2023-07-13T21:48:38.451 INFO:root:teuthology version: 0.0.1.dev143+g073e6eb 2023-07-13T21:48:38.452 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --name rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} --archive /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271 -- /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/orig.config.yaml 2023-07-13T21:48:38.502 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-13T21:48:38.602 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271 branch: wip-rishabh-2023Jul13 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} email: null first_in_suite: false job_id: '7336271' kernel: flavor: default kdb: true sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-rishabh-2023Jul13 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi060.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8nVQIrQ7b3w8ZPuA5eRJy7ykeR6z2+B/1lmJQu3oCFhge/KRmZhTWueiqzcfJEpS3NJkyU0jvPF/5kuU+lvcWyfsI1+Z53LxbyncMhk8AZiJXJ5RVsxpymZZAkn39Ab+fnO8vGyx50dH5XAGBR5OWqruHYPpcWw4bH3ddIvGOpXjxR7IKEKAULhkf9YZ/j11O4uaHiN9Xzv+K5N/nMdk4kxCPS21Gtst3U0KGWrR+wC2WNSJVB7I/LL48GAX4X3Nlh4gvr3jtEKEvpNCduUi8vnEBDc3xhb+jOkMZtdZB96z0ZxT/LmBZvyDQ3Zr9eigrYDz7EzbAgOMRaaHMonPY3xrJM5LuQUzjFbga2yPTZDVkBATKlxGmlAxDgtuuKA9fYpsaXvMn7vpjK/jlReKedb2lS7heNL4EY1ZebMxhAudvqljlLgQeQMBRhZ4W+lShI8nssgvHLsBKf5IVmlcmMbCTAFnNOcw4cVZNECQrwVW6TbZ6V0YJ6WN+9SLkpG0= smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLA2TVUdkdWf/GTZ+fWrjN14P+IxbNz6W3xXXjoshJQIYoB4jbdyPIdkjNaHeN8ANEoVUFDLZVy4k5cCqpwxTkEs2TZviwA0NyjGu+J660iXZM9f3izPw766fe8IE1duDu3xe0oyFZY30eJeh2CooQVeT11iLbr5ESZRGKlWRcnARsYAg52je2Rbcuus70/KQ5O07M+Y9lP4BlHkqg5/ycHSodgSe1TAr63oA2FZOsM5lYpsc+QdBIU4D4oOM48NBGuX0yzP7Fzga9QSKEYlzF1A5GWo4jmvj0EZygyx+hiRQEUjI7MlB+nTMM8qGVEB6ID30Xid9gCT8Mi7NhIgAvc5U6JH3Ey4wzGaV3x7vrz1bFvQhgK2xwR1ITNfQSafEOzr7fhI/mWcUzlUX/ouH8AbjkAtNYgO18ez7fzgIEtA+01SBhR6Q/rdDHSDA9Wb0dMzrnFEkLdcXjqF5fswqeTR7rA1JXG4jX9Cb2OXd6iSUqiNWtznHs4eI1bmBf2/E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 073e6eb1231451146b4a01d0b85b4f490f71d266 timestamp: 2023-07-13_21:35:13 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-13T21:48:38.602 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa; will attempt to use it 2023-07-13T21:48:38.603 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks 2023-07-13T21:48:38.603 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-13T21:48:38.605 INFO:teuthology.task.internal:Checking packages... 2023-07-13T21:48:38.632 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3fc0089cc1082313a98853d48b892acdc1385b8b' 2023-07-13T21:48:38.633 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-13T21:48:38.633 INFO:teuthology.packaging:ref: None 2023-07-13T21:48:38.633 INFO:teuthology.packaging:tag: None 2023-07-13T21:48:38.633 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13 2023-07-13T21:48:38.634 INFO:teuthology.packaging:sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T21:48:38.634 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13 2023-07-13T21:48:38.996 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4757.g3fc0089c 2023-07-13T21:48:38.996 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-13T21:48:39.006 INFO:teuthology.task.internal:no buildpackages task found 2023-07-13T21:48:39.006 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-13T21:48:39.014 INFO:teuthology.task.internal:Saving configuration 2023-07-13T21:48:39.032 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-13T21:48:39.038 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-13T21:48:39.067 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:15.314340', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAqpTsh4P1EAOijJ/Q7rQE57eaDXWIZoXdj9ozRDyHmXXAvGJTYe8DLi7da97jDit9Si26f+g6DIAVaHN1ecx1s='} 2023-07-13T21:48:39.096 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:15.316596', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLPUOKVcaypQf5MBTBKJFDKcuxZazB4Sad0RSAiuq7j3VJ3HcMixP3ZWQYYtTDJ4Gghv6inxEfrlQuqvh+YE4GY='} 2023-07-13T21:48:39.096 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-13T21:48:39.104 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-07-13T21:48:39.104 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-07-13T21:48:39.104 INFO:teuthology.run_tasks:Running task console_log... 2023-07-13T21:48:39.183 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2a63a6a430>, signals=[15]) 2023-07-13T21:48:39.184 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-13T21:48:39.191 INFO:teuthology.task.internal:Opening connections... 2023-07-13T21:48:39.192 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T21:48:39.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:48:39.266 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T21:48:39.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:48:39.336 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-13T21:48:39.349 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2023-07-13T21:48:39.373 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2023-07-13T21:48:39.374 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2023-07-13T21:48:39.437 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2023-07-13T21:48:39.438 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="8" 2023-07-13T21:48:39.438 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2023-07-13T21:48:39.438 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2023-07-13T21:48:39.438 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="8" 2023-07-13T21:48:39.438 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el8" 2023-07-13T21:48:39.438 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-13T21:48:39.438 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2023-07-13T21:48:39.439 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-13T21:48:39.439 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2023-07-13T21:48:39.439 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-13T21:48:39.439 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-13T21:48:39.439 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-13T21:48:39.440 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2023-07-13T21:48:39.460 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2023-07-13T21:48:39.482 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2023-07-13T21:48:39.482 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2023-07-13T21:48:39.543 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2023-07-13T21:48:39.544 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8" 2023-07-13T21:48:39.544 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2023-07-13T21:48:39.544 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2023-07-13T21:48:39.545 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8" 2023-07-13T21:48:39.545 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2023-07-13T21:48:39.545 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-13T21:48:39.546 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2023-07-13T21:48:39.546 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-13T21:48:39.546 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2023-07-13T21:48:39.546 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-13T21:48:39.546 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-13T21:48:39.547 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-13T21:48:39.547 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-07-13T21:48:39.568 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-13T21:48:39.578 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-13T21:48:39.585 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-13T21:48:39.585 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2023-07-13T21:48:39.588 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2023-07-13T21:48:39.608 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-13T21:48:39.616 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-13T21:48:39.616 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2023-07-13T21:48:39.651 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2023-07-13T21:48:39.679 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-13T21:48:39.748 INFO:teuthology.run_tasks:Running task kernel... 2023-07-13T21:48:39.768 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'} 2023-07-13T21:48:39.770 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}}, timeout 300 2023-07-13T21:48:39.770 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-13T21:48:39.771 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-13T21:48:39.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:48:39.793 DEBUG:teuthology.task.kernel:sha1 for mon.b is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:48:39.793 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-13T21:48:39.793 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2023-07-13T21:48:39.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:48:39.794 DEBUG:teuthology.task.kernel:sha1 for mon.a is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:48:39.795 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-13T21:48:39.795 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2023-07-13T21:48:39.854 INFO:teuthology.orchestra.run.smithi105.stdout:4.18.0-489.el8.x86_64 2023-07-13T21:48:39.855 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:48:39.856 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-13T21:48:39.856 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:48:39.860 INFO:teuthology.orchestra.run.smithi060.stdout:4.18.0-489.el8.x86_64 2023-07-13T21:48:39.860 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:48:39.860 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-13T21:48:39.860 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:48:40.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum upgrade -y linux-firmware 2023-07-13T21:48:40.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum upgrade -y linux-firmware 2023-07-13T21:48:54.310 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 8 - AppStream 2.7 MB/s | 31 MB 00:11 2023-07-13T21:49:02.643 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 8 - BaseOS 15 MB/s | 41 MB 00:02 2023-07-13T21:49:09.301 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 8 - Extras 25 kB/s | 18 kB 00:00 2023-07-13T21:49:10.077 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 8 - Extras common packages 11 kB/s | 6.6 kB 00:00 2023-07-13T21:49:12.221 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 8 - AppStream 1.0 MB/s | 31 MB 00:29 2023-07-13T21:49:12.995 INFO:teuthology.orchestra.run.smithi105.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-13T21:49:14.771 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 16 MB 00:01 2023-07-13T21:49:18.537 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 584 kB/s | 24 kB 00:00 2023-07-13T21:49:21.429 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 41 MB 00:03 2023-07-13T21:49:22.027 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-07-13T21:49:22.028 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-07-13T21:49:22.028 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repo Size 2023-07-13T21:49:22.028 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-07-13T21:49:22.029 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2023-07-13T21:49:22.029 INFO:teuthology.orchestra.run.smithi105.stdout: linux-firmware noarch 20230515-115.gitd1962891.el8 baseos 266 M 2023-07-13T21:49:22.029 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T21:49:22.029 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-07-13T21:49:22.029 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-07-13T21:49:22.029 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 1 Package 2023-07-13T21:49:22.029 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T21:49:22.030 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 266 M 2023-07-13T21:49:22.030 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-07-13T21:49:28.471 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 8 - Extras 26 kB/s | 18 kB 00:00 2023-07-13T21:49:29.155 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 8 - Extras common packages 13 kB/s | 6.6 kB 00:00 2023-07-13T21:49:32.148 INFO:teuthology.orchestra.run.smithi060.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-13T21:49:34.707 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 6.5 MB/s | 16 MB 00:02 2023-07-13T21:49:38.749 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 429 kB/s | 24 kB 00:00 2023-07-13T21:49:42.557 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-07-13T21:49:42.557 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-07-13T21:49:42.557 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2023-07-13T21:49:42.558 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-07-13T21:49:42.558 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2023-07-13T21:49:42.558 INFO:teuthology.orchestra.run.smithi060.stdout: linux-firmware noarch 20230515-115.gitd1962891.el8 baseos 266 M 2023-07-13T21:49:42.558 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T21:49:42.558 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-07-13T21:49:42.558 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-07-13T21:49:42.558 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 1 Package 2023-07-13T21:49:42.559 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T21:49:42.559 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 266 M 2023-07-13T21:49:42.559 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-07-13T21:49:46.362 INFO:teuthology.orchestra.run.smithi105.stdout:linux-firmware-20230515-115.gitd1962891.el8.noa 11 MB/s | 266 MB 00:24 2023-07-13T21:49:46.362 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-07-13T21:49:46.362 INFO:teuthology.orchestra.run.smithi105.stdout:Total 11 MB/s | 266 MB 00:24 2023-07-13T21:49:47.940 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-07-13T21:49:48.005 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-07-13T21:49:48.005 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-07-13T21:49:49.666 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-07-13T21:49:49.701 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-07-13T21:50:07.551 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-07-13T21:50:07.932 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-13T21:50:08.177 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-13T21:50:08.397 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-13T21:50:08.397 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-13T21:50:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-13T21:50:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T21:50:09.104 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2023-07-13T21:50:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: linux-firmware-20230515-115.gitd1962891.el8.noarch 2023-07-13T21:50:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T21:50:09.105 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-07-13T21:50:09.263 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.b... 2023-07-13T21:50:09.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:50:09.635 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-13T21:50:09.635 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-13T21:50:09.638 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi105.front.sepia.ceph.com to complete... 2023-07-13T21:50:17.514 INFO:teuthology.orchestra.run.smithi060.stdout:linux-firmware-20230515-115.gitd1962891.el8.noa 7.6 MB/s | 266 MB 00:34 2023-07-13T21:50:17.514 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-07-13T21:50:17.515 INFO:teuthology.orchestra.run.smithi060.stdout:Total 7.6 MB/s | 266 MB 00:34 2023-07-13T21:50:19.175 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-07-13T21:50:19.274 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-07-13T21:50:19.274 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-07-13T21:50:21.013 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-07-13T21:50:21.035 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-07-13T21:50:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-07-13T21:50:40.184 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-13T21:50:40.456 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-13T21:50:40.704 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-13T21:50:40.705 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-13T21:50:41.447 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-13T21:50:41.448 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T21:50:41.448 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2023-07-13T21:50:41.448 INFO:teuthology.orchestra.run.smithi060.stdout: linux-firmware-20230515-115.gitd1962891.el8.noarch 2023-07-13T21:50:41.448 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T21:50:41.448 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-07-13T21:50:41.637 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.a... 2023-07-13T21:50:41.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:50:42.008 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-13T21:50:42.009 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-13T21:50:42.011 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi060.front.sepia.ceph.com to complete... 2023-07-13T21:53:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13 21:53:18 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-13T21:53:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:FINISHED --2023-07-13 21:53:18-- 2023-07-13T21:53:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:Total wall clock time: 3m 9s 2023-07-13T21:53:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:Downloaded: 1 files, 363M in 3m 8s (1.93 MB/s) 2023-07-13T21:53:18.925 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.b... 2023-07-13T21:53:18.925 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-13T21:53:20.589 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying... ######################################## 2023-07-13T21:53:20.632 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing... ######################################## 2023-07-13T21:53:20.742 INFO:teuthology.orchestra.run.smithi105.stdout:Updating / installing... 2023-07-13T21:53:21.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13 21:53:21 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-13T21:53:21.413 INFO:teuthology.orchestra.run.smithi060.stderr:FINISHED --2023-07-13 21:53:21-- 2023-07-13T21:53:21.413 INFO:teuthology.orchestra.run.smithi060.stderr:Total wall clock time: 2m 39s 2023-07-13T21:53:21.413 INFO:teuthology.orchestra.run.smithi060.stderr:Downloaded: 1 files, 363M in 2m 39s (2.29 MB/s) 2023-07-13T21:53:21.414 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.a... 2023-07-13T21:53:21.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-13T21:53:23.175 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying... ######################################## 2023-07-13T21:53:23.222 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing... ######################################## 2023-07-13T21:53:23.338 INFO:teuthology.orchestra.run.smithi060.stdout:Updating / installing... 2023-07-13T21:53:58.627 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-13T21:54:04.231 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-13T21:54:24.950 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-13T21:54:24.951 DEBUG:teuthology.orchestra.run.smithi105:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-13T21:54:25.009 INFO:teuthology.orchestra.run.smithi105.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:25.009 INFO:teuthology.orchestra.run.smithi105.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:25.009 INFO:teuthology.orchestra.run.smithi105.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:25.009 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:25.009 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-13T21:54:25.010 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-13T21:54:25.010 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-13T21:54:25.010 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-13T21:54:25.010 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-13T21:54:25.010 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-13T21:54:25.010 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-13T21:54:25.011 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-13T21:54:25.011 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-13T21:54:25.011 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-13T21:54:25.011 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-13T21:54:25.011 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-13T21:54:25.011 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-13T21:54:25.011 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-13T21:54:25.012 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-13T21:54:25.012 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-13T21:54:25.012 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-13T21:54:25.012 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-13T21:54:25.012 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-13T21:54:25.012 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-13T21:54:25.013 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-13T21:54:25.013 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-13T21:54:25.013 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-13T21:54:25.013 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-13T21:54:25.013 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-13T21:54:25.013 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-13T21:54:25.013 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-13T21:54:25.014 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-13T21:54:25.014 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-13T21:54:25.014 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-13T21:54:25.014 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-13T21:54:25.014 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-13T21:54:25.014 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-13T21:54:25.014 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-13T21:54:25.015 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-13T21:54:25.015 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-13T21:54:25.015 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-13T21:54:25.015 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-13T21:54:25.015 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-13T21:54:25.015 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-13T21:54:25.015 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-13T21:54:25.016 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-13T21:54:25.016 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-13T21:54:25.016 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-13T21:54:25.016 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-13T21:54:25.016 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-13T21:54:25.016 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-13T21:54:25.016 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-13T21:54:25.017 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-13T21:54:25.017 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-13T21:54:25.017 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-13T21:54:25.017 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-13T21:54:25.017 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-13T21:54:25.017 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-13T21:54:25.017 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-13T21:54:25.018 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-13T21:54:25.018 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-13T21:54:25.018 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-13T21:54:25.018 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-13T21:54:25.018 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-13T21:54:25.018 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-13T21:54:25.019 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-13T21:54:25.019 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-13T21:54:25.019 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-13T21:54:25.019 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-13T21:54:25.019 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-13T21:54:25.019 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-13T21:54:25.019 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-13T21:54:25.019 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-13T21:54:25.020 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-13T21:54:25.020 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-13T21:54:25.020 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-13T21:54:25.020 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-13T21:54:25.020 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-13T21:54:25.020 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-13T21:54:25.021 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-13T21:54:25.021 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-13T21:54:25.021 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-13T21:54:25.021 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-13T21:54:25.021 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-13T21:54:25.021 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-13T21:54:25.021 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-13T21:54:25.022 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-13T21:54:25.022 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-13T21:54:25.022 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-13T21:54:25.022 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-13T21:54:25.022 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-13T21:54:25.022 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-13T21:54:25.022 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-13T21:54:25.023 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-13T21:54:25.023 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-13T21:54:25.023 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-13T21:54:25.023 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-13T21:54:25.023 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-13T21:54:25.023 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-13T21:54:25.023 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-13T21:54:25.024 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-13T21:54:25.024 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-13T21:54:25.024 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-13T21:54:25.024 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-13T21:54:25.024 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-13T21:54:25.024 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-13T21:54:25.024 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-13T21:54:25.025 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-13T21:54:25.025 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-13T21:54:25.025 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-13T21:54:25.025 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-13T21:54:25.025 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-13T21:54:25.025 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-13T21:54:25.025 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-13T21:54:25.026 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-13T21:54:25.026 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-13T21:54:25.026 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-13T21:54:25.026 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-13T21:54:25.026 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-13T21:54:25.026 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-13T21:54:25.026 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-13T21:54:25.027 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-13T21:54:25.027 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-13T21:54:25.027 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-13T21:54:25.027 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-13T21:54:25.027 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-13T21:54:25.027 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-13T21:54:25.028 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-13T21:54:25.028 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-13T21:54:25.028 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-13T21:54:25.028 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-13T21:54:25.028 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-13T21:54:25.028 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-13T21:54:25.028 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-13T21:54:25.029 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-13T21:54:25.029 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-13T21:54:25.029 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-13T21:54:25.029 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-13T21:54:25.029 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-13T21:54:25.029 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-13T21:54:25.029 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-13T21:54:25.030 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-13T21:54:25.030 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-13T21:54:25.030 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-13T21:54:25.030 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-13T21:54:25.030 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-13T21:54:25.030 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-13T21:54:25.030 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-13T21:54:25.031 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-13T21:54:25.031 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-13T21:54:25.031 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-13T21:54:25.031 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-13T21:54:25.031 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-13T21:54:25.031 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-13T21:54:25.031 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-13T21:54:25.032 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-13T21:54:25.032 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-13T21:54:25.032 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-13T21:54:25.032 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-13T21:54:25.032 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-13T21:54:25.032 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-13T21:54:25.032 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-13T21:54:25.033 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-13T21:54:25.033 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-13T21:54:25.033 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-13T21:54:25.033 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-13T21:54:25.033 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-13T21:54:25.033 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-13T21:54:25.033 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-13T21:54:25.034 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-13T21:54:25.034 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-13T21:54:25.034 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-13T21:54:25.034 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-13T21:54:25.034 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-13T21:54:25.034 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-13T21:54:25.034 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-13T21:54:25.035 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-13T21:54:25.035 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-13T21:54:25.035 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-13T21:54:25.035 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-13T21:54:25.035 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-13T21:54:25.035 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-13T21:54:25.035 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-13T21:54:25.036 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-13T21:54:25.036 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-13T21:54:25.036 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-13T21:54:25.036 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-13T21:54:25.036 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-13T21:54:25.036 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-13T21:54:25.036 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-13T21:54:25.037 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-13T21:54:25.037 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-13T21:54:25.037 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-13T21:54:25.037 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-13T21:54:25.037 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-13T21:54:25.037 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-13T21:54:25.037 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-13T21:54:25.038 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-13T21:54:25.038 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-13T21:54:25.038 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-13T21:54:25.038 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-13T21:54:25.038 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-13T21:54:25.038 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-13T21:54:25.038 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-13T21:54:25.039 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-13T21:54:25.039 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-13T21:54:25.039 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-13T21:54:25.039 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-13T21:54:25.039 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-13T21:54:25.039 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-13T21:54:25.039 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-13T21:54:25.040 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-13T21:54:25.040 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-13T21:54:25.040 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-13T21:54:25.040 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-13T21:54:25.040 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-13T21:54:25.040 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-13T21:54:25.041 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-13T21:54:25.041 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-13T21:54:25.041 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-13T21:54:25.041 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-13T21:54:25.041 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-13T21:54:25.041 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-13T21:54:25.041 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-13T21:54:25.042 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-13T21:54:25.042 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-13T21:54:25.042 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-13T21:54:25.042 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-13T21:54:25.042 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-13T21:54:25.042 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-13T21:54:25.042 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-13T21:54:25.043 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-13T21:54:25.043 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-13T21:54:25.043 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-13T21:54:25.043 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-13T21:54:25.043 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-13T21:54:25.043 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-13T21:54:25.043 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-13T21:54:25.044 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-13T21:54:25.044 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-13T21:54:25.044 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-13T21:54:25.045 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-13T21:54:25.045 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-13T21:54:25.045 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-13T21:54:25.045 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-13T21:54:25.045 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-13T21:54:25.045 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-13T21:54:25.046 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-13T21:54:25.046 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-13T21:54:25.046 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-13T21:54:25.046 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-13T21:54:25.046 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-13T21:54:25.046 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-13T21:54:25.046 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-13T21:54:25.047 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-13T21:54:25.047 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-13T21:54:25.047 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-13T21:54:25.047 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-13T21:54:25.047 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-13T21:54:25.047 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-13T21:54:25.047 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-13T21:54:25.048 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-13T21:54:25.048 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-13T21:54:25.048 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-13T21:54:25.048 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-13T21:54:25.048 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-13T21:54:25.048 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-13T21:54:25.048 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-13T21:54:25.049 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-13T21:54:25.049 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-13T21:54:25.049 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-13T21:54:25.049 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-13T21:54:25.049 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-13T21:54:25.049 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-13T21:54:25.049 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-13T21:54:25.050 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-13T21:54:25.050 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-13T21:54:25.050 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-13T21:54:25.050 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-13T21:54:25.050 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-13T21:54:25.050 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-13T21:54:25.050 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-13T21:54:25.051 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-13T21:54:25.051 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-13T21:54:25.051 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-13T21:54:25.051 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-13T21:54:25.051 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-13T21:54:25.051 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-13T21:54:25.051 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-13T21:54:25.052 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-13T21:54:25.052 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-13T21:54:25.052 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-13T21:54:25.052 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-13T21:54:25.052 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-13T21:54:25.052 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-13T21:54:25.052 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-13T21:54:25.053 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-13T21:54:25.053 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-13T21:54:25.053 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-13T21:54:25.053 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-13T21:54:25.053 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-13T21:54:25.053 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-13T21:54:25.053 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-13T21:54:25.054 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-13T21:54:25.054 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-13T21:54:25.054 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-13T21:54:25.054 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-13T21:54:25.054 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-13T21:54:25.054 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-13T21:54:25.054 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-13T21:54:25.055 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-13T21:54:25.055 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-13T21:54:25.055 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-13T21:54:25.055 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-13T21:54:25.055 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-13T21:54:25.055 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-13T21:54:25.055 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-13T21:54:25.056 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-13T21:54:25.056 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-13T21:54:25.056 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-13T21:54:25.056 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-13T21:54:25.056 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-13T21:54:25.056 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-13T21:54:25.056 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-13T21:54:25.057 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-13T21:54:25.057 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-13T21:54:25.057 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-13T21:54:25.057 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-13T21:54:25.057 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-13T21:54:25.057 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-13T21:54:25.057 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-13T21:54:25.058 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-13T21:54:25.058 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-13T21:54:25.058 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-13T21:54:25.058 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-13T21:54:25.058 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-13T21:54:25.058 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-13T21:54:25.058 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-13T21:54:25.059 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-13T21:54:25.059 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-13T21:54:25.059 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-13T21:54:25.059 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-13T21:54:25.059 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-13T21:54:25.059 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-13T21:54:25.059 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-13T21:54:25.060 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-13T21:54:25.060 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-13T21:54:25.060 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-13T21:54:25.060 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-13T21:54:25.060 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-13T21:54:25.060 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-13T21:54:25.060 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-13T21:54:25.061 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-13T21:54:25.061 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-13T21:54:25.061 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-13T21:54:25.061 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-13T21:54:25.061 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-13T21:54:25.061 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-13T21:54:25.061 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-13T21:54:25.062 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-13T21:54:25.062 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-13T21:54:25.062 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-13T21:54:25.062 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-13T21:54:25.062 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-13T21:54:25.062 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-13T21:54:25.062 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-13T21:54:25.063 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-13T21:54:25.063 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-13T21:54:25.063 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-13T21:54:25.063 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-13T21:54:25.063 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-13T21:54:25.063 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-13T21:54:25.063 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-13T21:54:25.064 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-13T21:54:25.064 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-13T21:54:25.064 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-13T21:54:25.064 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-13T21:54:25.064 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-13T21:54:25.064 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-13T21:54:25.064 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-13T21:54:25.065 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-13T21:54:25.065 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-13T21:54:25.065 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-13T21:54:25.065 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-13T21:54:25.065 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-13T21:54:25.065 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-13T21:54:25.065 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-13T21:54:25.066 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-13T21:54:25.066 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-13T21:54:25.066 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-13T21:54:25.066 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-13T21:54:25.066 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-13T21:54:25.066 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-13T21:54:25.066 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-13T21:54:25.067 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-13T21:54:25.067 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-13T21:54:25.067 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-13T21:54:25.067 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-13T21:54:25.067 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-13T21:54:25.067 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-13T21:54:25.067 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-13T21:54:25.068 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-13T21:54:25.068 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-13T21:54:25.068 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-13T21:54:25.068 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-13T21:54:25.068 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-13T21:54:25.068 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-13T21:54:25.069 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-13T21:54:25.069 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-13T21:54:25.069 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-13T21:54:25.069 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-13T21:54:25.069 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-13T21:54:25.069 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-13T21:54:25.069 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-13T21:54:25.070 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-13T21:54:25.070 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-13T21:54:25.070 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-13T21:54:25.070 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-13T21:54:25.070 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-13T21:54:25.070 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-13T21:54:25.070 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-13T21:54:25.071 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-13T21:54:25.071 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-13T21:54:25.071 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-13T21:54:25.071 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-13T21:54:25.071 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-13T21:54:25.071 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-13T21:54:25.071 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-13T21:54:25.072 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-13T21:54:25.072 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-13T21:54:25.072 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-13T21:54:25.072 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-13T21:54:25.072 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-13T21:54:25.072 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-13T21:54:25.072 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-13T21:54:25.073 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-13T21:54:25.073 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-13T21:54:25.073 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-13T21:54:25.073 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-13T21:54:25.073 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-13T21:54:25.073 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-13T21:54:25.073 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-13T21:54:25.074 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-13T21:54:25.074 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-13T21:54:25.074 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-13T21:54:25.074 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-13T21:54:25.074 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-13T21:54:25.074 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-13T21:54:25.074 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-13T21:54:25.075 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-13T21:54:25.075 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-13T21:54:25.075 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-13T21:54:25.075 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-13T21:54:25.075 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-13T21:54:25.075 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-13T21:54:25.075 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-13T21:54:25.076 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-13T21:54:25.076 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-13T21:54:25.076 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-13T21:54:25.076 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-13T21:54:25.076 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-13T21:54:25.076 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-13T21:54:25.076 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-13T21:54:25.076 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-13T21:54:25.077 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-13T21:54:25.077 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-13T21:54:25.077 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-13T21:54:25.077 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-13T21:54:25.077 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-13T21:54:25.077 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-13T21:54:25.077 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-13T21:54:25.078 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-13T21:54:25.078 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-13T21:54:25.078 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-13T21:54:25.078 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-13T21:54:25.078 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-13T21:54:25.078 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-13T21:54:25.079 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-13T21:54:25.079 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-13T21:54:25.079 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-13T21:54:25.079 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-13T21:54:25.079 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-13T21:54:25.079 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-13T21:54:25.079 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-13T21:54:25.080 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-13T21:54:25.080 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-13T21:54:25.080 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-13T21:54:25.080 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-13T21:54:25.080 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-13T21:54:25.080 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-13T21:54:25.080 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-13T21:54:25.081 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-13T21:54:25.081 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-13T21:54:25.081 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-13T21:54:25.081 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-13T21:54:25.081 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-13T21:54:25.081 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-13T21:54:25.081 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-13T21:54:25.082 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-13T21:54:25.082 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-13T21:54:25.082 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-13T21:54:25.082 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-13T21:54:25.082 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-13T21:54:25.082 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-13T21:54:25.082 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-13T21:54:25.083 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-13T21:54:25.083 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-13T21:54:25.083 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-13T21:54:25.083 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-13T21:54:25.083 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-13T21:54:25.083 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-13T21:54:25.083 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-13T21:54:25.084 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-13T21:54:25.084 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-13T21:54:25.084 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-13T21:54:25.084 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-13T21:54:25.084 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-13T21:54:25.084 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-13T21:54:25.084 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-13T21:54:25.085 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-13T21:54:25.085 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-13T21:54:25.085 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-13T21:54:25.085 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-13T21:54:25.085 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-13T21:54:25.085 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-13T21:54:25.085 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-13T21:54:25.086 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-13T21:54:25.086 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-13T21:54:25.086 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-13T21:54:25.086 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-13T21:54:25.086 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-13T21:54:25.086 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-13T21:54:25.086 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-13T21:54:25.087 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-13T21:54:25.087 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-13T21:54:25.087 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-13T21:54:25.087 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-13T21:54:25.087 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-13T21:54:25.087 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-13T21:54:25.087 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-13T21:54:25.088 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-13T21:54:25.088 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-13T21:54:25.088 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-13T21:54:25.088 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-13T21:54:25.088 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-13T21:54:25.088 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-13T21:54:25.088 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-13T21:54:25.089 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-13T21:54:25.089 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-13T21:54:25.089 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-13T21:54:25.089 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-13T21:54:25.089 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-13T21:54:25.089 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-13T21:54:25.089 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-13T21:54:25.090 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-13T21:54:25.090 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-13T21:54:25.090 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-13T21:54:25.090 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-13T21:54:25.090 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-13T21:54:25.090 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-13T21:54:25.090 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-13T21:54:25.091 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-13T21:54:25.091 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-13T21:54:25.091 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-13T21:54:25.091 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-13T21:54:25.091 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-13T21:54:25.091 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-13T21:54:25.091 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-13T21:54:25.092 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-13T21:54:25.092 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-13T21:54:25.092 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-13T21:54:25.092 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-13T21:54:25.092 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-13T21:54:25.092 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-13T21:54:25.092 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-13T21:54:25.093 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-13T21:54:25.093 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-13T21:54:25.093 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-13T21:54:25.093 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-13T21:54:25.093 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-13T21:54:25.093 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-13T21:54:25.094 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-13T21:54:25.094 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-13T21:54:25.094 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-13T21:54:25.094 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-13T21:54:25.094 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-13T21:54:25.094 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-13T21:54:25.094 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-13T21:54:25.094 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-13T21:54:25.095 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-13T21:54:25.095 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-13T21:54:25.095 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-13T21:54:25.095 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-13T21:54:25.095 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-13T21:54:25.095 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-13T21:54:25.096 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-13T21:54:25.096 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-13T21:54:25.096 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-13T21:54:25.096 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-13T21:54:25.096 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-13T21:54:25.096 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-13T21:54:25.096 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-13T21:54:25.097 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-13T21:54:25.097 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-13T21:54:25.097 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-13T21:54:25.097 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-13T21:54:25.097 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-13T21:54:25.097 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-13T21:54:25.097 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-13T21:54:25.097 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-13T21:54:25.098 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-13T21:54:25.098 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-13T21:54:25.098 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-13T21:54:25.098 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-13T21:54:25.098 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-13T21:54:25.098 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-13T21:54:25.098 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-13T21:54:25.099 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-13T21:54:25.099 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-13T21:54:25.099 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-13T21:54:25.099 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-13T21:54:25.099 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-13T21:54:25.099 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-13T21:54:25.099 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-13T21:54:25.100 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-13T21:54:25.100 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-13T21:54:25.100 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-13T21:54:25.100 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-13T21:54:25.100 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-13T21:54:25.100 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-13T21:54:25.100 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-13T21:54:25.101 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-13T21:54:25.101 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-13T21:54:25.101 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-13T21:54:25.101 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-13T21:54:25.101 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-13T21:54:25.101 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-13T21:54:25.101 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-13T21:54:25.102 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-13T21:54:25.102 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-13T21:54:25.102 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-13T21:54:25.102 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-13T21:54:25.102 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-13T21:54:25.102 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-13T21:54:25.102 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-13T21:54:25.103 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-13T21:54:25.103 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-13T21:54:25.103 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-13T21:54:25.103 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-13T21:54:25.103 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-13T21:54:25.103 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-13T21:54:25.103 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-13T21:54:25.104 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-13T21:54:25.104 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-13T21:54:25.104 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-13T21:54:25.104 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-13T21:54:25.104 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-13T21:54:25.104 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-13T21:54:25.104 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-13T21:54:25.105 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-13T21:54:25.105 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-13T21:54:25.105 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-13T21:54:25.105 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-13T21:54:25.105 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-13T21:54:25.105 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-13T21:54:25.105 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-13T21:54:25.106 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-13T21:54:25.106 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-13T21:54:25.106 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-13T21:54:25.106 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-13T21:54:25.106 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-13T21:54:25.106 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-13T21:54:25.106 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-13T21:54:25.106 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-13T21:54:25.107 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-13T21:54:25.107 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-13T21:54:25.107 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-13T21:54:25.107 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-13T21:54:25.107 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-13T21:54:25.107 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-13T21:54:25.107 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-13T21:54:25.108 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-13T21:54:25.108 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-13T21:54:25.108 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-13T21:54:25.108 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-13T21:54:25.108 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-13T21:54:25.108 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-13T21:54:25.108 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-13T21:54:25.109 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-13T21:54:25.109 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-13T21:54:25.109 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-13T21:54:25.109 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-13T21:54:25.109 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-13T21:54:25.109 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-13T21:54:25.109 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-13T21:54:25.110 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-13T21:54:25.110 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-13T21:54:25.110 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-13T21:54:25.110 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-13T21:54:25.110 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-13T21:54:25.110 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-13T21:54:25.110 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-13T21:54:25.111 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-13T21:54:25.111 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-13T21:54:25.111 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-13T21:54:25.111 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-13T21:54:25.111 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-13T21:54:25.111 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-13T21:54:25.111 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-13T21:54:25.112 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-13T21:54:25.112 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-13T21:54:25.112 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-13T21:54:25.112 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-13T21:54:25.112 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-13T21:54:25.112 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-13T21:54:25.112 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-13T21:54:25.113 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-13T21:54:25.113 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-13T21:54:25.113 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-13T21:54:25.113 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-13T21:54:25.113 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-13T21:54:25.113 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-13T21:54:25.113 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-13T21:54:25.114 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-13T21:54:25.114 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-13T21:54:25.114 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-13T21:54:25.114 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-13T21:54:25.114 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-13T21:54:25.114 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-13T21:54:25.114 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-13T21:54:25.115 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-13T21:54:25.115 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-13T21:54:25.115 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-13T21:54:25.115 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-13T21:54:25.115 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-13T21:54:25.115 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-13T21:54:25.115 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-13T21:54:25.116 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-13T21:54:25.116 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-13T21:54:25.116 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-13T21:54:25.116 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-13T21:54:25.116 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-13T21:54:25.116 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-13T21:54:25.116 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-13T21:54:25.117 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-13T21:54:25.117 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-13T21:54:25.117 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-13T21:54:25.117 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-13T21:54:25.117 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-13T21:54:25.117 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-13T21:54:25.117 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-13T21:54:25.118 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-13T21:54:25.118 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-13T21:54:25.118 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-13T21:54:25.118 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-13T21:54:25.118 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-13T21:54:25.118 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-13T21:54:25.118 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-13T21:54:25.119 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-13T21:54:25.119 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-13T21:54:25.119 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-13T21:54:25.119 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-13T21:54:25.119 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-13T21:54:25.119 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-13T21:54:25.119 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-13T21:54:25.120 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-13T21:54:25.120 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-13T21:54:25.120 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-13T21:54:25.120 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-13T21:54:25.120 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-13T21:54:25.120 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-13T21:54:25.120 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-13T21:54:25.121 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-13T21:54:25.121 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-13T21:54:25.121 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-13T21:54:25.121 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-13T21:54:25.121 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-13T21:54:25.121 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-13T21:54:25.121 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-13T21:54:25.122 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-13T21:54:25.122 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-13T21:54:25.122 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-13T21:54:25.122 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-13T21:54:25.122 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-13T21:54:25.122 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-13T21:54:25.122 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-13T21:54:25.123 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-13T21:54:25.123 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-13T21:54:25.123 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-13T21:54:25.123 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-13T21:54:25.123 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-13T21:54:25.123 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-13T21:54:25.123 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-13T21:54:25.123 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-13T21:54:25.124 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-13T21:54:25.124 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-13T21:54:25.124 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-13T21:54:25.124 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-13T21:54:25.124 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-13T21:54:25.124 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-13T21:54:25.125 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-13T21:54:25.125 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-13T21:54:25.125 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-13T21:54:25.125 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-13T21:54:25.125 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-13T21:54:25.125 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-13T21:54:25.125 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-13T21:54:25.125 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-13T21:54:25.126 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-13T21:54:25.126 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-13T21:54:25.126 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-13T21:54:25.126 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-13T21:54:25.126 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-13T21:54:25.126 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-13T21:54:25.126 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-13T21:54:25.127 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-13T21:54:25.127 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-13T21:54:25.127 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-13T21:54:25.127 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-13T21:54:25.127 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-13T21:54:25.127 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-13T21:54:25.127 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-13T21:54:25.128 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-13T21:54:25.128 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-13T21:54:25.128 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-13T21:54:25.128 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-13T21:54:25.128 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-13T21:54:25.128 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-13T21:54:25.128 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-13T21:54:25.129 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-13T21:54:25.129 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-13T21:54:25.129 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-13T21:54:25.129 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-13T21:54:25.129 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-13T21:54:25.129 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-13T21:54:25.129 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-13T21:54:25.130 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-13T21:54:25.130 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-13T21:54:25.130 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-13T21:54:25.130 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-13T21:54:25.130 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-13T21:54:25.130 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-13T21:54:25.130 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-13T21:54:25.131 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-13T21:54:25.131 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-13T21:54:25.131 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-13T21:54:25.131 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-13T21:54:25.131 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-13T21:54:25.131 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-13T21:54:25.131 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-13T21:54:25.132 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-13T21:54:25.132 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-13T21:54:25.132 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-13T21:54:25.132 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-13T21:54:25.132 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-13T21:54:25.132 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-13T21:54:25.132 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-13T21:54:25.133 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-13T21:54:25.133 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-13T21:54:25.133 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-13T21:54:25.133 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-13T21:54:25.133 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-13T21:54:25.133 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-13T21:54:25.133 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-13T21:54:25.134 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-13T21:54:25.134 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-13T21:54:25.134 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-13T21:54:25.134 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-13T21:54:25.134 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-13T21:54:25.134 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-13T21:54:25.134 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-13T21:54:25.135 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-13T21:54:25.135 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-13T21:54:25.135 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-13T21:54:25.135 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-13T21:54:25.135 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-13T21:54:25.135 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-13T21:54:25.135 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-13T21:54:25.135 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-13T21:54:25.136 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-13T21:54:25.136 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-13T21:54:25.136 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-13T21:54:25.136 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-13T21:54:25.136 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-13T21:54:25.136 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-13T21:54:25.136 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-13T21:54:25.137 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-13T21:54:25.137 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-13T21:54:25.137 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-13T21:54:25.137 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-13T21:54:25.137 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-13T21:54:25.137 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-13T21:54:25.137 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-13T21:54:25.138 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-13T21:54:25.138 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-13T21:54:25.138 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-13T21:54:25.138 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-13T21:54:25.138 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-13T21:54:25.138 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-13T21:54:25.138 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-13T21:54:25.139 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-13T21:54:25.139 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-13T21:54:25.139 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-13T21:54:25.139 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-13T21:54:25.139 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-13T21:54:25.139 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-13T21:54:25.139 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-13T21:54:25.140 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-13T21:54:25.140 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-13T21:54:25.140 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-13T21:54:25.140 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-13T21:54:25.140 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-13T21:54:25.140 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-13T21:54:25.140 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-13T21:54:25.141 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-13T21:54:25.141 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-13T21:54:25.141 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-13T21:54:25.141 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-13T21:54:25.141 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-13T21:54:25.141 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-13T21:54:25.141 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-13T21:54:25.142 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-13T21:54:25.142 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-13T21:54:25.142 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-13T21:54:25.142 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-13T21:54:25.142 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-13T21:54:25.142 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-13T21:54:25.142 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-13T21:54:25.143 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-13T21:54:25.143 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-13T21:54:25.143 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-13T21:54:25.143 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-13T21:54:25.143 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-13T21:54:25.143 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-13T21:54:25.143 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-13T21:54:25.143 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-13T21:54:25.144 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-13T21:54:25.144 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-13T21:54:25.144 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-13T21:54:25.144 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-13T21:54:25.144 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-13T21:54:25.144 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-13T21:54:25.145 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-13T21:54:25.145 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-13T21:54:25.145 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-13T21:54:25.145 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-13T21:54:25.145 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-13T21:54:25.145 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-13T21:54:25.145 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-13T21:54:25.146 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-13T21:54:25.146 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-13T21:54:25.146 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-13T21:54:25.146 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-13T21:54:25.146 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-13T21:54:25.146 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-13T21:54:25.146 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-13T21:54:25.146 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-13T21:54:25.147 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-13T21:54:25.147 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-13T21:54:25.147 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-13T21:54:25.147 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-13T21:54:25.147 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-13T21:54:25.147 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-13T21:54:25.147 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-13T21:54:25.148 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-13T21:54:25.148 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-13T21:54:25.148 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-13T21:54:25.148 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-13T21:54:25.148 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-13T21:54:25.148 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-13T21:54:25.148 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-13T21:54:25.149 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-13T21:54:25.149 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-13T21:54:25.149 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-13T21:54:25.149 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-13T21:54:25.149 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-13T21:54:25.149 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-13T21:54:25.149 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-13T21:54:25.150 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-13T21:54:25.150 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-13T21:54:25.150 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-13T21:54:25.150 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-13T21:54:25.150 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-13T21:54:25.150 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-13T21:54:25.150 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-13T21:54:25.151 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-13T21:54:25.151 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-13T21:54:25.151 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-13T21:54:25.151 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-13T21:54:25.151 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-13T21:54:25.151 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-13T21:54:25.151 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-13T21:54:25.152 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-13T21:54:25.152 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-13T21:54:25.152 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-13T21:54:25.152 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-13T21:54:25.152 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-13T21:54:25.152 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-13T21:54:25.152 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-13T21:54:25.152 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-13T21:54:25.153 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-13T21:54:25.153 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-13T21:54:25.153 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-13T21:54:25.153 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-13T21:54:25.153 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-13T21:54:25.153 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-13T21:54:25.153 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-13T21:54:25.154 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-13T21:54:25.154 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-13T21:54:25.154 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-13T21:54:25.154 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-13T21:54:25.154 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-13T21:54:25.154 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-13T21:54:25.154 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-13T21:54:25.155 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-13T21:54:25.155 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-13T21:54:25.155 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-13T21:54:25.155 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-13T21:54:25.155 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-13T21:54:25.155 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-13T21:54:25.155 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-13T21:54:25.155 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-13T21:54:25.156 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-13T21:54:25.156 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-13T21:54:25.156 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-13T21:54:25.156 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-13T21:54:25.156 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-13T21:54:25.156 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-13T21:54:25.157 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-13T21:54:25.157 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-13T21:54:25.157 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-13T21:54:25.157 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-13T21:54:25.157 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-13T21:54:25.157 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-13T21:54:25.157 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-13T21:54:25.158 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-13T21:54:25.158 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-13T21:54:25.158 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-13T21:54:25.158 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-13T21:54:25.158 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-13T21:54:25.158 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-13T21:54:25.158 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-13T21:54:25.158 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-13T21:54:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-13T21:54:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-13T21:54:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-13T21:54:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-13T21:54:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-13T21:54:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-13T21:54:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-13T21:54:25.160 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-13T21:54:25.160 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-13T21:54:25.160 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-13T21:54:25.160 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-13T21:54:25.160 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-13T21:54:25.160 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-13T21:54:25.160 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-13T21:54:25.161 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-13T21:54:25.161 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-13T21:54:25.161 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-13T21:54:25.161 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-13T21:54:25.161 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-13T21:54:25.161 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-13T21:54:25.161 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-13T21:54:25.162 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-13T21:54:25.162 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-13T21:54:25.162 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-13T21:54:25.162 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-13T21:54:25.162 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-13T21:54:25.162 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-13T21:54:25.162 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-13T21:54:25.163 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-13T21:54:25.163 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-13T21:54:25.163 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-13T21:54:25.163 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-13T21:54:25.163 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-13T21:54:25.163 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-13T21:54:25.163 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-13T21:54:25.163 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-13T21:54:25.164 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-13T21:54:25.164 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-13T21:54:25.164 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-13T21:54:25.164 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-13T21:54:25.164 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-13T21:54:25.164 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-13T21:54:25.164 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-13T21:54:25.165 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-13T21:54:25.165 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-13T21:54:25.165 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-13T21:54:25.165 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-13T21:54:25.165 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-13T21:54:25.165 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-13T21:54:25.165 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-13T21:54:25.166 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-13T21:54:25.166 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-13T21:54:25.166 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-13T21:54:25.166 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-13T21:54:25.166 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-13T21:54:25.166 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-13T21:54:25.166 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-13T21:54:25.167 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-13T21:54:25.167 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-13T21:54:25.167 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-13T21:54:25.167 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-13T21:54:25.167 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-13T21:54:25.167 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-13T21:54:25.167 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-13T21:54:25.168 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-13T21:54:25.168 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-13T21:54:25.168 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-13T21:54:25.168 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-13T21:54:25.168 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-13T21:54:25.168 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-13T21:54:25.168 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-13T21:54:25.169 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-13T21:54:25.174 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-13T21:54:25.174 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-13T21:54:25.174 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-13T21:54:25.174 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-13T21:54:25.175 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-13T21:54:25.175 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-13T21:54:25.175 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-13T21:54:25.175 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-13T21:54:25.175 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-13T21:54:25.175 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-13T21:54:25.175 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-13T21:54:25.176 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-13T21:54:25.176 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-13T21:54:25.176 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-13T21:54:25.176 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-13T21:54:25.176 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-13T21:54:25.176 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-13T21:54:25.176 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-13T21:54:25.177 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-13T21:54:25.177 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-13T21:54:25.177 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-13T21:54:25.177 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-13T21:54:25.177 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-13T21:54:25.177 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-13T21:54:25.177 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-13T21:54:25.178 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-13T21:54:25.178 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-13T21:54:25.178 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-13T21:54:25.178 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-13T21:54:25.178 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-13T21:54:25.178 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-13T21:54:25.178 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-13T21:54:25.179 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-13T21:54:25.179 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-13T21:54:25.179 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-13T21:54:25.179 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-13T21:54:25.179 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-13T21:54:25.179 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-13T21:54:25.179 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-13T21:54:25.179 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-13T21:54:25.180 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-13T21:54:25.180 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-13T21:54:25.180 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-13T21:54:25.180 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-13T21:54:25.180 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-13T21:54:25.180 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-13T21:54:25.180 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-13T21:54:25.181 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-13T21:54:25.181 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-13T21:54:25.181 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-13T21:54:25.181 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-13T21:54:25.181 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-13T21:54:25.181 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-13T21:54:25.181 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-13T21:54:25.182 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-13T21:54:25.182 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-13T21:54:25.182 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-13T21:54:25.182 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-13T21:54:25.182 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-13T21:54:25.182 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-13T21:54:25.182 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-13T21:54:25.183 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-13T21:54:25.183 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-13T21:54:25.183 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-13T21:54:25.183 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-13T21:54:25.183 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-13T21:54:25.183 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-13T21:54:25.183 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-13T21:54:25.184 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-13T21:54:25.184 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-13T21:54:25.184 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-13T21:54:25.184 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-13T21:54:25.184 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-13T21:54:25.184 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-13T21:54:25.184 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-13T21:54:25.185 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-13T21:54:25.185 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-13T21:54:25.185 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-13T21:54:25.185 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-13T21:54:25.185 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-13T21:54:25.185 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-13T21:54:25.185 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-13T21:54:25.186 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-13T21:54:25.186 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-13T21:54:25.186 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-13T21:54:25.186 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-13T21:54:25.186 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-13T21:54:25.186 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-13T21:54:25.186 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-13T21:54:25.187 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-13T21:54:25.187 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-13T21:54:25.187 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-13T21:54:25.187 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-13T21:54:25.187 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-13T21:54:25.187 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-13T21:54:25.187 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-13T21:54:25.187 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-13T21:54:25.188 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-13T21:54:25.188 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-13T21:54:25.188 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-13T21:54:25.188 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-13T21:54:25.188 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-13T21:54:25.188 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-13T21:54:25.188 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-13T21:54:25.189 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-13T21:54:25.189 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-13T21:54:25.189 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-13T21:54:25.189 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-13T21:54:25.189 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-13T21:54:25.189 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-13T21:54:25.189 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-13T21:54:25.190 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-13T21:54:25.190 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-13T21:54:25.190 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-13T21:54:25.190 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-13T21:54:25.190 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-13T21:54:25.190 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-13T21:54:25.190 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-13T21:54:25.191 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-13T21:54:25.191 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-13T21:54:25.191 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-13T21:54:25.191 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-13T21:54:25.191 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-13T21:54:25.191 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-13T21:54:25.191 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-13T21:54:25.192 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-13T21:54:25.192 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-13T21:54:25.192 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-13T21:54:25.192 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-13T21:54:25.192 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-13T21:54:25.192 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-13T21:54:25.192 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-13T21:54:25.193 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-13T21:54:25.193 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-13T21:54:25.193 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-13T21:54:25.193 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-13T21:54:25.193 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-13T21:54:25.193 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-13T21:54:25.193 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-13T21:54:25.194 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-13T21:54:25.194 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-13T21:54:25.194 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-13T21:54:25.194 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-13T21:54:25.194 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-13T21:54:25.194 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-13T21:54:25.194 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-13T21:54:25.195 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-13T21:54:25.195 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-13T21:54:25.195 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-13T21:54:25.195 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-13T21:54:25.195 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-13T21:54:25.195 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-13T21:54:25.195 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-13T21:54:25.195 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-13T21:54:25.196 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-13T21:54:25.196 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-13T21:54:25.196 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-13T21:54:25.196 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-13T21:54:25.196 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-13T21:54:25.196 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-13T21:54:25.196 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-13T21:54:25.197 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-13T21:54:25.197 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-13T21:54:25.197 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-13T21:54:25.197 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-13T21:54:25.197 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-13T21:54:25.197 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-13T21:54:25.197 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-13T21:54:25.198 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-13T21:54:25.198 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-13T21:54:25.198 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-13T21:54:25.198 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-13T21:54:25.198 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-13T21:54:25.198 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-13T21:54:25.198 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-13T21:54:25.199 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-13T21:54:25.199 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-13T21:54:25.199 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-13T21:54:25.199 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-13T21:54:25.199 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-13T21:54:25.199 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-13T21:54:25.199 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-13T21:54:25.200 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-13T21:54:25.200 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-13T21:54:25.200 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-13T21:54:25.200 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-13T21:54:25.200 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-13T21:54:25.200 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-13T21:54:25.200 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-13T21:54:25.200 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-13T21:54:25.201 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-13T21:54:25.201 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-13T21:54:25.201 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-13T21:54:25.201 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-13T21:54:25.201 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-13T21:54:25.201 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-13T21:54:25.201 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-13T21:54:25.202 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-13T21:54:25.202 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-13T21:54:25.202 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-13T21:54:25.202 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-13T21:54:25.202 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-13T21:54:25.202 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-13T21:54:25.202 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-13T21:54:25.203 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-13T21:54:25.203 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-13T21:54:25.203 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-13T21:54:25.203 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-13T21:54:25.203 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-13T21:54:25.203 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-13T21:54:25.203 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-13T21:54:25.204 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-13T21:54:25.204 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-13T21:54:25.204 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-13T21:54:25.204 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-13T21:54:25.204 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-13T21:54:25.204 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-13T21:54:25.204 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-13T21:54:25.205 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-13T21:54:25.205 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-13T21:54:25.205 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-13T21:54:25.205 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-13T21:54:25.205 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-13T21:54:25.205 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-13T21:54:25.205 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-13T21:54:25.205 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-13T21:54:25.206 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-13T21:54:25.206 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-13T21:54:25.206 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-13T21:54:25.206 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-13T21:54:25.206 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-13T21:54:25.206 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-13T21:54:25.206 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-13T21:54:25.207 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-13T21:54:25.207 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-13T21:54:25.207 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-13T21:54:25.207 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-13T21:54:25.207 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-13T21:54:25.207 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-13T21:54:25.207 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-13T21:54:25.208 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-13T21:54:25.208 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-13T21:54:25.208 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-13T21:54:25.208 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-13T21:54:25.208 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-13T21:54:25.208 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-13T21:54:25.208 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-13T21:54:25.209 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-13T21:54:25.209 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-13T21:54:25.209 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-13T21:54:25.209 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-13T21:54:25.209 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-13T21:54:25.209 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-13T21:54:25.209 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-13T21:54:25.210 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-13T21:54:25.210 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-13T21:54:25.210 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-13T21:54:25.210 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-13T21:54:25.210 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-13T21:54:25.210 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-13T21:54:25.210 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-13T21:54:25.211 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-13T21:54:25.211 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-13T21:54:25.211 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-13T21:54:25.211 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-13T21:54:25.211 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-13T21:54:25.211 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-13T21:54:25.211 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-13T21:54:25.211 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-13T21:54:25.212 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-13T21:54:25.212 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-13T21:54:25.212 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-13T21:54:25.212 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-13T21:54:25.212 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-13T21:54:25.212 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-13T21:54:25.212 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-13T21:54:25.213 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-13T21:54:25.213 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-13T21:54:25.213 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-13T21:54:25.213 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-13T21:54:25.213 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-13T21:54:25.213 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-13T21:54:25.213 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-13T21:54:25.214 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-13T21:54:25.214 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-13T21:54:25.214 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-13T21:54:25.214 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-13T21:54:25.214 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-13T21:54:25.214 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-13T21:54:25.214 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-13T21:54:25.215 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-13T21:54:25.215 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-13T21:54:25.215 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-13T21:54:25.215 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-13T21:54:25.215 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-13T21:54:25.215 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-13T21:54:25.215 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-13T21:54:25.215 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-13T21:54:25.216 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-13T21:54:25.216 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-13T21:54:25.216 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-13T21:54:25.216 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-13T21:54:25.216 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-13T21:54:25.216 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-13T21:54:25.216 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-13T21:54:25.217 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-13T21:54:25.217 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-13T21:54:25.217 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-13T21:54:25.217 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-13T21:54:25.217 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-13T21:54:25.217 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-13T21:54:25.217 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-13T21:54:25.218 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-13T21:54:25.218 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-13T21:54:25.218 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-13T21:54:25.218 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-13T21:54:25.218 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-13T21:54:25.218 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-13T21:54:25.218 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-13T21:54:25.219 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-13T21:54:25.219 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-13T21:54:25.219 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-13T21:54:25.219 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-13T21:54:25.219 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-13T21:54:25.219 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-13T21:54:25.219 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-13T21:54:25.219 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-13T21:54:25.220 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-13T21:54:25.220 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-13T21:54:25.220 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-13T21:54:25.220 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-13T21:54:25.220 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-13T21:54:25.220 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-13T21:54:25.220 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-13T21:54:25.221 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-13T21:54:25.221 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-13T21:54:25.221 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-13T21:54:25.221 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-13T21:54:25.221 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-13T21:54:25.221 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-13T21:54:25.221 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-13T21:54:25.222 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-13T21:54:25.222 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-13T21:54:25.222 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-13T21:54:25.222 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-13T21:54:25.222 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-13T21:54:25.222 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-13T21:54:25.222 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-13T21:54:25.223 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-13T21:54:25.223 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-13T21:54:25.223 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-13T21:54:25.223 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-13T21:54:25.223 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-13T21:54:25.223 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-13T21:54:25.223 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-13T21:54:25.224 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-13T21:54:25.224 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-13T21:54:25.224 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-13T21:54:25.224 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-13T21:54:25.224 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-13T21:54:25.224 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-13T21:54:25.224 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-13T21:54:25.225 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-13T21:54:25.225 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-13T21:54:25.225 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-13T21:54:25.225 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-13T21:54:25.225 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-13T21:54:25.225 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-13T21:54:25.225 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-13T21:54:25.225 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-13T21:54:25.226 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-13T21:54:25.226 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-13T21:54:25.226 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-13T21:54:25.226 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-13T21:54:25.226 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-13T21:54:25.226 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-13T21:54:25.226 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-13T21:54:25.227 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-13T21:54:25.227 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-13T21:54:25.227 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-13T21:54:25.227 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-13T21:54:25.227 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-13T21:54:25.227 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-13T21:54:25.227 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-13T21:54:25.228 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-13T21:54:25.228 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-13T21:54:25.228 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-13T21:54:25.228 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-13T21:54:25.228 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-13T21:54:25.228 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-13T21:54:25.228 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-13T21:54:25.229 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-13T21:54:25.229 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-13T21:54:25.229 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-13T21:54:25.229 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-13T21:54:25.229 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-13T21:54:25.229 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-13T21:54:25.229 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-13T21:54:25.230 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-13T21:54:25.230 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-13T21:54:25.230 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-13T21:54:25.230 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-13T21:54:25.230 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-13T21:54:25.230 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-13T21:54:25.230 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-13T21:54:25.230 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-13T21:54:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-13T21:54:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-13T21:54:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-13T21:54:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-13T21:54:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-13T21:54:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-13T21:54:25.231 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-13T21:54:25.232 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-13T21:54:25.232 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-13T21:54:25.232 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-13T21:54:25.232 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-13T21:54:25.232 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-13T21:54:25.232 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-13T21:54:25.232 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-13T21:54:25.233 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-13T21:54:25.233 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-13T21:54:25.233 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-13T21:54:25.233 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-13T21:54:25.233 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-13T21:54:25.233 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-13T21:54:25.233 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-13T21:54:25.234 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-13T21:54:25.234 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-13T21:54:25.234 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-13T21:54:25.234 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-13T21:54:25.234 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-13T21:54:25.234 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-13T21:54:25.234 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-13T21:54:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-13T21:54:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-13T21:54:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-13T21:54:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-13T21:54:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-13T21:54:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-13T21:54:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-13T21:54:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-13T21:54:25.236 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-13T21:54:25.236 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-13T21:54:25.236 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-13T21:54:25.236 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-13T21:54:25.236 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-13T21:54:25.236 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-13T21:54:25.236 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-13T21:54:25.237 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-13T21:54:25.237 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-13T21:54:25.237 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-13T21:54:25.237 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-13T21:54:25.237 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-13T21:54:25.237 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-13T21:54:25.237 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-13T21:54:25.238 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-13T21:54:25.238 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-13T21:54:25.238 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-13T21:54:25.238 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-13T21:54:25.238 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-13T21:54:25.238 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-13T21:54:25.238 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-13T21:54:25.239 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-13T21:54:25.239 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-13T21:54:25.239 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-13T21:54:25.239 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-13T21:54:25.239 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-13T21:54:25.239 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-13T21:54:25.239 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-13T21:54:25.240 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-13T21:54:25.240 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-13T21:54:25.240 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-13T21:54:25.240 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-13T21:54:25.240 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-13T21:54:25.240 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-13T21:54:25.240 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-13T21:54:25.240 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-13T21:54:25.241 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-13T21:54:25.241 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-13T21:54:25.241 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-13T21:54:25.241 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-13T21:54:25.241 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-13T21:54:25.241 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-13T21:54:25.241 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-13T21:54:25.242 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-13T21:54:25.242 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-13T21:54:25.242 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-13T21:54:25.242 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-13T21:54:25.242 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-13T21:54:25.242 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-13T21:54:25.242 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-13T21:54:25.243 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-13T21:54:25.243 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-13T21:54:25.243 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-13T21:54:25.243 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-13T21:54:25.243 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-13T21:54:25.243 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-13T21:54:25.243 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-13T21:54:25.244 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-13T21:54:25.244 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-13T21:54:25.244 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-13T21:54:25.244 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-13T21:54:25.244 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-13T21:54:25.244 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-13T21:54:25.244 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-13T21:54:25.245 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-13T21:54:25.245 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-13T21:54:25.245 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-13T21:54:25.245 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-13T21:54:25.245 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-13T21:54:25.245 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-13T21:54:25.245 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-13T21:54:25.246 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-13T21:54:25.246 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-13T21:54:25.246 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-13T21:54:25.246 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-13T21:54:25.246 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-13T21:54:25.246 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-13T21:54:25.246 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-13T21:54:25.246 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-13T21:54:25.247 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-13T21:54:25.247 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-13T21:54:25.247 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-13T21:54:25.247 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-13T21:54:25.247 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-13T21:54:25.247 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-13T21:54:25.247 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-13T21:54:25.248 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-13T21:54:25.248 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-13T21:54:25.248 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-13T21:54:25.248 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-13T21:54:25.248 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-13T21:54:25.248 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-13T21:54:25.248 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-13T21:54:25.249 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-13T21:54:25.249 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-13T21:54:25.249 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-13T21:54:25.249 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-13T21:54:25.249 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-13T21:54:25.249 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-13T21:54:25.249 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-13T21:54:25.250 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-13T21:54:25.250 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-13T21:54:25.250 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-13T21:54:25.250 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-13T21:54:25.250 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-13T21:54:25.250 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-13T21:54:25.250 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-13T21:54:25.251 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-13T21:54:25.251 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-13T21:54:25.251 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-13T21:54:25.251 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-13T21:54:25.251 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-13T21:54:25.251 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-13T21:54:25.251 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-13T21:54:25.252 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-13T21:54:25.252 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-13T21:54:25.252 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-13T21:54:25.252 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-13T21:54:25.252 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-13T21:54:25.252 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-13T21:54:25.252 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-13T21:54:25.252 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-13T21:54:25.253 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-13T21:54:25.253 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-13T21:54:25.253 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-13T21:54:25.253 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-13T21:54:25.253 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-13T21:54:25.253 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-13T21:54:25.253 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-13T21:54:25.254 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-13T21:54:25.254 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-13T21:54:25.254 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-13T21:54:25.254 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-13T21:54:25.254 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-13T21:54:25.254 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-13T21:54:25.254 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-13T21:54:25.255 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-13T21:54:25.255 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-13T21:54:25.255 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-13T21:54:25.255 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-13T21:54:25.255 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-13T21:54:25.255 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-13T21:54:25.255 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-13T21:54:25.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-13T21:54:25.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-13T21:54:25.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-13T21:54:25.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-13T21:54:25.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-13T21:54:25.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-13T21:54:25.256 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-13T21:54:25.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-13T21:54:25.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-13T21:54:25.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-13T21:54:25.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-13T21:54:25.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-13T21:54:25.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-13T21:54:25.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-13T21:54:25.257 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-13T21:54:25.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-13T21:54:25.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-13T21:54:25.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-13T21:54:25.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-13T21:54:25.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-13T21:54:25.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-13T21:54:25.258 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-13T21:54:25.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-13T21:54:25.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-13T21:54:25.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-13T21:54:25.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-13T21:54:25.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-13T21:54:25.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-13T21:54:25.259 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-13T21:54:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-13T21:54:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-13T21:54:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-13T21:54:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-13T21:54:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-13T21:54:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-13T21:54:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-13T21:54:25.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-13T21:54:25.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-13T21:54:25.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-13T21:54:25.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-13T21:54:25.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-13T21:54:25.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-13T21:54:25.261 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-13T21:54:25.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-13T21:54:25.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-13T21:54:25.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-13T21:54:25.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-13T21:54:25.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-13T21:54:25.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-13T21:54:25.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-13T21:54:25.262 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-13T21:54:25.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-13T21:54:25.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-13T21:54:25.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-13T21:54:25.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-13T21:54:25.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-13T21:54:25.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-13T21:54:25.263 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-13T21:54:25.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-13T21:54:25.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-13T21:54:25.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-13T21:54:25.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-13T21:54:25.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-13T21:54:25.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-13T21:54:25.264 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-13T21:54:25.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-13T21:54:25.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-13T21:54:25.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-13T21:54:25.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-13T21:54:25.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-13T21:54:25.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-13T21:54:25.265 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-13T21:54:25.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-13T21:54:25.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-13T21:54:25.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-13T21:54:25.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-13T21:54:25.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-13T21:54:25.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-13T21:54:25.266 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-13T21:54:25.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-13T21:54:25.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-13T21:54:25.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-13T21:54:25.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-13T21:54:25.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-13T21:54:25.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-13T21:54:25.267 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-13T21:54:25.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-13T21:54:25.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-13T21:54:25.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-13T21:54:25.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-13T21:54:25.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-13T21:54:25.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-13T21:54:25.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-13T21:54:25.268 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-13T21:54:25.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-13T21:54:25.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-13T21:54:25.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-13T21:54:25.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-13T21:54:25.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-13T21:54:25.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-13T21:54:25.269 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-13T21:54:25.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-13T21:54:25.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-13T21:54:25.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-13T21:54:25.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-13T21:54:25.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-13T21:54:25.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-13T21:54:25.270 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-13T21:54:25.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-13T21:54:25.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-13T21:54:25.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-13T21:54:25.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-13T21:54:25.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-13T21:54:25.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-13T21:54:25.271 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-13T21:54:25.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-13T21:54:25.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-13T21:54:25.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-13T21:54:25.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-13T21:54:25.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-13T21:54:25.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-13T21:54:25.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-13T21:54:25.272 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-13T21:54:25.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-13T21:54:25.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-13T21:54:25.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-13T21:54:25.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-13T21:54:25.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-13T21:54:25.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-13T21:54:25.273 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-13T21:54:25.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-13T21:54:25.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-13T21:54:25.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-13T21:54:25.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-13T21:54:25.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-13T21:54:25.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-13T21:54:25.274 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-13T21:54:25.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-13T21:54:25.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-13T21:54:25.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-13T21:54:25.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-13T21:54:25.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-13T21:54:25.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-13T21:54:25.275 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-13T21:54:25.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-13T21:54:25.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-13T21:54:25.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-13T21:54:25.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-13T21:54:25.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-13T21:54:25.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-13T21:54:25.276 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-13T21:54:25.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-13T21:54:25.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-13T21:54:25.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-13T21:54:25.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-13T21:54:25.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-13T21:54:25.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-13T21:54:25.277 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-13T21:54:25.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-13T21:54:25.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-13T21:54:25.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-13T21:54:25.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-13T21:54:25.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-13T21:54:25.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-13T21:54:25.278 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-13T21:54:25.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-13T21:54:25.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-13T21:54:25.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-13T21:54:25.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-13T21:54:25.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-13T21:54:25.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-13T21:54:25.279 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-13T21:54:25.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-13T21:54:25.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-13T21:54:25.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-13T21:54:25.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-13T21:54:25.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-13T21:54:25.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-13T21:54:25.280 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-13T21:54:25.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-13T21:54:25.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-13T21:54:25.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-13T21:54:25.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-13T21:54:25.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-13T21:54:25.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-13T21:54:25.281 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-13T21:54:25.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-13T21:54:25.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-13T21:54:25.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-13T21:54:25.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-13T21:54:25.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-13T21:54:25.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-13T21:54:25.282 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-13T21:54:25.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-13T21:54:25.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-13T21:54:25.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-13T21:54:25.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-13T21:54:25.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-13T21:54:25.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-13T21:54:25.283 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-13T21:54:25.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-13T21:54:25.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-13T21:54:25.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-13T21:54:25.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-13T21:54:25.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-13T21:54:25.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-13T21:54:25.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-13T21:54:25.284 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-13T21:54:25.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-13T21:54:25.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-13T21:54:25.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-13T21:54:25.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-13T21:54:25.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-13T21:54:25.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-13T21:54:25.285 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-13T21:54:25.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-13T21:54:25.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-13T21:54:25.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-13T21:54:25.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-13T21:54:25.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-13T21:54:25.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-13T21:54:25.286 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-13T21:54:25.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-13T21:54:25.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-13T21:54:25.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-13T21:54:25.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-13T21:54:25.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-13T21:54:25.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-13T21:54:25.287 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-13T21:54:25.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-13T21:54:25.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-13T21:54:25.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-13T21:54:25.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-13T21:54:25.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-13T21:54:25.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-13T21:54:25.288 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-13T21:54:25.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-13T21:54:25.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-13T21:54:25.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-13T21:54:25.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-13T21:54:25.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-13T21:54:25.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-13T21:54:25.289 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-13T21:54:25.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-13T21:54:25.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-13T21:54:25.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-13T21:54:25.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-13T21:54:25.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-13T21:54:25.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-13T21:54:25.290 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-13T21:54:25.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-13T21:54:25.291 INFO:teuthology.orchestra.run.smithi105.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-13T21:54:25.291 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:25.292 DEBUG:teuthology.orchestra.run.smithi105:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-13T21:54:25.320 INFO:teuthology.orchestra.run.smithi105.stdout:postinstall scriptlet (using /bin/sh): 2023-07-13T21:54:25.321 INFO:teuthology.orchestra.run.smithi105.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-13T21:54:25.321 INFO:teuthology.orchestra.run.smithi105.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:54:25.321 INFO:teuthology.orchestra.run.smithi105.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:54:25.321 INFO:teuthology.orchestra.run.smithi105.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:25.321 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:54:25.321 INFO:teuthology.orchestra.run.smithi105.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:54:25.322 INFO:teuthology.orchestra.run.smithi105.stdout:fi 2023-07-13T21:54:25.322 INFO:teuthology.orchestra.run.smithi105.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-13T21:54:25.322 INFO:teuthology.orchestra.run.smithi105.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-13T21:54:25.322 INFO:teuthology.orchestra.run.smithi105.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-13T21:54:25.322 INFO:teuthology.orchestra.run.smithi105.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-13T21:54:25.322 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:25.322 INFO:teuthology.orchestra.run.smithi105.stdout:fi 2023-07-13T21:54:25.323 INFO:teuthology.orchestra.run.smithi105.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-13T21:54:25.323 INFO:teuthology.orchestra.run.smithi105.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-13T21:54:25.323 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:25.323 INFO:teuthology.orchestra.run.smithi105.stdout:fi 2023-07-13T21:54:25.324 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2023-07-13T21:54:25.403 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2023-07-13T21:54:25.404 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2023-07-13T21:54:25.404 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.02 2023-07-13T21:54:25.404 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 129.el8 2023-07-13T21:54:25.404 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2023-07-13T21:54:25.404 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-07-13T21:54:25.405 INFO:teuthology.orchestra.run.smithi105.stdout:Group : System Environment/Base 2023-07-13T21:54:25.405 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 9265586 2023-07-13T21:54:25.405 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2023-07-13T21:54:25.405 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-07-13T21:54:25.405 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-07-13T21:54:25.405 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-07-13T21:54:25.405 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : kbuilder.bsys.centos.org 2023-07-13T21:54:25.406 INFO:teuthology.orchestra.run.smithi105.stdout:Relocations : (not relocatable) 2023-07-13T21:54:25.406 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : CentOS BuildSystem 2023-07-13T21:54:25.406 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2023-07-13T21:54:25.406 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-13T21:54:25.406 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2023-07-13T21:54:25.406 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2023-07-13T21:54:25.406 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T21:54:25.406 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-13T21:54:25.407 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-13T21:54:25.407 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-13T21:54:25.407 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2023-07-13T21:54:25.407 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T21:54:25.407 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2023-07-13T21:54:25.408 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-13T21:54:25.408 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:25.409 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-13T21:54:27.075 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-13T21:54:27.075 DEBUG:teuthology.orchestra.run.smithi060:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-13T21:54:27.138 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:27.139 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:27.139 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:27.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:27.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-13T21:54:27.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-13T21:54:27.140 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-13T21:54:27.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-13T21:54:27.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-13T21:54:27.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-13T21:54:27.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-13T21:54:27.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-13T21:54:27.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-13T21:54:27.141 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-13T21:54:27.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-13T21:54:27.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-13T21:54:27.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-13T21:54:27.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-13T21:54:27.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-13T21:54:27.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-13T21:54:27.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-13T21:54:27.142 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-13T21:54:27.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-13T21:54:27.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-13T21:54:27.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-13T21:54:27.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-13T21:54:27.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-13T21:54:27.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-13T21:54:27.143 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-13T21:54:27.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-13T21:54:27.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-13T21:54:27.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-13T21:54:27.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-13T21:54:27.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-13T21:54:27.144 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-13T21:54:27.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-13T21:54:27.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-13T21:54:27.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-13T21:54:27.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-13T21:54:27.145 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-13T21:54:27.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-13T21:54:27.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-13T21:54:27.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-13T21:54:27.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-13T21:54:27.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-13T21:54:27.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-13T21:54:27.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-13T21:54:27.146 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-13T21:54:27.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-13T21:54:27.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-13T21:54:27.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-13T21:54:27.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-13T21:54:27.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-13T21:54:27.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-13T21:54:27.147 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-13T21:54:27.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-13T21:54:27.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-13T21:54:27.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-13T21:54:27.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-13T21:54:27.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-13T21:54:27.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-13T21:54:27.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-13T21:54:27.148 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-13T21:54:27.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-13T21:54:27.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-13T21:54:27.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-13T21:54:27.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-13T21:54:27.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-13T21:54:27.149 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-13T21:54:27.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-13T21:54:27.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-13T21:54:27.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-13T21:54:27.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-13T21:54:27.150 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-13T21:54:27.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-13T21:54:27.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-13T21:54:27.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-13T21:54:27.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-13T21:54:27.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-13T21:54:27.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-13T21:54:27.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-13T21:54:27.151 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-13T21:54:27.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-13T21:54:27.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-13T21:54:27.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-13T21:54:27.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-13T21:54:27.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-13T21:54:27.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-13T21:54:27.152 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-13T21:54:27.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-13T21:54:27.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-13T21:54:27.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-13T21:54:27.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-13T21:54:27.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-13T21:54:27.153 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-13T21:54:27.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-13T21:54:27.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-13T21:54:27.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-13T21:54:27.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-13T21:54:27.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-13T21:54:27.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-13T21:54:27.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-13T21:54:27.154 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-13T21:54:27.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-13T21:54:27.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-13T21:54:27.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-13T21:54:27.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-13T21:54:27.155 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-13T21:54:27.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-13T21:54:27.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-13T21:54:27.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-13T21:54:27.156 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-13T21:54:27.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-13T21:54:27.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-13T21:54:27.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-13T21:54:27.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-13T21:54:27.157 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-13T21:54:27.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-13T21:54:27.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-13T21:54:27.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-13T21:54:27.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-13T21:54:27.158 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-13T21:54:27.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-13T21:54:27.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-13T21:54:27.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-13T21:54:27.159 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-13T21:54:27.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-13T21:54:27.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-13T21:54:27.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-13T21:54:27.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-13T21:54:27.160 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-13T21:54:27.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-13T21:54:27.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-13T21:54:27.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-13T21:54:27.161 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-13T21:54:27.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-13T21:54:27.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-13T21:54:27.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-13T21:54:27.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-13T21:54:27.162 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-13T21:54:27.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-13T21:54:27.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-13T21:54:27.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-13T21:54:27.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-13T21:54:27.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-13T21:54:27.163 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-13T21:54:27.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-13T21:54:27.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-13T21:54:27.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-13T21:54:27.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-13T21:54:27.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-13T21:54:27.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-13T21:54:27.164 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-13T21:54:27.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-13T21:54:27.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-13T21:54:27.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-13T21:54:27.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-13T21:54:27.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-13T21:54:27.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-13T21:54:27.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-13T21:54:27.165 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-13T21:54:27.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-13T21:54:27.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-13T21:54:27.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-13T21:54:27.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-13T21:54:27.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-13T21:54:27.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-13T21:54:27.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-13T21:54:27.166 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-13T21:54:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-13T21:54:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-13T21:54:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-13T21:54:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-13T21:54:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-13T21:54:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-13T21:54:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-13T21:54:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-13T21:54:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-13T21:54:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-13T21:54:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-13T21:54:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-13T21:54:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-13T21:54:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-13T21:54:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-13T21:54:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-13T21:54:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-13T21:54:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-13T21:54:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-13T21:54:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-13T21:54:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-13T21:54:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-13T21:54:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-13T21:54:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-13T21:54:27.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-13T21:54:27.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-13T21:54:27.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-13T21:54:27.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-13T21:54:27.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-13T21:54:27.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-13T21:54:27.170 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-13T21:54:27.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-13T21:54:27.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-13T21:54:27.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-13T21:54:27.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-13T21:54:27.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-13T21:54:27.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-13T21:54:27.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-13T21:54:27.171 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-13T21:54:27.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-13T21:54:27.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-13T21:54:27.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-13T21:54:27.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-13T21:54:27.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-13T21:54:27.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-13T21:54:27.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-13T21:54:27.172 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-13T21:54:27.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-13T21:54:27.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-13T21:54:27.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-13T21:54:27.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-13T21:54:27.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-13T21:54:27.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-13T21:54:27.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-13T21:54:27.173 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-13T21:54:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-13T21:54:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-13T21:54:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-13T21:54:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-13T21:54:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-13T21:54:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-13T21:54:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-13T21:54:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-13T21:54:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-13T21:54:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-13T21:54:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-13T21:54:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-13T21:54:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-13T21:54:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-13T21:54:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-13T21:54:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-13T21:54:27.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-13T21:54:27.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-13T21:54:27.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-13T21:54:27.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-13T21:54:27.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-13T21:54:27.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-13T21:54:27.176 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-13T21:54:27.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-13T21:54:27.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-13T21:54:27.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-13T21:54:27.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-13T21:54:27.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-13T21:54:27.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-13T21:54:27.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-13T21:54:27.177 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-13T21:54:27.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-13T21:54:27.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-13T21:54:27.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-13T21:54:27.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-13T21:54:27.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-13T21:54:27.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-13T21:54:27.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-13T21:54:27.178 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-13T21:54:27.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-13T21:54:27.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-13T21:54:27.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-13T21:54:27.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-13T21:54:27.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-13T21:54:27.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-13T21:54:27.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-13T21:54:27.179 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-13T21:54:27.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-13T21:54:27.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-13T21:54:27.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-13T21:54:27.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-13T21:54:27.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-13T21:54:27.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-13T21:54:27.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-13T21:54:27.180 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-13T21:54:27.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-13T21:54:27.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-13T21:54:27.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-13T21:54:27.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-13T21:54:27.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-13T21:54:27.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-13T21:54:27.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-13T21:54:27.181 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-13T21:54:27.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-13T21:54:27.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-13T21:54:27.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-13T21:54:27.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-13T21:54:27.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-13T21:54:27.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-13T21:54:27.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-13T21:54:27.182 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-13T21:54:27.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-13T21:54:27.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-13T21:54:27.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-13T21:54:27.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-13T21:54:27.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-13T21:54:27.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-13T21:54:27.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-13T21:54:27.183 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-13T21:54:27.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-13T21:54:27.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-13T21:54:27.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-13T21:54:27.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-13T21:54:27.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-13T21:54:27.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-13T21:54:27.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-13T21:54:27.184 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-13T21:54:27.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-13T21:54:27.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-13T21:54:27.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-13T21:54:27.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-13T21:54:27.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-13T21:54:27.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-13T21:54:27.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-13T21:54:27.185 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-13T21:54:27.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-13T21:54:27.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-13T21:54:27.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-13T21:54:27.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-13T21:54:27.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-13T21:54:27.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-13T21:54:27.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-13T21:54:27.186 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-13T21:54:27.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-13T21:54:27.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-13T21:54:27.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-13T21:54:27.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-13T21:54:27.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-13T21:54:27.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-13T21:54:27.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-13T21:54:27.187 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-13T21:54:27.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-13T21:54:27.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-13T21:54:27.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-13T21:54:27.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-13T21:54:27.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-13T21:54:27.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-13T21:54:27.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-13T21:54:27.188 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-13T21:54:27.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-13T21:54:27.189 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-13T21:54:27.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-13T21:54:27.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-13T21:54:27.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-13T21:54:27.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-13T21:54:27.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-13T21:54:27.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-13T21:54:27.191 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-13T21:54:27.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-13T21:54:27.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-13T21:54:27.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-13T21:54:27.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-13T21:54:27.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-13T21:54:27.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-13T21:54:27.192 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-13T21:54:27.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-13T21:54:27.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-13T21:54:27.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-13T21:54:27.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-13T21:54:27.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-13T21:54:27.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-13T21:54:27.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-13T21:54:27.193 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-13T21:54:27.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-13T21:54:27.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-13T21:54:27.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-13T21:54:27.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-13T21:54:27.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-13T21:54:27.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-13T21:54:27.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-13T21:54:27.194 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-13T21:54:27.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-13T21:54:27.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-13T21:54:27.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-13T21:54:27.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-13T21:54:27.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-13T21:54:27.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-13T21:54:27.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-13T21:54:27.195 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-13T21:54:27.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-13T21:54:27.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-13T21:54:27.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-13T21:54:27.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-13T21:54:27.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-13T21:54:27.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-13T21:54:27.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-13T21:54:27.196 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-13T21:54:27.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-13T21:54:27.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-13T21:54:27.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-13T21:54:27.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-13T21:54:27.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-13T21:54:27.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-13T21:54:27.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-13T21:54:27.197 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-13T21:54:27.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-13T21:54:27.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-13T21:54:27.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-13T21:54:27.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-13T21:54:27.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-13T21:54:27.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-13T21:54:27.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-13T21:54:27.198 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-13T21:54:27.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-13T21:54:27.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-13T21:54:27.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-13T21:54:27.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-13T21:54:27.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-13T21:54:27.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-13T21:54:27.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-13T21:54:27.199 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-13T21:54:27.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-13T21:54:27.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-13T21:54:27.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-13T21:54:27.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-13T21:54:27.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-13T21:54:27.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-13T21:54:27.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-13T21:54:27.200 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-13T21:54:27.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-13T21:54:27.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-13T21:54:27.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-13T21:54:27.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-13T21:54:27.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-13T21:54:27.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-13T21:54:27.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-13T21:54:27.201 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-13T21:54:27.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-13T21:54:27.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-13T21:54:27.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-13T21:54:27.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-13T21:54:27.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-13T21:54:27.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-13T21:54:27.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-13T21:54:27.202 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-13T21:54:27.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-13T21:54:27.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-13T21:54:27.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-13T21:54:27.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-13T21:54:27.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-13T21:54:27.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-13T21:54:27.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-13T21:54:27.203 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-13T21:54:27.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-13T21:54:27.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-13T21:54:27.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-13T21:54:27.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-13T21:54:27.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-13T21:54:27.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-13T21:54:27.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-13T21:54:27.204 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-13T21:54:27.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-13T21:54:27.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-13T21:54:27.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-13T21:54:27.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-13T21:54:27.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-13T21:54:27.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-13T21:54:27.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-13T21:54:27.205 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-13T21:54:27.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-13T21:54:27.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-13T21:54:27.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-13T21:54:27.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-13T21:54:27.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-13T21:54:27.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-13T21:54:27.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-13T21:54:27.206 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-13T21:54:27.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-13T21:54:27.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-13T21:54:27.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-13T21:54:27.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-13T21:54:27.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-13T21:54:27.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-13T21:54:27.207 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-13T21:54:27.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-13T21:54:27.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-13T21:54:27.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-13T21:54:27.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-13T21:54:27.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-13T21:54:27.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-13T21:54:27.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-13T21:54:27.208 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-13T21:54:27.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-13T21:54:27.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-13T21:54:27.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-13T21:54:27.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-13T21:54:27.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-13T21:54:27.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-13T21:54:27.209 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-13T21:54:27.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-13T21:54:27.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-13T21:54:27.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-13T21:54:27.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-13T21:54:27.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-13T21:54:27.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-13T21:54:27.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-13T21:54:27.210 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-13T21:54:27.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-13T21:54:27.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-13T21:54:27.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-13T21:54:27.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-13T21:54:27.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-13T21:54:27.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-13T21:54:27.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-13T21:54:27.211 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-13T21:54:27.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-13T21:54:27.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-13T21:54:27.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-13T21:54:27.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-13T21:54:27.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-13T21:54:27.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-13T21:54:27.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-13T21:54:27.212 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-13T21:54:27.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-13T21:54:27.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-13T21:54:27.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-13T21:54:27.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-13T21:54:27.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-13T21:54:27.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-13T21:54:27.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-13T21:54:27.213 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-13T21:54:27.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-13T21:54:27.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-13T21:54:27.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-13T21:54:27.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-13T21:54:27.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-13T21:54:27.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-13T21:54:27.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-13T21:54:27.214 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-13T21:54:27.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-13T21:54:27.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-13T21:54:27.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-13T21:54:27.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-13T21:54:27.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-13T21:54:27.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-13T21:54:27.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-13T21:54:27.215 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-13T21:54:27.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-13T21:54:27.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-13T21:54:27.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-13T21:54:27.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-13T21:54:27.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-13T21:54:27.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-13T21:54:27.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-13T21:54:27.216 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-13T21:54:27.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-13T21:54:27.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-13T21:54:27.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-13T21:54:27.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-13T21:54:27.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-13T21:54:27.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-13T21:54:27.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-13T21:54:27.217 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-13T21:54:27.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-13T21:54:27.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-13T21:54:27.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-13T21:54:27.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-13T21:54:27.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-13T21:54:27.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-13T21:54:27.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-13T21:54:27.218 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-13T21:54:27.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-13T21:54:27.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-13T21:54:27.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-13T21:54:27.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-13T21:54:27.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-13T21:54:27.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-13T21:54:27.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-13T21:54:27.219 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-13T21:54:27.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-13T21:54:27.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-13T21:54:27.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-13T21:54:27.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-13T21:54:27.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-13T21:54:27.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-13T21:54:27.220 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-13T21:54:27.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-13T21:54:27.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-13T21:54:27.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-13T21:54:27.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-13T21:54:27.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-13T21:54:27.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-13T21:54:27.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-13T21:54:27.221 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-13T21:54:27.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-13T21:54:27.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-13T21:54:27.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-13T21:54:27.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-13T21:54:27.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-13T21:54:27.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-13T21:54:27.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-13T21:54:27.222 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-13T21:54:27.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-13T21:54:27.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-13T21:54:27.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-13T21:54:27.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-13T21:54:27.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-13T21:54:27.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-13T21:54:27.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-13T21:54:27.223 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-13T21:54:27.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-13T21:54:27.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-13T21:54:27.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-13T21:54:27.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-13T21:54:27.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-13T21:54:27.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-13T21:54:27.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-13T21:54:27.224 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-13T21:54:27.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-13T21:54:27.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-13T21:54:27.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-13T21:54:27.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-13T21:54:27.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-13T21:54:27.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-13T21:54:27.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-13T21:54:27.225 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-13T21:54:27.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-13T21:54:27.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-13T21:54:27.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-13T21:54:27.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-13T21:54:27.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-13T21:54:27.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-13T21:54:27.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-13T21:54:27.226 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-13T21:54:27.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-13T21:54:27.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-13T21:54:27.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-13T21:54:27.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-13T21:54:27.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-13T21:54:27.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-13T21:54:27.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-13T21:54:27.227 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-13T21:54:27.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-13T21:54:27.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-13T21:54:27.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-13T21:54:27.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-13T21:54:27.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-13T21:54:27.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-13T21:54:27.228 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-13T21:54:27.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-13T21:54:27.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-13T21:54:27.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-13T21:54:27.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-13T21:54:27.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-13T21:54:27.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-13T21:54:27.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-13T21:54:27.229 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-13T21:54:27.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-13T21:54:27.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-13T21:54:27.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-13T21:54:27.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-13T21:54:27.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-13T21:54:27.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-13T21:54:27.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-13T21:54:27.230 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-13T21:54:27.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-13T21:54:27.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-13T21:54:27.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-13T21:54:27.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-13T21:54:27.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-13T21:54:27.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-13T21:54:27.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-13T21:54:27.231 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-13T21:54:27.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-13T21:54:27.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-13T21:54:27.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-13T21:54:27.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-13T21:54:27.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-13T21:54:27.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-13T21:54:27.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-13T21:54:27.232 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-13T21:54:27.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-13T21:54:27.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-13T21:54:27.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-13T21:54:27.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-13T21:54:27.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-13T21:54:27.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-13T21:54:27.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-13T21:54:27.233 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-13T21:54:27.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-13T21:54:27.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-13T21:54:27.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-13T21:54:27.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-13T21:54:27.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-13T21:54:27.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-13T21:54:27.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-13T21:54:27.234 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-13T21:54:27.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-13T21:54:27.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-13T21:54:27.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-13T21:54:27.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-13T21:54:27.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-13T21:54:27.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-13T21:54:27.235 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-13T21:54:27.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-13T21:54:27.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-13T21:54:27.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-13T21:54:27.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-13T21:54:27.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-13T21:54:27.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-13T21:54:27.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-13T21:54:27.236 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-13T21:54:27.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-13T21:54:27.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-13T21:54:27.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-13T21:54:27.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-13T21:54:27.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-13T21:54:27.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-13T21:54:27.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-13T21:54:27.237 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-13T21:54:27.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-13T21:54:27.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-13T21:54:27.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-13T21:54:27.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-13T21:54:27.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-13T21:54:27.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-13T21:54:27.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-13T21:54:27.238 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-13T21:54:27.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-13T21:54:27.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-13T21:54:27.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-13T21:54:27.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-13T21:54:27.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-13T21:54:27.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-13T21:54:27.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-13T21:54:27.239 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-13T21:54:27.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-13T21:54:27.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-13T21:54:27.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-13T21:54:27.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-13T21:54:27.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-13T21:54:27.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-13T21:54:27.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-13T21:54:27.240 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-13T21:54:27.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-13T21:54:27.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-13T21:54:27.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-13T21:54:27.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-13T21:54:27.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-13T21:54:27.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-13T21:54:27.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-13T21:54:27.241 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-13T21:54:27.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-13T21:54:27.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-13T21:54:27.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-13T21:54:27.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-13T21:54:27.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-13T21:54:27.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-13T21:54:27.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-13T21:54:27.242 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-13T21:54:27.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-13T21:54:27.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-13T21:54:27.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-13T21:54:27.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-13T21:54:27.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-13T21:54:27.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-13T21:54:27.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-13T21:54:27.243 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-13T21:54:27.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-13T21:54:27.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-13T21:54:27.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-13T21:54:27.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-13T21:54:27.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-13T21:54:27.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-13T21:54:27.244 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-13T21:54:27.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-13T21:54:27.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-13T21:54:27.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-13T21:54:27.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-13T21:54:27.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-13T21:54:27.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-13T21:54:27.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-13T21:54:27.245 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-13T21:54:27.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-13T21:54:27.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-13T21:54:27.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-13T21:54:27.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-13T21:54:27.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-13T21:54:27.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-13T21:54:27.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-13T21:54:27.246 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-13T21:54:27.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-13T21:54:27.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-13T21:54:27.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-13T21:54:27.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-13T21:54:27.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-13T21:54:27.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-13T21:54:27.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-13T21:54:27.247 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-13T21:54:27.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-13T21:54:27.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-13T21:54:27.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-13T21:54:27.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-13T21:54:27.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-13T21:54:27.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-13T21:54:27.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-13T21:54:27.248 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-13T21:54:27.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-13T21:54:27.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-13T21:54:27.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-13T21:54:27.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-13T21:54:27.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-13T21:54:27.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-13T21:54:27.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-13T21:54:27.249 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-13T21:54:27.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-13T21:54:27.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-13T21:54:27.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-13T21:54:27.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-13T21:54:27.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-13T21:54:27.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-13T21:54:27.250 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-13T21:54:27.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-13T21:54:27.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-13T21:54:27.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-13T21:54:27.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-13T21:54:27.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-13T21:54:27.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-13T21:54:27.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-13T21:54:27.251 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-13T21:54:27.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-13T21:54:27.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-13T21:54:27.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-13T21:54:27.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-13T21:54:27.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-13T21:54:27.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-13T21:54:27.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-13T21:54:27.252 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-13T21:54:27.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-13T21:54:27.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-13T21:54:27.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-13T21:54:27.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-13T21:54:27.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-13T21:54:27.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-13T21:54:27.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-13T21:54:27.253 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-13T21:54:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-13T21:54:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-13T21:54:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-13T21:54:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-13T21:54:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-13T21:54:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-13T21:54:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-13T21:54:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-13T21:54:27.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-13T21:54:27.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-13T21:54:27.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-13T21:54:27.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-13T21:54:27.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-13T21:54:27.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-13T21:54:27.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-13T21:54:27.255 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-13T21:54:27.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-13T21:54:27.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-13T21:54:27.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-13T21:54:27.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-13T21:54:27.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-13T21:54:27.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-13T21:54:27.256 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-13T21:54:27.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-13T21:54:27.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-13T21:54:27.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-13T21:54:27.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-13T21:54:27.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-13T21:54:27.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-13T21:54:27.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-13T21:54:27.257 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-13T21:54:27.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-13T21:54:27.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-13T21:54:27.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-13T21:54:27.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-13T21:54:27.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-13T21:54:27.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-13T21:54:27.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-13T21:54:27.258 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-13T21:54:27.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-13T21:54:27.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-13T21:54:27.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-13T21:54:27.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-13T21:54:27.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-13T21:54:27.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-13T21:54:27.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-13T21:54:27.259 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-13T21:54:27.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-13T21:54:27.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-13T21:54:27.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-13T21:54:27.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-13T21:54:27.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-13T21:54:27.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-13T21:54:27.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-13T21:54:27.260 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-13T21:54:27.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-13T21:54:27.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-13T21:54:27.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-13T21:54:27.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-13T21:54:27.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-13T21:54:27.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-13T21:54:27.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-13T21:54:27.261 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-13T21:54:27.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-13T21:54:27.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-13T21:54:27.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-13T21:54:27.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-13T21:54:27.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-13T21:54:27.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-13T21:54:27.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-13T21:54:27.262 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-13T21:54:27.263 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-13T21:54:27.263 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-13T21:54:27.263 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-13T21:54:27.263 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-13T21:54:27.263 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-13T21:54:27.263 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-13T21:54:27.263 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-13T21:54:27.263 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-13T21:54:27.264 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-13T21:54:27.264 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-13T21:54:27.264 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-13T21:54:27.264 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-13T21:54:27.264 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-13T21:54:27.264 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-13T21:54:27.264 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-13T21:54:27.264 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-13T21:54:27.265 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-13T21:54:27.265 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-13T21:54:27.265 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-13T21:54:27.265 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-13T21:54:27.265 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-13T21:54:27.265 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-13T21:54:27.265 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-13T21:54:27.266 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-13T21:54:27.266 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-13T21:54:27.266 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-13T21:54:27.266 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-13T21:54:27.266 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-13T21:54:27.266 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-13T21:54:27.266 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-13T21:54:27.266 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-13T21:54:27.267 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-13T21:54:27.267 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-13T21:54:27.267 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-13T21:54:27.267 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-13T21:54:27.267 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-13T21:54:27.267 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-13T21:54:27.267 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-13T21:54:27.267 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-13T21:54:27.268 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-13T21:54:27.268 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-13T21:54:27.268 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-13T21:54:27.268 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-13T21:54:27.268 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-13T21:54:27.268 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-13T21:54:27.268 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-13T21:54:27.268 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-13T21:54:27.269 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-13T21:54:27.269 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-13T21:54:27.269 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-13T21:54:27.269 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-13T21:54:27.269 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-13T21:54:27.269 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-13T21:54:27.269 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-13T21:54:27.269 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-13T21:54:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-13T21:54:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-13T21:54:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-13T21:54:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-13T21:54:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-13T21:54:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-13T21:54:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-13T21:54:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-13T21:54:27.271 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-13T21:54:27.271 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-13T21:54:27.271 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-13T21:54:27.271 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-13T21:54:27.271 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-13T21:54:27.271 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-13T21:54:27.271 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-13T21:54:27.271 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-13T21:54:27.272 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-13T21:54:27.272 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-13T21:54:27.272 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-13T21:54:27.272 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-13T21:54:27.272 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-13T21:54:27.272 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-13T21:54:27.272 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-13T21:54:27.272 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-13T21:54:27.273 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-13T21:54:27.273 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-13T21:54:27.273 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-13T21:54:27.273 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-13T21:54:27.273 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-13T21:54:27.273 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-13T21:54:27.273 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-13T21:54:27.273 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-13T21:54:27.274 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-13T21:54:27.274 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-13T21:54:27.274 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-13T21:54:27.274 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-13T21:54:27.274 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-13T21:54:27.274 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-13T21:54:27.274 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-13T21:54:27.274 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-13T21:54:27.275 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-13T21:54:27.275 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-13T21:54:27.275 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-13T21:54:27.275 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-13T21:54:27.275 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-13T21:54:27.275 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-13T21:54:27.275 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-13T21:54:27.275 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-13T21:54:27.276 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-13T21:54:27.276 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-13T21:54:27.276 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-13T21:54:27.276 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-13T21:54:27.276 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-13T21:54:27.276 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-13T21:54:27.276 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-13T21:54:27.276 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-13T21:54:27.277 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-13T21:54:27.277 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-13T21:54:27.277 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-13T21:54:27.277 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-13T21:54:27.277 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-13T21:54:27.277 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-13T21:54:27.277 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-13T21:54:27.277 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-13T21:54:27.278 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-13T21:54:27.278 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-13T21:54:27.278 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-13T21:54:27.278 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-13T21:54:27.278 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-13T21:54:27.278 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-13T21:54:27.278 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-13T21:54:27.278 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-13T21:54:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-13T21:54:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-13T21:54:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-13T21:54:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-13T21:54:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-13T21:54:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-13T21:54:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-13T21:54:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-13T21:54:27.280 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-13T21:54:27.280 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-13T21:54:27.280 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-13T21:54:27.280 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-13T21:54:27.280 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-13T21:54:27.280 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-13T21:54:27.280 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-13T21:54:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-13T21:54:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-13T21:54:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-13T21:54:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-13T21:54:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-13T21:54:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-13T21:54:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-13T21:54:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-13T21:54:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-13T21:54:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-13T21:54:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-13T21:54:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-13T21:54:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-13T21:54:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-13T21:54:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-13T21:54:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-13T21:54:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-13T21:54:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-13T21:54:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-13T21:54:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-13T21:54:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-13T21:54:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-13T21:54:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-13T21:54:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-13T21:54:27.283 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-13T21:54:27.284 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-13T21:54:27.284 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-13T21:54:27.284 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-13T21:54:27.284 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-13T21:54:27.284 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-13T21:54:27.284 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-13T21:54:27.284 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-13T21:54:27.285 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-13T21:54:27.285 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-13T21:54:27.285 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-13T21:54:27.285 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-13T21:54:27.285 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-13T21:54:27.285 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-13T21:54:27.285 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-13T21:54:27.285 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-13T21:54:27.286 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-13T21:54:27.286 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-13T21:54:27.286 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-13T21:54:27.286 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-13T21:54:27.286 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-13T21:54:27.286 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-13T21:54:27.286 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-13T21:54:27.286 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-13T21:54:27.287 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-13T21:54:27.287 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-13T21:54:27.287 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-13T21:54:27.287 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-13T21:54:27.287 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-13T21:54:27.287 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-13T21:54:27.287 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-13T21:54:27.287 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-13T21:54:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-13T21:54:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-13T21:54:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-13T21:54:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-13T21:54:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-13T21:54:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-13T21:54:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-13T21:54:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-13T21:54:27.289 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-13T21:54:27.289 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-13T21:54:27.289 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-13T21:54:27.289 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-13T21:54:27.289 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-13T21:54:27.289 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-13T21:54:27.289 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-13T21:54:27.289 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-13T21:54:27.290 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-13T21:54:27.290 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-13T21:54:27.290 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-13T21:54:27.290 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-13T21:54:27.290 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-13T21:54:27.290 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-13T21:54:27.290 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-13T21:54:27.290 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-13T21:54:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-13T21:54:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-13T21:54:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-13T21:54:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-13T21:54:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-13T21:54:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-13T21:54:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-13T21:54:27.291 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-13T21:54:27.292 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-13T21:54:27.292 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-13T21:54:27.292 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-13T21:54:27.292 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-13T21:54:27.292 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-13T21:54:27.292 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-13T21:54:27.292 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-13T21:54:27.293 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-13T21:54:27.293 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-13T21:54:27.293 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-13T21:54:27.293 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-13T21:54:27.293 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-13T21:54:27.293 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-13T21:54:27.293 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-13T21:54:27.293 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-13T21:54:27.294 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-13T21:54:27.294 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-13T21:54:27.294 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-13T21:54:27.294 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-13T21:54:27.294 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-13T21:54:27.294 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-13T21:54:27.294 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-13T21:54:27.294 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-13T21:54:27.295 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-13T21:54:27.295 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-13T21:54:27.295 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-13T21:54:27.295 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-13T21:54:27.295 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-13T21:54:27.295 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-13T21:54:27.295 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-13T21:54:27.295 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-13T21:54:27.296 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-13T21:54:27.296 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-13T21:54:27.296 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-13T21:54:27.296 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-13T21:54:27.296 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-13T21:54:27.296 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-13T21:54:27.296 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-13T21:54:27.296 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-13T21:54:27.297 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-13T21:54:27.297 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-13T21:54:27.297 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-13T21:54:27.297 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-13T21:54:27.297 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-13T21:54:27.297 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-13T21:54:27.297 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-13T21:54:27.297 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-13T21:54:27.298 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-13T21:54:27.298 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-13T21:54:27.298 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-13T21:54:27.298 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-13T21:54:27.298 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-13T21:54:27.298 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-13T21:54:27.298 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-13T21:54:27.298 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-13T21:54:27.299 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-13T21:54:27.299 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-13T21:54:27.299 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-13T21:54:27.299 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-13T21:54:27.299 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-13T21:54:27.299 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-13T21:54:27.299 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-13T21:54:27.299 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-13T21:54:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-13T21:54:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-13T21:54:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-13T21:54:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-13T21:54:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-13T21:54:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-13T21:54:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-13T21:54:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-13T21:54:27.301 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-13T21:54:27.301 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-13T21:54:27.301 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-13T21:54:27.301 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-13T21:54:27.301 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-13T21:54:27.301 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-13T21:54:27.301 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-13T21:54:27.301 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-13T21:54:27.302 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-13T21:54:27.302 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-13T21:54:27.302 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-13T21:54:27.302 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-13T21:54:27.302 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-13T21:54:27.302 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-13T21:54:27.302 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-13T21:54:27.302 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-13T21:54:27.303 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-13T21:54:27.303 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-13T21:54:27.303 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-13T21:54:27.303 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-13T21:54:27.303 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-13T21:54:27.303 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-13T21:54:27.303 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-13T21:54:27.303 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-13T21:54:27.304 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-13T21:54:27.304 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-13T21:54:27.304 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-13T21:54:27.304 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-13T21:54:27.304 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-13T21:54:27.304 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-13T21:54:27.304 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-13T21:54:27.305 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-13T21:54:27.305 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-13T21:54:27.305 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-13T21:54:27.305 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-13T21:54:27.305 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-13T21:54:27.305 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-13T21:54:27.305 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-13T21:54:27.305 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-13T21:54:27.306 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-13T21:54:27.306 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-13T21:54:27.306 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-13T21:54:27.306 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-13T21:54:27.306 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-13T21:54:27.306 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-13T21:54:27.306 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-13T21:54:27.306 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-13T21:54:27.307 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-13T21:54:27.307 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-13T21:54:27.307 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-13T21:54:27.307 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-13T21:54:27.307 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-13T21:54:27.307 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-13T21:54:27.307 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-13T21:54:27.307 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-13T21:54:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-13T21:54:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-13T21:54:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-13T21:54:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-13T21:54:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-13T21:54:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-13T21:54:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-13T21:54:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-13T21:54:27.309 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-13T21:54:27.309 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-13T21:54:27.309 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-13T21:54:27.309 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-13T21:54:27.309 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-13T21:54:27.309 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-13T21:54:27.309 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-13T21:54:27.309 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-13T21:54:27.310 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-13T21:54:27.310 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-13T21:54:27.310 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-13T21:54:27.310 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-13T21:54:27.310 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-13T21:54:27.310 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-13T21:54:27.310 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-13T21:54:27.311 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-13T21:54:27.311 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-13T21:54:27.311 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-13T21:54:27.311 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-13T21:54:27.311 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-13T21:54:27.311 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-13T21:54:27.311 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-13T21:54:27.311 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-13T21:54:27.312 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-13T21:54:27.312 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-13T21:54:27.312 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-13T21:54:27.312 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-13T21:54:27.312 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-13T21:54:27.312 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-13T21:54:27.312 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-13T21:54:27.312 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-13T21:54:27.313 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-13T21:54:27.313 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-13T21:54:27.313 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-13T21:54:27.313 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-13T21:54:27.313 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-13T21:54:27.313 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-13T21:54:27.313 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-13T21:54:27.313 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-13T21:54:27.314 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-13T21:54:27.314 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-13T21:54:27.314 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-13T21:54:27.314 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-13T21:54:27.314 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-13T21:54:27.314 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-13T21:54:27.314 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-13T21:54:27.314 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-13T21:54:27.315 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-13T21:54:27.315 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-13T21:54:27.315 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-13T21:54:27.315 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-13T21:54:27.315 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-13T21:54:27.315 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-13T21:54:27.315 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-13T21:54:27.315 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-13T21:54:27.316 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-13T21:54:27.316 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-13T21:54:27.316 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-13T21:54:27.316 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-13T21:54:27.316 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-13T21:54:27.316 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-13T21:54:27.316 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-13T21:54:27.316 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-13T21:54:27.317 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-13T21:54:27.317 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-13T21:54:27.317 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-13T21:54:27.317 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-13T21:54:27.317 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-13T21:54:27.317 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-13T21:54:27.317 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-13T21:54:27.317 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-13T21:54:27.318 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-13T21:54:27.318 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-13T21:54:27.318 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-13T21:54:27.318 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-13T21:54:27.318 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-13T21:54:27.318 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-13T21:54:27.318 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-13T21:54:27.318 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-13T21:54:27.319 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-13T21:54:27.319 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-13T21:54:27.319 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-13T21:54:27.319 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-13T21:54:27.319 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-13T21:54:27.319 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-13T21:54:27.319 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-13T21:54:27.319 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-13T21:54:27.320 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-13T21:54:27.320 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-13T21:54:27.320 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-13T21:54:27.320 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-13T21:54:27.320 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-13T21:54:27.320 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-13T21:54:27.320 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-13T21:54:27.320 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-13T21:54:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-13T21:54:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-13T21:54:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-13T21:54:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-13T21:54:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-13T21:54:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-13T21:54:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-13T21:54:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-13T21:54:27.322 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-13T21:54:27.322 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-13T21:54:27.322 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-13T21:54:27.322 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-13T21:54:27.322 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-13T21:54:27.322 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-13T21:54:27.322 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-13T21:54:27.322 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-13T21:54:27.323 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-13T21:54:27.323 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-13T21:54:27.323 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-13T21:54:27.323 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-13T21:54:27.323 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-13T21:54:27.323 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-13T21:54:27.323 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-13T21:54:27.323 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-13T21:54:27.324 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-13T21:54:27.324 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-13T21:54:27.324 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-13T21:54:27.324 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-13T21:54:27.324 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-13T21:54:27.324 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-13T21:54:27.324 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-13T21:54:27.324 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-13T21:54:27.325 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-13T21:54:27.325 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-13T21:54:27.325 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-13T21:54:27.325 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-13T21:54:27.325 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-13T21:54:27.325 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-13T21:54:27.325 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-13T21:54:27.325 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-13T21:54:27.326 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-13T21:54:27.326 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-13T21:54:27.326 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-13T21:54:27.326 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-13T21:54:27.326 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-13T21:54:27.326 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-13T21:54:27.326 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-13T21:54:27.326 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-13T21:54:27.327 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-13T21:54:27.327 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-13T21:54:27.327 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-13T21:54:27.327 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-13T21:54:27.327 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-13T21:54:27.327 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-13T21:54:27.327 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-13T21:54:27.327 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-13T21:54:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-13T21:54:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-13T21:54:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-13T21:54:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-13T21:54:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-13T21:54:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-13T21:54:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-13T21:54:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-13T21:54:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-13T21:54:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-13T21:54:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-13T21:54:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-13T21:54:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-13T21:54:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-13T21:54:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-13T21:54:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-13T21:54:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-13T21:54:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-13T21:54:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-13T21:54:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-13T21:54:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-13T21:54:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-13T21:54:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-13T21:54:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-13T21:54:27.331 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-13T21:54:27.331 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-13T21:54:27.331 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-13T21:54:27.331 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-13T21:54:27.331 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-13T21:54:27.331 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-13T21:54:27.331 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-13T21:54:27.331 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-13T21:54:27.332 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-13T21:54:27.332 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-13T21:54:27.332 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-13T21:54:27.332 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-13T21:54:27.332 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-13T21:54:27.332 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-13T21:54:27.332 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-13T21:54:27.332 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-13T21:54:27.333 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-13T21:54:27.333 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-13T21:54:27.333 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-13T21:54:27.333 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-13T21:54:27.333 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-13T21:54:27.333 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-13T21:54:27.333 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-13T21:54:27.333 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-13T21:54:27.334 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-13T21:54:27.334 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-13T21:54:27.334 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-13T21:54:27.334 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-13T21:54:27.334 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-13T21:54:27.334 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-13T21:54:27.334 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-13T21:54:27.334 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-13T21:54:27.335 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-13T21:54:27.335 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-13T21:54:27.335 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-13T21:54:27.335 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-13T21:54:27.335 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-13T21:54:27.335 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-13T21:54:27.335 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-13T21:54:27.335 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-13T21:54:27.336 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-13T21:54:27.336 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-13T21:54:27.336 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-13T21:54:27.336 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-13T21:54:27.336 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-13T21:54:27.336 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-13T21:54:27.336 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-13T21:54:27.337 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-13T21:54:27.337 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-13T21:54:27.337 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-13T21:54:27.337 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-13T21:54:27.337 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-13T21:54:27.337 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-13T21:54:27.337 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-13T21:54:27.337 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-13T21:54:27.338 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-13T21:54:27.338 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-13T21:54:27.338 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-13T21:54:27.338 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-13T21:54:27.338 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-13T21:54:27.338 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-13T21:54:27.338 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-13T21:54:27.338 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-13T21:54:27.339 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-13T21:54:27.339 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-13T21:54:27.339 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-13T21:54:27.339 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-13T21:54:27.339 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-13T21:54:27.339 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-13T21:54:27.339 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-13T21:54:27.339 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-13T21:54:27.340 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-13T21:54:27.340 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-13T21:54:27.340 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-13T21:54:27.340 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-13T21:54:27.340 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-13T21:54:27.340 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-13T21:54:27.340 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-13T21:54:27.340 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-13T21:54:27.341 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-13T21:54:27.341 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-13T21:54:27.341 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-13T21:54:27.341 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-13T21:54:27.341 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-13T21:54:27.341 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-13T21:54:27.341 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-13T21:54:27.341 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-13T21:54:27.342 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-13T21:54:27.342 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-13T21:54:27.342 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-13T21:54:27.342 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-13T21:54:27.342 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-13T21:54:27.342 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-13T21:54:27.342 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-13T21:54:27.342 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-13T21:54:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-13T21:54:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-13T21:54:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-13T21:54:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-13T21:54:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-13T21:54:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-13T21:54:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-13T21:54:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-13T21:54:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-13T21:54:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-13T21:54:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-13T21:54:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-13T21:54:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-13T21:54:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-13T21:54:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-13T21:54:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-13T21:54:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-13T21:54:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-13T21:54:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-13T21:54:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-13T21:54:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-13T21:54:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-13T21:54:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-13T21:54:27.346 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-13T21:54:27.346 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-13T21:54:27.346 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-13T21:54:27.346 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-13T21:54:27.346 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-13T21:54:27.346 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-13T21:54:27.346 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-13T21:54:27.346 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-13T21:54:27.347 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-13T21:54:27.347 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-13T21:54:27.347 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-13T21:54:27.347 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-13T21:54:27.347 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-13T21:54:27.347 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-13T21:54:27.347 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-13T21:54:27.347 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-13T21:54:27.348 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-13T21:54:27.348 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-13T21:54:27.348 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-13T21:54:27.348 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-13T21:54:27.348 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-13T21:54:27.348 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-13T21:54:27.348 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-13T21:54:27.348 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-13T21:54:27.349 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-13T21:54:27.349 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-13T21:54:27.349 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-13T21:54:27.349 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-13T21:54:27.349 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-13T21:54:27.349 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-13T21:54:27.349 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-13T21:54:27.349 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-13T21:54:27.350 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-13T21:54:27.350 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-13T21:54:27.350 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-13T21:54:27.350 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-13T21:54:27.350 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-13T21:54:27.350 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-13T21:54:27.350 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-13T21:54:27.350 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-13T21:54:27.351 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-13T21:54:27.351 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-13T21:54:27.351 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-13T21:54:27.351 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-13T21:54:27.351 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-13T21:54:27.351 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-13T21:54:27.351 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-13T21:54:27.351 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-13T21:54:27.352 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-13T21:54:27.352 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-13T21:54:27.352 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-13T21:54:27.352 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-13T21:54:27.352 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-13T21:54:27.352 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-13T21:54:27.352 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-13T21:54:27.352 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-13T21:54:27.353 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-13T21:54:27.353 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-13T21:54:27.353 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-13T21:54:27.353 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-13T21:54:27.353 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-13T21:54:27.353 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-13T21:54:27.353 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-13T21:54:27.353 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-13T21:54:27.354 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-13T21:54:27.354 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-13T21:54:27.354 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-13T21:54:27.354 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-13T21:54:27.354 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-13T21:54:27.354 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-13T21:54:27.354 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-13T21:54:27.354 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-13T21:54:27.355 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-13T21:54:27.355 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-13T21:54:27.355 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-13T21:54:27.355 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-13T21:54:27.355 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-13T21:54:27.355 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-13T21:54:27.355 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-13T21:54:27.355 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-13T21:54:27.356 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-13T21:54:27.356 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-13T21:54:27.356 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-13T21:54:27.356 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-13T21:54:27.356 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-13T21:54:27.356 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-13T21:54:27.356 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-13T21:54:27.356 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-13T21:54:27.357 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-13T21:54:27.357 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-13T21:54:27.357 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-13T21:54:27.357 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-13T21:54:27.357 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-13T21:54:27.357 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-13T21:54:27.357 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-13T21:54:27.357 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-13T21:54:27.358 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-13T21:54:27.358 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-13T21:54:27.358 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-13T21:54:27.358 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-13T21:54:27.358 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-13T21:54:27.358 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-13T21:54:27.358 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-13T21:54:27.358 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-13T21:54:27.359 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-13T21:54:27.359 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-13T21:54:27.359 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-13T21:54:27.359 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-13T21:54:27.359 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-13T21:54:27.359 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-13T21:54:27.359 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-13T21:54:27.359 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-13T21:54:27.360 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-13T21:54:27.360 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-13T21:54:27.360 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-13T21:54:27.360 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-13T21:54:27.360 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-13T21:54:27.360 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-13T21:54:27.360 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-13T21:54:27.361 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-13T21:54:27.361 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-13T21:54:27.361 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-13T21:54:27.361 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-13T21:54:27.361 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-13T21:54:27.361 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-13T21:54:27.361 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-13T21:54:27.361 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-13T21:54:27.362 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-13T21:54:27.362 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-13T21:54:27.362 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-13T21:54:27.362 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-13T21:54:27.362 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-13T21:54:27.362 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-13T21:54:27.362 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-13T21:54:27.362 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-13T21:54:27.363 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-13T21:54:27.363 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-13T21:54:27.363 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-13T21:54:27.363 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-13T21:54:27.363 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-13T21:54:27.363 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-13T21:54:27.363 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-13T21:54:27.363 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-13T21:54:27.364 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-13T21:54:27.364 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-13T21:54:27.364 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-13T21:54:27.364 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-13T21:54:27.364 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-13T21:54:27.364 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-13T21:54:27.364 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-13T21:54:27.364 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-13T21:54:27.365 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-13T21:54:27.365 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-13T21:54:27.365 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-13T21:54:27.365 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-13T21:54:27.365 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-13T21:54:27.365 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-13T21:54:27.365 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-13T21:54:27.365 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-13T21:54:27.366 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-13T21:54:27.366 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-13T21:54:27.366 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-13T21:54:27.366 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-13T21:54:27.366 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-13T21:54:27.366 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-13T21:54:27.366 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-13T21:54:27.366 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-13T21:54:27.367 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-13T21:54:27.367 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-13T21:54:27.367 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-13T21:54:27.367 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-13T21:54:27.367 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-13T21:54:27.367 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-13T21:54:27.367 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-13T21:54:27.367 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-13T21:54:27.368 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-13T21:54:27.368 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-13T21:54:27.368 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-13T21:54:27.368 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-13T21:54:27.368 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-13T21:54:27.368 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-13T21:54:27.368 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-13T21:54:27.368 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-13T21:54:27.369 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-13T21:54:27.369 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-13T21:54:27.369 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-13T21:54:27.369 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-13T21:54:27.369 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-13T21:54:27.369 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-13T21:54:27.369 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-13T21:54:27.369 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-13T21:54:27.370 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-13T21:54:27.370 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-13T21:54:27.370 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-13T21:54:27.370 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-13T21:54:27.370 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-13T21:54:27.370 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-13T21:54:27.370 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-13T21:54:27.370 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-13T21:54:27.371 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-13T21:54:27.371 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-13T21:54:27.371 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-13T21:54:27.371 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-13T21:54:27.371 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-13T21:54:27.371 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-13T21:54:27.371 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-13T21:54:27.372 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-13T21:54:27.372 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-13T21:54:27.372 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-13T21:54:27.372 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-13T21:54:27.372 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-13T21:54:27.372 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-13T21:54:27.372 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-13T21:54:27.372 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-13T21:54:27.373 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-13T21:54:27.373 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-13T21:54:27.373 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-13T21:54:27.373 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-13T21:54:27.373 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-13T21:54:27.373 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-13T21:54:27.373 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-13T21:54:27.373 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-13T21:54:27.374 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-13T21:54:27.374 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-13T21:54:27.374 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-13T21:54:27.374 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-13T21:54:27.374 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-13T21:54:27.374 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-13T21:54:27.374 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-13T21:54:27.374 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-13T21:54:27.375 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-13T21:54:27.375 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-13T21:54:27.375 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-13T21:54:27.375 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-13T21:54:27.375 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-13T21:54:27.375 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-13T21:54:27.375 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-13T21:54:27.375 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-13T21:54:27.376 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-13T21:54:27.376 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-13T21:54:27.376 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-13T21:54:27.376 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-13T21:54:27.376 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-13T21:54:27.376 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-13T21:54:27.376 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-13T21:54:27.377 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-13T21:54:27.377 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-13T21:54:27.377 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-13T21:54:27.377 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-13T21:54:27.377 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-13T21:54:27.377 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-13T21:54:27.377 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-13T21:54:27.377 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-13T21:54:27.378 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-13T21:54:27.378 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-13T21:54:27.378 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-13T21:54:27.378 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-13T21:54:27.378 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-13T21:54:27.378 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-13T21:54:27.378 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-13T21:54:27.378 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-13T21:54:27.379 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-13T21:54:27.379 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-13T21:54:27.379 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-13T21:54:27.379 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-13T21:54:27.379 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-13T21:54:27.379 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-13T21:54:27.379 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-13T21:54:27.379 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-13T21:54:27.380 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-13T21:54:27.380 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-13T21:54:27.380 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-13T21:54:27.380 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-13T21:54:27.380 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-13T21:54:27.380 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-13T21:54:27.380 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-13T21:54:27.380 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-13T21:54:27.381 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-13T21:54:27.381 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-13T21:54:27.381 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-13T21:54:27.381 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-13T21:54:27.381 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-13T21:54:27.381 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-13T21:54:27.381 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-13T21:54:27.381 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-13T21:54:27.382 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-13T21:54:27.382 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-13T21:54:27.382 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-13T21:54:27.382 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-13T21:54:27.382 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-13T21:54:27.382 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-13T21:54:27.382 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-13T21:54:27.382 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-13T21:54:27.383 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-13T21:54:27.383 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-13T21:54:27.383 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-13T21:54:27.383 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-13T21:54:27.383 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-13T21:54:27.383 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-13T21:54:27.383 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-13T21:54:27.383 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-13T21:54:27.384 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-13T21:54:27.384 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-13T21:54:27.384 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-13T21:54:27.384 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-13T21:54:27.384 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-13T21:54:27.384 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-13T21:54:27.384 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-13T21:54:27.384 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-13T21:54:27.385 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-13T21:54:27.385 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-13T21:54:27.385 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-13T21:54:27.385 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-13T21:54:27.385 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-13T21:54:27.385 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-13T21:54:27.385 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-13T21:54:27.385 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-13T21:54:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-13T21:54:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-13T21:54:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-13T21:54:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-13T21:54:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-13T21:54:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-13T21:54:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-13T21:54:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-13T21:54:27.387 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-13T21:54:27.387 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-13T21:54:27.387 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-13T21:54:27.387 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-13T21:54:27.387 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-13T21:54:27.387 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-13T21:54:27.387 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-13T21:54:27.387 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-13T21:54:27.388 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-13T21:54:27.388 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-13T21:54:27.388 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-13T21:54:27.388 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-13T21:54:27.388 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-13T21:54:27.388 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-13T21:54:27.388 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-13T21:54:27.389 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-13T21:54:27.389 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-13T21:54:27.389 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-13T21:54:27.389 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-13T21:54:27.389 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-13T21:54:27.389 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-13T21:54:27.389 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-13T21:54:27.389 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-13T21:54:27.390 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-13T21:54:27.390 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-13T21:54:27.390 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-13T21:54:27.390 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-13T21:54:27.390 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-13T21:54:27.390 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-13T21:54:27.390 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-13T21:54:27.390 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-13T21:54:27.391 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-13T21:54:27.391 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-13T21:54:27.391 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-13T21:54:27.391 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-13T21:54:27.391 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-13T21:54:27.391 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-13T21:54:27.391 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-13T21:54:27.391 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-13T21:54:27.392 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-13T21:54:27.392 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-13T21:54:27.392 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-13T21:54:27.392 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-13T21:54:27.392 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-13T21:54:27.392 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-13T21:54:27.392 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-13T21:54:27.392 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-13T21:54:27.393 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-13T21:54:27.393 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-13T21:54:27.393 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-13T21:54:27.393 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-13T21:54:27.393 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-13T21:54:27.393 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-13T21:54:27.393 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-13T21:54:27.393 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-13T21:54:27.394 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-13T21:54:27.394 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-13T21:54:27.394 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-13T21:54:27.394 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-13T21:54:27.395 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:27.395 DEBUG:teuthology.orchestra.run.smithi060:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-13T21:54:27.423 INFO:teuthology.orchestra.run.smithi060.stdout:postinstall scriptlet (using /bin/sh): 2023-07-13T21:54:27.423 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-13T21:54:27.424 INFO:teuthology.orchestra.run.smithi060.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:54:27.424 INFO:teuthology.orchestra.run.smithi060.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:54:27.424 INFO:teuthology.orchestra.run.smithi060.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:27.424 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:54:27.424 INFO:teuthology.orchestra.run.smithi060.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:54:27.425 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2023-07-13T21:54:27.425 INFO:teuthology.orchestra.run.smithi060.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-13T21:54:27.425 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-13T21:54:27.425 INFO:teuthology.orchestra.run.smithi060.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-13T21:54:27.425 INFO:teuthology.orchestra.run.smithi060.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-13T21:54:27.425 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:27.425 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2023-07-13T21:54:27.425 INFO:teuthology.orchestra.run.smithi060.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-13T21:54:27.426 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-13T21:54:27.426 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:27.426 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2023-07-13T21:54:27.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2023-07-13T21:54:27.511 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2023-07-13T21:54:27.511 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2023-07-13T21:54:27.511 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.02 2023-07-13T21:54:27.511 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 129.el8 2023-07-13T21:54:27.512 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2023-07-13T21:54:27.512 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-07-13T21:54:27.512 INFO:teuthology.orchestra.run.smithi060.stdout:Group : System Environment/Base 2023-07-13T21:54:27.512 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 9265586 2023-07-13T21:54:27.512 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2023-07-13T21:54:27.512 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-07-13T21:54:27.513 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-07-13T21:54:27.513 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-07-13T21:54:27.513 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : kbuilder.bsys.centos.org 2023-07-13T21:54:27.513 INFO:teuthology.orchestra.run.smithi060.stdout:Relocations : (not relocatable) 2023-07-13T21:54:27.513 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : CentOS BuildSystem 2023-07-13T21:54:27.513 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2023-07-13T21:54:27.513 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-13T21:54:27.514 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2023-07-13T21:54:27.514 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2023-07-13T21:54:27.514 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T21:54:27.514 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-13T21:54:27.515 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-13T21:54:27.515 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-13T21:54:27.516 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2023-07-13T21:54:27.516 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T21:54:27.517 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2023-07-13T21:54:27.519 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-13T21:54:27.519 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:27.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-13T21:54:28.297 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2023-07-13T21:54:30.256 INFO:teuthology.orchestra.run.smithi105.stderr:done 2023-07-13T21:54:30.257 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2023-07-13T21:54:30.281 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.AJCEvgNhRs 2023-07-13T21:54:30.282 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AJCEvgNhRs 2023-07-13T21:54:30.356 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.AJCEvgNhRs 2023-07-13T21:54:30.516 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.AJCEvgNhRs is 6KB 2023-07-13T21:54:30.564 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.AJCEvgNhRs 2023-07-13T21:54:30.582 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2023-07-13T21:54:30.663 INFO:teuthology.orchestra.run.smithi105.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-07-13T21:54:30.663 INFO:teuthology.orchestra.run.smithi105.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-07-13T21:54:30.664 INFO:teuthology.orchestra.run.smithi105.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-07-13T21:54:30.664 INFO:teuthology.orchestra.run.smithi105.stdout:a1b29d6c289747bdb22930598d39874a-6.5.0-rc1-ga440c79eee02.conf 2023-07-13T21:54:30.665 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:30.829 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2023-07-13T21:54:32.741 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2023-07-13T21:54:34.849 INFO:teuthology.orchestra.run.smithi060.stderr:done 2023-07-13T21:54:34.855 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2023-07-13T21:54:34.893 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.Y4pkUanGck 2023-07-13T21:54:34.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Y4pkUanGck 2023-07-13T21:54:34.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.Y4pkUanGck 2023-07-13T21:54:35.143 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.Y4pkUanGck is 6KB 2023-07-13T21:54:35.193 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.Y4pkUanGck 2023-07-13T21:54:35.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2023-07-13T21:54:35.278 INFO:teuthology.orchestra.run.smithi060.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-07-13T21:54:35.278 INFO:teuthology.orchestra.run.smithi060.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-07-13T21:54:35.279 INFO:teuthology.orchestra.run.smithi060.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-07-13T21:54:35.279 INFO:teuthology.orchestra.run.smithi060.stdout:a1b29d6c289747bdb22930598d39874a-6.5.0-rc1-ga440c79eee02.conf 2023-07-13T21:54:35.280 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.5.0-rc1-ga440c79eee02 2023-07-13T21:54:35.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2023-07-13T21:55:00.856 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-07-13T21:55:00.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-07-13T21:55:00.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:55:05.436 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-07-13T21:55:05.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-07-13T21:55:05.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:56:00.920 DEBUG:teuthology.orchestra.remote:timed out 2023-07-13T21:56:05.440 DEBUG:teuthology.orchestra.remote:timed out 2023-07-13T21:56:09.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-07-13T21:56:09.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:56:11.363 DEBUG:teuthology.orchestra.run.smithi105:> true 2023-07-13T21:56:11.898 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-07-13T21:56:11.899 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc1-ga440c79eee02"... 2023-07-13T21:56:11.899 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2023-07-13T21:56:11.943 INFO:teuthology.orchestra.run.smithi105.stdout:6.5.0-rc1-ga440c79eee02 2023-07-13T21:56:11.944 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-13T21:56:11.944 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-13T21:56:11.944 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-07-13T21:56:12.946 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-07-13T21:56:12.947 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-13T21:56:13.080 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2023-07-13T21:56:13.103 DEBUG:teuthology.parallel:result is None 2023-07-13T21:56:14.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-07-13T21:56:14.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:56:14.749 DEBUG:teuthology.orchestra.run.smithi060:> true 2023-07-13T21:56:15.185 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-07-13T21:56:15.185 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc1-ga440c79eee02"... 2023-07-13T21:56:15.185 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2023-07-13T21:56:15.231 INFO:teuthology.orchestra.run.smithi060.stdout:6.5.0-rc1-ga440c79eee02 2023-07-13T21:56:15.232 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-13T21:56:15.232 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-13T21:56:15.233 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-07-13T21:56:16.234 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-07-13T21:56:16.235 DEBUG:teuthology.orchestra.run.smithi060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-13T21:56:16.348 INFO:teuthology.orchestra.run.smithi060.stdout:ttyS1 2023-07-13T21:56:16.370 DEBUG:teuthology.parallel:result is None 2023-07-13T21:56:16.371 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-13T21:56:16.379 INFO:teuthology.task.internal:Creating test directory... 2023-07-13T21:56:16.379 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-13T21:56:16.384 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-13T21:56:16.435 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-13T21:56:16.443 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-13T21:56:16.518 INFO:teuthology.task.internal:Creating archive directory... 2023-07-13T21:56:16.518 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-13T21:56:16.524 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-13T21:56:16.590 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-13T21:56:16.604 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-13T21:56:16.606 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-13T21:56:16.616 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-13T21:56:16.704 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:56:16.712 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:56:16.731 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:56:16.739 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:56:16.742 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-13T21:56:16.751 INFO:teuthology.task.internal:Configuring sudo... 2023-07-13T21:56:16.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-13T21:56:16.760 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-13T21:56:16.830 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-13T21:56:16.849 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-13T21:56:16.850 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-13T21:56:16.856 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-13T21:56:16.902 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:56:17.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:56:17.098 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:56:17.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:56:17.247 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T21:56:17.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-13T21:56:17.319 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:56:17.402 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:56:17.472 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:56:17.553 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:56:17.624 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T21:56:17.625 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-13T21:56:17.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2023-07-13T21:56:17.700 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2023-07-13T21:56:17.819 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T21:56:17.871 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T21:56:18.301 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-13T21:56:18.312 INFO:teuthology.task.internal:Starting timer... 2023-07-13T21:56:18.312 INFO:teuthology.run_tasks:Running task pcp... 2023-07-13T21:56:18.331 INFO:teuthology.run_tasks:Running task selinux... 2023-07-13T21:56:18.352 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-13T21:56:18.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2023-07-13T21:56:18.501 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2023-07-13T21:56:18.511 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2023-07-13T21:56:18.638 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2023-07-13T21:56:18.645 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-13T21:56:18.645 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2023-07-13T21:56:18.704 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2023-07-13T21:56:18.705 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2023-07-13T21:56:18.761 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2023-07-13T21:56:18.761 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi105.front.sepia.ceph.com': 'permissive'} 2023-07-13T21:56:18.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T21:56:18.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:56:18.834 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T21:56:18.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:56:18.905 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-13T21:56:18.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2023-07-13T21:56:18.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2023-07-13T21:56:19.055 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-13T21:56:19.067 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-13T21:56:19.080 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-13T21:56:19.261 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-13T21:56:19.307 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-13T21:56:19.309 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi105.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-13T22:08:20.041 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi105.front.sepia.ceph.com')] 2023-07-13T22:08:20.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-07-13T22:08:20.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T22:08:20.129 DEBUG:teuthology.orchestra.run.smithi060:> true 2023-07-13T22:08:20.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-07-13T22:08:20.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-07-13T22:08:20.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T22:08:20.336 DEBUG:teuthology.orchestra.run.smithi105:> true 2023-07-13T22:08:20.451 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-07-13T22:08:20.451 INFO:teuthology.run_tasks:Running task clock... 2023-07-13T22:08:20.464 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-13T22:08:20.465 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-13T22:08:20.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:08:20.470 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-13T22:08:20.470 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:08:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-13T22:08:20.546 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-13T22:08:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-13T22:08:20.579 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-13T22:08:20.668 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2023-07-13T22:08:20.672 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2023-07-13T22:08:20.697 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2023-07-13T22:08:20.699 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2023-07-13T22:08:20.731 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-13T22:08:20.732 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-13T22:08:20.765 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-13T22:08:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-13T22:08:20.849 INFO:teuthology.orchestra.run.smithi105.stderr:bash: ntpq: command not found 2023-07-13T22:08:20.857 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:08:20.857 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2023-07-13T22:08:20.857 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:08:20.857 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:08:20.857 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:08:20.857 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:08:20.867 INFO:teuthology.orchestra.run.smithi060.stderr:bash: ntpq: command not found 2023-07-13T22:08:20.874 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:08:20.875 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-07-13T22:08:20.875 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:08:20.875 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:08:20.875 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:08:20.875 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:08:20.876 INFO:teuthology.run_tasks:Running task install... 2023-07-13T22:08:20.885 DEBUG:teuthology.task.install:project ceph 2023-07-13T22:08:20.885 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}} 2023-07-13T22:08:20.885 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'} 2023-07-13T22:08:20.886 INFO:teuthology.task.install:Using flavor: default 2023-07-13T22:08:20.892 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-13T22:08:20.893 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-13T22:08:20.893 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-13T22:08:20.894 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:08:20.896 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-13T22:08:20.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:08:21.599 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-13T22:08:21.599 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-13T22:08:22.096 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-13T22:08:22.096 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:08:22.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-13T22:08:22.174 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-13T22:08:22.175 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-13T22:08:22.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-13T22:08:22.387 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-13T22:08:22.457 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2023-07-13T22:08:22.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2023-07-13T22:08:22.475 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-13T22:08:22.476 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-13T22:08:22.830 INFO:teuthology.orchestra.run.smithi060.stdout:73 files removed 2023-07-13T22:08:22.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-13T22:08:22.939 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-13T22:08:22.939 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:08:22.939 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-13T22:08:23.016 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-13T22:08:23.017 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-13T22:08:23.131 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-13T22:08:23.226 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-13T22:08:23.295 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2023-07-13T22:08:23.298 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2023-07-13T22:08:23.646 INFO:teuthology.orchestra.run.smithi105.stdout:73 files removed 2023-07-13T22:08:23.680 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-13T22:08:24.891 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 56 kB/s | 81 kB 00:01 2023-07-13T22:08:25.510 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 63 kB/s | 81 kB 00:01 2023-07-13T22:08:25.824 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 11 kB/s | 9.4 kB 00:00 2023-07-13T22:08:26.477 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 10 kB/s | 9.4 kB 00:00 2023-07-13T22:08:26.833 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 1.5 kB/s | 1.5 kB 00:00 2023-07-13T22:08:27.544 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 1.4 kB/s | 1.5 kB 00:01 2023-07-13T22:09:32.954 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 8 - AppStream 481 kB/s | 31 MB 01:05 2023-07-13T22:09:43.704 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 8 - BaseOS 8.5 MB/s | 41 MB 00:04 2023-07-13T22:09:50.804 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 8 - Extras 21 kB/s | 18 kB 00:00 2023-07-13T22:09:51.813 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 8 - Extras common packages 7.9 kB/s | 6.6 kB 00:00 2023-07-13T22:09:53.931 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 8 - PowerTools 3.2 MB/s | 6.3 MB 00:01 2023-07-13T22:09:58.157 INFO:teuthology.orchestra.run.smithi105.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-13T22:10:24.261 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 8 - AppStream 268 kB/s | 31 MB 01:57 2023-07-13T22:10:46.247 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 333 kB/s | 16 MB 00:47 2023-07-13T22:10:48.852 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 8 - BaseOS 2.3 MB/s | 41 MB 00:17 2023-07-13T22:10:50.561 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 488 kB/s | 24 kB 00:00 2023-07-13T22:10:54.516 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:10:54.517 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:10:54.530 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-07-13T22:10:54.530 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-07-13T22:10:54.531 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-07-13T22:10:54.532 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-07-13T22:10:54.533 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-13T22:10:54.535 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-13T22:10:54.538 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-13T22:10:54.539 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-13T22:10:54.726 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-07-13T22:10:54.740 INFO:teuthology.orchestra.run.smithi105.stdout:==================================================================================================== 2023-07-13T22:10:54.740 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-07-13T22:10:54.740 INFO:teuthology.orchestra.run.smithi105.stdout:==================================================================================================== 2023-07-13T22:10:54.741 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-07-13T22:10:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-07-13T22:10:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-13T22:10:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-13T22:10:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-13T22:10:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-13T22:10:54.741 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-13T22:10:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-13T22:10:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-13T22:10:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-13T22:10:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-13T22:10:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-13T22:10:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-13T22:10:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-13T22:10:54.742 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-13T22:10:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-13T22:10:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-13T22:10:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.189-2.el8 baseos 61 k 2023-07-13T22:10:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-07-13T22:10:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-07-13T22:10:54.743 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-13T22:10:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-13T22:10:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-13T22:10:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-07-13T22:10:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.32.1-42.el8 baseos 99 k 2023-07-13T22:10:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-07-13T22:10:54.744 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c appstream 1.6 M 2023-07-13T22:10:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c appstream 903 k 2023-07-13T22:10:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c appstream 5.1 M 2023-07-13T22:10:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-13T22:10:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-13T22:10:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-13T22:10:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-13T22:10:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-13T22:10:54.745 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-13T22:10:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-13T22:10:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-07-13T22:10:54.746 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2023-07-13T22:10:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-8.el8 baseos 143 k 2023-07-13T22:10:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.189-2.el8 baseos 76 k 2023-07-13T22:10:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.189-2.el8 baseos 232 k 2023-07-13T22:10:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.189-2.el8 baseos 302 k 2023-07-13T22:10:54.746 INFO:teuthology.orchestra.run.smithi105.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-07-13T22:10:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid x86_64 2.32.1-42.el8 baseos 220 k 2023-07-13T22:10:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk x86_64 2.32.1-42.el8 baseos 253 k 2023-07-13T22:10:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: libmount x86_64 2.32.1-42.el8 baseos 236 k 2023-07-13T22:10:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-13T22:10:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-13T22:10:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols x86_64 2.32.1-42.el8 baseos 179 k 2023-07-13T22:10:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid x86_64 2.32.1-42.el8 baseos 98 k 2023-07-13T22:10:54.747 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux x86_64 2.32.1-42.el8 baseos 2.6 M 2023-07-13T22:10:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user x86_64 2.32.1-42.el8 baseos 102 k 2023-07-13T22:10:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-07-13T22:10:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-07-13T22:10:54.748 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-07-13T22:10:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.1-7.el8 appstream 713 k 2023-07-13T22:10:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-13T22:10:54.748 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-13T22:10:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-13T22:10:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-13T22:10:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-13T22:10:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-13T22:10:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-13T22:10:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-13T22:10:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-07-13T22:10:54.749 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-13T22:10:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-07-13T22:10:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-07-13T22:10:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-07-13T22:10:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-13T22:10:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-07-13T22:10:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-07-13T22:10:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-13T22:10:54.750 INFO:teuthology.orchestra.run.smithi105.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-07-13T22:10:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-07-13T22:10:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-07-13T22:10:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-13T22:10:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-07-13T22:10:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-13T22:10:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-07-13T22:10:54.751 INFO:teuthology.orchestra.run.smithi105.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-07-13T22:10:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-13T22:10:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-07-13T22:10:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-07-13T22:10:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-07-13T22:10:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-07-13T22:10:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-07-13T22:10:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-07-13T22:10:54.752 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-07-13T22:10:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-07-13T22:10:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-07-13T22:10:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-13T22:10:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-07-13T22:10:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-13T22:10:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-13T22:10:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-13T22:10:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-13T22:10:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-13T22:10:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-13T22:10:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-07-13T22:10:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-13T22:10:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-13T22:10:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-07-13T22:10:54.754 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-13T22:10:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-13T22:10:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-13T22:10:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-07-13T22:10:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-07-13T22:10:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-07-13T22:10:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-07-13T22:10:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-13T22:10:54.755 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-13T22:10:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-07-13T22:10:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-07-13T22:10:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-13T22:10:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-13T22:10:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-07-13T22:10:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-07-13T22:10:54.756 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-07-13T22:10:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-13T22:10:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-13T22:10:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-07-13T22:10:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-07-13T22:10:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-07-13T22:10:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-07-13T22:10:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-07-13T22:10:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2017.2-9.el8 appstream 54 k 2023-07-13T22:10:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-13T22:10:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-07-13T22:10:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-13T22:10:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-13T22:10:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-07-13T22:10:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-13T22:10:54.758 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-13T22:10:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-13T22:10:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-13T22:10:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-13T22:10:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-13T22:10:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-13T22:10:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-13T22:10:54.759 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-07-13T22:10:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-13T22:10:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-07-13T22:10:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-13T22:10:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-07-13T22:10:54.760 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-07-13T22:10:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-07-13T22:10:54.761 INFO:teuthology.orchestra.run.smithi105.stdout:Enabling module streams: 2023-07-13T22:10:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql 10 2023-07-13T22:10:54.761 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:10:54.761 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-07-13T22:10:54.761 INFO:teuthology.orchestra.run.smithi105.stdout:==================================================================================================== 2023-07-13T22:10:54.761 INFO:teuthology.orchestra.run.smithi105.stdout:Install 128 Packages 2023-07-13T22:10:54.762 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 16 Packages 2023-07-13T22:10:54.762 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:10:54.762 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 212 M 2023-07-13T22:10:54.762 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-07-13T22:10:56.005 INFO:teuthology.orchestra.run.smithi105.stdout:(1/144): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 11 kB/s | 6.4 kB 00:00 2023-07-13T22:10:56.360 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 8 - Extras 26 kB/s | 18 kB 00:00 2023-07-13T22:10:56.944 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 8 - Extras common packages 16 kB/s | 6.6 kB 00:00 2023-07-13T22:10:57.696 INFO:teuthology.orchestra.run.smithi105.stdout:(2/144): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 502 kB/s | 848 kB 00:01 2023-07-13T22:10:57.863 INFO:teuthology.orchestra.run.smithi105.stdout:(3/144): ceph-immutable-object-cache-18.0.0-475 995 kB/s | 165 kB 00:00 2023-07-13T22:10:58.321 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 8 - PowerTools 5.1 MB/s | 6.3 MB 00:01 2023-07-13T22:11:04.122 INFO:teuthology.orchestra.run.smithi060.stdout:Copr repo for python3-asyncssh owned by ceph 818 B/s | 3.5 kB 00:04 2023-07-13T22:11:04.913 INFO:teuthology.orchestra.run.smithi105.stdout:(4/144): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 333 kB/s | 2.3 MB 00:07 2023-07-13T22:11:06.104 INFO:teuthology.orchestra.run.smithi105.stdout:(5/144): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 516 kB/s | 5.4 MB 00:10 2023-07-13T22:11:07.763 INFO:teuthology.orchestra.run.smithi105.stdout:(6/144): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 556 kB/s | 1.5 MB 00:02 2023-07-13T22:11:11.496 INFO:teuthology.orchestra.run.smithi105.stdout:(7/144): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 878 kB/s | 4.6 MB 00:05 2023-07-13T22:11:25.128 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 767 kB/s | 16 MB 00:20 2023-07-13T22:11:29.853 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 377 kB/s | 24 kB 00:00 2023-07-13T22:11:34.053 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:11:34.054 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:11:34.068 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-07-13T22:11:34.068 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-07-13T22:11:34.069 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-07-13T22:11:34.070 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-07-13T22:11:34.071 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-13T22:11:34.073 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-13T22:11:34.077 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-13T22:11:34.078 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-13T22:11:34.286 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-07-13T22:11:34.301 INFO:teuthology.orchestra.run.smithi060.stdout:==================================================================================================== 2023-07-13T22:11:34.301 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-07-13T22:11:34.301 INFO:teuthology.orchestra.run.smithi060.stdout:==================================================================================================== 2023-07-13T22:11:34.302 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-07-13T22:11:34.302 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-07-13T22:11:34.302 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-13T22:11:34.302 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-13T22:11:34.302 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-13T22:11:34.303 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-13T22:11:34.303 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-13T22:11:34.303 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-13T22:11:34.303 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-13T22:11:34.303 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-13T22:11:34.303 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-13T22:11:34.303 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-13T22:11:34.304 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-13T22:11:34.304 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-13T22:11:34.304 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-13T22:11:34.304 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-13T22:11:34.304 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-13T22:11:34.304 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.189-2.el8 baseos 61 k 2023-07-13T22:11:34.304 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-07-13T22:11:34.305 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-07-13T22:11:34.305 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-13T22:11:34.305 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-13T22:11:34.305 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-13T22:11:34.305 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-07-13T22:11:34.305 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.32.1-42.el8 baseos 99 k 2023-07-13T22:11:34.306 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-07-13T22:11:34.306 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c appstream 1.6 M 2023-07-13T22:11:34.306 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c appstream 903 k 2023-07-13T22:11:34.306 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c appstream 5.1 M 2023-07-13T22:11:34.306 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-13T22:11:34.306 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-13T22:11:34.306 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-13T22:11:34.307 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-13T22:11:34.307 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-13T22:11:34.307 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-13T22:11:34.307 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-13T22:11:34.307 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-07-13T22:11:34.307 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2023-07-13T22:11:34.307 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-8.el8 baseos 143 k 2023-07-13T22:11:34.308 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client x86_64 0.189-2.el8 baseos 76 k 2023-07-13T22:11:34.308 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf x86_64 0.189-2.el8 baseos 232 k 2023-07-13T22:11:34.308 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs x86_64 0.189-2.el8 baseos 302 k 2023-07-13T22:11:34.308 INFO:teuthology.orchestra.run.smithi060.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-07-13T22:11:34.308 INFO:teuthology.orchestra.run.smithi060.stdout: libblkid x86_64 2.32.1-42.el8 baseos 220 k 2023-07-13T22:11:34.308 INFO:teuthology.orchestra.run.smithi060.stdout: libfdisk x86_64 2.32.1-42.el8 baseos 253 k 2023-07-13T22:11:34.308 INFO:teuthology.orchestra.run.smithi060.stdout: libmount x86_64 2.32.1-42.el8 baseos 236 k 2023-07-13T22:11:34.309 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-13T22:11:34.309 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-13T22:11:34.309 INFO:teuthology.orchestra.run.smithi060.stdout: libsmartcols x86_64 2.32.1-42.el8 baseos 179 k 2023-07-13T22:11:34.309 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid x86_64 2.32.1-42.el8 baseos 98 k 2023-07-13T22:11:34.309 INFO:teuthology.orchestra.run.smithi060.stdout: util-linux x86_64 2.32.1-42.el8 baseos 2.6 M 2023-07-13T22:11:34.309 INFO:teuthology.orchestra.run.smithi060.stdout: util-linux-user x86_64 2.32.1-42.el8 baseos 102 k 2023-07-13T22:11:34.310 INFO:teuthology.orchestra.run.smithi060.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-07-13T22:11:34.310 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-07-13T22:11:34.310 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-07-13T22:11:34.310 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.1-7.el8 appstream 713 k 2023-07-13T22:11:34.310 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-13T22:11:34.310 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-13T22:11:34.310 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-13T22:11:34.311 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-13T22:11:34.311 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-13T22:11:34.311 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-13T22:11:34.311 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-13T22:11:34.311 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-13T22:11:34.311 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-07-13T22:11:34.311 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-13T22:11:34.312 INFO:teuthology.orchestra.run.smithi060.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-07-13T22:11:34.312 INFO:teuthology.orchestra.run.smithi060.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-07-13T22:11:34.312 INFO:teuthology.orchestra.run.smithi060.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-07-13T22:11:34.312 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-13T22:11:34.312 INFO:teuthology.orchestra.run.smithi060.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-07-13T22:11:34.313 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-07-13T22:11:34.313 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-13T22:11:34.313 INFO:teuthology.orchestra.run.smithi060.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-07-13T22:11:34.313 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-07-13T22:11:34.313 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-07-13T22:11:34.313 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-13T22:11:34.314 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-07-13T22:11:34.314 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-13T22:11:34.314 INFO:teuthology.orchestra.run.smithi060.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-07-13T22:11:34.314 INFO:teuthology.orchestra.run.smithi060.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-07-13T22:11:34.315 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-13T22:11:34.315 INFO:teuthology.orchestra.run.smithi060.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-07-13T22:11:34.315 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-07-13T22:11:34.315 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-07-13T22:11:34.315 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-07-13T22:11:34.315 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-07-13T22:11:34.316 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-07-13T22:11:34.316 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-07-13T22:11:34.316 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-07-13T22:11:34.316 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-07-13T22:11:34.316 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-13T22:11:34.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-07-13T22:11:34.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-13T22:11:34.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-13T22:11:34.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-13T22:11:34.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-13T22:11:34.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-13T22:11:34.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-13T22:11:34.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-07-13T22:11:34.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-13T22:11:34.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-13T22:11:34.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-07-13T22:11:34.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-13T22:11:34.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-13T22:11:34.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-13T22:11:34.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-07-13T22:11:34.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-07-13T22:11:34.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-07-13T22:11:34.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-07-13T22:11:34.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-13T22:11:34.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-13T22:11:34.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-07-13T22:11:34.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-07-13T22:11:34.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-13T22:11:34.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-13T22:11:34.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-07-13T22:11:34.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-07-13T22:11:34.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-07-13T22:11:34.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-13T22:11:34.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-13T22:11:34.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-07-13T22:11:34.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-07-13T22:11:34.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-07-13T22:11:34.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-07-13T22:11:34.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-07-13T22:11:34.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2017.2-9.el8 appstream 54 k 2023-07-13T22:11:34.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-13T22:11:34.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-07-13T22:11:34.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-13T22:11:34.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-13T22:11:34.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-07-13T22:11:34.323 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-13T22:11:34.323 INFO:teuthology.orchestra.run.smithi060.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-13T22:11:34.323 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-13T22:11:34.323 INFO:teuthology.orchestra.run.smithi060.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-13T22:11:34.323 INFO:teuthology.orchestra.run.smithi060.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-13T22:11:34.323 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-13T22:11:34.324 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-13T22:11:34.324 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-13T22:11:34.324 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-07-13T22:11:34.324 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-13T22:11:34.324 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-07-13T22:11:34.324 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-13T22:11:34.325 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-07-13T22:11:34.325 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-07-13T22:11:34.325 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-07-13T22:11:34.325 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling module streams: 2023-07-13T22:11:34.325 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql 10 2023-07-13T22:11:34.325 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:11:34.325 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-07-13T22:11:34.326 INFO:teuthology.orchestra.run.smithi060.stdout:==================================================================================================== 2023-07-13T22:11:34.326 INFO:teuthology.orchestra.run.smithi060.stdout:Install 128 Packages 2023-07-13T22:11:34.326 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 16 Packages 2023-07-13T22:11:34.326 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:11:34.326 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 212 M 2023-07-13T22:11:34.327 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-07-13T22:11:35.371 INFO:teuthology.orchestra.run.smithi060.stdout:(1/144): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2023-07-13T22:11:35.510 INFO:teuthology.orchestra.run.smithi105.stdout:(8/144): ceph-radosgw-18.0.0-4757.g3fc0089c.el8 575 kB/s | 13 MB 00:24 2023-07-13T22:11:35.602 INFO:teuthology.orchestra.run.smithi105.stdout:(9/144): ceph-selinux-18.0.0-4757.g3fc0089c.el8 268 kB/s | 24 kB 00:00 2023-07-13T22:11:37.321 INFO:teuthology.orchestra.run.smithi060.stdout:(2/144): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 435 kB/s | 848 kB 00:01 2023-07-13T22:11:37.605 INFO:teuthology.orchestra.run.smithi060.stdout:(3/144): ceph-immutable-object-cache-18.0.0-475 584 kB/s | 165 kB 00:00 2023-07-13T22:11:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:(4/144): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 1.0 MB/s | 2.3 MB 00:02 2023-07-13T22:11:41.788 INFO:teuthology.orchestra.run.smithi060.stdout:(5/144): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 817 kB/s | 1.5 MB 00:01 2023-07-13T22:11:42.793 INFO:teuthology.orchestra.run.smithi105.stdout:(10/144): ceph-common-18.0.0-4757.g3fc0089c.el8 477 kB/s | 22 MB 00:47 2023-07-13T22:11:42.877 INFO:teuthology.orchestra.run.smithi105.stdout:(11/144): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 517 kB/s | 18 MB 00:35 2023-07-13T22:11:42.977 INFO:teuthology.orchestra.run.smithi105.stdout:(12/144): libcephfs-devel-18.0.0-4757.g3fc0089c 312 kB/s | 31 kB 00:00 2023-07-13T22:11:43.444 INFO:teuthology.orchestra.run.smithi105.stdout:(13/144): cephfs-mirror-18.0.0-4757.g3fc0089c.e 397 kB/s | 258 kB 00:00 2023-07-13T22:11:43.802 INFO:teuthology.orchestra.run.smithi105.stdout:(14/144): libcephsqlite-18.0.0-4757.g3fc0089c.e 383 kB/s | 137 kB 00:00 2023-07-13T22:11:44.244 INFO:teuthology.orchestra.run.smithi105.stdout:(15/144): librados-devel-18.0.0-4757.g3fc0089c. 315 kB/s | 139 kB 00:00 2023-07-13T22:11:45.336 INFO:teuthology.orchestra.run.smithi105.stdout:(16/144): libcephfs2-18.0.0-4757.g3fc0089c.el8. 327 kB/s | 771 kB 00:02 2023-07-13T22:11:45.420 INFO:teuthology.orchestra.run.smithi105.stdout:(17/144): libradosstriper1-18.0.0-4757.g3fc0089 402 kB/s | 473 kB 00:01 2023-07-13T22:11:45.537 INFO:teuthology.orchestra.run.smithi105.stdout:(18/144): python3-ceph-argparse-18.0.0-4757.g3f 384 kB/s | 45 kB 00:00 2023-07-13T22:11:45.962 INFO:teuthology.orchestra.run.smithi105.stdout:(19/144): python3-ceph-common-18.0.0-4757.g3fc0 279 kB/s | 119 kB 00:00 2023-07-13T22:11:46.987 INFO:teuthology.orchestra.run.smithi105.stdout:(20/144): python3-cephfs-18.0.0-4757.g3fc0089c. 192 kB/s | 197 kB 00:01 2023-07-13T22:11:48.154 INFO:teuthology.orchestra.run.smithi105.stdout:(21/144): python3-rados-18.0.0-4757.g3fc0089c.e 318 kB/s | 371 kB 00:01 2023-07-13T22:11:48.529 INFO:teuthology.orchestra.run.smithi060.stdout:(6/144): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 408 kB/s | 5.4 MB 00:13 2023-07-13T22:11:49.363 INFO:teuthology.orchestra.run.smithi105.stdout:(22/144): python3-rbd-18.0.0-4757.g3fc0089c.el8 302 kB/s | 364 kB 00:01 2023-07-13T22:11:49.846 INFO:teuthology.orchestra.run.smithi105.stdout:(23/144): python3-rgw-18.0.0-4757.g3fc0089c.el8 221 kB/s | 107 kB 00:00 2023-07-13T22:11:50.588 INFO:teuthology.orchestra.run.smithi105.stdout:(24/144): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 127 kB/s | 94 kB 00:00 2023-07-13T22:11:52.621 INFO:teuthology.orchestra.run.smithi060.stdout:(7/144): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 437 kB/s | 4.6 MB 00:10 2023-07-13T22:11:57.704 INFO:teuthology.orchestra.run.smithi105.stdout:(25/144): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 496 kB/s | 3.4 MB 00:07 2023-07-13T22:11:58.038 INFO:teuthology.orchestra.run.smithi105.stdout:(26/144): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 539 kB/s | 179 kB 00:00 2023-07-13T22:11:58.177 INFO:teuthology.orchestra.run.smithi105.stdout:(27/144): ceph-grafana-dashboards-18.0.0-4757.g 166 kB/s | 23 kB 00:00 2023-07-13T22:11:58.455 INFO:teuthology.orchestra.run.smithi105.stdout:(28/144): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 464 kB/s | 129 kB 00:00 2023-07-13T22:11:58.870 INFO:teuthology.orchestra.run.smithi060.stdout:(8/144): ceph-common-18.0.0-4757.g3fc0089c.el8. 948 kB/s | 22 MB 00:23 2023-07-13T22:11:58.962 INFO:teuthology.orchestra.run.smithi060.stdout:(9/144): ceph-selinux-18.0.0-4757.g3fc0089c.el8 267 kB/s | 24 kB 00:00 2023-07-13T22:12:00.031 INFO:teuthology.orchestra.run.smithi105.stdout:(29/144): ceph-mgr-dashboard-18.0.0-4757.g3fc00 1.1 MB/s | 1.7 MB 00:01 2023-07-13T22:12:00.389 INFO:teuthology.orchestra.run.smithi105.stdout:(30/144): librgw2-18.0.0-4757.g3fc0089c.el8.x86 459 kB/s | 6.7 MB 00:15 2023-07-13T22:12:00.839 INFO:teuthology.orchestra.run.smithi105.stdout:(31/144): ceph-mgr-modules-core-18.0.0-4757.g3f 535 kB/s | 240 kB 00:00 2023-07-13T22:12:01.040 INFO:teuthology.orchestra.run.smithi105.stdout:(32/144): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 237 kB/s | 47 kB 00:00 2023-07-13T22:12:01.140 INFO:teuthology.orchestra.run.smithi105.stdout:(33/144): ceph-prometheus-alerts-18.0.0-4757.g3 145 kB/s | 15 kB 00:00 2023-07-13T22:12:01.632 INFO:teuthology.orchestra.run.smithi105.stdout:(34/144): ceph-volume-18.0.0-4757.g3fc0089c.el8 526 kB/s | 258 kB 00:00 2023-07-13T22:12:01.941 INFO:teuthology.orchestra.run.smithi105.stdout:(35/144): cephadm-18.0.0-4757.g3fc0089c.el8.noa 540 kB/s | 166 kB 00:00 2023-07-13T22:12:02.041 INFO:teuthology.orchestra.run.smithi105.stdout:(36/144): cephfs-top-18.0.0-4757.g3fc0089c.el8. 180 kB/s | 18 kB 00:00 2023-07-13T22:12:17.265 INFO:teuthology.orchestra.run.smithi105.stdout:(37/144): ceph-mgr-diskprediction-local-18.0.0- 440 kB/s | 7.4 MB 00:17 2023-07-13T22:12:22.801 INFO:teuthology.orchestra.run.smithi060.stdout:(10/144): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 529 kB/s | 18 MB 00:34 2023-07-13T22:12:22.952 INFO:teuthology.orchestra.run.smithi060.stdout:(11/144): ceph-radosgw-18.0.0-4757.g3fc0089c.el 455 kB/s | 13 MB 00:30 2023-07-13T22:12:23.218 INFO:teuthology.orchestra.run.smithi060.stdout:(12/144): cephfs-mirror-18.0.0-4757.g3fc0089c.e 619 kB/s | 258 kB 00:00 2023-07-13T22:12:23.244 INFO:teuthology.orchestra.run.smithi060.stdout:(13/144): libcephfs-devel-18.0.0-4757.g3fc0089c 106 kB/s | 31 kB 00:00 2023-07-13T22:12:23.502 INFO:teuthology.orchestra.run.smithi060.stdout:(14/144): libcephsqlite-18.0.0-4757.g3fc0089c.e 530 kB/s | 137 kB 00:00 2023-07-13T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout:(15/144): librados-devel-18.0.0-4757.g3fc0089c. 556 kB/s | 139 kB 00:00 2023-07-13T22:12:24.395 INFO:teuthology.orchestra.run.smithi060.stdout:(16/144): libradosstriper1-18.0.0-4757.g3fc0089 737 kB/s | 473 kB 00:00 2023-07-13T22:12:25.153 INFO:teuthology.orchestra.run.smithi060.stdout:(17/144): libcephfs2-18.0.0-4757.g3fc0089c.el8. 399 kB/s | 771 kB 00:01 2023-07-13T22:12:25.279 INFO:teuthology.orchestra.run.smithi060.stdout:(18/144): python3-ceph-argparse-18.0.0-4757.g3f 357 kB/s | 45 kB 00:00 2023-07-13T22:12:25.629 INFO:teuthology.orchestra.run.smithi060.stdout:(19/144): python3-ceph-common-18.0.0-4757.g3fc0 339 kB/s | 119 kB 00:00 2023-07-13T22:12:26.221 INFO:teuthology.orchestra.run.smithi060.stdout:(20/144): python3-cephfs-18.0.0-4757.g3fc0089c. 332 kB/s | 197 kB 00:00 2023-07-13T22:12:27.080 INFO:teuthology.orchestra.run.smithi060.stdout:(21/144): python3-rados-18.0.0-4757.g3fc0089c.e 433 kB/s | 371 kB 00:00 2023-07-13T22:12:28.188 INFO:teuthology.orchestra.run.smithi060.stdout:(22/144): python3-rbd-18.0.0-4757.g3fc0089c.el8 329 kB/s | 364 kB 00:01 2023-07-13T22:12:28.621 INFO:teuthology.orchestra.run.smithi060.stdout:(23/144): python3-rgw-18.0.0-4757.g3fc0089c.el8 246 kB/s | 107 kB 00:00 2023-07-13T22:12:29.055 INFO:teuthology.orchestra.run.smithi060.stdout:(24/144): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 218 kB/s | 94 kB 00:00 2023-07-13T22:12:36.896 INFO:teuthology.orchestra.run.smithi060.stdout:(25/144): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 450 kB/s | 3.4 MB 00:07 2023-07-13T22:12:37.413 INFO:teuthology.orchestra.run.smithi060.stdout:(26/144): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 347 kB/s | 179 kB 00:00 2023-07-13T22:12:37.555 INFO:teuthology.orchestra.run.smithi060.stdout:(27/144): ceph-grafana-dashboards-18.0.0-4757.g 163 kB/s | 23 kB 00:00 2023-07-13T22:12:38.022 INFO:teuthology.orchestra.run.smithi060.stdout:(28/144): librgw2-18.0.0-4757.g3fc0089c.el8.x86 507 kB/s | 6.7 MB 00:13 2023-07-13T22:12:38.039 INFO:teuthology.orchestra.run.smithi060.stdout:(29/144): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 267 kB/s | 129 kB 00:00 2023-07-13T22:12:42.647 INFO:teuthology.orchestra.run.smithi060.stdout:(30/144): ceph-mgr-dashboard-18.0.0-4757.g3fc00 379 kB/s | 1.7 MB 00:04 2023-07-13T22:12:43.255 INFO:teuthology.orchestra.run.smithi060.stdout:(31/144): ceph-mgr-modules-core-18.0.0-4757.g3f 395 kB/s | 240 kB 00:00 2023-07-13T22:12:43.406 INFO:teuthology.orchestra.run.smithi060.stdout:(32/144): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 316 kB/s | 47 kB 00:00 2023-07-13T22:12:43.564 INFO:teuthology.orchestra.run.smithi060.stdout:(33/144): ceph-prometheus-alerts-18.0.0-4757.g3 92 kB/s | 15 kB 00:00 2023-07-13T22:12:44.231 INFO:teuthology.orchestra.run.smithi060.stdout:(34/144): ceph-volume-18.0.0-4757.g3fc0089c.el8 387 kB/s | 258 kB 00:00 2023-07-13T22:12:44.606 INFO:teuthology.orchestra.run.smithi060.stdout:(35/144): cephadm-18.0.0-4757.g3fc0089c.el8.noa 444 kB/s | 166 kB 00:00 2023-07-13T22:12:44.757 INFO:teuthology.orchestra.run.smithi060.stdout:(36/144): cephfs-top-18.0.0-4757.g3fc0089c.el8. 120 kB/s | 18 kB 00:00 2023-07-13T22:12:48.798 INFO:teuthology.orchestra.run.smithi060.stdout:(37/144): automake-1.16.1-7.el8.noarch.rpm 177 kB/s | 713 kB 00:04 2023-07-13T22:12:54.040 INFO:teuthology.orchestra.run.smithi060.stdout:(38/144): ceph-mgr-diskprediction-local-18.0.0- 474 kB/s | 7.4 MB 00:16 2023-07-13T22:12:55.356 INFO:teuthology.orchestra.run.smithi060.stdout:(39/144): flex-2.6.1-9.el8.x86_64.rpm 243 kB/s | 320 kB 00:01 2023-07-13T22:12:55.427 INFO:teuthology.orchestra.run.smithi105.stdout:(38/144): bison-3.0.4-10.el8.x86_64.rpm 18 kB/s | 688 kB 00:38 2023-07-13T22:12:57.665 INFO:teuthology.orchestra.run.smithi060.stdout:(40/144): jq-1.6-7.el8.x86_64.rpm 88 kB/s | 202 kB 00:02 2023-07-13T22:13:01.065 INFO:teuthology.orchestra.run.smithi060.stdout:(41/144): libpq-13.5-1.el8.x86_64.rpm 58 kB/s | 198 kB 00:03 2023-07-13T22:13:02.157 INFO:teuthology.orchestra.run.smithi060.stdout:(42/144): bison-3.0.4-10.el8.x86_64.rpm 52 kB/s | 688 kB 00:13 2023-07-13T22:13:03.218 INFO:teuthology.orchestra.run.smithi105.stdout:(39/144): flex-2.6.1-9.el8.x86_64.rpm 41 kB/s | 320 kB 00:07 2023-07-13T22:13:05.599 INFO:teuthology.orchestra.run.smithi060.stdout:(43/144): librdkafka-0.11.4-3.el8.x86_64.rpm 78 kB/s | 354 kB 00:04 2023-07-13T22:13:08.682 INFO:teuthology.orchestra.run.smithi060.stdout:(44/144): lttng-ust-2.8.1-11.el8.x86_64.rpm 84 kB/s | 259 kB 00:03 2023-07-13T22:13:09.440 INFO:teuthology.orchestra.run.smithi060.stdout:(45/144): libtool-2.4.6-25.el8.x86_64.rpm 97 kB/s | 709 kB 00:07 2023-07-13T22:13:14.765 INFO:teuthology.orchestra.run.smithi060.stdout:(46/144): oniguruma-6.8.2-2.el8.x86_64.rpm 31 kB/s | 187 kB 00:06 2023-07-13T22:13:20.633 INFO:teuthology.orchestra.run.smithi105.stdout:(40/144): ceph-test-18.0.0-4757.g3fc0089c.el8.x 466 kB/s | 48 MB 01:45 2023-07-13T22:13:32.130 INFO:teuthology.orchestra.run.smithi060.stdout:(47/144): ceph-test-18.0.0-4757.g3fc0089c.el8.x 525 kB/s | 48 MB 01:33 2023-07-13T22:13:37.280 INFO:teuthology.orchestra.run.smithi060.stdout:(48/144): openblas-threads-0.3.15-4.el8.x86_64. 221 kB/s | 4.9 MB 00:22 2023-07-13T22:13:41.621 INFO:teuthology.orchestra.run.smithi060.stdout:(49/144): postgresql-10.23-1.module_el8+300+2c5 169 kB/s | 1.6 MB 00:09 2023-07-13T22:13:56.428 INFO:teuthology.orchestra.run.smithi060.stdout:(50/144): postgresql-contrib-10.23-1.module_el8 47 kB/s | 903 kB 00:19 2023-07-13T22:14:08.051 INFO:teuthology.orchestra.run.smithi060.stdout:(51/144): postgresql-server-10.23-1.module_el8+ 198 kB/s | 5.1 MB 00:26 2023-07-13T22:14:11.485 INFO:teuthology.orchestra.run.smithi060.stdout:(52/144): python3-babel-2.5.1-7.el8.noarch.rpm 326 kB/s | 4.8 MB 00:15 2023-07-13T22:14:14.060 INFO:teuthology.orchestra.run.smithi060.stdout:(53/144): python3-jsonpatch-1.21-2.el8.noarch.r 10 kB/s | 27 kB 00:02 2023-07-13T22:14:16.826 INFO:teuthology.orchestra.run.smithi060.stdout:(54/144): python3-jinja2-2.10.1-3.el8.noarch.rp 61 kB/s | 538 kB 00:08 2023-07-13T22:14:20.284 INFO:teuthology.orchestra.run.smithi060.stdout:(55/144): python3-mako-1.0.6-14.el8.noarch.rpm 45 kB/s | 157 kB 00:03 2023-07-13T22:14:22.951 INFO:teuthology.orchestra.run.smithi060.stdout:(56/144): python3-markupsafe-0.23-19.el8.x86_64 15 kB/s | 39 kB 00:02 2023-07-13T22:14:25.843 INFO:teuthology.orchestra.run.smithi060.stdout:(57/144): python3-jsonpointer-1.10-11.el8.noarc 1.7 kB/s | 20 kB 00:11 2023-07-13T22:14:28.684 INFO:teuthology.orchestra.run.smithi060.stdout:(58/144): python3-numpy-f2py-1.14.3-10.el8.x86_ 79 kB/s | 225 kB 00:02 2023-07-13T22:14:31.415 INFO:teuthology.orchestra.run.smithi105.stdout:(41/144): libpq-13.5-1.el8.x86_64.rpm 2.8 kB/s | 198 kB 01:10 2023-07-13T22:14:33.450 INFO:teuthology.orchestra.run.smithi060.stdout:(59/144): python3-prettytable-0.7.2-14.el8.noar 9.2 kB/s | 44 kB 00:04 2023-07-13T22:14:34.426 INFO:teuthology.orchestra.run.smithi060.stdout:(60/144): python3-numpy-1.14.3-10.el8.x86_64.rp 333 kB/s | 3.7 MB 00:11 2023-07-13T22:14:35.065 INFO:teuthology.orchestra.run.smithi105.stdout:(42/144): librdkafka-0.11.4-3.el8.x86_64.rpm 97 kB/s | 354 kB 00:03 2023-07-13T22:14:38.850 INFO:teuthology.orchestra.run.smithi060.stdout:(61/144): python3-pyOpenSSL-19.0.0-1.el8.noarch 19 kB/s | 103 kB 00:05 2023-07-13T22:14:44.916 INFO:teuthology.orchestra.run.smithi060.stdout:(62/144): python3-pyasn1-modules-0.3.7-6.el8.no 18 kB/s | 110 kB 00:06 2023-07-13T22:14:45.683 INFO:teuthology.orchestra.run.smithi060.stdout:(63/144): python3-pytz-2017.2-9.el8.noarch.rpm 71 kB/s | 54 kB 00:00 2023-07-13T22:14:46.708 INFO:teuthology.orchestra.run.smithi060.stdout:(64/144): python3-pyasn1-0.3.7-6.el8.noarch.rpm 10 kB/s | 126 kB 00:12 2023-07-13T22:14:50.267 INFO:teuthology.orchestra.run.smithi060.stdout:(65/144): python3-werkzeug-0.12.2-4.el8.noarch. 129 kB/s | 457 kB 00:03 2023-07-13T22:14:52.908 INFO:teuthology.orchestra.run.smithi060.stdout:(66/144): socat-1.7.4.1-1.el8.x86_64.rpm 122 kB/s | 323 kB 00:02 2023-07-13T22:15:18.462 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] jq-1.6-7.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirrors.rcs.alaska.edu/centos/8-stream/AppStream/x86_64/os/Packages/jq-1.6-7.el8.x86_64.rpm [Failed to connect to mirrors.rcs.alaska.edu port 80: Connection timed out] 2023-07-13T22:15:18.818 INFO:teuthology.orchestra.run.smithi105.stdout:(43/144): jq-1.6-7.el8.x86_64.rpm 1.5 kB/s | 202 kB 02:15 2023-07-13T22:15:19.001 INFO:teuthology.orchestra.run.smithi105.stdout:(44/144): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2023-07-13T22:15:19.152 INFO:teuthology.orchestra.run.smithi105.stdout:(45/144): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-07-13T22:15:20.593 INFO:teuthology.orchestra.run.smithi105.stdout:(46/144): openblas-0.3.15-4.el8.x86_64.rpm 3.3 MB/s | 4.7 MB 00:01 2023-07-13T22:15:25.476 INFO:teuthology.orchestra.run.smithi105.stdout:(47/144): openblas-threads-0.3.15-4.el8.x86_64. 1.0 MB/s | 4.9 MB 00:04 2023-07-13T22:15:26.468 INFO:teuthology.orchestra.run.smithi105.stdout:(48/144): postgresql-10.23-1.module_el8+300+2c5 1.6 MB/s | 1.6 MB 00:00 2023-07-13T22:15:27.593 INFO:teuthology.orchestra.run.smithi105.stdout:(49/144): postgresql-contrib-10.23-1.module_el8 803 kB/s | 903 kB 00:01 2023-07-13T22:15:34.842 INFO:teuthology.orchestra.run.smithi105.stdout:(50/144): postgresql-server-10.23-1.module_el8+ 722 kB/s | 5.1 MB 00:07 2023-07-13T22:15:38.842 INFO:teuthology.orchestra.run.smithi105.stdout:(51/144): python3-babel-2.5.1-7.el8.noarch.rpm 1.2 MB/s | 4.8 MB 00:03 2023-07-13T22:15:39.201 INFO:teuthology.orchestra.run.smithi105.stdout:(52/144): python3-jinja2-2.10.1-3.el8.noarch.rp 1.5 MB/s | 538 kB 00:00 2023-07-13T22:15:39.259 INFO:teuthology.orchestra.run.smithi105.stdout:(53/144): python3-jsonpatch-1.21-2.el8.noarch.r 457 kB/s | 27 kB 00:00 2023-07-13T22:15:39.350 INFO:teuthology.orchestra.run.smithi105.stdout:(54/144): python3-jsonpointer-1.10-11.el8.noarc 221 kB/s | 20 kB 00:00 2023-07-13T22:15:39.468 INFO:teuthology.orchestra.run.smithi105.stdout:(55/144): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2023-07-13T22:15:39.535 INFO:teuthology.orchestra.run.smithi105.stdout:(56/144): python3-markupsafe-0.23-19.el8.x86_64 582 kB/s | 39 kB 00:00 2023-07-13T22:15:42.443 INFO:teuthology.orchestra.run.smithi105.stdout:(57/144): python3-numpy-1.14.3-10.el8.x86_64.rp 1.3 MB/s | 3.7 MB 00:02 2023-07-13T22:15:42.635 INFO:teuthology.orchestra.run.smithi105.stdout:(58/144): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.1 MB/s | 225 kB 00:00 2023-07-13T22:15:42.685 INFO:teuthology.orchestra.run.smithi105.stdout:(59/144): python3-prettytable-0.7.2-14.el8.noar 882 kB/s | 44 kB 00:00 2023-07-13T22:15:42.786 INFO:teuthology.orchestra.run.smithi105.stdout:(60/144): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2023-07-13T22:15:42.895 INFO:teuthology.orchestra.run.smithi105.stdout:(61/144): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-07-13T22:15:42.995 INFO:teuthology.orchestra.run.smithi105.stdout:(62/144): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-07-13T22:15:43.087 INFO:teuthology.orchestra.run.smithi105.stdout:(63/144): python3-pytz-2017.2-9.el8.noarch.rpm 590 kB/s | 54 kB 00:00 2023-07-13T22:15:43.878 INFO:teuthology.orchestra.run.smithi060.stdout:(67/144): python3-scipy-1.0.0-21.module_el8.5.0 241 kB/s | 14 MB 00:58 2023-07-13T22:15:49.752 INFO:teuthology.orchestra.run.smithi060.stdout:(68/144): xmlstarlet-1.6.1-20.el8.x86_64.rpm 12 kB/s | 71 kB 00:05 2023-07-13T22:15:49.961 INFO:teuthology.orchestra.run.smithi060.stdout:(69/144): elfutils-libelf-devel-0.189-2.el8.x86 295 kB/s | 61 kB 00:00 2023-07-13T22:15:50.186 INFO:teuthology.orchestra.run.smithi060.stdout:(70/144): fuse-2.9.7-17.el8.x86_64.rpm 369 kB/s | 83 kB 00:00 2023-07-13T22:15:50.253 INFO:teuthology.orchestra.run.smithi060.stdout:(71/144): keyutils-libs-devel-1.5.10-9.el8.x86_ 722 kB/s | 48 kB 00:00 2023-07-13T22:15:50.645 INFO:teuthology.orchestra.run.smithi060.stdout:(72/144): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.4 MB/s | 561 kB 00:00 2023-07-13T22:15:50.711 INFO:teuthology.orchestra.run.smithi060.stdout:(73/144): libaio-devel-0.3.112-1.el8.x86_64.rpm 280 kB/s | 19 kB 00:00 2023-07-13T22:15:50.770 INFO:teuthology.orchestra.run.smithi060.stdout:(74/144): libcom_err-devel-1.45.6-5.el8.x86_64. 664 kB/s | 39 kB 00:00 2023-07-13T22:15:51.154 INFO:teuthology.orchestra.run.smithi060.stdout:(75/144): libgfortran-8.5.0-20.el8.x86_64.rpm 1.6 MB/s | 645 kB 00:00 2023-07-13T22:15:51.262 INFO:teuthology.orchestra.run.smithi060.stdout:(76/144): libquadmath-8.5.0-20.el8.x86_64.rpm 1.6 MB/s | 172 kB 00:00 2023-07-13T22:15:51.338 INFO:teuthology.orchestra.run.smithi060.stdout:(77/144): librabbitmq-0.9.0-4.el8.x86_64.rpm 626 kB/s | 47 kB 00:00 2023-07-13T22:15:51.521 INFO:teuthology.orchestra.run.smithi060.stdout:(78/144): libselinux-devel-2.9-8.el8.x86_64.rpm 1.1 MB/s | 200 kB 00:00 2023-07-13T22:15:51.605 INFO:teuthology.orchestra.run.smithi060.stdout:(79/144): libsepol-devel-2.9-3.el8.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2023-07-13T22:15:51.746 INFO:teuthology.orchestra.run.smithi060.stdout:(80/144): libuuid-devel-2.32.1-42.el8.x86_64.rp 702 kB/s | 99 kB 00:00 2023-07-13T22:15:51.839 INFO:teuthology.orchestra.run.smithi060.stdout:(81/144): libverto-devel-0.3.2-2.el8.x86_64.rpm 196 kB/s | 18 kB 00:00 2023-07-13T22:15:51.914 INFO:teuthology.orchestra.run.smithi060.stdout:(82/144): libzstd-devel-1.4.4-1.el8.x86_64.rpm 587 kB/s | 44 kB 00:00 2023-07-13T22:15:53.814 INFO:teuthology.orchestra.run.smithi060.stdout:(83/144): openssl-devel-1.1.1k-9.el8.x86_64.rpm 1.7 MB/s | 3.2 MB 00:01 2023-07-13T22:15:54.180 INFO:teuthology.orchestra.run.smithi060.stdout:(84/144): pcre2-devel-10.32-3.el8.x86_64.rpm 1.6 MB/s | 605 kB 00:00 2023-07-13T22:15:54.347 INFO:teuthology.orchestra.run.smithi060.stdout:(85/144): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2023-07-13T22:15:54.514 INFO:teuthology.orchestra.run.smithi060.stdout:(86/144): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2023-07-13T22:15:54.731 INFO:teuthology.orchestra.run.smithi060.stdout:(87/144): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.1 MB/s | 249 kB 00:00 2023-07-13T22:15:54.780 INFO:teuthology.orchestra.run.smithi105.stdout:(64/144): python3-scipy-1.0.0-21.module_el8.5.0 1.2 MB/s | 14 MB 00:11 2023-07-13T22:15:55.245 INFO:teuthology.orchestra.run.smithi105.stdout:(65/144): python3-werkzeug-0.12.2-4.el8.noarch. 987 kB/s | 457 kB 00:00 2023-07-13T22:15:55.490 INFO:teuthology.orchestra.run.smithi060.stdout:(88/144): python3-cryptography-3.2.1-6.el8.x86_ 845 kB/s | 641 kB 00:00 2023-07-13T22:15:55.553 INFO:teuthology.orchestra.run.smithi105.stdout:(66/144): socat-1.7.4.1-1.el8.x86_64.rpm 1.0 MB/s | 323 kB 00:00 2023-07-13T22:15:55.582 INFO:teuthology.orchestra.run.smithi060.stdout:(89/144): python3-jwt-1.6.1-2.el8.noarch.rpm 466 kB/s | 43 kB 00:00 2023-07-13T22:15:55.774 INFO:teuthology.orchestra.run.smithi060.stdout:(90/144): python3-oauthlib-2.1.0-1.el8.noarch.r 809 kB/s | 155 kB 00:00 2023-07-13T22:15:55.899 INFO:teuthology.orchestra.run.smithi060.stdout:(91/144): python3-pycparser-2.14-14.el8.noarch. 872 kB/s | 109 kB 00:00 2023-07-13T22:15:55.937 INFO:teuthology.orchestra.run.smithi105.stdout:(67/144): uuid-1.6.2-43.el8.x86_64.rpm 167 kB/s | 64 kB 00:00 2023-07-13T22:15:55.983 INFO:teuthology.orchestra.run.smithi060.stdout:(92/144): python3-requests-oauthlib-1.0.0-1.el8 512 kB/s | 43 kB 00:00 2023-07-13T22:15:56.037 INFO:teuthology.orchestra.run.smithi105.stdout:(68/144): xmlstarlet-1.6.1-20.el8.x86_64.rpm 706 kB/s | 71 kB 00:00 2023-07-13T22:15:56.074 INFO:teuthology.orchestra.run.smithi060.stdout:(93/144): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 814 kB/s | 74 kB 00:00 2023-07-13T22:15:56.194 INFO:teuthology.orchestra.run.smithi060.stdout:(94/144): zlib-devel-1.2.11-21.el8.x86_64.rpm 485 kB/s | 58 kB 00:00 2023-07-13T22:15:56.762 INFO:teuthology.orchestra.run.smithi105.stdout:(69/144): elfutils-libelf-devel-0.189-2.el8.x86 85 kB/s | 61 kB 00:00 2023-07-13T22:15:56.809 INFO:teuthology.orchestra.run.smithi060.stdout:(95/144): gperftools-libs-2.7-9.el8.x86_64.rpm 499 kB/s | 306 kB 00:00 2023-07-13T22:15:56.904 INFO:teuthology.orchestra.run.smithi105.stdout:(70/144): fuse-2.9.7-17.el8.x86_64.rpm 586 kB/s | 83 kB 00:00 2023-07-13T22:15:56.959 INFO:teuthology.orchestra.run.smithi060.stdout:(96/144): liboath-2.6.2-3.el8.x86_64.rpm 393 kB/s | 59 kB 00:00 2023-07-13T22:15:57.084 INFO:teuthology.orchestra.run.smithi060.stdout:(97/144): libunwind-1.3.1-3.el8.x86_64.rpm 600 kB/s | 75 kB 00:00 2023-07-13T22:15:57.463 INFO:teuthology.orchestra.run.smithi105.stdout:(71/144): keyutils-libs-devel-1.5.10-9.el8.x86_ 86 kB/s | 48 kB 00:00 2023-07-13T22:15:57.498 INFO:teuthology.orchestra.run.smithi060.stdout:(98/144): python3-asyncssh-2.7.0-2.el8.noarch.r 1.1 MB/s | 476 kB 00:00 2023-07-13T22:15:57.593 INFO:teuthology.orchestra.run.smithi060.stdout:(99/144): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 462 kB/s | 44 kB 00:00 2023-07-13T22:15:57.718 INFO:teuthology.orchestra.run.smithi060.stdout:(100/144): python3-beautifulsoup4-4.6.3-2.el8.1 1.4 MB/s | 185 kB 00:00 2023-07-13T22:15:57.844 INFO:teuthology.orchestra.run.smithi060.stdout:(101/144): python3-cachetools-3.1.1-4.el8.noarc 264 kB/s | 33 kB 00:00 2023-07-13T22:15:57.969 INFO:teuthology.orchestra.run.smithi060.stdout:(102/144): python3-certifi-2018.10.15-7.el8.noa 130 kB/s | 16 kB 00:00 2023-07-13T22:15:58.102 INFO:teuthology.orchestra.run.smithi060.stdout:(103/144): python3-cheroot-8.5.2-1.el8.noarch.r 1.3 MB/s | 173 kB 00:00 2023-07-13T22:15:58.419 INFO:teuthology.orchestra.run.smithi060.stdout:(104/144): python3-cherrypy-18.4.0-1.el8.noarch 1.2 MB/s | 384 kB 00:00 2023-07-13T22:15:58.513 INFO:teuthology.orchestra.run.smithi105.stdout:(72/144): krb5-devel-1.18.2-22.el8.x86_64.rpm 534 kB/s | 561 kB 00:01 2023-07-13T22:15:58.545 INFO:teuthology.orchestra.run.smithi060.stdout:(105/144): python3-google-auth-1.1.1-10.el8.noa 651 kB/s | 81 kB 00:00 2023-07-13T22:15:58.630 INFO:teuthology.orchestra.run.smithi105.stdout:(73/144): libaio-devel-0.3.112-1.el8.x86_64.rpm 160 kB/s | 19 kB 00:00 2023-07-13T22:15:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:(106/144): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-07-13T22:15:58.747 INFO:teuthology.orchestra.run.smithi105.stdout:(74/144): libcom_err-devel-1.45.6-5.el8.x86_64. 332 kB/s | 39 kB 00:00 2023-07-13T22:15:58.812 INFO:teuthology.orchestra.run.smithi060.stdout:(107/144): python3-jaraco-functools-2.0-4.el8.n 132 kB/s | 18 kB 00:00 2023-07-13T22:15:59.504 INFO:teuthology.orchestra.run.smithi060.stdout:(108/144): python3-kubernetes-11.0.0-6.el8.noar 1.7 MB/s | 1.2 MB 00:00 2023-07-13T22:15:59.637 INFO:teuthology.orchestra.run.smithi060.stdout:(109/144): python3-logutils-0.3.5-11.el8.noarch 365 kB/s | 49 kB 00:00 2023-07-13T22:15:59.779 INFO:teuthology.orchestra.run.smithi060.stdout:(110/144): python3-more-itertools-7.2.0-3.el8.n 415 kB/s | 59 kB 00:00 2023-07-13T22:15:59.929 INFO:teuthology.orchestra.run.smithi060.stdout:(111/144): python3-natsort-7.1.1-2.el8.noarch.r 401 kB/s | 60 kB 00:00 2023-07-13T22:15:59.939 INFO:teuthology.orchestra.run.smithi105.stdout:(75/144): libgfortran-8.5.0-20.el8.x86_64.rpm 541 kB/s | 645 kB 00:01 2023-07-13T22:16:00.180 INFO:teuthology.orchestra.run.smithi060.stdout:(112/144): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-07-13T22:16:00.322 INFO:teuthology.orchestra.run.smithi060.stdout:(113/144): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2023-07-13T22:16:00.330 INFO:teuthology.orchestra.run.smithi105.stdout:(76/144): libquadmath-8.5.0-20.el8.x86_64.rpm 440 kB/s | 172 kB 00:00 2023-07-13T22:16:00.447 INFO:teuthology.orchestra.run.smithi105.stdout:(77/144): librabbitmq-0.9.0-4.el8.x86_64.rpm 402 kB/s | 47 kB 00:00 2023-07-13T22:16:00.455 INFO:teuthology.orchestra.run.smithi060.stdout:(114/144): python3-repoze-lru-0.7-6.el8.noarch. 252 kB/s | 34 kB 00:00 2023-07-13T22:16:00.589 INFO:teuthology.orchestra.run.smithi060.stdout:(115/144): python3-routes-2.4.1-12.el8.noarch.r 1.4 MB/s | 196 kB 00:00 2023-07-13T22:16:00.714 INFO:teuthology.orchestra.run.smithi060.stdout:(116/144): python3-rsa-4.8-1.el8.noarch.rpm 492 kB/s | 61 kB 00:00 2023-07-13T22:16:00.839 INFO:teuthology.orchestra.run.smithi105.stdout:(78/144): libselinux-devel-2.9-8.el8.x86_64.rpm 511 kB/s | 200 kB 00:00 2023-07-13T22:16:00.872 INFO:teuthology.orchestra.run.smithi060.stdout:(117/144): python3-simplegeneric-0.8.1-17.el8.n 118 kB/s | 19 kB 00:00 2023-07-13T22:16:01.323 INFO:teuthology.orchestra.run.smithi105.stdout:(79/144): libsepol-devel-2.9-3.el8.x86_64.rpm 180 kB/s | 87 kB 00:00 2023-07-13T22:16:01.356 INFO:teuthology.orchestra.run.smithi060.stdout:(118/144): python3-singledispatch-3.4.0.3-18.el 49 kB/s | 24 kB 00:00 2023-07-13T22:16:01.473 INFO:teuthology.orchestra.run.smithi060.stdout:(119/144): python3-tempora-1.14.1-5.el8.noarch. 251 kB/s | 29 kB 00:00 2023-07-13T22:16:01.598 INFO:teuthology.orchestra.run.smithi105.stdout:(80/144): libuuid-devel-2.32.1-42.el8.x86_64.rp 362 kB/s | 99 kB 00:00 2023-07-13T22:16:01.607 INFO:teuthology.orchestra.run.smithi060.stdout:(120/144): python3-trustme-0.6.0-4.el8.noarch.r 206 kB/s | 27 kB 00:00 2023-07-13T22:16:01.723 INFO:teuthology.orchestra.run.smithi105.stdout:(81/144): libverto-devel-0.3.2-2.el8.x86_64.rpm 144 kB/s | 18 kB 00:00 2023-07-13T22:16:01.832 INFO:teuthology.orchestra.run.smithi105.stdout:(82/144): libzstd-devel-1.4.4-1.el8.x86_64.rpm 404 kB/s | 44 kB 00:00 2023-07-13T22:16:01.857 INFO:teuthology.orchestra.run.smithi060.stdout:(121/144): python3-waitress-1.4.3-1.el8.noarch. 1.0 MB/s | 255 kB 00:00 2023-07-13T22:16:02.099 INFO:teuthology.orchestra.run.smithi060.stdout:(122/144): python3-webob-1.8.5-1.el8.1.noarch.r 1.0 MB/s | 251 kB 00:00 2023-07-13T22:16:02.224 INFO:teuthology.orchestra.run.smithi060.stdout:(123/144): python3-websocket-client-0.56.0-5.el 490 kB/s | 61 kB 00:00 2023-07-13T22:16:02.366 INFO:teuthology.orchestra.run.smithi060.stdout:(124/144): python3-webtest-2.0.33-1.el8.noarch. 603 kB/s | 85 kB 00:00 2023-07-13T22:16:02.516 INFO:teuthology.orchestra.run.smithi060.stdout:(125/144): python3-zc-lockfile-2.0-2.el8.noarch 153 kB/s | 23 kB 00:00 2023-07-13T22:16:03.658 INFO:teuthology.orchestra.run.smithi060.stdout:(126/144): thrift-0.13.0-2.el8.x86_64.rpm 1.5 MB/s | 1.7 MB 00:01 2023-07-13T22:16:06.631 INFO:teuthology.orchestra.run.smithi105.stdout:(83/144): openssl-devel-1.1.1k-9.el8.x86_64.rpm 678 kB/s | 3.2 MB 00:04 2023-07-13T22:16:07.506 INFO:teuthology.orchestra.run.smithi105.stdout:(84/144): pcre2-devel-10.32-3.el8.x86_64.rpm 691 kB/s | 605 kB 00:00 2023-07-13T22:16:07.940 INFO:teuthology.orchestra.run.smithi105.stdout:(85/144): pcre2-utf16-10.32-3.el8.x86_64.rpm 529 kB/s | 229 kB 00:00 2023-07-13T22:16:08.315 INFO:teuthology.orchestra.run.smithi105.stdout:(86/144): pcre2-utf32-10.32-3.el8.x86_64.rpm 588 kB/s | 220 kB 00:00 2023-07-13T22:16:08.615 INFO:teuthology.orchestra.run.smithi105.stdout:(87/144): python3-cffi-1.11.5-6.el8.x86_64.rpm 829 kB/s | 249 kB 00:00 2023-07-13T22:16:09.416 INFO:teuthology.orchestra.run.smithi105.stdout:(88/144): python3-cryptography-3.2.1-6.el8.x86_ 802 kB/s | 641 kB 00:00 2023-07-13T22:16:09.516 INFO:teuthology.orchestra.run.smithi105.stdout:(89/144): python3-jwt-1.6.1-2.el8.noarch.rpm 426 kB/s | 43 kB 00:00 2023-07-13T22:16:09.883 INFO:teuthology.orchestra.run.smithi105.stdout:(90/144): python3-oauthlib-2.1.0-1.el8.noarch.r 423 kB/s | 155 kB 00:00 2023-07-13T22:16:10.183 INFO:teuthology.orchestra.run.smithi105.stdout:(91/144): python3-pycparser-2.14-14.el8.noarch. 363 kB/s | 109 kB 00:00 2023-07-13T22:16:10.409 INFO:teuthology.orchestra.run.smithi105.stdout:(92/144): python3-requests-oauthlib-1.0.0-1.el8 190 kB/s | 43 kB 00:00 2023-07-13T22:16:10.609 INFO:teuthology.orchestra.run.smithi105.stdout:(93/144): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 373 kB/s | 74 kB 00:00 2023-07-13T22:16:10.826 INFO:teuthology.orchestra.run.smithi105.stdout:(94/144): zlib-devel-1.2.11-21.el8.x86_64.rpm 268 kB/s | 58 kB 00:00 2023-07-13T22:16:11.766 INFO:teuthology.orchestra.run.smithi060.stdout:(127/144): librados2-18.0.0-4757.g3fc0089c.el8. 463 kB/s | 3.7 MB 00:08 2023-07-13T22:16:11.976 INFO:teuthology.orchestra.run.smithi105.stdout:(95/144): gperftools-libs-2.7-9.el8.x86_64.rpm 266 kB/s | 306 kB 00:01 2023-07-13T22:16:12.201 INFO:teuthology.orchestra.run.smithi105.stdout:(96/144): liboath-2.6.2-3.el8.x86_64.rpm 261 kB/s | 59 kB 00:00 2023-07-13T22:16:12.393 INFO:teuthology.orchestra.run.smithi105.stdout:(97/144): libunwind-1.3.1-3.el8.x86_64.rpm 392 kB/s | 75 kB 00:00 2023-07-13T22:16:12.752 INFO:teuthology.orchestra.run.smithi105.stdout:(98/144): python3-asyncssh-2.7.0-2.el8.noarch.r 1.3 MB/s | 476 kB 00:00 2023-07-13T22:16:12.927 INFO:teuthology.orchestra.run.smithi105.stdout:(99/144): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 249 kB/s | 44 kB 00:00 2023-07-13T22:16:13.127 INFO:teuthology.orchestra.run.smithi105.stdout:(100/144): python3-beautifulsoup4-4.6.3-2.el8.1 927 kB/s | 185 kB 00:00 2023-07-13T22:16:13.311 INFO:teuthology.orchestra.run.smithi105.stdout:(101/144): python3-cachetools-3.1.1-4.el8.noarc 180 kB/s | 33 kB 00:00 2023-07-13T22:16:13.503 INFO:teuthology.orchestra.run.smithi105.stdout:(102/144): python3-certifi-2018.10.15-7.el8.noa 85 kB/s | 16 kB 00:00 2023-07-13T22:16:13.728 INFO:teuthology.orchestra.run.smithi105.stdout:(103/144): python3-cheroot-8.5.2-1.el8.noarch.r 768 kB/s | 173 kB 00:00 2023-07-13T22:16:13.962 INFO:teuthology.orchestra.run.smithi105.stdout:(104/144): python3-cherrypy-18.4.0-1.el8.noarch 1.6 MB/s | 384 kB 00:00 2023-07-13T22:16:14.145 INFO:teuthology.orchestra.run.smithi105.stdout:(105/144): python3-google-auth-1.1.1-10.el8.noa 443 kB/s | 81 kB 00:00 2023-07-13T22:16:14.320 INFO:teuthology.orchestra.run.smithi105.stdout:(106/144): python3-jaraco-6.2-6.el8.noarch.rpm 63 kB/s | 11 kB 00:00 2023-07-13T22:16:14.513 INFO:teuthology.orchestra.run.smithi105.stdout:(107/144): python3-jaraco-functools-2.0-4.el8.n 92 kB/s | 18 kB 00:00 2023-07-13T22:16:14.896 INFO:teuthology.orchestra.run.smithi105.stdout:(108/144): python3-kubernetes-11.0.0-6.el8.noar 3.0 MB/s | 1.2 MB 00:00 2023-07-13T22:16:15.105 INFO:teuthology.orchestra.run.smithi105.stdout:(109/144): python3-logutils-0.3.5-11.el8.noarch 234 kB/s | 49 kB 00:00 2023-07-13T22:16:15.296 INFO:teuthology.orchestra.run.smithi105.stdout:(110/144): python3-more-itertools-7.2.0-3.el8.n 307 kB/s | 59 kB 00:00 2023-07-13T22:16:15.480 INFO:teuthology.orchestra.run.smithi105.stdout:(111/144): python3-natsort-7.1.1-2.el8.noarch.r 329 kB/s | 60 kB 00:00 2023-07-13T22:16:15.714 INFO:teuthology.orchestra.run.smithi105.stdout:(112/144): python3-pecan-1.3.2-9.el8.noarch.rpm 1.2 MB/s | 283 kB 00:00 2023-07-13T22:16:15.906 INFO:teuthology.orchestra.run.smithi105.stdout:(113/144): python3-portend-2.6-1.el8.noarch.rpm 81 kB/s | 16 kB 00:00 2023-07-13T22:16:16.089 INFO:teuthology.orchestra.run.smithi105.stdout:(114/144): python3-repoze-lru-0.7-6.el8.noarch. 183 kB/s | 34 kB 00:00 2023-07-13T22:16:16.298 INFO:teuthology.orchestra.run.smithi105.stdout:(115/144): python3-routes-2.4.1-12.el8.noarch.r 943 kB/s | 196 kB 00:00 2023-07-13T22:16:16.498 INFO:teuthology.orchestra.run.smithi105.stdout:(116/144): python3-rsa-4.8-1.el8.noarch.rpm 308 kB/s | 61 kB 00:00 2023-07-13T22:16:16.665 INFO:teuthology.orchestra.run.smithi105.stdout:(117/144): python3-simplegeneric-0.8.1-17.el8.n 112 kB/s | 19 kB 00:00 2023-07-13T22:16:16.824 INFO:teuthology.orchestra.run.smithi105.stdout:(118/144): python3-singledispatch-3.4.0.3-18.el 151 kB/s | 24 kB 00:00 2023-07-13T22:16:16.991 INFO:teuthology.orchestra.run.smithi105.stdout:(119/144): python3-tempora-1.14.1-5.el8.noarch. 175 kB/s | 29 kB 00:00 2023-07-13T22:16:17.133 INFO:teuthology.orchestra.run.smithi105.stdout:(120/144): python3-trustme-0.6.0-4.el8.noarch.r 194 kB/s | 27 kB 00:00 2023-07-13T22:16:17.291 INFO:teuthology.orchestra.run.smithi105.stdout:(121/144): python3-waitress-1.4.3-1.el8.noarch. 1.6 MB/s | 255 kB 00:00 2023-07-13T22:16:17.650 INFO:teuthology.orchestra.run.smithi105.stdout:(122/144): python3-webob-1.8.5-1.el8.1.noarch.r 701 kB/s | 251 kB 00:00 2023-07-13T22:16:17.817 INFO:teuthology.orchestra.run.smithi105.stdout:(123/144): python3-websocket-client-0.56.0-5.el 367 kB/s | 61 kB 00:00 2023-07-13T22:16:17.984 INFO:teuthology.orchestra.run.smithi105.stdout:(124/144): python3-webtest-2.0.33-1.el8.noarch. 513 kB/s | 85 kB 00:00 2023-07-13T22:16:18.176 INFO:teuthology.orchestra.run.smithi105.stdout:(125/144): python3-zc-lockfile-2.0-2.el8.noarch 120 kB/s | 23 kB 00:00 2023-07-13T22:16:18.307 INFO:teuthology.orchestra.run.smithi060.stdout:(128/144): librbd1-18.0.0-4757.g3fc0089c.el8.x8 557 kB/s | 3.6 MB 00:06 2023-07-13T22:16:18.618 INFO:teuthology.orchestra.run.smithi105.stdout:(126/144): thrift-0.13.0-2.el8.x86_64.rpm 3.9 MB/s | 1.7 MB 00:00 2023-07-13T22:16:19.007 INFO:teuthology.orchestra.run.smithi060.stdout:(129/144): NetworkManager-initscripts-updown-1. 205 kB/s | 143 kB 00:00 2023-07-13T22:16:19.391 INFO:teuthology.orchestra.run.smithi060.stdout:(130/144): elfutils-debuginfod-client-0.189-2.e 198 kB/s | 76 kB 00:00 2023-07-13T22:16:20.233 INFO:teuthology.orchestra.run.smithi060.stdout:(131/144): elfutils-libelf-0.189-2.el8.x86_64.r 275 kB/s | 232 kB 00:00 2023-07-13T22:16:20.941 INFO:teuthology.orchestra.run.smithi060.stdout:(132/144): elfutils-libs-0.189-2.el8.x86_64.rpm 427 kB/s | 302 kB 00:00 2023-07-13T22:16:22.276 INFO:teuthology.orchestra.run.smithi105.stdout:(127/144): librados2-18.0.0-4757.g3fc0089c.el8. 1.0 MB/s | 3.7 MB 00:03 2023-07-13T22:16:22.500 INFO:teuthology.orchestra.run.smithi060.stdout:(133/144): iproute-6.2.0-2.el8.x86_64.rpm 565 kB/s | 880 kB 00:01 2023-07-13T22:16:22.933 INFO:teuthology.orchestra.run.smithi060.stdout:(134/144): libblkid-2.32.1-42.el8.x86_64.rpm 508 kB/s | 220 kB 00:00 2023-07-13T22:16:23.333 INFO:teuthology.orchestra.run.smithi060.stdout:(135/144): libfdisk-2.32.1-42.el8.x86_64.rpm 634 kB/s | 253 kB 00:00 2023-07-13T22:16:23.775 INFO:teuthology.orchestra.run.smithi060.stdout:(136/144): libmount-2.32.1-42.el8.x86_64.rpm 535 kB/s | 236 kB 00:00 2023-07-13T22:16:24.009 INFO:teuthology.orchestra.run.smithi060.stdout:(137/144): libsmartcols-2.32.1-42.el8.x86_64.rp 768 kB/s | 179 kB 00:00 2023-07-13T22:16:24.201 INFO:teuthology.orchestra.run.smithi060.stdout:(138/144): libuuid-2.32.1-42.el8.x86_64.rpm 514 kB/s | 98 kB 00:00 2023-07-13T22:16:24.418 INFO:teuthology.orchestra.run.smithi105.stdout:(128/144): librbd1-18.0.0-4757.g3fc0089c.el8.x8 1.7 MB/s | 3.6 MB 00:02 2023-07-13T22:16:24.860 INFO:teuthology.orchestra.run.smithi105.stdout:(129/144): NetworkManager-initscripts-updown-1. 325 kB/s | 143 kB 00:00 2023-07-13T22:16:24.993 INFO:teuthology.orchestra.run.smithi105.stdout:(130/144): elfutils-debuginfod-client-0.189-2.e 569 kB/s | 76 kB 00:00 2023-07-13T22:16:25.168 INFO:teuthology.orchestra.run.smithi105.stdout:(131/144): elfutils-libelf-0.189-2.el8.x86_64.r 1.3 MB/s | 232 kB 00:00 2023-07-13T22:16:25.327 INFO:teuthology.orchestra.run.smithi105.stdout:(132/144): elfutils-libs-0.189-2.el8.x86_64.rpm 1.9 MB/s | 302 kB 00:00 2023-07-13T22:16:25.502 INFO:teuthology.orchestra.run.smithi105.stdout:(133/144): iproute-6.2.0-2.el8.x86_64.rpm 4.9 MB/s | 880 kB 00:00 2023-07-13T22:16:25.619 INFO:teuthology.orchestra.run.smithi105.stdout:(134/144): libblkid-2.32.1-42.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-07-13T22:16:25.719 INFO:teuthology.orchestra.run.smithi105.stdout:(135/144): libfdisk-2.32.1-42.el8.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2023-07-13T22:16:26.220 INFO:teuthology.orchestra.run.smithi105.stdout:(136/144): libmount-2.32.1-42.el8.x86_64.rpm 473 kB/s | 236 kB 00:00 2023-07-13T22:16:26.345 INFO:teuthology.orchestra.run.smithi105.stdout:(137/144): libsmartcols-2.32.1-42.el8.x86_64.rp 1.4 MB/s | 179 kB 00:00 2023-07-13T22:16:26.445 INFO:teuthology.orchestra.run.smithi105.stdout:(138/144): libuuid-2.32.1-42.el8.x86_64.rpm 985 kB/s | 98 kB 00:00 2023-07-13T22:16:26.754 INFO:teuthology.orchestra.run.smithi105.stdout:(139/144): util-linux-2.32.1-42.el8.x86_64.rpm 8.4 MB/s | 2.6 MB 00:00 2023-07-13T22:16:26.871 INFO:teuthology.orchestra.run.smithi105.stdout:(140/144): util-linux-user-2.32.1-42.el8.x86_64 878 kB/s | 102 kB 00:00 2023-07-13T22:16:26.950 INFO:teuthology.orchestra.run.smithi060.stdout:(139/144): util-linux-2.32.1-42.el8.x86_64.rpm 966 kB/s | 2.6 MB 00:02 2023-07-13T22:16:27.004 INFO:teuthology.orchestra.run.smithi105.stdout:(141/144): xfsdump-3.1.8-6.el8.x86_64.rpm 2.4 MB/s | 334 kB 00:00 2023-07-13T22:16:27.201 INFO:teuthology.orchestra.run.smithi060.stdout:(140/144): util-linux-user-2.32.1-42.el8.x86_64 409 kB/s | 102 kB 00:00 2023-07-13T22:16:27.255 INFO:teuthology.orchestra.run.smithi105.stdout:(142/144): xfsprogs-5.0.0-12.el8.x86_64.rpm 4.3 MB/s | 1.1 MB 00:00 2023-07-13T22:16:27.559 INFO:teuthology.orchestra.run.smithi060.stdout:(141/144): xfsdump-3.1.8-6.el8.x86_64.rpm 932 kB/s | 334 kB 00:00 2023-07-13T22:16:28.326 INFO:teuthology.orchestra.run.smithi060.stdout:(142/144): xfsprogs-5.0.0-12.el8.x86_64.rpm 1.4 MB/s | 1.1 MB 00:00 2023-07-13T22:16:48.555 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] libtool-2.4.6-25.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirrors.rcs.alaska.edu/centos/8-stream/AppStream/x86_64/os/Packages/libtool-2.4.6-25.el8.x86_64.rpm [Failed to connect to mirrors.rcs.alaska.edu port 80: Connection timed out] 2023-07-13T22:16:52.268 INFO:teuthology.orchestra.run.smithi105.stdout:(143/144): libtool-2.4.6-25.el8.x86_64.rpm 5.2 kB/s | 709 kB 02:17 2023-07-13T22:17:04.040 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] automake-1.16.1-7.el8.noarch.rpm: Curl error (28): Timeout was reached for http://mirrors.rcs.alaska.edu/centos/8-stream/AppStream/x86_64/os/Packages/automake-1.16.1-7.el8.noarch.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2023-07-13T22:17:05.258 INFO:teuthology.orchestra.run.smithi105.stdout:(144/144): automake-1.16.1-7.el8.noarch.rpm 2.4 kB/s | 713 kB 05:03 2023-07-13T22:17:05.261 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-07-13T22:17:05.262 INFO:teuthology.orchestra.run.smithi105.stdout:Total 587 kB/s | 212 MB 06:10 2023-07-13T22:17:07.014 INFO:teuthology.orchestra.run.smithi060.stdout:[MIRROR] uuid-1.6.2-43.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirrors.rcs.alaska.edu/centos/8-stream/AppStream/x86_64/os/Packages/uuid-1.6.2-43.el8.x86_64.rpm [Failed to connect to mirrors.rcs.alaska.edu port 80: Connection timed out] 2023-07-13T22:17:07.093 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-07-13T22:17:07.180 INFO:teuthology.orchestra.run.smithi060.stdout:(143/144): uuid-1.6.2-43.el8.x86_64.rpm 487 B/s | 64 kB 02:14 2023-07-13T22:17:07.505 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-07-13T22:17:07.505 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-07-13T22:17:08.989 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-07-13T22:17:08.992 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-07-13T22:17:14.379 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-07-13T22:17:14.600 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-07-13T22:17:14.802 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/160 2023-07-13T22:17:14.947 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-13T22:17:15.039 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-13T22:17:15.205 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-13T22:17:15.325 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-13T22:17:15.552 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/160 2023-07-13T22:17:15.761 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.189-2.el8.x86_64 5/160 2023-07-13T22:17:15.995 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/160 2023-07-13T22:17:16.180 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-13T22:17:16.657 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-13T22:17:16.863 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/160 2023-07-13T22:17:17.329 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/160 2023-07-13T22:17:17.580 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/160 2023-07-13T22:17:17.726 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/160 2023-07-13T22:17:17.921 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/160 2023-07-13T22:17:18.037 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/160 2023-07-13T22:17:18.257 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/160 2023-07-13T22:17:18.421 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-13T22:17:18.646 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-13T22:17:18.829 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-13T22:17:18.907 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-13T22:17:19.130 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/160 2023-07-13T22:17:19.323 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/160 2023-07-13T22:17:19.795 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpq-13.5-1.el8.x86_64 16/160 2023-07-13T22:17:19.993 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/160 2023-07-13T22:17:20.146 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 18/160 2023-07-13T22:17:20.271 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 18/160 2023-07-13T22:17:20.429 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 19/160 2023-07-13T22:17:20.569 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 19/160 2023-07-13T22:17:20.818 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/160 2023-07-13T22:17:20.969 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 21/160 2023-07-13T22:17:21.113 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-13T22:17:21.275 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-13T22:17:21.454 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-13T22:17:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-13T22:17:22.321 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/160 2023-07-13T22:17:23.020 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/160 2023-07-13T22:17:23.204 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-13T22:17:24.038 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-13T22:17:24.389 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/160 2023-07-13T22:17:24.588 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/160 2023-07-13T22:17:24.753 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/160 2023-07-13T22:17:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/160 2023-07-13T22:17:25.228 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 30/160 2023-07-13T22:17:25.389 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 31/160 2023-07-13T22:17:25.630 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 32/160 2023-07-13T22:17:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/160 2023-07-13T22:17:27.078 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/160 2023-07-13T22:17:27.425 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/160 2023-07-13T22:17:27.629 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-13T22:17:29.722 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-13T22:17:30.042 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/160 2023-07-13T22:17:30.263 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/160 2023-07-13T22:17:30.466 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/160 2023-07-13T22:17:30.702 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 40/160 2023-07-13T22:17:30.881 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 41/160 2023-07-13T22:17:31.066 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.189-2.el8.x86_64 42/160 2023-07-13T22:17:31.264 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.189-2.el8.x86_64 43/160 2023-07-13T22:17:31.503 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-13T22:17:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-13T22:17:31.828 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libmount-2.32.1-42.el8.x86_64 45/160 2023-07-13T22:17:31.920 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libmount-2.32.1-42.el8.x86_64 45/160 2023-07-13T22:17:32.091 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.32.1-42.el8.x86_64 46/160 2023-07-13T22:17:32.286 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-13T22:17:32.886 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-13T22:17:33.170 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-13T22:17:33.198 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-13T22:17:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/160 2023-07-13T22:17:33.712 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/160 2023-07-13T22:17:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/160 2023-07-13T22:17:34.108 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/160 2023-07-13T22:17:34.265 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/160 2023-07-13T22:17:34.467 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/160 2023-07-13T22:17:34.668 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/160 2023-07-13T22:17:35.204 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/160 2023-07-13T22:17:35.398 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/160 2023-07-13T22:17:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/160 2023-07-13T22:17:35.716 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/160 2023-07-13T22:17:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/160 2023-07-13T22:17:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/160 2023-07-13T22:17:36.272 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/160 2023-07-13T22:17:36.507 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/160 2023-07-13T22:17:36.743 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/160 2023-07-13T22:17:36.944 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/160 2023-07-13T22:17:37.322 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/160 2023-07-13T22:17:37.563 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/160 2023-07-13T22:17:37.752 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/160 2023-07-13T22:17:38.576 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/160 2023-07-13T22:17:38.798 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 70/160 2023-07-13T22:17:38.891 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 70/160 2023-07-13T22:17:39.122 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 71/160 2023-07-13T22:17:39.389 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 72/160 2023-07-13T22:17:39.797 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 73/160 2023-07-13T22:17:40.076 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 74/160 2023-07-13T22:17:40.258 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 75/160 2023-07-13T22:17:40.540 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 76/160 2023-07-13T22:17:40.770 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 77/160 2023-07-13T22:17:42.067 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/160 2023-07-13T22:17:42.448 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/160 2023-07-13T22:17:42.695 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 80/160 2023-07-13T22:17:42.939 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 81/160 2023-07-13T22:17:43.245 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 82/160 2023-07-13T22:17:43.500 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/160 2023-07-13T22:17:43.755 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 84/160 2023-07-13T22:17:43.996 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 85/160 2023-07-13T22:17:44.344 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 86/160 2023-07-13T22:17:44.565 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 87/160 2023-07-13T22:17:44.745 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 88/160 2023-07-13T22:17:44.937 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/160 2023-07-13T22:17:45.348 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/160 2023-07-13T22:17:45.589 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 91/160 2023-07-13T22:17:45.783 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/160 2023-07-13T22:17:45.990 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/160 2023-07-13T22:17:46.212 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/160 2023-07-13T22:17:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.7-17.el8.x86_64 95/160 2023-07-13T22:17:46.673 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/160 2023-07-13T22:17:46.900 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/160 2023-07-13T22:17:47.028 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/160 2023-07-13T22:17:47.227 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 98/160 2023-07-13T22:17:47.368 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 99/160 2023-07-13T22:17:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/160 2023-07-13T22:17:50.784 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/160 2023-07-13T22:17:51.477 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/160 2023-07-13T22:17:51.756 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 101/160 2023-07-13T22:17:51.955 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 101/160 2023-07-13T22:17:52.142 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 102/160 2023-07-13T22:18:10.439 INFO:teuthology.orchestra.run.smithi060.stdout:[MIRROR] openblas-0.3.15-4.el8.x86_64.rpm: Curl error (28): Timeout was reached for http://mirrors.rcs.alaska.edu/centos/8-stream/AppStream/x86_64/os/Packages/openblas-0.3.15-4.el8.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2023-07-13T22:18:11.821 INFO:teuthology.orchestra.run.smithi060.stdout:(144/144): openblas-0.3.15-4.el8.x86_64.rpm 16 kB/s | 4.7 MB 05:02 2023-07-13T22:18:11.822 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-07-13T22:18:11.823 INFO:teuthology.orchestra.run.smithi060.stdout:Total 547 kB/s | 212 MB 06:37 2023-07-13T22:18:12.454 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 102/160 2023-07-13T22:18:12.454 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2023-07-13T22:18:12.454 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2023-07-13T22:18:12.454 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /dev 2023-07-13T22:18:12.455 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /run 2023-07-13T22:18:12.455 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2023-07-13T22:18:12.455 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2023-07-13T22:18:12.455 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2023-07-13T22:18:12.455 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2023-07-13T22:18:12.455 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /dev 2023-07-13T22:18:12.455 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:18:12.897 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 103/160 2023-07-13T22:18:13.705 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-07-13T22:18:14.145 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-07-13T22:18:14.146 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-07-13T22:18:14.657 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 103/160 2023-07-13T22:18:14.902 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 104/160 2023-07-13T22:18:15.171 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 104/160 2023-07-13T22:18:15.397 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 105/160 2023-07-13T22:18:15.684 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-07-13T22:18:15.689 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-07-13T22:18:15.928 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 105/160 2023-07-13T22:18:16.180 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 106/160 2023-07-13T22:18:16.229 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 106/160 2023-07-13T22:18:16.438 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 107/160 2023-07-13T22:18:16.660 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 108/160 2023-07-13T22:18:16.838 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-13T22:18:16.931 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-13T22:18:17.240 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-7.el8.x86_64 110/160 2023-07-13T22:18:17.455 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.1-7.el8.noarch 111/160 2023-07-13T22:18:18.065 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 112/160 2023-07-13T22:18:18.246 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 112/160 2023-07-13T22:18:18.447 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 113/160 2023-07-13T22:18:18.660 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 114/160 2023-07-13T22:18:18.863 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 115/160 2023-07-13T22:18:19.122 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 115/160 2023-07-13T22:18:19.359 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 116/160 2023-07-13T22:18:20.480 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 116/160 2023-07-13T22:18:20.662 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 117/160 2023-07-13T22:18:20.793 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 117/160 2023-07-13T22:18:21.167 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 118/160 2023-07-13T22:18:21.176 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-07-13T22:18:21.368 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/160 2023-07-13T22:18:21.414 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-07-13T22:18:21.433 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/160 2023-07-13T22:18:21.602 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/160 2023-07-13T22:18:21.620 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 120/160 2023-07-13T22:18:21.648 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 120/160 2023-07-13T22:18:21.739 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-13T22:18:21.831 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-13T22:18:21.835 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 121/160 2023-07-13T22:18:22.004 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-25.el8.x86_64 122/160 2023-07-13T22:18:22.030 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-13T22:18:22.105 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-13T22:18:22.324 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/160 2023-07-13T22:18:22.527 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libelf-0.189-2.el8.x86_64 5/160 2023-07-13T22:18:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/160 2023-07-13T22:18:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-13T22:18:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-13T22:18:23.572 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/160 2023-07-13T22:18:24.045 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/160 2023-07-13T22:18:24.197 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/160 2023-07-13T22:18:24.350 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/160 2023-07-13T22:18:24.622 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/160 2023-07-13T22:18:24.777 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/160 2023-07-13T22:18:24.967 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/160 2023-07-13T22:18:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-13T22:18:25.353 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-13T22:18:25.513 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-13T22:18:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-13T22:18:25.802 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/160 2023-07-13T22:18:26.024 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/160 2023-07-13T22:18:26.493 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpq-13.5-1.el8.x86_64 16/160 2023-07-13T22:18:26.660 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/160 2023-07-13T22:18:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 18/160 2023-07-13T22:18:26.936 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 18/160 2023-07-13T22:18:27.020 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 122/160 2023-07-13T22:18:27.071 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 19/160 2023-07-13T22:18:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 19/160 2023-07-13T22:18:27.451 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/160 2023-07-13T22:18:27.613 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 21/160 2023-07-13T22:18:27.738 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-13T22:18:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-13T22:18:28.037 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-13T22:18:28.597 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 123/160 2023-07-13T22:18:28.744 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-13T22:18:28.800 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 124/160 2023-07-13T22:18:28.862 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 124/160 2023-07-13T22:18:28.896 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/160 2023-07-13T22:18:29.062 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 125/160 2023-07-13T22:18:29.132 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 125/160 2023-07-13T22:18:29.395 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/160 2023-07-13T22:18:29.640 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/160 2023-07-13T22:18:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/160 2023-07-13T22:18:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-13T22:18:30.011 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 127/160 2023-07-13T22:18:30.296 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 127/160 2023-07-13T22:18:30.624 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 128/160 2023-07-13T22:18:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-13T22:18:31.032 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/160 2023-07-13T22:18:31.240 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/160 2023-07-13T22:18:31.401 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/160 2023-07-13T22:18:31.588 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/160 2023-07-13T22:18:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 30/160 2023-07-13T22:18:31.862 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/160 2023-07-13T22:18:31.953 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 31/160 2023-07-13T22:18:32.183 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 130/160 2023-07-13T22:18:32.194 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 32/160 2023-07-13T22:18:32.409 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.189-2.el8.x86_64 131/160 2023-07-13T22:18:32.658 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-user-2.32.1-42.el8.x86_64 132/160 2023-07-13T22:18:32.833 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/160 2023-07-13T22:18:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 134/160 2023-07-13T22:18:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/160 2023-07-13T22:18:33.733 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/160 2023-07-13T22:18:34.040 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/160 2023-07-13T22:18:34.229 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-13T22:18:34.262 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-13T22:18:34.520 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-13T22:18:34.608 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-13T22:18:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 136/160 2023-07-13T22:18:35.111 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 137/160 2023-07-13T22:18:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 138/160 2023-07-13T22:18:35.709 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 139/160 2023-07-13T22:18:36.004 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 140/160 2023-07-13T22:18:36.185 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-13T22:18:36.215 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-13T22:18:36.413 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 142/160 2023-07-13T22:18:36.515 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-13T22:18:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/160 2023-07-13T22:18:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/160 2023-07-13T22:18:36.829 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/160 2023-07-13T22:18:37.024 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/160 2023-07-13T22:18:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/160 2023-07-13T22:18:37.087 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/160 2023-07-13T22:18:37.219 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/160 2023-07-13T22:18:37.279 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 145/160 2023-07-13T22:18:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 40/160 2023-07-13T22:18:37.476 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 146/160 2023-07-13T22:18:37.620 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 41/160 2023-07-13T22:18:37.640 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 147/160 2023-07-13T22:18:37.640 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-13T22:18:37.832 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libs-0.189-2.el8.x86_64 42/160 2023-07-13T22:18:37.956 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-13T22:18:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-13T22:18:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-debuginfod-client-0.189-2.el8.x86_64 43/160 2023-07-13T22:18:38.221 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-13T22:18:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-13T22:18:38.237 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-13T22:18:38.323 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-13T22:18:38.375 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-13T22:18:38.375 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-13T22:18:38.503 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libmount-2.32.1-42.el8.x86_64 45/160 2023-07-13T22:18:38.591 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libmount-2.32.1-42.el8.x86_64 45/160 2023-07-13T22:18:38.651 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-13T22:18:38.717 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.32.1-42.el8.x86_64 46/160 2023-07-13T22:18:38.862 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 152/160 2023-07-13T22:18:38.903 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-13T22:18:39.056 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 153/160 2023-07-13T22:18:39.278 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-13T22:18:39.279 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-13T22:18:39.521 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-13T22:18:39.522 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 156/160 2023-07-13T22:18:39.531 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-13T22:18:39.738 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 156/160 2023-07-13T22:18:39.739 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-13T22:18:39.770 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-13T22:18:39.800 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-13T22:18:39.971 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-13T22:18:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-13T22:18:39.996 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/160 2023-07-13T22:18:40.238 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-13T22:18:40.238 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-13T22:18:40.454 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-13T22:18:40.609 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/160 2023-07-13T22:18:40.652 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-13T22:18:40.796 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/160 2023-07-13T22:18:40.883 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-13T22:18:40.883 INFO:teuthology.orchestra.run.smithi105.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-13T22:18:40.884 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:18:40.951 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/160 2023-07-13T22:18:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/160 2023-07-13T22:18:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/160 2023-07-13T22:18:41.497 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/160 2023-07-13T22:18:41.603 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/160 2023-07-13T22:18:41.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/160 2023-07-13T22:18:41.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/160 2023-07-13T22:18:41.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/160 2023-07-13T22:18:41.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/160 2023-07-13T22:18:41.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/160 2023-07-13T22:18:41.604 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/160 2023-07-13T22:18:41.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/160 2023-07-13T22:18:41.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/160 2023-07-13T22:18:41.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/160 2023-07-13T22:18:41.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/160 2023-07-13T22:18:41.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/160 2023-07-13T22:18:41.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/160 2023-07-13T22:18:41.605 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/160 2023-07-13T22:18:41.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/160 2023-07-13T22:18:41.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/160 2023-07-13T22:18:41.606 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/160 2023-07-13T22:18:41.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/160 2023-07-13T22:18:41.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/160 2023-07-13T22:18:41.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/160 2023-07-13T22:18:41.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/160 2023-07-13T22:18:41.607 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/160 2023-07-13T22:18:41.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/160 2023-07-13T22:18:41.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/160 2023-07-13T22:18:41.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/160 2023-07-13T22:18:41.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/160 2023-07-13T22:18:41.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/160 2023-07-13T22:18:41.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/160 2023-07-13T22:18:41.608 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/160 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/160 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/160 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/160 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/160 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/160 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/160 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/160 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/160 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/160 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/160 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/160 2023-07-13T22:18:41.619 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/160 2023-07-13T22:18:41.619 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-7.el8.x86_64 42/160 2023-07-13T22:18:41.619 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/160 2023-07-13T22:18:41.619 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/160 2023-07-13T22:18:41.619 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/160 2023-07-13T22:18:41.619 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/160 2023-07-13T22:18:41.619 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/160 2023-07-13T22:18:41.620 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/160 2023-07-13T22:18:41.620 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/160 2023-07-13T22:18:41.620 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/160 2023-07-13T22:18:41.620 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/160 2023-07-13T22:18:41.620 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/160 2023-07-13T22:18:41.620 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/160 2023-07-13T22:18:41.620 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/160 2023-07-13T22:18:41.620 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/160 2023-07-13T22:18:41.621 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/160 2023-07-13T22:18:41.621 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/160 2023-07-13T22:18:41.621 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/160 2023-07-13T22:18:41.621 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/160 2023-07-13T22:18:41.621 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/160 2023-07-13T22:18:41.621 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/160 2023-07-13T22:18:41.621 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/160 2023-07-13T22:18:41.622 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/160 2023-07-13T22:18:41.622 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/160 2023-07-13T22:18:41.622 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/160 2023-07-13T22:18:41.622 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/160 2023-07-13T22:18:41.622 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/160 2023-07-13T22:18:41.622 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/160 2023-07-13T22:18:41.622 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/160 2023-07-13T22:18:41.622 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/160 2023-07-13T22:18:41.623 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.189-2.el8.x86_64 71/160 2023-07-13T22:18:41.623 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/160 2023-07-13T22:18:41.623 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/160 2023-07-13T22:18:41.623 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/160 2023-07-13T22:18:41.623 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/160 2023-07-13T22:18:41.623 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/160 2023-07-13T22:18:41.623 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 77/160 2023-07-13T22:18:41.623 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 78/160 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/160 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/160 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/160 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.32.1-42.el8.x86_64 82/160 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/160 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/160 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 85/160 2023-07-13T22:18:41.625 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/160 2023-07-13T22:18:41.625 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/160 2023-07-13T22:18:41.625 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/160 2023-07-13T22:18:41.625 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/160 2023-07-13T22:18:41.625 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/160 2023-07-13T22:18:41.625 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/160 2023-07-13T22:18:41.625 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/160 2023-07-13T22:18:41.626 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/160 2023-07-13T22:18:41.626 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/160 2023-07-13T22:18:41.626 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/160 2023-07-13T22:18:41.626 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/160 2023-07-13T22:18:41.626 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/160 2023-07-13T22:18:41.626 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/160 2023-07-13T22:18:41.626 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/160 2023-07-13T22:18:41.627 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/160 2023-07-13T22:18:41.627 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/160 2023-07-13T22:18:41.627 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/160 2023-07-13T22:18:41.627 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/160 2023-07-13T22:18:41.627 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/160 2023-07-13T22:18:41.627 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/160 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/160 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/160 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/160 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/160 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/160 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/160 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/160 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/160 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/160 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/160 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/160 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/160 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 118/160 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/160 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/160 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/160 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/160 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/160 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/160 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/160 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/160 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/160 2023-07-13T22:18:41.631 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/160 2023-07-13T22:18:41.631 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/160 2023-07-13T22:18:41.631 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/160 2023-07-13T22:18:41.631 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/160 2023-07-13T22:18:41.631 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/160 2023-07-13T22:18:41.631 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-8.el 133/160 2023-07-13T22:18:41.631 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/160 2023-07-13T22:18:41.631 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.189-2.el8.x86_64 135/160 2023-07-13T22:18:41.632 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 136/160 2023-07-13T22:18:41.632 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.189-2.el8.x86_64 137/160 2023-07-13T22:18:41.632 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 138/160 2023-07-13T22:18:41.632 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.189-2.el8.x86_64 139/160 2023-07-13T22:18:41.632 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 140/160 2023-07-13T22:18:41.632 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 141/160 2023-07-13T22:18:41.632 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 142/160 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.32.1-42.el8.x86_64 143/160 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 144/160 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.32.1-42.el8.x86_64 145/160 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 146/160 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.32.1-42.el8.x86_64 147/160 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 148/160 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.32.1-42.el8.x86_64 149/160 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 150/160 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.32.1-42.el8.x86_64 151/160 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 152/160 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.32.1-42.el8.x86_64 153/160 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.32.1-42.el8.x86_64 155/160 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/160 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 157/160 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 158/160 2023-07-13T22:18:41.635 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 159/160 2023-07-13T22:18:41.704 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/160 2023-07-13T22:18:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/160 2023-07-13T22:18:42.052 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/160 2023-07-13T22:18:42.220 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/160 2023-07-13T22:18:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/160 2023-07-13T22:18:42.565 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/160 2023-07-13T22:18:42.760 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/160 2023-07-13T22:18:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/160 2023-07-13T22:18:43.231 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/160 2023-07-13T22:18:43.430 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/160 2023-07-13T22:18:43.789 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/160 2023-07-13T22:18:43.804 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 160/160 2023-07-13T22:18:43.804 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:18:43.804 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2023-07-13T22:18:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-initscripts-updown-1:1.40.16-8.el8.noarch 2023-07-13T22:18:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.189-2.el8.x86_64 2023-07-13T22:18:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.189-2.el8.x86_64 2023-07-13T22:18:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.189-2.el8.x86_64 2023-07-13T22:18:43.805 INFO:teuthology.orchestra.run.smithi105.stdout: iproute-6.2.0-2.el8.x86_64 2023-07-13T22:18:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid-2.32.1-42.el8.x86_64 2023-07-13T22:18:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk-2.32.1-42.el8.x86_64 2023-07-13T22:18:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: libmount-2.32.1-42.el8.x86_64 2023-07-13T22:18:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols-2.32.1-42.el8.x86_64 2023-07-13T22:18:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-2.32.1-42.el8.x86_64 2023-07-13T22:18:43.806 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-2.32.1-42.el8.x86_64 2023-07-13T22:18:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user-2.32.1-42.el8.x86_64 2023-07-13T22:18:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-07-13T22:18:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-07-13T22:18:43.807 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-07-13T22:18:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.1-7.el8.noarch 2023-07-13T22:18:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.0.4-10.el8.x86_64 2023-07-13T22:18:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.808 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.809 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:18:43.810 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.189-2.el8.x86_64 2023-07-13T22:18:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.1-9.el8.x86_64 2023-07-13T22:18:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.7-17.el8.x86_64 2023-07-13T22:18:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-13T22:18:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-7.el8.x86_64 2023-07-13T22:18:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-13T22:18:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-07-13T22:18:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-13T22:18:43.811 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-07-13T22:18:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-07-13T22:18:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-13T22:18:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: libpq-13.5-1.el8.x86_64 2023-07-13T22:18:43.812 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-07-13T22:18:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-07-13T22:18:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-13T22:18:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-07-13T22:18:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-13T22:18:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-13T22:18:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-13T22:18:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.32.1-42.el8.x86_64 2023-07-13T22:18:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-07-13T22:18:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-07-13T22:18:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-13T22:18:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-13T22:18:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.15-4.el8.x86_64 2023-07-13T22:18:43.814 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-07-13T22:18:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-07-13T22:18:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-07-13T22:18:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-07-13T22:18:43.815 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-07-13T22:18:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-13T22:18:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-13T22:18:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-13T22:18:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-13T22:18:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-13T22:18:43.816 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-13T22:18:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-13T22:18:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-13T22:18:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-13T22:18:43.817 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-07-13T22:18:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-13T22:18:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-13T22:18:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-07-13T22:18:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-13T22:18:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-13T22:18:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-13T22:18:43.818 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-13T22:18:43.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-13T22:18:43.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-13T22:18:43.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-13T22:18:43.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-13T22:18:43.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-13T22:18:43.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.0.6-14.el8.noarch 2023-07-13T22:18:43.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-13T22:18:43.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-13T22:18:43.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-13T22:18:43.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-13T22:18:43.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-13T22:18:43.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-13T22:18:43.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-13T22:18:43.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-2.6-1.el8.noarch 2023-07-13T22:18:43.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-13T22:18:43.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-13T22:18:43.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-13T22:18:43.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-13T22:18:43.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-13T22:18:43.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-13T22:18:43.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-13T22:18:43.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-13T22:18:43.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-13T22:18:43.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-13T22:18:43.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-07-13T22:18:43.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-13T22:18:43.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-13T22:18:43.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-13T22:18:43.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-13T22:18:43.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-13T22:18:43.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-13T22:18:43.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-13T22:18:43.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-13T22:18:43.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-13T22:18:43.824 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-13T22:18:43.824 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.824 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.824 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:18:43.824 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-13T22:18:43.824 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-13T22:18:43.824 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-13T22:18:43.825 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-07-13T22:18:43.825 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-13T22:18:43.825 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-07-13T22:18:43.825 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:18:43.826 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-07-13T22:18:44.072 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/160 2023-07-13T22:18:44.106 DEBUG:teuthology.parallel:result is None 2023-07-13T22:18:44.299 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/160 2023-07-13T22:18:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/160 2023-07-13T22:18:45.460 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 70/160 2023-07-13T22:18:45.536 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 70/160 2023-07-13T22:18:45.749 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 71/160 2023-07-13T22:18:45.968 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 72/160 2023-07-13T22:18:46.376 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 73/160 2023-07-13T22:18:46.645 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 74/160 2023-07-13T22:18:46.819 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 75/160 2023-07-13T22:18:47.090 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 76/160 2023-07-13T22:18:47.306 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 77/160 2023-07-13T22:18:48.676 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/160 2023-07-13T22:18:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/160 2023-07-13T22:18:49.262 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 80/160 2023-07-13T22:18:49.511 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 81/160 2023-07-13T22:18:49.821 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 82/160 2023-07-13T22:18:50.079 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/160 2023-07-13T22:18:50.341 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 84/160 2023-07-13T22:18:50.584 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 85/160 2023-07-13T22:18:50.946 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 86/160 2023-07-13T22:18:51.158 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 87/160 2023-07-13T22:18:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 88/160 2023-07-13T22:18:51.499 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/160 2023-07-13T22:18:51.892 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/160 2023-07-13T22:18:52.106 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 91/160 2023-07-13T22:18:52.243 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/160 2023-07-13T22:18:52.431 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/160 2023-07-13T22:18:52.605 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/160 2023-07-13T22:18:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.7-17.el8.x86_64 95/160 2023-07-13T22:18:53.022 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/160 2023-07-13T22:18:53.176 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/160 2023-07-13T22:18:53.290 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/160 2023-07-13T22:18:53.453 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 98/160 2023-07-13T22:18:53.576 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 99/160 2023-07-13T22:18:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/160 2023-07-13T22:18:56.884 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/160 2023-07-13T22:18:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/160 2023-07-13T22:18:57.888 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 101/160 2023-07-13T22:18:58.133 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 101/160 2023-07-13T22:18:58.300 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 102/160 2023-07-13T22:19:19.865 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 102/160 2023-07-13T22:19:19.865 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2023-07-13T22:19:19.866 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2023-07-13T22:19:19.866 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /dev 2023-07-13T22:19:19.866 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /run 2023-07-13T22:19:19.866 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2023-07-13T22:19:19.866 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2023-07-13T22:19:19.867 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2023-07-13T22:19:19.867 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2023-07-13T22:19:19.867 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /dev 2023-07-13T22:19:19.867 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 103/160 2023-07-13T22:19:22.129 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 103/160 2023-07-13T22:19:22.360 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 104/160 2023-07-13T22:19:22.643 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 104/160 2023-07-13T22:19:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 105/160 2023-07-13T22:19:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 105/160 2023-07-13T22:19:23.622 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 106/160 2023-07-13T22:19:23.674 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 106/160 2023-07-13T22:19:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 107/160 2023-07-13T22:19:24.070 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 108/160 2023-07-13T22:19:24.188 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-13T22:19:24.270 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-13T22:19:24.543 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : jq-1.6-7.el8.x86_64 110/160 2023-07-13T22:19:24.755 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.1-7.el8.noarch 111/160 2023-07-13T22:19:25.424 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 112/160 2023-07-13T22:19:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 112/160 2023-07-13T22:19:25.745 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 113/160 2023-07-13T22:19:25.957 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 114/160 2023-07-13T22:19:26.155 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 115/160 2023-07-13T22:19:26.432 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 115/160 2023-07-13T22:19:26.692 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 116/160 2023-07-13T22:19:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 116/160 2023-07-13T22:19:28.046 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 117/160 2023-07-13T22:19:28.184 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 117/160 2023-07-13T22:19:28.685 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 118/160 2023-07-13T22:19:28.875 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/160 2023-07-13T22:19:28.943 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/160 2023-07-13T22:19:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 120/160 2023-07-13T22:19:29.191 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 120/160 2023-07-13T22:19:29.390 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 121/160 2023-07-13T22:19:29.520 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-25.el8.x86_64 122/160 2023-07-13T22:19:34.859 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 122/160 2023-07-13T22:19:36.466 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 123/160 2023-07-13T22:19:36.683 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 124/160 2023-07-13T22:19:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 124/160 2023-07-13T22:19:36.937 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 125/160 2023-07-13T22:19:37.010 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 125/160 2023-07-13T22:19:37.171 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/160 2023-07-13T22:19:37.603 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/160 2023-07-13T22:19:37.803 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 127/160 2023-07-13T22:19:38.113 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 127/160 2023-07-13T22:19:38.438 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 128/160 2023-07-13T22:19:39.792 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/160 2023-07-13T22:19:40.118 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 130/160 2023-07-13T22:19:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.189-2.el8.x86_64 131/160 2023-07-13T22:19:40.518 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : util-linux-user-2.32.1-42.el8.x86_64 132/160 2023-07-13T22:19:40.709 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/160 2023-07-13T22:19:40.887 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 134/160 2023-07-13T22:19:42.156 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-13T22:19:42.386 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-13T22:19:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-13T22:19:42.691 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 136/160 2023-07-13T22:19:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 137/160 2023-07-13T22:19:43.163 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 138/160 2023-07-13T22:19:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 139/160 2023-07-13T22:19:43.813 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 140/160 2023-07-13T22:19:43.960 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-13T22:19:43.987 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-13T22:19:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 142/160 2023-07-13T22:19:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/160 2023-07-13T22:19:44.530 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/160 2023-07-13T22:19:44.752 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/160 2023-07-13T22:19:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/160 2023-07-13T22:19:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 145/160 2023-07-13T22:19:45.150 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 146/160 2023-07-13T22:19:45.291 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 147/160 2023-07-13T22:19:45.291 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-13T22:19:45.549 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-13T22:19:45.549 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-13T22:19:45.839 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-13T22:19:45.856 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-13T22:19:45.967 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-13T22:19:45.967 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-13T22:19:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-13T22:19:46.413 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 152/160 2023-07-13T22:19:46.570 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 153/160 2023-07-13T22:19:46.779 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-13T22:19:46.779 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-13T22:19:46.997 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-13T22:19:46.998 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 156/160 2023-07-13T22:19:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 156/160 2023-07-13T22:19:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-13T22:19:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-13T22:19:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-13T22:19:47.672 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-13T22:19:47.672 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-13T22:19:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-13T22:19:48.087 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-13T22:19:48.284 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-13T22:19:48.284 INFO:teuthology.orchestra.run.smithi060.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-13T22:19:48.284 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/160 2023-07-13T22:19:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/160 2023-07-13T22:19:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/160 2023-07-13T22:19:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/160 2023-07-13T22:19:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/160 2023-07-13T22:19:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/160 2023-07-13T22:19:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/160 2023-07-13T22:19:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/160 2023-07-13T22:19:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/160 2023-07-13T22:19:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/160 2023-07-13T22:19:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/160 2023-07-13T22:19:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/160 2023-07-13T22:19:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/160 2023-07-13T22:19:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/160 2023-07-13T22:19:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/160 2023-07-13T22:19:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/160 2023-07-13T22:19:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/160 2023-07-13T22:19:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/160 2023-07-13T22:19:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/160 2023-07-13T22:19:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/160 2023-07-13T22:19:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/160 2023-07-13T22:19:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/160 2023-07-13T22:19:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/160 2023-07-13T22:19:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/160 2023-07-13T22:19:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/160 2023-07-13T22:19:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/160 2023-07-13T22:19:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/160 2023-07-13T22:19:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/160 2023-07-13T22:19:49.027 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/160 2023-07-13T22:19:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/160 2023-07-13T22:19:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/160 2023-07-13T22:19:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/160 2023-07-13T22:19:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/160 2023-07-13T22:19:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/160 2023-07-13T22:19:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/160 2023-07-13T22:19:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/160 2023-07-13T22:19:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/160 2023-07-13T22:19:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/160 2023-07-13T22:19:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/160 2023-07-13T22:19:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/160 2023-07-13T22:19:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/160 2023-07-13T22:19:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : jq-1.6-7.el8.x86_64 42/160 2023-07-13T22:19:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/160 2023-07-13T22:19:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/160 2023-07-13T22:19:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/160 2023-07-13T22:19:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/160 2023-07-13T22:19:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/160 2023-07-13T22:19:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/160 2023-07-13T22:19:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/160 2023-07-13T22:19:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/160 2023-07-13T22:19:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/160 2023-07-13T22:19:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/160 2023-07-13T22:19:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/160 2023-07-13T22:19:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/160 2023-07-13T22:19:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/160 2023-07-13T22:19:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/160 2023-07-13T22:19:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/160 2023-07-13T22:19:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/160 2023-07-13T22:19:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/160 2023-07-13T22:19:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/160 2023-07-13T22:19:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/160 2023-07-13T22:19:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/160 2023-07-13T22:19:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/160 2023-07-13T22:19:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/160 2023-07-13T22:19:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/160 2023-07-13T22:19:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/160 2023-07-13T22:19:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/160 2023-07-13T22:19:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/160 2023-07-13T22:19:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/160 2023-07-13T22:19:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/160 2023-07-13T22:19:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.189-2.el8.x86_64 71/160 2023-07-13T22:19:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/160 2023-07-13T22:19:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/160 2023-07-13T22:19:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/160 2023-07-13T22:19:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/160 2023-07-13T22:19:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/160 2023-07-13T22:19:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 77/160 2023-07-13T22:19:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 78/160 2023-07-13T22:19:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/160 2023-07-13T22:19:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/160 2023-07-13T22:19:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/160 2023-07-13T22:19:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.32.1-42.el8.x86_64 82/160 2023-07-13T22:19:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/160 2023-07-13T22:19:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/160 2023-07-13T22:19:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 85/160 2023-07-13T22:19:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/160 2023-07-13T22:19:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/160 2023-07-13T22:19:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/160 2023-07-13T22:19:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/160 2023-07-13T22:19:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/160 2023-07-13T22:19:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/160 2023-07-13T22:19:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/160 2023-07-13T22:19:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/160 2023-07-13T22:19:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/160 2023-07-13T22:19:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/160 2023-07-13T22:19:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/160 2023-07-13T22:19:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/160 2023-07-13T22:19:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/160 2023-07-13T22:19:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/160 2023-07-13T22:19:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/160 2023-07-13T22:19:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/160 2023-07-13T22:19:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/160 2023-07-13T22:19:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/160 2023-07-13T22:19:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/160 2023-07-13T22:19:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/160 2023-07-13T22:19:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/160 2023-07-13T22:19:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/160 2023-07-13T22:19:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/160 2023-07-13T22:19:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/160 2023-07-13T22:19:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/160 2023-07-13T22:19:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/160 2023-07-13T22:19:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/160 2023-07-13T22:19:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/160 2023-07-13T22:19:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/160 2023-07-13T22:19:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/160 2023-07-13T22:19:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/160 2023-07-13T22:19:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/160 2023-07-13T22:19:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 118/160 2023-07-13T22:19:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/160 2023-07-13T22:19:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/160 2023-07-13T22:19:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/160 2023-07-13T22:19:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/160 2023-07-13T22:19:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/160 2023-07-13T22:19:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/160 2023-07-13T22:19:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/160 2023-07-13T22:19:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/160 2023-07-13T22:19:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/160 2023-07-13T22:19:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/160 2023-07-13T22:19:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/160 2023-07-13T22:19:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/160 2023-07-13T22:19:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/160 2023-07-13T22:19:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/160 2023-07-13T22:19:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-8.el 133/160 2023-07-13T22:19:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/160 2023-07-13T22:19:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.189-2.el8.x86_64 135/160 2023-07-13T22:19:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 136/160 2023-07-13T22:19:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.189-2.el8.x86_64 137/160 2023-07-13T22:19:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 138/160 2023-07-13T22:19:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.189-2.el8.x86_64 139/160 2023-07-13T22:19:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 140/160 2023-07-13T22:19:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 141/160 2023-07-13T22:19:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 142/160 2023-07-13T22:19:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libblkid-2.32.1-42.el8.x86_64 143/160 2023-07-13T22:19:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 144/160 2023-07-13T22:19:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libfdisk-2.32.1-42.el8.x86_64 145/160 2023-07-13T22:19:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 146/160 2023-07-13T22:19:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libmount-2.32.1-42.el8.x86_64 147/160 2023-07-13T22:19:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 148/160 2023-07-13T22:19:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libsmartcols-2.32.1-42.el8.x86_64 149/160 2023-07-13T22:19:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 150/160 2023-07-13T22:19:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-2.32.1-42.el8.x86_64 151/160 2023-07-13T22:19:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 152/160 2023-07-13T22:19:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : util-linux-2.32.1-42.el8.x86_64 153/160 2023-07-13T22:19:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-13T22:19:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : util-linux-user-2.32.1-42.el8.x86_64 155/160 2023-07-13T22:19:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/160 2023-07-13T22:19:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 157/160 2023-07-13T22:19:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 158/160 2023-07-13T22:19:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 159/160 2023-07-13T22:19:51.150 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 160/160 2023-07-13T22:19:51.151 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:51.151 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2023-07-13T22:19:51.151 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-initscripts-updown-1:1.40.16-8.el8.noarch 2023-07-13T22:19:51.151 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client-0.189-2.el8.x86_64 2023-07-13T22:19:51.151 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-0.189-2.el8.x86_64 2023-07-13T22:19:51.151 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs-0.189-2.el8.x86_64 2023-07-13T22:19:51.151 INFO:teuthology.orchestra.run.smithi060.stdout: iproute-6.2.0-2.el8.x86_64 2023-07-13T22:19:51.152 INFO:teuthology.orchestra.run.smithi060.stdout: libblkid-2.32.1-42.el8.x86_64 2023-07-13T22:19:51.152 INFO:teuthology.orchestra.run.smithi060.stdout: libfdisk-2.32.1-42.el8.x86_64 2023-07-13T22:19:51.152 INFO:teuthology.orchestra.run.smithi060.stdout: libmount-2.32.1-42.el8.x86_64 2023-07-13T22:19:51.152 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.152 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.152 INFO:teuthology.orchestra.run.smithi060.stdout: libsmartcols-2.32.1-42.el8.x86_64 2023-07-13T22:19:51.152 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-2.32.1-42.el8.x86_64 2023-07-13T22:19:51.152 INFO:teuthology.orchestra.run.smithi060.stdout: util-linux-2.32.1-42.el8.x86_64 2023-07-13T22:19:51.153 INFO:teuthology.orchestra.run.smithi060.stdout: util-linux-user-2.32.1-42.el8.x86_64 2023-07-13T22:19:51.153 INFO:teuthology.orchestra.run.smithi060.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-07-13T22:19:51.153 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-07-13T22:19:51.153 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-07-13T22:19:51.153 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.1-7.el8.noarch 2023-07-13T22:19:51.153 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.0.4-10.el8.x86_64 2023-07-13T22:19:51.153 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.154 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.154 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.154 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.154 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.154 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.154 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.154 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.154 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.155 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.155 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.155 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.155 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.155 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.155 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.155 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.155 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.156 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.156 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.156 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.156 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.156 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.156 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:19:51.156 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.189-2.el8.x86_64 2023-07-13T22:19:51.157 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.1-9.el8.x86_64 2023-07-13T22:19:51.157 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.7-17.el8.x86_64 2023-07-13T22:19:51.157 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-13T22:19:51.157 INFO:teuthology.orchestra.run.smithi060.stdout: jq-1.6-7.el8.x86_64 2023-07-13T22:19:51.157 INFO:teuthology.orchestra.run.smithi060.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-13T22:19:51.157 INFO:teuthology.orchestra.run.smithi060.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-07-13T22:19:51.157 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-13T22:19:51.157 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.158 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.158 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.158 INFO:teuthology.orchestra.run.smithi060.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-07-13T22:19:51.158 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-07-13T22:19:51.158 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-13T22:19:51.158 INFO:teuthology.orchestra.run.smithi060.stdout: libpq-13.5-1.el8.x86_64 2023-07-13T22:19:51.158 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-07-13T22:19:51.158 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-07-13T22:19:51.159 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.159 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.159 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-13T22:19:51.159 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.159 INFO:teuthology.orchestra.run.smithi060.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-07-13T22:19:51.159 INFO:teuthology.orchestra.run.smithi060.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-13T22:19:51.159 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-13T22:19:51.159 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-13T22:19:51.160 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.32.1-42.el8.x86_64 2023-07-13T22:19:51.160 INFO:teuthology.orchestra.run.smithi060.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-07-13T22:19:51.160 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-07-13T22:19:51.160 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-13T22:19:51.160 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-13T22:19:51.160 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.15-4.el8.x86_64 2023-07-13T22:19:51.160 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-07-13T22:19:51.160 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-07-13T22:19:51.161 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-07-13T22:19:51.161 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-07-13T22:19:51.161 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-07-13T22:19:51.161 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-13T22:19:51.161 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-13T22:19:51.161 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-13T22:19:51.161 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-13T22:19:51.161 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-13T22:19:51.162 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-13T22:19:51.162 INFO:teuthology.orchestra.run.smithi060.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-13T22:19:51.162 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-13T22:19:51.162 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.162 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.162 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.162 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-13T22:19:51.163 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-07-13T22:19:51.163 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-13T22:19:51.163 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-13T22:19:51.163 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-07-13T22:19:51.163 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-13T22:19:51.163 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-13T22:19:51.163 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-13T22:19:51.163 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-13T22:19:51.164 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-13T22:19:51.164 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-13T22:19:51.164 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-13T22:19:51.164 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-13T22:19:51.164 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-13T22:19:51.164 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.0.6-14.el8.noarch 2023-07-13T22:19:51.164 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-13T22:19:51.164 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-13T22:19:51.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-13T22:19:51.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-13T22:19:51.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-13T22:19:51.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-13T22:19:51.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-13T22:19:51.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-2.6-1.el8.noarch 2023-07-13T22:19:51.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-13T22:19:51.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-13T22:19:51.166 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-13T22:19:51.166 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-13T22:19:51.166 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-13T22:19:51.166 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-13T22:19:51.166 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.166 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.166 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-13T22:19:51.166 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-13T22:19:51.167 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.167 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-13T22:19:51.167 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-13T22:19:51.167 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-07-13T22:19:51.167 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-13T22:19:51.167 INFO:teuthology.orchestra.run.smithi060.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-13T22:19:51.167 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-13T22:19:51.168 INFO:teuthology.orchestra.run.smithi060.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-13T22:19:51.168 INFO:teuthology.orchestra.run.smithi060.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-13T22:19:51.168 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-13T22:19:51.168 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-13T22:19:51.168 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-13T22:19:51.168 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-13T22:19:51.168 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-13T22:19:51.168 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.169 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.169 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:19:51.169 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-13T22:19:51.169 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-13T22:19:51.169 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-13T22:19:51.169 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-07-13T22:19:51.169 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-13T22:19:51.170 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-07-13T22:19:51.170 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:51.170 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-07-13T22:19:51.480 DEBUG:teuthology.parallel:result is None 2023-07-13T22:19:51.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:19:51.850 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-13T22:19:51.906 INFO:teuthology.orchestra.run.smithi060.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-13T22:19:51.907 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-13T22:19:51.907 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-13T22:19:51.908 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:19:52.252 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-13T22:19:52.313 INFO:teuthology.orchestra.run.smithi105.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-13T22:19:52.313 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-13T22:19:52.313 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-13T22:19:52.314 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-13T22:19:52.314 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:52.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-13T22:19:52.390 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:19:52.391 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-13T22:19:52.466 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-13T22:19:52.467 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:52.467 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2023-07-13T22:19:52.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-13T22:19:52.652 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:19:52.652 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2023-07-13T22:19:52.722 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-13T22:19:52.831 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-13T22:19:52.832 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:52.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-13T22:19:52.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-13T22:19:53.016 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:19:53.017 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-13T22:19:53.085 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-13T22:19:53.193 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-13T22:19:53.193 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:53.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2023-07-13T22:19:53.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-13T22:19:53.379 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:19:53.379 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2023-07-13T22:19:53.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-13T22:19:53.555 INFO:teuthology.run_tasks:Running task ceph... 2023-07-13T22:19:53.628 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-13T22:19:53.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /var/log/ceph 2023-07-13T22:19:53.633 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2023-07-13T22:19:53.702 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-13T22:19:53.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-13T22:19:53.706 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-13T22:19:53.775 INFO:tasks.ceph:Creating extra log directories... 2023-07-13T22:19:53.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-13T22:19:53.780 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-13T22:19:53.857 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-07-13T22:19:53.857 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-07-13T22:19:53.858 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-07-13T22:19:53.858 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:53.858 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-13T22:19:53.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-13T22:19:54.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-13T22:19:54.198 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:19:54.199 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-13T22:19:54.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-13T22:19:54.466 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-13T22:19:54.539 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-13T22:19:54.540 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-13T22:19:54.541 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271', 'branch': 'wip-rishabh-2023Jul13', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}', 'email': None, 'first_in_suite': False, 'job_id': '7336271', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023Jul13'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023Jul13', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023Jul13', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi060.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8nVQIrQ7b3w8ZPuA5eRJy7ykeR6z2+B/1lmJQu3oCFhge/KRmZhTWueiqzcfJEpS3NJkyU0jvPF/5kuU+lvcWyfsI1+Z53LxbyncMhk8AZiJXJ5RVsxpymZZAkn39Ab+fnO8vGyx50dH5XAGBR5OWqruHYPpcWw4bH3ddIvGOpXjxR7IKEKAULhkf9YZ/j11O4uaHiN9Xzv+K5N/nMdk4kxCPS21Gtst3U0KGWrR+wC2WNSJVB7I/LL48GAX4X3Nlh4gvr3jtEKEvpNCduUi8vnEBDc3xhb+jOkMZtdZB96z0ZxT/LmBZvyDQ3Zr9eigrYDz7EzbAgOMRaaHMonPY3xrJM5LuQUzjFbga2yPTZDVkBATKlxGmlAxDgtuuKA9fYpsaXvMn7vpjK/jlReKedb2lS7heNL4EY1ZebMxhAudvqljlLgQeQMBRhZ4W+lShI8nssgvHLsBKf5IVmlcmMbCTAFnNOcw4cVZNECQrwVW6TbZ6V0YJ6WN+9SLkpG0=', 'smithi105.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLA2TVUdkdWf/GTZ+fWrjN14P+IxbNz6W3xXXjoshJQIYoB4jbdyPIdkjNaHeN8ANEoVUFDLZVy4k5cCqpwxTkEs2TZviwA0NyjGu+J660iXZM9f3izPw766fe8IE1duDu3xe0oyFZY30eJeh2CooQVeT11iLbr5ESZRGKlWRcnARsYAg52je2Rbcuus70/KQ5O07M+Y9lP4BlHkqg5/ycHSodgSe1TAr63oA2FZOsM5lYpsc+QdBIU4D4oOM48NBGuX0yzP7Fzga9QSKEYlzF1A5GWo4jmvj0EZygyx+hiRQEUjI7MlB+nTMM8qGVEB6ID30Xid9gCT8Mi7NhIgAvc5U6JH3Ey4wzGaV3x7vrz1bFvQhgK2xwR1ITNfQSafEOzr7fhI/mWcUzlUX/ouH8AbjkAtNYgO18ez7fzgIEtA+01SBhR6Q/rdDHSDA9Wb0dMzrnFEkLdcXjqF5fswqeTR7rA1JXG4jX9Cb2OXd6iSUqiNWtznHs4eI1bmBf2/E='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '073e6eb1231451146b4a01d0b85b4f490f71d266', 'timestamp': '2023-07-13_21:35:13', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-13T22:19:54.542 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-13T22:19:54.546 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-13T22:19:54.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/run/ceph 2023-07-13T22:19:54.596 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2023-07-13T22:19:54.700 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:54.701 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2023-07-13T22:19:54.748 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-13T22:19:54.749 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2023-07-13T22:19:54.834 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-13T22:19:54.834 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:19:54.835 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 442 Links: 1 2023-07-13T22:19:54.835 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:19:54.835 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:19:54.835 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-07-13 22:19:48.510825734 +0000 2023-07-13T22:19:54.835 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-07-13 22:06:50.350476990 +0000 2023-07-13T22:19:54.836 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-07-13 22:06:50.350476990 +0000 2023-07-13T22:19:54.836 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-07-13 22:06:50.329477194 +0000 2023-07-13T22:19:54.836 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-13T22:19:54.906 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-07-13T22:19:54.906 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-07-13T22:19:54.906 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000206261 s, 2.5 MB/s 2023-07-13T22:19:54.908 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-13T22:19:54.957 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2023-07-13T22:19:55.005 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-13T22:19:55.005 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:19:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-07-13T22:19:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:19:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:19:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-07-13 22:19:48.510825734 +0000 2023-07-13T22:19:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-07-13 22:06:51.225468482 +0000 2023-07-13T22:19:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-07-13 22:06:51.225468482 +0000 2023-07-13T22:19:55.006 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-07-13 22:06:51.198468745 +0000 2023-07-13T22:19:55.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-13T22:19:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-07-13T22:19:55.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-07-13T22:19:55.077 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.00016071 s, 3.2 MB/s 2023-07-13T22:19:55.079 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-13T22:19:55.128 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2023-07-13T22:19:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-13T22:19:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:19:55.177 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-07-13T22:19:55.177 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:19:55.177 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:19:55.177 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-07-13 22:19:48.510825734 +0000 2023-07-13T22:19:55.177 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-07-13 22:06:52.102459954 +0000 2023-07-13T22:19:55.178 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-07-13 22:06:52.102459954 +0000 2023-07-13T22:19:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-07-13 22:06:52.080460168 +0000 2023-07-13T22:19:55.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-13T22:19:55.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-07-13T22:19:55.247 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-07-13T22:19:55.247 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000156969 s, 3.3 MB/s 2023-07-13T22:19:55.250 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-13T22:19:55.301 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2023-07-13T22:19:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-13T22:19:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:19:55.351 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-07-13T22:19:55.351 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:19:55.351 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:19:55.351 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-07-13 22:19:48.511825725 +0000 2023-07-13T22:19:55.352 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-07-13 22:06:52.985451368 +0000 2023-07-13T22:19:55.352 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-07-13 22:06:52.985451368 +0000 2023-07-13T22:19:55.352 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-07-13 22:06:52.966451553 +0000 2023-07-13T22:19:55.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-13T22:19:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-07-13T22:19:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-07-13T22:19:55.422 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.00016218 s, 3.2 MB/s 2023-07-13T22:19:55.424 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-13T22:19:55.475 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-13T22:19:55.475 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:19:55.476 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2023-07-13T22:19:55.525 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-13T22:19:55.525 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2023-07-13T22:19:55.572 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-13T22:19:55.572 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:19:55.572 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 442 Links: 1 2023-07-13T22:19:55.573 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:19:55.573 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:19:55.573 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-07-13 22:18:41.105318885 +0000 2023-07-13T22:19:55.573 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-07-13 22:06:40.283339156 +0000 2023-07-13T22:19:55.573 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-07-13 22:06:40.283339156 +0000 2023-07-13T22:19:55.573 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2023-07-13 22:06:40.259339643 +0000 2023-07-13T22:19:55.574 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-13T22:19:55.640 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-07-13T22:19:55.640 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-07-13T22:19:55.640 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000294948 s, 1.7 MB/s 2023-07-13T22:19:55.642 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-13T22:19:55.690 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2023-07-13T22:19:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-13T22:19:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:19:55.737 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-07-13T22:19:55.737 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:19:55.737 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:19:55.737 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-07-13 22:18:41.105318885 +0000 2023-07-13T22:19:55.737 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-07-13 22:06:41.119322183 +0000 2023-07-13T22:19:55.737 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-07-13 22:06:41.119322183 +0000 2023-07-13T22:19:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2023-07-13 22:06:41.096322650 +0000 2023-07-13T22:19:55.738 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-13T22:19:55.842 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-07-13T22:19:55.842 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-07-13T22:19:55.842 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000290864 s, 1.8 MB/s 2023-07-13T22:19:55.844 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-13T22:19:55.892 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2023-07-13T22:19:55.938 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-13T22:19:55.938 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:19:55.938 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-07-13T22:19:55.938 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:19:55.938 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:19:55.938 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-07-13 22:18:41.105318885 +0000 2023-07-13T22:19:55.939 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-07-13 22:06:42.034303606 +0000 2023-07-13T22:19:55.939 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-07-13 22:06:42.034303606 +0000 2023-07-13T22:19:55.939 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2023-07-13 22:06:42.006304174 +0000 2023-07-13T22:19:55.939 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-13T22:19:56.043 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-07-13T22:19:56.043 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-07-13T22:19:56.043 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000284994 s, 1.8 MB/s 2023-07-13T22:19:56.046 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-13T22:19:56.093 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2023-07-13T22:19:56.139 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-13T22:19:56.139 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:19:56.139 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-07-13T22:19:56.140 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:19:56.140 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:19:56.140 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-07-13 22:18:41.105318885 +0000 2023-07-13T22:19:56.140 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-07-13 22:06:42.878286470 +0000 2023-07-13T22:19:56.140 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-07-13 22:06:42.878286470 +0000 2023-07-13T22:19:56.140 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2023-07-13 22:06:42.853286978 +0000 2023-07-13T22:19:56.140 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-13T22:19:56.245 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-07-13T22:19:56.245 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-07-13T22:19:56.245 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000263971 s, 1.9 MB/s 2023-07-13T22:19:56.247 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-13T22:19:56.294 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-13T22:19:56.294 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-13T22:19:56.295 INFO:tasks.ceph:Generating config... 2023-07-13T22:19:56.296 INFO:tasks.ceph:[client] client mount timeout = 600 2023-07-13T22:19:56.297 INFO:tasks.ceph:[client] debug client = 20 2023-07-13T22:19:56.297 INFO:tasks.ceph:[client] debug ms = 1 2023-07-13T22:19:56.297 INFO:tasks.ceph:[client] pid file = 2023-07-13T22:19:56.297 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-07-13T22:19:56.297 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-07-13T22:19:56.298 INFO:tasks.ceph:[mds] debug mds = 20 2023-07-13T22:19:56.298 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-07-13T22:19:56.298 INFO:tasks.ceph:[mds] debug ms = 1 2023-07-13T22:19:56.298 INFO:tasks.ceph:[mds] mds debug frag = True 2023-07-13T22:19:56.298 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-07-13T22:19:56.298 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-07-13T22:19:56.299 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-07-13T22:19:56.299 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-07-13T22:19:56.299 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-07-13T22:19:56.299 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-07-13T22:19:56.299 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-13T22:19:56.300 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-13T22:19:56.300 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-13T22:19:56.300 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-13T22:19:56.300 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-13T22:19:56.300 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-07-13T22:19:56.301 INFO:tasks.ceph:[osd] bdev async discard = True 2023-07-13T22:19:56.301 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-07-13T22:19:56.301 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-07-13T22:19:56.301 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-13T22:19:56.301 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-13T22:19:56.302 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-07-13T22:19:56.302 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-07-13T22:19:56.302 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-13T22:19:56.302 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-13T22:19:56.302 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-07-13T22:19:56.302 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-13T22:19:56.303 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-13T22:19:56.303 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-13T22:19:56.303 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-13T22:19:56.303 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-13T22:19:56.303 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-07-13T22:19:56.304 INFO:tasks.ceph:Setting up mon.a... 2023-07-13T22:19:56.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-13T22:19:56.422 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.keyring 2023-07-13T22:19:56.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-13T22:19:56.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-13T22:19:56.608 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.60'), ('mon.b', '172.21.15.105'), ('mon.c', '[v2:172.21.15.105:3301,v1:172.21.15.105:6790]')] 2023-07-13T22:19:56.608 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-07-13T22:19:56.609 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:56.610 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-13T22:19:56.658 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.60 --add b 172.21.15.105 --addv c '[v2:172.21.15.105:3301,v1:172.21.15.105:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:19:56.754 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:19:56.754 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: generated fsid fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:19:56.754 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = pacific 2023-07-13T22:19:56.754 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 0 2023-07-13T22:19:56.755 INFO:teuthology.orchestra.run.smithi060.stdout:fsid fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:19:56.755 INFO:teuthology.orchestra.run.smithi060.stdout:last_changed 2023-07-13T22:19:56.769714+0000 2023-07-13T22:19:56.755 INFO:teuthology.orchestra.run.smithi060.stdout:created 2023-07-13T22:19:56.769714+0000 2023-07-13T22:19:56.755 INFO:teuthology.orchestra.run.smithi060.stdout:min_mon_release 16 (pacific) 2023-07-13T22:19:56.755 INFO:teuthology.orchestra.run.smithi060.stdout:election_strategy: 1 2023-07-13T22:19:56.755 INFO:teuthology.orchestra.run.smithi060.stdout:0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2023-07-13T22:19:56.755 INFO:teuthology.orchestra.run.smithi060.stdout:1: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2023-07-13T22:19:56.755 INFO:teuthology.orchestra.run.smithi060.stdout:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.c 2023-07-13T22:19:56.756 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-13T22:19:56.757 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-13T22:19:56.806 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce... 2023-07-13T22:19:56.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:19:56.812 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:19:56.929 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-07-13T22:19:56.929 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-07-13T22:19:56.929 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:19:56.929 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-07-13T22:19:56.929 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.929 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-07-13T22:19:56.930 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.930 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-07-13T22:19:56.930 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.930 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-07-13T22:19:56.930 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-07-13T22:19:56.930 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.930 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-07-13T22:19:56.930 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-07-13T22:19:56.931 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.931 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:19:56.931 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-07-13T22:19:56.931 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.931 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:19:56.931 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:19:56.931 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.932 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-07-13T22:19:56.932 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.932 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-07-13T22:19:56.932 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-07-13T22:19:56.932 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:19:56.932 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:19:56.932 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-07-13T22:19:56.932 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:19:56.933 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-07-13T22:19:56.933 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:19:56.933 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:19:56.933 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-07-13T22:19:56.933 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.933 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:19:56.933 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.934 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-07-13T22:19:56.934 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.934 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-07-13T22:19:56.934 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.934 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-07-13T22:19:56.934 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-07-13T22:19:56.934 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-07-13T22:19:56.934 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:19:56.935 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:19:56.935 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.935 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-07-13T22:19:56.935 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-07-13T22:19:56.935 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.935 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-07-13T22:19:56.935 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2023-07-13T22:19:56.935 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:19:56.936 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.936 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-07-13T22:19:56.936 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-07-13T22:19:56.936 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.936 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-07-13T22:19:56.936 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-07-13T22:19:56.936 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-07-13T22:19:56.937 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.937 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-07-13T22:19:56.937 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-07-13T22:19:56.937 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.937 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:19:56.937 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.937 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-07-13T22:19:56.937 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.938 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-07-13T22:19:56.938 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.938 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:19:56.938 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-07-13T22:19:56.938 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.938 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-07-13T22:19:56.938 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-07-13T22:19:56.938 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2023-07-13T22:19:56.939 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2023-07-13T22:19:56.939 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2023-07-13T22:19:56.939 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2023-07-13T22:19:56.939 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2023-07-13T22:19:56.939 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2023-07-13T22:19:56.939 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2023-07-13T22:19:56.939 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:19:56.939 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-07-13T22:19:56.940 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2023-07-13T22:19:56.940 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:19:56.940 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2023-07-13T22:19:56.940 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:19:56.940 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:19:56.940 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2023-07-13T22:19:56.940 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:19:56.941 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.941 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-07-13T22:19:56.941 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:19:56.941 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-07-13T22:19:56.941 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:19:56.941 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:19:56.941 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:19:56.941 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:19:56.942 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-07-13T22:19:56.942 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.942 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-07-13T22:19:56.942 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:19:56.942 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:19:56.942 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-07-13T22:19:56.942 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:19:56.942 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-07-13T22:19:56.943 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-07-13T22:19:56.943 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:19:56.943 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-07-13T22:19:56.943 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-07-13T22:19:56.943 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:19:56.943 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.943 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:19:56.943 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:19:56.944 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:19:56.944 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.944 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:19:56.944 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:19:56.944 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:19:56.944 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.944 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-07-13T22:19:56.945 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:19:56.945 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.945 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:19:56.945 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2023-07-13T22:19:56.945 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:19:56.945 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-07-13T22:19:56.945 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-07-13T22:19:56.945 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-07-13T22:19:56.946 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-07-13T22:19:56.946 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:19:56.946 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:19:56.946 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2023-07-13T22:19:56.946 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2023-07-13T22:19:56.946 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:19:56.946 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2023-07-13T22:19:56.946 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:19:56.947 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:19:56.947 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-07-13T22:19:56.947 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2023-07-13T22:19:56.947 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2023-07-13T22:19:56.947 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-07-13T22:19:56.947 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-07-13T22:19:56.947 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-07-13T22:19:56.947 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-07-13T22:19:56.948 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2023-07-13T22:19:56.948 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2023-07-13T22:19:56.948 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2023-07-13T22:19:56.948 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:19:56.948 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2023-07-13T22:19:56.948 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2023-07-13T22:19:56.948 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2023-07-13T22:19:56.948 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2023-07-13T22:19:56.949 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:19:56.949 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:19:56.949 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:19:56.949 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2023-07-13T22:19:56.950 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2023-07-13T22:19:56.950 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:19:56.950 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2023-07-13T22:19:56.950 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.950 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2023-07-13T22:19:56.950 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.950 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2023-07-13T22:19:56.950 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.951 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2023-07-13T22:19:56.951 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2023-07-13T22:19:56.951 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.951 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2023-07-13T22:19:56.951 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2023-07-13T22:19:56.951 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.951 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:19:56.952 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2023-07-13T22:19:56.952 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.952 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:19:56.952 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:19:56.952 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.952 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2023-07-13T22:19:56.952 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.952 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2023-07-13T22:19:56.953 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2023-07-13T22:19:56.953 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:19:56.953 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:19:56.953 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2023-07-13T22:19:56.953 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:19:56.953 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2023-07-13T22:19:56.953 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:19:56.953 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:19:56.954 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2023-07-13T22:19:56.954 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.954 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:19:56.954 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.954 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2023-07-13T22:19:56.954 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.954 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2023-07-13T22:19:56.955 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.955 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2023-07-13T22:19:56.955 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2023-07-13T22:19:56.955 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2023-07-13T22:19:56.955 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:19:56.955 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:19:56.956 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.956 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2023-07-13T22:19:56.956 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2023-07-13T22:19:56.956 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.956 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2023-07-13T22:19:56.956 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2023-07-13T22:19:56.956 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:19:56.956 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.957 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2023-07-13T22:19:56.957 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2023-07-13T22:19:56.957 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.957 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2023-07-13T22:19:56.957 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2023-07-13T22:19:56.957 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2023-07-13T22:19:56.957 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.957 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2023-07-13T22:19:56.958 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2023-07-13T22:19:56.958 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.958 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:19:56.958 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.958 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2023-07-13T22:19:56.958 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.958 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2023-07-13T22:19:56.959 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.959 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:19:56.959 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2023-07-13T22:19:56.959 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.959 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2023-07-13T22:19:56.959 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2023-07-13T22:19:56.959 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2023-07-13T22:19:56.959 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2023-07-13T22:19:56.960 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2023-07-13T22:19:56.960 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2023-07-13T22:19:56.960 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2023-07-13T22:19:56.960 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2023-07-13T22:19:56.960 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2023-07-13T22:19:56.960 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:19:56.960 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2023-07-13T22:19:56.961 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2023-07-13T22:19:56.961 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:19:56.961 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2023-07-13T22:19:56.961 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:19:56.961 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:19:56.961 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2023-07-13T22:19:56.961 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:19:56.961 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.962 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2023-07-13T22:19:56.962 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:19:56.962 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2023-07-13T22:19:56.962 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:19:56.962 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:19:56.962 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:19:56.962 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:19:56.963 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2023-07-13T22:19:56.963 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.963 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2023-07-13T22:19:56.963 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:19:56.963 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:19:56.963 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2023-07-13T22:19:56.963 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:19:56.963 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2023-07-13T22:19:56.964 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2023-07-13T22:19:56.964 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:19:56.964 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2023-07-13T22:19:56.964 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2023-07-13T22:19:56.964 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:19:56.964 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.964 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:19:56.965 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:19:56.965 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:19:56.965 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.965 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:19:56.965 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:19:56.965 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:19:56.965 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.965 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2023-07-13T22:19:56.966 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:19:56.966 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.966 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:19:56.966 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2023-07-13T22:19:56.966 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:19:56.966 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2023-07-13T22:19:56.966 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2023-07-13T22:19:56.967 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2023-07-13T22:19:56.967 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2023-07-13T22:19:56.967 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:19:56.967 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:19:56.967 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2023-07-13T22:19:56.967 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2023-07-13T22:19:56.967 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:19:56.968 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2023-07-13T22:19:56.968 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:19:56.968 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:19:56.968 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2023-07-13T22:19:56.968 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2023-07-13T22:19:56.968 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2023-07-13T22:19:56.968 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-07-13T22:19:56.968 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-07-13T22:19:56.969 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-07-13T22:19:56.969 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-07-13T22:19:56.969 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2023-07-13T22:19:56.969 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2023-07-13T22:19:56.969 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2023-07-13T22:19:56.969 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:19:56.969 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2023-07-13T22:19:56.969 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2023-07-13T22:19:56.970 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2023-07-13T22:19:56.970 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2023-07-13T22:19:56.970 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:19:56.970 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:19:56.970 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:19:56.971 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-13T22:19:56.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-13T22:19:57.090 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-13T22:19:57.091 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:57.091 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-13T22:19:57.139 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:57.139 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-13T22:19:57.225 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:19:57.226 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:57.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-13T22:19:57.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-13T22:19:57.369 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:19:57.370 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:19:57.458 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:19:57.459 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:19:57.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-13T22:19:57.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-13T22:19:57.553 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:19:57.554 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:19:57.638 INFO:tasks.ceph:Setting up mon nodes... 2023-07-13T22:19:57.639 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-13T22:19:57.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-13T22:19:57.769 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-13T22:19:57.776 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-13T22:19:57.910 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-13T22:19:57.916 INFO:tasks.ceph:Setting up mds nodes... 2023-07-13T22:19:57.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-07-13T22:19:58.046 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-07-13T22:19:58.051 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-07-13T22:19:58.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-07-13T22:19:58.251 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-07-13T22:19:58.256 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-07-13T22:19:58.326 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-07-13T22:19:58.449 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-07-13T22:19:58.454 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-07-13T22:19:58.525 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-07-13T22:19:58.647 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-07-13T22:19:58.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-07-13T22:19:58.720 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-13T22:19:58.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-13T22:19:58.821 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-13T22:19:58.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-07-13T22:19:58.949 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-07-13T22:19:58.980 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-13T22:19:59.077 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-13T22:19:59.107 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-07-13T22:19:59.202 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-07-13T22:19:59.232 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-13T22:19:59.233 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-13T22:19:59.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-13T22:19:59.306 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-13T22:19:59.307 INFO:tasks.ceph:role: osd.0 2023-07-13T22:19:59.307 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:19:59.307 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-13T22:19:59.378 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-13T22:19:59.378 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-13T22:19:59.378 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-13T22:19:59.378 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-13T22:19:59.379 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-13T22:19:59.379 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-07-13T22:19:59.379 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-13T22:19:59.379 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-13T22:19:59.379 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-13T22:19:59.379 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-13T22:19:59.404 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-07-13T22:19:59.408 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-07-13T22:19:59.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-13T22:19:59.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-13T22:19:59.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-13T22:19:59.665 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-13T22:19:59.665 INFO:tasks.ceph:role: osd.1 2023-07-13T22:19:59.665 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:19:59.666 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-13T22:19:59.735 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-13T22:19:59.736 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-13T22:19:59.736 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-13T22:19:59.736 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-13T22:19:59.736 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-13T22:19:59.736 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-07-13T22:19:59.736 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-13T22:19:59.737 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-13T22:19:59.737 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-13T22:19:59.737 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-13T22:19:59.762 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-07-13T22:19:59.766 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-07-13T22:19:59.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-13T22:19:59.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-13T22:19:59.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-13T22:20:00.025 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-13T22:20:00.026 INFO:tasks.ceph:role: osd.2 2023-07-13T22:20:00.026 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:20:00.026 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-13T22:20:00.121 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-07-13T22:20:00.125 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-07-13T22:20:00.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-13T22:20:00.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-13T22:20:00.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-13T22:20:00.377 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-13T22:20:00.378 INFO:tasks.ceph:role: osd.3 2023-07-13T22:20:00.378 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:20:00.378 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-13T22:20:00.448 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-13T22:20:00.448 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-13T22:20:00.448 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-13T22:20:00.448 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-13T22:20:00.449 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-13T22:20:00.449 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-07-13T22:20:00.449 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-13T22:20:00.449 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-13T22:20:00.449 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-13T22:20:00.449 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-13T22:20:00.474 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-07-13T22:20:00.478 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-07-13T22:20:00.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-13T22:20:00.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-13T22:20:00.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:00.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:00.801+0000 7faea8319640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-13T22:20:00.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:00.801+0000 7faea8319640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-13T22:20:00.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:00.801+0000 7faea8319640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-13T22:20:00.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:00.801+0000 7faea8319640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-13T22:20:00.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:00.801+0000 7faea8319640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-13T22:20:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:00.803+0000 7faea8319640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-13T22:20:03.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-13T22:20:03.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:03.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:03.879+0000 7f60e4767640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-13T22:20:03.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:03.879+0000 7f60e4767640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-13T22:20:03.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:03.880+0000 7f60e4767640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-13T22:20:03.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:03.880+0000 7f60e4767640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-13T22:20:03.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:03.880+0000 7f60e4767640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-13T22:20:03.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:03.881+0000 7f60e4767640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-13T22:20:06.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-13T22:20:06.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:06.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:06.954+0000 7f435da1c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-13T22:20:06.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:06.954+0000 7f435da1c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-13T22:20:06.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:06.954+0000 7f435da1c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-13T22:20:06.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:06.954+0000 7f435da1c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-13T22:20:06.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:06.954+0000 7f435da1c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-13T22:20:06.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:06.955+0000 7f435da1c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-13T22:20:09.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-13T22:20:09.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:10.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:10.022+0000 7f5040f81640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-13T22:20:10.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:10.023+0000 7f5040f81640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-13T22:20:10.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:10.023+0000 7f5040f81640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-13T22:20:10.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:10.023+0000 7f5040f81640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-13T22:20:10.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:10.023+0000 7f5040f81640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-13T22:20:10.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:10.024+0000 7f5040f81640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-13T22:20:12.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-13T22:20:12.971 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-13T22:20:13.048 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-13T22:20:13.048 INFO:tasks.ceph:role: osd.4 2023-07-13T22:20:13.048 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:20:13.049 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-13T22:20:13.127 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-13T22:20:13.127 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-13T22:20:13.127 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-13T22:20:13.127 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-13T22:20:13.127 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-13T22:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-07-13T22:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-13T22:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-13T22:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-13T22:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-13T22:20:13.557 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-07-13T22:20:13.561 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-07-13T22:20:13.562 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-13T22:20:13.751 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-13T22:20:13.830 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-13T22:20:13.900 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-13T22:20:13.900 INFO:tasks.ceph:role: osd.5 2023-07-13T22:20:13.901 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:20:13.901 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-13T22:20:13.977 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-13T22:20:13.977 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-13T22:20:13.978 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-13T22:20:13.978 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-13T22:20:13.978 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-13T22:20:13.978 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-07-13T22:20:13.978 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-13T22:20:13.978 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-13T22:20:13.978 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-13T22:20:13.979 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-13T22:20:14.387 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-07-13T22:20:14.390 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-07-13T22:20:14.391 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-13T22:20:14.529 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-13T22:20:14.604 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-13T22:20:14.673 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-13T22:20:14.674 INFO:tasks.ceph:role: osd.6 2023-07-13T22:20:14.674 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:20:14.674 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-13T22:20:14.751 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-13T22:20:14.751 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-13T22:20:14.751 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-13T22:20:14.752 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-13T22:20:14.752 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-13T22:20:14.752 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-07-13T22:20:14.752 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-13T22:20:14.752 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-13T22:20:14.752 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-13T22:20:14.752 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-13T22:20:15.206 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-07-13T22:20:15.210 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-07-13T22:20:15.211 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-13T22:20:15.407 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-13T22:20:15.485 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-13T22:20:15.555 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-13T22:20:15.555 INFO:tasks.ceph:role: osd.7 2023-07-13T22:20:15.556 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:20:15.556 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-13T22:20:15.631 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-13T22:20:15.632 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-13T22:20:15.632 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-13T22:20:15.632 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-13T22:20:15.632 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-13T22:20:15.632 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-07-13T22:20:15.632 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-13T22:20:15.632 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-13T22:20:15.633 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-13T22:20:15.633 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-13T22:20:16.069 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-07-13T22:20:16.073 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-07-13T22:20:16.073 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-13T22:20:16.196 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-13T22:20:16.271 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:16.394 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:16.409+0000 7f7152024640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-13T22:20:16.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:16.410+0000 7f7152024640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-13T22:20:16.395 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:16.410+0000 7f7152024640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-13T22:20:16.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:16.410+0000 7f7152024640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-13T22:20:16.396 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:16.410+0000 7f7152024640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-13T22:20:16.397 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:16.412+0000 7f7152024640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-13T22:20:19.280 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-13T22:20:19.352 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:19.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:19.471+0000 7f465ec80640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-13T22:20:19.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:19.472+0000 7f465ec80640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-13T22:20:19.457 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:19.472+0000 7f465ec80640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-13T22:20:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:19.472+0000 7f465ec80640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-13T22:20:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:19.472+0000 7f465ec80640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-13T22:20:19.458 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:19.473+0000 7f465ec80640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-13T22:20:22.343 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-13T22:20:22.417 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:22.539+0000 7f932fe00640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-13T22:20:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:22.540+0000 7f932fe00640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-13T22:20:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:22.540+0000 7f932fe00640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-13T22:20:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:22.540+0000 7f932fe00640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-13T22:20:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:22.540+0000 7f932fe00640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-13T22:20:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:22.541+0000 7f932fe00640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-13T22:20:24.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:20:24.548 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:20:25.411 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-13T22:20:25.481 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:25.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:25.601+0000 7fecc5bbb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-13T22:20:25.587 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:25.601+0000 7fecc5bbb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-13T22:20:25.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:25.602+0000 7fecc5bbb640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-13T22:20:25.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:25.602+0000 7fecc5bbb640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-13T22:20:25.588 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:25.602+0000 7fecc5bbb640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-13T22:20:25.589 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:25.603+0000 7fecc5bbb640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-13T22:20:28.470 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-13T22:20:28.540 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-13T22:20:28.541 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:28.541 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-13T22:20:28.618 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:28.618 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-07-13T22:20:28.690 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:28.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-07-13T22:20:28.760 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:28.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-13T22:20:28.870 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:28.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-13T22:20:28.979 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:28.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-13T22:20:29.089 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:29.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-13T22:20:29.198 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:29.199 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-13T22:20:29.271 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:29.271 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-07-13T22:20:29.380 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:29.381 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-07-13T22:20:29.489 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:29.489 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-13T22:20:29.598 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:29.599 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-13T22:20:29.707 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:29.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-13T22:20:29.815 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:29.816 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-13T22:20:29.924 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:29.924 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-13T22:20:29.975 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:29.975 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-07-13T22:20:30.023 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:30.023 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-13T22:20:30.068 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:30.069 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-07-13T22:20:30.152 INFO:tasks.ceph:Adding keys to all mons... 2023-07-13T22:20:30.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-13T22:20:30.157 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-13T22:20:30.222 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.y] 2023-07-13T22:20:30.222 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ== 2023-07-13T22:20:30.223 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-07-13T22:20:30.223 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw== 2023-07-13T22:20:30.223 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-07-13T22:20:30.223 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA== 2023-07-13T22:20:30.223 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.0] 2023-07-13T22:20:30.223 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ== 2023-07-13T22:20:30.223 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.1] 2023-07-13T22:20:30.224 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw== 2023-07-13T22:20:30.224 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.2] 2023-07-13T22:20:30.224 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q== 2023-07-13T22:20:30.224 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.3] 2023-07-13T22:20:30.224 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ== 2023-07-13T22:20:30.224 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.x] 2023-07-13T22:20:30.224 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ== 2023-07-13T22:20:30.225 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-07-13T22:20:30.225 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g== 2023-07-13T22:20:30.225 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-07-13T22:20:30.225 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ== 2023-07-13T22:20:30.225 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.4] 2023-07-13T22:20:30.225 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg== 2023-07-13T22:20:30.225 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.5] 2023-07-13T22:20:30.226 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA== 2023-07-13T22:20:30.226 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.6] 2023-07-13T22:20:30.226 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw== 2023-07-13T22:20:30.226 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.7] 2023-07-13T22:20:30.226 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ== 2023-07-13T22:20:30.226 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:20:30.226 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:20:30.226 INFO:teuthology.orchestra.run.smithi060.stdout:[client.2] 2023-07-13T22:20:30.227 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg== 2023-07-13T22:20:30.227 INFO:teuthology.orchestra.run.smithi060.stdout:[client.1] 2023-07-13T22:20:30.227 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw== 2023-07-13T22:20:30.227 INFO:teuthology.orchestra.run.smithi060.stdout:[client.3] 2023-07-13T22:20:30.227 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q== 2023-07-13T22:20:30.258 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2023-07-13T22:20:30.259 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ== 2023-07-13T22:20:30.259 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-07-13T22:20:30.259 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw== 2023-07-13T22:20:30.259 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-07-13T22:20:30.259 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA== 2023-07-13T22:20:30.260 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2023-07-13T22:20:30.260 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ== 2023-07-13T22:20:30.260 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2023-07-13T22:20:30.260 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw== 2023-07-13T22:20:30.260 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2023-07-13T22:20:30.260 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q== 2023-07-13T22:20:30.261 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2023-07-13T22:20:30.261 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ== 2023-07-13T22:20:30.261 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2023-07-13T22:20:30.262 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ== 2023-07-13T22:20:30.262 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-07-13T22:20:30.262 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g== 2023-07-13T22:20:30.262 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-07-13T22:20:30.263 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ== 2023-07-13T22:20:30.263 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2023-07-13T22:20:30.263 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg== 2023-07-13T22:20:30.264 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2023-07-13T22:20:30.264 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA== 2023-07-13T22:20:30.264 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2023-07-13T22:20:30.264 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw== 2023-07-13T22:20:30.265 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2023-07-13T22:20:30.265 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ== 2023-07-13T22:20:30.265 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-07-13T22:20:30.265 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:20:30.266 INFO:teuthology.orchestra.run.smithi105.stdout:[client.2] 2023-07-13T22:20:30.266 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg== 2023-07-13T22:20:30.266 INFO:teuthology.orchestra.run.smithi105.stdout:[client.1] 2023-07-13T22:20:30.266 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw== 2023-07-13T22:20:30.266 INFO:teuthology.orchestra.run.smithi105.stdout:[client.3] 2023-07-13T22:20:30.266 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q== 2023-07-13T22:20:30.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-13T22:20:30.272 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-13T22:20:30.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-13T22:20:30.419 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-13T22:20:30.530 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-13T22:20:30.535 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-13T22:20:30.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:30.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:30.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:30.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:30.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:30.877 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:30.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:30.990 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:31.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-13T22:20:31.104 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-13T22:20:31.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-13T22:20:31.216 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-13T22:20:31.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-13T22:20:31.331 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-13T22:20:31.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:31.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:31.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:31.560 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:31.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:31.672 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:31.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:31.785 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-13T22:20:31.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-13T22:20:31.897 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-13T22:20:32.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-13T22:20:32.013 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-13T22:20:32.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-13T22:20:32.129 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-13T22:20:32.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-13T22:20:32.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-13T22:20:32.355 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-13T22:20:32.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-13T22:20:32.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-13T22:20:32.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-13T22:20:32.623 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-13T22:20:32.694 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-13T22:20:32.817 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-13T22:20:32.887 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-13T22:20:32.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-13T22:20:33.078 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-13T22:20:33.147 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:33.152 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-13T22:20:33.197 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-13T22:20:33.198 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-13T22:20:33.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-07-13T22:20:33.202 INFO:tasks.ceph.mon.a:Started 2023-07-13T22:20:33.203 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-13T22:20:33.203 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-07-13T22:20:33.242 INFO:tasks.ceph.mon.b:Started 2023-07-13T22:20:33.242 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-13T22:20:33.242 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-07-13T22:20:33.246 INFO:tasks.ceph.mon.c:Started 2023-07-13T22:20:33.247 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-13T22:20:33.247 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-13T22:20:33.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-07-13T22:20:33.251 INFO:tasks.ceph.mgr.y:Started 2023-07-13T22:20:33.252 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-13T22:20:33.252 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-07-13T22:20:33.256 INFO:tasks.ceph.mgr.x:Started 2023-07-13T22:20:33.256 INFO:tasks.ceph:Setting crush tunables to default 2023-07-13T22:20:33.256 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd crush tunables default 2023-07-13T22:20:33.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.478+0000 7f8d5b9ff700 1 Processor -- start 2023-07-13T22:20:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.478+0000 7f8d5b9ff700 1 -- start start 2023-07-13T22:20:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.479+0000 7f8d5b9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 0x7f8d540cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.479+0000 7f8d5b9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 0x7f8d540cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.479+0000 7f8d5b9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d5415ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.479+0000 7f8d5b9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540d1870 con 0x7f8d540b1ac0 2023-07-13T22:20:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.479+0000 7f8d5b9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540cf370 con 0x7f8d540b8200 2023-07-13T22:20:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.479+0000 7f8d5b9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540d12b0 con 0x7f8d540cede0 2023-07-13T22:20:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d58f9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 0x7f8d540cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d58f9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 0x7f8d540cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35838/0 (socket says 172.21.15.60:35838) 2023-07-13T22:20:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d58f9a700 1 -- 172.21.15.60:0/2369959439 learned_addr learned my addr 172.21.15.60:0/2369959439 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d5979b700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 0x7f8d540cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d59f9c700 1 --2- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d5415ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d5979b700 1 -- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 msgr2=0x7f8d540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-13T22:20:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d5979b700 1 -- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 msgr2=0x7f8d540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-13T22:20:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d5979b700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 0x7f8d540cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-13T22:20:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d59f9c700 1 -- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 msgr2=0x7f8d5415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-07-13T22:20:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d59f9c700 1 -- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 msgr2=0x7f8d5415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-13T22:20:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d59f9c700 1 --2- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d5415ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-13T22:20:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d5979b700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 0x7f8d540cf640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:20:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d59f9c700 1 --2- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d5415ff60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:20:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d58f9a700 1 -- 172.21.15.60:0/2369959439 >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 msgr2=0x7f8d540cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-13T22:20:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d58f9a700 1 -- 172.21.15.60:0/2369959439 >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 msgr2=0x7f8d540cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-13T22:20:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d58f9a700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 0x7f8d540cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-13T22:20:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.480+0000 7f8d58f9a700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 0x7f8d540cfb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:20:33.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.681+0000 7f8d5979b700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 0x7f8d540cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:33.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.681+0000 7f8d59f9c700 1 --2- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d5415ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:33.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.681+0000 7f8d58f9a700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 0x7f8d540cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:33.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.681+0000 7f8d59f9c700 1 -- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 msgr2=0x7f8d540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:33.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.681+0000 7f8d59f9c700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 0x7f8d540cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.681+0000 7f8d59f9c700 1 -- 172.21.15.60:0/2369959439 >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 msgr2=0x7f8d540cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:33.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.681+0000 7f8d59f9c700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 0x7f8d540cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.681+0000 7f8d59f9c700 1 -- 172.21.15.60:0/2369959439 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540d1590 con 0x7f8d540cede0 2023-07-13T22:20:33.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.681+0000 7f8d5979b700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 0x7f8d540cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:33.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.682+0000 7f8d59f9c700 1 --2- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d5415ff60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d540b2660 tx=0x7f8d4c007610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00df3f827e3c32d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.682+0000 7f8d4bfff700 1 -- 172.21.15.60:0/2369959439 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c014440 con 0x7f8d540cede0 2023-07-13T22:20:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.682+0000 7f8d4bfff700 1 -- 172.21.15.60:0/2369959439 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c0145a0 con 0x7f8d540cede0 2023-07-13T22:20:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.682+0000 7f8d4bfff700 1 -- 172.21.15.60:0/2369959439 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c014970 con 0x7f8d540cede0 2023-07-13T22:20:33.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.683+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 msgr2=0x7f8d5415ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:33.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.683+0000 7f8d5b9ff700 1 --2- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d5415ff60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d540b2660 tx=0x7f8d4c007610 comp rx=0 tx=0).stop 2023-07-13T22:20:33.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.683+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2369959439 shutdown_connections 2023-07-13T22:20:33.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.683+0000 7f8d5b9ff700 1 --2- 172.21.15.60:0/2369959439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d5415ff60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.683+0000 7f8d5b9ff700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.105:3300/0 conn(0x7f8d540b8200 0x7f8d540cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.683+0000 7f8d5b9ff700 1 --2- 172.21.15.60:0/2369959439 >> v2:172.21.15.60:3300/0 conn(0x7f8d540b1ac0 0x7f8d540cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.683+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2369959439 >> 172.21.15.60:0/2369959439 conn(0x7f8d5406b2f0 msgr2=0x7f8d5406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:33.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.683+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2369959439 shutdown_connections 2023-07-13T22:20:33.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.683+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2369959439 wait complete. 2023-07-13T22:20:33.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.687+0000 7f8d5b9ff700 1 Processor -- start 2023-07-13T22:20:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.687+0000 7f8d5b9ff700 1 -- start start 2023-07-13T22:20:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d5b9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b1ac0 0x7f8d540c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d5b9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d540b8200 0x7f8d540c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d5b9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d540ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d5b9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540caae0 con 0x7f8d540b8200 2023-07-13T22:20:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d5b9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540cadc0 con 0x7f8d540b1ac0 2023-07-13T22:20:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d5b9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540cb0a0 con 0x7f8d540cede0 2023-07-13T22:20:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d58f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d540b8200 0x7f8d540c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d58f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d540b8200 0x7f8d540c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35870/0 (socket says 172.21.15.60:35870) 2023-07-13T22:20:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d58f9a700 1 -- 172.21.15.60:0/2540680719 learned_addr learned my addr 172.21.15.60:0/2540680719 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d5979b700 1 --2- 172.21.15.60:0/2540680719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b1ac0 0x7f8d540c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d59f9c700 1 --2- 172.21.15.60:0/2540680719 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d540ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d58f9a700 1 -- 172.21.15.60:0/2540680719 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 msgr2=0x7f8d540ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d58f9a700 1 --2- 172.21.15.60:0/2540680719 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d540ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d58f9a700 1 -- 172.21.15.60:0/2540680719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b1ac0 msgr2=0x7f8d540c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d58f9a700 1 --2- 172.21.15.60:0/2540680719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b1ac0 0x7f8d540c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d58f9a700 1 -- 172.21.15.60:0/2540680719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50009500 con 0x7f8d540b8200 2023-07-13T22:20:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d58f9a700 1 --2- 172.21.15.60:0/2540680719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d540b8200 0x7f8d540c9d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8d440116a0 tx=0x7f8d440119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d49ffb700 1 -- 172.21.15.60:0/2540680719 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400c070 con 0x7f8d540b8200 2023-07-13T22:20:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d49ffb700 1 -- 172.21.15.60:0/2540680719 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d44004220 con 0x7f8d540b8200 2023-07-13T22:20:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d49ffb700 1 -- 172.21.15.60:0/2540680719 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d440036a0 con 0x7f8d540b8200 2023-07-13T22:20:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2540680719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c009ad0 con 0x7f8d540b8200 2023-07-13T22:20:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.689+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2540680719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d540c6800 con 0x7f8d540b8200 2023-07-13T22:20:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.690+0000 7f8d49ffb700 1 -- 172.21.15.60:0/2540680719 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8d44003800 con 0x7f8d540b8200 2023-07-13T22:20:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.690+0000 7f8d49ffb700 1 -- 172.21.15.60:0/2540680719 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8d44015aa0 con 0x7f8d540b8200 2023-07-13T22:20:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.691+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2540680719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d540ca850 con 0x7f8d540b8200 2023-07-13T22:20:33.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.696+0000 7f8d49ffb700 1 -- 172.21.15.60:0/2540680719 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f8d44022060 con 0x7f8d540b8200 2023-07-13T22:20:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.776+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2540680719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8d5404e210 con 0x7f8d540b8200 2023-07-13T22:20:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.779+0000 7f8d49ffb700 1 -- 172.21.15.60:0/2540680719 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8d4402b020 con 0x7f8d540b8200 2023-07-13T22:20:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.780+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2540680719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d540b8200 msgr2=0x7f8d540c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.780+0000 7f8d5b9ff700 1 --2- 172.21.15.60:0/2540680719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d540b8200 0x7f8d540c9d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8d440116a0 tx=0x7f8d440119b0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.780+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2540680719 shutdown_connections 2023-07-13T22:20:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.780+0000 7f8d5b9ff700 1 --2- 172.21.15.60:0/2540680719 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d540cede0 0x7f8d540ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.780+0000 7f8d5b9ff700 1 --2- 172.21.15.60:0/2540680719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b1ac0 0x7f8d540c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.780+0000 7f8d5b9ff700 1 --2- 172.21.15.60:0/2540680719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d540b8200 0x7f8d540c9d70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.780+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2540680719 >> 172.21.15.60:0/2540680719 conn(0x7f8d5406b2f0 msgr2=0x7f8d5415c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.781+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2540680719 shutdown_connections 2023-07-13T22:20:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:33.781+0000 7f8d5b9ff700 1 -- 172.21.15.60:0/2540680719 wait complete. 2023-07-13T22:20:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2023-07-13T22:20:33.783 INFO:tasks.ceph:check_enable_crimson: False 2023-07-13T22:20:33.784 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-13T22:20:33.785 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:33.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-13T22:20:33.789 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:33.802+0000 7f95c665e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-13T22:20:33.855 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:33.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-13T22:20:34.000 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:34.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-13T22:20:34.026 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:34.040+0000 7f95c665e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-13T22:20:34.071 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:20:34.071 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-13T22:20:34.160 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:34.175+0000 7f1d0e646200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-13T22:20:34.181 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:34.182 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-13T22:20:34.251 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:34.252 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-13T22:20:34.375 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:34.390+0000 7f1d0e646200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-13T22:20:34.381 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:34.382 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-13T22:20:34.451 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:20:34.451 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-13T22:20:34.482 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:34.496+0000 7f95c665e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-13T22:20:34.560 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new b74ca5bc-389c-43b7-8f76-69cfa5793530 0 2023-07-13T22:20:34.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.808+0000 7fb57e5ff700 1 Processor -- start 2023-07-13T22:20:34.794 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.808+0000 7fb57e5ff700 1 -- start start 2023-07-13T22:20:34.795 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.809+0000 7fb57e5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5780b8200 0x7fb5780cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.809+0000 7fb57e5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5780b1ac0 0x7fb5780cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.809+0000 7fb57e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780cede0 0x7fb57815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.809+0000 7fb57e5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5780d1870 con 0x7fb5780b8200 2023-07-13T22:20:34.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb57e5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5780cf370 con 0x7fb5780b1ac0 2023-07-13T22:20:34.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb57e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5780d12b0 con 0x7fb5780cede0 2023-07-13T22:20:34.796 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb57cb9c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780cede0 0x7fb57815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb5777fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5780b1ac0 0x7fb5780cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb57cb9c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780cede0 0x7fb57815ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35774/0 (socket says 172.21.15.105:35774) 2023-07-13T22:20:34.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb57cb9c700 1 -- 172.21.15.105:0/3167595274 learned_addr learned my addr 172.21.15.105:0/3167595274 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:34.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb577fff700 1 --2- 172.21.15.105:0/3167595274 >> v2:172.21.15.60:3300/0 conn(0x7fb5780b8200 0x7fb5780cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb5777fe700 1 -- 172.21.15.105:0/3167595274 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780cede0 msgr2=0x7fb57815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:34.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb5777fe700 1 --2- 172.21.15.105:0/3167595274 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780cede0 0x7fb57815ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.797 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb5777fe700 1 -- 172.21.15.105:0/3167595274 >> v2:172.21.15.60:3300/0 conn(0x7fb5780b8200 msgr2=0x7fb5780cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:34.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb5777fe700 1 --2- 172.21.15.105:0/3167595274 >> v2:172.21.15.60:3300/0 conn(0x7fb5780b8200 0x7fb5780cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb5777fe700 1 -- 172.21.15.105:0/3167595274 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5780d1590 con 0x7fb5780b1ac0 2023-07-13T22:20:34.798 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.810+0000 7fb57cb9c700 1 --2- 172.21.15.105:0/3167595274 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780cede0 0x7fb57815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.811+0000 7fb5777fe700 1 --2- 172.21.15.105:0/3167595274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780b1ac0 0x7fb5780cfb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb5780d07b0 tx=0x7fb56c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f2b45fe47553158 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.811+0000 7fb576ffd700 1 -- 172.21.15.105:0/3167595274 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c011830 con 0x7fb5780b1ac0 2023-07-13T22:20:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.811+0000 7fb576ffd700 1 -- 172.21.15.105:0/3167595274 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c011990 con 0x7fb5780b1ac0 2023-07-13T22:20:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.811+0000 7fb576ffd700 1 -- 172.21.15.105:0/3167595274 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c011d20 con 0x7fb5780b1ac0 2023-07-13T22:20:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.812+0000 7fb57e5ff700 1 -- 172.21.15.105:0/3167595274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780b1ac0 msgr2=0x7fb5780cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.799 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.812+0000 7fb57e5ff700 1 --2- 172.21.15.105:0/3167595274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780b1ac0 0x7fb5780cfb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb5780d07b0 tx=0x7fb56c009b20 comp rx=0 tx=0).stop 2023-07-13T22:20:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.812+0000 7fb57e5ff700 1 -- 172.21.15.105:0/3167595274 shutdown_connections 2023-07-13T22:20:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.812+0000 7fb57e5ff700 1 --2- 172.21.15.105:0/3167595274 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780cede0 0x7fb57815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.812+0000 7fb57e5ff700 1 --2- 172.21.15.105:0/3167595274 >> v2:172.21.15.60:3300/0 conn(0x7fb5780b8200 0x7fb5780cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.812+0000 7fb57e5ff700 1 --2- 172.21.15.105:0/3167595274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780b1ac0 0x7fb5780cfb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.812+0000 7fb57e5ff700 1 -- 172.21.15.105:0/3167595274 >> 172.21.15.105:0/3167595274 conn(0x7fb57806b2f0 msgr2=0x7fb57806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.812+0000 7fb57e5ff700 1 -- 172.21.15.105:0/3167595274 shutdown_connections 2023-07-13T22:20:34.800 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.812+0000 7fb57e5ff700 1 -- 172.21.15.105:0/3167595274 wait complete. 2023-07-13T22:20:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.814+0000 7fb57e5ff700 1 Processor -- start 2023-07-13T22:20:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.815+0000 7fb57e5ff700 1 -- start start 2023-07-13T22:20:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb57e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780b1ac0 0x7fb57814fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb57e5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5780b8200 0x7fb5781f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb57e5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780cede0 0x7fb578157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb57e5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb578150100 con 0x7fb5780b8200 2023-07-13T22:20:34.803 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb57e5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781f2540 con 0x7fb5780cede0 2023-07-13T22:20:34.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb57e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781f2820 con 0x7fb5780b1ac0 2023-07-13T22:20:34.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb577fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780b1ac0 0x7fb57814fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb577fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780b1ac0 0x7fb57814fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35786/0 (socket says 172.21.15.105:35786) 2023-07-13T22:20:34.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb577fff700 1 -- 172.21.15.105:0/2802460366 learned_addr learned my addr 172.21.15.105:0/2802460366 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:34.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb57cb9c700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780cede0 0x7fb578157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.816+0000 7fb5777fe700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5780b8200 0x7fb5781f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.804 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb57cb9c700 1 -- 172.21.15.105:0/2802460366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780b1ac0 msgr2=0x7fb57814fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb57cb9c700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780b1ac0 0x7fb57814fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb57cb9c700 1 -- 172.21.15.105:0/2802460366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5780b8200 msgr2=0x7fb5781f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb57cb9c700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5780b8200 0x7fb5781f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb57cb9c700 1 -- 172.21.15.105:0/2802460366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570009b30 con 0x7fb5780cede0 2023-07-13T22:20:34.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb5777fe700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5780b8200 0x7fb5781f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:34.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb577fff700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780b1ac0 0x7fb57814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:34.805 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb57cb9c700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780cede0 0x7fb578157ab0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb57000efc0 tx=0x7fb57000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:34.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb574ff9700 1 -- 172.21.15.105:0/2802460366 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570018070 con 0x7fb5780cede0 2023-07-13T22:20:34.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb574ff9700 1 -- 172.21.15.105:0/2802460366 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57000f040 con 0x7fb5780cede0 2023-07-13T22:20:34.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb574ff9700 1 -- 172.21.15.105:0/2802460366 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570003c40 con 0x7fb5780cede0 2023-07-13T22:20:34.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb57e5ff700 1 -- 172.21.15.105:0/2802460366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c0094a0 con 0x7fb5780cede0 2023-07-13T22:20:34.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.817+0000 7fb57e5ff700 1 -- 172.21.15.105:0/2802460366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5781f2de0 con 0x7fb5780cede0 2023-07-13T22:20:34.806 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.818+0000 7fb574ff9700 1 -- 172.21.15.105:0/2802460366 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb57000cb80 con 0x7fb5780cede0 2023-07-13T22:20:34.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.818+0000 7fb574ff9700 1 -- 172.21.15.105:0/2802460366 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb57000ce30 con 0x7fb5780cede0 2023-07-13T22:20:34.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.819+0000 7fb57e5ff700 1 -- 172.21.15.105:0/2802460366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57804e210 con 0x7fb5780cede0 2023-07-13T22:20:34.807 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.822+0000 7fb574ff9700 1 -- 172.21.15.105:0/2802460366 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb570003da0 con 0x7fb5780cede0 2023-07-13T22:20:34.814 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:34.829+0000 7f1d0e646200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-13T22:20:34.893 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.908+0000 7fb57e5ff700 1 -- 172.21.15.105:0/2802460366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b74ca5bc-389c-43b7-8f76-69cfa5793530", "id": 0} v 0) v1 -- 0x7fb5780b4f70 con 0x7fb5780cede0 2023-07-13T22:20:34.897 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.912+0000 7fb574ff9700 1 -- 172.21.15.105:0/2802460366 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b74ca5bc-389c-43b7-8f76-69cfa5793530", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb57002c0b0 con 0x7fb5780cede0 2023-07-13T22:20:34.897 INFO:teuthology.orchestra.run.smithi105.stdout:0 2023-07-13T22:20:34.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.913+0000 7fb57e5ff700 1 -- 172.21.15.105:0/2802460366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780cede0 msgr2=0x7fb578157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.913+0000 7fb57e5ff700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780cede0 0x7fb578157ab0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb57000efc0 tx=0x7fb57000c3a0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.913+0000 7fb57e5ff700 1 -- 172.21.15.105:0/2802460366 shutdown_connections 2023-07-13T22:20:34.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.913+0000 7fb57e5ff700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780cede0 0x7fb578157ab0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.899 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.913+0000 7fb57e5ff700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780b1ac0 0x7fb57814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.913+0000 7fb57e5ff700 1 --2- 172.21.15.105:0/2802460366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5780b8200 0x7fb5781f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.913+0000 7fb57e5ff700 1 -- 172.21.15.105:0/2802460366 >> 172.21.15.105:0/2802460366 conn(0x7fb57806b2f0 msgr2=0x7fb57806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.913+0000 7fb57e5ff700 1 -- 172.21.15.105:0/2802460366 shutdown_connections 2023-07-13T22:20:34.900 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:34.914+0000 7fb57e5ff700 1 -- 172.21.15.105:0/2802460366 wait complete. 2023-07-13T22:20:34.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new e3f29053-af1f-4c50-92f9-fd2eb879967b 1 2023-07-13T22:20:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.124+0000 7f48f3fff700 1 Processor -- start 2023-07-13T22:20:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.124+0000 7f48f3fff700 1 -- start start 2023-07-13T22:20:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.126+0000 7f48f3fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0cfd50 0x7f48ec0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.126+0000 7f48f3fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0d1870 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.126+0000 7f48f3fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0cf370 con 0x7f48ec0b8200 2023-07-13T22:20:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.126+0000 7f48f3fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0d12b0 con 0x7f48ec0cfd50 2023-07-13T22:20:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.126+0000 7f48f159a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0cfd50 0x7f48ec0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.126+0000 7f48f1d9b700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f48ec0b8200 0x7f48ec0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51046/0 (socket says 172.21.15.105:51046) 2023-07-13T22:20:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.126+0000 7f48f1d9b700 1 -- 172.21.15.105:0/1968203001 learned_addr learned my addr 172.21.15.105:0/1968203001 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-07-13T22:20:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.126+0000 7f48f159a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0cfd50 0x7f48ec0d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35800/0 (socket says 172.21.15.105:35800) 2023-07-13T22:20:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.127+0000 7f48f159a700 1 -- 172.21.15.105:0/1968203001 >> v1:172.21.15.105:6789/0 conn(0x7f48ec0b8200 legacy=0x7f48ec0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.127+0000 7f48f0d99700 1 -- 172.21.15.105:0/1968203001 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585375181 0 0) 0x7f48ec0cf370 con 0x7f48ec0b8200 2023-07-13T22:20:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.127+0000 7f48f159a700 1 -- 172.21.15.105:0/1968203001 >> v1:172.21.15.60:6789/0 conn(0x7f48ec0b1ac0 legacy=0x7f48ec0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.127+0000 7f48f159a700 1 -- 172.21.15.105:0/1968203001 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec0d1590 con 0x7f48ec0cfd50 2023-07-13T22:20:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.127+0000 7f48f159a700 1 --2- 172.21.15.105:0/1968203001 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0cfd50 0x7f48ec0d23c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0098a0 tx=0x7f48dc009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232f51d6e28f12e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.127+0000 7f48f0d99700 1 -- 172.21.15.105:0/1968203001 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc011830 con 0x7f48ec0cfd50 2023-07-13T22:20:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.127+0000 7f48f0d99700 1 -- 172.21.15.105:0/1968203001 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc011990 con 0x7f48ec0cfd50 2023-07-13T22:20:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.127+0000 7f48f0d99700 1 -- 172.21.15.105:0/1968203001 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc011d40 con 0x7f48ec0cfd50 2023-07-13T22:20:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.128+0000 7f48f3fff700 1 -- 172.21.15.105:0/1968203001 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0cfd50 msgr2=0x7f48ec0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.128+0000 7f48f3fff700 1 --2- 172.21.15.105:0/1968203001 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0cfd50 0x7f48ec0d23c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0098a0 tx=0x7f48dc009bb0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.128+0000 7f48f3fff700 1 -- 172.21.15.105:0/1968203001 shutdown_connections 2023-07-13T22:20:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.128+0000 7f48f3fff700 1 --2- 172.21.15.105:0/1968203001 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0cfd50 0x7f48ec0d23c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.128+0000 7f48f3fff700 1 -- 172.21.15.105:0/1968203001 >> 172.21.15.105:0/1968203001 conn(0x7f48ec06b2f0 msgr2=0x7f48ec06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.128+0000 7f48f3fff700 1 -- 172.21.15.105:0/1968203001 shutdown_connections 2023-07-13T22:20:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.128+0000 7f48f3fff700 1 -- 172.21.15.105:0/1968203001 wait complete. 2023-07-13T22:20:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.130+0000 7f48f3fff700 1 Processor -- start 2023-07-13T22:20:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.130+0000 7f48f3fff700 1 -- start start 2023-07-13T22:20:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f3fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0b1ac0 0x7f48ec14d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48ec0b8200 0x7f48ec0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f3fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48ec0cfd50 0x7f48ec14d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f3fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0cd0e0 con 0x7f48ec0b8200 2023-07-13T22:20:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f3fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0ca350 con 0x7f48ec0cfd50 2023-07-13T22:20:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f3fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec14d5b0 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f1d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0b1ac0 0x7f48ec14d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f1d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0b1ac0 0x7f48ec14d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35812/0 (socket says 172.21.15.105:35812) 2023-07-13T22:20:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f1d9b700 1 -- 172.21.15.105:0/3063598145 learned_addr learned my addr 172.21.15.105:0/3063598145 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f159a700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48ec0b8200 0x7f48ec0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.131+0000 7f48f259c700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48ec0cfd50 0x7f48ec14d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48f1d9b700 1 -- 172.21.15.105:0/3063598145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48ec0cfd50 msgr2=0x7f48ec14d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48f1d9b700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48ec0cfd50 0x7f48ec14d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48f1d9b700 1 -- 172.21.15.105:0/3063598145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48ec0b8200 msgr2=0x7f48ec0ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48f1d9b700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48ec0b8200 0x7f48ec0ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48f1d9b700 1 -- 172.21.15.105:0/3063598145 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0094a0 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48f259c700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48ec0cfd50 0x7f48ec14d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48f159a700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48ec0b8200 0x7f48ec0ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48f1d9b700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0b1ac0 0x7f48ec14d890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c520 tx=0x7f48e800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48e27fc700 1 -- 172.21.15.105:0/3063598145 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004c80 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48e27fc700 1 -- 172.21.15.105:0/3063598145 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e8004e00 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48e27fc700 1 -- 172.21.15.105:0/3063598145 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8007310 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.132+0000 7f48f3fff700 1 -- 172.21.15.105:0/3063598145 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec0ca070 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.133+0000 7f48f3fff700 1 -- 172.21.15.105:0/3063598145 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec15e680 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.133+0000 7f48e27fc700 1 -- 172.21.15.105:0/3063598145 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f48e801d3f0 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.133+0000 7f48e27fc700 1 -- 172.21.15.105:0/3063598145 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f48e8013770 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.134+0000 7f48f3fff700 1 -- 172.21.15.105:0/3063598145 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec04e210 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.137+0000 7f48e27fc700 1 -- 172.21.15.105:0/3063598145 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f48e8013930 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.126 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:35.141+0000 7f95c665e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-13T22:20:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.223+0000 7f48f3fff700 1 -- 172.21.15.105:0/3063598145 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e3f29053-af1f-4c50-92f9-fd2eb879967b", "id": 1} v 0) v1 -- 0x7f48ec0b4f70 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.227+0000 7f48e27fc700 1 -- 172.21.15.105:0/3063598145 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e3f29053-af1f-4c50-92f9-fd2eb879967b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f48e8013ba0 con 0x7f48ec0b1ac0 2023-07-13T22:20:35.212 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:20:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.228+0000 7f48f3fff700 1 -- 172.21.15.105:0/3063598145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0b1ac0 msgr2=0x7f48ec14d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.228+0000 7f48f3fff700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0b1ac0 0x7f48ec14d890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c520 tx=0x7f48e800c830 comp rx=0 tx=0).stop 2023-07-13T22:20:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.228+0000 7f48f3fff700 1 -- 172.21.15.105:0/3063598145 shutdown_connections 2023-07-13T22:20:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.228+0000 7f48f3fff700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48ec0cfd50 0x7f48ec14d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.228+0000 7f48f3fff700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48ec0b1ac0 0x7f48ec14d890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.228+0000 7f48f3fff700 1 --2- 172.21.15.105:0/3063598145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48ec0b8200 0x7f48ec0ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.228+0000 7f48f3fff700 1 -- 172.21.15.105:0/3063598145 >> 172.21.15.105:0/3063598145 conn(0x7f48ec06b2f0 msgr2=0x7f48ec0c8770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.228+0000 7f48f3fff700 1 -- 172.21.15.105:0/3063598145 shutdown_connections 2023-07-13T22:20:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.228+0000 7f48f3fff700 1 -- 172.21.15.105:0/3063598145 wait complete. 2023-07-13T22:20:35.231 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new d04e4f57-6185-4898-8417-14b13b6707be 2 2023-07-13T22:20:35.368 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:35.384+0000 7f95c665e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-13T22:20:35.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.437+0000 7f40281ff700 1 Processor -- start 2023-07-13T22:20:35.422 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.437+0000 7f40281ff700 1 -- start start 2023-07-13T22:20:35.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.438+0000 7f40281ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200cee90 0x7f40200d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.423 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.438+0000 7f40281ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f40200d0ce0 0x7f40200d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.438+0000 7f40281ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40200cd960 con 0x7f402015cf00 2023-07-13T22:20:35.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.438+0000 7f40281ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402015d9e0 con 0x7f40200d0ce0 2023-07-13T22:20:35.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.438+0000 7f40281ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40201539b0 con 0x7f40200cee90 2023-07-13T22:20:35.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.438+0000 7f4025f9b700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f402015cf00 0x7f402015d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51050/0 (socket says 172.21.15.105:51050) 2023-07-13T22:20:35.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.438+0000 7f4025f9b700 1 -- 172.21.15.105:0/472955781 learned_addr learned my addr 172.21.15.105:0/472955781 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-07-13T22:20:35.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.438+0000 7f402679c700 1 --2- 172.21.15.105:0/472955781 >> v2:172.21.15.105:3300/0 conn(0x7f40200d0ce0 0x7f40200d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.424 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.438+0000 7f402579a700 1 --2- 172.21.15.105:0/472955781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200cee90 0x7f40200d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.425 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f4024f99700 1 -- 172.21.15.105:0/472955781 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802070868 0 0) 0x7f40200cd960 con 0x7f402015cf00 2023-07-13T22:20:35.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f402579a700 1 -- 172.21.15.105:0/472955781 >> v2:172.21.15.105:3300/0 conn(0x7f40200d0ce0 msgr2=0x7f40200d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f402579a700 1 --2- 172.21.15.105:0/472955781 >> v2:172.21.15.105:3300/0 conn(0x7f40200d0ce0 0x7f40200d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f402579a700 1 -- 172.21.15.105:0/472955781 >> v1:172.21.15.105:6789/0 conn(0x7f402015cf00 legacy=0x7f402015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f402579a700 1 -- 172.21.15.105:0/472955781 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40200cd3a0 con 0x7f40200cee90 2023-07-13T22:20:35.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f402679c700 1 --2- 172.21.15.105:0/472955781 >> v2:172.21.15.105:3300/0 conn(0x7f40200d0ce0 0x7f40200d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:35.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f402579a700 1 --2- 172.21.15.105:0/472955781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200cee90 0x7f40200d07a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4010009840 tx=0x7f4010009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad35f5b4ae674a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.426 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f4024f99700 1 -- 172.21.15.105:0/472955781 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010011830 con 0x7f40200cee90 2023-07-13T22:20:35.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f4024f99700 1 -- 172.21.15.105:0/472955781 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4010011990 con 0x7f40200cee90 2023-07-13T22:20:35.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.439+0000 7f4024f99700 1 -- 172.21.15.105:0/472955781 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010011d20 con 0x7f40200cee90 2023-07-13T22:20:35.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.440+0000 7f40281ff700 1 -- 172.21.15.105:0/472955781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200cee90 msgr2=0x7f40200d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.440+0000 7f40281ff700 1 --2- 172.21.15.105:0/472955781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200cee90 0x7f40200d07a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4010009840 tx=0x7f4010009b50 comp rx=0 tx=0).stop 2023-07-13T22:20:35.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.440+0000 7f40281ff700 1 -- 172.21.15.105:0/472955781 shutdown_connections 2023-07-13T22:20:35.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.440+0000 7f40281ff700 1 --2- 172.21.15.105:0/472955781 >> v2:172.21.15.105:3300/0 conn(0x7f40200d0ce0 0x7f40200d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.427 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.440+0000 7f40281ff700 1 --2- 172.21.15.105:0/472955781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200cee90 0x7f40200d07a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.440+0000 7f40281ff700 1 -- 172.21.15.105:0/472955781 >> 172.21.15.105:0/472955781 conn(0x7f4020071600 msgr2=0x7f4020071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.440+0000 7f40281ff700 1 -- 172.21.15.105:0/472955781 shutdown_connections 2023-07-13T22:20:35.428 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.440+0000 7f40281ff700 1 -- 172.21.15.105:0/472955781 wait complete. 2023-07-13T22:20:35.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.442+0000 7f40281ff700 1 Processor -- start 2023-07-13T22:20:35.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.442+0000 7f40281ff700 1 -- start start 2023-07-13T22:20:35.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f40281ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40200cee90 0x7f40201f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f40281ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200d0ce0 0x7f40200cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f40281ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f402015cf00 0x7f4020150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f40281ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40200cb230 con 0x7f402015cf00 2023-07-13T22:20:35.430 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f40281ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40200cb490 con 0x7f40200cee90 2023-07-13T22:20:35.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f40281ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40200cb770 con 0x7f40200d0ce0 2023-07-13T22:20:35.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f4025f9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40200cee90 0x7f40201f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f402579a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200d0ce0 0x7f40200cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f4025f9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40200cee90 0x7f40201f2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49318/0 (socket says 172.21.15.105:49318) 2023-07-13T22:20:35.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f4025f9b700 1 -- 172.21.15.105:0/1616790278 learned_addr learned my addr 172.21.15.105:0/1616790278 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:35.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f402679c700 1 --2- 172.21.15.105:0/1616790278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f402015cf00 0x7f4020150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.431 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.443+0000 7f4025f9b700 1 -- 172.21.15.105:0/1616790278 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200d0ce0 msgr2=0x7f40200cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f4025f9b700 1 --2- 172.21.15.105:0/1616790278 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200d0ce0 0x7f40200cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f4025f9b700 1 -- 172.21.15.105:0/1616790278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f402015cf00 msgr2=0x7f4020150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f4025f9b700 1 --2- 172.21.15.105:0/1616790278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f402015cf00 0x7f4020150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f4025f9b700 1 -- 172.21.15.105:0/1616790278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018009a70 con 0x7f40200cee90 2023-07-13T22:20:35.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f402579a700 1 --2- 172.21.15.105:0/1616790278 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200d0ce0 0x7f40200cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:35.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f4025f9b700 1 --2- 172.21.15.105:0/1616790278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40200cee90 0x7f40201f2570 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f401c0011e0 tx=0x7f401c00ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.432 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f40167fc700 1 -- 172.21.15.105:0/1616790278 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c004c80 con 0x7f40200cee90 2023-07-13T22:20:35.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f40167fc700 1 -- 172.21.15.105:0/1616790278 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c004de0 con 0x7f40200cee90 2023-07-13T22:20:35.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f40167fc700 1 -- 172.21.15.105:0/1616790278 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c007310 con 0x7f40200cee90 2023-07-13T22:20:35.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f40281ff700 1 -- 172.21.15.105:0/1616790278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40100094a0 con 0x7f40200cee90 2023-07-13T22:20:35.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f40281ff700 1 -- 172.21.15.105:0/1616790278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402015dca0 con 0x7f40200cee90 2023-07-13T22:20:35.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.444+0000 7f40167fc700 1 -- 172.21.15.105:0/1616790278 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f401c01d3f0 con 0x7f40200cee90 2023-07-13T22:20:35.433 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.445+0000 7f40167fc700 1 -- 172.21.15.105:0/1616790278 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f401c01dac0 con 0x7f40200cee90 2023-07-13T22:20:35.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.445+0000 7f40281ff700 1 -- 172.21.15.105:0/1616790278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40200cbc90 con 0x7f40200cee90 2023-07-13T22:20:35.434 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.448+0000 7f40167fc700 1 -- 172.21.15.105:0/1616790278 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f401c01dd10 con 0x7f40200cee90 2023-07-13T22:20:35.453 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:35.468+0000 7f1d0e646200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-13T22:20:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.532+0000 7f40281ff700 1 -- 172.21.15.105:0/1616790278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d04e4f57-6185-4898-8417-14b13b6707be", "id": 2} v 0) v1 -- 0x7f402015dec0 con 0x7f40200cee90 2023-07-13T22:20:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.536+0000 7f40167fc700 1 -- 172.21.15.105:0/1616790278 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d04e4f57-6185-4898-8417-14b13b6707be", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f401c01d770 con 0x7f40200cee90 2023-07-13T22:20:35.521 INFO:teuthology.orchestra.run.smithi105.stdout:2 2023-07-13T22:20:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.537+0000 7f40281ff700 1 -- 172.21.15.105:0/1616790278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40200cee90 msgr2=0x7f40201f2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.537+0000 7f40281ff700 1 --2- 172.21.15.105:0/1616790278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40200cee90 0x7f40201f2570 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f401c0011e0 tx=0x7f401c00ca30 comp rx=0 tx=0).stop 2023-07-13T22:20:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.537+0000 7f40281ff700 1 -- 172.21.15.105:0/1616790278 shutdown_connections 2023-07-13T22:20:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.537+0000 7f40281ff700 1 --2- 172.21.15.105:0/1616790278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f402015cf00 0x7f4020150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.537+0000 7f40281ff700 1 --2- 172.21.15.105:0/1616790278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40200cee90 0x7f40201f2570 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.537+0000 7f40281ff700 1 --2- 172.21.15.105:0/1616790278 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40200d0ce0 0x7f40200cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.537+0000 7f40281ff700 1 -- 172.21.15.105:0/1616790278 >> 172.21.15.105:0/1616790278 conn(0x7f4020071600 msgr2=0x7f40200d2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.537+0000 7f40281ff700 1 -- 172.21.15.105:0/1616790278 shutdown_connections 2023-07-13T22:20:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.537+0000 7f40281ff700 1 -- 172.21.15.105:0/1616790278 wait complete. 2023-07-13T22:20:35.539 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new d56053c2-fa6e-465f-944f-7b7c268b6d96 3 2023-07-13T22:20:35.702 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:35.717+0000 7f1d0e646200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-13T22:20:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.755+0000 7f87e23ff700 1 Processor -- start 2023-07-13T22:20:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.755+0000 7f87e23ff700 1 -- start start 2023-07-13T22:20:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87e23ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87dc0b8200 0x7f87dc0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87e23ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87e23ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87dc0cede0 0x7f87dc15ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87e23ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0d1870 con 0x7f87dc0b8200 2023-07-13T22:20:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87e23ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0cf370 con 0x7f87dc0cede0 2023-07-13T22:20:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87e23ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0d12b0 con 0x7f87dc0b1ac0 2023-07-13T22:20:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87e099c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87dc0cede0 0x7f87dc15ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87dbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87dc0b8200 0x7f87dc0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35834/0 (socket says 172.21.15.105:35834) 2023-07-13T22:20:35.743 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.756+0000 7f87db7fe700 1 -- 172.21.15.105:0/2186363029 learned_addr learned my addr 172.21.15.105:0/2186363029 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.757+0000 7f87db7fe700 1 -- 172.21.15.105:0/2186363029 >> v2:172.21.15.105:3300/0 conn(0x7f87dc0cede0 msgr2=0x7f87dc15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.757+0000 7f87db7fe700 1 --2- 172.21.15.105:0/2186363029 >> v2:172.21.15.105:3300/0 conn(0x7f87dc0cede0 0x7f87dc15ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.757+0000 7f87db7fe700 1 -- 172.21.15.105:0/2186363029 >> v2:172.21.15.60:3300/0 conn(0x7f87dc0b8200 msgr2=0x7f87dc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.757+0000 7f87db7fe700 1 --2- 172.21.15.105:0/2186363029 >> v2:172.21.15.60:3300/0 conn(0x7f87dc0b8200 0x7f87dc0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.757+0000 7f87db7fe700 1 -- 172.21.15.105:0/2186363029 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0d1590 con 0x7f87dc0b1ac0 2023-07-13T22:20:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.757+0000 7f87db7fe700 1 --2- 172.21.15.105:0/2186363029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0cfb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0b8d70 tx=0x7f87d0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f0ae8e62e71ed6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.757+0000 7f87daffd700 1 -- 172.21.15.105:0/2186363029 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0011830 con 0x7f87dc0b1ac0 2023-07-13T22:20:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.757+0000 7f87daffd700 1 -- 172.21.15.105:0/2186363029 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d0011990 con 0x7f87dc0b1ac0 2023-07-13T22:20:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.757+0000 7f87daffd700 1 -- 172.21.15.105:0/2186363029 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0011d20 con 0x7f87dc0b1ac0 2023-07-13T22:20:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.758+0000 7f87e23ff700 1 -- 172.21.15.105:0/2186363029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 msgr2=0x7f87dc0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.758+0000 7f87e23ff700 1 --2- 172.21.15.105:0/2186363029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0cfb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0b8d70 tx=0x7f87d0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.758+0000 7f87e23ff700 1 -- 172.21.15.105:0/2186363029 shutdown_connections 2023-07-13T22:20:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.758+0000 7f87e23ff700 1 --2- 172.21.15.105:0/2186363029 >> v2:172.21.15.105:3300/0 conn(0x7f87dc0cede0 0x7f87dc15ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.758+0000 7f87e23ff700 1 --2- 172.21.15.105:0/2186363029 >> v2:172.21.15.60:3300/0 conn(0x7f87dc0b8200 0x7f87dc0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.758+0000 7f87e23ff700 1 --2- 172.21.15.105:0/2186363029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0cfb80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.758+0000 7f87e23ff700 1 -- 172.21.15.105:0/2186363029 >> 172.21.15.105:0/2186363029 conn(0x7f87dc06b2f0 msgr2=0x7f87dc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.758+0000 7f87e23ff700 1 -- 172.21.15.105:0/2186363029 shutdown_connections 2023-07-13T22:20:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.758+0000 7f87e23ff700 1 -- 172.21.15.105:0/2186363029 wait complete. 2023-07-13T22:20:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.761+0000 7f87e23ff700 1 Processor -- start 2023-07-13T22:20:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.761+0000 7f87e23ff700 1 -- start start 2023-07-13T22:20:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e23ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e23ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87dc0b8200 0x7f87dc0cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e23ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc0cede0 0x7f87dc0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e23ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc155760 con 0x7f87dc0b8200 2023-07-13T22:20:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e23ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0c91e0 con 0x7f87dc0cede0 2023-07-13T22:20:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e23ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0c7c00 con 0x7f87dc0b1ac0 2023-07-13T22:20:35.749 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e099c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc0cede0 0x7f87dc0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87dc0b8200 0x7f87dc0cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e099c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc0cede0 0x7f87dc0c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49344/0 (socket says 172.21.15.105:49344) 2023-07-13T22:20:35.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35840/0 (socket says 172.21.15.105:35840) 2023-07-13T22:20:35.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e099c700 1 -- 172.21.15.105:0/3117379891 learned_addr learned my addr 172.21.15.105:0/3117379891 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:35.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e099c700 1 -- 172.21.15.105:0/3117379891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 msgr2=0x7f87dc0c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.750 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e099c700 1 --2- 172.21.15.105:0/3117379891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e099c700 1 -- 172.21.15.105:0/3117379891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87dc0b8200 msgr2=0x7f87dc0cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e099c700 1 --2- 172.21.15.105:0/3117379891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87dc0b8200 0x7f87dc0cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87e099c700 1 -- 172.21.15.105:0/3117379891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0009550 con 0x7f87dc0cede0 2023-07-13T22:20:35.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.763+0000 7f87dbfff700 1 --2- 172.21.15.105:0/3117379891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:35.751 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.764+0000 7f87e099c700 1 --2- 172.21.15.105:0/3117379891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc0cede0 0x7f87dc0c9f70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f87d400efc0 tx=0x7f87d400c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.764+0000 7f87d8ff9700 1 -- 172.21.15.105:0/3117379891 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400cad0 con 0x7f87dc0cede0 2023-07-13T22:20:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.764+0000 7f87d8ff9700 1 -- 172.21.15.105:0/3117379891 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d400f040 con 0x7f87dc0cede0 2023-07-13T22:20:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.764+0000 7f87d8ff9700 1 -- 172.21.15.105:0/3117379891 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400ce70 con 0x7f87dc0cede0 2023-07-13T22:20:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.764+0000 7f87e23ff700 1 -- 172.21.15.105:0/3117379891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc0c7ee0 con 0x7f87dc0cede0 2023-07-13T22:20:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.764+0000 7f87d8ff9700 1 -- 172.21.15.105:0/3117379891 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f87d4014780 con 0x7f87dc0cede0 2023-07-13T22:20:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.764+0000 7f87e23ff700 1 -- 172.21.15.105:0/3117379891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc0c84a0 con 0x7f87dc0cede0 2023-07-13T22:20:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.765+0000 7f87d8ff9700 1 -- 172.21.15.105:0/3117379891 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f87d40133e0 con 0x7f87dc0cede0 2023-07-13T22:20:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.765+0000 7f87e23ff700 1 -- 172.21.15.105:0/3117379891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc04e210 con 0x7f87dc0cede0 2023-07-13T22:20:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.768+0000 7f87d8ff9700 1 -- 172.21.15.105:0/3117379891 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f87d4018020 con 0x7f87dc0cede0 2023-07-13T22:20:35.829 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:35.844+0000 7f95c665e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-13T22:20:35.839 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.854+0000 7f87e23ff700 1 -- 172.21.15.105:0/3117379891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d56053c2-fa6e-465f-944f-7b7c268b6d96", "id": 3} v 0) v1 -- 0x7f87dc0b4f70 con 0x7f87dc0cede0 2023-07-13T22:20:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.858+0000 7f87d8ff9700 1 -- 172.21.15.105:0/3117379891 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d56053c2-fa6e-465f-944f-7b7c268b6d96", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f87d4014b00 con 0x7f87dc0cede0 2023-07-13T22:20:35.844 INFO:teuthology.orchestra.run.smithi105.stdout:3 2023-07-13T22:20:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.859+0000 7f87e23ff700 1 -- 172.21.15.105:0/3117379891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc0cede0 msgr2=0x7f87dc0c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.859+0000 7f87e23ff700 1 --2- 172.21.15.105:0/3117379891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc0cede0 0x7f87dc0c9f70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f87d400efc0 tx=0x7f87d400c0c0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.859+0000 7f87e23ff700 1 -- 172.21.15.105:0/3117379891 shutdown_connections 2023-07-13T22:20:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.859+0000 7f87e23ff700 1 --2- 172.21.15.105:0/3117379891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87dc0cede0 0x7f87dc0c9f70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.859+0000 7f87e23ff700 1 --2- 172.21.15.105:0/3117379891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87dc0b1ac0 0x7f87dc0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.859+0000 7f87e23ff700 1 --2- 172.21.15.105:0/3117379891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87dc0b8200 0x7f87dc0cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.859+0000 7f87e23ff700 1 -- 172.21.15.105:0/3117379891 >> 172.21.15.105:0/3117379891 conn(0x7f87dc06b2f0 msgr2=0x7f87dc0b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.859+0000 7f87e23ff700 1 -- 172.21.15.105:0/3117379891 shutdown_connections 2023-07-13T22:20:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:35.859+0000 7f87e23ff700 1 -- 172.21.15.105:0/3117379891 wait complete. 2023-07-13T22:20:35.863 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 63d94abf-4298-48ad-987c-77af756648eb 4 2023-07-13T22:20:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.070+0000 7f98803ff700 1 Processor -- start 2023-07-13T22:20:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.070+0000 7f98803ff700 1 -- start start 2023-07-13T22:20:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f98803ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780cfd50 0x7f98780d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f98803ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98780d1870 con 0x7f98780b8200 2023-07-13T22:20:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f98803ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98780cf370 con 0x7f98780b1ac0 2023-07-13T22:20:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f98803ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98780d12b0 con 0x7f98780cfd50 2023-07-13T22:20:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f987d99a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780cfd50 0x7f98780d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f987e99c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f98780b1ac0 0x7f98780d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51056/0 (socket says 172.21.15.105:51056) 2023-07-13T22:20:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f987e99c700 1 -- 172.21.15.105:0/1880358077 learned_addr learned my addr 172.21.15.105:0/1880358077 (peer_addr_for_me v1:172.21.15.105:0/0) 2023-07-13T22:20:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f987d99a700 1 -- 172.21.15.105:0/1880358077 >> v1:172.21.15.105:6789/0 conn(0x7f98780b1ac0 legacy=0x7f98780d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f987d199700 1 -- 172.21.15.105:0/1880358077 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443479037 0 0) 0x7f98780cf370 con 0x7f98780b1ac0 2023-07-13T22:20:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f987d99a700 1 -- 172.21.15.105:0/1880358077 >> v1:172.21.15.60:6789/0 conn(0x7f98780b8200 legacy=0x7f98780cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.072+0000 7f987d99a700 1 -- 172.21.15.105:0/1880358077 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780d1590 con 0x7f98780cfd50 2023-07-13T22:20:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.073+0000 7f987d99a700 1 --2- 172.21.15.105:0/1880358077 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780cfd50 0x7f98780d23c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f98680098a0 tx=0x7f9868009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968dfd32ad9eafb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.073+0000 7f987d199700 1 -- 172.21.15.105:0/1880358077 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868004230 con 0x7f98780cfd50 2023-07-13T22:20:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.073+0000 7f987d199700 1 -- 172.21.15.105:0/1880358077 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9868004390 con 0x7f98780cfd50 2023-07-13T22:20:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.073+0000 7f987d199700 1 -- 172.21.15.105:0/1880358077 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98680036a0 con 0x7f98780cfd50 2023-07-13T22:20:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.073+0000 7f98803ff700 1 -- 172.21.15.105:0/1880358077 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780cfd50 msgr2=0x7f98780d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.073+0000 7f98803ff700 1 --2- 172.21.15.105:0/1880358077 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780cfd50 0x7f98780d23c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f98680098a0 tx=0x7f9868009bb0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.074+0000 7f98803ff700 1 -- 172.21.15.105:0/1880358077 shutdown_connections 2023-07-13T22:20:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.074+0000 7f98803ff700 1 --2- 172.21.15.105:0/1880358077 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780cfd50 0x7f98780d23c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.074+0000 7f98803ff700 1 -- 172.21.15.105:0/1880358077 >> 172.21.15.105:0/1880358077 conn(0x7f987806b2f0 msgr2=0x7f987806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.074+0000 7f98803ff700 1 -- 172.21.15.105:0/1880358077 shutdown_connections 2023-07-13T22:20:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.074+0000 7f98803ff700 1 -- 172.21.15.105:0/1880358077 wait complete. 2023-07-13T22:20:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.075+0000 7f98803ff700 1 Processor -- start 2023-07-13T22:20:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.075+0000 7f98803ff700 1 -- start start 2023-07-13T22:20:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.076+0000 7f98803ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98780b1ac0 0x7f9878156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.076+0000 7f98803ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780b8200 0x7f98780cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.076+0000 7f98803ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98780cfd50 0x7f98780cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.076+0000 7f98803ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98780cb420 con 0x7f98780cfd50 2023-07-13T22:20:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.076+0000 7f98803ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98780b74f0 con 0x7f98780b1ac0 2023-07-13T22:20:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.076+0000 7f98803ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98780b77d0 con 0x7f98780b8200 2023-07-13T22:20:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987d99a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780b8200 0x7f98780cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987e19b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98780b1ac0 0x7f9878156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987d99a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780b8200 0x7f98780cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35856/0 (socket says 172.21.15.105:35856) 2023-07-13T22:20:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987d99a700 1 -- 172.21.15.105:0/1224511603 learned_addr learned my addr 172.21.15.105:0/1224511603 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987e99c700 1 --2- 172.21.15.105:0/1224511603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98780cfd50 0x7f98780cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987e19b700 1 -- 172.21.15.105:0/1224511603 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780b8200 msgr2=0x7f98780cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987e19b700 1 --2- 172.21.15.105:0/1224511603 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780b8200 0x7f98780cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987e19b700 1 -- 172.21.15.105:0/1224511603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98780cfd50 msgr2=0x7f98780cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987e19b700 1 --2- 172.21.15.105:0/1224511603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98780cfd50 0x7f98780cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987e19b700 1 -- 172.21.15.105:0/1224511603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680094a0 con 0x7f98780b1ac0 2023-07-13T22:20:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.077+0000 7f987e19b700 1 --2- 172.21.15.105:0/1224511603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98780b1ac0 0x7f9878156370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f987400c750 tx=0x7f987400ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.078+0000 7f986effd700 1 -- 172.21.15.105:0/1224511603 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98740054e0 con 0x7f98780b1ac0 2023-07-13T22:20:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.078+0000 7f986effd700 1 -- 172.21.15.105:0/1224511603 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9874003680 con 0x7f98780b1ac0 2023-07-13T22:20:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.078+0000 7f986effd700 1 -- 172.21.15.105:0/1224511603 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874003a30 con 0x7f98780b1ac0 2023-07-13T22:20:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.078+0000 7f98803ff700 1 -- 172.21.15.105:0/1224511603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98780cb670 con 0x7f98780b1ac0 2023-07-13T22:20:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.078+0000 7f98803ff700 1 -- 172.21.15.105:0/1224511603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98781568b0 con 0x7f98780b1ac0 2023-07-13T22:20:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.078+0000 7f986effd700 1 -- 172.21.15.105:0/1224511603 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9874006840 con 0x7f98780b1ac0 2023-07-13T22:20:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.078+0000 7f986effd700 1 -- 172.21.15.105:0/1224511603 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9874006af0 con 0x7f98780b1ac0 2023-07-13T22:20:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.079+0000 7f98803ff700 1 -- 172.21.15.105:0/1224511603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98780cede0 con 0x7f98780b1ac0 2023-07-13T22:20:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.081+0000 7f986effd700 1 -- 172.21.15.105:0/1224511603 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9874017070 con 0x7f98780b1ac0 2023-07-13T22:20:36.153 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.167+0000 7f98803ff700 1 -- 172.21.15.105:0/1224511603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "63d94abf-4298-48ad-987c-77af756648eb", "id": 4} v 0) v1 -- 0x7f987804e210 con 0x7f98780b1ac0 2023-07-13T22:20:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.171+0000 7f986effd700 1 -- 172.21.15.105:0/1224511603 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "63d94abf-4298-48ad-987c-77af756648eb", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9874006d40 con 0x7f98780b1ac0 2023-07-13T22:20:36.157 INFO:teuthology.orchestra.run.smithi105.stdout:4 2023-07-13T22:20:36.158 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.172+0000 7f98803ff700 1 -- 172.21.15.105:0/1224511603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98780b1ac0 msgr2=0x7f9878156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.172+0000 7f98803ff700 1 --2- 172.21.15.105:0/1224511603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98780b1ac0 0x7f9878156370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f987400c750 tx=0x7f987400ca60 comp rx=0 tx=0).stop 2023-07-13T22:20:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.172+0000 7f98803ff700 1 -- 172.21.15.105:0/1224511603 shutdown_connections 2023-07-13T22:20:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.172+0000 7f98803ff700 1 --2- 172.21.15.105:0/1224511603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98780cfd50 0x7f98780cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.172+0000 7f98803ff700 1 --2- 172.21.15.105:0/1224511603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98780b1ac0 0x7f9878156370 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.172+0000 7f98803ff700 1 --2- 172.21.15.105:0/1224511603 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98780b8200 0x7f98780cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.172+0000 7f98803ff700 1 -- 172.21.15.105:0/1224511603 >> 172.21.15.105:0/1224511603 conn(0x7f987806b2f0 msgr2=0x7f98780b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.172+0000 7f98803ff700 1 -- 172.21.15.105:0/1224511603 shutdown_connections 2023-07-13T22:20:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.172+0000 7f98803ff700 1 -- 172.21.15.105:0/1224511603 wait complete. 2023-07-13T22:20:36.174 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new e3017c6e-22fe-4dc0-83b5-1bfa67440281 5 2023-07-13T22:20:36.179 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:36.194+0000 7f1d0e646200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-13T22:20:36.364 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.379+0000 7f64e55ff700 1 Processor -- start 2023-07-13T22:20:36.365 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.379+0000 7f64e55ff700 1 -- start start 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.380+0000 7f64e55ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64e015c530 0x7f64e00b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.380+0000 7f64e55ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e00cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.380+0000 7f64e55ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64e00cf9f0 0x7f64e00d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.380+0000 7f64e55ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e00cefd0 con 0x7f64e015c530 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.380+0000 7f64e55ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e00d1780 con 0x7f64e00cf9f0 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.380+0000 7f64e55ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00cd2d0 con 0x7f64e00d11c0 2023-07-13T22:20:36.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64de7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e00cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64df7fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64e00cf9f0 0x7f64e00d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64deffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64e015c530 0x7f64e00b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64de7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e00cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35864/0 (socket says 172.21.15.105:35864) 2023-07-13T22:20:36.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64df7fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64e00cf9f0 0x7f64e00d2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49376/0 (socket says 172.21.15.105:49376) 2023-07-13T22:20:36.367 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64de7fc700 1 -- 172.21.15.105:0/2851034864 learned_addr learned my addr 172.21.15.105:0/2851034864 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64de7fc700 1 -- 172.21.15.105:0/2851034864 >> v2:172.21.15.105:3300/0 conn(0x7f64e00cf9f0 msgr2=0x7f64e00d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64de7fc700 1 --2- 172.21.15.105:0/2851034864 >> v2:172.21.15.105:3300/0 conn(0x7f64e00cf9f0 0x7f64e00d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64de7fc700 1 -- 172.21.15.105:0/2851034864 >> v2:172.21.15.60:3300/0 conn(0x7f64e015c530 msgr2=0x7f64e00b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64de7fc700 1 --2- 172.21.15.105:0/2851034864 >> v2:172.21.15.60:3300/0 conn(0x7f64e015c530 0x7f64e00b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64de7fc700 1 -- 172.21.15.105:0/2851034864 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00cd5b0 con 0x7f64e00d11c0 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.381+0000 7f64de7fc700 1 --2- 172.21.15.105:0/2851034864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e00cf4b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f64e00b4660 tx=0x7f64d4009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18dcad859c9615e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64ddffb700 1 -- 172.21.15.105:0/2851034864 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4011830 con 0x7f64e00d11c0 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64ddffb700 1 -- 172.21.15.105:0/2851034864 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d4011990 con 0x7f64e00d11c0 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64ddffb700 1 -- 172.21.15.105:0/2851034864 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4011d20 con 0x7f64e00d11c0 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64e55ff700 1 -- 172.21.15.105:0/2851034864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 msgr2=0x7f64e00cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64e55ff700 1 --2- 172.21.15.105:0/2851034864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e00cf4b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f64e00b4660 tx=0x7f64d4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64e55ff700 1 -- 172.21.15.105:0/2851034864 shutdown_connections 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64e55ff700 1 --2- 172.21.15.105:0/2851034864 >> v2:172.21.15.105:3300/0 conn(0x7f64e00cf9f0 0x7f64e00d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64e55ff700 1 --2- 172.21.15.105:0/2851034864 >> v2:172.21.15.60:3300/0 conn(0x7f64e015c530 0x7f64e00b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64e55ff700 1 --2- 172.21.15.105:0/2851034864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e00cf4b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64e55ff700 1 -- 172.21.15.105:0/2851034864 >> 172.21.15.105:0/2851034864 conn(0x7f64e006b400 msgr2=0x7f64e006b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64e55ff700 1 -- 172.21.15.105:0/2851034864 shutdown_connections 2023-07-13T22:20:36.371 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.382+0000 7f64e55ff700 1 -- 172.21.15.105:0/2851034864 wait complete. 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.384+0000 7f64e55ff700 1 Processor -- start 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.384+0000 7f64e55ff700 1 -- start start 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64e55ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00cf9f0 0x7f64e0159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64e55ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64e55ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0156ce0 0x7f64e0156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64e55ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01592e0 con 0x7f64e0156ce0 2023-07-13T22:20:36.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64e55ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0158410 con 0x7f64e00cf9f0 2023-07-13T22:20:36.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64e55ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01e70f0 con 0x7f64e00d11c0 2023-07-13T22:20:36.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64de7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64deffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00cf9f0 0x7f64e0159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64de7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e015b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35868/0 (socket says 172.21.15.105:35868) 2023-07-13T22:20:36.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64de7fc700 1 -- 172.21.15.105:0/3056982130 learned_addr learned my addr 172.21.15.105:0/3056982130 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:36.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.385+0000 7f64df7fe700 1 --2- 172.21.15.105:0/3056982130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0156ce0 0x7f64e0156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.374 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64deffd700 1 -- 172.21.15.105:0/3056982130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 msgr2=0x7f64e015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64deffd700 1 --2- 172.21.15.105:0/3056982130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e015b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64deffd700 1 -- 172.21.15.105:0/3056982130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0156ce0 msgr2=0x7f64e0156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64deffd700 1 --2- 172.21.15.105:0/3056982130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0156ce0 0x7f64e0156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64deffd700 1 -- 172.21.15.105:0/3056982130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d4009550 con 0x7f64e00cf9f0 2023-07-13T22:20:36.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64de7fc700 1 --2- 172.21.15.105:0/3056982130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:36.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64deffd700 1 --2- 172.21.15.105:0/3056982130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00cf9f0 0x7f64e0159700 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e8a0 tx=0x7f64d000ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64c77fe700 1 -- 172.21.15.105:0/3056982130 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0013680 con 0x7f64e00cf9f0 2023-07-13T22:20:36.375 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64c77fe700 1 -- 172.21.15.105:0/3056982130 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d00137e0 con 0x7f64e00cf9f0 2023-07-13T22:20:36.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64e55ff700 1 -- 172.21.15.105:0/3056982130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e0158660 con 0x7f64e00cf9f0 2023-07-13T22:20:36.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64c77fe700 1 -- 172.21.15.105:0/3056982130 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0013b70 con 0x7f64e00cf9f0 2023-07-13T22:20:36.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.386+0000 7f64e55ff700 1 -- 172.21.15.105:0/3056982130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e0158c20 con 0x7f64e00cf9f0 2023-07-13T22:20:36.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.387+0000 7f64c77fe700 1 -- 172.21.15.105:0/3056982130 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f64d0005a20 con 0x7f64e00cf9f0 2023-07-13T22:20:36.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.387+0000 7f64c77fe700 1 -- 172.21.15.105:0/3056982130 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f64d0005cd0 con 0x7f64e00cf9f0 2023-07-13T22:20:36.376 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.387+0000 7f64e55ff700 1 -- 172.21.15.105:0/3056982130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e004e210 con 0x7f64e00cf9f0 2023-07-13T22:20:36.377 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.390+0000 7f64c77fe700 1 -- 172.21.15.105:0/3056982130 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f64d0021060 con 0x7f64e00cf9f0 2023-07-13T22:20:36.460 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.475+0000 7f64e55ff700 1 -- 172.21.15.105:0/3056982130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e3017c6e-22fe-4dc0-83b5-1bfa67440281", "id": 5} v 0) v1 -- 0x7f64e00d2500 con 0x7f64e00cf9f0 2023-07-13T22:20:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.479+0000 7f64c77fe700 1 -- 172.21.15.105:0/3056982130 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e3017c6e-22fe-4dc0-83b5-1bfa67440281", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f64d0029020 con 0x7f64e00cf9f0 2023-07-13T22:20:36.464 INFO:teuthology.orchestra.run.smithi105.stdout:5 2023-07-13T22:20:36.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.480+0000 7f64e55ff700 1 -- 172.21.15.105:0/3056982130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00cf9f0 msgr2=0x7f64e0159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.465 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.480+0000 7f64e55ff700 1 --2- 172.21.15.105:0/3056982130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00cf9f0 0x7f64e0159700 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e8a0 tx=0x7f64d000ebb0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.480+0000 7f64e55ff700 1 -- 172.21.15.105:0/3056982130 shutdown_connections 2023-07-13T22:20:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.480+0000 7f64e55ff700 1 --2- 172.21.15.105:0/3056982130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e0156ce0 0x7f64e0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.480+0000 7f64e55ff700 1 --2- 172.21.15.105:0/3056982130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00cf9f0 0x7f64e0159700 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.480+0000 7f64e55ff700 1 --2- 172.21.15.105:0/3056982130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d11c0 0x7f64e015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.480+0000 7f64e55ff700 1 -- 172.21.15.105:0/3056982130 >> 172.21.15.105:0/3056982130 conn(0x7f64e006b400 msgr2=0x7f64e00d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.480+0000 7f64e55ff700 1 -- 172.21.15.105:0/3056982130 shutdown_connections 2023-07-13T22:20:36.466 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.480+0000 7f64e55ff700 1 -- 172.21.15.105:0/3056982130 wait complete. 2023-07-13T22:20:36.482 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 3650aa71-6ed6-4187-aa47-1b8ab38c7158 6 2023-07-13T22:20:36.690 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:36.706+0000 7f95c665e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-13T22:20:36.722 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.737+0000 7fb0123ff700 1 Processor -- start 2023-07-13T22:20:36.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.737+0000 7fb0123ff700 1 -- start start 2023-07-13T22:20:36.723 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb0123ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb00c0b8200 0x7fb00c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb0123ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b1ac0 0x7fb00c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb0123ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb00c0cede0 0x7fb00c15ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb0123ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c0d1870 con 0x7fb00c0b8200 2023-07-13T22:20:36.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb0123ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c0cf370 con 0x7fb00c0cede0 2023-07-13T22:20:36.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb0123ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0d12b0 con 0x7fb00c0b1ac0 2023-07-13T22:20:36.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb003fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b1ac0 0x7fb00c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.724 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb01099c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb00c0cede0 0x7fb00c15ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb00bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb00c0b8200 0x7fb00c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb01099c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb00c0cede0 0x7fb00c15ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49404/0 (socket says 172.21.15.105:49404) 2023-07-13T22:20:36.725 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.738+0000 7fb01099c700 1 -- 172.21.15.105:0/2795275132 learned_addr learned my addr 172.21.15.105:0/2795275132 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:36.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb01099c700 1 -- 172.21.15.105:0/2795275132 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b1ac0 msgr2=0x7fb00c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb01099c700 1 --2- 172.21.15.105:0/2795275132 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b1ac0 0x7fb00c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.726 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb01099c700 1 -- 172.21.15.105:0/2795275132 >> v2:172.21.15.60:3300/0 conn(0x7fb00c0b8200 msgr2=0x7fb00c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb01099c700 1 --2- 172.21.15.105:0/2795275132 >> v2:172.21.15.60:3300/0 conn(0x7fb00c0b8200 0x7fb00c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb01099c700 1 -- 172.21.15.105:0/2795275132 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c0d1590 con 0x7fb00c0cede0 2023-07-13T22:20:36.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb003fff700 1 --2- 172.21.15.105:0/2795275132 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b1ac0 0x7fb00c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:36.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb01099c700 1 --2- 172.21.15.105:0/2795275132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0cede0 0x7fb00c15ff60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0b8d70 tx=0x7fb00400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f29edc156ca163c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb00b7fe700 1 -- 172.21.15.105:0/2795275132 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400e070 con 0x7fb00c0cede0 2023-07-13T22:20:36.727 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb00b7fe700 1 -- 172.21.15.105:0/2795275132 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00400b910 con 0x7fb00c0cede0 2023-07-13T22:20:36.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.739+0000 7fb00b7fe700 1 -- 172.21.15.105:0/2795275132 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400bcc0 con 0x7fb00c0cede0 2023-07-13T22:20:36.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.740+0000 7fb0123ff700 1 -- 172.21.15.105:0/2795275132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0cede0 msgr2=0x7fb00c15ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.740+0000 7fb0123ff700 1 --2- 172.21.15.105:0/2795275132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0cede0 0x7fb00c15ff60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0b8d70 tx=0x7fb00400b330 comp rx=0 tx=0).stop 2023-07-13T22:20:36.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.740+0000 7fb0123ff700 1 -- 172.21.15.105:0/2795275132 shutdown_connections 2023-07-13T22:20:36.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.740+0000 7fb0123ff700 1 --2- 172.21.15.105:0/2795275132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0cede0 0x7fb00c15ff60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.740+0000 7fb0123ff700 1 --2- 172.21.15.105:0/2795275132 >> v2:172.21.15.60:3300/0 conn(0x7fb00c0b8200 0x7fb00c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.728 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.740+0000 7fb0123ff700 1 --2- 172.21.15.105:0/2795275132 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b1ac0 0x7fb00c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.740+0000 7fb0123ff700 1 -- 172.21.15.105:0/2795275132 >> 172.21.15.105:0/2795275132 conn(0x7fb00c06b2f0 msgr2=0x7fb00c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.740+0000 7fb0123ff700 1 -- 172.21.15.105:0/2795275132 shutdown_connections 2023-07-13T22:20:36.729 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.740+0000 7fb0123ff700 1 -- 172.21.15.105:0/2795275132 wait complete. 2023-07-13T22:20:36.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.742+0000 7fb0123ff700 1 Processor -- start 2023-07-13T22:20:36.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.742+0000 7fb0123ff700 1 -- start start 2023-07-13T22:20:36.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb0123ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0b1ac0 0x7fb00c1e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb0123ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b8200 0x7fb00c0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb0123ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cede0 0x7fb00c1e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.731 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb0123ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c155fe0 con 0x7fb00c0cede0 2023-07-13T22:20:36.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb0123ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1e4090 con 0x7fb00c0b1ac0 2023-07-13T22:20:36.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb0123ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0c9080 con 0x7fb00c0b8200 2023-07-13T22:20:36.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb003fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b8200 0x7fb00c0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb00bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0b1ac0 0x7fb00c1e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb003fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b8200 0x7fb00c0c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:35880/0 (socket says 172.21.15.105:35880) 2023-07-13T22:20:36.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb003fff700 1 -- 172.21.15.105:0/3016235706 learned_addr learned my addr 172.21.15.105:0/3016235706 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:36.732 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.743+0000 7fb01099c700 1 --2- 172.21.15.105:0/3016235706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cede0 0x7fb00c1e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb003fff700 1 -- 172.21.15.105:0/3016235706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0b1ac0 msgr2=0x7fb00c1e3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb003fff700 1 --2- 172.21.15.105:0/3016235706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0b1ac0 0x7fb00c1e3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb003fff700 1 -- 172.21.15.105:0/3016235706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cede0 msgr2=0x7fb00c1e2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb003fff700 1 --2- 172.21.15.105:0/3016235706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cede0 0x7fb00c1e2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb003fff700 1 -- 172.21.15.105:0/3016235706 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40094a0 con 0x7fb00c0b8200 2023-07-13T22:20:36.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb003fff700 1 --2- 172.21.15.105:0/3016235706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b8200 0x7fb00c0c8b40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faff400ebe0 tx=0x7faff400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.733 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb0097fa700 1 -- 172.21.15.105:0/3016235706 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff400c850 con 0x7fb00c0b8200 2023-07-13T22:20:36.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb0097fa700 1 -- 172.21.15.105:0/3016235706 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff400c9b0 con 0x7fb00c0b8200 2023-07-13T22:20:36.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb0097fa700 1 -- 172.21.15.105:0/3016235706 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff400cd40 con 0x7fb00c0b8200 2023-07-13T22:20:36.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb0123ff700 1 -- 172.21.15.105:0/3016235706 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb004009b30 con 0x7fb00c0b8200 2023-07-13T22:20:36.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.744+0000 7fb0123ff700 1 -- 172.21.15.105:0/3016235706 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c1e4840 con 0x7fb00c0b8200 2023-07-13T22:20:36.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.745+0000 7fb0097fa700 1 -- 172.21.15.105:0/3016235706 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faff4005d20 con 0x7fb00c0b8200 2023-07-13T22:20:36.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.745+0000 7fb0097fa700 1 -- 172.21.15.105:0/3016235706 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7faff4013a60 con 0x7fb00c0b8200 2023-07-13T22:20:36.734 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.745+0000 7fb0123ff700 1 -- 172.21.15.105:0/3016235706 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c04e210 con 0x7fb00c0b8200 2023-07-13T22:20:36.735 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.748+0000 7fb0097fa700 1 -- 172.21.15.105:0/3016235706 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7faff4021060 con 0x7fb00c0b8200 2023-07-13T22:20:36.787 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:36.802+0000 7f95c665e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-13T22:20:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.833+0000 7fb0123ff700 1 -- 172.21.15.105:0/3016235706 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3650aa71-6ed6-4187-aa47-1b8ab38c7158", "id": 6} v 0) v1 -- 0x7fb00c0b4f70 con 0x7fb00c0b8200 2023-07-13T22:20:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.837+0000 7fb0097fa700 1 -- 172.21.15.105:0/3016235706 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3650aa71-6ed6-4187-aa47-1b8ab38c7158", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7faff4017070 con 0x7fb00c0b8200 2023-07-13T22:20:36.823 INFO:teuthology.orchestra.run.smithi105.stdout:6 2023-07-13T22:20:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.838+0000 7fb0123ff700 1 -- 172.21.15.105:0/3016235706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b8200 msgr2=0x7fb00c0c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.839+0000 7fb0123ff700 1 --2- 172.21.15.105:0/3016235706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b8200 0x7fb00c0c8b40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faff400ebe0 tx=0x7faff400eef0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.839+0000 7fb0123ff700 1 -- 172.21.15.105:0/3016235706 shutdown_connections 2023-07-13T22:20:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.839+0000 7fb0123ff700 1 --2- 172.21.15.105:0/3016235706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cede0 0x7fb00c1e2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.839+0000 7fb0123ff700 1 --2- 172.21.15.105:0/3016235706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0b1ac0 0x7fb00c1e3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.839+0000 7fb0123ff700 1 --2- 172.21.15.105:0/3016235706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0b8200 0x7fb00c0c8b40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.839+0000 7fb0123ff700 1 -- 172.21.15.105:0/3016235706 >> 172.21.15.105:0/3016235706 conn(0x7fb00c06b2f0 msgr2=0x7fb00c0b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.839+0000 7fb0123ff700 1 -- 172.21.15.105:0/3016235706 shutdown_connections 2023-07-13T22:20:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:36.839+0000 7fb0123ff700 1 -- 172.21.15.105:0/3016235706 wait complete. 2023-07-13T22:20:36.842 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 37f5f3b5-f0f5-41f0-88b8-6d9495170cfc 7 2023-07-13T22:20:36.963 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:36.979+0000 7f95c665e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-13T22:20:37.019 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:37.034+0000 7f1d0e646200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-13T22:20:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.049+0000 7f1be57ff700 1 Processor -- start 2023-07-13T22:20:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.050+0000 7f1be57ff700 1 -- start start 2023-07-13T22:20:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1be57ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1be00b8200 0x7f1be00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1be57ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1be00cede0 0x7f1be00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1be57ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be00d1870 con 0x7f1be00b1ac0 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1be57ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be00cf370 con 0x7f1be00cede0 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1be57ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be00d12b0 con 0x7f1be00b8200 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1bdeffd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1be00b8200 0x7f1be00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1bdeffd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1be00b8200 0x7f1be00cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49430/0 (socket says 172.21.15.105:49430) 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1bdeffd700 1 -- 172.21.15.105:0/2023994083 learned_addr learned my addr 172.21.15.105:0/2023994083 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1bdf7fe700 1 --2- 172.21.15.105:0/2023994083 >> v2:172.21.15.60:3300/0 conn(0x7f1be00cede0 0x7f1be00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1bdeffd700 1 -- 172.21.15.105:0/2023994083 >> v2:172.21.15.60:3300/0 conn(0x7f1be00cede0 msgr2=0x7f1be00cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.052+0000 7f1bdeffd700 1 --2- 172.21.15.105:0/2023994083 >> v2:172.21.15.60:3300/0 conn(0x7f1be00cede0 0x7f1be00cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.052+0000 7f1bdeffd700 1 -- 172.21.15.105:0/2023994083 >> v1:172.21.15.60:6789/0 conn(0x7f1be00b1ac0 legacy=0x7f1be015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.052+0000 7f1bdeffd700 1 -- 172.21.15.105:0/2023994083 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00d1590 con 0x7f1be00b8200 2023-07-13T22:20:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.051+0000 7f1bde7fc700 1 -- 172.21.15.105:0/2023994083 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227461565 0 0) 0x7f1be00d1870 con 0x7f1be00b1ac0 2023-07-13T22:20:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.052+0000 7f1bdeffd700 1 --2- 172.21.15.105:0/2023994083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00b8200 0x7f1be00cf640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1be00b2660 tx=0x7f1bd0009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80d141aa5150ee9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.052+0000 7f1bde7fc700 1 -- 172.21.15.105:0/2023994083 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0011830 con 0x7f1be00b8200 2023-07-13T22:20:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.052+0000 7f1bde7fc700 1 -- 172.21.15.105:0/2023994083 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd0011990 con 0x7f1be00b8200 2023-07-13T22:20:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.052+0000 7f1bde7fc700 1 -- 172.21.15.105:0/2023994083 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0011d20 con 0x7f1be00b8200 2023-07-13T22:20:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.053+0000 7f1be57ff700 1 -- 172.21.15.105:0/2023994083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00b8200 msgr2=0x7f1be00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.053+0000 7f1be57ff700 1 --2- 172.21.15.105:0/2023994083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00b8200 0x7f1be00cf640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1be00b2660 tx=0x7f1bd0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.053+0000 7f1be57ff700 1 -- 172.21.15.105:0/2023994083 shutdown_connections 2023-07-13T22:20:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.053+0000 7f1be57ff700 1 --2- 172.21.15.105:0/2023994083 >> v2:172.21.15.60:3300/0 conn(0x7f1be00cede0 0x7f1be00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.053+0000 7f1be57ff700 1 --2- 172.21.15.105:0/2023994083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00b8200 0x7f1be00cf640 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.053+0000 7f1be57ff700 1 -- 172.21.15.105:0/2023994083 >> 172.21.15.105:0/2023994083 conn(0x7f1be006b2f0 msgr2=0x7f1be006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.053+0000 7f1be57ff700 1 -- 172.21.15.105:0/2023994083 shutdown_connections 2023-07-13T22:20:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.053+0000 7f1be57ff700 1 -- 172.21.15.105:0/2023994083 wait complete. 2023-07-13T22:20:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.055+0000 7f1be57ff700 1 Processor -- start 2023-07-13T22:20:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.055+0000 7f1be57ff700 1 -- start start 2023-07-13T22:20:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1be57ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be00b1ac0 0x7f1be01e1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1be57ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1be00b8200 0x7f1be00b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1be57ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00cede0 0x7f1be00b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1be57ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be01e4c40 con 0x7f1be00b1ac0 2023-07-13T22:20:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1be57ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be014aec0 con 0x7f1be00cede0 2023-07-13T22:20:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1be57ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be014a2b0 con 0x7f1be00b8200 2023-07-13T22:20:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdf7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00cede0 0x7f1be00b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bd7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1be00b8200 0x7f1be00b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be00b1ac0 0x7f1be01e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be00b1ac0 0x7f1be01e1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.105:47816/0 (socket says 172.21.15.105:47816) 2023-07-13T22:20:37.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdf7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00cede0 0x7f1be00b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49438/0 (socket says 172.21.15.105:49438) 2023-07-13T22:20:37.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdeffd700 1 -- 172.21.15.105:0/3495204266 learned_addr learned my addr 172.21.15.105:0/3495204266 (peer_addr_for_me v2:172.21.15.105:0/0) 2023-07-13T22:20:37.049 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdf7fe700 1 -- 172.21.15.105:0/3495204266 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1be00b8200 msgr2=0x7f1be00b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdf7fe700 1 --2- 172.21.15.105:0/3495204266 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1be00b8200 0x7f1be00b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdf7fe700 1 -- 172.21.15.105:0/3495204266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be00b1ac0 msgr2=0x7f1be01e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdf7fe700 1 --2- 172.21.15.105:0/3495204266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be00b1ac0 0x7f1be01e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.056+0000 7f1bdf7fe700 1 -- 172.21.15.105:0/3495204266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd0009550 con 0x7f1be00cede0 2023-07-13T22:20:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.057+0000 7f1bdf7fe700 1 --2- 172.21.15.105:0/3495204266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00cede0 0x7f1be00b6c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800efc0 tx=0x7f1bd800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.057+0000 7f1bd77fe700 1 -- 172.21.15.105:0/3495204266 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800cc90 con 0x7f1be00cede0 2023-07-13T22:20:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.057+0000 7f1bd77fe700 1 -- 172.21.15.105:0/3495204266 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd800f040 con 0x7f1be00cede0 2023-07-13T22:20:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.057+0000 7f1bd77fe700 1 -- 172.21.15.105:0/3495204266 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80146c0 con 0x7f1be00cede0 2023-07-13T22:20:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.057+0000 7f1be57ff700 1 -- 172.21.15.105:0/3495204266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be014a590 con 0x7f1be00cede0 2023-07-13T22:20:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.057+0000 7f1be57ff700 1 -- 172.21.15.105:0/3495204266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be014ab50 con 0x7f1be00cede0 2023-07-13T22:20:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.057+0000 7f1bd77fe700 1 -- 172.21.15.105:0/3495204266 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1bd8014b20 con 0x7f1be00cede0 2023-07-13T22:20:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.057+0000 7f1bd77fe700 1 -- 172.21.15.105:0/3495204266 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f1bd8014dd0 con 0x7f1be00cede0 2023-07-13T22:20:37.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.058+0000 7f1be57ff700 1 -- 172.21.15.105:0/3495204266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be0147eb0 con 0x7f1be00cede0 2023-07-13T22:20:37.053 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.060+0000 7f1bd77fe700 1 -- 172.21.15.105:0/3495204266 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f1bd803bb20 con 0x7f1be00cede0 2023-07-13T22:20:37.056 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:37.072+0000 7f95c665e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-13T22:20:37.116 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:37.131+0000 7f1d0e646200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-13T22:20:37.131 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.146+0000 7f1be57ff700 1 -- 172.21.15.105:0/3495204266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "37f5f3b5-f0f5-41f0-88b8-6d9495170cfc", "id": 7} v 0) v1 -- 0x7f1be004e210 con 0x7f1be00cede0 2023-07-13T22:20:37.134 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.149+0000 7f1bd77fe700 1 -- 172.21.15.105:0/3495204266 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37f5f3b5-f0f5-41f0-88b8-6d9495170cfc", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1bd802a030 con 0x7f1be00cede0 2023-07-13T22:20:37.135 INFO:teuthology.orchestra.run.smithi105.stdout:7 2023-07-13T22:20:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.150+0000 7f1be57ff700 1 -- 172.21.15.105:0/3495204266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00cede0 msgr2=0x7f1be00b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.150+0000 7f1be57ff700 1 --2- 172.21.15.105:0/3495204266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00cede0 0x7f1be00b6c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800efc0 tx=0x7f1bd800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.150+0000 7f1be57ff700 1 -- 172.21.15.105:0/3495204266 shutdown_connections 2023-07-13T22:20:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.150+0000 7f1be57ff700 1 --2- 172.21.15.105:0/3495204266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1be00b1ac0 0x7f1be01e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.150+0000 7f1be57ff700 1 --2- 172.21.15.105:0/3495204266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1be00cede0 0x7f1be00b6c00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.136 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.150+0000 7f1be57ff700 1 --2- 172.21.15.105:0/3495204266 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1be00b8200 0x7f1be00b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.150+0000 7f1be57ff700 1 -- 172.21.15.105:0/3495204266 >> 172.21.15.105:0/3495204266 conn(0x7f1be006b2f0 msgr2=0x7f1be00b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.151+0000 7f1be57ff700 1 -- 172.21.15.105:0/3495204266 shutdown_connections 2023-07-13T22:20:37.137 INFO:teuthology.orchestra.run.smithi105.stderr:2023-07-13T22:20:37.151+0000 7f1be57ff700 1 -- 172.21.15.105:0/3495204266 wait complete. 2023-07-13T22:20:37.152 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-13T22:20:37.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-07-13T22:20:37.157 INFO:tasks.ceph.osd.0:Started 2023-07-13T22:20:37.158 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-13T22:20:37.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-07-13T22:20:37.162 INFO:tasks.ceph.osd.1:Started 2023-07-13T22:20:37.162 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-13T22:20:37.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-07-13T22:20:37.167 INFO:tasks.ceph.osd.2:Started 2023-07-13T22:20:37.167 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-13T22:20:37.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-07-13T22:20:37.173 INFO:tasks.ceph.osd.3:Started 2023-07-13T22:20:37.174 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-13T22:20:37.174 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-07-13T22:20:37.178 INFO:tasks.ceph.osd.4:Started 2023-07-13T22:20:37.178 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-13T22:20:37.179 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-07-13T22:20:37.183 INFO:tasks.ceph.osd.5:Started 2023-07-13T22:20:37.183 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-13T22:20:37.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-07-13T22:20:37.187 INFO:tasks.ceph.osd.6:Started 2023-07-13T22:20:37.188 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-13T22:20:37.188 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-07-13T22:20:37.193 INFO:tasks.ceph.osd.7:Started 2023-07-13T22:20:37.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:20:37.317 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:37.332+0000 7f1d0e646200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-13T22:20:37.334 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:37.349+0000 7f95c665e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-13T22:20:37.419 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:37.434+0000 7f1d0e646200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-13T22:20:37.452 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:37.468+0000 7f95c665e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-13T22:20:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.484+0000 7f579abff700 1 Processor -- start 2023-07-13T22:20:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.485+0000 7f579abff700 1 -- start start 2023-07-13T22:20:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.486+0000 7f579abff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f57940d3750 0x7f57940cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.486+0000 7f579abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 0x7f57940d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.486+0000 7f579abff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57940cd950 con 0x7f57940b8400 2023-07-13T22:20:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.486+0000 7f579abff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57940cff50 con 0x7f57940d3750 2023-07-13T22:20:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.486+0000 7f579abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57940d0ac0 con 0x7f57940cfad0 2023-07-13T22:20:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.486+0000 7f5793fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f57940d3750 0x7f57940cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.486+0000 7f579919c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 0x7f57940d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.486+0000 7f5793fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f57940d3750 0x7f57940cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:38970/0 (socket says 172.21.15.60:38970) 2023-07-13T22:20:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.486+0000 7f5793fff700 1 -- 172.21.15.60:0/3155680535 learned_addr learned my addr 172.21.15.60:0/3155680535 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f57937fe700 1 -- 172.21.15.60:0/3155680535 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151177602 0 0) 0x7f5788001110 con 0x7f57940b8400 2023-07-13T22:20:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f57937fe700 1 -- 172.21.15.60:0/3155680535 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57840035b0 con 0x7f57940b8400 2023-07-13T22:20:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f5793fff700 1 -- 172.21.15.60:0/3155680535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 msgr2=0x7f57940d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f5793fff700 1 --2- 172.21.15.60:0/3155680535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 0x7f57940d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f5793fff700 1 -- 172.21.15.60:0/3155680535 >> v1:172.21.15.105:6789/0 conn(0x7f57940b8400 legacy=0x7f57940cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f5793fff700 1 -- 172.21.15.60:0/3155680535 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57940b87d0 con 0x7f57940d3750 2023-07-13T22:20:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f579919c700 1 --2- 172.21.15.60:0/3155680535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 0x7f57940d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f5793fff700 1 --2- 172.21.15.60:0/3155680535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57940d3750 0x7f57940cf590 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f57800098f0 tx=0x7f5780009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5f334c6ec51cdf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f57937fe700 1 -- 172.21.15.60:0/3155680535 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780011830 con 0x7f57940d3750 2023-07-13T22:20:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f57937fe700 1 -- 172.21.15.60:0/3155680535 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780011990 con 0x7f57940d3750 2023-07-13T22:20:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.487+0000 7f57937fe700 1 -- 172.21.15.60:0/3155680535 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780011d20 con 0x7f57940d3750 2023-07-13T22:20:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.488+0000 7f579abff700 1 -- 172.21.15.60:0/3155680535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57940d3750 msgr2=0x7f57940cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.488+0000 7f579abff700 1 --2- 172.21.15.60:0/3155680535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57940d3750 0x7f57940cf590 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f57800098f0 tx=0x7f5780009c00 comp rx=0 tx=0).stop 2023-07-13T22:20:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.488+0000 7f579abff700 1 -- 172.21.15.60:0/3155680535 shutdown_connections 2023-07-13T22:20:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.488+0000 7f579abff700 1 --2- 172.21.15.60:0/3155680535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 0x7f57940d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.488+0000 7f579abff700 1 --2- 172.21.15.60:0/3155680535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57940d3750 0x7f57940cf590 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.488+0000 7f579abff700 1 -- 172.21.15.60:0/3155680535 >> 172.21.15.60:0/3155680535 conn(0x7f579406b390 msgr2=0x7f579406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.488+0000 7f579abff700 1 -- 172.21.15.60:0/3155680535 shutdown_connections 2023-07-13T22:20:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.488+0000 7f579abff700 1 -- 172.21.15.60:0/3155680535 wait complete. 2023-07-13T22:20:37.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.490+0000 7f579abff700 1 Processor -- start 2023-07-13T22:20:37.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.490+0000 7f579abff700 1 -- start start 2023-07-13T22:20:37.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579abff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940b8400 0x7f57941f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 0x7f57941f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579abff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57940d3750 0x7f57941f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579abff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57941f2da0 con 0x7f57940b8400 2023-07-13T22:20:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579abff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57941f5e70 con 0x7f57940d3750 2023-07-13T22:20:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57941f6150 con 0x7f57940cfad0 2023-07-13T22:20:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940b8400 0x7f57941f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940b8400 0x7f57941f2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51582/0 (socket says 172.21.15.60:51582) 2023-07-13T22:20:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579899b700 1 -- 172.21.15.60:0/651943124 learned_addr learned my addr 172.21.15.60:0/651943124 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f5793fff700 1 --2- 172.21.15.60:0/651943124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 0x7f57941f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.491+0000 7f579919c700 1 --2- 172.21.15.60:0/651943124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57940d3750 0x7f57941f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f579899b700 1 -- 172.21.15.60:0/651943124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 msgr2=0x7f57941f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f579899b700 1 --2- 172.21.15.60:0/651943124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 0x7f57941f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f579899b700 1 -- 172.21.15.60:0/651943124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57940d3750 msgr2=0x7f57941f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f579899b700 1 --2- 172.21.15.60:0/651943124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57940d3750 0x7f57941f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f579899b700 1 -- 172.21.15.60:0/651943124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009a70 con 0x7f57940b8400 2023-07-13T22:20:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f579899b700 1 --2- 172.21.15.60:0/651943124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940b8400 0x7f57941f2860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f578800c4d0 tx=0x7f578800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f57917fa700 1 -- 172.21.15.60:0/651943124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57880052d0 con 0x7f57940b8400 2023-07-13T22:20:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f57917fa700 1 -- 172.21.15.60:0/651943124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5788005450 con 0x7f57940b8400 2023-07-13T22:20:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f57917fa700 1 -- 172.21.15.60:0/651943124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57880038c0 con 0x7f57940b8400 2023-07-13T22:20:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f579abff700 1 -- 172.21.15.60:0/651943124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5780009550 con 0x7f57940b8400 2023-07-13T22:20:37.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f579abff700 1 -- 172.21.15.60:0/651943124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57941f2340 con 0x7f57940b8400 2023-07-13T22:20:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.492+0000 7f57917fa700 1 -- 172.21.15.60:0/651943124 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5788003d20 con 0x7f57940b8400 2023-07-13T22:20:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.493+0000 7f57917fa700 1 -- 172.21.15.60:0/651943124 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5788011db0 con 0x7f57940b8400 2023-07-13T22:20:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.493+0000 7f579abff700 1 -- 172.21.15.60:0/651943124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57941f6370 con 0x7f57940b8400 2023-07-13T22:20:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.496+0000 7f57917fa700 1 -- 172.21.15.60:0/651943124 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f5788017070 con 0x7f57940b8400 2023-07-13T22:20:37.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.570+0000 7f579abff700 1 -- 172.21.15.60:0/651943124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57940b66a0 con 0x7f57940b8400 2023-07-13T22:20:37.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.571+0000 7f57917fa700 1 -- 172.21.15.60:0/651943124 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f57940cd950 con 0x7f57940b8400 2023-07-13T22:20:37.555 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:37.556 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":10,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:37.148188+0000","last_up_change":"0.000000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:20:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.572+0000 7f579abff700 1 -- 172.21.15.60:0/651943124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940b8400 msgr2=0x7f57941f2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.572+0000 7f579abff700 1 --2- 172.21.15.60:0/651943124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940b8400 0x7f57941f2860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f578800c4d0 tx=0x7f578800c7e0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.572+0000 7f579abff700 1 -- 172.21.15.60:0/651943124 shutdown_connections 2023-07-13T22:20:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.572+0000 7f579abff700 1 --2- 172.21.15.60:0/651943124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940b8400 0x7f57941f2860 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.572+0000 7f579abff700 1 --2- 172.21.15.60:0/651943124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57940d3750 0x7f57941f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.572+0000 7f579abff700 1 --2- 172.21.15.60:0/651943124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57940cfad0 0x7f57941f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.572+0000 7f579abff700 1 -- 172.21.15.60:0/651943124 >> 172.21.15.60:0/651943124 conn(0x7f579406b390 msgr2=0x7f579415ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:37.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.572+0000 7f579abff700 1 -- 172.21.15.60:0/651943124 shutdown_connections 2023-07-13T22:20:37.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:37.572+0000 7f579abff700 1 -- 172.21.15.60:0/651943124 wait complete. 2023-07-13T22:20:37.558 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:37.573+0000 7f95c665e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-13T22:20:37.574 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-13T22:20:37.575 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-13T22:20:37.636 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:37.650+0000 7f1d0e646200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-13T22:20:37.674 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:37.690+0000 7f95c665e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-13T22:20:37.736 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:37.751+0000 7f1d0e646200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-13T22:20:37.834 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:37.848+0000 7f1d0e646200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-13T22:20:37.880 INFO:tasks.ceph.osd.6.smithi105.stderr:2023-07-13T22:20:37.895+0000 7ff31a373640 -1 Falling back to public interface 2023-07-13T22:20:37.883 INFO:tasks.ceph.osd.4.smithi105.stderr:2023-07-13T22:20:37.898+0000 7f73cc3ec640 -1 Falling back to public interface 2023-07-13T22:20:37.886 INFO:tasks.ceph.osd.0.smithi060.stderr:2023-07-13T22:20:37.902+0000 7f7b6ce94640 -1 Falling back to public interface 2023-07-13T22:20:37.887 INFO:tasks.ceph.osd.2.smithi060.stderr:2023-07-13T22:20:37.902+0000 7f8b65eaa640 -1 Falling back to public interface 2023-07-13T22:20:37.887 INFO:tasks.ceph.osd.2.smithi060.stderr:2023-07-13T22:20:37.902+0000 7f8b57dfd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-07-13T22:20:37.897 INFO:tasks.ceph.osd.5.smithi105.stderr:2023-07-13T22:20:37.912+0000 7f9dea3d7640 -1 Falling back to public interface 2023-07-13T22:20:37.910 INFO:tasks.ceph.osd.7.smithi105.stderr:2023-07-13T22:20:37.925+0000 7fea906b2640 -1 Falling back to public interface 2023-07-13T22:20:37.911 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:37.926+0000 7f95c665e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-13T22:20:37.911 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-07-13T22:20:37.927+0000 7fd20871b640 -1 Falling back to public interface 2023-07-13T22:20:37.912 INFO:tasks.ceph.osd.3.smithi060.stderr:2023-07-13T22:20:37.928+0000 7f94ccc20640 -1 Falling back to public interface 2023-07-13T22:20:37.955 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:37.970+0000 7f1d0e646200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-13T22:20:38.022 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:38.038+0000 7f95c665e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-13T22:20:38.181 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:38.196+0000 7f1d0e646200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-13T22:20:38.284 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:38.299+0000 7f1d0e646200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-13T22:20:38.380 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-13T22:20:38.386 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:38.400+0000 7f95c665e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-13T22:20:38.482 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:38.498+0000 7f95c665e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-13T22:20:38.564 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.579+0000 7fd4aafff700 1 Processor -- start 2023-07-13T22:20:38.564 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.580+0000 7fd4aafff700 1 -- start start 2023-07-13T22:20:38.565 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4aafff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 0x7fd4a40b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.566 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4aafff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4a40d1140 0x7fd4a40d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.566 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4aafff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a40d16b0 con 0x7fd4a40d3400 2023-07-13T22:20:38.566 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4aafff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a40cefe0 con 0x7fd4a40d1140 2023-07-13T22:20:38.566 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4aafff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40cd250 con 0x7fd4a40d2af0 2023-07-13T22:20:38.566 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4a3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4a40d1140 0x7fd4a40d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.566 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4a3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4a40d1140 0x7fd4a40d2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51584/0 (socket says 172.21.15.60:51584) 2023-07-13T22:20:38.566 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4a3fff700 1 -- 172.21.15.60:0/2636265135 learned_addr learned my addr 172.21.15.60:0/2636265135 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:38.567 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4a8d9b700 1 --2- 172.21.15.60:0/2636265135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 0x7fd4a40b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.568 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4a3fff700 1 -- 172.21.15.60:0/2636265135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 msgr2=0x7fd4a40b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:38.568 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4a3fff700 1 --2- 172.21.15.60:0/2636265135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 0x7fd4a40b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.568 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4a3fff700 1 -- 172.21.15.60:0/2636265135 >> v1:172.21.15.105:6789/0 conn(0x7fd4a40d3400 legacy=0x7fd4a40cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.568 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4a3fff700 1 -- 172.21.15.60:0/2636265135 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a40cd530 con 0x7fd4a40d1140 2023-07-13T22:20:38.568 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.581+0000 7fd4a8d9b700 1 --2- 172.21.15.60:0/2636265135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 0x7fd4a40b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:20:38.568 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.582+0000 7fd4a3fff700 1 --2- 172.21.15.60:0/2636265135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 0x7fd4a40d2ec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd4900098f0 tx=0x7fd490009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab885e7c5dcda89 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:38.568 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.582+0000 7fd4a37fe700 1 -- 172.21.15.60:0/2636265135 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49000b8a0 con 0x7fd4a40d1140 2023-07-13T22:20:38.569 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.582+0000 7fd4a37fe700 1 -- 172.21.15.60:0/2636265135 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49000ba00 con 0x7fd4a40d1140 2023-07-13T22:20:38.569 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.582+0000 7fd4a37fe700 1 -- 172.21.15.60:0/2636265135 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49000bd90 con 0x7fd4a40d1140 2023-07-13T22:20:38.569 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.582+0000 7fd4aafff700 1 -- 172.21.15.60:0/2636265135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 msgr2=0x7fd4a40d2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.569 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.582+0000 7fd4aafff700 1 --2- 172.21.15.60:0/2636265135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 0x7fd4a40d2ec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd4900098f0 tx=0x7fd490009c00 comp rx=0 tx=0).stop 2023-07-13T22:20:38.569 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.583+0000 7fd4aafff700 1 -- 172.21.15.60:0/2636265135 shutdown_connections 2023-07-13T22:20:38.569 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.583+0000 7fd4aafff700 1 --2- 172.21.15.60:0/2636265135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 0x7fd4a40b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.569 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.583+0000 7fd4aafff700 1 --2- 172.21.15.60:0/2636265135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 0x7fd4a40d2ec0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.570 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.583+0000 7fd4aafff700 1 -- 172.21.15.60:0/2636265135 >> 172.21.15.60:0/2636265135 conn(0x7fd4a406b3f0 msgr2=0x7fd4a406b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:38.570 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.583+0000 7fd4aafff700 1 -- 172.21.15.60:0/2636265135 shutdown_connections 2023-07-13T22:20:38.570 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.583+0000 7fd4aafff700 1 -- 172.21.15.60:0/2636265135 wait complete. 2023-07-13T22:20:38.570 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.585+0000 7fd4aafff700 1 Processor -- start 2023-07-13T22:20:38.570 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.585+0000 7fd4aafff700 1 -- start start 2023-07-13T22:20:38.571 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4aafff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 0x7fd4a4153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.572 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4aafff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 0x7fd4a415c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.572 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4aafff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4a40d3400 0x7fd4a415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.572 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4aafff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a415ce40 con 0x7fd4a40d1140 2023-07-13T22:20:38.572 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4aafff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40caa50 con 0x7fd4a40d3400 2023-07-13T22:20:38.572 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4aafff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40ca750 con 0x7fd4a40d2af0 2023-07-13T22:20:38.572 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a8d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 0x7fd4a4153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.573 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a8d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 0x7fd4a4153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51598/0 (socket says 172.21.15.60:51598) 2023-07-13T22:20:38.573 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a8d9b700 1 -- 172.21.15.60:0/310812314 learned_addr learned my addr 172.21.15.60:0/310812314 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:38.573 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a3fff700 1 --2- 172.21.15.60:0/310812314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 0x7fd4a415c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.573 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a959c700 1 --2- 172.21.15.60:0/310812314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4a40d3400 0x7fd4a415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.573 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a8d9b700 1 -- 172.21.15.60:0/310812314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 msgr2=0x7fd4a415c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.573 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a8d9b700 1 --2- 172.21.15.60:0/310812314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 0x7fd4a415c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.573 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a8d9b700 1 -- 172.21.15.60:0/310812314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4a40d3400 msgr2=0x7fd4a415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.573 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a8d9b700 1 --2- 172.21.15.60:0/310812314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4a40d3400 0x7fd4a415ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.574 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.587+0000 7fd4a8d9b700 1 -- 172.21.15.60:0/310812314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490009550 con 0x7fd4a40d1140 2023-07-13T22:20:38.574 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.588+0000 7fd4a8d9b700 1 --2- 172.21.15.60:0/310812314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 0x7fd4a4153820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e840 tx=0x7fd49800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:38.575 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.588+0000 7fd4a17fa700 1 -- 172.21.15.60:0/310812314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498013610 con 0x7fd4a40d1140 2023-07-13T22:20:38.575 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.588+0000 7fd4a17fa700 1 -- 172.21.15.60:0/310812314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd498013770 con 0x7fd4a40d1140 2023-07-13T22:20:38.575 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.588+0000 7fd4a17fa700 1 -- 172.21.15.60:0/310812314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498013b20 con 0x7fd4a40d1140 2023-07-13T22:20:38.575 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.588+0000 7fd4aafff700 1 -- 172.21.15.60:0/310812314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a415d070 con 0x7fd4a40d1140 2023-07-13T22:20:38.575 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.588+0000 7fd4aafff700 1 -- 172.21.15.60:0/310812314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a415d630 con 0x7fd4a40d1140 2023-07-13T22:20:38.575 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.588+0000 7fd4a17fa700 1 -- 172.21.15.60:0/310812314 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd49801c450 con 0x7fd4a40d1140 2023-07-13T22:20:38.575 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.588+0000 7fd4a17fa700 1 -- 172.21.15.60:0/310812314 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd49801cdd0 con 0x7fd4a40d1140 2023-07-13T22:20:38.576 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.589+0000 7fd4aafff700 1 -- 172.21.15.60:0/310812314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a40ca890 con 0x7fd4a40d1140 2023-07-13T22:20:38.576 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.592+0000 7fd4a17fa700 1 -- 172.21.15.60:0/310812314 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fd49803bb20 con 0x7fd4a40d1140 2023-07-13T22:20:38.637 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:38.652+0000 7f1d0e646200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-13T22:20:38.650 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.666+0000 7fd4aafff700 1 -- 172.21.15.60:0/310812314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4a40b4e50 con 0x7fd4a40d1140 2023-07-13T22:20:38.651 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.667+0000 7fd4a17fa700 1 -- 172.21.15.60:0/310812314 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd49802a020 con 0x7fd4a40d1140 2023-07-13T22:20:38.651 INFO:teuthology.misc.health.smithi060.stdout: 2023-07-13T22:20:38.651 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":10,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:37.148188+0000","last_up_change":"0.000000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:20:38.652 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.668+0000 7fd4aafff700 1 -- 172.21.15.60:0/310812314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 msgr2=0x7fd4a4153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.653 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.668+0000 7fd4aafff700 1 --2- 172.21.15.60:0/310812314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 0x7fd4a4153820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e840 tx=0x7fd49800eb50 comp rx=0 tx=0).stop 2023-07-13T22:20:38.653 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.668+0000 7fd4aafff700 1 -- 172.21.15.60:0/310812314 shutdown_connections 2023-07-13T22:20:38.653 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.668+0000 7fd4aafff700 1 --2- 172.21.15.60:0/310812314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4a40d1140 0x7fd4a4153820 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.653 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.668+0000 7fd4aafff700 1 --2- 172.21.15.60:0/310812314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4a40d3400 0x7fd4a415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.653 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.668+0000 7fd4aafff700 1 --2- 172.21.15.60:0/310812314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4a40d2af0 0x7fd4a415c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.653 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.668+0000 7fd4aafff700 1 -- 172.21.15.60:0/310812314 >> 172.21.15.60:0/310812314 conn(0x7fd4a406b3f0 msgr2=0x7fd4a40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:38.653 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.668+0000 7fd4aafff700 1 -- 172.21.15.60:0/310812314 shutdown_connections 2023-07-13T22:20:38.654 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:38.668+0000 7fd4aafff700 1 -- 172.21.15.60:0/310812314 wait complete. 2023-07-13T22:20:38.667 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-13T22:20:38.730 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:38.745+0000 7f1d0e646200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-13T22:20:39.143 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:39.158+0000 7f95c665e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-13T22:20:39.351 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:39.365+0000 7f1d0e646200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-13T22:20:39.534 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:39.549+0000 7f95c665e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-13T22:20:39.720 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:39.736+0000 7f95c665e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:20:39.729 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:39.744+0000 7f1d0e646200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-13T22:20:39.910 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:39.925+0000 7f1d0e646200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:20:40.184 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:40.199+0000 7f95c665e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-13T22:20:40.358 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:40.373+0000 7f1d0e646200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-13T22:20:40.386 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:40.401+0000 7f95c665e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:20:40.555 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:40.570+0000 7f1d0e646200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:20:40.595 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:40.611+0000 7f95c665e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-13T22:20:40.852 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:40.867+0000 7f1d0e646200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-13T22:20:41.085 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:41.101+0000 7f95c665e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-13T22:20:41.245 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-07-13T22:20:41.260+0000 7f1d0e646200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-13T22:20:41.395 INFO:tasks.ceph.osd.6.smithi105.stderr:2023-07-13T22:20:41.410+0000 7ff31a373640 -1 osd.6 0 log_to_monitors true 2023-07-13T22:20:41.455 INFO:tasks.ceph.osd.0.smithi060.stderr:2023-07-13T22:20:41.470+0000 7f7b6ce94640 -1 osd.0 0 log_to_monitors true 2023-07-13T22:20:41.456 INFO:tasks.ceph.osd.2.smithi060.stderr:2023-07-13T22:20:41.472+0000 7f8b65eaa640 -1 osd.2 0 log_to_monitors true 2023-07-13T22:20:41.480 INFO:tasks.ceph.osd.7.smithi105.stderr:2023-07-13T22:20:41.495+0000 7fea906b2640 -1 osd.7 0 log_to_monitors true 2023-07-13T22:20:41.484 INFO:tasks.ceph.osd.5.smithi105.stderr:2023-07-13T22:20:41.499+0000 7f9dea3d7640 -1 osd.5 0 log_to_monitors true 2023-07-13T22:20:41.486 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-07-13T22:20:41.502+0000 7fd20871b640 -1 osd.1 0 log_to_monitors true 2023-07-13T22:20:41.707 INFO:tasks.ceph.osd.4.smithi105.stderr:2023-07-13T22:20:41.721+0000 7f73cc3ec640 -1 osd.4 0 log_to_monitors true 2023-07-13T22:20:41.724 INFO:tasks.ceph.osd.3.smithi060.stderr:2023-07-13T22:20:41.740+0000 7f94ccc20640 -1 osd.3 0 log_to_monitors true 2023-07-13T22:20:42.106 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:42.121+0000 7f957fffa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-13T22:20:42.106 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-07-13T22:20:42.122+0000 7f957fffa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-13T22:20:44.672 INFO:tasks.ceph.osd.0.smithi060.stderr:2023-07-13T22:20:44.680+0000 7f7b5b5f6700 -1 osd.0 0 waiting for initial osdmap 2023-07-13T22:20:44.681 INFO:tasks.ceph.osd.2.smithi060.stderr:2023-07-13T22:20:44.696+0000 7f8b545f6700 -1 osd.2 0 waiting for initial osdmap 2023-07-13T22:20:44.685 INFO:tasks.ceph.osd.0.smithi060.stderr:2023-07-13T22:20:44.701+0000 7f7b535fb700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:20:44.696 INFO:tasks.ceph.osd.2.smithi060.stderr:2023-07-13T22:20:44.712+0000 7f8b4c5fb700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:20:44.779 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-07-13T22:20:44.795+0000 7fd1f6df6700 -1 osd.1 0 waiting for initial osdmap 2023-07-13T22:20:44.790 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-07-13T22:20:44.806+0000 7fd1eedfb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:20:44.861 INFO:tasks.ceph.osd.3.smithi060.stderr:2023-07-13T22:20:44.877+0000 7f94bb3f6700 -1 osd.3 0 waiting for initial osdmap 2023-07-13T22:20:44.881 INFO:tasks.ceph.osd.3.smithi060.stderr:2023-07-13T22:20:44.896+0000 7f94b33fb700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:20:45.318 INFO:tasks.ceph.osd.4.smithi105.stderr:2023-07-13T22:20:45.333+0000 7f73bc1f9700 -1 osd.4 0 waiting for initial osdmap 2023-07-13T22:20:45.338 INFO:tasks.ceph.osd.4.smithi105.stderr:2023-07-13T22:20:45.353+0000 7f73b29fb700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:20:45.348 INFO:tasks.ceph.osd.5.smithi105.stderr:2023-07-13T22:20:45.362+0000 7f9dda1f9700 -1 osd.5 0 waiting for initial osdmap 2023-07-13T22:20:45.356 INFO:tasks.ceph.osd.7.smithi105.stderr:2023-07-13T22:20:45.370+0000 7fea7edf6700 -1 osd.7 0 waiting for initial osdmap 2023-07-13T22:20:45.360 INFO:tasks.ceph.osd.5.smithi105.stderr:2023-07-13T22:20:45.375+0000 7f9dd09fb700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:20:45.364 INFO:tasks.ceph.osd.6.smithi105.stderr:2023-07-13T22:20:45.379+0000 7ff3089f6700 -1 osd.6 0 waiting for initial osdmap 2023-07-13T22:20:45.373 INFO:tasks.ceph.osd.7.smithi105.stderr:2023-07-13T22:20:45.388+0000 7fea76dfb700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:20:45.378 INFO:tasks.ceph.osd.6.smithi105.stderr:2023-07-13T22:20:45.393+0000 7ff3009fb700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:20:45.473 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-13T22:20:45.653 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.668+0000 7fee1f1ff700 1 Processor -- start 2023-07-13T22:20:45.653 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.668+0000 7fee1f1ff700 1 -- start start 2023-07-13T22:20:45.654 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.669+0000 7fee1f1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fee180d2af0 0x7fee180b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:45.654 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee1f1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee180d16b0 con 0x7fee180d1140 2023-07-13T22:20:45.654 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee1f1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee180cd250 con 0x7fee180d35d0 2023-07-13T22:20:45.654 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee1f1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee180cefe0 con 0x7fee180d2af0 2023-07-13T22:20:45.655 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee1cf9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fee180d2af0 0x7fee180b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:45.656 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee17fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fee180d1140 0x7fee180d2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43938/0 (socket says 172.21.15.60:43938) 2023-07-13T22:20:45.656 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee1cf9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fee180d2af0 0x7fee180b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42374/0 (socket says 172.21.15.60:42374) 2023-07-13T22:20:45.656 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee17fff700 1 -- 172.21.15.60:0/3370163 learned_addr learned my addr 172.21.15.60:0/3370163 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:45.656 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee1cf9b700 1 -- 172.21.15.60:0/3370163 >> v1:172.21.15.105:6789/0 conn(0x7fee180d35d0 legacy=0x7fee180cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:45.656 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee1cf9b700 1 -- 172.21.15.60:0/3370163 >> v1:172.21.15.60:6789/0 conn(0x7fee180d1140 legacy=0x7fee180d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:45.656 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee1cf9b700 1 -- 172.21.15.60:0/3370163 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee180cd530 con 0x7fee180d2af0 2023-07-13T22:20:45.657 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee177fe700 1 -- 172.21.15.60:0/3370163 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961612546 0 0) 0x7fee180d16b0 con 0x7fee180d1140 2023-07-13T22:20:45.657 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.670+0000 7fee1cf9b700 1 --2- 172.21.15.60:0/3370163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d2af0 0x7fee180b59b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009840 tx=0x7fee0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bbcd4290530199 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:45.657 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee177fe700 1 -- 172.21.15.60:0/3370163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c004230 con 0x7fee180d2af0 2023-07-13T22:20:45.657 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee177fe700 1 -- 172.21.15.60:0/3370163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c004390 con 0x7fee180d2af0 2023-07-13T22:20:45.657 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee177fe700 1 -- 172.21.15.60:0/3370163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0036a0 con 0x7fee180d2af0 2023-07-13T22:20:45.657 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee1f1ff700 1 -- 172.21.15.60:0/3370163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d2af0 msgr2=0x7fee180b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:45.657 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee1f1ff700 1 --2- 172.21.15.60:0/3370163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d2af0 0x7fee180b59b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009840 tx=0x7fee0c009b50 comp rx=0 tx=0).stop 2023-07-13T22:20:45.658 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee1f1ff700 1 -- 172.21.15.60:0/3370163 shutdown_connections 2023-07-13T22:20:45.658 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee1f1ff700 1 --2- 172.21.15.60:0/3370163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d2af0 0x7fee180b59b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:45.658 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee1f1ff700 1 -- 172.21.15.60:0/3370163 >> 172.21.15.60:0/3370163 conn(0x7fee1806b3f0 msgr2=0x7fee1806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:45.658 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee1f1ff700 1 -- 172.21.15.60:0/3370163 shutdown_connections 2023-07-13T22:20:45.658 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.671+0000 7fee1f1ff700 1 -- 172.21.15.60:0/3370163 wait complete. 2023-07-13T22:20:45.660 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.673+0000 7fee1f1ff700 1 Processor -- start 2023-07-13T22:20:45.660 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.673+0000 7fee1f1ff700 1 -- start start 2023-07-13T22:20:45.660 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1f1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d1140 0x7fee18155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:45.660 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1f1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee180d2af0 0x7fee181f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:45.660 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee180d35d0 0x7fee18156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:45.660 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1f1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee180ca280 con 0x7fee180d1140 2023-07-13T22:20:45.660 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1f1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee18155d70 con 0x7fee180d2af0 2023-07-13T22:20:45.661 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee18156050 con 0x7fee180d35d0 2023-07-13T22:20:45.661 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1cf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d1140 0x7fee18155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:45.661 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1cf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d1140 0x7fee18155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42386/0 (socket says 172.21.15.60:42386) 2023-07-13T22:20:45.661 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1cf9b700 1 -- 172.21.15.60:0/690283225 learned_addr learned my addr 172.21.15.60:0/690283225 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:45.661 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee17fff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee180d2af0 0x7fee181f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:45.661 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1d79c700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee180d35d0 0x7fee18156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:45.661 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1cf9b700 1 -- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee180d35d0 msgr2=0x7fee18156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:45.662 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1cf9b700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee180d35d0 0x7fee18156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:45.662 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1cf9b700 1 -- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee180d2af0 msgr2=0x7fee181f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:45.662 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1cf9b700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee180d2af0 0x7fee181f11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:45.662 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1cf9b700 1 -- 172.21.15.60:0/690283225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c0094a0 con 0x7fee180d1140 2023-07-13T22:20:45.662 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee17fff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee180d2af0 0x7fee181f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:20:45.662 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.675+0000 7fee1d79c700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee180d35d0 0x7fee18156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:20:45.662 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.676+0000 7fee1cf9b700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d1140 0x7fee18155520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009b90 tx=0x7fee0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:45.663 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.676+0000 7fee157fa700 1 -- 172.21.15.60:0/690283225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c011630 con 0x7fee180d1140 2023-07-13T22:20:45.664 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.676+0000 7fee157fa700 1 -- 172.21.15.60:0/690283225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c01e070 con 0x7fee180d1140 2023-07-13T22:20:45.664 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.676+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee180c9c30 con 0x7fee180d1140 2023-07-13T22:20:45.664 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.676+0000 7fee157fa700 1 -- 172.21.15.60:0/690283225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0119b0 con 0x7fee180d1140 2023-07-13T22:20:45.664 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.676+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee18155a60 con 0x7fee180d1140 2023-07-13T22:20:45.664 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.677+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee181561c0 con 0x7fee180d1140 2023-07-13T22:20:45.664 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.678+0000 7fee157fa700 1 -- 172.21.15.60:0/690283225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fee0c011b10 con 0x7fee180d1140 2023-07-13T22:20:45.665 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.679+0000 7fee157fa700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedfc06ac50 0x7fedfc06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:45.665 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.679+0000 7fee157fa700 1 -- 172.21.15.60:0/690283225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fee0c089920 con 0x7fee180d1140 2023-07-13T22:20:45.665 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.679+0000 7fee17fff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedfc06ac50 0x7fedfc06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:45.665 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.680+0000 7fee17fff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedfc06ac50 0x7fedfc06d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee180b2990 tx=0x7fee04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:45.668 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.684+0000 7fee157fa700 1 -- 172.21.15.60:0/690283225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fee0c055990 con 0x7fee180d1140 2023-07-13T22:20:45.885 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.901+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee180b5ed0 con 0x7fee180d1140 2023-07-13T22:20:45.886 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.902+0000 7fee157fa700 1 -- 172.21.15.60:0/690283225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10872 (secure 0 0 0) 0x7fee180b5ed0 con 0x7fee180d1140 2023-07-13T22:20:45.886 INFO:teuthology.misc.health.smithi060.stdout: 2023-07-13T22:20:45.886 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":13,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:45.703751+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:20:45.888 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedfc06ac50 msgr2=0x7fedfc06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:45.889 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedfc06ac50 0x7fedfc06d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee180b2990 tx=0x7fee04009040 comp rx=0 tx=0).stop 2023-07-13T22:20:45.889 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d1140 msgr2=0x7fee18155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:45.889 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d1140 0x7fee18155520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009b90 tx=0x7fee0c004750 comp rx=0 tx=0).stop 2023-07-13T22:20:45.889 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 shutdown_connections 2023-07-13T22:20:45.889 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedfc06ac50 0x7fedfc06d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:45.889 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee180d1140 0x7fee18155520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:45.890 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee180d35d0 0x7fee18156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:45.890 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 --2- 172.21.15.60:0/690283225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee180d2af0 0x7fee181f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:45.890 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 >> 172.21.15.60:0/690283225 conn(0x7fee1806b3f0 msgr2=0x7fee180b6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:45.890 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 shutdown_connections 2023-07-13T22:20:45.891 INFO:teuthology.misc.health.smithi060.stderr:2023-07-13T22:20:45.904+0000 7fee1f1ff700 1 -- 172.21.15.60:0/690283225 wait complete. 2023-07-13T22:20:45.904 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-13T22:20:45.907 INFO:tasks.ceph:Creating RBD pool 2023-07-13T22:20:45.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-13T22:20:46.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.120+0000 7ffa5b1ff700 1 Processor -- start 2023-07-13T22:20:46.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.121+0000 7ffa5b1ff700 1 -- start start 2023-07-13T22:20:46.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.122+0000 7ffa5b1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540b1ac0 0x7ffa540d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.122+0000 7ffa5b1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa540d1870 con 0x7ffa540cfd50 2023-07-13T22:20:46.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.122+0000 7ffa5b1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa540cf370 con 0x7ffa540b8200 2023-07-13T22:20:46.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.122+0000 7ffa5b1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540d12b0 con 0x7ffa540b1ac0 2023-07-13T22:20:46.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.122+0000 7ffa53fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffa540cfd50 0x7ffa540d23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43952/0 (socket says 172.21.15.60:43952) 2023-07-13T22:20:46.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.122+0000 7ffa53fff700 1 -- 172.21.15.60:0/1078153794 learned_addr learned my addr 172.21.15.60:0/1078153794 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:46.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.122+0000 7ffa5979c700 1 --2- 172.21.15.60:0/1078153794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540b1ac0 0x7ffa540d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.122+0000 7ffa537fe700 1 -- 172.21.15.60:0/1078153794 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508926403 0 0) 0x7ffa540d1870 con 0x7ffa540cfd50 2023-07-13T22:20:46.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.123+0000 7ffa537fe700 1 -- 172.21.15.60:0/1078153794 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa440035b0 con 0x7ffa540cfd50 2023-07-13T22:20:46.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.123+0000 7ffa5979c700 1 -- 172.21.15.60:0/1078153794 >> v1:172.21.15.105:6789/0 conn(0x7ffa540b8200 legacy=0x7ffa540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.123+0000 7ffa537fe700 1 -- 172.21.15.60:0/1078153794 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467070932 0 0) 0x7ffa540cf370 con 0x7ffa540b8200 2023-07-13T22:20:46.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.123+0000 7ffa5979c700 1 -- 172.21.15.60:0/1078153794 >> v1:172.21.15.60:6789/0 conn(0x7ffa540cfd50 legacy=0x7ffa540d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.123+0000 7ffa5979c700 1 -- 172.21.15.60:0/1078153794 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa540d1590 con 0x7ffa540b1ac0 2023-07-13T22:20:46.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.123+0000 7ffa5979c700 1 --2- 172.21.15.60:0/1078153794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540b1ac0 0x7ffa540d2ad0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009dc0 tx=0x7ffa4c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f842bcbbec62e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:46.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.123+0000 7ffa537fe700 1 -- 172.21.15.60:0/1078153794 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00e070 con 0x7ffa540b1ac0 2023-07-13T22:20:46.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.123+0000 7ffa537fe700 1 -- 172.21.15.60:0/1078153794 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c00bb20 con 0x7ffa540b1ac0 2023-07-13T22:20:46.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.123+0000 7ffa537fe700 1 -- 172.21.15.60:0/1078153794 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00bed0 con 0x7ffa540b1ac0 2023-07-13T22:20:46.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.124+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/1078153794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540b1ac0 msgr2=0x7ffa540d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.124+0000 7ffa5b1ff700 1 --2- 172.21.15.60:0/1078153794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540b1ac0 0x7ffa540d2ad0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009dc0 tx=0x7ffa4c00b330 comp rx=0 tx=0).stop 2023-07-13T22:20:46.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.124+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/1078153794 shutdown_connections 2023-07-13T22:20:46.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.124+0000 7ffa5b1ff700 1 --2- 172.21.15.60:0/1078153794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540b1ac0 0x7ffa540d2ad0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.124+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/1078153794 >> 172.21.15.60:0/1078153794 conn(0x7ffa5406b2f0 msgr2=0x7ffa5406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:46.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.124+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/1078153794 shutdown_connections 2023-07-13T22:20:46.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.124+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/1078153794 wait complete. 2023-07-13T22:20:46.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.126+0000 7ffa5b1ff700 1 Processor -- start 2023-07-13T22:20:46.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.126+0000 7ffa5b1ff700 1 -- start start 2023-07-13T22:20:46.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa5b1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa540b1ac0 0x7ffa540b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa5b1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa540b8200 0x7ffa540c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa5b1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540cfd50 0x7ffa541568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa5b1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540c6b10 con 0x7ffa540b1ac0 2023-07-13T22:20:46.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa5b1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa54157a50 con 0x7ffa540b8200 2023-07-13T22:20:46.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa5b1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa54157cf0 con 0x7ffa540cfd50 2023-07-13T22:20:46.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa58f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa540b1ac0 0x7ffa540b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa58f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa540b1ac0 0x7ffa540b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42398/0 (socket says 172.21.15.60:42398) 2023-07-13T22:20:46.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa58f9b700 1 -- 172.21.15.60:0/3488268331 learned_addr learned my addr 172.21.15.60:0/3488268331 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:46.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa5979c700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540cfd50 0x7ffa541568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa53fff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa540b8200 0x7ffa540c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa58f9b700 1 -- 172.21.15.60:0/3488268331 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540cfd50 msgr2=0x7ffa541568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa58f9b700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540cfd50 0x7ffa541568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa58f9b700 1 -- 172.21.15.60:0/3488268331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa540b8200 msgr2=0x7ffa540c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa58f9b700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa540b8200 0x7ffa540c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa58f9b700 1 -- 172.21.15.60:0/3488268331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c009a70 con 0x7ffa540b1ac0 2023-07-13T22:20:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.127+0000 7ffa53fff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa540b8200 0x7ffa540c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:20:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.128+0000 7ffa58f9b700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa540b1ac0 0x7ffa540b3d30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffa480011e0 tx=0x7ffa4800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.128+0000 7ffa517fa700 1 -- 172.21.15.60:0/3488268331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480054e0 con 0x7ffa540b1ac0 2023-07-13T22:20:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.128+0000 7ffa517fa700 1 -- 172.21.15.60:0/3488268331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa48003680 con 0x7ffa540b1ac0 2023-07-13T22:20:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.128+0000 7ffa517fa700 1 -- 172.21.15.60:0/3488268331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48003a30 con 0x7ffa540b1ac0 2023-07-13T22:20:46.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.128+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa54157fd0 con 0x7ffa540b1ac0 2023-07-13T22:20:46.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.128+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa54158870 con 0x7ffa540b1ac0 2023-07-13T22:20:46.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.129+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5404e210 con 0x7ffa540b1ac0 2023-07-13T22:20:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.130+0000 7ffa517fa700 1 -- 172.21.15.60:0/3488268331 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7ffa48003b90 con 0x7ffa540b1ac0 2023-07-13T22:20:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.131+0000 7ffa517fa700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ac50 0x7ffa3806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.131+0000 7ffa517fa700 1 -- 172.21.15.60:0/3488268331 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7ffa480882d0 con 0x7ffa540b1ac0 2023-07-13T22:20:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.132+0000 7ffa53fff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ac50 0x7ffa3806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.132+0000 7ffa53fff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ac50 0x7ffa3806d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa540d2bf0 tx=0x7ffa40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:46.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.135+0000 7ffa517fa700 1 -- 172.21.15.60:0/3488268331 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffa48053a00 con 0x7ffa540b1ac0 2023-07-13T22:20:46.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.348+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7ffa540b4f70 con 0x7ffa540b1ac0 2023-07-13T22:20:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.710+0000 7ffa517fa700 1 -- 172.21.15.60:0/3488268331 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7ffa48057020 con 0x7ffa540b1ac0 2023-07-13T22:20:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.713+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ac50 msgr2=0x7ffa3806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.713+0000 7ffa5b1ff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ac50 0x7ffa3806d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa540d2bf0 tx=0x7ffa40009040 comp rx=0 tx=0).stop 2023-07-13T22:20:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.713+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa540b1ac0 msgr2=0x7ffa540b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.714+0000 7ffa5b1ff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa540b1ac0 0x7ffa540b3d30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffa480011e0 tx=0x7ffa4800ca30 comp rx=0 tx=0).stop 2023-07-13T22:20:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.714+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 shutdown_connections 2023-07-13T22:20:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.714+0000 7ffa5b1ff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ac50 0x7ffa3806d100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.714+0000 7ffa5b1ff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa540b1ac0 0x7ffa540b3d30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.714+0000 7ffa5b1ff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa540cfd50 0x7ffa541568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.714+0000 7ffa5b1ff700 1 --2- 172.21.15.60:0/3488268331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa540b8200 0x7ffa540c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.714+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 >> 172.21.15.60:0/3488268331 conn(0x7ffa5406b2f0 msgr2=0x7ffa540d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.714+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 shutdown_connections 2023-07-13T22:20:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.714+0000 7ffa5b1ff700 1 -- 172.21.15.60:0/3488268331 wait complete. 2023-07-13T22:20:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'rbd' created 2023-07-13T22:20:46.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-13T22:20:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.931+0000 7fdc753ff700 1 Processor -- start 2023-07-13T22:20:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.932+0000 7fdc753ff700 1 -- start start 2023-07-13T22:20:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.933+0000 7fdc753ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc7015c320 0x7fdc701609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.933+0000 7fdc753ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdc700b1280 0x7fdc700b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.933+0000 7fdc753ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc70069ed0 con 0x7fdc7006eeb0 2023-07-13T22:20:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.933+0000 7fdc753ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc700b1c40 con 0x7fdc700b1280 2023-07-13T22:20:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.933+0000 7fdc753ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700cd7b0 con 0x7fdc7015c320 2023-07-13T22:20:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.933+0000 7fdc6f7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdc7006eeb0 0x7fdc7006f2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43954/0 (socket says 172.21.15.60:43954) 2023-07-13T22:20:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.933+0000 7fdc6f7fe700 1 -- 172.21.15.60:0/1377925292 learned_addr learned my addr 172.21.15.60:0/1377925292 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:46.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.933+0000 7fdc6e7fc700 1 --2- 172.21.15.60:0/1377925292 >> v2:172.21.15.105:3300/0 conn(0x7fdc700b1280 0x7fdc700b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.933+0000 7fdc6effd700 1 --2- 172.21.15.60:0/1377925292 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc7015c320 0x7fdc701609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.934+0000 7fdc6dffb700 1 -- 172.21.15.60:0/1377925292 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997978357 0 0) 0x7fdc70069ed0 con 0x7fdc7006eeb0 2023-07-13T22:20:46.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.934+0000 7fdc6dffb700 1 -- 172.21.15.60:0/1377925292 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc580035b0 con 0x7fdc7006eeb0 2023-07-13T22:20:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.934+0000 7fdc6e7fc700 1 -- 172.21.15.60:0/1377925292 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc7015c320 msgr2=0x7fdc701609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.934+0000 7fdc6e7fc700 1 --2- 172.21.15.60:0/1377925292 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc7015c320 0x7fdc701609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.934+0000 7fdc6e7fc700 1 -- 172.21.15.60:0/1377925292 >> v1:172.21.15.60:6789/0 conn(0x7fdc7006eeb0 legacy=0x7fdc7006f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.934+0000 7fdc6e7fc700 1 -- 172.21.15.60:0/1377925292 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc700cda90 con 0x7fdc700b1280 2023-07-13T22:20:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.934+0000 7fdc6effd700 1 --2- 172.21.15.60:0/1377925292 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc7015c320 0x7fdc701609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.934+0000 7fdc6e7fc700 1 --2- 172.21.15.60:0/1377925292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc700b1280 0x7fdc700b1700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009840 tx=0x7fdc64009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1e2aba4e95dd05a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.935+0000 7fdc6dffb700 1 -- 172.21.15.60:0/1377925292 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64011830 con 0x7fdc700b1280 2023-07-13T22:20:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.935+0000 7fdc6dffb700 1 -- 172.21.15.60:0/1377925292 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc64011990 con 0x7fdc700b1280 2023-07-13T22:20:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.935+0000 7fdc6dffb700 1 -- 172.21.15.60:0/1377925292 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64011d20 con 0x7fdc700b1280 2023-07-13T22:20:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.935+0000 7fdc753ff700 1 -- 172.21.15.60:0/1377925292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc700b1280 msgr2=0x7fdc700b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.935+0000 7fdc753ff700 1 --2- 172.21.15.60:0/1377925292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc700b1280 0x7fdc700b1700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009840 tx=0x7fdc64009b50 comp rx=0 tx=0).stop 2023-07-13T22:20:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.935+0000 7fdc753ff700 1 -- 172.21.15.60:0/1377925292 shutdown_connections 2023-07-13T22:20:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.935+0000 7fdc753ff700 1 --2- 172.21.15.60:0/1377925292 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc7015c320 0x7fdc701609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.935+0000 7fdc753ff700 1 --2- 172.21.15.60:0/1377925292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc700b1280 0x7fdc700b1700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.935+0000 7fdc753ff700 1 -- 172.21.15.60:0/1377925292 >> 172.21.15.60:0/1377925292 conn(0x7fdc700cc280 msgr2=0x7fdc700cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.936+0000 7fdc753ff700 1 -- 172.21.15.60:0/1377925292 shutdown_connections 2023-07-13T22:20:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.936+0000 7fdc753ff700 1 -- 172.21.15.60:0/1377925292 wait complete. 2023-07-13T22:20:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.937+0000 7fdc753ff700 1 Processor -- start 2023-07-13T22:20:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.937+0000 7fdc753ff700 1 -- start start 2023-07-13T22:20:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.938+0000 7fdc753ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc7006eeb0 0x7fdc700c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.938+0000 7fdc753ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc700b1280 0x7fdc70156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.938+0000 7fdc753ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc7015c320 0x7fdc700b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.938+0000 7fdc753ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700c6e10 con 0x7fdc7006eeb0 2023-07-13T22:20:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.938+0000 7fdc753ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700c8230 con 0x7fdc7015c320 2023-07-13T22:20:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.938+0000 7fdc753ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc70156f90 con 0x7fdc700b1280 2023-07-13T22:20:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.938+0000 7fdc6effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc7006eeb0 0x7fdc700c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc7006eeb0 0x7fdc700c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42410/0 (socket says 172.21.15.60:42410) 2023-07-13T22:20:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6effd700 1 -- 172.21.15.60:0/3101650654 learned_addr learned my addr 172.21.15.60:0/3101650654 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6f7fe700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc7015c320 0x7fdc700b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6e7fc700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc700b1280 0x7fdc70156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6effd700 1 -- 172.21.15.60:0/3101650654 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc700b1280 msgr2=0x7fdc70156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6effd700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc700b1280 0x7fdc70156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6effd700 1 -- 172.21.15.60:0/3101650654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc7015c320 msgr2=0x7fdc700b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6effd700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc7015c320 0x7fdc700b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6effd700 1 -- 172.21.15.60:0/3101650654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc600094a0 con 0x7fdc7006eeb0 2023-07-13T22:20:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc6effd700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc7006eeb0 0x7fdc700c68d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000ebe0 tx=0x7fdc6000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc577fe700 1 -- 172.21.15.60:0/3101650654 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000c850 con 0x7fdc7006eeb0 2023-07-13T22:20:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.939+0000 7fdc577fe700 1 -- 172.21.15.60:0/3101650654 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6000c9b0 con 0x7fdc7006eeb0 2023-07-13T22:20:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.940+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc640094a0 con 0x7fdc7006eeb0 2023-07-13T22:20:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.940+0000 7fdc577fe700 1 -- 172.21.15.60:0/3101650654 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000cd40 con 0x7fdc7006eeb0 2023-07-13T22:20:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.940+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc700b3700 con 0x7fdc7006eeb0 2023-07-13T22:20:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.941+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc700aef90 con 0x7fdc7006eeb0 2023-07-13T22:20:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.942+0000 7fdc577fe700 1 -- 172.21.15.60:0/3101650654 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fdc60013470 con 0x7fdc7006eeb0 2023-07-13T22:20:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.943+0000 7fdc577fe700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc5c06ac50 0x7fdc5c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.943+0000 7fdc577fe700 1 -- 172.21.15.60:0/3101650654 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4813+0+0 (secure 0 0 0) 0x7fdc60088fa0 con 0x7fdc7006eeb0 2023-07-13T22:20:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.943+0000 7fdc6e7fc700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc5c06ac50 0x7fdc5c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:46.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.944+0000 7fdc6e7fc700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc5c06ac50 0x7fdc5c06d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc64000c00 tx=0x7fdc64008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:46.948+0000 7fdc577fe700 1 -- 172.21.15.60:0/3101650654 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdc60054600 con 0x7fdc7006eeb0 2023-07-13T22:20:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.158+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdc7004e210 con 0x7fdc7006eeb0 2023-07-13T22:20:47.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.718+0000 7fdc577fe700 1 -- 172.21.15.60:0/3101650654 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdc6000cea0 con 0x7fdc7006eeb0 2023-07-13T22:20:47.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.721+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc5c06ac50 msgr2=0x7fdc5c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:47.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.721+0000 7fdc753ff700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc5c06ac50 0x7fdc5c06d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc64000c00 tx=0x7fdc64008030 comp rx=0 tx=0).stop 2023-07-13T22:20:47.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.721+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc7006eeb0 msgr2=0x7fdc700c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:47.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.721+0000 7fdc753ff700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc7006eeb0 0x7fdc700c68d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000ebe0 tx=0x7fdc6000eef0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.722+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 shutdown_connections 2023-07-13T22:20:47.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.722+0000 7fdc753ff700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc5c06ac50 0x7fdc5c06d100 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.722+0000 7fdc753ff700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc7006eeb0 0x7fdc700c68d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.722+0000 7fdc753ff700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc7015c320 0x7fdc700b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.722+0000 7fdc753ff700 1 --2- 172.21.15.60:0/3101650654 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc700b1280 0x7fdc70156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.722+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 >> 172.21.15.60:0/3101650654 conn(0x7fdc700cc280 msgr2=0x7fdc7006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:47.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.722+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 shutdown_connections 2023-07-13T22:20:47.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.722+0000 7fdc753ff700 1 -- 172.21.15.60:0/3101650654 wait complete. 2023-07-13T22:20:47.710 INFO:teuthology.orchestra.run.smithi060.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-13T22:20:47.726 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-13T22:20:47.727 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:20:47.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:20:47.731 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:20:47.732 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:20:47.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:20:47.736 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:20:47.736 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:20:47.736 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:20:47.741 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:20:47.741 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:20:47.741 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:20:47.745 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:20:47.746 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-07-13T22:20:47.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:20:47.886 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:20:47.887 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:20:47.894 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:20:47.896 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:20:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.976+0000 7fd4933ff700 1 Processor -- start 2023-07-13T22:20:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.976+0000 7fd4933ff700 1 -- start start 2023-07-13T22:20:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd4933ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0b8400 0x7fd48c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd4933ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd48c0d3750 0x7fd48c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd4933ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c0cff20 con 0x7fd48c0cfad0 2023-07-13T22:20:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd4933ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0d0ac0 con 0x7fd48c0d3750 2023-07-13T22:20:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd4933ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0b87d0 con 0x7fd48c0b8400 2023-07-13T22:20:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd49099a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd48c0d3750 0x7fd48c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd49199c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd48c0cfad0 0x7fd48c15fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43978/0 (socket says 172.21.15.60:43978) 2023-07-13T22:20:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd49099a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd48c0d3750 0x7fd48c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42462/0 (socket says 172.21.15.60:42462) 2023-07-13T22:20:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd49199c700 1 -- 172.21.15.60:0/1060689016 learned_addr learned my addr 172.21.15.60:0/1060689016 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd49119b700 1 --2- 172.21.15.60:0/1060689016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0b8400 0x7fd48c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd49099a700 1 -- 172.21.15.60:0/1060689016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0b8400 msgr2=0x7fd48c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd49099a700 1 --2- 172.21.15.60:0/1060689016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0b8400 0x7fd48c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd49099a700 1 -- 172.21.15.60:0/1060689016 >> v1:172.21.15.60:6789/0 conn(0x7fd48c0cfad0 legacy=0x7fd48c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd49099a700 1 -- 172.21.15.60:0/1060689016 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c0b4eb0 con 0x7fd48c0d3750 2023-07-13T22:20:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.978+0000 7fd483fff700 1 -- 172.21.15.60:0/1060689016 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123902275 0 0) 0x7fd484001110 con 0x7fd48c0cfad0 2023-07-13T22:20:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.979+0000 7fd49099a700 1 --2- 172.21.15.60:0/1060689016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0d3750 0x7fd48c0cf590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009840 tx=0x7fd47c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6482c24ffb26b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.979+0000 7fd483fff700 1 -- 172.21.15.60:0/1060689016 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c011830 con 0x7fd48c0d3750 2023-07-13T22:20:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.979+0000 7fd483fff700 1 -- 172.21.15.60:0/1060689016 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c011990 con 0x7fd48c0d3750 2023-07-13T22:20:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.979+0000 7fd483fff700 1 -- 172.21.15.60:0/1060689016 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c011d20 con 0x7fd48c0d3750 2023-07-13T22:20:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.979+0000 7fd4933ff700 1 -- 172.21.15.60:0/1060689016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0d3750 msgr2=0x7fd48c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.979+0000 7fd4933ff700 1 --2- 172.21.15.60:0/1060689016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0d3750 0x7fd48c0cf590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009840 tx=0x7fd47c009b50 comp rx=0 tx=0).stop 2023-07-13T22:20:47.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.980+0000 7fd4933ff700 1 -- 172.21.15.60:0/1060689016 shutdown_connections 2023-07-13T22:20:47.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.980+0000 7fd4933ff700 1 --2- 172.21.15.60:0/1060689016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0b8400 0x7fd48c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.980+0000 7fd4933ff700 1 --2- 172.21.15.60:0/1060689016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0d3750 0x7fd48c0cf590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.980+0000 7fd4933ff700 1 -- 172.21.15.60:0/1060689016 >> 172.21.15.60:0/1060689016 conn(0x7fd48c06b390 msgr2=0x7fd48c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:47.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.980+0000 7fd4933ff700 1 -- 172.21.15.60:0/1060689016 shutdown_connections 2023-07-13T22:20:47.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.980+0000 7fd4933ff700 1 -- 172.21.15.60:0/1060689016 wait complete. 2023-07-13T22:20:47.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.982+0000 7fd4933ff700 1 Processor -- start 2023-07-13T22:20:47.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.982+0000 7fd4933ff700 1 -- start start 2023-07-13T22:20:47.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.983+0000 7fd4933ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0b8400 0x7fd48c1f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.983+0000 7fd4933ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0cfad0 0x7fd48c0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.983+0000 7fd4933ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0d3750 0x7fd48c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd4933ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0c6990 con 0x7fd48c0b8400 2023-07-13T22:20:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd4933ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0c9600 con 0x7fd48c0cfad0 2023-07-13T22:20:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd4933ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c15db60 con 0x7fd48c0d3750 2023-07-13T22:20:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0b8400 0x7fd48c1f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0b8400 0x7fd48c1f3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42474/0 (socket says 172.21.15.60:42474) 2023-07-13T22:20:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49119b700 1 -- 172.21.15.60:0/321651872 learned_addr learned my addr 172.21.15.60:0/321651872 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49199c700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0d3750 0x7fd48c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49099a700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0cfad0 0x7fd48c0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49119b700 1 -- 172.21.15.60:0/321651872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0d3750 msgr2=0x7fd48c0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49119b700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0d3750 0x7fd48c0c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49119b700 1 -- 172.21.15.60:0/321651872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0cfad0 msgr2=0x7fd48c0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49119b700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0cfad0 0x7fd48c0c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49119b700 1 -- 172.21.15.60:0/321651872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c0094a0 con 0x7fd48c0b8400 2023-07-13T22:20:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.984+0000 7fd49119b700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0b8400 0x7fd48c1f3160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd488007e40 tx=0x7fd48800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.985+0000 7fd481ffb700 1 -- 172.21.15.60:0/321651872 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800ed30 con 0x7fd48c0b8400 2023-07-13T22:20:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.985+0000 7fd481ffb700 1 -- 172.21.15.60:0/321651872 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800ee90 con 0x7fd48c0b8400 2023-07-13T22:20:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.985+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c15de40 con 0x7fd48c0b8400 2023-07-13T22:20:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.985+0000 7fd481ffb700 1 -- 172.21.15.60:0/321651872 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488014660 con 0x7fd48c0b8400 2023-07-13T22:20:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.985+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c1ef7e0 con 0x7fd48c0b8400 2023-07-13T22:20:47.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.986+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c15d9c0 con 0x7fd48c0b8400 2023-07-13T22:20:47.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.987+0000 7fd481ffb700 1 -- 172.21.15.60:0/321651872 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fd4880147c0 con 0x7fd48c0b8400 2023-07-13T22:20:47.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.988+0000 7fd481ffb700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd47006ac50 0x7fd47006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:47.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.988+0000 7fd481ffb700 1 -- 172.21.15.60:0/321651872 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7fd488089750 con 0x7fd48c0b8400 2023-07-13T22:20:47.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.988+0000 7fd49099a700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd47006ac50 0x7fd47006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:47.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.989+0000 7fd49099a700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd47006ac50 0x7fd47006d100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0d3b60 tx=0x7fd47c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:47.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:47.993+0000 7fd481ffb700 1 -- 172.21.15.60:0/321651872 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd48c15d9c0 con 0x7fd48c0b8400 2023-07-13T22:20:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.198+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd48c15d9c0 con 0x7fd48c0b8400 2023-07-13T22:20:48.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.199+0000 7fd481ffb700 1 -- 172.21.15.60:0/321651872 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14561 (secure 0 0 0) 0x7fd48c15d9c0 con 0x7fd48c0b8400 2023-07-13T22:20:48.183 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:48.183 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:47.708813+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:20:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd47006ac50 msgr2=0x7fd47006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd47006ac50 0x7fd47006d100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0d3b60 tx=0x7fd47c01c040 comp rx=0 tx=0).stop 2023-07-13T22:20:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0b8400 msgr2=0x7fd48c1f3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0b8400 0x7fd48c1f3160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd488007e40 tx=0x7fd48800e3c0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 shutdown_connections 2023-07-13T22:20:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd47006ac50 0x7fd47006d100 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd48c0b8400 0x7fd48c1f3160 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd48c0d3750 0x7fd48c0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 --2- 172.21.15.60:0/321651872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd48c0cfad0 0x7fd48c0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 >> 172.21.15.60:0/321651872 conn(0x7fd48c06b390 msgr2=0x7fd48c15bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 shutdown_connections 2023-07-13T22:20:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.201+0000 7fd4933ff700 1 -- 172.21.15.60:0/321651872 wait complete. 2023-07-13T22:20:48.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:20:48.204 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:20:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.425+0000 7ffb3a7ff700 1 Processor -- start 2023-07-13T22:20:48.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.426+0000 7ffb3a7ff700 1 -- start start 2023-07-13T22:20:48.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.427+0000 7ffb3a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340b8200 0x7ffb340cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.427+0000 7ffb3a7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffb340b1ac0 0x7ffb340cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.427+0000 7ffb3a7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb340d1870 con 0x7ffb340cede0 2023-07-13T22:20:48.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.427+0000 7ffb3a7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb340cf370 con 0x7ffb340b1ac0 2023-07-13T22:20:48.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.427+0000 7ffb3a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340d12b0 con 0x7ffb340b8200 2023-07-13T22:20:48.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.427+0000 7ffb38d9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffb340cede0 0x7ffb3415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43982/0 (socket says 172.21.15.60:43982) 2023-07-13T22:20:48.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.427+0000 7ffb38d9c700 1 -- 172.21.15.60:0/1653212734 learned_addr learned my addr 172.21.15.60:0/1653212734 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:48.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.427+0000 7ffb33fff700 1 --2- 172.21.15.60:0/1653212734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340b8200 0x7ffb340cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.427+0000 7ffb337fe700 1 --2- 172.21.15.60:0/1653212734 >> v2:172.21.15.105:3300/0 conn(0x7ffb340b1ac0 0x7ffb340cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.428+0000 7ffb32ffd700 1 -- 172.21.15.60:0/1653212734 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572313278 0 0) 0x7ffb340d1870 con 0x7ffb340cede0 2023-07-13T22:20:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.428+0000 7ffb32ffd700 1 -- 172.21.15.60:0/1653212734 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb1c0035b0 con 0x7ffb340cede0 2023-07-13T22:20:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.428+0000 7ffb33fff700 1 -- 172.21.15.60:0/1653212734 >> v2:172.21.15.105:3300/0 conn(0x7ffb340b1ac0 msgr2=0x7ffb340cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.428+0000 7ffb33fff700 1 --2- 172.21.15.60:0/1653212734 >> v2:172.21.15.105:3300/0 conn(0x7ffb340b1ac0 0x7ffb340cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.428+0000 7ffb33fff700 1 -- 172.21.15.60:0/1653212734 >> v1:172.21.15.60:6789/0 conn(0x7ffb340cede0 legacy=0x7ffb3415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.428+0000 7ffb33fff700 1 -- 172.21.15.60:0/1653212734 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340d1590 con 0x7ffb340b8200 2023-07-13T22:20:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.428+0000 7ffb33fff700 1 --2- 172.21.15.60:0/1653212734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340b8200 0x7ffb340cf640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009840 tx=0x7ffb24009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7302c9b7ac33022 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.428+0000 7ffb32ffd700 1 -- 172.21.15.60:0/1653212734 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24011830 con 0x7ffb340b8200 2023-07-13T22:20:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.428+0000 7ffb32ffd700 1 -- 172.21.15.60:0/1653212734 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb24011990 con 0x7ffb340b8200 2023-07-13T22:20:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.429+0000 7ffb32ffd700 1 -- 172.21.15.60:0/1653212734 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24011d20 con 0x7ffb340b8200 2023-07-13T22:20:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.429+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/1653212734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340b8200 msgr2=0x7ffb340cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.429+0000 7ffb3a7ff700 1 --2- 172.21.15.60:0/1653212734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340b8200 0x7ffb340cf640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009840 tx=0x7ffb24009b50 comp rx=0 tx=0).stop 2023-07-13T22:20:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.429+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/1653212734 shutdown_connections 2023-07-13T22:20:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.429+0000 7ffb3a7ff700 1 --2- 172.21.15.60:0/1653212734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340b8200 0x7ffb340cf640 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.429+0000 7ffb3a7ff700 1 --2- 172.21.15.60:0/1653212734 >> v2:172.21.15.105:3300/0 conn(0x7ffb340b1ac0 0x7ffb340cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.429+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/1653212734 >> 172.21.15.60:0/1653212734 conn(0x7ffb3406b2f0 msgr2=0x7ffb3406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.429+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/1653212734 shutdown_connections 2023-07-13T22:20:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.429+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/1653212734 wait complete. 2023-07-13T22:20:48.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.431+0000 7ffb3a7ff700 1 Processor -- start 2023-07-13T22:20:48.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.431+0000 7ffb3a7ff700 1 -- start start 2023-07-13T22:20:48.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb3a7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb340b1ac0 0x7ffb340cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb3a7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb340b8200 0x7ffb341f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb3a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340cede0 0x7ffb341f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb3a7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340c78a0 con 0x7ffb340b8200 2023-07-13T22:20:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb3a7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340c7b20 con 0x7ffb340b1ac0 2023-07-13T22:20:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb3a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340cc1b0 con 0x7ffb340cede0 2023-07-13T22:20:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb337fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb340b8200 0x7ffb341f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb337fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb340b8200 0x7ffb341f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42490/0 (socket says 172.21.15.60:42490) 2023-07-13T22:20:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb337fe700 1 -- 172.21.15.60:0/2618519624 learned_addr learned my addr 172.21.15.60:0/2618519624 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb38d9c700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340cede0 0x7ffb341f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.432+0000 7ffb33fff700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb340b1ac0 0x7ffb340cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb337fe700 1 -- 172.21.15.60:0/2618519624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340cede0 msgr2=0x7ffb341f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb337fe700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340cede0 0x7ffb341f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb337fe700 1 -- 172.21.15.60:0/2618519624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb340b1ac0 msgr2=0x7ffb340cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb337fe700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb340b1ac0 0x7ffb340cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb337fe700 1 -- 172.21.15.60:0/2618519624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb240094a0 con 0x7ffb340b8200 2023-07-13T22:20:48.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb38d9c700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340cede0 0x7ffb341f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:48.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb337fe700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb340b8200 0x7ffb341f5110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800e9b0 tx=0x7ffb2800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:48.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb30ff9700 1 -- 172.21.15.60:0/2618519624 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28009710 con 0x7ffb340b8200 2023-07-13T22:20:48.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb30ff9700 1 -- 172.21.15.60:0/2618519624 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2800ee50 con 0x7ffb340b8200 2023-07-13T22:20:48.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb340cc490 con 0x7ffb340b8200 2023-07-13T22:20:48.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb30ff9700 1 -- 172.21.15.60:0/2618519624 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2800ca50 con 0x7ffb340b8200 2023-07-13T22:20:48.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.433+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3414b710 con 0x7ffb340b8200 2023-07-13T22:20:48.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.434+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb340c8150 con 0x7ffb340b8200 2023-07-13T22:20:48.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.436+0000 7ffb30ff9700 1 -- 172.21.15.60:0/2618519624 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7ffb2800cbb0 con 0x7ffb340b8200 2023-07-13T22:20:48.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.436+0000 7ffb30ff9700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb2006ac50 0x7ffb2006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.437+0000 7ffb30ff9700 1 -- 172.21.15.60:0/2618519624 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7ffb28089040 con 0x7ffb340b8200 2023-07-13T22:20:48.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.437+0000 7ffb33fff700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb2006ac50 0x7ffb2006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.437+0000 7ffb33fff700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb2006ac50 0x7ffb2006d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffb24000c00 tx=0x7ffb24008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:48.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.441+0000 7ffb30ff9700 1 -- 172.21.15.60:0/2618519624 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffb28054520 con 0x7ffb340b8200 2023-07-13T22:20:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.647+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffb3404e210 con 0x7ffb340b8200 2023-07-13T22:20:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.647+0000 7ffb30ff9700 1 -- 172.21.15.60:0/2618519624 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffb2800ce60 con 0x7ffb340b8200 2023-07-13T22:20:48.632 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:20:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb2006ac50 msgr2=0x7ffb2006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb2006ac50 0x7ffb2006d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffb24000c00 tx=0x7ffb24008030 comp rx=0 tx=0).stop 2023-07-13T22:20:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb340b8200 msgr2=0x7ffb341f5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb340b8200 0x7ffb341f5110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800e9b0 tx=0x7ffb2800ecc0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 shutdown_connections 2023-07-13T22:20:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb2006ac50 0x7ffb2006d100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb340cede0 0x7ffb341f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb340b1ac0 0x7ffb340cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 --2- 172.21.15.60:0/2618519624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb340b8200 0x7ffb341f5110 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 >> 172.21.15.60:0/2618519624 conn(0x7ffb3406b2f0 msgr2=0x7ffb340b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:48.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 shutdown_connections 2023-07-13T22:20:48.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.649+0000 7ffb3a7ff700 1 -- 172.21.15.60:0/2618519624 wait complete. 2023-07-13T22:20:48.651 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:20:48.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.888+0000 7f9e2ebff700 1 Processor -- start 2023-07-13T22:20:48.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.888+0000 7f9e2ebff700 1 -- start start 2023-07-13T22:20:48.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.889+0000 7f9e2ebff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9e280b8200 0x7f9e280cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.889+0000 7f9e2ebff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9e280cede0 0x7f9e280cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.889+0000 7f9e2ebff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e280d1870 con 0x7f9e280b1ac0 2023-07-13T22:20:48.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.889+0000 7f9e2ebff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e280cf370 con 0x7f9e280cede0 2023-07-13T22:20:48.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.889+0000 7f9e2ebff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e280d12b0 con 0x7f9e280b8200 2023-07-13T22:20:48.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.889+0000 7f9e2d19c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9e280cede0 0x7f9e280cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e2d19c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9e280cede0 0x7f9e280cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42528/0 (socket says 172.21.15.60:42528) 2023-07-13T22:20:48.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e2d19c700 1 -- 172.21.15.60:0/1806768698 learned_addr learned my addr 172.21.15.60:0/1806768698 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:48.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e2c99b700 1 --2- 172.21.15.60:0/1806768698 >> v2:172.21.15.105:3300/0 conn(0x7f9e280b8200 0x7f9e280cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e2d19c700 1 -- 172.21.15.60:0/1806768698 >> v2:172.21.15.105:3300/0 conn(0x7f9e280b8200 msgr2=0x7f9e280cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:48.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e2d19c700 1 --2- 172.21.15.60:0/1806768698 >> v2:172.21.15.105:3300/0 conn(0x7f9e280b8200 0x7f9e280cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e2d19c700 1 -- 172.21.15.60:0/1806768698 >> v1:172.21.15.105:6789/0 conn(0x7f9e280b1ac0 legacy=0x7f9e2815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e2d19c700 1 -- 172.21.15.60:0/1806768698 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e280d1590 con 0x7f9e280cede0 2023-07-13T22:20:48.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e277fe700 1 -- 172.21.15.60:0/1806768698 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350826455 0 0) 0x7f9e280d1870 con 0x7f9e280b1ac0 2023-07-13T22:20:48.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e2d19c700 1 --2- 172.21.15.60:0/1806768698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280cede0 0x7f9e280cfb80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9e280b2660 tx=0x7f9e2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7a98cd2ba4d903 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:48.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e277fe700 1 -- 172.21.15.60:0/1806768698 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000e070 con 0x7f9e280cede0 2023-07-13T22:20:48.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e277fe700 1 -- 172.21.15.60:0/1806768698 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2000b910 con 0x7f9e280cede0 2023-07-13T22:20:48.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.890+0000 7f9e277fe700 1 -- 172.21.15.60:0/1806768698 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000bcc0 con 0x7f9e280cede0 2023-07-13T22:20:48.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.891+0000 7f9e2ebff700 1 -- 172.21.15.60:0/1806768698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280cede0 msgr2=0x7f9e280cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.891+0000 7f9e2ebff700 1 --2- 172.21.15.60:0/1806768698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280cede0 0x7f9e280cfb80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9e280b2660 tx=0x7f9e2000b330 comp rx=0 tx=0).stop 2023-07-13T22:20:48.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.891+0000 7f9e2ebff700 1 -- 172.21.15.60:0/1806768698 shutdown_connections 2023-07-13T22:20:48.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.891+0000 7f9e2ebff700 1 --2- 172.21.15.60:0/1806768698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280cede0 0x7f9e280cfb80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.891+0000 7f9e2ebff700 1 --2- 172.21.15.60:0/1806768698 >> v2:172.21.15.105:3300/0 conn(0x7f9e280b8200 0x7f9e280cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.891+0000 7f9e2ebff700 1 -- 172.21.15.60:0/1806768698 >> 172.21.15.60:0/1806768698 conn(0x7f9e2806b2f0 msgr2=0x7f9e2806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.891+0000 7f9e2ebff700 1 -- 172.21.15.60:0/1806768698 shutdown_connections 2023-07-13T22:20:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.891+0000 7f9e2ebff700 1 -- 172.21.15.60:0/1806768698 wait complete. 2023-07-13T22:20:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.893+0000 7f9e2ebff700 1 Processor -- start 2023-07-13T22:20:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.893+0000 7f9e2ebff700 1 -- start start 2023-07-13T22:20:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e2ebff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e280b1ac0 0x7f9e280c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e2ebff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280b8200 0x7f9e280cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e2ebff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280cede0 0x7f9e28155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e2ebff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e281588f0 con 0x7f9e280b8200 2023-07-13T22:20:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e2ebff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e280cd0b0 con 0x7f9e280b1ac0 2023-07-13T22:20:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e2ebff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e280ccb30 con 0x7f9e280cede0 2023-07-13T22:20:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e27fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280b8200 0x7f9e280cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e27fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280b8200 0x7f9e280cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42542/0 (socket says 172.21.15.60:42542) 2023-07-13T22:20:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e27fff700 1 -- 172.21.15.60:0/4021232165 learned_addr learned my addr 172.21.15.60:0/4021232165 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e2d19c700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280cede0 0x7f9e28155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.894+0000 7f9e2c99b700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e280b1ac0 0x7f9e280c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e27fff700 1 -- 172.21.15.60:0/4021232165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280cede0 msgr2=0x7f9e28155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e27fff700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280cede0 0x7f9e28155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e27fff700 1 -- 172.21.15.60:0/4021232165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e280b1ac0 msgr2=0x7f9e280c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e27fff700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e280b1ac0 0x7f9e280c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e27fff700 1 -- 172.21.15.60:0/4021232165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e20009b30 con 0x7f9e280b8200 2023-07-13T22:20:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e27fff700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280b8200 0x7f9e280cc5f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400c480 tx=0x7f9e1400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:48.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e257fa700 1 -- 172.21.15.60:0/4021232165 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e140054e0 con 0x7f9e280b8200 2023-07-13T22:20:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e257fa700 1 -- 172.21.15.60:0/4021232165 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1400ce50 con 0x7f9e280b8200 2023-07-13T22:20:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e280ccdd0 con 0x7f9e280b8200 2023-07-13T22:20:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e257fa700 1 -- 172.21.15.60:0/4021232165 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e140038c0 con 0x7f9e280b8200 2023-07-13T22:20:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.895+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e28158510 con 0x7f9e280b8200 2023-07-13T22:20:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.896+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e28158090 con 0x7f9e280b8200 2023-07-13T22:20:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.898+0000 7f9e257fa700 1 -- 172.21.15.60:0/4021232165 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f9e14003a20 con 0x7f9e280b8200 2023-07-13T22:20:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.900+0000 7f9e257fa700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9e0c06ac50 0x7f9e0c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.900+0000 7f9e257fa700 1 -- 172.21.15.60:0/4021232165 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9e14088380 con 0x7f9e280b8200 2023-07-13T22:20:48.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.900+0000 7f9e2c99b700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9e0c06ac50 0x7f9e0c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:48.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.901+0000 7f9e2c99b700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9e0c06ac50 0x7f9e0c06d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009be0 tx=0x7f9e1c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:48.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:48.904+0000 7f9e257fa700 1 -- 172.21.15.60:0/4021232165 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9e28158090 con 0x7f9e280b8200 2023-07-13T22:20:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.124+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e28158090 con 0x7f9e280b8200 2023-07-13T22:20:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.124+0000 7f9e257fa700 1 -- 172.21.15.60:0/4021232165 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e28158090 con 0x7f9e280b8200 2023-07-13T22:20:49.109 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:20:49.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9e0c06ac50 msgr2=0x7f9e0c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9e0c06ac50 0x7f9e0c06d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009be0 tx=0x7f9e1c009210 comp rx=0 tx=0).stop 2023-07-13T22:20:49.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280b8200 msgr2=0x7f9e280cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280b8200 0x7f9e280cc5f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400c480 tx=0x7f9e1400c790 comp rx=0 tx=0).stop 2023-07-13T22:20:49.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 shutdown_connections 2023-07-13T22:20:49.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9e0c06ac50 0x7f9e0c06d100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280cede0 0x7f9e28155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e280b1ac0 0x7f9e280c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 --2- 172.21.15.60:0/4021232165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e280b8200 0x7f9e280cc5f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 >> 172.21.15.60:0/4021232165 conn(0x7f9e2806b2f0 msgr2=0x7f9e280b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:49.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 shutdown_connections 2023-07-13T22:20:49.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.127+0000 7f9e2ebff700 1 -- 172.21.15.60:0/4021232165 wait complete. 2023-07-13T22:20:49.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:20:49.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.351+0000 7f91049ff700 1 Processor -- start 2023-07-13T22:20:49.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.351+0000 7f91049ff700 1 -- start start 2023-07-13T22:20:49.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f91049ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f910015b590 0x7f910015e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f91049ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f91000cba00 0x7f9100069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f91049ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91000ca5d0 con 0x7f91000b12a0 2023-07-13T22:20:49.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f91049ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91000ca010 con 0x7f91000cba00 2023-07-13T22:20:49.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f91049ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91000ca2f0 con 0x7f910015b590 2023-07-13T22:20:49.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f90fe59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f91000b12a0 0x7f91000b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43998/0 (socket says 172.21.15.60:43998) 2023-07-13T22:20:49.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f90fe59c700 1 -- 172.21.15.60:0/3183589299 learned_addr learned my addr 172.21.15.60:0/3183589299 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f90fed9d700 1 --2- 172.21.15.60:0/3183589299 >> v2:172.21.15.60:3300/0 conn(0x7f91000cba00 0x7f9100069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f90f7fff700 1 --2- 172.21.15.60:0/3183589299 >> v2:172.21.15.105:3300/0 conn(0x7f910015b590 0x7f910015e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f90fdd9b700 1 -- 172.21.15.60:0/3183589299 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751833048 0 0) 0x7f91000ca5d0 con 0x7f91000b12a0 2023-07-13T22:20:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.353+0000 7f90fdd9b700 1 -- 172.21.15.60:0/3183589299 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90ec0035b0 con 0x7f91000b12a0 2023-07-13T22:20:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.354+0000 7f90fed9d700 1 -- 172.21.15.60:0/3183589299 >> v2:172.21.15.105:3300/0 conn(0x7f910015b590 msgr2=0x7f910015e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.354+0000 7f90fed9d700 1 --2- 172.21.15.60:0/3183589299 >> v2:172.21.15.105:3300/0 conn(0x7f910015b590 0x7f910015e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.354+0000 7f90fed9d700 1 -- 172.21.15.60:0/3183589299 >> v1:172.21.15.60:6789/0 conn(0x7f91000b12a0 legacy=0x7f91000b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.354+0000 7f90fed9d700 1 -- 172.21.15.60:0/3183589299 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910015dfe0 con 0x7f91000cba00 2023-07-13T22:20:49.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.354+0000 7f90f7fff700 1 --2- 172.21.15.60:0/3183589299 >> v2:172.21.15.105:3300/0 conn(0x7f910015b590 0x7f910015e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:49.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.354+0000 7f90fed9d700 1 --2- 172.21.15.60:0/3183589299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91000cba00 0x7f9100069ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f910006e8d0 tx=0x7f90f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6fc5b3147b14c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:49.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.354+0000 7f90fdd9b700 1 -- 172.21.15.60:0/3183589299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f800e070 con 0x7f91000cba00 2023-07-13T22:20:49.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.354+0000 7f90fdd9b700 1 -- 172.21.15.60:0/3183589299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f800bb20 con 0x7f91000cba00 2023-07-13T22:20:49.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.354+0000 7f90fdd9b700 1 -- 172.21.15.60:0/3183589299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8012430 con 0x7f91000cba00 2023-07-13T22:20:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.355+0000 7f91049ff700 1 -- 172.21.15.60:0/3183589299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91000cba00 msgr2=0x7f9100069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.355+0000 7f91049ff700 1 --2- 172.21.15.60:0/3183589299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91000cba00 0x7f9100069ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f910006e8d0 tx=0x7f90f800b330 comp rx=0 tx=0).stop 2023-07-13T22:20:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.355+0000 7f91049ff700 1 -- 172.21.15.60:0/3183589299 shutdown_connections 2023-07-13T22:20:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.355+0000 7f91049ff700 1 --2- 172.21.15.60:0/3183589299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91000cba00 0x7f9100069ed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.355+0000 7f91049ff700 1 --2- 172.21.15.60:0/3183589299 >> v2:172.21.15.105:3300/0 conn(0x7f910015b590 0x7f910015e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.355+0000 7f91049ff700 1 -- 172.21.15.60:0/3183589299 >> 172.21.15.60:0/3183589299 conn(0x7f91000cc2a0 msgr2=0x7f91000cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:49.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.355+0000 7f91049ff700 1 -- 172.21.15.60:0/3183589299 shutdown_connections 2023-07-13T22:20:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.355+0000 7f91049ff700 1 -- 172.21.15.60:0/3183589299 wait complete. 2023-07-13T22:20:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.357+0000 7f91049ff700 1 Processor -- start 2023-07-13T22:20:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.357+0000 7f91049ff700 1 -- start start 2023-07-13T22:20:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.358+0000 7f91049ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91000b12a0 0x7f9100159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.358+0000 7f91049ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91000cba00 0x7f9100149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.358+0000 7f91049ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f910015b590 0x7f910014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.358+0000 7f91049ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000c7c20 con 0x7f910015b590 2023-07-13T22:20:49.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.358+0000 7f91049ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000c7f00 con 0x7f91000b12a0 2023-07-13T22:20:49.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.358+0000 7f91049ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91000c81e0 con 0x7f91000cba00 2023-07-13T22:20:49.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f910015b590 0x7f910014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f910015b590 0x7f910014a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42552/0 (socket says 172.21.15.60:42552) 2023-07-13T22:20:49.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fed9d700 1 -- 172.21.15.60:0/3641909851 learned_addr learned my addr 172.21.15.60:0/3641909851 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:49.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90f7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91000cba00 0x7f9100149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fe59c700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91000b12a0 0x7f9100159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fed9d700 1 -- 172.21.15.60:0/3641909851 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91000cba00 msgr2=0x7f9100149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fed9d700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91000cba00 0x7f9100149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fed9d700 1 -- 172.21.15.60:0/3641909851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91000b12a0 msgr2=0x7f9100159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fed9d700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91000b12a0 0x7f9100159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fed9d700 1 -- 172.21.15.60:0/3641909851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e80094a0 con 0x7f910015b590 2023-07-13T22:20:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.359+0000 7f90fed9d700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f910015b590 0x7f910014a010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f90f8014040 tx=0x7f90f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.360+0000 7f90f6ffd700 1 -- 172.21.15.60:0/3641909851 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8012430 con 0x7f910015b590 2023-07-13T22:20:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.360+0000 7f90f6ffd700 1 -- 172.21.15.60:0/3641909851 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f8020070 con 0x7f910015b590 2023-07-13T22:20:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.360+0000 7f90f6ffd700 1 -- 172.21.15.60:0/3641909851 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f80098f0 con 0x7f910015b590 2023-07-13T22:20:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.360+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f8009a70 con 0x7f910015b590 2023-07-13T22:20:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.360+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91000c87a0 con 0x7f910015b590 2023-07-13T22:20:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.361+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91000c89b0 con 0x7f910015b590 2023-07-13T22:20:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.363+0000 7f90f6ffd700 1 -- 172.21.15.60:0/3641909851 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f90f8004040 con 0x7f910015b590 2023-07-13T22:20:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.364+0000 7f90f6ffd700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f90e006ac50 0x7f90e006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.364+0000 7f90f6ffd700 1 -- 172.21.15.60:0/3641909851 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f90f8022070 con 0x7f910015b590 2023-07-13T22:20:49.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.365+0000 7f90fe59c700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f90e006ac50 0x7f90e006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.365+0000 7f90fe59c700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f90e006ac50 0x7f90e006d100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007fd0 tx=0x7f90f0007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:49.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.367+0000 7f90f6ffd700 1 -- 172.21.15.60:0/3641909851 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f90f8056710 con 0x7f910015b590 2023-07-13T22:20:49.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.627+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f910004e280 con 0x7f910015b590 2023-07-13T22:20:49.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.628+0000 7f90f6ffd700 1 -- 172.21.15.60:0/3641909851 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f90f802c090 con 0x7f910015b590 2023-07-13T22:20:49.612 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:49.613 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2550952354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2550952354},{"type":"v1","addr":"172.21.15.60:6837","nonce":2550952354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4134045869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4134045869},{"type":"v1","addr":"172.21.15.60:6836","nonce":4134045869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2202330082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2202330082},{"type":"v1","addr":"172.21.15.105:6834","nonce":2202330082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/453882022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":453882022},{"type":"v1","addr":"172.21.15.105:6835","nonce":453882022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-13T22:20:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f90e006ac50 msgr2=0x7f90e006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f90e006ac50 0x7f90e006d100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007fd0 tx=0x7f90f0007dc0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f910015b590 msgr2=0x7f910014a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f910015b590 0x7f910014a010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f90f8014040 tx=0x7f90f8004ad0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 shutdown_connections 2023-07-13T22:20:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f90e006ac50 0x7f90e006d100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f910015b590 0x7f910014a010 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91000b12a0 0x7f9100159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 --2- 172.21.15.60:0/3641909851 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91000cba00 0x7f9100149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 >> 172.21.15.60:0/3641909851 conn(0x7f91000cc2a0 msgr2=0x7f91000ca530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:49.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 shutdown_connections 2023-07-13T22:20:49.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.630+0000 7f91049ff700 1 -- 172.21.15.60:0/3641909851 wait complete. 2023-07-13T22:20:49.616 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2023-07-13T22:20:49.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:20:49.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.855+0000 7fc1ccdff700 1 Processor -- start 2023-07-13T22:20:49.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.856+0000 7fc1ccdff700 1 -- start start 2023-07-13T22:20:49.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.857+0000 7fc1ccdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1c80cfca0 0x7fc1c8167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.857+0000 7fc1ccdff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c80cd950 con 0x7fc1c80d3750 2023-07-13T22:20:49.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.857+0000 7fc1ccdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c80d0ac0 con 0x7fc1c80b8400 2023-07-13T22:20:49.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.857+0000 7fc1ccdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c80b87d0 con 0x7fc1c80cfca0 2023-07-13T22:20:49.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.857+0000 7fc1c6d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1c80cfca0 0x7fc1c8167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.857+0000 7fc1bffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc1c80d3750 0x7fc1c80cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44008/0 (socket says 172.21.15.60:44008) 2023-07-13T22:20:49.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.857+0000 7fc1c6d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1c80cfca0 0x7fc1c8167050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42556/0 (socket says 172.21.15.60:42556) 2023-07-13T22:20:49.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.857+0000 7fc1bffff700 1 -- 172.21.15.60:0/36678415 learned_addr learned my addr 172.21.15.60:0/36678415 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:49.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.857+0000 7fc1c6d9d700 1 -- 172.21.15.60:0/36678415 >> v1:172.21.15.105:6789/0 conn(0x7fc1c80b8400 legacy=0x7fc1c80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.858+0000 7fc1c6d9d700 1 -- 172.21.15.60:0/36678415 >> v1:172.21.15.60:6789/0 conn(0x7fc1c80d3750 legacy=0x7fc1c80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.858+0000 7fc1c5d9b700 1 -- 172.21.15.60:0/36678415 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680764450 0 0) 0x7fc1c80d0ac0 con 0x7fc1c80b8400 2023-07-13T22:20:49.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.858+0000 7fc1c6d9d700 1 -- 172.21.15.60:0/36678415 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80b4eb0 con 0x7fc1c80cfca0 2023-07-13T22:20:49.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.858+0000 7fc1c6d9d700 1 --2- 172.21.15.60:0/36678415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80cfca0 0x7fc1c8167050 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009d90 tx=0x7fc1c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1afe9dc4187e0a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:49.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.858+0000 7fc1c5d9b700 1 -- 172.21.15.60:0/36678415 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000e070 con 0x7fc1c80cfca0 2023-07-13T22:20:49.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.858+0000 7fc1c5d9b700 1 -- 172.21.15.60:0/36678415 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c000bad0 con 0x7fc1c80cfca0 2023-07-13T22:20:49.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.858+0000 7fc1c5d9b700 1 -- 172.21.15.60:0/36678415 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000be80 con 0x7fc1c80cfca0 2023-07-13T22:20:49.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.859+0000 7fc1ccdff700 1 -- 172.21.15.60:0/36678415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80cfca0 msgr2=0x7fc1c8167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.859+0000 7fc1ccdff700 1 --2- 172.21.15.60:0/36678415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80cfca0 0x7fc1c8167050 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009d90 tx=0x7fc1c000b330 comp rx=0 tx=0).stop 2023-07-13T22:20:49.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.859+0000 7fc1ccdff700 1 -- 172.21.15.60:0/36678415 shutdown_connections 2023-07-13T22:20:49.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.859+0000 7fc1ccdff700 1 --2- 172.21.15.60:0/36678415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80cfca0 0x7fc1c8167050 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.859+0000 7fc1ccdff700 1 -- 172.21.15.60:0/36678415 >> 172.21.15.60:0/36678415 conn(0x7fc1c806b390 msgr2=0x7fc1c806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:49.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.859+0000 7fc1ccdff700 1 -- 172.21.15.60:0/36678415 shutdown_connections 2023-07-13T22:20:49.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.859+0000 7fc1ccdff700 1 -- 172.21.15.60:0/36678415 wait complete. 2023-07-13T22:20:49.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.860+0000 7fc1ccdff700 1 Processor -- start 2023-07-13T22:20:49.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.861+0000 7fc1ccdff700 1 -- start start 2023-07-13T22:20:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1ccdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80b8400 0x7fc1c81f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1ccdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80cfca0 0x7fc1c815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1ccdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80d3750 0x7fc1c80b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1ccdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c814ab00 con 0x7fc1c80b8400 2023-07-13T22:20:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1ccdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c81f5370 con 0x7fc1c80d3750 2023-07-13T22:20:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1ccdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80ccaa0 con 0x7fc1c80cfca0 2023-07-13T22:20:49.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1c659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80b8400 0x7fc1c81f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1c659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80b8400 0x7fc1c81f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42570/0 (socket says 172.21.15.60:42570) 2023-07-13T22:20:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1c659c700 1 -- 172.21.15.60:0/1633824787 learned_addr learned my addr 172.21.15.60:0/1633824787 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1c6d9d700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80d3750 0x7fc1c80b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1bffff700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80cfca0 0x7fc1c815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1c659c700 1 -- 172.21.15.60:0/1633824787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80cfca0 msgr2=0x7fc1c815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1c659c700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80cfca0 0x7fc1c815dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1c659c700 1 -- 172.21.15.60:0/1633824787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80d3750 msgr2=0x7fc1c80b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1c659c700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80d3750 0x7fc1c80b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.862+0000 7fc1c659c700 1 -- 172.21.15.60:0/1633824787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c0009a70 con 0x7fc1c80b8400 2023-07-13T22:20:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.863+0000 7fc1c659c700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80b8400 0x7fc1c81f4e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c480 tx=0x7fc1b800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.863+0000 7fc1beffd700 1 -- 172.21.15.60:0/1633824787 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80054e0 con 0x7fc1c80b8400 2023-07-13T22:20:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.863+0000 7fc1beffd700 1 -- 172.21.15.60:0/1633824787 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b800ce50 con 0x7fc1c80b8400 2023-07-13T22:20:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.863+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c80cbe90 con 0x7fc1c80b8400 2023-07-13T22:20:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.863+0000 7fc1beffd700 1 -- 172.21.15.60:0/1633824787 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80038c0 con 0x7fc1c80b8400 2023-07-13T22:20:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.863+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c80cc450 con 0x7fc1c80b8400 2023-07-13T22:20:49.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.864+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c815a9e0 con 0x7fc1c80b8400 2023-07-13T22:20:49.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.865+0000 7fc1beffd700 1 -- 172.21.15.60:0/1633824787 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fc1b8003a20 con 0x7fc1c80b8400 2023-07-13T22:20:49.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.866+0000 7fc1beffd700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1a806ac80 0x7fc1a806d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:49.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.866+0000 7fc1beffd700 1 -- 172.21.15.60:0/1633824787 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc1b80884c0 con 0x7fc1c80b8400 2023-07-13T22:20:49.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.866+0000 7fc1bffff700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1a806ac80 0x7fc1a806d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:49.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.867+0000 7fc1bffff700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1a806ac80 0x7fc1a806d130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1c815d470 tx=0x7fc1b0007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:49.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:49.871+0000 7fc1beffd700 1 -- 172.21.15.60:0/1633824787 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc1b8053870 con 0x7fc1c80b8400 2023-07-13T22:20:50.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.050+0000 7fc1beffd700 1 -- 172.21.15.60:0/1633824787 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc1b8003cd0 con 0x7fc1c80b8400 2023-07-13T22:20:50.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.080+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1c815a700 con 0x7fc1c80b8400 2023-07-13T22:20:50.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.081+0000 7fc1beffd700 1 -- 172.21.15.60:0/1633824787 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14543 (secure 0 0 0) 0x7fc1b8075db0 con 0x7fc1c80b8400 2023-07-13T22:20:50.065 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:50.066 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:49.718882+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:20:50.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1a806ac80 msgr2=0x7fc1a806d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1a806ac80 0x7fc1a806d130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1c815d470 tx=0x7fc1b0007eb0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80b8400 msgr2=0x7fc1c81f4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80b8400 0x7fc1c81f4e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c480 tx=0x7fc1b800c790 comp rx=0 tx=0).stop 2023-07-13T22:20:50.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 shutdown_connections 2023-07-13T22:20:50.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1a806ac80 0x7fc1a806d130 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80b8400 0x7fc1c81f4e30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80d3750 0x7fc1c80b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 --2- 172.21.15.60:0/1633824787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80cfca0 0x7fc1c815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 >> 172.21.15.60:0/1633824787 conn(0x7fc1c806b390 msgr2=0x7fc1c80d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:50.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 shutdown_connections 2023-07-13T22:20:50.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.083+0000 7fc1ccdff700 1 -- 172.21.15.60:0/1633824787 wait complete. 2023-07-13T22:20:50.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:20:50.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:20:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.309+0000 7f227efff700 1 Processor -- start 2023-07-13T22:20:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.310+0000 7f227efff700 1 -- start start 2023-07-13T22:20:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.311+0000 7f227efff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22780b8200 0x7f22780cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.311+0000 7f227efff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22780cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.311+0000 7f227efff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22780d1870 con 0x7f22780cede0 2023-07-13T22:20:50.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.311+0000 7f227efff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22780cf370 con 0x7f22780b8200 2023-07-13T22:20:50.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.311+0000 7f227efff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22780d12b0 con 0x7f22780b1ac0 2023-07-13T22:20:50.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.311+0000 7f2277fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22780cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.311+0000 7f2277fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22780cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:56368/0 (socket says 172.21.15.60:56368) 2023-07-13T22:20:50.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.311+0000 7f2277fff700 1 -- 172.21.15.60:0/1477718573 learned_addr learned my addr 172.21.15.60:0/1477718573 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:50.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.311+0000 7f227cd9b700 1 --2- 172.21.15.60:0/1477718573 >> v2:172.21.15.60:3300/0 conn(0x7f22780b8200 0x7f22780cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.312+0000 7f2277fff700 1 -- 172.21.15.60:0/1477718573 >> v2:172.21.15.60:3300/0 conn(0x7f22780b8200 msgr2=0x7f22780cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:50.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.312+0000 7f2277fff700 1 --2- 172.21.15.60:0/1477718573 >> v2:172.21.15.60:3300/0 conn(0x7f22780b8200 0x7f22780cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.312+0000 7f2277fff700 1 -- 172.21.15.60:0/1477718573 >> v1:172.21.15.60:6789/0 conn(0x7f22780cede0 legacy=0x7f227815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.312+0000 7f2277fff700 1 -- 172.21.15.60:0/1477718573 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780d1590 con 0x7f22780b1ac0 2023-07-13T22:20:50.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.312+0000 7f2277fff700 1 --2- 172.21.15.60:0/1477718573 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22780cfb80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22640098f0 tx=0x7f2264009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d55c839804183d48 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:50.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.312+0000 7f22777fe700 1 -- 172.21.15.60:0/1477718573 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400b8a0 con 0x7f22780b1ac0 2023-07-13T22:20:50.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.312+0000 7f22777fe700 1 -- 172.21.15.60:0/1477718573 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400ba00 con 0x7f22780b1ac0 2023-07-13T22:20:50.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.312+0000 7f22777fe700 1 -- 172.21.15.60:0/1477718573 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400bd90 con 0x7f22780b1ac0 2023-07-13T22:20:50.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.313+0000 7f227efff700 1 -- 172.21.15.60:0/1477718573 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 msgr2=0x7f22780cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.313+0000 7f227efff700 1 --2- 172.21.15.60:0/1477718573 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22780cfb80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22640098f0 tx=0x7f2264009c00 comp rx=0 tx=0).stop 2023-07-13T22:20:50.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.313+0000 7f227efff700 1 -- 172.21.15.60:0/1477718573 shutdown_connections 2023-07-13T22:20:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.313+0000 7f227efff700 1 --2- 172.21.15.60:0/1477718573 >> v2:172.21.15.60:3300/0 conn(0x7f22780b8200 0x7f22780cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.313+0000 7f227efff700 1 --2- 172.21.15.60:0/1477718573 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22780cfb80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.313+0000 7f227efff700 1 -- 172.21.15.60:0/1477718573 >> 172.21.15.60:0/1477718573 conn(0x7f227806b2f0 msgr2=0x7f227806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:50.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.313+0000 7f227efff700 1 -- 172.21.15.60:0/1477718573 shutdown_connections 2023-07-13T22:20:50.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.313+0000 7f227efff700 1 -- 172.21.15.60:0/1477718573 wait complete. 2023-07-13T22:20:50.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.315+0000 7f227efff700 1 Processor -- start 2023-07-13T22:20:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.315+0000 7f227efff700 1 -- start start 2023-07-13T22:20:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227efff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22781f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227efff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22780b8200 0x7f227814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227efff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22780cede0 0x7f2278149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227efff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278148f20 con 0x7f22780cede0 2023-07-13T22:20:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227efff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278147b10 con 0x7f22780b8200 2023-07-13T22:20:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227efff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22781482a0 con 0x7f22780b1ac0 2023-07-13T22:20:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227d59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22780cede0 0x7f2278149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227d59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22780cede0 0x7f2278149340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42600/0 (socket says 172.21.15.60:42600) 2023-07-13T22:20:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227d59c700 1 -- 172.21.15.60:0/4099968287 learned_addr learned my addr 172.21.15.60:0/4099968287 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f227cd9b700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22781f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.316+0000 7f2277fff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22780b8200 0x7f227814b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.317+0000 7f2277fff700 1 -- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 msgr2=0x7f22781f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.317+0000 7f2277fff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22781f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.317+0000 7f2277fff700 1 -- 172.21.15.60:0/4099968287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22780cede0 msgr2=0x7f2278149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.317+0000 7f2277fff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22780cede0 0x7f2278149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.317+0000 7f2277fff700 1 -- 172.21.15.60:0/4099968287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009550 con 0x7f22780b8200 2023-07-13T22:20:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.317+0000 7f227cd9b700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22781f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.317+0000 7f2277fff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22780b8200 0x7f227814b290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2264016000 tx=0x7f2264004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.318+0000 7f22757fa700 1 -- 172.21.15.60:0/4099968287 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264004300 con 0x7f22780b8200 2023-07-13T22:20:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.318+0000 7f22757fa700 1 -- 172.21.15.60:0/4099968287 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226401f070 con 0x7f22780b8200 2023-07-13T22:20:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.318+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2278148580 con 0x7f22780b8200 2023-07-13T22:20:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.318+0000 7f22757fa700 1 -- 172.21.15.60:0/4099968287 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22640036a0 con 0x7f22780b8200 2023-07-13T22:20:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.318+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2278148b40 con 0x7f22780b8200 2023-07-13T22:20:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.319+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2278149060 con 0x7f22780b8200 2023-07-13T22:20:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.320+0000 7f22757fa700 1 -- 172.21.15.60:0/4099968287 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2264021030 con 0x7f22780b8200 2023-07-13T22:20:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.321+0000 7f22757fa700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f225c06ace0 0x7f225c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.321+0000 7f22757fa700 1 -- 172.21.15.60:0/4099968287 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f226408a420 con 0x7f22780b8200 2023-07-13T22:20:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.321+0000 7f227cd9b700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f225c06ace0 0x7f225c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.322+0000 7f227cd9b700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f225c06ace0 0x7f225c06d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f226c009dc0 tx=0x7f226c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.326+0000 7f22757fa700 1 -- 172.21.15.60:0/4099968287 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2278149060 con 0x7f22780b8200 2023-07-13T22:20:50.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.539+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2278149060 con 0x7f22780b8200 2023-07-13T22:20:50.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.539+0000 7f22757fa700 1 -- 172.21.15.60:0/4099968287 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2278149060 con 0x7f22780b8200 2023-07-13T22:20:50.524 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:20:50.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f225c06ace0 msgr2=0x7f225c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f225c06ace0 0x7f225c06d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f226c009dc0 tx=0x7f226c008040 comp rx=0 tx=0).stop 2023-07-13T22:20:50.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22780b8200 msgr2=0x7f227814b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22780b8200 0x7f227814b290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2264016000 tx=0x7f2264004750 comp rx=0 tx=0).stop 2023-07-13T22:20:50.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 shutdown_connections 2023-07-13T22:20:50.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f225c06ace0 0x7f225c06d190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22780cede0 0x7f2278149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22780b1ac0 0x7f22781f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 --2- 172.21.15.60:0/4099968287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22780b8200 0x7f227814b290 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 >> 172.21.15.60:0/4099968287 conn(0x7f227806b2f0 msgr2=0x7f22780b7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:50.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 shutdown_connections 2023-07-13T22:20:50.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.542+0000 7f227efff700 1 -- 172.21.15.60:0/4099968287 wait complete. 2023-07-13T22:20:50.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:20:50.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.766+0000 7f36071ff700 1 Processor -- start 2023-07-13T22:20:50.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.767+0000 7f36071ff700 1 -- start start 2023-07-13T22:20:50.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f36071ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36000b8200 0x7f36000cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f36071ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36000cede0 0x7f36000cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f36071ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36000d1870 con 0x7f36000b1ac0 2023-07-13T22:20:50.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f36071ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36000cf370 con 0x7f36000b8200 2023-07-13T22:20:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f36071ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36000d12b0 con 0x7f36000cede0 2023-07-13T22:20:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f35fffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f36000b1ac0 0x7f360015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44028/0 (socket says 172.21.15.60:44028) 2023-07-13T22:20:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f35fffff700 1 -- 172.21.15.60:0/2061266321 learned_addr learned my addr 172.21.15.60:0/2061266321 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:50.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f360579c700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36000cede0 0x7f36000cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f3604f9b700 1 --2- 172.21.15.60:0/2061266321 >> v2:172.21.15.105:3300/0 conn(0x7f36000b8200 0x7f36000cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139181445 0 0) 0x7f36000d1870 con 0x7f36000b1ac0 2023-07-13T22:20:50.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.768+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f00035b0 con 0x7f36000b1ac0 2023-07-13T22:20:50.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1129001821 0 0) 0x7f35f00035b0 con 0x7f36000b1ac0 2023-07-13T22:20:50.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36000cede0 msgr2=0x7f36000cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:50.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36000cede0 0x7f36000cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 >> v2:172.21.15.105:3300/0 conn(0x7f36000b8200 msgr2=0x7f36000cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:50.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 --2- 172.21.15.60:0/2061266321 >> v2:172.21.15.105:3300/0 conn(0x7f36000b8200 0x7f36000cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36000d1590 con 0x7f36000b1ac0 2023-07-13T22:20:50.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f35ec001670 con 0x7f36000b1ac0 2023-07-13T22:20:50.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 >> v1:172.21.15.60:6789/0 conn(0x7f36000b1ac0 legacy=0x7f360015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f0003a00 0x7f35f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36000cede0 0x7f35f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f0004d90 0x7f35f0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36000cf370 con 0x7f35f0003a00 2023-07-13T22:20:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36000d12b0 con 0x7f35f0004d90 2023-07-13T22:20:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f000ba10 con 0x7f36000cede0 2023-07-13T22:20:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f35fffff700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36000cede0 0x7f35f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f3604f9b700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f0003a00 0x7f35f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.769+0000 7f360579c700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f0004d90 0x7f35f0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f360579c700 1 -- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36000cede0 msgr2=0x7f35f0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:50.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f360579c700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36000cede0 0x7f35f0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f360579c700 1 -- 172.21.15.60:0/2061266321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f0003a00 msgr2=0x7f35f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:50.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f360579c700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f0003a00 0x7f35f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f360579c700 1 -- 172.21.15.60:0/2061266321 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00035b0 con 0x7f35f0004d90 2023-07-13T22:20:50.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f35fffff700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36000cede0 0x7f35f0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:50.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f360579c700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f0004d90 0x7f35f0005200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009fc0 tx=0x7f35f0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33bc076eac2869ce server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:50.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f800ebd0 con 0x7f35f0004d90 2023-07-13T22:20:50.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f800f040 con 0x7f35f0004d90 2023-07-13T22:20:50.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.770+0000 7f35ff7fe700 1 -- 172.21.15.60:0/2061266321 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8014480 con 0x7f35f0004d90 2023-07-13T22:20:50.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.771+0000 7f36071ff700 1 -- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f0004d90 msgr2=0x7f35f0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.771+0000 7f36071ff700 1 --2- 172.21.15.60:0/2061266321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f0004d90 0x7f35f0005200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009fc0 tx=0x7f35f0005880 comp rx=0 tx=0).stop 2023-07-13T22:20:50.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.771+0000 7f360579c700 1 -- 172.21.15.60:0/2061266321 reap_dead start 2023-07-13T22:20:50.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.771+0000 7f36071ff700 1 -- 172.21.15.60:0/2061266321 shutdown_connections 2023-07-13T22:20:50.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.771+0000 7f36071ff700 1 -- 172.21.15.60:0/2061266321 >> 172.21.15.60:0/2061266321 conn(0x7f360006b2f0 msgr2=0x7f360006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:50.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.771+0000 7f36071ff700 1 -- 172.21.15.60:0/2061266321 shutdown_connections 2023-07-13T22:20:50.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.771+0000 7f36071ff700 1 -- 172.21.15.60:0/2061266321 wait complete. 2023-07-13T22:20:50.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.773+0000 7f36071ff700 1 Processor -- start 2023-07-13T22:20:50.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.773+0000 7f36071ff700 1 -- start start 2023-07-13T22:20:50.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f36071ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35f0004d90 0x7f3600155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f36071ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3600152d00 0x7f36001f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f36071ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3600153170 0x7f36001eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f36071ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3600154ce0 con 0x7f3600153170 2023-07-13T22:20:50.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f36071ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3600154fc0 con 0x7f3600152d00 2023-07-13T22:20:50.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f36071ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36001552a0 con 0x7f35f0004d90 2023-07-13T22:20:50.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f360579c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3600153170 0x7f36001eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f35fffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3600152d00 0x7f36001f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f360579c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3600153170 0x7f36001eeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42628/0 (socket says 172.21.15.60:42628) 2023-07-13T22:20:50.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f360579c700 1 -- 172.21.15.60:0/2042631231 learned_addr learned my addr 172.21.15.60:0/2042631231 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:50.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.774+0000 7f3604f9b700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35f0004d90 0x7f3600155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f360579c700 1 -- 172.21.15.60:0/2042631231 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35f0004d90 msgr2=0x7f3600155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f360579c700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35f0004d90 0x7f3600155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f360579c700 1 -- 172.21.15.60:0/2042631231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3600152d00 msgr2=0x7f36001f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f360579c700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3600152d00 0x7f36001f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f360579c700 1 -- 172.21.15.60:0/2042631231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600155580 con 0x7f3600153170 2023-07-13T22:20:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f35fffff700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3600152d00 0x7f36001f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f360579c700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3600153170 0x7f36001eeed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f35f8016000 tx=0x7f35f8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f35fd7fa700 1 -- 172.21.15.60:0/2042631231 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8007000 con 0x7f3600153170 2023-07-13T22:20:50.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f35fd7fa700 1 -- 172.21.15.60:0/2042631231 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f8004470 con 0x7f3600153170 2023-07-13T22:20:50.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3600154900 con 0x7f3600153170 2023-07-13T22:20:50.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.775+0000 7f35fd7fa700 1 -- 172.21.15.60:0/2042631231 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8004820 con 0x7f3600153170 2023-07-13T22:20:50.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.776+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36001f2cb0 con 0x7f3600153170 2023-07-13T22:20:50.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.777+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360004e210 con 0x7f3600153170 2023-07-13T22:20:50.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.778+0000 7f35fd7fa700 1 -- 172.21.15.60:0/2042631231 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35f8014480 con 0x7f3600153170 2023-07-13T22:20:50.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.779+0000 7f35fd7fa700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35e406ace0 0x7f35e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:50.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.779+0000 7f35fd7fa700 1 -- 172.21.15.60:0/2042631231 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f35f808aa80 con 0x7f3600153170 2023-07-13T22:20:50.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.779+0000 7f3604f9b700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35e406ace0 0x7f35e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:50.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.780+0000 7f3604f9b700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35e406ace0 0x7f35e406d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f360015be50 tx=0x7f35f4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:50.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.784+0000 7f35fd7fa700 1 -- 172.21.15.60:0/2042631231 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f35f8055eb0 con 0x7f3600153170 2023-07-13T22:20:50.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.992+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3600154e20 con 0x7f3600153170 2023-07-13T22:20:50.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.993+0000 7f35fd7fa700 1 -- 172.21.15.60:0/2042631231 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3600154e20 con 0x7f3600153170 2023-07-13T22:20:50.978 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:20:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35e406ace0 msgr2=0x7f35e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35e406ace0 0x7f35e406d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f360015be50 tx=0x7f35f4009210 comp rx=0 tx=0).stop 2023-07-13T22:20:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3600153170 msgr2=0x7f36001eeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3600153170 0x7f36001eeed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f35f8016000 tx=0x7f35f8009750 comp rx=0 tx=0).stop 2023-07-13T22:20:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 shutdown_connections 2023-07-13T22:20:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35e406ace0 0x7f35e406d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3600153170 0x7f36001eeed0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35f0004d90 0x7f3600155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 --2- 172.21.15.60:0/2042631231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3600152d00 0x7f36001f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 >> 172.21.15.60:0/2042631231 conn(0x7f360006b2f0 msgr2=0x7f36000734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.995+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 shutdown_connections 2023-07-13T22:20:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:50.996+0000 7f36071ff700 1 -- 172.21.15.60:0/2042631231 wait complete. 2023-07-13T22:20:50.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:20:51.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.216+0000 7fe1c31ff700 1 Processor -- start 2023-07-13T22:20:51.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.216+0000 7fe1c31ff700 1 -- start start 2023-07-13T22:20:51.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.217+0000 7fe1c31ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe1bc0d3750 0x7fe1bc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.217+0000 7fe1c31ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1bc0cfad0 0x7fe1bc0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.217+0000 7fe1c31ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc0cd950 con 0x7fe1bc0b8400 2023-07-13T22:20:51.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.217+0000 7fe1c31ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1bc0cff50 con 0x7fe1bc0cfad0 2023-07-13T22:20:51.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.217+0000 7fe1c31ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1bc0d0ac0 con 0x7fe1bc0d3750 2023-07-13T22:20:51.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.218+0000 7fe1c179c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1bc0cfad0 0x7fe1bc0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.218+0000 7fe1c179c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1bc0cfad0 0x7fe1bc0d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42634/0 (socket says 172.21.15.60:42634) 2023-07-13T22:20:51.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.218+0000 7fe1c179c700 1 -- 172.21.15.60:0/3727240960 learned_addr learned my addr 172.21.15.60:0/3727240960 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:51.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.218+0000 7fe1bbfff700 1 --2- 172.21.15.60:0/3727240960 >> v2:172.21.15.105:3300/0 conn(0x7fe1bc0d3750 0x7fe1bc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.218+0000 7fe1c179c700 1 -- 172.21.15.60:0/3727240960 >> v2:172.21.15.105:3300/0 conn(0x7fe1bc0d3750 msgr2=0x7fe1bc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:51.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.218+0000 7fe1c179c700 1 --2- 172.21.15.60:0/3727240960 >> v2:172.21.15.105:3300/0 conn(0x7fe1bc0d3750 0x7fe1bc0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.218+0000 7fe1c179c700 1 -- 172.21.15.60:0/3727240960 >> v1:172.21.15.105:6789/0 conn(0x7fe1bc0b8400 legacy=0x7fe1bc0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.218+0000 7fe1c179c700 1 -- 172.21.15.60:0/3727240960 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0b87d0 con 0x7fe1bc0cfad0 2023-07-13T22:20:51.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.218+0000 7fe1c179c700 1 --2- 172.21.15.60:0/3727240960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0cfad0 0x7fe1bc0d0f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0b8cf0 tx=0x7fe1b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8024f6b074edaddc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:51.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1bb7fe700 1 -- 172.21.15.60:0/3727240960 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400e070 con 0x7fe1bc0cfad0 2023-07-13T22:20:51.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1bb7fe700 1 -- 172.21.15.60:0/3727240960 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b400b910 con 0x7fe1bc0cfad0 2023-07-13T22:20:51.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1bb7fe700 1 -- 172.21.15.60:0/3727240960 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400bcc0 con 0x7fe1bc0cfad0 2023-07-13T22:20:51.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1c31ff700 1 -- 172.21.15.60:0/3727240960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0cfad0 msgr2=0x7fe1bc0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1c31ff700 1 --2- 172.21.15.60:0/3727240960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0cfad0 0x7fe1bc0d0f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0b8cf0 tx=0x7fe1b400b330 comp rx=0 tx=0).stop 2023-07-13T22:20:51.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1c31ff700 1 -- 172.21.15.60:0/3727240960 shutdown_connections 2023-07-13T22:20:51.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1c31ff700 1 --2- 172.21.15.60:0/3727240960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0cfad0 0x7fe1bc0d0f50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1c31ff700 1 --2- 172.21.15.60:0/3727240960 >> v2:172.21.15.105:3300/0 conn(0x7fe1bc0d3750 0x7fe1bc0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1c31ff700 1 -- 172.21.15.60:0/3727240960 >> 172.21.15.60:0/3727240960 conn(0x7fe1bc06b390 msgr2=0x7fe1bc06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:51.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1c31ff700 1 -- 172.21.15.60:0/3727240960 shutdown_connections 2023-07-13T22:20:51.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.219+0000 7fe1c31ff700 1 -- 172.21.15.60:0/3727240960 wait complete. 2023-07-13T22:20:51.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.221+0000 7fe1c31ff700 1 Processor -- start 2023-07-13T22:20:51.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.221+0000 7fe1c31ff700 1 -- start start 2023-07-13T22:20:51.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.222+0000 7fe1c31ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0b8400 0x7fe1bc1f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.222+0000 7fe1c31ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1bc0cfad0 0x7fe1bc150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.222+0000 7fe1c31ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1bc0d3750 0x7fe1bc15d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.222+0000 7fe1c31ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc15cd10 con 0x7fe1bc0b8400 2023-07-13T22:20:51.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.222+0000 7fe1c31ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc15c970 con 0x7fe1bc0cfad0 2023-07-13T22:20:51.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.222+0000 7fe1c31ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc1f2660 con 0x7fe1bc0d3750 2023-07-13T22:20:51.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.222+0000 7fe1c0f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0b8400 0x7fe1bc1f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c0f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0b8400 0x7fe1bc1f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42636/0 (socket says 172.21.15.60:42636) 2023-07-13T22:20:51.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c0f9b700 1 -- 172.21.15.60:0/408210228 learned_addr learned my addr 172.21.15.60:0/408210228 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:51.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c179c700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1bc0d3750 0x7fe1bc15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1bbfff700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1bc0cfad0 0x7fe1bc150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c0f9b700 1 -- 172.21.15.60:0/408210228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1bc0d3750 msgr2=0x7fe1bc15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c0f9b700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1bc0d3750 0x7fe1bc15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c0f9b700 1 -- 172.21.15.60:0/408210228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1bc0cfad0 msgr2=0x7fe1bc150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c0f9b700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1bc0cfad0 0x7fe1bc150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c0f9b700 1 -- 172.21.15.60:0/408210228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4009b30 con 0x7fe1bc0b8400 2023-07-13T22:20:51.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c0f9b700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0b8400 0x7fe1bc1f1df0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c450 tx=0x7fe1b000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:51.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1b97fa700 1 -- 172.21.15.60:0/408210228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00052d0 con 0x7fe1bc0b8400 2023-07-13T22:20:51.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1b97fa700 1 -- 172.21.15.60:0/408210228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b0005430 con 0x7fe1bc0b8400 2023-07-13T22:20:51.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc1f2940 con 0x7fe1bc0b8400 2023-07-13T22:20:51.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.223+0000 7fe1b97fa700 1 -- 172.21.15.60:0/408210228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00038c0 con 0x7fe1bc0b8400 2023-07-13T22:20:51.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.224+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc1f2330 con 0x7fe1bc0b8400 2023-07-13T22:20:51.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.225+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc15ce50 con 0x7fe1bc0b8400 2023-07-13T22:20:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.226+0000 7fe1b97fa700 1 -- 172.21.15.60:0/408210228 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe1b0003a20 con 0x7fe1bc0b8400 2023-07-13T22:20:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.227+0000 7fe1b97fa700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1a006ace0 0x7fe1a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.227+0000 7fe1b97fa700 1 -- 172.21.15.60:0/408210228 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe1b0088420 con 0x7fe1bc0b8400 2023-07-13T22:20:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.227+0000 7fe1bbfff700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1a006ace0 0x7fe1a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.228+0000 7fe1bbfff700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1a006ace0 0x7fe1a006d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc1546b0 tx=0x7fe1a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.231+0000 7fe1b97fa700 1 -- 172.21.15.60:0/408210228 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe1bc15ce50 con 0x7fe1bc0b8400 2023-07-13T22:20:51.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.438+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe1bc0cd950 con 0x7fe1bc0b8400 2023-07-13T22:20:51.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.439+0000 7fe1b97fa700 1 -- 172.21.15.60:0/408210228 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe1bc0cd950 con 0x7fe1bc0b8400 2023-07-13T22:20:51.423 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:51.424 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:20:51.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:20:51.424 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:20:51.424 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:20:51.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:20:51.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:20:51.424 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:20:51.425 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:20:51.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:20:51.425 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:20:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1a006ace0 msgr2=0x7fe1a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1a006ace0 0x7fe1a006d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc1546b0 tx=0x7fe1a8008040 comp rx=0 tx=0).stop 2023-07-13T22:20:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0b8400 msgr2=0x7fe1bc1f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0b8400 0x7fe1bc1f1df0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c450 tx=0x7fe1b000c810 comp rx=0 tx=0).stop 2023-07-13T22:20:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 shutdown_connections 2023-07-13T22:20:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1a006ace0 0x7fe1a006d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1bc0b8400 0x7fe1bc1f1df0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1bc0d3750 0x7fe1bc15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 --2- 172.21.15.60:0/408210228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1bc0cfad0 0x7fe1bc150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 >> 172.21.15.60:0/408210228 conn(0x7fe1bc06b390 msgr2=0x7fe1bc0b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:51.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 shutdown_connections 2023-07-13T22:20:51.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.441+0000 7fe1c31ff700 1 -- 172.21.15.60:0/408210228 wait complete. 2023-07-13T22:20:51.443 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:20:51.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:20:51.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.667+0000 7f5da5fff700 1 Processor -- start 2023-07-13T22:20:51.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.667+0000 7f5da5fff700 1 -- start start 2023-07-13T22:20:51.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.668+0000 7f5da5fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5da0159ea0 0x7f5da00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.668+0000 7f5da5fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5da00d0890 0x7f5da00d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.668+0000 7f5da5fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da015d200 con 0x7f5da015c9b0 2023-07-13T22:20:51.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.668+0000 7f5da5fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da00d2e60 con 0x7f5da00d0890 2023-07-13T22:20:51.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.668+0000 7f5da5fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da00d2b50 con 0x7f5da0159ea0 2023-07-13T22:20:51.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.668+0000 7f5d9effd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5da015c9b0 0x7f5da00d3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44040/0 (socket says 172.21.15.60:44040) 2023-07-13T22:20:51.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.668+0000 7f5d9effd700 1 -- 172.21.15.60:0/4088620501 learned_addr learned my addr 172.21.15.60:0/4088620501 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:51.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.668+0000 7f5d9ffff700 1 --2- 172.21.15.60:0/4088620501 >> v2:172.21.15.60:3300/0 conn(0x7f5da00d0890 0x7f5da00d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9f7fe700 1 --2- 172.21.15.60:0/4088620501 >> v2:172.21.15.105:3300/0 conn(0x7f5da0159ea0 0x7f5da00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9e7fc700 1 -- 172.21.15.60:0/4088620501 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084964763 0 0) 0x7f5da015d200 con 0x7f5da015c9b0 2023-07-13T22:20:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9e7fc700 1 -- 172.21.15.60:0/4088620501 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d880035b0 con 0x7f5da015c9b0 2023-07-13T22:20:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9ffff700 1 -- 172.21.15.60:0/4088620501 >> v2:172.21.15.105:3300/0 conn(0x7f5da0159ea0 msgr2=0x7f5da00cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9ffff700 1 --2- 172.21.15.60:0/4088620501 >> v2:172.21.15.105:3300/0 conn(0x7f5da0159ea0 0x7f5da00cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9ffff700 1 -- 172.21.15.60:0/4088620501 >> v1:172.21.15.60:6789/0 conn(0x7f5da015c9b0 legacy=0x7f5da00d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9ffff700 1 -- 172.21.15.60:0/4088620501 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da015d4e0 con 0x7f5da00d0890 2023-07-13T22:20:51.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9ffff700 1 --2- 172.21.15.60:0/4088620501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 0x7f5da00d0cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5da00cf6a0 tx=0x7f5d9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79dd2a3766beb02 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:51.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9e7fc700 1 -- 172.21.15.60:0/4088620501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800e070 con 0x7f5da00d0890 2023-07-13T22:20:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9e7fc700 1 -- 172.21.15.60:0/4088620501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9800bb20 con 0x7f5da00d0890 2023-07-13T22:20:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.669+0000 7f5d9e7fc700 1 -- 172.21.15.60:0/4088620501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98012430 con 0x7f5da00d0890 2023-07-13T22:20:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.670+0000 7f5da5fff700 1 -- 172.21.15.60:0/4088620501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 msgr2=0x7f5da00d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.670+0000 7f5da5fff700 1 --2- 172.21.15.60:0/4088620501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 0x7f5da00d0cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5da00cf6a0 tx=0x7f5d9800b330 comp rx=0 tx=0).stop 2023-07-13T22:20:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.670+0000 7f5da5fff700 1 -- 172.21.15.60:0/4088620501 shutdown_connections 2023-07-13T22:20:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.670+0000 7f5da5fff700 1 --2- 172.21.15.60:0/4088620501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 0x7f5da00d0cc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.670+0000 7f5da5fff700 1 --2- 172.21.15.60:0/4088620501 >> v2:172.21.15.105:3300/0 conn(0x7f5da0159ea0 0x7f5da00cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.670+0000 7f5da5fff700 1 -- 172.21.15.60:0/4088620501 >> 172.21.15.60:0/4088620501 conn(0x7f5da0071600 msgr2=0x7f5da00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.670+0000 7f5da5fff700 1 -- 172.21.15.60:0/4088620501 shutdown_connections 2023-07-13T22:20:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.670+0000 7f5da5fff700 1 -- 172.21.15.60:0/4088620501 wait complete. 2023-07-13T22:20:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.672+0000 7f5da5fff700 1 Processor -- start 2023-07-13T22:20:51.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.672+0000 7f5da5fff700 1 -- start start 2023-07-13T22:20:51.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5da5fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 0x7f5da00b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5da5fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5da0159ea0 0x7f5da00b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5da5fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5da015c9b0 0x7f5da0147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5da5fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00b5af0 con 0x7f5da00d0890 2023-07-13T22:20:51.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5da5fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00b5dd0 con 0x7f5da0159ea0 2023-07-13T22:20:51.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5da5fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00b60b0 con 0x7f5da015c9b0 2023-07-13T22:20:51.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5d9f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 0x7f5da00b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5d9f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 0x7f5da00b85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42644/0 (socket says 172.21.15.60:42644) 2023-07-13T22:20:51.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5d9f7fe700 1 -- 172.21.15.60:0/4083736554 learned_addr learned my addr 172.21.15.60:0/4083736554 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:51.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5d9effd700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5da0159ea0 0x7f5da00b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5d9ffff700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5da015c9b0 0x7f5da0147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5d9f7fe700 1 -- 172.21.15.60:0/4083736554 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5da015c9b0 msgr2=0x7f5da0147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5d9f7fe700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5da015c9b0 0x7f5da0147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.673+0000 7f5d9f7fe700 1 -- 172.21.15.60:0/4083736554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5da0159ea0 msgr2=0x7f5da00b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:51.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.674+0000 7f5d9f7fe700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5da0159ea0 0x7f5da00b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:51.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.674+0000 7f5d9f7fe700 1 -- 172.21.15.60:0/4083736554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d98009a70 con 0x7f5da00d0890 2023-07-13T22:20:51.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.674+0000 7f5d9f7fe700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 0x7f5da00b85d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e9b0 tx=0x7f5d9000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:51.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.674+0000 7f5d87fff700 1 -- 172.21.15.60:0/4083736554 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90009710 con 0x7f5da00d0890 2023-07-13T22:20:51.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.674+0000 7f5d87fff700 1 -- 172.21.15.60:0/4083736554 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9000c850 con 0x7f5da00d0890 2023-07-13T22:20:51.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.674+0000 7f5d87fff700 1 -- 172.21.15.60:0/4083736554 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000cbc0 con 0x7f5da00d0890 2023-07-13T22:20:51.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.674+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da00b6390 con 0x7f5da00d0890 2023-07-13T22:20:51.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.674+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da00b6950 con 0x7f5da00d0890 2023-07-13T22:20:51.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.675+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da0067a70 con 0x7f5da00d0890 2023-07-13T22:20:51.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.676+0000 7f5d87fff700 1 -- 172.21.15.60:0/4083736554 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d9000cd20 con 0x7f5da00d0890 2023-07-13T22:20:51.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.677+0000 7f5d87fff700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d8c06ace0 0x7f5d8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:51.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.677+0000 7f5d87fff700 1 -- 172.21.15.60:0/4083736554 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5d90089260 con 0x7f5da00d0890 2023-07-13T22:20:51.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.677+0000 7f5d9effd700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d8c06ace0 0x7f5d8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:51.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.678+0000 7f5d9effd700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d8c06ace0 0x7f5d8c06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5da015e4e0 tx=0x7f5d94007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:51.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.681+0000 7f5d87fff700 1 -- 172.21.15.60:0/4083736554 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5d900545c0 con 0x7f5da00d0890 2023-07-13T22:20:51.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:51.889+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5da00ce040 con 0x7f5da00d0890 2023-07-13T22:20:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.507+0000 7f5d87fff700 1 -- 172.21.15.60:0/4083736554 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5d90057be0 con 0x7f5da00d0890 2023-07-13T22:20:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.510+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d8c06ace0 msgr2=0x7f5d8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.510+0000 7f5da5fff700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d8c06ace0 0x7f5d8c06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5da015e4e0 tx=0x7f5d94007dc0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.510+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 msgr2=0x7f5da00b85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.510+0000 7f5da5fff700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 0x7f5da00b85d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e9b0 tx=0x7f5d9000ecc0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.511+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 shutdown_connections 2023-07-13T22:20:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.511+0000 7f5da5fff700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d8c06ace0 0x7f5d8c06d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.511+0000 7f5da5fff700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da00d0890 0x7f5da00b85d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.511+0000 7f5da5fff700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5da015c9b0 0x7f5da0147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.511+0000 7f5da5fff700 1 --2- 172.21.15.60:0/4083736554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5da0159ea0 0x7f5da00b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.511+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 >> 172.21.15.60:0/4083736554 conn(0x7f5da0071600 msgr2=0x7f5da015a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.511+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 shutdown_connections 2023-07-13T22:20:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.511+0000 7f5da5fff700 1 -- 172.21.15.60:0/4083736554 wait complete. 2023-07-13T22:20:52.498 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:20:52.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:20:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.751+0000 7fd9ad9ff700 1 Processor -- start 2023-07-13T22:20:52.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.751+0000 7fd9ad9ff700 1 -- start start 2023-07-13T22:20:52.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.752+0000 7fd9ad9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd9a815c6e0 0x7fd9a815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:52.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.752+0000 7fd9ad9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a80c8da0 0x7fd9a80cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:52.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.752+0000 7fd9ad9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a815cab0 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.752+0000 7fd9ad9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a80cafd0 con 0x7fd9a815c6e0 2023-07-13T22:20:52.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.752+0000 7fd9ad9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80ce570 con 0x7fd9a80c8da0 2023-07-13T22:20:52.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.753+0000 7fd9a67fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a80c8da0 0x7fd9a80cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:52.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.753+0000 7fd9a67fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a80c8da0 0x7fd9a80cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:56448/0 (socket says 172.21.15.60:56448) 2023-07-13T22:20:52.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.753+0000 7fd9a67fc700 1 -- 172.21.15.60:0/2793849273 learned_addr learned my addr 172.21.15.60:0/2793849273 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:52.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.753+0000 7fd9a6ffd700 1 --2- 172.21.15.60:0/2793849273 >> v2:172.21.15.105:3300/0 conn(0x7fd9a815c6e0 0x7fd9a815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:52.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.753+0000 7fd9a77fe700 1 --1- 172.21.15.60:0/2793849273 >> v1:172.21.15.105:6789/0 conn(0x7fd9a80ce0f0 0x7fd9a80cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:56010/0 (socket says 172.21.15.60:56010) 2023-07-13T22:20:52.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.753+0000 7fd9a5ffb700 1 -- 172.21.15.60:0/2793849273 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238757549 0 0) 0x7fd9a815cab0 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.753+0000 7fd9a5ffb700 1 -- 172.21.15.60:0/2793849273 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9900035b0 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.753+0000 7fd9a67fc700 1 -- 172.21.15.60:0/2793849273 >> v2:172.21.15.105:3300/0 conn(0x7fd9a815c6e0 msgr2=0x7fd9a815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:52.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.753+0000 7fd9a67fc700 1 --2- 172.21.15.60:0/2793849273 >> v2:172.21.15.105:3300/0 conn(0x7fd9a815c6e0 0x7fd9a815d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.754+0000 7fd9a67fc700 1 -- 172.21.15.60:0/2793849273 >> v1:172.21.15.105:6789/0 conn(0x7fd9a80ce0f0 legacy=0x7fd9a80cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:52.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.754+0000 7fd9a67fc700 1 -- 172.21.15.60:0/2793849273 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a80ca820 con 0x7fd9a80c8da0 2023-07-13T22:20:52.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.754+0000 7fd9a6ffd700 1 --2- 172.21.15.60:0/2793849273 >> v2:172.21.15.105:3300/0 conn(0x7fd9a815c6e0 0x7fd9a815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:52.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.754+0000 7fd9a67fc700 1 --2- 172.21.15.60:0/2793849273 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a80c8da0 0x7fd9a80cdbb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0098f0 tx=0x7fd99c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13fdaefa9813edb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:52.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.754+0000 7fd9a5ffb700 1 -- 172.21.15.60:0/2793849273 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c011830 con 0x7fd9a80c8da0 2023-07-13T22:20:52.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.754+0000 7fd9a5ffb700 1 -- 172.21.15.60:0/2793849273 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c011990 con 0x7fd9a80c8da0 2023-07-13T22:20:52.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.754+0000 7fd9a5ffb700 1 -- 172.21.15.60:0/2793849273 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c011d20 con 0x7fd9a80c8da0 2023-07-13T22:20:52.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.755+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/2793849273 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a80c8da0 msgr2=0x7fd9a80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:52.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.755+0000 7fd9ad9ff700 1 --2- 172.21.15.60:0/2793849273 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a80c8da0 0x7fd9a80cdbb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0098f0 tx=0x7fd99c009c00 comp rx=0 tx=0).stop 2023-07-13T22:20:52.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.755+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/2793849273 shutdown_connections 2023-07-13T22:20:52.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.755+0000 7fd9ad9ff700 1 --2- 172.21.15.60:0/2793849273 >> v2:172.21.15.105:3300/0 conn(0x7fd9a815c6e0 0x7fd9a815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.755+0000 7fd9ad9ff700 1 --2- 172.21.15.60:0/2793849273 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a80c8da0 0x7fd9a80cdbb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.755+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/2793849273 >> 172.21.15.60:0/2793849273 conn(0x7fd9a80cf3b0 msgr2=0x7fd9a80cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:52.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.755+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/2793849273 shutdown_connections 2023-07-13T22:20:52.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.755+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/2793849273 wait complete. 2023-07-13T22:20:52.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.757+0000 7fd9ad9ff700 1 Processor -- start 2023-07-13T22:20:52.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.757+0000 7fd9ad9ff700 1 -- start start 2023-07-13T22:20:52.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9ad9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a80c8da0 0x7fd9a80c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:52.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9ad9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a80ce0f0 0x7fd9a8143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:52.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9ad9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a815c6e0 0x7fd9a80c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:52.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9ad9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8143770 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9ad9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c84c0 con 0x7fd9a80c8da0 2023-07-13T22:20:52.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9ad9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c68a0 con 0x7fd9a815c6e0 2023-07-13T22:20:52.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a80ce0f0 0x7fd9a8143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:52.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a80ce0f0 0x7fd9a8143230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42658/0 (socket says 172.21.15.60:42658) 2023-07-13T22:20:52.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a67fc700 1 -- 172.21.15.60:0/1622304751 learned_addr learned my addr 172.21.15.60:0/1622304751 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:52.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a6ffd700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a80c8da0 0x7fd9a80c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:52.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a77fe700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a815c6e0 0x7fd9a80c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:52.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a67fc700 1 -- 172.21.15.60:0/1622304751 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a815c6e0 msgr2=0x7fd9a80c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:52.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a67fc700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a815c6e0 0x7fd9a80c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a67fc700 1 -- 172.21.15.60:0/1622304751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a80c8da0 msgr2=0x7fd9a80c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:52.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a67fc700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a80c8da0 0x7fd9a80c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:52.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.758+0000 7fd9a67fc700 1 -- 172.21.15.60:0/1622304751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9980094a0 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.759+0000 7fd9a67fc700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a80ce0f0 0x7fd9a8143230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009c40 tx=0x7fd99c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:52.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.759+0000 7fd98ffff700 1 -- 172.21.15.60:0/1622304751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c004300 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.759+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99c009550 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.759+0000 7fd98ffff700 1 -- 172.21.15.60:0/1622304751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c01f070 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.759+0000 7fd98ffff700 1 -- 172.21.15.60:0/1622304751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0036a0 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.759+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a80c6e60 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.760+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a804e210 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.761+0000 7fd98ffff700 1 -- 172.21.15.60:0/1622304751 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd99c021030 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.762+0000 7fd98ffff700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd99406ace0 0x7fd99406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:52.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.762+0000 7fd98ffff700 1 -- 172.21.15.60:0/1622304751 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fd99c08a9f0 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.762+0000 7fd9a6ffd700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd99406ace0 0x7fd99406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:52.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.763+0000 7fd9a6ffd700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd99406ace0 0x7fd99406d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd998009cd0 tx=0x7fd998009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:52.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.766+0000 7fd98ffff700 1 -- 172.21.15.60:0/1622304751 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd99c055c70 con 0x7fd9a80ce0f0 2023-07-13T22:20:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:52.972+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd9a80cb9e0 con 0x7fd9a80ce0f0 2023-07-13T22:20:53.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.514+0000 7fd98ffff700 1 -- 172.21.15.60:0/1622304751 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd99c0150d0 con 0x7fd9a80ce0f0 2023-07-13T22:20:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.519+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd99406ace0 msgr2=0x7fd99406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.519+0000 7fd9ad9ff700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd99406ace0 0x7fd99406d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd998009cd0 tx=0x7fd998009210 comp rx=0 tx=0).stop 2023-07-13T22:20:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.519+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a80ce0f0 msgr2=0x7fd9a8143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.519+0000 7fd9ad9ff700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a80ce0f0 0x7fd9a8143230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009c40 tx=0x7fd99c004750 comp rx=0 tx=0).stop 2023-07-13T22:20:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.521+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 shutdown_connections 2023-07-13T22:20:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.521+0000 7fd9ad9ff700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd99406ace0 0x7fd99406d190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.521+0000 7fd9ad9ff700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9a815c6e0 0x7fd9a80c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.521+0000 7fd9ad9ff700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a80c8da0 0x7fd9a80c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.521+0000 7fd9ad9ff700 1 --2- 172.21.15.60:0/1622304751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a80ce0f0 0x7fd9a8143230 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009c40 tx=0x7fd99c004750 comp rx=0 tx=0).stop 2023-07-13T22:20:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.521+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 >> 172.21.15.60:0/1622304751 conn(0x7fd9a80cf3b0 msgr2=0x7fd9a80cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.529+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 shutdown_connections 2023-07-13T22:20:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.531+0000 7fd9ad9ff700 1 -- 172.21.15.60:0/1622304751 wait complete. 2023-07-13T22:20:53.525 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:20:53.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:20:53.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.767+0000 7fcda75ff700 1 Processor -- start 2023-07-13T22:20:53.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.768+0000 7fcda75ff700 1 -- start start 2023-07-13T22:20:53.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.769+0000 7fcda75ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcda00b8200 0x7fcda00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:53.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.769+0000 7fcda75ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcda00b1ac0 0x7fcda00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:53.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.769+0000 7fcda75ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda00d1870 con 0x7fcda00cede0 2023-07-13T22:20:53.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.769+0000 7fcda75ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda00cf370 con 0x7fcda00b8200 2023-07-13T22:20:53.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.769+0000 7fcda75ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda00d12b0 con 0x7fcda00b1ac0 2023-07-13T22:20:53.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.769+0000 7fcda539b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcda00b8200 0x7fcda00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:53.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.769+0000 7fcda539b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcda00b8200 0x7fcda00cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42666/0 (socket says 172.21.15.60:42666) 2023-07-13T22:20:53.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.769+0000 7fcda539b700 1 -- 172.21.15.60:0/3886366337 learned_addr learned my addr 172.21.15.60:0/3886366337 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:53.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcda4b9a700 1 --2- 172.21.15.60:0/3886366337 >> v2:172.21.15.105:3300/0 conn(0x7fcda00b1ac0 0x7fcda00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:53.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcda539b700 1 -- 172.21.15.60:0/3886366337 >> v2:172.21.15.105:3300/0 conn(0x7fcda00b1ac0 msgr2=0x7fcda00cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:53.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcda539b700 1 --2- 172.21.15.60:0/3886366337 >> v2:172.21.15.105:3300/0 conn(0x7fcda00b1ac0 0x7fcda00cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:53.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcda539b700 1 -- 172.21.15.60:0/3886366337 >> v1:172.21.15.60:6789/0 conn(0x7fcda00cede0 legacy=0x7fcda015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:53.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcda539b700 1 -- 172.21.15.60:0/3886366337 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00d1590 con 0x7fcda00b8200 2023-07-13T22:20:53.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcd97fff700 1 -- 172.21.15.60:0/3886366337 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738575281 0 0) 0x7fcda00d1870 con 0x7fcda00cede0 2023-07-13T22:20:53.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcda4b9a700 1 --2- 172.21.15.60:0/3886366337 >> v2:172.21.15.105:3300/0 conn(0x7fcda00b1ac0 0x7fcda00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:20:53.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcda539b700 1 --2- 172.21.15.60:0/3886366337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 0x7fcda00cf640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcda00d07b0 tx=0x7fcd9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7424659e3584f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:53.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcd97fff700 1 -- 172.21.15.60:0/3886366337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c00b870 con 0x7fcda00b8200 2023-07-13T22:20:53.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcd97fff700 1 -- 172.21.15.60:0/3886366337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c00b9d0 con 0x7fcda00b8200 2023-07-13T22:20:53.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.770+0000 7fcd97fff700 1 -- 172.21.15.60:0/3886366337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c00bd60 con 0x7fcda00b8200 2023-07-13T22:20:53.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.771+0000 7fcda75ff700 1 -- 172.21.15.60:0/3886366337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 msgr2=0x7fcda00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:53.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.771+0000 7fcda75ff700 1 --2- 172.21.15.60:0/3886366337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 0x7fcda00cf640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcda00d07b0 tx=0x7fcd9c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:20:53.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.771+0000 7fcda75ff700 1 -- 172.21.15.60:0/3886366337 shutdown_connections 2023-07-13T22:20:53.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.771+0000 7fcda75ff700 1 --2- 172.21.15.60:0/3886366337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 0x7fcda00cf640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:53.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.771+0000 7fcda75ff700 1 --2- 172.21.15.60:0/3886366337 >> v2:172.21.15.105:3300/0 conn(0x7fcda00b1ac0 0x7fcda00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:53.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.771+0000 7fcda75ff700 1 -- 172.21.15.60:0/3886366337 >> 172.21.15.60:0/3886366337 conn(0x7fcda006b2f0 msgr2=0x7fcda006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:53.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.771+0000 7fcda75ff700 1 -- 172.21.15.60:0/3886366337 shutdown_connections 2023-07-13T22:20:53.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.771+0000 7fcda75ff700 1 -- 172.21.15.60:0/3886366337 wait complete. 2023-07-13T22:20:53.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.772+0000 7fcda75ff700 1 Processor -- start 2023-07-13T22:20:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.773+0000 7fcda75ff700 1 -- start start 2023-07-13T22:20:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda75ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcda00b1ac0 0x7fcda0156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda75ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 0x7fcda00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda75ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda00cede0 0x7fcda00d2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda75ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00cb430 con 0x7fcda00b8200 2023-07-13T22:20:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda75ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00cb640 con 0x7fcda00cede0 2023-07-13T22:20:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda75ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00cb920 con 0x7fcda00b1ac0 2023-07-13T22:20:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda4b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 0x7fcda00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda4b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 0x7fcda00cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42680/0 (socket says 172.21.15.60:42680) 2023-07-13T22:20:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda4b9a700 1 -- 172.21.15.60:0/2034910037 learned_addr learned my addr 172.21.15.60:0/2034910037 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda539b700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcda00b1ac0 0x7fcda0156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda5b9c700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda00cede0 0x7fcda00d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda4b9a700 1 -- 172.21.15.60:0/2034910037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcda00b1ac0 msgr2=0x7fcda0156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda4b9a700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcda00b1ac0 0x7fcda0156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda4b9a700 1 -- 172.21.15.60:0/2034910037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda00cede0 msgr2=0x7fcda00d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda4b9a700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda00cede0 0x7fcda00d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda4b9a700 1 -- 172.21.15.60:0/2034910037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c009550 con 0x7fcda00b8200 2023-07-13T22:20:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.774+0000 7fcda539b700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcda00b1ac0 0x7fcda0156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:20:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.775+0000 7fcda4b9a700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 0x7fcda00cc6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000c7f0 tx=0x7fcd9000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.775+0000 7fcd95ffb700 1 -- 172.21.15.60:0/2034910037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90011800 con 0x7fcda00b8200 2023-07-13T22:20:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.775+0000 7fcd95ffb700 1 -- 172.21.15.60:0/2034910037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd90011960 con 0x7fcda00b8200 2023-07-13T22:20:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.775+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda00cbc00 con 0x7fcda00b8200 2023-07-13T22:20:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.775+0000 7fcd95ffb700 1 -- 172.21.15.60:0/2034910037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90011d10 con 0x7fcda00b8200 2023-07-13T22:20:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.775+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda01567b0 con 0x7fcda00b8200 2023-07-13T22:20:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.776+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda00cbff0 con 0x7fcda00b8200 2023-07-13T22:20:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.777+0000 7fcd95ffb700 1 -- 172.21.15.60:0/2034910037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd90018360 con 0x7fcda00b8200 2023-07-13T22:20:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.778+0000 7fcd95ffb700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8406ace0 0x7fcd8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.778+0000 7fcd95ffb700 1 -- 172.21.15.60:0/2034910037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fcd900894a0 con 0x7fcda00b8200 2023-07-13T22:20:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.778+0000 7fcda539b700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8406ace0 0x7fcd8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.779+0000 7fcda539b700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8406ace0 0x7fcd8406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c000c00 tx=0x7fcd9c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:53.783+0000 7fcd95ffb700 1 -- 172.21.15.60:0/2034910037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcd90054510 con 0x7fcda00b8200 2023-07-13T22:20:54.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.037+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcda00cbdf0 con 0x7fcda00b8200 2023-07-13T22:20:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.517+0000 7fcd95ffb700 1 -- 172.21.15.60:0/2034910037 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcd90057b30 con 0x7fcda00b8200 2023-07-13T22:20:54.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.532+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8406ace0 msgr2=0x7fcd8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:54.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.532+0000 7fcda75ff700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8406ace0 0x7fcd8406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c000c00 tx=0x7fcd9c008030 comp rx=0 tx=0).stop 2023-07-13T22:20:54.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.532+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 msgr2=0x7fcda00cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:54.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.532+0000 7fcda75ff700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 0x7fcda00cc6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000c7f0 tx=0x7fcd9000cbb0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.533+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 shutdown_connections 2023-07-13T22:20:54.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.533+0000 7fcda75ff700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8406ace0 0x7fcd8406d190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.533+0000 7fcda75ff700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda00cede0 0x7fcda00d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.533+0000 7fcda75ff700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcda00b1ac0 0x7fcda0156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.533+0000 7fcda75ff700 1 --2- 172.21.15.60:0/2034910037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcda00b8200 0x7fcda00cc6c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.533+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 >> 172.21.15.60:0/2034910037 conn(0x7fcda006b2f0 msgr2=0x7fcda00cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:54.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.536+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 shutdown_connections 2023-07-13T22:20:54.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.536+0000 7fcda75ff700 1 -- 172.21.15.60:0/2034910037 wait complete. 2023-07-13T22:20:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 3 and data pool 4 2023-07-13T22:20:54.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:20:54.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:20:54.727 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:20:54.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.771+0000 7fd35bbff700 1 Processor -- start 2023-07-13T22:20:54.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.771+0000 7fd35bbff700 1 -- start start 2023-07-13T22:20:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.772+0000 7fd35bbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 0x7fd3540cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd35bbff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3540cd950 con 0x7fd3540b8400 2023-07-13T22:20:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd35bbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3540cff20 con 0x7fd3540cfad0 2023-07-13T22:20:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd35bbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540d0ac0 con 0x7fd3540d3750 2023-07-13T22:20:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd35999b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd3540b8400 0x7fd3540cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44054/0 (socket says 172.21.15.60:44054) 2023-07-13T22:20:54.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd35999b700 1 -- 172.21.15.60:0/4231342069 learned_addr learned my addr 172.21.15.60:0/4231342069 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:54.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd35919a700 1 --2- 172.21.15.60:0/4231342069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 0x7fd3540cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:54.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd358999700 1 -- 172.21.15.60:0/4231342069 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847826059 0 0) 0x7fd350001110 con 0x7fd3540b8400 2023-07-13T22:20:54.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd358999700 1 -- 172.21.15.60:0/4231342069 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3400035b0 con 0x7fd3540b8400 2023-07-13T22:20:54.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd35919a700 1 -- 172.21.15.60:0/4231342069 >> v1:172.21.15.105:6789/0 conn(0x7fd3540cfad0 legacy=0x7fd3540d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:54.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd358999700 1 -- 172.21.15.60:0/4231342069 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230547392 0 0) 0x7fd34c001490 con 0x7fd3540cfad0 2023-07-13T22:20:54.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd35919a700 1 -- 172.21.15.60:0/4231342069 >> v1:172.21.15.60:6789/0 conn(0x7fd3540b8400 legacy=0x7fd3540cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.773+0000 7fd35919a700 1 -- 172.21.15.60:0/4231342069 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3540b87d0 con 0x7fd3540d3750 2023-07-13T22:20:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.774+0000 7fd35919a700 1 --2- 172.21.15.60:0/4231342069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 0x7fd3540cf590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd344009870 tx=0x7fd344009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96760612633b9eba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.774+0000 7fd358999700 1 -- 172.21.15.60:0/4231342069 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344011830 con 0x7fd3540d3750 2023-07-13T22:20:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.774+0000 7fd358999700 1 -- 172.21.15.60:0/4231342069 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd344011990 con 0x7fd3540d3750 2023-07-13T22:20:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.774+0000 7fd358999700 1 -- 172.21.15.60:0/4231342069 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344011d20 con 0x7fd3540d3750 2023-07-13T22:20:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.775+0000 7fd35bbff700 1 -- 172.21.15.60:0/4231342069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 msgr2=0x7fd3540cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.775+0000 7fd35bbff700 1 --2- 172.21.15.60:0/4231342069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 0x7fd3540cf590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd344009870 tx=0x7fd344009b80 comp rx=0 tx=0).stop 2023-07-13T22:20:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.775+0000 7fd35bbff700 1 -- 172.21.15.60:0/4231342069 shutdown_connections 2023-07-13T22:20:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.775+0000 7fd35bbff700 1 --2- 172.21.15.60:0/4231342069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 0x7fd3540cf590 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.775+0000 7fd35bbff700 1 -- 172.21.15.60:0/4231342069 >> 172.21.15.60:0/4231342069 conn(0x7fd35406b390 msgr2=0x7fd35406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.775+0000 7fd35bbff700 1 -- 172.21.15.60:0/4231342069 shutdown_connections 2023-07-13T22:20:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.775+0000 7fd35bbff700 1 -- 172.21.15.60:0/4231342069 wait complete. 2023-07-13T22:20:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.777+0000 7fd35bbff700 1 Processor -- start 2023-07-13T22:20:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.777+0000 7fd35bbff700 1 -- start start 2023-07-13T22:20:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35bbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3540b8400 0x7fd3540c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35bbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3540cfad0 0x7fd35415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35bbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 0x7fd3540ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35bbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540cb080 con 0x7fd3540cfad0 2023-07-13T22:20:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35bbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540cb320 con 0x7fd3540b8400 2023-07-13T22:20:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35bbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540cb600 con 0x7fd3540d3750 2023-07-13T22:20:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3540cfad0 0x7fd35415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35999b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3540b8400 0x7fd3540c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3540cfad0 0x7fd35415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42692/0 (socket says 172.21.15.60:42692) 2023-07-13T22:20:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35919a700 1 -- 172.21.15.60:0/637544204 learned_addr learned my addr 172.21.15.60:0/637544204 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35999b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3540b8400 0x7fd3540c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:42476/0 (socket says 172.21.15.60:42476) 2023-07-13T22:20:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35a19c700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 0x7fd3540ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35919a700 1 -- 172.21.15.60:0/637544204 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 msgr2=0x7fd3540ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35919a700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 0x7fd3540ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35919a700 1 -- 172.21.15.60:0/637544204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3540b8400 msgr2=0x7fd3540c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.778+0000 7fd35919a700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3540b8400 0x7fd3540c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.779+0000 7fd35919a700 1 -- 172.21.15.60:0/637544204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3440094a0 con 0x7fd3540cfad0 2023-07-13T22:20:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.779+0000 7fd35919a700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3540cfad0 0x7fd35415d5d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd3440090b0 tx=0x7fd344004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.779+0000 7fd34a7fc700 1 -- 172.21.15.60:0/637544204 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34401a7e0 con 0x7fd3540cfad0 2023-07-13T22:20:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.779+0000 7fd34a7fc700 1 -- 172.21.15.60:0/637544204 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34401a940 con 0x7fd3540cfad0 2023-07-13T22:20:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.779+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3540cb8e0 con 0x7fd3540cfad0 2023-07-13T22:20:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.779+0000 7fd34a7fc700 1 -- 172.21.15.60:0/637544204 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34401acd0 con 0x7fd3540cfad0 2023-07-13T22:20:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.779+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35415db10 con 0x7fd3540cfad0 2023-07-13T22:20:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.781+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35415aba0 con 0x7fd3540cfad0 2023-07-13T22:20:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.782+0000 7fd34a7fc700 1 -- 172.21.15.60:0/637544204 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd344022070 con 0x7fd3540cfad0 2023-07-13T22:20:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.783+0000 7fd34a7fc700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd33806ace0 0x7fd33806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.783+0000 7fd34a7fc700 1 -- 172.21.15.60:0/637544204 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd34408a1c0 con 0x7fd3540cfad0 2023-07-13T22:20:54.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.784+0000 7fd35999b700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd33806ace0 0x7fd33806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:54.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.784+0000 7fd35999b700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd33806ace0 0x7fd33806d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd350005910 tx=0x7fd350009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:54.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.788+0000 7fd34a7fc700 1 -- 172.21.15.60:0/637544204 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd344055260 con 0x7fd3540cfad0 2023-07-13T22:20:54.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.997+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd35415c490 con 0x7fd3540cfad0 2023-07-13T22:20:54.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:54.998+0000 7fd34a7fc700 1 -- 172.21.15.60:0/637544204 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18198 (secure 0 0 0) 0x7fd344058880 con 0x7fd3540cfad0 2023-07-13T22:20:54.983 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:54.983 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":20,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:54.038045+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:20:54.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd33806ace0 msgr2=0x7fd33806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:54.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd33806ace0 0x7fd33806d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd350005910 tx=0x7fd350009040 comp rx=0 tx=0).stop 2023-07-13T22:20:54.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3540cfad0 msgr2=0x7fd35415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:54.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3540cfad0 0x7fd35415d5d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd3440090b0 tx=0x7fd344004750 comp rx=0 tx=0).stop 2023-07-13T22:20:54.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 shutdown_connections 2023-07-13T22:20:54.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd33806ace0 0x7fd33806d190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3540d3750 0x7fd3540ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3540b8400 0x7fd3540c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 --2- 172.21.15.60:0/637544204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3540cfad0 0x7fd35415d5d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:54.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.000+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 >> 172.21.15.60:0/637544204 conn(0x7fd35406b390 msgr2=0x7fd35415b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:54.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.001+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 shutdown_connections 2023-07-13T22:20:54.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.001+0000 7fd35bbff700 1 -- 172.21.15.60:0/637544204 wait complete. 2023-07-13T22:20:55.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:20:55.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.221+0000 7fefbb3ff700 1 Processor -- start 2023-07-13T22:20:55.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.222+0000 7fefbb3ff700 1 -- start start 2023-07-13T22:20:55.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.223+0000 7fefbb3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40cfca0 0x7fefb4167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.223+0000 7fefbb3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb40cd950 con 0x7fefb40d3750 2023-07-13T22:20:55.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.223+0000 7fefbb3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb40d0ac0 con 0x7fefb40b8400 2023-07-13T22:20:55.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.223+0000 7fefbb3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40b87d0 con 0x7fefb40cfca0 2023-07-13T22:20:55.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.223+0000 7fefb899a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fefb40d3750 0x7fefb40cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39484/0 (socket says 172.21.15.60:39484) 2023-07-13T22:20:55.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.223+0000 7fefb899a700 1 -- 172.21.15.60:0/2553163747 learned_addr learned my addr 172.21.15.60:0/2553163747 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:55.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.223+0000 7fefb999c700 1 --2- 172.21.15.60:0/2553163747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40cfca0 0x7fefb4167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.223+0000 7fefabfff700 1 -- 172.21.15.60:0/2553163747 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100719696 0 0) 0x7fefa4001110 con 0x7fefb40d3750 2023-07-13T22:20:55.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.223+0000 7fefabfff700 1 -- 172.21.15.60:0/2553163747 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefa00035b0 con 0x7fefb40d3750 2023-07-13T22:20:55.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefabfff700 1 -- 172.21.15.60:0/2553163747 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345484499 0 0) 0x7fefb40d0ac0 con 0x7fefb40b8400 2023-07-13T22:20:55.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefabfff700 1 -- 172.21.15.60:0/2553163747 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefa4001110 con 0x7fefb40b8400 2023-07-13T22:20:55.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefb999c700 1 -- 172.21.15.60:0/2553163747 >> v1:172.21.15.105:6789/0 conn(0x7fefb40b8400 legacy=0x7fefb40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefabfff700 1 -- 172.21.15.60:0/2553163747 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749367110 0 0) 0x7fefa00035b0 con 0x7fefb40d3750 2023-07-13T22:20:55.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefb999c700 1 -- 172.21.15.60:0/2553163747 >> v1:172.21.15.60:6789/0 conn(0x7fefb40d3750 legacy=0x7fefb40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefb999c700 1 -- 172.21.15.60:0/2553163747 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb40b4eb0 con 0x7fefb40cfca0 2023-07-13T22:20:55.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefb999c700 1 --2- 172.21.15.60:0/2553163747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40cfca0 0x7fefb4167050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fefac009dc0 tx=0x7fefac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc3641076b2c1d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:55.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefabfff700 1 -- 172.21.15.60:0/2553163747 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00e070 con 0x7fefb40cfca0 2023-07-13T22:20:55.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefabfff700 1 -- 172.21.15.60:0/2553163747 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac003a20 con 0x7fefb40cfca0 2023-07-13T22:20:55.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.224+0000 7fefabfff700 1 -- 172.21.15.60:0/2553163747 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4001800 con 0x7fefb40cfca0 2023-07-13T22:20:55.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.225+0000 7fefbb3ff700 1 -- 172.21.15.60:0/2553163747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40cfca0 msgr2=0x7fefb4167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.225+0000 7fefbb3ff700 1 --2- 172.21.15.60:0/2553163747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40cfca0 0x7fefb4167050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fefac009dc0 tx=0x7fefac00b330 comp rx=0 tx=0).stop 2023-07-13T22:20:55.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.225+0000 7fefbb3ff700 1 -- 172.21.15.60:0/2553163747 shutdown_connections 2023-07-13T22:20:55.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.225+0000 7fefbb3ff700 1 --2- 172.21.15.60:0/2553163747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40cfca0 0x7fefb4167050 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.225+0000 7fefbb3ff700 1 -- 172.21.15.60:0/2553163747 >> 172.21.15.60:0/2553163747 conn(0x7fefb406b390 msgr2=0x7fefb406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:55.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.225+0000 7fefbb3ff700 1 -- 172.21.15.60:0/2553163747 shutdown_connections 2023-07-13T22:20:55.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.225+0000 7fefbb3ff700 1 -- 172.21.15.60:0/2553163747 wait complete. 2023-07-13T22:20:55.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.227+0000 7fefbb3ff700 1 Processor -- start 2023-07-13T22:20:55.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.227+0000 7fefbb3ff700 1 -- start start 2023-07-13T22:20:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.228+0000 7fefbb3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefb40b8400 0x7fefb41596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.228+0000 7fefbb3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb40cfca0 0x7fefb4153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.228+0000 7fefbb3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40d3750 0x7fefb415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.228+0000 7fefbb3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb4151130 con 0x7fefb40b8400 2023-07-13T22:20:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.228+0000 7fefbb3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40cb670 con 0x7fefb40cfca0 2023-07-13T22:20:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.228+0000 7fefbb3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb4152640 con 0x7fefb40d3750 2023-07-13T22:20:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.228+0000 7fefb919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefb40b8400 0x7fefb41596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.228+0000 7fefb919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefb40b8400 0x7fefb41596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33656/0 (socket says 172.21.15.60:33656) 2023-07-13T22:20:55.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.228+0000 7fefb919b700 1 -- 172.21.15.60:0/3010336850 learned_addr learned my addr 172.21.15.60:0/3010336850 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:55.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefb999c700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40d3750 0x7fefb415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefb899a700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb40cfca0 0x7fefb4153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefb919b700 1 -- 172.21.15.60:0/3010336850 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40d3750 msgr2=0x7fefb415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefb919b700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40d3750 0x7fefb415c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefb919b700 1 -- 172.21.15.60:0/3010336850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb40cfca0 msgr2=0x7fefb4153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefb919b700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb40cfca0 0x7fefb4153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefb919b700 1 -- 172.21.15.60:0/3010336850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac009a70 con 0x7fefb40b8400 2023-07-13T22:20:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefb999c700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40d3750 0x7fefb415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:20:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefb919b700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefb40b8400 0x7fefb41596d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fefb000c500 tx=0x7fefb000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefa9ffb700 1 -- 172.21.15.60:0/3010336850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb00052d0 con 0x7fefb40b8400 2023-07-13T22:20:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefa9ffb700 1 -- 172.21.15.60:0/3010336850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb0005450 con 0x7fefb40b8400 2023-07-13T22:20:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb4152920 con 0x7fefb40b8400 2023-07-13T22:20:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefa9ffb700 1 -- 172.21.15.60:0/3010336850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb00038c0 con 0x7fefb40b8400 2023-07-13T22:20:55.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.229+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb40cbba0 con 0x7fefb40b8400 2023-07-13T22:20:55.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.230+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb415b410 con 0x7fefb40b8400 2023-07-13T22:20:55.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.231+0000 7fefa9ffb700 1 -- 172.21.15.60:0/3010336850 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefb0003a20 con 0x7fefb40b8400 2023-07-13T22:20:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.232+0000 7fefa9ffb700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef9806ace0 0x7fef9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.232+0000 7fefa9ffb700 1 -- 172.21.15.60:0/3010336850 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fefb00889c0 con 0x7fefb40b8400 2023-07-13T22:20:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.233+0000 7fefb899a700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef9806ace0 0x7fef9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.233+0000 7fefb899a700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef9806ace0 0x7fef9806d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fefa4007fd0 tx=0x7fefa4007a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:55.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.236+0000 7fefa9ffb700 1 -- 172.21.15.60:0/3010336850 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fefb00539d0 con 0x7fefb40b8400 2023-07-13T22:20:55.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.443+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefb415b130 con 0x7fefb40b8400 2023-07-13T22:20:55.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.444+0000 7fefa9ffb700 1 -- 172.21.15.60:0/3010336850 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18198 (secure 0 0 0) 0x7fefb0056ff0 con 0x7fefb40b8400 2023-07-13T22:20:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:55.429 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":20,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:54.038045+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:20:55.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.446+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef9806ace0 msgr2=0x7fef9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.446+0000 7fefbb3ff700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef9806ace0 0x7fef9806d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fefa4007fd0 tx=0x7fefa4007a00 comp rx=0 tx=0).stop 2023-07-13T22:20:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.446+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefb40b8400 msgr2=0x7fefb41596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.446+0000 7fefbb3ff700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefb40b8400 0x7fefb41596d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fefb000c500 tx=0x7fefb000c8c0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.447+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 shutdown_connections 2023-07-13T22:20:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.447+0000 7fefbb3ff700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef9806ace0 0x7fef9806d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.447+0000 7fefbb3ff700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefb40b8400 0x7fefb41596d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.447+0000 7fefbb3ff700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefb40d3750 0x7fefb415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.447+0000 7fefbb3ff700 1 --2- 172.21.15.60:0/3010336850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefb40cfca0 0x7fefb4153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.447+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 >> 172.21.15.60:0/3010336850 conn(0x7fefb406b390 msgr2=0x7fefb40d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.447+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 shutdown_connections 2023-07-13T22:20:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.447+0000 7fefbb3ff700 1 -- 172.21.15.60:0/3010336850 wait complete. 2023-07-13T22:20:55.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:20:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.675+0000 7ff6a99ff700 1 Processor -- start 2023-07-13T22:20:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.675+0000 7ff6a99ff700 1 -- start start 2023-07-13T22:20:55.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.676+0000 7ff6a99ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6a40b8200 0x7ff6a40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.676+0000 7ff6a99ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a40d1870 con 0x7ff6a40b1ac0 2023-07-13T22:20:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.676+0000 7ff6a99ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a4160d80 con 0x7ff6a40cede0 2023-07-13T22:20:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.676+0000 7ff6a99ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a40cf370 con 0x7ff6a40b8200 2023-07-13T22:20:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.676+0000 7ff6a2ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6a40b8200 0x7ff6a40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.677+0000 7ff6a2ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6a40b8200 0x7ff6a40cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33670/0 (socket says 172.21.15.60:33670) 2023-07-13T22:20:55.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.677+0000 7ff6a2ffd700 1 -- 172.21.15.60:0/2777893290 learned_addr learned my addr 172.21.15.60:0/2777893290 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:55.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.677+0000 7ff6a2ffd700 1 -- 172.21.15.60:0/2777893290 >> v1:172.21.15.105:6789/0 conn(0x7ff6a40cede0 legacy=0x7ff6a4160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.677+0000 7ff6a2ffd700 1 -- 172.21.15.60:0/2777893290 >> v1:172.21.15.60:6789/0 conn(0x7ff6a40b1ac0 legacy=0x7ff6a415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.677+0000 7ff6a2ffd700 1 -- 172.21.15.60:0/2777893290 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a40d12b0 con 0x7ff6a40b8200 2023-07-13T22:20:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.677+0000 7ff6a2ffd700 1 --2- 172.21.15.60:0/2777893290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 0x7ff6a40cf640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff6940098f0 tx=0x7ff694009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccd1e5989d56139 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.677+0000 7ff6a1ffb700 1 -- 172.21.15.60:0/2777893290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69400b8a0 con 0x7ff6a40b8200 2023-07-13T22:20:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.678+0000 7ff6a1ffb700 1 -- 172.21.15.60:0/2777893290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69400ba00 con 0x7ff6a40b8200 2023-07-13T22:20:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.678+0000 7ff6a1ffb700 1 -- 172.21.15.60:0/2777893290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69400bd90 con 0x7ff6a40b8200 2023-07-13T22:20:55.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.678+0000 7ff6a99ff700 1 -- 172.21.15.60:0/2777893290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 msgr2=0x7ff6a40cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.678+0000 7ff6a99ff700 1 --2- 172.21.15.60:0/2777893290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 0x7ff6a40cf640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff6940098f0 tx=0x7ff694009c00 comp rx=0 tx=0).stop 2023-07-13T22:20:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.678+0000 7ff6a99ff700 1 -- 172.21.15.60:0/2777893290 shutdown_connections 2023-07-13T22:20:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.678+0000 7ff6a99ff700 1 --2- 172.21.15.60:0/2777893290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 0x7ff6a40cf640 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.678+0000 7ff6a99ff700 1 -- 172.21.15.60:0/2777893290 >> 172.21.15.60:0/2777893290 conn(0x7ff6a406b2f0 msgr2=0x7ff6a406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.678+0000 7ff6a99ff700 1 -- 172.21.15.60:0/2777893290 shutdown_connections 2023-07-13T22:20:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.678+0000 7ff6a99ff700 1 -- 172.21.15.60:0/2777893290 wait complete. 2023-07-13T22:20:55.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.680+0000 7ff6a99ff700 1 Processor -- start 2023-07-13T22:20:55.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.680+0000 7ff6a99ff700 1 -- start start 2023-07-13T22:20:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.681+0000 7ff6a99ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a40b1ac0 0x7ff6a4154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.681+0000 7ff6a99ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 0x7ff6a40cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.681+0000 7ff6a99ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6a40cede0 0x7ff6a40c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.681+0000 7ff6a99ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40caab0 con 0x7ff6a40b8200 2023-07-13T22:20:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.681+0000 7ff6a99ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40c9ea0 con 0x7ff6a40b1ac0 2023-07-13T22:20:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.681+0000 7ff6a99ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40ca180 con 0x7ff6a40cede0 2023-07-13T22:20:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.681+0000 7ff6a27fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 0x7ff6a40cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.681+0000 7ff6a27fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 0x7ff6a40cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33684/0 (socket says 172.21.15.60:33684) 2023-07-13T22:20:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.681+0000 7ff6a27fc700 1 -- 172.21.15.60:0/171349967 learned_addr learned my addr 172.21.15.60:0/171349967 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a2ffd700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a40b1ac0 0x7ff6a4154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a37fe700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6a40cede0 0x7ff6a40c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a27fc700 1 -- 172.21.15.60:0/171349967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6a40cede0 msgr2=0x7ff6a40c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a27fc700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6a40cede0 0x7ff6a40c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a27fc700 1 -- 172.21.15.60:0/171349967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a40b1ac0 msgr2=0x7ff6a4154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a27fc700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a40b1ac0 0x7ff6a4154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a27fc700 1 -- 172.21.15.60:0/171349967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff694009550 con 0x7ff6a40b8200 2023-07-13T22:20:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a37fe700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6a40cede0 0x7ff6a40c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:20:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a27fc700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 0x7ff6a40cbe30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c4e0 tx=0x7ff69800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff68bfff700 1 -- 172.21.15.60:0/171349967 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6980052d0 con 0x7ff6a40b8200 2023-07-13T22:20:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff68bfff700 1 -- 172.21.15.60:0/171349967 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff698005430 con 0x7ff6a40b8200 2023-07-13T22:20:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff68bfff700 1 -- 172.21.15.60:0/171349967 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6980038c0 con 0x7ff6a40b8200 2023-07-13T22:20:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.682+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a40ca460 con 0x7ff6a40b8200 2023-07-13T22:20:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.683+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a4154740 con 0x7ff6a40b8200 2023-07-13T22:20:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.684+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a40cac60 con 0x7ff6a40b8200 2023-07-13T22:20:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.685+0000 7ff68bfff700 1 -- 172.21.15.60:0/171349967 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff698003a20 con 0x7ff6a40b8200 2023-07-13T22:20:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.685+0000 7ff68bfff700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff69006ace0 0x7ff69006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.686+0000 7ff68bfff700 1 -- 172.21.15.60:0/171349967 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff698088900 con 0x7ff6a40b8200 2023-07-13T22:20:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.686+0000 7ff6a2ffd700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff69006ace0 0x7ff69006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.686+0000 7ff6a2ffd700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff69006ace0 0x7ff69006d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff694009c40 tx=0x7ff694008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.690+0000 7ff68bfff700 1 -- 172.21.15.60:0/171349967 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff6a40cac60 con 0x7ff6a40b8200 2023-07-13T22:20:55.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:55.943+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff6a404e210 con 0x7ff6a40b8200 2023-07-13T22:20:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.537+0000 7ff68bfff700 1 -- 172.21.15.60:0/171349967 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff698053910 con 0x7ff6a40b8200 2023-07-13T22:20:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.539+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff69006ace0 msgr2=0x7ff69006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.539+0000 7ff6a99ff700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff69006ace0 0x7ff69006d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff694009c40 tx=0x7ff694008030 comp rx=0 tx=0).stop 2023-07-13T22:20:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 msgr2=0x7ff6a40cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 0x7ff6a40cbe30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c4e0 tx=0x7ff69800c7f0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 shutdown_connections 2023-07-13T22:20:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff69006ace0 0x7ff69006d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6a40cede0 0x7ff6a40c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a40b1ac0 0x7ff6a4154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 --2- 172.21.15.60:0/171349967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a40b8200 0x7ff6a40cbe30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 >> 172.21.15.60:0/171349967 conn(0x7ff6a406b2f0 msgr2=0x7ff6a4071600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 shutdown_connections 2023-07-13T22:20:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.540+0000 7ff6a99ff700 1 -- 172.21.15.60:0/171349967 wait complete. 2023-07-13T22:20:56.543 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:20:56.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:20:56.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.768+0000 7fd1759ff700 1 Processor -- start 2023-07-13T22:20:56.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.768+0000 7fd1759ff700 1 -- start start 2023-07-13T22:20:56.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.769+0000 7fd1759ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700cfd50 0x7fd1700d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:56.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.769+0000 7fd1759ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1700b1ac0 0x7fd1700d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:56.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.769+0000 7fd1759ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1700d1870 con 0x7fd1700b8200 2023-07-13T22:20:56.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.769+0000 7fd1759ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1700cf370 con 0x7fd1700b1ac0 2023-07-13T22:20:56.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.769+0000 7fd1759ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700d12b0 con 0x7fd1700cfd50 2023-07-13T22:20:56.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.769+0000 7fd16effd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd1700b8200 0x7fd1700cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39508/0 (socket says 172.21.15.60:39508) 2023-07-13T22:20:56.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.769+0000 7fd16effd700 1 -- 172.21.15.60:0/917869074 learned_addr learned my addr 172.21.15.60:0/917869074 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:56.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16f7fe700 1 --2- 172.21.15.60:0/917869074 >> v2:172.21.15.105:3300/0 conn(0x7fd1700b1ac0 0x7fd1700d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:56.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16e7fc700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700cfd50 0x7fd1700d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:56.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741292607 0 0) 0x7fd1700d1870 con 0x7fd1700b8200 2023-07-13T22:20:56.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1580035b0 con 0x7fd1700b8200 2023-07-13T22:20:56.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3470532292 0 0) 0x7fd1580035b0 con 0x7fd1700b8200 2023-07-13T22:20:56.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700cfd50 msgr2=0x7fd1700d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:56.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700cfd50 0x7fd1700d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 >> v2:172.21.15.105:3300/0 conn(0x7fd1700b1ac0 msgr2=0x7fd1700d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:56.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 --2- 172.21.15.60:0/917869074 >> v2:172.21.15.105:3300/0 conn(0x7fd1700b1ac0 0x7fd1700d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700d1590 con 0x7fd1700b8200 2023-07-13T22:20:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fd160001670 con 0x7fd1700b8200 2023-07-13T22:20:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 >> v1:172.21.15.60:6789/0 conn(0x7fd1700b8200 legacy=0x7fd1700cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd158003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd158004430 0x7fd1580048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700cfd50 0x7fd158004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700cf370 con 0x7fd158003a00 2023-07-13T22:20:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700d12b0 con 0x7fd158004430 2023-07-13T22:20:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15800ba10 con 0x7fd1700cfd50 2023-07-13T22:20:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.770+0000 7fd16effd700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd158003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16e7fc700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd158004430 0x7fd1580048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16f7fe700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700cfd50 0x7fd158004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16effd700 1 -- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700cfd50 msgr2=0x7fd158004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16effd700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700cfd50 0x7fd158004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16effd700 1 -- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd158004430 msgr2=0x7fd1580048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16effd700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd158004430 0x7fd1580048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16effd700 1 -- 172.21.15.60:0/917869074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580035b0 con 0x7fd158003a00 2023-07-13T22:20:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16effd700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd158003e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c2a0 tx=0x7fd158005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773be5dc1df1c824 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160007c10 con 0x7fd158003a00 2023-07-13T22:20:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16000cde0 con 0x7fd158003a00 2023-07-13T22:20:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.771+0000 7fd16dffb700 1 -- 172.21.15.60:0/917869074 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160005ba0 con 0x7fd158003a00 2023-07-13T22:20:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.772+0000 7fd1759ff700 1 -- 172.21.15.60:0/917869074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 msgr2=0x7fd158003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.772+0000 7fd1759ff700 1 --2- 172.21.15.60:0/917869074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd158003e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c2a0 tx=0x7fd158005880 comp rx=0 tx=0).stop 2023-07-13T22:20:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.772+0000 7fd16f7fe700 1 -- 172.21.15.60:0/917869074 reap_dead start 2023-07-13T22:20:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.772+0000 7fd1759ff700 1 -- 172.21.15.60:0/917869074 shutdown_connections 2023-07-13T22:20:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.772+0000 7fd1759ff700 1 -- 172.21.15.60:0/917869074 >> 172.21.15.60:0/917869074 conn(0x7fd17006b2f0 msgr2=0x7fd17006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.772+0000 7fd1759ff700 1 -- 172.21.15.60:0/917869074 shutdown_connections 2023-07-13T22:20:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.772+0000 7fd1759ff700 1 -- 172.21.15.60:0/917869074 wait complete. 2023-07-13T22:20:56.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.774+0000 7fd1759ff700 1 Processor -- start 2023-07-13T22:20:56.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.774+0000 7fd1759ff700 1 -- start start 2023-07-13T22:20:56.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd1759ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd1700c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:56.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd1759ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1700cb380 0x7fd1701f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:56.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd1759ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700c85e0 0x7fd1700c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:56.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd1759ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700c8f90 con 0x7fd158003a00 2023-07-13T22:20:56.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd1759ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700caef0 con 0x7fd1700cb380 2023-07-13T22:20:56.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd1759ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700ca2e0 con 0x7fd1700c85e0 2023-07-13T22:20:56.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd16effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd1700c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:56.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd16effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd1700c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33702/0 (socket says 172.21.15.60:33702) 2023-07-13T22:20:56.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd16effd700 1 -- 172.21.15.60:0/3992161077 learned_addr learned my addr 172.21.15.60:0/3992161077 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:56.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd16e7fc700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1700cb380 0x7fd1701f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:56.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.775+0000 7fd16f7fe700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700c85e0 0x7fd1700c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd16effd700 1 -- 172.21.15.60:0/3992161077 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700c85e0 msgr2=0x7fd1700c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd16effd700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700c85e0 0x7fd1700c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd16effd700 1 -- 172.21.15.60:0/3992161077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1700cb380 msgr2=0x7fd1701f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd16effd700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1700cb380 0x7fd1701f5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd16effd700 1 -- 172.21.15.60:0/3992161077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700ca5c0 con 0x7fd158003a00 2023-07-13T22:20:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd16effd700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd1700c9d00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c890 tx=0x7fd1700ce0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd157fff700 1 -- 172.21.15.60:0/3992161077 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160001630 con 0x7fd158003a00 2023-07-13T22:20:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd157fff700 1 -- 172.21.15.60:0/3992161077 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1600070d0 con 0x7fd158003a00 2023-07-13T22:20:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd157fff700 1 -- 172.21.15.60:0/3992161077 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160007460 con 0x7fd158003a00 2023-07-13T22:20:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1700ca8a0 con 0x7fd158003a00 2023-07-13T22:20:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.776+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1700c6990 con 0x7fd158003a00 2023-07-13T22:20:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.777+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1700cb1d0 con 0x7fd158003a00 2023-07-13T22:20:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.778+0000 7fd157fff700 1 -- 172.21.15.60:0/3992161077 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd160019030 con 0x7fd158003a00 2023-07-13T22:20:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.779+0000 7fd157fff700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd15c06ace0 0x7fd15c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.779+0000 7fd157fff700 1 -- 172.21.15.60:0/3992161077 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd16008aa60 con 0x7fd158003a00 2023-07-13T22:20:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.779+0000 7fd16e7fc700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd15c06ace0 0x7fd15c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.780+0000 7fd16e7fc700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd15c06ace0 0x7fd15c06d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1700b3e80 tx=0x7fd164009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:56.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:56.783+0000 7fd157fff700 1 -- 172.21.15.60:0/3992161077 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd160055a80 con 0x7fd158003a00 2023-07-13T22:20:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.040+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd17004e210 con 0x7fd158003a00 2023-07-13T22:20:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.544+0000 7fd157fff700 1 -- 172.21.15.60:0/3992161077 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1600590a0 con 0x7fd158003a00 2023-07-13T22:20:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd15c06ace0 msgr2=0x7fd15c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd15c06ace0 0x7fd15c06d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1700b3e80 tx=0x7fd164009210 comp rx=0 tx=0).stop 2023-07-13T22:20:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 msgr2=0x7fd1700c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd1700c9d00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c890 tx=0x7fd1700ce0b0 comp rx=0 tx=0).stop 2023-07-13T22:20:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 shutdown_connections 2023-07-13T22:20:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd15c06ace0 0x7fd15c06d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd158003a00 0x7fd1700c9d00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1700c85e0 0x7fd1700c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 --2- 172.21.15.60:0/3992161077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1700cb380 0x7fd1701f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 >> 172.21.15.60:0/3992161077 conn(0x7fd17006b2f0 msgr2=0x7fd1700b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:57.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 shutdown_connections 2023-07-13T22:20:57.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.547+0000 7fd1759ff700 1 -- 172.21.15.60:0/3992161077 wait complete. 2023-07-13T22:20:57.551 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:20:57.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:20:57.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.772+0000 7f66c51ff700 1 Processor -- start 2023-07-13T22:20:57.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.772+0000 7f66c51ff700 1 -- start start 2023-07-13T22:20:57.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.773+0000 7f66c51ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66c00cfca0 0x7f66c0167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:57.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.773+0000 7f66c51ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c00cd950 con 0x7f66c00d3750 2023-07-13T22:20:57.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.773+0000 7f66c51ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c00d0ac0 con 0x7f66c00b8400 2023-07-13T22:20:57.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.773+0000 7f66c51ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c00b87d0 con 0x7f66c00cfca0 2023-07-13T22:20:57.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.773+0000 7f66bf59e700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66c00cfca0 0x7f66c0167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:57.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bf59e700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66c00cfca0 0x7f66c0167050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33718/0 (socket says 172.21.15.60:33718) 2023-07-13T22:20:57.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bf59e700 1 -- 172.21.15.60:0/2619151105 learned_addr learned my addr 172.21.15.60:0/2619151105 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:57.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bf59e700 1 -- 172.21.15.60:0/2619151105 >> v1:172.21.15.105:6789/0 conn(0x7f66c00b8400 legacy=0x7f66c00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bf59e700 1 -- 172.21.15.60:0/2619151105 >> v1:172.21.15.60:6789/0 conn(0x7f66c00d3750 legacy=0x7f66c00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bf59e700 1 -- 172.21.15.60:0/2619151105 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00b4eb0 con 0x7f66c00cfca0 2023-07-13T22:20:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bdd9b700 1 -- 172.21.15.60:0/2619151105 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824584611 0 0) 0x7f66b4001110 con 0x7f66c00d3750 2023-07-13T22:20:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bf59e700 1 --2- 172.21.15.60:0/2619151105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00cfca0 0x7f66c0167050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009e50 tx=0x7f66b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f6a74afce91967 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bdd9b700 1 -- 172.21.15.60:0/2619151105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b800e070 con 0x7f66c00cfca0 2023-07-13T22:20:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bdd9b700 1 -- 172.21.15.60:0/2619151105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b800b9c0 con 0x7f66c00cfca0 2023-07-13T22:20:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.774+0000 7f66bdd9b700 1 -- 172.21.15.60:0/2619151105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b800bd70 con 0x7f66c00cfca0 2023-07-13T22:20:57.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.775+0000 7f66c51ff700 1 -- 172.21.15.60:0/2619151105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00cfca0 msgr2=0x7f66c0167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:57.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.775+0000 7f66c51ff700 1 --2- 172.21.15.60:0/2619151105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00cfca0 0x7f66c0167050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009e50 tx=0x7f66b800b330 comp rx=0 tx=0).stop 2023-07-13T22:20:57.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.775+0000 7f66c51ff700 1 -- 172.21.15.60:0/2619151105 shutdown_connections 2023-07-13T22:20:57.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.775+0000 7f66c51ff700 1 --2- 172.21.15.60:0/2619151105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00cfca0 0x7f66c0167050 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:57.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.775+0000 7f66c51ff700 1 -- 172.21.15.60:0/2619151105 >> 172.21.15.60:0/2619151105 conn(0x7f66c006b390 msgr2=0x7f66c006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:57.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.775+0000 7f66c51ff700 1 -- 172.21.15.60:0/2619151105 shutdown_connections 2023-07-13T22:20:57.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.775+0000 7f66c51ff700 1 -- 172.21.15.60:0/2619151105 wait complete. 2023-07-13T22:20:57.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.777+0000 7f66c51ff700 1 Processor -- start 2023-07-13T22:20:57.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.778+0000 7f66c51ff700 1 -- start start 2023-07-13T22:20:57.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.779+0000 7f66c51ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00b8400 0x7f66c00b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:57.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.779+0000 7f66c51ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66c00cfca0 0x7f66c00b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:57.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.779+0000 7f66c51ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c00d3750 0x7f66c015d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:57.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.779+0000 7f66c51ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00c7880 con 0x7f66c00b8400 2023-07-13T22:20:57.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.779+0000 7f66c51ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c015d010 con 0x7f66c00d3750 2023-07-13T22:20:57.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.779+0000 7f66c51ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00c6c70 con 0x7f66c00cfca0 2023-07-13T22:20:57.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00b8400 0x7f66c00b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:57.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00b8400 0x7f66c00b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33728/0 (socket says 172.21.15.60:33728) 2023-07-13T22:20:57.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bed9d700 1 -- 172.21.15.60:0/2752569489 learned_addr learned my addr 172.21.15.60:0/2752569489 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:57.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66be59c700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66c00cfca0 0x7f66c00b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:57.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bf59e700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c00d3750 0x7f66c015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:57.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bed9d700 1 -- 172.21.15.60:0/2752569489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66c00cfca0 msgr2=0x7f66c00b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:57.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bed9d700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66c00cfca0 0x7f66c00b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:57.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bed9d700 1 -- 172.21.15.60:0/2752569489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c00d3750 msgr2=0x7f66c015d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:57.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bed9d700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c00d3750 0x7f66c015d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:57.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bed9d700 1 -- 172.21.15.60:0/2752569489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b8009b30 con 0x7f66c00b8400 2023-07-13T22:20:57.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.780+0000 7f66bed9d700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00b8400 0x7f66c00b6a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c480 tx=0x7f66b000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:57.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.781+0000 7f66a77fe700 1 -- 172.21.15.60:0/2752569489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b00052d0 con 0x7f66c00b8400 2023-07-13T22:20:57.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.781+0000 7f66a77fe700 1 -- 172.21.15.60:0/2752569489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b0005430 con 0x7f66c00b8400 2023-07-13T22:20:57.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.781+0000 7f66a77fe700 1 -- 172.21.15.60:0/2752569489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b00038c0 con 0x7f66c00b8400 2023-07-13T22:20:57.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.781+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c00c6f50 con 0x7f66c00b8400 2023-07-13T22:20:57.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.781+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c00c7510 con 0x7f66c00b8400 2023-07-13T22:20:57.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.782+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c00008d0 con 0x7f66c00b8400 2023-07-13T22:20:57.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.783+0000 7f66a77fe700 1 -- 172.21.15.60:0/2752569489 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66b0003a20 con 0x7f66c00b8400 2023-07-13T22:20:57.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.784+0000 7f66a77fe700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66ac06ace0 0x7f66ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:57.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.784+0000 7f66a77fe700 1 -- 172.21.15.60:0/2752569489 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f66b0088850 con 0x7f66c00b8400 2023-07-13T22:20:57.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.784+0000 7f66be59c700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66ac06ace0 0x7f66ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:57.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.784+0000 7f66be59c700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66ac06ace0 0x7f66ac06d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66c00cd160 tx=0x7f66b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:57.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:57.788+0000 7f66a77fe700 1 -- 172.21.15.60:0/2752569489 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f66b0053860 con 0x7f66c00b8400 2023-07-13T22:20:58.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.041+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66c006a2d0 con 0x7f66c00b8400 2023-07-13T22:20:58.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.042+0000 7f66a77fe700 1 -- 172.21.15.60:0/2752569489 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f66b0056e80 con 0x7f66c00b8400 2023-07-13T22:20:58.026 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:58.026 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2550952354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2550952354},{"type":"v1","addr":"172.21.15.60:6837","nonce":2550952354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4134045869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4134045869},{"type":"v1","addr":"172.21.15.60:6836","nonce":4134045869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2202330082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2202330082},{"type":"v1","addr":"172.21.15.105:6834","nonce":2202330082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:20:54.038443+0000","modified":"2023-07-13T22:20:57.536915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/453882022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":453882022},{"type":"v1","addr":"172.21.15.105:6835","nonce":453882022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:20:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66ac06ace0 msgr2=0x7f66ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66ac06ace0 0x7f66ac06d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66c00cd160 tx=0x7f66b4009040 comp rx=0 tx=0).stop 2023-07-13T22:20:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00b8400 msgr2=0x7f66c00b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00b8400 0x7f66c00b6a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c480 tx=0x7f66b000c790 comp rx=0 tx=0).stop 2023-07-13T22:20:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 shutdown_connections 2023-07-13T22:20:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66ac06ace0 0x7f66ac06d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c00b8400 0x7f66c00b6a00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c00d3750 0x7f66c015d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 --2- 172.21.15.60:0/2752569489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66c00cfca0 0x7f66c00b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 >> 172.21.15.60:0/2752569489 conn(0x7f66c006b390 msgr2=0x7f66c015a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 shutdown_connections 2023-07-13T22:20:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.044+0000 7f66c51ff700 1 -- 172.21.15.60:0/2752569489 wait complete. 2023-07-13T22:20:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2023-07-13T22:20:58.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:20:58.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.264+0000 7f085e9ff700 1 Processor -- start 2023-07-13T22:20:58.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.264+0000 7f085e9ff700 1 -- start start 2023-07-13T22:20:58.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.265+0000 7f085e9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08580b8400 0x7f08580cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.265+0000 7f085e9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f08580cfca0 0x7f08580d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.265+0000 7f085e9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08580d0ac0 con 0x7f08580d3750 2023-07-13T22:20:58.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.265+0000 7f085e9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08580b87d0 con 0x7f08580b8400 2023-07-13T22:20:58.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f085e9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08580b4eb0 con 0x7f08580cfca0 2023-07-13T22:20:58.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f0857fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08580b8400 0x7f08580cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f0857fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08580b8400 0x7f08580cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33734/0 (socket says 172.21.15.60:33734) 2023-07-13T22:20:58.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f08577fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f08580d3750 0x7f08580cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39514/0 (socket says 172.21.15.60:39514) 2023-07-13T22:20:58.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f0857fff700 1 -- 172.21.15.60:0/3687175622 learned_addr learned my addr 172.21.15.60:0/3687175622 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:58.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f085cf9c700 1 --2- 172.21.15.60:0/3687175622 >> v2:172.21.15.105:3300/0 conn(0x7f08580cfca0 0x7f08580d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f0857fff700 1 -- 172.21.15.60:0/3687175622 >> v2:172.21.15.105:3300/0 conn(0x7f08580cfca0 msgr2=0x7f08580d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:58.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f0857fff700 1 --2- 172.21.15.60:0/3687175622 >> v2:172.21.15.105:3300/0 conn(0x7f08580cfca0 0x7f08580d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f0857fff700 1 -- 172.21.15.60:0/3687175622 >> v1:172.21.15.60:6789/0 conn(0x7f08580d3750 legacy=0x7f08580cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f0857fff700 1 -- 172.21.15.60:0/3687175622 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580b5190 con 0x7f08580b8400 2023-07-13T22:20:58.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f0856ffd700 1 -- 172.21.15.60:0/3687175622 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164172299 0 0) 0x7f08580d0ac0 con 0x7f08580d3750 2023-07-13T22:20:58.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.266+0000 7f0857fff700 1 --2- 172.21.15.60:0/3687175622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 0x7f08580cd240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f08580d0070 tx=0x7f0848009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967c73952f20cc6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:58.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f0856ffd700 1 -- 172.21.15.60:0/3687175622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848011830 con 0x7f08580b8400 2023-07-13T22:20:58.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f0856ffd700 1 -- 172.21.15.60:0/3687175622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0848011990 con 0x7f08580b8400 2023-07-13T22:20:58.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f0856ffd700 1 -- 172.21.15.60:0/3687175622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848011d20 con 0x7f08580b8400 2023-07-13T22:20:58.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f085e9ff700 1 -- 172.21.15.60:0/3687175622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 msgr2=0x7f08580cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f085e9ff700 1 --2- 172.21.15.60:0/3687175622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 0x7f08580cd240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f08580d0070 tx=0x7f0848009bd0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f085e9ff700 1 -- 172.21.15.60:0/3687175622 shutdown_connections 2023-07-13T22:20:58.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f085e9ff700 1 --2- 172.21.15.60:0/3687175622 >> v2:172.21.15.105:3300/0 conn(0x7f08580cfca0 0x7f08580d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f085e9ff700 1 --2- 172.21.15.60:0/3687175622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 0x7f08580cd240 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f085e9ff700 1 -- 172.21.15.60:0/3687175622 >> 172.21.15.60:0/3687175622 conn(0x7f085806b390 msgr2=0x7f085806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:58.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f085e9ff700 1 -- 172.21.15.60:0/3687175622 shutdown_connections 2023-07-13T22:20:58.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.267+0000 7f085e9ff700 1 -- 172.21.15.60:0/3687175622 wait complete. 2023-07-13T22:20:58.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.269+0000 7f085e9ff700 1 Processor -- start 2023-07-13T22:20:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.269+0000 7f085e9ff700 1 -- start start 2023-07-13T22:20:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.270+0000 7f085e9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 0x7f0858153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.270+0000 7f085e9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08580cfca0 0x7f085815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.270+0000 7f085e9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08580d3750 0x7f08580b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.270+0000 7f085e9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085815ca70 con 0x7f08580b8400 2023-07-13T22:20:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.270+0000 7f085e9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085815cd50 con 0x7f08580d3750 2023-07-13T22:20:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.270+0000 7f085e9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085815d030 con 0x7f08580cfca0 2023-07-13T22:20:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.270+0000 7f0857fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 0x7f0858153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.270+0000 7f0857fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 0x7f0858153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33746/0 (socket says 172.21.15.60:33746) 2023-07-13T22:20:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.270+0000 7f0857fff700 1 -- 172.21.15.60:0/2877137176 learned_addr learned my addr 172.21.15.60:0/2877137176 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f085cf9c700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08580d3750 0x7f08580b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f08577fe700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08580cfca0 0x7f085815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f0857fff700 1 -- 172.21.15.60:0/2877137176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08580cfca0 msgr2=0x7f085815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f0857fff700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08580cfca0 0x7f085815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f0857fff700 1 -- 172.21.15.60:0/2877137176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08580d3750 msgr2=0x7f08580b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f0857fff700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08580d3750 0x7f08580b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f0857fff700 1 -- 172.21.15.60:0/2877137176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0848009550 con 0x7f08580b8400 2023-07-13T22:20:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f085cf9c700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08580d3750 0x7f08580b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:20:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f0857fff700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 0x7f0858153670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0848009c10 tx=0x7f0848004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f0854ff9700 1 -- 172.21.15.60:0/2877137176 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08480036a0 con 0x7f08580b8400 2023-07-13T22:20:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f0854ff9700 1 -- 172.21.15.60:0/2877137176 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084801e070 con 0x7f08580b8400 2023-07-13T22:20:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085815d310 con 0x7f08580b8400 2023-07-13T22:20:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.271+0000 7f0854ff9700 1 -- 172.21.15.60:0/2877137176 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848003a20 con 0x7f08580b8400 2023-07-13T22:20:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.272+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085815e050 con 0x7f08580b8400 2023-07-13T22:20:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.272+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085815d830 con 0x7f08580b8400 2023-07-13T22:20:58.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.273+0000 7f0854ff9700 1 -- 172.21.15.60:0/2877137176 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0848003b80 con 0x7f08580b8400 2023-07-13T22:20:58.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.274+0000 7f0854ff9700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f084406ace0 0x7f084406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.274+0000 7f0854ff9700 1 -- 172.21.15.60:0/2877137176 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f084808a540 con 0x7f08580b8400 2023-07-13T22:20:58.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.275+0000 7f08577fe700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f084406ace0 0x7f084406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.275+0000 7f08577fe700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f084406ace0 0x7f084406d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08580b6ac0 tx=0x7f084c007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:58.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.278+0000 7f0854ff9700 1 -- 172.21.15.60:0/2877137176 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f085815d830 con 0x7f08580b8400 2023-07-13T22:20:58.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.487+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f085815d830 con 0x7f08580b8400 2023-07-13T22:20:58.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.488+0000 7f0854ff9700 1 -- 172.21.15.60:0/2877137176 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f085815d830 con 0x7f08580b8400 2023-07-13T22:20:58.472 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:58.473 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:20:58.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.490+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f084406ace0 msgr2=0x7f084406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.490+0000 7f085e9ff700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f084406ace0 0x7f084406d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08580b6ac0 tx=0x7f084c007be0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.490+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 msgr2=0x7f0858153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.490+0000 7f085e9ff700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 0x7f0858153670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0848009c10 tx=0x7f0848004750 comp rx=0 tx=0).stop 2023-07-13T22:20:58.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.491+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 shutdown_connections 2023-07-13T22:20:58.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.491+0000 7f085e9ff700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f084406ace0 0x7f084406d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.491+0000 7f085e9ff700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08580b8400 0x7f0858153670 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.491+0000 7f085e9ff700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08580d3750 0x7f08580b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.491+0000 7f085e9ff700 1 --2- 172.21.15.60:0/2877137176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08580cfca0 0x7f085815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.491+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 >> 172.21.15.60:0/2877137176 conn(0x7f085806b390 msgr2=0x7f08580b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.491+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 shutdown_connections 2023-07-13T22:20:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.491+0000 7f085e9ff700 1 -- 172.21.15.60:0/2877137176 wait complete. 2023-07-13T22:20:58.493 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:20:58.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:20:58.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.712+0000 7f0cd69ff700 1 Processor -- start 2023-07-13T22:20:58.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.712+0000 7f0cd69ff700 1 -- start start 2023-07-13T22:20:58.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.713+0000 7f0cd69ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0cd00b8400 0x7f0cd00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.713+0000 7f0cd69ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cd00cfca0 0x7f0cd00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.713+0000 7f0cd69ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd00d0ac0 con 0x7f0cd00d3750 2023-07-13T22:20:58.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.713+0000 7f0cd69ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd00b87d0 con 0x7f0cd00b8400 2023-07-13T22:20:58.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.713+0000 7f0cd69ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd00b4eb0 con 0x7f0cd00cfca0 2023-07-13T22:20:58.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.713+0000 7f0ccffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0cd00b8400 0x7f0cd00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.713+0000 7f0ccf7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0cd00d3750 0x7f0cd00cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39526/0 (socket says 172.21.15.60:39526) 2023-07-13T22:20:58.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.713+0000 7f0ccffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0cd00b8400 0x7f0cd00cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33752/0 (socket says 172.21.15.60:33752) 2023-07-13T22:20:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.713+0000 7f0ccf7fe700 1 -- 172.21.15.60:0/1996151731 learned_addr learned my addr 172.21.15.60:0/1996151731 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.714+0000 7f0ccffff700 1 -- 172.21.15.60:0/1996151731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cd00cfca0 msgr2=0x7f0cd00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.714+0000 7f0ccffff700 1 --2- 172.21.15.60:0/1996151731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cd00cfca0 0x7f0cd00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.714+0000 7f0ccffff700 1 -- 172.21.15.60:0/1996151731 >> v1:172.21.15.60:6789/0 conn(0x7f0cd00d3750 legacy=0x7f0cd00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.714+0000 7f0ccffff700 1 -- 172.21.15.60:0/1996151731 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd00b5190 con 0x7f0cd00b8400 2023-07-13T22:20:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.714+0000 7f0ccffff700 1 --2- 172.21.15.60:0/1996151731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 0x7f0cd00cd240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009840 tx=0x7f0cc0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989e8ee2c0f5deb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.714+0000 7f0cceffd700 1 -- 172.21.15.60:0/1996151731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0011830 con 0x7f0cd00b8400 2023-07-13T22:20:58.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.714+0000 7f0cceffd700 1 -- 172.21.15.60:0/1996151731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc0011990 con 0x7f0cd00b8400 2023-07-13T22:20:58.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.714+0000 7f0cceffd700 1 -- 172.21.15.60:0/1996151731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0011d20 con 0x7f0cd00b8400 2023-07-13T22:20:58.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.715+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1996151731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 msgr2=0x7f0cd00cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.715+0000 7f0cd69ff700 1 --2- 172.21.15.60:0/1996151731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 0x7f0cd00cd240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009840 tx=0x7f0cc0009b50 comp rx=0 tx=0).stop 2023-07-13T22:20:58.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.715+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1996151731 shutdown_connections 2023-07-13T22:20:58.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.715+0000 7f0cd69ff700 1 --2- 172.21.15.60:0/1996151731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cd00cfca0 0x7f0cd00d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.715+0000 7f0cd69ff700 1 --2- 172.21.15.60:0/1996151731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 0x7f0cd00cd240 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.715+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1996151731 >> 172.21.15.60:0/1996151731 conn(0x7f0cd006b390 msgr2=0x7f0cd006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:58.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.715+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1996151731 shutdown_connections 2023-07-13T22:20:58.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.715+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1996151731 wait complete. 2023-07-13T22:20:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.717+0000 7f0cd69ff700 1 Processor -- start 2023-07-13T22:20:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.717+0000 7f0cd69ff700 1 -- start start 2023-07-13T22:20:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0cd69ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 0x7f0cd01566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0cd69ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cd00cfca0 0x7f0cd0151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0cd69ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cd00d3750 0x7f0cd0159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0cd69ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01f57b0 con 0x7f0cd00b8400 2023-07-13T22:20:58.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0cd69ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01f5a50 con 0x7f0cd00cfca0 2023-07-13T22:20:58.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0cd69ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01f5d30 con 0x7f0cd00d3750 2023-07-13T22:20:58.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0ccffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 0x7f0cd01566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0ccffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 0x7f0cd01566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33766/0 (socket says 172.21.15.60:33766) 2023-07-13T22:20:58.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0ccffff700 1 -- 172.21.15.60:0/1257592915 learned_addr learned my addr 172.21.15.60:0/1257592915 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:58.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0ccf7fe700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cd00cfca0 0x7f0cd0151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0cd4f9c700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cd00d3750 0x7f0cd0159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0ccffff700 1 -- 172.21.15.60:0/1257592915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cd00d3750 msgr2=0x7f0cd0159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0ccffff700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cd00d3750 0x7f0cd0159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0ccffff700 1 -- 172.21.15.60:0/1257592915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cd00cfca0 msgr2=0x7f0cd0151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0ccffff700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cd00cfca0 0x7f0cd0151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.718+0000 7f0ccffff700 1 -- 172.21.15.60:0/1257592915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc00094a0 con 0x7f0cd00b8400 2023-07-13T22:20:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.719+0000 7f0ccffff700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 0x7f0cd01566a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00090b0 tx=0x7f0cc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.719+0000 7f0cccff9700 1 -- 172.21.15.60:0/1257592915 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00045f0 con 0x7f0cd00b8400 2023-07-13T22:20:58.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.719+0000 7f0cccff9700 1 -- 172.21.15.60:0/1257592915 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc001e070 con 0x7f0cd00b8400 2023-07-13T22:20:58.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.719+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd01f6010 con 0x7f0cd00b8400 2023-07-13T22:20:58.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.719+0000 7f0cccff9700 1 -- 172.21.15.60:0/1257592915 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0003b00 con 0x7f0cd00b8400 2023-07-13T22:20:58.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.719+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd01515d0 con 0x7f0cd00b8400 2023-07-13T22:20:58.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.720+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd00cf8e0 con 0x7f0cd00b8400 2023-07-13T22:20:58.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.721+0000 7f0cccff9700 1 -- 172.21.15.60:0/1257592915 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0cc0003c60 con 0x7f0cd00b8400 2023-07-13T22:20:58.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.722+0000 7f0cccff9700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0cbc06ace0 0x7f0cbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:58.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.722+0000 7f0cccff9700 1 -- 172.21.15.60:0/1257592915 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0cc008a680 con 0x7f0cd00b8400 2023-07-13T22:20:58.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.723+0000 7f0ccf7fe700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0cbc06ace0 0x7f0cbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:58.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.723+0000 7f0ccf7fe700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0cbc06ace0 0x7f0cbc06d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00d27a0 tx=0x7f0cc4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:58.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.727+0000 7f0cccff9700 1 -- 172.21.15.60:0/1257592915 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0cc0055720 con 0x7f0cd00b8400 2023-07-13T22:20:58.906 INFO:tasks.ceph.mds.d.smithi105.stderr:2023-07-13T22:20:58.921+0000 7f3ffb9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:20:58.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.928+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0cd01f5bc0 con 0x7f0cbc06ace0 2023-07-13T22:20:58.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.939+0000 7f0cccff9700 1 -- 172.21.15.60:0/1257592915 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287021 (secure 0 0 0) 0x7f0cd01f5bc0 con 0x7f0cbc06ace0 2023-07-13T22:20:58.926 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:58.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0cbc06ace0 msgr2=0x7f0cbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0cbc06ace0 0x7f0cbc06d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00d27a0 tx=0x7f0cc4009040 comp rx=0 tx=0).stop 2023-07-13T22:20:58.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 msgr2=0x7f0cd01566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:58.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 0x7f0cd01566a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00090b0 tx=0x7f0cc0004750 comp rx=0 tx=0).stop 2023-07-13T22:20:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 shutdown_connections 2023-07-13T22:20:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0cbc06ace0 0x7f0cbc06d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd00b8400 0x7f0cd01566a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cd00d3750 0x7f0cd0159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 --2- 172.21.15.60:0/1257592915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cd00cfca0 0x7f0cd0151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 >> 172.21.15.60:0/1257592915 conn(0x7f0cd006b390 msgr2=0x7f0cd00d1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.943+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 shutdown_connections 2023-07-13T22:20:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:58.944+0000 7f0cd69ff700 1 -- 172.21.15.60:0/1257592915 wait complete. 2023-07-13T22:20:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:20:58.946 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-07-13T22:20:57.117979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218480,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756240,"statfs":{"total":773094113280,"available":772870389760,"internally_reserved":0,"allocated":2932736,"data_stored":1792556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.609206"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611623+0000","last_change":"2023-07-13T22:20:54.539241+0000","last_active":"2023-07-13T22:20:55.611623+0000","last_peered":"2023-07-13T22:20:55.611623+0000","last_clean":"2023-07-13T22:20:55.611623+0000","last_became_active":"2023-07-13T22:20:54.538987+0000","last_became_peered":"2023-07-13T22:20:54.538987+0000","last_unstale":"2023-07-13T22:20:55.611623+0000","last_undegraded":"2023-07-13T22:20:55.611623+0000","last_fullsized":"2023-07-13T22:20:55.611623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:34:22.186098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532509+0000","last_change":"2023-07-13T22:20:54.537349+0000","last_active":"2023-07-13T22:20:55.532509+0000","last_peered":"2023-07-13T22:20:55.532509+0000","last_clean":"2023-07-13T22:20:55.532509+0000","last_became_active":"2023-07-13T22:20:54.536260+0000","last_became_peered":"2023-07-13T22:20:54.536260+0000","last_unstale":"2023-07-13T22:20:55.532509+0000","last_undegraded":"2023-07-13T22:20:55.532509+0000","last_fullsized":"2023-07-13T22:20:55.532509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532790+0000","last_change":"2023-07-13T22:20:54.543864+0000","last_active":"2023-07-13T22:20:55.532790+0000","last_peered":"2023-07-13T22:20:55.532790+0000","last_clean":"2023-07-13T22:20:55.532790+0000","last_became_active":"2023-07-13T22:20:54.543344+0000","last_became_peered":"2023-07-13T22:20:54.543344+0000","last_unstale":"2023-07-13T22:20:55.532790+0000","last_undegraded":"2023-07-13T22:20:55.532790+0000","last_fullsized":"2023-07-13T22:20:55.532790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610982+0000","last_change":"2023-07-13T22:20:54.540616+0000","last_active":"2023-07-13T22:20:55.610982+0000","last_peered":"2023-07-13T22:20:55.610982+0000","last_clean":"2023-07-13T22:20:55.610982+0000","last_became_active":"2023-07-13T22:20:54.540326+0000","last_became_peered":"2023-07-13T22:20:54.540326+0000","last_unstale":"2023-07-13T22:20:55.610982+0000","last_undegraded":"2023-07-13T22:20:55.610982+0000","last_fullsized":"2023-07-13T22:20:55.610982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611292+0000","last_change":"2023-07-13T22:20:54.550022+0000","last_active":"2023-07-13T22:20:55.611292+0000","last_peered":"2023-07-13T22:20:55.611292+0000","last_clean":"2023-07-13T22:20:55.611292+0000","last_became_active":"2023-07-13T22:20:54.549249+0000","last_became_peered":"2023-07-13T22:20:54.549249+0000","last_unstale":"2023-07-13T22:20:55.611292+0000","last_undegraded":"2023-07-13T22:20:55.611292+0000","last_fullsized":"2023-07-13T22:20:55.611292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533320+0000","last_change":"2023-07-13T22:20:54.544989+0000","last_active":"2023-07-13T22:20:55.533320+0000","last_peered":"2023-07-13T22:20:55.533320+0000","last_clean":"2023-07-13T22:20:55.533320+0000","last_became_active":"2023-07-13T22:20:54.544719+0000","last_became_peered":"2023-07-13T22:20:54.544719+0000","last_unstale":"2023-07-13T22:20:55.533320+0000","last_undegraded":"2023-07-13T22:20:55.533320+0000","last_fullsized":"2023-07-13T22:20:55.533320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:55.546960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610444+0000","last_change":"2023-07-13T22:20:54.543266+0000","last_active":"2023-07-13T22:20:55.610444+0000","last_peered":"2023-07-13T22:20:55.610444+0000","last_clean":"2023-07-13T22:20:55.610444+0000","last_became_active":"2023-07-13T22:20:54.542870+0000","last_became_peered":"2023-07-13T22:20:54.542870+0000","last_unstale":"2023-07-13T22:20:55.610444+0000","last_undegraded":"2023-07-13T22:20:55.610444+0000","last_fullsized":"2023-07-13T22:20:55.610444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532392+0000","last_change":"2023-07-13T22:20:54.546394+0000","last_active":"2023-07-13T22:20:55.532392+0000","last_peered":"2023-07-13T22:20:55.532392+0000","last_clean":"2023-07-13T22:20:55.532392+0000","last_became_active":"2023-07-13T22:20:54.545682+0000","last_became_peered":"2023-07-13T22:20:54.545682+0000","last_unstale":"2023-07-13T22:20:55.532392+0000","last_undegraded":"2023-07-13T22:20:55.532392+0000","last_fullsized":"2023-07-13T22:20:55.532392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611695+0000","last_change":"2023-07-13T22:20:54.549925+0000","last_active":"2023-07-13T22:20:55.611695+0000","last_peered":"2023-07-13T22:20:55.611695+0000","last_clean":"2023-07-13T22:20:55.611695+0000","last_became_active":"2023-07-13T22:20:54.549641+0000","last_became_peered":"2023-07-13T22:20:54.549641+0000","last_unstale":"2023-07-13T22:20:55.611695+0000","last_undegraded":"2023-07-13T22:20:55.611695+0000","last_fullsized":"2023-07-13T22:20:55.611695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:45.329435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532781+0000","last_change":"2023-07-13T22:20:54.543479+0000","last_active":"2023-07-13T22:20:55.532781+0000","last_peered":"2023-07-13T22:20:55.532781+0000","last_clean":"2023-07-13T22:20:55.532781+0000","last_became_active":"2023-07-13T22:20:54.543143+0000","last_became_peered":"2023-07-13T22:20:54.543143+0000","last_unstale":"2023-07-13T22:20:55.532781+0000","last_undegraded":"2023-07-13T22:20:55.532781+0000","last_fullsized":"2023-07-13T22:20:55.532781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532989+0000","last_change":"2023-07-13T22:20:54.543686+0000","last_active":"2023-07-13T22:20:55.532989+0000","last_peered":"2023-07-13T22:20:55.532989+0000","last_clean":"2023-07-13T22:20:55.532989+0000","last_became_active":"2023-07-13T22:20:54.543001+0000","last_became_peered":"2023-07-13T22:20:54.543001+0000","last_unstale":"2023-07-13T22:20:55.532989+0000","last_undegraded":"2023-07-13T22:20:55.532989+0000","last_fullsized":"2023-07-13T22:20:55.532989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611996+0000","last_change":"2023-07-13T22:20:54.547637+0000","last_active":"2023-07-13T22:20:55.611996+0000","last_peered":"2023-07-13T22:20:55.611996+0000","last_clean":"2023-07-13T22:20:55.611996+0000","last_became_active":"2023-07-13T22:20:54.547280+0000","last_became_peered":"2023-07-13T22:20:54.547280+0000","last_unstale":"2023-07-13T22:20:55.611996+0000","last_undegraded":"2023-07-13T22:20:55.611996+0000","last_fullsized":"2023-07-13T22:20:55.611996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:18:24.767625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611891+0000","last_change":"2023-07-13T22:20:54.538682+0000","last_active":"2023-07-13T22:20:55.611891+0000","last_peered":"2023-07-13T22:20:55.611891+0000","last_clean":"2023-07-13T22:20:55.611891+0000","last_became_active":"2023-07-13T22:20:54.538356+0000","last_became_peered":"2023-07-13T22:20:54.538356+0000","last_unstale":"2023-07-13T22:20:55.611891+0000","last_undegraded":"2023-07-13T22:20:55.611891+0000","last_fullsized":"2023-07-13T22:20:55.611891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:51:10.800579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533030+0000","last_change":"2023-07-13T22:20:54.540925+0000","last_active":"2023-07-13T22:20:55.533030+0000","last_peered":"2023-07-13T22:20:55.533030+0000","last_clean":"2023-07-13T22:20:55.533030+0000","last_became_active":"2023-07-13T22:20:54.540646+0000","last_became_peered":"2023-07-13T22:20:54.540646+0000","last_unstale":"2023-07-13T22:20:55.533030+0000","last_undegraded":"2023-07-13T22:20:55.533030+0000","last_fullsized":"2023-07-13T22:20:55.533030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610864+0000","last_change":"2023-07-13T22:20:54.543844+0000","last_active":"2023-07-13T22:20:55.610864+0000","last_peered":"2023-07-13T22:20:55.610864+0000","last_clean":"2023-07-13T22:20:55.610864+0000","last_became_active":"2023-07-13T22:20:54.543527+0000","last_became_peered":"2023-07-13T22:20:54.543527+0000","last_unstale":"2023-07-13T22:20:55.610864+0000","last_undegraded":"2023-07-13T22:20:55.610864+0000","last_fullsized":"2023-07-13T22:20:55.610864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533022+0000","last_change":"2023-07-13T22:20:54.537914+0000","last_active":"2023-07-13T22:20:55.533022+0000","last_peered":"2023-07-13T22:20:55.533022+0000","last_clean":"2023-07-13T22:20:55.533022+0000","last_became_active":"2023-07-13T22:20:54.537562+0000","last_became_peered":"2023-07-13T22:20:54.537562+0000","last_unstale":"2023-07-13T22:20:55.533022+0000","last_undegraded":"2023-07-13T22:20:55.533022+0000","last_fullsized":"2023-07-13T22:20:55.533022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:26.867364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611835+0000","last_change":"2023-07-13T22:20:54.549793+0000","last_active":"2023-07-13T22:20:55.611835+0000","last_peered":"2023-07-13T22:20:55.611835+0000","last_clean":"2023-07-13T22:20:55.611835+0000","last_became_active":"2023-07-13T22:20:54.549473+0000","last_became_peered":"2023-07-13T22:20:54.549473+0000","last_unstale":"2023-07-13T22:20:55.611835+0000","last_undegraded":"2023-07-13T22:20:55.611835+0000","last_fullsized":"2023-07-13T22:20:55.611835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535307+0000","last_change":"2023-07-13T22:20:54.541942+0000","last_active":"2023-07-13T22:20:55.535307+0000","last_peered":"2023-07-13T22:20:55.535307+0000","last_clean":"2023-07-13T22:20:55.535307+0000","last_became_active":"2023-07-13T22:20:54.541314+0000","last_became_peered":"2023-07-13T22:20:54.541314+0000","last_unstale":"2023-07-13T22:20:55.535307+0000","last_undegraded":"2023-07-13T22:20:55.535307+0000","last_fullsized":"2023-07-13T22:20:55.535307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:10:09.528351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611873+0000","last_change":"2023-07-13T22:20:54.532924+0000","last_active":"2023-07-13T22:20:55.611873+0000","last_peered":"2023-07-13T22:20:55.611873+0000","last_clean":"2023-07-13T22:20:55.611873+0000","last_became_active":"2023-07-13T22:20:54.531937+0000","last_became_peered":"2023-07-13T22:20:54.531937+0000","last_unstale":"2023-07-13T22:20:55.611873+0000","last_undegraded":"2023-07-13T22:20:55.611873+0000","last_fullsized":"2023-07-13T22:20:55.611873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:20:04.410391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533904+0000","last_change":"2023-07-13T22:20:54.552416+0000","last_active":"2023-07-13T22:20:55.533904+0000","last_peered":"2023-07-13T22:20:55.533904+0000","last_clean":"2023-07-13T22:20:55.533904+0000","last_became_active":"2023-07-13T22:20:54.551516+0000","last_became_peered":"2023-07-13T22:20:54.551516+0000","last_unstale":"2023-07-13T22:20:55.533904+0000","last_undegraded":"2023-07-13T22:20:55.533904+0000","last_fullsized":"2023-07-13T22:20:55.533904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611325+0000","last_change":"2023-07-13T22:20:54.551898+0000","last_active":"2023-07-13T22:20:55.611325+0000","last_peered":"2023-07-13T22:20:55.611325+0000","last_clean":"2023-07-13T22:20:55.611325+0000","last_became_active":"2023-07-13T22:20:54.551538+0000","last_became_peered":"2023-07-13T22:20:54.551538+0000","last_unstale":"2023-07-13T22:20:55.611325+0000","last_undegraded":"2023-07-13T22:20:55.611325+0000","last_fullsized":"2023-07-13T22:20:55.611325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:38.898796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610399+0000","last_change":"2023-07-13T22:20:54.540580+0000","last_active":"2023-07-13T22:20:55.610399+0000","last_peered":"2023-07-13T22:20:55.610399+0000","last_clean":"2023-07-13T22:20:55.610399+0000","last_became_active":"2023-07-13T22:20:54.540289+0000","last_became_peered":"2023-07-13T22:20:54.540289+0000","last_unstale":"2023-07-13T22:20:55.610399+0000","last_undegraded":"2023-07-13T22:20:55.610399+0000","last_fullsized":"2023-07-13T22:20:55.610399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:43.402535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610169+0000","last_change":"2023-07-13T22:20:54.536952+0000","last_active":"2023-07-13T22:20:55.610169+0000","last_peered":"2023-07-13T22:20:55.610169+0000","last_clean":"2023-07-13T22:20:55.610169+0000","last_became_active":"2023-07-13T22:20:54.536696+0000","last_became_peered":"2023-07-13T22:20:54.536696+0000","last_unstale":"2023-07-13T22:20:55.610169+0000","last_undegraded":"2023-07-13T22:20:55.610169+0000","last_fullsized":"2023-07-13T22:20:55.610169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:26.867364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611257+0000","last_change":"2023-07-13T22:20:54.550043+0000","last_active":"2023-07-13T22:20:55.611257+0000","last_peered":"2023-07-13T22:20:55.611257+0000","last_clean":"2023-07-13T22:20:55.611257+0000","last_became_active":"2023-07-13T22:20:54.549754+0000","last_became_peered":"2023-07-13T22:20:54.549754+0000","last_unstale":"2023-07-13T22:20:55.611257+0000","last_undegraded":"2023-07-13T22:20:55.611257+0000","last_fullsized":"2023-07-13T22:20:55.611257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531264+0000","last_change":"2023-07-13T22:20:54.534540+0000","last_active":"2023-07-13T22:20:55.531264+0000","last_peered":"2023-07-13T22:20:55.531264+0000","last_clean":"2023-07-13T22:20:55.531264+0000","last_became_active":"2023-07-13T22:20:54.534251+0000","last_became_peered":"2023-07-13T22:20:54.534251+0000","last_unstale":"2023-07-13T22:20:55.531264+0000","last_undegraded":"2023-07-13T22:20:55.531264+0000","last_fullsized":"2023-07-13T22:20:55.531264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531735+0000","last_change":"2023-07-13T22:20:54.529221+0000","last_active":"2023-07-13T22:20:55.531735+0000","last_peered":"2023-07-13T22:20:55.531735+0000","last_clean":"2023-07-13T22:20:55.531735+0000","last_became_active":"2023-07-13T22:20:54.528934+0000","last_became_peered":"2023-07-13T22:20:54.528934+0000","last_unstale":"2023-07-13T22:20:55.531735+0000","last_undegraded":"2023-07-13T22:20:55.531735+0000","last_fullsized":"2023-07-13T22:20:55.531735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533014+0000","last_change":"2023-07-13T22:20:54.539287+0000","last_active":"2023-07-13T22:20:55.533014+0000","last_peered":"2023-07-13T22:20:55.533014+0000","last_clean":"2023-07-13T22:20:55.533014+0000","last_became_active":"2023-07-13T22:20:54.537991+0000","last_became_peered":"2023-07-13T22:20:54.537991+0000","last_unstale":"2023-07-13T22:20:55.533014+0000","last_undegraded":"2023-07-13T22:20:55.533014+0000","last_fullsized":"2023-07-13T22:20:55.533014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533580+0000","last_change":"2023-07-13T22:20:53.534188+0000","last_active":"2023-07-13T22:20:55.533580+0000","last_peered":"2023-07-13T22:20:55.533580+0000","last_clean":"2023-07-13T22:20:55.533580+0000","last_became_active":"2023-07-13T22:20:53.533888+0000","last_became_peered":"2023-07-13T22:20:53.533888+0000","last_unstale":"2023-07-13T22:20:55.533580+0000","last_undegraded":"2023-07-13T22:20:55.533580+0000","last_fullsized":"2023-07-13T22:20:55.533580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:55.228110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610249+0000","last_change":"2023-07-13T22:20:54.541751+0000","last_active":"2023-07-13T22:20:55.610249+0000","last_peered":"2023-07-13T22:20:55.610249+0000","last_clean":"2023-07-13T22:20:55.610249+0000","last_became_active":"2023-07-13T22:20:54.541506+0000","last_became_peered":"2023-07-13T22:20:54.541506+0000","last_unstale":"2023-07-13T22:20:55.610249+0000","last_undegraded":"2023-07-13T22:20:55.610249+0000","last_fullsized":"2023-07-13T22:20:55.610249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:55.546960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531119+0000","last_change":"2023-07-13T22:20:53.531325+0000","last_active":"2023-07-13T22:20:55.531119+0000","last_peered":"2023-07-13T22:20:55.531119+0000","last_clean":"2023-07-13T22:20:55.531119+0000","last_became_active":"2023-07-13T22:20:53.530929+0000","last_became_peered":"2023-07-13T22:20:53.530929+0000","last_unstale":"2023-07-13T22:20:55.531119+0000","last_undegraded":"2023-07-13T22:20:55.531119+0000","last_fullsized":"2023-07-13T22:20:55.531119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:07:16.260303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611424+0000","last_change":"2023-07-13T22:20:54.541835+0000","last_active":"2023-07-13T22:20:55.611424+0000","last_peered":"2023-07-13T22:20:55.611424+0000","last_clean":"2023-07-13T22:20:55.611424+0000","last_became_active":"2023-07-13T22:20:54.541371+0000","last_became_peered":"2023-07-13T22:20:54.541371+0000","last_unstale":"2023-07-13T22:20:55.611424+0000","last_undegraded":"2023-07-13T22:20:55.611424+0000","last_fullsized":"2023-07-13T22:20:55.611424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610264+0000","last_change":"2023-07-13T22:20:53.534081+0000","last_active":"2023-07-13T22:20:55.610264+0000","last_peered":"2023-07-13T22:20:55.610264+0000","last_clean":"2023-07-13T22:20:55.610264+0000","last_became_active":"2023-07-13T22:20:53.532985+0000","last_became_peered":"2023-07-13T22:20:53.532985+0000","last_unstale":"2023-07-13T22:20:55.610264+0000","last_undegraded":"2023-07-13T22:20:55.610264+0000","last_fullsized":"2023-07-13T22:20:55.610264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:16.293217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610466+0000","last_change":"2023-07-13T22:20:54.533387+0000","last_active":"2023-07-13T22:20:55.610466+0000","last_peered":"2023-07-13T22:20:55.610466+0000","last_clean":"2023-07-13T22:20:55.610466+0000","last_became_active":"2023-07-13T22:20:54.533023+0000","last_became_peered":"2023-07-13T22:20:54.533023+0000","last_unstale":"2023-07-13T22:20:55.610466+0000","last_undegraded":"2023-07-13T22:20:55.610466+0000","last_fullsized":"2023-07-13T22:20:55.610466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611957+0000","last_change":"2023-07-13T22:20:53.538156+0000","last_active":"2023-07-13T22:20:55.611957+0000","last_peered":"2023-07-13T22:20:55.611957+0000","last_clean":"2023-07-13T22:20:55.611957+0000","last_became_active":"2023-07-13T22:20:53.536726+0000","last_became_peered":"2023-07-13T22:20:53.536726+0000","last_unstale":"2023-07-13T22:20:55.611957+0000","last_undegraded":"2023-07-13T22:20:55.611957+0000","last_fullsized":"2023-07-13T22:20:55.611957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:25.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610113+0000","last_change":"2023-07-13T22:20:54.541258+0000","last_active":"2023-07-13T22:20:55.610113+0000","last_peered":"2023-07-13T22:20:55.610113+0000","last_clean":"2023-07-13T22:20:55.610113+0000","last_became_active":"2023-07-13T22:20:54.540986+0000","last_became_peered":"2023-07-13T22:20:54.540986+0000","last_unstale":"2023-07-13T22:20:55.610113+0000","last_undegraded":"2023-07-13T22:20:55.610113+0000","last_fullsized":"2023-07-13T22:20:55.610113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532649+0000","last_change":"2023-07-13T22:20:53.536025+0000","last_active":"2023-07-13T22:20:55.532649+0000","last_peered":"2023-07-13T22:20:55.532649+0000","last_clean":"2023-07-13T22:20:55.532649+0000","last_became_active":"2023-07-13T22:20:53.535697+0000","last_became_peered":"2023-07-13T22:20:53.535697+0000","last_unstale":"2023-07-13T22:20:55.532649+0000","last_undegraded":"2023-07-13T22:20:55.532649+0000","last_fullsized":"2023-07-13T22:20:55.532649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:53.812690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610659+0000","last_change":"2023-07-13T22:20:54.537549+0000","last_active":"2023-07-13T22:20:55.610659+0000","last_peered":"2023-07-13T22:20:55.610659+0000","last_clean":"2023-07-13T22:20:55.610659+0000","last_became_active":"2023-07-13T22:20:54.537260+0000","last_became_peered":"2023-07-13T22:20:54.537260+0000","last_unstale":"2023-07-13T22:20:55.610659+0000","last_undegraded":"2023-07-13T22:20:55.610659+0000","last_fullsized":"2023-07-13T22:20:55.610659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611165+0000","last_change":"2023-07-13T22:20:53.543492+0000","last_active":"2023-07-13T22:20:55.611165+0000","last_peered":"2023-07-13T22:20:55.611165+0000","last_clean":"2023-07-13T22:20:55.611165+0000","last_became_active":"2023-07-13T22:20:53.543265+0000","last_became_peered":"2023-07-13T22:20:53.543265+0000","last_unstale":"2023-07-13T22:20:55.611165+0000","last_undegraded":"2023-07-13T22:20:55.611165+0000","last_fullsized":"2023-07-13T22:20:55.611165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:35.694626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531212+0000","last_change":"2023-07-13T22:20:54.532326+0000","last_active":"2023-07-13T22:20:55.531212+0000","last_peered":"2023-07-13T22:20:55.531212+0000","last_clean":"2023-07-13T22:20:55.531212+0000","last_became_active":"2023-07-13T22:20:54.532144+0000","last_became_peered":"2023-07-13T22:20:54.532144+0000","last_unstale":"2023-07-13T22:20:55.531212+0000","last_undegraded":"2023-07-13T22:20:55.531212+0000","last_fullsized":"2023-07-13T22:20:55.531212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:24.401992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.534193+0000","last_change":"2023-07-13T22:20:53.531850+0000","last_active":"2023-07-13T22:20:55.534193+0000","last_peered":"2023-07-13T22:20:55.534193+0000","last_clean":"2023-07-13T22:20:55.534193+0000","last_became_active":"2023-07-13T22:20:53.531567+0000","last_became_peered":"2023-07-13T22:20:53.531567+0000","last_unstale":"2023-07-13T22:20:55.534193+0000","last_undegraded":"2023-07-13T22:20:55.534193+0000","last_fullsized":"2023-07-13T22:20:55.534193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:37.890575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533168+0000","last_change":"2023-07-13T22:20:54.552096+0000","last_active":"2023-07-13T22:20:55.533168+0000","last_peered":"2023-07-13T22:20:55.533168+0000","last_clean":"2023-07-13T22:20:55.533168+0000","last_became_active":"2023-07-13T22:20:54.551842+0000","last_became_peered":"2023-07-13T22:20:54.551842+0000","last_unstale":"2023-07-13T22:20:55.533168+0000","last_undegraded":"2023-07-13T22:20:55.533168+0000","last_fullsized":"2023-07-13T22:20:55.533168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:08.460983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610860+0000","last_change":"2023-07-13T22:20:53.538459+0000","last_active":"2023-07-13T22:20:55.610860+0000","last_peered":"2023-07-13T22:20:55.610860+0000","last_clean":"2023-07-13T22:20:55.610860+0000","last_became_active":"2023-07-13T22:20:53.538180+0000","last_became_peered":"2023-07-13T22:20:53.538180+0000","last_unstale":"2023-07-13T22:20:55.610860+0000","last_undegraded":"2023-07-13T22:20:55.610860+0000","last_fullsized":"2023-07-13T22:20:55.610860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:54.538739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610410+0000","last_change":"2023-07-13T22:20:54.542603+0000","last_active":"2023-07-13T22:20:55.610410+0000","last_peered":"2023-07-13T22:20:55.610410+0000","last_clean":"2023-07-13T22:20:55.610410+0000","last_became_active":"2023-07-13T22:20:54.542348+0000","last_became_peered":"2023-07-13T22:20:54.542348+0000","last_unstale":"2023-07-13T22:20:55.610410+0000","last_undegraded":"2023-07-13T22:20:55.610410+0000","last_fullsized":"2023-07-13T22:20:55.610410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:48.562581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610578+0000","last_change":"2023-07-13T22:20:53.539294+0000","last_active":"2023-07-13T22:20:55.610578+0000","last_peered":"2023-07-13T22:20:55.610578+0000","last_clean":"2023-07-13T22:20:55.610578+0000","last_became_active":"2023-07-13T22:20:53.538780+0000","last_became_peered":"2023-07-13T22:20:53.538780+0000","last_unstale":"2023-07-13T22:20:55.610578+0000","last_undegraded":"2023-07-13T22:20:55.610578+0000","last_fullsized":"2023-07-13T22:20:55.610578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:23.393770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533163+0000","last_change":"2023-07-13T22:20:54.538797+0000","last_active":"2023-07-13T22:20:55.533163+0000","last_peered":"2023-07-13T22:20:55.533163+0000","last_clean":"2023-07-13T22:20:55.533163+0000","last_became_active":"2023-07-13T22:20:54.537618+0000","last_became_peered":"2023-07-13T22:20:54.537618+0000","last_unstale":"2023-07-13T22:20:55.533163+0000","last_undegraded":"2023-07-13T22:20:55.533163+0000","last_fullsized":"2023-07-13T22:20:55.533163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610621+0000","last_change":"2023-07-13T22:20:53.540448+0000","last_active":"2023-07-13T22:20:55.610621+0000","last_peered":"2023-07-13T22:20:55.610621+0000","last_clean":"2023-07-13T22:20:55.610621+0000","last_became_active":"2023-07-13T22:20:53.540124+0000","last_became_peered":"2023-07-13T22:20:53.540124+0000","last_unstale":"2023-07-13T22:20:55.610621+0000","last_undegraded":"2023-07-13T22:20:55.610621+0000","last_fullsized":"2023-07-13T22:20:55.610621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611081+0000","last_change":"2023-07-13T22:20:54.552868+0000","last_active":"2023-07-13T22:20:55.611081+0000","last_peered":"2023-07-13T22:20:55.611081+0000","last_clean":"2023-07-13T22:20:55.611081+0000","last_became_active":"2023-07-13T22:20:54.552531+0000","last_became_peered":"2023-07-13T22:20:54.552531+0000","last_unstale":"2023-07-13T22:20:55.611081+0000","last_undegraded":"2023-07-13T22:20:55.611081+0000","last_fullsized":"2023-07-13T22:20:55.611081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610438+0000","last_change":"2023-07-13T22:20:53.533055+0000","last_active":"2023-07-13T22:20:55.610438+0000","last_peered":"2023-07-13T22:20:55.610438+0000","last_clean":"2023-07-13T22:20:55.610438+0000","last_became_active":"2023-07-13T22:20:53.532792+0000","last_became_peered":"2023-07-13T22:20:53.532792+0000","last_unstale":"2023-07-13T22:20:55.610438+0000","last_undegraded":"2023-07-13T22:20:55.610438+0000","last_fullsized":"2023-07-13T22:20:55.610438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:28.638041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610888+0000","last_change":"2023-07-13T22:20:54.536719+0000","last_active":"2023-07-13T22:20:55.610888+0000","last_peered":"2023-07-13T22:20:55.610888+0000","last_clean":"2023-07-13T22:20:55.610888+0000","last_became_active":"2023-07-13T22:20:54.535865+0000","last_became_peered":"2023-07-13T22:20:54.535865+0000","last_unstale":"2023-07-13T22:20:55.610888+0000","last_undegraded":"2023-07-13T22:20:55.610888+0000","last_fullsized":"2023-07-13T22:20:55.610888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:04:58.712616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610902+0000","last_change":"2023-07-13T22:20:53.537777+0000","last_active":"2023-07-13T22:20:55.610902+0000","last_peered":"2023-07-13T22:20:55.610902+0000","last_clean":"2023-07-13T22:20:55.610902+0000","last_became_active":"2023-07-13T22:20:53.537464+0000","last_became_peered":"2023-07-13T22:20:53.537464+0000","last_unstale":"2023-07-13T22:20:55.610902+0000","last_undegraded":"2023-07-13T22:20:55.610902+0000","last_fullsized":"2023-07-13T22:20:55.610902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:47.554360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532565+0000","last_change":"2023-07-13T22:20:54.536450+0000","last_active":"2023-07-13T22:20:55.532565+0000","last_peered":"2023-07-13T22:20:55.532565+0000","last_clean":"2023-07-13T22:20:55.532565+0000","last_became_active":"2023-07-13T22:20:54.536099+0000","last_became_peered":"2023-07-13T22:20:54.536099+0000","last_unstale":"2023-07-13T22:20:55.532565+0000","last_undegraded":"2023-07-13T22:20:55.532565+0000","last_fullsized":"2023-07-13T22:20:55.532565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610785+0000","last_change":"2023-07-13T22:20:53.535131+0000","last_active":"2023-07-13T22:20:55.610785+0000","last_peered":"2023-07-13T22:20:55.610785+0000","last_clean":"2023-07-13T22:20:55.610785+0000","last_became_active":"2023-07-13T22:20:53.534737+0000","last_became_peered":"2023-07-13T22:20:53.534737+0000","last_unstale":"2023-07-13T22:20:55.610785+0000","last_undegraded":"2023-07-13T22:20:55.610785+0000","last_fullsized":"2023-07-13T22:20:55.610785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:16.293217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533196+0000","last_change":"2023-07-13T22:20:54.546160+0000","last_active":"2023-07-13T22:20:55.533196+0000","last_peered":"2023-07-13T22:20:55.533196+0000","last_clean":"2023-07-13T22:20:55.533196+0000","last_became_active":"2023-07-13T22:20:54.545713+0000","last_became_peered":"2023-07-13T22:20:54.545713+0000","last_unstale":"2023-07-13T22:20:55.533196+0000","last_undegraded":"2023-07-13T22:20:55.533196+0000","last_fullsized":"2023-07-13T22:20:55.533196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:17.301438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610989+0000","last_change":"2023-07-13T22:20:53.536620+0000","last_active":"2023-07-13T22:20:55.610989+0000","last_peered":"2023-07-13T22:20:55.610989+0000","last_clean":"2023-07-13T22:20:55.610989+0000","last_became_active":"2023-07-13T22:20:53.536322+0000","last_became_peered":"2023-07-13T22:20:53.536322+0000","last_unstale":"2023-07-13T22:20:55.610989+0000","last_undegraded":"2023-07-13T22:20:55.610989+0000","last_fullsized":"2023-07-13T22:20:55.610989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:55.228110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610640+0000","last_change":"2023-07-13T22:20:54.544907+0000","last_active":"2023-07-13T22:20:55.610640+0000","last_peered":"2023-07-13T22:20:55.610640+0000","last_clean":"2023-07-13T22:20:55.610640+0000","last_became_active":"2023-07-13T22:20:54.544596+0000","last_became_peered":"2023-07-13T22:20:54.544596+0000","last_unstale":"2023-07-13T22:20:55.610640+0000","last_undegraded":"2023-07-13T22:20:55.610640+0000","last_fullsized":"2023-07-13T22:20:55.610640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533567+0000","last_change":"2023-07-13T22:20:53.530981+0000","last_active":"2023-07-13T22:20:55.533567+0000","last_peered":"2023-07-13T22:20:55.533567+0000","last_clean":"2023-07-13T22:20:55.533567+0000","last_became_active":"2023-07-13T22:20:53.530568+0000","last_became_peered":"2023-07-13T22:20:53.530568+0000","last_unstale":"2023-07-13T22:20:55.533567+0000","last_undegraded":"2023-07-13T22:20:55.533567+0000","last_fullsized":"2023-07-13T22:20:55.533567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610942+0000","last_change":"2023-07-13T22:20:54.540498+0000","last_active":"2023-07-13T22:20:55.610942+0000","last_peered":"2023-07-13T22:20:55.610942+0000","last_clean":"2023-07-13T22:20:55.610942+0000","last_became_active":"2023-07-13T22:20:54.540161+0000","last_became_peered":"2023-07-13T22:20:54.540161+0000","last_unstale":"2023-07-13T22:20:55.610942+0000","last_undegraded":"2023-07-13T22:20:55.610942+0000","last_fullsized":"2023-07-13T22:20:55.610942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:12.599393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610764+0000","last_change":"2023-07-13T22:20:53.533470+0000","last_active":"2023-07-13T22:20:55.610764+0000","last_peered":"2023-07-13T22:20:55.610764+0000","last_clean":"2023-07-13T22:20:55.610764+0000","last_became_active":"2023-07-13T22:20:53.532854+0000","last_became_peered":"2023-07-13T22:20:53.532854+0000","last_unstale":"2023-07-13T22:20:55.610764+0000","last_undegraded":"2023-07-13T22:20:55.610764+0000","last_fullsized":"2023-07-13T22:20:55.610764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:37.890575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.534049+0000","last_change":"2023-07-13T22:20:54.552213+0000","last_active":"2023-07-13T22:20:55.534049+0000","last_peered":"2023-07-13T22:20:55.534049+0000","last_clean":"2023-07-13T22:20:55.534049+0000","last_became_active":"2023-07-13T22:20:54.548415+0000","last_became_peered":"2023-07-13T22:20:54.548415+0000","last_unstale":"2023-07-13T22:20:55.534049+0000","last_undegraded":"2023-07-13T22:20:55.534049+0000","last_fullsized":"2023-07-13T22:20:55.534049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:50.171609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610542+0000","last_change":"2023-07-13T22:20:53.529929+0000","last_active":"2023-07-13T22:20:55.610542+0000","last_peered":"2023-07-13T22:20:55.610542+0000","last_clean":"2023-07-13T22:20:55.610542+0000","last_became_active":"2023-07-13T22:20:53.529645+0000","last_became_peered":"2023-07-13T22:20:53.529645+0000","last_unstale":"2023-07-13T22:20:55.610542+0000","last_undegraded":"2023-07-13T22:20:55.610542+0000","last_fullsized":"2023-07-13T22:20:55.610542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:53.812690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535287+0000","last_change":"2023-07-13T22:20:54.550835+0000","last_active":"2023-07-13T22:20:55.535287+0000","last_peered":"2023-07-13T22:20:55.535287+0000","last_clean":"2023-07-13T22:20:55.535287+0000","last_became_active":"2023-07-13T22:20:54.547187+0000","last_became_peered":"2023-07-13T22:20:54.547187+0000","last_unstale":"2023-07-13T22:20:55.535287+0000","last_undegraded":"2023-07-13T22:20:55.535287+0000","last_fullsized":"2023-07-13T22:20:55.535287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:04:58.712616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533482+0000","last_change":"2023-07-13T22:20:53.534667+0000","last_active":"2023-07-13T22:20:55.533482+0000","last_peered":"2023-07-13T22:20:55.533482+0000","last_clean":"2023-07-13T22:20:55.533482+0000","last_became_active":"2023-07-13T22:20:53.534433+0000","last_became_peered":"2023-07-13T22:20:53.534433+0000","last_unstale":"2023-07-13T22:20:55.533482+0000","last_undegraded":"2023-07-13T22:20:55.533482+0000","last_fullsized":"2023-07-13T22:20:55.533482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:28.070264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535373+0000","last_change":"2023-07-13T22:20:54.550679+0000","last_active":"2023-07-13T22:20:55.535373+0000","last_peered":"2023-07-13T22:20:55.535373+0000","last_clean":"2023-07-13T22:20:55.535373+0000","last_became_active":"2023-07-13T22:20:54.546483+0000","last_became_peered":"2023-07-13T22:20:54.546483+0000","last_unstale":"2023-07-13T22:20:55.535373+0000","last_undegraded":"2023-07-13T22:20:55.535373+0000","last_fullsized":"2023-07-13T22:20:55.535373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:16:54.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610029+0000","last_change":"2023-07-13T22:20:53.531815+0000","last_active":"2023-07-13T22:20:55.610029+0000","last_peered":"2023-07-13T22:20:55.610029+0000","last_clean":"2023-07-13T22:20:55.610029+0000","last_became_active":"2023-07-13T22:20:53.531551+0000","last_became_peered":"2023-07-13T22:20:53.531551+0000","last_unstale":"2023-07-13T22:20:55.610029+0000","last_undegraded":"2023-07-13T22:20:55.610029+0000","last_fullsized":"2023-07-13T22:20:55.610029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531764+0000","last_change":"2023-07-13T22:20:54.542085+0000","last_active":"2023-07-13T22:20:55.531764+0000","last_peered":"2023-07-13T22:20:55.531764+0000","last_clean":"2023-07-13T22:20:55.531764+0000","last_became_active":"2023-07-13T22:20:54.541749+0000","last_became_peered":"2023-07-13T22:20:54.541749+0000","last_unstale":"2023-07-13T22:20:55.531764+0000","last_undegraded":"2023-07-13T22:20:55.531764+0000","last_fullsized":"2023-07-13T22:20:55.531764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610668+0000","last_change":"2023-07-13T22:20:53.539618+0000","last_active":"2023-07-13T22:20:55.610668+0000","last_peered":"2023-07-13T22:20:55.610668+0000","last_clean":"2023-07-13T22:20:55.610668+0000","last_became_active":"2023-07-13T22:20:53.539014+0000","last_became_peered":"2023-07-13T22:20:53.539014+0000","last_unstale":"2023-07-13T22:20:55.610668+0000","last_undegraded":"2023-07-13T22:20:55.610668+0000","last_fullsized":"2023-07-13T22:20:55.610668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:49.163388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535108+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:20:55.535108+0000","last_peered":"2023-07-13T22:20:55.535108+0000","last_clean":"2023-07-13T22:20:55.535108+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:20:55.535108+0000","last_undegraded":"2023-07-13T22:20:55.535108+0000","last_fullsized":"2023-07-13T22:20:55.535108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:11.435573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.609968+0000","last_change":"2023-07-13T22:20:54.538149+0000","last_active":"2023-07-13T22:20:55.609968+0000","last_peered":"2023-07-13T22:20:55.609968+0000","last_clean":"2023-07-13T22:20:55.609968+0000","last_became_active":"2023-07-13T22:20:54.537883+0000","last_became_peered":"2023-07-13T22:20:54.537883+0000","last_unstale":"2023-07-13T22:20:55.609968+0000","last_undegraded":"2023-07-13T22:20:55.609968+0000","last_fullsized":"2023-07-13T22:20:55.609968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610497+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:20:55.610497+0000","last_peered":"2023-07-13T22:20:55.610497+0000","last_clean":"2023-07-13T22:20:55.610497+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:20:55.610497+0000","last_undegraded":"2023-07-13T22:20:55.610497+0000","last_fullsized":"2023-07-13T22:20:55.610497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:49.435942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611546+0000","last_change":"2023-07-13T22:20:53.541916+0000","last_active":"2023-07-13T22:20:55.611546+0000","last_peered":"2023-07-13T22:20:55.611546+0000","last_clean":"2023-07-13T22:20:55.611546+0000","last_became_active":"2023-07-13T22:20:53.541602+0000","last_became_peered":"2023-07-13T22:20:53.541602+0000","last_unstale":"2023-07-13T22:20:55.611546+0000","last_undegraded":"2023-07-13T22:20:55.611546+0000","last_fullsized":"2023-07-13T22:20:55.611546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:50.801197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533347+0000","last_change":"2023-07-13T22:20:54.545955+0000","last_active":"2023-07-13T22:20:55.533347+0000","last_peered":"2023-07-13T22:20:55.533347+0000","last_clean":"2023-07-13T22:20:55.533347+0000","last_became_active":"2023-07-13T22:20:54.545361+0000","last_became_peered":"2023-07-13T22:20:54.545361+0000","last_unstale":"2023-07-13T22:20:55.533347+0000","last_undegraded":"2023-07-13T22:20:55.533347+0000","last_fullsized":"2023-07-13T22:20:55.533347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531064+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:20:55.531064+0000","last_peered":"2023-07-13T22:20:55.531064+0000","last_clean":"2023-07-13T22:20:55.531064+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:20:55.531064+0000","last_undegraded":"2023-07-13T22:20:55.531064+0000","last_fullsized":"2023-07-13T22:20:55.531064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:15:56.981684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611242+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:20:55.611242+0000","last_peered":"2023-07-13T22:20:55.611242+0000","last_clean":"2023-07-13T22:20:55.611242+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:20:55.611242+0000","last_undegraded":"2023-07-13T22:20:55.611242+0000","last_fullsized":"2023-07-13T22:20:55.611242+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:44.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610851+0000","last_change":"2023-07-13T22:20:53.537543+0000","last_active":"2023-07-13T22:20:55.610851+0000","last_peered":"2023-07-13T22:20:55.610851+0000","last_clean":"2023-07-13T22:20:55.610851+0000","last_became_active":"2023-07-13T22:20:53.537125+0000","last_became_peered":"2023-07-13T22:20:53.537125+0000","last_unstale":"2023-07-13T22:20:55.610851+0000","last_undegraded":"2023-07-13T22:20:55.610851+0000","last_fullsized":"2023-07-13T22:20:55.610851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:48.574628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611216+0000","last_change":"2023-07-13T22:20:54.541651+0000","last_active":"2023-07-13T22:20:55.611216+0000","last_peered":"2023-07-13T22:20:55.611216+0000","last_clean":"2023-07-13T22:20:55.611216+0000","last_became_active":"2023-07-13T22:20:54.541059+0000","last_became_peered":"2023-07-13T22:20:54.541059+0000","last_unstale":"2023-07-13T22:20:55.611216+0000","last_undegraded":"2023-07-13T22:20:55.611216+0000","last_fullsized":"2023-07-13T22:20:55.611216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:54.820912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610481+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:20:55.610481+0000","last_peered":"2023-07-13T22:20:55.610481+0000","last_clean":"2023-07-13T22:20:55.610481+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:20:55.610481+0000","last_undegraded":"2023-07-13T22:20:55.610481+0000","last_fullsized":"2023-07-13T22:20:55.610481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:20.066975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.530855+0000","last_change":"2023-07-13T22:20:53.533784+0000","last_active":"2023-07-13T22:20:55.530855+0000","last_peered":"2023-07-13T22:20:55.530855+0000","last_clean":"2023-07-13T22:20:55.530855+0000","last_became_active":"2023-07-13T22:20:53.533270+0000","last_became_peered":"2023-07-13T22:20:53.533270+0000","last_unstale":"2023-07-13T22:20:55.530855+0000","last_undegraded":"2023-07-13T22:20:55.530855+0000","last_fullsized":"2023-07-13T22:20:55.530855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:29:28.555785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531674+0000","last_change":"2023-07-13T22:20:54.529627+0000","last_active":"2023-07-13T22:20:55.531674+0000","last_peered":"2023-07-13T22:20:55.531674+0000","last_clean":"2023-07-13T22:20:55.531674+0000","last_became_active":"2023-07-13T22:20:54.528020+0000","last_became_peered":"2023-07-13T22:20:54.528020+0000","last_unstale":"2023-07-13T22:20:55.531674+0000","last_undegraded":"2023-07-13T22:20:55.531674+0000","last_fullsized":"2023-07-13T22:20:55.531674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:54.820912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610893+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:20:55.610893+0000","last_peered":"2023-07-13T22:20:55.610893+0000","last_clean":"2023-07-13T22:20:55.610893+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:20:55.610893+0000","last_undegraded":"2023-07-13T22:20:55.610893+0000","last_fullsized":"2023-07-13T22:20:55.610893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:22.278097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610262+0000","last_change":"2023-07-13T22:20:53.535232+0000","last_active":"2023-07-13T22:20:55.610262+0000","last_peered":"2023-07-13T22:20:55.610262+0000","last_clean":"2023-07-13T22:20:55.610262+0000","last_became_active":"2023-07-13T22:20:53.533935+0000","last_became_peered":"2023-07-13T22:20:53.533935+0000","last_unstale":"2023-07-13T22:20:55.610262+0000","last_undegraded":"2023-07-13T22:20:55.610262+0000","last_fullsized":"2023-07-13T22:20:55.610262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610591+0000","last_change":"2023-07-13T22:20:54.549217+0000","last_active":"2023-07-13T22:20:55.610591+0000","last_peered":"2023-07-13T22:20:55.610591+0000","last_clean":"2023-07-13T22:20:55.610591+0000","last_became_active":"2023-07-13T22:20:54.548902+0000","last_became_peered":"2023-07-13T22:20:54.548902+0000","last_unstale":"2023-07-13T22:20:55.610591+0000","last_undegraded":"2023-07-13T22:20:55.610591+0000","last_fullsized":"2023-07-13T22:20:55.610591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532394+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:20:55.532394+0000","last_peered":"2023-07-13T22:20:55.532394+0000","last_clean":"2023-07-13T22:20:55.532394+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:20:55.532394+0000","last_undegraded":"2023-07-13T22:20:55.532394+0000","last_fullsized":"2023-07-13T22:20:55.532394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:05.799004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532935+0000","last_change":"2023-07-13T22:20:53.537640+0000","last_active":"2023-07-13T22:20:55.532935+0000","last_peered":"2023-07-13T22:20:55.532935+0000","last_clean":"2023-07-13T22:20:55.532935+0000","last_became_active":"2023-07-13T22:20:53.537351+0000","last_became_peered":"2023-07-13T22:20:53.537351+0000","last_unstale":"2023-07-13T22:20:55.532935+0000","last_undegraded":"2023-07-13T22:20:55.532935+0000","last_fullsized":"2023-07-13T22:20:55.532935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531434+0000","last_change":"2023-07-13T22:20:54.530546+0000","last_active":"2023-07-13T22:20:55.531434+0000","last_peered":"2023-07-13T22:20:55.531434+0000","last_clean":"2023-07-13T22:20:55.531434+0000","last_became_active":"2023-07-13T22:20:54.530234+0000","last_became_peered":"2023-07-13T22:20:54.530234+0000","last_unstale":"2023-07-13T22:20:55.531434+0000","last_undegraded":"2023-07-13T22:20:55.531434+0000","last_fullsized":"2023-07-13T22:20:55.531434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610282+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:20:55.610282+0000","last_peered":"2023-07-13T22:20:55.610282+0000","last_clean":"2023-07-13T22:20:55.610282+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:20:55.610282+0000","last_undegraded":"2023-07-13T22:20:55.610282+0000","last_fullsized":"2023-07-13T22:20:55.610282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:05.799004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531292+0000","last_change":"2023-07-13T22:20:53.533697+0000","last_active":"2023-07-13T22:20:55.531292+0000","last_peered":"2023-07-13T22:20:55.531292+0000","last_clean":"2023-07-13T22:20:55.531292+0000","last_became_active":"2023-07-13T22:20:53.533261+0000","last_became_peered":"2023-07-13T22:20:53.533261+0000","last_unstale":"2023-07-13T22:20:55.531292+0000","last_undegraded":"2023-07-13T22:20:55.531292+0000","last_fullsized":"2023-07-13T22:20:55.531292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:11.591172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611449+0000","last_change":"2023-07-13T22:20:54.537844+0000","last_active":"2023-07-13T22:20:55.611449+0000","last_peered":"2023-07-13T22:20:55.611449+0000","last_clean":"2023-07-13T22:20:55.611449+0000","last_became_active":"2023-07-13T22:20:54.537526+0000","last_became_peered":"2023-07-13T22:20:54.537526+0000","last_unstale":"2023-07-13T22:20:55.611449+0000","last_undegraded":"2023-07-13T22:20:55.611449+0000","last_fullsized":"2023-07-13T22:20:55.611449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610286+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:20:55.610286+0000","last_peered":"2023-07-13T22:20:55.610286+0000","last_clean":"2023-07-13T22:20:55.610286+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:20:55.610286+0000","last_undegraded":"2023-07-13T22:20:55.610286+0000","last_fullsized":"2023-07-13T22:20:55.610286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:17.601603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533295+0000","last_change":"2023-07-13T22:20:53.532273+0000","last_active":"2023-07-13T22:20:55.533295+0000","last_peered":"2023-07-13T22:20:55.533295+0000","last_clean":"2023-07-13T22:20:55.533295+0000","last_became_active":"2023-07-13T22:20:53.531699+0000","last_became_peered":"2023-07-13T22:20:53.531699+0000","last_unstale":"2023-07-13T22:20:55.533295+0000","last_undegraded":"2023-07-13T22:20:55.533295+0000","last_fullsized":"2023-07-13T22:20:55.533295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531599+0000","last_change":"2023-07-13T22:20:54.530026+0000","last_active":"2023-07-13T22:20:55.531599+0000","last_peered":"2023-07-13T22:20:55.531599+0000","last_clean":"2023-07-13T22:20:55.531599+0000","last_became_active":"2023-07-13T22:20:54.528217+0000","last_became_peered":"2023-07-13T22:20:54.528217+0000","last_unstale":"2023-07-13T22:20:55.531599+0000","last_undegraded":"2023-07-13T22:20:55.531599+0000","last_fullsized":"2023-07-13T22:20:55.531599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611152+0000","last_change":"2023-07-13T22:20:53.541066+0000","last_active":"2023-07-13T22:20:55.611152+0000","last_peered":"2023-07-13T22:20:55.611152+0000","last_clean":"2023-07-13T22:20:55.611152+0000","last_became_active":"2023-07-13T22:20:53.540748+0000","last_became_peered":"2023-07-13T22:20:53.540748+0000","last_unstale":"2023-07-13T22:20:55.611152+0000","last_undegraded":"2023-07-13T22:20:55.611152+0000","last_fullsized":"2023-07-13T22:20:55.611152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:25.360393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.612132+0000","last_change":"2023-07-13T22:20:54.531788+0000","last_active":"2023-07-13T22:20:55.612132+0000","last_peered":"2023-07-13T22:20:55.612132+0000","last_clean":"2023-07-13T22:20:55.612132+0000","last_became_active":"2023-07-13T22:20:54.531396+0000","last_became_peered":"2023-07-13T22:20:54.531396+0000","last_unstale":"2023-07-13T22:20:55.612132+0000","last_undegraded":"2023-07-13T22:20:55.612132+0000","last_fullsized":"2023-07-13T22:20:55.612132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:45:27.037988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533306+0000","last_change":"2023-07-13T22:20:53.530819+0000","last_active":"2023-07-13T22:20:55.533306+0000","last_peered":"2023-07-13T22:20:55.533306+0000","last_clean":"2023-07-13T22:20:55.533306+0000","last_became_active":"2023-07-13T22:20:53.530223+0000","last_became_peered":"2023-07-13T22:20:53.530223+0000","last_unstale":"2023-07-13T22:20:55.533306+0000","last_undegraded":"2023-07-13T22:20:55.533306+0000","last_fullsized":"2023-07-13T22:20:55.533306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:15:58.999140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532780+0000","last_change":"2023-07-13T22:20:54.551556+0000","last_active":"2023-07-13T22:20:55.532780+0000","last_peered":"2023-07-13T22:20:55.532780+0000","last_clean":"2023-07-13T22:20:55.532780+0000","last_became_active":"2023-07-13T22:20:54.547048+0000","last_became_peered":"2023-07-13T22:20:54.547048+0000","last_unstale":"2023-07-13T22:20:55.532780+0000","last_undegraded":"2023-07-13T22:20:55.532780+0000","last_fullsized":"2023-07-13T22:20:55.532780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533441+0000","last_change":"2023-07-13T22:20:53.532040+0000","last_active":"2023-07-13T22:20:55.533441+0000","last_peered":"2023-07-13T22:20:55.533441+0000","last_clean":"2023-07-13T22:20:55.533441+0000","last_became_active":"2023-07-13T22:20:53.531414+0000","last_became_peered":"2023-07-13T22:20:53.531414+0000","last_unstale":"2023-07-13T22:20:55.533441+0000","last_undegraded":"2023-07-13T22:20:55.533441+0000","last_fullsized":"2023-07-13T22:20:55.533441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:25.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533812+0000","last_change":"2023-07-13T22:20:54.551351+0000","last_active":"2023-07-13T22:20:55.533812+0000","last_peered":"2023-07-13T22:20:55.533812+0000","last_clean":"2023-07-13T22:20:55.533812+0000","last_became_active":"2023-07-13T22:20:54.550932+0000","last_became_peered":"2023-07-13T22:20:54.550932+0000","last_unstale":"2023-07-13T22:20:55.533812+0000","last_undegraded":"2023-07-13T22:20:55.533812+0000","last_fullsized":"2023-07-13T22:20:55.533812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:36.702847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610605+0000","last_change":"2023-07-13T22:20:53.530713+0000","last_active":"2023-07-13T22:20:55.610605+0000","last_peered":"2023-07-13T22:20:55.610605+0000","last_clean":"2023-07-13T22:20:55.610605+0000","last_became_active":"2023-07-13T22:20:53.530419+0000","last_became_peered":"2023-07-13T22:20:53.530419+0000","last_unstale":"2023-07-13T22:20:55.610605+0000","last_undegraded":"2023-07-13T22:20:55.610605+0000","last_fullsized":"2023-07-13T22:20:55.610605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:35.694626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610476+0000","last_change":"2023-07-13T22:20:54.541258+0000","last_active":"2023-07-13T22:20:55.610476+0000","last_peered":"2023-07-13T22:20:55.610476+0000","last_clean":"2023-07-13T22:20:55.610476+0000","last_became_active":"2023-07-13T22:20:54.540610+0000","last_became_peered":"2023-07-13T22:20:54.540610+0000","last_unstale":"2023-07-13T22:20:55.610476+0000","last_undegraded":"2023-07-13T22:20:55.610476+0000","last_fullsized":"2023-07-13T22:20:55.610476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:29.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.612057+0000","last_change":"2023-07-13T22:20:53.538279+0000","last_active":"2023-07-13T22:20:55.612057+0000","last_peered":"2023-07-13T22:20:55.612057+0000","last_clean":"2023-07-13T22:20:55.612057+0000","last_became_active":"2023-07-13T22:20:53.537651+0000","last_became_peered":"2023-07-13T22:20:53.537651+0000","last_unstale":"2023-07-13T22:20:55.612057+0000","last_undegraded":"2023-07-13T22:20:55.612057+0000","last_fullsized":"2023-07-13T22:20:55.612057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532866+0000","last_change":"2023-07-13T22:20:54.541753+0000","last_active":"2023-07-13T22:20:55.532866+0000","last_peered":"2023-07-13T22:20:55.532866+0000","last_clean":"2023-07-13T22:20:55.532866+0000","last_became_active":"2023-07-13T22:20:54.540946+0000","last_became_peered":"2023-07-13T22:20:54.540946+0000","last_unstale":"2023-07-13T22:20:55.532866+0000","last_undegraded":"2023-07-13T22:20:55.532866+0000","last_fullsized":"2023-07-13T22:20:55.532866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610348+0000","last_change":"2023-07-13T22:20:54.542872+0000","last_active":"2023-07-13T22:20:55.610348+0000","last_peered":"2023-07-13T22:20:55.610348+0000","last_clean":"2023-07-13T22:20:55.610348+0000","last_became_active":"2023-07-13T22:20:54.542534+0000","last_became_peered":"2023-07-13T22:20:54.542534+0000","last_unstale":"2023-07-13T22:20:55.610348+0000","last_undegraded":"2023-07-13T22:20:55.610348+0000","last_fullsized":"2023-07-13T22:20:55.610348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610319+0000","last_change":"2023-07-13T22:20:54.541488+0000","last_active":"2023-07-13T22:20:55.610319+0000","last_peered":"2023-07-13T22:20:55.610319+0000","last_clean":"2023-07-13T22:20:55.610319+0000","last_became_active":"2023-07-13T22:20:54.541003+0000","last_became_peered":"2023-07-13T22:20:54.541003+0000","last_unstale":"2023-07-13T22:20:55.610319+0000","last_undegraded":"2023-07-13T22:20:55.610319+0000","last_fullsized":"2023-07-13T22:20:55.610319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533452+0000","last_change":"2023-07-13T22:20:54.552460+0000","last_active":"2023-07-13T22:20:55.533452+0000","last_peered":"2023-07-13T22:20:55.533452+0000","last_clean":"2023-07-13T22:20:55.533452+0000","last_became_active":"2023-07-13T22:20:54.552208+0000","last_became_peered":"2023-07-13T22:20:54.552208+0000","last_unstale":"2023-07-13T22:20:55.533452+0000","last_undegraded":"2023-07-13T22:20:55.533452+0000","last_fullsized":"2023-07-13T22:20:55.533452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:48.562581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610888+0000","last_change":"2023-07-13T22:20:54.535876+0000","last_active":"2023-07-13T22:20:55.610888+0000","last_peered":"2023-07-13T22:20:55.610888+0000","last_clean":"2023-07-13T22:20:55.610888+0000","last_became_active":"2023-07-13T22:20:54.535570+0000","last_became_peered":"2023-07-13T22:20:54.535570+0000","last_unstale":"2023-07-13T22:20:55.610888+0000","last_undegraded":"2023-07-13T22:20:55.610888+0000","last_fullsized":"2023-07-13T22:20:55.610888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:18.235962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531178+0000","last_change":"2023-07-13T22:20:54.535911+0000","last_active":"2023-07-13T22:20:55.531178+0000","last_peered":"2023-07-13T22:20:55.531178+0000","last_clean":"2023-07-13T22:20:55.531178+0000","last_became_active":"2023-07-13T22:20:54.535593+0000","last_became_peered":"2023-07-13T22:20:54.535593+0000","last_unstale":"2023-07-13T22:20:55.531178+0000","last_undegraded":"2023-07-13T22:20:55.531178+0000","last_fullsized":"2023-07-13T22:20:55.531178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:43.402535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":671608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":72807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":72403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":200704,"data_stored":71821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":81080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":73459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:20:58.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:20:59.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.203+0000 7ff19d9ff700 1 Processor -- start 2023-07-13T22:20:59.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.204+0000 7ff19d9ff700 1 -- start start 2023-07-13T22:20:59.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff19d9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff1980d3750 0x7ff1980cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff19d9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff1980d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff19d9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1980cd950 con 0x7ff1980b8400 2023-07-13T22:20:59.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff19d9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1980cff50 con 0x7ff1980d3750 2023-07-13T22:20:59.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff19d9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980d0ac0 con 0x7ff1980cfad0 2023-07-13T22:20:59.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff196ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff1980b8400 0x7ff1980cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39532/0 (socket says 172.21.15.60:39532) 2023-07-13T22:20:59.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff196ffd700 1 -- 172.21.15.60:0/2686712218 learned_addr learned my addr 172.21.15.60:0/2686712218 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:59.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff1977fe700 1 --2- 172.21.15.60:0/2686712218 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff1980d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff1967fc700 1 --2- 172.21.15.60:0/2686712218 >> v2:172.21.15.105:3300/0 conn(0x7ff1980d3750 0x7ff1980cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff195ffb700 1 -- 172.21.15.60:0/2686712218 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564223598 0 0) 0x7ff188001110 con 0x7ff1980b8400 2023-07-13T22:20:59.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.205+0000 7ff195ffb700 1 -- 172.21.15.60:0/2686712218 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1800035b0 con 0x7ff1980b8400 2023-07-13T22:20:59.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff195ffb700 1 -- 172.21.15.60:0/2686712218 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2131019298 0 0) 0x7ff1800035b0 con 0x7ff1980b8400 2023-07-13T22:20:59.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff1967fc700 1 -- 172.21.15.60:0/2686712218 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 msgr2=0x7ff1980d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:59.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff1967fc700 1 --2- 172.21.15.60:0/2686712218 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff1980d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff1967fc700 1 -- 172.21.15.60:0/2686712218 >> v1:172.21.15.60:6789/0 conn(0x7ff1980b8400 legacy=0x7ff1980cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff1967fc700 1 -- 172.21.15.60:0/2686712218 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980b87d0 con 0x7ff1980d3750 2023-07-13T22:20:59.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff1977fe700 1 --2- 172.21.15.60:0/2686712218 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff1980d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:59.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff1967fc700 1 --2- 172.21.15.60:0/2686712218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 0x7ff1980cf590 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009840 tx=0x7ff18c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c957e4762fb7a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:59.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff195ffb700 1 -- 172.21.15.60:0/2686712218 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188001800 con 0x7ff1980d3750 2023-07-13T22:20:59.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff195ffb700 1 -- 172.21.15.60:0/2686712218 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c004030 con 0x7ff1980d3750 2023-07-13T22:20:59.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.206+0000 7ff195ffb700 1 -- 172.21.15.60:0/2686712218 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0043c0 con 0x7ff1980d3750 2023-07-13T22:20:59.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.207+0000 7ff19d9ff700 1 -- 172.21.15.60:0/2686712218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 msgr2=0x7ff1980cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.207+0000 7ff19d9ff700 1 --2- 172.21.15.60:0/2686712218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 0x7ff1980cf590 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009840 tx=0x7ff18c009b50 comp rx=0 tx=0).stop 2023-07-13T22:20:59.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.207+0000 7ff19d9ff700 1 -- 172.21.15.60:0/2686712218 shutdown_connections 2023-07-13T22:20:59.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.207+0000 7ff19d9ff700 1 --2- 172.21.15.60:0/2686712218 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff1980d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.207+0000 7ff19d9ff700 1 --2- 172.21.15.60:0/2686712218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 0x7ff1980cf590 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.207+0000 7ff19d9ff700 1 -- 172.21.15.60:0/2686712218 >> 172.21.15.60:0/2686712218 conn(0x7ff19806b390 msgr2=0x7ff19806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:59.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.207+0000 7ff19d9ff700 1 -- 172.21.15.60:0/2686712218 shutdown_connections 2023-07-13T22:20:59.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.207+0000 7ff19d9ff700 1 -- 172.21.15.60:0/2686712218 wait complete. 2023-07-13T22:20:59.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.209+0000 7ff19d9ff700 1 Processor -- start 2023-07-13T22:20:59.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.209+0000 7ff19d9ff700 1 -- start start 2023-07-13T22:20:59.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff19d9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1980b8400 0x7ff1981f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff19d9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff198150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff19d9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 0x7ff1980d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff19d9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1981f2f00 con 0x7ff1980b8400 2023-07-13T22:20:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff19d9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1981f35f0 con 0x7ff1980d3750 2023-07-13T22:20:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff19d9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1981f2c20 con 0x7ff1980cfad0 2023-07-13T22:20:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff196ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1980b8400 0x7ff1981f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff196ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1980b8400 0x7ff1981f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33768/0 (socket says 172.21.15.60:33768) 2023-07-13T22:20:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff196ffd700 1 -- 172.21.15.60:0/1322875067 learned_addr learned my addr 172.21.15.60:0/1322875067 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff1967fc700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff198150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff1977fe700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 0x7ff1980d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff196ffd700 1 -- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 msgr2=0x7ff198150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff196ffd700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff198150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff196ffd700 1 -- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 msgr2=0x7ff1980d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff196ffd700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 0x7ff1980d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff196ffd700 1 -- 172.21.15.60:0/1322875067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009a70 con 0x7ff1980b8400 2023-07-13T22:20:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff1977fe700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 0x7ff1980d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:20:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.210+0000 7ff1967fc700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff198150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:20:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.211+0000 7ff196ffd700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1980b8400 0x7ff1981f30b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff18800c500 tx=0x7ff18800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.211+0000 7ff17ffff700 1 -- 172.21.15.60:0/1322875067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188004c80 con 0x7ff1980b8400 2023-07-13T22:20:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.211+0000 7ff17ffff700 1 -- 172.21.15.60:0/1322875067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff188004e00 con 0x7ff1980b8400 2023-07-13T22:20:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.211+0000 7ff17ffff700 1 -- 172.21.15.60:0/1322875067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188007310 con 0x7ff1980b8400 2023-07-13T22:20:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.211+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c0094a0 con 0x7ff1980b8400 2023-07-13T22:20:59.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.211+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19815e4d0 con 0x7ff1980b8400 2023-07-13T22:20:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.212+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19815e050 con 0x7ff1980b8400 2023-07-13T22:20:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.213+0000 7ff17ffff700 1 -- 172.21.15.60:0/1322875067 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff188007470 con 0x7ff1980b8400 2023-07-13T22:20:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.215+0000 7ff17ffff700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff18406ace0 0x7ff18406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.215+0000 7ff17ffff700 1 -- 172.21.15.60:0/1322875067 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1880890f0 con 0x7ff1980b8400 2023-07-13T22:20:59.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.215+0000 7ff1967fc700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff18406ace0 0x7ff18406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.215+0000 7ff1967fc700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff18406ace0 0x7ff18406d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff1980d3b60 tx=0x7ff18c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:59.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.219+0000 7ff17ffff700 1 -- 172.21.15.60:0/1322875067 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff19815e050 con 0x7ff1980b8400 2023-07-13T22:20:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.420+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1980b5190 con 0x7ff18406ace0 2023-07-13T22:20:59.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.428+0000 7ff17ffff700 1 -- 172.21.15.60:0/1322875067 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287021 (secure 0 0 0) 0x7ff18c004520 con 0x7ff18406ace0 2023-07-13T22:20:59.416 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:59.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff18406ace0 msgr2=0x7ff18406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff18406ace0 0x7ff18406d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff1980d3b60 tx=0x7ff18c008030 comp rx=0 tx=0).stop 2023-07-13T22:20:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1980b8400 msgr2=0x7ff1981f30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1980b8400 0x7ff1981f30b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff18800c500 tx=0x7ff18800c8c0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 shutdown_connections 2023-07-13T22:20:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff18406ace0 0x7ff18406d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1980b8400 0x7ff1981f30b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1980d3750 0x7ff1980d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 --2- 172.21.15.60:0/1322875067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1980cfad0 0x7ff198150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 >> 172.21.15.60:0/1322875067 conn(0x7ff19806b390 msgr2=0x7ff1980d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 shutdown_connections 2023-07-13T22:20:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.433+0000 7ff19d9ff700 1 -- 172.21.15.60:0/1322875067 wait complete. 2023-07-13T22:20:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:20:59.435 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-07-13T22:20:59.118528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218480,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756240,"statfs":{"total":773094113280,"available":772870389760,"internally_reserved":0,"allocated":2932736,"data_stored":1792556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001495"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611623+0000","last_change":"2023-07-13T22:20:54.539241+0000","last_active":"2023-07-13T22:20:55.611623+0000","last_peered":"2023-07-13T22:20:55.611623+0000","last_clean":"2023-07-13T22:20:55.611623+0000","last_became_active":"2023-07-13T22:20:54.538987+0000","last_became_peered":"2023-07-13T22:20:54.538987+0000","last_unstale":"2023-07-13T22:20:55.611623+0000","last_undegraded":"2023-07-13T22:20:55.611623+0000","last_fullsized":"2023-07-13T22:20:55.611623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:34:22.186098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532509+0000","last_change":"2023-07-13T22:20:54.537349+0000","last_active":"2023-07-13T22:20:55.532509+0000","last_peered":"2023-07-13T22:20:55.532509+0000","last_clean":"2023-07-13T22:20:55.532509+0000","last_became_active":"2023-07-13T22:20:54.536260+0000","last_became_peered":"2023-07-13T22:20:54.536260+0000","last_unstale":"2023-07-13T22:20:55.532509+0000","last_undegraded":"2023-07-13T22:20:55.532509+0000","last_fullsized":"2023-07-13T22:20:55.532509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532790+0000","last_change":"2023-07-13T22:20:54.543864+0000","last_active":"2023-07-13T22:20:55.532790+0000","last_peered":"2023-07-13T22:20:55.532790+0000","last_clean":"2023-07-13T22:20:55.532790+0000","last_became_active":"2023-07-13T22:20:54.543344+0000","last_became_peered":"2023-07-13T22:20:54.543344+0000","last_unstale":"2023-07-13T22:20:55.532790+0000","last_undegraded":"2023-07-13T22:20:55.532790+0000","last_fullsized":"2023-07-13T22:20:55.532790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610982+0000","last_change":"2023-07-13T22:20:54.540616+0000","last_active":"2023-07-13T22:20:55.610982+0000","last_peered":"2023-07-13T22:20:55.610982+0000","last_clean":"2023-07-13T22:20:55.610982+0000","last_became_active":"2023-07-13T22:20:54.540326+0000","last_became_peered":"2023-07-13T22:20:54.540326+0000","last_unstale":"2023-07-13T22:20:55.610982+0000","last_undegraded":"2023-07-13T22:20:55.610982+0000","last_fullsized":"2023-07-13T22:20:55.610982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611292+0000","last_change":"2023-07-13T22:20:54.550022+0000","last_active":"2023-07-13T22:20:55.611292+0000","last_peered":"2023-07-13T22:20:55.611292+0000","last_clean":"2023-07-13T22:20:55.611292+0000","last_became_active":"2023-07-13T22:20:54.549249+0000","last_became_peered":"2023-07-13T22:20:54.549249+0000","last_unstale":"2023-07-13T22:20:55.611292+0000","last_undegraded":"2023-07-13T22:20:55.611292+0000","last_fullsized":"2023-07-13T22:20:55.611292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533320+0000","last_change":"2023-07-13T22:20:54.544989+0000","last_active":"2023-07-13T22:20:55.533320+0000","last_peered":"2023-07-13T22:20:55.533320+0000","last_clean":"2023-07-13T22:20:55.533320+0000","last_became_active":"2023-07-13T22:20:54.544719+0000","last_became_peered":"2023-07-13T22:20:54.544719+0000","last_unstale":"2023-07-13T22:20:55.533320+0000","last_undegraded":"2023-07-13T22:20:55.533320+0000","last_fullsized":"2023-07-13T22:20:55.533320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:55.546960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610444+0000","last_change":"2023-07-13T22:20:54.543266+0000","last_active":"2023-07-13T22:20:55.610444+0000","last_peered":"2023-07-13T22:20:55.610444+0000","last_clean":"2023-07-13T22:20:55.610444+0000","last_became_active":"2023-07-13T22:20:54.542870+0000","last_became_peered":"2023-07-13T22:20:54.542870+0000","last_unstale":"2023-07-13T22:20:55.610444+0000","last_undegraded":"2023-07-13T22:20:55.610444+0000","last_fullsized":"2023-07-13T22:20:55.610444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532392+0000","last_change":"2023-07-13T22:20:54.546394+0000","last_active":"2023-07-13T22:20:55.532392+0000","last_peered":"2023-07-13T22:20:55.532392+0000","last_clean":"2023-07-13T22:20:55.532392+0000","last_became_active":"2023-07-13T22:20:54.545682+0000","last_became_peered":"2023-07-13T22:20:54.545682+0000","last_unstale":"2023-07-13T22:20:55.532392+0000","last_undegraded":"2023-07-13T22:20:55.532392+0000","last_fullsized":"2023-07-13T22:20:55.532392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611695+0000","last_change":"2023-07-13T22:20:54.549925+0000","last_active":"2023-07-13T22:20:55.611695+0000","last_peered":"2023-07-13T22:20:55.611695+0000","last_clean":"2023-07-13T22:20:55.611695+0000","last_became_active":"2023-07-13T22:20:54.549641+0000","last_became_peered":"2023-07-13T22:20:54.549641+0000","last_unstale":"2023-07-13T22:20:55.611695+0000","last_undegraded":"2023-07-13T22:20:55.611695+0000","last_fullsized":"2023-07-13T22:20:55.611695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:45.329435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532781+0000","last_change":"2023-07-13T22:20:54.543479+0000","last_active":"2023-07-13T22:20:55.532781+0000","last_peered":"2023-07-13T22:20:55.532781+0000","last_clean":"2023-07-13T22:20:55.532781+0000","last_became_active":"2023-07-13T22:20:54.543143+0000","last_became_peered":"2023-07-13T22:20:54.543143+0000","last_unstale":"2023-07-13T22:20:55.532781+0000","last_undegraded":"2023-07-13T22:20:55.532781+0000","last_fullsized":"2023-07-13T22:20:55.532781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532989+0000","last_change":"2023-07-13T22:20:54.543686+0000","last_active":"2023-07-13T22:20:55.532989+0000","last_peered":"2023-07-13T22:20:55.532989+0000","last_clean":"2023-07-13T22:20:55.532989+0000","last_became_active":"2023-07-13T22:20:54.543001+0000","last_became_peered":"2023-07-13T22:20:54.543001+0000","last_unstale":"2023-07-13T22:20:55.532989+0000","last_undegraded":"2023-07-13T22:20:55.532989+0000","last_fullsized":"2023-07-13T22:20:55.532989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611996+0000","last_change":"2023-07-13T22:20:54.547637+0000","last_active":"2023-07-13T22:20:55.611996+0000","last_peered":"2023-07-13T22:20:55.611996+0000","last_clean":"2023-07-13T22:20:55.611996+0000","last_became_active":"2023-07-13T22:20:54.547280+0000","last_became_peered":"2023-07-13T22:20:54.547280+0000","last_unstale":"2023-07-13T22:20:55.611996+0000","last_undegraded":"2023-07-13T22:20:55.611996+0000","last_fullsized":"2023-07-13T22:20:55.611996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:18:24.767625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611891+0000","last_change":"2023-07-13T22:20:54.538682+0000","last_active":"2023-07-13T22:20:55.611891+0000","last_peered":"2023-07-13T22:20:55.611891+0000","last_clean":"2023-07-13T22:20:55.611891+0000","last_became_active":"2023-07-13T22:20:54.538356+0000","last_became_peered":"2023-07-13T22:20:54.538356+0000","last_unstale":"2023-07-13T22:20:55.611891+0000","last_undegraded":"2023-07-13T22:20:55.611891+0000","last_fullsized":"2023-07-13T22:20:55.611891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:51:10.800579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533030+0000","last_change":"2023-07-13T22:20:54.540925+0000","last_active":"2023-07-13T22:20:55.533030+0000","last_peered":"2023-07-13T22:20:55.533030+0000","last_clean":"2023-07-13T22:20:55.533030+0000","last_became_active":"2023-07-13T22:20:54.540646+0000","last_became_peered":"2023-07-13T22:20:54.540646+0000","last_unstale":"2023-07-13T22:20:55.533030+0000","last_undegraded":"2023-07-13T22:20:55.533030+0000","last_fullsized":"2023-07-13T22:20:55.533030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610864+0000","last_change":"2023-07-13T22:20:54.543844+0000","last_active":"2023-07-13T22:20:55.610864+0000","last_peered":"2023-07-13T22:20:55.610864+0000","last_clean":"2023-07-13T22:20:55.610864+0000","last_became_active":"2023-07-13T22:20:54.543527+0000","last_became_peered":"2023-07-13T22:20:54.543527+0000","last_unstale":"2023-07-13T22:20:55.610864+0000","last_undegraded":"2023-07-13T22:20:55.610864+0000","last_fullsized":"2023-07-13T22:20:55.610864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533022+0000","last_change":"2023-07-13T22:20:54.537914+0000","last_active":"2023-07-13T22:20:55.533022+0000","last_peered":"2023-07-13T22:20:55.533022+0000","last_clean":"2023-07-13T22:20:55.533022+0000","last_became_active":"2023-07-13T22:20:54.537562+0000","last_became_peered":"2023-07-13T22:20:54.537562+0000","last_unstale":"2023-07-13T22:20:55.533022+0000","last_undegraded":"2023-07-13T22:20:55.533022+0000","last_fullsized":"2023-07-13T22:20:55.533022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:26.867364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611835+0000","last_change":"2023-07-13T22:20:54.549793+0000","last_active":"2023-07-13T22:20:55.611835+0000","last_peered":"2023-07-13T22:20:55.611835+0000","last_clean":"2023-07-13T22:20:55.611835+0000","last_became_active":"2023-07-13T22:20:54.549473+0000","last_became_peered":"2023-07-13T22:20:54.549473+0000","last_unstale":"2023-07-13T22:20:55.611835+0000","last_undegraded":"2023-07-13T22:20:55.611835+0000","last_fullsized":"2023-07-13T22:20:55.611835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535307+0000","last_change":"2023-07-13T22:20:54.541942+0000","last_active":"2023-07-13T22:20:55.535307+0000","last_peered":"2023-07-13T22:20:55.535307+0000","last_clean":"2023-07-13T22:20:55.535307+0000","last_became_active":"2023-07-13T22:20:54.541314+0000","last_became_peered":"2023-07-13T22:20:54.541314+0000","last_unstale":"2023-07-13T22:20:55.535307+0000","last_undegraded":"2023-07-13T22:20:55.535307+0000","last_fullsized":"2023-07-13T22:20:55.535307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:10:09.528351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611873+0000","last_change":"2023-07-13T22:20:54.532924+0000","last_active":"2023-07-13T22:20:55.611873+0000","last_peered":"2023-07-13T22:20:55.611873+0000","last_clean":"2023-07-13T22:20:55.611873+0000","last_became_active":"2023-07-13T22:20:54.531937+0000","last_became_peered":"2023-07-13T22:20:54.531937+0000","last_unstale":"2023-07-13T22:20:55.611873+0000","last_undegraded":"2023-07-13T22:20:55.611873+0000","last_fullsized":"2023-07-13T22:20:55.611873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:20:04.410391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533904+0000","last_change":"2023-07-13T22:20:54.552416+0000","last_active":"2023-07-13T22:20:55.533904+0000","last_peered":"2023-07-13T22:20:55.533904+0000","last_clean":"2023-07-13T22:20:55.533904+0000","last_became_active":"2023-07-13T22:20:54.551516+0000","last_became_peered":"2023-07-13T22:20:54.551516+0000","last_unstale":"2023-07-13T22:20:55.533904+0000","last_undegraded":"2023-07-13T22:20:55.533904+0000","last_fullsized":"2023-07-13T22:20:55.533904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611325+0000","last_change":"2023-07-13T22:20:54.551898+0000","last_active":"2023-07-13T22:20:55.611325+0000","last_peered":"2023-07-13T22:20:55.611325+0000","last_clean":"2023-07-13T22:20:55.611325+0000","last_became_active":"2023-07-13T22:20:54.551538+0000","last_became_peered":"2023-07-13T22:20:54.551538+0000","last_unstale":"2023-07-13T22:20:55.611325+0000","last_undegraded":"2023-07-13T22:20:55.611325+0000","last_fullsized":"2023-07-13T22:20:55.611325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:38.898796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610399+0000","last_change":"2023-07-13T22:20:54.540580+0000","last_active":"2023-07-13T22:20:55.610399+0000","last_peered":"2023-07-13T22:20:55.610399+0000","last_clean":"2023-07-13T22:20:55.610399+0000","last_became_active":"2023-07-13T22:20:54.540289+0000","last_became_peered":"2023-07-13T22:20:54.540289+0000","last_unstale":"2023-07-13T22:20:55.610399+0000","last_undegraded":"2023-07-13T22:20:55.610399+0000","last_fullsized":"2023-07-13T22:20:55.610399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:43.402535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610169+0000","last_change":"2023-07-13T22:20:54.536952+0000","last_active":"2023-07-13T22:20:55.610169+0000","last_peered":"2023-07-13T22:20:55.610169+0000","last_clean":"2023-07-13T22:20:55.610169+0000","last_became_active":"2023-07-13T22:20:54.536696+0000","last_became_peered":"2023-07-13T22:20:54.536696+0000","last_unstale":"2023-07-13T22:20:55.610169+0000","last_undegraded":"2023-07-13T22:20:55.610169+0000","last_fullsized":"2023-07-13T22:20:55.610169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:26.867364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611257+0000","last_change":"2023-07-13T22:20:54.550043+0000","last_active":"2023-07-13T22:20:55.611257+0000","last_peered":"2023-07-13T22:20:55.611257+0000","last_clean":"2023-07-13T22:20:55.611257+0000","last_became_active":"2023-07-13T22:20:54.549754+0000","last_became_peered":"2023-07-13T22:20:54.549754+0000","last_unstale":"2023-07-13T22:20:55.611257+0000","last_undegraded":"2023-07-13T22:20:55.611257+0000","last_fullsized":"2023-07-13T22:20:55.611257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531264+0000","last_change":"2023-07-13T22:20:54.534540+0000","last_active":"2023-07-13T22:20:55.531264+0000","last_peered":"2023-07-13T22:20:55.531264+0000","last_clean":"2023-07-13T22:20:55.531264+0000","last_became_active":"2023-07-13T22:20:54.534251+0000","last_became_peered":"2023-07-13T22:20:54.534251+0000","last_unstale":"2023-07-13T22:20:55.531264+0000","last_undegraded":"2023-07-13T22:20:55.531264+0000","last_fullsized":"2023-07-13T22:20:55.531264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531735+0000","last_change":"2023-07-13T22:20:54.529221+0000","last_active":"2023-07-13T22:20:55.531735+0000","last_peered":"2023-07-13T22:20:55.531735+0000","last_clean":"2023-07-13T22:20:55.531735+0000","last_became_active":"2023-07-13T22:20:54.528934+0000","last_became_peered":"2023-07-13T22:20:54.528934+0000","last_unstale":"2023-07-13T22:20:55.531735+0000","last_undegraded":"2023-07-13T22:20:55.531735+0000","last_fullsized":"2023-07-13T22:20:55.531735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533014+0000","last_change":"2023-07-13T22:20:54.539287+0000","last_active":"2023-07-13T22:20:55.533014+0000","last_peered":"2023-07-13T22:20:55.533014+0000","last_clean":"2023-07-13T22:20:55.533014+0000","last_became_active":"2023-07-13T22:20:54.537991+0000","last_became_peered":"2023-07-13T22:20:54.537991+0000","last_unstale":"2023-07-13T22:20:55.533014+0000","last_undegraded":"2023-07-13T22:20:55.533014+0000","last_fullsized":"2023-07-13T22:20:55.533014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533580+0000","last_change":"2023-07-13T22:20:53.534188+0000","last_active":"2023-07-13T22:20:55.533580+0000","last_peered":"2023-07-13T22:20:55.533580+0000","last_clean":"2023-07-13T22:20:55.533580+0000","last_became_active":"2023-07-13T22:20:53.533888+0000","last_became_peered":"2023-07-13T22:20:53.533888+0000","last_unstale":"2023-07-13T22:20:55.533580+0000","last_undegraded":"2023-07-13T22:20:55.533580+0000","last_fullsized":"2023-07-13T22:20:55.533580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:55.228110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610249+0000","last_change":"2023-07-13T22:20:54.541751+0000","last_active":"2023-07-13T22:20:55.610249+0000","last_peered":"2023-07-13T22:20:55.610249+0000","last_clean":"2023-07-13T22:20:55.610249+0000","last_became_active":"2023-07-13T22:20:54.541506+0000","last_became_peered":"2023-07-13T22:20:54.541506+0000","last_unstale":"2023-07-13T22:20:55.610249+0000","last_undegraded":"2023-07-13T22:20:55.610249+0000","last_fullsized":"2023-07-13T22:20:55.610249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:55.546960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531119+0000","last_change":"2023-07-13T22:20:53.531325+0000","last_active":"2023-07-13T22:20:55.531119+0000","last_peered":"2023-07-13T22:20:55.531119+0000","last_clean":"2023-07-13T22:20:55.531119+0000","last_became_active":"2023-07-13T22:20:53.530929+0000","last_became_peered":"2023-07-13T22:20:53.530929+0000","last_unstale":"2023-07-13T22:20:55.531119+0000","last_undegraded":"2023-07-13T22:20:55.531119+0000","last_fullsized":"2023-07-13T22:20:55.531119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:07:16.260303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611424+0000","last_change":"2023-07-13T22:20:54.541835+0000","last_active":"2023-07-13T22:20:55.611424+0000","last_peered":"2023-07-13T22:20:55.611424+0000","last_clean":"2023-07-13T22:20:55.611424+0000","last_became_active":"2023-07-13T22:20:54.541371+0000","last_became_peered":"2023-07-13T22:20:54.541371+0000","last_unstale":"2023-07-13T22:20:55.611424+0000","last_undegraded":"2023-07-13T22:20:55.611424+0000","last_fullsized":"2023-07-13T22:20:55.611424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610264+0000","last_change":"2023-07-13T22:20:53.534081+0000","last_active":"2023-07-13T22:20:55.610264+0000","last_peered":"2023-07-13T22:20:55.610264+0000","last_clean":"2023-07-13T22:20:55.610264+0000","last_became_active":"2023-07-13T22:20:53.532985+0000","last_became_peered":"2023-07-13T22:20:53.532985+0000","last_unstale":"2023-07-13T22:20:55.610264+0000","last_undegraded":"2023-07-13T22:20:55.610264+0000","last_fullsized":"2023-07-13T22:20:55.610264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:16.293217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610466+0000","last_change":"2023-07-13T22:20:54.533387+0000","last_active":"2023-07-13T22:20:55.610466+0000","last_peered":"2023-07-13T22:20:55.610466+0000","last_clean":"2023-07-13T22:20:55.610466+0000","last_became_active":"2023-07-13T22:20:54.533023+0000","last_became_peered":"2023-07-13T22:20:54.533023+0000","last_unstale":"2023-07-13T22:20:55.610466+0000","last_undegraded":"2023-07-13T22:20:55.610466+0000","last_fullsized":"2023-07-13T22:20:55.610466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611957+0000","last_change":"2023-07-13T22:20:53.538156+0000","last_active":"2023-07-13T22:20:55.611957+0000","last_peered":"2023-07-13T22:20:55.611957+0000","last_clean":"2023-07-13T22:20:55.611957+0000","last_became_active":"2023-07-13T22:20:53.536726+0000","last_became_peered":"2023-07-13T22:20:53.536726+0000","last_unstale":"2023-07-13T22:20:55.611957+0000","last_undegraded":"2023-07-13T22:20:55.611957+0000","last_fullsized":"2023-07-13T22:20:55.611957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:25.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610113+0000","last_change":"2023-07-13T22:20:54.541258+0000","last_active":"2023-07-13T22:20:55.610113+0000","last_peered":"2023-07-13T22:20:55.610113+0000","last_clean":"2023-07-13T22:20:55.610113+0000","last_became_active":"2023-07-13T22:20:54.540986+0000","last_became_peered":"2023-07-13T22:20:54.540986+0000","last_unstale":"2023-07-13T22:20:55.610113+0000","last_undegraded":"2023-07-13T22:20:55.610113+0000","last_fullsized":"2023-07-13T22:20:55.610113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532649+0000","last_change":"2023-07-13T22:20:53.536025+0000","last_active":"2023-07-13T22:20:55.532649+0000","last_peered":"2023-07-13T22:20:55.532649+0000","last_clean":"2023-07-13T22:20:55.532649+0000","last_became_active":"2023-07-13T22:20:53.535697+0000","last_became_peered":"2023-07-13T22:20:53.535697+0000","last_unstale":"2023-07-13T22:20:55.532649+0000","last_undegraded":"2023-07-13T22:20:55.532649+0000","last_fullsized":"2023-07-13T22:20:55.532649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:53.812690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610659+0000","last_change":"2023-07-13T22:20:54.537549+0000","last_active":"2023-07-13T22:20:55.610659+0000","last_peered":"2023-07-13T22:20:55.610659+0000","last_clean":"2023-07-13T22:20:55.610659+0000","last_became_active":"2023-07-13T22:20:54.537260+0000","last_became_peered":"2023-07-13T22:20:54.537260+0000","last_unstale":"2023-07-13T22:20:55.610659+0000","last_undegraded":"2023-07-13T22:20:55.610659+0000","last_fullsized":"2023-07-13T22:20:55.610659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611165+0000","last_change":"2023-07-13T22:20:53.543492+0000","last_active":"2023-07-13T22:20:55.611165+0000","last_peered":"2023-07-13T22:20:55.611165+0000","last_clean":"2023-07-13T22:20:55.611165+0000","last_became_active":"2023-07-13T22:20:53.543265+0000","last_became_peered":"2023-07-13T22:20:53.543265+0000","last_unstale":"2023-07-13T22:20:55.611165+0000","last_undegraded":"2023-07-13T22:20:55.611165+0000","last_fullsized":"2023-07-13T22:20:55.611165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:35.694626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531212+0000","last_change":"2023-07-13T22:20:54.532326+0000","last_active":"2023-07-13T22:20:55.531212+0000","last_peered":"2023-07-13T22:20:55.531212+0000","last_clean":"2023-07-13T22:20:55.531212+0000","last_became_active":"2023-07-13T22:20:54.532144+0000","last_became_peered":"2023-07-13T22:20:54.532144+0000","last_unstale":"2023-07-13T22:20:55.531212+0000","last_undegraded":"2023-07-13T22:20:55.531212+0000","last_fullsized":"2023-07-13T22:20:55.531212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:24.401992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.534193+0000","last_change":"2023-07-13T22:20:53.531850+0000","last_active":"2023-07-13T22:20:55.534193+0000","last_peered":"2023-07-13T22:20:55.534193+0000","last_clean":"2023-07-13T22:20:55.534193+0000","last_became_active":"2023-07-13T22:20:53.531567+0000","last_became_peered":"2023-07-13T22:20:53.531567+0000","last_unstale":"2023-07-13T22:20:55.534193+0000","last_undegraded":"2023-07-13T22:20:55.534193+0000","last_fullsized":"2023-07-13T22:20:55.534193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:37.890575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533168+0000","last_change":"2023-07-13T22:20:54.552096+0000","last_active":"2023-07-13T22:20:55.533168+0000","last_peered":"2023-07-13T22:20:55.533168+0000","last_clean":"2023-07-13T22:20:55.533168+0000","last_became_active":"2023-07-13T22:20:54.551842+0000","last_became_peered":"2023-07-13T22:20:54.551842+0000","last_unstale":"2023-07-13T22:20:55.533168+0000","last_undegraded":"2023-07-13T22:20:55.533168+0000","last_fullsized":"2023-07-13T22:20:55.533168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:08.460983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610860+0000","last_change":"2023-07-13T22:20:53.538459+0000","last_active":"2023-07-13T22:20:55.610860+0000","last_peered":"2023-07-13T22:20:55.610860+0000","last_clean":"2023-07-13T22:20:55.610860+0000","last_became_active":"2023-07-13T22:20:53.538180+0000","last_became_peered":"2023-07-13T22:20:53.538180+0000","last_unstale":"2023-07-13T22:20:55.610860+0000","last_undegraded":"2023-07-13T22:20:55.610860+0000","last_fullsized":"2023-07-13T22:20:55.610860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:54.538739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610410+0000","last_change":"2023-07-13T22:20:54.542603+0000","last_active":"2023-07-13T22:20:55.610410+0000","last_peered":"2023-07-13T22:20:55.610410+0000","last_clean":"2023-07-13T22:20:55.610410+0000","last_became_active":"2023-07-13T22:20:54.542348+0000","last_became_peered":"2023-07-13T22:20:54.542348+0000","last_unstale":"2023-07-13T22:20:55.610410+0000","last_undegraded":"2023-07-13T22:20:55.610410+0000","last_fullsized":"2023-07-13T22:20:55.610410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:48.562581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610578+0000","last_change":"2023-07-13T22:20:53.539294+0000","last_active":"2023-07-13T22:20:55.610578+0000","last_peered":"2023-07-13T22:20:55.610578+0000","last_clean":"2023-07-13T22:20:55.610578+0000","last_became_active":"2023-07-13T22:20:53.538780+0000","last_became_peered":"2023-07-13T22:20:53.538780+0000","last_unstale":"2023-07-13T22:20:55.610578+0000","last_undegraded":"2023-07-13T22:20:55.610578+0000","last_fullsized":"2023-07-13T22:20:55.610578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:23.393770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533163+0000","last_change":"2023-07-13T22:20:54.538797+0000","last_active":"2023-07-13T22:20:55.533163+0000","last_peered":"2023-07-13T22:20:55.533163+0000","last_clean":"2023-07-13T22:20:55.533163+0000","last_became_active":"2023-07-13T22:20:54.537618+0000","last_became_peered":"2023-07-13T22:20:54.537618+0000","last_unstale":"2023-07-13T22:20:55.533163+0000","last_undegraded":"2023-07-13T22:20:55.533163+0000","last_fullsized":"2023-07-13T22:20:55.533163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610621+0000","last_change":"2023-07-13T22:20:53.540448+0000","last_active":"2023-07-13T22:20:55.610621+0000","last_peered":"2023-07-13T22:20:55.610621+0000","last_clean":"2023-07-13T22:20:55.610621+0000","last_became_active":"2023-07-13T22:20:53.540124+0000","last_became_peered":"2023-07-13T22:20:53.540124+0000","last_unstale":"2023-07-13T22:20:55.610621+0000","last_undegraded":"2023-07-13T22:20:55.610621+0000","last_fullsized":"2023-07-13T22:20:55.610621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611081+0000","last_change":"2023-07-13T22:20:54.552868+0000","last_active":"2023-07-13T22:20:55.611081+0000","last_peered":"2023-07-13T22:20:55.611081+0000","last_clean":"2023-07-13T22:20:55.611081+0000","last_became_active":"2023-07-13T22:20:54.552531+0000","last_became_peered":"2023-07-13T22:20:54.552531+0000","last_unstale":"2023-07-13T22:20:55.611081+0000","last_undegraded":"2023-07-13T22:20:55.611081+0000","last_fullsized":"2023-07-13T22:20:55.611081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610438+0000","last_change":"2023-07-13T22:20:53.533055+0000","last_active":"2023-07-13T22:20:55.610438+0000","last_peered":"2023-07-13T22:20:55.610438+0000","last_clean":"2023-07-13T22:20:55.610438+0000","last_became_active":"2023-07-13T22:20:53.532792+0000","last_became_peered":"2023-07-13T22:20:53.532792+0000","last_unstale":"2023-07-13T22:20:55.610438+0000","last_undegraded":"2023-07-13T22:20:55.610438+0000","last_fullsized":"2023-07-13T22:20:55.610438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:28.638041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610888+0000","last_change":"2023-07-13T22:20:54.536719+0000","last_active":"2023-07-13T22:20:55.610888+0000","last_peered":"2023-07-13T22:20:55.610888+0000","last_clean":"2023-07-13T22:20:55.610888+0000","last_became_active":"2023-07-13T22:20:54.535865+0000","last_became_peered":"2023-07-13T22:20:54.535865+0000","last_unstale":"2023-07-13T22:20:55.610888+0000","last_undegraded":"2023-07-13T22:20:55.610888+0000","last_fullsized":"2023-07-13T22:20:55.610888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:04:58.712616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610902+0000","last_change":"2023-07-13T22:20:53.537777+0000","last_active":"2023-07-13T22:20:55.610902+0000","last_peered":"2023-07-13T22:20:55.610902+0000","last_clean":"2023-07-13T22:20:55.610902+0000","last_became_active":"2023-07-13T22:20:53.537464+0000","last_became_peered":"2023-07-13T22:20:53.537464+0000","last_unstale":"2023-07-13T22:20:55.610902+0000","last_undegraded":"2023-07-13T22:20:55.610902+0000","last_fullsized":"2023-07-13T22:20:55.610902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:47.554360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532565+0000","last_change":"2023-07-13T22:20:54.536450+0000","last_active":"2023-07-13T22:20:55.532565+0000","last_peered":"2023-07-13T22:20:55.532565+0000","last_clean":"2023-07-13T22:20:55.532565+0000","last_became_active":"2023-07-13T22:20:54.536099+0000","last_became_peered":"2023-07-13T22:20:54.536099+0000","last_unstale":"2023-07-13T22:20:55.532565+0000","last_undegraded":"2023-07-13T22:20:55.532565+0000","last_fullsized":"2023-07-13T22:20:55.532565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610785+0000","last_change":"2023-07-13T22:20:53.535131+0000","last_active":"2023-07-13T22:20:55.610785+0000","last_peered":"2023-07-13T22:20:55.610785+0000","last_clean":"2023-07-13T22:20:55.610785+0000","last_became_active":"2023-07-13T22:20:53.534737+0000","last_became_peered":"2023-07-13T22:20:53.534737+0000","last_unstale":"2023-07-13T22:20:55.610785+0000","last_undegraded":"2023-07-13T22:20:55.610785+0000","last_fullsized":"2023-07-13T22:20:55.610785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:16.293217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533196+0000","last_change":"2023-07-13T22:20:54.546160+0000","last_active":"2023-07-13T22:20:55.533196+0000","last_peered":"2023-07-13T22:20:55.533196+0000","last_clean":"2023-07-13T22:20:55.533196+0000","last_became_active":"2023-07-13T22:20:54.545713+0000","last_became_peered":"2023-07-13T22:20:54.545713+0000","last_unstale":"2023-07-13T22:20:55.533196+0000","last_undegraded":"2023-07-13T22:20:55.533196+0000","last_fullsized":"2023-07-13T22:20:55.533196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:17.301438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610989+0000","last_change":"2023-07-13T22:20:53.536620+0000","last_active":"2023-07-13T22:20:55.610989+0000","last_peered":"2023-07-13T22:20:55.610989+0000","last_clean":"2023-07-13T22:20:55.610989+0000","last_became_active":"2023-07-13T22:20:53.536322+0000","last_became_peered":"2023-07-13T22:20:53.536322+0000","last_unstale":"2023-07-13T22:20:55.610989+0000","last_undegraded":"2023-07-13T22:20:55.610989+0000","last_fullsized":"2023-07-13T22:20:55.610989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:55.228110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610640+0000","last_change":"2023-07-13T22:20:54.544907+0000","last_active":"2023-07-13T22:20:55.610640+0000","last_peered":"2023-07-13T22:20:55.610640+0000","last_clean":"2023-07-13T22:20:55.610640+0000","last_became_active":"2023-07-13T22:20:54.544596+0000","last_became_peered":"2023-07-13T22:20:54.544596+0000","last_unstale":"2023-07-13T22:20:55.610640+0000","last_undegraded":"2023-07-13T22:20:55.610640+0000","last_fullsized":"2023-07-13T22:20:55.610640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533567+0000","last_change":"2023-07-13T22:20:53.530981+0000","last_active":"2023-07-13T22:20:55.533567+0000","last_peered":"2023-07-13T22:20:55.533567+0000","last_clean":"2023-07-13T22:20:55.533567+0000","last_became_active":"2023-07-13T22:20:53.530568+0000","last_became_peered":"2023-07-13T22:20:53.530568+0000","last_unstale":"2023-07-13T22:20:55.533567+0000","last_undegraded":"2023-07-13T22:20:55.533567+0000","last_fullsized":"2023-07-13T22:20:55.533567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610942+0000","last_change":"2023-07-13T22:20:54.540498+0000","last_active":"2023-07-13T22:20:55.610942+0000","last_peered":"2023-07-13T22:20:55.610942+0000","last_clean":"2023-07-13T22:20:55.610942+0000","last_became_active":"2023-07-13T22:20:54.540161+0000","last_became_peered":"2023-07-13T22:20:54.540161+0000","last_unstale":"2023-07-13T22:20:55.610942+0000","last_undegraded":"2023-07-13T22:20:55.610942+0000","last_fullsized":"2023-07-13T22:20:55.610942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:12.599393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610764+0000","last_change":"2023-07-13T22:20:53.533470+0000","last_active":"2023-07-13T22:20:55.610764+0000","last_peered":"2023-07-13T22:20:55.610764+0000","last_clean":"2023-07-13T22:20:55.610764+0000","last_became_active":"2023-07-13T22:20:53.532854+0000","last_became_peered":"2023-07-13T22:20:53.532854+0000","last_unstale":"2023-07-13T22:20:55.610764+0000","last_undegraded":"2023-07-13T22:20:55.610764+0000","last_fullsized":"2023-07-13T22:20:55.610764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:37.890575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.534049+0000","last_change":"2023-07-13T22:20:54.552213+0000","last_active":"2023-07-13T22:20:55.534049+0000","last_peered":"2023-07-13T22:20:55.534049+0000","last_clean":"2023-07-13T22:20:55.534049+0000","last_became_active":"2023-07-13T22:20:54.548415+0000","last_became_peered":"2023-07-13T22:20:54.548415+0000","last_unstale":"2023-07-13T22:20:55.534049+0000","last_undegraded":"2023-07-13T22:20:55.534049+0000","last_fullsized":"2023-07-13T22:20:55.534049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:50.171609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610542+0000","last_change":"2023-07-13T22:20:53.529929+0000","last_active":"2023-07-13T22:20:55.610542+0000","last_peered":"2023-07-13T22:20:55.610542+0000","last_clean":"2023-07-13T22:20:55.610542+0000","last_became_active":"2023-07-13T22:20:53.529645+0000","last_became_peered":"2023-07-13T22:20:53.529645+0000","last_unstale":"2023-07-13T22:20:55.610542+0000","last_undegraded":"2023-07-13T22:20:55.610542+0000","last_fullsized":"2023-07-13T22:20:55.610542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:53.812690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535287+0000","last_change":"2023-07-13T22:20:54.550835+0000","last_active":"2023-07-13T22:20:55.535287+0000","last_peered":"2023-07-13T22:20:55.535287+0000","last_clean":"2023-07-13T22:20:55.535287+0000","last_became_active":"2023-07-13T22:20:54.547187+0000","last_became_peered":"2023-07-13T22:20:54.547187+0000","last_unstale":"2023-07-13T22:20:55.535287+0000","last_undegraded":"2023-07-13T22:20:55.535287+0000","last_fullsized":"2023-07-13T22:20:55.535287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:04:58.712616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533482+0000","last_change":"2023-07-13T22:20:53.534667+0000","last_active":"2023-07-13T22:20:55.533482+0000","last_peered":"2023-07-13T22:20:55.533482+0000","last_clean":"2023-07-13T22:20:55.533482+0000","last_became_active":"2023-07-13T22:20:53.534433+0000","last_became_peered":"2023-07-13T22:20:53.534433+0000","last_unstale":"2023-07-13T22:20:55.533482+0000","last_undegraded":"2023-07-13T22:20:55.533482+0000","last_fullsized":"2023-07-13T22:20:55.533482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:28.070264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535373+0000","last_change":"2023-07-13T22:20:54.550679+0000","last_active":"2023-07-13T22:20:55.535373+0000","last_peered":"2023-07-13T22:20:55.535373+0000","last_clean":"2023-07-13T22:20:55.535373+0000","last_became_active":"2023-07-13T22:20:54.546483+0000","last_became_peered":"2023-07-13T22:20:54.546483+0000","last_unstale":"2023-07-13T22:20:55.535373+0000","last_undegraded":"2023-07-13T22:20:55.535373+0000","last_fullsized":"2023-07-13T22:20:55.535373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:16:54.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610029+0000","last_change":"2023-07-13T22:20:53.531815+0000","last_active":"2023-07-13T22:20:55.610029+0000","last_peered":"2023-07-13T22:20:55.610029+0000","last_clean":"2023-07-13T22:20:55.610029+0000","last_became_active":"2023-07-13T22:20:53.531551+0000","last_became_peered":"2023-07-13T22:20:53.531551+0000","last_unstale":"2023-07-13T22:20:55.610029+0000","last_undegraded":"2023-07-13T22:20:55.610029+0000","last_fullsized":"2023-07-13T22:20:55.610029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531764+0000","last_change":"2023-07-13T22:20:54.542085+0000","last_active":"2023-07-13T22:20:55.531764+0000","last_peered":"2023-07-13T22:20:55.531764+0000","last_clean":"2023-07-13T22:20:55.531764+0000","last_became_active":"2023-07-13T22:20:54.541749+0000","last_became_peered":"2023-07-13T22:20:54.541749+0000","last_unstale":"2023-07-13T22:20:55.531764+0000","last_undegraded":"2023-07-13T22:20:55.531764+0000","last_fullsized":"2023-07-13T22:20:55.531764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610668+0000","last_change":"2023-07-13T22:20:53.539618+0000","last_active":"2023-07-13T22:20:55.610668+0000","last_peered":"2023-07-13T22:20:55.610668+0000","last_clean":"2023-07-13T22:20:55.610668+0000","last_became_active":"2023-07-13T22:20:53.539014+0000","last_became_peered":"2023-07-13T22:20:53.539014+0000","last_unstale":"2023-07-13T22:20:55.610668+0000","last_undegraded":"2023-07-13T22:20:55.610668+0000","last_fullsized":"2023-07-13T22:20:55.610668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:49.163388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535108+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:20:55.535108+0000","last_peered":"2023-07-13T22:20:55.535108+0000","last_clean":"2023-07-13T22:20:55.535108+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:20:55.535108+0000","last_undegraded":"2023-07-13T22:20:55.535108+0000","last_fullsized":"2023-07-13T22:20:55.535108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:11.435573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.609968+0000","last_change":"2023-07-13T22:20:54.538149+0000","last_active":"2023-07-13T22:20:55.609968+0000","last_peered":"2023-07-13T22:20:55.609968+0000","last_clean":"2023-07-13T22:20:55.609968+0000","last_became_active":"2023-07-13T22:20:54.537883+0000","last_became_peered":"2023-07-13T22:20:54.537883+0000","last_unstale":"2023-07-13T22:20:55.609968+0000","last_undegraded":"2023-07-13T22:20:55.609968+0000","last_fullsized":"2023-07-13T22:20:55.609968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610497+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:20:55.610497+0000","last_peered":"2023-07-13T22:20:55.610497+0000","last_clean":"2023-07-13T22:20:55.610497+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:20:55.610497+0000","last_undegraded":"2023-07-13T22:20:55.610497+0000","last_fullsized":"2023-07-13T22:20:55.610497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:49.435942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611546+0000","last_change":"2023-07-13T22:20:53.541916+0000","last_active":"2023-07-13T22:20:55.611546+0000","last_peered":"2023-07-13T22:20:55.611546+0000","last_clean":"2023-07-13T22:20:55.611546+0000","last_became_active":"2023-07-13T22:20:53.541602+0000","last_became_peered":"2023-07-13T22:20:53.541602+0000","last_unstale":"2023-07-13T22:20:55.611546+0000","last_undegraded":"2023-07-13T22:20:55.611546+0000","last_fullsized":"2023-07-13T22:20:55.611546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:50.801197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533347+0000","last_change":"2023-07-13T22:20:54.545955+0000","last_active":"2023-07-13T22:20:55.533347+0000","last_peered":"2023-07-13T22:20:55.533347+0000","last_clean":"2023-07-13T22:20:55.533347+0000","last_became_active":"2023-07-13T22:20:54.545361+0000","last_became_peered":"2023-07-13T22:20:54.545361+0000","last_unstale":"2023-07-13T22:20:55.533347+0000","last_undegraded":"2023-07-13T22:20:55.533347+0000","last_fullsized":"2023-07-13T22:20:55.533347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531064+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:20:55.531064+0000","last_peered":"2023-07-13T22:20:55.531064+0000","last_clean":"2023-07-13T22:20:55.531064+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:20:55.531064+0000","last_undegraded":"2023-07-13T22:20:55.531064+0000","last_fullsized":"2023-07-13T22:20:55.531064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:15:56.981684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611242+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:20:55.611242+0000","last_peered":"2023-07-13T22:20:55.611242+0000","last_clean":"2023-07-13T22:20:55.611242+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:20:55.611242+0000","last_undegraded":"2023-07-13T22:20:55.611242+0000","last_fullsized":"2023-07-13T22:20:55.611242+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:44.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610851+0000","last_change":"2023-07-13T22:20:53.537543+0000","last_active":"2023-07-13T22:20:55.610851+0000","last_peered":"2023-07-13T22:20:55.610851+0000","last_clean":"2023-07-13T22:20:55.610851+0000","last_became_active":"2023-07-13T22:20:53.537125+0000","last_became_peered":"2023-07-13T22:20:53.537125+0000","last_unstale":"2023-07-13T22:20:55.610851+0000","last_undegraded":"2023-07-13T22:20:55.610851+0000","last_fullsized":"2023-07-13T22:20:55.610851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:48.574628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611216+0000","last_change":"2023-07-13T22:20:54.541651+0000","last_active":"2023-07-13T22:20:55.611216+0000","last_peered":"2023-07-13T22:20:55.611216+0000","last_clean":"2023-07-13T22:20:55.611216+0000","last_became_active":"2023-07-13T22:20:54.541059+0000","last_became_peered":"2023-07-13T22:20:54.541059+0000","last_unstale":"2023-07-13T22:20:55.611216+0000","last_undegraded":"2023-07-13T22:20:55.611216+0000","last_fullsized":"2023-07-13T22:20:55.611216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:54.820912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610481+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:20:55.610481+0000","last_peered":"2023-07-13T22:20:55.610481+0000","last_clean":"2023-07-13T22:20:55.610481+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:20:55.610481+0000","last_undegraded":"2023-07-13T22:20:55.610481+0000","last_fullsized":"2023-07-13T22:20:55.610481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:20.066975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.530855+0000","last_change":"2023-07-13T22:20:53.533784+0000","last_active":"2023-07-13T22:20:55.530855+0000","last_peered":"2023-07-13T22:20:55.530855+0000","last_clean":"2023-07-13T22:20:55.530855+0000","last_became_active":"2023-07-13T22:20:53.533270+0000","last_became_peered":"2023-07-13T22:20:53.533270+0000","last_unstale":"2023-07-13T22:20:55.530855+0000","last_undegraded":"2023-07-13T22:20:55.530855+0000","last_fullsized":"2023-07-13T22:20:55.530855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:29:28.555785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531674+0000","last_change":"2023-07-13T22:20:54.529627+0000","last_active":"2023-07-13T22:20:55.531674+0000","last_peered":"2023-07-13T22:20:55.531674+0000","last_clean":"2023-07-13T22:20:55.531674+0000","last_became_active":"2023-07-13T22:20:54.528020+0000","last_became_peered":"2023-07-13T22:20:54.528020+0000","last_unstale":"2023-07-13T22:20:55.531674+0000","last_undegraded":"2023-07-13T22:20:55.531674+0000","last_fullsized":"2023-07-13T22:20:55.531674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:54.820912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610893+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:20:55.610893+0000","last_peered":"2023-07-13T22:20:55.610893+0000","last_clean":"2023-07-13T22:20:55.610893+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:20:55.610893+0000","last_undegraded":"2023-07-13T22:20:55.610893+0000","last_fullsized":"2023-07-13T22:20:55.610893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:22.278097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610262+0000","last_change":"2023-07-13T22:20:53.535232+0000","last_active":"2023-07-13T22:20:55.610262+0000","last_peered":"2023-07-13T22:20:55.610262+0000","last_clean":"2023-07-13T22:20:55.610262+0000","last_became_active":"2023-07-13T22:20:53.533935+0000","last_became_peered":"2023-07-13T22:20:53.533935+0000","last_unstale":"2023-07-13T22:20:55.610262+0000","last_undegraded":"2023-07-13T22:20:55.610262+0000","last_fullsized":"2023-07-13T22:20:55.610262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610591+0000","last_change":"2023-07-13T22:20:54.549217+0000","last_active":"2023-07-13T22:20:55.610591+0000","last_peered":"2023-07-13T22:20:55.610591+0000","last_clean":"2023-07-13T22:20:55.610591+0000","last_became_active":"2023-07-13T22:20:54.548902+0000","last_became_peered":"2023-07-13T22:20:54.548902+0000","last_unstale":"2023-07-13T22:20:55.610591+0000","last_undegraded":"2023-07-13T22:20:55.610591+0000","last_fullsized":"2023-07-13T22:20:55.610591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532394+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:20:55.532394+0000","last_peered":"2023-07-13T22:20:55.532394+0000","last_clean":"2023-07-13T22:20:55.532394+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:20:55.532394+0000","last_undegraded":"2023-07-13T22:20:55.532394+0000","last_fullsized":"2023-07-13T22:20:55.532394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:05.799004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532935+0000","last_change":"2023-07-13T22:20:53.537640+0000","last_active":"2023-07-13T22:20:55.532935+0000","last_peered":"2023-07-13T22:20:55.532935+0000","last_clean":"2023-07-13T22:20:55.532935+0000","last_became_active":"2023-07-13T22:20:53.537351+0000","last_became_peered":"2023-07-13T22:20:53.537351+0000","last_unstale":"2023-07-13T22:20:55.532935+0000","last_undegraded":"2023-07-13T22:20:55.532935+0000","last_fullsized":"2023-07-13T22:20:55.532935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531434+0000","last_change":"2023-07-13T22:20:54.530546+0000","last_active":"2023-07-13T22:20:55.531434+0000","last_peered":"2023-07-13T22:20:55.531434+0000","last_clean":"2023-07-13T22:20:55.531434+0000","last_became_active":"2023-07-13T22:20:54.530234+0000","last_became_peered":"2023-07-13T22:20:54.530234+0000","last_unstale":"2023-07-13T22:20:55.531434+0000","last_undegraded":"2023-07-13T22:20:55.531434+0000","last_fullsized":"2023-07-13T22:20:55.531434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610282+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:20:55.610282+0000","last_peered":"2023-07-13T22:20:55.610282+0000","last_clean":"2023-07-13T22:20:55.610282+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:20:55.610282+0000","last_undegraded":"2023-07-13T22:20:55.610282+0000","last_fullsized":"2023-07-13T22:20:55.610282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:05.799004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531292+0000","last_change":"2023-07-13T22:20:53.533697+0000","last_active":"2023-07-13T22:20:55.531292+0000","last_peered":"2023-07-13T22:20:55.531292+0000","last_clean":"2023-07-13T22:20:55.531292+0000","last_became_active":"2023-07-13T22:20:53.533261+0000","last_became_peered":"2023-07-13T22:20:53.533261+0000","last_unstale":"2023-07-13T22:20:55.531292+0000","last_undegraded":"2023-07-13T22:20:55.531292+0000","last_fullsized":"2023-07-13T22:20:55.531292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:11.591172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611449+0000","last_change":"2023-07-13T22:20:54.537844+0000","last_active":"2023-07-13T22:20:55.611449+0000","last_peered":"2023-07-13T22:20:55.611449+0000","last_clean":"2023-07-13T22:20:55.611449+0000","last_became_active":"2023-07-13T22:20:54.537526+0000","last_became_peered":"2023-07-13T22:20:54.537526+0000","last_unstale":"2023-07-13T22:20:55.611449+0000","last_undegraded":"2023-07-13T22:20:55.611449+0000","last_fullsized":"2023-07-13T22:20:55.611449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610286+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:20:55.610286+0000","last_peered":"2023-07-13T22:20:55.610286+0000","last_clean":"2023-07-13T22:20:55.610286+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:20:55.610286+0000","last_undegraded":"2023-07-13T22:20:55.610286+0000","last_fullsized":"2023-07-13T22:20:55.610286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:17.601603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533295+0000","last_change":"2023-07-13T22:20:53.532273+0000","last_active":"2023-07-13T22:20:55.533295+0000","last_peered":"2023-07-13T22:20:55.533295+0000","last_clean":"2023-07-13T22:20:55.533295+0000","last_became_active":"2023-07-13T22:20:53.531699+0000","last_became_peered":"2023-07-13T22:20:53.531699+0000","last_unstale":"2023-07-13T22:20:55.533295+0000","last_undegraded":"2023-07-13T22:20:55.533295+0000","last_fullsized":"2023-07-13T22:20:55.533295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531599+0000","last_change":"2023-07-13T22:20:54.530026+0000","last_active":"2023-07-13T22:20:55.531599+0000","last_peered":"2023-07-13T22:20:55.531599+0000","last_clean":"2023-07-13T22:20:55.531599+0000","last_became_active":"2023-07-13T22:20:54.528217+0000","last_became_peered":"2023-07-13T22:20:54.528217+0000","last_unstale":"2023-07-13T22:20:55.531599+0000","last_undegraded":"2023-07-13T22:20:55.531599+0000","last_fullsized":"2023-07-13T22:20:55.531599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611152+0000","last_change":"2023-07-13T22:20:53.541066+0000","last_active":"2023-07-13T22:20:55.611152+0000","last_peered":"2023-07-13T22:20:55.611152+0000","last_clean":"2023-07-13T22:20:55.611152+0000","last_became_active":"2023-07-13T22:20:53.540748+0000","last_became_peered":"2023-07-13T22:20:53.540748+0000","last_unstale":"2023-07-13T22:20:55.611152+0000","last_undegraded":"2023-07-13T22:20:55.611152+0000","last_fullsized":"2023-07-13T22:20:55.611152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:25.360393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.612132+0000","last_change":"2023-07-13T22:20:54.531788+0000","last_active":"2023-07-13T22:20:55.612132+0000","last_peered":"2023-07-13T22:20:55.612132+0000","last_clean":"2023-07-13T22:20:55.612132+0000","last_became_active":"2023-07-13T22:20:54.531396+0000","last_became_peered":"2023-07-13T22:20:54.531396+0000","last_unstale":"2023-07-13T22:20:55.612132+0000","last_undegraded":"2023-07-13T22:20:55.612132+0000","last_fullsized":"2023-07-13T22:20:55.612132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:45:27.037988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533306+0000","last_change":"2023-07-13T22:20:53.530819+0000","last_active":"2023-07-13T22:20:55.533306+0000","last_peered":"2023-07-13T22:20:55.533306+0000","last_clean":"2023-07-13T22:20:55.533306+0000","last_became_active":"2023-07-13T22:20:53.530223+0000","last_became_peered":"2023-07-13T22:20:53.530223+0000","last_unstale":"2023-07-13T22:20:55.533306+0000","last_undegraded":"2023-07-13T22:20:55.533306+0000","last_fullsized":"2023-07-13T22:20:55.533306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:15:58.999140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532780+0000","last_change":"2023-07-13T22:20:54.551556+0000","last_active":"2023-07-13T22:20:55.532780+0000","last_peered":"2023-07-13T22:20:55.532780+0000","last_clean":"2023-07-13T22:20:55.532780+0000","last_became_active":"2023-07-13T22:20:54.547048+0000","last_became_peered":"2023-07-13T22:20:54.547048+0000","last_unstale":"2023-07-13T22:20:55.532780+0000","last_undegraded":"2023-07-13T22:20:55.532780+0000","last_fullsized":"2023-07-13T22:20:55.532780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533441+0000","last_change":"2023-07-13T22:20:53.532040+0000","last_active":"2023-07-13T22:20:55.533441+0000","last_peered":"2023-07-13T22:20:55.533441+0000","last_clean":"2023-07-13T22:20:55.533441+0000","last_became_active":"2023-07-13T22:20:53.531414+0000","last_became_peered":"2023-07-13T22:20:53.531414+0000","last_unstale":"2023-07-13T22:20:55.533441+0000","last_undegraded":"2023-07-13T22:20:55.533441+0000","last_fullsized":"2023-07-13T22:20:55.533441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:25.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533812+0000","last_change":"2023-07-13T22:20:54.551351+0000","last_active":"2023-07-13T22:20:55.533812+0000","last_peered":"2023-07-13T22:20:55.533812+0000","last_clean":"2023-07-13T22:20:55.533812+0000","last_became_active":"2023-07-13T22:20:54.550932+0000","last_became_peered":"2023-07-13T22:20:54.550932+0000","last_unstale":"2023-07-13T22:20:55.533812+0000","last_undegraded":"2023-07-13T22:20:55.533812+0000","last_fullsized":"2023-07-13T22:20:55.533812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:36.702847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610605+0000","last_change":"2023-07-13T22:20:53.530713+0000","last_active":"2023-07-13T22:20:55.610605+0000","last_peered":"2023-07-13T22:20:55.610605+0000","last_clean":"2023-07-13T22:20:55.610605+0000","last_became_active":"2023-07-13T22:20:53.530419+0000","last_became_peered":"2023-07-13T22:20:53.530419+0000","last_unstale":"2023-07-13T22:20:55.610605+0000","last_undegraded":"2023-07-13T22:20:55.610605+0000","last_fullsized":"2023-07-13T22:20:55.610605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:35.694626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610476+0000","last_change":"2023-07-13T22:20:54.541258+0000","last_active":"2023-07-13T22:20:55.610476+0000","last_peered":"2023-07-13T22:20:55.610476+0000","last_clean":"2023-07-13T22:20:55.610476+0000","last_became_active":"2023-07-13T22:20:54.540610+0000","last_became_peered":"2023-07-13T22:20:54.540610+0000","last_unstale":"2023-07-13T22:20:55.610476+0000","last_undegraded":"2023-07-13T22:20:55.610476+0000","last_fullsized":"2023-07-13T22:20:55.610476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:29.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.612057+0000","last_change":"2023-07-13T22:20:53.538279+0000","last_active":"2023-07-13T22:20:55.612057+0000","last_peered":"2023-07-13T22:20:55.612057+0000","last_clean":"2023-07-13T22:20:55.612057+0000","last_became_active":"2023-07-13T22:20:53.537651+0000","last_became_peered":"2023-07-13T22:20:53.537651+0000","last_unstale":"2023-07-13T22:20:55.612057+0000","last_undegraded":"2023-07-13T22:20:55.612057+0000","last_fullsized":"2023-07-13T22:20:55.612057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532866+0000","last_change":"2023-07-13T22:20:54.541753+0000","last_active":"2023-07-13T22:20:55.532866+0000","last_peered":"2023-07-13T22:20:55.532866+0000","last_clean":"2023-07-13T22:20:55.532866+0000","last_became_active":"2023-07-13T22:20:54.540946+0000","last_became_peered":"2023-07-13T22:20:54.540946+0000","last_unstale":"2023-07-13T22:20:55.532866+0000","last_undegraded":"2023-07-13T22:20:55.532866+0000","last_fullsized":"2023-07-13T22:20:55.532866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610348+0000","last_change":"2023-07-13T22:20:54.542872+0000","last_active":"2023-07-13T22:20:55.610348+0000","last_peered":"2023-07-13T22:20:55.610348+0000","last_clean":"2023-07-13T22:20:55.610348+0000","last_became_active":"2023-07-13T22:20:54.542534+0000","last_became_peered":"2023-07-13T22:20:54.542534+0000","last_unstale":"2023-07-13T22:20:55.610348+0000","last_undegraded":"2023-07-13T22:20:55.610348+0000","last_fullsized":"2023-07-13T22:20:55.610348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610319+0000","last_change":"2023-07-13T22:20:54.541488+0000","last_active":"2023-07-13T22:20:55.610319+0000","last_peered":"2023-07-13T22:20:55.610319+0000","last_clean":"2023-07-13T22:20:55.610319+0000","last_became_active":"2023-07-13T22:20:54.541003+0000","last_became_peered":"2023-07-13T22:20:54.541003+0000","last_unstale":"2023-07-13T22:20:55.610319+0000","last_undegraded":"2023-07-13T22:20:55.610319+0000","last_fullsized":"2023-07-13T22:20:55.610319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533452+0000","last_change":"2023-07-13T22:20:54.552460+0000","last_active":"2023-07-13T22:20:55.533452+0000","last_peered":"2023-07-13T22:20:55.533452+0000","last_clean":"2023-07-13T22:20:55.533452+0000","last_became_active":"2023-07-13T22:20:54.552208+0000","last_became_peered":"2023-07-13T22:20:54.552208+0000","last_unstale":"2023-07-13T22:20:55.533452+0000","last_undegraded":"2023-07-13T22:20:55.533452+0000","last_fullsized":"2023-07-13T22:20:55.533452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:48.562581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610888+0000","last_change":"2023-07-13T22:20:54.535876+0000","last_active":"2023-07-13T22:20:55.610888+0000","last_peered":"2023-07-13T22:20:55.610888+0000","last_clean":"2023-07-13T22:20:55.610888+0000","last_became_active":"2023-07-13T22:20:54.535570+0000","last_became_peered":"2023-07-13T22:20:54.535570+0000","last_unstale":"2023-07-13T22:20:55.610888+0000","last_undegraded":"2023-07-13T22:20:55.610888+0000","last_fullsized":"2023-07-13T22:20:55.610888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:18.235962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531178+0000","last_change":"2023-07-13T22:20:54.535911+0000","last_active":"2023-07-13T22:20:55.531178+0000","last_peered":"2023-07-13T22:20:55.531178+0000","last_clean":"2023-07-13T22:20:55.531178+0000","last_became_active":"2023-07-13T22:20:54.535593+0000","last_became_peered":"2023-07-13T22:20:54.535593+0000","last_unstale":"2023-07-13T22:20:55.531178+0000","last_undegraded":"2023-07-13T22:20:55.531178+0000","last_fullsized":"2023-07-13T22:20:55.531178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:43.402535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":671608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":72807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":72403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":200704,"data_stored":71821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":81080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":73459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:20:59.442 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:20:59.442 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-13T22:20:59.443 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-13T22:20:59.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-13T22:20:59.443 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-13T22:20:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.693+0000 7f0bc01ff700 1 Processor -- start 2023-07-13T22:20:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.694+0000 7f0bc01ff700 1 -- start start 2023-07-13T22:20:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.695+0000 7f0bc01ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b8400 0x7f0bb80cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.695+0000 7f0bc01ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb80d0ac0 con 0x7f0bb80cfca0 2023-07-13T22:20:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.695+0000 7f0bc01ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb80b87d0 con 0x7f0bb80d3750 2023-07-13T22:20:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.695+0000 7f0bc01ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb80b4eb0 con 0x7f0bb80b8400 2023-07-13T22:20:59.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.695+0000 7f0bbe79c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0bb80cfca0 0x7f0bb80d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39534/0 (socket says 172.21.15.60:39534) 2023-07-13T22:20:59.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.695+0000 7f0bbe79c700 1 -- 172.21.15.60:0/3305324302 learned_addr learned my addr 172.21.15.60:0/3305324302 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:20:59.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.695+0000 7f0bbdf9b700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b8400 0x7f0bb80cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.695+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022864270 0 0) 0x7f0bb80d0ac0 con 0x7f0bb80cfca0 2023-07-13T22:20:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.695+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ba40035b0 con 0x7f0bb80cfca0 2023-07-13T22:20:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1348539474 0 0) 0x7f0ba40035b0 con 0x7f0bb80cfca0 2023-07-13T22:20:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b8400 msgr2=0x7f0bb80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b8400 0x7f0bb80cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 >> v1:172.21.15.105:6789/0 conn(0x7f0bb80d3750 legacy=0x7f0bb80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb80b5190 con 0x7f0bb80cfca0 2023-07-13T22:20:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f0bb00019f0 con 0x7f0bb80cfca0 2023-07-13T22:20:59.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 >> v1:172.21.15.60:6789/0 conn(0x7f0bb80cfca0 legacy=0x7f0bb80d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b8400 0x7f0ba4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0ba4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4004d90 0x7f0ba4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb80b4eb0 con 0x7f0ba4003f70 2023-07-13T22:20:59.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba400bb50 con 0x7f0ba4004d90 2023-07-13T22:20:59.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba400bd10 con 0x7f0bb80b8400 2023-07-13T22:20:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbd79a700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0ba4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbdf9b700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b8400 0x7f0ba4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.696+0000 7f0bbe79c700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4004d90 0x7f0ba4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.697+0000 7f0bbd79a700 1 -- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b8400 msgr2=0x7f0ba4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.697+0000 7f0bbd79a700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b8400 0x7f0ba4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.697+0000 7f0bbd79a700 1 -- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4004d90 msgr2=0x7f0ba4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.697+0000 7f0bbd79a700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4004d90 0x7f0ba4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.697+0000 7f0bbd79a700 1 -- 172.21.15.60:0/3305324302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40035b0 con 0x7f0ba4003f70 2023-07-13T22:20:59.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.697+0000 7f0bbd79a700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0ba4004850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8007fd0 tx=0x7f0ba4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615d906405f09506 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:59.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.697+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8012620 con 0x7f0ba4003f70 2023-07-13T22:20:59.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.697+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba8012780 con 0x7f0ba4003f70 2023-07-13T22:20:59.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.697+0000 7f0bbcf99700 1 -- 172.21.15.60:0/3305324302 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8012b30 con 0x7f0ba4003f70 2023-07-13T22:20:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.698+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3305324302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 msgr2=0x7f0ba4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.698+0000 7f0bc01ff700 1 --2- 172.21.15.60:0/3305324302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0ba4004850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8007fd0 tx=0x7f0ba4005880 comp rx=0 tx=0).stop 2023-07-13T22:20:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.698+0000 7f0bbe79c700 1 -- 172.21.15.60:0/3305324302 reap_dead start 2023-07-13T22:20:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.698+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3305324302 shutdown_connections 2023-07-13T22:20:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.698+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3305324302 >> 172.21.15.60:0/3305324302 conn(0x7f0bb806b390 msgr2=0x7f0bb806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.698+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3305324302 shutdown_connections 2023-07-13T22:20:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.698+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3305324302 wait complete. 2023-07-13T22:20:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.700+0000 7f0bc01ff700 1 Processor -- start 2023-07-13T22:20:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.700+0000 7f0bc01ff700 1 -- start start 2023-07-13T22:20:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bc01ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0bb80ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bc01ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bb81549e0 0x7f0bb8153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bc01ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b6a00 0x7f0bb80b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bc01ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb80b73c0 con 0x7f0ba4003f70 2023-07-13T22:20:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bc01ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb81577c0 con 0x7f0bb81549e0 2023-07-13T22:20:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bc01ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb80caba0 con 0x7f0bb80b6a00 2023-07-13T22:20:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbdf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0bb80ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbdf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0bb80ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33786/0 (socket says 172.21.15.60:33786) 2023-07-13T22:20:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbdf9b700 1 -- 172.21.15.60:0/3469753952 learned_addr learned my addr 172.21.15.60:0/3469753952 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:20:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbe79c700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b6a00 0x7f0bb80b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbd79a700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bb81549e0 0x7f0bb8153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbdf9b700 1 -- 172.21.15.60:0/3469753952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b6a00 msgr2=0x7f0bb80b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbdf9b700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b6a00 0x7f0bb80b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbdf9b700 1 -- 172.21.15.60:0/3469753952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bb81549e0 msgr2=0x7f0bb8153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbdf9b700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bb81549e0 0x7f0bb8153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbdf9b700 1 -- 172.21.15.60:0/3469753952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8154ea0 con 0x7f0ba4003f70 2023-07-13T22:20:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.701+0000 7f0bbd79a700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bb81549e0 0x7f0bb8153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:20:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.702+0000 7f0bbdf9b700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0bb80ca660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4011aa0 tx=0x7f0bb4011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.702+0000 7f0bae7fc700 1 -- 172.21.15.60:0/3469753952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400f940 con 0x7f0ba4003f70 2023-07-13T22:20:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.702+0000 7f0bae7fc700 1 -- 172.21.15.60:0/3469753952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb400faa0 con 0x7f0ba4003f70 2023-07-13T22:20:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.702+0000 7f0bae7fc700 1 -- 172.21.15.60:0/3469753952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400fe50 con 0x7f0ba4003f70 2023-07-13T22:20:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.702+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb8155180 con 0x7f0ba4003f70 2023-07-13T22:20:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.702+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb8155740 con 0x7f0ba4003f70 2023-07-13T22:20:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.703+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb80b4920 con 0x7f0ba4003f70 2023-07-13T22:20:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.704+0000 7f0bae7fc700 1 -- 172.21.15.60:0/3469753952 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0bb4014440 con 0x7f0ba4003f70 2023-07-13T22:20:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.705+0000 7f0bae7fc700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b9c06ace0 0x7f0b9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.705+0000 7f0bae7fc700 1 -- 172.21.15.60:0/3469753952 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0bb4089c30 con 0x7f0ba4003f70 2023-07-13T22:20:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.705+0000 7f0bbd79a700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b9c06ace0 0x7f0b9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.706+0000 7f0bbd79a700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b9c06ace0 0x7f0b9c06d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8000c00 tx=0x7f0ba8005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:59.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.709+0000 7f0bae7fc700 1 -- 172.21.15.60:0/3469753952 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0bb4054e40 con 0x7f0ba4003f70 2023-07-13T22:20:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.972+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0bb8153bb0 con 0x7f0ba4003f70 2023-07-13T22:20:59.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.976+0000 7f0bae7fc700 1 -- 172.21.15.60:0/3469753952 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170841 (secure 0 0 0) 0x7f0bb4022090 con 0x7f0ba4003f70 2023-07-13T22:20:59.962 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:20:59.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b9c06ace0 msgr2=0x7f0b9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b9c06ace0 0x7f0b9c06d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8000c00 tx=0x7f0ba8005040 comp rx=0 tx=0).stop 2023-07-13T22:20:59.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 msgr2=0x7f0bb80ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0bb80ca660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4011aa0 tx=0x7f0bb4011db0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 shutdown_connections 2023-07-13T22:20:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b9c06ace0 0x7f0b9c06d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ba4003f70 0x7f0bb80ca660 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb80b6a00 0x7f0bb80b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 --2- 172.21.15.60:0/3469753952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bb81549e0 0x7f0bb8153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 >> 172.21.15.60:0/3469753952 conn(0x7f0bb806b390 msgr2=0x7f0bb80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 shutdown_connections 2023-07-13T22:20:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:20:59.979+0000 7f0bc01ff700 1 -- 172.21.15.60:0/3469753952 wait complete. 2023-07-13T22:20:59.982 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3777735812},{"type":"v1","addr":"172.21.15.60:6833","nonce":3777735812}]},"active_addr":"172.21.15.60:6833/3777735812","active_change":"2023-07-13T22:20:41.104469+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4114,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":4179500418}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":830321050}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":3548966272}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":3629766779}]}]} 2023-07-13T22:20:59.986 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-13T22:20:59.986 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-13T22:20:59.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.204+0000 7f4e099ff700 1 Processor -- start 2023-07-13T22:21:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.204+0000 7f4e099ff700 1 -- start start 2023-07-13T22:21:00.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e099ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e040d3750 0x7f4e040cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e099ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e040cd950 con 0x7f4e040cfad0 2023-07-13T22:21:00.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e099ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e040d0ac0 con 0x7f4e040b8400 2023-07-13T22:21:00.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e099ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e040cff50 con 0x7f4e040d3750 2023-07-13T22:21:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e027fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e040d3750 0x7f4e040cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e027fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e040d3750 0x7f4e040cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33792/0 (socket says 172.21.15.60:33792) 2023-07-13T22:21:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e037fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4e040cfad0 0x7f4e040d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39546/0 (socket says 172.21.15.60:39546) 2023-07-13T22:21:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e027fc700 1 -- 172.21.15.60:0/2701278946 learned_addr learned my addr 172.21.15.60:0/2701278946 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e027fc700 1 -- 172.21.15.60:0/2701278946 >> v1:172.21.15.105:6789/0 conn(0x7f4e040b8400 legacy=0x7f4e040cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e027fc700 1 -- 172.21.15.60:0/2701278946 >> v1:172.21.15.60:6789/0 conn(0x7f4e040cfad0 legacy=0x7f4e040d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e027fc700 1 -- 172.21.15.60:0/2701278946 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e040b87d0 con 0x7f4e040d3750 2023-07-13T22:21:00.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.206+0000 7f4e01ffb700 1 -- 172.21.15.60:0/2701278946 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990285516 0 0) 0x7f4dfc001490 con 0x7f4e040cfad0 2023-07-13T22:21:00.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.207+0000 7f4e027fc700 1 --2- 172.21.15.60:0/2701278946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040d3750 0x7f4e040cf590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4df80098f0 tx=0x7f4df8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae06992dae4d4f59 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:00.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.207+0000 7f4e01ffb700 1 -- 172.21.15.60:0/2701278946 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800b8a0 con 0x7f4e040d3750 2023-07-13T22:21:00.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.207+0000 7f4e01ffb700 1 -- 172.21.15.60:0/2701278946 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df800ba00 con 0x7f4e040d3750 2023-07-13T22:21:00.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.207+0000 7f4e01ffb700 1 -- 172.21.15.60:0/2701278946 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800bd90 con 0x7f4e040d3750 2023-07-13T22:21:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.207+0000 7f4e099ff700 1 -- 172.21.15.60:0/2701278946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040d3750 msgr2=0x7f4e040cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.207+0000 7f4e099ff700 1 --2- 172.21.15.60:0/2701278946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040d3750 0x7f4e040cf590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4df80098f0 tx=0x7f4df8009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.208+0000 7f4e099ff700 1 -- 172.21.15.60:0/2701278946 shutdown_connections 2023-07-13T22:21:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.208+0000 7f4e099ff700 1 --2- 172.21.15.60:0/2701278946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040d3750 0x7f4e040cf590 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.208+0000 7f4e099ff700 1 -- 172.21.15.60:0/2701278946 >> 172.21.15.60:0/2701278946 conn(0x7f4e0406b390 msgr2=0x7f4e0406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.208+0000 7f4e099ff700 1 -- 172.21.15.60:0/2701278946 shutdown_connections 2023-07-13T22:21:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.208+0000 7f4e099ff700 1 -- 172.21.15.60:0/2701278946 wait complete. 2023-07-13T22:21:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.209+0000 7f4e099ff700 1 Processor -- start 2023-07-13T22:21:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.209+0000 7f4e099ff700 1 -- start start 2023-07-13T22:21:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.210+0000 7f4e099ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e040b8400 0x7f4e0415de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.210+0000 7f4e099ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040cfad0 0x7f4e040b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.210+0000 7f4e099ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e040d3750 0x7f4e040b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.210+0000 7f4e099ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0415cb80 con 0x7f4e040cfad0 2023-07-13T22:21:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.210+0000 7f4e099ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0415d9e0 con 0x7f4e040d3750 2023-07-13T22:21:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.210+0000 7f4e099ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0415cdd0 con 0x7f4e040b8400 2023-07-13T22:21:00.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e027fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040cfad0 0x7f4e040b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e027fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040cfad0 0x7f4e040b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33804/0 (socket says 172.21.15.60:33804) 2023-07-13T22:21:00.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e027fc700 1 -- 172.21.15.60:0/2761980058 learned_addr learned my addr 172.21.15.60:0/2761980058 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:00.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e037fe700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e040d3750 0x7f4e040b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e02ffd700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e040b8400 0x7f4e0415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e027fc700 1 -- 172.21.15.60:0/2761980058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e040b8400 msgr2=0x7f4e0415de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e027fc700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e040b8400 0x7f4e0415de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e027fc700 1 -- 172.21.15.60:0/2761980058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e040d3750 msgr2=0x7f4e040b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e027fc700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e040d3750 0x7f4e040b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e027fc700 1 -- 172.21.15.60:0/2761980058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df8009550 con 0x7f4e040cfad0 2023-07-13T22:21:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.211+0000 7f4e027fc700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040cfad0 0x7f4e040b31c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4df8014040 tx=0x7f4df8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.212+0000 7f4debfff700 1 -- 172.21.15.60:0/2761980058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df801a8f0 con 0x7f4e040cfad0 2023-07-13T22:21:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.212+0000 7f4debfff700 1 -- 172.21.15.60:0/2761980058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df801e070 con 0x7f4e040cfad0 2023-07-13T22:21:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.212+0000 7f4debfff700 1 -- 172.21.15.60:0/2761980058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df801ac50 con 0x7f4e040cfad0 2023-07-13T22:21:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.212+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0415d0b0 con 0x7f4e040cfad0 2023-07-13T22:21:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.212+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0415d670 con 0x7f4e040cfad0 2023-07-13T22:21:00.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.213+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e040ce7a0 con 0x7f4e040cfad0 2023-07-13T22:21:00.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.214+0000 7f4debfff700 1 -- 172.21.15.60:0/2761980058 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4df802d450 con 0x7f4e040cfad0 2023-07-13T22:21:00.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.215+0000 7f4debfff700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4df006ace0 0x7f4df006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.215+0000 7f4debfff700 1 -- 172.21.15.60:0/2761980058 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4df808a510 con 0x7f4e040cfad0 2023-07-13T22:21:00.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.215+0000 7f4e02ffd700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4df006ace0 0x7f4df006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.216+0000 7f4e02ffd700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4df006ace0 0x7f4df006d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4df4007fd0 tx=0x7f4df4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:00.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.220+0000 7f4debfff700 1 -- 172.21.15.60:0/2761980058 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4df8055530 con 0x7f4e040cfad0 2023-07-13T22:21:00.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.428+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e040d00c0 con 0x7f4e040cfad0 2023-07-13T22:21:00.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.429+0000 7f4debfff700 1 -- 172.21.15.60:0/2761980058 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f4df801adb0 con 0x7f4e040cfad0 2023-07-13T22:21:00.414 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:00.414 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:00.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4df006ace0 msgr2=0x7f4df006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4df006ace0 0x7f4df006d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4df4007fd0 tx=0x7f4df4009040 comp rx=0 tx=0).stop 2023-07-13T22:21:00.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040cfad0 msgr2=0x7f4e040b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040cfad0 0x7f4e040b31c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4df8014040 tx=0x7f4df8004750 comp rx=0 tx=0).stop 2023-07-13T22:21:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 shutdown_connections 2023-07-13T22:21:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4df006ace0 0x7f4df006d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e040d3750 0x7f4e040b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e040b8400 0x7f4e0415de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 --2- 172.21.15.60:0/2761980058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e040cfad0 0x7f4e040b31c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 >> 172.21.15.60:0/2761980058 conn(0x7f4e0406b390 msgr2=0x7f4e040b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:00.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 shutdown_connections 2023-07-13T22:21:00.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.432+0000 7f4e099ff700 1 -- 172.21.15.60:0/2761980058 wait complete. 2023-07-13T22:21:00.435 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-13T22:21:00.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:00.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.654+0000 7fa61a1ff700 1 Processor -- start 2023-07-13T22:21:00.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.654+0000 7fa61a1ff700 1 -- start start 2023-07-13T22:21:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.655+0000 7fa61a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa614070270 0x7fa6140706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.655+0000 7fa61a1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61415d970 con 0x7fa6140c9c50 2023-07-13T22:21:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.655+0000 7fa61a1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa614070c00 con 0x7fa614159170 2023-07-13T22:21:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.655+0000 7fa61a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61415cd60 con 0x7fa614070270 2023-07-13T22:21:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.655+0000 7fa613fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa6140c9c50 0x7fa6140ca070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39550/0 (socket says 172.21.15.60:39550) 2023-07-13T22:21:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.655+0000 7fa613fff700 1 -- 172.21.15.60:0/1904639357 learned_addr learned my addr 172.21.15.60:0/1904639357 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:00.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.656+0000 7fa612ffd700 1 --2- 172.21.15.60:0/1904639357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa614070270 0x7fa6140706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.656+0000 7fa6127fc700 1 -- 172.21.15.60:0/1904639357 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917910364 0 0) 0x7fa61415d970 con 0x7fa6140c9c50 2023-07-13T22:21:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.656+0000 7fa6127fc700 1 -- 172.21.15.60:0/1904639357 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5fc0057c0 con 0x7fa6140c9c50 2023-07-13T22:21:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.656+0000 7fa6127fc700 1 -- 172.21.15.60:0/1904639357 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066934611 0 0) 0x7fa604001110 con 0x7fa614159170 2023-07-13T22:21:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.656+0000 7fa6127fc700 1 -- 172.21.15.60:0/1904639357 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa61415d970 con 0x7fa614159170 2023-07-13T22:21:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.656+0000 7fa612ffd700 1 -- 172.21.15.60:0/1904639357 >> v1:172.21.15.105:6789/0 conn(0x7fa614159170 legacy=0x7fa6140c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.656+0000 7fa612ffd700 1 -- 172.21.15.60:0/1904639357 >> v1:172.21.15.60:6789/0 conn(0x7fa6140c9c50 legacy=0x7fa6140ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.656+0000 7fa612ffd700 1 -- 172.21.15.60:0/1904639357 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61415d040 con 0x7fa614070270 2023-07-13T22:21:00.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.656+0000 7fa612ffd700 1 --2- 172.21.15.60:0/1904639357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa614070270 0x7fa6140706c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa608009870 tx=0x7fa608009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcbb268ed2d7211e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:00.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa6127fc700 1 -- 172.21.15.60:0/1904639357 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608011830 con 0x7fa614070270 2023-07-13T22:21:00.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa6127fc700 1 -- 172.21.15.60:0/1904639357 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa608011990 con 0x7fa614070270 2023-07-13T22:21:00.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa6127fc700 1 -- 172.21.15.60:0/1904639357 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608011d20 con 0x7fa614070270 2023-07-13T22:21:00.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa61a1ff700 1 -- 172.21.15.60:0/1904639357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa614070270 msgr2=0x7fa6140706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa61a1ff700 1 --2- 172.21.15.60:0/1904639357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa614070270 0x7fa6140706c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa608009870 tx=0x7fa608009b80 comp rx=0 tx=0).stop 2023-07-13T22:21:00.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa61a1ff700 1 -- 172.21.15.60:0/1904639357 shutdown_connections 2023-07-13T22:21:00.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa61a1ff700 1 --2- 172.21.15.60:0/1904639357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa614070270 0x7fa6140706c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa61a1ff700 1 -- 172.21.15.60:0/1904639357 >> 172.21.15.60:0/1904639357 conn(0x7fa6140cf410 msgr2=0x7fa6140cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:00.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa61a1ff700 1 -- 172.21.15.60:0/1904639357 shutdown_connections 2023-07-13T22:21:00.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.657+0000 7fa61a1ff700 1 -- 172.21.15.60:0/1904639357 wait complete. 2023-07-13T22:21:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.659+0000 7fa61a1ff700 1 Processor -- start 2023-07-13T22:21:00.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.659+0000 7fa61a1ff700 1 -- start start 2023-07-13T22:21:00.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.660+0000 7fa61a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa614070270 0x7fa61414d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.660+0000 7fa61a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6140c9c50 0x7fa6140af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.660+0000 7fa61a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa614159170 0x7fa6140c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.660+0000 7fa61a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6140afae0 con 0x7fa614070270 2023-07-13T22:21:00.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.660+0000 7fa61a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61406dd00 con 0x7fa614159170 2023-07-13T22:21:00.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.660+0000 7fa61a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61414daf0 con 0x7fa6140c9c50 2023-07-13T22:21:00.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.660+0000 7fa6137fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa614070270 0x7fa61414d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.660+0000 7fa6137fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa614070270 0x7fa61414d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33806/0 (socket says 172.21.15.60:33806) 2023-07-13T22:21:00.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.660+0000 7fa6137fe700 1 -- 172.21.15.60:0/3077024974 learned_addr learned my addr 172.21.15.60:0/3077024974 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:00.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa612ffd700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6140c9c50 0x7fa6140af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa613fff700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa614159170 0x7fa6140c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa6137fe700 1 -- 172.21.15.60:0/3077024974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6140c9c50 msgr2=0x7fa6140af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa6137fe700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6140c9c50 0x7fa6140af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa6137fe700 1 -- 172.21.15.60:0/3077024974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa614159170 msgr2=0x7fa6140c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa6137fe700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa614159170 0x7fa6140c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa6137fe700 1 -- 172.21.15.60:0/3077024974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6080094a0 con 0x7fa614070270 2023-07-13T22:21:00.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa6137fe700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa614070270 0x7fa61414d5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c700 tx=0x7fa60400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:00.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa5fbfff700 1 -- 172.21.15.60:0/3077024974 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040052d0 con 0x7fa614070270 2023-07-13T22:21:00.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa5fbfff700 1 -- 172.21.15.60:0/3077024974 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604005430 con 0x7fa614070270 2023-07-13T22:21:00.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61406d3d0 con 0x7fa614070270 2023-07-13T22:21:00.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa5fbfff700 1 -- 172.21.15.60:0/3077024974 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040038a0 con 0x7fa614070270 2023-07-13T22:21:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.661+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61406d990 con 0x7fa614070270 2023-07-13T22:21:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.662+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6141553a0 con 0x7fa614070270 2023-07-13T22:21:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.663+0000 7fa5fbfff700 1 -- 172.21.15.60:0/3077024974 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa604003a00 con 0x7fa614070270 2023-07-13T22:21:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.664+0000 7fa5fbfff700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa60006ace0 0x7fa60006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.664+0000 7fa5fbfff700 1 -- 172.21.15.60:0/3077024974 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa604088860 con 0x7fa614070270 2023-07-13T22:21:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.665+0000 7fa612ffd700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa60006ace0 0x7fa60006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.665+0000 7fa612ffd700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa60006ace0 0x7fa60006d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa6080097c0 tx=0x7fa60801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:00.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.669+0000 7fa5fbfff700 1 -- 172.21.15.60:0/3077024974 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa604053880 con 0x7fa614070270 2023-07-13T22:21:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.877+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa614070c00 con 0x7fa614070270 2023-07-13T22:21:00.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.879+0000 7fa5fbfff700 1 -- 172.21.15.60:0/3077024974 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7fa614070c00 con 0x7fa614070270 2023-07-13T22:21:00.863 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:00.864 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa60006ace0 msgr2=0x7fa60006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa60006ace0 0x7fa60006d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa6080097c0 tx=0x7fa60801c040 comp rx=0 tx=0).stop 2023-07-13T22:21:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa614070270 msgr2=0x7fa61414d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa614070270 0x7fa61414d5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c700 tx=0x7fa60400cac0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 shutdown_connections 2023-07-13T22:21:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa60006ace0 0x7fa60006d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa614070270 0x7fa61414d5b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa614159170 0x7fa6140c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 --2- 172.21.15.60:0/3077024974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6140c9c50 0x7fa6140af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.881+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 >> 172.21.15.60:0/3077024974 conn(0x7fa6140cf410 msgr2=0x7fa6140ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.882+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 shutdown_connections 2023-07-13T22:21:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:00.882+0000 7fa61a1ff700 1 -- 172.21.15.60:0/3077024974 wait complete. 2023-07-13T22:21:00.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-13T22:21:00.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-13T22:21:00.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-13T22:21:00.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-13T22:21:00.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-13T22:21:00.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-13T22:21:00.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-13T22:21:00.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-13T22:21:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.271+0000 7f4daa1ff700 1 Processor -- start 2023-07-13T22:21:01.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.271+0000 7f4daa1ff700 1 -- start start 2023-07-13T22:21:01.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.274+0000 7f4daa1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 0x7f4da40adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.274+0000 7f4daa1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4da40c3640 0x7f4da4069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.274+0000 7f4daa1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da406a870 con 0x7f4da406a410 2023-07-13T22:21:01.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.274+0000 7f4daa1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da40ae060 con 0x7f4da40c3640 2023-07-13T22:21:01.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.274+0000 7f4daa1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40cd070 con 0x7f4da4160870 2023-07-13T22:21:01.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.271+0000 7f6ccdfff700 1 Processor -- start 2023-07-13T22:21:01.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.271+0000 7f6ccdfff700 1 -- start start 2023-07-13T22:21:01.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.273+0000 7f6ccdfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc8069ed0 0x7f6cc815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.273+0000 7f6ccdfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cc80c3850 0x7f6cc806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.273+0000 7f6ccdfff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc80cc8d0 con 0x7f6cc80cc3d0 2023-07-13T22:21:01.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.273+0000 7f6ccdfff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc806de00 con 0x7f6cc80c3850 2023-07-13T22:21:01.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.273+0000 7f6ccdfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80ce2d0 con 0x7f6cc8069ed0 2023-07-13T22:21:01.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.274+0000 7f6cc77fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc8069ed0 0x7f6cc815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.274+0000 7f6cc77fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc8069ed0 0x7f6cc815ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:49202/0 (socket says 172.21.15.60:49202) 2023-07-13T22:21:01.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.274+0000 7f6cc77fe700 1 -- 172.21.15.60:0/189131296 learned_addr learned my addr 172.21.15.60:0/189131296 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.275+0000 7f4da3fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4da406a410 0x7f4da4070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:49168/0 (socket says 172.21.15.60:49168) 2023-07-13T22:21:01.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.275+0000 7f4da3fff700 1 -- 172.21.15.60:0/3203790962 learned_addr learned my addr 172.21.15.60:0/3203790962 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:01.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.275+0000 7f6cc77fe700 1 -- 172.21.15.60:0/189131296 >> v2:172.21.15.60:3300/0 conn(0x7f6cc80c3850 msgr2=0x7f6cc806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.275+0000 7f6cc77fe700 1 --2- 172.21.15.60:0/189131296 >> v2:172.21.15.60:3300/0 conn(0x7f6cc80c3850 0x7f6cc806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.275+0000 7f6cc77fe700 1 -- 172.21.15.60:0/189131296 >> v1:172.21.15.60:6789/0 conn(0x7f6cc80cc3d0 legacy=0x7f6cc806a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.275+0000 7f6cc77fe700 1 -- 172.21.15.60:0/189131296 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc815cf90 con 0x7f6cc8069ed0 2023-07-13T22:21:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.276+0000 7f6cc77fe700 1 --2- 172.21.15.60:0/189131296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc8069ed0 0x7f6cc815ca50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80092e0 tx=0x7f6cb80095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6990b2f18f91113 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.276+0000 7f6cc67fc700 1 -- 172.21.15.60:0/189131296 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8004230 con 0x7f6cc8069ed0 2023-07-13T22:21:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.276+0000 7f6cc67fc700 1 -- 172.21.15.60:0/189131296 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb8004390 con 0x7f6cc8069ed0 2023-07-13T22:21:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.276+0000 7f6cc67fc700 1 -- 172.21.15.60:0/189131296 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800c770 con 0x7f6cc8069ed0 2023-07-13T22:21:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.276+0000 7f4da37fe700 1 --2- 172.21.15.60:0/3203790962 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 0x7f4da40adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.277+0000 7f6ccdfff700 1 -- 172.21.15.60:0/189131296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc8069ed0 msgr2=0x7f6cc815ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.277+0000 7f6ccdfff700 1 --2- 172.21.15.60:0/189131296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc8069ed0 0x7f6cc815ca50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80092e0 tx=0x7f6cb80095f0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.277+0000 7f6ccdfff700 1 -- 172.21.15.60:0/189131296 shutdown_connections 2023-07-13T22:21:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.277+0000 7f6ccdfff700 1 --2- 172.21.15.60:0/189131296 >> v2:172.21.15.60:3300/0 conn(0x7f6cc80c3850 0x7f6cc806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.277+0000 7f6ccdfff700 1 --2- 172.21.15.60:0/189131296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc8069ed0 0x7f6cc815ca50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.277+0000 7f6ccdfff700 1 -- 172.21.15.60:0/189131296 >> 172.21.15.60:0/189131296 conn(0x7f6cc80cf410 msgr2=0x7f6cc80cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.277+0000 7f6ccdfff700 1 -- 172.21.15.60:0/189131296 shutdown_connections 2023-07-13T22:21:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.277+0000 7f6ccdfff700 1 -- 172.21.15.60:0/189131296 wait complete. 2023-07-13T22:21:01.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.278+0000 7f4da37fe700 1 -- 172.21.15.60:0/3203790962 >> v2:172.21.15.60:3300/0 conn(0x7f4da40c3640 msgr2=0x7f4da4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.278+0000 7f4da37fe700 1 --2- 172.21.15.60:0/3203790962 >> v2:172.21.15.60:3300/0 conn(0x7f4da40c3640 0x7f4da4069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.278+0000 7f4da27fc700 1 -- 172.21.15.60:0/3203790962 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027719662 0 0) 0x7f4da406a870 con 0x7f4da406a410 2023-07-13T22:21:01.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.278+0000 7f4da37fe700 1 -- 172.21.15.60:0/3203790962 >> v1:172.21.15.105:6789/0 conn(0x7f4da406a410 legacy=0x7f4da4070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.278+0000 7f4da37fe700 1 -- 172.21.15.60:0/3203790962 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da4161c50 con 0x7f4da4160870 2023-07-13T22:21:01.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.282+0000 7f4da37fe700 1 --2- 172.21.15.60:0/3203790962 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 0x7f4da40adb20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009d90 tx=0x7f4d9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75020a89ddff35c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.280+0000 7f6ccdfff700 1 Processor -- start 2023-07-13T22:21:01.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.280+0000 7f6ccdfff700 1 -- start start 2023-07-13T22:21:01.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.282+0000 7ff60a5ff700 1 Processor -- start 2023-07-13T22:21:01.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.282+0000 7ff60a5ff700 1 -- start start 2023-07-13T22:21:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.282+0000 7f4da27fc700 1 -- 172.21.15.60:0/3203790962 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c00e070 con 0x7f4da4160870 2023-07-13T22:21:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.282+0000 7f4da27fc700 1 -- 172.21.15.60:0/3203790962 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c00bbd0 con 0x7f4da4160870 2023-07-13T22:21:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.282+0000 7f4da27fc700 1 -- 172.21.15.60:0/3203790962 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c012430 con 0x7f4da4160870 2023-07-13T22:21:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.283+0000 7f4daa1ff700 1 -- 172.21.15.60:0/3203790962 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 msgr2=0x7f4da40adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.283+0000 7f4daa1ff700 1 --2- 172.21.15.60:0/3203790962 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 0x7f4da40adb20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009d90 tx=0x7f4d9c00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f4daa1ff700 1 -- 172.21.15.60:0/3203790962 shutdown_connections 2023-07-13T22:21:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f4daa1ff700 1 --2- 172.21.15.60:0/3203790962 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 0x7f4da40adb20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f4daa1ff700 1 --2- 172.21.15.60:0/3203790962 >> v2:172.21.15.60:3300/0 conn(0x7f4da40c3640 0x7f4da4069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f4daa1ff700 1 -- 172.21.15.60:0/3203790962 >> 172.21.15.60:0/3203790962 conn(0x7f4da40cf410 msgr2=0x7f4da40cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f0805bff700 1 Processor -- start 2023-07-13T22:21:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7fca205ff700 1 Processor -- start 2023-07-13T22:21:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f6ccdfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc8069ed0 0x7f6cc8089950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f6ccdfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cc80c3850 0x7f6cc80af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f6ccdfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc80cc3d0 0x7f6cc80b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f6ccdfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81e4a90 con 0x7f6cc80c3850 2023-07-13T22:21:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f6ccdfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8089e90 con 0x7f6cc8069ed0 2023-07-13T22:21:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.284+0000 7f6ccdfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81ff150 con 0x7f6cc80cc3d0 2023-07-13T22:21:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7f4daa1ff700 1 -- 172.21.15.60:0/3203790962 shutdown_connections 2023-07-13T22:21:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7ff60a5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff604160870 0x7ff6040adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7ff60a5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6040c3640 0x7ff604069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7ff60a5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60406a830 con 0x7ff60406a410 2023-07-13T22:21:01.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7ff60a5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6040ae060 con 0x7ff6040c3640 2023-07-13T22:21:01.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7ff60a5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6040cd070 con 0x7ff604160870 2023-07-13T22:21:01.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7ff608b9c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff60406a410 0x7ff604070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:49180/0 (socket says 172.21.15.60:49180) 2023-07-13T22:21:01.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7ff608b9c700 1 -- 172.21.15.60:0/4084041115 learned_addr learned my addr 172.21.15.60:0/4084041115 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:01.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7fca205ff700 1 -- start start 2023-07-13T22:21:01.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.286+0000 7ff603fff700 1 --2- 172.21.15.60:0/4084041115 >> v2:172.21.15.105:3300/0 conn(0x7ff604160870 0x7ff6040adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.286+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213161937 0 0) 0x7ff60406a830 con 0x7ff60406a410 2023-07-13T22:21:01.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.286+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f00035b0 con 0x7ff60406a410 2023-07-13T22:21:01.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.286+0000 7f6cc6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cc80c3850 0x7f6cc80af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.286+0000 7f6cc6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cc80c3850 0x7f6cc80af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33846/0 (socket says 172.21.15.60:33846) 2023-07-13T22:21:01.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.286+0000 7f6cc6ffd700 1 -- 172.21.15.60:0/109084235 learned_addr learned my addr 172.21.15.60:0/109084235 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.286+0000 7f6cc7fff700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc80cc3d0 0x7f6cc80b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.286+0000 7ff6037fe700 1 --2- 172.21.15.60:0/4084041115 >> v2:172.21.15.60:3300/0 conn(0x7ff6040c3640 0x7ff604069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 <== mon.1 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3717047675 0 0) 0x7ff5f00035b0 con 0x7ff60406a410 2023-07-13T22:21:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 >> v2:172.21.15.105:3300/0 conn(0x7ff604160870 msgr2=0x7ff6040adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 --2- 172.21.15.60:0/4084041115 >> v2:172.21.15.105:3300/0 conn(0x7ff604160870 0x7ff6040adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 >> v2:172.21.15.60:3300/0 conn(0x7ff6040c3640 msgr2=0x7ff604069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 --2- 172.21.15.60:0/4084041115 >> v2:172.21.15.60:3300/0 conn(0x7ff6040c3640 0x7ff604069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff604161c50 con 0x7ff60406a410 2023-07-13T22:21:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 <== mon.1 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7ff5fc0019f0 con 0x7ff60406a410 2023-07-13T22:21:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 >> v1:172.21.15.105:6789/0 conn(0x7ff60406a410 legacy=0x7ff604070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f0003a00 0x7ff5f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5f0004850 0x7ff5f0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff5f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040ae060 con 0x7ff5f0005200 2023-07-13T22:21:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040cd070 con 0x7ff5f0004850 2023-07-13T22:21:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f000c0e0 con 0x7ff5f0003a00 2023-07-13T22:21:01.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.285+0000 7f4daa1ff700 1 -- 172.21.15.60:0/3203790962 wait complete. 2023-07-13T22:21:01.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7f4daa1ff700 1 Processor -- start 2023-07-13T22:21:01.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.288+0000 7ff608b9c700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff5f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.288+0000 7ff6037fe700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5f0004850 0x7ff5f0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7ff608b9c700 1 -- 172.21.15.60:0/4084041115 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f0003a00 msgr2=0x7ff5f0003e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:01.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7ff608b9c700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f0003a00 0x7ff5f0003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7ff608b9c700 1 -- 172.21.15.60:0/4084041115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5f0004850 msgr2=0x7ff5f0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7ff608b9c700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5f0004850 0x7ff5f0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7ff608b9c700 1 -- 172.21.15.60:0/4084041115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff604161c50 con 0x7ff5f0005200 2023-07-13T22:21:01.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7ff608b9c700 1 -- 172.21.15.60:0/4084041115 reap_dead start 2023-07-13T22:21:01.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7f6cc6ffd700 1 -- 172.21.15.60:0/109084235 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc80cc3d0 msgr2=0x7f6cc80b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7f6cc6ffd700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cc80cc3d0 0x7f6cc80b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7f6cc6ffd700 1 -- 172.21.15.60:0/109084235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc8069ed0 msgr2=0x7f6cc8089950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:01.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7f6cc6ffd700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc8069ed0 0x7f6cc8089950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7f6cc6ffd700 1 -- 172.21.15.60:0/109084235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb8009de0 con 0x7f6cc80c3850 2023-07-13T22:21:01.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.288+0000 7f6cc6ffd700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cc80c3850 0x7f6cc80af140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c4a0 tx=0x7f6cbc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.287+0000 7f0805bff700 1 -- start start 2023-07-13T22:21:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7f0805bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08000b8400 0x7f08000cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7f0805bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 0x7f08000d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7f0805bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08000d0ac0 con 0x7f08000d3750 2023-07-13T22:21:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7f0805bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08000b87d0 con 0x7f08000b8400 2023-07-13T22:21:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7f0805bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000b4eb0 con 0x7f08000cfca0 2023-07-13T22:21:01.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7f6caffff700 1 -- 172.21.15.60:0/109084235 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0054e0 con 0x7f6cc80c3850 2023-07-13T22:21:01.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f6caffff700 1 -- 172.21.15.60:0/109084235 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc003680 con 0x7f6cc80c3850 2023-07-13T22:21:01.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f6caffff700 1 -- 172.21.15.60:0/109084235 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc003a30 con 0x7f6cc80c3850 2023-07-13T22:21:01.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.288+0000 7f4daa1ff700 1 -- start start 2023-07-13T22:21:01.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.289+0000 7f4daa1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da406a410 0x7f4da414c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f07ff7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08000b8400 0x7f08000cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7ff608b9c700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff5f000ba10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00a8c0 tx=0x7ff5f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5634ca75eede4722 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f07ff7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08000b8400 0x7f08000cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33880/0 (socket says 172.21.15.60:33880) 2023-07-13T22:21:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f07ff7fe700 1 -- 172.21.15.60:0/3488399940 learned_addr learned my addr 172.21.15.60:0/3488399940 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc012600 con 0x7ff5f0005200 2023-07-13T22:21:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc012780 con 0x7ff5f0005200 2023-07-13T22:21:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7ff602ffd700 1 -- 172.21.15.60:0/4084041115 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc012b30 con 0x7ff5f0005200 2023-07-13T22:21:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4daa1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4da40c3640 0x7f4da414ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4daa1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 0x7f4da40af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4daa1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da414ca20 con 0x7f4da406a410 2023-07-13T22:21:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4daa1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da414b350 con 0x7f4da40c3640 2023-07-13T22:21:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4daa1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40b3ac0 con 0x7f4da4160870 2023-07-13T22:21:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4da37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da406a410 0x7f4da414c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4da37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da406a410 0x7f4da414c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33882/0 (socket says 172.21.15.60:33882) 2023-07-13T22:21:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4da37fe700 1 -- 172.21.15.60:0/91703743 learned_addr learned my addr 172.21.15.60:0/91703743 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4da2ffd700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4da40c3640 0x7f4da414ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f07ff7fe700 1 -- 172.21.15.60:0/3488399940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 msgr2=0x7f08000d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f07fffff700 1 --2- 172.21.15.60:0/3488399940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 0x7f08000d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f07ff7fe700 1 --2- 172.21.15.60:0/3488399940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 0x7f08000d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f07ff7fe700 1 -- 172.21.15.60:0/3488399940 >> v1:172.21.15.105:6789/0 conn(0x7f08000d3750 legacy=0x7f08000cf590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:01.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f07ff7fe700 1 -- 172.21.15.60:0/3488399940 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000b5190 con 0x7f08000b8400 2023-07-13T22:21:01.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.290+0000 7f4da3fff700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 0x7f4da40af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f07fffff700 1 --2- 172.21.15.60:0/3488399940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 0x7f08000d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:01.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7ff60a5ff700 1 -- 172.21.15.60:0/4084041115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 msgr2=0x7ff5f000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7ff60a5ff700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff5f000ba10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00a8c0 tx=0x7ff5f0005880 comp rx=0 tx=0).stop 2023-07-13T22:21:01.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7ff60a5ff700 1 -- 172.21.15.60:0/4084041115 shutdown_connections 2023-07-13T22:21:01.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7ff60a5ff700 1 --2- 172.21.15.60:0/4084041115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff5f000ba10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7ff60a5ff700 1 -- 172.21.15.60:0/4084041115 >> 172.21.15.60:0/4084041115 conn(0x7ff6040cf410 msgr2=0x7ff6040cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7ff60a5ff700 1 -- 172.21.15.60:0/4084041115 shutdown_connections 2023-07-13T22:21:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f4da2ffd700 1 -- 172.21.15.60:0/91703743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 msgr2=0x7f4da40af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f4da2ffd700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4da4160870 0x7f4da40af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f4da2ffd700 1 -- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da406a410 msgr2=0x7f4da414c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f4da2ffd700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da406a410 0x7f4da414c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f4da2ffd700 1 -- 172.21.15.60:0/91703743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c009a70 con 0x7f4da40c3640 2023-07-13T22:21:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f4da2ffd700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4da40c3640 0x7f4da414ae10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400b450 tx=0x7f4d9400b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc8205850 con 0x7f6cc80c3850 2023-07-13T22:21:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc81e40f0 con 0x7f6cc80c3850 2023-07-13T22:21:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6cc80c4030 con 0x7f6cc80c3850 2023-07-13T22:21:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.291+0000 7f07ff7fe700 1 --2- 172.21.15.60:0/3488399940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08000b8400 0x7f08000cd240 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07f00098f0 tx=0x7f07f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52420a1d928226b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7f07fe7fc700 1 -- 172.21.15.60:0/3488399940 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0004230 con 0x7f08000b8400 2023-07-13T22:21:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7f07fe7fc700 1 -- 172.21.15.60:0/3488399940 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f0004390 con 0x7f08000b8400 2023-07-13T22:21:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.292+0000 7f07fe7fc700 1 -- 172.21.15.60:0/3488399940 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00036a0 con 0x7f08000b8400 2023-07-13T22:21:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.293+0000 7f0805bff700 1 -- 172.21.15.60:0/3488399940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08000b8400 msgr2=0x7f08000cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.293+0000 7f0805bff700 1 --2- 172.21.15.60:0/3488399940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08000b8400 0x7f08000cd240 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07f00098f0 tx=0x7f07f0009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.293+0000 7f0805bff700 1 -- 172.21.15.60:0/3488399940 shutdown_connections 2023-07-13T22:21:01.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.293+0000 7f0805bff700 1 --2- 172.21.15.60:0/3488399940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 0x7f08000d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.293+0000 7f0805bff700 1 --2- 172.21.15.60:0/3488399940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08000b8400 0x7f08000cd240 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07f00098f0 tx=0x7f07f0009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:01.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.293+0000 7f0805bff700 1 -- 172.21.15.60:0/3488399940 >> 172.21.15.60:0/3488399940 conn(0x7f080006b390 msgr2=0x7f080006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.294+0000 7f0805bff700 1 -- 172.21.15.60:0/3488399940 shutdown_connections 2023-07-13T22:21:01.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.293+0000 7ff60a5ff700 1 -- 172.21.15.60:0/4084041115 wait complete. 2023-07-13T22:21:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.293+0000 7f4d8bfff700 1 -- 172.21.15.60:0/91703743 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94011830 con 0x7f4da40c3640 2023-07-13T22:21:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.294+0000 7f4d8bfff700 1 -- 172.21.15.60:0/91703743 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94011990 con 0x7f4da40c3640 2023-07-13T22:21:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.294+0000 7f4d8bfff700 1 -- 172.21.15.60:0/91703743 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94011d40 con 0x7f4da40c3640 2023-07-13T22:21:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da40b3da0 con 0x7f4da40c3640 2023-07-13T22:21:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da40b4360 con 0x7f4da40c3640 2023-07-13T22:21:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.294+0000 7fca205ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca18160870 0x7fca180adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.294+0000 7fca205ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fca1806a5e0 0x7fca18160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.294+0000 7fca205ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca180ae060 con 0x7fca180c3640 2023-07-13T22:21:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.294+0000 7fca205ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca180cd070 con 0x7fca1806a5e0 2023-07-13T22:21:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.294+0000 7fca205ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca181643b0 con 0x7fca18160870 2023-07-13T22:21:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7fca1e39b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca18160870 0x7fca180adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7fca1e39b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca18160870 0x7fca180adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:49264/0 (socket says 172.21.15.60:49264) 2023-07-13T22:21:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7fca1e39b700 1 -- 172.21.15.60:0/3532546042 learned_addr learned my addr 172.21.15.60:0/3532546042 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7fca1e39b700 1 -- 172.21.15.60:0/3532546042 >> v2:172.21.15.105:3300/0 conn(0x7fca1806a5e0 msgr2=0x7fca18160c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7fca1e39b700 1 --2- 172.21.15.60:0/3532546042 >> v2:172.21.15.105:3300/0 conn(0x7fca1806a5e0 0x7fca18160c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7fca1e39b700 1 -- 172.21.15.60:0/3532546042 >> v1:172.21.15.105:6789/0 conn(0x7fca180c3640 legacy=0x7fca18069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7fca1e39b700 1 -- 172.21.15.60:0/3532546042 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca18164690 con 0x7fca18160870 2023-07-13T22:21:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7f20e5fff700 1 Processor -- start 2023-07-13T22:21:01.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7f20e5fff700 1 -- start start 2023-07-13T22:21:01.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.296+0000 7fca1e39b700 1 --2- 172.21.15.60:0/3532546042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca18160870 0x7fca180adb20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca10009e50 tx=0x7fca1000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f647f2b8054b74 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.294+0000 7f0805bff700 1 -- 172.21.15.60:0/3488399940 wait complete. 2023-07-13T22:21:01.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.296+0000 7fca1d399700 1 -- 172.21.15.60:0/3532546042 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1000e070 con 0x7fca18160870 2023-07-13T22:21:01.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.296+0000 7fca1d399700 1 -- 172.21.15.60:0/3532546042 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1000b9c0 con 0x7fca18160870 2023-07-13T22:21:01.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.296+0000 7fca1d399700 1 -- 172.21.15.60:0/3532546042 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1000bd70 con 0x7fca18160870 2023-07-13T22:21:01.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7ff60a5ff700 1 Processor -- start 2023-07-13T22:21:01.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.295+0000 7f6caffff700 1 -- 172.21.15.60:0/109084235 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6cbc003b90 con 0x7f6cc80c3850 2023-07-13T22:21:01.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7f8731fff700 1 Processor -- start 2023-07-13T22:21:01.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7f0805bff700 1 Processor -- start 2023-07-13T22:21:01.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7fca205ff700 1 -- 172.21.15.60:0/3532546042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca18160870 msgr2=0x7fca180adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7fca205ff700 1 --2- 172.21.15.60:0/3532546042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca18160870 0x7fca180adb20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca10009e50 tx=0x7fca1000b330 comp rx=0 tx=0).stop 2023-07-13T22:21:01.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7f0805bff700 1 -- start start 2023-07-13T22:21:01.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.296+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4da4164690 con 0x7f4da40c3640 2023-07-13T22:21:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7ff60a5ff700 1 -- start start 2023-07-13T22:21:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7f20e5fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20e0164ed0 0x7f20e00cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7f20e5fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f20e00cfd50 0x7f20e01642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7f20e5fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00ce770 con 0x7f20e01605d0 2023-07-13T22:21:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7f20e5fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e00d1840 con 0x7f20e00cfd50 2023-07-13T22:21:01.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7f20e5fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e00cea50 con 0x7f20e0164ed0 2023-07-13T22:21:01.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.298+0000 7f20d6ffd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20e0164ed0 0x7f20e00cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.298+0000 7ff60a5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff604161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.298+0000 7ff60a5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60414f890 0x7ff60406d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.298+0000 7ff60a5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60414e820 0x7ff604148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.298+0000 7ff60a5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff604163ad0 con 0x7ff5f0005200 2023-07-13T22:21:01.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.298+0000 7ff60a5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff604163db0 con 0x7ff60414e820 2023-07-13T22:21:01.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.298+0000 7ff60a5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff604164090 con 0x7ff60414f890 2023-07-13T22:21:01.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.298+0000 7f20d6ffd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20e0164ed0 0x7f20e00cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:54872/0 (socket says 172.21.15.60:54872) 2023-07-13T22:21:01.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.298+0000 7f20d6ffd700 1 -- 172.21.15.60:0/1813594027 learned_addr learned my addr 172.21.15.60:0/1813594027 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f0805bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 0x7f08000ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f0805bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08000d3750 0x7f08000b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f0805bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08001542c0 0x7f08000b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f0805bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0800155e80 con 0x7f08001542c0 2023-07-13T22:21:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f0805bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0800155270 con 0x7f08000d3750 2023-07-13T22:21:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f0805bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0800155550 con 0x7f08000cfca0 2023-07-13T22:21:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7fca205ff700 1 -- 172.21.15.60:0/3532546042 shutdown_connections 2023-07-13T22:21:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7fca205ff700 1 --2- 172.21.15.60:0/3532546042 >> v2:172.21.15.105:3300/0 conn(0x7fca1806a5e0 0x7fca18160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7fca205ff700 1 --2- 172.21.15.60:0/3532546042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca18160870 0x7fca180adb20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7fca205ff700 1 -- 172.21.15.60:0/3532546042 >> 172.21.15.60:0/3532546042 conn(0x7fca180cf410 msgr2=0x7fca180cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.297+0000 7f8731fff700 1 -- start start 2023-07-13T22:21:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f20d6ffd700 1 -- 172.21.15.60:0/1813594027 >> v2:172.21.15.60:3300/0 conn(0x7f20e00cfd50 msgr2=0x7f20e01642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f20d6ffd700 1 --2- 172.21.15.60:0/1813594027 >> v2:172.21.15.60:3300/0 conn(0x7f20e00cfd50 0x7f20e01642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f20d6ffd700 1 -- 172.21.15.60:0/1813594027 >> v1:172.21.15.60:6789/0 conn(0x7f20e01605d0 legacy=0x7f20e00d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f20d6ffd700 1 -- 172.21.15.60:0/1813594027 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00ced30 con 0x7f20e0164ed0 2023-07-13T22:21:01.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7ff603fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff604161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7ff603fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff604161e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33904/0 (socket says 172.21.15.60:33904) 2023-07-13T22:21:01.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7ff603fff700 1 -- 172.21.15.60:0/3296208013 learned_addr learned my addr 172.21.15.60:0/3296208013 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f8731fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f872c0cbd10 0x7f872c0adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f8731fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f872c06a5e0 0x7f872c158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f8731fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872c159aa0 con 0x7f872c15cd00 2023-07-13T22:21:01.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f8731fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872c0ae1d0 con 0x7f872c0cbd10 2023-07-13T22:21:01.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f8731fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c1597c0 con 0x7f872c06a5e0 2023-07-13T22:21:01.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f20d6ffd700 1 --2- 172.21.15.60:0/1813594027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e0164ed0 0x7f20e00cf810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20e00b8cd0 tx=0x7f20c8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53844734d4ae93f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f6caffff700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cb406adb0 0x7f6cb406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.299+0000 7f6caffff700 1 -- 172.21.15.60:0/109084235 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6cbc0890a0 con 0x7f6cc80c3850 2023-07-13T22:21:01.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f6caffff700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f6cb4073660 0x7f6cb4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f6caffff700 1 -- 172.21.15.60:0/109084235 --> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cb4076120 con 0x7f6cb4073660 2023-07-13T22:21:01.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f6caffff700 1 -- 172.21.15.60:0/109084235 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cbc089450 con 0x7f6cc80c3850 2023-07-13T22:21:01.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f20deffd700 1 -- 172.21.15.60:0/1813594027 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8004230 con 0x7f20e0164ed0 2023-07-13T22:21:01.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f20deffd700 1 -- 172.21.15.60:0/1813594027 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c8004390 con 0x7f20e0164ed0 2023-07-13T22:21:01.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f20deffd700 1 -- 172.21.15.60:0/1813594027 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c80036a0 con 0x7f20e0164ed0 2023-07-13T22:21:01.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f8722ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f872c15cd00 0x7f872c069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39574/0 (socket says 172.21.15.60:39574) 2023-07-13T22:21:01.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f8722ffd700 1 -- 172.21.15.60:0/2248909649 learned_addr learned my addr 172.21.15.60:0/2248909649 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:01.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7f872b7fe700 1 --2- 172.21.15.60:0/2248909649 >> v2:172.21.15.60:3300/0 conn(0x7f872c0cbd10 0x7f872c0adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7f6cc7fff700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f6cb4073660 0x7f6cb4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7f07feffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08000d3750 0x7f08000b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7f07feffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08000d3750 0x7f08000b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:54902/0 (socket says 172.21.15.60:54902) 2023-07-13T22:21:01.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7f07feffd700 1 -- 172.21.15.60:0/2260830757 learned_addr learned my addr 172.21.15.60:0/2260830757 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7f6cc77fe700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cb406adb0 0x7f6cb406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7f07ff7fe700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 0x7f08000ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7ff6037fe700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60414f890 0x7ff60406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7ff603fff700 1 -- 172.21.15.60:0/3296208013 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60414f890 msgr2=0x7ff60406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7ff603fff700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60414f890 0x7ff60406d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7ff603fff700 1 -- 172.21.15.60:0/3296208013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60414e820 msgr2=0x7ff604148f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7ff603fff700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60414e820 0x7ff604148f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.300+0000 7ff603fff700 1 -- 172.21.15.60:0/3296208013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60414e020 con 0x7ff5f0005200 2023-07-13T22:21:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7f20e5fff700 1 -- 172.21.15.60:0/1813594027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e0164ed0 msgr2=0x7f20e00cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7f20e5fff700 1 --2- 172.21.15.60:0/1813594027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e0164ed0 0x7f20e00cf810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20e00b8cd0 tx=0x7f20c8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f20e5fff700 1 -- 172.21.15.60:0/1813594027 shutdown_connections 2023-07-13T22:21:01.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f20e5fff700 1 --2- 172.21.15.60:0/1813594027 >> v2:172.21.15.60:3300/0 conn(0x7f20e00cfd50 0x7f20e01642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f20e5fff700 1 --2- 172.21.15.60:0/1813594027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e0164ed0 0x7f20e00cf810 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f20e5fff700 1 -- 172.21.15.60:0/1813594027 >> 172.21.15.60:0/1813594027 conn(0x7f20e006b390 msgr2=0x7f20e006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f872bfff700 1 --2- 172.21.15.60:0/2248909649 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f872c06a5e0 0x7f872c158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7ff603fff700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff604161e40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6040b1440 tx=0x7ff5f400e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f872affd700 1 -- 172.21.15.60:0/2248909649 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106885624 0 0) 0x7f872c159aa0 con 0x7f872c15cd00 2023-07-13T22:21:01.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f872affd700 1 -- 172.21.15.60:0/2248909649 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87180035b0 con 0x7f872c15cd00 2023-07-13T22:21:01.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f20e5fff700 1 -- 172.21.15.60:0/1813594027 shutdown_connections 2023-07-13T22:21:01.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f6cc7fff700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f6cb4073660 0x7f6cb4075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.303+0000 7ff600ff9700 1 -- 172.21.15.60:0/3296208013 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400c070 con 0x7ff5f0005200 2023-07-13T22:21:01.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.303+0000 7ff600ff9700 1 -- 172.21.15.60:0/3296208013 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f4013610 con 0x7ff5f0005200 2023-07-13T22:21:01.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.303+0000 7ff600ff9700 1 -- 172.21.15.60:0/3296208013 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f40139c0 con 0x7ff5f0005200 2023-07-13T22:21:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60414e300 con 0x7ff5f0005200 2023-07-13T22:21:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60406dc40 con 0x7ff5f0005200 2023-07-13T22:21:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7f6cc77fe700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cb406adb0 0x7f6cb406d260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009630 tx=0x7f6cb8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.303+0000 7f07fffff700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08001542c0 0x7f08000b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7f07ff7fe700 1 -- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08000d3750 msgr2=0x7f08000b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7f07ff7fe700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08000d3750 0x7f08000b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7f07ff7fe700 1 -- 172.21.15.60:0/2260830757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08001542c0 msgr2=0x7f08000b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7f07ff7fe700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08001542c0 0x7f08000b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7f07ff7fe700 1 -- 172.21.15.60:0/2260830757 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0009550 con 0x7f08000cfca0 2023-07-13T22:21:01.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.303+0000 7f872b7fe700 1 -- 172.21.15.60:0/2248909649 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f872c06a5e0 msgr2=0x7f872c158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.303+0000 7f872b7fe700 1 --2- 172.21.15.60:0/2248909649 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f872c06a5e0 0x7f872c158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7f872b7fe700 1 -- 172.21.15.60:0/2248909649 >> v1:172.21.15.60:6789/0 conn(0x7f872c15cd00 legacy=0x7f872c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7f872b7fe700 1 -- 172.21.15.60:0/2248909649 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c15c220 con 0x7f872c0cbd10 2023-07-13T22:21:01.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f20e5fff700 1 -- 172.21.15.60:0/1813594027 wait complete. 2023-07-13T22:21:01.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.305+0000 7f20e5fff700 1 Processor -- start 2023-07-13T22:21:01.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.305+0000 7f872b7fe700 1 --2- 172.21.15.60:0/2248909649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c0cbd10 0x7f872c0adc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f871c009dc0 tx=0x7f871c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bcb4c0770e926c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.305+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff604164690 con 0x7ff5f0005200 2023-07-13T22:21:01.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.305+0000 7f6caffff700 1 -- 172.21.15.60:0/109084235 <== osd.3 v2:172.21.15.60:6818/3719659161 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6cb4076120 con 0x7f6cb4073660 2023-07-13T22:21:01.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.305+0000 7f07ff7fe700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 0x7f08000ca660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f07f0015000 tx=0x7f07f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7fca205ff700 1 -- 172.21.15.60:0/3532546042 shutdown_connections 2023-07-13T22:21:01.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7fca205ff700 1 -- 172.21.15.60:0/3532546042 wait complete. 2023-07-13T22:21:01.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7fca205ff700 1 Processor -- start 2023-07-13T22:21:01.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.305+0000 7fca205ff700 1 -- start start 2023-07-13T22:21:01.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7fca205ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca1806a5e0 0x7fca1820aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7fca205ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca180c3640 0x7fca180b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7fca205ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca18160870 0x7fca18163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7fca205ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1820af60 con 0x7fca1806a5e0 2023-07-13T22:21:01.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7fca205ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca180b2020 con 0x7fca18160870 2023-07-13T22:21:01.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7fca205ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca180b2300 con 0x7fca180c3640 2023-07-13T22:21:01.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.306+0000 7f07e7fff700 1 -- 172.21.15.60:0/2260830757 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0003ea0 con 0x7f08000cfca0 2023-07-13T22:21:01.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f07e7fff700 1 -- 172.21.15.60:0/2260830757 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f000b660 con 0x7f08000cfca0 2023-07-13T22:21:01.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f07e7fff700 1 -- 172.21.15.60:0/2260830757 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f000ba30 con 0x7f08000cfca0 2023-07-13T22:21:01.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0800155830 con 0x7f08000cfca0 2023-07-13T22:21:01.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.305+0000 7f20e5fff700 1 -- start start 2023-07-13T22:21:01.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f20e5fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20e01605d0 0x7f20e00b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f20e5fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e0164ed0 0x7f20e0202ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f20e5fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e00b7e70 0x7f20e00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f20e5fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e0202290 con 0x7f20e0164ed0 2023-07-13T22:21:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f20e5fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00b5ef0 con 0x7f20e00b7e70 2023-07-13T22:21:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f20e5fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e0202810 con 0x7f20e01605d0 2023-07-13T22:21:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f20dffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e00b7e70 0x7f20e00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f20dffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e00b7e70 0x7f20e00b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:54922/0 (socket says 172.21.15.60:54922) 2023-07-13T22:21:01.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f20dffff700 1 -- 172.21.15.60:0/2892053753 learned_addr learned my addr 172.21.15.60:0/2892053753 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.306+0000 7f872affd700 1 -- 172.21.15.60:0/2248909649 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00e070 con 0x7f872c0cbd10 2023-07-13T22:21:01.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.306+0000 7f872affd700 1 -- 172.21.15.60:0/2248909649 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c00bae0 con 0x7f872c0cbd10 2023-07-13T22:21:01.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.306+0000 7f872affd700 1 -- 172.21.15.60:0/2248909649 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00be90 con 0x7f872c0cbd10 2023-07-13T22:21:01.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f8731fff700 1 -- 172.21.15.60:0/2248909649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c0cbd10 msgr2=0x7f872c0adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.307+0000 7f8731fff700 1 --2- 172.21.15.60:0/2248909649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c0cbd10 0x7f872c0adc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f871c009dc0 tx=0x7f871c00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:01.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.301+0000 7f4d8bfff700 1 -- 172.21.15.60:0/91703743 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d9401a400 con 0x7f4da40c3640 2023-07-13T22:21:01.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f4d8bfff700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d9006adb0 0x7f4d9006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.302+0000 7f4d8bfff700 1 -- 172.21.15.60:0/91703743 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d94089680 con 0x7f4da40c3640 2023-07-13T22:21:01.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.303+0000 7f4d8bfff700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f4d90073660 0x7f4d90075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.303+0000 7f4d8bfff700 1 -- 172.21.15.60:0/91703743 --> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d90076120 con 0x7f4d90073660 2023-07-13T22:21:01.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.303+0000 7f4d8bfff700 1 -- 172.21.15.60:0/91703743 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d94089a30 con 0x7f4da40c3640 2023-07-13T22:21:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.304+0000 7f4da37fe700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d9006adb0 0x7f4d9006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.306+0000 7f4da3fff700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f4d90073660 0x7f4d90075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7fca1db9a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca180c3640 0x7fca180b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7fca1db9a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca180c3640 0x7fca180b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:49290/0 (socket says 172.21.15.60:49290) 2023-07-13T22:21:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7fca1db9a700 1 -- 172.21.15.60:0/4080475664 learned_addr learned my addr 172.21.15.60:0/4080475664 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7fca1e39b700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca1806a5e0 0x7fca1820aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7fca1eb9c700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca18160870 0x7fca18163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f20df7fe700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20e01605d0 0x7f20e00b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f8731fff700 1 -- 172.21.15.60:0/2248909649 shutdown_connections 2023-07-13T22:21:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f8731fff700 1 --2- 172.21.15.60:0/2248909649 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f872c06a5e0 0x7f872c158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f8731fff700 1 --2- 172.21.15.60:0/2248909649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c0cbd10 0x7f872c0adc90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f8731fff700 1 -- 172.21.15.60:0/2248909649 >> 172.21.15.60:0/2248909649 conn(0x7f872c0cb200 msgr2=0x7f872c0cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f4da37fe700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d9006adb0 0x7f4d9006d260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c000f80 tx=0x7f4d9c011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f20dffff700 1 -- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20e01605d0 msgr2=0x7f20e00b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f20dffff700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20e01605d0 0x7f20e00b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f20dffff700 1 -- 172.21.15.60:0/2892053753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e0164ed0 msgr2=0x7f20e0202ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f20dffff700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e0164ed0 0x7f20e0202ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.308+0000 7f20dffff700 1 -- 172.21.15.60:0/2892053753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c8009550 con 0x7f20e00b7e70 2023-07-13T22:21:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08000caba0 con 0x7f08000cfca0 2023-07-13T22:21:01.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7ff600ff9700 1 -- 172.21.15.60:0/3296208013 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff5f4013b20 con 0x7ff5f0005200 2023-07-13T22:21:01.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7f20dffff700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e00b7e70 0x7f20e00b3240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f20d800b550 tx=0x7f20d800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7fca1db9a700 1 -- 172.21.15.60:0/4080475664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca18160870 msgr2=0x7fca18163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7fca1db9a700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca18160870 0x7fca18163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7fca1db9a700 1 -- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca1806a5e0 msgr2=0x7fca1820aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7fca1db9a700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca1806a5e0 0x7fca1820aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7fca1db9a700 1 -- 172.21.15.60:0/4080475664 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca10009b30 con 0x7fca180c3640 2023-07-13T22:21:01.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7fca1e39b700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca1806a5e0 0x7fca1820aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7f4da3fff700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f4d90073660 0x7f4d90075a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.310+0000 7ff600ff9700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5e406adb0 0x7ff5e406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.310+0000 7ff600ff9700 1 -- 172.21.15.60:0/3296208013 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5f40896c0 con 0x7ff5f0005200 2023-07-13T22:21:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.311+0000 7ff600ff9700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7ff5e4073660 0x7ff5e4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.311+0000 7ff600ff9700 1 -- 172.21.15.60:0/3296208013 --> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5e4076120 con 0x7ff5e4073660 2023-07-13T22:21:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.311+0000 7ff600ff9700 1 -- 172.21.15.60:0/3296208013 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5f4089a70 con 0x7ff5f0005200 2023-07-13T22:21:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7fca1db9a700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca180c3640 0x7fca180b16d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca0800c4e0 tx=0x7fca0800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7f20dcff9700 1 -- 172.21.15.60:0/2892053753 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8011830 con 0x7f20e00b7e70 2023-07-13T22:21:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7f20dcff9700 1 -- 172.21.15.60:0/2892053753 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8011990 con 0x7f20e00b7e70 2023-07-13T22:21:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7f20dcff9700 1 -- 172.21.15.60:0/2892053753 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8011d40 con 0x7f20e00b7e70 2023-07-13T22:21:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.309+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e02031e0 con 0x7f20e00b7e70 2023-07-13T22:21:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.310+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e00b86e0 con 0x7f20e00b7e70 2023-07-13T22:21:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.311+0000 7ff608b9c700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7ff5e4073660 0x7ff5e4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.312+0000 7ff6037fe700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5e406adb0 0x7ff5e406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.312+0000 7ff608b9c700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7ff5e4073660 0x7ff5e4075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.313+0000 7ff6037fe700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5e406adb0 0x7ff5e406d260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6040c4970 tx=0x7ff5ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.314+0000 7ff600ff9700 1 -- 172.21.15.60:0/3296208013 <== osd.0 v2:172.21.15.60:6800/1806898910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff5e4076120 con 0x7ff5e4073660 2023-07-13T22:21:01.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.311+0000 7f4d8bfff700 1 -- 172.21.15.60:0/91703743 <== osd.1 v2:172.21.15.60:6816/4106681594 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4d90076120 con 0x7f4d90073660 2023-07-13T22:21:01.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.310+0000 7fca0effd700 1 -- 172.21.15.60:0/4080475664 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca080052d0 con 0x7fca180c3640 2023-07-13T22:21:01.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.310+0000 7fca0effd700 1 -- 172.21.15.60:0/4080475664 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca08005430 con 0x7fca180c3640 2023-07-13T22:21:01.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.310+0000 7fca0effd700 1 -- 172.21.15.60:0/4080475664 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca080038a0 con 0x7fca180c3640 2023-07-13T22:21:01.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.311+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca180b25e0 con 0x7fca180c3640 2023-07-13T22:21:01.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.311+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca180af110 con 0x7fca180c3640 2023-07-13T22:21:01.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.313+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca18164c50 con 0x7fca180c3640 2023-07-13T22:21:01.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.314+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08000d0750 con 0x7f08000cfca0 2023-07-13T22:21:01.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.311+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20c4000fc0 con 0x7f20e00b7e70 2023-07-13T22:21:01.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.312+0000 7f8731fff700 1 -- 172.21.15.60:0/2248909649 shutdown_connections 2023-07-13T22:21:01.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.315+0000 7fca0effd700 1 -- 172.21.15.60:0/4080475664 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca08003a00 con 0x7fca180c3640 2023-07-13T22:21:01.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7fca0effd700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9fc06adb0 0x7fc9fc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7fca0effd700 1 -- 172.21.15.60:0/4080475664 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fca08089360 con 0x7fca180c3640 2023-07-13T22:21:01.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7fca0effd700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7fc9fc073660 0x7fc9fc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7fca0effd700 1 -- 172.21.15.60:0/4080475664 --> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc9fc076120 con 0x7fc9fc073660 2023-07-13T22:21:01.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7fca0effd700 1 -- 172.21.15.60:0/4080475664 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca08089710 con 0x7fca180c3640 2023-07-13T22:21:01.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.315+0000 7f07e7fff700 1 -- 172.21.15.60:0/2260830757 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f07f0022070 con 0x7f08000cfca0 2023-07-13T22:21:01.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7f07e7fff700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f07ec06adb0 0x7f07ec06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7f07e7fff700 1 -- 172.21.15.60:0/2260830757 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f07f008ae90 con 0x7f08000cfca0 2023-07-13T22:21:01.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7f07e7fff700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7f07ec0735e0 0x7f07ec0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7f07e7fff700 1 -- 172.21.15.60:0/2260830757 --> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07ec0760a0 con 0x7f07ec0735e0 2023-07-13T22:21:01.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.314+0000 7f8731fff700 1 -- 172.21.15.60:0/2248909649 wait complete. 2023-07-13T22:21:01.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7f8731fff700 1 Processor -- start 2023-07-13T22:21:01.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f07fffff700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7f07ec0735e0 0x7f07ec0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f07e7fff700 1 -- 172.21.15.60:0/2260830757 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f0056070 con 0x7f08000cfca0 2023-07-13T22:21:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.316+0000 7f20dcff9700 1 -- 172.21.15.60:0/2892053753 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20d8005a50 con 0x7f20e00b7e70 2023-07-13T22:21:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7f20dcff9700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20c006adb0 0x7f20c006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f20dcff9700 1 -- 172.21.15.60:0/2892053753 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f20d80894a0 con 0x7f20e00b7e70 2023-07-13T22:21:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f20dcff9700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7f20c0073660 0x7f20c0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f20dcff9700 1 -- 172.21.15.60:0/2892053753 --> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20c0076120 con 0x7f20c0073660 2023-07-13T22:21:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f20dcff9700 1 -- 172.21.15.60:0/2892053753 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20d8089850 con 0x7f20e00b7e70 2023-07-13T22:21:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f20d6ffd700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7f20c0073660 0x7f20c0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.319+0000 7f20d6ffd700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7f20c0073660 0x7f20c0075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.317+0000 7f8731fff700 1 -- start start 2023-07-13T22:21:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f8731fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872c06a5e0 0x7f872c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f8731fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f872c0cbd10 0x7f872c06ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f8731fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c15cd00 0x7f872c0b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.318+0000 7f8731fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c13faf0 con 0x7f872c15cd00 2023-07-13T22:21:01.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.319+0000 7f8731fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c0b2cd0 con 0x7f872c06a5e0 2023-07-13T22:21:01.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.319+0000 7f8731fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c0b2fb0 con 0x7f872c0cbd10 2023-07-13T22:21:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.319+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872c06a5e0 0x7f872c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.319+0000 7f872b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872c06a5e0 0x7f872c0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:54928/0 (socket says 172.21.15.60:54928) 2023-07-13T22:21:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.319+0000 7f872b7fe700 1 -- 172.21.15.60:0/2929405774 learned_addr learned my addr 172.21.15.60:0/2929405774 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.319+0000 7f8722ffd700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f872c0cbd10 0x7f872c06ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7f20df7fe700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20c006adb0 0x7f20c006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7f872b7fe700 1 -- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f872c0cbd10 msgr2=0x7f872c06ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7f872b7fe700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f872c0cbd10 0x7f872c06ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7f872b7fe700 1 -- 172.21.15.60:0/2929405774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c15cd00 msgr2=0x7f872c0b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7f872b7fe700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c15cd00 0x7f872c0b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7f872b7fe700 1 -- 172.21.15.60:0/2929405774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c009a70 con 0x7f872c06a5e0 2023-07-13T22:21:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7f872b7fe700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872c06a5e0 0x7f872c0b26d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f871c000f80 tx=0x7f871c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7f20dcff9700 1 -- 172.21.15.60:0/2892053753 <== osd.5 v2:172.21.15.105:6816/584130633 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f20c0076120 con 0x7f20c0073660 2023-07-13T22:21:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7fca1e39b700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9fc06adb0 0x7fc9fc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.320+0000 7fca1eb9c700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7fc9fc073660 0x7fc9fc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.321+0000 7f07feffd700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f07ec06adb0 0x7f07ec06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.321+0000 7f07fffff700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7f07ec0735e0 0x7f07ec0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.321+0000 7f8728ff9700 1 -- 172.21.15.60:0/2929405774 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c004680 con 0x7f872c06a5e0 2023-07-13T22:21:01.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.321+0000 7f8728ff9700 1 -- 172.21.15.60:0/2929405774 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c020070 con 0x7f872c06a5e0 2023-07-13T22:21:01.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.321+0000 7f8728ff9700 1 -- 172.21.15.60:0/2929405774 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c027480 con 0x7f872c06a5e0 2023-07-13T22:21:01.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.321+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c0b3290 con 0x7f872c06a5e0 2023-07-13T22:21:01.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.321+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872c0b3850 con 0x7f872c06a5e0 2023-07-13T22:21:01.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.322+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8710000fc0 con 0x7f872c06a5e0 2023-07-13T22:21:01.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.322+0000 7f07e7fff700 1 -- 172.21.15.60:0/2260830757 <== osd.4 v2:172.21.15.105:6808/922467711 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f07ec0760a0 con 0x7f07ec0735e0 2023-07-13T22:21:01.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.322+0000 7fca1e39b700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9fc06adb0 0x7fc9fc06d260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca10000f80 tx=0x7fca10009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.323+0000 7f20df7fe700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20c006adb0 0x7f20c006d260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001d60 tx=0x7f20d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.325+0000 7f07feffd700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f07ec06adb0 0x7f07ec06d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f08001f5260 tx=0x7f07f4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.323+0000 7fca1eb9c700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7fc9fc073660 0x7fc9fc075a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.327+0000 7fca0effd700 1 -- 172.21.15.60:0/4080475664 <== osd.2 v2:172.21.15.60:6801/4022134141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc9fc076120 con 0x7fc9fc073660 2023-07-13T22:21:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.328+0000 7f8728ff9700 1 -- 172.21.15.60:0/2929405774 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f871c0275e0 con 0x7f872c06a5e0 2023-07-13T22:21:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.329+0000 7f8728ff9700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f870c06adb0 0x7f870c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.329+0000 7f8728ff9700 1 -- 172.21.15.60:0/2929405774 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f871c022070 con 0x7f872c06a5e0 2023-07-13T22:21:01.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.330+0000 7f8728ff9700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7f870c073660 0x7f870c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.330+0000 7f8728ff9700 1 -- 172.21.15.60:0/2929405774 --> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f870c076120 con 0x7f870c073660 2023-07-13T22:21:01.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.330+0000 7f8728ff9700 1 -- 172.21.15.60:0/2929405774 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f871c08bed0 con 0x7f872c06a5e0 2023-07-13T22:21:01.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.330+0000 7f8722ffd700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f870c06adb0 0x7f870c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.332+0000 7f8722ffd700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f870c06adb0 0x7f870c06d260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f872c15bb40 tx=0x7f8714009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.329+0000 7f65ab7ff700 1 Processor -- start 2023-07-13T22:21:01.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.329+0000 7f65ab7ff700 1 -- start start 2023-07-13T22:21:01.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.332+0000 7f65ab7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65a406a5e0 0x7f65a4158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.332+0000 7f65ab7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65a415cd00 0x7f65a4159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.332+0000 7f65ab7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a4159aa0 con 0x7f65a40add20 2023-07-13T22:21:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.332+0000 7f65ab7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a4159550 con 0x7f65a415cd00 2023-07-13T22:21:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.332+0000 7f65ab7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a41597c0 con 0x7f65a406a5e0 2023-07-13T22:21:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.333+0000 7f65a9d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65a415cd00 0x7f65a4159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.333+0000 7f65a9d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65a415cd00 0x7f65a4159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33968/0 (socket says 172.21.15.60:33968) 2023-07-13T22:21:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.333+0000 7f65a9d9c700 1 -- 172.21.15.60:0/2567178308 learned_addr learned my addr 172.21.15.60:0/2567178308 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.333+0000 7f65a8d9a700 1 --2- 172.21.15.60:0/2567178308 >> v2:172.21.15.105:3300/0 conn(0x7f65a406a5e0 0x7f65a4158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.333+0000 7f65a8d9a700 1 -- 172.21.15.60:0/2567178308 >> v2:172.21.15.60:3300/0 conn(0x7f65a415cd00 msgr2=0x7f65a4159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.333+0000 7f65a8d9a700 1 --2- 172.21.15.60:0/2567178308 >> v2:172.21.15.60:3300/0 conn(0x7f65a415cd00 0x7f65a4159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.333+0000 7f65a8d9a700 1 -- 172.21.15.60:0/2567178308 >> v1:172.21.15.60:6789/0 conn(0x7f65a40add20 legacy=0x7f65a4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.334+0000 7f65a8d9a700 1 -- 172.21.15.60:0/2567178308 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a415c220 con 0x7f65a406a5e0 2023-07-13T22:21:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.334+0000 7f65a8d9a700 1 --2- 172.21.15.60:0/2567178308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65a406a5e0 0x7f65a4158ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65a40700b0 tx=0x7f65a000df40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e8030a84a2074a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.334+0000 7f659bfff700 1 -- 172.21.15.60:0/2567178308 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0012070 con 0x7f65a406a5e0 2023-07-13T22:21:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.334+0000 7f659bfff700 1 -- 172.21.15.60:0/2567178308 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a000d2e0 con 0x7f65a406a5e0 2023-07-13T22:21:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.334+0000 7f659bfff700 1 -- 172.21.15.60:0/2567178308 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0004230 con 0x7f65a406a5e0 2023-07-13T22:21:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.335+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2567178308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65a406a5e0 msgr2=0x7f65a4158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.335+0000 7f65ab7ff700 1 --2- 172.21.15.60:0/2567178308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65a406a5e0 0x7f65a4158ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65a40700b0 tx=0x7f65a000df40 comp rx=0 tx=0).stop 2023-07-13T22:21:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.336+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2567178308 shutdown_connections 2023-07-13T22:21:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.336+0000 7f65ab7ff700 1 --2- 172.21.15.60:0/2567178308 >> v2:172.21.15.60:3300/0 conn(0x7f65a415cd00 0x7f65a4159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.336+0000 7f65ab7ff700 1 --2- 172.21.15.60:0/2567178308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65a406a5e0 0x7f65a4158ad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.336+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2567178308 >> 172.21.15.60:0/2567178308 conn(0x7f65a40cb200 msgr2=0x7f65a40cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.336+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2567178308 shutdown_connections 2023-07-13T22:21:01.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.338+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 --> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6cc80c3d50 con 0x7f6cb4073660 2023-07-13T22:21:01.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.336+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2567178308 wait complete. 2023-07-13T22:21:01.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.338+0000 7f872bfff700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7f870c073660 0x7f870c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.339+0000 7f65ab7ff700 1 Processor -- start 2023-07-13T22:21:01.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.339+0000 7f6caffff700 1 -- 172.21.15.60:0/109084235 <== osd.3 v2:172.21.15.60:6818/3719659161 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6cc0009bb0 con 0x7f6cb4073660 2023-07-13T22:21:01.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.339+0000 7f65ab7ff700 1 -- start start 2023-07-13T22:21:01.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f65ab7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65a406a5e0 0x7f65a406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f65ab7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65a40add20 0x7f65a4154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f65ab7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65a415cd00 0x7f65a4155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f65ab7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a406d2d0 con 0x7f65a406a5e0 2023-07-13T22:21:01.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f65ab7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a406d620 con 0x7f65a40add20 2023-07-13T22:21:01.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f65ab7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a406d900 con 0x7f65a415cd00 2023-07-13T22:21:01.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f6cb4073660 msgr2=0x7f6cb4075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f6ccdfff700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f6cb4073660 0x7f6cb4075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cb406adb0 msgr2=0x7f6cb406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f6ccdfff700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cb406adb0 0x7f6cb406d260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009630 tx=0x7f6cb8008030 comp rx=0 tx=0).stop 2023-07-13T22:21:01.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cc80c3850 msgr2=0x7f6cc80af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.341+0000 7f6ccdfff700 1 --2- 172.21.15.60:0/109084235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cc80c3850 0x7f6cc80af140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c4a0 tx=0x7f6cbc00c7b0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.342+0000 7f65a8d9a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65a40add20 0x7f65a4154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.342+0000 7f65a959b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65a406a5e0 0x7f65a406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.342+0000 7f65a959b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65a406a5e0 0x7f65a406e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33976/0 (socket says 172.21.15.60:33976) 2023-07-13T22:21:01.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.342+0000 7f65a959b700 1 -- 172.21.15.60:0/2216358623 learned_addr learned my addr 172.21.15.60:0/2216358623 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.342+0000 7f65a9d9c700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65a415cd00 0x7f65a4155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.343+0000 7f65a959b700 1 -- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65a415cd00 msgr2=0x7f65a4155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.343+0000 7f65a959b700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65a415cd00 0x7f65a4155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.343+0000 7f65a959b700 1 -- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65a40add20 msgr2=0x7f65a4154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.343+0000 7f65a959b700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65a40add20 0x7f65a4154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.343+0000 7f65a959b700 1 -- 172.21.15.60:0/2216358623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a000d940 con 0x7f65a406a5e0 2023-07-13T22:21:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.343+0000 7f65a959b700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65a406a5e0 0x7f65a406e3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f659c00cc50 tx=0x7f659c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.344+0000 7f6599ffb700 1 -- 172.21.15.60:0/2216358623 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00ab80 con 0x7f65a406a5e0 2023-07-13T22:21:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.344+0000 7f6599ffb700 1 -- 172.21.15.60:0/2216358623 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c00ace0 con 0x7f65a406a5e0 2023-07-13T22:21:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.344+0000 7f6599ffb700 1 -- 172.21.15.60:0/2216358623 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c012660 con 0x7f65a406a5e0 2023-07-13T22:21:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.344+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a406dbe0 con 0x7f65a406a5e0 2023-07-13T22:21:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.344+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a406e920 con 0x7f65a406a5e0 2023-07-13T22:21:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.345+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 --> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4da4164970 con 0x7f4d90073660 2023-07-13T22:21:01.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.345+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65a406d790 con 0x7f65a406a5e0 2023-07-13T22:21:01.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.346+0000 7f6cc7fff700 1 -- 172.21.15.60:0/109084235 reap_dead start 2023-07-13T22:21:01.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.346+0000 7f4d8bfff700 1 -- 172.21.15.60:0/91703743 <== osd.1 v2:172.21.15.60:6816/4106681594 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4d98004b10 con 0x7f4d90073660 2023-07-13T22:21:01.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.346+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 --> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08000b5470 con 0x7f07ec0735e0 2023-07-13T22:21:01.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f872bfff700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7f870c073660 0x7f870c075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.346+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 shutdown_connections 2023-07-13T22:21:01.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.346+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 >> 172.21.15.60:0/109084235 conn(0x7f6cc80cf410 msgr2=0x7f6cc80ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 shutdown_connections 2023-07-13T22:21:01.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f6ccdfff700 1 -- 172.21.15.60:0/109084235 wait complete. 2023-07-13T22:21:01.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f4d90073660 msgr2=0x7f4d90075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f4daa1ff700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f4d90073660 0x7f4d90075a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d9006adb0 msgr2=0x7f4d9006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f4daa1ff700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d9006adb0 0x7f4d9006d260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c000f80 tx=0x7f4d9c011000 comp rx=0 tx=0).stop 2023-07-13T22:21:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4da40c3640 msgr2=0x7f4da414ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f4daa1ff700 1 --2- 172.21.15.60:0/91703743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4da40c3640 0x7f4da414ae10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400b450 tx=0x7f4d9400b760 comp rx=0 tx=0).stop 2023-07-13T22:21:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.347+0000 7f4da3fff700 1 -- 172.21.15.60:0/91703743 reap_dead start 2023-07-13T22:21:01.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.348+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 shutdown_connections 2023-07-13T22:21:01.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.348+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 >> 172.21.15.60:0/91703743 conn(0x7f4da40cf410 msgr2=0x7f4da4155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.348+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 shutdown_connections 2023-07-13T22:21:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.348+0000 7f8728ff9700 1 -- 172.21.15.60:0/2929405774 <== osd.6 v2:172.21.15.105:6800/58712571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f870c076120 con 0x7f870c073660 2023-07-13T22:21:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.348+0000 7f6599ffb700 1 -- 172.21.15.60:0/2216358623 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f659c0127c0 con 0x7f65a406a5e0 2023-07-13T22:21:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.348+0000 7f4daa1ff700 1 -- 172.21.15.60:0/91703743 wait complete. 2023-07-13T22:21:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.348+0000 7f07e7fff700 1 -- 172.21.15.60:0/2260830757 <== osd.4 v2:172.21.15.105:6808/922467711 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f07f8007c40 con 0x7f07ec0735e0 2023-07-13T22:21:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.349+0000 7f6599ffb700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f658806adb0 0x7f658806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.349+0000 7f6599ffb700 1 -- 172.21.15.60:0/2216358623 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f659c08a210 con 0x7f65a406a5e0 2023-07-13T22:21:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.350+0000 7f6599ffb700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f6588073660 0x7f6588075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.350+0000 7f65a8d9a700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f658806adb0 0x7f658806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.350+0000 7f6599ffb700 1 -- 172.21.15.60:0/2216358623 --> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6588076120 con 0x7f6588073660 2023-07-13T22:21:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.350+0000 7f6599ffb700 1 -- 172.21.15.60:0/2216358623 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f659c08a5c0 con 0x7f65a406a5e0 2023-07-13T22:21:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.350+0000 7f65a9d9c700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f6588073660 0x7f6588075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.351+0000 7f65a9d9c700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f6588073660 0x7f6588075a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.351+0000 7f65a8d9a700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f658806adb0 0x7f658806d260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65a0000c00 tx=0x7f65a000d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.350+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 --> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff6041643b0 con 0x7ff5e4073660 2023-07-13T22:21:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.351+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7f07ec0735e0 msgr2=0x7f07ec0759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.351+0000 7f0805bff700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7f07ec0735e0 0x7f07ec0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.351+0000 7ff600ff9700 1 -- 172.21.15.60:0/3296208013 <== osd.0 v2:172.21.15.60:6800/1806898910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff5fc012520 con 0x7ff5e4073660 2023-07-13T22:21:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.352+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f07ec06adb0 msgr2=0x7f07ec06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.352+0000 7f0805bff700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f07ec06adb0 0x7f07ec06d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f08001f5260 tx=0x7f07f4009210 comp rx=0 tx=0).stop 2023-07-13T22:21:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.352+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 msgr2=0x7f08000ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.352+0000 7f0805bff700 1 --2- 172.21.15.60:0/2260830757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08000cfca0 0x7f08000ca660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f07f0015000 tx=0x7f07f0004750 comp rx=0 tx=0).stop 2023-07-13T22:21:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.352+0000 7f07fffff700 1 -- 172.21.15.60:0/2260830757 reap_dead start 2023-07-13T22:21:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.352+0000 7f6599ffb700 1 -- 172.21.15.60:0/2216358623 <== osd.7 v2:172.21.15.105:6824/2508216201 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6588076120 con 0x7f6588073660 2023-07-13T22:21:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.352+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 shutdown_connections 2023-07-13T22:21:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.352+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 >> 172.21.15.60:0/2260830757 conn(0x7f080006b390 msgr2=0x7f08000d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.353+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 shutdown_connections 2023-07-13T22:21:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.353+0000 7f0805bff700 1 -- 172.21.15.60:0/2260830757 wait complete. 2023-07-13T22:21:01.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.360+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7ff5e4073660 msgr2=0x7ff5e4075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.360+0000 7ff60a5ff700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7ff5e4073660 0x7ff5e4075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.361+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5e406adb0 msgr2=0x7ff5e406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.361+0000 7ff60a5ff700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5e406adb0 0x7ff5e406d260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6040c4970 tx=0x7ff5ec008040 comp rx=0 tx=0).stop 2023-07-13T22:21:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.361+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 msgr2=0x7ff604161e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.361+0000 7ff60a5ff700 1 --2- 172.21.15.60:0/3296208013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f0005200 0x7ff604161e40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6040b1440 tx=0x7ff5f400e8f0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.361+0000 7ff608b9c700 1 -- 172.21.15.60:0/3296208013 reap_dead start 2023-07-13T22:21:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.361+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 shutdown_connections 2023-07-13T22:21:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.361+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 >> 172.21.15.60:0/3296208013 conn(0x7ff6040cf410 msgr2=0x7ff6040c3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.362+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 shutdown_connections 2023-07-13T22:21:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.362+0000 7ff60a5ff700 1 -- 172.21.15.60:0/3296208013 wait complete. 2023-07-13T22:21:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.362+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 --> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f20c4002d40 con 0x7f20c0073660 2023-07-13T22:21:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.363+0000 7f20dcff9700 1 -- 172.21.15.60:0/2892053753 <== osd.5 v2:172.21.15.105:6816/584130633 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f20c4002d40 con 0x7f20c0073660 2023-07-13T22:21:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.368+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 --> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca18164970 con 0x7fc9fc073660 2023-07-13T22:21:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.368+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7f20c0073660 msgr2=0x7f20c0075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.368+0000 7f20e5fff700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7f20c0073660 0x7f20c0075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.368+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20c006adb0 msgr2=0x7f20c006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.368+0000 7f20e5fff700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20c006adb0 0x7f20c006d260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001d60 tx=0x7f20d0009040 comp rx=0 tx=0).stop 2023-07-13T22:21:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.368+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e00b7e70 msgr2=0x7f20e00b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.368+0000 7f20e5fff700 1 --2- 172.21.15.60:0/2892053753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20e00b7e70 0x7f20e00b3240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f20d800b550 tx=0x7f20d800b910 comp rx=0 tx=0).stop 2023-07-13T22:21:01.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.369+0000 7f20dffff700 1 -- 172.21.15.60:0/2892053753 reap_dead start 2023-07-13T22:21:01.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.369+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 shutdown_connections 2023-07-13T22:21:01.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.369+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 >> 172.21.15.60:0/2892053753 conn(0x7f20e006b390 msgr2=0x7f20e015ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.371+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 shutdown_connections 2023-07-13T22:21:01.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.371+0000 7f20e5fff700 1 -- 172.21.15.60:0/2892053753 wait complete. 2023-07-13T22:21:01.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.371+0000 7fca0effd700 1 -- 172.21.15.60:0/4080475664 <== osd.2 v2:172.21.15.60:6801/4022134141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fca140094d0 con 0x7fc9fc073660 2023-07-13T22:21:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.373+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7fc9fc073660 msgr2=0x7fc9fc075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.373+0000 7fca205ff700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7fc9fc073660 0x7fc9fc075a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.375+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9fc06adb0 msgr2=0x7fc9fc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.375+0000 7fca205ff700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9fc06adb0 0x7fc9fc06d260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca10000f80 tx=0x7fca10009000 comp rx=0 tx=0).stop 2023-07-13T22:21:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.375+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca180c3640 msgr2=0x7fca180b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.375+0000 7fca205ff700 1 --2- 172.21.15.60:0/4080475664 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca180c3640 0x7fca180b16d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca0800c4e0 tx=0x7fca0800c8a0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.377+0000 7fca1eb9c700 1 -- 172.21.15.60:0/4080475664 reap_dead start 2023-07-13T22:21:01.398 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:01.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-13T22:21:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.377+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 shutdown_connections 2023-07-13T22:21:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.377+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 >> 172.21.15.60:0/4080475664 conn(0x7fca180cf410 msgr2=0x7fca180c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.400 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:01.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-13T22:21:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.377+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 --> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65a415c7e0 con 0x7f6588073660 2023-07-13T22:21:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.378+0000 7f6599ffb700 1 -- 172.21.15.60:0/2216358623 <== osd.7 v2:172.21.15.105:6824/2508216201 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f65940042a0 con 0x7f6588073660 2023-07-13T22:21:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.378+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f6588073660 msgr2=0x7f6588075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.378+0000 7f65ab7ff700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f6588073660 0x7f6588075a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.379+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f658806adb0 msgr2=0x7f658806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.379+0000 7f65ab7ff700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f658806adb0 0x7f658806d260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65a0000c00 tx=0x7f65a000d000 comp rx=0 tx=0).stop 2023-07-13T22:21:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.379+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65a406a5e0 msgr2=0x7f65a406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.379+0000 7f65ab7ff700 1 --2- 172.21.15.60:0/2216358623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65a406a5e0 0x7f65a406e3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f659c00cc50 tx=0x7f659c00cf60 comp rx=0 tx=0).stop 2023-07-13T22:21:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.379+0000 7f65a9d9c700 1 -- 172.21.15.60:0/2216358623 reap_dead start 2023-07-13T22:21:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.379+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 shutdown_connections 2023-07-13T22:21:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.379+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 >> 172.21.15.60:0/2216358623 conn(0x7f65a40cb200 msgr2=0x7f65a4157ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.379+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 shutdown_connections 2023-07-13T22:21:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.379+0000 7f65ab7ff700 1 -- 172.21.15.60:0/2216358623 wait complete. 2023-07-13T22:21:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.380+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 shutdown_connections 2023-07-13T22:21:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.380+0000 7fca205ff700 1 -- 172.21.15.60:0/4080475664 wait complete. 2023-07-13T22:21:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.382+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 --> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8710002d40 con 0x7f870c073660 2023-07-13T22:21:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.383+0000 7f8728ff9700 1 -- 172.21.15.60:0/2929405774 <== osd.6 v2:172.21.15.105:6800/58712571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8710002d40 con 0x7f870c073660 2023-07-13T22:21:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.383+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7f870c073660 msgr2=0x7f870c075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.383+0000 7f8731fff700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7f870c073660 0x7f870c075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.406 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:01.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-13T22:21:01.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.384+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f870c06adb0 msgr2=0x7f870c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.384+0000 7f8731fff700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f870c06adb0 0x7f870c06d260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f872c15bb40 tx=0x7f8714009040 comp rx=0 tx=0).stop 2023-07-13T22:21:01.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.384+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872c06a5e0 msgr2=0x7f872c0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.384+0000 7f8731fff700 1 --2- 172.21.15.60:0/2929405774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872c06a5e0 0x7f872c0b26d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f871c000f80 tx=0x7f871c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.384+0000 7f872bfff700 1 -- 172.21.15.60:0/2929405774 reap_dead start 2023-07-13T22:21:01.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.385+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 shutdown_connections 2023-07-13T22:21:01.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.385+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 >> 172.21.15.60:0/2929405774 conn(0x7f872c0cb200 msgr2=0x7f872c15a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.385+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 shutdown_connections 2023-07-13T22:21:01.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.385+0000 7f8731fff700 1 -- 172.21.15.60:0/2929405774 wait complete. 2023-07-13T22:21:01.408 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2023-07-13T22:21:01.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-13T22:21:01.410 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:01.410 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-13T22:21:01.411 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-07-13T22:21:01.411 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-13T22:21:01.412 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2023-07-13T22:21:01.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-13T22:21:01.413 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:01.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-13T22:21:01.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.822+0000 7f98039ff700 1 Processor -- start 2023-07-13T22:21:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.822+0000 7f98039ff700 1 -- start start 2023-07-13T22:21:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.824+0000 7f98039ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f97fc160920 0x7f97fc069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.825+0000 7f98039ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97fc0c4e10 0x7f97fc06a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.825+0000 7f98039ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc1648d0 con 0x7f97fc0c5330 2023-07-13T22:21:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.825+0000 7f98039ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc0c9200 con 0x7f97fc0c4e10 2023-07-13T22:21:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.825+0000 7f98039ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc164bb0 con 0x7f97fc160920 2023-07-13T22:21:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.826+0000 7f9800f9a700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f97fc0c5330 0x7f97fc070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:49198/0 (socket says 172.21.15.60:49198) 2023-07-13T22:21:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.826+0000 7f9800f9a700 1 -- 172.21.15.60:0/3615937311 learned_addr learned my addr 172.21.15.60:0/3615937311 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.826+0000 7f980179b700 1 --2- 172.21.15.60:0/3615937311 >> v2:172.21.15.105:3300/0 conn(0x7f97fc160920 0x7f97fc069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.827+0000 7f97f3fff700 1 -- 172.21.15.60:0/3615937311 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735558030 0 0) 0x7f97fc1648d0 con 0x7f97fc0c5330 2023-07-13T22:21:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.827+0000 7f97f3fff700 1 -- 172.21.15.60:0/3615937311 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e80035b0 con 0x7f97fc0c5330 2023-07-13T22:21:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.827+0000 7f9801f9c700 1 --2- 172.21.15.60:0/3615937311 >> v2:172.21.15.60:3300/0 conn(0x7f97fc0c4e10 0x7f97fc06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.827+0000 7f980179b700 1 -- 172.21.15.60:0/3615937311 >> v2:172.21.15.60:3300/0 conn(0x7f97fc0c4e10 msgr2=0x7f97fc06a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.827+0000 7f980179b700 1 --2- 172.21.15.60:0/3615937311 >> v2:172.21.15.60:3300/0 conn(0x7f97fc0c4e10 0x7f97fc06a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.827+0000 7f980179b700 1 -- 172.21.15.60:0/3615937311 >> v1:172.21.15.105:6789/0 conn(0x7f97fc0c5330 legacy=0x7f97fc070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.827+0000 7f980179b700 1 -- 172.21.15.60:0/3615937311 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc164e90 con 0x7f97fc160920 2023-07-13T22:21:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.828+0000 7f980179b700 1 --2- 172.21.15.60:0/3615937311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc160920 0x7f97fc069ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0b4990 tx=0x7f97f8009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991b82345596e945 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.828+0000 7f97f3fff700 1 -- 172.21.15.60:0/3615937311 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8011830 con 0x7f97fc160920 2023-07-13T22:21:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.828+0000 7f97f3fff700 1 -- 172.21.15.60:0/3615937311 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f8011990 con 0x7f97fc160920 2023-07-13T22:21:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.828+0000 7f97f3fff700 1 -- 172.21.15.60:0/3615937311 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8011d20 con 0x7f97fc160920 2023-07-13T22:21:01.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.829+0000 7f98039ff700 1 -- 172.21.15.60:0/3615937311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc160920 msgr2=0x7f97fc069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.829+0000 7f98039ff700 1 --2- 172.21.15.60:0/3615937311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc160920 0x7f97fc069ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0b4990 tx=0x7f97f8009b20 comp rx=0 tx=0).stop 2023-07-13T22:21:01.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.830+0000 7f98039ff700 1 -- 172.21.15.60:0/3615937311 shutdown_connections 2023-07-13T22:21:01.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.830+0000 7f98039ff700 1 --2- 172.21.15.60:0/3615937311 >> v2:172.21.15.60:3300/0 conn(0x7f97fc0c4e10 0x7f97fc06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.830+0000 7f98039ff700 1 --2- 172.21.15.60:0/3615937311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc160920 0x7f97fc069ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.830+0000 7f98039ff700 1 -- 172.21.15.60:0/3615937311 >> 172.21.15.60:0/3615937311 conn(0x7f97fc0cb200 msgr2=0x7f97fc0cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.830+0000 7f98039ff700 1 -- 172.21.15.60:0/3615937311 shutdown_connections 2023-07-13T22:21:01.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.830+0000 7f98039ff700 1 -- 172.21.15.60:0/3615937311 wait complete. 2023-07-13T22:21:01.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.833+0000 7f98039ff700 1 Processor -- start 2023-07-13T22:21:01.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.833+0000 7fcc451ff700 1 Processor -- start 2023-07-13T22:21:01.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.834+0000 7fcc451ff700 1 -- start start 2023-07-13T22:21:01.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.833+0000 7f98039ff700 1 -- start start 2023-07-13T22:21:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7f98039ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc0c4e10 0x7f97fc150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7f98039ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0c5330 0x7f97fc141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7f98039ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97fc160920 0x7f97fc13cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7f98039ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc13c5d0 con 0x7f97fc160920 2023-07-13T22:21:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7f98039ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc13f030 con 0x7f97fc0c4e10 2023-07-13T22:21:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7f98039ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc13f310 con 0x7f97fc0c5330 2023-07-13T22:21:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7fe1a41ff700 1 Processor -- start 2023-07-13T22:21:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7f980179b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc0c4e10 0x7f97fc150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7f9801f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97fc160920 0x7f97fc13cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.836+0000 7f980179b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc0c4e10 0x7f97fc150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:54952/0 (socket says 172.21.15.60:54952) 2023-07-13T22:21:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.836+0000 7f980179b700 1 -- 172.21.15.60:0/1344238456 learned_addr learned my addr 172.21.15.60:0/1344238456 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.836+0000 7f9800f9a700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0c5330 0x7f97fc141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.836+0000 7f980179b700 1 -- 172.21.15.60:0/1344238456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0c5330 msgr2=0x7f97fc141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.836+0000 7f980179b700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0c5330 0x7f97fc141490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.836+0000 7f980179b700 1 -- 172.21.15.60:0/1344238456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97fc160920 msgr2=0x7f97fc13cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.836+0000 7f980179b700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97fc160920 0x7f97fc13cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.836+0000 7f980179b700 1 -- 172.21.15.60:0/1344238456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80094a0 con 0x7f97fc0c4e10 2023-07-13T22:21:01.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.835+0000 7fe1a41ff700 1 -- start start 2023-07-13T22:21:01.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7fe1a41ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c160870 0x7fe19c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7fe1a41ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe19c06a5e0 0x7fe19c160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7f980179b700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc0c4e10 0x7f97fc150bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009b60 tx=0x7f97f8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7fe1a41ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19c0ae060 con 0x7fe19c0c3640 2023-07-13T22:21:01.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7fe1a41ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19c0cd070 con 0x7fe19c06a5e0 2023-07-13T22:21:01.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7fe1a41ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c1643b0 con 0x7fe19c160870 2023-07-13T22:21:01.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7fe1a1f9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c160870 0x7fe19c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7fe1a1f9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c160870 0x7fe19c0adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:49332/0 (socket says 172.21.15.60:49332) 2023-07-13T22:21:01.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7fe1a1f9b700 1 -- 172.21.15.60:0/2844544112 learned_addr learned my addr 172.21.15.60:0/2844544112 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.838+0000 7fe1a279c700 1 --2- 172.21.15.60:0/2844544112 >> v2:172.21.15.105:3300/0 conn(0x7fe19c06a5e0 0x7fe19c160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7fa8623ff700 1 Processor -- start 2023-07-13T22:21:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7f97f1ffb700 1 -- 172.21.15.60:0/1344238456 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f80036a0 con 0x7f97fc0c4e10 2023-07-13T22:21:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7f97f1ffb700 1 -- 172.21.15.60:0/1344238456 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f801f070 con 0x7f97fc0c4e10 2023-07-13T22:21:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7f97f1ffb700 1 -- 172.21.15.60:0/1344238456 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8003a40 con 0x7f97fc0c4e10 2023-07-13T22:21:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc13f5f0 con 0x7f97fc0c4e10 2023-07-13T22:21:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.840+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc13ec40 con 0x7f97fc0c4e10 2023-07-13T22:21:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7fe1a1f9b700 1 -- 172.21.15.60:0/2844544112 >> v2:172.21.15.105:3300/0 conn(0x7fe19c06a5e0 msgr2=0x7fe19c160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7fe1a1f9b700 1 --2- 172.21.15.60:0/2844544112 >> v2:172.21.15.105:3300/0 conn(0x7fe19c06a5e0 0x7fe19c160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.840+0000 7fe1a1f9b700 1 -- 172.21.15.60:0/2844544112 >> v1:172.21.15.60:6789/0 conn(0x7fe19c0c3640 legacy=0x7fe19c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.840+0000 7fe1a1f9b700 1 -- 172.21.15.60:0/2844544112 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c164690 con 0x7fe19c160870 2023-07-13T22:21:01.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7fcc451ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcc40160920 0x7fcc40069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7fcc451ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc4006a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7fcc451ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcc4006e630 0x7fcc40070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7fcc451ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc400c9200 con 0x7fcc40160920 2023-07-13T22:21:01.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7fcc451ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc401648d0 con 0x7fcc4006e630 2023-07-13T22:21:01.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.839+0000 7fcc451ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40164bb0 con 0x7fcc400c5330 2023-07-13T22:21:01.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.840+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc4006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.840+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc4006a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:49346/0 (socket says 172.21.15.60:49346) 2023-07-13T22:21:01.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.840+0000 7fcc3e59c700 1 -- 172.21.15.60:0/557776569 learned_addr learned my addr 172.21.15.60:0/557776569 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.840+0000 7fcc3f59e700 1 --2- 172.21.15.60:0/557776569 >> v2:172.21.15.105:3300/0 conn(0x7fcc4006e630 0x7fcc40070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.841+0000 7fcc3ed9d700 1 --2- 172.21.15.60:0/557776569 >> v2:172.21.15.60:3300/0 conn(0x7fcc40160920 0x7fcc40069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.842+0000 7fcc3ed9d700 1 -- 172.21.15.60:0/557776569 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 msgr2=0x7fcc4006a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.842+0000 7fcc3ed9d700 1 --2- 172.21.15.60:0/557776569 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc4006a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.842+0000 7fcc3ed9d700 1 -- 172.21.15.60:0/557776569 >> v2:172.21.15.105:3300/0 conn(0x7fcc4006e630 msgr2=0x7fcc40070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.842+0000 7fcc3ed9d700 1 --2- 172.21.15.60:0/557776569 >> v2:172.21.15.105:3300/0 conn(0x7fcc4006e630 0x7fcc40070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.842+0000 7fcc3ed9d700 1 -- 172.21.15.60:0/557776569 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc40164e90 con 0x7fcc40160920 2023-07-13T22:21:01.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.842+0000 7fcc3ed9d700 1 --2- 172.21.15.60:0/557776569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc40160920 0x7fcc40069ed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcc4006b6c0 tx=0x7fcc38009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f82933acd171836 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fcc3dd9b700 1 -- 172.21.15.60:0/557776569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38011830 con 0x7fcc40160920 2023-07-13T22:21:01.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fcc3dd9b700 1 -- 172.21.15.60:0/557776569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc38011990 con 0x7fcc40160920 2023-07-13T22:21:01.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fcc3dd9b700 1 -- 172.21.15.60:0/557776569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38011d20 con 0x7fcc40160920 2023-07-13T22:21:01.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.844+0000 7fe1a1f9b700 1 --2- 172.21.15.60:0/2844544112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c160870 0x7fe19c0adb20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe1980098f0 tx=0x7fe198009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1849a2e233a3578a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7f97f1ffb700 1 -- 172.21.15.60:0/1344238456 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97f8003ba0 con 0x7f97fc0c4e10 2023-07-13T22:21:01.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.841+0000 7fa8623ff700 1 -- start start 2023-07-13T22:21:01.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fa8623ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85c06a5e0 0x7fa85c158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fa8623ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa85c15cd00 0x7fa85c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fa8623ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c159aa0 con 0x7fa85c0add20 2023-07-13T22:21:01.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fa8623ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85c159550 con 0x7fa85c15cd00 2023-07-13T22:21:01.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fa8623ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c1597c0 con 0x7fa85c06a5e0 2023-07-13T22:21:01.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fa85bfff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa85c0add20 0x7fa85c069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:49210/0 (socket says 172.21.15.60:49210) 2023-07-13T22:21:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.843+0000 7fa85bfff700 1 -- 172.21.15.60:0/859640103 learned_addr learned my addr 172.21.15.60:0/859640103 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.844+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423892652 0 0) 0x7fa85c159aa0 con 0x7fa85c0add20 2023-07-13T22:21:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.844+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8440035b0 con 0x7fa85c0add20 2023-07-13T22:21:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.844+0000 7fa85b7fe700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85c06a5e0 0x7fa85c158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.844+0000 7fcc451ff700 1 -- 172.21.15.60:0/557776569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc40160920 msgr2=0x7fcc40069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.844+0000 7fcc451ff700 1 --2- 172.21.15.60:0/557776569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc40160920 0x7fcc40069ed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcc4006b6c0 tx=0x7fcc38009b20 comp rx=0 tx=0).stop 2023-07-13T22:21:01.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fe1a0f99700 1 -- 172.21.15.60:0/2844544112 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800b8a0 con 0x7fe19c160870 2023-07-13T22:21:01.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fe1a0f99700 1 -- 172.21.15.60:0/2844544112 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19800ba00 con 0x7fe19c160870 2023-07-13T22:21:01.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fe1a0f99700 1 -- 172.21.15.60:0/2844544112 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800bd90 con 0x7fe19c160870 2023-07-13T22:21:01.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fe1a41ff700 1 -- 172.21.15.60:0/2844544112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c160870 msgr2=0x7fe19c0adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fe1a41ff700 1 --2- 172.21.15.60:0/2844544112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c160870 0x7fe19c0adb20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe1980098f0 tx=0x7fe198009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7f97f1ffb700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e006adb0 0x7f97e006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7f97f1ffb700 1 -- 172.21.15.60:0/1344238456 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97f8021070 con 0x7f97fc0c4e10 2023-07-13T22:21:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7f9800f9a700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e006adb0 0x7f97e006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 <== mon.1 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3890365053 0 0) 0x7fa8440035b0 con 0x7fa85c0add20 2023-07-13T22:21:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85c06a5e0 msgr2=0x7fa85c158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85c06a5e0 0x7fa85c158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 >> v2:172.21.15.60:3300/0 conn(0x7fa85c15cd00 msgr2=0x7fa85c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 --2- 172.21.15.60:0/859640103 >> v2:172.21.15.60:3300/0 conn(0x7fa85c15cd00 0x7fa85c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c15c220 con 0x7fa85c0add20 2023-07-13T22:21:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 <== mon.1 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fa84c001670 con 0x7fa85c0add20 2023-07-13T22:21:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 >> v1:172.21.15.105:6789/0 conn(0x7fa85c0add20 legacy=0x7fa85c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8440037f0 0x7fa844003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa844004140 0x7fa84400b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 0x7fa84400d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c159550 con 0x7fa844004140 2023-07-13T22:21:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c1597c0 con 0x7fa8440037f0 2023-07-13T22:21:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.845+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84400e260 con 0x7fa84400b5e0 2023-07-13T22:21:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fa86099c700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 0x7fa84400d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fa85bfff700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8440037f0 0x7fa844003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fa85b7fe700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa844004140 0x7fa84400b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7fcc451ff700 1 -- 172.21.15.60:0/557776569 shutdown_connections 2023-07-13T22:21:01.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7fcc451ff700 1 --2- 172.21.15.60:0/557776569 >> v2:172.21.15.105:3300/0 conn(0x7fcc4006e630 0x7fcc40070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7fcc451ff700 1 --2- 172.21.15.60:0/557776569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc40160920 0x7fcc40069ed0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7fcc451ff700 1 --2- 172.21.15.60:0/557776569 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc4006a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7fcc451ff700 1 -- 172.21.15.60:0/557776569 >> 172.21.15.60:0/557776569 conn(0x7fcc400cb200 msgr2=0x7fcc400cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fa86099c700 1 -- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8440037f0 msgr2=0x7fa844003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fa86099c700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8440037f0 0x7fa844003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fa86099c700 1 -- 172.21.15.60:0/859640103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa844004140 msgr2=0x7fa84400b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fa86099c700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa844004140 0x7fa84400b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.846+0000 7fa86099c700 1 -- 172.21.15.60:0/859640103 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c15c220 con 0x7fa84400b5e0 2023-07-13T22:21:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7fa86099c700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 0x7fa84400d9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa854009e10 tx=0x7fa844004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0355b4e3d2b245 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7fcc451ff700 1 -- 172.21.15.60:0/557776569 shutdown_connections 2023-07-13T22:21:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7fcc451ff700 1 -- 172.21.15.60:0/557776569 wait complete. 2023-07-13T22:21:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7f9800f9a700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e006adb0 0x7f97e006d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97fc151100 tx=0x7f97ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.847+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc0052f0 con 0x7f97fc0c4e10 2023-07-13T22:21:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.848+0000 7fe1a41ff700 1 -- 172.21.15.60:0/2844544112 shutdown_connections 2023-07-13T22:21:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.848+0000 7fe1a41ff700 1 --2- 172.21.15.60:0/2844544112 >> v2:172.21.15.105:3300/0 conn(0x7fe19c06a5e0 0x7fe19c160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.848+0000 7fe1a41ff700 1 --2- 172.21.15.60:0/2844544112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c160870 0x7fe19c0adb20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.848+0000 7fe1a41ff700 1 -- 172.21.15.60:0/2844544112 >> 172.21.15.60:0/2844544112 conn(0x7fe19c0cf410 msgr2=0x7fe19c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.848+0000 7fe1a41ff700 1 -- 172.21.15.60:0/2844544112 shutdown_connections 2023-07-13T22:21:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.848+0000 7fe1a41ff700 1 -- 172.21.15.60:0/2844544112 wait complete. 2023-07-13T22:21:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.848+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540098b0 con 0x7fa84400b5e0 2023-07-13T22:21:01.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.848+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85400baa0 con 0x7fa84400b5e0 2023-07-13T22:21:01.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.848+0000 7fa85affd700 1 -- 172.21.15.60:0/859640103 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400be50 con 0x7fa84400b5e0 2023-07-13T22:21:01.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.849+0000 7fa8623ff700 1 -- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 msgr2=0x7fa84400d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.849+0000 7fa8623ff700 1 --2- 172.21.15.60:0/859640103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 0x7fa84400d9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa854009e10 tx=0x7fa844004870 comp rx=0 tx=0).stop 2023-07-13T22:21:01.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.849+0000 7fa86099c700 1 -- 172.21.15.60:0/859640103 reap_dead start 2023-07-13T22:21:01.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.850+0000 7fcc451ff700 1 Processor -- start 2023-07-13T22:21:01.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.850+0000 7fcc451ff700 1 -- start start 2023-07-13T22:21:01.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.851+0000 7fe1a41ff700 1 Processor -- start 2023-07-13T22:21:01.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc451ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc4006e630 0x7fcc400b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc451ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc401ea510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc451ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc40160920 0x7fcc401eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc451ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40161330 con 0x7fcc4006e630 2023-07-13T22:21:01.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc451ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400afeb0 con 0x7fcc40160920 2023-07-13T22:21:01.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc451ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc400b0190 con 0x7fcc400c5330 2023-07-13T22:21:01.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fa8623ff700 1 -- 172.21.15.60:0/859640103 shutdown_connections 2023-07-13T22:21:01.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fa8623ff700 1 -- 172.21.15.60:0/859640103 >> 172.21.15.60:0/859640103 conn(0x7fa85c0cb200 msgr2=0x7fa85c0cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc401ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc401ea510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:49376/0 (socket says 172.21.15.60:49376) 2023-07-13T22:21:01.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc3e59c700 1 -- 172.21.15.60:0/2793074930 learned_addr learned my addr 172.21.15.60:0/2793074930 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7fcc3f59e700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc40160920 0x7fcc401eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7f47da3ff700 1 Processor -- start 2023-07-13T22:21:01.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.852+0000 7f47da3ff700 1 -- start start 2023-07-13T22:21:01.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fcc3e59c700 1 -- 172.21.15.60:0/2793074930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc40160920 msgr2=0x7fcc401eaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fcc3e59c700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc40160920 0x7fcc401eaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fcc3e59c700 1 -- 172.21.15.60:0/2793074930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc4006e630 msgr2=0x7fcc400b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fcc3e59c700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc4006e630 0x7fcc400b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fcc3e59c700 1 -- 172.21.15.60:0/2793074930 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc380094a0 con 0x7fcc400c5330 2023-07-13T22:21:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.851+0000 7fe1a41ff700 1 -- start start 2023-07-13T22:21:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fe1a41ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe19c06a5e0 0x7fe19c0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fe1a41ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c0c3640 0x7fe19c0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fe1a41ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c160870 0x7fe19c154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fe1a41ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c0b2620 con 0x7fe19c160870 2023-07-13T22:21:01.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fe1a41ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c1550b0 con 0x7fe19c06a5e0 2023-07-13T22:21:01.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.853+0000 7fe1a41ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c0b2880 con 0x7fe19c0c3640 2023-07-13T22:21:01.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fcc3e59c700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc401ea510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000efc0 tx=0x7fcc3000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fe1a1f9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe19c06a5e0 0x7fe19c0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fe1a1f9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe19c06a5e0 0x7fe19c0c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:54992/0 (socket says 172.21.15.60:54992) 2023-07-13T22:21:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fe1a1f9b700 1 -- 172.21.15.60:0/3300092811 learned_addr learned my addr 172.21.15.60:0/3300092811 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fe1a279c700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c160870 0x7fe19c154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fe1a179a700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c0c3640 0x7fe19c0b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fa8623ff700 1 -- 172.21.15.60:0/859640103 shutdown_connections 2023-07-13T22:21:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7f47da3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f47d40b8400 0x7f47d40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7f47da3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47d40d3750 0x7f47d40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7f47da3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40cfad0 0x7f47d415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47da3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d40cff50 con 0x7f47d40d3750 2023-07-13T22:21:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47da3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d40d0ac0 con 0x7f47d40b8400 2023-07-13T22:21:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47da3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40b87d0 con 0x7f47d40cfad0 2023-07-13T22:21:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f97f1ffb700 1 -- 172.21.15.60:0/1344238456 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f97f8055d00 con 0x7f97fc0c4e10 2023-07-13T22:21:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fcc277fe700 1 -- 172.21.15.60:0/2793074930 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3000cbb0 con 0x7fcc400c5330 2023-07-13T22:21:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fcc277fe700 1 -- 172.21.15.60:0/2793074930 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3000f040 con 0x7fcc400c5330 2023-07-13T22:21:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.854+0000 7fcc277fe700 1 -- 172.21.15.60:0/2793074930 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30009e60 con 0x7fcc400c5330 2023-07-13T22:21:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7fe1a1f9b700 1 -- 172.21.15.60:0/3300092811 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c0c3640 msgr2=0x7fe19c0b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7fe1a1f9b700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c0c3640 0x7fe19c0b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7fe1a1f9b700 1 -- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c160870 msgr2=0x7fe19c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7fe1a1f9b700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c160870 0x7fe19c154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7fe1a1f9b700 1 -- 172.21.15.60:0/3300092811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe198009550 con 0x7fe19c06a5e0 2023-07-13T22:21:01.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47d899c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40cfad0 0x7f47d415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47d3fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f47d40b8400 0x7f47d40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47d899c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40cfad0 0x7f47d415fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:49386/0 (socket says 172.21.15.60:49386) 2023-07-13T22:21:01.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47d899c700 1 -- 172.21.15.60:0/3938523425 learned_addr learned my addr 172.21.15.60:0/3938523425 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47d3fff700 1 -- 172.21.15.60:0/3938523425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40cfad0 msgr2=0x7f47d415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47d3fff700 1 --2- 172.21.15.60:0/3938523425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40cfad0 0x7f47d415fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47d3fff700 1 -- 172.21.15.60:0/3938523425 >> v2:172.21.15.60:3300/0 conn(0x7f47d40d3750 msgr2=0x7f47d40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47d3fff700 1 --2- 172.21.15.60:0/3938523425 >> v2:172.21.15.60:3300/0 conn(0x7f47d40d3750 0x7f47d40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.855+0000 7f47d3fff700 1 -- 172.21.15.60:0/3938523425 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40b4eb0 con 0x7f47d40b8400 2023-07-13T22:21:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.856+0000 7f47d3fff700 1 --2- 172.21.15.60:0/3938523425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 0x7f47d40cd240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47d40cf560 tx=0x7f47c4009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41650fa21e37b481 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.856+0000 7f47d2ffd700 1 -- 172.21.15.60:0/3938523425 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4011830 con 0x7f47d40b8400 2023-07-13T22:21:01.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.856+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc400b0470 con 0x7fcc400c5330 2023-07-13T22:21:01.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.856+0000 7f47d2ffd700 1 -- 172.21.15.60:0/3938523425 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c4011990 con 0x7f47d40b8400 2023-07-13T22:21:01.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.856+0000 7f47d2ffd700 1 -- 172.21.15.60:0/3938523425 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4011d20 con 0x7f47d40b8400 2023-07-13T22:21:01.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.856+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4015e960 con 0x7fcc400c5330 2023-07-13T22:21:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.857+0000 7f47da3ff700 1 -- 172.21.15.60:0/3938523425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 msgr2=0x7f47d40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.857+0000 7f47da3ff700 1 --2- 172.21.15.60:0/3938523425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 0x7f47d40cd240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47d40cf560 tx=0x7f47c4009b20 comp rx=0 tx=0).stop 2023-07-13T22:21:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.858+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc400b0690 con 0x7fcc400c5330 2023-07-13T22:21:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.858+0000 7f47da3ff700 1 -- 172.21.15.60:0/3938523425 shutdown_connections 2023-07-13T22:21:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.858+0000 7f47da3ff700 1 --2- 172.21.15.60:0/3938523425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40cfad0 0x7f47d415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.858+0000 7f47da3ff700 1 --2- 172.21.15.60:0/3938523425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 0x7f47d40cd240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.858+0000 7f47da3ff700 1 --2- 172.21.15.60:0/3938523425 >> v2:172.21.15.60:3300/0 conn(0x7f47d40d3750 0x7f47d40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.858+0000 7f47da3ff700 1 -- 172.21.15.60:0/3938523425 >> 172.21.15.60:0/3938523425 conn(0x7f47d406b390 msgr2=0x7f47d406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.859+0000 7fe1a1f9b700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe19c06a5e0 0x7fe19c0c7550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe198014040 tx=0x7fe198004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.857+0000 7fa8623ff700 1 -- 172.21.15.60:0/859640103 wait complete. 2023-07-13T22:21:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.859+0000 7fe1927fc700 1 -- 172.21.15.60:0/3300092811 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198004590 con 0x7fe19c06a5e0 2023-07-13T22:21:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.859+0000 7fe1927fc700 1 -- 172.21.15.60:0/3300092811 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1980036a0 con 0x7fe19c06a5e0 2023-07-13T22:21:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.859+0000 7fe1927fc700 1 -- 172.21.15.60:0/3300092811 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198003a50 con 0x7fe19c06a5e0 2023-07-13T22:21:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.859+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19c0b2b60 con 0x7fe19c06a5e0 2023-07-13T22:21:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.860+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19c0b3120 con 0x7fe19c06a5e0 2023-07-13T22:21:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.860+0000 7fa8623ff700 1 Processor -- start 2023-07-13T22:21:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.861+0000 7fe0dc1ff700 1 Processor -- start 2023-07-13T22:21:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.862+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19c04e280 con 0x7fe19c06a5e0 2023-07-13T22:21:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.861+0000 7fcc277fe700 1 -- 172.21.15.60:0/2793074930 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc30014480 con 0x7fcc400c5330 2023-07-13T22:21:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.862+0000 7fcc277fe700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcc2c06adb0 0x7fcc2c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.862+0000 7fcc277fe700 1 -- 172.21.15.60:0/2793074930 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc30089880 con 0x7fcc400c5330 2023-07-13T22:21:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.860+0000 7fa8623ff700 1 -- start start 2023-07-13T22:21:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.862+0000 7fa8623ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 0x7fa85c14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.862+0000 7fa8623ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa85c155390 0x7fa85c155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.862+0000 7fa8623ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa85c06d890 0x7fa85c06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.862+0000 7fa8623ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c150890 con 0x7fa85c155390 2023-07-13T22:21:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.862+0000 7fa8623ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c155d40 con 0x7fa85c06d890 2023-07-13T22:21:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.862+0000 7fa8623ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0aef10 con 0x7fa84400b5e0 2023-07-13T22:21:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fa86099c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa85c06d890 0x7fa85c06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fa86099c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa85c06d890 0x7fa85c06dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55004/0 (socket says 172.21.15.60:55004) 2023-07-13T22:21:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fa86099c700 1 -- 172.21.15.60:0/3147039826 learned_addr learned my addr 172.21.15.60:0/3147039826 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fa85bfff700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 0x7fa85c14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fcc3ed9d700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcc2c06adb0 0x7fcc2c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7f47da3ff700 1 -- 172.21.15.60:0/3938523425 shutdown_connections 2023-07-13T22:21:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fcc3ed9d700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcc2c06adb0 0x7fcc2c06d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc380090b0 tx=0x7fcc38010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fa86099c700 1 -- 172.21.15.60:0/3147039826 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 msgr2=0x7fa85c14fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fa86099c700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 0x7fa85c14fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fa86099c700 1 -- 172.21.15.60:0/3147039826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa85c155390 msgr2=0x7fa85c155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fa86099c700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa85c155390 0x7fa85c155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fa86099c700 1 -- 172.21.15.60:0/3147039826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0ae370 con 0x7fa85c06d890 2023-07-13T22:21:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fa86099c700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa85c06d890 0x7fa85c06dd00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa854017040 tx=0x7fa854004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.861+0000 7fe0dc1ff700 1 -- start start 2023-07-13T22:21:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fe0dc1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe0d40c3640 0x7fe0d4069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fe0dc1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0d406a410 0x7fe0d4160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fe0dc1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d406a830 con 0x7fe0d4160870 2023-07-13T22:21:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fe0dc1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d40cd070 con 0x7fe0d406a410 2023-07-13T22:21:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7fe0dc1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d41643b0 con 0x7fe0d40c3640 2023-07-13T22:21:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fe0d979a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe0d40c3640 0x7fe0d4069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fe0d979a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe0d40c3640 0x7fe0d4069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55018/0 (socket says 172.21.15.60:55018) 2023-07-13T22:21:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.864+0000 7fe0d979a700 1 -- 172.21.15.60:0/1062821014 learned_addr learned my addr 172.21.15.60:0/1062821014 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.865+0000 7fe0d979a700 1 -- 172.21.15.60:0/1062821014 >> v2:172.21.15.60:3300/0 conn(0x7fe0d406a410 msgr2=0x7fe0d4160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.865+0000 7fe0d979a700 1 --2- 172.21.15.60:0/1062821014 >> v2:172.21.15.60:3300/0 conn(0x7fe0d406a410 0x7fe0d4160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.865+0000 7fe0d979a700 1 -- 172.21.15.60:0/1062821014 >> v1:172.21.15.60:6789/0 conn(0x7fe0d4160870 legacy=0x7fe0d40adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.865+0000 7fe0d979a700 1 -- 172.21.15.60:0/1062821014 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d4164690 con 0x7fe0d40c3640 2023-07-13T22:21:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.865+0000 7fe0d979a700 1 --2- 172.21.15.60:0/1062821014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0d40c3640 0x7fe0d4069ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40c8cd0 tx=0x7fe0c4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd1453ea4ed4b48c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.865+0000 7fe1927fc700 1 -- 172.21.15.60:0/3300092811 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe198003bb0 con 0x7fe19c06a5e0 2023-07-13T22:21:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.863+0000 7f47da3ff700 1 -- 172.21.15.60:0/3938523425 wait complete. 2023-07-13T22:21:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.866+0000 7f47da3ff700 1 Processor -- start 2023-07-13T22:21:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.866+0000 7f47da3ff700 1 -- start start 2023-07-13T22:21:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.866+0000 7fe1927fc700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe18006adb0 0x7fe18006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.866+0000 7fe1927fc700 1 -- 172.21.15.60:0/3300092811 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe19808aac0 con 0x7fe19c06a5e0 2023-07-13T22:21:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7f1b1f3ff700 1 Processor -- start 2023-07-13T22:21:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7fe1a179a700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe18006adb0 0x7fe18006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.868+0000 7fcc277fe700 1 -- 172.21.15.60:0/2793074930 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcc30054a60 con 0x7fcc400c5330 2023-07-13T22:21:01.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.868+0000 7fe1927fc700 1 -- 172.21.15.60:0/3300092811 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe198055c70 con 0x7fe19c06a5e0 2023-07-13T22:21:01.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7fa858ff9700 1 -- 172.21.15.60:0/3147039826 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854026070 con 0x7fa85c06d890 2023-07-13T22:21:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c06e240 con 0x7fa85c06d890 2023-07-13T22:21:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.868+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c0ae8c0 con 0x7fa85c06d890 2023-07-13T22:21:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.868+0000 7fa858ff9700 1 -- 172.21.15.60:0/3147039826 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854021680 con 0x7fa85c06d890 2023-07-13T22:21:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.868+0000 7fa858ff9700 1 -- 172.21.15.60:0/3147039826 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854021a10 con 0x7fa85c06d890 2023-07-13T22:21:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7f47da3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 0x7f47d4156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7f47da3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d40cfad0 0x7f47d40b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7f47da3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40d3750 0x7f47d40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7f47da3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4157220 con 0x7f47d40cfad0 2023-07-13T22:21:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7f47da3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41f1cc0 con 0x7f47d40b8400 2023-07-13T22:21:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7f47da3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41f1fa0 con 0x7f47d40d3750 2023-07-13T22:21:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.868+0000 7f47d37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d40cfad0 0x7f47d40b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.868+0000 7f47d37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d40cfad0 0x7f47d40b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34080/0 (socket says 172.21.15.60:34080) 2023-07-13T22:21:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.868+0000 7f47d37fe700 1 -- 172.21.15.60:0/4168987903 learned_addr learned my addr 172.21.15.60:0/4168987903 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f47d3fff700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 0x7f47d4156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f47d899c700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40d3750 0x7f47d40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.867+0000 7f1b1f3ff700 1 -- start start 2023-07-13T22:21:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f1b1f3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1b180b8400 0x7f1b180cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f1b1f3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1b180d3750 0x7f1b180cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f1b1f3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180cfad0 0x7f1b1815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f1b1f3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b180cff50 con 0x7f1b180b8400 2023-07-13T22:21:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f1b1f3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b180d0ac0 con 0x7f1b180d3750 2023-07-13T22:21:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f1b1f3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180b87d0 con 0x7f1b180cfad0 2023-07-13T22:21:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7efc60bff700 1 Processor -- start 2023-07-13T22:21:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f1b1c99a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1b180d3750 0x7f1b180cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7efc60bff700 1 -- start start 2023-07-13T22:21:01.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f1b1c99a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1b180d3750 0x7f1b180cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55038/0 (socket says 172.21.15.60:55038) 2023-07-13T22:21:01.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7f1b1c99a700 1 -- 172.21.15.60:0/1510217387 learned_addr learned my addr 172.21.15.60:0/1510217387 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7fe1a179a700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe18006adb0 0x7fe18006d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0c87c0 tx=0x7fe194009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.869+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8380052f0 con 0x7fa85c06d890 2023-07-13T22:21:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f47d3fff700 1 -- 172.21.15.60:0/4168987903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40d3750 msgr2=0x7f47d40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f47d3fff700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40d3750 0x7f47d40b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f47d3fff700 1 -- 172.21.15.60:0/4168987903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d40cfad0 msgr2=0x7f47d40b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f47d3fff700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d40cfad0 0x7f47d40b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f47d3fff700 1 -- 172.21.15.60:0/4168987903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40094a0 con 0x7f47d40b8400 2023-07-13T22:21:01.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7f47d3fff700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 0x7f47d4156ce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47c4016000 tx=0x7f47c4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7fe0d8f99700 1 -- 172.21.15.60:0/1062821014 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4004230 con 0x7fe0d40c3640 2023-07-13T22:21:01.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7fe0d8f99700 1 -- 172.21.15.60:0/1062821014 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c4004390 con 0x7fe0d40c3640 2023-07-13T22:21:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7fe0d8f99700 1 -- 172.21.15.60:0/1062821014 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40036a0 con 0x7fe0d40c3640 2023-07-13T22:21:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/1062821014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0d40c3640 msgr2=0x7fe0d4069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7fe0dc1ff700 1 --2- 172.21.15.60:0/1062821014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0d40c3640 0x7fe0d4069ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40c8cd0 tx=0x7fe0c4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/1062821014 shutdown_connections 2023-07-13T22:21:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7fe0dc1ff700 1 --2- 172.21.15.60:0/1062821014 >> v2:172.21.15.60:3300/0 conn(0x7fe0d406a410 0x7fe0d4160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7fe0dc1ff700 1 --2- 172.21.15.60:0/1062821014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0d40c3640 0x7fe0d4069ed0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/1062821014 >> 172.21.15.60:0/1062821014 conn(0x7fe0d40cf410 msgr2=0x7fe0d40cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/1062821014 shutdown_connections 2023-07-13T22:21:01.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f1b1c99a700 1 -- 172.21.15.60:0/1510217387 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180cfad0 msgr2=0x7f1b1815fef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:01.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f1b1c99a700 1 --2- 172.21.15.60:0/1510217387 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180cfad0 0x7f1b1815fef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f1b1c99a700 1 -- 172.21.15.60:0/1510217387 >> v2:172.21.15.60:3300/0 conn(0x7f1b180b8400 msgr2=0x7f1b180cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f1b1c99a700 1 --2- 172.21.15.60:0/1510217387 >> v2:172.21.15.60:3300/0 conn(0x7f1b180b8400 0x7f1b180cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.870+0000 7f1b1c99a700 1 -- 172.21.15.60:0/1510217387 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180b4eb0 con 0x7f1b180d3750 2023-07-13T22:21:01.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7f1b1c99a700 1 --2- 172.21.15.60:0/1510217387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b180d3750 0x7f1b180cf590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b180b8cf0 tx=0x7f1b04009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f70e2dd3aba02f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7f47d0ff9700 1 -- 172.21.15.60:0/4168987903 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4004510 con 0x7f47d40b8400 2023-07-13T22:21:01.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7f47d0ff9700 1 -- 172.21.15.60:0/4168987903 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c401e070 con 0x7f47d40b8400 2023-07-13T22:21:01.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7f47d0ff9700 1 -- 172.21.15.60:0/4168987903 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40038c0 con 0x7f47d40b8400 2023-07-13T22:21:01.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7f1b17fff700 1 -- 172.21.15.60:0/1510217387 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400b870 con 0x7f1b180d3750 2023-07-13T22:21:01.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7f1b17fff700 1 -- 172.21.15.60:0/1510217387 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0400b9d0 con 0x7f1b180d3750 2023-07-13T22:21:01.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7f1b17fff700 1 -- 172.21.15.60:0/1510217387 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400bd60 con 0x7f1b180d3750 2023-07-13T22:21:01.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.872+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/1510217387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b180d3750 msgr2=0x7f1b180cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.872+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/1510217387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b180d3750 0x7f1b180cf590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b180b8cf0 tx=0x7f1b04009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/1062821014 wait complete. 2023-07-13T22:21:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7efc60bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc5c0d3750 0x7efc5c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7efc60bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc5c0cfad0 0x7efc5c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7efc60bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c0cd950 con 0x7efc5c0b8400 2023-07-13T22:21:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7efc60bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c0cff20 con 0x7efc5c0cfad0 2023-07-13T22:21:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.871+0000 7efc60bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0d0ac0 con 0x7efc5c0d3750 2023-07-13T22:21:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.872+0000 7fa858ff9700 1 -- 172.21.15.60:0/3147039826 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa854021b70 con 0x7fa85c06d890 2023-07-13T22:21:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.873+0000 7efc5ad9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc5c0cfad0 0x7efc5c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.873+0000 7efc5ad9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc5c0cfad0 0x7efc5c0d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55048/0 (socket says 172.21.15.60:55048) 2023-07-13T22:21:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.873+0000 7efc5ad9d700 1 -- 172.21.15.60:0/2397472758 learned_addr learned my addr 172.21.15.60:0/2397472758 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.874+0000 7efc5ad9d700 1 -- 172.21.15.60:0/2397472758 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc5c0d3750 msgr2=0x7efc5c0cf590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.874+0000 7efc5ad9d700 1 --2- 172.21.15.60:0/2397472758 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc5c0d3750 0x7efc5c0cf590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.874+0000 7efc5ad9d700 1 -- 172.21.15.60:0/2397472758 >> v1:172.21.15.60:6789/0 conn(0x7efc5c0b8400 legacy=0x7efc5c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:01.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.874+0000 7efc5ad9d700 1 -- 172.21.15.60:0/2397472758 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0b87d0 con 0x7efc5c0cfad0 2023-07-13T22:21:01.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.874+0000 7fe0dc1ff700 1 Processor -- start 2023-07-13T22:21:01.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.875+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/1510217387 shutdown_connections 2023-07-13T22:21:01.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.875+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/1510217387 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180cfad0 0x7f1b1815fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.875+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/1510217387 >> v2:172.21.15.60:3300/0 conn(0x7f1b180b8400 0x7f1b180cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.875+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/1510217387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b180d3750 0x7f1b180cf590 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.875+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/1510217387 >> 172.21.15.60:0/1510217387 conn(0x7f1b1806b390 msgr2=0x7f1b1806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.874+0000 7efc5ad9d700 1 --2- 172.21.15.60:0/2397472758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc5c0cfad0 0x7efc5c0d0f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098f0 tx=0x7efc4c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb7d3f1149b4a2d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.875+0000 7fa858ff9700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006acb0 0x7fa84006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.875+0000 7fa858ff9700 1 -- 172.21.15.60:0/3147039826 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa854011070 con 0x7fa85c06d890 2023-07-13T22:21:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.875+0000 7fa85bfff700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006acb0 0x7fa84006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.875+0000 7fe0dc1ff700 1 -- start start 2023-07-13T22:21:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/1510217387 shutdown_connections 2023-07-13T22:21:01.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7fe0dc1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0d406a410 0x7fe0d4140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/1510217387 wait complete. 2023-07-13T22:21:01.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7fe0dc1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0d40c3640 0x7fe0d415e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7fe0dc1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d4160870 0x7fe0d415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7fe0dc1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d4138cb0 con 0x7fe0d4160870 2023-07-13T22:21:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7fe0dc1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d4141420 con 0x7fe0d406a410 2023-07-13T22:21:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7fe0dc1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d4141700 con 0x7fe0d40c3640 2023-07-13T22:21:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7fe0da79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d4160870 0x7fe0d415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.873+0000 7f47da3ff700 1 -- 172.21.15.60:0/4168987903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d41f2280 con 0x7f47d40b8400 2023-07-13T22:21:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.873+0000 7f47da3ff700 1 -- 172.21.15.60:0/4168987903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d41f2840 con 0x7f47d40b8400 2023-07-13T22:21:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7fe0da79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d4160870 0x7fe0d415a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34086/0 (socket says 172.21.15.60:34086) 2023-07-13T22:21:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.876+0000 7fe0da79c700 1 -- 172.21.15.60:0/2497671025 learned_addr learned my addr 172.21.15.60:0/2497671025 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.877+0000 7fa85bfff700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006acb0 0x7fa84006d160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa84c007fd0 tx=0x7fa84c012040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.877+0000 7f47ba7fc700 1 -- 172.21.15.60:0/4168987903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d415a8a0 con 0x7f47d40b8400 2023-07-13T22:21:01.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.877+0000 7fe0da79c700 1 -- 172.21.15.60:0/2497671025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0d40c3640 msgr2=0x7fe0d415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.877+0000 7fe0da79c700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0d40c3640 0x7fe0d415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.877+0000 7fe0da79c700 1 -- 172.21.15.60:0/2497671025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0d406a410 msgr2=0x7fe0d4140e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:01.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.877+0000 7fe0da79c700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0d406a410 0x7fe0d4140e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.877+0000 7fe0da79c700 1 -- 172.21.15.60:0/2497671025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c4009550 con 0x7fe0d4160870 2023-07-13T22:21:01.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.878+0000 7fe0da79c700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d4160870 0x7fe0d415a8e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00b550 tx=0x7fe0cc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.878+0000 7fa858ff9700 1 -- 172.21.15.60:0/3147039826 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa854055f80 con 0x7fa85c06d890 2023-07-13T22:21:01.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.878+0000 7efc5959a700 1 -- 172.21.15.60:0/2397472758 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c004230 con 0x7efc5c0cfad0 2023-07-13T22:21:01.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.878+0000 7efc5959a700 1 -- 172.21.15.60:0/2397472758 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c004390 con 0x7efc5c0cfad0 2023-07-13T22:21:01.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.878+0000 7efc5959a700 1 -- 172.21.15.60:0/2397472758 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c0036a0 con 0x7efc5c0cfad0 2023-07-13T22:21:01.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.878+0000 7fe0ca7fc700 1 -- 172.21.15.60:0/2497671025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc011830 con 0x7fe0d4160870 2023-07-13T22:21:01.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.878+0000 7fe0ca7fc700 1 -- 172.21.15.60:0/2497671025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0cc0119b0 con 0x7fe0d4160870 2023-07-13T22:21:01.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.878+0000 7fe0ca7fc700 1 -- 172.21.15.60:0/2497671025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc011d60 con 0x7fe0d4160870 2023-07-13T22:21:01.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.879+0000 7f1b1f3ff700 1 Processor -- start 2023-07-13T22:21:01.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.879+0000 7efc60bff700 1 -- 172.21.15.60:0/2397472758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc5c0cfad0 msgr2=0x7efc5c0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.879+0000 7efc60bff700 1 --2- 172.21.15.60:0/2397472758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc5c0cfad0 0x7efc5c0d0f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098f0 tx=0x7efc4c009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:01.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.879+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d413f9c0 con 0x7fe0d4160870 2023-07-13T22:21:01.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.879+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d414a8a0 con 0x7fe0d4160870 2023-07-13T22:21:01.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.880+0000 7f47d0ff9700 1 -- 172.21.15.60:0/4168987903 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47c4003a20 con 0x7f47d40b8400 2023-07-13T22:21:01.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.879+0000 7f1b1f3ff700 1 -- start start 2023-07-13T22:21:01.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.881+0000 7f1b1f3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b180b8400 0x7f1b18148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.881+0000 7f1b1f3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b180cfad0 0x7f1b180b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.881+0000 7f1b1f3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180d3750 0x7f1b180b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.881+0000 7f1b1f3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b18148860 con 0x7f1b180b8400 2023-07-13T22:21:01.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.881+0000 7f1b1f3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180ca1c0 con 0x7f1b180cfad0 2023-07-13T22:21:01.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.881+0000 7f1b1f3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180ca4a0 con 0x7f1b180d3750 2023-07-13T22:21:01.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.881+0000 7f47d0ff9700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47c006adb0 0x7f47c006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.881+0000 7f47d0ff9700 1 -- 172.21.15.60:0/4168987903 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47c408a540 con 0x7f47d40b8400 2023-07-13T22:21:01.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.883+0000 7f47d0ff9700 1 -- 172.21.15.60:0/4168987903 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f47c4055660 con 0x7f47d40b8400 2023-07-13T22:21:01.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.881+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d413fee0 con 0x7fe0d4160870 2023-07-13T22:21:01.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.882+0000 7f1b1d99c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180d3750 0x7f1b180b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.882+0000 7f1b1d99c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180d3750 0x7f1b180b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:49438/0 (socket says 172.21.15.60:49438) 2023-07-13T22:21:01.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.882+0000 7f1b1d99c700 1 -- 172.21.15.60:0/3672630140 learned_addr learned my addr 172.21.15.60:0/3672630140 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.882+0000 7f1b1d99c700 1 -- 172.21.15.60:0/3672630140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b180cfad0 msgr2=0x7f1b180b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.882+0000 7f1b1d99c700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b180cfad0 0x7f1b180b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.882+0000 7f1b1d99c700 1 -- 172.21.15.60:0/3672630140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b180b8400 msgr2=0x7f1b18148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.882+0000 7f1b1d99c700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b180b8400 0x7f1b18148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.882+0000 7f1b1d99c700 1 -- 172.21.15.60:0/3672630140 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b04009550 con 0x7f1b180d3750 2023-07-13T22:21:01.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.883+0000 7f1b1d99c700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180d3750 0x7f1b180b33e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000b3e0 tx=0x7f1b1000b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.885+0000 7f47d37fe700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47c006adb0 0x7f47c006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.886+0000 7f47d37fe700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47c006adb0 0x7f47c006d260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47d415bbe0 tx=0x7f47c8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.885+0000 7f1b15ffb700 1 -- 172.21.15.60:0/3672630140 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10011830 con 0x7f1b180d3750 2023-07-13T22:21:01.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.885+0000 7f1b15ffb700 1 -- 172.21.15.60:0/3672630140 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b10011990 con 0x7f1b180d3750 2023-07-13T22:21:01.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.885+0000 7f1b15ffb700 1 -- 172.21.15.60:0/3672630140 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10011d40 con 0x7f1b180d3750 2023-07-13T22:21:01.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.885+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b180ca780 con 0x7f1b180d3750 2023-07-13T22:21:01.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.885+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b180cad40 con 0x7f1b180d3750 2023-07-13T22:21:01.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.887+0000 7fe0ca7fc700 1 -- 172.21.15.60:0/2497671025 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe0cc01a400 con 0x7fe0d4160870 2023-07-13T22:21:01.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.889+0000 7fe0ca7fc700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0b806adb0 0x7fe0b806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.889+0000 7fe0ca7fc700 1 -- 172.21.15.60:0/2497671025 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0cc089610 con 0x7fe0d4160870 2023-07-13T22:21:01.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.887+0000 7efc60bff700 1 -- 172.21.15.60:0/2397472758 shutdown_connections 2023-07-13T22:21:01.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.887+0000 7efc60bff700 1 --2- 172.21.15.60:0/2397472758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc5c0cfad0 0x7efc5c0d0f50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.887+0000 7efc60bff700 1 --2- 172.21.15.60:0/2397472758 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc5c0d3750 0x7efc5c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.887+0000 7efc60bff700 1 -- 172.21.15.60:0/2397472758 >> 172.21.15.60:0/2397472758 conn(0x7efc5c06b390 msgr2=0x7efc5c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:01.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.890+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af80052f0 con 0x7f1b180d3750 2023-07-13T22:21:01.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.891+0000 7efc60bff700 1 -- 172.21.15.60:0/2397472758 shutdown_connections 2023-07-13T22:21:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.891+0000 7efc60bff700 1 -- 172.21.15.60:0/2397472758 wait complete. 2023-07-13T22:21:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.895+0000 7fe0ca7fc700 1 -- 172.21.15.60:0/2497671025 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe0d413fee0 con 0x7fe0d4160870 2023-07-13T22:21:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.894+0000 7f1b15ffb700 1 -- 172.21.15.60:0/3672630140 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b1001a400 con 0x7f1b180d3750 2023-07-13T22:21:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.895+0000 7f1b15ffb700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1afc06adb0 0x7f1afc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.895+0000 7f1b15ffb700 1 -- 172.21.15.60:0/3672630140 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b10089440 con 0x7f1b180d3750 2023-07-13T22:21:01.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.894+0000 7efc60bff700 1 Processor -- start 2023-07-13T22:21:01.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.894+0000 7efc60bff700 1 -- start start 2023-07-13T22:21:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc60bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc5c0b8400 0x7efc5c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc60bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc5c0cfad0 0x7efc5c151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc60bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc5c0d3750 0x7efc5c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc60bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0c6a30 con 0x7efc5c0b8400 2023-07-13T22:21:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc60bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c15cf40 con 0x7efc5c0cfad0 2023-07-13T22:21:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc60bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0c90f0 con 0x7efc5c0d3750 2023-07-13T22:21:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc5a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc5c0b8400 0x7efc5c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc5a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc5c0b8400 0x7efc5c151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34098/0 (socket says 172.21.15.60:34098) 2023-07-13T22:21:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc5a59c700 1 -- 172.21.15.60:0/3211752842 learned_addr learned my addr 172.21.15.60:0/3211752842 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.896+0000 7efc5ad9d700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc5c0d3750 0x7efc5c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.897+0000 7efc5a59c700 1 -- 172.21.15.60:0/3211752842 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc5c0d3750 msgr2=0x7efc5c151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.897+0000 7efc5a59c700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc5c0d3750 0x7efc5c151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.897+0000 7efc5a59c700 1 -- 172.21.15.60:0/3211752842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc5c0cfad0 msgr2=0x7efc5c151840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.897+0000 7efc5a59c700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc5c0cfad0 0x7efc5c151840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:01.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.897+0000 7efc5a59c700 1 -- 172.21.15.60:0/3211752842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c009550 con 0x7efc5c0b8400 2023-07-13T22:21:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.897+0000 7efc5a59c700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc5c0b8400 0x7efc5c151220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b470 tx=0x7efc5000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.898+0000 7f1b1d19b700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1afc06adb0 0x7f1afc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.899+0000 7f1b15ffb700 1 -- 172.21.15.60:0/3672630140 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1b100544d0 con 0x7f1b180d3750 2023-07-13T22:21:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.901+0000 7fe0d9f9b700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0b806adb0 0x7fe0b806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.902+0000 7fe0d9f9b700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0b806adb0 0x7fe0b806d260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000ab90 tx=0x7fe0d0009230 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.907+0000 7f1b1d19b700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1afc06adb0 0x7f1afc06d260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b180c6220 tx=0x7f1b0c009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.908+0000 7efc4affd700 1 -- 172.21.15.60:0/3211752842 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50011830 con 0x7efc5c0b8400 2023-07-13T22:21:01.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.908+0000 7efc4affd700 1 -- 172.21.15.60:0/3211752842 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50011990 con 0x7efc5c0b8400 2023-07-13T22:21:01.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.908+0000 7efc4affd700 1 -- 172.21.15.60:0/3211752842 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50011d20 con 0x7efc5c0b8400 2023-07-13T22:21:01.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.908+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c0c9390 con 0x7efc5c0b8400 2023-07-13T22:21:01.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.908+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c15d430 con 0x7efc5c0b8400 2023-07-13T22:21:01.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.913+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc400052f0 con 0x7efc5c0b8400 2023-07-13T22:21:01.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.920+0000 7efc4affd700 1 -- 172.21.15.60:0/3211752842 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc5001a410 con 0x7efc5c0b8400 2023-07-13T22:21:01.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.921+0000 7efc4affd700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc3c06ade0 0x7efc3c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:01.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.921+0000 7efc4affd700 1 -- 172.21.15.60:0/3211752842 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc50088ef0 con 0x7efc5c0b8400 2023-07-13T22:21:01.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.923+0000 7efc59d9b700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc3c06ade0 0x7efc3c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:01.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.924+0000 7efc59d9b700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc3c06ade0 0x7efc3c06d290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efc5c151760 tx=0x7efc54009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:01.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:01.925+0000 7efc4affd700 1 -- 172.21.15.60:0/3211752842 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efc50053f80 con 0x7efc5c0b8400 2023-07-13T22:21:02.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.307+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f97dc0055a0 con 0x7f97fc0c4e10 2023-07-13T22:21:02.296 INFO:teuthology.orchestra.run.smithi060.stdout:55834574852 2023-07-13T22:21:02.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.308+0000 7f97f1ffb700 1 -- 172.21.15.60:0/1344238456 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f97f8015070 con 0x7f97fc0c4e10 2023-07-13T22:21:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.321+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e006adb0 msgr2=0x7f97e006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.321+0000 7f98039ff700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e006adb0 0x7f97e006d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97fc151100 tx=0x7f97ec009040 comp rx=0 tx=0).stop 2023-07-13T22:21:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.321+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc0c4e10 msgr2=0x7f97fc150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.321+0000 7f98039ff700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc0c4e10 0x7f97fc150bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009b60 tx=0x7f97f8004750 comp rx=0 tx=0).stop 2023-07-13T22:21:02.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.322+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 shutdown_connections 2023-07-13T22:21:02.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.322+0000 7f98039ff700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e006adb0 0x7f97e006d260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.322+0000 7f98039ff700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97fc160920 0x7f97fc13cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.322+0000 7f98039ff700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc0c4e10 0x7f97fc150bc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.322+0000 7f98039ff700 1 --2- 172.21.15.60:0/1344238456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0c5330 0x7f97fc141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.322+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 >> 172.21.15.60:0/1344238456 conn(0x7f97fc0cb200 msgr2=0x7f97fc0c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:02.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.322+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 shutdown_connections 2023-07-13T22:21:02.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.323+0000 7f98039ff700 1 -- 172.21.15.60:0/1344238456 wait complete. 2023-07-13T22:21:02.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.323+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe19c0cbe60 con 0x7fe19c06a5e0 2023-07-13T22:21:02.312 INFO:teuthology.orchestra.run.smithi060.stdout:55834574852 2023-07-13T22:21:02.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.324+0000 7fe1927fc700 1 -- 172.21.15.60:0/3300092811 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe198003e60 con 0x7fe19c06a5e0 2023-07-13T22:21:02.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.331+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcc400c3820 con 0x7fcc400c5330 2023-07-13T22:21:02.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.331+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe18006adb0 msgr2=0x7fe18006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.331+0000 7fe1a41ff700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe18006adb0 0x7fe18006d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0c87c0 tx=0x7fe194009040 comp rx=0 tx=0).stop 2023-07-13T22:21:02.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.331+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe19c06a5e0 msgr2=0x7fe19c0c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.331+0000 7fe1a41ff700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe19c06a5e0 0x7fe19c0c7550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe198014040 tx=0x7fe198004750 comp rx=0 tx=0).stop 2023-07-13T22:21:02.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.332+0000 7fcc277fe700 1 -- 172.21.15.60:0/2793074930 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcc30058080 con 0x7fcc400c5330 2023-07-13T22:21:02.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.332+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 shutdown_connections 2023-07-13T22:21:02.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.332+0000 7fe1a41ff700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe18006adb0 0x7fe18006d260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.332+0000 7fe1a41ff700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c160870 0x7fe19c154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.332+0000 7fe1a41ff700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe19c06a5e0 0x7fe19c0c7550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.332+0000 7fe1a41ff700 1 --2- 172.21.15.60:0/3300092811 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe19c0c3640 0x7fe19c0b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.332+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 >> 172.21.15.60:0/3300092811 conn(0x7fe19c0cf410 msgr2=0x7fe19c0c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:02.319 INFO:teuthology.orchestra.run.smithi060.stdout:55834574852 2023-07-13T22:21:02.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.333+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 shutdown_connections 2023-07-13T22:21:02.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.333+0000 7fe1a41ff700 1 -- 172.21.15.60:0/3300092811 wait complete. 2023-07-13T22:21:02.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.336+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcc2c06adb0 msgr2=0x7fcc2c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.336+0000 7fcc451ff700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcc2c06adb0 0x7fcc2c06d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc380090b0 tx=0x7fcc38010000 comp rx=0 tx=0).stop 2023-07-13T22:21:02.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.337+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 msgr2=0x7fcc401ea510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.337+0000 7fcc451ff700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc401ea510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000efc0 tx=0x7fcc3000c3a0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.338+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 shutdown_connections 2023-07-13T22:21:02.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.338+0000 7fcc451ff700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcc2c06adb0 0x7fcc2c06d260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.338+0000 7fcc451ff700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc4006e630 0x7fcc400b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.338+0000 7fcc451ff700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc40160920 0x7fcc401eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.338+0000 7fcc451ff700 1 --2- 172.21.15.60:0/2793074930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc400c5330 0x7fcc401ea510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.338+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 >> 172.21.15.60:0/2793074930 conn(0x7fcc400cb200 msgr2=0x7fcc4006d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.338+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 shutdown_connections 2023-07-13T22:21:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.338+0000 7fcc451ff700 1 -- 172.21.15.60:0/2793074930 wait complete. 2023-07-13T22:21:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.352+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1af80055a0 con 0x7f1b180d3750 2023-07-13T22:21:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.352+0000 7f47ba7fc700 1 -- 172.21.15.60:0/4168987903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f47d40d2590 con 0x7f47d40b8400 2023-07-13T22:21:02.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.353+0000 7f47d0ff9700 1 -- 172.21.15.60:0/4168987903 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47c4058c80 con 0x7f47d40b8400 2023-07-13T22:21:02.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.353+0000 7f1b15ffb700 1 -- 172.21.15.60:0/3672630140 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b10057af0 con 0x7f1b180d3750 2023-07-13T22:21:02.341 INFO:teuthology.orchestra.run.smithi060.stdout:55834574852 2023-07-13T22:21:02.341 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-07-13T22:21:02.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.4 2023-07-13T22:21:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.356+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1afc06adb0 msgr2=0x7f1afc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.356+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1afc06adb0 0x7f1afc06d260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b180c6220 tx=0x7f1b0c009250 comp rx=0 tx=0).stop 2023-07-13T22:21:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.356+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180d3750 msgr2=0x7f1b180b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.356+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180d3750 0x7f1b180b33e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000b3e0 tx=0x7f1b1000b7a0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 shutdown_connections 2023-07-13T22:21:02.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1afc06adb0 0x7f1afc06d260 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b180c6220 tx=0x7f1b0c009250 comp rx=0 tx=0).stop 2023-07-13T22:21:02.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b180b8400 0x7f1b18148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b180d3750 0x7f1b180b33e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f1b1f3ff700 1 --2- 172.21.15.60:0/3672630140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b180cfad0 0x7f1b180b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 >> 172.21.15.60:0/3672630140 conn(0x7f1b1806b390 msgr2=0x7f1b180d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:02.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 -- 172.21.15.60:0/4168987903 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47c006adb0 msgr2=0x7f47c006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47c006adb0 0x7f47c006d260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47d415bbe0 tx=0x7f47c8009190 comp rx=0 tx=0).stop 2023-07-13T22:21:02.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 -- 172.21.15.60:0/4168987903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 msgr2=0x7f47d4156ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 0x7f47d4156ce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47c4016000 tx=0x7f47c4004750 comp rx=0 tx=0).stop 2023-07-13T22:21:02.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 -- 172.21.15.60:0/4168987903 shutdown_connections 2023-07-13T22:21:02.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47c006adb0 0x7f47c006d260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d40d3750 0x7f47d40b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d40b8400 0x7f47d4156ce0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 --2- 172.21.15.60:0/4168987903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d40cfad0 0x7f47d40b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.357+0000 7f47ba7fc700 1 -- 172.21.15.60:0/4168987903 >> 172.21.15.60:0/4168987903 conn(0x7f47d406b390 msgr2=0x7f47d415b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.358+0000 7f47ba7fc700 1 -- 172.21.15.60:0/4168987903 shutdown_connections 2023-07-13T22:21:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.358+0000 7f47ba7fc700 1 -- 172.21.15.60:0/4168987903 wait complete. 2023-07-13T22:21:02.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.359+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 shutdown_connections 2023-07-13T22:21:02.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.359+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa8380055a0 con 0x7fa85c06d890 2023-07-13T22:21:02.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.360+0000 7fa858ff9700 1 -- 172.21.15.60:0/3147039826 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa85405de60 con 0x7fa85c06d890 2023-07-13T22:21:02.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.361+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe0d413fee0 con 0x7fe0d4160870 2023-07-13T22:21:02.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.361+0000 7f1b1f3ff700 1 -- 172.21.15.60:0/3672630140 wait complete. 2023-07-13T22:21:02.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.361+0000 7fe0ca7fc700 1 -- 172.21.15.60:0/2497671025 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0d413fee0 con 0x7fe0d4160870 2023-07-13T22:21:02.347 INFO:teuthology.orchestra.run.smithi060.stdout:55834574851 2023-07-13T22:21:02.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2023-07-13T22:21:02.349 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-07-13T22:21:02.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0b806adb0 msgr2=0x7fe0b806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0b806adb0 0x7fe0b806d260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000ab90 tx=0x7fe0d0009230 comp rx=0 tx=0).stop 2023-07-13T22:21:02.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d4160870 msgr2=0x7fe0d415a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d4160870 0x7fe0d415a8e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00b550 tx=0x7fe0cc00b910 comp rx=0 tx=0).stop 2023-07-13T22:21:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 shutdown_connections 2023-07-13T22:21:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0b806adb0 0x7fe0b806d260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d4160870 0x7fe0d415a8e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0d406a410 0x7fe0d4140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 --2- 172.21.15.60:0/2497671025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0d40c3640 0x7fe0d415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.365+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 >> 172.21.15.60:0/2497671025 conn(0x7fe0d40cf410 msgr2=0x7fe0d406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006acb0 msgr2=0x7fa84006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006acb0 0x7fa84006d160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa84c007fd0 tx=0x7fa84c012040 comp rx=0 tx=0).stop 2023-07-13T22:21:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa85c06d890 msgr2=0x7fa85c06dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa85c06d890 0x7fa85c06dd00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa854017040 tx=0x7fa854004be0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 shutdown_connections 2023-07-13T22:21:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006acb0 0x7fa84006d160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa85c06d890 0x7fa85c06dd00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa84400b5e0 0x7fa85c14fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 --2- 172.21.15.60:0/3147039826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa85c155390 0x7fa85c155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 >> 172.21.15.60:0/3147039826 conn(0x7fa85c0cb200 msgr2=0x7fa85c069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 shutdown_connections 2023-07-13T22:21:02.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.368+0000 7fa8623ff700 1 -- 172.21.15.60:0/3147039826 wait complete. 2023-07-13T22:21:02.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.1 2023-07-13T22:21:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.370+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 shutdown_connections 2023-07-13T22:21:02.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.371+0000 7fe0dc1ff700 1 -- 172.21.15.60:0/2497671025 wait complete. 2023-07-13T22:21:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.379+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efc400055a0 con 0x7efc5c0b8400 2023-07-13T22:21:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.379+0000 7efc4affd700 1 -- 172.21.15.60:0/3211752842 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efc5005be60 con 0x7efc5c0b8400 2023-07-13T22:21:02.364 INFO:teuthology.orchestra.run.smithi060.stdout:55834574852 2023-07-13T22:21:02.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.382+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc3c06ade0 msgr2=0x7efc3c06d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.382+0000 7efc60bff700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc3c06ade0 0x7efc3c06d290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efc5c151760 tx=0x7efc54009250 comp rx=0 tx=0).stop 2023-07-13T22:21:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.382+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc5c0b8400 msgr2=0x7efc5c151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.382+0000 7efc60bff700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc5c0b8400 0x7efc5c151220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b470 tx=0x7efc5000b830 comp rx=0 tx=0).stop 2023-07-13T22:21:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.383+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 shutdown_connections 2023-07-13T22:21:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.383+0000 7efc60bff700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc3c06ade0 0x7efc3c06d290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.383+0000 7efc60bff700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc5c0b8400 0x7efc5c151220 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.383+0000 7efc60bff700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc5c0d3750 0x7efc5c151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.383+0000 7efc60bff700 1 --2- 172.21.15.60:0/3211752842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc5c0cfad0 0x7efc5c151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.383+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 >> 172.21.15.60:0/3211752842 conn(0x7efc5c06b390 msgr2=0x7efc5c0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.383+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 shutdown_connections 2023-07-13T22:21:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:02.383+0000 7efc60bff700 1 -- 172.21.15.60:0/3211752842 wait complete. 2023-07-13T22:21:02.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.5 2023-07-13T22:21:02.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.7 2023-07-13T22:21:02.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.0 2023-07-13T22:21:02.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.6 2023-07-13T22:21:02.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.2 2023-07-13T22:21:03.342 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-13T22:21:03.349 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-13T22:21:03.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-13T22:21:03.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-13T22:21:03.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-13T22:21:03.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-13T22:21:03.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-13T22:21:03.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-13T22:21:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.698+0000 7fc06f3ff700 1 Processor -- start 2023-07-13T22:21:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.698+0000 7fc06f3ff700 1 -- start start 2023-07-13T22:21:03.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.700+0000 7fc06f3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc068160920 0x7fc068069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.700+0000 7fc06f3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc068071090 con 0x7fc0680c5330 2023-07-13T22:21:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.700+0000 7fc06f3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0681648d0 con 0x7fc0680c4e10 2023-07-13T22:21:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.700+0000 7fc06f3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680c9200 con 0x7fc068160920 2023-07-13T22:21:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.702+0000 7fc06d99c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc0680c4e10 0x7fc068070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:49214/0 (socket says 172.21.15.60:49214) 2023-07-13T22:21:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.702+0000 7fc06d99c700 1 -- 172.21.15.60:0/3266021340 learned_addr learned my addr 172.21.15.60:0/3266021340 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.704+0000 7fc06d19b700 1 --2- 172.21.15.60:0/3266021340 >> v2:172.21.15.105:3300/0 conn(0x7fc068160920 0x7fc068069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.705+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929929108 0 0) 0x7fc068071090 con 0x7fc0680c5330 2023-07-13T22:21:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.705+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0540035b0 con 0x7fc0680c5330 2023-07-13T22:21:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.705+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3632734625 0 0) 0x7fc0540035b0 con 0x7fc0680c5330 2023-07-13T22:21:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.705+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 >> v2:172.21.15.105:3300/0 conn(0x7fc068160920 msgr2=0x7fc068069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.705+0000 7fc05ffff700 1 --2- 172.21.15.60:0/3266021340 >> v2:172.21.15.105:3300/0 conn(0x7fc068160920 0x7fc068069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.705+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 >> v1:172.21.15.105:6789/0 conn(0x7fc0680c4e10 legacy=0x7fc068070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.706+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068164bb0 con 0x7fc0680c5330 2023-07-13T22:21:03.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.706+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fc060001670 con 0x7fc0680c5330 2023-07-13T22:21:03.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.706+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 >> v1:172.21.15.60:6789/0 conn(0x7fc0680c5330 legacy=0x7fc068070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.706+0000 7fc05ffff700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc054003a00 0x7fc054003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.706+0000 7fc05ffff700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc054004850 0x7fc054004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.706+0000 7fc05ffff700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0540052d0 0x7fc05401ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.706+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680c9200 con 0x7fc0540052d0 2023-07-13T22:21:03.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.706+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05401d1f0 con 0x7fc054003a00 2023-07-13T22:21:03.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.706+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05401d3b0 con 0x7fc054004850 2023-07-13T22:21:03.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.707+0000 7fc06d99c700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0540052d0 0x7fc05401ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.708+0000 7fc06c99a700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc054004850 0x7fc054004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.708+0000 7fc06d19b700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc054003a00 0x7fc054003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.708+0000 7fc06d99c700 1 -- 172.21.15.60:0/3266021340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc054004850 msgr2=0x7fc054004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.708+0000 7fc06d99c700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc054004850 0x7fc054004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.708+0000 7fc06d99c700 1 -- 172.21.15.60:0/3266021340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc054003a00 msgr2=0x7fc054003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.708+0000 7fc06d99c700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc054003a00 0x7fc054003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.708+0000 7fc06d99c700 1 -- 172.21.15.60:0/3266021340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068164bb0 con 0x7fc0540052d0 2023-07-13T22:21:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.708+0000 7fc06d99c700 1 -- 172.21.15.60:0/3266021340 reap_dead start 2023-07-13T22:21:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.709+0000 7fc06d99c700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0540052d0 0x7fc05401ca20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc0580018b0 tx=0x7fc054005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525f696b22879505 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.709+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058005a20 con 0x7fc0540052d0 2023-07-13T22:21:03.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.709+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc058005ba0 con 0x7fc0540052d0 2023-07-13T22:21:03.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.709+0000 7fc05ffff700 1 -- 172.21.15.60:0/3266021340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0580070d0 con 0x7fc0540052d0 2023-07-13T22:21:03.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.710+0000 7fc06f3ff700 1 -- 172.21.15.60:0/3266021340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0540052d0 msgr2=0x7fc05401ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.710+0000 7fc06f3ff700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0540052d0 0x7fc05401ca20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc0580018b0 tx=0x7fc054005880 comp rx=0 tx=0).stop 2023-07-13T22:21:03.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.711+0000 7fc06f3ff700 1 -- 172.21.15.60:0/3266021340 shutdown_connections 2023-07-13T22:21:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.711+0000 7fc06f3ff700 1 --2- 172.21.15.60:0/3266021340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0540052d0 0x7fc05401ca20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.711+0000 7fc06f3ff700 1 -- 172.21.15.60:0/3266021340 >> 172.21.15.60:0/3266021340 conn(0x7fc0680cb200 msgr2=0x7fc0680cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.715+0000 7fc06f3ff700 1 -- 172.21.15.60:0/3266021340 shutdown_connections 2023-07-13T22:21:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.715+0000 7fc06f3ff700 1 -- 172.21.15.60:0/3266021340 wait complete. 2023-07-13T22:21:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.718+0000 7fc06f3ff700 1 Processor -- start 2023-07-13T22:21:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.718+0000 7fc06f3ff700 1 -- start start 2023-07-13T22:21:03.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.720+0000 7fc06f3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0540052d0 0x7fc0680b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.720+0000 7fc06f3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc06814c000 0x7fc0680ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.720+0000 7fc06f3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0681f14c0 0x7fc0681f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.720+0000 7fc06f3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680b45a0 con 0x7fc06814c000 2023-07-13T22:21:03.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.720+0000 7fc06f3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068161470 con 0x7fc0681f14c0 2023-07-13T22:21:03.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.720+0000 7fc06f3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06814bdb0 con 0x7fc0540052d0 2023-07-13T22:21:03.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.721+0000 7fc06d99c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0681f14c0 0x7fc0681f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.721+0000 7fc06d99c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0681f14c0 0x7fc0681f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55116/0 (socket says 172.21.15.60:55116) 2023-07-13T22:21:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.721+0000 7fc06d99c700 1 -- 172.21.15.60:0/4046144545 learned_addr learned my addr 172.21.15.60:0/4046144545 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.723+0000 7fc06c99a700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc06814c000 0x7fc0680ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.724+0000 7fc06c99a700 1 -- 172.21.15.60:0/4046144545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0540052d0 msgr2=0x7fc0680b38e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:03.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.724+0000 7fc06c99a700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0540052d0 0x7fc0680b38e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.724+0000 7fc06c99a700 1 -- 172.21.15.60:0/4046144545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0681f14c0 msgr2=0x7fc0681f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.724+0000 7fc06c99a700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0681f14c0 0x7fc0681f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.724+0000 7fc06c99a700 1 -- 172.21.15.60:0/4046144545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680b3fe0 con 0x7fc06814c000 2023-07-13T22:21:03.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.725+0000 7fc06c99a700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc06814c000 0x7fc0680ada40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc068063c00 tx=0x7fc06000e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.727+0000 7fc05dffb700 1 -- 172.21.15.60:0/4046144545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600052e0 con 0x7fc06814c000 2023-07-13T22:21:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.727+0000 7fc05dffb700 1 -- 172.21.15.60:0/4046144545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc060005460 con 0x7fc06814c000 2023-07-13T22:21:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.727+0000 7fc05dffb700 1 -- 172.21.15.60:0/4046144545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600038c0 con 0x7fc06814c000 2023-07-13T22:21:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.727+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0680b42c0 con 0x7fc06814c000 2023-07-13T22:21:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.727+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0680ae340 con 0x7fc06814c000 2023-07-13T22:21:03.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.729+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc068160f60 con 0x7fc06814c000 2023-07-13T22:21:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.731+0000 7fc05dffb700 1 -- 172.21.15.60:0/4046144545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc060003a20 con 0x7fc06814c000 2023-07-13T22:21:03.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.733+0000 7fc05dffb700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc04c06adb0 0x7fc04c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.733+0000 7fc05dffb700 1 -- 172.21.15.60:0/4046144545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc060088db0 con 0x7fc06814c000 2023-07-13T22:21:03.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.744+0000 7fc06d19b700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc04c06adb0 0x7fc04c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.745+0000 7fc05dffb700 1 -- 172.21.15.60:0/4046144545 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc068160f60 con 0x7fc06814c000 2023-07-13T22:21:03.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.752+0000 7fc06d19b700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc04c06adb0 0x7fc04c06d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc068068f70 tx=0x7fc064003c80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.761+0000 7f7ea01ff700 1 Processor -- start 2023-07-13T22:21:03.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.761+0000 7f7ea01ff700 1 -- start start 2023-07-13T22:21:03.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.763+0000 7f7ea01ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e98160870 0x7f7e980adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.763+0000 7f7ea01ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7e980c3640 0x7f7e98069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.763+0000 7f7ea01ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e9806a410 0x7f7e98070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.763+0000 7f7ea01ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9806a870 con 0x7f7e980c3640 2023-07-13T22:21:03.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.763+0000 7f7ea01ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980ae060 con 0x7f7e9806a410 2023-07-13T22:21:03.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.763+0000 7f7ea01ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980cd070 con 0x7f7e98160870 2023-07-13T22:21:03.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.765+0000 7f7e9d79a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7e980c3640 0x7f7e98069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.765+0000 7f7e9d79a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7e980c3640 0x7f7e98069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34126/0 (socket says 172.21.15.60:34126) 2023-07-13T22:21:03.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.765+0000 7f7e9d79a700 1 -- 172.21.15.60:0/1014743890 learned_addr learned my addr 172.21.15.60:0/1014743890 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.766+0000 7f7e9d79a700 1 -- 172.21.15.60:0/1014743890 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e98160870 msgr2=0x7f7e980adb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:03.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.766+0000 7f7e9d79a700 1 --2- 172.21.15.60:0/1014743890 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e98160870 0x7f7e980adb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.766+0000 7f7e9d79a700 1 -- 172.21.15.60:0/1014743890 >> v2:172.21.15.105:3300/0 conn(0x7f7e9806a410 msgr2=0x7f7e98070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.766+0000 7f7e9d79a700 1 --2- 172.21.15.60:0/1014743890 >> v2:172.21.15.105:3300/0 conn(0x7f7e9806a410 0x7f7e98070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.766+0000 7f7e9d79a700 1 -- 172.21.15.60:0/1014743890 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e981643b0 con 0x7f7e980c3640 2023-07-13T22:21:03.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.767+0000 7f7e9d79a700 1 --2- 172.21.15.60:0/1014743890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e980c3640 0x7f7e98069ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e980cbb70 tx=0x7f7e9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6db95f379385fc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.767+0000 7f7e9cf99700 1 -- 172.21.15.60:0/1014743890 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000e070 con 0x7f7e980c3640 2023-07-13T22:21:03.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.768+0000 7f7e9cf99700 1 -- 172.21.15.60:0/1014743890 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9000b9c0 con 0x7f7e980c3640 2023-07-13T22:21:03.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.768+0000 7f7e9cf99700 1 -- 172.21.15.60:0/1014743890 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000bd50 con 0x7f7e980c3640 2023-07-13T22:21:03.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.769+0000 7f7ea01ff700 1 -- 172.21.15.60:0/1014743890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e980c3640 msgr2=0x7f7e98069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.769+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/1014743890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e980c3640 0x7f7e98069ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e980cbb70 tx=0x7f7e9000b330 comp rx=0 tx=0).stop 2023-07-13T22:21:03.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.769+0000 7f7ea01ff700 1 -- 172.21.15.60:0/1014743890 shutdown_connections 2023-07-13T22:21:03.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.769+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/1014743890 >> v2:172.21.15.105:3300/0 conn(0x7f7e9806a410 0x7f7e98070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.769+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/1014743890 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e98160870 0x7f7e980adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.769+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/1014743890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e980c3640 0x7f7e98069ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.769+0000 7f7ea01ff700 1 -- 172.21.15.60:0/1014743890 >> 172.21.15.60:0/1014743890 conn(0x7f7e980cf410 msgr2=0x7f7e980cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:03.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.770+0000 7f7ea01ff700 1 -- 172.21.15.60:0/1014743890 shutdown_connections 2023-07-13T22:21:03.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.770+0000 7f7ea01ff700 1 -- 172.21.15.60:0/1014743890 wait complete. 2023-07-13T22:21:03.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.773+0000 7f7ea01ff700 1 Processor -- start 2023-07-13T22:21:03.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.773+0000 7f7ea01ff700 1 -- start start 2023-07-13T22:21:03.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.775+0000 7f7ea01ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9806a410 0x7f7e9814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.775+0000 7f7ea01ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e980c3640 0x7f7e9815b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.775+0000 7f7ea01ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e98160870 0x7f7e981569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.775+0000 7f7ea01ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9815b830 con 0x7f7e9806a410 2023-07-13T22:21:03.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.775+0000 7f7ea01ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e98156f20 con 0x7f7e98160870 2023-07-13T22:21:03.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.775+0000 7f7ea01ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9815be70 con 0x7f7e980c3640 2023-07-13T22:21:03.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.776+0000 7f7e9df9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9806a410 0x7f7e9814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.776+0000 7f7e9df9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9806a410 0x7f7e9814e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34140/0 (socket says 172.21.15.60:34140) 2023-07-13T22:21:03.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.776+0000 7f7e9df9b700 1 -- 172.21.15.60:0/2466263555 learned_addr learned my addr 172.21.15.60:0/2466263555 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.776+0000 7f7e9d79a700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e980c3640 0x7f7e9815b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.777+0000 7f7e9df9b700 1 -- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e980c3640 msgr2=0x7f7e9815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.777+0000 7f7e9df9b700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e980c3640 0x7f7e9815b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.777+0000 7f7e9df9b700 1 -- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e98160870 msgr2=0x7f7e981569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.777+0000 7f7e9e79c700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e98160870 0x7f7e981569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.778+0000 7f7e9df9b700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e98160870 0x7f7e981569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.778+0000 7f7e9df9b700 1 -- 172.21.15.60:0/2466263555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e90009b30 con 0x7f7e9806a410 2023-07-13T22:21:03.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.778+0000 7f7e9df9b700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9806a410 0x7f7e9814e3b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400b470 tx=0x7f7e9400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.779+0000 7f7e8e7fc700 1 -- 172.21.15.60:0/2466263555 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94011830 con 0x7f7e9806a410 2023-07-13T22:21:03.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.779+0000 7f7e8e7fc700 1 -- 172.21.15.60:0/2466263555 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e940119b0 con 0x7f7e9806a410 2023-07-13T22:21:03.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.779+0000 7f7e8e7fc700 1 -- 172.21.15.60:0/2466263555 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94011d60 con 0x7f7e9806a410 2023-07-13T22:21:03.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.779+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e98156550 con 0x7f7e9806a410 2023-07-13T22:21:03.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.779+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9815c0c0 con 0x7f7e9806a410 2023-07-13T22:21:03.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.780+0000 7f7e9e79c700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e98160870 0x7f7e981569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:03.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.780+0000 7f7e9d79a700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e980c3640 0x7f7e9815b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:03.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.781+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9806db20 con 0x7f7e9806a410 2023-07-13T22:21:03.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.788+0000 7fad75fff700 1 Processor -- start 2023-07-13T22:21:03.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.789+0000 7fad75fff700 1 -- start start 2023-07-13T22:21:03.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.789+0000 7f6cbc3ff700 1 Processor -- start 2023-07-13T22:21:03.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.789+0000 7f6cbc3ff700 1 -- start start 2023-07-13T22:21:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.790+0000 7f7e8e7fc700 1 -- 172.21.15.60:0/2466263555 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e9401a400 con 0x7f7e9806a410 2023-07-13T22:21:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.792+0000 7f7e8e7fc700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e7c06adb0 0x7f7e7c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.792+0000 7f7e8e7fc700 1 -- 172.21.15.60:0/2466263555 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e94089bf0 con 0x7f7e9806a410 2023-07-13T22:21:03.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.792+0000 7f7e9d79a700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e7c06adb0 0x7f7e7c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.788+0000 7fc741bff700 1 Processor -- start 2023-07-13T22:21:03.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.788+0000 7fc741bff700 1 -- start start 2023-07-13T22:21:03.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.790+0000 7fc741bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc73c0add90 0x7fc73c16cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.790+0000 7fc741bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c06a5e0 0x7fc73c0c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.791+0000 7fc741bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73c16dbd0 con 0x7fc73c089520 2023-07-13T22:21:03.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.791+0000 7fc741bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73c0c9200 con 0x7fc73c0add90 2023-07-13T22:21:03.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.791+0000 7fc741bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c16cb20 con 0x7fc73c06a5e0 2023-07-13T22:21:03.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.792+0000 7fc73b7fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc73c0add90 0x7fc73c16cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.792+0000 7fc73bfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c06a5e0 0x7fc73c0c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.792+0000 7fc73b7fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc73c0add90 0x7fc73c16cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55146/0 (socket says 172.21.15.60:55146) 2023-07-13T22:21:03.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.792+0000 7fc73b7fe700 1 -- 172.21.15.60:0/2931518730 learned_addr learned my addr 172.21.15.60:0/2931518730 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.791+0000 7fad75fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fad70160870 0x7fad700adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.791+0000 7fad75fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad70160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.791+0000 7fad75fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad700ae060 con 0x7fad700c3640 2023-07-13T22:21:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.791+0000 7fad75fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad700cd070 con 0x7fad70160870 2023-07-13T22:21:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.791+0000 7fad75fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad701643b0 con 0x7fad7006a5e0 2023-07-13T22:21:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.793+0000 7f7e8e7fc700 1 -- 172.21.15.60:0/2466263555 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7e94054ca0 con 0x7f7e9806a410 2023-07-13T22:21:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.793+0000 7fad6f7fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fad70160870 0x7fad700adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.793+0000 7fad6effd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fad700c3640 0x7fad70069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:49216/0 (socket says 172.21.15.60:49216) 2023-07-13T22:21:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.793+0000 7fad6effd700 1 -- 172.21.15.60:0/2162436402 learned_addr learned my addr 172.21.15.60:0/2162436402 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.793+0000 7fad6ffff700 1 --2- 172.21.15.60:0/2162436402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad70160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.793+0000 7fc73bfff700 1 -- 172.21.15.60:0/2931518730 >> v2:172.21.15.105:3300/0 conn(0x7fc73c0add90 msgr2=0x7fc73c16cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.793+0000 7fc73bfff700 1 --2- 172.21.15.60:0/2931518730 >> v2:172.21.15.105:3300/0 conn(0x7fc73c0add90 0x7fc73c16cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.793+0000 7fc73bfff700 1 -- 172.21.15.60:0/2931518730 >> v1:172.21.15.60:6789/0 conn(0x7fc73c089520 legacy=0x7fc73c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.793+0000 7fc73bfff700 1 -- 172.21.15.60:0/2931518730 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c0c3e90 con 0x7fc73c06a5e0 2023-07-13T22:21:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.794+0000 7fc73bfff700 1 --2- 172.21.15.60:0/2931518730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c06a5e0 0x7fc73c0c4320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00f370 tx=0x7fc72c00f680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19bb9ccdf1791998 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.794+0000 7fad6ffff700 1 -- 172.21.15.60:0/2162436402 >> v2:172.21.15.105:3300/0 conn(0x7fad70160870 msgr2=0x7fad700adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.794+0000 7fad6ffff700 1 --2- 172.21.15.60:0/2162436402 >> v2:172.21.15.105:3300/0 conn(0x7fad70160870 0x7fad700adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.794+0000 7fad6ffff700 1 -- 172.21.15.60:0/2162436402 >> v1:172.21.15.105:6789/0 conn(0x7fad700c3640 legacy=0x7fad70069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.794+0000 7fad6ffff700 1 -- 172.21.15.60:0/2162436402 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70164690 con 0x7fad7006a5e0 2023-07-13T22:21:03.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.794+0000 7fc73a7fc700 1 -- 172.21.15.60:0/2931518730 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c012070 con 0x7fc73c06a5e0 2023-07-13T22:21:03.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.794+0000 7fc73a7fc700 1 -- 172.21.15.60:0/2931518730 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00fe30 con 0x7fc73c06a5e0 2023-07-13T22:21:03.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.794+0000 7fc73a7fc700 1 -- 172.21.15.60:0/2931518730 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c003c60 con 0x7fc73c06a5e0 2023-07-13T22:21:03.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.795+0000 7f7e9d79a700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e7c06adb0 0x7f7e7c06d260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7e980ad860 tx=0x7f7e90009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.795+0000 7fad6ffff700 1 --2- 172.21.15.60:0/2162436402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad70160c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad68009d90 tx=0x7fad6800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76a1f98c2d86ca0c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.795+0000 7fad6e7fc700 1 -- 172.21.15.60:0/2162436402 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800e070 con 0x7fad7006a5e0 2023-07-13T22:21:03.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.795+0000 7fad6e7fc700 1 -- 172.21.15.60:0/2162436402 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6800bb90 con 0x7fad7006a5e0 2023-07-13T22:21:03.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.795+0000 7fad6e7fc700 1 -- 172.21.15.60:0/2162436402 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68003a20 con 0x7fad7006a5e0 2023-07-13T22:21:03.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.795+0000 7fc741bff700 1 -- 172.21.15.60:0/2931518730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c06a5e0 msgr2=0x7fc73c0c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.795+0000 7fc741bff700 1 --2- 172.21.15.60:0/2931518730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c06a5e0 0x7fc73c0c4320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00f370 tx=0x7fc72c00f680 comp rx=0 tx=0).stop 2023-07-13T22:21:03.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fc741bff700 1 -- 172.21.15.60:0/2931518730 shutdown_connections 2023-07-13T22:21:03.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fc741bff700 1 --2- 172.21.15.60:0/2931518730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c06a5e0 0x7fc73c0c4320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fc741bff700 1 --2- 172.21.15.60:0/2931518730 >> v2:172.21.15.105:3300/0 conn(0x7fc73c0add90 0x7fc73c16cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fc741bff700 1 -- 172.21.15.60:0/2931518730 >> 172.21.15.60:0/2931518730 conn(0x7fc73c0cb200 msgr2=0x7fc73c0cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:03.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7f6cbc3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cb4160920 0x7f6cb4069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7f6cbc3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cb40c5330 0x7f6cb406a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7f6cbc3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb41648d0 con 0x7f6cb406e630 2023-07-13T22:21:03.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7f6cbc3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb40c9200 con 0x7f6cb40c5330 2023-07-13T22:21:03.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7f6cbc3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb4164bb0 con 0x7f6cb4160920 2023-07-13T22:21:03.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fad75fff700 1 -- 172.21.15.60:0/2162436402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 msgr2=0x7fad70160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fad75fff700 1 --2- 172.21.15.60:0/2162436402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad70160c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad68009d90 tx=0x7fad6800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:03.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fad75fff700 1 -- 172.21.15.60:0/2162436402 shutdown_connections 2023-07-13T22:21:03.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fad75fff700 1 --2- 172.21.15.60:0/2162436402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad70160c40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fad75fff700 1 --2- 172.21.15.60:0/2162436402 >> v2:172.21.15.105:3300/0 conn(0x7fad70160870 0x7fad700adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7fad75fff700 1 -- 172.21.15.60:0/2162436402 >> 172.21.15.60:0/2162436402 conn(0x7fad700cf410 msgr2=0x7fad700cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:03.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7f6cb999a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cb40c5330 0x7f6cb406a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7f6cb999a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cb40c5330 0x7f6cb406a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34154/0 (socket says 172.21.15.60:34154) 2023-07-13T22:21:03.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.796+0000 7f6cb999a700 1 -- 172.21.15.60:0/4182253761 learned_addr learned my addr 172.21.15.60:0/4182253761 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.797+0000 7fc741bff700 1 -- 172.21.15.60:0/2931518730 shutdown_connections 2023-07-13T22:21:03.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.797+0000 7fc741bff700 1 -- 172.21.15.60:0/2931518730 wait complete. 2023-07-13T22:21:03.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.797+0000 7fad75fff700 1 -- 172.21.15.60:0/2162436402 shutdown_connections 2023-07-13T22:21:03.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.797+0000 7f6cb999a700 1 -- 172.21.15.60:0/4182253761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cb4160920 msgr2=0x7f6cb4069ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:03.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.797+0000 7f6cb999a700 1 --2- 172.21.15.60:0/4182253761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cb4160920 0x7f6cb4069ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.797+0000 7f6cb999a700 1 -- 172.21.15.60:0/4182253761 >> v1:172.21.15.105:6789/0 conn(0x7f6cb406e630 legacy=0x7f6cb4070270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:03.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.797+0000 7f6cb999a700 1 -- 172.21.15.60:0/4182253761 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb4164e90 con 0x7f6cb40c5330 2023-07-13T22:21:03.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.798+0000 7f6cb999a700 1 --2- 172.21.15.60:0/4182253761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb40c5330 0x7f6cb406a410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009e50 tx=0x7f6cac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a816da3e4c245 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.800+0000 7fc741bff700 1 Processor -- start 2023-07-13T22:21:03.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.798+0000 7f6cb9199700 1 -- 172.21.15.60:0/4182253761 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00e070 con 0x7f6cb40c5330 2023-07-13T22:21:03.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.799+0000 7f6cb9199700 1 -- 172.21.15.60:0/4182253761 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac003a20 con 0x7f6cb40c5330 2023-07-13T22:21:03.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.799+0000 7f6cb9199700 1 -- 172.21.15.60:0/4182253761 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac003dd0 con 0x7f6cb40c5330 2023-07-13T22:21:03.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.800+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/4182253761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb40c5330 msgr2=0x7f6cb406a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.800+0000 7f6cbc3ff700 1 --2- 172.21.15.60:0/4182253761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb40c5330 0x7f6cb406a410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009e50 tx=0x7f6cac00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:03.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.800+0000 7fc741bff700 1 -- start start 2023-07-13T22:21:03.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.801+0000 7fc741bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c06a5e0 0x7fc73c0c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.801+0000 7fc741bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c089520 0x7fc73c2137d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fc741bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0add90 0x7fc73c213d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fc741bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c168690 con 0x7fc73c06a5e0 2023-07-13T22:21:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fc741bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0c6c40 con 0x7fc73c0add90 2023-07-13T22:21:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fc741bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c214330 con 0x7fc73c089520 2023-07-13T22:21:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.803+0000 7fc73bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0add90 0x7fc73c213d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.803+0000 7fc73bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0add90 0x7fc73c213d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55170/0 (socket says 172.21.15.60:55170) 2023-07-13T22:21:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.801+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/4182253761 shutdown_connections 2023-07-13T22:21:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.801+0000 7f6cbc3ff700 1 --2- 172.21.15.60:0/4182253761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cb4160920 0x7f6cb4069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.801+0000 7f6cbc3ff700 1 --2- 172.21.15.60:0/4182253761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb40c5330 0x7f6cb406a410 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.801+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/4182253761 >> 172.21.15.60:0/4182253761 conn(0x7f6cb40cb200 msgr2=0x7f6cb40cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.797+0000 7fad75fff700 1 -- 172.21.15.60:0/2162436402 wait complete. 2023-07-13T22:21:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.800+0000 7fad75fff700 1 Processor -- start 2023-07-13T22:21:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.800+0000 7fad75fff700 1 -- start start 2023-07-13T22:21:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fad75fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad701569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fad75fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad700c3640 0x7fad700c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fad75fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad70160870 0x7fad700c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fad75fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700c7650 con 0x7fad70160870 2023-07-13T22:21:03.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fad75fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700c57c0 con 0x7fad700c3640 2023-07-13T22:21:03.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7fad75fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad700c7370 con 0x7fad7006a5e0 2023-07-13T22:21:03.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.803+0000 7fc73bfff700 1 -- 172.21.15.60:0/2445116992 learned_addr learned my addr 172.21.15.60:0/2445116992 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.803+0000 7fc73b7fe700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c06a5e0 0x7fc73c0c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.800+0000 7f07013ff700 1 Processor -- start 2023-07-13T22:21:03.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.800+0000 7f07013ff700 1 -- start start 2023-07-13T22:21:03.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7f07013ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06fc0add90 0x7f06fc16cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7f07013ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06fc089520 0x7f06fc069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7f07013ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06fc06a410 0x7f06fc070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7f07013ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc0c9200 con 0x7f06fc06a410 2023-07-13T22:21:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7f07013ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc06a890 con 0x7f06fc0add90 2023-07-13T22:21:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.802+0000 7f07013ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc16cb20 con 0x7f06fc089520 2023-07-13T22:21:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.803+0000 7fc73affd700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c089520 0x7fc73c2137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fc73bfff700 1 -- 172.21.15.60:0/2445116992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c089520 msgr2=0x7fc73c2137d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fc73bfff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c089520 0x7fc73c2137d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fc73bfff700 1 -- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c06a5e0 msgr2=0x7fc73c0c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fc73bfff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c06a5e0 0x7fc73c0c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fc73bfff700 1 -- 172.21.15.60:0/2445116992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c00f050 con 0x7fc73c0add90 2023-07-13T22:21:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.803+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/4182253761 shutdown_connections 2023-07-13T22:21:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fad6effd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad700c3640 0x7fad700c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fad6effd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad700c3640 0x7fad700c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55172/0 (socket says 172.21.15.60:55172) 2023-07-13T22:21:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fad6effd700 1 -- 172.21.15.60:0/3754376413 learned_addr learned my addr 172.21.15.60:0/3754376413 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fad6f7fe700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad701569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7fad6ffff700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad70160870 0x7fad700c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7fc73bfff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0add90 0x7fc73c213d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc72c019040 tx=0x7fc72c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7fc733fff700 1 -- 172.21.15.60:0/2445116992 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00b940 con 0x7fc73c0add90 2023-07-13T22:21:03.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7fc733fff700 1 -- 172.21.15.60:0/2445116992 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00bac0 con 0x7fc73c0add90 2023-07-13T22:21:03.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7fc733fff700 1 -- 172.21.15.60:0/2445116992 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00be70 con 0x7fc73c0add90 2023-07-13T22:21:03.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.803+0000 7f06faffd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06fc0add90 0x7f06fc16cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.803+0000 7f06faffd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06fc0add90 0x7f06fc16cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55186/0 (socket says 172.21.15.60:55186) 2023-07-13T22:21:03.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.803+0000 7f06faffd700 1 -- 172.21.15.60:0/3211495418 learned_addr learned my addr 172.21.15.60:0/3211495418 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7f06fa7fc700 1 --2- 172.21.15.60:0/3211495418 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06fc089520 0x7f06fc069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7f06faffd700 1 -- 172.21.15.60:0/3211495418 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06fc089520 msgr2=0x7f06fc069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7f06faffd700 1 --2- 172.21.15.60:0/3211495418 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06fc089520 0x7f06fc069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7f06faffd700 1 -- 172.21.15.60:0/3211495418 >> v2:172.21.15.60:3300/0 conn(0x7f06fc06a410 msgr2=0x7f06fc070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7f06faffd700 1 --2- 172.21.15.60:0/3211495418 >> v2:172.21.15.60:3300/0 conn(0x7f06fc06a410 0x7f06fc070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.804+0000 7f06faffd700 1 -- 172.21.15.60:0/3211495418 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc16dbd0 con 0x7f06fc0add90 2023-07-13T22:21:03.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7f06faffd700 1 --2- 172.21.15.60:0/3211495418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 0x7f06fc16cd10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f06fc06b760 tx=0x7f06f000a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e00384fc8e1978d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7fad6f7fe700 1 -- 172.21.15.60:0/3754376413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad700c3640 msgr2=0x7fad700c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7fad6f7fe700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad700c3640 0x7fad700c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7fad6f7fe700 1 -- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad70160870 msgr2=0x7fad700c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7fad6f7fe700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad70160870 0x7fad700c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.805+0000 7fad6f7fe700 1 -- 172.21.15.60:0/3754376413 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad68009a70 con 0x7fad7006a5e0 2023-07-13T22:21:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.806+0000 7f89eabff700 1 Processor -- start 2023-07-13T22:21:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.806+0000 7fad6f7fe700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad701569b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad6000d5c0 tx=0x7fad6000d8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.806+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73c1688e0 con 0x7fc73c0add90 2023-07-13T22:21:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.806+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73c168ea0 con 0x7fc73c0add90 2023-07-13T22:21:03.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.806+0000 7fad57fff700 1 -- 172.21.15.60:0/3754376413 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600125f0 con 0x7fad7006a5e0 2023-07-13T22:21:03.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.807+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad700c5a10 con 0x7fad7006a5e0 2023-07-13T22:21:03.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.807+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad700c5fd0 con 0x7fad7006a5e0 2023-07-13T22:21:03.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.807+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/4182253761 wait complete. 2023-07-13T22:21:03.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f6cbc3ff700 1 Processor -- start 2023-07-13T22:21:03.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f6cbc3ff700 1 -- start start 2023-07-13T22:21:03.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.807+0000 7f89eabff700 1 -- start start 2023-07-13T22:21:03.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f89eabff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 0x7f89e4069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f89eabff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89e40c5330 0x7f89e406a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad70156f50 con 0x7fad7006a5e0 2023-07-13T22:21:03.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f89eabff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f89e406e630 0x7f89e4070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f89eabff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e40c9200 con 0x7f89e406e630 2023-07-13T22:21:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f89eabff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e41648d0 con 0x7f89e40c5330 2023-07-13T22:21:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f89eabff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4164bb0 con 0x7f89e4160920 2023-07-13T22:21:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f89e899b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 0x7f89e4069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f89e899b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 0x7f89e4069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:49550/0 (socket says 172.21.15.60:49550) 2023-07-13T22:21:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f89e899b700 1 -- 172.21.15.60:0/4044126619 learned_addr learned my addr 172.21.15.60:0/4044126619 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f89e3fff700 1 --2- 172.21.15.60:0/4044126619 >> v2:172.21.15.105:3300/0 conn(0x7f89e40c5330 0x7f89e406a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73c1690c0 con 0x7fc73c0add90 2023-07-13T22:21:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.808+0000 7f06f9ffb700 1 -- 172.21.15.60:0/3211495418 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f0015df0 con 0x7f06fc0add90 2023-07-13T22:21:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.808+0000 7f06f9ffb700 1 -- 172.21.15.60:0/3211495418 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f000c660 con 0x7f06fc0add90 2023-07-13T22:21:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.808+0000 7f06f9ffb700 1 -- 172.21.15.60:0/3211495418 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f000ca10 con 0x7f06fc0add90 2023-07-13T22:21:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f07013ff700 1 -- 172.21.15.60:0/3211495418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 msgr2=0x7f06fc16cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.809+0000 7f07013ff700 1 --2- 172.21.15.60:0/3211495418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 0x7f06fc16cd10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f06fc06b760 tx=0x7f06f000a3f0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f07013ff700 1 -- 172.21.15.60:0/3211495418 shutdown_connections 2023-07-13T22:21:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f07013ff700 1 --2- 172.21.15.60:0/3211495418 >> v2:172.21.15.60:3300/0 conn(0x7f06fc06a410 0x7f06fc070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f07013ff700 1 --2- 172.21.15.60:0/3211495418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 0x7f06fc16cd10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f07013ff700 1 --2- 172.21.15.60:0/3211495418 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06fc089520 0x7f06fc069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f07013ff700 1 -- 172.21.15.60:0/3211495418 >> 172.21.15.60:0/3211495418 conn(0x7f06fc0cb200 msgr2=0x7f06fc0cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f07013ff700 1 -- 172.21.15.60:0/3211495418 shutdown_connections 2023-07-13T22:21:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7fad57fff700 1 -- 172.21.15.60:0/3754376413 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad60012750 con 0x7fad7006a5e0 2023-07-13T22:21:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7fad57fff700 1 -- 172.21.15.60:0/3754376413 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60012ae0 con 0x7fad7006a5e0 2023-07-13T22:21:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f89e919c700 1 --2- 172.21.15.60:0/4044126619 >> v2:172.21.15.60:3300/0 conn(0x7f89e406e630 0x7f89e4070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.811+0000 7f89e919c700 1 -- 172.21.15.60:0/4044126619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 msgr2=0x7f89e4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.811+0000 7f89e919c700 1 --2- 172.21.15.60:0/4044126619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 0x7f89e4069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.811+0000 7f89e919c700 1 -- 172.21.15.60:0/4044126619 >> v2:172.21.15.105:3300/0 conn(0x7f89e40c5330 msgr2=0x7f89e406a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.811+0000 7f89e919c700 1 --2- 172.21.15.60:0/4044126619 >> v2:172.21.15.105:3300/0 conn(0x7f89e40c5330 0x7f89e406a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.811+0000 7f89e919c700 1 -- 172.21.15.60:0/4044126619 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4164e90 con 0x7f89e406e630 2023-07-13T22:21:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.812+0000 7f89e919c700 1 --2- 172.21.15.60:0/4044126619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 0x7f89e4070270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89e406b7d0 tx=0x7f89dc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e94e27f671678b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.811+0000 7f6cbc3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb406e630 0x7f6cb40c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.812+0000 7f6cbc3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb40c5330 0x7f6cb4147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.812+0000 7f6cbc3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cb4160920 0x7f6cb4147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.812+0000 7f6cbc3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb4147fa0 con 0x7f6cb406e630 2023-07-13T22:21:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.812+0000 7f6cbc3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40c7b30 con 0x7f6cb40c5330 2023-07-13T22:21:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.812+0000 7f6cbc3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb4160e70 con 0x7f6cb4160920 2023-07-13T22:21:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.813+0000 7f6cba19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb406e630 0x7f6cb40c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.813+0000 7f6cba19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb406e630 0x7f6cb40c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34210/0 (socket says 172.21.15.60:34210) 2023-07-13T22:21:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.813+0000 7f6cba19b700 1 -- 172.21.15.60:0/3204280398 learned_addr learned my addr 172.21.15.60:0/3204280398 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.813+0000 7f6cba19b700 1 -- 172.21.15.60:0/3204280398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cb4160920 msgr2=0x7f6cb4147a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.813+0000 7f6cba19b700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cb4160920 0x7f6cb4147a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.814+0000 7f6cba19b700 1 -- 172.21.15.60:0/3204280398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb40c5330 msgr2=0x7f6cb4147520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.814+0000 7f6cba19b700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb40c5330 0x7f6cb4147520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.814+0000 7f6cba19b700 1 -- 172.21.15.60:0/3204280398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac009b30 con 0x7f6cb406e630 2023-07-13T22:21:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.812+0000 7fc733fff700 1 -- 172.21.15.60:0/2445116992 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc72c004030 con 0x7fc73c0add90 2023-07-13T22:21:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.813+0000 7fc733fff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc71c06adb0 0x7fc71c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.814+0000 7fc733fff700 1 -- 172.21.15.60:0/2445116992 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc72c08a290 con 0x7fc73c0add90 2023-07-13T22:21:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.814+0000 7f6cba19b700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb406e630 0x7f6cb40c6a80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000b470 tx=0x7f6cb000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.815+0000 7fc73b7fe700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc71c06adb0 0x7fc71c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.810+0000 7f07013ff700 1 -- 172.21.15.60:0/3211495418 wait complete. 2023-07-13T22:21:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.813+0000 7f07013ff700 1 Processor -- start 2023-07-13T22:21:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.813+0000 7f07013ff700 1 -- start start 2023-07-13T22:21:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.815+0000 7f07013ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06fc06a410 0x7f06fc15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.815+0000 7f07013ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06fc089520 0x7f06fc0b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.815+0000 7f07013ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 0x7f06fc06d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.815+0000 7f07013ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0b4690 con 0x7f06fc089520 2023-07-13T22:21:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.815+0000 7f07013ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc15d060 con 0x7f06fc0add90 2023-07-13T22:21:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.815+0000 7f07013ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0b3a10 con 0x7f06fc06a410 2023-07-13T22:21:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.815+0000 7f06fb7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 0x7f06fc06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f06fb7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 0x7f06fc06d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55216/0 (socket says 172.21.15.60:55216) 2023-07-13T22:21:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f06fb7fe700 1 -- 172.21.15.60:0/1051643555 learned_addr learned my addr 172.21.15.60:0/1051643555 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f06fa7fc700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06fc089520 0x7f06fc0b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7fc73b7fe700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc71c06adb0 0x7fc71c06d260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc73400cb30 tx=0x7fc73400c230 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7fad57fff700 1 -- 172.21.15.60:0/3754376413 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad60012c40 con 0x7fad7006a5e0 2023-07-13T22:21:03.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f06fb7fe700 1 -- 172.21.15.60:0/1051643555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06fc06a410 msgr2=0x7f06fc15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f06fb7fe700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06fc06a410 0x7f06fc15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f06fb7fe700 1 -- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06fc089520 msgr2=0x7f06fc0b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f06fb7fe700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06fc089520 0x7f06fc0b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f06fb7fe700 1 -- 172.21.15.60:0/1051643555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f000a6f0 con 0x7f06fc0add90 2023-07-13T22:21:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f06fb7fe700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 0x7f06fc06d640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06f400b970 tx=0x7f06f400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.816+0000 7f6caaffd700 1 -- 172.21.15.60:0/3204280398 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0011830 con 0x7f6cb406e630 2023-07-13T22:21:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.817+0000 7f6caaffd700 1 -- 172.21.15.60:0/3204280398 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0011990 con 0x7f6cb406e630 2023-07-13T22:21:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.817+0000 7f6caaffd700 1 -- 172.21.15.60:0/3204280398 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0011d40 con 0x7f6cb406e630 2023-07-13T22:21:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.818+0000 7f89e37fe700 1 -- 172.21.15.60:0/4044126619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc011830 con 0x7f89e406e630 2023-07-13T22:21:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.818+0000 7f89e37fe700 1 -- 172.21.15.60:0/4044126619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc011990 con 0x7f89e406e630 2023-07-13T22:21:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.818+0000 7f89e37fe700 1 -- 172.21.15.60:0/4044126619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc011d00 con 0x7f89e406e630 2023-07-13T22:21:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.819+0000 7fc733fff700 1 -- 172.21.15.60:0/2445116992 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc72c0553b0 con 0x7fc73c0add90 2023-07-13T22:21:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.817+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb4161150 con 0x7f6cb406e630 2023-07-13T22:21:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.817+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb4161710 con 0x7f6cb406e630 2023-07-13T22:21:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.817+0000 7fad57fff700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad5c06ace0 0x7fad5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.818+0000 7fad57fff700 1 -- 172.21.15.60:0/3754376413 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad60089ef0 con 0x7fad7006a5e0 2023-07-13T22:21:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.819+0000 7fad57fff700 1 -- 172.21.15.60:0/3754376413 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fad60055010 con 0x7fad7006a5e0 2023-07-13T22:21:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.818+0000 7f06e37fe700 1 -- 172.21.15.60:0/1051643555 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f400c930 con 0x7f06fc0add90 2023-07-13T22:21:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.818+0000 7f06e37fe700 1 -- 172.21.15.60:0/1051643555 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f400cab0 con 0x7f06fc0add90 2023-07-13T22:21:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.818+0000 7f06e37fe700 1 -- 172.21.15.60:0/1051643555 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f400ce60 con 0x7f06fc0add90 2023-07-13T22:21:03.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.819+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc0b3cf0 con 0x7f06fc0add90 2023-07-13T22:21:03.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.819+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc0b42b0 con 0x7f06fc0add90 2023-07-13T22:21:03.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.819+0000 7fad6effd700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad5c06ace0 0x7fad5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.820+0000 7fad6effd700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad5c06ace0 0x7fad5c06d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad64007db0 tx=0x7fad64007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.821+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc04e280 con 0x7f06fc0add90 2023-07-13T22:21:03.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.821+0000 7f6caaffd700 1 -- 172.21.15.60:0/3204280398 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6cb001a400 con 0x7f6cb406e630 2023-07-13T22:21:03.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.822+0000 7f6caaffd700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c9806ad80 0x7f6c9806d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.823+0000 7f6caaffd700 1 -- 172.21.15.60:0/3204280398 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6cb0089440 con 0x7f6cb406e630 2023-07-13T22:21:03.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.823+0000 7f6cb999a700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c9806ad80 0x7f6c9806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.823+0000 7f06e37fe700 1 -- 172.21.15.60:0/1051643555 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f06f401e440 con 0x7f06fc0add90 2023-07-13T22:21:03.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.824+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb4161900 con 0x7f6cb406e630 2023-07-13T22:21:03.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.824+0000 7f6cb999a700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c9806ad80 0x7f6c9806d230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6cb41f9bf0 tx=0x7f6cac009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.825+0000 7f06e37fe700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e806adb0 0x7f06e806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.825+0000 7f06e37fe700 1 -- 172.21.15.60:0/1051643555 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06f4006e40 con 0x7f06fc0add90 2023-07-13T22:21:03.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.822+0000 7f89eabff700 1 -- 172.21.15.60:0/4044126619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 msgr2=0x7f89e4070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.822+0000 7f89eabff700 1 --2- 172.21.15.60:0/4044126619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 0x7f89e4070270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89e406b7d0 tx=0x7f89dc009b20 comp rx=0 tx=0).stop 2023-07-13T22:21:03.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.824+0000 7f89eabff700 1 -- 172.21.15.60:0/4044126619 shutdown_connections 2023-07-13T22:21:03.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.824+0000 7f89eabff700 1 --2- 172.21.15.60:0/4044126619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 0x7f89e4070270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.824+0000 7f89eabff700 1 --2- 172.21.15.60:0/4044126619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 0x7f89e4069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.824+0000 7f89eabff700 1 --2- 172.21.15.60:0/4044126619 >> v2:172.21.15.105:3300/0 conn(0x7f89e40c5330 0x7f89e406a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.824+0000 7f89eabff700 1 -- 172.21.15.60:0/4044126619 >> 172.21.15.60:0/4044126619 conn(0x7f89e40cb200 msgr2=0x7f89e40cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:03.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.825+0000 7f06faffd700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e806adb0 0x7f06e806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.826+0000 7f06faffd700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e806adb0 0x7f06e806d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f06f001f040 tx=0x7f06f0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.827+0000 7f06e37fe700 1 -- 172.21.15.60:0/1051643555 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f06f4054580 con 0x7f06fc0add90 2023-07-13T22:21:03.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.826+0000 7f89eabff700 1 -- 172.21.15.60:0/4044126619 shutdown_connections 2023-07-13T22:21:03.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.826+0000 7f89eabff700 1 -- 172.21.15.60:0/4044126619 wait complete. 2023-07-13T22:21:03.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.829+0000 7f89eabff700 1 Processor -- start 2023-07-13T22:21:03.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.829+0000 7f89eabff700 1 -- start start 2023-07-13T22:21:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.831+0000 7f89eabff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 0x7f89e40afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.831+0000 7f89eabff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89e40c5330 0x7f89e415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.831+0000 7f89eabff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 0x7f89e40ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.831+0000 7f89eabff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41637f0 con 0x7f89e406e630 2023-07-13T22:21:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.831+0000 7f89eabff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40b02c0 con 0x7f89e40c5330 2023-07-13T22:21:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.831+0000 7f89eabff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4162be0 con 0x7f89e4160920 2023-07-13T22:21:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.831+0000 7f89e899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 0x7f89e40afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.831+0000 7f89e899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 0x7f89e40afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34236/0 (socket says 172.21.15.60:34236) 2023-07-13T22:21:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.831+0000 7f89e899b700 1 -- 172.21.15.60:0/3463731255 learned_addr learned my addr 172.21.15.60:0/3463731255 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.832+0000 7f89e899b700 1 -- 172.21.15.60:0/3463731255 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 msgr2=0x7f89e40ca000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:21:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.832+0000 7f89e899b700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 0x7f89e40ca000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.832+0000 7f89e899b700 1 -- 172.21.15.60:0/3463731255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89e40c5330 msgr2=0x7f89e415d6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:21:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.832+0000 7f89e899b700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89e40c5330 0x7f89e415d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.832+0000 7f89e899b700 1 -- 172.21.15.60:0/3463731255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc0094a0 con 0x7f89e406e630 2023-07-13T22:21:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.833+0000 7f89e899b700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 0x7f89e40afd80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f89d800efc0 tx=0x7f89d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.834+0000 7f89e17fa700 1 -- 172.21.15.60:0/3463731255 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d800cbb0 con 0x7f89e406e630 2023-07-13T22:21:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.834+0000 7f89e17fa700 1 -- 172.21.15.60:0/3463731255 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d800f040 con 0x7f89e406e630 2023-07-13T22:21:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.834+0000 7f89e17fa700 1 -- 172.21.15.60:0/3463731255 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d80243f0 con 0x7f89e406e630 2023-07-13T22:21:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.834+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e4162ec0 con 0x7f89e406e630 2023-07-13T22:21:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.834+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e4163480 con 0x7f89e406e630 2023-07-13T22:21:03.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.834+0000 7f6caaffd700 1 -- 172.21.15.60:0/3204280398 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6cb0054560 con 0x7f6cb406e630 2023-07-13T22:21:03.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.841+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c40052f0 con 0x7f89e406e630 2023-07-13T22:21:03.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.848+0000 7f89e17fa700 1 -- 172.21.15.60:0/3463731255 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f89d8024550 con 0x7f89e406e630 2023-07-13T22:21:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.850+0000 7f89e17fa700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f89c806adb0 0x7f89c806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.850+0000 7f89e17fa700 1 -- 172.21.15.60:0/3463731255 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f89d809a910 con 0x7f89e406e630 2023-07-13T22:21:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.851+0000 7f89e3fff700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f89c806adb0 0x7f89c806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.853+0000 7f89e3fff700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f89c806adb0 0x7f89c806d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f89e40b20c0 tx=0x7f89d0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.862+0000 7f89e17fa700 1 -- 172.21.15.60:0/3463731255 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f89d8065a30 con 0x7f89e406e630 2023-07-13T22:21:03.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.857+0000 7f2cd8bff700 1 Processor -- start 2023-07-13T22:21:03.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.858+0000 7f2cd8bff700 1 -- start start 2023-07-13T22:21:03.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.860+0000 7f2cd8bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2cd4160920 0x7f2cd4069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.860+0000 7f2cd8bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd4071090 con 0x7f2cd40c4e10 2023-07-13T22:21:03.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.860+0000 7f2cd8bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd41648d0 con 0x7f2cd40c5330 2023-07-13T22:21:03.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.860+0000 7f2cd8bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40c9200 con 0x7f2cd4160920 2023-07-13T22:21:03.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.862+0000 7f2cd1d9b700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2cd40c5330 0x7f2cd4070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:49226/0 (socket says 172.21.15.60:49226) 2023-07-13T22:21:03.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.862+0000 7f2cd1d9b700 1 -- 172.21.15.60:0/923193462 learned_addr learned my addr 172.21.15.60:0/923193462 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:03.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.862+0000 7f2cd259c700 1 --2- 172.21.15.60:0/923193462 >> v2:172.21.15.60:3300/0 conn(0x7f2cd4160920 0x7f2cd4069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.862+0000 7f2cd259c700 1 -- 172.21.15.60:0/923193462 >> v1:172.21.15.105:6789/0 conn(0x7f2cd40c5330 legacy=0x7f2cd4070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:03.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.863+0000 7f2cd259c700 1 -- 172.21.15.60:0/923193462 >> v1:172.21.15.60:6789/0 conn(0x7f2cd40c4e10 legacy=0x7f2cd4070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.863+0000 7f2cd259c700 1 -- 172.21.15.60:0/923193462 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4164bb0 con 0x7f2cd4160920 2023-07-13T22:21:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.863+0000 7f2cd259c700 1 --2- 172.21.15.60:0/923193462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd4160920 0x7f2cd4069ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009840 tx=0x7f2cc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1306d5c9ff607f8f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.864+0000 7f2cd159a700 1 -- 172.21.15.60:0/923193462 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8004230 con 0x7f2cd4160920 2023-07-13T22:21:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.864+0000 7f2cd159a700 1 -- 172.21.15.60:0/923193462 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc8004390 con 0x7f2cd4160920 2023-07-13T22:21:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.864+0000 7f2cd159a700 1 -- 172.21.15.60:0/923193462 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80036a0 con 0x7f2cd4160920 2023-07-13T22:21:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.865+0000 7f2cd8bff700 1 -- 172.21.15.60:0/923193462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd4160920 msgr2=0x7f2cd4069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.865+0000 7f2cd8bff700 1 --2- 172.21.15.60:0/923193462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd4160920 0x7f2cd4069ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009840 tx=0x7f2cc8009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:03.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.866+0000 7f2cd8bff700 1 -- 172.21.15.60:0/923193462 shutdown_connections 2023-07-13T22:21:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.866+0000 7f2cd8bff700 1 --2- 172.21.15.60:0/923193462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd4160920 0x7f2cd4069ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.866+0000 7f2cd8bff700 1 -- 172.21.15.60:0/923193462 >> 172.21.15.60:0/923193462 conn(0x7f2cd40cb200 msgr2=0x7f2cd40cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:03.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.867+0000 7f2cd8bff700 1 -- 172.21.15.60:0/923193462 shutdown_connections 2023-07-13T22:21:03.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.867+0000 7f2cd8bff700 1 -- 172.21.15.60:0/923193462 wait complete. 2023-07-13T22:21:03.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.870+0000 7f2cd8bff700 1 Processor -- start 2023-07-13T22:21:03.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.870+0000 7f2cd8bff700 1 -- start start 2023-07-13T22:21:03.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.871+0000 7f2cd8bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd40c4e10 0x7f2cd412eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.871+0000 7f2cd8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cd40c5330 0x7f2cd412dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.871+0000 7f2cd8bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cd4160920 0x7f2cd40890f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.872+0000 7f2cd8bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd412eca0 con 0x7f2cd40c4e10 2023-07-13T22:21:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.872+0000 7f2cd8bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd412e6e0 con 0x7f2cd4160920 2023-07-13T22:21:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.872+0000 7f2cd8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd412e9c0 con 0x7f2cd40c5330 2023-07-13T22:21:03.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.876+0000 7f2cd259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd40c4e10 0x7f2cd412eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.876+0000 7f2cd259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd40c4e10 0x7f2cd412eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34252/0 (socket says 172.21.15.60:34252) 2023-07-13T22:21:03.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.876+0000 7f2cd259c700 1 -- 172.21.15.60:0/1078917833 learned_addr learned my addr 172.21.15.60:0/1078917833 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:03.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.877+0000 7f2cd259c700 1 -- 172.21.15.60:0/1078917833 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cd40c5330 msgr2=0x7f2cd412dd20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:03.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.877+0000 7f2cd259c700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cd40c5330 0x7f2cd412dd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.877+0000 7f2cd259c700 1 -- 172.21.15.60:0/1078917833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cd4160920 msgr2=0x7f2cd40890f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:03.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.877+0000 7f2cd259c700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cd4160920 0x7f2cd40890f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:03.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.877+0000 7f2cd259c700 1 -- 172.21.15.60:0/1078917833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc80094a0 con 0x7f2cd40c4e10 2023-07-13T22:21:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.877+0000 7f2cd259c700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd40c4e10 0x7f2cd412eec0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2cd406ff90 tx=0x7f2cc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.878+0000 7f2cc2ffd700 1 -- 172.21.15.60:0/1078917833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8003df0 con 0x7f2cd40c4e10 2023-07-13T22:21:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.878+0000 7f2cc2ffd700 1 -- 172.21.15.60:0/1078917833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc801f070 con 0x7f2cd40c4e10 2023-07-13T22:21:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.878+0000 7f2cc2ffd700 1 -- 172.21.15.60:0/1078917833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8011850 con 0x7f2cd40c4e10 2023-07-13T22:21:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.878+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd4161030 con 0x7f2cd40c4e10 2023-07-13T22:21:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.879+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd412e260 con 0x7f2cd40c4e10 2023-07-13T22:21:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.883+0000 7f2cc2ffd700 1 -- 172.21.15.60:0/1078917833 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2cc8021030 con 0x7f2cd40c4e10 2023-07-13T22:21:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.883+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd412e4b0 con 0x7f2cd40c4e10 2023-07-13T22:21:03.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.885+0000 7f2cc2ffd700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cb406adb0 0x7f2cb406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:03.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.886+0000 7f2cc2ffd700 1 -- 172.21.15.60:0/1078917833 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2cc808aa20 con 0x7f2cd40c4e10 2023-07-13T22:21:03.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.891+0000 7f2cd1d9b700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cb406adb0 0x7f2cb406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:03.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.892+0000 7f2cd1d9b700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cb406adb0 0x7f2cb406d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4164010 tx=0x7f2cc4007b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:03.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:03.896+0000 7f2cc2ffd700 1 -- 172.21.15.60:0/1078917833 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2cc8055b40 con 0x7f2cd40c4e10 2023-07-13T22:21:04.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.204+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc068160f60 con 0x7fc06814c000 2023-07-13T22:21:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.205+0000 7fc05dffb700 1 -- 172.21.15.60:0/4046144545 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc068160f60 con 0x7fc06814c000 2023-07-13T22:21:04.192 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:04.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.209+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc04c06adb0 msgr2=0x7fc04c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.209+0000 7fc06f3ff700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc04c06adb0 0x7fc04c06d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc068068f70 tx=0x7fc064003c80 comp rx=0 tx=0).stop 2023-07-13T22:21:04.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.209+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc06814c000 msgr2=0x7fc0680ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.209+0000 7fc06f3ff700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc06814c000 0x7fc0680ada40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc068063c00 tx=0x7fc06000e9b0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.210+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 shutdown_connections 2023-07-13T22:21:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.210+0000 7fc06f3ff700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc04c06adb0 0x7fc04c06d260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.210+0000 7fc06f3ff700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0681f14c0 0x7fc0681f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.210+0000 7fc06f3ff700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0540052d0 0x7fc0680b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.210+0000 7fc06f3ff700 1 --2- 172.21.15.60:0/4046144545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc06814c000 0x7fc0680ada40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.210+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 >> 172.21.15.60:0/4046144545 conn(0x7fc0680cb200 msgr2=0x7fc0680c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.211+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 shutdown_connections 2023-07-13T22:21:04.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.211+0000 7fc06f3ff700 1 -- 172.21.15.60:0/4046144545 wait complete. 2023-07-13T22:21:04.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-07-13T22:21:04.227 DEBUG:teuthology.parallel:result is None 2023-07-13T22:21:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.256+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7e980cbe60 con 0x7f7e9806a410 2023-07-13T22:21:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.257+0000 7f7e8e7fc700 1 -- 172.21.15.60:0/2466263555 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7e94015020 con 0x7f7e9806a410 2023-07-13T22:21:04.241 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:04.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e7c06adb0 msgr2=0x7f7e7c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e7c06adb0 0x7f7e7c06d260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7e980ad860 tx=0x7f7e90009000 comp rx=0 tx=0).stop 2023-07-13T22:21:04.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9806a410 msgr2=0x7f7e9814e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9806a410 0x7f7e9814e3b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400b470 tx=0x7f7e9400b830 comp rx=0 tx=0).stop 2023-07-13T22:21:04.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 shutdown_connections 2023-07-13T22:21:04.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e7c06adb0 0x7f7e7c06d260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9806a410 0x7f7e9814e3b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e98160870 0x7f7e981569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 --2- 172.21.15.60:0/2466263555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e980c3640 0x7f7e9815b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 >> 172.21.15.60:0/2466263555 conn(0x7f7e980cf410 msgr2=0x7f7e98070270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.261+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 shutdown_connections 2023-07-13T22:21:04.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.262+0000 7f7ea01ff700 1 -- 172.21.15.60:0/2466263555 wait complete. 2023-07-13T22:21:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.271+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6cb40ce390 con 0x7f6cb406e630 2023-07-13T22:21:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.272+0000 7f6caaffd700 1 -- 172.21.15.60:0/3204280398 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6cb0015070 con 0x7f6cb406e630 2023-07-13T22:21:04.256 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2023-07-13T22:21:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.274+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c9806ad80 msgr2=0x7f6c9806d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.274+0000 7f6cbc3ff700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c9806ad80 0x7f6c9806d230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6cb41f9bf0 tx=0x7f6cac009000 comp rx=0 tx=0).stop 2023-07-13T22:21:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.274+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb406e630 msgr2=0x7f6cb40c6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.274+0000 7f6cbc3ff700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb406e630 0x7f6cb40c6a80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000b470 tx=0x7f6cb000b830 comp rx=0 tx=0).stop 2023-07-13T22:21:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.275+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 shutdown_connections 2023-07-13T22:21:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.275+0000 7f6cbc3ff700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c9806ad80 0x7f6c9806d230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.275+0000 7f6cbc3ff700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cb406e630 0x7f6cb40c6a80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.275+0000 7f6cbc3ff700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cb4160920 0x7f6cb4147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.275+0000 7f6cbc3ff700 1 --2- 172.21.15.60:0/3204280398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb40c5330 0x7f6cb4147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.275+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 >> 172.21.15.60:0/3204280398 conn(0x7f6cb40cb200 msgr2=0x7f6cb406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.275+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 shutdown_connections 2023-07-13T22:21:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.275+0000 7f6cbc3ff700 1 -- 172.21.15.60:0/3204280398 wait complete. 2023-07-13T22:21:04.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.283+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fad7004e280 con 0x7fad7006a5e0 2023-07-13T22:21:04.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.284+0000 7fad57fff700 1 -- 172.21.15.60:0/3754376413 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fad60058630 con 0x7fad7006a5e0 2023-07-13T22:21:04.269 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-07-13T22:21:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.287+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc73c04e280 con 0x7fc73c0add90 2023-07-13T22:21:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.287+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f06fc15ae50 con 0x7f06fc0add90 2023-07-13T22:21:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.287+0000 7fc733fff700 1 -- 172.21.15.60:0/2445116992 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc72c0589d0 con 0x7fc73c0add90 2023-07-13T22:21:04.272 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:04.273 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2023-07-13T22:21:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.288+0000 7f06e37fe700 1 -- 172.21.15.60:0/1051643555 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f06f4019020 con 0x7f06fc0add90 2023-07-13T22:21:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.288+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad5c06ace0 msgr2=0x7fad5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.288+0000 7fad75fff700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad5c06ace0 0x7fad5c06d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad64007db0 tx=0x7fad64007d00 comp rx=0 tx=0).stop 2023-07-13T22:21:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.289+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 msgr2=0x7fad701569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.289+0000 7fad75fff700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad701569b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad6000d5c0 tx=0x7fad6000d8d0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.289+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 shutdown_connections 2023-07-13T22:21:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.289+0000 7fad75fff700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad5c06ace0 0x7fad5c06d190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.289+0000 7fad75fff700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad70160870 0x7fad700c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.289+0000 7fad75fff700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad7006a5e0 0x7fad701569b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.289+0000 7fad75fff700 1 --2- 172.21.15.60:0/3754376413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad700c3640 0x7fad700c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.289+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 >> 172.21.15.60:0/3754376413 conn(0x7fad700cf410 msgr2=0x7fad7006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.289+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 shutdown_connections 2023-07-13T22:21:04.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.291+0000 7fad75fff700 1 -- 172.21.15.60:0/3754376413 wait complete. 2023-07-13T22:21:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc71c06adb0 msgr2=0x7fc71c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc71c06adb0 0x7fc71c06d260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc73400cb30 tx=0x7fc73400c230 comp rx=0 tx=0).stop 2023-07-13T22:21:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0add90 msgr2=0x7fc73c213d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0add90 0x7fc73c213d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc72c019040 tx=0x7fc72c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 shutdown_connections 2023-07-13T22:21:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc71c06adb0 0x7fc71c06d260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c06a5e0 0x7fc73c0c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0add90 0x7fc73c213d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 --2- 172.21.15.60:0/2445116992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c089520 0x7fc73c2137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 >> 172.21.15.60:0/2445116992 conn(0x7fc73c0cb200 msgr2=0x7fc73c069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e806adb0 msgr2=0x7f06e806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7f07013ff700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e806adb0 0x7f06e806d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f06f001f040 tx=0x7f06f0017000 comp rx=0 tx=0).stop 2023-07-13T22:21:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 msgr2=0x7f06fc06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7f07013ff700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 0x7f06fc06d640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06f400b970 tx=0x7f06f400bc80 comp rx=0 tx=0).stop 2023-07-13T22:21:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.292+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 shutdown_connections 2023-07-13T22:21:04.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.293+0000 7fc741bff700 1 -- 172.21.15.60:0/2445116992 wait complete. 2023-07-13T22:21:04.279 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-07-13T22:21:04.280 DEBUG:teuthology.parallel:result is None 2023-07-13T22:21:04.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.293+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 shutdown_connections 2023-07-13T22:21:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.293+0000 7f07013ff700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e806adb0 0x7f06e806d260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.293+0000 7f07013ff700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06fc0add90 0x7f06fc06d640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.293+0000 7f07013ff700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06fc06a410 0x7f06fc15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.293+0000 7f07013ff700 1 --2- 172.21.15.60:0/1051643555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06fc089520 0x7f06fc0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.293+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 >> 172.21.15.60:0/1051643555 conn(0x7f06fc0cb200 msgr2=0x7f06fc0ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.293+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 shutdown_connections 2023-07-13T22:21:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.293+0000 7f07013ff700 1 -- 172.21.15.60:0/1051643555 wait complete. 2023-07-13T22:21:04.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-07-13T22:21:04.284 DEBUG:teuthology.parallel:result is None 2023-07-13T22:21:04.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.312+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f89c40055a0 con 0x7f89e406e630 2023-07-13T22:21:04.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.312+0000 7f89e17fa700 1 -- 172.21.15.60:0/3463731255 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f89d8069050 con 0x7f89e406e630 2023-07-13T22:21:04.297 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:04.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f89c806adb0 msgr2=0x7f89c806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f89c806adb0 0x7f89c806d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f89e40b20c0 tx=0x7f89d0009210 comp rx=0 tx=0).stop 2023-07-13T22:21:04.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 msgr2=0x7f89e40afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 0x7f89e40afd80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f89d800efc0 tx=0x7f89d800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 shutdown_connections 2023-07-13T22:21:04.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f89c806adb0 0x7f89c806d260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89e406e630 0x7f89e40afd80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89e4160920 0x7f89e40ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 --2- 172.21.15.60:0/3463731255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89e40c5330 0x7f89e415d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 >> 172.21.15.60:0/3463731255 conn(0x7f89e40cb200 msgr2=0x7f89e40cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 shutdown_connections 2023-07-13T22:21:04.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.315+0000 7f89eabff700 1 -- 172.21.15.60:0/3463731255 wait complete. 2023-07-13T22:21:04.308 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2023-07-13T22:21:04.308 DEBUG:teuthology.parallel:result is None 2023-07-13T22:21:04.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-07-13T22:21:04.309 DEBUG:teuthology.parallel:result is None 2023-07-13T22:21:04.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-07-13T22:21:04.311 DEBUG:teuthology.parallel:result is None 2023-07-13T22:21:04.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-07-13T22:21:04.320 DEBUG:teuthology.parallel:result is None 2023-07-13T22:21:04.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.339+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2cd412e4b0 con 0x7f2cd40c4e10 2023-07-13T22:21:04.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.339+0000 7f2cc2ffd700 1 -- 172.21.15.60:0/1078917833 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2cc8059160 con 0x7f2cd40c4e10 2023-07-13T22:21:04.323 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-07-13T22:21:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.341+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cb406adb0 msgr2=0x7f2cb406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.341+0000 7f2cd8bff700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cb406adb0 0x7f2cb406d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4164010 tx=0x7f2cc4007b30 comp rx=0 tx=0).stop 2023-07-13T22:21:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.341+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd40c4e10 msgr2=0x7f2cd412eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.341+0000 7f2cd8bff700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd40c4e10 0x7f2cd412eec0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2cd406ff90 tx=0x7f2cc8004750 comp rx=0 tx=0).stop 2023-07-13T22:21:04.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.342+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 shutdown_connections 2023-07-13T22:21:04.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.342+0000 7f2cd8bff700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cb406adb0 0x7f2cb406d260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.342+0000 7f2cd8bff700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cd40c4e10 0x7f2cd412eec0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.342+0000 7f2cd8bff700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cd4160920 0x7f2cd40890f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.342+0000 7f2cd8bff700 1 --2- 172.21.15.60:0/1078917833 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cd40c5330 0x7f2cd412dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.342+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 >> 172.21.15.60:0/1078917833 conn(0x7f2cd40cb200 msgr2=0x7f2cd40ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.342+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 shutdown_connections 2023-07-13T22:21:04.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.342+0000 7f2cd8bff700 1 -- 172.21.15.60:0/1078917833 wait complete. 2023-07-13T22:21:04.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-07-13T22:21:04.344 DEBUG:teuthology.parallel:result is None 2023-07-13T22:21:04.344 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-13T22:21:04.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:21:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.602+0000 7f431ffff700 1 Processor -- start 2023-07-13T22:21:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.603+0000 7f431ffff700 1 -- start start 2023-07-13T22:21:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43180b8400 0x7f43180cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431ffff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43180cfca0 0x7f43180d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431ffff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43180d0ac0 con 0x7f43180d3750 2023-07-13T22:21:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431ffff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43180b87d0 con 0x7f43180cfca0 2023-07-13T22:21:04.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43180b4eb0 con 0x7f43180b8400 2023-07-13T22:21:04.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431d59a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43180d3750 0x7f43180cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39640/0 (socket says 172.21.15.60:39640) 2023-07-13T22:21:04.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431d59a700 1 -- 172.21.15.60:0/3971235762 learned_addr learned my addr 172.21.15.60:0/3971235762 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:04.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431e59c700 1 --2- 172.21.15.60:0/3971235762 >> v2:172.21.15.105:3300/0 conn(0x7f43180cfca0 0x7f43180d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:04.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431dd9b700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43180b8400 0x7f43180cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:04.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.604+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897909281 0 0) 0x7f43180d0ac0 con 0x7f43180d3750 2023-07-13T22:21:04.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43040035b0 con 0x7f43180d3750 2023-07-13T22:21:04.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2551235792 0 0) 0x7f43040035b0 con 0x7f43180d3750 2023-07-13T22:21:04.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43180b8400 msgr2=0x7f43180cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:04.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43180b8400 0x7f43180cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 >> v2:172.21.15.105:3300/0 conn(0x7f43180cfca0 msgr2=0x7f43180d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:04.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 --2- 172.21.15.60:0/3971235762 >> v2:172.21.15.105:3300/0 conn(0x7f43180cfca0 0x7f43180d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180b5190 con 0x7f43180d3750 2023-07-13T22:21:04.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f4308001670 con 0x7f43180d3750 2023-07-13T22:21:04.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 >> v1:172.21.15.60:6789/0 conn(0x7f43180d3750 legacy=0x7f43180cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4304003a00 0x7f4304003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:04.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4304004850 0x7f4304004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:04.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43180b8400 0x7f4304005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:04.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43180b87d0 con 0x7f4304003a00 2023-07-13T22:21:04.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43180b4eb0 con 0x7f4304004850 2023-07-13T22:21:04.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430400ba40 con 0x7f43180b8400 2023-07-13T22:21:04.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431dd9b700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4304003a00 0x7f4304003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:04.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.605+0000 7f431d59a700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4304004850 0x7f4304004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:04.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431e59c700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43180b8400 0x7f4304005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:04.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431dd9b700 1 -- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43180b8400 msgr2=0x7f4304005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:04.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431dd9b700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43180b8400 0x7f4304005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431dd9b700 1 -- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4304004850 msgr2=0x7f4304004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:04.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431dd9b700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4304004850 0x7f4304004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431dd9b700 1 -- 172.21.15.60:0/3971235762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43040035b0 con 0x7f4304003a00 2023-07-13T22:21:04.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431dd9b700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4304003a00 0x7f4304003e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4314009b60 tx=0x7f4304005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edc6eb41860bbd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:04.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400e6c0 con 0x7f4304003a00 2023-07-13T22:21:04.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431400e820 con 0x7f4304003a00 2023-07-13T22:21:04.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.606+0000 7f431cd99700 1 -- 172.21.15.60:0/3971235762 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400eb90 con 0x7f4304003a00 2023-07-13T22:21:04.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.607+0000 7f431ffff700 1 -- 172.21.15.60:0/3971235762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4304003a00 msgr2=0x7f4304003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.607+0000 7f431ffff700 1 --2- 172.21.15.60:0/3971235762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4304003a00 0x7f4304003e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4314009b60 tx=0x7f4304005880 comp rx=0 tx=0).stop 2023-07-13T22:21:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.607+0000 7f431e59c700 1 -- 172.21.15.60:0/3971235762 reap_dead start 2023-07-13T22:21:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.607+0000 7f431ffff700 1 -- 172.21.15.60:0/3971235762 shutdown_connections 2023-07-13T22:21:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.607+0000 7f431ffff700 1 -- 172.21.15.60:0/3971235762 >> 172.21.15.60:0/3971235762 conn(0x7f431806b390 msgr2=0x7f431806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.607+0000 7f431ffff700 1 -- 172.21.15.60:0/3971235762 shutdown_connections 2023-07-13T22:21:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.607+0000 7f431ffff700 1 -- 172.21.15.60:0/3971235762 wait complete. 2023-07-13T22:21:04.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.609+0000 7f431ffff700 1 Processor -- start 2023-07-13T22:21:04.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.609+0000 7f431ffff700 1 -- start start 2023-07-13T22:21:04.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4304003a00 0x7f4318153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:04.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43181f1f30 0x7f43180c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:04.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43180cbaf0 0x7f43180b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:04.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43180c9c50 con 0x7f43181f1f30 2023-07-13T22:21:04.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431ffff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43181f2480 con 0x7f43180cbaf0 2023-07-13T22:21:04.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43181f2760 con 0x7f4304003a00 2023-07-13T22:21:04.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431d59a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43181f1f30 0x7f43180c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:04.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431d59a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43181f1f30 0x7f43180c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34272/0 (socket says 172.21.15.60:34272) 2023-07-13T22:21:04.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431d59a700 1 -- 172.21.15.60:0/363377940 learned_addr learned my addr 172.21.15.60:0/363377940 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:04.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431dd9b700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4304003a00 0x7f4318153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:04.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431e59c700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43180cbaf0 0x7f43180b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:04.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431d59a700 1 -- 172.21.15.60:0/363377940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4304003a00 msgr2=0x7f4318153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431d59a700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4304003a00 0x7f4318153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431d59a700 1 -- 172.21.15.60:0/363377940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43180cbaf0 msgr2=0x7f43180b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431d59a700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43180cbaf0 0x7f43180b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431d59a700 1 -- 172.21.15.60:0/363377940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43181f2a40 con 0x7f43181f1f30 2023-07-13T22:21:04.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.610+0000 7f431e59c700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43180cbaf0 0x7f43180b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:04.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.611+0000 7f431d59a700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43181f1f30 0x7f43180c66b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4308000c00 tx=0x7f43080075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:04.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.611+0000 7f430e7fc700 1 -- 172.21.15.60:0/363377940 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308011580 con 0x7f43181f1f30 2023-07-13T22:21:04.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.611+0000 7f430e7fc700 1 -- 172.21.15.60:0/363377940 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43080116e0 con 0x7f43181f1f30 2023-07-13T22:21:04.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.611+0000 7f430e7fc700 1 -- 172.21.15.60:0/363377940 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308011a90 con 0x7f43181f1f30 2023-07-13T22:21:04.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.611+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43181f2d20 con 0x7f43181f1f30 2023-07-13T22:21:04.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.611+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43180c6bf0 con 0x7f43181f1f30 2023-07-13T22:21:04.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.612+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43180b4a30 con 0x7f43181f1f30 2023-07-13T22:21:04.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.613+0000 7f430e7fc700 1 -- 172.21.15.60:0/363377940 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4308011bf0 con 0x7f43181f1f30 2023-07-13T22:21:04.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.614+0000 7f430e7fc700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42fc06ace0 0x7f42fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:04.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.614+0000 7f430e7fc700 1 -- 172.21.15.60:0/363377940 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4308089580 con 0x7f43181f1f30 2023-07-13T22:21:04.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.615+0000 7f431dd9b700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42fc06ace0 0x7f42fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:04.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.615+0000 7f431dd9b700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42fc06ace0 0x7f42fc06d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f43140090b0 tx=0x7f431401f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:04.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.619+0000 7f430e7fc700 1 -- 172.21.15.60:0/363377940 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4308054590 con 0x7f43181f1f30 2023-07-13T22:21:04.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.823+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43180d0750 con 0x7f42fc06ace0 2023-07-13T22:21:04.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.832+0000 7f430e7fc700 1 -- 172.21.15.60:0/363377940 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286826 (secure 0 0 0) 0x7f431400ea90 con 0x7f42fc06ace0 2023-07-13T22:21:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:04.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42fc06ace0 msgr2=0x7f42fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42fc06ace0 0x7f42fc06d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f43140090b0 tx=0x7f431401f040 comp rx=0 tx=0).stop 2023-07-13T22:21:04.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43181f1f30 msgr2=0x7f43180c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43181f1f30 0x7f43180c66b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4308000c00 tx=0x7f43080075f0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 shutdown_connections 2023-07-13T22:21:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42fc06ace0 0x7f42fc06d190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43180cbaf0 0x7f43180b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4304003a00 0x7f4318153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 --2- 172.21.15.60:0/363377940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43181f1f30 0x7f43180c66b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 >> 172.21.15.60:0/363377940 conn(0x7f431806b390 msgr2=0x7f43180b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:04.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 shutdown_connections 2023-07-13T22:21:04.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:04.837+0000 7f431ffff700 1 -- 172.21.15.60:0/363377940 wait complete. 2023-07-13T22:21:04.823 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:21:04.839 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-07-13T22:21:03.126941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218528,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756192,"statfs":{"total":773094113280,"available":772870340608,"internally_reserved":0,"allocated":2981888,"data_stored":1819168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.611638"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611623+0000","last_change":"2023-07-13T22:20:54.539241+0000","last_active":"2023-07-13T22:20:55.611623+0000","last_peered":"2023-07-13T22:20:55.611623+0000","last_clean":"2023-07-13T22:20:55.611623+0000","last_became_active":"2023-07-13T22:20:54.538987+0000","last_became_peered":"2023-07-13T22:20:54.538987+0000","last_unstale":"2023-07-13T22:20:55.611623+0000","last_undegraded":"2023-07-13T22:20:55.611623+0000","last_fullsized":"2023-07-13T22:20:55.611623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:34:22.186098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532509+0000","last_change":"2023-07-13T22:20:54.537349+0000","last_active":"2023-07-13T22:20:55.532509+0000","last_peered":"2023-07-13T22:20:55.532509+0000","last_clean":"2023-07-13T22:20:55.532509+0000","last_became_active":"2023-07-13T22:20:54.536260+0000","last_became_peered":"2023-07-13T22:20:54.536260+0000","last_unstale":"2023-07-13T22:20:55.532509+0000","last_undegraded":"2023-07-13T22:20:55.532509+0000","last_fullsized":"2023-07-13T22:20:55.532509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532790+0000","last_change":"2023-07-13T22:20:54.543864+0000","last_active":"2023-07-13T22:20:55.532790+0000","last_peered":"2023-07-13T22:20:55.532790+0000","last_clean":"2023-07-13T22:20:55.532790+0000","last_became_active":"2023-07-13T22:20:54.543344+0000","last_became_peered":"2023-07-13T22:20:54.543344+0000","last_unstale":"2023-07-13T22:20:55.532790+0000","last_undegraded":"2023-07-13T22:20:55.532790+0000","last_fullsized":"2023-07-13T22:20:55.532790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610982+0000","last_change":"2023-07-13T22:20:54.540616+0000","last_active":"2023-07-13T22:20:55.610982+0000","last_peered":"2023-07-13T22:20:55.610982+0000","last_clean":"2023-07-13T22:20:55.610982+0000","last_became_active":"2023-07-13T22:20:54.540326+0000","last_became_peered":"2023-07-13T22:20:54.540326+0000","last_unstale":"2023-07-13T22:20:55.610982+0000","last_undegraded":"2023-07-13T22:20:55.610982+0000","last_fullsized":"2023-07-13T22:20:55.610982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611292+0000","last_change":"2023-07-13T22:20:54.550022+0000","last_active":"2023-07-13T22:20:55.611292+0000","last_peered":"2023-07-13T22:20:55.611292+0000","last_clean":"2023-07-13T22:20:55.611292+0000","last_became_active":"2023-07-13T22:20:54.549249+0000","last_became_peered":"2023-07-13T22:20:54.549249+0000","last_unstale":"2023-07-13T22:20:55.611292+0000","last_undegraded":"2023-07-13T22:20:55.611292+0000","last_fullsized":"2023-07-13T22:20:55.611292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533320+0000","last_change":"2023-07-13T22:20:54.544989+0000","last_active":"2023-07-13T22:20:55.533320+0000","last_peered":"2023-07-13T22:20:55.533320+0000","last_clean":"2023-07-13T22:20:55.533320+0000","last_became_active":"2023-07-13T22:20:54.544719+0000","last_became_peered":"2023-07-13T22:20:54.544719+0000","last_unstale":"2023-07-13T22:20:55.533320+0000","last_undegraded":"2023-07-13T22:20:55.533320+0000","last_fullsized":"2023-07-13T22:20:55.533320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:55.546960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610444+0000","last_change":"2023-07-13T22:20:54.543266+0000","last_active":"2023-07-13T22:20:55.610444+0000","last_peered":"2023-07-13T22:20:55.610444+0000","last_clean":"2023-07-13T22:20:55.610444+0000","last_became_active":"2023-07-13T22:20:54.542870+0000","last_became_peered":"2023-07-13T22:20:54.542870+0000","last_unstale":"2023-07-13T22:20:55.610444+0000","last_undegraded":"2023-07-13T22:20:55.610444+0000","last_fullsized":"2023-07-13T22:20:55.610444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532392+0000","last_change":"2023-07-13T22:20:54.546394+0000","last_active":"2023-07-13T22:20:55.532392+0000","last_peered":"2023-07-13T22:20:55.532392+0000","last_clean":"2023-07-13T22:20:55.532392+0000","last_became_active":"2023-07-13T22:20:54.545682+0000","last_became_peered":"2023-07-13T22:20:54.545682+0000","last_unstale":"2023-07-13T22:20:55.532392+0000","last_undegraded":"2023-07-13T22:20:55.532392+0000","last_fullsized":"2023-07-13T22:20:55.532392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611695+0000","last_change":"2023-07-13T22:20:54.549925+0000","last_active":"2023-07-13T22:20:55.611695+0000","last_peered":"2023-07-13T22:20:55.611695+0000","last_clean":"2023-07-13T22:20:55.611695+0000","last_became_active":"2023-07-13T22:20:54.549641+0000","last_became_peered":"2023-07-13T22:20:54.549641+0000","last_unstale":"2023-07-13T22:20:55.611695+0000","last_undegraded":"2023-07-13T22:20:55.611695+0000","last_fullsized":"2023-07-13T22:20:55.611695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:45.329435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532781+0000","last_change":"2023-07-13T22:20:54.543479+0000","last_active":"2023-07-13T22:20:55.532781+0000","last_peered":"2023-07-13T22:20:55.532781+0000","last_clean":"2023-07-13T22:20:55.532781+0000","last_became_active":"2023-07-13T22:20:54.543143+0000","last_became_peered":"2023-07-13T22:20:54.543143+0000","last_unstale":"2023-07-13T22:20:55.532781+0000","last_undegraded":"2023-07-13T22:20:55.532781+0000","last_fullsized":"2023-07-13T22:20:55.532781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532989+0000","last_change":"2023-07-13T22:20:54.543686+0000","last_active":"2023-07-13T22:20:55.532989+0000","last_peered":"2023-07-13T22:20:55.532989+0000","last_clean":"2023-07-13T22:20:55.532989+0000","last_became_active":"2023-07-13T22:20:54.543001+0000","last_became_peered":"2023-07-13T22:20:54.543001+0000","last_unstale":"2023-07-13T22:20:55.532989+0000","last_undegraded":"2023-07-13T22:20:55.532989+0000","last_fullsized":"2023-07-13T22:20:55.532989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611996+0000","last_change":"2023-07-13T22:20:54.547637+0000","last_active":"2023-07-13T22:20:55.611996+0000","last_peered":"2023-07-13T22:20:55.611996+0000","last_clean":"2023-07-13T22:20:55.611996+0000","last_became_active":"2023-07-13T22:20:54.547280+0000","last_became_peered":"2023-07-13T22:20:54.547280+0000","last_unstale":"2023-07-13T22:20:55.611996+0000","last_undegraded":"2023-07-13T22:20:55.611996+0000","last_fullsized":"2023-07-13T22:20:55.611996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:18:24.767625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611891+0000","last_change":"2023-07-13T22:20:54.538682+0000","last_active":"2023-07-13T22:20:55.611891+0000","last_peered":"2023-07-13T22:20:55.611891+0000","last_clean":"2023-07-13T22:20:55.611891+0000","last_became_active":"2023-07-13T22:20:54.538356+0000","last_became_peered":"2023-07-13T22:20:54.538356+0000","last_unstale":"2023-07-13T22:20:55.611891+0000","last_undegraded":"2023-07-13T22:20:55.611891+0000","last_fullsized":"2023-07-13T22:20:55.611891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:51:10.800579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533030+0000","last_change":"2023-07-13T22:20:54.540925+0000","last_active":"2023-07-13T22:20:55.533030+0000","last_peered":"2023-07-13T22:20:55.533030+0000","last_clean":"2023-07-13T22:20:55.533030+0000","last_became_active":"2023-07-13T22:20:54.540646+0000","last_became_peered":"2023-07-13T22:20:54.540646+0000","last_unstale":"2023-07-13T22:20:55.533030+0000","last_undegraded":"2023-07-13T22:20:55.533030+0000","last_fullsized":"2023-07-13T22:20:55.533030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610864+0000","last_change":"2023-07-13T22:20:54.543844+0000","last_active":"2023-07-13T22:20:55.610864+0000","last_peered":"2023-07-13T22:20:55.610864+0000","last_clean":"2023-07-13T22:20:55.610864+0000","last_became_active":"2023-07-13T22:20:54.543527+0000","last_became_peered":"2023-07-13T22:20:54.543527+0000","last_unstale":"2023-07-13T22:20:55.610864+0000","last_undegraded":"2023-07-13T22:20:55.610864+0000","last_fullsized":"2023-07-13T22:20:55.610864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533022+0000","last_change":"2023-07-13T22:20:54.537914+0000","last_active":"2023-07-13T22:20:55.533022+0000","last_peered":"2023-07-13T22:20:55.533022+0000","last_clean":"2023-07-13T22:20:55.533022+0000","last_became_active":"2023-07-13T22:20:54.537562+0000","last_became_peered":"2023-07-13T22:20:54.537562+0000","last_unstale":"2023-07-13T22:20:55.533022+0000","last_undegraded":"2023-07-13T22:20:55.533022+0000","last_fullsized":"2023-07-13T22:20:55.533022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:26.867364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611835+0000","last_change":"2023-07-13T22:20:54.549793+0000","last_active":"2023-07-13T22:20:55.611835+0000","last_peered":"2023-07-13T22:20:55.611835+0000","last_clean":"2023-07-13T22:20:55.611835+0000","last_became_active":"2023-07-13T22:20:54.549473+0000","last_became_peered":"2023-07-13T22:20:54.549473+0000","last_unstale":"2023-07-13T22:20:55.611835+0000","last_undegraded":"2023-07-13T22:20:55.611835+0000","last_fullsized":"2023-07-13T22:20:55.611835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535307+0000","last_change":"2023-07-13T22:20:54.541942+0000","last_active":"2023-07-13T22:20:55.535307+0000","last_peered":"2023-07-13T22:20:55.535307+0000","last_clean":"2023-07-13T22:20:55.535307+0000","last_became_active":"2023-07-13T22:20:54.541314+0000","last_became_peered":"2023-07-13T22:20:54.541314+0000","last_unstale":"2023-07-13T22:20:55.535307+0000","last_undegraded":"2023-07-13T22:20:55.535307+0000","last_fullsized":"2023-07-13T22:20:55.535307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:10:09.528351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611873+0000","last_change":"2023-07-13T22:20:54.532924+0000","last_active":"2023-07-13T22:20:55.611873+0000","last_peered":"2023-07-13T22:20:55.611873+0000","last_clean":"2023-07-13T22:20:55.611873+0000","last_became_active":"2023-07-13T22:20:54.531937+0000","last_became_peered":"2023-07-13T22:20:54.531937+0000","last_unstale":"2023-07-13T22:20:55.611873+0000","last_undegraded":"2023-07-13T22:20:55.611873+0000","last_fullsized":"2023-07-13T22:20:55.611873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:20:04.410391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533904+0000","last_change":"2023-07-13T22:20:54.552416+0000","last_active":"2023-07-13T22:20:55.533904+0000","last_peered":"2023-07-13T22:20:55.533904+0000","last_clean":"2023-07-13T22:20:55.533904+0000","last_became_active":"2023-07-13T22:20:54.551516+0000","last_became_peered":"2023-07-13T22:20:54.551516+0000","last_unstale":"2023-07-13T22:20:55.533904+0000","last_undegraded":"2023-07-13T22:20:55.533904+0000","last_fullsized":"2023-07-13T22:20:55.533904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611325+0000","last_change":"2023-07-13T22:20:54.551898+0000","last_active":"2023-07-13T22:20:55.611325+0000","last_peered":"2023-07-13T22:20:55.611325+0000","last_clean":"2023-07-13T22:20:55.611325+0000","last_became_active":"2023-07-13T22:20:54.551538+0000","last_became_peered":"2023-07-13T22:20:54.551538+0000","last_unstale":"2023-07-13T22:20:55.611325+0000","last_undegraded":"2023-07-13T22:20:55.611325+0000","last_fullsized":"2023-07-13T22:20:55.611325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:38.898796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610399+0000","last_change":"2023-07-13T22:20:54.540580+0000","last_active":"2023-07-13T22:20:55.610399+0000","last_peered":"2023-07-13T22:20:55.610399+0000","last_clean":"2023-07-13T22:20:55.610399+0000","last_became_active":"2023-07-13T22:20:54.540289+0000","last_became_peered":"2023-07-13T22:20:54.540289+0000","last_unstale":"2023-07-13T22:20:55.610399+0000","last_undegraded":"2023-07-13T22:20:55.610399+0000","last_fullsized":"2023-07-13T22:20:55.610399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:43.402535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610169+0000","last_change":"2023-07-13T22:20:54.536952+0000","last_active":"2023-07-13T22:20:55.610169+0000","last_peered":"2023-07-13T22:20:55.610169+0000","last_clean":"2023-07-13T22:20:55.610169+0000","last_became_active":"2023-07-13T22:20:54.536696+0000","last_became_peered":"2023-07-13T22:20:54.536696+0000","last_unstale":"2023-07-13T22:20:55.610169+0000","last_undegraded":"2023-07-13T22:20:55.610169+0000","last_fullsized":"2023-07-13T22:20:55.610169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:26.867364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611257+0000","last_change":"2023-07-13T22:20:54.550043+0000","last_active":"2023-07-13T22:20:55.611257+0000","last_peered":"2023-07-13T22:20:55.611257+0000","last_clean":"2023-07-13T22:20:55.611257+0000","last_became_active":"2023-07-13T22:20:54.549754+0000","last_became_peered":"2023-07-13T22:20:54.549754+0000","last_unstale":"2023-07-13T22:20:55.611257+0000","last_undegraded":"2023-07-13T22:20:55.611257+0000","last_fullsized":"2023-07-13T22:20:55.611257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531264+0000","last_change":"2023-07-13T22:20:54.534540+0000","last_active":"2023-07-13T22:20:55.531264+0000","last_peered":"2023-07-13T22:20:55.531264+0000","last_clean":"2023-07-13T22:20:55.531264+0000","last_became_active":"2023-07-13T22:20:54.534251+0000","last_became_peered":"2023-07-13T22:20:54.534251+0000","last_unstale":"2023-07-13T22:20:55.531264+0000","last_undegraded":"2023-07-13T22:20:55.531264+0000","last_fullsized":"2023-07-13T22:20:55.531264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531735+0000","last_change":"2023-07-13T22:20:54.529221+0000","last_active":"2023-07-13T22:20:55.531735+0000","last_peered":"2023-07-13T22:20:55.531735+0000","last_clean":"2023-07-13T22:20:55.531735+0000","last_became_active":"2023-07-13T22:20:54.528934+0000","last_became_peered":"2023-07-13T22:20:54.528934+0000","last_unstale":"2023-07-13T22:20:55.531735+0000","last_undegraded":"2023-07-13T22:20:55.531735+0000","last_fullsized":"2023-07-13T22:20:55.531735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533014+0000","last_change":"2023-07-13T22:20:54.539287+0000","last_active":"2023-07-13T22:20:55.533014+0000","last_peered":"2023-07-13T22:20:55.533014+0000","last_clean":"2023-07-13T22:20:55.533014+0000","last_became_active":"2023-07-13T22:20:54.537991+0000","last_became_peered":"2023-07-13T22:20:54.537991+0000","last_unstale":"2023-07-13T22:20:55.533014+0000","last_undegraded":"2023-07-13T22:20:55.533014+0000","last_fullsized":"2023-07-13T22:20:55.533014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533580+0000","last_change":"2023-07-13T22:20:53.534188+0000","last_active":"2023-07-13T22:20:55.533580+0000","last_peered":"2023-07-13T22:20:55.533580+0000","last_clean":"2023-07-13T22:20:55.533580+0000","last_became_active":"2023-07-13T22:20:53.533888+0000","last_became_peered":"2023-07-13T22:20:53.533888+0000","last_unstale":"2023-07-13T22:20:55.533580+0000","last_undegraded":"2023-07-13T22:20:55.533580+0000","last_fullsized":"2023-07-13T22:20:55.533580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:55.228110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610249+0000","last_change":"2023-07-13T22:20:54.541751+0000","last_active":"2023-07-13T22:20:55.610249+0000","last_peered":"2023-07-13T22:20:55.610249+0000","last_clean":"2023-07-13T22:20:55.610249+0000","last_became_active":"2023-07-13T22:20:54.541506+0000","last_became_peered":"2023-07-13T22:20:54.541506+0000","last_unstale":"2023-07-13T22:20:55.610249+0000","last_undegraded":"2023-07-13T22:20:55.610249+0000","last_fullsized":"2023-07-13T22:20:55.610249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:55.546960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531119+0000","last_change":"2023-07-13T22:20:53.531325+0000","last_active":"2023-07-13T22:20:55.531119+0000","last_peered":"2023-07-13T22:20:55.531119+0000","last_clean":"2023-07-13T22:20:55.531119+0000","last_became_active":"2023-07-13T22:20:53.530929+0000","last_became_peered":"2023-07-13T22:20:53.530929+0000","last_unstale":"2023-07-13T22:20:55.531119+0000","last_undegraded":"2023-07-13T22:20:55.531119+0000","last_fullsized":"2023-07-13T22:20:55.531119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:07:16.260303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611424+0000","last_change":"2023-07-13T22:20:54.541835+0000","last_active":"2023-07-13T22:20:55.611424+0000","last_peered":"2023-07-13T22:20:55.611424+0000","last_clean":"2023-07-13T22:20:55.611424+0000","last_became_active":"2023-07-13T22:20:54.541371+0000","last_became_peered":"2023-07-13T22:20:54.541371+0000","last_unstale":"2023-07-13T22:20:55.611424+0000","last_undegraded":"2023-07-13T22:20:55.611424+0000","last_fullsized":"2023-07-13T22:20:55.611424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610264+0000","last_change":"2023-07-13T22:20:53.534081+0000","last_active":"2023-07-13T22:20:55.610264+0000","last_peered":"2023-07-13T22:20:55.610264+0000","last_clean":"2023-07-13T22:20:55.610264+0000","last_became_active":"2023-07-13T22:20:53.532985+0000","last_became_peered":"2023-07-13T22:20:53.532985+0000","last_unstale":"2023-07-13T22:20:55.610264+0000","last_undegraded":"2023-07-13T22:20:55.610264+0000","last_fullsized":"2023-07-13T22:20:55.610264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:16.293217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610466+0000","last_change":"2023-07-13T22:20:54.533387+0000","last_active":"2023-07-13T22:20:55.610466+0000","last_peered":"2023-07-13T22:20:55.610466+0000","last_clean":"2023-07-13T22:20:55.610466+0000","last_became_active":"2023-07-13T22:20:54.533023+0000","last_became_peered":"2023-07-13T22:20:54.533023+0000","last_unstale":"2023-07-13T22:20:55.610466+0000","last_undegraded":"2023-07-13T22:20:55.610466+0000","last_fullsized":"2023-07-13T22:20:55.610466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611957+0000","last_change":"2023-07-13T22:20:53.538156+0000","last_active":"2023-07-13T22:20:55.611957+0000","last_peered":"2023-07-13T22:20:55.611957+0000","last_clean":"2023-07-13T22:20:55.611957+0000","last_became_active":"2023-07-13T22:20:53.536726+0000","last_became_peered":"2023-07-13T22:20:53.536726+0000","last_unstale":"2023-07-13T22:20:55.611957+0000","last_undegraded":"2023-07-13T22:20:55.611957+0000","last_fullsized":"2023-07-13T22:20:55.611957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:25.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610113+0000","last_change":"2023-07-13T22:20:54.541258+0000","last_active":"2023-07-13T22:20:55.610113+0000","last_peered":"2023-07-13T22:20:55.610113+0000","last_clean":"2023-07-13T22:20:55.610113+0000","last_became_active":"2023-07-13T22:20:54.540986+0000","last_became_peered":"2023-07-13T22:20:54.540986+0000","last_unstale":"2023-07-13T22:20:55.610113+0000","last_undegraded":"2023-07-13T22:20:55.610113+0000","last_fullsized":"2023-07-13T22:20:55.610113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532649+0000","last_change":"2023-07-13T22:20:53.536025+0000","last_active":"2023-07-13T22:20:55.532649+0000","last_peered":"2023-07-13T22:20:55.532649+0000","last_clean":"2023-07-13T22:20:55.532649+0000","last_became_active":"2023-07-13T22:20:53.535697+0000","last_became_peered":"2023-07-13T22:20:53.535697+0000","last_unstale":"2023-07-13T22:20:55.532649+0000","last_undegraded":"2023-07-13T22:20:55.532649+0000","last_fullsized":"2023-07-13T22:20:55.532649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:53.812690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610659+0000","last_change":"2023-07-13T22:20:54.537549+0000","last_active":"2023-07-13T22:20:55.610659+0000","last_peered":"2023-07-13T22:20:55.610659+0000","last_clean":"2023-07-13T22:20:55.610659+0000","last_became_active":"2023-07-13T22:20:54.537260+0000","last_became_peered":"2023-07-13T22:20:54.537260+0000","last_unstale":"2023-07-13T22:20:55.610659+0000","last_undegraded":"2023-07-13T22:20:55.610659+0000","last_fullsized":"2023-07-13T22:20:55.610659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611165+0000","last_change":"2023-07-13T22:20:53.543492+0000","last_active":"2023-07-13T22:20:55.611165+0000","last_peered":"2023-07-13T22:20:55.611165+0000","last_clean":"2023-07-13T22:20:55.611165+0000","last_became_active":"2023-07-13T22:20:53.543265+0000","last_became_peered":"2023-07-13T22:20:53.543265+0000","last_unstale":"2023-07-13T22:20:55.611165+0000","last_undegraded":"2023-07-13T22:20:55.611165+0000","last_fullsized":"2023-07-13T22:20:55.611165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:35.694626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531212+0000","last_change":"2023-07-13T22:20:54.532326+0000","last_active":"2023-07-13T22:20:55.531212+0000","last_peered":"2023-07-13T22:20:55.531212+0000","last_clean":"2023-07-13T22:20:55.531212+0000","last_became_active":"2023-07-13T22:20:54.532144+0000","last_became_peered":"2023-07-13T22:20:54.532144+0000","last_unstale":"2023-07-13T22:20:55.531212+0000","last_undegraded":"2023-07-13T22:20:55.531212+0000","last_fullsized":"2023-07-13T22:20:55.531212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:24.401992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.534193+0000","last_change":"2023-07-13T22:20:53.531850+0000","last_active":"2023-07-13T22:20:55.534193+0000","last_peered":"2023-07-13T22:20:55.534193+0000","last_clean":"2023-07-13T22:20:55.534193+0000","last_became_active":"2023-07-13T22:20:53.531567+0000","last_became_peered":"2023-07-13T22:20:53.531567+0000","last_unstale":"2023-07-13T22:20:55.534193+0000","last_undegraded":"2023-07-13T22:20:55.534193+0000","last_fullsized":"2023-07-13T22:20:55.534193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:37.890575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533168+0000","last_change":"2023-07-13T22:20:54.552096+0000","last_active":"2023-07-13T22:20:55.533168+0000","last_peered":"2023-07-13T22:20:55.533168+0000","last_clean":"2023-07-13T22:20:55.533168+0000","last_became_active":"2023-07-13T22:20:54.551842+0000","last_became_peered":"2023-07-13T22:20:54.551842+0000","last_unstale":"2023-07-13T22:20:55.533168+0000","last_undegraded":"2023-07-13T22:20:55.533168+0000","last_fullsized":"2023-07-13T22:20:55.533168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:08.460983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610860+0000","last_change":"2023-07-13T22:20:53.538459+0000","last_active":"2023-07-13T22:20:55.610860+0000","last_peered":"2023-07-13T22:20:55.610860+0000","last_clean":"2023-07-13T22:20:55.610860+0000","last_became_active":"2023-07-13T22:20:53.538180+0000","last_became_peered":"2023-07-13T22:20:53.538180+0000","last_unstale":"2023-07-13T22:20:55.610860+0000","last_undegraded":"2023-07-13T22:20:55.610860+0000","last_fullsized":"2023-07-13T22:20:55.610860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:54.538739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610410+0000","last_change":"2023-07-13T22:20:54.542603+0000","last_active":"2023-07-13T22:20:55.610410+0000","last_peered":"2023-07-13T22:20:55.610410+0000","last_clean":"2023-07-13T22:20:55.610410+0000","last_became_active":"2023-07-13T22:20:54.542348+0000","last_became_peered":"2023-07-13T22:20:54.542348+0000","last_unstale":"2023-07-13T22:20:55.610410+0000","last_undegraded":"2023-07-13T22:20:55.610410+0000","last_fullsized":"2023-07-13T22:20:55.610410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:48.562581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610578+0000","last_change":"2023-07-13T22:20:53.539294+0000","last_active":"2023-07-13T22:20:55.610578+0000","last_peered":"2023-07-13T22:20:55.610578+0000","last_clean":"2023-07-13T22:20:55.610578+0000","last_became_active":"2023-07-13T22:20:53.538780+0000","last_became_peered":"2023-07-13T22:20:53.538780+0000","last_unstale":"2023-07-13T22:20:55.610578+0000","last_undegraded":"2023-07-13T22:20:55.610578+0000","last_fullsized":"2023-07-13T22:20:55.610578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:23.393770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533163+0000","last_change":"2023-07-13T22:20:54.538797+0000","last_active":"2023-07-13T22:20:55.533163+0000","last_peered":"2023-07-13T22:20:55.533163+0000","last_clean":"2023-07-13T22:20:55.533163+0000","last_became_active":"2023-07-13T22:20:54.537618+0000","last_became_peered":"2023-07-13T22:20:54.537618+0000","last_unstale":"2023-07-13T22:20:55.533163+0000","last_undegraded":"2023-07-13T22:20:55.533163+0000","last_fullsized":"2023-07-13T22:20:55.533163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610621+0000","last_change":"2023-07-13T22:20:53.540448+0000","last_active":"2023-07-13T22:20:55.610621+0000","last_peered":"2023-07-13T22:20:55.610621+0000","last_clean":"2023-07-13T22:20:55.610621+0000","last_became_active":"2023-07-13T22:20:53.540124+0000","last_became_peered":"2023-07-13T22:20:53.540124+0000","last_unstale":"2023-07-13T22:20:55.610621+0000","last_undegraded":"2023-07-13T22:20:55.610621+0000","last_fullsized":"2023-07-13T22:20:55.610621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611081+0000","last_change":"2023-07-13T22:20:54.552868+0000","last_active":"2023-07-13T22:20:55.611081+0000","last_peered":"2023-07-13T22:20:55.611081+0000","last_clean":"2023-07-13T22:20:55.611081+0000","last_became_active":"2023-07-13T22:20:54.552531+0000","last_became_peered":"2023-07-13T22:20:54.552531+0000","last_unstale":"2023-07-13T22:20:55.611081+0000","last_undegraded":"2023-07-13T22:20:55.611081+0000","last_fullsized":"2023-07-13T22:20:55.611081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610438+0000","last_change":"2023-07-13T22:20:53.533055+0000","last_active":"2023-07-13T22:20:55.610438+0000","last_peered":"2023-07-13T22:20:55.610438+0000","last_clean":"2023-07-13T22:20:55.610438+0000","last_became_active":"2023-07-13T22:20:53.532792+0000","last_became_peered":"2023-07-13T22:20:53.532792+0000","last_unstale":"2023-07-13T22:20:55.610438+0000","last_undegraded":"2023-07-13T22:20:55.610438+0000","last_fullsized":"2023-07-13T22:20:55.610438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:28.638041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610888+0000","last_change":"2023-07-13T22:20:54.536719+0000","last_active":"2023-07-13T22:20:55.610888+0000","last_peered":"2023-07-13T22:20:55.610888+0000","last_clean":"2023-07-13T22:20:55.610888+0000","last_became_active":"2023-07-13T22:20:54.535865+0000","last_became_peered":"2023-07-13T22:20:54.535865+0000","last_unstale":"2023-07-13T22:20:55.610888+0000","last_undegraded":"2023-07-13T22:20:55.610888+0000","last_fullsized":"2023-07-13T22:20:55.610888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:04:58.712616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610902+0000","last_change":"2023-07-13T22:20:53.537777+0000","last_active":"2023-07-13T22:20:55.610902+0000","last_peered":"2023-07-13T22:20:55.610902+0000","last_clean":"2023-07-13T22:20:55.610902+0000","last_became_active":"2023-07-13T22:20:53.537464+0000","last_became_peered":"2023-07-13T22:20:53.537464+0000","last_unstale":"2023-07-13T22:20:55.610902+0000","last_undegraded":"2023-07-13T22:20:55.610902+0000","last_fullsized":"2023-07-13T22:20:55.610902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:47.554360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532565+0000","last_change":"2023-07-13T22:20:54.536450+0000","last_active":"2023-07-13T22:20:55.532565+0000","last_peered":"2023-07-13T22:20:55.532565+0000","last_clean":"2023-07-13T22:20:55.532565+0000","last_became_active":"2023-07-13T22:20:54.536099+0000","last_became_peered":"2023-07-13T22:20:54.536099+0000","last_unstale":"2023-07-13T22:20:55.532565+0000","last_undegraded":"2023-07-13T22:20:55.532565+0000","last_fullsized":"2023-07-13T22:20:55.532565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610785+0000","last_change":"2023-07-13T22:20:53.535131+0000","last_active":"2023-07-13T22:20:55.610785+0000","last_peered":"2023-07-13T22:20:55.610785+0000","last_clean":"2023-07-13T22:20:55.610785+0000","last_became_active":"2023-07-13T22:20:53.534737+0000","last_became_peered":"2023-07-13T22:20:53.534737+0000","last_unstale":"2023-07-13T22:20:55.610785+0000","last_undegraded":"2023-07-13T22:20:55.610785+0000","last_fullsized":"2023-07-13T22:20:55.610785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:16.293217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533196+0000","last_change":"2023-07-13T22:20:54.546160+0000","last_active":"2023-07-13T22:20:55.533196+0000","last_peered":"2023-07-13T22:20:55.533196+0000","last_clean":"2023-07-13T22:20:55.533196+0000","last_became_active":"2023-07-13T22:20:54.545713+0000","last_became_peered":"2023-07-13T22:20:54.545713+0000","last_unstale":"2023-07-13T22:20:55.533196+0000","last_undegraded":"2023-07-13T22:20:55.533196+0000","last_fullsized":"2023-07-13T22:20:55.533196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:17.301438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610989+0000","last_change":"2023-07-13T22:20:53.536620+0000","last_active":"2023-07-13T22:20:55.610989+0000","last_peered":"2023-07-13T22:20:55.610989+0000","last_clean":"2023-07-13T22:20:55.610989+0000","last_became_active":"2023-07-13T22:20:53.536322+0000","last_became_peered":"2023-07-13T22:20:53.536322+0000","last_unstale":"2023-07-13T22:20:55.610989+0000","last_undegraded":"2023-07-13T22:20:55.610989+0000","last_fullsized":"2023-07-13T22:20:55.610989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:55.228110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610640+0000","last_change":"2023-07-13T22:20:54.544907+0000","last_active":"2023-07-13T22:20:55.610640+0000","last_peered":"2023-07-13T22:20:55.610640+0000","last_clean":"2023-07-13T22:20:55.610640+0000","last_became_active":"2023-07-13T22:20:54.544596+0000","last_became_peered":"2023-07-13T22:20:54.544596+0000","last_unstale":"2023-07-13T22:20:55.610640+0000","last_undegraded":"2023-07-13T22:20:55.610640+0000","last_fullsized":"2023-07-13T22:20:55.610640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533567+0000","last_change":"2023-07-13T22:20:53.530981+0000","last_active":"2023-07-13T22:20:55.533567+0000","last_peered":"2023-07-13T22:20:55.533567+0000","last_clean":"2023-07-13T22:20:55.533567+0000","last_became_active":"2023-07-13T22:20:53.530568+0000","last_became_peered":"2023-07-13T22:20:53.530568+0000","last_unstale":"2023-07-13T22:20:55.533567+0000","last_undegraded":"2023-07-13T22:20:55.533567+0000","last_fullsized":"2023-07-13T22:20:55.533567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610942+0000","last_change":"2023-07-13T22:20:54.540498+0000","last_active":"2023-07-13T22:20:55.610942+0000","last_peered":"2023-07-13T22:20:55.610942+0000","last_clean":"2023-07-13T22:20:55.610942+0000","last_became_active":"2023-07-13T22:20:54.540161+0000","last_became_peered":"2023-07-13T22:20:54.540161+0000","last_unstale":"2023-07-13T22:20:55.610942+0000","last_undegraded":"2023-07-13T22:20:55.610942+0000","last_fullsized":"2023-07-13T22:20:55.610942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:12.599393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610764+0000","last_change":"2023-07-13T22:20:53.533470+0000","last_active":"2023-07-13T22:20:55.610764+0000","last_peered":"2023-07-13T22:20:55.610764+0000","last_clean":"2023-07-13T22:20:55.610764+0000","last_became_active":"2023-07-13T22:20:53.532854+0000","last_became_peered":"2023-07-13T22:20:53.532854+0000","last_unstale":"2023-07-13T22:20:55.610764+0000","last_undegraded":"2023-07-13T22:20:55.610764+0000","last_fullsized":"2023-07-13T22:20:55.610764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:37.890575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.534049+0000","last_change":"2023-07-13T22:20:54.552213+0000","last_active":"2023-07-13T22:20:55.534049+0000","last_peered":"2023-07-13T22:20:55.534049+0000","last_clean":"2023-07-13T22:20:55.534049+0000","last_became_active":"2023-07-13T22:20:54.548415+0000","last_became_peered":"2023-07-13T22:20:54.548415+0000","last_unstale":"2023-07-13T22:20:55.534049+0000","last_undegraded":"2023-07-13T22:20:55.534049+0000","last_fullsized":"2023-07-13T22:20:55.534049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:50.171609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610542+0000","last_change":"2023-07-13T22:20:53.529929+0000","last_active":"2023-07-13T22:20:55.610542+0000","last_peered":"2023-07-13T22:20:55.610542+0000","last_clean":"2023-07-13T22:20:55.610542+0000","last_became_active":"2023-07-13T22:20:53.529645+0000","last_became_peered":"2023-07-13T22:20:53.529645+0000","last_unstale":"2023-07-13T22:20:55.610542+0000","last_undegraded":"2023-07-13T22:20:55.610542+0000","last_fullsized":"2023-07-13T22:20:55.610542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:53.812690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535287+0000","last_change":"2023-07-13T22:20:54.550835+0000","last_active":"2023-07-13T22:20:55.535287+0000","last_peered":"2023-07-13T22:20:55.535287+0000","last_clean":"2023-07-13T22:20:55.535287+0000","last_became_active":"2023-07-13T22:20:54.547187+0000","last_became_peered":"2023-07-13T22:20:54.547187+0000","last_unstale":"2023-07-13T22:20:55.535287+0000","last_undegraded":"2023-07-13T22:20:55.535287+0000","last_fullsized":"2023-07-13T22:20:55.535287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:04:58.712616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533482+0000","last_change":"2023-07-13T22:20:53.534667+0000","last_active":"2023-07-13T22:20:55.533482+0000","last_peered":"2023-07-13T22:20:55.533482+0000","last_clean":"2023-07-13T22:20:55.533482+0000","last_became_active":"2023-07-13T22:20:53.534433+0000","last_became_peered":"2023-07-13T22:20:53.534433+0000","last_unstale":"2023-07-13T22:20:55.533482+0000","last_undegraded":"2023-07-13T22:20:55.533482+0000","last_fullsized":"2023-07-13T22:20:55.533482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:28.070264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535373+0000","last_change":"2023-07-13T22:20:54.550679+0000","last_active":"2023-07-13T22:20:55.535373+0000","last_peered":"2023-07-13T22:20:55.535373+0000","last_clean":"2023-07-13T22:20:55.535373+0000","last_became_active":"2023-07-13T22:20:54.546483+0000","last_became_peered":"2023-07-13T22:20:54.546483+0000","last_unstale":"2023-07-13T22:20:55.535373+0000","last_undegraded":"2023-07-13T22:20:55.535373+0000","last_fullsized":"2023-07-13T22:20:55.535373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:16:54.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610029+0000","last_change":"2023-07-13T22:20:53.531815+0000","last_active":"2023-07-13T22:20:55.610029+0000","last_peered":"2023-07-13T22:20:55.610029+0000","last_clean":"2023-07-13T22:20:55.610029+0000","last_became_active":"2023-07-13T22:20:53.531551+0000","last_became_peered":"2023-07-13T22:20:53.531551+0000","last_unstale":"2023-07-13T22:20:55.610029+0000","last_undegraded":"2023-07-13T22:20:55.610029+0000","last_fullsized":"2023-07-13T22:20:55.610029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531764+0000","last_change":"2023-07-13T22:20:54.542085+0000","last_active":"2023-07-13T22:20:55.531764+0000","last_peered":"2023-07-13T22:20:55.531764+0000","last_clean":"2023-07-13T22:20:55.531764+0000","last_became_active":"2023-07-13T22:20:54.541749+0000","last_became_peered":"2023-07-13T22:20:54.541749+0000","last_unstale":"2023-07-13T22:20:55.531764+0000","last_undegraded":"2023-07-13T22:20:55.531764+0000","last_fullsized":"2023-07-13T22:20:55.531764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610668+0000","last_change":"2023-07-13T22:20:53.539618+0000","last_active":"2023-07-13T22:20:55.610668+0000","last_peered":"2023-07-13T22:20:55.610668+0000","last_clean":"2023-07-13T22:20:55.610668+0000","last_became_active":"2023-07-13T22:20:53.539014+0000","last_became_peered":"2023-07-13T22:20:53.539014+0000","last_unstale":"2023-07-13T22:20:55.610668+0000","last_undegraded":"2023-07-13T22:20:55.610668+0000","last_fullsized":"2023-07-13T22:20:55.610668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:49.163388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535108+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:20:55.535108+0000","last_peered":"2023-07-13T22:20:55.535108+0000","last_clean":"2023-07-13T22:20:55.535108+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:20:55.535108+0000","last_undegraded":"2023-07-13T22:20:55.535108+0000","last_fullsized":"2023-07-13T22:20:55.535108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:11.435573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.609968+0000","last_change":"2023-07-13T22:20:54.538149+0000","last_active":"2023-07-13T22:20:55.609968+0000","last_peered":"2023-07-13T22:20:55.609968+0000","last_clean":"2023-07-13T22:20:55.609968+0000","last_became_active":"2023-07-13T22:20:54.537883+0000","last_became_peered":"2023-07-13T22:20:54.537883+0000","last_unstale":"2023-07-13T22:20:55.609968+0000","last_undegraded":"2023-07-13T22:20:55.609968+0000","last_fullsized":"2023-07-13T22:20:55.609968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610497+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:20:55.610497+0000","last_peered":"2023-07-13T22:20:55.610497+0000","last_clean":"2023-07-13T22:20:55.610497+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:20:55.610497+0000","last_undegraded":"2023-07-13T22:20:55.610497+0000","last_fullsized":"2023-07-13T22:20:55.610497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:49.435942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:56.195931+0000","last_change":"2023-07-13T22:20:53.541916+0000","last_active":"2023-07-13T22:20:56.195931+0000","last_peered":"2023-07-13T22:20:56.195931+0000","last_clean":"2023-07-13T22:20:56.195931+0000","last_became_active":"2023-07-13T22:20:53.541602+0000","last_became_peered":"2023-07-13T22:20:53.541602+0000","last_unstale":"2023-07-13T22:20:56.195931+0000","last_undegraded":"2023-07-13T22:20:56.195931+0000","last_fullsized":"2023-07-13T22:20:56.195931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:50.801197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533347+0000","last_change":"2023-07-13T22:20:54.545955+0000","last_active":"2023-07-13T22:20:55.533347+0000","last_peered":"2023-07-13T22:20:55.533347+0000","last_clean":"2023-07-13T22:20:55.533347+0000","last_became_active":"2023-07-13T22:20:54.545361+0000","last_became_peered":"2023-07-13T22:20:54.545361+0000","last_unstale":"2023-07-13T22:20:55.533347+0000","last_undegraded":"2023-07-13T22:20:55.533347+0000","last_fullsized":"2023-07-13T22:20:55.533347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531064+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:20:55.531064+0000","last_peered":"2023-07-13T22:20:55.531064+0000","last_clean":"2023-07-13T22:20:55.531064+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:20:55.531064+0000","last_undegraded":"2023-07-13T22:20:55.531064+0000","last_fullsized":"2023-07-13T22:20:55.531064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:15:56.981684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611242+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:20:55.611242+0000","last_peered":"2023-07-13T22:20:55.611242+0000","last_clean":"2023-07-13T22:20:55.611242+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:20:55.611242+0000","last_undegraded":"2023-07-13T22:20:55.611242+0000","last_fullsized":"2023-07-13T22:20:55.611242+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:44.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610851+0000","last_change":"2023-07-13T22:20:53.537543+0000","last_active":"2023-07-13T22:20:55.610851+0000","last_peered":"2023-07-13T22:20:55.610851+0000","last_clean":"2023-07-13T22:20:55.610851+0000","last_became_active":"2023-07-13T22:20:53.537125+0000","last_became_peered":"2023-07-13T22:20:53.537125+0000","last_unstale":"2023-07-13T22:20:55.610851+0000","last_undegraded":"2023-07-13T22:20:55.610851+0000","last_fullsized":"2023-07-13T22:20:55.610851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:48.574628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611216+0000","last_change":"2023-07-13T22:20:54.541651+0000","last_active":"2023-07-13T22:20:55.611216+0000","last_peered":"2023-07-13T22:20:55.611216+0000","last_clean":"2023-07-13T22:20:55.611216+0000","last_became_active":"2023-07-13T22:20:54.541059+0000","last_became_peered":"2023-07-13T22:20:54.541059+0000","last_unstale":"2023-07-13T22:20:55.611216+0000","last_undegraded":"2023-07-13T22:20:55.611216+0000","last_fullsized":"2023-07-13T22:20:55.611216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:54.820912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610481+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:20:55.610481+0000","last_peered":"2023-07-13T22:20:55.610481+0000","last_clean":"2023-07-13T22:20:55.610481+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:20:55.610481+0000","last_undegraded":"2023-07-13T22:20:55.610481+0000","last_fullsized":"2023-07-13T22:20:55.610481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:20.066975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.530855+0000","last_change":"2023-07-13T22:20:53.533784+0000","last_active":"2023-07-13T22:20:55.530855+0000","last_peered":"2023-07-13T22:20:55.530855+0000","last_clean":"2023-07-13T22:20:55.530855+0000","last_became_active":"2023-07-13T22:20:53.533270+0000","last_became_peered":"2023-07-13T22:20:53.533270+0000","last_unstale":"2023-07-13T22:20:55.530855+0000","last_undegraded":"2023-07-13T22:20:55.530855+0000","last_fullsized":"2023-07-13T22:20:55.530855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:29:28.555785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531674+0000","last_change":"2023-07-13T22:20:54.529627+0000","last_active":"2023-07-13T22:20:55.531674+0000","last_peered":"2023-07-13T22:20:55.531674+0000","last_clean":"2023-07-13T22:20:55.531674+0000","last_became_active":"2023-07-13T22:20:54.528020+0000","last_became_peered":"2023-07-13T22:20:54.528020+0000","last_unstale":"2023-07-13T22:20:55.531674+0000","last_undegraded":"2023-07-13T22:20:55.531674+0000","last_fullsized":"2023-07-13T22:20:55.531674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:54.820912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610893+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:20:55.610893+0000","last_peered":"2023-07-13T22:20:55.610893+0000","last_clean":"2023-07-13T22:20:55.610893+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:20:55.610893+0000","last_undegraded":"2023-07-13T22:20:55.610893+0000","last_fullsized":"2023-07-13T22:20:55.610893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:22.278097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610262+0000","last_change":"2023-07-13T22:20:53.535232+0000","last_active":"2023-07-13T22:20:55.610262+0000","last_peered":"2023-07-13T22:20:55.610262+0000","last_clean":"2023-07-13T22:20:55.610262+0000","last_became_active":"2023-07-13T22:20:53.533935+0000","last_became_peered":"2023-07-13T22:20:53.533935+0000","last_unstale":"2023-07-13T22:20:55.610262+0000","last_undegraded":"2023-07-13T22:20:55.610262+0000","last_fullsized":"2023-07-13T22:20:55.610262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610591+0000","last_change":"2023-07-13T22:20:54.549217+0000","last_active":"2023-07-13T22:20:55.610591+0000","last_peered":"2023-07-13T22:20:55.610591+0000","last_clean":"2023-07-13T22:20:55.610591+0000","last_became_active":"2023-07-13T22:20:54.548902+0000","last_became_peered":"2023-07-13T22:20:54.548902+0000","last_unstale":"2023-07-13T22:20:55.610591+0000","last_undegraded":"2023-07-13T22:20:55.610591+0000","last_fullsized":"2023-07-13T22:20:55.610591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532394+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:20:55.532394+0000","last_peered":"2023-07-13T22:20:55.532394+0000","last_clean":"2023-07-13T22:20:55.532394+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:20:55.532394+0000","last_undegraded":"2023-07-13T22:20:55.532394+0000","last_fullsized":"2023-07-13T22:20:55.532394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:05.799004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532935+0000","last_change":"2023-07-13T22:20:53.537640+0000","last_active":"2023-07-13T22:20:55.532935+0000","last_peered":"2023-07-13T22:20:55.532935+0000","last_clean":"2023-07-13T22:20:55.532935+0000","last_became_active":"2023-07-13T22:20:53.537351+0000","last_became_peered":"2023-07-13T22:20:53.537351+0000","last_unstale":"2023-07-13T22:20:55.532935+0000","last_undegraded":"2023-07-13T22:20:55.532935+0000","last_fullsized":"2023-07-13T22:20:55.532935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531434+0000","last_change":"2023-07-13T22:20:54.530546+0000","last_active":"2023-07-13T22:20:55.531434+0000","last_peered":"2023-07-13T22:20:55.531434+0000","last_clean":"2023-07-13T22:20:55.531434+0000","last_became_active":"2023-07-13T22:20:54.530234+0000","last_became_peered":"2023-07-13T22:20:54.530234+0000","last_unstale":"2023-07-13T22:20:55.531434+0000","last_undegraded":"2023-07-13T22:20:55.531434+0000","last_fullsized":"2023-07-13T22:20:55.531434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610282+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:20:55.610282+0000","last_peered":"2023-07-13T22:20:55.610282+0000","last_clean":"2023-07-13T22:20:55.610282+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:20:55.610282+0000","last_undegraded":"2023-07-13T22:20:55.610282+0000","last_fullsized":"2023-07-13T22:20:55.610282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:05.799004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531292+0000","last_change":"2023-07-13T22:20:53.533697+0000","last_active":"2023-07-13T22:20:55.531292+0000","last_peered":"2023-07-13T22:20:55.531292+0000","last_clean":"2023-07-13T22:20:55.531292+0000","last_became_active":"2023-07-13T22:20:53.533261+0000","last_became_peered":"2023-07-13T22:20:53.533261+0000","last_unstale":"2023-07-13T22:20:55.531292+0000","last_undegraded":"2023-07-13T22:20:55.531292+0000","last_fullsized":"2023-07-13T22:20:55.531292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:11.591172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611449+0000","last_change":"2023-07-13T22:20:54.537844+0000","last_active":"2023-07-13T22:20:55.611449+0000","last_peered":"2023-07-13T22:20:55.611449+0000","last_clean":"2023-07-13T22:20:55.611449+0000","last_became_active":"2023-07-13T22:20:54.537526+0000","last_became_peered":"2023-07-13T22:20:54.537526+0000","last_unstale":"2023-07-13T22:20:55.611449+0000","last_undegraded":"2023-07-13T22:20:55.611449+0000","last_fullsized":"2023-07-13T22:20:55.611449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610286+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:20:55.610286+0000","last_peered":"2023-07-13T22:20:55.610286+0000","last_clean":"2023-07-13T22:20:55.610286+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:20:55.610286+0000","last_undegraded":"2023-07-13T22:20:55.610286+0000","last_fullsized":"2023-07-13T22:20:55.610286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:17.601603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533295+0000","last_change":"2023-07-13T22:20:53.532273+0000","last_active":"2023-07-13T22:20:55.533295+0000","last_peered":"2023-07-13T22:20:55.533295+0000","last_clean":"2023-07-13T22:20:55.533295+0000","last_became_active":"2023-07-13T22:20:53.531699+0000","last_became_peered":"2023-07-13T22:20:53.531699+0000","last_unstale":"2023-07-13T22:20:55.533295+0000","last_undegraded":"2023-07-13T22:20:55.533295+0000","last_fullsized":"2023-07-13T22:20:55.533295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531599+0000","last_change":"2023-07-13T22:20:54.530026+0000","last_active":"2023-07-13T22:20:55.531599+0000","last_peered":"2023-07-13T22:20:55.531599+0000","last_clean":"2023-07-13T22:20:55.531599+0000","last_became_active":"2023-07-13T22:20:54.528217+0000","last_became_peered":"2023-07-13T22:20:54.528217+0000","last_unstale":"2023-07-13T22:20:55.531599+0000","last_undegraded":"2023-07-13T22:20:55.531599+0000","last_fullsized":"2023-07-13T22:20:55.531599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611152+0000","last_change":"2023-07-13T22:20:53.541066+0000","last_active":"2023-07-13T22:20:55.611152+0000","last_peered":"2023-07-13T22:20:55.611152+0000","last_clean":"2023-07-13T22:20:55.611152+0000","last_became_active":"2023-07-13T22:20:53.540748+0000","last_became_peered":"2023-07-13T22:20:53.540748+0000","last_unstale":"2023-07-13T22:20:55.611152+0000","last_undegraded":"2023-07-13T22:20:55.611152+0000","last_fullsized":"2023-07-13T22:20:55.611152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:25.360393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.612132+0000","last_change":"2023-07-13T22:20:54.531788+0000","last_active":"2023-07-13T22:20:55.612132+0000","last_peered":"2023-07-13T22:20:55.612132+0000","last_clean":"2023-07-13T22:20:55.612132+0000","last_became_active":"2023-07-13T22:20:54.531396+0000","last_became_peered":"2023-07-13T22:20:54.531396+0000","last_unstale":"2023-07-13T22:20:55.612132+0000","last_undegraded":"2023-07-13T22:20:55.612132+0000","last_fullsized":"2023-07-13T22:20:55.612132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:45:27.037988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533306+0000","last_change":"2023-07-13T22:20:53.530819+0000","last_active":"2023-07-13T22:20:55.533306+0000","last_peered":"2023-07-13T22:20:55.533306+0000","last_clean":"2023-07-13T22:20:55.533306+0000","last_became_active":"2023-07-13T22:20:53.530223+0000","last_became_peered":"2023-07-13T22:20:53.530223+0000","last_unstale":"2023-07-13T22:20:55.533306+0000","last_undegraded":"2023-07-13T22:20:55.533306+0000","last_fullsized":"2023-07-13T22:20:55.533306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:15:58.999140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532780+0000","last_change":"2023-07-13T22:20:54.551556+0000","last_active":"2023-07-13T22:20:55.532780+0000","last_peered":"2023-07-13T22:20:55.532780+0000","last_clean":"2023-07-13T22:20:55.532780+0000","last_became_active":"2023-07-13T22:20:54.547048+0000","last_became_peered":"2023-07-13T22:20:54.547048+0000","last_unstale":"2023-07-13T22:20:55.532780+0000","last_undegraded":"2023-07-13T22:20:55.532780+0000","last_fullsized":"2023-07-13T22:20:55.532780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533441+0000","last_change":"2023-07-13T22:20:53.532040+0000","last_active":"2023-07-13T22:20:55.533441+0000","last_peered":"2023-07-13T22:20:55.533441+0000","last_clean":"2023-07-13T22:20:55.533441+0000","last_became_active":"2023-07-13T22:20:53.531414+0000","last_became_peered":"2023-07-13T22:20:53.531414+0000","last_unstale":"2023-07-13T22:20:55.533441+0000","last_undegraded":"2023-07-13T22:20:55.533441+0000","last_fullsized":"2023-07-13T22:20:55.533441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:25.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533812+0000","last_change":"2023-07-13T22:20:54.551351+0000","last_active":"2023-07-13T22:20:55.533812+0000","last_peered":"2023-07-13T22:20:55.533812+0000","last_clean":"2023-07-13T22:20:55.533812+0000","last_became_active":"2023-07-13T22:20:54.550932+0000","last_became_peered":"2023-07-13T22:20:54.550932+0000","last_unstale":"2023-07-13T22:20:55.533812+0000","last_undegraded":"2023-07-13T22:20:55.533812+0000","last_fullsized":"2023-07-13T22:20:55.533812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:36.702847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610605+0000","last_change":"2023-07-13T22:20:53.530713+0000","last_active":"2023-07-13T22:20:55.610605+0000","last_peered":"2023-07-13T22:20:55.610605+0000","last_clean":"2023-07-13T22:20:55.610605+0000","last_became_active":"2023-07-13T22:20:53.530419+0000","last_became_peered":"2023-07-13T22:20:53.530419+0000","last_unstale":"2023-07-13T22:20:55.610605+0000","last_undegraded":"2023-07-13T22:20:55.610605+0000","last_fullsized":"2023-07-13T22:20:55.610605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:35.694626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610476+0000","last_change":"2023-07-13T22:20:54.541258+0000","last_active":"2023-07-13T22:20:55.610476+0000","last_peered":"2023-07-13T22:20:55.610476+0000","last_clean":"2023-07-13T22:20:55.610476+0000","last_became_active":"2023-07-13T22:20:54.540610+0000","last_became_peered":"2023-07-13T22:20:54.540610+0000","last_unstale":"2023-07-13T22:20:55.610476+0000","last_undegraded":"2023-07-13T22:20:55.610476+0000","last_fullsized":"2023-07-13T22:20:55.610476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:29.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.612057+0000","last_change":"2023-07-13T22:20:53.538279+0000","last_active":"2023-07-13T22:20:55.612057+0000","last_peered":"2023-07-13T22:20:55.612057+0000","last_clean":"2023-07-13T22:20:55.612057+0000","last_became_active":"2023-07-13T22:20:53.537651+0000","last_became_peered":"2023-07-13T22:20:53.537651+0000","last_unstale":"2023-07-13T22:20:55.612057+0000","last_undegraded":"2023-07-13T22:20:55.612057+0000","last_fullsized":"2023-07-13T22:20:55.612057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532866+0000","last_change":"2023-07-13T22:20:54.541753+0000","last_active":"2023-07-13T22:20:55.532866+0000","last_peered":"2023-07-13T22:20:55.532866+0000","last_clean":"2023-07-13T22:20:55.532866+0000","last_became_active":"2023-07-13T22:20:54.540946+0000","last_became_peered":"2023-07-13T22:20:54.540946+0000","last_unstale":"2023-07-13T22:20:55.532866+0000","last_undegraded":"2023-07-13T22:20:55.532866+0000","last_fullsized":"2023-07-13T22:20:55.532866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610348+0000","last_change":"2023-07-13T22:20:54.542872+0000","last_active":"2023-07-13T22:20:55.610348+0000","last_peered":"2023-07-13T22:20:55.610348+0000","last_clean":"2023-07-13T22:20:55.610348+0000","last_became_active":"2023-07-13T22:20:54.542534+0000","last_became_peered":"2023-07-13T22:20:54.542534+0000","last_unstale":"2023-07-13T22:20:55.610348+0000","last_undegraded":"2023-07-13T22:20:55.610348+0000","last_fullsized":"2023-07-13T22:20:55.610348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610319+0000","last_change":"2023-07-13T22:20:54.541488+0000","last_active":"2023-07-13T22:20:55.610319+0000","last_peered":"2023-07-13T22:20:55.610319+0000","last_clean":"2023-07-13T22:20:55.610319+0000","last_became_active":"2023-07-13T22:20:54.541003+0000","last_became_peered":"2023-07-13T22:20:54.541003+0000","last_unstale":"2023-07-13T22:20:55.610319+0000","last_undegraded":"2023-07-13T22:20:55.610319+0000","last_fullsized":"2023-07-13T22:20:55.610319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533452+0000","last_change":"2023-07-13T22:20:54.552460+0000","last_active":"2023-07-13T22:20:55.533452+0000","last_peered":"2023-07-13T22:20:55.533452+0000","last_clean":"2023-07-13T22:20:55.533452+0000","last_became_active":"2023-07-13T22:20:54.552208+0000","last_became_peered":"2023-07-13T22:20:54.552208+0000","last_unstale":"2023-07-13T22:20:55.533452+0000","last_undegraded":"2023-07-13T22:20:55.533452+0000","last_fullsized":"2023-07-13T22:20:55.533452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:48.562581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610888+0000","last_change":"2023-07-13T22:20:54.535876+0000","last_active":"2023-07-13T22:20:55.610888+0000","last_peered":"2023-07-13T22:20:55.610888+0000","last_clean":"2023-07-13T22:20:55.610888+0000","last_became_active":"2023-07-13T22:20:54.535570+0000","last_became_peered":"2023-07-13T22:20:54.535570+0000","last_unstale":"2023-07-13T22:20:55.610888+0000","last_undegraded":"2023-07-13T22:20:55.610888+0000","last_fullsized":"2023-07-13T22:20:55.610888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:18.235962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531178+0000","last_change":"2023-07-13T22:20:54.535911+0000","last_active":"2023-07-13T22:20:55.531178+0000","last_peered":"2023-07-13T22:20:55.531178+0000","last_clean":"2023-07-13T22:20:55.531178+0000","last_became_active":"2023-07-13T22:20:54.535593+0000","last_became_peered":"2023-07-13T22:20:54.535593+0000","last_unstale":"2023-07-13T22:20:55.531178+0000","last_undegraded":"2023-07-13T22:20:55.531178+0000","last_fullsized":"2023-07-13T22:20:55.531178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:43.402535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":671608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":78474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":81080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":80112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:21:04.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:21:05.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.097+0000 7ff417bff700 1 Processor -- start 2023-07-13T22:21:05.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.098+0000 7ff417bff700 1 -- start start 2023-07-13T22:21:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.099+0000 7ff417bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4100cfca0 0x7ff410167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.099+0000 7ff417bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4100cd950 con 0x7ff4100d3750 2023-07-13T22:21:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.099+0000 7ff417bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4100d0ac0 con 0x7ff4100b8400 2023-07-13T22:21:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.099+0000 7ff417bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4100b87d0 con 0x7ff4100cfca0 2023-07-13T22:21:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.099+0000 7ff41519a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff4100d3750 0x7ff4100cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47724/0 (socket says 172.21.15.60:47724) 2023-07-13T22:21:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.099+0000 7ff41519a700 1 -- 172.21.15.60:0/2436821587 learned_addr learned my addr 172.21.15.60:0/2436821587 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:05.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.099+0000 7ff41619c700 1 --2- 172.21.15.60:0/2436821587 >> v2:172.21.15.60:3300/0 conn(0x7ff4100cfca0 0x7ff410167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.100+0000 7ff41619c700 1 -- 172.21.15.60:0/2436821587 >> v1:172.21.15.105:6789/0 conn(0x7ff4100b8400 legacy=0x7ff4100cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.100+0000 7ff41619c700 1 -- 172.21.15.60:0/2436821587 >> v1:172.21.15.60:6789/0 conn(0x7ff4100d3750 legacy=0x7ff4100cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.100+0000 7ff41619c700 1 -- 172.21.15.60:0/2436821587 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4100b4eb0 con 0x7ff4100cfca0 2023-07-13T22:21:05.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.100+0000 7ff414999700 1 -- 172.21.15.60:0/2436821587 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167404070 0 0) 0x7ff400001110 con 0x7ff4100d3750 2023-07-13T22:21:05.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.100+0000 7ff41619c700 1 --2- 172.21.15.60:0/2436821587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100cfca0 0x7ff410167050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff408009d90 tx=0x7ff40800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ec282fc2d58ea9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:05.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.100+0000 7ff414999700 1 -- 172.21.15.60:0/2436821587 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40800e070 con 0x7ff4100cfca0 2023-07-13T22:21:05.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.100+0000 7ff414999700 1 -- 172.21.15.60:0/2436821587 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40800bbd0 con 0x7ff4100cfca0 2023-07-13T22:21:05.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.100+0000 7ff414999700 1 -- 172.21.15.60:0/2436821587 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408012430 con 0x7ff4100cfca0 2023-07-13T22:21:05.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.101+0000 7ff417bff700 1 -- 172.21.15.60:0/2436821587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100cfca0 msgr2=0x7ff410167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.101+0000 7ff417bff700 1 --2- 172.21.15.60:0/2436821587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100cfca0 0x7ff410167050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff408009d90 tx=0x7ff40800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:05.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.101+0000 7ff417bff700 1 -- 172.21.15.60:0/2436821587 shutdown_connections 2023-07-13T22:21:05.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.101+0000 7ff417bff700 1 --2- 172.21.15.60:0/2436821587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100cfca0 0x7ff410167050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.101+0000 7ff417bff700 1 -- 172.21.15.60:0/2436821587 >> 172.21.15.60:0/2436821587 conn(0x7ff41006b390 msgr2=0x7ff41006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:05.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.101+0000 7ff417bff700 1 -- 172.21.15.60:0/2436821587 shutdown_connections 2023-07-13T22:21:05.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.101+0000 7ff417bff700 1 -- 172.21.15.60:0/2436821587 wait complete. 2023-07-13T22:21:05.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.102+0000 7ff417bff700 1 Processor -- start 2023-07-13T22:21:05.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.103+0000 7ff417bff700 1 -- start start 2023-07-13T22:21:05.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.103+0000 7ff417bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100b8400 0x7ff410151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.103+0000 7ff417bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4100cfca0 0x7ff4100c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.103+0000 7ff417bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4100d3750 0x7ff4100b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.103+0000 7ff417bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100c95f0 con 0x7ff4100b8400 2023-07-13T22:21:05.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.103+0000 7ff417bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100c8a70 con 0x7ff4100d3750 2023-07-13T22:21:05.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.103+0000 7ff417bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100c8d50 con 0x7ff4100cfca0 2023-07-13T22:21:05.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100b8400 0x7ff410151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100b8400 0x7ff410151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45668/0 (socket says 172.21.15.60:45668) 2023-07-13T22:21:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41599b700 1 -- 172.21.15.60:0/1762856391 learned_addr learned my addr 172.21.15.60:0/1762856391 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41619c700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4100d3750 0x7ff4100b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41519a700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4100cfca0 0x7ff4100c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41599b700 1 -- 172.21.15.60:0/1762856391 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4100cfca0 msgr2=0x7ff4100c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41599b700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4100cfca0 0x7ff4100c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41599b700 1 -- 172.21.15.60:0/1762856391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4100d3750 msgr2=0x7ff4100b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41599b700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4100d3750 0x7ff4100b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41599b700 1 -- 172.21.15.60:0/1762856391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408009a70 con 0x7ff4100b8400 2023-07-13T22:21:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.104+0000 7ff41599b700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100b8400 0x7ff410151190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c450 tx=0x7ff40c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.105+0000 7ff4067fc700 1 -- 172.21.15.60:0/1762856391 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0052d0 con 0x7ff4100b8400 2023-07-13T22:21:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.105+0000 7ff4067fc700 1 -- 172.21.15.60:0/1762856391 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40c005430 con 0x7ff4100b8400 2023-07-13T22:21:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.105+0000 7ff4067fc700 1 -- 172.21.15.60:0/1762856391 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0038c0 con 0x7ff4100b8400 2023-07-13T22:21:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.105+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4100c9030 con 0x7ff4100b8400 2023-07-13T22:21:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.105+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4100c7620 con 0x7ff4100b8400 2023-07-13T22:21:05.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.106+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41015a350 con 0x7ff4100b8400 2023-07-13T22:21:05.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.107+0000 7ff4067fc700 1 -- 172.21.15.60:0/1762856391 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff40c003a20 con 0x7ff4100b8400 2023-07-13T22:21:05.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.108+0000 7ff4067fc700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f406ace0 0x7ff3f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.108+0000 7ff4067fc700 1 -- 172.21.15.60:0/1762856391 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff40c088820 con 0x7ff4100b8400 2023-07-13T22:21:05.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.108+0000 7ff41519a700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f406ace0 0x7ff3f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.109+0000 7ff41519a700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f406ace0 0x7ff3f406d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff400007e80 tx=0x7ff400007e10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:05.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.112+0000 7ff4067fc700 1 -- 172.21.15.60:0/1762856391 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff40c053830 con 0x7ff4100b8400 2023-07-13T22:21:05.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.312+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4100b5470 con 0x7ff3f406ace0 2023-07-13T22:21:05.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.320+0000 7ff4067fc700 1 -- 172.21.15.60:0/1762856391 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286827 (secure 0 0 0) 0x7ff400004a70 con 0x7ff3f406ace0 2023-07-13T22:21:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.324+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f406ace0 msgr2=0x7ff3f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.324+0000 7ff417bff700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f406ace0 0x7ff3f406d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff400007e80 tx=0x7ff400007e10 comp rx=0 tx=0).stop 2023-07-13T22:21:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.324+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100b8400 msgr2=0x7ff410151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.324+0000 7ff417bff700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100b8400 0x7ff410151190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c450 tx=0x7ff40c00c760 comp rx=0 tx=0).stop 2023-07-13T22:21:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.325+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 shutdown_connections 2023-07-13T22:21:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.325+0000 7ff417bff700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f406ace0 0x7ff3f406d190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.325+0000 7ff417bff700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4100b8400 0x7ff410151190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.325+0000 7ff417bff700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4100d3750 0x7ff4100b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.325+0000 7ff417bff700 1 --2- 172.21.15.60:0/1762856391 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4100cfca0 0x7ff4100c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.325+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 >> 172.21.15.60:0/1762856391 conn(0x7ff41006b390 msgr2=0x7ff41015b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:05.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.325+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 shutdown_connections 2023-07-13T22:21:05.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.325+0000 7ff417bff700 1 -- 172.21.15.60:0/1762856391 wait complete. 2023-07-13T22:21:05.311 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:21:05.327 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-07-13T22:21:05.127418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218528,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756192,"statfs":{"total":773094113280,"available":772870340608,"internally_reserved":0,"allocated":2981888,"data_stored":1819168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.009895"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611623+0000","last_change":"2023-07-13T22:20:54.539241+0000","last_active":"2023-07-13T22:20:55.611623+0000","last_peered":"2023-07-13T22:20:55.611623+0000","last_clean":"2023-07-13T22:20:55.611623+0000","last_became_active":"2023-07-13T22:20:54.538987+0000","last_became_peered":"2023-07-13T22:20:54.538987+0000","last_unstale":"2023-07-13T22:20:55.611623+0000","last_undegraded":"2023-07-13T22:20:55.611623+0000","last_fullsized":"2023-07-13T22:20:55.611623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:34:22.186098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532509+0000","last_change":"2023-07-13T22:20:54.537349+0000","last_active":"2023-07-13T22:20:55.532509+0000","last_peered":"2023-07-13T22:20:55.532509+0000","last_clean":"2023-07-13T22:20:55.532509+0000","last_became_active":"2023-07-13T22:20:54.536260+0000","last_became_peered":"2023-07-13T22:20:54.536260+0000","last_unstale":"2023-07-13T22:20:55.532509+0000","last_undegraded":"2023-07-13T22:20:55.532509+0000","last_fullsized":"2023-07-13T22:20:55.532509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532790+0000","last_change":"2023-07-13T22:20:54.543864+0000","last_active":"2023-07-13T22:20:55.532790+0000","last_peered":"2023-07-13T22:20:55.532790+0000","last_clean":"2023-07-13T22:20:55.532790+0000","last_became_active":"2023-07-13T22:20:54.543344+0000","last_became_peered":"2023-07-13T22:20:54.543344+0000","last_unstale":"2023-07-13T22:20:55.532790+0000","last_undegraded":"2023-07-13T22:20:55.532790+0000","last_fullsized":"2023-07-13T22:20:55.532790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610982+0000","last_change":"2023-07-13T22:20:54.540616+0000","last_active":"2023-07-13T22:20:55.610982+0000","last_peered":"2023-07-13T22:20:55.610982+0000","last_clean":"2023-07-13T22:20:55.610982+0000","last_became_active":"2023-07-13T22:20:54.540326+0000","last_became_peered":"2023-07-13T22:20:54.540326+0000","last_unstale":"2023-07-13T22:20:55.610982+0000","last_undegraded":"2023-07-13T22:20:55.610982+0000","last_fullsized":"2023-07-13T22:20:55.610982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611292+0000","last_change":"2023-07-13T22:20:54.550022+0000","last_active":"2023-07-13T22:20:55.611292+0000","last_peered":"2023-07-13T22:20:55.611292+0000","last_clean":"2023-07-13T22:20:55.611292+0000","last_became_active":"2023-07-13T22:20:54.549249+0000","last_became_peered":"2023-07-13T22:20:54.549249+0000","last_unstale":"2023-07-13T22:20:55.611292+0000","last_undegraded":"2023-07-13T22:20:55.611292+0000","last_fullsized":"2023-07-13T22:20:55.611292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533320+0000","last_change":"2023-07-13T22:20:54.544989+0000","last_active":"2023-07-13T22:20:55.533320+0000","last_peered":"2023-07-13T22:20:55.533320+0000","last_clean":"2023-07-13T22:20:55.533320+0000","last_became_active":"2023-07-13T22:20:54.544719+0000","last_became_peered":"2023-07-13T22:20:54.544719+0000","last_unstale":"2023-07-13T22:20:55.533320+0000","last_undegraded":"2023-07-13T22:20:55.533320+0000","last_fullsized":"2023-07-13T22:20:55.533320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:55.546960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610444+0000","last_change":"2023-07-13T22:20:54.543266+0000","last_active":"2023-07-13T22:20:55.610444+0000","last_peered":"2023-07-13T22:20:55.610444+0000","last_clean":"2023-07-13T22:20:55.610444+0000","last_became_active":"2023-07-13T22:20:54.542870+0000","last_became_peered":"2023-07-13T22:20:54.542870+0000","last_unstale":"2023-07-13T22:20:55.610444+0000","last_undegraded":"2023-07-13T22:20:55.610444+0000","last_fullsized":"2023-07-13T22:20:55.610444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532392+0000","last_change":"2023-07-13T22:20:54.546394+0000","last_active":"2023-07-13T22:20:55.532392+0000","last_peered":"2023-07-13T22:20:55.532392+0000","last_clean":"2023-07-13T22:20:55.532392+0000","last_became_active":"2023-07-13T22:20:54.545682+0000","last_became_peered":"2023-07-13T22:20:54.545682+0000","last_unstale":"2023-07-13T22:20:55.532392+0000","last_undegraded":"2023-07-13T22:20:55.532392+0000","last_fullsized":"2023-07-13T22:20:55.532392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611695+0000","last_change":"2023-07-13T22:20:54.549925+0000","last_active":"2023-07-13T22:20:55.611695+0000","last_peered":"2023-07-13T22:20:55.611695+0000","last_clean":"2023-07-13T22:20:55.611695+0000","last_became_active":"2023-07-13T22:20:54.549641+0000","last_became_peered":"2023-07-13T22:20:54.549641+0000","last_unstale":"2023-07-13T22:20:55.611695+0000","last_undegraded":"2023-07-13T22:20:55.611695+0000","last_fullsized":"2023-07-13T22:20:55.611695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:45.329435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532781+0000","last_change":"2023-07-13T22:20:54.543479+0000","last_active":"2023-07-13T22:20:55.532781+0000","last_peered":"2023-07-13T22:20:55.532781+0000","last_clean":"2023-07-13T22:20:55.532781+0000","last_became_active":"2023-07-13T22:20:54.543143+0000","last_became_peered":"2023-07-13T22:20:54.543143+0000","last_unstale":"2023-07-13T22:20:55.532781+0000","last_undegraded":"2023-07-13T22:20:55.532781+0000","last_fullsized":"2023-07-13T22:20:55.532781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532989+0000","last_change":"2023-07-13T22:20:54.543686+0000","last_active":"2023-07-13T22:20:55.532989+0000","last_peered":"2023-07-13T22:20:55.532989+0000","last_clean":"2023-07-13T22:20:55.532989+0000","last_became_active":"2023-07-13T22:20:54.543001+0000","last_became_peered":"2023-07-13T22:20:54.543001+0000","last_unstale":"2023-07-13T22:20:55.532989+0000","last_undegraded":"2023-07-13T22:20:55.532989+0000","last_fullsized":"2023-07-13T22:20:55.532989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611996+0000","last_change":"2023-07-13T22:20:54.547637+0000","last_active":"2023-07-13T22:20:55.611996+0000","last_peered":"2023-07-13T22:20:55.611996+0000","last_clean":"2023-07-13T22:20:55.611996+0000","last_became_active":"2023-07-13T22:20:54.547280+0000","last_became_peered":"2023-07-13T22:20:54.547280+0000","last_unstale":"2023-07-13T22:20:55.611996+0000","last_undegraded":"2023-07-13T22:20:55.611996+0000","last_fullsized":"2023-07-13T22:20:55.611996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:18:24.767625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611891+0000","last_change":"2023-07-13T22:20:54.538682+0000","last_active":"2023-07-13T22:20:55.611891+0000","last_peered":"2023-07-13T22:20:55.611891+0000","last_clean":"2023-07-13T22:20:55.611891+0000","last_became_active":"2023-07-13T22:20:54.538356+0000","last_became_peered":"2023-07-13T22:20:54.538356+0000","last_unstale":"2023-07-13T22:20:55.611891+0000","last_undegraded":"2023-07-13T22:20:55.611891+0000","last_fullsized":"2023-07-13T22:20:55.611891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:51:10.800579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533030+0000","last_change":"2023-07-13T22:20:54.540925+0000","last_active":"2023-07-13T22:20:55.533030+0000","last_peered":"2023-07-13T22:20:55.533030+0000","last_clean":"2023-07-13T22:20:55.533030+0000","last_became_active":"2023-07-13T22:20:54.540646+0000","last_became_peered":"2023-07-13T22:20:54.540646+0000","last_unstale":"2023-07-13T22:20:55.533030+0000","last_undegraded":"2023-07-13T22:20:55.533030+0000","last_fullsized":"2023-07-13T22:20:55.533030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610864+0000","last_change":"2023-07-13T22:20:54.543844+0000","last_active":"2023-07-13T22:20:55.610864+0000","last_peered":"2023-07-13T22:20:55.610864+0000","last_clean":"2023-07-13T22:20:55.610864+0000","last_became_active":"2023-07-13T22:20:54.543527+0000","last_became_peered":"2023-07-13T22:20:54.543527+0000","last_unstale":"2023-07-13T22:20:55.610864+0000","last_undegraded":"2023-07-13T22:20:55.610864+0000","last_fullsized":"2023-07-13T22:20:55.610864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533022+0000","last_change":"2023-07-13T22:20:54.537914+0000","last_active":"2023-07-13T22:20:55.533022+0000","last_peered":"2023-07-13T22:20:55.533022+0000","last_clean":"2023-07-13T22:20:55.533022+0000","last_became_active":"2023-07-13T22:20:54.537562+0000","last_became_peered":"2023-07-13T22:20:54.537562+0000","last_unstale":"2023-07-13T22:20:55.533022+0000","last_undegraded":"2023-07-13T22:20:55.533022+0000","last_fullsized":"2023-07-13T22:20:55.533022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:26.867364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611835+0000","last_change":"2023-07-13T22:20:54.549793+0000","last_active":"2023-07-13T22:20:55.611835+0000","last_peered":"2023-07-13T22:20:55.611835+0000","last_clean":"2023-07-13T22:20:55.611835+0000","last_became_active":"2023-07-13T22:20:54.549473+0000","last_became_peered":"2023-07-13T22:20:54.549473+0000","last_unstale":"2023-07-13T22:20:55.611835+0000","last_undegraded":"2023-07-13T22:20:55.611835+0000","last_fullsized":"2023-07-13T22:20:55.611835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535307+0000","last_change":"2023-07-13T22:20:54.541942+0000","last_active":"2023-07-13T22:20:55.535307+0000","last_peered":"2023-07-13T22:20:55.535307+0000","last_clean":"2023-07-13T22:20:55.535307+0000","last_became_active":"2023-07-13T22:20:54.541314+0000","last_became_peered":"2023-07-13T22:20:54.541314+0000","last_unstale":"2023-07-13T22:20:55.535307+0000","last_undegraded":"2023-07-13T22:20:55.535307+0000","last_fullsized":"2023-07-13T22:20:55.535307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:10:09.528351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611873+0000","last_change":"2023-07-13T22:20:54.532924+0000","last_active":"2023-07-13T22:20:55.611873+0000","last_peered":"2023-07-13T22:20:55.611873+0000","last_clean":"2023-07-13T22:20:55.611873+0000","last_became_active":"2023-07-13T22:20:54.531937+0000","last_became_peered":"2023-07-13T22:20:54.531937+0000","last_unstale":"2023-07-13T22:20:55.611873+0000","last_undegraded":"2023-07-13T22:20:55.611873+0000","last_fullsized":"2023-07-13T22:20:55.611873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:20:04.410391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533904+0000","last_change":"2023-07-13T22:20:54.552416+0000","last_active":"2023-07-13T22:20:55.533904+0000","last_peered":"2023-07-13T22:20:55.533904+0000","last_clean":"2023-07-13T22:20:55.533904+0000","last_became_active":"2023-07-13T22:20:54.551516+0000","last_became_peered":"2023-07-13T22:20:54.551516+0000","last_unstale":"2023-07-13T22:20:55.533904+0000","last_undegraded":"2023-07-13T22:20:55.533904+0000","last_fullsized":"2023-07-13T22:20:55.533904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611325+0000","last_change":"2023-07-13T22:20:54.551898+0000","last_active":"2023-07-13T22:20:55.611325+0000","last_peered":"2023-07-13T22:20:55.611325+0000","last_clean":"2023-07-13T22:20:55.611325+0000","last_became_active":"2023-07-13T22:20:54.551538+0000","last_became_peered":"2023-07-13T22:20:54.551538+0000","last_unstale":"2023-07-13T22:20:55.611325+0000","last_undegraded":"2023-07-13T22:20:55.611325+0000","last_fullsized":"2023-07-13T22:20:55.611325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:38.898796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610399+0000","last_change":"2023-07-13T22:20:54.540580+0000","last_active":"2023-07-13T22:20:55.610399+0000","last_peered":"2023-07-13T22:20:55.610399+0000","last_clean":"2023-07-13T22:20:55.610399+0000","last_became_active":"2023-07-13T22:20:54.540289+0000","last_became_peered":"2023-07-13T22:20:54.540289+0000","last_unstale":"2023-07-13T22:20:55.610399+0000","last_undegraded":"2023-07-13T22:20:55.610399+0000","last_fullsized":"2023-07-13T22:20:55.610399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:43.402535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610169+0000","last_change":"2023-07-13T22:20:54.536952+0000","last_active":"2023-07-13T22:20:55.610169+0000","last_peered":"2023-07-13T22:20:55.610169+0000","last_clean":"2023-07-13T22:20:55.610169+0000","last_became_active":"2023-07-13T22:20:54.536696+0000","last_became_peered":"2023-07-13T22:20:54.536696+0000","last_unstale":"2023-07-13T22:20:55.610169+0000","last_undegraded":"2023-07-13T22:20:55.610169+0000","last_fullsized":"2023-07-13T22:20:55.610169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:26.867364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611257+0000","last_change":"2023-07-13T22:20:54.550043+0000","last_active":"2023-07-13T22:20:55.611257+0000","last_peered":"2023-07-13T22:20:55.611257+0000","last_clean":"2023-07-13T22:20:55.611257+0000","last_became_active":"2023-07-13T22:20:54.549754+0000","last_became_peered":"2023-07-13T22:20:54.549754+0000","last_unstale":"2023-07-13T22:20:55.611257+0000","last_undegraded":"2023-07-13T22:20:55.611257+0000","last_fullsized":"2023-07-13T22:20:55.611257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531264+0000","last_change":"2023-07-13T22:20:54.534540+0000","last_active":"2023-07-13T22:20:55.531264+0000","last_peered":"2023-07-13T22:20:55.531264+0000","last_clean":"2023-07-13T22:20:55.531264+0000","last_became_active":"2023-07-13T22:20:54.534251+0000","last_became_peered":"2023-07-13T22:20:54.534251+0000","last_unstale":"2023-07-13T22:20:55.531264+0000","last_undegraded":"2023-07-13T22:20:55.531264+0000","last_fullsized":"2023-07-13T22:20:55.531264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531735+0000","last_change":"2023-07-13T22:20:54.529221+0000","last_active":"2023-07-13T22:20:55.531735+0000","last_peered":"2023-07-13T22:20:55.531735+0000","last_clean":"2023-07-13T22:20:55.531735+0000","last_became_active":"2023-07-13T22:20:54.528934+0000","last_became_peered":"2023-07-13T22:20:54.528934+0000","last_unstale":"2023-07-13T22:20:55.531735+0000","last_undegraded":"2023-07-13T22:20:55.531735+0000","last_fullsized":"2023-07-13T22:20:55.531735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533014+0000","last_change":"2023-07-13T22:20:54.539287+0000","last_active":"2023-07-13T22:20:55.533014+0000","last_peered":"2023-07-13T22:20:55.533014+0000","last_clean":"2023-07-13T22:20:55.533014+0000","last_became_active":"2023-07-13T22:20:54.537991+0000","last_became_peered":"2023-07-13T22:20:54.537991+0000","last_unstale":"2023-07-13T22:20:55.533014+0000","last_undegraded":"2023-07-13T22:20:55.533014+0000","last_fullsized":"2023-07-13T22:20:55.533014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533580+0000","last_change":"2023-07-13T22:20:53.534188+0000","last_active":"2023-07-13T22:20:55.533580+0000","last_peered":"2023-07-13T22:20:55.533580+0000","last_clean":"2023-07-13T22:20:55.533580+0000","last_became_active":"2023-07-13T22:20:53.533888+0000","last_became_peered":"2023-07-13T22:20:53.533888+0000","last_unstale":"2023-07-13T22:20:55.533580+0000","last_undegraded":"2023-07-13T22:20:55.533580+0000","last_fullsized":"2023-07-13T22:20:55.533580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:55.228110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610249+0000","last_change":"2023-07-13T22:20:54.541751+0000","last_active":"2023-07-13T22:20:55.610249+0000","last_peered":"2023-07-13T22:20:55.610249+0000","last_clean":"2023-07-13T22:20:55.610249+0000","last_became_active":"2023-07-13T22:20:54.541506+0000","last_became_peered":"2023-07-13T22:20:54.541506+0000","last_unstale":"2023-07-13T22:20:55.610249+0000","last_undegraded":"2023-07-13T22:20:55.610249+0000","last_fullsized":"2023-07-13T22:20:55.610249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:55.546960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531119+0000","last_change":"2023-07-13T22:20:53.531325+0000","last_active":"2023-07-13T22:20:55.531119+0000","last_peered":"2023-07-13T22:20:55.531119+0000","last_clean":"2023-07-13T22:20:55.531119+0000","last_became_active":"2023-07-13T22:20:53.530929+0000","last_became_peered":"2023-07-13T22:20:53.530929+0000","last_unstale":"2023-07-13T22:20:55.531119+0000","last_undegraded":"2023-07-13T22:20:55.531119+0000","last_fullsized":"2023-07-13T22:20:55.531119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:07:16.260303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611424+0000","last_change":"2023-07-13T22:20:54.541835+0000","last_active":"2023-07-13T22:20:55.611424+0000","last_peered":"2023-07-13T22:20:55.611424+0000","last_clean":"2023-07-13T22:20:55.611424+0000","last_became_active":"2023-07-13T22:20:54.541371+0000","last_became_peered":"2023-07-13T22:20:54.541371+0000","last_unstale":"2023-07-13T22:20:55.611424+0000","last_undegraded":"2023-07-13T22:20:55.611424+0000","last_fullsized":"2023-07-13T22:20:55.611424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610264+0000","last_change":"2023-07-13T22:20:53.534081+0000","last_active":"2023-07-13T22:20:55.610264+0000","last_peered":"2023-07-13T22:20:55.610264+0000","last_clean":"2023-07-13T22:20:55.610264+0000","last_became_active":"2023-07-13T22:20:53.532985+0000","last_became_peered":"2023-07-13T22:20:53.532985+0000","last_unstale":"2023-07-13T22:20:55.610264+0000","last_undegraded":"2023-07-13T22:20:55.610264+0000","last_fullsized":"2023-07-13T22:20:55.610264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:16.293217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610466+0000","last_change":"2023-07-13T22:20:54.533387+0000","last_active":"2023-07-13T22:20:55.610466+0000","last_peered":"2023-07-13T22:20:55.610466+0000","last_clean":"2023-07-13T22:20:55.610466+0000","last_became_active":"2023-07-13T22:20:54.533023+0000","last_became_peered":"2023-07-13T22:20:54.533023+0000","last_unstale":"2023-07-13T22:20:55.610466+0000","last_undegraded":"2023-07-13T22:20:55.610466+0000","last_fullsized":"2023-07-13T22:20:55.610466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611957+0000","last_change":"2023-07-13T22:20:53.538156+0000","last_active":"2023-07-13T22:20:55.611957+0000","last_peered":"2023-07-13T22:20:55.611957+0000","last_clean":"2023-07-13T22:20:55.611957+0000","last_became_active":"2023-07-13T22:20:53.536726+0000","last_became_peered":"2023-07-13T22:20:53.536726+0000","last_unstale":"2023-07-13T22:20:55.611957+0000","last_undegraded":"2023-07-13T22:20:55.611957+0000","last_fullsized":"2023-07-13T22:20:55.611957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:25.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610113+0000","last_change":"2023-07-13T22:20:54.541258+0000","last_active":"2023-07-13T22:20:55.610113+0000","last_peered":"2023-07-13T22:20:55.610113+0000","last_clean":"2023-07-13T22:20:55.610113+0000","last_became_active":"2023-07-13T22:20:54.540986+0000","last_became_peered":"2023-07-13T22:20:54.540986+0000","last_unstale":"2023-07-13T22:20:55.610113+0000","last_undegraded":"2023-07-13T22:20:55.610113+0000","last_fullsized":"2023-07-13T22:20:55.610113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:00.007361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532649+0000","last_change":"2023-07-13T22:20:53.536025+0000","last_active":"2023-07-13T22:20:55.532649+0000","last_peered":"2023-07-13T22:20:55.532649+0000","last_clean":"2023-07-13T22:20:55.532649+0000","last_became_active":"2023-07-13T22:20:53.535697+0000","last_became_peered":"2023-07-13T22:20:53.535697+0000","last_unstale":"2023-07-13T22:20:55.532649+0000","last_undegraded":"2023-07-13T22:20:55.532649+0000","last_fullsized":"2023-07-13T22:20:55.532649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:53.812690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610659+0000","last_change":"2023-07-13T22:20:54.537549+0000","last_active":"2023-07-13T22:20:55.610659+0000","last_peered":"2023-07-13T22:20:55.610659+0000","last_clean":"2023-07-13T22:20:55.610659+0000","last_became_active":"2023-07-13T22:20:54.537260+0000","last_became_peered":"2023-07-13T22:20:54.537260+0000","last_unstale":"2023-07-13T22:20:55.610659+0000","last_undegraded":"2023-07-13T22:20:55.610659+0000","last_fullsized":"2023-07-13T22:20:55.610659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:51.434265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611165+0000","last_change":"2023-07-13T22:20:53.543492+0000","last_active":"2023-07-13T22:20:55.611165+0000","last_peered":"2023-07-13T22:20:55.611165+0000","last_clean":"2023-07-13T22:20:55.611165+0000","last_became_active":"2023-07-13T22:20:53.543265+0000","last_became_peered":"2023-07-13T22:20:53.543265+0000","last_unstale":"2023-07-13T22:20:55.611165+0000","last_undegraded":"2023-07-13T22:20:55.611165+0000","last_fullsized":"2023-07-13T22:20:55.611165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:35.694626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531212+0000","last_change":"2023-07-13T22:20:54.532326+0000","last_active":"2023-07-13T22:20:55.531212+0000","last_peered":"2023-07-13T22:20:55.531212+0000","last_clean":"2023-07-13T22:20:55.531212+0000","last_became_active":"2023-07-13T22:20:54.532144+0000","last_became_peered":"2023-07-13T22:20:54.532144+0000","last_unstale":"2023-07-13T22:20:55.531212+0000","last_undegraded":"2023-07-13T22:20:55.531212+0000","last_fullsized":"2023-07-13T22:20:55.531212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:24.401992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.534193+0000","last_change":"2023-07-13T22:20:53.531850+0000","last_active":"2023-07-13T22:20:55.534193+0000","last_peered":"2023-07-13T22:20:55.534193+0000","last_clean":"2023-07-13T22:20:55.534193+0000","last_became_active":"2023-07-13T22:20:53.531567+0000","last_became_peered":"2023-07-13T22:20:53.531567+0000","last_unstale":"2023-07-13T22:20:55.534193+0000","last_undegraded":"2023-07-13T22:20:55.534193+0000","last_fullsized":"2023-07-13T22:20:55.534193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:37.890575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533168+0000","last_change":"2023-07-13T22:20:54.552096+0000","last_active":"2023-07-13T22:20:55.533168+0000","last_peered":"2023-07-13T22:20:55.533168+0000","last_clean":"2023-07-13T22:20:55.533168+0000","last_became_active":"2023-07-13T22:20:54.551842+0000","last_became_peered":"2023-07-13T22:20:54.551842+0000","last_unstale":"2023-07-13T22:20:55.533168+0000","last_undegraded":"2023-07-13T22:20:55.533168+0000","last_fullsized":"2023-07-13T22:20:55.533168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:08.460983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610860+0000","last_change":"2023-07-13T22:20:53.538459+0000","last_active":"2023-07-13T22:20:55.610860+0000","last_peered":"2023-07-13T22:20:55.610860+0000","last_clean":"2023-07-13T22:20:55.610860+0000","last_became_active":"2023-07-13T22:20:53.538180+0000","last_became_peered":"2023-07-13T22:20:53.538180+0000","last_unstale":"2023-07-13T22:20:55.610860+0000","last_undegraded":"2023-07-13T22:20:55.610860+0000","last_fullsized":"2023-07-13T22:20:55.610860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:54.538739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610410+0000","last_change":"2023-07-13T22:20:54.542603+0000","last_active":"2023-07-13T22:20:55.610410+0000","last_peered":"2023-07-13T22:20:55.610410+0000","last_clean":"2023-07-13T22:20:55.610410+0000","last_became_active":"2023-07-13T22:20:54.542348+0000","last_became_peered":"2023-07-13T22:20:54.542348+0000","last_unstale":"2023-07-13T22:20:55.610410+0000","last_undegraded":"2023-07-13T22:20:55.610410+0000","last_fullsized":"2023-07-13T22:20:55.610410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:48.562581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610578+0000","last_change":"2023-07-13T22:20:53.539294+0000","last_active":"2023-07-13T22:20:55.610578+0000","last_peered":"2023-07-13T22:20:55.610578+0000","last_clean":"2023-07-13T22:20:55.610578+0000","last_became_active":"2023-07-13T22:20:53.538780+0000","last_became_peered":"2023-07-13T22:20:53.538780+0000","last_unstale":"2023-07-13T22:20:55.610578+0000","last_undegraded":"2023-07-13T22:20:55.610578+0000","last_fullsized":"2023-07-13T22:20:55.610578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:23.393770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533163+0000","last_change":"2023-07-13T22:20:54.538797+0000","last_active":"2023-07-13T22:20:55.533163+0000","last_peered":"2023-07-13T22:20:55.533163+0000","last_clean":"2023-07-13T22:20:55.533163+0000","last_became_active":"2023-07-13T22:20:54.537618+0000","last_became_peered":"2023-07-13T22:20:54.537618+0000","last_unstale":"2023-07-13T22:20:55.533163+0000","last_undegraded":"2023-07-13T22:20:55.533163+0000","last_fullsized":"2023-07-13T22:20:55.533163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610621+0000","last_change":"2023-07-13T22:20:53.540448+0000","last_active":"2023-07-13T22:20:55.610621+0000","last_peered":"2023-07-13T22:20:55.610621+0000","last_clean":"2023-07-13T22:20:55.610621+0000","last_became_active":"2023-07-13T22:20:53.540124+0000","last_became_peered":"2023-07-13T22:20:53.540124+0000","last_unstale":"2023-07-13T22:20:55.610621+0000","last_undegraded":"2023-07-13T22:20:55.610621+0000","last_fullsized":"2023-07-13T22:20:55.610621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611081+0000","last_change":"2023-07-13T22:20:54.552868+0000","last_active":"2023-07-13T22:20:55.611081+0000","last_peered":"2023-07-13T22:20:55.611081+0000","last_clean":"2023-07-13T22:20:55.611081+0000","last_became_active":"2023-07-13T22:20:54.552531+0000","last_became_peered":"2023-07-13T22:20:54.552531+0000","last_unstale":"2023-07-13T22:20:55.611081+0000","last_undegraded":"2023-07-13T22:20:55.611081+0000","last_fullsized":"2023-07-13T22:20:55.611081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610438+0000","last_change":"2023-07-13T22:20:53.533055+0000","last_active":"2023-07-13T22:20:55.610438+0000","last_peered":"2023-07-13T22:20:55.610438+0000","last_clean":"2023-07-13T22:20:55.610438+0000","last_became_active":"2023-07-13T22:20:53.532792+0000","last_became_peered":"2023-07-13T22:20:53.532792+0000","last_unstale":"2023-07-13T22:20:55.610438+0000","last_undegraded":"2023-07-13T22:20:55.610438+0000","last_fullsized":"2023-07-13T22:20:55.610438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:28.638041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610888+0000","last_change":"2023-07-13T22:20:54.536719+0000","last_active":"2023-07-13T22:20:55.610888+0000","last_peered":"2023-07-13T22:20:55.610888+0000","last_clean":"2023-07-13T22:20:55.610888+0000","last_became_active":"2023-07-13T22:20:54.535865+0000","last_became_peered":"2023-07-13T22:20:54.535865+0000","last_unstale":"2023-07-13T22:20:55.610888+0000","last_undegraded":"2023-07-13T22:20:55.610888+0000","last_fullsized":"2023-07-13T22:20:55.610888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:04:58.712616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610902+0000","last_change":"2023-07-13T22:20:53.537777+0000","last_active":"2023-07-13T22:20:55.610902+0000","last_peered":"2023-07-13T22:20:55.610902+0000","last_clean":"2023-07-13T22:20:55.610902+0000","last_became_active":"2023-07-13T22:20:53.537464+0000","last_became_peered":"2023-07-13T22:20:53.537464+0000","last_unstale":"2023-07-13T22:20:55.610902+0000","last_undegraded":"2023-07-13T22:20:55.610902+0000","last_fullsized":"2023-07-13T22:20:55.610902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:47.554360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532565+0000","last_change":"2023-07-13T22:20:54.536450+0000","last_active":"2023-07-13T22:20:55.532565+0000","last_peered":"2023-07-13T22:20:55.532565+0000","last_clean":"2023-07-13T22:20:55.532565+0000","last_became_active":"2023-07-13T22:20:54.536099+0000","last_became_peered":"2023-07-13T22:20:54.536099+0000","last_unstale":"2023-07-13T22:20:55.532565+0000","last_undegraded":"2023-07-13T22:20:55.532565+0000","last_fullsized":"2023-07-13T22:20:55.532565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610785+0000","last_change":"2023-07-13T22:20:53.535131+0000","last_active":"2023-07-13T22:20:55.610785+0000","last_peered":"2023-07-13T22:20:55.610785+0000","last_clean":"2023-07-13T22:20:55.610785+0000","last_became_active":"2023-07-13T22:20:53.534737+0000","last_became_peered":"2023-07-13T22:20:53.534737+0000","last_unstale":"2023-07-13T22:20:55.610785+0000","last_undegraded":"2023-07-13T22:20:55.610785+0000","last_fullsized":"2023-07-13T22:20:55.610785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:16.293217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533196+0000","last_change":"2023-07-13T22:20:54.546160+0000","last_active":"2023-07-13T22:20:55.533196+0000","last_peered":"2023-07-13T22:20:55.533196+0000","last_clean":"2023-07-13T22:20:55.533196+0000","last_became_active":"2023-07-13T22:20:54.545713+0000","last_became_peered":"2023-07-13T22:20:54.545713+0000","last_unstale":"2023-07-13T22:20:55.533196+0000","last_undegraded":"2023-07-13T22:20:55.533196+0000","last_fullsized":"2023-07-13T22:20:55.533196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:17.301438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610989+0000","last_change":"2023-07-13T22:20:53.536620+0000","last_active":"2023-07-13T22:20:55.610989+0000","last_peered":"2023-07-13T22:20:55.610989+0000","last_clean":"2023-07-13T22:20:55.610989+0000","last_became_active":"2023-07-13T22:20:53.536322+0000","last_became_peered":"2023-07-13T22:20:53.536322+0000","last_unstale":"2023-07-13T22:20:55.610989+0000","last_undegraded":"2023-07-13T22:20:55.610989+0000","last_fullsized":"2023-07-13T22:20:55.610989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:55.228110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610640+0000","last_change":"2023-07-13T22:20:54.544907+0000","last_active":"2023-07-13T22:20:55.610640+0000","last_peered":"2023-07-13T22:20:55.610640+0000","last_clean":"2023-07-13T22:20:55.610640+0000","last_became_active":"2023-07-13T22:20:54.544596+0000","last_became_peered":"2023-07-13T22:20:54.544596+0000","last_unstale":"2023-07-13T22:20:55.610640+0000","last_undegraded":"2023-07-13T22:20:55.610640+0000","last_fullsized":"2023-07-13T22:20:55.610640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533567+0000","last_change":"2023-07-13T22:20:53.530981+0000","last_active":"2023-07-13T22:20:55.533567+0000","last_peered":"2023-07-13T22:20:55.533567+0000","last_clean":"2023-07-13T22:20:55.533567+0000","last_became_active":"2023-07-13T22:20:53.530568+0000","last_became_peered":"2023-07-13T22:20:53.530568+0000","last_unstale":"2023-07-13T22:20:55.533567+0000","last_undegraded":"2023-07-13T22:20:55.533567+0000","last_fullsized":"2023-07-13T22:20:55.533567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610942+0000","last_change":"2023-07-13T22:20:54.540498+0000","last_active":"2023-07-13T22:20:55.610942+0000","last_peered":"2023-07-13T22:20:55.610942+0000","last_clean":"2023-07-13T22:20:55.610942+0000","last_became_active":"2023-07-13T22:20:54.540161+0000","last_became_peered":"2023-07-13T22:20:54.540161+0000","last_unstale":"2023-07-13T22:20:55.610942+0000","last_undegraded":"2023-07-13T22:20:55.610942+0000","last_fullsized":"2023-07-13T22:20:55.610942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:12.599393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610764+0000","last_change":"2023-07-13T22:20:53.533470+0000","last_active":"2023-07-13T22:20:55.610764+0000","last_peered":"2023-07-13T22:20:55.610764+0000","last_clean":"2023-07-13T22:20:55.610764+0000","last_became_active":"2023-07-13T22:20:53.532854+0000","last_became_peered":"2023-07-13T22:20:53.532854+0000","last_unstale":"2023-07-13T22:20:55.610764+0000","last_undegraded":"2023-07-13T22:20:55.610764+0000","last_fullsized":"2023-07-13T22:20:55.610764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:37.890575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.534049+0000","last_change":"2023-07-13T22:20:54.552213+0000","last_active":"2023-07-13T22:20:55.534049+0000","last_peered":"2023-07-13T22:20:55.534049+0000","last_clean":"2023-07-13T22:20:55.534049+0000","last_became_active":"2023-07-13T22:20:54.548415+0000","last_became_peered":"2023-07-13T22:20:54.548415+0000","last_unstale":"2023-07-13T22:20:55.534049+0000","last_undegraded":"2023-07-13T22:20:55.534049+0000","last_fullsized":"2023-07-13T22:20:55.534049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:50.171609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610542+0000","last_change":"2023-07-13T22:20:53.529929+0000","last_active":"2023-07-13T22:20:55.610542+0000","last_peered":"2023-07-13T22:20:55.610542+0000","last_clean":"2023-07-13T22:20:55.610542+0000","last_became_active":"2023-07-13T22:20:53.529645+0000","last_became_peered":"2023-07-13T22:20:53.529645+0000","last_unstale":"2023-07-13T22:20:55.610542+0000","last_undegraded":"2023-07-13T22:20:55.610542+0000","last_fullsized":"2023-07-13T22:20:55.610542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:53.812690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535287+0000","last_change":"2023-07-13T22:20:54.550835+0000","last_active":"2023-07-13T22:20:55.535287+0000","last_peered":"2023-07-13T22:20:55.535287+0000","last_clean":"2023-07-13T22:20:55.535287+0000","last_became_active":"2023-07-13T22:20:54.547187+0000","last_became_peered":"2023-07-13T22:20:54.547187+0000","last_unstale":"2023-07-13T22:20:55.535287+0000","last_undegraded":"2023-07-13T22:20:55.535287+0000","last_fullsized":"2023-07-13T22:20:55.535287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:04:58.712616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533482+0000","last_change":"2023-07-13T22:20:53.534667+0000","last_active":"2023-07-13T22:20:55.533482+0000","last_peered":"2023-07-13T22:20:55.533482+0000","last_clean":"2023-07-13T22:20:55.533482+0000","last_became_active":"2023-07-13T22:20:53.534433+0000","last_became_peered":"2023-07-13T22:20:53.534433+0000","last_unstale":"2023-07-13T22:20:55.533482+0000","last_undegraded":"2023-07-13T22:20:55.533482+0000","last_fullsized":"2023-07-13T22:20:55.533482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:28.070264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535373+0000","last_change":"2023-07-13T22:20:54.550679+0000","last_active":"2023-07-13T22:20:55.535373+0000","last_peered":"2023-07-13T22:20:55.535373+0000","last_clean":"2023-07-13T22:20:55.535373+0000","last_became_active":"2023-07-13T22:20:54.546483+0000","last_became_peered":"2023-07-13T22:20:54.546483+0000","last_unstale":"2023-07-13T22:20:55.535373+0000","last_undegraded":"2023-07-13T22:20:55.535373+0000","last_fullsized":"2023-07-13T22:20:55.535373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:16:54.293487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610029+0000","last_change":"2023-07-13T22:20:53.531815+0000","last_active":"2023-07-13T22:20:55.610029+0000","last_peered":"2023-07-13T22:20:55.610029+0000","last_clean":"2023-07-13T22:20:55.610029+0000","last_became_active":"2023-07-13T22:20:53.531551+0000","last_became_peered":"2023-07-13T22:20:53.531551+0000","last_unstale":"2023-07-13T22:20:55.610029+0000","last_undegraded":"2023-07-13T22:20:55.610029+0000","last_fullsized":"2023-07-13T22:20:55.610029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531764+0000","last_change":"2023-07-13T22:20:54.542085+0000","last_active":"2023-07-13T22:20:55.531764+0000","last_peered":"2023-07-13T22:20:55.531764+0000","last_clean":"2023-07-13T22:20:55.531764+0000","last_became_active":"2023-07-13T22:20:54.541749+0000","last_became_peered":"2023-07-13T22:20:54.541749+0000","last_unstale":"2023-07-13T22:20:55.531764+0000","last_undegraded":"2023-07-13T22:20:55.531764+0000","last_fullsized":"2023-07-13T22:20:55.531764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610668+0000","last_change":"2023-07-13T22:20:53.539618+0000","last_active":"2023-07-13T22:20:55.610668+0000","last_peered":"2023-07-13T22:20:55.610668+0000","last_clean":"2023-07-13T22:20:55.610668+0000","last_became_active":"2023-07-13T22:20:53.539014+0000","last_became_peered":"2023-07-13T22:20:53.539014+0000","last_unstale":"2023-07-13T22:20:55.610668+0000","last_undegraded":"2023-07-13T22:20:55.610668+0000","last_fullsized":"2023-07-13T22:20:55.610668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:49.163388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.535108+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:20:55.535108+0000","last_peered":"2023-07-13T22:20:55.535108+0000","last_clean":"2023-07-13T22:20:55.535108+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:20:55.535108+0000","last_undegraded":"2023-07-13T22:20:55.535108+0000","last_fullsized":"2023-07-13T22:20:55.535108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:11.435573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.609968+0000","last_change":"2023-07-13T22:20:54.538149+0000","last_active":"2023-07-13T22:20:55.609968+0000","last_peered":"2023-07-13T22:20:55.609968+0000","last_clean":"2023-07-13T22:20:55.609968+0000","last_became_active":"2023-07-13T22:20:54.537883+0000","last_became_peered":"2023-07-13T22:20:54.537883+0000","last_unstale":"2023-07-13T22:20:55.609968+0000","last_undegraded":"2023-07-13T22:20:55.609968+0000","last_fullsized":"2023-07-13T22:20:55.609968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610497+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:20:55.610497+0000","last_peered":"2023-07-13T22:20:55.610497+0000","last_clean":"2023-07-13T22:20:55.610497+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:20:55.610497+0000","last_undegraded":"2023-07-13T22:20:55.610497+0000","last_fullsized":"2023-07-13T22:20:55.610497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:49.435942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:56.195931+0000","last_change":"2023-07-13T22:20:53.541916+0000","last_active":"2023-07-13T22:20:56.195931+0000","last_peered":"2023-07-13T22:20:56.195931+0000","last_clean":"2023-07-13T22:20:56.195931+0000","last_became_active":"2023-07-13T22:20:53.541602+0000","last_became_peered":"2023-07-13T22:20:53.541602+0000","last_unstale":"2023-07-13T22:20:56.195931+0000","last_undegraded":"2023-07-13T22:20:56.195931+0000","last_fullsized":"2023-07-13T22:20:56.195931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:50.801197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533347+0000","last_change":"2023-07-13T22:20:54.545955+0000","last_active":"2023-07-13T22:20:55.533347+0000","last_peered":"2023-07-13T22:20:55.533347+0000","last_clean":"2023-07-13T22:20:55.533347+0000","last_became_active":"2023-07-13T22:20:54.545361+0000","last_became_peered":"2023-07-13T22:20:54.545361+0000","last_unstale":"2023-07-13T22:20:55.533347+0000","last_undegraded":"2023-07-13T22:20:55.533347+0000","last_fullsized":"2023-07-13T22:20:55.533347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531064+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:20:55.531064+0000","last_peered":"2023-07-13T22:20:55.531064+0000","last_clean":"2023-07-13T22:20:55.531064+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:20:55.531064+0000","last_undegraded":"2023-07-13T22:20:55.531064+0000","last_fullsized":"2023-07-13T22:20:55.531064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:15:56.981684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611242+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:20:55.611242+0000","last_peered":"2023-07-13T22:20:55.611242+0000","last_clean":"2023-07-13T22:20:55.611242+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:20:55.611242+0000","last_undegraded":"2023-07-13T22:20:55.611242+0000","last_fullsized":"2023-07-13T22:20:55.611242+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:44.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610851+0000","last_change":"2023-07-13T22:20:53.537543+0000","last_active":"2023-07-13T22:20:55.610851+0000","last_peered":"2023-07-13T22:20:55.610851+0000","last_clean":"2023-07-13T22:20:55.610851+0000","last_became_active":"2023-07-13T22:20:53.537125+0000","last_became_peered":"2023-07-13T22:20:53.537125+0000","last_unstale":"2023-07-13T22:20:55.610851+0000","last_undegraded":"2023-07-13T22:20:55.610851+0000","last_fullsized":"2023-07-13T22:20:55.610851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:48.574628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611216+0000","last_change":"2023-07-13T22:20:54.541651+0000","last_active":"2023-07-13T22:20:55.611216+0000","last_peered":"2023-07-13T22:20:55.611216+0000","last_clean":"2023-07-13T22:20:55.611216+0000","last_became_active":"2023-07-13T22:20:54.541059+0000","last_became_peered":"2023-07-13T22:20:54.541059+0000","last_unstale":"2023-07-13T22:20:55.611216+0000","last_undegraded":"2023-07-13T22:20:55.611216+0000","last_fullsized":"2023-07-13T22:20:55.611216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:54.820912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610481+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:20:55.610481+0000","last_peered":"2023-07-13T22:20:55.610481+0000","last_clean":"2023-07-13T22:20:55.610481+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:20:55.610481+0000","last_undegraded":"2023-07-13T22:20:55.610481+0000","last_fullsized":"2023-07-13T22:20:55.610481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:20.066975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.530855+0000","last_change":"2023-07-13T22:20:53.533784+0000","last_active":"2023-07-13T22:20:55.530855+0000","last_peered":"2023-07-13T22:20:55.530855+0000","last_clean":"2023-07-13T22:20:55.530855+0000","last_became_active":"2023-07-13T22:20:53.533270+0000","last_became_peered":"2023-07-13T22:20:53.533270+0000","last_unstale":"2023-07-13T22:20:55.530855+0000","last_undegraded":"2023-07-13T22:20:55.530855+0000","last_fullsized":"2023-07-13T22:20:55.530855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:29:28.555785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531674+0000","last_change":"2023-07-13T22:20:54.529627+0000","last_active":"2023-07-13T22:20:55.531674+0000","last_peered":"2023-07-13T22:20:55.531674+0000","last_clean":"2023-07-13T22:20:55.531674+0000","last_became_active":"2023-07-13T22:20:54.528020+0000","last_became_peered":"2023-07-13T22:20:54.528020+0000","last_unstale":"2023-07-13T22:20:55.531674+0000","last_undegraded":"2023-07-13T22:20:55.531674+0000","last_fullsized":"2023-07-13T22:20:55.531674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:54.820912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610893+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:20:55.610893+0000","last_peered":"2023-07-13T22:20:55.610893+0000","last_clean":"2023-07-13T22:20:55.610893+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:20:55.610893+0000","last_undegraded":"2023-07-13T22:20:55.610893+0000","last_fullsized":"2023-07-13T22:20:55.610893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:22.278097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610262+0000","last_change":"2023-07-13T22:20:53.535232+0000","last_active":"2023-07-13T22:20:55.610262+0000","last_peered":"2023-07-13T22:20:55.610262+0000","last_clean":"2023-07-13T22:20:55.610262+0000","last_became_active":"2023-07-13T22:20:53.533935+0000","last_became_peered":"2023-07-13T22:20:53.533935+0000","last_unstale":"2023-07-13T22:20:55.610262+0000","last_undegraded":"2023-07-13T22:20:55.610262+0000","last_fullsized":"2023-07-13T22:20:55.610262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610591+0000","last_change":"2023-07-13T22:20:54.549217+0000","last_active":"2023-07-13T22:20:55.610591+0000","last_peered":"2023-07-13T22:20:55.610591+0000","last_clean":"2023-07-13T22:20:55.610591+0000","last_became_active":"2023-07-13T22:20:54.548902+0000","last_became_peered":"2023-07-13T22:20:54.548902+0000","last_unstale":"2023-07-13T22:20:55.610591+0000","last_undegraded":"2023-07-13T22:20:55.610591+0000","last_fullsized":"2023-07-13T22:20:55.610591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:35.477313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532394+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:20:55.532394+0000","last_peered":"2023-07-13T22:20:55.532394+0000","last_clean":"2023-07-13T22:20:55.532394+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:20:55.532394+0000","last_undegraded":"2023-07-13T22:20:55.532394+0000","last_fullsized":"2023-07-13T22:20:55.532394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:05.799004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532935+0000","last_change":"2023-07-13T22:20:53.537640+0000","last_active":"2023-07-13T22:20:55.532935+0000","last_peered":"2023-07-13T22:20:55.532935+0000","last_clean":"2023-07-13T22:20:55.532935+0000","last_became_active":"2023-07-13T22:20:53.537351+0000","last_became_peered":"2023-07-13T22:20:53.537351+0000","last_unstale":"2023-07-13T22:20:55.532935+0000","last_undegraded":"2023-07-13T22:20:55.532935+0000","last_fullsized":"2023-07-13T22:20:55.532935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531434+0000","last_change":"2023-07-13T22:20:54.530546+0000","last_active":"2023-07-13T22:20:55.531434+0000","last_peered":"2023-07-13T22:20:55.531434+0000","last_clean":"2023-07-13T22:20:55.531434+0000","last_became_active":"2023-07-13T22:20:54.530234+0000","last_became_peered":"2023-07-13T22:20:54.530234+0000","last_unstale":"2023-07-13T22:20:55.531434+0000","last_undegraded":"2023-07-13T22:20:55.531434+0000","last_fullsized":"2023-07-13T22:20:55.531434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:29.646263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610282+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:20:55.610282+0000","last_peered":"2023-07-13T22:20:55.610282+0000","last_clean":"2023-07-13T22:20:55.610282+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:20:55.610282+0000","last_undegraded":"2023-07-13T22:20:55.610282+0000","last_fullsized":"2023-07-13T22:20:55.610282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:05.799004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531292+0000","last_change":"2023-07-13T22:20:53.533697+0000","last_active":"2023-07-13T22:20:55.531292+0000","last_peered":"2023-07-13T22:20:55.531292+0000","last_clean":"2023-07-13T22:20:55.531292+0000","last_became_active":"2023-07-13T22:20:53.533261+0000","last_became_peered":"2023-07-13T22:20:53.533261+0000","last_unstale":"2023-07-13T22:20:55.531292+0000","last_undegraded":"2023-07-13T22:20:55.531292+0000","last_fullsized":"2023-07-13T22:20:55.531292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:37:11.591172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611449+0000","last_change":"2023-07-13T22:20:54.537844+0000","last_active":"2023-07-13T22:20:55.611449+0000","last_peered":"2023-07-13T22:20:55.611449+0000","last_clean":"2023-07-13T22:20:55.611449+0000","last_became_active":"2023-07-13T22:20:54.537526+0000","last_became_peered":"2023-07-13T22:20:54.537526+0000","last_unstale":"2023-07-13T22:20:55.611449+0000","last_undegraded":"2023-07-13T22:20:55.611449+0000","last_fullsized":"2023-07-13T22:20:55.611449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:31.756152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610286+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:20:55.610286+0000","last_peered":"2023-07-13T22:20:55.610286+0000","last_clean":"2023-07-13T22:20:55.610286+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:20:55.610286+0000","last_undegraded":"2023-07-13T22:20:55.610286+0000","last_fullsized":"2023-07-13T22:20:55.610286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:44:17.601603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533295+0000","last_change":"2023-07-13T22:20:53.532273+0000","last_active":"2023-07-13T22:20:55.533295+0000","last_peered":"2023-07-13T22:20:55.533295+0000","last_clean":"2023-07-13T22:20:55.533295+0000","last_became_active":"2023-07-13T22:20:53.531699+0000","last_became_peered":"2023-07-13T22:20:53.531699+0000","last_unstale":"2023-07-13T22:20:55.533295+0000","last_undegraded":"2023-07-13T22:20:55.533295+0000","last_fullsized":"2023-07-13T22:20:55.533295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:05:44.321214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531599+0000","last_change":"2023-07-13T22:20:54.530026+0000","last_active":"2023-07-13T22:20:55.531599+0000","last_peered":"2023-07-13T22:20:55.531599+0000","last_clean":"2023-07-13T22:20:55.531599+0000","last_became_active":"2023-07-13T22:20:54.528217+0000","last_became_peered":"2023-07-13T22:20:54.528217+0000","last_unstale":"2023-07-13T22:20:55.531599+0000","last_undegraded":"2023-07-13T22:20:55.531599+0000","last_fullsized":"2023-07-13T22:20:55.531599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:26.368614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.611152+0000","last_change":"2023-07-13T22:20:53.541066+0000","last_active":"2023-07-13T22:20:55.611152+0000","last_peered":"2023-07-13T22:20:55.611152+0000","last_clean":"2023-07-13T22:20:55.611152+0000","last_became_active":"2023-07-13T22:20:53.540748+0000","last_became_peered":"2023-07-13T22:20:53.540748+0000","last_unstale":"2023-07-13T22:20:55.611152+0000","last_undegraded":"2023-07-13T22:20:55.611152+0000","last_fullsized":"2023-07-13T22:20:55.611152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:25.360393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.612132+0000","last_change":"2023-07-13T22:20:54.531788+0000","last_active":"2023-07-13T22:20:55.612132+0000","last_peered":"2023-07-13T22:20:55.612132+0000","last_clean":"2023-07-13T22:20:55.612132+0000","last_became_active":"2023-07-13T22:20:54.531396+0000","last_became_peered":"2023-07-13T22:20:54.531396+0000","last_unstale":"2023-07-13T22:20:55.612132+0000","last_undegraded":"2023-07-13T22:20:55.612132+0000","last_fullsized":"2023-07-13T22:20:55.612132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:45:27.037988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533306+0000","last_change":"2023-07-13T22:20:53.530819+0000","last_active":"2023-07-13T22:20:55.533306+0000","last_peered":"2023-07-13T22:20:55.533306+0000","last_clean":"2023-07-13T22:20:55.533306+0000","last_became_active":"2023-07-13T22:20:53.530223+0000","last_became_peered":"2023-07-13T22:20:53.530223+0000","last_unstale":"2023-07-13T22:20:55.533306+0000","last_undegraded":"2023-07-13T22:20:55.533306+0000","last_fullsized":"2023-07-13T22:20:55.533306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:15:58.999140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532780+0000","last_change":"2023-07-13T22:20:54.551556+0000","last_active":"2023-07-13T22:20:55.532780+0000","last_peered":"2023-07-13T22:20:55.532780+0000","last_clean":"2023-07-13T22:20:55.532780+0000","last_became_active":"2023-07-13T22:20:54.547048+0000","last_became_peered":"2023-07-13T22:20:54.547048+0000","last_unstale":"2023-07-13T22:20:55.532780+0000","last_undegraded":"2023-07-13T22:20:55.532780+0000","last_fullsized":"2023-07-13T22:20:55.532780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:01.665235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533441+0000","last_change":"2023-07-13T22:20:53.532040+0000","last_active":"2023-07-13T22:20:55.533441+0000","last_peered":"2023-07-13T22:20:55.533441+0000","last_clean":"2023-07-13T22:20:55.533441+0000","last_became_active":"2023-07-13T22:20:53.531414+0000","last_became_peered":"2023-07-13T22:20:53.531414+0000","last_unstale":"2023-07-13T22:20:55.533441+0000","last_undegraded":"2023-07-13T22:20:55.533441+0000","last_fullsized":"2023-07-13T22:20:55.533441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:25.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533812+0000","last_change":"2023-07-13T22:20:54.551351+0000","last_active":"2023-07-13T22:20:55.533812+0000","last_peered":"2023-07-13T22:20:55.533812+0000","last_clean":"2023-07-13T22:20:55.533812+0000","last_became_active":"2023-07-13T22:20:54.550932+0000","last_became_peered":"2023-07-13T22:20:54.550932+0000","last_unstale":"2023-07-13T22:20:55.533812+0000","last_undegraded":"2023-07-13T22:20:55.533812+0000","last_fullsized":"2023-07-13T22:20:55.533812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:36.702847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610605+0000","last_change":"2023-07-13T22:20:53.530713+0000","last_active":"2023-07-13T22:20:55.610605+0000","last_peered":"2023-07-13T22:20:55.610605+0000","last_clean":"2023-07-13T22:20:55.610605+0000","last_became_active":"2023-07-13T22:20:53.530419+0000","last_became_peered":"2023-07-13T22:20:53.530419+0000","last_unstale":"2023-07-13T22:20:55.610605+0000","last_undegraded":"2023-07-13T22:20:55.610605+0000","last_fullsized":"2023-07-13T22:20:55.610605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:51:35.694626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610476+0000","last_change":"2023-07-13T22:20:54.541258+0000","last_active":"2023-07-13T22:20:55.610476+0000","last_peered":"2023-07-13T22:20:55.610476+0000","last_clean":"2023-07-13T22:20:55.610476+0000","last_became_active":"2023-07-13T22:20:54.540610+0000","last_became_peered":"2023-07-13T22:20:54.540610+0000","last_unstale":"2023-07-13T22:20:55.610476+0000","last_undegraded":"2023-07-13T22:20:55.610476+0000","last_fullsized":"2023-07-13T22:20:55.610476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:29.078486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.612057+0000","last_change":"2023-07-13T22:20:53.538279+0000","last_active":"2023-07-13T22:20:55.612057+0000","last_peered":"2023-07-13T22:20:55.612057+0000","last_clean":"2023-07-13T22:20:55.612057+0000","last_became_active":"2023-07-13T22:20:53.537651+0000","last_became_peered":"2023-07-13T22:20:53.537651+0000","last_unstale":"2023-07-13T22:20:55.612057+0000","last_undegraded":"2023-07-13T22:20:55.612057+0000","last_fullsized":"2023-07-13T22:20:55.612057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:52.498261+0000","last_clean_scrub_stamp":"2023-07-13T22:20:52.498261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:42:07.452762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.532866+0000","last_change":"2023-07-13T22:20:54.541753+0000","last_active":"2023-07-13T22:20:55.532866+0000","last_peered":"2023-07-13T22:20:55.532866+0000","last_clean":"2023-07-13T22:20:55.532866+0000","last_became_active":"2023-07-13T22:20:54.540946+0000","last_became_peered":"2023-07-13T22:20:54.540946+0000","last_unstale":"2023-07-13T22:20:55.532866+0000","last_undegraded":"2023-07-13T22:20:55.532866+0000","last_fullsized":"2023-07-13T22:20:55.532866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:51.809418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610348+0000","last_change":"2023-07-13T22:20:54.542872+0000","last_active":"2023-07-13T22:20:55.610348+0000","last_peered":"2023-07-13T22:20:55.610348+0000","last_clean":"2023-07-13T22:20:55.610348+0000","last_became_active":"2023-07-13T22:20:54.542534+0000","last_became_peered":"2023-07-13T22:20:54.542534+0000","last_unstale":"2023-07-13T22:20:55.610348+0000","last_undegraded":"2023-07-13T22:20:55.610348+0000","last_fullsized":"2023-07-13T22:20:55.610348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:49.582850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610319+0000","last_change":"2023-07-13T22:20:54.541488+0000","last_active":"2023-07-13T22:20:55.610319+0000","last_peered":"2023-07-13T22:20:55.610319+0000","last_clean":"2023-07-13T22:20:55.610319+0000","last_became_active":"2023-07-13T22:20:54.541003+0000","last_became_peered":"2023-07-13T22:20:54.541003+0000","last_unstale":"2023-07-13T22:20:55.610319+0000","last_undegraded":"2023-07-13T22:20:55.610319+0000","last_fullsized":"2023-07-13T22:20:55.610319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:19.364913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.533452+0000","last_change":"2023-07-13T22:20:54.552460+0000","last_active":"2023-07-13T22:20:55.533452+0000","last_peered":"2023-07-13T22:20:55.533452+0000","last_clean":"2023-07-13T22:20:55.533452+0000","last_became_active":"2023-07-13T22:20:54.552208+0000","last_became_peered":"2023-07-13T22:20:54.552208+0000","last_unstale":"2023-07-13T22:20:55.533452+0000","last_undegraded":"2023-07-13T22:20:55.533452+0000","last_fullsized":"2023-07-13T22:20:55.533452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:43:48.562581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.610888+0000","last_change":"2023-07-13T22:20:54.535876+0000","last_active":"2023-07-13T22:20:55.610888+0000","last_peered":"2023-07-13T22:20:55.610888+0000","last_clean":"2023-07-13T22:20:55.610888+0000","last_became_active":"2023-07-13T22:20:54.535570+0000","last_became_peered":"2023-07-13T22:20:54.535570+0000","last_unstale":"2023-07-13T22:20:55.610888+0000","last_undegraded":"2023-07-13T22:20:55.610888+0000","last_fullsized":"2023-07-13T22:20:55.610888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:18.235962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-13T22:20:55.531178+0000","last_change":"2023-07-13T22:20:54.535911+0000","last_active":"2023-07-13T22:20:55.531178+0000","last_peered":"2023-07-13T22:20:55.531178+0000","last_clean":"2023-07-13T22:20:55.531178+0000","last_became_active":"2023-07-13T22:20:54.535593+0000","last_became_peered":"2023-07-13T22:20:54.535593+0000","last_unstale":"2023-07-13T22:20:55.531178+0000","last_undegraded":"2023-07-13T22:20:55.531178+0000","last_fullsized":"2023-07-13T22:20:55.531178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:53.506482+0000","last_clean_scrub_stamp":"2023-07-13T22:20:53.506482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:43.402535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":671608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":669828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":78474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":81080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":80112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:21:05.333 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-13T22:21:05.334 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-13T22:21:05.334 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-13T22:21:05.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:21:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.583+0000 7f9bd3bff700 1 Processor -- start 2023-07-13T22:21:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.583+0000 7f9bd3bff700 1 -- start start 2023-07-13T22:21:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd3bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9bcc0d0a50 0x7f9bcc0cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd3bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 0x7f9bcc0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd3bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bcc0d2410 con 0x7f9bcc0cd260 2023-07-13T22:21:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd3bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc15c900 con 0x7f9bcc0d0a50 2023-07-13T22:21:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd3bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0d2660 con 0x7f9bcc0d0e20 2023-07-13T22:21:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd199b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9bcc0d0a50 0x7f9bcc0cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd119a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9bcc0cd260 0x7f9bcc0d3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47738/0 (socket says 172.21.15.60:47738) 2023-07-13T22:21:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd119a700 1 -- 172.21.15.60:0/3023016353 learned_addr learned my addr 172.21.15.60:0/3023016353 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd219c700 1 --2- 172.21.15.60:0/3023016353 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 0x7f9bcc0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd199b700 1 -- 172.21.15.60:0/3023016353 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 msgr2=0x7f9bcc0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd199b700 1 --2- 172.21.15.60:0/3023016353 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 0x7f9bcc0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd199b700 1 -- 172.21.15.60:0/3023016353 >> v1:172.21.15.60:6789/0 conn(0x7f9bcc0cd260 legacy=0x7f9bcc0d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd199b700 1 -- 172.21.15.60:0/3023016353 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0d2940 con 0x7f9bcc0d0a50 2023-07-13T22:21:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.585+0000 7f9bd0999700 1 -- 172.21.15.60:0/3023016353 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406015715 0 0) 0x7f9bcc0d2410 con 0x7f9bcc0cd260 2023-07-13T22:21:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd199b700 1 --2- 172.21.15.60:0/3023016353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0d0a50 0x7f9bcc0cf8a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009840 tx=0x7f9bc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13de56ad84a02caf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd0999700 1 -- 172.21.15.60:0/3023016353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8011830 con 0x7f9bcc0d0a50 2023-07-13T22:21:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd0999700 1 -- 172.21.15.60:0/3023016353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc8011990 con 0x7f9bcc0d0a50 2023-07-13T22:21:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd0999700 1 -- 172.21.15.60:0/3023016353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8011d20 con 0x7f9bcc0d0a50 2023-07-13T22:21:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd3bff700 1 -- 172.21.15.60:0/3023016353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0d0a50 msgr2=0x7f9bcc0cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd3bff700 1 --2- 172.21.15.60:0/3023016353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0d0a50 0x7f9bcc0cf8a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009840 tx=0x7f9bc8009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd3bff700 1 -- 172.21.15.60:0/3023016353 shutdown_connections 2023-07-13T22:21:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd3bff700 1 --2- 172.21.15.60:0/3023016353 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 0x7f9bcc0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd3bff700 1 --2- 172.21.15.60:0/3023016353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0d0a50 0x7f9bcc0cf8a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.586+0000 7f9bd3bff700 1 -- 172.21.15.60:0/3023016353 >> 172.21.15.60:0/3023016353 conn(0x7f9bcc071600 msgr2=0x7f9bcc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.587+0000 7f9bd3bff700 1 -- 172.21.15.60:0/3023016353 shutdown_connections 2023-07-13T22:21:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.587+0000 7f9bd3bff700 1 -- 172.21.15.60:0/3023016353 wait complete. 2023-07-13T22:21:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.588+0000 7f9bd3bff700 1 Processor -- start 2023-07-13T22:21:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.588+0000 7f9bd3bff700 1 -- start start 2023-07-13T22:21:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.589+0000 7f9bd3bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0cd260 0x7f9bcc14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.589+0000 7f9bd3bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bcc0d0a50 0x7f9bcc15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.589+0000 7f9bd3bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 0x7f9bcc0c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.589+0000 7f9bd3bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc14d5e0 con 0x7f9bcc0cd260 2023-07-13T22:21:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.589+0000 7f9bd3bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc15a580 con 0x7f9bcc0d0a50 2023-07-13T22:21:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.589+0000 7f9bd3bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc15a860 con 0x7f9bcc0d0e20 2023-07-13T22:21:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0cd260 0x7f9bcc14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0cd260 0x7f9bcc14d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45690/0 (socket says 172.21.15.60:45690) 2023-07-13T22:21:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd199b700 1 -- 172.21.15.60:0/1832589702 learned_addr learned my addr 172.21.15.60:0/1832589702 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd119a700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bcc0d0a50 0x7f9bcc15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd219c700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 0x7f9bcc0c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd199b700 1 -- 172.21.15.60:0/1832589702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 msgr2=0x7f9bcc0c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd199b700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 0x7f9bcc0c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd199b700 1 -- 172.21.15.60:0/1832589702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bcc0d0a50 msgr2=0x7f9bcc15b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd199b700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bcc0d0a50 0x7f9bcc15b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd199b700 1 -- 172.21.15.60:0/1832589702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc80094a0 con 0x7f9bcc0cd260 2023-07-13T22:21:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bd199b700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0cd260 0x7f9bcc14d0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80090b0 tx=0x7f9bc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.590+0000 7f9bc27fc700 1 -- 172.21.15.60:0/1832589702 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80038a0 con 0x7f9bcc0cd260 2023-07-13T22:21:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.591+0000 7f9bc27fc700 1 -- 172.21.15.60:0/1832589702 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc801f070 con 0x7f9bcc0cd260 2023-07-13T22:21:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.591+0000 7f9bc27fc700 1 -- 172.21.15.60:0/1832589702 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8003c20 con 0x7f9bcc0cd260 2023-07-13T22:21:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.591+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc15ab40 con 0x7f9bcc0cd260 2023-07-13T22:21:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.591+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc15b100 con 0x7f9bcc0cd260 2023-07-13T22:21:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.592+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc04e280 con 0x7f9bcc0cd260 2023-07-13T22:21:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.593+0000 7f9bc27fc700 1 -- 172.21.15.60:0/1832589702 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9bc801a430 con 0x7f9bcc0cd260 2023-07-13T22:21:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.593+0000 7f9bc27fc700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9bb006ad10 0x7f9bb006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.594+0000 7f9bc27fc700 1 -- 172.21.15.60:0/1832589702 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9bc8021070 con 0x7f9bcc0cd260 2023-07-13T22:21:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.594+0000 7f9bd119a700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9bb006ad10 0x7f9bb006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.594+0000 7f9bd119a700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9bb006ad10 0x7f9bb006d1c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc15bf00 tx=0x7f9bbc007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.598+0000 7f9bc27fc700 1 -- 172.21.15.60:0/1832589702 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9bc8055860 con 0x7f9bcc0cd260 2023-07-13T22:21:05.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.846+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9bcc0cd040 con 0x7f9bcc0cd260 2023-07-13T22:21:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.847+0000 7f9bc27fc700 1 -- 172.21.15.60:0/1832589702 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9bc8058e80 con 0x7f9bcc0cd260 2023-07-13T22:21:05.831 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:05.831 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:21:05.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9bb006ad10 msgr2=0x7f9bb006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9bb006ad10 0x7f9bb006d1c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc15bf00 tx=0x7f9bbc007be0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0cd260 msgr2=0x7f9bcc14d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:05.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0cd260 0x7f9bcc14d0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80090b0 tx=0x7f9bc8004750 comp rx=0 tx=0).stop 2023-07-13T22:21:05.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 shutdown_connections 2023-07-13T22:21:05.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9bb006ad10 0x7f9bb006d1c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bcc0cd260 0x7f9bcc14d0a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9bcc0d0e20 0x7f9bcc0c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 --2- 172.21.15.60:0/1832589702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bcc0d0a50 0x7f9bcc15b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:05.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 >> 172.21.15.60:0/1832589702 conn(0x7f9bcc071600 msgr2=0x7f9bcc0732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:05.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 shutdown_connections 2023-07-13T22:21:05.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:05.849+0000 7f9bd3bff700 1 -- 172.21.15.60:0/1832589702 wait complete. 2023-07-13T22:21:05.852 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-13T22:21:05.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:06.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.070+0000 7f6b76dff700 1 Processor -- start 2023-07-13T22:21:06.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.070+0000 7f6b76dff700 1 -- start start 2023-07-13T22:21:06.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.071+0000 7f6b76dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6b700b5a00 0x7f6b700ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b76dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b700cd860 con 0x7f6b700b1f80 2023-07-13T22:21:06.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b76dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b700d2750 con 0x7f6b700cf560 2023-07-13T22:21:06.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b76dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b700b2350 con 0x7f6b700b5a00 2023-07-13T22:21:06.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b74b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6b700b1f80 0x7f6b700d1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47744/0 (socket says 172.21.15.60:47744) 2023-07-13T22:21:06.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b74b9b700 1 -- 172.21.15.60:0/3122850339 learned_addr learned my addr 172.21.15.60:0/3122850339 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:06.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b6ffff700 1 --2- 172.21.15.60:0/3122850339 >> v2:172.21.15.105:3300/0 conn(0x7f6b700b5a00 0x7f6b700ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b6f7fe700 1 -- 172.21.15.60:0/3122850339 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632915267 0 0) 0x7f6b700cd860 con 0x7f6b700b1f80 2023-07-13T22:21:06.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b6f7fe700 1 -- 172.21.15.60:0/3122850339 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b600035b0 con 0x7f6b700b1f80 2023-07-13T22:21:06.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b6f7fe700 1 -- 172.21.15.60:0/3122850339 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494547753 0 0) 0x7f6b700d2750 con 0x7f6b700cf560 2023-07-13T22:21:06.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b6f7fe700 1 -- 172.21.15.60:0/3122850339 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b700cd860 con 0x7f6b700cf560 2023-07-13T22:21:06.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b6ffff700 1 -- 172.21.15.60:0/3122850339 >> v1:172.21.15.105:6789/0 conn(0x7f6b700cf560 legacy=0x7f6b700cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b6ffff700 1 -- 172.21.15.60:0/3122850339 >> v1:172.21.15.60:6789/0 conn(0x7f6b700b1f80 legacy=0x7f6b700d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.072+0000 7f6b6ffff700 1 -- 172.21.15.60:0/3122850339 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b700cd2a0 con 0x7f6b700b5a00 2023-07-13T22:21:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.073+0000 7f6b6ffff700 1 --2- 172.21.15.60:0/3122850339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b5a00 0x7f6b700ced60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009840 tx=0x7f6b5c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db327f7033db4fc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.073+0000 7f6b6f7fe700 1 -- 172.21.15.60:0/3122850339 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c011830 con 0x7f6b700b5a00 2023-07-13T22:21:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.073+0000 7f6b6f7fe700 1 -- 172.21.15.60:0/3122850339 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c011990 con 0x7f6b700b5a00 2023-07-13T22:21:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.073+0000 7f6b6f7fe700 1 -- 172.21.15.60:0/3122850339 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c011d20 con 0x7f6b700b5a00 2023-07-13T22:21:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.074+0000 7f6b76dff700 1 -- 172.21.15.60:0/3122850339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b5a00 msgr2=0x7f6b700ced60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.074+0000 7f6b76dff700 1 --2- 172.21.15.60:0/3122850339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b5a00 0x7f6b700ced60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009840 tx=0x7f6b5c009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.074+0000 7f6b76dff700 1 -- 172.21.15.60:0/3122850339 shutdown_connections 2023-07-13T22:21:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.074+0000 7f6b76dff700 1 --2- 172.21.15.60:0/3122850339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b5a00 0x7f6b700ced60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.074+0000 7f6b76dff700 1 -- 172.21.15.60:0/3122850339 >> 172.21.15.60:0/3122850339 conn(0x7f6b70071600 msgr2=0x7f6b700719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.074+0000 7f6b76dff700 1 -- 172.21.15.60:0/3122850339 shutdown_connections 2023-07-13T22:21:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.074+0000 7f6b76dff700 1 -- 172.21.15.60:0/3122850339 wait complete. 2023-07-13T22:21:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.075+0000 7f6b76dff700 1 Processor -- start 2023-07-13T22:21:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.076+0000 7f6b76dff700 1 -- start start 2023-07-13T22:21:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.076+0000 7f6b76dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b1f80 0x7f6b7015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.076+0000 7f6b76dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b700b5a00 0x7f6b700cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.076+0000 7f6b76dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6b700cf560 0x7f6b7015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.076+0000 7f6b76dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7015a570 con 0x7f6b700b5a00 2023-07-13T22:21:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.076+0000 7f6b76dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7015c100 con 0x7f6b700b1f80 2023-07-13T22:21:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b76dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700cb6b0 con 0x7f6b700cf560 2023-07-13T22:21:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b6ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b700b5a00 0x7f6b700cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b6ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b700b5a00 0x7f6b700cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45704/0 (socket says 172.21.15.60:45704) 2023-07-13T22:21:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b6ffff700 1 -- 172.21.15.60:0/500114383 learned_addr learned my addr 172.21.15.60:0/500114383 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b74b9b700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b1f80 0x7f6b7015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b6ffff700 1 -- 172.21.15.60:0/500114383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6b700cf560 msgr2=0x7f6b7015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b6ffff700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6b700cf560 0x7f6b7015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b6ffff700 1 -- 172.21.15.60:0/500114383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b1f80 msgr2=0x7f6b7015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b6ffff700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b1f80 0x7f6b7015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b6ffff700 1 -- 172.21.15.60:0/500114383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0094a0 con 0x7f6b700b5a00 2023-07-13T22:21:06.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b74b9b700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b1f80 0x7f6b7015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:06.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.077+0000 7f6b6ffff700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b700b5a00 0x7f6b700cbe50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009fd0 tx=0x7f6b5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:06.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.078+0000 7f6b6d7fa700 1 -- 172.21.15.60:0/500114383 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c01aa70 con 0x7f6b700b5a00 2023-07-13T22:21:06.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.078+0000 7f6b6d7fa700 1 -- 172.21.15.60:0/500114383 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c01abd0 con 0x7f6b700b5a00 2023-07-13T22:21:06.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.078+0000 7f6b6d7fa700 1 -- 172.21.15.60:0/500114383 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0243f0 con 0x7f6b700b5a00 2023-07-13T22:21:06.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.078+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7015c370 con 0x7f6b700b5a00 2023-07-13T22:21:06.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.078+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7015d690 con 0x7f6b700b5a00 2023-07-13T22:21:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.079+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b700cbb70 con 0x7f6b700b5a00 2023-07-13T22:21:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.080+0000 7f6b6d7fa700 1 -- 172.21.15.60:0/500114383 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b5c024550 con 0x7f6b700b5a00 2023-07-13T22:21:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.081+0000 7f6b6d7fa700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6b5406ace0 0x7f6b5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.081+0000 7f6b6d7fa700 1 -- 172.21.15.60:0/500114383 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b5c08aaf0 con 0x7f6b700b5a00 2023-07-13T22:21:06.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.081+0000 7f6b74b9b700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6b5406ace0 0x7f6b5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.082+0000 7f6b74b9b700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6b5406ace0 0x7f6b5406d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000f60 tx=0x7f6b64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:06.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.085+0000 7f6b6d7fa700 1 -- 172.21.15.60:0/500114383 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6b700cbb70 con 0x7f6b700b5a00 2023-07-13T22:21:06.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.293+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b700cbb70 con 0x7f6b700b5a00 2023-07-13T22:21:06.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.294+0000 7f6b6d7fa700 1 -- 172.21.15.60:0/500114383 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f6b700cbb70 con 0x7f6b700b5a00 2023-07-13T22:21:06.279 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:06.279 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.296+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6b5406ace0 msgr2=0x7f6b5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.296+0000 7f6b76dff700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6b5406ace0 0x7f6b5406d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000f60 tx=0x7f6b64009040 comp rx=0 tx=0).stop 2023-07-13T22:21:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.296+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b700b5a00 msgr2=0x7f6b700cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.296+0000 7f6b76dff700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b700b5a00 0x7f6b700cbe50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009fd0 tx=0x7f6b5c004750 comp rx=0 tx=0).stop 2023-07-13T22:21:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.297+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 shutdown_connections 2023-07-13T22:21:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.297+0000 7f6b76dff700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6b5406ace0 0x7f6b5406d190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.297+0000 7f6b76dff700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6b700cf560 0x7f6b7015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.297+0000 7f6b76dff700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b700b1f80 0x7f6b7015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.297+0000 7f6b76dff700 1 --2- 172.21.15.60:0/500114383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b700b5a00 0x7f6b700cbe50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.297+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 >> 172.21.15.60:0/500114383 conn(0x7f6b70071600 msgr2=0x7f6b700c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.297+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 shutdown_connections 2023-07-13T22:21:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.297+0000 7f6b76dff700 1 -- 172.21.15.60:0/500114383 wait complete. 2023-07-13T22:21:06.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:06.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:06.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.520+0000 7f9262dff700 1 Processor -- start 2023-07-13T22:21:06.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.520+0000 7f9262dff700 1 -- start start 2023-07-13T22:21:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.521+0000 7f9262dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f925c0b8200 0x7f925c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.521+0000 7f9262dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f925c0b1ac0 0x7f925c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9262dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c0d1870 con 0x7f925c0cede0 2023-07-13T22:21:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9262dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925c0cf370 con 0x7f925c0b8200 2023-07-13T22:21:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9262dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925c0d12b0 con 0x7f925c0b1ac0 2023-07-13T22:21:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9260b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f925c0b8200 0x7f925c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9260b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f925c0b8200 0x7f925c0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45706/0 (socket says 172.21.15.60:45706) 2023-07-13T22:21:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9260b9b700 1 -- 172.21.15.60:0/2712022901 learned_addr learned my addr 172.21.15.60:0/2712022901 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f925bfff700 1 --2- 172.21.15.60:0/2712022901 >> v2:172.21.15.105:3300/0 conn(0x7f925c0b1ac0 0x7f925c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9260b9b700 1 -- 172.21.15.60:0/2712022901 >> v2:172.21.15.105:3300/0 conn(0x7f925c0b1ac0 msgr2=0x7f925c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:06.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9260b9b700 1 --2- 172.21.15.60:0/2712022901 >> v2:172.21.15.105:3300/0 conn(0x7f925c0b1ac0 0x7f925c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9260b9b700 1 -- 172.21.15.60:0/2712022901 >> v1:172.21.15.105:6789/0 conn(0x7f925c0cede0 legacy=0x7f925c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9260b9b700 1 -- 172.21.15.60:0/2712022901 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c0d1590 con 0x7f925c0b8200 2023-07-13T22:21:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.522+0000 7f9260b9b700 1 --2- 172.21.15.60:0/2712022901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b8200 0x7f925c0cf640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f925c0d07b0 tx=0x7f9250009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523fb3475f28c422 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f925b7fe700 1 -- 172.21.15.60:0/2712022901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250011830 con 0x7f925c0b8200 2023-07-13T22:21:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f925b7fe700 1 -- 172.21.15.60:0/2712022901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9250011990 con 0x7f925c0b8200 2023-07-13T22:21:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f925b7fe700 1 -- 172.21.15.60:0/2712022901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250011d20 con 0x7f925c0b8200 2023-07-13T22:21:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f9262dff700 1 -- 172.21.15.60:0/2712022901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b8200 msgr2=0x7f925c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f9262dff700 1 --2- 172.21.15.60:0/2712022901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b8200 0x7f925c0cf640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f925c0d07b0 tx=0x7f9250009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f9262dff700 1 -- 172.21.15.60:0/2712022901 shutdown_connections 2023-07-13T22:21:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f9262dff700 1 --2- 172.21.15.60:0/2712022901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b8200 0x7f925c0cf640 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f9262dff700 1 --2- 172.21.15.60:0/2712022901 >> v2:172.21.15.105:3300/0 conn(0x7f925c0b1ac0 0x7f925c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f9262dff700 1 -- 172.21.15.60:0/2712022901 >> 172.21.15.60:0/2712022901 conn(0x7f925c06b2f0 msgr2=0x7f925c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.523+0000 7f9262dff700 1 -- 172.21.15.60:0/2712022901 shutdown_connections 2023-07-13T22:21:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.524+0000 7f9262dff700 1 -- 172.21.15.60:0/2712022901 wait complete. 2023-07-13T22:21:06.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.525+0000 7f9262dff700 1 Processor -- start 2023-07-13T22:21:06.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.525+0000 7f9262dff700 1 -- start start 2023-07-13T22:21:06.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.526+0000 7f9262dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b1ac0 0x7f925c1557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.526+0000 7f9262dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f925c0b8200 0x7f925c1f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.526+0000 7f9262dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f925c0cede0 0x7f925c1f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.526+0000 7f9262dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c1587f0 con 0x7f925c0b1ac0 2023-07-13T22:21:06.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.526+0000 7f9262dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c155d10 con 0x7f925c0cede0 2023-07-13T22:21:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.526+0000 7f9262dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0cb6a0 con 0x7f925c0b8200 2023-07-13T22:21:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f9260b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b1ac0 0x7f925c1557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f9260b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b1ac0 0x7f925c1557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45716/0 (socket says 172.21.15.60:45716) 2023-07-13T22:21:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f9260b9b700 1 -- 172.21.15.60:0/1500602366 learned_addr learned my addr 172.21.15.60:0/1500602366 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f926139c700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f925c0cede0 0x7f925c1f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f925bfff700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f925c0b8200 0x7f925c1f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f9260b9b700 1 -- 172.21.15.60:0/1500602366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f925c0b8200 msgr2=0x7f925c1f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f9260b9b700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f925c0b8200 0x7f925c1f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f9260b9b700 1 -- 172.21.15.60:0/1500602366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f925c0cede0 msgr2=0x7f925c1f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f9260b9b700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f925c0cede0 0x7f925c1f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f9260b9b700 1 -- 172.21.15.60:0/1500602366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9250009550 con 0x7f925c0b1ac0 2023-07-13T22:21:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.527+0000 7f9260b9b700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b1ac0 0x7f925c1557d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9250000c00 tx=0x7f9250004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.528+0000 7f92597fa700 1 -- 172.21.15.60:0/1500602366 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250004380 con 0x7f925c0b1ac0 2023-07-13T22:21:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.528+0000 7f92597fa700 1 -- 172.21.15.60:0/1500602366 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925001f070 con 0x7f925c0b1ac0 2023-07-13T22:21:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.528+0000 7f92597fa700 1 -- 172.21.15.60:0/1500602366 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500036a0 con 0x7f925c0b1ac0 2023-07-13T22:21:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.528+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c0cb980 con 0x7f925c0b1ac0 2023-07-13T22:21:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.528+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c0cbf40 con 0x7f925c0b1ac0 2023-07-13T22:21:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.529+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c04e210 con 0x7f925c0b1ac0 2023-07-13T22:21:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.530+0000 7f92597fa700 1 -- 172.21.15.60:0/1500602366 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9250021030 con 0x7f925c0b1ac0 2023-07-13T22:21:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.531+0000 7f92597fa700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f924006ace0 0x7f924006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.531+0000 7f92597fa700 1 -- 172.21.15.60:0/1500602366 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f925008a8b0 con 0x7f925c0b1ac0 2023-07-13T22:21:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.531+0000 7f925bfff700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f924006ace0 0x7f924006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.532+0000 7f925bfff700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f924006ace0 0x7f924006d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b4650 tx=0x7f9248009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.535+0000 7f92597fa700 1 -- 172.21.15.60:0/1500602366 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f92500558d0 con 0x7f925c0b1ac0 2023-07-13T22:21:06.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.741+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f925c0b4f70 con 0x7f925c0b1ac0 2023-07-13T22:21:06.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.741+0000 7f92597fa700 1 -- 172.21.15.60:0/1500602366 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9250058ef0 con 0x7f925c0b1ac0 2023-07-13T22:21:06.725 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.743+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f924006ace0 msgr2=0x7f924006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.743+0000 7f9262dff700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f924006ace0 0x7f924006d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b4650 tx=0x7f9248009210 comp rx=0 tx=0).stop 2023-07-13T22:21:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.743+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b1ac0 msgr2=0x7f925c1557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.743+0000 7f9262dff700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b1ac0 0x7f925c1557d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9250000c00 tx=0x7f9250004750 comp rx=0 tx=0).stop 2023-07-13T22:21:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.744+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 shutdown_connections 2023-07-13T22:21:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.744+0000 7f9262dff700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f924006ace0 0x7f924006d190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.744+0000 7f9262dff700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925c0b1ac0 0x7f925c1557d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.744+0000 7f9262dff700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f925c0cede0 0x7f925c1f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.744+0000 7f9262dff700 1 --2- 172.21.15.60:0/1500602366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f925c0b8200 0x7f925c1f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.744+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 >> 172.21.15.60:0/1500602366 conn(0x7f925c06b2f0 msgr2=0x7f925c0cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.744+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 shutdown_connections 2023-07-13T22:21:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.744+0000 7f9262dff700 1 -- 172.21.15.60:0/1500602366 wait complete. 2023-07-13T22:21:06.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.967+0000 7fc2b51ff700 1 Processor -- start 2023-07-13T22:21:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.967+0000 7fc2b51ff700 1 -- start start 2023-07-13T22:21:06.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.968+0000 7fc2b51ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2b00cfd50 0x7fc2b00d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.968+0000 7fc2b51ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc2b00b1ac0 0x7fc2b00d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.968+0000 7fc2b51ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b00d1870 con 0x7fc2b00b8200 2023-07-13T22:21:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.968+0000 7fc2b51ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b00cf370 con 0x7fc2b00cfd50 2023-07-13T22:21:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.968+0000 7fc2b51ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b00d12b0 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.968+0000 7fc2aed9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc2b00b8200 0x7fc2b00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47756/0 (socket says 172.21.15.60:47756) 2023-07-13T22:21:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.968+0000 7fc2aed9d700 1 -- 172.21.15.60:0/1696144530 learned_addr learned my addr 172.21.15.60:0/1696144530 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.968+0000 7fc2a7fff700 1 --2- 172.21.15.60:0/1696144530 >> v2:172.21.15.60:3300/0 conn(0x7fc2b00cfd50 0x7fc2b00d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2af59e700 1 --2- 172.21.15.60:0/1696144530 >> v2:172.21.15.105:3300/0 conn(0x7fc2b00b1ac0 0x7fc2b00d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2ae59c700 1 -- 172.21.15.60:0/1696144530 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919783719 0 0) 0x7fc2b00d1870 con 0x7fc2b00b8200 2023-07-13T22:21:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2ae59c700 1 -- 172.21.15.60:0/1696144530 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc29c0035b0 con 0x7fc2b00b8200 2023-07-13T22:21:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2a7fff700 1 -- 172.21.15.60:0/1696144530 >> v2:172.21.15.105:3300/0 conn(0x7fc2b00b1ac0 msgr2=0x7fc2b00d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2a7fff700 1 --2- 172.21.15.60:0/1696144530 >> v2:172.21.15.105:3300/0 conn(0x7fc2b00b1ac0 0x7fc2b00d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2a7fff700 1 -- 172.21.15.60:0/1696144530 >> v1:172.21.15.60:6789/0 conn(0x7fc2b00b8200 legacy=0x7fc2b00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2a7fff700 1 -- 172.21.15.60:0/1696144530 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b00d1590 con 0x7fc2b00cfd50 2023-07-13T22:21:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2a7fff700 1 --2- 172.21.15.60:0/1696144530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00cfd50 0x7fc2b00d23c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00b8d70 tx=0x7fc298009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55197ceca47e814 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2ae59c700 1 -- 172.21.15.60:0/1696144530 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298011830 con 0x7fc2b00cfd50 2023-07-13T22:21:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.969+0000 7fc2ae59c700 1 -- 172.21.15.60:0/1696144530 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298011990 con 0x7fc2b00cfd50 2023-07-13T22:21:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.970+0000 7fc2ae59c700 1 -- 172.21.15.60:0/1696144530 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298011d20 con 0x7fc2b00cfd50 2023-07-13T22:21:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.970+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1696144530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00cfd50 msgr2=0x7fc2b00d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.970+0000 7fc2b51ff700 1 --2- 172.21.15.60:0/1696144530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00cfd50 0x7fc2b00d23c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00b8d70 tx=0x7fc298009b20 comp rx=0 tx=0).stop 2023-07-13T22:21:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.970+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1696144530 shutdown_connections 2023-07-13T22:21:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.970+0000 7fc2b51ff700 1 --2- 172.21.15.60:0/1696144530 >> v2:172.21.15.105:3300/0 conn(0x7fc2b00b1ac0 0x7fc2b00d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.970+0000 7fc2b51ff700 1 --2- 172.21.15.60:0/1696144530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00cfd50 0x7fc2b00d23c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.970+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1696144530 >> 172.21.15.60:0/1696144530 conn(0x7fc2b006b2f0 msgr2=0x7fc2b006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.970+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1696144530 shutdown_connections 2023-07-13T22:21:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.970+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1696144530 wait complete. 2023-07-13T22:21:06.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.972+0000 7fc2b51ff700 1 Processor -- start 2023-07-13T22:21:06.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.972+0000 7fc2b51ff700 1 -- start start 2023-07-13T22:21:06.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2b51ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00b1ac0 0x7fc2b0159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2b51ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b00b8200 0x7fc2b00ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2b51ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2b00cfd50 0x7fc2b00cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2b51ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00ca040 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2b51ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00b2690 con 0x7fc2b00b8200 2023-07-13T22:21:06.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2b51ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b015f840 con 0x7fc2b00cfd50 2023-07-13T22:21:06.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2aed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00b1ac0 0x7fc2b0159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2aed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00b1ac0 0x7fc2b0159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45736/0 (socket says 172.21.15.60:45736) 2023-07-13T22:21:06.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2aed9d700 1 -- 172.21.15.60:0/1077003193 learned_addr learned my addr 172.21.15.60:0/1077003193 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:06.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.973+0000 7fc2a7fff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b00b8200 0x7fc2b00ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2af59e700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2b00cfd50 0x7fc2b00cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2aed9d700 1 -- 172.21.15.60:0/1077003193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2b00cfd50 msgr2=0x7fc2b00cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2aed9d700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2b00cfd50 0x7fc2b00cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2aed9d700 1 -- 172.21.15.60:0/1077003193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b00b8200 msgr2=0x7fc2b00ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:06.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2aed9d700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b00b8200 0x7fc2b00ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:06.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2aed9d700 1 -- 172.21.15.60:0/1077003193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2980094a0 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2a7fff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b00b8200 0x7fc2b00ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:06.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2aed9d700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00b1ac0 0x7fc2b0159530 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c700 tx=0x7fc2a000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:06.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2a77fe700 1 -- 172.21.15.60:0/1077003193 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0005d70 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2a77fe700 1 -- 172.21.15.60:0/1077003193 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a00070d0 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2a77fe700 1 -- 172.21.15.60:0/1077003193 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0007460 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b015fb20 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.974+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b00b2bc0 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.975+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b004e210 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.976+0000 7fc2a77fe700 1 -- 172.21.15.60:0/1077003193 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2a00075c0 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.977+0000 7fc2a77fe700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc29006ad10 0x7fc29006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:06.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.977+0000 7fc2a77fe700 1 -- 172.21.15.60:0/1077003193 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2a0088af0 con 0x7fc2b00b1ac0 2023-07-13T22:21:06.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.978+0000 7fc2a7fff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc29006ad10 0x7fc29006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:06.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.978+0000 7fc2a7fff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc29006ad10 0x7fc29006d1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00d0140 tx=0x7fc298008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:06.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:06.982+0000 7fc2a77fe700 1 -- 172.21.15.60:0/1077003193 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc2a0053b10 con 0x7fc2b00b1ac0 2023-07-13T22:21:07.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.188+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2b00b4f70 con 0x7fc2b00b1ac0 2023-07-13T22:21:07.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.189+0000 7fc2a77fe700 1 -- 172.21.15.60:0/1077003193 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2a0057130 con 0x7fc2b00b1ac0 2023-07-13T22:21:07.173 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:07.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc29006ad10 msgr2=0x7fc29006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc29006ad10 0x7fc29006d1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00d0140 tx=0x7fc298008030 comp rx=0 tx=0).stop 2023-07-13T22:21:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00b1ac0 msgr2=0x7fc2b0159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00b1ac0 0x7fc2b0159530 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c700 tx=0x7fc2a000cac0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 shutdown_connections 2023-07-13T22:21:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc29006ad10 0x7fc29006d1c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2b00b1ac0 0x7fc2b0159530 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2b00cfd50 0x7fc2b00cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 --2- 172.21.15.60:0/1077003193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2b00b8200 0x7fc2b00ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 >> 172.21.15.60:0/1077003193 conn(0x7fc2b006b2f0 msgr2=0x7fc2b00d2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.191+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 shutdown_connections 2023-07-13T22:21:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.192+0000 7fc2b51ff700 1 -- 172.21.15.60:0/1077003193 wait complete. 2023-07-13T22:21:07.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.415+0000 7f8d253ff700 1 Processor -- start 2023-07-13T22:21:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.415+0000 7f8d253ff700 1 -- start start 2023-07-13T22:21:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.417+0000 7f8d253ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d200ca900 0x7f8d200703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.417+0000 7f8d253ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8d20070920 0x7f8d2006c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.417+0000 7f8d253ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d20070d80 con 0x7f8d20159260 2023-07-13T22:21:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.417+0000 7f8d253ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d200cad70 con 0x7f8d20070920 2023-07-13T22:21:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.417+0000 7f8d253ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200cf640 con 0x7f8d200ca900 2023-07-13T22:21:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.417+0000 7f8d1e7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d200ca900 0x7f8d200703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.417+0000 7f8d1f7fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8d20070920 0x7f8d2006c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.417+0000 7f8d1effd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8d20159260 0x7f8d200cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:33074/0 (socket says 172.21.15.60:33074) 2023-07-13T22:21:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.417+0000 7f8d1effd700 1 -- 172.21.15.60:0/1112570103 learned_addr learned my addr 172.21.15.60:0/1112570103 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.418+0000 7f8d1dffb700 1 -- 172.21.15.60:0/1112570103 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075184138 0 0) 0x7f8d20070d80 con 0x7f8d20159260 2023-07-13T22:21:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.418+0000 7f8d1f7fe700 1 -- 172.21.15.60:0/1112570103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d200ca900 msgr2=0x7f8d200703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.418+0000 7f8d1f7fe700 1 --2- 172.21.15.60:0/1112570103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d200ca900 0x7f8d200703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.418+0000 7f8d1f7fe700 1 -- 172.21.15.60:0/1112570103 >> v1:172.21.15.105:6789/0 conn(0x7f8d20159260 legacy=0x7f8d200cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.418+0000 7f8d1f7fe700 1 -- 172.21.15.60:0/1112570103 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200ccf80 con 0x7f8d20070920 2023-07-13T22:21:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.418+0000 7f8d1f7fe700 1 --2- 172.21.15.60:0/1112570103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20070920 0x7f8d2006c440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009d90 tx=0x7f8d1800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65193ac1225d6ebc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.418+0000 7f8d1dffb700 1 -- 172.21.15.60:0/1112570103 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1800e070 con 0x7f8d20070920 2023-07-13T22:21:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d1dffb700 1 -- 172.21.15.60:0/1112570103 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1800bb20 con 0x7f8d20070920 2023-07-13T22:21:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d1dffb700 1 -- 172.21.15.60:0/1112570103 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18012430 con 0x7f8d20070920 2023-07-13T22:21:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d253ff700 1 -- 172.21.15.60:0/1112570103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20070920 msgr2=0x7f8d2006c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d253ff700 1 --2- 172.21.15.60:0/1112570103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20070920 0x7f8d2006c440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009d90 tx=0x7f8d1800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:07.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d253ff700 1 -- 172.21.15.60:0/1112570103 shutdown_connections 2023-07-13T22:21:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d253ff700 1 --2- 172.21.15.60:0/1112570103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20070920 0x7f8d2006c440 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d253ff700 1 --2- 172.21.15.60:0/1112570103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d200ca900 0x7f8d200703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d253ff700 1 -- 172.21.15.60:0/1112570103 >> 172.21.15.60:0/1112570103 conn(0x7f8d200cbf50 msgr2=0x7f8d200ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d253ff700 1 -- 172.21.15.60:0/1112570103 shutdown_connections 2023-07-13T22:21:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.419+0000 7f8d253ff700 1 -- 172.21.15.60:0/1112570103 wait complete. 2023-07-13T22:21:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.421+0000 7f8d253ff700 1 Processor -- start 2023-07-13T22:21:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.421+0000 7f8d253ff700 1 -- start start 2023-07-13T22:21:07.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d253ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20070920 0x7f8d200c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d253ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d200ca900 0x7f8d20150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d253ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d20159260 0x7f8d200ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d253ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200af5c0 con 0x7f8d200ca900 2023-07-13T22:21:07.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d253ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200ae9b0 con 0x7f8d20070920 2023-07-13T22:21:07.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d253ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200aec90 con 0x7f8d20159260 2023-07-13T22:21:07.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d1e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d200ca900 0x7f8d20150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d1e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d200ca900 0x7f8d20150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45750/0 (socket says 172.21.15.60:45750) 2023-07-13T22:21:07.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d1e7fc700 1 -- 172.21.15.60:0/3453678500 learned_addr learned my addr 172.21.15.60:0/3453678500 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:07.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d1f7fe700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d20159260 0x7f8d200ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.422+0000 7f8d1effd700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20070920 0x7f8d200c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d1e7fc700 1 -- 172.21.15.60:0/3453678500 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d20159260 msgr2=0x7f8d200ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d1e7fc700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d20159260 0x7f8d200ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d1e7fc700 1 -- 172.21.15.60:0/3453678500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20070920 msgr2=0x7f8d200c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d1e7fc700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20070920 0x7f8d200c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d1e7fc700 1 -- 172.21.15.60:0/3453678500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d18009a70 con 0x7f8d200ca900 2023-07-13T22:21:07.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d1e7fc700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d200ca900 0x7f8d20150d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d14010df0 tx=0x7f8d1400e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:07.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d077fe700 1 -- 172.21.15.60:0/3453678500 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400b830 con 0x7f8d200ca900 2023-07-13T22:21:07.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d077fe700 1 -- 172.21.15.60:0/3453678500 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d14011450 con 0x7f8d200ca900 2023-07-13T22:21:07.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d077fe700 1 -- 172.21.15.60:0/3453678500 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14010ee0 con 0x7f8d200ca900 2023-07-13T22:21:07.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d200aef70 con 0x7f8d200ca900 2023-07-13T22:21:07.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.423+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d201512d0 con 0x7f8d200ca900 2023-07-13T22:21:07.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.424+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d200af730 con 0x7f8d200ca900 2023-07-13T22:21:07.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.425+0000 7f8d077fe700 1 -- 172.21.15.60:0/3453678500 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d140117c0 con 0x7f8d200ca900 2023-07-13T22:21:07.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.426+0000 7f8d077fe700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d0c06ace0 0x7f8d0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.426+0000 7f8d077fe700 1 -- 172.21.15.60:0/3453678500 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d1408bf50 con 0x7f8d200ca900 2023-07-13T22:21:07.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.426+0000 7f8d1effd700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d0c06ace0 0x7f8d0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.427+0000 7f8d1effd700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d0c06ace0 0x7f8d0c06d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d10007fd0 tx=0x7f8d10007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:07.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.431+0000 7f8d077fe700 1 -- 172.21.15.60:0/3453678500 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8d14056ff0 con 0x7f8d200ca900 2023-07-13T22:21:07.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.633+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d2004e210 con 0x7f8d200ca900 2023-07-13T22:21:07.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.634+0000 7f8d077fe700 1 -- 172.21.15.60:0/3453678500 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d1405a610 con 0x7f8d200ca900 2023-07-13T22:21:07.618 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:07.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d0c06ace0 msgr2=0x7f8d0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d0c06ace0 0x7f8d0c06d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d10007fd0 tx=0x7f8d10007de0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d200ca900 msgr2=0x7f8d20150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d200ca900 0x7f8d20150d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d14010df0 tx=0x7f8d1400e640 comp rx=0 tx=0).stop 2023-07-13T22:21:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 shutdown_connections 2023-07-13T22:21:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d0c06ace0 0x7f8d0c06d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d20159260 0x7f8d200ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d20070920 0x7f8d200c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 --2- 172.21.15.60:0/3453678500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d200ca900 0x7f8d20150d90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 >> 172.21.15.60:0/3453678500 conn(0x7f8d200cbf50 msgr2=0x7f8d2006d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:07.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 shutdown_connections 2023-07-13T22:21:07.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.636+0000 7f8d253ff700 1 -- 172.21.15.60:0/3453678500 wait complete. 2023-07-13T22:21:07.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:07.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.861+0000 7f6a499ff700 1 Processor -- start 2023-07-13T22:21:07.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.861+0000 7f6a499ff700 1 -- start start 2023-07-13T22:21:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.862+0000 7f6a499ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a440b8200 0x7f6a440cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.862+0000 7f6a499ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b1ac0 0x7f6a440cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.862+0000 7f6a499ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a440d1870 con 0x7f6a440cede0 2023-07-13T22:21:07.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.862+0000 7f6a499ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a440cf370 con 0x7f6a440b8200 2023-07-13T22:21:07.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.862+0000 7f6a499ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440d12b0 con 0x7f6a440b1ac0 2023-07-13T22:21:07.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.862+0000 7f6a42ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a440b8200 0x7f6a440cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a42ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a440b8200 0x7f6a440cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45758/0 (socket says 172.21.15.60:45758) 2023-07-13T22:21:07.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a42ffd700 1 -- 172.21.15.60:0/3597601297 learned_addr learned my addr 172.21.15.60:0/3597601297 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a427fc700 1 --2- 172.21.15.60:0/3597601297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b1ac0 0x7f6a440cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a42ffd700 1 -- 172.21.15.60:0/3597601297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b1ac0 msgr2=0x7f6a440cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a42ffd700 1 --2- 172.21.15.60:0/3597601297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b1ac0 0x7f6a440cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a42ffd700 1 -- 172.21.15.60:0/3597601297 >> v1:172.21.15.105:6789/0 conn(0x7f6a440cede0 legacy=0x7f6a4415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a42ffd700 1 -- 172.21.15.60:0/3597601297 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440d1590 con 0x7f6a440b8200 2023-07-13T22:21:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a42ffd700 1 --2- 172.21.15.60:0/3597601297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440b8200 0x7f6a440cf640 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6a340098f0 tx=0x7f6a34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33bb488bca6bd95 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a41ffb700 1 -- 172.21.15.60:0/3597601297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34011830 con 0x7f6a440b8200 2023-07-13T22:21:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a41ffb700 1 -- 172.21.15.60:0/3597601297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a34011990 con 0x7f6a440b8200 2023-07-13T22:21:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.863+0000 7f6a41ffb700 1 -- 172.21.15.60:0/3597601297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34011d20 con 0x7f6a440b8200 2023-07-13T22:21:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.864+0000 7f6a499ff700 1 -- 172.21.15.60:0/3597601297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440b8200 msgr2=0x7f6a440cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.864+0000 7f6a499ff700 1 --2- 172.21.15.60:0/3597601297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440b8200 0x7f6a440cf640 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6a340098f0 tx=0x7f6a34009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.864+0000 7f6a499ff700 1 -- 172.21.15.60:0/3597601297 shutdown_connections 2023-07-13T22:21:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.864+0000 7f6a499ff700 1 --2- 172.21.15.60:0/3597601297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440b8200 0x7f6a440cf640 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.864+0000 7f6a499ff700 1 --2- 172.21.15.60:0/3597601297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b1ac0 0x7f6a440cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.864+0000 7f6a499ff700 1 -- 172.21.15.60:0/3597601297 >> 172.21.15.60:0/3597601297 conn(0x7f6a4406b2f0 msgr2=0x7f6a4406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:07.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.864+0000 7f6a499ff700 1 -- 172.21.15.60:0/3597601297 shutdown_connections 2023-07-13T22:21:07.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.864+0000 7f6a499ff700 1 -- 172.21.15.60:0/3597601297 wait complete. 2023-07-13T22:21:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.866+0000 7f6a499ff700 1 Processor -- start 2023-07-13T22:21:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.866+0000 7f6a499ff700 1 -- start start 2023-07-13T22:21:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a499ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a440b1ac0 0x7f6a4414ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a499ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b8200 0x7f6a441e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a499ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440cede0 0x7f6a4414c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a499ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4414c420 con 0x7f6a440cede0 2023-07-13T22:21:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a499ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4414b810 con 0x7f6a440b1ac0 2023-07-13T22:21:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a499ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4414baf0 con 0x7f6a440b8200 2023-07-13T22:21:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440cede0 0x7f6a4414c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440cede0 0x7f6a4414c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45766/0 (socket says 172.21.15.60:45766) 2023-07-13T22:21:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a437fe700 1 -- 172.21.15.60:0/3945562871 learned_addr learned my addr 172.21.15.60:0/3945562871 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a42ffd700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a440b1ac0 0x7f6a4414ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.867+0000 7f6a427fc700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b8200 0x7f6a441e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a437fe700 1 -- 172.21.15.60:0/3945562871 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b8200 msgr2=0x7f6a441e28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a437fe700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b8200 0x7f6a441e28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a437fe700 1 -- 172.21.15.60:0/3945562871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a440b1ac0 msgr2=0x7f6a4414ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a437fe700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a440b1ac0 0x7f6a4414ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a437fe700 1 -- 172.21.15.60:0/3945562871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a34009550 con 0x7f6a440cede0 2023-07-13T22:21:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a427fc700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b8200 0x7f6a441e28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a437fe700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440cede0 0x7f6a4414c8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00cb50 tx=0x7f6a3c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a2bfff700 1 -- 172.21.15.60:0/3945562871 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c003c00 con 0x7f6a440cede0 2023-07-13T22:21:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a2bfff700 1 -- 172.21.15.60:0/3945562871 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c003d60 con 0x7f6a440cede0 2023-07-13T22:21:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a2bfff700 1 -- 172.21.15.60:0/3945562871 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c00aac0 con 0x7f6a440cede0 2023-07-13T22:21:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4414bdd0 con 0x7f6a440cede0 2023-07-13T22:21:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.868+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4415b790 con 0x7f6a440cede0 2023-07-13T22:21:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.870+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4414c5d0 con 0x7f6a440cede0 2023-07-13T22:21:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.870+0000 7f6a2bfff700 1 -- 172.21.15.60:0/3945562871 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a3c00ac20 con 0x7f6a440cede0 2023-07-13T22:21:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.871+0000 7f6a2bfff700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a3006ace0 0x7f6a3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.871+0000 7f6a2bfff700 1 -- 172.21.15.60:0/3945562871 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a3c0894e0 con 0x7f6a440cede0 2023-07-13T22:21:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.871+0000 7f6a42ffd700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a3006ace0 0x7f6a3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.872+0000 7f6a42ffd700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a3006ace0 0x7f6a3006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009c40 tx=0x7f6a34008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:07.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:07.875+0000 7f6a2bfff700 1 -- 172.21.15.60:0/3945562871 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6a4414c5d0 con 0x7f6a440cede0 2023-07-13T22:21:08.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.086+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a440ccd20 con 0x7f6a440cede0 2023-07-13T22:21:08.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.086+0000 7f6a2bfff700 1 -- 172.21.15.60:0/3945562871 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a3c0544f0 con 0x7f6a440cede0 2023-07-13T22:21:08.071 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:08.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a3006ace0 msgr2=0x7f6a3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a3006ace0 0x7f6a3006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009c40 tx=0x7f6a34008030 comp rx=0 tx=0).stop 2023-07-13T22:21:08.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440cede0 msgr2=0x7f6a4414c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440cede0 0x7f6a4414c8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00cb50 tx=0x7f6a3c00ce60 comp rx=0 tx=0).stop 2023-07-13T22:21:08.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 shutdown_connections 2023-07-13T22:21:08.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a3006ace0 0x7f6a3006d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a440cede0 0x7f6a4414c8b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a440b1ac0 0x7f6a4414ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 --2- 172.21.15.60:0/3945562871 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a440b8200 0x7f6a441e28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 >> 172.21.15.60:0/3945562871 conn(0x7f6a4406b2f0 msgr2=0x7f6a440b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:08.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 shutdown_connections 2023-07-13T22:21:08.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.089+0000 7f6a499ff700 1 -- 172.21.15.60:0/3945562871 wait complete. 2023-07-13T22:21:08.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:21:08.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.309+0000 7f8b777ff700 1 Processor -- start 2023-07-13T22:21:08.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.309+0000 7f8b777ff700 1 -- start start 2023-07-13T22:21:08.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.310+0000 7f8b777ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b700b8400 0x7f8b700cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.310+0000 7f8b777ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700d3750 0x7f8b700cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.310+0000 7f8b777ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700cff20 con 0x7f8b700cfad0 2023-07-13T22:21:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.310+0000 7f8b777ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b700d0ac0 con 0x7f8b700b8400 2023-07-13T22:21:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.310+0000 7f8b777ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700b87d0 con 0x7f8b700d3750 2023-07-13T22:21:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.310+0000 7f8b7559b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b700b8400 0x7f8b700cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.310+0000 7f8b7559b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b700b8400 0x7f8b700cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45774/0 (socket says 172.21.15.60:45774) 2023-07-13T22:21:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.310+0000 7f8b7559b700 1 -- 172.21.15.60:0/1218941129 learned_addr learned my addr 172.21.15.60:0/1218941129 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.310+0000 7f8b74d9a700 1 --2- 172.21.15.60:0/1218941129 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700d3750 0x7f8b700cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.311+0000 7f8b7559b700 1 -- 172.21.15.60:0/1218941129 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700d3750 msgr2=0x7f8b700cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.311+0000 7f8b7559b700 1 --2- 172.21.15.60:0/1218941129 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700d3750 0x7f8b700cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.311+0000 7f8b7559b700 1 -- 172.21.15.60:0/1218941129 >> v1:172.21.15.105:6789/0 conn(0x7f8b700cfad0 legacy=0x7f8b7015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.311+0000 7f8b7559b700 1 -- 172.21.15.60:0/1218941129 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700b4eb0 con 0x7f8b700b8400 2023-07-13T22:21:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.311+0000 7f8b7559b700 1 --2- 172.21.15.60:0/1218941129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700b8400 0x7f8b700cd240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0098f0 tx=0x7f8b6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8422ae33f12d4a1d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.311+0000 7f8b67fff700 1 -- 172.21.15.60:0/1218941129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c011830 con 0x7f8b700b8400 2023-07-13T22:21:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.311+0000 7f8b67fff700 1 -- 172.21.15.60:0/1218941129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c011990 con 0x7f8b700b8400 2023-07-13T22:21:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.311+0000 7f8b67fff700 1 -- 172.21.15.60:0/1218941129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c011d20 con 0x7f8b700b8400 2023-07-13T22:21:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.312+0000 7f8b777ff700 1 -- 172.21.15.60:0/1218941129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700b8400 msgr2=0x7f8b700cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.312+0000 7f8b777ff700 1 --2- 172.21.15.60:0/1218941129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700b8400 0x7f8b700cd240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0098f0 tx=0x7f8b6c009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.312+0000 7f8b777ff700 1 -- 172.21.15.60:0/1218941129 shutdown_connections 2023-07-13T22:21:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.312+0000 7f8b777ff700 1 --2- 172.21.15.60:0/1218941129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700b8400 0x7f8b700cd240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.312+0000 7f8b777ff700 1 --2- 172.21.15.60:0/1218941129 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700d3750 0x7f8b700cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.312+0000 7f8b777ff700 1 -- 172.21.15.60:0/1218941129 >> 172.21.15.60:0/1218941129 conn(0x7f8b7006b390 msgr2=0x7f8b7006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.312+0000 7f8b777ff700 1 -- 172.21.15.60:0/1218941129 shutdown_connections 2023-07-13T22:21:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.312+0000 7f8b777ff700 1 -- 172.21.15.60:0/1218941129 wait complete. 2023-07-13T22:21:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.314+0000 7f8b777ff700 1 Processor -- start 2023-07-13T22:21:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.314+0000 7f8b777ff700 1 -- start start 2023-07-13T22:21:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b777ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700b8400 0x7f8b70153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b777ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700cfad0 0x7f8b70153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b777ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b700d3750 0x7f8b700c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b777ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c66b0 con 0x7f8b700cfad0 2023-07-13T22:21:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b777ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c9ff0 con 0x7f8b700d3750 2023-07-13T22:21:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b777ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700cacc0 con 0x7f8b700b8400 2023-07-13T22:21:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b74d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700cfad0 0x7f8b70153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b74d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700cfad0 0x7f8b70153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45782/0 (socket says 172.21.15.60:45782) 2023-07-13T22:21:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b74d9a700 1 -- 172.21.15.60:0/2107090841 learned_addr learned my addr 172.21.15.60:0/2107090841 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b7559b700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700b8400 0x7f8b70153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b75d9c700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b700d3750 0x7f8b700c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b74d9a700 1 -- 172.21.15.60:0/2107090841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700b8400 msgr2=0x7f8b70153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b74d9a700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700b8400 0x7f8b70153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b74d9a700 1 -- 172.21.15.60:0/2107090841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b700d3750 msgr2=0x7f8b700c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b74d9a700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b700d3750 0x7f8b700c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b74d9a700 1 -- 172.21.15.60:0/2107090841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c009550 con 0x7f8b700cfad0 2023-07-13T22:21:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.315+0000 7f8b7559b700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700b8400 0x7f8b70153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.316+0000 7f8b74d9a700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700cfad0 0x7f8b70153eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e870 tx=0x7f8b6000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.316+0000 7f8b65ffb700 1 -- 172.21.15.60:0/2107090841 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60009590 con 0x7f8b700cfad0 2023-07-13T22:21:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.316+0000 7f8b65ffb700 1 -- 172.21.15.60:0/2107090841 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6000ee70 con 0x7f8b700cfad0 2023-07-13T22:21:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.316+0000 7f8b65ffb700 1 -- 172.21.15.60:0/2107090841 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000ca90 con 0x7f8b700cfad0 2023-07-13T22:21:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.316+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b700cafa0 con 0x7f8b700cfad0 2023-07-13T22:21:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.316+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b700c6900 con 0x7f8b700cfad0 2023-07-13T22:21:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.317+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b700c6b20 con 0x7f8b700cfad0 2023-07-13T22:21:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.318+0000 7f8b65ffb700 1 -- 172.21.15.60:0/2107090841 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b6000cbf0 con 0x7f8b700cfad0 2023-07-13T22:21:08.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.319+0000 7f8b65ffb700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b5406ad10 0x7f8b5406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.319+0000 7f8b65ffb700 1 -- 172.21.15.60:0/2107090841 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b600893f0 con 0x7f8b700cfad0 2023-07-13T22:21:08.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.319+0000 7f8b7559b700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b5406ad10 0x7f8b5406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.320+0000 7f8b7559b700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b5406ad10 0x7f8b5406d1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c014040 tx=0x7f8b6c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:08.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.323+0000 7f8b65ffb700 1 -- 172.21.15.60:0/2107090841 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8b60054490 con 0x7f8b700cfad0 2023-07-13T22:21:08.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.574+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b700c9910 con 0x7f8b700cfad0 2023-07-13T22:21:08.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.575+0000 7f8b65ffb700 1 -- 172.21.15.60:0/2107090841 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f8b6000cea0 con 0x7f8b700cfad0 2023-07-13T22:21:08.560 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:08.560 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2550952354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2550952354},{"type":"v1","addr":"172.21.15.60:6837","nonce":2550952354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4134045869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4134045869},{"type":"v1","addr":"172.21.15.60:6836","nonce":4134045869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2202330082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2202330082},{"type":"v1","addr":"172.21.15.105:6834","nonce":2202330082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:20:54.038443+0000","modified":"2023-07-13T22:20:57.536915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/453882022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":453882022},{"type":"v1","addr":"172.21.15.105:6835","nonce":453882022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:21:08.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b5406ad10 msgr2=0x7f8b5406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b5406ad10 0x7f8b5406d1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c014040 tx=0x7f8b6c008030 comp rx=0 tx=0).stop 2023-07-13T22:21:08.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700cfad0 msgr2=0x7f8b70153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700cfad0 0x7f8b70153eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e870 tx=0x7f8b6000ec30 comp rx=0 tx=0).stop 2023-07-13T22:21:08.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 shutdown_connections 2023-07-13T22:21:08.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b5406ad10 0x7f8b5406d1c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b700d3750 0x7f8b700c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b700b8400 0x7f8b70153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 --2- 172.21.15.60:0/2107090841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b700cfad0 0x7f8b70153eb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.577+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 >> 172.21.15.60:0/2107090841 conn(0x7f8b7006b390 msgr2=0x7f8b7015bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:08.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.578+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 shutdown_connections 2023-07-13T22:21:08.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.578+0000 7f8b777ff700 1 -- 172.21.15.60:0/2107090841 wait complete. 2023-07-13T22:21:08.563 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2023-07-13T22:21:08.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:08.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.799+0000 7ffa51fff700 1 Processor -- start 2023-07-13T22:21:08.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.799+0000 7ffa51fff700 1 -- start start 2023-07-13T22:21:08.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.800+0000 7ffa51fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0d3750 0x7ffa4c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.800+0000 7ffa51fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa4c0cfad0 0x7ffa4c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.800+0000 7ffa51fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c0cd950 con 0x7ffa4c0b8400 2023-07-13T22:21:08.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.800+0000 7ffa51fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c0cff20 con 0x7ffa4c0cfad0 2023-07-13T22:21:08.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.800+0000 7ffa51fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0d0ac0 con 0x7ffa4c0d3750 2023-07-13T22:21:08.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0d3750 0x7ffa4c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4bfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa4c0cfad0 0x7ffa4c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4b7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffa4c0b8400 0x7ffa4c0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:33088/0 (socket says 172.21.15.60:33088) 2023-07-13T22:21:08.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0d3750 0x7ffa4c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:39752/0 (socket says 172.21.15.60:39752) 2023-07-13T22:21:08.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4bfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa4c0cfad0 0x7ffa4c0d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:43148/0 (socket says 172.21.15.60:43148) 2023-07-13T22:21:08.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4b7fe700 1 -- 172.21.15.60:0/3384359740 learned_addr learned my addr 172.21.15.60:0/3384359740 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:08.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4a7fc700 1 -- 172.21.15.60:0/3384359740 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011942068 0 0) 0x7ffa3c001110 con 0x7ffa4c0b8400 2023-07-13T22:21:08.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4a7fc700 1 -- 172.21.15.60:0/3384359740 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa340035b0 con 0x7ffa4c0b8400 2023-07-13T22:21:08.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4affd700 1 -- 172.21.15.60:0/3384359740 >> v2:172.21.15.105:3300/0 conn(0x7ffa4c0cfad0 msgr2=0x7ffa4c0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:08.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4affd700 1 --2- 172.21.15.60:0/3384359740 >> v2:172.21.15.105:3300/0 conn(0x7ffa4c0cfad0 0x7ffa4c0d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4affd700 1 -- 172.21.15.60:0/3384359740 >> v1:172.21.15.105:6789/0 conn(0x7ffa4c0b8400 legacy=0x7ffa4c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.801+0000 7ffa4affd700 1 -- 172.21.15.60:0/3384359740 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c0b87d0 con 0x7ffa4c0d3750 2023-07-13T22:21:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.802+0000 7ffa4bfff700 1 --2- 172.21.15.60:0/3384359740 >> v2:172.21.15.105:3300/0 conn(0x7ffa4c0cfad0 0x7ffa4c0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.802+0000 7ffa4affd700 1 --2- 172.21.15.60:0/3384359740 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0d3750 0x7ffa4c0cf590 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009840 tx=0x7ffa40009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b077163ff35330 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:08.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.802+0000 7ffa4a7fc700 1 -- 172.21.15.60:0/3384359740 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40011830 con 0x7ffa4c0d3750 2023-07-13T22:21:08.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.802+0000 7ffa4a7fc700 1 -- 172.21.15.60:0/3384359740 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa40011990 con 0x7ffa4c0d3750 2023-07-13T22:21:08.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.802+0000 7ffa4a7fc700 1 -- 172.21.15.60:0/3384359740 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40011d20 con 0x7ffa4c0d3750 2023-07-13T22:21:08.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.803+0000 7ffa51fff700 1 -- 172.21.15.60:0/3384359740 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0d3750 msgr2=0x7ffa4c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.803+0000 7ffa51fff700 1 --2- 172.21.15.60:0/3384359740 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0d3750 0x7ffa4c0cf590 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009840 tx=0x7ffa40009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:08.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.803+0000 7ffa51fff700 1 -- 172.21.15.60:0/3384359740 shutdown_connections 2023-07-13T22:21:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.803+0000 7ffa51fff700 1 --2- 172.21.15.60:0/3384359740 >> v2:172.21.15.105:3300/0 conn(0x7ffa4c0cfad0 0x7ffa4c0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.803+0000 7ffa51fff700 1 --2- 172.21.15.60:0/3384359740 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0d3750 0x7ffa4c0cf590 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.803+0000 7ffa51fff700 1 -- 172.21.15.60:0/3384359740 >> 172.21.15.60:0/3384359740 conn(0x7ffa4c06b390 msgr2=0x7ffa4c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.803+0000 7ffa51fff700 1 -- 172.21.15.60:0/3384359740 shutdown_connections 2023-07-13T22:21:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.803+0000 7ffa51fff700 1 -- 172.21.15.60:0/3384359740 wait complete. 2023-07-13T22:21:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.804+0000 7ffa51fff700 1 Processor -- start 2023-07-13T22:21:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.805+0000 7ffa51fff700 1 -- start start 2023-07-13T22:21:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.805+0000 7ffa51fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa4c0b8400 0x7ffa4c149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.805+0000 7ffa51fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0cfad0 0x7ffa4c0b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa51fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c0d3750 0x7ffa4c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa51fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c15e130 con 0x7ffa4c0d3750 2023-07-13T22:21:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa51fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c154c80 con 0x7ffa4c0b8400 2023-07-13T22:21:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa51fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c155520 con 0x7ffa4c0cfad0 2023-07-13T22:21:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c0d3750 0x7ffa4c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c0d3750 0x7ffa4c0b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45788/0 (socket says 172.21.15.60:45788) 2023-07-13T22:21:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4bfff700 1 -- 172.21.15.60:0/548442073 learned_addr learned my addr 172.21.15.60:0/548442073 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4affd700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0cfad0 0x7ffa4c0b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4b7fe700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa4c0b8400 0x7ffa4c149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4bfff700 1 -- 172.21.15.60:0/548442073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0cfad0 msgr2=0x7ffa4c0b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4bfff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0cfad0 0x7ffa4c0b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4bfff700 1 -- 172.21.15.60:0/548442073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa4c0b8400 msgr2=0x7ffa4c149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4bfff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa4c0b8400 0x7ffa4c149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4bfff700 1 -- 172.21.15.60:0/548442073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa44009a70 con 0x7ffa4c0d3750 2023-07-13T22:21:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.806+0000 7ffa4affd700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0cfad0 0x7ffa4c0b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.807+0000 7ffa4bfff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c0d3750 0x7ffa4c0b3820 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400efc0 tx=0x7ffa4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.807+0000 7ffa33fff700 1 -- 172.21.15.60:0/548442073 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44009cd0 con 0x7ffa4c0d3750 2023-07-13T22:21:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.807+0000 7ffa33fff700 1 -- 172.21.15.60:0/548442073 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4400f040 con 0x7ffa4c0d3750 2023-07-13T22:21:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.807+0000 7ffa33fff700 1 -- 172.21.15.60:0/548442073 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4400cd60 con 0x7ffa4c0d3750 2023-07-13T22:21:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.807+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa400094a0 con 0x7ffa4c0d3750 2023-07-13T22:21:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.807+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c154ed0 con 0x7ffa4c0d3750 2023-07-13T22:21:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.808+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c155660 con 0x7ffa4c0d3750 2023-07-13T22:21:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.809+0000 7ffa33fff700 1 -- 172.21.15.60:0/548442073 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa44014480 con 0x7ffa4c0d3750 2023-07-13T22:21:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.810+0000 7ffa33fff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ace0 0x7ffa3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.810+0000 7ffa33fff700 1 -- 172.21.15.60:0/548442073 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa44089bb0 con 0x7ffa4c0d3750 2023-07-13T22:21:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.810+0000 7ffa4b7fe700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ace0 0x7ffa3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.811+0000 7ffa4b7fe700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ace0 0x7ffa3806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c001270 tx=0x7ffa3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:08.814+0000 7ffa33fff700 1 -- 172.21.15.60:0/548442073 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffa44054bd0 con 0x7ffa4c0d3750 2023-07-13T22:21:09.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.022+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa4c15b260 con 0x7ffa4c0d3750 2023-07-13T22:21:09.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.023+0000 7ffa33fff700 1 -- 172.21.15.60:0/548442073 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7ffa440581f0 con 0x7ffa4c0d3750 2023-07-13T22:21:09.007 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:09.008 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:09.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ace0 msgr2=0x7ffa3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ace0 0x7ffa3806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c001270 tx=0x7ffa3c009040 comp rx=0 tx=0).stop 2023-07-13T22:21:09.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c0d3750 msgr2=0x7ffa4c0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c0d3750 0x7ffa4c0b3820 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400efc0 tx=0x7ffa4400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 shutdown_connections 2023-07-13T22:21:09.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa3806ace0 0x7ffa3806d190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa4c0d3750 0x7ffa4c0b3820 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa4c0b8400 0x7ffa4c149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 --2- 172.21.15.60:0/548442073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa4c0cfad0 0x7ffa4c0b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.025+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 >> 172.21.15.60:0/548442073 conn(0x7ffa4c06b390 msgr2=0x7ffa4c0d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:09.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.026+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 shutdown_connections 2023-07-13T22:21:09.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.026+0000 7ffa51fff700 1 -- 172.21.15.60:0/548442073 wait complete. 2023-07-13T22:21:09.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:09.029 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.250+0000 7ff83e3ff700 1 Processor -- start 2023-07-13T22:21:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.250+0000 7ff83e3ff700 1 -- start start 2023-07-13T22:21:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.251+0000 7ff83e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff8380cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.251+0000 7ff83e3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff8380cede0 0x7ff8380cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.251+0000 7ff83e3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8380d1870 con 0x7ff8380b1ac0 2023-07-13T22:21:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.251+0000 7ff83e3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8380cf370 con 0x7ff8380cede0 2023-07-13T22:21:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.251+0000 7ff83e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380d12b0 con 0x7ff8380b8200 2023-07-13T22:21:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff8377fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff8380b1ac0 0x7ff83815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47766/0 (socket says 172.21.15.60:47766) 2023-07-13T22:21:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff8377fe700 1 -- 172.21.15.60:0/1057581929 learned_addr learned my addr 172.21.15.60:0/1057581929 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff83c99c700 1 --2- 172.21.15.60:0/1057581929 >> v2:172.21.15.105:3300/0 conn(0x7ff8380cede0 0x7ff8380cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff837fff700 1 --2- 172.21.15.60:0/1057581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff8380cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff836ffd700 1 -- 172.21.15.60:0/1057581929 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23119739 0 0) 0x7ff8380d1870 con 0x7ff8380b1ac0 2023-07-13T22:21:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff836ffd700 1 -- 172.21.15.60:0/1057581929 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8200035b0 con 0x7ff8380b1ac0 2023-07-13T22:21:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff837fff700 1 -- 172.21.15.60:0/1057581929 >> v2:172.21.15.105:3300/0 conn(0x7ff8380cede0 msgr2=0x7ff8380cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff837fff700 1 --2- 172.21.15.60:0/1057581929 >> v2:172.21.15.105:3300/0 conn(0x7ff8380cede0 0x7ff8380cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff837fff700 1 -- 172.21.15.60:0/1057581929 >> v1:172.21.15.60:6789/0 conn(0x7ff8380b1ac0 legacy=0x7ff83815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff837fff700 1 -- 172.21.15.60:0/1057581929 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380d1590 con 0x7ff8380b8200 2023-07-13T22:21:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.252+0000 7ff83c99c700 1 --2- 172.21.15.60:0/1057581929 >> v2:172.21.15.105:3300/0 conn(0x7ff8380cede0 0x7ff8380cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff837fff700 1 --2- 172.21.15.60:0/1057581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff8380cf640 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff828009840 tx=0x7ff828009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50363a036f5de863 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff836ffd700 1 -- 172.21.15.60:0/1057581929 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828011830 con 0x7ff8380b8200 2023-07-13T22:21:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff836ffd700 1 -- 172.21.15.60:0/1057581929 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828011990 con 0x7ff8380b8200 2023-07-13T22:21:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff836ffd700 1 -- 172.21.15.60:0/1057581929 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828011d20 con 0x7ff8380b8200 2023-07-13T22:21:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1057581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 msgr2=0x7ff8380cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff83e3ff700 1 --2- 172.21.15.60:0/1057581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff8380cf640 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff828009840 tx=0x7ff828009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1057581929 shutdown_connections 2023-07-13T22:21:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff83e3ff700 1 --2- 172.21.15.60:0/1057581929 >> v2:172.21.15.105:3300/0 conn(0x7ff8380cede0 0x7ff8380cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff83e3ff700 1 --2- 172.21.15.60:0/1057581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff8380cf640 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.253+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1057581929 >> 172.21.15.60:0/1057581929 conn(0x7ff83806b2f0 msgr2=0x7ff83806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.254+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1057581929 shutdown_connections 2023-07-13T22:21:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.254+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1057581929 wait complete. 2023-07-13T22:21:09.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.255+0000 7ff83e3ff700 1 Processor -- start 2023-07-13T22:21:09.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.256+0000 7ff83e3ff700 1 -- start start 2023-07-13T22:21:09.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.256+0000 7ff83e3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8380b1ac0 0x7ff8381e6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff83e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff83814d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff83e3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8380cede0 0x7ff8381f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff83e3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380cb540 con 0x7ff8380b1ac0 2023-07-13T22:21:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff83e3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380cb820 con 0x7ff8380cede0 2023-07-13T22:21:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff83e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380cbb00 con 0x7ff8380b8200 2023-07-13T22:21:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff837fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8380b1ac0 0x7ff8381e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff837fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8380b1ac0 0x7ff8381e6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45796/0 (socket says 172.21.15.60:45796) 2023-07-13T22:21:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff837fff700 1 -- 172.21.15.60:0/1403673775 learned_addr learned my addr 172.21.15.60:0/1403673775 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff8377fe700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff83814d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff83c99c700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8380cede0 0x7ff8381f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff837fff700 1 -- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 msgr2=0x7ff83814d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff837fff700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff83814d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff837fff700 1 -- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8380cede0 msgr2=0x7ff8381f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff837fff700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8380cede0 0x7ff8381f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff837fff700 1 -- 172.21.15.60:0/1403673775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff830009a70 con 0x7ff8380b1ac0 2023-07-13T22:21:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff8377fe700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff83814d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff83c99c700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8380cede0 0x7ff8381f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.257+0000 7ff837fff700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8380b1ac0 0x7ff8381e6940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff8280090b0 tx=0x7ff828004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.258+0000 7ff834ff9700 1 -- 172.21.15.60:0/1403673775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8280036a0 con 0x7ff8380b1ac0 2023-07-13T22:21:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.258+0000 7ff834ff9700 1 -- 172.21.15.60:0/1403673775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82801f070 con 0x7ff8380b1ac0 2023-07-13T22:21:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.258+0000 7ff834ff9700 1 -- 172.21.15.60:0/1403673775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828003a40 con 0x7ff8380b1ac0 2023-07-13T22:21:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.258+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8280094a0 con 0x7ff8380b1ac0 2023-07-13T22:21:09.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.258+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83814d5b0 con 0x7ff8380b1ac0 2023-07-13T22:21:09.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.259+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83804e210 con 0x7ff8380b1ac0 2023-07-13T22:21:09.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.260+0000 7ff834ff9700 1 -- 172.21.15.60:0/1403673775 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff828003ba0 con 0x7ff8380b1ac0 2023-07-13T22:21:09.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.261+0000 7ff834ff9700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff82406ace0 0x7ff82406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.261+0000 7ff834ff9700 1 -- 172.21.15.60:0/1403673775 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff828021070 con 0x7ff8380b1ac0 2023-07-13T22:21:09.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.261+0000 7ff8377fe700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff82406ace0 0x7ff82406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.261+0000 7ff8377fe700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff82406ace0 0x7ff82406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0011e0 tx=0x7ff82c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:09.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.265+0000 7ff834ff9700 1 -- 172.21.15.60:0/1403673775 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff828055dd0 con 0x7ff8380b1ac0 2023-07-13T22:21:09.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.469+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff8380cb680 con 0x7ff8380b1ac0 2023-07-13T22:21:09.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.469+0000 7ff834ff9700 1 -- 172.21.15.60:0/1403673775 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff8380cb680 con 0x7ff8380b1ac0 2023-07-13T22:21:09.454 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.471+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff82406ace0 msgr2=0x7ff82406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.471+0000 7ff83e3ff700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff82406ace0 0x7ff82406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0011e0 tx=0x7ff82c009040 comp rx=0 tx=0).stop 2023-07-13T22:21:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.471+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8380b1ac0 msgr2=0x7ff8381e6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.471+0000 7ff83e3ff700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8380b1ac0 0x7ff8381e6940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff8280090b0 tx=0x7ff828004750 comp rx=0 tx=0).stop 2023-07-13T22:21:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.472+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 shutdown_connections 2023-07-13T22:21:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.472+0000 7ff83e3ff700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff82406ace0 0x7ff82406d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.472+0000 7ff83e3ff700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8380b1ac0 0x7ff8381e6940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.472+0000 7ff83e3ff700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8380cede0 0x7ff8381f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.472+0000 7ff83e3ff700 1 --2- 172.21.15.60:0/1403673775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8380b8200 0x7ff83814d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.472+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 >> 172.21.15.60:0/1403673775 conn(0x7ff83806b2f0 msgr2=0x7ff8380b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.472+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 shutdown_connections 2023-07-13T22:21:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.472+0000 7ff83e3ff700 1 -- 172.21.15.60:0/1403673775 wait complete. 2023-07-13T22:21:09.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.698+0000 7fab6e3ff700 1 Processor -- start 2023-07-13T22:21:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.699+0000 7fab6e3ff700 1 -- start start 2023-07-13T22:21:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.700+0000 7fab6e3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fab680b8200 0x7fab680cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.700+0000 7fab6e3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab680d1870 con 0x7fab680b1ac0 2023-07-13T22:21:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.700+0000 7fab6e3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab68160d80 con 0x7fab680cede0 2023-07-13T22:21:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.700+0000 7fab6e3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab680cf370 con 0x7fab680b8200 2023-07-13T22:21:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.700+0000 7fab677fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fab680b1ac0 0x7fab6815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47778/0 (socket says 172.21.15.60:47778) 2023-07-13T22:21:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.700+0000 7fab677fe700 1 -- 172.21.15.60:0/3349239872 learned_addr learned my addr 172.21.15.60:0/3349239872 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.700+0000 7fab67fff700 1 --2- 172.21.15.60:0/3349239872 >> v2:172.21.15.105:3300/0 conn(0x7fab680b8200 0x7fab680cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.700+0000 7fab66ffd700 1 -- 172.21.15.60:0/3349239872 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533876895 0 0) 0x7fab680d1870 con 0x7fab680b1ac0 2023-07-13T22:21:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab66ffd700 1 -- 172.21.15.60:0/3349239872 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab500035b0 con 0x7fab680b1ac0 2023-07-13T22:21:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab66ffd700 1 -- 172.21.15.60:0/3349239872 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 603429060 0 0) 0x7fab68160d80 con 0x7fab680cede0 2023-07-13T22:21:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab66ffd700 1 -- 172.21.15.60:0/3349239872 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab680d1870 con 0x7fab680cede0 2023-07-13T22:21:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab67fff700 1 -- 172.21.15.60:0/3349239872 >> v1:172.21.15.105:6789/0 conn(0x7fab680cede0 legacy=0x7fab68160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab67fff700 1 -- 172.21.15.60:0/3349239872 >> v1:172.21.15.60:6789/0 conn(0x7fab680b1ac0 legacy=0x7fab6815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab67fff700 1 -- 172.21.15.60:0/3349239872 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680d12b0 con 0x7fab680b8200 2023-07-13T22:21:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab67fff700 1 --2- 172.21.15.60:0/3349239872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab680b8200 0x7fab680cf640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab58009840 tx=0x7fab58009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e98802889aa056a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab66ffd700 1 -- 172.21.15.60:0/3349239872 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58011830 con 0x7fab680b8200 2023-07-13T22:21:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab66ffd700 1 -- 172.21.15.60:0/3349239872 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab58011990 con 0x7fab680b8200 2023-07-13T22:21:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.701+0000 7fab66ffd700 1 -- 172.21.15.60:0/3349239872 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58011d20 con 0x7fab680b8200 2023-07-13T22:21:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.702+0000 7fab6e3ff700 1 -- 172.21.15.60:0/3349239872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab680b8200 msgr2=0x7fab680cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.702+0000 7fab6e3ff700 1 --2- 172.21.15.60:0/3349239872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab680b8200 0x7fab680cf640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab58009840 tx=0x7fab58009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.702+0000 7fab6e3ff700 1 -- 172.21.15.60:0/3349239872 shutdown_connections 2023-07-13T22:21:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.702+0000 7fab6e3ff700 1 --2- 172.21.15.60:0/3349239872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab680b8200 0x7fab680cf640 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.702+0000 7fab6e3ff700 1 -- 172.21.15.60:0/3349239872 >> 172.21.15.60:0/3349239872 conn(0x7fab6806b2f0 msgr2=0x7fab6806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.702+0000 7fab6e3ff700 1 -- 172.21.15.60:0/3349239872 shutdown_connections 2023-07-13T22:21:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.702+0000 7fab6e3ff700 1 -- 172.21.15.60:0/3349239872 wait complete. 2023-07-13T22:21:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.704+0000 7fab6e3ff700 1 Processor -- start 2023-07-13T22:21:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.704+0000 7fab6e3ff700 1 -- start start 2023-07-13T22:21:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab680b1ac0 0x7fab68153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6e3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab680b8200 0x7fab681e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6e3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab680cede0 0x7fab68071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6e3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab681e5d20 con 0x7fab680cede0 2023-07-13T22:21:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6e3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab681e5fc0 con 0x7fab680b8200 2023-07-13T22:21:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab681e62a0 con 0x7fab680b1ac0 2023-07-13T22:21:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6c99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab680cede0 0x7fab68071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6c99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab680cede0 0x7fab68071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45806/0 (socket says 172.21.15.60:45806) 2023-07-13T22:21:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6c99c700 1 -- 172.21.15.60:0/2795539647 learned_addr learned my addr 172.21.15.60:0/2795539647 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab677fe700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab680b8200 0x7fab681e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab67fff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab680b1ac0 0x7fab68153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6c99c700 1 -- 172.21.15.60:0/2795539647 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab680b1ac0 msgr2=0x7fab68153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6c99c700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab680b1ac0 0x7fab68153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.705+0000 7fab6c99c700 1 -- 172.21.15.60:0/2795539647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab680b8200 msgr2=0x7fab681e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.706+0000 7fab6c99c700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab680b8200 0x7fab681e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.706+0000 7fab6c99c700 1 -- 172.21.15.60:0/2795539647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab580094a0 con 0x7fab680cede0 2023-07-13T22:21:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.706+0000 7fab67fff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab680b1ac0 0x7fab68153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.706+0000 7fab6c99c700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab680cede0 0x7fab68071600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab6000cbc0 tx=0x7fab6000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.706+0000 7fab64ff9700 1 -- 172.21.15.60:0/2795539647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60003c00 con 0x7fab680cede0 2023-07-13T22:21:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.706+0000 7fab64ff9700 1 -- 172.21.15.60:0/2795539647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab60003d60 con 0x7fab680cede0 2023-07-13T22:21:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.706+0000 7fab64ff9700 1 -- 172.21.15.60:0/2795539647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6000ab00 con 0x7fab680cede0 2023-07-13T22:21:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.706+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab681e6580 con 0x7fab680cede0 2023-07-13T22:21:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.706+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab681e82e0 con 0x7fab680cede0 2023-07-13T22:21:09.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.707+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab681e6aa0 con 0x7fab680cede0 2023-07-13T22:21:09.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.709+0000 7fab64ff9700 1 -- 172.21.15.60:0/2795539647 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fab6000ac60 con 0x7fab680cede0 2023-07-13T22:21:09.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.710+0000 7fab64ff9700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fab5406ace0 0x7fab5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:09.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.710+0000 7fab64ff9700 1 -- 172.21.15.60:0/2795539647 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab60089460 con 0x7fab680cede0 2023-07-13T22:21:09.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.710+0000 7fab67fff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fab5406ace0 0x7fab5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:09.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.710+0000 7fab67fff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fab5406ace0 0x7fab5406d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab58009fd0 tx=0x7fab5801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:09.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.714+0000 7fab64ff9700 1 -- 172.21.15.60:0/2795539647 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fab681e6aa0 con 0x7fab680cede0 2023-07-13T22:21:09.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.922+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab681e6aa0 con 0x7fab680cede0 2023-07-13T22:21:09.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.922+0000 7fab64ff9700 1 -- 172.21.15.60:0/2795539647 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab681e6aa0 con 0x7fab680cede0 2023-07-13T22:21:09.906 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:09.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fab5406ace0 msgr2=0x7fab5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fab5406ace0 0x7fab5406d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab58009fd0 tx=0x7fab5801c040 comp rx=0 tx=0).stop 2023-07-13T22:21:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab680cede0 msgr2=0x7fab68071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab680cede0 0x7fab68071600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab6000cbc0 tx=0x7fab6000ced0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 shutdown_connections 2023-07-13T22:21:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fab5406ace0 0x7fab5406d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab680cede0 0x7fab68071600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab680b1ac0 0x7fab68153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 --2- 172.21.15.60:0/2795539647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab680b8200 0x7fab681e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.924+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 >> 172.21.15.60:0/2795539647 conn(0x7fab6806b2f0 msgr2=0x7fab680b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.925+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 shutdown_connections 2023-07-13T22:21:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:09.925+0000 7fab6e3ff700 1 -- 172.21.15.60:0/2795539647 wait complete. 2023-07-13T22:21:09.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:10.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.148+0000 7fe6c29ff700 1 Processor -- start 2023-07-13T22:21:10.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.149+0000 7fe6c29ff700 1 -- start start 2023-07-13T22:21:10.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6c29ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0b8200 0x7fe6bc0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6c29ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe6bc0b1ac0 0x7fe6bc0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6c29ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc0d1870 con 0x7fe6bc0cede0 2023-07-13T22:21:10.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6c29ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6bc0cf370 con 0x7fe6bc0b1ac0 2023-07-13T22:21:10.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6c29ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0d12b0 con 0x7fe6bc0b8200 2023-07-13T22:21:10.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6bbfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0b8200 0x7fe6bc0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6bb7fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe6bc0b1ac0 0x7fe6bc0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6c0f9c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe6bc0cede0 0x7fe6bc15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:33096/0 (socket says 172.21.15.60:33096) 2023-07-13T22:21:10.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6c0f9c700 1 -- 172.21.15.60:0/807773592 learned_addr learned my addr 172.21.15.60:0/807773592 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:10.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.150+0000 7fe6bbfff700 1 --2- 172.21.15.60:0/807773592 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0b8200 0x7fe6bc0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:39810/0 (socket says 172.21.15.60:39810) 2023-07-13T22:21:10.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6baffd700 1 -- 172.21.15.60:0/807773592 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972692641 0 0) 0x7fe6bc0d1870 con 0x7fe6bc0cede0 2023-07-13T22:21:10.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6baffd700 1 -- 172.21.15.60:0/807773592 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a40035b0 con 0x7fe6bc0cede0 2023-07-13T22:21:10.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6bbfff700 1 -- 172.21.15.60:0/807773592 >> v2:172.21.15.105:3300/0 conn(0x7fe6bc0b1ac0 msgr2=0x7fe6bc0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:10.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6bbfff700 1 --2- 172.21.15.60:0/807773592 >> v2:172.21.15.105:3300/0 conn(0x7fe6bc0b1ac0 0x7fe6bc0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6bbfff700 1 -- 172.21.15.60:0/807773592 >> v1:172.21.15.105:6789/0 conn(0x7fe6bc0cede0 legacy=0x7fe6bc15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6bbfff700 1 -- 172.21.15.60:0/807773592 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0d1590 con 0x7fe6bc0b8200 2023-07-13T22:21:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6bb7fe700 1 --2- 172.21.15.60:0/807773592 >> v2:172.21.15.105:3300/0 conn(0x7fe6bc0b1ac0 0x7fe6bc0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6bbfff700 1 --2- 172.21.15.60:0/807773592 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0b8200 0x7fe6bc0cf640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009840 tx=0x7fe6ac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af5f5517fa1ca204 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6baffd700 1 -- 172.21.15.60:0/807773592 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac011830 con 0x7fe6bc0b8200 2023-07-13T22:21:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6baffd700 1 -- 172.21.15.60:0/807773592 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac011990 con 0x7fe6bc0b8200 2023-07-13T22:21:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.151+0000 7fe6baffd700 1 -- 172.21.15.60:0/807773592 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac011d20 con 0x7fe6bc0b8200 2023-07-13T22:21:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.152+0000 7fe6c29ff700 1 -- 172.21.15.60:0/807773592 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0b8200 msgr2=0x7fe6bc0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.152+0000 7fe6c29ff700 1 --2- 172.21.15.60:0/807773592 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0b8200 0x7fe6bc0cf640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009840 tx=0x7fe6ac009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.152+0000 7fe6c29ff700 1 -- 172.21.15.60:0/807773592 shutdown_connections 2023-07-13T22:21:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.152+0000 7fe6c29ff700 1 --2- 172.21.15.60:0/807773592 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0b8200 0x7fe6bc0cf640 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.152+0000 7fe6c29ff700 1 --2- 172.21.15.60:0/807773592 >> v2:172.21.15.105:3300/0 conn(0x7fe6bc0b1ac0 0x7fe6bc0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.152+0000 7fe6c29ff700 1 -- 172.21.15.60:0/807773592 >> 172.21.15.60:0/807773592 conn(0x7fe6bc06b2f0 msgr2=0x7fe6bc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.152+0000 7fe6c29ff700 1 -- 172.21.15.60:0/807773592 shutdown_connections 2023-07-13T22:21:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.152+0000 7fe6c29ff700 1 -- 172.21.15.60:0/807773592 wait complete. 2023-07-13T22:21:10.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.154+0000 7fe6c29ff700 1 Processor -- start 2023-07-13T22:21:10.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.155+0000 7fe6c29ff700 1 -- start start 2023-07-13T22:21:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6c29ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6bc0b1ac0 0x7fe6bc158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6c29ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6bc0b8200 0x7fe6bc1e6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6c29ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0cede0 0x7fe6bc1e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6c29ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0c6760 con 0x7fe6bc0b8200 2023-07-13T22:21:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6c29ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1e6ea0 con 0x7fe6bc0b1ac0 2023-07-13T22:21:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6c29ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc158790 con 0x7fe6bc0cede0 2023-07-13T22:21:10.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6bb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6bc0b8200 0x7fe6bc1e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6bb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6bc0b8200 0x7fe6bc1e6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45808/0 (socket says 172.21.15.60:45808) 2023-07-13T22:21:10.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6bb7fe700 1 -- 172.21.15.60:0/4101091364 learned_addr learned my addr 172.21.15.60:0/4101091364 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:10.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6bbfff700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6bc0b1ac0 0x7fe6bc158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.156+0000 7fe6c0f9c700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0cede0 0x7fe6bc1e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6bb7fe700 1 -- 172.21.15.60:0/4101091364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0cede0 msgr2=0x7fe6bc1e2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6bb7fe700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0cede0 0x7fe6bc1e2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6bb7fe700 1 -- 172.21.15.60:0/4101091364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6bc0b1ac0 msgr2=0x7fe6bc158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6bb7fe700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6bc0b1ac0 0x7fe6bc158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6bb7fe700 1 -- 172.21.15.60:0/4101091364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b00094a0 con 0x7fe6bc0b8200 2023-07-13T22:21:10.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6bb7fe700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6bc0b8200 0x7fe6bc1e6960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000ebe0 tx=0x7fe6b000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6b8ff9700 1 -- 172.21.15.60:0/4101091364 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0009800 con 0x7fe6bc0b8200 2023-07-13T22:21:10.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac0094a0 con 0x7fe6bc0b8200 2023-07-13T22:21:10.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6b8ff9700 1 -- 172.21.15.60:0/4101091364 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b000c910 con 0x7fe6bc0b8200 2023-07-13T22:21:10.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6b8ff9700 1 -- 172.21.15.60:0/4101091364 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b000cc80 con 0x7fe6bc0b8200 2023-07-13T22:21:10.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.157+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc0c6c50 con 0x7fe6bc0b8200 2023-07-13T22:21:10.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.158+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc04e210 con 0x7fe6bc0b8200 2023-07-13T22:21:10.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.159+0000 7fe6b8ff9700 1 -- 172.21.15.60:0/4101091364 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe6b0013470 con 0x7fe6bc0b8200 2023-07-13T22:21:10.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.160+0000 7fe6b8ff9700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6a806ace0 0x7fe6a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.160+0000 7fe6b8ff9700 1 -- 172.21.15.60:0/4101091364 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6b0089620 con 0x7fe6bc0b8200 2023-07-13T22:21:10.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.160+0000 7fe6bbfff700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6a806ace0 0x7fe6a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.161+0000 7fe6bbfff700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6a806ace0 0x7fe6a806d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac000c00 tx=0x7fe6ac008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.164+0000 7fe6b8ff9700 1 -- 172.21.15.60:0/4101091364 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe6b00580d0 con 0x7fe6bc0b8200 2023-07-13T22:21:10.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.369+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6bc0cfd40 con 0x7fe6bc0b8200 2023-07-13T22:21:10.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.369+0000 7fe6b8ff9700 1 -- 172.21.15.60:0/4101091364 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6b0057c60 con 0x7fe6bc0b8200 2023-07-13T22:21:10.354 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:10.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6a806ace0 msgr2=0x7fe6a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6a806ace0 0x7fe6a806d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac000c00 tx=0x7fe6ac008030 comp rx=0 tx=0).stop 2023-07-13T22:21:10.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6bc0b8200 msgr2=0x7fe6bc1e6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6bc0b8200 0x7fe6bc1e6960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000ebe0 tx=0x7fe6b000eef0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 shutdown_connections 2023-07-13T22:21:10.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6a806ace0 0x7fe6a806d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6bc0cede0 0x7fe6bc1e2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6bc0b1ac0 0x7fe6bc158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 --2- 172.21.15.60:0/4101091364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6bc0b8200 0x7fe6bc1e6960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 >> 172.21.15.60:0/4101091364 conn(0x7fe6bc06b2f0 msgr2=0x7fe6bc0b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 shutdown_connections 2023-07-13T22:21:10.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.372+0000 7fe6c29ff700 1 -- 172.21.15.60:0/4101091364 wait complete. 2023-07-13T22:21:10.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:10.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.597+0000 7f47b4fff700 1 Processor -- start 2023-07-13T22:21:10.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.597+0000 7f47b4fff700 1 -- start start 2023-07-13T22:21:10.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47b4fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47b00cfd50 0x7f47b00d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47b4fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f47b00b1ac0 0x7f47b00d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47b4fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b00d1870 con 0x7f47b00b8200 2023-07-13T22:21:10.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47b4fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b00cf370 con 0x7f47b00cfd50 2023-07-13T22:21:10.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47b4fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b00d12b0 con 0x7f47b00b1ac0 2023-07-13T22:21:10.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47add9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47b00cfd50 0x7f47b00d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47ae59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f47b00b8200 0x7f47b00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47792/0 (socket says 172.21.15.60:47792) 2023-07-13T22:21:10.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47ae59c700 1 -- 172.21.15.60:0/4140627909 learned_addr learned my addr 172.21.15.60:0/4140627909 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:10.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47aed9d700 1 --2- 172.21.15.60:0/4140627909 >> v2:172.21.15.105:3300/0 conn(0x7f47b00b1ac0 0x7f47b00d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47ad59a700 1 -- 172.21.15.60:0/4140627909 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324638897 0 0) 0x7f47b00d1870 con 0x7f47b00b8200 2023-07-13T22:21:10.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47add9b700 1 -- 172.21.15.60:0/4140627909 >> v2:172.21.15.105:3300/0 conn(0x7f47b00b1ac0 msgr2=0x7f47b00d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:10.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47add9b700 1 --2- 172.21.15.60:0/4140627909 >> v2:172.21.15.105:3300/0 conn(0x7f47b00b1ac0 0x7f47b00d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47add9b700 1 -- 172.21.15.60:0/4140627909 >> v1:172.21.15.60:6789/0 conn(0x7f47b00b8200 legacy=0x7f47b00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.599+0000 7f47add9b700 1 -- 172.21.15.60:0/4140627909 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00d1590 con 0x7f47b00cfd50 2023-07-13T22:21:10.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.600+0000 7f47add9b700 1 --2- 172.21.15.60:0/4140627909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00cfd50 0x7f47b00d23c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47b00b8d70 tx=0x7f47a4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d13cce489d1194 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.600+0000 7f47ad59a700 1 -- 172.21.15.60:0/4140627909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4011830 con 0x7f47b00cfd50 2023-07-13T22:21:10.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.600+0000 7f47ad59a700 1 -- 172.21.15.60:0/4140627909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a4011990 con 0x7f47b00cfd50 2023-07-13T22:21:10.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.600+0000 7f47ad59a700 1 -- 172.21.15.60:0/4140627909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4011d20 con 0x7f47b00cfd50 2023-07-13T22:21:10.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.600+0000 7f47b4fff700 1 -- 172.21.15.60:0/4140627909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00cfd50 msgr2=0x7f47b00d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.600+0000 7f47b4fff700 1 --2- 172.21.15.60:0/4140627909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00cfd50 0x7f47b00d23c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47b00b8d70 tx=0x7f47a4009b20 comp rx=0 tx=0).stop 2023-07-13T22:21:10.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.601+0000 7f47b4fff700 1 -- 172.21.15.60:0/4140627909 shutdown_connections 2023-07-13T22:21:10.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.601+0000 7f47b4fff700 1 --2- 172.21.15.60:0/4140627909 >> v2:172.21.15.105:3300/0 conn(0x7f47b00b1ac0 0x7f47b00d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.601+0000 7f47b4fff700 1 --2- 172.21.15.60:0/4140627909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00cfd50 0x7f47b00d23c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.601+0000 7f47b4fff700 1 -- 172.21.15.60:0/4140627909 >> 172.21.15.60:0/4140627909 conn(0x7f47b006b2f0 msgr2=0x7f47b006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.601+0000 7f47b4fff700 1 -- 172.21.15.60:0/4140627909 shutdown_connections 2023-07-13T22:21:10.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.601+0000 7f47b4fff700 1 -- 172.21.15.60:0/4140627909 wait complete. 2023-07-13T22:21:10.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.603+0000 7f47b4fff700 1 Processor -- start 2023-07-13T22:21:10.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.603+0000 7f47b4fff700 1 -- start start 2023-07-13T22:21:10.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47b4fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00b1ac0 0x7f47b0155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47b4fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47b00b8200 0x7f47b015d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47b4fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b00cfd50 0x7f47b015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47b4fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0156b40 con 0x7f47b00b1ac0 2023-07-13T22:21:10.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47b4fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b015dda0 con 0x7f47b00cfd50 2023-07-13T22:21:10.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47b4fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b015e080 con 0x7f47b00b8200 2023-07-13T22:21:10.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47ae59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00b1ac0 0x7f47b0155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47ae59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00b1ac0 0x7f47b0155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45818/0 (socket says 172.21.15.60:45818) 2023-07-13T22:21:10.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47ae59c700 1 -- 172.21.15.60:0/792887901 learned_addr learned my addr 172.21.15.60:0/792887901 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:10.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47aed9d700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b00cfd50 0x7f47b015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47add9b700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47b00b8200 0x7f47b015d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47ae59c700 1 -- 172.21.15.60:0/792887901 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47b00b8200 msgr2=0x7f47b015d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47ae59c700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47b00b8200 0x7f47b015d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47ae59c700 1 -- 172.21.15.60:0/792887901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b00cfd50 msgr2=0x7f47b015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47ae59c700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b00cfd50 0x7f47b015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.604+0000 7f47ae59c700 1 -- 172.21.15.60:0/792887901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a40094a0 con 0x7f47b00b1ac0 2023-07-13T22:21:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.605+0000 7f47ae59c700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00b1ac0 0x7f47b0155d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f47a00011e0 tx=0x7f47a000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.605+0000 7f479effd700 1 -- 172.21.15.60:0/792887901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0004c80 con 0x7f47b00b1ac0 2023-07-13T22:21:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.605+0000 7f479effd700 1 -- 172.21.15.60:0/792887901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a0004e00 con 0x7f47b00b1ac0 2023-07-13T22:21:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.605+0000 7f479effd700 1 -- 172.21.15.60:0/792887901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0007310 con 0x7f47b00b1ac0 2023-07-13T22:21:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.605+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b015e360 con 0x7f47b00b1ac0 2023-07-13T22:21:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.605+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b0156280 con 0x7f47b00b1ac0 2023-07-13T22:21:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.606+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b00b8940 con 0x7f47b00b1ac0 2023-07-13T22:21:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.607+0000 7f479effd700 1 -- 172.21.15.60:0/792887901 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47a0007470 con 0x7f47b00b1ac0 2023-07-13T22:21:10.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.608+0000 7f479effd700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f479006ace0 0x7f479006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.608+0000 7f479effd700 1 -- 172.21.15.60:0/792887901 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47a00890c0 con 0x7f47b00b1ac0 2023-07-13T22:21:10.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.608+0000 7f47add9b700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f479006ace0 0x7f479006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.609+0000 7f47add9b700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f479006ace0 0x7f479006d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f47b00d0140 tx=0x7f47a4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.612+0000 7f479effd700 1 -- 172.21.15.60:0/792887901 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f47a0054160 con 0x7f47b00b1ac0 2023-07-13T22:21:10.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.819+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47b004e210 con 0x7f47b00b1ac0 2023-07-13T22:21:10.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.820+0000 7f479effd700 1 -- 172.21.15.60:0/792887901 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47a0057780 con 0x7f47b00b1ac0 2023-07-13T22:21:10.804 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:10.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f479006ace0 msgr2=0x7f479006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f479006ace0 0x7f479006d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f47b00d0140 tx=0x7f47a4008030 comp rx=0 tx=0).stop 2023-07-13T22:21:10.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00b1ac0 msgr2=0x7f47b0155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00b1ac0 0x7f47b0155d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f47a00011e0 tx=0x7f47a000ca30 comp rx=0 tx=0).stop 2023-07-13T22:21:10.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 shutdown_connections 2023-07-13T22:21:10.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f479006ace0 0x7f479006d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47b00b1ac0 0x7f47b0155d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b00cfd50 0x7f47b015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 --2- 172.21.15.60:0/792887901 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47b00b8200 0x7f47b015d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 >> 172.21.15.60:0/792887901 conn(0x7f47b006b2f0 msgr2=0x7f47b015c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 shutdown_connections 2023-07-13T22:21:10.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:10.822+0000 7f47b4fff700 1 -- 172.21.15.60:0/792887901 wait complete. 2023-07-13T22:21:10.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:21:11.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.042+0000 7f29513ff700 1 Processor -- start 2023-07-13T22:21:11.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.042+0000 7f29513ff700 1 -- start start 2023-07-13T22:21:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.043+0000 7f29513ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0d3750 0x7f294c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.043+0000 7f29513ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f294c0cfad0 0x7f294c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.043+0000 7f29513ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c0cd950 con 0x7f294c0b8400 2023-07-13T22:21:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.043+0000 7f29513ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0cff20 con 0x7f294c0cfad0 2023-07-13T22:21:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.043+0000 7f29513ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0d0ac0 con 0x7f294c0d3750 2023-07-13T22:21:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.043+0000 7f294affd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f294c0b8400 0x7f294c0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47802/0 (socket says 172.21.15.60:47802) 2023-07-13T22:21:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.043+0000 7f294affd700 1 -- 172.21.15.60:0/136299913 learned_addr learned my addr 172.21.15.60:0/136299913 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.044+0000 7f294b7fe700 1 --2- 172.21.15.60:0/136299913 >> v2:172.21.15.105:3300/0 conn(0x7f294c0cfad0 0x7f294c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.044+0000 7f294a7fc700 1 --2- 172.21.15.60:0/136299913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0d3750 0x7f294c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.044+0000 7f2949ffb700 1 -- 172.21.15.60:0/136299913 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778425688 0 0) 0x7f293c001110 con 0x7f294c0b8400 2023-07-13T22:21:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.044+0000 7f2949ffb700 1 -- 172.21.15.60:0/136299913 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29340035b0 con 0x7f294c0b8400 2023-07-13T22:21:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.044+0000 7f294b7fe700 1 -- 172.21.15.60:0/136299913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0d3750 msgr2=0x7f294c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.044+0000 7f294b7fe700 1 --2- 172.21.15.60:0/136299913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0d3750 0x7f294c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.044+0000 7f294b7fe700 1 -- 172.21.15.60:0/136299913 >> v1:172.21.15.60:6789/0 conn(0x7f294c0b8400 legacy=0x7f294c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.044+0000 7f294b7fe700 1 -- 172.21.15.60:0/136299913 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c0b87d0 con 0x7f294c0cfad0 2023-07-13T22:21:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f294b7fe700 1 --2- 172.21.15.60:0/136299913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c0cfad0 0x7f294c0d0f50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2944009d90 tx=0x7f294400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36ba2a0515516fe1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f2949ffb700 1 -- 172.21.15.60:0/136299913 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294400e070 con 0x7f294c0cfad0 2023-07-13T22:21:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f2949ffb700 1 -- 172.21.15.60:0/136299913 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294400bb20 con 0x7f294c0cfad0 2023-07-13T22:21:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f2949ffb700 1 -- 172.21.15.60:0/136299913 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944012430 con 0x7f294c0cfad0 2023-07-13T22:21:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f29513ff700 1 -- 172.21.15.60:0/136299913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c0cfad0 msgr2=0x7f294c0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f29513ff700 1 --2- 172.21.15.60:0/136299913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c0cfad0 0x7f294c0d0f50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2944009d90 tx=0x7f294400b330 comp rx=0 tx=0).stop 2023-07-13T22:21:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f29513ff700 1 -- 172.21.15.60:0/136299913 shutdown_connections 2023-07-13T22:21:11.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f29513ff700 1 --2- 172.21.15.60:0/136299913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c0cfad0 0x7f294c0d0f50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f29513ff700 1 --2- 172.21.15.60:0/136299913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0d3750 0x7f294c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f29513ff700 1 -- 172.21.15.60:0/136299913 >> 172.21.15.60:0/136299913 conn(0x7f294c06b390 msgr2=0x7f294c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:11.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.045+0000 7f29513ff700 1 -- 172.21.15.60:0/136299913 shutdown_connections 2023-07-13T22:21:11.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.046+0000 7f29513ff700 1 -- 172.21.15.60:0/136299913 wait complete. 2023-07-13T22:21:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.047+0000 7f29513ff700 1 Processor -- start 2023-07-13T22:21:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.047+0000 7f29513ff700 1 -- start start 2023-07-13T22:21:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.048+0000 7f29513ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0b8400 0x7f294c1f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.048+0000 7f29513ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0cfad0 0x7f294c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.048+0000 7f29513ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c0d3750 0x7f294c1f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.048+0000 7f29513ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c1f5bf0 con 0x7f294c0b8400 2023-07-13T22:21:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.048+0000 7f29513ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c1544e0 con 0x7f294c0d3750 2023-07-13T22:21:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.048+0000 7f29513ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c1f2b00 con 0x7f294c0cfad0 2023-07-13T22:21:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.048+0000 7f294affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0b8400 0x7f294c1f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0b8400 0x7f294c1f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45824/0 (socket says 172.21.15.60:45824) 2023-07-13T22:21:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294affd700 1 -- 172.21.15.60:0/1931647350 learned_addr learned my addr 172.21.15.60:0/1931647350 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294b7fe700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c0d3750 0x7f294c1f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294a7fc700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0cfad0 0x7f294c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294affd700 1 -- 172.21.15.60:0/1931647350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0cfad0 msgr2=0x7f294c153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294affd700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0cfad0 0x7f294c153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294affd700 1 -- 172.21.15.60:0/1931647350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c0d3750 msgr2=0x7f294c1f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294affd700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c0d3750 0x7f294c1f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294affd700 1 -- 172.21.15.60:0/1931647350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2944009a70 con 0x7f294c0b8400 2023-07-13T22:21:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f294affd700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0b8400 0x7f294c1f6080 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f293c007fd0 tx=0x7f293c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f29337fe700 1 -- 172.21.15.60:0/1931647350 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c005d70 con 0x7f294c0b8400 2023-07-13T22:21:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f29337fe700 1 -- 172.21.15.60:0/1931647350 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c0070d0 con 0x7f294c0b8400 2023-07-13T22:21:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c1f2de0 con 0x7f294c0b8400 2023-07-13T22:21:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.049+0000 7f29337fe700 1 -- 172.21.15.60:0/1931647350 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c007460 con 0x7f294c0b8400 2023-07-13T22:21:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.050+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c1f5880 con 0x7f294c0b8400 2023-07-13T22:21:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.051+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c1f5ed0 con 0x7f294c0b8400 2023-07-13T22:21:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.052+0000 7f29337fe700 1 -- 172.21.15.60:0/1931647350 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f293c0075c0 con 0x7f294c0b8400 2023-07-13T22:21:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.052+0000 7f29337fe700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.052+0000 7f29337fe700 1 -- 172.21.15.60:0/1931647350 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f293c088680 con 0x7f294c0b8400 2023-07-13T22:21:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.053+0000 7f294a7fc700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.053+0000 7f294a7fc700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f294c0b7360 tx=0x7f2940009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.056+0000 7f29337fe700 1 -- 172.21.15.60:0/1931647350 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f293c0537c0 con 0x7f294c0b8400 2023-07-13T22:21:11.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.307+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f294c0cd950 con 0x7f294c0b8400 2023-07-13T22:21:11.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.308+0000 7f29337fe700 1 -- 172.21.15.60:0/1931647350 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f294c0cd950 con 0x7f294c0b8400 2023-07-13T22:21:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:11.292 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2550952354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2550952354},{"type":"v1","addr":"172.21.15.60:6837","nonce":2550952354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4134045869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4134045869},{"type":"v1","addr":"172.21.15.60:6836","nonce":4134045869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2202330082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2202330082},{"type":"v1","addr":"172.21.15.105:6834","nonce":2202330082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:20:54.038443+0000","modified":"2023-07-13T22:20:57.536915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/453882022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":453882022},{"type":"v1","addr":"172.21.15.105:6835","nonce":453882022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:21:11.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 msgr2=0x7f293806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f294c0b7360 tx=0x7f2940009210 comp rx=0 tx=0).stop 2023-07-13T22:21:11.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0b8400 msgr2=0x7f294c1f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0b8400 0x7f294c1f6080 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f293c007fd0 tx=0x7f293c00c7b0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 shutdown_connections 2023-07-13T22:21:11.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0b8400 0x7f294c1f6080 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c0d3750 0x7f294c1f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 --2- 172.21.15.60:0/1931647350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c0cfad0 0x7f294c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 >> 172.21.15.60:0/1931647350 conn(0x7f294c06b390 msgr2=0x7f294c15ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:11.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 shutdown_connections 2023-07-13T22:21:11.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.310+0000 7f29513ff700 1 -- 172.21.15.60:0/1931647350 wait complete. 2023-07-13T22:21:11.296 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2023-07-13T22:21:11.312 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.531+0000 7f1452bff700 1 Processor -- start 2023-07-13T22:21:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.531+0000 7f1452bff700 1 -- start start 2023-07-13T22:21:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.532+0000 7f1452bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f144c0b8400 0x7f144c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.532+0000 7f1452bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f144c0d3750 0x7f144c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f1452bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144c0d0ac0 con 0x7f144c0cfad0 2023-07-13T22:21:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f1452bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c0cff50 con 0x7f144c0b8400 2023-07-13T22:21:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f1452bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c0b87d0 con 0x7f144c0d3750 2023-07-13T22:21:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f145099b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f144c0b8400 0x7f144c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f145099b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f144c0b8400 0x7f144c0cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45828/0 (socket says 172.21.15.60:45828) 2023-07-13T22:21:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f145099b700 1 -- 172.21.15.60:0/2930536678 learned_addr learned my addr 172.21.15.60:0/2930536678 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f144bfff700 1 --2- 172.21.15.60:0/2930536678 >> v2:172.21.15.105:3300/0 conn(0x7f144c0d3750 0x7f144c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f145099b700 1 -- 172.21.15.60:0/2930536678 >> v2:172.21.15.105:3300/0 conn(0x7f144c0d3750 msgr2=0x7f144c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f145099b700 1 --2- 172.21.15.60:0/2930536678 >> v2:172.21.15.105:3300/0 conn(0x7f144c0d3750 0x7f144c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f145099b700 1 -- 172.21.15.60:0/2930536678 >> v1:172.21.15.105:6789/0 conn(0x7f144c0cfad0 legacy=0x7f144c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f145099b700 1 -- 172.21.15.60:0/2930536678 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c0b4eb0 con 0x7f144c0b8400 2023-07-13T22:21:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.533+0000 7f145099b700 1 --2- 172.21.15.60:0/2930536678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0b8400 0x7f144c0cd240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f144c0b8cf0 tx=0x7f1440009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4792027de77c826f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f144b7fe700 1 -- 172.21.15.60:0/2930536678 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440011830 con 0x7f144c0b8400 2023-07-13T22:21:11.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f144b7fe700 1 -- 172.21.15.60:0/2930536678 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1440011990 con 0x7f144c0b8400 2023-07-13T22:21:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f144b7fe700 1 -- 172.21.15.60:0/2930536678 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440011d20 con 0x7f144c0b8400 2023-07-13T22:21:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f1452bff700 1 -- 172.21.15.60:0/2930536678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0b8400 msgr2=0x7f144c0cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f1452bff700 1 --2- 172.21.15.60:0/2930536678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0b8400 0x7f144c0cd240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f144c0b8cf0 tx=0x7f1440009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f1452bff700 1 -- 172.21.15.60:0/2930536678 shutdown_connections 2023-07-13T22:21:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f1452bff700 1 --2- 172.21.15.60:0/2930536678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0b8400 0x7f144c0cd240 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f1452bff700 1 --2- 172.21.15.60:0/2930536678 >> v2:172.21.15.105:3300/0 conn(0x7f144c0d3750 0x7f144c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f1452bff700 1 -- 172.21.15.60:0/2930536678 >> 172.21.15.60:0/2930536678 conn(0x7f144c06b390 msgr2=0x7f144c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f1452bff700 1 -- 172.21.15.60:0/2930536678 shutdown_connections 2023-07-13T22:21:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.534+0000 7f1452bff700 1 -- 172.21.15.60:0/2930536678 wait complete. 2023-07-13T22:21:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.536+0000 7f1452bff700 1 Processor -- start 2023-07-13T22:21:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.536+0000 7f1452bff700 1 -- start start 2023-07-13T22:21:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f1452bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f144c0b8400 0x7f144c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f1452bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0cfad0 0x7f144c0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f1452bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f144c0d3750 0x7f144c0b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f1452bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0c7d90 con 0x7f144c0cfad0 2023-07-13T22:21:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f1452bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c0c79d0 con 0x7f144c0d3750 2023-07-13T22:21:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f1452bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c154770 con 0x7f144c0b8400 2023-07-13T22:21:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f144bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0cfad0 0x7f144c0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f144bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0cfad0 0x7f144c0c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45836/0 (socket says 172.21.15.60:45836) 2023-07-13T22:21:11.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f144bfff700 1 -- 172.21.15.60:0/4029736501 learned_addr learned my addr 172.21.15.60:0/4029736501 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:11.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.537+0000 7f145099b700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f144c0b8400 0x7f144c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f145119c700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f144c0d3750 0x7f144c0b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f144bfff700 1 -- 172.21.15.60:0/4029736501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f144c0b8400 msgr2=0x7f144c154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f144bfff700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f144c0b8400 0x7f144c154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f144bfff700 1 -- 172.21.15.60:0/4029736501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f144c0d3750 msgr2=0x7f144c0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f144bfff700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f144c0d3750 0x7f144c0b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f144bfff700 1 -- 172.21.15.60:0/4029736501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440009550 con 0x7f144c0cfad0 2023-07-13T22:21:11.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f145119c700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f144c0d3750 0x7f144c0b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:11.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f144bfff700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0cfad0 0x7f144c0c66b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f143800e840 tx=0x7f143800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f14497fa700 1 -- 172.21.15.60:0/4029736501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438009590 con 0x7f144c0cfad0 2023-07-13T22:21:11.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f14497fa700 1 -- 172.21.15.60:0/4029736501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438013680 con 0x7f144c0cfad0 2023-07-13T22:21:11.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144c0c7130 con 0x7f144c0cfad0 2023-07-13T22:21:11.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f14497fa700 1 -- 172.21.15.60:0/4029736501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438013a30 con 0x7f144c0cfad0 2023-07-13T22:21:11.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.538+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c0c76f0 con 0x7f144c0cfad0 2023-07-13T22:21:11.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.539+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144c15b040 con 0x7f144c0cfad0 2023-07-13T22:21:11.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.540+0000 7f14497fa700 1 -- 172.21.15.60:0/4029736501 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1438013b90 con 0x7f144c0cfad0 2023-07-13T22:21:11.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.542+0000 7f14497fa700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f143006ace0 0x7f143006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.542+0000 7f14497fa700 1 -- 172.21.15.60:0/4029736501 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1438089380 con 0x7f144c0cfad0 2023-07-13T22:21:11.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.542+0000 7f145099b700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f143006ace0 0x7f143006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.542+0000 7f145099b700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f143006ace0 0x7f143006d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1440000c00 tx=0x7f1440008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.545+0000 7f14497fa700 1 -- 172.21.15.60:0/4029736501 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1438054420 con 0x7f144c0cfad0 2023-07-13T22:21:11.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.753+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f144c0b5440 con 0x7f144c0cfad0 2023-07-13T22:21:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.754+0000 7f14497fa700 1 -- 172.21.15.60:0/4029736501 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f1438057a40 con 0x7f144c0cfad0 2023-07-13T22:21:11.739 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:11.739 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:11.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f143006ace0 msgr2=0x7f143006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f143006ace0 0x7f143006d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1440000c00 tx=0x7f1440008030 comp rx=0 tx=0).stop 2023-07-13T22:21:11.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0cfad0 msgr2=0x7f144c0c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0cfad0 0x7f144c0c66b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f143800e840 tx=0x7f143800eb50 comp rx=0 tx=0).stop 2023-07-13T22:21:11.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 shutdown_connections 2023-07-13T22:21:11.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f143006ace0 0x7f143006d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f144c0d3750 0x7f144c0b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f144c0b8400 0x7f144c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 --2- 172.21.15.60:0/4029736501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c0cfad0 0x7f144c0c66b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.756+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 >> 172.21.15.60:0/4029736501 conn(0x7f144c06b390 msgr2=0x7f144c15bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:11.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.757+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 shutdown_connections 2023-07-13T22:21:11.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.757+0000 7f1452bff700 1 -- 172.21.15.60:0/4029736501 wait complete. 2023-07-13T22:21:11.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:21:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.976+0000 7f70e6bff700 1 Processor -- start 2023-07-13T22:21:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.976+0000 7f70e6bff700 1 -- start start 2023-07-13T22:21:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.977+0000 7f70e6bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70e00b8400 0x7f70e00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.977+0000 7f70e6bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00cfca0 0x7f70e00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.977+0000 7f70e6bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e00d0ac0 con 0x7f70e00d3750 2023-07-13T22:21:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.977+0000 7f70e6bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e00b87d0 con 0x7f70e00b8400 2023-07-13T22:21:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.977+0000 7f70e6bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00b4eb0 con 0x7f70e00cfca0 2023-07-13T22:21:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.977+0000 7f70e499b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70e00b8400 0x7f70e00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.977+0000 7f70e499b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70e00b8400 0x7f70e00cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45840/0 (socket says 172.21.15.60:45840) 2023-07-13T22:21:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.977+0000 7f70e499b700 1 -- 172.21.15.60:0/3270451325 learned_addr learned my addr 172.21.15.60:0/3270451325 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.977+0000 7f70e519c700 1 --2- 172.21.15.60:0/3270451325 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00cfca0 0x7f70e00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.978+0000 7f70e499b700 1 -- 172.21.15.60:0/3270451325 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00cfca0 msgr2=0x7f70e00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:11.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.978+0000 7f70e499b700 1 --2- 172.21.15.60:0/3270451325 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00cfca0 0x7f70e00d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.978+0000 7f70e499b700 1 -- 172.21.15.60:0/3270451325 >> v1:172.21.15.105:6789/0 conn(0x7f70e00d3750 legacy=0x7f70e00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.978+0000 7f70e499b700 1 -- 172.21.15.60:0/3270451325 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00b5190 con 0x7f70e00b8400 2023-07-13T22:21:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.978+0000 7f70df7fe700 1 -- 172.21.15.60:0/3270451325 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615029771 0 0) 0x7f70e00d0ac0 con 0x7f70e00d3750 2023-07-13T22:21:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.978+0000 7f70e499b700 1 --2- 172.21.15.60:0/3270451325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00b8400 0x7f70e00cd240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f70d40098f0 tx=0x7f70d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f1162d580b0d54 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.978+0000 7f70df7fe700 1 -- 172.21.15.60:0/3270451325 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4011830 con 0x7f70e00b8400 2023-07-13T22:21:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.978+0000 7f70df7fe700 1 -- 172.21.15.60:0/3270451325 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4011990 con 0x7f70e00b8400 2023-07-13T22:21:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.978+0000 7f70df7fe700 1 -- 172.21.15.60:0/3270451325 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4011d20 con 0x7f70e00b8400 2023-07-13T22:21:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.979+0000 7f70e6bff700 1 -- 172.21.15.60:0/3270451325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00b8400 msgr2=0x7f70e00cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.979+0000 7f70e6bff700 1 --2- 172.21.15.60:0/3270451325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00b8400 0x7f70e00cd240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f70d40098f0 tx=0x7f70d4009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.979+0000 7f70e6bff700 1 -- 172.21.15.60:0/3270451325 shutdown_connections 2023-07-13T22:21:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.979+0000 7f70e6bff700 1 --2- 172.21.15.60:0/3270451325 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00cfca0 0x7f70e00d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.979+0000 7f70e6bff700 1 --2- 172.21.15.60:0/3270451325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00b8400 0x7f70e00cd240 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.979+0000 7f70e6bff700 1 -- 172.21.15.60:0/3270451325 >> 172.21.15.60:0/3270451325 conn(0x7f70e006b390 msgr2=0x7f70e006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.979+0000 7f70e6bff700 1 -- 172.21.15.60:0/3270451325 shutdown_connections 2023-07-13T22:21:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.979+0000 7f70e6bff700 1 -- 172.21.15.60:0/3270451325 wait complete. 2023-07-13T22:21:11.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.981+0000 7f70e6bff700 1 Processor -- start 2023-07-13T22:21:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.981+0000 7f70e6bff700 1 -- start start 2023-07-13T22:21:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e6bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00b8400 0x7f70e01f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e6bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70e00cfca0 0x7f70e00b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e6bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00d3750 0x7f70e01f1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e6bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e015eaa0 con 0x7f70e00d3750 2023-07-13T22:21:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e6bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e01f6280 con 0x7f70e00cfca0 2023-07-13T22:21:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e6bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e015c630 con 0x7f70e00b8400 2023-07-13T22:21:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e519c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00d3750 0x7f70e01f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e519c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00d3750 0x7f70e01f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45854/0 (socket says 172.21.15.60:45854) 2023-07-13T22:21:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e519c700 1 -- 172.21.15.60:0/3826488887 learned_addr learned my addr 172.21.15.60:0/3826488887 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e499b700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00b8400 0x7f70e01f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e519c700 1 -- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00b8400 msgr2=0x7f70e01f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e519c700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00b8400 0x7f70e01f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70dffff700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70e00cfca0 0x7f70e00b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.982+0000 7f70e519c700 1 -- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70e00cfca0 msgr2=0x7f70e00b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70e519c700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70e00cfca0 0x7f70e00b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70e519c700 1 -- 172.21.15.60:0/3826488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4009550 con 0x7f70e00d3750 2023-07-13T22:21:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70e499b700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00b8400 0x7f70e01f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70dffff700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70e00cfca0 0x7f70e00b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70e519c700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00d3750 0x7f70e01f1240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f70d800efc0 tx=0x7f70d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70dd7fa700 1 -- 172.21.15.60:0/3826488887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8009b20 con 0x7f70e00d3750 2023-07-13T22:21:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70dd7fa700 1 -- 172.21.15.60:0/3826488887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d800f040 con 0x7f70e00d3750 2023-07-13T22:21:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70dd7fa700 1 -- 172.21.15.60:0/3826488887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8009e50 con 0x7f70e00d3750 2023-07-13T22:21:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e015c910 con 0x7f70e00d3750 2023-07-13T22:21:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.983+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e015ced0 con 0x7f70e00d3750 2023-07-13T22:21:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.984+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e00cf690 con 0x7f70e00d3750 2023-07-13T22:21:11.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.985+0000 7f70dd7fa700 1 -- 172.21.15.60:0/3826488887 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70d8014480 con 0x7f70e00d3750 2023-07-13T22:21:11.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.986+0000 7f70dd7fa700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f70c406ace0 0x7f70c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.987+0000 7f70dd7fa700 1 -- 172.21.15.60:0/3826488887 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70d8089ff0 con 0x7f70e00d3750 2023-07-13T22:21:11.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.987+0000 7f70e499b700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f70c406ace0 0x7f70c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.987+0000 7f70e499b700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f70c406ace0 0x7f70c406d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f70d4000c00 tx=0x7f70d4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:11.991+0000 7f70dd7fa700 1 -- 172.21.15.60:0/3826488887 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f70d8058b80 con 0x7f70e00d3750 2023-07-13T22:21:12.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.240+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70e00cf090 con 0x7f70e00d3750 2023-07-13T22:21:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.241+0000 7f70dd7fa700 1 -- 172.21.15.60:0/3826488887 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f70d8058710 con 0x7f70e00d3750 2023-07-13T22:21:12.226 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:12.227 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2550952354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2550952354},{"type":"v1","addr":"172.21.15.60:6837","nonce":2550952354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4134045869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4134045869},{"type":"v1","addr":"172.21.15.60:6836","nonce":4134045869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2202330082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2202330082},{"type":"v1","addr":"172.21.15.105:6834","nonce":2202330082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:20:54.038443+0000","modified":"2023-07-13T22:20:57.536915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/453882022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":453882022},{"type":"v1","addr":"172.21.15.105:6835","nonce":453882022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:21:12.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f70c406ace0 msgr2=0x7f70c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f70c406ace0 0x7f70c406d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f70d4000c00 tx=0x7f70d4008030 comp rx=0 tx=0).stop 2023-07-13T22:21:12.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00d3750 msgr2=0x7f70e01f1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00d3750 0x7f70e01f1240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f70d800efc0 tx=0x7f70d800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 shutdown_connections 2023-07-13T22:21:12.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f70c406ace0 0x7f70c406d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70e00d3750 0x7f70e01f1240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70e00b8400 0x7f70e01f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 --2- 172.21.15.60:0/3826488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70e00cfca0 0x7f70e00b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 >> 172.21.15.60:0/3826488887 conn(0x7f70e006b390 msgr2=0x7f70e00d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.243+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 shutdown_connections 2023-07-13T22:21:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.244+0000 7f70e6bff700 1 -- 172.21.15.60:0/3826488887 wait complete. 2023-07-13T22:21:12.231 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2023-07-13T22:21:12.648 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:20:54.038443+0000', 'modified': '2023-07-13T22:20:57.536915+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6835/453882022', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6833', 'nonce': 453882022}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 453882022}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:21:12.649 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:21:12.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-13T22:21:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.869+0000 7efffa7ff700 1 Processor -- start 2023-07-13T22:21:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.869+0000 7efffa7ff700 1 -- start start 2023-07-13T22:21:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.870+0000 7efffa7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efff40d3750 0x7efff40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efffa7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff40cd950 con 0x7efff40b8400 2023-07-13T22:21:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efffa7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff40d0ac0 con 0x7efff40cfad0 2023-07-13T22:21:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efffa7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff40cff50 con 0x7efff40d3750 2023-07-13T22:21:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efff37fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efff40d3750 0x7efff40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efff3fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efff40b8400 0x7efff40cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47812/0 (socket says 172.21.15.60:47812) 2023-07-13T22:21:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efff3fff700 1 -- 172.21.15.60:0/3178477884 learned_addr learned my addr 172.21.15.60:0/3178477884 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efff37fe700 1 -- 172.21.15.60:0/3178477884 >> v1:172.21.15.105:6789/0 conn(0x7efff40cfad0 legacy=0x7efff40d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efff37fe700 1 -- 172.21.15.60:0/3178477884 >> v1:172.21.15.60:6789/0 conn(0x7efff40b8400 legacy=0x7efff40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efff37fe700 1 -- 172.21.15.60:0/3178477884 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40b87d0 con 0x7efff40d3750 2023-07-13T22:21:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.871+0000 7efff37fe700 1 --2- 172.21.15.60:0/3178477884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 0x7efff40cf590 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7effe8009840 tx=0x7effe8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb3e29b98f92a71 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efff2ffd700 1 -- 172.21.15.60:0/3178477884 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8004230 con 0x7efff40d3750 2023-07-13T22:21:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efff2ffd700 1 -- 172.21.15.60:0/3178477884 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe8004390 con 0x7efff40d3750 2023-07-13T22:21:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efff2ffd700 1 -- 172.21.15.60:0/3178477884 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe80036a0 con 0x7efff40d3750 2023-07-13T22:21:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efffa7ff700 1 -- 172.21.15.60:0/3178477884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 msgr2=0x7efff40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efffa7ff700 1 --2- 172.21.15.60:0/3178477884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 0x7efff40cf590 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7effe8009840 tx=0x7effe8009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:12.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efffa7ff700 1 -- 172.21.15.60:0/3178477884 shutdown_connections 2023-07-13T22:21:12.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efffa7ff700 1 --2- 172.21.15.60:0/3178477884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 0x7efff40cf590 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efffa7ff700 1 -- 172.21.15.60:0/3178477884 >> 172.21.15.60:0/3178477884 conn(0x7efff406b390 msgr2=0x7efff406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efffa7ff700 1 -- 172.21.15.60:0/3178477884 shutdown_connections 2023-07-13T22:21:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.872+0000 7efffa7ff700 1 -- 172.21.15.60:0/3178477884 wait complete. 2023-07-13T22:21:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.874+0000 7efffa7ff700 1 Processor -- start 2023-07-13T22:21:12.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.874+0000 7efffa7ff700 1 -- start start 2023-07-13T22:21:12.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.875+0000 7efffa7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efff40b8400 0x7efff40cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.875+0000 7efffa7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff40cfad0 0x7efff40b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.875+0000 7efffa7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 0x7efff4150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.875+0000 7efffa7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40ca820 con 0x7efff40d3750 2023-07-13T22:21:12.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.875+0000 7efffa7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40c9c50 con 0x7efff40cfad0 2023-07-13T22:21:12.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.875+0000 7efffa7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff40cb1f0 con 0x7efff40b8400 2023-07-13T22:21:12.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff8d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 0x7efff4150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff3fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efff40b8400 0x7efff40cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff8d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 0x7efff4150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45876/0 (socket says 172.21.15.60:45876) 2023-07-13T22:21:12.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff8d9c700 1 -- 172.21.15.60:0/109807150 learned_addr learned my addr 172.21.15.60:0/109807150 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:12.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff37fe700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff40cfad0 0x7efff40b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff8d9c700 1 -- 172.21.15.60:0/109807150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efff40b8400 msgr2=0x7efff40cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff8d9c700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efff40b8400 0x7efff40cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff8d9c700 1 -- 172.21.15.60:0/109807150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff40cfad0 msgr2=0x7efff40b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff8d9c700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff40cfad0 0x7efff40b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff8d9c700 1 -- 172.21.15.60:0/109807150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe80094a0 con 0x7efff40d3750 2023-07-13T22:21:12.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff8d9c700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 0x7efff4150f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7effec00b970 tx=0x7effec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff0ff9700 1 -- 172.21.15.60:0/109807150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec00d860 con 0x7efff40d3750 2023-07-13T22:21:12.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff0ff9700 1 -- 172.21.15.60:0/109807150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec00d9c0 con 0x7efff40d3750 2023-07-13T22:21:12.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efff0ff9700 1 -- 172.21.15.60:0/109807150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec00dd70 con 0x7efff40d3750 2023-07-13T22:21:12.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.876+0000 7efffa7ff700 1 -- 172.21.15.60:0/109807150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff40c9ef0 con 0x7efff40d3750 2023-07-13T22:21:12.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.877+0000 7efffa7ff700 1 -- 172.21.15.60:0/109807150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff40ca4b0 con 0x7efff40d3750 2023-07-13T22:21:12.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.879+0000 7efff0ff9700 1 -- 172.21.15.60:0/109807150 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effec00e430 con 0x7efff40d3750 2023-07-13T22:21:12.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.880+0000 7efff0ff9700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effe006ace0 0x7effe006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.880+0000 7efff0ff9700 1 -- 172.21.15.60:0/109807150 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effec0895e0 con 0x7efff40d3750 2023-07-13T22:21:12.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.880+0000 7efff3fff700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effe006ace0 0x7effe006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.881+0000 7efff3fff700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effe006ace0 0x7effe006d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7effe4000f60 tx=0x7effe4007f10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.882+0000 7f000e1b2b80 1 Processor -- start 2023-07-13T22:21:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.882+0000 7f000e1b2b80 1 -- start start 2023-07-13T22:21:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d665f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d665fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 0x5576d664a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d65a9ea0 con 0x5576d6631dc0 2023-07-13T22:21:12.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d651d600 con 0x5576d662fc50 2023-07-13T22:21:12.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576d658c910 con 0x5576d662f880 2023-07-13T22:21:12.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d665fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d665fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45884/0 (socket says 172.21.15.60:45884) 2023-07-13T22:21:12.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff37fe700 1 -- 172.21.15.60:0/221278118 learned_addr learned my addr 172.21.15.60:0/221278118 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:12.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff8d9c700 1 --2- 172.21.15.60:0/221278118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 0x5576d664a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff3fff700 1 --2- 172.21.15.60:0/221278118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d665f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff37fe700 1 -- 172.21.15.60:0/221278118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 msgr2=0x5576d665f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff37fe700 1 --2- 172.21.15.60:0/221278118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d665f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff37fe700 1 -- 172.21.15.60:0/221278118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 msgr2=0x5576d664a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff37fe700 1 --2- 172.21.15.60:0/221278118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 0x5576d664a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.883+0000 7efff37fe700 1 -- 172.21.15.60:0/221278118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576d6379790 con 0x5576d6631dc0 2023-07-13T22:21:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.884+0000 7efff37fe700 1 --2- 172.21.15.60:0/221278118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d665fb20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efff406c720 tx=0x7effe8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959a3d992c1dbfdf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.884+0000 7effda7fc700 1 -- 172.21.15.60:0/221278118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8003850 con 0x5576d6631dc0 2023-07-13T22:21:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.884+0000 7effda7fc700 1 -- 172.21.15.60:0/221278118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe80039b0 con 0x5576d6631dc0 2023-07-13T22:21:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.884+0000 7effda7fc700 1 -- 172.21.15.60:0/221278118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8003d40 con 0x5576d6631dc0 2023-07-13T22:21:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.884+0000 7f000e1b2b80 1 -- 172.21.15.60:0/221278118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 msgr2=0x5576d665fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.884+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/221278118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d665fb20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efff406c720 tx=0x7effe8010000 comp rx=0 tx=0).stop 2023-07-13T22:21:12.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.885+0000 7f000e1b2b80 1 -- 172.21.15.60:0/221278118 shutdown_connections 2023-07-13T22:21:12.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.885+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/221278118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 0x5576d664a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.885+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/221278118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d665f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.885+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/221278118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d665fb20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.885+0000 7f000e1b2b80 1 -- 172.21.15.60:0/221278118 >> 172.21.15.60:0/221278118 conn(0x5576d662a3b0 msgr2=0x5576d6635550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.885+0000 7f000e1b2b80 1 -- 172.21.15.60:0/221278118 shutdown_connections 2023-07-13T22:21:12.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.885+0000 7f000e1b2b80 1 -- 172.21.15.60:0/221278118 wait complete. 2023-07-13T22:21:12.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.885+0000 7f000e1b2b80 1 -- start start 2023-07-13T22:21:12.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.886+0000 7f000e1b2b80 1 Processor -- start 2023-07-13T22:21:12.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d6674eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d662fc50 0x5576d6639980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d6631dc0 0x5576d668cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d6585be0 con 0x5576d662fc50 2023-07-13T22:21:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d65435b0 con 0x5576d6631dc0 2023-07-13T22:21:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576d6529910 con 0x5576d662f880 2023-07-13T22:21:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7f000e1b2b80 10 client.0 _ll_register_callbacks cb 0x5576d6541b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d662fc50 0x5576d6639980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d662fc50 0x5576d6639980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45890/0 (socket says 172.21.15.60:45890) 2023-07-13T22:21:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff37fe700 1 -- 172.21.15.60:0/2389008237 learned_addr learned my addr 172.21.15.60:0/2389008237 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff3fff700 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d6674eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff8d9c700 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d6631dc0 0x5576d668cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff37fe700 1 -- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 msgr2=0x5576d6674eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff37fe700 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d6674eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff37fe700 1 -- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d6631dc0 msgr2=0x5576d668cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff37fe700 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d6631dc0 0x5576d668cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.887+0000 7efff37fe700 1 -- 172.21.15.60:0/2389008237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff40b87d0 con 0x5576d662fc50 2023-07-13T22:21:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7efff37fe700 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d662fc50 0x5576d6639980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effe80090b0 tx=0x7effe80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3767951085394466 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7effd2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:21:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7effd2ffd700 1 -- 172.21.15.60:0/2389008237 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8006b10 con 0x5576d662fc50 2023-07-13T22:21:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7effd2ffd700 1 -- 172.21.15.60:0/2389008237 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effe802d070 con 0x5576d662fc50 2023-07-13T22:21:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2389008237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576d66279c0 con 0x5576d662fc50 2023-07-13T22:21:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7effd2ffd700 1 -- 172.21.15.60:0/2389008237 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7effe8011630 con 0x5576d662fc50 2023-07-13T22:21:12.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7f000e1b2b80 10 client.4662 fetch_fsmap learned FSMap version 8 2023-07-13T22:21:12.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2389008237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5576d64660a0 con 0x5576d662fc50 2023-07-13T22:21:12.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7effd2ffd700 1 -- 172.21.15.60:0/2389008237 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5576d66279c0 con 0x5576d662fc50 2023-07-13T22:21:12.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7effd2ffd700 1 -- 172.21.15.60:0/2389008237 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7effe8018070 con 0x5576d662fc50 2023-07-13T22:21:12.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7f000e1b2b80 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2023-07-13T22:21:12.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7f000e1b2b80 10 client.4662 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-13T22:21:12.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.888+0000 7f000e1b2b80 20 client.4662 populate_metadata read hostname 'smithi060' 2023-07-13T22:21:12.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.889+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] conn(0x5576d668d510 0x5576d66730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.889+0000 7f000e1b2b80 4 client.4662 mds_command: new command op to 4279 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:12.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.889+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2389008237 --> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576d6604170 con 0x5576d668d510 2023-07-13T22:21:12.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.889+0000 7efff3fff700 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] conn(0x5576d668d510 0x5576d66730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.889+0000 7efff3fff700 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] conn(0x5576d668d510 0x5576d66730b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395285bfcc4ca799 server_cookie=6a6cb944663dd41 in_seq=0 out_seq=0 2023-07-13T22:21:12.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.889+0000 7effd2ffd700 10 client.4662 ms_handle_connect on v2:172.21.15.105:6833/453882022 2023-07-13T22:21:12.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.890+0000 7effd2ffd700 1 -- 172.21.15.60:0/2389008237 <== mds.0 v2:172.21.15.105:6833/453882022 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5576d6604170 con 0x5576d668d510 2023-07-13T22:21:12.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.890+0000 7effd2ffd700 10 client.4662 handle_command_reply: tid=0 2023-07-13T22:21:12.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.890+0000 7f000e1b2b80 1 client.4662 shutdown 2023-07-13T22:21:12.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2389008237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d662fc50 msgr2=0x5576d6639980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d662fc50 0x5576d6639980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effe80090b0 tx=0x7effe80047c0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2389008237 shutdown_connections 2023-07-13T22:21:12.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] conn(0x5576d668d510 0x5576d66730b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d6631dc0 0x5576d668cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d6674eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2389008237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d662fc50 0x5576d6639980 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2389008237 >> 172.21.15.60:0/2389008237 conn(0x5576d662a3b0 msgr2=0x5576d665f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7effd2ffd700 0 client.4662 ms_handle_reset on v2:172.21.15.105:6833/453882022 2023-07-13T22:21:12.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2389008237 shutdown_connections 2023-07-13T22:21:12.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2389008237 wait complete. 2023-07-13T22:21:12.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.891+0000 7f000e1b2b80 20 client.4662 trim_cache size 0 max 16384 2023-07-13T22:21:12.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.909+0000 7f000e1b2b80 1 Processor -- start 2023-07-13T22:21:12.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.910+0000 7f000e1b2b80 1 -- start start 2023-07-13T22:21:12.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662f880 0x5576d66317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662fc50 0x5576d664aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d637a390 con 0x5576d6631dc0 2023-07-13T22:21:12.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d664afb0 con 0x5576d662f880 2023-07-13T22:21:12.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576d669d6e0 con 0x5576d662fc50 2023-07-13T22:21:12.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff8d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff8d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45892/0 (socket says 172.21.15.60:45892) 2023-07-13T22:21:12.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff8d9c700 1 -- 172.21.15.60:0/763937341 learned_addr learned my addr 172.21.15.60:0/763937341 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:12.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff37fe700 1 --2- 172.21.15.60:0/763937341 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662fc50 0x5576d664aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff3fff700 1 --2- 172.21.15.60:0/763937341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662f880 0x5576d66317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff8d9c700 1 -- 172.21.15.60:0/763937341 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662fc50 msgr2=0x5576d664aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff8d9c700 1 --2- 172.21.15.60:0/763937341 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662fc50 0x5576d664aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff8d9c700 1 -- 172.21.15.60:0/763937341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662f880 msgr2=0x5576d66317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff8d9c700 1 --2- 172.21.15.60:0/763937341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662f880 0x5576d66317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.911+0000 7efff8d9c700 1 -- 172.21.15.60:0/763937341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4155c70 con 0x5576d6631dc0 2023-07-13T22:21:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7efff8d9c700 1 --2- 172.21.15.60:0/763937341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667de80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7effec054840 tx=0x7effec0547b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d93a2de591b7e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7effdaffd700 1 -- 172.21.15.60:0/763937341 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec0566e0 con 0x5576d6631dc0 2023-07-13T22:21:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7effdaffd700 1 -- 172.21.15.60:0/763937341 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec05e0b0 con 0x5576d6631dc0 2023-07-13T22:21:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7effdaffd700 1 -- 172.21.15.60:0/763937341 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec057830 con 0x5576d6631dc0 2023-07-13T22:21:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7f000e1b2b80 1 -- 172.21.15.60:0/763937341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 msgr2=0x5576d667de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/763937341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667de80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7effec054840 tx=0x7effec0547b0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7f000e1b2b80 1 -- 172.21.15.60:0/763937341 shutdown_connections 2023-07-13T22:21:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/763937341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667de80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/763937341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662f880 0x5576d66317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/763937341 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662fc50 0x5576d664aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7f000e1b2b80 1 -- 172.21.15.60:0/763937341 >> 172.21.15.60:0/763937341 conn(0x5576d662a3b0 msgr2=0x5576d662b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7f000e1b2b80 1 -- 172.21.15.60:0/763937341 shutdown_connections 2023-07-13T22:21:12.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.912+0000 7f000e1b2b80 1 -- 172.21.15.60:0/763937341 wait complete. 2023-07-13T22:21:12.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.913+0000 7f000e1b2b80 1 -- start start 2023-07-13T22:21:12.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.913+0000 7f000e1b2b80 1 Processor -- start 2023-07-13T22:21:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d66e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 0x5576d6636b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7f000e1b2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d66e28f0 con 0x5576d6631dc0 2023-07-13T22:21:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d66602e0 con 0x5576d662fc50 2023-07-13T22:21:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7f000e1b2b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576d6660530 con 0x5576d662f880 2023-07-13T22:21:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7f000e1b2b80 10 client.0 _ll_register_callbacks cb 0x7effe8006ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7efff8d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7efff8d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45900/0 (socket says 172.21.15.60:45900) 2023-07-13T22:21:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.915+0000 7efff8d9c700 1 -- 172.21.15.60:0/2516660415 learned_addr learned my addr 172.21.15.60:0/2516660415 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7efff3fff700 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d66e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7efff37fe700 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 0x5576d6636b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7efff8d9c700 1 -- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 msgr2=0x5576d66e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7efff8d9c700 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d66e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7efff8d9c700 1 -- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 msgr2=0x5576d6636b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7efff8d9c700 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 0x5576d6636b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7efff8d9c700 1 -- 172.21.15.60:0/2516660415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff40ca4b0 con 0x5576d6631dc0 2023-07-13T22:21:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7efff37fe700 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 0x5576d6636b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7efff8d9c700 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667c990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7effec00b940 tx=0x7effec004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbe57b2d5e9b7b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7effd2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:21:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7effd2ffd700 1 -- 172.21.15.60:0/2516660415 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec057a30 con 0x5576d6631dc0 2023-07-13T22:21:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7effd2ffd700 1 -- 172.21.15.60:0/2516660415 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effec060200 con 0x5576d6631dc0 2023-07-13T22:21:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2516660415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effe80039b0 con 0x5576d6631dc0 2023-07-13T22:21:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7effd2ffd700 1 -- 172.21.15.60:0/2516660415 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7effec0603c0 con 0x5576d6631dc0 2023-07-13T22:21:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7f000e1b2b80 10 client.4668 fetch_fsmap learned FSMap version 8 2023-07-13T22:21:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.916+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2516660415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5576d654bb10 con 0x5576d6631dc0 2023-07-13T22:21:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.917+0000 7effd2ffd700 1 -- 172.21.15.60:0/2516660415 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe80039b0 con 0x5576d6631dc0 2023-07-13T22:21:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.917+0000 7effd2ffd700 1 -- 172.21.15.60:0/2516660415 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7effec015020 con 0x5576d6631dc0 2023-07-13T22:21:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.917+0000 7f000e1b2b80 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2023-07-13T22:21:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.917+0000 7f000e1b2b80 10 client.4668 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-13T22:21:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.917+0000 7f000e1b2b80 20 client.4668 populate_metadata read hostname 'smithi060' 2023-07-13T22:21:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.917+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] conn(0x5576d6694070 0x5576d6694440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.917+0000 7f000e1b2b80 4 client.4668 mds_command: new command op to 4279 tid=0[{"prefix": "status"}] 2023-07-13T22:21:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.917+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2516660415 --> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5576d657ed20 con 0x5576d6694070 2023-07-13T22:21:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.917+0000 7efff3fff700 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] conn(0x5576d6694070 0x5576d6694440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7efff3fff700 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] conn(0x5576d6694070 0x5576d6694440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5971f7dfaccaa919 server_cookie=30587b00c085c62e in_seq=0 out_seq=0 2023-07-13T22:21:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7effd2ffd700 10 client.4668 ms_handle_connect on v2:172.21.15.105:6833/453882022 2023-07-13T22:21:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7effd2ffd700 1 -- 172.21.15.60:0/2516660415 <== mds.0 v2:172.21.15.105:6833/453882022 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5576d657ed20 con 0x5576d6694070 2023-07-13T22:21:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7effd2ffd700 10 client.4668 handle_command_reply: tid=0 2023-07-13T22:21:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 client.4668 shutdown 2023-07-13T22:21:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2516660415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 msgr2=0x5576d667c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667c990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7effec00b940 tx=0x7effec004b00 comp rx=0 tx=0).stop 2023-07-13T22:21:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2516660415 shutdown_connections 2023-07-13T22:21:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:6833/453882022,v1:172.21.15.105:6835/453882022] conn(0x5576d6694070 0x5576d6694440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576d6631dc0 0x5576d667c990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7effd2ffd700 0 client.4668 ms_handle_reset on v2:172.21.15.105:6833/453882022 2023-07-13T22:21:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5576d662f880 0x5576d66e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 --2- 172.21.15.60:0/2516660415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d662fc50 0x5576d6636b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2516660415 >> 172.21.15.60:0/2516660415 conn(0x5576d662a3b0 msgr2=0x5576d667de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2516660415 shutdown_connections 2023-07-13T22:21:12.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.918+0000 7f000e1b2b80 1 -- 172.21.15.60:0/2516660415 wait complete. 2023-07-13T22:21:12.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7f000e1b2b80 20 client.4668 trim_cache size 0 max 16384 2023-07-13T22:21:12.915 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:21:12.915 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:21:12.916 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:21:12.916 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4279, 2023-07-13T22:21:12.916 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:21:12.916 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:21:12.916 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:21:12.916 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 18.404301594, 2023-07-13T22:21:12.917 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 8, 2023-07-13T22:21:12.917 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 21, 2023-07-13T22:21:12.917 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 21, 2023-07-13T22:21:12.917 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 25.003187743000002 2023-07-13T22:21:12.917 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:21:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 -- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effe006ace0 msgr2=0x7effe006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effe006ace0 0x7effe006d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7effe4000f60 tx=0x7effe4007f10 comp rx=0 tx=0).stop 2023-07-13T22:21:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 -- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 msgr2=0x7efff4150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 0x7efff4150f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7effec00b970 tx=0x7effec00bc80 comp rx=0 tx=0).stop 2023-07-13T22:21:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 -- 172.21.15.60:0/109807150 shutdown_connections 2023-07-13T22:21:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effe006ace0 0x7effe006d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efff40d3750 0x7efff4150f00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efff40b8400 0x7efff40cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 --2- 172.21.15.60:0/109807150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff40cfad0 0x7efff40b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 -- 172.21.15.60:0/109807150 >> 172.21.15.60:0/109807150 conn(0x7efff406b390 msgr2=0x7efff415fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 -- 172.21.15.60:0/109807150 shutdown_connections 2023-07-13T22:21:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:12.919+0000 7efffa7ff700 1 -- 172.21.15.60:0/109807150 wait complete. 2023-07-13T22:21:12.922 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-07-13T22:21:12.945 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-07-13T22:21:12.945 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-07-13T22:21:12.946 INFO:tasks.ceph_fuse:client.0 config is {} 2023-07-13T22:21:12.946 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-13T22:21:12.946 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:21:12.947 INFO:tasks.ceph_fuse:client.2 config is {} 2023-07-13T22:21:12.947 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-13T22:21:12.947 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-13T22:21:12.947 INFO:tasks.ceph_fuse:client.1 config is {} 2023-07-13T22:21:12.948 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-13T22:21:12.948 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:21:12.948 INFO:tasks.ceph_fuse:client.3 config is {} 2023-07-13T22:21:12.948 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-13T22:21:12.948 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-13T22:21:12.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:12.949 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:21:13.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:13.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2023-07-13T22:21:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2023-07-13T22:21:13.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:21:13.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:13.083 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:21:13.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:13.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2023-07-13T22:21:13.206 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2023-07-13T22:21:13.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:21:13.210 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-07-13T22:21:13.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:21:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.429+0000 7fa70a7ff700 1 Processor -- start 2023-07-13T22:21:13.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.429+0000 7fa70a7ff700 1 -- start start 2023-07-13T22:21:13.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.430+0000 7fa70a7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa704159170 0x7fa7040c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa70a7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70415d970 con 0x7fa704070980 2023-07-13T22:21:13.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa70a7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70415cd60 con 0x7fa7040c5040 2023-07-13T22:21:13.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa70a7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70415d040 con 0x7fa704159170 2023-07-13T22:21:13.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa703fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa704159170 0x7fa7040c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa708d9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa704070980 0x7fa7040c9c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47818/0 (socket says 172.21.15.60:47818) 2023-07-13T22:21:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa708d9c700 1 -- 172.21.15.60:0/3093671636 learned_addr learned my addr 172.21.15.60:0/3093671636 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa7037fe700 1 --1- 172.21.15.60:0/3093671636 >> v1:172.21.15.105:6789/0 conn(0x7fa7040c5040 0x7fa704070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:33134/0 (socket says 172.21.15.60:33134) 2023-07-13T22:21:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa702ffd700 1 -- 172.21.15.60:0/3093671636 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890018844 0 0) 0x7fa70415d970 con 0x7fa704070980 2023-07-13T22:21:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa702ffd700 1 -- 172.21.15.60:0/3093671636 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6ec0035b0 con 0x7fa704070980 2023-07-13T22:21:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa703fff700 1 -- 172.21.15.60:0/3093671636 >> v1:172.21.15.105:6789/0 conn(0x7fa7040c5040 legacy=0x7fa704070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa702ffd700 1 -- 172.21.15.60:0/3093671636 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311855054 0 0) 0x7fa70415cd60 con 0x7fa7040c5040 2023-07-13T22:21:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa703fff700 1 -- 172.21.15.60:0/3093671636 >> v1:172.21.15.60:6789/0 conn(0x7fa704070980 legacy=0x7fa7040c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.431+0000 7fa703fff700 1 -- 172.21.15.60:0/3093671636 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70415d320 con 0x7fa704159170 2023-07-13T22:21:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.432+0000 7fa703fff700 1 --2- 172.21.15.60:0/3093671636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 0x7fa7040c4b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400bae0 tx=0x7fa6f400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57aa6ec18e2c1be server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.432+0000 7fa702ffd700 1 -- 172.21.15.60:0/3093671636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400dc70 con 0x7fa704159170 2023-07-13T22:21:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.432+0000 7fa702ffd700 1 -- 172.21.15.60:0/3093671636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f400ddd0 con 0x7fa704159170 2023-07-13T22:21:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.432+0000 7fa702ffd700 1 -- 172.21.15.60:0/3093671636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4010690 con 0x7fa704159170 2023-07-13T22:21:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.432+0000 7fa70a7ff700 1 -- 172.21.15.60:0/3093671636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 msgr2=0x7fa7040c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.432+0000 7fa70a7ff700 1 --2- 172.21.15.60:0/3093671636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 0x7fa7040c4b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400bae0 tx=0x7fa6f400bdf0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.433+0000 7fa70a7ff700 1 -- 172.21.15.60:0/3093671636 shutdown_connections 2023-07-13T22:21:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.433+0000 7fa70a7ff700 1 --2- 172.21.15.60:0/3093671636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 0x7fa7040c4b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.433+0000 7fa70a7ff700 1 -- 172.21.15.60:0/3093671636 >> 172.21.15.60:0/3093671636 conn(0x7fa7040cf410 msgr2=0x7fa7040cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:13.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.433+0000 7fa70a7ff700 1 -- 172.21.15.60:0/3093671636 shutdown_connections 2023-07-13T22:21:13.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.433+0000 7fa70a7ff700 1 -- 172.21.15.60:0/3093671636 wait complete. 2023-07-13T22:21:13.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.435+0000 7fa70a7ff700 1 Processor -- start 2023-07-13T22:21:13.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.435+0000 7fa70a7ff700 1 -- start start 2023-07-13T22:21:13.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.436+0000 7fa70a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa704070980 0x7fa7040c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.436+0000 7fa70a7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7040c5040 0x7fa70413ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.436+0000 7fa70a7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 0x7fa7041599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.436+0000 7fa70a7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa704159f10 con 0x7fa704159170 2023-07-13T22:21:13.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.436+0000 7fa70a7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa704150ea0 con 0x7fa7040c5040 2023-07-13T22:21:13.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.436+0000 7fa70a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa704150880 con 0x7fa704070980 2023-07-13T22:21:13.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa703fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa704070980 0x7fa7040c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa7037fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7040c5040 0x7fa70413ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa708d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 0x7fa7041599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa703fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa704070980 0x7fa7040c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:39922/0 (socket says 172.21.15.60:39922) 2023-07-13T22:21:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa703fff700 1 -- 172.21.15.60:0/961707951 learned_addr learned my addr 172.21.15.60:0/961707951 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa703fff700 1 -- 172.21.15.60:0/961707951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7040c5040 msgr2=0x7fa70413ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa703fff700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7040c5040 0x7fa70413ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa703fff700 1 -- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 msgr2=0x7fa7041599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa703fff700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 0x7fa7041599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa703fff700 1 -- 172.21.15.60:0/961707951 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f400b740 con 0x7fa704070980 2023-07-13T22:21:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa708d9c700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 0x7fa7041599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:13.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.437+0000 7fa703fff700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa704070980 0x7fa7040c7fb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa70406b790 tx=0x7fa6f40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:13.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.438+0000 7fa700ff9700 1 -- 172.21.15.60:0/961707951 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f401e040 con 0x7fa704070980 2023-07-13T22:21:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.438+0000 7fa700ff9700 1 -- 172.21.15.60:0/961707951 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f4019620 con 0x7fa704070980 2023-07-13T22:21:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.438+0000 7fa700ff9700 1 -- 172.21.15.60:0/961707951 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40199d0 con 0x7fa704070980 2023-07-13T22:21:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.438+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7040c5860 con 0x7fa704070980 2023-07-13T22:21:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.438+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7040c5de0 con 0x7fa704070980 2023-07-13T22:21:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.439+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa704071440 con 0x7fa704070980 2023-07-13T22:21:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.440+0000 7fa700ff9700 1 -- 172.21.15.60:0/961707951 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa6f4019b30 con 0x7fa704070980 2023-07-13T22:21:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.441+0000 7fa700ff9700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f006ace0 0x7fa6f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.441+0000 7fa700ff9700 1 -- 172.21.15.60:0/961707951 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa6f408cff0 con 0x7fa704070980 2023-07-13T22:21:13.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.441+0000 7fa7037fe700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f006ace0 0x7fa6f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.442+0000 7fa7037fe700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f006ace0 0x7fa6f006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa7040cbcf0 tx=0x7fa6f8007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:13.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.446+0000 7fa700ff9700 1 -- 172.21.15.60:0/961707951 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa6f40bf330 con 0x7fa704070980 2023-07-13T22:21:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.696+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa704070740 con 0x7fa704070980 2023-07-13T22:21:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.697+0000 7fa700ff9700 1 -- 172.21.15.60:0/961707951 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa6f4029070 con 0x7fa704070980 2023-07-13T22:21:13.682 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:21:13.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f006ace0 msgr2=0x7fa6f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f006ace0 0x7fa6f006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa7040cbcf0 tx=0x7fa6f8007de0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa704070980 msgr2=0x7fa7040c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa704070980 0x7fa7040c7fb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa70406b790 tx=0x7fa6f40047c0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 shutdown_connections 2023-07-13T22:21:13.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f006ace0 0x7fa6f006d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa704159170 0x7fa7041599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa704070980 0x7fa7040c7fb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 --2- 172.21.15.60:0/961707951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7040c5040 0x7fa70413ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 >> 172.21.15.60:0/961707951 conn(0x7fa7040cf410 msgr2=0x7fa704069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:13.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 shutdown_connections 2023-07-13T22:21:13.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:13.699+0000 7fa70a7ff700 1 -- 172.21.15.60:0/961707951 wait complete. 2023-07-13T22:21:13.702 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:21:13.702 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:21:13.702 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:21:13.703 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:21:13.703 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:21:13.703 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:21:13.703 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:21:13.703 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:21:13.704 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:21:13.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:13.704 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:21:13.750 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:21:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:21:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:21:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:21:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:13.751 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:21:13.752 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:21:13.752 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:21:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:21:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:21:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41700sec preferred_lft 41700sec 2023-07-13T22:21:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:21:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:13.753 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:21:13.754 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-13T22:21:13.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:13.755 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:21:13.755 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:21:13.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2023-07-13T22:21:13.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2023-07-13T22:21:13.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2023-07-13T22:21:13.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-13T22:21:13.756 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:21:14.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:14.219 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:21:14.326 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:21:14.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:14.330 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:21:14.396 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:21:14.397 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:21:14.397 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:21:14.397 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:21:14.397 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:21:14.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:14.398 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:21:14.399 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:21:14.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:21:14.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:21:14.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:21:14.400 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:21:15.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:15.106 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:21:15.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:15.154 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2023-07-13T22:21:15.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:15.201 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:21:15.202 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:21:15.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:21:15.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-13T22:21:15.202 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:21:15.348 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-13T22:21:15.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:15.349 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:21:15.349 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:21:15.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-13T22:21:15.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-13T22:21:15.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-13T22:21:15.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-13T22:21:15.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-13T22:21:15.351 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:21:15.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:15.631 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:21:15.631 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:21:15.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:21:15.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2023-07-13T22:21:15.632 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:21:15.806 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:21:15.807 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:21:15.807 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:21:15.853 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:21:15.853 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:21:15.853 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:21:15.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:21:16.049 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:21:16.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:21:16.120 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:21:16.122 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:21:16.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:16.123 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:21:16.169 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:21:16.170 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:21:16.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:21:16.286 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:21:16.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:21:16.357 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:21:16.360 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:21:16.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:16.361 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:21:16.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:21:16.405+0000 7f696c60d5c0 -1 init, newargv = 0x557b2da6d110 newargc=15 2023-07-13T22:21:16.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[120832]: starting ceph client 2023-07-13T22:21:16.400 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[120832]: starting fuse 2023-07-13T22:21:16.411 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:21:16.411 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:21:16.411 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:21:16.412 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:21:16.412 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:21:16.412 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:21:16.413 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:21:16.413 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:21:16.413 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:21:16.413 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:21:16.413 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:21:16.414 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:21:16.414 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:21:16.414 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:21:16.415 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:21:16.415 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:21:16.415 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:21:16.415 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:21:16.416 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:21:16.416 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:21:16.416 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:21:16.416 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:21:16.417 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:21:16.417 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:21:16.417 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:21:16.417 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:21:16.418 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:21:16.418 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:21:16.418 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:21:16.599 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.120832.asok 2023-07-13T22:21:16.775 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.120832.asok 2023-07-13T22:21:16.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:16.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.120832.asok status 2023-07-13T22:21:17.007 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:21:17.008 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:21:17.008 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:21:17.008 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:21:17.008 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:21:17.008 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:21:17.008 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:21:17.009 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "120832", 2023-07-13T22:21:17.009 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:21:17.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:21:17.009 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:21:17.009 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:21:17.009 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4683, 2023-07-13T22:21:17.009 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:21:17.009 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:21:17.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:21:17.010 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 4683 2023-07-13T22:21:17.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:21:17.010 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:21:17.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:21:17.010 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:21:17.010 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1795789191 2023-07-13T22:21:17.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:21:17.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:21:17.011 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:21:17.011 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:21:17.011 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:21:17.011 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1795789191 2023-07-13T22:21:17.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:21:17.011 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.4683 192.168.0.1:0/1795789191", 2023-07-13T22:21:17.011 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/1795789191", 2023-07-13T22:21:17.012 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:21:17.012 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 8, 2023-07-13T22:21:17.012 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 21, 2023-07-13T22:21:17.012 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:21:17.012 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:21:17.012 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:21:17.012 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:21:17.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:21:17.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.240+0000 7f4d72fff700 1 Processor -- start 2023-07-13T22:21:17.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.241+0000 7f4d72fff700 1 -- start start 2023-07-13T22:21:17.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.242+0000 7f4d72fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d6c0b8400 0x7f4d6c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.242+0000 7f4d72fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 0x7f4d6c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.242+0000 7f4d72fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c0cff20 con 0x7f4d6c0cfad0 2023-07-13T22:21:17.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.242+0000 7f4d72fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6c0d0ac0 con 0x7f4d6c0b8400 2023-07-13T22:21:17.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.242+0000 7f4d72fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0b87d0 con 0x7f4d6c0d3750 2023-07-13T22:21:17.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.242+0000 7f4d70d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d6c0b8400 0x7f4d6c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.242+0000 7f4d70d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d6c0b8400 0x7f4d6c0cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54336/0 (socket says 172.21.15.60:54336) 2023-07-13T22:21:17.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.242+0000 7f4d70d9b700 1 -- 172.21.15.60:0/125558157 learned_addr learned my addr 172.21.15.60:0/125558157 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:17.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.242+0000 7f4d6bfff700 1 --2- 172.21.15.60:0/125558157 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 0x7f4d6c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.243+0000 7f4d70d9b700 1 -- 172.21.15.60:0/125558157 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 msgr2=0x7f4d6c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:17.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.243+0000 7f4d70d9b700 1 --2- 172.21.15.60:0/125558157 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 0x7f4d6c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.243+0000 7f4d70d9b700 1 -- 172.21.15.60:0/125558157 >> v1:172.21.15.60:6789/0 conn(0x7f4d6c0cfad0 legacy=0x7f4d6c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.243+0000 7f4d70d9b700 1 -- 172.21.15.60:0/125558157 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0b4eb0 con 0x7f4d6c0b8400 2023-07-13T22:21:17.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.243+0000 7f4d70d9b700 1 --2- 172.21.15.60:0/125558157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 0x7f4d6c0cd240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4d600098f0 tx=0x7f4d60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5171fadb5e523017 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:17.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.243+0000 7f4d6b7fe700 1 -- 172.21.15.60:0/125558157 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60011830 con 0x7f4d6c0b8400 2023-07-13T22:21:17.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.243+0000 7f4d6b7fe700 1 -- 172.21.15.60:0/125558157 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d60011990 con 0x7f4d6c0b8400 2023-07-13T22:21:17.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.243+0000 7f4d6b7fe700 1 -- 172.21.15.60:0/125558157 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60011d20 con 0x7f4d6c0b8400 2023-07-13T22:21:17.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.244+0000 7f4d72fff700 1 -- 172.21.15.60:0/125558157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 msgr2=0x7f4d6c0cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.244+0000 7f4d72fff700 1 --2- 172.21.15.60:0/125558157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 0x7f4d6c0cd240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4d600098f0 tx=0x7f4d60009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:17.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.244+0000 7f4d72fff700 1 -- 172.21.15.60:0/125558157 shutdown_connections 2023-07-13T22:21:17.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.244+0000 7f4d72fff700 1 --2- 172.21.15.60:0/125558157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 0x7f4d6c0cd240 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.244+0000 7f4d72fff700 1 --2- 172.21.15.60:0/125558157 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 0x7f4d6c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.244+0000 7f4d72fff700 1 -- 172.21.15.60:0/125558157 >> 172.21.15.60:0/125558157 conn(0x7f4d6c06b390 msgr2=0x7f4d6c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:17.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.244+0000 7f4d72fff700 1 -- 172.21.15.60:0/125558157 shutdown_connections 2023-07-13T22:21:17.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.244+0000 7f4d72fff700 1 -- 172.21.15.60:0/125558157 wait complete. 2023-07-13T22:21:17.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.246+0000 7f4d72fff700 1 Processor -- start 2023-07-13T22:21:17.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.246+0000 7f4d72fff700 1 -- start start 2023-07-13T22:21:17.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d72fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 0x7f4d6c153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d72fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d6c0cfad0 0x7f4d6c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d72fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 0x7f4d6c0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d72fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0c66b0 con 0x7f4d6c0b8400 2023-07-13T22:21:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d72fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0c9ff0 con 0x7f4d6c0cfad0 2023-07-13T22:21:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d72fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0cacc0 con 0x7f4d6c0d3750 2023-07-13T22:21:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d70d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 0x7f4d6c153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d70d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 0x7f4d6c153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54350/0 (socket says 172.21.15.60:54350) 2023-07-13T22:21:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d70d9b700 1 -- 172.21.15.60:0/1665625331 learned_addr learned my addr 172.21.15.60:0/1665625331 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d7159c700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 0x7f4d6c0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d6bfff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d6c0cfad0 0x7f4d6c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d70d9b700 1 -- 172.21.15.60:0/1665625331 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 msgr2=0x7f4d6c0c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d70d9b700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 0x7f4d6c0c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d70d9b700 1 -- 172.21.15.60:0/1665625331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d6c0cfad0 msgr2=0x7f4d6c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d70d9b700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d6c0cfad0 0x7f4d6c153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d70d9b700 1 -- 172.21.15.60:0/1665625331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60009550 con 0x7f4d6c0b8400 2023-07-13T22:21:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.247+0000 7f4d6bfff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d6c0cfad0 0x7f4d6c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.248+0000 7f4d70d9b700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 0x7f4d6c153670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d60014040 tx=0x7f4d60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.248+0000 7f4d697fa700 1 -- 172.21.15.60:0/1665625331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60004590 con 0x7f4d6c0b8400 2023-07-13T22:21:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.248+0000 7f4d697fa700 1 -- 172.21.15.60:0/1665625331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6001e070 con 0x7f4d6c0b8400 2023-07-13T22:21:17.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.248+0000 7f4d697fa700 1 -- 172.21.15.60:0/1665625331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d600038e0 con 0x7f4d6c0b8400 2023-07-13T22:21:17.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.248+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c0cafa0 con 0x7f4d6c0b8400 2023-07-13T22:21:17.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.248+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c0c6900 con 0x7f4d6c0b8400 2023-07-13T22:21:17.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.249+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c0c9910 con 0x7f4d6c0b8400 2023-07-13T22:21:17.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.250+0000 7f4d697fa700 1 -- 172.21.15.60:0/1665625331 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d60003a40 con 0x7f4d6c0b8400 2023-07-13T22:21:17.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.252+0000 7f4d697fa700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d5006ad10 0x7f4d5006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.252+0000 7f4d697fa700 1 -- 172.21.15.60:0/1665625331 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d6008a3b0 con 0x7f4d6c0b8400 2023-07-13T22:21:17.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.252+0000 7f4d6bfff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d5006ad10 0x7f4d5006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.253+0000 7f4d6bfff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d5006ad10 0x7f4d5006d1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c154900 tx=0x7f4d58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:17.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.257+0000 7f4d697fa700 1 -- 172.21.15.60:0/1665625331 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4d60055450 con 0x7f4d6c0b8400 2023-07-13T22:21:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.511+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d6c0cf590 con 0x7f4d6c0b8400 2023-07-13T22:21:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.512+0000 7f4d697fa700 1 -- 172.21.15.60:0/1665625331 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d60058a70 con 0x7f4d6c0b8400 2023-07-13T22:21:17.497 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:21:17.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d5006ad10 msgr2=0x7f4d5006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d5006ad10 0x7f4d5006d1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c154900 tx=0x7f4d58008040 comp rx=0 tx=0).stop 2023-07-13T22:21:17.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 msgr2=0x7f4d6c153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 0x7f4d6c153670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d60014040 tx=0x7f4d60004750 comp rx=0 tx=0).stop 2023-07-13T22:21:17.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 shutdown_connections 2023-07-13T22:21:17.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d5006ad10 0x7f4d5006d1c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d6c0b8400 0x7f4d6c153670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d6c0d3750 0x7f4d6c0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 --2- 172.21.15.60:0/1665625331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d6c0cfad0 0x7f4d6c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 >> 172.21.15.60:0/1665625331 conn(0x7f4d6c06b390 msgr2=0x7f4d6c15bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:17.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 shutdown_connections 2023-07-13T22:21:17.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:17.514+0000 7f4d72fff700 1 -- 172.21.15.60:0/1665625331 wait complete. 2023-07-13T22:21:17.517 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:21:17.517 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:21:17.517 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:21:17.518 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-07-13T22:21:17.518 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-13T22:21:17.518 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:21:17.518 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:21:17.518 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:21:17.519 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-07-13T22:21:17.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:17.519 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:21:17.566 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:21:17.566 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:21:17.566 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:21:17.566 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:21:17.568 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:21:17.568 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:17.568 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:21:17.568 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:21:17.568 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41696sec preferred_lft 41696sec 2023-07-13T22:21:17.568 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:21:17.568 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:17.568 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:17.569 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:17.569 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:21:17.569 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:21:17.569 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:17.569 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:21:17.569 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:17.569 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:21:17.570 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:17.570 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:21:17.570 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:21:17.570 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link tentative 2023-07-13T22:21:17.570 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:17.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:17.571 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:21:17.641 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:21:17.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:17.644 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:21:17.693 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:21:17.693 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:21:17.693 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:21:17.694 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:21:17.694 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:21:17.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:17.695 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:21:17.695 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:21:17.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:21:17.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:21:17.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:21:17.696 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:21:17.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:17.868 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:21:17.914 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:21:17.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:17.915 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2023-07-13T22:21:17.961 INFO:teuthology.orchestra.run.smithi060.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-07-13T22:21:17.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:17.963 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:21:17.963 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:21:17.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:21:17.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-07-13T22:21:17.964 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:21:18.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:18.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-13T22:21:18.179 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:21:18.179 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:21:18.179 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:21:18.180 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:18.180 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:21:18.180 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:18.180 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:21:18.180 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 8a:24:66:2e:13:5e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-13T22:21:18.180 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-13T22:21:18.180 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:18.181 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8824:66ff:fe2e:135e/64 scope link 2023-07-13T22:21:18.181 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:18.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:18.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-13T22:21:18.254 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 8a:24:66:2e:13:5e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-13T22:21:18.256 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-13T22:21:18.256 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:18.256 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8824:66ff:fe2e:135e/64 scope link 2023-07-13T22:21:18.256 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:18.258 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-07-13T22:21:18.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:18.258 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:21:18.258 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:21:18.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-07-13T22:21:18.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-13T22:21:18.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-07-13T22:21:18.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-07-13T22:21:18.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-07-13T22:21:18.260 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:21:18.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:18.515 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:21:18.515 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:21:18.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.1 up 2023-07-13T22:21:18.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.1 master ceph-brx 2023-07-13T22:21:18.516 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:21:18.693 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-07-13T22:21:18.693 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:21:18.693 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-07-13T22:21:18.739 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-07-13T22:21:18.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:21:18.740 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-07-13T22:21:18.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:21:18.895 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:21:18.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:21:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:21:18.967 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:21:18.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:18.968 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:21:19.014 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:21:19.014 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-07-13T22:21:19.015 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-07-13T22:21:19.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:21:19.092 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:21:19.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:21:19.165 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:21:19.167 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:21:19.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:19.169 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:21:19.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi060.stderr:2023-07-13T22:21:19.212+0000 7fd2f3e565c0 -1 init, newargv = 0x564494c4ada0 newargc=15 2023-07-13T22:21:19.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi060.stderr:ceph-fuse[121556]: starting ceph client 2023-07-13T22:21:19.206 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi060.stderr:ceph-fuse[121556]: starting fuse 2023-07-13T22:21:19.221 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:21:19.221 INFO:teuthology.orchestra.run.smithi060.stdout:45 2023-07-13T22:21:19.221 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 45] 2023-07-13T22:21:19.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:21:19.222 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:21:19.222 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:21:19.223 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:21:19.223 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:21:19.223 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:21:19.223 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:21:19.224 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-07-13T22:21:19.224 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:21:19.224 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-07-13T22:21:19.224 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:21:19.224 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:21:19.225 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:21:19.225 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:21:19.225 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:21:19.225 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:21:19.226 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:21:19.226 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:21:19.226 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:21:19.226 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:21:19.226 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:21:19.227 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:21:19.227 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:21:19.227 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:21:19.227 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:21:19.228 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.2")) 2023-07-13T22:21:19.228 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:21:19.409 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.2.121556.asok 2023-07-13T22:21:19.586 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.121556.asok 2023-07-13T22:21:19.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:19.587 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.121556.asok status 2023-07-13T22:21:19.819 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:21:19.819 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:21:19.820 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:21:19.820 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:21:19.820 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "2", 2023-07-13T22:21:19.820 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:21:19.820 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-07-13T22:21:19.820 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "121556", 2023-07-13T22:21:19.820 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:21:19.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:21:19.821 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:21:19.821 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:21:19.821 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4675, 2023-07-13T22:21:19.821 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:21:19.821 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:21:19.821 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:21:19.821 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 4675 2023-07-13T22:21:19.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:21:19.822 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:21:19.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:21:19.822 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2023-07-13T22:21:19.822 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2126494314 2023-07-13T22:21:19.822 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:21:19.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:21:19.822 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:21:19.823 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:21:19.823 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:0", 2023-07-13T22:21:19.823 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2126494314 2023-07-13T22:21:19.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:21:19.823 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.4675 172.21.15.60:0/2126494314", 2023-07-13T22:21:19.823 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "172.21.15.60:0/2126494314", 2023-07-13T22:21:19.823 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:21:19.823 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 8, 2023-07-13T22:21:19.824 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 21, 2023-07-13T22:21:19.824 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:21:19.824 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:21:19.824 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:21:19.824 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:21:19.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:21:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.050+0000 7ffbb9dff700 1 Processor -- start 2023-07-13T22:21:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.050+0000 7ffbb9dff700 1 -- start start 2023-07-13T22:21:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb9dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffbb40b8570 0x7ffbb40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb9dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40d35c0 0x7ffbb40d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb9dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffbb40cf3f0 0x7ffbb40d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb9dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb40d3b30 con 0x7ffbb40b8570 2023-07-13T22:21:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb9dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb40d1930 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb9dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40d09d0 con 0x7ffbb40d35c0 2023-07-13T22:21:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb37fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffbb40b8570 0x7ffbb40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb37fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffbb40b8570 0x7ffbb40ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54352/0 (socket says 172.21.15.60:54352) 2023-07-13T22:21:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb37fe700 1 -- 172.21.15.60:0/2442161951 learned_addr learned my addr 172.21.15.60:0/2442161951 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbabfff700 1 --2- 172.21.15.60:0/2442161951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40d35c0 0x7ffbb40d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb3fff700 1 --2- 172.21.15.60:0/2442161951 >> v2:172.21.15.105:3300/0 conn(0x7ffbb40cf3f0 0x7ffbb40d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb37fe700 1 -- 172.21.15.60:0/2442161951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40d35c0 msgr2=0x7ffbb40d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb37fe700 1 --2- 172.21.15.60:0/2442161951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40d35c0 0x7ffbb40d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb37fe700 1 -- 172.21.15.60:0/2442161951 >> v2:172.21.15.105:3300/0 conn(0x7ffbb40cf3f0 msgr2=0x7ffbb40d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb37fe700 1 --2- 172.21.15.60:0/2442161951 >> v2:172.21.15.105:3300/0 conn(0x7ffbb40cf3f0 0x7ffbb40d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.052+0000 7ffbb37fe700 1 -- 172.21.15.60:0/2442161951 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40cda10 con 0x7ffbb40b8570 2023-07-13T22:21:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.053+0000 7ffbb37fe700 1 --2- 172.21.15.60:0/2442161951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40b8570 0x7ffbb40ceeb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40cd240 tx=0x7ffba4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19a91ce4f4d4280 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.053+0000 7ffbb2ffd700 1 -- 172.21.15.60:0/2442161951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4005e30 con 0x7ffbb40b8570 2023-07-13T22:21:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.053+0000 7ffbb2ffd700 1 -- 172.21.15.60:0/2442161951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba4004030 con 0x7ffbb40b8570 2023-07-13T22:21:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.053+0000 7ffbb2ffd700 1 -- 172.21.15.60:0/2442161951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba40043e0 con 0x7ffbb40b8570 2023-07-13T22:21:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.053+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2442161951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40b8570 msgr2=0x7ffbb40ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.053+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2442161951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40b8570 0x7ffbb40ceeb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40cd240 tx=0x7ffba4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.054+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2442161951 shutdown_connections 2023-07-13T22:21:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.054+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2442161951 >> v2:172.21.15.105:3300/0 conn(0x7ffbb40cf3f0 0x7ffbb40d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.054+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2442161951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40b8570 0x7ffbb40ceeb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.054+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2442161951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40d35c0 0x7ffbb40d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.054+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2442161951 >> 172.21.15.60:0/2442161951 conn(0x7ffbb4071600 msgr2=0x7ffbb40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.054+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2442161951 shutdown_connections 2023-07-13T22:21:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.054+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2442161951 wait complete. 2023-07-13T22:21:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.055+0000 7ffbb9dff700 1 Processor -- start 2023-07-13T22:21:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.056+0000 7ffbb9dff700 1 -- start start 2023-07-13T22:21:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.056+0000 7ffbb9dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40b8570 0x7ffbb41f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.056+0000 7ffbb9dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40cf3f0 0x7ffbb40c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.056+0000 7ffbb9dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb40d35c0 0x7ffbb40cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.056+0000 7ffbb9dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40c7d80 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbb9dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40c7170 con 0x7ffbb40d35c0 2023-07-13T22:21:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbb9dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40c7450 con 0x7ffbb40b8570 2023-07-13T22:21:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbabfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40cf3f0 0x7ffbb40c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbabfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40cf3f0 0x7ffbb40c6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54368/0 (socket says 172.21.15.60:54368) 2023-07-13T22:21:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbabfff700 1 -- 172.21.15.60:0/2334193959 learned_addr learned my addr 172.21.15.60:0/2334193959 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbb37fe700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40b8570 0x7ffbb41f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbb3fff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb40d35c0 0x7ffbb40cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbabfff700 1 -- 172.21.15.60:0/2334193959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40b8570 msgr2=0x7ffbb41f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbabfff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40b8570 0x7ffbb41f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbabfff700 1 -- 172.21.15.60:0/2334193959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb40d35c0 msgr2=0x7ffbb40cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbabfff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb40d35c0 0x7ffbb40cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbabfff700 1 -- 172.21.15.60:0/2334193959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4009500 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.057+0000 7ffbabfff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40cf3f0 0x7ffbb40c6790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00e9b0 tx=0x7ffb9c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.058+0000 7ffbb0ff9700 1 -- 172.21.15.60:0/2334193959 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c009710 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.058+0000 7ffbb0ff9700 1 -- 172.21.15.60:0/2334193959 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c00ee50 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.058+0000 7ffbb0ff9700 1 -- 172.21.15.60:0/2334193959 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00ca50 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.058+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb40c7730 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.058+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb41f5650 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.059+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb404e280 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.060+0000 7ffbb0ff9700 1 -- 172.21.15.60:0/2334193959 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb9c00cbb0 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.061+0000 7ffbb0ff9700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb9406ace0 0x7ffb9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.061+0000 7ffbb0ff9700 1 -- 172.21.15.60:0/2334193959 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb9c089460 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.061+0000 7ffbb37fe700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb9406ace0 0x7ffb9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:20.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.062+0000 7ffbb37fe700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb9406ace0 0x7ffb9406d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009c10 tx=0x7ffba4008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.066+0000 7ffbb0ff9700 1 -- 172.21.15.60:0/2334193959 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffb9c057f90 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.321+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffbb415d640 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.321+0000 7ffbb0ff9700 1 -- 172.21.15.60:0/2334193959 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffb9c00ce60 con 0x7ffbb40cf3f0 2023-07-13T22:21:20.307 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:21:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb9406ace0 msgr2=0x7ffb9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb9406ace0 0x7ffb9406d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009c10 tx=0x7ffba4008070 comp rx=0 tx=0).stop 2023-07-13T22:21:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40cf3f0 msgr2=0x7ffbb40c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40cf3f0 0x7ffbb40c6790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00e9b0 tx=0x7ffb9c00ecc0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 shutdown_connections 2023-07-13T22:21:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb9406ace0 0x7ffb9406d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb40d35c0 0x7ffbb40cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbb40b8570 0x7ffbb41f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 --2- 172.21.15.60:0/2334193959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb40cf3f0 0x7ffbb40c6790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 >> 172.21.15.60:0/2334193959 conn(0x7ffbb4071600 msgr2=0x7ffbb415cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 shutdown_connections 2023-07-13T22:21:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:20.324+0000 7ffbb9dff700 1 -- 172.21.15.60:0/2334193959 wait complete. 2023-07-13T22:21:20.326 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:21:20.326 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:21:20.327 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:21:20.327 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:21:20.327 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:21:20.327 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:21:20.327 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:21:20.327 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:21:20.328 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:21:20.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:20.328 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:21:20.380 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:21:20.380 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:21:20.380 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:21:20.380 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:20.380 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:21:20.381 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:20.381 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:20.381 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:20.381 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:21:20.381 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:20.381 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:20.381 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:21:20.382 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:20.382 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:20.382 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:21:20.382 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:20.382 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:21:20.382 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41690sec preferred_lft 41690sec 2023-07-13T22:21:20.382 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:21:20.383 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:20.383 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-13T22:21:20.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:20.384 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:21:20.384 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:21:20.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2023-07-13T22:21:20.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2023-07-13T22:21:20.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2023-07-13T22:21:20.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-13T22:21:20.386 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:21:20.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:20.820 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:21:20.931 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:21:20.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:20.935 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:21:21.004 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:21:21.005 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:21:21.005 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:21:21.005 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:21:21.005 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:21:21.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:21.006 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:21:21.006 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:21:21.007 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:21:21.007 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:21:21.007 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:21:21.007 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:21:21.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:21.703 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:21:21.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:21.750 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2023-07-13T22:21:21.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:21.797 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:21:21.797 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:21:21.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:21:21.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-07-13T22:21:21.798 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:21:21.942 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-07-13T22:21:21.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:21.942 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:21:21.943 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:21:21.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-07-13T22:21:21.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-13T22:21:21.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-07-13T22:21:21.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-07-13T22:21:21.944 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-07-13T22:21:21.944 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:21:22.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:22.219 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:21:22.219 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:21:22.219 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:21:22.219 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2023-07-13T22:21:22.220 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:21:22.391 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:21:22.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:21:22.392 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:21:22.437 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:21:22.438 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:21:22.438 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:21:22.523 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:21:22.634 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:21:22.635 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:21:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:21:22.710 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:21:22.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:22.710 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:21:22.756 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:21:22.756 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:21:22.801 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:21:22.871 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:21:22.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:21:22.942 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:21:22.944 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:21:22.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:22.945 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:21:22.968 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:21:22.984+0000 7f26e97255c0 -1 init, newargv = 0x5603a189d150 newargc=15 2023-07-13T22:21:22.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[119170]: starting ceph client 2023-07-13T22:21:22.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[119170]: starting fuse 2023-07-13T22:21:22.994 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:21:22.994 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:21:22.994 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:21:22.995 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:21:22.995 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:21:22.995 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:21:22.995 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:21:22.996 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:21:22.996 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:21:22.996 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:21:22.996 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:21:22.996 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:21:22.997 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:21:22.997 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:21:22.997 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:21:22.997 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:21:22.998 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:21:22.998 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:21:22.998 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:21:22.998 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:21:22.998 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:21:22.999 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:21:22.999 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:21:22.999 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:21:22.999 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:21:23.000 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:21:23.000 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:21:23.000 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:21:23.000 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:21:23.178 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.119170.asok 2023-07-13T22:21:23.355 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.119170.asok 2023-07-13T22:21:23.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:23.356 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.119170.asok status 2023-07-13T22:21:23.582 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:21:23.582 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:21:23.582 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:21:23.582 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:21:23.582 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:21:23.582 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:21:23.583 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:21:23.583 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "119170", 2023-07-13T22:21:23.583 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:21:23.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:21:23.583 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:21:23.583 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:21:23.583 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4562, 2023-07-13T22:21:23.583 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:21:23.584 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:21:23.584 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:21:23.584 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4562 2023-07-13T22:21:23.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:21:23.584 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:21:23.584 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:21:23.584 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:21:23.584 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3606164576 2023-07-13T22:21:23.585 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:21:23.585 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:21:23.585 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:21:23.585 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:21:23.585 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:21:23.585 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3606164576 2023-07-13T22:21:23.585 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:21:23.585 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.4562 192.168.0.1:0/3606164576", 2023-07-13T22:21:23.586 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/3606164576", 2023-07-13T22:21:23.586 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:21:23.586 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 8, 2023-07-13T22:21:23.586 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 21, 2023-07-13T22:21:23.586 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:21:23.586 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:21:23.586 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:21:23.587 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:21:23.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:21:23.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.825+0000 7f88499ff700 1 Processor -- start 2023-07-13T22:21:23.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.825+0000 7f88499ff700 1 -- start start 2023-07-13T22:21:23.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.826+0000 7f88499ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f88440b8400 0x7f88440cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:23.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.826+0000 7f88499ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440d3750 0x7f88440cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:23.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.826+0000 7f88499ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f88440cfad0 0x7f884415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:23.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.826+0000 7f88499ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88440cff20 con 0x7f88440cfad0 2023-07-13T22:21:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.826+0000 7f88499ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88440d0ac0 con 0x7f88440b8400 2023-07-13T22:21:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.826+0000 7f88499ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88440b87d0 con 0x7f88440d3750 2023-07-13T22:21:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88437fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f88440cfad0 0x7f884415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88437fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f88440cfad0 0x7f884415fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54384/0 (socket says 172.21.15.60:54384) 2023-07-13T22:21:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88437fe700 1 -- 172.21.15.60:0/1577379187 learned_addr learned my addr 172.21.15.60:0/1577379187 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88427fc700 1 --2- 172.21.15.60:0/1577379187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440d3750 0x7f88440cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f8842ffd700 1 --2- 172.21.15.60:0/1577379187 >> v2:172.21.15.105:3300/0 conn(0x7f88440b8400 0x7f88440cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88437fe700 1 -- 172.21.15.60:0/1577379187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440d3750 msgr2=0x7f88440cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88437fe700 1 --2- 172.21.15.60:0/1577379187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440d3750 0x7f88440cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88437fe700 1 -- 172.21.15.60:0/1577379187 >> v2:172.21.15.105:3300/0 conn(0x7f88440b8400 msgr2=0x7f88440cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88437fe700 1 --2- 172.21.15.60:0/1577379187 >> v2:172.21.15.105:3300/0 conn(0x7f88440b8400 0x7f88440cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88437fe700 1 -- 172.21.15.60:0/1577379187 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88440b4eb0 con 0x7f88440cfad0 2023-07-13T22:21:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f88437fe700 1 --2- 172.21.15.60:0/1577379187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440cfad0 0x7f884415fef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f88440cf560 tx=0x7f883c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd90f39338a39fc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.827+0000 7f8841ffb700 1 -- 172.21.15.60:0/1577379187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00e070 con 0x7f88440cfad0 2023-07-13T22:21:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f8841ffb700 1 -- 172.21.15.60:0/1577379187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c00b9c0 con 0x7f88440cfad0 2023-07-13T22:21:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f8841ffb700 1 -- 172.21.15.60:0/1577379187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00bd70 con 0x7f88440cfad0 2023-07-13T22:21:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f88499ff700 1 -- 172.21.15.60:0/1577379187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440cfad0 msgr2=0x7f884415fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f88499ff700 1 --2- 172.21.15.60:0/1577379187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440cfad0 0x7f884415fef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f88440cf560 tx=0x7f883c00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:23.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f88499ff700 1 -- 172.21.15.60:0/1577379187 shutdown_connections 2023-07-13T22:21:23.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f88499ff700 1 --2- 172.21.15.60:0/1577379187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440cfad0 0x7f884415fef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:23.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f88499ff700 1 --2- 172.21.15.60:0/1577379187 >> v2:172.21.15.105:3300/0 conn(0x7f88440b8400 0x7f88440cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:23.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f88499ff700 1 --2- 172.21.15.60:0/1577379187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440d3750 0x7f88440cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:23.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f88499ff700 1 -- 172.21.15.60:0/1577379187 >> 172.21.15.60:0/1577379187 conn(0x7f884406b390 msgr2=0x7f884406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:23.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f88499ff700 1 -- 172.21.15.60:0/1577379187 shutdown_connections 2023-07-13T22:21:23.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.828+0000 7f88499ff700 1 -- 172.21.15.60:0/1577379187 wait complete. 2023-07-13T22:21:23.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.830+0000 7f88499ff700 1 Processor -- start 2023-07-13T22:21:23.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.830+0000 7f88499ff700 1 -- start start 2023-07-13T22:21:23.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.831+0000 7f88499ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440b8400 0x7f884414d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:23.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.831+0000 7f88499ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88440cfad0 0x7f88440c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:23.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.831+0000 7f88499ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440d3750 0x7f88440b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.831+0000 7f88499ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884415bc80 con 0x7f88440d3750 2023-07-13T22:21:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.831+0000 7f88499ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884415bed0 con 0x7f88440cfad0 2023-07-13T22:21:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.831+0000 7f88499ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884415c1b0 con 0x7f88440b8400 2023-07-13T22:21:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.831+0000 7f88437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440d3750 0x7f88440b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.831+0000 7f88437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440d3750 0x7f88440b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54390/0 (socket says 172.21.15.60:54390) 2023-07-13T22:21:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.831+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440b8400 0x7f884414d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f88437fe700 1 -- 172.21.15.60:0/1897236054 learned_addr learned my addr 172.21.15.60:0/1897236054 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f88427fc700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88440cfad0 0x7f88440c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f88437fe700 1 -- 172.21.15.60:0/1897236054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440b8400 msgr2=0x7f884414d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f88437fe700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440b8400 0x7f884414d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f88437fe700 1 -- 172.21.15.60:0/1897236054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88440cfad0 msgr2=0x7f88440c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f88437fe700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88440cfad0 0x7f88440c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f88437fe700 1 -- 172.21.15.60:0/1897236054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c009b30 con 0x7f88440d3750 2023-07-13T22:21:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f88437fe700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440d3750 0x7f88440b2950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f883c00b370 tx=0x7f883c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f882bfff700 1 -- 172.21.15.60:0/1897236054 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0099b0 con 0x7f88440d3750 2023-07-13T22:21:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f882bfff700 1 -- 172.21.15.60:0/1897236054 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c004250 con 0x7f88440d3750 2023-07-13T22:21:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f882bfff700 1 -- 172.21.15.60:0/1897236054 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0045e0 con 0x7f88440d3750 2023-07-13T22:21:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.832+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884415c490 con 0x7f88440d3750 2023-07-13T22:21:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.833+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88440c9190 con 0x7f88440d3750 2023-07-13T22:21:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.834+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88440c9320 con 0x7f88440d3750 2023-07-13T22:21:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.835+0000 7f882bfff700 1 -- 172.21.15.60:0/1897236054 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f883c004740 con 0x7f88440d3750 2023-07-13T22:21:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.835+0000 7f882bfff700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f883006ace0 0x7f883006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.836+0000 7f882bfff700 1 -- 172.21.15.60:0/1897236054 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f883c08a570 con 0x7f88440d3750 2023-07-13T22:21:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.836+0000 7f8842ffd700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f883006ace0 0x7f883006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.836+0000 7f8842ffd700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f883006ace0 0x7f883006d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8834006b20 tx=0x7f8834008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:23.840+0000 7f882bfff700 1 -- 172.21.15.60:0/1897236054 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f883c055610 con 0x7f88440d3750 2023-07-13T22:21:24.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.093+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f884415c9b0 con 0x7f88440d3750 2023-07-13T22:21:24.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.094+0000 7f882bfff700 1 -- 172.21.15.60:0/1897236054 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f883c058c30 con 0x7f88440d3750 2023-07-13T22:21:24.079 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:21:24.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f883006ace0 msgr2=0x7f883006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:24.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f883006ace0 0x7f883006d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8834006b20 tx=0x7f8834008040 comp rx=0 tx=0).stop 2023-07-13T22:21:24.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440d3750 msgr2=0x7f88440b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:24.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440d3750 0x7f88440b2950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f883c00b370 tx=0x7f883c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:24.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 shutdown_connections 2023-07-13T22:21:24.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f883006ace0 0x7f883006d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:24.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88440d3750 0x7f88440b2950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:24.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88440b8400 0x7f884414d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:24.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 --2- 172.21.15.60:0/1897236054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88440cfad0 0x7f88440c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:24.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 >> 172.21.15.60:0/1897236054 conn(0x7f884406b390 msgr2=0x7f88440d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:24.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 shutdown_connections 2023-07-13T22:21:24.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:24.096+0000 7f88499ff700 1 -- 172.21.15.60:0/1897236054 wait complete. 2023-07-13T22:21:24.098 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:21:24.099 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:21:24.099 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:21:24.099 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-07-13T22:21:24.099 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-13T22:21:24.100 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:21:24.100 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:21:24.100 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:21:24.100 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-07-13T22:21:24.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.101 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:21:24.149 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:21:24.150 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:21:24.150 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:21:24.151 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.151 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:21:24.151 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.151 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:24.152 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:24.152 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:21:24.152 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:24.152 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:24.153 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:21:24.153 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:21:24.153 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:24.153 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:21:24.154 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:24.154 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:21:24.154 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41687sec preferred_lft 41687sec 2023-07-13T22:21:24.154 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:21:24.154 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.155 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:21:24.155 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:21:24.155 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:21:24.155 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.155 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:21:24.155 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.156 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:21:24.156 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:21:24.156 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:21:24.156 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.157 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:21:24.224 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:21:24.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.228 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:21:24.276 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:21:24.276 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:21:24.276 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:21:24.276 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:21:24.276 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:21:24.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.278 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:21:24.278 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:21:24.278 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:21:24.278 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:21:24.279 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:21:24.279 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:21:24.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.447 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:21:24.492 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:21:24.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.494 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2023-07-13T22:21:24.540 INFO:teuthology.orchestra.run.smithi105.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-07-13T22:21:24.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.542 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:21:24.542 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:21:24.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:21:24.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-07-13T22:21:24.544 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:21:24.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-13T22:21:24.794 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:21:24.794 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:21:24.795 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:21:24.795 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.795 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:21:24.795 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.795 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:21:24.795 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ca:ab:c7:9c:12:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-13T22:21:24.795 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-13T22:21:24.795 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.796 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c8ab:c7ff:fe9c:1201/64 scope link 2023-07-13T22:21:24.796 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-13T22:21:24.868 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:21:24.868 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:21:24.868 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:21:24.869 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.869 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:21:24.869 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.869 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:21:24.869 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ca:ab:c7:9c:12:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-13T22:21:24.869 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-13T22:21:24.870 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.870 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::c8ab:c7ff:fe9c:1201/64 scope link 2023-07-13T22:21:24.870 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:21:24.871 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-07-13T22:21:24.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:24.871 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:21:24.872 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:21:24.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-07-13T22:21:24.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-13T22:21:24.873 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-07-13T22:21:24.873 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-07-13T22:21:24.873 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-07-13T22:21:24.873 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:21:24.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:21:24.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:21:25.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:25.127 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:21:25.127 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:21:25.127 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.1 up 2023-07-13T22:21:25.127 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.1 master ceph-brx 2023-07-13T22:21:25.128 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:21:25.299 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-07-13T22:21:25.300 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:21:25.300 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-07-13T22:21:25.346 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-07-13T22:21:25.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:21:25.346 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-07-13T22:21:25.432 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:21:25.543 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:21:25.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:21:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:21:25.614 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:21:25.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:25.615 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:21:25.661 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:21:25.661 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-07-13T22:21:25.661 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-07-13T22:21:25.666 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:21:25.737 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:21:25.737 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:21:25.805 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:21:25.808 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:21:25.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:25.809 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:21:25.832 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi105.stderr:2023-07-13T22:21:25.848+0000 7fe6eee365c0 -1 init, newargv = 0x55bc6190bf70 newargc=15 2023-07-13T22:21:25.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi105.stderr:ceph-fuse[119869]: starting ceph client 2023-07-13T22:21:25.840 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi105.stderr:ceph-fuse[119869]: starting fuse 2023-07-13T22:21:25.858 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:21:25.858 INFO:teuthology.orchestra.run.smithi105.stdout:45 2023-07-13T22:21:25.858 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 45] 2023-07-13T22:21:25.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:21:25.859 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:21:25.859 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:21:25.860 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:21:25.860 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:21:25.860 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:21:25.860 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:21:25.860 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-07-13T22:21:25.861 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:21:25.861 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-07-13T22:21:25.861 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:21:25.861 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:21:25.862 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:21:25.862 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:21:25.862 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:21:25.862 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:21:25.863 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:21:25.863 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:21:25.863 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:21:25.863 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:21:25.863 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:21:25.864 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:21:25.864 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:21:25.864 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:21:25.864 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:21:25.865 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.3")) 2023-07-13T22:21:25.865 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:21:26.041 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.3.119869.asok 2023-07-13T22:21:26.220 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.119869.asok 2023-07-13T22:21:26.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:26.222 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.119869.asok status 2023-07-13T22:21:26.445 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:21:26.445 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:21:26.445 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:21:26.445 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:21:26.445 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "3", 2023-07-13T22:21:26.445 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:21:26.446 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-07-13T22:21:26.446 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "119869", 2023-07-13T22:21:26.446 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:21:26.446 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:21:26.446 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:21:26.446 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:21:26.446 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4574, 2023-07-13T22:21:26.446 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:21:26.447 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:21:26.447 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:21:26.447 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4574 2023-07-13T22:21:26.447 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:21:26.447 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:21:26.447 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:21:26.448 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.2:0", 2023-07-13T22:21:26.448 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1651629538 2023-07-13T22:21:26.448 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:21:26.448 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:21:26.448 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:21:26.448 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:21:26.448 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.2:0", 2023-07-13T22:21:26.449 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1651629538 2023-07-13T22:21:26.449 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:21:26.449 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.4574 192.168.0.2:0/1651629538", 2023-07-13T22:21:26.449 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.2:0/1651629538", 2023-07-13T22:21:26.449 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:21:26.449 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 8, 2023-07-13T22:21:26.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 21, 2023-07-13T22:21:26.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:21:26.450 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:21:26.450 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:21:26.450 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:21:26.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:26.459 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:21:26.459 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:21:26.512 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:21:26.512 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:21:26.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:26.513 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:21:26.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:26.592 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:21:26.592 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.2 2023-07-13T22:21:26.641 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:21:26.641 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-07-13T22:21:26.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:26.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-07-13T22:21:26.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:26.716 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:21:26.716 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:21:26.763 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:21:26.764 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:21:26.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:26.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:21:26.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:26.835 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:21:26.835 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.3 2023-07-13T22:21:26.882 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:21:26.882 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-07-13T22:21:26.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:26.882 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-07-13T22:21:26.953 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-07-13T22:21:26.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:27.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.189+0000 7f657c1ff700 1 Processor -- start 2023-07-13T22:21:27.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.190+0000 7f657c1ff700 1 -- start start 2023-07-13T22:21:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657c1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6574159170 0x7f65740c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657c1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65740c5040 0x7f6574070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657c1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65740707b0 0x7f65740cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657c1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657415d970 con 0x7f65740c5040 2023-07-13T22:21:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657c1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6574070bd0 con 0x7f65740707b0 2023-07-13T22:21:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657c1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657415cd60 con 0x7f6574159170 2023-07-13T22:21:27.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657979a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65740c5040 0x7f6574070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657979a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65740c5040 0x7f6574070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47730/0 (socket says 172.21.15.60:47730) 2023-07-13T22:21:27.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657979a700 1 -- 172.21.15.60:0/3975628853 learned_addr learned my addr 172.21.15.60:0/3975628853 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:27.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f657a79c700 1 --2- 172.21.15.60:0/3975628853 >> v2:172.21.15.105:3300/0 conn(0x7f65740707b0 0x7f65740cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.191+0000 7f6579f9b700 1 --2- 172.21.15.60:0/3975628853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6574159170 0x7f65740c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.192+0000 7f657979a700 1 -- 172.21.15.60:0/3975628853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6574159170 msgr2=0x7f65740c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:27.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.192+0000 7f657979a700 1 --2- 172.21.15.60:0/3975628853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6574159170 0x7f65740c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.192+0000 7f657979a700 1 -- 172.21.15.60:0/3975628853 >> v2:172.21.15.105:3300/0 conn(0x7f65740707b0 msgr2=0x7f65740cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:27.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.192+0000 7f657979a700 1 --2- 172.21.15.60:0/3975628853 >> v2:172.21.15.105:3300/0 conn(0x7f65740707b0 0x7f65740cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.192+0000 7f657979a700 1 -- 172.21.15.60:0/3975628853 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657415d040 con 0x7f65740c5040 2023-07-13T22:21:27.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.192+0000 7f657979a700 1 --2- 172.21.15.60:0/3975628853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740c5040 0x7f6574070270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f65740b0360 tx=0x7f6564009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc88e23f2b7eec5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:27.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.192+0000 7f6578f99700 1 -- 172.21.15.60:0/3975628853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564011830 con 0x7f65740c5040 2023-07-13T22:21:27.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.192+0000 7f6578f99700 1 -- 172.21.15.60:0/3975628853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564011990 con 0x7f65740c5040 2023-07-13T22:21:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.192+0000 7f6578f99700 1 -- 172.21.15.60:0/3975628853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564011d20 con 0x7f65740c5040 2023-07-13T22:21:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.193+0000 7f657c1ff700 1 -- 172.21.15.60:0/3975628853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740c5040 msgr2=0x7f6574070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.193+0000 7f657c1ff700 1 --2- 172.21.15.60:0/3975628853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740c5040 0x7f6574070270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f65740b0360 tx=0x7f6564009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.193+0000 7f657c1ff700 1 -- 172.21.15.60:0/3975628853 shutdown_connections 2023-07-13T22:21:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.193+0000 7f657c1ff700 1 --2- 172.21.15.60:0/3975628853 >> v2:172.21.15.105:3300/0 conn(0x7f65740707b0 0x7f65740cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.193+0000 7f657c1ff700 1 --2- 172.21.15.60:0/3975628853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6574159170 0x7f65740c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.193+0000 7f657c1ff700 1 --2- 172.21.15.60:0/3975628853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740c5040 0x7f6574070270 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.193+0000 7f657c1ff700 1 -- 172.21.15.60:0/3975628853 >> 172.21.15.60:0/3975628853 conn(0x7f65740cf410 msgr2=0x7f65740cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:27.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.193+0000 7f657c1ff700 1 -- 172.21.15.60:0/3975628853 shutdown_connections 2023-07-13T22:21:27.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.193+0000 7f657c1ff700 1 -- 172.21.15.60:0/3975628853 wait complete. 2023-07-13T22:21:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.195+0000 7f657c1ff700 1 Processor -- start 2023-07-13T22:21:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.196+0000 7f657c1ff700 1 -- start start 2023-07-13T22:21:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f657c1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740707b0 0x7f657414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f657c1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65740c5040 0x7f65740acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f657c1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6574159170 0x7f65740ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f657c1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65741f2680 con 0x7f65740707b0 2023-07-13T22:21:27.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f657c1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65741f34a0 con 0x7f6574159170 2023-07-13T22:21:27.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f657c1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65741f2890 con 0x7f65740c5040 2023-07-13T22:21:27.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f6579f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740707b0 0x7f657414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f6579f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740707b0 0x7f657414b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47736/0 (socket says 172.21.15.60:47736) 2023-07-13T22:21:27.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f6579f9b700 1 -- 172.21.15.60:0/2431207957 learned_addr learned my addr 172.21.15.60:0/2431207957 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:27.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f657979a700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65740c5040 0x7f65740acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.197+0000 7f657a79c700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6574159170 0x7f65740ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f6579f9b700 1 -- 172.21.15.60:0/2431207957 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65740c5040 msgr2=0x7f65740acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f6579f9b700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65740c5040 0x7f65740acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f6579f9b700 1 -- 172.21.15.60:0/2431207957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6574159170 msgr2=0x7f65740ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f6579f9b700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6574159170 0x7f65740ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f6579f9b700 1 -- 172.21.15.60:0/2431207957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6564009550 con 0x7f65740707b0 2023-07-13T22:21:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f6579f9b700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740707b0 0x7f657414b040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f657000e9b0 tx=0x7f657000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f656a7fc700 1 -- 172.21.15.60:0/2431207957 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570009710 con 0x7f65740707b0 2023-07-13T22:21:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f656a7fc700 1 -- 172.21.15.60:0/2431207957 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657000c850 con 0x7f65740707b0 2023-07-13T22:21:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f656a7fc700 1 -- 172.21.15.60:0/2431207957 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000cbc0 con 0x7f65740707b0 2023-07-13T22:21:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65741f2b70 con 0x7f65740707b0 2023-07-13T22:21:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.198+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65741f3130 con 0x7f65740707b0 2023-07-13T22:21:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.199+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65740c7550 con 0x7f65740707b0 2023-07-13T22:21:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.200+0000 7f656a7fc700 1 -- 172.21.15.60:0/2431207957 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f657000cd20 con 0x7f65740707b0 2023-07-13T22:21:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.201+0000 7f656a7fc700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f655806ace0 0x7f655806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.201+0000 7f656a7fc700 1 -- 172.21.15.60:0/2431207957 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65700895c0 con 0x7f65740707b0 2023-07-13T22:21:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.201+0000 7f657979a700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f655806ace0 0x7f655806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.202+0000 7f657979a700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f655806ace0 0x7f655806d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6564014040 tx=0x7f6564008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:27.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.205+0000 7f656a7fc700 1 -- 172.21.15.60:0/2431207957 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6570058060 con 0x7f65740707b0 2023-07-13T22:21:27.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.412+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6574071440 con 0x7f65740707b0 2023-07-13T22:21:27.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.413+0000 7f656a7fc700 1 -- 172.21.15.60:0/2431207957 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f6570057bf0 con 0x7f65740707b0 2023-07-13T22:21:27.398 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:27.399 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:27.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f655806ace0 msgr2=0x7f655806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f655806ace0 0x7f655806d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6564014040 tx=0x7f6564008030 comp rx=0 tx=0).stop 2023-07-13T22:21:27.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740707b0 msgr2=0x7f657414b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740707b0 0x7f657414b040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f657000e9b0 tx=0x7f657000ed70 comp rx=0 tx=0).stop 2023-07-13T22:21:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 shutdown_connections 2023-07-13T22:21:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f655806ace0 0x7f655806d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65740707b0 0x7f657414b040 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6574159170 0x7f65740ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 --2- 172.21.15.60:0/2431207957 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65740c5040 0x7f65740acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 >> 172.21.15.60:0/2431207957 conn(0x7f65740cf410 msgr2=0x7f65740b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 shutdown_connections 2023-07-13T22:21:27.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.416+0000 7f657c1ff700 1 -- 172.21.15.60:0/2431207957 wait complete. 2023-07-13T22:21:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:27.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.640+0000 7feb11dff700 1 Processor -- start 2023-07-13T22:21:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.641+0000 7feb11dff700 1 -- start start 2023-07-13T22:21:27.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.642+0000 7feb11dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb0c0b8200 0x7feb0c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.642+0000 7feb11dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b1ac0 0x7feb0c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.642+0000 7feb11dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0d1870 con 0x7feb0c0cede0 2023-07-13T22:21:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.642+0000 7feb11dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0cf370 con 0x7feb0c0b8200 2023-07-13T22:21:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.642+0000 7feb11dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0d12b0 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.642+0000 7feb0b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb0c0b8200 0x7feb0c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.642+0000 7feb0bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feb0c0cede0 0x7feb0c15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47452/0 (socket says 172.21.15.60:47452) 2023-07-13T22:21:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.642+0000 7feb0bfff700 1 -- 172.21.15.60:0/1025657410 learned_addr learned my addr 172.21.15.60:0/1025657410 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.642+0000 7feb0affd700 1 --2- 172.21.15.60:0/1025657410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b1ac0 0x7feb0c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.643+0000 7feb0b7fe700 1 -- 172.21.15.60:0/1025657410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b1ac0 msgr2=0x7feb0c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.643+0000 7feb0b7fe700 1 --2- 172.21.15.60:0/1025657410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b1ac0 0x7feb0c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.643+0000 7feb0b7fe700 1 -- 172.21.15.60:0/1025657410 >> v1:172.21.15.60:6789/0 conn(0x7feb0c0cede0 legacy=0x7feb0c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.643+0000 7feb0b7fe700 1 -- 172.21.15.60:0/1025657410 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0d1590 con 0x7feb0c0b8200 2023-07-13T22:21:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.643+0000 7feb0a7fc700 1 -- 172.21.15.60:0/1025657410 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612689506 0 0) 0x7feb0c0d1870 con 0x7feb0c0cede0 2023-07-13T22:21:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.643+0000 7feb0b7fe700 1 --2- 172.21.15.60:0/1025657410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b8200 0x7feb0c0cf640 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feafc009840 tx=0x7feafc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9b381a68963251 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.643+0000 7feb0a7fc700 1 -- 172.21.15.60:0/1025657410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc011830 con 0x7feb0c0b8200 2023-07-13T22:21:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.643+0000 7feb0a7fc700 1 -- 172.21.15.60:0/1025657410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc011990 con 0x7feb0c0b8200 2023-07-13T22:21:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.643+0000 7feb0a7fc700 1 -- 172.21.15.60:0/1025657410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc011d20 con 0x7feb0c0b8200 2023-07-13T22:21:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.644+0000 7feb11dff700 1 -- 172.21.15.60:0/1025657410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b8200 msgr2=0x7feb0c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.644+0000 7feb11dff700 1 --2- 172.21.15.60:0/1025657410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b8200 0x7feb0c0cf640 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feafc009840 tx=0x7feafc009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.644+0000 7feb11dff700 1 -- 172.21.15.60:0/1025657410 shutdown_connections 2023-07-13T22:21:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.644+0000 7feb11dff700 1 --2- 172.21.15.60:0/1025657410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b8200 0x7feb0c0cf640 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.644+0000 7feb11dff700 1 --2- 172.21.15.60:0/1025657410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b1ac0 0x7feb0c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.644+0000 7feb11dff700 1 -- 172.21.15.60:0/1025657410 >> 172.21.15.60:0/1025657410 conn(0x7feb0c06b2f0 msgr2=0x7feb0c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.644+0000 7feb11dff700 1 -- 172.21.15.60:0/1025657410 shutdown_connections 2023-07-13T22:21:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.644+0000 7feb11dff700 1 -- 172.21.15.60:0/1025657410 wait complete. 2023-07-13T22:21:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.646+0000 7feb11dff700 1 Processor -- start 2023-07-13T22:21:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.646+0000 7feb11dff700 1 -- start start 2023-07-13T22:21:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb11dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b1ac0 0x7feb0c1e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb11dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b8200 0x7feb0c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb11dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb0c0cede0 0x7feb0c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb11dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1e3b20 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb11dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1e3e00 con 0x7feb0c0cede0 2023-07-13T22:21:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb11dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1e40e0 con 0x7feb0c0b8200 2023-07-13T22:21:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b1ac0 0x7feb0c1e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b1ac0 0x7feb0c1e6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47758/0 (socket says 172.21.15.60:47758) 2023-07-13T22:21:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0b7fe700 1 -- 172.21.15.60:0/453422623 learned_addr learned my addr 172.21.15.60:0/453422623 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0bfff700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb0c0cede0 0x7feb0c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0affd700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b8200 0x7feb0c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0b7fe700 1 -- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b8200 msgr2=0x7feb0c0c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0b7fe700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b8200 0x7feb0c0c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0b7fe700 1 -- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb0c0cede0 msgr2=0x7feb0c15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0b7fe700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb0c0cede0 0x7feb0c15a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0b7fe700 1 -- 172.21.15.60:0/453422623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc0094a0 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0bfff700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb0c0cede0 0x7feb0c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.647+0000 7feb0affd700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b8200 0x7feb0c0c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.648+0000 7feb0b7fe700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b1ac0 0x7feb0c1e6410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feafc0090b0 tx=0x7feafc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.648+0000 7feaf3fff700 1 -- 172.21.15.60:0/453422623 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0038a0 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.648+0000 7feaf3fff700 1 -- 172.21.15.60:0/453422623 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc01f070 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.648+0000 7feaf3fff700 1 -- 172.21.15.60:0/453422623 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc003c20 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.648+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c1e43c0 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.648+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1e6950 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.649+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c065ee0 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.650+0000 7feaf3fff700 1 -- 172.21.15.60:0/453422623 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feafc01a430 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.651+0000 7feaf3fff700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaf806ace0 0x7feaf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.651+0000 7feaf3fff700 1 -- 172.21.15.60:0/453422623 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feafc021070 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.651+0000 7feb0affd700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaf806ace0 0x7feaf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.652+0000 7feb0affd700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaf806ace0 0x7feaf806d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0cce00 tx=0x7feb00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:27.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.656+0000 7feaf3fff700 1 -- 172.21.15.60:0/453422623 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7feafc055ee0 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.861+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feb0c1e6ae0 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.861+0000 7feaf3fff700 1 -- 172.21.15.60:0/453422623 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7feafc0150a0 con 0x7feb0c0b1ac0 2023-07-13T22:21:27.846 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:27.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaf806ace0 msgr2=0x7feaf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaf806ace0 0x7feaf806d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0cce00 tx=0x7feb00008040 comp rx=0 tx=0).stop 2023-07-13T22:21:27.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b1ac0 msgr2=0x7feb0c1e6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:27.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b1ac0 0x7feb0c1e6410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feafc0090b0 tx=0x7feafc004750 comp rx=0 tx=0).stop 2023-07-13T22:21:27.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 shutdown_connections 2023-07-13T22:21:27.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaf806ace0 0x7feaf806d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb0c0b1ac0 0x7feb0c1e6410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb0c0cede0 0x7feb0c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 --2- 172.21.15.60:0/453422623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feb0c0b8200 0x7feb0c0c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:27.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 >> 172.21.15.60:0/453422623 conn(0x7feb0c06b2f0 msgr2=0x7feb0c0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:27.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 shutdown_connections 2023-07-13T22:21:27.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:27.864+0000 7feb11dff700 1 -- 172.21.15.60:0/453422623 wait complete. 2023-07-13T22:21:27.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:28.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.088+0000 7fdee7fff700 1 Processor -- start 2023-07-13T22:21:28.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.088+0000 7fdee7fff700 1 -- start start 2023-07-13T22:21:28.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.089+0000 7fdee7fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdee00b8200 0x7fdee00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.089+0000 7fdee7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 0x7fdee00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.089+0000 7fdee7fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee00d1870 con 0x7fdee00cede0 2023-07-13T22:21:28.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.089+0000 7fdee7fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee00cf370 con 0x7fdee00b8200 2023-07-13T22:21:28.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.089+0000 7fdee7fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00d12b0 con 0x7fdee00b1ac0 2023-07-13T22:21:28.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.090+0000 7fdee559a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 0x7fdee00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.090+0000 7fdee5d9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdee00b8200 0x7fdee00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.090+0000 7fdee5d9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdee00b8200 0x7fdee00cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:49204/0 (socket says 172.21.15.60:49204) 2023-07-13T22:21:28.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.090+0000 7fdee5d9b700 1 -- 172.21.15.60:0/3283906124 learned_addr learned my addr 172.21.15.60:0/3283906124 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:28.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.090+0000 7fdee559a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 0x7fdee00cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:60980/0 (socket says 172.21.15.60:60980) 2023-07-13T22:21:28.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.090+0000 7fdee5d9b700 1 -- 172.21.15.60:0/3283906124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 msgr2=0x7fdee00cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:28.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.090+0000 7fdee5d9b700 1 --2- 172.21.15.60:0/3283906124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 0x7fdee00cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.090+0000 7fdee5d9b700 1 -- 172.21.15.60:0/3283906124 >> v1:172.21.15.105:6789/0 conn(0x7fdee00cede0 legacy=0x7fdee015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.090+0000 7fdee5d9b700 1 -- 172.21.15.60:0/3283906124 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00d1590 con 0x7fdee00b8200 2023-07-13T22:21:28.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.091+0000 7fdee5d9b700 1 --2- 172.21.15.60:0/3283906124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdee00b8200 0x7fdee00cf640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0098f0 tx=0x7fdedc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16f147481938cbff server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:28.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.091+0000 7fdee4d99700 1 -- 172.21.15.60:0/3283906124 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc011830 con 0x7fdee00b8200 2023-07-13T22:21:28.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.091+0000 7fdee4d99700 1 -- 172.21.15.60:0/3283906124 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc011990 con 0x7fdee00b8200 2023-07-13T22:21:28.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.091+0000 7fdee4d99700 1 -- 172.21.15.60:0/3283906124 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc011d20 con 0x7fdee00b8200 2023-07-13T22:21:28.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.091+0000 7fdee7fff700 1 -- 172.21.15.60:0/3283906124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdee00b8200 msgr2=0x7fdee00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.091+0000 7fdee7fff700 1 --2- 172.21.15.60:0/3283906124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdee00b8200 0x7fdee00cf640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0098f0 tx=0x7fdedc009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:28.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.092+0000 7fdee7fff700 1 -- 172.21.15.60:0/3283906124 shutdown_connections 2023-07-13T22:21:28.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.092+0000 7fdee7fff700 1 --2- 172.21.15.60:0/3283906124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdee00b8200 0x7fdee00cf640 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.092+0000 7fdee7fff700 1 --2- 172.21.15.60:0/3283906124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 0x7fdee00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.092+0000 7fdee7fff700 1 -- 172.21.15.60:0/3283906124 >> 172.21.15.60:0/3283906124 conn(0x7fdee006b2f0 msgr2=0x7fdee006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:28.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.092+0000 7fdee7fff700 1 -- 172.21.15.60:0/3283906124 shutdown_connections 2023-07-13T22:21:28.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.092+0000 7fdee7fff700 1 -- 172.21.15.60:0/3283906124 wait complete. 2023-07-13T22:21:28.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.094+0000 7fdee7fff700 1 Processor -- start 2023-07-13T22:21:28.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.094+0000 7fdee7fff700 1 -- start start 2023-07-13T22:21:28.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.095+0000 7fdee7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 0x7fdee014ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.095+0000 7fdee7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdee00b8200 0x7fdee01e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdee00cede0 0x7fdee014c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee7fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee014c420 con 0x7fdee00cede0 2023-07-13T22:21:28.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee7fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee014b810 con 0x7fdee00b8200 2023-07-13T22:21:28.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee7fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee014baf0 con 0x7fdee00b1ac0 2023-07-13T22:21:28.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdee00cede0 0x7fdee014c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee559a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdee00b8200 0x7fdee01e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee5d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 0x7fdee014ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdee00cede0 0x7fdee014c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47772/0 (socket says 172.21.15.60:47772) 2023-07-13T22:21:28.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee659c700 1 -- 172.21.15.60:0/3996126586 learned_addr learned my addr 172.21.15.60:0/3996126586 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:28.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee659c700 1 -- 172.21.15.60:0/3996126586 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 msgr2=0x7fdee014ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee659c700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 0x7fdee014ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee659c700 1 -- 172.21.15.60:0/3996126586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdee00b8200 msgr2=0x7fdee01e28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee659c700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdee00b8200 0x7fdee01e28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.096+0000 7fdee659c700 1 -- 172.21.15.60:0/3996126586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc009550 con 0x7fdee00cede0 2023-07-13T22:21:28.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.097+0000 7fdee659c700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdee00cede0 0x7fdee014c8b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fded800cb50 tx=0x7fded800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:28.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.097+0000 7fded67fc700 1 -- 172.21.15.60:0/3996126586 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8003c00 con 0x7fdee00cede0 2023-07-13T22:21:28.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.097+0000 7fded67fc700 1 -- 172.21.15.60:0/3996126586 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8003d60 con 0x7fdee00cede0 2023-07-13T22:21:28.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.097+0000 7fded67fc700 1 -- 172.21.15.60:0/3996126586 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded800aa20 con 0x7fdee00cede0 2023-07-13T22:21:28.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.097+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee014bdd0 con 0x7fdee00cede0 2023-07-13T22:21:28.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.097+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee015b790 con 0x7fdee00cede0 2023-07-13T22:21:28.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.098+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee014c5d0 con 0x7fdee00cede0 2023-07-13T22:21:28.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.099+0000 7fded67fc700 1 -- 172.21.15.60:0/3996126586 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fded800ab80 con 0x7fdee00cede0 2023-07-13T22:21:28.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.100+0000 7fded67fc700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec406ace0 0x7fdec406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.100+0000 7fded67fc700 1 -- 172.21.15.60:0/3996126586 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fded80894a0 con 0x7fdee00cede0 2023-07-13T22:21:28.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.100+0000 7fdee5d9b700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec406ace0 0x7fdec406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.101+0000 7fdee5d9b700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec406ace0 0x7fdec406d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009c40 tx=0x7fdedc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:28.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.105+0000 7fded67fc700 1 -- 172.21.15.60:0/3996126586 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdee014c5d0 con 0x7fdee00cede0 2023-07-13T22:21:28.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.310+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdee00ccd20 con 0x7fdee00cede0 2023-07-13T22:21:28.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.311+0000 7fded67fc700 1 -- 172.21.15.60:0/3996126586 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fded8057f40 con 0x7fdee00cede0 2023-07-13T22:21:28.296 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:28.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec406ace0 msgr2=0x7fdec406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec406ace0 0x7fdec406d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009c40 tx=0x7fdedc008030 comp rx=0 tx=0).stop 2023-07-13T22:21:28.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdee00cede0 msgr2=0x7fdee014c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdee00cede0 0x7fdee014c8b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fded800cb50 tx=0x7fded800ce60 comp rx=0 tx=0).stop 2023-07-13T22:21:28.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 shutdown_connections 2023-07-13T22:21:28.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec406ace0 0x7fdec406d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdee00cede0 0x7fdee014c8b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdee00b1ac0 0x7fdee014ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 --2- 172.21.15.60:0/3996126586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdee00b8200 0x7fdee01e28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 >> 172.21.15.60:0/3996126586 conn(0x7fdee006b2f0 msgr2=0x7fdee00b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:28.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 shutdown_connections 2023-07-13T22:21:28.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.313+0000 7fdee7fff700 1 -- 172.21.15.60:0/3996126586 wait complete. 2023-07-13T22:21:28.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.536+0000 7fd079bff700 1 Processor -- start 2023-07-13T22:21:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.536+0000 7fd079bff700 1 -- start start 2023-07-13T22:21:28.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.537+0000 7fd079bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0740b8200 0x7fd0740cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.537+0000 7fd079bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740cede0 0x7fd0740cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.537+0000 7fd079bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740d1870 con 0x7fd0740b1ac0 2023-07-13T22:21:28.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.537+0000 7fd079bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0740cf370 con 0x7fd0740b8200 2023-07-13T22:21:28.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.537+0000 7fd079bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740d12b0 con 0x7fd0740cede0 2023-07-13T22:21:28.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.538+0000 7fd0737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0740b8200 0x7fd0740cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.538+0000 7fd0737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0740b8200 0x7fd0740cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47776/0 (socket says 172.21.15.60:47776) 2023-07-13T22:21:28.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.538+0000 7fd0737fe700 1 -- 172.21.15.60:0/912691423 learned_addr learned my addr 172.21.15.60:0/912691423 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:28.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.538+0000 7fd073fff700 1 --2- 172.21.15.60:0/912691423 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740cede0 0x7fd0740cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.538+0000 7fd0737fe700 1 -- 172.21.15.60:0/912691423 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740cede0 msgr2=0x7fd0740cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:28.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.538+0000 7fd0737fe700 1 --2- 172.21.15.60:0/912691423 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740cede0 0x7fd0740cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.538+0000 7fd0737fe700 1 -- 172.21.15.60:0/912691423 >> v1:172.21.15.60:6789/0 conn(0x7fd0740b1ac0 legacy=0x7fd07415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.538+0000 7fd0737fe700 1 -- 172.21.15.60:0/912691423 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740d1590 con 0x7fd0740b8200 2023-07-13T22:21:28.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.538+0000 7fd0737fe700 1 --2- 172.21.15.60:0/912691423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b8200 0x7fd0740cf640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd0640098f0 tx=0x7fd064009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf31218f686648c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:28.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd0727fc700 1 -- 172.21.15.60:0/912691423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400b8a0 con 0x7fd0740b8200 2023-07-13T22:21:28.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd0727fc700 1 -- 172.21.15.60:0/912691423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06400ba00 con 0x7fd0740b8200 2023-07-13T22:21:28.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd0727fc700 1 -- 172.21.15.60:0/912691423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400bd90 con 0x7fd0740b8200 2023-07-13T22:21:28.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd079bff700 1 -- 172.21.15.60:0/912691423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b8200 msgr2=0x7fd0740cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd079bff700 1 --2- 172.21.15.60:0/912691423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b8200 0x7fd0740cf640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd0640098f0 tx=0x7fd064009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:28.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd079bff700 1 -- 172.21.15.60:0/912691423 shutdown_connections 2023-07-13T22:21:28.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd079bff700 1 --2- 172.21.15.60:0/912691423 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740cede0 0x7fd0740cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd079bff700 1 --2- 172.21.15.60:0/912691423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b8200 0x7fd0740cf640 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd079bff700 1 -- 172.21.15.60:0/912691423 >> 172.21.15.60:0/912691423 conn(0x7fd07406b2f0 msgr2=0x7fd07406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:28.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd079bff700 1 -- 172.21.15.60:0/912691423 shutdown_connections 2023-07-13T22:21:28.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.539+0000 7fd079bff700 1 -- 172.21.15.60:0/912691423 wait complete. 2023-07-13T22:21:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.541+0000 7fd079bff700 1 Processor -- start 2023-07-13T22:21:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.541+0000 7fd079bff700 1 -- start start 2023-07-13T22:21:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.542+0000 7fd079bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b1ac0 0x7fd0741e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.542+0000 7fd079bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740b8200 0x7fd0740a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.542+0000 7fd079bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0740cede0 0x7fd0741e2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.542+0000 7fd079bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741e2d90 con 0x7fd0740b1ac0 2023-07-13T22:21:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.542+0000 7fd079bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741e47d0 con 0x7fd0740cede0 2023-07-13T22:21:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.542+0000 7fd079bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740c9ea0 con 0x7fd0740b8200 2023-07-13T22:21:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.542+0000 7fd0737fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b1ac0 0x7fd0741e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd0737fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b1ac0 0x7fd0741e3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47778/0 (socket says 172.21.15.60:47778) 2023-07-13T22:21:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd0737fe700 1 -- 172.21.15.60:0/4118206138 learned_addr learned my addr 172.21.15.60:0/4118206138 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd073fff700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0740cede0 0x7fd0741e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd072ffd700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740b8200 0x7fd0740a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd0737fe700 1 -- 172.21.15.60:0/4118206138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740b8200 msgr2=0x7fd0740a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd0737fe700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740b8200 0x7fd0740a9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd0737fe700 1 -- 172.21.15.60:0/4118206138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0740cede0 msgr2=0x7fd0741e2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd0737fe700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0740cede0 0x7fd0741e2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd0737fe700 1 -- 172.21.15.60:0/4118206138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064009550 con 0x7fd0740b1ac0 2023-07-13T22:21:28.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd0737fe700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b1ac0 0x7fd0741e3b60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd064000c00 tx=0x7fd064004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:28.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd05bfff700 1 -- 172.21.15.60:0/4118206138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064004480 con 0x7fd0740b1ac0 2023-07-13T22:21:28.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd05bfff700 1 -- 172.21.15.60:0/4118206138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06401f070 con 0x7fd0740b1ac0 2023-07-13T22:21:28.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.543+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0740ca180 con 0x7fd0740b1ac0 2023-07-13T22:21:28.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.544+0000 7fd05bfff700 1 -- 172.21.15.60:0/4118206138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0640038c0 con 0x7fd0740b1ac0 2023-07-13T22:21:28.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.544+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0740ca740 con 0x7fd0740b1ac0 2023-07-13T22:21:28.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.545+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07404e210 con 0x7fd0740b1ac0 2023-07-13T22:21:28.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.546+0000 7fd05bfff700 1 -- 172.21.15.60:0/4118206138 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd064021030 con 0x7fd0740b1ac0 2023-07-13T22:21:28.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.546+0000 7fd05bfff700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd06006ace0 0x7fd06006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.547+0000 7fd05bfff700 1 -- 172.21.15.60:0/4118206138 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd06408aa40 con 0x7fd0740b1ac0 2023-07-13T22:21:28.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.547+0000 7fd072ffd700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd06006ace0 0x7fd06006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.547+0000 7fd072ffd700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd06006ace0 0x7fd06006d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd0741e3690 tx=0x7fd068009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:28.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.551+0000 7fd05bfff700 1 -- 172.21.15.60:0/4118206138 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd064055c00 con 0x7fd0740b1ac0 2023-07-13T22:21:28.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.756+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0740b4f70 con 0x7fd0740b1ac0 2023-07-13T22:21:28.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.757+0000 7fd05bfff700 1 -- 172.21.15.60:0/4118206138 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd064059220 con 0x7fd0740b1ac0 2023-07-13T22:21:28.742 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:28.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd06006ace0 msgr2=0x7fd06006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd06006ace0 0x7fd06006d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd0741e3690 tx=0x7fd068009040 comp rx=0 tx=0).stop 2023-07-13T22:21:28.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b1ac0 msgr2=0x7fd0741e3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b1ac0 0x7fd0741e3b60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd064000c00 tx=0x7fd064004750 comp rx=0 tx=0).stop 2023-07-13T22:21:28.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 shutdown_connections 2023-07-13T22:21:28.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd06006ace0 0x7fd06006d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0740b1ac0 0x7fd0741e3b60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0740cede0 0x7fd0741e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 --2- 172.21.15.60:0/4118206138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0740b8200 0x7fd0740a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 >> 172.21.15.60:0/4118206138 conn(0x7fd07406b2f0 msgr2=0x7fd0740d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:28.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 shutdown_connections 2023-07-13T22:21:28.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.759+0000 7fd079bff700 1 -- 172.21.15.60:0/4118206138 wait complete. 2023-07-13T22:21:28.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.982+0000 7f3901dff700 1 Processor -- start 2023-07-13T22:21:28.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.982+0000 7f3901dff700 1 -- start start 2023-07-13T22:21:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.983+0000 7f3901dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f38fc0b8200 0x7f38fc0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.983+0000 7f3901dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38fc0cede0 0x7f38fc0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.983+0000 7f3901dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc0d1870 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.983+0000 7f3901dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0cf370 con 0x7f38fc0cede0 2023-07-13T22:21:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.983+0000 7f3901dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0d12b0 con 0x7f38fc0b8200 2023-07-13T22:21:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.984+0000 7f38fbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38fc0cede0 0x7f38fc0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.984+0000 7f38fbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38fc0cede0 0x7f38fc0cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47780/0 (socket says 172.21.15.60:47780) 2023-07-13T22:21:28.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.984+0000 7f38fbfff700 1 -- 172.21.15.60:0/585987459 learned_addr learned my addr 172.21.15.60:0/585987459 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:28.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.984+0000 7f38fb7fe700 1 --2- 172.21.15.60:0/585987459 >> v2:172.21.15.105:3300/0 conn(0x7f38fc0b8200 0x7f38fc0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.984+0000 7f38fbfff700 1 -- 172.21.15.60:0/585987459 >> v2:172.21.15.105:3300/0 conn(0x7f38fc0b8200 msgr2=0x7f38fc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.984+0000 7f38fbfff700 1 --2- 172.21.15.60:0/585987459 >> v2:172.21.15.105:3300/0 conn(0x7f38fc0b8200 0x7f38fc0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.984+0000 7f38fbfff700 1 -- 172.21.15.60:0/585987459 >> v1:172.21.15.105:6789/0 conn(0x7f38fc0b1ac0 legacy=0x7f38fc15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.984+0000 7f38fbfff700 1 -- 172.21.15.60:0/585987459 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0d1590 con 0x7f38fc0cede0 2023-07-13T22:21:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.984+0000 7f38fbfff700 1 --2- 172.21.15.60:0/585987459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0cede0 0x7f38fc0cfb80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0b2660 tx=0x7f38f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d18d7ca9f9ba12e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f38fa7fc700 1 -- 172.21.15.60:0/585987459 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400e070 con 0x7f38fc0cede0 2023-07-13T22:21:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f38fa7fc700 1 -- 172.21.15.60:0/585987459 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4003a20 con 0x7f38fc0cede0 2023-07-13T22:21:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f38fa7fc700 1 -- 172.21.15.60:0/585987459 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4003dd0 con 0x7f38fc0cede0 2023-07-13T22:21:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f3901dff700 1 -- 172.21.15.60:0/585987459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0cede0 msgr2=0x7f38fc0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f3901dff700 1 --2- 172.21.15.60:0/585987459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0cede0 0x7f38fc0cfb80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0b2660 tx=0x7f38f400b330 comp rx=0 tx=0).stop 2023-07-13T22:21:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f3901dff700 1 -- 172.21.15.60:0/585987459 shutdown_connections 2023-07-13T22:21:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f3901dff700 1 --2- 172.21.15.60:0/585987459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0cede0 0x7f38fc0cfb80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f3901dff700 1 --2- 172.21.15.60:0/585987459 >> v2:172.21.15.105:3300/0 conn(0x7f38fc0b8200 0x7f38fc0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f3901dff700 1 -- 172.21.15.60:0/585987459 >> 172.21.15.60:0/585987459 conn(0x7f38fc06b2f0 msgr2=0x7f38fc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:28.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.985+0000 7f3901dff700 1 -- 172.21.15.60:0/585987459 shutdown_connections 2023-07-13T22:21:28.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.986+0000 7f3901dff700 1 -- 172.21.15.60:0/585987459 wait complete. 2023-07-13T22:21:28.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.988+0000 7f3901dff700 1 Processor -- start 2023-07-13T22:21:28.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.988+0000 7f3901dff700 1 -- start start 2023-07-13T22:21:28.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.989+0000 7f3901dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0b1ac0 0x7f38fc155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.989+0000 7f3901dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38fc0b8200 0x7f38fc0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.989+0000 7f3901dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38fc0cede0 0x7f38fc159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.989+0000 7f3901dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0cb3e0 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.989+0000 7f3901dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0cb630 con 0x7f38fc0cede0 2023-07-13T22:21:28.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.989+0000 7f3901dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0cb910 con 0x7f38fc0b8200 2023-07-13T22:21:28.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0b1ac0 0x7f38fc155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0b1ac0 0x7f38fc155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47792/0 (socket says 172.21.15.60:47792) 2023-07-13T22:21:28.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fb7fe700 1 -- 172.21.15.60:0/389702388 learned_addr learned my addr 172.21.15.60:0/389702388 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:28.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fbfff700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38fc0cede0 0x7f38fc159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38faffd700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38fc0b8200 0x7f38fc0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fb7fe700 1 -- 172.21.15.60:0/389702388 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38fc0b8200 msgr2=0x7f38fc0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fb7fe700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38fc0b8200 0x7f38fc0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fb7fe700 1 -- 172.21.15.60:0/389702388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38fc0cede0 msgr2=0x7f38fc159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:28.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fb7fe700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38fc0cede0 0x7f38fc159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:28.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fb7fe700 1 -- 172.21.15.60:0/389702388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4009b30 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fbfff700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38fc0cede0 0x7f38fc159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.990+0000 7f38fb7fe700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0b1ac0 0x7f38fc155b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00e810 tx=0x7f38ec00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.991+0000 7f38e3fff700 1 -- 172.21.15.60:0/389702388 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec013610 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.991+0000 7f38e3fff700 1 -- 172.21.15.60:0/389702388 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec013770 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.991+0000 7f38e3fff700 1 -- 172.21.15.60:0/389702388 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec013b20 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.991+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc0cbbf0 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.991+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc0cc1b0 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.992+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc0cc520 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.993+0000 7f38e3fff700 1 -- 172.21.15.60:0/389702388 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38ec013c80 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.994+0000 7f38e3fff700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38e806ace0 0x7f38e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:28.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.994+0000 7f38e3fff700 1 -- 172.21.15.60:0/389702388 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38ec0891a0 con 0x7f38fc0b1ac0 2023-07-13T22:21:28.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.994+0000 7f38faffd700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38e806ace0 0x7f38e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:28.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.995+0000 7f38faffd700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38e806ace0 0x7f38e806d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0cfef0 tx=0x7f38f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:28.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:28.998+0000 7f38e3fff700 1 -- 172.21.15.60:0/389702388 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f38ec0541c0 con 0x7f38fc0b1ac0 2023-07-13T22:21:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.203+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38fc04e210 con 0x7f38fc0b1ac0 2023-07-13T22:21:29.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.204+0000 7f38e3fff700 1 -- 172.21.15.60:0/389702388 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38ec0577e0 con 0x7f38fc0b1ac0 2023-07-13T22:21:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38e806ace0 msgr2=0x7f38e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38e806ace0 0x7f38e806d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0cfef0 tx=0x7f38f0008040 comp rx=0 tx=0).stop 2023-07-13T22:21:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0b1ac0 msgr2=0x7f38fc155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0b1ac0 0x7f38fc155b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00e810 tx=0x7f38ec00ebd0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 shutdown_connections 2023-07-13T22:21:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38e806ace0 0x7f38e806d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc0b1ac0 0x7f38fc155b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38fc0cede0 0x7f38fc159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 --2- 172.21.15.60:0/389702388 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38fc0b8200 0x7f38fc0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 >> 172.21.15.60:0/389702388 conn(0x7f38fc06b2f0 msgr2=0x7f38fc0cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 shutdown_connections 2023-07-13T22:21:29.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.206+0000 7f3901dff700 1 -- 172.21.15.60:0/389702388 wait complete. 2023-07-13T22:21:29.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:29.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.425+0000 7f9a937ff700 1 Processor -- start 2023-07-13T22:21:29.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.425+0000 7f9a937ff700 1 -- start start 2023-07-13T22:21:29.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.426+0000 7f9a937ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a8c0b8400 0x7f9a8c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.426+0000 7f9a937ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a8c0cfca0 0x7f9a8c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.426+0000 7f9a937ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8c0d0ac0 con 0x7f9a8c0d3750 2023-07-13T22:21:29.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.426+0000 7f9a937ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c0b87d0 con 0x7f9a8c0cfca0 2023-07-13T22:21:29.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.426+0000 7f9a937ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c0b4eb0 con 0x7f9a8c0b8400 2023-07-13T22:21:29.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a91d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a8c0cfca0 0x7f9a8c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a90d9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9a8c0d3750 0x7f9a8c0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47474/0 (socket says 172.21.15.60:47474) 2023-07-13T22:21:29.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a90d9a700 1 -- 172.21.15.60:0/2271746772 learned_addr learned my addr 172.21.15.60:0/2271746772 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:29.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a9159b700 1 --2- 172.21.15.60:0/2271746772 >> v2:172.21.15.105:3300/0 conn(0x7f9a8c0b8400 0x7f9a8c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a91d9c700 1 -- 172.21.15.60:0/2271746772 >> v2:172.21.15.105:3300/0 conn(0x7f9a8c0b8400 msgr2=0x7f9a8c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:29.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a91d9c700 1 --2- 172.21.15.60:0/2271746772 >> v2:172.21.15.105:3300/0 conn(0x7f9a8c0b8400 0x7f9a8c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a91d9c700 1 -- 172.21.15.60:0/2271746772 >> v1:172.21.15.60:6789/0 conn(0x7f9a8c0d3750 legacy=0x7f9a8c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a91d9c700 1 -- 172.21.15.60:0/2271746772 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c0b5190 con 0x7f9a8c0cfca0 2023-07-13T22:21:29.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a83fff700 1 -- 172.21.15.60:0/2271746772 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33395999 0 0) 0x7f9a8c0d0ac0 con 0x7f9a8c0d3750 2023-07-13T22:21:29.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.427+0000 7f9a91d9c700 1 --2- 172.21.15.60:0/2271746772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0cfca0 0x7f9a8c0d0f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0cf560 tx=0x7f9a8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04de71ed3cc6dff server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:29.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a83fff700 1 -- 172.21.15.60:0/2271746772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8400e070 con 0x7f9a8c0cfca0 2023-07-13T22:21:29.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a83fff700 1 -- 172.21.15.60:0/2271746772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8400bb20 con 0x7f9a8c0cfca0 2023-07-13T22:21:29.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a83fff700 1 -- 172.21.15.60:0/2271746772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84012430 con 0x7f9a8c0cfca0 2023-07-13T22:21:29.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a937ff700 1 -- 172.21.15.60:0/2271746772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0cfca0 msgr2=0x7f9a8c0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a937ff700 1 --2- 172.21.15.60:0/2271746772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0cfca0 0x7f9a8c0d0f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0cf560 tx=0x7f9a8400b330 comp rx=0 tx=0).stop 2023-07-13T22:21:29.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a937ff700 1 -- 172.21.15.60:0/2271746772 shutdown_connections 2023-07-13T22:21:29.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a937ff700 1 --2- 172.21.15.60:0/2271746772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0cfca0 0x7f9a8c0d0f50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a937ff700 1 --2- 172.21.15.60:0/2271746772 >> v2:172.21.15.105:3300/0 conn(0x7f9a8c0b8400 0x7f9a8c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a937ff700 1 -- 172.21.15.60:0/2271746772 >> 172.21.15.60:0/2271746772 conn(0x7f9a8c06b390 msgr2=0x7f9a8c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:29.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a937ff700 1 -- 172.21.15.60:0/2271746772 shutdown_connections 2023-07-13T22:21:29.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.428+0000 7f9a937ff700 1 -- 172.21.15.60:0/2271746772 wait complete. 2023-07-13T22:21:29.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.430+0000 7f9a937ff700 1 Processor -- start 2023-07-13T22:21:29.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.430+0000 7f9a937ff700 1 -- start start 2023-07-13T22:21:29.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.431+0000 7f9a937ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c0b8400 0x7f9a8c0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.431+0000 7f9a937ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a8c0cfca0 0x7f9a8c1f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.431+0000 7f9a937ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0d3750 0x7f9a8c15d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.431+0000 7f9a937ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c1f28c0 con 0x7f9a8c0d3750 2023-07-13T22:21:29.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.431+0000 7f9a937ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c1f2300 con 0x7f9a8c0b8400 2023-07-13T22:21:29.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.431+0000 7f9a937ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c1f25e0 con 0x7f9a8c0cfca0 2023-07-13T22:21:29.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a91d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0d3750 0x7f9a8c15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a91d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0d3750 0x7f9a8c15d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47822/0 (socket says 172.21.15.60:47822) 2023-07-13T22:21:29.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a91d9c700 1 -- 172.21.15.60:0/2343712210 learned_addr learned my addr 172.21.15.60:0/2343712210 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:29.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a90d9a700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a8c0cfca0 0x7f9a8c1f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a9159b700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c0b8400 0x7f9a8c0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a91d9c700 1 -- 172.21.15.60:0/2343712210 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a8c0cfca0 msgr2=0x7f9a8c1f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a91d9c700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a8c0cfca0 0x7f9a8c1f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a91d9c700 1 -- 172.21.15.60:0/2343712210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c0b8400 msgr2=0x7f9a8c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a91d9c700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c0b8400 0x7f9a8c0cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a91d9c700 1 -- 172.21.15.60:0/2343712210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84009a70 con 0x7f9a8c0d3750 2023-07-13T22:21:29.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.432+0000 7f9a91d9c700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0d3750 0x7f9a8c15d570 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a84013000 tx=0x7f9a84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:29.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.433+0000 7f9a81ffb700 1 -- 172.21.15.60:0/2343712210 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84012430 con 0x7f9a8c0d3750 2023-07-13T22:21:29.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.433+0000 7f9a81ffb700 1 -- 172.21.15.60:0/2343712210 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a84020070 con 0x7f9a8c0d3750 2023-07-13T22:21:29.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.433+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c15cd70 con 0x7f9a8c0d3750 2023-07-13T22:21:29.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.433+0000 7f9a81ffb700 1 -- 172.21.15.60:0/2343712210 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004280 con 0x7f9a8c0d3750 2023-07-13T22:21:29.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.433+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c0cc640 con 0x7f9a8c0d3750 2023-07-13T22:21:29.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.434+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c15d290 con 0x7f9a8c0d3750 2023-07-13T22:21:29.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.435+0000 7f9a81ffb700 1 -- 172.21.15.60:0/2343712210 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a840043e0 con 0x7f9a8c0d3750 2023-07-13T22:21:29.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.435+0000 7f9a81ffb700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7006ace0 0x7f9a7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.435+0000 7f9a81ffb700 1 -- 172.21.15.60:0/2343712210 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a84022070 con 0x7f9a8c0d3750 2023-07-13T22:21:29.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.436+0000 7f9a9159b700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7006ace0 0x7f9a7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.436+0000 7f9a9159b700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7006ace0 0x7f9a7006d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009ad0 tx=0x7f9a88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.440+0000 7f9a81ffb700 1 -- 172.21.15.60:0/2343712210 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9a8c15d290 con 0x7f9a8c0d3750 2023-07-13T22:21:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.644+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a8c15d290 con 0x7f9a8c0d3750 2023-07-13T22:21:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.645+0000 7f9a81ffb700 1 -- 172.21.15.60:0/2343712210 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f9a8c15d290 con 0x7f9a8c0d3750 2023-07-13T22:21:29.631 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:29.631 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:29.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7006ace0 msgr2=0x7f9a7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7006ace0 0x7f9a7006d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009ad0 tx=0x7f9a88008040 comp rx=0 tx=0).stop 2023-07-13T22:21:29.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0d3750 msgr2=0x7f9a8c15d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0d3750 0x7f9a8c15d570 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a84013000 tx=0x7f9a84004ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 shutdown_connections 2023-07-13T22:21:29.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7006ace0 0x7f9a7006d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a8c0d3750 0x7f9a8c15d570 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a8c0b8400 0x7f9a8c0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 --2- 172.21.15.60:0/2343712210 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a8c0cfca0 0x7f9a8c1f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 >> 172.21.15.60:0/2343712210 conn(0x7f9a8c06b390 msgr2=0x7f9a8c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:29.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 shutdown_connections 2023-07-13T22:21:29.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.648+0000 7f9a937ff700 1 -- 172.21.15.60:0/2343712210 wait complete. 2023-07-13T22:21:29.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:29.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:29.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.872+0000 7fbf313ff700 1 Processor -- start 2023-07-13T22:21:29.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.872+0000 7fbf313ff700 1 -- start start 2023-07-13T22:21:29.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf313ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf2c0b8200 0x7fbf2c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf313ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf313ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2c0d1870 con 0x7fbf2c0cede0 2023-07-13T22:21:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf313ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2c0cf370 con 0x7fbf2c0b8200 2023-07-13T22:21:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf313ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c0d12b0 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf2affd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf2c0b8200 0x7fbf2c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf2affd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf2c0b8200 0x7fbf2c0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47836/0 (socket says 172.21.15.60:47836) 2023-07-13T22:21:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf2affd700 1 -- 172.21.15.60:0/2500553799 learned_addr learned my addr 172.21.15.60:0/2500553799 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:29.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf2b7fe700 1 --1- 172.21.15.60:0/2500553799 >> v1:172.21.15.60:6789/0 conn(0x7fbf2c0cede0 0x7fbf2c15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47478/0 (socket says 172.21.15.60:47478) 2023-07-13T22:21:29.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf23fff700 1 --2- 172.21.15.60:0/2500553799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf2affd700 1 -- 172.21.15.60:0/2500553799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0b1ac0 msgr2=0x7fbf2c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:29.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf2affd700 1 --2- 172.21.15.60:0/2500553799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf2affd700 1 -- 172.21.15.60:0/2500553799 >> v1:172.21.15.60:6789/0 conn(0x7fbf2c0cede0 legacy=0x7fbf2c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf2affd700 1 -- 172.21.15.60:0/2500553799 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c0d1590 con 0x7fbf2c0b8200 2023-07-13T22:21:29.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf23fff700 1 --2- 172.21.15.60:0/2500553799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:29.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.874+0000 7fbf2a7fc700 1 -- 172.21.15.60:0/2500553799 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1867428748 0 0) 0x7fbf2c0d1870 con 0x7fbf2c0cede0 2023-07-13T22:21:29.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.875+0000 7fbf2affd700 1 --2- 172.21.15.60:0/2500553799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b8200 0x7fbf2c0cf640 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0098f0 tx=0x7fbf1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7f215bcad424aa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:29.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.875+0000 7fbf2a7fc700 1 -- 172.21.15.60:0/2500553799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00b8a0 con 0x7fbf2c0b8200 2023-07-13T22:21:29.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.875+0000 7fbf2a7fc700 1 -- 172.21.15.60:0/2500553799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c00ba00 con 0x7fbf2c0b8200 2023-07-13T22:21:29.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.875+0000 7fbf2a7fc700 1 -- 172.21.15.60:0/2500553799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00bd90 con 0x7fbf2c0b8200 2023-07-13T22:21:29.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.875+0000 7fbf313ff700 1 -- 172.21.15.60:0/2500553799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b8200 msgr2=0x7fbf2c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.875+0000 7fbf313ff700 1 --2- 172.21.15.60:0/2500553799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b8200 0x7fbf2c0cf640 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0098f0 tx=0x7fbf1c009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:29.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.876+0000 7fbf313ff700 1 -- 172.21.15.60:0/2500553799 shutdown_connections 2023-07-13T22:21:29.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.876+0000 7fbf313ff700 1 --2- 172.21.15.60:0/2500553799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b8200 0x7fbf2c0cf640 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.876+0000 7fbf313ff700 1 --2- 172.21.15.60:0/2500553799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.876+0000 7fbf313ff700 1 -- 172.21.15.60:0/2500553799 >> 172.21.15.60:0/2500553799 conn(0x7fbf2c06b2f0 msgr2=0x7fbf2c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:29.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.876+0000 7fbf313ff700 1 -- 172.21.15.60:0/2500553799 shutdown_connections 2023-07-13T22:21:29.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.876+0000 7fbf313ff700 1 -- 172.21.15.60:0/2500553799 wait complete. 2023-07-13T22:21:29.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.877+0000 7fbf313ff700 1 Processor -- start 2023-07-13T22:21:29.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.878+0000 7fbf313ff700 1 -- start start 2023-07-13T22:21:29.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf313ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf313ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf2c0b8200 0x7fbf2c1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf313ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0cede0 0x7fbf2c145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf313ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c0ccdc0 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf313ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c0c8210 con 0x7fbf2c0b8200 2023-07-13T22:21:29.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf313ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c0c84f0 con 0x7fbf2c0cede0 2023-07-13T22:21:29.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf2affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf2affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47842/0 (socket says 172.21.15.60:47842) 2023-07-13T22:21:29.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf2affd700 1 -- 172.21.15.60:0/1342884929 learned_addr learned my addr 172.21.15.60:0/1342884929 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:29.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf23fff700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf2c0b8200 0x7fbf2c1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf2b7fe700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0cede0 0x7fbf2c145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf2affd700 1 -- 172.21.15.60:0/1342884929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0cede0 msgr2=0x7fbf2c145e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf2affd700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0cede0 0x7fbf2c145e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf2affd700 1 -- 172.21.15.60:0/1342884929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf2c0b8200 msgr2=0x7fbf2c1549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:29.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf2affd700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf2c0b8200 0x7fbf2c1549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:29.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.879+0000 7fbf2affd700 1 -- 172.21.15.60:0/1342884929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c009550 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.880+0000 7fbf2affd700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0c78e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009c40 tx=0x7fbf1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:29.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.880+0000 7fbf237fe700 1 -- 172.21.15.60:0/1342884929 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c004380 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.880+0000 7fbf237fe700 1 -- 172.21.15.60:0/1342884929 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c01f070 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.880+0000 7fbf237fe700 1 -- 172.21.15.60:0/1342884929 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0036a0 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.880+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2c0c87d0 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.880+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2c0c8d90 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.881+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2c04e210 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.882+0000 7fbf237fe700 1 -- 172.21.15.60:0/1342884929 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf1c021030 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.883+0000 7fbf237fe700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf0c06ace0 0x7fbf0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:29.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.883+0000 7fbf237fe700 1 -- 172.21.15.60:0/1342884929 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf1c08a880 con 0x7fbf2c0b1ac0 2023-07-13T22:21:29.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.883+0000 7fbf23fff700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf0c06ace0 0x7fbf0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:29.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.884+0000 7fbf23fff700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf0c06ace0 0x7fbf0c06d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c14a530 tx=0x7fbf14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:29.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:29.887+0000 7fbf237fe700 1 -- 172.21.15.60:0/1342884929 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbf1c055920 con 0x7fbf2c0b1ac0 2023-07-13T22:21:30.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.093+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbf2c0d3800 con 0x7fbf2c0b1ac0 2023-07-13T22:21:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.093+0000 7fbf237fe700 1 -- 172.21.15.60:0/1342884929 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbf1c006d80 con 0x7fbf2c0b1ac0 2023-07-13T22:21:30.079 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:30.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf0c06ace0 msgr2=0x7fbf0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf0c06ace0 0x7fbf0c06d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c14a530 tx=0x7fbf14008040 comp rx=0 tx=0).stop 2023-07-13T22:21:30.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b1ac0 msgr2=0x7fbf2c0c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0c78e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009c40 tx=0x7fbf1c004750 comp rx=0 tx=0).stop 2023-07-13T22:21:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 shutdown_connections 2023-07-13T22:21:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf0c06ace0 0x7fbf0c06d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf2c0b1ac0 0x7fbf2c0c78e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf2c0cede0 0x7fbf2c145e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 --2- 172.21.15.60:0/1342884929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf2c0b8200 0x7fbf2c1549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 >> 172.21.15.60:0/1342884929 conn(0x7fbf2c06b2f0 msgr2=0x7fbf2c0b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:30.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 shutdown_connections 2023-07-13T22:21:30.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.096+0000 7fbf313ff700 1 -- 172.21.15.60:0/1342884929 wait complete. 2023-07-13T22:21:30.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.319+0000 7fe4f7fff700 1 Processor -- start 2023-07-13T22:21:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.319+0000 7fe4f7fff700 1 -- start start 2023-07-13T22:21:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.320+0000 7fe4f7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 0x7fe4f00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f7fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00d1870 con 0x7fe4f00cede0 2023-07-13T22:21:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f7fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f0160d80 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f7fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00cf370 con 0x7fe4f00b8200 2023-07-13T22:21:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f659c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe4f00cede0 0x7fe4f0160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47486/0 (socket says 172.21.15.60:47486) 2023-07-13T22:21:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f659c700 1 -- 172.21.15.60:0/4193054564 learned_addr learned my addr 172.21.15.60:0/4193054564 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f5d9b700 1 --2- 172.21.15.60:0/4193054564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 0x7fe4f00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f4d99700 1 -- 172.21.15.60:0/4193054564 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156394068 0 0) 0x7fe4f00d1870 con 0x7fe4f00cede0 2023-07-13T22:21:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f4d99700 1 -- 172.21.15.60:0/4193054564 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4dc0035b0 con 0x7fe4f00cede0 2023-07-13T22:21:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f5d9b700 1 -- 172.21.15.60:0/4193054564 >> v1:172.21.15.105:6789/0 conn(0x7fe4f00b1ac0 legacy=0x7fe4f015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.321+0000 7fe4f4d99700 1 -- 172.21.15.60:0/4193054564 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612010218 0 0) 0x7fe4f0160d80 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.322+0000 7fe4f5d9b700 1 -- 172.21.15.60:0/4193054564 >> v1:172.21.15.60:6789/0 conn(0x7fe4f00cede0 legacy=0x7fe4f0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.322+0000 7fe4f5d9b700 1 -- 172.21.15.60:0/4193054564 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00d12b0 con 0x7fe4f00b8200 2023-07-13T22:21:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.322+0000 7fe4f5d9b700 1 --2- 172.21.15.60:0/4193054564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 0x7fe4f00cf640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009870 tx=0x7fe4ec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab73b27fc3cb9469 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.322+0000 7fe4f4d99700 1 -- 172.21.15.60:0/4193054564 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec011830 con 0x7fe4f00b8200 2023-07-13T22:21:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.322+0000 7fe4f4d99700 1 -- 172.21.15.60:0/4193054564 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec011990 con 0x7fe4f00b8200 2023-07-13T22:21:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.322+0000 7fe4f4d99700 1 -- 172.21.15.60:0/4193054564 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec011d00 con 0x7fe4f00b8200 2023-07-13T22:21:30.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.323+0000 7fe4f7fff700 1 -- 172.21.15.60:0/4193054564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 msgr2=0x7fe4f00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.323+0000 7fe4f7fff700 1 --2- 172.21.15.60:0/4193054564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 0x7fe4f00cf640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009870 tx=0x7fe4ec009b80 comp rx=0 tx=0).stop 2023-07-13T22:21:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.323+0000 7fe4f7fff700 1 -- 172.21.15.60:0/4193054564 shutdown_connections 2023-07-13T22:21:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.323+0000 7fe4f7fff700 1 --2- 172.21.15.60:0/4193054564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 0x7fe4f00cf640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.323+0000 7fe4f7fff700 1 -- 172.21.15.60:0/4193054564 >> 172.21.15.60:0/4193054564 conn(0x7fe4f006b2f0 msgr2=0x7fe4f006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.323+0000 7fe4f7fff700 1 -- 172.21.15.60:0/4193054564 shutdown_connections 2023-07-13T22:21:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.323+0000 7fe4f7fff700 1 -- 172.21.15.60:0/4193054564 wait complete. 2023-07-13T22:21:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.324+0000 7fe4f7fff700 1 Processor -- start 2023-07-13T22:21:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.325+0000 7fe4f7fff700 1 -- start start 2023-07-13T22:21:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4f00b1ac0 0x7fe4f015c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 0x7fe4f0157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f00cede0 0x7fe4f01577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f7fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0158ec0 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f7fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0157e20 con 0x7fe4f00cede0 2023-07-13T22:21:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f7fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0158100 con 0x7fe4f00b8200 2023-07-13T22:21:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4f00b1ac0 0x7fe4f015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4f00b1ac0 0x7fe4f015c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47854/0 (socket says 172.21.15.60:47854) 2023-07-13T22:21:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f5d9b700 1 -- 172.21.15.60:0/1648691904 learned_addr learned my addr 172.21.15.60:0/1648691904 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f659c700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f00cede0 0x7fe4f01577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f559a700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 0x7fe4f0157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f5d9b700 1 -- 172.21.15.60:0/1648691904 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 msgr2=0x7fe4f0157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f5d9b700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 0x7fe4f0157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f5d9b700 1 -- 172.21.15.60:0/1648691904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f00cede0 msgr2=0x7fe4f01577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f5d9b700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f00cede0 0x7fe4f01577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f5d9b700 1 -- 172.21.15.60:0/1648691904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec0094a0 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.326+0000 7fe4f659c700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f00cede0 0x7fe4f01577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.327+0000 7fe4f5d9b700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4f00b1ac0 0x7fe4f015c030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0097c0 tx=0x7fe4ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.327+0000 7fe4e67fc700 1 -- 172.21.15.60:0/1648691904 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0045d0 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.327+0000 7fe4e67fc700 1 -- 172.21.15.60:0/1648691904 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec01f070 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.327+0000 7fe4e67fc700 1 -- 172.21.15.60:0/1648691904 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0038c0 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.327+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f01583e0 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.327+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01589a0 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.328+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f00cd8b0 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.329+0000 7fe4e67fc700 1 -- 172.21.15.60:0/1648691904 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4ec003a20 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.330+0000 7fe4e67fc700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4d406ace0 0x7fe4d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.330+0000 7fe4e67fc700 1 -- 172.21.15.60:0/1648691904 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4ec021070 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.330+0000 7fe4f559a700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4d406ace0 0x7fe4d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.331+0000 7fe4f559a700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4d406ace0 0x7fe4d406d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0155b10 tx=0x7fe4e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:30.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.334+0000 7fe4e67fc700 1 -- 172.21.15.60:0/1648691904 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe4ec055e10 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.537+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4f004e210 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.538+0000 7fe4e67fc700 1 -- 172.21.15.60:0/1648691904 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4ec0150a0 con 0x7fe4f00b1ac0 2023-07-13T22:21:30.523 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4d406ace0 msgr2=0x7fe4d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4d406ace0 0x7fe4d406d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0155b10 tx=0x7fe4e0009040 comp rx=0 tx=0).stop 2023-07-13T22:21:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4f00b1ac0 msgr2=0x7fe4f015c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4f00b1ac0 0x7fe4f015c030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0097c0 tx=0x7fe4ec004750 comp rx=0 tx=0).stop 2023-07-13T22:21:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 shutdown_connections 2023-07-13T22:21:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4d406ace0 0x7fe4d406d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4f00b1ac0 0x7fe4f015c030 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f00cede0 0x7fe4f01577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 --2- 172.21.15.60:0/1648691904 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4f00b8200 0x7fe4f0157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 >> 172.21.15.60:0/1648691904 conn(0x7fe4f006b2f0 msgr2=0x7fe4f00b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:30.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 shutdown_connections 2023-07-13T22:21:30.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.540+0000 7fe4f7fff700 1 -- 172.21.15.60:0/1648691904 wait complete. 2023-07-13T22:21:30.542 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.763+0000 7f9148bff700 1 Processor -- start 2023-07-13T22:21:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.763+0000 7f9148bff700 1 -- start start 2023-07-13T22:21:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.764+0000 7f9148bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91440cfd50 0x7f91440d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.764+0000 7f9148bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91440d1870 con 0x7f91440b8200 2023-07-13T22:21:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.764+0000 7f9148bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91440cf370 con 0x7f91440b1ac0 2023-07-13T22:21:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.764+0000 7f9148bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91440d12b0 con 0x7f91440cfd50 2023-07-13T22:21:30.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.764+0000 7f914259c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f91440b8200 0x7f91440cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47496/0 (socket says 172.21.15.60:47496) 2023-07-13T22:21:30.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914259c700 1 -- 172.21.15.60:0/1976302248 learned_addr learned my addr 172.21.15.60:0/1976302248 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f9141d9b700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91440cfd50 0x7f91440d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466657522 0 0) 0x7f91440d1870 con 0x7f91440b8200 2023-07-13T22:21:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f912c0035b0 con 0x7f91440b8200 2023-07-13T22:21:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298823338 0 0) 0x7f912c0035b0 con 0x7f91440b8200 2023-07-13T22:21:30.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91440cfd50 msgr2=0x7f91440d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:30.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914159a700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91440cfd50 0x7f91440d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 >> v1:172.21.15.105:6789/0 conn(0x7f91440b1ac0 legacy=0x7f91440d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440d1590 con 0x7f91440b8200 2023-07-13T22:21:30.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f9134001670 con 0x7f91440b8200 2023-07-13T22:21:30.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.765+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 >> v1:172.21.15.60:6789/0 conn(0x7f91440b8200 legacy=0x7f91440cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914159a700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f912c003a30 0x7f912c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914159a700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c004850 0x7f912c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914159a700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91440cfd50 0x7f912c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440d12b0 con 0x7f912c003a30 2023-07-13T22:21:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c00bac0 con 0x7f912c004850 2023-07-13T22:21:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c00bc50 con 0x7f91440cfd50 2023-07-13T22:21:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914259c700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f912c003a30 0x7f912c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f9141d9b700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c004850 0x7f912c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f9142d9d700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91440cfd50 0x7f912c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914259c700 1 -- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91440cfd50 msgr2=0x7f912c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914259c700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91440cfd50 0x7f912c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914259c700 1 -- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c004850 msgr2=0x7f912c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914259c700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c004850 0x7f912c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914259c700 1 -- 172.21.15.60:0/1976302248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0035b0 con 0x7f912c003a30 2023-07-13T22:21:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f9142d9d700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91440cfd50 0x7f912c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f9141d9b700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c004850 0x7f912c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.766+0000 7f914259c700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f912c003a30 0x7f912c003e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f913400c2a0 tx=0x7f912c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91123455400d4191 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134007c10 con 0x7f912c003a30 2023-07-13T22:21:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913400cdc0 con 0x7f912c003a30 2023-07-13T22:21:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f914159a700 1 -- 172.21.15.60:0/1976302248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134005ba0 con 0x7f912c003a30 2023-07-13T22:21:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f9148bff700 1 -- 172.21.15.60:0/1976302248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f912c003a30 msgr2=0x7f912c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f9148bff700 1 --2- 172.21.15.60:0/1976302248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f912c003a30 0x7f912c003e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f913400c2a0 tx=0x7f912c005880 comp rx=0 tx=0).stop 2023-07-13T22:21:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f9142d9d700 1 -- 172.21.15.60:0/1976302248 reap_dead start 2023-07-13T22:21:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f9148bff700 1 -- 172.21.15.60:0/1976302248 shutdown_connections 2023-07-13T22:21:30.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f9148bff700 1 -- 172.21.15.60:0/1976302248 >> 172.21.15.60:0/1976302248 conn(0x7f914406b2f0 msgr2=0x7f914406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:30.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f9148bff700 1 -- 172.21.15.60:0/1976302248 shutdown_connections 2023-07-13T22:21:30.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.767+0000 7f9148bff700 1 -- 172.21.15.60:0/1976302248 wait complete. 2023-07-13T22:21:30.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.769+0000 7f9148bff700 1 Processor -- start 2023-07-13T22:21:30.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.769+0000 7f9148bff700 1 -- start start 2023-07-13T22:21:30.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.770+0000 7f9148bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c003a30 0x7f9144155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.770+0000 7f9148bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91440c7bf0 0x7f91440c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.770+0000 7f9148bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914415f660 0x7f914415a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.770+0000 7f9148bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9144156260 con 0x7f91440c7bf0 2023-07-13T22:21:30.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.770+0000 7f9148bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c9750 con 0x7f912c003a30 2023-07-13T22:21:30.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.770+0000 7f9148bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c8170 con 0x7f914415f660 2023-07-13T22:21:30.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.770+0000 7f9141d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91440c7bf0 0x7f91440c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9141d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91440c7bf0 0x7f91440c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47864/0 (socket says 172.21.15.60:47864) 2023-07-13T22:21:30.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9141d9b700 1 -- 172.21.15.60:0/2789055645 learned_addr learned my addr 172.21.15.60:0/2789055645 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:30.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9142d9d700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914415f660 0x7f914415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f914259c700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c003a30 0x7f9144155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9141d9b700 1 -- 172.21.15.60:0/2789055645 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914415f660 msgr2=0x7f914415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9141d9b700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914415f660 0x7f914415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9141d9b700 1 -- 172.21.15.60:0/2789055645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c003a30 msgr2=0x7f9144155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9141d9b700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c003a30 0x7f9144155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9141d9b700 1 -- 172.21.15.60:0/2789055645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440c8450 con 0x7f91440c7bf0 2023-07-13T22:21:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f914259c700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c003a30 0x7f9144155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9141d9b700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91440c7bf0 0x7f91440c9210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f91380116a0 tx=0x7f91380119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9132ffd700 1 -- 172.21.15.60:0/2789055645 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138004220 con 0x7f91440c7bf0 2023-07-13T22:21:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9132ffd700 1 -- 172.21.15.60:0/2789055645 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9138004380 con 0x7f91440c7bf0 2023-07-13T22:21:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.771+0000 7f9148bff700 1 -- 172.21.15.60:0/2789055645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91440c8730 con 0x7f91440c7bf0 2023-07-13T22:21:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.772+0000 7f9132ffd700 1 -- 172.21.15.60:0/2789055645 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380036a0 con 0x7f91440c7bf0 2023-07-13T22:21:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.772+0000 7f9148bff700 1 -- 172.21.15.60:0/2789055645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91440c8cf0 con 0x7f91440c7bf0 2023-07-13T22:21:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.773+0000 7f9130ff9700 1 -- 172.21.15.60:0/2789055645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914404e210 con 0x7f91440c7bf0 2023-07-13T22:21:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.774+0000 7f9132ffd700 1 -- 172.21.15.60:0/2789055645 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9138003800 con 0x7f91440c7bf0 2023-07-13T22:21:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.774+0000 7f9132ffd700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f912406ace0 0x7f912406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.775+0000 7f9132ffd700 1 -- 172.21.15.60:0/2789055645 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91380897f0 con 0x7f91440c7bf0 2023-07-13T22:21:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.775+0000 7f914259c700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f912406ace0 0x7f912406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.775+0000 7f914259c700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f912406ace0 0x7f912406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9134001930 tx=0x7f9134007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.779+0000 7f9132ffd700 1 -- 172.21.15.60:0/2789055645 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9138054800 con 0x7f91440c7bf0 2023-07-13T22:21:30.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.981+0000 7f9130ff9700 1 -- 172.21.15.60:0/2789055645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91440d3230 con 0x7f91440c7bf0 2023-07-13T22:21:30.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.981+0000 7f9132ffd700 1 -- 172.21.15.60:0/2789055645 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9138057e20 con 0x7f91440c7bf0 2023-07-13T22:21:30.966 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:30.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.983+0000 7f9130ff9700 1 -- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f912406ace0 msgr2=0x7f912406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.983+0000 7f9130ff9700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f912406ace0 0x7f912406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9134001930 tx=0x7f9134007000 comp rx=0 tx=0).stop 2023-07-13T22:21:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.983+0000 7f9130ff9700 1 -- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91440c7bf0 msgr2=0x7f91440c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.983+0000 7f9130ff9700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91440c7bf0 0x7f91440c9210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f91380116a0 tx=0x7f91380119b0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.983+0000 7f9130ff9700 1 -- 172.21.15.60:0/2789055645 shutdown_connections 2023-07-13T22:21:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.983+0000 7f9130ff9700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f912406ace0 0x7f912406d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.983+0000 7f9130ff9700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914415f660 0x7f914415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.983+0000 7f9130ff9700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f912c003a30 0x7f9144155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.983+0000 7f9130ff9700 1 --2- 172.21.15.60:0/2789055645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91440c7bf0 0x7f91440c9210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.984+0000 7f9130ff9700 1 -- 172.21.15.60:0/2789055645 >> 172.21.15.60:0/2789055645 conn(0x7f914406b2f0 msgr2=0x7f91440d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.984+0000 7f9130ff9700 1 -- 172.21.15.60:0/2789055645 shutdown_connections 2023-07-13T22:21:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:30.984+0000 7f9130ff9700 1 -- 172.21.15.60:0/2789055645 wait complete. 2023-07-13T22:21:30.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:31.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.207+0000 7f97fddff700 1 Processor -- start 2023-07-13T22:21:31.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.207+0000 7f97fddff700 1 -- start start 2023-07-13T22:21:31.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.208+0000 7f97fddff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f80b45b0 0x7f97f8160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.208+0000 7f97fddff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97f80cd0f0 0x7f97f80b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.208+0000 7f97fddff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80d1040 con 0x7f97f806b1e0 2023-07-13T22:21:31.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.208+0000 7f97fddff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80b4ec0 con 0x7f97f80cd0f0 2023-07-13T22:21:31.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.208+0000 7f97fddff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80cd840 con 0x7f97f80b45b0 2023-07-13T22:21:31.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.208+0000 7f97f6ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97f80cd0f0 0x7f97f80b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.208+0000 7f97f6ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97f80cd0f0 0x7f97f80b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47878/0 (socket says 172.21.15.60:47878) 2023-07-13T22:21:31.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.208+0000 7f97f6ffd700 1 -- 172.21.15.60:0/3186291825 learned_addr learned my addr 172.21.15.60:0/3186291825 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:31.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f77fe700 1 --2- 172.21.15.60:0/3186291825 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f80b45b0 0x7f97f8160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f6ffd700 1 -- 172.21.15.60:0/3186291825 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f80b45b0 msgr2=0x7f97f8160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:31.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f6ffd700 1 --2- 172.21.15.60:0/3186291825 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f80b45b0 0x7f97f8160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f6ffd700 1 -- 172.21.15.60:0/3186291825 >> v1:172.21.15.105:6789/0 conn(0x7f97f806b1e0 legacy=0x7f97f806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:31.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f6ffd700 1 -- 172.21.15.60:0/3186291825 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80d0710 con 0x7f97f80cd0f0 2023-07-13T22:21:31.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f77fe700 1 --2- 172.21.15.60:0/3186291825 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f80b45b0 0x7f97f8160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:31.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f6ffd700 1 --2- 172.21.15.60:0/3186291825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 0x7f97f80b4980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0098f0 tx=0x7f97ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c11f86eaf54b6a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:31.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f67fc700 1 -- 172.21.15.60:0/3186291825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec004230 con 0x7f97f80cd0f0 2023-07-13T22:21:31.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f67fc700 1 -- 172.21.15.60:0/3186291825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec004390 con 0x7f97f80cd0f0 2023-07-13T22:21:31.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.209+0000 7f97f67fc700 1 -- 172.21.15.60:0/3186291825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0036a0 con 0x7f97f80cd0f0 2023-07-13T22:21:31.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.210+0000 7f97fddff700 1 -- 172.21.15.60:0/3186291825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 msgr2=0x7f97f80b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.210+0000 7f97fddff700 1 --2- 172.21.15.60:0/3186291825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 0x7f97f80b4980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0098f0 tx=0x7f97ec009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:31.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.210+0000 7f97fddff700 1 -- 172.21.15.60:0/3186291825 shutdown_connections 2023-07-13T22:21:31.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.210+0000 7f97fddff700 1 --2- 172.21.15.60:0/3186291825 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f80b45b0 0x7f97f8160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.210+0000 7f97fddff700 1 --2- 172.21.15.60:0/3186291825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 0x7f97f80b4980 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.210+0000 7f97fddff700 1 -- 172.21.15.60:0/3186291825 >> 172.21.15.60:0/3186291825 conn(0x7f97f8072240 msgr2=0x7f97f80d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:31.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.210+0000 7f97fddff700 1 -- 172.21.15.60:0/3186291825 shutdown_connections 2023-07-13T22:21:31.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.210+0000 7f97fddff700 1 -- 172.21.15.60:0/3186291825 wait complete. 2023-07-13T22:21:31.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.212+0000 7f97fddff700 1 Processor -- start 2023-07-13T22:21:31.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.212+0000 7f97fddff700 1 -- start start 2023-07-13T22:21:31.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97fddff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f806b1e0 0x7f97f815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97fddff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97f80b45b0 0x7f97f8149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97fddff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 0x7f97f815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97fddff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815f7e0 con 0x7f97f80cd0f0 2023-07-13T22:21:31.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97fddff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815efd0 con 0x7f97f80b45b0 2023-07-13T22:21:31.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97fddff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81579e0 con 0x7f97f806b1e0 2023-07-13T22:21:31.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 0x7f97f815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 0x7f97f815a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47886/0 (socket says 172.21.15.60:47886) 2023-07-13T22:21:31.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f7fff700 1 -- 172.21.15.60:0/1604789138 learned_addr learned my addr 172.21.15.60:0/1604789138 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:31.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f6ffd700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97f80b45b0 0x7f97f8149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f77fe700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f806b1e0 0x7f97f815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f7fff700 1 -- 172.21.15.60:0/1604789138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f806b1e0 msgr2=0x7f97f815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f7fff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f806b1e0 0x7f97f815daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f7fff700 1 -- 172.21.15.60:0/1604789138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97f80b45b0 msgr2=0x7f97f8149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f7fff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97f80b45b0 0x7f97f8149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.213+0000 7f97f7fff700 1 -- 172.21.15.60:0/1604789138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec009550 con 0x7f97f80cd0f0 2023-07-13T22:21:31.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.214+0000 7f97f7fff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 0x7f97f815a720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f97f000cc10 tx=0x7f97f000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:31.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.214+0000 7f97dffff700 1 -- 172.21.15.60:0/1604789138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000aa60 con 0x7f97f80cd0f0 2023-07-13T22:21:31.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.214+0000 7f97dffff700 1 -- 172.21.15.60:0/1604789138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f000abc0 con 0x7f97f80cd0f0 2023-07-13T22:21:31.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.214+0000 7f97dffff700 1 -- 172.21.15.60:0/1604789138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0012420 con 0x7f97f80cd0f0 2023-07-13T22:21:31.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.214+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f815f220 con 0x7f97f80cd0f0 2023-07-13T22:21:31.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.214+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f815d2f0 con 0x7f97f80cd0f0 2023-07-13T22:21:31.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.215+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f815d150 con 0x7f97f80cd0f0 2023-07-13T22:21:31.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.216+0000 7f97dffff700 1 -- 172.21.15.60:0/1604789138 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97f0012580 con 0x7f97f80cd0f0 2023-07-13T22:21:31.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.217+0000 7f97dffff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e406ace0 0x7f97e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.217+0000 7f97dffff700 1 -- 172.21.15.60:0/1604789138 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97f0089840 con 0x7f97f80cd0f0 2023-07-13T22:21:31.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.217+0000 7f97f77fe700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e406ace0 0x7f97e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.218+0000 7f97f77fe700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e406ace0 0x7f97e406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97e8006fd0 tx=0x7f97e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:31.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.221+0000 7f97dffff700 1 -- 172.21.15.60:0/1604789138 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f97f815d150 con 0x7f97f80cd0f0 2023-07-13T22:21:31.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.427+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97f815d150 con 0x7f97f80cd0f0 2023-07-13T22:21:31.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.427+0000 7f97dffff700 1 -- 172.21.15.60:0/1604789138 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97f815d150 con 0x7f97f80cd0f0 2023-07-13T22:21:31.412 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:31.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e406ace0 msgr2=0x7f97e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e406ace0 0x7f97e406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97e8006fd0 tx=0x7f97e8008040 comp rx=0 tx=0).stop 2023-07-13T22:21:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 msgr2=0x7f97f815a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 0x7f97f815a720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f97f000cc10 tx=0x7f97f000cf20 comp rx=0 tx=0).stop 2023-07-13T22:21:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 shutdown_connections 2023-07-13T22:21:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f97e406ace0 0x7f97e406d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80cd0f0 0x7f97f815a720 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97f806b1e0 0x7f97f815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 --2- 172.21.15.60:0/1604789138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97f80b45b0 0x7f97f8149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 >> 172.21.15.60:0/1604789138 conn(0x7f97f8072240 msgr2=0x7f97f80cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 shutdown_connections 2023-07-13T22:21:31.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.430+0000 7f97fddff700 1 -- 172.21.15.60:0/1604789138 wait complete. 2023-07-13T22:21:31.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:31.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.651+0000 7f454a5ff700 1 Processor -- start 2023-07-13T22:21:31.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.651+0000 7f454a5ff700 1 -- start start 2023-07-13T22:21:31.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.652+0000 7f454a5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f45440b8400 0x7f45440cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.652+0000 7f454a5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45440cfca0 0x7f45440d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.652+0000 7f454a5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45440d0ac0 con 0x7f45440d3750 2023-07-13T22:21:31.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.652+0000 7f454a5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45440b87d0 con 0x7f45440cfca0 2023-07-13T22:21:31.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.652+0000 7f454a5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45440b4eb0 con 0x7f45440b8400 2023-07-13T22:21:31.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.652+0000 7f4548b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45440cfca0 0x7f45440d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.652+0000 7f4548b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45440cfca0 0x7f45440d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47896/0 (socket says 172.21.15.60:47896) 2023-07-13T22:21:31.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.652+0000 7f4548b9c700 1 -- 172.21.15.60:0/3334812860 learned_addr learned my addr 172.21.15.60:0/3334812860 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:31.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4543fff700 1 --2- 172.21.15.60:0/3334812860 >> v2:172.21.15.105:3300/0 conn(0x7f45440b8400 0x7f45440cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4548b9c700 1 -- 172.21.15.60:0/3334812860 >> v2:172.21.15.105:3300/0 conn(0x7f45440b8400 msgr2=0x7f45440cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:31.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4548b9c700 1 --2- 172.21.15.60:0/3334812860 >> v2:172.21.15.105:3300/0 conn(0x7f45440b8400 0x7f45440cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4548b9c700 1 -- 172.21.15.60:0/3334812860 >> v1:172.21.15.105:6789/0 conn(0x7f45440d3750 legacy=0x7f45440cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:31.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4548b9c700 1 -- 172.21.15.60:0/3334812860 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440b5190 con 0x7f45440cfca0 2023-07-13T22:21:31.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4543fff700 1 --2- 172.21.15.60:0/3334812860 >> v2:172.21.15.105:3300/0 conn(0x7f45440b8400 0x7f45440cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:31.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4548b9c700 1 --2- 172.21.15.60:0/3334812860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 0x7f45440d0f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f45440cf560 tx=0x7f453c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a965a9277cf4d6de server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:31.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4542ffd700 1 -- 172.21.15.60:0/3334812860 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c00e070 con 0x7f45440cfca0 2023-07-13T22:21:31.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4542ffd700 1 -- 172.21.15.60:0/3334812860 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c003a20 con 0x7f45440cfca0 2023-07-13T22:21:31.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.653+0000 7f4542ffd700 1 -- 172.21.15.60:0/3334812860 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c003dd0 con 0x7f45440cfca0 2023-07-13T22:21:31.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.654+0000 7f454a5ff700 1 -- 172.21.15.60:0/3334812860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 msgr2=0x7f45440d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.654+0000 7f454a5ff700 1 --2- 172.21.15.60:0/3334812860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 0x7f45440d0f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f45440cf560 tx=0x7f453c00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:31.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.654+0000 7f454a5ff700 1 -- 172.21.15.60:0/3334812860 shutdown_connections 2023-07-13T22:21:31.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.654+0000 7f454a5ff700 1 --2- 172.21.15.60:0/3334812860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 0x7f45440d0f50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.654+0000 7f454a5ff700 1 --2- 172.21.15.60:0/3334812860 >> v2:172.21.15.105:3300/0 conn(0x7f45440b8400 0x7f45440cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.654+0000 7f454a5ff700 1 -- 172.21.15.60:0/3334812860 >> 172.21.15.60:0/3334812860 conn(0x7f454406b390 msgr2=0x7f454406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:31.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.654+0000 7f454a5ff700 1 -- 172.21.15.60:0/3334812860 shutdown_connections 2023-07-13T22:21:31.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.654+0000 7f454a5ff700 1 -- 172.21.15.60:0/3334812860 wait complete. 2023-07-13T22:21:31.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.656+0000 7f454a5ff700 1 Processor -- start 2023-07-13T22:21:31.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.656+0000 7f454a5ff700 1 -- start start 2023-07-13T22:21:31.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f454a5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45440b8400 0x7f454414bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f454a5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 0x7f45440ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f454a5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f45440d3750 0x7f45441f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f454a5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45440cc630 con 0x7f45440cfca0 2023-07-13T22:21:31.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f454a5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45441f2aa0 con 0x7f45440b8400 2023-07-13T22:21:31.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f454a5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45440cbfe0 con 0x7f45440d3750 2023-07-13T22:21:31.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f45437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 0x7f45440ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f45437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 0x7f45440ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47910/0 (socket says 172.21.15.60:47910) 2023-07-13T22:21:31.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f45437fe700 1 -- 172.21.15.60:0/1112437225 learned_addr learned my addr 172.21.15.60:0/1112437225 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f4543fff700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45440b8400 0x7f454414bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.657+0000 7f4548b9c700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f45440d3750 0x7f45441f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f45437fe700 1 -- 172.21.15.60:0/1112437225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f45440d3750 msgr2=0x7f45441f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f45437fe700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f45440d3750 0x7f45441f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f45437fe700 1 -- 172.21.15.60:0/1112437225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45440b8400 msgr2=0x7f454414bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f45437fe700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45440b8400 0x7f454414bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f45437fe700 1 -- 172.21.15.60:0/1112437225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c009b30 con 0x7f45440cfca0 2023-07-13T22:21:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f4548b9c700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f45440d3750 0x7f45441f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f45437fe700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 0x7f45440ccac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f453800c4a0 tx=0x7f453800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f4540ff9700 1 -- 172.21.15.60:0/1112437225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380054e0 con 0x7f45440cfca0 2023-07-13T22:21:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f4540ff9700 1 -- 172.21.15.60:0/1112437225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538003680 con 0x7f45440cfca0 2023-07-13T22:21:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45440cc2c0 con 0x7f45440cfca0 2023-07-13T22:21:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f4540ff9700 1 -- 172.21.15.60:0/1112437225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538003a10 con 0x7f45440cfca0 2023-07-13T22:21:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.658+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454414c480 con 0x7f45440cfca0 2023-07-13T22:21:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.659+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45440cc7a0 con 0x7f45440cfca0 2023-07-13T22:21:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.660+0000 7f4540ff9700 1 -- 172.21.15.60:0/1112437225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4538003b70 con 0x7f45440cfca0 2023-07-13T22:21:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.661+0000 7f4540ff9700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f453006ace0 0x7f453006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.661+0000 7f4540ff9700 1 -- 172.21.15.60:0/1112437225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4538088760 con 0x7f45440cfca0 2023-07-13T22:21:31.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.661+0000 7f4543fff700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f453006ace0 0x7f453006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:31.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.662+0000 7f4543fff700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f453006ace0 0x7f453006d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4534006fd0 tx=0x7f4534008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:31.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.665+0000 7f4540ff9700 1 -- 172.21.15.60:0/1112437225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4538053770 con 0x7f45440cfca0 2023-07-13T22:21:31.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.874+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45440d1400 con 0x7f45440cfca0 2023-07-13T22:21:31.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.876+0000 7f4540ff9700 1 -- 172.21.15.60:0/1112437225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f4538056d90 con 0x7f45440cfca0 2023-07-13T22:21:31.861 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:31.861 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:31.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f453006ace0 msgr2=0x7f453006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f453006ace0 0x7f453006d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4534006fd0 tx=0x7f4534008040 comp rx=0 tx=0).stop 2023-07-13T22:21:31.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 msgr2=0x7f45440ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:31.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 0x7f45440ccac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f453800c4a0 tx=0x7f453800c860 comp rx=0 tx=0).stop 2023-07-13T22:21:31.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 shutdown_connections 2023-07-13T22:21:31.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f453006ace0 0x7f453006d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f45440d3750 0x7f45441f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45440b8400 0x7f454414bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 --2- 172.21.15.60:0/1112437225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45440cfca0 0x7f45440ccac0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:31.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 >> 172.21.15.60:0/1112437225 conn(0x7f454406b390 msgr2=0x7f45440cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:31.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 shutdown_connections 2023-07-13T22:21:31.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:31.878+0000 7f454a5ff700 1 -- 172.21.15.60:0/1112437225 wait complete. 2023-07-13T22:21:31.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:31.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:32.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.103+0000 7f16869ff700 1 Processor -- start 2023-07-13T22:21:32.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.103+0000 7f16869ff700 1 -- start start 2023-07-13T22:21:32.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.105+0000 7f16869ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f16800cfd50 0x7f16800d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.105+0000 7f16869ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f16800b1ac0 0x7f16800d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.105+0000 7f16869ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16800d1870 con 0x7f16800b8200 2023-07-13T22:21:32.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.105+0000 7f16869ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16800cf370 con 0x7f16800b1ac0 2023-07-13T22:21:32.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.105+0000 7f16869ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16800d12b0 con 0x7f16800cfd50 2023-07-13T22:21:32.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.105+0000 7f167ffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f16800b8200 0x7f16800cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47512/0 (socket says 172.21.15.60:47512) 2023-07-13T22:21:32.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.105+0000 7f167ffff700 1 -- 172.21.15.60:0/3864719760 learned_addr learned my addr 172.21.15.60:0/3864719760 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:32.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.105+0000 7f1684f9c700 1 --2- 172.21.15.60:0/3864719760 >> v2:172.21.15.60:3300/0 conn(0x7f16800b1ac0 0x7f16800d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.105+0000 7f167f7fe700 1 --2- 172.21.15.60:0/3864719760 >> v2:172.21.15.105:3300/0 conn(0x7f16800cfd50 0x7f16800d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f167effd700 1 -- 172.21.15.60:0/3864719760 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304049436 0 0) 0x7f16800d1870 con 0x7f16800b8200 2023-07-13T22:21:32.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f167effd700 1 -- 172.21.15.60:0/3864719760 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16680035b0 con 0x7f16800b8200 2023-07-13T22:21:32.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f1684f9c700 1 -- 172.21.15.60:0/3864719760 >> v2:172.21.15.105:3300/0 conn(0x7f16800cfd50 msgr2=0x7f16800d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:32.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f1684f9c700 1 --2- 172.21.15.60:0/3864719760 >> v2:172.21.15.105:3300/0 conn(0x7f16800cfd50 0x7f16800d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f1684f9c700 1 -- 172.21.15.60:0/3864719760 >> v1:172.21.15.60:6789/0 conn(0x7f16800b8200 legacy=0x7f16800cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f1684f9c700 1 -- 172.21.15.60:0/3864719760 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16800d1590 con 0x7f16800b1ac0 2023-07-13T22:21:32.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f167f7fe700 1 --2- 172.21.15.60:0/3864719760 >> v2:172.21.15.105:3300/0 conn(0x7f16800cfd50 0x7f16800d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:32.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f1684f9c700 1 --2- 172.21.15.60:0/3864719760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b1ac0 0x7f16800d2900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16800d07b0 tx=0x7f167800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bf59402bfd3902 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:32.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f167effd700 1 -- 172.21.15.60:0/3864719760 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167800e070 con 0x7f16800b1ac0 2023-07-13T22:21:32.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f167effd700 1 -- 172.21.15.60:0/3864719760 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167800bb20 con 0x7f16800b1ac0 2023-07-13T22:21:32.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.106+0000 7f167effd700 1 -- 172.21.15.60:0/3864719760 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678012430 con 0x7f16800b1ac0 2023-07-13T22:21:32.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.107+0000 7f16869ff700 1 -- 172.21.15.60:0/3864719760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b1ac0 msgr2=0x7f16800d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.107+0000 7f16869ff700 1 --2- 172.21.15.60:0/3864719760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b1ac0 0x7f16800d2900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16800d07b0 tx=0x7f167800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:32.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.107+0000 7f16869ff700 1 -- 172.21.15.60:0/3864719760 shutdown_connections 2023-07-13T22:21:32.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.107+0000 7f16869ff700 1 --2- 172.21.15.60:0/3864719760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b1ac0 0x7f16800d2900 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.107+0000 7f16869ff700 1 --2- 172.21.15.60:0/3864719760 >> v2:172.21.15.105:3300/0 conn(0x7f16800cfd50 0x7f16800d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.107+0000 7f16869ff700 1 -- 172.21.15.60:0/3864719760 >> 172.21.15.60:0/3864719760 conn(0x7f168006b2f0 msgr2=0x7f168006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.107+0000 7f16869ff700 1 -- 172.21.15.60:0/3864719760 shutdown_connections 2023-07-13T22:21:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.107+0000 7f16869ff700 1 -- 172.21.15.60:0/3864719760 wait complete. 2023-07-13T22:21:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.109+0000 7f16869ff700 1 Processor -- start 2023-07-13T22:21:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.109+0000 7f16869ff700 1 -- start start 2023-07-13T22:21:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f16869ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f16800b1ac0 0x7f1680158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f16869ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b8200 0x7f16801f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f16869ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16800cfd50 0x7f1680157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f16869ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1680157da0 con 0x7f16800b8200 2023-07-13T22:21:32.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f16869ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1680158080 con 0x7f16800cfd50 2023-07-13T22:21:32.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f16869ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1680158360 con 0x7f16800b1ac0 2023-07-13T22:21:32.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f167f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b8200 0x7f16801f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f167f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b8200 0x7f16801f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47936/0 (socket says 172.21.15.60:47936) 2023-07-13T22:21:32.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f167f7fe700 1 -- 172.21.15.60:0/914592251 learned_addr learned my addr 172.21.15.60:0/914592251 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:32.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f1684f9c700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16800cfd50 0x7f1680157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.110+0000 7f167ffff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f16800b1ac0 0x7f1680158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167f7fe700 1 -- 172.21.15.60:0/914592251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f16800b1ac0 msgr2=0x7f1680158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167f7fe700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f16800b1ac0 0x7f1680158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167f7fe700 1 -- 172.21.15.60:0/914592251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16800cfd50 msgr2=0x7f1680157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167f7fe700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16800cfd50 0x7f1680157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167f7fe700 1 -- 172.21.15.60:0/914592251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678009a70 con 0x7f16800b8200 2023-07-13T22:21:32.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167ffff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f16800b1ac0 0x7f1680158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:32.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167f7fe700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b8200 0x7f16801f1830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f167400e810 tx=0x7f167400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:32.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167cff9700 1 -- 172.21.15.60:0/914592251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674013610 con 0x7f16800b8200 2023-07-13T22:21:32.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167cff9700 1 -- 172.21.15.60:0/914592251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1674013770 con 0x7f16800b8200 2023-07-13T22:21:32.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f167cff9700 1 -- 172.21.15.60:0/914592251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674013b00 con 0x7f16800b8200 2023-07-13T22:21:32.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1680158640 con 0x7f16800b8200 2023-07-13T22:21:32.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.111+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16801f1d70 con 0x7f16800b8200 2023-07-13T22:21:32.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.112+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168004e210 con 0x7f16800b8200 2023-07-13T22:21:32.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.113+0000 7f167cff9700 1 -- 172.21.15.60:0/914592251 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1674013c60 con 0x7f16800b8200 2023-07-13T22:21:32.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.114+0000 7f167cff9700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f166c06ace0 0x7f166c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.114+0000 7f167cff9700 1 -- 172.21.15.60:0/914592251 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1674089050 con 0x7f16800b8200 2023-07-13T22:21:32.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.115+0000 7f167ffff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f166c06ace0 0x7f166c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.115+0000 7f167ffff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f166c06ace0 0x7f166c06d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16700011e0 tx=0x7f1670009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:32.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.118+0000 7f167cff9700 1 -- 172.21.15.60:0/914592251 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1674057ba0 con 0x7f16800b8200 2023-07-13T22:21:32.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.324+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f16800b4f70 con 0x7f16800b8200 2023-07-13T22:21:32.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.324+0000 7f167cff9700 1 -- 172.21.15.60:0/914592251 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1674057730 con 0x7f16800b8200 2023-07-13T22:21:32.309 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:32.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f166c06ace0 msgr2=0x7f166c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f166c06ace0 0x7f166c06d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16700011e0 tx=0x7f1670009040 comp rx=0 tx=0).stop 2023-07-13T22:21:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b8200 msgr2=0x7f16801f1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b8200 0x7f16801f1830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f167400e810 tx=0x7f167400ebd0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 shutdown_connections 2023-07-13T22:21:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f166c06ace0 0x7f166c06d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16800cfd50 0x7f1680157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f16800b1ac0 0x7f1680158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 --2- 172.21.15.60:0/914592251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16800b8200 0x7f16801f1830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 >> 172.21.15.60:0/914592251 conn(0x7f168006b2f0 msgr2=0x7f16800d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:32.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 shutdown_connections 2023-07-13T22:21:32.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.327+0000 7f16869ff700 1 -- 172.21.15.60:0/914592251 wait complete. 2023-07-13T22:21:32.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:32.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.550+0000 7fbee0bff700 1 Processor -- start 2023-07-13T22:21:32.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.550+0000 7fbee0bff700 1 -- start start 2023-07-13T22:21:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbee0bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbedc0b8200 0x7fbedc0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbee0bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 0x7fbedc0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbee0bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc0d1870 con 0x7fbedc0cede0 2023-07-13T22:21:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbee0bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc0cf370 con 0x7fbedc0b8200 2023-07-13T22:21:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbee0bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0d12b0 con 0x7fbedc0b1ac0 2023-07-13T22:21:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbeda59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbedc0b8200 0x7fbedc0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbedad9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbedc0cede0 0x7fbedc15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47522/0 (socket says 172.21.15.60:47522) 2023-07-13T22:21:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbeda59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbedc0b8200 0x7fbedc0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47942/0 (socket says 172.21.15.60:47942) 2023-07-13T22:21:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbedad9d700 1 -- 172.21.15.60:0/939686659 learned_addr learned my addr 172.21.15.60:0/939686659 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbed3fff700 1 --2- 172.21.15.60:0/939686659 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 0x7fbedc0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbeda59c700 1 -- 172.21.15.60:0/939686659 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 msgr2=0x7fbedc0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbeda59c700 1 --2- 172.21.15.60:0/939686659 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 0x7fbedc0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbeda59c700 1 -- 172.21.15.60:0/939686659 >> v1:172.21.15.60:6789/0 conn(0x7fbedc0cede0 legacy=0x7fbedc15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbeda59c700 1 -- 172.21.15.60:0/939686659 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc0d1590 con 0x7fbedc0b8200 2023-07-13T22:21:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.552+0000 7fbed9d9b700 1 -- 172.21.15.60:0/939686659 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746088781 0 0) 0x7fbedc0d1870 con 0x7fbedc0cede0 2023-07-13T22:21:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.553+0000 7fbeda59c700 1 --2- 172.21.15.60:0/939686659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0b8200 0x7fbedc0cf640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009840 tx=0x7fbecc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305070ac20e94a93 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.553+0000 7fbed9d9b700 1 -- 172.21.15.60:0/939686659 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc011830 con 0x7fbedc0b8200 2023-07-13T22:21:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.553+0000 7fbed9d9b700 1 -- 172.21.15.60:0/939686659 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc011990 con 0x7fbedc0b8200 2023-07-13T22:21:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.553+0000 7fbed9d9b700 1 -- 172.21.15.60:0/939686659 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc011d20 con 0x7fbedc0b8200 2023-07-13T22:21:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.554+0000 7fbee0bff700 1 -- 172.21.15.60:0/939686659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0b8200 msgr2=0x7fbedc0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.554+0000 7fbee0bff700 1 --2- 172.21.15.60:0/939686659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0b8200 0x7fbedc0cf640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009840 tx=0x7fbecc009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.554+0000 7fbee0bff700 1 -- 172.21.15.60:0/939686659 shutdown_connections 2023-07-13T22:21:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.554+0000 7fbee0bff700 1 --2- 172.21.15.60:0/939686659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0b8200 0x7fbedc0cf640 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.554+0000 7fbee0bff700 1 --2- 172.21.15.60:0/939686659 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 0x7fbedc0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.554+0000 7fbee0bff700 1 -- 172.21.15.60:0/939686659 >> 172.21.15.60:0/939686659 conn(0x7fbedc06b2f0 msgr2=0x7fbedc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.554+0000 7fbee0bff700 1 -- 172.21.15.60:0/939686659 shutdown_connections 2023-07-13T22:21:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.554+0000 7fbee0bff700 1 -- 172.21.15.60:0/939686659 wait complete. 2023-07-13T22:21:32.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.556+0000 7fbee0bff700 1 Processor -- start 2023-07-13T22:21:32.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.556+0000 7fbee0bff700 1 -- start start 2023-07-13T22:21:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbee0bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 0x7fbedc1e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbee0bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbedc0b8200 0x7fbedc0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbee0bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0cede0 0x7fbedc15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbee0bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc1e3b20 con 0x7fbedc0cede0 2023-07-13T22:21:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbee0bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc1e3e00 con 0x7fbedc0b8200 2023-07-13T22:21:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbee0bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc1e40e0 con 0x7fbedc0b1ac0 2023-07-13T22:21:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbedad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0cede0 0x7fbedc15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbedad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0cede0 0x7fbedc15a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47946/0 (socket says 172.21.15.60:47946) 2023-07-13T22:21:32.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbedad9d700 1 -- 172.21.15.60:0/1329572507 learned_addr learned my addr 172.21.15.60:0/1329572507 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:32.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbeda59c700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 0x7fbedc1e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.557+0000 7fbed3fff700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbedc0b8200 0x7fbedc0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbedad9d700 1 -- 172.21.15.60:0/1329572507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 msgr2=0x7fbedc1e6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbedad9d700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 0x7fbedc1e6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbedad9d700 1 -- 172.21.15.60:0/1329572507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbedc0b8200 msgr2=0x7fbedc0c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbedad9d700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbedc0b8200 0x7fbedc0c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbedad9d700 1 -- 172.21.15.60:0/1329572507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0094a0 con 0x7fbedc0cede0 2023-07-13T22:21:32.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbedad9d700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0cede0 0x7fbedc15a1f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbed400cba0 tx=0x7fbed400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:32.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbed2ffd700 1 -- 172.21.15.60:0/1329572507 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed400aaa0 con 0x7fbedc0cede0 2023-07-13T22:21:32.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbed2ffd700 1 -- 172.21.15.60:0/1329572507 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed400ac20 con 0x7fbedc0cede0 2023-07-13T22:21:32.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbed2ffd700 1 -- 172.21.15.60:0/1329572507 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed40070d0 con 0x7fbedc0cede0 2023-07-13T22:21:32.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc1e43c0 con 0x7fbedc0cede0 2023-07-13T22:21:32.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.558+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc1e6950 con 0x7fbedc0cede0 2023-07-13T22:21:32.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.559+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc065ee0 con 0x7fbedc0cede0 2023-07-13T22:21:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.560+0000 7fbed2ffd700 1 -- 172.21.15.60:0/1329572507 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbed4007230 con 0x7fbedc0cede0 2023-07-13T22:21:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.561+0000 7fbed2ffd700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbebc06ace0 0x7fbebc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.561+0000 7fbed2ffd700 1 -- 172.21.15.60:0/1329572507 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbed4088f50 con 0x7fbedc0cede0 2023-07-13T22:21:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.561+0000 7fbeda59c700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbebc06ace0 0x7fbebc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.562+0000 7fbeda59c700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbebc06ace0 0x7fbebc06d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0090b0 tx=0x7fbecc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:32.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.565+0000 7fbed2ffd700 1 -- 172.21.15.60:0/1329572507 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbed4053ff0 con 0x7fbedc0cede0 2023-07-13T22:21:32.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.771+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbedc04e210 con 0x7fbedc0cede0 2023-07-13T22:21:32.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.772+0000 7fbed2ffd700 1 -- 172.21.15.60:0/1329572507 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbed4057610 con 0x7fbedc0cede0 2023-07-13T22:21:32.757 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:32.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbebc06ace0 msgr2=0x7fbebc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbebc06ace0 0x7fbebc06d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0090b0 tx=0x7fbecc008030 comp rx=0 tx=0).stop 2023-07-13T22:21:32.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0cede0 msgr2=0x7fbedc15a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0cede0 0x7fbedc15a1f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbed400cba0 tx=0x7fbed400ceb0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 shutdown_connections 2023-07-13T22:21:32.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbebc06ace0 0x7fbebc06d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbedc0cede0 0x7fbedc15a1f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbedc0b1ac0 0x7fbedc1e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 --2- 172.21.15.60:0/1329572507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbedc0b8200 0x7fbedc0c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 >> 172.21.15.60:0/1329572507 conn(0x7fbedc06b2f0 msgr2=0x7fbedc0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:32.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 shutdown_connections 2023-07-13T22:21:32.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.774+0000 7fbee0bff700 1 -- 172.21.15.60:0/1329572507 wait complete. 2023-07-13T22:21:32.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:32.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.998+0000 7fb8babff700 1 Processor -- start 2023-07-13T22:21:32.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.998+0000 7fb8babff700 1 -- start start 2023-07-13T22:21:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.999+0000 7fb8babff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8b40cfd50 0x7fb8b40d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:32.999+0000 7fb8babff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 0x7fb8b40d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8babff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b40d1870 con 0x7fb8b40b8200 2023-07-13T22:21:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8babff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b40cf370 con 0x7fb8b40cfd50 2023-07-13T22:21:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8babff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40d12b0 con 0x7fb8b40b1ac0 2023-07-13T22:21:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8b40cfd50 0x7fb8b40d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b899b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb8b40b8200 0x7fb8b40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47526/0 (socket says 172.21.15.60:47526) 2023-07-13T22:21:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b899b700 1 -- 172.21.15.60:0/1792021364 learned_addr learned my addr 172.21.15.60:0/1792021364 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b919c700 1 --2- 172.21.15.60:0/1792021364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 0x7fb8b40d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b37fe700 1 -- 172.21.15.60:0/1792021364 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175145498 0 0) 0x7fb8b40d1870 con 0x7fb8b40b8200 2023-07-13T22:21:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b37fe700 1 -- 172.21.15.60:0/1792021364 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8a40035b0 con 0x7fb8b40b8200 2023-07-13T22:21:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b3fff700 1 -- 172.21.15.60:0/1792021364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 msgr2=0x7fb8b40d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b3fff700 1 --2- 172.21.15.60:0/1792021364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 0x7fb8b40d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b3fff700 1 -- 172.21.15.60:0/1792021364 >> v1:172.21.15.60:6789/0 conn(0x7fb8b40b8200 legacy=0x7fb8b40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.000+0000 7fb8b3fff700 1 -- 172.21.15.60:0/1792021364 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b40d1590 con 0x7fb8b40cfd50 2023-07-13T22:21:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.001+0000 7fb8b3fff700 1 --2- 172.21.15.60:0/1792021364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40cfd50 0x7fb8b40d23c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009840 tx=0x7fb8a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e454b17b576643c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.001+0000 7fb8b37fe700 1 -- 172.21.15.60:0/1792021364 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0011830 con 0x7fb8b40cfd50 2023-07-13T22:21:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.001+0000 7fb8b37fe700 1 -- 172.21.15.60:0/1792021364 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0011990 con 0x7fb8b40cfd50 2023-07-13T22:21:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.001+0000 7fb8b37fe700 1 -- 172.21.15.60:0/1792021364 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0011d20 con 0x7fb8b40cfd50 2023-07-13T22:21:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.001+0000 7fb8babff700 1 -- 172.21.15.60:0/1792021364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40cfd50 msgr2=0x7fb8b40d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.001+0000 7fb8babff700 1 --2- 172.21.15.60:0/1792021364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40cfd50 0x7fb8b40d23c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009840 tx=0x7fb8a0009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:32.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.001+0000 7fb8babff700 1 -- 172.21.15.60:0/1792021364 shutdown_connections 2023-07-13T22:21:32.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.002+0000 7fb8babff700 1 --2- 172.21.15.60:0/1792021364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 0x7fb8b40d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.002+0000 7fb8babff700 1 --2- 172.21.15.60:0/1792021364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40cfd50 0x7fb8b40d23c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.002+0000 7fb8babff700 1 -- 172.21.15.60:0/1792021364 >> 172.21.15.60:0/1792021364 conn(0x7fb8b406b2f0 msgr2=0x7fb8b406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:32.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.002+0000 7fb8babff700 1 -- 172.21.15.60:0/1792021364 shutdown_connections 2023-07-13T22:21:32.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.002+0000 7fb8babff700 1 -- 172.21.15.60:0/1792021364 wait complete. 2023-07-13T22:21:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.004+0000 7fb8babff700 1 Processor -- start 2023-07-13T22:21:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.004+0000 7fb8babff700 1 -- start start 2023-07-13T22:21:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.005+0000 7fb8babff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 0x7fb8b4147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.005+0000 7fb8babff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40b8200 0x7fb8b40cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.005+0000 7fb8babff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b40cfd50 0x7fb8b4147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8babff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40cabd0 con 0x7fb8b40b8200 2023-07-13T22:21:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8babff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40c9d00 con 0x7fb8b40cfd50 2023-07-13T22:21:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8babff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40c9f50 con 0x7fb8b40b1ac0 2023-07-13T22:21:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40b8200 0x7fb8b40cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40b8200 0x7fb8b40cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47976/0 (socket says 172.21.15.60:47976) 2023-07-13T22:21:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b3fff700 1 -- 172.21.15.60:0/3847266534 learned_addr learned my addr 172.21.15.60:0/3847266534 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b899b700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 0x7fb8b4147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b919c700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b40cfd50 0x7fb8b4147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b3fff700 1 -- 172.21.15.60:0/3847266534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 msgr2=0x7fb8b4147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b3fff700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 0x7fb8b4147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b3fff700 1 -- 172.21.15.60:0/3847266534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b40cfd50 msgr2=0x7fb8b4147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b3fff700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b40cfd50 0x7fb8b4147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.006+0000 7fb8b3fff700 1 -- 172.21.15.60:0/3847266534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a00094a0 con 0x7fb8b40b8200 2023-07-13T22:21:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.007+0000 7fb8b3fff700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40b8200 0x7fb8b40cc3f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00090b0 tx=0x7fb8a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.007+0000 7fb8b17fa700 1 -- 172.21.15.60:0/3847266534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0004590 con 0x7fb8b40b8200 2023-07-13T22:21:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.007+0000 7fb8b17fa700 1 -- 172.21.15.60:0/3847266534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a001f070 con 0x7fb8b40b8200 2023-07-13T22:21:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.007+0000 7fb8b17fa700 1 -- 172.21.15.60:0/3847266534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00038a0 con 0x7fb8b40b8200 2023-07-13T22:21:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.007+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b40ca230 con 0x7fb8b40b8200 2023-07-13T22:21:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.007+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b40ca7f0 con 0x7fb8b40b8200 2023-07-13T22:21:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.008+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b40cad10 con 0x7fb8b40b8200 2023-07-13T22:21:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.009+0000 7fb8b17fa700 1 -- 172.21.15.60:0/3847266534 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8a0003a00 con 0x7fb8b40b8200 2023-07-13T22:21:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.010+0000 7fb8b17fa700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb89806ace0 0x7fb89806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.010+0000 7fb8b17fa700 1 -- 172.21.15.60:0/3847266534 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb8a0021070 con 0x7fb8b40b8200 2023-07-13T22:21:32.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.010+0000 7fb8b899b700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb89806ace0 0x7fb89806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:32.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.011+0000 7fb8b899b700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb89806ace0 0x7fb89806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8007fd0 tx=0x7fb8a8007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:32.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.014+0000 7fb8b17fa700 1 -- 172.21.15.60:0/3847266534 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb8b40cad10 con 0x7fb8b40b8200 2023-07-13T22:21:33.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.223+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8b40cad10 con 0x7fb8b40b8200 2023-07-13T22:21:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.223+0000 7fb8b17fa700 1 -- 172.21.15.60:0/3847266534 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8b40cad10 con 0x7fb8b40b8200 2023-07-13T22:21:33.208 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb89806ace0 msgr2=0x7fb89806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb89806ace0 0x7fb89806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8007fd0 tx=0x7fb8a8007dc0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40b8200 msgr2=0x7fb8b40cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40b8200 0x7fb8b40cc3f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00090b0 tx=0x7fb8a0004750 comp rx=0 tx=0).stop 2023-07-13T22:21:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 shutdown_connections 2023-07-13T22:21:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb89806ace0 0x7fb89806d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8b40cfd50 0x7fb8b4147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8b40b1ac0 0x7fb8b4147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 --2- 172.21.15.60:0/3847266534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b40b8200 0x7fb8b40cc3f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 >> 172.21.15.60:0/3847266534 conn(0x7fb8b406b2f0 msgr2=0x7fb8b40d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:33.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 shutdown_connections 2023-07-13T22:21:33.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.226+0000 7fb8babff700 1 -- 172.21.15.60:0/3847266534 wait complete. 2023-07-13T22:21:33.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:33.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.450+0000 7fb2cd7ff700 1 Processor -- start 2023-07-13T22:21:33.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.451+0000 7fb2cd7ff700 1 -- start start 2023-07-13T22:21:33.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.452+0000 7fb2cd7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b8200 0x7fb2c80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.452+0000 7fb2cd7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb2c80cede0 0x7fb2c80cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.452+0000 7fb2cd7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c80d1870 con 0x7fb2c80b1ac0 2023-07-13T22:21:33.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.452+0000 7fb2cd7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c80cf370 con 0x7fb2c80cede0 2023-07-13T22:21:33.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.452+0000 7fb2cd7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80d12b0 con 0x7fb2c80b8200 2023-07-13T22:21:33.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.452+0000 7fb2c77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb2c80cede0 0x7fb2c80cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.452+0000 7fb2c67fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb2c80b1ac0 0x7fb2c815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47540/0 (socket says 172.21.15.60:47540) 2023-07-13T22:21:33.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.452+0000 7fb2c67fc700 1 -- 172.21.15.60:0/1148263381 learned_addr learned my addr 172.21.15.60:0/1148263381 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:33.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.452+0000 7fb2c6ffd700 1 --2- 172.21.15.60:0/1148263381 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b8200 0x7fb2c80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c77fe700 1 -- 172.21.15.60:0/1148263381 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b8200 msgr2=0x7fb2c80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:33.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c77fe700 1 --2- 172.21.15.60:0/1148263381 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b8200 0x7fb2c80cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c77fe700 1 -- 172.21.15.60:0/1148263381 >> v1:172.21.15.60:6789/0 conn(0x7fb2c80b1ac0 legacy=0x7fb2c815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c77fe700 1 -- 172.21.15.60:0/1148263381 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80d1590 con 0x7fb2c80cede0 2023-07-13T22:21:33.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c5ffb700 1 -- 172.21.15.60:0/1148263381 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779828183 0 0) 0x7fb2c80d1870 con 0x7fb2c80b1ac0 2023-07-13T22:21:33.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c6ffd700 1 --2- 172.21.15.60:0/1148263381 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b8200 0x7fb2c80cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:33.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c77fe700 1 --2- 172.21.15.60:0/1148263381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80cede0 0x7fb2c80cfb80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009d90 tx=0x7fb2c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33e78314793de74 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c5ffb700 1 -- 172.21.15.60:0/1148263381 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c000e070 con 0x7fb2c80cede0 2023-07-13T22:21:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c5ffb700 1 -- 172.21.15.60:0/1148263381 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c000bbd0 con 0x7fb2c80cede0 2023-07-13T22:21:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.453+0000 7fb2c5ffb700 1 -- 172.21.15.60:0/1148263381 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0012430 con 0x7fb2c80cede0 2023-07-13T22:21:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.454+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/1148263381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80cede0 msgr2=0x7fb2c80cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.454+0000 7fb2cd7ff700 1 --2- 172.21.15.60:0/1148263381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80cede0 0x7fb2c80cfb80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009d90 tx=0x7fb2c000b330 comp rx=0 tx=0).stop 2023-07-13T22:21:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.454+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/1148263381 shutdown_connections 2023-07-13T22:21:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.454+0000 7fb2cd7ff700 1 --2- 172.21.15.60:0/1148263381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80cede0 0x7fb2c80cfb80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.454+0000 7fb2cd7ff700 1 --2- 172.21.15.60:0/1148263381 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b8200 0x7fb2c80cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.454+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/1148263381 >> 172.21.15.60:0/1148263381 conn(0x7fb2c806b2f0 msgr2=0x7fb2c806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:33.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.454+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/1148263381 shutdown_connections 2023-07-13T22:21:33.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.454+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/1148263381 wait complete. 2023-07-13T22:21:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.456+0000 7fb2cd7ff700 1 Processor -- start 2023-07-13T22:21:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.456+0000 7fb2cd7ff700 1 -- start start 2023-07-13T22:21:33.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2cd7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b1ac0 0x7fb2c81e6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2cd7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80b8200 0x7fb2c8159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2cd7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c80cede0 0x7fb2c8159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2cd7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81e4470 con 0x7fb2c80b8200 2023-07-13T22:21:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2cd7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81e6ab0 con 0x7fb2c80cede0 2023-07-13T22:21:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2cd7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81e3bb0 con 0x7fb2c80b1ac0 2023-07-13T22:21:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2c67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80b8200 0x7fb2c8159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2c67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80b8200 0x7fb2c8159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47984/0 (socket says 172.21.15.60:47984) 2023-07-13T22:21:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2c67fc700 1 -- 172.21.15.60:0/4271892044 learned_addr learned my addr 172.21.15.60:0/4271892044 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2c6ffd700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b1ac0 0x7fb2c81e6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.457+0000 7fb2c77fe700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c80cede0 0x7fb2c8159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2c67fc700 1 -- 172.21.15.60:0/4271892044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b1ac0 msgr2=0x7fb2c81e6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2c67fc700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b1ac0 0x7fb2c81e6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2c67fc700 1 -- 172.21.15.60:0/4271892044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c80cede0 msgr2=0x7fb2c8159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2c67fc700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c80cede0 0x7fb2c8159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2c67fc700 1 -- 172.21.15.60:0/4271892044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c0009a70 con 0x7fb2c80b8200 2023-07-13T22:21:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2c67fc700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80b8200 0x7fb2c8159a90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c750 tx=0x7fb2bc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2af7fe700 1 -- 172.21.15.60:0/4271892044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc004c80 con 0x7fb2c80b8200 2023-07-13T22:21:33.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2af7fe700 1 -- 172.21.15.60:0/4271892044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc004e00 con 0x7fb2c80b8200 2023-07-13T22:21:33.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2af7fe700 1 -- 172.21.15.60:0/4271892044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc007310 con 0x7fb2c80b8200 2023-07-13T22:21:33.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c81e4900 con 0x7fb2c80b8200 2023-07-13T22:21:33.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.458+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c81e4100 con 0x7fb2c80b8200 2023-07-13T22:21:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.459+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c81e4620 con 0x7fb2c80b8200 2023-07-13T22:21:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.460+0000 7fb2af7fe700 1 -- 172.21.15.60:0/4271892044 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb2bc007470 con 0x7fb2c80b8200 2023-07-13T22:21:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.461+0000 7fb2af7fe700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb2b406ace0 0x7fb2b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.461+0000 7fb2af7fe700 1 -- 172.21.15.60:0/4271892044 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2bc089150 con 0x7fb2c80b8200 2023-07-13T22:21:33.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.461+0000 7fb2c6ffd700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb2b406ace0 0x7fb2b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.462+0000 7fb2c6ffd700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb2b406ace0 0x7fb2b406d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006f10 tx=0x7fb2b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:33.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.466+0000 7fb2af7fe700 1 -- 172.21.15.60:0/4271892044 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb2c81e4620 con 0x7fb2c80b8200 2023-07-13T22:21:33.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.674+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2c81e4620 con 0x7fb2c80b8200 2023-07-13T22:21:33.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.674+0000 7fb2af7fe700 1 -- 172.21.15.60:0/4271892044 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2c81e4620 con 0x7fb2c80b8200 2023-07-13T22:21:33.659 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:33.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.676+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb2b406ace0 msgr2=0x7fb2b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.676+0000 7fb2cd7ff700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb2b406ace0 0x7fb2b406d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006f10 tx=0x7fb2b8008040 comp rx=0 tx=0).stop 2023-07-13T22:21:33.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.676+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80b8200 msgr2=0x7fb2c8159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.676+0000 7fb2cd7ff700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80b8200 0x7fb2c8159a90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c750 tx=0x7fb2bc00ca60 comp rx=0 tx=0).stop 2023-07-13T22:21:33.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.677+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 shutdown_connections 2023-07-13T22:21:33.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.677+0000 7fb2cd7ff700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb2b406ace0 0x7fb2b406d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.677+0000 7fb2cd7ff700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2c80cede0 0x7fb2c8159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.677+0000 7fb2cd7ff700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2c80b1ac0 0x7fb2c81e6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.677+0000 7fb2cd7ff700 1 --2- 172.21.15.60:0/4271892044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2c80b8200 0x7fb2c8159a90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.677+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 >> 172.21.15.60:0/4271892044 conn(0x7fb2c806b2f0 msgr2=0x7fb2c80b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:33.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.677+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 shutdown_connections 2023-07-13T22:21:33.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.677+0000 7fb2cd7ff700 1 -- 172.21.15.60:0/4271892044 wait complete. 2023-07-13T22:21:33.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:33.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.899+0000 7f793bbff700 1 Processor -- start 2023-07-13T22:21:33.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.899+0000 7f793bbff700 1 -- start start 2023-07-13T22:21:33.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.900+0000 7f793bbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340cfca0 0x7f7934167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.900+0000 7f793bbff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79340cd950 con 0x7f79340b8400 2023-07-13T22:21:33.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.900+0000 7f793bbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79340d0ac0 con 0x7f79340d3750 2023-07-13T22:21:33.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.900+0000 7f793bbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79340b87d0 con 0x7f79340cfca0 2023-07-13T22:21:33.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f793999b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f79340b8400 0x7f79340cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47550/0 (socket says 172.21.15.60:47550) 2023-07-13T22:21:33.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f793999b700 1 -- 172.21.15.60:0/4164299546 learned_addr learned my addr 172.21.15.60:0/4164299546 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:33.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f793a19c700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340cfca0 0x7f7934167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379329936 0 0) 0x7f7930001110 con 0x7f79340b8400 2023-07-13T22:21:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79200035b0 con 0x7f79340b8400 2023-07-13T22:21:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033727113 0 0) 0x7f79340d0ac0 con 0x7f79340d3750 2023-07-13T22:21:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7930001110 con 0x7f79340d3750 2023-07-13T22:21:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3112630538 0 0) 0x7f79200035b0 con 0x7f79340b8400 2023-07-13T22:21:33.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340cfca0 msgr2=0x7f7934167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:33.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.901+0000 7f7938999700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340cfca0 0x7f7934167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 >> v1:172.21.15.105:6789/0 conn(0x7f79340d3750 legacy=0x7f79340cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340b4eb0 con 0x7f79340b8400 2023-07-13T22:21:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f7930001800 con 0x7f79340b8400 2023-07-13T22:21:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f793a19c700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340cfca0 0x7f7934167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 >> v1:172.21.15.60:6789/0 conn(0x7f79340b8400 legacy=0x7f79340cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7920003bc0 0x7f7920003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340cfca0 0x7f7920004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7920004d90 0x7f7920005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79340b87d0 con 0x7f7920004d90 2023-07-13T22:21:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792000baf0 con 0x7f7920003bc0 2023-07-13T22:21:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792000bcb0 con 0x7f79340cfca0 2023-07-13T22:21:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f793a19c700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7920004d90 0x7f7920005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f793999b700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7920003bc0 0x7f7920003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f793919a700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340cfca0 0x7f7920004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f793a19c700 1 -- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340cfca0 msgr2=0x7f7920004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f793a19c700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340cfca0 0x7f7920004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f793a19c700 1 -- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7920003bc0 msgr2=0x7f7920003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f793a19c700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7920003bc0 0x7f7920003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.902+0000 7f793a19c700 1 -- 172.21.15.60:0/4164299546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c009a70 con 0x7f7920004d90 2023-07-13T22:21:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.903+0000 7f793a19c700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7920004d90 0x7f7920005200 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f792c00e320 tx=0x7f7920005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3f6e0e29ab1199 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.903+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c00edc0 con 0x7f7920004d90 2023-07-13T22:21:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.903+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c00f040 con 0x7f7920004d90 2023-07-13T22:21:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.903+0000 7f7938999700 1 -- 172.21.15.60:0/4164299546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0146a0 con 0x7f7920004d90 2023-07-13T22:21:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.903+0000 7f793bbff700 1 -- 172.21.15.60:0/4164299546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7920004d90 msgr2=0x7f7920005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.903+0000 7f793bbff700 1 --2- 172.21.15.60:0/4164299546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7920004d90 0x7f7920005200 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f792c00e320 tx=0x7f7920005880 comp rx=0 tx=0).stop 2023-07-13T22:21:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.903+0000 7f793a19c700 1 -- 172.21.15.60:0/4164299546 reap_dead start 2023-07-13T22:21:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.904+0000 7f793bbff700 1 -- 172.21.15.60:0/4164299546 shutdown_connections 2023-07-13T22:21:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.904+0000 7f793bbff700 1 -- 172.21.15.60:0/4164299546 >> 172.21.15.60:0/4164299546 conn(0x7f793406b390 msgr2=0x7f793406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.904+0000 7f793bbff700 1 -- 172.21.15.60:0/4164299546 shutdown_connections 2023-07-13T22:21:33.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.904+0000 7f793bbff700 1 -- 172.21.15.60:0/4164299546 wait complete. 2023-07-13T22:21:33.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.905+0000 7f793bbff700 1 Processor -- start 2023-07-13T22:21:33.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.905+0000 7f793bbff700 1 -- start start 2023-07-13T22:21:33.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.906+0000 7f793bbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7920004d90 0x7f7934150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.906+0000 7f793bbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79340c8cc0 0x7f79340c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.906+0000 7f793bbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340b6560 0x7f79340b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793bbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79340b6f20 con 0x7f79340c8cc0 2023-07-13T22:21:33.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793bbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79340c9a80 con 0x7f7920004d90 2023-07-13T22:21:33.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793bbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79340c97a0 con 0x7f79340b6560 2023-07-13T22:21:33.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79340c8cc0 0x7f79340c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79340c8cc0 0x7f79340c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47990/0 (socket says 172.21.15.60:47990) 2023-07-13T22:21:33.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793919a700 1 -- 172.21.15.60:0/2372318164 learned_addr learned my addr 172.21.15.60:0/2372318164 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:33.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793999b700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7920004d90 0x7f7934150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793a19c700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340b6560 0x7f79340b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793919a700 1 -- 172.21.15.60:0/2372318164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340b6560 msgr2=0x7f79340b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793919a700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340b6560 0x7f79340b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793919a700 1 -- 172.21.15.60:0/2372318164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7920004d90 msgr2=0x7f7934150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:33.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793919a700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7920004d90 0x7f7934150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:33.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793919a700 1 -- 172.21.15.60:0/2372318164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793415cfe0 con 0x7f79340c8cc0 2023-07-13T22:21:33.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793a19c700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340b6560 0x7f79340b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:33.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.907+0000 7f793919a700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79340c8cc0 0x7f79340c9110 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f792400f810 tx=0x7f792400fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:33.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.908+0000 7f792a7fc700 1 -- 172.21.15.60:0/2372318164 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240052e0 con 0x7f79340c8cc0 2023-07-13T22:21:33.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.908+0000 7f792a7fc700 1 -- 172.21.15.60:0/2372318164 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7924005440 con 0x7f79340c8cc0 2023-07-13T22:21:33.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.908+0000 7f792a7fc700 1 -- 172.21.15.60:0/2372318164 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240038c0 con 0x7f79340c8cc0 2023-07-13T22:21:33.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.908+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793415d2c0 con 0x7f79340c8cc0 2023-07-13T22:21:33.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.908+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79341f23d0 con 0x7f79340c8cc0 2023-07-13T22:21:33.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.909+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793415d7e0 con 0x7f79340c8cc0 2023-07-13T22:21:33.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.910+0000 7f792a7fc700 1 -- 172.21.15.60:0/2372318164 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7924003a20 con 0x7f79340c8cc0 2023-07-13T22:21:33.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.911+0000 7f792a7fc700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f791806ad10 0x7f791806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:33.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.911+0000 7f792a7fc700 1 -- 172.21.15.60:0/2372318164 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79240892c0 con 0x7f79340c8cc0 2023-07-13T22:21:33.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.911+0000 7f793999b700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f791806ad10 0x7f791806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:33.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.912+0000 7f793999b700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f791806ad10 0x7f791806d1c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f79300010e0 tx=0x7f7930007a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:33.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:33.916+0000 7f792a7fc700 1 -- 172.21.15.60:0/2372318164 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f793415d7e0 con 0x7f79340c8cc0 2023-07-13T22:21:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.124+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79341609f0 con 0x7f79340c8cc0 2023-07-13T22:21:34.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.125+0000 7f792a7fc700 1 -- 172.21.15.60:0/2372318164 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f7924054360 con 0x7f79340c8cc0 2023-07-13T22:21:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:34.111 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:34.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f791806ad10 msgr2=0x7f791806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f791806ad10 0x7f791806d1c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f79300010e0 tx=0x7f7930007a80 comp rx=0 tx=0).stop 2023-07-13T22:21:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79340c8cc0 msgr2=0x7f79340c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79340c8cc0 0x7f79340c9110 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f792400f810 tx=0x7f792400fb20 comp rx=0 tx=0).stop 2023-07-13T22:21:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 shutdown_connections 2023-07-13T22:21:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f791806ad10 0x7f791806d1c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79340b6560 0x7f79340b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7920004d90 0x7f7934150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 --2- 172.21.15.60:0/2372318164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79340c8cc0 0x7f79340c9110 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 >> 172.21.15.60:0/2372318164 conn(0x7f793406b390 msgr2=0x7f79340cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 shutdown_connections 2023-07-13T22:21:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.128+0000 7f793bbff700 1 -- 172.21.15.60:0/2372318164 wait complete. 2023-07-13T22:21:34.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:34.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:34.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.351+0000 7f56921ff700 1 Processor -- start 2023-07-13T22:21:34.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.351+0000 7f56921ff700 1 -- start start 2023-07-13T22:21:34.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f56921ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f568c0ac490 0x7f568c0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f56921ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f568c0ad9a0 0x7f568c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f56921ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c0cf6d0 con 0x7f568c15c790 2023-07-13T22:21:34.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f56921ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c0ae2e0 con 0x7f568c0ad9a0 2023-07-13T22:21:34.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f56921ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c0ac900 con 0x7f568c0ac490 2023-07-13T22:21:34.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f568bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f568c0ad9a0 0x7f568c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f568bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f568c0ad9a0 0x7f568c0adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47994/0 (socket says 172.21.15.60:47994) 2023-07-13T22:21:34.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f568bfff700 1 -- 172.21.15.60:0/4281727695 learned_addr learned my addr 172.21.15.60:0/4281727695 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:34.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f568affd700 1 --2- 172.21.15.60:0/4281727695 >> v2:172.21.15.105:3300/0 conn(0x7f568c0ac490 0x7f568c0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f568bfff700 1 -- 172.21.15.60:0/4281727695 >> v2:172.21.15.105:3300/0 conn(0x7f568c0ac490 msgr2=0x7f568c0ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:34.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f568bfff700 1 --2- 172.21.15.60:0/4281727695 >> v2:172.21.15.105:3300/0 conn(0x7f568c0ac490 0x7f568c0ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f568bfff700 1 -- 172.21.15.60:0/4281727695 >> v1:172.21.15.105:6789/0 conn(0x7f568c15c790 legacy=0x7f568c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.353+0000 7f568bfff700 1 -- 172.21.15.60:0/4281727695 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0acbe0 con 0x7f568c0ad9a0 2023-07-13T22:21:34.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f568bfff700 1 --2- 172.21.15.60:0/4281727695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ad9a0 0x7f568c0adda0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f568c0c3310 tx=0x7f568400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4142be6a04f0e28 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:34.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f568a7fc700 1 -- 172.21.15.60:0/4281727695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400e070 con 0x7f568c0ad9a0 2023-07-13T22:21:34.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f568a7fc700 1 -- 172.21.15.60:0/4281727695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568400b910 con 0x7f568c0ad9a0 2023-07-13T22:21:34.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f568a7fc700 1 -- 172.21.15.60:0/4281727695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400bcc0 con 0x7f568c0ad9a0 2023-07-13T22:21:34.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f56921ff700 1 -- 172.21.15.60:0/4281727695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ad9a0 msgr2=0x7f568c0adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f56921ff700 1 --2- 172.21.15.60:0/4281727695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ad9a0 0x7f568c0adda0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f568c0c3310 tx=0x7f568400b330 comp rx=0 tx=0).stop 2023-07-13T22:21:34.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f56921ff700 1 -- 172.21.15.60:0/4281727695 shutdown_connections 2023-07-13T22:21:34.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f56921ff700 1 --2- 172.21.15.60:0/4281727695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ad9a0 0x7f568c0adda0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f56921ff700 1 --2- 172.21.15.60:0/4281727695 >> v2:172.21.15.105:3300/0 conn(0x7f568c0ac490 0x7f568c0ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.354+0000 7f56921ff700 1 -- 172.21.15.60:0/4281727695 >> 172.21.15.60:0/4281727695 conn(0x7f568c06b900 msgr2=0x7f568c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:34.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.355+0000 7f56921ff700 1 -- 172.21.15.60:0/4281727695 shutdown_connections 2023-07-13T22:21:34.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.355+0000 7f56921ff700 1 -- 172.21.15.60:0/4281727695 wait complete. 2023-07-13T22:21:34.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.356+0000 7f56921ff700 1 Processor -- start 2023-07-13T22:21:34.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.357+0000 7f56921ff700 1 -- start start 2023-07-13T22:21:34.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.357+0000 7f56921ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ac490 0x7f568c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f56921ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c0ad9a0 0x7f568c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f56921ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f568c15c790 0x7f568c0c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f56921ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c15ba40 con 0x7f568c0ac490 2023-07-13T22:21:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f56921ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0b51f0 con 0x7f568c0ad9a0 2023-07-13T22:21:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f56921ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0b54d0 con 0x7f568c15c790 2023-07-13T22:21:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ac490 0x7f568c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ac490 0x7f568c15b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47998/0 (socket says 172.21.15.60:47998) 2023-07-13T22:21:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568b7fe700 1 -- 172.21.15.60:0/424755361 learned_addr learned my addr 172.21.15.60:0/424755361 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568affd700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c0ad9a0 0x7f568c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568bfff700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f568c15c790 0x7f568c0c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568b7fe700 1 -- 172.21.15.60:0/424755361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f568c15c790 msgr2=0x7f568c0c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568b7fe700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f568c15c790 0x7f568c0c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568b7fe700 1 -- 172.21.15.60:0/424755361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c0ad9a0 msgr2=0x7f568c0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568b7fe700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c0ad9a0 0x7f568c0b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568b7fe700 1 -- 172.21.15.60:0/424755361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684009b30 con 0x7f568c0ac490 2023-07-13T22:21:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.358+0000 7f568affd700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c0ad9a0 0x7f568c0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.359+0000 7f568b7fe700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ac490 0x7f568c15b130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c4d0 tx=0x7f567c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.359+0000 7f5673fff700 1 -- 172.21.15.60:0/424755361 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0052d0 con 0x7f568c0ac490 2023-07-13T22:21:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.359+0000 7f5673fff700 1 -- 172.21.15.60:0/424755361 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c005430 con 0x7f568c0ac490 2023-07-13T22:21:34.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.359+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c0c68e0 con 0x7f568c0ac490 2023-07-13T22:21:34.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.359+0000 7f5673fff700 1 -- 172.21.15.60:0/424755361 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0038f0 con 0x7f568c0ac490 2023-07-13T22:21:34.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.359+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c0c6ea0 con 0x7f568c0ac490 2023-07-13T22:21:34.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.360+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c0c70c0 con 0x7f568c0ac490 2023-07-13T22:21:34.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.361+0000 7f5673fff700 1 -- 172.21.15.60:0/424755361 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f567c003a50 con 0x7f568c0ac490 2023-07-13T22:21:34.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.362+0000 7f5673fff700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f567806ace0 0x7f567806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.362+0000 7f5673fff700 1 -- 172.21.15.60:0/424755361 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f567c088940 con 0x7f568c0ac490 2023-07-13T22:21:34.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.362+0000 7f568affd700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f567806ace0 0x7f567806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.363+0000 7f568affd700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f567806ace0 0x7f567806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f568c0c82b0 tx=0x7f5680008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:34.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.366+0000 7f5673fff700 1 -- 172.21.15.60:0/424755361 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f567c0539e0 con 0x7f568c0ac490 2023-07-13T22:21:34.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.574+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f568c0cdad0 con 0x7f568c0ac490 2023-07-13T22:21:34.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.574+0000 7f5673fff700 1 -- 172.21.15.60:0/424755361 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f567c017020 con 0x7f568c0ac490 2023-07-13T22:21:34.559 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.576+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f567806ace0 msgr2=0x7f567806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.576+0000 7f56921ff700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f567806ace0 0x7f567806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f568c0c82b0 tx=0x7f5680008040 comp rx=0 tx=0).stop 2023-07-13T22:21:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.576+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ac490 msgr2=0x7f568c15b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.576+0000 7f56921ff700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ac490 0x7f568c15b130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c4d0 tx=0x7f567c00c890 comp rx=0 tx=0).stop 2023-07-13T22:21:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.576+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 shutdown_connections 2023-07-13T22:21:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.577+0000 7f56921ff700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f567806ace0 0x7f567806d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.577+0000 7f56921ff700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f568c0ac490 0x7f568c15b130 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.577+0000 7f56921ff700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f568c15c790 0x7f568c0c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.577+0000 7f56921ff700 1 --2- 172.21.15.60:0/424755361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f568c0ad9a0 0x7f568c0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.577+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 >> 172.21.15.60:0/424755361 conn(0x7f568c06b900 msgr2=0x7f568c0ca020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:34.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.577+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 shutdown_connections 2023-07-13T22:21:34.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.577+0000 7f56921ff700 1 -- 172.21.15.60:0/424755361 wait complete. 2023-07-13T22:21:34.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.800+0000 7f9cbb5ff700 1 Processor -- start 2023-07-13T22:21:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.800+0000 7f9cbb5ff700 1 -- start start 2023-07-13T22:21:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.801+0000 7f9cbb5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 0x7f9cb40d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.801+0000 7f9cbb5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9cb40b1ac0 0x7f9cb40d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.801+0000 7f9cbb5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb40d1870 con 0x7f9cb40b8200 2023-07-13T22:21:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cbb5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb40cf370 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cbb5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40d12b0 con 0x7f9cb40cfd50 2023-07-13T22:21:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cb939b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9cb40b8200 0x7f9cb40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47556/0 (socket says 172.21.15.60:47556) 2023-07-13T22:21:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cb939b700 1 -- 172.21.15.60:0/1067168619 learned_addr learned my addr 172.21.15.60:0/1067168619 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:34.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cb9b9c700 1 --2- 172.21.15.60:0/1067168619 >> v2:172.21.15.105:3300/0 conn(0x7f9cb40b1ac0 0x7f9cb40d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cb8b9a700 1 --2- 172.21.15.60:0/1067168619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 0x7f9cb40d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cabfff700 1 -- 172.21.15.60:0/1067168619 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194520776 0 0) 0x7f9cb40d1870 con 0x7f9cb40b8200 2023-07-13T22:21:34.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cabfff700 1 -- 172.21.15.60:0/1067168619 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ca00035b0 con 0x7f9cb40b8200 2023-07-13T22:21:34.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cb9b9c700 1 -- 172.21.15.60:0/1067168619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 msgr2=0x7f9cb40d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:34.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cb9b9c700 1 --2- 172.21.15.60:0/1067168619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 0x7f9cb40d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.802+0000 7f9cb9b9c700 1 -- 172.21.15.60:0/1067168619 >> v1:172.21.15.60:6789/0 conn(0x7f9cb40b8200 legacy=0x7f9cb40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.803+0000 7f9cabfff700 1 -- 172.21.15.60:0/1067168619 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1457569100 0 0) 0x7f9ca00035b0 con 0x7f9cb40b8200 2023-07-13T22:21:34.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.803+0000 7f9cb9b9c700 1 -- 172.21.15.60:0/1067168619 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb40d1590 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.803+0000 7f9cb8b9a700 1 --2- 172.21.15.60:0/1067168619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 0x7f9cb40d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:34.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.803+0000 7f9cb9b9c700 1 --2- 172.21.15.60:0/1067168619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cb40b1ac0 0x7f9cb40d2900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009dc0 tx=0x7f9cac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11d4006aa37ef239 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:34.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.803+0000 7f9cabfff700 1 -- 172.21.15.60:0/1067168619 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00e070 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.803+0000 7f9cabfff700 1 -- 172.21.15.60:0/1067168619 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac003a20 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.803+0000 7f9cabfff700 1 -- 172.21.15.60:0/1067168619 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0001670 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.804+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1067168619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cb40b1ac0 msgr2=0x7f9cb40d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.804+0000 7f9cbb5ff700 1 --2- 172.21.15.60:0/1067168619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cb40b1ac0 0x7f9cb40d2900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009dc0 tx=0x7f9cac00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:34.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.804+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1067168619 shutdown_connections 2023-07-13T22:21:34.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.804+0000 7f9cbb5ff700 1 --2- 172.21.15.60:0/1067168619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cb40b1ac0 0x7f9cb40d2900 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.804+0000 7f9cbb5ff700 1 --2- 172.21.15.60:0/1067168619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 0x7f9cb40d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.804+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1067168619 >> 172.21.15.60:0/1067168619 conn(0x7f9cb406b2f0 msgr2=0x7f9cb406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:34.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.804+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1067168619 shutdown_connections 2023-07-13T22:21:34.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.804+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1067168619 wait complete. 2023-07-13T22:21:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.806+0000 7f9cbb5ff700 1 Processor -- start 2023-07-13T22:21:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.806+0000 7f9cbb5ff700 1 -- start start 2023-07-13T22:21:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cbb5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb40b1ac0 0x7f9cb415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cbb5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cb40b8200 0x7f9cb40cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cbb5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 0x7f9cb41f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cbb5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40ca070 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cbb5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40ca350 con 0x7f9cb40b8200 2023-07-13T22:21:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cbb5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40ca630 con 0x7f9cb40cfd50 2023-07-13T22:21:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cb939b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb40b1ac0 0x7f9cb415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cb939b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb40b1ac0 0x7f9cb415ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48008/0 (socket says 172.21.15.60:48008) 2023-07-13T22:21:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cb939b700 1 -- 172.21.15.60:0/1374111304 learned_addr learned my addr 172.21.15.60:0/1374111304 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cb939b700 1 -- 172.21.15.60:0/1374111304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 msgr2=0x7f9cb41f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cb939b700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 0x7f9cb41f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cb939b700 1 -- 172.21.15.60:0/1374111304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cb40b8200 msgr2=0x7f9cb40cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cb939b700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cb40b8200 0x7f9cb40cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cb939b700 1 -- 172.21.15.60:0/1374111304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40094a0 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.807+0000 7f9cb939b700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb40b1ac0 0x7f9cb415ad00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000c780 tx=0x7f9cb000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.808+0000 7f9ca9ffb700 1 -- 172.21.15.60:0/1374111304 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0007560 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.808+0000 7f9ca9ffb700 1 -- 172.21.15.60:0/1374111304 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb00076e0 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.808+0000 7f9ca9ffb700 1 -- 172.21.15.60:0/1374111304 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0004ab0 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.808+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac009a70 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.808+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb40cabf0 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.809+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb40ce6e0 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.810+0000 7f9ca9ffb700 1 -- 172.21.15.60:0/1374111304 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9cb0004c10 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.811+0000 7f9ca9ffb700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c9806ace0 0x7f9c9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.811+0000 7f9ca9ffb700 1 -- 172.21.15.60:0/1374111304 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9cb00886f0 con 0x7f9cb40b1ac0 2023-07-13T22:21:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.811+0000 7f9cb8b9a700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c9806ace0 0x7f9c9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.812+0000 7f9cb8b9a700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c9806ace0 0x7f9c9806d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006f10 tx=0x7f9ca4006e60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:34.815+0000 7f9ca9ffb700 1 -- 172.21.15.60:0/1374111304 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9cb0053790 con 0x7f9cb40b1ac0 2023-07-13T22:21:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.019+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9cb40ca490 con 0x7f9cb40b1ac0 2023-07-13T22:21:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.019+0000 7f9ca9ffb700 1 -- 172.21.15.60:0/1374111304 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9cb0018020 con 0x7f9cb40b1ac0 2023-07-13T22:21:35.004 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.021+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c9806ace0 msgr2=0x7f9c9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.021+0000 7f9cbb5ff700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c9806ace0 0x7f9c9806d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006f10 tx=0x7f9ca4006e60 comp rx=0 tx=0).stop 2023-07-13T22:21:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb40b1ac0 msgr2=0x7f9cb415ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb40b1ac0 0x7f9cb415ad00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000c780 tx=0x7f9cb000cb40 comp rx=0 tx=0).stop 2023-07-13T22:21:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 shutdown_connections 2023-07-13T22:21:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c9806ace0 0x7f9c9806d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb40b1ac0 0x7f9cb415ad00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9cb40cfd50 0x7f9cb41f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 --2- 172.21.15.60:0/1374111304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cb40b8200 0x7f9cb40cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 >> 172.21.15.60:0/1374111304 conn(0x7f9cb406b2f0 msgr2=0x7f9cb40d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 shutdown_connections 2023-07-13T22:21:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.022+0000 7f9cbb5ff700 1 -- 172.21.15.60:0/1374111304 wait complete. 2023-07-13T22:21:35.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.246+0000 7f708bdff700 1 Processor -- start 2023-07-13T22:21:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.246+0000 7f708bdff700 1 -- start start 2023-07-13T22:21:35.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.247+0000 7f708bdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70840b1ac0 0x7f70840d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.247+0000 7f708bdff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70840d1870 con 0x7f70840cfd50 2023-07-13T22:21:35.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.247+0000 7f708bdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70840cf370 con 0x7f70840b8200 2023-07-13T22:21:35.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.247+0000 7f708bdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70840d12b0 con 0x7f70840b1ac0 2023-07-13T22:21:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.248+0000 7f708939a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f70840cfd50 0x7f70840d23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39176/0 (socket says 172.21.15.60:39176) 2023-07-13T22:21:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.248+0000 7f708939a700 1 -- 172.21.15.60:0/1724958952 learned_addr learned my addr 172.21.15.60:0/1724958952 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.248+0000 7f708a39c700 1 --2- 172.21.15.60:0/1724958952 >> v2:172.21.15.60:3300/0 conn(0x7f70840b1ac0 0x7f70840d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.248+0000 7f7088b99700 1 -- 172.21.15.60:0/1724958952 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976941569 0 0) 0x7f70840d1870 con 0x7f70840cfd50 2023-07-13T22:21:35.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.248+0000 7f7088b99700 1 -- 172.21.15.60:0/1724958952 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70700035b0 con 0x7f70840cfd50 2023-07-13T22:21:35.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.248+0000 7f708a39c700 1 -- 172.21.15.60:0/1724958952 >> v1:172.21.15.105:6789/0 conn(0x7f70840b8200 legacy=0x7f70840cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.248+0000 7f708a39c700 1 -- 172.21.15.60:0/1724958952 >> v1:172.21.15.60:6789/0 conn(0x7f70840cfd50 legacy=0x7f70840d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.248+0000 7f708a39c700 1 -- 172.21.15.60:0/1724958952 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70840d1590 con 0x7f70840b1ac0 2023-07-13T22:21:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.248+0000 7f708a39c700 1 --2- 172.21.15.60:0/1724958952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b1ac0 0x7f70840d2ad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f707c009d90 tx=0x7f707c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a89b2876229b103 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f7088b99700 1 -- 172.21.15.60:0/1724958952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c00e070 con 0x7f70840b1ac0 2023-07-13T22:21:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f7088b99700 1 -- 172.21.15.60:0/1724958952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c00bb20 con 0x7f70840b1ac0 2023-07-13T22:21:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f7088b99700 1 -- 172.21.15.60:0/1724958952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c012430 con 0x7f70840b1ac0 2023-07-13T22:21:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f708bdff700 1 -- 172.21.15.60:0/1724958952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b1ac0 msgr2=0x7f70840d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f708bdff700 1 --2- 172.21.15.60:0/1724958952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b1ac0 0x7f70840d2ad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f707c009d90 tx=0x7f707c00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f708bdff700 1 -- 172.21.15.60:0/1724958952 shutdown_connections 2023-07-13T22:21:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f708bdff700 1 --2- 172.21.15.60:0/1724958952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b1ac0 0x7f70840d2ad0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f708bdff700 1 -- 172.21.15.60:0/1724958952 >> 172.21.15.60:0/1724958952 conn(0x7f708406b2f0 msgr2=0x7f708406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f708bdff700 1 -- 172.21.15.60:0/1724958952 shutdown_connections 2023-07-13T22:21:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.249+0000 7f708bdff700 1 -- 172.21.15.60:0/1724958952 wait complete. 2023-07-13T22:21:35.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.252+0000 7f708bdff700 1 Processor -- start 2023-07-13T22:21:35.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.252+0000 7f708bdff700 1 -- start start 2023-07-13T22:21:35.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.253+0000 7f708bdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70840b1ac0 0x7f70841e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.253+0000 7f708bdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b8200 0x7f70840b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.253+0000 7f708bdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70840cfd50 0x7f70841e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.253+0000 7f708bdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084156940 con 0x7f70840b8200 2023-07-13T22:21:35.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.253+0000 7f708bdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70841e6190 con 0x7f70840cfd50 2023-07-13T22:21:35.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.253+0000 7f708bdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840b6ed0 con 0x7f70840b1ac0 2023-07-13T22:21:35.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.253+0000 7f708939a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b8200 0x7f70840b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708939a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b8200 0x7f70840b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59902/0 (socket says 172.21.15.60:59902) 2023-07-13T22:21:35.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708939a700 1 -- 172.21.15.60:0/2502967193 learned_addr learned my addr 172.21.15.60:0/2502967193 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:35.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708a39c700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70840cfd50 0x7f70841e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708939a700 1 -- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70840b1ac0 msgr2=0x7f70841e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f7089b9b700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70840b1ac0 0x7f70841e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708939a700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70840b1ac0 0x7f70841e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708939a700 1 -- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70840cfd50 msgr2=0x7f70841e6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708939a700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70840cfd50 0x7f70841e6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708939a700 1 -- 172.21.15.60:0/2502967193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c009a70 con 0x7f70840b8200 2023-07-13T22:21:35.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f7089b9b700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70840b1ac0 0x7f70841e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:35.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708a39c700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70840cfd50 0x7f70841e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:35.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f708939a700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b8200 0x7f70840b6990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7074005910 tx=0x7f707400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:35.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f707a7fc700 1 -- 172.21.15.60:0/2502967193 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074005d70 con 0x7f70840b8200 2023-07-13T22:21:35.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.254+0000 7f707a7fc700 1 -- 172.21.15.60:0/2502967193 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70740070d0 con 0x7f70840b8200 2023-07-13T22:21:35.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.255+0000 7f707a7fc700 1 -- 172.21.15.60:0/2502967193 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074007440 con 0x7f70840b8200 2023-07-13T22:21:35.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.255+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70841e6eb0 con 0x7f70840b8200 2023-07-13T22:21:35.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.255+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70840b7440 con 0x7f70840b8200 2023-07-13T22:21:35.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.256+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708404e210 con 0x7f70840b8200 2023-07-13T22:21:35.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.257+0000 7f707a7fc700 1 -- 172.21.15.60:0/2502967193 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70740075a0 con 0x7f70840b8200 2023-07-13T22:21:35.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.258+0000 7f707a7fc700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f706806ace0 0x7f706806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.258+0000 7f707a7fc700 1 -- 172.21.15.60:0/2502967193 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7074088860 con 0x7f70840b8200 2023-07-13T22:21:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.258+0000 7f7089b9b700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f706806ace0 0x7f706806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.259+0000 7f7089b9b700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f706806ace0 0x7f706806d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7080000f60 tx=0x7f7080009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:35.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.262+0000 7f707a7fc700 1 -- 172.21.15.60:0/2502967193 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f70740539b0 con 0x7f70840b8200 2023-07-13T22:21:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.468+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70840b4f70 con 0x7f70840b8200 2023-07-13T22:21:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.469+0000 7f707a7fc700 1 -- 172.21.15.60:0/2502967193 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7074056fd0 con 0x7f70840b8200 2023-07-13T22:21:35.454 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f706806ace0 msgr2=0x7f706806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f706806ace0 0x7f706806d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7080000f60 tx=0x7f7080009040 comp rx=0 tx=0).stop 2023-07-13T22:21:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b8200 msgr2=0x7f70840b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b8200 0x7f70840b6990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7074005910 tx=0x7f707400c860 comp rx=0 tx=0).stop 2023-07-13T22:21:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 shutdown_connections 2023-07-13T22:21:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f706806ace0 0x7f706806d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70840cfd50 0x7f70841e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70840b1ac0 0x7f70841e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 --2- 172.21.15.60:0/2502967193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840b8200 0x7f70840b6990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.471+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 >> 172.21.15.60:0/2502967193 conn(0x7f708406b2f0 msgr2=0x7f70840c8770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.472+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 shutdown_connections 2023-07-13T22:21:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.472+0000 7f708bdff700 1 -- 172.21.15.60:0/2502967193 wait complete. 2023-07-13T22:21:35.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:35.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.696+0000 7fd037dff700 1 Processor -- start 2023-07-13T22:21:35.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.696+0000 7fd037dff700 1 -- start start 2023-07-13T22:21:35.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.697+0000 7fd037dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0300b1ac0 0x7fd0300d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.697+0000 7fd037dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0300d1870 con 0x7fd0300b8200 2023-07-13T22:21:35.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.697+0000 7fd037dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0300cf370 con 0x7fd0300cfd50 2023-07-13T22:21:35.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.697+0000 7fd037dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0300d12b0 con 0x7fd0300b1ac0 2023-07-13T22:21:35.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.698+0000 7fd035b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd0300b8200 0x7fd0300cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39190/0 (socket says 172.21.15.60:39190) 2023-07-13T22:21:35.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.698+0000 7fd035b9b700 1 -- 172.21.15.60:0/2931822123 learned_addr learned my addr 172.21.15.60:0/2931822123 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:35.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.698+0000 7fd03639c700 1 --2- 172.21.15.60:0/2931822123 >> v2:172.21.15.60:3300/0 conn(0x7fd0300b1ac0 0x7fd0300d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.698+0000 7fd03639c700 1 -- 172.21.15.60:0/2931822123 >> v1:172.21.15.105:6789/0 conn(0x7fd0300cfd50 legacy=0x7fd0300d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:35.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.698+0000 7fd03639c700 1 -- 172.21.15.60:0/2931822123 >> v1:172.21.15.60:6789/0 conn(0x7fd0300b8200 legacy=0x7fd0300cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.698+0000 7fd03639c700 1 -- 172.21.15.60:0/2931822123 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0300d1590 con 0x7fd0300b1ac0 2023-07-13T22:21:35.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.698+0000 7fd034b99700 1 -- 172.21.15.60:0/2931822123 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209255533 0 0) 0x7fd0300d1870 con 0x7fd0300b8200 2023-07-13T22:21:35.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.698+0000 7fd03639c700 1 --2- 172.21.15.60:0/2931822123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b1ac0 0x7fd0300d2ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd028009d90 tx=0x7fd02800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b75e2a8c3105dac server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:35.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd034b99700 1 -- 172.21.15.60:0/2931822123 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02800e070 con 0x7fd0300b1ac0 2023-07-13T22:21:35.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd034b99700 1 -- 172.21.15.60:0/2931822123 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02800bb90 con 0x7fd0300b1ac0 2023-07-13T22:21:35.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd034b99700 1 -- 172.21.15.60:0/2931822123 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028003a20 con 0x7fd0300b1ac0 2023-07-13T22:21:35.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd037dff700 1 -- 172.21.15.60:0/2931822123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b1ac0 msgr2=0x7fd0300d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd037dff700 1 --2- 172.21.15.60:0/2931822123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b1ac0 0x7fd0300d2ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd028009d90 tx=0x7fd02800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:35.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd037dff700 1 -- 172.21.15.60:0/2931822123 shutdown_connections 2023-07-13T22:21:35.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd037dff700 1 --2- 172.21.15.60:0/2931822123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b1ac0 0x7fd0300d2ad0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd037dff700 1 -- 172.21.15.60:0/2931822123 >> 172.21.15.60:0/2931822123 conn(0x7fd03006b2f0 msgr2=0x7fd03006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:35.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd037dff700 1 -- 172.21.15.60:0/2931822123 shutdown_connections 2023-07-13T22:21:35.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.699+0000 7fd037dff700 1 -- 172.21.15.60:0/2931822123 wait complete. 2023-07-13T22:21:35.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.701+0000 7fd037dff700 1 Processor -- start 2023-07-13T22:21:35.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.701+0000 7fd037dff700 1 -- start start 2023-07-13T22:21:35.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.702+0000 7fd037dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0300b1ac0 0x7fd030156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.702+0000 7fd037dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b8200 0x7fd03015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.702+0000 7fd037dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0300cfd50 0x7fd03015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.702+0000 7fd037dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd030156590 con 0x7fd0300b8200 2023-07-13T22:21:35.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.702+0000 7fd037dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03015e5a0 con 0x7fd0300b1ac0 2023-07-13T22:21:35.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.702+0000 7fd037dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03015e880 con 0x7fd0300cfd50 2023-07-13T22:21:35.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03539a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b8200 0x7fd03015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03539a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b8200 0x7fd03015db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59920/0 (socket says 172.21.15.60:59920) 2023-07-13T22:21:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03539a700 1 -- 172.21.15.60:0/669774636 learned_addr learned my addr 172.21.15.60:0/669774636 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd035b9b700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0300b1ac0 0x7fd030156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03639c700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0300cfd50 0x7fd03015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03539a700 1 -- 172.21.15.60:0/669774636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0300cfd50 msgr2=0x7fd03015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03539a700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0300cfd50 0x7fd03015a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03539a700 1 -- 172.21.15.60:0/669774636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0300b1ac0 msgr2=0x7fd030156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03539a700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0300b1ac0 0x7fd030156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03539a700 1 -- 172.21.15.60:0/669774636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd028009a70 con 0x7fd0300b8200 2023-07-13T22:21:35.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd03539a700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b8200 0x7fd03015db20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd02000b450 tx=0x7fd02000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:35.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.703+0000 7fd0267fc700 1 -- 172.21.15.60:0/669774636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd020011830 con 0x7fd0300b8200 2023-07-13T22:21:35.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.704+0000 7fd0267fc700 1 -- 172.21.15.60:0/669774636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd020011990 con 0x7fd0300b8200 2023-07-13T22:21:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.704+0000 7fd0267fc700 1 -- 172.21.15.60:0/669774636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd020011d40 con 0x7fd0300b8200 2023-07-13T22:21:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.704+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03015eb60 con 0x7fd0300b8200 2023-07-13T22:21:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.704+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03015f120 con 0x7fd0300b8200 2023-07-13T22:21:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.705+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03015f310 con 0x7fd0300b8200 2023-07-13T22:21:35.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.705+0000 7fd0267fc700 1 -- 172.21.15.60:0/669774636 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd02001a400 con 0x7fd0300b8200 2023-07-13T22:21:35.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.706+0000 7fd0267fc700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd01406acb0 0x7fd01406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:35.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.707+0000 7fd0267fc700 1 -- 172.21.15.60:0/669774636 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd020089070 con 0x7fd0300b8200 2023-07-13T22:21:35.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.707+0000 7fd035b9b700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd01406acb0 0x7fd01406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.707+0000 7fd035b9b700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd01406acb0 0x7fd01406d160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd02c007fd0 tx=0x7fd02c007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.711+0000 7fd0267fc700 1 -- 172.21.15.60:0/669774636 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd0200bb330 con 0x7fd0300b8200 2023-07-13T22:21:35.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.914+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd03004e210 con 0x7fd0300b8200 2023-07-13T22:21:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.915+0000 7fd0267fc700 1 -- 172.21.15.60:0/669774636 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0200150a0 con 0x7fd0300b8200 2023-07-13T22:21:35.900 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:35.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd01406acb0 msgr2=0x7fd01406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd01406acb0 0x7fd01406d160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd02c007fd0 tx=0x7fd02c007de0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b8200 msgr2=0x7fd03015db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b8200 0x7fd03015db20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd02000b450 tx=0x7fd02000b760 comp rx=0 tx=0).stop 2023-07-13T22:21:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 shutdown_connections 2023-07-13T22:21:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd01406acb0 0x7fd01406d160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd0300cfd50 0x7fd03015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0300b1ac0 0x7fd030156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 --2- 172.21.15.60:0/669774636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0300b8200 0x7fd03015db20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 >> 172.21.15.60:0/669774636 conn(0x7fd03006b2f0 msgr2=0x7fd0300d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 shutdown_connections 2023-07-13T22:21:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:35.917+0000 7fd037dff700 1 -- 172.21.15.60:0/669774636 wait complete. 2023-07-13T22:21:35.930 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:21:35.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-07-13T22:21:36.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.151+0000 7f74755ff700 1 Processor -- start 2023-07-13T22:21:36.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.151+0000 7f74755ff700 1 -- start start 2023-07-13T22:21:36.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.152+0000 7f74755ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700d0a50 0x7f74700cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.152+0000 7f74755ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74700cd260 0x7f74700d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.152+0000 7f74755ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74700d2410 con 0x7f74700d0e20 2023-07-13T22:21:36.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.152+0000 7f74755ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747015c900 con 0x7f74700cd260 2023-07-13T22:21:36.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.152+0000 7f74755ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74700d2660 con 0x7f74700d0a50 2023-07-13T22:21:36.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.153+0000 7f746e7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74700cd260 0x7f74700d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.153+0000 7f746e7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74700cd260 0x7f74700d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59928/0 (socket says 172.21.15.60:59928) 2023-07-13T22:21:36.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.153+0000 7f746e7fc700 1 -- 172.21.15.60:0/2839970271 learned_addr learned my addr 172.21.15.60:0/2839970271 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:36.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.153+0000 7f746effd700 1 --2- 172.21.15.60:0/2839970271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700d0a50 0x7f74700cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.153+0000 7f746e7fc700 1 -- 172.21.15.60:0/2839970271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700d0a50 msgr2=0x7f74700cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:36.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.153+0000 7f746e7fc700 1 --2- 172.21.15.60:0/2839970271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700d0a50 0x7f74700cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.153+0000 7f746e7fc700 1 -- 172.21.15.60:0/2839970271 >> v1:172.21.15.105:6789/0 conn(0x7f74700d0e20 legacy=0x7f74700b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.153+0000 7f746e7fc700 1 -- 172.21.15.60:0/2839970271 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74700d2940 con 0x7f74700cd260 2023-07-13T22:21:36.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.153+0000 7f746e7fc700 1 --2- 172.21.15.60:0/2839970271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700cd260 0x7f74700d3420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74640098f0 tx=0x7f7464009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29043cc106510029 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:36.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f746dffb700 1 -- 172.21.15.60:0/2839970271 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746400b8a0 con 0x7f74700cd260 2023-07-13T22:21:36.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f746dffb700 1 -- 172.21.15.60:0/2839970271 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746400ba00 con 0x7f74700cd260 2023-07-13T22:21:36.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f746dffb700 1 -- 172.21.15.60:0/2839970271 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746400bd90 con 0x7f74700cd260 2023-07-13T22:21:36.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f74755ff700 1 -- 172.21.15.60:0/2839970271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700cd260 msgr2=0x7f74700d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f74755ff700 1 --2- 172.21.15.60:0/2839970271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700cd260 0x7f74700d3420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74640098f0 tx=0x7f7464009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:36.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f74755ff700 1 -- 172.21.15.60:0/2839970271 shutdown_connections 2023-07-13T22:21:36.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f74755ff700 1 --2- 172.21.15.60:0/2839970271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700d0a50 0x7f74700cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f74755ff700 1 --2- 172.21.15.60:0/2839970271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700cd260 0x7f74700d3420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f74755ff700 1 -- 172.21.15.60:0/2839970271 >> 172.21.15.60:0/2839970271 conn(0x7f7470071600 msgr2=0x7f74700719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:36.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f74755ff700 1 -- 172.21.15.60:0/2839970271 shutdown_connections 2023-07-13T22:21:36.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.154+0000 7f74755ff700 1 -- 172.21.15.60:0/2839970271 wait complete. 2023-07-13T22:21:36.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.156+0000 7f74755ff700 1 Processor -- start 2023-07-13T22:21:36.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.156+0000 7f74755ff700 1 -- start start 2023-07-13T22:21:36.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.157+0000 7f74755ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700cd260 0x7f747015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.157+0000 7f74755ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74700d0a50 0x7f7470150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.157+0000 7f74755ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700d0e20 0x7f74700ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.157+0000 7f74755ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74700b6280 con 0x7f74700d0e20 2023-07-13T22:21:36.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.157+0000 7f74755ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747015cf80 con 0x7f74700d0a50 2023-07-13T22:21:36.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.157+0000 7f74755ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74700b7ea0 con 0x7f74700cd260 2023-07-13T22:21:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700d0e20 0x7f74700ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700d0e20 0x7f74700ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59936/0 (socket says 172.21.15.60:59936) 2023-07-13T22:21:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746f7fe700 1 -- 172.21.15.60:0/1236965540 learned_addr learned my addr 172.21.15.60:0/1236965540 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746e7fc700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74700d0a50 0x7f7470150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746effd700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700cd260 0x7f747015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746f7fe700 1 -- 172.21.15.60:0/1236965540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700cd260 msgr2=0x7f747015ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746f7fe700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700cd260 0x7f747015ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746f7fe700 1 -- 172.21.15.60:0/1236965540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74700d0a50 msgr2=0x7f7470150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746f7fe700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74700d0a50 0x7f7470150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746f7fe700 1 -- 172.21.15.60:0/1236965540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7464009550 con 0x7f74700d0e20 2023-07-13T22:21:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.158+0000 7f746f7fe700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700d0e20 0x7f74700ca6d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f746800cb50 tx=0x7f746800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.159+0000 7f74577fe700 1 -- 172.21.15.60:0/1236965540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468003c00 con 0x7f74700d0e20 2023-07-13T22:21:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.159+0000 7f74577fe700 1 -- 172.21.15.60:0/1236965540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7468003d60 con 0x7f74700d0e20 2023-07-13T22:21:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.159+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74700b64d0 con 0x7f74700d0e20 2023-07-13T22:21:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.159+0000 7f74577fe700 1 -- 172.21.15.60:0/1236965540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746800aa20 con 0x7f74700d0e20 2023-07-13T22:21:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.159+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7470150ba0 con 0x7f74700d0e20 2023-07-13T22:21:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.160+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747004e280 con 0x7f74700d0e20 2023-07-13T22:21:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.161+0000 7f74577fe700 1 -- 172.21.15.60:0/1236965540 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f746800ab80 con 0x7f74700d0e20 2023-07-13T22:21:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.162+0000 7f74577fe700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f745c06ace0 0x7f745c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.162+0000 7f74577fe700 1 -- 172.21.15.60:0/1236965540 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7468089530 con 0x7f74700d0e20 2023-07-13T22:21:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.162+0000 7f746effd700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f745c06ace0 0x7f745c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.163+0000 7f746effd700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f745c06ace0 0x7f745c06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74700b5ef0 tx=0x7f7460008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:36.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.167+0000 7f74577fe700 1 -- 172.21.15.60:0/1236965540 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7468054540 con 0x7f74700d0e20 2023-07-13T22:21:36.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.455+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7f74701e5290 con 0x7f74700d0e20 2023-07-13T22:21:36.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.667+0000 7f74577fe700 1 -- 172.21.15.60:0/1236965540 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v54) v1 ==== 168+0+0 (secure 0 0 0) 0x7f7468057b60 con 0x7f74700d0e20 2023-07-13T22:21:36.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f745c06ace0 msgr2=0x7f745c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f745c06ace0 0x7f745c06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74700b5ef0 tx=0x7f7460008040 comp rx=0 tx=0).stop 2023-07-13T22:21:36.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700d0e20 msgr2=0x7f74700ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700d0e20 0x7f74700ca6d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f746800cb50 tx=0x7f746800ce60 comp rx=0 tx=0).stop 2023-07-13T22:21:36.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 shutdown_connections 2023-07-13T22:21:36.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f745c06ace0 0x7f745c06d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700d0e20 0x7f74700ca6d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74700cd260 0x7f747015ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 --2- 172.21.15.60:0/1236965540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74700d0a50 0x7f7470150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.669+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 >> 172.21.15.60:0/1236965540 conn(0x7f7470071600 msgr2=0x7f7470156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:36.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.670+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 shutdown_connections 2023-07-13T22:21:36.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.670+0000 7f74755ff700 1 -- 172.21.15.60:0/1236965540 wait complete. 2023-07-13T22:21:36.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-13T22:21:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.894+0000 7fdd0a7ff700 1 Processor -- start 2023-07-13T22:21:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.894+0000 7fdd0a7ff700 1 -- start start 2023-07-13T22:21:36.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.895+0000 7fdd0a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040b8200 0x7fdd040cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.895+0000 7fdd0a7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd040b1ac0 0x7fdd040cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.895+0000 7fdd0a7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd040d1870 con 0x7fdd040cede0 2023-07-13T22:21:36.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.895+0000 7fdd0a7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd040cf370 con 0x7fdd040b1ac0 2023-07-13T22:21:36.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.895+0000 7fdd0a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd040d12b0 con 0x7fdd040b8200 2023-07-13T22:21:36.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd037fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd040b1ac0 0x7fdd040cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd037fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd040b1ac0 0x7fdd040cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59942/0 (socket says 172.21.15.60:59942) 2023-07-13T22:21:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd037fe700 1 -- 172.21.15.60:0/73532299 learned_addr learned my addr 172.21.15.60:0/73532299 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd08d9c700 1 --1- 172.21.15.60:0/73532299 >> v1:172.21.15.60:6789/0 conn(0x7fdd040cede0 0x7fdd0415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39204/0 (socket says 172.21.15.60:39204) 2023-07-13T22:21:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd03fff700 1 --2- 172.21.15.60:0/73532299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040b8200 0x7fdd040cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd037fe700 1 -- 172.21.15.60:0/73532299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040b8200 msgr2=0x7fdd040cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd037fe700 1 --2- 172.21.15.60:0/73532299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040b8200 0x7fdd040cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd037fe700 1 -- 172.21.15.60:0/73532299 >> v1:172.21.15.60:6789/0 conn(0x7fdd040cede0 legacy=0x7fdd0415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd037fe700 1 -- 172.21.15.60:0/73532299 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd040d1590 con 0x7fdd040b1ac0 2023-07-13T22:21:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.896+0000 7fdd02ffd700 1 -- 172.21.15.60:0/73532299 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600004725 0 0) 0x7fdd040d1870 con 0x7fdd040cede0 2023-07-13T22:21:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd037fe700 1 --2- 172.21.15.60:0/73532299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b1ac0 0x7fdd040cfb80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80098f0 tx=0x7fdcf8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adebe75cfd983779 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd02ffd700 1 -- 172.21.15.60:0/73532299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8011830 con 0x7fdd040b1ac0 2023-07-13T22:21:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd02ffd700 1 -- 172.21.15.60:0/73532299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf8011990 con 0x7fdd040b1ac0 2023-07-13T22:21:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd02ffd700 1 -- 172.21.15.60:0/73532299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8011d20 con 0x7fdd040b1ac0 2023-07-13T22:21:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/73532299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b1ac0 msgr2=0x7fdd040cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd0a7ff700 1 --2- 172.21.15.60:0/73532299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b1ac0 0x7fdd040cfb80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80098f0 tx=0x7fdcf8009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/73532299 shutdown_connections 2023-07-13T22:21:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd0a7ff700 1 --2- 172.21.15.60:0/73532299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040b8200 0x7fdd040cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd0a7ff700 1 --2- 172.21.15.60:0/73532299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b1ac0 0x7fdd040cfb80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/73532299 >> 172.21.15.60:0/73532299 conn(0x7fdd0406b2f0 msgr2=0x7fdd0406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/73532299 shutdown_connections 2023-07-13T22:21:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.897+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/73532299 wait complete. 2023-07-13T22:21:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.899+0000 7fdd0a7ff700 1 Processor -- start 2023-07-13T22:21:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.899+0000 7fdd0a7ff700 1 -- start start 2023-07-13T22:21:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.900+0000 7fdd0a7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd040b1ac0 0x7fdd040cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.900+0000 7fdd0a7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b8200 0x7fdd04150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.900+0000 7fdd0a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040cede0 0x7fdd040c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.900+0000 7fdd0a7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04158b50 con 0x7fdd040b8200 2023-07-13T22:21:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.900+0000 7fdd0a7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd041511f0 con 0x7fdd040b1ac0 2023-07-13T22:21:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.900+0000 7fdd0a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd041514d0 con 0x7fdd040cede0 2023-07-13T22:21:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd037fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b8200 0x7fdd04150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd037fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b8200 0x7fdd04150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59950/0 (socket says 172.21.15.60:59950) 2023-07-13T22:21:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd037fe700 1 -- 172.21.15.60:0/3601735141 learned_addr learned my addr 172.21.15.60:0/3601735141 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd08d9c700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040cede0 0x7fdd040c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd03fff700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd040b1ac0 0x7fdd040cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd037fe700 1 -- 172.21.15.60:0/3601735141 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040cede0 msgr2=0x7fdd040c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd037fe700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040cede0 0x7fdd040c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd037fe700 1 -- 172.21.15.60:0/3601735141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd040b1ac0 msgr2=0x7fdd040cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd037fe700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd040b1ac0 0x7fdd040cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd037fe700 1 -- 172.21.15.60:0/3601735141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf8009550 con 0x7fdd040b8200 2023-07-13T22:21:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.901+0000 7fdd037fe700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b8200 0x7fdd04150810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009c40 tx=0x7fdcf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.902+0000 7fdd00ff9700 1 -- 172.21.15.60:0/3601735141 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80036a0 con 0x7fdd040b8200 2023-07-13T22:21:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.902+0000 7fdd00ff9700 1 -- 172.21.15.60:0/3601735141 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf801f070 con 0x7fdd040b8200 2023-07-13T22:21:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.902+0000 7fdd00ff9700 1 -- 172.21.15.60:0/3601735141 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8003a40 con 0x7fdd040b8200 2023-07-13T22:21:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.902+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd041517b0 con 0x7fdd040b8200 2023-07-13T22:21:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.902+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd040cbf20 con 0x7fdd040b8200 2023-07-13T22:21:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.903+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd04158c90 con 0x7fdd040b8200 2023-07-13T22:21:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.904+0000 7fdd00ff9700 1 -- 172.21.15.60:0/3601735141 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdcf8003ba0 con 0x7fdd040b8200 2023-07-13T22:21:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.905+0000 7fdd00ff9700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdcf006ace0 0x7fdcf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.905+0000 7fdd00ff9700 1 -- 172.21.15.60:0/3601735141 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdcf8021070 con 0x7fdd040b8200 2023-07-13T22:21:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.905+0000 7fdd03fff700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdcf006ace0 0x7fdcf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.906+0000 7fdd03fff700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdcf006ace0 0x7fdcf006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009b70 tx=0x7fdcf4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:36.909+0000 7fdd00ff9700 1 -- 172.21.15.60:0/3601735141 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdd04158c90 con 0x7fdd040b8200 2023-07-13T22:21:37.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.117+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdd04158c90 con 0x7fdd040b8200 2023-07-13T22:21:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.121+0000 7fdd00ff9700 1 -- 172.21.15.60:0/3601735141 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdd04158c90 con 0x7fdd040b8200 2023-07-13T22:21:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdcf006ace0 msgr2=0x7fdcf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdcf006ace0 0x7fdcf006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009b70 tx=0x7fdcf4008040 comp rx=0 tx=0).stop 2023-07-13T22:21:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b8200 msgr2=0x7fdd04150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b8200 0x7fdd04150810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009c40 tx=0x7fdcf8004750 comp rx=0 tx=0).stop 2023-07-13T22:21:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 shutdown_connections 2023-07-13T22:21:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdcf006ace0 0x7fdcf006d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd040cede0 0x7fdd040c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd040b1ac0 0x7fdd040cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 --2- 172.21.15.60:0/3601735141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd040b8200 0x7fdd04150810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 >> 172.21.15.60:0/3601735141 conn(0x7fdd0406b2f0 msgr2=0x7fdd040d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 shutdown_connections 2023-07-13T22:21:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.123+0000 7fdd0a7ff700 1 -- 172.21.15.60:0/3601735141 wait complete. 2023-07-13T22:21:37.127 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:21:37.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:21:37.228 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:21:37.333 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:21:37.333 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:21:37.333 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:37.380 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:21:37.380 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:37.467 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:37.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.725+0000 7fe2533ff700 1 Processor -- start 2023-07-13T22:21:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.725+0000 7fe2533ff700 1 -- start start 2023-07-13T22:21:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.726+0000 7fe2533ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe24c069ed0 0x7fe24c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.726+0000 7fe2533ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 0x7fe24c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.726+0000 7fe2533ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24c06a860 con 0x7fe24c158870 2023-07-13T22:21:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.726+0000 7fe2533ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24c0c3fc0 con 0x7fe24c069ed0 2023-07-13T22:21:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.726+0000 7fe2533ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0c42a0 con 0x7fe24c0c9300 2023-07-13T22:21:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.726+0000 7fe25119b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe24c158870 0x7fe24c0c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39216/0 (socket says 172.21.15.60:39216) 2023-07-13T22:21:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.726+0000 7fe25119b700 1 -- 172.21.15.60:0/526451628 learned_addr learned my addr 172.21.15.60:0/526451628 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:37.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.727+0000 7fe25099a700 1 --2- 172.21.15.60:0/526451628 >> v2:172.21.15.105:3300/0 conn(0x7fe24c069ed0 0x7fe24c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:37.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.727+0000 7fe25199c700 1 --2- 172.21.15.60:0/526451628 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 0x7fe24c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:37.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.727+0000 7fe243fff700 1 -- 172.21.15.60:0/526451628 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756001860 0 0) 0x7fe248001110 con 0x7fe24c158870 2023-07-13T22:21:37.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.727+0000 7fe243fff700 1 -- 172.21.15.60:0/526451628 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2380035b0 con 0x7fe24c158870 2023-07-13T22:21:37.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.727+0000 7fe25199c700 1 -- 172.21.15.60:0/526451628 >> v2:172.21.15.105:3300/0 conn(0x7fe24c069ed0 msgr2=0x7fe24c06a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:37.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.727+0000 7fe25199c700 1 --2- 172.21.15.60:0/526451628 >> v2:172.21.15.105:3300/0 conn(0x7fe24c069ed0 0x7fe24c06a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.727+0000 7fe25199c700 1 -- 172.21.15.60:0/526451628 >> v1:172.21.15.60:6789/0 conn(0x7fe24c158870 legacy=0x7fe24c0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:37.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.727+0000 7fe25199c700 1 -- 172.21.15.60:0/526451628 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c0c4580 con 0x7fe24c0c9300 2023-07-13T22:21:37.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.727+0000 7fe25099a700 1 --2- 172.21.15.60:0/526451628 >> v2:172.21.15.105:3300/0 conn(0x7fe24c069ed0 0x7fe24c06a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:37.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe25199c700 1 --2- 172.21.15.60:0/526451628 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 0x7fe24c0c9740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe244009d90 tx=0x7fe24400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0b505c7164c3927 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:37.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe243fff700 1 -- 172.21.15.60:0/526451628 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24400bd20 con 0x7fe24c0c9300 2023-07-13T22:21:37.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe243fff700 1 -- 172.21.15.60:0/526451628 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24400be80 con 0x7fe24c0c9300 2023-07-13T22:21:37.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe243fff700 1 -- 172.21.15.60:0/526451628 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2440116a0 con 0x7fe24c0c9300 2023-07-13T22:21:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe2533ff700 1 -- 172.21.15.60:0/526451628 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 msgr2=0x7fe24c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe2533ff700 1 --2- 172.21.15.60:0/526451628 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 0x7fe24c0c9740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe244009d90 tx=0x7fe24400b330 comp rx=0 tx=0).stop 2023-07-13T22:21:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe2533ff700 1 -- 172.21.15.60:0/526451628 shutdown_connections 2023-07-13T22:21:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe2533ff700 1 --2- 172.21.15.60:0/526451628 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 0x7fe24c0c9740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe2533ff700 1 --2- 172.21.15.60:0/526451628 >> v2:172.21.15.105:3300/0 conn(0x7fe24c069ed0 0x7fe24c06a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe2533ff700 1 -- 172.21.15.60:0/526451628 >> 172.21.15.60:0/526451628 conn(0x7fe24c0cf410 msgr2=0x7fe24c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.728+0000 7fe2533ff700 1 -- 172.21.15.60:0/526451628 shutdown_connections 2023-07-13T22:21:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.729+0000 7fe2533ff700 1 -- 172.21.15.60:0/526451628 wait complete. 2023-07-13T22:21:37.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.731+0000 7fe2533ff700 1 Processor -- start 2023-07-13T22:21:37.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.731+0000 7fe2533ff700 1 -- start start 2023-07-13T22:21:37.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.732+0000 7fe2533ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe24c069ed0 0x7fe24c142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:37.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.732+0000 7fe2533ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 0x7fe24c1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:37.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.732+0000 7fe2533ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe24c158870 0x7fe24c0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:37.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.732+0000 7fe2533ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c154a00 con 0x7fe24c069ed0 2023-07-13T22:21:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.732+0000 7fe2533ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0b0360 con 0x7fe24c158870 2023-07-13T22:21:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.732+0000 7fe2533ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0af750 con 0x7fe24c0c9300 2023-07-13T22:21:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe24c069ed0 0x7fe24c142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe24c069ed0 0x7fe24c142490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59962/0 (socket says 172.21.15.60:59962) 2023-07-13T22:21:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25119b700 1 -- 172.21.15.60:0/2565366523 learned_addr learned my addr 172.21.15.60:0/2565366523 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25099a700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 0x7fe24c1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25199c700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe24c158870 0x7fe24c0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25119b700 1 -- 172.21.15.60:0/2565366523 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 msgr2=0x7fe24c1544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25119b700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 0x7fe24c1544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25119b700 1 -- 172.21.15.60:0/2565366523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe24c158870 msgr2=0x7fe24c0b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25119b700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe24c158870 0x7fe24c0b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25119b700 1 -- 172.21.15.60:0/2565366523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0094a0 con 0x7fe24c069ed0 2023-07-13T22:21:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.733+0000 7fe25119b700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe24c069ed0 0x7fe24c142490 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe248007fd0 tx=0x7fe24800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:37.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.734+0000 7fe241ffb700 1 -- 172.21.15.60:0/2565366523 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248005d70 con 0x7fe24c069ed0 2023-07-13T22:21:37.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.734+0000 7fe241ffb700 1 -- 172.21.15.60:0/2565366523 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2480070d0 con 0x7fe24c069ed0 2023-07-13T22:21:37.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.734+0000 7fe241ffb700 1 -- 172.21.15.60:0/2565366523 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248007460 con 0x7fe24c069ed0 2023-07-13T22:21:37.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.734+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe244009a70 con 0x7fe24c069ed0 2023-07-13T22:21:37.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.734+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c0afd10 con 0x7fe24c069ed0 2023-07-13T22:21:37.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.735+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24c06cd50 con 0x7fe24c069ed0 2023-07-13T22:21:37.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.736+0000 7fe241ffb700 1 -- 172.21.15.60:0/2565366523 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe2480075c0 con 0x7fe24c069ed0 2023-07-13T22:21:37.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.737+0000 7fe241ffb700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe23006ace0 0x7fe23006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:37.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.737+0000 7fe241ffb700 1 -- 172.21.15.60:0/2565366523 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe2480886f0 con 0x7fe24c069ed0 2023-07-13T22:21:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.737+0000 7fe25099a700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe23006ace0 0x7fe23006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.738+0000 7fe25099a700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe23006ace0 0x7fe23006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe24c14b120 tx=0x7fe23c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.741+0000 7fe241ffb700 1 -- 172.21.15.60:0/2565366523 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe248053790 con 0x7fe24c069ed0 2023-07-13T22:21:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.947+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe24c06a860 con 0x7fe24c069ed0 2023-07-13T22:21:37.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.949+0000 7fe241ffb700 1 -- 172.21.15.60:0/2565366523 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7fe24c06a860 con 0x7fe24c069ed0 2023-07-13T22:21:37.934 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:37.934 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe23006ace0 msgr2=0x7fe23006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe23006ace0 0x7fe23006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe24c14b120 tx=0x7fe23c009210 comp rx=0 tx=0).stop 2023-07-13T22:21:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe24c069ed0 msgr2=0x7fe24c142490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe24c069ed0 0x7fe24c142490 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe248007fd0 tx=0x7fe24800c9e0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 shutdown_connections 2023-07-13T22:21:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe23006ace0 0x7fe23006d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe24c069ed0 0x7fe24c142490 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe24c158870 0x7fe24c0b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 --2- 172.21.15.60:0/2565366523 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe24c0c9300 0x7fe24c1544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 >> 172.21.15.60:0/2565366523 conn(0x7fe24c0cf410 msgr2=0x7fe24c06daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 shutdown_connections 2023-07-13T22:21:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:37.951+0000 7fe2533ff700 1 -- 172.21.15.60:0/2565366523 wait complete. 2023-07-13T22:21:37.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:37.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:38.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.176+0000 7f5c7bdff700 1 Processor -- start 2023-07-13T22:21:38.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.176+0000 7f5c7bdff700 1 -- start start 2023-07-13T22:21:38.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7bdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5c740b8200 0x7f5c740cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7bdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c740b1ac0 0x7f5c740cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7bdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c740d1870 con 0x7f5c740cede0 2023-07-13T22:21:38.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7bdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c740cf370 con 0x7f5c740b1ac0 2023-07-13T22:21:38.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7bdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c740d12b0 con 0x7f5c740b8200 2023-07-13T22:21:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7939a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c740b1ac0 0x7f5c740cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7939a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c740b1ac0 0x7f5c740cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59966/0 (socket says 172.21.15.60:59966) 2023-07-13T22:21:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7939a700 1 -- 172.21.15.60:0/1836344963 learned_addr learned my addr 172.21.15.60:0/1836344963 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c79b9b700 1 --2- 172.21.15.60:0/1836344963 >> v2:172.21.15.105:3300/0 conn(0x7f5c740b8200 0x7f5c740cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7939a700 1 -- 172.21.15.60:0/1836344963 >> v2:172.21.15.105:3300/0 conn(0x7f5c740b8200 msgr2=0x7f5c740cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7939a700 1 --2- 172.21.15.60:0/1836344963 >> v2:172.21.15.105:3300/0 conn(0x7f5c740b8200 0x7f5c740cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7939a700 1 -- 172.21.15.60:0/1836344963 >> v1:172.21.15.105:6789/0 conn(0x7f5c740cede0 legacy=0x7f5c7415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.178+0000 7f5c7939a700 1 -- 172.21.15.60:0/1836344963 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c740d1590 con 0x7f5c740b1ac0 2023-07-13T22:21:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.179+0000 7f5c7939a700 1 --2- 172.21.15.60:0/1836344963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 0x7f5c740cfb80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5c740b2660 tx=0x7f5c64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d86c8e381ca8db server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.179+0000 7f5c78b99700 1 -- 172.21.15.60:0/1836344963 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004230 con 0x7f5c740b1ac0 2023-07-13T22:21:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.179+0000 7f5c78b99700 1 -- 172.21.15.60:0/1836344963 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c64004390 con 0x7f5c740b1ac0 2023-07-13T22:21:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.179+0000 7f5c78b99700 1 -- 172.21.15.60:0/1836344963 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640036a0 con 0x7f5c740b1ac0 2023-07-13T22:21:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.179+0000 7f5c7bdff700 1 -- 172.21.15.60:0/1836344963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 msgr2=0x7f5c740cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.179+0000 7f5c7bdff700 1 --2- 172.21.15.60:0/1836344963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 0x7f5c740cfb80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5c740b2660 tx=0x7f5c64009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.179+0000 7f5c7bdff700 1 -- 172.21.15.60:0/1836344963 shutdown_connections 2023-07-13T22:21:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.179+0000 7f5c7bdff700 1 --2- 172.21.15.60:0/1836344963 >> v2:172.21.15.105:3300/0 conn(0x7f5c740b8200 0x7f5c740cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.179+0000 7f5c7bdff700 1 --2- 172.21.15.60:0/1836344963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 0x7f5c740cfb80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.180+0000 7f5c7bdff700 1 -- 172.21.15.60:0/1836344963 >> 172.21.15.60:0/1836344963 conn(0x7f5c7406b2f0 msgr2=0x7f5c7406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:38.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.180+0000 7f5c7bdff700 1 -- 172.21.15.60:0/1836344963 shutdown_connections 2023-07-13T22:21:38.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.180+0000 7f5c7bdff700 1 -- 172.21.15.60:0/1836344963 wait complete. 2023-07-13T22:21:38.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.181+0000 7f5c7bdff700 1 Processor -- start 2023-07-13T22:21:38.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.181+0000 7f5c7bdff700 1 -- start start 2023-07-13T22:21:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.182+0000 7f5c7bdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 0x7f5c74149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.182+0000 7f5c7bdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c740b8200 0x7f5c740cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.182+0000 7f5c7bdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c740cede0 0x7f5c740cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.182+0000 7f5c7bdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7415fa00 con 0x7f5c740b1ac0 2023-07-13T22:21:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.182+0000 7f5c7bdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740cc2f0 con 0x7f5c740cede0 2023-07-13T22:21:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.182+0000 7f5c7bdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740cb6e0 con 0x7f5c740b8200 2023-07-13T22:21:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c79b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 0x7f5c74149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c79b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 0x7f5c74149130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59980/0 (socket says 172.21.15.60:59980) 2023-07-13T22:21:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c79b9b700 1 -- 172.21.15.60:0/299797564 learned_addr learned my addr 172.21.15.60:0/299797564 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c7939a700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c740b8200 0x7f5c740cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c7a39c700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c740cede0 0x7f5c740cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c79b9b700 1 -- 172.21.15.60:0/299797564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c740b8200 msgr2=0x7f5c740cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c79b9b700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c740b8200 0x7f5c740cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c79b9b700 1 -- 172.21.15.60:0/299797564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c740cede0 msgr2=0x7f5c740cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c79b9b700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c740cede0 0x7f5c740cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c79b9b700 1 -- 172.21.15.60:0/299797564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64009550 con 0x7f5c740b1ac0 2023-07-13T22:21:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.183+0000 7f5c79b9b700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 0x7f5c74149130 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e810 tx=0x7f5c7000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.184+0000 7f5c6a7fc700 1 -- 172.21.15.60:0/299797564 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000c070 con 0x7f5c740b1ac0 2023-07-13T22:21:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.184+0000 7f5c6a7fc700 1 -- 172.21.15.60:0/299797564 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70013610 con 0x7f5c740b1ac0 2023-07-13T22:21:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.184+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c740cb9c0 con 0x7f5c740b1ac0 2023-07-13T22:21:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.184+0000 7f5c6a7fc700 1 -- 172.21.15.60:0/299797564 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c700139c0 con 0x7f5c740b1ac0 2023-07-13T22:21:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.184+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c740cbf80 con 0x7f5c740b1ac0 2023-07-13T22:21:38.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.185+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c740cc5d0 con 0x7f5c740b1ac0 2023-07-13T22:21:38.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.186+0000 7f5c6a7fc700 1 -- 172.21.15.60:0/299797564 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c70013b20 con 0x7f5c740b1ac0 2023-07-13T22:21:38.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.187+0000 7f5c6a7fc700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c5806ace0 0x7f5c5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.187+0000 7f5c6a7fc700 1 -- 172.21.15.60:0/299797564 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c700892c0 con 0x7f5c740b1ac0 2023-07-13T22:21:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.187+0000 7f5c7939a700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c5806ace0 0x7f5c5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.187+0000 7f5c7939a700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c5806ace0 0x7f5c5806d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009130 tx=0x7f5c64008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:38.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.191+0000 7f5c6a7fc700 1 -- 172.21.15.60:0/299797564 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5c700542e0 con 0x7f5c740b1ac0 2023-07-13T22:21:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.397+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c740cc140 con 0x7f5c740b1ac0 2023-07-13T22:21:38.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.397+0000 7f5c6a7fc700 1 -- 172.21.15.60:0/299797564 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c70057900 con 0x7f5c740b1ac0 2023-07-13T22:21:38.383 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:38.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c5806ace0 msgr2=0x7f5c5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c5806ace0 0x7f5c5806d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009130 tx=0x7f5c64008030 comp rx=0 tx=0).stop 2023-07-13T22:21:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 msgr2=0x7f5c74149130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 0x7f5c74149130 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e810 tx=0x7f5c7000eb20 comp rx=0 tx=0).stop 2023-07-13T22:21:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 shutdown_connections 2023-07-13T22:21:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c5806ace0 0x7f5c5806d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c740b1ac0 0x7f5c74149130 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c740cede0 0x7f5c740cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 --2- 172.21.15.60:0/299797564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c740b8200 0x7f5c740cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 >> 172.21.15.60:0/299797564 conn(0x7f5c7406b2f0 msgr2=0x7f5c740732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:38.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 shutdown_connections 2023-07-13T22:21:38.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.399+0000 7f5c7bdff700 1 -- 172.21.15.60:0/299797564 wait complete. 2023-07-13T22:21:38.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.623+0000 7fef451ff700 1 Processor -- start 2023-07-13T22:21:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.623+0000 7fef451ff700 1 -- start start 2023-07-13T22:21:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef451ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400cd280 0x7fef400d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef451ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef400d0e40 0x7fef400d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef451ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef400d2430 con 0x7fef400d0a70 2023-07-13T22:21:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef451ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4015c920 con 0x7fef400d0e40 2023-07-13T22:21:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef451ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef400d2680 con 0x7fef400cd280 2023-07-13T22:21:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef3f59e700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef400d0e40 0x7fef400d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef3f59e700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef400d0e40 0x7fef400d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59990/0 (socket says 172.21.15.60:59990) 2023-07-13T22:21:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef3f59e700 1 -- 172.21.15.60:0/2258033026 learned_addr learned my addr 172.21.15.60:0/2258033026 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef3e59c700 1 --2- 172.21.15.60:0/2258033026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400cd280 0x7fef400d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef3f59e700 1 -- 172.21.15.60:0/2258033026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400cd280 msgr2=0x7fef400d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef3f59e700 1 --2- 172.21.15.60:0/2258033026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400cd280 0x7fef400d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef3f59e700 1 -- 172.21.15.60:0/2258033026 >> v1:172.21.15.105:6789/0 conn(0x7fef400d0a70 legacy=0x7fef400cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef3f59e700 1 -- 172.21.15.60:0/2258033026 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400d2960 con 0x7fef400d0e40 2023-07-13T22:21:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.625+0000 7fef3e59c700 1 --2- 172.21.15.60:0/2258033026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400cd280 0x7fef400d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef3f59e700 1 --2- 172.21.15.60:0/2258033026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 0x7fef400d12a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef38009e50 tx=0x7fef3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19e38b80ec5f4b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef3dd9b700 1 -- 172.21.15.60:0/2258033026 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800e070 con 0x7fef400d0e40 2023-07-13T22:21:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef3dd9b700 1 -- 172.21.15.60:0/2258033026 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3800b9c0 con 0x7fef400d0e40 2023-07-13T22:21:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef3dd9b700 1 -- 172.21.15.60:0/2258033026 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800bd70 con 0x7fef400d0e40 2023-07-13T22:21:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef451ff700 1 -- 172.21.15.60:0/2258033026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 msgr2=0x7fef400d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef451ff700 1 --2- 172.21.15.60:0/2258033026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 0x7fef400d12a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef38009e50 tx=0x7fef3800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:38.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef451ff700 1 -- 172.21.15.60:0/2258033026 shutdown_connections 2023-07-13T22:21:38.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef451ff700 1 --2- 172.21.15.60:0/2258033026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 0x7fef400d12a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef451ff700 1 --2- 172.21.15.60:0/2258033026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400cd280 0x7fef400d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.626+0000 7fef451ff700 1 -- 172.21.15.60:0/2258033026 >> 172.21.15.60:0/2258033026 conn(0x7fef40071600 msgr2=0x7fef400719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:38.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.627+0000 7fef451ff700 1 -- 172.21.15.60:0/2258033026 shutdown_connections 2023-07-13T22:21:38.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.627+0000 7fef451ff700 1 -- 172.21.15.60:0/2258033026 wait complete. 2023-07-13T22:21:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.628+0000 7fef451ff700 1 Processor -- start 2023-07-13T22:21:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.628+0000 7fef451ff700 1 -- start start 2023-07-13T22:21:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.629+0000 7fef451ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef400cd280 0x7fef400c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.629+0000 7fef451ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400d0a70 0x7fef4013ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.629+0000 7fef451ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 0x7fef4015de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.629+0000 7fef451ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef401f1dc0 con 0x7fef400d0e40 2023-07-13T22:21:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.629+0000 7fef451ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40153280 con 0x7fef400cd280 2023-07-13T22:21:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.629+0000 7fef451ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef40152a00 con 0x7fef400d0a70 2023-07-13T22:21:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3f59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 0x7fef4015de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3f59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 0x7fef4015de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59998/0 (socket says 172.21.15.60:59998) 2023-07-13T22:21:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3f59e700 1 -- 172.21.15.60:0/1774030444 learned_addr learned my addr 172.21.15.60:0/1774030444 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3e59c700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400d0a70 0x7fef4013ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3ed9d700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef400cd280 0x7fef400c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3f59e700 1 -- 172.21.15.60:0/1774030444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400d0a70 msgr2=0x7fef4013ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3f59e700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400d0a70 0x7fef4013ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3f59e700 1 -- 172.21.15.60:0/1774030444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef400cd280 msgr2=0x7fef400c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3f59e700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef400cd280 0x7fef400c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3f59e700 1 -- 172.21.15.60:0/1774030444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38009b30 con 0x7fef400d0e40 2023-07-13T22:21:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3ed9d700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef400cd280 0x7fef400c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef3f59e700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 0x7fef4015de20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef38012000 tx=0x7fef38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.630+0000 7fef277fe700 1 -- 172.21.15.60:0/1774030444 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef380048b0 con 0x7fef400d0e40 2023-07-13T22:21:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.631+0000 7fef277fe700 1 -- 172.21.15.60:0/1774030444 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef380099b0 con 0x7fef400d0e40 2023-07-13T22:21:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.631+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef40155c60 con 0x7fef400d0e40 2023-07-13T22:21:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.631+0000 7fef277fe700 1 -- 172.21.15.60:0/1774030444 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3801b670 con 0x7fef400d0e40 2023-07-13T22:21:38.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.631+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef40152fa0 con 0x7fef400d0e40 2023-07-13T22:21:38.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.632+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef40067a70 con 0x7fef400d0e40 2023-07-13T22:21:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.633+0000 7fef277fe700 1 -- 172.21.15.60:0/1774030444 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef3801b7d0 con 0x7fef400d0e40 2023-07-13T22:21:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.634+0000 7fef277fe700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2c06ace0 0x7fef2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.634+0000 7fef277fe700 1 -- 172.21.15.60:0/1774030444 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef3808ad60 con 0x7fef400d0e40 2023-07-13T22:21:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.634+0000 7fef3ed9d700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2c06ace0 0x7fef2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.635+0000 7fef3ed9d700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2c06ace0 0x7fef2c06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef30000f60 tx=0x7fef30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:38.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.638+0000 7fef277fe700 1 -- 172.21.15.60:0/1774030444 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fef38055e00 con 0x7fef400d0e40 2023-07-13T22:21:38.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.843+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef4004e210 con 0x7fef400d0e40 2023-07-13T22:21:38.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.844+0000 7fef277fe700 1 -- 172.21.15.60:0/1774030444 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef38059420 con 0x7fef400d0e40 2023-07-13T22:21:38.829 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2c06ace0 msgr2=0x7fef2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2c06ace0 0x7fef2c06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef30000f60 tx=0x7fef30009040 comp rx=0 tx=0).stop 2023-07-13T22:21:38.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 msgr2=0x7fef4015de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:38.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 0x7fef4015de20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef38012000 tx=0x7fef38004ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 shutdown_connections 2023-07-13T22:21:38.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2c06ace0 0x7fef2c06d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef400d0e40 0x7fef4015de20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef400cd280 0x7fef400c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 --2- 172.21.15.60:0/1774030444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef400d0a70 0x7fef4013ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 >> 172.21.15.60:0/1774030444 conn(0x7fef40071600 msgr2=0x7fef400cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 shutdown_connections 2023-07-13T22:21:38.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:38.846+0000 7fef451ff700 1 -- 172.21.15.60:0/1774030444 wait complete. 2023-07-13T22:21:38.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:39.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.070+0000 7f3f823ff700 1 Processor -- start 2023-07-13T22:21:39.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.070+0000 7f3f823ff700 1 -- start start 2023-07-13T22:21:39.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.071+0000 7f3f823ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f7c0b8200 0x7f3f7c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.071+0000 7f3f823ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3f7c0cede0 0x7f3f7c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.071+0000 7f3f823ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f7c0d1870 con 0x7f3f7c0b1ac0 2023-07-13T22:21:39.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.071+0000 7f3f823ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7c0cf370 con 0x7f3f7c0b8200 2023-07-13T22:21:39.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.071+0000 7f3f823ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7c0d12b0 con 0x7f3f7c0cede0 2023-07-13T22:21:39.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.071+0000 7f3f7bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f7c0b8200 0x7f3f7c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f7bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f7c0b8200 0x7f3f7c0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60000/0 (socket says 172.21.15.60:60000) 2023-07-13T22:21:39.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f7bfff700 1 -- 172.21.15.60:0/2532462769 learned_addr learned my addr 172.21.15.60:0/2532462769 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:39.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f8099c700 1 --2- 172.21.15.60:0/2532462769 >> v2:172.21.15.105:3300/0 conn(0x7f3f7c0cede0 0x7f3f7c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f7bfff700 1 -- 172.21.15.60:0/2532462769 >> v2:172.21.15.105:3300/0 conn(0x7f3f7c0cede0 msgr2=0x7f3f7c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:39.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f7bfff700 1 --2- 172.21.15.60:0/2532462769 >> v2:172.21.15.105:3300/0 conn(0x7f3f7c0cede0 0x7f3f7c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f7bfff700 1 -- 172.21.15.60:0/2532462769 >> v1:172.21.15.105:6789/0 conn(0x7f3f7c0b1ac0 legacy=0x7f3f7c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:39.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f7bfff700 1 -- 172.21.15.60:0/2532462769 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c0d1590 con 0x7f3f7c0b8200 2023-07-13T22:21:39.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f8099c700 1 --2- 172.21.15.60:0/2532462769 >> v2:172.21.15.105:3300/0 conn(0x7f3f7c0cede0 0x7f3f7c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:39.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f7bfff700 1 --2- 172.21.15.60:0/2532462769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 0x7f3f7c0cf640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0c9c90 tx=0x7f3f6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8906bff41080fc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:39.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.072+0000 7f3f7affd700 1 -- 172.21.15.60:0/2532462769 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c004230 con 0x7f3f7c0b8200 2023-07-13T22:21:39.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f7affd700 1 -- 172.21.15.60:0/2532462769 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c004390 con 0x7f3f7c0b8200 2023-07-13T22:21:39.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f7affd700 1 -- 172.21.15.60:0/2532462769 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0036a0 con 0x7f3f7c0b8200 2023-07-13T22:21:39.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f823ff700 1 -- 172.21.15.60:0/2532462769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 msgr2=0x7f3f7c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f823ff700 1 --2- 172.21.15.60:0/2532462769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 0x7f3f7c0cf640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0c9c90 tx=0x7f3f6c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f823ff700 1 -- 172.21.15.60:0/2532462769 shutdown_connections 2023-07-13T22:21:39.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f823ff700 1 --2- 172.21.15.60:0/2532462769 >> v2:172.21.15.105:3300/0 conn(0x7f3f7c0cede0 0x7f3f7c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f823ff700 1 --2- 172.21.15.60:0/2532462769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 0x7f3f7c0cf640 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f823ff700 1 -- 172.21.15.60:0/2532462769 >> 172.21.15.60:0/2532462769 conn(0x7f3f7c06b2f0 msgr2=0x7f3f7c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:39.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f823ff700 1 -- 172.21.15.60:0/2532462769 shutdown_connections 2023-07-13T22:21:39.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.073+0000 7f3f823ff700 1 -- 172.21.15.60:0/2532462769 wait complete. 2023-07-13T22:21:39.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.076+0000 7f3f823ff700 1 Processor -- start 2023-07-13T22:21:39.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.076+0000 7f3f823ff700 1 -- start start 2023-07-13T22:21:39.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.077+0000 7f3f823ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f7c0b1ac0 0x7f3f7c1e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.077+0000 7f3f823ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 0x7f3f7c0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.077+0000 7f3f823ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f7c0cede0 0x7f3f7c1e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.077+0000 7f3f823ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c1e5e00 con 0x7f3f7c0b8200 2023-07-13T22:21:39.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.077+0000 7f3f823ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c1e1880 con 0x7f3f7c0cede0 2023-07-13T22:21:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.077+0000 7f3f823ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c1e1b60 con 0x7f3f7c0b1ac0 2023-07-13T22:21:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.077+0000 7f3f7b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 0x7f3f7c0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.077+0000 7f3f7b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 0x7f3f7c0c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60014/0 (socket says 172.21.15.60:60014) 2023-07-13T22:21:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.077+0000 7f3f7b7fe700 1 -- 172.21.15.60:0/2636967657 learned_addr learned my addr 172.21.15.60:0/2636967657 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f7bfff700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f7c0b1ac0 0x7f3f7c1e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f8099c700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f7c0cede0 0x7f3f7c1e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f7b7fe700 1 -- 172.21.15.60:0/2636967657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f7c0b1ac0 msgr2=0x7f3f7c1e58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f7b7fe700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f7c0b1ac0 0x7f3f7c1e58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f7b7fe700 1 -- 172.21.15.60:0/2636967657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f7c0cede0 msgr2=0x7f3f7c1e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f7b7fe700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f7c0cede0 0x7f3f7c1e3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f7b7fe700 1 -- 172.21.15.60:0/2636967657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c009550 con 0x7f3f7c0b8200 2023-07-13T22:21:39.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f8099c700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f7c0cede0 0x7f3f7c1e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:39.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f7b7fe700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 0x7f3f7c0c9410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000b4a0 tx=0x7f3f7000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:39.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f78ff9700 1 -- 172.21.15.60:0/2636967657 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70011830 con 0x7f3f7c0b8200 2023-07-13T22:21:39.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f78ff9700 1 -- 172.21.15.60:0/2636967657 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f70011990 con 0x7f3f7c0b8200 2023-07-13T22:21:39.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f78ff9700 1 -- 172.21.15.60:0/2636967657 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70011d40 con 0x7f3f7c0b8200 2023-07-13T22:21:39.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.078+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c1e1e40 con 0x7f3f7c0b8200 2023-07-13T22:21:39.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.079+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c1e2400 con 0x7f3f7c0b8200 2023-07-13T22:21:39.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.080+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c1e2620 con 0x7f3f7c0b8200 2023-07-13T22:21:39.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.081+0000 7f3f78ff9700 1 -- 172.21.15.60:0/2636967657 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f7001a400 con 0x7f3f7c0b8200 2023-07-13T22:21:39.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.082+0000 7f3f78ff9700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f6806ace0 0x7f3f6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.082+0000 7f3f78ff9700 1 -- 172.21.15.60:0/2636967657 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f70089050 con 0x7f3f7c0b8200 2023-07-13T22:21:39.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.082+0000 7f3f7bfff700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f6806ace0 0x7f3f6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.082+0000 7f3f7bfff700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f6806ace0 0x7f3f6806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009130 tx=0x7f3f6c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:39.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.086+0000 7f3f78ff9700 1 -- 172.21.15.60:0/2636967657 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3f70054190 con 0x7f3f7c0b8200 2023-07-13T22:21:39.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.292+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f7c04e210 con 0x7f3f7c0b8200 2023-07-13T22:21:39.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.293+0000 7f3f78ff9700 1 -- 172.21.15.60:0/2636967657 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f700577b0 con 0x7f3f7c0b8200 2023-07-13T22:21:39.279 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:39.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.295+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f6806ace0 msgr2=0x7f3f6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.295+0000 7f3f823ff700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f6806ace0 0x7f3f6806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009130 tx=0x7f3f6c008030 comp rx=0 tx=0).stop 2023-07-13T22:21:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.295+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 msgr2=0x7f3f7c0c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.295+0000 7f3f823ff700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 0x7f3f7c0c9410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000b4a0 tx=0x7f3f7000b7b0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.296+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 shutdown_connections 2023-07-13T22:21:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.296+0000 7f3f823ff700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f6806ace0 0x7f3f6806d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.296+0000 7f3f823ff700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f7c0cede0 0x7f3f7c1e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.296+0000 7f3f823ff700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f7c0b1ac0 0x7f3f7c1e58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.296+0000 7f3f823ff700 1 --2- 172.21.15.60:0/2636967657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f7c0b8200 0x7f3f7c0c9410 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.296+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 >> 172.21.15.60:0/2636967657 conn(0x7f3f7c06b2f0 msgr2=0x7f3f7c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.296+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 shutdown_connections 2023-07-13T22:21:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.296+0000 7f3f823ff700 1 -- 172.21.15.60:0/2636967657 wait complete. 2023-07-13T22:21:39.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:39.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.520+0000 7fdbf47ff700 1 Processor -- start 2023-07-13T22:21:39.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.520+0000 7fdbf47ff700 1 -- start start 2023-07-13T22:21:39.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.521+0000 7fdbf47ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdbec0c90e0 0x7fdbec069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.521+0000 7fdbf47ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec06d030 con 0x7fdbec158930 2023-07-13T22:21:39.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.521+0000 7fdbf47ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec06a870 con 0x7fdbec06a410 2023-07-13T22:21:39.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.521+0000 7fdbf47ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec06ca70 con 0x7fdbec0c90e0 2023-07-13T22:21:39.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.522+0000 7fdbf1d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdbec0c90e0 0x7fdbec069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.522+0000 7fdbf259b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdbec158930 0x7fdbec15c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39224/0 (socket says 172.21.15.60:39224) 2023-07-13T22:21:39.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.522+0000 7fdbf259b700 1 -- 172.21.15.60:0/1702301719 learned_addr learned my addr 172.21.15.60:0/1702301719 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:39.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.522+0000 7fdbf1d9a700 1 -- 172.21.15.60:0/1702301719 >> v1:172.21.15.105:6789/0 conn(0x7fdbec06a410 legacy=0x7fdbec06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.522+0000 7fdbf1d9a700 1 -- 172.21.15.60:0/1702301719 >> v1:172.21.15.60:6789/0 conn(0x7fdbec158930 legacy=0x7fdbec15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.522+0000 7fdbf1d9a700 1 -- 172.21.15.60:0/1702301719 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec0adc20 con 0x7fdbec0c90e0 2023-07-13T22:21:39.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.522+0000 7fdbf1d9a700 1 --2- 172.21.15.60:0/1702301719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec0c90e0 0x7fdbec069ed0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00a9e0 tx=0x7fdbdc00acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef4f68f8cfecd49 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:39.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf1599700 1 -- 172.21.15.60:0/1702301719 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00cb70 con 0x7fdbec0c90e0 2023-07-13T22:21:39.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf1599700 1 -- 172.21.15.60:0/1702301719 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc00ccd0 con 0x7fdbec0c90e0 2023-07-13T22:21:39.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf1599700 1 -- 172.21.15.60:0/1702301719 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc018620 con 0x7fdbec0c90e0 2023-07-13T22:21:39.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf47ff700 1 -- 172.21.15.60:0/1702301719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec0c90e0 msgr2=0x7fdbec069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf47ff700 1 --2- 172.21.15.60:0/1702301719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec0c90e0 0x7fdbec069ed0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00a9e0 tx=0x7fdbdc00acf0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf47ff700 1 -- 172.21.15.60:0/1702301719 shutdown_connections 2023-07-13T22:21:39.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf47ff700 1 --2- 172.21.15.60:0/1702301719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec0c90e0 0x7fdbec069ed0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf47ff700 1 -- 172.21.15.60:0/1702301719 >> 172.21.15.60:0/1702301719 conn(0x7fdbec0cbf50 msgr2=0x7fdbec0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:39.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf47ff700 1 -- 172.21.15.60:0/1702301719 shutdown_connections 2023-07-13T22:21:39.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.523+0000 7fdbf47ff700 1 -- 172.21.15.60:0/1702301719 wait complete. 2023-07-13T22:21:39.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.525+0000 7fdbf47ff700 1 Processor -- start 2023-07-13T22:21:39.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.525+0000 7fdbf47ff700 1 -- start start 2023-07-13T22:21:39.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.526+0000 7fdbf47ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec06a410 0x7fdbec144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.526+0000 7fdbf47ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbec0c90e0 0x7fdbec0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.526+0000 7fdbf47ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbec158930 0x7fdbec1f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.526+0000 7fdbf47ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0af920 con 0x7fdbec06a410 2023-07-13T22:21:39.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.526+0000 7fdbf47ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0afb70 con 0x7fdbec158930 2023-07-13T22:21:39.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.526+0000 7fdbf47ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0afe50 con 0x7fdbec0c90e0 2023-07-13T22:21:39.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf259b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec06a410 0x7fdbec144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf259b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec06a410 0x7fdbec144670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60046/0 (socket says 172.21.15.60:60046) 2023-07-13T22:21:39.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf259b700 1 -- 172.21.15.60:0/4203754595 learned_addr learned my addr 172.21.15.60:0/4203754595 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:39.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf1d9a700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbec0c90e0 0x7fdbec0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf2d9c700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbec158930 0x7fdbec1f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf259b700 1 -- 172.21.15.60:0/4203754595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbec0c90e0 msgr2=0x7fdbec0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf259b700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbec0c90e0 0x7fdbec0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf259b700 1 -- 172.21.15.60:0/4203754595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbec158930 msgr2=0x7fdbec1f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf259b700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbec158930 0x7fdbec1f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf259b700 1 -- 172.21.15.60:0/4203754595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc00a640 con 0x7fdbec06a410 2023-07-13T22:21:39.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf2d9c700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbec158930 0x7fdbec1f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:39.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.527+0000 7fdbf259b700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec06a410 0x7fdbec144670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800c480 tx=0x7fdbe800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:39.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.528+0000 7fdbe2ffd700 1 -- 172.21.15.60:0/4203754595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8005d70 con 0x7fdbec06a410 2023-07-13T22:21:39.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.528+0000 7fdbe2ffd700 1 -- 172.21.15.60:0/4203754595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe80070d0 con 0x7fdbec06a410 2023-07-13T22:21:39.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.528+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec0b0130 con 0x7fdbec06a410 2023-07-13T22:21:39.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.528+0000 7fdbe2ffd700 1 -- 172.21.15.60:0/4203754595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8007440 con 0x7fdbec06a410 2023-07-13T22:21:39.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.528+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec0b06f0 con 0x7fdbec06a410 2023-07-13T22:21:39.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.529+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbec04e210 con 0x7fdbec06a410 2023-07-13T22:21:39.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.530+0000 7fdbe2ffd700 1 -- 172.21.15.60:0/4203754595 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdbe80075a0 con 0x7fdbec06a410 2023-07-13T22:21:39.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.530+0000 7fdbe2ffd700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdbd006ace0 0x7fdbd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.531+0000 7fdbe2ffd700 1 -- 172.21.15.60:0/4203754595 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdbe8088980 con 0x7fdbec06a410 2023-07-13T22:21:39.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.531+0000 7fdbf1d9a700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdbd006ace0 0x7fdbd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.531+0000 7fdbf1d9a700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdbd006ace0 0x7fdbd006d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00a1c0 tx=0x7fdbdc01a040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:39.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.535+0000 7fdbe2ffd700 1 -- 172.21.15.60:0/4203754595 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdbe8053a40 con 0x7fdbec06a410 2023-07-13T22:21:39.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.741+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdbec0b47c0 con 0x7fdbec06a410 2023-07-13T22:21:39.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.742+0000 7fdbe2ffd700 1 -- 172.21.15.60:0/4203754595 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdbe8057060 con 0x7fdbec06a410 2023-07-13T22:21:39.727 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:39.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdbd006ace0 msgr2=0x7fdbd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdbd006ace0 0x7fdbd006d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00a1c0 tx=0x7fdbdc01a040 comp rx=0 tx=0).stop 2023-07-13T22:21:39.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec06a410 msgr2=0x7fdbec144670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec06a410 0x7fdbec144670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800c480 tx=0x7fdbe800c840 comp rx=0 tx=0).stop 2023-07-13T22:21:39.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 shutdown_connections 2023-07-13T22:21:39.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdbd006ace0 0x7fdbd006d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbec06a410 0x7fdbec144670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbec158930 0x7fdbec1f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 --2- 172.21.15.60:0/4203754595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbec0c90e0 0x7fdbec0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.744+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 >> 172.21.15.60:0/4203754595 conn(0x7fdbec0cbf50 msgr2=0x7fdbec0acb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:39.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.745+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 shutdown_connections 2023-07-13T22:21:39.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.745+0000 7fdbf47ff700 1 -- 172.21.15.60:0/4203754595 wait complete. 2023-07-13T22:21:39.747 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:21:39.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.966+0000 7f2cefdff700 1 Processor -- start 2023-07-13T22:21:39.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.966+0000 7f2cefdff700 1 -- start start 2023-07-13T22:21:39.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.967+0000 7f2cefdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b2010 0x7f2ce80cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cefdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ce80b5a00 0x7f2ce815fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cefdff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce80d14a0 con 0x7f2ce80d26d0 2023-07-13T22:21:39.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cefdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce80d2570 con 0x7f2ce80b5a00 2023-07-13T22:21:39.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cefdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80d1780 con 0x7f2ce80b2010 2023-07-13T22:21:39.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2ced39a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2ce80d26d0 0x7f2ce80cf6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39230/0 (socket says 172.21.15.60:39230) 2023-07-13T22:21:39.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2ced39a700 1 -- 172.21.15.60:0/3216904724 learned_addr learned my addr 172.21.15.60:0/3216904724 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:39.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cee39c700 1 --2- 172.21.15.60:0/3216904724 >> v2:172.21.15.60:3300/0 conn(0x7f2ce80b5a00 0x7f2ce815fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cedb9b700 1 --2- 172.21.15.60:0/3216904724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b2010 0x7f2ce80cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cecb99700 1 -- 172.21.15.60:0/3216904724 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780227369 0 0) 0x7f2ce80d14a0 con 0x7f2ce80d26d0 2023-07-13T22:21:39.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cecb99700 1 -- 172.21.15.60:0/3216904724 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd40035b0 con 0x7f2ce80d26d0 2023-07-13T22:21:39.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cee39c700 1 -- 172.21.15.60:0/3216904724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b2010 msgr2=0x7f2ce80cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:39.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cee39c700 1 --2- 172.21.15.60:0/3216904724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b2010 0x7f2ce80cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cee39c700 1 -- 172.21.15.60:0/3216904724 >> v1:172.21.15.60:6789/0 conn(0x7f2ce80d26d0 legacy=0x7f2ce80cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.968+0000 7f2cee39c700 1 -- 172.21.15.60:0/3216904724 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce80cd2a0 con 0x7f2ce80b5a00 2023-07-13T22:21:39.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.969+0000 7f2cee39c700 1 --2- 172.21.15.60:0/3216904724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80b5a00 0x7f2ce815fee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009d90 tx=0x7f2ce000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b337f69c716223c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:39.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.969+0000 7f2cecb99700 1 -- 172.21.15.60:0/3216904724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce000e070 con 0x7f2ce80b5a00 2023-07-13T22:21:39.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.969+0000 7f2cecb99700 1 -- 172.21.15.60:0/3216904724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce000bb20 con 0x7f2ce80b5a00 2023-07-13T22:21:39.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.969+0000 7f2cecb99700 1 -- 172.21.15.60:0/3216904724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0012430 con 0x7f2ce80b5a00 2023-07-13T22:21:39.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.969+0000 7f2cefdff700 1 -- 172.21.15.60:0/3216904724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80b5a00 msgr2=0x7f2ce815fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.969+0000 7f2cefdff700 1 --2- 172.21.15.60:0/3216904724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80b5a00 0x7f2ce815fee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009d90 tx=0x7f2ce000b330 comp rx=0 tx=0).stop 2023-07-13T22:21:39.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.970+0000 7f2cefdff700 1 -- 172.21.15.60:0/3216904724 shutdown_connections 2023-07-13T22:21:39.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.970+0000 7f2cefdff700 1 --2- 172.21.15.60:0/3216904724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80b5a00 0x7f2ce815fee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.970+0000 7f2cefdff700 1 --2- 172.21.15.60:0/3216904724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b2010 0x7f2ce80cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.970+0000 7f2cefdff700 1 -- 172.21.15.60:0/3216904724 >> 172.21.15.60:0/3216904724 conn(0x7f2ce806b380 msgr2=0x7f2ce806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:39.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.970+0000 7f2cefdff700 1 -- 172.21.15.60:0/3216904724 shutdown_connections 2023-07-13T22:21:39.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.970+0000 7f2cefdff700 1 -- 172.21.15.60:0/3216904724 wait complete. 2023-07-13T22:21:39.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.971+0000 7f2cefdff700 1 Processor -- start 2023-07-13T22:21:39.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.972+0000 7f2cefdff700 1 -- start start 2023-07-13T22:21:39.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.972+0000 7f2cefdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ce80b2010 0x7f2ce80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.972+0000 7f2cefdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b5a00 0x7f2ce815edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.972+0000 7f2cefdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80d26d0 0x7f2ce80a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.972+0000 7f2cefdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80c8bc0 con 0x7f2ce80d26d0 2023-07-13T22:21:39.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.972+0000 7f2cefdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80c7f40 con 0x7f2ce80b2010 2023-07-13T22:21:39.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.972+0000 7f2cefdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80c8220 con 0x7f2ce80b5a00 2023-07-13T22:21:39.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cee39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80d26d0 0x7f2ce80a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cee39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80d26d0 0x7f2ce80a9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60064/0 (socket says 172.21.15.60:60064) 2023-07-13T22:21:39.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cee39c700 1 -- 172.21.15.60:0/4064523442 learned_addr learned my addr 172.21.15.60:0/4064523442 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:39.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2ced39a700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b5a00 0x7f2ce815edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cedb9b700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ce80b2010 0x7f2ce80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cee39c700 1 -- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b5a00 msgr2=0x7f2ce815edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cee39c700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b5a00 0x7f2ce815edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cee39c700 1 -- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ce80b2010 msgr2=0x7f2ce80c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:39.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cee39c700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ce80b2010 0x7f2ce80c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:39.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cee39c700 1 -- 172.21.15.60:0/4064523442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0009a70 con 0x7f2ce80d26d0 2023-07-13T22:21:39.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2ced39a700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b5a00 0x7f2ce815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:39.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cedb9b700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ce80b2010 0x7f2ce80c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:39.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.973+0000 7f2cee39c700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80d26d0 0x7f2ce80a9510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0014040 tx=0x7f2ce0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:39.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.974+0000 7f2cde7fc700 1 -- 172.21.15.60:0/4064523442 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0012430 con 0x7f2ce80d26d0 2023-07-13T22:21:39.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.974+0000 7f2cde7fc700 1 -- 172.21.15.60:0/4064523442 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce0020070 con 0x7f2ce80d26d0 2023-07-13T22:21:39.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.974+0000 7f2cde7fc700 1 -- 172.21.15.60:0/4064523442 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0003ea0 con 0x7f2ce80d26d0 2023-07-13T22:21:39.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.974+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce80c8500 con 0x7f2ce80d26d0 2023-07-13T22:21:39.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.974+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce80c7c30 con 0x7f2ce80d26d0 2023-07-13T22:21:39.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.975+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce80d1150 con 0x7f2ce80d26d0 2023-07-13T22:21:39.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.976+0000 7f2cde7fc700 1 -- 172.21.15.60:0/4064523442 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ce0004280 con 0x7f2ce80d26d0 2023-07-13T22:21:39.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.977+0000 7f2cde7fc700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ccc06ace0 0x7f2ccc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:39.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.977+0000 7f2cde7fc700 1 -- 172.21.15.60:0/4064523442 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ce0022070 con 0x7f2ce80d26d0 2023-07-13T22:21:39.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.977+0000 7f2cedb9b700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ccc06ace0 0x7f2ccc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:39.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.977+0000 7f2cedb9b700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ccc06ace0 0x7f2ccc06d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80cab40 tx=0x7f2ce4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:39.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:39.981+0000 7f2cde7fc700 1 -- 172.21.15.60:0/4064523442 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2ce0056250 con 0x7f2ce80d26d0 2023-07-13T22:21:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.202+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2ce804e280 con 0x7f2ce80d26d0 2023-07-13T22:21:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.203+0000 7f2cde7fc700 1 -- 172.21.15.60:0/4064523442 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2ce002c080 con 0x7f2ce80d26d0 2023-07-13T22:21:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ccc06ace0 msgr2=0x7f2ccc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ccc06ace0 0x7f2ccc06d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80cab40 tx=0x7f2ce4008040 comp rx=0 tx=0).stop 2023-07-13T22:21:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80d26d0 msgr2=0x7f2ce80a9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80d26d0 0x7f2ce80a9510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0014040 tx=0x7f2ce0004ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 shutdown_connections 2023-07-13T22:21:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ccc06ace0 0x7f2ccc06d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80d26d0 0x7f2ce80a9510 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ce80b2010 0x7f2ce80c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 --2- 172.21.15.60:0/4064523442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ce80b5a00 0x7f2ce815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.205+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 >> 172.21.15.60:0/4064523442 conn(0x7f2ce806b380 msgr2=0x7f2ce80b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.206+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 shutdown_connections 2023-07-13T22:21:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.206+0000 7f2cefdff700 1 -- 172.21.15.60:0/4064523442 wait complete. 2023-07-13T22:21:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:21:40.208 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:21:40.209 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:40.255 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.513+0000 7f7fb55ff700 1 Processor -- start 2023-07-13T22:21:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.513+0000 7f7fb55ff700 1 -- start start 2023-07-13T22:21:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.514+0000 7f7fb55ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0070270 0x7f7fb00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.514+0000 7f7fb55ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb015d970 con 0x7f7fb00c9c50 2023-07-13T22:21:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.514+0000 7f7fb55ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb0070c00 con 0x7f7fb0159170 2023-07-13T22:21:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.514+0000 7f7fb55ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb015cd60 con 0x7f7fb0070270 2023-07-13T22:21:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.514+0000 7f7faf7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7fb00c9c50 0x7f7fb00ca070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39236/0 (socket says 172.21.15.60:39236) 2023-07-13T22:21:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.514+0000 7f7faf7fe700 1 -- 172.21.15.60:0/3389865245 learned_addr learned my addr 172.21.15.60:0/3389865245 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.515+0000 7f7fa7fff700 1 --2- 172.21.15.60:0/3389865245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0070270 0x7f7fb00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.515+0000 7f7fae7fc700 1 -- 172.21.15.60:0/3389865245 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728071900 0 0) 0x7f7fb015d970 con 0x7f7fb00c9c50 2023-07-13T22:21:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.515+0000 7f7fae7fc700 1 -- 172.21.15.60:0/3389865245 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f9c0035b0 con 0x7f7fb00c9c50 2023-07-13T22:21:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.515+0000 7f7fae7fc700 1 -- 172.21.15.60:0/3389865245 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926461782 0 0) 0x7f7fa0001110 con 0x7f7fb0159170 2023-07-13T22:21:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.515+0000 7f7fae7fc700 1 -- 172.21.15.60:0/3389865245 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fb015d970 con 0x7f7fb0159170 2023-07-13T22:21:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.515+0000 7f7fa7fff700 1 -- 172.21.15.60:0/3389865245 >> v1:172.21.15.105:6789/0 conn(0x7f7fb0159170 legacy=0x7f7fb00c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.515+0000 7f7fa7fff700 1 -- 172.21.15.60:0/3389865245 >> v1:172.21.15.60:6789/0 conn(0x7f7fb00c9c50 legacy=0x7f7fb00ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.515+0000 7f7fa7fff700 1 -- 172.21.15.60:0/3389865245 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb015d040 con 0x7f7fb0070270 2023-07-13T22:21:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.515+0000 7f7fa7fff700 1 --2- 172.21.15.60:0/3389865245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0070270 0x7f7fb00706c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009870 tx=0x7f7f98009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49d06f42ff324ea server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.516+0000 7f7fae7fc700 1 -- 172.21.15.60:0/3389865245 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98011830 con 0x7f7fb0070270 2023-07-13T22:21:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.516+0000 7f7fae7fc700 1 -- 172.21.15.60:0/3389865245 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f98011990 con 0x7f7fb0070270 2023-07-13T22:21:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.516+0000 7f7fae7fc700 1 -- 172.21.15.60:0/3389865245 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98011d00 con 0x7f7fb0070270 2023-07-13T22:21:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.516+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3389865245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0070270 msgr2=0x7f7fb00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.516+0000 7f7fb55ff700 1 --2- 172.21.15.60:0/3389865245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0070270 0x7f7fb00706c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009870 tx=0x7f7f98009b80 comp rx=0 tx=0).stop 2023-07-13T22:21:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.516+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3389865245 shutdown_connections 2023-07-13T22:21:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.516+0000 7f7fb55ff700 1 --2- 172.21.15.60:0/3389865245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0070270 0x7f7fb00706c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.516+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3389865245 >> 172.21.15.60:0/3389865245 conn(0x7f7fb00cf410 msgr2=0x7f7fb00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.516+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3389865245 shutdown_connections 2023-07-13T22:21:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.517+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3389865245 wait complete. 2023-07-13T22:21:40.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.518+0000 7f7fb55ff700 1 Processor -- start 2023-07-13T22:21:40.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.518+0000 7f7fb55ff700 1 -- start start 2023-07-13T22:21:40.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.519+0000 7f7fb55ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb0070270 0x7f7fb00c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.519+0000 7f7fb55ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb00c9c50 0x7f7fb01509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.519+0000 7f7fb55ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0159170 0x7f7fb0157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.519+0000 7f7fb55ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb00c6f00 con 0x7f7fb0070270 2023-07-13T22:21:40.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.519+0000 7f7fb55ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0150f00 con 0x7f7fb00c9c50 2023-07-13T22:21:40.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.519+0000 7f7fb55ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01511a0 con 0x7f7fb0159170 2023-07-13T22:21:40.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.519+0000 7f7faeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb0070270 0x7f7fb00c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7faeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb0070270 0x7f7fb00c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60074/0 (socket says 172.21.15.60:60074) 2023-07-13T22:21:40.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7faeffd700 1 -- 172.21.15.60:0/3438395501 learned_addr learned my addr 172.21.15.60:0/3438395501 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:40.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7faf7fe700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0159170 0x7f7fb0157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7fa7fff700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb00c9c50 0x7f7fb01509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7faeffd700 1 -- 172.21.15.60:0/3438395501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0159170 msgr2=0x7f7fb0157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7faeffd700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0159170 0x7f7fb0157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7faeffd700 1 -- 172.21.15.60:0/3438395501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb00c9c50 msgr2=0x7f7fb01509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7faeffd700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb00c9c50 0x7f7fb01509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7faeffd700 1 -- 172.21.15.60:0/3438395501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f980094a0 con 0x7f7fb0070270 2023-07-13T22:21:40.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7faeffd700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb0070270 0x7f7fb00c69c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000c700 tx=0x7f7fa000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:40.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7fa77fe700 1 -- 172.21.15.60:0/3438395501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00052d0 con 0x7f7fb0070270 2023-07-13T22:21:40.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.520+0000 7f7fa77fe700 1 -- 172.21.15.60:0/3438395501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa0005430 con 0x7f7fb0070270 2023-07-13T22:21:40.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.521+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb0151480 con 0x7f7fb0070270 2023-07-13T22:21:40.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.521+0000 7f7fa77fe700 1 -- 172.21.15.60:0/3438395501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00038c0 con 0x7f7fb0070270 2023-07-13T22:21:40.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.521+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb0151a40 con 0x7f7fb0070270 2023-07-13T22:21:40.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.522+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb0071440 con 0x7f7fb0070270 2023-07-13T22:21:40.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.523+0000 7f7fa77fe700 1 -- 172.21.15.60:0/3438395501 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7fa0003a20 con 0x7f7fb0070270 2023-07-13T22:21:40.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.524+0000 7f7fa77fe700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9006ace0 0x7f7f9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.524+0000 7f7fa77fe700 1 -- 172.21.15.60:0/3438395501 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7fa00886e0 con 0x7f7fb0070270 2023-07-13T22:21:40.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.524+0000 7f7fa7fff700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9006ace0 0x7f7f9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.524+0000 7f7fa7fff700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9006ace0 0x7f7f9006d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f980090b0 tx=0x7f7f98008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:40.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.528+0000 7f7fa77fe700 1 -- 172.21.15.60:0/3438395501 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7fa00536f0 con 0x7f7fb0070270 2023-07-13T22:21:40.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.741+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fb0070c00 con 0x7f7fb0070270 2023-07-13T22:21:40.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.742+0000 7f7fa77fe700 1 -- 172.21.15.60:0/3438395501 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f7fb0070c00 con 0x7f7fb0070270 2023-07-13T22:21:40.728 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:40.728 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:20:55.524264+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9006ace0 msgr2=0x7f7f9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9006ace0 0x7f7f9006d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f980090b0 tx=0x7f7f98008030 comp rx=0 tx=0).stop 2023-07-13T22:21:40.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb0070270 msgr2=0x7f7fb00c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb0070270 0x7f7fb00c69c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000c700 tx=0x7f7fa000ca10 comp rx=0 tx=0).stop 2023-07-13T22:21:40.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 shutdown_connections 2023-07-13T22:21:40.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9006ace0 0x7f7f9006d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb0070270 0x7f7fb00c69c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb0159170 0x7f7fb0157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 --2- 172.21.15.60:0/3438395501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb00c9c50 0x7f7fb01509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 >> 172.21.15.60:0/3438395501 conn(0x7f7fb00cf410 msgr2=0x7f7fb00ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:40.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 shutdown_connections 2023-07-13T22:21:40.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.745+0000 7f7fb55ff700 1 -- 172.21.15.60:0/3438395501 wait complete. 2023-07-13T22:21:40.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:40.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:40.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.972+0000 7f6870fff700 1 Processor -- start 2023-07-13T22:21:40.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.972+0000 7f6870fff700 1 -- start start 2023-07-13T22:21:40.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.973+0000 7f6870fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c0d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.973+0000 7f6870fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686c0d1870 con 0x7f686c0b8200 2023-07-13T22:21:40.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.973+0000 7f6870fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686c0cf370 con 0x7f686c0cfd50 2023-07-13T22:21:40.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.973+0000 7f6870fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0d12b0 con 0x7f686c0b1ac0 2023-07-13T22:21:40.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.973+0000 7f686a59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f686c0b8200 0x7f686c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39246/0 (socket says 172.21.15.60:39246) 2023-07-13T22:21:40.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.973+0000 7f686a59c700 1 -- 172.21.15.60:0/628804064 learned_addr learned my addr 172.21.15.60:0/628804064 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.974+0000 7f686ad9d700 1 --2- 172.21.15.60:0/628804064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c0d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.974+0000 7f6869d9b700 1 -- 172.21.15.60:0/628804064 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632179747 0 0) 0x7f686c0d1870 con 0x7f686c0b8200 2023-07-13T22:21:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.974+0000 7f6869d9b700 1 -- 172.21.15.60:0/628804064 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68580035b0 con 0x7f686c0b8200 2023-07-13T22:21:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.974+0000 7f6869d9b700 1 -- 172.21.15.60:0/628804064 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392082050 0 0) 0x7f686c0cf370 con 0x7f686c0cfd50 2023-07-13T22:21:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.974+0000 7f6869d9b700 1 -- 172.21.15.60:0/628804064 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f686c0d1870 con 0x7f686c0cfd50 2023-07-13T22:21:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.974+0000 7f6869d9b700 1 -- 172.21.15.60:0/628804064 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3370506571 0 0) 0x7f68580035b0 con 0x7f686c0b8200 2023-07-13T22:21:40.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.974+0000 7f686ad9d700 1 -- 172.21.15.60:0/628804064 >> v1:172.21.15.105:6789/0 conn(0x7f686c0cfd50 legacy=0x7f686c0d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.974+0000 7f686ad9d700 1 -- 172.21.15.60:0/628804064 >> v1:172.21.15.60:6789/0 conn(0x7f686c0b8200 legacy=0x7f686c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.974+0000 7f686ad9d700 1 -- 172.21.15.60:0/628804064 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c0d1590 con 0x7f686c0b1ac0 2023-07-13T22:21:40.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.975+0000 7f686ad9d700 1 --2- 172.21.15.60:0/628804064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c0d2ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6864009dc0 tx=0x7f686400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef722d7d244be4e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:40.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.975+0000 7f6869d9b700 1 -- 172.21.15.60:0/628804064 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686400e070 con 0x7f686c0b1ac0 2023-07-13T22:21:40.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.975+0000 7f6869d9b700 1 -- 172.21.15.60:0/628804064 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864003a20 con 0x7f686c0b1ac0 2023-07-13T22:21:40.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.975+0000 7f6869d9b700 1 -- 172.21.15.60:0/628804064 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c001670 con 0x7f686c0b1ac0 2023-07-13T22:21:40.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.975+0000 7f6870fff700 1 -- 172.21.15.60:0/628804064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 msgr2=0x7f686c0d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.975+0000 7f6870fff700 1 --2- 172.21.15.60:0/628804064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c0d2ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6864009dc0 tx=0x7f686400b330 comp rx=0 tx=0).stop 2023-07-13T22:21:40.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.976+0000 7f6870fff700 1 -- 172.21.15.60:0/628804064 shutdown_connections 2023-07-13T22:21:40.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.976+0000 7f6870fff700 1 --2- 172.21.15.60:0/628804064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c0d2ad0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.976+0000 7f6870fff700 1 -- 172.21.15.60:0/628804064 >> 172.21.15.60:0/628804064 conn(0x7f686c06b2f0 msgr2=0x7f686c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:40.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.976+0000 7f6870fff700 1 -- 172.21.15.60:0/628804064 shutdown_connections 2023-07-13T22:21:40.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.976+0000 7f6870fff700 1 -- 172.21.15.60:0/628804064 wait complete. 2023-07-13T22:21:40.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.977+0000 7f6870fff700 1 Processor -- start 2023-07-13T22:21:40.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.978+0000 7f6870fff700 1 -- start start 2023-07-13T22:21:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.978+0000 7f6870fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.978+0000 7f6870fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f686c0b8200 0x7f686c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.978+0000 7f6870fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0cfd50 0x7f686c1f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f6870fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c158240 con 0x7f686c0cfd50 2023-07-13T22:21:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f6870fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c157630 con 0x7f686c0b8200 2023-07-13T22:21:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f6870fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c157910 con 0x7f686c0b1ac0 2023-07-13T22:21:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686a59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0cfd50 0x7f686c1f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f6863fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f686c0b8200 0x7f686c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0cfd50 0x7f686c1f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60086/0 (socket says 172.21.15.60:60086) 2023-07-13T22:21:40.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686ad9d700 1 -- 172.21.15.60:0/1713244814 learned_addr learned my addr 172.21.15.60:0/1713244814 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:40.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686a59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c1586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:46866/0 (socket says 172.21.15.60:46866) 2023-07-13T22:21:40.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686ad9d700 1 -- 172.21.15.60:0/1713244814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 msgr2=0x7f686c1586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686ad9d700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c1586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686ad9d700 1 -- 172.21.15.60:0/1713244814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f686c0b8200 msgr2=0x7f686c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686ad9d700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f686c0b8200 0x7f686c156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.979+0000 7f686ad9d700 1 -- 172.21.15.60:0/1713244814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6864009a70 con 0x7f686c0cfd50 2023-07-13T22:21:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.980+0000 7f686ad9d700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0cfd50 0x7f686c1f1c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f686400b370 tx=0x7f6864004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.980+0000 7f6862ffd700 1 -- 172.21.15.60:0/1713244814 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68640044c0 con 0x7f686c0cfd50 2023-07-13T22:21:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.980+0000 7f6862ffd700 1 -- 172.21.15.60:0/1713244814 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864004620 con 0x7f686c0cfd50 2023-07-13T22:21:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.980+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c157bf0 con 0x7f686c0cfd50 2023-07-13T22:21:40.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.980+0000 7f6862ffd700 1 -- 172.21.15.60:0/1713244814 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686401d470 con 0x7f686c0cfd50 2023-07-13T22:21:40.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.980+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c158c10 con 0x7f686c0cfd50 2023-07-13T22:21:40.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.981+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c1583b0 con 0x7f686c0cfd50 2023-07-13T22:21:40.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.982+0000 7f6862ffd700 1 -- 172.21.15.60:0/1713244814 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f686401d5d0 con 0x7f686c0cfd50 2023-07-13T22:21:40.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.983+0000 7f6862ffd700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f684c06ace0 0x7f684c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:40.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.983+0000 7f6862ffd700 1 -- 172.21.15.60:0/1713244814 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f686408b0f0 con 0x7f686c0cfd50 2023-07-13T22:21:40.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.983+0000 7f686a59c700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f684c06ace0 0x7f684c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:40.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.984+0000 7f686a59c700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f684c06ace0 0x7f684c06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f685c000f60 tx=0x7f685c000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:40.987+0000 7f6862ffd700 1 -- 172.21.15.60:0/1713244814 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6864056190 con 0x7f686c0cfd50 2023-07-13T22:21:41.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.196+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f686c04e210 con 0x7f686c0cfd50 2023-07-13T22:21:41.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.196+0000 7f6862ffd700 1 -- 172.21.15.60:0/1713244814 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f68640597b0 con 0x7f686c0cfd50 2023-07-13T22:21:41.182 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.198+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f684c06ace0 msgr2=0x7f684c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.198+0000 7f6870fff700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f684c06ace0 0x7f684c06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f685c000f60 tx=0x7f685c000bc0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.198+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0cfd50 msgr2=0x7f686c1f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.198+0000 7f6870fff700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0cfd50 0x7f686c1f1c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f686400b370 tx=0x7f6864004ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.199+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 shutdown_connections 2023-07-13T22:21:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.199+0000 7f6870fff700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f684c06ace0 0x7f684c06d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.199+0000 7f6870fff700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686c0cfd50 0x7f686c1f1c00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.199+0000 7f6870fff700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686c0b1ac0 0x7f686c1586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.199+0000 7f6870fff700 1 --2- 172.21.15.60:0/1713244814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f686c0b8200 0x7f686c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.199+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 >> 172.21.15.60:0/1713244814 conn(0x7f686c06b2f0 msgr2=0x7f686c0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.199+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 shutdown_connections 2023-07-13T22:21:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.199+0000 7f6870fff700 1 -- 172.21.15.60:0/1713244814 wait complete. 2023-07-13T22:21:41.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:41.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.423+0000 7f2a3fbff700 1 Processor -- start 2023-07-13T22:21:41.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.423+0000 7f2a3fbff700 1 -- start start 2023-07-13T22:21:41.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.424+0000 7f2a3fbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a380b8200 0x7f2a380cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.424+0000 7f2a3fbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2a380cede0 0x7f2a380cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.424+0000 7f2a3fbff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a380d1870 con 0x7f2a380b1ac0 2023-07-13T22:21:41.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.424+0000 7f2a3fbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a380cf370 con 0x7f2a380b8200 2023-07-13T22:21:41.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.424+0000 7f2a3fbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a380d12b0 con 0x7f2a380cede0 2023-07-13T22:21:41.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.425+0000 7f2a3d99b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a380b8200 0x7f2a380cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.425+0000 7f2a3d19a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2a380b1ac0 0x7f2a3815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39256/0 (socket says 172.21.15.60:39256) 2023-07-13T22:21:41.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.425+0000 7f2a3d99b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a380b8200 0x7f2a380cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60096/0 (socket says 172.21.15.60:60096) 2023-07-13T22:21:41.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.425+0000 7f2a3d19a700 1 -- 172.21.15.60:0/4043433640 learned_addr learned my addr 172.21.15.60:0/4043433640 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:41.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.425+0000 7f2a3e19c700 1 --2- 172.21.15.60:0/4043433640 >> v2:172.21.15.105:3300/0 conn(0x7f2a380cede0 0x7f2a380cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.425+0000 7f2a3d99b700 1 -- 172.21.15.60:0/4043433640 >> v2:172.21.15.105:3300/0 conn(0x7f2a380cede0 msgr2=0x7f2a380cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:41.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.425+0000 7f2a3d99b700 1 --2- 172.21.15.60:0/4043433640 >> v2:172.21.15.105:3300/0 conn(0x7f2a380cede0 0x7f2a380cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.425+0000 7f2a3d99b700 1 -- 172.21.15.60:0/4043433640 >> v1:172.21.15.60:6789/0 conn(0x7f2a380b1ac0 legacy=0x7f2a3815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.425+0000 7f2a3d99b700 1 -- 172.21.15.60:0/4043433640 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a380d1590 con 0x7f2a380b8200 2023-07-13T22:21:41.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3d99b700 1 --2- 172.21.15.60:0/4043433640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380b8200 0x7f2a380cf640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2a380c9c90 tx=0x7f2a34009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcac83a5d13fd54d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:41.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3c999700 1 -- 172.21.15.60:0/4043433640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34011830 con 0x7f2a380b8200 2023-07-13T22:21:41.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3c999700 1 -- 172.21.15.60:0/4043433640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a34011990 con 0x7f2a380b8200 2023-07-13T22:21:41.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3c999700 1 -- 172.21.15.60:0/4043433640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34011d20 con 0x7f2a380b8200 2023-07-13T22:21:41.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3fbff700 1 -- 172.21.15.60:0/4043433640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380b8200 msgr2=0x7f2a380cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3fbff700 1 --2- 172.21.15.60:0/4043433640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380b8200 0x7f2a380cf640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2a380c9c90 tx=0x7f2a34009b20 comp rx=0 tx=0).stop 2023-07-13T22:21:41.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3fbff700 1 -- 172.21.15.60:0/4043433640 shutdown_connections 2023-07-13T22:21:41.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3fbff700 1 --2- 172.21.15.60:0/4043433640 >> v2:172.21.15.105:3300/0 conn(0x7f2a380cede0 0x7f2a380cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3fbff700 1 --2- 172.21.15.60:0/4043433640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380b8200 0x7f2a380cf640 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3fbff700 1 -- 172.21.15.60:0/4043433640 >> 172.21.15.60:0/4043433640 conn(0x7f2a3806b2f0 msgr2=0x7f2a3806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:41.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.426+0000 7f2a3fbff700 1 -- 172.21.15.60:0/4043433640 shutdown_connections 2023-07-13T22:21:41.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.427+0000 7f2a3fbff700 1 -- 172.21.15.60:0/4043433640 wait complete. 2023-07-13T22:21:41.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.428+0000 7f2a3fbff700 1 Processor -- start 2023-07-13T22:21:41.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.428+0000 7f2a3fbff700 1 -- start start 2023-07-13T22:21:41.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.429+0000 7f2a3fbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a380b1ac0 0x7f2a38142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.429+0000 7f2a3fbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a380b8200 0x7f2a38144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.429+0000 7f2a3fbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380cede0 0x7f2a38144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3fbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38142ae0 con 0x7f2a380cede0 2023-07-13T22:21:41.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3fbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38142dc0 con 0x7f2a380b1ac0 2023-07-13T22:21:41.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3fbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a381430a0 con 0x7f2a380b8200 2023-07-13T22:21:41.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3e19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380cede0 0x7f2a38144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3e19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380cede0 0x7f2a38144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60110/0 (socket says 172.21.15.60:60110) 2023-07-13T22:21:41.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3e19c700 1 -- 172.21.15.60:0/508418564 learned_addr learned my addr 172.21.15.60:0/508418564 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:41.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3d99b700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a380b1ac0 0x7f2a38142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3d19a700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a380b8200 0x7f2a38144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3e19c700 1 -- 172.21.15.60:0/508418564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a380b8200 msgr2=0x7f2a38144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3e19c700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a380b8200 0x7f2a38144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3e19c700 1 -- 172.21.15.60:0/508418564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a380b1ac0 msgr2=0x7f2a38142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3e19c700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a380b1ac0 0x7f2a38142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.430+0000 7f2a3e19c700 1 -- 172.21.15.60:0/508418564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340094a0 con 0x7f2a380cede0 2023-07-13T22:21:41.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.431+0000 7f2a3e19c700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380cede0 0x7f2a38144660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2a30007e40 tx=0x7f2a3000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.431+0000 7f2a2e7fc700 1 -- 172.21.15.60:0/508418564 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000ed30 con 0x7f2a380cede0 2023-07-13T22:21:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.431+0000 7f2a2e7fc700 1 -- 172.21.15.60:0/508418564 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a30014420 con 0x7f2a380cede0 2023-07-13T22:21:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.431+0000 7f2a2e7fc700 1 -- 172.21.15.60:0/508418564 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a300147d0 con 0x7f2a380cede0 2023-07-13T22:21:41.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.431+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a38143380 con 0x7f2a380cede0 2023-07-13T22:21:41.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.431+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a38144e00 con 0x7f2a380cede0 2023-07-13T22:21:41.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.432+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a38144fc0 con 0x7f2a380cede0 2023-07-13T22:21:41.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.433+0000 7f2a2e7fc700 1 -- 172.21.15.60:0/508418564 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a30014930 con 0x7f2a380cede0 2023-07-13T22:21:41.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.434+0000 7f2a2e7fc700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a1c06ace0 0x7f2a1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.434+0000 7f2a2e7fc700 1 -- 172.21.15.60:0/508418564 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a30089c10 con 0x7f2a380cede0 2023-07-13T22:21:41.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.434+0000 7f2a3d99b700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a1c06ace0 0x7f2a1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.435+0000 7f2a3d99b700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a1c06ace0 0x7f2a1c06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a340090b0 tx=0x7f2a34008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:41.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.438+0000 7f2a2e7fc700 1 -- 172.21.15.60:0/508418564 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2a38144fc0 con 0x7f2a380cede0 2023-07-13T22:21:41.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.648+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a381438a0 con 0x7f2a380cede0 2023-07-13T22:21:41.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.648+0000 7f2a2e7fc700 1 -- 172.21.15.60:0/508418564 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a381438a0 con 0x7f2a380cede0 2023-07-13T22:21:41.633 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:41.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a1c06ace0 msgr2=0x7f2a1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a1c06ace0 0x7f2a1c06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a340090b0 tx=0x7f2a34008030 comp rx=0 tx=0).stop 2023-07-13T22:21:41.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380cede0 msgr2=0x7f2a38144660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380cede0 0x7f2a38144660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2a30007e40 tx=0x7f2a3000e3c0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 shutdown_connections 2023-07-13T22:21:41.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a1c06ace0 0x7f2a1c06d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a380cede0 0x7f2a38144660 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a380b1ac0 0x7f2a38142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 --2- 172.21.15.60:0/508418564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a380b8200 0x7f2a38144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 >> 172.21.15.60:0/508418564 conn(0x7f2a3806b2f0 msgr2=0x7f2a380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:41.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 shutdown_connections 2023-07-13T22:21:41.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.651+0000 7f2a3fbff700 1 -- 172.21.15.60:0/508418564 wait complete. 2023-07-13T22:21:41.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:41.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.881+0000 7fc39a3ff700 1 Processor -- start 2023-07-13T22:21:41.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.881+0000 7fc39a3ff700 1 -- start start 2023-07-13T22:21:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.882+0000 7fc39a3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc3940d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.882+0000 7fc39a3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3940d1870 con 0x7fc3940b8200 2023-07-13T22:21:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.882+0000 7fc39a3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3940cf370 con 0x7fc3940cfd50 2023-07-13T22:21:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.882+0000 7fc39a3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940d12b0 con 0x7fc3940b1ac0 2023-07-13T22:21:41.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.882+0000 7fc393fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc3940b8200 0x7fc3940cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39258/0 (socket says 172.21.15.60:39258) 2023-07-13T22:21:41.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.882+0000 7fc393fff700 1 -- 172.21.15.60:0/1070026008 learned_addr learned my addr 172.21.15.60:0/1070026008 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.883+0000 7fc39899c700 1 --2- 172.21.15.60:0/1070026008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc3940d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.883+0000 7fc392ffd700 1 -- 172.21.15.60:0/1070026008 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853663838 0 0) 0x7fc3940d1870 con 0x7fc3940b8200 2023-07-13T22:21:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.883+0000 7fc392ffd700 1 -- 172.21.15.60:0/1070026008 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc37c0035b0 con 0x7fc3940b8200 2023-07-13T22:21:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.883+0000 7fc392ffd700 1 -- 172.21.15.60:0/1070026008 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088845473 0 0) 0x7fc3940cf370 con 0x7fc3940cfd50 2023-07-13T22:21:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.883+0000 7fc392ffd700 1 -- 172.21.15.60:0/1070026008 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3940d1870 con 0x7fc3940cfd50 2023-07-13T22:21:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.883+0000 7fc39899c700 1 -- 172.21.15.60:0/1070026008 >> v1:172.21.15.105:6789/0 conn(0x7fc3940cfd50 legacy=0x7fc3940d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.883+0000 7fc39899c700 1 -- 172.21.15.60:0/1070026008 >> v1:172.21.15.60:6789/0 conn(0x7fc3940b8200 legacy=0x7fc3940cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.883+0000 7fc392ffd700 1 -- 172.21.15.60:0/1070026008 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3615690770 0 0) 0x7fc37c0035b0 con 0x7fc3940b8200 2023-07-13T22:21:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.883+0000 7fc39899c700 1 -- 172.21.15.60:0/1070026008 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3940d1590 con 0x7fc3940b1ac0 2023-07-13T22:21:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.884+0000 7fc39899c700 1 --2- 172.21.15.60:0/1070026008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc3940d2ad0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009dc0 tx=0x7fc38c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632fee75b2e2bc73 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.884+0000 7fc392ffd700 1 -- 172.21.15.60:0/1070026008 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c00e070 con 0x7fc3940b1ac0 2023-07-13T22:21:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.884+0000 7fc392ffd700 1 -- 172.21.15.60:0/1070026008 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c00bb20 con 0x7fc3940b1ac0 2023-07-13T22:21:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.884+0000 7fc392ffd700 1 -- 172.21.15.60:0/1070026008 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c00bed0 con 0x7fc3940b1ac0 2023-07-13T22:21:41.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.885+0000 7fc39a3ff700 1 -- 172.21.15.60:0/1070026008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 msgr2=0x7fc3940d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.885+0000 7fc39a3ff700 1 --2- 172.21.15.60:0/1070026008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc3940d2ad0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009dc0 tx=0x7fc38c00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.885+0000 7fc39a3ff700 1 -- 172.21.15.60:0/1070026008 shutdown_connections 2023-07-13T22:21:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.885+0000 7fc39a3ff700 1 --2- 172.21.15.60:0/1070026008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc3940d2ad0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.885+0000 7fc39a3ff700 1 -- 172.21.15.60:0/1070026008 >> 172.21.15.60:0/1070026008 conn(0x7fc39406b2f0 msgr2=0x7fc39406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.885+0000 7fc39a3ff700 1 -- 172.21.15.60:0/1070026008 shutdown_connections 2023-07-13T22:21:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.885+0000 7fc39a3ff700 1 -- 172.21.15.60:0/1070026008 wait complete. 2023-07-13T22:21:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.886+0000 7fc39a3ff700 1 Processor -- start 2023-07-13T22:21:41.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.887+0000 7fc39a3ff700 1 -- start start 2023-07-13T22:21:41.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc39a3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc394152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc39a3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3940b8200 0x7fc3941f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc39a3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3940cfd50 0x7fc39415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc39a3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3941592e0 con 0x7fc3940b8200 2023-07-13T22:21:41.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc39a3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394158d20 con 0x7fc3940cfd50 2023-07-13T22:21:41.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc39a3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc394159000 con 0x7fc3940b1ac0 2023-07-13T22:21:41.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc3937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3940b8200 0x7fc3941f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc3937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3940b8200 0x7fc3941f6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60122/0 (socket says 172.21.15.60:60122) 2023-07-13T22:21:41.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc3937fe700 1 -- 172.21.15.60:0/4194960468 learned_addr learned my addr 172.21.15.60:0/4194960468 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:41.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc393fff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc394152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc39899c700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3940cfd50 0x7fc39415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc3937fe700 1 -- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 msgr2=0x7fc394152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc3937fe700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc394152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc3937fe700 1 -- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3940cfd50 msgr2=0x7fc39415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:41.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc3937fe700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3940cfd50 0x7fc39415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:41.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc3937fe700 1 -- 172.21.15.60:0/4194960468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c009a70 con 0x7fc3940b8200 2023-07-13T22:21:41.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc39899c700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3940cfd50 0x7fc39415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:41.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.888+0000 7fc393fff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc394152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:41.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.889+0000 7fc3937fe700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3940b8200 0x7fc3941f6190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c4d0 tx=0x7fc38800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.889+0000 7fc390ff9700 1 -- 172.21.15.60:0/4194960468 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880052d0 con 0x7fc3940b8200 2023-07-13T22:21:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.889+0000 7fc390ff9700 1 -- 172.21.15.60:0/4194960468 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc388005430 con 0x7fc3940b8200 2023-07-13T22:21:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.889+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3941f5990 con 0x7fc3940b8200 2023-07-13T22:21:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.889+0000 7fc390ff9700 1 -- 172.21.15.60:0/4194960468 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880038a0 con 0x7fc3940b8200 2023-07-13T22:21:41.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.889+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3941f66d0 con 0x7fc3940b8200 2023-07-13T22:21:41.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.890+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3941f5eb0 con 0x7fc3940b8200 2023-07-13T22:21:41.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.891+0000 7fc390ff9700 1 -- 172.21.15.60:0/4194960468 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc388003a00 con 0x7fc3940b8200 2023-07-13T22:21:41.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.892+0000 7fc390ff9700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc38006ace0 0x7fc38006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:41.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.892+0000 7fc390ff9700 1 -- 172.21.15.60:0/4194960468 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc388088980 con 0x7fc3940b8200 2023-07-13T22:21:41.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.892+0000 7fc393fff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc38006ace0 0x7fc38006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:41.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.893+0000 7fc393fff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc38006ace0 0x7fc38006d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc384000f60 tx=0x7fc384009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:41.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:41.896+0000 7fc390ff9700 1 -- 172.21.15.60:0/4194960468 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc3941f5eb0 con 0x7fc3940b8200 2023-07-13T22:21:42.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.102+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3941f5eb0 con 0x7fc3940b8200 2023-07-13T22:21:42.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.103+0000 7fc390ff9700 1 -- 172.21.15.60:0/4194960468 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3941f5eb0 con 0x7fc3940b8200 2023-07-13T22:21:42.088 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:42.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.105+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc38006ace0 msgr2=0x7fc38006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.105+0000 7fc39a3ff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc38006ace0 0x7fc38006d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc384000f60 tx=0x7fc384009040 comp rx=0 tx=0).stop 2023-07-13T22:21:42.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.105+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3940b8200 msgr2=0x7fc3941f6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.105+0000 7fc39a3ff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3940b8200 0x7fc3941f6190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c4d0 tx=0x7fc38800c890 comp rx=0 tx=0).stop 2023-07-13T22:21:42.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.106+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 shutdown_connections 2023-07-13T22:21:42.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.106+0000 7fc39a3ff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc38006ace0 0x7fc38006d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.106+0000 7fc39a3ff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3940cfd50 0x7fc39415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.106+0000 7fc39a3ff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3940b1ac0 0x7fc394152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.106+0000 7fc39a3ff700 1 --2- 172.21.15.60:0/4194960468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3940b8200 0x7fc3941f6190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.106+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 >> 172.21.15.60:0/4194960468 conn(0x7fc39406b2f0 msgr2=0x7fc3940d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:42.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.106+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 shutdown_connections 2023-07-13T22:21:42.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.106+0000 7fc39a3ff700 1 -- 172.21.15.60:0/4194960468 wait complete. 2023-07-13T22:21:42.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:42.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.328+0000 7fe9681ff700 1 Processor -- start 2023-07-13T22:21:42.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.329+0000 7fe9681ff700 1 -- start start 2023-07-13T22:21:42.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.330+0000 7fe9681ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe960159260 0x7fe9600ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.330+0000 7fe9681ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe960070af0 0x7fe9600cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.330+0000 7fe9681ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9600cf640 con 0x7fe9600cc410 2023-07-13T22:21:42.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.330+0000 7fe9681ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9600ccf80 con 0x7fe960070af0 2023-07-13T22:21:42.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.330+0000 7fe9681ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600c9b40 con 0x7fe960159260 2023-07-13T22:21:42.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.330+0000 7fe96679c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe960070af0 0x7fe9600cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.330+0000 7fe96579a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe9600cc410 0x7fe9600703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39264/0 (socket says 172.21.15.60:39264) 2023-07-13T22:21:42.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.330+0000 7fe96579a700 1 -- 172.21.15.60:0/2653897595 learned_addr learned my addr 172.21.15.60:0/2653897595 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:42.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.330+0000 7fe965f9b700 1 --2- 172.21.15.60:0/2653897595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe960159260 0x7fe9600ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe96679c700 1 -- 172.21.15.60:0/2653897595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe960159260 msgr2=0x7fe9600ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:42.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe96679c700 1 --2- 172.21.15.60:0/2653897595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe960159260 0x7fe9600ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe96679c700 1 -- 172.21.15.60:0/2653897595 >> v1:172.21.15.60:6789/0 conn(0x7fe9600cc410 legacy=0x7fe9600703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe96679c700 1 -- 172.21.15.60:0/2653897595 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600c9e20 con 0x7fe960070af0 2023-07-13T22:21:42.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe964f99700 1 -- 172.21.15.60:0/2653897595 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612126206 0 0) 0x7fe9600cf640 con 0x7fe9600cc410 2023-07-13T22:21:42.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe965f9b700 1 --2- 172.21.15.60:0/2653897595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe960159260 0x7fe9600ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:42.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe96679c700 1 --2- 172.21.15.60:0/2653897595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 0x7fe9600cc880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe958009d90 tx=0x7fe95800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262adba5f6ca1a6f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:42.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe964f99700 1 -- 172.21.15.60:0/2653897595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95800bd20 con 0x7fe960070af0 2023-07-13T22:21:42.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe964f99700 1 -- 172.21.15.60:0/2653897595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95800be80 con 0x7fe960070af0 2023-07-13T22:21:42.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.331+0000 7fe964f99700 1 -- 172.21.15.60:0/2653897595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9580116a0 con 0x7fe960070af0 2023-07-13T22:21:42.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.332+0000 7fe9681ff700 1 -- 172.21.15.60:0/2653897595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 msgr2=0x7fe9600cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.332+0000 7fe9681ff700 1 --2- 172.21.15.60:0/2653897595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 0x7fe9600cc880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe958009d90 tx=0x7fe95800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:42.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.332+0000 7fe9681ff700 1 -- 172.21.15.60:0/2653897595 shutdown_connections 2023-07-13T22:21:42.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.332+0000 7fe9681ff700 1 --2- 172.21.15.60:0/2653897595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 0x7fe9600cc880 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.332+0000 7fe9681ff700 1 --2- 172.21.15.60:0/2653897595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe960159260 0x7fe9600ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.332+0000 7fe9681ff700 1 -- 172.21.15.60:0/2653897595 >> 172.21.15.60:0/2653897595 conn(0x7fe9600cbf50 msgr2=0x7fe9600ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:42.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.332+0000 7fe9681ff700 1 -- 172.21.15.60:0/2653897595 shutdown_connections 2023-07-13T22:21:42.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.332+0000 7fe9681ff700 1 -- 172.21.15.60:0/2653897595 wait complete. 2023-07-13T22:21:42.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.334+0000 7fe9681ff700 1 Processor -- start 2023-07-13T22:21:42.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.334+0000 7fe9681ff700 1 -- start start 2023-07-13T22:21:42.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe9681ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 0x7fe9600afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe9681ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9600cc410 0x7fe9600ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe9681ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe960159260 0x7fe96014b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe9681ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600abe00 con 0x7fe960070af0 2023-07-13T22:21:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe9681ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96014c460 con 0x7fe960159260 2023-07-13T22:21:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe9681ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96014bea0 con 0x7fe9600cc410 2023-07-13T22:21:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe965f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 0x7fe9600afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe965f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 0x7fe9600afc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60150/0 (socket says 172.21.15.60:60150) 2023-07-13T22:21:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe965f9b700 1 -- 172.21.15.60:0/2310226058 learned_addr learned my addr 172.21.15.60:0/2310226058 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe96679c700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe960159260 0x7fe96014b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe96579a700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9600cc410 0x7fe9600ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe965f9b700 1 -- 172.21.15.60:0/2310226058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9600cc410 msgr2=0x7fe9600ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe965f9b700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9600cc410 0x7fe9600ac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe965f9b700 1 -- 172.21.15.60:0/2310226058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe960159260 msgr2=0x7fe96014b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe965f9b700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe960159260 0x7fe96014b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.335+0000 7fe965f9b700 1 -- 172.21.15.60:0/2310226058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c0094a0 con 0x7fe960070af0 2023-07-13T22:21:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.336+0000 7fe965f9b700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 0x7fe9600afc50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00ebe0 tx=0x7fe95c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.336+0000 7fe9567fc700 1 -- 172.21.15.60:0/2310226058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00c8b0 con 0x7fe960070af0 2023-07-13T22:21:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.336+0000 7fe9567fc700 1 -- 172.21.15.60:0/2310226058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c00ca10 con 0x7fe960070af0 2023-07-13T22:21:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.336+0000 7fe9567fc700 1 -- 172.21.15.60:0/2310226058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00cda0 con 0x7fe960070af0 2023-07-13T22:21:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.336+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe958009a70 con 0x7fe960070af0 2023-07-13T22:21:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.336+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9600aba20 con 0x7fe960070af0 2023-07-13T22:21:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.337+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9600ca0e0 con 0x7fe960070af0 2023-07-13T22:21:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.338+0000 7fe9567fc700 1 -- 172.21.15.60:0/2310226058 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe95c013470 con 0x7fe960070af0 2023-07-13T22:21:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.339+0000 7fe9567fc700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94406ace0 0x7fe94406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.339+0000 7fe9567fc700 1 -- 172.21.15.60:0/2310226058 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe95c0895e0 con 0x7fe960070af0 2023-07-13T22:21:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.339+0000 7fe96579a700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94406ace0 0x7fe94406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.340+0000 7fe96579a700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94406ace0 0x7fe94406d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe960159750 tx=0x7fe950007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.343+0000 7fe9567fc700 1 -- 172.21.15.60:0/2310226058 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe9600ca0e0 con 0x7fe960070af0 2023-07-13T22:21:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.551+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9600ca0e0 con 0x7fe960070af0 2023-07-13T22:21:42.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.552+0000 7fe9567fc700 1 -- 172.21.15.60:0/2310226058 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9600ca0e0 con 0x7fe960070af0 2023-07-13T22:21:42.537 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94406ace0 msgr2=0x7fe94406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94406ace0 0x7fe94406d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe960159750 tx=0x7fe950007be0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 msgr2=0x7fe9600afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 0x7fe9600afc50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00ebe0 tx=0x7fe95c009e20 comp rx=0 tx=0).stop 2023-07-13T22:21:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 shutdown_connections 2023-07-13T22:21:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94406ace0 0x7fe94406d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe960070af0 0x7fe9600afc50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe960159260 0x7fe96014b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 --2- 172.21.15.60:0/2310226058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9600cc410 0x7fe9600ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 >> 172.21.15.60:0/2310226058 conn(0x7fe9600cbf50 msgr2=0x7fe960069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 shutdown_connections 2023-07-13T22:21:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.555+0000 7fe9681ff700 1 -- 172.21.15.60:0/2310226058 wait complete. 2023-07-13T22:21:42.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:21:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.783+0000 7fb19c9ff700 1 Processor -- start 2023-07-13T22:21:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.783+0000 7fb19c9ff700 1 -- start start 2023-07-13T22:21:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb19c9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb1980b8570 0x7fb1980ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb19c9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 0x7fb1980d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb19c9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb1980cf3f0 0x7fb1980d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb19c9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1980d3b30 con 0x7fb1980cf3f0 2023-07-13T22:21:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb19c9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1980d1930 con 0x7fb1980b8570 2023-07-13T22:21:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb19c9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1980d09d0 con 0x7fb1980d35c0 2023-07-13T22:21:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb196d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb1980cf3f0 0x7fb1980d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb196d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb1980cf3f0 0x7fb1980d13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60166/0 (socket says 172.21.15.60:60166) 2023-07-13T22:21:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb196d9d700 1 -- 172.21.15.60:0/3874635042 learned_addr learned my addr 172.21.15.60:0/3874635042 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb18ffff700 1 --2- 172.21.15.60:0/3874635042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 0x7fb1980d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb19659c700 1 --2- 172.21.15.60:0/3874635042 >> v2:172.21.15.105:3300/0 conn(0x7fb1980b8570 0x7fb1980ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb196d9d700 1 -- 172.21.15.60:0/3874635042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 msgr2=0x7fb1980d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb196d9d700 1 --2- 172.21.15.60:0/3874635042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 0x7fb1980d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb196d9d700 1 -- 172.21.15.60:0/3874635042 >> v2:172.21.15.105:3300/0 conn(0x7fb1980b8570 msgr2=0x7fb1980ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb196d9d700 1 --2- 172.21.15.60:0/3874635042 >> v2:172.21.15.105:3300/0 conn(0x7fb1980b8570 0x7fb1980ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.785+0000 7fb196d9d700 1 -- 172.21.15.60:0/3874635042 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1980cda10 con 0x7fb1980cf3f0 2023-07-13T22:21:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.786+0000 7fb196d9d700 1 --2- 172.21.15.60:0/3874635042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 0x7fb1980d13f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb19806c6c0 tx=0x7fb19000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7659a5541721b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.786+0000 7fb195d9b700 1 -- 172.21.15.60:0/3874635042 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19000e070 con 0x7fb1980cf3f0 2023-07-13T22:21:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.786+0000 7fb195d9b700 1 -- 172.21.15.60:0/3874635042 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19000b9c0 con 0x7fb1980cf3f0 2023-07-13T22:21:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.786+0000 7fb195d9b700 1 -- 172.21.15.60:0/3874635042 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19000bd70 con 0x7fb1980cf3f0 2023-07-13T22:21:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.786+0000 7fb19c9ff700 1 -- 172.21.15.60:0/3874635042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 msgr2=0x7fb1980d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.786+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/3874635042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 0x7fb1980d13f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb19806c6c0 tx=0x7fb19000b330 comp rx=0 tx=0).stop 2023-07-13T22:21:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.787+0000 7fb19c9ff700 1 -- 172.21.15.60:0/3874635042 shutdown_connections 2023-07-13T22:21:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.787+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/3874635042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 0x7fb1980d13f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.787+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/3874635042 >> v2:172.21.15.105:3300/0 conn(0x7fb1980b8570 0x7fb1980ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.787+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/3874635042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 0x7fb1980d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.787+0000 7fb19c9ff700 1 -- 172.21.15.60:0/3874635042 >> 172.21.15.60:0/3874635042 conn(0x7fb198071600 msgr2=0x7fb1980719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.787+0000 7fb19c9ff700 1 -- 172.21.15.60:0/3874635042 shutdown_connections 2023-07-13T22:21:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.787+0000 7fb19c9ff700 1 -- 172.21.15.60:0/3874635042 wait complete. 2023-07-13T22:21:42.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.788+0000 7fb19c9ff700 1 Processor -- start 2023-07-13T22:21:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.789+0000 7fb19c9ff700 1 -- start start 2023-07-13T22:21:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.789+0000 7fb19c9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1980b8570 0x7fb1980c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb19c9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 0x7fb19814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb19c9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 0x7fb19814a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb19c9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1980c9120 con 0x7fb1980cf3f0 2023-07-13T22:21:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb19c9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1980c8b60 con 0x7fb1980b8570 2023-07-13T22:21:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb19c9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1980c8e40 con 0x7fb1980d35c0 2023-07-13T22:21:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb18ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 0x7fb19814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb18ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 0x7fb19814dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60176/0 (socket says 172.21.15.60:60176) 2023-07-13T22:21:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb18ffff700 1 -- 172.21.15.60:0/2692372246 learned_addr learned my addr 172.21.15.60:0/2692372246 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb19659c700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1980b8570 0x7fb1980c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb196d9d700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 0x7fb19814a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb18ffff700 1 -- 172.21.15.60:0/2692372246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 msgr2=0x7fb19814a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb18ffff700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 0x7fb19814a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb18ffff700 1 -- 172.21.15.60:0/2692372246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1980b8570 msgr2=0x7fb1980c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:42.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb18ffff700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1980b8570 0x7fb1980c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:42.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.790+0000 7fb18ffff700 1 -- 172.21.15.60:0/2692372246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb190009b30 con 0x7fb1980cf3f0 2023-07-13T22:21:42.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.791+0000 7fb18ffff700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 0x7fb19814dff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb18000e810 tx=0x7fb18000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:42.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.791+0000 7fb18effd700 1 -- 172.21.15.60:0/2692372246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18000c070 con 0x7fb1980cf3f0 2023-07-13T22:21:42.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.791+0000 7fb18effd700 1 -- 172.21.15.60:0/2692372246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb180013610 con 0x7fb1980cf3f0 2023-07-13T22:21:42.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.791+0000 7fb18effd700 1 -- 172.21.15.60:0/2692372246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1800139c0 con 0x7fb1980cf3f0 2023-07-13T22:21:42.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.791+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1981489a0 con 0x7fb1980cf3f0 2023-07-13T22:21:42.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.791+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1980c8730 con 0x7fb1980cf3f0 2023-07-13T22:21:42.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.792+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19804e280 con 0x7fb1980cf3f0 2023-07-13T22:21:42.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.793+0000 7fb18effd700 1 -- 172.21.15.60:0/2692372246 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb180013b20 con 0x7fb1980cf3f0 2023-07-13T22:21:42.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.794+0000 7fb18effd700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb17806ace0 0x7fb17806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:42.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.794+0000 7fb18effd700 1 -- 172.21.15.60:0/2692372246 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb180089250 con 0x7fb1980cf3f0 2023-07-13T22:21:42.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.794+0000 7fb19659c700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb17806ace0 0x7fb17806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:42.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.795+0000 7fb19659c700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb17806ace0 0x7fb17806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb188006fd0 tx=0x7fb188008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:42.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:42.798+0000 7fb18effd700 1 -- 172.21.15.60:0/2692372246 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb180054230 con 0x7fb1980cf3f0 2023-07-13T22:21:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.025+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb1980cea90 con 0x7fb1980cf3f0 2023-07-13T22:21:43.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.025+0000 7fb18effd700 1 -- 172.21.15.60:0/2692372246 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb180057850 con 0x7fb1980cf3f0 2023-07-13T22:21:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.027+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb17806ace0 msgr2=0x7fb17806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.027+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb17806ace0 0x7fb17806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb188006fd0 tx=0x7fb188008040 comp rx=0 tx=0).stop 2023-07-13T22:21:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.027+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 msgr2=0x7fb19814dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.027+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 0x7fb19814dff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb18000e810 tx=0x7fb18000eb20 comp rx=0 tx=0).stop 2023-07-13T22:21:43.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.027+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 shutdown_connections 2023-07-13T22:21:43.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.027+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb17806ace0 0x7fb17806d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.027+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1980d35c0 0x7fb19814a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.027+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1980b8570 0x7fb1980c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.027+0000 7fb19c9ff700 1 --2- 172.21.15.60:0/2692372246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1980cf3f0 0x7fb19814dff0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.028+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 >> 172.21.15.60:0/2692372246 conn(0x7fb198071600 msgr2=0x7fb1980d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:43.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.028+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 shutdown_connections 2023-07-13T22:21:43.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.028+0000 7fb19c9ff700 1 -- 172.21.15.60:0/2692372246 wait complete. 2023-07-13T22:21:43.014 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:21:43.030 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:21:43.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:43.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:21:43.103 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:21:43.104 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:21:43.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:43.105 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:21:43.159 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:21:43.159 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:43.211 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:21:43.211 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:43.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:43.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.480+0000 7f744f5ff700 1 Processor -- start 2023-07-13T22:21:43.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.480+0000 7f744f5ff700 1 -- start start 2023-07-13T22:21:43.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.481+0000 7f744f5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480b8400 0x7f74480cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744f5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74480cfca0 0x7f74480d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744f5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74480d0ac0 con 0x7f74480d3750 2023-07-13T22:21:43.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744f5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74480b87d0 con 0x7f74480cfca0 2023-07-13T22:21:43.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744f5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480b4eb0 con 0x7f74480b8400 2023-07-13T22:21:43.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744cb9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f74480d3750 0x7f74480cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39270/0 (socket says 172.21.15.60:39270) 2023-07-13T22:21:43.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744cb9a700 1 -- 172.21.15.60:0/2486258165 learned_addr learned my addr 172.21.15.60:0/2486258165 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:43.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744db9c700 1 --2- 172.21.15.60:0/2486258165 >> v2:172.21.15.60:3300/0 conn(0x7f74480cfca0 0x7f74480d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744d39b700 1 --2- 172.21.15.60:0/2486258165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480b8400 0x7f74480cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744db9c700 1 -- 172.21.15.60:0/2486258165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480b8400 msgr2=0x7f74480cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:43.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744db9c700 1 --2- 172.21.15.60:0/2486258165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480b8400 0x7f74480cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744db9c700 1 -- 172.21.15.60:0/2486258165 >> v1:172.21.15.60:6789/0 conn(0x7f74480d3750 legacy=0x7f74480cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f743ffff700 1 -- 172.21.15.60:0/2486258165 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129965640 0 0) 0x7f74480d0ac0 con 0x7f74480d3750 2023-07-13T22:21:43.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.482+0000 7f744db9c700 1 -- 172.21.15.60:0/2486258165 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74480b5190 con 0x7f74480cfca0 2023-07-13T22:21:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.483+0000 7f744db9c700 1 --2- 172.21.15.60:0/2486258165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480cfca0 0x7f74480d0f50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7440009d90 tx=0x7f744000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acef990498924761 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.483+0000 7f743ffff700 1 -- 172.21.15.60:0/2486258165 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000e070 con 0x7f74480cfca0 2023-07-13T22:21:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.483+0000 7f743ffff700 1 -- 172.21.15.60:0/2486258165 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744000bbd0 con 0x7f74480cfca0 2023-07-13T22:21:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.483+0000 7f743ffff700 1 -- 172.21.15.60:0/2486258165 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440012430 con 0x7f74480cfca0 2023-07-13T22:21:43.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.483+0000 7f744f5ff700 1 -- 172.21.15.60:0/2486258165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480cfca0 msgr2=0x7f74480d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.483+0000 7f744f5ff700 1 --2- 172.21.15.60:0/2486258165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480cfca0 0x7f74480d0f50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7440009d90 tx=0x7f744000b330 comp rx=0 tx=0).stop 2023-07-13T22:21:43.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.484+0000 7f744f5ff700 1 -- 172.21.15.60:0/2486258165 shutdown_connections 2023-07-13T22:21:43.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.484+0000 7f744f5ff700 1 --2- 172.21.15.60:0/2486258165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480cfca0 0x7f74480d0f50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.484+0000 7f744f5ff700 1 --2- 172.21.15.60:0/2486258165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480b8400 0x7f74480cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.484+0000 7f744f5ff700 1 -- 172.21.15.60:0/2486258165 >> 172.21.15.60:0/2486258165 conn(0x7f744806b390 msgr2=0x7f744806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:43.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.484+0000 7f744f5ff700 1 -- 172.21.15.60:0/2486258165 shutdown_connections 2023-07-13T22:21:43.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.484+0000 7f744f5ff700 1 -- 172.21.15.60:0/2486258165 wait complete. 2023-07-13T22:21:43.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.485+0000 7f744f5ff700 1 Processor -- start 2023-07-13T22:21:43.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.486+0000 7f744f5ff700 1 -- start start 2023-07-13T22:21:43.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.486+0000 7f744f5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480b8400 0x7f74480b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.486+0000 7f744f5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74480cfca0 0x7f744815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.486+0000 7f744f5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480d3750 0x7f744814e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.486+0000 7f744f5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c66b0 con 0x7f74480b8400 2023-07-13T22:21:43.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.486+0000 7f744f5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480cbbb0 con 0x7f74480cfca0 2023-07-13T22:21:43.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.486+0000 7f744f5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c6900 con 0x7f74480d3750 2023-07-13T22:21:43.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480b8400 0x7f74480b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480b8400 0x7f74480b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60202/0 (socket says 172.21.15.60:60202) 2023-07-13T22:21:43.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744d39b700 1 -- 172.21.15.60:0/1053936126 learned_addr learned my addr 172.21.15.60:0/1053936126 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:43.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744cb9a700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74480cfca0 0x7f744815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744db9c700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480d3750 0x7f744814e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744d39b700 1 -- 172.21.15.60:0/1053936126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480d3750 msgr2=0x7f744814e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744d39b700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480d3750 0x7f744814e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744d39b700 1 -- 172.21.15.60:0/1053936126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74480cfca0 msgr2=0x7f744815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744d39b700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74480cfca0 0x7f744815cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744d39b700 1 -- 172.21.15.60:0/1053936126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7440009a70 con 0x7f74480b8400 2023-07-13T22:21:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744cb9a700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74480cfca0 0x7f744815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.487+0000 7f744d39b700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480b8400 0x7f74480b6820 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f744400d800 tx=0x7f744400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.488+0000 7f743dffb700 1 -- 172.21.15.60:0/1053936126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440125f0 con 0x7f74480b8400 2023-07-13T22:21:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.488+0000 7f743dffb700 1 -- 172.21.15.60:0/1053936126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7444012750 con 0x7f74480b8400 2023-07-13T22:21:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.488+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74480c6be0 con 0x7f74480b8400 2023-07-13T22:21:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.488+0000 7f743dffb700 1 -- 172.21.15.60:0/1053936126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444012b00 con 0x7f74480b8400 2023-07-13T22:21:43.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.488+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74480b6d60 con 0x7f74480b8400 2023-07-13T22:21:43.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.489+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74480c7230 con 0x7f74480b8400 2023-07-13T22:21:43.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.490+0000 7f743dffb700 1 -- 172.21.15.60:0/1053936126 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7444012c60 con 0x7f74480b8400 2023-07-13T22:21:43.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.491+0000 7f743dffb700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f742c06ace0 0x7f742c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.491+0000 7f743dffb700 1 -- 172.21.15.60:0/1053936126 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7444089830 con 0x7f74480b8400 2023-07-13T22:21:43.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.491+0000 7f744cb9a700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f742c06ace0 0x7f742c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.492+0000 7f744cb9a700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f742c06ace0 0x7f742c06d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f744815ca90 tx=0x7f7438007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:43.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.495+0000 7f743dffb700 1 -- 172.21.15.60:0/1053936126 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7444054890 con 0x7f74480b8400 2023-07-13T22:21:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.704+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74480b6f20 con 0x7f74480b8400 2023-07-13T22:21:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.705+0000 7f743dffb700 1 -- 172.21.15.60:0/1053936126 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f7444057eb0 con 0x7f74480b8400 2023-07-13T22:21:43.690 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:43.690 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:21:42.683944+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:43.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.707+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f742c06ace0 msgr2=0x7f742c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.707+0000 7f744f5ff700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f742c06ace0 0x7f742c06d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f744815ca90 tx=0x7f7438007ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.707+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480b8400 msgr2=0x7f74480b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.707+0000 7f744f5ff700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480b8400 0x7f74480b6820 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f744400d800 tx=0x7f744400db10 comp rx=0 tx=0).stop 2023-07-13T22:21:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.708+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 shutdown_connections 2023-07-13T22:21:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.708+0000 7f744f5ff700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f742c06ace0 0x7f742c06d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.708+0000 7f744f5ff700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74480b8400 0x7f74480b6820 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.708+0000 7f744f5ff700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74480d3750 0x7f744814e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.708+0000 7f744f5ff700 1 --2- 172.21.15.60:0/1053936126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74480cfca0 0x7f744815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.708+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 >> 172.21.15.60:0/1053936126 conn(0x7f744806b390 msgr2=0x7f74480b7810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.708+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 shutdown_connections 2023-07-13T22:21:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.708+0000 7f744f5ff700 1 -- 172.21.15.60:0/1053936126 wait complete. 2023-07-13T22:21:43.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:43.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.933+0000 7fa10fbff700 1 Processor -- start 2023-07-13T22:21:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.933+0000 7fa10fbff700 1 -- start start 2023-07-13T22:21:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10fbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 0x7fa1080cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10fbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1080b1ac0 0x7fa1080cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10fbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1080cede0 0x7fa10815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10fbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1080d1870 con 0x7fa1080b1ac0 2023-07-13T22:21:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10fbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1080cf370 con 0x7fa1080cede0 2023-07-13T22:21:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10fbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080d12b0 con 0x7fa1080b8200 2023-07-13T22:21:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10d19a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1080b1ac0 0x7fa1080cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10d19a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1080b1ac0 0x7fa1080cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60204/0 (socket says 172.21.15.60:60204) 2023-07-13T22:21:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10d19a700 1 -- 172.21.15.60:0/955417755 learned_addr learned my addr 172.21.15.60:0/955417755 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.934+0000 7fa10e19c700 1 --2- 172.21.15.60:0/955417755 >> v2:172.21.15.105:3300/0 conn(0x7fa1080cede0 0x7fa10815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10d99b700 1 --2- 172.21.15.60:0/955417755 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 0x7fa1080cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10d19a700 1 -- 172.21.15.60:0/955417755 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 msgr2=0x7fa1080cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10d19a700 1 --2- 172.21.15.60:0/955417755 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 0x7fa1080cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10d19a700 1 -- 172.21.15.60:0/955417755 >> v2:172.21.15.105:3300/0 conn(0x7fa1080cede0 msgr2=0x7fa10815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10d19a700 1 --2- 172.21.15.60:0/955417755 >> v2:172.21.15.105:3300/0 conn(0x7fa1080cede0 0x7fa10815ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10d19a700 1 -- 172.21.15.60:0/955417755 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1080d1590 con 0x7fa1080b1ac0 2023-07-13T22:21:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10d19a700 1 --2- 172.21.15.60:0/955417755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 0x7fa1080cfb80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1080b8d70 tx=0x7fa0f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c546a59a2d2d075c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10c999700 1 -- 172.21.15.60:0/955417755 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f800b870 con 0x7fa1080b1ac0 2023-07-13T22:21:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10c999700 1 -- 172.21.15.60:0/955417755 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f800b9d0 con 0x7fa1080b1ac0 2023-07-13T22:21:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.935+0000 7fa10c999700 1 -- 172.21.15.60:0/955417755 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f800bd60 con 0x7fa1080b1ac0 2023-07-13T22:21:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.936+0000 7fa10fbff700 1 -- 172.21.15.60:0/955417755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 msgr2=0x7fa1080cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.936+0000 7fa10fbff700 1 --2- 172.21.15.60:0/955417755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 0x7fa1080cfb80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1080b8d70 tx=0x7fa0f8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.936+0000 7fa10fbff700 1 -- 172.21.15.60:0/955417755 shutdown_connections 2023-07-13T22:21:43.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.936+0000 7fa10fbff700 1 --2- 172.21.15.60:0/955417755 >> v2:172.21.15.105:3300/0 conn(0x7fa1080cede0 0x7fa10815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.936+0000 7fa10fbff700 1 --2- 172.21.15.60:0/955417755 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 0x7fa1080cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.936+0000 7fa10fbff700 1 --2- 172.21.15.60:0/955417755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 0x7fa1080cfb80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.936+0000 7fa10fbff700 1 -- 172.21.15.60:0/955417755 >> 172.21.15.60:0/955417755 conn(0x7fa10806b2f0 msgr2=0x7fa10806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.936+0000 7fa10fbff700 1 -- 172.21.15.60:0/955417755 shutdown_connections 2023-07-13T22:21:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.936+0000 7fa10fbff700 1 -- 172.21.15.60:0/955417755 wait complete. 2023-07-13T22:21:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.938+0000 7fa10fbff700 1 Processor -- start 2023-07-13T22:21:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.938+0000 7fa10fbff700 1 -- start start 2023-07-13T22:21:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10fbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 0x7fa108155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10fbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 0x7fa108159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10fbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1080cede0 0x7fa1080b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10fbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1081f28e0 con 0x7fa1080b1ac0 2023-07-13T22:21:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10fbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1081f2d70 con 0x7fa1080cede0 2023-07-13T22:21:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10fbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1081f1cd0 con 0x7fa1080b8200 2023-07-13T22:21:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10d99b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 0x7fa108155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10d99b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 0x7fa108155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60214/0 (socket says 172.21.15.60:60214) 2023-07-13T22:21:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10d99b700 1 -- 172.21.15.60:0/89286144 learned_addr learned my addr 172.21.15.60:0/89286144 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10d19a700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 0x7fa108159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10e19c700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1080cede0 0x7fa1080b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10d99b700 1 -- 172.21.15.60:0/89286144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 msgr2=0x7fa108159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10d99b700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 0x7fa108159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10d99b700 1 -- 172.21.15.60:0/89286144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1080cede0 msgr2=0x7fa1080b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10d99b700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1080cede0 0x7fa1080b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.939+0000 7fa10d99b700 1 -- 172.21.15.60:0/89286144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f8009550 con 0x7fa1080b1ac0 2023-07-13T22:21:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.940+0000 7fa10d99b700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 0x7fa108155c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e810 tx=0x7fa10400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.940+0000 7fa0fe7fc700 1 -- 172.21.15.60:0/89286144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104013610 con 0x7fa1080b1ac0 2023-07-13T22:21:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.940+0000 7fa0fe7fc700 1 -- 172.21.15.60:0/89286144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa104013770 con 0x7fa1080b1ac0 2023-07-13T22:21:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.940+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1081f1fb0 con 0x7fa1080b1ac0 2023-07-13T22:21:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.940+0000 7fa0fe7fc700 1 -- 172.21.15.60:0/89286144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104013b20 con 0x7fa1080b1ac0 2023-07-13T22:21:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.940+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1081f2570 con 0x7fa1080b1ac0 2023-07-13T22:21:43.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.941+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1081f2bc0 con 0x7fa1080b1ac0 2023-07-13T22:21:43.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.942+0000 7fa0fe7fc700 1 -- 172.21.15.60:0/89286144 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa104013c80 con 0x7fa1080b1ac0 2023-07-13T22:21:43.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.943+0000 7fa0fe7fc700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ec06ace0 0x7fa0ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:43.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.943+0000 7fa0fe7fc700 1 -- 172.21.15.60:0/89286144 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa104089240 con 0x7fa1080b1ac0 2023-07-13T22:21:43.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.943+0000 7fa10d19a700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ec06ace0 0x7fa0ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:43.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.944+0000 7fa10d19a700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ec06ace0 0x7fa0ec06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:43.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:43.947+0000 7fa0fe7fc700 1 -- 172.21.15.60:0/89286144 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa104057cb0 con 0x7fa1080b1ac0 2023-07-13T22:21:44.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.151+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa10804e210 con 0x7fa1080b1ac0 2023-07-13T22:21:44.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.151+0000 7fa0fe7fc700 1 -- 172.21.15.60:0/89286144 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa104057840 con 0x7fa1080b1ac0 2023-07-13T22:21:44.136 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ec06ace0 msgr2=0x7fa0ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ec06ace0 0x7fa0ec06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f8008030 comp rx=0 tx=0).stop 2023-07-13T22:21:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 msgr2=0x7fa108155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 0x7fa108155c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e810 tx=0x7fa10400ebd0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 shutdown_connections 2023-07-13T22:21:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ec06ace0 0x7fa0ec06d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1080b1ac0 0x7fa108155c90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1080cede0 0x7fa1080b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 --2- 172.21.15.60:0/89286144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1080b8200 0x7fa108159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 >> 172.21.15.60:0/89286144 conn(0x7fa10806b2f0 msgr2=0x7fa1080b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 shutdown_connections 2023-07-13T22:21:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.154+0000 7fa10fbff700 1 -- 172.21.15.60:0/89286144 wait complete. 2023-07-13T22:21:44.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:44.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.377+0000 7f15ac7ff700 1 Processor -- start 2023-07-13T22:21:44.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.377+0000 7f15ac7ff700 1 -- start start 2023-07-13T22:21:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.378+0000 7f15ac7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15a40cfd50 0x7f15a40d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.378+0000 7f15ac7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15a40b1ac0 0x7f15a40d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15ac7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a40d1870 con 0x7f15a40b8200 2023-07-13T22:21:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15ac7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a40cf370 con 0x7f15a40b1ac0 2023-07-13T22:21:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15ac7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a40d12b0 con 0x7f15a40cfd50 2023-07-13T22:21:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15aa59b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f15a40b8200 0x7f15a40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39272/0 (socket says 172.21.15.60:39272) 2023-07-13T22:21:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15aad9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15a40b1ac0 0x7f15a40d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15aa59b700 1 -- 172.21.15.60:0/2674033928 learned_addr learned my addr 172.21.15.60:0/2674033928 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:44.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15a9d9a700 1 --2- 172.21.15.60:0/2674033928 >> v2:172.21.15.105:3300/0 conn(0x7f15a40cfd50 0x7f15a40d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15aad9c700 1 -- 172.21.15.60:0/2674033928 >> v2:172.21.15.105:3300/0 conn(0x7f15a40cfd50 msgr2=0x7f15a40d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15aad9c700 1 --2- 172.21.15.60:0/2674033928 >> v2:172.21.15.105:3300/0 conn(0x7f15a40cfd50 0x7f15a40d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15aad9c700 1 -- 172.21.15.60:0/2674033928 >> v1:172.21.15.60:6789/0 conn(0x7f15a40b8200 legacy=0x7f15a40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15aad9c700 1 -- 172.21.15.60:0/2674033928 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a40d1590 con 0x7f15a40b1ac0 2023-07-13T22:21:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.379+0000 7f15a9599700 1 -- 172.21.15.60:0/2674033928 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673060813 0 0) 0x7f15a40d1870 con 0x7f15a40b8200 2023-07-13T22:21:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.380+0000 7f15aad9c700 1 --2- 172.21.15.60:0/2674033928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 0x7f15a40d2900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f15a40d07b0 tx=0x7f159c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9b6350febf9392 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.380+0000 7f15a9599700 1 -- 172.21.15.60:0/2674033928 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c00e070 con 0x7f15a40b1ac0 2023-07-13T22:21:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.380+0000 7f15a9599700 1 -- 172.21.15.60:0/2674033928 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c00bb20 con 0x7f15a40b1ac0 2023-07-13T22:21:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.380+0000 7f15a9599700 1 -- 172.21.15.60:0/2674033928 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c012430 con 0x7f15a40b1ac0 2023-07-13T22:21:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.380+0000 7f15ac7ff700 1 -- 172.21.15.60:0/2674033928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 msgr2=0x7f15a40d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.380+0000 7f15ac7ff700 1 --2- 172.21.15.60:0/2674033928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 0x7f15a40d2900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f15a40d07b0 tx=0x7f159c00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.381+0000 7f15ac7ff700 1 -- 172.21.15.60:0/2674033928 shutdown_connections 2023-07-13T22:21:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.381+0000 7f15ac7ff700 1 --2- 172.21.15.60:0/2674033928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 0x7f15a40d2900 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.381+0000 7f15ac7ff700 1 --2- 172.21.15.60:0/2674033928 >> v2:172.21.15.105:3300/0 conn(0x7f15a40cfd50 0x7f15a40d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.381+0000 7f15ac7ff700 1 -- 172.21.15.60:0/2674033928 >> 172.21.15.60:0/2674033928 conn(0x7f15a406b2f0 msgr2=0x7f15a406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.381+0000 7f15ac7ff700 1 -- 172.21.15.60:0/2674033928 shutdown_connections 2023-07-13T22:21:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.381+0000 7f15ac7ff700 1 -- 172.21.15.60:0/2674033928 wait complete. 2023-07-13T22:21:44.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.382+0000 7f15ac7ff700 1 Processor -- start 2023-07-13T22:21:44.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.383+0000 7f15ac7ff700 1 -- start start 2023-07-13T22:21:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.383+0000 7f15ac7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 0x7f15a414c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.383+0000 7f15ac7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15a40b8200 0x7f15a40b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15ac7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40cfd50 0x7f15a41f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15ac7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40b35e0 con 0x7f15a40b1ac0 2023-07-13T22:21:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15ac7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40b3220 con 0x7f15a40cfd50 2023-07-13T22:21:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15ac7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40cbb00 con 0x7f15a40b8200 2023-07-13T22:21:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aa59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 0x7f15a414c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aa59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 0x7f15a414c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60230/0 (socket says 172.21.15.60:60230) 2023-07-13T22:21:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aa59b700 1 -- 172.21.15.60:0/1512835650 learned_addr learned my addr 172.21.15.60:0/1512835650 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15a9d9a700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15a40b8200 0x7f15a40b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aad9c700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40cfd50 0x7f15a41f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aa59b700 1 -- 172.21.15.60:0/1512835650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15a40b8200 msgr2=0x7f15a40b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aa59b700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15a40b8200 0x7f15a40b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aa59b700 1 -- 172.21.15.60:0/1512835650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40cfd50 msgr2=0x7f15a41f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aa59b700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40cfd50 0x7f15a41f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aa59b700 1 -- 172.21.15.60:0/1512835650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c009a70 con 0x7f15a40b1ac0 2023-07-13T22:21:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aad9c700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40cfd50 0x7f15a41f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.384+0000 7f15aa59b700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 0x7f15a414c160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f15a00011e0 tx=0x7f15a000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:44.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.385+0000 7f159affd700 1 -- 172.21.15.60:0/1512835650 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0004c80 con 0x7f15a40b1ac0 2023-07-13T22:21:44.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.385+0000 7f159affd700 1 -- 172.21.15.60:0/1512835650 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a0004e00 con 0x7f15a40b1ac0 2023-07-13T22:21:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.385+0000 7f159affd700 1 -- 172.21.15.60:0/1512835650 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0007310 con 0x7f15a40b1ac0 2023-07-13T22:21:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.385+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a40cbde0 con 0x7f15a40b1ac0 2023-07-13T22:21:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.385+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a40cc8f0 con 0x7f15a40b1ac0 2023-07-13T22:21:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.386+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a40cc000 con 0x7f15a40b1ac0 2023-07-13T22:21:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.387+0000 7f159affd700 1 -- 172.21.15.60:0/1512835650 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15a0007470 con 0x7f15a40b1ac0 2023-07-13T22:21:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.388+0000 7f159affd700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f158806ace0 0x7f158806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.388+0000 7f159affd700 1 -- 172.21.15.60:0/1512835650 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f15a00892c0 con 0x7f15a40b1ac0 2023-07-13T22:21:44.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.388+0000 7f15a9d9a700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f158806ace0 0x7f158806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.389+0000 7f15a9d9a700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f158806ace0 0x7f158806d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f15a40d3ac0 tx=0x7f1594008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:44.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.392+0000 7f159affd700 1 -- 172.21.15.60:0/1512835650 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f15a0054320 con 0x7f15a40b1ac0 2023-07-13T22:21:44.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.596+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15a404e210 con 0x7f15a40b1ac0 2023-07-13T22:21:44.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.597+0000 7f159affd700 1 -- 172.21.15.60:0/1512835650 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15a0057940 con 0x7f15a40b1ac0 2023-07-13T22:21:44.582 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:44.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f158806ace0 msgr2=0x7f158806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f158806ace0 0x7f158806d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f15a40d3ac0 tx=0x7f1594008040 comp rx=0 tx=0).stop 2023-07-13T22:21:44.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 msgr2=0x7f15a414c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 0x7f15a414c160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f15a00011e0 tx=0x7f15a000cae0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 shutdown_connections 2023-07-13T22:21:44.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f158806ace0 0x7f158806d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a40b1ac0 0x7f15a414c160 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40cfd50 0x7f15a41f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 --2- 172.21.15.60:0/1512835650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15a40b8200 0x7f15a40b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 >> 172.21.15.60:0/1512835650 conn(0x7f15a406b2f0 msgr2=0x7f15a40b7360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:44.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 shutdown_connections 2023-07-13T22:21:44.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.599+0000 7f15ac7ff700 1 -- 172.21.15.60:0/1512835650 wait complete. 2023-07-13T22:21:44.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:44.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.823+0000 7f52c51ff700 1 Processor -- start 2023-07-13T22:21:44.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.823+0000 7f52c51ff700 1 -- start start 2023-07-13T22:21:44.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52c51ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00cfd50 0x7f52c00d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52c51ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f52c00b1ac0 0x7f52c00d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52c51ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00d1870 con 0x7f52c00b8200 2023-07-13T22:21:44.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52c51ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c00cf370 con 0x7f52c00b1ac0 2023-07-13T22:21:44.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52c51ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00d12b0 con 0x7f52c00cfd50 2023-07-13T22:21:44.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52bed9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f52c00b8200 0x7f52c00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39280/0 (socket says 172.21.15.60:39280) 2023-07-13T22:21:44.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52bed9d700 1 -- 172.21.15.60:0/975472913 learned_addr learned my addr 172.21.15.60:0/975472913 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52b7fff700 1 --2- 172.21.15.60:0/975472913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00cfd50 0x7f52c00d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52bf59e700 1 --2- 172.21.15.60:0/975472913 >> v2:172.21.15.105:3300/0 conn(0x7f52c00b1ac0 0x7f52c00d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52be59c700 1 -- 172.21.15.60:0/975472913 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716061852 0 0) 0x7f52c00d1870 con 0x7f52c00b8200 2023-07-13T22:21:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.825+0000 7f52be59c700 1 -- 172.21.15.60:0/975472913 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52ac0035b0 con 0x7f52c00b8200 2023-07-13T22:21:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.826+0000 7f52b7fff700 1 -- 172.21.15.60:0/975472913 >> v2:172.21.15.105:3300/0 conn(0x7f52c00b1ac0 msgr2=0x7f52c00d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.826+0000 7f52b7fff700 1 --2- 172.21.15.60:0/975472913 >> v2:172.21.15.105:3300/0 conn(0x7f52c00b1ac0 0x7f52c00d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.826+0000 7f52b7fff700 1 -- 172.21.15.60:0/975472913 >> v1:172.21.15.60:6789/0 conn(0x7f52c00b8200 legacy=0x7f52c00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.826+0000 7f52b7fff700 1 -- 172.21.15.60:0/975472913 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00d1590 con 0x7f52c00cfd50 2023-07-13T22:21:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.826+0000 7f52b7fff700 1 --2- 172.21.15.60:0/975472913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00cfd50 0x7f52c00d23c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009870 tx=0x7f52a8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25307eae9110392 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.826+0000 7f52be59c700 1 -- 172.21.15.60:0/975472913 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8011830 con 0x7f52c00cfd50 2023-07-13T22:21:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.826+0000 7f52be59c700 1 -- 172.21.15.60:0/975472913 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a8011990 con 0x7f52c00cfd50 2023-07-13T22:21:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.826+0000 7f52be59c700 1 -- 172.21.15.60:0/975472913 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8011d20 con 0x7f52c00cfd50 2023-07-13T22:21:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.827+0000 7f52c51ff700 1 -- 172.21.15.60:0/975472913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00cfd50 msgr2=0x7f52c00d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.827+0000 7f52c51ff700 1 --2- 172.21.15.60:0/975472913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00cfd50 0x7f52c00d23c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009870 tx=0x7f52a8009b80 comp rx=0 tx=0).stop 2023-07-13T22:21:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.827+0000 7f52c51ff700 1 -- 172.21.15.60:0/975472913 shutdown_connections 2023-07-13T22:21:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.827+0000 7f52c51ff700 1 --2- 172.21.15.60:0/975472913 >> v2:172.21.15.105:3300/0 conn(0x7f52c00b1ac0 0x7f52c00d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.827+0000 7f52c51ff700 1 --2- 172.21.15.60:0/975472913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00cfd50 0x7f52c00d23c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.827+0000 7f52c51ff700 1 -- 172.21.15.60:0/975472913 >> 172.21.15.60:0/975472913 conn(0x7f52c006b2f0 msgr2=0x7f52c006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.827+0000 7f52c51ff700 1 -- 172.21.15.60:0/975472913 shutdown_connections 2023-07-13T22:21:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.827+0000 7f52c51ff700 1 -- 172.21.15.60:0/975472913 wait complete. 2023-07-13T22:21:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.829+0000 7f52c51ff700 1 Processor -- start 2023-07-13T22:21:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.829+0000 7f52c51ff700 1 -- start start 2023-07-13T22:21:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.830+0000 7f52c51ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00b1ac0 0x7f52c00ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.830+0000 7f52c51ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c00b8200 0x7f52c0155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.830+0000 7f52c51ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00cfd50 0x7f52c00c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.830+0000 7f52c51ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0157010 con 0x7f52c00b8200 2023-07-13T22:21:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.830+0000 7f52c51ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00cae10 con 0x7f52c00cfd50 2023-07-13T22:21:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.830+0000 7f52c51ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0156950 con 0x7f52c00b1ac0 2023-07-13T22:21:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.830+0000 7f52b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c00b8200 0x7f52c0155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.830+0000 7f52b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c00b8200 0x7f52c0155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60246/0 (socket says 172.21.15.60:60246) 2023-07-13T22:21:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.830+0000 7f52b7fff700 1 -- 172.21.15.60:0/4170125495 learned_addr learned my addr 172.21.15.60:0/4170125495 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52bf59e700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00cfd50 0x7f52c00c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52b7fff700 1 -- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00b1ac0 msgr2=0x7f52c00ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52bed9d700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00b1ac0 0x7f52c00ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52b7fff700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00b1ac0 0x7f52c00ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52b7fff700 1 -- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00cfd50 msgr2=0x7f52c00c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52b7fff700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00cfd50 0x7f52c00c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52b7fff700 1 -- 172.21.15.60:0/4170125495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a80094a0 con 0x7f52c00b8200 2023-07-13T22:21:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52bf59e700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00cfd50 0x7f52c00c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52bed9d700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00b1ac0 0x7f52c00ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52b7fff700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c00b8200 0x7f52c0155cb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009840 tx=0x7f52a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52b77fe700 1 -- 172.21.15.60:0/4170125495 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80036a0 con 0x7f52c00b8200 2023-07-13T22:21:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52b77fe700 1 -- 172.21.15.60:0/4170125495 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a801e070 con 0x7f52c00b8200 2023-07-13T22:21:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52b77fe700 1 -- 172.21.15.60:0/4170125495 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8003a20 con 0x7f52c00b8200 2023-07-13T22:21:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c0156c30 con 0x7f52c00b8200 2023-07-13T22:21:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.831+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c00caa50 con 0x7f52c00b8200 2023-07-13T22:21:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.832+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c0157150 con 0x7f52c00b8200 2023-07-13T22:21:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.833+0000 7f52b77fe700 1 -- 172.21.15.60:0/4170125495 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52a8003b80 con 0x7f52c00b8200 2023-07-13T22:21:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.834+0000 7f52b77fe700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52a006ace0 0x7f52a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.834+0000 7f52b77fe700 1 -- 172.21.15.60:0/4170125495 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f52a808a520 con 0x7f52c00b8200 2023-07-13T22:21:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.835+0000 7f52bed9d700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52a006ace0 0x7f52a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.835+0000 7f52bed9d700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52a006ace0 0x7f52a006d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f52b00011e0 tx=0x7f52b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:44.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:44.839+0000 7f52b77fe700 1 -- 172.21.15.60:0/4170125495 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f52c0157150 con 0x7f52c00b8200 2023-07-13T22:21:45.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.041+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52c0157150 con 0x7f52c00b8200 2023-07-13T22:21:45.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.042+0000 7f52b77fe700 1 -- 172.21.15.60:0/4170125495 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52c0157150 con 0x7f52c00b8200 2023-07-13T22:21:45.027 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52a006ace0 msgr2=0x7f52a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52a006ace0 0x7f52a006d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f52b00011e0 tx=0x7f52b0009040 comp rx=0 tx=0).stop 2023-07-13T22:21:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c00b8200 msgr2=0x7f52c0155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c00b8200 0x7f52c0155cb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009840 tx=0x7f52a8004750 comp rx=0 tx=0).stop 2023-07-13T22:21:45.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 shutdown_connections 2023-07-13T22:21:45.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52a006ace0 0x7f52a006d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c00cfd50 0x7f52c00c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c00b1ac0 0x7f52c00ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 --2- 172.21.15.60:0/4170125495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c00b8200 0x7f52c0155cb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.044+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 >> 172.21.15.60:0/4170125495 conn(0x7f52c006b2f0 msgr2=0x7f52c015bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:45.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.045+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 shutdown_connections 2023-07-13T22:21:45.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.045+0000 7f52c51ff700 1 -- 172.21.15.60:0/4170125495 wait complete. 2023-07-13T22:21:45.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.268+0000 7fd2b3bff700 1 Processor -- start 2023-07-13T22:21:45.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.268+0000 7fd2b3bff700 1 -- start start 2023-07-13T22:21:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.269+0000 7fd2b3bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0d2030 0x7fd2ac0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.269+0000 7fd2b3bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac0d2510 con 0x7fd2ac0d2ae0 2023-07-13T22:21:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.269+0000 7fd2b3bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac15fbe0 con 0x7fd2ac0d35c0 2023-07-13T22:21:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.269+0000 7fd2b3bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0cd1a0 con 0x7fd2ac0d2030 2023-07-13T22:21:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.269+0000 7fd2b199b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd2ac0d2ae0 0x7fd2ac0d2eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51858/0 (socket says 172.21.15.60:51858) 2023-07-13T22:21:45.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.269+0000 7fd2b199b700 1 -- 172.21.15.60:0/3724163298 learned_addr learned my addr 172.21.15.60:0/3724163298 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b119a700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0d2030 0x7fd2ac0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637960911 0 0) 0x7fd2ac0d2510 con 0x7fd2ac0d2ae0 2023-07-13T22:21:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2980035b0 con 0x7fd2ac0d2ae0 2023-07-13T22:21:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550579619 0 0) 0x7fd2a4001490 con 0x7fd2ac0d35c0 2023-07-13T22:21:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ac0d2510 con 0x7fd2ac0d35c0 2023-07-13T22:21:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1886357254 0 0) 0x7fd2980035b0 con 0x7fd2ac0d2ae0 2023-07-13T22:21:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0d2030 msgr2=0x7fd2ac0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0d2030 0x7fd2ac0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 >> v1:172.21.15.105:6789/0 conn(0x7fd2ac0d35c0 legacy=0x7fd2ac0d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac0cd910 con 0x7fd2ac0d2ae0 2023-07-13T22:21:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fd2a8001670 con 0x7fd2ac0d2ae0 2023-07-13T22:21:45.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b119a700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0d2030 0x7fd2ac0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 >> v1:172.21.15.60:6789/0 conn(0x7fd2ac0d2ae0 legacy=0x7fd2ac0d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.270+0000 7fd2b0999700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0d2030 0x7fd298003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b0999700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd298004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b0999700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd298004d90 0x7fd298005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0cd1a0 con 0x7fd298004150 2023-07-13T22:21:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29800bba0 con 0x7fd298004d90 2023-07-13T22:21:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29800bd60 con 0x7fd2ac0d2030 2023-07-13T22:21:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b119a700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd298004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b199b700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0d2030 0x7fd298003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b219c700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd298004d90 0x7fd298005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b119a700 1 -- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0d2030 msgr2=0x7fd298003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b119a700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0d2030 0x7fd298003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b119a700 1 -- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd298004d90 msgr2=0x7fd298005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b119a700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd298004d90 0x7fd298005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b119a700 1 -- 172.21.15.60:0/3724163298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c0094a0 con 0x7fd298004150 2023-07-13T22:21:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b219c700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd298004d90 0x7fd298005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.271+0000 7fd2b119a700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd298004850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009ce0 tx=0x7fd298005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8615134a8af9081d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c004030 con 0x7fd298004150 2023-07-13T22:21:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c004190 con 0x7fd298004150 2023-07-13T22:21:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b0999700 1 -- 172.21.15.60:0/3724163298 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c004540 con 0x7fd298004150 2023-07-13T22:21:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b3bff700 1 -- 172.21.15.60:0/3724163298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 msgr2=0x7fd298004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b3bff700 1 --2- 172.21.15.60:0/3724163298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd298004850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009ce0 tx=0x7fd298005880 comp rx=0 tx=0).stop 2023-07-13T22:21:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b219c700 1 -- 172.21.15.60:0/3724163298 reap_dead start 2023-07-13T22:21:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b3bff700 1 -- 172.21.15.60:0/3724163298 shutdown_connections 2023-07-13T22:21:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b3bff700 1 -- 172.21.15.60:0/3724163298 >> 172.21.15.60:0/3724163298 conn(0x7fd2ac06b390 msgr2=0x7fd2ac06b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b3bff700 1 -- 172.21.15.60:0/3724163298 shutdown_connections 2023-07-13T22:21:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.272+0000 7fd2b3bff700 1 -- 172.21.15.60:0/3724163298 wait complete. 2023-07-13T22:21:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.274+0000 7fd2b3bff700 1 Processor -- start 2023-07-13T22:21:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.274+0000 7fd2b3bff700 1 -- start start 2023-07-13T22:21:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.275+0000 7fd2b3bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd2ac1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.275+0000 7fd2b3bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0c9270 0x7fd2ac0cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.275+0000 7fd2b3bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac0b61f0 0x7fd2ac0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.275+0000 7fd2b3bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac156720 con 0x7fd298004150 2023-07-13T22:21:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.275+0000 7fd2b3bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0cc200 con 0x7fd2ac0b61f0 2023-07-13T22:21:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.275+0000 7fd2b3bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0b6ba0 con 0x7fd2ac0c9270 2023-07-13T22:21:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.275+0000 7fd2b199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd2ac1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.275+0000 7fd2b199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd2ac1553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36292/0 (socket says 172.21.15.60:36292) 2023-07-13T22:21:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.275+0000 7fd2b199b700 1 -- 172.21.15.60:0/4199929439 learned_addr learned my addr 172.21.15.60:0/4199929439 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b219c700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac0b61f0 0x7fd2ac0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b119a700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0c9270 0x7fd2ac0cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b199b700 1 -- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0c9270 msgr2=0x7fd2ac0cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b199b700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0c9270 0x7fd2ac0cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b199b700 1 -- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac0b61f0 msgr2=0x7fd2ac0b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b199b700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac0b61f0 0x7fd2ac0b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b199b700 1 -- 172.21.15.60:0/4199929439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac155aa0 con 0x7fd298004150 2023-07-13T22:21:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b219c700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac0b61f0 0x7fd2ac0b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:45.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b119a700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0c9270 0x7fd2ac0cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b199b700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd2ac1553c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8004f70 tx=0x7fd2a80073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2a27fc700 1 -- 172.21.15.60:0/4199929439 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8011580 con 0x7fd298004150 2023-07-13T22:21:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2a27fc700 1 -- 172.21.15.60:0/4199929439 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8011720 con 0x7fd298004150 2023-07-13T22:21:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac155d80 con 0x7fd298004150 2023-07-13T22:21:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2a27fc700 1 -- 172.21.15.60:0/4199929439 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8011ad0 con 0x7fd298004150 2023-07-13T22:21:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.276+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac156340 con 0x7fd298004150 2023-07-13T22:21:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.278+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac04e210 con 0x7fd298004150 2023-07-13T22:21:45.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.278+0000 7fd2a27fc700 1 -- 172.21.15.60:0/4199929439 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2a8011c30 con 0x7fd298004150 2023-07-13T22:21:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.279+0000 7fd2a27fc700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd29006ace0 0x7fd29006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.279+0000 7fd2a27fc700 1 -- 172.21.15.60:0/4199929439 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd2a8089450 con 0x7fd298004150 2023-07-13T22:21:45.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.280+0000 7fd2b119a700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd29006ace0 0x7fd29006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.280+0000 7fd2b119a700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd29006ace0 0x7fd29006d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac157b20 tx=0x7fd29c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:45.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.284+0000 7fd2a27fc700 1 -- 172.21.15.60:0/4199929439 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd2a80544b0 con 0x7fd298004150 2023-07-13T22:21:45.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.489+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2ac15d4c0 con 0x7fd298004150 2023-07-13T22:21:45.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.490+0000 7fd2a27fc700 1 -- 172.21.15.60:0/4199929439 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd2a8057ad0 con 0x7fd298004150 2023-07-13T22:21:45.475 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:45.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd29006ace0 msgr2=0x7fd29006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd29006ace0 0x7fd29006d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac157b20 tx=0x7fd29c008030 comp rx=0 tx=0).stop 2023-07-13T22:21:45.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 msgr2=0x7fd2ac1553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd2ac1553c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8004f70 tx=0x7fd2a80073f0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 shutdown_connections 2023-07-13T22:21:45.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd29006ace0 0x7fd29006d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd298004150 0x7fd2ac1553c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2ac0b61f0 0x7fd2ac0b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 --2- 172.21.15.60:0/4199929439 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2ac0c9270 0x7fd2ac0cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.492+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 >> 172.21.15.60:0/4199929439 conn(0x7fd2ac06b390 msgr2=0x7fd2ac15f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:45.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.493+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 shutdown_connections 2023-07-13T22:21:45.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.493+0000 7fd2b3bff700 1 -- 172.21.15.60:0/4199929439 wait complete. 2023-07-13T22:21:45.495 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:21:45.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.715+0000 7fb36cfff700 1 Processor -- start 2023-07-13T22:21:45.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.715+0000 7fb36cfff700 1 -- start start 2023-07-13T22:21:45.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.716+0000 7fb36cfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680b8400 0x7fb3680cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.716+0000 7fb36cfff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3680d0ac0 con 0x7fb3680cfca0 2023-07-13T22:21:45.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.716+0000 7fb36cfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3680b87d0 con 0x7fb3680d3750 2023-07-13T22:21:45.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.716+0000 7fb36cfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3680b4eb0 con 0x7fb3680b8400 2023-07-13T22:21:45.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb366d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb3680cfca0 0x7fb3680d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51874/0 (socket says 172.21.15.60:51874) 2023-07-13T22:21:45.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb366d9d700 1 -- 172.21.15.60:0/2231105766 learned_addr learned my addr 172.21.15.60:0/2231105766 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:45.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb36659c700 1 --2- 172.21.15.60:0/2231105766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680b8400 0x7fb3680cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb36559a700 1 -- 172.21.15.60:0/2231105766 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729198028 0 0) 0x7fb3680d0ac0 con 0x7fb3680cfca0 2023-07-13T22:21:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb36559a700 1 -- 172.21.15.60:0/2231105766 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3500035b0 con 0x7fb3680cfca0 2023-07-13T22:21:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb36559a700 1 -- 172.21.15.60:0/2231105766 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637040552 0 0) 0x7fb3680b87d0 con 0x7fb3680d3750 2023-07-13T22:21:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb36559a700 1 -- 172.21.15.60:0/2231105766 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3680d0ac0 con 0x7fb3680d3750 2023-07-13T22:21:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb36559a700 1 -- 172.21.15.60:0/2231105766 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3550685715 0 0) 0x7fb3500035b0 con 0x7fb3680cfca0 2023-07-13T22:21:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb36659c700 1 -- 172.21.15.60:0/2231105766 >> v1:172.21.15.105:6789/0 conn(0x7fb3680d3750 legacy=0x7fb3680cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb36659c700 1 -- 172.21.15.60:0/2231105766 >> v1:172.21.15.60:6789/0 conn(0x7fb3680cfca0 legacy=0x7fb3680d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.717+0000 7fb36659c700 1 -- 172.21.15.60:0/2231105766 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3680b5190 con 0x7fb3680b8400 2023-07-13T22:21:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.718+0000 7fb36659c700 1 --2- 172.21.15.60:0/2231105766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680b8400 0x7fb3680cd240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb358009870 tx=0x7fb358009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c53b4628d80c5c01 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:45.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.718+0000 7fb36559a700 1 -- 172.21.15.60:0/2231105766 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600019f0 con 0x7fb3680b8400 2023-07-13T22:21:45.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.718+0000 7fb36559a700 1 -- 172.21.15.60:0/2231105766 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358004030 con 0x7fb3680b8400 2023-07-13T22:21:45.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.718+0000 7fb36559a700 1 -- 172.21.15.60:0/2231105766 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3580043a0 con 0x7fb3680b8400 2023-07-13T22:21:45.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.718+0000 7fb36cfff700 1 -- 172.21.15.60:0/2231105766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680b8400 msgr2=0x7fb3680cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.718+0000 7fb36cfff700 1 --2- 172.21.15.60:0/2231105766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680b8400 0x7fb3680cd240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb358009870 tx=0x7fb358009b80 comp rx=0 tx=0).stop 2023-07-13T22:21:45.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.719+0000 7fb36cfff700 1 -- 172.21.15.60:0/2231105766 shutdown_connections 2023-07-13T22:21:45.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.719+0000 7fb36cfff700 1 --2- 172.21.15.60:0/2231105766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680b8400 0x7fb3680cd240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.719+0000 7fb36cfff700 1 -- 172.21.15.60:0/2231105766 >> 172.21.15.60:0/2231105766 conn(0x7fb36806b390 msgr2=0x7fb36806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:45.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.719+0000 7fb36cfff700 1 -- 172.21.15.60:0/2231105766 shutdown_connections 2023-07-13T22:21:45.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.719+0000 7fb36cfff700 1 -- 172.21.15.60:0/2231105766 wait complete. 2023-07-13T22:21:45.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.720+0000 7fb36cfff700 1 Processor -- start 2023-07-13T22:21:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.721+0000 7fb36cfff700 1 -- start start 2023-07-13T22:21:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.721+0000 7fb36cfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3680b8400 0x7fb3681f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.721+0000 7fb36cfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680cfca0 0x7fb3681f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.721+0000 7fb36cfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3680d3750 0x7fb3680b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.721+0000 7fb36cfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36815de80 con 0x7fb3680b8400 2023-07-13T22:21:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.721+0000 7fb36cfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3681f29c0 con 0x7fb3680d3750 2023-07-13T22:21:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.721+0000 7fb36cfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3681f2ca0 con 0x7fb3680cfca0 2023-07-13T22:21:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb36659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3680b8400 0x7fb3681f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb36659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3680b8400 0x7fb3681f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36302/0 (socket says 172.21.15.60:36302) 2023-07-13T22:21:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb36659c700 1 -- 172.21.15.60:0/3416972685 learned_addr learned my addr 172.21.15.60:0/3416972685 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb365d9b700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680cfca0 0x7fb3681f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb366d9d700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3680d3750 0x7fb3680b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb36659c700 1 -- 172.21.15.60:0/3416972685 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680cfca0 msgr2=0x7fb3681f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb36659c700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680cfca0 0x7fb3681f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb36659c700 1 -- 172.21.15.60:0/3416972685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3680d3750 msgr2=0x7fb3680b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb36659c700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3680d3750 0x7fb3680b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb36659c700 1 -- 172.21.15.60:0/3416972685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3580094a0 con 0x7fb3680b8400 2023-07-13T22:21:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.722+0000 7fb366d9d700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3680d3750 0x7fb3680b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.723+0000 7fb36659c700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3680b8400 0x7fb3681f1e80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097c0 tx=0x7fb358004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.723+0000 7fb356ffd700 1 -- 172.21.15.60:0/3416972685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358003c70 con 0x7fb3680b8400 2023-07-13T22:21:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.723+0000 7fb356ffd700 1 -- 172.21.15.60:0/3416972685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35801f070 con 0x7fb3680b8400 2023-07-13T22:21:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.723+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36815e0d0 con 0x7fb3680b8400 2023-07-13T22:21:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.723+0000 7fb356ffd700 1 -- 172.21.15.60:0/3416972685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358011630 con 0x7fb3680b8400 2023-07-13T22:21:45.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.723+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3681f3090 con 0x7fb3680b8400 2023-07-13T22:21:45.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.724+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3681f3220 con 0x7fb3680b8400 2023-07-13T22:21:45.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.725+0000 7fb356ffd700 1 -- 172.21.15.60:0/3416972685 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb358021030 con 0x7fb3680b8400 2023-07-13T22:21:45.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.726+0000 7fb356ffd700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb34806ace0 0x7fb34806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:45.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.726+0000 7fb356ffd700 1 -- 172.21.15.60:0/3416972685 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb35808a8f0 con 0x7fb3680b8400 2023-07-13T22:21:45.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.727+0000 7fb365d9b700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb34806ace0 0x7fb34806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:45.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.727+0000 7fb365d9b700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb34806ace0 0x7fb34806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0011e0 tx=0x7fb35c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:45.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.731+0000 7fb356ffd700 1 -- 172.21.15.60:0/3416972685 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb3681f3220 con 0x7fb3680b8400 2023-07-13T22:21:45.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.951+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb3681f3220 con 0x7fb3680b8400 2023-07-13T22:21:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.951+0000 7fb356ffd700 1 -- 172.21.15.60:0/3416972685 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb3681f3220 con 0x7fb3680b8400 2023-07-13T22:21:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb34806ace0 msgr2=0x7fb34806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb34806ace0 0x7fb34806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0011e0 tx=0x7fb35c009040 comp rx=0 tx=0).stop 2023-07-13T22:21:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3680b8400 msgr2=0x7fb3681f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3680b8400 0x7fb3681f1e80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097c0 tx=0x7fb358004750 comp rx=0 tx=0).stop 2023-07-13T22:21:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 shutdown_connections 2023-07-13T22:21:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb34806ace0 0x7fb34806d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3680b8400 0x7fb3681f1e80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3680d3750 0x7fb3680b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 --2- 172.21.15.60:0/3416972685 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3680cfca0 0x7fb3681f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:45.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 >> 172.21.15.60:0/3416972685 conn(0x7fb36806b390 msgr2=0x7fb3680b7770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:45.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 shutdown_connections 2023-07-13T22:21:45.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:45.954+0000 7fb36cfff700 1 -- 172.21.15.60:0/3416972685 wait complete. 2023-07-13T22:21:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:21:45.956 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:21:45.957 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:46.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:46.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.226+0000 7f97805ff700 1 Processor -- start 2023-07-13T22:21:46.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.227+0000 7f97805ff700 1 -- start start 2023-07-13T22:21:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.228+0000 7f97805ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f97780d3750 0x7f97780cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.228+0000 7f97805ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97780cd950 con 0x7f97780b8400 2023-07-13T22:21:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.228+0000 7f97805ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97780d0ac0 con 0x7f97780cfad0 2023-07-13T22:21:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.228+0000 7f97805ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97780cff50 con 0x7f97780d3750 2023-07-13T22:21:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.228+0000 7f977e39b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f97780b8400 0x7f97780cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51876/0 (socket says 172.21.15.60:51876) 2023-07-13T22:21:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.228+0000 7f977e39b700 1 -- 172.21.15.60:0/1924033568 learned_addr learned my addr 172.21.15.60:0/1924033568 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:46.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.228+0000 7f977db9a700 1 --2- 172.21.15.60:0/1924033568 >> v2:172.21.15.105:3300/0 conn(0x7f97780d3750 0x7f97780cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.228+0000 7f977d399700 1 -- 172.21.15.60:0/1924033568 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435393471 0 0) 0x7f9774001110 con 0x7f97780b8400 2023-07-13T22:21:46.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977d399700 1 -- 172.21.15.60:0/1924033568 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97640035b0 con 0x7f97780b8400 2023-07-13T22:21:46.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977d399700 1 -- 172.21.15.60:0/1924033568 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648764103 0 0) 0x7f97780d0ac0 con 0x7f97780cfad0 2023-07-13T22:21:46.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977d399700 1 -- 172.21.15.60:0/1924033568 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9774001110 con 0x7f97780cfad0 2023-07-13T22:21:46.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977db9a700 1 -- 172.21.15.60:0/1924033568 >> v1:172.21.15.105:6789/0 conn(0x7f97780cfad0 legacy=0x7f97780d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977d399700 1 -- 172.21.15.60:0/1924033568 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2205865966 0 0) 0x7f97640035b0 con 0x7f97780b8400 2023-07-13T22:21:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977db9a700 1 -- 172.21.15.60:0/1924033568 >> v1:172.21.15.60:6789/0 conn(0x7f97780b8400 legacy=0x7f97780cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977db9a700 1 -- 172.21.15.60:0/1924033568 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780b87d0 con 0x7f97780d3750 2023-07-13T22:21:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977db9a700 1 --2- 172.21.15.60:0/1924033568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 0x7f97780cf590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9768009840 tx=0x7f9768009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23b0def5d338bb5f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977d399700 1 -- 172.21.15.60:0/1924033568 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774001800 con 0x7f97780d3750 2023-07-13T22:21:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977d399700 1 -- 172.21.15.60:0/1924033568 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768004030 con 0x7f97780d3750 2023-07-13T22:21:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.229+0000 7f977d399700 1 -- 172.21.15.60:0/1924033568 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97680043a0 con 0x7f97780d3750 2023-07-13T22:21:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.230+0000 7f97805ff700 1 -- 172.21.15.60:0/1924033568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 msgr2=0x7f97780cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.230+0000 7f97805ff700 1 --2- 172.21.15.60:0/1924033568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 0x7f97780cf590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9768009840 tx=0x7f9768009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.230+0000 7f97805ff700 1 -- 172.21.15.60:0/1924033568 shutdown_connections 2023-07-13T22:21:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.230+0000 7f97805ff700 1 --2- 172.21.15.60:0/1924033568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 0x7f97780cf590 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.230+0000 7f97805ff700 1 -- 172.21.15.60:0/1924033568 >> 172.21.15.60:0/1924033568 conn(0x7f977806b390 msgr2=0x7f977806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.230+0000 7f97805ff700 1 -- 172.21.15.60:0/1924033568 shutdown_connections 2023-07-13T22:21:46.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.230+0000 7f97805ff700 1 -- 172.21.15.60:0/1924033568 wait complete. 2023-07-13T22:21:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.232+0000 7f97805ff700 1 Processor -- start 2023-07-13T22:21:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.232+0000 7f97805ff700 1 -- start start 2023-07-13T22:21:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.233+0000 7f97805ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97780b8400 0x7f977815d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.233+0000 7f97805ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97780cfad0 0x7f97781f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.233+0000 7f97805ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 0x7f977814db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.233+0000 7f97805ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977815dc40 con 0x7f97780cfad0 2023-07-13T22:21:46.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.233+0000 7f97805ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977815d960 con 0x7f97780d3750 2023-07-13T22:21:46.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.233+0000 7f97805ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977815e1b0 con 0x7f97780b8400 2023-07-13T22:21:46.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.233+0000 7f977db9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97780cfad0 0x7f97781f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.233+0000 7f977db9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97780cfad0 0x7f97781f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36308/0 (socket says 172.21.15.60:36308) 2023-07-13T22:21:46.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.233+0000 7f977db9a700 1 -- 172.21.15.60:0/3975420486 learned_addr learned my addr 172.21.15.60:0/3975420486 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:46.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977eb9c700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 0x7f977814db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977e39b700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97780b8400 0x7f977815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977db9a700 1 -- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97780b8400 msgr2=0x7f977815d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977db9a700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97780b8400 0x7f977815d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977db9a700 1 -- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 msgr2=0x7f977814db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977db9a700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 0x7f977814db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977db9a700 1 -- 172.21.15.60:0/3975420486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97680094a0 con 0x7f97780cfad0 2023-07-13T22:21:46.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977e39b700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97780b8400 0x7f977815d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:46.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977eb9c700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 0x7f977814db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:46.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f977db9a700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97780cfad0 0x7f97781f31e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f97680090b0 tx=0x7f9768004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:46.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f976effd700 1 -- 172.21.15.60:0/3975420486 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768003df0 con 0x7f97780cfad0 2023-07-13T22:21:46.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f976effd700 1 -- 172.21.15.60:0/3975420486 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976801f070 con 0x7f97780cfad0 2023-07-13T22:21:46.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f976effd700 1 -- 172.21.15.60:0/3975420486 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768011870 con 0x7f97780cfad0 2023-07-13T22:21:46.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977815e490 con 0x7f97780cfad0 2023-07-13T22:21:46.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.234+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97780c7b50 con 0x7f97780cfad0 2023-07-13T22:21:46.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.236+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977815e010 con 0x7f97780cfad0 2023-07-13T22:21:46.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.236+0000 7f976effd700 1 -- 172.21.15.60:0/3975420486 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9768021030 con 0x7f97780cfad0 2023-07-13T22:21:46.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.237+0000 7f976effd700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f975c06ace0 0x7f975c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.237+0000 7f976effd700 1 -- 172.21.15.60:0/3975420486 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f976808a800 con 0x7f97780cfad0 2023-07-13T22:21:46.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.238+0000 7f977e39b700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f975c06ace0 0x7f975c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.238+0000 7f977e39b700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f975c06ace0 0x7f975c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9774007fd0 tx=0x7f9774009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:46.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.242+0000 7f976effd700 1 -- 172.21.15.60:0/3975420486 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f977815e010 con 0x7f97780cfad0 2023-07-13T22:21:46.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.448+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f977815e9b0 con 0x7f97780cfad0 2023-07-13T22:21:46.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.449+0000 7f976effd700 1 -- 172.21.15.60:0/3975420486 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f977815e9b0 con 0x7f97780cfad0 2023-07-13T22:21:46.434 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:46.434 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:21:42.683944+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:46.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f975c06ace0 msgr2=0x7f975c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f975c06ace0 0x7f975c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9774007fd0 tx=0x7f9774009040 comp rx=0 tx=0).stop 2023-07-13T22:21:46.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97780cfad0 msgr2=0x7f97781f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97780cfad0 0x7f97781f31e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f97680090b0 tx=0x7f9768004750 comp rx=0 tx=0).stop 2023-07-13T22:21:46.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 shutdown_connections 2023-07-13T22:21:46.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f975c06ace0 0x7f975c06d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97780d3750 0x7f977814db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97780b8400 0x7f977815d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 --2- 172.21.15.60:0/3975420486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97780cfad0 0x7f97781f31e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.451+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 >> 172.21.15.60:0/3975420486 conn(0x7f977806b390 msgr2=0x7f97780b6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:46.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.452+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 shutdown_connections 2023-07-13T22:21:46.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.452+0000 7f97805ff700 1 -- 172.21.15.60:0/3975420486 wait complete. 2023-07-13T22:21:46.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:46.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.676+0000 7fa7121ff700 1 Processor -- start 2023-07-13T22:21:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.676+0000 7fa7121ff700 1 -- start start 2023-07-13T22:21:46.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.677+0000 7fa7121ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0a70 0x7fa70c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.677+0000 7fa7121ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c0d2430 con 0x7fa70c0cd280 2023-07-13T22:21:46.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.677+0000 7fa7121ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c0d2680 con 0x7fa70c0d0e40 2023-07-13T22:21:46.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.677+0000 7fa7121ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c15c920 con 0x7fa70c0d0a70 2023-07-13T22:21:46.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70affd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa70c0cd280 0x7fa70c0d3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51892/0 (socket says 172.21.15.60:51892) 2023-07-13T22:21:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70affd700 1 -- 172.21.15.60:0/630637254 learned_addr learned my addr 172.21.15.60:0/630637254 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70b7fe700 1 --2- 172.21.15.60:0/630637254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0a70 0x7fa70c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70a7fc700 1 -- 172.21.15.60:0/630637254 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137507217 0 0) 0x7fa70c0d2430 con 0x7fa70c0cd280 2023-07-13T22:21:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70a7fc700 1 -- 172.21.15.60:0/630637254 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f40035b0 con 0x7fa70c0cd280 2023-07-13T22:21:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70a7fc700 1 -- 172.21.15.60:0/630637254 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147950127 0 0) 0x7fa70c0d2680 con 0x7fa70c0d0e40 2023-07-13T22:21:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70a7fc700 1 -- 172.21.15.60:0/630637254 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa70c0d2430 con 0x7fa70c0d0e40 2023-07-13T22:21:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70b7fe700 1 -- 172.21.15.60:0/630637254 >> v1:172.21.15.105:6789/0 conn(0x7fa70c0d0e40 legacy=0x7fa70c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70b7fe700 1 -- 172.21.15.60:0/630637254 >> v1:172.21.15.60:6789/0 conn(0x7fa70c0cd280 legacy=0x7fa70c0d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.678+0000 7fa70b7fe700 1 -- 172.21.15.60:0/630637254 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c0d2960 con 0x7fa70c0d0a70 2023-07-13T22:21:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.679+0000 7fa70b7fe700 1 --2- 172.21.15.60:0/630637254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0a70 0x7fa70c0cf8c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009870 tx=0x7fa6fc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfd53798af188141 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.679+0000 7fa70a7fc700 1 -- 172.21.15.60:0/630637254 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc011830 con 0x7fa70c0d0a70 2023-07-13T22:21:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.679+0000 7fa70a7fc700 1 -- 172.21.15.60:0/630637254 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc011990 con 0x7fa70c0d0a70 2023-07-13T22:21:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.679+0000 7fa70a7fc700 1 -- 172.21.15.60:0/630637254 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc011d20 con 0x7fa70c0d0a70 2023-07-13T22:21:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.679+0000 7fa7121ff700 1 -- 172.21.15.60:0/630637254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0a70 msgr2=0x7fa70c0cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.679+0000 7fa7121ff700 1 --2- 172.21.15.60:0/630637254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0a70 0x7fa70c0cf8c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009870 tx=0x7fa6fc009b80 comp rx=0 tx=0).stop 2023-07-13T22:21:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.680+0000 7fa7121ff700 1 -- 172.21.15.60:0/630637254 shutdown_connections 2023-07-13T22:21:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.680+0000 7fa7121ff700 1 --2- 172.21.15.60:0/630637254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0a70 0x7fa70c0cf8c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.680+0000 7fa7121ff700 1 -- 172.21.15.60:0/630637254 >> 172.21.15.60:0/630637254 conn(0x7fa70c071600 msgr2=0x7fa70c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.680+0000 7fa7121ff700 1 -- 172.21.15.60:0/630637254 shutdown_connections 2023-07-13T22:21:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.680+0000 7fa7121ff700 1 -- 172.21.15.60:0/630637254 wait complete. 2023-07-13T22:21:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.681+0000 7fa7121ff700 1 Processor -- start 2023-07-13T22:21:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.681+0000 7fa7121ff700 1 -- start start 2023-07-13T22:21:46.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.682+0000 7fa7121ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa70c0cd280 0x7fa70c1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.682+0000 7fa7121ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa70c0d0a70 0x7fa70c15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.682+0000 7fa7121ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0e40 0x7fa70c1f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.682+0000 7fa7121ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0cca50 con 0x7fa70c0cd280 2023-07-13T22:21:46.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.682+0000 7fa7121ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c1f3580 con 0x7fa70c0d0a70 2023-07-13T22:21:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.682+0000 7fa7121ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c1f3860 con 0x7fa70c0d0e40 2023-07-13T22:21:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa70c0cd280 0x7fa70c1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa70c0cd280 0x7fa70c1f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36318/0 (socket says 172.21.15.60:36318) 2023-07-13T22:21:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70b7fe700 1 -- 172.21.15.60:0/3739318112 learned_addr learned my addr 172.21.15.60:0/3739318112 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70affd700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa70c0d0a70 0x7fa70c15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70bfff700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0e40 0x7fa70c1f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70b7fe700 1 -- 172.21.15.60:0/3739318112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0e40 msgr2=0x7fa70c1f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70b7fe700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0e40 0x7fa70c1f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70b7fe700 1 -- 172.21.15.60:0/3739318112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa70c0d0a70 msgr2=0x7fa70c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70b7fe700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa70c0d0a70 0x7fa70c15dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70b7fe700 1 -- 172.21.15.60:0/3739318112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0094a0 con 0x7fa70c0cd280 2023-07-13T22:21:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70affd700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa70c0d0a70 0x7fa70c15dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.683+0000 7fa70b7fe700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa70c0cd280 0x7fa70c1f4620 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0090b0 tx=0x7fa6fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.684+0000 7fa6f3fff700 1 -- 172.21.15.60:0/3739318112 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc01a860 con 0x7fa70c0cd280 2023-07-13T22:21:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.684+0000 7fa6f3fff700 1 -- 172.21.15.60:0/3739318112 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc01a9c0 con 0x7fa70c0cd280 2023-07-13T22:21:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.684+0000 7fa6f3fff700 1 -- 172.21.15.60:0/3739318112 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc01ad30 con 0x7fa70c0cd280 2023-07-13T22:21:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.684+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70c1f3b40 con 0x7fa70c0cd280 2023-07-13T22:21:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.684+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70c1f4100 con 0x7fa70c0cd280 2023-07-13T22:21:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.685+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70c067a70 con 0x7fa70c0cd280 2023-07-13T22:21:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.686+0000 7fa6f3fff700 1 -- 172.21.15.60:0/3739318112 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa6fc022070 con 0x7fa70c0cd280 2023-07-13T22:21:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.686+0000 7fa6f3fff700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f806ace0 0x7fa6f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.687+0000 7fa6f3fff700 1 -- 172.21.15.60:0/3739318112 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa6fc08a1c0 con 0x7fa70c0cd280 2023-07-13T22:21:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.687+0000 7fa70affd700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f806ace0 0x7fa6f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.687+0000 7fa70affd700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f806ace0 0x7fa6f806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0b84b0 tx=0x7fa700009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.691+0000 7fa6f3fff700 1 -- 172.21.15.60:0/3739318112 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa6fc0553f0 con 0x7fa70c0cd280 2023-07-13T22:21:46.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.900+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa70c04e210 con 0x7fa70c0cd280 2023-07-13T22:21:46.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.900+0000 7fa6f3fff700 1 -- 172.21.15.60:0/3739318112 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa6fc029020 con 0x7fa70c0cd280 2023-07-13T22:21:46.885 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.902+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f806ace0 msgr2=0x7fa6f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.902+0000 7fa7121ff700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f806ace0 0x7fa6f806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0b84b0 tx=0x7fa700009040 comp rx=0 tx=0).stop 2023-07-13T22:21:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.902+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa70c0cd280 msgr2=0x7fa70c1f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.902+0000 7fa7121ff700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa70c0cd280 0x7fa70c1f4620 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0090b0 tx=0x7fa6fc004750 comp rx=0 tx=0).stop 2023-07-13T22:21:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.903+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 shutdown_connections 2023-07-13T22:21:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.903+0000 7fa7121ff700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6f806ace0 0x7fa6f806d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.903+0000 7fa7121ff700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa70c0cd280 0x7fa70c1f4620 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.903+0000 7fa7121ff700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa70c0d0e40 0x7fa70c1f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.903+0000 7fa7121ff700 1 --2- 172.21.15.60:0/3739318112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa70c0d0a70 0x7fa70c15dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:46.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.903+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 >> 172.21.15.60:0/3739318112 conn(0x7fa70c071600 msgr2=0x7fa70c0b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:46.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.903+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 shutdown_connections 2023-07-13T22:21:46.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:46.903+0000 7fa7121ff700 1 -- 172.21.15.60:0/3739318112 wait complete. 2023-07-13T22:21:46.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:47.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.127+0000 7f5c9c1ff700 1 Processor -- start 2023-07-13T22:21:47.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.127+0000 7f5c9c1ff700 1 -- start start 2023-07-13T22:21:47.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.128+0000 7f5c9c1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c940ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.128+0000 7f5c9c1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5c940cc410 0x7f5c940cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.129+0000 7f5c9c1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c940cf640 con 0x7f5c940703e0 2023-07-13T22:21:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.129+0000 7f5c9c1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c940ccf80 con 0x7f5c940cc410 2023-07-13T22:21:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.129+0000 7f5c9c1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940c9b40 con 0x7f5c94159260 2023-07-13T22:21:47.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.129+0000 7f5c9a79c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5c940703e0 0x7f5c9406c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51906/0 (socket says 172.21.15.60:51906) 2023-07-13T22:21:47.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.129+0000 7f5c9a79c700 1 -- 172.21.15.60:0/3652178734 learned_addr learned my addr 172.21.15.60:0/3652178734 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:47.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.129+0000 7f5c9979a700 1 --2- 172.21.15.60:0/3652178734 >> v2:172.21.15.105:3300/0 conn(0x7f5c940cc410 0x7f5c940cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.129+0000 7f5c99f9b700 1 --2- 172.21.15.60:0/3652178734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c940ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.129+0000 7f5c98f99700 1 -- 172.21.15.60:0/3652178734 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129238185 0 0) 0x7f5c940cf640 con 0x7f5c940703e0 2023-07-13T22:21:47.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.129+0000 7f5c98f99700 1 -- 172.21.15.60:0/3652178734 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c800035b0 con 0x7f5c940703e0 2023-07-13T22:21:47.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.130+0000 7f5c99f9b700 1 -- 172.21.15.60:0/3652178734 >> v2:172.21.15.105:3300/0 conn(0x7f5c940cc410 msgr2=0x7f5c940cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:47.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.130+0000 7f5c99f9b700 1 --2- 172.21.15.60:0/3652178734 >> v2:172.21.15.105:3300/0 conn(0x7f5c940cc410 0x7f5c940cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.130+0000 7f5c99f9b700 1 -- 172.21.15.60:0/3652178734 >> v1:172.21.15.60:6789/0 conn(0x7f5c940703e0 legacy=0x7f5c9406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.130+0000 7f5c99f9b700 1 -- 172.21.15.60:0/3652178734 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c940c9e20 con 0x7f5c94159260 2023-07-13T22:21:47.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.130+0000 7f5c9979a700 1 --2- 172.21.15.60:0/3652178734 >> v2:172.21.15.105:3300/0 conn(0x7f5c940cc410 0x7f5c940cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:47.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.130+0000 7f5c99f9b700 1 --2- 172.21.15.60:0/3652178734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c940ca900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009840 tx=0x7f5c90009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=392b7c79c7452735 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:47.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.130+0000 7f5c98f99700 1 -- 172.21.15.60:0/3652178734 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90011830 con 0x7f5c94159260 2023-07-13T22:21:47.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.130+0000 7f5c98f99700 1 -- 172.21.15.60:0/3652178734 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c90011990 con 0x7f5c94159260 2023-07-13T22:21:47.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.130+0000 7f5c98f99700 1 -- 172.21.15.60:0/3652178734 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90011d20 con 0x7f5c94159260 2023-07-13T22:21:47.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.131+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/3652178734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 msgr2=0x7f5c940ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.131+0000 7f5c9c1ff700 1 --2- 172.21.15.60:0/3652178734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c940ca900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009840 tx=0x7f5c90009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:47.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.131+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/3652178734 shutdown_connections 2023-07-13T22:21:47.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.131+0000 7f5c9c1ff700 1 --2- 172.21.15.60:0/3652178734 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c940ca900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.131+0000 7f5c9c1ff700 1 --2- 172.21.15.60:0/3652178734 >> v2:172.21.15.105:3300/0 conn(0x7f5c940cc410 0x7f5c940cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.131+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/3652178734 >> 172.21.15.60:0/3652178734 conn(0x7f5c940cbf50 msgr2=0x7f5c940ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:47.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.131+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/3652178734 shutdown_connections 2023-07-13T22:21:47.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.131+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/3652178734 wait complete. 2023-07-13T22:21:47.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.133+0000 7f5c9c1ff700 1 Processor -- start 2023-07-13T22:21:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.134+0000 7f5c9c1ff700 1 -- start start 2023-07-13T22:21:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9c1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c940703e0 0x7f5c940b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9c1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c940cc410 0x7f5c9415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9c1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c94154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9c1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94159630 con 0x7f5c940cc410 2023-07-13T22:21:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9c1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c941f1890 con 0x7f5c940703e0 2023-07-13T22:21:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9c1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94154ab0 con 0x7f5c94159260 2023-07-13T22:21:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9979a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c940cc410 0x7f5c9415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9979a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c940cc410 0x7f5c9415c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36332/0 (socket says 172.21.15.60:36332) 2023-07-13T22:21:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9979a700 1 -- 172.21.15.60:0/4285297757 learned_addr learned my addr 172.21.15.60:0/4285297757 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c99f9b700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c940703e0 0x7f5c940b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.135+0000 7f5c9a79c700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c94154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c9979a700 1 -- 172.21.15.60:0/4285297757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 msgr2=0x7f5c94154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c9979a700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c94154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c9979a700 1 -- 172.21.15.60:0/4285297757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c940703e0 msgr2=0x7f5c940b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c9979a700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c940703e0 0x7f5c940b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c9979a700 1 -- 172.21.15.60:0/4285297757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c840094a0 con 0x7f5c940cc410 2023-07-13T22:21:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c9a79c700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c94154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c9979a700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c940cc410 0x7f5c9415c470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400ec10 tx=0x7f5c8400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c8a7fc700 1 -- 172.21.15.60:0/4285297757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8400c910 con 0x7f5c940cc410 2023-07-13T22:21:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c8a7fc700 1 -- 172.21.15.60:0/4285297757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8400ca70 con 0x7f5c940cc410 2023-07-13T22:21:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c8a7fc700 1 -- 172.21.15.60:0/4285297757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8400cde0 con 0x7f5c940cc410 2023-07-13T22:21:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c900094a0 con 0x7f5c940cc410 2023-07-13T22:21:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.136+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c941f1ae0 con 0x7f5c940cc410 2023-07-13T22:21:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.137+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9404e210 con 0x7f5c940cc410 2023-07-13T22:21:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.138+0000 7f5c8a7fc700 1 -- 172.21.15.60:0/4285297757 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c84013470 con 0x7f5c940cc410 2023-07-13T22:21:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.140+0000 7f5c8a7fc700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7806ace0 0x7f5c7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.140+0000 7f5c8a7fc700 1 -- 172.21.15.60:0/4285297757 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5c84089780 con 0x7f5c940cc410 2023-07-13T22:21:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.140+0000 7f5c99f9b700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7806ace0 0x7f5c7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.140+0000 7f5c99f9b700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7806ace0 0x7f5c7806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c90000c00 tx=0x7f5c90008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:47.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.144+0000 7f5c8a7fc700 1 -- 172.21.15.60:0/4285297757 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5c840546c0 con 0x7f5c940cc410 2023-07-13T22:21:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.349+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c940cf050 con 0x7f5c940cc410 2023-07-13T22:21:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.350+0000 7f5c8a7fc700 1 -- 172.21.15.60:0/4285297757 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c840170a0 con 0x7f5c940cc410 2023-07-13T22:21:47.335 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7806ace0 msgr2=0x7f5c7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7806ace0 0x7f5c7806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c90000c00 tx=0x7f5c90008030 comp rx=0 tx=0).stop 2023-07-13T22:21:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c940cc410 msgr2=0x7f5c9415c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c940cc410 0x7f5c9415c470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400ec10 tx=0x7f5c8400ef20 comp rx=0 tx=0).stop 2023-07-13T22:21:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 shutdown_connections 2023-07-13T22:21:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7806ace0 0x7f5c7806d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c94159260 0x7f5c94154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c940703e0 0x7f5c940b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 --2- 172.21.15.60:0/4285297757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c940cc410 0x7f5c9415c470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 >> 172.21.15.60:0/4285297757 conn(0x7f5c940cbf50 msgr2=0x7f5c940ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:47.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 shutdown_connections 2023-07-13T22:21:47.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.352+0000 7f5c9c1ff700 1 -- 172.21.15.60:0/4285297757 wait complete. 2023-07-13T22:21:47.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.576+0000 7f62b61ff700 1 Processor -- start 2023-07-13T22:21:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.577+0000 7f62b61ff700 1 -- start start 2023-07-13T22:21:47.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62b61ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00cfd50 0x7f62b00d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62b61ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62b00b1ac0 0x7f62b00d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62b61ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b00d1870 con 0x7f62b00b8200 2023-07-13T22:21:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62b61ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b00cf370 con 0x7f62b00b1ac0 2023-07-13T22:21:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62b61ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00d12b0 con 0x7f62b00cfd50 2023-07-13T22:21:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62affff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62b00b1ac0 0x7f62b00d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62affff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62b00b1ac0 0x7f62b00d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36334/0 (socket says 172.21.15.60:36334) 2023-07-13T22:21:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62affff700 1 -- 172.21.15.60:0/4290556628 learned_addr learned my addr 172.21.15.60:0/4290556628 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62aeffd700 1 --2- 172.21.15.60:0/4290556628 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00cfd50 0x7f62b00d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62affff700 1 -- 172.21.15.60:0/4290556628 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00cfd50 msgr2=0x7f62b00d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62affff700 1 --2- 172.21.15.60:0/4290556628 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00cfd50 0x7f62b00d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62affff700 1 -- 172.21.15.60:0/4290556628 >> v1:172.21.15.105:6789/0 conn(0x7f62b00b8200 legacy=0x7f62b00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.578+0000 7f62affff700 1 -- 172.21.15.60:0/4290556628 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b00d1590 con 0x7f62b00b1ac0 2023-07-13T22:21:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.579+0000 7f62affff700 1 --2- 172.21.15.60:0/4290556628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 0x7f62b00d2900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009e80 tx=0x7f62a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587007aee45226e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.579+0000 7f62ae7fc700 1 -- 172.21.15.60:0/4290556628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a800e070 con 0x7f62b00b1ac0 2023-07-13T22:21:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.579+0000 7f62ae7fc700 1 -- 172.21.15.60:0/4290556628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a800b980 con 0x7f62b00b1ac0 2023-07-13T22:21:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.579+0000 7f62ae7fc700 1 -- 172.21.15.60:0/4290556628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a800bd30 con 0x7f62b00b1ac0 2023-07-13T22:21:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.579+0000 7f62b61ff700 1 -- 172.21.15.60:0/4290556628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 msgr2=0x7f62b00d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.579+0000 7f62b61ff700 1 --2- 172.21.15.60:0/4290556628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 0x7f62b00d2900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009e80 tx=0x7f62a800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.580+0000 7f62b61ff700 1 -- 172.21.15.60:0/4290556628 shutdown_connections 2023-07-13T22:21:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.580+0000 7f62b61ff700 1 --2- 172.21.15.60:0/4290556628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 0x7f62b00d2900 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.580+0000 7f62b61ff700 1 --2- 172.21.15.60:0/4290556628 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00cfd50 0x7f62b00d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.580+0000 7f62b61ff700 1 -- 172.21.15.60:0/4290556628 >> 172.21.15.60:0/4290556628 conn(0x7f62b006b2f0 msgr2=0x7f62b006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.580+0000 7f62b61ff700 1 -- 172.21.15.60:0/4290556628 shutdown_connections 2023-07-13T22:21:47.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.580+0000 7f62b61ff700 1 -- 172.21.15.60:0/4290556628 wait complete. 2023-07-13T22:21:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.581+0000 7f62b61ff700 1 Processor -- start 2023-07-13T22:21:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.582+0000 7f62b61ff700 1 -- start start 2023-07-13T22:21:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.582+0000 7f62b61ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 0x7f62b00cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.582+0000 7f62b61ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00b8200 0x7f62b01f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.582+0000 7f62b61ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b00cfd50 0x7f62b01e8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.582+0000 7f62b61ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00ca9b0 con 0x7f62b00b1ac0 2023-07-13T22:21:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.582+0000 7f62b61ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b01f2af0 con 0x7f62b00cfd50 2023-07-13T22:21:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62b61ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00c9da0 con 0x7f62b00b8200 2023-07-13T22:21:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62af7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 0x7f62b00cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62af7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 0x7f62b00cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36338/0 (socket says 172.21.15.60:36338) 2023-07-13T22:21:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62af7fe700 1 -- 172.21.15.60:0/2686787110 learned_addr learned my addr 172.21.15.60:0/2686787110 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62aeffd700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00b8200 0x7f62b01f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62affff700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b00cfd50 0x7f62b01e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62af7fe700 1 -- 172.21.15.60:0/2686787110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00b8200 msgr2=0x7f62b01f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62af7fe700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00b8200 0x7f62b01f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62af7fe700 1 -- 172.21.15.60:0/2686787110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b00cfd50 msgr2=0x7f62b01e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62af7fe700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b00cfd50 0x7f62b01e8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62af7fe700 1 -- 172.21.15.60:0/2686787110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a8009b30 con 0x7f62b00b1ac0 2023-07-13T22:21:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.583+0000 7f62af7fe700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 0x7f62b00cb960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c4e0 tx=0x7f62a000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.584+0000 7f6297fff700 1 -- 172.21.15.60:0/2686787110 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a00052d0 con 0x7f62b00b1ac0 2023-07-13T22:21:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.584+0000 7f6297fff700 1 -- 172.21.15.60:0/2686787110 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a0005430 con 0x7f62b00b1ac0 2023-07-13T22:21:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.584+0000 7f6297fff700 1 -- 172.21.15.60:0/2686787110 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a00038c0 con 0x7f62b00b1ac0 2023-07-13T22:21:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.584+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b00ca080 con 0x7f62b00b1ac0 2023-07-13T22:21:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.584+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b00ca640 con 0x7f62b00b1ac0 2023-07-13T22:21:47.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.585+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b004e210 con 0x7f62b00b1ac0 2023-07-13T22:21:47.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.586+0000 7f6297fff700 1 -- 172.21.15.60:0/2686787110 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f62a0003a20 con 0x7f62b00b1ac0 2023-07-13T22:21:47.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.587+0000 7f6297fff700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f629c06ace0 0x7f629c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:47.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.587+0000 7f6297fff700 1 -- 172.21.15.60:0/2686787110 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f62a0088870 con 0x7f62b00b1ac0 2023-07-13T22:21:47.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.587+0000 7f62aeffd700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f629c06ace0 0x7f629c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:47.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.588+0000 7f62aeffd700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f629c06ace0 0x7f629c06d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f62b01e8b20 tx=0x7f62a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:47.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.591+0000 7f6297fff700 1 -- 172.21.15.60:0/2686787110 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f62a00538d0 con 0x7f62b00b1ac0 2023-07-13T22:21:47.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.801+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62b00b4f70 con 0x7f62b00b1ac0 2023-07-13T22:21:47.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.801+0000 7f6297fff700 1 -- 172.21.15.60:0/2686787110 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62a0056ef0 con 0x7f62b00b1ac0 2023-07-13T22:21:47.786 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:47.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.803+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f629c06ace0 msgr2=0x7f629c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.803+0000 7f62b61ff700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f629c06ace0 0x7f629c06d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f62b01e8b20 tx=0x7f62a4008040 comp rx=0 tx=0).stop 2023-07-13T22:21:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.803+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 msgr2=0x7f62b00cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.803+0000 7f62b61ff700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 0x7f62b00cb960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c4e0 tx=0x7f62a000c7f0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.804+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 shutdown_connections 2023-07-13T22:21:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.804+0000 7f62b61ff700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f629c06ace0 0x7f629c06d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.804+0000 7f62b61ff700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62b00b1ac0 0x7f62b00cb960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.804+0000 7f62b61ff700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62b00cfd50 0x7f62b01e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.804+0000 7f62b61ff700 1 --2- 172.21.15.60:0/2686787110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62b00b8200 0x7f62b01f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:47.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.804+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 >> 172.21.15.60:0/2686787110 conn(0x7f62b006b2f0 msgr2=0x7f62b00d2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:47.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.804+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 shutdown_connections 2023-07-13T22:21:47.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:47.804+0000 7f62b61ff700 1 -- 172.21.15.60:0/2686787110 wait complete. 2023-07-13T22:21:47.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:48.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.027+0000 7f986f7ff700 1 Processor -- start 2023-07-13T22:21:48.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.027+0000 7f986f7ff700 1 -- start start 2023-07-13T22:21:48.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.028+0000 7f986f7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b8200 0x7f98680cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.028+0000 7f986f7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98680cede0 0x7f98680cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.028+0000 7f986f7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98680d1870 con 0x7f98680b1ac0 2023-07-13T22:21:48.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.028+0000 7f986f7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98680cf370 con 0x7f98680cede0 2023-07-13T22:21:48.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.028+0000 7f986f7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98680d12b0 con 0x7f98680b8200 2023-07-13T22:21:48.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.029+0000 7f986dd9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98680cede0 0x7f98680cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.029+0000 7f986d59b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b8200 0x7f98680cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.029+0000 7f986dd9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98680cede0 0x7f98680cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:39198/0 (socket says 172.21.15.60:39198) 2023-07-13T22:21:48.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.029+0000 7f986dd9c700 1 -- 172.21.15.60:0/2658204728 learned_addr learned my addr 172.21.15.60:0/2658204728 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:48.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.029+0000 7f986dd9c700 1 -- 172.21.15.60:0/2658204728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b8200 msgr2=0x7f98680cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:48.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.029+0000 7f986dd9c700 1 --2- 172.21.15.60:0/2658204728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b8200 0x7f98680cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.029+0000 7f986dd9c700 1 -- 172.21.15.60:0/2658204728 >> v1:172.21.15.105:6789/0 conn(0x7f98680b1ac0 legacy=0x7f986815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.029+0000 7f986dd9c700 1 -- 172.21.15.60:0/2658204728 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680d1590 con 0x7f98680cede0 2023-07-13T22:21:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f986dd9c700 1 --2- 172.21.15.60:0/2658204728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98680cede0 0x7f98680cfb80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9860009e50 tx=0x7f986000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a74fe77442f685 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f985ffff700 1 -- 172.21.15.60:0/2658204728 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986000e070 con 0x7f98680cede0 2023-07-13T22:21:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f985ffff700 1 -- 172.21.15.60:0/2658204728 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986000b910 con 0x7f98680cede0 2023-07-13T22:21:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f985ffff700 1 -- 172.21.15.60:0/2658204728 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986000bcc0 con 0x7f98680cede0 2023-07-13T22:21:48.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f986f7ff700 1 -- 172.21.15.60:0/2658204728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98680cede0 msgr2=0x7f98680cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f986f7ff700 1 --2- 172.21.15.60:0/2658204728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98680cede0 0x7f98680cfb80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9860009e50 tx=0x7f986000b330 comp rx=0 tx=0).stop 2023-07-13T22:21:48.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f986f7ff700 1 -- 172.21.15.60:0/2658204728 shutdown_connections 2023-07-13T22:21:48.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f986f7ff700 1 --2- 172.21.15.60:0/2658204728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98680cede0 0x7f98680cfb80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f986f7ff700 1 --2- 172.21.15.60:0/2658204728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b8200 0x7f98680cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.030+0000 7f986f7ff700 1 -- 172.21.15.60:0/2658204728 >> 172.21.15.60:0/2658204728 conn(0x7f986806b2f0 msgr2=0x7f986806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:48.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.031+0000 7f986f7ff700 1 -- 172.21.15.60:0/2658204728 shutdown_connections 2023-07-13T22:21:48.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.031+0000 7f986f7ff700 1 -- 172.21.15.60:0/2658204728 wait complete. 2023-07-13T22:21:48.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.032+0000 7f986f7ff700 1 Processor -- start 2023-07-13T22:21:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.032+0000 7f986f7ff700 1 -- start start 2023-07-13T22:21:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.033+0000 7f986f7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b1ac0 0x7f98680c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.033+0000 7f986f7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98680b8200 0x7f98680b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.033+0000 7f986f7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98680cede0 0x7f98680b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.033+0000 7f986f7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868156780 con 0x7f98680b8200 2023-07-13T22:21:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.033+0000 7f986f7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98680cc1d0 con 0x7f98680cede0 2023-07-13T22:21:48.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.033+0000 7f986f7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98680cc4b0 con 0x7f98680b1ac0 2023-07-13T22:21:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986cd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98680b8200 0x7f98680b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986cd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98680b8200 0x7f98680b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36346/0 (socket says 172.21.15.60:36346) 2023-07-13T22:21:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986cd9a700 1 -- 172.21.15.60:0/1345872743 learned_addr learned my addr 172.21.15.60:0/1345872743 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:48.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986d59b700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b1ac0 0x7f98680c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986dd9c700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98680cede0 0x7f98680b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986cd9a700 1 -- 172.21.15.60:0/1345872743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b1ac0 msgr2=0x7f98680c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986cd9a700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b1ac0 0x7f98680c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986cd9a700 1 -- 172.21.15.60:0/1345872743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98680cede0 msgr2=0x7f98680b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986cd9a700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98680cede0 0x7f98680b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.034+0000 7f986cd9a700 1 -- 172.21.15.60:0/1345872743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9860009b30 con 0x7f98680b8200 2023-07-13T22:21:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.035+0000 7f986cd9a700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98680b8200 0x7f98680b6510 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f985800c4e0 tx=0x7f985800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.035+0000 7f985dffb700 1 -- 172.21.15.60:0/1345872743 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580052d0 con 0x7f98680b8200 2023-07-13T22:21:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.035+0000 7f985dffb700 1 -- 172.21.15.60:0/1345872743 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9858005430 con 0x7f98680b8200 2023-07-13T22:21:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.035+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98681469c0 con 0x7f98680b8200 2023-07-13T22:21:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.035+0000 7f985dffb700 1 -- 172.21.15.60:0/1345872743 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580038c0 con 0x7f98680b8200 2023-07-13T22:21:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.035+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98680c5e40 con 0x7f98680b8200 2023-07-13T22:21:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.036+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9868146820 con 0x7f98680b8200 2023-07-13T22:21:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.037+0000 7f985dffb700 1 -- 172.21.15.60:0/1345872743 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9858003a20 con 0x7f98680b8200 2023-07-13T22:21:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.038+0000 7f985dffb700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984c06acb0 0x7f984c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.038+0000 7f985dffb700 1 -- 172.21.15.60:0/1345872743 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9858088940 con 0x7f98680b8200 2023-07-13T22:21:48.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.038+0000 7f986d59b700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984c06acb0 0x7f984c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.039+0000 7f986d59b700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984c06acb0 0x7f984c06d160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9864009be0 tx=0x7f9864009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.042+0000 7f985dffb700 1 -- 172.21.15.60:0/1345872743 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9868146820 con 0x7f98680b8200 2023-07-13T22:21:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.248+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9868146ee0 con 0x7f98680b8200 2023-07-13T22:21:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.248+0000 7f985dffb700 1 -- 172.21.15.60:0/1345872743 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9858017020 con 0x7f98680b8200 2023-07-13T22:21:48.233 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:48.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.250+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984c06acb0 msgr2=0x7f984c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.250+0000 7f986f7ff700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984c06acb0 0x7f984c06d160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9864009be0 tx=0x7f9864009210 comp rx=0 tx=0).stop 2023-07-13T22:21:48.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.250+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98680b8200 msgr2=0x7f98680b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.250+0000 7f986f7ff700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98680b8200 0x7f98680b6510 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f985800c4e0 tx=0x7f985800c7f0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.251+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 shutdown_connections 2023-07-13T22:21:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.251+0000 7f986f7ff700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984c06acb0 0x7f984c06d160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.251+0000 7f986f7ff700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98680cede0 0x7f98680b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.251+0000 7f986f7ff700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98680b1ac0 0x7f98680c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.251+0000 7f986f7ff700 1 --2- 172.21.15.60:0/1345872743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98680b8200 0x7f98680b6510 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.251+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 >> 172.21.15.60:0/1345872743 conn(0x7f986806b2f0 msgr2=0x7f98680b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.251+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 shutdown_connections 2023-07-13T22:21:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.251+0000 7f986f7ff700 1 -- 172.21.15.60:0/1345872743 wait complete. 2023-07-13T22:21:48.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:21:48.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.474+0000 7fe5cdbff700 1 Processor -- start 2023-07-13T22:21:48.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.475+0000 7fe5cdbff700 1 -- start start 2023-07-13T22:21:48.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.476+0000 7fe5cdbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe5c80b8570 0x7fe5c80ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.476+0000 7fe5cdbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80d35c0 0x7fe5c80d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.476+0000 7fe5cdbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c80d09d0 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.476+0000 7fe5cdbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c80d3b30 con 0x7fe5c80b8570 2023-07-13T22:21:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.476+0000 7fe5cdbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c80cda10 con 0x7fe5c80d35c0 2023-07-13T22:21:48.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.476+0000 7fe5c77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe5c80b8570 0x7fe5c80ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.476+0000 7fe5c77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe5c80b8570 0x7fe5c80ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36362/0 (socket says 172.21.15.60:36362) 2023-07-13T22:21:48.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.476+0000 7fe5c77fe700 1 -- 172.21.15.60:0/798807972 learned_addr learned my addr 172.21.15.60:0/798807972 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:48.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.476+0000 7fe5c6ffd700 1 --2- 172.21.15.60:0/798807972 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80d35c0 0x7fe5c80d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.477+0000 7fe5c77fe700 1 -- 172.21.15.60:0/798807972 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80d35c0 msgr2=0x7fe5c80d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:48.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.477+0000 7fe5c77fe700 1 --2- 172.21.15.60:0/798807972 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80d35c0 0x7fe5c80d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.477+0000 7fe5c77fe700 1 -- 172.21.15.60:0/798807972 >> v1:172.21.15.105:6789/0 conn(0x7fe5c80cf3f0 legacy=0x7fe5c815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.477+0000 7fe5c77fe700 1 -- 172.21.15.60:0/798807972 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c80cd450 con 0x7fe5c80b8570 2023-07-13T22:21:48.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.477+0000 7fe5c77fe700 1 --2- 172.21.15.60:0/798807972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80b8570 0x7fe5c80ceeb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8006fd0 tx=0x7fe5b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc33424d21dccfe server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.477+0000 7fe5c67fc700 1 -- 172.21.15.60:0/798807972 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8005e30 con 0x7fe5c80b8570 2023-07-13T22:21:48.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.477+0000 7fe5c67fc700 1 -- 172.21.15.60:0/798807972 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8004030 con 0x7fe5c80b8570 2023-07-13T22:21:48.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.477+0000 7fe5c67fc700 1 -- 172.21.15.60:0/798807972 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80043e0 con 0x7fe5c80b8570 2023-07-13T22:21:48.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.478+0000 7fe5cdbff700 1 -- 172.21.15.60:0/798807972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80b8570 msgr2=0x7fe5c80ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.478+0000 7fe5cdbff700 1 --2- 172.21.15.60:0/798807972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80b8570 0x7fe5c80ceeb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8006fd0 tx=0x7fe5b8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.478+0000 7fe5cdbff700 1 -- 172.21.15.60:0/798807972 shutdown_connections 2023-07-13T22:21:48.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.478+0000 7fe5cdbff700 1 --2- 172.21.15.60:0/798807972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80b8570 0x7fe5c80ceeb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.478+0000 7fe5cdbff700 1 --2- 172.21.15.60:0/798807972 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80d35c0 0x7fe5c80d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.478+0000 7fe5cdbff700 1 -- 172.21.15.60:0/798807972 >> 172.21.15.60:0/798807972 conn(0x7fe5c8071600 msgr2=0x7fe5c80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:48.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.478+0000 7fe5cdbff700 1 -- 172.21.15.60:0/798807972 shutdown_connections 2023-07-13T22:21:48.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.478+0000 7fe5cdbff700 1 -- 172.21.15.60:0/798807972 wait complete. 2023-07-13T22:21:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.480+0000 7fe5cdbff700 1 Processor -- start 2023-07-13T22:21:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.480+0000 7fe5cdbff700 1 -- start start 2023-07-13T22:21:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5cdbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80b8570 0x7fe5c80b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5cdbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80cf3f0 0x7fe5c8147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5cdbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c80d35c0 0x7fe5c80cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5cdbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c8148190 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5cdbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c80b2d50 con 0x7fe5c80d35c0 2023-07-13T22:21:48.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5cdbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c80b3030 con 0x7fe5c80b8570 2023-07-13T22:21:48.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80cf3f0 0x7fe5c8147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80cf3f0 0x7fe5c8147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36368/0 (socket says 172.21.15.60:36368) 2023-07-13T22:21:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5c6ffd700 1 -- 172.21.15.60:0/1071525319 learned_addr learned my addr 172.21.15.60:0/1071525319 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5c77fe700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80b8570 0x7fe5c80b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.481+0000 7fe5c7fff700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c80d35c0 0x7fe5c80cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5c6ffd700 1 -- 172.21.15.60:0/1071525319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80b8570 msgr2=0x7fe5c80b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5c6ffd700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80b8570 0x7fe5c80b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5c6ffd700 1 -- 172.21.15.60:0/1071525319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c80d35c0 msgr2=0x7fe5c80cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5c6ffd700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c80d35c0 0x7fe5c80cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5c6ffd700 1 -- 172.21.15.60:0/1071525319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b8009500 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5c6ffd700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80cf3f0 0x7fe5c8147c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00e870 tx=0x7fe5bc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5affff700 1 -- 172.21.15.60:0/1071525319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc009590 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5affff700 1 -- 172.21.15.60:0/1071525319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc013680 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5affff700 1 -- 172.21.15.60:0/1071525319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0139f0 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c80b3310 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.482+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c80b38d0 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.483+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c804e280 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.484+0000 7fe5affff700 1 -- 172.21.15.60:0/1071525319 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe5bc013b50 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.485+0000 7fe5affff700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe5b406ace0 0x7fe5b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.485+0000 7fe5affff700 1 -- 172.21.15.60:0/1071525319 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe5bc0893f0 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.485+0000 7fe5c77fe700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe5b406ace0 0x7fe5b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.486+0000 7fe5c77fe700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe5b406ace0 0x7fe5b406d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8006fd0 tx=0x7fe5b8008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.490+0000 7fe5affff700 1 -- 172.21.15.60:0/1071525319 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe5bc057e50 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.716+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe5c80b7a50 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.716+0000 7fe5affff700 1 -- 172.21.15.60:0/1071525319 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe5bc0579e0 con 0x7fe5c80cf3f0 2023-07-13T22:21:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe5b406ace0 msgr2=0x7fe5b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe5b406ace0 0x7fe5b406d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8006fd0 tx=0x7fe5b8008070 comp rx=0 tx=0).stop 2023-07-13T22:21:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80cf3f0 msgr2=0x7fe5c8147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80cf3f0 0x7fe5c8147c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00e870 tx=0x7fe5bc00eb80 comp rx=0 tx=0).stop 2023-07-13T22:21:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 shutdown_connections 2023-07-13T22:21:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe5b406ace0 0x7fe5b406d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c80d35c0 0x7fe5c80cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5c80b8570 0x7fe5c80b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 --2- 172.21.15.60:0/1071525319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5c80cf3f0 0x7fe5c8147c50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 >> 172.21.15.60:0/1071525319 conn(0x7fe5c8071600 msgr2=0x7fe5c80d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 shutdown_connections 2023-07-13T22:21:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:48.719+0000 7fe5cdbff700 1 -- 172.21.15.60:0/1071525319 wait complete. 2023-07-13T22:21:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:21:48.721 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:21:48.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:48.722 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:21:48.800 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:21:48.801 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:21:48.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:48.802 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:21:48.864 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:21:48.865 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:48.911 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:21:48.912 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:48.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:49.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.255+0000 7f64941ff700 1 Processor -- start 2023-07-13T22:21:49.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.255+0000 7f64941ff700 1 -- start start 2023-07-13T22:21:49.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.256+0000 7f64941ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f648c0b2010 0x7f648c0cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.256+0000 7f64941ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b5a00 0x7f648c15fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f64941ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648c0d14a0 con 0x7f648c0d26d0 2023-07-13T22:21:49.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f64941ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f648c0d2570 con 0x7f648c0b2010 2023-07-13T22:21:49.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f64941ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0d1780 con 0x7f648c0b5a00 2023-07-13T22:21:49.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f6491f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f648c0b2010 0x7f648c0cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f6491f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f648c0b2010 0x7f648c0cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36376/0 (socket says 172.21.15.60:36376) 2023-07-13T22:21:49.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f6491f9b700 1 -- 172.21.15.60:0/540292584 learned_addr learned my addr 172.21.15.60:0/540292584 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:49.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f649179a700 1 --1- 172.21.15.60:0/540292584 >> v1:172.21.15.60:6789/0 conn(0x7f648c0d26d0 0x7f648c0cf6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51922/0 (socket says 172.21.15.60:51922) 2023-07-13T22:21:49.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f649279c700 1 --2- 172.21.15.60:0/540292584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b5a00 0x7f648c15fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f6491f9b700 1 -- 172.21.15.60:0/540292584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b5a00 msgr2=0x7f648c15fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:49.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f6491f9b700 1 --2- 172.21.15.60:0/540292584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b5a00 0x7f648c15fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f6491f9b700 1 -- 172.21.15.60:0/540292584 >> v1:172.21.15.60:6789/0 conn(0x7f648c0d26d0 legacy=0x7f648c0cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.257+0000 7f6491f9b700 1 -- 172.21.15.60:0/540292584 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c0cd2a0 con 0x7f648c0b2010 2023-07-13T22:21:49.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f6491f9b700 1 --2- 172.21.15.60:0/540292584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b2010 0x7f648c0cf1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f64880098f0 tx=0x7f6488009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8089e361b762792 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:49.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f6490f99700 1 -- 172.21.15.60:0/540292584 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648800b8a0 con 0x7f648c0b2010 2023-07-13T22:21:49.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f6490f99700 1 -- 172.21.15.60:0/540292584 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648800ba00 con 0x7f648c0b2010 2023-07-13T22:21:49.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f6490f99700 1 -- 172.21.15.60:0/540292584 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648800bd90 con 0x7f648c0b2010 2023-07-13T22:21:49.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f64941ff700 1 -- 172.21.15.60:0/540292584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b2010 msgr2=0x7f648c0cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f64941ff700 1 --2- 172.21.15.60:0/540292584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b2010 0x7f648c0cf1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f64880098f0 tx=0x7f6488009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:49.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f64941ff700 1 -- 172.21.15.60:0/540292584 shutdown_connections 2023-07-13T22:21:49.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f64941ff700 1 --2- 172.21.15.60:0/540292584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b5a00 0x7f648c15fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f64941ff700 1 --2- 172.21.15.60:0/540292584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b2010 0x7f648c0cf1a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.258+0000 7f64941ff700 1 -- 172.21.15.60:0/540292584 >> 172.21.15.60:0/540292584 conn(0x7f648c06b380 msgr2=0x7f648c06b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:49.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.259+0000 7f64941ff700 1 -- 172.21.15.60:0/540292584 shutdown_connections 2023-07-13T22:21:49.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.259+0000 7f64941ff700 1 -- 172.21.15.60:0/540292584 wait complete. 2023-07-13T22:21:49.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.260+0000 7f64941ff700 1 Processor -- start 2023-07-13T22:21:49.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.260+0000 7f64941ff700 1 -- start start 2023-07-13T22:21:49.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.261+0000 7f64941ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b2010 0x7f648c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.261+0000 7f64941ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b5a00 0x7f648c1e6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.261+0000 7f64941ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f648c0d26d0 0x7f648c15ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.261+0000 7f64941ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c1e5d00 con 0x7f648c0b5a00 2023-07-13T22:21:49.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.261+0000 7f64941ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c15b140 con 0x7f648c0d26d0 2023-07-13T22:21:49.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.261+0000 7f64941ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c15f3d0 con 0x7f648c0b2010 2023-07-13T22:21:49.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b5a00 0x7f648c1e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b5a00 0x7f648c1e6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36378/0 (socket says 172.21.15.60:36378) 2023-07-13T22:21:49.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649179a700 1 -- 172.21.15.60:0/3128569789 learned_addr learned my addr 172.21.15.60:0/3128569789 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:49.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f6491f9b700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b2010 0x7f648c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649279c700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f648c0d26d0 0x7f648c15ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649179a700 1 -- 172.21.15.60:0/3128569789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b2010 msgr2=0x7f648c0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649179a700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b2010 0x7f648c0c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649179a700 1 -- 172.21.15.60:0/3128569789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f648c0d26d0 msgr2=0x7f648c15ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649179a700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f648c0d26d0 0x7f648c15ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649179a700 1 -- 172.21.15.60:0/3128569789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6488009550 con 0x7f648c0b5a00 2023-07-13T22:21:49.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.262+0000 7f649179a700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b5a00 0x7f648c1e6120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c4f0 tx=0x7f647c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:49.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.263+0000 7f64827fc700 1 -- 172.21.15.60:0/3128569789 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c005cb0 con 0x7f648c0b5a00 2023-07-13T22:21:49.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.263+0000 7f64827fc700 1 -- 172.21.15.60:0/3128569789 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c005e10 con 0x7f648c0b5a00 2023-07-13T22:21:49.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.263+0000 7f64827fc700 1 -- 172.21.15.60:0/3128569789 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c0072d0 con 0x7f648c0b5a00 2023-07-13T22:21:49.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.263+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648c1e5080 con 0x7f648c0b5a00 2023-07-13T22:21:49.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.263+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648c1e5640 con 0x7f648c0b5a00 2023-07-13T22:21:49.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.264+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648c1e5e40 con 0x7f648c0b5a00 2023-07-13T22:21:49.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.265+0000 7f64827fc700 1 -- 172.21.15.60:0/3128569789 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f647c007430 con 0x7f648c0b5a00 2023-07-13T22:21:49.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.266+0000 7f64827fc700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f647006ace0 0x7f647006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.266+0000 7f64827fc700 1 -- 172.21.15.60:0/3128569789 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f647c0887b0 con 0x7f648c0b5a00 2023-07-13T22:21:49.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.266+0000 7f6491f9b700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f647006ace0 0x7f647006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.267+0000 7f6491f9b700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f647006ace0 0x7f647006d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6488014040 tx=0x7f6488008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:49.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.270+0000 7f64827fc700 1 -- 172.21.15.60:0/3128569789 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f648c1e5e40 con 0x7f648c0b5a00 2023-07-13T22:21:49.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.477+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f648c1e5e40 con 0x7f648c0b5a00 2023-07-13T22:21:49.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.479+0000 7f64827fc700 1 -- 172.21.15.60:0/3128569789 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f648c1e5e40 con 0x7f648c0b5a00 2023-07-13T22:21:49.463 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:49.464 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:21:42.683944+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f647006ace0 msgr2=0x7f647006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f647006ace0 0x7f647006d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6488014040 tx=0x7f6488008030 comp rx=0 tx=0).stop 2023-07-13T22:21:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b5a00 msgr2=0x7f648c1e6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b5a00 0x7f648c1e6120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c4f0 tx=0x7f647c00c800 comp rx=0 tx=0).stop 2023-07-13T22:21:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 shutdown_connections 2023-07-13T22:21:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f647006ace0 0x7f647006d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f648c0d26d0 0x7f648c15ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f648c0b2010 0x7f648c0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 --2- 172.21.15.60:0/3128569789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f648c0b5a00 0x7f648c1e6120 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 >> 172.21.15.60:0/3128569789 conn(0x7f648c06b380 msgr2=0x7f648c0b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:49.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 shutdown_connections 2023-07-13T22:21:49.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.481+0000 7f64941ff700 1 -- 172.21.15.60:0/3128569789 wait complete. 2023-07-13T22:21:49.484 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:49.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.706+0000 7f3fee9ff700 1 Processor -- start 2023-07-13T22:21:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.706+0000 7f3fee9ff700 1 -- start start 2023-07-13T22:21:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.707+0000 7f3fee9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fe80b8200 0x7f3fe80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.707+0000 7f3fee9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80cede0 0x7f3fe80cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.707+0000 7f3fee9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe80d1870 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.707+0000 7f3fee9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe80cf370 con 0x7f3fe80b8200 2023-07-13T22:21:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.707+0000 7f3fee9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80d12b0 con 0x7f3fe80cede0 2023-07-13T22:21:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.707+0000 7f3fe7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fe80b8200 0x7f3fe80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.707+0000 7f3fe77fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3fe80b1ac0 0x7f3fe815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51930/0 (socket says 172.21.15.60:51930) 2023-07-13T22:21:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.707+0000 7f3fe7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fe80b8200 0x7f3fe80cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36380/0 (socket says 172.21.15.60:36380) 2023-07-13T22:21:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.707+0000 7f3fe77fe700 1 -- 172.21.15.60:0/3736035578 learned_addr learned my addr 172.21.15.60:0/3736035578 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.708+0000 7f3fecf9c700 1 --2- 172.21.15.60:0/3736035578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80cede0 0x7f3fe80cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.708+0000 7f3fe7fff700 1 -- 172.21.15.60:0/3736035578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80cede0 msgr2=0x7f3fe80cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.708+0000 7f3fe7fff700 1 --2- 172.21.15.60:0/3736035578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80cede0 0x7f3fe80cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.708+0000 7f3fe7fff700 1 -- 172.21.15.60:0/3736035578 >> v1:172.21.15.60:6789/0 conn(0x7f3fe80b1ac0 legacy=0x7f3fe815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.708+0000 7f3fe7fff700 1 -- 172.21.15.60:0/3736035578 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe80d1590 con 0x7f3fe80b8200 2023-07-13T22:21:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.708+0000 7f3fe7fff700 1 --2- 172.21.15.60:0/3736035578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b8200 0x7f3fe80cf640 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009840 tx=0x7f3fd8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749f9c8c6340e5fc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.708+0000 7f3fe6ffd700 1 -- 172.21.15.60:0/3736035578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8011830 con 0x7f3fe80b8200 2023-07-13T22:21:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.708+0000 7f3fe6ffd700 1 -- 172.21.15.60:0/3736035578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd8011990 con 0x7f3fe80b8200 2023-07-13T22:21:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.708+0000 7f3fe6ffd700 1 -- 172.21.15.60:0/3736035578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8011d20 con 0x7f3fe80b8200 2023-07-13T22:21:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.709+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3736035578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b8200 msgr2=0x7f3fe80cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.709+0000 7f3fee9ff700 1 --2- 172.21.15.60:0/3736035578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b8200 0x7f3fe80cf640 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009840 tx=0x7f3fd8009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.709+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3736035578 shutdown_connections 2023-07-13T22:21:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.709+0000 7f3fee9ff700 1 --2- 172.21.15.60:0/3736035578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80cede0 0x7f3fe80cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.709+0000 7f3fee9ff700 1 --2- 172.21.15.60:0/3736035578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b8200 0x7f3fe80cf640 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.709+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3736035578 >> 172.21.15.60:0/3736035578 conn(0x7f3fe806b2f0 msgr2=0x7f3fe806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.709+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3736035578 shutdown_connections 2023-07-13T22:21:49.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.709+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3736035578 wait complete. 2023-07-13T22:21:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.711+0000 7f3fee9ff700 1 Processor -- start 2023-07-13T22:21:49.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.712+0000 7f3fee9ff700 1 -- start start 2023-07-13T22:21:49.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fee9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b1ac0 0x7f3fe81e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fee9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80b8200 0x7f3fe80a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fee9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe80cede0 0x7f3fe81e2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fee9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81e2d90 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fee9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81e47d0 con 0x7f3fe80cede0 2023-07-13T22:21:49.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fee9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80c9ea0 con 0x7f3fe80b8200 2023-07-13T22:21:49.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b1ac0 0x7f3fe81e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b1ac0 0x7f3fe81e3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36388/0 (socket says 172.21.15.60:36388) 2023-07-13T22:21:49.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fe7fff700 1 -- 172.21.15.60:0/3392021993 learned_addr learned my addr 172.21.15.60:0/3392021993 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fecf9c700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe80cede0 0x7f3fe81e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.713+0000 7f3fe77fe700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80b8200 0x7f3fe80a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.714+0000 7f3fe7fff700 1 -- 172.21.15.60:0/3392021993 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80b8200 msgr2=0x7f3fe80a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.714+0000 7f3fe7fff700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80b8200 0x7f3fe80a9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.714+0000 7f3fe7fff700 1 -- 172.21.15.60:0/3392021993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe80cede0 msgr2=0x7f3fe81e2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.714+0000 7f3fe7fff700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe80cede0 0x7f3fe81e2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.714+0000 7f3fe7fff700 1 -- 172.21.15.60:0/3392021993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80094a0 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.714+0000 7f3fe7fff700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b1ac0 0x7f3fe81e3b60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009b90 tx=0x7f3fd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:49.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.716+0000 7f3fe4ff9700 1 -- 172.21.15.60:0/3392021993 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80036a0 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.716+0000 7f3fe4ff9700 1 -- 172.21.15.60:0/3392021993 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd801f070 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.716+0000 7f3fe4ff9700 1 -- 172.21.15.60:0/3392021993 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8003a40 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.716+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe80ca180 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.716+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe80ca740 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.717+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe80ca980 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.718+0000 7f3fe4ff9700 1 -- 172.21.15.60:0/3392021993 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3fd8003ba0 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.720+0000 7f3fe4ff9700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd406ace0 0x7f3fd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:49.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.720+0000 7f3fe4ff9700 1 -- 172.21.15.60:0/3392021993 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3fd8021070 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.720+0000 7f3fe77fe700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd406ace0 0x7f3fd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.721+0000 7f3fe77fe700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd406ace0 0x7f3fd406d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3fe81e3690 tx=0x7f3fdc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:49.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.726+0000 7f3fe4ff9700 1 -- 172.21.15.60:0/3392021993 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3fe80ca980 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.930+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3fe80ca980 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.931+0000 7f3fe4ff9700 1 -- 172.21.15.60:0/3392021993 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3fd80150a0 con 0x7f3fe80b1ac0 2023-07-13T22:21:49.916 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd406ace0 msgr2=0x7f3fd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd406ace0 0x7f3fd406d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3fe81e3690 tx=0x7f3fdc009040 comp rx=0 tx=0).stop 2023-07-13T22:21:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b1ac0 msgr2=0x7f3fe81e3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b1ac0 0x7f3fe81e3b60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009b90 tx=0x7f3fd8004750 comp rx=0 tx=0).stop 2023-07-13T22:21:49.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 shutdown_connections 2023-07-13T22:21:49.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd406ace0 0x7f3fd406d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe80b1ac0 0x7f3fe81e3b60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe80cede0 0x7f3fe81e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 --2- 172.21.15.60:0/3392021993 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fe80b8200 0x7f3fe80a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 >> 172.21.15.60:0/3392021993 conn(0x7f3fe806b2f0 msgr2=0x7f3fe80d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:49.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 shutdown_connections 2023-07-13T22:21:49.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:49.933+0000 7f3fee9ff700 1 -- 172.21.15.60:0/3392021993 wait complete. 2023-07-13T22:21:49.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.156+0000 7f9f41bff700 1 Processor -- start 2023-07-13T22:21:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.156+0000 7f9f41bff700 1 -- start start 2023-07-13T22:21:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.157+0000 7f9f41bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9f3c0b1ac0 0x7f9f3c0d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.157+0000 7f9f41bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3c0d1870 con 0x7f9f3c0b8200 2023-07-13T22:21:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.157+0000 7f9f41bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3c0cf370 con 0x7f9f3c0cfd50 2023-07-13T22:21:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.157+0000 7f9f41bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3c0d12b0 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.158+0000 7f9f3b7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9f3c0b8200 0x7f9f3c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51938/0 (socket says 172.21.15.60:51938) 2023-07-13T22:21:50.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.158+0000 7f9f3b7fe700 1 -- 172.21.15.60:0/1967047533 learned_addr learned my addr 172.21.15.60:0/1967047533 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.158+0000 7f9f3bfff700 1 --2- 172.21.15.60:0/1967047533 >> v2:172.21.15.105:3300/0 conn(0x7f9f3c0b1ac0 0x7f9f3c0d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.158+0000 7f9f3a7fc700 1 -- 172.21.15.60:0/1967047533 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757399249 0 0) 0x7f9f3c0d1870 con 0x7f9f3c0b8200 2023-07-13T22:21:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.158+0000 7f9f3a7fc700 1 -- 172.21.15.60:0/1967047533 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f240035b0 con 0x7f9f3c0b8200 2023-07-13T22:21:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.158+0000 7f9f3bfff700 1 -- 172.21.15.60:0/1967047533 >> v1:172.21.15.105:6789/0 conn(0x7f9f3c0cfd50 legacy=0x7f9f3c0d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.158+0000 7f9f3bfff700 1 -- 172.21.15.60:0/1967047533 >> v1:172.21.15.60:6789/0 conn(0x7f9f3c0b8200 legacy=0x7f9f3c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.158+0000 7f9f3bfff700 1 -- 172.21.15.60:0/1967047533 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c0d1590 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.159+0000 7f9f3bfff700 1 --2- 172.21.15.60:0/1967047533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0b1ac0 0x7f9f3c0d2ad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009d90 tx=0x7f9f3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3725b761bdbee432 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:50.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.159+0000 7f9f3a7fc700 1 -- 172.21.15.60:0/1967047533 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400e070 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.159+0000 7f9f3a7fc700 1 -- 172.21.15.60:0/1967047533 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3400bb20 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.159+0000 7f9f3a7fc700 1 -- 172.21.15.60:0/1967047533 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34012430 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.160+0000 7f9f41bff700 1 -- 172.21.15.60:0/1967047533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0b1ac0 msgr2=0x7f9f3c0d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.160+0000 7f9f41bff700 1 --2- 172.21.15.60:0/1967047533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0b1ac0 0x7f9f3c0d2ad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009d90 tx=0x7f9f3400b330 comp rx=0 tx=0).stop 2023-07-13T22:21:50.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.160+0000 7f9f41bff700 1 -- 172.21.15.60:0/1967047533 shutdown_connections 2023-07-13T22:21:50.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.160+0000 7f9f41bff700 1 --2- 172.21.15.60:0/1967047533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0b1ac0 0x7f9f3c0d2ad0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.160+0000 7f9f41bff700 1 -- 172.21.15.60:0/1967047533 >> 172.21.15.60:0/1967047533 conn(0x7f9f3c06b2f0 msgr2=0x7f9f3c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:50.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.160+0000 7f9f41bff700 1 -- 172.21.15.60:0/1967047533 shutdown_connections 2023-07-13T22:21:50.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.160+0000 7f9f41bff700 1 -- 172.21.15.60:0/1967047533 wait complete. 2023-07-13T22:21:50.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.161+0000 7f9f41bff700 1 Processor -- start 2023-07-13T22:21:50.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.162+0000 7f9f41bff700 1 -- start start 2023-07-13T22:21:50.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.162+0000 7f9f41bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3c0b1ac0 0x7f9f3c1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f41bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f3c0b8200 0x7f9f3c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f41bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0cfd50 0x7f9f3c0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f41bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c158850 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f41bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c159c80 con 0x7f9f3c0cfd50 2023-07-13T22:21:50.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f41bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c1533f0 con 0x7f9f3c0b8200 2023-07-13T22:21:50.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3c0b1ac0 0x7f9f3c1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3c0b1ac0 0x7f9f3c1529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36402/0 (socket says 172.21.15.60:36402) 2023-07-13T22:21:50.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3b7fe700 1 -- 172.21.15.60:0/787988510 learned_addr learned my addr 172.21.15.60:0/787988510 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:50.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3bfff700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0cfd50 0x7f9f3c0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3affd700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f3c0b8200 0x7f9f3c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3b7fe700 1 -- 172.21.15.60:0/787988510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f3c0b8200 msgr2=0x7f9f3c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3b7fe700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f3c0b8200 0x7f9f3c0cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3b7fe700 1 -- 172.21.15.60:0/787988510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0cfd50 msgr2=0x7f9f3c0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3b7fe700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0cfd50 0x7f9f3c0b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3b7fe700 1 -- 172.21.15.60:0/787988510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f34009a70 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.163+0000 7f9f3bfff700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0cfd50 0x7f9f3c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:50.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.164+0000 7f9f3b7fe700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3c0b1ac0 0x7f9f3c1529a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00c700 tx=0x7f9f2c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:50.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.164+0000 7f9f23fff700 1 -- 172.21.15.60:0/787988510 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c005d70 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.164+0000 7f9f23fff700 1 -- 172.21.15.60:0/787988510 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0070d0 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.164+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3c152ee0 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.164+0000 7f9f23fff700 1 -- 172.21.15.60:0/787988510 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c007460 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.164+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3c157eb0 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.165+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3c158ac0 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.166+0000 7f9f23fff700 1 -- 172.21.15.60:0/787988510 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f2c0075c0 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.167+0000 7f9f23fff700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f2806ace0 0x7f9f2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.167+0000 7f9f23fff700 1 -- 172.21.15.60:0/787988510 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9f2c088b40 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.167+0000 7f9f3affd700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f2806ace0 0x7f9f2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.168+0000 7f9f3affd700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f2806ace0 0x7f9f2806d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0cc800 tx=0x7f9f30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.171+0000 7f9f23fff700 1 -- 172.21.15.60:0/787988510 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9f2c053b20 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.377+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f3c0d39b0 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.377+0000 7f9f23fff700 1 -- 172.21.15.60:0/787988510 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f2c057140 con 0x7f9f3c0b1ac0 2023-07-13T22:21:50.377 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:50.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.379+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f2806ace0 msgr2=0x7f9f2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.379+0000 7f9f41bff700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f2806ace0 0x7f9f2806d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0cc800 tx=0x7f9f30009040 comp rx=0 tx=0).stop 2023-07-13T22:21:50.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.379+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3c0b1ac0 msgr2=0x7f9f3c1529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.379+0000 7f9f41bff700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3c0b1ac0 0x7f9f3c1529a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00c700 tx=0x7f9f2c00cac0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.379+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 shutdown_connections 2023-07-13T22:21:50.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.379+0000 7f9f41bff700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f2806ace0 0x7f9f2806d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.379+0000 7f9f41bff700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3c0b1ac0 0x7f9f3c1529a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.379+0000 7f9f41bff700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f3c0cfd50 0x7f9f3c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.379+0000 7f9f41bff700 1 --2- 172.21.15.60:0/787988510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f3c0b8200 0x7f9f3c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.380+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 >> 172.21.15.60:0/787988510 conn(0x7f9f3c06b2f0 msgr2=0x7f9f3c0d2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:50.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.380+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 shutdown_connections 2023-07-13T22:21:50.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.380+0000 7f9f41bff700 1 -- 172.21.15.60:0/787988510 wait complete. 2023-07-13T22:21:50.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.606+0000 7fb6af1ff700 1 Processor -- start 2023-07-13T22:21:50.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.606+0000 7fb6af1ff700 1 -- start start 2023-07-13T22:21:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.608+0000 7fb6af1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb6a815c790 0x7fb6a80ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.608+0000 7fb6af1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a80cf6d0 con 0x7fb6a80ac490 2023-07-13T22:21:50.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.608+0000 7fb6af1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a80ac900 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.608+0000 7fb6af1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a80acbe0 con 0x7fb6a815c790 2023-07-13T22:21:50.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.608+0000 7fb6a7fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb6a80ac490 0x7fb6a80c9e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51944/0 (socket says 172.21.15.60:51944) 2023-07-13T22:21:50.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.608+0000 7fb6a7fff700 1 -- 172.21.15.60:0/4109611124 learned_addr learned my addr 172.21.15.60:0/4109611124 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:50.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.608+0000 7fb6acf9b700 1 --2- 172.21.15.60:0/4109611124 >> v2:172.21.15.105:3300/0 conn(0x7fb6a815c790 0x7fb6a80ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.608+0000 7fb6a77fe700 1 -- 172.21.15.60:0/4109611124 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376940493 0 0) 0x7fb6a80cf6d0 con 0x7fb6a80ac490 2023-07-13T22:21:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.608+0000 7fb6a77fe700 1 -- 172.21.15.60:0/4109611124 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6980035b0 con 0x7fb6a80ac490 2023-07-13T22:21:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.609+0000 7fb6a77fe700 1 -- 172.21.15.60:0/4109611124 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874216379 0 0) 0x7fb6a80ac900 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.609+0000 7fb6a77fe700 1 -- 172.21.15.60:0/4109611124 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a80cf6d0 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.609+0000 7fb6acf9b700 1 -- 172.21.15.60:0/4109611124 >> v1:172.21.15.105:6789/0 conn(0x7fb6a80ad9a0 legacy=0x7fb6a80adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.609+0000 7fb6acf9b700 1 -- 172.21.15.60:0/4109611124 >> v1:172.21.15.60:6789/0 conn(0x7fb6a80ac490 legacy=0x7fb6a80c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.609+0000 7fb6acf9b700 1 -- 172.21.15.60:0/4109611124 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80acec0 con 0x7fb6a815c790 2023-07-13T22:21:50.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.609+0000 7fb6acf9b700 1 --2- 172.21.15.60:0/4109611124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6a815c790 0x7fb6a80ce4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009820 tx=0x7fb69c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a50a3303ed3154b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:50.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.609+0000 7fb6a77fe700 1 -- 172.21.15.60:0/4109611124 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c005e30 con 0x7fb6a815c790 2023-07-13T22:21:50.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.609+0000 7fb6a77fe700 1 -- 172.21.15.60:0/4109611124 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c006030 con 0x7fb6a815c790 2023-07-13T22:21:50.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.609+0000 7fb6a77fe700 1 -- 172.21.15.60:0/4109611124 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0063c0 con 0x7fb6a815c790 2023-07-13T22:21:50.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.610+0000 7fb6af1ff700 1 -- 172.21.15.60:0/4109611124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6a815c790 msgr2=0x7fb6a80ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.610+0000 7fb6af1ff700 1 --2- 172.21.15.60:0/4109611124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6a815c790 0x7fb6a80ce4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009820 tx=0x7fb69c009b30 comp rx=0 tx=0).stop 2023-07-13T22:21:50.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.610+0000 7fb6af1ff700 1 -- 172.21.15.60:0/4109611124 shutdown_connections 2023-07-13T22:21:50.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.610+0000 7fb6af1ff700 1 --2- 172.21.15.60:0/4109611124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6a815c790 0x7fb6a80ce4b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.610+0000 7fb6af1ff700 1 -- 172.21.15.60:0/4109611124 >> 172.21.15.60:0/4109611124 conn(0x7fb6a806b900 msgr2=0x7fb6a80cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:50.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.610+0000 7fb6af1ff700 1 -- 172.21.15.60:0/4109611124 shutdown_connections 2023-07-13T22:21:50.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.610+0000 7fb6af1ff700 1 -- 172.21.15.60:0/4109611124 wait complete. 2023-07-13T22:21:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.612+0000 7fb6af1ff700 1 Processor -- start 2023-07-13T22:21:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.612+0000 7fb6af1ff700 1 -- start start 2023-07-13T22:21:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6af1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6a80ac490 0x7fb6a81e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6af1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80ad9a0 0x7fb6a80b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6af1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6a815c790 0x7fb6a814d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6af1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81e6d90 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6af1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81e7070 con 0x7fb6a80ac490 2023-07-13T22:21:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6af1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81e7350 con 0x7fb6a815c790 2023-07-13T22:21:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6a7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80ad9a0 0x7fb6a80b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6a7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80ad9a0 0x7fb6a80b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36408/0 (socket says 172.21.15.60:36408) 2023-07-13T22:21:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6a7fff700 1 -- 172.21.15.60:0/241018137 learned_addr learned my addr 172.21.15.60:0/241018137 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:50.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6acf9b700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6a80ac490 0x7fb6a81e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6ad79c700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6a815c790 0x7fb6a814d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6a7fff700 1 -- 172.21.15.60:0/241018137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6a815c790 msgr2=0x7fb6a814d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6a7fff700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6a815c790 0x7fb6a814d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6a7fff700 1 -- 172.21.15.60:0/241018137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6a80ac490 msgr2=0x7fb6a81e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6a7fff700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6a80ac490 0x7fb6a81e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.613+0000 7fb6a7fff700 1 -- 172.21.15.60:0/241018137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c009430 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.614+0000 7fb6a7fff700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80ad9a0 0x7fb6a80b29d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c700 tx=0x7fb69400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:50.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.614+0000 7fb6a57fa700 1 -- 172.21.15.60:0/241018137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694005d70 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.614+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a81e7630 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.614+0000 7fb6a57fa700 1 -- 172.21.15.60:0/241018137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6940070d0 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.614+0000 7fb6a57fa700 1 -- 172.21.15.60:0/241018137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694007460 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.614+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a81e69f0 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.615+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a804e210 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.616+0000 7fb6a57fa700 1 -- 172.21.15.60:0/241018137 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb6940075c0 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.617+0000 7fb6a57fa700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb68c06ace0 0x7fb68c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.617+0000 7fb6a57fa700 1 -- 172.21.15.60:0/241018137 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb694088a10 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.618+0000 7fb6acf9b700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb68c06ace0 0x7fb68c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.618+0000 7fb6acf9b700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb68c06ace0 0x7fb68c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb69c013040 tx=0x7fb69c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:50.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.621+0000 7fb6a57fa700 1 -- 172.21.15.60:0/241018137 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb694053960 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.829+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb6a80c30f0 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.830+0000 7fb6a57fa700 1 -- 172.21.15.60:0/241018137 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb694017020 con 0x7fb6a80ad9a0 2023-07-13T22:21:50.815 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb68c06ace0 msgr2=0x7fb68c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb68c06ace0 0x7fb68c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb69c013040 tx=0x7fb69c008070 comp rx=0 tx=0).stop 2023-07-13T22:21:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80ad9a0 msgr2=0x7fb6a80b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80ad9a0 0x7fb6a80b29d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c700 tx=0x7fb69400ca10 comp rx=0 tx=0).stop 2023-07-13T22:21:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 shutdown_connections 2023-07-13T22:21:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb68c06ace0 0x7fb68c06d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6a815c790 0x7fb6a814d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6a80ac490 0x7fb6a81e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 --2- 172.21.15.60:0/241018137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80ad9a0 0x7fb6a80b29d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 >> 172.21.15.60:0/241018137 conn(0x7fb6a806b900 msgr2=0x7fb6a806d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 shutdown_connections 2023-07-13T22:21:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:50.832+0000 7fb6af1ff700 1 -- 172.21.15.60:0/241018137 wait complete. 2023-07-13T22:21:50.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.055+0000 7fbad83ff700 1 Processor -- start 2023-07-13T22:21:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.056+0000 7fbad83ff700 1 -- start start 2023-07-13T22:21:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.057+0000 7fbad83ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbad00b8200 0x7fbad00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.057+0000 7fbad83ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00cede0 0x7fbad00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.057+0000 7fbad83ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad00d1870 con 0x7fbad00b1ac0 2023-07-13T22:21:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.057+0000 7fbad83ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad00cf370 con 0x7fbad00b8200 2023-07-13T22:21:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.057+0000 7fbad83ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00d12b0 con 0x7fbad00cede0 2023-07-13T22:21:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.057+0000 7fbad619b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbad00b8200 0x7fbad00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.057+0000 7fbad599a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbad00b1ac0 0x7fbad015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51960/0 (socket says 172.21.15.60:51960) 2023-07-13T22:21:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.057+0000 7fbad619b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbad00b8200 0x7fbad00cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36416/0 (socket says 172.21.15.60:36416) 2023-07-13T22:21:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.057+0000 7fbad599a700 1 -- 172.21.15.60:0/2191620119 learned_addr learned my addr 172.21.15.60:0/2191620119 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad619b700 1 -- 172.21.15.60:0/2191620119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00cede0 msgr2=0x7fbad00cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad699c700 1 --2- 172.21.15.60:0/2191620119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00cede0 0x7fbad00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad5199700 1 -- 172.21.15.60:0/2191620119 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939784739 0 0) 0x7fbad00d1870 con 0x7fbad00b1ac0 2023-07-13T22:21:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad619b700 1 --2- 172.21.15.60:0/2191620119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00cede0 0x7fbad00cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad619b700 1 -- 172.21.15.60:0/2191620119 >> v1:172.21.15.60:6789/0 conn(0x7fbad00b1ac0 legacy=0x7fbad015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad619b700 1 -- 172.21.15.60:0/2191620119 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad00d1590 con 0x7fbad00b8200 2023-07-13T22:21:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad699c700 1 --2- 172.21.15.60:0/2191620119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00cede0 0x7fbad00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad619b700 1 --2- 172.21.15.60:0/2191620119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b8200 0x7fbad00cf640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009840 tx=0x7fbacc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687242b0a92bdc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad5199700 1 -- 172.21.15.60:0/2191620119 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc011830 con 0x7fbad00b8200 2023-07-13T22:21:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad5199700 1 -- 172.21.15.60:0/2191620119 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc011990 con 0x7fbad00b8200 2023-07-13T22:21:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.058+0000 7fbad5199700 1 -- 172.21.15.60:0/2191620119 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc011d20 con 0x7fbad00b8200 2023-07-13T22:21:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.059+0000 7fbad83ff700 1 -- 172.21.15.60:0/2191620119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b8200 msgr2=0x7fbad00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.059+0000 7fbad83ff700 1 --2- 172.21.15.60:0/2191620119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b8200 0x7fbad00cf640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009840 tx=0x7fbacc009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.059+0000 7fbad83ff700 1 -- 172.21.15.60:0/2191620119 shutdown_connections 2023-07-13T22:21:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.059+0000 7fbad83ff700 1 --2- 172.21.15.60:0/2191620119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00cede0 0x7fbad00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.059+0000 7fbad83ff700 1 --2- 172.21.15.60:0/2191620119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b8200 0x7fbad00cf640 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.059+0000 7fbad83ff700 1 -- 172.21.15.60:0/2191620119 >> 172.21.15.60:0/2191620119 conn(0x7fbad006b2f0 msgr2=0x7fbad006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.059+0000 7fbad83ff700 1 -- 172.21.15.60:0/2191620119 shutdown_connections 2023-07-13T22:21:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.059+0000 7fbad83ff700 1 -- 172.21.15.60:0/2191620119 wait complete. 2023-07-13T22:21:51.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.061+0000 7fbad83ff700 1 Processor -- start 2023-07-13T22:21:51.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.061+0000 7fbad83ff700 1 -- start start 2023-07-13T22:21:51.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad83ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad00b1ac0 0x7fbad01f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad83ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00b8200 0x7fbad00b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad83ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00cede0 0x7fbad0155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad83ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad0159ef0 con 0x7fbad00cede0 2023-07-13T22:21:51.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad83ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad0159450 con 0x7fbad00b1ac0 2023-07-13T22:21:51.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad83ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00b66b0 con 0x7fbad00b8200 2023-07-13T22:21:51.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad699c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00cede0 0x7fbad0155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad699c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00cede0 0x7fbad0155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36422/0 (socket says 172.21.15.60:36422) 2023-07-13T22:21:51.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad619b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad00b1ac0 0x7fbad01f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad699c700 1 -- 172.21.15.60:0/1315291069 learned_addr learned my addr 172.21.15.60:0/1315291069 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:51.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad599a700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00b8200 0x7fbad00b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad699c700 1 -- 172.21.15.60:0/1315291069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00b8200 msgr2=0x7fbad00b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad699c700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00b8200 0x7fbad00b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad699c700 1 -- 172.21.15.60:0/1315291069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad00b1ac0 msgr2=0x7fbad01f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad699c700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad00b1ac0 0x7fbad01f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.062+0000 7fbad699c700 1 -- 172.21.15.60:0/1315291069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc0094a0 con 0x7fbad00cede0 2023-07-13T22:21:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.063+0000 7fbad699c700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00cede0 0x7fbad0155ca0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbac800cd50 tx=0x7fbac800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.063+0000 7fbac6ffd700 1 -- 172.21.15.60:0/1315291069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800cea0 con 0x7fbad00cede0 2023-07-13T22:21:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.063+0000 7fbac6ffd700 1 -- 172.21.15.60:0/1315291069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac800f480 con 0x7fbad00cede0 2023-07-13T22:21:51.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.063+0000 7fbac6ffd700 1 -- 172.21.15.60:0/1315291069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800f830 con 0x7fbad00cede0 2023-07-13T22:21:51.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.063+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad0159970 con 0x7fbad00cede0 2023-07-13T22:21:51.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.063+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad00cb860 con 0x7fbad00cede0 2023-07-13T22:21:51.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.064+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad004e210 con 0x7fbad00cede0 2023-07-13T22:21:51.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.065+0000 7fbac6ffd700 1 -- 172.21.15.60:0/1315291069 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbac800b040 con 0x7fbad00cede0 2023-07-13T22:21:51.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.066+0000 7fbac6ffd700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbab406ace0 0x7fbab406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.066+0000 7fbac6ffd700 1 -- 172.21.15.60:0/1315291069 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbac8089bc0 con 0x7fbad00cede0 2023-07-13T22:21:51.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.066+0000 7fbad619b700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbab406ace0 0x7fbab406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.067+0000 7fbad619b700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbab406ace0 0x7fbab406d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0090b0 tx=0x7fbacc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.070+0000 7fbac6ffd700 1 -- 172.21.15.60:0/1315291069 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbac8054c20 con 0x7fbad00cede0 2023-07-13T22:21:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.276+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbad00cb3e0 con 0x7fbad00cede0 2023-07-13T22:21:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.277+0000 7fbac6ffd700 1 -- 172.21.15.60:0/1315291069 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbad00cb3e0 con 0x7fbad00cede0 2023-07-13T22:21:51.262 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbab406ace0 msgr2=0x7fbab406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbab406ace0 0x7fbab406d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0090b0 tx=0x7fbacc008030 comp rx=0 tx=0).stop 2023-07-13T22:21:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00cede0 msgr2=0x7fbad0155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00cede0 0x7fbad0155ca0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbac800cd50 tx=0x7fbac800a560 comp rx=0 tx=0).stop 2023-07-13T22:21:51.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 shutdown_connections 2023-07-13T22:21:51.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbab406ace0 0x7fbab406d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00cede0 0x7fbad0155ca0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad00b1ac0 0x7fbad01f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 --2- 172.21.15.60:0/1315291069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00b8200 0x7fbad00b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 >> 172.21.15.60:0/1315291069 conn(0x7fbad006b2f0 msgr2=0x7fbad00b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.279+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 shutdown_connections 2023-07-13T22:21:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.280+0000 7fbad83ff700 1 -- 172.21.15.60:0/1315291069 wait complete. 2023-07-13T22:21:51.282 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:21:51.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.501+0000 7f6364dff700 1 Processor -- start 2023-07-13T22:21:51.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.501+0000 7f6364dff700 1 -- start start 2023-07-13T22:21:51.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f6364dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600b8400 0x7f63600cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f6364dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f63600d3750 0x7f63600cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f6364dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63600cff20 con 0x7f63600cfad0 2023-07-13T22:21:51.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f6364dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63600d0ac0 con 0x7f63600d3750 2023-07-13T22:21:51.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f6364dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600b87d0 con 0x7f63600b8400 2023-07-13T22:21:51.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f635e59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600b8400 0x7f63600cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f635e59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600b8400 0x7f63600cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:39466/0 (socket says 172.21.15.60:39466) 2023-07-13T22:21:51.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f6357fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f63600d3750 0x7f63600cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f635e59c700 1 -- 172.21.15.60:0/2718419362 learned_addr learned my addr 172.21.15.60:0/2718419362 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:51.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.503+0000 7f635ed9d700 1 --1- 172.21.15.60:0/2718419362 >> v1:172.21.15.105:6789/0 conn(0x7f63600cfad0 0x7f636015fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:44518/0 (socket says 172.21.15.60:44518) 2023-07-13T22:21:51.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.504+0000 7f635e59c700 1 -- 172.21.15.60:0/2718419362 >> v2:172.21.15.105:3300/0 conn(0x7f63600d3750 msgr2=0x7f63600cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:51.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.504+0000 7f635e59c700 1 --2- 172.21.15.60:0/2718419362 >> v2:172.21.15.105:3300/0 conn(0x7f63600d3750 0x7f63600cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.504+0000 7f635e59c700 1 -- 172.21.15.60:0/2718419362 >> v1:172.21.15.105:6789/0 conn(0x7f63600cfad0 legacy=0x7f636015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.504+0000 7f635dd9b700 1 -- 172.21.15.60:0/2718419362 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586579457 0 0) 0x7f6358001490 con 0x7f63600cfad0 2023-07-13T22:21:51.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.504+0000 7f635e59c700 1 -- 172.21.15.60:0/2718419362 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600b4eb0 con 0x7f63600b8400 2023-07-13T22:21:51.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.504+0000 7f635e59c700 1 --2- 172.21.15.60:0/2718419362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600b8400 0x7f63600cd240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63500098f0 tx=0x7f6350009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6fca77caef645e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.504+0000 7f635dd9b700 1 -- 172.21.15.60:0/2718419362 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350011830 con 0x7f63600b8400 2023-07-13T22:21:51.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.504+0000 7f635dd9b700 1 -- 172.21.15.60:0/2718419362 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350011990 con 0x7f63600b8400 2023-07-13T22:21:51.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.504+0000 7f635dd9b700 1 -- 172.21.15.60:0/2718419362 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350011d20 con 0x7f63600b8400 2023-07-13T22:21:51.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.505+0000 7f6364dff700 1 -- 172.21.15.60:0/2718419362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600b8400 msgr2=0x7f63600cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.505+0000 7f6364dff700 1 --2- 172.21.15.60:0/2718419362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600b8400 0x7f63600cd240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63500098f0 tx=0x7f6350009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:51.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.505+0000 7f6364dff700 1 -- 172.21.15.60:0/2718419362 shutdown_connections 2023-07-13T22:21:51.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.505+0000 7f6364dff700 1 --2- 172.21.15.60:0/2718419362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600b8400 0x7f63600cd240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.505+0000 7f6364dff700 1 --2- 172.21.15.60:0/2718419362 >> v2:172.21.15.105:3300/0 conn(0x7f63600d3750 0x7f63600cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.505+0000 7f6364dff700 1 -- 172.21.15.60:0/2718419362 >> 172.21.15.60:0/2718419362 conn(0x7f636006b390 msgr2=0x7f636006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.505+0000 7f6364dff700 1 -- 172.21.15.60:0/2718419362 shutdown_connections 2023-07-13T22:21:51.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.505+0000 7f6364dff700 1 -- 172.21.15.60:0/2718419362 wait complete. 2023-07-13T22:21:51.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.507+0000 7f6364dff700 1 Processor -- start 2023-07-13T22:21:51.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.507+0000 7f6364dff700 1 -- start start 2023-07-13T22:21:51.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.508+0000 7f6364dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63600b8400 0x7f6360152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.508+0000 7f6364dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600cfad0 0x7f6360151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.508+0000 7f6364dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63600d3750 0x7f63601f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.508+0000 7f6364dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360153cb0 con 0x7f63600b8400 2023-07-13T22:21:51.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.508+0000 7f6364dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360155970 con 0x7f63600d3750 2023-07-13T22:21:51.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.508+0000 7f6364dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6360155040 con 0x7f63600cfad0 2023-07-13T22:21:51.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.508+0000 7f635e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63600b8400 0x7f6360152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.508+0000 7f635e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63600b8400 0x7f6360152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36436/0 (socket says 172.21.15.60:36436) 2023-07-13T22:21:51.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.508+0000 7f635e59c700 1 -- 172.21.15.60:0/2854378244 learned_addr learned my addr 172.21.15.60:0/2854378244 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:51.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f635ed9d700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63600d3750 0x7f63601f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f6357fff700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600cfad0 0x7f6360151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f635e59c700 1 -- 172.21.15.60:0/2854378244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600cfad0 msgr2=0x7f6360151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f635e59c700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600cfad0 0x7f6360151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f635e59c700 1 -- 172.21.15.60:0/2854378244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63600d3750 msgr2=0x7f63601f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f635e59c700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63600d3750 0x7f63601f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f635e59c700 1 -- 172.21.15.60:0/2854378244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350009550 con 0x7f63600b8400 2023-07-13T22:21:51.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f635e59c700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63600b8400 0x7f6360152f90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6350009c40 tx=0x7f6350004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f6356ffd700 1 -- 172.21.15.60:0/2854378244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500036a0 con 0x7f63600b8400 2023-07-13T22:21:51.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f6356ffd700 1 -- 172.21.15.60:0/2854378244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635001e070 con 0x7f63600b8400 2023-07-13T22:21:51.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f6356ffd700 1 -- 172.21.15.60:0/2854378244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350003a20 con 0x7f63600b8400 2023-07-13T22:21:51.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.509+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6360155320 con 0x7f63600b8400 2023-07-13T22:21:51.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.510+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6360151e80 con 0x7f63600b8400 2023-07-13T22:21:51.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.511+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636015b430 con 0x7f63600b8400 2023-07-13T22:21:51.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.512+0000 7f6356ffd700 1 -- 172.21.15.60:0/2854378244 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6350003b80 con 0x7f63600b8400 2023-07-13T22:21:51.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.512+0000 7f6356ffd700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f634006ace0 0x7f634006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.512+0000 7f6356ffd700 1 -- 172.21.15.60:0/2854378244 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f635008a440 con 0x7f63600b8400 2023-07-13T22:21:51.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.513+0000 7f6357fff700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f634006ace0 0x7f634006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.513+0000 7f6357fff700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f634006ace0 0x7f634006d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f63600b8ac0 tx=0x7f6348008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.517+0000 7f6356ffd700 1 -- 172.21.15.60:0/2854378244 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6350055540 con 0x7f63600b8400 2023-07-13T22:21:51.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.745+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f636015b150 con 0x7f63600b8400 2023-07-13T22:21:51.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.745+0000 7f6356ffd700 1 -- 172.21.15.60:0/2854378244 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f635002c3b0 con 0x7f63600b8400 2023-07-13T22:21:51.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f634006ace0 msgr2=0x7f634006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f634006ace0 0x7f634006d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f63600b8ac0 tx=0x7f6348008040 comp rx=0 tx=0).stop 2023-07-13T22:21:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63600b8400 msgr2=0x7f6360152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63600b8400 0x7f6360152f90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6350009c40 tx=0x7f6350004750 comp rx=0 tx=0).stop 2023-07-13T22:21:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 shutdown_connections 2023-07-13T22:21:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f634006ace0 0x7f634006d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63600b8400 0x7f6360152f90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63600d3750 0x7f63601f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 --2- 172.21.15.60:0/2854378244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63600cfad0 0x7f6360151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 >> 172.21.15.60:0/2854378244 conn(0x7f636006b390 msgr2=0x7f63600b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 shutdown_connections 2023-07-13T22:21:51.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:51.748+0000 7f6364dff700 1 -- 172.21.15.60:0/2854378244 wait complete. 2023-07-13T22:21:51.734 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:21:51.751 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:21:51.751 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:51.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.056+0000 7f6338fff700 1 Processor -- start 2023-07-13T22:21:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.056+0000 7f6338fff700 1 -- start start 2023-07-13T22:21:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.057+0000 7f6338fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6334069ed0 0x7f633406a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.057+0000 7f6338fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633406a860 con 0x7f63340c9300 2023-07-13T22:21:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.057+0000 7f6338fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340c3fc0 con 0x7f6334158870 2023-07-13T22:21:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.057+0000 7f6338fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63340c42a0 con 0x7f6334069ed0 2023-07-13T22:21:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.058+0000 7f6331d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6334069ed0 0x7f633406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.058+0000 7f6332d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f63340c9300 0x7f63340c9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51972/0 (socket says 172.21.15.60:51972) 2023-07-13T22:21:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.058+0000 7f6332d9d700 1 -- 172.21.15.60:0/785040065 learned_addr learned my addr 172.21.15.60:0/785040065 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.058+0000 7f6331d9b700 1 -- 172.21.15.60:0/785040065 >> v1:172.21.15.105:6789/0 conn(0x7f6334158870 legacy=0x7f63340c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.058+0000 7f6331d9b700 1 -- 172.21.15.60:0/785040065 >> v1:172.21.15.60:6789/0 conn(0x7f63340c9300 legacy=0x7f63340c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.058+0000 7f6331d9b700 1 -- 172.21.15.60:0/785040065 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340c4580 con 0x7f6334069ed0 2023-07-13T22:21:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.058+0000 7f633159a700 1 -- 172.21.15.60:0/785040065 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248640790 0 0) 0x7f632c001490 con 0x7f63340c9300 2023-07-13T22:21:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.058+0000 7f6331d9b700 1 --2- 172.21.15.60:0/785040065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334069ed0 0x7f633406a320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6328009840 tx=0x7f6328009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e798899f0087e4a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f633159a700 1 -- 172.21.15.60:0/785040065 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328011830 con 0x7f6334069ed0 2023-07-13T22:21:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f633159a700 1 -- 172.21.15.60:0/785040065 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328011990 con 0x7f6334069ed0 2023-07-13T22:21:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f633159a700 1 -- 172.21.15.60:0/785040065 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328011d20 con 0x7f6334069ed0 2023-07-13T22:21:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f6338fff700 1 -- 172.21.15.60:0/785040065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334069ed0 msgr2=0x7f633406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f6338fff700 1 --2- 172.21.15.60:0/785040065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334069ed0 0x7f633406a320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6328009840 tx=0x7f6328009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f6338fff700 1 -- 172.21.15.60:0/785040065 shutdown_connections 2023-07-13T22:21:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f6338fff700 1 --2- 172.21.15.60:0/785040065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334069ed0 0x7f633406a320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f6338fff700 1 -- 172.21.15.60:0/785040065 >> 172.21.15.60:0/785040065 conn(0x7f63340cf410 msgr2=0x7f63340cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f6338fff700 1 -- 172.21.15.60:0/785040065 shutdown_connections 2023-07-13T22:21:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.059+0000 7f6338fff700 1 -- 172.21.15.60:0/785040065 wait complete. 2023-07-13T22:21:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.061+0000 7f6338fff700 1 Processor -- start 2023-07-13T22:21:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.061+0000 7f6338fff700 1 -- start start 2023-07-13T22:21:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.062+0000 7f6338fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6334069ed0 0x7f633406de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.062+0000 7f6338fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63340c9300 0x7f633414a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.062+0000 7f6338fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334158870 0x7f63340ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.062+0000 7f6338fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633414a810 con 0x7f6334158870 2023-07-13T22:21:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.062+0000 7f6338fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341f17c0 con 0x7f63340c9300 2023-07-13T22:21:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.062+0000 7f6338fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63341f1aa0 con 0x7f6334069ed0 2023-07-13T22:21:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6332d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334158870 0x7f63340ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6332d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334158870 0x7f63340ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36446/0 (socket says 172.21.15.60:36446) 2023-07-13T22:21:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6332d9d700 1 -- 172.21.15.60:0/51719073 learned_addr learned my addr 172.21.15.60:0/51719073 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6331d9b700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63340c9300 0x7f633414a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f633259c700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6334069ed0 0x7f633406de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6332d9d700 1 -- 172.21.15.60:0/51719073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6334069ed0 msgr2=0x7f633406de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6332d9d700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6334069ed0 0x7f633406de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6332d9d700 1 -- 172.21.15.60:0/51719073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63340c9300 msgr2=0x7f633414a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6332d9d700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63340c9300 0x7f633414a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6332d9d700 1 -- 172.21.15.60:0/51719073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280094a0 con 0x7f6334158870 2023-07-13T22:21:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6332d9d700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334158870 0x7f63340ca150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f632c00cca0 tx=0x7f632c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.063+0000 7f6322ffd700 1 -- 172.21.15.60:0/51719073 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00ab10 con 0x7f6334158870 2023-07-13T22:21:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.064+0000 7f6322ffd700 1 -- 172.21.15.60:0/51719073 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c00ac90 con 0x7f6334158870 2023-07-13T22:21:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.064+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63341f1d80 con 0x7f6334158870 2023-07-13T22:21:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.064+0000 7f6322ffd700 1 -- 172.21.15.60:0/51719073 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c0070d0 con 0x7f6334158870 2023-07-13T22:21:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.064+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63340c5a80 con 0x7f6334158870 2023-07-13T22:21:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.065+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63340c39f0 con 0x7f6334158870 2023-07-13T22:21:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.066+0000 7f6322ffd700 1 -- 172.21.15.60:0/51719073 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f632c007250 con 0x7f6334158870 2023-07-13T22:21:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.067+0000 7f6322ffd700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f631406ace0 0x7f631406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.067+0000 7f6322ffd700 1 -- 172.21.15.60:0/51719073 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f632c088e60 con 0x7f6334158870 2023-07-13T22:21:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.067+0000 7f633259c700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f631406ace0 0x7f631406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.067+0000 7f633259c700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f631406ace0 0x7f631406d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6334154b10 tx=0x7f6324009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.071+0000 7f6322ffd700 1 -- 172.21.15.60:0/51719073 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f632c053e30 con 0x7f6334158870 2023-07-13T22:21:52.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.278+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f633406a860 con 0x7f6334158870 2023-07-13T22:21:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.280+0000 7f6322ffd700 1 -- 172.21.15.60:0/51719073 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f633406a860 con 0x7f6334158870 2023-07-13T22:21:52.265 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:52.265 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:21:42.683944+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:52.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f631406ace0 msgr2=0x7f631406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f631406ace0 0x7f631406d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6334154b10 tx=0x7f6324009040 comp rx=0 tx=0).stop 2023-07-13T22:21:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334158870 msgr2=0x7f63340ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334158870 0x7f63340ca150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f632c00cca0 tx=0x7f632c00a060 comp rx=0 tx=0).stop 2023-07-13T22:21:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 shutdown_connections 2023-07-13T22:21:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f631406ace0 0x7f631406d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6334158870 0x7f63340ca150 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6334069ed0 0x7f633406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 --2- 172.21.15.60:0/51719073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63340c9300 0x7f633414a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 >> 172.21.15.60:0/51719073 conn(0x7f63340cf410 msgr2=0x7f633414b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 shutdown_connections 2023-07-13T22:21:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.282+0000 7f6338fff700 1 -- 172.21.15.60:0/51719073 wait complete. 2023-07-13T22:21:52.285 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:52.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.507+0000 7fbe3a9ff700 1 Processor -- start 2023-07-13T22:21:52.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.507+0000 7fbe3a9ff700 1 -- start start 2023-07-13T22:21:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.508+0000 7fbe3a9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8200 0x7fbe340cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.508+0000 7fbe3a9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe340cede0 0x7fbe340cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.508+0000 7fbe3a9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe340d1870 con 0x7fbe340b1ac0 2023-07-13T22:21:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.508+0000 7fbe3a9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe340cf370 con 0x7fbe340cede0 2023-07-13T22:21:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.508+0000 7fbe3a9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340d12b0 con 0x7fbe340b8200 2023-07-13T22:21:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe337fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbe340b1ac0 0x7fbe3415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51988/0 (socket says 172.21.15.60:51988) 2023-07-13T22:21:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe337fe700 1 -- 172.21.15.60:0/3988666627 learned_addr learned my addr 172.21.15.60:0/3988666627 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe38f9c700 1 --2- 172.21.15.60:0/3988666627 >> v2:172.21.15.105:3300/0 conn(0x7fbe340cede0 0x7fbe340cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe33fff700 1 --2- 172.21.15.60:0/3988666627 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8200 0x7fbe340cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe32ffd700 1 -- 172.21.15.60:0/3988666627 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446879076 0 0) 0x7fbe340d1870 con 0x7fbe340b1ac0 2023-07-13T22:21:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe32ffd700 1 -- 172.21.15.60:0/3988666627 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe1c0035b0 con 0x7fbe340b1ac0 2023-07-13T22:21:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe33fff700 1 -- 172.21.15.60:0/3988666627 >> v2:172.21.15.105:3300/0 conn(0x7fbe340cede0 msgr2=0x7fbe340cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe33fff700 1 --2- 172.21.15.60:0/3988666627 >> v2:172.21.15.105:3300/0 conn(0x7fbe340cede0 0x7fbe340cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe33fff700 1 -- 172.21.15.60:0/3988666627 >> v1:172.21.15.60:6789/0 conn(0x7fbe340b1ac0 legacy=0x7fbe3415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe33fff700 1 -- 172.21.15.60:0/3988666627 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340d1590 con 0x7fbe340b8200 2023-07-13T22:21:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.509+0000 7fbe38f9c700 1 --2- 172.21.15.60:0/3988666627 >> v2:172.21.15.105:3300/0 conn(0x7fbe340cede0 0x7fbe340cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.510+0000 7fbe33fff700 1 --2- 172.21.15.60:0/3988666627 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8200 0x7fbe340cf640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009840 tx=0x7fbe24009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b28419873500f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.510+0000 7fbe32ffd700 1 -- 172.21.15.60:0/3988666627 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24011830 con 0x7fbe340b8200 2023-07-13T22:21:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.510+0000 7fbe32ffd700 1 -- 172.21.15.60:0/3988666627 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe24011990 con 0x7fbe340b8200 2023-07-13T22:21:52.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.510+0000 7fbe32ffd700 1 -- 172.21.15.60:0/3988666627 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24011d00 con 0x7fbe340b8200 2023-07-13T22:21:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.510+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/3988666627 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8200 msgr2=0x7fbe340cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.510+0000 7fbe3a9ff700 1 --2- 172.21.15.60:0/3988666627 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8200 0x7fbe340cf640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009840 tx=0x7fbe24009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.511+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/3988666627 shutdown_connections 2023-07-13T22:21:52.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.511+0000 7fbe3a9ff700 1 --2- 172.21.15.60:0/3988666627 >> v2:172.21.15.105:3300/0 conn(0x7fbe340cede0 0x7fbe340cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.511+0000 7fbe3a9ff700 1 --2- 172.21.15.60:0/3988666627 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8200 0x7fbe340cf640 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.511+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/3988666627 >> 172.21.15.60:0/3988666627 conn(0x7fbe3406b2f0 msgr2=0x7fbe3406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.511+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/3988666627 shutdown_connections 2023-07-13T22:21:52.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.511+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/3988666627 wait complete. 2023-07-13T22:21:52.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.513+0000 7fbe3a9ff700 1 Processor -- start 2023-07-13T22:21:52.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.513+0000 7fbe3a9ff700 1 -- start start 2023-07-13T22:21:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.514+0000 7fbe3a9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340b1ac0 0x7fbe340c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe3a9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340b8200 0x7fbe3414f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe3a9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340cede0 0x7fbe341e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe3a9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe341e6e10 con 0x7fbe340b1ac0 2023-07-13T22:21:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe3a9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe341e6570 con 0x7fbe340b8200 2023-07-13T22:21:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe3a9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe341e6850 con 0x7fbe340cede0 2023-07-13T22:21:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe33fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340b1ac0 0x7fbe340c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe33fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340b1ac0 0x7fbe340c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36450/0 (socket says 172.21.15.60:36450) 2023-07-13T22:21:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe33fff700 1 -- 172.21.15.60:0/1146056552 learned_addr learned my addr 172.21.15.60:0/1146056552 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe38f9c700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340cede0 0x7fbe341e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe337fe700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340b8200 0x7fbe3414f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe33fff700 1 -- 172.21.15.60:0/1146056552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340cede0 msgr2=0x7fbe341e5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe33fff700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340cede0 0x7fbe341e5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe33fff700 1 -- 172.21.15.60:0/1146056552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340b8200 msgr2=0x7fbe3414f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe33fff700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340b8200 0x7fbe3414f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe33fff700 1 -- 172.21.15.60:0/1146056552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c009a70 con 0x7fbe340b1ac0 2023-07-13T22:21:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe38f9c700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340cede0 0x7fbe341e5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.515+0000 7fbe33fff700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340b1ac0 0x7fbe340c7eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009b90 tx=0x7fbe24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.516+0000 7fbe30ff9700 1 -- 172.21.15.60:0/1146056552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240036a0 con 0x7fbe340b1ac0 2023-07-13T22:21:52.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.516+0000 7fbe30ff9700 1 -- 172.21.15.60:0/1146056552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2401f070 con 0x7fbe340b1ac0 2023-07-13T22:21:52.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.516+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe240094a0 con 0x7fbe340b1ac0 2023-07-13T22:21:52.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.516+0000 7fbe30ff9700 1 -- 172.21.15.60:0/1146056552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24003a40 con 0x7fbe340b1ac0 2023-07-13T22:21:52.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.516+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe340c9ea0 con 0x7fbe340b1ac0 2023-07-13T22:21:52.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.517+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe340c9ce0 con 0x7fbe340b1ac0 2023-07-13T22:21:52.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.518+0000 7fbe30ff9700 1 -- 172.21.15.60:0/1146056552 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe24003ba0 con 0x7fbe340b1ac0 2023-07-13T22:21:52.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.519+0000 7fbe30ff9700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe2006ace0 0x7fbe2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.519+0000 7fbe30ff9700 1 -- 172.21.15.60:0/1146056552 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbe24021070 con 0x7fbe340b1ac0 2023-07-13T22:21:52.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.519+0000 7fbe337fe700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe2006ace0 0x7fbe2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.519+0000 7fbe337fe700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe2006ace0 0x7fbe2006d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe340c9690 tx=0x7fbe28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.523+0000 7fbe30ff9700 1 -- 172.21.15.60:0/1146056552 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbe24055d90 con 0x7fbe340b1ac0 2023-07-13T22:21:52.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.725+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbe340ca0c0 con 0x7fbe340b1ac0 2023-07-13T22:21:52.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.725+0000 7fbe30ff9700 1 -- 172.21.15.60:0/1146056552 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbe240150a0 con 0x7fbe340b1ac0 2023-07-13T22:21:52.710 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:52.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe2006ace0 msgr2=0x7fbe2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe2006ace0 0x7fbe2006d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe340c9690 tx=0x7fbe28009040 comp rx=0 tx=0).stop 2023-07-13T22:21:52.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340b1ac0 msgr2=0x7fbe340c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340b1ac0 0x7fbe340c7eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009b90 tx=0x7fbe24004750 comp rx=0 tx=0).stop 2023-07-13T22:21:52.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 shutdown_connections 2023-07-13T22:21:52.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe2006ace0 0x7fbe2006d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340b1ac0 0x7fbe340c7eb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340cede0 0x7fbe341e5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 --2- 172.21.15.60:0/1146056552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340b8200 0x7fbe3414f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 >> 172.21.15.60:0/1146056552 conn(0x7fbe3406b2f0 msgr2=0x7fbe340b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 shutdown_connections 2023-07-13T22:21:52.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.728+0000 7fbe3a9ff700 1 -- 172.21.15.60:0/1146056552 wait complete. 2023-07-13T22:21:52.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.953+0000 7fc64cbff700 1 Processor -- start 2023-07-13T22:21:52.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.954+0000 7fc64cbff700 1 -- start start 2023-07-13T22:21:52.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc64cbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6480b45b0 0x7fc648160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc64cbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480cd0f0 0x7fc6480b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc64cbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc64806b1e0 0x7fc64806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc64cbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480d1040 con 0x7fc64806b1e0 2023-07-13T22:21:52.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc64cbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480b4ec0 con 0x7fc6480b45b0 2023-07-13T22:21:52.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc64cbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480cd840 con 0x7fc6480cd0f0 2023-07-13T22:21:52.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc646d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc64806b1e0 0x7fc64806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc646d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc64806b1e0 0x7fc64806b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36460/0 (socket says 172.21.15.60:36460) 2023-07-13T22:21:52.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc646d9d700 1 -- 172.21.15.60:0/2073025885 learned_addr learned my addr 172.21.15.60:0/2073025885 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:52.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc64659c700 1 --2- 172.21.15.60:0/2073025885 >> v2:172.21.15.105:3300/0 conn(0x7fc6480b45b0 0x7fc648160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.955+0000 7fc63ffff700 1 --2- 172.21.15.60:0/2073025885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480cd0f0 0x7fc6480b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc646d9d700 1 -- 172.21.15.60:0/2073025885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480cd0f0 msgr2=0x7fc6480b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc646d9d700 1 --2- 172.21.15.60:0/2073025885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480cd0f0 0x7fc6480b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc646d9d700 1 -- 172.21.15.60:0/2073025885 >> v2:172.21.15.105:3300/0 conn(0x7fc6480b45b0 msgr2=0x7fc648160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc646d9d700 1 --2- 172.21.15.60:0/2073025885 >> v2:172.21.15.105:3300/0 conn(0x7fc6480b45b0 0x7fc648160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc646d9d700 1 -- 172.21.15.60:0/2073025885 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480d0710 con 0x7fc64806b1e0 2023-07-13T22:21:52.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc63ffff700 1 --2- 172.21.15.60:0/2073025885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480cd0f0 0x7fc6480b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:52.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc646d9d700 1 --2- 172.21.15.60:0/2073025885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 0x7fc64806b660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc64806c4e0 tx=0x7fc64000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7516ed83142c7137 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc645d9b700 1 -- 172.21.15.60:0/2073025885 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000e070 con 0x7fc64806b1e0 2023-07-13T22:21:52.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc645d9b700 1 -- 172.21.15.60:0/2073025885 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64000b910 con 0x7fc64806b1e0 2023-07-13T22:21:52.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.956+0000 7fc645d9b700 1 -- 172.21.15.60:0/2073025885 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000bcc0 con 0x7fc64806b1e0 2023-07-13T22:21:52.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.957+0000 7fc64cbff700 1 -- 172.21.15.60:0/2073025885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 msgr2=0x7fc64806b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.957+0000 7fc64cbff700 1 --2- 172.21.15.60:0/2073025885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 0x7fc64806b660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc64806c4e0 tx=0x7fc64000b330 comp rx=0 tx=0).stop 2023-07-13T22:21:52.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.957+0000 7fc64cbff700 1 -- 172.21.15.60:0/2073025885 shutdown_connections 2023-07-13T22:21:52.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.957+0000 7fc64cbff700 1 --2- 172.21.15.60:0/2073025885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 0x7fc64806b660 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.957+0000 7fc64cbff700 1 --2- 172.21.15.60:0/2073025885 >> v2:172.21.15.105:3300/0 conn(0x7fc6480b45b0 0x7fc648160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.957+0000 7fc64cbff700 1 --2- 172.21.15.60:0/2073025885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480cd0f0 0x7fc6480b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.957+0000 7fc64cbff700 1 -- 172.21.15.60:0/2073025885 >> 172.21.15.60:0/2073025885 conn(0x7fc648072240 msgr2=0x7fc6480d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.957+0000 7fc64cbff700 1 -- 172.21.15.60:0/2073025885 shutdown_connections 2023-07-13T22:21:52.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.957+0000 7fc64cbff700 1 -- 172.21.15.60:0/2073025885 wait complete. 2023-07-13T22:21:52.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.958+0000 7fc64cbff700 1 Processor -- start 2023-07-13T22:21:52.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.959+0000 7fc64cbff700 1 -- start start 2023-07-13T22:21:52.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.959+0000 7fc64cbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 0x7fc64815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64cbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480b45b0 0x7fc64815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64cbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6480cd0f0 0x7fc6480cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64cbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64815fc90 con 0x7fc64806b1e0 2023-07-13T22:21:52.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64cbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64815ebf0 con 0x7fc6480cd0f0 2023-07-13T22:21:52.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64cbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64815eed0 con 0x7fc6480b45b0 2023-07-13T22:21:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 0x7fc64815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 0x7fc64815e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36470/0 (socket says 172.21.15.60:36470) 2023-07-13T22:21:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64659c700 1 -- 172.21.15.60:0/4241124725 learned_addr learned my addr 172.21.15.60:0/4241124725 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc646d9d700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6480cd0f0 0x7fc6480cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc63ffff700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480b45b0 0x7fc64815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64659c700 1 -- 172.21.15.60:0/4241124725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480b45b0 msgr2=0x7fc64815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64659c700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480b45b0 0x7fc64815e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64659c700 1 -- 172.21.15.60:0/4241124725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6480cd0f0 msgr2=0x7fc6480cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64659c700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6480cd0f0 0x7fc6480cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64659c700 1 -- 172.21.15.60:0/4241124725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640009b30 con 0x7fc64806b1e0 2023-07-13T22:21:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.960+0000 7fc64659c700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 0x7fc64815e0a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc63800e930 tx=0x7fc63800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.961+0000 7fc63effd700 1 -- 172.21.15.60:0/4241124725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638009690 con 0x7fc64806b1e0 2023-07-13T22:21:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.961+0000 7fc63effd700 1 -- 172.21.15.60:0/4241124725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63800c850 con 0x7fc64806b1e0 2023-07-13T22:21:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.961+0000 7fc63effd700 1 -- 172.21.15.60:0/4241124725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63800cbe0 con 0x7fc64806b1e0 2023-07-13T22:21:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.961+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64815f1b0 con 0x7fc64806b1e0 2023-07-13T22:21:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.961+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64815f770 con 0x7fc64806b1e0 2023-07-13T22:21:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.962+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64804e210 con 0x7fc64806b1e0 2023-07-13T22:21:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.963+0000 7fc63effd700 1 -- 172.21.15.60:0/4241124725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc63800cd40 con 0x7fc64806b1e0 2023-07-13T22:21:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.964+0000 7fc63effd700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc62806ace0 0x7fc62806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.964+0000 7fc63effd700 1 -- 172.21.15.60:0/4241124725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc638089650 con 0x7fc64806b1e0 2023-07-13T22:21:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.964+0000 7fc63ffff700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc62806ace0 0x7fc62806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.965+0000 7fc63ffff700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc62806ace0 0x7fc62806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc6481569c0 tx=0x7fc630008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:52.969+0000 7fc63effd700 1 -- 172.21.15.60:0/4241124725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc638054620 con 0x7fc64806b1e0 2023-07-13T22:21:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.176+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6480b2270 con 0x7fc64806b1e0 2023-07-13T22:21:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.177+0000 7fc63effd700 1 -- 172.21.15.60:0/4241124725 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc638057c40 con 0x7fc64806b1e0 2023-07-13T22:21:53.162 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:53.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc62806ace0 msgr2=0x7fc62806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc62806ace0 0x7fc62806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc6481569c0 tx=0x7fc630008040 comp rx=0 tx=0).stop 2023-07-13T22:21:53.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 msgr2=0x7fc64815e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 0x7fc64815e0a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc63800e930 tx=0x7fc63800ecf0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 shutdown_connections 2023-07-13T22:21:53.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc62806ace0 0x7fc62806d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc64806b1e0 0x7fc64815e0a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6480cd0f0 0x7fc6480cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 --2- 172.21.15.60:0/4241124725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6480b45b0 0x7fc64815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 >> 172.21.15.60:0/4241124725 conn(0x7fc648072240 msgr2=0x7fc6480b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 shutdown_connections 2023-07-13T22:21:53.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.179+0000 7fc64cbff700 1 -- 172.21.15.60:0/4241124725 wait complete. 2023-07-13T22:21:53.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.404+0000 7ff95c5ff700 1 Processor -- start 2023-07-13T22:21:53.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.404+0000 7ff95c5ff700 1 -- start start 2023-07-13T22:21:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.405+0000 7ff95c5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff9540b8200 0x7ff9540cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.405+0000 7ff95c5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 0x7ff9540cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.405+0000 7ff95c5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9540d1870 con 0x7ff9540cede0 2023-07-13T22:21:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.405+0000 7ff95c5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9540cf370 con 0x7ff9540b8200 2023-07-13T22:21:53.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.405+0000 7ff95c5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9540d12b0 con 0x7ff9540b1ac0 2023-07-13T22:21:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff95ab9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9540cede0 0x7ff95415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51994/0 (socket says 172.21.15.60:51994) 2023-07-13T22:21:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff95ab9c700 1 -- 172.21.15.60:0/3779608417 learned_addr learned my addr 172.21.15.60:0/3779608417 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff95a39b700 1 --2- 172.21.15.60:0/3779608417 >> v2:172.21.15.105:3300/0 conn(0x7ff9540b8200 0x7ff9540cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff959b9a700 1 --2- 172.21.15.60:0/3779608417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 0x7ff9540cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff959399700 1 -- 172.21.15.60:0/3779608417 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890473095 0 0) 0x7ff9540d1870 con 0x7ff9540cede0 2023-07-13T22:21:53.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff959399700 1 -- 172.21.15.60:0/3779608417 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9400035b0 con 0x7ff9540cede0 2023-07-13T22:21:53.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff95a39b700 1 -- 172.21.15.60:0/3779608417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 msgr2=0x7ff9540cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff95a39b700 1 --2- 172.21.15.60:0/3779608417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 0x7ff9540cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff95a39b700 1 -- 172.21.15.60:0/3779608417 >> v1:172.21.15.60:6789/0 conn(0x7ff9540cede0 legacy=0x7ff95415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.406+0000 7ff95a39b700 1 -- 172.21.15.60:0/3779608417 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9540d1590 con 0x7ff9540b8200 2023-07-13T22:21:53.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.407+0000 7ff95a39b700 1 --2- 172.21.15.60:0/3779608417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9540b8200 0x7ff9540cf640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff950009840 tx=0x7ff950009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f84c61ca5c22b4dd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.407+0000 7ff959399700 1 -- 172.21.15.60:0/3779608417 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950011830 con 0x7ff9540b8200 2023-07-13T22:21:53.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.407+0000 7ff959399700 1 -- 172.21.15.60:0/3779608417 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff950011990 con 0x7ff9540b8200 2023-07-13T22:21:53.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.407+0000 7ff959399700 1 -- 172.21.15.60:0/3779608417 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950011d00 con 0x7ff9540b8200 2023-07-13T22:21:53.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.407+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3779608417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9540b8200 msgr2=0x7ff9540cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.407+0000 7ff95c5ff700 1 --2- 172.21.15.60:0/3779608417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9540b8200 0x7ff9540cf640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff950009840 tx=0x7ff950009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:53.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.408+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3779608417 shutdown_connections 2023-07-13T22:21:53.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.408+0000 7ff95c5ff700 1 --2- 172.21.15.60:0/3779608417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9540b8200 0x7ff9540cf640 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.408+0000 7ff95c5ff700 1 --2- 172.21.15.60:0/3779608417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 0x7ff9540cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.408+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3779608417 >> 172.21.15.60:0/3779608417 conn(0x7ff95406b2f0 msgr2=0x7ff95406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.408+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3779608417 shutdown_connections 2023-07-13T22:21:53.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.408+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3779608417 wait complete. 2023-07-13T22:21:53.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.409+0000 7ff95c5ff700 1 Processor -- start 2023-07-13T22:21:53.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.410+0000 7ff95c5ff700 1 -- start start 2023-07-13T22:21:53.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.410+0000 7ff95c5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 0x7ff9540cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.410+0000 7ff95c5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9540b8200 0x7ff954159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.410+0000 7ff95c5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9540cede0 0x7ff9541e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.410+0000 7ff95c5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9541e6050 con 0x7ff9540cede0 2023-07-13T22:21:53.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95c5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9541e5130 con 0x7ff9540b8200 2023-07-13T22:21:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95c5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9541e53d0 con 0x7ff9540b1ac0 2023-07-13T22:21:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95ab9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9540cede0 0x7ff9541e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95ab9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9540cede0 0x7ff9541e4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36486/0 (socket says 172.21.15.60:36486) 2023-07-13T22:21:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95a39b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 0x7ff9540cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95ab9c700 1 -- 172.21.15.60:0/3502718997 learned_addr learned my addr 172.21.15.60:0/3502718997 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff959b9a700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9540b8200 0x7ff954159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95ab9c700 1 -- 172.21.15.60:0/3502718997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 msgr2=0x7ff9540cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95ab9c700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 0x7ff9540cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95ab9c700 1 -- 172.21.15.60:0/3502718997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9540b8200 msgr2=0x7ff954159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95ab9c700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9540b8200 0x7ff954159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95ab9c700 1 -- 172.21.15.60:0/3502718997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500094a0 con 0x7ff9540cede0 2023-07-13T22:21:53.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.411+0000 7ff95ab9c700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9540cede0 0x7ff9541e4bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00caa0 tx=0x7ff94c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.412+0000 7ff94affd700 1 -- 172.21.15.60:0/3502718997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00a9e0 con 0x7ff9540cede0 2023-07-13T22:21:53.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.412+0000 7ff94affd700 1 -- 172.21.15.60:0/3502718997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c00ab40 con 0x7ff9540cede0 2023-07-13T22:21:53.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.412+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9541e56b0 con 0x7ff9540cede0 2023-07-13T22:21:53.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.412+0000 7ff94affd700 1 -- 172.21.15.60:0/3502718997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c012400 con 0x7ff9540cede0 2023-07-13T22:21:53.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.412+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9541e5c70 con 0x7ff9540cede0 2023-07-13T22:21:53.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.413+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9541e6190 con 0x7ff9540cede0 2023-07-13T22:21:53.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.414+0000 7ff94affd700 1 -- 172.21.15.60:0/3502718997 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff94c01b3b0 con 0x7ff9540cede0 2023-07-13T22:21:53.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.415+0000 7ff94affd700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff93806ace0 0x7ff93806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.415+0000 7ff94affd700 1 -- 172.21.15.60:0/3502718997 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff94c089180 con 0x7ff9540cede0 2023-07-13T22:21:53.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.415+0000 7ff95a39b700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff93806ace0 0x7ff93806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.416+0000 7ff95a39b700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff93806ace0 0x7ff93806d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff950000c00 tx=0x7ff950008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.420+0000 7ff94affd700 1 -- 172.21.15.60:0/3502718997 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff9541e6190 con 0x7ff9540cede0 2023-07-13T22:21:53.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.625+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9541e6190 con 0x7ff9540cede0 2023-07-13T22:21:53.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.626+0000 7ff94affd700 1 -- 172.21.15.60:0/3502718997 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9541e6190 con 0x7ff9540cede0 2023-07-13T22:21:53.610 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:53.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff93806ace0 msgr2=0x7ff93806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff93806ace0 0x7ff93806d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff950000c00 tx=0x7ff950008030 comp rx=0 tx=0).stop 2023-07-13T22:21:53.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9540cede0 msgr2=0x7ff9541e4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9540cede0 0x7ff9541e4bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00caa0 tx=0x7ff94c00cdb0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 shutdown_connections 2023-07-13T22:21:53.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff93806ace0 0x7ff93806d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9540cede0 0x7ff9541e4bf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9540b1ac0 0x7ff9540cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 --2- 172.21.15.60:0/3502718997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9540b8200 0x7ff954159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 >> 172.21.15.60:0/3502718997 conn(0x7ff95406b2f0 msgr2=0x7ff95415c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 shutdown_connections 2023-07-13T22:21:53.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.628+0000 7ff95c5ff700 1 -- 172.21.15.60:0/3502718997 wait complete. 2023-07-13T22:21:53.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:53.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.852+0000 7fa12cfff700 1 Processor -- start 2023-07-13T22:21:53.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.853+0000 7fa12cfff700 1 -- start start 2023-07-13T22:21:53.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa12cfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1280b8200 0x7fa1280cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa12cfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1280b1ac0 0x7fa1280cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa12cfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280cede0 0x7fa12815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa12cfff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280d1870 con 0x7fa1280b8200 2023-07-13T22:21:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa12cfff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280cf370 con 0x7fa1280b1ac0 2023-07-13T22:21:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa12cfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280d12b0 con 0x7fa1280cede0 2023-07-13T22:21:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa12659c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1280b8200 0x7fa1280cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa12659c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1280b8200 0x7fa1280cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36498/0 (socket says 172.21.15.60:36498) 2023-07-13T22:21:53.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa12659c700 1 -- 172.21.15.60:0/988791258 learned_addr learned my addr 172.21.15.60:0/988791258 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:53.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa126d9d700 1 --2- 172.21.15.60:0/988791258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280cede0 0x7fa12815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.854+0000 7fa11ffff700 1 --2- 172.21.15.60:0/988791258 >> v2:172.21.15.105:3300/0 conn(0x7fa1280b1ac0 0x7fa1280cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.855+0000 7fa12659c700 1 -- 172.21.15.60:0/988791258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280cede0 msgr2=0x7fa12815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.855+0000 7fa12659c700 1 --2- 172.21.15.60:0/988791258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280cede0 0x7fa12815ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.855+0000 7fa12659c700 1 -- 172.21.15.60:0/988791258 >> v2:172.21.15.105:3300/0 conn(0x7fa1280b1ac0 msgr2=0x7fa1280cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.855+0000 7fa12659c700 1 --2- 172.21.15.60:0/988791258 >> v2:172.21.15.105:3300/0 conn(0x7fa1280b1ac0 0x7fa1280cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.855+0000 7fa12659c700 1 -- 172.21.15.60:0/988791258 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280d1590 con 0x7fa1280b8200 2023-07-13T22:21:53.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.855+0000 7fa12659c700 1 --2- 172.21.15.60:0/988791258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280b8200 0x7fa1280cf640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa1280d07b0 tx=0x7fa118009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9606dcfee13f977 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.855+0000 7fa125d9b700 1 -- 172.21.15.60:0/988791258 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800b870 con 0x7fa1280b8200 2023-07-13T22:21:53.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.855+0000 7fa125d9b700 1 -- 172.21.15.60:0/988791258 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11800b9d0 con 0x7fa1280b8200 2023-07-13T22:21:53.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.855+0000 7fa125d9b700 1 -- 172.21.15.60:0/988791258 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800bd60 con 0x7fa1280b8200 2023-07-13T22:21:53.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.856+0000 7fa12cfff700 1 -- 172.21.15.60:0/988791258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280b8200 msgr2=0x7fa1280cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.856+0000 7fa12cfff700 1 --2- 172.21.15.60:0/988791258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280b8200 0x7fa1280cf640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa1280d07b0 tx=0x7fa118009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.856+0000 7fa12cfff700 1 -- 172.21.15.60:0/988791258 shutdown_connections 2023-07-13T22:21:53.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.856+0000 7fa12cfff700 1 --2- 172.21.15.60:0/988791258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280cede0 0x7fa12815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.856+0000 7fa12cfff700 1 --2- 172.21.15.60:0/988791258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280b8200 0x7fa1280cf640 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.856+0000 7fa12cfff700 1 --2- 172.21.15.60:0/988791258 >> v2:172.21.15.105:3300/0 conn(0x7fa1280b1ac0 0x7fa1280cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.856+0000 7fa12cfff700 1 -- 172.21.15.60:0/988791258 >> 172.21.15.60:0/988791258 conn(0x7fa12806b2f0 msgr2=0x7fa12806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.856+0000 7fa12cfff700 1 -- 172.21.15.60:0/988791258 shutdown_connections 2023-07-13T22:21:53.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.856+0000 7fa12cfff700 1 -- 172.21.15.60:0/988791258 wait complete. 2023-07-13T22:21:53.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.858+0000 7fa12cfff700 1 Processor -- start 2023-07-13T22:21:53.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.858+0000 7fa12cfff700 1 -- start start 2023-07-13T22:21:53.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa12cfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1280b1ac0 0x7fa1281f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa12cfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280b8200 0x7fa128146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa12cfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cede0 0x7fa1280c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa12cfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa128158c00 con 0x7fa1280cede0 2023-07-13T22:21:53.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa12cfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c75b0 con 0x7fa1280b1ac0 2023-07-13T22:21:53.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa12cfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c8250 con 0x7fa1280b8200 2023-07-13T22:21:53.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa126d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cede0 0x7fa1280c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa126d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cede0 0x7fa1280c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36508/0 (socket says 172.21.15.60:36508) 2023-07-13T22:21:53.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa12659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1280b1ac0 0x7fa1281f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa126d9d700 1 -- 172.21.15.60:0/1183457401 learned_addr learned my addr 172.21.15.60:0/1183457401 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:53.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa11ffff700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280b8200 0x7fa128146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa126d9d700 1 -- 172.21.15.60:0/1183457401 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280b8200 msgr2=0x7fa128146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa126d9d700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280b8200 0x7fa128146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa126d9d700 1 -- 172.21.15.60:0/1183457401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1280b1ac0 msgr2=0x7fa1281f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa126d9d700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1280b1ac0 0x7fa1281f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.859+0000 7fa126d9d700 1 -- 172.21.15.60:0/1183457401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa118009550 con 0x7fa1280cede0 2023-07-13T22:21:53.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.860+0000 7fa126d9d700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cede0 0x7fa1280c8d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa120007e60 tx=0x7fa12000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.860+0000 7fa11effd700 1 -- 172.21.15.60:0/1183457401 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000ed30 con 0x7fa1280cede0 2023-07-13T22:21:53.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.860+0000 7fa11effd700 1 -- 172.21.15.60:0/1183457401 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa120014420 con 0x7fa1280cede0 2023-07-13T22:21:53.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.860+0000 7fa11effd700 1 -- 172.21.15.60:0/1183457401 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1200147d0 con 0x7fa1280cede0 2023-07-13T22:21:53.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.860+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1280c8530 con 0x7fa1280cede0 2023-07-13T22:21:53.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.860+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1281f2020 con 0x7fa1280cede0 2023-07-13T22:21:53.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.861+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1280c80b0 con 0x7fa1280cede0 2023-07-13T22:21:53.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.862+0000 7fa11effd700 1 -- 172.21.15.60:0/1183457401 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa120014930 con 0x7fa1280cede0 2023-07-13T22:21:53.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.863+0000 7fa11effd700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa10806ace0 0x7fa10806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.863+0000 7fa11effd700 1 -- 172.21.15.60:0/1183457401 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa120089ac0 con 0x7fa1280cede0 2023-07-13T22:21:53.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.863+0000 7fa12659c700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa10806ace0 0x7fa10806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.864+0000 7fa12659c700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa10806ace0 0x7fa10806d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa118000c00 tx=0x7fa118008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:53.868+0000 7fa11effd700 1 -- 172.21.15.60:0/1183457401 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa1280c80b0 con 0x7fa1280cede0 2023-07-13T22:21:54.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.073+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa1280c8a50 con 0x7fa1280cede0 2023-07-13T22:21:54.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.073+0000 7fa11effd700 1 -- 172.21.15.60:0/1183457401 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa1280c8a50 con 0x7fa1280cede0 2023-07-13T22:21:54.058 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.075+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa10806ace0 msgr2=0x7fa10806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.075+0000 7fa12cfff700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa10806ace0 0x7fa10806d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa118000c00 tx=0x7fa118008030 comp rx=0 tx=0).stop 2023-07-13T22:21:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.075+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cede0 msgr2=0x7fa1280c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.075+0000 7fa12cfff700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cede0 0x7fa1280c8d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa120007e60 tx=0x7fa12000e3c0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.076+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 shutdown_connections 2023-07-13T22:21:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.076+0000 7fa12cfff700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa10806ace0 0x7fa10806d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.076+0000 7fa12cfff700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cede0 0x7fa1280c8d30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.076+0000 7fa12cfff700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1280b1ac0 0x7fa1281f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.076+0000 7fa12cfff700 1 --2- 172.21.15.60:0/1183457401 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1280b8200 0x7fa128146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.076+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 >> 172.21.15.60:0/1183457401 conn(0x7fa12806b2f0 msgr2=0x7fa1280d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.076+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 shutdown_connections 2023-07-13T22:21:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.076+0000 7fa12cfff700 1 -- 172.21.15.60:0/1183457401 wait complete. 2023-07-13T22:21:54.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:21:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.298+0000 7f1f56bff700 1 Processor -- start 2023-07-13T22:21:54.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.298+0000 7f1f56bff700 1 -- start start 2023-07-13T22:21:54.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.299+0000 7f1f56bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500cefa0 0x7f1f5006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.299+0000 7f1f56bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f500cd760 0x7f1f500cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.299+0000 7f1f56bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f500cf3f0 con 0x7f1f500b12a0 2023-07-13T22:21:54.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.299+0000 7f1f56bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5006a5f0 con 0x7f1f500cd760 2023-07-13T22:21:54.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.299+0000 7f1f56bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f50160ed0 con 0x7f1f500cefa0 2023-07-13T22:21:54.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.300+0000 7f1f4ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500cefa0 0x7f1f5006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.300+0000 7f1f4ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500cefa0 0x7f1f5006bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:39570/0 (socket says 172.21.15.60:39570) 2023-07-13T22:21:54.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.300+0000 7f1f4ffff700 1 -- 172.21.15.60:0/508074380 learned_addr learned my addr 172.21.15.60:0/508074380 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:54.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.300+0000 7f1f5519c700 1 --2- 172.21.15.60:0/508074380 >> v2:172.21.15.105:3300/0 conn(0x7f1f500cd760 0x7f1f500cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.300+0000 7f1f4ffff700 1 -- 172.21.15.60:0/508074380 >> v2:172.21.15.105:3300/0 conn(0x7f1f500cd760 msgr2=0x7f1f500cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:54.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.300+0000 7f1f4ffff700 1 --2- 172.21.15.60:0/508074380 >> v2:172.21.15.105:3300/0 conn(0x7f1f500cd760 0x7f1f500cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.300+0000 7f1f4ffff700 1 -- 172.21.15.60:0/508074380 >> v1:172.21.15.105:6789/0 conn(0x7f1f500b12a0 legacy=0x7f1f500b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.300+0000 7f1f4ffff700 1 -- 172.21.15.60:0/508074380 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f501609a0 con 0x7f1f500cefa0 2023-07-13T22:21:54.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.300+0000 7f1f4ffff700 1 --2- 172.21.15.60:0/508074380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500cefa0 0x7f1f5006bad0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0098f0 tx=0x7f1f3c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f09b0036e0d776 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f4f7fe700 1 -- 172.21.15.60:0/508074380 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c011830 con 0x7f1f500cefa0 2023-07-13T22:21:54.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f4f7fe700 1 -- 172.21.15.60:0/508074380 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c011990 con 0x7f1f500cefa0 2023-07-13T22:21:54.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f4f7fe700 1 -- 172.21.15.60:0/508074380 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c011d20 con 0x7f1f500cefa0 2023-07-13T22:21:54.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f56bff700 1 -- 172.21.15.60:0/508074380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500cefa0 msgr2=0x7f1f5006bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f56bff700 1 --2- 172.21.15.60:0/508074380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500cefa0 0x7f1f5006bad0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0098f0 tx=0x7f1f3c009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:54.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f56bff700 1 -- 172.21.15.60:0/508074380 shutdown_connections 2023-07-13T22:21:54.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f56bff700 1 --2- 172.21.15.60:0/508074380 >> v2:172.21.15.105:3300/0 conn(0x7f1f500cd760 0x7f1f500cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f56bff700 1 --2- 172.21.15.60:0/508074380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500cefa0 0x7f1f5006bad0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f56bff700 1 -- 172.21.15.60:0/508074380 >> 172.21.15.60:0/508074380 conn(0x7f1f500cc2a0 msgr2=0x7f1f500cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:54.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f56bff700 1 -- 172.21.15.60:0/508074380 shutdown_connections 2023-07-13T22:21:54.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.301+0000 7f1f56bff700 1 -- 172.21.15.60:0/508074380 wait complete. 2023-07-13T22:21:54.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.303+0000 7f1f56bff700 1 Processor -- start 2023-07-13T22:21:54.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.303+0000 7f1f56bff700 1 -- start start 2023-07-13T22:21:54.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.304+0000 7f1f56bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500b12a0 0x7f1f500ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.304+0000 7f1f56bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f500cd760 0x7f1f501f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.304+0000 7f1f56bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f500cefa0 0x7f1f5014ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.304+0000 7f1f56bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f501518f0 con 0x7f1f500cd760 2023-07-13T22:21:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.304+0000 7f1f56bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500caef0 con 0x7f1f500cefa0 2023-07-13T22:21:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.304+0000 7f1f56bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500c68f0 con 0x7f1f500b12a0 2023-07-13T22:21:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f4ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f500cd760 0x7f1f501f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f5499b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500b12a0 0x7f1f500ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f4ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f500cd760 0x7f1f501f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36518/0 (socket says 172.21.15.60:36518) 2023-07-13T22:21:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f4ffff700 1 -- 172.21.15.60:0/3910385483 learned_addr learned my addr 172.21.15.60:0/3910385483 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f5519c700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f500cefa0 0x7f1f5014ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f4ffff700 1 -- 172.21.15.60:0/3910385483 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500b12a0 msgr2=0x7f1f500ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f4ffff700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500b12a0 0x7f1f500ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f4ffff700 1 -- 172.21.15.60:0/3910385483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f500cefa0 msgr2=0x7f1f5014ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f4ffff700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f500cefa0 0x7f1f5014ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f4ffff700 1 -- 172.21.15.60:0/3910385483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c009550 con 0x7f1f500cd760 2023-07-13T22:21:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.305+0000 7f1f4ffff700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f500cd760 0x7f1f501f1750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009c40 tx=0x7f1f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.306+0000 7f1f4d7fa700 1 -- 172.21.15.60:0/3910385483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c004300 con 0x7f1f500cd760 2023-07-13T22:21:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.306+0000 7f1f4d7fa700 1 -- 172.21.15.60:0/3910385483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c01f070 con 0x7f1f500cd760 2023-07-13T22:21:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.306+0000 7f1f4d7fa700 1 -- 172.21.15.60:0/3910385483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0036a0 con 0x7f1f500cd760 2023-07-13T22:21:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.306+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f500c6bd0 con 0x7f1f500cd760 2023-07-13T22:21:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.306+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f500c7190 con 0x7f1f500cd760 2023-07-13T22:21:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.307+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5014e410 con 0x7f1f500cd760 2023-07-13T22:21:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.308+0000 7f1f4d7fa700 1 -- 172.21.15.60:0/3910385483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f3c021030 con 0x7f1f500cd760 2023-07-13T22:21:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.310+0000 7f1f4d7fa700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f3406ace0 0x7f1f3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.310+0000 7f1f4d7fa700 1 -- 172.21.15.60:0/3910385483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1f3c08a9f0 con 0x7f1f500cd760 2023-07-13T22:21:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.310+0000 7f1f5499b700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f3406ace0 0x7f1f3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.311+0000 7f1f5499b700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f3406ace0 0x7f1f3406d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f44007fd0 tx=0x7f1f44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.314+0000 7f1f4d7fa700 1 -- 172.21.15.60:0/3910385483 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1f3c055af0 con 0x7f1f500cd760 2023-07-13T22:21:54.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.538+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1f50073810 con 0x7f1f500cd760 2023-07-13T22:21:54.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.538+0000 7f1f4d7fa700 1 -- 172.21.15.60:0/3910385483 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1f3c015080 con 0x7f1f500cd760 2023-07-13T22:21:54.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.540+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f3406ace0 msgr2=0x7f1f3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.540+0000 7f1f56bff700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f3406ace0 0x7f1f3406d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f44007fd0 tx=0x7f1f44009040 comp rx=0 tx=0).stop 2023-07-13T22:21:54.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.540+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f500cd760 msgr2=0x7f1f501f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.540+0000 7f1f56bff700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f500cd760 0x7f1f501f1750 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009c40 tx=0x7f1f3c004750 comp rx=0 tx=0).stop 2023-07-13T22:21:54.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.541+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 shutdown_connections 2023-07-13T22:21:54.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.541+0000 7f1f56bff700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f3406ace0 0x7f1f3406d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.541+0000 7f1f56bff700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f500cefa0 0x7f1f5014ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.541+0000 7f1f56bff700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f500b12a0 0x7f1f500ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.541+0000 7f1f56bff700 1 --2- 172.21.15.60:0/3910385483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f500cd760 0x7f1f501f1750 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.541+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 >> 172.21.15.60:0/3910385483 conn(0x7f1f500cc2a0 msgr2=0x7f1f500b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:54.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.541+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 shutdown_connections 2023-07-13T22:21:54.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.541+0000 7f1f56bff700 1 -- 172.21.15.60:0/3910385483 wait complete. 2023-07-13T22:21:54.527 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:21:54.543 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:21:54.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:54.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-07-13T22:21:54.615 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:21:54.615 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:21:54.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:21:54.616 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:21:54.663 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:21:54.664 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:54.716 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:21:54.716 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:54.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:54.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.983+0000 7f32a4bff700 1 Processor -- start 2023-07-13T22:21:54.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.983+0000 7f32a4bff700 1 -- start start 2023-07-13T22:21:54.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.984+0000 7f32a4bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32a00b8400 0x7f32a00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.984+0000 7f32a4bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 0x7f32a00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.984+0000 7f32a4bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00cff20 con 0x7f32a00cfad0 2023-07-13T22:21:54.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.984+0000 7f32a4bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a00d0ac0 con 0x7f32a00b8400 2023-07-13T22:21:54.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.985+0000 7f32a4bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00b87d0 con 0x7f32a00d3750 2023-07-13T22:21:54.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.985+0000 7f329e59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32a00b8400 0x7f32a00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.985+0000 7f329e59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32a00b8400 0x7f32a00cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50660/0 (socket says 172.21.15.60:50660) 2023-07-13T22:21:54.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.985+0000 7f329e59c700 1 -- 172.21.15.60:0/2686304314 learned_addr learned my addr 172.21.15.60:0/2686304314 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:54.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.985+0000 7f3297fff700 1 --2- 172.21.15.60:0/2686304314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 0x7f32a00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.985+0000 7f329e59c700 1 -- 172.21.15.60:0/2686304314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 msgr2=0x7f32a00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:54.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.985+0000 7f329e59c700 1 --2- 172.21.15.60:0/2686304314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 0x7f32a00cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.985+0000 7f329e59c700 1 -- 172.21.15.60:0/2686304314 >> v1:172.21.15.105:6789/0 conn(0x7f32a00cfad0 legacy=0x7f32a015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.985+0000 7f329e59c700 1 -- 172.21.15.60:0/2686304314 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00b4eb0 con 0x7f32a00b8400 2023-07-13T22:21:54.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f329e59c700 1 --2- 172.21.15.60:0/2686304314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00b8400 0x7f32a00cd240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f32900098f0 tx=0x7f3290009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b784966be56b3ab server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f329dd9b700 1 -- 172.21.15.60:0/2686304314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329000b8a0 con 0x7f32a00b8400 2023-07-13T22:21:54.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f329dd9b700 1 -- 172.21.15.60:0/2686304314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329000ba00 con 0x7f32a00b8400 2023-07-13T22:21:54.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f329dd9b700 1 -- 172.21.15.60:0/2686304314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329000bd90 con 0x7f32a00b8400 2023-07-13T22:21:54.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f32a4bff700 1 -- 172.21.15.60:0/2686304314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00b8400 msgr2=0x7f32a00cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f32a4bff700 1 --2- 172.21.15.60:0/2686304314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00b8400 0x7f32a00cd240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f32900098f0 tx=0x7f3290009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:54.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f32a4bff700 1 -- 172.21.15.60:0/2686304314 shutdown_connections 2023-07-13T22:21:54.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f32a4bff700 1 --2- 172.21.15.60:0/2686304314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00b8400 0x7f32a00cd240 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f32a4bff700 1 --2- 172.21.15.60:0/2686304314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 0x7f32a00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.986+0000 7f32a4bff700 1 -- 172.21.15.60:0/2686304314 >> 172.21.15.60:0/2686304314 conn(0x7f32a006b390 msgr2=0x7f32a006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:54.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.987+0000 7f32a4bff700 1 -- 172.21.15.60:0/2686304314 shutdown_connections 2023-07-13T22:21:54.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.987+0000 7f32a4bff700 1 -- 172.21.15.60:0/2686304314 wait complete. 2023-07-13T22:21:54.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.988+0000 7f32a4bff700 1 Processor -- start 2023-07-13T22:21:54.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.988+0000 7f32a4bff700 1 -- start start 2023-07-13T22:21:54.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.989+0000 7f32a4bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32a00b8400 0x7f32a0153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.989+0000 7f32a4bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00cfad0 0x7f32a0153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.989+0000 7f32a4bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 0x7f32a00c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.989+0000 7f32a4bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c66b0 con 0x7f32a00cfad0 2023-07-13T22:21:54.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.989+0000 7f32a4bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c9ff0 con 0x7f32a00b8400 2023-07-13T22:21:54.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.989+0000 7f32a4bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00cacc0 con 0x7f32a00d3750 2023-07-13T22:21:54.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f3297fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00cfad0 0x7f32a0153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f3297fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00cfad0 0x7f32a0153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50662/0 (socket says 172.21.15.60:50662) 2023-07-13T22:21:54.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f3297fff700 1 -- 172.21.15.60:0/2395310127 learned_addr learned my addr 172.21.15.60:0/2395310127 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:54.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f329e59c700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32a00b8400 0x7f32a0153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f329ed9d700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 0x7f32a00c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f3297fff700 1 -- 172.21.15.60:0/2395310127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 msgr2=0x7f32a00c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f3297fff700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 0x7f32a00c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f3297fff700 1 -- 172.21.15.60:0/2395310127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32a00b8400 msgr2=0x7f32a0153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f3297fff700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32a00b8400 0x7f32a0153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f3297fff700 1 -- 172.21.15.60:0/2395310127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3290009550 con 0x7f32a00cfad0 2023-07-13T22:21:54.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.990+0000 7f3297fff700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00cfad0 0x7f32a0153eb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f328800d830 tx=0x7f328800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.991+0000 7f3296ffd700 1 -- 172.21.15.60:0/2395310127 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32880125f0 con 0x7f32a00cfad0 2023-07-13T22:21:54.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.991+0000 7f3296ffd700 1 -- 172.21.15.60:0/2395310127 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3288012750 con 0x7f32a00cfad0 2023-07-13T22:21:54.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.991+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a00cafa0 con 0x7f32a00cfad0 2023-07-13T22:21:54.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.991+0000 7f3296ffd700 1 -- 172.21.15.60:0/2395310127 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288012ae0 con 0x7f32a00cfad0 2023-07-13T22:21:54.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.991+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a00c6900 con 0x7f32a00cfad0 2023-07-13T22:21:54.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.992+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a00c9910 con 0x7f32a00cfad0 2023-07-13T22:21:54.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.993+0000 7f3296ffd700 1 -- 172.21.15.60:0/2395310127 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3288012c40 con 0x7f32a00cfad0 2023-07-13T22:21:54.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.994+0000 7f3296ffd700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f328006ad10 0x7f328006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.994+0000 7f3296ffd700 1 -- 172.21.15.60:0/2395310127 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3288089870 con 0x7f32a00cfad0 2023-07-13T22:21:54.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.994+0000 7f329e59c700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f328006ad10 0x7f328006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.995+0000 7f329e59c700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f328006ad10 0x7f328006d1c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3290014040 tx=0x7f3290008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:54.998+0000 7f3296ffd700 1 -- 172.21.15.60:0/2395310127 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f32880548d0 con 0x7f32a00cfad0 2023-07-13T22:21:55.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:21:55.053 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:21:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.215+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32a00cf590 con 0x7f32a00cfad0 2023-07-13T22:21:55.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.216+0000 7f3296ffd700 1 -- 172.21.15.60:0/2395310127 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f3288057ef0 con 0x7f32a00cfad0 2023-07-13T22:21:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:55.201 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:21:42.683944+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:55.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f328006ad10 msgr2=0x7f328006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f328006ad10 0x7f328006d1c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3290014040 tx=0x7f3290008030 comp rx=0 tx=0).stop 2023-07-13T22:21:55.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00cfad0 msgr2=0x7f32a0153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00cfad0 0x7f32a0153eb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f328800d830 tx=0x7f328800dbf0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 shutdown_connections 2023-07-13T22:21:55.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f328006ad10 0x7f328006d1c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32a00d3750 0x7f32a00c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32a00b8400 0x7f32a0153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 --2- 172.21.15.60:0/2395310127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32a00cfad0 0x7f32a0153eb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.218+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 >> 172.21.15.60:0/2395310127 conn(0x7f32a006b390 msgr2=0x7f32a015bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.219+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 shutdown_connections 2023-07-13T22:21:55.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.219+0000 7f32a4bff700 1 -- 172.21.15.60:0/2395310127 wait complete. 2023-07-13T22:21:55.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:55.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:55.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.443+0000 7f4f665ff700 1 Processor -- start 2023-07-13T22:21:55.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.443+0000 7f4f665ff700 1 -- start start 2023-07-13T22:21:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.444+0000 7f4f665ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f600add00 0x7f4f6006ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.444+0000 7f4f665ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600c8e90 con 0x7f4f600ca9f0 2023-07-13T22:21:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.444+0000 7f4f665ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600ae0d0 con 0x7f4f6006a5e0 2023-07-13T22:21:55.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.444+0000 7f4f665ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6006d3b0 con 0x7f4f600add00 2023-07-13T22:21:55.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.444+0000 7f4f5f7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4f600ca9f0 0x7f4f60069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34016/0 (socket says 172.21.15.60:34016) 2023-07-13T22:21:55.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.445+0000 7f4f5f7fe700 1 -- 172.21.15.60:0/223331123 learned_addr learned my addr 172.21.15.60:0/223331123 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:55.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.445+0000 7f4f5ffff700 1 --2- 172.21.15.60:0/223331123 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f600add00 0x7f4f6006ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.445+0000 7f4f5effd700 1 -- 172.21.15.60:0/223331123 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951080720 0 0) 0x7f4f600c8e90 con 0x7f4f600ca9f0 2023-07-13T22:21:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.445+0000 7f4f5effd700 1 -- 172.21.15.60:0/223331123 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f480035b0 con 0x7f4f600ca9f0 2023-07-13T22:21:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.445+0000 7f4f5effd700 1 -- 172.21.15.60:0/223331123 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85753148 0 0) 0x7f4f600ae0d0 con 0x7f4f6006a5e0 2023-07-13T22:21:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.445+0000 7f4f5effd700 1 -- 172.21.15.60:0/223331123 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f600c8e90 con 0x7f4f6006a5e0 2023-07-13T22:21:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.445+0000 7f4f5ffff700 1 -- 172.21.15.60:0/223331123 >> v1:172.21.15.105:6789/0 conn(0x7f4f6006a5e0 legacy=0x7f4f600b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.445+0000 7f4f5ffff700 1 -- 172.21.15.60:0/223331123 >> v1:172.21.15.60:6789/0 conn(0x7f4f600ca9f0 legacy=0x7f4f60069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.445+0000 7f4f5ffff700 1 -- 172.21.15.60:0/223331123 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600c90e0 con 0x7f4f600add00 2023-07-13T22:21:55.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.446+0000 7f4f5ffff700 1 --2- 172.21.15.60:0/223331123 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f600add00 0x7f4f6006ce70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f50009870 tx=0x7f4f50009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b1ca3aebe36ba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.446+0000 7f4f5effd700 1 -- 172.21.15.60:0/223331123 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f50011830 con 0x7f4f600add00 2023-07-13T22:21:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.446+0000 7f4f5effd700 1 -- 172.21.15.60:0/223331123 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f50011990 con 0x7f4f600add00 2023-07-13T22:21:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.446+0000 7f4f5effd700 1 -- 172.21.15.60:0/223331123 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f50011d20 con 0x7f4f600add00 2023-07-13T22:21:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.446+0000 7f4f665ff700 1 -- 172.21.15.60:0/223331123 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f600add00 msgr2=0x7f4f6006ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.446+0000 7f4f665ff700 1 --2- 172.21.15.60:0/223331123 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f600add00 0x7f4f6006ce70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f50009870 tx=0x7f4f50009b80 comp rx=0 tx=0).stop 2023-07-13T22:21:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.446+0000 7f4f665ff700 1 -- 172.21.15.60:0/223331123 shutdown_connections 2023-07-13T22:21:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.446+0000 7f4f665ff700 1 --2- 172.21.15.60:0/223331123 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f600add00 0x7f4f6006ce70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.446+0000 7f4f665ff700 1 -- 172.21.15.60:0/223331123 >> 172.21.15.60:0/223331123 conn(0x7f4f600cbf50 msgr2=0x7f4f600ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.447+0000 7f4f665ff700 1 -- 172.21.15.60:0/223331123 shutdown_connections 2023-07-13T22:21:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.447+0000 7f4f665ff700 1 -- 172.21.15.60:0/223331123 wait complete. 2023-07-13T22:21:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.448+0000 7f4f665ff700 1 Processor -- start 2023-07-13T22:21:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.448+0000 7f4f665ff700 1 -- start start 2023-07-13T22:21:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.449+0000 7f4f665ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f6006a5e0 0x7f4f600cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.449+0000 7f4f665ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f600add00 0x7f4f6014fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.449+0000 7f4f665ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f600ca9f0 0x7f4f600c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.449+0000 7f4f665ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f600c74e0 con 0x7f4f600ca9f0 2023-07-13T22:21:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.449+0000 7f4f665ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f601502a0 con 0x7f4f600add00 2023-07-13T22:21:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.449+0000 7f4f665ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f600cd420 con 0x7f4f6006a5e0 2023-07-13T22:21:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f64b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f600ca9f0 0x7f4f600c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f64b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f600ca9f0 0x7f4f600c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50670/0 (socket says 172.21.15.60:50670) 2023-07-13T22:21:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f64b9c700 1 -- 172.21.15.60:0/2927735101 learned_addr learned my addr 172.21.15.60:0/2927735101 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f5ffff700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f6006a5e0 0x7f4f600cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f5f7fe700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f600add00 0x7f4f6014fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f64b9c700 1 -- 172.21.15.60:0/2927735101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f6006a5e0 msgr2=0x7f4f600cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f64b9c700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f6006a5e0 0x7f4f600cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f64b9c700 1 -- 172.21.15.60:0/2927735101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f600add00 msgr2=0x7f4f6014fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f64b9c700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f600add00 0x7f4f6014fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f64b9c700 1 -- 172.21.15.60:0/2927735101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f500094a0 con 0x7f4f600ca9f0 2023-07-13T22:21:55.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.450+0000 7f4f64b9c700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f600ca9f0 0x7f4f600c5980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800cbc0 tx=0x7f4f5800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.451+0000 7f4f5cff9700 1 -- 172.21.15.60:0/2927735101 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58003c00 con 0x7f4f600ca9f0 2023-07-13T22:21:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.451+0000 7f4f5cff9700 1 -- 172.21.15.60:0/2927735101 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f58003d60 con 0x7f4f600ca9f0 2023-07-13T22:21:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.451+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f600ad100 con 0x7f4f600ca9f0 2023-07-13T22:21:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.451+0000 7f4f5cff9700 1 -- 172.21.15.60:0/2927735101 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5800aae0 con 0x7f4f600ca9f0 2023-07-13T22:21:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.451+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f600ad6c0 con 0x7f4f600ca9f0 2023-07-13T22:21:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.452+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f600c7670 con 0x7f4f600ca9f0 2023-07-13T22:21:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.453+0000 7f4f5cff9700 1 -- 172.21.15.60:0/2927735101 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4f5800ac40 con 0x7f4f600ca9f0 2023-07-13T22:21:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.454+0000 7f4f5cff9700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4f4c06ace0 0x7f4f4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.454+0000 7f4f5cff9700 1 -- 172.21.15.60:0/2927735101 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4f580894a0 con 0x7f4f600ca9f0 2023-07-13T22:21:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.454+0000 7f4f5ffff700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4f4c06ace0 0x7f4f4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.455+0000 7f4f5ffff700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4f4c06ace0 0x7f4f4c06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4f500097c0 tx=0x7f4f5001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.458+0000 7f4f5cff9700 1 -- 172.21.15.60:0/2927735101 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4f600c7670 con 0x7f4f600ca9f0 2023-07-13T22:21:55.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.670+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4f600c7620 con 0x7f4f600ca9f0 2023-07-13T22:21:55.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.670+0000 7f4f5cff9700 1 -- 172.21.15.60:0/2927735101 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4f580545a0 con 0x7f4f600ca9f0 2023-07-13T22:21:55.656 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4f4c06ace0 msgr2=0x7f4f4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4f4c06ace0 0x7f4f4c06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4f500097c0 tx=0x7f4f5001c040 comp rx=0 tx=0).stop 2023-07-13T22:21:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f600ca9f0 msgr2=0x7f4f600c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f600ca9f0 0x7f4f600c5980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800cbc0 tx=0x7f4f5800ced0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 shutdown_connections 2023-07-13T22:21:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4f4c06ace0 0x7f4f4c06d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f600ca9f0 0x7f4f600c5980 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f6006a5e0 0x7f4f600cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 --2- 172.21.15.60:0/2927735101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f600add00 0x7f4f6014fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 >> 172.21.15.60:0/2927735101 conn(0x7f4f600cbf50 msgr2=0x7f4f600b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 shutdown_connections 2023-07-13T22:21:55.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.673+0000 7f4f665ff700 1 -- 172.21.15.60:0/2927735101 wait complete. 2023-07-13T22:21:55.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:55.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.896+0000 7f689ddff700 1 Processor -- start 2023-07-13T22:21:55.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.897+0000 7f689ddff700 1 -- start start 2023-07-13T22:21:55.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.898+0000 7f689ddff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f68980d11c0 0x7f68980cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.898+0000 7f689ddff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f68980cf9f0 0x7f6898165eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.898+0000 7f689ddff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68980cefd0 con 0x7f689815c530 2023-07-13T22:21:55.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.898+0000 7f689ddff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68980d1780 con 0x7f68980d11c0 2023-07-13T22:21:55.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.898+0000 7f689ddff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68980cd2d0 con 0x7f68980cf9f0 2023-07-13T22:21:55.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.898+0000 7f6896ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f68980d11c0 0x7f68980cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.898+0000 7f6896ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f68980d11c0 0x7f68980cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50674/0 (socket says 172.21.15.60:50674) 2023-07-13T22:21:55.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.898+0000 7f6896ffd700 1 -- 172.21.15.60:0/4155644384 learned_addr learned my addr 172.21.15.60:0/4155644384 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:55.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.898+0000 7f6897fff700 1 --2- 172.21.15.60:0/4155644384 >> v2:172.21.15.105:3300/0 conn(0x7f68980cf9f0 0x7f6898165eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.899+0000 7f6896ffd700 1 -- 172.21.15.60:0/4155644384 >> v2:172.21.15.105:3300/0 conn(0x7f68980cf9f0 msgr2=0x7f6898165eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:55.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.899+0000 7f6896ffd700 1 --2- 172.21.15.60:0/4155644384 >> v2:172.21.15.105:3300/0 conn(0x7f68980cf9f0 0x7f6898165eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.899+0000 7f6896ffd700 1 -- 172.21.15.60:0/4155644384 >> v1:172.21.15.105:6789/0 conn(0x7f689815c530 legacy=0x7f68980b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.899+0000 7f6896ffd700 1 -- 172.21.15.60:0/4155644384 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68980cd5b0 con 0x7f68980d11c0 2023-07-13T22:21:55.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.899+0000 7f6896ffd700 1 --2- 172.21.15.60:0/4155644384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980d11c0 0x7f68980cf4b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f68980b8910 tx=0x7f688c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3465838117d9164 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.899+0000 7f68967fc700 1 -- 172.21.15.60:0/4155644384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c011830 con 0x7f68980d11c0 2023-07-13T22:21:55.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.899+0000 7f68967fc700 1 -- 172.21.15.60:0/4155644384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c011990 con 0x7f68980d11c0 2023-07-13T22:21:55.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.899+0000 7f68967fc700 1 -- 172.21.15.60:0/4155644384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c011d20 con 0x7f68980d11c0 2023-07-13T22:21:55.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.900+0000 7f689ddff700 1 -- 172.21.15.60:0/4155644384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980d11c0 msgr2=0x7f68980cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.900+0000 7f689ddff700 1 --2- 172.21.15.60:0/4155644384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980d11c0 0x7f68980cf4b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f68980b8910 tx=0x7f688c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.900+0000 7f689ddff700 1 -- 172.21.15.60:0/4155644384 shutdown_connections 2023-07-13T22:21:55.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.900+0000 7f689ddff700 1 --2- 172.21.15.60:0/4155644384 >> v2:172.21.15.105:3300/0 conn(0x7f68980cf9f0 0x7f6898165eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.900+0000 7f689ddff700 1 --2- 172.21.15.60:0/4155644384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980d11c0 0x7f68980cf4b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.900+0000 7f689ddff700 1 -- 172.21.15.60:0/4155644384 >> 172.21.15.60:0/4155644384 conn(0x7f689806b400 msgr2=0x7f689806b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.900+0000 7f689ddff700 1 -- 172.21.15.60:0/4155644384 shutdown_connections 2023-07-13T22:21:55.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.900+0000 7f689ddff700 1 -- 172.21.15.60:0/4155644384 wait complete. 2023-07-13T22:21:55.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.902+0000 7f689ddff700 1 Processor -- start 2023-07-13T22:21:55.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.902+0000 7f689ddff700 1 -- start start 2023-07-13T22:21:55.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f689ddff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980cf9f0 0x7f68980ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f689ddff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68980d11c0 0x7f68981e6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f689ddff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68981e8070 0x7f689815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f689ddff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68981e6ea0 con 0x7f68980cf9f0 2023-07-13T22:21:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f689ddff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980cad50 con 0x7f68981e8070 2023-07-13T22:21:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f689ddff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980cb030 con 0x7f68980d11c0 2023-07-13T22:21:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f68977fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980cf9f0 0x7f68980ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f68977fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980cf9f0 0x7f68980ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50682/0 (socket says 172.21.15.60:50682) 2023-07-13T22:21:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f68977fe700 1 -- 172.21.15.60:0/862655038 learned_addr learned my addr 172.21.15.60:0/862655038 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f6896ffd700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68980d11c0 0x7f68981e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f6897fff700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68981e8070 0x7f689815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f68977fe700 1 -- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68980d11c0 msgr2=0x7f68981e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f68977fe700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68980d11c0 0x7f68981e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f68977fe700 1 -- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68981e8070 msgr2=0x7f689815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f68977fe700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68981e8070 0x7f689815b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f68977fe700 1 -- 172.21.15.60:0/862655038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c009550 con 0x7f68980cf9f0 2023-07-13T22:21:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f6896ffd700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68980d11c0 0x7f68981e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.903+0000 7f6897fff700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68981e8070 0x7f689815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.904+0000 7f68977fe700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980cf9f0 0x7f68980ca680 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f688800c450 tx=0x7f688800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.904+0000 7f687ffff700 1 -- 172.21.15.60:0/862655038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880052d0 con 0x7f68980cf9f0 2023-07-13T22:21:55.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.904+0000 7f687ffff700 1 -- 172.21.15.60:0/862655038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888005430 con 0x7f68980cf9f0 2023-07-13T22:21:55.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.904+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68980cb310 con 0x7f68980cf9f0 2023-07-13T22:21:55.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.904+0000 7f687ffff700 1 -- 172.21.15.60:0/862655038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880038c0 con 0x7f68980cf9f0 2023-07-13T22:21:55.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.904+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68981596a0 con 0x7f68980cf9f0 2023-07-13T22:21:55.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.905+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68980b8b70 con 0x7f68980cf9f0 2023-07-13T22:21:55.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.906+0000 7f687ffff700 1 -- 172.21.15.60:0/862655038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6888003a20 con 0x7f68980cf9f0 2023-07-13T22:21:55.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.907+0000 7f687ffff700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f688406ad10 0x7f688406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.907+0000 7f687ffff700 1 -- 172.21.15.60:0/862655038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6888088990 con 0x7f68980cf9f0 2023-07-13T22:21:55.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.907+0000 7f6896ffd700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f688406ad10 0x7f688406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.908+0000 7f6896ffd700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f688406ad10 0x7f688406d1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68981e8db0 tx=0x7f688c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:55.912+0000 7f687ffff700 1 -- 172.21.15.60:0/862655038 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f68880538d0 con 0x7f68980cf9f0 2023-07-13T22:21:56.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.117+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f689804e210 con 0x7f68980cf9f0 2023-07-13T22:21:56.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.117+0000 7f687ffff700 1 -- 172.21.15.60:0/862655038 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6888056ef0 con 0x7f68980cf9f0 2023-07-13T22:21:56.102 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:56.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f688406ad10 msgr2=0x7f688406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f688406ad10 0x7f688406d1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68981e8db0 tx=0x7f688c008030 comp rx=0 tx=0).stop 2023-07-13T22:21:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980cf9f0 msgr2=0x7f68980ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980cf9f0 0x7f68980ca680 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f688800c450 tx=0x7f688800c810 comp rx=0 tx=0).stop 2023-07-13T22:21:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 shutdown_connections 2023-07-13T22:21:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f688406ad10 0x7f688406d1c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68980cf9f0 0x7f68980ca680 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68981e8070 0x7f689815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 --2- 172.21.15.60:0/862655038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68980d11c0 0x7f68981e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 >> 172.21.15.60:0/862655038 conn(0x7f689806b400 msgr2=0x7f689815c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 shutdown_connections 2023-07-13T22:21:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.120+0000 7f689ddff700 1 -- 172.21.15.60:0/862655038 wait complete. 2023-07-13T22:21:56.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:56.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.343+0000 7f6aec3ff700 1 Processor -- start 2023-07-13T22:21:56.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.343+0000 7f6aec3ff700 1 -- start start 2023-07-13T22:21:56.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.344+0000 7f6aec3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ae40b45b0 0x7f6ae4160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.344+0000 7f6aec3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae40cd0f0 0x7f6ae40b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aec3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae40d1040 con 0x7f6ae406b1e0 2023-07-13T22:21:56.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aec3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae40b4ec0 con 0x7f6ae40b45b0 2023-07-13T22:21:56.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aec3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40cd840 con 0x7f6ae40cd0f0 2023-07-13T22:21:56.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aea19b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ae40b45b0 0x7f6ae4160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aea19b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ae40b45b0 0x7f6ae4160970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50698/0 (socket says 172.21.15.60:50698) 2023-07-13T22:21:56.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aea19b700 1 -- 172.21.15.60:0/1412694631 learned_addr learned my addr 172.21.15.60:0/1412694631 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:56.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6ae999a700 1 --2- 172.21.15.60:0/1412694631 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae40cd0f0 0x7f6ae40b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aea19b700 1 -- 172.21.15.60:0/1412694631 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae40cd0f0 msgr2=0x7f6ae40b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:56.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aea19b700 1 --2- 172.21.15.60:0/1412694631 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae40cd0f0 0x7f6ae40b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aea19b700 1 -- 172.21.15.60:0/1412694631 >> v1:172.21.15.105:6789/0 conn(0x7f6ae406b1e0 legacy=0x7f6ae406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.345+0000 7f6aea19b700 1 -- 172.21.15.60:0/1412694631 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae40d0710 con 0x7f6ae40b45b0 2023-07-13T22:21:56.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.346+0000 7f6aea19b700 1 --2- 172.21.15.60:0/1412694631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae40b45b0 0x7f6ae4160970 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00098f0 tx=0x7f6ae0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7cd44112b5acf1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:56.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.346+0000 7f6ae9199700 1 -- 172.21.15.60:0/1412694631 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000b8a0 con 0x7f6ae40b45b0 2023-07-13T22:21:56.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.346+0000 7f6ae9199700 1 -- 172.21.15.60:0/1412694631 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae000ba00 con 0x7f6ae40b45b0 2023-07-13T22:21:56.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.346+0000 7f6ae9199700 1 -- 172.21.15.60:0/1412694631 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000bd90 con 0x7f6ae40b45b0 2023-07-13T22:21:56.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.346+0000 7f6aec3ff700 1 -- 172.21.15.60:0/1412694631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae40b45b0 msgr2=0x7f6ae4160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.346+0000 7f6aec3ff700 1 --2- 172.21.15.60:0/1412694631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae40b45b0 0x7f6ae4160970 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00098f0 tx=0x7f6ae0009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:56.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.347+0000 7f6aec3ff700 1 -- 172.21.15.60:0/1412694631 shutdown_connections 2023-07-13T22:21:56.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.347+0000 7f6aec3ff700 1 --2- 172.21.15.60:0/1412694631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae40b45b0 0x7f6ae4160970 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.347+0000 7f6aec3ff700 1 --2- 172.21.15.60:0/1412694631 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae40cd0f0 0x7f6ae40b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.347+0000 7f6aec3ff700 1 -- 172.21.15.60:0/1412694631 >> 172.21.15.60:0/1412694631 conn(0x7f6ae4072240 msgr2=0x7f6ae40d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:56.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.347+0000 7f6aec3ff700 1 -- 172.21.15.60:0/1412694631 shutdown_connections 2023-07-13T22:21:56.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.347+0000 7f6aec3ff700 1 -- 172.21.15.60:0/1412694631 wait complete. 2023-07-13T22:21:56.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.348+0000 7f6aec3ff700 1 Processor -- start 2023-07-13T22:21:56.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.349+0000 7f6aec3ff700 1 -- start start 2023-07-13T22:21:56.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.349+0000 7f6aec3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae406b1e0 0x7f6ae4156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.349+0000 7f6aec3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ae40b45b0 0x7f6ae40b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.349+0000 7f6aec3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae40cd0f0 0x7f6ae40d2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.349+0000 7f6aec3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4158b90 con 0x7f6ae406b1e0 2023-07-13T22:21:56.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.349+0000 7f6aec3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae41570e0 con 0x7f6ae40b45b0 2023-07-13T22:21:56.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.349+0000 7f6aec3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae415a740 con 0x7f6ae40cd0f0 2023-07-13T22:21:56.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6aea19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae406b1e0 0x7f6ae4156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6aea19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae406b1e0 0x7f6ae4156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50704/0 (socket says 172.21.15.60:50704) 2023-07-13T22:21:56.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6aea19b700 1 -- 172.21.15.60:0/663009663 learned_addr learned my addr 172.21.15.60:0/663009663 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:56.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6aea19b700 1 -- 172.21.15.60:0/663009663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae40cd0f0 msgr2=0x7f6ae40d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6aea19b700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae40cd0f0 0x7f6ae40d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6ae999a700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ae40b45b0 0x7f6ae40b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6aea19b700 1 -- 172.21.15.60:0/663009663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ae40b45b0 msgr2=0x7f6ae40b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6aea19b700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ae40b45b0 0x7f6ae40b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6aea19b700 1 -- 172.21.15.60:0/663009663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae0009550 con 0x7f6ae406b1e0 2023-07-13T22:21:56.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6ae999a700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ae40b45b0 0x7f6ae40b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.350+0000 7f6aea19b700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae406b1e0 0x7f6ae4156ba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009c40 tx=0x7f6ae0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.351+0000 7f6adaffd700 1 -- 172.21.15.60:0/663009663 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004400 con 0x7f6ae406b1e0 2023-07-13T22:21:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.351+0000 7f6adaffd700 1 -- 172.21.15.60:0/663009663 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae001f070 con 0x7f6ae406b1e0 2023-07-13T22:21:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.351+0000 7f6adaffd700 1 -- 172.21.15.60:0/663009663 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00038e0 con 0x7f6ae406b1e0 2023-07-13T22:21:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.351+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae415a9e0 con 0x7f6ae406b1e0 2023-07-13T22:21:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.351+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae415afa0 con 0x7f6ae406b1e0 2023-07-13T22:21:56.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.352+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae404e210 con 0x7f6ae406b1e0 2023-07-13T22:21:56.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.353+0000 7f6adaffd700 1 -- 172.21.15.60:0/663009663 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ae0021030 con 0x7f6ae406b1e0 2023-07-13T22:21:56.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.354+0000 7f6adaffd700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6ac806ace0 0x7f6ac806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.354+0000 7f6adaffd700 1 -- 172.21.15.60:0/663009663 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6ae008aa10 con 0x7f6ae406b1e0 2023-07-13T22:21:56.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.354+0000 7f6ae999a700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6ac806ace0 0x7f6ac806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.355+0000 7f6ae999a700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6ac806ace0 0x7f6ac806d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40cbdf0 tx=0x7f6ad4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:56.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.358+0000 7f6adaffd700 1 -- 172.21.15.60:0/663009663 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6ae0055a70 con 0x7f6ae406b1e0 2023-07-13T22:21:56.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.563+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ae40b2270 con 0x7f6ae406b1e0 2023-07-13T22:21:56.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.564+0000 7f6adaffd700 1 -- 172.21.15.60:0/663009663 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ae0059090 con 0x7f6ae406b1e0 2023-07-13T22:21:56.549 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:56.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.566+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6ac806ace0 msgr2=0x7f6ac806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.566+0000 7f6aec3ff700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6ac806ace0 0x7f6ac806d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40cbdf0 tx=0x7f6ad4008040 comp rx=0 tx=0).stop 2023-07-13T22:21:56.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.566+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae406b1e0 msgr2=0x7f6ae4156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.566+0000 7f6aec3ff700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae406b1e0 0x7f6ae4156ba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009c40 tx=0x7f6ae0004750 comp rx=0 tx=0).stop 2023-07-13T22:21:56.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.567+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 shutdown_connections 2023-07-13T22:21:56.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.567+0000 7f6aec3ff700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6ac806ace0 0x7f6ac806d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.567+0000 7f6aec3ff700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae406b1e0 0x7f6ae4156ba0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.567+0000 7f6aec3ff700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae40cd0f0 0x7f6ae40d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.567+0000 7f6aec3ff700 1 --2- 172.21.15.60:0/663009663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ae40b45b0 0x7f6ae40b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.567+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 >> 172.21.15.60:0/663009663 conn(0x7f6ae4072240 msgr2=0x7f6ae40b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:56.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.567+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 shutdown_connections 2023-07-13T22:21:56.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.567+0000 7f6aec3ff700 1 -- 172.21.15.60:0/663009663 wait complete. 2023-07-13T22:21:56.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:56.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.790+0000 7f129ffff700 1 Processor -- start 2023-07-13T22:21:56.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.790+0000 7f129ffff700 1 -- start start 2023-07-13T22:21:56.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.791+0000 7f129ffff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f12980ca900 0x7f12980703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.791+0000 7f129ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1298070920 0x7f129806c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.791+0000 7f129ffff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1298070d20 con 0x7f1298159260 2023-07-13T22:21:56.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.791+0000 7f129ffff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12980cada0 con 0x7f1298070920 2023-07-13T22:21:56.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.791+0000 7f129ffff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12980cf640 con 0x7f12980ca900 2023-07-13T22:21:56.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.792+0000 7f129e59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1298070920 0x7f129806c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.792+0000 7f129e59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1298070920 0x7f129806c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50720/0 (socket says 172.21.15.60:50720) 2023-07-13T22:21:56.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.792+0000 7f129e59c700 1 -- 172.21.15.60:0/2087328121 learned_addr learned my addr 172.21.15.60:0/2087328121 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:56.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.792+0000 7f129d59a700 1 --2- 172.21.15.60:0/2087328121 >> v2:172.21.15.105:3300/0 conn(0x7f12980ca900 0x7f12980703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.792+0000 7f129e59c700 1 -- 172.21.15.60:0/2087328121 >> v2:172.21.15.105:3300/0 conn(0x7f12980ca900 msgr2=0x7f12980703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:56.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.792+0000 7f129e59c700 1 --2- 172.21.15.60:0/2087328121 >> v2:172.21.15.105:3300/0 conn(0x7f12980ca900 0x7f12980703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.792+0000 7f129e59c700 1 -- 172.21.15.60:0/2087328121 >> v1:172.21.15.105:6789/0 conn(0x7f1298159260 legacy=0x7f12980cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.792+0000 7f129e59c700 1 -- 172.21.15.60:0/2087328121 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980ccf80 con 0x7f1298070920 2023-07-13T22:21:56.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.792+0000 7f129e59c700 1 --2- 172.21.15.60:0/2087328121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 0x7f129806c440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f12980ad6d0 tx=0x7f129000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70dc5a24e1d4b9b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:56.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129cd99700 1 -- 172.21.15.60:0/2087328121 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290010070 con 0x7f1298070920 2023-07-13T22:21:56.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129cd99700 1 -- 172.21.15.60:0/2087328121 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129000dd90 con 0x7f1298070920 2023-07-13T22:21:56.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129cd99700 1 -- 172.21.15.60:0/2087328121 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129000b310 con 0x7f1298070920 2023-07-13T22:21:56.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129ffff700 1 -- 172.21.15.60:0/2087328121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 msgr2=0x7f129806c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129ffff700 1 --2- 172.21.15.60:0/2087328121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 0x7f129806c440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f12980ad6d0 tx=0x7f129000d430 comp rx=0 tx=0).stop 2023-07-13T22:21:56.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129ffff700 1 -- 172.21.15.60:0/2087328121 shutdown_connections 2023-07-13T22:21:56.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129ffff700 1 --2- 172.21.15.60:0/2087328121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 0x7f129806c440 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129ffff700 1 --2- 172.21.15.60:0/2087328121 >> v2:172.21.15.105:3300/0 conn(0x7f12980ca900 0x7f12980703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129ffff700 1 -- 172.21.15.60:0/2087328121 >> 172.21.15.60:0/2087328121 conn(0x7f12980cbf50 msgr2=0x7f12980ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:56.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129ffff700 1 -- 172.21.15.60:0/2087328121 shutdown_connections 2023-07-13T22:21:56.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.793+0000 7f129ffff700 1 -- 172.21.15.60:0/2087328121 wait complete. 2023-07-13T22:21:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.795+0000 7f129ffff700 1 Processor -- start 2023-07-13T22:21:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.795+0000 7f129ffff700 1 -- start start 2023-07-13T22:21:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.796+0000 7f129ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 0x7f12980c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.796+0000 7f129ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12980ca900 0x7f12980c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.796+0000 7f129ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1298159260 0x7f1298150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.796+0000 7f129ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12980ab9c0 con 0x7f1298070920 2023-07-13T22:21:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.796+0000 7f129ffff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129815a070 con 0x7f12980ca900 2023-07-13T22:21:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.796+0000 7f129ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12980c6860 con 0x7f1298159260 2023-07-13T22:21:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.796+0000 7f129dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 0x7f12980c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.796+0000 7f129dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 0x7f12980c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50726/0 (socket says 172.21.15.60:50726) 2023-07-13T22:21:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129dd9b700 1 -- 172.21.15.60:0/1849684293 learned_addr learned my addr 172.21.15.60:0/1849684293 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129d59a700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12980ca900 0x7f12980c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129e59c700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1298159260 0x7f1298150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129dd9b700 1 -- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1298159260 msgr2=0x7f1298150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129dd9b700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1298159260 0x7f1298150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129dd9b700 1 -- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12980ca900 msgr2=0x7f12980c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129dd9b700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12980ca900 0x7f12980c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129dd9b700 1 -- 172.21.15.60:0/1849684293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129000bd40 con 0x7f1298070920 2023-07-13T22:21:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129e59c700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1298159260 0x7f1298150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129d59a700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12980ca900 0x7f12980c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129dd9b700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 0x7f12980c7b20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f129400c4e0 tx=0x7f129400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f128e7fc700 1 -- 172.21.15.60:0/1849684293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12940052d0 con 0x7f1298070920 2023-07-13T22:21:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f128e7fc700 1 -- 172.21.15.60:0/1849684293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1294005430 con 0x7f1298070920 2023-07-13T22:21:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f128e7fc700 1 -- 172.21.15.60:0/1849684293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12940038c0 con 0x7f1298070920 2023-07-13T22:21:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.797+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12980c6b00 con 0x7f1298070920 2023-07-13T22:21:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.798+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12980abc30 con 0x7f1298070920 2023-07-13T22:21:56.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.799+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129804e210 con 0x7f1298070920 2023-07-13T22:21:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.799+0000 7f128e7fc700 1 -- 172.21.15.60:0/1849684293 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1294003a20 con 0x7f1298070920 2023-07-13T22:21:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.800+0000 7f128e7fc700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f127c06ace0 0x7f127c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.800+0000 7f128e7fc700 1 -- 172.21.15.60:0/1849684293 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1294088a80 con 0x7f1298070920 2023-07-13T22:21:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.801+0000 7f129d59a700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f127c06ace0 0x7f127c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.801+0000 7f129d59a700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f127c06ace0 0x7f127c06d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1298153dc0 tx=0x7f1288008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:56.805+0000 7f128e7fc700 1 -- 172.21.15.60:0/1849684293 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1294053a50 con 0x7f1298070920 2023-07-13T22:21:56.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.009+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12980cf050 con 0x7f1298070920 2023-07-13T22:21:56.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.009+0000 7f128e7fc700 1 -- 172.21.15.60:0/1849684293 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1294017020 con 0x7f1298070920 2023-07-13T22:21:56.994 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f127c06ace0 msgr2=0x7f127c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f127c06ace0 0x7f127c06d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1298153dc0 tx=0x7f1288008040 comp rx=0 tx=0).stop 2023-07-13T22:21:56.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 msgr2=0x7f12980c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 0x7f12980c7b20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f129400c4e0 tx=0x7f129400c8a0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 shutdown_connections 2023-07-13T22:21:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f127c06ace0 0x7f127c06d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1298070920 0x7f12980c7b20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1298159260 0x7f1298150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 --2- 172.21.15.60:0/1849684293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12980ca900 0x7f12980c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.011+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 >> 172.21.15.60:0/1849684293 conn(0x7f12980cbf50 msgr2=0x7f129806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.012+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 shutdown_connections 2023-07-13T22:21:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.012+0000 7f129ffff700 1 -- 172.21.15.60:0/1849684293 wait complete. 2023-07-13T22:21:57.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:21:57.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.233+0000 7f4d883ff700 1 Processor -- start 2023-07-13T22:21:57.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.233+0000 7f4d883ff700 1 -- start start 2023-07-13T22:21:57.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.235+0000 7f4d883ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d800b8400 0x7f4d800cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.235+0000 7f4d883ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800cfca0 0x7f4d800d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.235+0000 7f4d883ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d800d0ac0 con 0x7f4d800d3750 2023-07-13T22:21:57.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.235+0000 7f4d883ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800b87d0 con 0x7f4d800b8400 2023-07-13T22:21:57.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.235+0000 7f4d883ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800b4eb0 con 0x7f4d800cfca0 2023-07-13T22:21:57.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.235+0000 7f4d8699c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800cfca0 0x7f4d800d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.235+0000 7f4d8619b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d800b8400 0x7f4d800cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.235+0000 7f4d8599a700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4d800d3750 0x7f4d800cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:39328/0 (socket says 172.21.15.60:39328) 2023-07-13T22:21:57.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.235+0000 7f4d8599a700 1 -- 172.21.15.60:0/2560389705 learned_addr learned my addr 172.21.15.60:0/2560389705 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:57.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d85199700 1 -- 172.21.15.60:0/2560389705 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402215816 0 0) 0x7f4d800d0ac0 con 0x7f4d800d3750 2023-07-13T22:21:57.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d85199700 1 -- 172.21.15.60:0/2560389705 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d6c0035b0 con 0x7f4d800d3750 2023-07-13T22:21:57.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d8699c700 1 -- 172.21.15.60:0/2560389705 >> v2:172.21.15.105:3300/0 conn(0x7f4d800b8400 msgr2=0x7f4d800cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d8699c700 1 --2- 172.21.15.60:0/2560389705 >> v2:172.21.15.105:3300/0 conn(0x7f4d800b8400 0x7f4d800cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d8699c700 1 -- 172.21.15.60:0/2560389705 >> v1:172.21.15.105:6789/0 conn(0x7f4d800d3750 legacy=0x7f4d800cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d8699c700 1 -- 172.21.15.60:0/2560389705 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800b5190 con 0x7f4d800cfca0 2023-07-13T22:21:57.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d8619b700 1 --2- 172.21.15.60:0/2560389705 >> v2:172.21.15.105:3300/0 conn(0x7f4d800b8400 0x7f4d800cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:57.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d8699c700 1 --2- 172.21.15.60:0/2560389705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800cfca0 0x7f4d800d0f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009d90 tx=0x7f4d7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de914ac42f459f17 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d85199700 1 -- 172.21.15.60:0/2560389705 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800e070 con 0x7f4d800cfca0 2023-07-13T22:21:57.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d85199700 1 -- 172.21.15.60:0/2560389705 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7800bb20 con 0x7f4d800cfca0 2023-07-13T22:21:57.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.236+0000 7f4d85199700 1 -- 172.21.15.60:0/2560389705 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78012430 con 0x7f4d800cfca0 2023-07-13T22:21:57.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.237+0000 7f4d883ff700 1 -- 172.21.15.60:0/2560389705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800cfca0 msgr2=0x7f4d800d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.237+0000 7f4d883ff700 1 --2- 172.21.15.60:0/2560389705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800cfca0 0x7f4d800d0f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009d90 tx=0x7f4d7800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:57.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.237+0000 7f4d883ff700 1 -- 172.21.15.60:0/2560389705 shutdown_connections 2023-07-13T22:21:57.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.237+0000 7f4d883ff700 1 --2- 172.21.15.60:0/2560389705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800cfca0 0x7f4d800d0f50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.237+0000 7f4d883ff700 1 --2- 172.21.15.60:0/2560389705 >> v2:172.21.15.105:3300/0 conn(0x7f4d800b8400 0x7f4d800cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.237+0000 7f4d883ff700 1 -- 172.21.15.60:0/2560389705 >> 172.21.15.60:0/2560389705 conn(0x7f4d8006b390 msgr2=0x7f4d8006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.237+0000 7f4d883ff700 1 -- 172.21.15.60:0/2560389705 shutdown_connections 2023-07-13T22:21:57.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.237+0000 7f4d883ff700 1 -- 172.21.15.60:0/2560389705 wait complete. 2023-07-13T22:21:57.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.239+0000 7f4d883ff700 1 Processor -- start 2023-07-13T22:21:57.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.239+0000 7f4d883ff700 1 -- start start 2023-07-13T22:21:57.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d883ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800b8400 0x7f4d80156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d883ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d800cfca0 0x7f4d80158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d883ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800d3750 0x7f4d80160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d883ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d801571b0 con 0x7f4d800cfca0 2023-07-13T22:21:57.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d883ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80157310 con 0x7f4d800b8400 2023-07-13T22:21:57.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d883ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80158580 con 0x7f4d800d3750 2023-07-13T22:21:57.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d8599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d800cfca0 0x7f4d80158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d8599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d800cfca0 0x7f4d80158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50736/0 (socket says 172.21.15.60:50736) 2023-07-13T22:21:57.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d8699c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800d3750 0x7f4d80160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d8619b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800b8400 0x7f4d80156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.240+0000 7f4d8599a700 1 -- 172.21.15.60:0/2032654489 learned_addr learned my addr 172.21.15.60:0/2032654489 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:57.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d8599a700 1 -- 172.21.15.60:0/2032654489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800d3750 msgr2=0x7f4d80160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d8599a700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800d3750 0x7f4d80160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d8599a700 1 -- 172.21.15.60:0/2032654489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800b8400 msgr2=0x7f4d80156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d8599a700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800b8400 0x7f4d80156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d8599a700 1 -- 172.21.15.60:0/2032654489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c0094a0 con 0x7f4d800cfca0 2023-07-13T22:21:57.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d8619b700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800b8400 0x7f4d80156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:57.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d8699c700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800d3750 0x7f4d80160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:57.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d8599a700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d800cfca0 0x7f4d80158040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4d70005910 tx=0x7f4d7000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d76ffd700 1 -- 172.21.15.60:0/2032654489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70005d70 con 0x7f4d800cfca0 2023-07-13T22:21:57.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d76ffd700 1 -- 172.21.15.60:0/2032654489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d700070d0 con 0x7f4d800cfca0 2023-07-13T22:21:57.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d76ffd700 1 -- 172.21.15.60:0/2032654489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70007460 con 0x7f4d800cfca0 2023-07-13T22:21:57.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.241+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d78009a70 con 0x7f4d800cfca0 2023-07-13T22:21:57.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.242+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d800c97c0 con 0x7f4d800cfca0 2023-07-13T22:21:57.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.242+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d800ccf50 con 0x7f4d800cfca0 2023-07-13T22:21:57.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.244+0000 7f4d76ffd700 1 -- 172.21.15.60:0/2032654489 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d700075c0 con 0x7f4d800cfca0 2023-07-13T22:21:57.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.244+0000 7f4d76ffd700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6406ace0 0x7f4d6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.245+0000 7f4d76ffd700 1 -- 172.21.15.60:0/2032654489 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4d700889a0 con 0x7f4d800cfca0 2023-07-13T22:21:57.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.245+0000 7f4d8619b700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6406ace0 0x7f4d6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.245+0000 7f4d8619b700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6406ace0 0x7f4d6406d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c011380 tx=0x7f4d7c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.249+0000 7f4d76ffd700 1 -- 172.21.15.60:0/2032654489 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4d70053a00 con 0x7f4d800cfca0 2023-07-13T22:21:57.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.466+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4d800d1b60 con 0x7f4d800cfca0 2023-07-13T22:21:57.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.467+0000 7f4d76ffd700 1 -- 172.21.15.60:0/2032654489 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4d70057020 con 0x7f4d800cfca0 2023-07-13T22:21:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6406ace0 msgr2=0x7f4d6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6406ace0 0x7f4d6406d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c011380 tx=0x7f4d7c009210 comp rx=0 tx=0).stop 2023-07-13T22:21:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d800cfca0 msgr2=0x7f4d80158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d800cfca0 0x7f4d80158040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4d70005910 tx=0x7f4d7000c9e0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 shutdown_connections 2023-07-13T22:21:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6406ace0 0x7f4d6406d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d800d3750 0x7f4d80160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d800b8400 0x7f4d80156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 --2- 172.21.15.60:0/2032654489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d800cfca0 0x7f4d80158040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 >> 172.21.15.60:0/2032654489 conn(0x7f4d8006b390 msgr2=0x7f4d800b7770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 shutdown_connections 2023-07-13T22:21:57.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.469+0000 7f4d883ff700 1 -- 172.21.15.60:0/2032654489 wait complete. 2023-07-13T22:21:57.456 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:21:57.471 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:21:57.472 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:21:57.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:21:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.743+0000 7f384a1ff700 1 Processor -- start 2023-07-13T22:21:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.744+0000 7f384a1ff700 1 -- start start 2023-07-13T22:21:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.745+0000 7f384a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 0x7f38440ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.745+0000 7f384a1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f38440d15c0 0x7f384415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.745+0000 7f384a1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38440d09d0 con 0x7f38440d35c0 2023-07-13T22:21:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.745+0000 7f384a1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38440d3b30 con 0x7f38440d15c0 2023-07-13T22:21:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.745+0000 7f384a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38440cda10 con 0x7f38440b8570 2023-07-13T22:21:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f3842ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f38440d35c0 0x7f38440d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34022/0 (socket says 172.21.15.60:34022) 2023-07-13T22:21:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f3842ffd700 1 -- 172.21.15.60:0/2744797101 learned_addr learned my addr 172.21.15.60:0/2744797101 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f38437fe700 1 --2- 172.21.15.60:0/2744797101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 0x7f38440ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f3843fff700 1 --2- 172.21.15.60:0/2744797101 >> v2:172.21.15.105:3300/0 conn(0x7f38440d15c0 0x7f384415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f38427fc700 1 -- 172.21.15.60:0/2744797101 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646958833 0 0) 0x7f38440d09d0 con 0x7f38440d35c0 2023-07-13T22:21:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f38427fc700 1 -- 172.21.15.60:0/2744797101 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f382c0035b0 con 0x7f38440d35c0 2023-07-13T22:21:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f38437fe700 1 -- 172.21.15.60:0/2744797101 >> v2:172.21.15.105:3300/0 conn(0x7f38440d15c0 msgr2=0x7f384415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f38437fe700 1 --2- 172.21.15.60:0/2744797101 >> v2:172.21.15.105:3300/0 conn(0x7f38440d15c0 0x7f384415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f38437fe700 1 -- 172.21.15.60:0/2744797101 >> v1:172.21.15.60:6789/0 conn(0x7f38440d35c0 legacy=0x7f38440d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.746+0000 7f38437fe700 1 -- 172.21.15.60:0/2744797101 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38440cd450 con 0x7f38440b8570 2023-07-13T22:21:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.747+0000 7f38437fe700 1 --2- 172.21.15.60:0/2744797101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 0x7f38440ceeb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3834009820 tx=0x7f3834009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c3e83a2277851c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.747+0000 7f38427fc700 1 -- 172.21.15.60:0/2744797101 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834005e30 con 0x7f38440b8570 2023-07-13T22:21:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.747+0000 7f38427fc700 1 -- 172.21.15.60:0/2744797101 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834006030 con 0x7f38440b8570 2023-07-13T22:21:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.747+0000 7f38427fc700 1 -- 172.21.15.60:0/2744797101 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340063c0 con 0x7f38440b8570 2023-07-13T22:21:57.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.747+0000 7f384a1ff700 1 -- 172.21.15.60:0/2744797101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 msgr2=0x7f38440ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.747+0000 7f384a1ff700 1 --2- 172.21.15.60:0/2744797101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 0x7f38440ceeb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3834009820 tx=0x7f3834009b30 comp rx=0 tx=0).stop 2023-07-13T22:21:57.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.748+0000 7f384a1ff700 1 -- 172.21.15.60:0/2744797101 shutdown_connections 2023-07-13T22:21:57.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.748+0000 7f384a1ff700 1 --2- 172.21.15.60:0/2744797101 >> v2:172.21.15.105:3300/0 conn(0x7f38440d15c0 0x7f384415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.748+0000 7f384a1ff700 1 --2- 172.21.15.60:0/2744797101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 0x7f38440ceeb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.748+0000 7f384a1ff700 1 -- 172.21.15.60:0/2744797101 >> 172.21.15.60:0/2744797101 conn(0x7f3844071600 msgr2=0x7f38440719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.748+0000 7f384a1ff700 1 -- 172.21.15.60:0/2744797101 shutdown_connections 2023-07-13T22:21:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.748+0000 7f384a1ff700 1 -- 172.21.15.60:0/2744797101 wait complete. 2023-07-13T22:21:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.750+0000 7f384a1ff700 1 Processor -- start 2023-07-13T22:21:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.750+0000 7f384a1ff700 1 -- start start 2023-07-13T22:21:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f384a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 0x7f38440cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f384a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38440d15c0 0x7f38440c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f384a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38440d35c0 0x7f38441f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f384a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38440ca780 con 0x7f38440d35c0 2023-07-13T22:21:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f384a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38440caa60 con 0x7f38440d15c0 2023-07-13T22:21:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f384a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38440cad40 con 0x7f38440b8570 2023-07-13T22:21:57.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f3843fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38440d35c0 0x7f38441f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f38437fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 0x7f38440cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f3843fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38440d35c0 0x7f38441f1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50750/0 (socket says 172.21.15.60:50750) 2023-07-13T22:21:57.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f3843fff700 1 -- 172.21.15.60:0/703228377 learned_addr learned my addr 172.21.15.60:0/703228377 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:57.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.751+0000 7f3842ffd700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38440d15c0 0x7f38440c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f3843fff700 1 -- 172.21.15.60:0/703228377 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 msgr2=0x7f38440cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f3843fff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 0x7f38440cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f3843fff700 1 -- 172.21.15.60:0/703228377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38440d15c0 msgr2=0x7f38440c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f3843fff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38440d15c0 0x7f38440c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f3843fff700 1 -- 172.21.15.60:0/703228377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3834009430 con 0x7f38440d35c0 2023-07-13T22:21:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f3843fff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38440d35c0 0x7f38441f1920 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f383c00efc0 tx=0x7f383c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f382bfff700 1 -- 172.21.15.60:0/703228377 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c00cc90 con 0x7f38440d35c0 2023-07-13T22:21:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f382bfff700 1 -- 172.21.15.60:0/703228377 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c00f040 con 0x7f38440d35c0 2023-07-13T22:21:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f382bfff700 1 -- 172.21.15.60:0/703228377 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c0146c0 con 0x7f38440d35c0 2023-07-13T22:21:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38440cb020 con 0x7f38440d35c0 2023-07-13T22:21:57.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.752+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38440c9100 con 0x7f38440d35c0 2023-07-13T22:21:57.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.753+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384404e280 con 0x7f38440d35c0 2023-07-13T22:21:57.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.754+0000 7f382bfff700 1 -- 172.21.15.60:0/703228377 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f383c014820 con 0x7f38440d35c0 2023-07-13T22:21:57.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.755+0000 7f382bfff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f383006ad10 0x7f383006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.755+0000 7f382bfff700 1 -- 172.21.15.60:0/703228377 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f383c08a090 con 0x7f38440d35c0 2023-07-13T22:21:57.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.756+0000 7f38437fe700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f383006ad10 0x7f383006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.756+0000 7f38437fe700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f383006ad10 0x7f383006d1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3834000c00 tx=0x7f3834008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.760+0000 7f382bfff700 1 -- 172.21.15.60:0/703228377 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f383c0550f0 con 0x7f38440d35c0 2023-07-13T22:21:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.968+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f384415eeb0 con 0x7f38440d35c0 2023-07-13T22:21:57.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.969+0000 7f382bfff700 1 -- 172.21.15.60:0/703228377 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f383c009ca0 con 0x7f38440d35c0 2023-07-13T22:21:57.954 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:21:57.954 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:21:42.683944+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:21:57.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f383006ad10 msgr2=0x7f383006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f383006ad10 0x7f383006d1c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3834000c00 tx=0x7f3834008070 comp rx=0 tx=0).stop 2023-07-13T22:21:57.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38440d35c0 msgr2=0x7f38441f1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38440d35c0 0x7f38441f1920 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f383c00efc0 tx=0x7f383c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 shutdown_connections 2023-07-13T22:21:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f383006ad10 0x7f383006d1c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38440d35c0 0x7f38441f1920 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38440b8570 0x7f38440cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 --2- 172.21.15.60:0/703228377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38440d15c0 0x7f38440c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.971+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 >> 172.21.15.60:0/703228377 conn(0x7f3844071600 msgr2=0x7f384415f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.972+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 shutdown_connections 2023-07-13T22:21:57.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:57.972+0000 7f384a1ff700 1 -- 172.21.15.60:0/703228377 wait complete. 2023-07-13T22:21:57.974 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:21:57.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:21:58.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.196+0000 7fd55ebff700 1 Processor -- start 2023-07-13T22:21:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.196+0000 7fd55ebff700 1 -- start start 2023-07-13T22:21:58.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd55ebff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5580b8200 0x7fd5580cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd55ebff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd5580cede0 0x7fd5580cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd55ebff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5580d1870 con 0x7fd5580b1ac0 2023-07-13T22:21:58.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd55ebff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5580cf370 con 0x7fd5580cede0 2023-07-13T22:21:58.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd55ebff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580d12b0 con 0x7fd5580b8200 2023-07-13T22:21:58.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd557fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd5580b1ac0 0x7fd55815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34032/0 (socket says 172.21.15.60:34032) 2023-07-13T22:21:58.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd557fff700 1 -- 172.21.15.60:0/3822292080 learned_addr learned my addr 172.21.15.60:0/3822292080 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:58.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd55d19c700 1 --2- 172.21.15.60:0/3822292080 >> v2:172.21.15.60:3300/0 conn(0x7fd5580cede0 0x7fd5580cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd55c99b700 1 --2- 172.21.15.60:0/3822292080 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5580b8200 0x7fd5580cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd5577fe700 1 -- 172.21.15.60:0/3822292080 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524588241 0 0) 0x7fd5580d1870 con 0x7fd5580b1ac0 2023-07-13T22:21:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.198+0000 7fd5577fe700 1 -- 172.21.15.60:0/3822292080 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5480035b0 con 0x7fd5580b1ac0 2023-07-13T22:21:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.199+0000 7fd55d19c700 1 -- 172.21.15.60:0/3822292080 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5580b8200 msgr2=0x7fd5580cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.199+0000 7fd55d19c700 1 --2- 172.21.15.60:0/3822292080 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5580b8200 0x7fd5580cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.199+0000 7fd55d19c700 1 -- 172.21.15.60:0/3822292080 >> v1:172.21.15.60:6789/0 conn(0x7fd5580b1ac0 legacy=0x7fd55815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.199+0000 7fd55d19c700 1 -- 172.21.15.60:0/3822292080 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580d1590 con 0x7fd5580cede0 2023-07-13T22:21:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.199+0000 7fd55d19c700 1 --2- 172.21.15.60:0/3822292080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580cede0 0x7fd5580cfb80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd550009d90 tx=0x7fd55000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a052cc20ba0f9e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.199+0000 7fd5577fe700 1 -- 172.21.15.60:0/3822292080 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000e070 con 0x7fd5580cede0 2023-07-13T22:21:58.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.199+0000 7fd5577fe700 1 -- 172.21.15.60:0/3822292080 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55000bb20 con 0x7fd5580cede0 2023-07-13T22:21:58.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.199+0000 7fd5577fe700 1 -- 172.21.15.60:0/3822292080 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550012430 con 0x7fd5580cede0 2023-07-13T22:21:58.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.200+0000 7fd55ebff700 1 -- 172.21.15.60:0/3822292080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580cede0 msgr2=0x7fd5580cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.200+0000 7fd55ebff700 1 --2- 172.21.15.60:0/3822292080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580cede0 0x7fd5580cfb80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd550009d90 tx=0x7fd55000b330 comp rx=0 tx=0).stop 2023-07-13T22:21:58.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.200+0000 7fd55ebff700 1 -- 172.21.15.60:0/3822292080 shutdown_connections 2023-07-13T22:21:58.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.200+0000 7fd55ebff700 1 --2- 172.21.15.60:0/3822292080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580cede0 0x7fd5580cfb80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.200+0000 7fd55ebff700 1 --2- 172.21.15.60:0/3822292080 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5580b8200 0x7fd5580cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.200+0000 7fd55ebff700 1 -- 172.21.15.60:0/3822292080 >> 172.21.15.60:0/3822292080 conn(0x7fd55806b2f0 msgr2=0x7fd55806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.200+0000 7fd55ebff700 1 -- 172.21.15.60:0/3822292080 shutdown_connections 2023-07-13T22:21:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.200+0000 7fd55ebff700 1 -- 172.21.15.60:0/3822292080 wait complete. 2023-07-13T22:21:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.201+0000 7fd55ebff700 1 Processor -- start 2023-07-13T22:21:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.202+0000 7fd55ebff700 1 -- start start 2023-07-13T22:21:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55ebff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580b1ac0 0x7fd5580c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55ebff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5580b8200 0x7fd5580c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55ebff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5580cede0 0x7fd5580c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55ebff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580c7f50 con 0x7fd5580b1ac0 2023-07-13T22:21:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55ebff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580c8ee0 con 0x7fd5580b8200 2023-07-13T22:21:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55ebff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580c72d0 con 0x7fd5580cede0 2023-07-13T22:21:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55c99b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580b1ac0 0x7fd5580c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55c99b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580b1ac0 0x7fd5580c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50770/0 (socket says 172.21.15.60:50770) 2023-07-13T22:21:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55c99b700 1 -- 172.21.15.60:0/112754808 learned_addr learned my addr 172.21.15.60:0/112754808 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55c99b700 1 -- 172.21.15.60:0/112754808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5580cede0 msgr2=0x7fd5580c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55c99b700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5580cede0 0x7fd5580c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55c99b700 1 -- 172.21.15.60:0/112754808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5580b8200 msgr2=0x7fd5580c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd557fff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5580b8200 0x7fd5580c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55c99b700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5580b8200 0x7fd5580c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd55c99b700 1 -- 172.21.15.60:0/112754808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd550009a70 con 0x7fd5580b1ac0 2023-07-13T22:21:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.203+0000 7fd557fff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5580b8200 0x7fd5580c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.204+0000 7fd55c99b700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580b1ac0 0x7fd5580c89a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00e9b0 tx=0x7fd54c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.204+0000 7fd5557fa700 1 -- 172.21.15.60:0/112754808 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c009710 con 0x7fd5580b1ac0 2023-07-13T22:21:58.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.204+0000 7fd5557fa700 1 -- 172.21.15.60:0/112754808 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c00c850 con 0x7fd5580b1ac0 2023-07-13T22:21:58.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.204+0000 7fd5557fa700 1 -- 172.21.15.60:0/112754808 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00cbc0 con 0x7fd5580b1ac0 2023-07-13T22:21:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.204+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5580c75b0 con 0x7fd5580b1ac0 2023-07-13T22:21:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.204+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5580c7b70 con 0x7fd5580b1ac0 2023-07-13T22:21:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.205+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd558066700 con 0x7fd5580b1ac0 2023-07-13T22:21:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.206+0000 7fd5557fa700 1 -- 172.21.15.60:0/112754808 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd54c00cd20 con 0x7fd5580b1ac0 2023-07-13T22:21:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.207+0000 7fd5557fa700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd53c06ace0 0x7fd53c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.207+0000 7fd5557fa700 1 -- 172.21.15.60:0/112754808 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd54c0896d0 con 0x7fd5580b1ac0 2023-07-13T22:21:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.207+0000 7fd557fff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd53c06ace0 0x7fd53c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.208+0000 7fd557fff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd53c06ace0 0x7fd53c06d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd558149940 tx=0x7fd544009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.211+0000 7fd5557fa700 1 -- 172.21.15.60:0/112754808 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd54c058130 con 0x7fd5580b1ac0 2023-07-13T22:21:58.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.418+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd55804e210 con 0x7fd5580b1ac0 2023-07-13T22:21:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.418+0000 7fd5557fa700 1 -- 172.21.15.60:0/112754808 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd54c057cc0 con 0x7fd5580b1ac0 2023-07-13T22:21:58.403 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:21:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd53c06ace0 msgr2=0x7fd53c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd53c06ace0 0x7fd53c06d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd558149940 tx=0x7fd544009040 comp rx=0 tx=0).stop 2023-07-13T22:21:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580b1ac0 msgr2=0x7fd5580c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580b1ac0 0x7fd5580c89a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00e9b0 tx=0x7fd54c00ed70 comp rx=0 tx=0).stop 2023-07-13T22:21:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 shutdown_connections 2023-07-13T22:21:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd53c06ace0 0x7fd53c06d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5580b1ac0 0x7fd5580c89a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5580cede0 0x7fd5580c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 --2- 172.21.15.60:0/112754808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5580b8200 0x7fd5580c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 >> 172.21.15.60:0/112754808 conn(0x7fd55806b2f0 msgr2=0x7fd5580b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 shutdown_connections 2023-07-13T22:21:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.421+0000 7fd55ebff700 1 -- 172.21.15.60:0/112754808 wait complete. 2023-07-13T22:21:58.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:21:58.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.648+0000 7f24d67ff700 1 Processor -- start 2023-07-13T22:21:58.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.648+0000 7f24d67ff700 1 -- start start 2023-07-13T22:21:58.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.650+0000 7f24d67ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f24d00c90e0 0x7f24d0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.650+0000 7f24d67ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d006d030 con 0x7f24d0158930 2023-07-13T22:21:58.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.650+0000 7f24d67ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d006a870 con 0x7f24d006a410 2023-07-13T22:21:58.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.650+0000 7f24d67ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d006ca70 con 0x7f24d00c90e0 2023-07-13T22:21:58.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.650+0000 7f24cffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f24d0158930 0x7f24d015c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34038/0 (socket says 172.21.15.60:34038) 2023-07-13T22:21:58.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.650+0000 7f24cffff700 1 -- 172.21.15.60:0/774724832 learned_addr learned my addr 172.21.15.60:0/774724832 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:58.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.650+0000 7f24cf7fe700 1 --2- 172.21.15.60:0/774724832 >> v2:172.21.15.105:3300/0 conn(0x7f24d00c90e0 0x7f24d0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.650+0000 7f24ceffd700 1 -- 172.21.15.60:0/774724832 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055571423 0 0) 0x7f24d006d030 con 0x7f24d0158930 2023-07-13T22:21:58.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.650+0000 7f24ceffd700 1 -- 172.21.15.60:0/774724832 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b80035b0 con 0x7f24d0158930 2023-07-13T22:21:58.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24ceffd700 1 -- 172.21.15.60:0/774724832 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933543287 0 0) 0x7f24d006a870 con 0x7f24d006a410 2023-07-13T22:21:58.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24ceffd700 1 -- 172.21.15.60:0/774724832 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d006d030 con 0x7f24d006a410 2023-07-13T22:21:58.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24cf7fe700 1 -- 172.21.15.60:0/774724832 >> v1:172.21.15.105:6789/0 conn(0x7f24d006a410 legacy=0x7f24d006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24cf7fe700 1 -- 172.21.15.60:0/774724832 >> v1:172.21.15.60:6789/0 conn(0x7f24d0158930 legacy=0x7f24d015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24ceffd700 1 -- 172.21.15.60:0/774724832 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 443857890 0 0) 0x7f24b80035b0 con 0x7f24d0158930 2023-07-13T22:21:58.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24cf7fe700 1 -- 172.21.15.60:0/774724832 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00adc20 con 0x7f24d00c90e0 2023-07-13T22:21:58.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24cf7fe700 1 --2- 172.21.15.60:0/774724832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d00c90e0 0x7f24d0069ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009840 tx=0x7f24c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6224a975218305f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24ceffd700 1 -- 172.21.15.60:0/774724832 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0001670 con 0x7f24d00c90e0 2023-07-13T22:21:58.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24ceffd700 1 -- 172.21.15.60:0/774724832 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c4004030 con 0x7f24d00c90e0 2023-07-13T22:21:58.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.651+0000 7f24ceffd700 1 -- 172.21.15.60:0/774724832 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c40043c0 con 0x7f24d00c90e0 2023-07-13T22:21:58.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.652+0000 7f24d67ff700 1 -- 172.21.15.60:0/774724832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d00c90e0 msgr2=0x7f24d0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.652+0000 7f24d67ff700 1 --2- 172.21.15.60:0/774724832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d00c90e0 0x7f24d0069ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009840 tx=0x7f24c4009b50 comp rx=0 tx=0).stop 2023-07-13T22:21:58.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.652+0000 7f24d67ff700 1 -- 172.21.15.60:0/774724832 shutdown_connections 2023-07-13T22:21:58.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.652+0000 7f24d67ff700 1 --2- 172.21.15.60:0/774724832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d00c90e0 0x7f24d0069ed0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.652+0000 7f24d67ff700 1 -- 172.21.15.60:0/774724832 >> 172.21.15.60:0/774724832 conn(0x7f24d00cbf50 msgr2=0x7f24d00ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.652+0000 7f24d67ff700 1 -- 172.21.15.60:0/774724832 shutdown_connections 2023-07-13T22:21:58.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.652+0000 7f24d67ff700 1 -- 172.21.15.60:0/774724832 wait complete. 2023-07-13T22:21:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.654+0000 7f24d67ff700 1 Processor -- start 2023-07-13T22:21:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.654+0000 7f24d67ff700 1 -- start start 2023-07-13T22:21:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24d67ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24d006a410 0x7f24d0149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24d67ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24d00c90e0 0x7f24d00ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24d67ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d0158930 0x7f24d00b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24d67ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0158550 con 0x7f24d006a410 2023-07-13T22:21:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24d67ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00b2c10 con 0x7f24d0158930 2023-07-13T22:21:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24d67ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00ae040 con 0x7f24d00c90e0 2023-07-13T22:21:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24cffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24d006a410 0x7f24d0149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24cffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24d006a410 0x7f24d0149610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50772/0 (socket says 172.21.15.60:50772) 2023-07-13T22:21:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24cffff700 1 -- 172.21.15.60:0/1160173041 learned_addr learned my addr 172.21.15.60:0/1160173041 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24cf7fe700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24d00c90e0 0x7f24d00ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24d4d9c700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d0158930 0x7f24d00b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24cffff700 1 -- 172.21.15.60:0/1160173041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24d00c90e0 msgr2=0x7f24d00ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24cffff700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24d00c90e0 0x7f24d00ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24cffff700 1 -- 172.21.15.60:0/1160173041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d0158930 msgr2=0x7f24d00b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24cffff700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d0158930 0x7f24d00b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.655+0000 7f24cffff700 1 -- 172.21.15.60:0/1160173041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c40094a0 con 0x7f24d006a410 2023-07-13T22:21:58.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.656+0000 7f24cffff700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24d006a410 0x7f24d0149610 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c530 tx=0x7f24c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.656+0000 7f24ccff9700 1 -- 172.21.15.60:0/1160173041 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004c80 con 0x7f24d006a410 2023-07-13T22:21:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.656+0000 7f24ccff9700 1 -- 172.21.15.60:0/1160173041 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c0004e00 con 0x7f24d006a410 2023-07-13T22:21:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.656+0000 7f24ccff9700 1 -- 172.21.15.60:0/1160173041 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0007310 con 0x7f24d006a410 2023-07-13T22:21:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.656+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d00b2650 con 0x7f24d006a410 2023-07-13T22:21:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.656+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d00ae2e0 con 0x7f24d006a410 2023-07-13T22:21:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.657+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d00adea0 con 0x7f24d006a410 2023-07-13T22:21:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.658+0000 7f24ccff9700 1 -- 172.21.15.60:0/1160173041 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f24c0007470 con 0x7f24d006a410 2023-07-13T22:21:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.659+0000 7f24ccff9700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24bc06ace0 0x7f24bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.659+0000 7f24ccff9700 1 -- 172.21.15.60:0/1160173041 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f24c00892a0 con 0x7f24d006a410 2023-07-13T22:21:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.659+0000 7f24cf7fe700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24bc06ace0 0x7f24bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.660+0000 7f24cf7fe700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24bc06ace0 0x7f24bc06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f24c400a010 tx=0x7f24c4013040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.664+0000 7f24ccff9700 1 -- 172.21.15.60:0/1160173041 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f24d00adea0 con 0x7f24d006a410 2023-07-13T22:21:58.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.872+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24d00adea0 con 0x7f24d006a410 2023-07-13T22:21:58.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.873+0000 7f24ccff9700 1 -- 172.21.15.60:0/1160173041 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24d00adea0 con 0x7f24d006a410 2023-07-13T22:21:58.858 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:21:58.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24bc06ace0 msgr2=0x7f24bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24bc06ace0 0x7f24bc06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f24c400a010 tx=0x7f24c4013040 comp rx=0 tx=0).stop 2023-07-13T22:21:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24d006a410 msgr2=0x7f24d0149610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24d006a410 0x7f24d0149610 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c530 tx=0x7f24c000c8f0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 shutdown_connections 2023-07-13T22:21:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24bc06ace0 0x7f24bc06d190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24d006a410 0x7f24d0149610 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d0158930 0x7f24d00b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 --2- 172.21.15.60:0/1160173041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24d00c90e0 0x7f24d00ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.875+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 >> 172.21.15.60:0/1160173041 conn(0x7f24d00cbf50 msgr2=0x7f24d00abc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.876+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 shutdown_connections 2023-07-13T22:21:58.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:58.876+0000 7f24d67ff700 1 -- 172.21.15.60:0/1160173041 wait complete. 2023-07-13T22:21:58.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:21:59.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.099+0000 7fc3593ff700 1 Processor -- start 2023-07-13T22:21:59.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.099+0000 7fc3593ff700 1 -- start start 2023-07-13T22:21:59.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.100+0000 7fc3593ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc3540b8200 0x7fc3540cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.100+0000 7fc3593ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc3540cede0 0x7fc3540cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.100+0000 7fc3593ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3540d1870 con 0x7fc3540b1ac0 2023-07-13T22:21:59.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.100+0000 7fc3593ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3540cf370 con 0x7fc3540cede0 2023-07-13T22:21:59.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.100+0000 7fc3593ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3540d12b0 con 0x7fc3540b8200 2023-07-13T22:21:59.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc3527fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc3540b1ac0 0x7fc35415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34046/0 (socket says 172.21.15.60:34046) 2023-07-13T22:21:59.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc3537fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc3540cede0 0x7fc3540cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc3527fc700 1 -- 172.21.15.60:0/1894964068 learned_addr learned my addr 172.21.15.60:0/1894964068 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:21:59.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc352ffd700 1 --2- 172.21.15.60:0/1894964068 >> v2:172.21.15.105:3300/0 conn(0x7fc3540b8200 0x7fc3540cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc351ffb700 1 -- 172.21.15.60:0/1894964068 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995052717 0 0) 0x7fc3540d1870 con 0x7fc3540b1ac0 2023-07-13T22:21:59.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc351ffb700 1 -- 172.21.15.60:0/1894964068 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc33c0035b0 con 0x7fc3540b1ac0 2023-07-13T22:21:59.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc3537fe700 1 -- 172.21.15.60:0/1894964068 >> v2:172.21.15.105:3300/0 conn(0x7fc3540b8200 msgr2=0x7fc3540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:59.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc3537fe700 1 --2- 172.21.15.60:0/1894964068 >> v2:172.21.15.105:3300/0 conn(0x7fc3540b8200 0x7fc3540cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc3537fe700 1 -- 172.21.15.60:0/1894964068 >> v1:172.21.15.60:6789/0 conn(0x7fc3540b1ac0 legacy=0x7fc35415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc3537fe700 1 -- 172.21.15.60:0/1894964068 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540d1590 con 0x7fc3540cede0 2023-07-13T22:21:59.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.101+0000 7fc3537fe700 1 --2- 172.21.15.60:0/1894964068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540cede0 0x7fc3540cfb80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc3540b2660 tx=0x7fc34c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2c731543bf5e65 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:59.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc351ffb700 1 -- 172.21.15.60:0/1894964068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c00e070 con 0x7fc3540cede0 2023-07-13T22:21:59.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc351ffb700 1 -- 172.21.15.60:0/1894964068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c00bb20 con 0x7fc3540cede0 2023-07-13T22:21:59.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc351ffb700 1 -- 172.21.15.60:0/1894964068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c012430 con 0x7fc3540cede0 2023-07-13T22:21:59.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc3593ff700 1 -- 172.21.15.60:0/1894964068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540cede0 msgr2=0x7fc3540cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc3593ff700 1 --2- 172.21.15.60:0/1894964068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540cede0 0x7fc3540cfb80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc3540b2660 tx=0x7fc34c00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:59.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc3593ff700 1 -- 172.21.15.60:0/1894964068 shutdown_connections 2023-07-13T22:21:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc3593ff700 1 --2- 172.21.15.60:0/1894964068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540cede0 0x7fc3540cfb80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc3593ff700 1 --2- 172.21.15.60:0/1894964068 >> v2:172.21.15.105:3300/0 conn(0x7fc3540b8200 0x7fc3540cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc3593ff700 1 -- 172.21.15.60:0/1894964068 >> 172.21.15.60:0/1894964068 conn(0x7fc35406b2f0 msgr2=0x7fc35406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc3593ff700 1 -- 172.21.15.60:0/1894964068 shutdown_connections 2023-07-13T22:21:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.102+0000 7fc3593ff700 1 -- 172.21.15.60:0/1894964068 wait complete. 2023-07-13T22:21:59.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.104+0000 7fc3593ff700 1 Processor -- start 2023-07-13T22:21:59.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.104+0000 7fc3593ff700 1 -- start start 2023-07-13T22:21:59.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.105+0000 7fc3593ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540b1ac0 0x7fc3540c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.105+0000 7fc3593ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3540b8200 0x7fc3540cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.105+0000 7fc3593ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3540cede0 0x7fc3541e2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.105+0000 7fc3593ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540c71b0 con 0x7fc3540b1ac0 2023-07-13T22:21:59.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.105+0000 7fc3593ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540c7490 con 0x7fc3540cede0 2023-07-13T22:21:59.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.105+0000 7fc3593ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540cb7c0 con 0x7fc3540b8200 2023-07-13T22:21:59.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.105+0000 7fc352ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540b1ac0 0x7fc3540c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.105+0000 7fc352ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540b1ac0 0x7fc3540c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50784/0 (socket says 172.21.15.60:50784) 2023-07-13T22:21:59.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.105+0000 7fc352ffd700 1 -- 172.21.15.60:0/3517449829 learned_addr learned my addr 172.21.15.60:0/3517449829 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:59.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc352ffd700 1 -- 172.21.15.60:0/3517449829 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3540b8200 msgr2=0x7fc3540cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc352ffd700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3540b8200 0x7fc3540cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc352ffd700 1 -- 172.21.15.60:0/3517449829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3540cede0 msgr2=0x7fc3541e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc3537fe700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3540cede0 0x7fc3541e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc352ffd700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3540cede0 0x7fc3541e2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc352ffd700 1 -- 172.21.15.60:0/3517449829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c009a70 con 0x7fc3540b1ac0 2023-07-13T22:21:59.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc3537fe700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3540cede0 0x7fc3541e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:59.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc352ffd700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540b1ac0 0x7fc3540c90c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc34400e9b0 tx=0x7fc34400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:59.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc33b7fe700 1 -- 172.21.15.60:0/3517449829 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344009710 con 0x7fc3540b1ac0 2023-07-13T22:21:59.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc33b7fe700 1 -- 172.21.15.60:0/3517449829 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34400c850 con 0x7fc3540b1ac0 2023-07-13T22:21:59.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc33b7fe700 1 -- 172.21.15.60:0/3517449829 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34400cbc0 con 0x7fc3540b1ac0 2023-07-13T22:21:59.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.106+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3540cbaa0 con 0x7fc3540b1ac0 2023-07-13T22:21:59.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.107+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3540c9600 con 0x7fc3540b1ac0 2023-07-13T22:21:59.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.108+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3540cbcc0 con 0x7fc3540b1ac0 2023-07-13T22:21:59.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.108+0000 7fc33b7fe700 1 -- 172.21.15.60:0/3517449829 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc34400cd20 con 0x7fc3540b1ac0 2023-07-13T22:21:59.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.109+0000 7fc33b7fe700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34006ace0 0x7fc34006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.109+0000 7fc33b7fe700 1 -- 172.21.15.60:0/3517449829 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc3440895e0 con 0x7fc3540b1ac0 2023-07-13T22:21:59.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.110+0000 7fc3527fc700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34006ace0 0x7fc34006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.110+0000 7fc3527fc700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34006ace0 0x7fc34006d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3540b4310 tx=0x7fc3480079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:59.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.114+0000 7fc33b7fe700 1 -- 172.21.15.60:0/3517449829 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc3440580d0 con 0x7fc3540b1ac0 2023-07-13T22:21:59.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.319+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc35404e210 con 0x7fc3540b1ac0 2023-07-13T22:21:59.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.319+0000 7fc33b7fe700 1 -- 172.21.15.60:0/3517449829 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc344057c60 con 0x7fc3540b1ac0 2023-07-13T22:21:59.305 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:21:59.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34006ace0 msgr2=0x7fc34006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34006ace0 0x7fc34006d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3540b4310 tx=0x7fc3480079c0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540b1ac0 msgr2=0x7fc3540c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540b1ac0 0x7fc3540c90c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc34400e9b0 tx=0x7fc34400ed70 comp rx=0 tx=0).stop 2023-07-13T22:21:59.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 shutdown_connections 2023-07-13T22:21:59.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34006ace0 0x7fc34006d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3540b1ac0 0x7fc3540c90c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3540cede0 0x7fc3541e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 --2- 172.21.15.60:0/3517449829 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3540b8200 0x7fc3540cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 >> 172.21.15.60:0/3517449829 conn(0x7fc35406b2f0 msgr2=0x7fc3540b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:59.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 shutdown_connections 2023-07-13T22:21:59.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.322+0000 7fc3593ff700 1 -- 172.21.15.60:0/3517449829 wait complete. 2023-07-13T22:21:59.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:21:59.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.545+0000 7f607b5ff700 1 Processor -- start 2023-07-13T22:21:59.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.545+0000 7f607b5ff700 1 -- start start 2023-07-13T22:21:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.547+0000 7f607b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 0x7f60740cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.547+0000 7f607b5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60740cede0 0x7f60740cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.547+0000 7f607b5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60740d1870 con 0x7f60740b1ac0 2023-07-13T22:21:59.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.547+0000 7f607b5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60740cf370 con 0x7f60740cede0 2023-07-13T22:21:59.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.547+0000 7f607b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60740d12b0 con 0x7f60740b8200 2023-07-13T22:21:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.547+0000 7f6079b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60740cede0 0x7f60740cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f607939b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 0x7f60740cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f6079b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60740cede0 0x7f60740cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50790/0 (socket says 172.21.15.60:50790) 2023-07-13T22:21:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f6079b9c700 1 -- 172.21.15.60:0/580391071 learned_addr learned my addr 172.21.15.60:0/580391071 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f6079b9c700 1 -- 172.21.15.60:0/580391071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 msgr2=0x7f60740cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f6079b9c700 1 --2- 172.21.15.60:0/580391071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 0x7f60740cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f6079b9c700 1 -- 172.21.15.60:0/580391071 >> v1:172.21.15.105:6789/0 conn(0x7f60740b1ac0 legacy=0x7f607415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f6079b9c700 1 -- 172.21.15.60:0/580391071 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60740d1590 con 0x7f60740cede0 2023-07-13T22:21:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f606bfff700 1 -- 172.21.15.60:0/580391071 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121913497 0 0) 0x7f60740d1870 con 0x7f60740b1ac0 2023-07-13T22:21:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f6079b9c700 1 --2- 172.21.15.60:0/580391071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 0x7f60740cfb80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f606c009e50 tx=0x7f606c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d408964fb1e43c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.548+0000 7f606bfff700 1 -- 172.21.15.60:0/580391071 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c00e070 con 0x7f60740cede0 2023-07-13T22:21:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f606bfff700 1 -- 172.21.15.60:0/580391071 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c00b910 con 0x7f60740cede0 2023-07-13T22:21:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f606bfff700 1 -- 172.21.15.60:0/580391071 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c00bcc0 con 0x7f60740cede0 2023-07-13T22:21:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f607b5ff700 1 -- 172.21.15.60:0/580391071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 msgr2=0x7f60740cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f607b5ff700 1 --2- 172.21.15.60:0/580391071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 0x7f60740cfb80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f606c009e50 tx=0x7f606c00b330 comp rx=0 tx=0).stop 2023-07-13T22:21:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f607b5ff700 1 -- 172.21.15.60:0/580391071 shutdown_connections 2023-07-13T22:21:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f607b5ff700 1 --2- 172.21.15.60:0/580391071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 0x7f60740cfb80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f607b5ff700 1 --2- 172.21.15.60:0/580391071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 0x7f60740cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f607b5ff700 1 -- 172.21.15.60:0/580391071 >> 172.21.15.60:0/580391071 conn(0x7f607406b2f0 msgr2=0x7f607406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f607b5ff700 1 -- 172.21.15.60:0/580391071 shutdown_connections 2023-07-13T22:21:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.549+0000 7f607b5ff700 1 -- 172.21.15.60:0/580391071 wait complete. 2023-07-13T22:21:59.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.551+0000 7f607b5ff700 1 Processor -- start 2023-07-13T22:21:59.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.551+0000 7f607b5ff700 1 -- start start 2023-07-13T22:21:59.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.552+0000 7f607b5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60740b1ac0 0x7f60740cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.552+0000 7f607b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 0x7f60741e6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.552+0000 7f607b5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 0x7f60740c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.552+0000 7f607b5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60740c7740 con 0x7f60740cede0 2023-07-13T22:21:59.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.552+0000 7f607b5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60740cbc00 con 0x7f60740b1ac0 2023-07-13T22:21:59.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.552+0000 7f607b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60740c7180 con 0x7f60740b8200 2023-07-13T22:21:59.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.552+0000 7f6079b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 0x7f60740c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6079b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 0x7f60740c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50806/0 (socket says 172.21.15.60:50806) 2023-07-13T22:21:59.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6079b9c700 1 -- 172.21.15.60:0/3534805738 learned_addr learned my addr 172.21.15.60:0/3534805738 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:59.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f607939b700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60740b1ac0 0x7f60740cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6078b9a700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 0x7f60741e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6079b9c700 1 -- 172.21.15.60:0/3534805738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 msgr2=0x7f60741e6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6079b9c700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 0x7f60741e6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6079b9c700 1 -- 172.21.15.60:0/3534805738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60740b1ac0 msgr2=0x7f60740cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6079b9c700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60740b1ac0 0x7f60740cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6079b9c700 1 -- 172.21.15.60:0/3534805738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c009b30 con 0x7f60740cede0 2023-07-13T22:21:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6078b9a700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 0x7f60741e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.553+0000 7f6079b9c700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 0x7f60740c8c60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f606c004270 tx=0x7f606c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.554+0000 7f6069ffb700 1 -- 172.21.15.60:0/3534805738 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c004950 con 0x7f60740cede0 2023-07-13T22:21:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.554+0000 7f6069ffb700 1 -- 172.21.15.60:0/3534805738 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c0092e0 con 0x7f60740cede0 2023-07-13T22:21:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.554+0000 7f6069ffb700 1 -- 172.21.15.60:0/3534805738 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c01b650 con 0x7f60740cede0 2023-07-13T22:21:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.554+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60740c7460 con 0x7f60740cede0 2023-07-13T22:21:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.554+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60740cc5d0 con 0x7f60740cede0 2023-07-13T22:21:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.555+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607404e210 con 0x7f60740cede0 2023-07-13T22:21:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.556+0000 7f6069ffb700 1 -- 172.21.15.60:0/3534805738 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f606c01b7b0 con 0x7f60740cede0 2023-07-13T22:21:59.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.556+0000 7f6069ffb700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ad10 0x7f605806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.557+0000 7f6069ffb700 1 -- 172.21.15.60:0/3534805738 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f606c08aaa0 con 0x7f60740cede0 2023-07-13T22:21:59.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.557+0000 7f607939b700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ad10 0x7f605806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.557+0000 7f607939b700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ad10 0x7f605806d1c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6070009be0 tx=0x7f6070009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:59.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.561+0000 7f6069ffb700 1 -- 172.21.15.60:0/3534805738 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f606c055b00 con 0x7f60740cede0 2023-07-13T22:21:59.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.768+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60740cbd40 con 0x7f60740cede0 2023-07-13T22:21:59.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.769+0000 7f6069ffb700 1 -- 172.21.15.60:0/3534805738 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f606c059120 con 0x7f60740cede0 2023-07-13T22:21:59.754 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:21:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ad10 msgr2=0x7f605806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ad10 0x7f605806d1c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6070009be0 tx=0x7f6070009210 comp rx=0 tx=0).stop 2023-07-13T22:21:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 msgr2=0x7f60740c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 0x7f60740c8c60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f606c004270 tx=0x7f606c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 shutdown_connections 2023-07-13T22:21:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ad10 0x7f605806d1c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60740cede0 0x7f60740c8c60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60740b1ac0 0x7f60740cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 --2- 172.21.15.60:0/3534805738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60740b8200 0x7f60741e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.771+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 >> 172.21.15.60:0/3534805738 conn(0x7f607406b2f0 msgr2=0x7f60740b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.772+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 shutdown_connections 2023-07-13T22:21:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.772+0000 7f607b5ff700 1 -- 172.21.15.60:0/3534805738 wait complete. 2023-07-13T22:21:59.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:21:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.992+0000 7f5f50bff700 1 Processor -- start 2023-07-13T22:21:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.993+0000 7f5f50bff700 1 -- start start 2023-07-13T22:21:59.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.994+0000 7f5f50bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 0x7f5f4c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.994+0000 7f5f50bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f4c0d3750 0x7f5f4c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.994+0000 7f5f50bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c0cff20 con 0x7f5f4c0cfad0 2023-07-13T22:21:59.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.994+0000 7f5f50bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c0d0ac0 con 0x7f5f4c0d3750 2023-07-13T22:21:59.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.994+0000 7f5f50bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0b87d0 con 0x7f5f4c0b8400 2023-07-13T22:21:59.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.994+0000 7f5f43fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f4c0d3750 0x7f5f4c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.994+0000 7f5f43fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f4c0d3750 0x7f5f4c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50816/0 (socket says 172.21.15.60:50816) 2023-07-13T22:21:59.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.994+0000 7f5f43fff700 1 -- 172.21.15.60:0/186923265 learned_addr learned my addr 172.21.15.60:0/186923265 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:59.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.994+0000 7f5f4a59c700 1 --2- 172.21.15.60:0/186923265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 0x7f5f4c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.995+0000 7f5f43fff700 1 -- 172.21.15.60:0/186923265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 msgr2=0x7f5f4c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:59.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.995+0000 7f5f43fff700 1 --2- 172.21.15.60:0/186923265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 0x7f5f4c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.995+0000 7f5f43fff700 1 -- 172.21.15.60:0/186923265 >> v1:172.21.15.105:6789/0 conn(0x7f5f4c0cfad0 legacy=0x7f5f4c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.995+0000 7f5f43fff700 1 -- 172.21.15.60:0/186923265 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0b4eb0 con 0x7f5f4c0d3750 2023-07-13T22:21:59.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.995+0000 7f5f43fff700 1 --2- 172.21.15.60:0/186923265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 0x7f5f4c0cf590 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5f340098f0 tx=0x7f5f34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe10d4528d87afbd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:59.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.995+0000 7f5f49d9b700 1 -- 172.21.15.60:0/186923265 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400b8a0 con 0x7f5f4c0d3750 2023-07-13T22:21:59.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.995+0000 7f5f49d9b700 1 -- 172.21.15.60:0/186923265 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3400ba00 con 0x7f5f4c0d3750 2023-07-13T22:21:59.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.995+0000 7f5f49d9b700 1 -- 172.21.15.60:0/186923265 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400bd90 con 0x7f5f4c0d3750 2023-07-13T22:21:59.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.996+0000 7f5f50bff700 1 -- 172.21.15.60:0/186923265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 msgr2=0x7f5f4c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.996+0000 7f5f50bff700 1 --2- 172.21.15.60:0/186923265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 0x7f5f4c0cf590 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5f340098f0 tx=0x7f5f34009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:59.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.996+0000 7f5f50bff700 1 -- 172.21.15.60:0/186923265 shutdown_connections 2023-07-13T22:21:59.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.996+0000 7f5f50bff700 1 --2- 172.21.15.60:0/186923265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 0x7f5f4c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.996+0000 7f5f50bff700 1 --2- 172.21.15.60:0/186923265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 0x7f5f4c0cf590 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.996+0000 7f5f50bff700 1 -- 172.21.15.60:0/186923265 >> 172.21.15.60:0/186923265 conn(0x7f5f4c06b390 msgr2=0x7f5f4c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:59.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.996+0000 7f5f50bff700 1 -- 172.21.15.60:0/186923265 shutdown_connections 2023-07-13T22:21:59.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.996+0000 7f5f50bff700 1 -- 172.21.15.60:0/186923265 wait complete. 2023-07-13T22:21:59.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.998+0000 7f5f50bff700 1 Processor -- start 2023-07-13T22:21:59.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:21:59.999+0000 7f5f50bff700 1 -- start start 2023-07-13T22:21:59.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.000+0000 7f5f50bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 0x7f5f4c1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.000+0000 7f5f50bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4c0cfad0 0x7f5f4c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.000+0000 7f5f50bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 0x7f5f4c0b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.000+0000 7f5f50bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1f30d0 con 0x7f5f4c0d3750 2023-07-13T22:21:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.000+0000 7f5f50bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0ca5b0 con 0x7f5f4c0cfad0 2023-07-13T22:21:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.000+0000 7f5f50bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0c9cf0 con 0x7f5f4c0b8400 2023-07-13T22:21:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.000+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 0x7f5f4c0b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.000+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 0x7f5f4c0b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50824/0 (socket says 172.21.15.60:50824) 2023-07-13T22:21:59.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.000+0000 7f5f4ad9d700 1 -- 172.21.15.60:0/735385695 learned_addr learned my addr 172.21.15.60:0/735385695 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:21:59.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f4a59c700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 0x7f5f4c1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f43fff700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4c0cfad0 0x7f5f4c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f4ad9d700 1 -- 172.21.15.60:0/735385695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 msgr2=0x7f5f4c1f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f4ad9d700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 0x7f5f4c1f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f4ad9d700 1 -- 172.21.15.60:0/735385695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4c0cfad0 msgr2=0x7f5f4c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:59.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f4ad9d700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4c0cfad0 0x7f5f4c15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:59.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f4ad9d700 1 -- 172.21.15.60:0/735385695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34009550 con 0x7f5f4c0d3750 2023-07-13T22:21:59.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f4ad9d700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 0x7f5f4c0b3310 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400cb80 tx=0x7f5f4400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:59.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f42ffd700 1 -- 172.21.15.60:0/735385695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44003c00 con 0x7f5f4c0d3750 2023-07-13T22:21:59.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f42ffd700 1 -- 172.21.15.60:0/735385695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f44003d60 con 0x7f5f4c0d3750 2023-07-13T22:21:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c1f3300 con 0x7f5f4c0d3750 2023-07-13T22:21:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f42ffd700 1 -- 172.21.15.60:0/735385695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400abb0 con 0x7f5f4c0d3750 2023-07-13T22:21:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.001+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c0c9f60 con 0x7f5f4c0d3750 2023-07-13T22:21:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.003+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c0ca720 con 0x7f5f4c0d3750 2023-07-13T22:21:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.003+0000 7f5f42ffd700 1 -- 172.21.15.60:0/735385695 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f4400ad10 con 0x7f5f4c0d3750 2023-07-13T22:21:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.005+0000 7f5f42ffd700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f2c06ace0 0x7f5f2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.005+0000 7f5f42ffd700 1 -- 172.21.15.60:0/735385695 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5f44089400 con 0x7f5f4c0d3750 2023-07-13T22:21:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.005+0000 7f5f4a59c700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f2c06ace0 0x7f5f2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.006+0000 7f5f4a59c700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f2c06ace0 0x7f5f2c06d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00b9b0 tx=0x7f5f3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.009+0000 7f5f42ffd700 1 -- 172.21.15.60:0/735385695 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5f440543d0 con 0x7f5f4c0d3750 2023-07-13T22:22:00.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.230+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5f4c0cff20 con 0x7f5f4c0d3750 2023-07-13T22:22:00.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.231+0000 7f5f42ffd700 1 -- 172.21.15.60:0/735385695 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5f4c0cff20 con 0x7f5f4c0d3750 2023-07-13T22:22:00.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f2c06ace0 msgr2=0x7f5f2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:00.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f2c06ace0 0x7f5f2c06d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00b9b0 tx=0x7f5f3c008040 comp rx=0 tx=0).stop 2023-07-13T22:22:00.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 msgr2=0x7f5f4c0b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:00.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 0x7f5f4c0b3310 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400cb80 tx=0x7f5f4400cf40 comp rx=0 tx=0).stop 2023-07-13T22:22:00.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 shutdown_connections 2023-07-13T22:22:00.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f2c06ace0 0x7f5f2c06d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0d3750 0x7f5f4c0b3310 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f4c0b8400 0x7f5f4c1f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 --2- 172.21.15.60:0/735385695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f4c0cfad0 0x7f5f4c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 >> 172.21.15.60:0/735385695 conn(0x7f5f4c06b390 msgr2=0x7f5f4c0b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:00.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 shutdown_connections 2023-07-13T22:22:00.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.233+0000 7f5f50bff700 1 -- 172.21.15.60:0/735385695 wait complete. 2023-07-13T22:22:00.222 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:22:00.236 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:22:00.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:00.236 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-07-13T22:22:00.314 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:22:00.314 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:22:00.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:00.316 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:22:00.363 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-13T22:22:00.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:00.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.582+0000 7fd9e4fff700 1 Processor -- start 2023-07-13T22:22:00.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.582+0000 7fd9e4fff700 1 -- start start 2023-07-13T22:22:00.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.583+0000 7fd9e4fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd9e00b8400 0x7fd9e00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:00.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.583+0000 7fd9e4fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd9e00d3750 0x7fd9e00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:00.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.583+0000 7fd9e4fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00d0ac0 con 0x7fd9e00cfad0 2023-07-13T22:22:00.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.583+0000 7fd9e4fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e00cff50 con 0x7fd9e00d3750 2023-07-13T22:22:00.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.583+0000 7fd9e4fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e00b87d0 con 0x7fd9e00b8400 2023-07-13T22:22:00.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9ddd9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd9e00d3750 0x7fd9e00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:00.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9ded9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd9e00cfad0 0x7fd9e015fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34060/0 (socket says 172.21.15.60:34060) 2023-07-13T22:22:00.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9ded9d700 1 -- 172.21.15.60:0/1346961144 learned_addr learned my addr 172.21.15.60:0/1346961144 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:00.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9de59c700 1 --2- 172.21.15.60:0/1346961144 >> v2:172.21.15.105:3300/0 conn(0x7fd9e00b8400 0x7fd9e00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9ddd9b700 1 -- 172.21.15.60:0/1346961144 >> v2:172.21.15.105:3300/0 conn(0x7fd9e00b8400 msgr2=0x7fd9e00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9ddd9b700 1 --2- 172.21.15.60:0/1346961144 >> v2:172.21.15.105:3300/0 conn(0x7fd9e00b8400 0x7fd9e00cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9ddd9b700 1 -- 172.21.15.60:0/1346961144 >> v1:172.21.15.60:6789/0 conn(0x7fd9e00cfad0 legacy=0x7fd9e015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9ddd9b700 1 -- 172.21.15.60:0/1346961144 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00b4eb0 con 0x7fd9e00d3750 2023-07-13T22:22:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9dd59a700 1 -- 172.21.15.60:0/1346961144 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695898363 0 0) 0x7fd9e00d0ac0 con 0x7fd9e00cfad0 2023-07-13T22:22:00.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9de59c700 1 --2- 172.21.15.60:0/1346961144 >> v2:172.21.15.105:3300/0 conn(0x7fd9e00b8400 0x7fd9e00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:00.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.584+0000 7fd9ddd9b700 1 --2- 172.21.15.60:0/1346961144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 0x7fd9e00cf590 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00cf560 tx=0x7fd9d4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac5c24d6f99785 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:00.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9dd59a700 1 -- 172.21.15.60:0/1346961144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4011830 con 0x7fd9e00d3750 2023-07-13T22:22:00.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9dd59a700 1 -- 172.21.15.60:0/1346961144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4011990 con 0x7fd9e00d3750 2023-07-13T22:22:00.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9dd59a700 1 -- 172.21.15.60:0/1346961144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4011d20 con 0x7fd9e00d3750 2023-07-13T22:22:00.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1346961144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 msgr2=0x7fd9e00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:00.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9e4fff700 1 --2- 172.21.15.60:0/1346961144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 0x7fd9e00cf590 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00cf560 tx=0x7fd9d4009b20 comp rx=0 tx=0).stop 2023-07-13T22:22:00.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1346961144 shutdown_connections 2023-07-13T22:22:00.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9e4fff700 1 --2- 172.21.15.60:0/1346961144 >> v2:172.21.15.105:3300/0 conn(0x7fd9e00b8400 0x7fd9e00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9e4fff700 1 --2- 172.21.15.60:0/1346961144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 0x7fd9e00cf590 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1346961144 >> 172.21.15.60:0/1346961144 conn(0x7fd9e006b390 msgr2=0x7fd9e006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:00.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.585+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1346961144 shutdown_connections 2023-07-13T22:22:00.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.586+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1346961144 wait complete. 2023-07-13T22:22:00.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.587+0000 7fd9e4fff700 1 Processor -- start 2023-07-13T22:22:00.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.587+0000 7fd9e4fff700 1 -- start start 2023-07-13T22:22:00.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9e4fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9e00b8400 0x7fd9e0159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:00.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9e4fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9e00cfad0 0x7fd9e0154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:00.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9e4fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 0x7fd9e0156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:00.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9e4fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e0159830 con 0x7fd9e00d3750 2023-07-13T22:22:00.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9e4fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e0154650 con 0x7fd9e00cfad0 2023-07-13T22:22:00.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9e4fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e015e910 con 0x7fd9e00b8400 2023-07-13T22:22:00.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9ded9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 0x7fd9e0156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:00.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9ddd9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9e00cfad0 0x7fd9e0154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:00.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9ded9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 0x7fd9e0156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50852/0 (socket says 172.21.15.60:50852) 2023-07-13T22:22:00.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9ded9d700 1 -- 172.21.15.60:0/1703035040 learned_addr learned my addr 172.21.15.60:0/1703035040 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:00.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9de59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9e00b8400 0x7fd9e0159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:00.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9ded9d700 1 -- 172.21.15.60:0/1703035040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9e00b8400 msgr2=0x7fd9e0159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:00.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9ded9d700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9e00b8400 0x7fd9e0159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9ded9d700 1 -- 172.21.15.60:0/1703035040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9e00cfad0 msgr2=0x7fd9e0154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:00.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9ded9d700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9e00cfad0 0x7fd9e0154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.589+0000 7fd9ded9d700 1 -- 172.21.15.60:0/1703035040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40094a0 con 0x7fd9e00d3750 2023-07-13T22:22:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.590+0000 7fd9ded9d700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 0x7fd9e0156c20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800cba0 tx=0x7fd9d800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.590+0000 7fd9ceffd700 1 -- 172.21.15.60:0/1703035040 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800aaa0 con 0x7fd9e00d3750 2023-07-13T22:22:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.590+0000 7fd9ceffd700 1 -- 172.21.15.60:0/1703035040 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d800ac20 con 0x7fd9e00d3750 2023-07-13T22:22:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.590+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e0158bb0 con 0x7fd9e00d3750 2023-07-13T22:22:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.590+0000 7fd9ceffd700 1 -- 172.21.15.60:0/1703035040 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80070d0 con 0x7fd9e00d3750 2023-07-13T22:22:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.590+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e0159170 con 0x7fd9e00d3750 2023-07-13T22:22:00.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.591+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e0159970 con 0x7fd9e00d3750 2023-07-13T22:22:00.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.592+0000 7fd9ceffd700 1 -- 172.21.15.60:0/1703035040 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd9d8007230 con 0x7fd9e00d3750 2023-07-13T22:22:00.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.593+0000 7fd9ceffd700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9c006ace0 0x7fd9c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:00.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.594+0000 7fd9ceffd700 1 -- 172.21.15.60:0/1703035040 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd9d8089250 con 0x7fd9e00d3750 2023-07-13T22:22:00.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.594+0000 7fd9de59c700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9c006ace0 0x7fd9c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:00.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.594+0000 7fd9de59c700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9c006ace0 0x7fd9c006d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0006f10 tx=0x7fd9d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:00.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.598+0000 7fd9ceffd700 1 -- 172.21.15.60:0/1703035040 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd9e0159970 con 0x7fd9e00d3750 2023-07-13T22:22:00.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.851+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9e0159970 con 0x7fd9e00d3750 2023-07-13T22:22:00.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.851+0000 7fd9ceffd700 1 -- 172.21.15.60:0/1703035040 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fd9e0159970 con 0x7fd9e00d3750 2023-07-13T22:22:00.836 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:00.836 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2550952354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2550952354},{"type":"v1","addr":"172.21.15.60:6837","nonce":2550952354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4134045869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4134045869},{"type":"v1","addr":"172.21.15.60:6836","nonce":4134045869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2202330082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2202330082},{"type":"v1","addr":"172.21.15.105:6834","nonce":2202330082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:20:54.038443+0000","modified":"2023-07-13T22:20:57.536915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/453882022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":453882022},{"type":"v1","addr":"172.21.15.105:6835","nonce":453882022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:22:00.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9c006ace0 msgr2=0x7fd9c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:00.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9c006ace0 0x7fd9c006d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0006f10 tx=0x7fd9d0008040 comp rx=0 tx=0).stop 2023-07-13T22:22:00.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 msgr2=0x7fd9e0156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:00.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 0x7fd9e0156c20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800cba0 tx=0x7fd9d800ceb0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 shutdown_connections 2023-07-13T22:22:00.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9c006ace0 0x7fd9c006d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e00d3750 0x7fd9e0156c20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9e00b8400 0x7fd9e0159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 --2- 172.21.15.60:0/1703035040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9e00cfad0 0x7fd9e0154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:00.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 >> 172.21.15.60:0/1703035040 conn(0x7fd9e006b390 msgr2=0x7fd9e015baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:00.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 shutdown_connections 2023-07-13T22:22:00.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:00.854+0000 7fd9e4fff700 1 -- 172.21.15.60:0/1703035040 wait complete. 2023-07-13T22:22:00.840 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2023-07-13T22:22:00.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.076+0000 7fd4197ff700 1 Processor -- start 2023-07-13T22:22:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.076+0000 7fd4197ff700 1 -- start start 2023-07-13T22:22:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.077+0000 7fd4197ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4140b8400 0x7fd4140cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.077+0000 7fd4197ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd4140d3750 0x7fd4140cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.077+0000 7fd4197ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140cfad0 0x7fd41415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.077+0000 7fd4197ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4140cff50 con 0x7fd4140b8400 2023-07-13T22:22:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.077+0000 7fd4197ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4140d0ac0 con 0x7fd4140d3750 2023-07-13T22:22:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.077+0000 7fd4197ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140b87d0 con 0x7fd4140cfad0 2023-07-13T22:22:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.077+0000 7fd412ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4140b8400 0x7fd4140cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.077+0000 7fd412ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4140b8400 0x7fd4140cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50854/0 (socket says 172.21.15.60:50854) 2023-07-13T22:22:01.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.077+0000 7fd412ffd700 1 -- 172.21.15.60:0/1707770581 learned_addr learned my addr 172.21.15.60:0/1707770581 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd4137fe700 1 --2- 172.21.15.60:0/1707770581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140cfad0 0x7fd41415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd40bfff700 1 --2- 172.21.15.60:0/1707770581 >> v2:172.21.15.105:3300/0 conn(0x7fd4140d3750 0x7fd4140cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd412ffd700 1 -- 172.21.15.60:0/1707770581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140cfad0 msgr2=0x7fd41415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd412ffd700 1 --2- 172.21.15.60:0/1707770581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140cfad0 0x7fd41415fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd412ffd700 1 -- 172.21.15.60:0/1707770581 >> v2:172.21.15.105:3300/0 conn(0x7fd4140d3750 msgr2=0x7fd4140cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd412ffd700 1 --2- 172.21.15.60:0/1707770581 >> v2:172.21.15.105:3300/0 conn(0x7fd4140d3750 0x7fd4140cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd412ffd700 1 -- 172.21.15.60:0/1707770581 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140b4eb0 con 0x7fd4140b8400 2023-07-13T22:22:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd4137fe700 1 --2- 172.21.15.60:0/1707770581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140cfad0 0x7fd41415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd40bfff700 1 --2- 172.21.15.60:0/1707770581 >> v2:172.21.15.105:3300/0 conn(0x7fd4140d3750 0x7fd4140cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd412ffd700 1 --2- 172.21.15.60:0/1707770581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140b8400 0x7fd4140cd240 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd4140b8cf0 tx=0x7fd404009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5eed02f0bc6f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:01.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd4127fc700 1 -- 172.21.15.60:0/1707770581 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400b870 con 0x7fd4140b8400 2023-07-13T22:22:01.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd4127fc700 1 -- 172.21.15.60:0/1707770581 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40400b9d0 con 0x7fd4140b8400 2023-07-13T22:22:01.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.078+0000 7fd4127fc700 1 -- 172.21.15.60:0/1707770581 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400bd60 con 0x7fd4140b8400 2023-07-13T22:22:01.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.079+0000 7fd4197ff700 1 -- 172.21.15.60:0/1707770581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140b8400 msgr2=0x7fd4140cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.079+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1707770581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140b8400 0x7fd4140cd240 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd4140b8cf0 tx=0x7fd404009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.079+0000 7fd4197ff700 1 -- 172.21.15.60:0/1707770581 shutdown_connections 2023-07-13T22:22:01.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.079+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1707770581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140cfad0 0x7fd41415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.079+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1707770581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140b8400 0x7fd4140cd240 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.079+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1707770581 >> v2:172.21.15.105:3300/0 conn(0x7fd4140d3750 0x7fd4140cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.079+0000 7fd4197ff700 1 -- 172.21.15.60:0/1707770581 >> 172.21.15.60:0/1707770581 conn(0x7fd41406b390 msgr2=0x7fd41406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.079+0000 7fd4197ff700 1 -- 172.21.15.60:0/1707770581 shutdown_connections 2023-07-13T22:22:01.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.079+0000 7fd4197ff700 1 -- 172.21.15.60:0/1707770581 wait complete. 2023-07-13T22:22:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.081+0000 7fd4197ff700 1 Processor -- start 2023-07-13T22:22:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.081+0000 7fd4197ff700 1 -- start start 2023-07-13T22:22:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd4197ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140b8400 0x7fd4140b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd4197ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140cfad0 0x7fd4140c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd4197ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140d3750 0x7fd414150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd4197ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4141f2960 con 0x7fd4140cfad0 2023-07-13T22:22:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd4197ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4141f1de0 con 0x7fd4140b8400 2023-07-13T22:22:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd4197ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4141f20c0 con 0x7fd4140d3750 2023-07-13T22:22:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd40bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140cfad0 0x7fd4140c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd40bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140cfad0 0x7fd4140c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50860/0 (socket says 172.21.15.60:50860) 2023-07-13T22:22:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd40bfff700 1 -- 172.21.15.60:0/1675135060 learned_addr learned my addr 172.21.15.60:0/1675135060 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd412ffd700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140b8400 0x7fd4140b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd4137fe700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140d3750 0x7fd414150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd40bfff700 1 -- 172.21.15.60:0/1675135060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140d3750 msgr2=0x7fd414150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd40bfff700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140d3750 0x7fd414150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd40bfff700 1 -- 172.21.15.60:0/1675135060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140b8400 msgr2=0x7fd4140b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.082+0000 7fd40bfff700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140b8400 0x7fd4140b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.083+0000 7fd40bfff700 1 -- 172.21.15.60:0/1675135060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404009550 con 0x7fd4140cfad0 2023-07-13T22:22:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.083+0000 7fd412ffd700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140b8400 0x7fd4140b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.083+0000 7fd40bfff700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140cfad0 0x7fd4140c8ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c7f0 tx=0x7fd3fc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.083+0000 7fd40b7fe700 1 -- 172.21.15.60:0/1675135060 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc011800 con 0x7fd4140cfad0 2023-07-13T22:22:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.083+0000 7fd40b7fe700 1 -- 172.21.15.60:0/1675135060 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc011960 con 0x7fd4140cfad0 2023-07-13T22:22:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.083+0000 7fd40b7fe700 1 -- 172.21.15.60:0/1675135060 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc011d10 con 0x7fd4140cfad0 2023-07-13T22:22:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.083+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4141f23a0 con 0x7fd4140cfad0 2023-07-13T22:22:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.083+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4141f19f0 con 0x7fd4140cfad0 2023-07-13T22:22:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.084+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4140c6ea0 con 0x7fd4140cfad0 2023-07-13T22:22:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.085+0000 7fd40b7fe700 1 -- 172.21.15.60:0/1675135060 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd3fc018360 con 0x7fd4140cfad0 2023-07-13T22:22:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.086+0000 7fd40b7fe700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3f406ace0 0x7fd3f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.086+0000 7fd40b7fe700 1 -- 172.21.15.60:0/1675135060 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd3fc0894e0 con 0x7fd4140cfad0 2023-07-13T22:22:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.086+0000 7fd412ffd700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3f406ace0 0x7fd3f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.087+0000 7fd412ffd700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3f406ace0 0x7fd3f406d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd404009130 tx=0x7fd404008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.090+0000 7fd40b7fe700 1 -- 172.21.15.60:0/1675135060 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd3fc057f40 con 0x7fd4140cfad0 2023-07-13T22:22:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.306+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd41415b100 con 0x7fd4140cfad0 2023-07-13T22:22:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.307+0000 7fd40b7fe700 1 -- 172.21.15.60:0/1675135060 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7fd3fc057ad0 con 0x7fd4140cfad0 2023-07-13T22:22:01.292 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:01.292 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:21:42.683944+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3f406ace0 msgr2=0x7fd3f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3f406ace0 0x7fd3f406d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd404009130 tx=0x7fd404008030 comp rx=0 tx=0).stop 2023-07-13T22:22:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140cfad0 msgr2=0x7fd4140c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140cfad0 0x7fd4140c8ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c7f0 tx=0x7fd3fc00cbb0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 shutdown_connections 2023-07-13T22:22:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3f406ace0 0x7fd3f406d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4140d3750 0x7fd414150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140b8400 0x7fd4140b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 --2- 172.21.15.60:0/1675135060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4140cfad0 0x7fd4140c8ec0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 >> 172.21.15.60:0/1675135060 conn(0x7fd41406b390 msgr2=0x7fd4140cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 shutdown_connections 2023-07-13T22:22:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.310+0000 7fd4197ff700 1 -- 172.21.15.60:0/1675135060 wait complete. 2023-07-13T22:22:01.313 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:20:51.930543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:20:53.012506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:22:01.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:22:01.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.536+0000 7fa2ab3ff700 1 Processor -- start 2023-07-13T22:22:01.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.536+0000 7fa2ab3ff700 1 -- start start 2023-07-13T22:22:01.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.537+0000 7fa2ab3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40ca900 0x7fa2a40703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.537+0000 7fa2ab3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a4070d80 con 0x7fa2a4070920 2023-07-13T22:22:01.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.537+0000 7fa2ab3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a40cad70 con 0x7fa2a4159260 2023-07-13T22:22:01.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.537+0000 7fa2ab3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a40cf640 con 0x7fa2a40ca900 2023-07-13T22:22:01.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.537+0000 7fa2a999c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa2a4070920 0x7fa2a406c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34062/0 (socket says 172.21.15.60:34062) 2023-07-13T22:22:01.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa2a999c700 1 -- 172.21.15.60:0/3752042570 learned_addr learned my addr 172.21.15.60:0/3752042570 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:01.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa2a899a700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40ca900 0x7fa2a40703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445842485 0 0) 0x7fa2a4070d80 con 0x7fa2a4070920 2023-07-13T22:22:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2900035b0 con 0x7fa2a4070920 2023-07-13T22:22:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633208408 0 0) 0x7fa2a40cad70 con 0x7fa2a4159260 2023-07-13T22:22:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a4070d80 con 0x7fa2a4159260 2023-07-13T22:22:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 277874955 0 0) 0x7fa2900035b0 con 0x7fa2a4070920 2023-07-13T22:22:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40ca900 msgr2=0x7fa2a40703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40ca900 0x7fa2a40703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 >> v1:172.21.15.105:6789/0 conn(0x7fa2a4159260 legacy=0x7fa2a40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a40ccf80 con 0x7fa2a4070920 2023-07-13T22:22:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fa29c0019f0 con 0x7fa2a4070920 2023-07-13T22:22:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 >> v1:172.21.15.60:6789/0 conn(0x7fa2a4070920 legacy=0x7fa2a406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa290003bc0 0x7fa290003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.538+0000 7fa29bfff700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa290004850 0x7fa290004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa29bfff700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40ca900 0x7fa290005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a40cf640 con 0x7fa290003bc0 2023-07-13T22:22:01.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29000dcd0 con 0x7fa290004850 2023-07-13T22:22:01.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29000de90 con 0x7fa2a40ca900 2023-07-13T22:22:01.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa2a919b700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa290003bc0 0x7fa290003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa2a899a700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa290004850 0x7fa290004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa2a999c700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40ca900 0x7fa290005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa2a919b700 1 -- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40ca900 msgr2=0x7fa290005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:01.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa2a919b700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40ca900 0x7fa290005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa2a919b700 1 -- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa290004850 msgr2=0x7fa290004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa2a919b700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa290004850 0x7fa290004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa2a919b700 1 -- 172.21.15.60:0/3752042570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2900035b0 con 0x7fa290003bc0 2023-07-13T22:22:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.539+0000 7fa2a919b700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa290003bc0 0x7fa290003fb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0001940 tx=0x7fa290005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bddf1dd4119d35 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0001430 con 0x7fa290003bc0 2023-07-13T22:22:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a0001d70 con 0x7fa290003bc0 2023-07-13T22:22:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa29bfff700 1 -- 172.21.15.60:0/3752042570 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0005510 con 0x7fa290003bc0 2023-07-13T22:22:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3752042570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa290003bc0 msgr2=0x7fa290003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa2ab3ff700 1 --2- 172.21.15.60:0/3752042570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa290003bc0 0x7fa290003fb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0001940 tx=0x7fa290005880 comp rx=0 tx=0).stop 2023-07-13T22:22:01.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa2a999c700 1 -- 172.21.15.60:0/3752042570 reap_dead start 2023-07-13T22:22:01.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3752042570 shutdown_connections 2023-07-13T22:22:01.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3752042570 >> 172.21.15.60:0/3752042570 conn(0x7fa2a40cbf50 msgr2=0x7fa2a40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:01.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3752042570 shutdown_connections 2023-07-13T22:22:01.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.540+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3752042570 wait complete. 2023-07-13T22:22:01.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.542+0000 7fa2ab3ff700 1 Processor -- start 2023-07-13T22:22:01.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.542+0000 7fa2ab3ff700 1 -- start start 2023-07-13T22:22:01.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.543+0000 7fa2ab3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa290003bc0 0x7fa2a415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.543+0000 7fa2ab3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2a4158550 0x7fa2a414a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.543+0000 7fa2ab3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40af6c0 0x7fa2a40afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.543+0000 7fa2ab3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a40b0070 con 0x7fa2a4158550 2023-07-13T22:22:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.543+0000 7fa2ab3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a415a9d0 con 0x7fa290003bc0 2023-07-13T22:22:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.543+0000 7fa2ab3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a40c5910 con 0x7fa2a40af6c0 2023-07-13T22:22:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a899a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2a4158550 0x7fa2a414a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a899a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2a4158550 0x7fa2a414a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50884/0 (socket says 172.21.15.60:50884) 2023-07-13T22:22:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a899a700 1 -- 172.21.15.60:0/3565361196 learned_addr learned my addr 172.21.15.60:0/3565361196 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a999c700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40af6c0 0x7fa2a40afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a919b700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa290003bc0 0x7fa2a415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a899a700 1 -- 172.21.15.60:0/3565361196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40af6c0 msgr2=0x7fa2a40afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a899a700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40af6c0 0x7fa2a40afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a899a700 1 -- 172.21.15.60:0/3565361196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa290003bc0 msgr2=0x7fa2a415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a899a700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa290003bc0 0x7fa2a415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a899a700 1 -- 172.21.15.60:0/3565361196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a40c5bf0 con 0x7fa2a4158550 2023-07-13T22:22:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.544+0000 7fa2a899a700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2a4158550 0x7fa2a414a1e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa294011aa0 tx=0x7fa294011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.545+0000 7fa299ffb700 1 -- 172.21.15.60:0/3565361196 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294011ee0 con 0x7fa2a4158550 2023-07-13T22:22:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.545+0000 7fa299ffb700 1 -- 172.21.15.60:0/3565361196 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29400f940 con 0x7fa2a4158550 2023-07-13T22:22:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.545+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a40c5ed0 con 0x7fa2a4158550 2023-07-13T22:22:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.545+0000 7fa299ffb700 1 -- 172.21.15.60:0/3565361196 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29400fcf0 con 0x7fa2a4158550 2023-07-13T22:22:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.545+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a40c6490 con 0x7fa2a4158550 2023-07-13T22:22:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.546+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a40ab9b0 con 0x7fa2a4158550 2023-07-13T22:22:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.547+0000 7fa299ffb700 1 -- 172.21.15.60:0/3565361196 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa294014440 con 0x7fa2a4158550 2023-07-13T22:22:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.548+0000 7fa299ffb700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa28806ace0 0x7fa28806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.548+0000 7fa299ffb700 1 -- 172.21.15.60:0/3565361196 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa294089b90 con 0x7fa2a4158550 2023-07-13T22:22:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.549+0000 7fa2a919b700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa28806ace0 0x7fa28806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.549+0000 7fa2a919b700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa28806ace0 0x7fa28806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0015040 tx=0x7fa2a000a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.553+0000 7fa299ffb700 1 -- 172.21.15.60:0/3565361196 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa2940586b0 con 0x7fa2a4158550 2023-07-13T22:22:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.761+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa2a40c9e00 con 0x7fa2a4158550 2023-07-13T22:22:01.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.761+0000 7fa299ffb700 1 -- 172.21.15.60:0/3565361196 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa2a40c9e00 con 0x7fa2a4158550 2023-07-13T22:22:01.747 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:22:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.763+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa28806ace0 msgr2=0x7fa28806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.763+0000 7fa2ab3ff700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa28806ace0 0x7fa28806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0015040 tx=0x7fa2a000a000 comp rx=0 tx=0).stop 2023-07-13T22:22:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2a4158550 msgr2=0x7fa2a414a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2a4158550 0x7fa2a414a1e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa294011aa0 tx=0x7fa294011db0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 shutdown_connections 2023-07-13T22:22:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa28806ace0 0x7fa28806d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2a40af6c0 0x7fa2a40afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa290003bc0 0x7fa2a415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 --2- 172.21.15.60:0/3565361196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2a4158550 0x7fa2a414a1e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 >> 172.21.15.60:0/3565361196 conn(0x7fa2a40cbf50 msgr2=0x7fa2a40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:01.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 shutdown_connections 2023-07-13T22:22:01.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.764+0000 7fa2ab3ff700 1 -- 172.21.15.60:0/3565361196 wait complete. 2023-07-13T22:22:01.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:22:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.988+0000 7fbc077ff700 1 Processor -- start 2023-07-13T22:22:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.988+0000 7fbc077ff700 1 -- start start 2023-07-13T22:22:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.989+0000 7fbc077ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc000b8200 0x7fbc000cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.989+0000 7fbc077ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b1ac0 0x7fbc000cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.989+0000 7fbc077ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc000d1870 con 0x7fbc000cede0 2023-07-13T22:22:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.989+0000 7fbc077ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc000cf370 con 0x7fbc000b8200 2023-07-13T22:22:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.989+0000 7fbc077ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000d12b0 con 0x7fbc000b1ac0 2023-07-13T22:22:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbc05d9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbc000cede0 0x7fbc0015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34072/0 (socket says 172.21.15.60:34072) 2023-07-13T22:22:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbc05d9c700 1 -- 172.21.15.60:0/611134922 learned_addr learned my addr 172.21.15.60:0/611134922 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbc04d9a700 1 --2- 172.21.15.60:0/611134922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b1ac0 0x7fbc000cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbc0559b700 1 --2- 172.21.15.60:0/611134922 >> v2:172.21.15.105:3300/0 conn(0x7fbc000b8200 0x7fbc000cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbbf7fff700 1 -- 172.21.15.60:0/611134922 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649658382 0 0) 0x7fbc000d1870 con 0x7fbc000cede0 2023-07-13T22:22:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbbf7fff700 1 -- 172.21.15.60:0/611134922 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbec0035b0 con 0x7fbc000cede0 2023-07-13T22:22:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbbf7fff700 1 -- 172.21.15.60:0/611134922 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1088644728 0 0) 0x7fbbec0035b0 con 0x7fbc000cede0 2023-07-13T22:22:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbc04d9a700 1 -- 172.21.15.60:0/611134922 >> v2:172.21.15.105:3300/0 conn(0x7fbc000b8200 msgr2=0x7fbc000cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbc04d9a700 1 --2- 172.21.15.60:0/611134922 >> v2:172.21.15.105:3300/0 conn(0x7fbc000b8200 0x7fbc000cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbc04d9a700 1 -- 172.21.15.60:0/611134922 >> v1:172.21.15.60:6789/0 conn(0x7fbc000cede0 legacy=0x7fbc0015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbc04d9a700 1 -- 172.21.15.60:0/611134922 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc000d1590 con 0x7fbc000b1ac0 2023-07-13T22:22:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.990+0000 7fbc0559b700 1 --2- 172.21.15.60:0/611134922 >> v2:172.21.15.105:3300/0 conn(0x7fbc000b8200 0x7fbc000cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.991+0000 7fbc04d9a700 1 --2- 172.21.15.60:0/611134922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b1ac0 0x7fbc000cfb80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009840 tx=0x7fbbf0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a6b6cf67bf7aea server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.991+0000 7fbbf7fff700 1 -- 172.21.15.60:0/611134922 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80019f0 con 0x7fbc000b1ac0 2023-07-13T22:22:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.991+0000 7fbbf7fff700 1 -- 172.21.15.60:0/611134922 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf0004030 con 0x7fbc000b1ac0 2023-07-13T22:22:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.991+0000 7fbbf7fff700 1 -- 172.21.15.60:0/611134922 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00043c0 con 0x7fbc000b1ac0 2023-07-13T22:22:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.991+0000 7fbc077ff700 1 -- 172.21.15.60:0/611134922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b1ac0 msgr2=0x7fbc000cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.991+0000 7fbc077ff700 1 --2- 172.21.15.60:0/611134922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b1ac0 0x7fbc000cfb80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009840 tx=0x7fbbf0009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.992+0000 7fbc077ff700 1 -- 172.21.15.60:0/611134922 shutdown_connections 2023-07-13T22:22:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.992+0000 7fbc077ff700 1 --2- 172.21.15.60:0/611134922 >> v2:172.21.15.105:3300/0 conn(0x7fbc000b8200 0x7fbc000cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.992+0000 7fbc077ff700 1 --2- 172.21.15.60:0/611134922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b1ac0 0x7fbc000cfb80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.992+0000 7fbc077ff700 1 -- 172.21.15.60:0/611134922 >> 172.21.15.60:0/611134922 conn(0x7fbc0006b2f0 msgr2=0x7fbc0006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.992+0000 7fbc077ff700 1 -- 172.21.15.60:0/611134922 shutdown_connections 2023-07-13T22:22:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.992+0000 7fbc077ff700 1 -- 172.21.15.60:0/611134922 wait complete. 2023-07-13T22:22:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.993+0000 7fbc077ff700 1 Processor -- start 2023-07-13T22:22:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.994+0000 7fbc077ff700 1 -- start start 2023-07-13T22:22:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.994+0000 7fbc077ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc000b1ac0 0x7fbc001e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.994+0000 7fbc077ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b8200 0x7fbc00143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc077ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc000cede0 0x7fbc000c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc077ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000cb560 con 0x7fbc000cede0 2023-07-13T22:22:01.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc077ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000cb800 con 0x7fbc000b1ac0 2023-07-13T22:22:01.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc077ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000cbae0 con 0x7fbc000b8200 2023-07-13T22:22:01.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc05d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc000cede0 0x7fbc000c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc04d9a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b8200 0x7fbc00143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc04d9a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b8200 0x7fbc00143110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:43886/0 (socket says 172.21.15.60:43886) 2023-07-13T22:22:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc04d9a700 1 -- 172.21.15.60:0/2429310517 learned_addr learned my addr 172.21.15.60:0/2429310517 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc0559b700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc000b1ac0 0x7fbc001e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc05d9c700 1 -- 172.21.15.60:0/2429310517 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b8200 msgr2=0x7fbc00143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc05d9c700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b8200 0x7fbc00143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc05d9c700 1 -- 172.21.15.60:0/2429310517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc000b1ac0 msgr2=0x7fbc001e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc05d9c700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc000b1ac0 0x7fbc001e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:01.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc05d9c700 1 -- 172.21.15.60:0/2429310517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0094a0 con 0x7fbc000cede0 2023-07-13T22:22:01.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.995+0000 7fbc04d9a700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b8200 0x7fbc00143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:01.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.996+0000 7fbc05d9c700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc000cede0 0x7fbc000c8930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800cae0 tx=0x7fbbf800cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:01.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.996+0000 7fbbf5ffb700 1 -- 172.21.15.60:0/2429310517 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800a9e0 con 0x7fbc000cede0 2023-07-13T22:22:01.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.996+0000 7fbbf5ffb700 1 -- 172.21.15.60:0/2429310517 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf800ab60 con 0x7fbc000cede0 2023-07-13T22:22:01.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.996+0000 7fbbf5ffb700 1 -- 172.21.15.60:0/2429310517 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80070d0 con 0x7fbc000cede0 2023-07-13T22:22:01.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.996+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf00094a0 con 0x7fbc000cede0 2023-07-13T22:22:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.996+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc000cad80 con 0x7fbc000cede0 2023-07-13T22:22:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.997+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0004e210 con 0x7fbc000cede0 2023-07-13T22:22:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:01.998+0000 7fbbf5ffb700 1 -- 172.21.15.60:0/2429310517 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbbf8007230 con 0x7fbc000cede0 2023-07-13T22:22:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.000+0000 7fbbf5ffb700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbbe406ace0 0x7fbbe406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:01.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.000+0000 7fbbf5ffb700 1 -- 172.21.15.60:0/2429310517 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbbf8088ee0 con 0x7fbc000cede0 2023-07-13T22:22:01.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.000+0000 7fbc0559b700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbbe406ace0 0x7fbbe406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:01.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.000+0000 7fbc0559b700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbbe406ace0 0x7fbbe406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc000b3dd0 tx=0x7fbbfc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:01.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.004+0000 7fbbf5ffb700 1 -- 172.21.15.60:0/2429310517 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbbf8053eb0 con 0x7fbc000cede0 2023-07-13T22:22:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.209+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc000b4f70 con 0x7fbc000cede0 2023-07-13T22:22:02.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.209+0000 7fbbf5ffb700 1 -- 172.21.15.60:0/2429310517 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbf80180a0 con 0x7fbc000cede0 2023-07-13T22:22:02.194 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:22:02.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.211+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbbe406ace0 msgr2=0x7fbbe406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbbe406ace0 0x7fbbe406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc000b3dd0 tx=0x7fbbfc009210 comp rx=0 tx=0).stop 2023-07-13T22:22:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc000cede0 msgr2=0x7fbc000c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc000cede0 0x7fbc000c8930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800cae0 tx=0x7fbbf800cdf0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 shutdown_connections 2023-07-13T22:22:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbbe406ace0 0x7fbbe406d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc000cede0 0x7fbc000c8930 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc000b1ac0 0x7fbc001e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 --2- 172.21.15.60:0/2429310517 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc000b8200 0x7fbc00143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 >> 172.21.15.60:0/2429310517 conn(0x7fbc0006b2f0 msgr2=0x7fbc0006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:02.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 shutdown_connections 2023-07-13T22:22:02.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.212+0000 7fbc077ff700 1 -- 172.21.15.60:0/2429310517 wait complete. 2023-07-13T22:22:02.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:22:02.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.435+0000 7f9335fff700 1 Processor -- start 2023-07-13T22:22:02.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.435+0000 7f9335fff700 1 -- start start 2023-07-13T22:22:02.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.436+0000 7f9335fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93300cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.436+0000 7f9335fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93300d3220 0x7f93300d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.436+0000 7f9335fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f93300d12b0 0x7f93300b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.436+0000 7f9335fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93300b5050 con 0x7f93300d3220 2023-07-13T22:22:02.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.436+0000 7f9335fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93300cd570 con 0x7f93300d12b0 2023-07-13T22:22:02.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.436+0000 7f9335fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93300d2740 con 0x7f93300d09a0 2023-07-13T22:22:02.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.436+0000 7f932effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93300d3220 0x7f93300d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93300d3220 0x7f93300d0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50912/0 (socket says 172.21.15.60:50912) 2023-07-13T22:22:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932effd700 1 -- 172.21.15.60:0/1355100737 learned_addr learned my addr 172.21.15.60:0/1355100737 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:02.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932ffff700 1 --2- 172.21.15.60:0/1355100737 >> v2:172.21.15.105:3300/0 conn(0x7f93300d12b0 0x7f93300b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932f7fe700 1 --2- 172.21.15.60:0/1355100737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93300cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932effd700 1 -- 172.21.15.60:0/1355100737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 msgr2=0x7f93300cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932effd700 1 --2- 172.21.15.60:0/1355100737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93300cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932effd700 1 -- 172.21.15.60:0/1355100737 >> v2:172.21.15.105:3300/0 conn(0x7f93300d12b0 msgr2=0x7f93300b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932effd700 1 --2- 172.21.15.60:0/1355100737 >> v2:172.21.15.105:3300/0 conn(0x7f93300d12b0 0x7f93300b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932effd700 1 -- 172.21.15.60:0/1355100737 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93300d2a20 con 0x7f93300d3220 2023-07-13T22:22:02.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932f7fe700 1 --2- 172.21.15.60:0/1355100737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93300cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:02.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932ffff700 1 --2- 172.21.15.60:0/1355100737 >> v2:172.21.15.105:3300/0 conn(0x7f93300d12b0 0x7f93300b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:02.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932effd700 1 --2- 172.21.15.60:0/1355100737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d3220 0x7f93300d0d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f93300cd140 tx=0x7f9324009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11270191eb4fba4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:02.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932e7fc700 1 -- 172.21.15.60:0/1355100737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400b870 con 0x7f93300d3220 2023-07-13T22:22:02.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932e7fc700 1 -- 172.21.15.60:0/1355100737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932400b9d0 con 0x7f93300d3220 2023-07-13T22:22:02.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.437+0000 7f932e7fc700 1 -- 172.21.15.60:0/1355100737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400bd60 con 0x7f93300d3220 2023-07-13T22:22:02.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.438+0000 7f9335fff700 1 -- 172.21.15.60:0/1355100737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d3220 msgr2=0x7f93300d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.438+0000 7f9335fff700 1 --2- 172.21.15.60:0/1355100737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d3220 0x7f93300d0d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f93300cd140 tx=0x7f9324009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.438+0000 7f9335fff700 1 -- 172.21.15.60:0/1355100737 shutdown_connections 2023-07-13T22:22:02.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.438+0000 7f9335fff700 1 --2- 172.21.15.60:0/1355100737 >> v2:172.21.15.105:3300/0 conn(0x7f93300d12b0 0x7f93300b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.438+0000 7f9335fff700 1 --2- 172.21.15.60:0/1355100737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93300cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.438+0000 7f9335fff700 1 --2- 172.21.15.60:0/1355100737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d3220 0x7f93300d0d70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.438+0000 7f9335fff700 1 -- 172.21.15.60:0/1355100737 >> 172.21.15.60:0/1355100737 conn(0x7f9330071600 msgr2=0x7f93300719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:02.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.438+0000 7f9335fff700 1 -- 172.21.15.60:0/1355100737 shutdown_connections 2023-07-13T22:22:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.438+0000 7f9335fff700 1 -- 172.21.15.60:0/1355100737 wait complete. 2023-07-13T22:22:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.440+0000 7f9335fff700 1 Processor -- start 2023-07-13T22:22:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.440+0000 7f9335fff700 1 -- start start 2023-07-13T22:22:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.441+0000 7f9335fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93301538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.441+0000 7f9335fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d12b0 0x7f9330152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.441+0000 7f9335fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300d3220 0x7f9330159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.441+0000 7f9335fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93300cacc0 con 0x7f93300d12b0 2023-07-13T22:22:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.441+0000 7f9335fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93300cba50 con 0x7f93300d3220 2023-07-13T22:22:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.441+0000 7f9335fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933015a180 con 0x7f93300d09a0 2023-07-13T22:22:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.441+0000 7f932effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d12b0 0x7f9330152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.441+0000 7f932effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d12b0 0x7f9330152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50920/0 (socket says 172.21.15.60:50920) 2023-07-13T22:22:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.441+0000 7f932effd700 1 -- 172.21.15.60:0/878616038 learned_addr learned my addr 172.21.15.60:0/878616038 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932f7fe700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93301538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932ffff700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300d3220 0x7f9330159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932effd700 1 -- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 msgr2=0x7f93301538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932effd700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93301538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932effd700 1 -- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300d3220 msgr2=0x7f9330159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932effd700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300d3220 0x7f9330159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932effd700 1 -- 172.21.15.60:0/878616038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324009550 con 0x7f93300d12b0 2023-07-13T22:22:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932f7fe700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93301538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932ffff700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300d3220 0x7f9330159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f932effd700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d12b0 0x7f9330152380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f93300722c0 tx=0x7f9324004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f9317fff700 1 -- 172.21.15.60:0/878616038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324004540 con 0x7f93300d12b0 2023-07-13T22:22:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f9317fff700 1 -- 172.21.15.60:0/878616038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93240036a0 con 0x7f93300d12b0 2023-07-13T22:22:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f9317fff700 1 -- 172.21.15.60:0/878616038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324003a50 con 0x7f93300d12b0 2023-07-13T22:22:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93300ca9e0 con 0x7f93300d12b0 2023-07-13T22:22:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.442+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93300cb6e0 con 0x7f93300d12b0 2023-07-13T22:22:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.443+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93300cb260 con 0x7f93300d12b0 2023-07-13T22:22:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.444+0000 7f9317fff700 1 -- 172.21.15.60:0/878616038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9324021070 con 0x7f93300d12b0 2023-07-13T22:22:02.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.445+0000 7f9317fff700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f931c06ace0 0x7f931c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.445+0000 7f9317fff700 1 -- 172.21.15.60:0/878616038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f932408abf0 con 0x7f93300d12b0 2023-07-13T22:22:02.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.445+0000 7f932f7fe700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f931c06ace0 0x7f931c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.446+0000 7f932f7fe700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f931c06ace0 0x7f931c06d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93301f51f0 tx=0x7f9320008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:02.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.449+0000 7f9317fff700 1 -- 172.21.15.60:0/878616038 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f93300cb260 con 0x7f93300d12b0 2023-07-13T22:22:02.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.655+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93300cb260 con 0x7f93300d12b0 2023-07-13T22:22:02.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.655+0000 7f9317fff700 1 -- 172.21.15.60:0/878616038 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9324015020 con 0x7f93300d12b0 2023-07-13T22:22:02.640 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:22:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f931c06ace0 msgr2=0x7f931c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f931c06ace0 0x7f931c06d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93301f51f0 tx=0x7f9320008040 comp rx=0 tx=0).stop 2023-07-13T22:22:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d12b0 msgr2=0x7f9330152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d12b0 0x7f9330152380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f93300722c0 tx=0x7f9324004750 comp rx=0 tx=0).stop 2023-07-13T22:22:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 shutdown_connections 2023-07-13T22:22:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f931c06ace0 0x7f931c06d190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300d3220 0x7f9330159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93300d09a0 0x7f93301538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 --2- 172.21.15.60:0/878616038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93300d12b0 0x7f9330152380 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 >> 172.21.15.60:0/878616038 conn(0x7f9330071600 msgr2=0x7f93300b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:02.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 shutdown_connections 2023-07-13T22:22:02.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.658+0000 7f9335fff700 1 -- 172.21.15.60:0/878616038 wait complete. 2023-07-13T22:22:02.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:22:02.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.882+0000 7fb79e5ff700 1 Processor -- start 2023-07-13T22:22:02.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.883+0000 7fb79e5ff700 1 -- start start 2023-07-13T22:22:02.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb79e5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7980b8200 0x7fb7980cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb79e5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb7980b1ac0 0x7fb7980cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb79e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980cede0 0x7fb79815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb79e5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980d1870 con 0x7fb7980b8200 2023-07-13T22:22:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb79e5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980cf370 con 0x7fb7980b1ac0 2023-07-13T22:22:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb79e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980d12b0 con 0x7fb7980cede0 2023-07-13T22:22:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb797fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7980b8200 0x7fb7980cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb797fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7980b8200 0x7fb7980cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50936/0 (socket says 172.21.15.60:50936) 2023-07-13T22:22:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb797fff700 1 -- 172.21.15.60:0/391470228 learned_addr learned my addr 172.21.15.60:0/391470228 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb79cb9c700 1 --2- 172.21.15.60:0/391470228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980cede0 0x7fb79815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.884+0000 7fb7977fe700 1 --2- 172.21.15.60:0/391470228 >> v2:172.21.15.105:3300/0 conn(0x7fb7980b1ac0 0x7fb7980cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.885+0000 7fb797fff700 1 -- 172.21.15.60:0/391470228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980cede0 msgr2=0x7fb79815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.885+0000 7fb797fff700 1 --2- 172.21.15.60:0/391470228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980cede0 0x7fb79815ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.885+0000 7fb797fff700 1 -- 172.21.15.60:0/391470228 >> v2:172.21.15.105:3300/0 conn(0x7fb7980b1ac0 msgr2=0x7fb7980cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.885+0000 7fb797fff700 1 --2- 172.21.15.60:0/391470228 >> v2:172.21.15.105:3300/0 conn(0x7fb7980b1ac0 0x7fb7980cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.885+0000 7fb797fff700 1 -- 172.21.15.60:0/391470228 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980d1590 con 0x7fb7980b8200 2023-07-13T22:22:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.885+0000 7fb797fff700 1 --2- 172.21.15.60:0/391470228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980b8200 0x7fb7980cf640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb7980d07b0 tx=0x7fb788009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff9673b9ded56dc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.885+0000 7fb796ffd700 1 -- 172.21.15.60:0/391470228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78800b870 con 0x7fb7980b8200 2023-07-13T22:22:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.885+0000 7fb796ffd700 1 -- 172.21.15.60:0/391470228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78800b9d0 con 0x7fb7980b8200 2023-07-13T22:22:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.885+0000 7fb796ffd700 1 -- 172.21.15.60:0/391470228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78800bd60 con 0x7fb7980b8200 2023-07-13T22:22:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.886+0000 7fb79e5ff700 1 -- 172.21.15.60:0/391470228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980b8200 msgr2=0x7fb7980cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.886+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/391470228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980b8200 0x7fb7980cf640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb7980d07b0 tx=0x7fb788009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.886+0000 7fb79e5ff700 1 -- 172.21.15.60:0/391470228 shutdown_connections 2023-07-13T22:22:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.886+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/391470228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980cede0 0x7fb79815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.886+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/391470228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980b8200 0x7fb7980cf640 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.886+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/391470228 >> v2:172.21.15.105:3300/0 conn(0x7fb7980b1ac0 0x7fb7980cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.886+0000 7fb79e5ff700 1 -- 172.21.15.60:0/391470228 >> 172.21.15.60:0/391470228 conn(0x7fb79806b2f0 msgr2=0x7fb79806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.886+0000 7fb79e5ff700 1 -- 172.21.15.60:0/391470228 shutdown_connections 2023-07-13T22:22:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.886+0000 7fb79e5ff700 1 -- 172.21.15.60:0/391470228 wait complete. 2023-07-13T22:22:02.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.888+0000 7fb79e5ff700 1 Processor -- start 2023-07-13T22:22:02.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.888+0000 7fb79e5ff700 1 -- start start 2023-07-13T22:22:02.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb79e5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7980b1ac0 0x7fb7981f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb79e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980b8200 0x7fb798146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb79e5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980cede0 0x7fb7980c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb79e5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798158c00 con 0x7fb7980cede0 2023-07-13T22:22:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb79e5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980c75b0 con 0x7fb7980b1ac0 2023-07-13T22:22:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb79e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980c8250 con 0x7fb7980b8200 2023-07-13T22:22:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb79cb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980cede0 0x7fb7980c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb79cb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980cede0 0x7fb7980c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50938/0 (socket says 172.21.15.60:50938) 2023-07-13T22:22:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb79cb9c700 1 -- 172.21.15.60:0/2532374165 learned_addr learned my addr 172.21.15.60:0/2532374165 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb797fff700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7980b1ac0 0x7fb7981f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.889+0000 7fb7977fe700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980b8200 0x7fb798146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb79cb9c700 1 -- 172.21.15.60:0/2532374165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980b8200 msgr2=0x7fb798146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb79cb9c700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980b8200 0x7fb798146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb79cb9c700 1 -- 172.21.15.60:0/2532374165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7980b1ac0 msgr2=0x7fb7981f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb79cb9c700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7980b1ac0 0x7fb7981f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb79cb9c700 1 -- 172.21.15.60:0/2532374165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb788009550 con 0x7fb7980cede0 2023-07-13T22:22:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb79cb9c700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980cede0 0x7fb7980c8d30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb790007e60 tx=0x7fb79000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb794ff9700 1 -- 172.21.15.60:0/2532374165 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000ed30 con 0x7fb7980cede0 2023-07-13T22:22:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb794ff9700 1 -- 172.21.15.60:0/2532374165 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790014420 con 0x7fb7980cede0 2023-07-13T22:22:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7980c8530 con 0x7fb7980cede0 2023-07-13T22:22:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb794ff9700 1 -- 172.21.15.60:0/2532374165 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900147d0 con 0x7fb7980cede0 2023-07-13T22:22:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.890+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7981f2020 con 0x7fb7980cede0 2023-07-13T22:22:02.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.891+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7980c80b0 con 0x7fb7980cede0 2023-07-13T22:22:02.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.892+0000 7fb794ff9700 1 -- 172.21.15.60:0/2532374165 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb790014930 con 0x7fb7980cede0 2023-07-13T22:22:02.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.894+0000 7fb794ff9700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb78406ace0 0x7fb78406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:02.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.894+0000 7fb794ff9700 1 -- 172.21.15.60:0/2532374165 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb790089ac0 con 0x7fb7980cede0 2023-07-13T22:22:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.894+0000 7fb797fff700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb78406ace0 0x7fb78406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.895+0000 7fb797fff700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb78406ace0 0x7fb78406d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb788000c00 tx=0x7fb788008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:02.898+0000 7fb794ff9700 1 -- 172.21.15.60:0/2532374165 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb7980c80b0 con 0x7fb7980cede0 2023-07-13T22:22:03.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.102+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7980c8a50 con 0x7fb7980cede0 2023-07-13T22:22:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.103+0000 7fb794ff9700 1 -- 172.21.15.60:0/2532374165 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7980c8a50 con 0x7fb7980cede0 2023-07-13T22:22:03.088 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:22:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb78406ace0 msgr2=0x7fb78406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb78406ace0 0x7fb78406d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb788000c00 tx=0x7fb788008030 comp rx=0 tx=0).stop 2023-07-13T22:22:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980cede0 msgr2=0x7fb7980c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980cede0 0x7fb7980c8d30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb790007e60 tx=0x7fb79000e3c0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 shutdown_connections 2023-07-13T22:22:03.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb78406ace0 0x7fb78406d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7980cede0 0x7fb7980c8d30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7980b1ac0 0x7fb7981f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 --2- 172.21.15.60:0/2532374165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7980b8200 0x7fb798146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.105+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 >> 172.21.15.60:0/2532374165 conn(0x7fb79806b2f0 msgr2=0x7fb7980d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:03.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.106+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 shutdown_connections 2023-07-13T22:22:03.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.106+0000 7fb79e5ff700 1 -- 172.21.15.60:0/2532374165 wait complete. 2023-07-13T22:22:03.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.326+0000 7f15bb7ff700 1 Processor -- start 2023-07-13T22:22:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.326+0000 7f15bb7ff700 1 -- start start 2023-07-13T22:22:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15bb7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15b40b8400 0x7f15b40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15bb7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15b40d3750 0x7f15b40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15bb7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40cfad0 0x7f15b415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15bb7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b40cff50 con 0x7f15b40b8400 2023-07-13T22:22:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15bb7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b40d0ac0 con 0x7f15b40d3750 2023-07-13T22:22:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15bb7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40b87d0 con 0x7f15b40cfad0 2023-07-13T22:22:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15b959b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15b40b8400 0x7f15b40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15b959b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15b40b8400 0x7f15b40cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50954/0 (socket says 172.21.15.60:50954) 2023-07-13T22:22:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15b959b700 1 -- 172.21.15.60:0/2686497067 learned_addr learned my addr 172.21.15.60:0/2686497067 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:03.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15b8d9a700 1 --2- 172.21.15.60:0/2686497067 >> v2:172.21.15.105:3300/0 conn(0x7f15b40d3750 0x7f15b40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.327+0000 7f15b9d9c700 1 --2- 172.21.15.60:0/2686497067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40cfad0 0x7f15b415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.328+0000 7f15b959b700 1 -- 172.21.15.60:0/2686497067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40cfad0 msgr2=0x7f15b415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:03.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.328+0000 7f15b959b700 1 --2- 172.21.15.60:0/2686497067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40cfad0 0x7f15b415fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.328+0000 7f15b959b700 1 -- 172.21.15.60:0/2686497067 >> v2:172.21.15.105:3300/0 conn(0x7f15b40d3750 msgr2=0x7f15b40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:03.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.328+0000 7f15b959b700 1 --2- 172.21.15.60:0/2686497067 >> v2:172.21.15.105:3300/0 conn(0x7f15b40d3750 0x7f15b40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.328+0000 7f15b959b700 1 -- 172.21.15.60:0/2686497067 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40b4eb0 con 0x7f15b40b8400 2023-07-13T22:22:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.328+0000 7f15b959b700 1 --2- 172.21.15.60:0/2686497067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40b8400 0x7f15b40cd240 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f15b40b8cf0 tx=0x7f15b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7195bc077481ed6c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.328+0000 7f15abfff700 1 -- 172.21.15.60:0/2686497067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b000b870 con 0x7f15b40b8400 2023-07-13T22:22:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.328+0000 7f15abfff700 1 -- 172.21.15.60:0/2686497067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b000b9d0 con 0x7f15b40b8400 2023-07-13T22:22:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.328+0000 7f15abfff700 1 -- 172.21.15.60:0/2686497067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b000bd60 con 0x7f15b40b8400 2023-07-13T22:22:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.329+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2686497067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40b8400 msgr2=0x7f15b40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.329+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2686497067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40b8400 0x7f15b40cd240 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f15b40b8cf0 tx=0x7f15b0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.329+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2686497067 shutdown_connections 2023-07-13T22:22:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.329+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2686497067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40cfad0 0x7f15b415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.329+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2686497067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40b8400 0x7f15b40cd240 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.329+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2686497067 >> v2:172.21.15.105:3300/0 conn(0x7f15b40d3750 0x7f15b40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.329+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2686497067 >> 172.21.15.60:0/2686497067 conn(0x7f15b406b390 msgr2=0x7f15b406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.329+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2686497067 shutdown_connections 2023-07-13T22:22:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.329+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2686497067 wait complete. 2023-07-13T22:22:03.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.331+0000 7f15bb7ff700 1 Processor -- start 2023-07-13T22:22:03.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.331+0000 7f15bb7ff700 1 -- start start 2023-07-13T22:22:03.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15bb7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40b8400 0x7f15b414cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15bb7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40cfad0 0x7f15b40c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15bb7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15b40d3750 0x7f15b40c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15bb7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b414d4e0 con 0x7f15b40cfad0 2023-07-13T22:22:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15bb7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40c89c0 con 0x7f15b40d3750 2023-07-13T22:22:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15bb7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40ca7a0 con 0x7f15b40b8400 2023-07-13T22:22:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15b8d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40cfad0 0x7f15b40c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15b8d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40cfad0 0x7f15b40c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50958/0 (socket says 172.21.15.60:50958) 2023-07-13T22:22:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15b8d9a700 1 -- 172.21.15.60:0/2905362492 learned_addr learned my addr 172.21.15.60:0/2905362492 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15b9d9c700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15b40d3750 0x7f15b40c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15b959b700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40b8400 0x7f15b414cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15b8d9a700 1 -- 172.21.15.60:0/2905362492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40b8400 msgr2=0x7f15b414cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15b8d9a700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40b8400 0x7f15b414cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.332+0000 7f15b8d9a700 1 -- 172.21.15.60:0/2905362492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15b40d3750 msgr2=0x7f15b40c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.333+0000 7f15b8d9a700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15b40d3750 0x7f15b40c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.333+0000 7f15b8d9a700 1 -- 172.21.15.60:0/2905362492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0009550 con 0x7f15b40cfad0 2023-07-13T22:22:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.333+0000 7f15b959b700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40b8400 0x7f15b414cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.333+0000 7f15b8d9a700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40cfad0 0x7f15b40c8480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f15a400d800 tx=0x7f15a400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.333+0000 7f15a9ffb700 1 -- 172.21.15.60:0/2905362492 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40125f0 con 0x7f15b40cfad0 2023-07-13T22:22:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.333+0000 7f15a9ffb700 1 -- 172.21.15.60:0/2905362492 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a4012750 con 0x7f15b40cfad0 2023-07-13T22:22:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.333+0000 7f15a9ffb700 1 -- 172.21.15.60:0/2905362492 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4012ae0 con 0x7f15b40cfad0 2023-07-13T22:22:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.333+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b40c79a0 con 0x7f15b40cfad0 2023-07-13T22:22:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.333+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b40c7f60 con 0x7f15b40cfad0 2023-07-13T22:22:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.334+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b40c8120 con 0x7f15b40cfad0 2023-07-13T22:22:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.335+0000 7f15a9ffb700 1 -- 172.21.15.60:0/2905362492 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15a4012c40 con 0x7f15b40cfad0 2023-07-13T22:22:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.336+0000 7f15a9ffb700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f159806ace0 0x7f159806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.336+0000 7f15a9ffb700 1 -- 172.21.15.60:0/2905362492 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f15a4089850 con 0x7f15b40cfad0 2023-07-13T22:22:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.336+0000 7f15b959b700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f159806ace0 0x7f159806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.337+0000 7f15b959b700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f159806ace0 0x7f159806d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f15b0014040 tx=0x7f15b0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.340+0000 7f15a9ffb700 1 -- 172.21.15.60:0/2905362492 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f15a4054820 con 0x7f15b40cfad0 2023-07-13T22:22:03.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.595+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15b415abf0 con 0x7f15b40cfad0 2023-07-13T22:22:03.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.596+0000 7f15a9ffb700 1 -- 172.21.15.60:0/2905362492 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f15a4057e40 con 0x7f15b40cfad0 2023-07-13T22:22:03.580 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:03.581 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2550952354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2550952354},{"type":"v1","addr":"172.21.15.60:6837","nonce":2550952354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4134045869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4134045869},{"type":"v1","addr":"172.21.15.60:6836","nonce":4134045869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2202330082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2202330082},{"type":"v1","addr":"172.21.15.105:6834","nonce":2202330082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:20:54.038443+0000","modified":"2023-07-13T22:20:57.536915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/453882022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":453882022},{"type":"v1","addr":"172.21.15.105:6835","nonce":453882022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:22:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f159806ace0 msgr2=0x7f159806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f159806ace0 0x7f159806d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f15b0014040 tx=0x7f15b0008030 comp rx=0 tx=0).stop 2023-07-13T22:22:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40cfad0 msgr2=0x7f15b40c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40cfad0 0x7f15b40c8480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f15a400d800 tx=0x7f15a400dbc0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 shutdown_connections 2023-07-13T22:22:03.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f159806ace0 0x7f159806d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15b40d3750 0x7f15b40c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b40b8400 0x7f15b414cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 --2- 172.21.15.60:0/2905362492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15b40cfad0 0x7f15b40c8480 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 >> 172.21.15.60:0/2905362492 conn(0x7f15b406b390 msgr2=0x7f15b40b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:03.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 shutdown_connections 2023-07-13T22:22:03.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.598+0000 7f15bb7ff700 1 -- 172.21.15.60:0/2905362492 wait complete. 2023-07-13T22:22:03.585 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2023-07-13T22:22:03.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.820+0000 7f5144dff700 1 Processor -- start 2023-07-13T22:22:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.820+0000 7f5144dff700 1 -- start start 2023-07-13T22:22:03.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.821+0000 7f5144dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51400cfca0 0x7f5140167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.821+0000 7f5144dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51400cd950 con 0x7f51400d3750 2023-07-13T22:22:03.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.821+0000 7f5144dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51400d0ac0 con 0x7f51400b8400 2023-07-13T22:22:03.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.821+0000 7f5144dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51400b87d0 con 0x7f51400cfca0 2023-07-13T22:22:03.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.821+0000 7f513dd9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f51400d3750 0x7f51400cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34080/0 (socket says 172.21.15.60:34080) 2023-07-13T22:22:03.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513dd9b700 1 -- 172.21.15.60:0/3614906770 learned_addr learned my addr 172.21.15.60:0/3614906770 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513ed9d700 1 --2- 172.21.15.60:0/3614906770 >> v2:172.21.15.105:3300/0 conn(0x7f51400cfca0 0x7f5140167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173432051 0 0) 0x7f5134001110 con 0x7f51400d3750 2023-07-13T22:22:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51280035b0 con 0x7f51400d3750 2023-07-13T22:22:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533335245 0 0) 0x7f51400d0ac0 con 0x7f51400b8400 2023-07-13T22:22:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5134001110 con 0x7f51400b8400 2023-07-13T22:22:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2176851383 0 0) 0x7f51280035b0 con 0x7f51400d3750 2023-07-13T22:22:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 >> v2:172.21.15.105:3300/0 conn(0x7f51400cfca0 msgr2=0x7f5140167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 --2- 172.21.15.60:0/3614906770 >> v2:172.21.15.105:3300/0 conn(0x7f51400cfca0 0x7f5140167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 >> v1:172.21.15.105:6789/0 conn(0x7f51400b8400 legacy=0x7f51400cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51400b4eb0 con 0x7f51400d3750 2023-07-13T22:22:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f5134001800 con 0x7f51400d3750 2023-07-13T22:22:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 >> v1:172.21.15.60:6789/0 conn(0x7f51400d3750 legacy=0x7f51400cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5128003b90 0x7f5128003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f5128004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5128005200 0x7f512800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51400b87d0 con 0x7f5128004850 2023-07-13T22:22:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512800c110 con 0x7f5128003b90 2023-07-13T22:22:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.822+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512800c2d0 con 0x7f5128005200 2023-07-13T22:22:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513ed9d700 1 --2- 172.21.15.60:0/3614906770 >> v2:172.21.15.105:3300/0 conn(0x7f51400cfca0 0x7f5140167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513dd9b700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f5128004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513e59c700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5128003b90 0x7f5128003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513ed9d700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5128005200 0x7f512800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513dd9b700 1 -- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5128005200 msgr2=0x7f512800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513dd9b700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5128005200 0x7f512800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513dd9b700 1 -- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5128003b90 msgr2=0x7f5128003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513dd9b700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5128003b90 0x7f5128003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513dd9b700 1 -- 172.21.15.60:0/3614906770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280035b0 con 0x7f5128004850 2023-07-13T22:22:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513e59c700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5128003b90 0x7f5128003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513ed9d700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5128005200 0x7f512800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513ed9d700 1 -- 172.21.15.60:0/3614906770 reap_dead start 2023-07-13T22:22:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.823+0000 7f513dd9b700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f5128004cc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5134001eb0 tx=0x7f5128005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72beff4f67dd396d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400ccf0 con 0x7f5128004850 2023-07-13T22:22:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513400ce50 con 0x7f5128004850 2023-07-13T22:22:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f513d59a700 1 -- 172.21.15.60:0/3614906770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134005ba0 con 0x7f5128004850 2023-07-13T22:22:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f5144dff700 1 -- 172.21.15.60:0/3614906770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 msgr2=0x7f5128004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f5144dff700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f5128004cc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5134001eb0 tx=0x7f5128005880 comp rx=0 tx=0).stop 2023-07-13T22:22:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f5144dff700 1 -- 172.21.15.60:0/3614906770 shutdown_connections 2023-07-13T22:22:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f5144dff700 1 --2- 172.21.15.60:0/3614906770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f5128004cc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f5144dff700 1 -- 172.21.15.60:0/3614906770 >> 172.21.15.60:0/3614906770 conn(0x7f514006b390 msgr2=0x7f514006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f5144dff700 1 -- 172.21.15.60:0/3614906770 shutdown_connections 2023-07-13T22:22:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.824+0000 7f5144dff700 1 -- 172.21.15.60:0/3614906770 wait complete. 2023-07-13T22:22:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.826+0000 7f5144dff700 1 Processor -- start 2023-07-13T22:22:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.826+0000 7f5144dff700 1 -- start start 2023-07-13T22:22:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.827+0000 7f5144dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f51401f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.827+0000 7f5144dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5140149cd0 0x7f514015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.827+0000 7f5144dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f514015cea0 0x7f514015d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.827+0000 7f5144dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51400cb8b0 con 0x7f5128004850 2023-07-13T22:22:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.827+0000 7f5144dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514015d850 con 0x7f5140149cd0 2023-07-13T22:22:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.827+0000 7f5144dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514015c000 con 0x7f514015cea0 2023-07-13T22:22:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.827+0000 7f513e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f51401f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.827+0000 7f513e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f51401f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50984/0 (socket says 172.21.15.60:50984) 2023-07-13T22:22:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.827+0000 7f513e59c700 1 -- 172.21.15.60:0/2972205220 learned_addr learned my addr 172.21.15.60:0/2972205220 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513dd9b700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5140149cd0 0x7f514015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513ed9d700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f514015cea0 0x7f514015d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513e59c700 1 -- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f514015cea0 msgr2=0x7f514015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513e59c700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f514015cea0 0x7f514015d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513e59c700 1 -- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5140149cd0 msgr2=0x7f514015bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513e59c700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5140149cd0 0x7f514015bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513e59c700 1 -- 172.21.15.60:0/2972205220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514015c240 con 0x7f5128004850 2023-07-13T22:22:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513ed9d700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f514015cea0 0x7f514015d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513dd9b700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5140149cd0 0x7f514015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f513e59c700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f51401f5990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f513000e780 tx=0x7f513000eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f512effd700 1 -- 172.21.15.60:0/2972205220 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51300052e0 con 0x7f5128004850 2023-07-13T22:22:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f512effd700 1 -- 172.21.15.60:0/2972205220 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5130005440 con 0x7f5128004850 2023-07-13T22:22:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f512effd700 1 -- 172.21.15.60:0/2972205220 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130003880 con 0x7f5128004850 2023-07-13T22:22:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.828+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514015c520 con 0x7f5128004850 2023-07-13T22:22:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.829+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514015cae0 con 0x7f5128004850 2023-07-13T22:22:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.830+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51400d2140 con 0x7f5128004850 2023-07-13T22:22:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.830+0000 7f512effd700 1 -- 172.21.15.60:0/2972205220 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51300039e0 con 0x7f5128004850 2023-07-13T22:22:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.831+0000 7f512effd700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512006ace0 0x7f512006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.831+0000 7f512effd700 1 -- 172.21.15.60:0/2972205220 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5130088aa0 con 0x7f5128004850 2023-07-13T22:22:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.832+0000 7f513dd9b700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512006ace0 0x7f512006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.832+0000 7f513dd9b700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512006ace0 0x7f512006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51401f4dc0 tx=0x7f513400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:03.836+0000 7f512effd700 1 -- 172.21.15.60:0/2972205220 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5130053a80 con 0x7f5128004850 2023-07-13T22:22:04.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.044+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f514004e280 con 0x7f5128004850 2023-07-13T22:22:04.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.046+0000 7f512effd700 1 -- 172.21.15.60:0/2972205220 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f51300570a0 con 0x7f5128004850 2023-07-13T22:22:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:04.031 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:21:42.683944+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.048+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512006ace0 msgr2=0x7f512006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.048+0000 7f5144dff700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512006ace0 0x7f512006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51401f4dc0 tx=0x7f513400a000 comp rx=0 tx=0).stop 2023-07-13T22:22:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.048+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 msgr2=0x7f51401f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.048+0000 7f5144dff700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f51401f5990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f513000e780 tx=0x7f513000eb40 comp rx=0 tx=0).stop 2023-07-13T22:22:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.048+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 shutdown_connections 2023-07-13T22:22:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.048+0000 7f5144dff700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512006ace0 0x7f512006d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.048+0000 7f5144dff700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5128004850 0x7f51401f5990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.048+0000 7f5144dff700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f514015cea0 0x7f514015d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.049+0000 7f5144dff700 1 --2- 172.21.15.60:0/2972205220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5140149cd0 0x7f514015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.049+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 >> 172.21.15.60:0/2972205220 conn(0x7f514006b390 msgr2=0x7f51400b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.049+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 shutdown_connections 2023-07-13T22:22:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.049+0000 7f5144dff700 1 -- 172.21.15.60:0/2972205220 wait complete. 2023-07-13T22:22:04.051 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:22:04.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:04.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.269+0000 7fc7ac3ff700 1 Processor -- start 2023-07-13T22:22:04.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.269+0000 7fc7ac3ff700 1 -- start start 2023-07-13T22:22:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7ac3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a4154950 0x7fc7a4070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7ac3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc7a415a820 0x7fc7a415ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7ac3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7a40c8d40 0x7fc7a40c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7ac3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a4070270 con 0x7fc7a40c8d40 2023-07-13T22:22:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7ac3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a40c96c0 con 0x7fc7a415a820 2023-07-13T22:22:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7ac3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a415b2e0 con 0x7fc7a4154950 2023-07-13T22:22:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7aa99c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7a40c8d40 0x7fc7a40c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7aa99c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7a40c8d40 0x7fc7a40c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50994/0 (socket says 172.21.15.60:50994) 2023-07-13T22:22:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7aa99c700 1 -- 172.21.15.60:0/515963150 learned_addr learned my addr 172.21.15.60:0/515963150 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7aa19b700 1 --2- 172.21.15.60:0/515963150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a4154950 0x7fc7a4070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.271+0000 7fc7a999a700 1 --2- 172.21.15.60:0/515963150 >> v2:172.21.15.105:3300/0 conn(0x7fc7a415a820 0x7fc7a415ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.272+0000 7fc7aa99c700 1 -- 172.21.15.60:0/515963150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a4154950 msgr2=0x7fc7a4070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.272+0000 7fc7aa99c700 1 --2- 172.21.15.60:0/515963150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a4154950 0x7fc7a4070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.272+0000 7fc7aa99c700 1 -- 172.21.15.60:0/515963150 >> v2:172.21.15.105:3300/0 conn(0x7fc7a415a820 msgr2=0x7fc7a415ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.272+0000 7fc7aa99c700 1 --2- 172.21.15.60:0/515963150 >> v2:172.21.15.105:3300/0 conn(0x7fc7a415a820 0x7fc7a415ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.272+0000 7fc7aa99c700 1 -- 172.21.15.60:0/515963150 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a415b5c0 con 0x7fc7a40c8d40 2023-07-13T22:22:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.272+0000 7fc7aa99c700 1 --2- 172.21.15.60:0/515963150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a40c8d40 0x7fc7a40c9180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4070060 tx=0x7fc79c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3566cc348a369ec0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.272+0000 7fc7a9199700 1 -- 172.21.15.60:0/515963150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00e070 con 0x7fc7a40c8d40 2023-07-13T22:22:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.272+0000 7fc7a9199700 1 -- 172.21.15.60:0/515963150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c00b9c0 con 0x7fc7a40c8d40 2023-07-13T22:22:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.272+0000 7fc7a9199700 1 -- 172.21.15.60:0/515963150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00bd70 con 0x7fc7a40c8d40 2023-07-13T22:22:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.273+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/515963150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a40c8d40 msgr2=0x7fc7a40c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.273+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/515963150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a40c8d40 0x7fc7a40c9180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4070060 tx=0x7fc79c00b330 comp rx=0 tx=0).stop 2023-07-13T22:22:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.273+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/515963150 shutdown_connections 2023-07-13T22:22:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.273+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/515963150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a40c8d40 0x7fc7a40c9180 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.273+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/515963150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a4154950 0x7fc7a4070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.273+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/515963150 >> v2:172.21.15.105:3300/0 conn(0x7fc7a415a820 0x7fc7a415ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.273+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/515963150 >> 172.21.15.60:0/515963150 conn(0x7fc7a40cf410 msgr2=0x7fc7a40cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.273+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/515963150 shutdown_connections 2023-07-13T22:22:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.273+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/515963150 wait complete. 2023-07-13T22:22:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.274+0000 7fc7ac3ff700 1 Processor -- start 2023-07-13T22:22:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.275+0000 7fc7ac3ff700 1 -- start start 2023-07-13T22:22:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7ac3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40c8d40 0x7fc7a4142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7ac3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a4154950 0x7fc7a41475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7ac3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a415a820 0x7fc7a4148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7ac3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40b16d0 con 0x7fc7a4154950 2023-07-13T22:22:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7ac3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a406d8e0 con 0x7fc7a415a820 2023-07-13T22:22:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7ac3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a406dbc0 con 0x7fc7a40c8d40 2023-07-13T22:22:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7a999a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a4154950 0x7fc7a41475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7a999a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a4154950 0x7fc7a41475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50996/0 (socket says 172.21.15.60:50996) 2023-07-13T22:22:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7a999a700 1 -- 172.21.15.60:0/1072052712 learned_addr learned my addr 172.21.15.60:0/1072052712 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7aa19b700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40c8d40 0x7fc7a4142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7aa99c700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a415a820 0x7fc7a4148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7a999a700 1 -- 172.21.15.60:0/1072052712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40c8d40 msgr2=0x7fc7a4142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7a999a700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40c8d40 0x7fc7a4142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7a999a700 1 -- 172.21.15.60:0/1072052712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a415a820 msgr2=0x7fc7a4148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7a999a700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a415a820 0x7fc7a4148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7a999a700 1 -- 172.21.15.60:0/1072052712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c009b30 con 0x7fc7a4154950 2023-07-13T22:22:04.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.276+0000 7fc7aa19b700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40c8d40 0x7fc7a4142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:04.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.277+0000 7fc7a999a700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a4154950 0x7fc7a41475a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc79400e810 tx=0x7fc79400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:04.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.277+0000 7fc79affd700 1 -- 172.21.15.60:0/1072052712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794013610 con 0x7fc7a4154950 2023-07-13T22:22:04.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.277+0000 7fc79affd700 1 -- 172.21.15.60:0/1072052712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc794013770 con 0x7fc7a4154950 2023-07-13T22:22:04.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.277+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a406dea0 con 0x7fc7a4154950 2023-07-13T22:22:04.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.277+0000 7fc79affd700 1 -- 172.21.15.60:0/1072052712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794013b20 con 0x7fc7a4154950 2023-07-13T22:22:04.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.277+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a40b4060 con 0x7fc7a4154950 2023-07-13T22:22:04.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.278+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a406e4f0 con 0x7fc7a4154950 2023-07-13T22:22:04.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.279+0000 7fc79affd700 1 -- 172.21.15.60:0/1072052712 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc794013c80 con 0x7fc7a4154950 2023-07-13T22:22:04.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.280+0000 7fc79affd700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc78806ace0 0x7fc78806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.280+0000 7fc79affd700 1 -- 172.21.15.60:0/1072052712 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc7940891b0 con 0x7fc7a4154950 2023-07-13T22:22:04.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.280+0000 7fc7aa19b700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc78806ace0 0x7fc78806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.281+0000 7fc7aa19b700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc78806ace0 0x7fc78806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0003950 tx=0x7fc7a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.284+0000 7fc79affd700 1 -- 172.21.15.60:0/1072052712 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc794057c20 con 0x7fc7a4154950 2023-07-13T22:22:04.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.542+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7a40704c0 con 0x7fc7a4154950 2023-07-13T22:22:04.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.542+0000 7fc79affd700 1 -- 172.21.15.60:0/1072052712 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fc7940577b0 con 0x7fc7a4154950 2023-07-13T22:22:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:04.527 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2550952354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2550952354},{"type":"v1","addr":"172.21.15.60:6837","nonce":2550952354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4134045869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4134045869},{"type":"v1","addr":"172.21.15.60:6836","nonce":4134045869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2202330082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2202330082},{"type":"v1","addr":"172.21.15.105:6834","nonce":2202330082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:20:54.038443+0000","modified":"2023-07-13T22:20:57.536915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/453882022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":453882022},{"type":"v1","addr":"172.21.15.105:6835","nonce":453882022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:22:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc78806ace0 msgr2=0x7fc78806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc78806ace0 0x7fc78806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0003950 tx=0x7fc7a0008040 comp rx=0 tx=0).stop 2023-07-13T22:22:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a4154950 msgr2=0x7fc7a41475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a4154950 0x7fc7a41475a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc79400e810 tx=0x7fc79400eb20 comp rx=0 tx=0).stop 2023-07-13T22:22:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 shutdown_connections 2023-07-13T22:22:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc78806ace0 0x7fc78806d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a415a820 0x7fc7a4148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40c8d40 0x7fc7a4142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 --2- 172.21.15.60:0/1072052712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7a4154950 0x7fc7a41475a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 >> 172.21.15.60:0/1072052712 conn(0x7fc7a40cf410 msgr2=0x7fc7a40adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:04.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 shutdown_connections 2023-07-13T22:22:04.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.545+0000 7fc7ac3ff700 1 -- 172.21.15.60:0/1072052712 wait complete. 2023-07-13T22:22:04.531 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2023-07-13T22:22:04.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:04.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.766+0000 7fa6fafff700 1 Processor -- start 2023-07-13T22:22:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.766+0000 7fa6fafff700 1 -- start start 2023-07-13T22:22:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.767+0000 7fa6fafff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40b8400 0x7fa6f40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.767+0000 7fa6fafff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f40d0ac0 con 0x7fa6f40cfca0 2023-07-13T22:22:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.767+0000 7fa6fafff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f40b87d0 con 0x7fa6f40d3750 2023-07-13T22:22:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.767+0000 7fa6fafff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40b4eb0 con 0x7fa6f40b8400 2023-07-13T22:22:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.767+0000 7fa6f959c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa6f40cfca0 0x7fa6f40d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34094/0 (socket says 172.21.15.60:34094) 2023-07-13T22:22:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f959c700 1 -- 172.21.15.60:0/2510721577 learned_addr learned my addr 172.21.15.60:0/2510721577 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f8d9b700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40b8400 0x7fa6f40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197098620 0 0) 0x7fa6f40d0ac0 con 0x7fa6f40cfca0 2023-07-13T22:22:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e40035b0 con 0x7fa6f40cfca0 2023-07-13T22:22:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621456536 0 0) 0x7fa6f40b87d0 con 0x7fa6f40d3750 2023-07-13T22:22:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f40d0ac0 con 0x7fa6f40d3750 2023-07-13T22:22:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1845728733 0 0) 0x7fa6e40035b0 con 0x7fa6f40cfca0 2023-07-13T22:22:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40b8400 msgr2=0x7fa6f40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40b8400 0x7fa6f40cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 >> v1:172.21.15.105:6789/0 conn(0x7fa6f40d3750 legacy=0x7fa6f40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f40b5190 con 0x7fa6f40cfca0 2023-07-13T22:22:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fa6ec0019f0 con 0x7fa6f40cfca0 2023-07-13T22:22:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 >> v1:172.21.15.60:6789/0 conn(0x7fa6f40cfca0 legacy=0x7fa6f40d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40b8400 0x7fa6e4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e4004170 0x7fa6e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e4004d90 0x7fa6e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40b4eb0 con 0x7fa6e4004170 2023-07-13T22:22:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e400bba0 con 0x7fa6e4004d90 2023-07-13T22:22:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.768+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e400bd60 con 0x7fa6f40b8400 2023-07-13T22:22:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.769+0000 7fa6f8d9b700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40b8400 0x7fa6e4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.769+0000 7fa6f3fff700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e4004170 0x7fa6e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.769+0000 7fa6f959c700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e4004d90 0x7fa6e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.769+0000 7fa6f8d9b700 1 -- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e4004d90 msgr2=0x7fa6e4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.769+0000 7fa6f8d9b700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6e4004d90 0x7fa6e4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.769+0000 7fa6f8d9b700 1 -- 172.21.15.60:0/2510721577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e4004170 msgr2=0x7fa6e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.769+0000 7fa6f8d9b700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e4004170 0x7fa6e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.769+0000 7fa6f8d9b700 1 -- 172.21.15.60:0/2510721577 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e40035b0 con 0x7fa6f40b8400 2023-07-13T22:22:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.770+0000 7fa6f8d9b700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40b8400 0x7fa6e4003bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009b60 tx=0x7fa6e4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed0192ccd0971b81 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:04.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.770+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800e6c0 con 0x7fa6f40b8400 2023-07-13T22:22:04.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.770+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e800e820 con 0x7fa6f40b8400 2023-07-13T22:22:04.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.770+0000 7fa6f37fe700 1 -- 172.21.15.60:0/2510721577 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800eb90 con 0x7fa6f40b8400 2023-07-13T22:22:04.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.770+0000 7fa6fafff700 1 -- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40b8400 msgr2=0x7fa6e4003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.770+0000 7fa6fafff700 1 --2- 172.21.15.60:0/2510721577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40b8400 0x7fa6e4003bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009b60 tx=0x7fa6e4005880 comp rx=0 tx=0).stop 2023-07-13T22:22:04.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.770+0000 7fa6f959c700 1 -- 172.21.15.60:0/2510721577 reap_dead start 2023-07-13T22:22:04.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.771+0000 7fa6fafff700 1 -- 172.21.15.60:0/2510721577 shutdown_connections 2023-07-13T22:22:04.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.771+0000 7fa6fafff700 1 -- 172.21.15.60:0/2510721577 >> 172.21.15.60:0/2510721577 conn(0x7fa6f406b390 msgr2=0x7fa6f406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.771+0000 7fa6fafff700 1 -- 172.21.15.60:0/2510721577 shutdown_connections 2023-07-13T22:22:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.771+0000 7fa6fafff700 1 -- 172.21.15.60:0/2510721577 wait complete. 2023-07-13T22:22:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.772+0000 7fa6fafff700 1 Processor -- start 2023-07-13T22:22:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.773+0000 7fa6fafff700 1 -- start start 2023-07-13T22:22:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.773+0000 7fa6fafff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f40b8400 0x7fa6f4153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6fafff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6f40c8760 0x7fa6f415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6fafff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40c8bd0 0x7fa6f415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6fafff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40c79a0 con 0x7fa6f40c8760 2023-07-13T22:22:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6fafff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40c7c80 con 0x7fa6f40b8400 2023-07-13T22:22:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6fafff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40c7f60 con 0x7fa6f40c8bd0 2023-07-13T22:22:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6f40c8760 0x7fa6f415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6f40c8760 0x7fa6f415c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51018/0 (socket says 172.21.15.60:51018) 2023-07-13T22:22:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f3fff700 1 -- 172.21.15.60:0/1234918440 learned_addr learned my addr 172.21.15.60:0/1234918440 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:04.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f8d9b700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f40b8400 0x7fa6f4153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f959c700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40c8bd0 0x7fa6f415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f3fff700 1 -- 172.21.15.60:0/1234918440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40c8bd0 msgr2=0x7fa6f415c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f3fff700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40c8bd0 0x7fa6f415c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f3fff700 1 -- 172.21.15.60:0/1234918440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f40b8400 msgr2=0x7fa6f4153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:04.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f3fff700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f40b8400 0x7fa6f4153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:04.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.774+0000 7fa6f3fff700 1 -- 172.21.15.60:0/1234918440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f40c8240 con 0x7fa6f40c8760 2023-07-13T22:22:04.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.775+0000 7fa6f3fff700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6f40c8760 0x7fa6f415c370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000f7a0 tx=0x7fa6e000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:04.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.775+0000 7fa6f17fa700 1 -- 172.21.15.60:0/1234918440 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0005320 con 0x7fa6f40c8760 2023-07-13T22:22:04.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.775+0000 7fa6f17fa700 1 -- 172.21.15.60:0/1234918440 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e00054a0 con 0x7fa6f40c8760 2023-07-13T22:22:04.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.775+0000 7fa6f17fa700 1 -- 172.21.15.60:0/1234918440 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0006aa0 con 0x7fa6f40c8760 2023-07-13T22:22:04.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.775+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f4154440 con 0x7fa6f40c8760 2023-07-13T22:22:04.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.775+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f40c9e30 con 0x7fa6f40c8760 2023-07-13T22:22:04.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.776+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f40c7ae0 con 0x7fa6f40c8760 2023-07-13T22:22:04.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.777+0000 7fa6f17fa700 1 -- 172.21.15.60:0/1234918440 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa6e0006c00 con 0x7fa6f40c8760 2023-07-13T22:22:04.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.779+0000 7fa6f17fa700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6d806ace0 0x7fa6d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:04.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.779+0000 7fa6f17fa700 1 -- 172.21.15.60:0/1234918440 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa6e0088940 con 0x7fa6f40c8760 2023-07-13T22:22:04.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.779+0000 7fa6f8d9b700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6d806ace0 0x7fa6d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:04.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.780+0000 7fa6f8d9b700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6d806ace0 0x7fa6d806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80090b0 tx=0x7fa6e801f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:04.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:04.783+0000 7fa6f17fa700 1 -- 172.21.15.60:0/1234918440 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa6f40c7ae0 con 0x7fa6f40c8760 2023-07-13T22:22:05.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.038+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6f40c8480 con 0x7fa6f40c8760 2023-07-13T22:22:05.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.038+0000 7fa6f17fa700 1 -- 172.21.15.60:0/1234918440 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fa6f40c8480 con 0x7fa6f40c8760 2023-07-13T22:22:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:05.023 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2550952354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2550952354},{"type":"v1","addr":"172.21.15.60:6837","nonce":2550952354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4134045869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4134045869},{"type":"v1","addr":"172.21.15.60:6836","nonce":4134045869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2202330082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2202330082},{"type":"v1","addr":"172.21.15.105:6834","nonce":2202330082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:20:54.038443+0000","modified":"2023-07-13T22:20:57.536915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/453882022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":453882022},{"type":"v1","addr":"172.21.15.105:6835","nonce":453882022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:22:05.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6d806ace0 msgr2=0x7fa6d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6d806ace0 0x7fa6d806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80090b0 tx=0x7fa6e801f040 comp rx=0 tx=0).stop 2023-07-13T22:22:05.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6f40c8760 msgr2=0x7fa6f415c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6f40c8760 0x7fa6f415c370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000f7a0 tx=0x7fa6e000fab0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 shutdown_connections 2023-07-13T22:22:05.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6d806ace0 0x7fa6d806d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6f40c8bd0 0x7fa6f415c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f40b8400 0x7fa6f4153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 --2- 172.21.15.60:0/1234918440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6f40c8760 0x7fa6f415c370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 >> 172.21.15.60:0/1234918440 conn(0x7fa6f406b390 msgr2=0x7fa6f40b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:05.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 shutdown_connections 2023-07-13T22:22:05.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.041+0000 7fa6fafff700 1 -- 172.21.15.60:0/1234918440 wait complete. 2023-07-13T22:22:05.027 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2023-07-13T22:22:05.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:05.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.262+0000 7f51b81ff700 1 Processor -- start 2023-07-13T22:22:05.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.262+0000 7f51b81ff700 1 -- start start 2023-07-13T22:22:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.263+0000 7f51b81ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00b8400 0x7f51b00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.263+0000 7f51b81ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51b00d3750 0x7f51b00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.263+0000 7f51b81ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b00cff20 con 0x7f51b00cfad0 2023-07-13T22:22:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.263+0000 7f51b81ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b00d0ac0 con 0x7f51b00d3750 2023-07-13T22:22:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.263+0000 7f51b81ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b00b87d0 con 0x7f51b00b8400 2023-07-13T22:22:05.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b5f9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00b8400 0x7f51b00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b579a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51b00d3750 0x7f51b00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b5f9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00b8400 0x7f51b00cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:39372/0 (socket says 172.21.15.60:39372) 2023-07-13T22:22:05.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b5f9b700 1 -- 172.21.15.60:0/2605824340 learned_addr learned my addr 172.21.15.60:0/2605824340 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:05.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b679c700 1 --1- 172.21.15.60:0/2605824340 >> v1:172.21.15.105:6789/0 conn(0x7f51b00cfad0 0x7f51b015fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:48660/0 (socket says 172.21.15.60:48660) 2023-07-13T22:22:05.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b4f99700 1 -- 172.21.15.60:0/2605824340 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954191205 0 0) 0x7f51a8001490 con 0x7f51b00cfad0 2023-07-13T22:22:05.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b579a700 1 -- 172.21.15.60:0/2605824340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00b8400 msgr2=0x7f51b00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:05.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b579a700 1 --2- 172.21.15.60:0/2605824340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00b8400 0x7f51b00cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b579a700 1 -- 172.21.15.60:0/2605824340 >> v1:172.21.15.105:6789/0 conn(0x7f51b00cfad0 legacy=0x7f51b015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b579a700 1 -- 172.21.15.60:0/2605824340 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b00b4eb0 con 0x7f51b00d3750 2023-07-13T22:22:05.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.264+0000 7f51b5f9b700 1 --2- 172.21.15.60:0/2605824340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00b8400 0x7f51b00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:05.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.265+0000 7f51b579a700 1 --2- 172.21.15.60:0/2605824340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00d3750 0x7f51b00cf590 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009840 tx=0x7f51a0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86671b5b9cfcaf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:05.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.265+0000 7f51b4f99700 1 -- 172.21.15.60:0/2605824340 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0011830 con 0x7f51b00d3750 2023-07-13T22:22:05.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.265+0000 7f51b4f99700 1 -- 172.21.15.60:0/2605824340 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a0011990 con 0x7f51b00d3750 2023-07-13T22:22:05.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.265+0000 7f51b4f99700 1 -- 172.21.15.60:0/2605824340 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0011d20 con 0x7f51b00d3750 2023-07-13T22:22:05.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.266+0000 7f51b81ff700 1 -- 172.21.15.60:0/2605824340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00d3750 msgr2=0x7f51b00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.266+0000 7f51b81ff700 1 --2- 172.21.15.60:0/2605824340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00d3750 0x7f51b00cf590 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009840 tx=0x7f51a0009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:05.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.266+0000 7f51b81ff700 1 -- 172.21.15.60:0/2605824340 shutdown_connections 2023-07-13T22:22:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.266+0000 7f51b81ff700 1 --2- 172.21.15.60:0/2605824340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00b8400 0x7f51b00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.266+0000 7f51b81ff700 1 --2- 172.21.15.60:0/2605824340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00d3750 0x7f51b00cf590 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.266+0000 7f51b81ff700 1 -- 172.21.15.60:0/2605824340 >> 172.21.15.60:0/2605824340 conn(0x7f51b006b390 msgr2=0x7f51b006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.266+0000 7f51b81ff700 1 -- 172.21.15.60:0/2605824340 shutdown_connections 2023-07-13T22:22:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.266+0000 7f51b81ff700 1 -- 172.21.15.60:0/2605824340 wait complete. 2023-07-13T22:22:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.267+0000 7f51b81ff700 1 Processor -- start 2023-07-13T22:22:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.268+0000 7f51b81ff700 1 -- start start 2023-07-13T22:22:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.268+0000 7f51b81ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00b8400 0x7f51b0153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.268+0000 7f51b81ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00cfad0 0x7f51b00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.268+0000 7f51b81ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51b00d3750 0x7f51b00d1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b81ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0153880 con 0x7f51b00d3750 2023-07-13T22:22:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b81ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b01e8c70 con 0x7f51b00b8400 2023-07-13T22:22:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b81ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b00cb450 con 0x7f51b00cfad0 2023-07-13T22:22:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b679c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51b00d3750 0x7f51b00d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b679c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51b00d3750 0x7f51b00d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45330/0 (socket says 172.21.15.60:45330) 2023-07-13T22:22:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b679c700 1 -- 172.21.15.60:0/1450152723 learned_addr learned my addr 172.21.15.60:0/1450152723 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b5f9b700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00b8400 0x7f51b0153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b579a700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00cfad0 0x7f51b00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b679c700 1 -- 172.21.15.60:0/1450152723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00cfad0 msgr2=0x7f51b00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b679c700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00cfad0 0x7f51b00d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b679c700 1 -- 172.21.15.60:0/1450152723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00b8400 msgr2=0x7f51b0153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b679c700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00b8400 0x7f51b0153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b679c700 1 -- 172.21.15.60:0/1450152723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac009550 con 0x7f51b00d3750 2023-07-13T22:22:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b5f9b700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00b8400 0x7f51b0153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.269+0000 7f51b679c700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51b00d3750 0x7f51b00d1490 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f51a800cbd0 tx=0x7f51a800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.270+0000 7f51a67fc700 1 -- 172.21.15.60:0/1450152723 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800aa00 con 0x7f51b00d3750 2023-07-13T22:22:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.270+0000 7f51a67fc700 1 -- 172.21.15.60:0/1450152723 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a800ab80 con 0x7f51b00d3750 2023-07-13T22:22:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.270+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a00094a0 con 0x7f51b00d3750 2023-07-13T22:22:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.270+0000 7f51a67fc700 1 -- 172.21.15.60:0/1450152723 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a80070d0 con 0x7f51b00d3750 2023-07-13T22:22:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.270+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b00cae00 con 0x7f51b00d3750 2023-07-13T22:22:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.271+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b00cb590 con 0x7f51b00d3750 2023-07-13T22:22:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.272+0000 7f51a67fc700 1 -- 172.21.15.60:0/1450152723 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51a8007230 con 0x7f51b00d3750 2023-07-13T22:22:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.273+0000 7f51a67fc700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f519406ace0 0x7f519406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.273+0000 7f51a67fc700 1 -- 172.21.15.60:0/1450152723 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f51a8089070 con 0x7f51b00d3750 2023-07-13T22:22:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.273+0000 7f51b5f9b700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f519406ace0 0x7f519406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.274+0000 7f51b5f9b700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f519406ace0 0x7f519406d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51ac006d30 tx=0x7f51ac0092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:05.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.277+0000 7f51a67fc700 1 -- 172.21.15.60:0/1450152723 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f51a80bb330 con 0x7f51b00d3750 2023-07-13T22:22:05.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.486+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51b00cff20 con 0x7f51b00d3750 2023-07-13T22:22:05.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.487+0000 7f51a67fc700 1 -- 172.21.15.60:0/1450152723 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f51b00cff20 con 0x7f51b00d3750 2023-07-13T22:22:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:05.472 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:21:42.683944+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:20:51.930543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-13T22:20:53.012506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:05.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f519406ace0 msgr2=0x7f519406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f519406ace0 0x7f519406d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51ac006d30 tx=0x7f51ac0092c0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51b00d3750 msgr2=0x7f51b00d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51b00d3750 0x7f51b00d1490 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f51a800cbd0 tx=0x7f51a800cee0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 shutdown_connections 2023-07-13T22:22:05.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f519406ace0 0x7f519406d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51b00d3750 0x7f51b00d1490 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51b00b8400 0x7f51b0153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 --2- 172.21.15.60:0/1450152723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51b00cfad0 0x7f51b00d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 >> 172.21.15.60:0/1450152723 conn(0x7f51b006b390 msgr2=0x7f51b015bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:05.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 shutdown_connections 2023-07-13T22:22:05.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.490+0000 7f51b81ff700 1 -- 172.21.15.60:0/1450152723 wait complete. 2023-07-13T22:22:05.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:22:05.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.713+0000 7f412adff700 1 Processor -- start 2023-07-13T22:22:05.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.713+0000 7f412adff700 1 -- start start 2023-07-13T22:22:05.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.715+0000 7f412adff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240b8570 0x7f41240ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.715+0000 7f412adff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41240d09d0 con 0x7f41240d35c0 2023-07-13T22:22:05.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.715+0000 7f412adff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41240cda10 con 0x7f41240d15c0 2023-07-13T22:22:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.715+0000 7f412adff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41240d3b30 con 0x7f41240b8570 2023-07-13T22:22:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.715+0000 7f4123fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f41240d35c0 0x7f41240d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54242/0 (socket says 172.21.15.60:54242) 2023-07-13T22:22:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.715+0000 7f4123fff700 1 -- 172.21.15.60:0/2235085597 learned_addr learned my addr 172.21.15.60:0/2235085597 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:05.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.715+0000 7f4128b9b700 1 --2- 172.21.15.60:0/2235085597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240b8570 0x7f41240ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.715+0000 7f41237fe700 1 -- 172.21.15.60:0/2235085597 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463346710 0 0) 0x7f41240d09d0 con 0x7f41240d35c0 2023-07-13T22:22:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f41237fe700 1 -- 172.21.15.60:0/2235085597 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41140035b0 con 0x7f41240d35c0 2023-07-13T22:22:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f41237fe700 1 -- 172.21.15.60:0/2235085597 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859712898 0 0) 0x7f41240cda10 con 0x7f41240d15c0 2023-07-13T22:22:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f41237fe700 1 -- 172.21.15.60:0/2235085597 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41240d09d0 con 0x7f41240d15c0 2023-07-13T22:22:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f4128b9b700 1 -- 172.21.15.60:0/2235085597 >> v1:172.21.15.105:6789/0 conn(0x7f41240d15c0 legacy=0x7f412415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f4128b9b700 1 -- 172.21.15.60:0/2235085597 >> v1:172.21.15.60:6789/0 conn(0x7f41240d35c0 legacy=0x7f41240d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f4128b9b700 1 -- 172.21.15.60:0/2235085597 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41240cd450 con 0x7f41240b8570 2023-07-13T22:22:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f4128b9b700 1 --2- 172.21.15.60:0/2235085597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240b8570 0x7f41240ceeb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4118009850 tx=0x7f4118009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7aa2486881730d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f41237fe700 1 -- 172.21.15.60:0/2235085597 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118005e30 con 0x7f41240b8570 2023-07-13T22:22:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f41237fe700 1 -- 172.21.15.60:0/2235085597 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118006030 con 0x7f41240b8570 2023-07-13T22:22:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.716+0000 7f41237fe700 1 -- 172.21.15.60:0/2235085597 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41180063e0 con 0x7f41240b8570 2023-07-13T22:22:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.717+0000 7f412adff700 1 -- 172.21.15.60:0/2235085597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240b8570 msgr2=0x7f41240ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.717+0000 7f412adff700 1 --2- 172.21.15.60:0/2235085597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240b8570 0x7f41240ceeb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4118009850 tx=0x7f4118009b60 comp rx=0 tx=0).stop 2023-07-13T22:22:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.717+0000 7f412adff700 1 -- 172.21.15.60:0/2235085597 shutdown_connections 2023-07-13T22:22:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.717+0000 7f412adff700 1 --2- 172.21.15.60:0/2235085597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240b8570 0x7f41240ceeb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.717+0000 7f412adff700 1 -- 172.21.15.60:0/2235085597 >> 172.21.15.60:0/2235085597 conn(0x7f4124071600 msgr2=0x7f41240719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.717+0000 7f412adff700 1 -- 172.21.15.60:0/2235085597 shutdown_connections 2023-07-13T22:22:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.717+0000 7f412adff700 1 -- 172.21.15.60:0/2235085597 wait complete. 2023-07-13T22:22:05.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.719+0000 7f412adff700 1 Processor -- start 2023-07-13T22:22:05.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.719+0000 7f412adff700 1 -- start start 2023-07-13T22:22:05.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f412adff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41240b8570 0x7f412415df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f412adff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41240d15c0 0x7f41240c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f412adff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240d35c0 0x7f41241e6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f412adff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41241e6340 con 0x7f41240b8570 2023-07-13T22:22:05.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f412adff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41241e5d10 con 0x7f41240d15c0 2023-07-13T22:22:05.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f412adff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412415b0f0 con 0x7f41240d35c0 2023-07-13T22:22:05.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f4128b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41240b8570 0x7f412415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f4128b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41240b8570 0x7f412415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45338/0 (socket says 172.21.15.60:45338) 2023-07-13T22:22:05.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f4128b9b700 1 -- 172.21.15.60:0/933463166 learned_addr learned my addr 172.21.15.60:0/933463166 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:05.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f4123fff700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41240d15c0 0x7f41240c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f412939c700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240d35c0 0x7f41241e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f4128b9b700 1 -- 172.21.15.60:0/933463166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240d35c0 msgr2=0x7f41241e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f4128b9b700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240d35c0 0x7f41241e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f4128b9b700 1 -- 172.21.15.60:0/933463166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41240d15c0 msgr2=0x7f41240c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f4128b9b700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41240d15c0 0x7f41240c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f4128b9b700 1 -- 172.21.15.60:0/933463166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4118009430 con 0x7f41240b8570 2023-07-13T22:22:05.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.720+0000 7f412939c700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240d35c0 0x7f41241e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:05.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.721+0000 7f4128b9b700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41240b8570 0x7f412415df30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f41240d3d60 tx=0x7f4118004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:05.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.721+0000 7f41217fa700 1 -- 172.21.15.60:0/933463166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118025430 con 0x7f41240b8570 2023-07-13T22:22:05.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.721+0000 7f41217fa700 1 -- 172.21.15.60:0/933463166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118020070 con 0x7f41240b8570 2023-07-13T22:22:05.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.721+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41241e8480 con 0x7f41240b8570 2023-07-13T22:22:05.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.721+0000 7f41217fa700 1 -- 172.21.15.60:0/933463166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41180257d0 con 0x7f41240b8570 2023-07-13T22:22:05.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.721+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41241e8150 con 0x7f41240b8570 2023-07-13T22:22:05.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.722+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41241e6480 con 0x7f41240b8570 2023-07-13T22:22:05.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.723+0000 7f41217fa700 1 -- 172.21.15.60:0/933463166 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4118022030 con 0x7f41240b8570 2023-07-13T22:22:05.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.724+0000 7f41217fa700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f410806ace0 0x7f410806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:05.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.725+0000 7f41217fa700 1 -- 172.21.15.60:0/933463166 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f411808a9c0 con 0x7f41240b8570 2023-07-13T22:22:05.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.725+0000 7f4123fff700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f410806ace0 0x7f410806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:05.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.725+0000 7f4123fff700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f410806ace0 0x7f410806d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f412415db50 tx=0x7f4110009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.728+0000 7f41217fa700 1 -- 172.21.15.60:0/933463166 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4118055a20 con 0x7f41240b8570 2023-07-13T22:22:05.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.985+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f41240cef40 con 0x7f41240b8570 2023-07-13T22:22:05.970 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:22:05.986+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:22:05.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.991+0000 7f41217fa700 1 -- 172.21.15.60:0/933463166 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4118059040 con 0x7f41240b8570 2023-07-13T22:22:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f410806ace0 msgr2=0x7f410806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f410806ace0 0x7f410806d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f412415db50 tx=0x7f4110009040 comp rx=0 tx=0).stop 2023-07-13T22:22:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41240b8570 msgr2=0x7f412415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41240b8570 0x7f412415df30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f41240d3d60 tx=0x7f4118004750 comp rx=0 tx=0).stop 2023-07-13T22:22:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 shutdown_connections 2023-07-13T22:22:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f410806ace0 0x7f410806d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41240b8570 0x7f412415df30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41240d35c0 0x7f41241e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 --2- 172.21.15.60:0/933463166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41240d15c0 0x7f41240c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 >> 172.21.15.60:0/933463166 conn(0x7f4124071600 msgr2=0x7f41240d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 shutdown_connections 2023-07-13T22:22:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:05.993+0000 7f412adff700 1 -- 172.21.15.60:0/933463166 wait complete. 2023-07-13T22:22:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:22:05.986 INFO:tasks.ceph.mds.d.smithi105.stderr: -3760> 2023-07-13T22:20:58.921+0000 7f3ffb9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:22:05.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:22:06.040 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:22:06.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.218+0000 7f6f573ff700 1 Processor -- start 2023-07-13T22:22:06.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.218+0000 7f6f573ff700 1 -- start start 2023-07-13T22:22:06.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f573ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6f50158930 0x7f6f500c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:06.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f573ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6f5006a5e0 0x7f6f5015c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f573ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5006d030 con 0x7f6f5015c4a0 2023-07-13T22:22:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f573ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f500c9620 con 0x7f6f5006a5e0 2023-07-13T22:22:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f573ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5006ca70 con 0x7f6f50158930 2023-07-13T22:22:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f5599c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6f5006a5e0 0x7f6f5015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f5599c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6f5006a5e0 0x7f6f5015c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45348/0 (socket says 172.21.15.60:45348) 2023-07-13T22:22:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f5499a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6f5015c4a0 0x7f6f50069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54244/0 (socket says 172.21.15.60:54244) 2023-07-13T22:22:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f5599c700 1 -- 172.21.15.60:0/709429252 learned_addr learned my addr 172.21.15.60:0/709429252 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f5519b700 1 --2- 172.21.15.60:0/709429252 >> v2:172.21.15.105:3300/0 conn(0x7f6f50158930 0x7f6f500c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:06.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f5599c700 1 -- 172.21.15.60:0/709429252 >> v2:172.21.15.105:3300/0 conn(0x7f6f50158930 msgr2=0x7f6f500c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:06.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f5599c700 1 --2- 172.21.15.60:0/709429252 >> v2:172.21.15.105:3300/0 conn(0x7f6f50158930 0x7f6f500c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f5599c700 1 -- 172.21.15.60:0/709429252 >> v1:172.21.15.60:6789/0 conn(0x7f6f5015c4a0 legacy=0x7f6f50069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:06.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.220+0000 7f6f5599c700 1 -- 172.21.15.60:0/709429252 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f500adc20 con 0x7f6f5006a5e0 2023-07-13T22:22:06.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.221+0000 7f6f5599c700 1 --2- 172.21.15.60:0/709429252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5006a5e0 0x7f6f5015c910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f5006f160 tx=0x7f6f4800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a74be7303349517 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.221+0000 7f6f47fff700 1 -- 172.21.15.60:0/709429252 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800ca80 con 0x7f6f5006a5e0 2023-07-13T22:22:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.221+0000 7f6f47fff700 1 -- 172.21.15.60:0/709429252 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4800cbe0 con 0x7f6f5006a5e0 2023-07-13T22:22:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.221+0000 7f6f47fff700 1 -- 172.21.15.60:0/709429252 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48008500 con 0x7f6f5006a5e0 2023-07-13T22:22:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.221+0000 7f6f573ff700 1 -- 172.21.15.60:0/709429252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5006a5e0 msgr2=0x7f6f5015c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.221+0000 7f6f573ff700 1 --2- 172.21.15.60:0/709429252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5006a5e0 0x7f6f5015c910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f5006f160 tx=0x7f6f4800c330 comp rx=0 tx=0).stop 2023-07-13T22:22:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.222+0000 7f6f573ff700 1 -- 172.21.15.60:0/709429252 shutdown_connections 2023-07-13T22:22:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.222+0000 7f6f573ff700 1 --2- 172.21.15.60:0/709429252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5006a5e0 0x7f6f5015c910 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.222+0000 7f6f573ff700 1 --2- 172.21.15.60:0/709429252 >> v2:172.21.15.105:3300/0 conn(0x7f6f50158930 0x7f6f500c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.222+0000 7f6f573ff700 1 -- 172.21.15.60:0/709429252 >> 172.21.15.60:0/709429252 conn(0x7f6f500cbf50 msgr2=0x7f6f500ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:06.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.222+0000 7f6f573ff700 1 -- 172.21.15.60:0/709429252 shutdown_connections 2023-07-13T22:22:06.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.222+0000 7f6f573ff700 1 -- 172.21.15.60:0/709429252 wait complete. 2023-07-13T22:22:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.223+0000 7f6f573ff700 1 Processor -- start 2023-07-13T22:22:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.224+0000 7f6f573ff700 1 -- start start 2023-07-13T22:22:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.224+0000 7f6f573ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f5006a5e0 0x7f6f501f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.224+0000 7f6f573ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6f50158930 0x7f6f500b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f573ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5015c4a0 0x7f6f500b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f573ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500c53b0 con 0x7f6f5015c4a0 2023-07-13T22:22:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f573ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5015b4a0 con 0x7f6f5006a5e0 2023-07-13T22:22:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f573ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50159070 con 0x7f6f50158930 2023-07-13T22:22:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5599c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5015c4a0 0x7f6f500b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5599c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5015c4a0 0x7f6f500b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45352/0 (socket says 172.21.15.60:45352) 2023-07-13T22:22:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5599c700 1 -- 172.21.15.60:0/473268632 learned_addr learned my addr 172.21.15.60:0/473268632 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5519b700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f5006a5e0 0x7f6f501f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5499a700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6f50158930 0x7f6f500b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5599c700 1 -- 172.21.15.60:0/473268632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6f50158930 msgr2=0x7f6f500b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5599c700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6f50158930 0x7f6f500b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5599c700 1 -- 172.21.15.60:0/473268632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f5006a5e0 msgr2=0x7f6f501f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5599c700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f5006a5e0 0x7f6f501f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5599c700 1 -- 172.21.15.60:0/473268632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4800ac40 con 0x7f6f5015c4a0 2023-07-13T22:22:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.225+0000 7f6f5519b700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f5006a5e0 0x7f6f501f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.226+0000 7f6f5599c700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5015c4a0 0x7f6f500b3de0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f48014000 tx=0x7f6f48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:06.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.226+0000 7f6f45ffb700 1 -- 172.21.15.60:0/473268632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800ca80 con 0x7f6f5015c4a0 2023-07-13T22:22:06.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.226+0000 7f6f45ffb700 1 -- 172.21.15.60:0/473268632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f480047f0 con 0x7f6f5015c4a0 2023-07-13T22:22:06.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.226+0000 7f6f45ffb700 1 -- 172.21.15.60:0/473268632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48008740 con 0x7f6f5015c4a0 2023-07-13T22:22:06.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.226+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5015b740 con 0x7f6f5015c4a0 2023-07-13T22:22:06.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.226+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f50159350 con 0x7f6f5015c4a0 2023-07-13T22:22:06.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.227+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5015bc60 con 0x7f6f5015c4a0 2023-07-13T22:22:06.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.228+0000 7f6f45ffb700 1 -- 172.21.15.60:0/473268632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f480088a0 con 0x7f6f5015c4a0 2023-07-13T22:22:06.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.229+0000 7f6f45ffb700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6f3406ace0 0x7f6f3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:06.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.229+0000 7f6f45ffb700 1 -- 172.21.15.60:0/473268632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f6f4808a970 con 0x7f6f5015c4a0 2023-07-13T22:22:06.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.229+0000 7f6f5519b700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6f3406ace0 0x7f6f3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:06.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.230+0000 7f6f5519b700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6f3406ace0 0x7f6f3406d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c007a50 tx=0x7f6f4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:06.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.233+0000 7f6f45ffb700 1 -- 172.21.15.60:0/473268632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6f5015bc60 con 0x7f6f5015c4a0 2023-07-13T22:22:06.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.488+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6f5015bc60 con 0x7f6f5015c4a0 2023-07-13T22:22:06.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.991+0000 7f6f45ffb700 1 -- 172.21.15.60:0/473268632 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6f5015bc60 con 0x7f6f5015c4a0 2023-07-13T22:22:06.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6f3406ace0 msgr2=0x7f6f3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:06.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6f3406ace0 0x7f6f3406d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c007a50 tx=0x7f6f4c008040 comp rx=0 tx=0).stop 2023-07-13T22:22:06.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5015c4a0 msgr2=0x7f6f500b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:06.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5015c4a0 0x7f6f500b3de0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f48014000 tx=0x7f6f48004ad0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 shutdown_connections 2023-07-13T22:22:06.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6f3406ace0 0x7f6f3406d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f5015c4a0 0x7f6f500b3de0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f5006a5e0 0x7f6f501f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 --2- 172.21.15.60:0/473268632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6f50158930 0x7f6f500b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:06.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 >> 172.21.15.60:0/473268632 conn(0x7f6f500cbf50 msgr2=0x7f6f500c82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:06.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 shutdown_connections 2023-07-13T22:22:06.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:06.994+0000 7f6f573ff700 1 -- 172.21.15.60:0/473268632 wait complete. 2023-07-13T22:22:06.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:22:07.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.221+0000 7ff44f7ff700 1 Processor -- start 2023-07-13T22:22:07.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.221+0000 7ff44f7ff700 1 -- start start 2023-07-13T22:22:07.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.222+0000 7ff44f7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff4480d0890 0x7ff4480d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:07.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.222+0000 7ff44f7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4480cdb20 con 0x7ff448159ea0 2023-07-13T22:22:07.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.222+0000 7ff44f7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44815d200 con 0x7ff4480cd6d0 2023-07-13T22:22:07.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.222+0000 7ff44f7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4480d2e60 con 0x7ff4480d0890 2023-07-13T22:22:07.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff44d59b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff448159ea0 0x7ff44815c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54254/0 (socket says 172.21.15.60:54254) 2023-07-13T22:22:07.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff44d59b700 1 -- 172.21.15.60:0/2702622679 learned_addr learned my addr 172.21.15.60:0/2702622679 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:07.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff44dd9c700 1 --2- 172.21.15.60:0/2702622679 >> v2:172.21.15.105:3300/0 conn(0x7ff4480d0890 0x7ff4480d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:07.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952133821 0 0) 0x7ff4480cdb20 con 0x7ff448159ea0 2023-07-13T22:22:07.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4340035b0 con 0x7ff448159ea0 2023-07-13T22:22:07.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1967854644 0 0) 0x7ff4340035b0 con 0x7ff448159ea0 2023-07-13T22:22:07.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 >> v2:172.21.15.105:3300/0 conn(0x7ff4480d0890 msgr2=0x7ff4480d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:07.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff43ffff700 1 --2- 172.21.15.60:0/2702622679 >> v2:172.21.15.105:3300/0 conn(0x7ff4480d0890 0x7ff4480d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 >> v1:172.21.15.105:6789/0 conn(0x7ff4480cd6d0 legacy=0x7ff4480d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:07.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44815d4e0 con 0x7ff448159ea0 2023-07-13T22:22:07.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.223+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7ff4440019f0 con 0x7ff448159ea0 2023-07-13T22:22:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 >> v1:172.21.15.60:6789/0 conn(0x7ff448159ea0 legacy=0x7ff44815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff43ffff700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff434003a00 0x7ff434003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff43ffff700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4340043f0 0x7ff4340048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff43ffff700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4340050b0 0x7ff43400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480d2e60 con 0x7ff434003a00 2023-07-13T22:22:07.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43400c000 con 0x7ff4340043f0 2023-07-13T22:22:07.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43400c1c0 con 0x7ff4340050b0 2023-07-13T22:22:07.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff44d59b700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff434003a00 0x7ff434003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:07.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff44cd9a700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4340043f0 0x7ff4340048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:07.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff44dd9c700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4340050b0 0x7ff43400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:07.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff44d59b700 1 -- 172.21.15.60:0/2702622679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4340050b0 msgr2=0x7ff43400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:07.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff44d59b700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4340050b0 0x7ff43400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff44d59b700 1 -- 172.21.15.60:0/2702622679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4340043f0 msgr2=0x7ff4340048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:07.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff44d59b700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4340043f0 0x7ff4340048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff44d59b700 1 -- 172.21.15.60:0/2702622679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4340035b0 con 0x7ff434003a00 2023-07-13T22:22:07.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.224+0000 7ff44dd9c700 1 -- 172.21.15.60:0/2702622679 reap_dead start 2023-07-13T22:22:07.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.225+0000 7ff44d59b700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff434003a00 0x7ff434003dd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff44400a7d0 tx=0x7ff434005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffd02389746e67a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:07.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.225+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444012600 con 0x7ff434003a00 2023-07-13T22:22:07.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.225+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff444012760 con 0x7ff434003a00 2023-07-13T22:22:07.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.225+0000 7ff43ffff700 1 -- 172.21.15.60:0/2702622679 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444012b10 con 0x7ff434003a00 2023-07-13T22:22:07.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.225+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2702622679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff434003a00 msgr2=0x7ff434003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:07.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.225+0000 7ff44f7ff700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff434003a00 0x7ff434003dd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff44400a7d0 tx=0x7ff434005880 comp rx=0 tx=0).stop 2023-07-13T22:22:07.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.226+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2702622679 shutdown_connections 2023-07-13T22:22:07.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.226+0000 7ff44f7ff700 1 --2- 172.21.15.60:0/2702622679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff434003a00 0x7ff434003dd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.226+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2702622679 >> 172.21.15.60:0/2702622679 conn(0x7ff448071600 msgr2=0x7ff4480719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:07.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.226+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2702622679 shutdown_connections 2023-07-13T22:22:07.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.226+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2702622679 wait complete. 2023-07-13T22:22:07.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.227+0000 7ff44f7ff700 1 Processor -- start 2023-07-13T22:22:07.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.228+0000 7ff44f7ff700 1 -- start start 2023-07-13T22:22:07.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.228+0000 7ff44f7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff434003a00 0x7ff448153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:07.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.228+0000 7ff44f7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4480c7470 0x7ff4480c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:07.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44f7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4480c85c0 0x7ff44815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:07.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44f7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480c8d40 con 0x7ff4480c7470 2023-07-13T22:22:07.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44f7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480c6fe0 con 0x7ff4480c85c0 2023-07-13T22:22:07.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44f7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480c8a60 con 0x7ff434003a00 2023-07-13T22:22:07.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44cd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4480c7470 0x7ff4480c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:07.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44cd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4480c7470 0x7ff4480c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45372/0 (socket says 172.21.15.60:45372) 2023-07-13T22:22:07.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44cd9a700 1 -- 172.21.15.60:0/2990953668 learned_addr learned my addr 172.21.15.60:0/2990953668 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:07.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44d59b700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff434003a00 0x7ff448153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:07.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44dd9c700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4480c85c0 0x7ff44815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:07.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44cd9a700 1 -- 172.21.15.60:0/2990953668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff434003a00 msgr2=0x7ff448153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:07.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44cd9a700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff434003a00 0x7ff448153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44cd9a700 1 -- 172.21.15.60:0/2990953668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4480c85c0 msgr2=0x7ff44815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:07.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44cd9a700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4480c85c0 0x7ff44815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.229+0000 7ff44cd9a700 1 -- 172.21.15.60:0/2990953668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4480c6990 con 0x7ff4480c7470 2023-07-13T22:22:07.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.230+0000 7ff44cd9a700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4480c7470 0x7ff4480c8080 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff43800f7d0 tx=0x7ff43800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:07.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.230+0000 7ff43dffb700 1 -- 172.21.15.60:0/2990953668 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438005320 con 0x7ff4480c7470 2023-07-13T22:22:07.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.230+0000 7ff43dffb700 1 -- 172.21.15.60:0/2990953668 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438005480 con 0x7ff4480c7470 2023-07-13T22:22:07.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.230+0000 7ff43dffb700 1 -- 172.21.15.60:0/2990953668 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438006aa0 con 0x7ff4480c7470 2023-07-13T22:22:07.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.230+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4480c6c70 con 0x7ff4480c7470 2023-07-13T22:22:07.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.230+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44815f430 con 0x7ff4480c7470 2023-07-13T22:22:07.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.231+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4480c72c0 con 0x7ff4480c7470 2023-07-13T22:22:07.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.232+0000 7ff43dffb700 1 -- 172.21.15.60:0/2990953668 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff438006c00 con 0x7ff4480c7470 2023-07-13T22:22:07.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.233+0000 7ff43dffb700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff42c06ace0 0x7ff42c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:07.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.233+0000 7ff43dffb700 1 -- 172.21.15.60:0/2990953668 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6206+0+0 (secure 0 0 0) 0x7ff438088850 con 0x7ff4480c7470 2023-07-13T22:22:07.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.233+0000 7ff44d59b700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff42c06ace0 0x7ff42c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:07.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.234+0000 7ff44d59b700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff42c06ace0 0x7ff42c06d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff444000f80 tx=0x7ff444010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:07.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.237+0000 7ff43dffb700 1 -- 172.21.15.60:0/2990953668 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff4380537d0 con 0x7ff4480c7470 2023-07-13T22:22:07.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.445+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff448067a70 con 0x7ff4480c7470 2023-07-13T22:22:07.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:07.996+0000 7ff43dffb700 1 -- 172.21.15.60:0/2990953668 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff438056df0 con 0x7ff4480c7470 2023-07-13T22:22:07.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff42c06ace0 msgr2=0x7ff42c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:07.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff42c06ace0 0x7ff42c06d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff444000f80 tx=0x7ff444010000 comp rx=0 tx=0).stop 2023-07-13T22:22:07.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4480c7470 msgr2=0x7ff4480c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:07.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4480c7470 0x7ff4480c8080 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff43800f7d0 tx=0x7ff43800fae0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 shutdown_connections 2023-07-13T22:22:07.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff42c06ace0 0x7ff42c06d190 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff444000f80 tx=0x7ff444010000 comp rx=0 tx=0).stop 2023-07-13T22:22:07.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4480c85c0 0x7ff44815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff434003a00 0x7ff448153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 --2- 172.21.15.60:0/2990953668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4480c7470 0x7ff4480c8080 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:07.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.001+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 >> 172.21.15.60:0/2990953668 conn(0x7ff448071600 msgr2=0x7ff44815aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:07.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.002+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 shutdown_connections 2023-07-13T22:22:07.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.002+0000 7ff44f7ff700 1 -- 172.21.15.60:0/2990953668 wait complete. 2023-07-13T22:22:07.988 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:22:08.009 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:22:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.237+0000 7f2f797ff700 1 Processor -- start 2023-07-13T22:22:08.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.238+0000 7f2f797ff700 1 -- start start 2023-07-13T22:22:08.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.239+0000 7f2f797ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f74159ea0 0x7f2f740cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:08.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.239+0000 7f2f797ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f740d0890 0x7f2f740d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:08.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.239+0000 7f2f797ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7415d200 con 0x7f2f7415c9b0 2023-07-13T22:22:08.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.239+0000 7f2f797ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f740d2e60 con 0x7f2f740d0890 2023-07-13T22:22:08.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.239+0000 7f2f797ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740d2b50 con 0x7f2f74159ea0 2023-07-13T22:22:08.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.239+0000 7f2f737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f740d0890 0x7f2f740d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:08.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.239+0000 7f2f737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f740d0890 0x7f2f740d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45382/0 (socket says 172.21.15.60:45382) 2023-07-13T22:22:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.239+0000 7f2f737fe700 1 -- 172.21.15.60:0/2241036281 learned_addr learned my addr 172.21.15.60:0/2241036281 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.239+0000 7f2f72ffd700 1 --2- 172.21.15.60:0/2241036281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f74159ea0 0x7f2f740cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.240+0000 7f2f737fe700 1 -- 172.21.15.60:0/2241036281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f74159ea0 msgr2=0x7f2f740cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.240+0000 7f2f737fe700 1 --2- 172.21.15.60:0/2241036281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f74159ea0 0x7f2f740cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:08.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.240+0000 7f2f737fe700 1 -- 172.21.15.60:0/2241036281 >> v1:172.21.15.105:6789/0 conn(0x7f2f7415c9b0 legacy=0x7f2f740d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:08.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.240+0000 7f2f737fe700 1 -- 172.21.15.60:0/2241036281 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7415d4e0 con 0x7f2f740d0890 2023-07-13T22:22:08.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.240+0000 7f2f72ffd700 1 --2- 172.21.15.60:0/2241036281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f74159ea0 0x7f2f740cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.240+0000 7f2f737fe700 1 --2- 172.21.15.60:0/2241036281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f740d0890 0x7f2f740d0cf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009e50 tx=0x7f2f6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb48a8e0d175ade server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.240+0000 7f2f71ffb700 1 -- 172.21.15.60:0/2241036281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00e070 con 0x7f2f740d0890 2023-07-13T22:22:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.240+0000 7f2f71ffb700 1 -- 172.21.15.60:0/2241036281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c00b9c0 con 0x7f2f740d0890 2023-07-13T22:22:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.240+0000 7f2f71ffb700 1 -- 172.21.15.60:0/2241036281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00bd70 con 0x7f2f740d0890 2023-07-13T22:22:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.241+0000 7f2f797ff700 1 -- 172.21.15.60:0/2241036281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f740d0890 msgr2=0x7f2f740d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.241+0000 7f2f797ff700 1 --2- 172.21.15.60:0/2241036281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f740d0890 0x7f2f740d0cf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009e50 tx=0x7f2f6c00b330 comp rx=0 tx=0).stop 2023-07-13T22:22:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.241+0000 7f2f797ff700 1 -- 172.21.15.60:0/2241036281 shutdown_connections 2023-07-13T22:22:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.241+0000 7f2f797ff700 1 --2- 172.21.15.60:0/2241036281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f740d0890 0x7f2f740d0cf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.241+0000 7f2f797ff700 1 --2- 172.21.15.60:0/2241036281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f74159ea0 0x7f2f740cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.241+0000 7f2f797ff700 1 -- 172.21.15.60:0/2241036281 >> 172.21.15.60:0/2241036281 conn(0x7f2f74071600 msgr2=0x7f2f740719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.241+0000 7f2f797ff700 1 -- 172.21.15.60:0/2241036281 shutdown_connections 2023-07-13T22:22:08.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.241+0000 7f2f797ff700 1 -- 172.21.15.60:0/2241036281 wait complete. 2023-07-13T22:22:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.243+0000 7f2f797ff700 1 Processor -- start 2023-07-13T22:22:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.244+0000 7f2f797ff700 1 -- start start 2023-07-13T22:22:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f797ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f740d0890 0x7f2f7415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f797ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f74159ea0 0x7f2f741533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f797ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7415c9b0 0x7f2f741538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f797ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740c84b0 con 0x7f2f74159ea0 2023-07-13T22:22:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f797ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740c78e0 con 0x7f2f7415c9b0 2023-07-13T22:22:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f797ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740c7b80 con 0x7f2f740d0890 2023-07-13T22:22:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f727fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f74159ea0 0x7f2f741533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f727fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f74159ea0 0x7f2f741533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45388/0 (socket says 172.21.15.60:45388) 2023-07-13T22:22:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f727fc700 1 -- 172.21.15.60:0/1287869142 learned_addr learned my addr 172.21.15.60:0/1287869142 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:08.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f72ffd700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f740d0890 0x7f2f7415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:08.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.245+0000 7f2f737fe700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7415c9b0 0x7f2f741538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:08.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f727fc700 1 -- 172.21.15.60:0/1287869142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f740d0890 msgr2=0x7f2f7415ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:08.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f727fc700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f740d0890 0x7f2f7415ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:08.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f727fc700 1 -- 172.21.15.60:0/1287869142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7415c9b0 msgr2=0x7f2f741538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:08.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f727fc700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7415c9b0 0x7f2f741538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:08.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f727fc700 1 -- 172.21.15.60:0/1287869142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c009b30 con 0x7f2f74159ea0 2023-07-13T22:22:08.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f727fc700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f74159ea0 0x7f2f741533b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c450 tx=0x7f2f6800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:08.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f5b7fe700 1 -- 172.21.15.60:0/1287869142 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680052d0 con 0x7f2f74159ea0 2023-07-13T22:22:08.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f5b7fe700 1 -- 172.21.15.60:0/1287869142 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f68005430 con 0x7f2f74159ea0 2023-07-13T22:22:08.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f5b7fe700 1 -- 172.21.15.60:0/1287869142 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680038c0 con 0x7f2f74159ea0 2023-07-13T22:22:08.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f740c7e60 con 0x7f2f74159ea0 2023-07-13T22:22:08.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.246+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f740c8940 con 0x7f2f74159ea0 2023-07-13T22:22:08.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.247+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f74067a70 con 0x7f2f74159ea0 2023-07-13T22:22:08.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.248+0000 7f2f5b7fe700 1 -- 172.21.15.60:0/1287869142 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f68003a20 con 0x7f2f74159ea0 2023-07-13T22:22:08.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.249+0000 7f2f5b7fe700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2f6006ace0 0x7f2f6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:08.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.249+0000 7f2f5b7fe700 1 -- 172.21.15.60:0/1287869142 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f2f68088700 con 0x7f2f74159ea0 2023-07-13T22:22:08.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.250+0000 7f2f72ffd700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2f6006ace0 0x7f2f6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:08.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.250+0000 7f2f72ffd700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2f6006ace0 0x7f2f6006d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2f640099d0 tx=0x7f2f64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:08.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.254+0000 7f2f5b7fe700 1 -- 172.21.15.60:0/1287869142 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2f68053830 con 0x7f2f74159ea0 2023-07-13T22:22:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:08.462+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2f7404e210 con 0x7f2f74159ea0 2023-07-13T22:22:09.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.431+0000 7f2f5b7fe700 1 -- 172.21.15.60:0/1287869142 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2f6800ce60 con 0x7f2f74159ea0 2023-07-13T22:22:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.436+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2f6006ace0 msgr2=0x7f2f6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.436+0000 7f2f797ff700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2f6006ace0 0x7f2f6006d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2f640099d0 tx=0x7f2f64008040 comp rx=0 tx=0).stop 2023-07-13T22:22:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.436+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f74159ea0 msgr2=0x7f2f741533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:09.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.436+0000 7f2f797ff700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f74159ea0 0x7f2f741533b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c450 tx=0x7f2f6800c760 comp rx=0 tx=0).stop 2023-07-13T22:22:09.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.437+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 shutdown_connections 2023-07-13T22:22:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.437+0000 7f2f797ff700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2f6006ace0 0x7f2f6006d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.437+0000 7f2f797ff700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7415c9b0 0x7f2f741538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.437+0000 7f2f797ff700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f740d0890 0x7f2f7415ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.437+0000 7f2f797ff700 1 --2- 172.21.15.60:0/1287869142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f74159ea0 0x7f2f741533b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.437+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 >> 172.21.15.60:0/1287869142 conn(0x7f2f74071600 msgr2=0x7f2f740d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.438+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 shutdown_connections 2023-07-13T22:22:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.438+0000 7f2f797ff700 1 -- 172.21.15.60:0/1287869142 wait complete. 2023-07-13T22:22:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:22:09.444 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:22:09.445 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-13T22:22:09.445 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:22:09.445 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:22:09.446 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:22:09.446 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-13T22:22:09.446 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:22:09.446 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:22:09.447 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:22:09.447 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-13T22:22:09.447 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:22:09.448 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:22:09.448 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:22:09.448 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-13T22:22:09.448 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:22:09.449 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:22:09.471 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:22:09.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:22:09.473 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:22:09.473 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:22:09.474 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:22:09.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:22:09.475 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:22:09.475 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:22:09.517 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:22:09.517 DEBUG:teuthology.parallel:result is None 2023-07-13T22:22:09.518 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:22:09.519 DEBUG:teuthology.parallel:result is None 2023-07-13T22:22:09.520 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:22:09.520 DEBUG:teuthology.parallel:result is None 2023-07-13T22:22:09.521 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:22:09.521 DEBUG:teuthology.parallel:result is None 2023-07-13T22:22:09.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-13T22:22:09.666 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:22:09.667 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:22:09.672 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:22:09.676 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:22:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.756+0000 7f6cf6dff700 1 Processor -- start 2023-07-13T22:22:09.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.756+0000 7f6cf6dff700 1 -- start start 2023-07-13T22:22:09.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.757+0000 7f6cf6dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cf00b8570 0x7f6cf00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:09.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.757+0000 7f6cf6dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6cf00d15c0 0x7f6cf015c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:09.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.757+0000 7f6cf6dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf00d09d0 con 0x7f6cf00d35c0 2023-07-13T22:22:09.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.757+0000 7f6cf6dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf00d3b30 con 0x7f6cf00b8570 2023-07-13T22:22:09.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.757+0000 7f6cf6dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf00cda10 con 0x7f6cf00d15c0 2023-07-13T22:22:09.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.757+0000 7f6cf4b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cf00b8570 0x7f6cf00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:09.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.757+0000 7f6cf4b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cf00b8570 0x7f6cf00ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45414/0 (socket says 172.21.15.60:45414) 2023-07-13T22:22:09.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.757+0000 7f6cf4b9b700 1 -- 172.21.15.60:0/2001355189 learned_addr learned my addr 172.21.15.60:0/2001355189 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:09.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.757+0000 7f6cf539c700 1 --2- 172.21.15.60:0/2001355189 >> v2:172.21.15.105:3300/0 conn(0x7f6cf00d15c0 0x7f6cf015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:09.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.758+0000 7f6cf4b9b700 1 -- 172.21.15.60:0/2001355189 >> v2:172.21.15.105:3300/0 conn(0x7f6cf00d15c0 msgr2=0x7f6cf015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:09.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.758+0000 7f6cf4b9b700 1 --2- 172.21.15.60:0/2001355189 >> v2:172.21.15.105:3300/0 conn(0x7f6cf00d15c0 0x7f6cf015c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.758+0000 7f6cf4b9b700 1 -- 172.21.15.60:0/2001355189 >> v1:172.21.15.105:6789/0 conn(0x7f6cf00d35c0 legacy=0x7f6cf00d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:09.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.758+0000 7f6cf4b9b700 1 -- 172.21.15.60:0/2001355189 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00cd450 con 0x7f6cf00b8570 2023-07-13T22:22:09.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.758+0000 7f6cf539c700 1 --2- 172.21.15.60:0/2001355189 >> v2:172.21.15.105:3300/0 conn(0x7f6cf00d15c0 0x7f6cf015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:09.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.758+0000 7f6cf4b9b700 1 --2- 172.21.15.60:0/2001355189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00b8570 0x7f6cf00ceeb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00b2670 tx=0x7f6ce4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bacfac1e453164d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:09.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.758+0000 7f6cef7fe700 1 -- 172.21.15.60:0/2001355189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4005e30 con 0x7f6cf00b8570 2023-07-13T22:22:09.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.758+0000 7f6cef7fe700 1 -- 172.21.15.60:0/2001355189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce4004030 con 0x7f6cf00b8570 2023-07-13T22:22:09.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.758+0000 7f6cef7fe700 1 -- 172.21.15.60:0/2001355189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40043e0 con 0x7f6cf00b8570 2023-07-13T22:22:09.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.759+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2001355189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00b8570 msgr2=0x7f6cf00ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:09.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.759+0000 7f6cf6dff700 1 --2- 172.21.15.60:0/2001355189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00b8570 0x7f6cf00ceeb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00b2670 tx=0x7f6ce4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.759+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2001355189 shutdown_connections 2023-07-13T22:22:09.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.759+0000 7f6cf6dff700 1 --2- 172.21.15.60:0/2001355189 >> v2:172.21.15.105:3300/0 conn(0x7f6cf00d15c0 0x7f6cf015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.759+0000 7f6cf6dff700 1 --2- 172.21.15.60:0/2001355189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00b8570 0x7f6cf00ceeb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.759+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2001355189 >> 172.21.15.60:0/2001355189 conn(0x7f6cf0071600 msgr2=0x7f6cf00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:09.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.759+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2001355189 shutdown_connections 2023-07-13T22:22:09.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.759+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2001355189 wait complete. 2023-07-13T22:22:09.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.761+0000 7f6cf6dff700 1 Processor -- start 2023-07-13T22:22:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.761+0000 7f6cf6dff700 1 -- start start 2023-07-13T22:22:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf6dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cf00b8570 0x7f6cf01f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf6dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cf00d15c0 0x7f6cf00c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf6dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00d35c0 0x7f6cf00c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf6dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00c8220 con 0x7f6cf00d35c0 2023-07-13T22:22:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf6dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00ca240 con 0x7f6cf00d15c0 2023-07-13T22:22:09.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf6dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00c7c60 con 0x7f6cf00b8570 2023-07-13T22:22:09.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf539c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00d35c0 0x7f6cf00c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:09.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf539c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00d35c0 0x7f6cf00c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45426/0 (socket says 172.21.15.60:45426) 2023-07-13T22:22:09.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf539c700 1 -- 172.21.15.60:0/2995029185 learned_addr learned my addr 172.21.15.60:0/2995029185 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:09.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6cf4b9b700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cf00b8570 0x7f6cf01f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:09.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.762+0000 7f6ceffff700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cf00d15c0 0x7f6cf00c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:09.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6cf539c700 1 -- 172.21.15.60:0/2995029185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cf00b8570 msgr2=0x7f6cf01f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6cf539c700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cf00b8570 0x7f6cf01f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6cf539c700 1 -- 172.21.15.60:0/2995029185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cf00d15c0 msgr2=0x7f6cf00c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6cf539c700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cf00d15c0 0x7f6cf00c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6cf539c700 1 -- 172.21.15.60:0/2995029185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009500 con 0x7f6cf00d35c0 2023-07-13T22:22:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6cf539c700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00d35c0 0x7f6cf00c85e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8007ef0 tx=0x7f6ce800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6ced7fa700 1 -- 172.21.15.60:0/2995029185 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800dd80 con 0x7f6cf00d35c0 2023-07-13T22:22:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6ced7fa700 1 -- 172.21.15.60:0/2995029185 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce800f040 con 0x7f6cf00d35c0 2023-07-13T22:22:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6ced7fa700 1 -- 172.21.15.60:0/2995029185 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8003c60 con 0x7f6cf00d35c0 2023-07-13T22:22:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf00c7f40 con 0x7f6cf00d35c0 2023-07-13T22:22:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.763+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf0150780 con 0x7f6cf00d35c0 2023-07-13T22:22:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.764+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf00c9d30 con 0x7f6cf00d35c0 2023-07-13T22:22:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.765+0000 7f6ced7fa700 1 -- 172.21.15.60:0/2995029185 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ce8004030 con 0x7f6cf00d35c0 2023-07-13T22:22:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.766+0000 7f6ced7fa700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cd406ace0 0x7f6cd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:09.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.767+0000 7f6ced7fa700 1 -- 172.21.15.60:0/2995029185 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6ce80892b0 con 0x7f6cf00d35c0 2023-07-13T22:22:09.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.767+0000 7f6cf4b9b700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cd406ace0 0x7f6cd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:09.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.767+0000 7f6cf4b9b700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cd406ace0 0x7f6cd406d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00b2670 tx=0x7f6ce4008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:09.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.771+0000 7f6ced7fa700 1 -- 172.21.15.60:0/2995029185 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6cf00c9d30 con 0x7f6cf00d35c0 2023-07-13T22:22:09.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:09.994+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6cf00c9d30 con 0x7f6cf00d35c0 2023-07-13T22:22:10.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.013+0000 7f6ced7fa700 1 -- 172.21.15.60:0/2995029185 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6cf00c9d30 con 0x7f6cf00d35c0 2023-07-13T22:22:11.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cd406ace0 msgr2=0x7f6cd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cd406ace0 0x7f6cd406d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00b2670 tx=0x7f6ce4008070 comp rx=0 tx=0).stop 2023-07-13T22:22:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00d35c0 msgr2=0x7f6cf00c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00d35c0 0x7f6cf00c85e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8007ef0 tx=0x7f6ce800d3b0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 shutdown_connections 2023-07-13T22:22:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6cd406ace0 0x7f6cd406d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00d35c0 0x7f6cf00c85e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6cf00b8570 0x7f6cf01f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 --2- 172.21.15.60:0/2995029185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cf00d15c0 0x7f6cf00c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 >> 172.21.15.60:0/2995029185 conn(0x7f6cf0071600 msgr2=0x7f6cf015e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 shutdown_connections 2023-07-13T22:22:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.016+0000 7f6cf6dff700 1 -- 172.21.15.60:0/2995029185 wait complete. 2023-07-13T22:22:11.002 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-07-13T22:22:11.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-13T22:22:11.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.247+0000 7f0bc9bff700 1 Processor -- start 2023-07-13T22:22:11.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.247+0000 7f0bc9bff700 1 -- start start 2023-07-13T22:22:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.248+0000 7f0bc9bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bc40d3750 0x7f0bc40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.248+0000 7f0bc9bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40cfad0 0x7f0bc40d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.248+0000 7f0bc9bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc40cd950 con 0x7f0bc40b8400 2023-07-13T22:22:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.248+0000 7f0bc9bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc40cff50 con 0x7f0bc40d3750 2023-07-13T22:22:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.248+0000 7f0bc9bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40d0ac0 con 0x7f0bc40cfad0 2023-07-13T22:22:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.248+0000 7f0bc2ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bc40d3750 0x7f0bc40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.248+0000 7f0bc2ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bc40d3750 0x7f0bc40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45442/0 (socket says 172.21.15.60:45442) 2023-07-13T22:22:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.248+0000 7f0bc2ffd700 1 -- 172.21.15.60:0/492663812 learned_addr learned my addr 172.21.15.60:0/492663812 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.249+0000 7f0bc3fff700 1 --2- 172.21.15.60:0/492663812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40cfad0 0x7f0bc40d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.249+0000 7f0bc2ffd700 1 -- 172.21.15.60:0/492663812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40cfad0 msgr2=0x7f0bc40d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.249+0000 7f0bc2ffd700 1 --2- 172.21.15.60:0/492663812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40cfad0 0x7f0bc40d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.249+0000 7f0bc2ffd700 1 -- 172.21.15.60:0/492663812 >> v1:172.21.15.105:6789/0 conn(0x7f0bc40b8400 legacy=0x7f0bc40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.249+0000 7f0bc2ffd700 1 -- 172.21.15.60:0/492663812 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc40b87d0 con 0x7f0bc40d3750 2023-07-13T22:22:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.249+0000 7f0bc2ffd700 1 --2- 172.21.15.60:0/492663812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40d3750 0x7f0bc40cf590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80098f0 tx=0x7f0bb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6c7630e1d799c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.249+0000 7f0bc27fc700 1 -- 172.21.15.60:0/492663812 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800b8a0 con 0x7f0bc40d3750 2023-07-13T22:22:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.249+0000 7f0bc27fc700 1 -- 172.21.15.60:0/492663812 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb800ba00 con 0x7f0bc40d3750 2023-07-13T22:22:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.249+0000 7f0bc27fc700 1 -- 172.21.15.60:0/492663812 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800bd90 con 0x7f0bc40d3750 2023-07-13T22:22:11.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.250+0000 7f0bc9bff700 1 -- 172.21.15.60:0/492663812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40d3750 msgr2=0x7f0bc40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.250+0000 7f0bc9bff700 1 --2- 172.21.15.60:0/492663812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40d3750 0x7f0bc40cf590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80098f0 tx=0x7f0bb8009c00 comp rx=0 tx=0).stop 2023-07-13T22:22:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.250+0000 7f0bc9bff700 1 -- 172.21.15.60:0/492663812 shutdown_connections 2023-07-13T22:22:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.250+0000 7f0bc9bff700 1 --2- 172.21.15.60:0/492663812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40cfad0 0x7f0bc40d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.250+0000 7f0bc9bff700 1 --2- 172.21.15.60:0/492663812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40d3750 0x7f0bc40cf590 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.250+0000 7f0bc9bff700 1 -- 172.21.15.60:0/492663812 >> 172.21.15.60:0/492663812 conn(0x7f0bc406b390 msgr2=0x7f0bc406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.250+0000 7f0bc9bff700 1 -- 172.21.15.60:0/492663812 shutdown_connections 2023-07-13T22:22:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.250+0000 7f0bc9bff700 1 -- 172.21.15.60:0/492663812 wait complete. 2023-07-13T22:22:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.252+0000 7f0bc9bff700 1 Processor -- start 2023-07-13T22:22:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.252+0000 7f0bc9bff700 1 -- start start 2023-07-13T22:22:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc9bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40b8400 0x7f0bc415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc9bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40cfad0 0x7f0bc4150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc9bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bc40d3750 0x7f0bc4159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc9bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc415deb0 con 0x7f0bc40cfad0 2023-07-13T22:22:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc9bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40c82f0 con 0x7f0bc40d3750 2023-07-13T22:22:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc9bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc415d860 con 0x7f0bc40b8400 2023-07-13T22:22:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40cfad0 0x7f0bc4150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40cfad0 0x7f0bc4150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45456/0 (socket says 172.21.15.60:45456) 2023-07-13T22:22:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc2ffd700 1 -- 172.21.15.60:0/3483361648 learned_addr learned my addr 172.21.15.60:0/3483361648 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc37fe700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40b8400 0x7f0bc415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.253+0000 7f0bc3fff700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bc40d3750 0x7f0bc4159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0bc2ffd700 1 -- 172.21.15.60:0/3483361648 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40b8400 msgr2=0x7f0bc415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0bc2ffd700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40b8400 0x7f0bc415cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0bc2ffd700 1 -- 172.21.15.60:0/3483361648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bc40d3750 msgr2=0x7f0bc4159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0bc2ffd700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bc40d3750 0x7f0bc4159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0bc2ffd700 1 -- 172.21.15.60:0/3483361648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8009550 con 0x7f0bc40cfad0 2023-07-13T22:22:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0bc2ffd700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40cfad0 0x7f0bc4150f90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009c40 tx=0x7f0bb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0babfff700 1 -- 172.21.15.60:0/3483361648 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80036a0 con 0x7f0bc40cfad0 2023-07-13T22:22:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0babfff700 1 -- 172.21.15.60:0/3483361648 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb801e070 con 0x7f0bc40cfad0 2023-07-13T22:22:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc415db40 con 0x7f0bc40cfad0 2023-07-13T22:22:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0babfff700 1 -- 172.21.15.60:0/3483361648 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8003a60 con 0x7f0bc40cfad0 2023-07-13T22:22:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.254+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc415d480 con 0x7f0bc40cfad0 2023-07-13T22:22:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.255+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc415e020 con 0x7f0bc40cfad0 2023-07-13T22:22:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.256+0000 7f0babfff700 1 -- 172.21.15.60:0/3483361648 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0bb8003bc0 con 0x7f0bc40cfad0 2023-07-13T22:22:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.258+0000 7f0babfff700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0bb006ace0 0x7f0bb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.258+0000 7f0babfff700 1 -- 172.21.15.60:0/3483361648 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0bb8020070 con 0x7f0bc40cfad0 2023-07-13T22:22:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.258+0000 7f0bc37fe700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0bb006ace0 0x7f0bb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.259+0000 7f0bc37fe700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0bb006ace0 0x7f0bb006d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40c97b0 tx=0x7f0bb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:11.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.262+0000 7f0babfff700 1 -- 172.21.15.60:0/3483361648 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0bb8055b20 con 0x7f0bc40cfad0 2023-07-13T22:22:11.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.552+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0bc415d670 con 0x7f0bc40cfad0 2023-07-13T22:22:11.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.553+0000 7f0babfff700 1 -- 172.21.15.60:0/3483361648 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f0bb8059140 con 0x7f0bc40cfad0 2023-07-13T22:22:11.538 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:11.538 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:22:11.538 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-07-13T22:22:11.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.538 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-07-13T22:22:11.539 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw==", 2023-07-13T22:22:11.539 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.539 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:22:11.539 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:22:11.539 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:22:11.539 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.539 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.540 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-07-13T22:22:11.540 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g==", 2023-07-13T22:22:11.540 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.540 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:22:11.540 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:22:11.540 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:22:11.541 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.541 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.541 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-07-13T22:22:11.541 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA==", 2023-07-13T22:22:11.541 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.541 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:22:11.542 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:22:11.542 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:22:11.542 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.542 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.542 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-07-13T22:22:11.543 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ==", 2023-07-13T22:22:11.543 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.543 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:22:11.543 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:22:11.543 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:22:11.543 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.543 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.544 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-07-13T22:22:11.544 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ==", 2023-07-13T22:22:11.544 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.544 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:22:11.545 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:22:11.545 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.545 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.545 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-07-13T22:22:11.545 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw==", 2023-07-13T22:22:11.546 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.546 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:22:11.546 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:22:11.546 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.546 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.547 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-07-13T22:22:11.547 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q==", 2023-07-13T22:22:11.547 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.548 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:22:11.548 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:22:11.548 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.548 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.549 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-07-13T22:22:11.549 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ==", 2023-07-13T22:22:11.549 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.550 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:22:11.550 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:22:11.550 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.550 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.551 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-07-13T22:22:11.551 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg==", 2023-07-13T22:22:11.551 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.552 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:22:11.552 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:22:11.552 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.552 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.553 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-07-13T22:22:11.553 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA==", 2023-07-13T22:22:11.554 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.554 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:22:11.554 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:22:11.554 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.555 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.556 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-07-13T22:22:11.556 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw==", 2023-07-13T22:22:11.556 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.556 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:22:11.557 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:22:11.557 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.557 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.558 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-07-13T22:22:11.558 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ==", 2023-07-13T22:22:11.558 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.558 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:22:11.558 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:22:11.558 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.559 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.559 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-07-13T22:22:11.559 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w==", 2023-07-13T22:22:11.559 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.559 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:22:11.560 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:22:11.560 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:22:11.560 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:22:11.560 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.560 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-07-13T22:22:11.561 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw==", 2023-07-13T22:22:11.561 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.561 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:22:11.561 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:22:11.561 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:22:11.561 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:22:11.561 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-07-13T22:22:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg==", 2023-07-13T22:22:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:22:11.563 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:22:11.563 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:22:11.563 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:22:11.563 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.563 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-07-13T22:22:11.564 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q==", 2023-07-13T22:22:11.564 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.564 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:22:11.564 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:22:11.564 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:22:11.565 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:22:11.565 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.565 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.565 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-07-13T22:22:11.565 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBk7CgGBhAAaK88V4CEyrYfZx7wv+XG3g==", 2023-07-13T22:22:11.566 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.566 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:22:11.566 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:22:11.566 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:22:11.566 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.566 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.567 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mds", 2023-07-13T22:22:11.567 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCxeLBkhqtSJBAAacsry/ANqNOQmdefaxnE0g==", 2023-07-13T22:22:11.567 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.567 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mds" 2023-07-13T22:22:11.567 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.568 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mgr", 2023-07-13T22:22:11.568 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCxeLBkOrhSJBAA5uPM89PjzYGEqDdFu+3ljw==", 2023-07-13T22:22:11.569 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.569 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mgr" 2023-07-13T22:22:11.569 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-osd", 2023-07-13T22:22:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCxeLBkGsFSJBAAYrPkUs4s3Sw+ymva3LFrCQ==", 2023-07-13T22:22:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-osd" 2023-07-13T22:22:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd", 2023-07-13T22:22:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCxeLBkkMlSJBAAlwebkP/uFUIxAxkyaPOXHw==", 2023-07-13T22:22:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd" 2023-07-13T22:22:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-07-13T22:22:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCxeLBkUNJSJBAAGX1iJpBX7IzYkbnf72tIWA==", 2023-07-13T22:22:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-07-13T22:22:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rgw", 2023-07-13T22:22:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCxeLBkD9tSJBAApt73a4NQs1AIcXZMKBOOKA==", 2023-07-13T22:22:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rgw" 2023-07-13T22:22:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.574 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.574 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-07-13T22:22:11.574 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ==", 2023-07-13T22:22:11.574 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.574 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:22:11.574 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:22:11.574 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.574 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:11.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:11.575 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-07-13T22:22:11.575 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ==", 2023-07-13T22:22:11.575 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:22:11.575 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:22:11.576 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:22:11.576 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:22:11.576 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.576 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:11.576 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:22:11.576 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:22:11.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0bb006ace0 msgr2=0x7f0bb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0bb006ace0 0x7f0bb006d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40c97b0 tx=0x7f0bb4009040 comp rx=0 tx=0).stop 2023-07-13T22:22:11.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40cfad0 msgr2=0x7f0bc4150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40cfad0 0x7f0bc4150f90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009c40 tx=0x7f0bb8004750 comp rx=0 tx=0).stop 2023-07-13T22:22:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 shutdown_connections 2023-07-13T22:22:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0bb006ace0 0x7f0bb006d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bc40d3750 0x7f0bc4159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bc40b8400 0x7f0bc415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 --2- 172.21.15.60:0/3483361648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bc40cfad0 0x7f0bc4150f90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 >> 172.21.15.60:0/3483361648 conn(0x7f0bc406b390 msgr2=0x7f0bc40b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:11.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 shutdown_connections 2023-07-13T22:22:11.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.555+0000 7f0bc9bff700 1 -- 172.21.15.60:0/3483361648 wait complete. 2023-07-13T22:22:11.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-07-13T22:22:11.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.799+0000 7f5fbdbff700 1 Processor -- start 2023-07-13T22:22:11.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.800+0000 7f5fbdbff700 1 -- start start 2023-07-13T22:22:11.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fbdbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5fb80b2010 0x7f5fb80cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fbdbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fb80d26d0 0x7f5fb80cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fbdbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b5a00 0x7f5fb80cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fbdbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb80d2570 con 0x7f5fb80d26d0 2023-07-13T22:22:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fbdbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb80d14a0 con 0x7f5fb80b2010 2023-07-13T22:22:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fbdbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb80d1780 con 0x7f5fb80b5a00 2023-07-13T22:22:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fb6ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fb80d26d0 0x7f5fb80cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fb6ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fb80d26d0 0x7f5fb80cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45460/0 (socket says 172.21.15.60:45460) 2023-07-13T22:22:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fb6ffd700 1 -- 172.21.15.60:0/2763993013 learned_addr learned my addr 172.21.15.60:0/2763993013 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fb7fff700 1 --2- 172.21.15.60:0/2763993013 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b5a00 0x7f5fb80cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.801+0000 7f5fb77fe700 1 --2- 172.21.15.60:0/2763993013 >> v2:172.21.15.105:3300/0 conn(0x7f5fb80b2010 0x7f5fb80cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb6ffd700 1 -- 172.21.15.60:0/2763993013 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b5a00 msgr2=0x7f5fb80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb6ffd700 1 --2- 172.21.15.60:0/2763993013 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b5a00 0x7f5fb80cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb6ffd700 1 -- 172.21.15.60:0/2763993013 >> v2:172.21.15.105:3300/0 conn(0x7f5fb80b2010 msgr2=0x7f5fb80cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb6ffd700 1 --2- 172.21.15.60:0/2763993013 >> v2:172.21.15.105:3300/0 conn(0x7f5fb80b2010 0x7f5fb80cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb6ffd700 1 -- 172.21.15.60:0/2763993013 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb80cd2a0 con 0x7f5fb80d26d0 2023-07-13T22:22:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb77fe700 1 --2- 172.21.15.60:0/2763993013 >> v2:172.21.15.105:3300/0 conn(0x7f5fb80b2010 0x7f5fb80cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb6ffd700 1 --2- 172.21.15.60:0/2763993013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80d26d0 0x7f5fb80cf6e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5fb808bd20 tx=0x7f5fac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fa66e4090b1f4c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb67fc700 1 -- 172.21.15.60:0/2763993013 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00b870 con 0x7f5fb80d26d0 2023-07-13T22:22:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb67fc700 1 -- 172.21.15.60:0/2763993013 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac00b9d0 con 0x7f5fb80d26d0 2023-07-13T22:22:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.802+0000 7f5fb67fc700 1 -- 172.21.15.60:0/2763993013 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00bd60 con 0x7f5fb80d26d0 2023-07-13T22:22:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.803+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2763993013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80d26d0 msgr2=0x7f5fb80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.803+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2763993013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80d26d0 0x7f5fb80cf6e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5fb808bd20 tx=0x7f5fac009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.803+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2763993013 shutdown_connections 2023-07-13T22:22:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.803+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2763993013 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b5a00 0x7f5fb80cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.803+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2763993013 >> v2:172.21.15.105:3300/0 conn(0x7f5fb80b2010 0x7f5fb80cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.803+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2763993013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80d26d0 0x7f5fb80cf6e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.803+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2763993013 >> 172.21.15.60:0/2763993013 conn(0x7f5fb806b380 msgr2=0x7f5fb806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:11.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.803+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2763993013 shutdown_connections 2023-07-13T22:22:11.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.803+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2763993013 wait complete. 2023-07-13T22:22:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.805+0000 7f5fbdbff700 1 Processor -- start 2023-07-13T22:22:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.805+0000 7f5fbdbff700 1 -- start start 2023-07-13T22:22:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fbdbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b2010 0x7f5fb814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fbdbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80b5a00 0x7f5fb814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fbdbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb80d26d0 0x7f5fb80c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fbdbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb814cac0 con 0x7f5fb80b5a00 2023-07-13T22:22:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fbdbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb814c020 con 0x7f5fb80d26d0 2023-07-13T22:22:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fbdbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb814fe80 con 0x7f5fb80b2010 2023-07-13T22:22:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80b5a00 0x7f5fb814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80b5a00 0x7f5fb814bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45474/0 (socket says 172.21.15.60:45474) 2023-07-13T22:22:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb6ffd700 1 -- 172.21.15.60:0/2500199714 learned_addr learned my addr 172.21.15.60:0/2500199714 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb77fe700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b2010 0x7f5fb814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb7fff700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb80d26d0 0x7f5fb80c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb6ffd700 1 -- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b2010 msgr2=0x7f5fb814f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb6ffd700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b2010 0x7f5fb814f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb6ffd700 1 -- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb80d26d0 msgr2=0x7f5fb80c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb6ffd700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb80d26d0 0x7f5fb80c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb6ffd700 1 -- 172.21.15.60:0/2500199714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac009550 con 0x7f5fb80b5a00 2023-07-13T22:22:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb77fe700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b2010 0x7f5fb814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.806+0000 7f5fb7fff700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb80d26d0 0x7f5fb80c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.807+0000 7f5fb6ffd700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80b5a00 0x7f5fb814bae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000c00 tx=0x7f5fac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.807+0000 7f5f9ffff700 1 -- 172.21.15.60:0/2500199714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac004520 con 0x7f5fb80b5a00 2023-07-13T22:22:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.807+0000 7f5f9ffff700 1 -- 172.21.15.60:0/2500199714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac01e070 con 0x7f5fb80b5a00 2023-07-13T22:22:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.807+0000 7f5f9ffff700 1 -- 172.21.15.60:0/2500199714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0038c0 con 0x7f5fb80b5a00 2023-07-13T22:22:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.807+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb814c220 con 0x7f5fb80b5a00 2023-07-13T22:22:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.807+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb814c7e0 con 0x7f5fb80b5a00 2023-07-13T22:22:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.808+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb804e280 con 0x7f5fb80b5a00 2023-07-13T22:22:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.809+0000 7f5f9ffff700 1 -- 172.21.15.60:0/2500199714 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5fac003a20 con 0x7f5fb80b5a00 2023-07-13T22:22:11.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.810+0000 7f5f9ffff700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fa406ace0 0x7f5fa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.810+0000 7f5f9ffff700 1 -- 172.21.15.60:0/2500199714 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5fac020070 con 0x7f5fb80b5a00 2023-07-13T22:22:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.810+0000 7f5fb77fe700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fa406ace0 0x7f5fa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.811+0000 7f5fb77fe700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fa406ace0 0x7f5fa406d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8008940 tx=0x7f5fa8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:11.814+0000 7f5f9ffff700 1 -- 172.21.15.60:0/2500199714 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5fac055ac0 con 0x7f5fb80b5a00 2023-07-13T22:22:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.100+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f5fb80d2aa0 con 0x7f5fb80b5a00 2023-07-13T22:22:12.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.104+0000 7f5f9ffff700 1 -- 172.21.15.60:0/2500199714 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5fac08a030 con 0x7f5fb80b5a00 2023-07-13T22:22:12.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.106+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fa406ace0 msgr2=0x7f5fa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.106+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fa406ace0 0x7f5fa406d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8008940 tx=0x7f5fa8008040 comp rx=0 tx=0).stop 2023-07-13T22:22:12.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.106+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80b5a00 msgr2=0x7f5fb814bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.106+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80b5a00 0x7f5fb814bae0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000c00 tx=0x7f5fac004750 comp rx=0 tx=0).stop 2023-07-13T22:22:12.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.106+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 shutdown_connections 2023-07-13T22:22:12.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.106+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fa406ace0 0x7f5fa406d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.106+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb80d26d0 0x7f5fb80c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.106+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fb80b2010 0x7f5fb814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.106+0000 7f5fbdbff700 1 --2- 172.21.15.60:0/2500199714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fb80b5a00 0x7f5fb814bae0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.107+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 >> 172.21.15.60:0/2500199714 conn(0x7f5fb806b380 msgr2=0x7f5fb80732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:12.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.107+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 shutdown_connections 2023-07-13T22:22:12.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.107+0000 7f5fbdbff700 1 -- 172.21.15.60:0/2500199714 wait complete. 2023-07-13T22:22:12.109 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-07-13T22:22:12.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.329+0000 7f81f69ff700 1 Processor -- start 2023-07-13T22:22:12.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.329+0000 7f81f69ff700 1 -- start start 2023-07-13T22:22:12.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.330+0000 7f81f69ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00d3750 0x7f81f00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.330+0000 7f81f69ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81f00cfad0 0x7f81f00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.330+0000 7f81f69ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f00cd950 con 0x7f81f00b8400 2023-07-13T22:22:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.330+0000 7f81f69ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f00cff20 con 0x7f81f00cfad0 2023-07-13T22:22:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.330+0000 7f81f69ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00d0ac0 con 0x7f81f00d3750 2023-07-13T22:22:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.330+0000 7f81f4f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81f00cfad0 0x7f81f00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.330+0000 7f81effff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f81f00b8400 0x7f81f00cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54268/0 (socket says 172.21.15.60:54268) 2023-07-13T22:22:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.330+0000 7f81effff700 1 -- 172.21.15.60:0/1512363434 learned_addr learned my addr 172.21.15.60:0/1512363434 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:12.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81ef7fe700 1 --2- 172.21.15.60:0/1512363434 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00d3750 0x7f81f00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81eeffd700 1 -- 172.21.15.60:0/1512363434 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698452790 0 0) 0x7f81e0001110 con 0x7f81f00b8400 2023-07-13T22:22:12.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81f4f9c700 1 -- 172.21.15.60:0/1512363434 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00d3750 msgr2=0x7f81f00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:12.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81f4f9c700 1 --2- 172.21.15.60:0/1512363434 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00d3750 0x7f81f00cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81f4f9c700 1 -- 172.21.15.60:0/1512363434 >> v1:172.21.15.60:6789/0 conn(0x7f81f00b8400 legacy=0x7f81f00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81f4f9c700 1 -- 172.21.15.60:0/1512363434 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f00b87d0 con 0x7f81f00cfad0 2023-07-13T22:22:12.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81ef7fe700 1 --2- 172.21.15.60:0/1512363434 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00d3750 0x7f81f00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:12.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81f4f9c700 1 --2- 172.21.15.60:0/1512363434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 0x7f81f00d0f50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009d90 tx=0x7f81e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4543c2f8325c9155 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:12.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81eeffd700 1 -- 172.21.15.60:0/1512363434 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e800e070 con 0x7f81f00cfad0 2023-07-13T22:22:12.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81eeffd700 1 -- 172.21.15.60:0/1512363434 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e800bbd0 con 0x7f81f00cfad0 2023-07-13T22:22:12.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.331+0000 7f81eeffd700 1 -- 172.21.15.60:0/1512363434 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8012430 con 0x7f81f00cfad0 2023-07-13T22:22:12.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.332+0000 7f81f69ff700 1 -- 172.21.15.60:0/1512363434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 msgr2=0x7f81f00d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.332+0000 7f81f69ff700 1 --2- 172.21.15.60:0/1512363434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 0x7f81f00d0f50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009d90 tx=0x7f81e800b330 comp rx=0 tx=0).stop 2023-07-13T22:22:12.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.332+0000 7f81f69ff700 1 -- 172.21.15.60:0/1512363434 shutdown_connections 2023-07-13T22:22:12.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.332+0000 7f81f69ff700 1 --2- 172.21.15.60:0/1512363434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 0x7f81f00d0f50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.332+0000 7f81f69ff700 1 --2- 172.21.15.60:0/1512363434 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00d3750 0x7f81f00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.332+0000 7f81f69ff700 1 -- 172.21.15.60:0/1512363434 >> 172.21.15.60:0/1512363434 conn(0x7f81f006b390 msgr2=0x7f81f006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:12.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.332+0000 7f81f69ff700 1 -- 172.21.15.60:0/1512363434 shutdown_connections 2023-07-13T22:22:12.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.332+0000 7f81f69ff700 1 -- 172.21.15.60:0/1512363434 wait complete. 2023-07-13T22:22:12.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.334+0000 7f81f69ff700 1 Processor -- start 2023-07-13T22:22:12.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.335+0000 7f81f69ff700 1 -- start start 2023-07-13T22:22:12.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81f69ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00b8400 0x7f81f01544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81f69ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 0x7f81f00cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81f69ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81f00d3750 0x7f81f00ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81f69ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00b48c0 con 0x7f81f00cfad0 2023-07-13T22:22:12.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81f69ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f01e5d30 con 0x7f81f00d3750 2023-07-13T22:22:12.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81f69ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0154a10 con 0x7f81f00b8400 2023-07-13T22:22:12.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81ef7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 0x7f81f00cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81ef7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 0x7f81f00cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45496/0 (socket says 172.21.15.60:45496) 2023-07-13T22:22:12.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81ef7fe700 1 -- 172.21.15.60:0/1147442640 learned_addr learned my addr 172.21.15.60:0/1147442640 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:12.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81effff700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00b8400 0x7f81f01544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81f4f9c700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81f00d3750 0x7f81f00ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81ef7fe700 1 -- 172.21.15.60:0/1147442640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00b8400 msgr2=0x7f81f01544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81ef7fe700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00b8400 0x7f81f01544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81ef7fe700 1 -- 172.21.15.60:0/1147442640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81f00d3750 msgr2=0x7f81f00ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81ef7fe700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81f00d3750 0x7f81f00ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.336+0000 7f81ef7fe700 1 -- 172.21.15.60:0/1147442640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8009a70 con 0x7f81f00cfad0 2023-07-13T22:22:12.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.337+0000 7f81ef7fe700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 0x7f81f00cc2a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f81e400c820 tx=0x7f81e400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:12.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.337+0000 7f81ecff9700 1 -- 172.21.15.60:0/1147442640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011800 con 0x7f81f00cfad0 2023-07-13T22:22:12.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.337+0000 7f81ecff9700 1 -- 172.21.15.60:0/1147442640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4011960 con 0x7f81f00cfad0 2023-07-13T22:22:12.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.337+0000 7f81ecff9700 1 -- 172.21.15.60:0/1147442640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011d10 con 0x7f81f00cfad0 2023-07-13T22:22:12.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.337+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f01e5f80 con 0x7f81f00cfad0 2023-07-13T22:22:12.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.337+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f01e6540 con 0x7f81f00cfad0 2023-07-13T22:22:12.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.338+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f015a8f0 con 0x7f81f00cfad0 2023-07-13T22:22:12.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.339+0000 7f81ecff9700 1 -- 172.21.15.60:0/1147442640 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f81e4018360 con 0x7f81f00cfad0 2023-07-13T22:22:12.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.340+0000 7f81ecff9700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81dc06ace0 0x7f81dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.340+0000 7f81ecff9700 1 -- 172.21.15.60:0/1147442640 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f81e4089150 con 0x7f81f00cfad0 2023-07-13T22:22:12.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.340+0000 7f81effff700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81dc06ace0 0x7f81dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.341+0000 7f81effff700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81dc06ace0 0x7f81dc06d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f81e0000f60 tx=0x7f81e0007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:12.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.344+0000 7f81ecff9700 1 -- 172.21.15.60:0/1147442640 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f81e4057f40 con 0x7f81f00cfad0 2023-07-13T22:22:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.637+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f81f004e280 con 0x7f81f00cfad0 2023-07-13T22:22:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.641+0000 7f81ecff9700 1 -- 172.21.15.60:0/1147442640 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f81e4057ad0 con 0x7f81f00cfad0 2023-07-13T22:22:12.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81dc06ace0 msgr2=0x7f81dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81dc06ace0 0x7f81dc06d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f81e0000f60 tx=0x7f81e0007c90 comp rx=0 tx=0).stop 2023-07-13T22:22:12.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 msgr2=0x7f81f00cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 0x7f81f00cc2a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f81e400c820 tx=0x7f81e400cbe0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 shutdown_connections 2023-07-13T22:22:12.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81dc06ace0 0x7f81dc06d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81f00d3750 0x7f81f00ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81f00b8400 0x7f81f01544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 --2- 172.21.15.60:0/1147442640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f00cfad0 0x7f81f00cc2a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 >> 172.21.15.60:0/1147442640 conn(0x7f81f006b390 msgr2=0x7f81f015bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:12.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 shutdown_connections 2023-07-13T22:22:12.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.644+0000 7f81f69ff700 1 -- 172.21.15.60:0/1147442640 wait complete. 2023-07-13T22:22:12.647 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-07-13T22:22:12.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.867+0000 7ff51f9ff700 1 Processor -- start 2023-07-13T22:22:12.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.867+0000 7ff51f9ff700 1 -- start start 2023-07-13T22:22:12.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51f9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180b8570 0x7ff5180ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51f9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5180d09d0 con 0x7ff5180d35c0 2023-07-13T22:22:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51f9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5180cda10 con 0x7ff5180d15c0 2023-07-13T22:22:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51f9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180d3b30 con 0x7ff5180b8570 2023-07-13T22:22:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51cf9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff5180d35c0 0x7ff5180d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54278/0 (socket says 172.21.15.60:54278) 2023-07-13T22:22:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51cf9a700 1 -- 172.21.15.60:0/379587458 learned_addr learned my addr 172.21.15.60:0/379587458 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51d79b700 1 --2- 172.21.15.60:0/379587458 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180b8570 0x7ff5180ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff50ffff700 1 -- 172.21.15.60:0/379587458 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445521481 0 0) 0x7ff5180d09d0 con 0x7ff5180d35c0 2023-07-13T22:22:12.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff50ffff700 1 -- 172.21.15.60:0/379587458 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5040035b0 con 0x7ff5180d35c0 2023-07-13T22:22:12.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51d79b700 1 -- 172.21.15.60:0/379587458 >> v1:172.21.15.105:6789/0 conn(0x7ff5180d15c0 legacy=0x7ff51815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff50ffff700 1 -- 172.21.15.60:0/379587458 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922526093 0 0) 0x7ff5180cda10 con 0x7ff5180d15c0 2023-07-13T22:22:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51d79b700 1 -- 172.21.15.60:0/379587458 >> v1:172.21.15.60:6789/0 conn(0x7ff5180d35c0 legacy=0x7ff5180d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51d79b700 1 -- 172.21.15.60:0/379587458 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5180cd450 con 0x7ff5180b8570 2023-07-13T22:22:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff51d79b700 1 --2- 172.21.15.60:0/379587458 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180b8570 0x7ff5180ceeb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff514009850 tx=0x7ff514009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9929e92ab8ae1a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff50ffff700 1 -- 172.21.15.60:0/379587458 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514005e30 con 0x7ff5180b8570 2023-07-13T22:22:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff50ffff700 1 -- 172.21.15.60:0/379587458 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff514006030 con 0x7ff5180b8570 2023-07-13T22:22:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.869+0000 7ff50ffff700 1 -- 172.21.15.60:0/379587458 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5140063e0 con 0x7ff5180b8570 2023-07-13T22:22:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.870+0000 7ff51f9ff700 1 -- 172.21.15.60:0/379587458 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180b8570 msgr2=0x7ff5180ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.870+0000 7ff51f9ff700 1 --2- 172.21.15.60:0/379587458 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180b8570 0x7ff5180ceeb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff514009850 tx=0x7ff514009b60 comp rx=0 tx=0).stop 2023-07-13T22:22:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.870+0000 7ff51f9ff700 1 -- 172.21.15.60:0/379587458 shutdown_connections 2023-07-13T22:22:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.870+0000 7ff51f9ff700 1 --2- 172.21.15.60:0/379587458 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180b8570 0x7ff5180ceeb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.870+0000 7ff51f9ff700 1 -- 172.21.15.60:0/379587458 >> 172.21.15.60:0/379587458 conn(0x7ff518071600 msgr2=0x7ff5180719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.870+0000 7ff51f9ff700 1 -- 172.21.15.60:0/379587458 shutdown_connections 2023-07-13T22:22:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.870+0000 7ff51f9ff700 1 -- 172.21.15.60:0/379587458 wait complete. 2023-07-13T22:22:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.872+0000 7ff51f9ff700 1 Processor -- start 2023-07-13T22:22:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.872+0000 7ff51f9ff700 1 -- start start 2023-07-13T22:22:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.873+0000 7ff51f9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5180b8570 0x7ff5181f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.873+0000 7ff51f9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5180d15c0 0x7ff5181f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.873+0000 7ff51f9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180d35c0 0x7ff5181f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.873+0000 7ff51f9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180c9590 con 0x7ff5180b8570 2023-07-13T22:22:12.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.873+0000 7ff51f9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180cbf90 con 0x7ff5180d15c0 2023-07-13T22:22:12.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.873+0000 7ff51f9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180cc270 con 0x7ff5180d35c0 2023-07-13T22:22:12.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.873+0000 7ff51d79b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5180b8570 0x7ff5181f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.873+0000 7ff51d79b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5180b8570 0x7ff5181f1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45510/0 (socket says 172.21.15.60:45510) 2023-07-13T22:22:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.873+0000 7ff51d79b700 1 -- 172.21.15.60:0/4133496139 learned_addr learned my addr 172.21.15.60:0/4133496139 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51cf9a700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5180d15c0 0x7ff5181f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51df9c700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180d35c0 0x7ff5181f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51d79b700 1 -- 172.21.15.60:0/4133496139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180d35c0 msgr2=0x7ff5181f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51d79b700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180d35c0 0x7ff5181f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51d79b700 1 -- 172.21.15.60:0/4133496139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5180d15c0 msgr2=0x7ff5181f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:12.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51d79b700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5180d15c0 0x7ff5181f52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:12.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51d79b700 1 -- 172.21.15.60:0/4133496139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514009430 con 0x7ff5180b8570 2023-07-13T22:22:12.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51d79b700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5180b8570 0x7ff5181f1590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff514000c00 tx=0x7ff514004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:12.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff50dffb700 1 -- 172.21.15.60:0/4133496139 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514025430 con 0x7ff5180b8570 2023-07-13T22:22:12.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff50dffb700 1 -- 172.21.15.60:0/4133496139 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff514020070 con 0x7ff5180b8570 2023-07-13T22:22:12.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff50dffb700 1 -- 172.21.15.60:0/4133496139 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5140257d0 con 0x7ff5180b8570 2023-07-13T22:22:12.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5180cc550 con 0x7ff5180b8570 2023-07-13T22:22:12.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.874+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5181f1ad0 con 0x7ff5180b8570 2023-07-13T22:22:12.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.875+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5180d1ea0 con 0x7ff5180b8570 2023-07-13T22:22:12.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.876+0000 7ff50dffb700 1 -- 172.21.15.60:0/4133496139 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff514022030 con 0x7ff5180b8570 2023-07-13T22:22:12.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.877+0000 7ff50dffb700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4fc06ace0 0x7ff4fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:12.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.877+0000 7ff50dffb700 1 -- 172.21.15.60:0/4133496139 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff51408a700 con 0x7ff5180b8570 2023-07-13T22:22:12.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.877+0000 7ff51cf9a700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4fc06ace0 0x7ff4fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:12.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.878+0000 7ff51cf9a700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4fc06ace0 0x7ff4fc06d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff5180d2430 tx=0x7ff508009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:12.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:12.881+0000 7ff50dffb700 1 -- 172.21.15.60:0/4133496139 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff514055af0 con 0x7ff5180b8570 2023-07-13T22:22:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.168+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7ff5180d3170 con 0x7ff5180b8570 2023-07-13T22:22:13.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.172+0000 7ff50dffb700 1 -- 172.21.15.60:0/4133496139 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff514059110 con 0x7ff5180b8570 2023-07-13T22:22:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4fc06ace0 msgr2=0x7ff4fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4fc06ace0 0x7ff4fc06d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff5180d2430 tx=0x7ff508009040 comp rx=0 tx=0).stop 2023-07-13T22:22:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5180b8570 msgr2=0x7ff5181f1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5180b8570 0x7ff5181f1590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff514000c00 tx=0x7ff514004750 comp rx=0 tx=0).stop 2023-07-13T22:22:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 shutdown_connections 2023-07-13T22:22:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4fc06ace0 0x7ff4fc06d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5180b8570 0x7ff5181f1590 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5180d35c0 0x7ff5181f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 --2- 172.21.15.60:0/4133496139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5180d15c0 0x7ff5181f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 >> 172.21.15.60:0/4133496139 conn(0x7ff518071600 msgr2=0x7ff51815ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:13.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 shutdown_connections 2023-07-13T22:22:13.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.175+0000 7ff51f9ff700 1 -- 172.21.15.60:0/4133496139 wait complete. 2023-07-13T22:22:13.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-07-13T22:22:13.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.396+0000 7f09ac3ff700 1 Processor -- start 2023-07-13T22:22:13.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.396+0000 7f09ac3ff700 1 -- start start 2023-07-13T22:22:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.397+0000 7f09ac3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f09a40d3750 0x7f09a40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.397+0000 7f09ac3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40cd950 con 0x7f09a40cfad0 2023-07-13T22:22:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.397+0000 7f09ac3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40d0ac0 con 0x7f09a40b8400 2023-07-13T22:22:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.397+0000 7f09ac3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40cff50 con 0x7f09a40d3750 2023-07-13T22:22:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.397+0000 7f09aa99c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f09a40cfad0 0x7f09a40d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54294/0 (socket says 172.21.15.60:54294) 2023-07-13T22:22:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.397+0000 7f09aa99c700 1 -- 172.21.15.60:0/1154343033 learned_addr learned my addr 172.21.15.60:0/1154343033 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:13.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075246098 0 0) 0x7f099c001490 con 0x7f09a40cfad0 2023-07-13T22:22:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a999a700 1 --2- 172.21.15.60:0/1154343033 >> v2:172.21.15.105:3300/0 conn(0x7f09a40d3750 0x7f09a40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09900035b0 con 0x7f09a40cfad0 2023-07-13T22:22:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 705169350 0 0) 0x7f09900035b0 con 0x7f09a40cfad0 2023-07-13T22:22:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 >> v2:172.21.15.105:3300/0 conn(0x7f09a40d3750 msgr2=0x7f09a40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 --2- 172.21.15.60:0/1154343033 >> v2:172.21.15.105:3300/0 conn(0x7f09a40d3750 0x7f09a40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 >> v1:172.21.15.105:6789/0 conn(0x7f09a40b8400 legacy=0x7f09a40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40b87d0 con 0x7f09a40cfad0 2023-07-13T22:22:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f099c001b80 con 0x7f09a40cfad0 2023-07-13T22:22:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 >> v1:172.21.15.60:6789/0 conn(0x7f09a40cfad0 legacy=0x7f09a40d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0990003a00 0x7f0990003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0990004850 0x7f0990004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f099000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40cff50 con 0x7f0990005200 2023-07-13T22:22:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099000c110 con 0x7f0990004850 2023-07-13T22:22:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.398+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099000c2d0 con 0x7f0990003a00 2023-07-13T22:22:13.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09aa99c700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f099000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09aa19b700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0990003a00 0x7f0990003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09a999a700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0990004850 0x7f0990004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09aa99c700 1 -- 172.21.15.60:0/1154343033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0990003a00 msgr2=0x7f0990003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:13.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09aa99c700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0990003a00 0x7f0990003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09aa99c700 1 -- 172.21.15.60:0/1154343033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0990004850 msgr2=0x7f0990004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:13.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09aa99c700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0990004850 0x7f0990004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09aa99c700 1 -- 172.21.15.60:0/1154343033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900035b0 con 0x7f0990005200 2023-07-13T22:22:13.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09a999a700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0990004850 0x7f0990004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:13.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09aa99c700 1 -- 172.21.15.60:0/1154343033 reap_dead start 2023-07-13T22:22:13.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09aa99c700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f099000ba10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f099c00a830 tx=0x7f0990005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543a3fea278ea1e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:13.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c00a490 con 0x7f0990005200 2023-07-13T22:22:13.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c012620 con 0x7f0990005200 2023-07-13T22:22:13.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.399+0000 7f09a9199700 1 -- 172.21.15.60:0/1154343033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0129d0 con 0x7f0990005200 2023-07-13T22:22:13.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.400+0000 7f09ac3ff700 1 -- 172.21.15.60:0/1154343033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 msgr2=0x7f099000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.400+0000 7f09ac3ff700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f099000ba10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f099c00a830 tx=0x7f0990005880 comp rx=0 tx=0).stop 2023-07-13T22:22:13.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.400+0000 7f09ac3ff700 1 -- 172.21.15.60:0/1154343033 shutdown_connections 2023-07-13T22:22:13.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.400+0000 7f09ac3ff700 1 --2- 172.21.15.60:0/1154343033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f099000ba10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.400+0000 7f09ac3ff700 1 -- 172.21.15.60:0/1154343033 >> 172.21.15.60:0/1154343033 conn(0x7f09a406b390 msgr2=0x7f09a406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:13.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.400+0000 7f09ac3ff700 1 -- 172.21.15.60:0/1154343033 shutdown_connections 2023-07-13T22:22:13.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.400+0000 7f09ac3ff700 1 -- 172.21.15.60:0/1154343033 wait complete. 2023-07-13T22:22:13.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09ac3ff700 1 Processor -- start 2023-07-13T22:22:13.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09ac3ff700 1 -- start start 2023-07-13T22:22:13.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09ac3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f09a4151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09ac3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09a415eac0 0x7f09a415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09ac3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f09a40b6f70 0x7f09a40b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09ac3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40b7950 con 0x7f0990005200 2023-07-13T22:22:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09ac3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41515d0 con 0x7f09a415eac0 2023-07-13T22:22:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09ac3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41f2010 con 0x7f09a40b6f70 2023-07-13T22:22:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09aa19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f09a4151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09aa19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f09a4151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45526/0 (socket says 172.21.15.60:45526) 2023-07-13T22:22:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.403+0000 7f09aa19b700 1 -- 172.21.15.60:0/2121765019 learned_addr learned my addr 172.21.15.60:0/2121765019 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f09aa19b700 1 -- 172.21.15.60:0/2121765019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f09a40b6f70 msgr2=0x7f09a40b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f09aa19b700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f09a40b6f70 0x7f09a40b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f09aa19b700 1 -- 172.21.15.60:0/2121765019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09a415eac0 msgr2=0x7f09a415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f09aa19b700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09a415eac0 0x7f09a415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f09aa19b700 1 -- 172.21.15.60:0/2121765019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a41f22b0 con 0x7f0990005200 2023-07-13T22:22:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f09aa19b700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f09a4151090 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f09a000f7f0 tx=0x7f09a000fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f099affd700 1 -- 172.21.15.60:0/2121765019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00052e0 con 0x7f0990005200 2023-07-13T22:22:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f099affd700 1 -- 172.21.15.60:0/2121765019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0005440 con 0x7f0990005200 2023-07-13T22:22:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f099affd700 1 -- 172.21.15.60:0/2121765019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00038a0 con 0x7f0990005200 2023-07-13T22:22:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a41f2590 con 0x7f0990005200 2023-07-13T22:22:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.404+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a41f2b50 con 0x7f0990005200 2023-07-13T22:22:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.405+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a41f2ce0 con 0x7f0990005200 2023-07-13T22:22:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.406+0000 7f099affd700 1 -- 172.21.15.60:0/2121765019 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f09a0003a00 con 0x7f0990005200 2023-07-13T22:22:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.408+0000 7f099affd700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f098806ace0 0x7f098806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.408+0000 7f099affd700 1 -- 172.21.15.60:0/2121765019 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f09a0088f50 con 0x7f0990005200 2023-07-13T22:22:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.408+0000 7f09a999a700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f098806ace0 0x7f098806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.409+0000 7f09a999a700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f098806ace0 0x7f098806d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f09a40c6220 tx=0x7f0994008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:13.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.412+0000 7f099affd700 1 -- 172.21.15.60:0/2121765019 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f09a0054340 con 0x7f0990005200 2023-07-13T22:22:13.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.702+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f09a40cf8b0 con 0x7f0990005200 2023-07-13T22:22:13.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.706+0000 7f099affd700 1 -- 172.21.15.60:0/2121765019 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f09a0003cb0 con 0x7f0990005200 2023-07-13T22:22:13.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f098806ace0 msgr2=0x7f098806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f098806ace0 0x7f098806d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f09a40c6220 tx=0x7f0994008040 comp rx=0 tx=0).stop 2023-07-13T22:22:13.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 msgr2=0x7f09a4151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f09a4151090 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f09a000f7f0 tx=0x7f09a000fbb0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 shutdown_connections 2023-07-13T22:22:13.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f098806ace0 0x7f098806d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0990005200 0x7f09a4151090 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f09a40b6f70 0x7f09a40b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 --2- 172.21.15.60:0/2121765019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09a415eac0 0x7f09a415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 >> 172.21.15.60:0/2121765019 conn(0x7f09a406b390 msgr2=0x7f09a40d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:13.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 shutdown_connections 2023-07-13T22:22:13.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.708+0000 7f09ac3ff700 1 -- 172.21.15.60:0/2121765019 wait complete. 2023-07-13T22:22:13.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-07-13T22:22:13.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.931+0000 7fe7151ff700 1 Processor -- start 2023-07-13T22:22:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.931+0000 7fe7151ff700 1 -- start start 2023-07-13T22:22:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.932+0000 7fe7151ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe7100d3750 0x7fe7100cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.932+0000 7fe7151ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe7100cfad0 0x7fe7100d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.932+0000 7fe7151ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7100cd950 con 0x7fe7100b8400 2023-07-13T22:22:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.932+0000 7fe7151ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7100cff50 con 0x7fe7100d3750 2023-07-13T22:22:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.932+0000 7fe7151ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7100d0ac0 con 0x7fe7100cfad0 2023-07-13T22:22:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70e59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe7100d3750 0x7fe7100cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70ed9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe7100b8400 0x7fe7100cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54306/0 (socket says 172.21.15.60:54306) 2023-07-13T22:22:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70ed9d700 1 -- 172.21.15.60:0/1359380512 learned_addr learned my addr 172.21.15.60:0/1359380512 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70f59e700 1 --2- 172.21.15.60:0/1359380512 >> v2:172.21.15.105:3300/0 conn(0x7fe7100cfad0 0x7fe7100d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70e59c700 1 -- 172.21.15.60:0/1359380512 >> v2:172.21.15.105:3300/0 conn(0x7fe7100cfad0 msgr2=0x7fe7100d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70e59c700 1 --2- 172.21.15.60:0/1359380512 >> v2:172.21.15.105:3300/0 conn(0x7fe7100cfad0 0x7fe7100d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70e59c700 1 -- 172.21.15.60:0/1359380512 >> v1:172.21.15.60:6789/0 conn(0x7fe7100b8400 legacy=0x7fe7100cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70e59c700 1 -- 172.21.15.60:0/1359380512 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7100b87d0 con 0x7fe7100d3750 2023-07-13T22:22:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70f59e700 1 --2- 172.21.15.60:0/1359380512 >> v2:172.21.15.105:3300/0 conn(0x7fe7100cfad0 0x7fe7100d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70dd9b700 1 -- 172.21.15.60:0/1359380512 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396890601 0 0) 0x7fe700001110 con 0x7fe7100b8400 2023-07-13T22:22:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.933+0000 7fe70e59c700 1 --2- 172.21.15.60:0/1359380512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 0x7fe7100cf590 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe7100b1b40 tx=0x7fe704009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4db2a57ce10308 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe70dd9b700 1 -- 172.21.15.60:0/1359380512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704011830 con 0x7fe7100d3750 2023-07-13T22:22:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe70dd9b700 1 -- 172.21.15.60:0/1359380512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704011990 con 0x7fe7100d3750 2023-07-13T22:22:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe70dd9b700 1 -- 172.21.15.60:0/1359380512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704011d20 con 0x7fe7100d3750 2023-07-13T22:22:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe7151ff700 1 -- 172.21.15.60:0/1359380512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 msgr2=0x7fe7100cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe7151ff700 1 --2- 172.21.15.60:0/1359380512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 0x7fe7100cf590 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe7100b1b40 tx=0x7fe704009b20 comp rx=0 tx=0).stop 2023-07-13T22:22:13.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe7151ff700 1 -- 172.21.15.60:0/1359380512 shutdown_connections 2023-07-13T22:22:13.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe7151ff700 1 --2- 172.21.15.60:0/1359380512 >> v2:172.21.15.105:3300/0 conn(0x7fe7100cfad0 0x7fe7100d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe7151ff700 1 --2- 172.21.15.60:0/1359380512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 0x7fe7100cf590 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe7151ff700 1 -- 172.21.15.60:0/1359380512 >> 172.21.15.60:0/1359380512 conn(0x7fe71006b390 msgr2=0x7fe71006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:13.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe7151ff700 1 -- 172.21.15.60:0/1359380512 shutdown_connections 2023-07-13T22:22:13.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.934+0000 7fe7151ff700 1 -- 172.21.15.60:0/1359380512 wait complete. 2023-07-13T22:22:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.936+0000 7fe7151ff700 1 Processor -- start 2023-07-13T22:22:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.936+0000 7fe7151ff700 1 -- start start 2023-07-13T22:22:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.937+0000 7fe7151ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7100b8400 0x7fe710154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.937+0000 7fe7151ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7100cfad0 0x7fe7100cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.937+0000 7fe7151ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 0x7fe7100ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.937+0000 7fe7151ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71015e420 con 0x7fe7100d3750 2023-07-13T22:22:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.937+0000 7fe7151ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71015d810 con 0x7fe7100cfad0 2023-07-13T22:22:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.937+0000 7fe7151ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71015daf0 con 0x7fe7100b8400 2023-07-13T22:22:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.937+0000 7fe70f59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 0x7fe7100ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.937+0000 7fe70f59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 0x7fe7100ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45546/0 (socket says 172.21.15.60:45546) 2023-07-13T22:22:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.937+0000 7fe70f59e700 1 -- 172.21.15.60:0/2575597430 learned_addr learned my addr 172.21.15.60:0/2575597430 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe70ed9d700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7100b8400 0x7fe710154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe70e59c700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7100cfad0 0x7fe7100cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe70f59e700 1 -- 172.21.15.60:0/2575597430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7100b8400 msgr2=0x7fe710154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe70f59e700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7100b8400 0x7fe710154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe70f59e700 1 -- 172.21.15.60:0/2575597430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7100cfad0 msgr2=0x7fe7100cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe70f59e700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7100cfad0 0x7fe7100cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe70f59e700 1 -- 172.21.15.60:0/2575597430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7040094a0 con 0x7fe7100d3750 2023-07-13T22:22:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe70f59e700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 0x7fe7100ca5b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe708007e70 tx=0x7fe70800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:13.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe6f77fe700 1 -- 172.21.15.60:0/2575597430 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70800de40 con 0x7fe7100d3750 2023-07-13T22:22:13.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe6f77fe700 1 -- 172.21.15.60:0/2575597430 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70800f040 con 0x7fe7100d3750 2023-07-13T22:22:13.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71015ddd0 con 0x7fe7100d3750 2023-07-13T22:22:13.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe6f77fe700 1 -- 172.21.15.60:0/2575597430 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708003c60 con 0x7fe7100d3750 2023-07-13T22:22:13.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.938+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7100b7be0 con 0x7fe7100d3750 2023-07-13T22:22:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.939+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71015e7b0 con 0x7fe7100d3750 2023-07-13T22:22:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.940+0000 7fe6f77fe700 1 -- 172.21.15.60:0/2575597430 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe708004030 con 0x7fe7100d3750 2023-07-13T22:22:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.941+0000 7fe6f77fe700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6fc06ace0 0x7fe6fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.941+0000 7fe6f77fe700 1 -- 172.21.15.60:0/2575597430 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe708089310 con 0x7fe7100d3750 2023-07-13T22:22:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.942+0000 7fe70ed9d700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6fc06ace0 0x7fe6fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:13.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.942+0000 7fe70ed9d700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6fc06ace0 0x7fe6fc06d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe700000f60 tx=0x7fe700007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:13.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:13.946+0000 7fe6f77fe700 1 -- 172.21.15.60:0/2575597430 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe708054670 con 0x7fe7100d3750 2023-07-13T22:22:14.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.236+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fe71015e560 con 0x7fe7100d3750 2023-07-13T22:22:14.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.240+0000 7fe6f77fe700 1 -- 172.21.15.60:0/2575597430 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe708057c90 con 0x7fe7100d3750 2023-07-13T22:22:14.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6fc06ace0 msgr2=0x7fe6fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6fc06ace0 0x7fe6fc06d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe700000f60 tx=0x7fe700007c90 comp rx=0 tx=0).stop 2023-07-13T22:22:14.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 msgr2=0x7fe7100ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 0x7fe7100ca5b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe708007e70 tx=0x7fe70800d3b0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 shutdown_connections 2023-07-13T22:22:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6fc06ace0 0x7fe6fc06d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7100d3750 0x7fe7100ca5b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7100b8400 0x7fe710154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 --2- 172.21.15.60:0/2575597430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7100cfad0 0x7fe7100cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 >> 172.21.15.60:0/2575597430 conn(0x7fe71006b390 msgr2=0x7fe7100b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 shutdown_connections 2023-07-13T22:22:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.243+0000 7fe7151ff700 1 -- 172.21.15.60:0/2575597430 wait complete. 2023-07-13T22:22:14.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-07-13T22:22:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.465+0000 7f8305dff700 1 Processor -- start 2023-07-13T22:22:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.465+0000 7f8305dff700 1 -- start start 2023-07-13T22:22:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.466+0000 7f8305dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83000d3750 0x7f83000cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.466+0000 7f8305dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 0x7f83000d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.466+0000 7f8305dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000cd950 con 0x7f83000b8400 2023-07-13T22:22:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.466+0000 7f8305dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83000cff50 con 0x7f83000d3750 2023-07-13T22:22:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.466+0000 7f8305dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000d0ac0 con 0x7f83000cfad0 2023-07-13T22:22:14.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.466+0000 7f82ff7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f83000b8400 0x7f83000cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54308/0 (socket says 172.21.15.60:54308) 2023-07-13T22:22:14.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.466+0000 7f82ff7fe700 1 -- 172.21.15.60:0/4205258901 learned_addr learned my addr 172.21.15.60:0/4205258901 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:14.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.466+0000 7f82feffd700 1 --2- 172.21.15.60:0/4205258901 >> v2:172.21.15.60:3300/0 conn(0x7f83000d3750 0x7f83000cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.466+0000 7f82fffff700 1 --2- 172.21.15.60:0/4205258901 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 0x7f83000d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82fe7fc700 1 -- 172.21.15.60:0/4205258901 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583722336 0 0) 0x7f82f0001110 con 0x7f83000b8400 2023-07-13T22:22:14.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82fe7fc700 1 -- 172.21.15.60:0/4205258901 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e80035b0 con 0x7f83000b8400 2023-07-13T22:22:14.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82feffd700 1 -- 172.21.15.60:0/4205258901 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 msgr2=0x7f83000d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:14.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82feffd700 1 --2- 172.21.15.60:0/4205258901 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 0x7f83000d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82feffd700 1 -- 172.21.15.60:0/4205258901 >> v1:172.21.15.60:6789/0 conn(0x7f83000b8400 legacy=0x7f83000cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82feffd700 1 -- 172.21.15.60:0/4205258901 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000b87d0 con 0x7f83000d3750 2023-07-13T22:22:14.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82feffd700 1 --2- 172.21.15.60:0/4205258901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 0x7f83000cf590 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009840 tx=0x7f82f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d78e0e7c02e403 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82fe7fc700 1 -- 172.21.15.60:0/4205258901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4011830 con 0x7f83000d3750 2023-07-13T22:22:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82fe7fc700 1 -- 172.21.15.60:0/4205258901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4011990 con 0x7f83000d3750 2023-07-13T22:22:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.467+0000 7f82fe7fc700 1 -- 172.21.15.60:0/4205258901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4011d20 con 0x7f83000d3750 2023-07-13T22:22:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.468+0000 7f8305dff700 1 -- 172.21.15.60:0/4205258901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 msgr2=0x7f83000cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.468+0000 7f8305dff700 1 --2- 172.21.15.60:0/4205258901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 0x7f83000cf590 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009840 tx=0x7f82f4009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.468+0000 7f8305dff700 1 -- 172.21.15.60:0/4205258901 shutdown_connections 2023-07-13T22:22:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.468+0000 7f8305dff700 1 --2- 172.21.15.60:0/4205258901 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 0x7f83000d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.468+0000 7f8305dff700 1 --2- 172.21.15.60:0/4205258901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 0x7f83000cf590 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.468+0000 7f8305dff700 1 -- 172.21.15.60:0/4205258901 >> 172.21.15.60:0/4205258901 conn(0x7f830006b390 msgr2=0x7f830006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.468+0000 7f8305dff700 1 -- 172.21.15.60:0/4205258901 shutdown_connections 2023-07-13T22:22:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.468+0000 7f8305dff700 1 -- 172.21.15.60:0/4205258901 wait complete. 2023-07-13T22:22:14.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.470+0000 7f8305dff700 1 Processor -- start 2023-07-13T22:22:14.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.470+0000 7f8305dff700 1 -- start start 2023-07-13T22:22:14.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f8305dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83000b8400 0x7f83000c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f8305dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 0x7f8300150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f8305dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 0x7f83000b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f8305dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c7600 con 0x7f83000d3750 2023-07-13T22:22:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f8305dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c8750 con 0x7f83000b8400 2023-07-13T22:22:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f8305dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c8a30 con 0x7f83000cfad0 2023-07-13T22:22:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f82fffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 0x7f83000b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f82fffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 0x7f83000b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45554/0 (socket says 172.21.15.60:45554) 2023-07-13T22:22:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f82fffff700 1 -- 172.21.15.60:0/3252147278 learned_addr learned my addr 172.21.15.60:0/3252147278 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f82ff7fe700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83000b8400 0x7f83000c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.471+0000 7f82feffd700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 0x7f8300150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f82fffff700 1 -- 172.21.15.60:0/3252147278 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 msgr2=0x7f8300150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f82fffff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 0x7f8300150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f82fffff700 1 -- 172.21.15.60:0/3252147278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83000b8400 msgr2=0x7f83000c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f82fffff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83000b8400 0x7f83000c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f82fffff700 1 -- 172.21.15.60:0/3252147278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40094a0 con 0x7f83000d3750 2023-07-13T22:22:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f82fffff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 0x7f83000b31e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f82f800efc0 tx=0x7f82f800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f82e7fff700 1 -- 172.21.15.60:0/3252147278 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800cad0 con 0x7f83000d3750 2023-07-13T22:22:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f82e7fff700 1 -- 172.21.15.60:0/3252147278 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f800f040 con 0x7f83000d3750 2023-07-13T22:22:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f82e7fff700 1 -- 172.21.15.60:0/3252147278 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800ce70 con 0x7f83000d3750 2023-07-13T22:22:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83000c8d10 con 0x7f83000d3750 2023-07-13T22:22:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.472+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83001514b0 con 0x7f83000d3750 2023-07-13T22:22:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.473+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83000c9510 con 0x7f83000d3750 2023-07-13T22:22:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.475+0000 7f82e7fff700 1 -- 172.21.15.60:0/3252147278 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82f8014480 con 0x7f83000d3750 2023-07-13T22:22:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.475+0000 7f82e7fff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82ec06ace0 0x7f82ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.475+0000 7f82e7fff700 1 -- 172.21.15.60:0/3252147278 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f82f8089920 con 0x7f83000d3750 2023-07-13T22:22:14.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.476+0000 7f82ff7fe700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82ec06ace0 0x7f82ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.476+0000 7f82ff7fe700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82ec06ace0 0x7f82ec06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82f0007fd0 tx=0x7f82f0007c40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:14.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.481+0000 7f82e7fff700 1 -- 172.21.15.60:0/3252147278 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f83000c9510 con 0x7f83000d3750 2023-07-13T22:22:14.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.773+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f83000c9510 con 0x7f83000d3750 2023-07-13T22:22:14.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.777+0000 7f82e7fff700 1 -- 172.21.15.60:0/3252147278 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f83000c9510 con 0x7f83000d3750 2023-07-13T22:22:14.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82ec06ace0 msgr2=0x7f82ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82ec06ace0 0x7f82ec06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82f0007fd0 tx=0x7f82f0007c40 comp rx=0 tx=0).stop 2023-07-13T22:22:14.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 msgr2=0x7f83000b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 0x7f83000b31e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f82f800efc0 tx=0x7f82f800c0c0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 shutdown_connections 2023-07-13T22:22:14.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82ec06ace0 0x7f82ec06d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83000d3750 0x7f83000b31e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83000b8400 0x7f83000c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 --2- 172.21.15.60:0/3252147278 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83000cfad0 0x7f8300150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 >> 172.21.15.60:0/3252147278 conn(0x7f830006b390 msgr2=0x7f83000d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:14.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 shutdown_connections 2023-07-13T22:22:14.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:14.780+0000 7f8305dff700 1 -- 172.21.15.60:0/3252147278 wait complete. 2023-07-13T22:22:14.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:14.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.000+0000 7f72d81ff700 1 Processor -- start 2023-07-13T22:22:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.001+0000 7f72d81ff700 1 -- start start 2023-07-13T22:22:14.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.002+0000 7f72d81ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f72d0154950 0x7f72d0070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.002+0000 7f72d81ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f72d00c9450 0x7f72d015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.002+0000 7f72d81ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d0070270 con 0x7f72d015a820 2023-07-13T22:22:14.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.002+0000 7f72d81ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d015b2e0 con 0x7f72d0154950 2023-07-13T22:22:14.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.002+0000 7f72d81ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d015b5c0 con 0x7f72d00c9450 2023-07-13T22:22:14.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.002+0000 7f72d5f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f72d0154950 0x7f72d0070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.002+0000 7f72d579a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f72d015a820 0x7f72d00c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34158/0 (socket says 172.21.15.60:34158) 2023-07-13T22:22:14.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.002+0000 7f72d579a700 1 -- 172.21.15.60:0/4161491830 learned_addr learned my addr 172.21.15.60:0/4161491830 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:14.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d679c700 1 --2- 172.21.15.60:0/4161491830 >> v2:172.21.15.105:3300/0 conn(0x7f72d00c9450 0x7f72d015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d5f9b700 1 -- 172.21.15.60:0/4161491830 >> v2:172.21.15.105:3300/0 conn(0x7f72d00c9450 msgr2=0x7f72d015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:14.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d5f9b700 1 --2- 172.21.15.60:0/4161491830 >> v2:172.21.15.105:3300/0 conn(0x7f72d00c9450 0x7f72d015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d5f9b700 1 -- 172.21.15.60:0/4161491830 >> v1:172.21.15.60:6789/0 conn(0x7f72d015a820 legacy=0x7f72d00c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d5f9b700 1 -- 172.21.15.60:0/4161491830 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d00704c0 con 0x7f72d0154950 2023-07-13T22:22:14.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d679c700 1 --2- 172.21.15.60:0/4161491830 >> v2:172.21.15.105:3300/0 conn(0x7f72d00c9450 0x7f72d015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:14.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d5f9b700 1 --2- 172.21.15.60:0/4161491830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 0x7f72d0070cc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f72d006b060 tx=0x7f72cc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b7c2a2ddbb12b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:14.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d4f99700 1 -- 172.21.15.60:0/4161491830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc011830 con 0x7f72d0154950 2023-07-13T22:22:14.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d4f99700 1 -- 172.21.15.60:0/4161491830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc011990 con 0x7f72d0154950 2023-07-13T22:22:14.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.003+0000 7f72d4f99700 1 -- 172.21.15.60:0/4161491830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc011d20 con 0x7f72d0154950 2023-07-13T22:22:14.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.004+0000 7f72d81ff700 1 -- 172.21.15.60:0/4161491830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 msgr2=0x7f72d0070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.004+0000 7f72d81ff700 1 --2- 172.21.15.60:0/4161491830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 0x7f72d0070cc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f72d006b060 tx=0x7f72cc009b20 comp rx=0 tx=0).stop 2023-07-13T22:22:14.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.004+0000 7f72d81ff700 1 -- 172.21.15.60:0/4161491830 shutdown_connections 2023-07-13T22:22:14.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.004+0000 7f72d81ff700 1 --2- 172.21.15.60:0/4161491830 >> v2:172.21.15.105:3300/0 conn(0x7f72d00c9450 0x7f72d015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.004+0000 7f72d81ff700 1 --2- 172.21.15.60:0/4161491830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 0x7f72d0070cc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.004+0000 7f72d81ff700 1 -- 172.21.15.60:0/4161491830 >> 172.21.15.60:0/4161491830 conn(0x7f72d00cf410 msgr2=0x7f72d00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:14.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.004+0000 7f72d81ff700 1 -- 172.21.15.60:0/4161491830 shutdown_connections 2023-07-13T22:22:14.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.004+0000 7f72d81ff700 1 -- 172.21.15.60:0/4161491830 wait complete. 2023-07-13T22:22:14.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.006+0000 7f72d81ff700 1 Processor -- start 2023-07-13T22:22:14.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.006+0000 7f72d81ff700 1 -- start start 2023-07-13T22:22:14.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d81ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d00c9450 0x7f72d00c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d81ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 0x7f72d015dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d81ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d015a820 0x7f72d014b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d81ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015c270 con 0x7f72d0154950 2023-07-13T22:22:14.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d81ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015b9b0 con 0x7f72d015a820 2023-07-13T22:22:14.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d81ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015bc20 con 0x7f72d00c9450 2023-07-13T22:22:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d579a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 0x7f72d015dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d579a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 0x7f72d015dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46616/0 (socket says 172.21.15.60:46616) 2023-07-13T22:22:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d579a700 1 -- 172.21.15.60:0/4143490862 learned_addr learned my addr 172.21.15.60:0/4143490862 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d679c700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d015a820 0x7f72d014b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d5f9b700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d00c9450 0x7f72d00c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d579a700 1 -- 172.21.15.60:0/4143490862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d00c9450 msgr2=0x7f72d00c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d579a700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d00c9450 0x7f72d00c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d579a700 1 -- 172.21.15.60:0/4143490862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d015a820 msgr2=0x7f72d014b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:14.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d579a700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d015a820 0x7f72d014b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:14.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.007+0000 7f72d579a700 1 -- 172.21.15.60:0/4143490862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0094a0 con 0x7f72d0154950 2023-07-13T22:22:14.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.008+0000 7f72d579a700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 0x7f72d015dbd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c4b0 tx=0x7f72c000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:14.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.008+0000 7f72c67fc700 1 -- 172.21.15.60:0/4143490862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0005d70 con 0x7f72d0154950 2023-07-13T22:22:14.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.008+0000 7f72c67fc700 1 -- 172.21.15.60:0/4143490862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c00070d0 con 0x7f72d0154950 2023-07-13T22:22:14.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.008+0000 7f72c67fc700 1 -- 172.21.15.60:0/4143490862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0007440 con 0x7f72d0154950 2023-07-13T22:22:14.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.008+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d015bf00 con 0x7f72d0154950 2023-07-13T22:22:14.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.008+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d00c65a0 con 0x7f72d0154950 2023-07-13T22:22:14.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.009+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d015c420 con 0x7f72d0154950 2023-07-13T22:22:14.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.010+0000 7f72c67fc700 1 -- 172.21.15.60:0/4143490862 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72c00075a0 con 0x7f72d0154950 2023-07-13T22:22:14.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.012+0000 7f72c67fc700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b406ace0 0x7f72b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:14.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.012+0000 7f72c67fc700 1 -- 172.21.15.60:0/4143490862 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f72c0088510 con 0x7f72d0154950 2023-07-13T22:22:14.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.012+0000 7f72d5f9b700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b406ace0 0x7f72b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:14.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.012+0000 7f72d5f9b700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b406ace0 0x7f72b406d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0090b0 tx=0x7f72cc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:14.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.016+0000 7f72c67fc700 1 -- 172.21.15.60:0/4143490862 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f72d015c420 con 0x7f72d0154950 2023-07-13T22:22:15.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.224+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72d015c420 con 0x7f72d0154950 2023-07-13T22:22:15.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.225+0000 7f72c67fc700 1 -- 172.21.15.60:0/4143490862 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+14545 (secure 0 0 0) 0x7f72d015c420 con 0x7f72d0154950 2023-07-13T22:22:15.209 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:15.209 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":31,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:22:11.009748+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:15.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b406ace0 msgr2=0x7f72b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b406ace0 0x7f72b406d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0090b0 tx=0x7f72cc008030 comp rx=0 tx=0).stop 2023-07-13T22:22:15.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 msgr2=0x7f72d015dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 0x7f72d015dbd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c4b0 tx=0x7f72c000c870 comp rx=0 tx=0).stop 2023-07-13T22:22:15.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 shutdown_connections 2023-07-13T22:22:15.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b406ace0 0x7f72b406d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d015a820 0x7f72d014b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d00c9450 0x7f72d00c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 --2- 172.21.15.60:0/4143490862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d0154950 0x7f72d015dbd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 >> 172.21.15.60:0/4143490862 conn(0x7f72d00cf410 msgr2=0x7f72d00c44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:15.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 shutdown_connections 2023-07-13T22:22:15.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.228+0000 7f72d81ff700 1 -- 172.21.15.60:0/4143490862 wait complete. 2023-07-13T22:22:15.229 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:22:15.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:22:15.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.452+0000 7fa8eb9ff700 1 Processor -- start 2023-07-13T22:22:15.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.453+0000 7fa8eb9ff700 1 -- start start 2023-07-13T22:22:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8eb9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8e40cfd50 0x7fa8e40d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8eb9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e40b1ac0 0x7fa8e40d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8eb9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e40d1870 con 0x7fa8e40b8200 2023-07-13T22:22:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8eb9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e40cf370 con 0x7fa8e40cfd50 2023-07-13T22:22:15.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8eb9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40d12b0 con 0x7fa8e40b1ac0 2023-07-13T22:22:15.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8e979b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa8e40b8200 0x7fa8e40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34172/0 (socket says 172.21.15.60:34172) 2023-07-13T22:22:15.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8e979b700 1 -- 172.21.15.60:0/1058916139 learned_addr learned my addr 172.21.15.60:0/1058916139 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:15.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8e9f9c700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e40b1ac0 0x7fa8e40d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8e8f9a700 1 --2- 172.21.15.60:0/1058916139 >> v2:172.21.15.105:3300/0 conn(0x7fa8e40cfd50 0x7fa8e40d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168550159 0 0) 0x7fa8e40d1870 con 0x7fa8e40b8200 2023-07-13T22:22:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.454+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8d00035b0 con 0x7fa8e40b8200 2023-07-13T22:22:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2929746166 0 0) 0x7fa8d00035b0 con 0x7fa8e40b8200 2023-07-13T22:22:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e40b1ac0 msgr2=0x7fa8e40d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e40b1ac0 0x7fa8e40d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 >> v2:172.21.15.105:3300/0 conn(0x7fa8e40cfd50 msgr2=0x7fa8e40d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 --2- 172.21.15.60:0/1058916139 >> v2:172.21.15.105:3300/0 conn(0x7fa8e40cfd50 0x7fa8e40d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e40d1590 con 0x7fa8e40b8200 2023-07-13T22:22:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fa8e0001670 con 0x7fa8e40b8200 2023-07-13T22:22:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 >> v1:172.21.15.60:6789/0 conn(0x7fa8e40b8200 legacy=0x7fa8e40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e40b1ac0 0x7fa8d0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8d0003f80 0x7fa8d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40cf370 con 0x7fa8d0004d90 2023-07-13T22:22:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40d12b0 con 0x7fa8d0003f80 2023-07-13T22:22:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d000ba10 con 0x7fa8e40b1ac0 2023-07-13T22:22:15.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8e9f9c700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8e979b700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e40b1ac0 0x7fa8d0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.455+0000 7fa8e8f9a700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8d0003f80 0x7fa8d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8e9f9c700 1 -- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e40b1ac0 msgr2=0x7fa8d0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:15.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8e9f9c700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e40b1ac0 0x7fa8d0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8e9f9c700 1 -- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8d0003f80 msgr2=0x7fa8d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:15.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8e9f9c700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8d0003f80 0x7fa8d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8e9f9c700 1 -- 172.21.15.60:0/1058916139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00035b0 con 0x7fa8d0004d90 2023-07-13T22:22:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8e979b700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e40b1ac0 0x7fa8d0003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8e9f9c700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8d0005200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40b1ee0 tx=0x7fa8d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e8ae779a8b47df server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00ebd0 con 0x7fa8d0004d90 2023-07-13T22:22:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8dc00f040 con 0x7fa8d0004d90 2023-07-13T22:22:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8dbfff700 1 -- 172.21.15.60:0/1058916139 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc014480 con 0x7fa8d0004d90 2023-07-13T22:22:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.456+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/1058916139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 msgr2=0x7fa8d0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.457+0000 7fa8eb9ff700 1 --2- 172.21.15.60:0/1058916139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8d0005200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40b1ee0 tx=0x7fa8d0005880 comp rx=0 tx=0).stop 2023-07-13T22:22:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.457+0000 7fa8e9f9c700 1 -- 172.21.15.60:0/1058916139 reap_dead start 2023-07-13T22:22:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.457+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/1058916139 shutdown_connections 2023-07-13T22:22:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.457+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/1058916139 >> 172.21.15.60:0/1058916139 conn(0x7fa8e406b2f0 msgr2=0x7fa8e406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.457+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/1058916139 shutdown_connections 2023-07-13T22:22:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.457+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/1058916139 wait complete. 2023-07-13T22:22:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.458+0000 7fa8eb9ff700 1 Processor -- start 2023-07-13T22:22:15.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.459+0000 7fa8eb9ff700 1 -- start start 2023-07-13T22:22:15.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.459+0000 7fa8eb9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8e415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.459+0000 7fa8eb9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8e415aca0 0x7fa8e41e1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8eb9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e41e3670 0x7fa8e41e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8eb9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e41e4050 con 0x7fa8d0004d90 2023-07-13T22:22:15.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8eb9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40ca070 con 0x7fa8e415aca0 2023-07-13T22:22:15.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8eb9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e415aa10 con 0x7fa8e41e3670 2023-07-13T22:22:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e979b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8e415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e979b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8e415a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46636/0 (socket says 172.21.15.60:46636) 2023-07-13T22:22:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e979b700 1 -- 172.21.15.60:0/3502567882 learned_addr learned my addr 172.21.15.60:0/3502567882 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e8f9a700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8e415aca0 0x7fa8e41e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e9f9c700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e41e3670 0x7fa8e41e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e979b700 1 -- 172.21.15.60:0/3502567882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e41e3670 msgr2=0x7fa8e41e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e979b700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e41e3670 0x7fa8e41e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e979b700 1 -- 172.21.15.60:0/3502567882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8e415aca0 msgr2=0x7fa8e41e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e979b700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8e415aca0 0x7fa8e41e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.460+0000 7fa8e979b700 1 -- 172.21.15.60:0/3502567882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e41e20c0 con 0x7fa8d0004d90 2023-07-13T22:22:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.461+0000 7fa8e979b700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8e415a4d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000eb20 tx=0x7fa8e000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.461+0000 7fa8d9ffb700 1 -- 172.21.15.60:0/3502567882 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0017070 con 0x7fa8d0004d90 2023-07-13T22:22:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.461+0000 7fa8d9ffb700 1 -- 172.21.15.60:0/3502567882 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e0005e20 con 0x7fa8d0004d90 2023-07-13T22:22:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.461+0000 7fa8d9ffb700 1 -- 172.21.15.60:0/3502567882 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0004870 con 0x7fa8d0004d90 2023-07-13T22:22:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.461+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e41e23a0 con 0x7fa8d0004d90 2023-07-13T22:22:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.461+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e40c9d00 con 0x7fa8d0004d90 2023-07-13T22:22:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.462+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e404e210 con 0x7fa8d0004d90 2023-07-13T22:22:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.463+0000 7fa8d9ffb700 1 -- 172.21.15.60:0/3502567882 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa8e00049d0 con 0x7fa8d0004d90 2023-07-13T22:22:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.464+0000 7fa8d9ffb700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8c806ad10 0x7fa8c806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.464+0000 7fa8d9ffb700 1 -- 172.21.15.60:0/3502567882 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa8e0088930 con 0x7fa8d0004d90 2023-07-13T22:22:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.464+0000 7fa8e8f9a700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8c806ad10 0x7fa8c806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.465+0000 7fa8e8f9a700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8c806ad10 0x7fa8c806d1c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa8e41e30f0 tx=0x7fa8d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.468+0000 7fa8d9ffb700 1 -- 172.21.15.60:0/3502567882 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa8e0053c90 con 0x7fa8d0004d90 2023-07-13T22:22:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.678+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8e40b4f70 con 0x7fa8d0004d90 2023-07-13T22:22:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.678+0000 7fa8d9ffb700 1 -- 172.21.15.60:0/3502567882 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa8e00572b0 con 0x7fa8d0004d90 2023-07-13T22:22:15.662 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:22:15.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8c806ad10 msgr2=0x7fa8c806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8c806ad10 0x7fa8c806d1c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa8e41e30f0 tx=0x7fa8d4008040 comp rx=0 tx=0).stop 2023-07-13T22:22:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 msgr2=0x7fa8e415a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8e415a4d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000eb20 tx=0x7fa8e000ee30 comp rx=0 tx=0).stop 2023-07-13T22:22:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 shutdown_connections 2023-07-13T22:22:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8c806ad10 0x7fa8c806d1c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8d0004d90 0x7fa8e415a4d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e41e3670 0x7fa8e41e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 --2- 172.21.15.60:0/3502567882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8e415aca0 0x7fa8e41e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 >> 172.21.15.60:0/3502567882 conn(0x7fa8e406b2f0 msgr2=0x7fa8e40b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 shutdown_connections 2023-07-13T22:22:15.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.681+0000 7fa8eb9ff700 1 -- 172.21.15.60:0/3502567882 wait complete. 2023-07-13T22:22:15.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:22:15.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.906+0000 7f3f967ff700 1 Processor -- start 2023-07-13T22:22:15.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.906+0000 7f3f967ff700 1 -- start start 2023-07-13T22:22:15.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.907+0000 7f3f967ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b8200 0x7f3f900cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.907+0000 7f3f967ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3f900cede0 0x7f3f900cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.907+0000 7f3f967ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f900d1870 con 0x7f3f900b1ac0 2023-07-13T22:22:15.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.907+0000 7f3f967ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f900cf370 con 0x7f3f900cede0 2023-07-13T22:22:15.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.907+0000 7f3f967ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900d12b0 con 0x7f3f900b8200 2023-07-13T22:22:15.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.908+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b8200 0x7f3f900cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.908+0000 7f3f8f7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3f900b1ac0 0x7f3f9015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:36588/0 (socket says 172.21.15.60:36588) 2023-07-13T22:22:15.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.908+0000 7f3f8f7fe700 1 -- 172.21.15.60:0/762143347 learned_addr learned my addr 172.21.15.60:0/762143347 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:15.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.908+0000 7f3f94d9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3f900cede0 0x7f3f900cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.908+0000 7f3f8effd700 1 -- 172.21.15.60:0/762143347 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897942762 0 0) 0x7f3f900d1870 con 0x7f3f900b1ac0 2023-07-13T22:22:15.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.908+0000 7f3f8effd700 1 -- 172.21.15.60:0/762143347 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f780035b0 con 0x7f3f900b1ac0 2023-07-13T22:22:15.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.908+0000 7f3f94d9c700 1 -- 172.21.15.60:0/762143347 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b8200 msgr2=0x7f3f900cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:15.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.908+0000 7f3f94d9c700 1 --2- 172.21.15.60:0/762143347 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b8200 0x7f3f900cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.908+0000 7f3f94d9c700 1 -- 172.21.15.60:0/762143347 >> v1:172.21.15.105:6789/0 conn(0x7f3f900b1ac0 legacy=0x7f3f9015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.909+0000 7f3f94d9c700 1 -- 172.21.15.60:0/762143347 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900d1590 con 0x7f3f900cede0 2023-07-13T22:22:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.909+0000 7f3f94d9c700 1 --2- 172.21.15.60:0/762143347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f900cede0 0x7f3f900cfb80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009d90 tx=0x7f3f8800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=373994ea6dd9c73b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.909+0000 7f3f8effd700 1 -- 172.21.15.60:0/762143347 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800e070 con 0x7f3f900cede0 2023-07-13T22:22:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.909+0000 7f3f8effd700 1 -- 172.21.15.60:0/762143347 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8800bb20 con 0x7f3f900cede0 2023-07-13T22:22:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.909+0000 7f3f8effd700 1 -- 172.21.15.60:0/762143347 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88012430 con 0x7f3f900cede0 2023-07-13T22:22:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.909+0000 7f3f967ff700 1 -- 172.21.15.60:0/762143347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f900cede0 msgr2=0x7f3f900cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.909+0000 7f3f967ff700 1 --2- 172.21.15.60:0/762143347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f900cede0 0x7f3f900cfb80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009d90 tx=0x7f3f8800b330 comp rx=0 tx=0).stop 2023-07-13T22:22:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.910+0000 7f3f967ff700 1 -- 172.21.15.60:0/762143347 shutdown_connections 2023-07-13T22:22:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.910+0000 7f3f967ff700 1 --2- 172.21.15.60:0/762143347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f900cede0 0x7f3f900cfb80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.910+0000 7f3f967ff700 1 --2- 172.21.15.60:0/762143347 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b8200 0x7f3f900cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.910+0000 7f3f967ff700 1 -- 172.21.15.60:0/762143347 >> 172.21.15.60:0/762143347 conn(0x7f3f9006b2f0 msgr2=0x7f3f9006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.910+0000 7f3f967ff700 1 -- 172.21.15.60:0/762143347 shutdown_connections 2023-07-13T22:22:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.910+0000 7f3f967ff700 1 -- 172.21.15.60:0/762143347 wait complete. 2023-07-13T22:22:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.911+0000 7f3f967ff700 1 Processor -- start 2023-07-13T22:22:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.912+0000 7f3f967ff700 1 -- start start 2023-07-13T22:22:15.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.912+0000 7f3f967ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b1ac0 0x7f3f901e1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.912+0000 7f3f967ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f900b8200 0x7f3f90156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f967ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f900cede0 0x7f3f90158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f967ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90159a00 con 0x7f3f900cede0 2023-07-13T22:22:15.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f967ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90159ce0 con 0x7f3f900b8200 2023-07-13T22:22:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f967ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900cadb0 con 0x7f3f900b1ac0 2023-07-13T22:22:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f94d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f900cede0 0x7f3f90158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f94d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f900cede0 0x7f3f90158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46646/0 (socket says 172.21.15.60:46646) 2023-07-13T22:22:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f94d9c700 1 -- 172.21.15.60:0/3249423424 learned_addr learned my addr 172.21.15.60:0/3249423424 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f8f7fe700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f900b8200 0x7f3f90156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f8ffff700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b1ac0 0x7f3f901e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f94d9c700 1 -- 172.21.15.60:0/3249423424 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b1ac0 msgr2=0x7f3f901e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f94d9c700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b1ac0 0x7f3f901e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f94d9c700 1 -- 172.21.15.60:0/3249423424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f900b8200 msgr2=0x7f3f90156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f94d9c700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f900b8200 0x7f3f90156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f94d9c700 1 -- 172.21.15.60:0/3249423424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f88009a70 con 0x7f3f900cede0 2023-07-13T22:22:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.913+0000 7f3f94d9c700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f900cede0 0x7f3f90158f40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3f88014040 tx=0x7f3f88004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.914+0000 7f3f8cff9700 1 -- 172.21.15.60:0/3249423424 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f880129f0 con 0x7f3f900cede0 2023-07-13T22:22:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.914+0000 7f3f8cff9700 1 -- 172.21.15.60:0/3249423424 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f88020070 con 0x7f3f900cede0 2023-07-13T22:22:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.914+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f900cb090 con 0x7f3f900cede0 2023-07-13T22:22:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.914+0000 7f3f8cff9700 1 -- 172.21.15.60:0/3249423424 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88012dd0 con 0x7f3f900cede0 2023-07-13T22:22:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.914+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f900ccb30 con 0x7f3f900cede0 2023-07-13T22:22:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.915+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f900cb5b0 con 0x7f3f900cede0 2023-07-13T22:22:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.916+0000 7f3f8cff9700 1 -- 172.21.15.60:0/3249423424 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f88004040 con 0x7f3f900cede0 2023-07-13T22:22:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.917+0000 7f3f8cff9700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f7c06ace0 0x7f3f7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.917+0000 7f3f8cff9700 1 -- 172.21.15.60:0/3249423424 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3f88022070 con 0x7f3f900cede0 2023-07-13T22:22:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.917+0000 7f3f8ffff700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f7c06ace0 0x7f3f7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.918+0000 7f3f8ffff700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f7c06ace0 0x7f3f7c06d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009be0 tx=0x7f3f80009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:15.921+0000 7f3f8cff9700 1 -- 172.21.15.60:0/3249423424 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3f900cb5b0 con 0x7f3f900cede0 2023-07-13T22:22:16.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.126+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f900cb5b0 con 0x7f3f900cede0 2023-07-13T22:22:16.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.127+0000 7f3f8cff9700 1 -- 172.21.15.60:0/3249423424 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f900cb5b0 con 0x7f3f900cede0 2023-07-13T22:22:16.111 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:22:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f7c06ace0 msgr2=0x7f3f7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f7c06ace0 0x7f3f7c06d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009be0 tx=0x7f3f80009210 comp rx=0 tx=0).stop 2023-07-13T22:22:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f900cede0 msgr2=0x7f3f90158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f900cede0 0x7f3f90158f40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3f88014040 tx=0x7f3f88004ad0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 shutdown_connections 2023-07-13T22:22:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f7c06ace0 0x7f3f7c06d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f900cede0 0x7f3f90158f40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f900b1ac0 0x7f3f901e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 --2- 172.21.15.60:0/3249423424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f900b8200 0x7f3f90156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 >> 172.21.15.60:0/3249423424 conn(0x7f3f9006b2f0 msgr2=0x7f3f9006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 shutdown_connections 2023-07-13T22:22:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.129+0000 7f3f967ff700 1 -- 172.21.15.60:0/3249423424 wait complete. 2023-07-13T22:22:16.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:22:16.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.351+0000 7f08b2bff700 1 Processor -- start 2023-07-13T22:22:16.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.351+0000 7f08b2bff700 1 -- start start 2023-07-13T22:22:16.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.352+0000 7f08b2bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08ac0b12a0 0x7f08ac0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08b2bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac1609a0 con 0x7f08ac06bad0 2023-07-13T22:22:16.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08b2bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac15c860 con 0x7f08ac0cd760 2023-07-13T22:22:16.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08b2bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac06a5f0 con 0x7f08ac0b12a0 2023-07-13T22:22:16.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08abfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f08ac06bad0 0x7f08ac0b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34180/0 (socket says 172.21.15.60:34180) 2023-07-13T22:22:16.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08b099b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08ac0b12a0 0x7f08ac0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08abfff700 1 -- 172.21.15.60:0/528769287 learned_addr learned my addr 172.21.15.60:0/528769287 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:16.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08b099b700 1 -- 172.21.15.60:0/528769287 >> v1:172.21.15.105:6789/0 conn(0x7f08ac0cd760 legacy=0x7f08ac0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08b099b700 1 -- 172.21.15.60:0/528769287 >> v1:172.21.15.60:6789/0 conn(0x7f08ac06bad0 legacy=0x7f08ac0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08b099b700 1 -- 172.21.15.60:0/528769287 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac06bf20 con 0x7f08ac0b12a0 2023-07-13T22:22:16.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.353+0000 7f08ab7fe700 1 -- 172.21.15.60:0/528769287 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477607450 0 0) 0x7f08ac1609a0 con 0x7f08ac06bad0 2023-07-13T22:22:16.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.354+0000 7f08b099b700 1 --2- 172.21.15.60:0/528769287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 0x7f08ac0cefa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009840 tx=0x7f08a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b797ff17d9c7b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:16.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.354+0000 7f08ab7fe700 1 -- 172.21.15.60:0/528769287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0011830 con 0x7f08ac0b12a0 2023-07-13T22:22:16.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.354+0000 7f08ab7fe700 1 -- 172.21.15.60:0/528769287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0011990 con 0x7f08ac0b12a0 2023-07-13T22:22:16.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.354+0000 7f08ab7fe700 1 -- 172.21.15.60:0/528769287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0011d20 con 0x7f08ac0b12a0 2023-07-13T22:22:16.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.354+0000 7f08b2bff700 1 -- 172.21.15.60:0/528769287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 msgr2=0x7f08ac0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.354+0000 7f08b2bff700 1 --2- 172.21.15.60:0/528769287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 0x7f08ac0cefa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009840 tx=0x7f08a0009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:16.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.355+0000 7f08b2bff700 1 -- 172.21.15.60:0/528769287 shutdown_connections 2023-07-13T22:22:16.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.355+0000 7f08b2bff700 1 --2- 172.21.15.60:0/528769287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 0x7f08ac0cefa0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.355+0000 7f08b2bff700 1 -- 172.21.15.60:0/528769287 >> 172.21.15.60:0/528769287 conn(0x7f08ac0cc2a0 msgr2=0x7f08ac0cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:16.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.355+0000 7f08b2bff700 1 -- 172.21.15.60:0/528769287 shutdown_connections 2023-07-13T22:22:16.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.355+0000 7f08b2bff700 1 -- 172.21.15.60:0/528769287 wait complete. 2023-07-13T22:22:16.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.356+0000 7f08b2bff700 1 Processor -- start 2023-07-13T22:22:16.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.357+0000 7f08b2bff700 1 -- start start 2023-07-13T22:22:16.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.357+0000 7f08b2bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08ac06bad0 0x7f08ac0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.357+0000 7f08b2bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 0x7f08ac0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.357+0000 7f08b2bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08ac0cd760 0x7f08ac155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.357+0000 7f08b2bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac15fe10 con 0x7f08ac0b12a0 2023-07-13T22:22:16.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.357+0000 7f08b2bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac156360 con 0x7f08ac0cd760 2023-07-13T22:22:16.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.357+0000 7f08b2bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0b3650 con 0x7f08ac06bad0 2023-07-13T22:22:16.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08abfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 0x7f08ac0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08abfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 0x7f08ac0b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46664/0 (socket says 172.21.15.60:46664) 2023-07-13T22:22:16.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08abfff700 1 -- 172.21.15.60:0/3682650291 learned_addr learned my addr 172.21.15.60:0/3682650291 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:16.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08b119c700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08ac0cd760 0x7f08ac155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08b099b700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08ac06bad0 0x7f08ac0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08abfff700 1 -- 172.21.15.60:0/3682650291 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08ac06bad0 msgr2=0x7f08ac0b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08abfff700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08ac06bad0 0x7f08ac0b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08abfff700 1 -- 172.21.15.60:0/3682650291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08ac0cd760 msgr2=0x7f08ac155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08abfff700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08ac0cd760 0x7f08ac155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08abfff700 1 -- 172.21.15.60:0/3682650291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a00094a0 con 0x7f08ac0b12a0 2023-07-13T22:22:16.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.358+0000 7f08abfff700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 0x7f08ac0b2fc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f089800c520 tx=0x7f089800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:16.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.359+0000 7f08a97fa700 1 -- 172.21.15.60:0/3682650291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898004c80 con 0x7f08ac0b12a0 2023-07-13T22:22:16.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.359+0000 7f08a97fa700 1 -- 172.21.15.60:0/3682650291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0898004e00 con 0x7f08ac0b12a0 2023-07-13T22:22:16.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.359+0000 7f08a97fa700 1 -- 172.21.15.60:0/3682650291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898007310 con 0x7f08ac0b12a0 2023-07-13T22:22:16.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.359+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac15dd00 con 0x7f08ac0b12a0 2023-07-13T22:22:16.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.359+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac15e2c0 con 0x7f08ac0b12a0 2023-07-13T22:22:16.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.360+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac15e4e0 con 0x7f08ac0b12a0 2023-07-13T22:22:16.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.361+0000 7f08a97fa700 1 -- 172.21.15.60:0/3682650291 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0898007470 con 0x7f08ac0b12a0 2023-07-13T22:22:16.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.362+0000 7f08a97fa700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f089006ace0 0x7f089006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.362+0000 7f08a97fa700 1 -- 172.21.15.60:0/3682650291 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0898088e20 con 0x7f08ac0b12a0 2023-07-13T22:22:16.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.362+0000 7f08b099b700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f089006ace0 0x7f089006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.363+0000 7f08b099b700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f089006ace0 0x7f089006d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f08a00090b0 tx=0x7f08a001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:16.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.366+0000 7f08a97fa700 1 -- 172.21.15.60:0/3682650291 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0898054180 con 0x7f08ac0b12a0 2023-07-13T22:22:16.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.577+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f08ac071c50 con 0x7f08ac0b12a0 2023-07-13T22:22:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.578+0000 7f08a97fa700 1 -- 172.21.15.60:0/3682650291 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+126 (secure 0 0 0) 0x7f08980577a0 con 0x7f08ac0b12a0 2023-07-13T22:22:16.561 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:16.561 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:22:16.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:16.562 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:22:16.562 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:22:16.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:16.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:16.562 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:22:16.562 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:22:16.563 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:16.563 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:22:16.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f089006ace0 msgr2=0x7f089006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f089006ace0 0x7f089006d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f08a00090b0 tx=0x7f08a001c040 comp rx=0 tx=0).stop 2023-07-13T22:22:16.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 msgr2=0x7f08ac0b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 0x7f08ac0b2fc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f089800c520 tx=0x7f089800c830 comp rx=0 tx=0).stop 2023-07-13T22:22:16.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 shutdown_connections 2023-07-13T22:22:16.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f089006ace0 0x7f089006d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08ac0cd760 0x7f08ac155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08ac06bad0 0x7f08ac0b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 --2- 172.21.15.60:0/3682650291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08ac0b12a0 0x7f08ac0b2fc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 >> 172.21.15.60:0/3682650291 conn(0x7f08ac0cc2a0 msgr2=0x7f08ac0b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:16.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 shutdown_connections 2023-07-13T22:22:16.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.580+0000 7f08b2bff700 1 -- 172.21.15.60:0/3682650291 wait complete. 2023-07-13T22:22:16.581 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:22:16.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:22:16.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.807+0000 7fc049fff700 1 Processor -- start 2023-07-13T22:22:16.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.808+0000 7fc049fff700 1 -- start start 2023-07-13T22:22:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.809+0000 7fc049fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc044159ea0 0x7fc0440cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.809+0000 7fc049fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0440d0890 0x7fc0440d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.809+0000 7fc049fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04415d200 con 0x7fc04415c9b0 2023-07-13T22:22:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.809+0000 7fc049fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0440d2e60 con 0x7fc044159ea0 2023-07-13T22:22:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.809+0000 7fc049fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440d2b50 con 0x7fc0440d0890 2023-07-13T22:22:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.809+0000 7fc0437fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc044159ea0 0x7fc0440cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.809+0000 7fc0437fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc044159ea0 0x7fc0440cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46670/0 (socket says 172.21.15.60:46670) 2023-07-13T22:22:16.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.809+0000 7fc0437fe700 1 -- 172.21.15.60:0/3486956928 learned_addr learned my addr 172.21.15.60:0/3486956928 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:16.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.810+0000 7fc043fff700 1 --2- 172.21.15.60:0/3486956928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0440d0890 0x7fc0440d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.810+0000 7fc0437fe700 1 -- 172.21.15.60:0/3486956928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0440d0890 msgr2=0x7fc0440d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.810+0000 7fc0437fe700 1 --2- 172.21.15.60:0/3486956928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0440d0890 0x7fc0440d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.810+0000 7fc0437fe700 1 -- 172.21.15.60:0/3486956928 >> v1:172.21.15.105:6789/0 conn(0x7fc04415c9b0 legacy=0x7fc0440d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.810+0000 7fc0437fe700 1 -- 172.21.15.60:0/3486956928 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04415d4e0 con 0x7fc044159ea0 2023-07-13T22:22:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.810+0000 7fc0437fe700 1 --2- 172.21.15.60:0/3486956928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 0x7fc0440cd6d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc0340098f0 tx=0x7fc034009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24c670d82d56caa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.810+0000 7fc0427fc700 1 -- 172.21.15.60:0/3486956928 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034004230 con 0x7fc044159ea0 2023-07-13T22:22:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.810+0000 7fc0427fc700 1 -- 172.21.15.60:0/3486956928 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc034004390 con 0x7fc044159ea0 2023-07-13T22:22:16.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.810+0000 7fc0427fc700 1 -- 172.21.15.60:0/3486956928 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340036a0 con 0x7fc044159ea0 2023-07-13T22:22:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.811+0000 7fc049fff700 1 -- 172.21.15.60:0/3486956928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 msgr2=0x7fc0440cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.811+0000 7fc049fff700 1 --2- 172.21.15.60:0/3486956928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 0x7fc0440cd6d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc0340098f0 tx=0x7fc034009c00 comp rx=0 tx=0).stop 2023-07-13T22:22:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.811+0000 7fc049fff700 1 -- 172.21.15.60:0/3486956928 shutdown_connections 2023-07-13T22:22:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.811+0000 7fc049fff700 1 --2- 172.21.15.60:0/3486956928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0440d0890 0x7fc0440d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.811+0000 7fc049fff700 1 --2- 172.21.15.60:0/3486956928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 0x7fc0440cd6d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.811+0000 7fc049fff700 1 -- 172.21.15.60:0/3486956928 >> 172.21.15.60:0/3486956928 conn(0x7fc044071600 msgr2=0x7fc0440719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:16.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.811+0000 7fc049fff700 1 -- 172.21.15.60:0/3486956928 shutdown_connections 2023-07-13T22:22:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.811+0000 7fc049fff700 1 -- 172.21.15.60:0/3486956928 wait complete. 2023-07-13T22:22:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.813+0000 7fc049fff700 1 Processor -- start 2023-07-13T22:22:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.814+0000 7fc049fff700 1 -- start start 2023-07-13T22:22:16.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc049fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0440d0890 0x7fc0440c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc049fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 0x7fc04415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc049fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04415c9b0 0x7fc0441f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc049fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c8460 con 0x7fc044159ea0 2023-07-13T22:22:16.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc049fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c8740 con 0x7fc0440d0890 2023-07-13T22:22:16.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc049fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c8a20 con 0x7fc04415c9b0 2023-07-13T22:22:16.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc042ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 0x7fc04415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc042ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 0x7fc04415b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46682/0 (socket says 172.21.15.60:46682) 2023-07-13T22:22:16.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc042ffd700 1 -- 172.21.15.60:0/1104398652 learned_addr learned my addr 172.21.15.60:0/1104398652 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc043fff700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04415c9b0 0x7fc0441f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.815+0000 7fc0437fe700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0440d0890 0x7fc0440c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc042ffd700 1 -- 172.21.15.60:0/1104398652 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04415c9b0 msgr2=0x7fc0441f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc042ffd700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04415c9b0 0x7fc0441f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc042ffd700 1 -- 172.21.15.60:0/1104398652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0440d0890 msgr2=0x7fc0440c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc042ffd700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0440d0890 0x7fc0440c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc042ffd700 1 -- 172.21.15.60:0/1104398652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc034009550 con 0x7fc044159ea0 2023-07-13T22:22:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc042ffd700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 0x7fc04415b3d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc03800c480 tx=0x7fc03800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc02bfff700 1 -- 172.21.15.60:0/1104398652 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038011790 con 0x7fc044159ea0 2023-07-13T22:22:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc02bfff700 1 -- 172.21.15.60:0/1104398652 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03800ce90 con 0x7fc044159ea0 2023-07-13T22:22:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0440c8d00 con 0x7fc044159ea0 2023-07-13T22:22:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc02bfff700 1 -- 172.21.15.60:0/1104398652 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038011b50 con 0x7fc044159ea0 2023-07-13T22:22:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.816+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04415b910 con 0x7fc044159ea0 2023-07-13T22:22:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.817+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc044067a70 con 0x7fc044159ea0 2023-07-13T22:22:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.818+0000 7fc02bfff700 1 -- 172.21.15.60:0/1104398652 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc038011cb0 con 0x7fc044159ea0 2023-07-13T22:22:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.819+0000 7fc02bfff700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03006ace0 0x7fc03006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.819+0000 7fc02bfff700 1 -- 172.21.15.60:0/1104398652 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc038088e50 con 0x7fc044159ea0 2023-07-13T22:22:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.820+0000 7fc0437fe700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03006ace0 0x7fc03006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.820+0000 7fc0437fe700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03006ace0 0x7fc03006d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc034009130 tx=0x7fc034008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:16.824+0000 7fc02bfff700 1 -- 172.21.15.60:0/1104398652 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc0380541b0 con 0x7fc044159ea0 2023-07-13T22:22:17.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.030+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc044066700 con 0x7fc044159ea0 2023-07-13T22:22:17.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.655+0000 7fc02bfff700 1 -- 172.21.15.60:0/1104398652 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc0380577d0 con 0x7fc044159ea0 2023-07-13T22:22:17.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.661+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03006ace0 msgr2=0x7fc03006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:17.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.661+0000 7fc049fff700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03006ace0 0x7fc03006d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc034009130 tx=0x7fc034008030 comp rx=0 tx=0).stop 2023-07-13T22:22:17.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.661+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 msgr2=0x7fc04415b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:17.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.661+0000 7fc049fff700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 0x7fc04415b3d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc03800c480 tx=0x7fc03800c790 comp rx=0 tx=0).stop 2023-07-13T22:22:17.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.662+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 shutdown_connections 2023-07-13T22:22:17.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.662+0000 7fc049fff700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03006ace0 0x7fc03006d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.662+0000 7fc049fff700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04415c9b0 0x7fc0441f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.662+0000 7fc049fff700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0440d0890 0x7fc0440c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.662+0000 7fc049fff700 1 --2- 172.21.15.60:0/1104398652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc044159ea0 0x7fc04415b3d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.662+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 >> 172.21.15.60:0/1104398652 conn(0x7fc044071600 msgr2=0x7fc04415f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.662+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 shutdown_connections 2023-07-13T22:22:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.663+0000 7fc049fff700 1 -- 172.21.15.60:0/1104398652 wait complete. 2023-07-13T22:22:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:22:17.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:22:17.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.897+0000 7fc40dfff700 1 Processor -- start 2023-07-13T22:22:17.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.898+0000 7fc40dfff700 1 -- start start 2023-07-13T22:22:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.899+0000 7fc40dfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc408070a60 0x7fc40806ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.899+0000 7fc40dfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc4080ccfd0 0x7fc40806d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.899+0000 7fc40dfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc408160d10 con 0x7fc408160940 2023-07-13T22:22:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.899+0000 7fc40dfff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4080cd4e0 con 0x7fc408070a60 2023-07-13T22:22:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.899+0000 7fc40dfff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4080cd7c0 con 0x7fc4080ccfd0 2023-07-13T22:22:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.899+0000 7fc406ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc408070a60 0x7fc40806ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.899+0000 7fc406ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc408070a60 0x7fc40806ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46696/0 (socket says 172.21.15.60:46696) 2023-07-13T22:22:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.899+0000 7fc406ffd700 1 -- 172.21.15.60:0/2264457349 learned_addr learned my addr 172.21.15.60:0/2264457349 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.899+0000 7fc407fff700 1 --2- 172.21.15.60:0/2264457349 >> v2:172.21.15.105:3300/0 conn(0x7fc4080ccfd0 0x7fc40806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:17.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.900+0000 7fc406ffd700 1 -- 172.21.15.60:0/2264457349 >> v2:172.21.15.105:3300/0 conn(0x7fc4080ccfd0 msgr2=0x7fc40806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:17.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.900+0000 7fc406ffd700 1 --2- 172.21.15.60:0/2264457349 >> v2:172.21.15.105:3300/0 conn(0x7fc4080ccfd0 0x7fc40806d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.900+0000 7fc406ffd700 1 -- 172.21.15.60:0/2264457349 >> v1:172.21.15.105:6789/0 conn(0x7fc408160940 legacy=0x7fc408070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:17.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.900+0000 7fc406ffd700 1 -- 172.21.15.60:0/2264457349 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40806c4c0 con 0x7fc408070a60 2023-07-13T22:22:17.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.900+0000 7fc4067fc700 1 -- 172.21.15.60:0/2264457349 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756944363 0 0) 0x7fc408160d10 con 0x7fc408160940 2023-07-13T22:22:17.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.900+0000 7fc406ffd700 1 --2- 172.21.15.60:0/2264457349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc408070a60 0x7fc40806ccc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc4080cb170 tx=0x7fc3fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e91364324f912e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:17.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.900+0000 7fc4067fc700 1 -- 172.21.15.60:0/2264457349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc011830 con 0x7fc408070a60 2023-07-13T22:22:17.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.900+0000 7fc4067fc700 1 -- 172.21.15.60:0/2264457349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc011990 con 0x7fc408070a60 2023-07-13T22:22:17.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.900+0000 7fc4067fc700 1 -- 172.21.15.60:0/2264457349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc011d20 con 0x7fc408070a60 2023-07-13T22:22:17.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.901+0000 7fc40dfff700 1 -- 172.21.15.60:0/2264457349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc408070a60 msgr2=0x7fc40806ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:17.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.901+0000 7fc40dfff700 1 --2- 172.21.15.60:0/2264457349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc408070a60 0x7fc40806ccc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc4080cb170 tx=0x7fc3fc009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.901+0000 7fc40dfff700 1 -- 172.21.15.60:0/2264457349 shutdown_connections 2023-07-13T22:22:17.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.901+0000 7fc40dfff700 1 --2- 172.21.15.60:0/2264457349 >> v2:172.21.15.105:3300/0 conn(0x7fc4080ccfd0 0x7fc40806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.901+0000 7fc40dfff700 1 --2- 172.21.15.60:0/2264457349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc408070a60 0x7fc40806ccc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.901+0000 7fc40dfff700 1 -- 172.21.15.60:0/2264457349 >> 172.21.15.60:0/2264457349 conn(0x7fc4080cf3b0 msgr2=0x7fc4080cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:17.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.901+0000 7fc40dfff700 1 -- 172.21.15.60:0/2264457349 shutdown_connections 2023-07-13T22:22:17.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.901+0000 7fc40dfff700 1 -- 172.21.15.60:0/2264457349 wait complete. 2023-07-13T22:22:17.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.903+0000 7fc40dfff700 1 Processor -- start 2023-07-13T22:22:17.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.904+0000 7fc40dfff700 1 -- start start 2023-07-13T22:22:17.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc40dfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc408070a60 0x7fc408145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:17.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc40dfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4080ccfd0 0x7fc4080b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:17.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc40dfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc408160940 0x7fc40815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:17.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc40dfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080c8200 con 0x7fc4080ccfd0 2023-07-13T22:22:17.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc40dfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080c75f0 con 0x7fc408160940 2023-07-13T22:22:17.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc40dfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080c78d0 con 0x7fc408070a60 2023-07-13T22:22:17.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc406ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4080ccfd0 0x7fc4080b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:17.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc406ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4080ccfd0 0x7fc4080b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46708/0 (socket says 172.21.15.60:46708) 2023-07-13T22:22:17.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc406ffd700 1 -- 172.21.15.60:0/2645705409 learned_addr learned my addr 172.21.15.60:0/2645705409 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:17.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc4077fe700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc408070a60 0x7fc408145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:17.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.905+0000 7fc407fff700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc408160940 0x7fc40815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:17.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc406ffd700 1 -- 172.21.15.60:0/2645705409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc408070a60 msgr2=0x7fc408145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:17.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc406ffd700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc408070a60 0x7fc408145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc406ffd700 1 -- 172.21.15.60:0/2645705409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc408160940 msgr2=0x7fc40815a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:17.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc406ffd700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc408160940 0x7fc40815a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:17.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc406ffd700 1 -- 172.21.15.60:0/2645705409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc009550 con 0x7fc4080ccfd0 2023-07-13T22:22:17.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc4077fe700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc408070a60 0x7fc408145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:17.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc406ffd700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4080ccfd0 0x7fc4080b1660 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc000c00 tx=0x7fc3fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:17.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc3effff700 1 -- 172.21.15.60:0/2645705409 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0045d0 con 0x7fc4080ccfd0 2023-07-13T22:22:17.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc3effff700 1 -- 172.21.15.60:0/2645705409 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc01f070 con 0x7fc4080ccfd0 2023-07-13T22:22:17.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4080c7bb0 con 0x7fc4080ccfd0 2023-07-13T22:22:17.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.906+0000 7fc3effff700 1 -- 172.21.15.60:0/2645705409 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0038a0 con 0x7fc4080ccfd0 2023-07-13T22:22:17.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.907+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4080c7280 con 0x7fc4080ccfd0 2023-07-13T22:22:17.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.907+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4080c8370 con 0x7fc4080ccfd0 2023-07-13T22:22:17.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.909+0000 7fc3effff700 1 -- 172.21.15.60:0/2645705409 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3fc003a00 con 0x7fc4080ccfd0 2023-07-13T22:22:17.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.909+0000 7fc3effff700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3f406ace0 0x7fc3f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:17.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.910+0000 7fc3effff700 1 -- 172.21.15.60:0/2645705409 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fc3fc021070 con 0x7fc4080ccfd0 2023-07-13T22:22:17.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.910+0000 7fc4077fe700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3f406ace0 0x7fc3f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:17.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.910+0000 7fc4077fe700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3f406ace0 0x7fc3f406d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000f60 tx=0x7fc3f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:17.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:17.914+0000 7fc3effff700 1 -- 172.21.15.60:0/2645705409 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc3fc055dc0 con 0x7fc4080ccfd0 2023-07-13T22:22:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.121+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc40804e210 con 0x7fc4080ccfd0 2023-07-13T22:22:18.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.658+0000 7fc3effff700 1 -- 172.21.15.60:0/2645705409 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc3fc015020 con 0x7fc4080ccfd0 2023-07-13T22:22:18.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.664+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3f406ace0 msgr2=0x7fc3f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:18.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.664+0000 7fc40dfff700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3f406ace0 0x7fc3f406d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000f60 tx=0x7fc3f8009040 comp rx=0 tx=0).stop 2023-07-13T22:22:18.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.664+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4080ccfd0 msgr2=0x7fc4080b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:18.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.664+0000 7fc40dfff700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4080ccfd0 0x7fc4080b1660 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc000c00 tx=0x7fc3fc004750 comp rx=0 tx=0).stop 2023-07-13T22:22:18.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.665+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 shutdown_connections 2023-07-13T22:22:18.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.665+0000 7fc40dfff700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3f406ace0 0x7fc3f406d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:18.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.665+0000 7fc40dfff700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc408160940 0x7fc40815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:18.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.665+0000 7fc40dfff700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc408070a60 0x7fc408145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:18.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.665+0000 7fc40dfff700 1 --2- 172.21.15.60:0/2645705409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4080ccfd0 0x7fc4080b1660 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:18.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.665+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 >> 172.21.15.60:0/2645705409 conn(0x7fc4080cf3b0 msgr2=0x7fc40806d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.667+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 shutdown_connections 2023-07-13T22:22:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.667+0000 7fc40dfff700 1 -- 172.21.15.60:0/2645705409 wait complete. 2023-07-13T22:22:18.657 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:22:18.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:22:18.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.903+0000 7f66977ff700 1 Processor -- start 2023-07-13T22:22:18.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.903+0000 7f66977ff700 1 -- start start 2023-07-13T22:22:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.904+0000 7f66977ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66900b8200 0x7f66900cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.904+0000 7f66977ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 0x7f66900cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.904+0000 7f66977ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66900d1870 con 0x7f66900b1ac0 2023-07-13T22:22:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.904+0000 7f66977ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66900cf370 con 0x7f66900b8200 2023-07-13T22:22:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.904+0000 7f66977ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900d12b0 con 0x7f66900cede0 2023-07-13T22:22:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.904+0000 7f669559b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66900b8200 0x7f66900cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.904+0000 7f6694d9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f66900b1ac0 0x7f669015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34196/0 (socket says 172.21.15.60:34196) 2023-07-13T22:22:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.904+0000 7f6694d9a700 1 -- 172.21.15.60:0/1600596733 learned_addr learned my addr 172.21.15.60:0/1600596733 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.905+0000 7f6695d9c700 1 --2- 172.21.15.60:0/1600596733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 0x7f66900cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.905+0000 7f669559b700 1 -- 172.21.15.60:0/1600596733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 msgr2=0x7f66900cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.905+0000 7f669559b700 1 --2- 172.21.15.60:0/1600596733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 0x7f66900cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.905+0000 7f669559b700 1 -- 172.21.15.60:0/1600596733 >> v1:172.21.15.60:6789/0 conn(0x7f66900b1ac0 legacy=0x7f669015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:18.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.905+0000 7f669559b700 1 -- 172.21.15.60:0/1600596733 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66900d1590 con 0x7f66900b8200 2023-07-13T22:22:18.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.905+0000 7f669559b700 1 --2- 172.21.15.60:0/1600596733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 0x7f66900cf640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f668c009840 tx=0x7f668c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e986f8f4126c9435 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:18.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.905+0000 7f6687fff700 1 -- 172.21.15.60:0/1600596733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c011830 con 0x7f66900b8200 2023-07-13T22:22:18.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.905+0000 7f6687fff700 1 -- 172.21.15.60:0/1600596733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c011990 con 0x7f66900b8200 2023-07-13T22:22:18.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.905+0000 7f6687fff700 1 -- 172.21.15.60:0/1600596733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c011d20 con 0x7f66900b8200 2023-07-13T22:22:18.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.906+0000 7f66977ff700 1 -- 172.21.15.60:0/1600596733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 msgr2=0x7f66900cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:18.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.906+0000 7f66977ff700 1 --2- 172.21.15.60:0/1600596733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 0x7f66900cf640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f668c009840 tx=0x7f668c009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:18.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.906+0000 7f66977ff700 1 -- 172.21.15.60:0/1600596733 shutdown_connections 2023-07-13T22:22:18.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.906+0000 7f66977ff700 1 --2- 172.21.15.60:0/1600596733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 0x7f66900cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:18.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.906+0000 7f66977ff700 1 --2- 172.21.15.60:0/1600596733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 0x7f66900cf640 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:18.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.906+0000 7f66977ff700 1 -- 172.21.15.60:0/1600596733 >> 172.21.15.60:0/1600596733 conn(0x7f669006b2f0 msgr2=0x7f669006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:18.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.906+0000 7f66977ff700 1 -- 172.21.15.60:0/1600596733 shutdown_connections 2023-07-13T22:22:18.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.906+0000 7f66977ff700 1 -- 172.21.15.60:0/1600596733 wait complete. 2023-07-13T22:22:18.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.908+0000 7f66977ff700 1 Processor -- start 2023-07-13T22:22:18.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.908+0000 7f66977ff700 1 -- start start 2023-07-13T22:22:18.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f66977ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66900b1ac0 0x7f66901e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f66977ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 0x7f66900a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f66977ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 0x7f66901e2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f66977ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66901e2d90 con 0x7f66900b8200 2023-07-13T22:22:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f66977ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66901e47d0 con 0x7f66900b1ac0 2023-07-13T22:22:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f66977ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900c9ea0 con 0x7f66900cede0 2023-07-13T22:22:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f6694d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 0x7f66900a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f6694d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 0x7f66900a9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46716/0 (socket says 172.21.15.60:46716) 2023-07-13T22:22:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f6694d9a700 1 -- 172.21.15.60:0/3407475338 learned_addr learned my addr 172.21.15.60:0/3407475338 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.909+0000 7f669559b700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66900b1ac0 0x7f66901e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6695d9c700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 0x7f66901e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6694d9a700 1 -- 172.21.15.60:0/3407475338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 msgr2=0x7f66901e2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6694d9a700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 0x7f66901e2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6694d9a700 1 -- 172.21.15.60:0/3407475338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66900b1ac0 msgr2=0x7f66901e3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6694d9a700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66900b1ac0 0x7f66901e3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6694d9a700 1 -- 172.21.15.60:0/3407475338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0094a0 con 0x7f66900b8200 2023-07-13T22:22:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f669559b700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66900b1ac0 0x7f66901e3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6694d9a700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 0x7f66900a9590 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f668000c520 tx=0x7f668000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6685ffb700 1 -- 172.21.15.60:0/3407475338 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680005d70 con 0x7f66900b8200 2023-07-13T22:22:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6685ffb700 1 -- 172.21.15.60:0/3407475338 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66800070d0 con 0x7f66900b8200 2023-07-13T22:22:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f6685ffb700 1 -- 172.21.15.60:0/3407475338 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680007440 con 0x7f66900b8200 2023-07-13T22:22:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66900ca180 con 0x7f66900b8200 2023-07-13T22:22:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.910+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66900ca740 con 0x7f66900b8200 2023-07-13T22:22:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.911+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66900ca920 con 0x7f66900b8200 2023-07-13T22:22:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.913+0000 7f6685ffb700 1 -- 172.21.15.60:0/3407475338 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66800075a0 con 0x7f66900b8200 2023-07-13T22:22:18.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.913+0000 7f6685ffb700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f667406ace0 0x7f667406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:18.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.914+0000 7f6685ffb700 1 -- 172.21.15.60:0/3407475338 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f6680088840 con 0x7f66900b8200 2023-07-13T22:22:18.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.914+0000 7f669559b700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f667406ace0 0x7f667406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:18.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.914+0000 7f669559b700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f667406ace0 0x7f667406d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f668c0090b0 tx=0x7f668c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:18.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:18.919+0000 7f6685ffb700 1 -- 172.21.15.60:0/3407475338 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f66800539f0 con 0x7f66900b8200 2023-07-13T22:22:19.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.174+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f669004e210 con 0x7f66900b8200 2023-07-13T22:22:19.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.691+0000 7f6685ffb700 1 -- 172.21.15.60:0/3407475338 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6680057010 con 0x7f66900b8200 2023-07-13T22:22:19.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.698+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f667406ace0 msgr2=0x7f667406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:19.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.698+0000 7f66977ff700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f667406ace0 0x7f667406d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f668c0090b0 tx=0x7f668c008030 comp rx=0 tx=0).stop 2023-07-13T22:22:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.698+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 msgr2=0x7f66900a9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.698+0000 7f66977ff700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 0x7f66900a9590 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f668000c520 tx=0x7f668000c8e0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.699+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 shutdown_connections 2023-07-13T22:22:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.699+0000 7f66977ff700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f667406ace0 0x7f667406d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.699+0000 7f66977ff700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66900cede0 0x7f66901e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.699+0000 7f66977ff700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66900b1ac0 0x7f66901e3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.699+0000 7f66977ff700 1 --2- 172.21.15.60:0/3407475338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b8200 0x7f66900a9590 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.699+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 >> 172.21.15.60:0/3407475338 conn(0x7f669006b2f0 msgr2=0x7f66900d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:19.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.701+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 shutdown_connections 2023-07-13T22:22:19.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.701+0000 7f66977ff700 1 -- 172.21.15.60:0/3407475338 wait complete. 2023-07-13T22:22:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 5 and data pool 6 2023-07-13T22:22:19.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.930+0000 7f514f9ff700 1 Processor -- start 2023-07-13T22:22:19.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.931+0000 7f514f9ff700 1 -- start start 2023-07-13T22:22:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f514f9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480d35c0 0x7f51480d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f514f9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51480d13f0 0x7f514815c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f514f9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51480d09d0 con 0x7f51480b8570 2023-07-13T22:22:19.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f514f9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51480d3b30 con 0x7f51480d13f0 2023-07-13T22:22:19.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f514f9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51480d1870 con 0x7f51480d35c0 2023-07-13T22:22:19.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f514d79b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f51480b8570 0x7f51480ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34202/0 (socket says 172.21.15.60:34202) 2023-07-13T22:22:19.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f514d79b700 1 -- 172.21.15.60:0/163220571 learned_addr learned my addr 172.21.15.60:0/163220571 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:19.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f514cf9a700 1 --2- 172.21.15.60:0/163220571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480d35c0 0x7f51480d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:19.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f514df9c700 1 --2- 172.21.15.60:0/163220571 >> v2:172.21.15.105:3300/0 conn(0x7f51480d13f0 0x7f514815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:19.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.932+0000 7f513ffff700 1 -- 172.21.15.60:0/163220571 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065181036 0 0) 0x7f51480d09d0 con 0x7f51480b8570 2023-07-13T22:22:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.933+0000 7f513ffff700 1 -- 172.21.15.60:0/163220571 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51340035b0 con 0x7f51480b8570 2023-07-13T22:22:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.933+0000 7f514df9c700 1 -- 172.21.15.60:0/163220571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480d35c0 msgr2=0x7f51480d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.933+0000 7f514df9c700 1 --2- 172.21.15.60:0/163220571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480d35c0 0x7f51480d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.933+0000 7f514df9c700 1 -- 172.21.15.60:0/163220571 >> v1:172.21.15.60:6789/0 conn(0x7f51480b8570 legacy=0x7f51480ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.933+0000 7f514df9c700 1 -- 172.21.15.60:0/163220571 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480cda10 con 0x7f51480d13f0 2023-07-13T22:22:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.933+0000 7f514df9c700 1 --2- 172.21.15.60:0/163220571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51480d13f0 0x7f514815c920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5140009d90 tx=0x7f514000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4e5b85877f9a07 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:19.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.933+0000 7f513ffff700 1 -- 172.21.15.60:0/163220571 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514000e070 con 0x7f51480d13f0 2023-07-13T22:22:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.933+0000 7f513ffff700 1 -- 172.21.15.60:0/163220571 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514000bb20 con 0x7f51480d13f0 2023-07-13T22:22:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.933+0000 7f513ffff700 1 -- 172.21.15.60:0/163220571 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140012430 con 0x7f51480d13f0 2023-07-13T22:22:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.934+0000 7f514f9ff700 1 -- 172.21.15.60:0/163220571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51480d13f0 msgr2=0x7f514815c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.934+0000 7f514f9ff700 1 --2- 172.21.15.60:0/163220571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51480d13f0 0x7f514815c920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5140009d90 tx=0x7f514000b330 comp rx=0 tx=0).stop 2023-07-13T22:22:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.934+0000 7f514f9ff700 1 -- 172.21.15.60:0/163220571 shutdown_connections 2023-07-13T22:22:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.934+0000 7f514f9ff700 1 --2- 172.21.15.60:0/163220571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51480d13f0 0x7f514815c920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.934+0000 7f514f9ff700 1 --2- 172.21.15.60:0/163220571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480d35c0 0x7f51480d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.934+0000 7f514f9ff700 1 -- 172.21.15.60:0/163220571 >> 172.21.15.60:0/163220571 conn(0x7f5148071600 msgr2=0x7f51480719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.934+0000 7f514f9ff700 1 -- 172.21.15.60:0/163220571 shutdown_connections 2023-07-13T22:22:19.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.934+0000 7f514f9ff700 1 -- 172.21.15.60:0/163220571 wait complete. 2023-07-13T22:22:19.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.936+0000 7f514f9ff700 1 Processor -- start 2023-07-13T22:22:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.936+0000 7f514f9ff700 1 -- start start 2023-07-13T22:22:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514f9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480b8570 0x7f51480c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514f9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51480d13f0 0x7f51480b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514f9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51480d35c0 0x7f51481ef240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514f9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51481f5200 con 0x7f51480d13f0 2023-07-13T22:22:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514f9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51480c9230 con 0x7f51480d35c0 2023-07-13T22:22:19.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514f9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51480b5fd0 con 0x7f51480b8570 2023-07-13T22:22:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514cf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51480d13f0 0x7f51480b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514cf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51480d13f0 0x7f51480b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46730/0 (socket says 172.21.15.60:46730) 2023-07-13T22:22:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514cf9a700 1 -- 172.21.15.60:0/800655891 learned_addr learned my addr 172.21.15.60:0/800655891 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514d79b700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480b8570 0x7f51480c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.937+0000 7f514df9c700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51480d35c0 0x7f51481ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f514cf9a700 1 -- 172.21.15.60:0/800655891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480b8570 msgr2=0x7f51480c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f514cf9a700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480b8570 0x7f51480c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f514cf9a700 1 -- 172.21.15.60:0/800655891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51480d35c0 msgr2=0x7f51481ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:19.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f514cf9a700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51480d35c0 0x7f51481ef240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:19.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f514cf9a700 1 -- 172.21.15.60:0/800655891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5140009a70 con 0x7f51480d13f0 2023-07-13T22:22:19.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f514cf9a700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51480d13f0 0x7f51480b5a90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f513800e8a0 tx=0x7f513800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:19.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f513dffb700 1 -- 172.21.15.60:0/800655891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138009590 con 0x7f51480d13f0 2023-07-13T22:22:19.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f513dffb700 1 -- 172.21.15.60:0/800655891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5138013680 con 0x7f51480d13f0 2023-07-13T22:22:19.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f513dffb700 1 -- 172.21.15.60:0/800655891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51380139f0 con 0x7f51480d13f0 2023-07-13T22:22:19.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51480b6270 con 0x7f51480d13f0 2023-07-13T22:22:19.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.938+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51480b6830 con 0x7f51480d13f0 2023-07-13T22:22:19.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.939+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51480b6a20 con 0x7f51480d13f0 2023-07-13T22:22:19.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.940+0000 7f513dffb700 1 -- 172.21.15.60:0/800655891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5138013b50 con 0x7f51480d13f0 2023-07-13T22:22:19.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.941+0000 7f513dffb700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512c06ace0 0x7f512c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:19.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.941+0000 7f513dffb700 1 -- 172.21.15.60:0/800655891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5138089320 con 0x7f51480d13f0 2023-07-13T22:22:19.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.941+0000 7f514d79b700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512c06ace0 0x7f512c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:19.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.942+0000 7f514d79b700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512c06ace0 0x7f512c06d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f51440010e0 tx=0x7f5144007c70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:19.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:19.946+0000 7f513dffb700 1 -- 172.21.15.60:0/800655891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5138057dc0 con 0x7f51480d13f0 2023-07-13T22:22:20.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.153+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f514804e280 con 0x7f51480d13f0 2023-07-13T22:22:20.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.154+0000 7f513dffb700 1 -- 172.21.15.60:0/800655891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f5138057950 con 0x7f51480d13f0 2023-07-13T22:22:20.138 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:20.138 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":34,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:22:19.175328+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:22:17.071217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-13T22:22:18.161600+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:20.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512c06ace0 msgr2=0x7f512c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512c06ace0 0x7f512c06d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f51440010e0 tx=0x7f5144007c70 comp rx=0 tx=0).stop 2023-07-13T22:22:20.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51480d13f0 msgr2=0x7f51480b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51480d13f0 0x7f51480b5a90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f513800e8a0 tx=0x7f513800ebb0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 shutdown_connections 2023-07-13T22:22:20.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f512c06ace0 0x7f512c06d190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51480d35c0 0x7f51481ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51480b8570 0x7f51480c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 --2- 172.21.15.60:0/800655891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51480d13f0 0x7f51480b5a90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 >> 172.21.15.60:0/800655891 conn(0x7f5148071600 msgr2=0x7f514815da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:20.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 shutdown_connections 2023-07-13T22:22:20.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.157+0000 7f514f9ff700 1 -- 172.21.15.60:0/800655891 wait complete. 2023-07-13T22:22:20.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.378+0000 7f123f1ff700 1 Processor -- start 2023-07-13T22:22:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.378+0000 7f123f1ff700 1 -- start start 2023-07-13T22:22:20.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.379+0000 7f123f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12380d35c0 0x7f12380d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.379+0000 7f123f1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12380d09d0 con 0x7f12380d13f0 2023-07-13T22:22:20.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.379+0000 7f123f1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12380d1870 con 0x7f12380b8570 2023-07-13T22:22:20.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.379+0000 7f123f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12380d3b30 con 0x7f12380d35c0 2023-07-13T22:22:20.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.379+0000 7f123d79c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f12380d13f0 0x7f123815c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34214/0 (socket says 172.21.15.60:34214) 2023-07-13T22:22:20.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.379+0000 7f123d79c700 1 -- 172.21.15.60:0/3921759100 learned_addr learned my addr 172.21.15.60:0/3921759100 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:20.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f1237fff700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12380d35c0 0x7f12380d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581800342 0 0) 0x7f12380d09d0 con 0x7f12380d13f0 2023-07-13T22:22:20.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12280035b0 con 0x7f12380d13f0 2023-07-13T22:22:20.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 759215156 0 0) 0x7f12280035b0 con 0x7f12380d13f0 2023-07-13T22:22:20.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12380d35c0 msgr2=0x7f12380d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:20.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12380d35c0 0x7f12380d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 >> v1:172.21.15.105:6789/0 conn(0x7f12380b8570 legacy=0x7f12380ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380cda10 con 0x7f12380d13f0 2023-07-13T22:22:20.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f12300019f0 con 0x7f12380d13f0 2023-07-13T22:22:20.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 >> v1:172.21.15.60:6789/0 conn(0x7f12380d13f0 legacy=0x7f123815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1228003a30 0x7f1228003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12280043d0 0x7f1228004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12380d35c0 0x7f12280050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380d3b30 con 0x7f12280043d0 2023-07-13T22:22:20.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122800ba10 con 0x7f1228003a30 2023-07-13T22:22:20.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.380+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122800bba0 con 0x7f12380d35c0 2023-07-13T22:22:20.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.381+0000 7f1237fff700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12280043d0 0x7f1228004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.381+0000 7f123d79c700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12380d35c0 0x7f12280050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.381+0000 7f123cf9b700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1228003a30 0x7f1228003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.381+0000 7f1237fff700 1 -- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12380d35c0 msgr2=0x7f12280050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:20.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.381+0000 7f1237fff700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12380d35c0 0x7f12280050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.381+0000 7f1237fff700 1 -- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1228003a30 msgr2=0x7f1228003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:20.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.381+0000 7f1237fff700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1228003a30 0x7f1228003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.381+0000 7f1237fff700 1 -- 172.21.15.60:0/3921759100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12280035b0 con 0x7f12280043d0 2023-07-13T22:22:20.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.381+0000 7f1237fff700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12280043d0 0x7f1228004890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1224009b60 tx=0x7f1228005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de12c063ca028cf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:20.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400e6c0 con 0x7f12280043d0 2023-07-13T22:22:20.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122400e820 con 0x7f12280043d0 2023-07-13T22:22:20.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f12377fe700 1 -- 172.21.15.60:0/3921759100 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400eb90 con 0x7f12280043d0 2023-07-13T22:22:20.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f123f1ff700 1 -- 172.21.15.60:0/3921759100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12280043d0 msgr2=0x7f1228004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f123f1ff700 1 --2- 172.21.15.60:0/3921759100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12280043d0 0x7f1228004890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1224009b60 tx=0x7f1228005880 comp rx=0 tx=0).stop 2023-07-13T22:22:20.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f123d79c700 1 -- 172.21.15.60:0/3921759100 reap_dead start 2023-07-13T22:22:20.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f123f1ff700 1 -- 172.21.15.60:0/3921759100 shutdown_connections 2023-07-13T22:22:20.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f123f1ff700 1 -- 172.21.15.60:0/3921759100 >> 172.21.15.60:0/3921759100 conn(0x7f1238071600 msgr2=0x7f12380719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:20.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f123f1ff700 1 -- 172.21.15.60:0/3921759100 shutdown_connections 2023-07-13T22:22:20.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.382+0000 7f123f1ff700 1 -- 172.21.15.60:0/3921759100 wait complete. 2023-07-13T22:22:20.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.385+0000 7f123f1ff700 1 Processor -- start 2023-07-13T22:22:20.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.385+0000 7f123f1ff700 1 -- start start 2023-07-13T22:22:20.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.386+0000 7f123f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12280043d0 0x7f1238157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.386+0000 7f123f1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1238154bf0 0x7f12381f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.386+0000 7f123f1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12380cc7a0 0x7f1238155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.386+0000 7f123f1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12381f5420 con 0x7f12380cc7a0 2023-07-13T22:22:20.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.386+0000 7f123f1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12381f6370 con 0x7f1238154bf0 2023-07-13T22:22:20.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.386+0000 7f123f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1238157ac0 con 0x7f12280043d0 2023-07-13T22:22:20.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123cf9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12280043d0 0x7f1238157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123d79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12380cc7a0 0x7f1238155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123cf9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12280043d0 0x7f1238157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:46428/0 (socket says 172.21.15.60:46428) 2023-07-13T22:22:20.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123cf9b700 1 -- 172.21.15.60:0/2801139547 learned_addr learned my addr 172.21.15.60:0/2801139547 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:20.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f1237fff700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1238154bf0 0x7f12381f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123d79c700 1 -- 172.21.15.60:0/2801139547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12280043d0 msgr2=0x7f1238157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123d79c700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12280043d0 0x7f1238157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123d79c700 1 -- 172.21.15.60:0/2801139547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1238154bf0 msgr2=0x7f12381f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123d79c700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1238154bf0 0x7f12381f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123d79c700 1 -- 172.21.15.60:0/2801139547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1238157da0 con 0x7f12380cc7a0 2023-07-13T22:22:20.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123cf9b700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12280043d0 0x7f1238157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:20.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.387+0000 7f123d79c700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12380cc7a0 0x7f1238155590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f12380d0780 tx=0x7f123000a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:20.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.388+0000 7f12357fa700 1 -- 172.21.15.60:0/2801139547 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123000ad00 con 0x7f12380cc7a0 2023-07-13T22:22:20.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.388+0000 7f12357fa700 1 -- 172.21.15.60:0/2801139547 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123000ae60 con 0x7f12380cc7a0 2023-07-13T22:22:20.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.388+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12381f5630 con 0x7f12380cc7a0 2023-07-13T22:22:20.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.388+0000 7f12357fa700 1 -- 172.21.15.60:0/2801139547 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230013630 con 0x7f12380cc7a0 2023-07-13T22:22:20.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.388+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123815efa0 con 0x7f12380cc7a0 2023-07-13T22:22:20.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.389+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12381f5b50 con 0x7f12380cc7a0 2023-07-13T22:22:20.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.390+0000 7f12357fa700 1 -- 172.21.15.60:0/2801139547 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1230013790 con 0x7f12380cc7a0 2023-07-13T22:22:20.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.392+0000 7f12357fa700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f121c06ace0 0x7f121c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.392+0000 7f12357fa700 1 -- 172.21.15.60:0/2801139547 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1230089f70 con 0x7f12380cc7a0 2023-07-13T22:22:20.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.392+0000 7f123cf9b700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f121c06ace0 0x7f121c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.393+0000 7f123cf9b700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f121c06ace0 0x7f121c06d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f122c000ef0 tx=0x7f122c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:20.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.396+0000 7f12357fa700 1 -- 172.21.15.60:0/2801139547 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f12381f5b50 con 0x7f12380cc7a0 2023-07-13T22:22:20.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.602+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12381f5b50 con 0x7f12380cc7a0 2023-07-13T22:22:20.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.603+0000 7f12357fa700 1 -- 172.21.15.60:0/2801139547 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f12381f5b50 con 0x7f12380cc7a0 2023-07-13T22:22:20.587 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:20.587 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":34,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:22:19.175328+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:22:17.071217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-13T22:22:18.161600+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:20.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f121c06ace0 msgr2=0x7f121c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f121c06ace0 0x7f121c06d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f122c000ef0 tx=0x7f122c009040 comp rx=0 tx=0).stop 2023-07-13T22:22:20.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12380cc7a0 msgr2=0x7f1238155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12380cc7a0 0x7f1238155590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f12380d0780 tx=0x7f123000a640 comp rx=0 tx=0).stop 2023-07-13T22:22:20.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 shutdown_connections 2023-07-13T22:22:20.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f121c06ace0 0x7f121c06d190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12380cc7a0 0x7f1238155590 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12280043d0 0x7f1238157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 --2- 172.21.15.60:0/2801139547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1238154bf0 0x7f12381f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 >> 172.21.15.60:0/2801139547 conn(0x7f1238071600 msgr2=0x7f12380b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:20.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 shutdown_connections 2023-07-13T22:22:20.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.606+0000 7f123f1ff700 1 -- 172.21.15.60:0/2801139547 wait complete. 2023-07-13T22:22:20.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:22:20.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.834+0000 7f4c0e7ff700 1 Processor -- start 2023-07-13T22:22:20.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.834+0000 7f4c0e7ff700 1 -- start start 2023-07-13T22:22:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.835+0000 7f4c0e7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c08158930 0x7f4c080c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.835+0000 7f4c0e7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c0815c4a0 0x7f4c0815c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.835+0000 7f4c0e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08069ed0 0x7f4c0806a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.835+0000 7f4c0e7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c080c9620 con 0x7f4c0815c4a0 2023-07-13T22:22:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.835+0000 7f4c0e7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0806a890 con 0x7f4c08158930 2023-07-13T22:22:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.835+0000 7f4c0e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0806d030 con 0x7f4c08069ed0 2023-07-13T22:22:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.835+0000 7f4c077fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c0815c4a0 0x7f4c0815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c077fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c0815c4a0 0x7f4c0815c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46762/0 (socket says 172.21.15.60:46762) 2023-07-13T22:22:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c077fe700 1 -- 172.21.15.60:0/895832822 learned_addr learned my addr 172.21.15.60:0/895832822 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c07fff700 1 --2- 172.21.15.60:0/895832822 >> v2:172.21.15.105:3300/0 conn(0x7f4c08158930 0x7f4c080c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c0cd9c700 1 --2- 172.21.15.60:0/895832822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08069ed0 0x7f4c0806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c077fe700 1 -- 172.21.15.60:0/895832822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08069ed0 msgr2=0x7f4c0806a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c077fe700 1 --2- 172.21.15.60:0/895832822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08069ed0 0x7f4c0806a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c077fe700 1 -- 172.21.15.60:0/895832822 >> v2:172.21.15.105:3300/0 conn(0x7f4c08158930 msgr2=0x7f4c080c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c077fe700 1 --2- 172.21.15.60:0/895832822 >> v2:172.21.15.105:3300/0 conn(0x7f4c08158930 0x7f4c080c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c077fe700 1 -- 172.21.15.60:0/895832822 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0806ca70 con 0x7f4c0815c4a0 2023-07-13T22:22:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c0cd9c700 1 --2- 172.21.15.60:0/895832822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08069ed0 0x7f4c0806a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.836+0000 7f4c077fe700 1 --2- 172.21.15.60:0/895832822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c0815c4a0 0x7f4c0815c910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4c0806f160 tx=0x7f4bfc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6bfb02ecab7580 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c06ffd700 1 -- 172.21.15.60:0/895832822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc00dc70 con 0x7f4c0815c4a0 2023-07-13T22:22:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c06ffd700 1 -- 172.21.15.60:0/895832822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc00ddd0 con 0x7f4c0815c4a0 2023-07-13T22:22:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c06ffd700 1 -- 172.21.15.60:0/895832822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc010690 con 0x7f4c0815c4a0 2023-07-13T22:22:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/895832822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c0815c4a0 msgr2=0x7f4c0815c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/895832822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c0815c4a0 0x7f4c0815c910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4c0806f160 tx=0x7f4bfc00be70 comp rx=0 tx=0).stop 2023-07-13T22:22:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/895832822 shutdown_connections 2023-07-13T22:22:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/895832822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08069ed0 0x7f4c0806a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/895832822 >> v2:172.21.15.105:3300/0 conn(0x7f4c08158930 0x7f4c080c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/895832822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c0815c4a0 0x7f4c0815c910 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/895832822 >> 172.21.15.60:0/895832822 conn(0x7f4c080cbf50 msgr2=0x7f4c080ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/895832822 shutdown_connections 2023-07-13T22:22:20.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.837+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/895832822 wait complete. 2023-07-13T22:22:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.839+0000 7f4c0e7ff700 1 Processor -- start 2023-07-13T22:22:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.839+0000 7f4c0e7ff700 1 -- start start 2023-07-13T22:22:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.840+0000 7f4c0e7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08069ed0 0x7f4c08149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.840+0000 7f4c0e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08158930 0x7f4c0815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.840+0000 7f4c0e7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c0815c4a0 0x7f4c08158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.840+0000 7f4c0e7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0815c1a0 con 0x7f4c08069ed0 2023-07-13T22:22:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.840+0000 7f4c0e7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0815b900 con 0x7f4c0815c4a0 2023-07-13T22:22:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.840+0000 7f4c0e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0815bbe0 con 0x7f4c08158930 2023-07-13T22:22:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.840+0000 7f4c07fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08069ed0 0x7f4c08149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.840+0000 7f4c07fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08069ed0 0x7f4c08149c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46776/0 (socket says 172.21.15.60:46776) 2023-07-13T22:22:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.840+0000 7f4c07fff700 1 -- 172.21.15.60:0/691853175 learned_addr learned my addr 172.21.15.60:0/691853175 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c07fff700 1 -- 172.21.15.60:0/691853175 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08158930 msgr2=0x7f4c0815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c07fff700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08158930 0x7f4c0815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c0cd9c700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c0815c4a0 0x7f4c08158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c07fff700 1 -- 172.21.15.60:0/691853175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c0815c4a0 msgr2=0x7f4c08158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c07fff700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c0815c4a0 0x7f4c08158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c07fff700 1 -- 172.21.15.60:0/691853175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc00b7f0 con 0x7f4c08069ed0 2023-07-13T22:22:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c0cd9c700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c0815c4a0 0x7f4c08158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c07fff700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08069ed0 0x7f4c08149c20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e870 tx=0x7f4bf800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c04ff9700 1 -- 172.21.15.60:0/691853175 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8009590 con 0x7f4c08069ed0 2023-07-13T22:22:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c04ff9700 1 -- 172.21.15.60:0/691853175 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf800ee70 con 0x7f4c08069ed0 2023-07-13T22:22:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0815bec0 con 0x7f4c08069ed0 2023-07-13T22:22:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c04ff9700 1 -- 172.21.15.60:0/691853175 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800ca50 con 0x7f4c08069ed0 2023-07-13T22:22:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.841+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0814a160 con 0x7f4c08069ed0 2023-07-13T22:22:20.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.842+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0804e210 con 0x7f4c08069ed0 2023-07-13T22:22:20.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.844+0000 7f4c04ff9700 1 -- 172.21.15.60:0/691853175 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4bf800cbb0 con 0x7f4c08069ed0 2023-07-13T22:22:20.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.844+0000 7f4c04ff9700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4bf406ace0 0x7f4bf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:20.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.845+0000 7f4c04ff9700 1 -- 172.21.15.60:0/691853175 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4bf80893f0 con 0x7f4c08069ed0 2023-07-13T22:22:20.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.845+0000 7f4c077fe700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4bf406ace0 0x7f4bf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:20.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.845+0000 7f4c077fe700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4bf406ace0 0x7f4bf406d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c080ca9e0 tx=0x7f4bfc012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:20.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:20.849+0000 7f4c04ff9700 1 -- 172.21.15.60:0/691853175 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4bf8057f20 con 0x7f4c08069ed0 2023-07-13T22:22:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.102+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4c080b47c0 con 0x7f4c08069ed0 2023-07-13T22:22:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.693+0000 7f4c04ff9700 1 -- 172.21.15.60:0/691853175 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4bf800ce60 con 0x7f4c08069ed0 2023-07-13T22:22:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4bf406ace0 msgr2=0x7f4bf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4bf406ace0 0x7f4bf406d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c080ca9e0 tx=0x7f4bfc012000 comp rx=0 tx=0).stop 2023-07-13T22:22:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08069ed0 msgr2=0x7f4c08149c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08069ed0 0x7f4c08149c20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e870 tx=0x7f4bf800ec30 comp rx=0 tx=0).stop 2023-07-13T22:22:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 shutdown_connections 2023-07-13T22:22:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4bf406ace0 0x7f4bf406d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c08069ed0 0x7f4c08149c20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c0815c4a0 0x7f4c08158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 --2- 172.21.15.60:0/691853175 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c08158930 0x7f4c0815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 >> 172.21.15.60:0/691853175 conn(0x7f4c080cbf50 msgr2=0x7f4c080b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 shutdown_connections 2023-07-13T22:22:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.697+0000 7f4c0e7ff700 1 -- 172.21.15.60:0/691853175 wait complete. 2023-07-13T22:22:21.701 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:22:21.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:22:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.927+0000 7fedc97ff700 1 Processor -- start 2023-07-13T22:22:21.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.928+0000 7fedc97ff700 1 -- start start 2023-07-13T22:22:21.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.929+0000 7fedc97ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fedc40b8200 0x7fedc40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:21.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.929+0000 7fedc97ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40b1ac0 0x7fedc40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.929+0000 7fedc97ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc40d1870 con 0x7fedc40cede0 2023-07-13T22:22:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.929+0000 7fedc97ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc40cf370 con 0x7fedc40b8200 2023-07-13T22:22:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.929+0000 7fedc97ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40d12b0 con 0x7fedc40b1ac0 2023-07-13T22:22:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.929+0000 7fedc2ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fedc40b8200 0x7fedc40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.929+0000 7fedc2ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fedc40b8200 0x7fedc40cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46790/0 (socket says 172.21.15.60:46790) 2023-07-13T22:22:21.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.929+0000 7fedc2ffd700 1 -- 172.21.15.60:0/169591243 learned_addr learned my addr 172.21.15.60:0/169591243 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:21.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.929+0000 7fedc27fc700 1 --2- 172.21.15.60:0/169591243 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40b1ac0 0x7fedc40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.930+0000 7fedc2ffd700 1 -- 172.21.15.60:0/169591243 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40b1ac0 msgr2=0x7fedc40cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.930+0000 7fedc2ffd700 1 --2- 172.21.15.60:0/169591243 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40b1ac0 0x7fedc40cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.930+0000 7fedc2ffd700 1 -- 172.21.15.60:0/169591243 >> v1:172.21.15.60:6789/0 conn(0x7fedc40cede0 legacy=0x7fedc415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.930+0000 7fedc2ffd700 1 -- 172.21.15.60:0/169591243 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc40d1590 con 0x7fedc40b8200 2023-07-13T22:22:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.930+0000 7fedc2ffd700 1 --2- 172.21.15.60:0/169591243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b8200 0x7fedc40cf640 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fedb40098f0 tx=0x7fedb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a360250f6ccdd6c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.930+0000 7fedc1ffb700 1 -- 172.21.15.60:0/169591243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb400b8a0 con 0x7fedc40b8200 2023-07-13T22:22:21.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.930+0000 7fedc1ffb700 1 -- 172.21.15.60:0/169591243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb400ba00 con 0x7fedc40b8200 2023-07-13T22:22:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.930+0000 7fedc1ffb700 1 -- 172.21.15.60:0/169591243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb400bd90 con 0x7fedc40b8200 2023-07-13T22:22:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.931+0000 7fedc97ff700 1 -- 172.21.15.60:0/169591243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b8200 msgr2=0x7fedc40cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.931+0000 7fedc97ff700 1 --2- 172.21.15.60:0/169591243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b8200 0x7fedc40cf640 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fedb40098f0 tx=0x7fedb4009c00 comp rx=0 tx=0).stop 2023-07-13T22:22:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.931+0000 7fedc97ff700 1 -- 172.21.15.60:0/169591243 shutdown_connections 2023-07-13T22:22:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.931+0000 7fedc97ff700 1 --2- 172.21.15.60:0/169591243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b8200 0x7fedc40cf640 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.931+0000 7fedc97ff700 1 --2- 172.21.15.60:0/169591243 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40b1ac0 0x7fedc40cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.931+0000 7fedc97ff700 1 -- 172.21.15.60:0/169591243 >> 172.21.15.60:0/169591243 conn(0x7fedc406b2f0 msgr2=0x7fedc406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:21.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.931+0000 7fedc97ff700 1 -- 172.21.15.60:0/169591243 shutdown_connections 2023-07-13T22:22:21.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.931+0000 7fedc97ff700 1 -- 172.21.15.60:0/169591243 wait complete. 2023-07-13T22:22:21.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.933+0000 7fedc97ff700 1 Processor -- start 2023-07-13T22:22:21.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.933+0000 7fedc97ff700 1 -- start start 2023-07-13T22:22:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc97ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b1ac0 0x7fedc414ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc97ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc40b8200 0x7fedc41e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc97ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40cede0 0x7fedc414c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc97ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc414c420 con 0x7fedc40b1ac0 2023-07-13T22:22:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc97ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc414b810 con 0x7fedc40b8200 2023-07-13T22:22:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc97ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc414baf0 con 0x7fedc40cede0 2023-07-13T22:22:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b1ac0 0x7fedc414ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:21.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b1ac0 0x7fedc414ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46800/0 (socket says 172.21.15.60:46800) 2023-07-13T22:22:21.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc2ffd700 1 -- 172.21.15.60:0/2261012209 learned_addr learned my addr 172.21.15.60:0/2261012209 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:21.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc27fc700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc40b8200 0x7fedc41e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:21.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc37fe700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40cede0 0x7fedc414c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:21.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc2ffd700 1 -- 172.21.15.60:0/2261012209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40cede0 msgr2=0x7fedc414c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:21.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc2ffd700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40cede0 0x7fedc414c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:21.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc2ffd700 1 -- 172.21.15.60:0/2261012209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc40b8200 msgr2=0x7fedc41e28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:21.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc2ffd700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc40b8200 0x7fedc41e28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:21.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.934+0000 7fedc2ffd700 1 -- 172.21.15.60:0/2261012209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb4009550 con 0x7fedc40b1ac0 2023-07-13T22:22:21.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.935+0000 7fedc2ffd700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b1ac0 0x7fedc414ae50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fedb4000c00 tx=0x7fedb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:21.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.935+0000 7fedab7fe700 1 -- 172.21.15.60:0/2261012209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4004480 con 0x7fedc40b1ac0 2023-07-13T22:22:21.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.935+0000 7fedab7fe700 1 -- 172.21.15.60:0/2261012209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb401f070 con 0x7fedc40b1ac0 2023-07-13T22:22:21.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.935+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc414bdd0 con 0x7fedc40b1ac0 2023-07-13T22:22:21.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.935+0000 7fedab7fe700 1 -- 172.21.15.60:0/2261012209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb40038c0 con 0x7fedc40b1ac0 2023-07-13T22:22:21.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.935+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc415b790 con 0x7fedc40b1ac0 2023-07-13T22:22:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.936+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc414c700 con 0x7fedc40b1ac0 2023-07-13T22:22:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.937+0000 7fedab7fe700 1 -- 172.21.15.60:0/2261012209 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fedb4021030 con 0x7fedc40b1ac0 2023-07-13T22:22:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.938+0000 7fedab7fe700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb006ace0 0x7fedb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:21.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.938+0000 7fedab7fe700 1 -- 172.21.15.60:0/2261012209 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fedb408aa10 con 0x7fedc40b1ac0 2023-07-13T22:22:21.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.938+0000 7fedc27fc700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb006ace0 0x7fedb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:21.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.939+0000 7fedc27fc700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb006ace0 0x7fedb006d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fedc4147930 tx=0x7fedb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:21.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:21.942+0000 7fedab7fe700 1 -- 172.21.15.60:0/2261012209 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fedb4055bd0 con 0x7fedc40b1ac0 2023-07-13T22:22:22.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.195+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fedc40ccd20 con 0x7fedc40b1ac0 2023-07-13T22:22:22.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.694+0000 7fedab7fe700 1 -- 172.21.15.60:0/2261012209 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fedb40591f0 con 0x7fedc40b1ac0 2023-07-13T22:22:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb006ace0 msgr2=0x7fedb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb006ace0 0x7fedb006d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fedc4147930 tx=0x7fedb8008040 comp rx=0 tx=0).stop 2023-07-13T22:22:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b1ac0 msgr2=0x7fedc414ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b1ac0 0x7fedc414ae50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fedb4000c00 tx=0x7fedb4004750 comp rx=0 tx=0).stop 2023-07-13T22:22:22.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 shutdown_connections 2023-07-13T22:22:22.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb006ace0 0x7fedb006d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:22.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc40b1ac0 0x7fedc414ae50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:22.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc40cede0 0x7fedc414c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:22.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 --2- 172.21.15.60:0/2261012209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc40b8200 0x7fedc41e28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:22.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 >> 172.21.15.60:0/2261012209 conn(0x7fedc406b2f0 msgr2=0x7fedc40b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:22.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 shutdown_connections 2023-07-13T22:22:22.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.698+0000 7fedc97ff700 1 -- 172.21.15.60:0/2261012209 wait complete. 2023-07-13T22:22:22.702 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:22:22.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:22.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.925+0000 7fa0d0fff700 1 Processor -- start 2023-07-13T22:22:22.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.925+0000 7fa0d0fff700 1 -- start start 2023-07-13T22:22:22.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.926+0000 7fa0d0fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa0cc070c50 0x7fa0cc0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.926+0000 7fa0d0fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0cc15c700 0x7fa0cc15ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.926+0000 7fa0d0fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc06b290 con 0x7fa0cc06e990 2023-07-13T22:22:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.926+0000 7fa0d0fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc06ee30 con 0x7fa0cc15c700 2023-07-13T22:22:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.926+0000 7fa0d0fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc0cd330 con 0x7fa0cc070c50 2023-07-13T22:22:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.927+0000 7fa0cad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0cc15c700 0x7fa0cc15ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:22.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.927+0000 7fa0cad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0cc15c700 0x7fa0cc15ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46808/0 (socket says 172.21.15.60:46808) 2023-07-13T22:22:22.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.927+0000 7fa0cad9d700 1 -- 172.21.15.60:0/1581930623 learned_addr learned my addr 172.21.15.60:0/1581930623 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:22.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.927+0000 7fa0ca59c700 1 --2- 172.21.15.60:0/1581930623 >> v2:172.21.15.105:3300/0 conn(0x7fa0cc070c50 0x7fa0cc0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:22.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.927+0000 7fa0cad9d700 1 -- 172.21.15.60:0/1581930623 >> v2:172.21.15.105:3300/0 conn(0x7fa0cc070c50 msgr2=0x7fa0cc0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:22.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.927+0000 7fa0cad9d700 1 --2- 172.21.15.60:0/1581930623 >> v2:172.21.15.105:3300/0 conn(0x7fa0cc070c50 0x7fa0cc0ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:22.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.927+0000 7fa0cad9d700 1 -- 172.21.15.60:0/1581930623 >> v1:172.21.15.105:6789/0 conn(0x7fa0cc06e990 legacy=0x7fa0cc071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:22.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.927+0000 7fa0cad9d700 1 -- 172.21.15.60:0/1581930623 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0ccd70 con 0x7fa0cc15c700 2023-07-13T22:22:22.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0cad9d700 1 --2- 172.21.15.60:0/1581930623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 0x7fa0cc15ffc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc06c960 tx=0x7fa0c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae44945877beeb07 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:22.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0c9d9b700 1 -- 172.21.15.60:0/1581930623 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400e070 con 0x7fa0cc15c700 2023-07-13T22:22:22.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0c9d9b700 1 -- 172.21.15.60:0/1581930623 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c400b910 con 0x7fa0cc15c700 2023-07-13T22:22:22.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0c9d9b700 1 -- 172.21.15.60:0/1581930623 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400bcc0 con 0x7fa0cc15c700 2023-07-13T22:22:22.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0d0fff700 1 -- 172.21.15.60:0/1581930623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 msgr2=0x7fa0cc15ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:22.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0d0fff700 1 --2- 172.21.15.60:0/1581930623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 0x7fa0cc15ffc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc06c960 tx=0x7fa0c400b330 comp rx=0 tx=0).stop 2023-07-13T22:22:22.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0d0fff700 1 -- 172.21.15.60:0/1581930623 shutdown_connections 2023-07-13T22:22:22.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0d0fff700 1 --2- 172.21.15.60:0/1581930623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 0x7fa0cc15ffc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:22.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0d0fff700 1 --2- 172.21.15.60:0/1581930623 >> v2:172.21.15.105:3300/0 conn(0x7fa0cc070c50 0x7fa0cc0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:22.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.928+0000 7fa0d0fff700 1 -- 172.21.15.60:0/1581930623 >> 172.21.15.60:0/1581930623 conn(0x7fa0cc0b1820 msgr2=0x7fa0cc0cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:22.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.929+0000 7fa0d0fff700 1 -- 172.21.15.60:0/1581930623 shutdown_connections 2023-07-13T22:22:22.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.929+0000 7fa0d0fff700 1 -- 172.21.15.60:0/1581930623 wait complete. 2023-07-13T22:22:22.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.930+0000 7fa0d0fff700 1 Processor -- start 2023-07-13T22:22:22.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.931+0000 7fa0d0fff700 1 -- start start 2023-07-13T22:22:22.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.931+0000 7fa0d0fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0cc06e990 0x7fa0cc1f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:22.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.931+0000 7fa0d0fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0cc070c50 0x7fa0cc0c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:22.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0d0fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 0x7fa0cc0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:22.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0d0fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc148200 con 0x7fa0cc15c700 2023-07-13T22:22:22.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0d0fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc148470 con 0x7fa0cc06e990 2023-07-13T22:22:22.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0d0fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0af900 con 0x7fa0cc070c50 2023-07-13T22:22:22.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0cad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 0x7fa0cc0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:22.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0cad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 0x7fa0cc0c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46810/0 (socket says 172.21.15.60:46810) 2023-07-13T22:22:22.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0cad9d700 1 -- 172.21.15.60:0/604853254 learned_addr learned my addr 172.21.15.60:0/604853254 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:22.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0ca59c700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0cc06e990 0x7fa0cc1f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:22.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0c3fff700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0cc070c50 0x7fa0cc0c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:22.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0cad9d700 1 -- 172.21.15.60:0/604853254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0cc070c50 msgr2=0x7fa0cc0c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:22.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0cad9d700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0cc070c50 0x7fa0cc0c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:22.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0cad9d700 1 -- 172.21.15.60:0/604853254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0cc06e990 msgr2=0x7fa0cc1f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:22.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0cad9d700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0cc06e990 0x7fa0cc1f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:22.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.932+0000 7fa0cad9d700 1 -- 172.21.15.60:0/604853254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4009b30 con 0x7fa0cc15c700 2023-07-13T22:22:22.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.933+0000 7fa0cad9d700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 0x7fa0cc0c9380 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc065d60 tx=0x7fa0c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:22.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.933+0000 7fa0c2ffd700 1 -- 172.21.15.60:0/604853254 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4004880 con 0x7fa0cc15c700 2023-07-13T22:22:22.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.933+0000 7fa0c2ffd700 1 -- 172.21.15.60:0/604853254 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c40099b0 con 0x7fa0cc15c700 2023-07-13T22:22:22.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.933+0000 7fa0c2ffd700 1 -- 172.21.15.60:0/604853254 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c401b670 con 0x7fa0cc15c700 2023-07-13T22:22:22.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.933+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc148730 con 0x7fa0cc15c700 2023-07-13T22:22:22.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.933+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc148cf0 con 0x7fa0cc15c700 2023-07-13T22:22:22.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.934+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc04e280 con 0x7fa0cc15c700 2023-07-13T22:22:22.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.935+0000 7fa0c2ffd700 1 -- 172.21.15.60:0/604853254 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa0c401b7d0 con 0x7fa0cc15c700 2023-07-13T22:22:22.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.936+0000 7fa0c2ffd700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ac06ace0 0x7fa0ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:22.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.936+0000 7fa0c2ffd700 1 -- 172.21.15.60:0/604853254 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa0c408a9c0 con 0x7fa0cc15c700 2023-07-13T22:22:22.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.936+0000 7fa0ca59c700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ac06ace0 0x7fa0ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:22.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.937+0000 7fa0ca59c700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ac06ace0 0x7fa0ac06d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009be0 tx=0x7fa0bc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:22.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:22.940+0000 7fa0c2ffd700 1 -- 172.21.15.60:0/604853254 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa0c4055a60 con 0x7fa0cc15c700 2023-07-13T22:22:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.192+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0cc0b0940 con 0x7fa0cc15c700 2023-07-13T22:22:23.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.193+0000 7fa0c2ffd700 1 -- 172.21.15.60:0/604853254 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fa0c4059080 con 0x7fa0cc15c700 2023-07-13T22:22:23.176 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:23.177 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2157014443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2157014443},{"type":"v1","addr":"172.21.15.105:6834","nonce":2157014443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/302067138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":302067138},{"type":"v1","addr":"172.21.15.105:6835","nonce":302067138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3461450784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461450784},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461450784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:22:19.175830+0000","modified":"2023-07-13T22:22:22.691356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6837/2048874349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2048874349},{"type":"v1","addr":"172.21.15.60:6837","nonce":2048874349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-13T22:22:23.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ac06ace0 msgr2=0x7fa0ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ac06ace0 0x7fa0ac06d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009be0 tx=0x7fa0bc009210 comp rx=0 tx=0).stop 2023-07-13T22:22:23.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 msgr2=0x7fa0cc0c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 0x7fa0cc0c9380 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc065d60 tx=0x7fa0c4004ad0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 shutdown_connections 2023-07-13T22:22:23.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa0ac06ace0 0x7fa0ac06d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0cc15c700 0x7fa0cc0c9380 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0cc06e990 0x7fa0cc1f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 --2- 172.21.15.60:0/604853254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0cc070c50 0x7fa0cc0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 >> 172.21.15.60:0/604853254 conn(0x7fa0cc0b1820 msgr2=0x7fa0cc0b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:23.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 shutdown_connections 2023-07-13T22:22:23.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.196+0000 7fa0d0fff700 1 -- 172.21.15.60:0/604853254 wait complete. 2023-07-13T22:22:23.183 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2023-07-13T22:22:23.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:23.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.417+0000 7fbf80bff700 1 Processor -- start 2023-07-13T22:22:23.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.417+0000 7fbf80bff700 1 -- start start 2023-07-13T22:22:23.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.418+0000 7fbf80bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf7c0b8400 0x7fbf7c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.418+0000 7fbf80bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf7c0d3750 0x7fbf7c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.418+0000 7fbf80bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 0x7fbf7c15fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.418+0000 7fbf80bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c0cff50 con 0x7fbf7c0b8400 2023-07-13T22:22:23.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.418+0000 7fbf80bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c0d0ac0 con 0x7fbf7c0d3750 2023-07-13T22:22:23.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.418+0000 7fbf80bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0b87d0 con 0x7fbf7c0cfad0 2023-07-13T22:22:23.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.418+0000 7fbf7a59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf7c0b8400 0x7fbf7c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.418+0000 7fbf7a59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf7c0b8400 0x7fbf7c0cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46822/0 (socket says 172.21.15.60:46822) 2023-07-13T22:22:23.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.418+0000 7fbf7a59c700 1 -- 172.21.15.60:0/257580112 learned_addr learned my addr 172.21.15.60:0/257580112 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:23.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf73fff700 1 --2- 172.21.15.60:0/257580112 >> v2:172.21.15.105:3300/0 conn(0x7fbf7c0d3750 0x7fbf7c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf7ad9d700 1 --2- 172.21.15.60:0/257580112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 0x7fbf7c15fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf7a59c700 1 -- 172.21.15.60:0/257580112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 msgr2=0x7fbf7c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf7a59c700 1 --2- 172.21.15.60:0/257580112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 0x7fbf7c15fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf7a59c700 1 -- 172.21.15.60:0/257580112 >> v2:172.21.15.105:3300/0 conn(0x7fbf7c0d3750 msgr2=0x7fbf7c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf7a59c700 1 --2- 172.21.15.60:0/257580112 >> v2:172.21.15.105:3300/0 conn(0x7fbf7c0d3750 0x7fbf7c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf7a59c700 1 -- 172.21.15.60:0/257580112 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0b4eb0 con 0x7fbf7c0b8400 2023-07-13T22:22:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf73fff700 1 --2- 172.21.15.60:0/257580112 >> v2:172.21.15.105:3300/0 conn(0x7fbf7c0d3750 0x7fbf7c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf7a59c700 1 --2- 172.21.15.60:0/257580112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0b8400 0x7fbf7c0cd240 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0b8cf0 tx=0x7fbf6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601b5bc1c7de3542 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.419+0000 7fbf79d9b700 1 -- 172.21.15.60:0/257580112 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c011830 con 0x7fbf7c0b8400 2023-07-13T22:22:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf79d9b700 1 -- 172.21.15.60:0/257580112 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6c011990 con 0x7fbf7c0b8400 2023-07-13T22:22:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf79d9b700 1 -- 172.21.15.60:0/257580112 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c011d20 con 0x7fbf7c0b8400 2023-07-13T22:22:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf80bff700 1 -- 172.21.15.60:0/257580112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0b8400 msgr2=0x7fbf7c0cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf80bff700 1 --2- 172.21.15.60:0/257580112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0b8400 0x7fbf7c0cd240 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0b8cf0 tx=0x7fbf6c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf80bff700 1 -- 172.21.15.60:0/257580112 shutdown_connections 2023-07-13T22:22:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf80bff700 1 --2- 172.21.15.60:0/257580112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 0x7fbf7c15fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf80bff700 1 --2- 172.21.15.60:0/257580112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0b8400 0x7fbf7c0cd240 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf80bff700 1 --2- 172.21.15.60:0/257580112 >> v2:172.21.15.105:3300/0 conn(0x7fbf7c0d3750 0x7fbf7c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf80bff700 1 -- 172.21.15.60:0/257580112 >> 172.21.15.60:0/257580112 conn(0x7fbf7c06b390 msgr2=0x7fbf7c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:23.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf80bff700 1 -- 172.21.15.60:0/257580112 shutdown_connections 2023-07-13T22:22:23.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.420+0000 7fbf80bff700 1 -- 172.21.15.60:0/257580112 wait complete. 2023-07-13T22:22:23.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.422+0000 7fbf80bff700 1 Processor -- start 2023-07-13T22:22:23.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.422+0000 7fbf80bff700 1 -- start start 2023-07-13T22:22:23.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.423+0000 7fbf80bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf7c0b8400 0x7fbf7c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.423+0000 7fbf80bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 0x7fbf7c15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.423+0000 7fbf80bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0d3750 0x7fbf7c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.423+0000 7fbf80bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c15ea00 con 0x7fbf7c0d3750 2023-07-13T22:22:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.423+0000 7fbf80bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c15ece0 con 0x7fbf7c0b8400 2023-07-13T22:22:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.423+0000 7fbf80bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c15efc0 con 0x7fbf7c0cfad0 2023-07-13T22:22:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.423+0000 7fbf7ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0d3750 0x7fbf7c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.423+0000 7fbf7ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0d3750 0x7fbf7c15e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46830/0 (socket says 172.21.15.60:46830) 2023-07-13T22:22:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.423+0000 7fbf7ad9d700 1 -- 172.21.15.60:0/3786416166 learned_addr learned my addr 172.21.15.60:0/3786416166 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf7a59c700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf7c0b8400 0x7fbf7c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf73fff700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 0x7fbf7c15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf7ad9d700 1 -- 172.21.15.60:0/3786416166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 msgr2=0x7fbf7c15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf7ad9d700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 0x7fbf7c15de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf7ad9d700 1 -- 172.21.15.60:0/3786416166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf7c0b8400 msgr2=0x7fbf7c153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf7ad9d700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf7c0b8400 0x7fbf7c153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf7ad9d700 1 -- 172.21.15.60:0/3786416166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c009550 con 0x7fbf7c0d3750 2023-07-13T22:22:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf7a59c700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf7c0b8400 0x7fbf7c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf7ad9d700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0d3750 0x7fbf7c15e3d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400efc0 tx=0x7fbf7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf72ffd700 1 -- 172.21.15.60:0/3786416166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74009ba0 con 0x7fbf7c0d3750 2023-07-13T22:22:23.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf72ffd700 1 -- 172.21.15.60:0/3786416166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7400f040 con 0x7fbf7c0d3750 2023-07-13T22:22:23.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf72ffd700 1 -- 172.21.15.60:0/3786416166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74014480 con 0x7fbf7c0d3750 2023-07-13T22:22:23.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c15f2a0 con 0x7fbf7c0d3750 2023-07-13T22:22:23.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.424+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c1544e0 con 0x7fbf7c0d3750 2023-07-13T22:22:23.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.425+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c15f7c0 con 0x7fbf7c0d3750 2023-07-13T22:22:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.426+0000 7fbf72ffd700 1 -- 172.21.15.60:0/3786416166 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf740145e0 con 0x7fbf7c0d3750 2023-07-13T22:22:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.428+0000 7fbf72ffd700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5c06ace0 0x7fbf5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.428+0000 7fbf72ffd700 1 -- 172.21.15.60:0/3786416166 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf74089c40 con 0x7fbf7c0d3750 2023-07-13T22:22:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.428+0000 7fbf7a59c700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5c06ace0 0x7fbf5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.429+0000 7fbf7a59c700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5c06ace0 0x7fbf5c06d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c014040 tx=0x7fbf6c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:23.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.432+0000 7fbf72ffd700 1 -- 172.21.15.60:0/3786416166 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbf7c15f7c0 con 0x7fbf7c0d3750 2023-07-13T22:22:23.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.643+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf7c15f7c0 con 0x7fbf7c0d3750 2023-07-13T22:22:23.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.644+0000 7fbf72ffd700 1 -- 172.21.15.60:0/3786416166 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18257 (secure 0 0 0) 0x7fbf7c15f7c0 con 0x7fbf7c0d3750 2023-07-13T22:22:23.628 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:23.628 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:22:20.678957+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:22:17.071217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-13T22:22:18.161600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:23.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5c06ace0 msgr2=0x7fbf5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5c06ace0 0x7fbf5c06d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c014040 tx=0x7fbf6c008030 comp rx=0 tx=0).stop 2023-07-13T22:22:23.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0d3750 msgr2=0x7fbf7c15e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0d3750 0x7fbf7c15e3d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400efc0 tx=0x7fbf7400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 shutdown_connections 2023-07-13T22:22:23.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5c06ace0 0x7fbf5c06d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf7c0d3750 0x7fbf7c15e3d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf7c0b8400 0x7fbf7c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 --2- 172.21.15.60:0/3786416166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf7c0cfad0 0x7fbf7c15de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.646+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 >> 172.21.15.60:0/3786416166 conn(0x7fbf7c06b390 msgr2=0x7fbf7c15bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:23.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.647+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 shutdown_connections 2023-07-13T22:22:23.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.647+0000 7fbf80bff700 1 -- 172.21.15.60:0/3786416166 wait complete. 2023-07-13T22:22:23.648 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:22:23.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:22:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.868+0000 7f8e407ff700 1 Processor -- start 2023-07-13T22:22:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.868+0000 7f8e407ff700 1 -- start start 2023-07-13T22:22:23.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.869+0000 7f8e407ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e380b1810 0x7f8e38160a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.869+0000 7f8e407ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3815fe70 con 0x7f8e380cec40 2023-07-13T22:22:23.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.869+0000 7f8e407ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3806ce50 con 0x7f8e3815faa0 2023-07-13T22:22:23.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.869+0000 7f8e407ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e380b0a50 con 0x7f8e380b1810 2023-07-13T22:22:23.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.869+0000 7f8e3dd9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8e380cec40 0x7f8e38160320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34230/0 (socket says 172.21.15.60:34230) 2023-07-13T22:22:23.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.870+0000 7f8e3dd9a700 1 -- 172.21.15.60:0/4152821760 learned_addr learned my addr 172.21.15.60:0/4152821760 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:23.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.870+0000 7f8e3ed9c700 1 --2- 172.21.15.60:0/4152821760 >> v2:172.21.15.60:3300/0 conn(0x7f8e380b1810 0x7f8e38160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.870+0000 7f8e3d599700 1 -- 172.21.15.60:0/4152821760 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319125008 0 0) 0x7f8e28001110 con 0x7f8e380cec40 2023-07-13T22:22:23.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.870+0000 7f8e3d599700 1 -- 172.21.15.60:0/4152821760 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e240035b0 con 0x7f8e380cec40 2023-07-13T22:22:23.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.870+0000 7f8e3ed9c700 1 -- 172.21.15.60:0/4152821760 >> v1:172.21.15.105:6789/0 conn(0x7f8e3815faa0 legacy=0x7f8e380ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.870+0000 7f8e3ed9c700 1 -- 172.21.15.60:0/4152821760 >> v1:172.21.15.60:6789/0 conn(0x7f8e380cec40 legacy=0x7f8e38160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.870+0000 7f8e3ed9c700 1 -- 172.21.15.60:0/4152821760 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380b0d30 con 0x7f8e380b1810 2023-07-13T22:22:23.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.870+0000 7f8e3ed9c700 1 --2- 172.21.15.60:0/4152821760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380b1810 0x7f8e38160a30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009d90 tx=0x7f8e3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34d4f2dc3e0e6f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:23.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e3d599700 1 -- 172.21.15.60:0/4152821760 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3000e070 con 0x7f8e380b1810 2023-07-13T22:22:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e3d599700 1 -- 172.21.15.60:0/4152821760 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3000bb20 con 0x7f8e380b1810 2023-07-13T22:22:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e3d599700 1 -- 172.21.15.60:0/4152821760 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30012430 con 0x7f8e380b1810 2023-07-13T22:22:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e407ff700 1 -- 172.21.15.60:0/4152821760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380b1810 msgr2=0x7f8e38160a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e407ff700 1 --2- 172.21.15.60:0/4152821760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380b1810 0x7f8e38160a30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009d90 tx=0x7f8e3000b330 comp rx=0 tx=0).stop 2023-07-13T22:22:23.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e407ff700 1 -- 172.21.15.60:0/4152821760 shutdown_connections 2023-07-13T22:22:23.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e407ff700 1 --2- 172.21.15.60:0/4152821760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380b1810 0x7f8e38160a30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e407ff700 1 -- 172.21.15.60:0/4152821760 >> 172.21.15.60:0/4152821760 conn(0x7f8e3808a000 msgr2=0x7f8e38099d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:23.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e407ff700 1 -- 172.21.15.60:0/4152821760 shutdown_connections 2023-07-13T22:22:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.871+0000 7f8e407ff700 1 -- 172.21.15.60:0/4152821760 wait complete. 2023-07-13T22:22:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.874+0000 7f8e407ff700 1 Processor -- start 2023-07-13T22:22:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.874+0000 7f8e407ff700 1 -- start start 2023-07-13T22:22:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.875+0000 7f8e407ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e380b1810 0x7f8e38154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.875+0000 7f8e407ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380cec40 0x7f8e38159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.875+0000 7f8e407ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3815faa0 0x7f8e38157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.875+0000 7f8e407ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380b24f0 con 0x7f8e380cec40 2023-07-13T22:22:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.875+0000 7f8e407ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380b2790 con 0x7f8e3815faa0 2023-07-13T22:22:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.875+0000 7f8e407ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380b2a70 con 0x7f8e380b1810 2023-07-13T22:22:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.875+0000 7f8e3dd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380cec40 0x7f8e38159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.875+0000 7f8e3dd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380cec40 0x7f8e38159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46848/0 (socket says 172.21.15.60:46848) 2023-07-13T22:22:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.875+0000 7f8e3dd9a700 1 -- 172.21.15.60:0/2063567863 learned_addr learned my addr 172.21.15.60:0/2063567863 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e3e59b700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e380b1810 0x7f8e38154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e3ed9c700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3815faa0 0x7f8e38157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e3dd9a700 1 -- 172.21.15.60:0/2063567863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e380b1810 msgr2=0x7f8e38154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e3dd9a700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e380b1810 0x7f8e38154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e3dd9a700 1 -- 172.21.15.60:0/2063567863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3815faa0 msgr2=0x7f8e38157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e3dd9a700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3815faa0 0x7f8e38157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e3dd9a700 1 -- 172.21.15.60:0/2063567863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30009a70 con 0x7f8e380cec40 2023-07-13T22:22:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e3dd9a700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380cec40 0x7f8e38159bd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c520 tx=0x7f8e2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e2effd700 1 -- 172.21.15.60:0/2063567863 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28005d70 con 0x7f8e380cec40 2023-07-13T22:22:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e2effd700 1 -- 172.21.15.60:0/2063567863 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e280070d0 con 0x7f8e380cec40 2023-07-13T22:22:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e2effd700 1 -- 172.21.15.60:0/2063567863 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28007460 con 0x7f8e380cec40 2023-07-13T22:22:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.876+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e380b2d50 con 0x7f8e380cec40 2023-07-13T22:22:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.877+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e381550a0 con 0x7f8e380cec40 2023-07-13T22:22:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.878+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e380b3270 con 0x7f8e380cec40 2023-07-13T22:22:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.879+0000 7f8e2effd700 1 -- 172.21.15.60:0/2063567863 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e280075c0 con 0x7f8e380cec40 2023-07-13T22:22:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.879+0000 7f8e2effd700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e1c06ace0 0x7f8e1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.879+0000 7f8e2effd700 1 -- 172.21.15.60:0/2063567863 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8e28088970 con 0x7f8e380cec40 2023-07-13T22:22:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.880+0000 7f8e3e59b700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e1c06ace0 0x7f8e1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.880+0000 7f8e3e59b700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e1c06ace0 0x7f8e1c06d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e34007fd0 tx=0x7f8e34009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:23.884+0000 7f8e2effd700 1 -- 172.21.15.60:0/2063567863 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8e380b3270 con 0x7f8e380cec40 2023-07-13T22:22:24.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.088+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e380b2900 con 0x7f8e1c06ace0 2023-07-13T22:22:24.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.099+0000 7f8e2effd700 1 -- 172.21.15.60:0/2063567863 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311537 (secure 0 0 0) 0x7f8e380b2900 con 0x7f8e1c06ace0 2023-07-13T22:22:24.086 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:24.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e1c06ace0 msgr2=0x7f8e1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e1c06ace0 0x7f8e1c06d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e34007fd0 tx=0x7f8e34009040 comp rx=0 tx=0).stop 2023-07-13T22:22:24.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380cec40 msgr2=0x7f8e38159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380cec40 0x7f8e38159bd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c520 tx=0x7f8e2800c8e0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 shutdown_connections 2023-07-13T22:22:24.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e1c06ace0 0x7f8e1c06d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3815faa0 0x7f8e38157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e380b1810 0x7f8e38154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 --2- 172.21.15.60:0/2063567863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e380cec40 0x7f8e38159bd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 >> 172.21.15.60:0/2063567863 conn(0x7f8e3808a000 msgr2=0x7f8e380ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:24.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 shutdown_connections 2023-07-13T22:22:24.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.104+0000 7f8e407ff700 1 -- 172.21.15.60:0/2063567863 wait complete. 2023-07-13T22:22:24.090 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:22:24.105 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-07-13T22:22:23.141685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219876,"kb_used_data":4260,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754844,"statfs":{"total":773094113280,"available":772868960256,"internally_reserved":0,"allocated":4362240,"data_stored":2472215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.483828"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047399+0000","last_change":"2023-07-13T22:22:19.692656+0000","last_active":"2023-07-13T22:22:21.047399+0000","last_peered":"2023-07-13T22:22:21.047399+0000","last_clean":"2023-07-13T22:22:21.047399+0000","last_became_active":"2023-07-13T22:22:19.692354+0000","last_became_peered":"2023-07-13T22:22:19.692354+0000","last_unstale":"2023-07-13T22:22:21.047399+0000","last_undegraded":"2023-07-13T22:22:21.047399+0000","last_fullsized":"2023-07-13T22:22:21.047399+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:59:30.476619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047077+0000","last_change":"2023-07-13T22:22:19.703610+0000","last_active":"2023-07-13T22:22:21.047077+0000","last_peered":"2023-07-13T22:22:21.047077+0000","last_clean":"2023-07-13T22:22:21.047077+0000","last_became_active":"2023-07-13T22:22:19.703345+0000","last_became_peered":"2023-07-13T22:22:19.703345+0000","last_unstale":"2023-07-13T22:22:21.047077+0000","last_undegraded":"2023-07-13T22:22:21.047077+0000","last_fullsized":"2023-07-13T22:22:21.047077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:19:05.780353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794280+0000","last_change":"2023-07-13T22:22:19.705602+0000","last_active":"2023-07-13T22:22:20.794280+0000","last_peered":"2023-07-13T22:22:20.794280+0000","last_clean":"2023-07-13T22:22:20.794280+0000","last_became_active":"2023-07-13T22:22:19.705203+0000","last_became_peered":"2023-07-13T22:22:19.705203+0000","last_unstale":"2023-07-13T22:22:20.794280+0000","last_undegraded":"2023-07-13T22:22:20.794280+0000","last_fullsized":"2023-07-13T22:22:20.794280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:54:11.424287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894637+0000","last_change":"2023-07-13T22:22:19.704528+0000","last_active":"2023-07-13T22:22:20.894637+0000","last_peered":"2023-07-13T22:22:20.894637+0000","last_clean":"2023-07-13T22:22:20.894637+0000","last_became_active":"2023-07-13T22:22:19.704008+0000","last_became_peered":"2023-07-13T22:22:19.704008+0000","last_unstale":"2023-07-13T22:22:20.894637+0000","last_undegraded":"2023-07-13T22:22:20.894637+0000","last_fullsized":"2023-07-13T22:22:20.894637+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:37:33.348454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894342+0000","last_change":"2023-07-13T22:22:19.698937+0000","last_active":"2023-07-13T22:22:20.894342+0000","last_peered":"2023-07-13T22:22:20.894342+0000","last_clean":"2023-07-13T22:22:20.894342+0000","last_became_active":"2023-07-13T22:22:19.698617+0000","last_became_peered":"2023-07-13T22:22:19.698617+0000","last_unstale":"2023-07-13T22:22:20.894342+0000","last_undegraded":"2023-07-13T22:22:20.894342+0000","last_fullsized":"2023-07-13T22:22:20.894342+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:22:21.825394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687200+0000","last_change":"2023-07-13T22:22:19.694903+0000","last_active":"2023-07-13T22:22:20.687200+0000","last_peered":"2023-07-13T22:22:20.687200+0000","last_clean":"2023-07-13T22:22:20.687200+0000","last_became_active":"2023-07-13T22:22:19.694525+0000","last_became_peered":"2023-07-13T22:22:19.694525+0000","last_unstale":"2023-07-13T22:22:20.687200+0000","last_undegraded":"2023-07-13T22:22:20.687200+0000","last_fullsized":"2023-07-13T22:22:20.687200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:51.327980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688617+0000","last_change":"2023-07-13T22:22:19.693765+0000","last_active":"2023-07-13T22:22:20.688617+0000","last_peered":"2023-07-13T22:22:20.688617+0000","last_clean":"2023-07-13T22:22:20.688617+0000","last_became_active":"2023-07-13T22:22:19.693521+0000","last_became_peered":"2023-07-13T22:22:19.693521+0000","last_unstale":"2023-07-13T22:22:20.688617+0000","last_undegraded":"2023-07-13T22:22:20.688617+0000","last_fullsized":"2023-07-13T22:22:20.688617+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:37:40.537525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046800+0000","last_change":"2023-07-13T22:22:19.696608+0000","last_active":"2023-07-13T22:22:21.046800+0000","last_peered":"2023-07-13T22:22:21.046800+0000","last_clean":"2023-07-13T22:22:21.046800+0000","last_became_active":"2023-07-13T22:22:19.696316+0000","last_became_peered":"2023-07-13T22:22:19.696316+0000","last_unstale":"2023-07-13T22:22:21.046800+0000","last_undegraded":"2023-07-13T22:22:21.046800+0000","last_fullsized":"2023-07-13T22:22:21.046800+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:13:47.499901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794491+0000","last_change":"2023-07-13T22:22:19.704019+0000","last_active":"2023-07-13T22:22:20.794491+0000","last_peered":"2023-07-13T22:22:20.794491+0000","last_clean":"2023-07-13T22:22:20.794491+0000","last_became_active":"2023-07-13T22:22:19.703749+0000","last_became_peered":"2023-07-13T22:22:19.703749+0000","last_unstale":"2023-07-13T22:22:20.794491+0000","last_undegraded":"2023-07-13T22:22:20.794491+0000","last_fullsized":"2023-07-13T22:22:20.794491+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:53:30.822156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689408+0000","last_change":"2023-07-13T22:22:19.694418+0000","last_active":"2023-07-13T22:22:20.689408+0000","last_peered":"2023-07-13T22:22:20.689408+0000","last_clean":"2023-07-13T22:22:20.689408+0000","last_became_active":"2023-07-13T22:22:19.694012+0000","last_became_peered":"2023-07-13T22:22:19.694012+0000","last_unstale":"2023-07-13T22:22:20.689408+0000","last_undegraded":"2023-07-13T22:22:20.689408+0000","last_fullsized":"2023-07-13T22:22:20.689408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:36:15.663442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689974+0000","last_change":"2023-07-13T22:22:19.704107+0000","last_active":"2023-07-13T22:22:20.689974+0000","last_peered":"2023-07-13T22:22:20.689974+0000","last_clean":"2023-07-13T22:22:20.689974+0000","last_became_active":"2023-07-13T22:22:19.702838+0000","last_became_peered":"2023-07-13T22:22:19.702838+0000","last_unstale":"2023-07-13T22:22:20.689974+0000","last_undegraded":"2023-07-13T22:22:20.689974+0000","last_fullsized":"2023-07-13T22:22:20.689974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:59.864501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794193+0000","last_change":"2023-07-13T22:22:19.705496+0000","last_active":"2023-07-13T22:22:20.794193+0000","last_peered":"2023-07-13T22:22:20.794193+0000","last_clean":"2023-07-13T22:22:20.794193+0000","last_became_active":"2023-07-13T22:22:19.705020+0000","last_became_peered":"2023-07-13T22:22:19.705020+0000","last_unstale":"2023-07-13T22:22:20.794193+0000","last_undegraded":"2023-07-13T22:22:20.794193+0000","last_fullsized":"2023-07-13T22:22:20.794193+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:07.299044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689017+0000","last_change":"2023-07-13T22:22:19.698140+0000","last_active":"2023-07-13T22:22:20.689017+0000","last_peered":"2023-07-13T22:22:20.689017+0000","last_clean":"2023-07-13T22:22:20.689017+0000","last_became_active":"2023-07-13T22:22:19.697454+0000","last_became_peered":"2023-07-13T22:22:19.697454+0000","last_unstale":"2023-07-13T22:22:20.689017+0000","last_undegraded":"2023-07-13T22:22:20.689017+0000","last_fullsized":"2023-07-13T22:22:20.689017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:38:07.942207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689594+0000","last_change":"2023-07-13T22:22:19.705141+0000","last_active":"2023-07-13T22:22:20.689594+0000","last_peered":"2023-07-13T22:22:20.689594+0000","last_clean":"2023-07-13T22:22:20.689594+0000","last_became_active":"2023-07-13T22:22:19.704822+0000","last_became_peered":"2023-07-13T22:22:19.704822+0000","last_unstale":"2023-07-13T22:22:20.689594+0000","last_undegraded":"2023-07-13T22:22:20.689594+0000","last_fullsized":"2023-07-13T22:22:20.689594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:07:19.584493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688353+0000","last_change":"2023-07-13T22:22:19.703860+0000","last_active":"2023-07-13T22:22:20.688353+0000","last_peered":"2023-07-13T22:22:20.688353+0000","last_clean":"2023-07-13T22:22:20.688353+0000","last_became_active":"2023-07-13T22:22:19.703574+0000","last_became_peered":"2023-07-13T22:22:19.703574+0000","last_unstale":"2023-07-13T22:22:20.688353+0000","last_undegraded":"2023-07-13T22:22:20.688353+0000","last_fullsized":"2023-07-13T22:22:20.688353+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:46:23.714214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046473+0000","last_change":"2023-07-13T22:22:19.692840+0000","last_active":"2023-07-13T22:22:21.046473+0000","last_peered":"2023-07-13T22:22:21.046473+0000","last_clean":"2023-07-13T22:22:21.046473+0000","last_became_active":"2023-07-13T22:22:19.692577+0000","last_became_peered":"2023-07-13T22:22:19.692577+0000","last_unstale":"2023-07-13T22:22:21.046473+0000","last_undegraded":"2023-07-13T22:22:21.046473+0000","last_fullsized":"2023-07-13T22:22:21.046473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:32.666494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046997+0000","last_change":"2023-07-13T22:22:19.694284+0000","last_active":"2023-07-13T22:22:21.046997+0000","last_peered":"2023-07-13T22:22:21.046997+0000","last_clean":"2023-07-13T22:22:21.046997+0000","last_became_active":"2023-07-13T22:22:19.694006+0000","last_became_peered":"2023-07-13T22:22:19.694006+0000","last_unstale":"2023-07-13T22:22:21.046997+0000","last_undegraded":"2023-07-13T22:22:21.046997+0000","last_fullsized":"2023-07-13T22:22:21.046997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:13:47.499901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733170+0000","last_change":"2023-07-13T22:22:19.702652+0000","last_active":"2023-07-13T22:22:20.733170+0000","last_peered":"2023-07-13T22:22:20.733170+0000","last_clean":"2023-07-13T22:22:20.733170+0000","last_became_active":"2023-07-13T22:22:19.702286+0000","last_became_peered":"2023-07-13T22:22:19.702286+0000","last_unstale":"2023-07-13T22:22:20.733170+0000","last_undegraded":"2023-07-13T22:22:20.733170+0000","last_fullsized":"2023-07-13T22:22:20.733170+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:01:51.003257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894591+0000","last_change":"2023-07-13T22:22:19.708353+0000","last_active":"2023-07-13T22:22:20.894591+0000","last_peered":"2023-07-13T22:22:20.894591+0000","last_clean":"2023-07-13T22:22:20.894591+0000","last_became_active":"2023-07-13T22:22:19.704536+0000","last_became_peered":"2023-07-13T22:22:19.704536+0000","last_unstale":"2023-07-13T22:22:20.894591+0000","last_undegraded":"2023-07-13T22:22:20.894591+0000","last_fullsized":"2023-07-13T22:22:20.894591+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:07:05.817546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.732958+0000","last_change":"2023-07-13T22:22:19.706285+0000","last_active":"2023-07-13T22:22:20.732958+0000","last_peered":"2023-07-13T22:22:20.732958+0000","last_clean":"2023-07-13T22:22:20.732958+0000","last_became_active":"2023-07-13T22:22:19.705988+0000","last_became_peered":"2023-07-13T22:22:19.705988+0000","last_unstale":"2023-07-13T22:22:20.732958+0000","last_undegraded":"2023-07-13T22:22:20.732958+0000","last_fullsized":"2023-07-13T22:22:20.732958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:55:08.308467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689179+0000","last_change":"2023-07-13T22:22:19.698367+0000","last_active":"2023-07-13T22:22:20.689179+0000","last_peered":"2023-07-13T22:22:20.689179+0000","last_clean":"2023-07-13T22:22:20.689179+0000","last_became_active":"2023-07-13T22:22:19.697805+0000","last_became_peered":"2023-07-13T22:22:19.697805+0000","last_unstale":"2023-07-13T22:22:20.689179+0000","last_undegraded":"2023-07-13T22:22:20.689179+0000","last_fullsized":"2023-07-13T22:22:20.689179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:38:48.583259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689020+0000","last_change":"2023-07-13T22:22:19.703882+0000","last_active":"2023-07-13T22:22:20.689020+0000","last_peered":"2023-07-13T22:22:20.689020+0000","last_clean":"2023-07-13T22:22:20.689020+0000","last_became_active":"2023-07-13T22:22:19.703616+0000","last_became_peered":"2023-07-13T22:22:19.703616+0000","last_unstale":"2023-07-13T22:22:20.689020+0000","last_undegraded":"2023-07-13T22:22:20.689020+0000","last_fullsized":"2023-07-13T22:22:20.689020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:15:37.062173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047162+0000","last_change":"2023-07-13T22:22:19.703128+0000","last_active":"2023-07-13T22:22:21.047162+0000","last_peered":"2023-07-13T22:22:21.047162+0000","last_clean":"2023-07-13T22:22:21.047162+0000","last_became_active":"2023-07-13T22:22:19.702526+0000","last_became_peered":"2023-07-13T22:22:19.702526+0000","last_unstale":"2023-07-13T22:22:21.047162+0000","last_undegraded":"2023-07-13T22:22:21.047162+0000","last_fullsized":"2023-07-13T22:22:21.047162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:44:59.667785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894316+0000","last_change":"2023-07-13T22:22:19.700742+0000","last_active":"2023-07-13T22:22:20.894316+0000","last_peered":"2023-07-13T22:22:20.894316+0000","last_clean":"2023-07-13T22:22:20.894316+0000","last_became_active":"2023-07-13T22:22:19.700307+0000","last_became_peered":"2023-07-13T22:22:19.700307+0000","last_unstale":"2023-07-13T22:22:20.894316+0000","last_undegraded":"2023-07-13T22:22:20.894316+0000","last_fullsized":"2023-07-13T22:22:20.894316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:26:12.845198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733078+0000","last_change":"2023-07-13T22:22:19.694741+0000","last_active":"2023-07-13T22:22:20.733078+0000","last_peered":"2023-07-13T22:22:20.733078+0000","last_clean":"2023-07-13T22:22:20.733078+0000","last_became_active":"2023-07-13T22:22:19.694471+0000","last_became_peered":"2023-07-13T22:22:19.694471+0000","last_unstale":"2023-07-13T22:22:20.733078+0000","last_undegraded":"2023-07-13T22:22:20.733078+0000","last_fullsized":"2023-07-13T22:22:20.733078+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:10:13.478680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687617+0000","last_change":"2023-07-13T22:22:19.700424+0000","last_active":"2023-07-13T22:22:20.687617+0000","last_peered":"2023-07-13T22:22:20.687617+0000","last_clean":"2023-07-13T22:22:20.687617+0000","last_became_active":"2023-07-13T22:22:19.699831+0000","last_became_peered":"2023-07-13T22:22:19.699831+0000","last_unstale":"2023-07-13T22:22:20.687617+0000","last_undegraded":"2023-07-13T22:22:20.687617+0000","last_fullsized":"2023-07-13T22:22:20.687617+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:05:35.997727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733168+0000","last_change":"2023-07-13T22:22:19.698503+0000","last_active":"2023-07-13T22:22:20.733168+0000","last_peered":"2023-07-13T22:22:20.733168+0000","last_clean":"2023-07-13T22:22:20.733168+0000","last_became_active":"2023-07-13T22:22:19.698203+0000","last_became_peered":"2023-07-13T22:22:19.698203+0000","last_unstale":"2023-07-13T22:22:20.733168+0000","last_undegraded":"2023-07-13T22:22:20.733168+0000","last_fullsized":"2023-07-13T22:22:20.733168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:11:24.335253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689408+0000","last_change":"2023-07-13T22:22:19.691385+0000","last_active":"2023-07-13T22:22:20.689408+0000","last_peered":"2023-07-13T22:22:20.689408+0000","last_clean":"2023-07-13T22:22:20.689408+0000","last_became_active":"2023-07-13T22:22:19.690026+0000","last_became_peered":"2023-07-13T22:22:19.690026+0000","last_unstale":"2023-07-13T22:22:20.689408+0000","last_undegraded":"2023-07-13T22:22:20.689408+0000","last_fullsized":"2023-07-13T22:22:20.689408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:01:42.811942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733097+0000","last_change":"2023-07-13T22:22:19.700492+0000","last_active":"2023-07-13T22:22:20.733097+0000","last_peered":"2023-07-13T22:22:20.733097+0000","last_clean":"2023-07-13T22:22:20.733097+0000","last_became_active":"2023-07-13T22:22:19.700239+0000","last_became_peered":"2023-07-13T22:22:19.700239+0000","last_unstale":"2023-07-13T22:22:20.733097+0000","last_undegraded":"2023-07-13T22:22:20.733097+0000","last_fullsized":"2023-07-13T22:22:20.733097+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:24:18.621969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.732775+0000","last_change":"2023-07-13T22:22:19.680540+0000","last_active":"2023-07-13T22:22:20.732775+0000","last_peered":"2023-07-13T22:22:20.732775+0000","last_clean":"2023-07-13T22:22:20.732775+0000","last_became_active":"2023-07-13T22:22:19.680275+0000","last_became_peered":"2023-07-13T22:22:19.680275+0000","last_unstale":"2023-07-13T22:22:20.732775+0000","last_undegraded":"2023-07-13T22:22:20.732775+0000","last_fullsized":"2023-07-13T22:22:20.732775+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:06:19.287774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046628+0000","last_change":"2023-07-13T22:22:19.703295+0000","last_active":"2023-07-13T22:22:21.046628+0000","last_peered":"2023-07-13T22:22:21.046628+0000","last_clean":"2023-07-13T22:22:21.046628+0000","last_became_active":"2023-07-13T22:22:19.702866+0000","last_became_peered":"2023-07-13T22:22:19.702866+0000","last_unstale":"2023-07-13T22:22:21.046628+0000","last_undegraded":"2023-07-13T22:22:21.046628+0000","last_fullsized":"2023-07-13T22:22:21.046628+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:09:26.260912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687051+0000","last_change":"2023-07-13T22:22:19.704318+0000","last_active":"2023-07-13T22:22:20.687051+0000","last_peered":"2023-07-13T22:22:20.687051+0000","last_clean":"2023-07-13T22:22:20.687051+0000","last_became_active":"2023-07-13T22:22:19.703828+0000","last_became_peered":"2023-07-13T22:22:19.703828+0000","last_unstale":"2023-07-13T22:22:20.687051+0000","last_undegraded":"2023-07-13T22:22:20.687051+0000","last_fullsized":"2023-07-13T22:22:20.687051+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:56:44.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046791+0000","last_change":"2023-07-13T22:22:19.689928+0000","last_active":"2023-07-13T22:22:21.046791+0000","last_peered":"2023-07-13T22:22:21.046791+0000","last_clean":"2023-07-13T22:22:21.046791+0000","last_became_active":"2023-07-13T22:22:19.689662+0000","last_became_peered":"2023-07-13T22:22:19.689662+0000","last_unstale":"2023-07-13T22:22:21.046791+0000","last_undegraded":"2023-07-13T22:22:21.046791+0000","last_fullsized":"2023-07-13T22:22:21.046791+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:43:58.934620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733242+0000","last_change":"2023-07-13T22:22:18.683795+0000","last_active":"2023-07-13T22:22:20.733242+0000","last_peered":"2023-07-13T22:22:20.733242+0000","last_clean":"2023-07-13T22:22:20.733242+0000","last_became_active":"2023-07-13T22:22:18.683513+0000","last_became_peered":"2023-07-13T22:22:18.683513+0000","last_unstale":"2023-07-13T22:22:20.733242+0000","last_undegraded":"2023-07-13T22:22:20.733242+0000","last_fullsized":"2023-07-13T22:22:20.733242+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:56:10.492367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687480+0000","last_change":"2023-07-13T22:22:19.699638+0000","last_active":"2023-07-13T22:22:20.687480+0000","last_peered":"2023-07-13T22:22:20.687480+0000","last_clean":"2023-07-13T22:22:20.687480+0000","last_became_active":"2023-07-13T22:22:19.698969+0000","last_became_peered":"2023-07-13T22:22:19.698969+0000","last_unstale":"2023-07-13T22:22:20.687480+0000","last_undegraded":"2023-07-13T22:22:20.687480+0000","last_fullsized":"2023-07-13T22:22:20.687480+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:54:11.424287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794426+0000","last_change":"2023-07-13T22:22:18.688079+0000","last_active":"2023-07-13T22:22:20.794426+0000","last_peered":"2023-07-13T22:22:20.794426+0000","last_clean":"2023-07-13T22:22:20.794426+0000","last_became_active":"2023-07-13T22:22:18.687780+0000","last_became_peered":"2023-07-13T22:22:18.687780+0000","last_unstale":"2023-07-13T22:22:20.794426+0000","last_undegraded":"2023-07-13T22:22:20.794426+0000","last_fullsized":"2023-07-13T22:22:20.794426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:07.214702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689472+0000","last_change":"2023-07-13T22:22:19.706038+0000","last_active":"2023-07-13T22:22:20.689472+0000","last_peered":"2023-07-13T22:22:20.689472+0000","last_clean":"2023-07-13T22:22:20.689472+0000","last_became_active":"2023-07-13T22:22:19.705276+0000","last_became_peered":"2023-07-13T22:22:19.705276+0000","last_unstale":"2023-07-13T22:22:20.689472+0000","last_undegraded":"2023-07-13T22:22:20.689472+0000","last_fullsized":"2023-07-13T22:22:20.689472+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:37:55.620973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687570+0000","last_change":"2023-07-13T22:22:18.683090+0000","last_active":"2023-07-13T22:22:20.687570+0000","last_peered":"2023-07-13T22:22:20.687570+0000","last_clean":"2023-07-13T22:22:20.687570+0000","last_became_active":"2023-07-13T22:22:18.682835+0000","last_became_peered":"2023-07-13T22:22:18.682835+0000","last_unstale":"2023-07-13T22:22:20.687570+0000","last_undegraded":"2023-07-13T22:22:20.687570+0000","last_fullsized":"2023-07-13T22:22:20.687570+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:06:59.391553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047224+0000","last_change":"2023-07-13T22:22:19.682220+0000","last_active":"2023-07-13T22:22:21.047224+0000","last_peered":"2023-07-13T22:22:21.047224+0000","last_clean":"2023-07-13T22:22:21.047224+0000","last_became_active":"2023-07-13T22:22:19.681376+0000","last_became_peered":"2023-07-13T22:22:19.681376+0000","last_unstale":"2023-07-13T22:22:21.047224+0000","last_undegraded":"2023-07-13T22:22:21.047224+0000","last_fullsized":"2023-07-13T22:22:21.047224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:22.748465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689165+0000","last_change":"2023-07-13T22:22:18.682480+0000","last_active":"2023-07-13T22:22:20.689165+0000","last_peered":"2023-07-13T22:22:20.689165+0000","last_clean":"2023-07-13T22:22:20.689165+0000","last_became_active":"2023-07-13T22:22:18.682093+0000","last_became_peered":"2023-07-13T22:22:18.682093+0000","last_unstale":"2023-07-13T22:22:20.689165+0000","last_undegraded":"2023-07-13T22:22:20.689165+0000","last_fullsized":"2023-07-13T22:22:20.689165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:28:59.822612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794018+0000","last_change":"2023-07-13T22:22:19.694289+0000","last_active":"2023-07-13T22:22:20.794018+0000","last_peered":"2023-07-13T22:22:20.794018+0000","last_clean":"2023-07-13T22:22:20.794018+0000","last_became_active":"2023-07-13T22:22:19.693972+0000","last_became_peered":"2023-07-13T22:22:19.693972+0000","last_unstale":"2023-07-13T22:22:20.794018+0000","last_undegraded":"2023-07-13T22:22:20.794018+0000","last_fullsized":"2023-07-13T22:22:20.794018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:51.327980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689146+0000","last_change":"2023-07-13T22:22:18.680139+0000","last_active":"2023-07-13T22:22:20.689146+0000","last_peered":"2023-07-13T22:22:20.689146+0000","last_clean":"2023-07-13T22:22:20.689146+0000","last_became_active":"2023-07-13T22:22:18.679855+0000","last_became_peered":"2023-07-13T22:22:18.679855+0000","last_unstale":"2023-07-13T22:22:20.689146+0000","last_undegraded":"2023-07-13T22:22:20.689146+0000","last_fullsized":"2023-07-13T22:22:20.689146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:40:01.218615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689817+0000","last_change":"2023-07-13T22:22:19.701678+0000","last_active":"2023-07-13T22:22:20.689817+0000","last_peered":"2023-07-13T22:22:20.689817+0000","last_clean":"2023-07-13T22:22:20.689817+0000","last_became_active":"2023-07-13T22:22:19.696615+0000","last_became_peered":"2023-07-13T22:22:19.696615+0000","last_unstale":"2023-07-13T22:22:20.689817+0000","last_undegraded":"2023-07-13T22:22:20.689817+0000","last_fullsized":"2023-07-13T22:22:20.689817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:48:09.116672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047078+0000","last_change":"2023-07-13T22:22:18.684907+0000","last_active":"2023-07-13T22:22:21.047078+0000","last_peered":"2023-07-13T22:22:21.047078+0000","last_clean":"2023-07-13T22:22:21.047078+0000","last_became_active":"2023-07-13T22:22:18.684571+0000","last_became_peered":"2023-07-13T22:22:18.684571+0000","last_unstale":"2023-07-13T22:22:21.047078+0000","last_undegraded":"2023-07-13T22:22:21.047078+0000","last_fullsized":"2023-07-13T22:22:21.047078+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:38:20.355956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689188+0000","last_change":"2023-07-13T22:22:19.709929+0000","last_active":"2023-07-13T22:22:20.689188+0000","last_peered":"2023-07-13T22:22:20.689188+0000","last_clean":"2023-07-13T22:22:20.689188+0000","last_became_active":"2023-07-13T22:22:19.709626+0000","last_became_peered":"2023-07-13T22:22:19.709626+0000","last_unstale":"2023-07-13T22:22:20.689188+0000","last_undegraded":"2023-07-13T22:22:20.689188+0000","last_fullsized":"2023-07-13T22:22:20.689188+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:56:59.213235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687114+0000","last_change":"2023-07-13T22:22:18.680505+0000","last_active":"2023-07-13T22:22:20.687114+0000","last_peered":"2023-07-13T22:22:20.687114+0000","last_clean":"2023-07-13T22:22:20.687114+0000","last_became_active":"2023-07-13T22:22:18.680200+0000","last_became_peered":"2023-07-13T22:22:18.680200+0000","last_unstale":"2023-07-13T22:22:20.687114+0000","last_undegraded":"2023-07-13T22:22:20.687114+0000","last_fullsized":"2023-07-13T22:22:20.687114+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:13:23.676059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047261+0000","last_change":"2023-07-13T22:22:19.682712+0000","last_active":"2023-07-13T22:22:21.047261+0000","last_peered":"2023-07-13T22:22:21.047261+0000","last_clean":"2023-07-13T22:22:21.047261+0000","last_became_active":"2023-07-13T22:22:19.682379+0000","last_became_peered":"2023-07-13T22:22:19.682379+0000","last_unstale":"2023-07-13T22:22:21.047261+0000","last_undegraded":"2023-07-13T22:22:21.047261+0000","last_fullsized":"2023-07-13T22:22:21.047261+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:06.318949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688871+0000","last_change":"2023-07-13T22:22:18.681506+0000","last_active":"2023-07-13T22:22:20.688871+0000","last_peered":"2023-07-13T22:22:20.688871+0000","last_clean":"2023-07-13T22:22:20.688871+0000","last_became_active":"2023-07-13T22:22:18.681195+0000","last_became_peered":"2023-07-13T22:22:18.681195+0000","last_unstale":"2023-07-13T22:22:20.688871+0000","last_undegraded":"2023-07-13T22:22:20.688871+0000","last_fullsized":"2023-07-13T22:22:20.688871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:20.385029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894635+0000","last_change":"2023-07-13T22:22:19.704324+0000","last_active":"2023-07-13T22:22:20.894635+0000","last_peered":"2023-07-13T22:22:20.894635+0000","last_clean":"2023-07-13T22:22:20.894635+0000","last_became_active":"2023-07-13T22:22:19.703616+0000","last_became_peered":"2023-07-13T22:22:19.703616+0000","last_unstale":"2023-07-13T22:22:20.894635+0000","last_undegraded":"2023-07-13T22:22:20.894635+0000","last_fullsized":"2023-07-13T22:22:20.894635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:10:45.045964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047329+0000","last_change":"2023-07-13T22:22:18.667612+0000","last_active":"2023-07-13T22:22:21.047329+0000","last_peered":"2023-07-13T22:22:21.047329+0000","last_clean":"2023-07-13T22:22:21.047329+0000","last_became_active":"2023-07-13T22:22:18.667270+0000","last_became_peered":"2023-07-13T22:22:18.667270+0000","last_unstale":"2023-07-13T22:22:21.047329+0000","last_undegraded":"2023-07-13T22:22:21.047329+0000","last_fullsized":"2023-07-13T22:22:21.047329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:15:57.132576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047057+0000","last_change":"2023-07-13T22:22:19.685896+0000","last_active":"2023-07-13T22:22:21.047057+0000","last_peered":"2023-07-13T22:22:21.047057+0000","last_clean":"2023-07-13T22:22:21.047057+0000","last_became_active":"2023-07-13T22:22:19.685221+0000","last_became_peered":"2023-07-13T22:22:19.685221+0000","last_unstale":"2023-07-13T22:22:21.047057+0000","last_undegraded":"2023-07-13T22:22:21.047057+0000","last_fullsized":"2023-07-13T22:22:21.047057+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:31:13.446473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689715+0000","last_change":"2023-07-13T22:22:18.679413+0000","last_active":"2023-07-13T22:22:20.689715+0000","last_peered":"2023-07-13T22:22:20.689715+0000","last_clean":"2023-07-13T22:22:20.689715+0000","last_became_active":"2023-07-13T22:22:18.679111+0000","last_became_peered":"2023-07-13T22:22:18.679111+0000","last_unstale":"2023-07-13T22:22:20.689715+0000","last_undegraded":"2023-07-13T22:22:20.689715+0000","last_fullsized":"2023-07-13T22:22:20.689715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:41:45.164905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794403+0000","last_change":"2023-07-13T22:22:19.697191+0000","last_active":"2023-07-13T22:22:20.794403+0000","last_peered":"2023-07-13T22:22:20.794403+0000","last_clean":"2023-07-13T22:22:20.794403+0000","last_became_active":"2023-07-13T22:22:19.696913+0000","last_became_peered":"2023-07-13T22:22:19.696913+0000","last_unstale":"2023-07-13T22:22:20.794403+0000","last_undegraded":"2023-07-13T22:22:20.794403+0000","last_fullsized":"2023-07-13T22:22:20.794403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:05:35.997727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.690056+0000","last_change":"2023-07-13T22:22:18.679915+0000","last_active":"2023-07-13T22:22:20.690056+0000","last_peered":"2023-07-13T22:22:20.690056+0000","last_clean":"2023-07-13T22:22:20.690056+0000","last_became_active":"2023-07-13T22:22:18.679608+0000","last_became_peered":"2023-07-13T22:22:18.679608+0000","last_unstale":"2023-07-13T22:22:20.690056+0000","last_undegraded":"2023-07-13T22:22:20.690056+0000","last_fullsized":"2023-07-13T22:22:20.690056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:31.660354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689121+0000","last_change":"2023-07-13T22:22:19.704021+0000","last_active":"2023-07-13T22:22:20.689121+0000","last_peered":"2023-07-13T22:22:20.689121+0000","last_clean":"2023-07-13T22:22:20.689121+0000","last_became_active":"2023-07-13T22:22:19.703765+0000","last_became_peered":"2023-07-13T22:22:19.703765+0000","last_unstale":"2023-07-13T22:22:20.689121+0000","last_undegraded":"2023-07-13T22:22:20.689121+0000","last_fullsized":"2023-07-13T22:22:20.689121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:40:43.964736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047094+0000","last_change":"2023-07-13T22:22:18.688250+0000","last_active":"2023-07-13T22:22:21.047094+0000","last_peered":"2023-07-13T22:22:21.047094+0000","last_clean":"2023-07-13T22:22:21.047094+0000","last_became_active":"2023-07-13T22:22:18.687919+0000","last_became_peered":"2023-07-13T22:22:18.687919+0000","last_unstale":"2023-07-13T22:22:21.047094+0000","last_undegraded":"2023-07-13T22:22:21.047094+0000","last_fullsized":"2023-07-13T22:22:21.047094+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:05:16.510489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794460+0000","last_change":"2023-07-13T22:22:19.705213+0000","last_active":"2023-07-13T22:22:20.794460+0000","last_peered":"2023-07-13T22:22:20.794460+0000","last_clean":"2023-07-13T22:22:20.794460+0000","last_became_active":"2023-07-13T22:22:19.704914+0000","last_became_peered":"2023-07-13T22:22:19.704914+0000","last_unstale":"2023-07-13T22:22:20.794460+0000","last_undegraded":"2023-07-13T22:22:20.794460+0000","last_fullsized":"2023-07-13T22:22:20.794460+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:39:50.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047399+0000","last_change":"2023-07-13T22:22:18.677934+0000","last_active":"2023-07-13T22:22:21.047399+0000","last_peered":"2023-07-13T22:22:21.047399+0000","last_clean":"2023-07-13T22:22:21.047399+0000","last_became_active":"2023-07-13T22:22:18.677635+0000","last_became_peered":"2023-07-13T22:22:18.677635+0000","last_unstale":"2023-07-13T22:22:21.047399+0000","last_undegraded":"2023-07-13T22:22:21.047399+0000","last_fullsized":"2023-07-13T22:22:21.047399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:28:38.856054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046877+0000","last_change":"2023-07-13T22:22:19.693402+0000","last_active":"2023-07-13T22:22:21.046877+0000","last_peered":"2023-07-13T22:22:21.046877+0000","last_clean":"2023-07-13T22:22:21.046877+0000","last_became_active":"2023-07-13T22:22:19.693100+0000","last_became_peered":"2023-07-13T22:22:19.693100+0000","last_unstale":"2023-07-13T22:22:21.046877+0000","last_undegraded":"2023-07-13T22:22:21.046877+0000","last_fullsized":"2023-07-13T22:22:21.046877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:38:21.362096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687311+0000","last_change":"2023-07-13T22:22:18.680682+0000","last_active":"2023-07-13T22:22:20.687311+0000","last_peered":"2023-07-13T22:22:20.687311+0000","last_clean":"2023-07-13T22:22:20.687311+0000","last_became_active":"2023-07-13T22:22:18.680290+0000","last_became_peered":"2023-07-13T22:22:18.680290+0000","last_unstale":"2023-07-13T22:22:20.687311+0000","last_undegraded":"2023-07-13T22:22:20.687311+0000","last_fullsized":"2023-07-13T22:22:20.687311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:06.292904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046669+0000","last_change":"2023-07-13T22:22:19.702221+0000","last_active":"2023-07-13T22:22:21.046669+0000","last_peered":"2023-07-13T22:22:21.046669+0000","last_clean":"2023-07-13T22:22:21.046669+0000","last_became_active":"2023-07-13T22:22:19.701927+0000","last_became_peered":"2023-07-13T22:22:19.701927+0000","last_unstale":"2023-07-13T22:22:21.046669+0000","last_undegraded":"2023-07-13T22:22:21.046669+0000","last_fullsized":"2023-07-13T22:22:21.046669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:03:40.805354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.732749+0000","last_change":"2023-07-13T22:22:18.683147+0000","last_active":"2023-07-13T22:22:20.732749+0000","last_peered":"2023-07-13T22:22:20.732749+0000","last_clean":"2023-07-13T22:22:20.732749+0000","last_became_active":"2023-07-13T22:22:18.682827+0000","last_became_peered":"2023-07-13T22:22:18.682827+0000","last_unstale":"2023-07-13T22:22:20.732749+0000","last_undegraded":"2023-07-13T22:22:20.732749+0000","last_fullsized":"2023-07-13T22:22:20.732749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:58:10.746801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047233+0000","last_change":"2023-07-13T22:22:19.694063+0000","last_active":"2023-07-13T22:22:21.047233+0000","last_peered":"2023-07-13T22:22:21.047233+0000","last_clean":"2023-07-13T22:22:21.047233+0000","last_became_active":"2023-07-13T22:22:19.693779+0000","last_became_peered":"2023-07-13T22:22:19.693779+0000","last_unstale":"2023-07-13T22:22:21.047233+0000","last_undegraded":"2023-07-13T22:22:21.047233+0000","last_fullsized":"2023-07-13T22:22:21.047233+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:59:28.209710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046667+0000","last_change":"2023-07-13T22:22:18.688049+0000","last_active":"2023-07-13T22:22:21.046667+0000","last_peered":"2023-07-13T22:22:21.046667+0000","last_clean":"2023-07-13T22:22:21.046667+0000","last_became_active":"2023-07-13T22:22:18.687795+0000","last_became_peered":"2023-07-13T22:22:18.687795+0000","last_unstale":"2023-07-13T22:22:21.046667+0000","last_undegraded":"2023-07-13T22:22:21.046667+0000","last_fullsized":"2023-07-13T22:22:21.046667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:29:57.085734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047172+0000","last_change":"2023-07-13T22:22:19.689345+0000","last_active":"2023-07-13T22:22:21.047172+0000","last_peered":"2023-07-13T22:22:21.047172+0000","last_clean":"2023-07-13T22:22:21.047172+0000","last_became_active":"2023-07-13T22:22:19.689067+0000","last_became_peered":"2023-07-13T22:22:19.689067+0000","last_unstale":"2023-07-13T22:22:21.047172+0000","last_undegraded":"2023-07-13T22:22:21.047172+0000","last_fullsized":"2023-07-13T22:22:21.047172+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:29:32.779598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689222+0000","last_change":"2023-07-13T22:22:18.683350+0000","last_active":"2023-07-13T22:22:20.689222+0000","last_peered":"2023-07-13T22:22:20.689222+0000","last_clean":"2023-07-13T22:22:20.689222+0000","last_became_active":"2023-07-13T22:22:18.683053+0000","last_became_peered":"2023-07-13T22:22:18.683053+0000","last_unstale":"2023-07-13T22:22:20.689222+0000","last_undegraded":"2023-07-13T22:22:20.689222+0000","last_fullsized":"2023-07-13T22:22:20.689222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:05:35.442793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.690167+0000","last_change":"2023-07-13T22:22:19.701641+0000","last_active":"2023-07-13T22:22:20.690167+0000","last_peered":"2023-07-13T22:22:20.690167+0000","last_clean":"2023-07-13T22:22:20.690167+0000","last_became_active":"2023-07-13T22:22:19.701196+0000","last_became_peered":"2023-07-13T22:22:19.701196+0000","last_unstale":"2023-07-13T22:22:20.690167+0000","last_undegraded":"2023-07-13T22:22:20.690167+0000","last_fullsized":"2023-07-13T22:22:20.690167+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:12:34.777446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689931+0000","last_change":"2023-07-13T22:22:18.683089+0000","last_active":"2023-07-13T22:22:20.689931+0000","last_peered":"2023-07-13T22:22:20.689931+0000","last_clean":"2023-07-13T22:22:20.689931+0000","last_became_active":"2023-07-13T22:22:18.682735+0000","last_became_peered":"2023-07-13T22:22:18.682735+0000","last_unstale":"2023-07-13T22:22:20.689931+0000","last_undegraded":"2023-07-13T22:22:20.689931+0000","last_fullsized":"2023-07-13T22:22:20.689931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:46:06.117007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689635+0000","last_change":"2023-07-13T22:22:19.705671+0000","last_active":"2023-07-13T22:22:20.689635+0000","last_peered":"2023-07-13T22:22:20.689635+0000","last_clean":"2023-07-13T22:22:20.689635+0000","last_became_active":"2023-07-13T22:22:19.704886+0000","last_became_peered":"2023-07-13T22:22:19.704886+0000","last_unstale":"2023-07-13T22:22:20.689635+0000","last_undegraded":"2023-07-13T22:22:20.689635+0000","last_fullsized":"2023-07-13T22:22:20.689635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:32:53.170164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894534+0000","last_change":"2023-07-13T22:22:18.687950+0000","last_active":"2023-07-13T22:22:20.894534+0000","last_peered":"2023-07-13T22:22:20.894534+0000","last_clean":"2023-07-13T22:22:20.894534+0000","last_became_active":"2023-07-13T22:22:18.687656+0000","last_became_peered":"2023-07-13T22:22:18.687656+0000","last_unstale":"2023-07-13T22:22:20.894534+0000","last_undegraded":"2023-07-13T22:22:20.894534+0000","last_fullsized":"2023-07-13T22:22:20.894534+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:59:50.029440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894608+0000","last_change":"2023-07-13T22:22:19.697958+0000","last_active":"2023-07-13T22:22:20.894608+0000","last_peered":"2023-07-13T22:22:20.894608+0000","last_clean":"2023-07-13T22:22:20.894608+0000","last_became_active":"2023-07-13T22:22:19.697687+0000","last_became_peered":"2023-07-13T22:22:19.697687+0000","last_unstale":"2023-07-13T22:22:20.894608+0000","last_undegraded":"2023-07-13T22:22:20.894608+0000","last_fullsized":"2023-07-13T22:22:20.894608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:13.709947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687400+0000","last_change":"2023-07-13T22:22:18.683135+0000","last_active":"2023-07-13T22:22:20.687400+0000","last_peered":"2023-07-13T22:22:20.687400+0000","last_clean":"2023-07-13T22:22:20.687400+0000","last_became_active":"2023-07-13T22:22:18.682853+0000","last_became_peered":"2023-07-13T22:22:18.682853+0000","last_unstale":"2023-07-13T22:22:20.687400+0000","last_undegraded":"2023-07-13T22:22:20.687400+0000","last_fullsized":"2023-07-13T22:22:20.687400+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:21:34.358104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046537+0000","last_change":"2023-07-13T22:22:19.704285+0000","last_active":"2023-07-13T22:22:21.046537+0000","last_peered":"2023-07-13T22:22:21.046537+0000","last_clean":"2023-07-13T22:22:21.046537+0000","last_became_active":"2023-07-13T22:22:19.703643+0000","last_became_peered":"2023-07-13T22:22:19.703643+0000","last_unstale":"2023-07-13T22:22:21.046537+0000","last_undegraded":"2023-07-13T22:22:21.046537+0000","last_fullsized":"2023-07-13T22:22:21.046537+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:12:34.777446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894435+0000","last_change":"2023-07-13T22:22:18.678327+0000","last_active":"2023-07-13T22:22:20.894435+0000","last_peered":"2023-07-13T22:22:20.894435+0000","last_clean":"2023-07-13T22:22:20.894435+0000","last_became_active":"2023-07-13T22:22:18.678067+0000","last_became_peered":"2023-07-13T22:22:18.678067+0000","last_unstale":"2023-07-13T22:22:20.894435+0000","last_undegraded":"2023-07-13T22:22:20.894435+0000","last_fullsized":"2023-07-13T22:22:20.894435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:23:28.504684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794046+0000","last_change":"2023-07-13T22:22:19.689914+0000","last_active":"2023-07-13T22:22:20.794046+0000","last_peered":"2023-07-13T22:22:20.794046+0000","last_clean":"2023-07-13T22:22:20.794046+0000","last_became_active":"2023-07-13T22:22:19.689462+0000","last_became_peered":"2023-07-13T22:22:19.689462+0000","last_unstale":"2023-07-13T22:22:20.794046+0000","last_undegraded":"2023-07-13T22:22:20.794046+0000","last_fullsized":"2023-07-13T22:22:20.794046+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:51:16.874448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894688+0000","last_change":"2023-07-13T22:22:18.683882+0000","last_active":"2023-07-13T22:22:20.894688+0000","last_peered":"2023-07-13T22:22:20.894688+0000","last_clean":"2023-07-13T22:22:20.894688+0000","last_became_active":"2023-07-13T22:22:18.683559+0000","last_became_peered":"2023-07-13T22:22:18.683559+0000","last_unstale":"2023-07-13T22:22:20.894688+0000","last_undegraded":"2023-07-13T22:22:20.894688+0000","last_fullsized":"2023-07-13T22:22:20.894688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:49:18.750217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688335+0000","last_change":"2023-07-13T22:22:19.692843+0000","last_active":"2023-07-13T22:22:20.688335+0000","last_peered":"2023-07-13T22:22:20.688335+0000","last_clean":"2023-07-13T22:22:20.688335+0000","last_became_active":"2023-07-13T22:22:19.689222+0000","last_became_peered":"2023-07-13T22:22:19.689222+0000","last_unstale":"2023-07-13T22:22:20.688335+0000","last_undegraded":"2023-07-13T22:22:20.688335+0000","last_fullsized":"2023-07-13T22:22:20.688335+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:58:08.188540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046925+0000","last_change":"2023-07-13T22:22:18.687862+0000","last_active":"2023-07-13T22:22:21.046925+0000","last_peered":"2023-07-13T22:22:21.046925+0000","last_clean":"2023-07-13T22:22:21.046925+0000","last_became_active":"2023-07-13T22:22:18.687578+0000","last_became_peered":"2023-07-13T22:22:18.687578+0000","last_unstale":"2023-07-13T22:22:21.046925+0000","last_undegraded":"2023-07-13T22:22:21.046925+0000","last_fullsized":"2023-07-13T22:22:21.046925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:32:38.019370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047020+0000","last_change":"2023-07-13T22:22:19.680279+0000","last_active":"2023-07-13T22:22:21.047020+0000","last_peered":"2023-07-13T22:22:21.047020+0000","last_clean":"2023-07-13T22:22:21.047020+0000","last_became_active":"2023-07-13T22:22:19.680000+0000","last_became_peered":"2023-07-13T22:22:19.680000+0000","last_unstale":"2023-07-13T22:22:21.047020+0000","last_undegraded":"2023-07-13T22:22:21.047020+0000","last_fullsized":"2023-07-13T22:22:21.047020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:44:59.667785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894344+0000","last_change":"2023-07-13T22:22:18.678463+0000","last_active":"2023-07-13T22:22:20.894344+0000","last_peered":"2023-07-13T22:22:20.894344+0000","last_clean":"2023-07-13T22:22:20.894344+0000","last_became_active":"2023-07-13T22:22:18.678158+0000","last_became_peered":"2023-07-13T22:22:18.678158+0000","last_unstale":"2023-07-13T22:22:20.894344+0000","last_undegraded":"2023-07-13T22:22:20.894344+0000","last_fullsized":"2023-07-13T22:22:20.894344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:01.656282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689344+0000","last_change":"2023-07-13T22:22:19.693027+0000","last_active":"2023-07-13T22:22:20.689344+0000","last_peered":"2023-07-13T22:22:20.689344+0000","last_clean":"2023-07-13T22:22:20.689344+0000","last_became_active":"2023-07-13T22:22:19.692733+0000","last_became_peered":"2023-07-13T22:22:19.692733+0000","last_unstale":"2023-07-13T22:22:20.689344+0000","last_undegraded":"2023-07-13T22:22:20.689344+0000","last_fullsized":"2023-07-13T22:22:20.689344+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:02:24.245064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688499+0000","last_change":"2023-07-13T22:22:18.677191+0000","last_active":"2023-07-13T22:22:20.688499+0000","last_peered":"2023-07-13T22:22:20.688499+0000","last_clean":"2023-07-13T22:22:20.688499+0000","last_became_active":"2023-07-13T22:22:18.676904+0000","last_became_peered":"2023-07-13T22:22:18.676904+0000","last_unstale":"2023-07-13T22:22:20.688499+0000","last_undegraded":"2023-07-13T22:22:20.688499+0000","last_fullsized":"2023-07-13T22:22:20.688499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:25:19.932858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688818+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:22:20.688818+0000","last_peered":"2023-07-13T22:22:20.688818+0000","last_clean":"2023-07-13T22:22:20.688818+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:22:20.688818+0000","last_undegraded":"2023-07-13T22:22:20.688818+0000","last_fullsized":"2023-07-13T22:22:20.688818+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:52:04.409917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047158+0000","last_change":"2023-07-13T22:22:19.688642+0000","last_active":"2023-07-13T22:22:21.047158+0000","last_peered":"2023-07-13T22:22:21.047158+0000","last_clean":"2023-07-13T22:22:21.047158+0000","last_became_active":"2023-07-13T22:22:19.688378+0000","last_became_peered":"2023-07-13T22:22:19.688378+0000","last_unstale":"2023-07-13T22:22:21.047158+0000","last_undegraded":"2023-07-13T22:22:21.047158+0000","last_fullsized":"2023-07-13T22:22:21.047158+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:14:10.896364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046895+0000","last_change":"2023-07-13T22:22:18.667567+0000","last_active":"2023-07-13T22:22:21.046895+0000","last_peered":"2023-07-13T22:22:21.046895+0000","last_clean":"2023-07-13T22:22:21.046895+0000","last_became_active":"2023-07-13T22:22:18.667250+0000","last_became_peered":"2023-07-13T22:22:18.667250+0000","last_unstale":"2023-07-13T22:22:21.046895+0000","last_undegraded":"2023-07-13T22:22:21.046895+0000","last_fullsized":"2023-07-13T22:22:21.046895+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:54:59.714270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047185+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:22:21.047185+0000","last_peered":"2023-07-13T22:22:21.047185+0000","last_clean":"2023-07-13T22:22:21.047185+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:22:21.047185+0000","last_undegraded":"2023-07-13T22:22:21.047185+0000","last_fullsized":"2023-07-13T22:22:21.047185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:43:50.119354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687256+0000","last_change":"2023-07-13T22:22:19.698783+0000","last_active":"2023-07-13T22:22:20.687256+0000","last_peered":"2023-07-13T22:22:20.687256+0000","last_clean":"2023-07-13T22:22:20.687256+0000","last_became_active":"2023-07-13T22:22:19.698547+0000","last_became_peered":"2023-07-13T22:22:19.698547+0000","last_unstale":"2023-07-13T22:22:20.687256+0000","last_undegraded":"2023-07-13T22:22:20.687256+0000","last_fullsized":"2023-07-13T22:22:20.687256+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:34.008871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688763+0000","last_change":"2023-07-13T22:22:18.680294+0000","last_active":"2023-07-13T22:22:20.688763+0000","last_peered":"2023-07-13T22:22:20.688763+0000","last_clean":"2023-07-13T22:22:20.688763+0000","last_became_active":"2023-07-13T22:22:18.679989+0000","last_became_peered":"2023-07-13T22:22:18.679989+0000","last_unstale":"2023-07-13T22:22:20.688763+0000","last_undegraded":"2023-07-13T22:22:20.688763+0000","last_fullsized":"2023-07-13T22:22:20.688763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:05:34.616243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687231+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:22:20.687231+0000","last_peered":"2023-07-13T22:22:20.687231+0000","last_clean":"2023-07-13T22:22:20.687231+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:22:20.687231+0000","last_undegraded":"2023-07-13T22:22:20.687231+0000","last_fullsized":"2023-07-13T22:22:20.687231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:49:44.926269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":191,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047073+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:22:21.047073+0000","last_peered":"2023-07-13T22:22:21.047073+0000","last_clean":"2023-07-13T22:22:21.047073+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:22:21.047073+0000","last_undegraded":"2023-07-13T22:22:21.047073+0000","last_fullsized":"2023-07-13T22:22:21.047073+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:53:29.774952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047096+0000","last_change":"2023-07-13T22:22:19.685880+0000","last_active":"2023-07-13T22:22:21.047096+0000","last_peered":"2023-07-13T22:22:21.047096+0000","last_clean":"2023-07-13T22:22:21.047096+0000","last_became_active":"2023-07-13T22:22:19.685586+0000","last_became_peered":"2023-07-13T22:22:19.685586+0000","last_unstale":"2023-07-13T22:22:21.047096+0000","last_undegraded":"2023-07-13T22:22:21.047096+0000","last_fullsized":"2023-07-13T22:22:21.047096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:47.579372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894542+0000","last_change":"2023-07-13T22:22:18.681823+0000","last_active":"2023-07-13T22:22:20.894542+0000","last_peered":"2023-07-13T22:22:20.894542+0000","last_clean":"2023-07-13T22:22:20.894542+0000","last_became_active":"2023-07-13T22:22:18.681516+0000","last_became_peered":"2023-07-13T22:22:18.681516+0000","last_unstale":"2023-07-13T22:22:20.894542+0000","last_undegraded":"2023-07-13T22:22:20.894542+0000","last_fullsized":"2023-07-13T22:22:20.894542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:34:55.893996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046749+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:22:21.046749+0000","last_peered":"2023-07-13T22:22:21.046749+0000","last_clean":"2023-07-13T22:22:21.046749+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:22:21.046749+0000","last_undegraded":"2023-07-13T22:22:21.046749+0000","last_fullsized":"2023-07-13T22:22:21.046749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:59.847327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688607+0000","last_change":"2023-07-13T22:22:19.690141+0000","last_active":"2023-07-13T22:22:20.688607+0000","last_peered":"2023-07-13T22:22:20.688607+0000","last_clean":"2023-07-13T22:22:20.688607+0000","last_became_active":"2023-07-13T22:22:19.689874+0000","last_became_peered":"2023-07-13T22:22:19.689874+0000","last_unstale":"2023-07-13T22:22:20.688607+0000","last_undegraded":"2023-07-13T22:22:20.688607+0000","last_fullsized":"2023-07-13T22:22:20.688607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:42:36.503117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046990+0000","last_change":"2023-07-13T22:22:18.687553+0000","last_active":"2023-07-13T22:22:21.046990+0000","last_peered":"2023-07-13T22:22:21.046990+0000","last_clean":"2023-07-13T22:22:21.046990+0000","last_became_active":"2023-07-13T22:22:18.687306+0000","last_became_peered":"2023-07-13T22:22:18.687306+0000","last_unstale":"2023-07-13T22:22:21.046990+0000","last_undegraded":"2023-07-13T22:22:21.046990+0000","last_fullsized":"2023-07-13T22:22:21.046990+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:28.316674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046738+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:22:21.046738+0000","last_peered":"2023-07-13T22:22:21.046738+0000","last_clean":"2023-07-13T22:22:21.046738+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:22:21.046738+0000","last_undegraded":"2023-07-13T22:22:21.046738+0000","last_fullsized":"2023-07-13T22:22:21.046738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:28:58.208287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047063+0000","last_change":"2023-07-13T22:22:19.694828+0000","last_active":"2023-07-13T22:22:21.047063+0000","last_peered":"2023-07-13T22:22:21.047063+0000","last_clean":"2023-07-13T22:22:21.047063+0000","last_became_active":"2023-07-13T22:22:19.694555+0000","last_became_peered":"2023-07-13T22:22:19.694555+0000","last_unstale":"2023-07-13T22:22:21.047063+0000","last_undegraded":"2023-07-13T22:22:21.047063+0000","last_fullsized":"2023-07-13T22:22:21.047063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:14:10.896364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794276+0000","last_change":"2023-07-13T22:22:18.677467+0000","last_active":"2023-07-13T22:22:20.794276+0000","last_peered":"2023-07-13T22:22:20.794276+0000","last_clean":"2023-07-13T22:22:20.794276+0000","last_became_active":"2023-07-13T22:22:18.677152+0000","last_became_peered":"2023-07-13T22:22:18.677152+0000","last_unstale":"2023-07-13T22:22:20.794276+0000","last_undegraded":"2023-07-13T22:22:20.794276+0000","last_fullsized":"2023-07-13T22:22:20.794276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:06:59.391553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688712+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:22:20.688712+0000","last_peered":"2023-07-13T22:22:20.688712+0000","last_clean":"2023-07-13T22:22:20.688712+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:22:20.688712+0000","last_undegraded":"2023-07-13T22:22:20.688712+0000","last_fullsized":"2023-07-13T22:22:20.688712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:14:57.399173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687051+0000","last_change":"2023-07-13T22:22:19.691360+0000","last_active":"2023-07-13T22:22:20.687051+0000","last_peered":"2023-07-13T22:22:20.687051+0000","last_clean":"2023-07-13T22:22:20.687051+0000","last_became_active":"2023-07-13T22:22:19.691092+0000","last_became_peered":"2023-07-13T22:22:19.691092+0000","last_unstale":"2023-07-13T22:22:20.687051+0000","last_undegraded":"2023-07-13T22:22:20.687051+0000","last_fullsized":"2023-07-13T22:22:20.687051+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:45:45.050703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.732841+0000","last_change":"2023-07-13T22:22:18.678304+0000","last_active":"2023-07-13T22:22:20.732841+0000","last_peered":"2023-07-13T22:22:20.732841+0000","last_clean":"2023-07-13T22:22:20.732841+0000","last_became_active":"2023-07-13T22:22:18.677999+0000","last_became_peered":"2023-07-13T22:22:18.677999+0000","last_unstale":"2023-07-13T22:22:20.732841+0000","last_undegraded":"2023-07-13T22:22:20.732841+0000","last_fullsized":"2023-07-13T22:22:20.732841+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:24:14.444707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733041+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:22:20.733041+0000","last_peered":"2023-07-13T22:22:20.733041+0000","last_clean":"2023-07-13T22:22:20.733041+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:22:20.733041+0000","last_undegraded":"2023-07-13T22:22:20.733041+0000","last_fullsized":"2023-07-13T22:22:20.733041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:27:20.368439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794090+0000","last_change":"2023-07-13T22:22:19.690009+0000","last_active":"2023-07-13T22:22:20.794090+0000","last_peered":"2023-07-13T22:22:20.794090+0000","last_clean":"2023-07-13T22:22:20.794090+0000","last_became_active":"2023-07-13T22:22:19.689696+0000","last_became_peered":"2023-07-13T22:22:19.689696+0000","last_unstale":"2023-07-13T22:22:20.794090+0000","last_undegraded":"2023-07-13T22:22:20.794090+0000","last_fullsized":"2023-07-13T22:22:20.794090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:34.008871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794236+0000","last_change":"2023-07-13T22:22:18.681618+0000","last_active":"2023-07-13T22:22:20.794236+0000","last_peered":"2023-07-13T22:22:20.794236+0000","last_clean":"2023-07-13T22:22:20.794236+0000","last_became_active":"2023-07-13T22:22:18.681315+0000","last_became_peered":"2023-07-13T22:22:18.681315+0000","last_unstale":"2023-07-13T22:22:20.794236+0000","last_undegraded":"2023-07-13T22:22:20.794236+0000","last_fullsized":"2023-07-13T22:22:20.794236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:21:34.358104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733185+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:22:20.733185+0000","last_peered":"2023-07-13T22:22:20.733185+0000","last_clean":"2023-07-13T22:22:20.733185+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:22:20.733185+0000","last_undegraded":"2023-07-13T22:22:20.733185+0000","last_fullsized":"2023-07-13T22:22:20.733185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:39:58.614283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":995328,"data_stored":750598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.754}]},{"osd":1,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.4710000000000001,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.62}]},{"osd":3,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":165831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.751},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.80200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":749054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.495}]},{"osd":3,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.629},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.75}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":157644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":11.792,"5min":11.792,"15min":11.792},"last":0.38900000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.443},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.99299999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":11.718999999999999,"5min":11.718999999999999,"15min":11.718999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.97399999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.679,"5min":11.679,"15min":11.679},"last":1.0329999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":11.756,"5min":11.756,"15min":11.756},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.0569999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":157680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.371},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":0.40699999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.78100000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.79600000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":166847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.31},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.373},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.495}]},{"osd":4,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.629}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":165853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.038},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.0580000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":10.657999999999999,"5min":10.657999999999999,"15min":10.657999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":8.75,"5min":8.75,"15min":8.75},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.89200000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":1.173},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":1.099}]},{"osd":7,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":1},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":1.212}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":158708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.629}]},{"osd":1,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:22:24.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:22:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.364+0000 7f59a1dff700 1 Processor -- start 2023-07-13T22:22:24.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.364+0000 7f59a1dff700 1 -- start start 2023-07-13T22:22:24.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.365+0000 7f59a1dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f599c0b8400 0x7f599c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.365+0000 7f59a1dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f599c0d3750 0x7f599c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.365+0000 7f59a1dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599c0d0ac0 con 0x7f599c0cfad0 2023-07-13T22:22:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.365+0000 7f59a1dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599c0cff50 con 0x7f599c0b8400 2023-07-13T22:22:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.365+0000 7f59a1dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599c0b87d0 con 0x7f599c0d3750 2023-07-13T22:22:24.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.366+0000 7f599b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f599c0b8400 0x7f599c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.366+0000 7f599bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f599c0cfad0 0x7f599c15fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34236/0 (socket says 172.21.15.60:34236) 2023-07-13T22:22:24.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.366+0000 7f599bfff700 1 -- 172.21.15.60:0/1691789312 learned_addr learned my addr 172.21.15.60:0/1691789312 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:24.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.366+0000 7f5993fff700 1 --2- 172.21.15.60:0/1691789312 >> v2:172.21.15.105:3300/0 conn(0x7f599c0d3750 0x7f599c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.366+0000 7f599b7fe700 1 -- 172.21.15.60:0/1691789312 >> v2:172.21.15.105:3300/0 conn(0x7f599c0d3750 msgr2=0x7f599c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:24.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.366+0000 7f599b7fe700 1 --2- 172.21.15.60:0/1691789312 >> v2:172.21.15.105:3300/0 conn(0x7f599c0d3750 0x7f599c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.366+0000 7f599b7fe700 1 -- 172.21.15.60:0/1691789312 >> v1:172.21.15.60:6789/0 conn(0x7f599c0cfad0 legacy=0x7f599c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.366+0000 7f599b7fe700 1 -- 172.21.15.60:0/1691789312 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c0b4eb0 con 0x7f599c0b8400 2023-07-13T22:22:24.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.366+0000 7f599b7fe700 1 --2- 172.21.15.60:0/1691789312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0b8400 0x7f599c0cd240 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f599c0b8cf0 tx=0x7f598c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842a5368dcece8d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:24.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f599affd700 1 -- 172.21.15.60:0/1691789312 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c011830 con 0x7f599c0b8400 2023-07-13T22:22:24.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f599affd700 1 -- 172.21.15.60:0/1691789312 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c011990 con 0x7f599c0b8400 2023-07-13T22:22:24.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f599affd700 1 -- 172.21.15.60:0/1691789312 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c011d20 con 0x7f599c0b8400 2023-07-13T22:22:24.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f59a1dff700 1 -- 172.21.15.60:0/1691789312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0b8400 msgr2=0x7f599c0cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f59a1dff700 1 --2- 172.21.15.60:0/1691789312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0b8400 0x7f599c0cd240 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f599c0b8cf0 tx=0x7f598c009b20 comp rx=0 tx=0).stop 2023-07-13T22:22:24.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f59a1dff700 1 -- 172.21.15.60:0/1691789312 shutdown_connections 2023-07-13T22:22:24.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f59a1dff700 1 --2- 172.21.15.60:0/1691789312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0b8400 0x7f599c0cd240 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f59a1dff700 1 --2- 172.21.15.60:0/1691789312 >> v2:172.21.15.105:3300/0 conn(0x7f599c0d3750 0x7f599c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f59a1dff700 1 -- 172.21.15.60:0/1691789312 >> 172.21.15.60:0/1691789312 conn(0x7f599c06b390 msgr2=0x7f599c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:24.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f59a1dff700 1 -- 172.21.15.60:0/1691789312 shutdown_connections 2023-07-13T22:22:24.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.367+0000 7f59a1dff700 1 -- 172.21.15.60:0/1691789312 wait complete. 2023-07-13T22:22:24.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.369+0000 7f59a1dff700 1 Processor -- start 2023-07-13T22:22:24.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.369+0000 7f59a1dff700 1 -- start start 2023-07-13T22:22:24.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.370+0000 7f59a1dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f599c0b8400 0x7f599c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.370+0000 7f59a1dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0cfad0 0x7f599c0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.370+0000 7f59a1dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f599c0d3750 0x7f599c0b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.370+0000 7f59a1dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0c7d90 con 0x7f599c0cfad0 2023-07-13T22:22:24.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.370+0000 7f59a1dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0c79d0 con 0x7f599c0d3750 2023-07-13T22:22:24.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.370+0000 7f59a1dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c154770 con 0x7f599c0b8400 2023-07-13T22:22:24.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.370+0000 7f5993fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0cfad0 0x7f599c0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.370+0000 7f5993fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0cfad0 0x7f599c0c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46874/0 (socket says 172.21.15.60:46874) 2023-07-13T22:22:24.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5993fff700 1 -- 172.21.15.60:0/3535277047 learned_addr learned my addr 172.21.15.60:0/3535277047 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:24.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f599bfff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f599c0d3750 0x7f599c0b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f599b7fe700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f599c0b8400 0x7f599c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5993fff700 1 -- 172.21.15.60:0/3535277047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f599c0b8400 msgr2=0x7f599c154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5993fff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f599c0b8400 0x7f599c154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5993fff700 1 -- 172.21.15.60:0/3535277047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f599c0d3750 msgr2=0x7f599c0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5993fff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f599c0d3750 0x7f599c0b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5993fff700 1 -- 172.21.15.60:0/3535277047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0094a0 con 0x7f599c0cfad0 2023-07-13T22:22:24.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5993fff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0cfad0 0x7f599c0c66b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f598400e840 tx=0x7f598400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5998ff9700 1 -- 172.21.15.60:0/3535277047 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984009590 con 0x7f599c0cfad0 2023-07-13T22:22:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5998ff9700 1 -- 172.21.15.60:0/3535277047 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5984013680 con 0x7f599c0cfad0 2023-07-13T22:22:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f5998ff9700 1 -- 172.21.15.60:0/3535277047 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984013a30 con 0x7f599c0cfad0 2023-07-13T22:22:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.371+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c0c7130 con 0x7f599c0cfad0 2023-07-13T22:22:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.372+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c0c76f0 con 0x7f599c0cfad0 2023-07-13T22:22:24.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.373+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c15b040 con 0x7f599c0cfad0 2023-07-13T22:22:24.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.374+0000 7f5998ff9700 1 -- 172.21.15.60:0/3535277047 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5984013b90 con 0x7f599c0cfad0 2023-07-13T22:22:24.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.374+0000 7f5998ff9700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f597c06ace0 0x7f597c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.375+0000 7f5998ff9700 1 -- 172.21.15.60:0/3535277047 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5984089430 con 0x7f599c0cfad0 2023-07-13T22:22:24.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.375+0000 7f599b7fe700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f597c06ace0 0x7f597c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.375+0000 7f599b7fe700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f597c06ace0 0x7f597c06d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f598c000c00 tx=0x7f598c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.379+0000 7f5998ff9700 1 -- 172.21.15.60:0/3535277047 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f59840544d0 con 0x7f599c0cfad0 2023-07-13T22:22:24.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.579+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f599c0b5190 con 0x7f597c06ace0 2023-07-13T22:22:24.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.588+0000 7f5998ff9700 1 -- 172.21.15.60:0/3535277047 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311537 (secure 0 0 0) 0x7f598c011b80 con 0x7f597c06ace0 2023-07-13T22:22:24.575 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:24.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f597c06ace0 msgr2=0x7f597c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f597c06ace0 0x7f597c06d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f598c000c00 tx=0x7f598c01c040 comp rx=0 tx=0).stop 2023-07-13T22:22:24.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0cfad0 msgr2=0x7f599c0c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0cfad0 0x7f599c0c66b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f598400e840 tx=0x7f598400ec00 comp rx=0 tx=0).stop 2023-07-13T22:22:24.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 shutdown_connections 2023-07-13T22:22:24.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f597c06ace0 0x7f597c06d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f599c0d3750 0x7f599c0b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f599c0b8400 0x7f599c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 --2- 172.21.15.60:0/3535277047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599c0cfad0 0x7f599c0c66b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 >> 172.21.15.60:0/3535277047 conn(0x7f599c06b390 msgr2=0x7f599c15bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:24.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 shutdown_connections 2023-07-13T22:22:24.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.592+0000 7f59a1dff700 1 -- 172.21.15.60:0/3535277047 wait complete. 2023-07-13T22:22:24.578 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:22:24.593 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-07-13T22:22:23.141685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219876,"kb_used_data":4260,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754844,"statfs":{"total":773094113280,"available":772868960256,"internally_reserved":0,"allocated":4362240,"data_stored":2472215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.483828"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047399+0000","last_change":"2023-07-13T22:22:19.692656+0000","last_active":"2023-07-13T22:22:21.047399+0000","last_peered":"2023-07-13T22:22:21.047399+0000","last_clean":"2023-07-13T22:22:21.047399+0000","last_became_active":"2023-07-13T22:22:19.692354+0000","last_became_peered":"2023-07-13T22:22:19.692354+0000","last_unstale":"2023-07-13T22:22:21.047399+0000","last_undegraded":"2023-07-13T22:22:21.047399+0000","last_fullsized":"2023-07-13T22:22:21.047399+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:59:30.476619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047077+0000","last_change":"2023-07-13T22:22:19.703610+0000","last_active":"2023-07-13T22:22:21.047077+0000","last_peered":"2023-07-13T22:22:21.047077+0000","last_clean":"2023-07-13T22:22:21.047077+0000","last_became_active":"2023-07-13T22:22:19.703345+0000","last_became_peered":"2023-07-13T22:22:19.703345+0000","last_unstale":"2023-07-13T22:22:21.047077+0000","last_undegraded":"2023-07-13T22:22:21.047077+0000","last_fullsized":"2023-07-13T22:22:21.047077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:19:05.780353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794280+0000","last_change":"2023-07-13T22:22:19.705602+0000","last_active":"2023-07-13T22:22:20.794280+0000","last_peered":"2023-07-13T22:22:20.794280+0000","last_clean":"2023-07-13T22:22:20.794280+0000","last_became_active":"2023-07-13T22:22:19.705203+0000","last_became_peered":"2023-07-13T22:22:19.705203+0000","last_unstale":"2023-07-13T22:22:20.794280+0000","last_undegraded":"2023-07-13T22:22:20.794280+0000","last_fullsized":"2023-07-13T22:22:20.794280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:54:11.424287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894637+0000","last_change":"2023-07-13T22:22:19.704528+0000","last_active":"2023-07-13T22:22:20.894637+0000","last_peered":"2023-07-13T22:22:20.894637+0000","last_clean":"2023-07-13T22:22:20.894637+0000","last_became_active":"2023-07-13T22:22:19.704008+0000","last_became_peered":"2023-07-13T22:22:19.704008+0000","last_unstale":"2023-07-13T22:22:20.894637+0000","last_undegraded":"2023-07-13T22:22:20.894637+0000","last_fullsized":"2023-07-13T22:22:20.894637+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:37:33.348454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894342+0000","last_change":"2023-07-13T22:22:19.698937+0000","last_active":"2023-07-13T22:22:20.894342+0000","last_peered":"2023-07-13T22:22:20.894342+0000","last_clean":"2023-07-13T22:22:20.894342+0000","last_became_active":"2023-07-13T22:22:19.698617+0000","last_became_peered":"2023-07-13T22:22:19.698617+0000","last_unstale":"2023-07-13T22:22:20.894342+0000","last_undegraded":"2023-07-13T22:22:20.894342+0000","last_fullsized":"2023-07-13T22:22:20.894342+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:22:21.825394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687200+0000","last_change":"2023-07-13T22:22:19.694903+0000","last_active":"2023-07-13T22:22:20.687200+0000","last_peered":"2023-07-13T22:22:20.687200+0000","last_clean":"2023-07-13T22:22:20.687200+0000","last_became_active":"2023-07-13T22:22:19.694525+0000","last_became_peered":"2023-07-13T22:22:19.694525+0000","last_unstale":"2023-07-13T22:22:20.687200+0000","last_undegraded":"2023-07-13T22:22:20.687200+0000","last_fullsized":"2023-07-13T22:22:20.687200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:51.327980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688617+0000","last_change":"2023-07-13T22:22:19.693765+0000","last_active":"2023-07-13T22:22:20.688617+0000","last_peered":"2023-07-13T22:22:20.688617+0000","last_clean":"2023-07-13T22:22:20.688617+0000","last_became_active":"2023-07-13T22:22:19.693521+0000","last_became_peered":"2023-07-13T22:22:19.693521+0000","last_unstale":"2023-07-13T22:22:20.688617+0000","last_undegraded":"2023-07-13T22:22:20.688617+0000","last_fullsized":"2023-07-13T22:22:20.688617+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:37:40.537525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046800+0000","last_change":"2023-07-13T22:22:19.696608+0000","last_active":"2023-07-13T22:22:21.046800+0000","last_peered":"2023-07-13T22:22:21.046800+0000","last_clean":"2023-07-13T22:22:21.046800+0000","last_became_active":"2023-07-13T22:22:19.696316+0000","last_became_peered":"2023-07-13T22:22:19.696316+0000","last_unstale":"2023-07-13T22:22:21.046800+0000","last_undegraded":"2023-07-13T22:22:21.046800+0000","last_fullsized":"2023-07-13T22:22:21.046800+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:13:47.499901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794491+0000","last_change":"2023-07-13T22:22:19.704019+0000","last_active":"2023-07-13T22:22:20.794491+0000","last_peered":"2023-07-13T22:22:20.794491+0000","last_clean":"2023-07-13T22:22:20.794491+0000","last_became_active":"2023-07-13T22:22:19.703749+0000","last_became_peered":"2023-07-13T22:22:19.703749+0000","last_unstale":"2023-07-13T22:22:20.794491+0000","last_undegraded":"2023-07-13T22:22:20.794491+0000","last_fullsized":"2023-07-13T22:22:20.794491+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:53:30.822156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689408+0000","last_change":"2023-07-13T22:22:19.694418+0000","last_active":"2023-07-13T22:22:20.689408+0000","last_peered":"2023-07-13T22:22:20.689408+0000","last_clean":"2023-07-13T22:22:20.689408+0000","last_became_active":"2023-07-13T22:22:19.694012+0000","last_became_peered":"2023-07-13T22:22:19.694012+0000","last_unstale":"2023-07-13T22:22:20.689408+0000","last_undegraded":"2023-07-13T22:22:20.689408+0000","last_fullsized":"2023-07-13T22:22:20.689408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:36:15.663442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689974+0000","last_change":"2023-07-13T22:22:19.704107+0000","last_active":"2023-07-13T22:22:20.689974+0000","last_peered":"2023-07-13T22:22:20.689974+0000","last_clean":"2023-07-13T22:22:20.689974+0000","last_became_active":"2023-07-13T22:22:19.702838+0000","last_became_peered":"2023-07-13T22:22:19.702838+0000","last_unstale":"2023-07-13T22:22:20.689974+0000","last_undegraded":"2023-07-13T22:22:20.689974+0000","last_fullsized":"2023-07-13T22:22:20.689974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:59.864501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794193+0000","last_change":"2023-07-13T22:22:19.705496+0000","last_active":"2023-07-13T22:22:20.794193+0000","last_peered":"2023-07-13T22:22:20.794193+0000","last_clean":"2023-07-13T22:22:20.794193+0000","last_became_active":"2023-07-13T22:22:19.705020+0000","last_became_peered":"2023-07-13T22:22:19.705020+0000","last_unstale":"2023-07-13T22:22:20.794193+0000","last_undegraded":"2023-07-13T22:22:20.794193+0000","last_fullsized":"2023-07-13T22:22:20.794193+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:07.299044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689017+0000","last_change":"2023-07-13T22:22:19.698140+0000","last_active":"2023-07-13T22:22:20.689017+0000","last_peered":"2023-07-13T22:22:20.689017+0000","last_clean":"2023-07-13T22:22:20.689017+0000","last_became_active":"2023-07-13T22:22:19.697454+0000","last_became_peered":"2023-07-13T22:22:19.697454+0000","last_unstale":"2023-07-13T22:22:20.689017+0000","last_undegraded":"2023-07-13T22:22:20.689017+0000","last_fullsized":"2023-07-13T22:22:20.689017+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:38:07.942207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689594+0000","last_change":"2023-07-13T22:22:19.705141+0000","last_active":"2023-07-13T22:22:20.689594+0000","last_peered":"2023-07-13T22:22:20.689594+0000","last_clean":"2023-07-13T22:22:20.689594+0000","last_became_active":"2023-07-13T22:22:19.704822+0000","last_became_peered":"2023-07-13T22:22:19.704822+0000","last_unstale":"2023-07-13T22:22:20.689594+0000","last_undegraded":"2023-07-13T22:22:20.689594+0000","last_fullsized":"2023-07-13T22:22:20.689594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:07:19.584493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688353+0000","last_change":"2023-07-13T22:22:19.703860+0000","last_active":"2023-07-13T22:22:20.688353+0000","last_peered":"2023-07-13T22:22:20.688353+0000","last_clean":"2023-07-13T22:22:20.688353+0000","last_became_active":"2023-07-13T22:22:19.703574+0000","last_became_peered":"2023-07-13T22:22:19.703574+0000","last_unstale":"2023-07-13T22:22:20.688353+0000","last_undegraded":"2023-07-13T22:22:20.688353+0000","last_fullsized":"2023-07-13T22:22:20.688353+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:46:23.714214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046473+0000","last_change":"2023-07-13T22:22:19.692840+0000","last_active":"2023-07-13T22:22:21.046473+0000","last_peered":"2023-07-13T22:22:21.046473+0000","last_clean":"2023-07-13T22:22:21.046473+0000","last_became_active":"2023-07-13T22:22:19.692577+0000","last_became_peered":"2023-07-13T22:22:19.692577+0000","last_unstale":"2023-07-13T22:22:21.046473+0000","last_undegraded":"2023-07-13T22:22:21.046473+0000","last_fullsized":"2023-07-13T22:22:21.046473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:32.666494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046997+0000","last_change":"2023-07-13T22:22:19.694284+0000","last_active":"2023-07-13T22:22:21.046997+0000","last_peered":"2023-07-13T22:22:21.046997+0000","last_clean":"2023-07-13T22:22:21.046997+0000","last_became_active":"2023-07-13T22:22:19.694006+0000","last_became_peered":"2023-07-13T22:22:19.694006+0000","last_unstale":"2023-07-13T22:22:21.046997+0000","last_undegraded":"2023-07-13T22:22:21.046997+0000","last_fullsized":"2023-07-13T22:22:21.046997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:13:47.499901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733170+0000","last_change":"2023-07-13T22:22:19.702652+0000","last_active":"2023-07-13T22:22:20.733170+0000","last_peered":"2023-07-13T22:22:20.733170+0000","last_clean":"2023-07-13T22:22:20.733170+0000","last_became_active":"2023-07-13T22:22:19.702286+0000","last_became_peered":"2023-07-13T22:22:19.702286+0000","last_unstale":"2023-07-13T22:22:20.733170+0000","last_undegraded":"2023-07-13T22:22:20.733170+0000","last_fullsized":"2023-07-13T22:22:20.733170+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:01:51.003257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894591+0000","last_change":"2023-07-13T22:22:19.708353+0000","last_active":"2023-07-13T22:22:20.894591+0000","last_peered":"2023-07-13T22:22:20.894591+0000","last_clean":"2023-07-13T22:22:20.894591+0000","last_became_active":"2023-07-13T22:22:19.704536+0000","last_became_peered":"2023-07-13T22:22:19.704536+0000","last_unstale":"2023-07-13T22:22:20.894591+0000","last_undegraded":"2023-07-13T22:22:20.894591+0000","last_fullsized":"2023-07-13T22:22:20.894591+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:07:05.817546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.732958+0000","last_change":"2023-07-13T22:22:19.706285+0000","last_active":"2023-07-13T22:22:20.732958+0000","last_peered":"2023-07-13T22:22:20.732958+0000","last_clean":"2023-07-13T22:22:20.732958+0000","last_became_active":"2023-07-13T22:22:19.705988+0000","last_became_peered":"2023-07-13T22:22:19.705988+0000","last_unstale":"2023-07-13T22:22:20.732958+0000","last_undegraded":"2023-07-13T22:22:20.732958+0000","last_fullsized":"2023-07-13T22:22:20.732958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:55:08.308467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689179+0000","last_change":"2023-07-13T22:22:19.698367+0000","last_active":"2023-07-13T22:22:20.689179+0000","last_peered":"2023-07-13T22:22:20.689179+0000","last_clean":"2023-07-13T22:22:20.689179+0000","last_became_active":"2023-07-13T22:22:19.697805+0000","last_became_peered":"2023-07-13T22:22:19.697805+0000","last_unstale":"2023-07-13T22:22:20.689179+0000","last_undegraded":"2023-07-13T22:22:20.689179+0000","last_fullsized":"2023-07-13T22:22:20.689179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:38:48.583259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689020+0000","last_change":"2023-07-13T22:22:19.703882+0000","last_active":"2023-07-13T22:22:20.689020+0000","last_peered":"2023-07-13T22:22:20.689020+0000","last_clean":"2023-07-13T22:22:20.689020+0000","last_became_active":"2023-07-13T22:22:19.703616+0000","last_became_peered":"2023-07-13T22:22:19.703616+0000","last_unstale":"2023-07-13T22:22:20.689020+0000","last_undegraded":"2023-07-13T22:22:20.689020+0000","last_fullsized":"2023-07-13T22:22:20.689020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:15:37.062173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047162+0000","last_change":"2023-07-13T22:22:19.703128+0000","last_active":"2023-07-13T22:22:21.047162+0000","last_peered":"2023-07-13T22:22:21.047162+0000","last_clean":"2023-07-13T22:22:21.047162+0000","last_became_active":"2023-07-13T22:22:19.702526+0000","last_became_peered":"2023-07-13T22:22:19.702526+0000","last_unstale":"2023-07-13T22:22:21.047162+0000","last_undegraded":"2023-07-13T22:22:21.047162+0000","last_fullsized":"2023-07-13T22:22:21.047162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:44:59.667785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894316+0000","last_change":"2023-07-13T22:22:19.700742+0000","last_active":"2023-07-13T22:22:20.894316+0000","last_peered":"2023-07-13T22:22:20.894316+0000","last_clean":"2023-07-13T22:22:20.894316+0000","last_became_active":"2023-07-13T22:22:19.700307+0000","last_became_peered":"2023-07-13T22:22:19.700307+0000","last_unstale":"2023-07-13T22:22:20.894316+0000","last_undegraded":"2023-07-13T22:22:20.894316+0000","last_fullsized":"2023-07-13T22:22:20.894316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:26:12.845198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733078+0000","last_change":"2023-07-13T22:22:19.694741+0000","last_active":"2023-07-13T22:22:20.733078+0000","last_peered":"2023-07-13T22:22:20.733078+0000","last_clean":"2023-07-13T22:22:20.733078+0000","last_became_active":"2023-07-13T22:22:19.694471+0000","last_became_peered":"2023-07-13T22:22:19.694471+0000","last_unstale":"2023-07-13T22:22:20.733078+0000","last_undegraded":"2023-07-13T22:22:20.733078+0000","last_fullsized":"2023-07-13T22:22:20.733078+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:10:13.478680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687617+0000","last_change":"2023-07-13T22:22:19.700424+0000","last_active":"2023-07-13T22:22:20.687617+0000","last_peered":"2023-07-13T22:22:20.687617+0000","last_clean":"2023-07-13T22:22:20.687617+0000","last_became_active":"2023-07-13T22:22:19.699831+0000","last_became_peered":"2023-07-13T22:22:19.699831+0000","last_unstale":"2023-07-13T22:22:20.687617+0000","last_undegraded":"2023-07-13T22:22:20.687617+0000","last_fullsized":"2023-07-13T22:22:20.687617+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:05:35.997727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733168+0000","last_change":"2023-07-13T22:22:19.698503+0000","last_active":"2023-07-13T22:22:20.733168+0000","last_peered":"2023-07-13T22:22:20.733168+0000","last_clean":"2023-07-13T22:22:20.733168+0000","last_became_active":"2023-07-13T22:22:19.698203+0000","last_became_peered":"2023-07-13T22:22:19.698203+0000","last_unstale":"2023-07-13T22:22:20.733168+0000","last_undegraded":"2023-07-13T22:22:20.733168+0000","last_fullsized":"2023-07-13T22:22:20.733168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:11:24.335253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689408+0000","last_change":"2023-07-13T22:22:19.691385+0000","last_active":"2023-07-13T22:22:20.689408+0000","last_peered":"2023-07-13T22:22:20.689408+0000","last_clean":"2023-07-13T22:22:20.689408+0000","last_became_active":"2023-07-13T22:22:19.690026+0000","last_became_peered":"2023-07-13T22:22:19.690026+0000","last_unstale":"2023-07-13T22:22:20.689408+0000","last_undegraded":"2023-07-13T22:22:20.689408+0000","last_fullsized":"2023-07-13T22:22:20.689408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:01:42.811942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733097+0000","last_change":"2023-07-13T22:22:19.700492+0000","last_active":"2023-07-13T22:22:20.733097+0000","last_peered":"2023-07-13T22:22:20.733097+0000","last_clean":"2023-07-13T22:22:20.733097+0000","last_became_active":"2023-07-13T22:22:19.700239+0000","last_became_peered":"2023-07-13T22:22:19.700239+0000","last_unstale":"2023-07-13T22:22:20.733097+0000","last_undegraded":"2023-07-13T22:22:20.733097+0000","last_fullsized":"2023-07-13T22:22:20.733097+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:24:18.621969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.732775+0000","last_change":"2023-07-13T22:22:19.680540+0000","last_active":"2023-07-13T22:22:20.732775+0000","last_peered":"2023-07-13T22:22:20.732775+0000","last_clean":"2023-07-13T22:22:20.732775+0000","last_became_active":"2023-07-13T22:22:19.680275+0000","last_became_peered":"2023-07-13T22:22:19.680275+0000","last_unstale":"2023-07-13T22:22:20.732775+0000","last_undegraded":"2023-07-13T22:22:20.732775+0000","last_fullsized":"2023-07-13T22:22:20.732775+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:06:19.287774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046628+0000","last_change":"2023-07-13T22:22:19.703295+0000","last_active":"2023-07-13T22:22:21.046628+0000","last_peered":"2023-07-13T22:22:21.046628+0000","last_clean":"2023-07-13T22:22:21.046628+0000","last_became_active":"2023-07-13T22:22:19.702866+0000","last_became_peered":"2023-07-13T22:22:19.702866+0000","last_unstale":"2023-07-13T22:22:21.046628+0000","last_undegraded":"2023-07-13T22:22:21.046628+0000","last_fullsized":"2023-07-13T22:22:21.046628+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:09:26.260912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687051+0000","last_change":"2023-07-13T22:22:19.704318+0000","last_active":"2023-07-13T22:22:20.687051+0000","last_peered":"2023-07-13T22:22:20.687051+0000","last_clean":"2023-07-13T22:22:20.687051+0000","last_became_active":"2023-07-13T22:22:19.703828+0000","last_became_peered":"2023-07-13T22:22:19.703828+0000","last_unstale":"2023-07-13T22:22:20.687051+0000","last_undegraded":"2023-07-13T22:22:20.687051+0000","last_fullsized":"2023-07-13T22:22:20.687051+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:56:44.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046791+0000","last_change":"2023-07-13T22:22:19.689928+0000","last_active":"2023-07-13T22:22:21.046791+0000","last_peered":"2023-07-13T22:22:21.046791+0000","last_clean":"2023-07-13T22:22:21.046791+0000","last_became_active":"2023-07-13T22:22:19.689662+0000","last_became_peered":"2023-07-13T22:22:19.689662+0000","last_unstale":"2023-07-13T22:22:21.046791+0000","last_undegraded":"2023-07-13T22:22:21.046791+0000","last_fullsized":"2023-07-13T22:22:21.046791+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:43:58.934620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733242+0000","last_change":"2023-07-13T22:22:18.683795+0000","last_active":"2023-07-13T22:22:20.733242+0000","last_peered":"2023-07-13T22:22:20.733242+0000","last_clean":"2023-07-13T22:22:20.733242+0000","last_became_active":"2023-07-13T22:22:18.683513+0000","last_became_peered":"2023-07-13T22:22:18.683513+0000","last_unstale":"2023-07-13T22:22:20.733242+0000","last_undegraded":"2023-07-13T22:22:20.733242+0000","last_fullsized":"2023-07-13T22:22:20.733242+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:56:10.492367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687480+0000","last_change":"2023-07-13T22:22:19.699638+0000","last_active":"2023-07-13T22:22:20.687480+0000","last_peered":"2023-07-13T22:22:20.687480+0000","last_clean":"2023-07-13T22:22:20.687480+0000","last_became_active":"2023-07-13T22:22:19.698969+0000","last_became_peered":"2023-07-13T22:22:19.698969+0000","last_unstale":"2023-07-13T22:22:20.687480+0000","last_undegraded":"2023-07-13T22:22:20.687480+0000","last_fullsized":"2023-07-13T22:22:20.687480+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:54:11.424287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794426+0000","last_change":"2023-07-13T22:22:18.688079+0000","last_active":"2023-07-13T22:22:20.794426+0000","last_peered":"2023-07-13T22:22:20.794426+0000","last_clean":"2023-07-13T22:22:20.794426+0000","last_became_active":"2023-07-13T22:22:18.687780+0000","last_became_peered":"2023-07-13T22:22:18.687780+0000","last_unstale":"2023-07-13T22:22:20.794426+0000","last_undegraded":"2023-07-13T22:22:20.794426+0000","last_fullsized":"2023-07-13T22:22:20.794426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:07.214702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689472+0000","last_change":"2023-07-13T22:22:19.706038+0000","last_active":"2023-07-13T22:22:20.689472+0000","last_peered":"2023-07-13T22:22:20.689472+0000","last_clean":"2023-07-13T22:22:20.689472+0000","last_became_active":"2023-07-13T22:22:19.705276+0000","last_became_peered":"2023-07-13T22:22:19.705276+0000","last_unstale":"2023-07-13T22:22:20.689472+0000","last_undegraded":"2023-07-13T22:22:20.689472+0000","last_fullsized":"2023-07-13T22:22:20.689472+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:37:55.620973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687570+0000","last_change":"2023-07-13T22:22:18.683090+0000","last_active":"2023-07-13T22:22:20.687570+0000","last_peered":"2023-07-13T22:22:20.687570+0000","last_clean":"2023-07-13T22:22:20.687570+0000","last_became_active":"2023-07-13T22:22:18.682835+0000","last_became_peered":"2023-07-13T22:22:18.682835+0000","last_unstale":"2023-07-13T22:22:20.687570+0000","last_undegraded":"2023-07-13T22:22:20.687570+0000","last_fullsized":"2023-07-13T22:22:20.687570+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:06:59.391553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047224+0000","last_change":"2023-07-13T22:22:19.682220+0000","last_active":"2023-07-13T22:22:21.047224+0000","last_peered":"2023-07-13T22:22:21.047224+0000","last_clean":"2023-07-13T22:22:21.047224+0000","last_became_active":"2023-07-13T22:22:19.681376+0000","last_became_peered":"2023-07-13T22:22:19.681376+0000","last_unstale":"2023-07-13T22:22:21.047224+0000","last_undegraded":"2023-07-13T22:22:21.047224+0000","last_fullsized":"2023-07-13T22:22:21.047224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:22.748465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689165+0000","last_change":"2023-07-13T22:22:18.682480+0000","last_active":"2023-07-13T22:22:20.689165+0000","last_peered":"2023-07-13T22:22:20.689165+0000","last_clean":"2023-07-13T22:22:20.689165+0000","last_became_active":"2023-07-13T22:22:18.682093+0000","last_became_peered":"2023-07-13T22:22:18.682093+0000","last_unstale":"2023-07-13T22:22:20.689165+0000","last_undegraded":"2023-07-13T22:22:20.689165+0000","last_fullsized":"2023-07-13T22:22:20.689165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:28:59.822612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794018+0000","last_change":"2023-07-13T22:22:19.694289+0000","last_active":"2023-07-13T22:22:20.794018+0000","last_peered":"2023-07-13T22:22:20.794018+0000","last_clean":"2023-07-13T22:22:20.794018+0000","last_became_active":"2023-07-13T22:22:19.693972+0000","last_became_peered":"2023-07-13T22:22:19.693972+0000","last_unstale":"2023-07-13T22:22:20.794018+0000","last_undegraded":"2023-07-13T22:22:20.794018+0000","last_fullsized":"2023-07-13T22:22:20.794018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:51.327980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689146+0000","last_change":"2023-07-13T22:22:18.680139+0000","last_active":"2023-07-13T22:22:20.689146+0000","last_peered":"2023-07-13T22:22:20.689146+0000","last_clean":"2023-07-13T22:22:20.689146+0000","last_became_active":"2023-07-13T22:22:18.679855+0000","last_became_peered":"2023-07-13T22:22:18.679855+0000","last_unstale":"2023-07-13T22:22:20.689146+0000","last_undegraded":"2023-07-13T22:22:20.689146+0000","last_fullsized":"2023-07-13T22:22:20.689146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:40:01.218615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689817+0000","last_change":"2023-07-13T22:22:19.701678+0000","last_active":"2023-07-13T22:22:20.689817+0000","last_peered":"2023-07-13T22:22:20.689817+0000","last_clean":"2023-07-13T22:22:20.689817+0000","last_became_active":"2023-07-13T22:22:19.696615+0000","last_became_peered":"2023-07-13T22:22:19.696615+0000","last_unstale":"2023-07-13T22:22:20.689817+0000","last_undegraded":"2023-07-13T22:22:20.689817+0000","last_fullsized":"2023-07-13T22:22:20.689817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:48:09.116672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047078+0000","last_change":"2023-07-13T22:22:18.684907+0000","last_active":"2023-07-13T22:22:21.047078+0000","last_peered":"2023-07-13T22:22:21.047078+0000","last_clean":"2023-07-13T22:22:21.047078+0000","last_became_active":"2023-07-13T22:22:18.684571+0000","last_became_peered":"2023-07-13T22:22:18.684571+0000","last_unstale":"2023-07-13T22:22:21.047078+0000","last_undegraded":"2023-07-13T22:22:21.047078+0000","last_fullsized":"2023-07-13T22:22:21.047078+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:38:20.355956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689188+0000","last_change":"2023-07-13T22:22:19.709929+0000","last_active":"2023-07-13T22:22:20.689188+0000","last_peered":"2023-07-13T22:22:20.689188+0000","last_clean":"2023-07-13T22:22:20.689188+0000","last_became_active":"2023-07-13T22:22:19.709626+0000","last_became_peered":"2023-07-13T22:22:19.709626+0000","last_unstale":"2023-07-13T22:22:20.689188+0000","last_undegraded":"2023-07-13T22:22:20.689188+0000","last_fullsized":"2023-07-13T22:22:20.689188+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:56:59.213235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687114+0000","last_change":"2023-07-13T22:22:18.680505+0000","last_active":"2023-07-13T22:22:20.687114+0000","last_peered":"2023-07-13T22:22:20.687114+0000","last_clean":"2023-07-13T22:22:20.687114+0000","last_became_active":"2023-07-13T22:22:18.680200+0000","last_became_peered":"2023-07-13T22:22:18.680200+0000","last_unstale":"2023-07-13T22:22:20.687114+0000","last_undegraded":"2023-07-13T22:22:20.687114+0000","last_fullsized":"2023-07-13T22:22:20.687114+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:13:23.676059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047261+0000","last_change":"2023-07-13T22:22:19.682712+0000","last_active":"2023-07-13T22:22:21.047261+0000","last_peered":"2023-07-13T22:22:21.047261+0000","last_clean":"2023-07-13T22:22:21.047261+0000","last_became_active":"2023-07-13T22:22:19.682379+0000","last_became_peered":"2023-07-13T22:22:19.682379+0000","last_unstale":"2023-07-13T22:22:21.047261+0000","last_undegraded":"2023-07-13T22:22:21.047261+0000","last_fullsized":"2023-07-13T22:22:21.047261+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:06.318949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688871+0000","last_change":"2023-07-13T22:22:18.681506+0000","last_active":"2023-07-13T22:22:20.688871+0000","last_peered":"2023-07-13T22:22:20.688871+0000","last_clean":"2023-07-13T22:22:20.688871+0000","last_became_active":"2023-07-13T22:22:18.681195+0000","last_became_peered":"2023-07-13T22:22:18.681195+0000","last_unstale":"2023-07-13T22:22:20.688871+0000","last_undegraded":"2023-07-13T22:22:20.688871+0000","last_fullsized":"2023-07-13T22:22:20.688871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:20.385029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894635+0000","last_change":"2023-07-13T22:22:19.704324+0000","last_active":"2023-07-13T22:22:20.894635+0000","last_peered":"2023-07-13T22:22:20.894635+0000","last_clean":"2023-07-13T22:22:20.894635+0000","last_became_active":"2023-07-13T22:22:19.703616+0000","last_became_peered":"2023-07-13T22:22:19.703616+0000","last_unstale":"2023-07-13T22:22:20.894635+0000","last_undegraded":"2023-07-13T22:22:20.894635+0000","last_fullsized":"2023-07-13T22:22:20.894635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:10:45.045964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047329+0000","last_change":"2023-07-13T22:22:18.667612+0000","last_active":"2023-07-13T22:22:21.047329+0000","last_peered":"2023-07-13T22:22:21.047329+0000","last_clean":"2023-07-13T22:22:21.047329+0000","last_became_active":"2023-07-13T22:22:18.667270+0000","last_became_peered":"2023-07-13T22:22:18.667270+0000","last_unstale":"2023-07-13T22:22:21.047329+0000","last_undegraded":"2023-07-13T22:22:21.047329+0000","last_fullsized":"2023-07-13T22:22:21.047329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:15:57.132576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047057+0000","last_change":"2023-07-13T22:22:19.685896+0000","last_active":"2023-07-13T22:22:21.047057+0000","last_peered":"2023-07-13T22:22:21.047057+0000","last_clean":"2023-07-13T22:22:21.047057+0000","last_became_active":"2023-07-13T22:22:19.685221+0000","last_became_peered":"2023-07-13T22:22:19.685221+0000","last_unstale":"2023-07-13T22:22:21.047057+0000","last_undegraded":"2023-07-13T22:22:21.047057+0000","last_fullsized":"2023-07-13T22:22:21.047057+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:31:13.446473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689715+0000","last_change":"2023-07-13T22:22:18.679413+0000","last_active":"2023-07-13T22:22:20.689715+0000","last_peered":"2023-07-13T22:22:20.689715+0000","last_clean":"2023-07-13T22:22:20.689715+0000","last_became_active":"2023-07-13T22:22:18.679111+0000","last_became_peered":"2023-07-13T22:22:18.679111+0000","last_unstale":"2023-07-13T22:22:20.689715+0000","last_undegraded":"2023-07-13T22:22:20.689715+0000","last_fullsized":"2023-07-13T22:22:20.689715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:41:45.164905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794403+0000","last_change":"2023-07-13T22:22:19.697191+0000","last_active":"2023-07-13T22:22:20.794403+0000","last_peered":"2023-07-13T22:22:20.794403+0000","last_clean":"2023-07-13T22:22:20.794403+0000","last_became_active":"2023-07-13T22:22:19.696913+0000","last_became_peered":"2023-07-13T22:22:19.696913+0000","last_unstale":"2023-07-13T22:22:20.794403+0000","last_undegraded":"2023-07-13T22:22:20.794403+0000","last_fullsized":"2023-07-13T22:22:20.794403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:05:35.997727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.690056+0000","last_change":"2023-07-13T22:22:18.679915+0000","last_active":"2023-07-13T22:22:20.690056+0000","last_peered":"2023-07-13T22:22:20.690056+0000","last_clean":"2023-07-13T22:22:20.690056+0000","last_became_active":"2023-07-13T22:22:18.679608+0000","last_became_peered":"2023-07-13T22:22:18.679608+0000","last_unstale":"2023-07-13T22:22:20.690056+0000","last_undegraded":"2023-07-13T22:22:20.690056+0000","last_fullsized":"2023-07-13T22:22:20.690056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:31.660354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689121+0000","last_change":"2023-07-13T22:22:19.704021+0000","last_active":"2023-07-13T22:22:20.689121+0000","last_peered":"2023-07-13T22:22:20.689121+0000","last_clean":"2023-07-13T22:22:20.689121+0000","last_became_active":"2023-07-13T22:22:19.703765+0000","last_became_peered":"2023-07-13T22:22:19.703765+0000","last_unstale":"2023-07-13T22:22:20.689121+0000","last_undegraded":"2023-07-13T22:22:20.689121+0000","last_fullsized":"2023-07-13T22:22:20.689121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:40:43.964736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047094+0000","last_change":"2023-07-13T22:22:18.688250+0000","last_active":"2023-07-13T22:22:21.047094+0000","last_peered":"2023-07-13T22:22:21.047094+0000","last_clean":"2023-07-13T22:22:21.047094+0000","last_became_active":"2023-07-13T22:22:18.687919+0000","last_became_peered":"2023-07-13T22:22:18.687919+0000","last_unstale":"2023-07-13T22:22:21.047094+0000","last_undegraded":"2023-07-13T22:22:21.047094+0000","last_fullsized":"2023-07-13T22:22:21.047094+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:05:16.510489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794460+0000","last_change":"2023-07-13T22:22:19.705213+0000","last_active":"2023-07-13T22:22:20.794460+0000","last_peered":"2023-07-13T22:22:20.794460+0000","last_clean":"2023-07-13T22:22:20.794460+0000","last_became_active":"2023-07-13T22:22:19.704914+0000","last_became_peered":"2023-07-13T22:22:19.704914+0000","last_unstale":"2023-07-13T22:22:20.794460+0000","last_undegraded":"2023-07-13T22:22:20.794460+0000","last_fullsized":"2023-07-13T22:22:20.794460+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:39:50.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047399+0000","last_change":"2023-07-13T22:22:18.677934+0000","last_active":"2023-07-13T22:22:21.047399+0000","last_peered":"2023-07-13T22:22:21.047399+0000","last_clean":"2023-07-13T22:22:21.047399+0000","last_became_active":"2023-07-13T22:22:18.677635+0000","last_became_peered":"2023-07-13T22:22:18.677635+0000","last_unstale":"2023-07-13T22:22:21.047399+0000","last_undegraded":"2023-07-13T22:22:21.047399+0000","last_fullsized":"2023-07-13T22:22:21.047399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:28:38.856054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046877+0000","last_change":"2023-07-13T22:22:19.693402+0000","last_active":"2023-07-13T22:22:21.046877+0000","last_peered":"2023-07-13T22:22:21.046877+0000","last_clean":"2023-07-13T22:22:21.046877+0000","last_became_active":"2023-07-13T22:22:19.693100+0000","last_became_peered":"2023-07-13T22:22:19.693100+0000","last_unstale":"2023-07-13T22:22:21.046877+0000","last_undegraded":"2023-07-13T22:22:21.046877+0000","last_fullsized":"2023-07-13T22:22:21.046877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:38:21.362096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687311+0000","last_change":"2023-07-13T22:22:18.680682+0000","last_active":"2023-07-13T22:22:20.687311+0000","last_peered":"2023-07-13T22:22:20.687311+0000","last_clean":"2023-07-13T22:22:20.687311+0000","last_became_active":"2023-07-13T22:22:18.680290+0000","last_became_peered":"2023-07-13T22:22:18.680290+0000","last_unstale":"2023-07-13T22:22:20.687311+0000","last_undegraded":"2023-07-13T22:22:20.687311+0000","last_fullsized":"2023-07-13T22:22:20.687311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:06.292904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046669+0000","last_change":"2023-07-13T22:22:19.702221+0000","last_active":"2023-07-13T22:22:21.046669+0000","last_peered":"2023-07-13T22:22:21.046669+0000","last_clean":"2023-07-13T22:22:21.046669+0000","last_became_active":"2023-07-13T22:22:19.701927+0000","last_became_peered":"2023-07-13T22:22:19.701927+0000","last_unstale":"2023-07-13T22:22:21.046669+0000","last_undegraded":"2023-07-13T22:22:21.046669+0000","last_fullsized":"2023-07-13T22:22:21.046669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:03:40.805354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.732749+0000","last_change":"2023-07-13T22:22:18.683147+0000","last_active":"2023-07-13T22:22:20.732749+0000","last_peered":"2023-07-13T22:22:20.732749+0000","last_clean":"2023-07-13T22:22:20.732749+0000","last_became_active":"2023-07-13T22:22:18.682827+0000","last_became_peered":"2023-07-13T22:22:18.682827+0000","last_unstale":"2023-07-13T22:22:20.732749+0000","last_undegraded":"2023-07-13T22:22:20.732749+0000","last_fullsized":"2023-07-13T22:22:20.732749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:58:10.746801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047233+0000","last_change":"2023-07-13T22:22:19.694063+0000","last_active":"2023-07-13T22:22:21.047233+0000","last_peered":"2023-07-13T22:22:21.047233+0000","last_clean":"2023-07-13T22:22:21.047233+0000","last_became_active":"2023-07-13T22:22:19.693779+0000","last_became_peered":"2023-07-13T22:22:19.693779+0000","last_unstale":"2023-07-13T22:22:21.047233+0000","last_undegraded":"2023-07-13T22:22:21.047233+0000","last_fullsized":"2023-07-13T22:22:21.047233+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:59:28.209710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046667+0000","last_change":"2023-07-13T22:22:18.688049+0000","last_active":"2023-07-13T22:22:21.046667+0000","last_peered":"2023-07-13T22:22:21.046667+0000","last_clean":"2023-07-13T22:22:21.046667+0000","last_became_active":"2023-07-13T22:22:18.687795+0000","last_became_peered":"2023-07-13T22:22:18.687795+0000","last_unstale":"2023-07-13T22:22:21.046667+0000","last_undegraded":"2023-07-13T22:22:21.046667+0000","last_fullsized":"2023-07-13T22:22:21.046667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:29:57.085734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047172+0000","last_change":"2023-07-13T22:22:19.689345+0000","last_active":"2023-07-13T22:22:21.047172+0000","last_peered":"2023-07-13T22:22:21.047172+0000","last_clean":"2023-07-13T22:22:21.047172+0000","last_became_active":"2023-07-13T22:22:19.689067+0000","last_became_peered":"2023-07-13T22:22:19.689067+0000","last_unstale":"2023-07-13T22:22:21.047172+0000","last_undegraded":"2023-07-13T22:22:21.047172+0000","last_fullsized":"2023-07-13T22:22:21.047172+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:29:32.779598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689222+0000","last_change":"2023-07-13T22:22:18.683350+0000","last_active":"2023-07-13T22:22:20.689222+0000","last_peered":"2023-07-13T22:22:20.689222+0000","last_clean":"2023-07-13T22:22:20.689222+0000","last_became_active":"2023-07-13T22:22:18.683053+0000","last_became_peered":"2023-07-13T22:22:18.683053+0000","last_unstale":"2023-07-13T22:22:20.689222+0000","last_undegraded":"2023-07-13T22:22:20.689222+0000","last_fullsized":"2023-07-13T22:22:20.689222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:05:35.442793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.690167+0000","last_change":"2023-07-13T22:22:19.701641+0000","last_active":"2023-07-13T22:22:20.690167+0000","last_peered":"2023-07-13T22:22:20.690167+0000","last_clean":"2023-07-13T22:22:20.690167+0000","last_became_active":"2023-07-13T22:22:19.701196+0000","last_became_peered":"2023-07-13T22:22:19.701196+0000","last_unstale":"2023-07-13T22:22:20.690167+0000","last_undegraded":"2023-07-13T22:22:20.690167+0000","last_fullsized":"2023-07-13T22:22:20.690167+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:12:34.777446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689931+0000","last_change":"2023-07-13T22:22:18.683089+0000","last_active":"2023-07-13T22:22:20.689931+0000","last_peered":"2023-07-13T22:22:20.689931+0000","last_clean":"2023-07-13T22:22:20.689931+0000","last_became_active":"2023-07-13T22:22:18.682735+0000","last_became_peered":"2023-07-13T22:22:18.682735+0000","last_unstale":"2023-07-13T22:22:20.689931+0000","last_undegraded":"2023-07-13T22:22:20.689931+0000","last_fullsized":"2023-07-13T22:22:20.689931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:46:06.117007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689635+0000","last_change":"2023-07-13T22:22:19.705671+0000","last_active":"2023-07-13T22:22:20.689635+0000","last_peered":"2023-07-13T22:22:20.689635+0000","last_clean":"2023-07-13T22:22:20.689635+0000","last_became_active":"2023-07-13T22:22:19.704886+0000","last_became_peered":"2023-07-13T22:22:19.704886+0000","last_unstale":"2023-07-13T22:22:20.689635+0000","last_undegraded":"2023-07-13T22:22:20.689635+0000","last_fullsized":"2023-07-13T22:22:20.689635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:32:53.170164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894534+0000","last_change":"2023-07-13T22:22:18.687950+0000","last_active":"2023-07-13T22:22:20.894534+0000","last_peered":"2023-07-13T22:22:20.894534+0000","last_clean":"2023-07-13T22:22:20.894534+0000","last_became_active":"2023-07-13T22:22:18.687656+0000","last_became_peered":"2023-07-13T22:22:18.687656+0000","last_unstale":"2023-07-13T22:22:20.894534+0000","last_undegraded":"2023-07-13T22:22:20.894534+0000","last_fullsized":"2023-07-13T22:22:20.894534+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:59:50.029440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894608+0000","last_change":"2023-07-13T22:22:19.697958+0000","last_active":"2023-07-13T22:22:20.894608+0000","last_peered":"2023-07-13T22:22:20.894608+0000","last_clean":"2023-07-13T22:22:20.894608+0000","last_became_active":"2023-07-13T22:22:19.697687+0000","last_became_peered":"2023-07-13T22:22:19.697687+0000","last_unstale":"2023-07-13T22:22:20.894608+0000","last_undegraded":"2023-07-13T22:22:20.894608+0000","last_fullsized":"2023-07-13T22:22:20.894608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:13.709947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687400+0000","last_change":"2023-07-13T22:22:18.683135+0000","last_active":"2023-07-13T22:22:20.687400+0000","last_peered":"2023-07-13T22:22:20.687400+0000","last_clean":"2023-07-13T22:22:20.687400+0000","last_became_active":"2023-07-13T22:22:18.682853+0000","last_became_peered":"2023-07-13T22:22:18.682853+0000","last_unstale":"2023-07-13T22:22:20.687400+0000","last_undegraded":"2023-07-13T22:22:20.687400+0000","last_fullsized":"2023-07-13T22:22:20.687400+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:21:34.358104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046537+0000","last_change":"2023-07-13T22:22:19.704285+0000","last_active":"2023-07-13T22:22:21.046537+0000","last_peered":"2023-07-13T22:22:21.046537+0000","last_clean":"2023-07-13T22:22:21.046537+0000","last_became_active":"2023-07-13T22:22:19.703643+0000","last_became_peered":"2023-07-13T22:22:19.703643+0000","last_unstale":"2023-07-13T22:22:21.046537+0000","last_undegraded":"2023-07-13T22:22:21.046537+0000","last_fullsized":"2023-07-13T22:22:21.046537+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:12:34.777446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894435+0000","last_change":"2023-07-13T22:22:18.678327+0000","last_active":"2023-07-13T22:22:20.894435+0000","last_peered":"2023-07-13T22:22:20.894435+0000","last_clean":"2023-07-13T22:22:20.894435+0000","last_became_active":"2023-07-13T22:22:18.678067+0000","last_became_peered":"2023-07-13T22:22:18.678067+0000","last_unstale":"2023-07-13T22:22:20.894435+0000","last_undegraded":"2023-07-13T22:22:20.894435+0000","last_fullsized":"2023-07-13T22:22:20.894435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:23:28.504684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794046+0000","last_change":"2023-07-13T22:22:19.689914+0000","last_active":"2023-07-13T22:22:20.794046+0000","last_peered":"2023-07-13T22:22:20.794046+0000","last_clean":"2023-07-13T22:22:20.794046+0000","last_became_active":"2023-07-13T22:22:19.689462+0000","last_became_peered":"2023-07-13T22:22:19.689462+0000","last_unstale":"2023-07-13T22:22:20.794046+0000","last_undegraded":"2023-07-13T22:22:20.794046+0000","last_fullsized":"2023-07-13T22:22:20.794046+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:51:16.874448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894688+0000","last_change":"2023-07-13T22:22:18.683882+0000","last_active":"2023-07-13T22:22:20.894688+0000","last_peered":"2023-07-13T22:22:20.894688+0000","last_clean":"2023-07-13T22:22:20.894688+0000","last_became_active":"2023-07-13T22:22:18.683559+0000","last_became_peered":"2023-07-13T22:22:18.683559+0000","last_unstale":"2023-07-13T22:22:20.894688+0000","last_undegraded":"2023-07-13T22:22:20.894688+0000","last_fullsized":"2023-07-13T22:22:20.894688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:49:18.750217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688335+0000","last_change":"2023-07-13T22:22:19.692843+0000","last_active":"2023-07-13T22:22:20.688335+0000","last_peered":"2023-07-13T22:22:20.688335+0000","last_clean":"2023-07-13T22:22:20.688335+0000","last_became_active":"2023-07-13T22:22:19.689222+0000","last_became_peered":"2023-07-13T22:22:19.689222+0000","last_unstale":"2023-07-13T22:22:20.688335+0000","last_undegraded":"2023-07-13T22:22:20.688335+0000","last_fullsized":"2023-07-13T22:22:20.688335+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:58:08.188540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046925+0000","last_change":"2023-07-13T22:22:18.687862+0000","last_active":"2023-07-13T22:22:21.046925+0000","last_peered":"2023-07-13T22:22:21.046925+0000","last_clean":"2023-07-13T22:22:21.046925+0000","last_became_active":"2023-07-13T22:22:18.687578+0000","last_became_peered":"2023-07-13T22:22:18.687578+0000","last_unstale":"2023-07-13T22:22:21.046925+0000","last_undegraded":"2023-07-13T22:22:21.046925+0000","last_fullsized":"2023-07-13T22:22:21.046925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:32:38.019370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047020+0000","last_change":"2023-07-13T22:22:19.680279+0000","last_active":"2023-07-13T22:22:21.047020+0000","last_peered":"2023-07-13T22:22:21.047020+0000","last_clean":"2023-07-13T22:22:21.047020+0000","last_became_active":"2023-07-13T22:22:19.680000+0000","last_became_peered":"2023-07-13T22:22:19.680000+0000","last_unstale":"2023-07-13T22:22:21.047020+0000","last_undegraded":"2023-07-13T22:22:21.047020+0000","last_fullsized":"2023-07-13T22:22:21.047020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:44:59.667785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894344+0000","last_change":"2023-07-13T22:22:18.678463+0000","last_active":"2023-07-13T22:22:20.894344+0000","last_peered":"2023-07-13T22:22:20.894344+0000","last_clean":"2023-07-13T22:22:20.894344+0000","last_became_active":"2023-07-13T22:22:18.678158+0000","last_became_peered":"2023-07-13T22:22:18.678158+0000","last_unstale":"2023-07-13T22:22:20.894344+0000","last_undegraded":"2023-07-13T22:22:20.894344+0000","last_fullsized":"2023-07-13T22:22:20.894344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:01.656282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.689344+0000","last_change":"2023-07-13T22:22:19.693027+0000","last_active":"2023-07-13T22:22:20.689344+0000","last_peered":"2023-07-13T22:22:20.689344+0000","last_clean":"2023-07-13T22:22:20.689344+0000","last_became_active":"2023-07-13T22:22:19.692733+0000","last_became_peered":"2023-07-13T22:22:19.692733+0000","last_unstale":"2023-07-13T22:22:20.689344+0000","last_undegraded":"2023-07-13T22:22:20.689344+0000","last_fullsized":"2023-07-13T22:22:20.689344+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:02:24.245064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688499+0000","last_change":"2023-07-13T22:22:18.677191+0000","last_active":"2023-07-13T22:22:20.688499+0000","last_peered":"2023-07-13T22:22:20.688499+0000","last_clean":"2023-07-13T22:22:20.688499+0000","last_became_active":"2023-07-13T22:22:18.676904+0000","last_became_peered":"2023-07-13T22:22:18.676904+0000","last_unstale":"2023-07-13T22:22:20.688499+0000","last_undegraded":"2023-07-13T22:22:20.688499+0000","last_fullsized":"2023-07-13T22:22:20.688499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:25:19.932858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688818+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:22:20.688818+0000","last_peered":"2023-07-13T22:22:20.688818+0000","last_clean":"2023-07-13T22:22:20.688818+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:22:20.688818+0000","last_undegraded":"2023-07-13T22:22:20.688818+0000","last_fullsized":"2023-07-13T22:22:20.688818+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:52:04.409917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047158+0000","last_change":"2023-07-13T22:22:19.688642+0000","last_active":"2023-07-13T22:22:21.047158+0000","last_peered":"2023-07-13T22:22:21.047158+0000","last_clean":"2023-07-13T22:22:21.047158+0000","last_became_active":"2023-07-13T22:22:19.688378+0000","last_became_peered":"2023-07-13T22:22:19.688378+0000","last_unstale":"2023-07-13T22:22:21.047158+0000","last_undegraded":"2023-07-13T22:22:21.047158+0000","last_fullsized":"2023-07-13T22:22:21.047158+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:14:10.896364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046895+0000","last_change":"2023-07-13T22:22:18.667567+0000","last_active":"2023-07-13T22:22:21.046895+0000","last_peered":"2023-07-13T22:22:21.046895+0000","last_clean":"2023-07-13T22:22:21.046895+0000","last_became_active":"2023-07-13T22:22:18.667250+0000","last_became_peered":"2023-07-13T22:22:18.667250+0000","last_unstale":"2023-07-13T22:22:21.046895+0000","last_undegraded":"2023-07-13T22:22:21.046895+0000","last_fullsized":"2023-07-13T22:22:21.046895+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:54:59.714270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047185+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:22:21.047185+0000","last_peered":"2023-07-13T22:22:21.047185+0000","last_clean":"2023-07-13T22:22:21.047185+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:22:21.047185+0000","last_undegraded":"2023-07-13T22:22:21.047185+0000","last_fullsized":"2023-07-13T22:22:21.047185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:43:50.119354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687256+0000","last_change":"2023-07-13T22:22:19.698783+0000","last_active":"2023-07-13T22:22:20.687256+0000","last_peered":"2023-07-13T22:22:20.687256+0000","last_clean":"2023-07-13T22:22:20.687256+0000","last_became_active":"2023-07-13T22:22:19.698547+0000","last_became_peered":"2023-07-13T22:22:19.698547+0000","last_unstale":"2023-07-13T22:22:20.687256+0000","last_undegraded":"2023-07-13T22:22:20.687256+0000","last_fullsized":"2023-07-13T22:22:20.687256+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:34.008871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688763+0000","last_change":"2023-07-13T22:22:18.680294+0000","last_active":"2023-07-13T22:22:20.688763+0000","last_peered":"2023-07-13T22:22:20.688763+0000","last_clean":"2023-07-13T22:22:20.688763+0000","last_became_active":"2023-07-13T22:22:18.679989+0000","last_became_peered":"2023-07-13T22:22:18.679989+0000","last_unstale":"2023-07-13T22:22:20.688763+0000","last_undegraded":"2023-07-13T22:22:20.688763+0000","last_fullsized":"2023-07-13T22:22:20.688763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:05:34.616243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687231+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:22:20.687231+0000","last_peered":"2023-07-13T22:22:20.687231+0000","last_clean":"2023-07-13T22:22:20.687231+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:22:20.687231+0000","last_undegraded":"2023-07-13T22:22:20.687231+0000","last_fullsized":"2023-07-13T22:22:20.687231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:49:44.926269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":191,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047073+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:22:21.047073+0000","last_peered":"2023-07-13T22:22:21.047073+0000","last_clean":"2023-07-13T22:22:21.047073+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:22:21.047073+0000","last_undegraded":"2023-07-13T22:22:21.047073+0000","last_fullsized":"2023-07-13T22:22:21.047073+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:53:29.774952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047096+0000","last_change":"2023-07-13T22:22:19.685880+0000","last_active":"2023-07-13T22:22:21.047096+0000","last_peered":"2023-07-13T22:22:21.047096+0000","last_clean":"2023-07-13T22:22:21.047096+0000","last_became_active":"2023-07-13T22:22:19.685586+0000","last_became_peered":"2023-07-13T22:22:19.685586+0000","last_unstale":"2023-07-13T22:22:21.047096+0000","last_undegraded":"2023-07-13T22:22:21.047096+0000","last_fullsized":"2023-07-13T22:22:21.047096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:47.579372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.894542+0000","last_change":"2023-07-13T22:22:18.681823+0000","last_active":"2023-07-13T22:22:20.894542+0000","last_peered":"2023-07-13T22:22:20.894542+0000","last_clean":"2023-07-13T22:22:20.894542+0000","last_became_active":"2023-07-13T22:22:18.681516+0000","last_became_peered":"2023-07-13T22:22:18.681516+0000","last_unstale":"2023-07-13T22:22:20.894542+0000","last_undegraded":"2023-07-13T22:22:20.894542+0000","last_fullsized":"2023-07-13T22:22:20.894542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:34:55.893996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046749+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:22:21.046749+0000","last_peered":"2023-07-13T22:22:21.046749+0000","last_clean":"2023-07-13T22:22:21.046749+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:22:21.046749+0000","last_undegraded":"2023-07-13T22:22:21.046749+0000","last_fullsized":"2023-07-13T22:22:21.046749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:59.847327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688607+0000","last_change":"2023-07-13T22:22:19.690141+0000","last_active":"2023-07-13T22:22:20.688607+0000","last_peered":"2023-07-13T22:22:20.688607+0000","last_clean":"2023-07-13T22:22:20.688607+0000","last_became_active":"2023-07-13T22:22:19.689874+0000","last_became_peered":"2023-07-13T22:22:19.689874+0000","last_unstale":"2023-07-13T22:22:20.688607+0000","last_undegraded":"2023-07-13T22:22:20.688607+0000","last_fullsized":"2023-07-13T22:22:20.688607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:42:36.503117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046990+0000","last_change":"2023-07-13T22:22:18.687553+0000","last_active":"2023-07-13T22:22:21.046990+0000","last_peered":"2023-07-13T22:22:21.046990+0000","last_clean":"2023-07-13T22:22:21.046990+0000","last_became_active":"2023-07-13T22:22:18.687306+0000","last_became_peered":"2023-07-13T22:22:18.687306+0000","last_unstale":"2023-07-13T22:22:21.046990+0000","last_undegraded":"2023-07-13T22:22:21.046990+0000","last_fullsized":"2023-07-13T22:22:21.046990+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:28.316674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.046738+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:22:21.046738+0000","last_peered":"2023-07-13T22:22:21.046738+0000","last_clean":"2023-07-13T22:22:21.046738+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:22:21.046738+0000","last_undegraded":"2023-07-13T22:22:21.046738+0000","last_fullsized":"2023-07-13T22:22:21.046738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:28:58.208287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:21.047063+0000","last_change":"2023-07-13T22:22:19.694828+0000","last_active":"2023-07-13T22:22:21.047063+0000","last_peered":"2023-07-13T22:22:21.047063+0000","last_clean":"2023-07-13T22:22:21.047063+0000","last_became_active":"2023-07-13T22:22:19.694555+0000","last_became_peered":"2023-07-13T22:22:19.694555+0000","last_unstale":"2023-07-13T22:22:21.047063+0000","last_undegraded":"2023-07-13T22:22:21.047063+0000","last_fullsized":"2023-07-13T22:22:21.047063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:14:10.896364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794276+0000","last_change":"2023-07-13T22:22:18.677467+0000","last_active":"2023-07-13T22:22:20.794276+0000","last_peered":"2023-07-13T22:22:20.794276+0000","last_clean":"2023-07-13T22:22:20.794276+0000","last_became_active":"2023-07-13T22:22:18.677152+0000","last_became_peered":"2023-07-13T22:22:18.677152+0000","last_unstale":"2023-07-13T22:22:20.794276+0000","last_undegraded":"2023-07-13T22:22:20.794276+0000","last_fullsized":"2023-07-13T22:22:20.794276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:06:59.391553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.688712+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:22:20.688712+0000","last_peered":"2023-07-13T22:22:20.688712+0000","last_clean":"2023-07-13T22:22:20.688712+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:22:20.688712+0000","last_undegraded":"2023-07-13T22:22:20.688712+0000","last_fullsized":"2023-07-13T22:22:20.688712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:14:57.399173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.687051+0000","last_change":"2023-07-13T22:22:19.691360+0000","last_active":"2023-07-13T22:22:20.687051+0000","last_peered":"2023-07-13T22:22:20.687051+0000","last_clean":"2023-07-13T22:22:20.687051+0000","last_became_active":"2023-07-13T22:22:19.691092+0000","last_became_peered":"2023-07-13T22:22:19.691092+0000","last_unstale":"2023-07-13T22:22:20.687051+0000","last_undegraded":"2023-07-13T22:22:20.687051+0000","last_fullsized":"2023-07-13T22:22:20.687051+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:45:45.050703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.732841+0000","last_change":"2023-07-13T22:22:18.678304+0000","last_active":"2023-07-13T22:22:20.732841+0000","last_peered":"2023-07-13T22:22:20.732841+0000","last_clean":"2023-07-13T22:22:20.732841+0000","last_became_active":"2023-07-13T22:22:18.677999+0000","last_became_peered":"2023-07-13T22:22:18.677999+0000","last_unstale":"2023-07-13T22:22:20.732841+0000","last_undegraded":"2023-07-13T22:22:20.732841+0000","last_fullsized":"2023-07-13T22:22:20.732841+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:24:14.444707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733041+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:22:20.733041+0000","last_peered":"2023-07-13T22:22:20.733041+0000","last_clean":"2023-07-13T22:22:20.733041+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:22:20.733041+0000","last_undegraded":"2023-07-13T22:22:20.733041+0000","last_fullsized":"2023-07-13T22:22:20.733041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:27:20.368439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794090+0000","last_change":"2023-07-13T22:22:19.690009+0000","last_active":"2023-07-13T22:22:20.794090+0000","last_peered":"2023-07-13T22:22:20.794090+0000","last_clean":"2023-07-13T22:22:20.794090+0000","last_became_active":"2023-07-13T22:22:19.689696+0000","last_became_peered":"2023-07-13T22:22:19.689696+0000","last_unstale":"2023-07-13T22:22:20.794090+0000","last_undegraded":"2023-07-13T22:22:20.794090+0000","last_fullsized":"2023-07-13T22:22:20.794090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:18.654272+0000","last_clean_scrub_stamp":"2023-07-13T22:22:18.654272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:34.008871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.794236+0000","last_change":"2023-07-13T22:22:18.681618+0000","last_active":"2023-07-13T22:22:20.794236+0000","last_peered":"2023-07-13T22:22:20.794236+0000","last_clean":"2023-07-13T22:22:20.794236+0000","last_became_active":"2023-07-13T22:22:18.681315+0000","last_became_peered":"2023-07-13T22:22:18.681315+0000","last_unstale":"2023-07-13T22:22:20.794236+0000","last_undegraded":"2023-07-13T22:22:20.794236+0000","last_fullsized":"2023-07-13T22:22:20.794236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:22:17.648131+0000","last_clean_scrub_stamp":"2023-07-13T22:22:17.648131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:21:34.358104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-07-13T22:22:20.733185+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:22:20.733185+0000","last_peered":"2023-07-13T22:22:20.733185+0000","last_clean":"2023-07-13T22:22:20.733185+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:22:20.733185+0000","last_undegraded":"2023-07-13T22:22:20.733185+0000","last_fullsized":"2023-07-13T22:22:20.733185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:39:58.614283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":995328,"data_stored":750598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.754}]},{"osd":1,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.4710000000000001,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.62}]},{"osd":3,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:21:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":165831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.751},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.80200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":749054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.495}]},{"osd":3,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.629},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.75}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":157644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":11.792,"5min":11.792,"15min":11.792},"last":0.38900000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.443},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.99299999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":11.718999999999999,"5min":11.718999999999999,"15min":11.718999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.97399999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.679,"5min":11.679,"15min":11.679},"last":1.0329999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":11.756,"5min":11.756,"15min":11.756},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.0569999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":157680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.371},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":0.40699999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.78100000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.79600000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":166847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.31},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.373},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.495}]},{"osd":4,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.629}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":165853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.038},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.0580000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":10.657999999999999,"5min":10.657999999999999,"15min":10.657999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":8.75,"5min":8.75,"15min":8.75},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.89200000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":1.173},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":1.099}]},{"osd":7,"last update":"Thu Jul 13 22:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":1},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":1.212}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":158708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.629}]},{"osd":1,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:22:24.601 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:22:24.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-13T22:22:24.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.861+0000 7fe845fff700 1 Processor -- start 2023-07-13T22:22:24.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.861+0000 7fe845fff700 1 -- start start 2023-07-13T22:22:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.862+0000 7fe845fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe84006b6f0 0x7fe84006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.862+0000 7fe845fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84006ffd0 con 0x7fe8400cf4a0 2023-07-13T22:22:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.862+0000 7fe845fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84006e5f0 con 0x7fe84015c5f0 2023-07-13T22:22:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.862+0000 7fe845fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84006f430 con 0x7fe84006b6f0 2023-07-13T22:22:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.862+0000 7fe837fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe8400cf4a0 0x7fe84006afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34238/0 (socket says 172.21.15.60:34238) 2023-07-13T22:22:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.862+0000 7fe837fff700 1 -- 172.21.15.60:0/652052098 learned_addr learned my addr 172.21.15.60:0/652052098 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:24.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995913075 0 0) 0x7fe84006ffd0 con 0x7fe8400cf4a0 2023-07-13T22:22:24.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83ffff700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe84006b6f0 0x7fe84006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe82c0035b0 con 0x7fe8400cf4a0 2023-07-13T22:22:24.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484119004 0 0) 0x7fe84006e5f0 con 0x7fe84015c5f0 2023-07-13T22:22:24.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe84006ffd0 con 0x7fe84015c5f0 2023-07-13T22:22:24.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3706134832 0 0) 0x7fe82c0035b0 con 0x7fe8400cf4a0 2023-07-13T22:22:24.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe84006b6f0 msgr2=0x7fe84006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:24.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe84006b6f0 0x7fe84006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 >> v1:172.21.15.105:6789/0 conn(0x7fe84015c5f0 legacy=0x7fe84006f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84006e820 con 0x7fe8400cf4a0 2023-07-13T22:22:24.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fe828001670 con 0x7fe8400cf4a0 2023-07-13T22:22:24.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 >> v1:172.21.15.60:6789/0 conn(0x7fe8400cf4a0 legacy=0x7fe84006afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe84006b6f0 0x7fe82c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c004170 0x7fe82c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe82c004d90 0x7fe82c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84006f430 con 0x7fe82c004d90 2023-07-13T22:22:24.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c00bc50 con 0x7fe82c004170 2023-07-13T22:22:24.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.863+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c00be10 con 0x7fe84006b6f0 2023-07-13T22:22:24.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83ffff700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe82c004d90 0x7fe82c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83f7fe700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe84006b6f0 0x7fe82c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe837fff700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c004170 0x7fe82c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83ffff700 1 -- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe84006b6f0 msgr2=0x7fe82c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:24.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83ffff700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe84006b6f0 0x7fe82c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83ffff700 1 -- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c004170 msgr2=0x7fe82c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:24.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83ffff700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c004170 0x7fe82c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83ffff700 1 -- 172.21.15.60:0/652052098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c0035b0 con 0x7fe82c004d90 2023-07-13T22:22:24.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83ffff700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe82c004d90 0x7fe82c005200 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe840066fd0 tx=0x7fe82c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323c1bb6478f1d15 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:24.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83800ebd0 con 0x7fe82c004d90 2023-07-13T22:22:24.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.864+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83800f040 con 0x7fe82c004d90 2023-07-13T22:22:24.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.865+0000 7fe83effd700 1 -- 172.21.15.60:0/652052098 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838014480 con 0x7fe82c004d90 2023-07-13T22:22:24.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.865+0000 7fe845fff700 1 -- 172.21.15.60:0/652052098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe82c004d90 msgr2=0x7fe82c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.865+0000 7fe845fff700 1 --2- 172.21.15.60:0/652052098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe82c004d90 0x7fe82c005200 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe840066fd0 tx=0x7fe82c005880 comp rx=0 tx=0).stop 2023-07-13T22:22:24.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.865+0000 7fe83ffff700 1 -- 172.21.15.60:0/652052098 reap_dead start 2023-07-13T22:22:24.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.865+0000 7fe845fff700 1 -- 172.21.15.60:0/652052098 shutdown_connections 2023-07-13T22:22:24.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.865+0000 7fe845fff700 1 -- 172.21.15.60:0/652052098 >> 172.21.15.60:0/652052098 conn(0x7fe840099c30 msgr2=0x7fe8400d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:24.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.865+0000 7fe845fff700 1 -- 172.21.15.60:0/652052098 shutdown_connections 2023-07-13T22:22:24.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.865+0000 7fe845fff700 1 -- 172.21.15.60:0/652052098 wait complete. 2023-07-13T22:22:24.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.867+0000 7fe845fff700 1 Processor -- start 2023-07-13T22:22:24.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.867+0000 7fe845fff700 1 -- start start 2023-07-13T22:22:24.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.868+0000 7fe845fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c004d90 0x7fe8401e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.868+0000 7fe845fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe840156ec0 0x7fe8401e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.868+0000 7fe845fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8401e5fc0 0x7fe8401e6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.868+0000 7fe845fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8401e4100 con 0x7fe8401e5fc0 2023-07-13T22:22:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.868+0000 7fe845fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400b1000 con 0x7fe82c004d90 2023-07-13T22:22:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.868+0000 7fe845fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400b12e0 con 0x7fe840156ec0 2023-07-13T22:22:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.868+0000 7fe83ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8401e5fc0 0x7fe8401e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.868+0000 7fe83ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8401e5fc0 0x7fe8401e6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46886/0 (socket says 172.21.15.60:46886) 2023-07-13T22:22:24.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe83ffff700 1 -- 172.21.15.60:0/88442322 learned_addr learned my addr 172.21.15.60:0/88442322 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:24.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe83f7fe700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c004d90 0x7fe8401e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe837fff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe840156ec0 0x7fe8401e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe83ffff700 1 -- 172.21.15.60:0/88442322 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe840156ec0 msgr2=0x7fe8401e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe83ffff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe840156ec0 0x7fe8401e4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe83ffff700 1 -- 172.21.15.60:0/88442322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c004d90 msgr2=0x7fe8401e3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:24.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe83ffff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c004d90 0x7fe8401e3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:24.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe83ffff700 1 -- 172.21.15.60:0/88442322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400b15c0 con 0x7fe8401e5fc0 2023-07-13T22:22:24.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe83ffff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8401e5fc0 0x7fe8401e6460 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00bc20 tx=0x7fe838009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:24.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.869+0000 7fe83cff9700 1 -- 172.21.15.60:0/88442322 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838007000 con 0x7fe8401e5fc0 2023-07-13T22:22:24.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.870+0000 7fe83cff9700 1 -- 172.21.15.60:0/88442322 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838022070 con 0x7fe8401e5fc0 2023-07-13T22:22:24.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.870+0000 7fe83cff9700 1 -- 172.21.15.60:0/88442322 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838003c40 con 0x7fe8401e5fc0 2023-07-13T22:22:24.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.870+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8400b18a0 con 0x7fe8401e5fc0 2023-07-13T22:22:24.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.870+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8401e4ac0 con 0x7fe8401e5fc0 2023-07-13T22:22:24.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.871+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe840066700 con 0x7fe8401e5fc0 2023-07-13T22:22:24.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.872+0000 7fe83cff9700 1 -- 172.21.15.60:0/88442322 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8380149c0 con 0x7fe8401e5fc0 2023-07-13T22:22:24.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.873+0000 7fe83cff9700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe82006ace0 0x7fe82006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:24.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.873+0000 7fe83cff9700 1 -- 172.21.15.60:0/88442322 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe838024070 con 0x7fe8401e5fc0 2023-07-13T22:22:24.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.873+0000 7fe83f7fe700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe82006ace0 0x7fe82006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:24.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.874+0000 7fe83f7fe700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe82006ace0 0x7fe82006d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe8401f5660 tx=0x7fe830009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:24.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:24.877+0000 7fe83cff9700 1 -- 172.21.15.60:0/88442322 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe838055c10 con 0x7fe8401e5fc0 2023-07-13T22:22:25.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.166+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe8400675c0 con 0x7fe8401e5fc0 2023-07-13T22:22:25.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.170+0000 7fe83cff9700 1 -- 172.21.15.60:0/88442322 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe83802d0b0 con 0x7fe8401e5fc0 2023-07-13T22:22:25.154 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:22:25.154 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:22:25.154 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-07-13T22:22:25.154 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-07-13T22:22:25.154 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-13T22:22:25.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:22:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe82006ace0 msgr2=0x7fe82006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:25.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe82006ace0 0x7fe82006d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe8401f5660 tx=0x7fe830009040 comp rx=0 tx=0).stop 2023-07-13T22:22:25.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8401e5fc0 msgr2=0x7fe8401e6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:25.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8401e5fc0 0x7fe8401e6460 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00bc20 tx=0x7fe838009750 comp rx=0 tx=0).stop 2023-07-13T22:22:25.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 shutdown_connections 2023-07-13T22:22:25.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe82006ace0 0x7fe82006d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8401e5fc0 0x7fe8401e6460 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe82c004d90 0x7fe8401e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 --2- 172.21.15.60:0/88442322 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe840156ec0 0x7fe8401e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 >> 172.21.15.60:0/88442322 conn(0x7fe840099c30 msgr2=0x7fe84015b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:25.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 shutdown_connections 2023-07-13T22:22:25.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.173+0000 7fe845fff700 1 -- 172.21.15.60:0/88442322 wait complete. 2023-07-13T22:22:25.160 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-07-13T22:22:25.161 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:22:25.175 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.434+0000 7f1de73ff700 1 Processor -- start 2023-07-13T22:22:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.435+0000 7f1de73ff700 1 -- start start 2023-07-13T22:22:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.436+0000 7f1de73ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1de0070270 0x7f1de00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.436+0000 7f1de73ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de015d970 con 0x7f1de0159170 2023-07-13T22:22:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.436+0000 7f1de73ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de0070c00 con 0x7f1de00c9c50 2023-07-13T22:22:25.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.436+0000 7f1de73ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de015cd60 con 0x7f1de0070270 2023-07-13T22:22:25.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.436+0000 7f1de519b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1de0159170 0x7f1de00c4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48586/0 (socket says 172.21.15.60:48586) 2023-07-13T22:22:25.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.436+0000 7f1de519b700 1 -- 172.21.15.60:0/3787248 learned_addr learned my addr 172.21.15.60:0/3787248 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1de499a700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1de0070270 0x7f1de00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:25.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609072210 0 0) 0x7f1de015d970 con 0x7f1de0159170 2023-07-13T22:22:25.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dcc0035b0 con 0x7f1de0159170 2023-07-13T22:22:25.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660257647 0 0) 0x7f1dd8001490 con 0x7f1de00c9c50 2023-07-13T22:22:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de015d970 con 0x7f1de00c9c50 2023-07-13T22:22:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3345292375 0 0) 0x7f1dcc0035b0 con 0x7f1de0159170 2023-07-13T22:22:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1de0070270 msgr2=0x7f1de00706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1de0070270 0x7f1de00706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 >> v1:172.21.15.105:6789/0 conn(0x7f1de00c9c50 legacy=0x7f1de00ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de015d040 con 0x7f1de0159170 2023-07-13T22:22:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f1ddc001670 con 0x7f1de0159170 2023-07-13T22:22:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 >> v1:172.21.15.60:6789/0 conn(0x7f1de0159170 legacy=0x7f1de00c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dcc003bc0 0x7f1dcc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1dcc004850 0x7f1dcc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1de0070270 0x7f1dcc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:25.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de015cd60 con 0x7f1dcc003bc0 2023-07-13T22:22:25.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc00dcd0 con 0x7f1dcc004850 2023-07-13T22:22:25.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc00de90 con 0x7f1de0070270 2023-07-13T22:22:25.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1de519b700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dcc003bc0 0x7f1dcc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:25.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.437+0000 7f1de499a700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1dcc004850 0x7f1dcc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:25.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.438+0000 7f1de599c700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1de0070270 0x7f1dcc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.438+0000 7f1de499a700 1 -- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1de0070270 msgr2=0x7f1dcc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.438+0000 7f1de499a700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1de0070270 0x7f1dcc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.438+0000 7f1de499a700 1 -- 172.21.15.60:0/3787248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dcc003bc0 msgr2=0x7f1dcc003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.438+0000 7f1de499a700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dcc003bc0 0x7f1dcc003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.438+0000 7f1de499a700 1 -- 172.21.15.60:0/3787248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc0035b0 con 0x7f1dcc004850 2023-07-13T22:22:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.438+0000 7f1de519b700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dcc003bc0 0x7f1dcc003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.438+0000 7f1de499a700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1dcc004850 0x7f1dcc004cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009b60 tx=0x7f1dcc005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6145b6384a670035 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:25.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.438+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd000e6c0 con 0x7f1dcc004850 2023-07-13T22:22:25.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.439+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd000e820 con 0x7f1dcc004850 2023-07-13T22:22:25.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.439+0000 7f1dd7fff700 1 -- 172.21.15.60:0/3787248 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd000eb90 con 0x7f1dcc004850 2023-07-13T22:22:25.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.439+0000 7f1de73ff700 1 -- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1dcc004850 msgr2=0x7f1dcc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.439+0000 7f1de73ff700 1 --2- 172.21.15.60:0/3787248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1dcc004850 0x7f1dcc004cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009b60 tx=0x7f1dcc005880 comp rx=0 tx=0).stop 2023-07-13T22:22:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.439+0000 7f1de599c700 1 -- 172.21.15.60:0/3787248 reap_dead start 2023-07-13T22:22:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.439+0000 7f1de73ff700 1 -- 172.21.15.60:0/3787248 shutdown_connections 2023-07-13T22:22:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.439+0000 7f1de73ff700 1 -- 172.21.15.60:0/3787248 >> 172.21.15.60:0/3787248 conn(0x7f1de00cf410 msgr2=0x7f1de00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.439+0000 7f1de73ff700 1 -- 172.21.15.60:0/3787248 shutdown_connections 2023-07-13T22:22:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.439+0000 7f1de73ff700 1 -- 172.21.15.60:0/3787248 wait complete. 2023-07-13T22:22:25.430 INFO:tasks.ceph.mds.c.smithi060.stderr:2023-07-13T22:22:25.440+0000 7fcd6e3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:22:25.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.441+0000 7f1de73ff700 1 Processor -- start 2023-07-13T22:22:25.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.441+0000 7f1de73ff700 1 -- start start 2023-07-13T22:22:25.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.442+0000 7f1de73ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1dcc004850 0x7f1de00ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:25.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.442+0000 7f1de73ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1de01492c0 0x7f1de00afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:25.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.442+0000 7f1de73ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de00b0170 0x7f1de00c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:25.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.442+0000 7f1de73ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de00c57c0 con 0x7f1de00b0170 2023-07-13T22:22:25.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.442+0000 7f1de73ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de00c6060 con 0x7f1de01492c0 2023-07-13T22:22:25.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.442+0000 7f1de73ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de00b3f60 con 0x7f1dcc004850 2023-07-13T22:22:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.442+0000 7f1de599c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de00b0170 0x7f1de00c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.442+0000 7f1de599c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de00b0170 0x7f1de00c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57722/0 (socket says 172.21.15.60:57722) 2023-07-13T22:22:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.442+0000 7f1de599c700 1 -- 172.21.15.60:0/502644003 learned_addr learned my addr 172.21.15.60:0/502644003 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1de499a700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1de01492c0 0x7f1de00afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1de519b700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1dcc004850 0x7f1de00ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1de599c700 1 -- 172.21.15.60:0/502644003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1dcc004850 msgr2=0x7f1de00ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1de599c700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1dcc004850 0x7f1de00ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1de599c700 1 -- 172.21.15.60:0/502644003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1de01492c0 msgr2=0x7f1de00afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1de599c700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1de01492c0 0x7f1de00afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1de599c700 1 -- 172.21.15.60:0/502644003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de00c5a10 con 0x7f1de00b0170 2023-07-13T22:22:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1de599c700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de00b0170 0x7f1de00c64f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1de0070060 tx=0x7f1dd800d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1dd5ffb700 1 -- 172.21.15.60:0/502644003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800fed0 con 0x7f1de00b0170 2023-07-13T22:22:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1dd5ffb700 1 -- 172.21.15.60:0/502644003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd800dd20 con 0x7f1de00b0170 2023-07-13T22:22:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de00c5cf0 con 0x7f1de00b0170 2023-07-13T22:22:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.443+0000 7f1dd5ffb700 1 -- 172.21.15.60:0/502644003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8003c20 con 0x7f1de00b0170 2023-07-13T22:22:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.444+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de0149710 con 0x7f1de00b0170 2023-07-13T22:22:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.445+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de00c61a0 con 0x7f1de00b0170 2023-07-13T22:22:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.446+0000 7f1dd5ffb700 1 -- 172.21.15.60:0/502644003 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1dd80050d0 con 0x7f1de00b0170 2023-07-13T22:22:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.446+0000 7f1dd5ffb700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1dc406ace0 0x7f1dc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.447+0000 7f1dd5ffb700 1 -- 172.21.15.60:0/502644003 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1dd8089400 con 0x7f1de00b0170 2023-07-13T22:22:25.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.447+0000 7f1de519b700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1dc406ace0 0x7f1dc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:25.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.447+0000 7f1de519b700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1dc406ace0 0x7f1dc406d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc000c00 tx=0x7f1ddc0011c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:25.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.451+0000 7f1dd5ffb700 1 -- 172.21.15.60:0/502644003 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1dd8054540 con 0x7f1de00b0170 2023-07-13T22:22:25.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.704+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1de0070c00 con 0x7f1de00b0170 2023-07-13T22:22:25.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.704+0000 7f1dd5ffb700 1 -- 172.21.15.60:0/502644003 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f1de0070c00 con 0x7f1de00b0170 2023-07-13T22:22:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:25.688 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2157014443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2157014443},{"type":"v1","addr":"172.21.15.105:6834","nonce":2157014443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/302067138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":302067138},{"type":"v1","addr":"172.21.15.105:6835","nonce":302067138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3461450784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461450784},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461450784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:22:19.175830+0000","modified":"2023-07-13T22:22:22.691356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6837/2048874349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2048874349},{"type":"v1","addr":"172.21.15.60:6837","nonce":2048874349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-13T22:22:25.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1dc406ace0 msgr2=0x7f1dc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:25.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1dc406ace0 0x7f1dc406d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc000c00 tx=0x7f1ddc0011c0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de00b0170 msgr2=0x7f1de00c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:25.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de00b0170 0x7f1de00c64f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1de0070060 tx=0x7f1dd800d660 comp rx=0 tx=0).stop 2023-07-13T22:22:25.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 shutdown_connections 2023-07-13T22:22:25.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1dc406ace0 0x7f1dc406d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de00b0170 0x7f1de00c64f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1dcc004850 0x7f1de00ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 --2- 172.21.15.60:0/502644003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1de01492c0 0x7f1de00afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:25.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 >> 172.21.15.60:0/502644003 conn(0x7f1de00cf410 msgr2=0x7f1de0154620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:25.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 shutdown_connections 2023-07-13T22:22:25.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:25.708+0000 7f1de73ff700 1 -- 172.21.15.60:0/502644003 wait complete. 2023-07-13T22:22:25.693 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2023-07-13T22:22:26.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:22:19.175830+0000', 'modified': '2023-07-13T22:22:22.691356+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5322}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5322': {'gid': 5322, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6837/2048874349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6836', 'nonce': 2048874349}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 2048874349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:22:26.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:22:26.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-13T22:22:26.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.339+0000 7fd4b75ff700 1 Processor -- start 2023-07-13T22:22:26.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.339+0000 7fd4b75ff700 1 -- start start 2023-07-13T22:22:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.340+0000 7fd4b75ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00b8570 0x7fd4b00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.340+0000 7fd4b75ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4b00d15c0 0x7fd4b015c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.340+0000 7fd4b75ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b00d09d0 con 0x7fd4b00d35c0 2023-07-13T22:22:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.340+0000 7fd4b75ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00d3b30 con 0x7fd4b00d15c0 2023-07-13T22:22:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.340+0000 7fd4b75ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00cda10 con 0x7fd4b00b8570 2023-07-13T22:22:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.340+0000 7fd4b5b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4b00d15c0 0x7fd4b015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.341+0000 7fd4b5b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4b00d15c0 0x7fd4b015c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57732/0 (socket says 172.21.15.60:57732) 2023-07-13T22:22:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.341+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/3313624531 learned_addr learned my addr 172.21.15.60:0/3313624531 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.341+0000 7fd4b539b700 1 --2- 172.21.15.60:0/3313624531 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00b8570 0x7fd4b00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.341+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/3313624531 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00b8570 msgr2=0x7fd4b00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.341+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/3313624531 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00b8570 0x7fd4b00ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.341+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/3313624531 >> v1:172.21.15.105:6789/0 conn(0x7fd4b00d35c0 legacy=0x7fd4b00d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.341+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/3313624531 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00cd450 con 0x7fd4b00d15c0 2023-07-13T22:22:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.341+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/3313624531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00d15c0 0x7fd4b015c920 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009e50 tx=0x7fd4a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58175e3f6c32b66 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4a7fff700 1 -- 172.21.15.60:0/3313624531 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800e070 con 0x7fd4b00d15c0 2023-07-13T22:22:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4a7fff700 1 -- 172.21.15.60:0/3313624531 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a800b910 con 0x7fd4b00d15c0 2023-07-13T22:22:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4a7fff700 1 -- 172.21.15.60:0/3313624531 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800bcc0 con 0x7fd4b00d15c0 2023-07-13T22:22:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4b75ff700 1 -- 172.21.15.60:0/3313624531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00d15c0 msgr2=0x7fd4b015c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4b75ff700 1 --2- 172.21.15.60:0/3313624531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00d15c0 0x7fd4b015c920 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009e50 tx=0x7fd4a800b330 comp rx=0 tx=0).stop 2023-07-13T22:22:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4b75ff700 1 -- 172.21.15.60:0/3313624531 shutdown_connections 2023-07-13T22:22:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4b75ff700 1 --2- 172.21.15.60:0/3313624531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00d15c0 0x7fd4b015c920 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4b75ff700 1 --2- 172.21.15.60:0/3313624531 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00b8570 0x7fd4b00ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4b75ff700 1 -- 172.21.15.60:0/3313624531 >> 172.21.15.60:0/3313624531 conn(0x7fd4b0071600 msgr2=0x7fd4b00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4b75ff700 1 -- 172.21.15.60:0/3313624531 shutdown_connections 2023-07-13T22:22:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.342+0000 7fd4b75ff700 1 -- 172.21.15.60:0/3313624531 wait complete. 2023-07-13T22:22:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.344+0000 7fd4b75ff700 1 Processor -- start 2023-07-13T22:22:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.344+0000 7fd4b75ff700 1 -- start start 2023-07-13T22:22:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.345+0000 7fd4b75ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00b8570 0x7fd4b00c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.345+0000 7fd4b75ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00d15c0 0x7fd4b01f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.345+0000 7fd4b75ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b00d35c0 0x7fd4b014f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.345+0000 7fd4b75ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00cb040 con 0x7fd4b00b8570 2023-07-13T22:22:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.345+0000 7fd4b75ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b01f2450 con 0x7fd4b00d35c0 2023-07-13T22:22:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.345+0000 7fd4b75ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b01f2730 con 0x7fd4b00d15c0 2023-07-13T22:22:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.345+0000 7fd4b539b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00b8570 0x7fd4b00c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.345+0000 7fd4b539b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00b8570 0x7fd4b00c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57734/0 (socket says 172.21.15.60:57734) 2023-07-13T22:22:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.345+0000 7fd4b539b700 1 -- 172.21.15.60:0/2170009188 learned_addr learned my addr 172.21.15.60:0/2170009188 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b00d35c0 0x7fd4b014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00d15c0 0x7fd4b01f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4b539b700 1 -- 172.21.15.60:0/2170009188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00d15c0 msgr2=0x7fd4b01f1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4b539b700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00d15c0 0x7fd4b01f1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4b539b700 1 -- 172.21.15.60:0/2170009188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b00d35c0 msgr2=0x7fd4b014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4b539b700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b00d35c0 0x7fd4b014f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4b539b700 1 -- 172.21.15.60:0/2170009188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8009b30 con 0x7fd4b00b8570 2023-07-13T22:22:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4b539b700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00b8570 0x7fd4b00c9640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00e990 tx=0x7fd4ac00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4a5ffb700 1 -- 172.21.15.60:0/2170009188 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac0096f0 con 0x7fd4b00b8570 2023-07-13T22:22:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4a5ffb700 1 -- 172.21.15.60:0/2170009188 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac00c850 con 0x7fd4b00b8570 2023-07-13T22:22:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4b75ff700 1 -- 172.21.15.60:0/2170009188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b01f2a10 con 0x7fd4b00b8570 2023-07-13T22:22:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.346+0000 7fd4a5ffb700 1 -- 172.21.15.60:0/2170009188 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00cbc0 con 0x7fd4b00b8570 2023-07-13T22:22:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.347+0000 7fd4b75ff700 1 -- 172.21.15.60:0/2170009188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b014f670 con 0x7fd4b00b8570 2023-07-13T22:22:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.349+0000 7fd4a5ffb700 1 -- 172.21.15.60:0/2170009188 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd4ac00cd20 con 0x7fd4b00b8570 2023-07-13T22:22:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.349+0000 7fd4a5ffb700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd49406ace0 0x7fd49406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.350+0000 7fd4a5ffb700 1 -- 172.21.15.60:0/2170009188 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4ac089a00 con 0x7fd4b00b8570 2023-07-13T22:22:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.350+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd49406ace0 0x7fd49406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.350+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd49406ace0 0x7fd49406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd4b006c410 tx=0x7fd4a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.352+0000 7fd4caf18b80 1 Processor -- start 2023-07-13T22:22:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.352+0000 7fd4caf18b80 1 -- start start 2023-07-13T22:22:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 0x55bd7cfbe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cfbeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfa9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd7cf08ea0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd7ce7c600 con 0x55bd7cf8e880 2023-07-13T22:22:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd7ceeb910 con 0x55bd7cf90dc0 2023-07-13T22:22:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b5b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfa9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b5b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfa9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57738/0 (socket says 172.21.15.60:57738) 2023-07-13T22:22:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/636313933 learned_addr learned my addr 172.21.15.60:0/636313933 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b539b700 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 0x55bd7cfbe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cfbeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 msgr2=0x55bd7cfbeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cfbeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 msgr2=0x55bd7cfbe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 0x55bd7cfbe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/636313933 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd7ccd8790 con 0x55bd7cf8ec50 2023-07-13T22:22:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b539b700 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 0x55bd7cfbe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.353+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cfbeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.354+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfa9120 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009360 tx=0x7fd4a8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c248cd1d6cffa3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.354+0000 7fd49affd700 1 -- 172.21.15.60:0/636313933 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8003a20 con 0x55bd7cf8ec50 2023-07-13T22:22:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.354+0000 7fd49affd700 1 -- 172.21.15.60:0/636313933 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a8003b80 con 0x55bd7cf8ec50 2023-07-13T22:22:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.354+0000 7fd49affd700 1 -- 172.21.15.60:0/636313933 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8004040 con 0x55bd7cf8ec50 2023-07-13T22:22:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 -- 172.21.15.60:0/636313933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 msgr2=0x55bd7cfa9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfa9120 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009360 tx=0x7fd4a8010000 comp rx=0 tx=0).stop 2023-07-13T22:22:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 -- 172.21.15.60:0/636313933 shutdown_connections 2023-07-13T22:22:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfa9120 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 0x55bd7cfbe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/636313933 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cfbeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 -- 172.21.15.60:0/636313933 >> 172.21.15.60:0/636313933 conn(0x55bd7cf893b0 msgr2=0x55bd7cf94550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 -- 172.21.15.60:0/636313933 shutdown_connections 2023-07-13T22:22:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 -- 172.21.15.60:0/636313933 wait complete. 2023-07-13T22:22:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.355+0000 7fd4caf18b80 1 -- start start 2023-07-13T22:22:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.356+0000 7fd4caf18b80 1 Processor -- start 2023-07-13T22:22:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 0x55bd7cfbb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfbb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cf8f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd7cee4be0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd7cea25b0 con 0x55bd7cf8e880 2023-07-13T22:22:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd7ce88910 con 0x55bd7cf90dc0 2023-07-13T22:22:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4caf18b80 10 client.0 _ll_register_callbacks cb 0x55bd7cea0b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b4b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfbb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b4b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfbb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57740/0 (socket says 172.21.15.60:57740) 2023-07-13T22:22:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b4b9a700 1 -- 172.21.15.60:0/2206698266 learned_addr learned my addr 172.21.15.60:0/2206698266 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b539b700 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 0x55bd7cfbb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cf8f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b4b9a700 1 -- 172.21.15.60:0/2206698266 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 msgr2=0x55bd7cf8f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cf8f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b4b9a700 1 -- 172.21.15.60:0/2206698266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 msgr2=0x55bd7cfbb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 0x55bd7cfbb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b4b9a700 1 -- 172.21.15.60:0/2206698266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a800be70 con 0x55bd7cf8ec50 2023-07-13T22:22:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.357+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cf8f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.358+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfbb720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00039b0 tx=0x7fd4a0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f721897999570b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.358+0000 7fd48ffff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:22:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.358+0000 7fd48ffff700 1 -- 172.21.15.60:0/2206698266 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00155c0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.358+0000 7fd48ffff700 1 -- 172.21.15.60:0/2206698266 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4a0022bf0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.358+0000 7fd4caf18b80 1 -- 172.21.15.60:0/2206698266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd7cf869c0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.358+0000 7fd48ffff700 1 -- 172.21.15.60:0/2206698266 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a0022db0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.358+0000 7fd4caf18b80 10 client.5508 fetch_fsmap learned FSMap version 16 2023-07-13T22:22:26.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.358+0000 7fd4caf18b80 1 -- 172.21.15.60:0/2206698266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bd7cdc50a0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd48ffff700 1 -- 172.21.15.60:0/2206698266 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bd7cf869c0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd48ffff700 1 -- 172.21.15.60:0/2206698266 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd4a0015720 con 0x55bd7cf8ec50 2023-07-13T22:22:26.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd4caf18b80 10 client.5508 fetch_fsmap finished waiting for FSMap version 16 2023-07-13T22:22:26.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd4caf18b80 10 client.5508 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-13T22:22:26.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd4caf18b80 20 client.5508 populate_metadata read hostname 'smithi060' 2023-07-13T22:22:26.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] conn(0x55bd7cf8f6f0 0x55bd7cfbde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd4caf18b80 4 client.5508 mds_command: new command op to 5322 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:22:26.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd4caf18b80 1 -- 172.21.15.60:0/2206698266 --> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd7cf63170 con 0x55bd7cf8f6f0 2023-07-13T22:22:26.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd4b539b700 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] conn(0x55bd7cf8f6f0 0x55bd7cfbde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd4b539b700 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] conn(0x55bd7cf8f6f0 0x55bd7cfbde50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3e4f2e18892285 server_cookie=17811c0bdcac993b in_seq=0 out_seq=0 2023-07-13T22:22:26.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.359+0000 7fd48ffff700 10 client.5508 ms_handle_connect on v2:172.21.15.60:6836/2048874349 2023-07-13T22:22:26.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.360+0000 7fd48ffff700 1 -- 172.21.15.60:0/2206698266 <== mds.0 v2:172.21.15.60:6836/2048874349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bd7cf63170 con 0x55bd7cf8f6f0 2023-07-13T22:22:26.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.360+0000 7fd48ffff700 10 client.5508 handle_command_reply: tid=0 2023-07-13T22:22:26.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.360+0000 7fd4caf18b80 1 client.5508 shutdown 2023-07-13T22:22:26.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 -- 172.21.15.60:0/2206698266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 msgr2=0x55bd7cfbb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfbb720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00039b0 tx=0x7fd4a0000ed0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 -- 172.21.15.60:0/2206698266 shutdown_connections 2023-07-13T22:22:26.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] conn(0x55bd7cf8f6f0 0x55bd7cfbde50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf90dc0 0x55bd7cf8f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8e880 0x55bd7cfbb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd48ffff700 0 client.5508 ms_handle_reset on v2:172.21.15.60:6836/2048874349 2023-07-13T22:22:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/2206698266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8ec50 0x55bd7cfbb720 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 -- 172.21.15.60:0/2206698266 >> 172.21.15.60:0/2206698266 conn(0x55bd7cf893b0 msgr2=0x55bd7cf91a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:26.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 -- 172.21.15.60:0/2206698266 shutdown_connections 2023-07-13T22:22:26.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 1 -- 172.21.15.60:0/2206698266 wait complete. 2023-07-13T22:22:26.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.361+0000 7fd4caf18b80 20 client.5508 trim_cache size 0 max 16384 2023-07-13T22:22:26.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.380+0000 7fd4caf18b80 1 Processor -- start 2023-07-13T22:22:26.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.380+0000 7fd4caf18b80 1 -- start start 2023-07-13T22:22:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8e880 0x55bd7cfeb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8ec50 0x55bd7cf91650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf90dc0 0x55bd7cf91b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd7ccd9390 con 0x55bd7cf8e880 2023-07-13T22:22:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd7cf90100 con 0x55bd7cf90dc0 2023-07-13T22:22:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd7cfdd5f0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b539b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8e880 0x55bd7cfeb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b539b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8e880 0x55bd7cfeb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57756/0 (socket says 172.21.15.60:57756) 2023-07-13T22:22:26.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b539b700 1 -- 172.21.15.60:0/3827762317 learned_addr learned my addr 172.21.15.60:0/3827762317 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:26.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/3827762317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf90dc0 0x55bd7cf91b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/3827762317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8ec50 0x55bd7cf91650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b539b700 1 -- 172.21.15.60:0/3827762317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8ec50 msgr2=0x55bd7cf91650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:26.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b539b700 1 --2- 172.21.15.60:0/3827762317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8ec50 0x55bd7cf91650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b539b700 1 -- 172.21.15.60:0/3827762317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf90dc0 msgr2=0x55bd7cf91b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:26.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b539b700 1 --2- 172.21.15.60:0/3827762317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf90dc0 0x55bd7cf91b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.381+0000 7fd4b539b700 1 -- 172.21.15.60:0/3827762317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd7ceaab10 con 0x55bd7cf8e880 2023-07-13T22:22:26.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd4b539b700 1 --2- 172.21.15.60:0/3827762317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8e880 0x55bd7cfeb140 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac05c850 tx=0x7fd4ac05b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10837cba0ed58e5b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd49b7fe700 1 -- 172.21.15.60:0/3827762317 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac0568d0 con 0x55bd7cf8e880 2023-07-13T22:22:26.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd49b7fe700 1 -- 172.21.15.60:0/3827762317 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac057bc0 con 0x55bd7cf8e880 2023-07-13T22:22:26.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd49b7fe700 1 -- 172.21.15.60:0/3827762317 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac060590 con 0x55bd7cf8e880 2023-07-13T22:22:26.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3827762317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8e880 msgr2=0x55bd7cfeb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3827762317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8e880 0x55bd7cfeb140 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac05c850 tx=0x7fd4ac05b040 comp rx=0 tx=0).stop 2023-07-13T22:22:26.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3827762317 shutdown_connections 2023-07-13T22:22:26.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3827762317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf8e880 0x55bd7cfeb140 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3827762317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf90dc0 0x55bd7cf91b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3827762317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8ec50 0x55bd7cf91650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.382+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3827762317 >> 172.21.15.60:0/3827762317 conn(0x55bd7cf893b0 msgr2=0x55bd7cf94520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:26.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.383+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3827762317 shutdown_connections 2023-07-13T22:22:26.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.383+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3827762317 wait complete. 2023-07-13T22:22:26.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.383+0000 7fd4caf18b80 1 -- start start 2023-07-13T22:22:26.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.383+0000 7fd4caf18b80 1 Processor -- start 2023-07-13T22:22:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.384+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8e880 0x55bd7cfa97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.384+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8ec50 0x55bd7d03fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4caf18b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf90dc0 0x55bd7d025350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd7d03a050 con 0x55bd7cf90dc0 2023-07-13T22:22:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd7d0404f0 con 0x55bd7cf8ec50 2023-07-13T22:22:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4caf18b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd7cfa89e0 con 0x55bd7cf8e880 2023-07-13T22:22:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4caf18b80 10 client.0 _ll_register_callbacks cb 0x7fd4a0005520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b5b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf90dc0 0x55bd7d025350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b5b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf90dc0 0x55bd7d025350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57770/0 (socket says 172.21.15.60:57770) 2023-07-13T22:22:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/3196984858 learned_addr learned my addr 172.21.15.60:0/3196984858 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b539b700 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8e880 0x55bd7cfa97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b4b9a700 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8ec50 0x55bd7d03fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/3196984858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8e880 msgr2=0x55bd7cfa97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8e880 0x55bd7cfa97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/3196984858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8ec50 msgr2=0x55bd7d03fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8ec50 0x55bd7d03fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.385+0000 7fd4b5b9c700 1 -- 172.21.15.60:0/3196984858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac081e70 con 0x55bd7cf90dc0 2023-07-13T22:22:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd4b5b9c700 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf90dc0 0x55bd7d025350 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800d010 tx=0x7fd4a8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d60a5bf72ec9e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd48ffff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:22:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd48ffff700 1 -- 172.21.15.60:0/3196984858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8003a20 con 0x55bd7cf90dc0 2023-07-13T22:22:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3196984858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4a8003b80 con 0x55bd7cf90dc0 2023-07-13T22:22:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd48ffff700 1 -- 172.21.15.60:0/3196984858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4a802a040 con 0x55bd7cf90dc0 2023-07-13T22:22:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd48ffff700 1 -- 172.21.15.60:0/3196984858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a801db70 con 0x55bd7cf90dc0 2023-07-13T22:22:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd4caf18b80 10 client.5514 fetch_fsmap learned FSMap version 16 2023-07-13T22:22:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3196984858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bd7cd391d0 con 0x55bd7cf90dc0 2023-07-13T22:22:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd48ffff700 1 -- 172.21.15.60:0/3196984858 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a8003b80 con 0x55bd7cf90dc0 2023-07-13T22:22:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.386+0000 7fd48ffff700 1 -- 172.21.15.60:0/3196984858 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd4a8023030 con 0x55bd7cf90dc0 2023-07-13T22:22:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd4caf18b80 10 client.5514 fetch_fsmap finished waiting for FSMap version 16 2023-07-13T22:22:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd4caf18b80 10 client.5514 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-13T22:22:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd4caf18b80 20 client.5514 populate_metadata read hostname 'smithi060' 2023-07-13T22:22:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] conn(0x55bd7cf91650 0x55bd7cf91a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd4caf18b80 4 client.5514 mds_command: new command op to 5322 tid=0[{"prefix": "status"}] 2023-07-13T22:22:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3196984858 --> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bd7ceddd20 con 0x55bd7cf91650 2023-07-13T22:22:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd4b539b700 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] conn(0x55bd7cf91650 0x55bd7cf91a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd4b539b700 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] conn(0x55bd7cf91650 0x55bd7cf91a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74ece85d07648210 server_cookie=b9fb3befd24def56 in_seq=0 out_seq=0 2023-07-13T22:22:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd48ffff700 10 client.5514 ms_handle_connect on v2:172.21.15.60:6836/2048874349 2023-07-13T22:22:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd48ffff700 1 -- 172.21.15.60:0/3196984858 <== mds.0 v2:172.21.15.60:6836/2048874349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bd7ceddd20 con 0x55bd7cf91650 2023-07-13T22:22:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.387+0000 7fd48ffff700 10 client.5514 handle_command_reply: tid=0 2023-07-13T22:22:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 client.5514 shutdown 2023-07-13T22:22:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3196984858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf90dc0 msgr2=0x55bd7d025350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf90dc0 0x55bd7d025350 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800d010 tx=0x7fd4a8004b40 comp rx=0 tx=0).stop 2023-07-13T22:22:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3196984858 shutdown_connections 2023-07-13T22:22:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.60:6836/2048874349,v1:172.21.15.60:6837/2048874349] conn(0x55bd7cf91650 0x55bd7cf91a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd7cf90dc0 0x55bd7d025350 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55bd7cf8e880 0x55bd7cfa97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 --2- 172.21.15.60:0/3196984858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd7cf8ec50 0x55bd7d03fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3196984858 >> 172.21.15.60:0/3196984858 conn(0x55bd7cf893b0 msgr2=0x55bd7cf92a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:26.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd48ffff700 0 client.5514 ms_handle_reset on v2:172.21.15.60:6836/2048874349 2023-07-13T22:22:26.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3196984858 shutdown_connections 2023-07-13T22:22:26.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 1 -- 172.21.15.60:0/3196984858 wait complete. 2023-07-13T22:22:26.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.388+0000 7fd4caf18b80 20 client.5514 trim_cache size 0 max 16384 2023-07-13T22:22:26.393 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:22:26.393 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:22:26.393 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:22:26.393 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5322, 2023-07-13T22:22:26.393 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:22:26.393 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:22:26.394 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:22:26.394 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.7076989380000001, 2023-07-13T22:22:26.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 16, 2023-07-13T22:22:26.394 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 35, 2023-07-13T22:22:26.394 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 35, 2023-07-13T22:22:26.394 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 16.688544447000002 2023-07-13T22:22:26.394 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:22:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 -- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd49406ace0 msgr2=0x7fd49406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd49406ace0 0x7fd49406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd4b006c410 tx=0x7fd4a0009040 comp rx=0 tx=0).stop 2023-07-13T22:22:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 -- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00b8570 msgr2=0x7fd4b00c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00b8570 0x7fd4b00c9640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00e990 tx=0x7fd4ac00ed50 comp rx=0 tx=0).stop 2023-07-13T22:22:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 -- 172.21.15.60:0/2170009188 shutdown_connections 2023-07-13T22:22:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd49406ace0 0x7fd49406d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4b00b8570 0x7fd4b00c9640 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b00d35c0 0x7fd4b014f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 --2- 172.21.15.60:0/2170009188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4b00d15c0 0x7fd4b01f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 -- 172.21.15.60:0/2170009188 >> 172.21.15.60:0/2170009188 conn(0x7fd4b0071600 msgr2=0x7fd4b00d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:26.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 -- 172.21.15.60:0/2170009188 shutdown_connections 2023-07-13T22:22:26.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.389+0000 7fd4b75ff700 1 -- 172.21.15.60:0/2170009188 wait complete. 2023-07-13T22:22:26.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:22:26.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.615+0000 7ff822bff700 1 Processor -- start 2023-07-13T22:22:26.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.615+0000 7ff822bff700 1 -- start start 2023-07-13T22:22:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff822bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0cfca0 0x7ff81c167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff822bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c0cd950 con 0x7ff81c0b8400 2023-07-13T22:22:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff822bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c0d0ac0 con 0x7ff81c0d3750 2023-07-13T22:22:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff822bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0b87d0 con 0x7ff81c0cfca0 2023-07-13T22:22:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff82099b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff81c0b8400 0x7ff81c0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48600/0 (socket says 172.21.15.60:48600) 2023-07-13T22:22:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff82099b700 1 -- 172.21.15.60:0/3746954321 learned_addr learned my addr 172.21.15.60:0/3746954321 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff82119c700 1 --2- 172.21.15.60:0/3746954321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0cfca0 0x7ff81c167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff81b7fe700 1 -- 172.21.15.60:0/3746954321 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369631426 0 0) 0x7ff810001110 con 0x7ff81c0b8400 2023-07-13T22:22:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff81b7fe700 1 -- 172.21.15.60:0/3746954321 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff80c0035b0 con 0x7ff81c0b8400 2023-07-13T22:22:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff81b7fe700 1 -- 172.21.15.60:0/3746954321 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161890583 0 0) 0x7ff81c0d0ac0 con 0x7ff81c0d3750 2023-07-13T22:22:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.617+0000 7ff81b7fe700 1 -- 172.21.15.60:0/3746954321 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff810001110 con 0x7ff81c0d3750 2023-07-13T22:22:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.618+0000 7ff82119c700 1 -- 172.21.15.60:0/3746954321 >> v1:172.21.15.105:6789/0 conn(0x7ff81c0d3750 legacy=0x7ff81c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.618+0000 7ff82119c700 1 -- 172.21.15.60:0/3746954321 >> v1:172.21.15.60:6789/0 conn(0x7ff81c0b8400 legacy=0x7ff81c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.618+0000 7ff82119c700 1 -- 172.21.15.60:0/3746954321 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0b4eb0 con 0x7ff81c0cfca0 2023-07-13T22:22:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.618+0000 7ff82119c700 1 --2- 172.21.15.60:0/3746954321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0cfca0 0x7ff81c167050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff814009dc0 tx=0x7ff81400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94fb6f0f3a856d58 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.618+0000 7ff81b7fe700 1 -- 172.21.15.60:0/3746954321 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400e070 con 0x7ff81c0cfca0 2023-07-13T22:22:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.618+0000 7ff81b7fe700 1 -- 172.21.15.60:0/3746954321 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400bb20 con 0x7ff81c0cfca0 2023-07-13T22:22:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.618+0000 7ff81b7fe700 1 -- 172.21.15.60:0/3746954321 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814012430 con 0x7ff81c0cfca0 2023-07-13T22:22:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.619+0000 7ff822bff700 1 -- 172.21.15.60:0/3746954321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0cfca0 msgr2=0x7ff81c167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.619+0000 7ff822bff700 1 --2- 172.21.15.60:0/3746954321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0cfca0 0x7ff81c167050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff814009dc0 tx=0x7ff81400b330 comp rx=0 tx=0).stop 2023-07-13T22:22:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.619+0000 7ff822bff700 1 -- 172.21.15.60:0/3746954321 shutdown_connections 2023-07-13T22:22:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.619+0000 7ff822bff700 1 --2- 172.21.15.60:0/3746954321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0cfca0 0x7ff81c167050 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.619+0000 7ff822bff700 1 -- 172.21.15.60:0/3746954321 >> 172.21.15.60:0/3746954321 conn(0x7ff81c06b390 msgr2=0x7ff81c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.619+0000 7ff822bff700 1 -- 172.21.15.60:0/3746954321 shutdown_connections 2023-07-13T22:22:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.619+0000 7ff822bff700 1 -- 172.21.15.60:0/3746954321 wait complete. 2023-07-13T22:22:26.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.621+0000 7ff822bff700 1 Processor -- start 2023-07-13T22:22:26.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.621+0000 7ff822bff700 1 -- start start 2023-07-13T22:22:26.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff822bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0b8400 0x7ff81c154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff822bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff81c0cfca0 0x7ff81c149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff822bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff81c0d3750 0x7ff81c1f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff822bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c15e340 con 0x7ff81c0cfca0 2023-07-13T22:22:26.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff822bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c15c970 con 0x7ff81c0d3750 2023-07-13T22:22:26.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff822bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c15e590 con 0x7ff81c0b8400 2023-07-13T22:22:26.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff82099b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0b8400 0x7ff81c154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff82119c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff81c0d3750 0x7ff81c1f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff81bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff81c0cfca0 0x7ff81c149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff82099b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0b8400 0x7ff81c154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:52312/0 (socket says 172.21.15.60:52312) 2023-07-13T22:22:26.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.622+0000 7ff82099b700 1 -- 172.21.15.60:0/4132691372 learned_addr learned my addr 172.21.15.60:0/4132691372 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:26.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff81bfff700 1 -- 172.21.15.60:0/4132691372 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0b8400 msgr2=0x7ff81c154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff81bfff700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0b8400 0x7ff81c154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff81bfff700 1 -- 172.21.15.60:0/4132691372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff81c0d3750 msgr2=0x7ff81c1f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff81bfff700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff81c0d3750 0x7ff81c1f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff81bfff700 1 -- 172.21.15.60:0/4132691372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814009a70 con 0x7ff81c0cfca0 2023-07-13T22:22:26.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff82119c700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff81c0d3750 0x7ff81c1f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:26.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff81bfff700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff81c0cfca0 0x7ff81c149cd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c390 tx=0x7ff80800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff8197fa700 1 -- 172.21.15.60:0/4132691372 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8080052d0 con 0x7ff81c0cfca0 2023-07-13T22:22:26.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c15e870 con 0x7ff81c0cfca0 2023-07-13T22:22:26.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff8197fa700 1 -- 172.21.15.60:0/4132691372 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff808005430 con 0x7ff81c0cfca0 2023-07-13T22:22:26.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff8197fa700 1 -- 172.21.15.60:0/4132691372 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8080038a0 con 0x7ff81c0cfca0 2023-07-13T22:22:26.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.623+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c15ee30 con 0x7ff81c0cfca0 2023-07-13T22:22:26.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.624+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c15b4b0 con 0x7ff81c0cfca0 2023-07-13T22:22:26.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.626+0000 7ff8197fa700 1 -- 172.21.15.60:0/4132691372 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff808003a00 con 0x7ff81c0cfca0 2023-07-13T22:22:26.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.627+0000 7ff8197fa700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff80006ace0 0x7ff80006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:26.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.627+0000 7ff8197fa700 1 -- 172.21.15.60:0/4132691372 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff8080888f0 con 0x7ff81c0cfca0 2023-07-13T22:22:26.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.627+0000 7ff82099b700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff80006ace0 0x7ff80006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:26.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.627+0000 7ff82099b700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff80006ace0 0x7ff80006d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff810000f60 tx=0x7ff810007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:26.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.631+0000 7ff8197fa700 1 -- 172.21.15.60:0/4132691372 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff8080537f0 con 0x7ff81c0cfca0 2023-07-13T22:22:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.884+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff81c15b1d0 con 0x7ff81c0cfca0 2023-07-13T22:22:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.885+0000 7ff8197fa700 1 -- 172.21.15.60:0/4132691372 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff808017060 con 0x7ff81c0cfca0 2023-07-13T22:22:26.868 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:22:26.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff80006ace0 msgr2=0x7ff80006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff80006ace0 0x7ff80006d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff810000f60 tx=0x7ff810007ea0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff81c0cfca0 msgr2=0x7ff81c149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff81c0cfca0 0x7ff81c149cd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c390 tx=0x7ff80800c6a0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 shutdown_connections 2023-07-13T22:22:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff80006ace0 0x7ff80006d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff81c0d3750 0x7ff81c1f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff81c0b8400 0x7ff81c154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 --2- 172.21.15.60:0/4132691372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff81c0cfca0 0x7ff81c149cd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 >> 172.21.15.60:0/4132691372 conn(0x7ff81c06b390 msgr2=0x7ff81c0b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 shutdown_connections 2023-07-13T22:22:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:26.887+0000 7ff822bff700 1 -- 172.21.15.60:0/4132691372 wait complete. 2023-07-13T22:22:26.888 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:22:26.889 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:22:26.889 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:22:26.889 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:22:26.890 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:22:26.890 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:22:26.890 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:22:26.890 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:22:26.891 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:22:26.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:26.891 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:22:26.939 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:22:26.939 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:22:26.940 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:22:26.940 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:26.940 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:22:26.940 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:26.940 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:22:26.940 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:26.940 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:22:26.941 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:22:26.941 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:26.941 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:22:26.941 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:22:26.941 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:26.941 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:22:26.941 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:22:26.941 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41626sec preferred_lft 41626sec 2023-07-13T22:22:26.942 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:22:26.942 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:26.942 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:22:26.942 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:26.942 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:22:26.942 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:22:26.942 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:26.943 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:22:26.943 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:26.943 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:22:26.943 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:26.943 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:22:26.943 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:22:26.943 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:22:26.943 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:26.944 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:22:26.944 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:22:26.944 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:22:26.944 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:26.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:26.945 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:22:27.013 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:22:27.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:27.017 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:22:27.066 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:22:27.066 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:22:27.067 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:22:27.067 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:22:27.067 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:22:27.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:27.068 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:22:27.069 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:22:27.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:22:27.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:22:27.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:22:27.070 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:22:27.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:27.242 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:22:27.288 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:22:27.289 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:22:27.289 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:22:27.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:27.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:22:27.362 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:22:27.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:22:27.362 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:22:27.408 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:22:27.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:22:27.409 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:22:27.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:22:27.605 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:22:27.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:22:27.676 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:22:27.678 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:22:27.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:27.680 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:22:27.727 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:22:27.728 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:22:27.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:22:27.805 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:22:27.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:22:27.878 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:22:27.880 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:22:27.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:27.882 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:22:27.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:22:27.922+0000 7f20a04dd5c0 -1 init, newargv = 0x55ec6b401c30 newargc=15 2023-07-13T22:22:27.906 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[128160]: starting ceph client 2023-07-13T22:22:27.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[128160]: starting fuse 2023-07-13T22:22:27.934 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:22:27.934 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:22:27.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:22:27.935 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:22:27.935 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:22:27.936 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:22:27.936 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:22:27.937 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:27.937 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:22:27.937 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:22:27.938 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:22:27.938 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:22:27.938 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:27.939 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:22:27.939 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:22:27.940 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:22:27.940 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:22:27.940 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:27.940 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:22:27.941 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:22:27.941 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:22:27.941 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:22:27.942 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:22:27.942 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:22:27.942 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:22:27.942 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:22:27.943 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:27.943 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:22:27.943 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:22:28.121 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.128160.asok 2023-07-13T22:22:28.298 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.128160.asok 2023-07-13T22:22:28.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:28.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.128160.asok status 2023-07-13T22:22:28.530 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:22:28.530 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:22:28.530 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:22:28.530 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:22:28.530 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:22:28.531 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:22:28.531 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:22:28.531 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "128160", 2023-07-13T22:22:28.531 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:22:28.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:28.531 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:22:28.531 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:22:28.532 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5529, 2023-07-13T22:22:28.532 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:22:28.532 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:22:28.532 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:22:28.532 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5529 2023-07-13T22:22:28.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:28.532 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:22:28.533 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:22:28.533 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:22:28.533 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1582720856 2023-07-13T22:22:28.533 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:28.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:28.533 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:22:28.533 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:22:28.533 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:22:28.534 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1582720856 2023-07-13T22:22:28.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:28.534 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5529 192.168.0.1:0/1582720856", 2023-07-13T22:22:28.534 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/1582720856", 2023-07-13T22:22:28.534 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:22:28.534 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 16, 2023-07-13T22:22:28.535 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 35, 2023-07-13T22:22:28.535 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:22:28.535 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:22:28.535 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:22:28.535 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:22:28.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:28.544 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:22:28.545 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:22:28.591 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:22:28.592 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:22:28.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:28.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:22:28.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:22:28.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.921+0000 7f587a5ff700 1 Processor -- start 2023-07-13T22:22:28.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.921+0000 7f587a5ff700 1 -- start start 2023-07-13T22:22:28.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.923+0000 7f587a5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5874154950 0x7f5874070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:28.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.923+0000 7f587a5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f587415a820 0x7f587415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:28.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.923+0000 7f587a5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5874070270 con 0x7f58740c8d40 2023-07-13T22:22:28.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.923+0000 7f587a5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587415b2e0 con 0x7f587415a820 2023-07-13T22:22:28.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.923+0000 7f587a5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587415b5c0 con 0x7f5874154950 2023-07-13T22:22:28.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.923+0000 7f58737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f587415a820 0x7f587415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:28.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.923+0000 7f58737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f587415a820 0x7f587415ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57788/0 (socket says 172.21.15.60:57788) 2023-07-13T22:22:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.923+0000 7f58737fe700 1 -- 172.21.15.60:0/931875659 learned_addr learned my addr 172.21.15.60:0/931875659 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.923+0000 7f5873fff700 1 --2- 172.21.15.60:0/931875659 >> v2:172.21.15.105:3300/0 conn(0x7f5874154950 0x7f5874070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.924+0000 7f58737fe700 1 -- 172.21.15.60:0/931875659 >> v2:172.21.15.105:3300/0 conn(0x7f5874154950 msgr2=0x7f5874070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.924+0000 7f58737fe700 1 --2- 172.21.15.60:0/931875659 >> v2:172.21.15.105:3300/0 conn(0x7f5874154950 0x7f5874070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.924+0000 7f58737fe700 1 -- 172.21.15.60:0/931875659 >> v1:172.21.15.105:6789/0 conn(0x7f58740c8d40 legacy=0x7f58740c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.924+0000 7f58737fe700 1 -- 172.21.15.60:0/931875659 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740704c0 con 0x7f587415a820 2023-07-13T22:22:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.924+0000 7f58737fe700 1 --2- 172.21.15.60:0/931875659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587415a820 0x7f587415ac70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f58740ce6e0 tx=0x7f5868009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ae6fe1ffafb05d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.924+0000 7f5872ffd700 1 -- 172.21.15.60:0/931875659 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586800b870 con 0x7f587415a820 2023-07-13T22:22:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.924+0000 7f5872ffd700 1 -- 172.21.15.60:0/931875659 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586800b9d0 con 0x7f587415a820 2023-07-13T22:22:28.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.924+0000 7f5872ffd700 1 -- 172.21.15.60:0/931875659 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586800bd60 con 0x7f587415a820 2023-07-13T22:22:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.925+0000 7f587a5ff700 1 -- 172.21.15.60:0/931875659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587415a820 msgr2=0x7f587415ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.925+0000 7f587a5ff700 1 --2- 172.21.15.60:0/931875659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587415a820 0x7f587415ac70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f58740ce6e0 tx=0x7f5868009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.925+0000 7f587a5ff700 1 -- 172.21.15.60:0/931875659 shutdown_connections 2023-07-13T22:22:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.925+0000 7f587a5ff700 1 --2- 172.21.15.60:0/931875659 >> v2:172.21.15.105:3300/0 conn(0x7f5874154950 0x7f5874070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.925+0000 7f587a5ff700 1 --2- 172.21.15.60:0/931875659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587415a820 0x7f587415ac70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.925+0000 7f587a5ff700 1 -- 172.21.15.60:0/931875659 >> 172.21.15.60:0/931875659 conn(0x7f58740cf410 msgr2=0x7f58740cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:28.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.925+0000 7f587a5ff700 1 -- 172.21.15.60:0/931875659 shutdown_connections 2023-07-13T22:22:28.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.925+0000 7f587a5ff700 1 -- 172.21.15.60:0/931875659 wait complete. 2023-07-13T22:22:28.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.927+0000 7f587a5ff700 1 Processor -- start 2023-07-13T22:22:28.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.928+0000 7f587a5ff700 1 -- start start 2023-07-13T22:22:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f587a5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58740c8d40 0x7f587406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:28.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f587a5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5874154950 0x7f58740c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f587a5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f587415a820 0x7f587414ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f587a5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58740c7fa0 con 0x7f58740c8d40 2023-07-13T22:22:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f587a5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58740c8280 con 0x7f587415a820 2023-07-13T22:22:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f587a5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58740c8560 con 0x7f5874154950 2023-07-13T22:22:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f5873fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58740c8d40 0x7f587406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f5873fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58740c8d40 0x7f587406d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57800/0 (socket says 172.21.15.60:57800) 2023-07-13T22:22:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f5873fff700 1 -- 172.21.15.60:0/542766565 learned_addr learned my addr 172.21.15.60:0/542766565 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f5873fff700 1 -- 172.21.15.60:0/542766565 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5874154950 msgr2=0x7f58740c7550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:22:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f5873fff700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5874154950 0x7f58740c7550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f5873fff700 1 -- 172.21.15.60:0/542766565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f587415a820 msgr2=0x7f587414ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f5873fff700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f587415a820 0x7f587414ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:28.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.929+0000 7f5873fff700 1 -- 172.21.15.60:0/542766565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868009550 con 0x7f58740c8d40 2023-07-13T22:22:28.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.930+0000 7f5873fff700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58740c8d40 0x7f587406d4c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f586400e810 tx=0x7f586400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:28.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.930+0000 7f5870ff9700 1 -- 172.21.15.60:0/542766565 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864013610 con 0x7f58740c8d40 2023-07-13T22:22:28.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.930+0000 7f5870ff9700 1 -- 172.21.15.60:0/542766565 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5864013770 con 0x7f58740c8d40 2023-07-13T22:22:28.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.930+0000 7f5870ff9700 1 -- 172.21.15.60:0/542766565 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864013b20 con 0x7f58740c8d40 2023-07-13T22:22:28.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.930+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58740cc2c0 con 0x7f58740c8d40 2023-07-13T22:22:28.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.930+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587415d7c0 con 0x7f58740c8d40 2023-07-13T22:22:28.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.931+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58740cc7e0 con 0x7f58740c8d40 2023-07-13T22:22:28.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.932+0000 7f5870ff9700 1 -- 172.21.15.60:0/542766565 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5864013c80 con 0x7f58740c8d40 2023-07-13T22:22:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.933+0000 7f5870ff9700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f586006ad10 0x7f586006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.933+0000 7f5870ff9700 1 -- 172.21.15.60:0/542766565 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5864089210 con 0x7f58740c8d40 2023-07-13T22:22:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.933+0000 7f58737fe700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f586006ad10 0x7f586006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.934+0000 7f58737fe700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f586006ad10 0x7f586006d1c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f58741f56d0 tx=0x7f5868008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:28.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:28.937+0000 7f5870ff9700 1 -- 172.21.15.60:0/542766565 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5864054220 con 0x7f58740c8d40 2023-07-13T22:22:29.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.189+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f58740c80e0 con 0x7f58740c8d40 2023-07-13T22:22:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.190+0000 7f5870ff9700 1 -- 172.21.15.60:0/542766565 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f58740c80e0 con 0x7f58740c8d40 2023-07-13T22:22:29.173 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:22:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f586006ad10 msgr2=0x7f586006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f586006ad10 0x7f586006d1c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f58741f56d0 tx=0x7f5868008030 comp rx=0 tx=0).stop 2023-07-13T22:22:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58740c8d40 msgr2=0x7f587406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58740c8d40 0x7f587406d4c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f586400e810 tx=0x7f586400ebd0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 shutdown_connections 2023-07-13T22:22:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f586006ad10 0x7f586006d1c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58740c8d40 0x7f587406d4c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f587415a820 0x7f587414ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 --2- 172.21.15.60:0/542766565 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5874154950 0x7f58740c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.192+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 >> 172.21.15.60:0/542766565 conn(0x7f58740cf410 msgr2=0x7f5874151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.193+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 shutdown_connections 2023-07-13T22:22:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:29.193+0000 7f587a5ff700 1 -- 172.21.15.60:0/542766565 wait complete. 2023-07-13T22:22:29.193 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:22:29.194 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:22:29.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:22:29.194 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:22:29.195 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:22:29.195 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:22:29.195 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:22:29.196 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:22:29.196 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:22:29.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:29.197 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:22:29.249 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:22:29.249 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:22:29.249 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:22:29.249 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:29.249 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:22:29.250 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:29.250 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:22:29.250 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:29.250 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:22:29.250 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:22:29.250 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:29.250 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:22:29.250 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:22:29.251 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:29.251 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:22:29.251 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:29.251 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:22:29.251 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41621sec preferred_lft 41621sec 2023-07-13T22:22:29.251 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:22:29.251 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:29.251 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:22:29.252 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:22:29.252 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:22:29.252 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:29.252 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:22:29.252 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:29.252 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:22:29.252 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:22:29.253 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:22:29.253 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:29.253 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:22:29.253 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:22:29.253 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::14e3:bfff:fea8:bf57/64 scope link 2023-07-13T22:22:29.253 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:22:29.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:29.255 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:22:29.325 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:22:29.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:29.329 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:22:29.378 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:22:29.378 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:22:29.378 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:22:29.379 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:22:29.379 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:22:29.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:29.380 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:22:29.381 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:22:29.381 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:22:29.381 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:22:29.382 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:22:29.383 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:22:29.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:29.553 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:22:29.599 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:22:29.599 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:22:29.600 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:22:29.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:29.602 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:22:29.711 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:22:29.712 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:22:29.712 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:22:29.758 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:22:29.758 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:22:29.759 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:22:29.843 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:22:29.953 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:22:29.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:22:30.022 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:22:30.025 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:22:30.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:30.026 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:22:30.072 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:22:30.073 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:22:30.117 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:22:30.188 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:22:30.188 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:22:30.257 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:22:30.259 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:22:30.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:30.260 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:22:30.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:22:30.303+0000 7f336660e5c0 -1 init, newargv = 0x56426d613060 newargc=15 2023-07-13T22:22:30.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[120845]: starting ceph client 2023-07-13T22:22:30.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[120845]: starting fuse 2023-07-13T22:22:30.308 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:22:30.308 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:22:30.309 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:22:30.309 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:22:30.310 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:22:30.310 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:22:30.310 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:22:30.311 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:22:30.311 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:22:30.311 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:22:30.311 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:22:30.312 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:22:30.312 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:22:30.312 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:22:30.313 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:22:30.313 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:22:30.313 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:22:30.314 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:22:30.314 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:22:30.315 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:22:30.315 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:22:30.315 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:22:30.316 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:22:30.316 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:22:30.316 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:22:30.316 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:22:30.317 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:22:30.317 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:22:30.317 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:22:30.492 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.120845.asok 2023-07-13T22:22:30.671 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.120845.asok 2023-07-13T22:22:30.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:30.672 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.120845.asok status 2023-07-13T22:22:30.895 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:22:30.896 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:22:30.896 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:22:30.896 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:22:30.896 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:22:30.896 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:22:30.896 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:22:30.896 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "120845", 2023-07-13T22:22:30.897 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:22:30.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:22:30.897 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:22:30.897 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:22:30.897 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5114, 2023-07-13T22:22:30.897 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:22:30.897 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:22:30.897 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:22:30.898 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 5114 2023-07-13T22:22:30.898 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:22:30.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:22:30.898 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:22:30.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:22:30.898 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1448251474 2023-07-13T22:22:30.898 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:22:30.898 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:22:30.899 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:22:30.899 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:22:30.899 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:22:30.899 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1448251474 2023-07-13T22:22:30.899 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:22:30.899 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.5114 192.168.0.1:0/1448251474", 2023-07-13T22:22:30.899 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/1448251474", 2023-07-13T22:22:30.899 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:22:30.900 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 16, 2023-07-13T22:22:30.900 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 35, 2023-07-13T22:22:30.900 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:22:30.900 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:22:30.900 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:22:30.900 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:22:30.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:30.910 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:22:30.910 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:22:30.956 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:22:30.956 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:22:30.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:30.957 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:22:31.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 500 2023-07-13T22:22:31.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.260+0000 7fcd4edff700 1 Processor -- start 2023-07-13T22:22:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.261+0000 7fcd4edff700 1 -- start start 2023-07-13T22:22:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd4edff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b8200 0x7fcd480cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd4edff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd480cede0 0x7fcd480cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd4edff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd480d1870 con 0x7fcd480b1ac0 2023-07-13T22:22:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd4edff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd480cf370 con 0x7fcd480cede0 2023-07-13T22:22:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd4edff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd480d12b0 con 0x7fcd480b8200 2023-07-13T22:22:31.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd4d39c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd480cede0 0x7fcd480cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd4d39c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd480cede0 0x7fcd480cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57804/0 (socket says 172.21.15.60:57804) 2023-07-13T22:22:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd47fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcd480b1ac0 0x7fcd4815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48608/0 (socket says 172.21.15.60:48608) 2023-07-13T22:22:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd4d39c700 1 -- 172.21.15.60:0/3905369247 learned_addr learned my addr 172.21.15.60:0/3905369247 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.262+0000 7fcd4cb9b700 1 --2- 172.21.15.60:0/3905369247 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b8200 0x7fcd480cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd4d39c700 1 -- 172.21.15.60:0/3905369247 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b8200 msgr2=0x7fcd480cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd4d39c700 1 --2- 172.21.15.60:0/3905369247 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b8200 0x7fcd480cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd4d39c700 1 -- 172.21.15.60:0/3905369247 >> v1:172.21.15.60:6789/0 conn(0x7fcd480b1ac0 legacy=0x7fcd4815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd4d39c700 1 -- 172.21.15.60:0/3905369247 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd480d1590 con 0x7fcd480cede0 2023-07-13T22:22:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd477fe700 1 -- 172.21.15.60:0/3905369247 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158462578 0 0) 0x7fcd480d1870 con 0x7fcd480b1ac0 2023-07-13T22:22:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd4cb9b700 1 --2- 172.21.15.60:0/3905369247 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b8200 0x7fcd480cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd4d39c700 1 --2- 172.21.15.60:0/3905369247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480cede0 0x7fcd480cfb80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009e50 tx=0x7fcd4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98e39cd4ab3575e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd477fe700 1 -- 172.21.15.60:0/3905369247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000e070 con 0x7fcd480cede0 2023-07-13T22:22:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd477fe700 1 -- 172.21.15.60:0/3905369247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4000b9c0 con 0x7fcd480cede0 2023-07-13T22:22:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.263+0000 7fcd477fe700 1 -- 172.21.15.60:0/3905369247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000bd70 con 0x7fcd480cede0 2023-07-13T22:22:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.264+0000 7fcd4edff700 1 -- 172.21.15.60:0/3905369247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480cede0 msgr2=0x7fcd480cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.264+0000 7fcd4edff700 1 --2- 172.21.15.60:0/3905369247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480cede0 0x7fcd480cfb80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009e50 tx=0x7fcd4000b330 comp rx=0 tx=0).stop 2023-07-13T22:22:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.264+0000 7fcd4edff700 1 -- 172.21.15.60:0/3905369247 shutdown_connections 2023-07-13T22:22:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.264+0000 7fcd4edff700 1 --2- 172.21.15.60:0/3905369247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480cede0 0x7fcd480cfb80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.264+0000 7fcd4edff700 1 --2- 172.21.15.60:0/3905369247 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b8200 0x7fcd480cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.264+0000 7fcd4edff700 1 -- 172.21.15.60:0/3905369247 >> 172.21.15.60:0/3905369247 conn(0x7fcd4806b2f0 msgr2=0x7fcd4806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.264+0000 7fcd4edff700 1 -- 172.21.15.60:0/3905369247 shutdown_connections 2023-07-13T22:22:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.264+0000 7fcd4edff700 1 -- 172.21.15.60:0/3905369247 wait complete. 2023-07-13T22:22:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.266+0000 7fcd4edff700 1 Processor -- start 2023-07-13T22:22:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.266+0000 7fcd4edff700 1 -- start start 2023-07-13T22:22:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd4edff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b1ac0 0x7fcd480c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd4edff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480b8200 0x7fcd480c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd4edff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd480cede0 0x7fcd480b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd4edff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd481e54e0 con 0x7fcd480b8200 2023-07-13T22:22:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd4edff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd481e6300 con 0x7fcd480cede0 2023-07-13T22:22:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd4edff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd480c82f0 con 0x7fcd480b1ac0 2023-07-13T22:22:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd47fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480b8200 0x7fcd480c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd47fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480b8200 0x7fcd480c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57820/0 (socket says 172.21.15.60:57820) 2023-07-13T22:22:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd47fff700 1 -- 172.21.15.60:0/1843294222 learned_addr learned my addr 172.21.15.60:0/1843294222 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd4cb9b700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b1ac0 0x7fcd480c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.267+0000 7fcd4d39c700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd480cede0 0x7fcd480b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd47fff700 1 -- 172.21.15.60:0/1843294222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b1ac0 msgr2=0x7fcd480c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd47fff700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b1ac0 0x7fcd480c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd47fff700 1 -- 172.21.15.60:0/1843294222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd480cede0 msgr2=0x7fcd480b6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd47fff700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd480cede0 0x7fcd480b6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd47fff700 1 -- 172.21.15.60:0/1843294222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40009b30 con 0x7fcd480b8200 2023-07-13T22:22:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd47fff700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480b8200 0x7fcd480c8ed0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcd340011e0 tx=0x7fcd3400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd457fa700 1 -- 172.21.15.60:0/1843294222 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34004c80 con 0x7fcd480b8200 2023-07-13T22:22:31.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd457fa700 1 -- 172.21.15.60:0/1843294222 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd34004e00 con 0x7fcd480b8200 2023-07-13T22:22:31.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd481e56f0 con 0x7fcd480b8200 2023-07-13T22:22:31.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd457fa700 1 -- 172.21.15.60:0/1843294222 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34007310 con 0x7fcd480b8200 2023-07-13T22:22:31.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.268+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd481e5cb0 con 0x7fcd480b8200 2023-07-13T22:22:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.269+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd481e64b0 con 0x7fcd480b8200 2023-07-13T22:22:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.270+0000 7fcd457fa700 1 -- 172.21.15.60:0/1843294222 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd34007470 con 0x7fcd480b8200 2023-07-13T22:22:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.271+0000 7fcd457fa700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2c06ace0 0x7fcd2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.272+0000 7fcd457fa700 1 -- 172.21.15.60:0/1843294222 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd34088fe0 con 0x7fcd480b8200 2023-07-13T22:22:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.272+0000 7fcd4cb9b700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2c06ace0 0x7fcd2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.272+0000 7fcd4cb9b700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2c06ace0 0x7fcd2c06d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c006f10 tx=0x7fcd3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.276+0000 7fcd457fa700 1 -- 172.21.15.60:0/1843294222 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcd481e64b0 con 0x7fcd480b8200 2023-07-13T22:22:31.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.483+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) v1 -- 0x7fcd481e6470 con 0x7fcd480b8200 2023-07-13T22:22:31.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.486+0000 7fcd457fa700 1 -- 172.21.15.60:0/1843294222 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v11)=0 v11) v1 ==== 123+0+0 (secure 0 0 0) 0x7fcd34054120 con 0x7fcd480b8200 2023-07-13T22:22:31.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.488+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2c06ace0 msgr2=0x7fcd2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.488+0000 7fcd4edff700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2c06ace0 0x7fcd2c06d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c006f10 tx=0x7fcd3c008040 comp rx=0 tx=0).stop 2023-07-13T22:22:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.488+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480b8200 msgr2=0x7fcd480c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.488+0000 7fcd4edff700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480b8200 0x7fcd480c8ed0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcd340011e0 tx=0x7fcd3400ca30 comp rx=0 tx=0).stop 2023-07-13T22:22:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.488+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 shutdown_connections 2023-07-13T22:22:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.489+0000 7fcd4edff700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2c06ace0 0x7fcd2c06d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.489+0000 7fcd4edff700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd480cede0 0x7fcd480b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.489+0000 7fcd4edff700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd480b1ac0 0x7fcd480c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.489+0000 7fcd4edff700 1 --2- 172.21.15.60:0/1843294222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd480b8200 0x7fcd480c8ed0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.489+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 >> 172.21.15.60:0/1843294222 conn(0x7fcd4806b2f0 msgr2=0x7fcd480b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.489+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 shutdown_connections 2023-07-13T22:22:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.489+0000 7fcd4edff700 1 -- 172.21.15.60:0/1843294222 wait complete. 2023-07-13T22:22:31.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 250 2023-07-13T22:22:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.711+0000 7f0f3cbff700 1 Processor -- start 2023-07-13T22:22:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.711+0000 7f0f3cbff700 1 -- start start 2023-07-13T22:22:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.712+0000 7f0f3cbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f380add00 0x7f0f3806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.712+0000 7f0f3cbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0f3806a5e0 0x7f0f380b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.712+0000 7f0f3cbff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f380c8e90 con 0x7f0f380ca9f0 2023-07-13T22:22:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.712+0000 7f0f3cbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3806d3b0 con 0x7f0f3806a5e0 2023-07-13T22:22:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.712+0000 7f0f3cbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380ae0d0 con 0x7f0f380add00 2023-07-13T22:22:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f35d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0f380ca9f0 0x7f0f38069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48622/0 (socket says 172.21.15.60:48622) 2023-07-13T22:22:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f35d9b700 1 -- 172.21.15.60:0/1750014072 learned_addr learned my addr 172.21.15.60:0/1750014072 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f3659c700 1 --2- 172.21.15.60:0/1750014072 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f380add00 0x7f0f3806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f36d9d700 1 --2- 172.21.15.60:0/1750014072 >> v2:172.21.15.105:3300/0 conn(0x7f0f3806a5e0 0x7f0f380b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f3559a700 1 -- 172.21.15.60:0/1750014072 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914066923 0 0) 0x7f0f380c8e90 con 0x7f0f380ca9f0 2023-07-13T22:22:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f3559a700 1 -- 172.21.15.60:0/1750014072 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f1c0035b0 con 0x7f0f380ca9f0 2023-07-13T22:22:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f3659c700 1 -- 172.21.15.60:0/1750014072 >> v2:172.21.15.105:3300/0 conn(0x7f0f3806a5e0 msgr2=0x7f0f380b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f3659c700 1 --2- 172.21.15.60:0/1750014072 >> v2:172.21.15.105:3300/0 conn(0x7f0f3806a5e0 0x7f0f380b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f3659c700 1 -- 172.21.15.60:0/1750014072 >> v1:172.21.15.60:6789/0 conn(0x7f0f380ca9f0 legacy=0x7f0f38069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f3659c700 1 -- 172.21.15.60:0/1750014072 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f380c90e0 con 0x7f0f380add00 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.713+0000 7f0f36d9d700 1 --2- 172.21.15.60:0/1750014072 >> v2:172.21.15.105:3300/0 conn(0x7f0f3806a5e0 0x7f0f380b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.714+0000 7f0f3659c700 1 --2- 172.21.15.60:0/1750014072 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f380add00 0x7f0f3806ce70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009840 tx=0x7f0f28009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=613c1b615d643905 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.714+0000 7f0f3559a700 1 -- 172.21.15.60:0/1750014072 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28011830 con 0x7f0f380add00 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.714+0000 7f0f3559a700 1 -- 172.21.15.60:0/1750014072 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f28011990 con 0x7f0f380add00 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.714+0000 7f0f3559a700 1 -- 172.21.15.60:0/1750014072 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28011d20 con 0x7f0f380add00 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.714+0000 7f0f3cbff700 1 -- 172.21.15.60:0/1750014072 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f380add00 msgr2=0x7f0f3806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.714+0000 7f0f3cbff700 1 --2- 172.21.15.60:0/1750014072 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f380add00 0x7f0f3806ce70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009840 tx=0x7f0f28009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.715+0000 7f0f3cbff700 1 -- 172.21.15.60:0/1750014072 shutdown_connections 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.715+0000 7f0f3cbff700 1 --2- 172.21.15.60:0/1750014072 >> v2:172.21.15.105:3300/0 conn(0x7f0f3806a5e0 0x7f0f380b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.715+0000 7f0f3cbff700 1 --2- 172.21.15.60:0/1750014072 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f380add00 0x7f0f3806ce70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.715+0000 7f0f3cbff700 1 -- 172.21.15.60:0/1750014072 >> 172.21.15.60:0/1750014072 conn(0x7f0f380cbf50 msgr2=0x7f0f380ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.715+0000 7f0f3cbff700 1 -- 172.21.15.60:0/1750014072 shutdown_connections 2023-07-13T22:22:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.715+0000 7f0f3cbff700 1 -- 172.21.15.60:0/1750014072 wait complete. 2023-07-13T22:22:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.716+0000 7f0f3cbff700 1 Processor -- start 2023-07-13T22:22:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.717+0000 7f0f3cbff700 1 -- start start 2023-07-13T22:22:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f3cbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f3806a5e0 0x7f0f381f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f3cbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f380add00 0x7f0f381f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f3cbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f380ca9f0 0x7f0f381556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f3cbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380b0b90 con 0x7f0f380ca9f0 2023-07-13T22:22:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f3cbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380cc7d0 con 0x7f0f380add00 2023-07-13T22:22:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f3cbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f381f2ea0 con 0x7f0f3806a5e0 2023-07-13T22:22:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f36d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f380ca9f0 0x7f0f381556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f36d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f380ca9f0 0x7f0f381556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57828/0 (socket says 172.21.15.60:57828) 2023-07-13T22:22:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f36d9d700 1 -- 172.21.15.60:0/398938358 learned_addr learned my addr 172.21.15.60:0/398938358 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f3659c700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f3806a5e0 0x7f0f381f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f35d9b700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f380add00 0x7f0f381f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f36d9d700 1 -- 172.21.15.60:0/398938358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f3806a5e0 msgr2=0x7f0f381f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f36d9d700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f3806a5e0 0x7f0f381f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f36d9d700 1 -- 172.21.15.60:0/398938358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f380add00 msgr2=0x7f0f381f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f36d9d700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f380add00 0x7f0f381f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.718+0000 7f0f36d9d700 1 -- 172.21.15.60:0/398938358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30009a70 con 0x7f0f380ca9f0 2023-07-13T22:22:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.719+0000 7f0f36d9d700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f380ca9f0 0x7f0f381556d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000efc0 tx=0x7f0f3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.719+0000 7f0f2effd700 1 -- 172.21.15.60:0/398938358 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000cc90 con 0x7f0f380ca9f0 2023-07-13T22:22:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.719+0000 7f0f2effd700 1 -- 172.21.15.60:0/398938358 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3000f040 con 0x7f0f380ca9f0 2023-07-13T22:22:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.719+0000 7f0f2effd700 1 -- 172.21.15.60:0/398938358 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f300146c0 con 0x7f0f380ca9f0 2023-07-13T22:22:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.719+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f280094a0 con 0x7f0f380ca9f0 2023-07-13T22:22:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.719+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f381f3460 con 0x7f0f380ca9f0 2023-07-13T22:22:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.720+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f381f35f0 con 0x7f0f380ca9f0 2023-07-13T22:22:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.721+0000 7f0f2effd700 1 -- 172.21.15.60:0/398938358 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f30014820 con 0x7f0f380ca9f0 2023-07-13T22:22:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.722+0000 7f0f2effd700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0f2006ace0 0x7f0f2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.722+0000 7f0f2effd700 1 -- 172.21.15.60:0/398938358 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f3008a0e0 con 0x7f0f380ca9f0 2023-07-13T22:22:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.722+0000 7f0f3659c700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0f2006ace0 0x7f0f2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.723+0000 7f0f3659c700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0f2006ace0 0x7f0f2006d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0f28000c00 tx=0x7f0f28008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.727+0000 7f0f2effd700 1 -- 172.21.15.60:0/398938358 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0f300550f0 con 0x7f0f380ca9f0 2023-07-13T22:22:31.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.932+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) v1 -- 0x7f0f3804e210 con 0x7f0f380ca9f0 2023-07-13T22:22:31.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.937+0000 7f0f2effd700 1 -- 172.21.15.60:0/398938358 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v12)=0 v12) v1 ==== 136+0+0 (secure 0 0 0) 0x7f0f30058710 con 0x7f0f380ca9f0 2023-07-13T22:22:31.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0f2006ace0 msgr2=0x7f0f2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0f2006ace0 0x7f0f2006d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0f28000c00 tx=0x7f0f28008030 comp rx=0 tx=0).stop 2023-07-13T22:22:31.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f380ca9f0 msgr2=0x7f0f381556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f380ca9f0 0x7f0f381556d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000efc0 tx=0x7f0f3000c3a0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 shutdown_connections 2023-07-13T22:22:31.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0f2006ace0 0x7f0f2006d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f380ca9f0 0x7f0f381556d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f3806a5e0 0x7f0f381f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 --2- 172.21.15.60:0/398938358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f380add00 0x7f0f381f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 >> 172.21.15.60:0/398938358 conn(0x7f0f380cbf50 msgr2=0x7f0f3815aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 shutdown_connections 2023-07-13T22:22:31.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:31.939+0000 7f0f3cbff700 1 -- 172.21.15.60:0/398938358 wait complete. 2023-07-13T22:22:31.940 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:22:31.940 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:31.987 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:22:31.987 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:22:31.987 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2023-07-13T22:22:31.988 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:31.988 DEBUG:teuthology.orchestra.run.smithi060:> n = 250 2023-07-13T22:22:31.988 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2023-07-13T22:22:31.988 DEBUG:teuthology.orchestra.run.smithi060:> if 0 > 0: 2023-07-13T22:22:31.989 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2023-07-13T22:22:31.989 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2023-07-13T22:22:31.989 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:31.990 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2023-07-13T22:22:31.990 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2023-07-13T22:22:31.990 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2023-07-13T22:22:31.990 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:31.991 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-07-13T22:22:31.991 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-13T22:22:31.991 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:31.991 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2023-07-13T22:22:31.992 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-13T22:22:31.992 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2023-07-13T22:22:31.992 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2023-07-13T22:22:31.992 DEBUG:teuthology.orchestra.run.smithi060:> if True: 2023-07-13T22:22:31.993 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2023-07-13T22:22:31.993 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2023-07-13T22:22:31.993 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:31.993 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2023-07-13T22:22:31.994 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:31.994 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:31.994 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2023-07-13T22:22:31.994 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(0): 2023-07-13T22:22:31.995 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-13T22:22:31.995 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:31.995 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:31.995 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2023-07-13T22:22:31.996 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2023-07-13T22:22:31.996 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:22:33.527 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:22:33.528 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:33.614 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:22:33.615 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:22:33.615 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2023-07-13T22:22:33.615 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:33.615 DEBUG:teuthology.orchestra.run.smithi060:> n = 250 2023-07-13T22:22:33.616 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2023-07-13T22:22:33.616 DEBUG:teuthology.orchestra.run.smithi060:> if 0 > 0: 2023-07-13T22:22:33.616 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2023-07-13T22:22:33.617 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2023-07-13T22:22:33.617 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:33.617 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2023-07-13T22:22:33.617 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2023-07-13T22:22:33.618 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2023-07-13T22:22:33.618 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:33.618 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-07-13T22:22:33.618 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-13T22:22:33.619 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:33.619 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2023-07-13T22:22:33.619 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-13T22:22:33.619 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2023-07-13T22:22:33.620 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2023-07-13T22:22:33.620 DEBUG:teuthology.orchestra.run.smithi060:> if True: 2023-07-13T22:22:33.620 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2023-07-13T22:22:33.620 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2023-07-13T22:22:33.621 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:33.621 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2023-07-13T22:22:33.621 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:33.622 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:33.622 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2023-07-13T22:22:33.622 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(0): 2023-07-13T22:22:33.622 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-13T22:22:33.623 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:33.623 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:33.623 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2023-07-13T22:22:33.623 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2023-07-13T22:22:33.624 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:22:35.154 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:22:35.154 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:35.240 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:22:35.241 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:22:35.241 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2023-07-13T22:22:35.241 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:35.241 DEBUG:teuthology.orchestra.run.smithi060:> n = 250 2023-07-13T22:22:35.242 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2023-07-13T22:22:35.242 DEBUG:teuthology.orchestra.run.smithi060:> if 0 > 0: 2023-07-13T22:22:35.242 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2023-07-13T22:22:35.242 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2023-07-13T22:22:35.243 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:35.243 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2023-07-13T22:22:35.243 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2023-07-13T22:22:35.244 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2023-07-13T22:22:35.244 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:35.244 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-07-13T22:22:35.244 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-13T22:22:35.245 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:35.245 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2023-07-13T22:22:35.245 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-13T22:22:35.245 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2023-07-13T22:22:35.246 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2023-07-13T22:22:35.246 DEBUG:teuthology.orchestra.run.smithi060:> if True: 2023-07-13T22:22:35.246 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2023-07-13T22:22:35.246 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2023-07-13T22:22:35.247 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:35.247 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2023-07-13T22:22:35.247 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:35.248 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:35.248 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2023-07-13T22:22:35.248 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(0): 2023-07-13T22:22:35.248 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-13T22:22:35.249 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:35.249 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:35.249 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2023-07-13T22:22:35.249 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2023-07-13T22:22:35.250 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:22:36.981 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:22:36.982 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:37.068 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:22:37.068 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:22:37.069 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2023-07-13T22:22:37.069 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:37.069 DEBUG:teuthology.orchestra.run.smithi060:> n = 250 2023-07-13T22:22:37.070 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2023-07-13T22:22:37.070 DEBUG:teuthology.orchestra.run.smithi060:> if 0 > 0: 2023-07-13T22:22:37.070 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2023-07-13T22:22:37.070 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2023-07-13T22:22:37.071 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:37.071 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2023-07-13T22:22:37.071 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2023-07-13T22:22:37.071 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2023-07-13T22:22:37.072 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:37.072 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-07-13T22:22:37.072 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-13T22:22:37.073 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:37.073 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2023-07-13T22:22:37.073 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-13T22:22:37.073 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2023-07-13T22:22:37.074 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2023-07-13T22:22:37.074 DEBUG:teuthology.orchestra.run.smithi060:> if True: 2023-07-13T22:22:37.074 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2023-07-13T22:22:37.074 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2023-07-13T22:22:37.075 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:37.075 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2023-07-13T22:22:37.075 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:37.075 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:37.076 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2023-07-13T22:22:37.076 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(0): 2023-07-13T22:22:37.076 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-13T22:22:37.077 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:37.077 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:37.080 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2023-07-13T22:22:37.081 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2023-07-13T22:22:37.081 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:22:38.808 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:22:38.809 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:38.895 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:22:38.896 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:22:38.896 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2023-07-13T22:22:38.896 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:38.897 DEBUG:teuthology.orchestra.run.smithi060:> n = 250 2023-07-13T22:22:38.897 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2023-07-13T22:22:38.897 DEBUG:teuthology.orchestra.run.smithi060:> if 0 > 0: 2023-07-13T22:22:38.898 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2023-07-13T22:22:38.898 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/dir5/file" 2023-07-13T22:22:38.898 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:38.898 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2023-07-13T22:22:38.899 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2023-07-13T22:22:38.899 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2023-07-13T22:22:38.899 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:38.899 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-07-13T22:22:38.900 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-13T22:22:38.900 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:38.900 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2023-07-13T22:22:38.901 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-13T22:22:38.901 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2023-07-13T22:22:38.901 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2023-07-13T22:22:38.901 DEBUG:teuthology.orchestra.run.smithi060:> if True: 2023-07-13T22:22:38.902 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2023-07-13T22:22:38.902 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2023-07-13T22:22:38.902 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:38.902 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2023-07-13T22:22:38.903 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:38.903 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:38.903 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2023-07-13T22:22:38.904 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(0): 2023-07-13T22:22:38.904 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-13T22:22:38.904 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:38.904 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:38.905 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2023-07-13T22:22:38.905 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2023-07-13T22:22:38.905 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:22:40.637 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:22:40.637 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:40.724 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:22:40.724 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:22:40.725 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2023-07-13T22:22:40.725 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:40.725 DEBUG:teuthology.orchestra.run.smithi060:> n = 250 2023-07-13T22:22:40.726 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2023-07-13T22:22:40.726 DEBUG:teuthology.orchestra.run.smithi060:> if 0 > 0: 2023-07-13T22:22:40.726 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2023-07-13T22:22:40.726 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/dir6/file" 2023-07-13T22:22:40.727 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:40.727 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2023-07-13T22:22:40.727 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2023-07-13T22:22:40.728 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2023-07-13T22:22:40.728 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:40.728 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-07-13T22:22:40.728 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-13T22:22:40.729 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:40.729 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2023-07-13T22:22:40.730 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-13T22:22:40.730 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2023-07-13T22:22:40.730 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2023-07-13T22:22:40.730 DEBUG:teuthology.orchestra.run.smithi060:> if True: 2023-07-13T22:22:40.731 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2023-07-13T22:22:40.731 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2023-07-13T22:22:40.731 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:40.732 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2023-07-13T22:22:40.732 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:40.732 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:40.733 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2023-07-13T22:22:40.733 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(0): 2023-07-13T22:22:40.733 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-13T22:22:40.733 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:22:40.734 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:22:40.734 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2023-07-13T22:22:40.734 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2023-07-13T22:22:40.735 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:22:42.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:22:42.666 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:22:42.666 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:22:42.666 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:22:42.667 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:22:42.667 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:42.667 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:22:42.668 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:22:42.668 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:22:42.668 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:22:42.669 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:42.669 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:22:42.669 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:22:42.669 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:22:42.670 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:22:42.670 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:42.670 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:22:42.671 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:22:42.671 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:22:42.671 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:22:42.672 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:22:42.672 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:22:42.672 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:22:42.673 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:22:42.673 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:22:42.673 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:22:42.674 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:22:42.855 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.128160.asok 2023-07-13T22:22:43.031 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.128160.asok 2023-07-13T22:22:43.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:43.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.128160.asok mds_sessions 2023-07-13T22:22:43.262 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:22:43.263 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5529, 2023-07-13T22:22:43.263 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:22:43.263 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:22:43.263 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:22:43.264 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5529 2023-07-13T22:22:43.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:43.264 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:22:43.264 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:22:43.264 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:22:43.265 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1582720856 2023-07-13T22:22:43.265 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:43.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:43.265 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5529 v1:192.168.0.1:0/1582720856", 2023-07-13T22:22:43.265 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "v1:192.168.0.1:0/1582720856", 2023-07-13T22:22:43.265 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions": [ 2023-07-13T22:22:43.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:43.266 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": 0, 2023-07-13T22:22:43.266 INFO:teuthology.orchestra.run.smithi060.stdout: "addrs": { 2023-07-13T22:22:43.266 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2023-07-13T22:22:43.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:43.266 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2023-07-13T22:22:43.266 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6836", 2023-07-13T22:22:43.267 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2048874349 2023-07-13T22:22:43.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:43.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:22:43.267 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:22:43.267 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6837", 2023-07-13T22:22:43.267 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2048874349 2023-07-13T22:22:43.268 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:43.268 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:22:43.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:22:43.268 INFO:teuthology.orchestra.run.smithi060.stdout: "seq": 3006, 2023-07-13T22:22:43.268 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_gen": 0, 2023-07-13T22:22:43.268 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_ttl": "2023-07-13T22:23:28.925861+0000", 2023-07-13T22:22:43.269 INFO:teuthology.orchestra.run.smithi060.stdout: "last_cap_renew_request": "2023-07-13T22:22:28.925861+0000", 2023-07-13T22:22:43.269 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_renew_seq": 2, 2023-07-13T22:22:43.269 INFO:teuthology.orchestra.run.smithi060.stdout: "num_caps": 198, 2023-07-13T22:22:43.269 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "open" 2023-07-13T22:22:43.269 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:22:43.269 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2023-07-13T22:22:43.270 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 16 2023-07-13T22:22:43.270 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:22:43.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:43.277 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'find | wc') 2023-07-13T22:22:47.022 INFO:teuthology.orchestra.run.smithi060.stdout: 1507 1507 23384 2023-07-13T22:22:47.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:47.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.252+0000 7fcb1a3ff700 1 Processor -- start 2023-07-13T22:22:47.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.253+0000 7fcb1a3ff700 1 -- start start 2023-07-13T22:22:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.254+0000 7fcb1a3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcb140c9450 0x7fcb1415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.254+0000 7fcb1a3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb14070270 con 0x7fcb1415a820 2023-07-13T22:22:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.254+0000 7fcb1a3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1415b2e0 con 0x7fcb14154950 2023-07-13T22:22:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.254+0000 7fcb1a3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb140713d0 con 0x7fcb140c9450 2023-07-13T22:22:47.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.254+0000 7fcb137fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcb1415a820 0x7fcb140c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45178/0 (socket says 172.21.15.60:45178) 2023-07-13T22:22:47.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.254+0000 7fcb137fe700 1 -- 172.21.15.60:0/675651633 learned_addr learned my addr 172.21.15.60:0/675651633 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:47.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb1899c700 1 --2- 172.21.15.60:0/675651633 >> v2:172.21.15.105:3300/0 conn(0x7fcb140c9450 0x7fcb1415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251536293 0 0) 0x7fcb14070270 con 0x7fcb1415a820 2023-07-13T22:22:47.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcafc0035b0 con 0x7fcb1415a820 2023-07-13T22:22:47.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 534200064 0 0) 0x7fcafc0035b0 con 0x7fcb1415a820 2023-07-13T22:22:47.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 >> v2:172.21.15.105:3300/0 conn(0x7fcb140c9450 msgr2=0x7fcb1415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:47.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 --2- 172.21.15.60:0/675651633 >> v2:172.21.15.105:3300/0 conn(0x7fcb140c9450 0x7fcb1415ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 >> v1:172.21.15.105:6789/0 conn(0x7fcb14154950 legacy=0x7fcb14070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1415b5c0 con 0x7fcb1415a820 2023-07-13T22:22:47.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fcb08001670 con 0x7fcb1415a820 2023-07-13T22:22:47.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 >> v1:172.21.15.60:6789/0 conn(0x7fcb1415a820 legacy=0x7fcb140c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcafc003a50 0x7fcafc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcafc004850 0x7fcafc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcafc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc00c130 con 0x7fcafc005250 2023-07-13T22:22:47.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc00c2a0 con 0x7fcafc004850 2023-07-13T22:22:47.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc00c460 con 0x7fcafc003a50 2023-07-13T22:22:47.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.255+0000 7fcb1899c700 1 --2- 172.21.15.60:0/675651633 >> v2:172.21.15.105:3300/0 conn(0x7fcb140c9450 0x7fcb1415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:47.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb13fff700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcafc003a50 0x7fcafc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb1899c700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcafc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb137fe700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcafc004850 0x7fcafc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb1899c700 1 -- 172.21.15.60:0/675651633 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcafc003a50 msgr2=0x7fcafc003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:47.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb1899c700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcafc003a50 0x7fcafc003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb1899c700 1 -- 172.21.15.60:0/675651633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcafc004850 msgr2=0x7fcafc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:47.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb1899c700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcafc004850 0x7fcafc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb1899c700 1 -- 172.21.15.60:0/675651633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0035b0 con 0x7fcafc005250 2023-07-13T22:22:47.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb1899c700 1 -- 172.21.15.60:0/675651633 reap_dead start 2023-07-13T22:22:47.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb1899c700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcafc00ba10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e350 tx=0x7fcafc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da77005e0b99a5c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:47.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.256+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00ee10 con 0x7fcafc005250 2023-07-13T22:22:47.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.257+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c00f040 con 0x7fcafc005250 2023-07-13T22:22:47.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.257+0000 7fcb12ffd700 1 -- 172.21.15.60:0/675651633 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c003c60 con 0x7fcafc005250 2023-07-13T22:22:47.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.257+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/675651633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 msgr2=0x7fcafc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.257+0000 7fcb1a3ff700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcafc00ba10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e350 tx=0x7fcafc005880 comp rx=0 tx=0).stop 2023-07-13T22:22:47.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.257+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/675651633 shutdown_connections 2023-07-13T22:22:47.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.257+0000 7fcb1a3ff700 1 --2- 172.21.15.60:0/675651633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcafc00ba10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.257+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/675651633 >> 172.21.15.60:0/675651633 conn(0x7fcb140cf410 msgr2=0x7fcb140cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:47.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.257+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/675651633 shutdown_connections 2023-07-13T22:22:47.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.257+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/675651633 wait complete. 2023-07-13T22:22:47.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.260+0000 7fcb1a3ff700 1 Processor -- start 2023-07-13T22:22:47.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.260+0000 7fcb1a3ff700 1 -- start start 2023-07-13T22:22:47.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.261+0000 7fcb1a3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcb1414f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.261+0000 7fcb1a3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb1414fd60 0x7fcb140c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.261+0000 7fcb1a3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb140aed50 0x7fcb140c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.261+0000 7fcb1a3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1414fba0 con 0x7fcafc005250 2023-07-13T22:22:47.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.261+0000 7fcb1a3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140af310 con 0x7fcb1414fd60 2023-07-13T22:22:47.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.261+0000 7fcb1a3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140af5f0 con 0x7fcb140aed50 2023-07-13T22:22:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.261+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcb1414f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.261+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcb1414f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40806/0 (socket says 172.21.15.60:40806) 2023-07-13T22:22:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.261+0000 7fcb13fff700 1 -- 172.21.15.60:0/191040207 learned_addr learned my addr 172.21.15.60:0/191040207 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb137fe700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb1414fd60 0x7fcb140c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb1899c700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb140aed50 0x7fcb140c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb13fff700 1 -- 172.21.15.60:0/191040207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb140aed50 msgr2=0x7fcb140c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb13fff700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb140aed50 0x7fcb140c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb13fff700 1 -- 172.21.15.60:0/191040207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb1414fd60 msgr2=0x7fcb140c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb13fff700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb1414fd60 0x7fcb140c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb13fff700 1 -- 172.21.15.60:0/191040207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140af8d0 con 0x7fcafc005250 2023-07-13T22:22:47.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb13fff700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcb1414f660 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400f7f0 tx=0x7fcb0400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:47.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb10ff9700 1 -- 172.21.15.60:0/191040207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb040052e0 con 0x7fcafc005250 2023-07-13T22:22:47.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb10ff9700 1 -- 172.21.15.60:0/191040207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04005440 con 0x7fcafc005250 2023-07-13T22:22:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb10ff9700 1 -- 172.21.15.60:0/191040207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb040038c0 con 0x7fcafc005250 2023-07-13T22:22:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb140afbb0 con 0x7fcafc005250 2023-07-13T22:22:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.262+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb14150520 con 0x7fcafc005250 2023-07-13T22:22:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.263+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb140b00d0 con 0x7fcafc005250 2023-07-13T22:22:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.265+0000 7fcb10ff9700 1 -- 172.21.15.60:0/191040207 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb04003a20 con 0x7fcafc005250 2023-07-13T22:22:47.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.266+0000 7fcb10ff9700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcb0006ace0 0x7fcb0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.266+0000 7fcb10ff9700 1 -- 172.21.15.60:0/191040207 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fcb040891b0 con 0x7fcafc005250 2023-07-13T22:22:47.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.266+0000 7fcb137fe700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcb0006ace0 0x7fcb0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.267+0000 7fcb137fe700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcb0006ace0 0x7fcb0006d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb08000c00 tx=0x7fcb08000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:47.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.271+0000 7fcb10ff9700 1 -- 172.21.15.60:0/191040207 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcb140b00d0 con 0x7fcafc005250 2023-07-13T22:22:47.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.523+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb140b00d0 con 0x7fcafc005250 2023-07-13T22:22:47.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.524+0000 7fcb10ff9700 1 -- 172.21.15.60:0/191040207 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fcb140b00d0 con 0x7fcafc005250 2023-07-13T22:22:47.507 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:47.507 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2157014443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2157014443},{"type":"v1","addr":"172.21.15.105:6834","nonce":2157014443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/302067138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":302067138},{"type":"v1","addr":"172.21.15.105:6835","nonce":302067138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3461450784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461450784},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461450784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:22:19.175830+0000","modified":"2023-07-13T22:22:22.691356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6837/2048874349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2048874349},{"type":"v1","addr":"172.21.15.60:6837","nonce":2048874349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-13T22:22:47.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcb0006ace0 msgr2=0x7fcb0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcb0006ace0 0x7fcb0006d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb08000c00 tx=0x7fcb08000ed0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 msgr2=0x7fcb1414f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcb1414f660 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400f7f0 tx=0x7fcb0400fb00 comp rx=0 tx=0).stop 2023-07-13T22:22:47.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 shutdown_connections 2023-07-13T22:22:47.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcb0006ace0 0x7fcb0006d190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcafc005250 0x7fcb1414f660 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb140aed50 0x7fcb140c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 --2- 172.21.15.60:0/191040207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb1414fd60 0x7fcb140c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.526+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 >> 172.21.15.60:0/191040207 conn(0x7fcb140cf410 msgr2=0x7fcb140b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.527+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 shutdown_connections 2023-07-13T22:22:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.527+0000 7fcb1a3ff700 1 -- 172.21.15.60:0/191040207 wait complete. 2023-07-13T22:22:47.512 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2023-07-13T22:22:47.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-13T22:22:47.756 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5114,"entity":{"name":{"type":"client","num":5114},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1448251474}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":17.449814517,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.8898061987658532,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5114 192.168.0.1:0/1448251474","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"120845","root":"/"}},{"id":5529,"entity":{"name":{"type":"client","num":5529},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1582720856}},"state":"open","num_leases":6,"num_caps":350,"request_load_avg":1554,"uptime":19.831789222000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":1971.6906155212673,"halflife":60},"release_caps":{"value":2323.5057245522212,"halflife":60},"recall_caps_throttle":{"value":830.99551104365526,"halflife":1.5},"recall_caps_throttle2o":{"value":158.94346424405452,"halflife":0.5},"session_cache_liveness":{"value":4374.1629339081846,"halflife":300},"cap_acquisition":{"value":1209.885895892493,"halflife":10},"last_trim_completed_requests_tid":1742,"last_trim_completed_flushes_tid":1501,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5529 192.168.0.1:0/1582720856","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000007d6","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"128160","root":"/"}}] 2023-07-13T22:22:47.757 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "120845", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1448251474, "type": "any" }, "name": { "num": 5114, "type": "client" } }, "id": 5114, "inst": "client.5114 192.168.0.1:0/1448251474", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.889806198765853 }, "state": "open", "uptime": 17.449814517 }, { "cap_acquisition": { "halflife": 10, "value": 1209.885895892493 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "128160", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1582720856, "type": "any" }, "name": { "num": 5529, "type": "client" } }, "id": 5529, "inst": "client.5529 192.168.0.1:0/1582720856", "last_trim_completed_flushes_tid": 1501, "last_trim_completed_requests_tid": 1742, "num_caps": 350, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 6, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 1000, "start": "0x100000007d6" } ], "recall_caps": { "halflife": 60, "value": 1971.6906155212673 }, "recall_caps_throttle": { "halflife": 1.5, "value": 830.9955110436553 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 158.94346424405452 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 2323.505724552221 }, "request_load_avg": 1554, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 4374.162933908185 }, "state": "open", "uptime": 19.831789222 } ] 2023-07-13T22:22:47.757 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:47.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.977+0000 7f42767ff700 1 Processor -- start 2023-07-13T22:22:47.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.977+0000 7f42767ff700 1 -- start start 2023-07-13T22:22:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.978+0000 7f42767ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f427015c6e0 0x7f427006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.978+0000 7f42767ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 0x7f42700b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.978+0000 7f42767ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427006b8e0 con 0x7f42700b3050 2023-07-13T22:22:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.978+0000 7f42767ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427006b6f0 con 0x7f427015c6e0 2023-07-13T22:22:47.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.978+0000 7f42767ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42700b2e90 con 0x7f42700b24b0 2023-07-13T22:22:47.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.978+0000 7f426f7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f42700b3050 0x7f427006afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45186/0 (socket says 172.21.15.60:45186) 2023-07-13T22:22:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.978+0000 7f426f7fe700 1 -- 172.21.15.60:0/3484393963 learned_addr learned my addr 172.21.15.60:0/3484393963 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.978+0000 7f4274d9c700 1 --2- 172.21.15.60:0/3484393963 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 0x7f42700b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.978+0000 7f426ffff700 1 --2- 172.21.15.60:0/3484393963 >> v2:172.21.15.105:3300/0 conn(0x7f427015c6e0 0x7f427006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.979+0000 7f426effd700 1 -- 172.21.15.60:0/3484393963 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067817519 0 0) 0x7f427006b8e0 con 0x7f42700b3050 2023-07-13T22:22:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.979+0000 7f426effd700 1 -- 172.21.15.60:0/3484393963 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42580035b0 con 0x7f42700b3050 2023-07-13T22:22:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.979+0000 7f4274d9c700 1 -- 172.21.15.60:0/3484393963 >> v2:172.21.15.105:3300/0 conn(0x7f427015c6e0 msgr2=0x7f427006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:47.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.979+0000 7f4274d9c700 1 --2- 172.21.15.60:0/3484393963 >> v2:172.21.15.105:3300/0 conn(0x7f427015c6e0 0x7f427006c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.979+0000 7f4274d9c700 1 -- 172.21.15.60:0/3484393963 >> v1:172.21.15.60:6789/0 conn(0x7f42700b3050 legacy=0x7f427006afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.979+0000 7f4274d9c700 1 -- 172.21.15.60:0/3484393963 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427006bbc0 con 0x7f42700b24b0 2023-07-13T22:22:47.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.979+0000 7f426ffff700 1 --2- 172.21.15.60:0/3484393963 >> v2:172.21.15.105:3300/0 conn(0x7f427015c6e0 0x7f427006c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:47.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.979+0000 7f4274d9c700 1 --2- 172.21.15.60:0/3484393963 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 0x7f42700b2950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4268009d90 tx=0x7f426800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5830b93b6ce05ec2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:47.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.979+0000 7f426effd700 1 -- 172.21.15.60:0/3484393963 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426800e070 con 0x7f42700b24b0 2023-07-13T22:22:47.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f426effd700 1 -- 172.21.15.60:0/3484393963 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426800bb20 con 0x7f42700b24b0 2023-07-13T22:22:47.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f426effd700 1 -- 172.21.15.60:0/3484393963 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268012430 con 0x7f42700b24b0 2023-07-13T22:22:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f42767ff700 1 -- 172.21.15.60:0/3484393963 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 msgr2=0x7f42700b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f42767ff700 1 --2- 172.21.15.60:0/3484393963 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 0x7f42700b2950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4268009d90 tx=0x7f426800b330 comp rx=0 tx=0).stop 2023-07-13T22:22:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f42767ff700 1 -- 172.21.15.60:0/3484393963 shutdown_connections 2023-07-13T22:22:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f42767ff700 1 --2- 172.21.15.60:0/3484393963 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 0x7f42700b2950 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f42767ff700 1 --2- 172.21.15.60:0/3484393963 >> v2:172.21.15.105:3300/0 conn(0x7f427015c6e0 0x7f427006c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f42767ff700 1 -- 172.21.15.60:0/3484393963 >> 172.21.15.60:0/3484393963 conn(0x7f427006e680 msgr2=0x7f427006ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f42767ff700 1 -- 172.21.15.60:0/3484393963 shutdown_connections 2023-07-13T22:22:47.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.980+0000 7f42767ff700 1 -- 172.21.15.60:0/3484393963 wait complete. 2023-07-13T22:22:47.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.982+0000 7f42767ff700 1 Processor -- start 2023-07-13T22:22:47.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.982+0000 7f42767ff700 1 -- start start 2023-07-13T22:22:47.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.983+0000 7f42767ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 0x7f4270157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.983+0000 7f42767ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42700b3050 0x7f42700b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.983+0000 7f42767ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427015c6e0 0x7f42700c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.983+0000 7f42767ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427015dff0 con 0x7f42700b3050 2023-07-13T22:22:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.983+0000 7f42767ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427015e2d0 con 0x7f427015c6e0 2023-07-13T22:22:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.983+0000 7f42767ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427015e5b0 con 0x7f42700b24b0 2023-07-13T22:22:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.983+0000 7f426f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42700b3050 0x7f42700b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.983+0000 7f426f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42700b3050 0x7f42700b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40814/0 (socket says 172.21.15.60:40814) 2023-07-13T22:22:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.983+0000 7f426f7fe700 1 -- 172.21.15.60:0/3865959594 learned_addr learned my addr 172.21.15.60:0/3865959594 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426ffff700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 0x7f4270157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f4274d9c700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427015c6e0 0x7f42700c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426f7fe700 1 -- 172.21.15.60:0/3865959594 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 msgr2=0x7f4270157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426f7fe700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 0x7f4270157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426f7fe700 1 -- 172.21.15.60:0/3865959594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427015c6e0 msgr2=0x7f42700c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426f7fe700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427015c6e0 0x7f42700c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426f7fe700 1 -- 172.21.15.60:0/3865959594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600094a0 con 0x7f42700b3050 2023-07-13T22:22:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426f7fe700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42700b3050 0x7f42700b5ed0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f426400c700 tx=0x7f426400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:47.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426cff9700 1 -- 172.21.15.60:0/3865959594 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264005d70 con 0x7f42700b3050 2023-07-13T22:22:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426cff9700 1 -- 172.21.15.60:0/3865959594 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42640070d0 con 0x7f42700b3050 2023-07-13T22:22:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f426cff9700 1 -- 172.21.15.60:0/3865959594 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264007460 con 0x7f42700b3050 2023-07-13T22:22:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.984+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4268009a70 con 0x7f42700b3050 2023-07-13T22:22:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.985+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427015eb70 con 0x7f42700b3050 2023-07-13T22:22:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.985+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427004e280 con 0x7f42700b3050 2023-07-13T22:22:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.986+0000 7f426cff9700 1 -- 172.21.15.60:0/3865959594 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42640075c0 con 0x7f42700b3050 2023-07-13T22:22:47.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.987+0000 7f426cff9700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:47.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.987+0000 7f426cff9700 1 -- 172.21.15.60:0/3865959594 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4264088ab0 con 0x7f42700b3050 2023-07-13T22:22:47.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.988+0000 7f426ffff700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:47.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.988+0000 7f426ffff700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4260009cd0 tx=0x7f4260009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:47.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:47.992+0000 7f426cff9700 1 -- 172.21.15.60:0/3865959594 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4264053aa0 con 0x7f42700b3050 2023-07-13T22:22:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.247+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4270070a00 con 0x7f42700b3050 2023-07-13T22:22:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.247+0000 7f426cff9700 1 -- 172.21.15.60:0/3865959594 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f42640570c0 con 0x7f42700b3050 2023-07-13T22:22:48.230 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:48.231 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2157014443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2157014443},{"type":"v1","addr":"172.21.15.105:6834","nonce":2157014443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/302067138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":302067138},{"type":"v1","addr":"172.21.15.105:6835","nonce":302067138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3461450784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461450784},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461450784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:22:19.175830+0000","modified":"2023-07-13T22:22:22.691356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6837/2048874349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2048874349},{"type":"v1","addr":"172.21.15.60:6837","nonce":2048874349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-13T22:22:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.249+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 msgr2=0x7f425c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.249+0000 7f42767ff700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4260009cd0 tx=0x7f4260009210 comp rx=0 tx=0).stop 2023-07-13T22:22:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.249+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42700b3050 msgr2=0x7f42700b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.249+0000 7f42767ff700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42700b3050 0x7f42700b5ed0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f426400c700 tx=0x7f426400ca10 comp rx=0 tx=0).stop 2023-07-13T22:22:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.250+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 shutdown_connections 2023-07-13T22:22:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.250+0000 7f42767ff700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.250+0000 7f42767ff700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427015c6e0 0x7f42700c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.250+0000 7f42767ff700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42700b24b0 0x7f4270157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.250+0000 7f42767ff700 1 --2- 172.21.15.60:0/3865959594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42700b3050 0x7f42700b5ed0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.250+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 >> 172.21.15.60:0/3865959594 conn(0x7f427006e680 msgr2=0x7f4270159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.250+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 shutdown_connections 2023-07-13T22:22:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:48.250+0000 7f42767ff700 1 -- 172.21.15.60:0/3865959594 wait complete. 2023-07-13T22:22:48.235 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2023-07-13T22:22:48.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2023-07-13T22:22:48.478 INFO:teuthology.orchestra.run.smithi060.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":177,"msgr_send_messages":153,"msgr_recv_bytes":146388,"msgr_send_bytes":1287517,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.183508252,"msgr_running_send_time":0.007770175,"msgr_running_recv_time":0.010653541,"msgr_running_fast_dispatch_time":0.162639322,"msgr_send_messages_queue_lat":{"avgcount":152,"sum":0.138675678,"avgtime":0.000912339},"msgr_handle_ack_lat":{"avgcount":28,"sum":0.000004230,"avgtime":0.000000151},"msgr_recv_encrypted_bytes":112384,"msgr_send_encrypted_bytes":12976},"AsyncMessenger::Worker-1":{"msgr_recv_messages":655,"msgr_send_messages":663,"msgr_recv_bytes":161880,"msgr_send_bytes":3186326,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.056035957,"msgr_running_send_time":0.022352884,"msgr_running_recv_time":0.014738032,"msgr_running_fast_dispatch_time":0.013934494,"msgr_send_messages_queue_lat":{"avgcount":662,"sum":0.106104773,"avgtime":0.000160279},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":97952},"AsyncMessenger::Worker-2":{"msgr_recv_messages":5647,"msgr_send_messages":7137,"msgr_recv_bytes":1556771,"msgr_send_bytes":7684996,"msgr_created_connections":8,"msgr_active_connections":7,"msgr_running_total_time":0.606352662,"msgr_running_send_time":0.314219515,"msgr_running_recv_time":0.378303534,"msgr_running_fast_dispatch_time":0.017307461,"msgr_send_messages_queue_lat":{"avgcount":7136,"sum":0.286203013,"avgtime":0.000040106},"msgr_handle_ack_lat":{"avgcount":11147,"sum":0.002599309,"avgtime":0.000000233},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":3057,"sum":1.347696485,"avgtime":0.000440855}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000017252,"avgtime":0.000008626}},"mds":{"request":1744,"reply":1744,"reply_latency":{"avgcount":1744,"sum":4.502448011,"avgtime":0.002581678},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":1516,"inodes_top":0,"inodes_bottom":1516,"inodes_pin_tail":0,"inodes_pinned":1516,"inodes_expired":0,"inodes_with_caps":350,"caps":351,"subtrees":2,"traverse":1981,"traverse_hit":1564,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":146847,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1500,"root_rbytes":3840,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":66,"ceph_cap_op_grant":1500,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1566,"handle_client_caps_dirty":1500,"handle_client_cap_release":6,"process_request_cap_release":1507},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4519,"evex":0,"evtrm":0,"ev":4519,"evexg":0,"evexd":0,"segadd":5,"segex":0,"segtrm":0,"seg":5,"segexg":0,"segexd":0,"expos":4194304,"wrpos":13742247,"rdpos":0,"jlat":{"avgcount":1514,"sum":2.801992646,"avgtime":0.001850721},"replayed":0},"mds_mem":{"ino":1519,"ino+":1519,"ino-":0,"dir":18,"dir+":18,"dir-":0,"dn":1516,"dn+":1516,"dn-":0,"cap":351,"cap+":2819,"cap-":2468,"rss":61304,"heap":207196},"mds_server":{"dispatch_client_request":1762,"dispatch_server_request":0,"handle_client_request":1744,"handle_client_session":1506,"handle_peer_request":0,"req_create_latency":{"avgcount":1500,"sum":0.793676206,"avgtime":0.000529117},"req_getattr_latency":{"avgcount":23,"sum":0.137180385,"avgtime":0.005964364},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":207,"sum":0.032562782,"avgtime":0.000157308},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":6,"sum":0.006886087,"avgtime":0.001147681},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":7,"sum":3.531867493,"avgtime":0.504552499},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000275058,"avgtime":0.000275058},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":7,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":1568,"average_load":784,"avg_session_uptime":15},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":242218,"buffer_anon_items":95,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":6272,"osdmap_items":80,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":4562520,"mds_co_items":29821,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":1558,"op_send_bytes":9639806,"op_resend":26,"op_reply":1558,"op_latency":{"avgcount":1558,"sum":3.650961602,"avgtime":0.002343364},"op_inflight":0,"oplen_avg":{"avgcount":1558,"sum":1587},"op":1558,"op_r":0,"op_w":1558,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":1514,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":8,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":37,"map_full":0,"map_inc":3,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":21,"omap_rd":0,"omap_del":3},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":1744},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":6475,"get_sum":1278150,"get_or_fail_fail":0,"get_or_fail_success":6475,"take":0,"take_sum":0,"put":6475,"put_sum":1278150,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":1558,"take_sum":9639806,"put":1558,"put_sum":9639806,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":1558,"take_sum":1558,"put":1558,"put_sum":1558,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55fb4301fcc0":{"val":0,"max":3758096384,"get_started":0,"get":4519,"get_sum":9547943,"get_or_fail_fail":0,"get_or_fail_success":4519,"take":0,"take_sum":0,"put":1514,"put_sum":9547943,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-07-13T22:22:48.481 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 28, "avgtime": 1.51e-07, "sum": 4.23e-06 }, "msgr_recv_bytes": 146388, "msgr_recv_encrypted_bytes": 112384, "msgr_recv_messages": 177, "msgr_running_fast_dispatch_time": 0.162639322, "msgr_running_recv_time": 0.010653541, "msgr_running_send_time": 0.007770175, "msgr_running_total_time": 0.183508252, "msgr_send_bytes": 1287517, "msgr_send_encrypted_bytes": 12976, "msgr_send_messages": 153, "msgr_send_messages_queue_lat": { "avgcount": 152, "avgtime": 0.000912339, "sum": 0.138675678 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 161880, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 655, "msgr_running_fast_dispatch_time": 0.013934494, "msgr_running_recv_time": 0.014738032, "msgr_running_send_time": 0.022352884, "msgr_running_total_time": 0.056035957, "msgr_send_bytes": 3186326, "msgr_send_encrypted_bytes": 97952, "msgr_send_messages": 663, "msgr_send_messages_queue_lat": { "avgcount": 662, "avgtime": 0.000160279, "sum": 0.106104773 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 7, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 11147, "avgtime": 2.33e-07, "sum": 0.002599309 }, "msgr_recv_bytes": 1556771, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 5647, "msgr_running_fast_dispatch_time": 0.017307461, "msgr_running_recv_time": 0.378303534, "msgr_running_send_time": 0.314219515, "msgr_running_total_time": 0.606352662, "msgr_send_bytes": 7684996, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 7137, "msgr_send_messages_queue_lat": { "avgcount": 7136, "avgtime": 4.0106e-05, "sum": 0.286203013 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 3057, "avgtime": 0.000440855, "sum": 1.347696485 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 8.626e-06, "sum": 1.7252e-05 }, "queue_len": 0 }, "mds": { "caps": 351, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1500, "ceph_cap_op_revoke": 66, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 6, "handle_client_caps": 1566, "handle_client_caps_dirty": 1500, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 1516, "inodes_bottom": 1516, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 1516, "inodes_top": 0, "inodes_with_caps": 350, "load_cent": 146847, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1507, "q": 0, "reply": 1744, "reply_latency": { "avgcount": 1744, "avgtime": 0.002581678, "sum": 4.502448011 }, "request": 1744, "root_rbytes": 3840, "root_rfiles": 1500, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 1981, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 1564, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 4519, "evadd": 4519, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 1514, "avgtime": 0.001850721, "sum": 2.801992646 }, "rdpos": 0, "replayed": 0, "seg": 5, "segadd": 5, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 13742247 }, "mds_mem": { "cap": 351, "cap+": 2819, "cap-": 2468, "dir": 18, "dir+": 18, "dir-": 0, "dn": 1516, "dn+": 1516, "dn-": 0, "heap": 207196, "ino": 1519, "ino+": 1519, "ino-": 0, "rss": 61304 }, "mds_server": { "cap_acquisition_throttle": 7, "cap_revoke_eviction": 0, "dispatch_client_request": 1762, "dispatch_server_request": 0, "handle_client_request": 1744, "handle_client_session": 1506, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1500, "avgtime": 0.000529117, "sum": 0.793676206 }, "req_getattr_latency": { "avgcount": 23, "avgtime": 0.005964364, "sum": 0.137180385 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 207, "avgtime": 0.000157308, "sum": 0.032562782 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 6, "avgtime": 0.001147681, "sum": 0.006886087 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 7, "avgtime": 0.504552499, "sum": 3.531867493 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000275058, "sum": 0.000275058 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 784, "avg_session_uptime": 15, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 1568 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 242218, "buffer_anon_items": 95, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 4562520, "mds_co_items": 29821, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 6272, "osdmap_items": 80, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 37, "map_full": 0, "map_inc": 3, "omap_del": 3, "omap_rd": 0, "omap_wr": 21, "op": 1558, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 1558, "avgtime": 0.002343364, "sum": 3.650961602 }, "op_pg": 0, "op_r": 0, "op_reply": 1558, "op_resend": 26, "op_rmw": 0, "op_send": 1558, "op_send_bytes": 9639806, "op_w": 1558, "oplen_avg": { "avgcount": 1558, "sum": 1587 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 8, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 1514, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 1744 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 6475, "get_or_fail_fail": 0, "get_or_fail_success": 6475, "get_started": 0, "get_sum": 1278150, "max": 104857600, "put": 6475, "put_sum": 1278150, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 1558, "put_sum": 9639806, "take": 1558, "take_sum": 9639806, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 1558, "put_sum": 1558, "take": 1558, "take_sum": 1558, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55fb4301fcc0": { "get": 4519, "get_or_fail_fail": 0, "get_or_fail_success": 4519, "get_started": 0, "get_sum": 9547943, "max": 3758096384, "put": 1514, "put_sum": 9547943, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-07-13T22:22:48.481 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:22:48.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:22:48.584 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:22:48.695 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:22:48.695 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:22:48.696 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:48.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:48.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.001+0000 7fdaa0dff700 1 Processor -- start 2023-07-13T22:22:48.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.002+0000 7fdaa0dff700 1 -- start start 2023-07-13T22:22:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.003+0000 7fdaa0dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda9c154950 0x7fda9c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.003+0000 7fdaa0dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c0c9450 0x7fda9c15ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.003+0000 7fdaa0dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9c070270 con 0x7fda9c15a820 2023-07-13T22:22:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.003+0000 7fdaa0dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9c15b2e0 con 0x7fda9c154950 2023-07-13T22:22:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.003+0000 7fdaa0dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c15b5c0 con 0x7fda9c0c9450 2023-07-13T22:22:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.003+0000 7fda9a59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda9c154950 0x7fda9c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.003+0000 7fda9a59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda9c154950 0x7fda9c070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40826/0 (socket says 172.21.15.60:40826) 2023-07-13T22:22:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.003+0000 7fda9a59c700 1 -- 172.21.15.60:0/2460334173 learned_addr learned my addr 172.21.15.60:0/2460334173 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9ad9d700 1 --2- 172.21.15.60:0/2460334173 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c0c9450 0x7fda9c15ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9a59c700 1 -- 172.21.15.60:0/2460334173 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c0c9450 msgr2=0x7fda9c15ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9a59c700 1 --2- 172.21.15.60:0/2460334173 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c0c9450 0x7fda9c15ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9a59c700 1 -- 172.21.15.60:0/2460334173 >> v1:172.21.15.105:6789/0 conn(0x7fda9c15a820 legacy=0x7fda9c0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:48.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9a59c700 1 -- 172.21.15.60:0/2460334173 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c0704c0 con 0x7fda9c154950 2023-07-13T22:22:48.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9ad9d700 1 --2- 172.21.15.60:0/2460334173 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c0c9450 0x7fda9c15ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:48.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9a59c700 1 --2- 172.21.15.60:0/2460334173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 0x7fda9c070cc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0098f0 tx=0x7fda8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bfa829c475df45 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:48.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9959a700 1 -- 172.21.15.60:0/2460334173 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00b8a0 con 0x7fda9c154950 2023-07-13T22:22:48.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9959a700 1 -- 172.21.15.60:0/2460334173 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c00ba00 con 0x7fda9c154950 2023-07-13T22:22:48.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.004+0000 7fda9959a700 1 -- 172.21.15.60:0/2460334173 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00bd90 con 0x7fda9c154950 2023-07-13T22:22:48.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.005+0000 7fdaa0dff700 1 -- 172.21.15.60:0/2460334173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 msgr2=0x7fda9c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.005+0000 7fdaa0dff700 1 --2- 172.21.15.60:0/2460334173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 0x7fda9c070cc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0098f0 tx=0x7fda8c009c00 comp rx=0 tx=0).stop 2023-07-13T22:22:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.005+0000 7fdaa0dff700 1 -- 172.21.15.60:0/2460334173 shutdown_connections 2023-07-13T22:22:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.005+0000 7fdaa0dff700 1 --2- 172.21.15.60:0/2460334173 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c0c9450 0x7fda9c15ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.005+0000 7fdaa0dff700 1 --2- 172.21.15.60:0/2460334173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 0x7fda9c070cc0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.005+0000 7fdaa0dff700 1 -- 172.21.15.60:0/2460334173 >> 172.21.15.60:0/2460334173 conn(0x7fda9c0cf410 msgr2=0x7fda9c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.005+0000 7fdaa0dff700 1 -- 172.21.15.60:0/2460334173 shutdown_connections 2023-07-13T22:22:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.005+0000 7fdaa0dff700 1 -- 172.21.15.60:0/2460334173 wait complete. 2023-07-13T22:22:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.007+0000 7fdaa0dff700 1 Processor -- start 2023-07-13T22:22:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.008+0000 7fdaa0dff700 1 -- start start 2023-07-13T22:22:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fdaa0dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda9c0c9450 0x7fda9c0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fdaa0dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 0x7fda9c06cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fdaa0dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c15a820 0x7fda9c151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fdaa0dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0c82e0 con 0x7fda9c154950 2023-07-13T22:22:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fdaa0dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0c85c0 con 0x7fda9c0c9450 2023-07-13T22:22:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fdaa0dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0c57c0 con 0x7fda9c15a820 2023-07-13T22:22:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fda99d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 0x7fda9c06cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fda99d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 0x7fda9c06cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40834/0 (socket says 172.21.15.60:40834) 2023-07-13T22:22:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fda99d9b700 1 -- 172.21.15.60:0/1972766774 learned_addr learned my addr 172.21.15.60:0/1972766774 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.009+0000 7fda9ad9d700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c15a820 0x7fda9c151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda9a59c700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda9c0c9450 0x7fda9c0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda99d9b700 1 -- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c15a820 msgr2=0x7fda9c151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda99d9b700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c15a820 0x7fda9c151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda99d9b700 1 -- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda9c0c9450 msgr2=0x7fda9c0c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda99d9b700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda9c0c9450 0x7fda9c0c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda99d9b700 1 -- 172.21.15.60:0/1972766774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c009550 con 0x7fda9c154950 2023-07-13T22:22:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda9ad9d700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c15a820 0x7fda9c151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda9a59c700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda9c0c9450 0x7fda9c0c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda99d9b700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 0x7fda9c06cab0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fda9000c450 tx=0x7fda9000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda8affd700 1 -- 172.21.15.60:0/1972766774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900052d0 con 0x7fda9c154950 2023-07-13T22:22:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda8affd700 1 -- 172.21.15.60:0/1972766774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90005430 con 0x7fda9c154950 2023-07-13T22:22:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fda8affd700 1 -- 172.21.15.60:0/1972766774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900038c0 con 0x7fda9c154950 2023-07-13T22:22:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c0c5aa0 con 0x7fda9c154950 2023-07-13T22:22:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.010+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c0c7f60 con 0x7fda9c154950 2023-07-13T22:22:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.011+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c0c5fc0 con 0x7fda9c154950 2023-07-13T22:22:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.012+0000 7fda8affd700 1 -- 172.21.15.60:0/1972766774 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda90003a20 con 0x7fda9c154950 2023-07-13T22:22:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.014+0000 7fda8affd700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda7c06ace0 0x7fda7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.014+0000 7fda8affd700 1 -- 172.21.15.60:0/1972766774 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fda900888f0 con 0x7fda9c154950 2023-07-13T22:22:48.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.014+0000 7fda9a59c700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda7c06ace0 0x7fda7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:48.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.015+0000 7fda9a59c700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda7c06ace0 0x7fda7c06d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0098f0 tx=0x7fda8c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.018+0000 7fda8affd700 1 -- 172.21.15.60:0/1972766774 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fda9c0c5fc0 con 0x7fda9c154950 2023-07-13T22:22:49.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.223+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda9c0c5fc0 con 0x7fda9c154950 2023-07-13T22:22:49.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.224+0000 7fda8affd700 1 -- 172.21.15.60:0/1972766774 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18250 (secure 0 0 0) 0x7fda9c0c5fc0 con 0x7fda9c154950 2023-07-13T22:22:49.207 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:49.208 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:22:43.507730+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:22:17.071217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-13T22:22:18.161600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:49.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda7c06ace0 msgr2=0x7fda7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda7c06ace0 0x7fda7c06d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0098f0 tx=0x7fda8c008030 comp rx=0 tx=0).stop 2023-07-13T22:22:49.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 msgr2=0x7fda9c06cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 0x7fda9c06cab0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fda9000c450 tx=0x7fda9000c810 comp rx=0 tx=0).stop 2023-07-13T22:22:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 shutdown_connections 2023-07-13T22:22:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda7c06ace0 0x7fda7c06d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda9c15a820 0x7fda9c151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda9c0c9450 0x7fda9c0c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 --2- 172.21.15.60:0/1972766774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda9c154950 0x7fda9c06cab0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 >> 172.21.15.60:0/1972766774 conn(0x7fda9c0cf410 msgr2=0x7fda9c0b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:49.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 shutdown_connections 2023-07-13T22:22:49.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.227+0000 7fdaa0dff700 1 -- 172.21.15.60:0/1972766774 wait complete. 2023-07-13T22:22:49.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:22:17.071217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:22:18.161600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:22:49.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:22:49.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.451+0000 7f8f9f5ff700 1 Processor -- start 2023-07-13T22:22:49.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.451+0000 7f8f9f5ff700 1 -- start start 2023-07-13T22:22:49.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.452+0000 7f8f9f5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8f980add00 0x7f8f9806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.452+0000 7f8f9f5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f980ca9f0 0x7f8f98069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.452+0000 7f8f9f5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f980c8e90 con 0x7f8f9806a410 2023-07-13T22:22:49.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.452+0000 7f8f9f5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9806d3b0 con 0x7f8f980ca9f0 2023-07-13T22:22:49.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.452+0000 7f8f9f5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f980ae0d0 con 0x7f8f980add00 2023-07-13T22:22:49.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.453+0000 7f8f9cb9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f980ca9f0 0x7f8f98069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.453+0000 7f8f9cb9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f980ca9f0 0x7f8f98069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40846/0 (socket says 172.21.15.60:40846) 2023-07-13T22:22:49.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.453+0000 7f8f9cb9a700 1 -- 172.21.15.60:0/1286884234 learned_addr learned my addr 172.21.15.60:0/1286884234 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:49.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.453+0000 7f8f9d39b700 1 --2- 172.21.15.60:0/1286884234 >> v2:172.21.15.105:3300/0 conn(0x7f8f980add00 0x7f8f9806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.453+0000 7f8f9cb9a700 1 -- 172.21.15.60:0/1286884234 >> v2:172.21.15.105:3300/0 conn(0x7f8f980add00 msgr2=0x7f8f9806ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:49.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.453+0000 7f8f9cb9a700 1 --2- 172.21.15.60:0/1286884234 >> v2:172.21.15.105:3300/0 conn(0x7f8f980add00 0x7f8f9806ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.453+0000 7f8f9cb9a700 1 -- 172.21.15.60:0/1286884234 >> v1:172.21.15.105:6789/0 conn(0x7f8f9806a410 legacy=0x7f8f980703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.453+0000 7f8f9cb9a700 1 -- 172.21.15.60:0/1286884234 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9806a820 con 0x7f8f980ca9f0 2023-07-13T22:22:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f9cb9a700 1 --2- 172.21.15.60:0/1286884234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980ca9f0 0x7f8f98069ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8f980cb220 tx=0x7f8f88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aec89f9f2fc8630 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f8ffff700 1 -- 172.21.15.60:0/1286884234 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88011830 con 0x7f8f980ca9f0 2023-07-13T22:22:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f8ffff700 1 -- 172.21.15.60:0/1286884234 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88011990 con 0x7f8f980ca9f0 2023-07-13T22:22:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f8ffff700 1 -- 172.21.15.60:0/1286884234 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88011d20 con 0x7f8f980ca9f0 2023-07-13T22:22:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/1286884234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980ca9f0 msgr2=0x7f8f98069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f9f5ff700 1 --2- 172.21.15.60:0/1286884234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980ca9f0 0x7f8f98069ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8f980cb220 tx=0x7f8f88009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/1286884234 shutdown_connections 2023-07-13T22:22:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f9f5ff700 1 --2- 172.21.15.60:0/1286884234 >> v2:172.21.15.105:3300/0 conn(0x7f8f980add00 0x7f8f9806ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f9f5ff700 1 --2- 172.21.15.60:0/1286884234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980ca9f0 0x7f8f98069ed0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.454+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/1286884234 >> 172.21.15.60:0/1286884234 conn(0x7f8f980cbf50 msgr2=0x7f8f980ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.455+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/1286884234 shutdown_connections 2023-07-13T22:22:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.455+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/1286884234 wait complete. 2023-07-13T22:22:49.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.456+0000 7f8f9f5ff700 1 Processor -- start 2023-07-13T22:22:49.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.456+0000 7f8f9f5ff700 1 -- start start 2023-07-13T22:22:49.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.457+0000 7f8f9f5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f9806a410 0x7f8f980b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.457+0000 7f8f9f5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980add00 0x7f8f98146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.457+0000 7f8f9f5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f980ca9f0 0x7f8f980aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.457+0000 7f8f9f5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980b23e0 con 0x7f8f980add00 2023-07-13T22:22:49.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.457+0000 7f8f9f5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980b2680 con 0x7f8f9806a410 2023-07-13T22:22:49.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.457+0000 7f8f9f5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980b2960 con 0x7f8f980ca9f0 2023-07-13T22:22:49.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9cb9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980add00 0x7f8f98146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9cb9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980add00 0x7f8f98146b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40856/0 (socket says 172.21.15.60:40856) 2023-07-13T22:22:49.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9cb9a700 1 -- 172.21.15.60:0/3705797858 learned_addr learned my addr 172.21.15.60:0/3705797858 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:49.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9d39b700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f9806a410 0x7f8f980b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9cb9a700 1 -- 172.21.15.60:0/3705797858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f980ca9f0 msgr2=0x7f8f980aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9cb9a700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f980ca9f0 0x7f8f980aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9cb9a700 1 -- 172.21.15.60:0/3705797858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f9806a410 msgr2=0x7f8f980b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9cb9a700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f9806a410 0x7f8f980b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9cb9a700 1 -- 172.21.15.60:0/3705797858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f88009550 con 0x7f8f980add00 2023-07-13T22:22:49.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9d39b700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f9806a410 0x7f8f980b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:49.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f9cb9a700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980add00 0x7f8f98146b10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8f88000c00 tx=0x7f8f88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.458+0000 7f8f8dffb700 1 -- 172.21.15.60:0/3705797858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004570 con 0x7f8f980add00 2023-07-13T22:22:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.459+0000 7f8f8dffb700 1 -- 172.21.15.60:0/3705797858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8801f070 con 0x7f8f980add00 2023-07-13T22:22:49.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.459+0000 7f8f8dffb700 1 -- 172.21.15.60:0/3705797858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f880038e0 con 0x7f8f980add00 2023-07-13T22:22:49.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.459+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f980b2c40 con 0x7f8f980add00 2023-07-13T22:22:49.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.459+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f98147050 con 0x7f8f980add00 2023-07-13T22:22:49.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.460+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f980b2e30 con 0x7f8f980add00 2023-07-13T22:22:49.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.461+0000 7f8f8dffb700 1 -- 172.21.15.60:0/3705797858 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f88021030 con 0x7f8f980add00 2023-07-13T22:22:49.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.462+0000 7f8f8dffb700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8f7c06ace0 0x7f8f7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.462+0000 7f8f8dffb700 1 -- 172.21.15.60:0/3705797858 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8f8808a910 con 0x7f8f980add00 2023-07-13T22:22:49.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.462+0000 7f8f9d39b700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8f7c06ace0 0x7f8f7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.462+0000 7f8f9d39b700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8f7c06ace0 0x7f8f7c06d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009b60 tx=0x7f8f94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:49.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.466+0000 7f8f8dffb700 1 -- 172.21.15.60:0/3705797858 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8f88055990 con 0x7f8f980add00 2023-07-13T22:22:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.670+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8f9804e210 con 0x7f8f980add00 2023-07-13T22:22:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.671+0000 7f8f8dffb700 1 -- 172.21.15.60:0/3705797858 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8f88058fb0 con 0x7f8f980add00 2023-07-13T22:22:49.655 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:22:49.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8f7c06ace0 msgr2=0x7f8f7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8f7c06ace0 0x7f8f7c06d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009b60 tx=0x7f8f94008040 comp rx=0 tx=0).stop 2023-07-13T22:22:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980add00 msgr2=0x7f8f98146b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980add00 0x7f8f98146b10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8f88000c00 tx=0x7f8f88004750 comp rx=0 tx=0).stop 2023-07-13T22:22:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 shutdown_connections 2023-07-13T22:22:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8f7c06ace0 0x7f8f7c06d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f980ca9f0 0x7f8f980aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f9806a410 0x7f8f980b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 --2- 172.21.15.60:0/3705797858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f980add00 0x7f8f98146b10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 >> 172.21.15.60:0/3705797858 conn(0x7f8f980cbf50 msgr2=0x7f8f9815aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 shutdown_connections 2023-07-13T22:22:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.673+0000 7f8f9f5ff700 1 -- 172.21.15.60:0/3705797858 wait complete. 2023-07-13T22:22:49.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:22:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.899+0000 7f83ca5ff700 1 Processor -- start 2023-07-13T22:22:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.899+0000 7f83ca5ff700 1 -- start start 2023-07-13T22:22:49.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.900+0000 7f83ca5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83c40add00 0x7f83c406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.900+0000 7f83ca5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c406a5e0 0x7f83c40b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.900+0000 7f83ca5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c40c8e90 con 0x7f83c40ca9f0 2023-07-13T22:22:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.900+0000 7f83ca5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c406d3b0 con 0x7f83c40add00 2023-07-13T22:22:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.900+0000 7f83ca5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40ae0d0 con 0x7f83c406a5e0 2023-07-13T22:22:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.901+0000 7f83c3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83c40add00 0x7f83c406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.901+0000 7f83c3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83c40add00 0x7f83c406ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40858/0 (socket says 172.21.15.60:40858) 2023-07-13T22:22:49.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.901+0000 7f83c3fff700 1 -- 172.21.15.60:0/3891314152 learned_addr learned my addr 172.21.15.60:0/3891314152 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.901+0000 7f83c8b9c700 1 --2- 172.21.15.60:0/3891314152 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c406a5e0 0x7f83c40b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.901+0000 7f83c3fff700 1 -- 172.21.15.60:0/3891314152 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c406a5e0 msgr2=0x7f83c40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.901+0000 7f83c3fff700 1 --2- 172.21.15.60:0/3891314152 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c406a5e0 0x7f83c40b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.901+0000 7f83c3fff700 1 -- 172.21.15.60:0/3891314152 >> v1:172.21.15.60:6789/0 conn(0x7f83c40ca9f0 legacy=0x7f83c4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.901+0000 7f83c3fff700 1 -- 172.21.15.60:0/3891314152 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c40c90e0 con 0x7f83c40add00 2023-07-13T22:22:49.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.901+0000 7f83c2ffd700 1 -- 172.21.15.60:0/3891314152 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748853712 0 0) 0x7f83c40c8e90 con 0x7f83c40ca9f0 2023-07-13T22:22:49.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83c3fff700 1 --2- 172.21.15.60:0/3891314152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c40add00 0x7f83c406ce70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f83b40098f0 tx=0x7f83b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8653d47eb9fa7d65 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:49.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83c2ffd700 1 -- 172.21.15.60:0/3891314152 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400b8a0 con 0x7f83c40add00 2023-07-13T22:22:49.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83c2ffd700 1 -- 172.21.15.60:0/3891314152 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b400ba00 con 0x7f83c40add00 2023-07-13T22:22:49.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83c2ffd700 1 -- 172.21.15.60:0/3891314152 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400bd90 con 0x7f83c40add00 2023-07-13T22:22:49.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83ca5ff700 1 -- 172.21.15.60:0/3891314152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c40add00 msgr2=0x7f83c406ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83ca5ff700 1 --2- 172.21.15.60:0/3891314152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c40add00 0x7f83c406ce70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f83b40098f0 tx=0x7f83b4009c00 comp rx=0 tx=0).stop 2023-07-13T22:22:49.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83ca5ff700 1 -- 172.21.15.60:0/3891314152 shutdown_connections 2023-07-13T22:22:49.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83ca5ff700 1 --2- 172.21.15.60:0/3891314152 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c406a5e0 0x7f83c40b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83ca5ff700 1 --2- 172.21.15.60:0/3891314152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c40add00 0x7f83c406ce70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.902+0000 7f83ca5ff700 1 -- 172.21.15.60:0/3891314152 >> 172.21.15.60:0/3891314152 conn(0x7f83c40cbf50 msgr2=0x7f83c40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:49.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.903+0000 7f83ca5ff700 1 -- 172.21.15.60:0/3891314152 shutdown_connections 2023-07-13T22:22:49.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.903+0000 7f83ca5ff700 1 -- 172.21.15.60:0/3891314152 wait complete. 2023-07-13T22:22:49.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.904+0000 7f83ca5ff700 1 Processor -- start 2023-07-13T22:22:49.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.904+0000 7f83ca5ff700 1 -- start start 2023-07-13T22:22:49.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.905+0000 7f83ca5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c406a5e0 0x7f83c40c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.905+0000 7f83ca5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c40add00 0x7f83c40cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.905+0000 7f83ca5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83c40ca9f0 0x7f83c41f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.905+0000 7f83ca5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40ae330 con 0x7f83c406a5e0 2023-07-13T22:22:49.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.905+0000 7f83ca5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c41f6350 con 0x7f83c40ca9f0 2023-07-13T22:22:49.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.905+0000 7f83ca5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c41f6630 con 0x7f83c40add00 2023-07-13T22:22:49.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c406a5e0 0x7f83c40c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c406a5e0 0x7f83c40c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40872/0 (socket says 172.21.15.60:40872) 2023-07-13T22:22:49.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c3fff700 1 -- 172.21.15.60:0/633110921 learned_addr learned my addr 172.21.15.60:0/633110921 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:49.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c37fe700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c40add00 0x7f83c40cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c8b9c700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83c40ca9f0 0x7f83c41f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c3fff700 1 -- 172.21.15.60:0/633110921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c40add00 msgr2=0x7f83c40cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c3fff700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c40add00 0x7f83c40cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c3fff700 1 -- 172.21.15.60:0/633110921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83c40ca9f0 msgr2=0x7f83c41f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c3fff700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83c40ca9f0 0x7f83c41f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c3fff700 1 -- 172.21.15.60:0/633110921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4009550 con 0x7f83c406a5e0 2023-07-13T22:22:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c8b9c700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83c40ca9f0 0x7f83c41f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.906+0000 7f83c3fff700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c406a5e0 0x7f83c40c81c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009c40 tx=0x7f83b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.907+0000 7f83c0ff9700 1 -- 172.21.15.60:0/633110921 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4004300 con 0x7f83c406a5e0 2023-07-13T22:22:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.907+0000 7f83c0ff9700 1 -- 172.21.15.60:0/633110921 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b401f070 con 0x7f83c406a5e0 2023-07-13T22:22:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.907+0000 7f83c0ff9700 1 -- 172.21.15.60:0/633110921 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b40036a0 con 0x7f83c406a5e0 2023-07-13T22:22:49.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.907+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c40c76e0 con 0x7f83c406a5e0 2023-07-13T22:22:49.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.907+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c40c7ca0 con 0x7f83c406a5e0 2023-07-13T22:22:49.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.908+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c404e210 con 0x7f83c406a5e0 2023-07-13T22:22:49.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.909+0000 7f83c0ff9700 1 -- 172.21.15.60:0/633110921 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83b4021030 con 0x7f83c406a5e0 2023-07-13T22:22:49.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.910+0000 7f83c0ff9700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83b006ace0 0x7f83b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:49.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.910+0000 7f83c0ff9700 1 -- 172.21.15.60:0/633110921 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f83b408a810 con 0x7f83c406a5e0 2023-07-13T22:22:49.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.910+0000 7f83c37fe700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83b006ace0 0x7f83b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:49.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.911+0000 7f83c37fe700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83b006ace0 0x7f83b006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ad5d0 tx=0x7f83b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:49.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:49.914+0000 7f83c0ff9700 1 -- 172.21.15.60:0/633110921 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f83b4055890 con 0x7f83c406a5e0 2023-07-13T22:22:50.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.119+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83c406d7b0 con 0x7f83c406a5e0 2023-07-13T22:22:50.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.119+0000 7f83c0ff9700 1 -- 172.21.15.60:0/633110921 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83b4006d80 con 0x7f83c406a5e0 2023-07-13T22:22:50.104 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:22:50.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83b006ace0 msgr2=0x7f83b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83b006ace0 0x7f83b006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ad5d0 tx=0x7f83b8009040 comp rx=0 tx=0).stop 2023-07-13T22:22:50.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c406a5e0 msgr2=0x7f83c40c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c406a5e0 0x7f83c40c81c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009c40 tx=0x7f83b4004750 comp rx=0 tx=0).stop 2023-07-13T22:22:50.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 shutdown_connections 2023-07-13T22:22:50.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83b006ace0 0x7f83b006d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c406a5e0 0x7f83c40c81c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83c40ca9f0 0x7f83c41f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 --2- 172.21.15.60:0/633110921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83c40add00 0x7f83c40cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 >> 172.21.15.60:0/633110921 conn(0x7f83c40cbf50 msgr2=0x7f83c40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 shutdown_connections 2023-07-13T22:22:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.122+0000 7f83ca5ff700 1 -- 172.21.15.60:0/633110921 wait complete. 2023-07-13T22:22:50.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:22:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.345+0000 7f922edff700 1 Processor -- start 2023-07-13T22:22:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.346+0000 7f922edff700 1 -- start start 2023-07-13T22:22:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.347+0000 7f922edff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 0x7f922806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.347+0000 7f922edff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f92280ca9f0 0x7f9228069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.347+0000 7f922edff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92280c8e90 con 0x7f922806a410 2023-07-13T22:22:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.347+0000 7f922edff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922806d3b0 con 0x7f92280ca9f0 2023-07-13T22:22:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.347+0000 7f922edff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922806a890 con 0x7f92280add00 2023-07-13T22:22:50.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.347+0000 7f922d39c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f922806a410 0x7f92280703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45208/0 (socket says 172.21.15.60:45208) 2023-07-13T22:22:50.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.347+0000 7f922d39c700 1 -- 172.21.15.60:0/943350250 learned_addr learned my addr 172.21.15.60:0/943350250 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:50.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.347+0000 7f9227fff700 1 --2- 172.21.15.60:0/943350250 >> v2:172.21.15.105:3300/0 conn(0x7f92280ca9f0 0x7f9228069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.347+0000 7f922cb9b700 1 --2- 172.21.15.60:0/943350250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 0x7f922806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f92277fe700 1 -- 172.21.15.60:0/943350250 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661040684 0 0) 0x7f92280c8e90 con 0x7f922806a410 2023-07-13T22:22:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f92277fe700 1 -- 172.21.15.60:0/943350250 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92180035b0 con 0x7f922806a410 2023-07-13T22:22:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f922cb9b700 1 -- 172.21.15.60:0/943350250 >> v2:172.21.15.105:3300/0 conn(0x7f92280ca9f0 msgr2=0x7f9228069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f922cb9b700 1 --2- 172.21.15.60:0/943350250 >> v2:172.21.15.105:3300/0 conn(0x7f92280ca9f0 0x7f9228069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f922cb9b700 1 -- 172.21.15.60:0/943350250 >> v1:172.21.15.60:6789/0 conn(0x7f922806a410 legacy=0x7f92280703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f922cb9b700 1 -- 172.21.15.60:0/943350250 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280ae0d0 con 0x7f92280add00 2023-07-13T22:22:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f9227fff700 1 --2- 172.21.15.60:0/943350250 >> v2:172.21.15.105:3300/0 conn(0x7f92280ca9f0 0x7f9228069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:50.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f922cb9b700 1 --2- 172.21.15.60:0/943350250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 0x7f922806ce70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f921c009840 tx=0x7f921c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d523ce78f0c233 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f92277fe700 1 -- 172.21.15.60:0/943350250 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c011830 con 0x7f92280add00 2023-07-13T22:22:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f92277fe700 1 -- 172.21.15.60:0/943350250 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c011990 con 0x7f92280add00 2023-07-13T22:22:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.348+0000 7f92277fe700 1 -- 172.21.15.60:0/943350250 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c011d20 con 0x7f92280add00 2023-07-13T22:22:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.349+0000 7f922edff700 1 -- 172.21.15.60:0/943350250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 msgr2=0x7f922806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.349+0000 7f922edff700 1 --2- 172.21.15.60:0/943350250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 0x7f922806ce70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f921c009840 tx=0x7f921c009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.349+0000 7f922edff700 1 -- 172.21.15.60:0/943350250 shutdown_connections 2023-07-13T22:22:50.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.349+0000 7f922edff700 1 --2- 172.21.15.60:0/943350250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 0x7f922806ce70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.349+0000 7f922edff700 1 --2- 172.21.15.60:0/943350250 >> v2:172.21.15.105:3300/0 conn(0x7f92280ca9f0 0x7f9228069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.349+0000 7f922edff700 1 -- 172.21.15.60:0/943350250 >> 172.21.15.60:0/943350250 conn(0x7f92280cbf50 msgr2=0x7f92280ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.349+0000 7f922edff700 1 -- 172.21.15.60:0/943350250 shutdown_connections 2023-07-13T22:22:50.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.349+0000 7f922edff700 1 -- 172.21.15.60:0/943350250 wait complete. 2023-07-13T22:22:50.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.351+0000 7f922edff700 1 Processor -- start 2023-07-13T22:22:50.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.351+0000 7f922edff700 1 -- start start 2023-07-13T22:22:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922edff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f922806a410 0x7f92281e6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922edff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 0x7f92280ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922edff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92280ca9f0 0x7f92280ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922edff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9228151db0 con 0x7f92280ca9f0 2023-07-13T22:22:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922edff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922815cfd0 con 0x7f922806a410 2023-07-13T22:22:50.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922edff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9228152760 con 0x7f92280add00 2023-07-13T22:22:50.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922d39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92280ca9f0 0x7f92280ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922d39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92280ca9f0 0x7f92280ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40880/0 (socket says 172.21.15.60:40880) 2023-07-13T22:22:50.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922d39c700 1 -- 172.21.15.60:0/1656023342 learned_addr learned my addr 172.21.15.60:0/1656023342 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:50.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f922cb9b700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f922806a410 0x7f92281e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.352+0000 7f9227fff700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 0x7f92280ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f922d39c700 1 -- 172.21.15.60:0/1656023342 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 msgr2=0x7f92280ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f922d39c700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 0x7f92280ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f922d39c700 1 -- 172.21.15.60:0/1656023342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f922806a410 msgr2=0x7f92281e6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f922d39c700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f922806a410 0x7f92281e6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f922d39c700 1 -- 172.21.15.60:0/1656023342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140094a0 con 0x7f92280ca9f0 2023-07-13T22:22:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f922d39c700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92280ca9f0 0x7f92280ae7a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f922000cb40 tx=0x7f922000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f92257fa700 1 -- 172.21.15.60:0/1656023342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922000a9e0 con 0x7f92280ca9f0 2023-07-13T22:22:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f92257fa700 1 -- 172.21.15.60:0/1656023342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922000ab40 con 0x7f92280ca9f0 2023-07-13T22:22:50.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f92257fa700 1 -- 172.21.15.60:0/1656023342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9220012400 con 0x7f92280ca9f0 2023-07-13T22:22:50.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c0094a0 con 0x7f92280ca9f0 2023-07-13T22:22:50.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.353+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9228152d20 con 0x7f92280ca9f0 2023-07-13T22:22:50.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.355+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9228152f40 con 0x7f92280ca9f0 2023-07-13T22:22:50.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.355+0000 7f92257fa700 1 -- 172.21.15.60:0/1656023342 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f922001b3b0 con 0x7f92280ca9f0 2023-07-13T22:22:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.357+0000 7f92257fa700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f920c06ace0 0x7f920c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.357+0000 7f92257fa700 1 -- 172.21.15.60:0/1656023342 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f9220088ff0 con 0x7f92280ca9f0 2023-07-13T22:22:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.357+0000 7f922cb9b700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f920c06ace0 0x7f920c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.358+0000 7f922cb9b700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f920c06ace0 0x7f920c06d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f921c000c00 tx=0x7f921c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:50.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.361+0000 7f92257fa700 1 -- 172.21.15.60:0/1656023342 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f92200540f0 con 0x7f92280ca9f0 2023-07-13T22:22:50.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.570+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f922804e210 con 0x7f92280ca9f0 2023-07-13T22:22:50.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.570+0000 7f92257fa700 1 -- 172.21.15.60:0/1656023342 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9220057710 con 0x7f92280ca9f0 2023-07-13T22:22:50.554 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:22:50.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f920c06ace0 msgr2=0x7f920c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f920c06ace0 0x7f920c06d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f921c000c00 tx=0x7f921c008030 comp rx=0 tx=0).stop 2023-07-13T22:22:50.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92280ca9f0 msgr2=0x7f92280ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92280ca9f0 0x7f92280ae7a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f922000cb40 tx=0x7f922000ce50 comp rx=0 tx=0).stop 2023-07-13T22:22:50.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 shutdown_connections 2023-07-13T22:22:50.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f920c06ace0 0x7f920c06d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92280ca9f0 0x7f92280ae7a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f922806a410 0x7f92281e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 --2- 172.21.15.60:0/1656023342 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92280add00 0x7f92280ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 >> 172.21.15.60:0/1656023342 conn(0x7f92280cbf50 msgr2=0x7f92280cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:50.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 shutdown_connections 2023-07-13T22:22:50.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.573+0000 7f922edff700 1 -- 172.21.15.60:0/1656023342 wait complete. 2023-07-13T22:22:50.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:22:50.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.798+0000 7fa5b29ff700 1 Processor -- start 2023-07-13T22:22:50.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.798+0000 7fa5b29ff700 1 -- start start 2023-07-13T22:22:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5b29ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0b8200 0x7fa5ac0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5b29ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5ac0b1ac0 0x7fa5ac0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5b29ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ac0d1870 con 0x7fa5ac0cede0 2023-07-13T22:22:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5b29ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ac0cf370 con 0x7fa5ac0b1ac0 2023-07-13T22:22:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5b29ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0d12b0 con 0x7fa5ac0b8200 2023-07-13T22:22:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5ab7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5ac0b1ac0 0x7fa5ac0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5b0f9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa5ac0cede0 0x7fa5ac15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45210/0 (socket says 172.21.15.60:45210) 2023-07-13T22:22:50.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5b0f9c700 1 -- 172.21.15.60:0/2453280047 learned_addr learned my addr 172.21.15.60:0/2453280047 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5abfff700 1 --2- 172.21.15.60:0/2453280047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0b8200 0x7fa5ac0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5ab7fe700 1 -- 172.21.15.60:0/2453280047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0b8200 msgr2=0x7fa5ac0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5ab7fe700 1 --2- 172.21.15.60:0/2453280047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0b8200 0x7fa5ac0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5ab7fe700 1 -- 172.21.15.60:0/2453280047 >> v1:172.21.15.60:6789/0 conn(0x7fa5ac0cede0 legacy=0x7fa5ac15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5ab7fe700 1 -- 172.21.15.60:0/2453280047 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac0d1590 con 0x7fa5ac0b1ac0 2023-07-13T22:22:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.800+0000 7fa5aaffd700 1 -- 172.21.15.60:0/2453280047 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496818795 0 0) 0x7fa5ac0d1870 con 0x7fa5ac0cede0 2023-07-13T22:22:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.801+0000 7fa5ab7fe700 1 --2- 172.21.15.60:0/2453280047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b1ac0 0x7fa5ac0cfb80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009840 tx=0x7fa5a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8352a11d4153b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.801+0000 7fa5aaffd700 1 -- 172.21.15.60:0/2453280047 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0011830 con 0x7fa5ac0b1ac0 2023-07-13T22:22:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.801+0000 7fa5aaffd700 1 -- 172.21.15.60:0/2453280047 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0011990 con 0x7fa5ac0b1ac0 2023-07-13T22:22:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.801+0000 7fa5aaffd700 1 -- 172.21.15.60:0/2453280047 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0011d20 con 0x7fa5ac0b1ac0 2023-07-13T22:22:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.801+0000 7fa5b29ff700 1 -- 172.21.15.60:0/2453280047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b1ac0 msgr2=0x7fa5ac0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.801+0000 7fa5b29ff700 1 --2- 172.21.15.60:0/2453280047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b1ac0 0x7fa5ac0cfb80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009840 tx=0x7fa5a0009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.802+0000 7fa5b29ff700 1 -- 172.21.15.60:0/2453280047 shutdown_connections 2023-07-13T22:22:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.802+0000 7fa5b29ff700 1 --2- 172.21.15.60:0/2453280047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0b8200 0x7fa5ac0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.802+0000 7fa5b29ff700 1 --2- 172.21.15.60:0/2453280047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b1ac0 0x7fa5ac0cfb80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.802+0000 7fa5b29ff700 1 -- 172.21.15.60:0/2453280047 >> 172.21.15.60:0/2453280047 conn(0x7fa5ac06b2f0 msgr2=0x7fa5ac06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.802+0000 7fa5b29ff700 1 -- 172.21.15.60:0/2453280047 shutdown_connections 2023-07-13T22:22:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.802+0000 7fa5b29ff700 1 -- 172.21.15.60:0/2453280047 wait complete. 2023-07-13T22:22:50.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.803+0000 7fa5b29ff700 1 Processor -- start 2023-07-13T22:22:50.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.804+0000 7fa5b29ff700 1 -- start start 2023-07-13T22:22:50.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.804+0000 7fa5b29ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5ac0b1ac0 0x7fa5ac15b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5b29ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b8200 0x7fa5ac1e5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5b29ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0cede0 0x7fa5ac1e4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5b29ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0c6ed0 con 0x7fa5ac0b8200 2023-07-13T22:22:50.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5b29ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac15bd70 con 0x7fa5ac0b1ac0 2023-07-13T22:22:50.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5b29ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0c6bf0 con 0x7fa5ac0cede0 2023-07-13T22:22:50.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b8200 0x7fa5ac1e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5ab7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b8200 0x7fa5ac1e5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40900/0 (socket says 172.21.15.60:40900) 2023-07-13T22:22:50.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5ab7fe700 1 -- 172.21.15.60:0/174515891 learned_addr learned my addr 172.21.15.60:0/174515891 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:50.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5b0f9c700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0cede0 0x7fa5ac1e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5ab7fe700 1 -- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0cede0 msgr2=0x7fa5ac1e4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5ab7fe700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0cede0 0x7fa5ac1e4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5abfff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5ac0b1ac0 0x7fa5ac15b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5ab7fe700 1 -- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5ac0b1ac0 msgr2=0x7fa5ac15b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:50.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5ab7fe700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5ac0b1ac0 0x7fa5ac15b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:50.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5ab7fe700 1 -- 172.21.15.60:0/174515891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a00094a0 con 0x7fa5ac0b8200 2023-07-13T22:22:50.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5b0f9c700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0cede0 0x7fa5ac1e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:50.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.805+0000 7fa5abfff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5ac0b1ac0 0x7fa5ac15b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:50.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.806+0000 7fa5ab7fe700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b8200 0x7fa5ac1e5510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00090b0 tx=0x7fa5a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:50.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.806+0000 7fa5a8ff9700 1 -- 172.21.15.60:0/174515891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00038a0 con 0x7fa5ac0b8200 2023-07-13T22:22:50.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.806+0000 7fa5a8ff9700 1 -- 172.21.15.60:0/174515891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a001f070 con 0x7fa5ac0b8200 2023-07-13T22:22:50.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.806+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac1e3d90 con 0x7fa5ac0b8200 2023-07-13T22:22:50.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.806+0000 7fa5a8ff9700 1 -- 172.21.15.60:0/174515891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0003c20 con 0x7fa5ac0b8200 2023-07-13T22:22:50.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.806+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac1e4350 con 0x7fa5ac0b8200 2023-07-13T22:22:50.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.807+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ac1e3bf0 con 0x7fa5ac0b8200 2023-07-13T22:22:50.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.808+0000 7fa5a8ff9700 1 -- 172.21.15.60:0/174515891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5a001a430 con 0x7fa5ac0b8200 2023-07-13T22:22:50.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.809+0000 7fa5a8ff9700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa59806ace0 0x7fa59806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:50.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.809+0000 7fa5a8ff9700 1 -- 172.21.15.60:0/174515891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fa5a0021070 con 0x7fa5ac0b8200 2023-07-13T22:22:50.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.809+0000 7fa5abfff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa59806ace0 0x7fa59806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:50.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.810+0000 7fa5abfff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa59806ace0 0x7fa59806d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009a00 tx=0x7fa59c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:50.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:50.813+0000 7fa5a8ff9700 1 -- 172.21.15.60:0/174515891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa5ac1e3bf0 con 0x7fa5ac0b8200 2023-07-13T22:22:51.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.020+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5ac1e3bf0 con 0x7fa5ac0b8200 2023-07-13T22:22:51.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.021+0000 7fa5a8ff9700 1 -- 172.21.15.60:0/174515891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa5ac1e3bf0 con 0x7fa5ac0b8200 2023-07-13T22:22:51.005 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:22:51.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa59806ace0 msgr2=0x7fa59806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa59806ace0 0x7fa59806d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009a00 tx=0x7fa59c008040 comp rx=0 tx=0).stop 2023-07-13T22:22:51.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b8200 msgr2=0x7fa5ac1e5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b8200 0x7fa5ac1e5510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00090b0 tx=0x7fa5a0004750 comp rx=0 tx=0).stop 2023-07-13T22:22:51.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 shutdown_connections 2023-07-13T22:22:51.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa59806ace0 0x7fa59806d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5ac0cede0 0x7fa5ac1e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5ac0b1ac0 0x7fa5ac15b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 --2- 172.21.15.60:0/174515891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5ac0b8200 0x7fa5ac1e5510 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 >> 172.21.15.60:0/174515891 conn(0x7fa5ac06b2f0 msgr2=0x7fa5ac0b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:51.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 shutdown_connections 2023-07-13T22:22:51.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.023+0000 7fa5b29ff700 1 -- 172.21.15.60:0/174515891 wait complete. 2023-07-13T22:22:51.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:22:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.247+0000 7f11f09ff700 1 Processor -- start 2023-07-13T22:22:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.247+0000 7f11f09ff700 1 -- start start 2023-07-13T22:22:51.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.248+0000 7f11f09ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f11ec0b8400 0x7f11ec0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.249+0000 7f11f09ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ec0d0ac0 con 0x7f11ec0d3750 2023-07-13T22:22:51.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.249+0000 7f11f09ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ec0b87d0 con 0x7f11ec0cfca0 2023-07-13T22:22:51.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.249+0000 7f11f09ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ec0b4eb0 con 0x7f11ec0b8400 2023-07-13T22:22:51.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.249+0000 7f11e3fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f11ec0d3750 0x7f11ec0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45226/0 (socket says 172.21.15.60:45226) 2023-07-13T22:22:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.249+0000 7f11e3fff700 1 -- 172.21.15.60:0/918378041 learned_addr learned my addr 172.21.15.60:0/918378041 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.249+0000 7f11ea59c700 1 --2- 172.21.15.60:0/918378041 >> v2:172.21.15.105:3300/0 conn(0x7f11ec0b8400 0x7f11ec0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.249+0000 7f11e9d9b700 1 -- 172.21.15.60:0/918378041 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635893471 0 0) 0x7f11ec0d0ac0 con 0x7f11ec0d3750 2023-07-13T22:22:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.249+0000 7f11e9d9b700 1 -- 172.21.15.60:0/918378041 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d80035b0 con 0x7f11ec0d3750 2023-07-13T22:22:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.250+0000 7f11ea59c700 1 -- 172.21.15.60:0/918378041 >> v1:172.21.15.105:6789/0 conn(0x7f11ec0cfca0 legacy=0x7f11ec0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.250+0000 7f11ea59c700 1 -- 172.21.15.60:0/918378041 >> v1:172.21.15.60:6789/0 conn(0x7f11ec0d3750 legacy=0x7f11ec0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.250+0000 7f11ea59c700 1 -- 172.21.15.60:0/918378041 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec0b5190 con 0x7f11ec0b8400 2023-07-13T22:22:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.250+0000 7f11ea59c700 1 --2- 172.21.15.60:0/918378041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec0b8400 0x7f11ec0cd240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009840 tx=0x7f11dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e2b615a0b5be4ac server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.250+0000 7f11e9d9b700 1 -- 172.21.15.60:0/918378041 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc011830 con 0x7f11ec0b8400 2023-07-13T22:22:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.250+0000 7f11e9d9b700 1 -- 172.21.15.60:0/918378041 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc011990 con 0x7f11ec0b8400 2023-07-13T22:22:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.250+0000 7f11e9d9b700 1 -- 172.21.15.60:0/918378041 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc011d20 con 0x7f11ec0b8400 2023-07-13T22:22:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.251+0000 7f11f09ff700 1 -- 172.21.15.60:0/918378041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec0b8400 msgr2=0x7f11ec0cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.251+0000 7f11f09ff700 1 --2- 172.21.15.60:0/918378041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec0b8400 0x7f11ec0cd240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009840 tx=0x7f11dc009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.251+0000 7f11f09ff700 1 -- 172.21.15.60:0/918378041 shutdown_connections 2023-07-13T22:22:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.251+0000 7f11f09ff700 1 --2- 172.21.15.60:0/918378041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec0b8400 0x7f11ec0cd240 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.251+0000 7f11f09ff700 1 -- 172.21.15.60:0/918378041 >> 172.21.15.60:0/918378041 conn(0x7f11ec06b390 msgr2=0x7f11ec06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.251+0000 7f11f09ff700 1 -- 172.21.15.60:0/918378041 shutdown_connections 2023-07-13T22:22:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.251+0000 7f11f09ff700 1 -- 172.21.15.60:0/918378041 wait complete. 2023-07-13T22:22:51.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.253+0000 7f11f09ff700 1 Processor -- start 2023-07-13T22:22:51.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.253+0000 7f11f09ff700 1 -- start start 2023-07-13T22:22:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11f09ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11ec0b8400 0x7f11ec1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11f09ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec0cfca0 0x7f11ec0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11f09ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11ec0d3750 0x7f11ec0b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11f09ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec15d8f0 con 0x7f11ec0d3750 2023-07-13T22:22:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11f09ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0cc0f0 con 0x7f11ec0cfca0 2023-07-13T22:22:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11f09ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec150a10 con 0x7f11ec0b8400 2023-07-13T22:22:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ead9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11ec0d3750 0x7f11ec0b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ead9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11ec0d3750 0x7f11ec0b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40906/0 (socket says 172.21.15.60:40906) 2023-07-13T22:22:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ead9d700 1 -- 172.21.15.60:0/2170236696 learned_addr learned my addr 172.21.15.60:0/2170236696 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11e3fff700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec0cfca0 0x7f11ec0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ea59c700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11ec0b8400 0x7f11ec1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ead9d700 1 -- 172.21.15.60:0/2170236696 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11ec0b8400 msgr2=0x7f11ec1504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ead9d700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11ec0b8400 0x7f11ec1504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ead9d700 1 -- 172.21.15.60:0/2170236696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec0cfca0 msgr2=0x7f11ec0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ead9d700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec0cfca0 0x7f11ec0cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ead9d700 1 -- 172.21.15.60:0/2170236696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc0094a0 con 0x7f11ec0d3750 2023-07-13T22:22:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.254+0000 7f11ea59c700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11ec0b8400 0x7f11ec1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.255+0000 7f11ead9d700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11ec0d3750 0x7f11ec0b6a00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f11e400caf0 tx=0x7f11e400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:51.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.255+0000 7f11e2ffd700 1 -- 172.21.15.60:0/2170236696 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4003c00 con 0x7f11ec0d3750 2023-07-13T22:22:51.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.255+0000 7f11e2ffd700 1 -- 172.21.15.60:0/2170236696 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e4003d60 con 0x7f11ec0d3750 2023-07-13T22:22:51.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.255+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ec0cb4e0 con 0x7f11ec0d3750 2023-07-13T22:22:51.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.255+0000 7f11e2ffd700 1 -- 172.21.15.60:0/2170236696 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400aae0 con 0x7f11ec0d3750 2023-07-13T22:22:51.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.255+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec0cbaa0 con 0x7f11ec0d3750 2023-07-13T22:22:51.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.256+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ec0cc260 con 0x7f11ec0d3750 2023-07-13T22:22:51.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.257+0000 7f11e2ffd700 1 -- 172.21.15.60:0/2170236696 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f11e400ac40 con 0x7f11ec0d3750 2023-07-13T22:22:51.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.258+0000 7f11e2ffd700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f11cc06ace0 0x7f11cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.258+0000 7f11e2ffd700 1 -- 172.21.15.60:0/2170236696 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f11e40893e0 con 0x7f11ec0d3750 2023-07-13T22:22:51.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.258+0000 7f11ea59c700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f11cc06ace0 0x7f11cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.259+0000 7f11ea59c700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f11cc06ace0 0x7f11cc06d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0090b0 tx=0x7f11dc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.262+0000 7f11e2ffd700 1 -- 172.21.15.60:0/2170236696 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f11e4054460 con 0x7f11ec0d3750 2023-07-13T22:22:51.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.487+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f11ec0c6000 con 0x7f11ec0d3750 2023-07-13T22:22:51.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.487+0000 7f11e2ffd700 1 -- 172.21.15.60:0/2170236696 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f11e4057a80 con 0x7f11ec0d3750 2023-07-13T22:22:51.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f11cc06ace0 msgr2=0x7f11cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f11cc06ace0 0x7f11cc06d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0090b0 tx=0x7f11dc01c040 comp rx=0 tx=0).stop 2023-07-13T22:22:51.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11ec0d3750 msgr2=0x7f11ec0b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11ec0d3750 0x7f11ec0b6a00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f11e400caf0 tx=0x7f11e400ce00 comp rx=0 tx=0).stop 2023-07-13T22:22:51.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 shutdown_connections 2023-07-13T22:22:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f11cc06ace0 0x7f11cc06d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11ec0d3750 0x7f11ec0b6a00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11ec0b8400 0x7f11ec1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 --2- 172.21.15.60:0/2170236696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec0cfca0 0x7f11ec0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 >> 172.21.15.60:0/2170236696 conn(0x7f11ec06b390 msgr2=0x7f11ec15f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 shutdown_connections 2023-07-13T22:22:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.490+0000 7f11f09ff700 1 -- 172.21.15.60:0/2170236696 wait complete. 2023-07-13T22:22:51.476 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:22:51.491 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:22:51.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:51.492 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:22:51.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[128160]: fuse finished with error 0 and tester_r 0 2023-07-13T22:22:51.565 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:22:51.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:51.566 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:22:51.716 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:22:51.716 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:51.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:51.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.984+0000 7f8b1abff700 1 Processor -- start 2023-07-13T22:22:51.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.984+0000 7f8b1abff700 1 -- start start 2023-07-13T22:22:51.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.985+0000 7f8b1abff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b140b12a0 0x7f8b140cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 0x7f8b140b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1abff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b141609a0 con 0x7f8b1406eed0 2023-07-13T22:22:51.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1abff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1406a5f0 con 0x7f8b140b12a0 2023-07-13T22:22:51.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b14160ed0 con 0x7f8b1406bad0 2023-07-13T22:22:51.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1899b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b140b12a0 0x7f8b140cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1899b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b140b12a0 0x7f8b140cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40918/0 (socket says 172.21.15.60:40918) 2023-07-13T22:22:51.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1899b700 1 -- 172.21.15.60:0/1720745920 learned_addr learned my addr 172.21.15.60:0/1720745920 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:51.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1899b700 1 -- 172.21.15.60:0/1720745920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 msgr2=0x7f8b140b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:51.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b13fff700 1 --2- 172.21.15.60:0/1720745920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 0x7f8b140b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1899b700 1 --2- 172.21.15.60:0/1720745920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 0x7f8b140b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1899b700 1 -- 172.21.15.60:0/1720745920 >> v1:172.21.15.105:6789/0 conn(0x7f8b1406eed0 legacy=0x7f8b1406f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:51.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b1899b700 1 -- 172.21.15.60:0/1720745920 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1415c860 con 0x7f8b140b12a0 2023-07-13T22:22:51.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.986+0000 7f8b13fff700 1 --2- 172.21.15.60:0/1720745920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 0x7f8b140b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:51.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.987+0000 7f8b1899b700 1 --2- 172.21.15.60:0/1720745920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b140b12a0 0x7f8b140cefa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009920 tx=0x7f8b08009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb5f8eff4ff9529 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:51.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.987+0000 7f8b137fe700 1 -- 172.21.15.60:0/1720745920 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800ba20 con 0x7f8b140b12a0 2023-07-13T22:22:51.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.987+0000 7f8b137fe700 1 -- 172.21.15.60:0/1720745920 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0800bb80 con 0x7f8b140b12a0 2023-07-13T22:22:51.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.987+0000 7f8b137fe700 1 -- 172.21.15.60:0/1720745920 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08009e70 con 0x7f8b140b12a0 2023-07-13T22:22:51.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.988+0000 7f8b1abff700 1 -- 172.21.15.60:0/1720745920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b140b12a0 msgr2=0x7f8b140cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.988+0000 7f8b1abff700 1 --2- 172.21.15.60:0/1720745920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b140b12a0 0x7f8b140cefa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009920 tx=0x7f8b08009c30 comp rx=0 tx=0).stop 2023-07-13T22:22:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.988+0000 7f8b1abff700 1 -- 172.21.15.60:0/1720745920 shutdown_connections 2023-07-13T22:22:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.988+0000 7f8b1abff700 1 --2- 172.21.15.60:0/1720745920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b140b12a0 0x7f8b140cefa0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.988+0000 7f8b1abff700 1 --2- 172.21.15.60:0/1720745920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 0x7f8b140b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.988+0000 7f8b1abff700 1 -- 172.21.15.60:0/1720745920 >> 172.21.15.60:0/1720745920 conn(0x7f8b140cc2a0 msgr2=0x7f8b140cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.988+0000 7f8b1abff700 1 -- 172.21.15.60:0/1720745920 shutdown_connections 2023-07-13T22:22:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.988+0000 7f8b1abff700 1 -- 172.21.15.60:0/1720745920 wait complete. 2023-07-13T22:22:51.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.989+0000 7f8b1abff700 1 Processor -- start 2023-07-13T22:22:51.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.990+0000 7f8b1abff700 1 -- start start 2023-07-13T22:22:51.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.990+0000 7f8b1abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 0x7f8b1415bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b1abff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b1406eed0 0x7f8b140c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b1abff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b140b12a0 0x7f8b14158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b1abff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140c9cb0 con 0x7f8b1406eed0 2023-07-13T22:22:51.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b1abff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140c9f90 con 0x7f8b140b12a0 2023-07-13T22:22:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b1abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140ca270 con 0x7f8b1406bad0 2023-07-13T22:22:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b13fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b1406eed0 0x7f8b140c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b13fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b1406eed0 0x7f8b140c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40934/0 (socket says 172.21.15.60:40934) 2023-07-13T22:22:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b13fff700 1 -- 172.21.15.60:0/3691340485 learned_addr learned my addr 172.21.15.60:0/3691340485 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b1919c700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b140b12a0 0x7f8b14158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b1899b700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 0x7f8b1415bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b13fff700 1 -- 172.21.15.60:0/3691340485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 msgr2=0x7f8b1415bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b13fff700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 0x7f8b1415bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b13fff700 1 -- 172.21.15.60:0/3691340485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b140b12a0 msgr2=0x7f8b14158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:51.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b13fff700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b140b12a0 0x7f8b14158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:51.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.991+0000 7f8b13fff700 1 -- 172.21.15.60:0/3691340485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08009550 con 0x7f8b1406eed0 2023-07-13T22:22:51.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.992+0000 7f8b13fff700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b1406eed0 0x7f8b140c9410 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c7f0 tx=0x7f8b0000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:51.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.992+0000 7f8b117fa700 1 -- 172.21.15.60:0/3691340485 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00011800 con 0x7f8b1406eed0 2023-07-13T22:22:51.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.992+0000 7f8b117fa700 1 -- 172.21.15.60:0/3691340485 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b00011960 con 0x7f8b1406eed0 2023-07-13T22:22:51.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.992+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b140ca550 con 0x7f8b1406eed0 2023-07-13T22:22:51.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.992+0000 7f8b117fa700 1 -- 172.21.15.60:0/3691340485 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00011cf0 con 0x7f8b1406eed0 2023-07-13T22:22:51.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.992+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b14159160 con 0x7f8b1406eed0 2023-07-13T22:22:51.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.993+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b140ca6e0 con 0x7f8b1406eed0 2023-07-13T22:22:51.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.994+0000 7f8b117fa700 1 -- 172.21.15.60:0/3691340485 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b00018360 con 0x7f8b1406eed0 2023-07-13T22:22:51.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.995+0000 7f8b117fa700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8af806ace0 0x7f8af806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:51.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.995+0000 7f8b117fa700 1 -- 172.21.15.60:0/3691340485 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8b00089400 con 0x7f8b1406eed0 2023-07-13T22:22:51.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.995+0000 7f8b1899b700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8af806ace0 0x7f8af806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:51.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.996+0000 7f8b1899b700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8af806ace0 0x7f8af806d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b08000c00 tx=0x7f8b08008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:51.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:51.999+0000 7f8b117fa700 1 -- 172.21.15.60:0/3691340485 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8b000543f0 con 0x7f8b1406eed0 2023-07-13T22:22:52.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.205+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b14071c90 con 0x7f8b1406eed0 2023-07-13T22:22:52.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.206+0000 7f8b117fa700 1 -- 172.21.15.60:0/3691340485 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18250 (secure 0 0 0) 0x7f8b00057a10 con 0x7f8b1406eed0 2023-07-13T22:22:52.190 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:52.191 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:22:43.507730+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:22:17.071217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-13T22:22:18.161600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8af806ace0 msgr2=0x7f8af806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8af806ace0 0x7f8af806d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b08000c00 tx=0x7f8b08008030 comp rx=0 tx=0).stop 2023-07-13T22:22:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b1406eed0 msgr2=0x7f8b140c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b1406eed0 0x7f8b140c9410 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c7f0 tx=0x7f8b0000cb00 comp rx=0 tx=0).stop 2023-07-13T22:22:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 shutdown_connections 2023-07-13T22:22:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8af806ace0 0x7f8af806d190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b140b12a0 0x7f8b14158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b1406bad0 0x7f8b1415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 --2- 172.21.15.60:0/3691340485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b1406eed0 0x7f8b140c9410 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 >> 172.21.15.60:0/3691340485 conn(0x7f8b140cc2a0 msgr2=0x7f8b140b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 shutdown_connections 2023-07-13T22:22:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.209+0000 7f8b1abff700 1 -- 172.21.15.60:0/3691340485 wait complete. 2023-07-13T22:22:52.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:22:17.071217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:22:18.161600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:22:52.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:22:52.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.434+0000 7f7d4b1ff700 1 Processor -- start 2023-07-13T22:22:52.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.434+0000 7f7d4b1ff700 1 -- start start 2023-07-13T22:22:52.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d4b1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d44158700 0x7f7d4415c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d4b1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 0x7f7d4406a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d4b1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4406a870 con 0x7f7d440ca710 2023-07-13T22:22:52.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d4b1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d44158ad0 con 0x7f7d44158700 2023-07-13T22:22:52.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d4b1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4415ca30 con 0x7f7d44069ed0 2023-07-13T22:22:52.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d48f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d44158700 0x7f7d4415c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d43fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7d440ca710 0x7f7d440c8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45242/0 (socket says 172.21.15.60:45242) 2023-07-13T22:22:52.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d43fff700 1 -- 172.21.15.60:0/1704067655 learned_addr learned my addr 172.21.15.60:0/1704067655 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:52.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d4979c700 1 --2- 172.21.15.60:0/1704067655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 0x7f7d4406a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d48f9b700 1 -- 172.21.15.60:0/1704067655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 msgr2=0x7f7d4406a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:52.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.436+0000 7f7d48f9b700 1 --2- 172.21.15.60:0/1704067655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 0x7f7d4406a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.437+0000 7f7d48f9b700 1 -- 172.21.15.60:0/1704067655 >> v1:172.21.15.60:6789/0 conn(0x7f7d440ca710 legacy=0x7f7d440c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.437+0000 7f7d48f9b700 1 -- 172.21.15.60:0/1704067655 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4415cd10 con 0x7f7d44158700 2023-07-13T22:22:52.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.437+0000 7f7d437fe700 1 -- 172.21.15.60:0/1704067655 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477437322 0 0) 0x7f7d4406a870 con 0x7f7d440ca710 2023-07-13T22:22:52.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.437+0000 7f7d48f9b700 1 --2- 172.21.15.60:0/1704067655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d44158700 0x7f7d4415c1e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800a9e0 tx=0x7f7d3800acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acc18332f62bd1f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:52.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.437+0000 7f7d437fe700 1 -- 172.21.15.60:0/1704067655 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800cb70 con 0x7f7d44158700 2023-07-13T22:22:52.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.437+0000 7f7d437fe700 1 -- 172.21.15.60:0/1704067655 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800ccd0 con 0x7f7d44158700 2023-07-13T22:22:52.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.437+0000 7f7d437fe700 1 -- 172.21.15.60:0/1704067655 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38018620 con 0x7f7d44158700 2023-07-13T22:22:52.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.438+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/1704067655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d44158700 msgr2=0x7f7d4415c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.438+0000 7f7d4b1ff700 1 --2- 172.21.15.60:0/1704067655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d44158700 0x7f7d4415c1e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800a9e0 tx=0x7f7d3800acf0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.438+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/1704067655 shutdown_connections 2023-07-13T22:22:52.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.438+0000 7f7d4b1ff700 1 --2- 172.21.15.60:0/1704067655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 0x7f7d4406a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.438+0000 7f7d4b1ff700 1 --2- 172.21.15.60:0/1704067655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d44158700 0x7f7d4415c1e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.438+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/1704067655 >> 172.21.15.60:0/1704067655 conn(0x7f7d440cbf50 msgr2=0x7f7d440ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:52.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.438+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/1704067655 shutdown_connections 2023-07-13T22:22:52.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.438+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/1704067655 wait complete. 2023-07-13T22:22:52.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.439+0000 7f7d4b1ff700 1 Processor -- start 2023-07-13T22:22:52.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.440+0000 7f7d4b1ff700 1 -- start start 2023-07-13T22:22:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.440+0000 7f7d4b1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 0x7f7d440c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d4b1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d440ca710 0x7f7d441f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d4b1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d44158700 0x7f7d44156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d4b1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d441f6620 con 0x7f7d440ca710 2023-07-13T22:22:52.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d4b1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440af160 con 0x7f7d44158700 2023-07-13T22:22:52.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d4b1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440af440 con 0x7f7d44069ed0 2023-07-13T22:22:52.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d43fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d440ca710 0x7f7d441f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d43fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d440ca710 0x7f7d441f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40950/0 (socket says 172.21.15.60:40950) 2023-07-13T22:22:52.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d43fff700 1 -- 172.21.15.60:0/2521939787 learned_addr learned my addr 172.21.15.60:0/2521939787 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:52.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d48f9b700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 0x7f7d440c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d4979c700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d44158700 0x7f7d44156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d43fff700 1 -- 172.21.15.60:0/2521939787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 msgr2=0x7f7d440c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d43fff700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 0x7f7d440c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d43fff700 1 -- 172.21.15.60:0/2521939787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d44158700 msgr2=0x7f7d44156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d43fff700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d44158700 0x7f7d44156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d43fff700 1 -- 172.21.15.60:0/2521939787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3800a640 con 0x7f7d440ca710 2023-07-13T22:22:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.441+0000 7f7d43fff700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d440ca710 0x7f7d441f60e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c550 tx=0x7f7d3000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.442+0000 7f7d417fa700 1 -- 172.21.15.60:0/2521939787 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30004c80 con 0x7f7d440ca710 2023-07-13T22:22:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.442+0000 7f7d417fa700 1 -- 172.21.15.60:0/2521939787 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d30004e00 con 0x7f7d440ca710 2023-07-13T22:22:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.442+0000 7f7d417fa700 1 -- 172.21.15.60:0/2521939787 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30007310 con 0x7f7d440ca710 2023-07-13T22:22:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.442+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d440af720 con 0x7f7d440ca710 2023-07-13T22:22:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.442+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d440afce0 con 0x7f7d440ca710 2023-07-13T22:22:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.443+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4406c3c0 con 0x7f7d440ca710 2023-07-13T22:22:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.444+0000 7f7d417fa700 1 -- 172.21.15.60:0/2521939787 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d30007470 con 0x7f7d440ca710 2023-07-13T22:22:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.446+0000 7f7d417fa700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2806ace0 0x7f7d2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.446+0000 7f7d417fa700 1 -- 172.21.15.60:0/2521939787 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7d30089180 con 0x7f7d440ca710 2023-07-13T22:22:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.446+0000 7f7d48f9b700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2806ace0 0x7f7d2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.446+0000 7f7d48f9b700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2806ace0 0x7f7d2806d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800a1c0 tx=0x7f7d38012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:52.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.450+0000 7f7d417fa700 1 -- 172.21.15.60:0/2521939787 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7d30054200 con 0x7f7d440ca710 2023-07-13T22:22:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.654+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7d4404e210 con 0x7f7d440ca710 2023-07-13T22:22:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.654+0000 7f7d417fa700 1 -- 172.21.15.60:0/2521939787 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7d30057820 con 0x7f7d440ca710 2023-07-13T22:22:52.638 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:22:52.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.656+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2806ace0 msgr2=0x7f7d2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.656+0000 7f7d4b1ff700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2806ace0 0x7f7d2806d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800a1c0 tx=0x7f7d38012000 comp rx=0 tx=0).stop 2023-07-13T22:22:52.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.656+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d440ca710 msgr2=0x7f7d441f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.656+0000 7f7d4b1ff700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d440ca710 0x7f7d441f60e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c550 tx=0x7f7d3000c860 comp rx=0 tx=0).stop 2023-07-13T22:22:52.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.657+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 shutdown_connections 2023-07-13T22:22:52.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.657+0000 7f7d4b1ff700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2806ace0 0x7f7d2806d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.657+0000 7f7d4b1ff700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d44158700 0x7f7d44156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.657+0000 7f7d4b1ff700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d44069ed0 0x7f7d440c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.657+0000 7f7d4b1ff700 1 --2- 172.21.15.60:0/2521939787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d440ca710 0x7f7d441f60e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.657+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 >> 172.21.15.60:0/2521939787 conn(0x7f7d440cbf50 msgr2=0x7f7d440c5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.657+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 shutdown_connections 2023-07-13T22:22:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.657+0000 7f7d4b1ff700 1 -- 172.21.15.60:0/2521939787 wait complete. 2023-07-13T22:22:52.659 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:22:52.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.880+0000 7f1c6e9ff700 1 Processor -- start 2023-07-13T22:22:52.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.880+0000 7f1c6e9ff700 1 -- start start 2023-07-13T22:22:52.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c6e9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c6815c530 0x7f1c680b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c6e9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1c680d11c0 0x7f1c680cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c6e9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c680cefd0 con 0x7f1c680cf9f0 2023-07-13T22:22:52.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c6e9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c680d1780 con 0x7f1c6815c530 2023-07-13T22:22:52.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c6e9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c680cd2d0 con 0x7f1c680d11c0 2023-07-13T22:22:52.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c67fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c6815c530 0x7f1c680b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c67fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c6815c530 0x7f1c680b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40966/0 (socket says 172.21.15.60:40966) 2023-07-13T22:22:52.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c67fff700 1 -- 172.21.15.60:0/3125503129 learned_addr learned my addr 172.21.15.60:0/3125503129 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:52.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c67fff700 1 -- 172.21.15.60:0/3125503129 >> v2:172.21.15.105:3300/0 conn(0x7f1c680d11c0 msgr2=0x7f1c680cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:52.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c67fff700 1 --2- 172.21.15.60:0/3125503129 >> v2:172.21.15.105:3300/0 conn(0x7f1c680d11c0 0x7f1c680cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c67fff700 1 -- 172.21.15.60:0/3125503129 >> v1:172.21.15.105:6789/0 conn(0x7f1c680cf9f0 legacy=0x7f1c680d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:52.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.882+0000 7f1c67fff700 1 -- 172.21.15.60:0/3125503129 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c680cd5b0 con 0x7f1c6815c530 2023-07-13T22:22:52.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.883+0000 7f1c67fff700 1 --2- 172.21.15.60:0/3125503129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6815c530 0x7f1c680b5a10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1c6806c680 tx=0x7f1c58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df54919454f487ff server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:52.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.883+0000 7f1c66ffd700 1 -- 172.21.15.60:0/3125503129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58004230 con 0x7f1c6815c530 2023-07-13T22:22:52.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.883+0000 7f1c66ffd700 1 -- 172.21.15.60:0/3125503129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c58004390 con 0x7f1c6815c530 2023-07-13T22:22:52.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.883+0000 7f1c66ffd700 1 -- 172.21.15.60:0/3125503129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c580036a0 con 0x7f1c6815c530 2023-07-13T22:22:52.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.883+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/3125503129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6815c530 msgr2=0x7f1c680b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.883+0000 7f1c6e9ff700 1 --2- 172.21.15.60:0/3125503129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6815c530 0x7f1c680b5a10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1c6806c680 tx=0x7f1c58009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.884+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/3125503129 shutdown_connections 2023-07-13T22:22:52.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.884+0000 7f1c6e9ff700 1 --2- 172.21.15.60:0/3125503129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6815c530 0x7f1c680b5a10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.884+0000 7f1c6e9ff700 1 --2- 172.21.15.60:0/3125503129 >> v2:172.21.15.105:3300/0 conn(0x7f1c680d11c0 0x7f1c680cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.884+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/3125503129 >> 172.21.15.60:0/3125503129 conn(0x7f1c6806b400 msgr2=0x7f1c6806b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:52.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.884+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/3125503129 shutdown_connections 2023-07-13T22:22:52.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.884+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/3125503129 wait complete. 2023-07-13T22:22:52.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.886+0000 7f1c6e9ff700 1 Processor -- start 2023-07-13T22:22:52.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.886+0000 7f1c6e9ff700 1 -- start start 2023-07-13T22:22:52.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.887+0000 7f1c6e9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c680cf9f0 0x7f1c68158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6e9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c680d11c0 0x7f1c680c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6e9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c68154ef0 0x7f1c6815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6e9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68155720 con 0x7f1c68154ef0 2023-07-13T22:22:52.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6e9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68159180 con 0x7f1c680cf9f0 2023-07-13T22:22:52.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6e9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68159460 con 0x7f1c680d11c0 2023-07-13T22:22:52.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6cf9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c68154ef0 0x7f1c6815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6cf9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c68154ef0 0x7f1c6815a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40970/0 (socket says 172.21.15.60:40970) 2023-07-13T22:22:52.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6cf9c700 1 -- 172.21.15.60:0/1069384498 learned_addr learned my addr 172.21.15.60:0/1069384498 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:52.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c67fff700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c680cf9f0 0x7f1c68158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c677fe700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c680d11c0 0x7f1c680c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6cf9c700 1 -- 172.21.15.60:0/1069384498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c680d11c0 msgr2=0x7f1c680c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6cf9c700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c680d11c0 0x7f1c680c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6cf9c700 1 -- 172.21.15.60:0/1069384498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c680cf9f0 msgr2=0x7f1c68158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6cf9c700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c680cf9f0 0x7f1c68158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.888+0000 7f1c6cf9c700 1 -- 172.21.15.60:0/1069384498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58009550 con 0x7f1c68154ef0 2023-07-13T22:22:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.889+0000 7f1c6cf9c700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c68154ef0 0x7f1c6815a220 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c7a0 tx=0x7f1c6000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.889+0000 7f1c64ff9700 1 -- 172.21.15.60:0/1069384498 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60003e10 con 0x7f1c68154ef0 2023-07-13T22:22:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.889+0000 7f1c64ff9700 1 -- 172.21.15.60:0/1069384498 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c60005070 con 0x7f1c68154ef0 2023-07-13T22:22:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.889+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c68159740 con 0x7f1c68154ef0 2023-07-13T22:22:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.889+0000 7f1c64ff9700 1 -- 172.21.15.60:0/1069384498 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60005420 con 0x7f1c68154ef0 2023-07-13T22:22:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.889+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c68159d00 con 0x7f1c68154ef0 2023-07-13T22:22:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.890+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6804e210 con 0x7f1c68154ef0 2023-07-13T22:22:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.891+0000 7f1c64ff9700 1 -- 172.21.15.60:0/1069384498 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1c60005580 con 0x7f1c68154ef0 2023-07-13T22:22:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.892+0000 7f1c64ff9700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1c5406ace0 0x7f1c5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.892+0000 7f1c64ff9700 1 -- 172.21.15.60:0/1069384498 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f1c600886f0 con 0x7f1c68154ef0 2023-07-13T22:22:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.892+0000 7f1c67fff700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1c5406ace0 0x7f1c5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:52.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.893+0000 7f1c67fff700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1c5406ace0 0x7f1c5406d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009130 tx=0x7f1c58008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:52.897+0000 7f1c64ff9700 1 -- 172.21.15.60:0/1069384498 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1c600536e0 con 0x7f1c68154ef0 2023-07-13T22:22:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.100+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c680d2500 con 0x7f1c68154ef0 2023-07-13T22:22:53.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.100+0000 7f1c64ff9700 1 -- 172.21.15.60:0/1069384498 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c60056d00 con 0x7f1c68154ef0 2023-07-13T22:22:53.084 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:22:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1c5406ace0 msgr2=0x7f1c5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1c5406ace0 0x7f1c5406d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009130 tx=0x7f1c58008030 comp rx=0 tx=0).stop 2023-07-13T22:22:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c68154ef0 msgr2=0x7f1c6815a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c68154ef0 0x7f1c6815a220 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c7a0 tx=0x7f1c6000cab0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 shutdown_connections 2023-07-13T22:22:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1c5406ace0 0x7f1c5406d190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c68154ef0 0x7f1c6815a220 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c680cf9f0 0x7f1c68158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 --2- 172.21.15.60:0/1069384498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c680d11c0 0x7f1c680c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.102+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 >> 172.21.15.60:0/1069384498 conn(0x7f1c6806b400 msgr2=0x7f1c680b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.103+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 shutdown_connections 2023-07-13T22:22:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.103+0000 7f1c6e9ff700 1 -- 172.21.15.60:0/1069384498 wait complete. 2023-07-13T22:22:53.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:22:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.327+0000 7f5c92dff700 1 Processor -- start 2023-07-13T22:22:53.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.327+0000 7f5c92dff700 1 -- start start 2023-07-13T22:22:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.328+0000 7f5c92dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5c8c0b8200 0x7f5c8c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.328+0000 7f5c92dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8c0d1870 con 0x7f5c8c0b1ac0 2023-07-13T22:22:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.328+0000 7f5c92dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8c160d80 con 0x7f5c8c0cede0 2023-07-13T22:22:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.328+0000 7f5c92dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8c0cf370 con 0x7f5c8c0b8200 2023-07-13T22:22:53.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.328+0000 7f5c8bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5c8c0b1ac0 0x7f5c8c15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45258/0 (socket says 172.21.15.60:45258) 2023-07-13T22:22:53.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.328+0000 7f5c8bfff700 1 -- 172.21.15.60:0/807410791 learned_addr learned my addr 172.21.15.60:0/807410791 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.329+0000 7f5c90b9b700 1 --2- 172.21.15.60:0/807410791 >> v2:172.21.15.105:3300/0 conn(0x7f5c8c0b8200 0x7f5c8c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.329+0000 7f5c8b7fe700 1 -- 172.21.15.60:0/807410791 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145497959 0 0) 0x7f5c8c0d1870 con 0x7f5c8c0b1ac0 2023-07-13T22:22:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.329+0000 7f5c8b7fe700 1 -- 172.21.15.60:0/807410791 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c7c0035b0 con 0x7f5c8c0b1ac0 2023-07-13T22:22:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.329+0000 7f5c8b7fe700 1 -- 172.21.15.60:0/807410791 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 898164948 0 0) 0x7f5c7c0035b0 con 0x7f5c8c0b1ac0 2023-07-13T22:22:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.329+0000 7f5c90b9b700 1 -- 172.21.15.60:0/807410791 >> v1:172.21.15.105:6789/0 conn(0x7f5c8c0cede0 legacy=0x7f5c8c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.329+0000 7f5c90b9b700 1 -- 172.21.15.60:0/807410791 >> v1:172.21.15.60:6789/0 conn(0x7f5c8c0b1ac0 legacy=0x7f5c8c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.329+0000 7f5c90b9b700 1 -- 172.21.15.60:0/807410791 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c0d12b0 con 0x7f5c8c0b8200 2023-07-13T22:22:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c90b9b700 1 --2- 172.21.15.60:0/807410791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8c0b8200 0x7f5c8c0cf640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009840 tx=0x7f5c80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf09cb5dbbe8b452 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c8b7fe700 1 -- 172.21.15.60:0/807410791 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78001670 con 0x7f5c8c0b8200 2023-07-13T22:22:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c8b7fe700 1 -- 172.21.15.60:0/807410791 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c80004030 con 0x7f5c8c0b8200 2023-07-13T22:22:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c8b7fe700 1 -- 172.21.15.60:0/807410791 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800043a0 con 0x7f5c8c0b8200 2023-07-13T22:22:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c92dff700 1 -- 172.21.15.60:0/807410791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8c0b8200 msgr2=0x7f5c8c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c92dff700 1 --2- 172.21.15.60:0/807410791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8c0b8200 0x7f5c8c0cf640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009840 tx=0x7f5c80009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c92dff700 1 -- 172.21.15.60:0/807410791 shutdown_connections 2023-07-13T22:22:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c92dff700 1 --2- 172.21.15.60:0/807410791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8c0b8200 0x7f5c8c0cf640 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c92dff700 1 -- 172.21.15.60:0/807410791 >> 172.21.15.60:0/807410791 conn(0x7f5c8c06b2f0 msgr2=0x7f5c8c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:53.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c92dff700 1 -- 172.21.15.60:0/807410791 shutdown_connections 2023-07-13T22:22:53.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.330+0000 7f5c92dff700 1 -- 172.21.15.60:0/807410791 wait complete. 2023-07-13T22:22:53.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.333+0000 7f5c92dff700 1 Processor -- start 2023-07-13T22:22:53.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.333+0000 7f5c92dff700 1 -- start start 2023-07-13T22:22:53.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.334+0000 7f5c92dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8c0b1ac0 0x7f5c8c1e6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.334+0000 7f5c92dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c8c0b8200 0x7f5c8c1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.334+0000 7f5c92dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c8c0cede0 0x7f5c8c1e7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.334+0000 7f5c92dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c1e6d40 con 0x7f5c8c0b8200 2023-07-13T22:22:53.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.334+0000 7f5c92dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c1e6370 con 0x7f5c8c0b1ac0 2023-07-13T22:22:53.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.334+0000 7f5c92dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c1e8090 con 0x7f5c8c0cede0 2023-07-13T22:22:53.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c8bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c8c0b8200 0x7f5c8c1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c8bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c8c0b8200 0x7f5c8c1536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40980/0 (socket says 172.21.15.60:40980) 2023-07-13T22:22:53.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c8bfff700 1 -- 172.21.15.60:0/2360675150 learned_addr learned my addr 172.21.15.60:0/2360675150 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:53.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c90b9b700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8c0b1ac0 0x7f5c8c1e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c9139c700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c8c0cede0 0x7f5c8c1e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c8bfff700 1 -- 172.21.15.60:0/2360675150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c8c0cede0 msgr2=0x7f5c8c1e7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c8bfff700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c8c0cede0 0x7f5c8c1e7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c8bfff700 1 -- 172.21.15.60:0/2360675150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8c0b1ac0 msgr2=0x7f5c8c1e6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c8bfff700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8c0b1ac0 0x7f5c8c1e6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c8bfff700 1 -- 172.21.15.60:0/2360675150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800094a0 con 0x7f5c8c0b8200 2023-07-13T22:22:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c9139c700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c8c0cede0 0x7f5c8c1e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:53.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.335+0000 7f5c8bfff700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c8c0b8200 0x7f5c8c1536b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c550 tx=0x7f5c7800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:53.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.336+0000 7f5c897fa700 1 -- 172.21.15.60:0/2360675150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78004c80 con 0x7f5c8c0b8200 2023-07-13T22:22:53.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.336+0000 7f5c897fa700 1 -- 172.21.15.60:0/2360675150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c78004de0 con 0x7f5c8c0b8200 2023-07-13T22:22:53.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.336+0000 7f5c897fa700 1 -- 172.21.15.60:0/2360675150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c780072f0 con 0x7f5c8c0b8200 2023-07-13T22:22:53.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.336+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8c1e5d20 con 0x7f5c8c0b8200 2023-07-13T22:22:53.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.336+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8c153bf0 con 0x7f5c8c0b8200 2023-07-13T22:22:53.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.337+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8c04e210 con 0x7f5c8c0b8200 2023-07-13T22:22:53.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.338+0000 7f5c897fa700 1 -- 172.21.15.60:0/2360675150 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c78007450 con 0x7f5c8c0b8200 2023-07-13T22:22:53.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.339+0000 7f5c897fa700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7006ace0 0x7f5c7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.339+0000 7f5c897fa700 1 -- 172.21.15.60:0/2360675150 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5c78089270 con 0x7f5c8c0b8200 2023-07-13T22:22:53.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.339+0000 7f5c90b9b700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7006ace0 0x7f5c7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.340+0000 7f5c90b9b700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7006ace0 0x7f5c7006d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c800090b0 tx=0x7f5c80008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:53.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.343+0000 7f5c897fa700 1 -- 172.21.15.60:0/2360675150 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5c78054260 con 0x7f5c8c0b8200 2023-07-13T22:22:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.547+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c8c0b4f70 con 0x7f5c8c0b8200 2023-07-13T22:22:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.547+0000 7f5c897fa700 1 -- 172.21.15.60:0/2360675150 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c78007700 con 0x7f5c8c0b8200 2023-07-13T22:22:53.532 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:22:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7006ace0 msgr2=0x7f5c7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7006ace0 0x7f5c7006d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c800090b0 tx=0x7f5c80008030 comp rx=0 tx=0).stop 2023-07-13T22:22:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c8c0b8200 msgr2=0x7f5c8c1536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c8c0b8200 0x7f5c8c1536b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c550 tx=0x7f5c7800c860 comp rx=0 tx=0).stop 2023-07-13T22:22:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 shutdown_connections 2023-07-13T22:22:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c7006ace0 0x7f5c7006d190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c8c0cede0 0x7f5c8c1e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c8c0b1ac0 0x7f5c8c1e6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 --2- 172.21.15.60:0/2360675150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c8c0b8200 0x7f5c8c1536b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 >> 172.21.15.60:0/2360675150 conn(0x7f5c8c06b2f0 msgr2=0x7f5c8c0d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 shutdown_connections 2023-07-13T22:22:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.550+0000 7f5c92dff700 1 -- 172.21.15.60:0/2360675150 wait complete. 2023-07-13T22:22:53.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:22:53.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.775+0000 7f36205ff700 1 Processor -- start 2023-07-13T22:22:53.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.775+0000 7f36205ff700 1 -- start start 2023-07-13T22:22:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f36205ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36180cfd50 0x7f36180d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f36205ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b1ac0 0x7f36180d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f36205ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36180d1870 con 0x7f36180b8200 2023-07-13T22:22:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f36205ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36180cf370 con 0x7f36180cfd50 2023-07-13T22:22:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f36205ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36180d12b0 con 0x7f36180b1ac0 2023-07-13T22:22:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f361db9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36180cfd50 0x7f36180d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f361db9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36180cfd50 0x7f36180d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40992/0 (socket says 172.21.15.60:40992) 2023-07-13T22:22:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f361db9a700 1 -- 172.21.15.60:0/1938262392 learned_addr learned my addr 172.21.15.60:0/1938262392 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f361eb9c700 1 --2- 172.21.15.60:0/1938262392 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b1ac0 0x7f36180d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f361db9a700 1 -- 172.21.15.60:0/1938262392 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b1ac0 msgr2=0x7f36180d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f361db9a700 1 --2- 172.21.15.60:0/1938262392 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b1ac0 0x7f36180d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f361db9a700 1 -- 172.21.15.60:0/1938262392 >> v1:172.21.15.105:6789/0 conn(0x7f36180b8200 legacy=0x7f36180cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.777+0000 7f361db9a700 1 -- 172.21.15.60:0/1938262392 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180d1590 con 0x7f36180cfd50 2023-07-13T22:22:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.778+0000 7f361db9a700 1 --2- 172.21.15.60:0/1938262392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 0x7f36180d23c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f36080098f0 tx=0x7f3608009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186d61cd08aa86f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.778+0000 7f361d399700 1 -- 172.21.15.60:0/1938262392 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800b8a0 con 0x7f36180cfd50 2023-07-13T22:22:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.778+0000 7f361d399700 1 -- 172.21.15.60:0/1938262392 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800ba00 con 0x7f36180cfd50 2023-07-13T22:22:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.778+0000 7f361d399700 1 -- 172.21.15.60:0/1938262392 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800bd90 con 0x7f36180cfd50 2023-07-13T22:22:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.778+0000 7f36205ff700 1 -- 172.21.15.60:0/1938262392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 msgr2=0x7f36180d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.778+0000 7f36205ff700 1 --2- 172.21.15.60:0/1938262392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 0x7f36180d23c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f36080098f0 tx=0x7f3608009c00 comp rx=0 tx=0).stop 2023-07-13T22:22:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.779+0000 7f36205ff700 1 -- 172.21.15.60:0/1938262392 shutdown_connections 2023-07-13T22:22:53.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.779+0000 7f36205ff700 1 --2- 172.21.15.60:0/1938262392 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b1ac0 0x7f36180d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.779+0000 7f36205ff700 1 --2- 172.21.15.60:0/1938262392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 0x7f36180d23c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.779+0000 7f36205ff700 1 -- 172.21.15.60:0/1938262392 >> 172.21.15.60:0/1938262392 conn(0x7f361806b2f0 msgr2=0x7f361806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:53.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.779+0000 7f36205ff700 1 -- 172.21.15.60:0/1938262392 shutdown_connections 2023-07-13T22:22:53.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.779+0000 7f36205ff700 1 -- 172.21.15.60:0/1938262392 wait complete. 2023-07-13T22:22:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.780+0000 7f36205ff700 1 Processor -- start 2023-07-13T22:22:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.781+0000 7f36205ff700 1 -- start start 2023-07-13T22:22:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.781+0000 7f36205ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36180b1ac0 0x7f3618158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.781+0000 7f36205ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b8200 0x7f36180b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f36205ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 0x7f3618157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f36205ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3618157ac0 con 0x7f36180cfd50 2023-07-13T22:22:53.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f36205ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3618157da0 con 0x7f36180b1ac0 2023-07-13T22:22:53.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f36205ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3618158080 con 0x7f36180b8200 2023-07-13T22:22:53.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361eb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 0x7f3618157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361eb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 0x7f3618157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41008/0 (socket says 172.21.15.60:41008) 2023-07-13T22:22:53.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361eb9c700 1 -- 172.21.15.60:0/1919887429 learned_addr learned my addr 172.21.15.60:0/1919887429 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:53.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361e39b700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36180b1ac0 0x7f3618158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361db9a700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b8200 0x7f36180b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361eb9c700 1 -- 172.21.15.60:0/1919887429 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b8200 msgr2=0x7f36180b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361eb9c700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b8200 0x7f36180b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361eb9c700 1 -- 172.21.15.60:0/1919887429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36180b1ac0 msgr2=0x7f3618158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361eb9c700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36180b1ac0 0x7f3618158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.782+0000 7f361eb9c700 1 -- 172.21.15.60:0/1919887429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608009550 con 0x7f36180cfd50 2023-07-13T22:22:53.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.783+0000 7f361eb9c700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 0x7f3618157000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f361000cd20 tx=0x7f361000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:53.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.783+0000 7f360effd700 1 -- 172.21.15.60:0/1919887429 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610003a20 con 0x7f36180cfd50 2023-07-13T22:22:53.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.783+0000 7f360effd700 1 -- 172.21.15.60:0/1919887429 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3610003b80 con 0x7f36180cfd50 2023-07-13T22:22:53.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.783+0000 7f360effd700 1 -- 172.21.15.60:0/1919887429 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610004020 con 0x7f36180cfd50 2023-07-13T22:22:53.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.783+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3618158360 con 0x7f36180cfd50 2023-07-13T22:22:53.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.783+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3618153240 con 0x7f36180cfd50 2023-07-13T22:22:53.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.784+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3618158880 con 0x7f36180cfd50 2023-07-13T22:22:53.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.785+0000 7f360effd700 1 -- 172.21.15.60:0/1919887429 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3610004180 con 0x7f36180cfd50 2023-07-13T22:22:53.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.786+0000 7f360effd700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:53.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.786+0000 7f360effd700 1 -- 172.21.15.60:0/1919887429 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3610012070 con 0x7f36180cfd50 2023-07-13T22:22:53.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.786+0000 7f361e39b700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:53.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.787+0000 7f361e39b700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3614000f60 tx=0x7f3614009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:53.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.790+0000 7f360effd700 1 -- 172.21.15.60:0/1919887429 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3610054d80 con 0x7f36180cfd50 2023-07-13T22:22:53.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.996+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36180cd360 con 0x7f36180cfd50 2023-07-13T22:22:53.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.997+0000 7f360effd700 1 -- 172.21.15.60:0/1919887429 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f36100583a0 con 0x7f36180cfd50 2023-07-13T22:22:53.981 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:22:53.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 msgr2=0x7f35fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3614000f60 tx=0x7f3614009040 comp rx=0 tx=0).stop 2023-07-13T22:22:53.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 msgr2=0x7f3618157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:53.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 0x7f3618157000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f361000cd20 tx=0x7f361000a570 comp rx=0 tx=0).stop 2023-07-13T22:22:53.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 shutdown_connections 2023-07-13T22:22:53.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36180cfd50 0x7f3618157000 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36180b1ac0 0x7f3618158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 --2- 172.21.15.60:0/1919887429 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36180b8200 0x7f36180b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:53.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 >> 172.21.15.60:0/1919887429 conn(0x7f361806b2f0 msgr2=0x7f36180d2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:53.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:53.999+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 shutdown_connections 2023-07-13T22:22:53.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.000+0000 7f36205ff700 1 -- 172.21.15.60:0/1919887429 wait complete. 2023-07-13T22:22:54.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:22:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.221+0000 7fc7cb1ff700 1 Processor -- start 2023-07-13T22:22:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.221+0000 7fc7cb1ff700 1 -- start start 2023-07-13T22:22:54.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7cb1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7c40b8400 0x7fc7c40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:54.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7cb1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc7c40cfca0 0x7fc7c40d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:54.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7cb1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c40d0ac0 con 0x7fc7c40d3750 2023-07-13T22:22:54.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7cb1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c40b87d0 con 0x7fc7c40b8400 2023-07-13T22:22:54.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7cb1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c40b4eb0 con 0x7fc7c40cfca0 2023-07-13T22:22:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7c3fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc7c40d3750 0x7fc7c40cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45266/0 (socket says 172.21.15.60:45266) 2023-07-13T22:22:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7c3fff700 1 -- 172.21.15.60:0/3600158145 learned_addr learned my addr 172.21.15.60:0/3600158145 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7c8f9b700 1 --2- 172.21.15.60:0/3600158145 >> v2:172.21.15.60:3300/0 conn(0x7fc7c40b8400 0x7fc7c40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7c979c700 1 --2- 172.21.15.60:0/3600158145 >> v2:172.21.15.105:3300/0 conn(0x7fc7c40cfca0 0x7fc7c40d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7c37fe700 1 -- 172.21.15.60:0/3600158145 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657941682 0 0) 0x7fc7c40d0ac0 con 0x7fc7c40d3750 2023-07-13T22:22:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7c8f9b700 1 -- 172.21.15.60:0/3600158145 >> v2:172.21.15.105:3300/0 conn(0x7fc7c40cfca0 msgr2=0x7fc7c40d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:54.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7c8f9b700 1 --2- 172.21.15.60:0/3600158145 >> v2:172.21.15.105:3300/0 conn(0x7fc7c40cfca0 0x7fc7c40d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:54.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.223+0000 7fc7c8f9b700 1 -- 172.21.15.60:0/3600158145 >> v1:172.21.15.60:6789/0 conn(0x7fc7c40d3750 legacy=0x7fc7c40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:54.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.224+0000 7fc7c8f9b700 1 -- 172.21.15.60:0/3600158145 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40b5190 con 0x7fc7c40b8400 2023-07-13T22:22:54.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.224+0000 7fc7c8f9b700 1 --2- 172.21.15.60:0/3600158145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40b8400 0x7fc7c40cd240 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40d0070 tx=0x7fc7b8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7a04380171a3dd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:54.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.224+0000 7fc7c37fe700 1 -- 172.21.15.60:0/3600158145 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8011830 con 0x7fc7c40b8400 2023-07-13T22:22:54.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.224+0000 7fc7c37fe700 1 -- 172.21.15.60:0/3600158145 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8011990 con 0x7fc7c40b8400 2023-07-13T22:22:54.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.224+0000 7fc7c37fe700 1 -- 172.21.15.60:0/3600158145 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8011d20 con 0x7fc7c40b8400 2023-07-13T22:22:54.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.225+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/3600158145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40b8400 msgr2=0x7fc7c40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:54.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.225+0000 7fc7cb1ff700 1 --2- 172.21.15.60:0/3600158145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40b8400 0x7fc7c40cd240 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40d0070 tx=0x7fc7b8009b20 comp rx=0 tx=0).stop 2023-07-13T22:22:54.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.225+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/3600158145 shutdown_connections 2023-07-13T22:22:54.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.225+0000 7fc7cb1ff700 1 --2- 172.21.15.60:0/3600158145 >> v2:172.21.15.105:3300/0 conn(0x7fc7c40cfca0 0x7fc7c40d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:54.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.225+0000 7fc7cb1ff700 1 --2- 172.21.15.60:0/3600158145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40b8400 0x7fc7c40cd240 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:54.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.225+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/3600158145 >> 172.21.15.60:0/3600158145 conn(0x7fc7c406b390 msgr2=0x7fc7c406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:54.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.225+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/3600158145 shutdown_connections 2023-07-13T22:22:54.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.225+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/3600158145 wait complete. 2023-07-13T22:22:54.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.227+0000 7fc7cb1ff700 1 Processor -- start 2023-07-13T22:22:54.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.227+0000 7fc7cb1ff700 1 -- start start 2023-07-13T22:22:54.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7cb1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c40b8400 0x7fc7c4153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7cb1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7c40cfca0 0x7fc7c415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7cb1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40d3750 0x7fc7c40b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7cb1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c415ca70 con 0x7fc7c40d3750 2023-07-13T22:22:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7cb1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c415cd50 con 0x7fc7c40b8400 2023-07-13T22:22:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7cb1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c415d030 con 0x7fc7c40cfca0 2023-07-13T22:22:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c979c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40d3750 0x7fc7c40b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c979c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40d3750 0x7fc7c40b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41018/0 (socket says 172.21.15.60:41018) 2023-07-13T22:22:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c979c700 1 -- 172.21.15.60:0/1753551176 learned_addr learned my addr 172.21.15.60:0/1753551176 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c8f9b700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c40b8400 0x7fc7c4153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c3fff700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7c40cfca0 0x7fc7c415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c979c700 1 -- 172.21.15.60:0/1753551176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7c40cfca0 msgr2=0x7fc7c415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c979c700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7c40cfca0 0x7fc7c415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c979c700 1 -- 172.21.15.60:0/1753551176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c40b8400 msgr2=0x7fc7c4153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c979c700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c40b8400 0x7fc7c4153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c979c700 1 -- 172.21.15.60:0/1753551176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b80094a0 con 0x7fc7c40d3750 2023-07-13T22:22:54.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.228+0000 7fc7c3fff700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7c40cfca0 0x7fc7c415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:54.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.229+0000 7fc7c979c700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40d3750 0x7fc7c40b59b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00ef90 tx=0x7fc7bc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:54.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.229+0000 7fc7c17fa700 1 -- 172.21.15.60:0/1753551176 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc009b20 con 0x7fc7c40d3750 2023-07-13T22:22:54.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.229+0000 7fc7c17fa700 1 -- 172.21.15.60:0/1753551176 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc00f040 con 0x7fc7c40d3750 2023-07-13T22:22:54.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.229+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c415d310 con 0x7fc7c40d3750 2023-07-13T22:22:54.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.229+0000 7fc7c17fa700 1 -- 172.21.15.60:0/1753551176 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00cd20 con 0x7fc7c40d3750 2023-07-13T22:22:54.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.229+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c415e050 con 0x7fc7c40d3750 2023-07-13T22:22:54.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.230+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c415d830 con 0x7fc7c40d3750 2023-07-13T22:22:54.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.231+0000 7fc7c17fa700 1 -- 172.21.15.60:0/1753551176 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc7bc014480 con 0x7fc7c40d3750 2023-07-13T22:22:54.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.232+0000 7fc7c17fa700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc7a806ace0 0x7fc7a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:54.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.232+0000 7fc7c17fa700 1 -- 172.21.15.60:0/1753551176 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fc7bc089fc0 con 0x7fc7c40d3750 2023-07-13T22:22:54.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.233+0000 7fc7c8f9b700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc7a806ace0 0x7fc7a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:54.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.233+0000 7fc7c8f9b700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc7a806ace0 0x7fc7a806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80090b0 tx=0x7fc7b801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:54.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.236+0000 7fc7c17fa700 1 -- 172.21.15.60:0/1753551176 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc7c415d830 con 0x7fc7c40d3750 2023-07-13T22:22:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.461+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc7c415d830 con 0x7fc7c40d3750 2023-07-13T22:22:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.461+0000 7fc7c17fa700 1 -- 172.21.15.60:0/1753551176 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc7c415d830 con 0x7fc7c40d3750 2023-07-13T22:22:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc7a806ace0 msgr2=0x7fc7a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc7a806ace0 0x7fc7a806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80090b0 tx=0x7fc7b801c040 comp rx=0 tx=0).stop 2023-07-13T22:22:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40d3750 msgr2=0x7fc7c40b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40d3750 0x7fc7c40b59b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00ef90 tx=0x7fc7bc009f60 comp rx=0 tx=0).stop 2023-07-13T22:22:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 shutdown_connections 2023-07-13T22:22:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc7a806ace0 0x7fc7a806d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c40d3750 0x7fc7c40b59b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7c40b8400 0x7fc7c4153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 --2- 172.21.15.60:0/1753551176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7c40cfca0 0x7fc7c415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 >> 172.21.15.60:0/1753551176 conn(0x7fc7c406b390 msgr2=0x7fc7c40b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 shutdown_connections 2023-07-13T22:22:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:54.464+0000 7fc7cb1ff700 1 -- 172.21.15.60:0/1753551176 wait complete. 2023-07-13T22:22:54.450 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:22:54.466 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:22:54.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:54.466 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:22:54.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[120845]: fuse finished with error 0 and tester_r 0 2023-07-13T22:22:54.544 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:22:54.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:54.545 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:22:54.703 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:22:54.703 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:54.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:22:54.750 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:22:54.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:54.751 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:22:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-13T22:22:54.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:22:54.836 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:22:54.837 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:22:54.884 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:22:54.884 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:22:54.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:22:54.885 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:22:54.969 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-13T22:22:54.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:22:54.971 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:55.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.188+0000 7ff4131ff700 1 Processor -- start 2023-07-13T22:22:55.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.189+0000 7ff4131ff700 1 -- start start 2023-07-13T22:22:55.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.190+0000 7ff4131ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c154950 0x7ff40c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.190+0000 7ff4131ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff40c0c9450 0x7ff40c15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.190+0000 7ff4131ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c070270 con 0x7ff40c15a820 2023-07-13T22:22:55.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.190+0000 7ff4131ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40c15b2e0 con 0x7ff40c0c9450 2023-07-13T22:22:55.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.190+0000 7ff4131ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c15b5c0 con 0x7ff40c154950 2023-07-13T22:22:55.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.190+0000 7ff40bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff40c15a820 0x7ff40c0c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60964/0 (socket says 172.21.15.60:60964) 2023-07-13T22:22:55.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.190+0000 7ff40bfff700 1 -- 172.21.15.60:0/4081519724 learned_addr learned my addr 172.21.15.60:0/4081519724 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:55.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.190+0000 7ff41179c700 1 --2- 172.21.15.60:0/4081519724 >> v2:172.21.15.105:3300/0 conn(0x7ff40c0c9450 0x7ff40c15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.190+0000 7ff410f9b700 1 --2- 172.21.15.60:0/4081519724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c154950 0x7ff40c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.191+0000 7ff40b7fe700 1 -- 172.21.15.60:0/4081519724 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938829318 0 0) 0x7ff40c070270 con 0x7ff40c15a820 2023-07-13T22:22:55.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.191+0000 7ff40b7fe700 1 -- 172.21.15.60:0/4081519724 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3fc0035b0 con 0x7ff40c15a820 2023-07-13T22:22:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.191+0000 7ff41179c700 1 -- 172.21.15.60:0/4081519724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c154950 msgr2=0x7ff40c070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.191+0000 7ff41179c700 1 --2- 172.21.15.60:0/4081519724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c154950 0x7ff40c070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.191+0000 7ff41179c700 1 -- 172.21.15.60:0/4081519724 >> v1:172.21.15.60:6789/0 conn(0x7ff40c15a820 legacy=0x7ff40c0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.191+0000 7ff41179c700 1 -- 172.21.15.60:0/4081519724 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0704c0 con 0x7ff40c0c9450 2023-07-13T22:22:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.191+0000 7ff410f9b700 1 --2- 172.21.15.60:0/4081519724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c154950 0x7ff40c070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.191+0000 7ff41179c700 1 --2- 172.21.15.60:0/4081519724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff40c0c9450 0x7ff40c15ac90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff404009d90 tx=0x7ff40400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e1a027682c8c0e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.191+0000 7ff40b7fe700 1 -- 172.21.15.60:0/4081519724 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40400e070 con 0x7ff40c0c9450 2023-07-13T22:22:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff40b7fe700 1 -- 172.21.15.60:0/4081519724 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40400bb20 con 0x7ff40c0c9450 2023-07-13T22:22:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff40b7fe700 1 -- 172.21.15.60:0/4081519724 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404012430 con 0x7ff40c0c9450 2023-07-13T22:22:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff4131ff700 1 -- 172.21.15.60:0/4081519724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff40c0c9450 msgr2=0x7ff40c15ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff4131ff700 1 --2- 172.21.15.60:0/4081519724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff40c0c9450 0x7ff40c15ac90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff404009d90 tx=0x7ff40400b330 comp rx=0 tx=0).stop 2023-07-13T22:22:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff4131ff700 1 -- 172.21.15.60:0/4081519724 shutdown_connections 2023-07-13T22:22:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff4131ff700 1 --2- 172.21.15.60:0/4081519724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff40c0c9450 0x7ff40c15ac90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff4131ff700 1 --2- 172.21.15.60:0/4081519724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c154950 0x7ff40c070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff4131ff700 1 -- 172.21.15.60:0/4081519724 >> 172.21.15.60:0/4081519724 conn(0x7ff40c0cf410 msgr2=0x7ff40c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:55.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff4131ff700 1 -- 172.21.15.60:0/4081519724 shutdown_connections 2023-07-13T22:22:55.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.192+0000 7ff4131ff700 1 -- 172.21.15.60:0/4081519724 wait complete. 2023-07-13T22:22:55.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.194+0000 7ff4131ff700 1 Processor -- start 2023-07-13T22:22:55.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.194+0000 7ff4131ff700 1 -- start start 2023-07-13T22:22:55.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff4131ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40c0c9450 0x7ff40c144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff4131ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff40c154950 0x7ff40c0aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff4131ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c15a820 0x7ff40c0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff4131ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c159460 con 0x7ff40c0c9450 2023-07-13T22:22:55.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff4131ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c159740 con 0x7ff40c154950 2023-07-13T22:22:55.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff4131ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c159a20 con 0x7ff40c15a820 2023-07-13T22:22:55.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff410f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40c0c9450 0x7ff40c144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff410f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40c0c9450 0x7ff40c144170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52894/0 (socket says 172.21.15.60:52894) 2023-07-13T22:22:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff410f9b700 1 -- 172.21.15.60:0/4162510037 learned_addr learned my addr 172.21.15.60:0/4162510037 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff41179c700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c15a820 0x7ff40c0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.195+0000 7ff40bfff700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff40c154950 0x7ff40c0aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff410f9b700 1 -- 172.21.15.60:0/4162510037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c15a820 msgr2=0x7ff40c0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff410f9b700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c15a820 0x7ff40c0af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff410f9b700 1 -- 172.21.15.60:0/4162510037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff40c154950 msgr2=0x7ff40c0aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff410f9b700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff40c154950 0x7ff40c0aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff410f9b700 1 -- 172.21.15.60:0/4162510037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4000094a0 con 0x7ff40c0c9450 2023-07-13T22:22:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff410f9b700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40c0c9450 0x7ff40c144170 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff40000ebe0 tx=0x7ff40000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff4097fa700 1 -- 172.21.15.60:0/4162510037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40000c850 con 0x7ff40c0c9450 2023-07-13T22:22:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff4097fa700 1 -- 172.21.15.60:0/4162510037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40000c9b0 con 0x7ff40c0c9450 2023-07-13T22:22:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404009a70 con 0x7ff40c0c9450 2023-07-13T22:22:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff4097fa700 1 -- 172.21.15.60:0/4162510037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40000cd40 con 0x7ff40c0c9450 2023-07-13T22:22:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.196+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c0cc360 con 0x7ff40c0c9450 2023-07-13T22:22:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.197+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c04e280 con 0x7ff40c0c9450 2023-07-13T22:22:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.198+0000 7ff4097fa700 1 -- 172.21.15.60:0/4162510037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff400013470 con 0x7ff40c0c9450 2023-07-13T22:22:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.200+0000 7ff4097fa700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f006ace0 0x7ff3f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.200+0000 7ff4097fa700 1 -- 172.21.15.60:0/4162510037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7ff4000895f0 con 0x7ff40c0c9450 2023-07-13T22:22:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.200+0000 7ff40bfff700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f006ace0 0x7ff3f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.201+0000 7ff40bfff700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f006ace0 0x7ff3f006d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8007fd0 tx=0x7ff3f8007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.204+0000 7ff4097fa700 1 -- 172.21.15.60:0/4162510037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff4000545e0 con 0x7ff40c0c9450 2023-07-13T22:22:55.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:22:55.275 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:22:55.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.456+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff40c0cbc20 con 0x7ff40c0c9450 2023-07-13T22:22:55.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.456+0000 7ff4097fa700 1 -- 172.21.15.60:0/4162510037 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7ff40000cea0 con 0x7ff40c0c9450 2023-07-13T22:22:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:55.441 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2157014443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2157014443},{"type":"v1","addr":"172.21.15.105:6834","nonce":2157014443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/302067138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":302067138},{"type":"v1","addr":"172.21.15.105:6835","nonce":302067138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3461450784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461450784},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461450784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:22:19.175830+0000","modified":"2023-07-13T22:22:22.691356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6837/2048874349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2048874349},{"type":"v1","addr":"172.21.15.60:6837","nonce":2048874349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-13T22:22:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f006ace0 msgr2=0x7ff3f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f006ace0 0x7ff3f006d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8007fd0 tx=0x7ff3f8007dc0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40c0c9450 msgr2=0x7ff40c144170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40c0c9450 0x7ff40c144170 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff40000ebe0 tx=0x7ff40000eef0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 shutdown_connections 2023-07-13T22:22:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3f006ace0 0x7ff3f006d190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40c0c9450 0x7ff40c144170 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff40c15a820 0x7ff40c0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 --2- 172.21.15.60:0/4162510037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff40c154950 0x7ff40c0aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 >> 172.21.15.60:0/4162510037 conn(0x7ff40c0cf410 msgr2=0x7ff40c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 shutdown_connections 2023-07-13T22:22:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.459+0000 7ff4131ff700 1 -- 172.21.15.60:0/4162510037 wait complete. 2023-07-13T22:22:55.445 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2023-07-13T22:22:55.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.680+0000 7fcd545ff700 1 Processor -- start 2023-07-13T22:22:55.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.680+0000 7fcd545ff700 1 -- start start 2023-07-13T22:22:55.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.682+0000 7fcd545ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcd4c154950 0x7fcd4c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.682+0000 7fcd545ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd4c0c9450 0x7fcd4c15ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.682+0000 7fcd545ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4c070270 con 0x7fcd4c15a820 2023-07-13T22:22:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.682+0000 7fcd545ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4c15b2e0 con 0x7fcd4c0c9450 2023-07-13T22:22:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.682+0000 7fcd545ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4c15b5c0 con 0x7fcd4c154950 2023-07-13T22:22:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.682+0000 7fcd51b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcd4c15a820 0x7fcd4c0c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60966/0 (socket says 172.21.15.60:60966) 2023-07-13T22:22:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.682+0000 7fcd51b9a700 1 -- 172.21.15.60:0/2822582820 learned_addr learned my addr 172.21.15.60:0/2822582820 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.682+0000 7fcd52b9c700 1 --2- 172.21.15.60:0/2822582820 >> v2:172.21.15.60:3300/0 conn(0x7fcd4c0c9450 0x7fcd4c15ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.682+0000 7fcd5239b700 1 --2- 172.21.15.60:0/2822582820 >> v2:172.21.15.105:3300/0 conn(0x7fcd4c154950 0x7fcd4c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.683+0000 7fcd51399700 1 -- 172.21.15.60:0/2822582820 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651815185 0 0) 0x7fcd4c070270 con 0x7fcd4c15a820 2023-07-13T22:22:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.683+0000 7fcd52b9c700 1 -- 172.21.15.60:0/2822582820 >> v2:172.21.15.105:3300/0 conn(0x7fcd4c154950 msgr2=0x7fcd4c070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.683+0000 7fcd52b9c700 1 --2- 172.21.15.60:0/2822582820 >> v2:172.21.15.105:3300/0 conn(0x7fcd4c154950 0x7fcd4c070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.683+0000 7fcd52b9c700 1 -- 172.21.15.60:0/2822582820 >> v1:172.21.15.60:6789/0 conn(0x7fcd4c15a820 legacy=0x7fcd4c0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.683+0000 7fcd52b9c700 1 -- 172.21.15.60:0/2822582820 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c0704c0 con 0x7fcd4c0c9450 2023-07-13T22:22:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.683+0000 7fcd52b9c700 1 --2- 172.21.15.60:0/2822582820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c0c9450 0x7fcd4c15ac70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0ce6e0 tx=0x7fcd4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c827e0c443a9d13 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.683+0000 7fcd51399700 1 -- 172.21.15.60:0/2822582820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400e070 con 0x7fcd4c0c9450 2023-07-13T22:22:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.683+0000 7fcd51399700 1 -- 172.21.15.60:0/2822582820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4400bb20 con 0x7fcd4c0c9450 2023-07-13T22:22:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.683+0000 7fcd51399700 1 -- 172.21.15.60:0/2822582820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44012430 con 0x7fcd4c0c9450 2023-07-13T22:22:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.684+0000 7fcd545ff700 1 -- 172.21.15.60:0/2822582820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c0c9450 msgr2=0x7fcd4c15ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.684+0000 7fcd545ff700 1 --2- 172.21.15.60:0/2822582820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c0c9450 0x7fcd4c15ac70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0ce6e0 tx=0x7fcd4400b330 comp rx=0 tx=0).stop 2023-07-13T22:22:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.684+0000 7fcd545ff700 1 -- 172.21.15.60:0/2822582820 shutdown_connections 2023-07-13T22:22:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.684+0000 7fcd545ff700 1 --2- 172.21.15.60:0/2822582820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c0c9450 0x7fcd4c15ac70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.684+0000 7fcd545ff700 1 --2- 172.21.15.60:0/2822582820 >> v2:172.21.15.105:3300/0 conn(0x7fcd4c154950 0x7fcd4c070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.684+0000 7fcd545ff700 1 -- 172.21.15.60:0/2822582820 >> 172.21.15.60:0/2822582820 conn(0x7fcd4c0cf410 msgr2=0x7fcd4c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.684+0000 7fcd545ff700 1 -- 172.21.15.60:0/2822582820 shutdown_connections 2023-07-13T22:22:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.684+0000 7fcd545ff700 1 -- 172.21.15.60:0/2822582820 wait complete. 2023-07-13T22:22:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.685+0000 7fcd545ff700 1 Processor -- start 2023-07-13T22:22:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.686+0000 7fcd545ff700 1 -- start start 2023-07-13T22:22:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd545ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c0c9450 0x7fcd4c0c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd545ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c154950 0x7fcd4c144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd545ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c15a820 0x7fcd4c0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd545ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0c57c0 con 0x7fcd4c15a820 2023-07-13T22:22:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd545ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c144740 con 0x7fcd4c154950 2023-07-13T22:22:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd545ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0c5a10 con 0x7fcd4c0c9450 2023-07-13T22:22:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd52b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c15a820 0x7fcd4c0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd52b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c15a820 0x7fcd4c0b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52906/0 (socket says 172.21.15.60:52906) 2023-07-13T22:22:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd52b9c700 1 -- 172.21.15.60:0/1276965819 learned_addr learned my addr 172.21.15.60:0/1276965819 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd5239b700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c0c9450 0x7fcd4c0c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd51b9a700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c154950 0x7fcd4c144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd52b9c700 1 -- 172.21.15.60:0/1276965819 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c0c9450 msgr2=0x7fcd4c0c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd52b9c700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c0c9450 0x7fcd4c0c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd52b9c700 1 -- 172.21.15.60:0/1276965819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c154950 msgr2=0x7fcd4c144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd52b9c700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c154950 0x7fcd4c144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.687+0000 7fcd52b9c700 1 -- 172.21.15.60:0/1276965819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd44009a70 con 0x7fcd4c15a820 2023-07-13T22:22:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.688+0000 7fcd52b9c700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c15a820 0x7fcd4c0b3830 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400b680 tx=0x7fcd44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.688+0000 7fcd42ffd700 1 -- 172.21.15.60:0/1276965819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44012430 con 0x7fcd4c15a820 2023-07-13T22:22:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.688+0000 7fcd42ffd700 1 -- 172.21.15.60:0/1276965819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd44020070 con 0x7fcd4c15a820 2023-07-13T22:22:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.688+0000 7fcd42ffd700 1 -- 172.21.15.60:0/1276965819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44004280 con 0x7fcd4c15a820 2023-07-13T22:22:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.688+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4c0c5cf0 con 0x7fcd4c15a820 2023-07-13T22:22:55.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.688+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c0c62b0 con 0x7fcd4c15a820 2023-07-13T22:22:55.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.689+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4c04e280 con 0x7fcd4c15a820 2023-07-13T22:22:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.690+0000 7fcd42ffd700 1 -- 172.21.15.60:0/1276965819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd440043e0 con 0x7fcd4c15a820 2023-07-13T22:22:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.691+0000 7fcd42ffd700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd3006ace0 0x7fcd3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.691+0000 7fcd42ffd700 1 -- 172.21.15.60:0/1276965819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fcd44022070 con 0x7fcd4c15a820 2023-07-13T22:22:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.691+0000 7fcd5239b700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd3006ace0 0x7fcd3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.692+0000 7fcd5239b700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd3006ace0 0x7fcd3006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009b70 tx=0x7fcd48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:55.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.695+0000 7fcd42ffd700 1 -- 172.21.15.60:0/1276965819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcd440bd330 con 0x7fcd4c15a820 2023-07-13T22:22:55.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.902+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd4c0cbc20 con 0x7fcd4c15a820 2023-07-13T22:22:55.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.903+0000 7fcd42ffd700 1 -- 172.21.15.60:0/1276965819 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18250 (secure 0 0 0) 0x7fcd440562a0 con 0x7fcd4c15a820 2023-07-13T22:22:55.887 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:55.888 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:22:43.507730+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:22:17.071217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-13T22:22:18.161600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:55.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd3006ace0 msgr2=0x7fcd3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd3006ace0 0x7fcd3006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009b70 tx=0x7fcd48008040 comp rx=0 tx=0).stop 2023-07-13T22:22:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c15a820 msgr2=0x7fcd4c0b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c15a820 0x7fcd4c0b3830 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400b680 tx=0x7fcd44004ad0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 shutdown_connections 2023-07-13T22:22:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd3006ace0 0x7fcd3006d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd4c15a820 0x7fcd4c0b3830 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c0c9450 0x7fcd4c0c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 --2- 172.21.15.60:0/1276965819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c154950 0x7fcd4c144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 >> 172.21.15.60:0/1276965819 conn(0x7fcd4c0cf410 msgr2=0x7fcd4c159130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 shutdown_connections 2023-07-13T22:22:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:55.906+0000 7fcd545ff700 1 -- 172.21.15.60:0/1276965819 wait complete. 2023-07-13T22:22:55.908 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:22:17.071217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:22:18.161600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:22:55.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:22:56.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.131+0000 7fe4d63ff700 1 Processor -- start 2023-07-13T22:22:56.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.131+0000 7fe4d63ff700 1 -- start start 2023-07-13T22:22:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.132+0000 7fe4d63ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe4d015c790 0x7fe4d00ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.132+0000 7fe4d63ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 0x7fe4d00c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4d63ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d00cf6d0 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4d63ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d00ac900 con 0x7fe4d015c790 2023-07-13T22:22:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4d63ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00acbe0 con 0x7fe4d00ac490 2023-07-13T22:22:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 0x7fe4d00c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4d499c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe4d00ca3a0 0x7fe4d00ad9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:58320/0 (socket says 172.21.15.60:58320) 2023-07-13T22:22:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4cffff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe4d015c790 0x7fe4d00ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4d499c700 1 -- 172.21.15.60:0/650088011 learned_addr learned my addr 172.21.15.60:0/650088011 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4c7fff700 1 -- 172.21.15.60:0/650088011 >> v2:172.21.15.105:3300/0 conn(0x7fe4d015c790 msgr2=0x7fe4d00ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4c7fff700 1 --2- 172.21.15.60:0/650088011 >> v2:172.21.15.105:3300/0 conn(0x7fe4d015c790 0x7fe4d00ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4c7fff700 1 -- 172.21.15.60:0/650088011 >> v1:172.21.15.105:6789/0 conn(0x7fe4d00ca3a0 legacy=0x7fe4d00ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4c7fff700 1 -- 172.21.15.60:0/650088011 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d00acec0 con 0x7fe4d00ac490 2023-07-13T22:22:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.133+0000 7fe4cf7fe700 1 -- 172.21.15.60:0/650088011 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501424834 0 0) 0x7fe4d00cf6d0 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.134+0000 7fe4c7fff700 1 --2- 172.21.15.60:0/650088011 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 0x7fe4d00c9e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009840 tx=0x7fe4b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc3684150d5c5f71 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.134+0000 7fe4cf7fe700 1 -- 172.21.15.60:0/650088011 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8011830 con 0x7fe4d00ac490 2023-07-13T22:22:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.134+0000 7fe4cf7fe700 1 -- 172.21.15.60:0/650088011 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b8011990 con 0x7fe4d00ac490 2023-07-13T22:22:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.134+0000 7fe4cf7fe700 1 -- 172.21.15.60:0/650088011 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8011d20 con 0x7fe4d00ac490 2023-07-13T22:22:56.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.135+0000 7fe4d63ff700 1 -- 172.21.15.60:0/650088011 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 msgr2=0x7fe4d00c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.135+0000 7fe4d63ff700 1 --2- 172.21.15.60:0/650088011 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 0x7fe4d00c9e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009840 tx=0x7fe4b8009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.135+0000 7fe4d63ff700 1 -- 172.21.15.60:0/650088011 shutdown_connections 2023-07-13T22:22:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.135+0000 7fe4d63ff700 1 --2- 172.21.15.60:0/650088011 >> v2:172.21.15.105:3300/0 conn(0x7fe4d015c790 0x7fe4d00ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.135+0000 7fe4d63ff700 1 --2- 172.21.15.60:0/650088011 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 0x7fe4d00c9e60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.135+0000 7fe4d63ff700 1 -- 172.21.15.60:0/650088011 >> 172.21.15.60:0/650088011 conn(0x7fe4d006b900 msgr2=0x7fe4d00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.135+0000 7fe4d63ff700 1 -- 172.21.15.60:0/650088011 shutdown_connections 2023-07-13T22:22:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.135+0000 7fe4d63ff700 1 -- 172.21.15.60:0/650088011 wait complete. 2023-07-13T22:22:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.137+0000 7fe4d63ff700 1 Processor -- start 2023-07-13T22:22:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.137+0000 7fe4d63ff700 1 -- start start 2023-07-13T22:22:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4d63ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 0x7fe4d015f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4d63ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d00ca3a0 0x7fe4d01f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4d63ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d015c790 0x7fe4d00b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4d63ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d015d0d0 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4d63ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00c8160 con 0x7fe4d015c790 2023-07-13T22:22:56.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4d63ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00c8440 con 0x7fe4d00ac490 2023-07-13T22:22:56.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d00ca3a0 0x7fe4d01f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d00ca3a0 0x7fe4d01f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52910/0 (socket says 172.21.15.60:52910) 2023-07-13T22:22:56.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4c7fff700 1 -- 172.21.15.60:0/789349504 learned_addr learned my addr 172.21.15.60:0/789349504 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:56.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4cffff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 0x7fe4d015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4d499c700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d015c790 0x7fe4d00b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4c7fff700 1 -- 172.21.15.60:0/789349504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 msgr2=0x7fe4d015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4c7fff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 0x7fe4d015f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4c7fff700 1 -- 172.21.15.60:0/789349504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d015c790 msgr2=0x7fe4d00b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4c7fff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d015c790 0x7fe4d00b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.138+0000 7fe4c7fff700 1 -- 172.21.15.60:0/789349504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b80094a0 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.139+0000 7fe4c7fff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d00ca3a0 0x7fe4d01f22f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009b90 tx=0x7fe4b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:56.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.139+0000 7fe4cd7fa700 1 -- 172.21.15.60:0/789349504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80036a0 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.139+0000 7fe4cd7fa700 1 -- 172.21.15.60:0/789349504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b801e070 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.139+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d00c8720 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.139+0000 7fe4cd7fa700 1 -- 172.21.15.60:0/789349504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8003a60 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.139+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d015b220 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.140+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d004e210 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.141+0000 7fe4cd7fa700 1 -- 172.21.15.60:0/789349504 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4b8003bc0 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.142+0000 7fe4cd7fa700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4b006ace0 0x7fe4b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.142+0000 7fe4cd7fa700 1 -- 172.21.15.60:0/789349504 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fe4b808a450 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.142+0000 7fe4cffff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4b006ace0 0x7fe4b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.143+0000 7fe4cffff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4b006ace0 0x7fe4b006d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009bc0 tx=0x7fe4c00091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:56.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.146+0000 7fe4cd7fa700 1 -- 172.21.15.60:0/789349504 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe4b8055440 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.355+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe4d00cddb0 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.355+0000 7fe4cd7fa700 1 -- 172.21.15.60:0/789349504 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe4b8058a60 con 0x7fe4d00ca3a0 2023-07-13T22:22:56.339 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:22:56.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.357+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4b006ace0 msgr2=0x7fe4b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.357+0000 7fe4d63ff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4b006ace0 0x7fe4b006d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009bc0 tx=0x7fe4c00091c0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.357+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d00ca3a0 msgr2=0x7fe4d01f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.357+0000 7fe4d63ff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d00ca3a0 0x7fe4d01f22f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009b90 tx=0x7fe4b8004750 comp rx=0 tx=0).stop 2023-07-13T22:22:56.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.358+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 shutdown_connections 2023-07-13T22:22:56.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.358+0000 7fe4d63ff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4b006ace0 0x7fe4b006d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.358+0000 7fe4d63ff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d015c790 0x7fe4d00b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.358+0000 7fe4d63ff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d00ac490 0x7fe4d015f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.358+0000 7fe4d63ff700 1 --2- 172.21.15.60:0/789349504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d00ca3a0 0x7fe4d01f22f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.358+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 >> 172.21.15.60:0/789349504 conn(0x7fe4d006b900 msgr2=0x7fe4d00b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:56.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.358+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 shutdown_connections 2023-07-13T22:22:56.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.358+0000 7fe4d63ff700 1 -- 172.21.15.60:0/789349504 wait complete. 2023-07-13T22:22:56.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:22:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.582+0000 7f3ba95ff700 1 Processor -- start 2023-07-13T22:22:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.582+0000 7f3ba95ff700 1 -- start start 2023-07-13T22:22:56.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.583+0000 7f3ba95ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3ba40ca900 0x7f3ba40703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.583+0000 7f3ba95ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba4070920 0x7f3ba406c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.583+0000 7f3ba95ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba4070d60 con 0x7f3ba4159260 2023-07-13T22:22:56.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.583+0000 7f3ba95ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba40cada0 con 0x7f3ba40ca900 2023-07-13T22:22:56.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.583+0000 7f3ba95ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40cf640 con 0x7f3ba4070920 2023-07-13T22:22:56.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba2ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3ba4159260 0x7f3ba40cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60982/0 (socket says 172.21.15.60:60982) 2023-07-13T22:22:56.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba2ffd700 1 -- 172.21.15.60:0/487955524 learned_addr learned my addr 172.21.15.60:0/487955524 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:56.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba37fe700 1 --2- 172.21.15.60:0/487955524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba4070920 0x7f3ba406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3b9bfff700 1 --2- 172.21.15.60:0/487955524 >> v2:172.21.15.105:3300/0 conn(0x7f3ba40ca900 0x7f3ba40703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba27fc700 1 -- 172.21.15.60:0/487955524 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122118321 0 0) 0x7f3ba4070d60 con 0x7f3ba4159260 2023-07-13T22:22:56.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba27fc700 1 -- 172.21.15.60:0/487955524 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b900035b0 con 0x7f3ba4159260 2023-07-13T22:22:56.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba37fe700 1 -- 172.21.15.60:0/487955524 >> v2:172.21.15.105:3300/0 conn(0x7f3ba40ca900 msgr2=0x7f3ba40703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:56.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba37fe700 1 --2- 172.21.15.60:0/487955524 >> v2:172.21.15.105:3300/0 conn(0x7f3ba40ca900 0x7f3ba40703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba37fe700 1 -- 172.21.15.60:0/487955524 >> v1:172.21.15.60:6789/0 conn(0x7f3ba4159260 legacy=0x7f3ba40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba27fc700 1 -- 172.21.15.60:0/487955524 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3802450334 0 0) 0x7f3b900035b0 con 0x7f3ba4159260 2023-07-13T22:22:56.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.584+0000 7f3ba37fe700 1 -- 172.21.15.60:0/487955524 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba40ccf80 con 0x7f3ba4070920 2023-07-13T22:22:56.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.585+0000 7f3ba37fe700 1 --2- 172.21.15.60:0/487955524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba4070920 0x7f3ba406c440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009d90 tx=0x7f3b9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=258200ebde0d7058 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:56.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.585+0000 7f3ba27fc700 1 -- 172.21.15.60:0/487955524 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00e070 con 0x7f3ba4070920 2023-07-13T22:22:56.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.585+0000 7f3ba27fc700 1 -- 172.21.15.60:0/487955524 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c003a20 con 0x7f3ba4070920 2023-07-13T22:22:56.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.585+0000 7f3ba27fc700 1 -- 172.21.15.60:0/487955524 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94001670 con 0x7f3ba4070920 2023-07-13T22:22:56.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.585+0000 7f3ba95ff700 1 -- 172.21.15.60:0/487955524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba4070920 msgr2=0x7f3ba406c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.586+0000 7f3ba95ff700 1 --2- 172.21.15.60:0/487955524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba4070920 0x7f3ba406c440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009d90 tx=0x7f3b9c00b330 comp rx=0 tx=0).stop 2023-07-13T22:22:56.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.586+0000 7f3ba95ff700 1 -- 172.21.15.60:0/487955524 shutdown_connections 2023-07-13T22:22:56.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.586+0000 7f3ba95ff700 1 --2- 172.21.15.60:0/487955524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba4070920 0x7f3ba406c440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.586+0000 7f3ba95ff700 1 --2- 172.21.15.60:0/487955524 >> v2:172.21.15.105:3300/0 conn(0x7f3ba40ca900 0x7f3ba40703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.586+0000 7f3ba95ff700 1 -- 172.21.15.60:0/487955524 >> 172.21.15.60:0/487955524 conn(0x7f3ba40cbf50 msgr2=0x7f3ba40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:56.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.586+0000 7f3ba95ff700 1 -- 172.21.15.60:0/487955524 shutdown_connections 2023-07-13T22:22:56.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.586+0000 7f3ba95ff700 1 -- 172.21.15.60:0/487955524 wait complete. 2023-07-13T22:22:56.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.588+0000 7f3ba95ff700 1 Processor -- start 2023-07-13T22:22:56.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.588+0000 7f3ba95ff700 1 -- start start 2023-07-13T22:22:56.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.589+0000 7f3ba95ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ba4070920 0x7f3ba415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba95ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba40ca900 0x7f3ba40ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba95ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba4159260 0x7f3ba40ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba95ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40b11b0 con 0x7f3ba4070920 2023-07-13T22:22:56.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba95ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40b0ed0 con 0x7f3ba4159260 2023-07-13T22:22:56.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba95ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4164130 con 0x7f3ba40ca900 2023-07-13T22:22:56.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ba4070920 0x7f3ba415c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ba4070920 0x7f3ba415c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52912/0 (socket says 172.21.15.60:52912) 2023-07-13T22:22:56.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba2ffd700 1 -- 172.21.15.60:0/650103309 learned_addr learned my addr 172.21.15.60:0/650103309 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:56.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3b9bfff700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba40ca900 0x7f3ba40ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba37fe700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba4159260 0x7f3ba40ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba2ffd700 1 -- 172.21.15.60:0/650103309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba40ca900 msgr2=0x7f3ba40ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba2ffd700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba40ca900 0x7f3ba40ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba2ffd700 1 -- 172.21.15.60:0/650103309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba4159260 msgr2=0x7f3ba40ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba2ffd700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba4159260 0x7f3ba40ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.590+0000 7f3ba2ffd700 1 -- 172.21.15.60:0/650103309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009a70 con 0x7f3ba4070920 2023-07-13T22:22:56.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.591+0000 7f3ba2ffd700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ba4070920 0x7f3ba415c0c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3b940011e0 tx=0x7f3b9400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:56.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.591+0000 7f3b9b7fe700 1 -- 172.21.15.60:0/650103309 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94004c80 con 0x7f3ba4070920 2023-07-13T22:22:56.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.591+0000 7f3b9b7fe700 1 -- 172.21.15.60:0/650103309 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b94004e00 con 0x7f3ba4070920 2023-07-13T22:22:56.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.591+0000 7f3b9b7fe700 1 -- 172.21.15.60:0/650103309 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94007310 con 0x7f3ba4070920 2023-07-13T22:22:56.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.591+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba4164410 con 0x7f3ba4070920 2023-07-13T22:22:56.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.591+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba4159630 con 0x7f3ba4070920 2023-07-13T22:22:56.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.592+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba4164930 con 0x7f3ba4070920 2023-07-13T22:22:56.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.593+0000 7f3b9b7fe700 1 -- 172.21.15.60:0/650103309 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b94007470 con 0x7f3ba4070920 2023-07-13T22:22:56.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.594+0000 7f3b9b7fe700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b8406ace0 0x7f3b8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:56.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.594+0000 7f3b9b7fe700 1 -- 172.21.15.60:0/650103309 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3b94089200 con 0x7f3ba4070920 2023-07-13T22:22:56.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.594+0000 7f3b9bfff700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b8406ace0 0x7f3b8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:56.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.595+0000 7f3b9bfff700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b8406ace0 0x7f3b8406d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40b1b20 tx=0x7f3b8c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:56.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.598+0000 7f3b9b7fe700 1 -- 172.21.15.60:0/650103309 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3ba4164930 con 0x7f3ba4070920 2023-07-13T22:22:56.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.803+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ba4164930 con 0x7f3ba4070920 2023-07-13T22:22:56.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.803+0000 7f3b9b7fe700 1 -- 172.21.15.60:0/650103309 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b940541f0 con 0x7f3ba4070920 2023-07-13T22:22:56.787 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:22:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b8406ace0 msgr2=0x7f3b8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b8406ace0 0x7f3b8406d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40b1b20 tx=0x7f3b8c009210 comp rx=0 tx=0).stop 2023-07-13T22:22:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ba4070920 msgr2=0x7f3ba415c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ba4070920 0x7f3ba415c0c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3b940011e0 tx=0x7f3b9400c9e0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 shutdown_connections 2023-07-13T22:22:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b8406ace0 0x7f3b8406d190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ba4070920 0x7f3ba415c0c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ba4159260 0x7f3ba40ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 --2- 172.21.15.60:0/650103309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ba40ca900 0x7f3ba40ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.805+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 >> 172.21.15.60:0/650103309 conn(0x7f3ba40cbf50 msgr2=0x7f3ba406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:56.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.806+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 shutdown_connections 2023-07-13T22:22:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:56.806+0000 7f3ba95ff700 1 -- 172.21.15.60:0/650103309 wait complete. 2023-07-13T22:22:56.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:22:57.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.031+0000 7f8d3b5ff700 1 Processor -- start 2023-07-13T22:22:57.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.032+0000 7f8d3b5ff700 1 -- start start 2023-07-13T22:22:57.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.033+0000 7f8d3b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d3406ce70 0x7f8d340ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.033+0000 7f8d3b5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d340c8e90 con 0x7f8d340add00 2023-07-13T22:22:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.033+0000 7f8d3b5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d340caf30 con 0x7f8d3406a5e0 2023-07-13T22:22:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.033+0000 7f8d3b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340ae0d0 con 0x7f8d3406ce70 2023-07-13T22:22:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.033+0000 7f8d3939b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8d340add00 0x7f8d34069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60990/0 (socket says 172.21.15.60:60990) 2023-07-13T22:22:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.033+0000 7f8d3939b700 1 -- 172.21.15.60:0/2810228398 learned_addr learned my addr 172.21.15.60:0/2810228398 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.033+0000 7f8d39b9c700 1 --2- 172.21.15.60:0/2810228398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d3406ce70 0x7f8d340ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.033+0000 7f8d2bfff700 1 -- 172.21.15.60:0/2810228398 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427368460 0 0) 0x7f8d340c8e90 con 0x7f8d340add00 2023-07-13T22:22:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d2bfff700 1 -- 172.21.15.60:0/2810228398 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d200035b0 con 0x7f8d340add00 2023-07-13T22:22:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d2bfff700 1 -- 172.21.15.60:0/2810228398 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936027941 0 0) 0x7f8d340caf30 con 0x7f8d3406a5e0 2023-07-13T22:22:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d2bfff700 1 -- 172.21.15.60:0/2810228398 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d340c8e90 con 0x7f8d3406a5e0 2023-07-13T22:22:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d39b9c700 1 -- 172.21.15.60:0/2810228398 >> v1:172.21.15.105:6789/0 conn(0x7f8d3406a5e0 legacy=0x7f8d340b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d39b9c700 1 -- 172.21.15.60:0/2810228398 >> v1:172.21.15.60:6789/0 conn(0x7f8d340add00 legacy=0x7f8d34069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d39b9c700 1 -- 172.21.15.60:0/2810228398 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340c90e0 con 0x7f8d3406ce70 2023-07-13T22:22:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d39b9c700 1 --2- 172.21.15.60:0/2810228398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d3406ce70 0x7f8d340ca9f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009dc0 tx=0x7f8d2c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaf8761c104534a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d2bfff700 1 -- 172.21.15.60:0/2810228398 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00e070 con 0x7f8d3406ce70 2023-07-13T22:22:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d2bfff700 1 -- 172.21.15.60:0/2810228398 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c00bb20 con 0x7f8d3406ce70 2023-07-13T22:22:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.034+0000 7f8d2bfff700 1 -- 172.21.15.60:0/2810228398 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c012430 con 0x7f8d3406ce70 2023-07-13T22:22:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.035+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/2810228398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d3406ce70 msgr2=0x7f8d340ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.035+0000 7f8d3b5ff700 1 --2- 172.21.15.60:0/2810228398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d3406ce70 0x7f8d340ca9f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009dc0 tx=0x7f8d2c00b330 comp rx=0 tx=0).stop 2023-07-13T22:22:57.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.035+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/2810228398 shutdown_connections 2023-07-13T22:22:57.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.035+0000 7f8d3b5ff700 1 --2- 172.21.15.60:0/2810228398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d3406ce70 0x7f8d340ca9f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.035+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/2810228398 >> 172.21.15.60:0/2810228398 conn(0x7f8d340cbf50 msgr2=0x7f8d340ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:57.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.035+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/2810228398 shutdown_connections 2023-07-13T22:22:57.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.035+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/2810228398 wait complete. 2023-07-13T22:22:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.037+0000 7f8d3b5ff700 1 Processor -- start 2023-07-13T22:22:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.037+0000 7f8d3b5ff700 1 -- start start 2023-07-13T22:22:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d3b5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d3406a5e0 0x7f8d34158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d3b5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d3406ce70 0x7f8d340c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d3b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d340add00 0x7f8d340c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d3b5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d341f2590 con 0x7f8d3406ce70 2023-07-13T22:22:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d3b5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d341f2870 con 0x7f8d3406a5e0 2023-07-13T22:22:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d3b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d341f2b50 con 0x7f8d340add00 2023-07-13T22:22:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d38b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d3406ce70 0x7f8d340c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d38b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d3406ce70 0x7f8d340c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52922/0 (socket says 172.21.15.60:52922) 2023-07-13T22:22:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d38b9a700 1 -- 172.21.15.60:0/1719626590 learned_addr learned my addr 172.21.15.60:0/1719626590 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d3939b700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d3406a5e0 0x7f8d34158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.038+0000 7f8d39b9c700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d340add00 0x7f8d340c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d38b9a700 1 -- 172.21.15.60:0/1719626590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d340add00 msgr2=0x7f8d340c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d38b9a700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d340add00 0x7f8d340c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d38b9a700 1 -- 172.21.15.60:0/1719626590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d3406a5e0 msgr2=0x7f8d34158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d38b9a700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d3406a5e0 0x7f8d34158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d38b9a700 1 -- 172.21.15.60:0/1719626590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c009a70 con 0x7f8d3406ce70 2023-07-13T22:22:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d3939b700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d3406a5e0 0x7f8d34158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:57.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d38b9a700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d3406ce70 0x7f8d340c7600 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c500 tx=0x7f8d2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:57.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d29ffb700 1 -- 172.21.15.60:0/1719626590 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d240052d0 con 0x7f8d3406ce70 2023-07-13T22:22:57.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d29ffb700 1 -- 172.21.15.60:0/1719626590 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d24005450 con 0x7f8d3406ce70 2023-07-13T22:22:57.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d341f2e30 con 0x7f8d3406ce70 2023-07-13T22:22:57.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d29ffb700 1 -- 172.21.15.60:0/1719626590 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d240038c0 con 0x7f8d3406ce70 2023-07-13T22:22:57.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.039+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d341f21d0 con 0x7f8d3406ce70 2023-07-13T22:22:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.040+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3404e210 con 0x7f8d3406ce70 2023-07-13T22:22:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.041+0000 7f8d29ffb700 1 -- 172.21.15.60:0/1719626590 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d24003a20 con 0x7f8d3406ce70 2023-07-13T22:22:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.042+0000 7f8d29ffb700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d1806ace0 0x7f8d1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.042+0000 7f8d29ffb700 1 -- 172.21.15.60:0/1719626590 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8d240889a0 con 0x7f8d3406ce70 2023-07-13T22:22:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.043+0000 7f8d3939b700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d1806ace0 0x7f8d1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.043+0000 7f8d3939b700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d1806ace0 0x7f8d1806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8d300011e0 tx=0x7f8d30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:57.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.047+0000 7f8d29ffb700 1 -- 172.21.15.60:0/1719626590 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8d24053a20 con 0x7f8d3406ce70 2023-07-13T22:22:57.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.254+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d340c9770 con 0x7f8d3406ce70 2023-07-13T22:22:57.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.255+0000 7f8d29ffb700 1 -- 172.21.15.60:0/1719626590 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d24057040 con 0x7f8d3406ce70 2023-07-13T22:22:57.239 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:22:57.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d1806ace0 msgr2=0x7f8d1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d1806ace0 0x7f8d1806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8d300011e0 tx=0x7f8d30009040 comp rx=0 tx=0).stop 2023-07-13T22:22:57.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d3406ce70 msgr2=0x7f8d340c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d3406ce70 0x7f8d340c7600 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c500 tx=0x7f8d2400c8c0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 shutdown_connections 2023-07-13T22:22:57.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d1806ace0 0x7f8d1806d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d340add00 0x7f8d340c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d3406a5e0 0x7f8d34158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 --2- 172.21.15.60:0/1719626590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d3406ce70 0x7f8d340c7600 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 >> 172.21.15.60:0/1719626590 conn(0x7f8d340cbf50 msgr2=0x7f8d34159dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 shutdown_connections 2023-07-13T22:22:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.257+0000 7f8d3b5ff700 1 -- 172.21.15.60:0/1719626590 wait complete. 2023-07-13T22:22:57.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:22:57.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.482+0000 7f82debff700 1 Processor -- start 2023-07-13T22:22:57.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.482+0000 7f82debff700 1 -- start start 2023-07-13T22:22:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82debff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80b8200 0x7f82d80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82debff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82d80cede0 0x7f82d80cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82debff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d80d1870 con 0x7f82d80b1ac0 2023-07-13T22:22:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82debff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d80cf370 con 0x7f82d80cede0 2023-07-13T22:22:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82debff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80d12b0 con 0x7f82d80b8200 2023-07-13T22:22:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82dd19c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82d80cede0 0x7f82d80cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82dc99b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80b8200 0x7f82d80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82dd19c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82d80cede0 0x7f82d80cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52930/0 (socket says 172.21.15.60:52930) 2023-07-13T22:22:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82d7fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f82d80b1ac0 0x7f82d815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:58340/0 (socket says 172.21.15.60:58340) 2023-07-13T22:22:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.483+0000 7f82dd19c700 1 -- 172.21.15.60:0/1147793657 learned_addr learned my addr 172.21.15.60:0/1147793657 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82dd19c700 1 -- 172.21.15.60:0/1147793657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80b8200 msgr2=0x7f82d80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82dd19c700 1 --2- 172.21.15.60:0/1147793657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80b8200 0x7f82d80cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82dd19c700 1 -- 172.21.15.60:0/1147793657 >> v1:172.21.15.105:6789/0 conn(0x7f82d80b1ac0 legacy=0x7f82d815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82d77fe700 1 -- 172.21.15.60:0/1147793657 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970698509 0 0) 0x7f82d80d1870 con 0x7f82d80b1ac0 2023-07-13T22:22:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82dd19c700 1 -- 172.21.15.60:0/1147793657 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80d1590 con 0x7f82d80cede0 2023-07-13T22:22:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82dc99b700 1 --2- 172.21.15.60:0/1147793657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80b8200 0x7f82d80cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82dd19c700 1 --2- 172.21.15.60:0/1147793657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80cede0 0x7f82d80cfb80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009e50 tx=0x7f82d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfe4e48f993c388 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82d77fe700 1 -- 172.21.15.60:0/1147793657 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000e070 con 0x7f82d80cede0 2023-07-13T22:22:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82d77fe700 1 -- 172.21.15.60:0/1147793657 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d000b910 con 0x7f82d80cede0 2023-07-13T22:22:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.484+0000 7f82d77fe700 1 -- 172.21.15.60:0/1147793657 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000bcc0 con 0x7f82d80cede0 2023-07-13T22:22:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.485+0000 7f82debff700 1 -- 172.21.15.60:0/1147793657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80cede0 msgr2=0x7f82d80cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.485+0000 7f82debff700 1 --2- 172.21.15.60:0/1147793657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80cede0 0x7f82d80cfb80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009e50 tx=0x7f82d000b330 comp rx=0 tx=0).stop 2023-07-13T22:22:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.485+0000 7f82debff700 1 -- 172.21.15.60:0/1147793657 shutdown_connections 2023-07-13T22:22:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.485+0000 7f82debff700 1 --2- 172.21.15.60:0/1147793657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80cede0 0x7f82d80cfb80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.485+0000 7f82debff700 1 --2- 172.21.15.60:0/1147793657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80b8200 0x7f82d80cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.485+0000 7f82debff700 1 -- 172.21.15.60:0/1147793657 >> 172.21.15.60:0/1147793657 conn(0x7f82d806b2f0 msgr2=0x7f82d806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.485+0000 7f82debff700 1 -- 172.21.15.60:0/1147793657 shutdown_connections 2023-07-13T22:22:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.485+0000 7f82debff700 1 -- 172.21.15.60:0/1147793657 wait complete. 2023-07-13T22:22:57.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.487+0000 7f82debff700 1 Processor -- start 2023-07-13T22:22:57.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.487+0000 7f82debff700 1 -- start start 2023-07-13T22:22:57.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82debff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82d80b1ac0 0x7f82d81492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82debff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80b8200 0x7f82d8145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82debff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80cede0 0x7f82d80cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82debff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80ccce0 con 0x7f82d80b8200 2023-07-13T22:22:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82debff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8146970 con 0x7f82d80b1ac0 2023-07-13T22:22:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82debff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80cc440 con 0x7f82d80cede0 2023-07-13T22:22:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80b8200 0x7f82d8145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80b8200 0x7f82d8145fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52932/0 (socket says 172.21.15.60:52932) 2023-07-13T22:22:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82d7fff700 1 -- 172.21.15.60:0/2695725781 learned_addr learned my addr 172.21.15.60:0/2695725781 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.488+0000 7f82dc99b700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82d80b1ac0 0x7f82d81492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82dd19c700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80cede0 0x7f82d80cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82d7fff700 1 -- 172.21.15.60:0/2695725781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80cede0 msgr2=0x7f82d80cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82d7fff700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80cede0 0x7f82d80cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82d7fff700 1 -- 172.21.15.60:0/2695725781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82d80b1ac0 msgr2=0x7f82d81492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82d7fff700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82d80b1ac0 0x7f82d81492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82d7fff700 1 -- 172.21.15.60:0/2695725781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc0094a0 con 0x7f82d80b8200 2023-07-13T22:22:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82dd19c700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80cede0 0x7f82d80cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82d7fff700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80b8200 0x7f82d8145fe0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f82c40011e0 tx=0x7f82c400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82d57fa700 1 -- 172.21.15.60:0/2695725781 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4004c80 con 0x7f82d80b8200 2023-07-13T22:22:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82d57fa700 1 -- 172.21.15.60:0/2695725781 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4004de0 con 0x7f82d80b8200 2023-07-13T22:22:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d0009b30 con 0x7f82d80b8200 2023-07-13T22:22:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.489+0000 7f82d57fa700 1 -- 172.21.15.60:0/2695725781 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4007310 con 0x7f82d80b8200 2023-07-13T22:22:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.490+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d80cca00 con 0x7f82d80b8200 2023-07-13T22:22:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.490+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d8146740 con 0x7f82d80b8200 2023-07-13T22:22:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.492+0000 7f82d57fa700 1 -- 172.21.15.60:0/2695725781 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82c4007470 con 0x7f82d80b8200 2023-07-13T22:22:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.493+0000 7f82d57fa700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82bc06ace0 0x7f82bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.493+0000 7f82d57fa700 1 -- 172.21.15.60:0/2695725781 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f82c4089280 con 0x7f82d80b8200 2023-07-13T22:22:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.493+0000 7f82dc99b700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82bc06ace0 0x7f82bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.494+0000 7f82dc99b700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82bc06ace0 0x7f82bc06d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009d70 tx=0x7f82cc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:57.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.497+0000 7f82d57fa700 1 -- 172.21.15.60:0/2695725781 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f82d8146740 con 0x7f82d80b8200 2023-07-13T22:22:57.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.702+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82d8146740 con 0x7f82d80b8200 2023-07-13T22:22:57.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.703+0000 7f82d57fa700 1 -- 172.21.15.60:0/2695725781 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82d8146740 con 0x7f82d80b8200 2023-07-13T22:22:57.687 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:22:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.705+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82bc06ace0 msgr2=0x7f82bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.705+0000 7f82debff700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82bc06ace0 0x7f82bc06d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009d70 tx=0x7f82cc009210 comp rx=0 tx=0).stop 2023-07-13T22:22:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.705+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80b8200 msgr2=0x7f82d8145fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.705+0000 7f82debff700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80b8200 0x7f82d8145fe0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f82c40011e0 tx=0x7f82c400ca30 comp rx=0 tx=0).stop 2023-07-13T22:22:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.706+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 shutdown_connections 2023-07-13T22:22:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.706+0000 7f82debff700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f82bc06ace0 0x7f82bc06d190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.706+0000 7f82debff700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82d80cede0 0x7f82d80cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.706+0000 7f82debff700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82d80b1ac0 0x7f82d81492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.706+0000 7f82debff700 1 --2- 172.21.15.60:0/2695725781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82d80b8200 0x7f82d8145fe0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.706+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 >> 172.21.15.60:0/2695725781 conn(0x7f82d806b2f0 msgr2=0x7f82d815adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.706+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 shutdown_connections 2023-07-13T22:22:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.706+0000 7f82debff700 1 -- 172.21.15.60:0/2695725781 wait complete. 2023-07-13T22:22:57.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.925+0000 7fc16f1ff700 1 Processor -- start 2023-07-13T22:22:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.926+0000 7fc16f1ff700 1 -- start start 2023-07-13T22:22:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.927+0000 7fc16f1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1680b8400 0x7fc1680cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.927+0000 7fc16f1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1680d3750 0x7fc1680cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.927+0000 7fc16f1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1680d0ac0 con 0x7fc1680cfad0 2023-07-13T22:22:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.927+0000 7fc16f1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1680cff50 con 0x7fc1680b8400 2023-07-13T22:22:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.927+0000 7fc16f1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1680b87d0 con 0x7fc1680d3750 2023-07-13T22:22:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.927+0000 7fc16cf9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1680b8400 0x7fc1680cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.927+0000 7fc16cf9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1680b8400 0x7fc1680cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52934/0 (socket says 172.21.15.60:52934) 2023-07-13T22:22:57.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.927+0000 7fc16cf9b700 1 -- 172.21.15.60:0/3765851876 learned_addr learned my addr 172.21.15.60:0/3765851876 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:57.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc167fff700 1 --2- 172.21.15.60:0/3765851876 >> v2:172.21.15.105:3300/0 conn(0x7fc1680d3750 0x7fc1680cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc16cf9b700 1 -- 172.21.15.60:0/3765851876 >> v2:172.21.15.105:3300/0 conn(0x7fc1680d3750 msgr2=0x7fc1680cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:57.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc16cf9b700 1 --2- 172.21.15.60:0/3765851876 >> v2:172.21.15.105:3300/0 conn(0x7fc1680d3750 0x7fc1680cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc16cf9b700 1 -- 172.21.15.60:0/3765851876 >> v1:172.21.15.105:6789/0 conn(0x7fc1680cfad0 legacy=0x7fc16815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc16cf9b700 1 -- 172.21.15.60:0/3765851876 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1680b4eb0 con 0x7fc1680b8400 2023-07-13T22:22:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc167fff700 1 --2- 172.21.15.60:0/3765851876 >> v2:172.21.15.105:3300/0 conn(0x7fc1680d3750 0x7fc1680cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:22:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc16cf9b700 1 --2- 172.21.15.60:0/3765851876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680b8400 0x7fc1680cd240 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc1680b8cf0 tx=0x7fc15c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22120f2496f8b44c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc1677fe700 1 -- 172.21.15.60:0/3765851876 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c00b870 con 0x7fc1680b8400 2023-07-13T22:22:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc1677fe700 1 -- 172.21.15.60:0/3765851876 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c00b9d0 con 0x7fc1680b8400 2023-07-13T22:22:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.928+0000 7fc1677fe700 1 -- 172.21.15.60:0/3765851876 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c00bd60 con 0x7fc1680b8400 2023-07-13T22:22:57.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.929+0000 7fc16f1ff700 1 -- 172.21.15.60:0/3765851876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680b8400 msgr2=0x7fc1680cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.929+0000 7fc16f1ff700 1 --2- 172.21.15.60:0/3765851876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680b8400 0x7fc1680cd240 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc1680b8cf0 tx=0x7fc15c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.929+0000 7fc16f1ff700 1 -- 172.21.15.60:0/3765851876 shutdown_connections 2023-07-13T22:22:57.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.929+0000 7fc16f1ff700 1 --2- 172.21.15.60:0/3765851876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680b8400 0x7fc1680cd240 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.929+0000 7fc16f1ff700 1 --2- 172.21.15.60:0/3765851876 >> v2:172.21.15.105:3300/0 conn(0x7fc1680d3750 0x7fc1680cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.929+0000 7fc16f1ff700 1 -- 172.21.15.60:0/3765851876 >> 172.21.15.60:0/3765851876 conn(0x7fc16806b390 msgr2=0x7fc16806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:57.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.929+0000 7fc16f1ff700 1 -- 172.21.15.60:0/3765851876 shutdown_connections 2023-07-13T22:22:57.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.929+0000 7fc16f1ff700 1 -- 172.21.15.60:0/3765851876 wait complete. 2023-07-13T22:22:57.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.931+0000 7fc16f1ff700 1 Processor -- start 2023-07-13T22:22:57.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.931+0000 7fc16f1ff700 1 -- start start 2023-07-13T22:22:57.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc16f1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1680b8400 0x7fc1680c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc16f1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680cfad0 0x7fc1680b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc16f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1680d3750 0x7fc168153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc16f1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680c78a0 con 0x7fc1680cfad0 2023-07-13T22:22:57.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc16f1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680c7000 con 0x7fc1680b8400 2023-07-13T22:22:57.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc16f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680c72e0 con 0x7fc1680d3750 2023-07-13T22:22:57.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc167fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680cfad0 0x7fc1680b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc167fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680cfad0 0x7fc1680b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52946/0 (socket says 172.21.15.60:52946) 2023-07-13T22:22:57.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc167fff700 1 -- 172.21.15.60:0/126503420 learned_addr learned my addr 172.21.15.60:0/126503420 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:57.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc16cf9b700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1680b8400 0x7fc1680c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc16d79c700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1680d3750 0x7fc168153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.932+0000 7fc167fff700 1 -- 172.21.15.60:0/126503420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1680d3750 msgr2=0x7fc168153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc167fff700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1680d3750 0x7fc168153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc167fff700 1 -- 172.21.15.60:0/126503420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1680b8400 msgr2=0x7fc1680c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:57.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc167fff700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1680b8400 0x7fc1680c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:57.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc167fff700 1 -- 172.21.15.60:0/126503420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c009550 con 0x7fc1680cfad0 2023-07-13T22:22:57.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc167fff700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680cfad0 0x7fc1680b6c70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c7f0 tx=0x7fc15400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:57.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc1657fa700 1 -- 172.21.15.60:0/126503420 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154011800 con 0x7fc1680cfad0 2023-07-13T22:22:57.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc1657fa700 1 -- 172.21.15.60:0/126503420 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154011960 con 0x7fc1680cfad0 2023-07-13T22:22:57.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1680c75c0 con 0x7fc1680cfad0 2023-07-13T22:22:57.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc1657fa700 1 -- 172.21.15.60:0/126503420 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154011d10 con 0x7fc1680cfad0 2023-07-13T22:22:57.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.933+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1680c6bf0 con 0x7fc1680cfad0 2023-07-13T22:22:57.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.934+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16815b040 con 0x7fc1680cfad0 2023-07-13T22:22:57.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.935+0000 7fc1657fa700 1 -- 172.21.15.60:0/126503420 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc154018360 con 0x7fc1680cfad0 2023-07-13T22:22:57.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.937+0000 7fc1657fa700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc14c06ace0 0x7fc14c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:57.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.937+0000 7fc1657fa700 1 -- 172.21.15.60:0/126503420 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fc1540892e0 con 0x7fc1680cfad0 2023-07-13T22:22:57.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.937+0000 7fc16cf9b700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc14c06ace0 0x7fc14c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:57.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.938+0000 7fc16cf9b700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc14c06ace0 0x7fc14c06d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc15c014040 tx=0x7fc15c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:57.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:57.941+0000 7fc1657fa700 1 -- 172.21.15.60:0/126503420 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc154057df0 con 0x7fc1680cfad0 2023-07-13T22:22:58.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.194+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1680b5440 con 0x7fc1680cfad0 2023-07-13T22:22:58.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.195+0000 7fc1657fa700 1 -- 172.21.15.60:0/126503420 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fc154057980 con 0x7fc1680cfad0 2023-07-13T22:22:58.179 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:58.179 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2157014443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2157014443},{"type":"v1","addr":"172.21.15.105:6834","nonce":2157014443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/302067138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":302067138},{"type":"v1","addr":"172.21.15.105:6835","nonce":302067138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3461450784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461450784},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461450784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:22:19.175830+0000","modified":"2023-07-13T22:22:22.691356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6837/2048874349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2048874349},{"type":"v1","addr":"172.21.15.60:6837","nonce":2048874349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-13T22:22:58.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc14c06ace0 msgr2=0x7fc14c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc14c06ace0 0x7fc14c06d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc15c014040 tx=0x7fc15c008030 comp rx=0 tx=0).stop 2023-07-13T22:22:58.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680cfad0 msgr2=0x7fc1680b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680cfad0 0x7fc1680b6c70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c7f0 tx=0x7fc15400cb00 comp rx=0 tx=0).stop 2023-07-13T22:22:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 shutdown_connections 2023-07-13T22:22:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc14c06ace0 0x7fc14c06d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1680d3750 0x7fc168153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1680b8400 0x7fc1680c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 --2- 172.21.15.60:0/126503420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680cfad0 0x7fc1680b6c70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 >> 172.21.15.60:0/126503420 conn(0x7fc16806b390 msgr2=0x7fc16815bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 shutdown_connections 2023-07-13T22:22:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.197+0000 7fc16f1ff700 1 -- 172.21.15.60:0/126503420 wait complete. 2023-07-13T22:22:58.183 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2023-07-13T22:22:58.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:58.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.417+0000 7f64e5dff700 1 Processor -- start 2023-07-13T22:22:58.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.418+0000 7f64e5dff700 1 -- start start 2023-07-13T22:22:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.419+0000 7f64e5dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64e00d3750 0x7f64e00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.419+0000 7f64e5dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00cfad0 0x7f64e00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64e5dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00cd950 con 0x7f64e00b8400 2023-07-13T22:22:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64e5dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e00cff50 con 0x7f64e00d3750 2023-07-13T22:22:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64e5dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00d0ac0 con 0x7f64e00cfad0 2023-07-13T22:22:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64d7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64e00d3750 0x7f64e00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64d7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64e00d3750 0x7f64e00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52958/0 (socket says 172.21.15.60:52958) 2023-07-13T22:22:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64d7fff700 1 -- 172.21.15.60:0/2877192138 learned_addr learned my addr 172.21.15.60:0/2877192138 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64dffff700 1 --2- 172.21.15.60:0/2877192138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00cfad0 0x7f64e00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64d7fff700 1 -- 172.21.15.60:0/2877192138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00cfad0 msgr2=0x7f64e00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64d7fff700 1 --2- 172.21.15.60:0/2877192138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00cfad0 0x7f64e00d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64d7fff700 1 -- 172.21.15.60:0/2877192138 >> v1:172.21.15.105:6789/0 conn(0x7f64e00b8400 legacy=0x7f64e00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.420+0000 7f64d7fff700 1 -- 172.21.15.60:0/2877192138 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00b87d0 con 0x7f64e00d3750 2023-07-13T22:22:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64d7fff700 1 --2- 172.21.15.60:0/2877192138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00d3750 0x7f64e00cf590 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098f0 tx=0x7f64c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401f04d56871acf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64deffd700 1 -- 172.21.15.60:0/2877192138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8011830 con 0x7f64e00d3750 2023-07-13T22:22:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64deffd700 1 -- 172.21.15.60:0/2877192138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8011990 con 0x7f64e00d3750 2023-07-13T22:22:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64deffd700 1 -- 172.21.15.60:0/2877192138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8011d20 con 0x7f64e00d3750 2023-07-13T22:22:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64e5dff700 1 -- 172.21.15.60:0/2877192138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00d3750 msgr2=0x7f64e00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64e5dff700 1 --2- 172.21.15.60:0/2877192138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00d3750 0x7f64e00cf590 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098f0 tx=0x7f64c8009c00 comp rx=0 tx=0).stop 2023-07-13T22:22:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64e5dff700 1 -- 172.21.15.60:0/2877192138 shutdown_connections 2023-07-13T22:22:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64e5dff700 1 --2- 172.21.15.60:0/2877192138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00cfad0 0x7f64e00d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64e5dff700 1 --2- 172.21.15.60:0/2877192138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00d3750 0x7f64e00cf590 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64e5dff700 1 -- 172.21.15.60:0/2877192138 >> 172.21.15.60:0/2877192138 conn(0x7f64e006b390 msgr2=0x7f64e006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.421+0000 7f64e5dff700 1 -- 172.21.15.60:0/2877192138 shutdown_connections 2023-07-13T22:22:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.422+0000 7f64e5dff700 1 -- 172.21.15.60:0/2877192138 wait complete. 2023-07-13T22:22:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.423+0000 7f64e5dff700 1 Processor -- start 2023-07-13T22:22:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.423+0000 7f64e5dff700 1 -- start start 2023-07-13T22:22:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.424+0000 7f64e5dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00b8400 0x7f64e0150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.424+0000 7f64e5dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00cfad0 0x7f64e01509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.424+0000 7f64e5dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d3750 0x7f64e0150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.424+0000 7f64e5dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00cc690 con 0x7f64e00cfad0 2023-07-13T22:22:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.424+0000 7f64e5dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00cc970 con 0x7f64e00b8400 2023-07-13T22:22:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.424+0000 7f64e5dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00ca700 con 0x7f64e00d3750 2023-07-13T22:22:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.424+0000 7f64d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00cfad0 0x7f64e01509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00cfad0 0x7f64e01509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52972/0 (socket says 172.21.15.60:52972) 2023-07-13T22:22:58.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64d7fff700 1 -- 172.21.15.60:0/3726321781 learned_addr learned my addr 172.21.15.60:0/3726321781 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:58.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64dffff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d3750 0x7f64e0150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64df7fe700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00b8400 0x7f64e0150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64d7fff700 1 -- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d3750 msgr2=0x7f64e0150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64d7fff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d3750 0x7f64e0150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64d7fff700 1 -- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00b8400 msgr2=0x7f64e0150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64d7fff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00b8400 0x7f64e0150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64d7fff700 1 -- 172.21.15.60:0/3726321781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009550 con 0x7f64e00cfad0 2023-07-13T22:22:58.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64dffff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d3750 0x7f64e0150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:58.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64df7fe700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00b8400 0x7f64e0150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:58.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.425+0000 7f64d7fff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00cfad0 0x7f64e01509a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64c8000c00 tx=0x7f64c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:58.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.426+0000 7f64dcff9700 1 -- 172.21.15.60:0/3726321781 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8004570 con 0x7f64e00cfad0 2023-07-13T22:22:58.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.426+0000 7f64dcff9700 1 -- 172.21.15.60:0/3726321781 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c801f070 con 0x7f64e00cfad0 2023-07-13T22:22:58.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.426+0000 7f64dcff9700 1 -- 172.21.15.60:0/3726321781 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80038c0 con 0x7f64e00cfad0 2023-07-13T22:22:58.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.426+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e00ca9e0 con 0x7f64e00cfad0 2023-07-13T22:22:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.426+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e00cc2d0 con 0x7f64e00cfad0 2023-07-13T22:22:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.427+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e00cc460 con 0x7f64e00cfad0 2023-07-13T22:22:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.428+0000 7f64dcff9700 1 -- 172.21.15.60:0/3726321781 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f64c8021030 con 0x7f64e00cfad0 2023-07-13T22:22:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.429+0000 7f64dcff9700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c006ace0 0x7f64c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.429+0000 7f64dcff9700 1 -- 172.21.15.60:0/3726321781 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f64c808a9e0 con 0x7f64e00cfad0 2023-07-13T22:22:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.429+0000 7f64df7fe700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c006ace0 0x7f64c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.430+0000 7f64df7fe700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c006ace0 0x7f64c006d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f64d0000f60 tx=0x7f64d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.433+0000 7f64dcff9700 1 -- 172.21.15.60:0/3726321781 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f64c8055a60 con 0x7f64e00cfad0 2023-07-13T22:22:58.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.645+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64e00cc7d0 con 0x7f64e00cfad0 2023-07-13T22:22:58.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.647+0000 7f64dcff9700 1 -- 172.21.15.60:0/3726321781 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18250 (secure 0 0 0) 0x7f64e00cc7d0 con 0x7f64e00cfad0 2023-07-13T22:22:58.631 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:22:43.507730+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:22:17.071217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-13T22:22:18.161600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:22:58.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c006ace0 msgr2=0x7f64c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c006ace0 0x7f64c006d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f64d0000f60 tx=0x7f64d0009040 comp rx=0 tx=0).stop 2023-07-13T22:22:58.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00cfad0 msgr2=0x7f64e01509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00cfad0 0x7f64e01509a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64c8000c00 tx=0x7f64c8004750 comp rx=0 tx=0).stop 2023-07-13T22:22:58.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 shutdown_connections 2023-07-13T22:22:58.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c006ace0 0x7f64c006d190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00d3750 0x7f64e0150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e00b8400 0x7f64e0150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 --2- 172.21.15.60:0/3726321781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00cfad0 0x7f64e01509a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 >> 172.21.15.60:0/3726321781 conn(0x7f64e006b390 msgr2=0x7f64e00b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:58.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 shutdown_connections 2023-07-13T22:22:58.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.649+0000 7f64e5dff700 1 -- 172.21.15.60:0/3726321781 wait complete. 2023-07-13T22:22:58.652 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:22:58.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:58.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.872+0000 7f14469ff700 1 Processor -- start 2023-07-13T22:22:58.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.872+0000 7f14469ff700 1 -- start start 2023-07-13T22:22:58.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f14469ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14400d3750 0x7f14400cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f14469ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14400cd950 con 0x7f14400b8400 2023-07-13T22:22:58.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f14469ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14400d0ac0 con 0x7f14400cfad0 2023-07-13T22:22:58.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f14469ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14400cff50 con 0x7f14400d3750 2023-07-13T22:22:58.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f143f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14400d3750 0x7f14400cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f143ffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f14400b8400 0x7f14400cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60996/0 (socket says 172.21.15.60:60996) 2023-07-13T22:22:58.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f143ffff700 1 -- 172.21.15.60:0/299156118 learned_addr learned my addr 172.21.15.60:0/299156118 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:58.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f143f7fe700 1 -- 172.21.15.60:0/299156118 >> v1:172.21.15.105:6789/0 conn(0x7f14400cfad0 legacy=0x7f14400d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f143effd700 1 -- 172.21.15.60:0/299156118 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482316530 0 0) 0x7f1430001110 con 0x7f14400b8400 2023-07-13T22:22:58.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f143f7fe700 1 -- 172.21.15.60:0/299156118 >> v1:172.21.15.60:6789/0 conn(0x7f14400b8400 legacy=0x7f14400cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.874+0000 7f143f7fe700 1 -- 172.21.15.60:0/299156118 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400b87d0 con 0x7f14400d3750 2023-07-13T22:22:58.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.875+0000 7f143f7fe700 1 --2- 172.21.15.60:0/299156118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400d3750 0x7f14400cf590 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1434009840 tx=0x7f1434009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b3f0d93242fd89 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:58.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.875+0000 7f143effd700 1 -- 172.21.15.60:0/299156118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434011830 con 0x7f14400d3750 2023-07-13T22:22:58.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.875+0000 7f143effd700 1 -- 172.21.15.60:0/299156118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1434011990 con 0x7f14400d3750 2023-07-13T22:22:58.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.875+0000 7f143effd700 1 -- 172.21.15.60:0/299156118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434011d20 con 0x7f14400d3750 2023-07-13T22:22:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.875+0000 7f14469ff700 1 -- 172.21.15.60:0/299156118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400d3750 msgr2=0x7f14400cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.875+0000 7f14469ff700 1 --2- 172.21.15.60:0/299156118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400d3750 0x7f14400cf590 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1434009840 tx=0x7f1434009b50 comp rx=0 tx=0).stop 2023-07-13T22:22:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.876+0000 7f14469ff700 1 -- 172.21.15.60:0/299156118 shutdown_connections 2023-07-13T22:22:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.876+0000 7f14469ff700 1 --2- 172.21.15.60:0/299156118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400d3750 0x7f14400cf590 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.876+0000 7f14469ff700 1 -- 172.21.15.60:0/299156118 >> 172.21.15.60:0/299156118 conn(0x7f144006b390 msgr2=0x7f144006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:58.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.876+0000 7f14469ff700 1 -- 172.21.15.60:0/299156118 shutdown_connections 2023-07-13T22:22:58.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.876+0000 7f14469ff700 1 -- 172.21.15.60:0/299156118 wait complete. 2023-07-13T22:22:58.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.877+0000 7f14469ff700 1 Processor -- start 2023-07-13T22:22:58.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.877+0000 7f14469ff700 1 -- start start 2023-07-13T22:22:58.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.878+0000 7f14469ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14400b8400 0x7f1440150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.878+0000 7f14469ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400cfad0 0x7f144015c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.878+0000 7f14469ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14400d3750 0x7f14401529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.878+0000 7f14469ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400cbb50 con 0x7f14400cfad0 2023-07-13T22:22:58.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.878+0000 7f14469ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440150f40 con 0x7f14400d3750 2023-07-13T22:22:58.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.878+0000 7f14469ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14400cb260 con 0x7f14400b8400 2023-07-13T22:22:58.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f143f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400cfad0 0x7f144015c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f143f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400cfad0 0x7f144015c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52998/0 (socket says 172.21.15.60:52998) 2023-07-13T22:22:58.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f143f7fe700 1 -- 172.21.15.60:0/322076958 learned_addr learned my addr 172.21.15.60:0/322076958 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:58.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f143ffff700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14400b8400 0x7f1440150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f1444f9c700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14400d3750 0x7f14401529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f143f7fe700 1 -- 172.21.15.60:0/322076958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14400b8400 msgr2=0x7f1440150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f143f7fe700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14400b8400 0x7f1440150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f143f7fe700 1 -- 172.21.15.60:0/322076958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14400d3750 msgr2=0x7f14401529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:58.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f143f7fe700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14400d3750 0x7f14401529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:58.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.879+0000 7f143f7fe700 1 -- 172.21.15.60:0/322076958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340094a0 con 0x7f14400cfad0 2023-07-13T22:22:58.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.880+0000 7f143f7fe700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400cfad0 0x7f144015c970 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1434014040 tx=0x7f1434004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:58.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.880+0000 7f143cff9700 1 -- 172.21.15.60:0/322076958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143401a7e0 con 0x7f14400cfad0 2023-07-13T22:22:58.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.880+0000 7f143cff9700 1 -- 172.21.15.60:0/322076958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143401a940 con 0x7f14400cfad0 2023-07-13T22:22:58.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.880+0000 7f143cff9700 1 -- 172.21.15.60:0/322076958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143401acb0 con 0x7f14400cfad0 2023-07-13T22:22:58.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.880+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14400cb500 con 0x7f14400cfad0 2023-07-13T22:22:58.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.880+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1440151350 con 0x7f14400cfad0 2023-07-13T22:22:58.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.881+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1440154a10 con 0x7f14400cfad0 2023-07-13T22:22:58.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.882+0000 7f143cff9700 1 -- 172.21.15.60:0/322076958 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1434022070 con 0x7f14400cfad0 2023-07-13T22:22:58.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.883+0000 7f143cff9700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f142c06ace0 0x7f142c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:58.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.883+0000 7f143cff9700 1 -- 172.21.15.60:0/322076958 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f143408a1f0 con 0x7f14400cfad0 2023-07-13T22:22:58.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.883+0000 7f143ffff700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f142c06ace0 0x7f142c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:58.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.884+0000 7f143ffff700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f142c06ace0 0x7f142c06d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1430000f60 tx=0x7f1430004a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:58.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:58.888+0000 7f143cff9700 1 -- 172.21.15.60:0/322076958 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1434055440 con 0x7f14400cfad0 2023-07-13T22:22:59.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.137+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14400cbc90 con 0x7f14400cfad0 2023-07-13T22:22:59.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.137+0000 7f143cff9700 1 -- 172.21.15.60:0/322076958 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f1434006cc0 con 0x7f14400cfad0 2023-07-13T22:22:59.121 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:59.121 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2157014443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2157014443},{"type":"v1","addr":"172.21.15.105:6834","nonce":2157014443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/302067138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":302067138},{"type":"v1","addr":"172.21.15.105:6835","nonce":302067138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3461450784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461450784},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461450784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:22:19.175830+0000","modified":"2023-07-13T22:22:22.691356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6837/2048874349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2048874349},{"type":"v1","addr":"172.21.15.60:6837","nonce":2048874349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-13T22:22:59.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f142c06ace0 msgr2=0x7f142c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f142c06ace0 0x7f142c06d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1430000f60 tx=0x7f1430004a50 comp rx=0 tx=0).stop 2023-07-13T22:22:59.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400cfad0 msgr2=0x7f144015c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400cfad0 0x7f144015c970 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1434014040 tx=0x7f1434004750 comp rx=0 tx=0).stop 2023-07-13T22:22:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 shutdown_connections 2023-07-13T22:22:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f142c06ace0 0x7f142c06d190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14400d3750 0x7f14401529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14400b8400 0x7f1440150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 --2- 172.21.15.60:0/322076958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14400cfad0 0x7f144015c970 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.139+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 >> 172.21.15.60:0/322076958 conn(0x7f144006b390 msgr2=0x7f14400d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:59.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.140+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 shutdown_connections 2023-07-13T22:22:59.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.140+0000 7f14469ff700 1 -- 172.21.15.60:0/322076958 wait complete. 2023-07-13T22:22:59.125 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2023-07-13T22:22:59.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:59.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.361+0000 7f3ea05ff700 1 Processor -- start 2023-07-13T22:22:59.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.361+0000 7f3ea05ff700 1 -- start start 2023-07-13T22:22:59.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.362+0000 7f3ea05ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e980d26d0 0x7f3e980cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.362+0000 7f3ea05ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3e980b5a00 0x7f3e9815fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.362+0000 7f3ea05ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e980d14a0 con 0x7f3e980b2010 2023-07-13T22:22:59.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.362+0000 7f3ea05ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e980d2570 con 0x7f3e980b5a00 2023-07-13T22:22:59.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.362+0000 7f3ea05ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e980d1780 con 0x7f3e980d26d0 2023-07-13T22:22:59.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9e39b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3e980b2010 0x7f3e980cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:32776/0 (socket says 172.21.15.60:32776) 2023-07-13T22:22:59.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9e39b700 1 -- 172.21.15.60:0/1021384530 learned_addr learned my addr 172.21.15.60:0/1021384530 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:59.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9eb9c700 1 --2- 172.21.15.60:0/1021384530 >> v2:172.21.15.60:3300/0 conn(0x7f3e980b5a00 0x7f3e9815fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9db9a700 1 --2- 172.21.15.60:0/1021384530 >> v2:172.21.15.105:3300/0 conn(0x7f3e980d26d0 0x7f3e980cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9eb9c700 1 -- 172.21.15.60:0/1021384530 >> v2:172.21.15.105:3300/0 conn(0x7f3e980d26d0 msgr2=0x7f3e980cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9eb9c700 1 --2- 172.21.15.60:0/1021384530 >> v2:172.21.15.105:3300/0 conn(0x7f3e980d26d0 0x7f3e980cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9d399700 1 -- 172.21.15.60:0/1021384530 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471533393 0 0) 0x7f3e980d14a0 con 0x7f3e980b2010 2023-07-13T22:22:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9eb9c700 1 -- 172.21.15.60:0/1021384530 >> v1:172.21.15.60:6789/0 conn(0x7f3e980b2010 legacy=0x7f3e980cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9eb9c700 1 -- 172.21.15.60:0/1021384530 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e980cd2a0 con 0x7f3e980b5a00 2023-07-13T22:22:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.363+0000 7f3e9eb9c700 1 --2- 172.21.15.60:0/1021384530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980b5a00 0x7f3e9815fee0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3e980b57a0 tx=0x7f3e9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfeb259b8e3b2945 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3e9d399700 1 -- 172.21.15.60:0/1021384530 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000e070 con 0x7f3e980b5a00 2023-07-13T22:22:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3e9d399700 1 -- 172.21.15.60:0/1021384530 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000bb20 con 0x7f3e980b5a00 2023-07-13T22:22:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3e9d399700 1 -- 172.21.15.60:0/1021384530 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90012430 con 0x7f3e980b5a00 2023-07-13T22:22:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3ea05ff700 1 -- 172.21.15.60:0/1021384530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980b5a00 msgr2=0x7f3e9815fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3ea05ff700 1 --2- 172.21.15.60:0/1021384530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980b5a00 0x7f3e9815fee0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3e980b57a0 tx=0x7f3e9000b330 comp rx=0 tx=0).stop 2023-07-13T22:22:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3ea05ff700 1 -- 172.21.15.60:0/1021384530 shutdown_connections 2023-07-13T22:22:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3ea05ff700 1 --2- 172.21.15.60:0/1021384530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980b5a00 0x7f3e9815fee0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3ea05ff700 1 --2- 172.21.15.60:0/1021384530 >> v2:172.21.15.105:3300/0 conn(0x7f3e980d26d0 0x7f3e980cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3ea05ff700 1 -- 172.21.15.60:0/1021384530 >> 172.21.15.60:0/1021384530 conn(0x7f3e9806b380 msgr2=0x7f3e9806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3ea05ff700 1 -- 172.21.15.60:0/1021384530 shutdown_connections 2023-07-13T22:22:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.364+0000 7f3ea05ff700 1 -- 172.21.15.60:0/1021384530 wait complete. 2023-07-13T22:22:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.367+0000 7f3ea05ff700 1 Processor -- start 2023-07-13T22:22:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.367+0000 7f3ea05ff700 1 -- start start 2023-07-13T22:22:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.368+0000 7f3ea05ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e980b2010 0x7f3e980aa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.368+0000 7f3ea05ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e980b5a00 0x7f3e980abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.368+0000 7f3ea05ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980d26d0 0x7f3e981f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.368+0000 7f3ea05ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980cb6f0 con 0x7f3e980d26d0 2023-07-13T22:22:59.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.368+0000 7f3ea05ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980aac80 con 0x7f3e980b5a00 2023-07-13T22:22:59.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.368+0000 7f3ea05ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980cae00 con 0x7f3e980b2010 2023-07-13T22:22:59.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9eb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980d26d0 0x7f3e981f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9eb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980d26d0 0x7f3e981f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53020/0 (socket says 172.21.15.60:53020) 2023-07-13T22:22:59.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9eb9c700 1 -- 172.21.15.60:0/638228002 learned_addr learned my addr 172.21.15.60:0/638228002 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:59.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9db9a700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e980b5a00 0x7f3e980abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9e39b700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e980b2010 0x7f3e980aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9eb9c700 1 -- 172.21.15.60:0/638228002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e980b2010 msgr2=0x7f3e980aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9eb9c700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e980b2010 0x7f3e980aa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9eb9c700 1 -- 172.21.15.60:0/638228002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e980b5a00 msgr2=0x7f3e980abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9eb9c700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e980b5a00 0x7f3e980abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9eb9c700 1 -- 172.21.15.60:0/638228002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e90009a70 con 0x7f3e980d26d0 2023-07-13T22:22:59.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.369+0000 7f3e9eb9c700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980d26d0 0x7f3e981f45e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3e9806bfd0 tx=0x7f3e90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:59.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.370+0000 7f3e8effd700 1 -- 172.21.15.60:0/638228002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90012430 con 0x7f3e980d26d0 2023-07-13T22:22:59.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.370+0000 7f3e8effd700 1 -- 172.21.15.60:0/638228002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e90020070 con 0x7f3e980d26d0 2023-07-13T22:22:59.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.370+0000 7f3e8effd700 1 -- 172.21.15.60:0/638228002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90004280 con 0x7f3e980d26d0 2023-07-13T22:22:59.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.370+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e980cb0a0 con 0x7f3e980d26d0 2023-07-13T22:22:59.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.370+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e980aa940 con 0x7f3e980d26d0 2023-07-13T22:22:59.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.371+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e980cb8a0 con 0x7f3e980d26d0 2023-07-13T22:22:59.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.372+0000 7f3e8effd700 1 -- 172.21.15.60:0/638228002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e900043e0 con 0x7f3e980d26d0 2023-07-13T22:22:59.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.373+0000 7f3e8effd700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e7c06ace0 0x7f3e7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.373+0000 7f3e8effd700 1 -- 172.21.15.60:0/638228002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3e90022070 con 0x7f3e980d26d0 2023-07-13T22:22:59.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.373+0000 7f3e9e39b700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e7c06ace0 0x7f3e7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.374+0000 7f3e9e39b700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e7c06ace0 0x7f3e7c06d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3e94000f60 tx=0x7f3e94007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:59.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.377+0000 7f3e8effd700 1 -- 172.21.15.60:0/638228002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3e90056160 con 0x7f3e980d26d0 2023-07-13T22:22:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.628+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e9804e280 con 0x7f3e980d26d0 2023-07-13T22:22:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.629+0000 7f3e8effd700 1 -- 172.21.15.60:0/638228002 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f3e9002c090 con 0x7f3e980d26d0 2023-07-13T22:22:59.613 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:22:59.613 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2157014443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2157014443},{"type":"v1","addr":"172.21.15.105:6834","nonce":2157014443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/302067138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":302067138},{"type":"v1","addr":"172.21.15.105:6835","nonce":302067138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3461450784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461450784},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461450784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:22:19.175830+0000","modified":"2023-07-13T22:22:22.691356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5322},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5322":{"gid":5322,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6837/2048874349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2048874349},{"type":"v1","addr":"172.21.15.60:6837","nonce":2048874349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-13T22:22:59.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e7c06ace0 msgr2=0x7f3e7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e7c06ace0 0x7f3e7c06d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3e94000f60 tx=0x7f3e94007e70 comp rx=0 tx=0).stop 2023-07-13T22:22:59.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980d26d0 msgr2=0x7f3e981f45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980d26d0 0x7f3e981f45e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3e9806bfd0 tx=0x7f3e90004ad0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 shutdown_connections 2023-07-13T22:22:59.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e7c06ace0 0x7f3e7c06d190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e980d26d0 0x7f3e981f45e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e980b2010 0x7f3e980aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 --2- 172.21.15.60:0/638228002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e980b5a00 0x7f3e980abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 >> 172.21.15.60:0/638228002 conn(0x7f3e9806b380 msgr2=0x7f3e980b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:59.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.631+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 shutdown_connections 2023-07-13T22:22:59.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.632+0000 7f3ea05ff700 1 -- 172.21.15.60:0/638228002 wait complete. 2023-07-13T22:22:59.617 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2023-07-13T22:22:59.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:22:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.854+0000 7f62979ff700 1 Processor -- start 2023-07-13T22:22:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.855+0000 7f62979ff700 1 -- start start 2023-07-13T22:22:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.856+0000 7f62979ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f629015a820 0x7f629015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.856+0000 7f62979ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6290070270 con 0x7f62900c8d40 2023-07-13T22:22:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.856+0000 7f62979ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629015b2e0 con 0x7f6290154950 2023-07-13T22:22:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.856+0000 7f62979ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900713d0 con 0x7f629015a820 2023-07-13T22:22:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.856+0000 7f6295f9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f62900c8d40 0x7f62900c9160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:32780/0 (socket says 172.21.15.60:32780) 2023-07-13T22:22:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.856+0000 7f6295f9c700 1 -- 172.21.15.60:0/1074896727 learned_addr learned my addr 172.21.15.60:0/1074896727 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:22:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.856+0000 7f6294f9a700 1 --2- 172.21.15.60:0/1074896727 >> v2:172.21.15.105:3300/0 conn(0x7f629015a820 0x7f629015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128043523 0 0) 0x7f6290070270 con 0x7f62900c8d40 2023-07-13T22:22:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f627c0035b0 con 0x7f62900c8d40 2023-07-13T22:22:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425415818 0 0) 0x7f629015b2e0 con 0x7f6290154950 2023-07-13T22:22:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6290070270 con 0x7f6290154950 2023-07-13T22:22:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 848335188 0 0) 0x7f627c0035b0 con 0x7f62900c8d40 2023-07-13T22:22:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 >> v2:172.21.15.105:3300/0 conn(0x7f629015a820 msgr2=0x7f629015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 --2- 172.21.15.60:0/1074896727 >> v2:172.21.15.105:3300/0 conn(0x7f629015a820 0x7f629015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 >> v1:172.21.15.105:6789/0 conn(0x7f6290154950 legacy=0x7f6290070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629015b5c0 con 0x7f62900c8d40 2023-07-13T22:22:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f62880019f0 con 0x7f62900c8d40 2023-07-13T22:22:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 >> v1:172.21.15.60:6789/0 conn(0x7f62900c8d40 legacy=0x7f62900c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f627c003be0 0x7f627c003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f627c004850 0x7f627c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c005250 0x7f627c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c00c130 con 0x7f627c004850 2023-07-13T22:22:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c00c2a0 con 0x7f627c003be0 2023-07-13T22:22:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c00c460 con 0x7f627c005250 2023-07-13T22:22:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f6294f9a700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f627c004850 0x7f627c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.857+0000 7f629579b700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f627c003be0 0x7f627c003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6295f9c700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c005250 0x7f627c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6294f9a700 1 -- 172.21.15.60:0/1074896727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c005250 msgr2=0x7f627c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6294f9a700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c005250 0x7f627c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6294f9a700 1 -- 172.21.15.60:0/1074896727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f627c003be0 msgr2=0x7f627c003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6294f9a700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f627c003be0 0x7f627c003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6294f9a700 1 -- 172.21.15.60:0/1074896727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c0035b0 con 0x7f627c004850 2023-07-13T22:22:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f629579b700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f627c003be0 0x7f627c003fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6295f9c700 1 -- 172.21.15.60:0/1074896727 reap_dead start 2023-07-13T22:22:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6294f9a700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f627c004850 0x7f627c004cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6280009b30 tx=0x7f627c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3b300b1eb284db server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000e6c0 con 0x7f627c004850 2023-07-13T22:22:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628000e820 con 0x7f627c004850 2023-07-13T22:22:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.858+0000 7f6287fff700 1 -- 172.21.15.60:0/1074896727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000ebb0 con 0x7f627c004850 2023-07-13T22:22:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.859+0000 7f62979ff700 1 -- 172.21.15.60:0/1074896727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f627c004850 msgr2=0x7f627c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.859+0000 7f62979ff700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f627c004850 0x7f627c004cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6280009b30 tx=0x7f627c005880 comp rx=0 tx=0).stop 2023-07-13T22:22:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.859+0000 7f62979ff700 1 -- 172.21.15.60:0/1074896727 shutdown_connections 2023-07-13T22:22:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.859+0000 7f62979ff700 1 --2- 172.21.15.60:0/1074896727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f627c004850 0x7f627c004cc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.859+0000 7f62979ff700 1 -- 172.21.15.60:0/1074896727 >> 172.21.15.60:0/1074896727 conn(0x7f62900cf410 msgr2=0x7f62900cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.859+0000 7f62979ff700 1 -- 172.21.15.60:0/1074896727 shutdown_connections 2023-07-13T22:22:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.859+0000 7f62979ff700 1 -- 172.21.15.60:0/1074896727 wait complete. 2023-07-13T22:22:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.862+0000 7f62979ff700 1 Processor -- start 2023-07-13T22:22:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.862+0000 7f62979ff700 1 -- start start 2023-07-13T22:22:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.863+0000 7f62979ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c004850 0x7f62900cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.863+0000 7f62979ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62900cdc00 0x7f6290159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.863+0000 7f62979ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6290159670 0x7f62900cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.863+0000 7f62979ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6290150000 con 0x7f6290159670 2023-07-13T22:22:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.863+0000 7f62979ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6290159af0 con 0x7f62900cdc00 2023-07-13T22:22:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.863+0000 7f62979ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62901f5290 con 0x7f627c004850 2023-07-13T22:22:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.863+0000 7f6295f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6290159670 0x7f62900cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.863+0000 7f6295f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6290159670 0x7f62900cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53038/0 (socket says 172.21.15.60:53038) 2023-07-13T22:22:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.863+0000 7f6295f9c700 1 -- 172.21.15.60:0/2523655034 learned_addr learned my addr 172.21.15.60:0/2523655034 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:22:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f629579b700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c004850 0x7f62900cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6294f9a700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62900cdc00 0x7f6290159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6295f9c700 1 -- 172.21.15.60:0/2523655034 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c004850 msgr2=0x7f62900cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6295f9c700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c004850 0x7f62900cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6295f9c700 1 -- 172.21.15.60:0/2523655034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62900cdc00 msgr2=0x7f6290159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6295f9c700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62900cdc00 0x7f6290159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6295f9c700 1 -- 172.21.15.60:0/2523655034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900ce180 con 0x7f6290159670 2023-07-13T22:22:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f629579b700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c004850 0x7f62900cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:22:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6295f9c700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6290159670 0x7f62900cc2c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f62900ca7c0 tx=0x7f6288007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6285ffb700 1 -- 172.21.15.60:0/2523655034 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800eb40 con 0x7f6290159670 2023-07-13T22:22:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6285ffb700 1 -- 172.21.15.60:0/2523655034 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628800eca0 con 0x7f6290159670 2023-07-13T22:22:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f6285ffb700 1 -- 172.21.15.60:0/2523655034 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288013420 con 0x7f6290159670 2023-07-13T22:22:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.864+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62901f54e0 con 0x7f6290159670 2023-07-13T22:22:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.865+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62901f5aa0 con 0x7f6290159670 2023-07-13T22:22:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.866+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62900cbc20 con 0x7f6290159670 2023-07-13T22:22:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.866+0000 7f6285ffb700 1 -- 172.21.15.60:0/2523655034 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6288013580 con 0x7f6290159670 2023-07-13T22:22:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.868+0000 7f6285ffb700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627406ace0 0x7f627406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.868+0000 7f6285ffb700 1 -- 172.21.15.60:0/2523655034 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f62880897e0 con 0x7f6290159670 2023-07-13T22:22:59.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.868+0000 7f629579b700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627406ace0 0x7f627406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:59.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.869+0000 7f629579b700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627406ace0 0x7f627406d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f629015a590 tx=0x7f628c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:22:59.872+0000 7f6285ffb700 1 -- 172.21.15.60:0/2523655034 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6288054890 con 0x7f6290159670 2023-07-13T22:23:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.080+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f629015a5e0 con 0x7f6290159670 2023-07-13T22:23:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.081+0000 7f6285ffb700 1 -- 172.21.15.60:0/2523655034 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18250 (secure 0 0 0) 0x7f6288057eb0 con 0x7f6290159670 2023-07-13T22:23:00.065 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:00.066 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:22:43.507730+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:22:17.071217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-13T22:22:18.161600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2b","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:23:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627406ace0 msgr2=0x7f627406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627406ace0 0x7f627406d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f629015a590 tx=0x7f628c009040 comp rx=0 tx=0).stop 2023-07-13T22:23:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6290159670 msgr2=0x7f62900cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6290159670 0x7f62900cc2c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f62900ca7c0 tx=0x7f6288007c90 comp rx=0 tx=0).stop 2023-07-13T22:23:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 shutdown_connections 2023-07-13T22:23:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627406ace0 0x7f627406d190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6290159670 0x7f62900cc2c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f627c004850 0x7f62900cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 --2- 172.21.15.60:0/2523655034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62900cdc00 0x7f6290159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 >> 172.21.15.60:0/2523655034 conn(0x7f62900cf410 msgr2=0x7f629015d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 shutdown_connections 2023-07-13T22:23:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.084+0000 7f62979ff700 1 -- 172.21.15.60:0/2523655034 wait complete. 2023-07-13T22:23:00.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:23:00.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.305+0000 7f0b535ff700 1 Processor -- start 2023-07-13T22:23:00.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.305+0000 7f0b535ff700 1 -- start start 2023-07-13T22:23:00.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.307+0000 7f0b535ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c0706f0 0x7f0b4c0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.307+0000 7f0b535ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4c15d970 con 0x7f0b4c070270 2023-07-13T22:23:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.307+0000 7f0b535ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4c15cd60 con 0x7f0b4c159170 2023-07-13T22:23:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.307+0000 7f0b535ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c15d040 con 0x7f0b4c0706f0 2023-07-13T22:23:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.307+0000 7f0b50b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0b4c070270 0x7f0b4c0c9c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:32782/0 (socket says 172.21.15.60:32782) 2023-07-13T22:23:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.307+0000 7f0b50b9a700 1 -- 172.21.15.60:0/3860264071 learned_addr learned my addr 172.21.15.60:0/3860264071 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:00.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.307+0000 7f0b51b9c700 1 --2- 172.21.15.60:0/3860264071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c0706f0 0x7f0b4c0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.307+0000 7f0b43fff700 1 -- 172.21.15.60:0/3860264071 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664728902 0 0) 0x7f0b4c15cd60 con 0x7f0b4c159170 2023-07-13T22:23:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b43fff700 1 -- 172.21.15.60:0/3860264071 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b380035b0 con 0x7f0b4c159170 2023-07-13T22:23:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b43fff700 1 -- 172.21.15.60:0/3860264071 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7853432 0 0) 0x7f0b4c15d970 con 0x7f0b4c070270 2023-07-13T22:23:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b43fff700 1 -- 172.21.15.60:0/3860264071 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b4c15cd60 con 0x7f0b4c070270 2023-07-13T22:23:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b51b9c700 1 -- 172.21.15.60:0/3860264071 >> v1:172.21.15.105:6789/0 conn(0x7f0b4c159170 legacy=0x7f0b4c0c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b51b9c700 1 -- 172.21.15.60:0/3860264071 >> v1:172.21.15.60:6789/0 conn(0x7f0b4c070270 legacy=0x7f0b4c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b51b9c700 1 -- 172.21.15.60:0/3860264071 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c15d320 con 0x7f0b4c0706f0 2023-07-13T22:23:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b51b9c700 1 --2- 172.21.15.60:0/3860264071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c0706f0 0x7f0b4c0ca360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009dc0 tx=0x7f0b4400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7deb3cbbc61650c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b43fff700 1 -- 172.21.15.60:0/3860264071 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400e070 con 0x7f0b4c0706f0 2023-07-13T22:23:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b43fff700 1 -- 172.21.15.60:0/3860264071 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4400bb20 con 0x7f0b4c0706f0 2023-07-13T22:23:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.308+0000 7f0b43fff700 1 -- 172.21.15.60:0/3860264071 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44012430 con 0x7f0b4c0706f0 2023-07-13T22:23:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.309+0000 7f0b535ff700 1 -- 172.21.15.60:0/3860264071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c0706f0 msgr2=0x7f0b4c0ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.309+0000 7f0b535ff700 1 --2- 172.21.15.60:0/3860264071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c0706f0 0x7f0b4c0ca360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009dc0 tx=0x7f0b4400b330 comp rx=0 tx=0).stop 2023-07-13T22:23:00.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.309+0000 7f0b535ff700 1 -- 172.21.15.60:0/3860264071 shutdown_connections 2023-07-13T22:23:00.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.309+0000 7f0b535ff700 1 --2- 172.21.15.60:0/3860264071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c0706f0 0x7f0b4c0ca360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.309+0000 7f0b535ff700 1 -- 172.21.15.60:0/3860264071 >> 172.21.15.60:0/3860264071 conn(0x7f0b4c0cf410 msgr2=0x7f0b4c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:00.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.309+0000 7f0b535ff700 1 -- 172.21.15.60:0/3860264071 shutdown_connections 2023-07-13T22:23:00.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.309+0000 7f0b535ff700 1 -- 172.21.15.60:0/3860264071 wait complete. 2023-07-13T22:23:00.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.311+0000 7f0b535ff700 1 Processor -- start 2023-07-13T22:23:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.311+0000 7f0b535ff700 1 -- start start 2023-07-13T22:23:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b535ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c070270 0x7f0b4c14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b535ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c0706f0 0x7f0b4c0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b535ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c159170 0x7f0b4c0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b535ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b2770 con 0x7f0b4c0706f0 2023-07-13T22:23:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b535ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b1b60 con 0x7f0b4c070270 2023-07-13T22:23:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b535ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b1e40 con 0x7f0b4c159170 2023-07-13T22:23:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b50b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c0706f0 0x7f0b4c0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b50b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c0706f0 0x7f0b4c0b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53048/0 (socket says 172.21.15.60:53048) 2023-07-13T22:23:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b50b9a700 1 -- 172.21.15.60:0/2273329969 learned_addr learned my addr 172.21.15.60:0/2273329969 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b5139b700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c070270 0x7f0b4c14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.312+0000 7f0b51b9c700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c159170 0x7f0b4c0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b50b9a700 1 -- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c159170 msgr2=0x7f0b4c0c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b50b9a700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c159170 0x7f0b4c0c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b50b9a700 1 -- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c070270 msgr2=0x7f0b4c14cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b50b9a700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c070270 0x7f0b4c14cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b50b9a700 1 -- 172.21.15.60:0/2273329969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b44009a70 con 0x7f0b4c0706f0 2023-07-13T22:23:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b51b9c700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c159170 0x7f0b4c0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b5139b700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c070270 0x7f0b4c14cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b50b9a700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c0706f0 0x7f0b4c0b2c00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00c700 tx=0x7f0b3c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:00.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b41ffb700 1 -- 172.21.15.60:0/2273329969 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c005d70 con 0x7f0b4c0706f0 2023-07-13T22:23:00.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b41ffb700 1 -- 172.21.15.60:0/2273329969 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0070d0 con 0x7f0b4c0706f0 2023-07-13T22:23:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c0b2120 con 0x7f0b4c0706f0 2023-07-13T22:23:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b41ffb700 1 -- 172.21.15.60:0/2273329969 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c007460 con 0x7f0b4c0706f0 2023-07-13T22:23:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.313+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c0b3140 con 0x7f0b4c0706f0 2023-07-13T22:23:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.314+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c0b1100 con 0x7f0b4c0706f0 2023-07-13T22:23:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.315+0000 7f0b41ffb700 1 -- 172.21.15.60:0/2273329969 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b3c0075c0 con 0x7f0b4c0706f0 2023-07-13T22:23:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.316+0000 7f0b41ffb700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b3006ace0 0x7f0b3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.316+0000 7f0b41ffb700 1 -- 172.21.15.60:0/2273329969 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f0b3c088bb0 con 0x7f0b4c0706f0 2023-07-13T22:23:00.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.316+0000 7f0b5139b700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b3006ace0 0x7f0b3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.317+0000 7f0b5139b700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b3006ace0 0x7f0b3006d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0b48007fd0 tx=0x7f0b48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.321+0000 7f0b41ffb700 1 -- 172.21.15.60:0/2273329969 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0b3c053bb0 con 0x7f0b4c0706f0 2023-07-13T22:23:00.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.571+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0b4c1553a0 con 0x7f0b4c0706f0 2023-07-13T22:23:00.556 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:23:00.572+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:23:00.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.576+0000 7f0b41ffb700 1 -- 172.21.15.60:0/2273329969 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0b3c0571d0 con 0x7f0b4c0706f0 2023-07-13T22:23:00.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b3006ace0 msgr2=0x7f0b3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b3006ace0 0x7f0b3006d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0b48007fd0 tx=0x7f0b48009040 comp rx=0 tx=0).stop 2023-07-13T22:23:00.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c0706f0 msgr2=0x7f0b4c0b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c0706f0 0x7f0b4c0b2c00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00c700 tx=0x7f0b3c00cac0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 shutdown_connections 2023-07-13T22:23:00.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0b3006ace0 0x7f0b3006d190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b4c159170 0x7f0b4c0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b4c070270 0x7f0b4c14cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 --2- 172.21.15.60:0/2273329969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c0706f0 0x7f0b4c0b2c00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 >> 172.21.15.60:0/2273329969 conn(0x7f0b4c0cf410 msgr2=0x7f0b4c069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:00.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 shutdown_connections 2023-07-13T22:23:00.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.579+0000 7f0b535ff700 1 -- 172.21.15.60:0/2273329969 wait complete. 2023-07-13T22:23:00.565 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:23:00.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:23:00.634 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:23:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.808+0000 7fb395fff700 1 Processor -- start 2023-07-13T22:23:00.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.808+0000 7fb395fff700 1 -- start start 2023-07-13T22:23:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.809+0000 7fb395fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb3900b8200 0x7fb3900cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.809+0000 7fb395fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900d1870 con 0x7fb3900cede0 2023-07-13T22:23:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.809+0000 7fb395fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb390160d80 con 0x7fb3900b1ac0 2023-07-13T22:23:00.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.809+0000 7fb395fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900cf370 con 0x7fb3900b8200 2023-07-13T22:23:00.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38ffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb3900cede0 0x7fb390160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:32796/0 (socket says 172.21.15.60:32796) 2023-07-13T22:23:00.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38ffff700 1 -- 172.21.15.60:0/743466872 learned_addr learned my addr 172.21.15.60:0/743466872 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:00.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38f7fe700 1 --2- 172.21.15.60:0/743466872 >> v2:172.21.15.105:3300/0 conn(0x7fb3900b8200 0x7fb3900cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775938285 0 0) 0x7fb3900d1870 con 0x7fb3900cede0 2023-07-13T22:23:00.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3780035b0 con 0x7fb3900cede0 2023-07-13T22:23:00.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930620408 0 0) 0x7fb390160d80 con 0x7fb3900b1ac0 2023-07-13T22:23:00.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3900d1870 con 0x7fb3900b1ac0 2023-07-13T22:23:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2834528968 0 0) 0x7fb3780035b0 con 0x7fb3900cede0 2023-07-13T22:23:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 >> v2:172.21.15.105:3300/0 conn(0x7fb3900b8200 msgr2=0x7fb3900cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.810+0000 7fb38e7fc700 1 --2- 172.21.15.60:0/743466872 >> v2:172.21.15.105:3300/0 conn(0x7fb3900b8200 0x7fb3900cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 >> v1:172.21.15.105:6789/0 conn(0x7fb3900b1ac0 legacy=0x7fb39015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900d12b0 con 0x7fb3900cede0 2023-07-13T22:23:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fb3880019f0 con 0x7fb3900cede0 2023-07-13T22:23:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 >> v1:172.21.15.60:6789/0 conn(0x7fb3900cede0 legacy=0x7fb390160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb378003b90 0x7fb378003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb378004850 0x7fb378004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb37800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900cf370 con 0x7fb378005200 2023-07-13T22:23:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800c060 con 0x7fb378003b90 2023-07-13T22:23:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800c220 con 0x7fb378004850 2023-07-13T22:23:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38f7fe700 1 --2- 172.21.15.60:0/743466872 >> v2:172.21.15.105:3300/0 conn(0x7fb3900b8200 0x7fb3900cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38ffff700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb37800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38effd700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb378004850 0x7fb378004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38f7fe700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb378003b90 0x7fb378003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38ffff700 1 -- 172.21.15.60:0/743466872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb378004850 msgr2=0x7fb378004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:00.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38ffff700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb378004850 0x7fb378004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38ffff700 1 -- 172.21.15.60:0/743466872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb378003b90 msgr2=0x7fb378003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:00.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38ffff700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb378003b90 0x7fb378003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.811+0000 7fb38ffff700 1 -- 172.21.15.60:0/743466872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3780035b0 con 0x7fb378005200 2023-07-13T22:23:00.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.812+0000 7fb38ffff700 1 -- 172.21.15.60:0/743466872 reap_dead start 2023-07-13T22:23:00.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.812+0000 7fb38ffff700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb37800ba10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb38800a7a0 tx=0x7fb378005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87107174f4d200e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:00.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.812+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388012600 con 0x7fb378005200 2023-07-13T22:23:00.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.812+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb388012780 con 0x7fb378005200 2023-07-13T22:23:00.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.812+0000 7fb38e7fc700 1 -- 172.21.15.60:0/743466872 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388012b30 con 0x7fb378005200 2023-07-13T22:23:00.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.812+0000 7fb395fff700 1 -- 172.21.15.60:0/743466872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 msgr2=0x7fb37800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.812+0000 7fb395fff700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb37800ba10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb38800a7a0 tx=0x7fb378005880 comp rx=0 tx=0).stop 2023-07-13T22:23:00.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.813+0000 7fb395fff700 1 -- 172.21.15.60:0/743466872 shutdown_connections 2023-07-13T22:23:00.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.813+0000 7fb395fff700 1 --2- 172.21.15.60:0/743466872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb37800ba10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.813+0000 7fb395fff700 1 -- 172.21.15.60:0/743466872 >> 172.21.15.60:0/743466872 conn(0x7fb39006b2f0 msgr2=0x7fb39006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:00.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.813+0000 7fb395fff700 1 -- 172.21.15.60:0/743466872 shutdown_connections 2023-07-13T22:23:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.813+0000 7fb395fff700 1 -- 172.21.15.60:0/743466872 wait complete. 2023-07-13T22:23:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.814+0000 7fb395fff700 1 Processor -- start 2023-07-13T22:23:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.815+0000 7fb395fff700 1 -- start start 2023-07-13T22:23:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.815+0000 7fb395fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb390151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.815+0000 7fb395fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39015e200 0x7fb3900b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.815+0000 7fb395fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39015e670 0x7fb3900b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.815+0000 7fb395fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015d950 con 0x7fb378005200 2023-07-13T22:23:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.815+0000 7fb395fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015ca60 con 0x7fb39015e200 2023-07-13T22:23:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.815+0000 7fb395fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015cd40 con 0x7fb39015e670 2023-07-13T22:23:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb390151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb390151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53084/0 (socket says 172.21.15.60:53084) 2023-07-13T22:23:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38f7fe700 1 -- 172.21.15.60:0/939249979 learned_addr learned my addr 172.21.15.60:0/939249979 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38effd700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39015e200 0x7fb3900b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38ffff700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39015e670 0x7fb3900b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38f7fe700 1 -- 172.21.15.60:0/939249979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39015e670 msgr2=0x7fb3900b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38f7fe700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39015e670 0x7fb3900b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38f7fe700 1 -- 172.21.15.60:0/939249979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39015e200 msgr2=0x7fb3900b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38f7fe700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39015e200 0x7fb3900b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38f7fe700 1 -- 172.21.15.60:0/939249979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39015d020 con 0x7fb378005200 2023-07-13T22:23:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38ffff700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39015e670 0x7fb3900b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.816+0000 7fb38f7fe700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb390151090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb380011ca0 tx=0x7fb38000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.817+0000 7fb377fff700 1 -- 172.21.15.60:0/939249979 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38000fb30 con 0x7fb378005200 2023-07-13T22:23:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.817+0000 7fb377fff700 1 -- 172.21.15.60:0/939249979 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38000fc90 con 0x7fb378005200 2023-07-13T22:23:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.817+0000 7fb377fff700 1 -- 172.21.15.60:0/939249979 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380004020 con 0x7fb378005200 2023-07-13T22:23:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.817+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39015d300 con 0x7fb378005200 2023-07-13T22:23:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.817+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3900ccdd0 con 0x7fb378005200 2023-07-13T22:23:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.818+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39015ea40 con 0x7fb378005200 2023-07-13T22:23:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.819+0000 7fb377fff700 1 -- 172.21.15.60:0/939249979 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb380004180 con 0x7fb378005200 2023-07-13T22:23:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.820+0000 7fb377fff700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb37c06ace0 0x7fb37c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.820+0000 7fb377fff700 1 -- 172.21.15.60:0/939249979 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fb380089a40 con 0x7fb378005200 2023-07-13T22:23:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.820+0000 7fb38effd700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb37c06ace0 0x7fb37c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.821+0000 7fb38effd700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb37c06ace0 0x7fb37c06d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb3900b1730 tx=0x7fb384009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:00.824+0000 7fb377fff700 1 -- 172.21.15.60:0/939249979 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb380054a60 con 0x7fb378005200 2023-07-13T22:23:01.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.074+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb39004e210 con 0x7fb378005200 2023-07-13T22:23:01.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.582+0000 7fb377fff700 1 -- 172.21.15.60:0/939249979 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb380058080 con 0x7fb378005200 2023-07-13T22:23:01.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.585+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb37c06ace0 msgr2=0x7fb37c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:01.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.585+0000 7fb395fff700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb37c06ace0 0x7fb37c06d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb3900b1730 tx=0x7fb384009040 comp rx=0 tx=0).stop 2023-07-13T22:23:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.585+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 msgr2=0x7fb390151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.585+0000 7fb395fff700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb390151090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb380011ca0 tx=0x7fb38000f520 comp rx=0 tx=0).stop 2023-07-13T22:23:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.586+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 shutdown_connections 2023-07-13T22:23:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.586+0000 7fb395fff700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb37c06ace0 0x7fb37c06d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.586+0000 7fb395fff700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb378005200 0x7fb390151090 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.586+0000 7fb395fff700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39015e670 0x7fb3900b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.586+0000 7fb395fff700 1 --2- 172.21.15.60:0/939249979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39015e200 0x7fb3900b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.586+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 >> 172.21.15.60:0/939249979 conn(0x7fb39006b2f0 msgr2=0x7fb390071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.586+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 shutdown_connections 2023-07-13T22:23:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.586+0000 7fb395fff700 1 -- 172.21.15.60:0/939249979 wait complete. 2023-07-13T22:23:01.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:23:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.816+0000 7f3b259ff700 1 Processor -- start 2023-07-13T22:23:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.816+0000 7f3b259ff700 1 -- start start 2023-07-13T22:23:01.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.817+0000 7f3b259ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 0x7f3b200cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:01.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.817+0000 7f3b259ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3b2015c9b0 0x7f3b200d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:01.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.817+0000 7f3b259ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3b200d37c0 0x7f3b200d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.817+0000 7f3b259ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b200d2e60 con 0x7f3b2015c9b0 2023-07-13T22:23:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.817+0000 7f3b259ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b200d2b50 con 0x7f3b200d37c0 2023-07-13T22:23:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.817+0000 7f3b259ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2015d200 con 0x7f3b20159ea0 2023-07-13T22:23:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.817+0000 7f3b167fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3b2015c9b0 0x7f3b200d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.817+0000 7f3b167fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3b2015c9b0 0x7f3b200d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53086/0 (socket says 172.21.15.60:53086) 2023-07-13T22:23:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.817+0000 7f3b167fc700 1 -- 172.21.15.60:0/2308952835 learned_addr learned my addr 172.21.15.60:0/2308952835 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b1effd700 1 --2- 172.21.15.60:0/2308952835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 0x7f3b200cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b1f7fe700 1 --2- 172.21.15.60:0/2308952835 >> v2:172.21.15.105:3300/0 conn(0x7f3b200d37c0 0x7f3b200d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b167fc700 1 -- 172.21.15.60:0/2308952835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 msgr2=0x7f3b200cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b167fc700 1 --2- 172.21.15.60:0/2308952835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 0x7f3b200cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b167fc700 1 -- 172.21.15.60:0/2308952835 >> v2:172.21.15.105:3300/0 conn(0x7f3b200d37c0 msgr2=0x7f3b200d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b167fc700 1 --2- 172.21.15.60:0/2308952835 >> v2:172.21.15.105:3300/0 conn(0x7f3b200d37c0 0x7f3b200d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b167fc700 1 -- 172.21.15.60:0/2308952835 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2015d4e0 con 0x7f3b2015c9b0 2023-07-13T22:23:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b1effd700 1 --2- 172.21.15.60:0/2308952835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 0x7f3b200cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b167fc700 1 --2- 172.21.15.60:0/2308952835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2015c9b0 0x7f3b200d3280 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3b200cd2a0 tx=0x7f3b08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ece7bcef10c0e45 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:01.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b1e7fc700 1 -- 172.21.15.60:0/2308952835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800b870 con 0x7f3b2015c9b0 2023-07-13T22:23:01.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b1e7fc700 1 -- 172.21.15.60:0/2308952835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0800b9d0 con 0x7f3b2015c9b0 2023-07-13T22:23:01.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.818+0000 7f3b1e7fc700 1 -- 172.21.15.60:0/2308952835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800bd60 con 0x7f3b2015c9b0 2023-07-13T22:23:01.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.819+0000 7f3b259ff700 1 -- 172.21.15.60:0/2308952835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2015c9b0 msgr2=0x7f3b200d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:01.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.819+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2308952835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2015c9b0 0x7f3b200d3280 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3b200cd2a0 tx=0x7f3b08009bd0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.819+0000 7f3b259ff700 1 -- 172.21.15.60:0/2308952835 shutdown_connections 2023-07-13T22:23:01.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.819+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2308952835 >> v2:172.21.15.105:3300/0 conn(0x7f3b200d37c0 0x7f3b200d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.819+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2308952835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 0x7f3b200cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.819+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2308952835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b2015c9b0 0x7f3b200d3280 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.819+0000 7f3b259ff700 1 -- 172.21.15.60:0/2308952835 >> 172.21.15.60:0/2308952835 conn(0x7f3b20071600 msgr2=0x7f3b200719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:01.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.819+0000 7f3b259ff700 1 -- 172.21.15.60:0/2308952835 shutdown_connections 2023-07-13T22:23:01.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.819+0000 7f3b259ff700 1 -- 172.21.15.60:0/2308952835 wait complete. 2023-07-13T22:23:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.821+0000 7f3b259ff700 1 Processor -- start 2023-07-13T22:23:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.821+0000 7f3b259ff700 1 -- start start 2023-07-13T22:23:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.822+0000 7f3b259ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b200d37c0 0x7f3b20157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.822+0000 7f3b259ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 0x7f3b20157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.822+0000 7f3b259ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b2015c9b0 0x7f3b200c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.822+0000 7f3b259ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20157ca0 con 0x7f3b200d37c0 2023-07-13T22:23:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.822+0000 7f3b259ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200c7230 con 0x7f3b2015c9b0 2023-07-13T22:23:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.822+0000 7f3b259ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200b8950 con 0x7f3b20159ea0 2023-07-13T22:23:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.822+0000 7f3b1effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b200d37c0 0x7f3b20157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.822+0000 7f3b1effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b200d37c0 0x7f3b20157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53102/0 (socket says 172.21.15.60:53102) 2023-07-13T22:23:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.822+0000 7f3b1effd700 1 -- 172.21.15.60:0/2193316880 learned_addr learned my addr 172.21.15.60:0/2193316880 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1f7fe700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b2015c9b0 0x7f3b200c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b167fc700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 0x7f3b20157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1effd700 1 -- 172.21.15.60:0/2193316880 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 msgr2=0x7f3b20157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1effd700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 0x7f3b20157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1effd700 1 -- 172.21.15.60:0/2193316880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b2015c9b0 msgr2=0x7f3b200c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1effd700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b2015c9b0 0x7f3b200c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1effd700 1 -- 172.21.15.60:0/2193316880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08009550 con 0x7f3b200d37c0 2023-07-13T22:23:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1effd700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b200d37c0 0x7f3b20157e60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3b10007fa0 tx=0x7f3b1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1cff9700 1 -- 172.21.15.60:0/2193316880 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000de10 con 0x7f3b200d37c0 2023-07-13T22:23:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1cff9700 1 -- 172.21.15.60:0/2193316880 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1000f040 con 0x7f3b200d37c0 2023-07-13T22:23:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b1cff9700 1 -- 172.21.15.60:0/2193316880 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10003c40 con 0x7f3b200d37c0 2023-07-13T22:23:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.823+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b200c6900 con 0x7f3b200d37c0 2023-07-13T22:23:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.824+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b200c6ec0 con 0x7f3b200d37c0 2023-07-13T22:23:01.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.825+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b200c7510 con 0x7f3b200d37c0 2023-07-13T22:23:01.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.826+0000 7f3b1cff9700 1 -- 172.21.15.60:0/2193316880 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b10004030 con 0x7f3b200d37c0 2023-07-13T22:23:01.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.827+0000 7f3b1cff9700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b0006ace0 0x7f3b0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:01.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.827+0000 7f3b1cff9700 1 -- 172.21.15.60:0/2193316880 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f3b10089480 con 0x7f3b200d37c0 2023-07-13T22:23:01.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.827+0000 7f3b167fc700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b0006ace0 0x7f3b0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:01.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.827+0000 7f3b167fc700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b0006ace0 0x7f3b0006d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b08014040 tx=0x7f3b08008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:01.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:01.831+0000 7f3b1cff9700 1 -- 172.21.15.60:0/2193316880 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3b10054560 con 0x7f3b200d37c0 2023-07-13T22:23:02.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.040+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3b20067a70 con 0x7f3b200d37c0 2023-07-13T22:23:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.585+0000 7f3b1cff9700 1 -- 172.21.15.60:0/2193316880 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3b10057b80 con 0x7f3b200d37c0 2023-07-13T22:23:02.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b0006ace0 msgr2=0x7f3b0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:02.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b0006ace0 0x7f3b0006d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b08014040 tx=0x7f3b08008030 comp rx=0 tx=0).stop 2023-07-13T22:23:02.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b200d37c0 msgr2=0x7f3b20157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:02.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b200d37c0 0x7f3b20157e60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3b10007fa0 tx=0x7f3b1000d3b0 comp rx=0 tx=0).stop 2023-07-13T22:23:02.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 shutdown_connections 2023-07-13T22:23:02.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b0006ace0 0x7f3b0006d190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:02.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b200d37c0 0x7f3b20157e60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:02.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b2015c9b0 0x7f3b200c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 --2- 172.21.15.60:0/2193316880 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b20159ea0 0x7f3b20157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.589+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 >> 172.21.15.60:0/2193316880 conn(0x7f3b20071600 msgr2=0x7f3b200d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.590+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 shutdown_connections 2023-07-13T22:23:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.590+0000 7f3b259ff700 1 -- 172.21.15.60:0/2193316880 wait complete. 2023-07-13T22:23:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:23:02.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:23:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.829+0000 7fce83dff700 1 Processor -- start 2023-07-13T22:23:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.829+0000 7fce83dff700 1 -- start start 2023-07-13T22:23:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.830+0000 7fce83dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce7c0d0890 0x7fce7c0d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.830+0000 7fce83dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c0cdb20 con 0x7fce7c159ea0 2023-07-13T22:23:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.830+0000 7fce83dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c15d200 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.830+0000 7fce83dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c0d2e60 con 0x7fce7c0d0890 2023-07-13T22:23:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.830+0000 7fce81b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fce7c159ea0 0x7fce7c15c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:32804/0 (socket says 172.21.15.60:32804) 2023-07-13T22:23:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.831+0000 7fce81b9b700 1 -- 172.21.15.60:0/3889114795 learned_addr learned my addr 172.21.15.60:0/3889114795 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:02.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.831+0000 7fce8239c700 1 --2- 172.21.15.60:0/3889114795 >> v2:172.21.15.105:3300/0 conn(0x7fce7c0d0890 0x7fce7c0d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:02.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.831+0000 7fce80b99700 1 -- 172.21.15.60:0/3889114795 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20793404 0 0) 0x7fce7c0cdb20 con 0x7fce7c159ea0 2023-07-13T22:23:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.831+0000 7fce80b99700 1 -- 172.21.15.60:0/3889114795 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce680035b0 con 0x7fce7c159ea0 2023-07-13T22:23:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.831+0000 7fce8239c700 1 -- 172.21.15.60:0/3889114795 >> v1:172.21.15.105:6789/0 conn(0x7fce7c0cd6d0 legacy=0x7fce7c0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.831+0000 7fce80b99700 1 -- 172.21.15.60:0/3889114795 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3336603337 0 0) 0x7fce680035b0 con 0x7fce7c159ea0 2023-07-13T22:23:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.831+0000 7fce8239c700 1 -- 172.21.15.60:0/3889114795 >> v1:172.21.15.60:6789/0 conn(0x7fce7c159ea0 legacy=0x7fce7c15c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.831+0000 7fce8239c700 1 -- 172.21.15.60:0/3889114795 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c15d4e0 con 0x7fce7c0d0890 2023-07-13T22:23:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.832+0000 7fce8239c700 1 --2- 172.21.15.60:0/3889114795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 0x7fce7c0d0cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce74009d90 tx=0x7fce7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d02567b04ee2b084 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.832+0000 7fce80b99700 1 -- 172.21.15.60:0/3889114795 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7400e070 con 0x7fce7c0d0890 2023-07-13T22:23:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.832+0000 7fce80b99700 1 -- 172.21.15.60:0/3889114795 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74003a20 con 0x7fce7c0d0890 2023-07-13T22:23:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.832+0000 7fce80b99700 1 -- 172.21.15.60:0/3889114795 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78001670 con 0x7fce7c0d0890 2023-07-13T22:23:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.832+0000 7fce83dff700 1 -- 172.21.15.60:0/3889114795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 msgr2=0x7fce7c0d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.832+0000 7fce83dff700 1 --2- 172.21.15.60:0/3889114795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 0x7fce7c0d0cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce74009d90 tx=0x7fce7400b330 comp rx=0 tx=0).stop 2023-07-13T22:23:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.833+0000 7fce83dff700 1 -- 172.21.15.60:0/3889114795 shutdown_connections 2023-07-13T22:23:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.833+0000 7fce83dff700 1 --2- 172.21.15.60:0/3889114795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 0x7fce7c0d0cc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.833+0000 7fce83dff700 1 -- 172.21.15.60:0/3889114795 >> 172.21.15.60:0/3889114795 conn(0x7fce7c071600 msgr2=0x7fce7c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:02.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.833+0000 7fce83dff700 1 -- 172.21.15.60:0/3889114795 shutdown_connections 2023-07-13T22:23:02.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.833+0000 7fce83dff700 1 -- 172.21.15.60:0/3889114795 wait complete. 2023-07-13T22:23:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.834+0000 7fce83dff700 1 Processor -- start 2023-07-13T22:23:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.835+0000 7fce83dff700 1 -- start start 2023-07-13T22:23:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.835+0000 7fce83dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce7c0cd6d0 0x7fce7c1f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.835+0000 7fce83dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 0x7fce7c14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce83dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce7c159ea0 0x7fce7c0b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce83dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0c9720 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce83dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0c9d90 con 0x7fce7c0d0890 2023-07-13T22:23:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce83dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0ca070 con 0x7fce7c159ea0 2023-07-13T22:23:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce81b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce7c0cd6d0 0x7fce7c1f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce81b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce7c0cd6d0 0x7fce7c1f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53108/0 (socket says 172.21.15.60:53108) 2023-07-13T22:23:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce81b9b700 1 -- 172.21.15.60:0/3092823877 learned_addr learned my addr 172.21.15.60:0/3092823877 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce8239c700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce7c159ea0 0x7fce7c0b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce8139a700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 0x7fce7c14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce81b9b700 1 -- 172.21.15.60:0/3092823877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce7c159ea0 msgr2=0x7fce7c0b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce81b9b700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce7c159ea0 0x7fce7c0b7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce81b9b700 1 -- 172.21.15.60:0/3092823877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 msgr2=0x7fce7c14cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce81b9b700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 0x7fce7c14cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce81b9b700 1 -- 172.21.15.60:0/3092823877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74009a70 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.836+0000 7fce8139a700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 0x7fce7c14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.837+0000 7fce81b9b700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce7c0cd6d0 0x7fce7c1f5680 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fce7800c700 tx=0x7fce7800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:02.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.837+0000 7fce727fc700 1 -- 172.21.15.60:0/3092823877 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78004c80 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.837+0000 7fce727fc700 1 -- 172.21.15.60:0/3092823877 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce78004de0 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.837+0000 7fce727fc700 1 -- 172.21.15.60:0/3092823877 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78007310 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.837+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c0b7680 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.837+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c0c9340 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.838+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c067a70 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.839+0000 7fce727fc700 1 -- 172.21.15.60:0/3092823877 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fce78007470 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.840+0000 7fce727fc700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6006ace0 0x7fce6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.840+0000 7fce727fc700 1 -- 172.21.15.60:0/3092823877 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5739+0+0 (secure 0 0 0) 0x7fce780890e0 con 0x7fce7c0cd6d0 2023-07-13T22:23:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.840+0000 7fce8139a700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6006ace0 0x7fce6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.841+0000 7fce8139a700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6006ace0 0x7fce6006d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fce7c15f030 tx=0x7fce6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:02.844+0000 7fce727fc700 1 -- 172.21.15.60:0/3092823877 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fce780542c0 con 0x7fce7c0cd6d0 2023-07-13T22:23:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.052+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce7c04e210 con 0x7fce7c0cd6d0 2023-07-13T22:23:03.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.593+0000 7fce727fc700 1 -- 172.21.15.60:0/3092823877 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7fce780578e0 con 0x7fce7c0cd6d0 2023-07-13T22:23:03.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.597+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6006ace0 msgr2=0x7fce6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:03.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.597+0000 7fce83dff700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6006ace0 0x7fce6006d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fce7c15f030 tx=0x7fce6c009040 comp rx=0 tx=0).stop 2023-07-13T22:23:03.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce7c0cd6d0 msgr2=0x7fce7c1f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:03.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce7c0cd6d0 0x7fce7c1f5680 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fce7800c700 tx=0x7fce7800ca10 comp rx=0 tx=0).stop 2023-07-13T22:23:03.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 shutdown_connections 2023-07-13T22:23:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6006ace0 0x7fce6006d190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce7c0cd6d0 0x7fce7c1f5680 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce7c159ea0 0x7fce7c0b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 --2- 172.21.15.60:0/3092823877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce7c0d0890 0x7fce7c14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 >> 172.21.15.60:0/3092823877 conn(0x7fce7c071600 msgr2=0x7fce7c0d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 shutdown_connections 2023-07-13T22:23:03.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.598+0000 7fce83dff700 1 -- 172.21.15.60:0/3092823877 wait complete. 2023-07-13T22:23:03.584 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:23:03.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2023-07-13T22:23:03.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.845+0000 7fa02a7ff700 1 Processor -- start 2023-07-13T22:23:03.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.845+0000 7fa02a7ff700 1 -- start start 2023-07-13T22:23:03.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.846+0000 7fa02a7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0240cfd50 0x7fa0240d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:03.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.846+0000 7fa02a7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240d1870 con 0x7fa0240b1ac0 2023-07-13T22:23:03.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.846+0000 7fa02a7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240cf370 con 0x7fa0240b8200 2023-07-13T22:23:03.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.846+0000 7fa02a7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0240d12b0 con 0x7fa0240cfd50 2023-07-13T22:23:03.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.847+0000 7fa0237fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0240cfd50 0x7fa0240d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:03.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.847+0000 7fa028d9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa0240b1ac0 0x7fa0240d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:32812/0 (socket says 172.21.15.60:32812) 2023-07-13T22:23:03.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.847+0000 7fa028d9c700 1 -- 172.21.15.60:0/2597884355 learned_addr learned my addr 172.21.15.60:0/2597884355 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:03.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.847+0000 7fa0237fe700 1 -- 172.21.15.60:0/2597884355 >> v1:172.21.15.105:6789/0 conn(0x7fa0240b8200 legacy=0x7fa0240cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.847+0000 7fa0237fe700 1 -- 172.21.15.60:0/2597884355 >> v1:172.21.15.60:6789/0 conn(0x7fa0240b1ac0 legacy=0x7fa0240d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.847+0000 7fa0237fe700 1 -- 172.21.15.60:0/2597884355 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240d1590 con 0x7fa0240cfd50 2023-07-13T22:23:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.847+0000 7fa022ffd700 1 -- 172.21.15.60:0/2597884355 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754728105 0 0) 0x7fa0240d1870 con 0x7fa0240b1ac0 2023-07-13T22:23:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.847+0000 7fa0237fe700 1 --2- 172.21.15.60:0/2597884355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240cfd50 0x7fa0240d23c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa018009840 tx=0x7fa018009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222cf9cce4d480f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa022ffd700 1 -- 172.21.15.60:0/2597884355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018011830 con 0x7fa0240cfd50 2023-07-13T22:23:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa022ffd700 1 -- 172.21.15.60:0/2597884355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa018011990 con 0x7fa0240cfd50 2023-07-13T22:23:03.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa022ffd700 1 -- 172.21.15.60:0/2597884355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018011d20 con 0x7fa0240cfd50 2023-07-13T22:23:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa02a7ff700 1 -- 172.21.15.60:0/2597884355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240cfd50 msgr2=0x7fa0240d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa02a7ff700 1 --2- 172.21.15.60:0/2597884355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240cfd50 0x7fa0240d23c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa018009840 tx=0x7fa018009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa02a7ff700 1 -- 172.21.15.60:0/2597884355 shutdown_connections 2023-07-13T22:23:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa02a7ff700 1 --2- 172.21.15.60:0/2597884355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240cfd50 0x7fa0240d23c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa02a7ff700 1 -- 172.21.15.60:0/2597884355 >> 172.21.15.60:0/2597884355 conn(0x7fa02406b2f0 msgr2=0x7fa02406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:03.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa02a7ff700 1 -- 172.21.15.60:0/2597884355 shutdown_connections 2023-07-13T22:23:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.848+0000 7fa02a7ff700 1 -- 172.21.15.60:0/2597884355 wait complete. 2023-07-13T22:23:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.851+0000 7fa02a7ff700 1 Processor -- start 2023-07-13T22:23:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.851+0000 7fa02a7ff700 1 -- start start 2023-07-13T22:23:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.852+0000 7fa02a7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240b1ac0 0x7fa0240c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.852+0000 7fa02a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0240b8200 0x7fa0240cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.852+0000 7fa02a7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0240cfd50 0x7fa0241e6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.852+0000 7fa02a7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240cbe70 con 0x7fa0240b1ac0 2023-07-13T22:23:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.852+0000 7fa02a7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240cc150 con 0x7fa0240cfd50 2023-07-13T22:23:03.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.852+0000 7fa02a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240cc430 con 0x7fa0240b8200 2023-07-13T22:23:03.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa023fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240b1ac0 0x7fa0240c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:03.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa023fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240b1ac0 0x7fa0240c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53126/0 (socket says 172.21.15.60:53126) 2023-07-13T22:23:03.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa023fff700 1 -- 172.21.15.60:0/1011258175 learned_addr learned my addr 172.21.15.60:0/1011258175 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:03.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa0237fe700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0240b8200 0x7fa0240cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:03.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa028d9c700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0240cfd50 0x7fa0241e6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:03.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa023fff700 1 -- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0240b8200 msgr2=0x7fa0240cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa023fff700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0240b8200 0x7fa0240cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa023fff700 1 -- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0240cfd50 msgr2=0x7fa0241e6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa023fff700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0240cfd50 0x7fa0241e6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa023fff700 1 -- 172.21.15.60:0/1011258175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180094a0 con 0x7fa0240b1ac0 2023-07-13T22:23:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa0237fe700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0240b8200 0x7fa0240cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa028d9c700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0240cfd50 0x7fa0241e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.853+0000 7fa023fff700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240b1ac0 0x7fa0240c8560 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c4e0 tx=0x7fa01400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:03.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.854+0000 7fa020ff9700 1 -- 172.21.15.60:0/1011258175 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140052d0 con 0x7fa0240b1ac0 2023-07-13T22:23:03.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.854+0000 7fa020ff9700 1 -- 172.21.15.60:0/1011258175 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014005430 con 0x7fa0240b1ac0 2023-07-13T22:23:03.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.854+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0240cc710 con 0x7fa0240b1ac0 2023-07-13T22:23:03.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.854+0000 7fa020ff9700 1 -- 172.21.15.60:0/1011258175 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140038c0 con 0x7fa0240b1ac0 2023-07-13T22:23:03.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.854+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0240c8c80 con 0x7fa0240b1ac0 2023-07-13T22:23:03.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.855+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02404e210 con 0x7fa0240b1ac0 2023-07-13T22:23:03.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.856+0000 7fa020ff9700 1 -- 172.21.15.60:0/1011258175 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa014003a20 con 0x7fa0240b1ac0 2023-07-13T22:23:03.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.857+0000 7fa020ff9700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01006ace0 0x7fa01006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:03.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.857+0000 7fa020ff9700 1 -- 172.21.15.60:0/1011258175 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa014088580 con 0x7fa0240b1ac0 2023-07-13T22:23:03.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.857+0000 7fa0237fe700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01006ace0 0x7fa01006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:03.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.858+0000 7fa0237fe700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01006ace0 0x7fa01006d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0240d0140 tx=0x7fa01801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:03.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:03.861+0000 7fa020ff9700 1 -- 172.21.15.60:0/1011258175 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa0140538b0 con 0x7fa0240b1ac0 2023-07-13T22:23:04.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.066+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) v1 -- 0x7fa0240c8c80 con 0x7fa0240b1ac0 2023-07-13T22:23:04.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.070+0000 7fa020ff9700 1 -- 172.21.15.60:0/1011258175 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v13) v1 ==== 106+0+0 (secure 0 0 0) 0x7fa0240c8c80 con 0x7fa0240b1ac0 2023-07-13T22:23:04.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01006ace0 msgr2=0x7fa01006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01006ace0 0x7fa01006d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0240d0140 tx=0x7fa01801c040 comp rx=0 tx=0).stop 2023-07-13T22:23:04.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240b1ac0 msgr2=0x7fa0240c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240b1ac0 0x7fa0240c8560 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c4e0 tx=0x7fa01400c8a0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 shutdown_connections 2023-07-13T22:23:04.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01006ace0 0x7fa01006d190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0240b1ac0 0x7fa0240c8560 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0240cfd50 0x7fa0241e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 --2- 172.21.15.60:0/1011258175 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0240b8200 0x7fa0240cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 >> 172.21.15.60:0/1011258175 conn(0x7fa02406b2f0 msgr2=0x7fa0240d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:04.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 shutdown_connections 2023-07-13T22:23:04.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.073+0000 7fa02a7ff700 1 -- 172.21.15.60:0/1011258175 wait complete. 2023-07-13T22:23:04.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_cap_acquisition_throttle 2023-07-13T22:23:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.294+0000 7f3c579ff700 1 Processor -- start 2023-07-13T22:23:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.294+0000 7f3c579ff700 1 -- start start 2023-07-13T22:23:04.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.295+0000 7f3c579ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 0x7f3c500ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.295+0000 7f3c579ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c500ac490 0x7f3c500c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.296+0000 7f3c579ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c500cf6d0 con 0x7f3c500ca3a0 2023-07-13T22:23:04.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.296+0000 7f3c579ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c500ac900 con 0x7f3c500ac490 2023-07-13T22:23:04.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.296+0000 7f3c579ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c500acbe0 con 0x7f3c5015c790 2023-07-13T22:23:04.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.296+0000 7f3c5579b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 0x7f3c500ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.296+0000 7f3c54f9a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c500ac490 0x7f3c500c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.296+0000 7f3c5579b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 0x7f3c500ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:38836/0 (socket says 172.21.15.60:38836) 2023-07-13T22:23:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.296+0000 7f3c5579b700 1 -- 172.21.15.60:0/1737268505 learned_addr learned my addr 172.21.15.60:0/1737268505 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.297+0000 7f3c5579b700 1 -- 172.21.15.60:0/1737268505 >> v2:172.21.15.105:3300/0 conn(0x7f3c500ac490 msgr2=0x7f3c500c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.297+0000 7f3c5579b700 1 --2- 172.21.15.60:0/1737268505 >> v2:172.21.15.105:3300/0 conn(0x7f3c500ac490 0x7f3c500c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.297+0000 7f3c5579b700 1 -- 172.21.15.60:0/1737268505 >> v1:172.21.15.105:6789/0 conn(0x7f3c500ca3a0 legacy=0x7f3c500ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.297+0000 7f3c5579b700 1 -- 172.21.15.60:0/1737268505 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c500acec0 con 0x7f3c5015c790 2023-07-13T22:23:04.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.297+0000 7f3c47fff700 1 -- 172.21.15.60:0/1737268505 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688165693 0 0) 0x7f3c500cf6d0 con 0x7f3c500ca3a0 2023-07-13T22:23:04.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.297+0000 7f3c5579b700 1 --2- 172.21.15.60:0/1737268505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 0x7f3c500ce4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006fd0 tx=0x7f3c4c009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=457593bf3e9c573 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:04.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.297+0000 7f3c47fff700 1 -- 172.21.15.60:0/1737268505 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c005e30 con 0x7f3c5015c790 2023-07-13T22:23:04.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.297+0000 7f3c47fff700 1 -- 172.21.15.60:0/1737268505 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c004030 con 0x7f3c5015c790 2023-07-13T22:23:04.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.297+0000 7f3c47fff700 1 -- 172.21.15.60:0/1737268505 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0043e0 con 0x7f3c5015c790 2023-07-13T22:23:04.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.298+0000 7f3c579ff700 1 -- 172.21.15.60:0/1737268505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 msgr2=0x7f3c500ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.298+0000 7f3c579ff700 1 --2- 172.21.15.60:0/1737268505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 0x7f3c500ce4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006fd0 tx=0x7f3c4c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.298+0000 7f3c579ff700 1 -- 172.21.15.60:0/1737268505 shutdown_connections 2023-07-13T22:23:04.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.298+0000 7f3c579ff700 1 --2- 172.21.15.60:0/1737268505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 0x7f3c500ce4b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.298+0000 7f3c579ff700 1 --2- 172.21.15.60:0/1737268505 >> v2:172.21.15.105:3300/0 conn(0x7f3c500ac490 0x7f3c500c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.298+0000 7f3c579ff700 1 -- 172.21.15.60:0/1737268505 >> 172.21.15.60:0/1737268505 conn(0x7f3c5006b900 msgr2=0x7f3c500cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:04.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.298+0000 7f3c579ff700 1 -- 172.21.15.60:0/1737268505 shutdown_connections 2023-07-13T22:23:04.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.298+0000 7f3c579ff700 1 -- 172.21.15.60:0/1737268505 wait complete. 2023-07-13T22:23:04.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.300+0000 7f3c579ff700 1 Processor -- start 2023-07-13T22:23:04.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.301+0000 7f3c579ff700 1 -- start start 2023-07-13T22:23:04.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c579ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c500ac490 0x7f3c5009b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c579ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c500ca3a0 0x7f3c5015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c579ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 0x7f3c5009b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c579ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5009acd0 con 0x7f3c500ac490 2023-07-13T22:23:04.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c579ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5009a0c0 con 0x7f3c500ca3a0 2023-07-13T22:23:04.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c579ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5009a3a0 con 0x7f3c5015c790 2023-07-13T22:23:04.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c5579b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c500ac490 0x7f3c5009b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:04.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c5579b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c500ac490 0x7f3c5009b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53142/0 (socket says 172.21.15.60:53142) 2023-07-13T22:23:04.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c5579b700 1 -- 172.21.15.60:0/2561932386 learned_addr learned my addr 172.21.15.60:0/2561932386 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:04.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c5579b700 1 -- 172.21.15.60:0/2561932386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 msgr2=0x7f3c5009b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c5579b700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 0x7f3c5009b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c5579b700 1 -- 172.21.15.60:0/2561932386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c500ca3a0 msgr2=0x7f3c5015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c5579b700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c500ca3a0 0x7f3c5015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.302+0000 7f3c5579b700 1 -- 172.21.15.60:0/2561932386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c009500 con 0x7f3c500ac490 2023-07-13T22:23:04.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.303+0000 7f3c5579b700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c500ac490 0x7f3c5009b160 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009c10 tx=0x7f3c4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:04.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.303+0000 7f3c45ffb700 1 -- 172.21.15.60:0/2561932386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c003ab0 con 0x7f3c500ac490 2023-07-13T22:23:04.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.303+0000 7f3c45ffb700 1 -- 172.21.15.60:0/2561932386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c003c10 con 0x7f3c500ac490 2023-07-13T22:23:04.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.303+0000 7f3c45ffb700 1 -- 172.21.15.60:0/2561932386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c006030 con 0x7f3c500ac490 2023-07-13T22:23:04.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.303+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5009a680 con 0x7f3c500ac490 2023-07-13T22:23:04.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.303+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5009b6a0 con 0x7f3c500ac490 2023-07-13T22:23:04.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.304+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5004e210 con 0x7f3c500ac490 2023-07-13T22:23:04.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.305+0000 7f3c45ffb700 1 -- 172.21.15.60:0/2561932386 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c4c006190 con 0x7f3c500ac490 2023-07-13T22:23:04.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.307+0000 7f3c45ffb700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3c3406ace0 0x7f3c3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.307+0000 7f3c45ffb700 1 -- 172.21.15.60:0/2561932386 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3c4c08a2c0 con 0x7f3c500ac490 2023-07-13T22:23:04.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.307+0000 7f3c54f9a700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3c3406ace0 0x7f3c3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:04.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.307+0000 7f3c54f9a700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3c3406ace0 0x7f3c3406d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3c40006b20 tx=0x7f3c40006ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:04.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.311+0000 7f3c45ffb700 1 -- 172.21.15.60:0/2561932386 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3c4c055680 con 0x7f3c500ac490 2023-07-13T22:23:04.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.516+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"} v 0) v1 -- 0x7f3c500720e0 con 0x7f3c500ac490 2023-07-13T22:23:04.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.520+0000 7f3c45ffb700 1 -- 172.21.15.60:0/2561932386 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"}]=0 v14) v1 ==== 119+0+0 (secure 0 0 0) 0x7f3c4c058ca0 con 0x7f3c500ac490 2023-07-13T22:23:04.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3c3406ace0 msgr2=0x7f3c3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3c3406ace0 0x7f3c3406d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3c40006b20 tx=0x7f3c40006ab0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c500ac490 msgr2=0x7f3c5009b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c500ac490 0x7f3c5009b160 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009c10 tx=0x7f3c4c004750 comp rx=0 tx=0).stop 2023-07-13T22:23:04.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 shutdown_connections 2023-07-13T22:23:04.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3c3406ace0 0x7f3c3406d190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c500ac490 0x7f3c5009b160 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c5015c790 0x7f3c5009b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 --2- 172.21.15.60:0/2561932386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c500ca3a0 0x7f3c5015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 >> 172.21.15.60:0/2561932386 conn(0x7f3c5006b900 msgr2=0x7f3c500ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:04.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 shutdown_connections 2023-07-13T22:23:04.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.523+0000 7f3c579ff700 1 -- 172.21.15.60:0/2561932386 wait complete. 2023-07-13T22:23:04.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-13T22:23:04.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.747+0000 7f65c29ff700 1 Processor -- start 2023-07-13T22:23:04.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.748+0000 7f65c29ff700 1 -- start start 2023-07-13T22:23:04.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.749+0000 7f65c29ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65bc0cfd50 0x7f65bc0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.749+0000 7f65c29ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65bc0d1870 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.749+0000 7f65c29ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65bc0cf370 con 0x7f65bc0b8200 2023-07-13T22:23:04.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.749+0000 7f65c29ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0d12b0 con 0x7f65bc0cfd50 2023-07-13T22:23:04.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.749+0000 7f65c0f9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f65bc0b1ac0 0x7f65bc0d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:32828/0 (socket says 172.21.15.60:32828) 2023-07-13T22:23:04.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.749+0000 7f65c0f9c700 1 -- 172.21.15.60:0/85825009 learned_addr learned my addr 172.21.15.60:0/85825009 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:04.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.749+0000 7f65bb7fe700 1 --2- 172.21.15.60:0/85825009 >> v2:172.21.15.105:3300/0 conn(0x7f65bc0cfd50 0x7f65bc0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:04.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.749+0000 7f65baffd700 1 -- 172.21.15.60:0/85825009 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006005927 0 0) 0x7f65bc0d1870 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.750+0000 7f65baffd700 1 -- 172.21.15.60:0/85825009 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65a40035b0 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.750+0000 7f65bb7fe700 1 -- 172.21.15.60:0/85825009 >> v1:172.21.15.105:6789/0 conn(0x7f65bc0b8200 legacy=0x7f65bc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.750+0000 7f65bb7fe700 1 -- 172.21.15.60:0/85825009 >> v1:172.21.15.60:6789/0 conn(0x7f65bc0b1ac0 legacy=0x7f65bc0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.750+0000 7f65bb7fe700 1 -- 172.21.15.60:0/85825009 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0d1590 con 0x7f65bc0cfd50 2023-07-13T22:23:04.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.750+0000 7f65bb7fe700 1 --2- 172.21.15.60:0/85825009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc0cfd50 0x7f65bc0d23c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009840 tx=0x7f65b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58444a5a75cb2ecc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:04.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.750+0000 7f65baffd700 1 -- 172.21.15.60:0/85825009 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b0011830 con 0x7f65bc0cfd50 2023-07-13T22:23:04.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.750+0000 7f65baffd700 1 -- 172.21.15.60:0/85825009 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b0011990 con 0x7f65bc0cfd50 2023-07-13T22:23:04.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.750+0000 7f65baffd700 1 -- 172.21.15.60:0/85825009 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b0011d20 con 0x7f65bc0cfd50 2023-07-13T22:23:04.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.751+0000 7f65c29ff700 1 -- 172.21.15.60:0/85825009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc0cfd50 msgr2=0x7f65bc0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.751+0000 7f65c29ff700 1 --2- 172.21.15.60:0/85825009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc0cfd50 0x7f65bc0d23c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009840 tx=0x7f65b0009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:04.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.751+0000 7f65c29ff700 1 -- 172.21.15.60:0/85825009 shutdown_connections 2023-07-13T22:23:04.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.751+0000 7f65c29ff700 1 --2- 172.21.15.60:0/85825009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc0cfd50 0x7f65bc0d23c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.751+0000 7f65c29ff700 1 -- 172.21.15.60:0/85825009 >> 172.21.15.60:0/85825009 conn(0x7f65bc06b2f0 msgr2=0x7f65bc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.751+0000 7f65c29ff700 1 -- 172.21.15.60:0/85825009 shutdown_connections 2023-07-13T22:23:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.751+0000 7f65c29ff700 1 -- 172.21.15.60:0/85825009 wait complete. 2023-07-13T22:23:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.753+0000 7f65c29ff700 1 Processor -- start 2023-07-13T22:23:04.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.753+0000 7f65c29ff700 1 -- start start 2023-07-13T22:23:04.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65c29ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0b1ac0 0x7f65bc0c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65c29ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc0b8200 0x7f65bc0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65c29ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65bc0cfd50 0x7f65bc0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65c29ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0cc850 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65c29ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0c6e60 con 0x7f65bc0b8200 2023-07-13T22:23:04.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65c29ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0c7140 con 0x7f65bc0cfd50 2023-07-13T22:23:04.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0b1ac0 0x7f65bc0c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:04.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0b1ac0 0x7f65bc0c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53152/0 (socket says 172.21.15.60:53152) 2023-07-13T22:23:04.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65bbfff700 1 -- 172.21.15.60:0/1669429589 learned_addr learned my addr 172.21.15.60:0/1669429589 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:04.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65bb7fe700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc0b8200 0x7f65bc0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:04.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.754+0000 7f65c0f9c700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65bc0cfd50 0x7f65bc0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:04.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65bbfff700 1 -- 172.21.15.60:0/1669429589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65bc0cfd50 msgr2=0x7f65bc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65bbfff700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65bc0cfd50 0x7f65bc0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65bbfff700 1 -- 172.21.15.60:0/1669429589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc0b8200 msgr2=0x7f65bc0c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65bbfff700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc0b8200 0x7f65bc0c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65bbfff700 1 -- 172.21.15.60:0/1669429589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b00094a0 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65c0f9c700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65bc0cfd50 0x7f65bc0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:04.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65bbfff700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0b1ac0 0x7f65bc0c7f00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c4e0 tx=0x7f65ac00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:04.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65b8ff9700 1 -- 172.21.15.60:0/1669429589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0052d0 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65b8ff9700 1 -- 172.21.15.60:0/1669429589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac005430 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc0c7420 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65b8ff9700 1 -- 172.21.15.60:0/1669429589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0038c0 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.755+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc0c79e0 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.756+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc04e210 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.757+0000 7f65b8ff9700 1 -- 172.21.15.60:0/1669429589 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65ac003a20 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.758+0000 7f65b8ff9700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a806ad10 0x7f65a806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.758+0000 7f65b8ff9700 1 -- 172.21.15.60:0/1669429589 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f65ac088730 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.758+0000 7f65bb7fe700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a806ad10 0x7f65a806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.759+0000 7f65bb7fe700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a806ad10 0x7f65a806d1c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0d0140 tx=0x7f65b0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.762+0000 7f65b8ff9700 1 -- 172.21.15.60:0/1669429589 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f65ac0539d0 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.968+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f65bc0d3360 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.972+0000 7f65b8ff9700 1 -- 172.21.15.60:0/1669429589 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f65ac056ff0 con 0x7f65bc0b1ac0 2023-07-13T22:23:04.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a806ad10 msgr2=0x7f65a806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a806ad10 0x7f65a806d1c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0d0140 tx=0x7f65b0008030 comp rx=0 tx=0).stop 2023-07-13T22:23:04.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0b1ac0 msgr2=0x7f65bc0c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:04.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0b1ac0 0x7f65bc0c7f00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c4e0 tx=0x7f65ac00c7f0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 shutdown_connections 2023-07-13T22:23:04.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a806ad10 0x7f65a806d1c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0b1ac0 0x7f65bc0c7f00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65bc0cfd50 0x7f65bc0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 --2- 172.21.15.60:0/1669429589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc0b8200 0x7f65bc0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:04.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 >> 172.21.15.60:0/1669429589 conn(0x7f65bc06b2f0 msgr2=0x7f65bc0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:04.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 shutdown_connections 2023-07-13T22:23:04.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:04.974+0000 7f65c29ff700 1 -- 172.21.15.60:0/1669429589 wait complete. 2023-07-13T22:23:04.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-07-13T22:23:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.196+0000 7f1417dff700 1 Processor -- start 2023-07-13T22:23:05.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.196+0000 7f1417dff700 1 -- start start 2023-07-13T22:23:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.197+0000 7f1417dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f141015a390 0x7f14100b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.197+0000 7f1417dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14100cf1f0 0x7f141006c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.197+0000 7f1417dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14100cf640 con 0x7f14100ca520 2023-07-13T22:23:05.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.197+0000 7f1417dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14100cab10 con 0x7f14100cf1f0 2023-07-13T22:23:05.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.197+0000 7f1417dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14100cadf0 con 0x7f141015a390 2023-07-13T22:23:05.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.198+0000 7f1415b9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f141015a390 0x7f14100b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.198+0000 7f141639c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14100cf1f0 0x7f141006c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.198+0000 7f1415b9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f141015a390 0x7f14100b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:55574/0 (socket says 172.21.15.60:55574) 2023-07-13T22:23:05.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.198+0000 7f1415b9b700 1 -- 172.21.15.60:0/10782607 learned_addr learned my addr 172.21.15.60:0/10782607 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:05.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.198+0000 7f141539a700 1 --1- 172.21.15.60:0/10782607 >> v1:172.21.15.105:6789/0 conn(0x7f14100ca520 0x7f1410070df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:39218/0 (socket says 172.21.15.60:39218) 2023-07-13T22:23:05.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.198+0000 7f1415b9b700 1 -- 172.21.15.60:0/10782607 >> v2:172.21.15.105:3300/0 conn(0x7f14100cf1f0 msgr2=0x7f141006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:05.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.198+0000 7f1415b9b700 1 --2- 172.21.15.60:0/10782607 >> v2:172.21.15.105:3300/0 conn(0x7f14100cf1f0 0x7f141006c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.198+0000 7f1415b9b700 1 -- 172.21.15.60:0/10782607 >> v1:172.21.15.105:6789/0 conn(0x7f14100ca520 legacy=0x7f1410070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.198+0000 7f1415b9b700 1 -- 172.21.15.60:0/10782607 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141006e030 con 0x7f141015a390 2023-07-13T22:23:05.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.199+0000 7f1415b9b700 1 --2- 172.21.15.60:0/10782607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f141015a390 0x7f14100b15b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f140c006fd0 tx=0x7f140c009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba330e07f316d95 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.199+0000 7f1414b99700 1 -- 172.21.15.60:0/10782607 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c005e30 con 0x7f141015a390 2023-07-13T22:23:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.199+0000 7f1414b99700 1 -- 172.21.15.60:0/10782607 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c004030 con 0x7f141015a390 2023-07-13T22:23:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.199+0000 7f1414b99700 1 -- 172.21.15.60:0/10782607 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c0043e0 con 0x7f141015a390 2023-07-13T22:23:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.199+0000 7f1417dff700 1 -- 172.21.15.60:0/10782607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f141015a390 msgr2=0x7f14100b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.199+0000 7f1417dff700 1 --2- 172.21.15.60:0/10782607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f141015a390 0x7f14100b15b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f140c006fd0 tx=0x7f140c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.200+0000 7f1417dff700 1 -- 172.21.15.60:0/10782607 shutdown_connections 2023-07-13T22:23:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.200+0000 7f1417dff700 1 --2- 172.21.15.60:0/10782607 >> v2:172.21.15.105:3300/0 conn(0x7f14100cf1f0 0x7f141006c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.200+0000 7f1417dff700 1 --2- 172.21.15.60:0/10782607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f141015a390 0x7f14100b15b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.200+0000 7f1417dff700 1 -- 172.21.15.60:0/10782607 >> 172.21.15.60:0/10782607 conn(0x7f14100cbe70 msgr2=0x7f14100ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.200+0000 7f1417dff700 1 -- 172.21.15.60:0/10782607 shutdown_connections 2023-07-13T22:23:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.200+0000 7f1417dff700 1 -- 172.21.15.60:0/10782607 wait complete. 2023-07-13T22:23:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.201+0000 7f1417dff700 1 Processor -- start 2023-07-13T22:23:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.202+0000 7f1417dff700 1 -- start start 2023-07-13T22:23:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1417dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14100ca520 0x7f14100c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1417dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14100cf1f0 0x7f14101f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1417dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f141015a390 0x7f14101f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1417dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14101f3d50 con 0x7f14100ca520 2023-07-13T22:23:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1417dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14100aef20 con 0x7f141015a390 2023-07-13T22:23:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1417dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14100c3f60 con 0x7f14100cf1f0 2023-07-13T22:23:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1415b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14100ca520 0x7f14100c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1415b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14100ca520 0x7f14100c22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52860/0 (socket says 172.21.15.60:52860) 2023-07-13T22:23:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1415b9b700 1 -- 172.21.15.60:0/3083549275 learned_addr learned my addr 172.21.15.60:0/3083549275 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f141539a700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14100cf1f0 0x7f14101f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f141639c700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f141015a390 0x7f14101f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1415b9b700 1 -- 172.21.15.60:0/3083549275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14100cf1f0 msgr2=0x7f14101f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1415b9b700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14100cf1f0 0x7f14101f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1415b9b700 1 -- 172.21.15.60:0/3083549275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f141015a390 msgr2=0x7f14101f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1415b9b700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f141015a390 0x7f14101f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f1415b9b700 1 -- 172.21.15.60:0/3083549275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c009500 con 0x7f14100ca520 2023-07-13T22:23:05.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.203+0000 7f141539a700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14100cf1f0 0x7f14101f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:05.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.204+0000 7f1415b9b700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14100ca520 0x7f14100c22b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f140c009c10 tx=0x7f140c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:05.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.204+0000 7f14067fc700 1 -- 172.21.15.60:0/3083549275 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c003ab0 con 0x7f14100ca520 2023-07-13T22:23:05.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.204+0000 7f14067fc700 1 -- 172.21.15.60:0/3083549275 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c003c10 con 0x7f14100ca520 2023-07-13T22:23:05.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.204+0000 7f14067fc700 1 -- 172.21.15.60:0/3083549275 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c006030 con 0x7f14100ca520 2023-07-13T22:23:05.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.204+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14100c3650 con 0x7f14100ca520 2023-07-13T22:23:05.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.204+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14100c4f60 con 0x7f14100ca520 2023-07-13T22:23:05.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.205+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141015c130 con 0x7f14100ca520 2023-07-13T22:23:05.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.206+0000 7f14067fc700 1 -- 172.21.15.60:0/3083549275 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f140c006190 con 0x7f14100ca520 2023-07-13T22:23:05.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.207+0000 7f14067fc700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13f406ace0 0x7f13f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.207+0000 7f14067fc700 1 -- 172.21.15.60:0/3083549275 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f140c08a220 con 0x7f14100ca520 2023-07-13T22:23:05.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.207+0000 7f141539a700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13f406ace0 0x7f13f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.208+0000 7f141539a700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13f406ace0 0x7f13f406d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f141006abe0 tx=0x7f1400007dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:05.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.211+0000 7f14067fc700 1 -- 172.21.15.60:0/3083549275 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f140c0555e0 con 0x7f14100ca520 2023-07-13T22:23:05.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.507+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7f141004e280 con 0x7f14100ca520 2023-07-13T22:23:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.595+0000 7f14067fc700 1 -- 172.21.15.60:0/3083549275 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v133) v1 ==== 165+0+0 (secure 0 0 0) 0x7f140c058c00 con 0x7f14100ca520 2023-07-13T22:23:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13f406ace0 msgr2=0x7f13f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13f406ace0 0x7f13f406d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f141006abe0 tx=0x7f1400007dd0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14100ca520 msgr2=0x7f14100c22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14100ca520 0x7f14100c22b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f140c009c10 tx=0x7f140c004750 comp rx=0 tx=0).stop 2023-07-13T22:23:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 shutdown_connections 2023-07-13T22:23:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13f406ace0 0x7f13f406d190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14100ca520 0x7f14100c22b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f141015a390 0x7f14101f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 --2- 172.21.15.60:0/3083549275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14100cf1f0 0x7f14101f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 >> 172.21.15.60:0/3083549275 conn(0x7f14100cbe70 msgr2=0x7f14100b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 shutdown_connections 2023-07-13T22:23:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.598+0000 7f1417dff700 1 -- 172.21.15.60:0/3083549275 wait complete. 2023-07-13T22:23:05.600 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-13T22:23:05.600 INFO:tasks.cephfs_test_runner:Starting test: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:23:05.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2023-07-13T22:23:05.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.819+0000 7f85667ff700 1 Processor -- start 2023-07-13T22:23:05.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.820+0000 7f85667ff700 1 -- start start 2023-07-13T22:23:05.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.821+0000 7f85667ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cd960 0x7f856015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.821+0000 7f85667ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f856015d160 0x7f85600d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.821+0000 7f85667ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85600d16b0 con 0x7f85600cfcf0 2023-07-13T22:23:05.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.821+0000 7f85667ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85600d0d80 con 0x7f856015d160 2023-07-13T22:23:05.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.821+0000 7f85667ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85600d1060 con 0x7f85600cd960 2023-07-13T22:23:05.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.821+0000 7f8564d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f856015d160 0x7f85600d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.821+0000 7f8564d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f856015d160 0x7f85600d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52870/0 (socket says 172.21.15.60:52870) 2023-07-13T22:23:05.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.821+0000 7f8564d9c700 1 -- 172.21.15.60:0/2345005592 learned_addr learned my addr 172.21.15.60:0/2345005592 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.821+0000 7f8557fff700 1 --2- 172.21.15.60:0/2345005592 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cd960 0x7f856015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.822+0000 7f8564d9c700 1 -- 172.21.15.60:0/2345005592 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cd960 msgr2=0x7f856015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.822+0000 7f8564d9c700 1 --2- 172.21.15.60:0/2345005592 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cd960 0x7f856015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.822+0000 7f8564d9c700 1 -- 172.21.15.60:0/2345005592 >> v1:172.21.15.105:6789/0 conn(0x7f85600cfcf0 legacy=0x7f85600cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.822+0000 7f8564d9c700 1 -- 172.21.15.60:0/2345005592 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600d1340 con 0x7f856015d160 2023-07-13T22:23:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.822+0000 7f8564d9c700 1 --2- 172.21.15.60:0/2345005592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 0x7f85600d28f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8558009e50 tx=0x7f855800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c970af83e466a746 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.822+0000 7f855f7fe700 1 -- 172.21.15.60:0/2345005592 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855800e070 con 0x7f856015d160 2023-07-13T22:23:05.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.822+0000 7f855f7fe700 1 -- 172.21.15.60:0/2345005592 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855800b910 con 0x7f856015d160 2023-07-13T22:23:05.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.822+0000 7f855f7fe700 1 -- 172.21.15.60:0/2345005592 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855800bcc0 con 0x7f856015d160 2023-07-13T22:23:05.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.823+0000 7f85667ff700 1 -- 172.21.15.60:0/2345005592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 msgr2=0x7f85600d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.823+0000 7f85667ff700 1 --2- 172.21.15.60:0/2345005592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 0x7f85600d28f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8558009e50 tx=0x7f855800b330 comp rx=0 tx=0).stop 2023-07-13T22:23:05.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.823+0000 7f85667ff700 1 -- 172.21.15.60:0/2345005592 shutdown_connections 2023-07-13T22:23:05.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.823+0000 7f85667ff700 1 --2- 172.21.15.60:0/2345005592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 0x7f85600d28f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.823+0000 7f85667ff700 1 --2- 172.21.15.60:0/2345005592 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cd960 0x7f856015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.823+0000 7f85667ff700 1 -- 172.21.15.60:0/2345005592 >> 172.21.15.60:0/2345005592 conn(0x7f856006b310 msgr2=0x7f856006b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:05.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.823+0000 7f85667ff700 1 -- 172.21.15.60:0/2345005592 shutdown_connections 2023-07-13T22:23:05.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.823+0000 7f85667ff700 1 -- 172.21.15.60:0/2345005592 wait complete. 2023-07-13T22:23:05.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.825+0000 7f85667ff700 1 Processor -- start 2023-07-13T22:23:05.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.825+0000 7f85667ff700 1 -- start start 2023-07-13T22:23:05.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f85667ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85600cd960 0x7f85601f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f85667ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cfcf0 0x7f856014d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f85667ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 0x7f85600cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f85667ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85600ca8c0 con 0x7f856015d160 2023-07-13T22:23:05.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f85667ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85600c9cd0 con 0x7f85600cd960 2023-07-13T22:23:05.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f85667ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856014d680 con 0x7f85600cfcf0 2023-07-13T22:23:05.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8564d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 0x7f85600cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8564d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 0x7f85600cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52886/0 (socket says 172.21.15.60:52886) 2023-07-13T22:23:05.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8564d9c700 1 -- 172.21.15.60:0/2374019176 learned_addr learned my addr 172.21.15.60:0/2374019176 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:05.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f855ffff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85600cd960 0x7f85601f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8557fff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cfcf0 0x7f856014d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8564d9c700 1 -- 172.21.15.60:0/2374019176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cfcf0 msgr2=0x7f856014d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8564d9c700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cfcf0 0x7f856014d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8564d9c700 1 -- 172.21.15.60:0/2374019176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85600cd960 msgr2=0x7f85601f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8564d9c700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85600cd960 0x7f85601f3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8564d9c700 1 -- 172.21.15.60:0/2374019176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8558009b30 con 0x7f856015d160 2023-07-13T22:23:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.826+0000 7f8557fff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cfcf0 0x7f856014d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.827+0000 7f8564d9c700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 0x7f85600cc4d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f855800b680 tx=0x7f8558004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.827+0000 7f855d7fa700 1 -- 172.21.15.60:0/2374019176 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558004910 con 0x7f856015d160 2023-07-13T22:23:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.827+0000 7f855d7fa700 1 -- 172.21.15.60:0/2374019176 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85580099b0 con 0x7f856015d160 2023-07-13T22:23:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.827+0000 7f855d7fa700 1 -- 172.21.15.60:0/2374019176 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855801c680 con 0x7f856015d160 2023-07-13T22:23:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.827+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85600c9f20 con 0x7f856015d160 2023-07-13T22:23:05.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.827+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85600ca4e0 con 0x7f856015d160 2023-07-13T22:23:05.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.828+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85600b8c10 con 0x7f856015d160 2023-07-13T22:23:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.829+0000 7f855d7fa700 1 -- 172.21.15.60:0/2374019176 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f855801c7e0 con 0x7f856015d160 2023-07-13T22:23:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.830+0000 7f855d7fa700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f854006ace0 0x7f854006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.830+0000 7f855d7fa700 1 -- 172.21.15.60:0/2374019176 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f855808ae00 con 0x7f856015d160 2023-07-13T22:23:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.830+0000 7f855ffff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f854006ace0 0x7f854006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.831+0000 7f855ffff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f854006ace0 0x7f854006d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8550000f60 tx=0x7f8550009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:05.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:05.834+0000 7f855d7fa700 1 -- 172.21.15.60:0/2374019176 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f85580561c0 con 0x7f856015d160 2023-07-13T22:23:06.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.122+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) v1 -- 0x7f85600caa00 con 0x7f856015d160 2023-07-13T22:23:06.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.598+0000 7f855d7fa700 1 -- 172.21.15.60:0/2374019176 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v134) v1 ==== 153+0+0 (secure 0 0 0) 0x7f85600caa00 con 0x7f856015d160 2023-07-13T22:23:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.601+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f854006ace0 msgr2=0x7f854006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:06.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.601+0000 7f85667ff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f854006ace0 0x7f854006d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8550000f60 tx=0x7f8550009040 comp rx=0 tx=0).stop 2023-07-13T22:23:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.601+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 msgr2=0x7f85600cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.601+0000 7f85667ff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 0x7f85600cc4d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f855800b680 tx=0x7f8558004ad0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.601+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 shutdown_connections 2023-07-13T22:23:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.602+0000 7f85667ff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f854006ace0 0x7f854006d190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.602+0000 7f85667ff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f856015d160 0x7f85600cc4d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.602+0000 7f85667ff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85600cd960 0x7f85601f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.602+0000 7f85667ff700 1 --2- 172.21.15.60:0/2374019176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85600cfcf0 0x7f856014d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.602+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 >> 172.21.15.60:0/2374019176 conn(0x7f856006b310 msgr2=0x7f85600732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.602+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 shutdown_connections 2023-07-13T22:23:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.602+0000 7f85667ff700 1 -- 172.21.15.60:0/2374019176 wait complete. 2023-07-13T22:23:06.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-13T22:23:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.836+0000 7f4c7adff700 1 Processor -- start 2023-07-13T22:23:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.837+0000 7f4c7adff700 1 -- start start 2023-07-13T22:23:06.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.838+0000 7f4c7adff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4c74159260 0x7f4c740ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:06.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.838+0000 7f4c7adff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c74070af0 0x7f4c740cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:06.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.838+0000 7f4c7adff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740cf640 con 0x7f4c740cc410 2023-07-13T22:23:06.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.838+0000 7f4c7adff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c740ccf80 con 0x7f4c74070af0 2023-07-13T22:23:06.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.838+0000 7f4c7adff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c740c9b40 con 0x7f4c74159260 2023-07-13T22:23:06.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.838+0000 7f4c73fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4c740cc410 0x7f4c740703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51952/0 (socket says 172.21.15.60:51952) 2023-07-13T22:23:06.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.838+0000 7f4c73fff700 1 -- 172.21.15.60:0/4248271665 learned_addr learned my addr 172.21.15.60:0/4248271665 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:06.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.838+0000 7f4c7939c700 1 --2- 172.21.15.60:0/4248271665 >> v2:172.21.15.60:3300/0 conn(0x7f4c74070af0 0x7f4c740cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:06.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.838+0000 7f4c78b9b700 1 --2- 172.21.15.60:0/4248271665 >> v2:172.21.15.105:3300/0 conn(0x7f4c74159260 0x7f4c740ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:06.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.839+0000 7f4c7939c700 1 -- 172.21.15.60:0/4248271665 >> v2:172.21.15.105:3300/0 conn(0x7f4c74159260 msgr2=0x7f4c740ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:06.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.839+0000 7f4c7939c700 1 --2- 172.21.15.60:0/4248271665 >> v2:172.21.15.105:3300/0 conn(0x7f4c74159260 0x7f4c740ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.839+0000 7f4c7939c700 1 -- 172.21.15.60:0/4248271665 >> v1:172.21.15.60:6789/0 conn(0x7f4c740cc410 legacy=0x7f4c740703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:06.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.839+0000 7f4c7939c700 1 -- 172.21.15.60:0/4248271665 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740c9e20 con 0x7f4c74070af0 2023-07-13T22:23:06.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.839+0000 7f4c7939c700 1 --2- 172.21.15.60:0/4248271665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 0x7f4c740cc8b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4c74071110 tx=0x7f4c6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c269717627911d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:06.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.839+0000 7f4c737fe700 1 -- 172.21.15.60:0/4248271665 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00e070 con 0x7f4c74070af0 2023-07-13T22:23:06.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.839+0000 7f4c737fe700 1 -- 172.21.15.60:0/4248271665 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c00bb20 con 0x7f4c74070af0 2023-07-13T22:23:06.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.839+0000 7f4c737fe700 1 -- 172.21.15.60:0/4248271665 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c012430 con 0x7f4c74070af0 2023-07-13T22:23:06.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.840+0000 7f4c7adff700 1 -- 172.21.15.60:0/4248271665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 msgr2=0x7f4c740cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:06.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.840+0000 7f4c7adff700 1 --2- 172.21.15.60:0/4248271665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 0x7f4c740cc8b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4c74071110 tx=0x7f4c6c00b330 comp rx=0 tx=0).stop 2023-07-13T22:23:06.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.840+0000 7f4c7adff700 1 -- 172.21.15.60:0/4248271665 shutdown_connections 2023-07-13T22:23:06.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.840+0000 7f4c7adff700 1 --2- 172.21.15.60:0/4248271665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 0x7f4c740cc8b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.840+0000 7f4c7adff700 1 --2- 172.21.15.60:0/4248271665 >> v2:172.21.15.105:3300/0 conn(0x7f4c74159260 0x7f4c740ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.840+0000 7f4c7adff700 1 -- 172.21.15.60:0/4248271665 >> 172.21.15.60:0/4248271665 conn(0x7f4c740cbf50 msgr2=0x7f4c740ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:06.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.840+0000 7f4c7adff700 1 -- 172.21.15.60:0/4248271665 shutdown_connections 2023-07-13T22:23:06.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.840+0000 7f4c7adff700 1 -- 172.21.15.60:0/4248271665 wait complete. 2023-07-13T22:23:06.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.842+0000 7f4c7adff700 1 Processor -- start 2023-07-13T22:23:06.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.843+0000 7f4c7adff700 1 -- start start 2023-07-13T22:23:06.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c7adff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 0x7f4c74151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:06.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c7adff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c740cc410 0x7f4c74152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:06.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c7adff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c74159260 0x7f4c74156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:06.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c7adff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c741521c0 con 0x7f4c74070af0 2023-07-13T22:23:06.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c7adff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740c5910 con 0x7f4c740cc410 2023-07-13T22:23:06.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c7adff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740c5b60 con 0x7f4c74159260 2023-07-13T22:23:06.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c78b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 0x7f4c74151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:06.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c78b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 0x7f4c74151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52908/0 (socket says 172.21.15.60:52908) 2023-07-13T22:23:06.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c78b9b700 1 -- 172.21.15.60:0/4112684619 learned_addr learned my addr 172.21.15.60:0/4112684619 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:06.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.844+0000 7f4c73fff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c740cc410 0x7f4c74152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:06.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c7939c700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c74159260 0x7f4c74156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:06.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c78b9b700 1 -- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c74159260 msgr2=0x7f4c74156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:06.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c78b9b700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c74159260 0x7f4c74156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c78b9b700 1 -- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c740cc410 msgr2=0x7f4c74152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c78b9b700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c740cc410 0x7f4c74152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c78b9b700 1 -- 172.21.15.60:0/4112684619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c009a70 con 0x7f4c74070af0 2023-07-13T22:23:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c73fff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c740cc410 0x7f4c74152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c7939c700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c74159260 0x7f4c74156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c78b9b700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 0x7f4c74151c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800e870 tx=0x7f4c6800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:06.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c717fa700 1 -- 172.21.15.60:0/4112684619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68009590 con 0x7f4c74070af0 2023-07-13T22:23:06.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c717fa700 1 -- 172.21.15.60:0/4112684619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6800ee70 con 0x7f4c74070af0 2023-07-13T22:23:06.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c717fa700 1 -- 172.21.15.60:0/4112684619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6800ca50 con 0x7f4c74070af0 2023-07-13T22:23:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c740c5e40 con 0x7f4c74070af0 2023-07-13T22:23:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.845+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c740c6400 con 0x7f4c74070af0 2023-07-13T22:23:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.846+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7406c3c0 con 0x7f4c74070af0 2023-07-13T22:23:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.847+0000 7f4c717fa700 1 -- 172.21.15.60:0/4112684619 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c6800cbb0 con 0x7f4c74070af0 2023-07-13T22:23:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.848+0000 7f4c717fa700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c5806ad10 0x7f4c5806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.848+0000 7f4c717fa700 1 -- 172.21.15.60:0/4112684619 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4c680891f0 con 0x7f4c74070af0 2023-07-13T22:23:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.849+0000 7f4c73fff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c5806ad10 0x7f4c5806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:06.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.849+0000 7f4c73fff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c5806ad10 0x7f4c5806d1c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4c7415c630 tx=0x7f4c60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:06.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:06.853+0000 7f4c717fa700 1 -- 172.21.15.60:0/4112684619 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4c68054490 con 0x7f4c74070af0 2023-07-13T22:23:07.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.062+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4c7404e210 con 0x7f4c74070af0 2023-07-13T22:23:07.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.066+0000 7f4c717fa700 1 -- 172.21.15.60:0/4112684619 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c6800ce60 con 0x7f4c74070af0 2023-07-13T22:23:07.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c5806ad10 msgr2=0x7f4c5806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:07.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c5806ad10 0x7f4c5806d1c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4c7415c630 tx=0x7f4c60009040 comp rx=0 tx=0).stop 2023-07-13T22:23:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 msgr2=0x7f4c74151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 0x7f4c74151c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800e870 tx=0x7f4c6800ec30 comp rx=0 tx=0).stop 2023-07-13T22:23:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 shutdown_connections 2023-07-13T22:23:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c5806ad10 0x7f4c5806d1c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c74070af0 0x7f4c74151c80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c74159260 0x7f4c74156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 --2- 172.21.15.60:0/4112684619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c740cc410 0x7f4c74152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 >> 172.21.15.60:0/4112684619 conn(0x7f4c740cbf50 msgr2=0x7f4c7406d420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 shutdown_connections 2023-07-13T22:23:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.068+0000 7f4c7adff700 1 -- 172.21.15.60:0/4112684619 wait complete. 2023-07-13T22:23:07.070 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:23:07.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:23:07.175 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:23:07.280 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:23:07.281 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:23:07.281 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:23:07.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:23:07.328 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:23:07.328 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:23:07.377 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:23:07.378 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:23:07.378 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:23:07.425 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:23:07.426 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:23:07.426 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:23:07.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:23:07.473 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-13T22:23:07.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:23:07.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.693+0000 7f6fde3ff700 1 Processor -- start 2023-07-13T22:23:07.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.694+0000 7f6fde3ff700 1 -- start start 2023-07-13T22:23:07.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.695+0000 7f6fde3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80d3750 0x7f6fd80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:07.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.695+0000 7f6fde3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6fd80cfad0 0x7f6fd80d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:07.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.695+0000 7f6fde3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd80cd950 con 0x7f6fd80b8400 2023-07-13T22:23:07.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fde3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd80cff20 con 0x7f6fd80cfad0 2023-07-13T22:23:07.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fde3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80d0ac0 con 0x7f6fd80d3750 2023-07-13T22:23:07.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fd7fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6fd80b8400 0x7f6fd80cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51960/0 (socket says 172.21.15.60:51960) 2023-07-13T22:23:07.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fd7fff700 1 -- 172.21.15.60:0/272827630 learned_addr learned my addr 172.21.15.60:0/272827630 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:07.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/272827630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80d3750 0x7f6fd80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:07.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fdc99c700 1 --2- 172.21.15.60:0/272827630 >> v2:172.21.15.105:3300/0 conn(0x7f6fd80cfad0 0x7f6fd80d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:07.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fd6ffd700 1 -- 172.21.15.60:0/272827630 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473734742 0 0) 0x7f6fc8001110 con 0x7f6fd80b8400 2023-07-13T22:23:07.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fd6ffd700 1 -- 172.21.15.60:0/272827630 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc00035b0 con 0x7f6fd80b8400 2023-07-13T22:23:07.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fd77fe700 1 -- 172.21.15.60:0/272827630 >> v2:172.21.15.105:3300/0 conn(0x7f6fd80cfad0 msgr2=0x7f6fd80d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:07.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/272827630 >> v2:172.21.15.105:3300/0 conn(0x7f6fd80cfad0 0x7f6fd80d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fd77fe700 1 -- 172.21.15.60:0/272827630 >> v1:172.21.15.60:6789/0 conn(0x7f6fd80b8400 legacy=0x7f6fd80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:07.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fd77fe700 1 -- 172.21.15.60:0/272827630 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd80b87d0 con 0x7f6fd80d3750 2023-07-13T22:23:07.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.696+0000 7f6fdc99c700 1 --2- 172.21.15.60:0/272827630 >> v2:172.21.15.105:3300/0 conn(0x7f6fd80cfad0 0x7f6fd80d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:07.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.697+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/272827630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80d3750 0x7f6fd80cf590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009840 tx=0x7f6fcc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4abc3094e7439d54 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:07.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.697+0000 7f6fd6ffd700 1 -- 172.21.15.60:0/272827630 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc011830 con 0x7f6fd80d3750 2023-07-13T22:23:07.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.697+0000 7f6fd6ffd700 1 -- 172.21.15.60:0/272827630 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc011990 con 0x7f6fd80d3750 2023-07-13T22:23:07.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.697+0000 7f6fd6ffd700 1 -- 172.21.15.60:0/272827630 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc011d20 con 0x7f6fd80d3750 2023-07-13T22:23:07.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.698+0000 7f6fde3ff700 1 -- 172.21.15.60:0/272827630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80d3750 msgr2=0x7f6fd80cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:07.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.698+0000 7f6fde3ff700 1 --2- 172.21.15.60:0/272827630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80d3750 0x7f6fd80cf590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009840 tx=0x7f6fcc009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:07.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.698+0000 7f6fde3ff700 1 -- 172.21.15.60:0/272827630 shutdown_connections 2023-07-13T22:23:07.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.698+0000 7f6fde3ff700 1 --2- 172.21.15.60:0/272827630 >> v2:172.21.15.105:3300/0 conn(0x7f6fd80cfad0 0x7f6fd80d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.698+0000 7f6fde3ff700 1 --2- 172.21.15.60:0/272827630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80d3750 0x7f6fd80cf590 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.698+0000 7f6fde3ff700 1 -- 172.21.15.60:0/272827630 >> 172.21.15.60:0/272827630 conn(0x7f6fd806b390 msgr2=0x7f6fd806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:07.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.698+0000 7f6fde3ff700 1 -- 172.21.15.60:0/272827630 shutdown_connections 2023-07-13T22:23:07.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.698+0000 7f6fde3ff700 1 -- 172.21.15.60:0/272827630 wait complete. 2023-07-13T22:23:07.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.700+0000 7f6fde3ff700 1 Processor -- start 2023-07-13T22:23:07.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.700+0000 7f6fde3ff700 1 -- start start 2023-07-13T22:23:07.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fde3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80b8400 0x7f6fd8151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fde3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80cfad0 0x7f6fd80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fde3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80d3750 0x7f6fd80b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fde3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd81515d0 con 0x7f6fd80b8400 2023-07-13T22:23:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fde3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80cb980 con 0x7f6fd80d3750 2023-07-13T22:23:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fde3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80cbc60 con 0x7f6fd80cfad0 2023-07-13T22:23:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fd7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80b8400 0x7f6fd8151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fd7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80b8400 0x7f6fd8151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52924/0 (socket says 172.21.15.60:52924) 2023-07-13T22:23:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fd7fff700 1 -- 172.21.15.60:0/3107383346 learned_addr learned my addr 172.21.15.60:0/3107383346 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:07.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80cfad0 0x7f6fd80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:07.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fdc99c700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80d3750 0x7f6fd80b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:07.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fd7fff700 1 -- 172.21.15.60:0/3107383346 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80cfad0 msgr2=0x7f6fd80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:07.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.701+0000 7f6fd7fff700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80cfad0 0x7f6fd80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.702+0000 7f6fd7fff700 1 -- 172.21.15.60:0/3107383346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80d3750 msgr2=0x7f6fd80b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:07.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.702+0000 7f6fd7fff700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80d3750 0x7f6fd80b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.702+0000 7f6fd7fff700 1 -- 172.21.15.60:0/3107383346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd0009a70 con 0x7f6fd80b8400 2023-07-13T22:23:07.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.702+0000 7f6fd7fff700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80b8400 0x7f6fd8151090 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8007fd0 tx=0x7f6fc800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:07.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.702+0000 7f6fd4ff9700 1 -- 172.21.15.60:0/3107383346 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800cea0 con 0x7f6fd80b8400 2023-07-13T22:23:07.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.702+0000 7f6fd4ff9700 1 -- 172.21.15.60:0/3107383346 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc8005d70 con 0x7f6fd80b8400 2023-07-13T22:23:07.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.702+0000 7f6fd4ff9700 1 -- 172.21.15.60:0/3107383346 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc80072f0 con 0x7f6fd80b8400 2023-07-13T22:23:07.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.702+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc0094a0 con 0x7f6fd80b8400 2023-07-13T22:23:07.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.702+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd80cc220 con 0x7f6fd80b8400 2023-07-13T22:23:07.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.703+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd815ad60 con 0x7f6fd80b8400 2023-07-13T22:23:07.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.704+0000 7f6fd4ff9700 1 -- 172.21.15.60:0/3107383346 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6fc8007450 con 0x7f6fd80b8400 2023-07-13T22:23:07.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.706+0000 7f6fd4ff9700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:07.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.706+0000 7f6fd4ff9700 1 -- 172.21.15.60:0/3107383346 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6fc80883d0 con 0x7f6fd80b8400 2023-07-13T22:23:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.706+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.707+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80d3b60 tx=0x7f6fcc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.710+0000 7f6fd4ff9700 1 -- 172.21.15.60:0/3107383346 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6fc8053790 con 0x7f6fd80b8400 2023-07-13T22:23:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.961+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fd80cd950 con 0x7f6fd80b8400 2023-07-13T22:23:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.961+0000 7f6fd4ff9700 1 -- 172.21.15.60:0/3107383346 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f6fc8017de0 con 0x7f6fd80b8400 2023-07-13T22:23:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:07.945 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2157014443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2157014443},{"type":"v1","addr":"172.21.15.105:6834","nonce":2157014443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/302067138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":302067138},{"type":"v1","addr":"172.21.15.105:6835","nonce":302067138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3461450784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461450784},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461450784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/3932355303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":3932355303},{"type":"v1","addr":"172.21.15.60:6837","nonce":3932355303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-07-13T22:23:07.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 msgr2=0x7f6fc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80d3b60 tx=0x7f6fcc01c040 comp rx=0 tx=0).stop 2023-07-13T22:23:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80b8400 msgr2=0x7f6fd8151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80b8400 0x7f6fd8151090 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8007fd0 tx=0x7f6fc800c7b0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 shutdown_connections 2023-07-13T22:23:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80b8400 0x7f6fd8151090 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80d3750 0x7f6fd80b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 --2- 172.21.15.60:0/3107383346 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd80cfad0 0x7f6fd80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 >> 172.21.15.60:0/3107383346 conn(0x7f6fd806b390 msgr2=0x7f6fd80d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 shutdown_connections 2023-07-13T22:23:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:07.964+0000 7f6fde3ff700 1 -- 172.21.15.60:0/3107383346 wait complete. 2023-07-13T22:23:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 18 2023-07-13T22:23:07.966 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:23:07.966 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-13T22:23:07.966 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:23:07.967 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:23:07.967 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:23:07.967 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-13T22:23:07.967 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:23:07.967 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:23:07.968 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:23:07.968 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-13T22:23:07.968 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:23:07.968 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:23:07.969 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:23:07.969 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-13T22:23:07.969 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:23:07.969 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:23:07.980 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:23:07.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:23:07.981 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:23:07.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:23:07.987 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:23:07.987 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:23:07.988 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:23:07.988 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:23:08.025 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:23:08.025 DEBUG:teuthology.parallel:result is None 2023-07-13T22:23:08.027 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:23:08.028 DEBUG:teuthology.parallel:result is None 2023-07-13T22:23:08.032 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:23:08.032 DEBUG:teuthology.parallel:result is None 2023-07-13T22:23:08.035 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:23:08.035 DEBUG:teuthology.parallel:result is None 2023-07-13T22:23:08.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-13T22:23:08.179 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:23:08.180 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:23:08.183 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:23:08.184 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:23:08.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.266+0000 7f7c32fff700 1 Processor -- start 2023-07-13T22:23:08.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.266+0000 7f7c32fff700 1 -- start start 2023-07-13T22:23:08.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.267+0000 7f7c32fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c2c070c50 0x7f7c2c0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.267+0000 7f7c32fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c15c700 0x7f7c2c15ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.267+0000 7f7c32fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c06b290 con 0x7f7c2c06e990 2023-07-13T22:23:08.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.267+0000 7f7c32fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2c06ee30 con 0x7f7c2c070c50 2023-07-13T22:23:08.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.267+0000 7f7c32fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0cd330 con 0x7f7c2c15c700 2023-07-13T22:23:08.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c30d9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c2c070c50 0x7f7c2c0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c3159c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c15c700 0x7f7c2c15ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c30d9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7c2c070c50 0x7f7c2c0ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:48750/0 (socket says 172.21.15.60:48750) 2023-07-13T22:23:08.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c30d9b700 1 -- 172.21.15.60:0/290379083 learned_addr learned my addr 172.21.15.60:0/290379083 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:08.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c2bfff700 1 --1- 172.21.15.60:0/290379083 >> v1:172.21.15.105:6789/0 conn(0x7f7c2c06e990 0x7f7c2c071020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:39232/0 (socket says 172.21.15.60:39232) 2023-07-13T22:23:08.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c2b7fe700 1 -- 172.21.15.60:0/290379083 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797752273 0 0) 0x7f7c2c06b290 con 0x7f7c2c06e990 2023-07-13T22:23:08.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c2b7fe700 1 -- 172.21.15.60:0/290379083 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c1c0035b0 con 0x7f7c2c06e990 2023-07-13T22:23:08.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c3159c700 1 -- 172.21.15.60:0/290379083 >> v2:172.21.15.105:3300/0 conn(0x7f7c2c070c50 msgr2=0x7f7c2c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:08.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c3159c700 1 --2- 172.21.15.60:0/290379083 >> v2:172.21.15.105:3300/0 conn(0x7f7c2c070c50 0x7f7c2c0ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c3159c700 1 -- 172.21.15.60:0/290379083 >> v1:172.21.15.105:6789/0 conn(0x7f7c2c06e990 legacy=0x7f7c2c071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c3159c700 1 -- 172.21.15.60:0/290379083 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0ccd70 con 0x7f7c2c15c700 2023-07-13T22:23:08.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.268+0000 7f7c30d9b700 1 --2- 172.21.15.60:0/290379083 >> v2:172.21.15.105:3300/0 conn(0x7f7c2c070c50 0x7f7c2c0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:08.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c3159c700 1 --2- 172.21.15.60:0/290379083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c15c700 0x7f7c2c15ffc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009d90 tx=0x7f7c2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e186239ad50734a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:08.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c2b7fe700 1 -- 172.21.15.60:0/290379083 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400e070 con 0x7f7c2c15c700 2023-07-13T22:23:08.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c2b7fe700 1 -- 172.21.15.60:0/290379083 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2400bb20 con 0x7f7c2c15c700 2023-07-13T22:23:08.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c2b7fe700 1 -- 172.21.15.60:0/290379083 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24012430 con 0x7f7c2c15c700 2023-07-13T22:23:08.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c32fff700 1 -- 172.21.15.60:0/290379083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c15c700 msgr2=0x7f7c2c15ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c32fff700 1 --2- 172.21.15.60:0/290379083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c15c700 0x7f7c2c15ffc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009d90 tx=0x7f7c2400b330 comp rx=0 tx=0).stop 2023-07-13T22:23:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c32fff700 1 -- 172.21.15.60:0/290379083 shutdown_connections 2023-07-13T22:23:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c32fff700 1 --2- 172.21.15.60:0/290379083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c15c700 0x7f7c2c15ffc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c32fff700 1 --2- 172.21.15.60:0/290379083 >> v2:172.21.15.105:3300/0 conn(0x7f7c2c070c50 0x7f7c2c0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c32fff700 1 -- 172.21.15.60:0/290379083 >> 172.21.15.60:0/290379083 conn(0x7f7c2c0b1820 msgr2=0x7f7c2c0cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.269+0000 7f7c32fff700 1 -- 172.21.15.60:0/290379083 shutdown_connections 2023-07-13T22:23:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.270+0000 7f7c32fff700 1 -- 172.21.15.60:0/290379083 wait complete. 2023-07-13T22:23:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.272+0000 7f7c32fff700 1 Processor -- start 2023-07-13T22:23:08.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.272+0000 7f7c32fff700 1 -- start start 2023-07-13T22:23:08.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.273+0000 7f7c32fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2c06e990 0x7f7c2c1e3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.273+0000 7f7c32fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c070c50 0x7f7c2c1f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.273+0000 7f7c32fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2c15c700 0x7f7c2c0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.273+0000 7f7c32fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0c4140 con 0x7f7c2c06e990 2023-07-13T22:23:08.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.273+0000 7f7c32fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0c4420 con 0x7f7c2c15c700 2023-07-13T22:23:08.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.273+0000 7f7c32fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0c4700 con 0x7f7c2c070c50 2023-07-13T22:23:08.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c30d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2c06e990 0x7f7c2c1e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c30d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2c06e990 0x7f7c2c1e3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52948/0 (socket says 172.21.15.60:52948) 2023-07-13T22:23:08.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c30d9b700 1 -- 172.21.15.60:0/2271399161 learned_addr learned my addr 172.21.15.60:0/2271399161 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:08.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c3159c700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2c15c700 0x7f7c2c0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c2bfff700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c070c50 0x7f7c2c1f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c30d9b700 1 -- 172.21.15.60:0/2271399161 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c070c50 msgr2=0x7f7c2c1f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c30d9b700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c070c50 0x7f7c2c1f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c30d9b700 1 -- 172.21.15.60:0/2271399161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2c15c700 msgr2=0x7f7c2c0c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c30d9b700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2c15c700 0x7f7c2c0c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c30d9b700 1 -- 172.21.15.60:0/2271399161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c20009550 con 0x7f7c2c06e990 2023-07-13T22:23:08.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.274+0000 7f7c30d9b700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2c06e990 0x7f7c2c1e3080 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000ebe0 tx=0x7f7c2000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:08.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.275+0000 7f7c297fa700 1 -- 172.21.15.60:0/2271399161 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000c850 con 0x7f7c2c06e990 2023-07-13T22:23:08.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.275+0000 7f7c297fa700 1 -- 172.21.15.60:0/2271399161 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2000c9b0 con 0x7f7c2c06e990 2023-07-13T22:23:08.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.275+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c24009a70 con 0x7f7c2c06e990 2023-07-13T22:23:08.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.275+0000 7f7c297fa700 1 -- 172.21.15.60:0/2271399161 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000cd40 con 0x7f7c2c06e990 2023-07-13T22:23:08.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.275+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c0c4cc0 con 0x7f7c2c06e990 2023-07-13T22:23:08.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.276+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c0c5030 con 0x7f7c2c06e990 2023-07-13T22:23:08.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.277+0000 7f7c297fa700 1 -- 172.21.15.60:0/2271399161 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7c20013470 con 0x7f7c2c06e990 2023-07-13T22:23:08.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.278+0000 7f7c297fa700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7c1006ace0 0x7f7c1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.278+0000 7f7c297fa700 1 -- 172.21.15.60:0/2271399161 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7c20089290 con 0x7f7c2c06e990 2023-07-13T22:23:08.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.278+0000 7f7c2bfff700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7c1006ace0 0x7f7c1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.279+0000 7f7c2bfff700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7c1006ace0 0x7f7c1006d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c1568c0 tx=0x7f7c18007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:08.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.282+0000 7f7c297fa700 1 -- 172.21.15.60:0/2271399161 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7c20054650 con 0x7f7c2c06e990 2023-07-13T22:23:08.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:08.507+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7c2c04e280 con 0x7f7c2c06e990 2023-07-13T22:23:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.458+0000 7f7c297fa700 1 -- 172.21.15.60:0/2271399161 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7c20057c70 con 0x7f7c2c06e990 2023-07-13T22:23:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7c1006ace0 msgr2=0x7f7c1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7c1006ace0 0x7f7c1006d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c1568c0 tx=0x7f7c18007dc0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2c06e990 msgr2=0x7f7c2c1e3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2c06e990 0x7f7c2c1e3080 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000ebe0 tx=0x7f7c2000eef0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 shutdown_connections 2023-07-13T22:23:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7c1006ace0 0x7f7c1006d190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2c06e990 0x7f7c2c1e3080 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2c15c700 0x7f7c2c0c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 --2- 172.21.15.60:0/2271399161 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7c2c070c50 0x7f7c2c1f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 >> 172.21.15.60:0/2271399161 conn(0x7f7c2c0b1820 msgr2=0x7f7c2c0b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 shutdown_connections 2023-07-13T22:23:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.461+0000 7f7c32fff700 1 -- 172.21.15.60:0/2271399161 wait complete. 2023-07-13T22:23:09.447 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-07-13T22:23:09.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-13T22:23:09.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.689+0000 7f96d53ff700 1 Processor -- start 2023-07-13T22:23:09.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.689+0000 7f96d53ff700 1 -- start start 2023-07-13T22:23:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.690+0000 7f96d53ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00b8570 0x7f96d00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.690+0000 7f96d53ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d00d09d0 con 0x7f96d00d35c0 2023-07-13T22:23:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.690+0000 7f96d53ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d00cda10 con 0x7f96d00d15c0 2023-07-13T22:23:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.690+0000 7f96d53ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00d3b30 con 0x7f96d00b8570 2023-07-13T22:23:09.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96ce7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f96d00d35c0 0x7f96d00d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51980/0 (socket says 172.21.15.60:51980) 2023-07-13T22:23:09.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96ce7fc700 1 -- 172.21.15.60:0/1599508374 learned_addr learned my addr 172.21.15.60:0/1599508374 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96ceffd700 1 --2- 172.21.15.60:0/1599508374 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00b8570 0x7f96d00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96cdffb700 1 -- 172.21.15.60:0/1599508374 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75707575 0 0) 0x7f96d00d09d0 con 0x7f96d00d35c0 2023-07-13T22:23:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96cdffb700 1 -- 172.21.15.60:0/1599508374 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96b80035b0 con 0x7f96d00d35c0 2023-07-13T22:23:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96cdffb700 1 -- 172.21.15.60:0/1599508374 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787048381 0 0) 0x7f96d00cda10 con 0x7f96d00d15c0 2023-07-13T22:23:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96cdffb700 1 -- 172.21.15.60:0/1599508374 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d00d09d0 con 0x7f96d00d15c0 2023-07-13T22:23:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96cdffb700 1 -- 172.21.15.60:0/1599508374 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2346100671 0 0) 0x7f96b80035b0 con 0x7f96d00d35c0 2023-07-13T22:23:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96ceffd700 1 -- 172.21.15.60:0/1599508374 >> v1:172.21.15.105:6789/0 conn(0x7f96d00d15c0 legacy=0x7f96d015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96ceffd700 1 -- 172.21.15.60:0/1599508374 >> v1:172.21.15.60:6789/0 conn(0x7f96d00d35c0 legacy=0x7f96d00d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.691+0000 7f96ceffd700 1 -- 172.21.15.60:0/1599508374 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00cd450 con 0x7f96d00b8570 2023-07-13T22:23:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.692+0000 7f96ceffd700 1 --2- 172.21.15.60:0/1599508374 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00b8570 0x7f96d00ceeb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009850 tx=0x7f96c0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d951d0c5b367bb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.692+0000 7f96cdffb700 1 -- 172.21.15.60:0/1599508374 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40015f0 con 0x7f96d00b8570 2023-07-13T22:23:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.692+0000 7f96cdffb700 1 -- 172.21.15.60:0/1599508374 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0005c30 con 0x7f96d00b8570 2023-07-13T22:23:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.692+0000 7f96cdffb700 1 -- 172.21.15.60:0/1599508374 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0004030 con 0x7f96d00b8570 2023-07-13T22:23:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.693+0000 7f96d53ff700 1 -- 172.21.15.60:0/1599508374 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00b8570 msgr2=0x7f96d00ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.693+0000 7f96d53ff700 1 --2- 172.21.15.60:0/1599508374 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00b8570 0x7f96d00ceeb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009850 tx=0x7f96c0009b60 comp rx=0 tx=0).stop 2023-07-13T22:23:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.693+0000 7f96d53ff700 1 -- 172.21.15.60:0/1599508374 shutdown_connections 2023-07-13T22:23:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.693+0000 7f96d53ff700 1 --2- 172.21.15.60:0/1599508374 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00b8570 0x7f96d00ceeb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.693+0000 7f96d53ff700 1 -- 172.21.15.60:0/1599508374 >> 172.21.15.60:0/1599508374 conn(0x7f96d0071600 msgr2=0x7f96d00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.693+0000 7f96d53ff700 1 -- 172.21.15.60:0/1599508374 shutdown_connections 2023-07-13T22:23:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.693+0000 7f96d53ff700 1 -- 172.21.15.60:0/1599508374 wait complete. 2023-07-13T22:23:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.694+0000 7f96d53ff700 1 Processor -- start 2023-07-13T22:23:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.695+0000 7f96d53ff700 1 -- start start 2023-07-13T22:23:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96d53ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d00b8570 0x7f96d0159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96d53ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96d00d15c0 0x7f96d01f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96d53ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00d35c0 0x7f96d01542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96d53ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d01547e0 con 0x7f96d00d15c0 2023-07-13T22:23:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96d53ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d01f2dc0 con 0x7f96d00b8570 2023-07-13T22:23:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96d53ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d01f2520 con 0x7f96d00d35c0 2023-07-13T22:23:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96ce7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96d00d15c0 0x7f96d01f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96ce7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96d00d15c0 0x7f96d01f1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52956/0 (socket says 172.21.15.60:52956) 2023-07-13T22:23:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96ce7fc700 1 -- 172.21.15.60:0/1147439998 learned_addr learned my addr 172.21.15.60:0/1147439998 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96ceffd700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d00b8570 0x7f96d0159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96cf7fe700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00d35c0 0x7f96d01542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96ce7fc700 1 -- 172.21.15.60:0/1147439998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00d35c0 msgr2=0x7f96d01542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96ce7fc700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00d35c0 0x7f96d01542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96ce7fc700 1 -- 172.21.15.60:0/1147439998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d00b8570 msgr2=0x7f96d0159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96ce7fc700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d00b8570 0x7f96d0159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.696+0000 7f96ce7fc700 1 -- 172.21.15.60:0/1147439998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c0009430 con 0x7f96d00d15c0 2023-07-13T22:23:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.697+0000 7f96ce7fc700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96d00d15c0 0x7f96d01f1e40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f96c400c450 tx=0x7f96c400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.697+0000 7f96b77fe700 1 -- 172.21.15.60:0/1147439998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4004c80 con 0x7f96d00d15c0 2023-07-13T22:23:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.697+0000 7f96b77fe700 1 -- 172.21.15.60:0/1147439998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c4004e00 con 0x7f96d00d15c0 2023-07-13T22:23:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.697+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d01f2800 con 0x7f96d00d15c0 2023-07-13T22:23:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.697+0000 7f96b77fe700 1 -- 172.21.15.60:0/1147439998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4007310 con 0x7f96d00d15c0 2023-07-13T22:23:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.697+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d0158000 con 0x7f96d00d15c0 2023-07-13T22:23:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.699+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d0158220 con 0x7f96d00d15c0 2023-07-13T22:23:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.700+0000 7f96b77fe700 1 -- 172.21.15.60:0/1147439998 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96c4007470 con 0x7f96d00d15c0 2023-07-13T22:23:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.700+0000 7f96b77fe700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f96bc06ace0 0x7f96bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.701+0000 7f96b77fe700 1 -- 172.21.15.60:0/1147439998 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f96c4088d70 con 0x7f96d00d15c0 2023-07-13T22:23:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.701+0000 7f96ceffd700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f96bc06ace0 0x7f96bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.701+0000 7f96ceffd700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f96bc06ace0 0x7f96bc06d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f96c00097a0 tx=0x7f96c0008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.705+0000 7f96b77fe700 1 -- 172.21.15.60:0/1147439998 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f96c4054160 con 0x7f96d00d15c0 2023-07-13T22:23:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.992+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f96d0065ee0 con 0x7f96d00d15c0 2023-07-13T22:23:09.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.992+0000 7f96b77fe700 1 -- 172.21.15.60:0/1147439998 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f96c400ce60 con 0x7f96d00d15c0 2023-07-13T22:23:09.976 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:09.976 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:09.977 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-07-13T22:23:09.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.977 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-07-13T22:23:09.977 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw==", 2023-07-13T22:23:09.977 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.977 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:23:09.977 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:23:09.978 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:23:09.978 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.978 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-07-13T22:23:09.978 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g==", 2023-07-13T22:23:09.979 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.979 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:23:09.979 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:23:09.979 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:23:09.979 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.980 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-07-13T22:23:09.980 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA==", 2023-07-13T22:23:09.980 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.980 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:23:09.980 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:23:09.980 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:23:09.980 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.981 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.981 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-07-13T22:23:09.981 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ==", 2023-07-13T22:23:09.981 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.981 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:23:09.981 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:23:09.982 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:23:09.982 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.982 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-07-13T22:23:09.982 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ==", 2023-07-13T22:23:09.982 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.983 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:23:09.983 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:23:09.983 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.983 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-07-13T22:23:09.984 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw==", 2023-07-13T22:23:09.984 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.984 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:23:09.984 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:23:09.984 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.985 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-07-13T22:23:09.985 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q==", 2023-07-13T22:23:09.985 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.985 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:23:09.985 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:23:09.985 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.985 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.986 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-07-13T22:23:09.986 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ==", 2023-07-13T22:23:09.986 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.986 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:23:09.986 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:23:09.986 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.986 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.987 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-07-13T22:23:09.987 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg==", 2023-07-13T22:23:09.987 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.987 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:23:09.987 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:23:09.988 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.988 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.988 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-07-13T22:23:09.988 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA==", 2023-07-13T22:23:09.988 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.988 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:23:09.989 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:23:09.989 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.989 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-07-13T22:23:09.989 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw==", 2023-07-13T22:23:09.989 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.990 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:23:09.990 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:23:09.990 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.990 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-07-13T22:23:09.990 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ==", 2023-07-13T22:23:09.991 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.991 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:23:09.991 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:23:09.991 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.991 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-07-13T22:23:09.992 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w==", 2023-07-13T22:23:09.992 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.992 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:23:09.992 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2023-07-13T22:23:09.992 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-13T22:23:09.992 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.993 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-07-13T22:23:09.993 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw==", 2023-07-13T22:23:09.993 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.993 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:23:09.993 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:23:09.993 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:23:09.993 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:23:09.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.994 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-07-13T22:23:09.994 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg==", 2023-07-13T22:23:09.994 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.994 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:23:09.994 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:23:09.994 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:23:09.995 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:23:09.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.995 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-07-13T22:23:09.995 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q==", 2023-07-13T22:23:09.995 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.996 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:23:09.996 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:23:09.996 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:23:09.996 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:23:09.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.996 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-07-13T22:23:09.997 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBk7CgGBhAAaK88V4CEyrYfZx7wv+XG3g==", 2023-07-13T22:23:09.997 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.997 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:23:09.997 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:23:09.997 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:23:09.997 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.998 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-07-13T22:23:09.998 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ==", 2023-07-13T22:23:09.998 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.998 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:23:09.998 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:23:09.998 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:09.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:09.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:09.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:09.999 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-07-13T22:23:09.999 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ==", 2023-07-13T22:23:09.999 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:23:09.999 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:23:09.999 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:23:09.999 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:23:10.000 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:10.000 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:10.000 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:23:10.000 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:10.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f96bc06ace0 msgr2=0x7f96bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f96bc06ace0 0x7f96bc06d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f96c00097a0 tx=0x7f96c0008070 comp rx=0 tx=0).stop 2023-07-13T22:23:10.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96d00d15c0 msgr2=0x7f96d01f1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96d00d15c0 0x7f96d01f1e40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f96c400c450 tx=0x7f96c400c760 comp rx=0 tx=0).stop 2023-07-13T22:23:10.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 shutdown_connections 2023-07-13T22:23:10.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f96bc06ace0 0x7f96bc06d190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96d00d35c0 0x7f96d01542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d00b8570 0x7f96d0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 --2- 172.21.15.60:0/1147439998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96d00d15c0 0x7f96d01f1e40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 >> 172.21.15.60:0/1147439998 conn(0x7f96d0071600 msgr2=0x7f96d0160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 shutdown_connections 2023-07-13T22:23:10.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:09.995+0000 7f96d53ff700 1 -- 172.21.15.60:0/1147439998 wait complete. 2023-07-13T22:23:10.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:23:10.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.224+0000 7f99893ff700 1 Processor -- start 2023-07-13T22:23:10.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.225+0000 7f99893ff700 1 -- start start 2023-07-13T22:23:10.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f99893ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840b8400 0x7f99840cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f99893ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f99840d3750 0x7f99840cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f99893ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99840cff20 con 0x7f99840cfad0 2023-07-13T22:23:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f99893ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99840d0ac0 con 0x7f99840d3750 2023-07-13T22:23:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f99893ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99840b87d0 con 0x7f99840b8400 2023-07-13T22:23:10.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f997bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f99840d3750 0x7f99840cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f99837fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f99840cfad0 0x7f998415fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51996/0 (socket says 172.21.15.60:51996) 2023-07-13T22:23:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f99837fe700 1 -- 172.21.15.60:0/4273316887 learned_addr learned my addr 172.21.15.60:0/4273316887 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f997bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f99840d3750 0x7f99840cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52960/0 (socket says 172.21.15.60:52960) 2023-07-13T22:23:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.226+0000 7f9982ffd700 1 --2- 172.21.15.60:0/4273316887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840b8400 0x7f99840cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.227+0000 7f997bfff700 1 -- 172.21.15.60:0/4273316887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840b8400 msgr2=0x7f99840cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.227+0000 7f997bfff700 1 --2- 172.21.15.60:0/4273316887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840b8400 0x7f99840cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.227+0000 7f997bfff700 1 -- 172.21.15.60:0/4273316887 >> v1:172.21.15.60:6789/0 conn(0x7f99840cfad0 legacy=0x7f998415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.227+0000 7f997bfff700 1 -- 172.21.15.60:0/4273316887 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840b4eb0 con 0x7f99840d3750 2023-07-13T22:23:10.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.227+0000 7f99827fc700 1 -- 172.21.15.60:0/4273316887 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217217659 0 0) 0x7f997c001490 con 0x7f99840cfad0 2023-07-13T22:23:10.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.227+0000 7f997bfff700 1 --2- 172.21.15.60:0/4273316887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840d3750 0x7f99840cf590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f996c009840 tx=0x7f996c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b8f0ee20c2d97c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.227+0000 7f99827fc700 1 -- 172.21.15.60:0/4273316887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c011830 con 0x7f99840d3750 2023-07-13T22:23:10.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.227+0000 7f99827fc700 1 -- 172.21.15.60:0/4273316887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c011990 con 0x7f99840d3750 2023-07-13T22:23:10.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.227+0000 7f99827fc700 1 -- 172.21.15.60:0/4273316887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c011d20 con 0x7f99840d3750 2023-07-13T22:23:10.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.228+0000 7f99893ff700 1 -- 172.21.15.60:0/4273316887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840d3750 msgr2=0x7f99840cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.228+0000 7f99893ff700 1 --2- 172.21.15.60:0/4273316887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840d3750 0x7f99840cf590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f996c009840 tx=0x7f996c009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.228+0000 7f99893ff700 1 -- 172.21.15.60:0/4273316887 shutdown_connections 2023-07-13T22:23:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.228+0000 7f99893ff700 1 --2- 172.21.15.60:0/4273316887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840b8400 0x7f99840cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.228+0000 7f99893ff700 1 --2- 172.21.15.60:0/4273316887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840d3750 0x7f99840cf590 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.228+0000 7f99893ff700 1 -- 172.21.15.60:0/4273316887 >> 172.21.15.60:0/4273316887 conn(0x7f998406b390 msgr2=0x7f998406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.228+0000 7f99893ff700 1 -- 172.21.15.60:0/4273316887 shutdown_connections 2023-07-13T22:23:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.228+0000 7f99893ff700 1 -- 172.21.15.60:0/4273316887 wait complete. 2023-07-13T22:23:10.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.230+0000 7f99893ff700 1 Processor -- start 2023-07-13T22:23:10.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.231+0000 7f99893ff700 1 -- start start 2023-07-13T22:23:10.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f99893ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840b8400 0x7f9984153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f99893ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99840cfad0 0x7f99840d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f99893ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840d3750 0x7f99840d1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f99893ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9984153880 con 0x7f99840b8400 2023-07-13T22:23:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f99893ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99841e8c70 con 0x7f99840cfad0 2023-07-13T22:23:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f99893ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99840cb450 con 0x7f99840d3750 2023-07-13T22:23:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f9982ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840b8400 0x7f9984153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f9982ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840b8400 0x7f9984153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52964/0 (socket says 172.21.15.60:52964) 2023-07-13T22:23:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f9982ffd700 1 -- 172.21.15.60:0/425779049 learned_addr learned my addr 172.21.15.60:0/425779049 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f997bfff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99840cfad0 0x7f99840d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f99837fe700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840d3750 0x7f99840d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f9982ffd700 1 -- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840d3750 msgr2=0x7f99840d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.232+0000 7f9982ffd700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840d3750 0x7f99840d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f9982ffd700 1 -- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99840cfad0 msgr2=0x7f99840d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f9982ffd700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99840cfad0 0x7f99840d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f9982ffd700 1 -- 172.21.15.60:0/425779049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c0094a0 con 0x7f99840b8400 2023-07-13T22:23:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f997bfff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99840cfad0 0x7f99840d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f99837fe700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840d3750 0x7f99840d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f9982ffd700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840b8400 0x7f9984153340 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f997400e8a0 tx=0x7f997400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f997b7fe700 1 -- 172.21.15.60:0/425779049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974009590 con 0x7f99840b8400 2023-07-13T22:23:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f997b7fe700 1 -- 172.21.15.60:0/425779049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997400eea0 con 0x7f99840b8400 2023-07-13T22:23:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99841e8990 con 0x7f99840b8400 2023-07-13T22:23:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f997b7fe700 1 -- 172.21.15.60:0/425779049 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400c9f0 con 0x7f99840b8400 2023-07-13T22:23:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.233+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99840cb0e0 con 0x7f99840b8400 2023-07-13T22:23:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.234+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998415add0 con 0x7f99840b8400 2023-07-13T22:23:10.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.235+0000 7f997b7fe700 1 -- 172.21.15.60:0/425779049 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f997400cb50 con 0x7f99840b8400 2023-07-13T22:23:10.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.236+0000 7f997b7fe700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f996406ace0 0x7f996406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.236+0000 7f997b7fe700 1 -- 172.21.15.60:0/425779049 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9974089100 con 0x7f99840b8400 2023-07-13T22:23:10.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.236+0000 7f997bfff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f996406ace0 0x7f996406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.237+0000 7f997bfff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f996406ace0 0x7f996406d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f99840d3b60 tx=0x7f996c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.240+0000 7f997b7fe700 1 -- 172.21.15.60:0/425779049 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9974057ef0 con 0x7f99840b8400 2023-07-13T22:23:10.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.448+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f998415aaf0 con 0x7f99840b8400 2023-07-13T22:23:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.449+0000 7f997b7fe700 1 -- 172.21.15.60:0/425779049 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14545 (secure 0 0 0) 0x7f9974057a80 con 0x7f99840b8400 2023-07-13T22:23:10.433 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:10.433 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":45,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:09.454261+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:23:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f996406ace0 msgr2=0x7f996406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f996406ace0 0x7f996406d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f99840d3b60 tx=0x7f996c008030 comp rx=0 tx=0).stop 2023-07-13T22:23:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840b8400 msgr2=0x7f9984153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840b8400 0x7f9984153340 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f997400e8a0 tx=0x7f997400ec60 comp rx=0 tx=0).stop 2023-07-13T22:23:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 shutdown_connections 2023-07-13T22:23:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f996406ace0 0x7f996406d190 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99840b8400 0x7f9984153340 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f99840d3750 0x7f99840d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 --2- 172.21.15.60:0/425779049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99840cfad0 0x7f99840d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 >> 172.21.15.60:0/425779049 conn(0x7f998406b390 msgr2=0x7f998415bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 shutdown_connections 2023-07-13T22:23:10.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.452+0000 7f99893ff700 1 -- 172.21.15.60:0/425779049 wait complete. 2023-07-13T22:23:10.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:23:10.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:23:10.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.676+0000 7f7ad31ff700 1 Processor -- start 2023-07-13T22:23:10.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.676+0000 7f7ad31ff700 1 -- start start 2023-07-13T22:23:10.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.677+0000 7f7ad31ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0cfd50 0x7f7acc0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.677+0000 7f7ad31ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7acc0b1ac0 0x7f7acc0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad31ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7acc0d1870 con 0x7f7acc0b8200 2023-07-13T22:23:10.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad31ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7acc0cf370 con 0x7f7acc0b1ac0 2023-07-13T22:23:10.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad31ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0d12b0 con 0x7f7acc0cfd50 2023-07-13T22:23:10.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad179c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7acc0b1ac0 0x7f7acc0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad179c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7acc0b1ac0 0x7f7acc0d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52968/0 (socket says 172.21.15.60:52968) 2023-07-13T22:23:10.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad179c700 1 -- 172.21.15.60:0/3721958247 learned_addr learned my addr 172.21.15.60:0/3721958247 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:10.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7acbfff700 1 --2- 172.21.15.60:0/3721958247 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0cfd50 0x7f7acc0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad179c700 1 -- 172.21.15.60:0/3721958247 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0cfd50 msgr2=0x7f7acc0d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad179c700 1 --2- 172.21.15.60:0/3721958247 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0cfd50 0x7f7acc0d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad179c700 1 -- 172.21.15.60:0/3721958247 >> v1:172.21.15.105:6789/0 conn(0x7f7acc0b8200 legacy=0x7f7acc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.678+0000 7f7ad179c700 1 -- 172.21.15.60:0/3721958247 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc0d1590 con 0x7f7acc0b1ac0 2023-07-13T22:23:10.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7ad179c700 1 --2- 172.21.15.60:0/3721958247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b1ac0 0x7f7acc0d2900 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009e80 tx=0x7f7ac400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87738ce148803285 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7acb7fe700 1 -- 172.21.15.60:0/3721958247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400e070 con 0x7f7acc0b1ac0 2023-07-13T22:23:10.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7acb7fe700 1 -- 172.21.15.60:0/3721958247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac400b980 con 0x7f7acc0b1ac0 2023-07-13T22:23:10.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7acb7fe700 1 -- 172.21.15.60:0/3721958247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400bd30 con 0x7f7acc0b1ac0 2023-07-13T22:23:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7ad31ff700 1 -- 172.21.15.60:0/3721958247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b1ac0 msgr2=0x7f7acc0d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7ad31ff700 1 --2- 172.21.15.60:0/3721958247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b1ac0 0x7f7acc0d2900 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009e80 tx=0x7f7ac400b330 comp rx=0 tx=0).stop 2023-07-13T22:23:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7ad31ff700 1 -- 172.21.15.60:0/3721958247 shutdown_connections 2023-07-13T22:23:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7ad31ff700 1 --2- 172.21.15.60:0/3721958247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b1ac0 0x7f7acc0d2900 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7ad31ff700 1 --2- 172.21.15.60:0/3721958247 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0cfd50 0x7f7acc0d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.679+0000 7f7ad31ff700 1 -- 172.21.15.60:0/3721958247 >> 172.21.15.60:0/3721958247 conn(0x7f7acc06b2f0 msgr2=0x7f7acc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.680+0000 7f7ad31ff700 1 -- 172.21.15.60:0/3721958247 shutdown_connections 2023-07-13T22:23:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.680+0000 7f7ad31ff700 1 -- 172.21.15.60:0/3721958247 wait complete. 2023-07-13T22:23:10.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.681+0000 7f7ad31ff700 1 Processor -- start 2023-07-13T22:23:10.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.681+0000 7f7ad31ff700 1 -- start start 2023-07-13T22:23:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.682+0000 7f7ad31ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0b1ac0 0x7f7acc15a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.682+0000 7f7ad31ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b8200 0x7f7acc15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.682+0000 7f7ad31ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7acc0cfd50 0x7f7acc15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.682+0000 7f7ad31ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc15d470 con 0x7f7acc0b8200 2023-07-13T22:23:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.682+0000 7f7ad31ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc15da30 con 0x7f7acc0cfd50 2023-07-13T22:23:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.682+0000 7f7ad31ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc15e400 con 0x7f7acc0b1ac0 2023-07-13T22:23:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7acbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b8200 0x7f7acc15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7acbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b8200 0x7f7acc15dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52974/0 (socket says 172.21.15.60:52974) 2023-07-13T22:23:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7acbfff700 1 -- 172.21.15.60:0/2382069446 learned_addr learned my addr 172.21.15.60:0/2382069446 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7ad0f9b700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0b1ac0 0x7f7acc15a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7ad179c700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7acc0cfd50 0x7f7acc15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7acbfff700 1 -- 172.21.15.60:0/2382069446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0b1ac0 msgr2=0x7f7acc15a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7acbfff700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0b1ac0 0x7f7acc15a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7acbfff700 1 -- 172.21.15.60:0/2382069446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7acc0cfd50 msgr2=0x7f7acc15a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7acbfff700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7acc0cfd50 0x7f7acc15a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7acbfff700 1 -- 172.21.15.60:0/2382069446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac4009b30 con 0x7f7acc0b8200 2023-07-13T22:23:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.683+0000 7f7acbfff700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b8200 0x7f7acc15dec0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800d830 tx=0x7f7ab800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.684+0000 7f7ac97fa700 1 -- 172.21.15.60:0/2382069446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80125f0 con 0x7f7acc0b8200 2023-07-13T22:23:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.684+0000 7f7ac97fa700 1 -- 172.21.15.60:0/2382069446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab8012750 con 0x7f7acc0b8200 2023-07-13T22:23:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.684+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc0cc6e0 con 0x7f7acc0b8200 2023-07-13T22:23:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.684+0000 7f7ac97fa700 1 -- 172.21.15.60:0/2382069446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8012ae0 con 0x7f7acc0b8200 2023-07-13T22:23:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.684+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc15d6c0 con 0x7f7acc0b8200 2023-07-13T22:23:10.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.685+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7acc0ccc00 con 0x7f7acc0b8200 2023-07-13T22:23:10.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.686+0000 7f7ac97fa700 1 -- 172.21.15.60:0/2382069446 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ab8012c40 con 0x7f7acc0b8200 2023-07-13T22:23:10.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.687+0000 7f7ac97fa700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ab006ace0 0x7f7ab006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.687+0000 7f7ac97fa700 1 -- 172.21.15.60:0/2382069446 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7ab8089260 con 0x7f7acc0b8200 2023-07-13T22:23:10.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.687+0000 7f7ad0f9b700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ab006ace0 0x7f7ab006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.688+0000 7f7ad0f9b700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ab006ace0 0x7f7ab006d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7acc15fbf0 tx=0x7f7ac0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.691+0000 7f7ac97fa700 1 -- 172.21.15.60:0/2382069446 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7ab80545c0 con 0x7f7acc0b8200 2023-07-13T22:23:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.894+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7acc15d880 con 0x7f7acc0b8200 2023-07-13T22:23:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.894+0000 7f7ac97fa700 1 -- 172.21.15.60:0/2382069446 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7ab8057be0 con 0x7f7acc0b8200 2023-07-13T22:23:10.878 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:23:10.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.896+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ab006ace0 msgr2=0x7f7ab006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.896+0000 7f7ad31ff700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ab006ace0 0x7f7ab006d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7acc15fbf0 tx=0x7f7ac0009040 comp rx=0 tx=0).stop 2023-07-13T22:23:10.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.896+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b8200 msgr2=0x7f7acc15dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.896+0000 7f7ad31ff700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b8200 0x7f7acc15dec0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800d830 tx=0x7f7ab800db40 comp rx=0 tx=0).stop 2023-07-13T22:23:10.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.897+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 shutdown_connections 2023-07-13T22:23:10.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.897+0000 7f7ad31ff700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ab006ace0 0x7f7ab006d190 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.897+0000 7f7ad31ff700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7acc0cfd50 0x7f7acc15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.897+0000 7f7ad31ff700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7acc0b1ac0 0x7f7acc15a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.897+0000 7f7ad31ff700 1 --2- 172.21.15.60:0/2382069446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7acc0b8200 0x7f7acc15dec0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.897+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 >> 172.21.15.60:0/2382069446 conn(0x7f7acc06b2f0 msgr2=0x7f7acc073350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.897+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 shutdown_connections 2023-07-13T22:23:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:10.897+0000 7f7ad31ff700 1 -- 172.21.15.60:0/2382069446 wait complete. 2023-07-13T22:23:10.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:23:11.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.123+0000 7f54459ff700 1 Processor -- start 2023-07-13T22:23:11.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.124+0000 7f54459ff700 1 -- start start 2023-07-13T22:23:11.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f54459ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 0x7f54400d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f54459ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54400b1ac0 0x7f54400d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f54459ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54400d1870 con 0x7f54400b8200 2023-07-13T22:23:11.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f54459ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54400cf370 con 0x7f54400b1ac0 2023-07-13T22:23:11.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f54459ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54400d12b0 con 0x7f54400cfd50 2023-07-13T22:23:11.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f543effd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f54400b8200 0x7f54400cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51998/0 (socket says 172.21.15.60:51998) 2023-07-13T22:23:11.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f543effd700 1 -- 172.21.15.60:0/2443344222 learned_addr learned my addr 172.21.15.60:0/2443344222 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:11.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f5437fff700 1 --2- 172.21.15.60:0/2443344222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 0x7f54400d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f543f7fe700 1 --2- 172.21.15.60:0/2443344222 >> v2:172.21.15.105:3300/0 conn(0x7f54400b1ac0 0x7f54400d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.125+0000 7f543e7fc700 1 -- 172.21.15.60:0/2443344222 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559678610 0 0) 0x7f54400d1870 con 0x7f54400b8200 2023-07-13T22:23:11.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.126+0000 7f543e7fc700 1 -- 172.21.15.60:0/2443344222 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f542c0035b0 con 0x7f54400b8200 2023-07-13T22:23:11.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.126+0000 7f543e7fc700 1 -- 172.21.15.60:0/2443344222 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1943797490 0 0) 0x7f542c0035b0 con 0x7f54400b8200 2023-07-13T22:23:11.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.126+0000 7f5437fff700 1 -- 172.21.15.60:0/2443344222 >> v2:172.21.15.105:3300/0 conn(0x7f54400b1ac0 msgr2=0x7f54400d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:11.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.126+0000 7f5437fff700 1 --2- 172.21.15.60:0/2443344222 >> v2:172.21.15.105:3300/0 conn(0x7f54400b1ac0 0x7f54400d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.126+0000 7f5437fff700 1 -- 172.21.15.60:0/2443344222 >> v1:172.21.15.60:6789/0 conn(0x7f54400b8200 legacy=0x7f54400cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.126+0000 7f5437fff700 1 -- 172.21.15.60:0/2443344222 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400d1590 con 0x7f54400cfd50 2023-07-13T22:23:11.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.126+0000 7f5437fff700 1 --2- 172.21.15.60:0/2443344222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 0x7f54400d23c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5428009870 tx=0x7f5428009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104b97cee4892134 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:11.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f543e7fc700 1 -- 172.21.15.60:0/2443344222 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430001670 con 0x7f54400cfd50 2023-07-13T22:23:11.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f543e7fc700 1 -- 172.21.15.60:0/2443344222 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428004030 con 0x7f54400cfd50 2023-07-13T22:23:11.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f543e7fc700 1 -- 172.21.15.60:0/2443344222 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54280043c0 con 0x7f54400cfd50 2023-07-13T22:23:11.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f54459ff700 1 -- 172.21.15.60:0/2443344222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 msgr2=0x7f54400d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f54459ff700 1 --2- 172.21.15.60:0/2443344222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 0x7f54400d23c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5428009870 tx=0x7f5428009b80 comp rx=0 tx=0).stop 2023-07-13T22:23:11.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f54459ff700 1 -- 172.21.15.60:0/2443344222 shutdown_connections 2023-07-13T22:23:11.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f54459ff700 1 --2- 172.21.15.60:0/2443344222 >> v2:172.21.15.105:3300/0 conn(0x7f54400b1ac0 0x7f54400d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f54459ff700 1 --2- 172.21.15.60:0/2443344222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 0x7f54400d23c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f54459ff700 1 -- 172.21.15.60:0/2443344222 >> 172.21.15.60:0/2443344222 conn(0x7f544006b2f0 msgr2=0x7f544006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:11.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f54459ff700 1 -- 172.21.15.60:0/2443344222 shutdown_connections 2023-07-13T22:23:11.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.127+0000 7f54459ff700 1 -- 172.21.15.60:0/2443344222 wait complete. 2023-07-13T22:23:11.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.129+0000 7f54459ff700 1 Processor -- start 2023-07-13T22:23:11.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.129+0000 7f54459ff700 1 -- start start 2023-07-13T22:23:11.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.130+0000 7f54459ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54400b1ac0 0x7f544015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.130+0000 7f54459ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54400b8200 0x7f544015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.130+0000 7f54459ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 0x7f5440154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.130+0000 7f54459ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54401f5f60 con 0x7f54400b1ac0 2023-07-13T22:23:11.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.130+0000 7f54459ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544015ef80 con 0x7f54400b8200 2023-07-13T22:23:11.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.130+0000 7f54459ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544015f790 con 0x7f54400cfd50 2023-07-13T22:23:11.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.130+0000 7f543effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54400b1ac0 0x7f544015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f543effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54400b1ac0 0x7f544015f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52980/0 (socket says 172.21.15.60:52980) 2023-07-13T22:23:11.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f543effd700 1 -- 172.21.15.60:0/2309106511 learned_addr learned my addr 172.21.15.60:0/2309106511 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:11.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f543f7fe700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 0x7f5440154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f5437fff700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54400b8200 0x7f544015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f543effd700 1 -- 172.21.15.60:0/2309106511 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 msgr2=0x7f5440154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f543effd700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 0x7f5440154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f543effd700 1 -- 172.21.15.60:0/2309106511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54400b8200 msgr2=0x7f544015ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f543effd700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54400b8200 0x7f544015ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f543effd700 1 -- 172.21.15.60:0/2309106511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54280094a0 con 0x7f54400b1ac0 2023-07-13T22:23:11.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.131+0000 7f543effd700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54400b1ac0 0x7f544015f1b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f543000c780 tx=0x7f543000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:11.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.132+0000 7f543cff9700 1 -- 172.21.15.60:0/2309106511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430004c80 con 0x7f54400b1ac0 2023-07-13T22:23:11.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.132+0000 7f543cff9700 1 -- 172.21.15.60:0/2309106511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430004de0 con 0x7f54400b1ac0 2023-07-13T22:23:11.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.132+0000 7f543cff9700 1 -- 172.21.15.60:0/2309106511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300072f0 con 0x7f54400b1ac0 2023-07-13T22:23:11.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.132+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54401f61b0 con 0x7f54400b1ac0 2023-07-13T22:23:11.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.132+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54401f6770 con 0x7f54400b1ac0 2023-07-13T22:23:11.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.133+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54400d2a30 con 0x7f54400b1ac0 2023-07-13T22:23:11.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.134+0000 7f543cff9700 1 -- 172.21.15.60:0/2309106511 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5430007450 con 0x7f54400b1ac0 2023-07-13T22:23:11.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.135+0000 7f543cff9700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f542006ace0 0x7f542006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.135+0000 7f543cff9700 1 -- 172.21.15.60:0/2309106511 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5430088f60 con 0x7f54400b1ac0 2023-07-13T22:23:11.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.135+0000 7f5437fff700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f542006ace0 0x7f542006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.135+0000 7f5437fff700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f542006ace0 0x7f542006d190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f54400d0140 tx=0x7f5428008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.139+0000 7f543cff9700 1 -- 172.21.15.60:0/2309106511 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f54300542d0 con 0x7f54400b1ac0 2023-07-13T22:23:11.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.349+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5440066700 con 0x7f54400b1ac0 2023-07-13T22:23:11.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.349+0000 7f543cff9700 1 -- 172.21.15.60:0/2309106511 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5430007700 con 0x7f54400b1ac0 2023-07-13T22:23:11.333 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:23:11.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.351+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f542006ace0 msgr2=0x7f542006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.351+0000 7f54459ff700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f542006ace0 0x7f542006d190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f54400d0140 tx=0x7f5428008030 comp rx=0 tx=0).stop 2023-07-13T22:23:11.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.351+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54400b1ac0 msgr2=0x7f544015f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.351+0000 7f54459ff700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54400b1ac0 0x7f544015f1b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f543000c780 tx=0x7f543000ca90 comp rx=0 tx=0).stop 2023-07-13T22:23:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.352+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 shutdown_connections 2023-07-13T22:23:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.352+0000 7f54459ff700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f542006ace0 0x7f542006d190 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.352+0000 7f54459ff700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54400b1ac0 0x7f544015f1b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.352+0000 7f54459ff700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54400cfd50 0x7f5440154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.352+0000 7f54459ff700 1 --2- 172.21.15.60:0/2309106511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54400b8200 0x7f544015ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.352+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 >> 172.21.15.60:0/2309106511 conn(0x7f544006b2f0 msgr2=0x7f54400c8770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.352+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 shutdown_connections 2023-07-13T22:23:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.352+0000 7f54459ff700 1 -- 172.21.15.60:0/2309106511 wait complete. 2023-07-13T22:23:11.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:23:11.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.576+0000 7f7fb89ff700 1 Processor -- start 2023-07-13T22:23:11.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.576+0000 7f7fb89ff700 1 -- start start 2023-07-13T22:23:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.577+0000 7f7fb89ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7fb40b8400 0x7f7fb40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.577+0000 7f7fb89ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40cfca0 0x7f7fb40d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.577+0000 7f7fb89ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb40d0ac0 con 0x7f7fb40d3750 2023-07-13T22:23:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.577+0000 7f7fb89ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb40b87d0 con 0x7f7fb40b8400 2023-07-13T22:23:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.577+0000 7f7fb89ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40b4eb0 con 0x7f7fb40cfca0 2023-07-13T22:23:11.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.578+0000 7f7fb259c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7fb40b8400 0x7f7fb40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.578+0000 7f7fb259c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7fb40b8400 0x7f7fb40cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:48816/0 (socket says 172.21.15.60:48816) 2023-07-13T22:23:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.578+0000 7f7fb259c700 1 -- 172.21.15.60:0/4068553985 learned_addr learned my addr 172.21.15.60:0/4068553985 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.578+0000 7f7fb2d9d700 1 --2- 172.21.15.60:0/4068553985 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40cfca0 0x7f7fb40d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.578+0000 7f7fb259c700 1 -- 172.21.15.60:0/4068553985 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40cfca0 msgr2=0x7f7fb40d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.578+0000 7f7fb259c700 1 --2- 172.21.15.60:0/4068553985 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40cfca0 0x7f7fb40d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.578+0000 7f7fb259c700 1 -- 172.21.15.60:0/4068553985 >> v1:172.21.15.105:6789/0 conn(0x7f7fb40d3750 legacy=0x7f7fb40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.578+0000 7f7fb259c700 1 -- 172.21.15.60:0/4068553985 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40b5190 con 0x7f7fb40b8400 2023-07-13T22:23:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.579+0000 7f7fb259c700 1 --2- 172.21.15.60:0/4068553985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 0x7f7fb40cd240 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40098f0 tx=0x7f7fa4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e86c8cb2bc8f4a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.579+0000 7f7fb159a700 1 -- 172.21.15.60:0/4068553985 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4011830 con 0x7f7fb40b8400 2023-07-13T22:23:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.579+0000 7f7fb159a700 1 -- 172.21.15.60:0/4068553985 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa4011990 con 0x7f7fb40b8400 2023-07-13T22:23:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.579+0000 7f7fb159a700 1 -- 172.21.15.60:0/4068553985 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4011d20 con 0x7f7fb40b8400 2023-07-13T22:23:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.579+0000 7f7fb89ff700 1 -- 172.21.15.60:0/4068553985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 msgr2=0x7f7fb40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.579+0000 7f7fb89ff700 1 --2- 172.21.15.60:0/4068553985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 0x7f7fb40cd240 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40098f0 tx=0x7f7fa4009c00 comp rx=0 tx=0).stop 2023-07-13T22:23:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.580+0000 7f7fb89ff700 1 -- 172.21.15.60:0/4068553985 shutdown_connections 2023-07-13T22:23:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.580+0000 7f7fb89ff700 1 --2- 172.21.15.60:0/4068553985 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40cfca0 0x7f7fb40d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.580+0000 7f7fb89ff700 1 --2- 172.21.15.60:0/4068553985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 0x7f7fb40cd240 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.580+0000 7f7fb89ff700 1 -- 172.21.15.60:0/4068553985 >> 172.21.15.60:0/4068553985 conn(0x7f7fb406b390 msgr2=0x7f7fb406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.580+0000 7f7fb89ff700 1 -- 172.21.15.60:0/4068553985 shutdown_connections 2023-07-13T22:23:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.580+0000 7f7fb89ff700 1 -- 172.21.15.60:0/4068553985 wait complete. 2023-07-13T22:23:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.581+0000 7f7fb89ff700 1 Processor -- start 2023-07-13T22:23:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.582+0000 7f7fb89ff700 1 -- start start 2023-07-13T22:23:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb89ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 0x7f7fb4150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb89ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb40cfca0 0x7f7fb40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb89ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40d3750 0x7f7fb40cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb89ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40cb550 con 0x7f7fb40cfca0 2023-07-13T22:23:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb89ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40ca660 con 0x7f7fb40b8400 2023-07-13T22:23:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb89ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40ca940 con 0x7f7fb40d3750 2023-07-13T22:23:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb40cfca0 0x7f7fb40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb40cfca0 0x7f7fb40cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52984/0 (socket says 172.21.15.60:52984) 2023-07-13T22:23:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb1d9b700 1 -- 172.21.15.60:0/3085924165 learned_addr learned my addr 172.21.15.60:0/3085924165 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb259c700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 0x7f7fb4150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb2d9d700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40d3750 0x7f7fb40cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb1d9b700 1 -- 172.21.15.60:0/3085924165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40d3750 msgr2=0x7f7fb40cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb1d9b700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40d3750 0x7f7fb40cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb1d9b700 1 -- 172.21.15.60:0/3085924165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 msgr2=0x7f7fb4150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb1d9b700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 0x7f7fb4150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb1d9b700 1 -- 172.21.15.60:0/3085924165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa4009550 con 0x7f7fb40cfca0 2023-07-13T22:23:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.583+0000 7f7fb259c700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 0x7f7fb4150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.584+0000 7f7fb1d9b700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb40cfca0 0x7f7fb40cc040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800c4b0 tx=0x7f7fa800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.584+0000 7f7fa2ffd700 1 -- 172.21.15.60:0/3085924165 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80052d0 con 0x7f7fb40cfca0 2023-07-13T22:23:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.584+0000 7f7fa2ffd700 1 -- 172.21.15.60:0/3085924165 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa8005430 con 0x7f7fb40cfca0 2023-07-13T22:23:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.584+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb40cac20 con 0x7f7fb40cfca0 2023-07-13T22:23:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.584+0000 7f7fa2ffd700 1 -- 172.21.15.60:0/3085924165 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80038c0 con 0x7f7fb40cfca0 2023-07-13T22:23:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.584+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb40cb1e0 con 0x7f7fb40cfca0 2023-07-13T22:23:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.585+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb40d1f60 con 0x7f7fb40cfca0 2023-07-13T22:23:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.586+0000 7f7fa2ffd700 1 -- 172.21.15.60:0/3085924165 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7fa8003a20 con 0x7f7fb40cfca0 2023-07-13T22:23:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.587+0000 7f7fa2ffd700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9406ace0 0x7f7f9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.587+0000 7f7fa2ffd700 1 -- 172.21.15.60:0/3085924165 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7fa8088500 con 0x7f7fb40cfca0 2023-07-13T22:23:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.587+0000 7f7fb259c700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9406ace0 0x7f7f9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.588+0000 7f7fb259c700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9406ace0 0x7f7f9406d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4014040 tx=0x7f7fa4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.591+0000 7f7fa2ffd700 1 -- 172.21.15.60:0/3085924165 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7fa8053860 con 0x7f7fb40cfca0 2023-07-13T22:23:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.806+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7fb40d24b0 con 0x7f7fb40cfca0 2023-07-13T22:23:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.807+0000 7f7fa2ffd700 1 -- 172.21.15.60:0/3085924165 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7fa8056e80 con 0x7f7fb40cfca0 2023-07-13T22:23:11.791 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:11.791 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:23:11.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:11.791 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:23:11.791 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:23:11.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:11.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:23:11.792 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:23:11.792 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:23:11.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:11.792 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:23:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9406ace0 msgr2=0x7f7f9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9406ace0 0x7f7f9406d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4014040 tx=0x7f7fa4008030 comp rx=0 tx=0).stop 2023-07-13T22:23:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb40cfca0 msgr2=0x7f7fb40cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb40cfca0 0x7f7fb40cc040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800c4b0 tx=0x7f7fa800c7c0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 shutdown_connections 2023-07-13T22:23:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f9406ace0 0x7f7f9406d190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fb40d3750 0x7f7fb40cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fb40b8400 0x7f7fb4150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 --2- 172.21.15.60:0/3085924165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fb40cfca0 0x7f7fb40cc040 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.809+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 >> 172.21.15.60:0/3085924165 conn(0x7f7fb406b390 msgr2=0x7f7fb415af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.810+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 shutdown_connections 2023-07-13T22:23:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:11.810+0000 7f7fb89ff700 1 -- 172.21.15.60:0/3085924165 wait complete. 2023-07-13T22:23:11.811 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:23:11.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:23:12.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.035+0000 7f39459ff700 1 Processor -- start 2023-07-13T22:23:12.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.036+0000 7f39459ff700 1 -- start start 2023-07-13T22:23:12.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.037+0000 7f39459ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f394015c5f0 0x7f394006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.037+0000 7f39459ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394006e5f0 con 0x7f394006f8c0 2023-07-13T22:23:12.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.037+0000 7f39459ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394006f430 con 0x7f394006b6f0 2023-07-13T22:23:12.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.037+0000 7f39459ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394006e820 con 0x7f394015c5f0 2023-07-13T22:23:12.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.037+0000 7f393effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f394015c5f0 0x7f394006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.037+0000 7f393effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f394015c5f0 0x7f394006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52986/0 (socket says 172.21.15.60:52986) 2023-07-13T22:23:12.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.037+0000 7f393effd700 1 -- 172.21.15.60:0/4089821177 learned_addr learned my addr 172.21.15.60:0/4089821177 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:12.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.037+0000 7f3937fff700 1 --1- 172.21.15.60:0/4089821177 >> v1:172.21.15.60:6789/0 conn(0x7f394006f8c0 0x7f394006afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52002/0 (socket says 172.21.15.60:52002) 2023-07-13T22:23:12.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.038+0000 7f393effd700 1 -- 172.21.15.60:0/4089821177 >> v1:172.21.15.105:6789/0 conn(0x7f394006b6f0 legacy=0x7f39400ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.038+0000 7f393effd700 1 -- 172.21.15.60:0/4089821177 >> v1:172.21.15.60:6789/0 conn(0x7f394006f8c0 legacy=0x7f394006afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.038+0000 7f393e7fc700 1 -- 172.21.15.60:0/4089821177 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628961428 0 0) 0x7f394006e5f0 con 0x7f394006f8c0 2023-07-13T22:23:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.038+0000 7f393effd700 1 -- 172.21.15.60:0/4089821177 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394006eb00 con 0x7f394015c5f0 2023-07-13T22:23:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.038+0000 7f393effd700 1 --2- 172.21.15.60:0/4089821177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394015c5f0 0x7f394006e0b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3930006fd0 tx=0x7f3930009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1be8d18634b1618 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.038+0000 7f393e7fc700 1 -- 172.21.15.60:0/4089821177 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930005e30 con 0x7f394015c5f0 2023-07-13T22:23:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.038+0000 7f393e7fc700 1 -- 172.21.15.60:0/4089821177 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3930006030 con 0x7f394015c5f0 2023-07-13T22:23:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.038+0000 7f393e7fc700 1 -- 172.21.15.60:0/4089821177 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39300063e0 con 0x7f394015c5f0 2023-07-13T22:23:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.039+0000 7f39459ff700 1 -- 172.21.15.60:0/4089821177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394015c5f0 msgr2=0x7f394006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.039+0000 7f39459ff700 1 --2- 172.21.15.60:0/4089821177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394015c5f0 0x7f394006e0b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3930006fd0 tx=0x7f3930009bd0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.039+0000 7f39459ff700 1 -- 172.21.15.60:0/4089821177 shutdown_connections 2023-07-13T22:23:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.039+0000 7f39459ff700 1 --2- 172.21.15.60:0/4089821177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394015c5f0 0x7f394006e0b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.039+0000 7f39459ff700 1 -- 172.21.15.60:0/4089821177 >> 172.21.15.60:0/4089821177 conn(0x7f3940099c30 msgr2=0x7f39400d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.039+0000 7f39459ff700 1 -- 172.21.15.60:0/4089821177 shutdown_connections 2023-07-13T22:23:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.039+0000 7f39459ff700 1 -- 172.21.15.60:0/4089821177 wait complete. 2023-07-13T22:23:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.041+0000 7f39459ff700 1 Processor -- start 2023-07-13T22:23:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.041+0000 7f39459ff700 1 -- start start 2023-07-13T22:23:12.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f39459ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f394006b6f0 0x7f3940159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f39459ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006f8c0 0x7f39400ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f39459ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f394015c5f0 0x7f39400b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f39459ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39400c9150 con 0x7f394006f8c0 2023-07-13T22:23:12.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f39459ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39401596b0 con 0x7f394006b6f0 2023-07-13T22:23:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f39459ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39400c8540 con 0x7f394015c5f0 2023-07-13T22:23:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f3937fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006f8c0 0x7f39400ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f3937fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006f8c0 0x7f39400ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52990/0 (socket says 172.21.15.60:52990) 2023-07-13T22:23:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f3937fff700 1 -- 172.21.15.60:0/3126307897 learned_addr learned my addr 172.21.15.60:0/3126307897 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f3937fff700 1 -- 172.21.15.60:0/3126307897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f394015c5f0 msgr2=0x7f39400b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f393effd700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f394006b6f0 0x7f3940159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f3937fff700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f394015c5f0 0x7f39400b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f3937fff700 1 -- 172.21.15.60:0/3126307897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f394006b6f0 msgr2=0x7f3940159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f3937fff700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f394006b6f0 0x7f3940159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f3937fff700 1 -- 172.21.15.60:0/3126307897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3930009500 con 0x7f394006f8c0 2023-07-13T22:23:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.042+0000 7f393effd700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f394006b6f0 0x7f3940159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.043+0000 7f3937fff700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006f8c0 0x7f39400ccf50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f392800c540 tx=0x7f392800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.043+0000 7f393cff9700 1 -- 172.21.15.60:0/3126307897 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928004c80 con 0x7f394006f8c0 2023-07-13T22:23:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.043+0000 7f393cff9700 1 -- 172.21.15.60:0/3126307897 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3928004de0 con 0x7f394006f8c0 2023-07-13T22:23:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.043+0000 7f393cff9700 1 -- 172.21.15.60:0/3126307897 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928007310 con 0x7f394006f8c0 2023-07-13T22:23:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.043+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39400c8820 con 0x7f394006f8c0 2023-07-13T22:23:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.043+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39400c8de0 con 0x7f394006f8c0 2023-07-13T22:23:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.044+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39400c92c0 con 0x7f394006f8c0 2023-07-13T22:23:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.045+0000 7f393cff9700 1 -- 172.21.15.60:0/3126307897 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3928007470 con 0x7f394006f8c0 2023-07-13T22:23:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.046+0000 7f393cff9700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f392006ace0 0x7f392006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.046+0000 7f393cff9700 1 -- 172.21.15.60:0/3126307897 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3928088fa0 con 0x7f394006f8c0 2023-07-13T22:23:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.046+0000 7f393effd700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f392006ace0 0x7f392006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.047+0000 7f393effd700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f392006ace0 0x7f392006d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3930013040 tx=0x7f3930008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.051+0000 7f393cff9700 1 -- 172.21.15.60:0/3126307897 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3928054300 con 0x7f394006f8c0 2023-07-13T22:23:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:12.263+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f394004e210 con 0x7f394006f8c0 2023-07-13T22:23:13.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.088+0000 7f393cff9700 1 -- 172.21.15.60:0/3126307897 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3928057920 con 0x7f394006f8c0 2023-07-13T22:23:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.095+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f392006ace0 msgr2=0x7f392006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.095+0000 7f39459ff700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f392006ace0 0x7f392006d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3930013040 tx=0x7f3930008070 comp rx=0 tx=0).stop 2023-07-13T22:23:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.095+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006f8c0 msgr2=0x7f39400ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.095+0000 7f39459ff700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006f8c0 0x7f39400ccf50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f392800c540 tx=0x7f392800c900 comp rx=0 tx=0).stop 2023-07-13T22:23:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.096+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 shutdown_connections 2023-07-13T22:23:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.096+0000 7f39459ff700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f392006ace0 0x7f392006d190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.096+0000 7f39459ff700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f394015c5f0 0x7f39400b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.096+0000 7f39459ff700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f394006b6f0 0x7f3940159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.096+0000 7f39459ff700 1 --2- 172.21.15.60:0/3126307897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006f8c0 0x7f39400ccf50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.096+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 >> 172.21.15.60:0/3126307897 conn(0x7f3940099c30 msgr2=0x7f39400ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.097+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 shutdown_connections 2023-07-13T22:23:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.097+0000 7f39459ff700 1 -- 172.21.15.60:0/3126307897 wait complete. 2023-07-13T22:23:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:23:13.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:23:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.332+0000 7fb697fff700 1 Processor -- start 2023-07-13T22:23:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.333+0000 7fb697fff700 1 -- start start 2023-07-13T22:23:13.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb697fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb6900d1f00 0x7fb6900b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb697fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900cd8a0 con 0x7fb69015c360 2023-07-13T22:23:13.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb697fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900d1360 con 0x7fb6900d25f0 2023-07-13T22:23:13.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb697fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900d10f0 con 0x7fb6900d1f00 2023-07-13T22:23:13.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb69559a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb6900d1f00 0x7fb6900b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb69559a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb6900d1f00 0x7fb6900b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53000/0 (socket says 172.21.15.60:53000) 2023-07-13T22:23:13.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb69559a700 1 -- 172.21.15.60:0/1318934544 learned_addr learned my addr 172.21.15.60:0/1318934544 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:13.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb69559a700 1 -- 172.21.15.60:0/1318934544 >> v1:172.21.15.105:6789/0 conn(0x7fb6900d25f0 legacy=0x7fb6900d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb69559a700 1 -- 172.21.15.60:0/1318934544 >> v1:172.21.15.60:6789/0 conn(0x7fb69015c360 legacy=0x7fb6900d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.334+0000 7fb69559a700 1 -- 172.21.15.60:0/1318934544 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6900b5e80 con 0x7fb6900d1f00 2023-07-13T22:23:13.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.335+0000 7fb69559a700 1 --2- 172.21.15.60:0/1318934544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 0x7fb6900b5940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb6800098f0 tx=0x7fb680009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4284bb74a64c1801 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:13.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.335+0000 7fb694d99700 1 -- 172.21.15.60:0/1318934544 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68000b8a0 con 0x7fb6900d1f00 2023-07-13T22:23:13.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.335+0000 7fb694d99700 1 -- 172.21.15.60:0/1318934544 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68000ba00 con 0x7fb6900d1f00 2023-07-13T22:23:13.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.335+0000 7fb694d99700 1 -- 172.21.15.60:0/1318934544 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68000bd90 con 0x7fb6900d1f00 2023-07-13T22:23:13.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.335+0000 7fb697fff700 1 -- 172.21.15.60:0/1318934544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 msgr2=0x7fb6900b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.335+0000 7fb697fff700 1 --2- 172.21.15.60:0/1318934544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 0x7fb6900b5940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb6800098f0 tx=0x7fb680009c00 comp rx=0 tx=0).stop 2023-07-13T22:23:13.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.336+0000 7fb697fff700 1 -- 172.21.15.60:0/1318934544 shutdown_connections 2023-07-13T22:23:13.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.336+0000 7fb697fff700 1 --2- 172.21.15.60:0/1318934544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 0x7fb6900b5940 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.336+0000 7fb697fff700 1 -- 172.21.15.60:0/1318934544 >> 172.21.15.60:0/1318934544 conn(0x7fb69006b330 msgr2=0x7fb69006b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.336+0000 7fb697fff700 1 -- 172.21.15.60:0/1318934544 shutdown_connections 2023-07-13T22:23:13.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.336+0000 7fb697fff700 1 -- 172.21.15.60:0/1318934544 wait complete. 2023-07-13T22:23:13.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.337+0000 7fb697fff700 1 Processor -- start 2023-07-13T22:23:13.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.338+0000 7fb697fff700 1 -- start start 2023-07-13T22:23:13.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.338+0000 7fb697fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 0x7fb69014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.338+0000 7fb697fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6900d25f0 0x7fb69015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.338+0000 7fb697fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb69015c360 0x7fb6900b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.338+0000 7fb697fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900c6c50 con 0x7fb6900d1f00 2023-07-13T22:23:13.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.338+0000 7fb697fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69015d300 con 0x7fb6900d25f0 2023-07-13T22:23:13.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.338+0000 7fb697fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69015d5e0 con 0x7fb69015c360 2023-07-13T22:23:13.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb695d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 0x7fb69014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb695d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 0x7fb69014d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53010/0 (socket says 172.21.15.60:53010) 2023-07-13T22:23:13.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb695d9b700 1 -- 172.21.15.60:0/3165218124 learned_addr learned my addr 172.21.15.60:0/3165218124 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:13.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb69659c700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb69015c360 0x7fb6900b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb69559a700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6900d25f0 0x7fb69015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb695d9b700 1 -- 172.21.15.60:0/3165218124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb69015c360 msgr2=0x7fb6900b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb695d9b700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb69015c360 0x7fb6900b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb695d9b700 1 -- 172.21.15.60:0/3165218124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6900d25f0 msgr2=0x7fb69015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb695d9b700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6900d25f0 0x7fb69015c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb695d9b700 1 -- 172.21.15.60:0/3165218124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb680009550 con 0x7fb6900d1f00 2023-07-13T22:23:13.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.339+0000 7fb695d9b700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 0x7fb69014d8e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c450 tx=0x7fb68c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:13.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.340+0000 7fb6867fc700 1 -- 172.21.15.60:0/3165218124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0052d0 con 0x7fb6900d1f00 2023-07-13T22:23:13.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.340+0000 7fb6867fc700 1 -- 172.21.15.60:0/3165218124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c005430 con 0x7fb6900d1f00 2023-07-13T22:23:13.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.340+0000 7fb6867fc700 1 -- 172.21.15.60:0/3165218124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0038c0 con 0x7fb6900d1f00 2023-07-13T22:23:13.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.340+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69015d8c0 con 0x7fb6900d1f00 2023-07-13T22:23:13.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.340+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69015e780 con 0x7fb6900d1f00 2023-07-13T22:23:13.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.341+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69015db00 con 0x7fb6900d1f00 2023-07-13T22:23:13.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.342+0000 7fb6867fc700 1 -- 172.21.15.60:0/3165218124 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb68c003a20 con 0x7fb6900d1f00 2023-07-13T22:23:13.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.343+0000 7fb6867fc700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb67406ace0 0x7fb67406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.344+0000 7fb6867fc700 1 -- 172.21.15.60:0/3165218124 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fb68c088560 con 0x7fb6900d1f00 2023-07-13T22:23:13.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.344+0000 7fb69559a700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb67406ace0 0x7fb67406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.344+0000 7fb69559a700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb67406ace0 0x7fb67406d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb680009c40 tx=0x7fb680008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:13.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.348+0000 7fb6867fc700 1 -- 172.21.15.60:0/3165218124 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb69015db00 con 0x7fb6900d1f00 2023-07-13T22:23:13.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:13.557+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb6900cd8a0 con 0x7fb6900d1f00 2023-07-13T22:23:14.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.096+0000 7fb6867fc700 1 -- 172.21.15.60:0/3165218124 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb6900cd8a0 con 0x7fb6900d1f00 2023-07-13T22:23:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.102+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb67406ace0 msgr2=0x7fb67406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.102+0000 7fb697fff700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb67406ace0 0x7fb67406d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb680009c40 tx=0x7fb680008030 comp rx=0 tx=0).stop 2023-07-13T22:23:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.102+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 msgr2=0x7fb69014d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.102+0000 7fb697fff700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 0x7fb69014d8e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00c450 tx=0x7fb68c00c760 comp rx=0 tx=0).stop 2023-07-13T22:23:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.104+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 shutdown_connections 2023-07-13T22:23:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.104+0000 7fb697fff700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb67406ace0 0x7fb67406d190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.104+0000 7fb697fff700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6900d1f00 0x7fb69014d8e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.104+0000 7fb697fff700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb69015c360 0x7fb6900b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.104+0000 7fb697fff700 1 --2- 172.21.15.60:0/3165218124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6900d25f0 0x7fb69015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.104+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 >> 172.21.15.60:0/3165218124 conn(0x7fb69006b330 msgr2=0x7fb6900b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.106+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 shutdown_connections 2023-07-13T22:23:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.111+0000 7fb697fff700 1 -- 172.21.15.60:0/3165218124 wait complete. 2023-07-13T22:23:14.102 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:23:14.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:23:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.344+0000 7fbc815ff700 1 Processor -- start 2023-07-13T22:23:14.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.344+0000 7fbc815ff700 1 -- start start 2023-07-13T22:23:14.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.345+0000 7fbc815ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbc7c0cfd50 0x7fbc7c0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.345+0000 7fbc815ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc7c0b1ac0 0x7fbc7c0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc815ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7c0d1870 con 0x7fbc7c0b8200 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc815ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc7c0cf370 con 0x7fbc7c0cfd50 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc815ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc7c0d12b0 con 0x7fbc7c0b1ac0 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc7affd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbc7c0b8200 0x7fbc7c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52028/0 (socket says 172.21.15.60:52028) 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc7affd700 1 -- 172.21.15.60:0/1694726160 learned_addr learned my addr 172.21.15.60:0/1694726160 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc73fff700 1 --2- 172.21.15.60:0/1694726160 >> v2:172.21.15.60:3300/0 conn(0x7fbc7c0cfd50 0x7fbc7c0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc7b7fe700 1 --2- 172.21.15.60:0/1694726160 >> v2:172.21.15.105:3300/0 conn(0x7fbc7c0b1ac0 0x7fbc7c0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc7a7fc700 1 -- 172.21.15.60:0/1694726160 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488287169 0 0) 0x7fbc7c0d1870 con 0x7fbc7c0b8200 2023-07-13T22:23:14.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc7a7fc700 1 -- 172.21.15.60:0/1694726160 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc680035b0 con 0x7fbc7c0b8200 2023-07-13T22:23:14.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc73fff700 1 -- 172.21.15.60:0/1694726160 >> v2:172.21.15.105:3300/0 conn(0x7fbc7c0b1ac0 msgr2=0x7fbc7c0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:14.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc73fff700 1 --2- 172.21.15.60:0/1694726160 >> v2:172.21.15.105:3300/0 conn(0x7fbc7c0b1ac0 0x7fbc7c0d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc73fff700 1 -- 172.21.15.60:0/1694726160 >> v1:172.21.15.60:6789/0 conn(0x7fbc7c0b8200 legacy=0x7fbc7c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.346+0000 7fbc73fff700 1 -- 172.21.15.60:0/1694726160 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c0d1590 con 0x7fbc7c0cfd50 2023-07-13T22:23:14.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.347+0000 7fbc73fff700 1 --2- 172.21.15.60:0/1694726160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0cfd50 0x7fbc7c0d23c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0b8d70 tx=0x7fbc64009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e41ae71bb76288 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:14.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.347+0000 7fbc7a7fc700 1 -- 172.21.15.60:0/1694726160 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011830 con 0x7fbc7c0cfd50 2023-07-13T22:23:14.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.347+0000 7fbc7a7fc700 1 -- 172.21.15.60:0/1694726160 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc64011990 con 0x7fbc7c0cfd50 2023-07-13T22:23:14.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.347+0000 7fbc7a7fc700 1 -- 172.21.15.60:0/1694726160 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011d20 con 0x7fbc7c0cfd50 2023-07-13T22:23:14.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.347+0000 7fbc815ff700 1 -- 172.21.15.60:0/1694726160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0cfd50 msgr2=0x7fbc7c0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.347+0000 7fbc815ff700 1 --2- 172.21.15.60:0/1694726160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0cfd50 0x7fbc7c0d23c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0b8d70 tx=0x7fbc64009b20 comp rx=0 tx=0).stop 2023-07-13T22:23:14.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.348+0000 7fbc815ff700 1 -- 172.21.15.60:0/1694726160 shutdown_connections 2023-07-13T22:23:14.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.348+0000 7fbc815ff700 1 --2- 172.21.15.60:0/1694726160 >> v2:172.21.15.105:3300/0 conn(0x7fbc7c0b1ac0 0x7fbc7c0d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.348+0000 7fbc815ff700 1 --2- 172.21.15.60:0/1694726160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0cfd50 0x7fbc7c0d23c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.348+0000 7fbc815ff700 1 -- 172.21.15.60:0/1694726160 >> 172.21.15.60:0/1694726160 conn(0x7fbc7c06b2f0 msgr2=0x7fbc7c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:14.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.348+0000 7fbc815ff700 1 -- 172.21.15.60:0/1694726160 shutdown_connections 2023-07-13T22:23:14.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.348+0000 7fbc815ff700 1 -- 172.21.15.60:0/1694726160 wait complete. 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.349+0000 7fbc815ff700 1 Processor -- start 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.350+0000 7fbc815ff700 1 -- start start 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.350+0000 7fbc815ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc7c0b1ac0 0x7fbc7c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.350+0000 7fbc815ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0b8200 0x7fbc7c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.350+0000 7fbc815ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc7c0cfd50 0x7fbc7c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc815ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c0ca750 con 0x7fbc7c0b8200 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc815ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c0caa30 con 0x7fbc7c0cfd50 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc815ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c0cad10 con 0x7fbc7c0b1ac0 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc73fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0b8200 0x7fbc7c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc73fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0b8200 0x7fbc7c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53024/0 (socket says 172.21.15.60:53024) 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc73fff700 1 -- 172.21.15.60:0/1391442248 learned_addr learned my addr 172.21.15.60:0/1391442248 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc7b7fe700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc7c0cfd50 0x7fbc7c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc7affd700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc7c0b1ac0 0x7fbc7c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc73fff700 1 -- 172.21.15.60:0/1391442248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc7c0b1ac0 msgr2=0x7fbc7c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc73fff700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc7c0b1ac0 0x7fbc7c0cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc73fff700 1 -- 172.21.15.60:0/1391442248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc7c0cfd50 msgr2=0x7fbc7c151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc73fff700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc7c0cfd50 0x7fbc7c151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.351+0000 7fbc73fff700 1 -- 172.21.15.60:0/1391442248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640094a0 con 0x7fbc7c0b8200 2023-07-13T22:23:14.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.352+0000 7fbc73fff700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0b8200 0x7fbc7c0c9d00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc64014040 tx=0x7fbc64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:14.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.352+0000 7fbc737fe700 1 -- 172.21.15.60:0/1391442248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64004590 con 0x7fbc7c0b8200 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.352+0000 7fbc737fe700 1 -- 172.21.15.60:0/1391442248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6401f070 con 0x7fbc7c0b8200 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.352+0000 7fbc737fe700 1 -- 172.21.15.60:0/1391442248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc640038e0 con 0x7fbc7c0b8200 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.352+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7c0caff0 con 0x7fbc7c0b8200 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.352+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7c0cb7f0 con 0x7fbc7c0b8200 2023-07-13T22:23:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.353+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7c04e210 con 0x7fbc7c0b8200 2023-07-13T22:23:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.354+0000 7fbc737fe700 1 -- 172.21.15.60:0/1391442248 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbc64003a40 con 0x7fbc7c0b8200 2023-07-13T22:23:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.355+0000 7fbc737fe700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc5c06ace0 0x7fbc5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.355+0000 7fbc737fe700 1 -- 172.21.15.60:0/1391442248 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fbc64021070 con 0x7fbc7c0b8200 2023-07-13T22:23:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.355+0000 7fbc7affd700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc5c06ace0 0x7fbc5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.356+0000 7fbc7affd700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc5c06ace0 0x7fbc5c06d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c007fd0 tx=0x7fbc6c007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.360+0000 7fbc737fe700 1 -- 172.21.15.60:0/1391442248 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbc64055c00 con 0x7fbc7c0b8200 2023-07-13T22:23:14.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:14.614+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbc7c0b4f70 con 0x7fbc7c0b8200 2023-07-13T22:23:15.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.098+0000 7fbc737fe700 1 -- 172.21.15.60:0/1391442248 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbc64059220 con 0x7fbc7c0b8200 2023-07-13T22:23:15.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.104+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc5c06ace0 msgr2=0x7fbc5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.104+0000 7fbc815ff700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc5c06ace0 0x7fbc5c06d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c007fd0 tx=0x7fbc6c007dc0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.104+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0b8200 msgr2=0x7fbc7c0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.104+0000 7fbc815ff700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0b8200 0x7fbc7c0c9d00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc64014040 tx=0x7fbc64004750 comp rx=0 tx=0).stop 2023-07-13T22:23:15.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.110+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 shutdown_connections 2023-07-13T22:23:15.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.110+0000 7fbc815ff700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc5c06ace0 0x7fbc5c06d190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.110+0000 7fbc815ff700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc7c0cfd50 0x7fbc7c151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.110+0000 7fbc815ff700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc7c0b1ac0 0x7fbc7c0cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.110+0000 7fbc815ff700 1 --2- 172.21.15.60:0/1391442248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc7c0b8200 0x7fbc7c0c9d00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.110+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 >> 172.21.15.60:0/1391442248 conn(0x7fbc7c06b2f0 msgr2=0x7fbc7c0d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:15.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.113+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 shutdown_connections 2023-07-13T22:23:15.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.114+0000 7fbc815ff700 1 -- 172.21.15.60:0/1391442248 wait complete. 2023-07-13T22:23:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 7 and data pool 8 2023-07-13T22:23:15.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:23:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.349+0000 7f618d5ff700 1 Processor -- start 2023-07-13T22:23:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.350+0000 7f618d5ff700 1 -- start start 2023-07-13T22:23:15.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.351+0000 7f618d5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61880cfca0 0x7f6188167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.351+0000 7f618d5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61880cd950 con 0x7f61880d3750 2023-07-13T22:23:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.351+0000 7f618d5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61880d0ac0 con 0x7f61880b8400 2023-07-13T22:23:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.351+0000 7f618d5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880b87d0 con 0x7f61880cfca0 2023-07-13T22:23:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.351+0000 7f61867fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f61880d3750 0x7f61880cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47224/0 (socket says 172.21.15.60:47224) 2023-07-13T22:23:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.351+0000 7f61867fc700 1 -- 172.21.15.60:0/3315654814 learned_addr learned my addr 172.21.15.60:0/3315654814 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:15.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175952502 0 0) 0x7f617c001110 con 0x7f61880d3750 2023-07-13T22:23:15.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61700035b0 con 0x7f61880d3750 2023-07-13T22:23:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f61877fe700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61880cfca0 0x7f6188167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3727830315 0 0) 0x7f61700035b0 con 0x7f61880d3750 2023-07-13T22:23:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61880cfca0 msgr2=0x7f6188167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61880cfca0 0x7f6188167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 >> v1:172.21.15.105:6789/0 conn(0x7f61880b8400 legacy=0x7f61880cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880b4eb0 con 0x7f61880d3750 2023-07-13T22:23:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f617c001800 con 0x7f61880d3750 2023-07-13T22:23:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 >> v1:172.21.15.60:6789/0 conn(0x7f61880d3750 legacy=0x7f61880cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61880cfca0 0x7f6170003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6170004060 0x7f6170004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6170004d90 0x7f6170005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880b87d0 con 0x7f6170004d90 2023-07-13T22:23:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617000bc20 con 0x7f6170004060 2023-07-13T22:23:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617000bde0 con 0x7f61880cfca0 2023-07-13T22:23:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f61877fe700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6170004d90 0x7f6170005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f6186ffd700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61880cfca0 0x7f6170003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.352+0000 7f61867fc700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6170004060 0x7f6170004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.353+0000 7f61877fe700 1 -- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61880cfca0 msgr2=0x7f6170003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.353+0000 7f61877fe700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61880cfca0 0x7f6170003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.353+0000 7f61877fe700 1 -- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6170004060 msgr2=0x7f6170004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.353+0000 7f61877fe700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6170004060 0x7f6170004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.353+0000 7f61877fe700 1 -- 172.21.15.60:0/3315654814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700035b0 con 0x7f6170004d90 2023-07-13T22:23:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.353+0000 7f61877fe700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6170004d90 0x7f6170005200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6180007330 tx=0x7f6170005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ee6647ed411f54 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.353+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180007e50 con 0x7f6170004d90 2023-07-13T22:23:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.353+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6180014400 con 0x7f6170004d90 2023-07-13T22:23:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.353+0000 7f6185ffb700 1 -- 172.21.15.60:0/3315654814 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61800147b0 con 0x7f6170004d90 2023-07-13T22:23:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.354+0000 7f618d5ff700 1 -- 172.21.15.60:0/3315654814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6170004d90 msgr2=0x7f6170005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.354+0000 7f618d5ff700 1 --2- 172.21.15.60:0/3315654814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6170004d90 0x7f6170005200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6180007330 tx=0x7f6170005880 comp rx=0 tx=0).stop 2023-07-13T22:23:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.354+0000 7f61877fe700 1 -- 172.21.15.60:0/3315654814 reap_dead start 2023-07-13T22:23:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.354+0000 7f618d5ff700 1 -- 172.21.15.60:0/3315654814 shutdown_connections 2023-07-13T22:23:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.354+0000 7f618d5ff700 1 -- 172.21.15.60:0/3315654814 >> 172.21.15.60:0/3315654814 conn(0x7f618806b390 msgr2=0x7f618806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.354+0000 7f618d5ff700 1 -- 172.21.15.60:0/3315654814 shutdown_connections 2023-07-13T22:23:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.354+0000 7f618d5ff700 1 -- 172.21.15.60:0/3315654814 wait complete. 2023-07-13T22:23:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.356+0000 7f618d5ff700 1 Processor -- start 2023-07-13T22:23:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.356+0000 7f618d5ff700 1 -- start start 2023-07-13T22:23:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f618d5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6170004d90 0x7f61881f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f618d5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61880c8f70 0x7f61881efb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f618d5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188159cd0 0x7f61880b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f618d5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880cbdc0 con 0x7f6188159cd0 2023-07-13T22:23:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f618d5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61881f24f0 con 0x7f61880c8f70 2023-07-13T22:23:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f618d5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61881f27d0 con 0x7f6170004d90 2023-07-13T22:23:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f61877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188159cd0 0x7f61880b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f61877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188159cd0 0x7f61880b62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34660/0 (socket says 172.21.15.60:34660) 2023-07-13T22:23:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f61877fe700 1 -- 172.21.15.60:0/2142247392 learned_addr learned my addr 172.21.15.60:0/2142247392 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f61877fe700 1 -- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6170004d90 msgr2=0x7f61881f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f6186ffd700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6170004d90 0x7f61881f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f61867fc700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61880c8f70 0x7f61881efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f61877fe700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6170004d90 0x7f61881f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f61877fe700 1 -- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61880c8f70 msgr2=0x7f61881efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f61877fe700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61880c8f70 0x7f61881efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f61877fe700 1 -- 172.21.15.60:0/2142247392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61881f2ab0 con 0x7f6188159cd0 2023-07-13T22:23:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.357+0000 7f6186ffd700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6170004d90 0x7f61881f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.358+0000 7f61867fc700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61880c8f70 0x7f61881efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.358+0000 7f61877fe700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188159cd0 0x7f61880b62d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6180017040 tx=0x7f6180004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.358+0000 7f616f7fe700 1 -- 172.21.15.60:0/2142247392 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180007000 con 0x7f6188159cd0 2023-07-13T22:23:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.358+0000 7f616f7fe700 1 -- 172.21.15.60:0/2142247392 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618001e410 con 0x7f6188159cd0 2023-07-13T22:23:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.358+0000 7f616f7fe700 1 -- 172.21.15.60:0/2142247392 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618001e7c0 con 0x7f6188159cd0 2023-07-13T22:23:15.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.358+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61881f0060 con 0x7f6188159cd0 2023-07-13T22:23:15.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.358+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61880c93e0 con 0x7f6188159cd0 2023-07-13T22:23:15.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.359+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618815ad10 con 0x7f6188159cd0 2023-07-13T22:23:15.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.360+0000 7f616f7fe700 1 -- 172.21.15.60:0/2142247392 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f618001b050 con 0x7f6188159cd0 2023-07-13T22:23:15.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.361+0000 7f616f7fe700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f617406ace0 0x7f617406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.361+0000 7f616f7fe700 1 -- 172.21.15.60:0/2142247392 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f618008a790 con 0x7f6188159cd0 2023-07-13T22:23:15.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.361+0000 7f6186ffd700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f617406ace0 0x7f617406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.362+0000 7f6186ffd700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f617406ace0 0x7f617406d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6178007fd0 tx=0x7f6178009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:15.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.366+0000 7f616f7fe700 1 -- 172.21.15.60:0/2142247392 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6180055950 con 0x7f6188159cd0 2023-07-13T22:23:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.572+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f618815aa30 con 0x7f6188159cd0 2023-07-13T22:23:15.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.573+0000 7f616f7fe700 1 -- 172.21.15.60:0/2142247392 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18270 (secure 0 0 0) 0x7f6180058f70 con 0x7f6188159cd0 2023-07-13T22:23:15.557 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:15.558 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:14.615880+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:23:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f617406ace0 msgr2=0x7f617406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f617406ace0 0x7f617406d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6178007fd0 tx=0x7f6178009040 comp rx=0 tx=0).stop 2023-07-13T22:23:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188159cd0 msgr2=0x7f61880b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188159cd0 0x7f61880b62d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6180017040 tx=0x7f6180004b40 comp rx=0 tx=0).stop 2023-07-13T22:23:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 shutdown_connections 2023-07-13T22:23:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f617406ace0 0x7f617406d190 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188159cd0 0x7f61880b62d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6170004d90 0x7f61881f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 --2- 172.21.15.60:0/2142247392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61880c8f70 0x7f61881efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 >> 172.21.15.60:0/2142247392 conn(0x7f618806b390 msgr2=0x7f61880b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 shutdown_connections 2023-07-13T22:23:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.576+0000 7f618d5ff700 1 -- 172.21.15.60:0/2142247392 wait complete. 2023-07-13T22:23:15.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:23:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.796+0000 7fa974fff700 1 Processor -- start 2023-07-13T22:23:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.796+0000 7fa974fff700 1 -- start start 2023-07-13T22:23:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa974fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa970154950 0x7fa970070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa974fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9700c9450 0x7fa97015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa974fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa970070270 con 0x7fa97015a820 2023-07-13T22:23:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa974fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97015b2e0 con 0x7fa970154950 2023-07-13T22:23:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa974fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97015b5c0 con 0x7fa9700c9450 2023-07-13T22:23:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa96e59c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa970154950 0x7fa970070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa96ed9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9700c9450 0x7fa97015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa96e59c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa970154950 0x7fa970070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:53770/0 (socket says 172.21.15.60:53770) 2023-07-13T22:23:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa96e59c700 1 -- 172.21.15.60:0/1306898178 learned_addr learned my addr 172.21.15.60:0/1306898178 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa96ed9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9700c9450 0x7fa97015ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:44638/0 (socket says 172.21.15.60:44638) 2023-07-13T22:23:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.798+0000 7fa967fff700 1 --1- 172.21.15.60:0/1306898178 >> v1:172.21.15.105:6789/0 conn(0x7fa97015a820 0x7fa9700c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:37474/0 (socket says 172.21.15.60:37474) 2023-07-13T22:23:15.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.799+0000 7fa96e59c700 1 -- 172.21.15.60:0/1306898178 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9700c9450 msgr2=0x7fa97015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:15.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.799+0000 7fa96e59c700 1 --2- 172.21.15.60:0/1306898178 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9700c9450 0x7fa97015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.799+0000 7fa96e59c700 1 -- 172.21.15.60:0/1306898178 >> v1:172.21.15.105:6789/0 conn(0x7fa97015a820 legacy=0x7fa9700c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.799+0000 7fa96e59c700 1 -- 172.21.15.60:0/1306898178 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9700704c0 con 0x7fa970154950 2023-07-13T22:23:15.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.799+0000 7fa96dd9b700 1 -- 172.21.15.60:0/1306898178 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101764910 0 0) 0x7fa970070270 con 0x7fa97015a820 2023-07-13T22:23:15.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.799+0000 7fa96e59c700 1 --2- 172.21.15.60:0/1306898178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa970154950 0x7fa970070cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa9600098f0 tx=0x7fa960009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63df71163576a10c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:15.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.799+0000 7fa96dd9b700 1 -- 172.21.15.60:0/1306898178 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960011830 con 0x7fa970154950 2023-07-13T22:23:15.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.799+0000 7fa96dd9b700 1 -- 172.21.15.60:0/1306898178 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa960011990 con 0x7fa970154950 2023-07-13T22:23:15.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.799+0000 7fa96dd9b700 1 -- 172.21.15.60:0/1306898178 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960011d20 con 0x7fa970154950 2023-07-13T22:23:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.800+0000 7fa974fff700 1 -- 172.21.15.60:0/1306898178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa970154950 msgr2=0x7fa970070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.800+0000 7fa974fff700 1 --2- 172.21.15.60:0/1306898178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa970154950 0x7fa970070cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa9600098f0 tx=0x7fa960009c00 comp rx=0 tx=0).stop 2023-07-13T22:23:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.800+0000 7fa974fff700 1 -- 172.21.15.60:0/1306898178 shutdown_connections 2023-07-13T22:23:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.800+0000 7fa974fff700 1 --2- 172.21.15.60:0/1306898178 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9700c9450 0x7fa97015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.800+0000 7fa974fff700 1 --2- 172.21.15.60:0/1306898178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa970154950 0x7fa970070cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.800+0000 7fa974fff700 1 -- 172.21.15.60:0/1306898178 >> 172.21.15.60:0/1306898178 conn(0x7fa9700cf410 msgr2=0x7fa9700cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.800+0000 7fa974fff700 1 -- 172.21.15.60:0/1306898178 shutdown_connections 2023-07-13T22:23:15.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.800+0000 7fa974fff700 1 -- 172.21.15.60:0/1306898178 wait complete. 2023-07-13T22:23:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.802+0000 7fa974fff700 1 Processor -- start 2023-07-13T22:23:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.802+0000 7fa974fff700 1 -- start start 2023-07-13T22:23:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa974fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9700c9450 0x7fa9700ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa974fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa970154950 0x7fa9700c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa974fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa97015a820 0x7fa9700c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa974fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700aeb50 con 0x7fa970154950 2023-07-13T22:23:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa974fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700b3130 con 0x7fa9700c9450 2023-07-13T22:23:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa974fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700ae880 con 0x7fa97015a820 2023-07-13T22:23:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa967fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa970154950 0x7fa9700c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa967fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa970154950 0x7fa9700c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34662/0 (socket says 172.21.15.60:34662) 2023-07-13T22:23:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa967fff700 1 -- 172.21.15.60:0/2331663173 learned_addr learned my addr 172.21.15.60:0/2331663173 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa96e59c700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9700c9450 0x7fa9700ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.803+0000 7fa96ed9d700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa97015a820 0x7fa9700c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa967fff700 1 -- 172.21.15.60:0/2331663173 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa97015a820 msgr2=0x7fa9700c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa967fff700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa97015a820 0x7fa9700c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa967fff700 1 -- 172.21.15.60:0/2331663173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9700c9450 msgr2=0x7fa9700ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa967fff700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9700c9450 0x7fa9700ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa967fff700 1 -- 172.21.15.60:0/2331663173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa960009550 con 0x7fa970154950 2023-07-13T22:23:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa96e59c700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9700c9450 0x7fa9700ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa967fff700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa970154950 0x7fa9700c7d00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa95800c550 tx=0x7fa95800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa966ffd700 1 -- 172.21.15.60:0/2331663173 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958004c80 con 0x7fa970154950 2023-07-13T22:23:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa966ffd700 1 -- 172.21.15.60:0/2331663173 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa958004e00 con 0x7fa970154950 2023-07-13T22:23:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9700b3340 con 0x7fa970154950 2023-07-13T22:23:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa966ffd700 1 -- 172.21.15.60:0/2331663173 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958007310 con 0x7fa970154950 2023-07-13T22:23:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.804+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9700b3900 con 0x7fa970154950 2023-07-13T22:23:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.806+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97004e280 con 0x7fa970154950 2023-07-13T22:23:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.806+0000 7fa966ffd700 1 -- 172.21.15.60:0/2331663173 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa958007470 con 0x7fa970154950 2023-07-13T22:23:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.807+0000 7fa966ffd700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa95006ace0 0x7fa95006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.807+0000 7fa966ffd700 1 -- 172.21.15.60:0/2331663173 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa958089130 con 0x7fa970154950 2023-07-13T22:23:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.808+0000 7fa96e59c700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa95006ace0 0x7fa95006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.808+0000 7fa96e59c700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa95006ace0 0x7fa95006d190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa960009c40 tx=0x7fa960008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:15.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:15.812+0000 7fa966ffd700 1 -- 172.21.15.60:0/2331663173 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa958054140 con 0x7fa970154950 2023-07-13T22:23:16.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.018+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9700cbc20 con 0x7fa970154950 2023-07-13T22:23:16.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.020+0000 7fa966ffd700 1 -- 172.21.15.60:0/2331663173 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18270 (secure 0 0 0) 0x7fa958057760 con 0x7fa970154950 2023-07-13T22:23:16.004 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:16.004 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:14.615880+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:23:16.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa95006ace0 msgr2=0x7fa95006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa95006ace0 0x7fa95006d190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa960009c40 tx=0x7fa960008030 comp rx=0 tx=0).stop 2023-07-13T22:23:16.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa970154950 msgr2=0x7fa9700c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa970154950 0x7fa9700c7d00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa95800c550 tx=0x7fa95800c860 comp rx=0 tx=0).stop 2023-07-13T22:23:16.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 shutdown_connections 2023-07-13T22:23:16.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa95006ace0 0x7fa95006d190 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa97015a820 0x7fa9700c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9700c9450 0x7fa9700ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 --2- 172.21.15.60:0/2331663173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa970154950 0x7fa9700c7d00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 >> 172.21.15.60:0/2331663173 conn(0x7fa9700cf410 msgr2=0x7fa97015d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:16.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.022+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 shutdown_connections 2023-07-13T22:23:16.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.023+0000 7fa974fff700 1 -- 172.21.15.60:0/2331663173 wait complete. 2023-07-13T22:23:16.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:23:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.252+0000 7fad9b7ff700 1 Processor -- start 2023-07-13T22:23:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.252+0000 7fad9b7ff700 1 -- start start 2023-07-13T22:23:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.253+0000 7fad9b7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fad9406afe0 0x7fad940cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.253+0000 7fad9b7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 0x7fad940d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad9b7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad940cd220 con 0x7fad9415a170 2023-07-13T22:23:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad9b7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9406bb20 con 0x7fad9406afe0 2023-07-13T22:23:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad9b7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad940d1270 con 0x7fad940d08b0 2023-07-13T22:23:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad98d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fad9406afe0 0x7fad940cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad9959b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fad9415a170 0x7fad9415d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47240/0 (socket says 172.21.15.60:47240) 2023-07-13T22:23:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad9959b700 1 -- 172.21.15.60:0/2059374444 learned_addr learned my addr 172.21.15.60:0/2059374444 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad99d9c700 1 --2- 172.21.15.60:0/2059374444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 0x7fad940d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad98d9a700 1 -- 172.21.15.60:0/2059374444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 msgr2=0x7fad940d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad98d9a700 1 --2- 172.21.15.60:0/2059374444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 0x7fad940d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad98d9a700 1 -- 172.21.15.60:0/2059374444 >> v1:172.21.15.60:6789/0 conn(0x7fad9415a170 legacy=0x7fad9415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad98d9a700 1 -- 172.21.15.60:0/2059374444 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9406b560 con 0x7fad9406afe0 2023-07-13T22:23:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.254+0000 7fad8bfff700 1 -- 172.21.15.60:0/2059374444 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434272412 0 0) 0x7fad940cd220 con 0x7fad9415a170 2023-07-13T22:23:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad98d9a700 1 --2- 172.21.15.60:0/2059374444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 0x7fad940cdc70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fad84009840 tx=0x7fad84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf9ca3372ad8f3b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad8bfff700 1 -- 172.21.15.60:0/2059374444 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84011830 con 0x7fad9406afe0 2023-07-13T22:23:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad8bfff700 1 -- 172.21.15.60:0/2059374444 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad84011990 con 0x7fad9406afe0 2023-07-13T22:23:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad8bfff700 1 -- 172.21.15.60:0/2059374444 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84011d20 con 0x7fad9406afe0 2023-07-13T22:23:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad9b7ff700 1 -- 172.21.15.60:0/2059374444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 msgr2=0x7fad940cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad9b7ff700 1 --2- 172.21.15.60:0/2059374444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 0x7fad940cdc70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fad84009840 tx=0x7fad84009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad9b7ff700 1 -- 172.21.15.60:0/2059374444 shutdown_connections 2023-07-13T22:23:16.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad9b7ff700 1 --2- 172.21.15.60:0/2059374444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 0x7fad940d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad9b7ff700 1 --2- 172.21.15.60:0/2059374444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 0x7fad940cdc70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.255+0000 7fad9b7ff700 1 -- 172.21.15.60:0/2059374444 >> 172.21.15.60:0/2059374444 conn(0x7fad9406a7f0 msgr2=0x7fad94099cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:16.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.256+0000 7fad9b7ff700 1 -- 172.21.15.60:0/2059374444 shutdown_connections 2023-07-13T22:23:16.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.256+0000 7fad9b7ff700 1 -- 172.21.15.60:0/2059374444 wait complete. 2023-07-13T22:23:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.257+0000 7fad9b7ff700 1 Processor -- start 2023-07-13T22:23:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.257+0000 7fad9b7ff700 1 -- start start 2023-07-13T22:23:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.258+0000 7fad9b7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 0x7fad940b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.258+0000 7fad9b7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 0x7fad94157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.258+0000 7fad9b7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad9415a170 0x7fad940c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.258+0000 7fad9b7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940b5690 con 0x7fad9406afe0 2023-07-13T22:23:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.258+0000 7fad9b7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940b3ce0 con 0x7fad9415a170 2023-07-13T22:23:16.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.258+0000 7fad9b7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad940b68c0 con 0x7fad940d08b0 2023-07-13T22:23:16.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.258+0000 7fad9959b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 0x7fad940b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad9959b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 0x7fad940b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34694/0 (socket says 172.21.15.60:34694) 2023-07-13T22:23:16.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad9959b700 1 -- 172.21.15.60:0/1648465941 learned_addr learned my addr 172.21.15.60:0/1648465941 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:16.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad98d9a700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 0x7fad94157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad99d9c700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad9415a170 0x7fad940c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad9959b700 1 -- 172.21.15.60:0/1648465941 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 msgr2=0x7fad94157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad9959b700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 0x7fad94157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad9959b700 1 -- 172.21.15.60:0/1648465941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad9415a170 msgr2=0x7fad940c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad9959b700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad9415a170 0x7fad940c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad9959b700 1 -- 172.21.15.60:0/1648465941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840094a0 con 0x7fad9406afe0 2023-07-13T22:23:16.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad9959b700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 0x7fad940b6380 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fad9000c4a0 tx=0x7fad9000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:16.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad89ffb700 1 -- 172.21.15.60:0/1648465941 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad900074e0 con 0x7fad9406afe0 2023-07-13T22:23:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad89ffb700 1 -- 172.21.15.60:0/1648465941 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad90007640 con 0x7fad9406afe0 2023-07-13T22:23:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad89ffb700 1 -- 172.21.15.60:0/1648465941 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90005a70 con 0x7fad9406afe0 2023-07-13T22:23:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.259+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad940b4a10 con 0x7fad9406afe0 2023-07-13T22:23:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.260+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad940b4fd0 con 0x7fad9406afe0 2023-07-13T22:23:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.261+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad94067a70 con 0x7fad9406afe0 2023-07-13T22:23:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.262+0000 7fad89ffb700 1 -- 172.21.15.60:0/1648465941 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad90005bd0 con 0x7fad9406afe0 2023-07-13T22:23:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.262+0000 7fad89ffb700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad7806ace0 0x7fad7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.263+0000 7fad89ffb700 1 -- 172.21.15.60:0/1648465941 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad900888e0 con 0x7fad9406afe0 2023-07-13T22:23:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.263+0000 7fad98d9a700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad7806ace0 0x7fad7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.263+0000 7fad98d9a700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad7806ace0 0x7fad7806d190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad840090b0 tx=0x7fad84008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:16.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.267+0000 7fad89ffb700 1 -- 172.21.15.60:0/1648465941 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fad90053980 con 0x7fad9406afe0 2023-07-13T22:23:16.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:16.517+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fad940b57d0 con 0x7fad9406afe0 2023-07-13T22:23:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.125+0000 7fad89ffb700 1 -- 172.21.15.60:0/1648465941 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fad940b57d0 con 0x7fad9406afe0 2023-07-13T22:23:17.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.128+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad7806ace0 msgr2=0x7fad7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.128+0000 7fad9b7ff700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad7806ace0 0x7fad7806d190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad840090b0 tx=0x7fad84008030 comp rx=0 tx=0).stop 2023-07-13T22:23:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.128+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 msgr2=0x7fad940b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.128+0000 7fad9b7ff700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 0x7fad940b6380 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fad9000c4a0 tx=0x7fad9000c7b0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.128+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 shutdown_connections 2023-07-13T22:23:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.128+0000 7fad9b7ff700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad7806ace0 0x7fad7806d190 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.128+0000 7fad9b7ff700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad9406afe0 0x7fad940b6380 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.128+0000 7fad9b7ff700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad9415a170 0x7fad940c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.128+0000 7fad9b7ff700 1 --2- 172.21.15.60:0/1648465941 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad940d08b0 0x7fad94157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.129+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 >> 172.21.15.60:0/1648465941 conn(0x7fad9406a7f0 msgr2=0x7fad9415c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.129+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 shutdown_connections 2023-07-13T22:23:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.129+0000 7fad9b7ff700 1 -- 172.21.15.60:0/1648465941 wait complete. 2023-07-13T22:23:17.134 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:23:17.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:23:17.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.360+0000 7fb2b35ff700 1 Processor -- start 2023-07-13T22:23:17.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.360+0000 7fb2b35ff700 1 -- start start 2023-07-13T22:23:17.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.361+0000 7fb2b35ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb2ac070b30 0x7fb2ac0caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2b35ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ac06bf40 con 0x7fb2ac0ce770 2023-07-13T22:23:17.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2b35ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ac0cebf0 con 0x7fb2ac15c460 2023-07-13T22:23:17.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2b35ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ac06b9f0 con 0x7fb2ac070b30 2023-07-13T22:23:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2b1b9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb2ac0ce770 0x7fb2ac0b23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47254/0 (socket says 172.21.15.60:47254) 2023-07-13T22:23:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2b1b9c700 1 -- 172.21.15.60:0/3186063090 learned_addr learned my addr 172.21.15.60:0/3186063090 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2b0b9a700 1 --2- 172.21.15.60:0/3186063090 >> v2:172.21.15.105:3300/0 conn(0x7fb2ac070b30 0x7fb2ac0caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2a3fff700 1 -- 172.21.15.60:0/3186063090 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114400898 0 0) 0x7fb2ac06bf40 con 0x7fb2ac0ce770 2023-07-13T22:23:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2a3fff700 1 -- 172.21.15.60:0/3186063090 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2980035b0 con 0x7fb2ac0ce770 2023-07-13T22:23:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2b0b9a700 1 -- 172.21.15.60:0/3186063090 >> v1:172.21.15.105:6789/0 conn(0x7fb2ac15c460 legacy=0x7fb2ac06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.362+0000 7fb2b0b9a700 1 -- 172.21.15.60:0/3186063090 >> v1:172.21.15.60:6789/0 conn(0x7fb2ac0ce770 legacy=0x7fb2ac0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.363+0000 7fb2b0b9a700 1 -- 172.21.15.60:0/3186063090 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac06d000 con 0x7fb2ac070b30 2023-07-13T22:23:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.363+0000 7fb2b0b9a700 1 --2- 172.21.15.60:0/3186063090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2ac070b30 0x7fb2ac0caf80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009840 tx=0x7fb29c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b27b0df09064fd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.363+0000 7fb2a3fff700 1 -- 172.21.15.60:0/3186063090 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c011830 con 0x7fb2ac070b30 2023-07-13T22:23:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.363+0000 7fb2a3fff700 1 -- 172.21.15.60:0/3186063090 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29c011990 con 0x7fb2ac070b30 2023-07-13T22:23:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.363+0000 7fb2a3fff700 1 -- 172.21.15.60:0/3186063090 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c011d20 con 0x7fb2ac070b30 2023-07-13T22:23:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.364+0000 7fb2b35ff700 1 -- 172.21.15.60:0/3186063090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2ac070b30 msgr2=0x7fb2ac0caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.364+0000 7fb2b35ff700 1 --2- 172.21.15.60:0/3186063090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2ac070b30 0x7fb2ac0caf80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009840 tx=0x7fb29c009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.364+0000 7fb2b35ff700 1 -- 172.21.15.60:0/3186063090 shutdown_connections 2023-07-13T22:23:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.364+0000 7fb2b35ff700 1 --2- 172.21.15.60:0/3186063090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2ac070b30 0x7fb2ac0caf80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.364+0000 7fb2b35ff700 1 -- 172.21.15.60:0/3186063090 >> 172.21.15.60:0/3186063090 conn(0x7fb2ac06e580 msgr2=0x7fb2ac06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:17.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.364+0000 7fb2b35ff700 1 -- 172.21.15.60:0/3186063090 shutdown_connections 2023-07-13T22:23:17.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.364+0000 7fb2b35ff700 1 -- 172.21.15.60:0/3186063090 wait complete. 2023-07-13T22:23:17.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.366+0000 7fb2b35ff700 1 Processor -- start 2023-07-13T22:23:17.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.366+0000 7fb2b35ff700 1 -- start start 2023-07-13T22:23:17.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b35ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ac070b30 0x7fb2ac0cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b35ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2ac0ce770 0x7fb2ac15db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b35ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2ac15c460 0x7fb2ac15e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b35ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0b58c0 con 0x7fb2ac070b30 2023-07-13T22:23:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b35ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac15e8a0 con 0x7fb2ac15c460 2023-07-13T22:23:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b35ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0b4f20 con 0x7fb2ac0ce770 2023-07-13T22:23:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b139b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ac070b30 0x7fb2ac0cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b139b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ac070b30 0x7fb2ac0cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34696/0 (socket says 172.21.15.60:34696) 2023-07-13T22:23:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b139b700 1 -- 172.21.15.60:0/2977136431 learned_addr learned my addr 172.21.15.60:0/2977136431 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b1b9c700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2ac15c460 0x7fb2ac15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b0b9a700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2ac0ce770 0x7fb2ac15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b139b700 1 -- 172.21.15.60:0/2977136431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2ac0ce770 msgr2=0x7fb2ac15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b139b700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2ac0ce770 0x7fb2ac15db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b139b700 1 -- 172.21.15.60:0/2977136431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2ac15c460 msgr2=0x7fb2ac15e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b139b700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2ac15c460 0x7fb2ac15e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.367+0000 7fb2b139b700 1 -- 172.21.15.60:0/2977136431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0094a0 con 0x7fb2ac070b30 2023-07-13T22:23:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.368+0000 7fb2b139b700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ac070b30 0x7fb2ac0cca80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800c4e0 tx=0x7fb2a800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.368+0000 7fb2a1ffb700 1 -- 172.21.15.60:0/2977136431 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80052d0 con 0x7fb2ac070b30 2023-07-13T22:23:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.368+0000 7fb2a1ffb700 1 -- 172.21.15.60:0/2977136431 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a8005430 con 0x7fb2ac070b30 2023-07-13T22:23:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.368+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac0b5200 con 0x7fb2ac070b30 2023-07-13T22:23:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.368+0000 7fb2a1ffb700 1 -- 172.21.15.60:0/2977136431 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80038c0 con 0x7fb2ac070b30 2023-07-13T22:23:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.368+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac0ccfc0 con 0x7fb2ac070b30 2023-07-13T22:23:17.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.369+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac0b53f0 con 0x7fb2ac070b30 2023-07-13T22:23:17.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.370+0000 7fb2a1ffb700 1 -- 172.21.15.60:0/2977136431 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb2a8003a20 con 0x7fb2ac070b30 2023-07-13T22:23:17.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.371+0000 7fb2a1ffb700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb29006ace0 0x7fb29006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.371+0000 7fb2a1ffb700 1 -- 172.21.15.60:0/2977136431 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2a80889b0 con 0x7fb2ac070b30 2023-07-13T22:23:17.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.371+0000 7fb2b0b9a700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb29006ace0 0x7fb29006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.372+0000 7fb2b0b9a700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb29006ace0 0x7fb29006d190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009fd0 tx=0x7fb29c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.375+0000 7fb2a1ffb700 1 -- 172.21.15.60:0/2977136431 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb2a8053a50 con 0x7fb2ac070b30 2023-07-13T22:23:17.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:17.626+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb2ac0b55f0 con 0x7fb2ac070b30 2023-07-13T22:23:18.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.136+0000 7fb2a1ffb700 1 -- 172.21.15.60:0/2977136431 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb2a8057070 con 0x7fb2ac070b30 2023-07-13T22:23:18.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb29006ace0 msgr2=0x7fb29006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb29006ace0 0x7fb29006d190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009fd0 tx=0x7fb29c01c040 comp rx=0 tx=0).stop 2023-07-13T22:23:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ac070b30 msgr2=0x7fb2ac0cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ac070b30 0x7fb2ac0cca80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800c4e0 tx=0x7fb2a800c8a0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 shutdown_connections 2023-07-13T22:23:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb29006ace0 0x7fb29006d190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ac070b30 0x7fb2ac0cca80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2ac15c460 0x7fb2ac15e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 --2- 172.21.15.60:0/2977136431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2ac0ce770 0x7fb2ac15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 >> 172.21.15.60:0/2977136431 conn(0x7fb2ac06e580 msgr2=0x7fb2ac0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:18.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 shutdown_connections 2023-07-13T22:23:18.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.139+0000 7fb2b35ff700 1 -- 172.21.15.60:0/2977136431 wait complete. 2023-07-13T22:23:18.143 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:23:18.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:23:18.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.367+0000 7f27319ff700 1 Processor -- start 2023-07-13T22:23:18.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.367+0000 7f27319ff700 1 -- start start 2023-07-13T22:23:18.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.368+0000 7f27319ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.368+0000 7f27319ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f272c0cfad0 0x7f272c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.368+0000 7f27319ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272c0cd950 con 0x7f272c0b8400 2023-07-13T22:23:18.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.368+0000 7f27319ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272c0cff20 con 0x7f272c0cfad0 2023-07-13T22:23:18.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.368+0000 7f27319ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0d0ac0 con 0x7f272c0d3750 2023-07-13T22:23:18.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.369+0000 7f272a7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.369+0000 7f272b7fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f272c0cfad0 0x7f272c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.369+0000 7f272a7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:44674/0 (socket says 172.21.15.60:44674) 2023-07-13T22:23:18.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.369+0000 7f272a7fc700 1 -- 172.21.15.60:0/3234872376 learned_addr learned my addr 172.21.15.60:0/3234872376 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:18.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.369+0000 7f272b7fe700 1 -- 172.21.15.60:0/3234872376 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 msgr2=0x7f272c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:18.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.369+0000 7f272b7fe700 1 --2- 172.21.15.60:0/3234872376 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.369+0000 7f272b7fe700 1 -- 172.21.15.60:0/3234872376 >> v1:172.21.15.105:6789/0 conn(0x7f272c0b8400 legacy=0x7f272c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.369+0000 7f272b7fe700 1 -- 172.21.15.60:0/3234872376 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c0b87d0 con 0x7f272c0cfad0 2023-07-13T22:23:18.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.369+0000 7f272a7fc700 1 --2- 172.21.15.60:0/3234872376 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:18.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.370+0000 7f272b7fe700 1 --2- 172.21.15.60:0/3234872376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272c0cfad0 0x7f272c0d0f50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2724009d90 tx=0x7f272400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81c120af55b91cf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:18.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.370+0000 7f2729ffb700 1 -- 172.21.15.60:0/3234872376 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272400e070 con 0x7f272c0cfad0 2023-07-13T22:23:18.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.370+0000 7f2729ffb700 1 -- 172.21.15.60:0/3234872376 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272400bb20 con 0x7f272c0cfad0 2023-07-13T22:23:18.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.370+0000 7f2729ffb700 1 -- 172.21.15.60:0/3234872376 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724012430 con 0x7f272c0cfad0 2023-07-13T22:23:18.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.370+0000 7f27319ff700 1 -- 172.21.15.60:0/3234872376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272c0cfad0 msgr2=0x7f272c0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.370+0000 7f27319ff700 1 --2- 172.21.15.60:0/3234872376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272c0cfad0 0x7f272c0d0f50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2724009d90 tx=0x7f272400b330 comp rx=0 tx=0).stop 2023-07-13T22:23:18.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.371+0000 7f27319ff700 1 -- 172.21.15.60:0/3234872376 shutdown_connections 2023-07-13T22:23:18.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.371+0000 7f27319ff700 1 --2- 172.21.15.60:0/3234872376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272c0cfad0 0x7f272c0d0f50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.371+0000 7f27319ff700 1 --2- 172.21.15.60:0/3234872376 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.371+0000 7f27319ff700 1 -- 172.21.15.60:0/3234872376 >> 172.21.15.60:0/3234872376 conn(0x7f272c06b390 msgr2=0x7f272c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:18.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.371+0000 7f27319ff700 1 -- 172.21.15.60:0/3234872376 shutdown_connections 2023-07-13T22:23:18.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.371+0000 7f27319ff700 1 -- 172.21.15.60:0/3234872376 wait complete. 2023-07-13T22:23:18.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.372+0000 7f27319ff700 1 Processor -- start 2023-07-13T22:23:18.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.373+0000 7f27319ff700 1 -- start start 2023-07-13T22:23:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.373+0000 7f27319ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272c0b8400 0x7f272c150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.373+0000 7f27319ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f272c0cfad0 0x7f272c15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.373+0000 7f27319ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.373+0000 7f27319ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c150e80 con 0x7f272c0cfad0 2023-07-13T22:23:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.373+0000 7f27319ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c15d4b0 con 0x7f272c0b8400 2023-07-13T22:23:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.373+0000 7f27319ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c15d790 con 0x7f272c0d3750 2023-07-13T22:23:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f272c0cfad0 0x7f272c15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f272c0cfad0 0x7f272c15f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34700/0 (socket says 172.21.15.60:34700) 2023-07-13T22:23:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272a7fc700 1 -- 172.21.15.60:0/1273914874 learned_addr learned my addr 172.21.15.60:0/1273914874 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272affd700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272c0b8400 0x7f272c150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272b7fe700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272a7fc700 1 -- 172.21.15.60:0/1273914874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 msgr2=0x7f272c0b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272a7fc700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272a7fc700 1 -- 172.21.15.60:0/1273914874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272c0b8400 msgr2=0x7f272c150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272a7fc700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272c0b8400 0x7f272c150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.374+0000 7f272a7fc700 1 -- 172.21.15.60:0/1273914874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2720009550 con 0x7f272c0cfad0 2023-07-13T22:23:18.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.375+0000 7f272a7fc700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f272c0cfad0 0x7f272c15f0a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f272000e980 tx=0x7f272000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:18.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.375+0000 7f2713fff700 1 -- 172.21.15.60:0/1273914874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272000c690 con 0x7f272c0cfad0 2023-07-13T22:23:18.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.375+0000 7f2713fff700 1 -- 172.21.15.60:0/1273914874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272000c7f0 con 0x7f272c0cfad0 2023-07-13T22:23:18.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.375+0000 7f2713fff700 1 -- 172.21.15.60:0/1273914874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272000cb60 con 0x7f272c0cfad0 2023-07-13T22:23:18.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.375+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2724009a70 con 0x7f272c0cfad0 2023-07-13T22:23:18.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.375+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c0c7470 con 0x7f272c0cfad0 2023-07-13T22:23:18.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.376+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c15ad40 con 0x7f272c0cfad0 2023-07-13T22:23:18.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.377+0000 7f2713fff700 1 -- 172.21.15.60:0/1273914874 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f272000ccc0 con 0x7f272c0cfad0 2023-07-13T22:23:18.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.378+0000 7f2713fff700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f271806ace0 0x7f271806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.378+0000 7f2713fff700 1 -- 172.21.15.60:0/1273914874 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27200894a0 con 0x7f272c0cfad0 2023-07-13T22:23:18.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.378+0000 7f272affd700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f271806ace0 0x7f271806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.379+0000 7f272affd700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f271806ace0 0x7f271806d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f271c007fd0 tx=0x7f271c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.382+0000 7f2713fff700 1 -- 172.21.15.60:0/1273914874 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2720054540 con 0x7f272c0cfad0 2023-07-13T22:23:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.636+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f272c15aa60 con 0x7f272c0cfad0 2023-07-13T22:23:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.637+0000 7f2713fff700 1 -- 172.21.15.60:0/1273914874 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f2720057b60 con 0x7f272c0cfad0 2023-07-13T22:23:18.621 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:18.621 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3226044033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3226044033},{"type":"v1","addr":"172.21.15.105:6834","nonce":3226044033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5599,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2171011214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":2171011214},{"type":"v1","addr":"172.21.15.60:6836","nonce":2171011214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1623228645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1623228645},{"type":"v1","addr":"172.21.15.60:6837","nonce":1623228645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:23:14.616210+0000","modified":"2023-07-13T22:23:18.125486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5608},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5608":{"gid":5608,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/3570367595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3570367595},{"type":"v1","addr":"172.21.15.105:6835","nonce":3570367595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-13T22:23:18.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.639+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f271806ace0 msgr2=0x7f271806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.639+0000 7f27319ff700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f271806ace0 0x7f271806d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f271c007fd0 tx=0x7f271c009040 comp rx=0 tx=0).stop 2023-07-13T22:23:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.639+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f272c0cfad0 msgr2=0x7f272c15f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.639+0000 7f27319ff700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f272c0cfad0 0x7f272c15f0a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f272000e980 tx=0x7f272000ec90 comp rx=0 tx=0).stop 2023-07-13T22:23:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.640+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 shutdown_connections 2023-07-13T22:23:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.640+0000 7f27319ff700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f271806ace0 0x7f271806d190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.640+0000 7f27319ff700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f272c0d3750 0x7f272c0b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.640+0000 7f27319ff700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f272c0b8400 0x7f272c150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.640+0000 7f27319ff700 1 --2- 172.21.15.60:0/1273914874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f272c0cfad0 0x7f272c15f0a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.640+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 >> 172.21.15.60:0/1273914874 conn(0x7f272c06b390 msgr2=0x7f272c0b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.640+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 shutdown_connections 2023-07-13T22:23:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.640+0000 7f27319ff700 1 -- 172.21.15.60:0/1273914874 wait complete. 2023-07-13T22:23:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2023-07-13T22:23:18.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:23:18.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.859+0000 7fac287ff700 1 Processor -- start 2023-07-13T22:23:18.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.860+0000 7fac287ff700 1 -- start start 2023-07-13T22:23:18.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac287ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fac200b8400 0x7fac200cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac287ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fac200d3750 0x7fac200cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac287ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200cfad0 0x7fac2015fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac287ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac200cff50 con 0x7fac200b8400 2023-07-13T22:23:18.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac287ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac200d0ac0 con 0x7fac200d3750 2023-07-13T22:23:18.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac287ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac200b87d0 con 0x7fac200cfad0 2023-07-13T22:23:18.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac2659b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fac200b8400 0x7fac200cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac2659b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fac200b8400 0x7fac200cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34716/0 (socket says 172.21.15.60:34716) 2023-07-13T22:23:18.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac2659b700 1 -- 172.21.15.60:0/2841020976 learned_addr learned my addr 172.21.15.60:0/2841020976 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:18.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac26d9c700 1 --2- 172.21.15.60:0/2841020976 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200cfad0 0x7fac2015fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac25d9a700 1 --2- 172.21.15.60:0/2841020976 >> v2:172.21.15.105:3300/0 conn(0x7fac200d3750 0x7fac200cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac2659b700 1 -- 172.21.15.60:0/2841020976 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200cfad0 msgr2=0x7fac2015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.861+0000 7fac2659b700 1 --2- 172.21.15.60:0/2841020976 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200cfad0 0x7fac2015fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.862+0000 7fac2659b700 1 -- 172.21.15.60:0/2841020976 >> v2:172.21.15.105:3300/0 conn(0x7fac200d3750 msgr2=0x7fac200cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.862+0000 7fac2659b700 1 --2- 172.21.15.60:0/2841020976 >> v2:172.21.15.105:3300/0 conn(0x7fac200d3750 0x7fac200cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.862+0000 7fac2659b700 1 -- 172.21.15.60:0/2841020976 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200b4eb0 con 0x7fac200b8400 2023-07-13T22:23:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.862+0000 7fac2659b700 1 --2- 172.21.15.60:0/2841020976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200b8400 0x7fac200cd240 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fac200b8cf0 tx=0x7fac1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495d4e0073e41f7f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.862+0000 7fac25599700 1 -- 172.21.15.60:0/2841020976 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00b870 con 0x7fac200b8400 2023-07-13T22:23:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.862+0000 7fac25599700 1 -- 172.21.15.60:0/2841020976 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c00b9d0 con 0x7fac200b8400 2023-07-13T22:23:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.862+0000 7fac25599700 1 -- 172.21.15.60:0/2841020976 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00bd60 con 0x7fac200b8400 2023-07-13T22:23:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.863+0000 7fac287ff700 1 -- 172.21.15.60:0/2841020976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200b8400 msgr2=0x7fac200cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.863+0000 7fac287ff700 1 --2- 172.21.15.60:0/2841020976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200b8400 0x7fac200cd240 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fac200b8cf0 tx=0x7fac1c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.863+0000 7fac287ff700 1 -- 172.21.15.60:0/2841020976 shutdown_connections 2023-07-13T22:23:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.863+0000 7fac287ff700 1 --2- 172.21.15.60:0/2841020976 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200cfad0 0x7fac2015fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.863+0000 7fac287ff700 1 --2- 172.21.15.60:0/2841020976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200b8400 0x7fac200cd240 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.863+0000 7fac287ff700 1 --2- 172.21.15.60:0/2841020976 >> v2:172.21.15.105:3300/0 conn(0x7fac200d3750 0x7fac200cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.863+0000 7fac287ff700 1 -- 172.21.15.60:0/2841020976 >> 172.21.15.60:0/2841020976 conn(0x7fac2006b390 msgr2=0x7fac2006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.863+0000 7fac287ff700 1 -- 172.21.15.60:0/2841020976 shutdown_connections 2023-07-13T22:23:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.863+0000 7fac287ff700 1 -- 172.21.15.60:0/2841020976 wait complete. 2023-07-13T22:23:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.865+0000 7fac287ff700 1 Processor -- start 2023-07-13T22:23:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.865+0000 7fac287ff700 1 -- start start 2023-07-13T22:23:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac287ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200b8400 0x7fac2014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac287ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200cfad0 0x7fac200c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac287ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac200d3750 0x7fac200c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac287ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2014d4e0 con 0x7fac200cfad0 2023-07-13T22:23:18.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac287ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200c89c0 con 0x7fac200d3750 2023-07-13T22:23:18.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac287ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac200ca7a0 con 0x7fac200b8400 2023-07-13T22:23:18.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac25d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200cfad0 0x7fac200c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac25d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200cfad0 0x7fac200c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34730/0 (socket says 172.21.15.60:34730) 2023-07-13T22:23:18.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac25d9a700 1 -- 172.21.15.60:0/1908007405 learned_addr learned my addr 172.21.15.60:0/1908007405 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:18.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac2659b700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200b8400 0x7fac2014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac26d9c700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac200d3750 0x7fac200c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac25d9a700 1 -- 172.21.15.60:0/1908007405 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200b8400 msgr2=0x7fac2014cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac25d9a700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200b8400 0x7fac2014cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac25d9a700 1 -- 172.21.15.60:0/1908007405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac200d3750 msgr2=0x7fac200c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac25d9a700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac200d3750 0x7fac200c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac25d9a700 1 -- 172.21.15.60:0/1908007405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009550 con 0x7fac200cfad0 2023-07-13T22:23:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.866+0000 7fac26d9c700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac200d3750 0x7fac200c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.867+0000 7fac25d9a700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200cfad0 0x7fac200c8480 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d800 tx=0x7fac1000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.867+0000 7fac16ffd700 1 -- 172.21.15.60:0/1908007405 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100125f0 con 0x7fac200cfad0 2023-07-13T22:23:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.867+0000 7fac16ffd700 1 -- 172.21.15.60:0/1908007405 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac10012750 con 0x7fac200cfad0 2023-07-13T22:23:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.867+0000 7fac16ffd700 1 -- 172.21.15.60:0/1908007405 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10012ae0 con 0x7fac200cfad0 2023-07-13T22:23:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.867+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac200c79a0 con 0x7fac200cfad0 2023-07-13T22:23:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.867+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac200c7f60 con 0x7fac200cfad0 2023-07-13T22:23:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.868+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac200c8120 con 0x7fac200cfad0 2023-07-13T22:23:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.869+0000 7fac16ffd700 1 -- 172.21.15.60:0/1908007405 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac10012c40 con 0x7fac200cfad0 2023-07-13T22:23:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.870+0000 7fac16ffd700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fac0406ace0 0x7fac0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.870+0000 7fac16ffd700 1 -- 172.21.15.60:0/1908007405 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fac100895e0 con 0x7fac200cfad0 2023-07-13T22:23:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.870+0000 7fac2659b700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fac0406ace0 0x7fac0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.871+0000 7fac2659b700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fac0406ace0 0x7fac0406d190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac1c014040 tx=0x7fac1c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:18.875+0000 7fac16ffd700 1 -- 172.21.15.60:0/1908007405 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fac100546a0 con 0x7fac200cfad0 2023-07-13T22:23:19.060 INFO:tasks.ceph.mds.d.smithi105.stderr:2023-07-13T22:23:19.077+0000 7f16239f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:23:19.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.084+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac2015abf0 con 0x7fac200cfad0 2023-07-13T22:23:19.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.085+0000 7fac16ffd700 1 -- 172.21.15.60:0/1908007405 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18257 (secure 0 0 0) 0x7fac10057cc0 con 0x7fac200cfad0 2023-07-13T22:23:19.069 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:19.069 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:16.111947+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:23:19.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fac0406ace0 msgr2=0x7fac0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fac0406ace0 0x7fac0406d190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac1c014040 tx=0x7fac1c008030 comp rx=0 tx=0).stop 2023-07-13T22:23:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200cfad0 msgr2=0x7fac200c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200cfad0 0x7fac200c8480 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d800 tx=0x7fac1000db10 comp rx=0 tx=0).stop 2023-07-13T22:23:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 shutdown_connections 2023-07-13T22:23:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fac0406ace0 0x7fac0406d190 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac200d3750 0x7fac200c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac200b8400 0x7fac2014cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 --2- 172.21.15.60:0/1908007405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac200cfad0 0x7fac200c8480 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 >> 172.21.15.60:0/1908007405 conn(0x7fac2006b390 msgr2=0x7fac200b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 shutdown_connections 2023-07-13T22:23:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.088+0000 7fac287ff700 1 -- 172.21.15.60:0/1908007405 wait complete. 2023-07-13T22:23:19.090 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:23:19.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:23:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.312+0000 7f55fa7ff700 1 Processor -- start 2023-07-13T22:23:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.312+0000 7f55fa7ff700 1 -- start start 2023-07-13T22:23:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.313+0000 7f55fa7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f55f40cefa0 0x7f55f406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.313+0000 7f55fa7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f40cf3f0 con 0x7f55f40cd760 2023-07-13T22:23:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.313+0000 7f55fa7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f41609a0 con 0x7f55f40b12a0 2023-07-13T22:23:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.314+0000 7f55fa7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f406a5f0 con 0x7f55f40cefa0 2023-07-13T22:23:19.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.314+0000 7f55f8d9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f55f40cd760 0x7f55f40cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47260/0 (socket says 172.21.15.60:47260) 2023-07-13T22:23:19.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.314+0000 7f55f8d9c700 1 -- 172.21.15.60:0/4168910052 learned_addr learned my addr 172.21.15.60:0/4168910052 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:19.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.314+0000 7f55f37fe700 1 --2- 172.21.15.60:0/4168910052 >> v2:172.21.15.105:3300/0 conn(0x7f55f40cefa0 0x7f55f406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.314+0000 7f55f2ffd700 1 -- 172.21.15.60:0/4168910052 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722312003 0 0) 0x7f55f40cf3f0 con 0x7f55f40cd760 2023-07-13T22:23:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.314+0000 7f55f2ffd700 1 -- 172.21.15.60:0/4168910052 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55dc0035b0 con 0x7f55f40cd760 2023-07-13T22:23:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.314+0000 7f55f37fe700 1 -- 172.21.15.60:0/4168910052 >> v1:172.21.15.105:6789/0 conn(0x7f55f40b12a0 legacy=0x7f55f40b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.314+0000 7f55f37fe700 1 -- 172.21.15.60:0/4168910052 >> v1:172.21.15.60:6789/0 conn(0x7f55f40cd760 legacy=0x7f55f40cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.315+0000 7f55f37fe700 1 -- 172.21.15.60:0/4168910052 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f415c860 con 0x7f55f40cefa0 2023-07-13T22:23:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.315+0000 7f55f37fe700 1 --2- 172.21.15.60:0/4168910052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cefa0 0x7f55f406bad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009840 tx=0x7f55e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac7b9a211bb449a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.315+0000 7f55f2ffd700 1 -- 172.21.15.60:0/4168910052 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011830 con 0x7f55f40cefa0 2023-07-13T22:23:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.315+0000 7f55f2ffd700 1 -- 172.21.15.60:0/4168910052 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e8011990 con 0x7f55f40cefa0 2023-07-13T22:23:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.315+0000 7f55f2ffd700 1 -- 172.21.15.60:0/4168910052 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011d20 con 0x7f55f40cefa0 2023-07-13T22:23:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.316+0000 7f55fa7ff700 1 -- 172.21.15.60:0/4168910052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cefa0 msgr2=0x7f55f406bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.316+0000 7f55fa7ff700 1 --2- 172.21.15.60:0/4168910052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cefa0 0x7f55f406bad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009840 tx=0x7f55e8009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.316+0000 7f55fa7ff700 1 -- 172.21.15.60:0/4168910052 shutdown_connections 2023-07-13T22:23:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.316+0000 7f55fa7ff700 1 --2- 172.21.15.60:0/4168910052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cefa0 0x7f55f406bad0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.316+0000 7f55fa7ff700 1 -- 172.21.15.60:0/4168910052 >> 172.21.15.60:0/4168910052 conn(0x7f55f40cc2a0 msgr2=0x7f55f40cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.316+0000 7f55fa7ff700 1 -- 172.21.15.60:0/4168910052 shutdown_connections 2023-07-13T22:23:19.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.316+0000 7f55fa7ff700 1 -- 172.21.15.60:0/4168910052 wait complete. 2023-07-13T22:23:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.318+0000 7f55fa7ff700 1 Processor -- start 2023-07-13T22:23:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.318+0000 7f55fa7ff700 1 -- start start 2023-07-13T22:23:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55fa7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f40b12a0 0x7f55f40c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55fa7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cd760 0x7f55f40c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55fa7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f40cefa0 0x7f55f415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55fa7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40c34f0 con 0x7f55f40cefa0 2023-07-13T22:23:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55fa7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40c7470 con 0x7f55f40cd760 2023-07-13T22:23:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55fa7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40c7750 con 0x7f55f40b12a0 2023-07-13T22:23:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55f8d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f40cefa0 0x7f55f415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55f8d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f40cefa0 0x7f55f415ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34736/0 (socket says 172.21.15.60:34736) 2023-07-13T22:23:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55f8d9c700 1 -- 172.21.15.60:0/1523624393 learned_addr learned my addr 172.21.15.60:0/1523624393 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55f3fff700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f40b12a0 0x7f55f40c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.319+0000 7f55f37fe700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cd760 0x7f55f40c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55f8d9c700 1 -- 172.21.15.60:0/1523624393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f40b12a0 msgr2=0x7f55f40c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55f8d9c700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f40b12a0 0x7f55f40c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55f8d9c700 1 -- 172.21.15.60:0/1523624393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cd760 msgr2=0x7f55f40c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55f8d9c700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cd760 0x7f55f40c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55f8d9c700 1 -- 172.21.15.60:0/1523624393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e80094a0 con 0x7f55f40cefa0 2023-07-13T22:23:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55f8d9c700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f40cefa0 0x7f55f415ec20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00cb40 tx=0x7f55ec00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55f0ff9700 1 -- 172.21.15.60:0/1523624393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00a9e0 con 0x7f55f40cefa0 2023-07-13T22:23:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55f0ff9700 1 -- 172.21.15.60:0/1523624393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec00ab40 con 0x7f55f40cefa0 2023-07-13T22:23:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55f0ff9700 1 -- 172.21.15.60:0/1523624393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec012400 con 0x7f55f40cefa0 2023-07-13T22:23:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40c7a30 con 0x7f55f40cefa0 2023-07-13T22:23:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.320+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f40c7d10 con 0x7f55f40cefa0 2023-07-13T22:23:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.321+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f40c3630 con 0x7f55f40cefa0 2023-07-13T22:23:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.322+0000 7f55f0ff9700 1 -- 172.21.15.60:0/1523624393 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f55ec01b3b0 con 0x7f55f40cefa0 2023-07-13T22:23:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.324+0000 7f55f0ff9700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f55e006ace0 0x7f55e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.324+0000 7f55f0ff9700 1 -- 172.21.15.60:0/1523624393 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f55ec089080 con 0x7f55f40cefa0 2023-07-13T22:23:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.324+0000 7f55f3fff700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f55e006ace0 0x7f55e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.324+0000 7f55f3fff700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f55e006ace0 0x7f55e006d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55f40c9f90 tx=0x7f55e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:19.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.328+0000 7f55f0ff9700 1 -- 172.21.15.60:0/1523624393 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f55ec0bb330 con 0x7f55f40cefa0 2023-07-13T22:23:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.529+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55f406a070 con 0x7f55e006ace0 2023-07-13T22:23:19.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.539+0000 7f55f0ff9700 1 -- 172.21.15.60:0/1523624393 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311684 (secure 0 0 0) 0x7f55e4001cc0 con 0x7f55e006ace0 2023-07-13T22:23:19.526 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f55e006ace0 msgr2=0x7f55e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f55e006ace0 0x7f55e006d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55f40c9f90 tx=0x7f55e4009040 comp rx=0 tx=0).stop 2023-07-13T22:23:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f40cefa0 msgr2=0x7f55f415ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f40cefa0 0x7f55f415ec20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00cb40 tx=0x7f55ec00ce50 comp rx=0 tx=0).stop 2023-07-13T22:23:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 shutdown_connections 2023-07-13T22:23:19.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f55e006ace0 0x7f55e006d190 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f40cefa0 0x7f55f415ec20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f40b12a0 0x7f55f40c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 --2- 172.21.15.60:0/1523624393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cd760 0x7f55f40c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 >> 172.21.15.60:0/1523624393 conn(0x7f55f40cc2a0 msgr2=0x7f55f40b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:19.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 shutdown_connections 2023-07-13T22:23:19.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.544+0000 7f55fa7ff700 1 -- 172.21.15.60:0/1523624393 wait complete. 2023-07-13T22:23:19.530 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:23:19.546 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-07-13T22:23:19.153075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249920,"kb_used_data":5632,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754724800,"statfs":{"total":773094113280,"available":772838195200,"internally_reserved":0,"allocated":5767168,"data_stored":3136512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001252"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119181+0000","last_change":"2023-07-13T22:23:15.117911+0000","last_active":"2023-07-13T22:23:16.119181+0000","last_peered":"2023-07-13T22:23:16.119181+0000","last_clean":"2023-07-13T22:23:16.119181+0000","last_became_active":"2023-07-13T22:23:15.117580+0000","last_became_peered":"2023-07-13T22:23:15.117580+0000","last_unstale":"2023-07-13T22:23:16.119181+0000","last_undegraded":"2023-07-13T22:23:16.119181+0000","last_fullsized":"2023-07-13T22:23:16.119181+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:25.640506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.126180+0000","last_change":"2023-07-13T22:23:15.122044+0000","last_active":"2023-07-13T22:23:15.126180+0000","last_peered":"2023-07-13T22:23:15.126180+0000","last_clean":"2023-07-13T22:23:15.126180+0000","last_became_active":"2023-07-13T22:23:15.121730+0000","last_became_peered":"2023-07-13T22:23:15.121730+0000","last_unstale":"2023-07-13T22:23:15.126180+0000","last_undegraded":"2023-07-13T22:23:15.126180+0000","last_fullsized":"2023-07-13T22:23:15.126180+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:52:03.136297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121888+0000","last_change":"2023-07-13T22:23:15.133128+0000","last_active":"2023-07-13T22:23:16.121888+0000","last_peered":"2023-07-13T22:23:16.121888+0000","last_clean":"2023-07-13T22:23:16.121888+0000","last_became_active":"2023-07-13T22:23:15.131529+0000","last_became_peered":"2023-07-13T22:23:15.131529+0000","last_unstale":"2023-07-13T22:23:16.121888+0000","last_undegraded":"2023-07-13T22:23:16.121888+0000","last_fullsized":"2023-07-13T22:23:16.121888+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:55:22.011271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.122942+0000","last_change":"2023-07-13T22:23:15.119521+0000","last_active":"2023-07-13T22:23:15.122942+0000","last_peered":"2023-07-13T22:23:15.122942+0000","last_clean":"2023-07-13T22:23:15.122942+0000","last_became_active":"2023-07-13T22:23:15.119226+0000","last_became_peered":"2023-07-13T22:23:15.119226+0000","last_unstale":"2023-07-13T22:23:15.122942+0000","last_undegraded":"2023-07-13T22:23:15.122942+0000","last_fullsized":"2023-07-13T22:23:15.122942+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:18:44.220318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130436+0000","last_change":"2023-07-13T22:23:15.123929+0000","last_active":"2023-07-13T22:23:15.130436+0000","last_peered":"2023-07-13T22:23:15.130436+0000","last_clean":"2023-07-13T22:23:15.130436+0000","last_became_active":"2023-07-13T22:23:15.122938+0000","last_became_peered":"2023-07-13T22:23:15.122938+0000","last_unstale":"2023-07-13T22:23:15.130436+0000","last_undegraded":"2023-07-13T22:23:15.130436+0000","last_fullsized":"2023-07-13T22:23:15.130436+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:21:07.140335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119197+0000","last_change":"2023-07-13T22:23:15.122245+0000","last_active":"2023-07-13T22:23:16.119197+0000","last_peered":"2023-07-13T22:23:16.119197+0000","last_clean":"2023-07-13T22:23:16.119197+0000","last_became_active":"2023-07-13T22:23:15.121966+0000","last_became_peered":"2023-07-13T22:23:15.121966+0000","last_unstale":"2023-07-13T22:23:16.119197+0000","last_undegraded":"2023-07-13T22:23:16.119197+0000","last_fullsized":"2023-07-13T22:23:16.119197+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:27:19.284955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121035+0000","last_change":"2023-07-13T22:23:15.120446+0000","last_active":"2023-07-13T22:23:16.121035+0000","last_peered":"2023-07-13T22:23:16.121035+0000","last_clean":"2023-07-13T22:23:16.121035+0000","last_became_active":"2023-07-13T22:23:15.119801+0000","last_became_peered":"2023-07-13T22:23:15.119801+0000","last_unstale":"2023-07-13T22:23:16.121035+0000","last_undegraded":"2023-07-13T22:23:16.121035+0000","last_fullsized":"2023-07-13T22:23:16.121035+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:46:43.573246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129672+0000","last_change":"2023-07-13T22:23:15.122196+0000","last_active":"2023-07-13T22:23:15.129672+0000","last_peered":"2023-07-13T22:23:15.129672+0000","last_clean":"2023-07-13T22:23:15.129672+0000","last_became_active":"2023-07-13T22:23:15.121909+0000","last_became_peered":"2023-07-13T22:23:15.121909+0000","last_unstale":"2023-07-13T22:23:15.129672+0000","last_undegraded":"2023-07-13T22:23:15.129672+0000","last_fullsized":"2023-07-13T22:23:15.129672+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:06.365938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.136773+0000","last_change":"2023-07-13T22:23:15.134981+0000","last_active":"2023-07-13T22:23:15.136773+0000","last_peered":"2023-07-13T22:23:15.136773+0000","last_clean":"2023-07-13T22:23:15.136773+0000","last_became_active":"2023-07-13T22:23:15.134456+0000","last_became_peered":"2023-07-13T22:23:15.134456+0000","last_unstale":"2023-07-13T22:23:15.136773+0000","last_undegraded":"2023-07-13T22:23:15.136773+0000","last_fullsized":"2023-07-13T22:23:15.136773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:19:36.133927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119545+0000","last_change":"2023-07-13T22:23:15.128219+0000","last_active":"2023-07-13T22:23:16.119545+0000","last_peered":"2023-07-13T22:23:16.119545+0000","last_clean":"2023-07-13T22:23:16.119545+0000","last_became_active":"2023-07-13T22:23:15.127923+0000","last_became_peered":"2023-07-13T22:23:15.127923+0000","last_unstale":"2023-07-13T22:23:16.119545+0000","last_undegraded":"2023-07-13T22:23:16.119545+0000","last_fullsized":"2023-07-13T22:23:16.119545+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:12:10.313272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132227+0000","last_change":"2023-07-13T22:23:15.121681+0000","last_active":"2023-07-13T22:23:15.132227+0000","last_peered":"2023-07-13T22:23:15.132227+0000","last_clean":"2023-07-13T22:23:15.132227+0000","last_became_active":"2023-07-13T22:23:15.121394+0000","last_became_peered":"2023-07-13T22:23:15.121394+0000","last_unstale":"2023-07-13T22:23:15.132227+0000","last_undegraded":"2023-07-13T22:23:15.132227+0000","last_fullsized":"2023-07-13T22:23:15.132227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:58:23.109679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131280+0000","last_change":"2023-07-13T22:23:15.128977+0000","last_active":"2023-07-13T22:23:15.131280+0000","last_peered":"2023-07-13T22:23:15.131280+0000","last_clean":"2023-07-13T22:23:15.131280+0000","last_became_active":"2023-07-13T22:23:15.128696+0000","last_became_peered":"2023-07-13T22:23:15.128696+0000","last_unstale":"2023-07-13T22:23:15.131280+0000","last_undegraded":"2023-07-13T22:23:15.131280+0000","last_fullsized":"2023-07-13T22:23:15.131280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:51:09.589836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119182+0000","last_change":"2023-07-13T22:23:15.128052+0000","last_active":"2023-07-13T22:23:16.119182+0000","last_peered":"2023-07-13T22:23:16.119182+0000","last_clean":"2023-07-13T22:23:16.119182+0000","last_became_active":"2023-07-13T22:23:15.127748+0000","last_became_peered":"2023-07-13T22:23:15.127748+0000","last_unstale":"2023-07-13T22:23:16.119182+0000","last_undegraded":"2023-07-13T22:23:16.119182+0000","last_fullsized":"2023-07-13T22:23:16.119182+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:09:03.155046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121338+0000","last_change":"2023-07-13T22:23:15.129902+0000","last_active":"2023-07-13T22:23:16.121338+0000","last_peered":"2023-07-13T22:23:16.121338+0000","last_clean":"2023-07-13T22:23:16.121338+0000","last_became_active":"2023-07-13T22:23:15.129531+0000","last_became_peered":"2023-07-13T22:23:15.129531+0000","last_unstale":"2023-07-13T22:23:16.121338+0000","last_undegraded":"2023-07-13T22:23:16.121338+0000","last_fullsized":"2023-07-13T22:23:16.121338+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:02.410960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132400+0000","last_change":"2023-07-13T22:23:15.129452+0000","last_active":"2023-07-13T22:23:15.132400+0000","last_peered":"2023-07-13T22:23:15.132400+0000","last_clean":"2023-07-13T22:23:15.132400+0000","last_became_active":"2023-07-13T22:23:15.129183+0000","last_became_peered":"2023-07-13T22:23:15.129183+0000","last_unstale":"2023-07-13T22:23:15.132400+0000","last_undegraded":"2023-07-13T22:23:15.132400+0000","last_fullsized":"2023-07-13T22:23:15.132400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:02.651843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.118996+0000","last_change":"2023-07-13T22:23:15.124772+0000","last_active":"2023-07-13T22:23:16.118996+0000","last_peered":"2023-07-13T22:23:16.118996+0000","last_clean":"2023-07-13T22:23:16.118996+0000","last_became_active":"2023-07-13T22:23:15.124489+0000","last_became_peered":"2023-07-13T22:23:15.124489+0000","last_unstale":"2023-07-13T22:23:16.118996+0000","last_undegraded":"2023-07-13T22:23:16.118996+0000","last_fullsized":"2023-07-13T22:23:16.118996+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:01:25.892366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.122146+0000","last_change":"2023-07-13T22:23:15.132250+0000","last_active":"2023-07-13T22:23:16.122146+0000","last_peered":"2023-07-13T22:23:16.122146+0000","last_clean":"2023-07-13T22:23:16.122146+0000","last_became_active":"2023-07-13T22:23:15.129701+0000","last_became_peered":"2023-07-13T22:23:15.129701+0000","last_unstale":"2023-07-13T22:23:16.122146+0000","last_undegraded":"2023-07-13T22:23:16.122146+0000","last_fullsized":"2023-07-13T22:23:16.122146+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:07.169138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.127055+0000","last_change":"2023-07-13T22:23:15.123375+0000","last_active":"2023-07-13T22:23:15.127055+0000","last_peered":"2023-07-13T22:23:15.127055+0000","last_clean":"2023-07-13T22:23:15.127055+0000","last_became_active":"2023-07-13T22:23:15.123109+0000","last_became_peered":"2023-07-13T22:23:15.123109+0000","last_unstale":"2023-07-13T22:23:15.127055+0000","last_undegraded":"2023-07-13T22:23:15.127055+0000","last_fullsized":"2023-07-13T22:23:15.127055+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:39:10.832174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121968+0000","last_change":"2023-07-13T22:23:15.133131+0000","last_active":"2023-07-13T22:23:16.121968+0000","last_peered":"2023-07-13T22:23:16.121968+0000","last_clean":"2023-07-13T22:23:16.121968+0000","last_became_active":"2023-07-13T22:23:15.130321+0000","last_became_peered":"2023-07-13T22:23:15.130321+0000","last_unstale":"2023-07-13T22:23:16.121968+0000","last_undegraded":"2023-07-13T22:23:16.121968+0000","last_fullsized":"2023-07-13T22:23:16.121968+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:19:22.881968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131857+0000","last_change":"2023-07-13T22:23:15.128934+0000","last_active":"2023-07-13T22:23:15.131857+0000","last_peered":"2023-07-13T22:23:15.131857+0000","last_clean":"2023-07-13T22:23:15.131857+0000","last_became_active":"2023-07-13T22:23:15.128058+0000","last_became_peered":"2023-07-13T22:23:15.128058+0000","last_unstale":"2023-07-13T22:23:15.131857+0000","last_undegraded":"2023-07-13T22:23:15.131857+0000","last_fullsized":"2023-07-13T22:23:15.131857+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:08:47.550844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131897+0000","last_change":"2023-07-13T22:23:15.128972+0000","last_active":"2023-07-13T22:23:15.131897+0000","last_peered":"2023-07-13T22:23:15.131897+0000","last_clean":"2023-07-13T22:23:15.131897+0000","last_became_active":"2023-07-13T22:23:15.128664+0000","last_became_peered":"2023-07-13T22:23:15.128664+0000","last_unstale":"2023-07-13T22:23:15.131897+0000","last_undegraded":"2023-07-13T22:23:15.131897+0000","last_fullsized":"2023-07-13T22:23:15.131897+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:43.829071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.136734+0000","last_change":"2023-07-13T22:23:15.134809+0000","last_active":"2023-07-13T22:23:15.136734+0000","last_peered":"2023-07-13T22:23:15.136734+0000","last_clean":"2023-07-13T22:23:15.136734+0000","last_became_active":"2023-07-13T22:23:15.134532+0000","last_became_peered":"2023-07-13T22:23:15.134532+0000","last_unstale":"2023-07-13T22:23:15.136734+0000","last_undegraded":"2023-07-13T22:23:15.136734+0000","last_fullsized":"2023-07-13T22:23:15.136734+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:16:19.713925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132184+0000","last_change":"2023-07-13T22:23:15.124507+0000","last_active":"2023-07-13T22:23:15.132184+0000","last_peered":"2023-07-13T22:23:15.132184+0000","last_clean":"2023-07-13T22:23:15.132184+0000","last_became_active":"2023-07-13T22:23:15.124211+0000","last_became_peered":"2023-07-13T22:23:15.124211+0000","last_unstale":"2023-07-13T22:23:15.132184+0000","last_undegraded":"2023-07-13T22:23:15.132184+0000","last_fullsized":"2023-07-13T22:23:15.132184+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:01:12.416364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120578+0000","last_change":"2023-07-13T22:23:15.120001+0000","last_active":"2023-07-13T22:23:16.120578+0000","last_peered":"2023-07-13T22:23:16.120578+0000","last_clean":"2023-07-13T22:23:16.120578+0000","last_became_active":"2023-07-13T22:23:15.119609+0000","last_became_peered":"2023-07-13T22:23:15.119609+0000","last_unstale":"2023-07-13T22:23:16.120578+0000","last_undegraded":"2023-07-13T22:23:16.120578+0000","last_fullsized":"2023-07-13T22:23:16.120578+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:51:44.861681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.122079+0000","last_change":"2023-07-13T22:23:15.132434+0000","last_active":"2023-07-13T22:23:16.122079+0000","last_peered":"2023-07-13T22:23:16.122079+0000","last_clean":"2023-07-13T22:23:16.122079+0000","last_became_active":"2023-07-13T22:23:15.130088+0000","last_became_peered":"2023-07-13T22:23:15.130088+0000","last_unstale":"2023-07-13T22:23:16.122079+0000","last_undegraded":"2023-07-13T22:23:16.122079+0000","last_fullsized":"2023-07-13T22:23:16.122079+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:22:22.174758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129752+0000","last_change":"2023-07-13T22:23:15.118112+0000","last_active":"2023-07-13T22:23:15.129752+0000","last_peered":"2023-07-13T22:23:15.129752+0000","last_clean":"2023-07-13T22:23:15.129752+0000","last_became_active":"2023-07-13T22:23:15.117813+0000","last_became_peered":"2023-07-13T22:23:15.117813+0000","last_unstale":"2023-07-13T22:23:15.129752+0000","last_undegraded":"2023-07-13T22:23:15.129752+0000","last_fullsized":"2023-07-13T22:23:15.129752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:18:44.220318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119457+0000","last_change":"2023-07-13T22:23:15.134875+0000","last_active":"2023-07-13T22:23:16.119457+0000","last_peered":"2023-07-13T22:23:16.119457+0000","last_clean":"2023-07-13T22:23:16.119457+0000","last_became_active":"2023-07-13T22:23:15.134483+0000","last_became_peered":"2023-07-13T22:23:15.134483+0000","last_unstale":"2023-07-13T22:23:16.119457+0000","last_undegraded":"2023-07-13T22:23:16.119457+0000","last_fullsized":"2023-07-13T22:23:16.119457+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:50:30.456993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129163+0000","last_change":"2023-07-13T22:23:15.122265+0000","last_active":"2023-07-13T22:23:15.129163+0000","last_peered":"2023-07-13T22:23:15.129163+0000","last_clean":"2023-07-13T22:23:15.129163+0000","last_became_active":"2023-07-13T22:23:15.121963+0000","last_became_peered":"2023-07-13T22:23:15.121963+0000","last_unstale":"2023-07-13T22:23:15.129163+0000","last_undegraded":"2023-07-13T22:23:15.129163+0000","last_fullsized":"2023-07-13T22:23:15.129163+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:42:27.411239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121397+0000","last_change":"2023-07-13T22:23:15.136420+0000","last_active":"2023-07-13T22:23:16.121397+0000","last_peered":"2023-07-13T22:23:16.121397+0000","last_clean":"2023-07-13T22:23:16.121397+0000","last_became_active":"2023-07-13T22:23:15.136117+0000","last_became_peered":"2023-07-13T22:23:15.136117+0000","last_unstale":"2023-07-13T22:23:16.121397+0000","last_undegraded":"2023-07-13T22:23:16.121397+0000","last_fullsized":"2023-07-13T22:23:16.121397+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:13.313970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.124799+0000","last_change":"2023-07-13T22:23:15.120511+0000","last_active":"2023-07-13T22:23:15.124799+0000","last_peered":"2023-07-13T22:23:15.124799+0000","last_clean":"2023-07-13T22:23:15.124799+0000","last_became_active":"2023-07-13T22:23:15.120168+0000","last_became_peered":"2023-07-13T22:23:15.120168+0000","last_unstale":"2023-07-13T22:23:15.124799+0000","last_undegraded":"2023-07-13T22:23:15.124799+0000","last_fullsized":"2023-07-13T22:23:15.124799+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:52:10.055506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120981+0000","last_change":"2023-07-13T22:23:15.120661+0000","last_active":"2023-07-13T22:23:16.120981+0000","last_peered":"2023-07-13T22:23:16.120981+0000","last_clean":"2023-07-13T22:23:16.120981+0000","last_became_active":"2023-07-13T22:23:15.120126+0000","last_became_peered":"2023-07-13T22:23:15.120126+0000","last_unstale":"2023-07-13T22:23:16.120981+0000","last_undegraded":"2023-07-13T22:23:16.120981+0000","last_fullsized":"2023-07-13T22:23:16.120981+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:40:54.737424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120280+0000","last_change":"2023-07-13T22:23:15.119079+0000","last_active":"2023-07-13T22:23:16.120280+0000","last_peered":"2023-07-13T22:23:16.120280+0000","last_clean":"2023-07-13T22:23:16.120280+0000","last_became_active":"2023-07-13T22:23:15.118779+0000","last_became_peered":"2023-07-13T22:23:15.118779+0000","last_unstale":"2023-07-13T22:23:16.120280+0000","last_undegraded":"2023-07-13T22:23:16.120280+0000","last_fullsized":"2023-07-13T22:23:16.120280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:48:32.606922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.123577+0000","last_change":"2023-07-13T22:23:15.121041+0000","last_active":"2023-07-13T22:23:15.123577+0000","last_peered":"2023-07-13T22:23:15.123577+0000","last_clean":"2023-07-13T22:23:15.123577+0000","last_became_active":"2023-07-13T22:23:15.120764+0000","last_became_peered":"2023-07-13T22:23:15.120764+0000","last_unstale":"2023-07-13T22:23:15.123577+0000","last_undegraded":"2023-07-13T22:23:15.123577+0000","last_fullsized":"2023-07-13T22:23:15.123577+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:28:01.610676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131216+0000","last_change":"2023-07-13T22:23:14.117111+0000","last_active":"2023-07-13T22:23:15.131216+0000","last_peered":"2023-07-13T22:23:15.131216+0000","last_clean":"2023-07-13T22:23:15.131216+0000","last_became_active":"2023-07-13T22:23:14.116811+0000","last_became_peered":"2023-07-13T22:23:14.116811+0000","last_unstale":"2023-07-13T22:23:15.131216+0000","last_undegraded":"2023-07-13T22:23:15.131216+0000","last_fullsized":"2023-07-13T22:23:15.131216+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:32:27.692851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120594+0000","last_change":"2023-07-13T22:23:15.126077+0000","last_active":"2023-07-13T22:23:16.120594+0000","last_peered":"2023-07-13T22:23:16.120594+0000","last_clean":"2023-07-13T22:23:16.120594+0000","last_became_active":"2023-07-13T22:23:15.125764+0000","last_became_peered":"2023-07-13T22:23:15.125764+0000","last_unstale":"2023-07-13T22:23:16.120594+0000","last_undegraded":"2023-07-13T22:23:16.120594+0000","last_fullsized":"2023-07-13T22:23:16.120594+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:29:06.882102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119651+0000","last_change":"2023-07-13T22:23:14.110977+0000","last_active":"2023-07-13T22:23:16.119651+0000","last_peered":"2023-07-13T22:23:16.119651+0000","last_clean":"2023-07-13T22:23:16.119651+0000","last_became_active":"2023-07-13T22:23:14.110679+0000","last_became_peered":"2023-07-13T22:23:14.110679+0000","last_unstale":"2023-07-13T22:23:16.119651+0000","last_undegraded":"2023-07-13T22:23:16.119651+0000","last_fullsized":"2023-07-13T22:23:16.119651+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:36:30.987495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130389+0000","last_change":"2023-07-13T22:23:15.128817+0000","last_active":"2023-07-13T22:23:15.130389+0000","last_peered":"2023-07-13T22:23:15.130389+0000","last_clean":"2023-07-13T22:23:15.130389+0000","last_became_active":"2023-07-13T22:23:15.128165+0000","last_became_peered":"2023-07-13T22:23:15.128165+0000","last_unstale":"2023-07-13T22:23:15.130389+0000","last_undegraded":"2023-07-13T22:23:15.130389+0000","last_fullsized":"2023-07-13T22:23:15.130389+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:54:13.910242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119560+0000","last_change":"2023-07-13T22:23:14.120162+0000","last_active":"2023-07-13T22:23:16.119560+0000","last_peered":"2023-07-13T22:23:16.119560+0000","last_clean":"2023-07-13T22:23:16.119560+0000","last_became_active":"2023-07-13T22:23:14.119783+0000","last_became_peered":"2023-07-13T22:23:14.119783+0000","last_unstale":"2023-07-13T22:23:16.119560+0000","last_undegraded":"2023-07-13T22:23:16.119560+0000","last_fullsized":"2023-07-13T22:23:16.119560+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:36:06.944629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.126229+0000","last_change":"2023-07-13T22:23:15.113836+0000","last_active":"2023-07-13T22:23:15.126229+0000","last_peered":"2023-07-13T22:23:15.126229+0000","last_clean":"2023-07-13T22:23:15.126229+0000","last_became_active":"2023-07-13T22:23:15.113515+0000","last_became_peered":"2023-07-13T22:23:15.113515+0000","last_unstale":"2023-07-13T22:23:15.126229+0000","last_undegraded":"2023-07-13T22:23:15.126229+0000","last_fullsized":"2023-07-13T22:23:15.126229+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:45:41.534962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120558+0000","last_change":"2023-07-13T22:23:14.114670+0000","last_active":"2023-07-13T22:23:16.120558+0000","last_peered":"2023-07-13T22:23:16.120558+0000","last_clean":"2023-07-13T22:23:16.120558+0000","last_became_active":"2023-07-13T22:23:14.114412+0000","last_became_peered":"2023-07-13T22:23:14.114412+0000","last_unstale":"2023-07-13T22:23:16.120558+0000","last_undegraded":"2023-07-13T22:23:16.120558+0000","last_fullsized":"2023-07-13T22:23:16.120558+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:10:59.775619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.133133+0000","last_change":"2023-07-13T22:23:15.131330+0000","last_active":"2023-07-13T22:23:15.133133+0000","last_peered":"2023-07-13T22:23:15.133133+0000","last_clean":"2023-07-13T22:23:15.133133+0000","last_became_active":"2023-07-13T22:23:15.131040+0000","last_became_peered":"2023-07-13T22:23:15.131040+0000","last_unstale":"2023-07-13T22:23:15.133133+0000","last_undegraded":"2023-07-13T22:23:15.133133+0000","last_fullsized":"2023-07-13T22:23:15.133133+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:47.330951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121398+0000","last_change":"2023-07-13T22:23:14.119598+0000","last_active":"2023-07-13T22:23:16.121398+0000","last_peered":"2023-07-13T22:23:16.121398+0000","last_clean":"2023-07-13T22:23:16.121398+0000","last_became_active":"2023-07-13T22:23:14.119167+0000","last_became_peered":"2023-07-13T22:23:14.119167+0000","last_unstale":"2023-07-13T22:23:16.121398+0000","last_undegraded":"2023-07-13T22:23:16.121398+0000","last_fullsized":"2023-07-13T22:23:16.121398+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:09:40.697987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.127332+0000","last_change":"2023-07-13T22:23:15.124471+0000","last_active":"2023-07-13T22:23:15.127332+0000","last_peered":"2023-07-13T22:23:15.127332+0000","last_clean":"2023-07-13T22:23:15.127332+0000","last_became_active":"2023-07-13T22:23:15.124186+0000","last_became_peered":"2023-07-13T22:23:15.124186+0000","last_unstale":"2023-07-13T22:23:15.127332+0000","last_undegraded":"2023-07-13T22:23:15.127332+0000","last_fullsized":"2023-07-13T22:23:15.127332+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:31:45.719375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130725+0000","last_change":"2023-07-13T22:23:14.120205+0000","last_active":"2023-07-13T22:23:15.130725+0000","last_peered":"2023-07-13T22:23:15.130725+0000","last_clean":"2023-07-13T22:23:15.130725+0000","last_became_active":"2023-07-13T22:23:14.119846+0000","last_became_peered":"2023-07-13T22:23:14.119846+0000","last_unstale":"2023-07-13T22:23:15.130725+0000","last_undegraded":"2023-07-13T22:23:15.130725+0000","last_fullsized":"2023-07-13T22:23:15.130725+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:14:20.398352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.127243+0000","last_change":"2023-07-13T22:23:15.113100+0000","last_active":"2023-07-13T22:23:15.127243+0000","last_peered":"2023-07-13T22:23:15.127243+0000","last_clean":"2023-07-13T22:23:15.127243+0000","last_became_active":"2023-07-13T22:23:15.112763+0000","last_became_peered":"2023-07-13T22:23:15.112763+0000","last_unstale":"2023-07-13T22:23:15.127243+0000","last_undegraded":"2023-07-13T22:23:15.127243+0000","last_fullsized":"2023-07-13T22:23:15.127243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:13:52.313599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.108115+0000","last_change":"2023-07-13T22:23:14.116977+0000","last_active":"2023-07-13T22:23:15.108115+0000","last_peered":"2023-07-13T22:23:15.108115+0000","last_clean":"2023-07-13T22:23:15.108115+0000","last_became_active":"2023-07-13T22:23:14.116679+0000","last_became_peered":"2023-07-13T22:23:14.116679+0000","last_unstale":"2023-07-13T22:23:15.108115+0000","last_undegraded":"2023-07-13T22:23:15.108115+0000","last_fullsized":"2023-07-13T22:23:15.108115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:09:14.618868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.134842+0000","last_change":"2023-07-13T22:23:15.134144+0000","last_active":"2023-07-13T22:23:15.134842+0000","last_peered":"2023-07-13T22:23:15.134842+0000","last_clean":"2023-07-13T22:23:15.134842+0000","last_became_active":"2023-07-13T22:23:15.133829+0000","last_became_peered":"2023-07-13T22:23:15.133829+0000","last_unstale":"2023-07-13T22:23:15.134842+0000","last_undegraded":"2023-07-13T22:23:15.134842+0000","last_fullsized":"2023-07-13T22:23:15.134842+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:18:54.363844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120708+0000","last_change":"2023-07-13T22:23:14.122179+0000","last_active":"2023-07-13T22:23:16.120708+0000","last_peered":"2023-07-13T22:23:16.120708+0000","last_clean":"2023-07-13T22:23:16.120708+0000","last_became_active":"2023-07-13T22:23:14.121796+0000","last_became_peered":"2023-07-13T22:23:14.121796+0000","last_unstale":"2023-07-13T22:23:16.120708+0000","last_undegraded":"2023-07-13T22:23:16.120708+0000","last_fullsized":"2023-07-13T22:23:16.120708+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:29:36.900241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130403+0000","last_change":"2023-07-13T22:23:15.123589+0000","last_active":"2023-07-13T22:23:15.130403+0000","last_peered":"2023-07-13T22:23:15.130403+0000","last_clean":"2023-07-13T22:23:15.130403+0000","last_became_active":"2023-07-13T22:23:15.123255+0000","last_became_peered":"2023-07-13T22:23:15.123255+0000","last_unstale":"2023-07-13T22:23:15.130403+0000","last_undegraded":"2023-07-13T22:23:15.130403+0000","last_fullsized":"2023-07-13T22:23:15.130403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:16:19.713925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.109475+0000","last_change":"2023-07-13T22:23:14.113874+0000","last_active":"2023-07-13T22:23:15.109475+0000","last_peered":"2023-07-13T22:23:15.109475+0000","last_clean":"2023-07-13T22:23:15.109475+0000","last_became_active":"2023-07-13T22:23:14.113571+0000","last_became_peered":"2023-07-13T22:23:14.113571+0000","last_unstale":"2023-07-13T22:23:15.109475+0000","last_undegraded":"2023-07-13T22:23:15.109475+0000","last_fullsized":"2023-07-13T22:23:15.109475+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:19:35.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120435+0000","last_change":"2023-07-13T22:23:15.125234+0000","last_active":"2023-07-13T22:23:16.120435+0000","last_peered":"2023-07-13T22:23:16.120435+0000","last_clean":"2023-07-13T22:23:16.120435+0000","last_became_active":"2023-07-13T22:23:15.124919+0000","last_became_peered":"2023-07-13T22:23:15.124919+0000","last_unstale":"2023-07-13T22:23:16.120435+0000","last_undegraded":"2023-07-13T22:23:16.120435+0000","last_fullsized":"2023-07-13T22:23:16.120435+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:15:48.975568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.105579+0000","last_change":"2023-07-13T22:23:14.120862+0000","last_active":"2023-07-13T22:23:15.105579+0000","last_peered":"2023-07-13T22:23:15.105579+0000","last_clean":"2023-07-13T22:23:15.105579+0000","last_became_active":"2023-07-13T22:23:14.120599+0000","last_became_peered":"2023-07-13T22:23:14.120599+0000","last_unstale":"2023-07-13T22:23:15.105579+0000","last_undegraded":"2023-07-13T22:23:15.105579+0000","last_fullsized":"2023-07-13T22:23:15.105579+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:51:08.582430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121450+0000","last_change":"2023-07-13T22:23:15.136457+0000","last_active":"2023-07-13T22:23:16.121450+0000","last_peered":"2023-07-13T22:23:16.121450+0000","last_clean":"2023-07-13T22:23:16.121450+0000","last_became_active":"2023-07-13T22:23:15.136166+0000","last_became_peered":"2023-07-13T22:23:15.136166+0000","last_unstale":"2023-07-13T22:23:16.121450+0000","last_undegraded":"2023-07-13T22:23:16.121450+0000","last_fullsized":"2023-07-13T22:23:16.121450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:36:01.074855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.128002+0000","last_change":"2023-07-13T22:23:14.114283+0000","last_active":"2023-07-13T22:23:15.128002+0000","last_peered":"2023-07-13T22:23:15.128002+0000","last_clean":"2023-07-13T22:23:15.128002+0000","last_became_active":"2023-07-13T22:23:14.113985+0000","last_became_peered":"2023-07-13T22:23:14.113985+0000","last_unstale":"2023-07-13T22:23:15.128002+0000","last_undegraded":"2023-07-13T22:23:15.128002+0000","last_fullsized":"2023-07-13T22:23:15.128002+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:40:54.316211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130455+0000","last_change":"2023-07-13T22:23:15.124130+0000","last_active":"2023-07-13T22:23:15.130455+0000","last_peered":"2023-07-13T22:23:15.130455+0000","last_clean":"2023-07-13T22:23:15.130455+0000","last_became_active":"2023-07-13T22:23:15.123355+0000","last_became_peered":"2023-07-13T22:23:15.123355+0000","last_unstale":"2023-07-13T22:23:15.130455+0000","last_undegraded":"2023-07-13T22:23:15.130455+0000","last_fullsized":"2023-07-13T22:23:15.130455+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:36.941256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.139081+0000","last_change":"2023-07-13T22:23:14.105197+0000","last_active":"2023-07-13T22:23:15.139081+0000","last_peered":"2023-07-13T22:23:15.139081+0000","last_clean":"2023-07-13T22:23:15.139081+0000","last_became_active":"2023-07-13T22:23:14.104913+0000","last_became_peered":"2023-07-13T22:23:14.104913+0000","last_unstale":"2023-07-13T22:23:15.139081+0000","last_undegraded":"2023-07-13T22:23:15.139081+0000","last_fullsized":"2023-07-13T22:23:15.139081+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:56:00.254478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120525+0000","last_change":"2023-07-13T22:23:15.117372+0000","last_active":"2023-07-13T22:23:16.120525+0000","last_peered":"2023-07-13T22:23:16.120525+0000","last_clean":"2023-07-13T22:23:16.120525+0000","last_became_active":"2023-07-13T22:23:15.117111+0000","last_became_peered":"2023-07-13T22:23:15.117111+0000","last_unstale":"2023-07-13T22:23:16.120525+0000","last_undegraded":"2023-07-13T22:23:16.120525+0000","last_fullsized":"2023-07-13T22:23:16.120525+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:34:21.416572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121547+0000","last_change":"2023-07-13T22:23:14.121723+0000","last_active":"2023-07-13T22:23:16.121547+0000","last_peered":"2023-07-13T22:23:16.121547+0000","last_clean":"2023-07-13T22:23:16.121547+0000","last_became_active":"2023-07-13T22:23:14.121451+0000","last_became_peered":"2023-07-13T22:23:14.121451+0000","last_unstale":"2023-07-13T22:23:16.121547+0000","last_undegraded":"2023-07-13T22:23:16.121547+0000","last_fullsized":"2023-07-13T22:23:16.121547+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:34:47.155749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121777+0000","last_change":"2023-07-13T22:23:15.131853+0000","last_active":"2023-07-13T22:23:16.121777+0000","last_peered":"2023-07-13T22:23:16.121777+0000","last_clean":"2023-07-13T22:23:16.121777+0000","last_became_active":"2023-07-13T22:23:15.129944+0000","last_became_peered":"2023-07-13T22:23:15.129944+0000","last_unstale":"2023-07-13T22:23:16.121777+0000","last_undegraded":"2023-07-13T22:23:16.121777+0000","last_fullsized":"2023-07-13T22:23:16.121777+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:36:54.293285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.110468+0000","last_change":"2023-07-13T22:23:14.121024+0000","last_active":"2023-07-13T22:23:15.110468+0000","last_peered":"2023-07-13T22:23:15.110468+0000","last_clean":"2023-07-13T22:23:15.110468+0000","last_became_active":"2023-07-13T22:23:14.120540+0000","last_became_peered":"2023-07-13T22:23:14.120540+0000","last_unstale":"2023-07-13T22:23:15.110468+0000","last_undegraded":"2023-07-13T22:23:15.110468+0000","last_fullsized":"2023-07-13T22:23:15.110468+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:02:20.754497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.127284+0000","last_change":"2023-07-13T22:23:15.116723+0000","last_active":"2023-07-13T22:23:15.127284+0000","last_peered":"2023-07-13T22:23:15.127284+0000","last_clean":"2023-07-13T22:23:15.127284+0000","last_became_active":"2023-07-13T22:23:15.116391+0000","last_became_peered":"2023-07-13T22:23:15.116391+0000","last_unstale":"2023-07-13T22:23:15.127284+0000","last_undegraded":"2023-07-13T22:23:15.127284+0000","last_fullsized":"2023-07-13T22:23:15.127284+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:54:17.085802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.114191+0000","last_change":"2023-07-13T22:23:14.120475+0000","last_active":"2023-07-13T22:23:15.114191+0000","last_peered":"2023-07-13T22:23:15.114191+0000","last_clean":"2023-07-13T22:23:15.114191+0000","last_became_active":"2023-07-13T22:23:14.120134+0000","last_became_peered":"2023-07-13T22:23:14.120134+0000","last_unstale":"2023-07-13T22:23:15.114191+0000","last_undegraded":"2023-07-13T22:23:15.114191+0000","last_fullsized":"2023-07-13T22:23:15.114191+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:14:41.654539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.122007+0000","last_change":"2023-07-13T22:23:15.133088+0000","last_active":"2023-07-13T22:23:16.122007+0000","last_peered":"2023-07-13T22:23:16.122007+0000","last_clean":"2023-07-13T22:23:16.122007+0000","last_became_active":"2023-07-13T22:23:15.130348+0000","last_became_peered":"2023-07-13T22:23:15.130348+0000","last_unstale":"2023-07-13T22:23:16.122007+0000","last_undegraded":"2023-07-13T22:23:16.122007+0000","last_fullsized":"2023-07-13T22:23:16.122007+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:18:12.038123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.134820+0000","last_change":"2023-07-13T22:23:14.106447+0000","last_active":"2023-07-13T22:23:15.134820+0000","last_peered":"2023-07-13T22:23:15.134820+0000","last_clean":"2023-07-13T22:23:15.134820+0000","last_became_active":"2023-07-13T22:23:14.106157+0000","last_became_peered":"2023-07-13T22:23:14.106157+0000","last_unstale":"2023-07-13T22:23:15.134820+0000","last_undegraded":"2023-07-13T22:23:15.134820+0000","last_fullsized":"2023-07-13T22:23:15.134820+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:10:22.920377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129651+0000","last_change":"2023-07-13T22:23:15.119012+0000","last_active":"2023-07-13T22:23:15.129651+0000","last_peered":"2023-07-13T22:23:15.129651+0000","last_clean":"2023-07-13T22:23:15.129651+0000","last_became_active":"2023-07-13T22:23:15.118757+0000","last_became_peered":"2023-07-13T22:23:15.118757+0000","last_unstale":"2023-07-13T22:23:15.129651+0000","last_undegraded":"2023-07-13T22:23:15.129651+0000","last_fullsized":"2023-07-13T22:23:15.129651+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:36.941256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121942+0000","last_change":"2023-07-13T22:23:14.121598+0000","last_active":"2023-07-13T22:23:16.121942+0000","last_peered":"2023-07-13T22:23:16.121942+0000","last_clean":"2023-07-13T22:23:16.121942+0000","last_became_active":"2023-07-13T22:23:14.119514+0000","last_became_peered":"2023-07-13T22:23:14.119514+0000","last_unstale":"2023-07-13T22:23:16.121942+0000","last_undegraded":"2023-07-13T22:23:16.121942+0000","last_fullsized":"2023-07-13T22:23:16.121942+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:05.834088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119384+0000","last_change":"2023-07-13T22:23:15.121648+0000","last_active":"2023-07-13T22:23:16.119384+0000","last_peered":"2023-07-13T22:23:16.119384+0000","last_clean":"2023-07-13T22:23:16.119384+0000","last_became_active":"2023-07-13T22:23:15.121343+0000","last_became_peered":"2023-07-13T22:23:15.121343+0000","last_unstale":"2023-07-13T22:23:16.119384+0000","last_undegraded":"2023-07-13T22:23:16.119384+0000","last_fullsized":"2023-07-13T22:23:16.119384+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:00:26.636577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131938+0000","last_change":"2023-07-13T22:23:14.116045+0000","last_active":"2023-07-13T22:23:15.131938+0000","last_peered":"2023-07-13T22:23:15.131938+0000","last_clean":"2023-07-13T22:23:15.131938+0000","last_became_active":"2023-07-13T22:23:14.115723+0000","last_became_peered":"2023-07-13T22:23:14.115723+0000","last_unstale":"2023-07-13T22:23:15.131938+0000","last_undegraded":"2023-07-13T22:23:15.131938+0000","last_fullsized":"2023-07-13T22:23:15.131938+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:44:10.271316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.133175+0000","last_change":"2023-07-13T22:23:15.131522+0000","last_active":"2023-07-13T22:23:15.133175+0000","last_peered":"2023-07-13T22:23:15.133175+0000","last_clean":"2023-07-13T22:23:15.133175+0000","last_became_active":"2023-07-13T22:23:15.130992+0000","last_became_peered":"2023-07-13T22:23:15.130992+0000","last_unstale":"2023-07-13T22:23:15.133175+0000","last_undegraded":"2023-07-13T22:23:15.133175+0000","last_fullsized":"2023-07-13T22:23:15.133175+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:10:23.927784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129765+0000","last_change":"2023-07-13T22:23:14.117408+0000","last_active":"2023-07-13T22:23:15.129765+0000","last_peered":"2023-07-13T22:23:15.129765+0000","last_clean":"2023-07-13T22:23:15.129765+0000","last_became_active":"2023-07-13T22:23:14.117110+0000","last_became_peered":"2023-07-13T22:23:14.117110+0000","last_unstale":"2023-07-13T22:23:15.129765+0000","last_undegraded":"2023-07-13T22:23:15.129765+0000","last_fullsized":"2023-07-13T22:23:15.129765+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:39:11.744835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119293+0000","last_change":"2023-07-13T22:23:15.120214+0000","last_active":"2023-07-13T22:23:16.119293+0000","last_peered":"2023-07-13T22:23:16.119293+0000","last_clean":"2023-07-13T22:23:16.119293+0000","last_became_active":"2023-07-13T22:23:15.119923+0000","last_became_peered":"2023-07-13T22:23:15.119923+0000","last_unstale":"2023-07-13T22:23:16.119293+0000","last_undegraded":"2023-07-13T22:23:16.119293+0000","last_fullsized":"2023-07-13T22:23:16.119293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:03.126633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.109785+0000","last_change":"2023-07-13T22:23:14.119721+0000","last_active":"2023-07-13T22:23:15.109785+0000","last_peered":"2023-07-13T22:23:15.109785+0000","last_clean":"2023-07-13T22:23:15.109785+0000","last_became_active":"2023-07-13T22:23:14.119419+0000","last_became_peered":"2023-07-13T22:23:14.119419+0000","last_unstale":"2023-07-13T22:23:15.109785+0000","last_undegraded":"2023-07-13T22:23:15.109785+0000","last_fullsized":"2023-07-13T22:23:15.109785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:18:11.792486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120685+0000","last_change":"2023-07-13T22:23:15.124984+0000","last_active":"2023-07-13T22:23:16.120685+0000","last_peered":"2023-07-13T22:23:16.120685+0000","last_clean":"2023-07-13T22:23:16.120685+0000","last_became_active":"2023-07-13T22:23:15.124692+0000","last_became_peered":"2023-07-13T22:23:15.124692+0000","last_unstale":"2023-07-13T22:23:16.120685+0000","last_undegraded":"2023-07-13T22:23:16.120685+0000","last_fullsized":"2023-07-13T22:23:16.120685+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:32:32.253408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119360+0000","last_change":"2023-07-13T22:23:14.112933+0000","last_active":"2023-07-13T22:23:16.119360+0000","last_peered":"2023-07-13T22:23:16.119360+0000","last_clean":"2023-07-13T22:23:16.119360+0000","last_became_active":"2023-07-13T22:23:14.112657+0000","last_became_peered":"2023-07-13T22:23:14.112657+0000","last_unstale":"2023-07-13T22:23:16.119360+0000","last_undegraded":"2023-07-13T22:23:16.119360+0000","last_fullsized":"2023-07-13T22:23:16.119360+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:46:16.097850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132517+0000","last_change":"2023-07-13T22:23:15.129908+0000","last_active":"2023-07-13T22:23:15.132517+0000","last_peered":"2023-07-13T22:23:15.132517+0000","last_clean":"2023-07-13T22:23:15.132517+0000","last_became_active":"2023-07-13T22:23:15.129549+0000","last_became_peered":"2023-07-13T22:23:15.129549+0000","last_unstale":"2023-07-13T22:23:15.132517+0000","last_undegraded":"2023-07-13T22:23:15.132517+0000","last_fullsized":"2023-07-13T22:23:15.132517+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:32:18.064197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.103704+0000","last_change":"2023-07-13T22:23:14.105848+0000","last_active":"2023-07-13T22:23:15.103704+0000","last_peered":"2023-07-13T22:23:15.103704+0000","last_clean":"2023-07-13T22:23:15.103704+0000","last_became_active":"2023-07-13T22:23:14.105495+0000","last_became_peered":"2023-07-13T22:23:14.105495+0000","last_unstale":"2023-07-13T22:23:15.103704+0000","last_undegraded":"2023-07-13T22:23:15.103704+0000","last_fullsized":"2023-07-13T22:23:15.103704+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:46.323544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121965+0000","last_change":"2023-07-13T22:23:15.131910+0000","last_active":"2023-07-13T22:23:16.121965+0000","last_peered":"2023-07-13T22:23:16.121965+0000","last_clean":"2023-07-13T22:23:16.121965+0000","last_became_active":"2023-07-13T22:23:15.131597+0000","last_became_peered":"2023-07-13T22:23:15.131597+0000","last_unstale":"2023-07-13T22:23:16.121965+0000","last_undegraded":"2023-07-13T22:23:16.121965+0000","last_fullsized":"2023-07-13T22:23:16.121965+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:48:33.136472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121106+0000","last_change":"2023-07-13T22:23:14.123031+0000","last_active":"2023-07-13T22:23:16.121106+0000","last_peered":"2023-07-13T22:23:16.121106+0000","last_clean":"2023-07-13T22:23:16.121106+0000","last_became_active":"2023-07-13T22:23:14.122712+0000","last_became_peered":"2023-07-13T22:23:14.122712+0000","last_unstale":"2023-07-13T22:23:16.121106+0000","last_undegraded":"2023-07-13T22:23:16.121106+0000","last_fullsized":"2023-07-13T22:23:16.121106+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:00:36.722157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119337+0000","last_change":"2023-07-13T22:23:15.120375+0000","last_active":"2023-07-13T22:23:16.119337+0000","last_peered":"2023-07-13T22:23:16.119337+0000","last_clean":"2023-07-13T22:23:16.119337+0000","last_became_active":"2023-07-13T22:23:15.120048+0000","last_became_peered":"2023-07-13T22:23:15.120048+0000","last_unstale":"2023-07-13T22:23:16.119337+0000","last_undegraded":"2023-07-13T22:23:16.119337+0000","last_fullsized":"2023-07-13T22:23:16.119337+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:26:58.375517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.104050+0000","last_change":"2023-07-13T22:23:14.109287+0000","last_active":"2023-07-13T22:23:15.104050+0000","last_peered":"2023-07-13T22:23:15.104050+0000","last_clean":"2023-07-13T22:23:15.104050+0000","last_became_active":"2023-07-13T22:23:14.108282+0000","last_became_peered":"2023-07-13T22:23:14.108282+0000","last_unstale":"2023-07-13T22:23:15.104050+0000","last_undegraded":"2023-07-13T22:23:15.104050+0000","last_fullsized":"2023-07-13T22:23:15.104050+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:18:27.457575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121636+0000","last_change":"2023-07-13T22:23:15.133329+0000","last_active":"2023-07-13T22:23:16.121636+0000","last_peered":"2023-07-13T22:23:16.121636+0000","last_clean":"2023-07-13T22:23:16.121636+0000","last_became_active":"2023-07-13T22:23:15.130125+0000","last_became_peered":"2023-07-13T22:23:15.130125+0000","last_unstale":"2023-07-13T22:23:16.121636+0000","last_undegraded":"2023-07-13T22:23:16.121636+0000","last_fullsized":"2023-07-13T22:23:16.121636+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:03:59.750400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.104452+0000","last_change":"2023-07-13T22:23:14.109029+0000","last_active":"2023-07-13T22:23:15.104452+0000","last_peered":"2023-07-13T22:23:15.104452+0000","last_clean":"2023-07-13T22:23:15.104452+0000","last_became_active":"2023-07-13T22:23:14.108769+0000","last_became_peered":"2023-07-13T22:23:14.108769+0000","last_unstale":"2023-07-13T22:23:15.104452+0000","last_undegraded":"2023-07-13T22:23:15.104452+0000","last_fullsized":"2023-07-13T22:23:15.104452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:18:53.356437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.105325+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:23:15.105325+0000","last_peered":"2023-07-13T22:23:15.105325+0000","last_clean":"2023-07-13T22:23:15.105325+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:23:15.105325+0000","last_undegraded":"2023-07-13T22:23:15.105325+0000","last_fullsized":"2023-07-13T22:23:15.105325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:11:06.229784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120374+0000","last_change":"2023-07-13T22:23:15.121431+0000","last_active":"2023-07-13T22:23:16.120374+0000","last_peered":"2023-07-13T22:23:16.120374+0000","last_clean":"2023-07-13T22:23:16.120374+0000","last_became_active":"2023-07-13T22:23:15.121162+0000","last_became_peered":"2023-07-13T22:23:15.121162+0000","last_unstale":"2023-07-13T22:23:16.120374+0000","last_undegraded":"2023-07-13T22:23:16.120374+0000","last_fullsized":"2023-07-13T22:23:16.120374+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:58:06.919357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.103872+0000","last_change":"2023-07-13T22:23:14.114841+0000","last_active":"2023-07-13T22:23:15.103872+0000","last_peered":"2023-07-13T22:23:15.103872+0000","last_clean":"2023-07-13T22:23:15.103872+0000","last_became_active":"2023-07-13T22:23:14.114343+0000","last_became_peered":"2023-07-13T22:23:14.114343+0000","last_unstale":"2023-07-13T22:23:15.103872+0000","last_undegraded":"2023-07-13T22:23:15.103872+0000","last_fullsized":"2023-07-13T22:23:15.103872+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:14:20.398352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.104188+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:23:15.104188+0000","last_peered":"2023-07-13T22:23:15.104188+0000","last_clean":"2023-07-13T22:23:15.104188+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:23:15.104188+0000","last_undegraded":"2023-07-13T22:23:15.104188+0000","last_fullsized":"2023-07-13T22:23:15.104188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:25:34.228422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119436+0000","last_change":"2023-07-13T22:23:15.108681+0000","last_active":"2023-07-13T22:23:16.119436+0000","last_peered":"2023-07-13T22:23:16.119436+0000","last_clean":"2023-07-13T22:23:16.119436+0000","last_became_active":"2023-07-13T22:23:15.108390+0000","last_became_peered":"2023-07-13T22:23:15.108390+0000","last_unstale":"2023-07-13T22:23:16.119436+0000","last_undegraded":"2023-07-13T22:23:16.119436+0000","last_fullsized":"2023-07-13T22:23:16.119436+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:53:34.657146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.112377+0000","last_change":"2023-07-13T22:23:14.113623+0000","last_active":"2023-07-13T22:23:15.112377+0000","last_peered":"2023-07-13T22:23:15.112377+0000","last_clean":"2023-07-13T22:23:15.112377+0000","last_became_active":"2023-07-13T22:23:14.113259+0000","last_became_peered":"2023-07-13T22:23:14.113259+0000","last_unstale":"2023-07-13T22:23:15.112377+0000","last_undegraded":"2023-07-13T22:23:15.112377+0000","last_fullsized":"2023-07-13T22:23:15.112377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:26:54.959880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119375+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:23:16.119375+0000","last_peered":"2023-07-13T22:23:16.119375+0000","last_clean":"2023-07-13T22:23:16.119375+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:23:16.119375+0000","last_undegraded":"2023-07-13T22:23:16.119375+0000","last_fullsized":"2023-07-13T22:23:16.119375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:55:46.456312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":214,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.108795+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:23:15.108795+0000","last_peered":"2023-07-13T22:23:15.108795+0000","last_clean":"2023-07-13T22:23:15.108795+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:23:15.108795+0000","last_undegraded":"2023-07-13T22:23:15.108795+0000","last_fullsized":"2023-07-13T22:23:15.108795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:53:34.882351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132497+0000","last_change":"2023-07-13T22:23:15.129170+0000","last_active":"2023-07-13T22:23:15.132497+0000","last_peered":"2023-07-13T22:23:15.132497+0000","last_clean":"2023-07-13T22:23:15.132497+0000","last_became_active":"2023-07-13T22:23:15.128313+0000","last_became_peered":"2023-07-13T22:23:15.128313+0000","last_unstale":"2023-07-13T22:23:15.132497+0000","last_undegraded":"2023-07-13T22:23:15.132497+0000","last_fullsized":"2023-07-13T22:23:15.132497+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:13:33.612038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121473+0000","last_change":"2023-07-13T22:23:14.119635+0000","last_active":"2023-07-13T22:23:16.121473+0000","last_peered":"2023-07-13T22:23:16.121473+0000","last_clean":"2023-07-13T22:23:16.121473+0000","last_became_active":"2023-07-13T22:23:14.119226+0000","last_became_peered":"2023-07-13T22:23:14.119226+0000","last_unstale":"2023-07-13T22:23:16.121473+0000","last_undegraded":"2023-07-13T22:23:16.121473+0000","last_fullsized":"2023-07-13T22:23:16.121473+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:51:40.327041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.110026+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:23:15.110026+0000","last_peered":"2023-07-13T22:23:15.110026+0000","last_clean":"2023-07-13T22:23:15.110026+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:23:15.110026+0000","last_undegraded":"2023-07-13T22:23:15.110026+0000","last_fullsized":"2023-07-13T22:23:15.110026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:11:24.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129180+0000","last_change":"2023-07-13T22:23:15.122571+0000","last_active":"2023-07-13T22:23:15.129180+0000","last_peered":"2023-07-13T22:23:15.129180+0000","last_clean":"2023-07-13T22:23:15.129180+0000","last_became_active":"2023-07-13T22:23:15.122246+0000","last_became_peered":"2023-07-13T22:23:15.122246+0000","last_unstale":"2023-07-13T22:23:15.129180+0000","last_undegraded":"2023-07-13T22:23:15.129180+0000","last_fullsized":"2023-07-13T22:23:15.129180+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:01:08.201816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121807+0000","last_change":"2023-07-13T22:23:14.119648+0000","last_active":"2023-07-13T22:23:16.121807+0000","last_peered":"2023-07-13T22:23:16.121807+0000","last_clean":"2023-07-13T22:23:16.121807+0000","last_became_active":"2023-07-13T22:23:14.119329+0000","last_became_peered":"2023-07-13T22:23:14.119329+0000","last_unstale":"2023-07-13T22:23:16.121807+0000","last_undegraded":"2023-07-13T22:23:16.121807+0000","last_fullsized":"2023-07-13T22:23:16.121807+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:33:32.807863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.108661+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:23:15.108661+0000","last_peered":"2023-07-13T22:23:15.108661+0000","last_clean":"2023-07-13T22:23:15.108661+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:23:15.108661+0000","last_undegraded":"2023-07-13T22:23:15.108661+0000","last_fullsized":"2023-07-13T22:23:15.108661+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:51:49.703548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120847+0000","last_change":"2023-07-13T22:23:15.118800+0000","last_active":"2023-07-13T22:23:16.120847+0000","last_peered":"2023-07-13T22:23:16.120847+0000","last_clean":"2023-07-13T22:23:16.120847+0000","last_became_active":"2023-07-13T22:23:15.118536+0000","last_became_peered":"2023-07-13T22:23:15.118536+0000","last_unstale":"2023-07-13T22:23:16.120847+0000","last_undegraded":"2023-07-13T22:23:16.120847+0000","last_fullsized":"2023-07-13T22:23:16.120847+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:45:25.191667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.134013+0000","last_change":"2023-07-13T22:23:14.120903+0000","last_active":"2023-07-13T22:23:15.134013+0000","last_peered":"2023-07-13T22:23:15.134013+0000","last_clean":"2023-07-13T22:23:15.134013+0000","last_became_active":"2023-07-13T22:23:14.120528+0000","last_became_peered":"2023-07-13T22:23:14.120528+0000","last_unstale":"2023-07-13T22:23:15.134013+0000","last_undegraded":"2023-07-13T22:23:15.134013+0000","last_fullsized":"2023-07-13T22:23:15.134013+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:38:25.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120694+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:23:16.120694+0000","last_peered":"2023-07-13T22:23:16.120694+0000","last_clean":"2023-07-13T22:23:16.120694+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:23:16.120694+0000","last_undegraded":"2023-07-13T22:23:16.120694+0000","last_fullsized":"2023-07-13T22:23:16.120694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:40.587332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.126259+0000","last_change":"2023-07-13T22:23:15.114584+0000","last_active":"2023-07-13T22:23:15.126259+0000","last_peered":"2023-07-13T22:23:15.126259+0000","last_clean":"2023-07-13T22:23:15.126259+0000","last_became_active":"2023-07-13T22:23:15.114191+0000","last_became_peered":"2023-07-13T22:23:15.114191+0000","last_unstale":"2023-07-13T22:23:15.126259+0000","last_undegraded":"2023-07-13T22:23:15.126259+0000","last_fullsized":"2023-07-13T22:23:15.126259+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:52:10.055506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120796+0000","last_change":"2023-07-13T22:23:14.116691+0000","last_active":"2023-07-13T22:23:16.120796+0000","last_peered":"2023-07-13T22:23:16.120796+0000","last_clean":"2023-07-13T22:23:16.120796+0000","last_became_active":"2023-07-13T22:23:14.116369+0000","last_became_peered":"2023-07-13T22:23:14.116369+0000","last_unstale":"2023-07-13T22:23:16.120796+0000","last_undegraded":"2023-07-13T22:23:16.120796+0000","last_fullsized":"2023-07-13T22:23:16.120796+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:36.648050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.107536+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:23:15.107536+0000","last_peered":"2023-07-13T22:23:15.107536+0000","last_clean":"2023-07-13T22:23:15.107536+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:23:15.107536+0000","last_undegraded":"2023-07-13T22:23:15.107536+0000","last_fullsized":"2023-07-13T22:23:15.107536+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:46:39.914110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121788+0000","last_change":"2023-07-13T22:23:15.130228+0000","last_active":"2023-07-13T22:23:16.121788+0000","last_peered":"2023-07-13T22:23:16.121788+0000","last_clean":"2023-07-13T22:23:16.121788+0000","last_became_active":"2023-07-13T22:23:15.129879+0000","last_became_peered":"2023-07-13T22:23:15.129879+0000","last_unstale":"2023-07-13T22:23:16.121788+0000","last_undegraded":"2023-07-13T22:23:16.121788+0000","last_fullsized":"2023-07-13T22:23:16.121788+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:26:21.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.104480+0000","last_change":"2023-07-13T22:23:14.112992+0000","last_active":"2023-07-13T22:23:15.104480+0000","last_peered":"2023-07-13T22:23:15.104480+0000","last_clean":"2023-07-13T22:23:15.104480+0000","last_became_active":"2023-07-13T22:23:14.112663+0000","last_became_peered":"2023-07-13T22:23:14.112663+0000","last_unstale":"2023-07-13T22:23:15.104480+0000","last_undegraded":"2023-07-13T22:23:15.104480+0000","last_fullsized":"2023-07-13T22:23:15.104480+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:05.358531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.108712+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:23:15.108712+0000","last_peered":"2023-07-13T22:23:15.108712+0000","last_clean":"2023-07-13T22:23:15.108712+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:23:15.108712+0000","last_undegraded":"2023-07-13T22:23:15.108712+0000","last_fullsized":"2023-07-13T22:23:15.108712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:28:59.688558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1167360,"data_stored":834620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.02,"5min":1.115,"15min":1.115},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.69599999999999995,"15min":0.71799999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77800000000000002,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88200000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.751}]},{"osd":2,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.80300000000000005,"15min":0.82399999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92300000000000004,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.79000000000000004,"15min":0.82199999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67500000000000004,"15min":0.68500000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90200000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76100000000000001,"15min":0.76500000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98799999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.79400000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90900000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.748,"15min":0.754},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84799999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66600000000000004,"15min":0.67300000000000004},"min":{"1min":0.40899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91300000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79700000000000004},"min":{"1min":0.56100000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93500000000000005,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.89100000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.47199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93600000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94799999999999995,"5min":1.042,"15min":1.042},"last":0.71799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":573440,"data_stored":244780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78200000000000003,"5min":0.875,"15min":0.875},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.68200000000000005,"15min":0.69499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88200000000000001,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.65900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95099999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81499999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89100000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91900000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.75900000000000001,"15min":0.76700000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90500000000000003,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92600000000000005,"5min":1.111,"15min":1.111},"last":0.83599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1167360,"data_stored":834656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":1.952,"15min":1.952},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.51800000000000002,"15min":0.53000000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63600000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60499999999999998,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.60999999999999999,"15min":0.627},"min":{"1min":0.377,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.87,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91500000000000004,"5min":2.008,"15min":2.008},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64300000000000002,"5min":0.87,"15min":0.87},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.81499999999999995,"15min":0.83499999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96799999999999997,"5min":2.028,"15min":2.028},"last":0.752},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.75600000000000001,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0189999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.85699999999999998,"15min":0.88200000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97899999999999998,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.79700000000000004,"15min":0.81499999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.95299999999999996,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.88400000000000001,"15min":0.90900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0389999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98099999999999998,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.82299999999999995,"15min":0.84099999999999997},"min":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.63300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":569344,"data_stored":243198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.60599999999999998,"15min":0.53500000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":11.791,"5min":11.791,"15min":11.791},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.55500000000000005,"15min":0.53600000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.657,"5min":4.657,"15min":4.657},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":4.5389999999999997,"5min":4.5389999999999997,"15min":4.5389999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.1930000000000001,"15min":1.1859999999999999},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":13.398999999999999,"5min":13.398999999999999,"15min":13.398999999999999},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0329999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.4470000000000001,"15min":1.456},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":13.667999999999999,"5min":13.667999999999999,"15min":13.667999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.81799999999999995,"15min":0.83799999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0980000000000001,"5min":1.776,"15min":1.776},"last":0.81499999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4610000000000001,"15min":1.466},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":13.349,"5min":13.349,"15min":13.349},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.91300000000000003,"15min":0.90400000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":4.7649999999999997,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.84199999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.86499999999999999,"15min":0.85699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.5039999999999996,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.3819999999999999,"15min":1.4379999999999999},"min":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":6.1870000000000003,"5min":11.679,"15min":11.679},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4630000000000001,"15min":1.4710000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":13.298,"5min":13.298,"15min":13.298},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.3500000000000001,"5min":0.98499999999999999,"15min":0.92400000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":11.456,"5min":11.456,"15min":11.456},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":573440,"data_stored":244706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.627,"5min":2.282,"15min":2.282},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46899999999999997,"15min":0.47199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59499999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.39200000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.61399999999999999,"15min":0.63300000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78300000000000003,"5min":2.319,"15min":2.319},"last":0.432}]},{"osd":3,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74099999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.95999999999999996,"5min":2.351,"15min":2.351},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.86299999999999999,"15min":0.88100000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3600000000000001,"5min":2.238,"15min":2.238},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94099999999999995,"5min":1.518,"15min":1.518},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.59299999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.86799999999999999,"15min":0.89200000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.92100000000000004,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94999999999999996,"5min":1.214,"15min":1.214},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88300000000000001,"5min":1.29,"15min":1.29},"last":0.625},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.84499999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.99099999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.69499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35712,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336128,"statfs":{"total":96636764160,"available":96600195072,"internally_reserved":0,"allocated":581632,"data_stored":245714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42199999999999999,"15min":0.42699999999999999},"min":{"1min":0.28999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57199999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.439,"15min":0.437},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63700000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.371},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67000000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68899999999999995,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.65100000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61699999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72999999999999998,"15min":0.73999999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92400000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.879},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73599999999999999,"15min":0.74099999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.022,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.88300000000000001,"5min":1.411,"15min":1.411},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92600000000000005,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.53100000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.075,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.82999999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80500000000000005,"15min":0.81499999999999995},"min":{"1min":0.57399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.97899999999999998,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92400000000000004,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.96799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":569344,"data_stored":244710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.59599999999999997,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68100000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.626}]},{"osd":1,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.038,"5min":1.298,"15min":1.298},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.82599999999999996,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92700000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.71099999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0580000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":1.0349999999999999,"15min":1.109},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":10.657999999999999,"15min":10.657999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":1,"15min":1.0529999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92500000000000004,"5min":8.75,"15min":8.75},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73599999999999999,"15min":0.74399999999999999},"min":{"1min":0.442,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97299999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69399999999999995,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1,"5min":1.004,"15min":1.004},"last":0.752},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.78900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":565248,"data_stored":244128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.52800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80400000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93200000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.627},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.54000000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88900000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.58799999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88600000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.68700000000000006,"15min":0.70099999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81799999999999995,"5min":0.995,"15min":0.995},"last":0.495}]},{"osd":6,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93799999999999994,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44900000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.82399999999999995,"5min":1.02,"15min":1.02},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.52400000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.86199999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.72099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:23:19.553 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:23:19.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.805+0000 7f9d81dff700 1 Processor -- start 2023-07-13T22:23:19.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.805+0000 7f9d81dff700 1 -- start start 2023-07-13T22:23:19.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.806+0000 7f9d81dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0b8570 0x7f9d7c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d81dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7c0d09d0 con 0x7f9d7c0d15c0 2023-07-13T22:23:19.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d81dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7c0cda10 con 0x7f9d7c0d35c0 2023-07-13T22:23:19.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d81dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0d3b30 con 0x7f9d7c0b8570 2023-07-13T22:23:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d7bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9d7c0d15c0 0x7f9d7c15c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47266/0 (socket says 172.21.15.60:47266) 2023-07-13T22:23:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d7bfff700 1 -- 172.21.15.60:0/2791637878 learned_addr learned my addr 172.21.15.60:0/2791637878 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:19.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d7b7fe700 1 --2- 172.21.15.60:0/2791637878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0b8570 0x7f9d7c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d7a7fc700 1 -- 172.21.15.60:0/2791637878 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067906138 0 0) 0x7f9d7c0cda10 con 0x7f9d7c0d35c0 2023-07-13T22:23:19.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d7b7fe700 1 -- 172.21.15.60:0/2791637878 >> v1:172.21.15.105:6789/0 conn(0x7f9d7c0d35c0 legacy=0x7f9d7c0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d7b7fe700 1 -- 172.21.15.60:0/2791637878 >> v1:172.21.15.60:6789/0 conn(0x7f9d7c0d15c0 legacy=0x7f9d7c15c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.807+0000 7f9d7b7fe700 1 -- 172.21.15.60:0/2791637878 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c0cd450 con 0x7f9d7c0b8570 2023-07-13T22:23:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.808+0000 7f9d7b7fe700 1 --2- 172.21.15.60:0/2791637878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0b8570 0x7f9d7c0ceeb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009850 tx=0x7f9d6c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb300f1d93a43a68 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.808+0000 7f9d7a7fc700 1 -- 172.21.15.60:0/2791637878 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c005e30 con 0x7f9d7c0b8570 2023-07-13T22:23:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.808+0000 7f9d7a7fc700 1 -- 172.21.15.60:0/2791637878 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c004030 con 0x7f9d7c0b8570 2023-07-13T22:23:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.808+0000 7f9d7a7fc700 1 -- 172.21.15.60:0/2791637878 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0043c0 con 0x7f9d7c0b8570 2023-07-13T22:23:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.809+0000 7f9d81dff700 1 -- 172.21.15.60:0/2791637878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0b8570 msgr2=0x7f9d7c0ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.809+0000 7f9d81dff700 1 --2- 172.21.15.60:0/2791637878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0b8570 0x7f9d7c0ceeb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009850 tx=0x7f9d6c009b60 comp rx=0 tx=0).stop 2023-07-13T22:23:19.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.809+0000 7f9d81dff700 1 -- 172.21.15.60:0/2791637878 shutdown_connections 2023-07-13T22:23:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.809+0000 7f9d81dff700 1 --2- 172.21.15.60:0/2791637878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0b8570 0x7f9d7c0ceeb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.809+0000 7f9d81dff700 1 -- 172.21.15.60:0/2791637878 >> 172.21.15.60:0/2791637878 conn(0x7f9d7c071600 msgr2=0x7f9d7c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.809+0000 7f9d81dff700 1 -- 172.21.15.60:0/2791637878 shutdown_connections 2023-07-13T22:23:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.809+0000 7f9d81dff700 1 -- 172.21.15.60:0/2791637878 wait complete. 2023-07-13T22:23:19.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.810+0000 7f9d81dff700 1 Processor -- start 2023-07-13T22:23:19.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.811+0000 7f9d81dff700 1 -- start start 2023-07-13T22:23:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.811+0000 7f9d81dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c0b8570 0x7f9d7c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.811+0000 7f9d81dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d7c0d15c0 0x7f9d7c0b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d81dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0d35c0 0x7f9d7c0b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d81dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0c6ae0 con 0x7f9d7c0b8570 2023-07-13T22:23:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d81dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0c6dc0 con 0x7f9d7c0d15c0 2023-07-13T22:23:19.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d81dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0c70a0 con 0x7f9d7c0d35c0 2023-07-13T22:23:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c0b8570 0x7f9d7c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c0b8570 0x7f9d7c1542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34746/0 (socket says 172.21.15.60:34746) 2023-07-13T22:23:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7b7fe700 1 -- 172.21.15.60:0/3751306515 learned_addr learned my addr 172.21.15.60:0/3751306515 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7affd700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d7c0d15c0 0x7f9d7c0b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7bfff700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0d35c0 0x7f9d7c0b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7b7fe700 1 -- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0d35c0 msgr2=0x7f9d7c0b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7b7fe700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0d35c0 0x7f9d7c0b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7b7fe700 1 -- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d7c0d15c0 msgr2=0x7f9d7c0b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7b7fe700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d7c0d15c0 0x7f9d7c0b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7b7fe700 1 -- 172.21.15.60:0/3751306515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c009430 con 0x7f9d7c0b8570 2023-07-13T22:23:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7affd700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d7c0d15c0 0x7f9d7c0b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.812+0000 7f9d7bfff700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0d35c0 0x7f9d7c0b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:19.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.813+0000 7f9d7b7fe700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c0b8570 0x7f9d7c1542c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009ba0 tx=0x7f9d6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:19.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.813+0000 7f9d63fff700 1 -- 172.21.15.60:0/3751306515 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0045e0 con 0x7f9d7c0b8570 2023-07-13T22:23:19.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.813+0000 7f9d63fff700 1 -- 172.21.15.60:0/3751306515 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c003e80 con 0x7f9d7c0b8570 2023-07-13T22:23:19.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.813+0000 7f9d63fff700 1 -- 172.21.15.60:0/3751306515 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c01c640 con 0x7f9d7c0b8570 2023-07-13T22:23:19.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.813+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c0c7380 con 0x7f9d7c0b8570 2023-07-13T22:23:19.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.813+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c0c6790 con 0x7f9d7c0b8570 2023-07-13T22:23:19.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.814+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c0c7b80 con 0x7f9d7c0b8570 2023-07-13T22:23:19.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.815+0000 7f9d63fff700 1 -- 172.21.15.60:0/3751306515 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d6c01c7a0 con 0x7f9d7c0b8570 2023-07-13T22:23:19.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.816+0000 7f9d63fff700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9d6806ace0 0x7f9d6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:19.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.816+0000 7f9d63fff700 1 -- 172.21.15.60:0/3751306515 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d6c08a3e0 con 0x7f9d7c0b8570 2023-07-13T22:23:19.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.816+0000 7f9d7affd700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9d6806ace0 0x7f9d6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:19.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.817+0000 7f9d7affd700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9d6806ace0 0x7f9d6806d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c15d650 tx=0x7f9d70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:19.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:19.820+0000 7f9d63fff700 1 -- 172.21.15.60:0/3751306515 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9d6c055480 con 0x7f9d7c0b8570 2023-07-13T22:23:20.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.035+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d7c0c6f30 con 0x7f9d6806ace0 2023-07-13T22:23:20.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.045+0000 7f9d63fff700 1 -- 172.21.15.60:0/3751306515 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311684 (secure 0 0 0) 0x7f9d7c0c6f30 con 0x7f9d6806ace0 2023-07-13T22:23:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9d6806ace0 msgr2=0x7f9d6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9d6806ace0 0x7f9d6806d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c15d650 tx=0x7f9d70009040 comp rx=0 tx=0).stop 2023-07-13T22:23:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c0b8570 msgr2=0x7f9d7c1542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c0b8570 0x7f9d7c1542c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009ba0 tx=0x7f9d6c004750 comp rx=0 tx=0).stop 2023-07-13T22:23:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 shutdown_connections 2023-07-13T22:23:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9d6806ace0 0x7f9d6806d190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c0b8570 0x7f9d7c1542c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9d7c0d35c0 0x7f9d7c0b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 --2- 172.21.15.60:0/3751306515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d7c0d15c0 0x7f9d7c0b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 >> 172.21.15.60:0/3751306515 conn(0x7f9d7c071600 msgr2=0x7f9d7c15b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 shutdown_connections 2023-07-13T22:23:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.049+0000 7f9d81dff700 1 -- 172.21.15.60:0/3751306515 wait complete. 2023-07-13T22:23:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:23:20.051 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-07-13T22:23:19.153075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249920,"kb_used_data":5632,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754724800,"statfs":{"total":773094113280,"available":772838195200,"internally_reserved":0,"allocated":5767168,"data_stored":3136512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001252"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119181+0000","last_change":"2023-07-13T22:23:15.117911+0000","last_active":"2023-07-13T22:23:16.119181+0000","last_peered":"2023-07-13T22:23:16.119181+0000","last_clean":"2023-07-13T22:23:16.119181+0000","last_became_active":"2023-07-13T22:23:15.117580+0000","last_became_peered":"2023-07-13T22:23:15.117580+0000","last_unstale":"2023-07-13T22:23:16.119181+0000","last_undegraded":"2023-07-13T22:23:16.119181+0000","last_fullsized":"2023-07-13T22:23:16.119181+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:25.640506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.126180+0000","last_change":"2023-07-13T22:23:15.122044+0000","last_active":"2023-07-13T22:23:15.126180+0000","last_peered":"2023-07-13T22:23:15.126180+0000","last_clean":"2023-07-13T22:23:15.126180+0000","last_became_active":"2023-07-13T22:23:15.121730+0000","last_became_peered":"2023-07-13T22:23:15.121730+0000","last_unstale":"2023-07-13T22:23:15.126180+0000","last_undegraded":"2023-07-13T22:23:15.126180+0000","last_fullsized":"2023-07-13T22:23:15.126180+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:52:03.136297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121888+0000","last_change":"2023-07-13T22:23:15.133128+0000","last_active":"2023-07-13T22:23:16.121888+0000","last_peered":"2023-07-13T22:23:16.121888+0000","last_clean":"2023-07-13T22:23:16.121888+0000","last_became_active":"2023-07-13T22:23:15.131529+0000","last_became_peered":"2023-07-13T22:23:15.131529+0000","last_unstale":"2023-07-13T22:23:16.121888+0000","last_undegraded":"2023-07-13T22:23:16.121888+0000","last_fullsized":"2023-07-13T22:23:16.121888+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:55:22.011271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.122942+0000","last_change":"2023-07-13T22:23:15.119521+0000","last_active":"2023-07-13T22:23:15.122942+0000","last_peered":"2023-07-13T22:23:15.122942+0000","last_clean":"2023-07-13T22:23:15.122942+0000","last_became_active":"2023-07-13T22:23:15.119226+0000","last_became_peered":"2023-07-13T22:23:15.119226+0000","last_unstale":"2023-07-13T22:23:15.122942+0000","last_undegraded":"2023-07-13T22:23:15.122942+0000","last_fullsized":"2023-07-13T22:23:15.122942+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:18:44.220318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130436+0000","last_change":"2023-07-13T22:23:15.123929+0000","last_active":"2023-07-13T22:23:15.130436+0000","last_peered":"2023-07-13T22:23:15.130436+0000","last_clean":"2023-07-13T22:23:15.130436+0000","last_became_active":"2023-07-13T22:23:15.122938+0000","last_became_peered":"2023-07-13T22:23:15.122938+0000","last_unstale":"2023-07-13T22:23:15.130436+0000","last_undegraded":"2023-07-13T22:23:15.130436+0000","last_fullsized":"2023-07-13T22:23:15.130436+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:21:07.140335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119197+0000","last_change":"2023-07-13T22:23:15.122245+0000","last_active":"2023-07-13T22:23:16.119197+0000","last_peered":"2023-07-13T22:23:16.119197+0000","last_clean":"2023-07-13T22:23:16.119197+0000","last_became_active":"2023-07-13T22:23:15.121966+0000","last_became_peered":"2023-07-13T22:23:15.121966+0000","last_unstale":"2023-07-13T22:23:16.119197+0000","last_undegraded":"2023-07-13T22:23:16.119197+0000","last_fullsized":"2023-07-13T22:23:16.119197+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:27:19.284955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121035+0000","last_change":"2023-07-13T22:23:15.120446+0000","last_active":"2023-07-13T22:23:16.121035+0000","last_peered":"2023-07-13T22:23:16.121035+0000","last_clean":"2023-07-13T22:23:16.121035+0000","last_became_active":"2023-07-13T22:23:15.119801+0000","last_became_peered":"2023-07-13T22:23:15.119801+0000","last_unstale":"2023-07-13T22:23:16.121035+0000","last_undegraded":"2023-07-13T22:23:16.121035+0000","last_fullsized":"2023-07-13T22:23:16.121035+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:46:43.573246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129672+0000","last_change":"2023-07-13T22:23:15.122196+0000","last_active":"2023-07-13T22:23:15.129672+0000","last_peered":"2023-07-13T22:23:15.129672+0000","last_clean":"2023-07-13T22:23:15.129672+0000","last_became_active":"2023-07-13T22:23:15.121909+0000","last_became_peered":"2023-07-13T22:23:15.121909+0000","last_unstale":"2023-07-13T22:23:15.129672+0000","last_undegraded":"2023-07-13T22:23:15.129672+0000","last_fullsized":"2023-07-13T22:23:15.129672+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:06.365938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.136773+0000","last_change":"2023-07-13T22:23:15.134981+0000","last_active":"2023-07-13T22:23:15.136773+0000","last_peered":"2023-07-13T22:23:15.136773+0000","last_clean":"2023-07-13T22:23:15.136773+0000","last_became_active":"2023-07-13T22:23:15.134456+0000","last_became_peered":"2023-07-13T22:23:15.134456+0000","last_unstale":"2023-07-13T22:23:15.136773+0000","last_undegraded":"2023-07-13T22:23:15.136773+0000","last_fullsized":"2023-07-13T22:23:15.136773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:19:36.133927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119545+0000","last_change":"2023-07-13T22:23:15.128219+0000","last_active":"2023-07-13T22:23:16.119545+0000","last_peered":"2023-07-13T22:23:16.119545+0000","last_clean":"2023-07-13T22:23:16.119545+0000","last_became_active":"2023-07-13T22:23:15.127923+0000","last_became_peered":"2023-07-13T22:23:15.127923+0000","last_unstale":"2023-07-13T22:23:16.119545+0000","last_undegraded":"2023-07-13T22:23:16.119545+0000","last_fullsized":"2023-07-13T22:23:16.119545+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:12:10.313272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132227+0000","last_change":"2023-07-13T22:23:15.121681+0000","last_active":"2023-07-13T22:23:15.132227+0000","last_peered":"2023-07-13T22:23:15.132227+0000","last_clean":"2023-07-13T22:23:15.132227+0000","last_became_active":"2023-07-13T22:23:15.121394+0000","last_became_peered":"2023-07-13T22:23:15.121394+0000","last_unstale":"2023-07-13T22:23:15.132227+0000","last_undegraded":"2023-07-13T22:23:15.132227+0000","last_fullsized":"2023-07-13T22:23:15.132227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:58:23.109679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131280+0000","last_change":"2023-07-13T22:23:15.128977+0000","last_active":"2023-07-13T22:23:15.131280+0000","last_peered":"2023-07-13T22:23:15.131280+0000","last_clean":"2023-07-13T22:23:15.131280+0000","last_became_active":"2023-07-13T22:23:15.128696+0000","last_became_peered":"2023-07-13T22:23:15.128696+0000","last_unstale":"2023-07-13T22:23:15.131280+0000","last_undegraded":"2023-07-13T22:23:15.131280+0000","last_fullsized":"2023-07-13T22:23:15.131280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:51:09.589836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119182+0000","last_change":"2023-07-13T22:23:15.128052+0000","last_active":"2023-07-13T22:23:16.119182+0000","last_peered":"2023-07-13T22:23:16.119182+0000","last_clean":"2023-07-13T22:23:16.119182+0000","last_became_active":"2023-07-13T22:23:15.127748+0000","last_became_peered":"2023-07-13T22:23:15.127748+0000","last_unstale":"2023-07-13T22:23:16.119182+0000","last_undegraded":"2023-07-13T22:23:16.119182+0000","last_fullsized":"2023-07-13T22:23:16.119182+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:09:03.155046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121338+0000","last_change":"2023-07-13T22:23:15.129902+0000","last_active":"2023-07-13T22:23:16.121338+0000","last_peered":"2023-07-13T22:23:16.121338+0000","last_clean":"2023-07-13T22:23:16.121338+0000","last_became_active":"2023-07-13T22:23:15.129531+0000","last_became_peered":"2023-07-13T22:23:15.129531+0000","last_unstale":"2023-07-13T22:23:16.121338+0000","last_undegraded":"2023-07-13T22:23:16.121338+0000","last_fullsized":"2023-07-13T22:23:16.121338+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:02.410960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132400+0000","last_change":"2023-07-13T22:23:15.129452+0000","last_active":"2023-07-13T22:23:15.132400+0000","last_peered":"2023-07-13T22:23:15.132400+0000","last_clean":"2023-07-13T22:23:15.132400+0000","last_became_active":"2023-07-13T22:23:15.129183+0000","last_became_peered":"2023-07-13T22:23:15.129183+0000","last_unstale":"2023-07-13T22:23:15.132400+0000","last_undegraded":"2023-07-13T22:23:15.132400+0000","last_fullsized":"2023-07-13T22:23:15.132400+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:02.651843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.118996+0000","last_change":"2023-07-13T22:23:15.124772+0000","last_active":"2023-07-13T22:23:16.118996+0000","last_peered":"2023-07-13T22:23:16.118996+0000","last_clean":"2023-07-13T22:23:16.118996+0000","last_became_active":"2023-07-13T22:23:15.124489+0000","last_became_peered":"2023-07-13T22:23:15.124489+0000","last_unstale":"2023-07-13T22:23:16.118996+0000","last_undegraded":"2023-07-13T22:23:16.118996+0000","last_fullsized":"2023-07-13T22:23:16.118996+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:01:25.892366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.122146+0000","last_change":"2023-07-13T22:23:15.132250+0000","last_active":"2023-07-13T22:23:16.122146+0000","last_peered":"2023-07-13T22:23:16.122146+0000","last_clean":"2023-07-13T22:23:16.122146+0000","last_became_active":"2023-07-13T22:23:15.129701+0000","last_became_peered":"2023-07-13T22:23:15.129701+0000","last_unstale":"2023-07-13T22:23:16.122146+0000","last_undegraded":"2023-07-13T22:23:16.122146+0000","last_fullsized":"2023-07-13T22:23:16.122146+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:23:07.169138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.127055+0000","last_change":"2023-07-13T22:23:15.123375+0000","last_active":"2023-07-13T22:23:15.127055+0000","last_peered":"2023-07-13T22:23:15.127055+0000","last_clean":"2023-07-13T22:23:15.127055+0000","last_became_active":"2023-07-13T22:23:15.123109+0000","last_became_peered":"2023-07-13T22:23:15.123109+0000","last_unstale":"2023-07-13T22:23:15.127055+0000","last_undegraded":"2023-07-13T22:23:15.127055+0000","last_fullsized":"2023-07-13T22:23:15.127055+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:39:10.832174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121968+0000","last_change":"2023-07-13T22:23:15.133131+0000","last_active":"2023-07-13T22:23:16.121968+0000","last_peered":"2023-07-13T22:23:16.121968+0000","last_clean":"2023-07-13T22:23:16.121968+0000","last_became_active":"2023-07-13T22:23:15.130321+0000","last_became_peered":"2023-07-13T22:23:15.130321+0000","last_unstale":"2023-07-13T22:23:16.121968+0000","last_undegraded":"2023-07-13T22:23:16.121968+0000","last_fullsized":"2023-07-13T22:23:16.121968+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:19:22.881968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131857+0000","last_change":"2023-07-13T22:23:15.128934+0000","last_active":"2023-07-13T22:23:15.131857+0000","last_peered":"2023-07-13T22:23:15.131857+0000","last_clean":"2023-07-13T22:23:15.131857+0000","last_became_active":"2023-07-13T22:23:15.128058+0000","last_became_peered":"2023-07-13T22:23:15.128058+0000","last_unstale":"2023-07-13T22:23:15.131857+0000","last_undegraded":"2023-07-13T22:23:15.131857+0000","last_fullsized":"2023-07-13T22:23:15.131857+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:08:47.550844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131897+0000","last_change":"2023-07-13T22:23:15.128972+0000","last_active":"2023-07-13T22:23:15.131897+0000","last_peered":"2023-07-13T22:23:15.131897+0000","last_clean":"2023-07-13T22:23:15.131897+0000","last_became_active":"2023-07-13T22:23:15.128664+0000","last_became_peered":"2023-07-13T22:23:15.128664+0000","last_unstale":"2023-07-13T22:23:15.131897+0000","last_undegraded":"2023-07-13T22:23:15.131897+0000","last_fullsized":"2023-07-13T22:23:15.131897+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:43.829071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.136734+0000","last_change":"2023-07-13T22:23:15.134809+0000","last_active":"2023-07-13T22:23:15.136734+0000","last_peered":"2023-07-13T22:23:15.136734+0000","last_clean":"2023-07-13T22:23:15.136734+0000","last_became_active":"2023-07-13T22:23:15.134532+0000","last_became_peered":"2023-07-13T22:23:15.134532+0000","last_unstale":"2023-07-13T22:23:15.136734+0000","last_undegraded":"2023-07-13T22:23:15.136734+0000","last_fullsized":"2023-07-13T22:23:15.136734+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:16:19.713925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132184+0000","last_change":"2023-07-13T22:23:15.124507+0000","last_active":"2023-07-13T22:23:15.132184+0000","last_peered":"2023-07-13T22:23:15.132184+0000","last_clean":"2023-07-13T22:23:15.132184+0000","last_became_active":"2023-07-13T22:23:15.124211+0000","last_became_peered":"2023-07-13T22:23:15.124211+0000","last_unstale":"2023-07-13T22:23:15.132184+0000","last_undegraded":"2023-07-13T22:23:15.132184+0000","last_fullsized":"2023-07-13T22:23:15.132184+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:01:12.416364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120578+0000","last_change":"2023-07-13T22:23:15.120001+0000","last_active":"2023-07-13T22:23:16.120578+0000","last_peered":"2023-07-13T22:23:16.120578+0000","last_clean":"2023-07-13T22:23:16.120578+0000","last_became_active":"2023-07-13T22:23:15.119609+0000","last_became_peered":"2023-07-13T22:23:15.119609+0000","last_unstale":"2023-07-13T22:23:16.120578+0000","last_undegraded":"2023-07-13T22:23:16.120578+0000","last_fullsized":"2023-07-13T22:23:16.120578+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:51:44.861681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.122079+0000","last_change":"2023-07-13T22:23:15.132434+0000","last_active":"2023-07-13T22:23:16.122079+0000","last_peered":"2023-07-13T22:23:16.122079+0000","last_clean":"2023-07-13T22:23:16.122079+0000","last_became_active":"2023-07-13T22:23:15.130088+0000","last_became_peered":"2023-07-13T22:23:15.130088+0000","last_unstale":"2023-07-13T22:23:16.122079+0000","last_undegraded":"2023-07-13T22:23:16.122079+0000","last_fullsized":"2023-07-13T22:23:16.122079+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:22:22.174758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129752+0000","last_change":"2023-07-13T22:23:15.118112+0000","last_active":"2023-07-13T22:23:15.129752+0000","last_peered":"2023-07-13T22:23:15.129752+0000","last_clean":"2023-07-13T22:23:15.129752+0000","last_became_active":"2023-07-13T22:23:15.117813+0000","last_became_peered":"2023-07-13T22:23:15.117813+0000","last_unstale":"2023-07-13T22:23:15.129752+0000","last_undegraded":"2023-07-13T22:23:15.129752+0000","last_fullsized":"2023-07-13T22:23:15.129752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:18:44.220318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119457+0000","last_change":"2023-07-13T22:23:15.134875+0000","last_active":"2023-07-13T22:23:16.119457+0000","last_peered":"2023-07-13T22:23:16.119457+0000","last_clean":"2023-07-13T22:23:16.119457+0000","last_became_active":"2023-07-13T22:23:15.134483+0000","last_became_peered":"2023-07-13T22:23:15.134483+0000","last_unstale":"2023-07-13T22:23:16.119457+0000","last_undegraded":"2023-07-13T22:23:16.119457+0000","last_fullsized":"2023-07-13T22:23:16.119457+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:50:30.456993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129163+0000","last_change":"2023-07-13T22:23:15.122265+0000","last_active":"2023-07-13T22:23:15.129163+0000","last_peered":"2023-07-13T22:23:15.129163+0000","last_clean":"2023-07-13T22:23:15.129163+0000","last_became_active":"2023-07-13T22:23:15.121963+0000","last_became_peered":"2023-07-13T22:23:15.121963+0000","last_unstale":"2023-07-13T22:23:15.129163+0000","last_undegraded":"2023-07-13T22:23:15.129163+0000","last_fullsized":"2023-07-13T22:23:15.129163+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:42:27.411239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121397+0000","last_change":"2023-07-13T22:23:15.136420+0000","last_active":"2023-07-13T22:23:16.121397+0000","last_peered":"2023-07-13T22:23:16.121397+0000","last_clean":"2023-07-13T22:23:16.121397+0000","last_became_active":"2023-07-13T22:23:15.136117+0000","last_became_peered":"2023-07-13T22:23:15.136117+0000","last_unstale":"2023-07-13T22:23:16.121397+0000","last_undegraded":"2023-07-13T22:23:16.121397+0000","last_fullsized":"2023-07-13T22:23:16.121397+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:47:13.313970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.124799+0000","last_change":"2023-07-13T22:23:15.120511+0000","last_active":"2023-07-13T22:23:15.124799+0000","last_peered":"2023-07-13T22:23:15.124799+0000","last_clean":"2023-07-13T22:23:15.124799+0000","last_became_active":"2023-07-13T22:23:15.120168+0000","last_became_peered":"2023-07-13T22:23:15.120168+0000","last_unstale":"2023-07-13T22:23:15.124799+0000","last_undegraded":"2023-07-13T22:23:15.124799+0000","last_fullsized":"2023-07-13T22:23:15.124799+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:52:10.055506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120981+0000","last_change":"2023-07-13T22:23:15.120661+0000","last_active":"2023-07-13T22:23:16.120981+0000","last_peered":"2023-07-13T22:23:16.120981+0000","last_clean":"2023-07-13T22:23:16.120981+0000","last_became_active":"2023-07-13T22:23:15.120126+0000","last_became_peered":"2023-07-13T22:23:15.120126+0000","last_unstale":"2023-07-13T22:23:16.120981+0000","last_undegraded":"2023-07-13T22:23:16.120981+0000","last_fullsized":"2023-07-13T22:23:16.120981+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:40:54.737424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120280+0000","last_change":"2023-07-13T22:23:15.119079+0000","last_active":"2023-07-13T22:23:16.120280+0000","last_peered":"2023-07-13T22:23:16.120280+0000","last_clean":"2023-07-13T22:23:16.120280+0000","last_became_active":"2023-07-13T22:23:15.118779+0000","last_became_peered":"2023-07-13T22:23:15.118779+0000","last_unstale":"2023-07-13T22:23:16.120280+0000","last_undegraded":"2023-07-13T22:23:16.120280+0000","last_fullsized":"2023-07-13T22:23:16.120280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:48:32.606922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.123577+0000","last_change":"2023-07-13T22:23:15.121041+0000","last_active":"2023-07-13T22:23:15.123577+0000","last_peered":"2023-07-13T22:23:15.123577+0000","last_clean":"2023-07-13T22:23:15.123577+0000","last_became_active":"2023-07-13T22:23:15.120764+0000","last_became_peered":"2023-07-13T22:23:15.120764+0000","last_unstale":"2023-07-13T22:23:15.123577+0000","last_undegraded":"2023-07-13T22:23:15.123577+0000","last_fullsized":"2023-07-13T22:23:15.123577+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:28:01.610676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131216+0000","last_change":"2023-07-13T22:23:14.117111+0000","last_active":"2023-07-13T22:23:15.131216+0000","last_peered":"2023-07-13T22:23:15.131216+0000","last_clean":"2023-07-13T22:23:15.131216+0000","last_became_active":"2023-07-13T22:23:14.116811+0000","last_became_peered":"2023-07-13T22:23:14.116811+0000","last_unstale":"2023-07-13T22:23:15.131216+0000","last_undegraded":"2023-07-13T22:23:15.131216+0000","last_fullsized":"2023-07-13T22:23:15.131216+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:32:27.692851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120594+0000","last_change":"2023-07-13T22:23:15.126077+0000","last_active":"2023-07-13T22:23:16.120594+0000","last_peered":"2023-07-13T22:23:16.120594+0000","last_clean":"2023-07-13T22:23:16.120594+0000","last_became_active":"2023-07-13T22:23:15.125764+0000","last_became_peered":"2023-07-13T22:23:15.125764+0000","last_unstale":"2023-07-13T22:23:16.120594+0000","last_undegraded":"2023-07-13T22:23:16.120594+0000","last_fullsized":"2023-07-13T22:23:16.120594+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:29:06.882102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119651+0000","last_change":"2023-07-13T22:23:14.110977+0000","last_active":"2023-07-13T22:23:16.119651+0000","last_peered":"2023-07-13T22:23:16.119651+0000","last_clean":"2023-07-13T22:23:16.119651+0000","last_became_active":"2023-07-13T22:23:14.110679+0000","last_became_peered":"2023-07-13T22:23:14.110679+0000","last_unstale":"2023-07-13T22:23:16.119651+0000","last_undegraded":"2023-07-13T22:23:16.119651+0000","last_fullsized":"2023-07-13T22:23:16.119651+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:36:30.987495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130389+0000","last_change":"2023-07-13T22:23:15.128817+0000","last_active":"2023-07-13T22:23:15.130389+0000","last_peered":"2023-07-13T22:23:15.130389+0000","last_clean":"2023-07-13T22:23:15.130389+0000","last_became_active":"2023-07-13T22:23:15.128165+0000","last_became_peered":"2023-07-13T22:23:15.128165+0000","last_unstale":"2023-07-13T22:23:15.130389+0000","last_undegraded":"2023-07-13T22:23:15.130389+0000","last_fullsized":"2023-07-13T22:23:15.130389+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:54:13.910242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119560+0000","last_change":"2023-07-13T22:23:14.120162+0000","last_active":"2023-07-13T22:23:16.119560+0000","last_peered":"2023-07-13T22:23:16.119560+0000","last_clean":"2023-07-13T22:23:16.119560+0000","last_became_active":"2023-07-13T22:23:14.119783+0000","last_became_peered":"2023-07-13T22:23:14.119783+0000","last_unstale":"2023-07-13T22:23:16.119560+0000","last_undegraded":"2023-07-13T22:23:16.119560+0000","last_fullsized":"2023-07-13T22:23:16.119560+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:36:06.944629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.126229+0000","last_change":"2023-07-13T22:23:15.113836+0000","last_active":"2023-07-13T22:23:15.126229+0000","last_peered":"2023-07-13T22:23:15.126229+0000","last_clean":"2023-07-13T22:23:15.126229+0000","last_became_active":"2023-07-13T22:23:15.113515+0000","last_became_peered":"2023-07-13T22:23:15.113515+0000","last_unstale":"2023-07-13T22:23:15.126229+0000","last_undegraded":"2023-07-13T22:23:15.126229+0000","last_fullsized":"2023-07-13T22:23:15.126229+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:45:41.534962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120558+0000","last_change":"2023-07-13T22:23:14.114670+0000","last_active":"2023-07-13T22:23:16.120558+0000","last_peered":"2023-07-13T22:23:16.120558+0000","last_clean":"2023-07-13T22:23:16.120558+0000","last_became_active":"2023-07-13T22:23:14.114412+0000","last_became_peered":"2023-07-13T22:23:14.114412+0000","last_unstale":"2023-07-13T22:23:16.120558+0000","last_undegraded":"2023-07-13T22:23:16.120558+0000","last_fullsized":"2023-07-13T22:23:16.120558+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:10:59.775619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.133133+0000","last_change":"2023-07-13T22:23:15.131330+0000","last_active":"2023-07-13T22:23:15.133133+0000","last_peered":"2023-07-13T22:23:15.133133+0000","last_clean":"2023-07-13T22:23:15.133133+0000","last_became_active":"2023-07-13T22:23:15.131040+0000","last_became_peered":"2023-07-13T22:23:15.131040+0000","last_unstale":"2023-07-13T22:23:15.133133+0000","last_undegraded":"2023-07-13T22:23:15.133133+0000","last_fullsized":"2023-07-13T22:23:15.133133+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:47.330951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121398+0000","last_change":"2023-07-13T22:23:14.119598+0000","last_active":"2023-07-13T22:23:16.121398+0000","last_peered":"2023-07-13T22:23:16.121398+0000","last_clean":"2023-07-13T22:23:16.121398+0000","last_became_active":"2023-07-13T22:23:14.119167+0000","last_became_peered":"2023-07-13T22:23:14.119167+0000","last_unstale":"2023-07-13T22:23:16.121398+0000","last_undegraded":"2023-07-13T22:23:16.121398+0000","last_fullsized":"2023-07-13T22:23:16.121398+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:09:40.697987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.127332+0000","last_change":"2023-07-13T22:23:15.124471+0000","last_active":"2023-07-13T22:23:15.127332+0000","last_peered":"2023-07-13T22:23:15.127332+0000","last_clean":"2023-07-13T22:23:15.127332+0000","last_became_active":"2023-07-13T22:23:15.124186+0000","last_became_peered":"2023-07-13T22:23:15.124186+0000","last_unstale":"2023-07-13T22:23:15.127332+0000","last_undegraded":"2023-07-13T22:23:15.127332+0000","last_fullsized":"2023-07-13T22:23:15.127332+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:31:45.719375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130725+0000","last_change":"2023-07-13T22:23:14.120205+0000","last_active":"2023-07-13T22:23:15.130725+0000","last_peered":"2023-07-13T22:23:15.130725+0000","last_clean":"2023-07-13T22:23:15.130725+0000","last_became_active":"2023-07-13T22:23:14.119846+0000","last_became_peered":"2023-07-13T22:23:14.119846+0000","last_unstale":"2023-07-13T22:23:15.130725+0000","last_undegraded":"2023-07-13T22:23:15.130725+0000","last_fullsized":"2023-07-13T22:23:15.130725+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:14:20.398352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.127243+0000","last_change":"2023-07-13T22:23:15.113100+0000","last_active":"2023-07-13T22:23:15.127243+0000","last_peered":"2023-07-13T22:23:15.127243+0000","last_clean":"2023-07-13T22:23:15.127243+0000","last_became_active":"2023-07-13T22:23:15.112763+0000","last_became_peered":"2023-07-13T22:23:15.112763+0000","last_unstale":"2023-07-13T22:23:15.127243+0000","last_undegraded":"2023-07-13T22:23:15.127243+0000","last_fullsized":"2023-07-13T22:23:15.127243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:13:52.313599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.108115+0000","last_change":"2023-07-13T22:23:14.116977+0000","last_active":"2023-07-13T22:23:15.108115+0000","last_peered":"2023-07-13T22:23:15.108115+0000","last_clean":"2023-07-13T22:23:15.108115+0000","last_became_active":"2023-07-13T22:23:14.116679+0000","last_became_peered":"2023-07-13T22:23:14.116679+0000","last_unstale":"2023-07-13T22:23:15.108115+0000","last_undegraded":"2023-07-13T22:23:15.108115+0000","last_fullsized":"2023-07-13T22:23:15.108115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:09:14.618868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.134842+0000","last_change":"2023-07-13T22:23:15.134144+0000","last_active":"2023-07-13T22:23:15.134842+0000","last_peered":"2023-07-13T22:23:15.134842+0000","last_clean":"2023-07-13T22:23:15.134842+0000","last_became_active":"2023-07-13T22:23:15.133829+0000","last_became_peered":"2023-07-13T22:23:15.133829+0000","last_unstale":"2023-07-13T22:23:15.134842+0000","last_undegraded":"2023-07-13T22:23:15.134842+0000","last_fullsized":"2023-07-13T22:23:15.134842+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:18:54.363844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120708+0000","last_change":"2023-07-13T22:23:14.122179+0000","last_active":"2023-07-13T22:23:16.120708+0000","last_peered":"2023-07-13T22:23:16.120708+0000","last_clean":"2023-07-13T22:23:16.120708+0000","last_became_active":"2023-07-13T22:23:14.121796+0000","last_became_peered":"2023-07-13T22:23:14.121796+0000","last_unstale":"2023-07-13T22:23:16.120708+0000","last_undegraded":"2023-07-13T22:23:16.120708+0000","last_fullsized":"2023-07-13T22:23:16.120708+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:29:36.900241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130403+0000","last_change":"2023-07-13T22:23:15.123589+0000","last_active":"2023-07-13T22:23:15.130403+0000","last_peered":"2023-07-13T22:23:15.130403+0000","last_clean":"2023-07-13T22:23:15.130403+0000","last_became_active":"2023-07-13T22:23:15.123255+0000","last_became_peered":"2023-07-13T22:23:15.123255+0000","last_unstale":"2023-07-13T22:23:15.130403+0000","last_undegraded":"2023-07-13T22:23:15.130403+0000","last_fullsized":"2023-07-13T22:23:15.130403+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:16:19.713925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.109475+0000","last_change":"2023-07-13T22:23:14.113874+0000","last_active":"2023-07-13T22:23:15.109475+0000","last_peered":"2023-07-13T22:23:15.109475+0000","last_clean":"2023-07-13T22:23:15.109475+0000","last_became_active":"2023-07-13T22:23:14.113571+0000","last_became_peered":"2023-07-13T22:23:14.113571+0000","last_unstale":"2023-07-13T22:23:15.109475+0000","last_undegraded":"2023-07-13T22:23:15.109475+0000","last_fullsized":"2023-07-13T22:23:15.109475+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:19:35.126520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120435+0000","last_change":"2023-07-13T22:23:15.125234+0000","last_active":"2023-07-13T22:23:16.120435+0000","last_peered":"2023-07-13T22:23:16.120435+0000","last_clean":"2023-07-13T22:23:16.120435+0000","last_became_active":"2023-07-13T22:23:15.124919+0000","last_became_peered":"2023-07-13T22:23:15.124919+0000","last_unstale":"2023-07-13T22:23:16.120435+0000","last_undegraded":"2023-07-13T22:23:16.120435+0000","last_fullsized":"2023-07-13T22:23:16.120435+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:15:48.975568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.105579+0000","last_change":"2023-07-13T22:23:14.120862+0000","last_active":"2023-07-13T22:23:15.105579+0000","last_peered":"2023-07-13T22:23:15.105579+0000","last_clean":"2023-07-13T22:23:15.105579+0000","last_became_active":"2023-07-13T22:23:14.120599+0000","last_became_peered":"2023-07-13T22:23:14.120599+0000","last_unstale":"2023-07-13T22:23:15.105579+0000","last_undegraded":"2023-07-13T22:23:15.105579+0000","last_fullsized":"2023-07-13T22:23:15.105579+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:51:08.582430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121450+0000","last_change":"2023-07-13T22:23:15.136457+0000","last_active":"2023-07-13T22:23:16.121450+0000","last_peered":"2023-07-13T22:23:16.121450+0000","last_clean":"2023-07-13T22:23:16.121450+0000","last_became_active":"2023-07-13T22:23:15.136166+0000","last_became_peered":"2023-07-13T22:23:15.136166+0000","last_unstale":"2023-07-13T22:23:16.121450+0000","last_undegraded":"2023-07-13T22:23:16.121450+0000","last_fullsized":"2023-07-13T22:23:16.121450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:36:01.074855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.128002+0000","last_change":"2023-07-13T22:23:14.114283+0000","last_active":"2023-07-13T22:23:15.128002+0000","last_peered":"2023-07-13T22:23:15.128002+0000","last_clean":"2023-07-13T22:23:15.128002+0000","last_became_active":"2023-07-13T22:23:14.113985+0000","last_became_peered":"2023-07-13T22:23:14.113985+0000","last_unstale":"2023-07-13T22:23:15.128002+0000","last_undegraded":"2023-07-13T22:23:15.128002+0000","last_fullsized":"2023-07-13T22:23:15.128002+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:40:54.316211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.130455+0000","last_change":"2023-07-13T22:23:15.124130+0000","last_active":"2023-07-13T22:23:15.130455+0000","last_peered":"2023-07-13T22:23:15.130455+0000","last_clean":"2023-07-13T22:23:15.130455+0000","last_became_active":"2023-07-13T22:23:15.123355+0000","last_became_peered":"2023-07-13T22:23:15.123355+0000","last_unstale":"2023-07-13T22:23:15.130455+0000","last_undegraded":"2023-07-13T22:23:15.130455+0000","last_fullsized":"2023-07-13T22:23:15.130455+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:36.941256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.139081+0000","last_change":"2023-07-13T22:23:14.105197+0000","last_active":"2023-07-13T22:23:15.139081+0000","last_peered":"2023-07-13T22:23:15.139081+0000","last_clean":"2023-07-13T22:23:15.139081+0000","last_became_active":"2023-07-13T22:23:14.104913+0000","last_became_peered":"2023-07-13T22:23:14.104913+0000","last_unstale":"2023-07-13T22:23:15.139081+0000","last_undegraded":"2023-07-13T22:23:15.139081+0000","last_fullsized":"2023-07-13T22:23:15.139081+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:56:00.254478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120525+0000","last_change":"2023-07-13T22:23:15.117372+0000","last_active":"2023-07-13T22:23:16.120525+0000","last_peered":"2023-07-13T22:23:16.120525+0000","last_clean":"2023-07-13T22:23:16.120525+0000","last_became_active":"2023-07-13T22:23:15.117111+0000","last_became_peered":"2023-07-13T22:23:15.117111+0000","last_unstale":"2023-07-13T22:23:16.120525+0000","last_undegraded":"2023-07-13T22:23:16.120525+0000","last_fullsized":"2023-07-13T22:23:16.120525+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:34:21.416572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121547+0000","last_change":"2023-07-13T22:23:14.121723+0000","last_active":"2023-07-13T22:23:16.121547+0000","last_peered":"2023-07-13T22:23:16.121547+0000","last_clean":"2023-07-13T22:23:16.121547+0000","last_became_active":"2023-07-13T22:23:14.121451+0000","last_became_peered":"2023-07-13T22:23:14.121451+0000","last_unstale":"2023-07-13T22:23:16.121547+0000","last_undegraded":"2023-07-13T22:23:16.121547+0000","last_fullsized":"2023-07-13T22:23:16.121547+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:34:47.155749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121777+0000","last_change":"2023-07-13T22:23:15.131853+0000","last_active":"2023-07-13T22:23:16.121777+0000","last_peered":"2023-07-13T22:23:16.121777+0000","last_clean":"2023-07-13T22:23:16.121777+0000","last_became_active":"2023-07-13T22:23:15.129944+0000","last_became_peered":"2023-07-13T22:23:15.129944+0000","last_unstale":"2023-07-13T22:23:16.121777+0000","last_undegraded":"2023-07-13T22:23:16.121777+0000","last_fullsized":"2023-07-13T22:23:16.121777+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:36:54.293285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.110468+0000","last_change":"2023-07-13T22:23:14.121024+0000","last_active":"2023-07-13T22:23:15.110468+0000","last_peered":"2023-07-13T22:23:15.110468+0000","last_clean":"2023-07-13T22:23:15.110468+0000","last_became_active":"2023-07-13T22:23:14.120540+0000","last_became_peered":"2023-07-13T22:23:14.120540+0000","last_unstale":"2023-07-13T22:23:15.110468+0000","last_undegraded":"2023-07-13T22:23:15.110468+0000","last_fullsized":"2023-07-13T22:23:15.110468+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:02:20.754497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.127284+0000","last_change":"2023-07-13T22:23:15.116723+0000","last_active":"2023-07-13T22:23:15.127284+0000","last_peered":"2023-07-13T22:23:15.127284+0000","last_clean":"2023-07-13T22:23:15.127284+0000","last_became_active":"2023-07-13T22:23:15.116391+0000","last_became_peered":"2023-07-13T22:23:15.116391+0000","last_unstale":"2023-07-13T22:23:15.127284+0000","last_undegraded":"2023-07-13T22:23:15.127284+0000","last_fullsized":"2023-07-13T22:23:15.127284+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:54:17.085802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.114191+0000","last_change":"2023-07-13T22:23:14.120475+0000","last_active":"2023-07-13T22:23:15.114191+0000","last_peered":"2023-07-13T22:23:15.114191+0000","last_clean":"2023-07-13T22:23:15.114191+0000","last_became_active":"2023-07-13T22:23:14.120134+0000","last_became_peered":"2023-07-13T22:23:14.120134+0000","last_unstale":"2023-07-13T22:23:15.114191+0000","last_undegraded":"2023-07-13T22:23:15.114191+0000","last_fullsized":"2023-07-13T22:23:15.114191+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:14:41.654539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.122007+0000","last_change":"2023-07-13T22:23:15.133088+0000","last_active":"2023-07-13T22:23:16.122007+0000","last_peered":"2023-07-13T22:23:16.122007+0000","last_clean":"2023-07-13T22:23:16.122007+0000","last_became_active":"2023-07-13T22:23:15.130348+0000","last_became_peered":"2023-07-13T22:23:15.130348+0000","last_unstale":"2023-07-13T22:23:16.122007+0000","last_undegraded":"2023-07-13T22:23:16.122007+0000","last_fullsized":"2023-07-13T22:23:16.122007+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:18:12.038123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.134820+0000","last_change":"2023-07-13T22:23:14.106447+0000","last_active":"2023-07-13T22:23:15.134820+0000","last_peered":"2023-07-13T22:23:15.134820+0000","last_clean":"2023-07-13T22:23:15.134820+0000","last_became_active":"2023-07-13T22:23:14.106157+0000","last_became_peered":"2023-07-13T22:23:14.106157+0000","last_unstale":"2023-07-13T22:23:15.134820+0000","last_undegraded":"2023-07-13T22:23:15.134820+0000","last_fullsized":"2023-07-13T22:23:15.134820+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:10:22.920377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129651+0000","last_change":"2023-07-13T22:23:15.119012+0000","last_active":"2023-07-13T22:23:15.129651+0000","last_peered":"2023-07-13T22:23:15.129651+0000","last_clean":"2023-07-13T22:23:15.129651+0000","last_became_active":"2023-07-13T22:23:15.118757+0000","last_became_peered":"2023-07-13T22:23:15.118757+0000","last_unstale":"2023-07-13T22:23:15.129651+0000","last_undegraded":"2023-07-13T22:23:15.129651+0000","last_fullsized":"2023-07-13T22:23:15.129651+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:36.941256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121942+0000","last_change":"2023-07-13T22:23:14.121598+0000","last_active":"2023-07-13T22:23:16.121942+0000","last_peered":"2023-07-13T22:23:16.121942+0000","last_clean":"2023-07-13T22:23:16.121942+0000","last_became_active":"2023-07-13T22:23:14.119514+0000","last_became_peered":"2023-07-13T22:23:14.119514+0000","last_unstale":"2023-07-13T22:23:16.121942+0000","last_undegraded":"2023-07-13T22:23:16.121942+0000","last_fullsized":"2023-07-13T22:23:16.121942+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:05.834088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119384+0000","last_change":"2023-07-13T22:23:15.121648+0000","last_active":"2023-07-13T22:23:16.119384+0000","last_peered":"2023-07-13T22:23:16.119384+0000","last_clean":"2023-07-13T22:23:16.119384+0000","last_became_active":"2023-07-13T22:23:15.121343+0000","last_became_peered":"2023-07-13T22:23:15.121343+0000","last_unstale":"2023-07-13T22:23:16.119384+0000","last_undegraded":"2023-07-13T22:23:16.119384+0000","last_fullsized":"2023-07-13T22:23:16.119384+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:00:26.636577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.131938+0000","last_change":"2023-07-13T22:23:14.116045+0000","last_active":"2023-07-13T22:23:15.131938+0000","last_peered":"2023-07-13T22:23:15.131938+0000","last_clean":"2023-07-13T22:23:15.131938+0000","last_became_active":"2023-07-13T22:23:14.115723+0000","last_became_peered":"2023-07-13T22:23:14.115723+0000","last_unstale":"2023-07-13T22:23:15.131938+0000","last_undegraded":"2023-07-13T22:23:15.131938+0000","last_fullsized":"2023-07-13T22:23:15.131938+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:44:10.271316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.133175+0000","last_change":"2023-07-13T22:23:15.131522+0000","last_active":"2023-07-13T22:23:15.133175+0000","last_peered":"2023-07-13T22:23:15.133175+0000","last_clean":"2023-07-13T22:23:15.133175+0000","last_became_active":"2023-07-13T22:23:15.130992+0000","last_became_peered":"2023-07-13T22:23:15.130992+0000","last_unstale":"2023-07-13T22:23:15.133175+0000","last_undegraded":"2023-07-13T22:23:15.133175+0000","last_fullsized":"2023-07-13T22:23:15.133175+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:10:23.927784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129765+0000","last_change":"2023-07-13T22:23:14.117408+0000","last_active":"2023-07-13T22:23:15.129765+0000","last_peered":"2023-07-13T22:23:15.129765+0000","last_clean":"2023-07-13T22:23:15.129765+0000","last_became_active":"2023-07-13T22:23:14.117110+0000","last_became_peered":"2023-07-13T22:23:14.117110+0000","last_unstale":"2023-07-13T22:23:15.129765+0000","last_undegraded":"2023-07-13T22:23:15.129765+0000","last_fullsized":"2023-07-13T22:23:15.129765+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:39:11.744835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119293+0000","last_change":"2023-07-13T22:23:15.120214+0000","last_active":"2023-07-13T22:23:16.119293+0000","last_peered":"2023-07-13T22:23:16.119293+0000","last_clean":"2023-07-13T22:23:16.119293+0000","last_became_active":"2023-07-13T22:23:15.119923+0000","last_became_peered":"2023-07-13T22:23:15.119923+0000","last_unstale":"2023-07-13T22:23:16.119293+0000","last_undegraded":"2023-07-13T22:23:16.119293+0000","last_fullsized":"2023-07-13T22:23:16.119293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:03.126633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.109785+0000","last_change":"2023-07-13T22:23:14.119721+0000","last_active":"2023-07-13T22:23:15.109785+0000","last_peered":"2023-07-13T22:23:15.109785+0000","last_clean":"2023-07-13T22:23:15.109785+0000","last_became_active":"2023-07-13T22:23:14.119419+0000","last_became_peered":"2023-07-13T22:23:14.119419+0000","last_unstale":"2023-07-13T22:23:15.109785+0000","last_undegraded":"2023-07-13T22:23:15.109785+0000","last_fullsized":"2023-07-13T22:23:15.109785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:18:11.792486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120685+0000","last_change":"2023-07-13T22:23:15.124984+0000","last_active":"2023-07-13T22:23:16.120685+0000","last_peered":"2023-07-13T22:23:16.120685+0000","last_clean":"2023-07-13T22:23:16.120685+0000","last_became_active":"2023-07-13T22:23:15.124692+0000","last_became_peered":"2023-07-13T22:23:15.124692+0000","last_unstale":"2023-07-13T22:23:16.120685+0000","last_undegraded":"2023-07-13T22:23:16.120685+0000","last_fullsized":"2023-07-13T22:23:16.120685+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:32:32.253408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119360+0000","last_change":"2023-07-13T22:23:14.112933+0000","last_active":"2023-07-13T22:23:16.119360+0000","last_peered":"2023-07-13T22:23:16.119360+0000","last_clean":"2023-07-13T22:23:16.119360+0000","last_became_active":"2023-07-13T22:23:14.112657+0000","last_became_peered":"2023-07-13T22:23:14.112657+0000","last_unstale":"2023-07-13T22:23:16.119360+0000","last_undegraded":"2023-07-13T22:23:16.119360+0000","last_fullsized":"2023-07-13T22:23:16.119360+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:46:16.097850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132517+0000","last_change":"2023-07-13T22:23:15.129908+0000","last_active":"2023-07-13T22:23:15.132517+0000","last_peered":"2023-07-13T22:23:15.132517+0000","last_clean":"2023-07-13T22:23:15.132517+0000","last_became_active":"2023-07-13T22:23:15.129549+0000","last_became_peered":"2023-07-13T22:23:15.129549+0000","last_unstale":"2023-07-13T22:23:15.132517+0000","last_undegraded":"2023-07-13T22:23:15.132517+0000","last_fullsized":"2023-07-13T22:23:15.132517+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:32:18.064197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.103704+0000","last_change":"2023-07-13T22:23:14.105848+0000","last_active":"2023-07-13T22:23:15.103704+0000","last_peered":"2023-07-13T22:23:15.103704+0000","last_clean":"2023-07-13T22:23:15.103704+0000","last_became_active":"2023-07-13T22:23:14.105495+0000","last_became_peered":"2023-07-13T22:23:14.105495+0000","last_unstale":"2023-07-13T22:23:15.103704+0000","last_undegraded":"2023-07-13T22:23:15.103704+0000","last_fullsized":"2023-07-13T22:23:15.103704+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:46.323544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121965+0000","last_change":"2023-07-13T22:23:15.131910+0000","last_active":"2023-07-13T22:23:16.121965+0000","last_peered":"2023-07-13T22:23:16.121965+0000","last_clean":"2023-07-13T22:23:16.121965+0000","last_became_active":"2023-07-13T22:23:15.131597+0000","last_became_peered":"2023-07-13T22:23:15.131597+0000","last_unstale":"2023-07-13T22:23:16.121965+0000","last_undegraded":"2023-07-13T22:23:16.121965+0000","last_fullsized":"2023-07-13T22:23:16.121965+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:48:33.136472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121106+0000","last_change":"2023-07-13T22:23:14.123031+0000","last_active":"2023-07-13T22:23:16.121106+0000","last_peered":"2023-07-13T22:23:16.121106+0000","last_clean":"2023-07-13T22:23:16.121106+0000","last_became_active":"2023-07-13T22:23:14.122712+0000","last_became_peered":"2023-07-13T22:23:14.122712+0000","last_unstale":"2023-07-13T22:23:16.121106+0000","last_undegraded":"2023-07-13T22:23:16.121106+0000","last_fullsized":"2023-07-13T22:23:16.121106+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:00:36.722157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119337+0000","last_change":"2023-07-13T22:23:15.120375+0000","last_active":"2023-07-13T22:23:16.119337+0000","last_peered":"2023-07-13T22:23:16.119337+0000","last_clean":"2023-07-13T22:23:16.119337+0000","last_became_active":"2023-07-13T22:23:15.120048+0000","last_became_peered":"2023-07-13T22:23:15.120048+0000","last_unstale":"2023-07-13T22:23:16.119337+0000","last_undegraded":"2023-07-13T22:23:16.119337+0000","last_fullsized":"2023-07-13T22:23:16.119337+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:26:58.375517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.104050+0000","last_change":"2023-07-13T22:23:14.109287+0000","last_active":"2023-07-13T22:23:15.104050+0000","last_peered":"2023-07-13T22:23:15.104050+0000","last_clean":"2023-07-13T22:23:15.104050+0000","last_became_active":"2023-07-13T22:23:14.108282+0000","last_became_peered":"2023-07-13T22:23:14.108282+0000","last_unstale":"2023-07-13T22:23:15.104050+0000","last_undegraded":"2023-07-13T22:23:15.104050+0000","last_fullsized":"2023-07-13T22:23:15.104050+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:18:27.457575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121636+0000","last_change":"2023-07-13T22:23:15.133329+0000","last_active":"2023-07-13T22:23:16.121636+0000","last_peered":"2023-07-13T22:23:16.121636+0000","last_clean":"2023-07-13T22:23:16.121636+0000","last_became_active":"2023-07-13T22:23:15.130125+0000","last_became_peered":"2023-07-13T22:23:15.130125+0000","last_unstale":"2023-07-13T22:23:16.121636+0000","last_undegraded":"2023-07-13T22:23:16.121636+0000","last_fullsized":"2023-07-13T22:23:16.121636+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:03:59.750400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.104452+0000","last_change":"2023-07-13T22:23:14.109029+0000","last_active":"2023-07-13T22:23:15.104452+0000","last_peered":"2023-07-13T22:23:15.104452+0000","last_clean":"2023-07-13T22:23:15.104452+0000","last_became_active":"2023-07-13T22:23:14.108769+0000","last_became_peered":"2023-07-13T22:23:14.108769+0000","last_unstale":"2023-07-13T22:23:15.104452+0000","last_undegraded":"2023-07-13T22:23:15.104452+0000","last_fullsized":"2023-07-13T22:23:15.104452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:18:53.356437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.105325+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:23:15.105325+0000","last_peered":"2023-07-13T22:23:15.105325+0000","last_clean":"2023-07-13T22:23:15.105325+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:23:15.105325+0000","last_undegraded":"2023-07-13T22:23:15.105325+0000","last_fullsized":"2023-07-13T22:23:15.105325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:11:06.229784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120374+0000","last_change":"2023-07-13T22:23:15.121431+0000","last_active":"2023-07-13T22:23:16.120374+0000","last_peered":"2023-07-13T22:23:16.120374+0000","last_clean":"2023-07-13T22:23:16.120374+0000","last_became_active":"2023-07-13T22:23:15.121162+0000","last_became_peered":"2023-07-13T22:23:15.121162+0000","last_unstale":"2023-07-13T22:23:16.120374+0000","last_undegraded":"2023-07-13T22:23:16.120374+0000","last_fullsized":"2023-07-13T22:23:16.120374+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:58:06.919357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.103872+0000","last_change":"2023-07-13T22:23:14.114841+0000","last_active":"2023-07-13T22:23:15.103872+0000","last_peered":"2023-07-13T22:23:15.103872+0000","last_clean":"2023-07-13T22:23:15.103872+0000","last_became_active":"2023-07-13T22:23:14.114343+0000","last_became_peered":"2023-07-13T22:23:14.114343+0000","last_unstale":"2023-07-13T22:23:15.103872+0000","last_undegraded":"2023-07-13T22:23:15.103872+0000","last_fullsized":"2023-07-13T22:23:15.103872+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:14:20.398352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.104188+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:23:15.104188+0000","last_peered":"2023-07-13T22:23:15.104188+0000","last_clean":"2023-07-13T22:23:15.104188+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:23:15.104188+0000","last_undegraded":"2023-07-13T22:23:15.104188+0000","last_fullsized":"2023-07-13T22:23:15.104188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:25:34.228422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119436+0000","last_change":"2023-07-13T22:23:15.108681+0000","last_active":"2023-07-13T22:23:16.119436+0000","last_peered":"2023-07-13T22:23:16.119436+0000","last_clean":"2023-07-13T22:23:16.119436+0000","last_became_active":"2023-07-13T22:23:15.108390+0000","last_became_peered":"2023-07-13T22:23:15.108390+0000","last_unstale":"2023-07-13T22:23:16.119436+0000","last_undegraded":"2023-07-13T22:23:16.119436+0000","last_fullsized":"2023-07-13T22:23:16.119436+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:53:34.657146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.112377+0000","last_change":"2023-07-13T22:23:14.113623+0000","last_active":"2023-07-13T22:23:15.112377+0000","last_peered":"2023-07-13T22:23:15.112377+0000","last_clean":"2023-07-13T22:23:15.112377+0000","last_became_active":"2023-07-13T22:23:14.113259+0000","last_became_peered":"2023-07-13T22:23:14.113259+0000","last_unstale":"2023-07-13T22:23:15.112377+0000","last_undegraded":"2023-07-13T22:23:15.112377+0000","last_fullsized":"2023-07-13T22:23:15.112377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:26:54.959880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.119375+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:23:16.119375+0000","last_peered":"2023-07-13T22:23:16.119375+0000","last_clean":"2023-07-13T22:23:16.119375+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:23:16.119375+0000","last_undegraded":"2023-07-13T22:23:16.119375+0000","last_fullsized":"2023-07-13T22:23:16.119375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:55:46.456312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":214,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.108795+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:23:15.108795+0000","last_peered":"2023-07-13T22:23:15.108795+0000","last_clean":"2023-07-13T22:23:15.108795+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:23:15.108795+0000","last_undegraded":"2023-07-13T22:23:15.108795+0000","last_fullsized":"2023-07-13T22:23:15.108795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:53:34.882351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.132497+0000","last_change":"2023-07-13T22:23:15.129170+0000","last_active":"2023-07-13T22:23:15.132497+0000","last_peered":"2023-07-13T22:23:15.132497+0000","last_clean":"2023-07-13T22:23:15.132497+0000","last_became_active":"2023-07-13T22:23:15.128313+0000","last_became_peered":"2023-07-13T22:23:15.128313+0000","last_unstale":"2023-07-13T22:23:15.132497+0000","last_undegraded":"2023-07-13T22:23:15.132497+0000","last_fullsized":"2023-07-13T22:23:15.132497+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:13:33.612038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121473+0000","last_change":"2023-07-13T22:23:14.119635+0000","last_active":"2023-07-13T22:23:16.121473+0000","last_peered":"2023-07-13T22:23:16.121473+0000","last_clean":"2023-07-13T22:23:16.121473+0000","last_became_active":"2023-07-13T22:23:14.119226+0000","last_became_peered":"2023-07-13T22:23:14.119226+0000","last_unstale":"2023-07-13T22:23:16.121473+0000","last_undegraded":"2023-07-13T22:23:16.121473+0000","last_fullsized":"2023-07-13T22:23:16.121473+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:51:40.327041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.110026+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:23:15.110026+0000","last_peered":"2023-07-13T22:23:15.110026+0000","last_clean":"2023-07-13T22:23:15.110026+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:23:15.110026+0000","last_undegraded":"2023-07-13T22:23:15.110026+0000","last_fullsized":"2023-07-13T22:23:15.110026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:11:24.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.129180+0000","last_change":"2023-07-13T22:23:15.122571+0000","last_active":"2023-07-13T22:23:15.129180+0000","last_peered":"2023-07-13T22:23:15.129180+0000","last_clean":"2023-07-13T22:23:15.129180+0000","last_became_active":"2023-07-13T22:23:15.122246+0000","last_became_peered":"2023-07-13T22:23:15.122246+0000","last_unstale":"2023-07-13T22:23:15.129180+0000","last_undegraded":"2023-07-13T22:23:15.129180+0000","last_fullsized":"2023-07-13T22:23:15.129180+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:01:08.201816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121807+0000","last_change":"2023-07-13T22:23:14.119648+0000","last_active":"2023-07-13T22:23:16.121807+0000","last_peered":"2023-07-13T22:23:16.121807+0000","last_clean":"2023-07-13T22:23:16.121807+0000","last_became_active":"2023-07-13T22:23:14.119329+0000","last_became_peered":"2023-07-13T22:23:14.119329+0000","last_unstale":"2023-07-13T22:23:16.121807+0000","last_undegraded":"2023-07-13T22:23:16.121807+0000","last_fullsized":"2023-07-13T22:23:16.121807+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:33:32.807863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.108661+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:23:15.108661+0000","last_peered":"2023-07-13T22:23:15.108661+0000","last_clean":"2023-07-13T22:23:15.108661+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:23:15.108661+0000","last_undegraded":"2023-07-13T22:23:15.108661+0000","last_fullsized":"2023-07-13T22:23:15.108661+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:51:49.703548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120847+0000","last_change":"2023-07-13T22:23:15.118800+0000","last_active":"2023-07-13T22:23:16.120847+0000","last_peered":"2023-07-13T22:23:16.120847+0000","last_clean":"2023-07-13T22:23:16.120847+0000","last_became_active":"2023-07-13T22:23:15.118536+0000","last_became_peered":"2023-07-13T22:23:15.118536+0000","last_unstale":"2023-07-13T22:23:16.120847+0000","last_undegraded":"2023-07-13T22:23:16.120847+0000","last_fullsized":"2023-07-13T22:23:16.120847+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:45:25.191667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.134013+0000","last_change":"2023-07-13T22:23:14.120903+0000","last_active":"2023-07-13T22:23:15.134013+0000","last_peered":"2023-07-13T22:23:15.134013+0000","last_clean":"2023-07-13T22:23:15.134013+0000","last_became_active":"2023-07-13T22:23:14.120528+0000","last_became_peered":"2023-07-13T22:23:14.120528+0000","last_unstale":"2023-07-13T22:23:15.134013+0000","last_undegraded":"2023-07-13T22:23:15.134013+0000","last_fullsized":"2023-07-13T22:23:15.134013+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:38:25.947741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120694+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:23:16.120694+0000","last_peered":"2023-07-13T22:23:16.120694+0000","last_clean":"2023-07-13T22:23:16.120694+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:23:16.120694+0000","last_undegraded":"2023-07-13T22:23:16.120694+0000","last_fullsized":"2023-07-13T22:23:16.120694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:40.587332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.126259+0000","last_change":"2023-07-13T22:23:15.114584+0000","last_active":"2023-07-13T22:23:15.126259+0000","last_peered":"2023-07-13T22:23:15.126259+0000","last_clean":"2023-07-13T22:23:15.126259+0000","last_became_active":"2023-07-13T22:23:15.114191+0000","last_became_peered":"2023-07-13T22:23:15.114191+0000","last_unstale":"2023-07-13T22:23:15.126259+0000","last_undegraded":"2023-07-13T22:23:15.126259+0000","last_fullsized":"2023-07-13T22:23:15.126259+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:52:10.055506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.120796+0000","last_change":"2023-07-13T22:23:14.116691+0000","last_active":"2023-07-13T22:23:16.120796+0000","last_peered":"2023-07-13T22:23:16.120796+0000","last_clean":"2023-07-13T22:23:16.120796+0000","last_became_active":"2023-07-13T22:23:14.116369+0000","last_became_peered":"2023-07-13T22:23:14.116369+0000","last_unstale":"2023-07-13T22:23:16.120796+0000","last_undegraded":"2023-07-13T22:23:16.120796+0000","last_fullsized":"2023-07-13T22:23:16.120796+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:36.648050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.107536+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:23:15.107536+0000","last_peered":"2023-07-13T22:23:15.107536+0000","last_clean":"2023-07-13T22:23:15.107536+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:23:15.107536+0000","last_undegraded":"2023-07-13T22:23:15.107536+0000","last_fullsized":"2023-07-13T22:23:15.107536+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:46:39.914110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-07-13T22:23:16.121788+0000","last_change":"2023-07-13T22:23:15.130228+0000","last_active":"2023-07-13T22:23:16.121788+0000","last_peered":"2023-07-13T22:23:16.121788+0000","last_clean":"2023-07-13T22:23:16.121788+0000","last_became_active":"2023-07-13T22:23:15.129879+0000","last_became_peered":"2023-07-13T22:23:15.129879+0000","last_unstale":"2023-07-13T22:23:16.121788+0000","last_undegraded":"2023-07-13T22:23:16.121788+0000","last_fullsized":"2023-07-13T22:23:16.121788+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:14.088347+0000","last_clean_scrub_stamp":"2023-07-13T22:23:14.088347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:26:21.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.104480+0000","last_change":"2023-07-13T22:23:14.112992+0000","last_active":"2023-07-13T22:23:15.104480+0000","last_peered":"2023-07-13T22:23:15.104480+0000","last_clean":"2023-07-13T22:23:15.104480+0000","last_became_active":"2023-07-13T22:23:14.112663+0000","last_became_peered":"2023-07-13T22:23:14.112663+0000","last_unstale":"2023-07-13T22:23:15.104480+0000","last_undegraded":"2023-07-13T22:23:15.104480+0000","last_fullsized":"2023-07-13T22:23:15.104480+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:23:13.080941+0000","last_clean_scrub_stamp":"2023-07-13T22:23:13.080941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:05.358531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-07-13T22:23:15.108712+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:23:15.108712+0000","last_peered":"2023-07-13T22:23:15.108712+0000","last_clean":"2023-07-13T22:23:15.108712+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:23:15.108712+0000","last_undegraded":"2023-07-13T22:23:15.108712+0000","last_fullsized":"2023-07-13T22:23:15.108712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:28:59.688558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1167360,"data_stored":834620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.02,"5min":1.115,"15min":1.115},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.69599999999999995,"15min":0.71799999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77800000000000002,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88200000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.751}]},{"osd":2,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.80300000000000005,"15min":0.82399999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92300000000000004,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.79000000000000004,"15min":0.82199999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67500000000000004,"15min":0.68500000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90200000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76100000000000001,"15min":0.76500000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98799999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.79400000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90900000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.748,"15min":0.754},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84799999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66600000000000004,"15min":0.67300000000000004},"min":{"1min":0.40899999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91300000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79700000000000004},"min":{"1min":0.56100000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93500000000000005,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.89100000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.47199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93600000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94799999999999995,"5min":1.042,"15min":1.042},"last":0.71799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":573440,"data_stored":244780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78200000000000003,"5min":0.875,"15min":0.875},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.68200000000000005,"15min":0.69499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88200000000000001,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.65900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95099999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81499999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89100000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91900000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.75900000000000001,"15min":0.76700000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90500000000000003,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.76800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92600000000000005,"5min":1.111,"15min":1.111},"last":0.83599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1167360,"data_stored":834656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":1.952,"15min":1.952},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.51800000000000002,"15min":0.53000000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63600000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60499999999999998,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.60999999999999999,"15min":0.627},"min":{"1min":0.377,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.87,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91500000000000004,"5min":2.008,"15min":2.008},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64300000000000002,"5min":0.87,"15min":0.87},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.81499999999999995,"15min":0.83499999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96799999999999997,"5min":2.028,"15min":2.028},"last":0.752},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.80200000000000005,"15min":0.82099999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.75600000000000001,"15min":0.77000000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0189999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.85699999999999998,"15min":0.88200000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97899999999999998,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.79700000000000004,"15min":0.81499999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.95299999999999996,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.88400000000000001,"15min":0.90900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0389999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98099999999999998,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.82299999999999995,"15min":0.84099999999999997},"min":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.63300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":569344,"data_stored":243198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.60599999999999998,"15min":0.53500000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":11.791,"5min":11.791,"15min":11.791},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.55500000000000005,"15min":0.53600000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.657,"5min":4.657,"15min":4.657},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":4.5389999999999997,"5min":4.5389999999999997,"15min":4.5389999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.1930000000000001,"15min":1.1859999999999999},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":13.398999999999999,"5min":13.398999999999999,"15min":13.398999999999999},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0329999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.4470000000000001,"15min":1.456},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":13.667999999999999,"5min":13.667999999999999,"15min":13.667999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.81799999999999995,"15min":0.83799999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0980000000000001,"5min":1.776,"15min":1.776},"last":0.81499999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4610000000000001,"15min":1.466},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":13.349,"5min":13.349,"15min":13.349},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.91300000000000003,"15min":0.90400000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":4.7649999999999997,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.84199999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.86499999999999999,"15min":0.85699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.5039999999999996,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.3819999999999999,"15min":1.4379999999999999},"min":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":6.1870000000000003,"5min":11.679,"15min":11.679},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4630000000000001,"15min":1.4710000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":13.298,"5min":13.298,"15min":13.298},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.3500000000000001,"5min":0.98499999999999999,"15min":0.92400000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":11.456,"5min":11.456,"15min":11.456},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":573440,"data_stored":244706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.627,"5min":2.282,"15min":2.282},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46899999999999997,"15min":0.47199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.59499999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.39200000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.61399999999999999,"15min":0.63300000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78300000000000003,"5min":2.319,"15min":2.319},"last":0.432}]},{"osd":3,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74099999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.95999999999999996,"5min":2.351,"15min":2.351},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.86299999999999999,"15min":0.88100000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3600000000000001,"5min":2.238,"15min":2.238},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94099999999999995,"5min":1.518,"15min":1.518},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.59299999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.86799999999999999,"15min":0.89200000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.92100000000000004,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94999999999999996,"5min":1.214,"15min":1.214},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88300000000000001,"5min":1.29,"15min":1.29},"last":0.625},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83299999999999996,"15min":0.84499999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.99099999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.69499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35712,"kb_used_data":568,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336128,"statfs":{"total":96636764160,"available":96600195072,"internally_reserved":0,"allocated":581632,"data_stored":245714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42199999999999999,"15min":0.42699999999999999},"min":{"1min":0.28999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57199999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.439,"15min":0.437},"min":{"1min":0.312,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63700000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.371},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67000000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68899999999999995,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.65100000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61699999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72999999999999998,"15min":0.73999999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92400000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.879},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73599999999999999,"15min":0.74099999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.022,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.88300000000000001,"5min":1.411,"15min":1.411},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92600000000000005,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.53100000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.77600000000000002,"15min":0.78500000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.075,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.82999999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:22:51 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80500000000000005,"15min":0.81499999999999995},"min":{"1min":0.57399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.97899999999999998,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92400000000000004,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.96799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":556,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":569344,"data_stored":244710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.59599999999999997,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68100000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.626}]},{"osd":1,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.038,"5min":1.298,"15min":1.298},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.82599999999999996,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92700000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.71099999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0580000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":1.0349999999999999,"15min":1.109},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":10.657999999999999,"15min":10.657999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":1,"15min":1.0529999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92500000000000004,"5min":8.75,"15min":8.75},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73599999999999999,"15min":0.74399999999999999},"min":{"1min":0.442,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97299999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69399999999999995,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1,"5min":1.004,"15min":1.004},"last":0.752},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.78900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":565248,"data_stored":244128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.52800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80400000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93200000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.627},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.54000000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88900000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.58799999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88600000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.68700000000000006,"15min":0.70099999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81799999999999995,"5min":0.995,"15min":0.995},"last":0.495}]},{"osd":6,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93799999999999994,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44900000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.82399999999999995,"5min":1.02,"15min":1.02},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.52400000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.86199999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.72099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:23:20.058 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:23:20.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-13T22:23:20.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.318+0000 7fc744bff700 1 Processor -- start 2023-07-13T22:23:20.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.319+0000 7fc744bff700 1 -- start start 2023-07-13T22:23:20.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc744bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc740159ea0 0x7fc7400cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc744bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7400d0890 0x7fc7400d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc744bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74015d200 con 0x7fc74015c9b0 2023-07-13T22:23:20.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc744bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7400d2e60 con 0x7fc7400d0890 2023-07-13T22:23:20.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc744bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7400d2b50 con 0x7fc740159ea0 2023-07-13T22:23:20.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc73ed9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7400d0890 0x7fc7400d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc73ed9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7400d0890 0x7fc7400d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34756/0 (socket says 172.21.15.60:34756) 2023-07-13T22:23:20.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc73ed9d700 1 -- 172.21.15.60:0/3276808240 learned_addr learned my addr 172.21.15.60:0/3276808240 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:20.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc73e59c700 1 --2- 172.21.15.60:0/3276808240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc740159ea0 0x7fc7400cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc73ed9d700 1 -- 172.21.15.60:0/3276808240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc740159ea0 msgr2=0x7fc7400cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:20.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.320+0000 7fc73ed9d700 1 --2- 172.21.15.60:0/3276808240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc740159ea0 0x7fc7400cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.321+0000 7fc73ed9d700 1 -- 172.21.15.60:0/3276808240 >> v1:172.21.15.105:6789/0 conn(0x7fc74015c9b0 legacy=0x7fc7400d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.321+0000 7fc73ed9d700 1 -- 172.21.15.60:0/3276808240 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74015d4e0 con 0x7fc7400d0890 2023-07-13T22:23:20.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.321+0000 7fc73e59c700 1 --2- 172.21.15.60:0/3276808240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc740159ea0 0x7fc7400cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:20.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.321+0000 7fc73ed9d700 1 --2- 172.21.15.60:0/3276808240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7400d0890 0x7fc7400d0cf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc738009e50 tx=0x7fc73800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b80209b23e2758 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.321+0000 7fc73d59a700 1 -- 172.21.15.60:0/3276808240 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73800e070 con 0x7fc7400d0890 2023-07-13T22:23:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.321+0000 7fc73d59a700 1 -- 172.21.15.60:0/3276808240 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73800b9c0 con 0x7fc7400d0890 2023-07-13T22:23:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.321+0000 7fc73d59a700 1 -- 172.21.15.60:0/3276808240 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73800bd70 con 0x7fc7400d0890 2023-07-13T22:23:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.322+0000 7fc744bff700 1 -- 172.21.15.60:0/3276808240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7400d0890 msgr2=0x7fc7400d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.322+0000 7fc744bff700 1 --2- 172.21.15.60:0/3276808240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7400d0890 0x7fc7400d0cf0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc738009e50 tx=0x7fc73800b330 comp rx=0 tx=0).stop 2023-07-13T22:23:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.322+0000 7fc744bff700 1 -- 172.21.15.60:0/3276808240 shutdown_connections 2023-07-13T22:23:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.322+0000 7fc744bff700 1 --2- 172.21.15.60:0/3276808240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7400d0890 0x7fc7400d0cf0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.322+0000 7fc744bff700 1 --2- 172.21.15.60:0/3276808240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc740159ea0 0x7fc7400cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.322+0000 7fc744bff700 1 -- 172.21.15.60:0/3276808240 >> 172.21.15.60:0/3276808240 conn(0x7fc740071600 msgr2=0x7fc7400719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.322+0000 7fc744bff700 1 -- 172.21.15.60:0/3276808240 shutdown_connections 2023-07-13T22:23:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.322+0000 7fc744bff700 1 -- 172.21.15.60:0/3276808240 wait complete. 2023-07-13T22:23:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.324+0000 7fc744bff700 1 Processor -- start 2023-07-13T22:23:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.324+0000 7fc744bff700 1 -- start start 2023-07-13T22:23:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc744bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7400d0890 0x7fc74015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc744bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc740159ea0 0x7fc7401533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc744bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc74015c9b0 0x7fc7401538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc744bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7400c84b0 con 0x7fc740159ea0 2023-07-13T22:23:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc744bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7400c78e0 con 0x7fc7400d0890 2023-07-13T22:23:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc744bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7400c7b80 con 0x7fc74015c9b0 2023-07-13T22:23:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc740159ea0 0x7fc7401533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc740159ea0 0x7fc7401533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34772/0 (socket says 172.21.15.60:34772) 2023-07-13T22:23:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73dd9b700 1 -- 172.21.15.60:0/3007521965 learned_addr learned my addr 172.21.15.60:0/3007521965 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73e59c700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7400d0890 0x7fc74015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73ed9d700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc74015c9b0 0x7fc7401538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73dd9b700 1 -- 172.21.15.60:0/3007521965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc74015c9b0 msgr2=0x7fc7401538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73dd9b700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc74015c9b0 0x7fc7401538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73dd9b700 1 -- 172.21.15.60:0/3007521965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7400d0890 msgr2=0x7fc74015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73dd9b700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7400d0890 0x7fc74015ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73dd9b700 1 -- 172.21.15.60:0/3007521965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc738009b30 con 0x7fc740159ea0 2023-07-13T22:23:20.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.325+0000 7fc73ed9d700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc74015c9b0 0x7fc7401538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:20.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.326+0000 7fc73dd9b700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc740159ea0 0x7fc7401533b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c450 tx=0x7fc73400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:20.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.326+0000 7fc72effd700 1 -- 172.21.15.60:0/3007521965 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7340052d0 con 0x7fc740159ea0 2023-07-13T22:23:20.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.326+0000 7fc72effd700 1 -- 172.21.15.60:0/3007521965 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc734005430 con 0x7fc740159ea0 2023-07-13T22:23:20.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.326+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7400c7e60 con 0x7fc740159ea0 2023-07-13T22:23:20.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.326+0000 7fc72effd700 1 -- 172.21.15.60:0/3007521965 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7340038c0 con 0x7fc740159ea0 2023-07-13T22:23:20.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.326+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7400c8940 con 0x7fc740159ea0 2023-07-13T22:23:20.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.327+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc740067a70 con 0x7fc740159ea0 2023-07-13T22:23:20.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.328+0000 7fc72effd700 1 -- 172.21.15.60:0/3007521965 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc734003a20 con 0x7fc740159ea0 2023-07-13T22:23:20.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.329+0000 7fc72effd700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.330+0000 7fc72effd700 1 -- 172.21.15.60:0/3007521965 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7340888a0 con 0x7fc740159ea0 2023-07-13T22:23:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.330+0000 7fc73e59c700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.330+0000 7fc73e59c700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc730009930 tx=0x7fc730008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:20.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.334+0000 7fc72effd700 1 -- 172.21.15.60:0/3007521965 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc7340538b0 con 0x7fc740159ea0 2023-07-13T22:23:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.625+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc74004e210 con 0x7fc740159ea0 2023-07-13T22:23:20.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.629+0000 7fc72effd700 1 -- 172.21.15.60:0/3007521965 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc734056ed0 con 0x7fc740159ea0 2023-07-13T22:23:20.613 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:23:20.613 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:23:20.613 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-07-13T22:23:20.613 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-07-13T22:23:20.614 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-13T22:23:20.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 msgr2=0x7fc72006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc730009930 tx=0x7fc730008040 comp rx=0 tx=0).stop 2023-07-13T22:23:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc740159ea0 msgr2=0x7fc7401533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc740159ea0 0x7fc7401533b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c450 tx=0x7fc73400c810 comp rx=0 tx=0).stop 2023-07-13T22:23:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 shutdown_connections 2023-07-13T22:23:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc74015c9b0 0x7fc7401538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7400d0890 0x7fc74015ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 --2- 172.21.15.60:0/3007521965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc740159ea0 0x7fc7401533b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 >> 172.21.15.60:0/3007521965 conn(0x7fc740071600 msgr2=0x7fc7400d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 shutdown_connections 2023-07-13T22:23:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.632+0000 7fc744bff700 1 -- 172.21.15.60:0/3007521965 wait complete. 2023-07-13T22:23:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-07-13T22:23:20.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:23:20.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.853+0000 7fdfbcfff700 1 Processor -- start 2023-07-13T22:23:20.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.853+0000 7fdfbcfff700 1 -- start start 2023-07-13T22:23:20.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.854+0000 7fdfbcfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdfb80b2010 0x7fdfb80cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.854+0000 7fdfbcfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80d26d0 0x7fdfb80cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.854+0000 7fdfbcfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb80d14a0 con 0x7fdfb80b5a00 2023-07-13T22:23:20.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.854+0000 7fdfbcfff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb80d2570 con 0x7fdfb80b2010 2023-07-13T22:23:20.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.854+0000 7fdfbcfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80d1780 con 0x7fdfb80d26d0 2023-07-13T22:23:20.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.854+0000 7fdfb5d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80d26d0 0x7fdfb80cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.854+0000 7fdfb659c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdfb80b2010 0x7fdfb80cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.855+0000 7fdfb659c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdfb80b2010 0x7fdfb80cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:53888/0 (socket says 172.21.15.60:53888) 2023-07-13T22:23:20.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.855+0000 7fdfb5d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80d26d0 0x7fdfb80cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:44748/0 (socket says 172.21.15.60:44748) 2023-07-13T22:23:20.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.855+0000 7fdfb659c700 1 -- 172.21.15.60:0/4004140363 learned_addr learned my addr 172.21.15.60:0/4004140363 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:20.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.855+0000 7fdfb5d9b700 1 -- 172.21.15.60:0/4004140363 >> v2:172.21.15.105:3300/0 conn(0x7fdfb80b2010 msgr2=0x7fdfb80cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:20.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.855+0000 7fdfb5d9b700 1 --2- 172.21.15.60:0/4004140363 >> v2:172.21.15.105:3300/0 conn(0x7fdfb80b2010 0x7fdfb80cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.855+0000 7fdfb5d9b700 1 -- 172.21.15.60:0/4004140363 >> v1:172.21.15.105:6789/0 conn(0x7fdfb80b5a00 legacy=0x7fdfb80d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.855+0000 7fdfb5d9b700 1 -- 172.21.15.60:0/4004140363 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb80cd2a0 con 0x7fdfb80d26d0 2023-07-13T22:23:20.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.855+0000 7fdfb5d9b700 1 --2- 172.21.15.60:0/4004140363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80d26d0 0x7fdfb80cf6e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009840 tx=0x7fdfac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459dd07d614f8923 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:20.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfb559a700 1 -- 172.21.15.60:0/4004140363 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac011830 con 0x7fdfb80d26d0 2023-07-13T22:23:20.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfb559a700 1 -- 172.21.15.60:0/4004140363 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac011990 con 0x7fdfb80d26d0 2023-07-13T22:23:20.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfb559a700 1 -- 172.21.15.60:0/4004140363 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac011d20 con 0x7fdfb80d26d0 2023-07-13T22:23:20.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfbcfff700 1 -- 172.21.15.60:0/4004140363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80d26d0 msgr2=0x7fdfb80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfbcfff700 1 --2- 172.21.15.60:0/4004140363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80d26d0 0x7fdfb80cf6e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009840 tx=0x7fdfac009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:20.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfbcfff700 1 -- 172.21.15.60:0/4004140363 shutdown_connections 2023-07-13T22:23:20.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfbcfff700 1 --2- 172.21.15.60:0/4004140363 >> v2:172.21.15.105:3300/0 conn(0x7fdfb80b2010 0x7fdfb80cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfbcfff700 1 --2- 172.21.15.60:0/4004140363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80d26d0 0x7fdfb80cf6e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfbcfff700 1 -- 172.21.15.60:0/4004140363 >> 172.21.15.60:0/4004140363 conn(0x7fdfb806b380 msgr2=0x7fdfb806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:20.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfbcfff700 1 -- 172.21.15.60:0/4004140363 shutdown_connections 2023-07-13T22:23:20.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.856+0000 7fdfbcfff700 1 -- 172.21.15.60:0/4004140363 wait complete. 2023-07-13T22:23:20.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.859+0000 7fdfbcfff700 1 Processor -- start 2023-07-13T22:23:20.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.859+0000 7fdfbcfff700 1 -- start start 2023-07-13T22:23:20.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.860+0000 7fdfbcfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80b2010 0x7fdfb815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.860+0000 7fdfbcfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfb80b5a00 0x7fdfb8154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.860+0000 7fdfbcfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfb80d26d0 0x7fdfb81f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.860+0000 7fdfbcfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb815de70 con 0x7fdfb80b5a00 2023-07-13T22:23:20.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.860+0000 7fdfbcfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80cae70 con 0x7fdfb80d26d0 2023-07-13T22:23:20.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.860+0000 7fdfbcfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb815d8b0 con 0x7fdfb80b2010 2023-07-13T22:23:20.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.860+0000 7fdfb5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfb80b5a00 0x7fdfb8154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfb80b5a00 0x7fdfb8154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34782/0 (socket says 172.21.15.60:34782) 2023-07-13T22:23:20.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb5d9b700 1 -- 172.21.15.60:0/3789605279 learned_addr learned my addr 172.21.15.60:0/3789605279 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb659c700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80b2010 0x7fdfb815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb6d9d700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfb80d26d0 0x7fdfb81f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb5d9b700 1 -- 172.21.15.60:0/3789605279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80b2010 msgr2=0x7fdfb815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb5d9b700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80b2010 0x7fdfb815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb5d9b700 1 -- 172.21.15.60:0/3789605279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfb80d26d0 msgr2=0x7fdfb81f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb5d9b700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfb80d26d0 0x7fdfb81f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:20.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb5d9b700 1 -- 172.21.15.60:0/3789605279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac0094a0 con 0x7fdfb80b5a00 2023-07-13T22:23:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb659c700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80b2010 0x7fdfb815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.861+0000 7fdfb5d9b700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfb80b5a00 0x7fdfb8154fe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdfac014040 tx=0x7fdfac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.862+0000 7fdfa6ffd700 1 -- 172.21.15.60:0/3789605279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0036a0 con 0x7fdfb80b5a00 2023-07-13T22:23:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.862+0000 7fdfa6ffd700 1 -- 172.21.15.60:0/3789605279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac01f070 con 0x7fdfb80b5a00 2023-07-13T22:23:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.862+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb815db90 con 0x7fdfb80b5a00 2023-07-13T22:23:20.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.862+0000 7fdfa6ffd700 1 -- 172.21.15.60:0/3789605279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac003a00 con 0x7fdfb80b5a00 2023-07-13T22:23:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.862+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb815d560 con 0x7fdfb80b5a00 2023-07-13T22:23:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.863+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb80cacd0 con 0x7fdfb80b5a00 2023-07-13T22:23:20.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.864+0000 7fdfa6ffd700 1 -- 172.21.15.60:0/3789605279 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdfac003b60 con 0x7fdfb80b5a00 2023-07-13T22:23:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.865+0000 7fdfa6ffd700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf9806ace0 0x7fdf9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.865+0000 7fdfa6ffd700 1 -- 172.21.15.60:0/3789605279 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdfac021070 con 0x7fdfb80b5a00 2023-07-13T22:23:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.865+0000 7fdfb659c700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf9806ace0 0x7fdf9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:20.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.866+0000 7fdfb659c700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf9806ace0 0x7fdf9806d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009c20 tx=0x7fdfa8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:20.869+0000 7fdfa6ffd700 1 -- 172.21.15.60:0/3789605279 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdfb80cacd0 con 0x7fdfb80b5a00 2023-07-13T22:23:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.121+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfb80cb350 con 0x7fdfb80b5a00 2023-07-13T22:23:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.122+0000 7fdfa6ffd700 1 -- 172.21.15.60:0/3789605279 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fdfb80cb350 con 0x7fdfb80b5a00 2023-07-13T22:23:21.106 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:21.106 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3226044033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3226044033},{"type":"v1","addr":"172.21.15.105:6834","nonce":3226044033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5599,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2171011214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":2171011214},{"type":"v1","addr":"172.21.15.60:6836","nonce":2171011214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1623228645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1623228645},{"type":"v1","addr":"172.21.15.60:6837","nonce":1623228645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:23:14.616210+0000","modified":"2023-07-13T22:23:18.125486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5608},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5608":{"gid":5608,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/3570367595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3570367595},{"type":"v1","addr":"172.21.15.105:6835","nonce":3570367595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-13T22:23:21.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.124+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf9806ace0 msgr2=0x7fdf9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.124+0000 7fdfbcfff700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf9806ace0 0x7fdf9806d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009c20 tx=0x7fdfa8008040 comp rx=0 tx=0).stop 2023-07-13T22:23:21.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.124+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfb80b5a00 msgr2=0x7fdfb8154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.124+0000 7fdfbcfff700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfb80b5a00 0x7fdfb8154fe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdfac014040 tx=0x7fdfac004750 comp rx=0 tx=0).stop 2023-07-13T22:23:21.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.125+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 shutdown_connections 2023-07-13T22:23:21.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.125+0000 7fdfbcfff700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf9806ace0 0x7fdf9806d190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.125+0000 7fdfbcfff700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfb80d26d0 0x7fdfb81f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.125+0000 7fdfbcfff700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfb80b2010 0x7fdfb815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.125+0000 7fdfbcfff700 1 --2- 172.21.15.60:0/3789605279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfb80b5a00 0x7fdfb8154fe0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.125+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 >> 172.21.15.60:0/3789605279 conn(0x7fdfb806b380 msgr2=0x7fdfb80c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.125+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 shutdown_connections 2023-07-13T22:23:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.125+0000 7fdfbcfff700 1 -- 172.21.15.60:0/3789605279 wait complete. 2023-07-13T22:23:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2023-07-13T22:23:21.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:23:14.616210+0000', 'modified': '2023-07-13T22:23:18.125486+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5608}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5608': {'gid': 5608, 'name': 'd', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6835/3570367595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6833', 'nonce': 3570367595}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 3570367595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:23:21.530 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:23:21.530 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-13T22:23:21.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.755+0000 7f57a77ff700 1 Processor -- start 2023-07-13T22:23:21.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.755+0000 7f57a77ff700 1 -- start start 2023-07-13T22:23:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.756+0000 7f57a77ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57a00b8400 0x7f57a00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.756+0000 7f57a77ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f57a00d3750 0x7f57a00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.756+0000 7f57a77ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a00d0ac0 con 0x7f57a00cfad0 2023-07-13T22:23:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.756+0000 7f57a77ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a00cff50 con 0x7f57a00b8400 2023-07-13T22:23:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.756+0000 7f57a77ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a00b87d0 con 0x7f57a00d3750 2023-07-13T22:23:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.757+0000 7f57a559b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57a00b8400 0x7f57a00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.757+0000 7f57a559b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57a00b8400 0x7f57a00cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34794/0 (socket says 172.21.15.60:34794) 2023-07-13T22:23:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.757+0000 7f57a559b700 1 -- 172.21.15.60:0/3358634807 learned_addr learned my addr 172.21.15.60:0/3358634807 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.757+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/3358634807 >> v2:172.21.15.105:3300/0 conn(0x7f57a00d3750 0x7f57a00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.757+0000 7f57a559b700 1 -- 172.21.15.60:0/3358634807 >> v2:172.21.15.105:3300/0 conn(0x7f57a00d3750 msgr2=0x7f57a00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.757+0000 7f57a559b700 1 --2- 172.21.15.60:0/3358634807 >> v2:172.21.15.105:3300/0 conn(0x7f57a00d3750 0x7f57a00cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.757+0000 7f57a559b700 1 -- 172.21.15.60:0/3358634807 >> v1:172.21.15.105:6789/0 conn(0x7f57a00cfad0 legacy=0x7f57a015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.757+0000 7f57a559b700 1 -- 172.21.15.60:0/3358634807 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a00b4eb0 con 0x7f57a00b8400 2023-07-13T22:23:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f57a559b700 1 --2- 172.21.15.60:0/3358634807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 0x7f57a00cd240 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f57a00b8cf0 tx=0x7f579c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c4af5e12982013 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f5797fff700 1 -- 172.21.15.60:0/3358634807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c011830 con 0x7f57a00b8400 2023-07-13T22:23:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f5797fff700 1 -- 172.21.15.60:0/3358634807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c011990 con 0x7f57a00b8400 2023-07-13T22:23:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f5797fff700 1 -- 172.21.15.60:0/3358634807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c011d20 con 0x7f57a00b8400 2023-07-13T22:23:21.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f57a77ff700 1 -- 172.21.15.60:0/3358634807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 msgr2=0x7f57a00cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f57a77ff700 1 --2- 172.21.15.60:0/3358634807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 0x7f57a00cd240 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f57a00b8cf0 tx=0x7f579c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f57a77ff700 1 -- 172.21.15.60:0/3358634807 shutdown_connections 2023-07-13T22:23:21.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f57a77ff700 1 --2- 172.21.15.60:0/3358634807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 0x7f57a00cd240 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f57a77ff700 1 --2- 172.21.15.60:0/3358634807 >> v2:172.21.15.105:3300/0 conn(0x7f57a00d3750 0x7f57a00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.758+0000 7f57a77ff700 1 -- 172.21.15.60:0/3358634807 >> 172.21.15.60:0/3358634807 conn(0x7f57a006b390 msgr2=0x7f57a006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:21.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.759+0000 7f57a77ff700 1 -- 172.21.15.60:0/3358634807 shutdown_connections 2023-07-13T22:23:21.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.759+0000 7f57a77ff700 1 -- 172.21.15.60:0/3358634807 wait complete. 2023-07-13T22:23:21.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.761+0000 7f57a77ff700 1 Processor -- start 2023-07-13T22:23:21.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.761+0000 7f57a77ff700 1 -- start start 2023-07-13T22:23:21.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.762+0000 7f57a77ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 0x7f57a00c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.762+0000 7f57a77ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57a00cfad0 0x7f57a0153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.762+0000 7f57a77ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a00d3750 0x7f57a015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.762+0000 7f57a77ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00c9590 con 0x7f57a00b8400 2023-07-13T22:23:21.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.762+0000 7f57a77ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00c8250 con 0x7f57a00d3750 2023-07-13T22:23:21.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.762+0000 7f57a77ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00c8530 con 0x7f57a00cfad0 2023-07-13T22:23:21.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a559b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 0x7f57a00c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a559b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 0x7f57a00c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34796/0 (socket says 172.21.15.60:34796) 2023-07-13T22:23:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a559b700 1 -- 172.21.15.60:0/835377777 learned_addr learned my addr 172.21.15.60:0/835377777 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57a00cfad0 0x7f57a0153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a5d9c700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a00d3750 0x7f57a015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a559b700 1 -- 172.21.15.60:0/835377777 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57a00cfad0 msgr2=0x7f57a0153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a559b700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57a00cfad0 0x7f57a0153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a559b700 1 -- 172.21.15.60:0/835377777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a00d3750 msgr2=0x7f57a015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a559b700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a00d3750 0x7f57a015cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a559b700 1 -- 172.21.15.60:0/835377777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c009550 con 0x7f57a00b8400 2023-07-13T22:23:21.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57a00cfad0 0x7f57a0153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:21.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.763+0000 7f57a559b700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 0x7f57a00c9050 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f579c009130 tx=0x7f579c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:21.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.764+0000 7f5795ffb700 1 -- 172.21.15.60:0/835377777 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c0036a0 con 0x7f57a00b8400 2023-07-13T22:23:21.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.764+0000 7f5795ffb700 1 -- 172.21.15.60:0/835377777 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c01e070 con 0x7f57a00b8400 2023-07-13T22:23:21.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.764+0000 7f5795ffb700 1 -- 172.21.15.60:0/835377777 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c003a00 con 0x7f57a00b8400 2023-07-13T22:23:21.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.764+0000 7f57a77ff700 1 -- 172.21.15.60:0/835377777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a00c8810 con 0x7f57a00b8400 2023-07-13T22:23:21.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.764+0000 7f57a77ff700 1 -- 172.21.15.60:0/835377777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a01e6ac0 con 0x7f57a00b8400 2023-07-13T22:23:21.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.767+0000 7f5795ffb700 1 -- 172.21.15.60:0/835377777 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f579c003b60 con 0x7f57a00b8400 2023-07-13T22:23:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.768+0000 7f5795ffb700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f578406ace0 0x7f578406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.768+0000 7f5795ffb700 1 -- 172.21.15.60:0/835377777 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f579c08a470 con 0x7f57a00b8400 2023-07-13T22:23:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.768+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f578406ace0 0x7f578406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.769+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f578406ace0 0x7f578406d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f57a01f53c0 tx=0x7f5790008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:21.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.770+0000 7f57bb052b80 1 Processor -- start 2023-07-13T22:23:21.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.770+0000 7f57bb052b80 1 -- start start 2023-07-13T22:23:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.770+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432880 0x558bd64625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.770+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd6462b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432c50 0x558bd644d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57bb052b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd63acea0 con 0x558bd6434dc0 2023-07-13T22:23:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57bb052b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd6320600 con 0x558bd6432c50 2023-07-13T22:23:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57bb052b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bd638f910 con 0x558bd6432880 2023-07-13T22:23:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a4d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd6462b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a4d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd6462b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34800/0 (socket says 172.21.15.60:34800) 2023-07-13T22:23:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a4d9a700 1 -- 172.21.15.60:0/608386658 learned_addr learned my addr 172.21.15.60:0/608386658 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a5d9c700 1 --2- 172.21.15.60:0/608386658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432c50 0x558bd644d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a559b700 1 --2- 172.21.15.60:0/608386658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432880 0x558bd64625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a4d9a700 1 -- 172.21.15.60:0/608386658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432880 msgr2=0x558bd64625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/608386658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432880 0x558bd64625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a4d9a700 1 -- 172.21.15.60:0/608386658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432c50 msgr2=0x558bd644d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/608386658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432c50 0x558bd644d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.771+0000 7f57a4d9a700 1 -- 172.21.15.60:0/608386658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bd617c790 con 0x558bd6434dc0 2023-07-13T22:23:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.772+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/608386658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd6462b20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5790011c90 tx=0x7f5790014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c8208e3e851d5f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.772+0000 7f5782ffd700 1 -- 172.21.15.60:0/608386658 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57900158d0 con 0x558bd6434dc0 2023-07-13T22:23:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.772+0000 7f5782ffd700 1 -- 172.21.15.60:0/608386658 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5790015a30 con 0x558bd6434dc0 2023-07-13T22:23:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.772+0000 7f5782ffd700 1 -- 172.21.15.60:0/608386658 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790015da0 con 0x558bd6434dc0 2023-07-13T22:23:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.772+0000 7f57bb052b80 1 -- 172.21.15.60:0/608386658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 msgr2=0x558bd6462b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.772+0000 7f57bb052b80 1 --2- 172.21.15.60:0/608386658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd6462b20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5790011c90 tx=0x7f5790014000 comp rx=0 tx=0).stop 2023-07-13T22:23:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.773+0000 7f57bb052b80 1 -- 172.21.15.60:0/608386658 shutdown_connections 2023-07-13T22:23:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.773+0000 7f57bb052b80 1 --2- 172.21.15.60:0/608386658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432c50 0x558bd644d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.773+0000 7f57bb052b80 1 --2- 172.21.15.60:0/608386658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432880 0x558bd64625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.773+0000 7f57bb052b80 1 --2- 172.21.15.60:0/608386658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd6462b20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.773+0000 7f57bb052b80 1 -- 172.21.15.60:0/608386658 >> 172.21.15.60:0/608386658 conn(0x558bd642d3b0 msgr2=0x558bd6438550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.773+0000 7f57bb052b80 1 -- 172.21.15.60:0/608386658 shutdown_connections 2023-07-13T22:23:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.773+0000 7f57bb052b80 1 -- 172.21.15.60:0/608386658 wait complete. 2023-07-13T22:23:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.773+0000 7f57bb052b80 1 -- start start 2023-07-13T22:23:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.774+0000 7f57bb052b80 1 Processor -- start 2023-07-13T22:23:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432880 0x558bd644c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432c50 0x558bd64766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6434dc0 0x558bd64331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57bb052b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd6388be0 con 0x558bd6432880 2023-07-13T22:23:21.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57bb052b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd63465b0 con 0x558bd6432c50 2023-07-13T22:23:21.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57bb052b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bd632c910 con 0x558bd6434dc0 2023-07-13T22:23:21.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57bb052b80 10 client.0 _ll_register_callbacks cb 0x558bd6344b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:21.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57a559b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432880 0x558bd644c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57a559b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432880 0x558bd644c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34806/0 (socket says 172.21.15.60:34806) 2023-07-13T22:23:21.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57a559b700 1 -- 172.21.15.60:0/340966680 learned_addr learned my addr 172.21.15.60:0/340966680 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:21.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57a559b700 1 -- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6434dc0 msgr2=0x558bd64331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:21.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57a559b700 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6434dc0 0x558bd64331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57a559b700 1 -- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432c50 msgr2=0x558bd64766a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:21.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57a559b700 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432c50 0x558bd64766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57a559b700 1 -- 172.21.15.60:0/340966680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5790011910 con 0x558bd6432880 2023-07-13T22:23:21.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f57a559b700 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432880 0x558bd644c1d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f579c012fd0 tx=0x7f579c058e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a577b9b818afc2e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:21.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.776+0000 7f577bfff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:23:21.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f577bfff700 1 -- 172.21.15.60:0/340966680 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c08a2b0 con 0x558bd6432880 2023-07-13T22:23:21.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f577bfff700 1 -- 172.21.15.60:0/340966680 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f579c05d190 con 0x558bd6432880 2023-07-13T22:23:21.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f57bb052b80 1 -- 172.21.15.60:0/340966680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bd642a9c0 con 0x558bd6432880 2023-07-13T22:23:21.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f577bfff700 1 -- 172.21.15.60:0/340966680 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f579c061520 con 0x558bd6432880 2023-07-13T22:23:21.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f57bb052b80 10 client.5943 fetch_fsmap learned FSMap version 24 2023-07-13T22:23:21.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f57bb052b80 1 -- 172.21.15.60:0/340966680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x558bd62690a0 con 0x558bd6432880 2023-07-13T22:23:21.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f577bfff700 1 -- 172.21.15.60:0/340966680 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c077030 con 0x558bd6432880 2023-07-13T22:23:21.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f577bfff700 1 -- 172.21.15.60:0/340966680 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f579c0586c0 con 0x558bd6432880 2023-07-13T22:23:21.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f57bb052b80 10 client.5943 fetch_fsmap finished waiting for FSMap version 24 2023-07-13T22:23:21.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f57bb052b80 10 client.5943 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-13T22:23:21.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f57bb052b80 20 client.5943 populate_metadata read hostname 'smithi060' 2023-07-13T22:23:21.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f57bb052b80 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] conn(0x558bd64752a0 0x558bd6475670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f57bb052b80 4 client.5943 mds_command: new command op to 5608 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.777+0000 7f57bb052b80 1 -- 172.21.15.60:0/340966680 --> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bd6407170 con 0x558bd64752a0 2023-07-13T22:23:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.778+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] conn(0x558bd64752a0 0x558bd6475670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.778+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] conn(0x558bd64752a0 0x558bd6475670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a85beb89b41c2c21 server_cookie=75f9fbd58622cd79 in_seq=0 out_seq=0 2023-07-13T22:23:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.778+0000 7f577bfff700 10 client.5943 ms_handle_connect on v2:172.21.15.105:6833/3570367595 2023-07-13T22:23:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.779+0000 7f577bfff700 1 -- 172.21.15.60:0/340966680 <== mds.0 v2:172.21.15.105:6833/3570367595 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558bd6407170 con 0x558bd64752a0 2023-07-13T22:23:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.779+0000 7f577bfff700 10 client.5943 handle_command_reply: tid=0 2023-07-13T22:23:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.779+0000 7f57bb052b80 1 client.5943 shutdown 2023-07-13T22:23:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.779+0000 7f57bb052b80 1 -- 172.21.15.60:0/340966680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432880 msgr2=0x558bd644c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.779+0000 7f57bb052b80 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432880 0x558bd644c1d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f579c012fd0 tx=0x7f579c058e30 comp rx=0 tx=0).stop 2023-07-13T22:23:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f57bb052b80 1 -- 172.21.15.60:0/340966680 shutdown_connections 2023-07-13T22:23:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f57bb052b80 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] conn(0x558bd64752a0 0x558bd6475670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f57bb052b80 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432880 0x558bd644c1d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f57bb052b80 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6434dc0 0x558bd64331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f57bb052b80 1 --2- 172.21.15.60:0/340966680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432c50 0x558bd64766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f577bfff700 0 client.5943 ms_handle_reset on v2:172.21.15.105:6833/3570367595 2023-07-13T22:23:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f57bb052b80 1 -- 172.21.15.60:0/340966680 >> 172.21.15.60:0/340966680 conn(0x558bd642d3b0 msgr2=0x558bd6437d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f57bb052b80 1 -- 172.21.15.60:0/340966680 shutdown_connections 2023-07-13T22:23:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f57bb052b80 1 -- 172.21.15.60:0/340966680 wait complete. 2023-07-13T22:23:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.780+0000 7f57bb052b80 20 client.5943 trim_cache size 0 max 16384 2023-07-13T22:23:21.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.798+0000 7f57bb052b80 1 Processor -- start 2023-07-13T22:23:21.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.798+0000 7f57bb052b80 1 -- start start 2023-07-13T22:23:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 0x558bd64625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432c50 0x558bd6462b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6434dc0 0x558bd6431640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57bb052b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd617d390 con 0x558bd6432c50 2023-07-13T22:23:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57bb052b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd647fbe0 con 0x558bd6432880 2023-07-13T22:23:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57bb052b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bd6414c50 con 0x558bd6434dc0 2023-07-13T22:23:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a4d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432c50 0x558bd6462b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a4d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432c50 0x558bd6462b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34812/0 (socket says 172.21.15.60:34812) 2023-07-13T22:23:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a4d9a700 1 -- 172.21.15.60:0/1166336110 learned_addr learned my addr 172.21.15.60:0/1166336110 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a559b700 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 0x558bd64625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a5d9c700 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6434dc0 0x558bd6431640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a4d9a700 1 -- 172.21.15.60:0/1166336110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6434dc0 msgr2=0x558bd6431640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6434dc0 0x558bd6431640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a4d9a700 1 -- 172.21.15.60:0/1166336110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 msgr2=0x558bd64625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 0x558bd64625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.799+0000 7f57a4d9a700 1 -- 172.21.15.60:0/1166336110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bd634eb10 con 0x558bd6432c50 2023-07-13T22:23:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.800+0000 7f57a559b700 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 0x558bd64625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.800+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432c50 0x558bd6462b10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5790008010 tx=0x7f5790020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd73ffb45f69176 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.800+0000 7f57837fe700 1 -- 172.21.15.60:0/1166336110 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790015ba0 con 0x558bd6432c50 2023-07-13T22:23:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.800+0000 7f57837fe700 1 -- 172.21.15.60:0/1166336110 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5790032040 con 0x558bd6432c50 2023-07-13T22:23:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.800+0000 7f57837fe700 1 -- 172.21.15.60:0/1166336110 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790015d00 con 0x558bd6432c50 2023-07-13T22:23:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.800+0000 7f57bb052b80 1 -- 172.21.15.60:0/1166336110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432c50 msgr2=0x558bd6462b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.801+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432c50 0x558bd6462b10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5790008010 tx=0x7f5790020000 comp rx=0 tx=0).stop 2023-07-13T22:23:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.801+0000 7f57bb052b80 1 -- 172.21.15.60:0/1166336110 shutdown_connections 2023-07-13T22:23:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.801+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6434dc0 0x558bd6431640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.801+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 0x558bd64625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.801+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1166336110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6432c50 0x558bd6462b10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.801+0000 7f57bb052b80 1 -- 172.21.15.60:0/1166336110 >> 172.21.15.60:0/1166336110 conn(0x558bd642d3b0 msgr2=0x558bd6438520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.801+0000 7f57bb052b80 1 -- 172.21.15.60:0/1166336110 shutdown_connections 2023-07-13T22:23:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.801+0000 7f57bb052b80 1 -- 172.21.15.60:0/1166336110 wait complete. 2023-07-13T22:23:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.801+0000 7f57bb052b80 1 -- start start 2023-07-13T22:23:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.802+0000 7f57bb052b80 1 Processor -- start 2023-07-13T22:23:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 0x558bd648ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432c50 0x558bd6431a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57bb052b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd64e5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57bb052b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd6496e50 con 0x558bd6434dc0 2023-07-13T22:23:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57bb052b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd64e5b50 con 0x558bd6432880 2023-07-13T22:23:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57bb052b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bd6431f80 con 0x558bd6432c50 2023-07-13T22:23:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57bb052b80 10 client.0 _ll_register_callbacks cb 0x7f579c058fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a5d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd64e5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a5d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd64e5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34828/0 (socket says 172.21.15.60:34828) 2023-07-13T22:23:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a5d9c700 1 -- 172.21.15.60:0/1349046006 learned_addr learned my addr 172.21.15.60:0/1349046006 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a4d9a700 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432c50 0x558bd6431a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a559b700 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 0x558bd648ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a5d9c700 1 -- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432c50 msgr2=0x558bd6431a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a5d9c700 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432c50 0x558bd6431a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a5d9c700 1 -- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 msgr2=0x558bd648ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a5d9c700 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 0x558bd648ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.803+0000 7f57a5d9c700 1 -- 172.21.15.60:0/1349046006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bd617c790 con 0x558bd6434dc0 2023-07-13T22:23:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.804+0000 7f57a5d9c700 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd64e5110 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f57a01e8b80 tx=0x7f5798005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2ef56503275462 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.804+0000 7f577bfff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:23:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.804+0000 7f577bfff700 1 -- 172.21.15.60:0/1349046006 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798015680 con 0x558bd6434dc0 2023-07-13T22:23:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.804+0000 7f577bfff700 1 -- 172.21.15.60:0/1349046006 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57980157e0 con 0x558bd6434dc0 2023-07-13T22:23:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.804+0000 7f57bb052b80 1 -- 172.21.15.60:0/1349046006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5790015a30 con 0x558bd6434dc0 2023-07-13T22:23:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.804+0000 7f577bfff700 1 -- 172.21.15.60:0/1349046006 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57980159a0 con 0x558bd6434dc0 2023-07-13T22:23:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.804+0000 7f57bb052b80 10 client.5949 fetch_fsmap learned FSMap version 24 2023-07-13T22:23:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.804+0000 7f57bb052b80 1 -- 172.21.15.60:0/1349046006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x558bd61dd1d0 con 0x558bd6434dc0 2023-07-13T22:23:21.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.804+0000 7f577bfff700 1 -- 172.21.15.60:0/1349046006 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5790015a30 con 0x558bd6434dc0 2023-07-13T22:23:21.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.805+0000 7f577bfff700 1 -- 172.21.15.60:0/1349046006 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5798015b00 con 0x558bd6434dc0 2023-07-13T22:23:21.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.805+0000 7f57bb052b80 10 client.5949 fetch_fsmap finished waiting for FSMap version 24 2023-07-13T22:23:21.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.805+0000 7f57bb052b80 10 client.5949 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-13T22:23:21.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.805+0000 7f57bb052b80 20 client.5949 populate_metadata read hostname 'smithi060' 2023-07-13T22:23:21.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.805+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] conn(0x558bd6473200 0x558bd64735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:21.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.805+0000 7f57bb052b80 4 client.5949 mds_command: new command op to 5608 tid=0[{"prefix": "status"}] 2023-07-13T22:23:21.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.805+0000 7f57bb052b80 1 -- 172.21.15.60:0/1349046006 --> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558bd6381d20 con 0x558bd6473200 2023-07-13T22:23:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.805+0000 7f57a559b700 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] conn(0x558bd6473200 0x558bd64735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57a559b700 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] conn(0x558bd6473200 0x558bd64735d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a08c3ec22caa1bf3 server_cookie=16a861afb6a9a5c5 in_seq=0 out_seq=0 2023-07-13T22:23:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f577bfff700 10 client.5949 ms_handle_connect on v2:172.21.15.105:6833/3570367595 2023-07-13T22:23:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f577bfff700 1 -- 172.21.15.60:0/1349046006 <== mds.0 v2:172.21.15.105:6833/3570367595 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x558bd6381d20 con 0x558bd6473200 2023-07-13T22:23:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f577bfff700 10 client.5949 handle_command_reply: tid=0 2023-07-13T22:23:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 client.5949 shutdown 2023-07-13T22:23:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 -- 172.21.15.60:0/1349046006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 msgr2=0x558bd64e5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd64e5110 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f57a01e8b80 tx=0x7f5798005560 comp rx=0 tx=0).stop 2023-07-13T22:23:21.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 -- 172.21.15.60:0/1349046006 shutdown_connections 2023-07-13T22:23:21.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:6833/3570367595,v1:172.21.15.105:6835/3570367595] conn(0x558bd6473200 0x558bd64735d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558bd6434dc0 0x558bd64e5110 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bd6432880 0x558bd648ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 --2- 172.21.15.60:0/1349046006 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x558bd6432c50 0x558bd6431a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f577bfff700 0 client.5949 ms_handle_reset on v2:172.21.15.105:6833/3570367595 2023-07-13T22:23:21.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 -- 172.21.15.60:0/1349046006 >> 172.21.15.60:0/1349046006 conn(0x558bd642d3b0 msgr2=0x558bd6438520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:21.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 -- 172.21.15.60:0/1349046006 shutdown_connections 2023-07-13T22:23:21.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.806+0000 7f57bb052b80 1 -- 172.21.15.60:0/1349046006 wait complete. 2023-07-13T22:23:21.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57bb052b80 20 client.5949 trim_cache size 0 max 16384 2023-07-13T22:23:21.797 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:21.797 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:23:21.797 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:23:21.798 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5608, 2023-07-13T22:23:21.798 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:23:21.798 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:23:21.798 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:23:21.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.709014421, 2023-07-13T22:23:21.798 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 24, 2023-07-13T22:23:21.798 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 49, 2023-07-13T22:23:21.799 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 49, 2023-07-13T22:23:21.799 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 13.599642187000001 2023-07-13T22:23:21.799 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:21.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 -- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f578406ace0 msgr2=0x7f578406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f578406ace0 0x7f578406d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f57a01f53c0 tx=0x7f5790008040 comp rx=0 tx=0).stop 2023-07-13T22:23:21.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 -- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 msgr2=0x7f57a00c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:21.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 0x7f57a00c9050 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f579c009130 tx=0x7f579c004750 comp rx=0 tx=0).stop 2023-07-13T22:23:21.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 -- 172.21.15.60:0/835377777 shutdown_connections 2023-07-13T22:23:21.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f578406ace0 0x7f578406d190 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00b8400 0x7f57a00c9050 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a00d3750 0x7f57a015cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 --2- 172.21.15.60:0/835377777 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57a00cfad0 0x7f57a0153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 -- 172.21.15.60:0/835377777 >> 172.21.15.60:0/835377777 conn(0x7f57a006b390 msgr2=0x7f57a00b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 -- 172.21.15.60:0/835377777 shutdown_connections 2023-07-13T22:23:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:21.807+0000 7f57a77ff700 1 -- 172.21.15.60:0/835377777 wait complete. 2023-07-13T22:23:21.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:23:22.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.026+0000 7ff79ffff700 1 Processor -- start 2023-07-13T22:23:22.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.027+0000 7ff79ffff700 1 -- start start 2023-07-13T22:23:22.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7980d3750 0x7ff7980cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:22.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980cfad0 0x7ff7980d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:22.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79ffff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7980cd950 con 0x7ff7980b8400 2023-07-13T22:23:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79ffff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980cff50 con 0x7ff7980d3750 2023-07-13T22:23:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980d0ac0 con 0x7ff7980cfad0 2023-07-13T22:23:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79d59a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7980d3750 0x7ff7980cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79d59a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7980d3750 0x7ff7980cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34836/0 (socket says 172.21.15.60:34836) 2023-07-13T22:23:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79d59a700 1 -- 172.21.15.60:0/4291125695 learned_addr learned my addr 172.21.15.60:0/4291125695 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79e59c700 1 --2- 172.21.15.60:0/4291125695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980cfad0 0x7ff7980d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79d59a700 1 -- 172.21.15.60:0/4291125695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980cfad0 msgr2=0x7ff7980d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79d59a700 1 --2- 172.21.15.60:0/4291125695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980cfad0 0x7ff7980d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.028+0000 7ff79d59a700 1 -- 172.21.15.60:0/4291125695 >> v1:172.21.15.105:6789/0 conn(0x7ff7980b8400 legacy=0x7ff7980cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.029+0000 7ff79d59a700 1 -- 172.21.15.60:0/4291125695 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980b87d0 con 0x7ff7980d3750 2023-07-13T22:23:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.029+0000 7ff79d59a700 1 --2- 172.21.15.60:0/4291125695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980d3750 0x7ff7980cf590 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff7880098f0 tx=0x7ff788009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3219e2b23a7d95ca server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.029+0000 7ff79cd99700 1 -- 172.21.15.60:0/4291125695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788004230 con 0x7ff7980d3750 2023-07-13T22:23:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.029+0000 7ff79cd99700 1 -- 172.21.15.60:0/4291125695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff788004390 con 0x7ff7980d3750 2023-07-13T22:23:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.029+0000 7ff79cd99700 1 -- 172.21.15.60:0/4291125695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7880036a0 con 0x7ff7980d3750 2023-07-13T22:23:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.029+0000 7ff79ffff700 1 -- 172.21.15.60:0/4291125695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980d3750 msgr2=0x7ff7980cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.029+0000 7ff79ffff700 1 --2- 172.21.15.60:0/4291125695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980d3750 0x7ff7980cf590 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff7880098f0 tx=0x7ff788009c00 comp rx=0 tx=0).stop 2023-07-13T22:23:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.030+0000 7ff79ffff700 1 -- 172.21.15.60:0/4291125695 shutdown_connections 2023-07-13T22:23:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.030+0000 7ff79ffff700 1 --2- 172.21.15.60:0/4291125695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980cfad0 0x7ff7980d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:22.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.030+0000 7ff79ffff700 1 --2- 172.21.15.60:0/4291125695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980d3750 0x7ff7980cf590 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:22.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.030+0000 7ff79ffff700 1 -- 172.21.15.60:0/4291125695 >> 172.21.15.60:0/4291125695 conn(0x7ff79806b390 msgr2=0x7ff79806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:22.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.030+0000 7ff79ffff700 1 -- 172.21.15.60:0/4291125695 shutdown_connections 2023-07-13T22:23:22.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.030+0000 7ff79ffff700 1 -- 172.21.15.60:0/4291125695 wait complete. 2023-07-13T22:23:22.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.032+0000 7ff79ffff700 1 Processor -- start 2023-07-13T22:23:22.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.032+0000 7ff79ffff700 1 -- start start 2023-07-13T22:23:22.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980b8400 0x7ff798150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:22.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7980cfad0 0x7ff7980cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:22.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980d3750 0x7ff7980d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:22.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980c66b0 con 0x7ff7980b8400 2023-07-13T22:23:22.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79ffff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff798151470 con 0x7ff7980cfad0 2023-07-13T22:23:22.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cab90 con 0x7ff7980d3750 2023-07-13T22:23:22.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980b8400 0x7ff798150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:22.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980b8400 0x7ff798150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34848/0 (socket says 172.21.15.60:34848) 2023-07-13T22:23:22.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79dd9b700 1 -- 172.21.15.60:0/655021824 learned_addr learned my addr 172.21.15.60:0/655021824 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:22.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79d59a700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7980cfad0 0x7ff7980cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:22.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79e59c700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980d3750 0x7ff7980d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:22.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79dd9b700 1 -- 172.21.15.60:0/655021824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980d3750 msgr2=0x7ff7980d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:22.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79dd9b700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980d3750 0x7ff7980d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:22.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79dd9b700 1 -- 172.21.15.60:0/655021824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7980cfad0 msgr2=0x7ff7980cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:22.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79dd9b700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7980cfad0 0x7ff7980cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:22.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.034+0000 7ff79dd9b700 1 -- 172.21.15.60:0/655021824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788009550 con 0x7ff7980b8400 2023-07-13T22:23:22.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.035+0000 7ff79dd9b700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980b8400 0x7ff798150980 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b4d0 tx=0x7ff79400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.035+0000 7ff78e7fc700 1 -- 172.21.15.60:0/655021824 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794011830 con 0x7ff7980b8400 2023-07-13T22:23:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.035+0000 7ff78e7fc700 1 -- 172.21.15.60:0/655021824 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff794011990 con 0x7ff7980b8400 2023-07-13T22:23:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.035+0000 7ff78e7fc700 1 -- 172.21.15.60:0/655021824 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794011d40 con 0x7ff7980b8400 2023-07-13T22:23:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.035+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7980ca010 con 0x7ff7980b8400 2023-07-13T22:23:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.035+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7980ca5d0 con 0x7ff7980b8400 2023-07-13T22:23:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.036+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79815c970 con 0x7ff7980b8400 2023-07-13T22:23:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.037+0000 7ff78e7fc700 1 -- 172.21.15.60:0/655021824 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff79401a400 con 0x7ff7980b8400 2023-07-13T22:23:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.038+0000 7ff78e7fc700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff77c06ace0 0x7ff77c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.038+0000 7ff78e7fc700 1 -- 172.21.15.60:0/655021824 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff7940892f0 con 0x7ff7980b8400 2023-07-13T22:23:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.038+0000 7ff79d59a700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff77c06ace0 0x7ff77c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.039+0000 7ff79d59a700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff77c06ace0 0x7ff77c06d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff7980d3b60 tx=0x7ff788008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.042+0000 7ff78e7fc700 1 -- 172.21.15.60:0/655021824 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff79815c970 con 0x7ff7980b8400 2023-07-13T22:23:22.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.297+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff79815c970 con 0x7ff7980b8400 2023-07-13T22:23:22.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.297+0000 7ff78e7fc700 1 -- 172.21.15.60:0/655021824 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff794015060 con 0x7ff7980b8400 2023-07-13T22:23:22.281 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:23:22.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff77c06ace0 msgr2=0x7ff77c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff77c06ace0 0x7ff77c06d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff7980d3b60 tx=0x7ff788008030 comp rx=0 tx=0).stop 2023-07-13T22:23:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980b8400 msgr2=0x7ff798150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980b8400 0x7ff798150980 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b4d0 tx=0x7ff79400b890 comp rx=0 tx=0).stop 2023-07-13T22:23:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 shutdown_connections 2023-07-13T22:23:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff77c06ace0 0x7ff77c06d190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7980b8400 0x7ff798150980 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:22.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7980d3750 0x7ff7980d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:22.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 --2- 172.21.15.60:0/655021824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7980cfad0 0x7ff7980cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:22.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 >> 172.21.15.60:0/655021824 conn(0x7ff79806b390 msgr2=0x7ff7980d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:22.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 shutdown_connections 2023-07-13T22:23:22.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:22.300+0000 7ff79ffff700 1 -- 172.21.15.60:0/655021824 wait complete. 2023-07-13T22:23:22.301 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:23:22.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:23:22.302 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:23:22.302 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:23:22.302 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:22.303 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:23:22.303 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:23:22.303 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:23:22.303 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:23:22.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:22.304 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:23:22.351 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:23:22.351 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:23:22.351 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:23:22.352 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:22.352 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:23:22.352 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:22.352 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:23:22.352 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:22.352 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:23:22.353 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:23:22.353 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:22.353 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:23:22.353 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:23:22.353 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:22.353 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:23:22.353 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:23:22.354 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41571sec preferred_lft 41571sec 2023-07-13T22:23:22.354 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:23:22.354 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:22.354 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:23:22.354 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:22.354 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:23:22.354 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:23:22.355 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:22.355 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:23:22.355 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:22.355 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:23:22.355 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:22.355 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:23:22.355 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:23:22.356 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:23:22.356 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:22.356 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:23:22.356 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:23:22.356 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:23:22.356 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:22.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:22.358 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:23:22.426 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:23:22.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:22.429 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:23:22.478 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:23:22.478 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:23:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:23:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:23:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:23:22.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:22.480 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:23:22.481 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:23:22.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:23:22.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:23:22.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:23:22.483 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:23:22.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:22.656 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:23:22.703 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:23:22.703 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:23:22.704 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:23:22.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:22.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:23:22.816 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:23:22.816 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:23:22.817 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:22.863 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:23:22.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:23:22.863 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:22.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:23:23.059 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:23:23.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:23:23.131 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:23:23.134 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:23:23.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:23.135 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:23:23.181 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:23:23.181 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:23:23.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:23:23.300 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:23:23.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:23:23.373 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:23:23.375 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:23:23.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:23.377 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:23:23.399 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:23:23.416+0000 7f1518d3d5c0 -1 init, newargv = 0x5587a6884ff0 newargc=15 2023-07-13T22:23:23.400 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[132428]: starting ceph client 2023-07-13T22:23:23.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[132428]: starting fuse 2023-07-13T22:23:23.427 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:23:23.427 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:23:23.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:23.428 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:23.428 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:23.428 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:23.429 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:23.429 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:23.429 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:23.430 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:23.430 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:23.430 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:23.430 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:23.431 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:23.431 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:23.431 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:23.431 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:23.432 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:23.432 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:23.432 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:23.433 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:23.433 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:23.433 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:23.434 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:23.434 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:23.434 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:23.434 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:23.435 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:23.435 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:23.614 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:23.791 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:23.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:23.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.132428.asok status 2023-07-13T22:23:24.022 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:23:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:23:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:23:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:23:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:23:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:23:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "132428", 2023-07-13T22:23:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:23:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:23:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:23:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5961, 2023-07-13T22:23:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:23:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:23:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:23:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5961 2023-07-13T22:23:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2452753753 2023-07-13T22:23:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2452753753 2023-07-13T22:23:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5961 192.168.0.1:0/2452753753", 2023-07-13T22:23:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2452753753", 2023-07-13T22:23:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:23:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 24, 2023-07-13T22:23:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 49, 2023-07-13T22:23:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:23:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:23:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:23:24.027 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:24.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:24.037 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:23:24.037 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:24.084 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:23:24.085 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:24.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:24.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:24.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:23:24.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.377+0000 7fa7b37ff700 1 Processor -- start 2023-07-13T22:23:24.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.377+0000 7fa7b37ff700 1 -- start start 2023-07-13T22:23:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.378+0000 7fa7b37ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7ac0d3750 0x7fa7ac0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.378+0000 7fa7b37ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0cfad0 0x7fa7ac0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.378+0000 7fa7b37ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7ac0cd950 con 0x7fa7ac0b8400 2023-07-13T22:23:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.378+0000 7fa7b37ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac0cff50 con 0x7fa7ac0d3750 2023-07-13T22:23:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.378+0000 7fa7b37ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0d0ac0 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.378+0000 7fa7b0d9a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7ac0d3750 0x7fa7ac0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:24.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.378+0000 7fa7b159b700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa7ac0b8400 0x7fa7ac0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:37556/0 (socket says 172.21.15.60:37556) 2023-07-13T22:23:24.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.379+0000 7fa7b159b700 1 -- 172.21.15.60:0/707141632 learned_addr learned my addr 172.21.15.60:0/707141632 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:24.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.378+0000 7fa7b1d9c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0cfad0 0x7fa7ac0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:24.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.379+0000 7fa7a3fff700 1 -- 172.21.15.60:0/707141632 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707794291 0 0) 0x7fa7a8001110 con 0x7fa7ac0b8400 2023-07-13T22:23:24.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.379+0000 7fa7a3fff700 1 -- 172.21.15.60:0/707141632 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7980035b0 con 0x7fa7ac0b8400 2023-07-13T22:23:24.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.379+0000 7fa7b0d9a700 1 -- 172.21.15.60:0/707141632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0cfad0 msgr2=0x7fa7ac0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:24.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.379+0000 7fa7b0d9a700 1 --2- 172.21.15.60:0/707141632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0cfad0 0x7fa7ac0d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.379+0000 7fa7b0d9a700 1 -- 172.21.15.60:0/707141632 >> v1:172.21.15.105:6789/0 conn(0x7fa7ac0b8400 legacy=0x7fa7ac0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.379+0000 7fa7b0d9a700 1 -- 172.21.15.60:0/707141632 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac0b87d0 con 0x7fa7ac0d3750 2023-07-13T22:23:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.379+0000 7fa7b1d9c700 1 --2- 172.21.15.60:0/707141632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0cfad0 0x7fa7ac0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.379+0000 7fa7b0d9a700 1 --2- 172.21.15.60:0/707141632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0d3750 0x7fa7ac0cf590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009840 tx=0x7fa79c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0b7a86410449b5d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7a3fff700 1 -- 172.21.15.60:0/707141632 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c011830 con 0x7fa7ac0d3750 2023-07-13T22:23:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7a3fff700 1 -- 172.21.15.60:0/707141632 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c011990 con 0x7fa7ac0d3750 2023-07-13T22:23:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7a3fff700 1 -- 172.21.15.60:0/707141632 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c011d20 con 0x7fa7ac0d3750 2023-07-13T22:23:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7b37ff700 1 -- 172.21.15.60:0/707141632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0d3750 msgr2=0x7fa7ac0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7b37ff700 1 --2- 172.21.15.60:0/707141632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0d3750 0x7fa7ac0cf590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009840 tx=0x7fa79c009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7b37ff700 1 -- 172.21.15.60:0/707141632 shutdown_connections 2023-07-13T22:23:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7b37ff700 1 --2- 172.21.15.60:0/707141632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0cfad0 0x7fa7ac0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7b37ff700 1 --2- 172.21.15.60:0/707141632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0d3750 0x7fa7ac0cf590 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7b37ff700 1 -- 172.21.15.60:0/707141632 >> 172.21.15.60:0/707141632 conn(0x7fa7ac06b390 msgr2=0x7fa7ac06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7b37ff700 1 -- 172.21.15.60:0/707141632 shutdown_connections 2023-07-13T22:23:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.380+0000 7fa7b37ff700 1 -- 172.21.15.60:0/707141632 wait complete. 2023-07-13T22:23:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.382+0000 7fa7b37ff700 1 Processor -- start 2023-07-13T22:23:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.382+0000 7fa7b37ff700 1 -- start start 2023-07-13T22:23:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.383+0000 7fa7b37ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7ac0b8400 0x7fa7ac1f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.383+0000 7fa7b37ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0cfad0 0x7fa7ac1f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.383+0000 7fa7b37ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0d3750 0x7fa7ac1f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.383+0000 7fa7b37ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac1f2da0 con 0x7fa7ac0b8400 2023-07-13T22:23:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.383+0000 7fa7b37ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac1f5e70 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.383+0000 7fa7b37ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac1f6150 con 0x7fa7ac0d3750 2023-07-13T22:23:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.383+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7ac0b8400 0x7fa7ac1f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.383+0000 7fa7b159b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7ac0b8400 0x7fa7ac1f2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34858/0 (socket says 172.21.15.60:34858) 2023-07-13T22:23:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.383+0000 7fa7b159b700 1 -- 172.21.15.60:0/4143089304 learned_addr learned my addr 172.21.15.60:0/4143089304 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.384+0000 7fa7b0d9a700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0cfad0 0x7fa7ac1f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.384+0000 7fa7b1d9c700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0d3750 0x7fa7ac1f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:24.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.384+0000 7fa7b0d9a700 1 -- 172.21.15.60:0/4143089304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0d3750 msgr2=0x7fa7ac1f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.384+0000 7fa7b0d9a700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0d3750 0x7fa7ac1f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.384+0000 7fa7b0d9a700 1 -- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7ac0b8400 msgr2=0x7fa7ac1f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.384+0000 7fa7b0d9a700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7ac0b8400 0x7fa7ac1f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.384+0000 7fa7b0d9a700 1 -- 172.21.15.60:0/4143089304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4009a70 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.384+0000 7fa7b159b700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7ac0b8400 0x7fa7ac1f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.384+0000 7fa7b0d9a700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0cfad0 0x7fa7ac1f15d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa79c016000 tx=0x7fa79c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.385+0000 7fa7a1ffb700 1 -- 172.21.15.60:0/4143089304 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c004550 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.385+0000 7fa7a1ffb700 1 -- 172.21.15.60:0/4143089304 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c01f070 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.385+0000 7fa7a1ffb700 1 -- 172.21.15.60:0/4143089304 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0038e0 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.385+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c0094a0 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.385+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac1f2340 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.386+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ac1f6310 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.387+0000 7fa7a1ffb700 1 -- 172.21.15.60:0/4143089304 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa79c003a40 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.388+0000 7fa7a1ffb700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa79006ace0 0x7fa79006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:24.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.388+0000 7fa7a1ffb700 1 -- 172.21.15.60:0/4143089304 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa79c021070 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.388+0000 7fa7b159b700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa79006ace0 0x7fa79006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:24.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.389+0000 7fa7b159b700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa79006ace0 0x7fa79006d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8007fd0 tx=0x7fa7a8007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:24.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.392+0000 7fa7a1ffb700 1 -- 172.21.15.60:0/4143089304 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa79c056200 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.643+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa7ac0b66a0 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.644+0000 7fa7a1ffb700 1 -- 172.21.15.60:0/4143089304 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa79c015070 con 0x7fa7ac0cfad0 2023-07-13T22:23:24.627 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:23:24.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa79006ace0 msgr2=0x7fa79006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:24.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa79006ace0 0x7fa79006d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8007fd0 tx=0x7fa7a8007d00 comp rx=0 tx=0).stop 2023-07-13T22:23:24.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0cfad0 msgr2=0x7fa7ac1f15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:24.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0cfad0 0x7fa7ac1f15d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa79c016000 tx=0x7fa79c004750 comp rx=0 tx=0).stop 2023-07-13T22:23:24.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 shutdown_connections 2023-07-13T22:23:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa79006ace0 0x7fa79006d190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7ac0b8400 0x7fa7ac1f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7ac0d3750 0x7fa7ac1f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 --2- 172.21.15.60:0/4143089304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac0cfad0 0x7fa7ac1f15d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 >> 172.21.15.60:0/4143089304 conn(0x7fa7ac06b390 msgr2=0x7fa7ac15ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 shutdown_connections 2023-07-13T22:23:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:24.646+0000 7fa7b37ff700 1 -- 172.21.15.60:0/4143089304 wait complete. 2023-07-13T22:23:24.647 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:23:24.648 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:23:24.648 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:23:24.648 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:23:24.648 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:23:24.649 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:23:24.649 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:23:24.649 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:23:24.649 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:23:24.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:24.650 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:23:24.702 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:23:24.703 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:23:24.703 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:23:24.703 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:24.703 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:23:24.703 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:24.703 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:23:24.703 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:24.703 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:23:24.704 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:23:24.704 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:24.704 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:23:24.704 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:23:24.704 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:24.704 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:23:24.704 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:24.705 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:23:24.705 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41566sec preferred_lft 41566sec 2023-07-13T22:23:24.705 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:23:24.705 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:24.705 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:23:24.705 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:24.705 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:23:24.705 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:24.706 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:23:24.706 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:24.706 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:23:24.706 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:23:24.706 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:23:24.706 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:24.706 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:23:24.706 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:23:24.707 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::14e3:bfff:fea8:bf57/64 scope link 2023-07-13T22:23:24.707 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:24.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:24.708 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:23:24.779 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:23:24.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:24.782 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:23:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:23:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:23:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:23:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:23:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:23:24.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:24.833 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:23:24.834 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:23:24.834 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:23:24.834 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:23:24.834 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:23:24.835 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:23:25.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:25.003 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:23:25.049 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:23:25.049 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:23:25.050 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:23:25.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:25.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:23:25.159 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:23:25.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:23:25.160 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:23:25.205 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:23:25.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:23:25.206 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:23:25.291 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:23:25.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:23:25.382 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:23:25.401 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:23:25.402 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:23:25.509 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:23:25.512 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:23:25.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:25.513 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:23:25.598 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:23:25.598 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:23:25.603 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:23:25.674 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:23:25.675 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:23:25.749 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:23:25.751 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:23:25.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:25.752 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:23:25.773 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:23:25.790+0000 7fc36afec5c0 -1 init, newargv = 0x55910fff7110 newargc=15 2023-07-13T22:23:25.774 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[121721]: starting ceph client 2023-07-13T22:23:25.781 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[121721]: starting fuse 2023-07-13T22:23:25.801 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:23:25.801 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:23:25.802 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:25.802 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:23:25.802 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:23:25.803 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:23:25.803 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:23:25.803 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:23:25.804 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:23:25.804 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:23:25.804 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:23:25.805 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:23:25.805 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:23:25.805 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:23:25.805 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:23:25.806 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:23:25.806 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:23:25.806 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:23:25.807 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:23:25.807 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:25.807 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:25.807 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:25.808 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:23:25.808 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:23:25.808 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:23:25.809 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:25.809 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:23:25.809 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:23:25.809 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:23:25.985 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.121721.asok 2023-07-13T22:23:26.163 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.121721.asok 2023-07-13T22:23:26.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:26.164 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.121721.asok status 2023-07-13T22:23:26.386 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:23:26.387 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:23:26.387 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:23:26.387 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:23:26.387 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:23:26.387 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:23:26.387 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:23:26.387 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "121721", 2023-07-13T22:23:26.388 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:23:26.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:23:26.388 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:23:26.388 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:23:26.388 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5414, 2023-07-13T22:23:26.388 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:23:26.388 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:23:26.389 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:23:26.389 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 5414 2023-07-13T22:23:26.389 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:23:26.389 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:23:26.389 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:23:26.389 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:26.389 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 672427362 2023-07-13T22:23:26.389 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:23:26.390 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:23:26.390 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:23:26.390 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:23:26.390 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:26.390 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 672427362 2023-07-13T22:23:26.390 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:23:26.391 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.5414 192.168.0.1:0/672427362", 2023-07-13T22:23:26.391 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/672427362", 2023-07-13T22:23:26.391 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:23:26.391 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 24, 2023-07-13T22:23:26.391 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 49, 2023-07-13T22:23:26.391 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:23:26.392 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:23:26.392 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:23:26.392 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:23:26.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:26.401 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:23:26.401 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:23:26.448 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:23:26.448 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:23:26.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:26.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:23:26.558 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:26.558 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:26.559 DEBUG:teuthology.orchestra.run.smithi060:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2023-07-13T22:23:26.559 DEBUG:teuthology.orchestra.run.smithi060:> for n in range(0, 1000): 2023-07-13T22:23:26.560 DEBUG:teuthology.orchestra.run.smithi060:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir/dir{0}".format(n)) 2023-07-13T22:23:26.561 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:28.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:28.673 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync) 2023-07-13T22:23:29.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:29.105 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:29.106 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:29.106 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:29.106 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:29.107 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:29.107 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:29.107 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:29.107 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:29.108 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:29.108 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:29.108 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:29.109 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:29.109 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:29.109 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:29.110 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:29.110 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:29.110 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:29.110 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:29.111 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:29.111 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:29.111 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:29.112 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:29.112 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:29.112 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:29.113 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:29.113 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:29.263 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:29.439 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:29.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:29.440 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.132428.asok status 2023-07-13T22:23:29.671 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:29.672 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:23:29.672 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:23:29.672 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:23:29.672 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:23:29.672 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:23:29.672 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:23:29.673 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "132428", 2023-07-13T22:23:29.673 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:23:29.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:29.673 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 1001, 2023-07-13T22:23:29.673 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 1001, 2023-07-13T22:23:29.673 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5961, 2023-07-13T22:23:29.673 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:23:29.673 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:23:29.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:23:29.674 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5961 2023-07-13T22:23:29.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:29.674 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:29.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:29.674 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:29.674 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2452753753 2023-07-13T22:23:29.675 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:29.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:29.675 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:29.675 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:29.675 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:29.675 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2452753753 2023-07-13T22:23:29.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:29.675 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5961 192.168.0.1:0/2452753753", 2023-07-13T22:23:29.676 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2452753753", 2023-07-13T22:23:29.676 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1002, 2023-07-13T22:23:29.676 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 24, 2023-07-13T22:23:29.676 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 49, 2023-07-13T22:23:29.676 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 49, 2023-07-13T22:23:29.676 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:23:29.676 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:23:29.676 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:29.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:29.686 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:29.686 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:29.686 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:29.687 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:29.687 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:29.687 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:29.687 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:29.688 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:29.688 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:29.688 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:29.689 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:29.689 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:29.689 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:29.689 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:29.690 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:29.690 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:29.690 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:29.691 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:29.691 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:29.691 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:29.691 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:29.692 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:29.692 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:29.692 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:29.693 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:29.693 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:29.839 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:30.016 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:30.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:30.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.132428.asok config set client_cache_size 100 2023-07-13T22:23:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "client_cache_size = '100' " 2023-07-13T22:23:30.247 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:30.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:30.262 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:30.262 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:30.263 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:30.263 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:30.263 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:30.263 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:30.264 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:30.264 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:30.264 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:30.265 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:30.265 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:30.265 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:30.266 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:30.266 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:30.266 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:30.267 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:30.267 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:30.267 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:30.267 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:30.268 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:30.268 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:30.268 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:30.269 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:30.269 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:30.269 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:30.269 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:30.416 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:30.593 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:30.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:30.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.132428.asok status 2023-07-13T22:23:30.825 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:30.825 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:23:30.825 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:23:30.826 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:23:30.826 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:23:30.826 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:23:30.826 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:23:30.826 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "132428", 2023-07-13T22:23:30.826 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:23:30.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:30.826 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 1001, 2023-07-13T22:23:30.827 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 165, 2023-07-13T22:23:30.827 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5961, 2023-07-13T22:23:30.827 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:23:30.827 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:23:30.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:23:30.827 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5961 2023-07-13T22:23:30.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:30.828 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:30.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:30.828 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:30.828 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2452753753 2023-07-13T22:23:30.828 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:30.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:30.828 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:30.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:30.829 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:30.829 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2452753753 2023-07-13T22:23:30.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:30.829 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5961 192.168.0.1:0/2452753753", 2023-07-13T22:23:30.829 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2452753753", 2023-07-13T22:23:30.829 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1002, 2023-07-13T22:23:30.829 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 24, 2023-07-13T22:23:30.829 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 49, 2023-07-13T22:23:30.830 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 49, 2023-07-13T22:23:30.830 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:23:30.830 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:23:30.830 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:30.839 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 1001, 165 2023-07-13T22:23:30.839 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-07-13T22:23:35.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:35.841 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:35.841 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:35.842 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:35.842 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:35.842 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:35.843 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:35.843 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:35.843 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:35.843 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:35.844 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:35.844 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:35.844 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:35.845 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:35.845 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:35.845 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:35.846 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:35.846 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:35.846 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:35.847 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:35.847 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:35.847 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:35.848 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:35.848 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:35.848 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:35.848 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:35.849 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:36.006 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:36.182 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.132428.asok 2023-07-13T22:23:36.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:36.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.132428.asok status 2023-07-13T22:23:36.416 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:36.416 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:23:36.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:23:36.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:23:36.416 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:23:36.417 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:23:36.417 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:23:36.417 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "132428", 2023-07-13T22:23:36.417 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:23:36.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:36.417 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 100, 2023-07-13T22:23:36.417 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 1, 2023-07-13T22:23:36.417 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5961, 2023-07-13T22:23:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:23:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:23:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:23:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5961 2023-07-13T22:23:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2452753753 2023-07-13T22:23:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2452753753 2023-07-13T22:23:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5961 192.168.0.1:0/2452753753", 2023-07-13T22:23:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2452753753", 2023-07-13T22:23:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 101, 2023-07-13T22:23:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 24, 2023-07-13T22:23:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 49, 2023-07-13T22:23:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 49, 2023-07-13T22:23:36.421 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:23:36.421 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:23:36.421 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:36.430 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 100, 1 2023-07-13T22:23:36.430 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-07-13T22:23:36.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:36.431 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-07-13T22:23:36.483 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:23:36.483 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:23:36.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:23:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.789+0000 7f40be3ff700 1 Processor -- start 2023-07-13T22:23:36.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.789+0000 7f40be3ff700 1 -- start start 2023-07-13T22:23:36.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.790+0000 7f40be3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40b815c6e0 0x7f40b806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:36.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.790+0000 7f40be3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b80b3050 0x7f40b806afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:36.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.790+0000 7f40be3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b806b8e0 con 0x7f40b80b24b0 2023-07-13T22:23:36.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.790+0000 7f40be3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b815cab0 con 0x7f40b815c6e0 2023-07-13T22:23:36.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.790+0000 7f40be3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b806bbc0 con 0x7f40b80b3050 2023-07-13T22:23:36.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40b7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40b815c6e0 0x7f40b806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40bc99c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f40b80b24b0 0x7f40b80cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55220/0 (socket says 172.21.15.60:55220) 2023-07-13T22:23:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40b7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40b815c6e0 0x7f40b806c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55832/0 (socket says 172.21.15.60:55832) 2023-07-13T22:23:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40bc99c700 1 -- 172.21.15.60:0/3437817973 learned_addr learned my addr 172.21.15.60:0/3437817973 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40b77fe700 1 --2- 172.21.15.60:0/3437817973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b80b3050 0x7f40b806afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40b7fff700 1 -- 172.21.15.60:0/3437817973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b80b3050 msgr2=0x7f40b806afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40b7fff700 1 --2- 172.21.15.60:0/3437817973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b80b3050 0x7f40b806afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40b7fff700 1 -- 172.21.15.60:0/3437817973 >> v1:172.21.15.60:6789/0 conn(0x7f40b80b24b0 legacy=0x7f40b80cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40b7fff700 1 -- 172.21.15.60:0/3437817973 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b806bea0 con 0x7f40b815c6e0 2023-07-13T22:23:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40b6ffd700 1 -- 172.21.15.60:0/3437817973 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566406007 0 0) 0x7f40b806b8e0 con 0x7f40b80b24b0 2023-07-13T22:23:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.791+0000 7f40b7fff700 1 --2- 172.21.15.60:0/3437817973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b815c6e0 0x7f40b806c980 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009840 tx=0x7f40a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fac33e60657bf56 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40b6ffd700 1 -- 172.21.15.60:0/3437817973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8011830 con 0x7f40b815c6e0 2023-07-13T22:23:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40b6ffd700 1 -- 172.21.15.60:0/3437817973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a8011990 con 0x7f40b815c6e0 2023-07-13T22:23:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40b6ffd700 1 -- 172.21.15.60:0/3437817973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8011d20 con 0x7f40b815c6e0 2023-07-13T22:23:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40be3ff700 1 -- 172.21.15.60:0/3437817973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b815c6e0 msgr2=0x7f40b806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40be3ff700 1 --2- 172.21.15.60:0/3437817973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b815c6e0 0x7f40b806c980 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009840 tx=0x7f40a8009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:36.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40be3ff700 1 -- 172.21.15.60:0/3437817973 shutdown_connections 2023-07-13T22:23:36.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40be3ff700 1 --2- 172.21.15.60:0/3437817973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b815c6e0 0x7f40b806c980 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:36.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40be3ff700 1 --2- 172.21.15.60:0/3437817973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b80b3050 0x7f40b806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:36.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40be3ff700 1 -- 172.21.15.60:0/3437817973 >> 172.21.15.60:0/3437817973 conn(0x7f40b806e680 msgr2=0x7f40b806ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:36.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40be3ff700 1 -- 172.21.15.60:0/3437817973 shutdown_connections 2023-07-13T22:23:36.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.792+0000 7f40be3ff700 1 -- 172.21.15.60:0/3437817973 wait complete. 2023-07-13T22:23:36.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.795+0000 7f40be3ff700 1 Processor -- start 2023-07-13T22:23:36.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.795+0000 7f40be3ff700 1 -- start start 2023-07-13T22:23:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.796+0000 7f40be3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b80b24b0 0x7f40b8154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.796+0000 7f40be3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40b80b3050 0x7f40b80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.796+0000 7f40be3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b815c6e0 0x7f40b815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.796+0000 7f40be3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b815ed10 con 0x7f40b80b24b0 2023-07-13T22:23:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.796+0000 7f40be3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b815e100 con 0x7f40b80b3050 2023-07-13T22:23:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.796+0000 7f40be3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b815e3e0 con 0x7f40b815c6e0 2023-07-13T22:23:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.796+0000 7f40b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b80b24b0 0x7f40b8154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.796+0000 7f40b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b80b24b0 0x7f40b8154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55844/0 (socket says 172.21.15.60:55844) 2023-07-13T22:23:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.796+0000 7f40b7fff700 1 -- 172.21.15.60:0/2344404597 learned_addr learned my addr 172.21.15.60:0/2344404597 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b77fe700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40b80b3050 0x7f40b80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:36.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40bc99c700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b815c6e0 0x7f40b815f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:36.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b7fff700 1 -- 172.21.15.60:0/2344404597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b815c6e0 msgr2=0x7f40b815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:36.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b7fff700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b815c6e0 0x7f40b815f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:36.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b7fff700 1 -- 172.21.15.60:0/2344404597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40b80b3050 msgr2=0x7f40b80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:36.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b7fff700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40b80b3050 0x7f40b80c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:36.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b7fff700 1 -- 172.21.15.60:0/2344404597 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a80094a0 con 0x7f40b80b24b0 2023-07-13T22:23:36.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b77fe700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40b80b3050 0x7f40b80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b7fff700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b80b24b0 0x7f40b8154850 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f40a8014040 tx=0x7f40a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b4ff9700 1 -- 172.21.15.60:0/2344404597 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a80038a0 con 0x7f40b80b24b0 2023-07-13T22:23:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b4ff9700 1 -- 172.21.15.60:0/2344404597 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a801e070 con 0x7f40b80b24b0 2023-07-13T22:23:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b815e6c0 con 0x7f40b80b24b0 2023-07-13T22:23:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.797+0000 7f40b4ff9700 1 -- 172.21.15.60:0/2344404597 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003c20 con 0x7f40b80b24b0 2023-07-13T22:23:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.798+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b8154d90 con 0x7f40b80b24b0 2023-07-13T22:23:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.799+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b804e280 con 0x7f40b80b24b0 2023-07-13T22:23:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.800+0000 7f40b4ff9700 1 -- 172.21.15.60:0/2344404597 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f40a802d450 con 0x7f40b80b24b0 2023-07-13T22:23:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.801+0000 7f40b4ff9700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f40a406ace0 0x7f40a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.801+0000 7f40b4ff9700 1 -- 172.21.15.60:0/2344404597 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40a808a5b0 con 0x7f40b80b24b0 2023-07-13T22:23:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.801+0000 7f40b77fe700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f40a406ace0 0x7f40a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.802+0000 7f40b77fe700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f40a406ace0 0x7f40a406d190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f40b815bc20 tx=0x7f40ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:36.805+0000 7f40b4ff9700 1 -- 172.21.15.60:0/2344404597 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f40a8055650 con 0x7f40b80b24b0 2023-07-13T22:23:36.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.014+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40b8070a00 con 0x7f40b80b24b0 2023-07-13T22:23:36.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.015+0000 7f40b4ff9700 1 -- 172.21.15.60:0/2344404597 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18257 (secure 0 0 0) 0x7f40a802c3b0 con 0x7f40b80b24b0 2023-07-13T22:23:36.999 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:36.999 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:16.111947+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:23:37.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f40a406ace0 msgr2=0x7f40a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f40a406ace0 0x7f40a406d190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f40b815bc20 tx=0x7f40ac008040 comp rx=0 tx=0).stop 2023-07-13T22:23:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b80b24b0 msgr2=0x7f40b8154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b80b24b0 0x7f40b8154850 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f40a8014040 tx=0x7f40a8004750 comp rx=0 tx=0).stop 2023-07-13T22:23:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 shutdown_connections 2023-07-13T22:23:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f40a406ace0 0x7f40a406d190 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40b80b24b0 0x7f40b8154850 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40b815c6e0 0x7f40b815f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 --2- 172.21.15.60:0/2344404597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40b80b3050 0x7f40b80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 >> 172.21.15.60:0/2344404597 conn(0x7f40b806e680 msgr2=0x7f40b80cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:37.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 shutdown_connections 2023-07-13T22:23:37.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.018+0000 7f40be3ff700 1 -- 172.21.15.60:0/2344404597 wait complete. 2023-07-13T22:23:37.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:23:12.304337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:23:13.598583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:23:37.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:23:37.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.244+0000 7fe813dff700 1 Processor -- start 2023-07-13T22:23:37.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.244+0000 7fe813dff700 1 -- start start 2023-07-13T22:23:37.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.245+0000 7fe813dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 0x7fe80c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe813dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe80c0cede0 0x7fe80c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe813dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80c0d1870 con 0x7fe80c0b1ac0 2023-07-13T22:23:37.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe813dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80c0cf370 con 0x7fe80c0cede0 2023-07-13T22:23:37.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe813dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c0d12b0 con 0x7fe80c0b8200 2023-07-13T22:23:37.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe81239c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe80c0cede0 0x7fe80c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe81239c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe80c0cede0 0x7fe80c0cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55852/0 (socket says 172.21.15.60:55852) 2023-07-13T22:23:37.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe811b9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 0x7fe80c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe81239c700 1 -- 172.21.15.60:0/1247304275 learned_addr learned my addr 172.21.15.60:0/1247304275 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:37.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe81139a700 1 --1- 172.21.15.60:0/1247304275 >> v1:172.21.15.105:6789/0 conn(0x7fe80c0b1ac0 0x7fe80c15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:52580/0 (socket says 172.21.15.60:52580) 2023-07-13T22:23:37.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe81239c700 1 -- 172.21.15.60:0/1247304275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 msgr2=0x7fe80c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:37.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe81239c700 1 --2- 172.21.15.60:0/1247304275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 0x7fe80c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe81239c700 1 -- 172.21.15.60:0/1247304275 >> v1:172.21.15.105:6789/0 conn(0x7fe80c0b1ac0 legacy=0x7fe80c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe81239c700 1 -- 172.21.15.60:0/1247304275 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c0d1590 con 0x7fe80c0cede0 2023-07-13T22:23:37.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.246+0000 7fe810b99700 1 -- 172.21.15.60:0/1247304275 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583972101 0 0) 0x7fe80c0d1870 con 0x7fe80c0b1ac0 2023-07-13T22:23:37.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.247+0000 7fe81239c700 1 --2- 172.21.15.60:0/1247304275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 0x7fe80c0cfb80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe804009e50 tx=0x7fe80400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ba2ee18bd2b28a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:37.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.247+0000 7fe810b99700 1 -- 172.21.15.60:0/1247304275 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80400e070 con 0x7fe80c0cede0 2023-07-13T22:23:37.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.247+0000 7fe810b99700 1 -- 172.21.15.60:0/1247304275 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80400b910 con 0x7fe80c0cede0 2023-07-13T22:23:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.247+0000 7fe810b99700 1 -- 172.21.15.60:0/1247304275 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80400bcc0 con 0x7fe80c0cede0 2023-07-13T22:23:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.247+0000 7fe813dff700 1 -- 172.21.15.60:0/1247304275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 msgr2=0x7fe80c0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.247+0000 7fe813dff700 1 --2- 172.21.15.60:0/1247304275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 0x7fe80c0cfb80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe804009e50 tx=0x7fe80400b330 comp rx=0 tx=0).stop 2023-07-13T22:23:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.248+0000 7fe813dff700 1 -- 172.21.15.60:0/1247304275 shutdown_connections 2023-07-13T22:23:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.248+0000 7fe813dff700 1 --2- 172.21.15.60:0/1247304275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 0x7fe80c0cfb80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.248+0000 7fe813dff700 1 --2- 172.21.15.60:0/1247304275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 0x7fe80c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.248+0000 7fe813dff700 1 -- 172.21.15.60:0/1247304275 >> 172.21.15.60:0/1247304275 conn(0x7fe80c06b2f0 msgr2=0x7fe80c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.248+0000 7fe813dff700 1 -- 172.21.15.60:0/1247304275 shutdown_connections 2023-07-13T22:23:37.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.248+0000 7fe813dff700 1 -- 172.21.15.60:0/1247304275 wait complete. 2023-07-13T22:23:37.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.249+0000 7fe813dff700 1 Processor -- start 2023-07-13T22:23:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.250+0000 7fe813dff700 1 -- start start 2023-07-13T22:23:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.250+0000 7fe813dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe80c0b1ac0 0x7fe80c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.250+0000 7fe813dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 0x7fe80c1e6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.250+0000 7fe813dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 0x7fe80c0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.250+0000 7fe813dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c0c7740 con 0x7fe80c0cede0 2023-07-13T22:23:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe813dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c0cbc00 con 0x7fe80c0b1ac0 2023-07-13T22:23:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe813dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c0c7180 con 0x7fe80c0b8200 2023-07-13T22:23:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81239c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 0x7fe80c0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81239c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 0x7fe80c0c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55858/0 (socket says 172.21.15.60:55858) 2023-07-13T22:23:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81239c700 1 -- 172.21.15.60:0/638618858 learned_addr learned my addr 172.21.15.60:0/638618858 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe811b9b700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe80c0b1ac0 0x7fe80c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81139a700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 0x7fe80c1e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81239c700 1 -- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 msgr2=0x7fe80c1e6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81239c700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 0x7fe80c1e6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81239c700 1 -- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe80c0b1ac0 msgr2=0x7fe80c0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81239c700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe80c0b1ac0 0x7fe80c0cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81239c700 1 -- 172.21.15.60:0/638618858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe804009b30 con 0x7fe80c0cede0 2023-07-13T22:23:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe811b9b700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe80c0b1ac0 0x7fe80c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81139a700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 0x7fe80c1e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.251+0000 7fe81239c700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 0x7fe80c0c8c60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe804004270 tx=0x7fe804004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.252+0000 7fe8027fc700 1 -- 172.21.15.60:0/638618858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8040092e0 con 0x7fe80c0cede0 2023-07-13T22:23:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.252+0000 7fe8027fc700 1 -- 172.21.15.60:0/638618858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80401b430 con 0x7fe80c0cede0 2023-07-13T22:23:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.252+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80c0c7460 con 0x7fe80c0cede0 2023-07-13T22:23:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.252+0000 7fe8027fc700 1 -- 172.21.15.60:0/638618858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80401b800 con 0x7fe80c0cede0 2023-07-13T22:23:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.252+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80c0cc5d0 con 0x7fe80c0cede0 2023-07-13T22:23:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.253+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80c04e210 con 0x7fe80c0cede0 2023-07-13T22:23:37.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.254+0000 7fe8027fc700 1 -- 172.21.15.60:0/638618858 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe80401b960 con 0x7fe80c0cede0 2023-07-13T22:23:37.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.255+0000 7fe8027fc700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe7f006ad10 0x7fe7f006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.255+0000 7fe8027fc700 1 -- 172.21.15.60:0/638618858 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe80408ab80 con 0x7fe80c0cede0 2023-07-13T22:23:37.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.255+0000 7fe811b9b700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe7f006ad10 0x7fe7f006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.256+0000 7fe811b9b700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe7f006ad10 0x7fe7f006d1c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe808009b60 tx=0x7fe808008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.259+0000 7fe8027fc700 1 -- 172.21.15.60:0/638618858 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe804055c20 con 0x7fe80c0cede0 2023-07-13T22:23:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.463+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe80c0cbd40 con 0x7fe80c0cede0 2023-07-13T22:23:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.464+0000 7fe8027fc700 1 -- 172.21.15.60:0/638618858 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe80401f020 con 0x7fe80c0cede0 2023-07-13T22:23:37.448 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:23:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe7f006ad10 msgr2=0x7fe7f006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe7f006ad10 0x7fe7f006d1c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe808009b60 tx=0x7fe808008040 comp rx=0 tx=0).stop 2023-07-13T22:23:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 msgr2=0x7fe80c0c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 0x7fe80c0c8c60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe804004270 tx=0x7fe804004ad0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 shutdown_connections 2023-07-13T22:23:37.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe7f006ad10 0x7fe7f006d1c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe80c0cede0 0x7fe80c0c8c60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe80c0b1ac0 0x7fe80c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 --2- 172.21.15.60:0/638618858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe80c0b8200 0x7fe80c1e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.466+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 >> 172.21.15.60:0/638618858 conn(0x7fe80c06b2f0 msgr2=0x7fe80c0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:37.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.467+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 shutdown_connections 2023-07-13T22:23:37.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.467+0000 7fe813dff700 1 -- 172.21.15.60:0/638618858 wait complete. 2023-07-13T22:23:37.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:23:37.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.690+0000 7f2d141ff700 1 Processor -- start 2023-07-13T22:23:37.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.691+0000 7f2d141ff700 1 -- start start 2023-07-13T22:23:37.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.692+0000 7f2d141ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d0c15c790 0x7f2d0c0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.692+0000 7f2d141ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d0c0ad9a0 0x7f2d0c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.692+0000 7f2d141ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c0cf6d0 con 0x7f2d0c0ac490 2023-07-13T22:23:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.692+0000 7f2d141ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c0ae2e0 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.692+0000 7f2d141ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c0ac900 con 0x7f2d0c15c790 2023-07-13T22:23:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.692+0000 7f2d1279c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d0c0ad9a0 0x7f2d0c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.692+0000 7f2d1279c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d0c0ad9a0 0x7f2d0c0adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55862/0 (socket says 172.21.15.60:55862) 2023-07-13T22:23:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.692+0000 7f2d1279c700 1 -- 172.21.15.60:0/3127831853 learned_addr learned my addr 172.21.15.60:0/3127831853 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:37.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.692+0000 7f2d11f9b700 1 --2- 172.21.15.60:0/3127831853 >> v2:172.21.15.105:3300/0 conn(0x7f2d0c15c790 0x7f2d0c0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.693+0000 7f2d1279c700 1 -- 172.21.15.60:0/3127831853 >> v2:172.21.15.105:3300/0 conn(0x7f2d0c15c790 msgr2=0x7f2d0c0ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.693+0000 7f2d1279c700 1 --2- 172.21.15.60:0/3127831853 >> v2:172.21.15.105:3300/0 conn(0x7f2d0c15c790 0x7f2d0c0ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.693+0000 7f2d1279c700 1 -- 172.21.15.60:0/3127831853 >> v1:172.21.15.105:6789/0 conn(0x7f2d0c0ac490 legacy=0x7f2d0c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.693+0000 7f2d1279c700 1 -- 172.21.15.60:0/3127831853 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0acbe0 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.693+0000 7f2d1279c700 1 --2- 172.21.15.60:0/3127831853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 0x7f2d0c0adda0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0726e0 tx=0x7f2d0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540c99ee3e6ae741 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.693+0000 7f2d10f99700 1 -- 172.21.15.60:0/3127831853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400e070 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.693+0000 7f2d10f99700 1 -- 172.21.15.60:0/3127831853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04003a20 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.693+0000 7f2d10f99700 1 -- 172.21.15.60:0/3127831853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04003dd0 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.694+0000 7f2d141ff700 1 -- 172.21.15.60:0/3127831853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 msgr2=0x7f2d0c0adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.694+0000 7f2d141ff700 1 --2- 172.21.15.60:0/3127831853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 0x7f2d0c0adda0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0726e0 tx=0x7f2d0400b330 comp rx=0 tx=0).stop 2023-07-13T22:23:37.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.694+0000 7f2d141ff700 1 -- 172.21.15.60:0/3127831853 shutdown_connections 2023-07-13T22:23:37.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.694+0000 7f2d141ff700 1 --2- 172.21.15.60:0/3127831853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 0x7f2d0c0adda0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.694+0000 7f2d141ff700 1 --2- 172.21.15.60:0/3127831853 >> v2:172.21.15.105:3300/0 conn(0x7f2d0c15c790 0x7f2d0c0ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.694+0000 7f2d141ff700 1 -- 172.21.15.60:0/3127831853 >> 172.21.15.60:0/3127831853 conn(0x7f2d0c06b900 msgr2=0x7f2d0c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:37.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.694+0000 7f2d141ff700 1 -- 172.21.15.60:0/3127831853 shutdown_connections 2023-07-13T22:23:37.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.694+0000 7f2d141ff700 1 -- 172.21.15.60:0/3127831853 wait complete. 2023-07-13T22:23:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.695+0000 7f2d141ff700 1 Processor -- start 2023-07-13T22:23:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.696+0000 7f2d141ff700 1 -- start start 2023-07-13T22:23:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.696+0000 7f2d141ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0ac490 0x7f2d0c1f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.696+0000 7f2d141ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 0x7f2d0c0c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.696+0000 7f2d141ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15c790 0x7f2d0c0c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.696+0000 7f2d141ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1f2950 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.696+0000 7f2d141ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0b38c0 con 0x7f2d0c0ac490 2023-07-13T22:23:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.696+0000 7f2d141ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0b3ba0 con 0x7f2d0c15c790 2023-07-13T22:23:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d1179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 0x7f2d0c0c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d1179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 0x7f2d0c0c5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55872/0 (socket says 172.21.15.60:55872) 2023-07-13T22:23:37.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d1179a700 1 -- 172.21.15.60:0/3750138379 learned_addr learned my addr 172.21.15.60:0/3750138379 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d11f9b700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0ac490 0x7f2d0c1f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d1279c700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15c790 0x7f2d0c0c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d1179a700 1 -- 172.21.15.60:0/3750138379 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15c790 msgr2=0x7f2d0c0c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d1179a700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15c790 0x7f2d0c0c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d1179a700 1 -- 172.21.15.60:0/3750138379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0ac490 msgr2=0x7f2d0c1f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d1179a700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0ac490 0x7f2d0c1f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.697+0000 7f2d1179a700 1 -- 172.21.15.60:0/3750138379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d04009b30 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.698+0000 7f2d1179a700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 0x7f2d0c0c5c60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c4d0 tx=0x7f2cfc00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:37.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.698+0000 7f2d027fc700 1 -- 172.21.15.60:0/3750138379 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc011790 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.698+0000 7f2d027fc700 1 -- 172.21.15.60:0/3750138379 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc0118f0 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.698+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c0b3e80 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.698+0000 7f2d027fc700 1 -- 172.21.15.60:0/3750138379 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc011ca0 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.698+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c0b4440 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.699+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c04e210 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.700+0000 7f2d027fc700 1 -- 172.21.15.60:0/3750138379 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2cfc01c460 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.701+0000 7f2d027fc700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cf006ace0 0x7f2cf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:37.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.701+0000 7f2d027fc700 1 -- 172.21.15.60:0/3750138379 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2cfc089290 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.701+0000 7f2d11f9b700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cf006ace0 0x7f2cf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:37.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.702+0000 7f2d11f9b700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cf006ace0 0x7f2cf006d190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d08006fd0 tx=0x7f2d08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:37.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.705+0000 7f2d027fc700 1 -- 172.21.15.60:0/3750138379 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2cfc054210 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.911+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d0c0c30f0 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.911+0000 7f2d027fc700 1 -- 172.21.15.60:0/3750138379 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cfc057830 con 0x7f2d0c0ad9a0 2023-07-13T22:23:37.895 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:23:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.913+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cf006ace0 msgr2=0x7f2cf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.913+0000 7f2d141ff700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cf006ace0 0x7f2cf006d190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d08006fd0 tx=0x7f2d08008040 comp rx=0 tx=0).stop 2023-07-13T22:23:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.913+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 msgr2=0x7f2d0c0c5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.913+0000 7f2d141ff700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 0x7f2d0c0c5c60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c4d0 tx=0x7f2cfc00c7e0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.913+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 shutdown_connections 2023-07-13T22:23:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.913+0000 7f2d141ff700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cf006ace0 0x7f2cf006d190 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.913+0000 7f2d141ff700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15c790 0x7f2d0c0c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.913+0000 7f2d141ff700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0ac490 0x7f2d0c1f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.914+0000 7f2d141ff700 1 --2- 172.21.15.60:0/3750138379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d0c0ad9a0 0x7f2d0c0c5c60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.914+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 >> 172.21.15.60:0/3750138379 conn(0x7f2d0c06b900 msgr2=0x7f2d0c06d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.914+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 shutdown_connections 2023-07-13T22:23:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:37.914+0000 7f2d141ff700 1 -- 172.21.15.60:0/3750138379 wait complete. 2023-07-13T22:23:37.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:23:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.138+0000 7f436a5ff700 1 Processor -- start 2023-07-13T22:23:38.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.138+0000 7f436a5ff700 1 -- start start 2023-07-13T22:23:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.139+0000 7f436a5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4364159260 0x7f43640ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.139+0000 7f436a5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640cf640 con 0x7f4364070af0 2023-07-13T22:23:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.140+0000 7f436a5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640ccf80 con 0x7f43640cc410 2023-07-13T22:23:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.140+0000 7f436a5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43640c9b40 con 0x7f4364159260 2023-07-13T22:23:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.140+0000 7f4368b9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4364070af0 0x7f436406c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55228/0 (socket says 172.21.15.60:55228) 2023-07-13T22:23:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.140+0000 7f4368b9c700 1 -- 172.21.15.60:0/3603005718 learned_addr learned my addr 172.21.15.60:0/3603005718 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.140+0000 7f4363fff700 1 --2- 172.21.15.60:0/3603005718 >> v2:172.21.15.105:3300/0 conn(0x7f4364159260 0x7f43640ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.140+0000 7f43637fe700 1 -- 172.21.15.60:0/3603005718 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518788247 0 0) 0x7f43640cf640 con 0x7f4364070af0 2023-07-13T22:23:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.140+0000 7f43637fe700 1 -- 172.21.15.60:0/3603005718 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43500057c0 con 0x7f4364070af0 2023-07-13T22:23:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.141+0000 7f43637fe700 1 -- 172.21.15.60:0/3603005718 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2626940070 0 0) 0x7f43500057c0 con 0x7f4364070af0 2023-07-13T22:23:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.141+0000 7f4363fff700 1 -- 172.21.15.60:0/3603005718 >> v1:172.21.15.105:6789/0 conn(0x7f43640cc410 legacy=0x7f43640703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.141+0000 7f4363fff700 1 -- 172.21.15.60:0/3603005718 >> v1:172.21.15.60:6789/0 conn(0x7f4364070af0 legacy=0x7f436406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.141+0000 7f4363fff700 1 -- 172.21.15.60:0/3603005718 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43640c9e20 con 0x7f4364159260 2023-07-13T22:23:38.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.141+0000 7f4363fff700 1 --2- 172.21.15.60:0/3603005718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364159260 0x7f43640ca900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4354009840 tx=0x7f4354009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e46bb10822bc00c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.141+0000 7f43637fe700 1 -- 172.21.15.60:0/3603005718 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c0019f0 con 0x7f4364159260 2023-07-13T22:23:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.141+0000 7f43637fe700 1 -- 172.21.15.60:0/3603005718 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4354004030 con 0x7f4364159260 2023-07-13T22:23:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.141+0000 7f43637fe700 1 -- 172.21.15.60:0/3603005718 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540043c0 con 0x7f4364159260 2023-07-13T22:23:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.142+0000 7f436a5ff700 1 -- 172.21.15.60:0/3603005718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364159260 msgr2=0x7f43640ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.142+0000 7f436a5ff700 1 --2- 172.21.15.60:0/3603005718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364159260 0x7f43640ca900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4354009840 tx=0x7f4354009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.142+0000 7f436a5ff700 1 -- 172.21.15.60:0/3603005718 shutdown_connections 2023-07-13T22:23:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.142+0000 7f436a5ff700 1 --2- 172.21.15.60:0/3603005718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364159260 0x7f43640ca900 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.142+0000 7f436a5ff700 1 -- 172.21.15.60:0/3603005718 >> 172.21.15.60:0/3603005718 conn(0x7f43640cbf50 msgr2=0x7f43640ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.142+0000 7f436a5ff700 1 -- 172.21.15.60:0/3603005718 shutdown_connections 2023-07-13T22:23:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.142+0000 7f436a5ff700 1 -- 172.21.15.60:0/3603005718 wait complete. 2023-07-13T22:23:38.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.144+0000 7f436a5ff700 1 Processor -- start 2023-07-13T22:23:38.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.144+0000 7f436a5ff700 1 -- start start 2023-07-13T22:23:38.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f436a5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364070af0 0x7f4364159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f436a5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43640cc410 0x7f436414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f436a5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4364159260 0x7f43640c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f436a5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43640b30e0 con 0x7f4364159260 2023-07-13T22:23:38.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f436a5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4364159b70 con 0x7f4364070af0 2023-07-13T22:23:38.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f436a5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436414f230 con 0x7f43640cc410 2023-07-13T22:23:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f4368b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4364159260 0x7f43640c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f4368b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4364159260 0x7f43640c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55874/0 (socket says 172.21.15.60:55874) 2023-07-13T22:23:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f4368b9c700 1 -- 172.21.15.60:0/2476308630 learned_addr learned my addr 172.21.15.60:0/2476308630 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f4363fff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364070af0 0x7f4364159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.145+0000 7f435bfff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43640cc410 0x7f436414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f4368b9c700 1 -- 172.21.15.60:0/2476308630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43640cc410 msgr2=0x7f436414e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f4368b9c700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43640cc410 0x7f436414e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f4368b9c700 1 -- 172.21.15.60:0/2476308630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364070af0 msgr2=0x7f4364159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f4368b9c700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364070af0 0x7f4364159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f4368b9c700 1 -- 172.21.15.60:0/2476308630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540094a0 con 0x7f4364159260 2023-07-13T22:23:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f4368b9c700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4364159260 0x7f43640c7100 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f435c001ba0 tx=0x7f435c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f43617fa700 1 -- 172.21.15.60:0/2476308630 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c00ab10 con 0x7f4364159260 2023-07-13T22:23:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f43617fa700 1 -- 172.21.15.60:0/2476308630 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c00ac70 con 0x7f4364159260 2023-07-13T22:23:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f43617fa700 1 -- 172.21.15.60:0/2476308630 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c0070d0 con 0x7f4364159260 2023-07-13T22:23:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43640b2d90 con 0x7f4364159260 2023-07-13T22:23:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.146+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436415a060 con 0x7f4364159260 2023-07-13T22:23:38.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.147+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43640b3220 con 0x7f4364159260 2023-07-13T22:23:38.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.148+0000 7f43617fa700 1 -- 172.21.15.60:0/2476308630 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f435c007230 con 0x7f4364159260 2023-07-13T22:23:38.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.150+0000 7f43617fa700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f434406ace0 0x7f434406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.150+0000 7f43617fa700 1 -- 172.21.15.60:0/2476308630 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f435c088f90 con 0x7f4364159260 2023-07-13T22:23:38.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.150+0000 7f4363fff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f434406ace0 0x7f434406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.151+0000 7f4363fff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f434406ace0 0x7f434406d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f43540090b0 tx=0x7f4354008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.154+0000 7f43617fa700 1 -- 172.21.15.60:0/2476308630 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f43640b3220 con 0x7f4364159260 2023-07-13T22:23:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.360+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43640b3220 con 0x7f4364159260 2023-07-13T22:23:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.361+0000 7f43617fa700 1 -- 172.21.15.60:0/2476308630 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f435c018020 con 0x7f4364159260 2023-07-13T22:23:38.345 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:23:38.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f434406ace0 msgr2=0x7f434406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f434406ace0 0x7f434406d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f43540090b0 tx=0x7f4354008030 comp rx=0 tx=0).stop 2023-07-13T22:23:38.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4364159260 msgr2=0x7f43640c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4364159260 0x7f43640c7100 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f435c001ba0 tx=0x7f435c00cf50 comp rx=0 tx=0).stop 2023-07-13T22:23:38.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 shutdown_connections 2023-07-13T22:23:38.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f434406ace0 0x7f434406d190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4364159260 0x7f43640c7100 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364070af0 0x7f4364159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 --2- 172.21.15.60:0/2476308630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43640cc410 0x7f436414e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.363+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 >> 172.21.15.60:0/2476308630 conn(0x7f43640cbf50 msgr2=0x7f436406df80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.364+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 shutdown_connections 2023-07-13T22:23:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.364+0000 7f436a5ff700 1 -- 172.21.15.60:0/2476308630 wait complete. 2023-07-13T22:23:38.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:23:38.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.588+0000 7f945e3ff700 1 Processor -- start 2023-07-13T22:23:38.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.588+0000 7f945e3ff700 1 -- start start 2023-07-13T22:23:38.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.589+0000 7f945e3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f945815c460 0x7f9458070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.589+0000 7f945e3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f94580caf80 0x7f94580ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.589+0000 7f945e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f945806c8f0 0x7f945815fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.589+0000 7f945e3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945806b9f0 con 0x7f94580caf80 2023-07-13T22:23:38.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.589+0000 7f945e3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945806c750 con 0x7f945815c460 2023-07-13T22:23:38.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.589+0000 7f945e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945806bf40 con 0x7f945806c8f0 2023-07-13T22:23:38.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f94577fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f94580caf80 0x7f94580ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f94577fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f94580caf80 0x7f94580ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55880/0 (socket says 172.21.15.60:55880) 2023-07-13T22:23:38.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f94577fe700 1 -- 172.21.15.60:0/2533963012 learned_addr learned my addr 172.21.15.60:0/2533963012 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:38.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f9457fff700 1 --2- 172.21.15.60:0/2533963012 >> v2:172.21.15.105:3300/0 conn(0x7f945815c460 0x7f9458070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f945c99c700 1 --2- 172.21.15.60:0/2533963012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f945806c8f0 0x7f945815fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f94577fe700 1 -- 172.21.15.60:0/2533963012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f945806c8f0 msgr2=0x7f945815fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f94577fe700 1 --2- 172.21.15.60:0/2533963012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f945806c8f0 0x7f945815fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f94577fe700 1 -- 172.21.15.60:0/2533963012 >> v2:172.21.15.105:3300/0 conn(0x7f945815c460 msgr2=0x7f9458070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f94577fe700 1 --2- 172.21.15.60:0/2533963012 >> v2:172.21.15.105:3300/0 conn(0x7f945815c460 0x7f9458070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f94577fe700 1 -- 172.21.15.60:0/2533963012 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94580cb3f0 con 0x7f94580caf80 2023-07-13T22:23:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f945c99c700 1 --2- 172.21.15.60:0/2533963012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f945806c8f0 0x7f945815fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.590+0000 7f94577fe700 1 --2- 172.21.15.60:0/2533963012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94580caf80 0x7f94580ce770 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9458072360 tx=0x7f944c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0d7c5a0c8c175e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:38.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f9456ffd700 1 -- 172.21.15.60:0/2533963012 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c011830 con 0x7f94580caf80 2023-07-13T22:23:38.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f9456ffd700 1 -- 172.21.15.60:0/2533963012 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c011990 con 0x7f94580caf80 2023-07-13T22:23:38.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f9456ffd700 1 -- 172.21.15.60:0/2533963012 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c011d20 con 0x7f94580caf80 2023-07-13T22:23:38.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f945e3ff700 1 -- 172.21.15.60:0/2533963012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94580caf80 msgr2=0x7f94580ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f945e3ff700 1 --2- 172.21.15.60:0/2533963012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94580caf80 0x7f94580ce770 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9458072360 tx=0x7f944c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f945e3ff700 1 -- 172.21.15.60:0/2533963012 shutdown_connections 2023-07-13T22:23:38.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f945e3ff700 1 --2- 172.21.15.60:0/2533963012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f945806c8f0 0x7f945815fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f945e3ff700 1 --2- 172.21.15.60:0/2533963012 >> v2:172.21.15.105:3300/0 conn(0x7f945815c460 0x7f9458070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f945e3ff700 1 --2- 172.21.15.60:0/2533963012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94580caf80 0x7f94580ce770 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f945e3ff700 1 -- 172.21.15.60:0/2533963012 >> 172.21.15.60:0/2533963012 conn(0x7f945806e580 msgr2=0x7f945806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:38.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f945e3ff700 1 -- 172.21.15.60:0/2533963012 shutdown_connections 2023-07-13T22:23:38.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.591+0000 7f945e3ff700 1 -- 172.21.15.60:0/2533963012 wait complete. 2023-07-13T22:23:38.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.593+0000 7f945e3ff700 1 Processor -- start 2023-07-13T22:23:38.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.593+0000 7f945e3ff700 1 -- start start 2023-07-13T22:23:38.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.594+0000 7f945e3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945806c8f0 0x7f945814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.594+0000 7f945e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94580caf80 0x7f94580c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.594+0000 7f945e3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f945815c460 0x7f94580ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.594+0000 7f945e3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580b5bd0 con 0x7f945806c8f0 2023-07-13T22:23:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.594+0000 7f945e3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580b5eb0 con 0x7f945815c460 2023-07-13T22:23:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.594+0000 7f945e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94580b6190 con 0x7f94580caf80 2023-07-13T22:23:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f9457fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945806c8f0 0x7f945814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f9457fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945806c8f0 0x7f945814bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55896/0 (socket says 172.21.15.60:55896) 2023-07-13T22:23:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f9457fff700 1 -- 172.21.15.60:0/3819250320 learned_addr learned my addr 172.21.15.60:0/3819250320 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f94577fe700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94580caf80 0x7f94580c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f945c99c700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f945815c460 0x7f94580ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f9457fff700 1 -- 172.21.15.60:0/3819250320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94580caf80 msgr2=0x7f94580c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f9457fff700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94580caf80 0x7f94580c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f9457fff700 1 -- 172.21.15.60:0/3819250320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f945815c460 msgr2=0x7f94580ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f9457fff700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f945815c460 0x7f94580ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f9457fff700 1 -- 172.21.15.60:0/3819250320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c009550 con 0x7f945806c8f0 2023-07-13T22:23:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f945c99c700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f945815c460 0x7f94580ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.595+0000 7f9457fff700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945806c8f0 0x7f945814bc50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f944800e9b0 tx=0x7f944800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:38.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.596+0000 7f9454ff9700 1 -- 172.21.15.60:0/3819250320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448009710 con 0x7f945806c8f0 2023-07-13T22:23:38.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.596+0000 7f9454ff9700 1 -- 172.21.15.60:0/3819250320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944800c850 con 0x7f945806c8f0 2023-07-13T22:23:38.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.596+0000 7f9454ff9700 1 -- 172.21.15.60:0/3819250320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800cbc0 con 0x7f945806c8f0 2023-07-13T22:23:38.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.596+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94580b6470 con 0x7f945806c8f0 2023-07-13T22:23:38.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.596+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945814c190 con 0x7f945806c8f0 2023-07-13T22:23:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.597+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9458158120 con 0x7f945806c8f0 2023-07-13T22:23:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.598+0000 7f9454ff9700 1 -- 172.21.15.60:0/3819250320 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f944800cd20 con 0x7f945806c8f0 2023-07-13T22:23:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.599+0000 7f9454ff9700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f944406ace0 0x7f944406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.599+0000 7f9454ff9700 1 -- 172.21.15.60:0/3819250320 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9448089460 con 0x7f945806c8f0 2023-07-13T22:23:38.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.599+0000 7f94577fe700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f944406ace0 0x7f944406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:38.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.600+0000 7f94577fe700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f944406ace0 0x7f944406d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f944c009840 tx=0x7f944c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:38.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.603+0000 7f9454ff9700 1 -- 172.21.15.60:0/3819250320 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9448054510 con 0x7f945806c8f0 2023-07-13T22:23:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.809+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f945804e210 con 0x7f945806c8f0 2023-07-13T22:23:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.809+0000 7f9454ff9700 1 -- 172.21.15.60:0/3819250320 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9448057b30 con 0x7f945806c8f0 2023-07-13T22:23:38.793 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:23:38.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f944406ace0 msgr2=0x7f944406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f944406ace0 0x7f944406d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f944c009840 tx=0x7f944c008030 comp rx=0 tx=0).stop 2023-07-13T22:23:38.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945806c8f0 msgr2=0x7f945814bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945806c8f0 0x7f945814bc50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f944800e9b0 tx=0x7f944800ecc0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 shutdown_connections 2023-07-13T22:23:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f944406ace0 0x7f944406d190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945806c8f0 0x7f945814bc50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f945815c460 0x7f94580ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 --2- 172.21.15.60:0/3819250320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94580caf80 0x7f94580c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:38.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 >> 172.21.15.60:0/3819250320 conn(0x7f945806e580 msgr2=0x7f94580b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 shutdown_connections 2023-07-13T22:23:38.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:38.812+0000 7f945e3ff700 1 -- 172.21.15.60:0/3819250320 wait complete. 2023-07-13T22:23:38.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:23:39.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.033+0000 7fa5fa1ff700 1 Processor -- start 2023-07-13T22:23:39.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.034+0000 7fa5fa1ff700 1 -- start start 2023-07-13T22:23:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.035+0000 7fa5fa1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 0x7fa5f40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.035+0000 7fa5fa1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5f40cfad0 0x7fa5f40d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.035+0000 7fa5fa1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f40cd950 con 0x7fa5f40b8400 2023-07-13T22:23:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.035+0000 7fa5fa1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f40cff20 con 0x7fa5f40cfad0 2023-07-13T22:23:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.035+0000 7fa5fa1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f40d0ac0 con 0x7fa5f40d3750 2023-07-13T22:23:39.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.035+0000 7fa5f37fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa5f40b8400 0x7fa5f40cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55242/0 (socket says 172.21.15.60:55242) 2023-07-13T22:23:39.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.035+0000 7fa5f37fe700 1 -- 172.21.15.60:0/90874053 learned_addr learned my addr 172.21.15.60:0/90874053 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:39.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.035+0000 7fa5f3fff700 1 --2- 172.21.15.60:0/90874053 >> v2:172.21.15.60:3300/0 conn(0x7fa5f40cfad0 0x7fa5f40d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.035+0000 7fa5f2ffd700 1 --2- 172.21.15.60:0/90874053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 0x7fa5f40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f27fc700 1 -- 172.21.15.60:0/90874053 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667740848 0 0) 0x7fa5e4001110 con 0x7fa5f40b8400 2023-07-13T22:23:39.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f27fc700 1 -- 172.21.15.60:0/90874053 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5dc0035b0 con 0x7fa5f40b8400 2023-07-13T22:23:39.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f3fff700 1 -- 172.21.15.60:0/90874053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 msgr2=0x7fa5f40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:39.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f3fff700 1 --2- 172.21.15.60:0/90874053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 0x7fa5f40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f3fff700 1 -- 172.21.15.60:0/90874053 >> v1:172.21.15.60:6789/0 conn(0x7fa5f40b8400 legacy=0x7fa5f40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f3fff700 1 -- 172.21.15.60:0/90874053 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40b87d0 con 0x7fa5f40cfad0 2023-07-13T22:23:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f2ffd700 1 --2- 172.21.15.60:0/90874053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 0x7fa5f40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f3fff700 1 --2- 172.21.15.60:0/90874053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40cfad0 0x7fa5f40d0f50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009d90 tx=0x7fa5ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff63a602d0c55df8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f27fc700 1 -- 172.21.15.60:0/90874053 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec00e070 con 0x7fa5f40cfad0 2023-07-13T22:23:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f27fc700 1 -- 172.21.15.60:0/90874053 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ec00bb20 con 0x7fa5f40cfad0 2023-07-13T22:23:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.036+0000 7fa5f27fc700 1 -- 172.21.15.60:0/90874053 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec012430 con 0x7fa5f40cfad0 2023-07-13T22:23:39.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.037+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/90874053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40cfad0 msgr2=0x7fa5f40d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.037+0000 7fa5fa1ff700 1 --2- 172.21.15.60:0/90874053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40cfad0 0x7fa5f40d0f50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009d90 tx=0x7fa5ec00b330 comp rx=0 tx=0).stop 2023-07-13T22:23:39.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.037+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/90874053 shutdown_connections 2023-07-13T22:23:39.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.037+0000 7fa5fa1ff700 1 --2- 172.21.15.60:0/90874053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40cfad0 0x7fa5f40d0f50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.037+0000 7fa5fa1ff700 1 --2- 172.21.15.60:0/90874053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 0x7fa5f40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.037+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/90874053 >> 172.21.15.60:0/90874053 conn(0x7fa5f406b390 msgr2=0x7fa5f406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:39.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.037+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/90874053 shutdown_connections 2023-07-13T22:23:39.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.037+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/90874053 wait complete. 2023-07-13T22:23:39.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.039+0000 7fa5fa1ff700 1 Processor -- start 2023-07-13T22:23:39.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.039+0000 7fa5fa1ff700 1 -- start start 2023-07-13T22:23:39.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.040+0000 7fa5fa1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40b8400 0x7fa5f415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.040+0000 7fa5fa1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40cfad0 0x7fa5f41ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.040+0000 7fa5fa1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 0x7fa5f415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.040+0000 7fa5fa1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f415e720 con 0x7fa5f40b8400 2023-07-13T22:23:39.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.040+0000 7fa5fa1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f415ea00 con 0x7fa5f40cfad0 2023-07-13T22:23:39.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.040+0000 7fa5fa1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f415ece0 con 0x7fa5f40d3750 2023-07-13T22:23:39.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.040+0000 7fa5f37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40b8400 0x7fa5f415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.040+0000 7fa5f37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40b8400 0x7fa5f415f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55922/0 (socket says 172.21.15.60:55922) 2023-07-13T22:23:39.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.040+0000 7fa5f37fe700 1 -- 172.21.15.60:0/2782678456 learned_addr learned my addr 172.21.15.60:0/2782678456 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:39.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5f37fe700 1 -- 172.21.15.60:0/2782678456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 msgr2=0x7fa5f415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5f37fe700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 0x7fa5f415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5f37fe700 1 -- 172.21.15.60:0/2782678456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40cfad0 msgr2=0x7fa5f41ef6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5f2ffd700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40cfad0 0x7fa5f41ef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5f37fe700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40cfad0 0x7fa5f41ef6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5f37fe700 1 -- 172.21.15.60:0/2782678456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec009a70 con 0x7fa5f40b8400 2023-07-13T22:23:39.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5f37fe700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40b8400 0x7fa5f415f4e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4007fd0 tx=0x7fa5e400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:39.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5dbfff700 1 -- 172.21.15.60:0/2782678456 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4005d70 con 0x7fa5f40b8400 2023-07-13T22:23:39.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5dbfff700 1 -- 172.21.15.60:0/2782678456 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e40070d0 con 0x7fa5f40b8400 2023-07-13T22:23:39.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f415efc0 con 0x7fa5f40b8400 2023-07-13T22:23:39.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5dbfff700 1 -- 172.21.15.60:0/2782678456 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4007460 con 0x7fa5f40b8400 2023-07-13T22:23:39.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.041+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f41f2870 con 0x7fa5f40b8400 2023-07-13T22:23:39.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.042+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f40d0750 con 0x7fa5f40b8400 2023-07-13T22:23:39.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.043+0000 7fa5dbfff700 1 -- 172.21.15.60:0/2782678456 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5e40075c0 con 0x7fa5f40b8400 2023-07-13T22:23:39.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.044+0000 7fa5dbfff700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5e006ad10 0x7fa5e006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.044+0000 7fa5dbfff700 1 -- 172.21.15.60:0/2782678456 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5e40888d0 con 0x7fa5f40b8400 2023-07-13T22:23:39.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.044+0000 7fa5f2ffd700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5e006ad10 0x7fa5e006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.045+0000 7fa5f2ffd700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5e006ad10 0x7fa5e006d1c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40b4750 tx=0x7fa5e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:39.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.048+0000 7fa5dbfff700 1 -- 172.21.15.60:0/2782678456 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa5e40538e0 con 0x7fa5f40b8400 2023-07-13T22:23:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.276+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa5f40cd950 con 0x7fa5f40b8400 2023-07-13T22:23:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.276+0000 7fa5dbfff700 1 -- 172.21.15.60:0/2782678456 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa5f40cd950 con 0x7fa5f40b8400 2023-07-13T22:23:39.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.278+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5e006ad10 msgr2=0x7fa5e006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.278+0000 7fa5fa1ff700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5e006ad10 0x7fa5e006d1c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40b4750 tx=0x7fa5e8008040 comp rx=0 tx=0).stop 2023-07-13T22:23:39.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40b8400 msgr2=0x7fa5f415f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40b8400 0x7fa5f415f4e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4007fd0 tx=0x7fa5e400c860 comp rx=0 tx=0).stop 2023-07-13T22:23:39.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 shutdown_connections 2023-07-13T22:23:39.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5e006ad10 0x7fa5e006d1c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5f40b8400 0x7fa5f415f4e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f40d3750 0x7fa5f415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 --2- 172.21.15.60:0/2782678456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40cfad0 0x7fa5f41ef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 >> 172.21.15.60:0/2782678456 conn(0x7fa5f406b390 msgr2=0x7fa5f40d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:39.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 shutdown_connections 2023-07-13T22:23:39.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.279+0000 7fa5fa1ff700 1 -- 172.21.15.60:0/2782678456 wait complete. 2023-07-13T22:23:39.264 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:23:39.280 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:23:39.280 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:23:39.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:23:39.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.591+0000 7f73599ff700 1 Processor -- start 2023-07-13T22:23:39.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.592+0000 7f73599ff700 1 -- start start 2023-07-13T22:23:39.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.593+0000 7f73599ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540b8570 0x7f73540ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.593+0000 7f73599ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f73540d15c0 0x7f735415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.593+0000 7f73599ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73540d09d0 con 0x7f73540d35c0 2023-07-13T22:23:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.593+0000 7f73599ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73540d3b30 con 0x7f73540d15c0 2023-07-13T22:23:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.593+0000 7f73599ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73540cda10 con 0x7f73540b8570 2023-07-13T22:23:39.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f73537fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f73540d15c0 0x7f735415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f7352ffd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540b8570 0x7f73540ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f73527fc700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f73540d35c0 0x7f73540d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:52604/0 (socket says 172.21.15.60:52604) 2023-07-13T22:23:39.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f73527fc700 1 -- 172.21.15.60:0/3694644238 learned_addr learned my addr 172.21.15.60:0/3694644238 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:39.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f7351ffb700 1 -- 172.21.15.60:0/3694644238 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487019887 0 0) 0x7f73540d09d0 con 0x7f73540d35c0 2023-07-13T22:23:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f7351ffb700 1 -- 172.21.15.60:0/3694644238 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f733c0035b0 con 0x7f73540d35c0 2023-07-13T22:23:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f7352ffd700 1 -- 172.21.15.60:0/3694644238 >> v2:172.21.15.105:3300/0 conn(0x7f73540d15c0 msgr2=0x7f735415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f7352ffd700 1 --2- 172.21.15.60:0/3694644238 >> v2:172.21.15.105:3300/0 conn(0x7f73540d15c0 0x7f735415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f7352ffd700 1 -- 172.21.15.60:0/3694644238 >> v1:172.21.15.105:6789/0 conn(0x7f73540d35c0 legacy=0x7f73540d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f7352ffd700 1 -- 172.21.15.60:0/3694644238 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73540cd450 con 0x7f73540b8570 2023-07-13T22:23:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f73537fe700 1 --2- 172.21.15.60:0/3694644238 >> v2:172.21.15.105:3300/0 conn(0x7f73540d15c0 0x7f735415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.594+0000 7f7352ffd700 1 --2- 172.21.15.60:0/3694644238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540b8570 0x7f73540ceeb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7344009820 tx=0x7f7344009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44009ab5f75926f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:39.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f7351ffb700 1 -- 172.21.15.60:0/3694644238 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344005e30 con 0x7f73540b8570 2023-07-13T22:23:39.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f7351ffb700 1 -- 172.21.15.60:0/3694644238 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7344006030 con 0x7f73540b8570 2023-07-13T22:23:39.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f7351ffb700 1 -- 172.21.15.60:0/3694644238 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73440063c0 con 0x7f73540b8570 2023-07-13T22:23:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f73599ff700 1 -- 172.21.15.60:0/3694644238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540b8570 msgr2=0x7f73540ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f73599ff700 1 --2- 172.21.15.60:0/3694644238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540b8570 0x7f73540ceeb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7344009820 tx=0x7f7344009b30 comp rx=0 tx=0).stop 2023-07-13T22:23:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f73599ff700 1 -- 172.21.15.60:0/3694644238 shutdown_connections 2023-07-13T22:23:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f73599ff700 1 --2- 172.21.15.60:0/3694644238 >> v2:172.21.15.105:3300/0 conn(0x7f73540d15c0 0x7f735415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f73599ff700 1 --2- 172.21.15.60:0/3694644238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540b8570 0x7f73540ceeb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f73599ff700 1 -- 172.21.15.60:0/3694644238 >> 172.21.15.60:0/3694644238 conn(0x7f7354071600 msgr2=0x7f73540719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f73599ff700 1 -- 172.21.15.60:0/3694644238 shutdown_connections 2023-07-13T22:23:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.595+0000 7f73599ff700 1 -- 172.21.15.60:0/3694644238 wait complete. 2023-07-13T22:23:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.597+0000 7f73599ff700 1 Processor -- start 2023-07-13T22:23:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.597+0000 7f73599ff700 1 -- start start 2023-07-13T22:23:39.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.598+0000 7f73599ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73540b8570 0x7f7354155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.598+0000 7f73599ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540d15c0 0x7f73540b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.598+0000 7f73599ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73540d35c0 0x7f735415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.598+0000 7f73599ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73540c6790 con 0x7f73540d35c0 2023-07-13T22:23:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.598+0000 7f73599ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73540c6d50 con 0x7f73540b8570 2023-07-13T22:23:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.598+0000 7f73599ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73541e8530 con 0x7f73540d15c0 2023-07-13T22:23:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73537fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73540d35c0 0x7f735415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f7352ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73540b8570 0x7f7354155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73527fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540d15c0 0x7f73540b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73537fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73540d35c0 0x7f735415ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55924/0 (socket says 172.21.15.60:55924) 2023-07-13T22:23:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73537fe700 1 -- 172.21.15.60:0/583124526 learned_addr learned my addr 172.21.15.60:0/583124526 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73537fe700 1 -- 172.21.15.60:0/583124526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540d15c0 msgr2=0x7f73540b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73537fe700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540d15c0 0x7f73540b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73537fe700 1 -- 172.21.15.60:0/583124526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73540b8570 msgr2=0x7f7354155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73537fe700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73540b8570 0x7f7354155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73537fe700 1 -- 172.21.15.60:0/583124526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c009a70 con 0x7f73540d35c0 2023-07-13T22:23:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.599+0000 7f73537fe700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73540d35c0 0x7f735415ea80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f734c00ef90 tx=0x7f734c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.600+0000 7f733bfff700 1 -- 172.21.15.60:0/583124526 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c00cc90 con 0x7f73540d35c0 2023-07-13T22:23:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.600+0000 7f733bfff700 1 -- 172.21.15.60:0/583124526 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c00f040 con 0x7f73540d35c0 2023-07-13T22:23:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.600+0000 7f733bfff700 1 -- 172.21.15.60:0/583124526 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c0146c0 con 0x7f73540d35c0 2023-07-13T22:23:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.600+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7344009430 con 0x7f73540d35c0 2023-07-13T22:23:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.600+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73540c69e0 con 0x7f73540d35c0 2023-07-13T22:23:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.601+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73540c6ec0 con 0x7f73540d35c0 2023-07-13T22:23:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.602+0000 7f733bfff700 1 -- 172.21.15.60:0/583124526 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f734c014820 con 0x7f73540d35c0 2023-07-13T22:23:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.603+0000 7f733bfff700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f734006ace0 0x7f734006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.603+0000 7f733bfff700 1 -- 172.21.15.60:0/583124526 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f734c08a170 con 0x7f73540d35c0 2023-07-13T22:23:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.603+0000 7f7352ffd700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f734006ace0 0x7f734006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:39.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.604+0000 7f7352ffd700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f734006ace0 0x7f734006d190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7344000c00 tx=0x7f7344008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:39.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.607+0000 7f733bfff700 1 -- 172.21.15.60:0/583124526 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f734c058c10 con 0x7f73540d35c0 2023-07-13T22:23:39.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.813+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f735404e280 con 0x7f73540d35c0 2023-07-13T22:23:39.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.815+0000 7f733bfff700 1 -- 172.21.15.60:0/583124526 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18257 (secure 0 0 0) 0x7f734c009d70 con 0x7f73540d35c0 2023-07-13T22:23:39.798 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:39.799 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:16.111947+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:23:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f734006ace0 msgr2=0x7f734006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f734006ace0 0x7f734006d190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7344000c00 tx=0x7f7344008070 comp rx=0 tx=0).stop 2023-07-13T22:23:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73540d35c0 msgr2=0x7f735415ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73540d35c0 0x7f735415ea80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f734c00ef90 tx=0x7f734c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 shutdown_connections 2023-07-13T22:23:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f734006ace0 0x7f734006d190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73540d35c0 0x7f735415ea80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73540b8570 0x7f7354155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 --2- 172.21.15.60:0/583124526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73540d15c0 0x7f73540b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 >> 172.21.15.60:0/583124526 conn(0x7f7354071600 msgr2=0x7f73540b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 shutdown_connections 2023-07-13T22:23:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:39.817+0000 7f73599ff700 1 -- 172.21.15.60:0/583124526 wait complete. 2023-07-13T22:23:39.818 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:23:12.304337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:23:13.598583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:23:39.819 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:23:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.043+0000 7f22897ff700 1 Processor -- start 2023-07-13T22:23:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.043+0000 7f22897ff700 1 -- start start 2023-07-13T22:23:40.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22897ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 0x7f22840ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22897ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22840ac490 0x7f22840c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22897ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22840cf6d0 con 0x7f22840ca3a0 2023-07-13T22:23:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22897ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22840ac900 con 0x7f22840ac490 2023-07-13T22:23:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22897ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22840acbe0 con 0x7f228415c790 2023-07-13T22:23:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22827fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22840ac490 0x7f22840c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22837fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f22840ca3a0 0x7f22840ad9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55244/0 (socket says 172.21.15.60:55244) 2023-07-13T22:23:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22827fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22840ac490 0x7f22840c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55938/0 (socket says 172.21.15.60:55938) 2023-07-13T22:23:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22837fe700 1 -- 172.21.15.60:0/1720870487 learned_addr learned my addr 172.21.15.60:0/1720870487 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:40.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f2282ffd700 1 --2- 172.21.15.60:0/1720870487 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 0x7f22840ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22827fc700 1 -- 172.21.15.60:0/1720870487 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 msgr2=0x7f22840ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22827fc700 1 --2- 172.21.15.60:0/1720870487 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 0x7f22840ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22827fc700 1 -- 172.21.15.60:0/1720870487 >> v1:172.21.15.60:6789/0 conn(0x7f22840ca3a0 legacy=0x7f22840ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.045+0000 7f22827fc700 1 -- 172.21.15.60:0/1720870487 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22840acec0 con 0x7f22840ac490 2023-07-13T22:23:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.046+0000 7f22827fc700 1 --2- 172.21.15.60:0/1720870487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 0x7f22840c9e60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2278009840 tx=0x7f2278009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06473fc64667e9d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.046+0000 7f2281ffb700 1 -- 172.21.15.60:0/1720870487 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278011830 con 0x7f22840ac490 2023-07-13T22:23:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.046+0000 7f2281ffb700 1 -- 172.21.15.60:0/1720870487 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2278011990 con 0x7f22840ac490 2023-07-13T22:23:40.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.046+0000 7f2281ffb700 1 -- 172.21.15.60:0/1720870487 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278011d20 con 0x7f22840ac490 2023-07-13T22:23:40.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.046+0000 7f22897ff700 1 -- 172.21.15.60:0/1720870487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 msgr2=0x7f22840c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.046+0000 7f22897ff700 1 --2- 172.21.15.60:0/1720870487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 0x7f22840c9e60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2278009840 tx=0x7f2278009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:40.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.047+0000 7f22897ff700 1 -- 172.21.15.60:0/1720870487 shutdown_connections 2023-07-13T22:23:40.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.047+0000 7f22897ff700 1 --2- 172.21.15.60:0/1720870487 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 0x7f22840ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.047+0000 7f22897ff700 1 --2- 172.21.15.60:0/1720870487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 0x7f22840c9e60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.047+0000 7f22897ff700 1 -- 172.21.15.60:0/1720870487 >> 172.21.15.60:0/1720870487 conn(0x7f228406b900 msgr2=0x7f22840cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:40.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.047+0000 7f22897ff700 1 -- 172.21.15.60:0/1720870487 shutdown_connections 2023-07-13T22:23:40.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.047+0000 7f22897ff700 1 -- 172.21.15.60:0/1720870487 wait complete. 2023-07-13T22:23:40.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.048+0000 7f22897ff700 1 Processor -- start 2023-07-13T22:23:40.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.049+0000 7f22897ff700 1 -- start start 2023-07-13T22:23:40.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.049+0000 7f22897ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 0x7f22841515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.049+0000 7f22897ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22840ca3a0 0x7f228415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f22897ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 0x7f22840b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f22897ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2284151b00 con 0x7f22840ac490 2023-07-13T22:23:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f22897ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22840b2800 con 0x7f22840ca3a0 2023-07-13T22:23:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f22897ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22840b2aa0 con 0x7f228415c790 2023-07-13T22:23:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f2282ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 0x7f22841515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f2282ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 0x7f22841515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55954/0 (socket says 172.21.15.60:55954) 2023-07-13T22:23:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f2282ffd700 1 -- 172.21.15.60:0/3213325923 learned_addr learned my addr 172.21.15.60:0/3213325923 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f22827fc700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22840ca3a0 0x7f228415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f22837fe700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 0x7f22840b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f2282ffd700 1 -- 172.21.15.60:0/3213325923 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 msgr2=0x7f22840b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f2282ffd700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 0x7f22840b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f2282ffd700 1 -- 172.21.15.60:0/3213325923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22840ca3a0 msgr2=0x7f228415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f2282ffd700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22840ca3a0 0x7f228415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f2282ffd700 1 -- 172.21.15.60:0/3213325923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780094a0 con 0x7f22840ac490 2023-07-13T22:23:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.050+0000 7f2282ffd700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 0x7f22841515c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f227400e990 tx=0x7f227400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.051+0000 7f226b7fe700 1 -- 172.21.15.60:0/3213325923 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740096f0 con 0x7f22840ac490 2023-07-13T22:23:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.051+0000 7f226b7fe700 1 -- 172.21.15.60:0/3213325923 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227400c850 con 0x7f22840ac490 2023-07-13T22:23:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.051+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22840b2d80 con 0x7f22840ac490 2023-07-13T22:23:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.051+0000 7f226b7fe700 1 -- 172.21.15.60:0/3213325923 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227400cbc0 con 0x7f22840ac490 2023-07-13T22:23:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.051+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22840b3340 con 0x7f22840ac490 2023-07-13T22:23:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.052+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22840cdad0 con 0x7f22840ac490 2023-07-13T22:23:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.053+0000 7f226b7fe700 1 -- 172.21.15.60:0/3213325923 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f227400cd20 con 0x7f22840ac490 2023-07-13T22:23:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.054+0000 7f226b7fe700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f227006ace0 0x7f227006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.054+0000 7f226b7fe700 1 -- 172.21.15.60:0/3213325923 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2274089970 con 0x7f22840ac490 2023-07-13T22:23:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.054+0000 7f22827fc700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f227006ace0 0x7f227006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.055+0000 7f22827fc700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f227006ace0 0x7f227006d190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2278000c00 tx=0x7f227801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.058+0000 7f226b7fe700 1 -- 172.21.15.60:0/3213325923 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2274054a10 con 0x7f22840ac490 2023-07-13T22:23:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.269+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f228404e210 con 0x7f22840ac490 2023-07-13T22:23:40.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.269+0000 7f226b7fe700 1 -- 172.21.15.60:0/3213325923 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f22740170a0 con 0x7f22840ac490 2023-07-13T22:23:40.253 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:23:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f227006ace0 msgr2=0x7f227006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f227006ace0 0x7f227006d190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2278000c00 tx=0x7f227801c040 comp rx=0 tx=0).stop 2023-07-13T22:23:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 msgr2=0x7f22841515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 0x7f22841515c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f227400e990 tx=0x7f227400ed50 comp rx=0 tx=0).stop 2023-07-13T22:23:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 shutdown_connections 2023-07-13T22:23:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f227006ace0 0x7f227006d190 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22840ac490 0x7f22841515c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f228415c790 0x7f22840b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 --2- 172.21.15.60:0/3213325923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22840ca3a0 0x7f228415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 >> 172.21.15.60:0/3213325923 conn(0x7f228406b900 msgr2=0x7f22840cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:40.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 shutdown_connections 2023-07-13T22:23:40.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.272+0000 7f22897ff700 1 -- 172.21.15.60:0/3213325923 wait complete. 2023-07-13T22:23:40.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:23:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.495+0000 7effb0fff700 1 Processor -- start 2023-07-13T22:23:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.496+0000 7effb0fff700 1 -- start start 2023-07-13T22:23:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.497+0000 7effb0fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7effac0b8200 0x7effac0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.497+0000 7effb0fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effac0b1ac0 0x7effac0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.497+0000 7effb0fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effac0d1870 con 0x7effac0cede0 2023-07-13T22:23:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.497+0000 7effb0fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effac0cf370 con 0x7effac0b1ac0 2023-07-13T22:23:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.497+0000 7effb0fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effac0d12b0 con 0x7effac0b8200 2023-07-13T22:23:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.497+0000 7effa3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effac0b1ac0 0x7effac0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.497+0000 7effa3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effac0b1ac0 0x7effac0cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55956/0 (socket says 172.21.15.60:55956) 2023-07-13T22:23:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.497+0000 7effa3fff700 1 -- 172.21.15.60:0/528188838 learned_addr learned my addr 172.21.15.60:0/528188838 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.497+0000 7effaa59c700 1 --2- 172.21.15.60:0/528188838 >> v2:172.21.15.105:3300/0 conn(0x7effac0b8200 0x7effac0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.498+0000 7effa3fff700 1 -- 172.21.15.60:0/528188838 >> v2:172.21.15.105:3300/0 conn(0x7effac0b8200 msgr2=0x7effac0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.498+0000 7effa3fff700 1 --2- 172.21.15.60:0/528188838 >> v2:172.21.15.105:3300/0 conn(0x7effac0b8200 0x7effac0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.498+0000 7effa3fff700 1 -- 172.21.15.60:0/528188838 >> v1:172.21.15.105:6789/0 conn(0x7effac0cede0 legacy=0x7effac15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.498+0000 7effa3fff700 1 -- 172.21.15.60:0/528188838 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac0d1590 con 0x7effac0b1ac0 2023-07-13T22:23:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.498+0000 7effaa59c700 1 --2- 172.21.15.60:0/528188838 >> v2:172.21.15.105:3300/0 conn(0x7effac0b8200 0x7effac0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.498+0000 7effa3fff700 1 --2- 172.21.15.60:0/528188838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b1ac0 0x7effac0cfb80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7effac0b2660 tx=0x7eff94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b8a15bf7f21618 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.498+0000 7effa9d9b700 1 -- 172.21.15.60:0/528188838 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94004230 con 0x7effac0b1ac0 2023-07-13T22:23:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.498+0000 7effa9d9b700 1 -- 172.21.15.60:0/528188838 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff94004390 con 0x7effac0b1ac0 2023-07-13T22:23:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.498+0000 7effa9d9b700 1 -- 172.21.15.60:0/528188838 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff940036a0 con 0x7effac0b1ac0 2023-07-13T22:23:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.499+0000 7effb0fff700 1 -- 172.21.15.60:0/528188838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b1ac0 msgr2=0x7effac0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.499+0000 7effb0fff700 1 --2- 172.21.15.60:0/528188838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b1ac0 0x7effac0cfb80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7effac0b2660 tx=0x7eff94009bd0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.499+0000 7effb0fff700 1 -- 172.21.15.60:0/528188838 shutdown_connections 2023-07-13T22:23:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.499+0000 7effb0fff700 1 --2- 172.21.15.60:0/528188838 >> v2:172.21.15.105:3300/0 conn(0x7effac0b8200 0x7effac0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.499+0000 7effb0fff700 1 --2- 172.21.15.60:0/528188838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b1ac0 0x7effac0cfb80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.499+0000 7effb0fff700 1 -- 172.21.15.60:0/528188838 >> 172.21.15.60:0/528188838 conn(0x7effac06b2f0 msgr2=0x7effac06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.499+0000 7effb0fff700 1 -- 172.21.15.60:0/528188838 shutdown_connections 2023-07-13T22:23:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.499+0000 7effb0fff700 1 -- 172.21.15.60:0/528188838 wait complete. 2023-07-13T22:23:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.501+0000 7effb0fff700 1 Processor -- start 2023-07-13T22:23:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.501+0000 7effb0fff700 1 -- start start 2023-07-13T22:23:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effb0fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effac0b1ac0 0x7effac1490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effb0fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b8200 0x7effac0c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effb0fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effac0cede0 0x7effac15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effb0fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac15ca60 con 0x7effac0b8200 2023-07-13T22:23:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effb0fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac15cd40 con 0x7effac0cede0 2023-07-13T22:23:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effb0fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac15d020 con 0x7effac0b1ac0 2023-07-13T22:23:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effa3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b8200 0x7effac0c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effa3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b8200 0x7effac0c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55958/0 (socket says 172.21.15.60:55958) 2023-07-13T22:23:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effa3fff700 1 -- 172.21.15.60:0/3303550773 learned_addr learned my addr 172.21.15.60:0/3303550773 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effaad9d700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effac0cede0 0x7effac15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.502+0000 7effaa59c700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effac0b1ac0 0x7effac1490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effa3fff700 1 -- 172.21.15.60:0/3303550773 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effac0b1ac0 msgr2=0x7effac1490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effa3fff700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effac0b1ac0 0x7effac1490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effa3fff700 1 -- 172.21.15.60:0/3303550773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effac0cede0 msgr2=0x7effac15dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effa3fff700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effac0cede0 0x7effac15dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effa3fff700 1 -- 172.21.15.60:0/3303550773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94009550 con 0x7effac0b8200 2023-07-13T22:23:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effa3fff700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b8200 0x7effac0c8ab0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7effac06bf10 tx=0x7eff94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effa2ffd700 1 -- 172.21.15.60:0/3303550773 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94003ea0 con 0x7effac0b8200 2023-07-13T22:23:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effa2ffd700 1 -- 172.21.15.60:0/3303550773 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff94011630 con 0x7effac0b8200 2023-07-13T22:23:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effac15d300 con 0x7effac0b8200 2023-07-13T22:23:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.503+0000 7effa2ffd700 1 -- 172.21.15.60:0/3303550773 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff940119e0 con 0x7effac0b8200 2023-07-13T22:23:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.504+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac15d8c0 con 0x7effac0b8200 2023-07-13T22:23:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.504+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effac15dab0 con 0x7effac0b8200 2023-07-13T22:23:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.506+0000 7effa2ffd700 1 -- 172.21.15.60:0/3303550773 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff94011b40 con 0x7effac0b8200 2023-07-13T22:23:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.507+0000 7effa2ffd700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.507+0000 7effa2ffd700 1 -- 172.21.15.60:0/3303550773 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff9408aa60 con 0x7effac0b8200 2023-07-13T22:23:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.507+0000 7effaa59c700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.508+0000 7effaa59c700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7eff9c006fd0 tx=0x7eff9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:40.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.511+0000 7effa2ffd700 1 -- 172.21.15.60:0/3303550773 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7eff94055b00 con 0x7effac0b8200 2023-07-13T22:23:40.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.718+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7effac04e210 con 0x7effac0b8200 2023-07-13T22:23:40.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.718+0000 7effa2ffd700 1 -- 172.21.15.60:0/3303550773 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff9401f020 con 0x7effac0b8200 2023-07-13T22:23:40.702 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:23:40.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 msgr2=0x7eff8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7eff9c006fd0 tx=0x7eff9c008040 comp rx=0 tx=0).stop 2023-07-13T22:23:40.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b8200 msgr2=0x7effac0c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b8200 0x7effac0c8ab0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7effac06bf10 tx=0x7eff94004750 comp rx=0 tx=0).stop 2023-07-13T22:23:40.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 shutdown_connections 2023-07-13T22:23:40.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effac0cede0 0x7effac15dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effac0b1ac0 0x7effac1490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 --2- 172.21.15.60:0/3303550773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effac0b8200 0x7effac0c8ab0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 >> 172.21.15.60:0/3303550773 conn(0x7effac06b2f0 msgr2=0x7effac0732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:40.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 shutdown_connections 2023-07-13T22:23:40.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.721+0000 7effb0fff700 1 -- 172.21.15.60:0/3303550773 wait complete. 2023-07-13T22:23:40.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:23:40.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.944+0000 7f4046fff700 1 Processor -- start 2023-07-13T22:23:40.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.944+0000 7f4046fff700 1 -- start start 2023-07-13T22:23:40.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.945+0000 7f4046fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40400b1ac0 0x7f40400d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.945+0000 7f4046fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40400d1870 con 0x7f40400b8200 2023-07-13T22:23:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.945+0000 7f4046fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40400cf370 con 0x7f40400cfd50 2023-07-13T22:23:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.945+0000 7f4046fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40400d12b0 con 0x7f40400b1ac0 2023-07-13T22:23:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.945+0000 7f404559c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40400b1ac0 0x7f40400d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.945+0000 7f404559c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40400b1ac0 0x7f40400d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55972/0 (socket says 172.21.15.60:55972) 2023-07-13T22:23:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.945+0000 7f404559c700 1 -- 172.21.15.60:0/1325030568 learned_addr learned my addr 172.21.15.60:0/1325030568 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:40.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.946+0000 7f404559c700 1 -- 172.21.15.60:0/1325030568 >> v1:172.21.15.105:6789/0 conn(0x7f40400cfd50 legacy=0x7f40400d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.946+0000 7f404559c700 1 -- 172.21.15.60:0/1325030568 >> v1:172.21.15.60:6789/0 conn(0x7f40400b8200 legacy=0x7f40400cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.946+0000 7f404559c700 1 -- 172.21.15.60:0/1325030568 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40400d1590 con 0x7f40400b1ac0 2023-07-13T22:23:40.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.946+0000 7f404559c700 1 --2- 172.21.15.60:0/1325030568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 0x7f40400d2ad0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4038009e50 tx=0x7f403800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367a97e47b2a90bb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:40.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.946+0000 7f403f7fe700 1 -- 172.21.15.60:0/1325030568 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403800e070 con 0x7f40400b1ac0 2023-07-13T22:23:40.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.946+0000 7f403f7fe700 1 -- 172.21.15.60:0/1325030568 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403800b9c0 con 0x7f40400b1ac0 2023-07-13T22:23:40.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.946+0000 7f403f7fe700 1 -- 172.21.15.60:0/1325030568 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403800bd70 con 0x7f40400b1ac0 2023-07-13T22:23:40.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.947+0000 7f4046fff700 1 -- 172.21.15.60:0/1325030568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 msgr2=0x7f40400d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.947+0000 7f4046fff700 1 --2- 172.21.15.60:0/1325030568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 0x7f40400d2ad0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4038009e50 tx=0x7f403800b330 comp rx=0 tx=0).stop 2023-07-13T22:23:40.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.947+0000 7f4046fff700 1 -- 172.21.15.60:0/1325030568 shutdown_connections 2023-07-13T22:23:40.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.947+0000 7f4046fff700 1 --2- 172.21.15.60:0/1325030568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 0x7f40400d2ad0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.947+0000 7f4046fff700 1 -- 172.21.15.60:0/1325030568 >> 172.21.15.60:0/1325030568 conn(0x7f404006b2f0 msgr2=0x7f404006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:40.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.947+0000 7f4046fff700 1 -- 172.21.15.60:0/1325030568 shutdown_connections 2023-07-13T22:23:40.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.947+0000 7f4046fff700 1 -- 172.21.15.60:0/1325030568 wait complete. 2023-07-13T22:23:40.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.949+0000 7f4046fff700 1 Processor -- start 2023-07-13T22:23:40.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.949+0000 7f4046fff700 1 -- start start 2023-07-13T22:23:40.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.950+0000 7f4046fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 0x7f4040155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.950+0000 7f4046fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40400b8200 0x7f40401598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.950+0000 7f4046fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40400cfd50 0x7f40401556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.950+0000 7f4046fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4040159f10 con 0x7f40400b1ac0 2023-07-13T22:23:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.950+0000 7f4046fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404015dfe0 con 0x7f40400cfd50 2023-07-13T22:23:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.950+0000 7f4046fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404015e2c0 con 0x7f40400b8200 2023-07-13T22:23:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.950+0000 7f4044d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 0x7f4040155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.950+0000 7f4044d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 0x7f4040155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55984/0 (socket says 172.21.15.60:55984) 2023-07-13T22:23:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.950+0000 7f4044d9b700 1 -- 172.21.15.60:0/2154858138 learned_addr learned my addr 172.21.15.60:0/2154858138 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f404559c700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40400cfd50 0x7f40401556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f403ffff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40400b8200 0x7f40401598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f4044d9b700 1 -- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40400b8200 msgr2=0x7f40401598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f4044d9b700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40400b8200 0x7f40401598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f4044d9b700 1 -- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40400cfd50 msgr2=0x7f40401556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f4044d9b700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40400cfd50 0x7f40401556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f4044d9b700 1 -- 172.21.15.60:0/2154858138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038009b30 con 0x7f40400b1ac0 2023-07-13T22:23:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f403ffff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40400b8200 0x7f40401598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f404559c700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40400cfd50 0x7f40401556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f4044d9b700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 0x7f4040155ee0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f403400c4e0 tx=0x7f403400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f403d7fa700 1 -- 172.21.15.60:0/2154858138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40340052d0 con 0x7f40400b1ac0 2023-07-13T22:23:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f403d7fa700 1 -- 172.21.15.60:0/2154858138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4034005430 con 0x7f40400b1ac0 2023-07-13T22:23:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f403d7fa700 1 -- 172.21.15.60:0/2154858138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40340038c0 con 0x7f40400b1ac0 2023-07-13T22:23:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404015e5a0 con 0x7f40400b1ac0 2023-07-13T22:23:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.951+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4040155bf0 con 0x7f40400b1ac0 2023-07-13T22:23:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.952+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404015eac0 con 0x7f40400b1ac0 2023-07-13T22:23:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.954+0000 7f403d7fa700 1 -- 172.21.15.60:0/2154858138 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4034003a20 con 0x7f40400b1ac0 2023-07-13T22:23:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.955+0000 7f403d7fa700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f402406ace0 0x7f402406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.955+0000 7f403d7fa700 1 -- 172.21.15.60:0/2154858138 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40340889b0 con 0x7f40400b1ac0 2023-07-13T22:23:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.955+0000 7f403ffff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f402406ace0 0x7f402406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:40.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.956+0000 7f403ffff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f402406ace0 0x7f402406d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f40400b8b60 tx=0x7f402c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:40.959+0000 7f403d7fa700 1 -- 172.21.15.60:0/2154858138 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f404015eac0 con 0x7f40400b1ac0 2023-07-13T22:23:41.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.166+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f404004e210 con 0x7f40400b1ac0 2023-07-13T22:23:41.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.167+0000 7f403d7fa700 1 -- 172.21.15.60:0/2154858138 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4034053a50 con 0x7f40400b1ac0 2023-07-13T22:23:41.150 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:23:41.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f402406ace0 msgr2=0x7f402406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f402406ace0 0x7f402406d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f40400b8b60 tx=0x7f402c009040 comp rx=0 tx=0).stop 2023-07-13T22:23:41.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 msgr2=0x7f4040155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 0x7f4040155ee0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f403400c4e0 tx=0x7f403400c8a0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 shutdown_connections 2023-07-13T22:23:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f402406ace0 0x7f402406d190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40400b1ac0 0x7f4040155ee0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40400cfd50 0x7f40401556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 --2- 172.21.15.60:0/2154858138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f40400b8200 0x7f40401598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.169+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 >> 172.21.15.60:0/2154858138 conn(0x7f404006b2f0 msgr2=0x7f40400cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.170+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 shutdown_connections 2023-07-13T22:23:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.170+0000 7f4046fff700 1 -- 172.21.15.60:0/2154858138 wait complete. 2023-07-13T22:23:41.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:23:41.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.394+0000 7f9c47fff700 1 Processor -- start 2023-07-13T22:23:41.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.394+0000 7f9c47fff700 1 -- start start 2023-07-13T22:23:41.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.395+0000 7f9c47fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9c400b8200 0x7f9c400cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.395+0000 7f9c47fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400cede0 0x7f9c400cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.395+0000 7f9c47fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c400d1870 con 0x7f9c400b1ac0 2023-07-13T22:23:41.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c47fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c400cf370 con 0x7f9c400b8200 2023-07-13T22:23:41.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c47fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400d12b0 con 0x7f9c400cede0 2023-07-13T22:23:41.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c45d9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9c400b8200 0x7f9c400cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c4659c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400cede0 0x7f9c400cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c45d9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9c400b8200 0x7f9c400cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:33896/0 (socket says 172.21.15.60:33896) 2023-07-13T22:23:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c45d9b700 1 -- 172.21.15.60:0/1373543863 learned_addr learned my addr 172.21.15.60:0/1373543863 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c4659c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400cede0 0x7f9c400cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:52714/0 (socket says 172.21.15.60:52714) 2023-07-13T22:23:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c4659c700 1 -- 172.21.15.60:0/1373543863 >> v2:172.21.15.105:3300/0 conn(0x7f9c400b8200 msgr2=0x7f9c400cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c4659c700 1 --2- 172.21.15.60:0/1373543863 >> v2:172.21.15.105:3300/0 conn(0x7f9c400b8200 0x7f9c400cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c4659c700 1 -- 172.21.15.60:0/1373543863 >> v1:172.21.15.105:6789/0 conn(0x7f9c400b1ac0 legacy=0x7f9c4015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.396+0000 7f9c4659c700 1 -- 172.21.15.60:0/1373543863 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c400d1590 con 0x7f9c400cede0 2023-07-13T22:23:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.397+0000 7f9c4659c700 1 --2- 172.21.15.60:0/1373543863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400cede0 0x7f9c400cfb80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009d90 tx=0x7f9c3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b578a63d320e342 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:41.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.397+0000 7f9c44d99700 1 -- 172.21.15.60:0/1373543863 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800e070 con 0x7f9c400cede0 2023-07-13T22:23:41.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.397+0000 7f9c44d99700 1 -- 172.21.15.60:0/1373543863 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3800bb20 con 0x7f9c400cede0 2023-07-13T22:23:41.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.397+0000 7f9c44d99700 1 -- 172.21.15.60:0/1373543863 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38012430 con 0x7f9c400cede0 2023-07-13T22:23:41.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.397+0000 7f9c47fff700 1 -- 172.21.15.60:0/1373543863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400cede0 msgr2=0x7f9c400cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.397+0000 7f9c47fff700 1 --2- 172.21.15.60:0/1373543863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400cede0 0x7f9c400cfb80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009d90 tx=0x7f9c3800b330 comp rx=0 tx=0).stop 2023-07-13T22:23:41.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.398+0000 7f9c47fff700 1 -- 172.21.15.60:0/1373543863 shutdown_connections 2023-07-13T22:23:41.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.398+0000 7f9c47fff700 1 --2- 172.21.15.60:0/1373543863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400cede0 0x7f9c400cfb80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.398+0000 7f9c47fff700 1 --2- 172.21.15.60:0/1373543863 >> v2:172.21.15.105:3300/0 conn(0x7f9c400b8200 0x7f9c400cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.398+0000 7f9c47fff700 1 -- 172.21.15.60:0/1373543863 >> 172.21.15.60:0/1373543863 conn(0x7f9c4006b2f0 msgr2=0x7f9c4006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:41.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.398+0000 7f9c47fff700 1 -- 172.21.15.60:0/1373543863 shutdown_connections 2023-07-13T22:23:41.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.398+0000 7f9c47fff700 1 -- 172.21.15.60:0/1373543863 wait complete. 2023-07-13T22:23:41.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.399+0000 7f9c47fff700 1 Processor -- start 2023-07-13T22:23:41.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.400+0000 7f9c47fff700 1 -- start start 2023-07-13T22:23:41.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c47fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c400b1ac0 0x7f9c401e25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c47fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400b8200 0x7f9c4015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c47fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400cede0 0x7f9c401e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c47fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401e1a80 con 0x7f9c400cede0 2023-07-13T22:23:41.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c47fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401e2b20 con 0x7f9c400b1ac0 2023-07-13T22:23:41.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c47fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401e2d60 con 0x7f9c400b8200 2023-07-13T22:23:41.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c4659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400cede0 0x7f9c401e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c4659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400cede0 0x7f9c401e37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55990/0 (socket says 172.21.15.60:55990) 2023-07-13T22:23:41.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c4659c700 1 -- 172.21.15.60:0/1802741730 learned_addr learned my addr 172.21.15.60:0/1802741730 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:41.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c45d9b700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c400b1ac0 0x7f9c401e25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c4559a700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400b8200 0x7f9c4015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c4659c700 1 -- 172.21.15.60:0/1802741730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400b8200 msgr2=0x7f9c4015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c4659c700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400b8200 0x7f9c4015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c4659c700 1 -- 172.21.15.60:0/1802741730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c400b1ac0 msgr2=0x7f9c401e25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c4659c700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c400b1ac0 0x7f9c401e25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.401+0000 7f9c4659c700 1 -- 172.21.15.60:0/1802741730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c38009a70 con 0x7f9c400cede0 2023-07-13T22:23:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.402+0000 7f9c4659c700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400cede0 0x7f9c401e37c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9c38014040 tx=0x7f9c38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.402+0000 7f9c367fc700 1 -- 172.21.15.60:0/1802741730 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38012430 con 0x7f9c400cede0 2023-07-13T22:23:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.402+0000 7f9c367fc700 1 -- 172.21.15.60:0/1802741730 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c38020070 con 0x7f9c400cede0 2023-07-13T22:23:41.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.402+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c401e3040 con 0x7f9c400cede0 2023-07-13T22:23:41.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.402+0000 7f9c367fc700 1 -- 172.21.15.60:0/1802741730 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38003ea0 con 0x7f9c400cede0 2023-07-13T22:23:41.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.402+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c400c6760 con 0x7f9c400cede0 2023-07-13T22:23:41.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.403+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4004e210 con 0x7f9c400cede0 2023-07-13T22:23:41.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.404+0000 7f9c367fc700 1 -- 172.21.15.60:0/1802741730 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9c38004040 con 0x7f9c400cede0 2023-07-13T22:23:41.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.405+0000 7f9c367fc700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c2406ace0 0x7f9c2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.405+0000 7f9c367fc700 1 -- 172.21.15.60:0/1802741730 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c38022070 con 0x7f9c400cede0 2023-07-13T22:23:41.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.405+0000 7f9c45d9b700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c2406ace0 0x7f9c2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.406+0000 7f9c45d9b700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c2406ace0 0x7f9c2406d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009bd0 tx=0x7f9c3c0092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:41.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.409+0000 7f9c367fc700 1 -- 172.21.15.60:0/1802741730 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9c38056610 con 0x7f9c400cede0 2023-07-13T22:23:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.617+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c400b4f70 con 0x7f9c400cede0 2023-07-13T22:23:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.617+0000 7f9c367fc700 1 -- 172.21.15.60:0/1802741730 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c38059c30 con 0x7f9c400cede0 2023-07-13T22:23:41.601 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:23:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c2406ace0 msgr2=0x7f9c2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c2406ace0 0x7f9c2406d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009bd0 tx=0x7f9c3c0092c0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400cede0 msgr2=0x7f9c401e37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400cede0 0x7f9c401e37c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9c38014040 tx=0x7f9c38004ad0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 shutdown_connections 2023-07-13T22:23:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9c2406ace0 0x7f9c2406d190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400cede0 0x7f9c401e37c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c400b1ac0 0x7f9c401e25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 --2- 172.21.15.60:0/1802741730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c400b8200 0x7f9c4015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 >> 172.21.15.60:0/1802741730 conn(0x7f9c4006b2f0 msgr2=0x7f9c400d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 shutdown_connections 2023-07-13T22:23:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.620+0000 7f9c47fff700 1 -- 172.21.15.60:0/1802741730 wait complete. 2023-07-13T22:23:41.621 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:23:41.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.841+0000 7f3a273ff700 1 Processor -- start 2023-07-13T22:23:41.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.841+0000 7f3a273ff700 1 -- start start 2023-07-13T22:23:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.842+0000 7f3a273ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3a2015a820 0x7f3a2015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.842+0000 7f3a273ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a200c8d40 0x7f3a200c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.843+0000 7f3a273ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a20070270 con 0x7f3a20154950 2023-07-13T22:23:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.843+0000 7f3a273ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a200713d0 con 0x7f3a2015a820 2023-07-13T22:23:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.843+0000 7f3a273ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200c96a0 con 0x7f3a200c8d40 2023-07-13T22:23:41.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.843+0000 7f3a2519b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3a20154950 0x7f3a20070cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55268/0 (socket says 172.21.15.60:55268) 2023-07-13T22:23:41.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.843+0000 7f3a2519b700 1 -- 172.21.15.60:0/1169218968 learned_addr learned my addr 172.21.15.60:0/1169218968 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:41.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.843+0000 7f3a2599c700 1 --2- 172.21.15.60:0/1169218968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a200c8d40 0x7f3a200c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.843+0000 7f3a2499a700 1 --2- 172.21.15.60:0/1169218968 >> v2:172.21.15.105:3300/0 conn(0x7f3a2015a820 0x7f3a2015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.843+0000 7f3a17fff700 1 -- 172.21.15.60:0/1169218968 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266903081 0 0) 0x7f3a20070270 con 0x7f3a20154950 2023-07-13T22:23:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.843+0000 7f3a17fff700 1 -- 172.21.15.60:0/1169218968 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a0c0035b0 con 0x7f3a20154950 2023-07-13T22:23:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a2599c700 1 -- 172.21.15.60:0/1169218968 >> v2:172.21.15.105:3300/0 conn(0x7f3a2015a820 msgr2=0x7f3a2015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a2599c700 1 --2- 172.21.15.60:0/1169218968 >> v2:172.21.15.105:3300/0 conn(0x7f3a2015a820 0x7f3a2015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a2599c700 1 -- 172.21.15.60:0/1169218968 >> v1:172.21.15.60:6789/0 conn(0x7f3a20154950 legacy=0x7f3a20070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a17fff700 1 -- 172.21.15.60:0/1169218968 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23788435 0 0) 0x7f3a0c0035b0 con 0x7f3a20154950 2023-07-13T22:23:41.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a2599c700 1 -- 172.21.15.60:0/1169218968 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2015b2e0 con 0x7f3a200c8d40 2023-07-13T22:23:41.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a2499a700 1 --2- 172.21.15.60:0/1169218968 >> v2:172.21.15.105:3300/0 conn(0x7f3a2015a820 0x7f3a2015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:41.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a2599c700 1 --2- 172.21.15.60:0/1169218968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a200c8d40 0x7f3a200c9160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009d90 tx=0x7f3a1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab6c20ede2481a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:41.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a17fff700 1 -- 172.21.15.60:0/1169218968 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800e070 con 0x7f3a200c8d40 2023-07-13T22:23:41.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a17fff700 1 -- 172.21.15.60:0/1169218968 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18003a20 con 0x7f3a200c8d40 2023-07-13T22:23:41.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.844+0000 7f3a17fff700 1 -- 172.21.15.60:0/1169218968 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c001670 con 0x7f3a200c8d40 2023-07-13T22:23:41.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.845+0000 7f3a273ff700 1 -- 172.21.15.60:0/1169218968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a200c8d40 msgr2=0x7f3a200c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.845+0000 7f3a273ff700 1 --2- 172.21.15.60:0/1169218968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a200c8d40 0x7f3a200c9160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009d90 tx=0x7f3a1800b330 comp rx=0 tx=0).stop 2023-07-13T22:23:41.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.845+0000 7f3a273ff700 1 -- 172.21.15.60:0/1169218968 shutdown_connections 2023-07-13T22:23:41.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.845+0000 7f3a273ff700 1 --2- 172.21.15.60:0/1169218968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a200c8d40 0x7f3a200c9160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.845+0000 7f3a273ff700 1 --2- 172.21.15.60:0/1169218968 >> v2:172.21.15.105:3300/0 conn(0x7f3a2015a820 0x7f3a2015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.845+0000 7f3a273ff700 1 -- 172.21.15.60:0/1169218968 >> 172.21.15.60:0/1169218968 conn(0x7f3a200cf410 msgr2=0x7f3a200cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:41.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.845+0000 7f3a273ff700 1 -- 172.21.15.60:0/1169218968 shutdown_connections 2023-07-13T22:23:41.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.845+0000 7f3a273ff700 1 -- 172.21.15.60:0/1169218968 wait complete. 2023-07-13T22:23:41.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.847+0000 7f3a273ff700 1 Processor -- start 2023-07-13T22:23:41.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.847+0000 7f3a273ff700 1 -- start start 2023-07-13T22:23:41.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a273ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a200c8d40 0x7f3a201503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a273ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a20154950 0x7f3a201f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a273ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2015a820 0x7f3a200c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a273ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201e3f60 con 0x7f3a200c8d40 2023-07-13T22:23:41.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a273ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201f0a30 con 0x7f3a2015a820 2023-07-13T22:23:41.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a273ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200b3180 con 0x7f3a20154950 2023-07-13T22:23:41.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a2519b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a200c8d40 0x7f3a201503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a2519b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a200c8d40 0x7f3a201503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55992/0 (socket says 172.21.15.60:55992) 2023-07-13T22:23:41.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a2519b700 1 -- 172.21.15.60:0/1070632945 learned_addr learned my addr 172.21.15.60:0/1070632945 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:41.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.848+0000 7f3a2499a700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a20154950 0x7f3a201f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a2599c700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2015a820 0x7f3a200c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a2519b700 1 -- 172.21.15.60:0/1070632945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a20154950 msgr2=0x7f3a201f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a2519b700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a20154950 0x7f3a201f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a2519b700 1 -- 172.21.15.60:0/1070632945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2015a820 msgr2=0x7f3a200c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:41.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a2519b700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2015a820 0x7f3a200c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:41.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a2519b700 1 -- 172.21.15.60:0/1070632945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100094a0 con 0x7f3a200c8d40 2023-07-13T22:23:41.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a2599c700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2015a820 0x7f3a200c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:41.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a2519b700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a200c8d40 0x7f3a201503c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0011e0 tx=0x7f3a1c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:41.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a15ffb700 1 -- 172.21.15.60:0/1070632945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c004c80 con 0x7f3a200c8d40 2023-07-13T22:23:41.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a15ffb700 1 -- 172.21.15.60:0/1070632945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c004e00 con 0x7f3a200c8d40 2023-07-13T22:23:41.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a15ffb700 1 -- 172.21.15.60:0/1070632945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c007310 con 0x7f3a200c8d40 2023-07-13T22:23:41.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.849+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a18009a70 con 0x7f3a200c8d40 2023-07-13T22:23:41.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.850+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a200b3740 con 0x7f3a200c8d40 2023-07-13T22:23:41.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.851+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2004e280 con 0x7f3a200c8d40 2023-07-13T22:23:41.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.851+0000 7f3a15ffb700 1 -- 172.21.15.60:0/1070632945 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a1c007470 con 0x7f3a200c8d40 2023-07-13T22:23:41.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.852+0000 7f3a15ffb700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0406ace0 0x7f3a0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:41.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.852+0000 7f3a15ffb700 1 -- 172.21.15.60:0/1070632945 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a1c089290 con 0x7f3a200c8d40 2023-07-13T22:23:41.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.853+0000 7f3a2499a700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0406ace0 0x7f3a0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:41.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.853+0000 7f3a2499a700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0406ace0 0x7f3a0406d190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3a20154360 tx=0x7f3a10009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:41.857+0000 7f3a15ffb700 1 -- 172.21.15.60:0/1070632945 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3a1c054340 con 0x7f3a200c8d40 2023-07-13T22:23:42.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.078+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3a200cbc20 con 0x7f3a200c8d40 2023-07-13T22:23:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.079+0000 7f3a15ffb700 1 -- 172.21.15.60:0/1070632945 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3a1c057960 con 0x7f3a200c8d40 2023-07-13T22:23:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0406ace0 msgr2=0x7f3a0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0406ace0 0x7f3a0406d190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3a20154360 tx=0x7f3a10009210 comp rx=0 tx=0).stop 2023-07-13T22:23:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a200c8d40 msgr2=0x7f3a201503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a200c8d40 0x7f3a201503c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0011e0 tx=0x7f3a1c00c9e0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 shutdown_connections 2023-07-13T22:23:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0406ace0 0x7f3a0406d190 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a200c8d40 0x7f3a201503c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2015a820 0x7f3a200c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 --2- 172.21.15.60:0/1070632945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a20154950 0x7f3a201f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 >> 172.21.15.60:0/1070632945 conn(0x7f3a200cf410 msgr2=0x7f3a2015dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:42.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 shutdown_connections 2023-07-13T22:23:42.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.081+0000 7f3a273ff700 1 -- 172.21.15.60:0/1070632945 wait complete. 2023-07-13T22:23:42.066 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:23:42.082 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:23:42.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:42.083 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:42.151 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[132428]: fuse finished with error 0 and tester_r 0 2023-07-13T22:23:42.156 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:23:42.205 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:23:42.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:42.206 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:23:42.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:23:42.257 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:23:42.373 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2023-07-13T22:23:42.373 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2023-07-13T22:23:42.374 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:23:42.374 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2023-07-13T22:23:42.374 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.374 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2023-07-13T22:23:42.374 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.374 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2023-07-13T22:23:42.374 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.374 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2023-07-13T22:23:42.375 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2023-07-13T22:23:42.375 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.375 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2023-07-13T22:23:42.375 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2023-07-13T22:23:42.375 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.375 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:23:42.375 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2023-07-13T22:23:42.376 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.376 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:23:42.376 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:23:42.376 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.376 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2023-07-13T22:23:42.376 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.376 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2023-07-13T22:23:42.377 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2023-07-13T22:23:42.377 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:23:42.377 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:23:42.377 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2023-07-13T22:23:42.377 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:23:42.377 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2023-07-13T22:23:42.377 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:23:42.377 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:23:42.378 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2023-07-13T22:23:42.378 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.378 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:23:42.378 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.378 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2023-07-13T22:23:42.378 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.378 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2023-07-13T22:23:42.378 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.379 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2023-07-13T22:23:42.379 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2023-07-13T22:23:42.379 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2023-07-13T22:23:42.379 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:23:42.379 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:23:42.379 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.379 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2023-07-13T22:23:42.379 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2023-07-13T22:23:42.380 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.380 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2023-07-13T22:23:42.380 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2023-07-13T22:23:42.380 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:23:42.380 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.381 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2023-07-13T22:23:42.381 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2023-07-13T22:23:42.381 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.381 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2023-07-13T22:23:42.381 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2023-07-13T22:23:42.381 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2023-07-13T22:23:42.381 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.381 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2023-07-13T22:23:42.382 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2023-07-13T22:23:42.382 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.382 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:23:42.382 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.382 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2023-07-13T22:23:42.382 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.382 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2023-07-13T22:23:42.382 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.383 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:23:42.383 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2023-07-13T22:23:42.383 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.383 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2023-07-13T22:23:42.383 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2023-07-13T22:23:42.383 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2023-07-13T22:23:42.383 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2023-07-13T22:23:42.383 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2023-07-13T22:23:42.384 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2023-07-13T22:23:42.384 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2023-07-13T22:23:42.384 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2023-07-13T22:23:42.384 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2023-07-13T22:23:42.384 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:23:42.384 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2023-07-13T22:23:42.384 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2023-07-13T22:23:42.384 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:23:42.385 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2023-07-13T22:23:42.385 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:23:42.385 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:23:42.385 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2023-07-13T22:23:42.385 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:23:42.385 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.385 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2023-07-13T22:23:42.385 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:23:42.386 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2023-07-13T22:23:42.386 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:23:42.386 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:23:42.386 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:23:42.386 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:23:42.386 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2023-07-13T22:23:42.386 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.387 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2023-07-13T22:23:42.387 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:23:42.387 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:23:42.387 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2023-07-13T22:23:42.387 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:23:42.387 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2023-07-13T22:23:42.387 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2023-07-13T22:23:42.387 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:23:42.388 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2023-07-13T22:23:42.388 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2023-07-13T22:23:42.388 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:23:42.388 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.388 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:23:42.388 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:23:42.388 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:23:42.389 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.389 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:23:42.389 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:23:42.389 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:23:42.389 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.389 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2023-07-13T22:23:42.389 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:23:42.390 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.390 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:23:42.390 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2023-07-13T22:23:42.390 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:42.390 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2023-07-13T22:23:42.390 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2023-07-13T22:23:42.390 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2023-07-13T22:23:42.391 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2023-07-13T22:23:42.391 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:23:42.391 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:23:42.391 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2023-07-13T22:23:42.391 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2023-07-13T22:23:42.391 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:23:42.391 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2023-07-13T22:23:42.392 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:23:42.392 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:23:42.392 INFO:teuthology.orchestra.run.smithi060.stdout: client mountpoint = /subdir 2023-07-13T22:23:42.392 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2023-07-13T22:23:42.392 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2023-07-13T22:23:42.392 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2023-07-13T22:23:42.392 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-07-13T22:23:42.392 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-07-13T22:23:42.393 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-07-13T22:23:42.393 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-07-13T22:23:42.393 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2023-07-13T22:23:42.393 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2023-07-13T22:23:42.393 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2023-07-13T22:23:42.393 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:23:42.393 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2023-07-13T22:23:42.394 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2023-07-13T22:23:42.394 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2023-07-13T22:23:42.394 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2023-07-13T22:23:42.394 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:23:42.394 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:23:42.394 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:23:42.395 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-07-13T22:23:42.395 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-07-13T22:23:42.395 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:23:42.395 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-07-13T22:23:42.395 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.395 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-07-13T22:23:42.396 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.396 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-07-13T22:23:42.396 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.396 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-07-13T22:23:42.396 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-07-13T22:23:42.396 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.396 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-07-13T22:23:42.397 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-07-13T22:23:42.397 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.397 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:23:42.397 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-07-13T22:23:42.397 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.397 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:23:42.397 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:23:42.397 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.398 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-07-13T22:23:42.398 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.398 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-07-13T22:23:42.398 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-07-13T22:23:42.398 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:23:42.398 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:23:42.398 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-07-13T22:23:42.398 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:23:42.399 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-07-13T22:23:42.399 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:23:42.399 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:23:42.399 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-07-13T22:23:42.399 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.399 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:23:42.399 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.399 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-07-13T22:23:42.400 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.400 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-07-13T22:23:42.400 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.400 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-07-13T22:23:42.400 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-07-13T22:23:42.400 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-07-13T22:23:42.400 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:23:42.400 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:23:42.401 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.401 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-07-13T22:23:42.401 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-07-13T22:23:42.401 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.401 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-07-13T22:23:42.401 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2023-07-13T22:23:42.401 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:23:42.402 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.402 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-07-13T22:23:42.402 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-07-13T22:23:42.402 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.402 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-07-13T22:23:42.402 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-07-13T22:23:42.402 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-07-13T22:23:42.402 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.403 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-07-13T22:23:42.403 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-07-13T22:23:42.403 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.403 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:23:42.403 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.403 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-07-13T22:23:42.403 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.403 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-07-13T22:23:42.404 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.404 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:23:42.404 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-07-13T22:23:42.404 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.404 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-07-13T22:23:42.404 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-07-13T22:23:42.404 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2023-07-13T22:23:42.405 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2023-07-13T22:23:42.405 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2023-07-13T22:23:42.405 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2023-07-13T22:23:42.405 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2023-07-13T22:23:42.405 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2023-07-13T22:23:42.405 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2023-07-13T22:23:42.405 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:23:42.405 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-07-13T22:23:42.406 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2023-07-13T22:23:42.406 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:23:42.406 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2023-07-13T22:23:42.406 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:23:42.406 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:23:42.406 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2023-07-13T22:23:42.406 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:23:42.406 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.407 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-07-13T22:23:42.407 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:23:42.407 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-07-13T22:23:42.407 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:23:42.407 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:23:42.407 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:23:42.407 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:23:42.407 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-07-13T22:23:42.408 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.408 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-07-13T22:23:42.408 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:23:42.408 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:23:42.408 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-07-13T22:23:42.408 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:23:42.409 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-07-13T22:23:42.409 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-07-13T22:23:42.409 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:23:42.409 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-07-13T22:23:42.409 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-07-13T22:23:42.409 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:23:42.409 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.410 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:23:42.410 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:23:42.410 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:23:42.410 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.410 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:23:42.410 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:23:42.410 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:23:42.410 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.411 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-07-13T22:23:42.411 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:23:42.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.411 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:23:42.411 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2023-07-13T22:23:42.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:23:42.411 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-07-13T22:23:42.412 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-07-13T22:23:42.412 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-07-13T22:23:42.412 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-07-13T22:23:42.412 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:23:42.412 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:23:42.412 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2023-07-13T22:23:42.412 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2023-07-13T22:23:42.413 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:23:42.413 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2023-07-13T22:23:42.413 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:23:42.413 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:23:42.413 INFO:teuthology.orchestra.run.smithi105.stdout: client mountpoint = /subdir 2023-07-13T22:23:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-07-13T22:23:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2023-07-13T22:23:42.413 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2023-07-13T22:23:42.414 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-07-13T22:23:42.414 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-07-13T22:23:42.414 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-07-13T22:23:42.414 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-07-13T22:23:42.414 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2023-07-13T22:23:42.414 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2023-07-13T22:23:42.414 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2023-07-13T22:23:42.414 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:23:42.415 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2023-07-13T22:23:42.415 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2023-07-13T22:23:42.415 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2023-07-13T22:23:42.415 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2023-07-13T22:23:42.415 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:23:42.415 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:23:42.415 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:23:42.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:23:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.636+0000 7fb3a17ff700 1 Processor -- start 2023-07-13T22:23:42.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.636+0000 7fb3a17ff700 1 -- start start 2023-07-13T22:23:42.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.637+0000 7fb3a17ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c06c7a0 0x7fb39c06cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.637+0000 7fb3a17ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb39c1597e0 0x7fb39c0ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.637+0000 7fb3a17ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39c0c98a0 con 0x7fb39c15dc50 2023-07-13T22:23:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb3a17ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c06d0b0 con 0x7fb39c1597e0 2023-07-13T22:23:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb3a17ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c0c9b00 con 0x7fb39c06c7a0 2023-07-13T22:23:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb39a7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb39c15dc50 0x7fb39c0c9190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55278/0 (socket says 172.21.15.60:55278) 2023-07-13T22:23:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb39a7fc700 1 -- 172.21.15.60:0/3511027563 learned_addr learned my addr 172.21.15.60:0/3511027563 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb39b7fe700 1 --2- 172.21.15.60:0/3511027563 >> v2:172.21.15.60:3300/0 conn(0x7fb39c1597e0 0x7fb39c0ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:42.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb39affd700 1 --2- 172.21.15.60:0/3511027563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c06c7a0 0x7fb39c06cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:42.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb399ffb700 1 -- 172.21.15.60:0/3511027563 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717324460 0 0) 0x7fb39c0c98a0 con 0x7fb39c15dc50 2023-07-13T22:23:42.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb399ffb700 1 -- 172.21.15.60:0/3511027563 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3840057c0 con 0x7fb39c15dc50 2023-07-13T22:23:42.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb39affd700 1 -- 172.21.15.60:0/3511027563 >> v2:172.21.15.60:3300/0 conn(0x7fb39c1597e0 msgr2=0x7fb39c0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:42.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb39affd700 1 --2- 172.21.15.60:0/3511027563 >> v2:172.21.15.60:3300/0 conn(0x7fb39c1597e0 0x7fb39c0ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb39affd700 1 -- 172.21.15.60:0/3511027563 >> v1:172.21.15.60:6789/0 conn(0x7fb39c15dc50 legacy=0x7fb39c0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:42.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb39affd700 1 -- 172.21.15.60:0/3511027563 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c159d70 con 0x7fb39c06c7a0 2023-07-13T22:23:42.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.638+0000 7fb39b7fe700 1 --2- 172.21.15.60:0/3511027563 >> v2:172.21.15.60:3300/0 conn(0x7fb39c1597e0 0x7fb39c0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.639+0000 7fb39affd700 1 --2- 172.21.15.60:0/3511027563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c06c7a0 0x7fb39c06cb70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009840 tx=0x7fb38c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5947ccbe0c87cb19 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.639+0000 7fb399ffb700 1 -- 172.21.15.60:0/3511027563 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c011830 con 0x7fb39c06c7a0 2023-07-13T22:23:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.639+0000 7fb399ffb700 1 -- 172.21.15.60:0/3511027563 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c011990 con 0x7fb39c06c7a0 2023-07-13T22:23:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.639+0000 7fb399ffb700 1 -- 172.21.15.60:0/3511027563 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c011d00 con 0x7fb39c06c7a0 2023-07-13T22:23:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.639+0000 7fb3a17ff700 1 -- 172.21.15.60:0/3511027563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c06c7a0 msgr2=0x7fb39c06cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.639+0000 7fb3a17ff700 1 --2- 172.21.15.60:0/3511027563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c06c7a0 0x7fb39c06cb70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009840 tx=0x7fb38c009b50 comp rx=0 tx=0).stop 2023-07-13T22:23:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.639+0000 7fb3a17ff700 1 -- 172.21.15.60:0/3511027563 shutdown_connections 2023-07-13T22:23:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.640+0000 7fb3a17ff700 1 --2- 172.21.15.60:0/3511027563 >> v2:172.21.15.60:3300/0 conn(0x7fb39c1597e0 0x7fb39c0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.640+0000 7fb3a17ff700 1 --2- 172.21.15.60:0/3511027563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c06c7a0 0x7fb39c06cb70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.640+0000 7fb3a17ff700 1 -- 172.21.15.60:0/3511027563 >> 172.21.15.60:0/3511027563 conn(0x7fb39c0cb390 msgr2=0x7fb39c0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.640+0000 7fb3a17ff700 1 -- 172.21.15.60:0/3511027563 shutdown_connections 2023-07-13T22:23:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.640+0000 7fb3a17ff700 1 -- 172.21.15.60:0/3511027563 wait complete. 2023-07-13T22:23:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.641+0000 7fb3a17ff700 1 Processor -- start 2023-07-13T22:23:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.642+0000 7fb3a17ff700 1 -- start start 2023-07-13T22:23:42.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.642+0000 7fb3a17ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39c06c7a0 0x7fb39c0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.642+0000 7fb3a17ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c1597e0 0x7fb39c1471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.642+0000 7fb3a17ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c15dc50 0x7fb39c1f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.642+0000 7fb3a17ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c1580d0 con 0x7fb39c1597e0 2023-07-13T22:23:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.642+0000 7fb3a17ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c147700 con 0x7fb39c06c7a0 2023-07-13T22:23:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.642+0000 7fb3a17ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c0c6f60 con 0x7fb39c15dc50 2023-07-13T22:23:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c1597e0 0x7fb39c1471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c1597e0 0x7fb39c1471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56000/0 (socket says 172.21.15.60:56000) 2023-07-13T22:23:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39a7fc700 1 -- 172.21.15.60:0/1903899246 learned_addr learned my addr 172.21.15.60:0/1903899246 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39affd700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39c06c7a0 0x7fb39c0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39b7fe700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c15dc50 0x7fb39c1f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39a7fc700 1 -- 172.21.15.60:0/1903899246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c15dc50 msgr2=0x7fb39c1f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39a7fc700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c15dc50 0x7fb39c1f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39a7fc700 1 -- 172.21.15.60:0/1903899246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39c06c7a0 msgr2=0x7fb39c0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39a7fc700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39c06c7a0 0x7fb39c0c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39a7fc700 1 -- 172.21.15.60:0/1903899246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb394009a70 con 0x7fb39c1597e0 2023-07-13T22:23:42.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.643+0000 7fb39a7fc700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c1597e0 0x7fb39c1471c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c400 tx=0x7fb39000c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:42.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.644+0000 7fb3837fe700 1 -- 172.21.15.60:0/1903899246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390005b70 con 0x7fb39c1597e0 2023-07-13T22:23:42.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.644+0000 7fb3837fe700 1 -- 172.21.15.60:0/1903899246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb390005cf0 con 0x7fb39c1597e0 2023-07-13T22:23:42.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.644+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0094a0 con 0x7fb39c1597e0 2023-07-13T22:23:42.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.644+0000 7fb3837fe700 1 -- 172.21.15.60:0/1903899246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390007310 con 0x7fb39c1597e0 2023-07-13T22:23:42.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.644+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c0c7520 con 0x7fb39c1597e0 2023-07-13T22:23:42.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.645+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39c072890 con 0x7fb39c1597e0 2023-07-13T22:23:42.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.646+0000 7fb3837fe700 1 -- 172.21.15.60:0/1903899246 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb390007470 con 0x7fb39c1597e0 2023-07-13T22:23:42.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.647+0000 7fb3837fe700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb38806ace0 0x7fb38806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:42.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.647+0000 7fb3837fe700 1 -- 172.21.15.60:0/1903899246 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb3900887d0 con 0x7fb39c1597e0 2023-07-13T22:23:42.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.647+0000 7fb39affd700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb38806ace0 0x7fb38806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:42.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.648+0000 7fb39affd700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb38806ace0 0x7fb38806d190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0090b0 tx=0x7fb38c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.651+0000 7fb3837fe700 1 -- 172.21.15.60:0/1903899246 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb390053940 con 0x7fb39c1597e0 2023-07-13T22:23:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.900+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb39c0cbfa0 con 0x7fb39c1597e0 2023-07-13T22:23:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.901+0000 7fb3837fe700 1 -- 172.21.15.60:0/1903899246 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb390017060 con 0x7fb39c1597e0 2023-07-13T22:23:42.885 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:23:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb38806ace0 msgr2=0x7fb38806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb38806ace0 0x7fb38806d190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0090b0 tx=0x7fb38c008030 comp rx=0 tx=0).stop 2023-07-13T22:23:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c1597e0 msgr2=0x7fb39c1471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c1597e0 0x7fb39c1471c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c400 tx=0x7fb39000c710 comp rx=0 tx=0).stop 2023-07-13T22:23:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 shutdown_connections 2023-07-13T22:23:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb38806ace0 0x7fb38806d190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb39c15dc50 0x7fb39c1f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb39c06c7a0 0x7fb39c0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 --2- 172.21.15.60:0/1903899246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c1597e0 0x7fb39c1471c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.903+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 >> 172.21.15.60:0/1903899246 conn(0x7fb39c0cb390 msgr2=0x7fb39c0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.904+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 shutdown_connections 2023-07-13T22:23:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:42.904+0000 7fb3a17ff700 1 -- 172.21.15.60:0/1903899246 wait complete. 2023-07-13T22:23:42.905 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:23:42.905 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:23:42.905 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:23:42.905 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:23:42.906 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:42.906 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:23:42.906 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:23:42.906 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:23:42.907 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:23:42.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:42.907 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:23:42.954 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:23:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:23:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:23:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:23:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:42.955 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:23:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:23:42.956 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:23:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:23:42.956 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:23:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:23:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:23:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41550sec preferred_lft 41550sec 2023-07-13T22:23:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:23:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:42.957 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:23:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:23:42.957 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:23:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:23:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:23:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:23:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:42.958 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:23:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:23:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:23:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:42.959 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:23:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:23:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:23:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:23:42.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:42.960 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:23:43.028 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:23:43.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:43.032 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:23:43.081 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:23:43.081 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:23:43.081 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:23:43.081 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:23:43.081 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:23:43.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:43.082 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:23:43.082 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:23:43.083 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:23:43.083 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:23:43.083 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:23:43.084 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:23:43.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:43.257 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:23:43.305 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:23:43.305 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:23:43.306 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:23:43.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:43.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:23:43.378 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:23:43.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:23:43.380 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:43.426 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:23:43.427 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:23:43.427 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:43.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:23:43.585 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:23:43.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:23:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:23:43.657 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:23:43.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:43.658 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:23:43.704 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:23:43.705 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:23:43.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:23:43.783 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:23:43.783 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:23:43.858 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:23:43.860 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:23:43.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:43.861 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:23:43.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:23:43.901+0000 7f0fb6a895c0 -1 init, newargv = 0x55bbddc0f0e0 newargc=15 2023-07-13T22:23:43.885 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[134050]: starting ceph client 2023-07-13T22:23:43.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[134050]: starting fuse 2023-07-13T22:23:43.913 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:23:43.913 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:23:43.914 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:43.914 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:43.914 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:43.915 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:43.915 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:43.915 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:43.916 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:43.916 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:43.916 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:43.917 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:43.917 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:43.917 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:43.917 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:43.918 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:43.918 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:43.918 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:43.919 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:43.919 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:43.919 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:43.920 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:43.920 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:43.920 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:43.920 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:43.921 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:43.921 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:43.921 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:43.922 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:44.100 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:44.277 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:44.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:44.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134050.asok status 2023-07-13T22:23:44.514 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:44.514 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:23:44.514 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:23:44.514 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:23:44.515 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:23:44.515 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:23:44.515 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:23:44.515 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "134050", 2023-07-13T22:23:44.515 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/subdir" 2023-07-13T22:23:44.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:44.516 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:23:44.516 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:23:44.516 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6063, 2023-07-13T22:23:44.516 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:23:44.516 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:23:44.516 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:23:44.516 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6063 2023-07-13T22:23:44.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:44.517 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:44.517 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:44.517 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:44.517 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3000387442 2023-07-13T22:23:44.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:44.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:44.517 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:44.518 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:44.518 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:44.518 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3000387442 2023-07-13T22:23:44.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:44.518 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6063 192.168.0.1:0/3000387442", 2023-07-13T22:23:44.518 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3000387442", 2023-07-13T22:23:44.518 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 2, 2023-07-13T22:23:44.518 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 24, 2023-07-13T22:23:44.519 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 50, 2023-07-13T22:23:44.519 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:23:44.519 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:23:44.519 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:23:44.519 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:44.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:44.529 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:23:44.529 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:44.576 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:23:44.576 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:44.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:44.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:44.648 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:44.649 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:44.649 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2023-07-13T22:23:44.649 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:44.649 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/.").st_ino) 2023-07-13T22:23:44.650 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:44.765 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:23:44.950 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:44.950 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:44.951 DEBUG:teuthology.orchestra.run.smithi060:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2023-07-13T22:23:44.952 DEBUG:teuthology.orchestra.run.smithi060:> for n in range(0, 1000): 2023-07-13T22:23:44.952 DEBUG:teuthology.orchestra.run.smithi060:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir/dir{0}".format(n)) 2023-07-13T22:23:44.953 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:47.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:47.294 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync) 2023-07-13T22:23:47.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:47.706 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:47.707 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:47.707 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:47.707 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:47.708 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:47.708 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:47.708 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:47.708 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:47.709 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:47.709 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:47.709 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:47.710 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:47.710 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:47.710 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:47.711 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:47.711 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:47.711 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:47.711 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:47.712 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:47.712 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:47.712 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:47.713 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:47.713 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:47.713 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:47.713 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:47.714 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:47.864 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:48.042 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:48.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:48.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134050.asok status 2023-07-13T22:23:48.279 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:48.279 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:23:48.279 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:23:48.279 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:23:48.279 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:23:48.279 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:23:48.280 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:23:48.280 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "134050", 2023-07-13T22:23:48.280 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/subdir" 2023-07-13T22:23:48.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:48.280 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 1001, 2023-07-13T22:23:48.280 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 1001, 2023-07-13T22:23:48.280 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6063, 2023-07-13T22:23:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:23:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:23:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:23:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6063 2023-07-13T22:23:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:48.282 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3000387442 2023-07-13T22:23:48.282 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:48.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:48.282 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:48.282 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:48.282 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:48.282 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3000387442 2023-07-13T22:23:48.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:48.283 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6063 192.168.0.1:0/3000387442", 2023-07-13T22:23:48.283 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3000387442", 2023-07-13T22:23:48.283 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1003, 2023-07-13T22:23:48.283 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 24, 2023-07-13T22:23:48.283 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 50, 2023-07-13T22:23:48.283 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 49, 2023-07-13T22:23:48.283 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:23:48.283 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:23:48.284 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:48.293 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:48.293 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:48.293 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:48.294 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:48.294 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:48.294 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:48.295 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:48.295 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:48.295 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:48.295 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:48.296 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:48.296 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:48.296 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:48.297 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:48.297 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:48.297 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:48.297 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:48.298 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:48.298 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:48.298 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:48.298 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:48.299 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:48.299 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:48.299 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:48.300 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:48.300 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:48.300 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:48.447 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:48.624 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:48.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:48.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134050.asok config set client_cache_size 100 2023-07-13T22:23:48.854 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:48.854 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "client_cache_size = '100' " 2023-07-13T22:23:48.854 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:48.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:48.868 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:48.869 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:48.869 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:48.869 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:48.870 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:48.870 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:48.870 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:48.870 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:48.871 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:48.871 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:48.871 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:48.872 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:48.872 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:48.872 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:48.873 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:48.873 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:48.873 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:48.874 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:48.874 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:48.874 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:48.875 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:48.875 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:48.875 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:48.875 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:48.876 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:48.876 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:49.024 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:49.201 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:49.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:49.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134050.asok status 2023-07-13T22:23:49.432 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:49.433 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:23:49.433 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:23:49.433 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:23:49.433 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:23:49.433 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:23:49.433 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:23:49.434 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "134050", 2023-07-13T22:23:49.434 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/subdir" 2023-07-13T22:23:49.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:49.434 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 1001, 2023-07-13T22:23:49.434 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 1, 2023-07-13T22:23:49.434 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6063, 2023-07-13T22:23:49.434 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:23:49.435 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:23:49.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:23:49.435 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6063 2023-07-13T22:23:49.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:49.435 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:49.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:49.435 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:49.435 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3000387442 2023-07-13T22:23:49.436 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:49.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:49.436 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:49.436 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:49.436 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:49.436 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3000387442 2023-07-13T22:23:49.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:49.436 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6063 192.168.0.1:0/3000387442", 2023-07-13T22:23:49.437 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3000387442", 2023-07-13T22:23:49.437 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1003, 2023-07-13T22:23:49.437 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 24, 2023-07-13T22:23:49.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 50, 2023-07-13T22:23:49.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 49, 2023-07-13T22:23:49.437 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:23:49.437 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:23:49.438 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:49.447 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 1001, 1 2023-07-13T22:23:49.447 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-07-13T22:23:54.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:23:54.448 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:23:54.448 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:23:54.449 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:23:54.449 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:23:54.449 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:54.450 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:23:54.450 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:23:54.450 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:23:54.451 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:23:54.451 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:54.451 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:23:54.451 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:23:54.452 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:23:54.452 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:23:54.452 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:54.453 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:23:54.453 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:23:54.453 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:23:54.453 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:23:54.454 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:23:54.454 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:23:54.454 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:23:54.455 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:23:54.455 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:23:54.455 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:23:54.455 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:23:54.614 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:54.791 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134050.asok 2023-07-13T22:23:54.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:54.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134050.asok status 2023-07-13T22:23:55.024 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:23:55.024 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:23:55.024 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:23:55.024 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:23:55.025 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:23:55.025 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:23:55.025 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:23:55.025 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "134050", 2023-07-13T22:23:55.025 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/subdir" 2023-07-13T22:23:55.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:55.025 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 100, 2023-07-13T22:23:55.026 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 1, 2023-07-13T22:23:55.026 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6063, 2023-07-13T22:23:55.026 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:23:55.026 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:23:55.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:23:55.026 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6063 2023-07-13T22:23:55.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:55.026 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:55.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:55.027 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:55.027 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3000387442 2023-07-13T22:23:55.027 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:23:55.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:55.027 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:23:55.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:23:55.027 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:23:55.028 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3000387442 2023-07-13T22:23:55.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:23:55.028 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6063 192.168.0.1:0/3000387442", 2023-07-13T22:23:55.028 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3000387442", 2023-07-13T22:23:55.028 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 102, 2023-07-13T22:23:55.028 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 24, 2023-07-13T22:23:55.028 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 50, 2023-07-13T22:23:55.028 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 49, 2023-07-13T22:23:55.029 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:23:55.029 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:23:55.029 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:23:55.038 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 100, 1 2023-07-13T22:23:55.038 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-07-13T22:23:55.039 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:23:55.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:23:55.137 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:23:55.241 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:23:55.241 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:23:55.242 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:23:55.288 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:23:55.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:23:55.514 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:23:55.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.549+0000 7f99369ff700 1 Processor -- start 2023-07-13T22:23:55.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.549+0000 7f99369ff700 1 -- start start 2023-07-13T22:23:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.551+0000 7f99369ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f99300aed40 0x7f9930069e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.551+0000 7f99369ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993006a3c0 0x7f993015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.551+0000 7f99369ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99300711a0 con 0x7f993015e730 2023-07-13T22:23:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.551+0000 7f99369ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993015ef60 con 0x7f99300aed40 2023-07-13T22:23:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.551+0000 7f99369ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993006a830 con 0x7f993006a3c0 2023-07-13T22:23:55.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.551+0000 7f992ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f99300aed40 0x7f9930069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:55.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.551+0000 7f992ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f99300aed40 0x7f9930069e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59096/0 (socket says 172.21.15.60:59096) 2023-07-13T22:23:55.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.551+0000 7f992ffff700 1 -- 172.21.15.60:0/654107128 learned_addr learned my addr 172.21.15.60:0/654107128 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.551+0000 7f992f7fe700 1 --2- 172.21.15.60:0/654107128 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993006a3c0 0x7f993015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.552+0000 7f992ffff700 1 -- 172.21.15.60:0/654107128 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993006a3c0 msgr2=0x7f993015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.552+0000 7f992ffff700 1 --2- 172.21.15.60:0/654107128 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993006a3c0 0x7f993015e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.552+0000 7f992ffff700 1 -- 172.21.15.60:0/654107128 >> v1:172.21.15.105:6789/0 conn(0x7f993015e730 legacy=0x7f99300b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.552+0000 7f992ffff700 1 -- 172.21.15.60:0/654107128 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930070870 con 0x7f99300aed40 2023-07-13T22:23:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.552+0000 7f992ffff700 1 --2- 172.21.15.60:0/654107128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300aed40 0x7f9930069e80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9920009920 tx=0x7f9920009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9633733bbed968f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.552+0000 7f992effd700 1 -- 172.21.15.60:0/654107128 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920011830 con 0x7f99300aed40 2023-07-13T22:23:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.552+0000 7f992effd700 1 -- 172.21.15.60:0/654107128 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9920011990 con 0x7f99300aed40 2023-07-13T22:23:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.552+0000 7f992effd700 1 -- 172.21.15.60:0/654107128 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920011d20 con 0x7f99300aed40 2023-07-13T22:23:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.553+0000 7f99369ff700 1 -- 172.21.15.60:0/654107128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300aed40 msgr2=0x7f9930069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.553+0000 7f99369ff700 1 --2- 172.21.15.60:0/654107128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300aed40 0x7f9930069e80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9920009920 tx=0x7f9920009c30 comp rx=0 tx=0).stop 2023-07-13T22:23:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.553+0000 7f99369ff700 1 -- 172.21.15.60:0/654107128 shutdown_connections 2023-07-13T22:23:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.553+0000 7f99369ff700 1 --2- 172.21.15.60:0/654107128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300aed40 0x7f9930069e80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.553+0000 7f99369ff700 1 --2- 172.21.15.60:0/654107128 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993006a3c0 0x7f993015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.553+0000 7f99369ff700 1 -- 172.21.15.60:0/654107128 >> 172.21.15.60:0/654107128 conn(0x7f993006bc20 msgr2=0x7f99300cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.553+0000 7f99369ff700 1 -- 172.21.15.60:0/654107128 shutdown_connections 2023-07-13T22:23:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.553+0000 7f99369ff700 1 -- 172.21.15.60:0/654107128 wait complete. 2023-07-13T22:23:55.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.556+0000 7f99369ff700 1 Processor -- start 2023-07-13T22:23:55.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.556+0000 7f99369ff700 1 -- start start 2023-07-13T22:23:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f99369ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993006a3c0 0x7f9930154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f99369ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300aed40 0x7f99301f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f99369ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015e730 0x7f99300b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f99369ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9930157180 con 0x7f993006a3c0 2023-07-13T22:23:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f99369ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9930157460 con 0x7f99300aed40 2023-07-13T22:23:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f99369ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9930157740 con 0x7f993015e730 2023-07-13T22:23:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993006a3c0 0x7f9930154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993006a3c0 0x7f9930154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59102/0 (socket says 172.21.15.60:59102) 2023-07-13T22:23:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992ffff700 1 -- 172.21.15.60:0/3010900801 learned_addr learned my addr 172.21.15.60:0/3010900801 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992f7fe700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300aed40 0x7f99301f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f9934f9c700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015e730 0x7f99300b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992ffff700 1 -- 172.21.15.60:0/3010900801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015e730 msgr2=0x7f99300b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992ffff700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015e730 0x7f99300b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992ffff700 1 -- 172.21.15.60:0/3010900801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300aed40 msgr2=0x7f99301f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992ffff700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300aed40 0x7f99301f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992ffff700 1 -- 172.21.15.60:0/3010900801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9920009550 con 0x7f993006a3c0 2023-07-13T22:23:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.557+0000 7f992f7fe700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300aed40 0x7f99301f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.558+0000 7f992ffff700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993006a3c0 0x7f9930154cf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9920009870 tx=0x7f9920004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.558+0000 7f992cff9700 1 -- 172.21.15.60:0/3010900801 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920004400 con 0x7f993006a3c0 2023-07-13T22:23:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.558+0000 7f992cff9700 1 -- 172.21.15.60:0/3010900801 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992001f070 con 0x7f993006a3c0 2023-07-13T22:23:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.558+0000 7f992cff9700 1 -- 172.21.15.60:0/3010900801 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99200038e0 con 0x7f993006a3c0 2023-07-13T22:23:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.558+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9930155e10 con 0x7f993006a3c0 2023-07-13T22:23:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.558+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9930157a20 con 0x7f993006a3c0 2023-07-13T22:23:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.559+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9930072890 con 0x7f993006a3c0 2023-07-13T22:23:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.560+0000 7f992cff9700 1 -- 172.21.15.60:0/3010900801 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9920021030 con 0x7f993006a3c0 2023-07-13T22:23:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.561+0000 7f992cff9700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991c06ace0 0x7f991c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.561+0000 7f992cff9700 1 -- 172.21.15.60:0/3010900801 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f992008a880 con 0x7f993006a3c0 2023-07-13T22:23:55.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.562+0000 7f992f7fe700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991c06ace0 0x7f991c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:55.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.562+0000 7f992f7fe700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991c06ace0 0x7f991c06d190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9930155d20 tx=0x7f9924008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.567+0000 7f992cff9700 1 -- 172.21.15.60:0/3010900801 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9920055a00 con 0x7f993006a3c0 2023-07-13T22:23:55.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.776+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9930069fa0 con 0x7f993006a3c0 2023-07-13T22:23:55.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.777+0000 7f992cff9700 1 -- 172.21.15.60:0/3010900801 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18353 (secure 0 0 0) 0x7f9920059020 con 0x7f993006a3c0 2023-07-13T22:23:55.760 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:55.760 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":51,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:44.502069+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.21","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:23:55.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.779+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991c06ace0 msgr2=0x7f991c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991c06ace0 0x7f991c06d190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9930155d20 tx=0x7f9924008040 comp rx=0 tx=0).stop 2023-07-13T22:23:55.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993006a3c0 msgr2=0x7f9930154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993006a3c0 0x7f9930154cf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9920009870 tx=0x7f9920004750 comp rx=0 tx=0).stop 2023-07-13T22:23:55.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 shutdown_connections 2023-07-13T22:23:55.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991c06ace0 0x7f991c06d190 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993006a3c0 0x7f9930154cf0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015e730 0x7f99300b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 --2- 172.21.15.60:0/3010900801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300aed40 0x7f99301f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 >> 172.21.15.60:0/3010900801 conn(0x7f993006bc20 msgr2=0x7f99300cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:55.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 shutdown_connections 2023-07-13T22:23:55.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:55.780+0000 7f99369ff700 1 -- 172.21.15.60:0/3010900801 wait complete. 2023-07-13T22:23:55.780 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:23:12.304337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:23:13.598583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:23:55.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:23:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.004+0000 7fb8229ff700 1 Processor -- start 2023-07-13T22:23:55.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.004+0000 7fb8229ff700 1 -- start start 2023-07-13T22:23:55.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.005+0000 7fb8229ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb81c15ff70 0x7fb81c0d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb8229ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb81c161030 0x7fb81c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb8229ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c1614b0 con 0x7fb81c0d0e60 2023-07-13T22:23:55.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb8229ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c0d12b0 con 0x7fb81c161030 2023-07-13T22:23:55.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb8229ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0cd3a0 con 0x7fb81c15ff70 2023-07-13T22:23:55.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb820f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb81c161030 0x7fb81c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:55.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb820f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb81c161030 0x7fb81c0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59116/0 (socket says 172.21.15.60:59116) 2023-07-13T22:23:55.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb820f9c700 1 -- 172.21.15.60:0/3399441378 learned_addr learned my addr 172.21.15.60:0/3399441378 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:55.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb81bfff700 1 --2- 172.21.15.60:0/3399441378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb81c15ff70 0x7fb81c0d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:55.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb820f9c700 1 -- 172.21.15.60:0/3399441378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb81c15ff70 msgr2=0x7fb81c0d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:55.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb820f9c700 1 --2- 172.21.15.60:0/3399441378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb81c15ff70 0x7fb81c0d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb820f9c700 1 -- 172.21.15.60:0/3399441378 >> v1:172.21.15.105:6789/0 conn(0x7fb81c0d0e60 legacy=0x7fb81c160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.006+0000 7fb820f9c700 1 -- 172.21.15.60:0/3399441378 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0cd5f0 con 0x7fb81c161030 2023-07-13T22:23:55.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb820f9c700 1 --2- 172.21.15.60:0/3399441378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c161030 0x7fb81c0ce3b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb814009e50 tx=0x7fb81400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4195e650aa5be1f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:55.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb81affd700 1 -- 172.21.15.60:0/3399441378 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81400e070 con 0x7fb81c161030 2023-07-13T22:23:55.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb81affd700 1 -- 172.21.15.60:0/3399441378 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81400b910 con 0x7fb81c161030 2023-07-13T22:23:55.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb81affd700 1 -- 172.21.15.60:0/3399441378 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81400bcc0 con 0x7fb81c161030 2023-07-13T22:23:55.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb8229ff700 1 -- 172.21.15.60:0/3399441378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c161030 msgr2=0x7fb81c0ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb8229ff700 1 --2- 172.21.15.60:0/3399441378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c161030 0x7fb81c0ce3b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb814009e50 tx=0x7fb81400b330 comp rx=0 tx=0).stop 2023-07-13T22:23:55.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb8229ff700 1 -- 172.21.15.60:0/3399441378 shutdown_connections 2023-07-13T22:23:55.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb8229ff700 1 --2- 172.21.15.60:0/3399441378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c161030 0x7fb81c0ce3b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb8229ff700 1 --2- 172.21.15.60:0/3399441378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb81c15ff70 0x7fb81c0d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.007+0000 7fb8229ff700 1 -- 172.21.15.60:0/3399441378 >> 172.21.15.60:0/3399441378 conn(0x7fb81c071ed0 msgr2=0x7fb81c09c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:55.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.008+0000 7fb8229ff700 1 -- 172.21.15.60:0/3399441378 shutdown_connections 2023-07-13T22:23:55.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.008+0000 7fb8229ff700 1 -- 172.21.15.60:0/3399441378 wait complete. 2023-07-13T22:23:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.009+0000 7fb8229ff700 1 Processor -- start 2023-07-13T22:23:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.009+0000 7fb8229ff700 1 -- start start 2023-07-13T22:23:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.010+0000 7fb8229ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0d0e60 0x7fb81c0cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.010+0000 7fb8229ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb81c15ff70 0x7fb81c153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.010+0000 7fb8229ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb81c161030 0x7fb81c0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.010+0000 7fb8229ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0c6ed0 con 0x7fb81c0d0e60 2023-07-13T22:23:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.010+0000 7fb8229ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1f5a90 con 0x7fb81c161030 2023-07-13T22:23:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.010+0000 7fb8229ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0c8c00 con 0x7fb81c15ff70 2023-07-13T22:23:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb81bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0d0e60 0x7fb81c0cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb81bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0d0e60 0x7fb81c0cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59124/0 (socket says 172.21.15.60:59124) 2023-07-13T22:23:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb81bfff700 1 -- 172.21.15.60:0/3092380265 learned_addr learned my addr 172.21.15.60:0/3092380265 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb81bfff700 1 -- 172.21.15.60:0/3092380265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb81c15ff70 msgr2=0x7fb81c153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb81bfff700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb81c15ff70 0x7fb81c153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb81bfff700 1 -- 172.21.15.60:0/3092380265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb81c161030 msgr2=0x7fb81c0c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb81bfff700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb81c161030 0x7fb81c0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb81bfff700 1 -- 172.21.15.60:0/3092380265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb814009b30 con 0x7fb81c0d0e60 2023-07-13T22:23:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb81bfff700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0d0e60 0x7fb81c0cb880 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e990 tx=0x7fb80c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.011+0000 7fb818ff9700 1 -- 172.21.15.60:0/3092380265 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0096f0 con 0x7fb81c0d0e60 2023-07-13T22:23:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.012+0000 7fb818ff9700 1 -- 172.21.15.60:0/3092380265 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c00c850 con 0x7fb81c0d0e60 2023-07-13T22:23:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.012+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c0cbf40 con 0x7fb81c0d0e60 2023-07-13T22:23:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.012+0000 7fb818ff9700 1 -- 172.21.15.60:0/3092380265 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00cbc0 con 0x7fb81c0d0e60 2023-07-13T22:23:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.012+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c0c6880 con 0x7fb81c0d0e60 2023-07-13T22:23:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.013+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c067a90 con 0x7fb81c0d0e60 2023-07-13T22:23:55.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.014+0000 7fb818ff9700 1 -- 172.21.15.60:0/3092380265 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb80c00cd20 con 0x7fb81c0d0e60 2023-07-13T22:23:56.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.015+0000 7fb818ff9700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80806ace0 0x7fb80806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.015+0000 7fb818ff9700 1 -- 172.21.15.60:0/3092380265 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb80c0899b0 con 0x7fb81c0d0e60 2023-07-13T22:23:56.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.015+0000 7fb81b7fe700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80806ace0 0x7fb80806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.016+0000 7fb81b7fe700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80806ace0 0x7fb80806d190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb810000f60 tx=0x7fb810007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.020+0000 7fb818ff9700 1 -- 172.21.15.60:0/3092380265 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb80c054a10 con 0x7fb81c0d0e60 2023-07-13T22:23:56.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.226+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb81c072890 con 0x7fb81c0d0e60 2023-07-13T22:23:56.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.226+0000 7fb818ff9700 1 -- 172.21.15.60:0/3092380265 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb80c021020 con 0x7fb81c0d0e60 2023-07-13T22:23:56.209 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:23:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80806ace0 msgr2=0x7fb80806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80806ace0 0x7fb80806d190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb810000f60 tx=0x7fb810007a40 comp rx=0 tx=0).stop 2023-07-13T22:23:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0d0e60 msgr2=0x7fb81c0cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0d0e60 0x7fb81c0cb880 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e990 tx=0x7fb80c00ed50 comp rx=0 tx=0).stop 2023-07-13T22:23:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 shutdown_connections 2023-07-13T22:23:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80806ace0 0x7fb80806d190 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0d0e60 0x7fb81c0cb880 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb81c161030 0x7fb81c0c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 --2- 172.21.15.60:0/3092380265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb81c15ff70 0x7fb81c153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 >> 172.21.15.60:0/3092380265 conn(0x7fb81c071ed0 msgr2=0x7fb81c15a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 shutdown_connections 2023-07-13T22:23:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.229+0000 7fb8229ff700 1 -- 172.21.15.60:0/3092380265 wait complete. 2023-07-13T22:23:56.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:23:56.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.452+0000 7fe881bff700 1 Processor -- start 2023-07-13T22:23:56.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.452+0000 7fe881bff700 1 -- start start 2023-07-13T22:23:56.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.454+0000 7fe881bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe87c15ff70 0x7fe87c0d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.454+0000 7fe881bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87c0cd3a0 con 0x7fe87c161030 2023-07-13T22:23:56.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.454+0000 7fe881bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87c161470 con 0x7fe87c0d0e60 2023-07-13T22:23:56.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.454+0000 7fe881bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87c0d12e0 con 0x7fe87c15ff70 2023-07-13T22:23:56.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.454+0000 7fe87bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe87c161030 0x7fe87c0ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38962/0 (socket says 172.21.15.60:38962) 2023-07-13T22:23:56.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.454+0000 7fe87bfff700 1 -- 172.21.15.60:0/1282087110 learned_addr learned my addr 172.21.15.60:0/1282087110 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.454+0000 7fe87b7fe700 1 --2- 172.21.15.60:0/1282087110 >> v2:172.21.15.105:3300/0 conn(0x7fe87c15ff70 0x7fe87c0d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.454+0000 7fe87affd700 1 -- 172.21.15.60:0/1282087110 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115531496 0 0) 0x7fe87c0cd3a0 con 0x7fe87c161030 2023-07-13T22:23:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.454+0000 7fe87affd700 1 -- 172.21.15.60:0/1282087110 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8680035b0 con 0x7fe87c161030 2023-07-13T22:23:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.455+0000 7fe87affd700 1 -- 172.21.15.60:0/1282087110 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822109547 0 0) 0x7fe87c161470 con 0x7fe87c0d0e60 2023-07-13T22:23:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.455+0000 7fe87affd700 1 -- 172.21.15.60:0/1282087110 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe87c0cd3a0 con 0x7fe87c0d0e60 2023-07-13T22:23:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.455+0000 7fe87b7fe700 1 -- 172.21.15.60:0/1282087110 >> v1:172.21.15.105:6789/0 conn(0x7fe87c0d0e60 legacy=0x7fe87c160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.455+0000 7fe87b7fe700 1 -- 172.21.15.60:0/1282087110 >> v1:172.21.15.60:6789/0 conn(0x7fe87c161030 legacy=0x7fe87c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.455+0000 7fe87b7fe700 1 -- 172.21.15.60:0/1282087110 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c0cd5f0 con 0x7fe87c15ff70 2023-07-13T22:23:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.455+0000 7fe87b7fe700 1 --2- 172.21.15.60:0/1282087110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c15ff70 0x7fe87c0d0920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009850 tx=0x7fe86c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91d7c97394793a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.455+0000 7fe87affd700 1 -- 172.21.15.60:0/1282087110 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c005e30 con 0x7fe87c15ff70 2023-07-13T22:23:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.455+0000 7fe87affd700 1 -- 172.21.15.60:0/1282087110 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c006030 con 0x7fe87c15ff70 2023-07-13T22:23:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.455+0000 7fe87affd700 1 -- 172.21.15.60:0/1282087110 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0063c0 con 0x7fe87c15ff70 2023-07-13T22:23:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.456+0000 7fe881bff700 1 -- 172.21.15.60:0/1282087110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c15ff70 msgr2=0x7fe87c0d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.456+0000 7fe881bff700 1 --2- 172.21.15.60:0/1282087110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c15ff70 0x7fe87c0d0920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009850 tx=0x7fe86c009b60 comp rx=0 tx=0).stop 2023-07-13T22:23:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.456+0000 7fe881bff700 1 -- 172.21.15.60:0/1282087110 shutdown_connections 2023-07-13T22:23:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.456+0000 7fe881bff700 1 --2- 172.21.15.60:0/1282087110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c15ff70 0x7fe87c0d0920 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.456+0000 7fe881bff700 1 -- 172.21.15.60:0/1282087110 >> 172.21.15.60:0/1282087110 conn(0x7fe87c071ed0 msgr2=0x7fe87c09c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.456+0000 7fe881bff700 1 -- 172.21.15.60:0/1282087110 shutdown_connections 2023-07-13T22:23:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.456+0000 7fe881bff700 1 -- 172.21.15.60:0/1282087110 wait complete. 2023-07-13T22:23:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.458+0000 7fe881bff700 1 Processor -- start 2023-07-13T22:23:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.458+0000 7fe881bff700 1 -- start start 2023-07-13T22:23:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe881bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c0d0e60 0x7fe87c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe881bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe87c15ff70 0x7fe87c15e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe881bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe87c161030 0x7fe87c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe881bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c15cee0 con 0x7fe87c161030 2023-07-13T22:23:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe881bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c15c2d0 con 0x7fe87c0d0e60 2023-07-13T22:23:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe881bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c15c5b0 con 0x7fe87c15ff70 2023-07-13T22:23:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe87c161030 0x7fe87c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe87c161030 0x7fe87c0c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59134/0 (socket says 172.21.15.60:59134) 2023-07-13T22:23:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87bfff700 1 -- 172.21.15.60:0/2743616058 learned_addr learned my addr 172.21.15.60:0/2743616058 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87b7fe700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c0d0e60 0x7fe87c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87bfff700 1 -- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe87c15ff70 msgr2=0x7fe87c15e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe873fff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe87c15ff70 0x7fe87c15e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87bfff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe87c15ff70 0x7fe87c15e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87bfff700 1 -- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c0d0e60 msgr2=0x7fe87c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87bfff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c0d0e60 0x7fe87c0cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87bfff700 1 -- 172.21.15.60:0/2743616058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c009430 con 0x7fe87c161030 2023-07-13T22:23:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe87b7fe700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c0d0e60 0x7fe87c0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.459+0000 7fe873fff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe87c15ff70 0x7fe87c15e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.460+0000 7fe87bfff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe87c161030 0x7fe87c0c9e20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe87400cb10 tx=0x7fe87400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.460+0000 7fe878ff9700 1 -- 172.21.15.60:0/2743616058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87400ab00 con 0x7fe87c161030 2023-07-13T22:23:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.460+0000 7fe878ff9700 1 -- 172.21.15.60:0/2743616058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87400ac60 con 0x7fe87c161030 2023-07-13T22:23:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.460+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87c15c890 con 0x7fe87c161030 2023-07-13T22:23:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.460+0000 7fe878ff9700 1 -- 172.21.15.60:0/2743616058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874012400 con 0x7fe87c161030 2023-07-13T22:23:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.460+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c0cc970 con 0x7fe87c161030 2023-07-13T22:23:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.461+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87c067a90 con 0x7fe87c161030 2023-07-13T22:23:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.462+0000 7fe878ff9700 1 -- 172.21.15.60:0/2743616058 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe87401b3b0 con 0x7fe87c161030 2023-07-13T22:23:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.463+0000 7fe878ff9700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe85c06ad10 0x7fe85c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.464+0000 7fe878ff9700 1 -- 172.21.15.60:0/2743616058 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe874089190 con 0x7fe87c161030 2023-07-13T22:23:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.464+0000 7fe87b7fe700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe85c06ad10 0x7fe85c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.464+0000 7fe87b7fe700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe85c06ad10 0x7fe85c06d1c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009820 tx=0x7fe86c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.468+0000 7fe878ff9700 1 -- 172.21.15.60:0/2743616058 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe874054210 con 0x7fe87c161030 2023-07-13T22:23:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.671+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe87c072890 con 0x7fe87c161030 2023-07-13T22:23:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.671+0000 7fe878ff9700 1 -- 172.21.15.60:0/2743616058 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe874057830 con 0x7fe87c161030 2023-07-13T22:23:56.654 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:23:56.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe85c06ad10 msgr2=0x7fe85c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe85c06ad10 0x7fe85c06d1c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009820 tx=0x7fe86c008070 comp rx=0 tx=0).stop 2023-07-13T22:23:56.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe87c161030 msgr2=0x7fe87c0c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe87c161030 0x7fe87c0c9e20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe87400cb10 tx=0x7fe87400ced0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 shutdown_connections 2023-07-13T22:23:56.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe85c06ad10 0x7fe85c06d1c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe87c161030 0x7fe87c0c9e20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe87c0d0e60 0x7fe87c0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 --2- 172.21.15.60:0/2743616058 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe87c15ff70 0x7fe87c15e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 >> 172.21.15.60:0/2743616058 conn(0x7fe87c071ed0 msgr2=0x7fe87c0b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 shutdown_connections 2023-07-13T22:23:56.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.674+0000 7fe881bff700 1 -- 172.21.15.60:0/2743616058 wait complete. 2023-07-13T22:23:56.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:23:56.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.897+0000 7ff1e95ff700 1 Processor -- start 2023-07-13T22:23:56.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.897+0000 7ff1e95ff700 1 -- start start 2023-07-13T22:23:56.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.899+0000 7ff1e95ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff1e40d1030 0x7ff1e4160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.899+0000 7ff1e95ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff1e4160e60 0x7ff1e40ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.899+0000 7ff1e95ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e40cd3a0 con 0x7ff1e415ff70 2023-07-13T22:23:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.899+0000 7ff1e95ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e40cd5f0 con 0x7ff1e40d1030 2023-07-13T22:23:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.899+0000 7ff1e95ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e40cd8d0 con 0x7ff1e4160e60 2023-07-13T22:23:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.899+0000 7ff1e27fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff1e40d1030 0x7ff1e4160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e27fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff1e40d1030 0x7ff1e4160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59136/0 (socket says 172.21.15.60:59136) 2023-07-13T22:23:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e27fc700 1 -- 172.21.15.60:0/1922271054 learned_addr learned my addr 172.21.15.60:0/1922271054 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e37fe700 1 --2- 172.21.15.60:0/1922271054 >> v2:172.21.15.105:3300/0 conn(0x7ff1e4160e60 0x7ff1e40ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e27fc700 1 -- 172.21.15.60:0/1922271054 >> v2:172.21.15.105:3300/0 conn(0x7ff1e4160e60 msgr2=0x7ff1e40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e27fc700 1 --2- 172.21.15.60:0/1922271054 >> v2:172.21.15.105:3300/0 conn(0x7ff1e4160e60 0x7ff1e40ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e27fc700 1 -- 172.21.15.60:0/1922271054 >> v1:172.21.15.105:6789/0 conn(0x7ff1e415ff70 legacy=0x7ff1e40d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e27fc700 1 -- 172.21.15.60:0/1922271054 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40cdbb0 con 0x7ff1e40d1030 2023-07-13T22:23:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e27fc700 1 --2- 172.21.15.60:0/1922271054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e40d1030 0x7ff1e4160920 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40d1400 tx=0x7ff1d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a4b33a615625f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e1ffb700 1 -- 172.21.15.60:0/1922271054 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8004230 con 0x7ff1e40d1030 2023-07-13T22:23:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e1ffb700 1 -- 172.21.15.60:0/1922271054 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d8004390 con 0x7ff1e40d1030 2023-07-13T22:23:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.900+0000 7ff1e1ffb700 1 -- 172.21.15.60:0/1922271054 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80036a0 con 0x7ff1e40d1030 2023-07-13T22:23:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.901+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1922271054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e40d1030 msgr2=0x7ff1e4160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.901+0000 7ff1e95ff700 1 --2- 172.21.15.60:0/1922271054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e40d1030 0x7ff1e4160920 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40d1400 tx=0x7ff1d8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.901+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1922271054 shutdown_connections 2023-07-13T22:23:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.901+0000 7ff1e95ff700 1 --2- 172.21.15.60:0/1922271054 >> v2:172.21.15.105:3300/0 conn(0x7ff1e4160e60 0x7ff1e40ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.901+0000 7ff1e95ff700 1 --2- 172.21.15.60:0/1922271054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e40d1030 0x7ff1e4160920 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.901+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1922271054 >> 172.21.15.60:0/1922271054 conn(0x7ff1e4071ed0 msgr2=0x7ff1e409c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.901+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1922271054 shutdown_connections 2023-07-13T22:23:56.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.901+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1922271054 wait complete. 2023-07-13T22:23:56.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.903+0000 7ff1e95ff700 1 Processor -- start 2023-07-13T22:23:56.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.903+0000 7ff1e95ff700 1 -- start start 2023-07-13T22:23:56.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e95ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1e40d1030 0x7ff1e415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e95ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e415ff70 0x7ff1e40cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e95ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4160e60 0x7ff1e40cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e95ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40ca550 con 0x7ff1e415ff70 2023-07-13T22:23:56.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e95ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e414fb20 con 0x7ff1e4160e60 2023-07-13T22:23:56.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e95ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e415bac0 con 0x7ff1e40d1030 2023-07-13T22:23:56.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e27fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e415ff70 0x7ff1e40cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e27fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e415ff70 0x7ff1e40cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59152/0 (socket says 172.21.15.60:59152) 2023-07-13T22:23:56.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e27fc700 1 -- 172.21.15.60:0/1272696472 learned_addr learned my addr 172.21.15.60:0/1272696472 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e37fe700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4160e60 0x7ff1e40cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.904+0000 7ff1e2ffd700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1e40d1030 0x7ff1e415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1e27fc700 1 -- 172.21.15.60:0/1272696472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1e40d1030 msgr2=0x7ff1e415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1e27fc700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1e40d1030 0x7ff1e415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1e27fc700 1 -- 172.21.15.60:0/1272696472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4160e60 msgr2=0x7ff1e40cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1e27fc700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4160e60 0x7ff1e40cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1e27fc700 1 -- 172.21.15.60:0/1272696472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d8009550 con 0x7ff1e415ff70 2023-07-13T22:23:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1e27fc700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e415ff70 0x7ff1e40cad30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009130 tx=0x7ff1d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1cb7fe700 1 -- 172.21.15.60:0/1272696472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8003e20 con 0x7ff1e415ff70 2023-07-13T22:23:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1cb7fe700 1 -- 172.21.15.60:0/1272696472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d8011630 con 0x7ff1e415ff70 2023-07-13T22:23:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e415b220 con 0x7ff1e415ff70 2023-07-13T22:23:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.905+0000 7ff1cb7fe700 1 -- 172.21.15.60:0/1272696472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80119e0 con 0x7ff1e415ff70 2023-07-13T22:23:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.906+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e415b7e0 con 0x7ff1e415ff70 2023-07-13T22:23:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.907+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e4067a90 con 0x7ff1e415ff70 2023-07-13T22:23:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.907+0000 7ff1cb7fe700 1 -- 172.21.15.60:0/1272696472 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1d8011b40 con 0x7ff1e415ff70 2023-07-13T22:23:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.908+0000 7ff1cb7fe700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1d006ace0 0x7ff1d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.908+0000 7ff1cb7fe700 1 -- 172.21.15.60:0/1272696472 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff1d808aaf0 con 0x7ff1e415ff70 2023-07-13T22:23:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.909+0000 7ff1e2ffd700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1d006ace0 0x7ff1d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.909+0000 7ff1e2ffd700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1d006ace0 0x7ff1d006d190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4004f80 tx=0x7ff1d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:56.913+0000 7ff1cb7fe700 1 -- 172.21.15.60:0/1272696472 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff1d8055ad0 con 0x7ff1e415ff70 2023-07-13T22:23:57.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.118+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1e4072890 con 0x7ff1e415ff70 2023-07-13T22:23:57.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.119+0000 7ff1cb7fe700 1 -- 172.21.15.60:0/1272696472 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1d80590f0 con 0x7ff1e415ff70 2023-07-13T22:23:57.101 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:23:57.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1d006ace0 msgr2=0x7ff1d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1d006ace0 0x7ff1d006d190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4004f80 tx=0x7ff1d4009040 comp rx=0 tx=0).stop 2023-07-13T22:23:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e415ff70 msgr2=0x7ff1e40cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e415ff70 0x7ff1e40cad30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009130 tx=0x7ff1d8004750 comp rx=0 tx=0).stop 2023-07-13T22:23:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 shutdown_connections 2023-07-13T22:23:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1d006ace0 0x7ff1d006d190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1e4160e60 0x7ff1e40cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1e40d1030 0x7ff1e415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 --2- 172.21.15.60:0/1272696472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1e415ff70 0x7ff1e40cad30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.121+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 >> 172.21.15.60:0/1272696472 conn(0x7ff1e4071ed0 msgr2=0x7ff1e40b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.122+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 shutdown_connections 2023-07-13T22:23:57.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.122+0000 7ff1e95ff700 1 -- 172.21.15.60:0/1272696472 wait complete. 2023-07-13T22:23:57.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:23:57.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.345+0000 7fe101dff700 1 Processor -- start 2023-07-13T22:23:57.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.345+0000 7fe101dff700 1 -- start start 2023-07-13T22:23:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe101dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0fc06be60 0x7fe0fc159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe101dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0fc06fe90 con 0x7fe0fc15d890 2023-07-13T22:23:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe101dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0fc15a2f0 con 0x7fe0fc159b70 2023-07-13T22:23:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe101dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc15a5d0 con 0x7fe0fc06be60 2023-07-13T22:23:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe0f3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0fc06be60 0x7fe0fc159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe0f3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0fc06be60 0x7fe0fc159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59160/0 (socket says 172.21.15.60:59160) 2023-07-13T22:23:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe0f3fff700 1 -- 172.21.15.60:0/2799941535 learned_addr learned my addr 172.21.15.60:0/2799941535 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:57.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe0f3fff700 1 -- 172.21.15.60:0/2799941535 >> v1:172.21.15.105:6789/0 conn(0x7fe0fc159b70 legacy=0x7fe0fc0caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe0f3fff700 1 -- 172.21.15.60:0/2799941535 >> v1:172.21.15.60:6789/0 conn(0x7fe0fc15d890 legacy=0x7fe0fc06b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe0f3fff700 1 -- 172.21.15.60:0/2799941535 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc06f840 con 0x7fe0fc06be60 2023-07-13T22:23:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.347+0000 7fe0faffd700 1 -- 172.21.15.60:0/2799941535 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943034557 0 0) 0x7fe0fc06fe90 con 0x7fe0fc15d890 2023-07-13T22:23:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.348+0000 7fe0f3fff700 1 --2- 172.21.15.60:0/2799941535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc06be60 0x7fe0fc159630 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400bb90 tx=0x7fe0e400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f35da0ff453e71 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.348+0000 7fe0faffd700 1 -- 172.21.15.60:0/2799941535 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400dd50 con 0x7fe0fc06be60 2023-07-13T22:23:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.348+0000 7fe0faffd700 1 -- 172.21.15.60:0/2799941535 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e4010470 con 0x7fe0fc06be60 2023-07-13T22:23:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.348+0000 7fe0faffd700 1 -- 172.21.15.60:0/2799941535 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4010800 con 0x7fe0fc06be60 2023-07-13T22:23:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.348+0000 7fe101dff700 1 -- 172.21.15.60:0/2799941535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc06be60 msgr2=0x7fe0fc159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.348+0000 7fe101dff700 1 --2- 172.21.15.60:0/2799941535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc06be60 0x7fe0fc159630 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400bb90 tx=0x7fe0e400bea0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.349+0000 7fe101dff700 1 -- 172.21.15.60:0/2799941535 shutdown_connections 2023-07-13T22:23:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.349+0000 7fe101dff700 1 --2- 172.21.15.60:0/2799941535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc06be60 0x7fe0fc159630 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.349+0000 7fe101dff700 1 -- 172.21.15.60:0/2799941535 >> 172.21.15.60:0/2799941535 conn(0x7fe0fc0b0580 msgr2=0x7fe0fc0b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.349+0000 7fe101dff700 1 -- 172.21.15.60:0/2799941535 shutdown_connections 2023-07-13T22:23:57.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.349+0000 7fe101dff700 1 -- 172.21.15.60:0/2799941535 wait complete. 2023-07-13T22:23:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.350+0000 7fe101dff700 1 Processor -- start 2023-07-13T22:23:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.351+0000 7fe101dff700 1 -- start start 2023-07-13T22:23:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.351+0000 7fe101dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0fc06be60 0x7fe0fc1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.351+0000 7fe101dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc159b70 0x7fe0fc0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.351+0000 7fe101dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0fc15d890 0x7fe0fc0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.351+0000 7fe101dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc14f7d0 con 0x7fe0fc159b70 2023-07-13T22:23:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.351+0000 7fe101dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc14ebc0 con 0x7fe0fc15d890 2023-07-13T22:23:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.351+0000 7fe101dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc14eea0 con 0x7fe0fc06be60 2023-07-13T22:23:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0f3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc159b70 0x7fe0fc0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0f3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc159b70 0x7fe0fc0ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59164/0 (socket says 172.21.15.60:59164) 2023-07-13T22:23:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0f3fff700 1 -- 172.21.15.60:0/1129640727 learned_addr learned my addr 172.21.15.60:0/1129640727 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0fb7fe700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0fc06be60 0x7fe0fc1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0f3fff700 1 -- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0fc06be60 msgr2=0x7fe0fc1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0fbfff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0fc15d890 0x7fe0fc0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0f3fff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0fc06be60 0x7fe0fc1465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0f3fff700 1 -- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0fc15d890 msgr2=0x7fe0fc0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0f3fff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0fc15d890 0x7fe0fc0acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0f3fff700 1 -- 172.21.15.60:0/1129640727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e400b7f0 con 0x7fe0fc159b70 2023-07-13T22:23:57.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0fbfff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0fc15d890 0x7fe0fc0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:57.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0fb7fe700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0fc06be60 0x7fe0fc1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:57.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.352+0000 7fe0f3fff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc159b70 0x7fe0fc0ac890 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4015000 tx=0x7fe0e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.353+0000 7fe0f8ff9700 1 -- 172.21.15.60:0/1129640727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4019810 con 0x7fe0fc159b70 2023-07-13T22:23:57.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.353+0000 7fe0f8ff9700 1 -- 172.21.15.60:0/1129640727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e4019970 con 0x7fe0fc159b70 2023-07-13T22:23:57.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.353+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc14f180 con 0x7fe0fc159b70 2023-07-13T22:23:57.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.353+0000 7fe0f8ff9700 1 -- 172.21.15.60:0/1129640727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4019d20 con 0x7fe0fc159b70 2023-07-13T22:23:57.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.353+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc146b00 con 0x7fe0fc159b70 2023-07-13T22:23:57.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.354+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc14f980 con 0x7fe0fc159b70 2023-07-13T22:23:57.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.355+0000 7fe0f8ff9700 1 -- 172.21.15.60:0/1129640727 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe0e402c430 con 0x7fe0fc159b70 2023-07-13T22:23:57.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.356+0000 7fe0f8ff9700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0dc06ace0 0x7fe0dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.356+0000 7fe0f8ff9700 1 -- 172.21.15.60:0/1129640727 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe0e408b630 con 0x7fe0fc159b70 2023-07-13T22:23:57.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.356+0000 7fe0fb7fe700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0dc06ace0 0x7fe0dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.356+0000 7fe0fb7fe700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0dc06ace0 0x7fe0dc06d190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec000f60 tx=0x7fe0ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.360+0000 7fe0f8ff9700 1 -- 172.21.15.60:0/1129640727 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe0fc14f980 con 0x7fe0fc159b70 2023-07-13T22:23:57.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.569+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0fc14f980 con 0x7fe0fc159b70 2023-07-13T22:23:57.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.569+0000 7fe0f8ff9700 1 -- 172.21.15.60:0/1129640727 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0fc14f980 con 0x7fe0fc159b70 2023-07-13T22:23:57.551 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:23:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.571+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0dc06ace0 msgr2=0x7fe0dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0dc06ace0 0x7fe0dc06d190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec000f60 tx=0x7fe0ec009040 comp rx=0 tx=0).stop 2023-07-13T22:23:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc159b70 msgr2=0x7fe0fc0ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc159b70 0x7fe0fc0ac890 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4015000 tx=0x7fe0e4004750 comp rx=0 tx=0).stop 2023-07-13T22:23:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 shutdown_connections 2023-07-13T22:23:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe0dc06ace0 0x7fe0dc06d190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0fc15d890 0x7fe0fc0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0fc06be60 0x7fe0fc1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 --2- 172.21.15.60:0/1129640727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0fc159b70 0x7fe0fc0ac890 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 >> 172.21.15.60:0/1129640727 conn(0x7fe0fc0b0580 msgr2=0x7fe0fc071180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 shutdown_connections 2023-07-13T22:23:57.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.572+0000 7fe101dff700 1 -- 172.21.15.60:0/1129640727 wait complete. 2023-07-13T22:23:57.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:23:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.793+0000 7ff4c85ff700 1 Processor -- start 2023-07-13T22:23:57.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.794+0000 7ff4c85ff700 1 -- start start 2023-07-13T22:23:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c85ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c0159560 0x7ff4c00c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c85ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff4c015d160 0x7ff4c00c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c85ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4c00cdfd0 0x7ff4c00b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c85ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c015d6d0 con 0x7ff4c00cdfd0 2023-07-13T22:23:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c85ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c015d9b0 con 0x7ff4c015d160 2023-07-13T22:23:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c85ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c015dc90 con 0x7ff4c0159560 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c6b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4c00cdfd0 0x7ff4c00b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c5b9a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff4c015d160 0x7ff4c00c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c639b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c0159560 0x7ff4c00c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c6b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4c00cdfd0 0x7ff4c00b02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59174/0 (socket says 172.21.15.60:59174) 2023-07-13T22:23:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.795+0000 7ff4c6b9c700 1 -- 172.21.15.60:0/732983705 learned_addr learned my addr 172.21.15.60:0/732983705 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.796+0000 7ff4c6b9c700 1 -- 172.21.15.60:0/732983705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c0159560 msgr2=0x7ff4c00c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.796+0000 7ff4c6b9c700 1 --2- 172.21.15.60:0/732983705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c0159560 0x7ff4c00c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.796+0000 7ff4c6b9c700 1 -- 172.21.15.60:0/732983705 >> v2:172.21.15.105:3300/0 conn(0x7ff4c015d160 msgr2=0x7ff4c00c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.796+0000 7ff4c6b9c700 1 --2- 172.21.15.60:0/732983705 >> v2:172.21.15.105:3300/0 conn(0x7ff4c015d160 0x7ff4c00c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.796+0000 7ff4c6b9c700 1 -- 172.21.15.60:0/732983705 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c015df70 con 0x7ff4c00cdfd0 2023-07-13T22:23:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.796+0000 7ff4c6b9c700 1 --2- 172.21.15.60:0/732983705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c00cdfd0 0x7ff4c00b02d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff4c006b2d0 tx=0x7ff4b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8577e9b871454d48 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.796+0000 7ff4c5399700 1 -- 172.21.15.60:0/732983705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b800e070 con 0x7ff4c00cdfd0 2023-07-13T22:23:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.796+0000 7ff4c5399700 1 -- 172.21.15.60:0/732983705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b800b910 con 0x7ff4c00cdfd0 2023-07-13T22:23:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.796+0000 7ff4c5399700 1 -- 172.21.15.60:0/732983705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b800bcc0 con 0x7ff4c00cdfd0 2023-07-13T22:23:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.797+0000 7ff4c85ff700 1 -- 172.21.15.60:0/732983705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c00cdfd0 msgr2=0x7ff4c00b02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.797+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/732983705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c00cdfd0 0x7ff4c00b02d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff4c006b2d0 tx=0x7ff4b800b330 comp rx=0 tx=0).stop 2023-07-13T22:23:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.797+0000 7ff4c85ff700 1 -- 172.21.15.60:0/732983705 shutdown_connections 2023-07-13T22:23:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.797+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/732983705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c00cdfd0 0x7ff4c00b02d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.797+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/732983705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c0159560 0x7ff4c00c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.797+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/732983705 >> v2:172.21.15.105:3300/0 conn(0x7ff4c015d160 0x7ff4c00c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.797+0000 7ff4c85ff700 1 -- 172.21.15.60:0/732983705 >> 172.21.15.60:0/732983705 conn(0x7ff4c00cb2d0 msgr2=0x7ff4c00cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.797+0000 7ff4c85ff700 1 -- 172.21.15.60:0/732983705 shutdown_connections 2023-07-13T22:23:57.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.797+0000 7ff4c85ff700 1 -- 172.21.15.60:0/732983705 wait complete. 2023-07-13T22:23:57.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.799+0000 7ff4c85ff700 1 Processor -- start 2023-07-13T22:23:57.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.799+0000 7ff4c85ff700 1 -- start start 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c85ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c00cdfd0 0x7ff4c00ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c85ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4c0159560 0x7ff4c00b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c85ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c015d160 0x7ff4c00aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c85ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00ae580 con 0x7ff4c015d160 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c85ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00b0d80 con 0x7ff4c0159560 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c85ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00af390 con 0x7ff4c00cdfd0 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c6b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c015d160 0x7ff4c00aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c6b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c015d160 0x7ff4c00aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59184/0 (socket says 172.21.15.60:59184) 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c6b9c700 1 -- 172.21.15.60:0/2151262539 learned_addr learned my addr 172.21.15.60:0/2151262539 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c5b9a700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4c0159560 0x7ff4c00b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c6b9c700 1 -- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c00cdfd0 msgr2=0x7ff4c00ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c639b700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c00cdfd0 0x7ff4c00ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c6b9c700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c00cdfd0 0x7ff4c00ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c6b9c700 1 -- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4c0159560 msgr2=0x7ff4c00b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.800+0000 7ff4c6b9c700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4c0159560 0x7ff4c00b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.801+0000 7ff4c6b9c700 1 -- 172.21.15.60:0/2151262539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8009b30 con 0x7ff4c015d160 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.801+0000 7ff4c5b9a700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4c0159560 0x7ff4c00b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.801+0000 7ff4c639b700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c00cdfd0 0x7ff4c00ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.801+0000 7ff4c6b9c700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c015d160 0x7ff4c00aee50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800b680 tx=0x7ff4b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.801+0000 7ff4b6ffd700 1 -- 172.21.15.60:0/2151262539 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b800ce80 con 0x7ff4c015d160 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.801+0000 7ff4b6ffd700 1 -- 172.21.15.60:0/2151262539 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b8004250 con 0x7ff4c015d160 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.801+0000 7ff4b6ffd700 1 -- 172.21.15.60:0/2151262539 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8004600 con 0x7ff4c015d160 2023-07-13T22:23:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.801+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c015b3c0 con 0x7ff4c015d160 2023-07-13T22:23:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.801+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c015b980 con 0x7ff4c015d160 2023-07-13T22:23:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.802+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c0072890 con 0x7ff4c015d160 2023-07-13T22:23:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.803+0000 7ff4b6ffd700 1 -- 172.21.15.60:0/2151262539 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff4b8004760 con 0x7ff4c015d160 2023-07-13T22:23:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.805+0000 7ff4b6ffd700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4a406ace0 0x7ff4a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.805+0000 7ff4b6ffd700 1 -- 172.21.15.60:0/2151262539 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff4b808a600 con 0x7ff4c015d160 2023-07-13T22:23:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.805+0000 7ff4c639b700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4a406ace0 0x7ff4a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.806+0000 7ff4c639b700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4a406ace0 0x7ff4a406d190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009b60 tx=0x7ff4bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:57.809+0000 7ff4b6ffd700 1 -- 172.21.15.60:0/2151262539 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff4b80555e0 con 0x7ff4c015d160 2023-07-13T22:23:58.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.030+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff4c00adcd0 con 0x7ff4c015d160 2023-07-13T22:23:58.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.030+0000 7ff4b6ffd700 1 -- 172.21.15.60:0/2151262539 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff4b8058c00 con 0x7ff4c015d160 2023-07-13T22:23:58.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4a406ace0 msgr2=0x7ff4a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4a406ace0 0x7ff4a406d190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009b60 tx=0x7ff4bc008040 comp rx=0 tx=0).stop 2023-07-13T22:23:58.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c015d160 msgr2=0x7ff4c00aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c015d160 0x7ff4c00aee50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800b680 tx=0x7ff4b8004ad0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 shutdown_connections 2023-07-13T22:23:58.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff4a406ace0 0x7ff4a406d190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4c015d160 0x7ff4c00aee50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4c00cdfd0 0x7ff4c00ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 --2- 172.21.15.60:0/2151262539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4c0159560 0x7ff4c00b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 >> 172.21.15.60:0/2151262539 conn(0x7ff4c00cb2d0 msgr2=0x7ff4c00cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:58.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 shutdown_connections 2023-07-13T22:23:58.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.033+0000 7ff4c85ff700 1 -- 172.21.15.60:0/2151262539 wait complete. 2023-07-13T22:23:58.017 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:23:58.033 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:23:58.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:58.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:23:58.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[134050]: fuse finished with error 0 and tester_r 0 2023-07-13T22:23:58.105 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:23:58.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:23:58.105 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:23:58.269 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:23:58.270 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:23:58.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:23:58.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.542+0000 7fbc447ff700 1 Processor -- start 2023-07-13T22:23:58.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.542+0000 7fbc447ff700 1 -- start start 2023-07-13T22:23:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc447ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbc3c15c770 0x7fbc3c0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc447ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbc3c0b1c10 0x7fbc3c0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc447ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 0x7fbc3c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc447ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3c0b1800 con 0x7fbc3c0b1c10 2023-07-13T22:23:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc447ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3c0d12f0 con 0x7fbc3c15c770 2023-07-13T22:23:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc447ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0d3cb0 con 0x7fbc3c0d0e70 2023-07-13T22:23:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc41d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbc3c0b1c10 0x7fbc3c0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc41d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbc3c0b1c10 0x7fbc3c0d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59188/0 (socket says 172.21.15.60:59188) 2023-07-13T22:23:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc41d9a700 1 -- 172.21.15.60:0/1311188386 learned_addr learned my addr 172.21.15.60:0/1311188386 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc4259b700 1 --2- 172.21.15.60:0/1311188386 >> v2:172.21.15.105:3300/0 conn(0x7fbc3c15c770 0x7fbc3c0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc42d9c700 1 --2- 172.21.15.60:0/1311188386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 0x7fbc3c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc41d9a700 1 -- 172.21.15.60:0/1311188386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 msgr2=0x7fbc3c0af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:58.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc41d9a700 1 --2- 172.21.15.60:0/1311188386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 0x7fbc3c0af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc41d9a700 1 -- 172.21.15.60:0/1311188386 >> v2:172.21.15.105:3300/0 conn(0x7fbc3c15c770 msgr2=0x7fbc3c0b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:58.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc41d9a700 1 --2- 172.21.15.60:0/1311188386 >> v2:172.21.15.105:3300/0 conn(0x7fbc3c15c770 0x7fbc3c0b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.544+0000 7fbc41d9a700 1 -- 172.21.15.60:0/1311188386 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c0b19f0 con 0x7fbc3c0b1c10 2023-07-13T22:23:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.545+0000 7fbc41d9a700 1 --2- 172.21.15.60:0/1311188386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c0b1c10 0x7fbc3c0d0930 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c066ed0 tx=0x7fbc2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8461e7585dda843 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.545+0000 7fbc41599700 1 -- 172.21.15.60:0/1311188386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c011830 con 0x7fbc3c0b1c10 2023-07-13T22:23:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.545+0000 7fbc41599700 1 -- 172.21.15.60:0/1311188386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c011990 con 0x7fbc3c0b1c10 2023-07-13T22:23:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.545+0000 7fbc41599700 1 -- 172.21.15.60:0/1311188386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c011d20 con 0x7fbc3c0b1c10 2023-07-13T22:23:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.545+0000 7fbc447ff700 1 -- 172.21.15.60:0/1311188386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c0b1c10 msgr2=0x7fbc3c0d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.545+0000 7fbc447ff700 1 --2- 172.21.15.60:0/1311188386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c0b1c10 0x7fbc3c0d0930 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c066ed0 tx=0x7fbc2c009c00 comp rx=0 tx=0).stop 2023-07-13T22:23:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.546+0000 7fbc447ff700 1 -- 172.21.15.60:0/1311188386 shutdown_connections 2023-07-13T22:23:58.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.546+0000 7fbc447ff700 1 --2- 172.21.15.60:0/1311188386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 0x7fbc3c0af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.546+0000 7fbc447ff700 1 --2- 172.21.15.60:0/1311188386 >> v2:172.21.15.105:3300/0 conn(0x7fbc3c15c770 0x7fbc3c0b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.546+0000 7fbc447ff700 1 --2- 172.21.15.60:0/1311188386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c0b1c10 0x7fbc3c0d0930 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.546+0000 7fbc447ff700 1 -- 172.21.15.60:0/1311188386 >> 172.21.15.60:0/1311188386 conn(0x7fbc3c09c070 msgr2=0x7fbc3c0d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:58.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.546+0000 7fbc447ff700 1 -- 172.21.15.60:0/1311188386 shutdown_connections 2023-07-13T22:23:58.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.546+0000 7fbc447ff700 1 -- 172.21.15.60:0/1311188386 wait complete. 2023-07-13T22:23:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.547+0000 7fbc447ff700 1 Processor -- start 2023-07-13T22:23:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.548+0000 7fbc447ff700 1 -- start start 2023-07-13T22:23:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.548+0000 7fbc447ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0b1c10 0x7fbc3c0cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc447ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 0x7fbc3c0c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc447ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c15c770 0x7fbc3c0b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc447ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0c7190 con 0x7fbc3c15c770 2023-07-13T22:23:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc447ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0c7470 con 0x7fbc3c0b1c10 2023-07-13T22:23:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc447ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0c7750 con 0x7fbc3c0d0e70 2023-07-13T22:23:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc42d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c15c770 0x7fbc3c0b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc42d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c15c770 0x7fbc3c0b7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59196/0 (socket says 172.21.15.60:59196) 2023-07-13T22:23:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc42d9c700 1 -- 172.21.15.60:0/2804318101 learned_addr learned my addr 172.21.15.60:0/2804318101 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc4259b700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0b1c10 0x7fbc3c0cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc41d9a700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 0x7fbc3c0c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc42d9c700 1 -- 172.21.15.60:0/2804318101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 msgr2=0x7fbc3c0c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc42d9c700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 0x7fbc3c0c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc42d9c700 1 -- 172.21.15.60:0/2804318101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0b1c10 msgr2=0x7fbc3c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc42d9c700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0b1c10 0x7fbc3c0cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.549+0000 7fbc42d9c700 1 -- 172.21.15.60:0/2804318101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c009550 con 0x7fbc3c15c770 2023-07-13T22:23:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.550+0000 7fbc42d9c700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c15c770 0x7fbc3c0b7d60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbc34007e40 tx=0x7fbc3400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.550+0000 7fbc32ffd700 1 -- 172.21.15.60:0/2804318101 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400ed30 con 0x7fbc3c15c770 2023-07-13T22:23:58.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.550+0000 7fbc32ffd700 1 -- 172.21.15.60:0/2804318101 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc34014420 con 0x7fbc3c15c770 2023-07-13T22:23:58.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.550+0000 7fbc32ffd700 1 -- 172.21.15.60:0/2804318101 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc340147d0 con 0x7fbc3c15c770 2023-07-13T22:23:58.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.550+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c0c7ee0 con 0x7fbc3c15c770 2023-07-13T22:23:58.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.550+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c0cd390 con 0x7fbc3c15c770 2023-07-13T22:23:58.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.551+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3c0c8400 con 0x7fbc3c15c770 2023-07-13T22:23:58.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.552+0000 7fbc32ffd700 1 -- 172.21.15.60:0/2804318101 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbc34014930 con 0x7fbc3c15c770 2023-07-13T22:23:58.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.553+0000 7fbc32ffd700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc2006ace0 0x7fbc2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.554+0000 7fbc32ffd700 1 -- 172.21.15.60:0/2804318101 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbc34089c20 con 0x7fbc3c15c770 2023-07-13T22:23:58.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.554+0000 7fbc4259b700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc2006ace0 0x7fbc2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.554+0000 7fbc4259b700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc2006ace0 0x7fbc2006d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009aa0 tx=0x7fbc380091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.558+0000 7fbc32ffd700 1 -- 172.21.15.60:0/2804318101 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbc3c0c8400 con 0x7fbc3c15c770 2023-07-13T22:23:58.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.765+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc3c0c8400 con 0x7fbc3c15c770 2023-07-13T22:23:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.766+0000 7fbc32ffd700 1 -- 172.21.15.60:0/2804318101 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18353 (secure 0 0 0) 0x7fbc3c0c8400 con 0x7fbc3c15c770 2023-07-13T22:23:58.749 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:23:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":51,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:44.502069+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.21","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:23:58.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc2006ace0 msgr2=0x7fbc2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc2006ace0 0x7fbc2006d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009aa0 tx=0x7fbc380091c0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c15c770 msgr2=0x7fbc3c0b7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c15c770 0x7fbc3c0b7d60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbc34007e40 tx=0x7fbc3400e3c0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 shutdown_connections 2023-07-13T22:23:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbc2006ace0 0x7fbc2006d190 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc3c15c770 0x7fbc3c0b7d60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc3c0b1c10 0x7fbc3c0cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 --2- 172.21.15.60:0/2804318101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbc3c0d0e70 0x7fbc3c0c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 >> 172.21.15.60:0/2804318101 conn(0x7fbc3c09c070 msgr2=0x7fbc3c0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 shutdown_connections 2023-07-13T22:23:58.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.769+0000 7fbc447ff700 1 -- 172.21.15.60:0/2804318101 wait complete. 2023-07-13T22:23:58.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:23:12.304337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:23:13.598583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:23:58.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:23:58.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.993+0000 7f9ceb5ff700 1 Processor -- start 2023-07-13T22:23:58.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.993+0000 7f9ceb5ff700 1 -- start start 2023-07-13T22:23:58.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.994+0000 7f9ceb5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce40d1030 0x7f9ce4160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.994+0000 7f9ceb5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9ce4160e60 0x7f9ce40ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.994+0000 7f9ceb5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce40cd3a0 con 0x7f9ce415ff70 2023-07-13T22:23:58.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.994+0000 7f9ceb5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce40cd5f0 con 0x7f9ce4160e60 2023-07-13T22:23:58.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.994+0000 7f9ceb5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40cd8d0 con 0x7f9ce40d1030 2023-07-13T22:23:58.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.994+0000 7f9ce939b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9ce415ff70 0x7f9ce40d0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38986/0 (socket says 172.21.15.60:38986) 2023-07-13T22:23:58.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.994+0000 7f9ce939b700 1 -- 172.21.15.60:0/3120498234 learned_addr learned my addr 172.21.15.60:0/3120498234 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:58.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.994+0000 7f9ce8b9a700 1 --2- 172.21.15.60:0/3120498234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce40d1030 0x7f9ce4160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.994+0000 7f9ce9b9c700 1 --2- 172.21.15.60:0/3120498234 >> v2:172.21.15.105:3300/0 conn(0x7f9ce4160e60 0x7f9ce40ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.995+0000 7f9cdbfff700 1 -- 172.21.15.60:0/3120498234 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336429350 0 0) 0x7f9ce40cd3a0 con 0x7f9ce415ff70 2023-07-13T22:23:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.995+0000 7f9cdbfff700 1 -- 172.21.15.60:0/3120498234 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cd00035b0 con 0x7f9ce415ff70 2023-07-13T22:23:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.995+0000 7f9ce9b9c700 1 -- 172.21.15.60:0/3120498234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce40d1030 msgr2=0x7f9ce4160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.995+0000 7f9ce9b9c700 1 --2- 172.21.15.60:0/3120498234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce40d1030 0x7f9ce4160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.995+0000 7f9ce9b9c700 1 -- 172.21.15.60:0/3120498234 >> v1:172.21.15.60:6789/0 conn(0x7f9ce415ff70 legacy=0x7f9ce40d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.995+0000 7f9ce9b9c700 1 -- 172.21.15.60:0/3120498234 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce40cdbb0 con 0x7f9ce4160e60 2023-07-13T22:23:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.995+0000 7f9ce8b9a700 1 --2- 172.21.15.60:0/3120498234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce40d1030 0x7f9ce4160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:58.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.995+0000 7f9ce9b9c700 1 --2- 172.21.15.60:0/3120498234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce4160e60 0x7f9ce40ce3b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009d90 tx=0x7f9cdc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=663f48b370808f83 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:58.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9cdbfff700 1 -- 172.21.15.60:0/3120498234 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00e070 con 0x7f9ce4160e60 2023-07-13T22:23:58.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9cdbfff700 1 -- 172.21.15.60:0/3120498234 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc00bb20 con 0x7f9ce4160e60 2023-07-13T22:23:58.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9cdbfff700 1 -- 172.21.15.60:0/3120498234 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc012430 con 0x7f9ce4160e60 2023-07-13T22:23:58.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3120498234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce4160e60 msgr2=0x7f9ce40ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9ceb5ff700 1 --2- 172.21.15.60:0/3120498234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce4160e60 0x7f9ce40ce3b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009d90 tx=0x7f9cdc00b330 comp rx=0 tx=0).stop 2023-07-13T22:23:58.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3120498234 shutdown_connections 2023-07-13T22:23:58.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9ceb5ff700 1 --2- 172.21.15.60:0/3120498234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce4160e60 0x7f9ce40ce3b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9ceb5ff700 1 --2- 172.21.15.60:0/3120498234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce40d1030 0x7f9ce4160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3120498234 >> 172.21.15.60:0/3120498234 conn(0x7f9ce4071ed0 msgr2=0x7f9ce409c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:58.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3120498234 shutdown_connections 2023-07-13T22:23:58.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.996+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3120498234 wait complete. 2023-07-13T22:23:58.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.998+0000 7f9ceb5ff700 1 Processor -- start 2023-07-13T22:23:58.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.998+0000 7f9ceb5ff700 1 -- start start 2023-07-13T22:23:58.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.999+0000 7f9ceb5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ce40d1030 0x7f9ce4154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.999+0000 7f9ceb5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce415ff70 0x7f9ce40cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.999+0000 7f9ceb5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce4160e60 0x7f9ce40c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.999+0000 7f9ceb5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40afdc0 con 0x7f9ce40d1030 2023-07-13T22:23:58.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.999+0000 7f9ceb5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40ca8e0 con 0x7f9ce4160e60 2023-07-13T22:23:58.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.999+0000 7f9ceb5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40cabc0 con 0x7f9ce415ff70 2023-07-13T22:23:58.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.999+0000 7f9ce939b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ce40d1030 0x7f9ce4154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.999+0000 7f9ce939b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ce40d1030 0x7f9ce4154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59204/0 (socket says 172.21.15.60:59204) 2023-07-13T22:23:58.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:58.999+0000 7f9ce939b700 1 -- 172.21.15.60:0/3754579067 learned_addr learned my addr 172.21.15.60:0/3754579067 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:58.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9ce9b9c700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce4160e60 0x7f9ce40c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9ce8b9a700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce415ff70 0x7f9ce40cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9ce939b700 1 -- 172.21.15.60:0/3754579067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce415ff70 msgr2=0x7f9ce40cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9ce939b700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce415ff70 0x7f9ce40cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9ce939b700 1 -- 172.21.15.60:0/3754579067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce4160e60 msgr2=0x7f9ce40c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:58.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9ce939b700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce4160e60 0x7f9ce40c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:58.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9ce939b700 1 -- 172.21.15.60:0/3754579067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd40094a0 con 0x7f9ce40d1030 2023-07-13T22:23:58.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9ce939b700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ce40d1030 0x7f9ce4154c10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0001260 tx=0x7f9ce000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:58.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9cd9ffb700 1 -- 172.21.15.60:0/3754579067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00074e0 con 0x7f9ce40d1030 2023-07-13T22:23:58.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9cd9ffb700 1 -- 172.21.15.60:0/3754579067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0007640 con 0x7f9ce40d1030 2023-07-13T22:23:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc009a70 con 0x7f9ce40d1030 2023-07-13T22:23:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.000+0000 7f9cd9ffb700 1 -- 172.21.15.60:0/3754579067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0005a50 con 0x7f9ce40d1030 2023-07-13T22:23:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.001+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce40b0080 con 0x7f9ce40d1030 2023-07-13T22:23:58.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.002+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce4067a90 con 0x7f9ce40d1030 2023-07-13T22:23:58.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.003+0000 7f9cd9ffb700 1 -- 172.21.15.60:0/3754579067 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ce0005bb0 con 0x7f9ce40d1030 2023-07-13T22:23:58.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.004+0000 7f9cd9ffb700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9cc806ace0 0x7f9cc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:58.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.004+0000 7f9cd9ffb700 1 -- 172.21.15.60:0/3754579067 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9ce00889a0 con 0x7f9ce40d1030 2023-07-13T22:23:58.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.004+0000 7f9ce8b9a700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9cc806ace0 0x7f9cc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:58.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.005+0000 7f9ce8b9a700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9cc806ace0 0x7f9cc806d190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40ccf90 tx=0x7f9cd4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:58.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.009+0000 7f9cd9ffb700 1 -- 172.21.15.60:0/3754579067 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9ce0057400 con 0x7f9ce40d1030 2023-07-13T22:23:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.215+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9ce40ce4d0 con 0x7f9ce40d1030 2023-07-13T22:23:59.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.216+0000 7f9cd9ffb700 1 -- 172.21.15.60:0/3754579067 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9ce00170a0 con 0x7f9ce40d1030 2023-07-13T22:23:59.198 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:23:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9cc806ace0 msgr2=0x7f9cc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9cc806ace0 0x7f9cc806d190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40ccf90 tx=0x7f9cd4009210 comp rx=0 tx=0).stop 2023-07-13T22:23:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ce40d1030 msgr2=0x7f9ce4154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ce40d1030 0x7f9ce4154c10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0001260 tx=0x7f9ce000c9e0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 shutdown_connections 2023-07-13T22:23:59.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9cc806ace0 0x7f9cc806d190 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ce40d1030 0x7f9ce4154c10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce4160e60 0x7f9ce40c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 --2- 172.21.15.60:0/3754579067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ce415ff70 0x7f9ce40cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.218+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 >> 172.21.15.60:0/3754579067 conn(0x7f9ce4071ed0 msgr2=0x7f9ce409c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:59.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.219+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 shutdown_connections 2023-07-13T22:23:59.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.219+0000 7f9ceb5ff700 1 -- 172.21.15.60:0/3754579067 wait complete. 2023-07-13T22:23:59.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:23:59.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.443+0000 7f56fdbff700 1 Processor -- start 2023-07-13T22:23:59.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.443+0000 7f56fdbff700 1 -- start start 2023-07-13T22:23:59.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.444+0000 7f56fdbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56f815ff70 0x7f56f80d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.444+0000 7f56fdbff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f80cd3a0 con 0x7f56f80d0e60 2023-07-13T22:23:59.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.444+0000 7f56fdbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f8161470 con 0x7f56f8161030 2023-07-13T22:23:59.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.444+0000 7f56fdbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f80d12e0 con 0x7f56f815ff70 2023-07-13T22:23:59.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.444+0000 7f56f6ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f56f80d0e60 0x7f56f8160920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39000/0 (socket says 172.21.15.60:39000) 2023-07-13T22:23:59.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.444+0000 7f56f6ffd700 1 -- 172.21.15.60:0/1253684192 learned_addr learned my addr 172.21.15.60:0/1253684192 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:23:59.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.444+0000 7f56f77fe700 1 --2- 172.21.15.60:0/1253684192 >> v2:172.21.15.105:3300/0 conn(0x7f56f815ff70 0x7f56f80d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.445+0000 7f56f67fc700 1 -- 172.21.15.60:0/1253684192 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403294961 0 0) 0x7f56f80cd3a0 con 0x7f56f80d0e60 2023-07-13T22:23:59.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.445+0000 7f56f67fc700 1 -- 172.21.15.60:0/1253684192 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e00035b0 con 0x7f56f80d0e60 2023-07-13T22:23:59.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.445+0000 7f56f67fc700 1 -- 172.21.15.60:0/1253684192 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415601156 0 0) 0x7f56f8161470 con 0x7f56f8161030 2023-07-13T22:23:59.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.445+0000 7f56f67fc700 1 -- 172.21.15.60:0/1253684192 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f80cd3a0 con 0x7f56f8161030 2023-07-13T22:23:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.445+0000 7f56f77fe700 1 -- 172.21.15.60:0/1253684192 >> v1:172.21.15.105:6789/0 conn(0x7f56f8161030 legacy=0x7f56f80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.445+0000 7f56f77fe700 1 -- 172.21.15.60:0/1253684192 >> v1:172.21.15.60:6789/0 conn(0x7f56f80d0e60 legacy=0x7f56f8160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.445+0000 7f56f77fe700 1 -- 172.21.15.60:0/1253684192 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f80cd5f0 con 0x7f56f815ff70 2023-07-13T22:23:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.445+0000 7f56f77fe700 1 --2- 172.21.15.60:0/1253684192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56f815ff70 0x7f56f80d0920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009850 tx=0x7f56e8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d8427bce889f608 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56f67fc700 1 -- 172.21.15.60:0/1253684192 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8005e30 con 0x7f56f815ff70 2023-07-13T22:23:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56f67fc700 1 -- 172.21.15.60:0/1253684192 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8006030 con 0x7f56f815ff70 2023-07-13T22:23:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56f67fc700 1 -- 172.21.15.60:0/1253684192 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e80063c0 con 0x7f56f815ff70 2023-07-13T22:23:59.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56fdbff700 1 -- 172.21.15.60:0/1253684192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56f815ff70 msgr2=0x7f56f80d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56fdbff700 1 --2- 172.21.15.60:0/1253684192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56f815ff70 0x7f56f80d0920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009850 tx=0x7f56e8009b60 comp rx=0 tx=0).stop 2023-07-13T22:23:59.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56fdbff700 1 -- 172.21.15.60:0/1253684192 shutdown_connections 2023-07-13T22:23:59.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56fdbff700 1 --2- 172.21.15.60:0/1253684192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56f815ff70 0x7f56f80d0920 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56fdbff700 1 -- 172.21.15.60:0/1253684192 >> 172.21.15.60:0/1253684192 conn(0x7f56f8071ed0 msgr2=0x7f56f809c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:59.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56fdbff700 1 -- 172.21.15.60:0/1253684192 shutdown_connections 2023-07-13T22:23:59.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.446+0000 7f56fdbff700 1 -- 172.21.15.60:0/1253684192 wait complete. 2023-07-13T22:23:59.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.448+0000 7f56fdbff700 1 Processor -- start 2023-07-13T22:23:59.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.448+0000 7f56fdbff700 1 -- start start 2023-07-13T22:23:59.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.449+0000 7f56fdbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56f80d0e60 0x7f56f80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.449+0000 7f56fdbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56f815ff70 0x7f56f814ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.449+0000 7f56fdbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56f8161030 0x7f56f814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.449+0000 7f56fdbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80cbdc0 con 0x7f56f815ff70 2023-07-13T22:23:59.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.449+0000 7f56fdbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f815f7f0 con 0x7f56f80d0e60 2023-07-13T22:23:59.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.449+0000 7f56fdbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80cb140 con 0x7f56f8161030 2023-07-13T22:23:59.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.449+0000 7f56f6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56f815ff70 0x7f56f814ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.449+0000 7f56f6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56f815ff70 0x7f56f814ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59206/0 (socket says 172.21.15.60:59206) 2023-07-13T22:23:59.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.449+0000 7f56f6ffd700 1 -- 172.21.15.60:0/153275736 learned_addr learned my addr 172.21.15.60:0/153275736 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:59.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56f77fe700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56f80d0e60 0x7f56f80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56f7fff700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56f8161030 0x7f56f814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56f6ffd700 1 -- 172.21.15.60:0/153275736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56f8161030 msgr2=0x7f56f814f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56f6ffd700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56f8161030 0x7f56f814f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56f6ffd700 1 -- 172.21.15.60:0/153275736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56f80d0e60 msgr2=0x7f56f80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56f6ffd700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56f80d0e60 0x7f56f80cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56f6ffd700 1 -- 172.21.15.60:0/153275736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e8009430 con 0x7f56f815ff70 2023-07-13T22:23:59.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56f7fff700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56f8161030 0x7f56f814f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:59.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56f6ffd700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56f815ff70 0x7f56f814ed80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c700 tx=0x7f56ec00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:59.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56dffff700 1 -- 172.21.15.60:0/153275736 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec005d70 con 0x7f56f815ff70 2023-07-13T22:23:59.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56dffff700 1 -- 172.21.15.60:0/153275736 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ec0070d0 con 0x7f56f815ff70 2023-07-13T22:23:59.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f80cb420 con 0x7f56f815ff70 2023-07-13T22:23:59.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56dffff700 1 -- 172.21.15.60:0/153275736 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec007480 con 0x7f56f815ff70 2023-07-13T22:23:59.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.450+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f80cb9e0 con 0x7f56f815ff70 2023-07-13T22:23:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.452+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f8067a90 con 0x7f56f815ff70 2023-07-13T22:23:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.453+0000 7f56dffff700 1 -- 172.21.15.60:0/153275736 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56ec0075e0 con 0x7f56f815ff70 2023-07-13T22:23:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.454+0000 7f56dffff700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56e406ace0 0x7f56e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.454+0000 7f56dffff700 1 -- 172.21.15.60:0/153275736 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f56ec088bc0 con 0x7f56f815ff70 2023-07-13T22:23:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.454+0000 7f56f77fe700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56e406ace0 0x7f56e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.455+0000 7f56f77fe700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56e406ace0 0x7f56e406d190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f56e8013040 tx=0x7f56e8008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:59.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.458+0000 7f56dffff700 1 -- 172.21.15.60:0/153275736 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f56ec053ba0 con 0x7f56f815ff70 2023-07-13T22:23:59.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.665+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56f80cde70 con 0x7f56f815ff70 2023-07-13T22:23:59.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.666+0000 7f56dffff700 1 -- 172.21.15.60:0/153275736 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56f80cde70 con 0x7f56f815ff70 2023-07-13T22:23:59.648 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:23:59.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56e406ace0 msgr2=0x7f56e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56e406ace0 0x7f56e406d190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f56e8013040 tx=0x7f56e8008070 comp rx=0 tx=0).stop 2023-07-13T22:23:59.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56f815ff70 msgr2=0x7f56f814ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56f815ff70 0x7f56f814ed80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c700 tx=0x7f56ec00cac0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 shutdown_connections 2023-07-13T22:23:59.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56e406ace0 0x7f56e406d190 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56f8161030 0x7f56f814f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56f80d0e60 0x7f56f80cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 --2- 172.21.15.60:0/153275736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56f815ff70 0x7f56f814ed80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 >> 172.21.15.60:0/153275736 conn(0x7f56f8071ed0 msgr2=0x7f56f806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:59.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 shutdown_connections 2023-07-13T22:23:59.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.668+0000 7f56fdbff700 1 -- 172.21.15.60:0/153275736 wait complete. 2023-07-13T22:23:59.669 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:23:59.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.892+0000 7f57cabff700 1 Processor -- start 2023-07-13T22:23:59.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.892+0000 7f57cabff700 1 -- start start 2023-07-13T22:23:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57cabff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57c415d890 0x7f57c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57cabff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f57c406bc90 0x7f57c4159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57cabff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c406fe90 con 0x7f57c4159b70 2023-07-13T22:23:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57cabff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c415a5d0 con 0x7f57c415d890 2023-07-13T22:23:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57cabff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c415cf50 con 0x7f57c406bc90 2023-07-13T22:23:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c899b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57c415d890 0x7f57c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c899b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57c415d890 0x7f57c406b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59222/0 (socket says 172.21.15.60:59222) 2023-07-13T22:23:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c899b700 1 -- 172.21.15.60:0/235206384 learned_addr learned my addr 172.21.15.60:0/235206384 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c919c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f57c4159b70 0x7f57c40caf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39012/0 (socket says 172.21.15.60:39012) 2023-07-13T22:23:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c3fff700 1 --2- 172.21.15.60:0/235206384 >> v2:172.21.15.105:3300/0 conn(0x7f57c406bc90 0x7f57c4159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c899b700 1 -- 172.21.15.60:0/235206384 >> v2:172.21.15.105:3300/0 conn(0x7f57c406bc90 msgr2=0x7f57c4159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:59.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c899b700 1 --2- 172.21.15.60:0/235206384 >> v2:172.21.15.105:3300/0 conn(0x7f57c406bc90 0x7f57c4159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c37fe700 1 -- 172.21.15.60:0/235206384 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296601783 0 0) 0x7f57c406fe90 con 0x7f57c4159b70 2023-07-13T22:23:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c899b700 1 -- 172.21.15.60:0/235206384 >> v1:172.21.15.60:6789/0 conn(0x7f57c4159b70 legacy=0x7f57c40caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.894+0000 7f57c899b700 1 -- 172.21.15.60:0/235206384 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c415a2f0 con 0x7f57c415d890 2023-07-13T22:23:59.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.895+0000 7f57c899b700 1 --2- 172.21.15.60:0/235206384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c415d890 0x7f57c406b750 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f57c406ad90 tx=0x7f57b800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1c6c9c2bcbd93a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:59.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.895+0000 7f57c37fe700 1 -- 172.21.15.60:0/235206384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800dd10 con 0x7f57c415d890 2023-07-13T22:23:59.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.895+0000 7f57c37fe700 1 -- 172.21.15.60:0/235206384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b800de70 con 0x7f57c415d890 2023-07-13T22:23:59.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.895+0000 7f57c37fe700 1 -- 172.21.15.60:0/235206384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8010670 con 0x7f57c415d890 2023-07-13T22:23:59.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.895+0000 7f57cabff700 1 -- 172.21.15.60:0/235206384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c415d890 msgr2=0x7f57c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.896+0000 7f57cabff700 1 --2- 172.21.15.60:0/235206384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c415d890 0x7f57c406b750 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f57c406ad90 tx=0x7f57b800bea0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.896+0000 7f57cabff700 1 -- 172.21.15.60:0/235206384 shutdown_connections 2023-07-13T22:23:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.896+0000 7f57cabff700 1 --2- 172.21.15.60:0/235206384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c415d890 0x7f57c406b750 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.896+0000 7f57cabff700 1 --2- 172.21.15.60:0/235206384 >> v2:172.21.15.105:3300/0 conn(0x7f57c406bc90 0x7f57c4159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.896+0000 7f57cabff700 1 -- 172.21.15.60:0/235206384 >> 172.21.15.60:0/235206384 conn(0x7f57c40b0580 msgr2=0x7f57c40b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:59.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.896+0000 7f57cabff700 1 -- 172.21.15.60:0/235206384 shutdown_connections 2023-07-13T22:23:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.896+0000 7f57cabff700 1 -- 172.21.15.60:0/235206384 wait complete. 2023-07-13T22:23:59.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.897+0000 7f57cabff700 1 Processor -- start 2023-07-13T22:23:59.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.898+0000 7f57cabff700 1 -- start start 2023-07-13T22:23:59.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57cabff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c406bc90 0x7f57c41f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57cabff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4159b70 0x7f57c40ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57cabff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57c415d890 0x7f57c40acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57cabff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40c9d70 con 0x7f57c406bc90 2023-07-13T22:23:59.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57cabff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40c98e0 con 0x7f57c4159b70 2023-07-13T22:23:59.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57cabff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40cbef0 con 0x7f57c415d890 2023-07-13T22:23:59.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c406bc90 0x7f57c41f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c406bc90 0x7f57c41f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59232/0 (socket says 172.21.15.60:59232) 2023-07-13T22:23:59.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c899b700 1 -- 172.21.15.60:0/2262241025 learned_addr learned my addr 172.21.15.60:0/2262241025 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:23:59.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c3fff700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4159b70 0x7f57c40ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c919c700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57c415d890 0x7f57c40acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c899b700 1 -- 172.21.15.60:0/2262241025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57c415d890 msgr2=0x7f57c40acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c899b700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57c415d890 0x7f57c40acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c899b700 1 -- 172.21.15.60:0/2262241025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4159b70 msgr2=0x7f57c40ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:59.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c899b700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4159b70 0x7f57c40ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:59.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c899b700 1 -- 172.21.15.60:0/2262241025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b800b7f0 con 0x7f57c406bc90 2023-07-13T22:23:59.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.899+0000 7f57c919c700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57c415d890 0x7f57c40acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.900+0000 7f57c899b700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c406bc90 0x7f57c41f1fd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b340 tx=0x7f57b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.900+0000 7f57c17fa700 1 -- 172.21.15.60:0/2262241025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8003cf0 con 0x7f57c406bc90 2023-07-13T22:23:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.900+0000 7f57c17fa700 1 -- 172.21.15.60:0/2262241025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b8003e70 con 0x7f57c406bc90 2023-07-13T22:23:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.900+0000 7f57c17fa700 1 -- 172.21.15.60:0/2262241025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8010690 con 0x7f57c406bc90 2023-07-13T22:23:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.900+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c40c8ff0 con 0x7f57c406bc90 2023-07-13T22:23:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.900+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c40c9570 con 0x7f57c406bc90 2023-07-13T22:23:59.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.901+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c4072890 con 0x7f57c406bc90 2023-07-13T22:23:59.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.902+0000 7f57c17fa700 1 -- 172.21.15.60:0/2262241025 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57b8019400 con 0x7f57c406bc90 2023-07-13T22:23:59.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.903+0000 7f57c17fa700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57a806ace0 0x7f57a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:59.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.903+0000 7f57c17fa700 1 -- 172.21.15.60:0/2262241025 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f57b808ac80 con 0x7f57c406bc90 2023-07-13T22:23:59.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.903+0000 7f57c3fff700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57a806ace0 0x7f57a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:59.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.904+0000 7f57c3fff700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57a806ace0 0x7f57a806d190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f57c40b4210 tx=0x7f57b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:59.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:23:59.907+0000 7f57c17fa700 1 -- 172.21.15.60:0/2262241025 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f57b8055ce0 con 0x7f57c406bc90 2023-07-13T22:24:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.114+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57c40ade70 con 0x7f57c406bc90 2023-07-13T22:24:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.115+0000 7f57c17fa700 1 -- 172.21.15.60:0/2262241025 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57b8059300 con 0x7f57c406bc90 2023-07-13T22:24:00.097 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:24:00.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57a806ace0 msgr2=0x7f57a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57a806ace0 0x7f57a806d190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f57c40b4210 tx=0x7f57b0008040 comp rx=0 tx=0).stop 2023-07-13T22:24:00.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c406bc90 msgr2=0x7f57c41f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c406bc90 0x7f57c41f1fd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b340 tx=0x7f57b8004750 comp rx=0 tx=0).stop 2023-07-13T22:24:00.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 shutdown_connections 2023-07-13T22:24:00.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57a806ace0 0x7f57a806d190 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57c406bc90 0x7f57c41f1fd0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57c415d890 0x7f57c40acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 --2- 172.21.15.60:0/2262241025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57c4159b70 0x7f57c40ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.117+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 >> 172.21.15.60:0/2262241025 conn(0x7f57c40b0580 msgr2=0x7f57c40ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:00.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.118+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 shutdown_connections 2023-07-13T22:24:00.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.118+0000 7f57cabff700 1 -- 172.21.15.60:0/2262241025 wait complete. 2023-07-13T22:24:00.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:24:00.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.342+0000 7fafe09ff700 1 Processor -- start 2023-07-13T22:24:00.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.342+0000 7fafe09ff700 1 -- start start 2023-07-13T22:24:00.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.343+0000 7fafe09ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc15ff70 0x7fafdc0d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.343+0000 7fafe09ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc1614b0 con 0x7fafdc161030 2023-07-13T22:24:00.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.343+0000 7fafe09ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc0d12b0 con 0x7fafdc0d0e60 2023-07-13T22:24:00.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.343+0000 7fafe09ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0cd3a0 con 0x7fafdc15ff70 2023-07-13T22:24:00.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.343+0000 7fafdad9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fafdc161030 0x7fafdc0ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39022/0 (socket says 172.21.15.60:39022) 2023-07-13T22:24:00.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.343+0000 7fafdad9d700 1 -- 172.21.15.60:0/2864160287 learned_addr learned my addr 172.21.15.60:0/2864160287 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:00.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.343+0000 7fafda59c700 1 --2- 172.21.15.60:0/2864160287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc15ff70 0x7fafdc0d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.344+0000 7fafd959a700 1 -- 172.21.15.60:0/2864160287 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060401979 0 0) 0x7fafdc1614b0 con 0x7fafdc161030 2023-07-13T22:24:00.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.344+0000 7fafd959a700 1 -- 172.21.15.60:0/2864160287 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc40035b0 con 0x7fafdc161030 2023-07-13T22:24:00.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.344+0000 7fafd959a700 1 -- 172.21.15.60:0/2864160287 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045146614 0 0) 0x7fafd0001110 con 0x7fafdc0d0e60 2023-07-13T22:24:00.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.344+0000 7fafd959a700 1 -- 172.21.15.60:0/2864160287 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafdc1614b0 con 0x7fafdc0d0e60 2023-07-13T22:24:00.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.344+0000 7fafda59c700 1 -- 172.21.15.60:0/2864160287 >> v1:172.21.15.105:6789/0 conn(0x7fafdc0d0e60 legacy=0x7fafdc160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.344+0000 7fafda59c700 1 -- 172.21.15.60:0/2864160287 >> v1:172.21.15.60:6789/0 conn(0x7fafdc161030 legacy=0x7fafdc0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.344+0000 7fafda59c700 1 -- 172.21.15.60:0/2864160287 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0cd5f0 con 0x7fafdc15ff70 2023-07-13T22:24:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.344+0000 7fafda59c700 1 --2- 172.21.15.60:0/2864160287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc15ff70 0x7fafdc0d0920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009850 tx=0x7fafcc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771698f3fcced4ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafd959a700 1 -- 172.21.15.60:0/2864160287 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc005e30 con 0x7fafdc15ff70 2023-07-13T22:24:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafd959a700 1 -- 172.21.15.60:0/2864160287 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc004030 con 0x7fafdc15ff70 2023-07-13T22:24:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafd959a700 1 -- 172.21.15.60:0/2864160287 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0043c0 con 0x7fafdc15ff70 2023-07-13T22:24:00.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafe09ff700 1 -- 172.21.15.60:0/2864160287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc15ff70 msgr2=0x7fafdc0d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafe09ff700 1 --2- 172.21.15.60:0/2864160287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc15ff70 0x7fafdc0d0920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009850 tx=0x7fafcc009b60 comp rx=0 tx=0).stop 2023-07-13T22:24:00.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafe09ff700 1 -- 172.21.15.60:0/2864160287 shutdown_connections 2023-07-13T22:24:00.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafe09ff700 1 --2- 172.21.15.60:0/2864160287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc15ff70 0x7fafdc0d0920 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafe09ff700 1 -- 172.21.15.60:0/2864160287 >> 172.21.15.60:0/2864160287 conn(0x7fafdc071ed0 msgr2=0x7fafdc09c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:00.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafe09ff700 1 -- 172.21.15.60:0/2864160287 shutdown_connections 2023-07-13T22:24:00.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.345+0000 7fafe09ff700 1 -- 172.21.15.60:0/2864160287 wait complete. 2023-07-13T22:24:00.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.347+0000 7fafe09ff700 1 Processor -- start 2023-07-13T22:24:00.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.347+0000 7fafe09ff700 1 -- start start 2023-07-13T22:24:00.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.348+0000 7fafe09ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafdc0d0e60 0x7fafdc0c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.348+0000 7fafe09ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc15ff70 0x7fafdc0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.348+0000 7fafe09ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc161030 0x7fafdc0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.348+0000 7fafe09ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0b7180 con 0x7fafdc15ff70 2023-07-13T22:24:00.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.348+0000 7fafe09ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0cce10 con 0x7fafdc0d0e60 2023-07-13T22:24:00.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.348+0000 7fafe09ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0b7360 con 0x7fafdc161030 2023-07-13T22:24:00.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.348+0000 7fafd9d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc15ff70 0x7fafdc0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafda59c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafdc0d0e60 0x7fafdc0c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafdad9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc161030 0x7fafdc0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafd9d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc15ff70 0x7fafdc0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59238/0 (socket says 172.21.15.60:59238) 2023-07-13T22:24:00.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafda59c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafdc0d0e60 0x7fafdc0c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:39256/0 (socket says 172.21.15.60:39256) 2023-07-13T22:24:00.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafd9d9b700 1 -- 172.21.15.60:0/1977404276 learned_addr learned my addr 172.21.15.60:0/1977404276 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:00.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafdad9d700 1 -- 172.21.15.60:0/1977404276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafdc0d0e60 msgr2=0x7fafdc0c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafdad9d700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafdc0d0e60 0x7fafdc0c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafdad9d700 1 -- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc15ff70 msgr2=0x7fafdc0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafdad9d700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc15ff70 0x7fafdc0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafdad9d700 1 -- 172.21.15.60:0/1977404276 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc009430 con 0x7fafdc161030 2023-07-13T22:24:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafd9d9b700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc15ff70 0x7fafdc0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafda59c700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafdc0d0e60 0x7fafdc0c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.349+0000 7fafdad9d700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc161030 0x7fafdc0ca510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fafd400caa0 tx=0x7fafd400cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.350+0000 7fafcaffd700 1 -- 172.21.15.60:0/1977404276 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400a9e0 con 0x7fafdc161030 2023-07-13T22:24:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.350+0000 7fafcaffd700 1 -- 172.21.15.60:0/1977404276 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd400ab40 con 0x7fafdc161030 2023-07-13T22:24:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.350+0000 7fafcaffd700 1 -- 172.21.15.60:0/1977404276 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4012400 con 0x7fafdc161030 2023-07-13T22:24:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.350+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc0b7640 con 0x7fafdc161030 2023-07-13T22:24:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.350+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc0b7c00 con 0x7fafdc161030 2023-07-13T22:24:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.351+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc067a90 con 0x7fafdc161030 2023-07-13T22:24:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.352+0000 7fafcaffd700 1 -- 172.21.15.60:0/1977404276 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fafd401b3b0 con 0x7fafdc161030 2023-07-13T22:24:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.353+0000 7fafcaffd700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafbc06ace0 0x7fafbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.353+0000 7fafcaffd700 1 -- 172.21.15.60:0/1977404276 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fafd40893a0 con 0x7fafdc161030 2023-07-13T22:24:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.354+0000 7fafda59c700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafbc06ace0 0x7fafbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.354+0000 7fafda59c700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafbc06ace0 0x7fafbc06d190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fafcc000c00 tx=0x7fafcc008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.357+0000 7fafcaffd700 1 -- 172.21.15.60:0/1977404276 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fafd4054530 con 0x7fafdc161030 2023-07-13T22:24:00.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.562+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fafdc0b57e0 con 0x7fafdc161030 2023-07-13T22:24:00.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.563+0000 7fafcaffd700 1 -- 172.21.15.60:0/1977404276 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7fafd4057b50 con 0x7fafdc161030 2023-07-13T22:24:00.545 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:24:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafbc06ace0 msgr2=0x7fafbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafbc06ace0 0x7fafbc06d190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fafcc000c00 tx=0x7fafcc008070 comp rx=0 tx=0).stop 2023-07-13T22:24:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc161030 msgr2=0x7fafdc0ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc161030 0x7fafdc0ca510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fafd400caa0 tx=0x7fafd400cdb0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 shutdown_connections 2023-07-13T22:24:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafbc06ace0 0x7fafbc06d190 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fafdc161030 0x7fafdc0ca510 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafdc0d0e60 0x7fafdc0c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 --2- 172.21.15.60:0/1977404276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafdc15ff70 0x7fafdc0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.565+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 >> 172.21.15.60:0/1977404276 conn(0x7fafdc071ed0 msgr2=0x7fafdc09c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.566+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 shutdown_connections 2023-07-13T22:24:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.566+0000 7fafe09ff700 1 -- 172.21.15.60:0/1977404276 wait complete. 2023-07-13T22:24:00.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:24:00.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.788+0000 7f177dfff700 1 Processor -- start 2023-07-13T22:24:00.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.788+0000 7f177dfff700 1 -- start start 2023-07-13T22:24:00.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.789+0000 7f177dfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780d0fa0 0x7f17780b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.789+0000 7f177dfff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17780af890 con 0x7f17780b1b90 2023-07-13T22:24:00.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.789+0000 7f177dfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f177815ca80 con 0x7f177815c6b0 2023-07-13T22:24:00.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.789+0000 7f177dfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17780afae0 con 0x7f17780d0fa0 2023-07-13T22:24:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f1777fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f17780b1b90 0x7f17780b2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39038/0 (socket says 172.21.15.60:39038) 2023-07-13T22:24:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f1777fff700 1 -- 172.21.15.60:0/2745501926 learned_addr learned my addr 172.21.15.60:0/2745501926 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f1776ffd700 1 --2- 172.21.15.60:0/2745501926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780d0fa0 0x7f17780b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f17767fc700 1 -- 172.21.15.60:0/2745501926 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719290838 0 0) 0x7f17780af890 con 0x7f17780b1b90 2023-07-13T22:24:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f17767fc700 1 -- 172.21.15.60:0/2745501926 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17600035b0 con 0x7f17780b1b90 2023-07-13T22:24:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f17767fc700 1 -- 172.21.15.60:0/2745501926 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659665919 0 0) 0x7f177815ca80 con 0x7f177815c6b0 2023-07-13T22:24:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f17767fc700 1 -- 172.21.15.60:0/2745501926 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17780af890 con 0x7f177815c6b0 2023-07-13T22:24:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f1776ffd700 1 -- 172.21.15.60:0/2745501926 >> v1:172.21.15.105:6789/0 conn(0x7f177815c6b0 legacy=0x7f17780d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f1776ffd700 1 -- 172.21.15.60:0/2745501926 >> v1:172.21.15.60:6789/0 conn(0x7f17780b1b90 legacy=0x7f17780b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.790+0000 7f17767fc700 1 -- 172.21.15.60:0/2745501926 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2193073559 0 0) 0x7f17600035b0 con 0x7f17780b1b90 2023-07-13T22:24:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.791+0000 7f1776ffd700 1 -- 172.21.15.60:0/2745501926 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17780afdc0 con 0x7f17780d0fa0 2023-07-13T22:24:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.791+0000 7f1776ffd700 1 --2- 172.21.15.60:0/2745501926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780d0fa0 0x7f17780b25a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f176c009870 tx=0x7f176c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ecacab6cc5bf7ca server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.791+0000 7f17767fc700 1 -- 172.21.15.60:0/2745501926 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17700019f0 con 0x7f17780d0fa0 2023-07-13T22:24:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.791+0000 7f17767fc700 1 -- 172.21.15.60:0/2745501926 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c004030 con 0x7f17780d0fa0 2023-07-13T22:24:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.791+0000 7f17767fc700 1 -- 172.21.15.60:0/2745501926 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c0043c0 con 0x7f17780d0fa0 2023-07-13T22:24:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.792+0000 7f177dfff700 1 -- 172.21.15.60:0/2745501926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780d0fa0 msgr2=0x7f17780b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.792+0000 7f177dfff700 1 --2- 172.21.15.60:0/2745501926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780d0fa0 0x7f17780b25a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f176c009870 tx=0x7f176c009b80 comp rx=0 tx=0).stop 2023-07-13T22:24:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.792+0000 7f177dfff700 1 -- 172.21.15.60:0/2745501926 shutdown_connections 2023-07-13T22:24:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.792+0000 7f177dfff700 1 --2- 172.21.15.60:0/2745501926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780d0fa0 0x7f17780b25a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.792+0000 7f177dfff700 1 -- 172.21.15.60:0/2745501926 >> 172.21.15.60:0/2745501926 conn(0x7f177809c030 msgr2=0x7f17780d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.792+0000 7f177dfff700 1 -- 172.21.15.60:0/2745501926 shutdown_connections 2023-07-13T22:24:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.792+0000 7f177dfff700 1 -- 172.21.15.60:0/2745501926 wait complete. 2023-07-13T22:24:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.793+0000 7f177dfff700 1 Processor -- start 2023-07-13T22:24:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.794+0000 7f177dfff700 1 -- start start 2023-07-13T22:24:00.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f177dfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780b1b90 0x7f17781f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f177dfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17780d0fa0 0x7f17780b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f177dfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f177815c6b0 0x7f177815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f177dfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17780c70c0 con 0x7f17780d0fa0 2023-07-13T22:24:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f177dfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17781f22b0 con 0x7f177815c6b0 2023-07-13T22:24:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f177dfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17780c6480 con 0x7f17780b1b90 2023-07-13T22:24:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1776ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17780d0fa0 0x7f17780b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1776ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17780d0fa0 0x7f17780b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59254/0 (socket says 172.21.15.60:59254) 2023-07-13T22:24:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1776ffd700 1 -- 172.21.15.60:0/248030738 learned_addr learned my addr 172.21.15.60:0/248030738 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1777fff700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f177815c6b0 0x7f177815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f17777fe700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780b1b90 0x7f17781f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1776ffd700 1 -- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780b1b90 msgr2=0x7f17781f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1776ffd700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780b1b90 0x7f17781f1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1776ffd700 1 -- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f177815c6b0 msgr2=0x7f177815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1776ffd700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f177815c6b0 0x7f177815eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1776ffd700 1 -- 172.21.15.60:0/248030738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0094a0 con 0x7f17780d0fa0 2023-07-13T22:24:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f1777fff700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f177815c6b0 0x7f177815eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.795+0000 7f17777fe700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780b1b90 0x7f17781f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.796+0000 7f1776ffd700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17780d0fa0 0x7f17780b7540 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f176c0097c0 tx=0x7f176c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.796+0000 7f175ffff700 1 -- 172.21.15.60:0/248030738 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c011830 con 0x7f17780d0fa0 2023-07-13T22:24:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.796+0000 7f175ffff700 1 -- 172.21.15.60:0/248030738 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c01f070 con 0x7f17780d0fa0 2023-07-13T22:24:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.796+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17780c6720 con 0x7f17780d0fa0 2023-07-13T22:24:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.796+0000 7f175ffff700 1 -- 172.21.15.60:0/248030738 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c011bd0 con 0x7f17780d0fa0 2023-07-13T22:24:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.796+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17780c6ab0 con 0x7f17780d0fa0 2023-07-13T22:24:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.797+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17780c7200 con 0x7f17780d0fa0 2023-07-13T22:24:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.798+0000 7f175ffff700 1 -- 172.21.15.60:0/248030738 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f176c011d30 con 0x7f17780d0fa0 2023-07-13T22:24:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.799+0000 7f175ffff700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f176406ace0 0x7f176406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.799+0000 7f175ffff700 1 -- 172.21.15.60:0/248030738 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f176c021070 con 0x7f17780d0fa0 2023-07-13T22:24:00.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.799+0000 7f17777fe700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f176406ace0 0x7f176406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:00.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.799+0000 7f17777fe700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f176406ace0 0x7f176406d190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f17680011e0 tx=0x7f1768009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:00.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:00.803+0000 7f175ffff700 1 -- 172.21.15.60:0/248030738 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f17780c7200 con 0x7f17780d0fa0 2023-07-13T22:24:01.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.024+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17780c7200 con 0x7f17780d0fa0 2023-07-13T22:24:01.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.024+0000 7f175ffff700 1 -- 172.21.15.60:0/248030738 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7f17780c7200 con 0x7f17780d0fa0 2023-07-13T22:24:01.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f176406ace0 msgr2=0x7f176406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:01.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f176406ace0 0x7f176406d190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f17680011e0 tx=0x7f1768009040 comp rx=0 tx=0).stop 2023-07-13T22:24:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17780d0fa0 msgr2=0x7f17780b7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17780d0fa0 0x7f17780b7540 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f176c0097c0 tx=0x7f176c004750 comp rx=0 tx=0).stop 2023-07-13T22:24:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 shutdown_connections 2023-07-13T22:24:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f176406ace0 0x7f176406d190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f177815c6b0 0x7f177815eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780b1b90 0x7f17781f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 --2- 172.21.15.60:0/248030738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17780d0fa0 0x7f17780b7540 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 >> 172.21.15.60:0/248030738 conn(0x7f177809c030 msgr2=0x7f17780d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 shutdown_connections 2023-07-13T22:24:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.027+0000 7f177dfff700 1 -- 172.21.15.60:0/248030738 wait complete. 2023-07-13T22:24:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:24:01.027 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:24:01.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:01.028 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:24:01.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[121721]: fuse finished with error 0 and tester_r 0 2023-07-13T22:24:01.105 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:24:01.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:01.106 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:24:01.208 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:24:01.209 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:01.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:24:01.256 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:24:01.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:01.257 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:24:01.341 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-13T22:24:01.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:24:01.343 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:24:01.343 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:01.389 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:24:01.390 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:24:01.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:01.390 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:24:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-13T22:24:01.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:24:01.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:01.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.698+0000 7f13735ff700 1 Processor -- start 2023-07-13T22:24:01.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.698+0000 7f13735ff700 1 -- start start 2023-07-13T22:24:01.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.699+0000 7f13735ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f136c0d1040 0x7f136c0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f13735ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c0d3cb0 con 0x7f136c0b1c10 2023-07-13T22:24:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f13735ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c0b19f0 con 0x7f136c15c770 2023-07-13T22:24:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f13735ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f136c0b1800 con 0x7f136c0d1040 2023-07-13T22:24:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f1371b9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f136c0b1c10 0x7f136c0b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39046/0 (socket says 172.21.15.60:39046) 2023-07-13T22:24:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f1371b9c700 1 -- 172.21.15.60:0/2290555024 learned_addr learned my addr 172.21.15.60:0/2290555024 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f1370b9a700 1 --2- 172.21.15.60:0/2290555024 >> v2:172.21.15.105:3300/0 conn(0x7f136c0d1040 0x7f136c0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f1363fff700 1 -- 172.21.15.60:0/2290555024 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099191561 0 0) 0x7f136c0d3cb0 con 0x7f136c0b1c10 2023-07-13T22:24:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f1363fff700 1 -- 172.21.15.60:0/2290555024 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13580035b0 con 0x7f136c0b1c10 2023-07-13T22:24:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f1363fff700 1 -- 172.21.15.60:0/2290555024 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607060661 0 0) 0x7f136c0b19f0 con 0x7f136c15c770 2023-07-13T22:24:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.700+0000 7f1363fff700 1 -- 172.21.15.60:0/2290555024 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f136c0d3cb0 con 0x7f136c15c770 2023-07-13T22:24:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.701+0000 7f1370b9a700 1 -- 172.21.15.60:0/2290555024 >> v1:172.21.15.105:6789/0 conn(0x7f136c15c770 legacy=0x7f136c0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.701+0000 7f1363fff700 1 -- 172.21.15.60:0/2290555024 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2975757869 0 0) 0x7f13580035b0 con 0x7f136c0b1c10 2023-07-13T22:24:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.701+0000 7f1370b9a700 1 -- 172.21.15.60:0/2290555024 >> v1:172.21.15.60:6789/0 conn(0x7f136c0b1c10 legacy=0x7f136c0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:01.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.701+0000 7f1370b9a700 1 -- 172.21.15.60:0/2290555024 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0b2290 con 0x7f136c0d1040 2023-07-13T22:24:01.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.701+0000 7f1370b9a700 1 --2- 172.21.15.60:0/2290555024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c0d1040 0x7f136c0b12c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f135c009840 tx=0x7f135c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92987e16e89c9990 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:01.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.701+0000 7f1363fff700 1 -- 172.21.15.60:0/2290555024 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13640019f0 con 0x7f136c0d1040 2023-07-13T22:24:01.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.701+0000 7f1363fff700 1 -- 172.21.15.60:0/2290555024 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c004030 con 0x7f136c0d1040 2023-07-13T22:24:01.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.701+0000 7f1363fff700 1 -- 172.21.15.60:0/2290555024 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c0043c0 con 0x7f136c0d1040 2023-07-13T22:24:01.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.702+0000 7f13735ff700 1 -- 172.21.15.60:0/2290555024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c0d1040 msgr2=0x7f136c0b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:01.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.702+0000 7f13735ff700 1 --2- 172.21.15.60:0/2290555024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c0d1040 0x7f136c0b12c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f135c009840 tx=0x7f135c009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:01.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.702+0000 7f13735ff700 1 -- 172.21.15.60:0/2290555024 shutdown_connections 2023-07-13T22:24:01.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.702+0000 7f13735ff700 1 --2- 172.21.15.60:0/2290555024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c0d1040 0x7f136c0b12c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.702+0000 7f13735ff700 1 -- 172.21.15.60:0/2290555024 >> 172.21.15.60:0/2290555024 conn(0x7f136c09c070 msgr2=0x7f136c0d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:01.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.702+0000 7f13735ff700 1 -- 172.21.15.60:0/2290555024 shutdown_connections 2023-07-13T22:24:01.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.702+0000 7f13735ff700 1 -- 172.21.15.60:0/2290555024 wait complete. 2023-07-13T22:24:01.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.703+0000 7f13735ff700 1 Processor -- start 2023-07-13T22:24:01.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.704+0000 7f13735ff700 1 -- start start 2023-07-13T22:24:01.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.704+0000 7f13735ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f136c0b1c10 0x7f136c0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:01.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f13735ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f136c0d1040 0x7f136c0b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:01.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f13735ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c15c770 0x7f136c0b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f13735ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0c7f90 con 0x7f136c0d1040 2023-07-13T22:24:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f13735ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0c8830 con 0x7f136c15c770 2023-07-13T22:24:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f13735ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1f5d80 con 0x7f136c0b1c10 2023-07-13T22:24:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f1370b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f136c0d1040 0x7f136c0b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f1370b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f136c0d1040 0x7f136c0b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59268/0 (socket says 172.21.15.60:59268) 2023-07-13T22:24:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f1370b9a700 1 -- 172.21.15.60:0/770772358 learned_addr learned my addr 172.21.15.60:0/770772358 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f1371b9c700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c15c770 0x7f136c0b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f137139b700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f136c0b1c10 0x7f136c0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f1370b9a700 1 -- 172.21.15.60:0/770772358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f136c0b1c10 msgr2=0x7f136c0b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f1370b9a700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f136c0b1c10 0x7f136c0b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f1370b9a700 1 -- 172.21.15.60:0/770772358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c15c770 msgr2=0x7f136c0b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f1370b9a700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c15c770 0x7f136c0b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.705+0000 7f1370b9a700 1 -- 172.21.15.60:0/770772358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c0094a0 con 0x7f136c0d1040 2023-07-13T22:24:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.706+0000 7f1370b9a700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f136c0d1040 0x7f136c0b7fa0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f135c0090b0 tx=0x7f135c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.706+0000 7f1361ffb700 1 -- 172.21.15.60:0/770772358 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c004570 con 0x7f136c0d1040 2023-07-13T22:24:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.706+0000 7f1361ffb700 1 -- 172.21.15.60:0/770772358 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c01f070 con 0x7f136c0d1040 2023-07-13T22:24:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.706+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c0c81e0 con 0x7f136c0d1040 2023-07-13T22:24:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.706+0000 7f1361ffb700 1 -- 172.21.15.60:0/770772358 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c011870 con 0x7f136c0d1040 2023-07-13T22:24:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.706+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c0b7790 con 0x7f136c0d1040 2023-07-13T22:24:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.707+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136c072890 con 0x7f136c0d1040 2023-07-13T22:24:01.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.708+0000 7f1361ffb700 1 -- 172.21.15.60:0/770772358 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f135c0119d0 con 0x7f136c0d1040 2023-07-13T22:24:01.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.709+0000 7f1361ffb700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f135006ace0 0x7f135006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:01.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.709+0000 7f1361ffb700 1 -- 172.21.15.60:0/770772358 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f135c021070 con 0x7f136c0d1040 2023-07-13T22:24:01.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.709+0000 7f137139b700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f135006ace0 0x7f135006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:01.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.709+0000 7f137139b700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f135006ace0 0x7f135006d190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1368007fd0 tx=0x7f1368009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:01.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.713+0000 7f1361ffb700 1 -- 172.21.15.60:0/770772358 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f135c08ce60 con 0x7f136c0d1040 2023-07-13T22:24:01.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.965+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f136c0c8970 con 0x7f136c0d1040 2023-07-13T22:24:01.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.965+0000 7f1361ffb700 1 -- 172.21.15.60:0/770772358 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f135c02b090 con 0x7f136c0d1040 2023-07-13T22:24:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:01.948 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3226044033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3226044033},{"type":"v1","addr":"172.21.15.105:6834","nonce":3226044033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5599,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2171011214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":2171011214},{"type":"v1","addr":"172.21.15.60:6836","nonce":2171011214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1623228645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1623228645},{"type":"v1","addr":"172.21.15.60:6837","nonce":1623228645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:23:14.616210+0000","modified":"2023-07-13T22:23:18.125486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5608},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5608":{"gid":5608,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/3570367595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3570367595},{"type":"v1","addr":"172.21.15.105:6835","nonce":3570367595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-13T22:24:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f135006ace0 msgr2=0x7f135006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f135006ace0 0x7f135006d190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1368007fd0 tx=0x7f1368009040 comp rx=0 tx=0).stop 2023-07-13T22:24:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f136c0d1040 msgr2=0x7f136c0b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f136c0d1040 0x7f136c0b7fa0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f135c0090b0 tx=0x7f135c004750 comp rx=0 tx=0).stop 2023-07-13T22:24:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 shutdown_connections 2023-07-13T22:24:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f135006ace0 0x7f135006d190 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c15c770 0x7f136c0b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f136c0b1c10 0x7f136c0b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 --2- 172.21.15.60:0/770772358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f136c0d1040 0x7f136c0b7fa0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 >> 172.21.15.60:0/770772358 conn(0x7f136c09c070 msgr2=0x7f136c159410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 shutdown_connections 2023-07-13T22:24:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:01.968+0000 7f13735ff700 1 -- 172.21.15.60:0/770772358 wait complete. 2023-07-13T22:24:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2023-07-13T22:24:01.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:02.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.193+0000 7fbf423ff700 1 Processor -- start 2023-07-13T22:24:02.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.193+0000 7fbf423ff700 1 -- start start 2023-07-13T22:24:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.194+0000 7fbf423ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf3c15dc50 0x7fbf3c0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.194+0000 7fbf423ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c0c9b00 con 0x7fbf3c1597e0 2023-07-13T22:24:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.194+0000 7fbf423ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c159d70 con 0x7fbf3c06c7a0 2023-07-13T22:24:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.194+0000 7fbf423ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c15a050 con 0x7fbf3c15dc50 2023-07-13T22:24:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.195+0000 7fbf3b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf3c15dc50 0x7fbf3c0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.195+0000 7fbf4099c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbf3c1597e0 0x7fbf3c0ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39050/0 (socket says 172.21.15.60:39050) 2023-07-13T22:24:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.195+0000 7fbf4099c700 1 -- 172.21.15.60:0/2358094014 learned_addr learned my addr 172.21.15.60:0/2358094014 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:02.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.195+0000 7fbf3b7fe700 1 -- 172.21.15.60:0/2358094014 >> v1:172.21.15.105:6789/0 conn(0x7fbf3c06c7a0 legacy=0x7fbf3c06cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:02.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.195+0000 7fbf3b7fe700 1 -- 172.21.15.60:0/2358094014 >> v1:172.21.15.60:6789/0 conn(0x7fbf3c1597e0 legacy=0x7fbf3c0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.195+0000 7fbf3affd700 1 -- 172.21.15.60:0/2358094014 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164553461 0 0) 0x7fbf3c0c9b00 con 0x7fbf3c1597e0 2023-07-13T22:24:02.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.195+0000 7fbf3b7fe700 1 -- 172.21.15.60:0/2358094014 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c15a330 con 0x7fbf3c15dc50 2023-07-13T22:24:02.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf3b7fe700 1 --2- 172.21.15.60:0/2358094014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c15dc50 0x7fbf3c0c9190 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009840 tx=0x7fbf30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e83d387a75bd965 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:02.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf3affd700 1 -- 172.21.15.60:0/2358094014 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30011830 con 0x7fbf3c15dc50 2023-07-13T22:24:02.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf3affd700 1 -- 172.21.15.60:0/2358094014 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf30011990 con 0x7fbf3c15dc50 2023-07-13T22:24:02.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf3affd700 1 -- 172.21.15.60:0/2358094014 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30011d20 con 0x7fbf3c15dc50 2023-07-13T22:24:02.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf423ff700 1 -- 172.21.15.60:0/2358094014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c15dc50 msgr2=0x7fbf3c0c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf423ff700 1 --2- 172.21.15.60:0/2358094014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c15dc50 0x7fbf3c0c9190 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009840 tx=0x7fbf30009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:02.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf423ff700 1 -- 172.21.15.60:0/2358094014 shutdown_connections 2023-07-13T22:24:02.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf423ff700 1 --2- 172.21.15.60:0/2358094014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c15dc50 0x7fbf3c0c9190 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf423ff700 1 -- 172.21.15.60:0/2358094014 >> 172.21.15.60:0/2358094014 conn(0x7fbf3c0cb390 msgr2=0x7fbf3c0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:02.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf423ff700 1 -- 172.21.15.60:0/2358094014 shutdown_connections 2023-07-13T22:24:02.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.196+0000 7fbf423ff700 1 -- 172.21.15.60:0/2358094014 wait complete. 2023-07-13T22:24:02.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.198+0000 7fbf423ff700 1 Processor -- start 2023-07-13T22:24:02.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.198+0000 7fbf423ff700 1 -- start start 2023-07-13T22:24:02.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.199+0000 7fbf423ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf3c06c7a0 0x7fbf3c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.199+0000 7fbf423ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c1597e0 0x7fbf3c157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.199+0000 7fbf423ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf3c15dc50 0x7fbf3c0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.199+0000 7fbf423ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0b3870 con 0x7fbf3c1597e0 2023-07-13T22:24:02.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.199+0000 7fbf423ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0b0d30 con 0x7fbf3c15dc50 2023-07-13T22:24:02.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.199+0000 7fbf423ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0afc90 con 0x7fbf3c06c7a0 2023-07-13T22:24:02.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.199+0000 7fbf3b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c1597e0 0x7fbf3c157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf3b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c1597e0 0x7fbf3c157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59286/0 (socket says 172.21.15.60:59286) 2023-07-13T22:24:02.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf3b7fe700 1 -- 172.21.15.60:0/4086641311 learned_addr learned my addr 172.21.15.60:0/4086641311 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:02.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf4099c700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf3c15dc50 0x7fbf3c0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf3bfff700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf3c06c7a0 0x7fbf3c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf3b7fe700 1 -- 172.21.15.60:0/4086641311 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf3c06c7a0 msgr2=0x7fbf3c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf3b7fe700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf3c06c7a0 0x7fbf3c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf3b7fe700 1 -- 172.21.15.60:0/4086641311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf3c15dc50 msgr2=0x7fbf3c0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf3b7fe700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf3c15dc50 0x7fbf3c0c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf3b7fe700 1 -- 172.21.15.60:0/4086641311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300094a0 con 0x7fbf3c1597e0 2023-07-13T22:24:02.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf3b7fe700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c1597e0 0x7fbf3c157dd0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbf30014040 tx=0x7fbf30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:02.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.200+0000 7fbf38ff9700 1 -- 172.21.15.60:0/4086641311 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3001a7e0 con 0x7fbf3c1597e0 2023-07-13T22:24:02.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.201+0000 7fbf38ff9700 1 -- 172.21.15.60:0/4086641311 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3001a940 con 0x7fbf3c1597e0 2023-07-13T22:24:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.201+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c15a9d0 con 0x7fbf3c1597e0 2023-07-13T22:24:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.201+0000 7fbf38ff9700 1 -- 172.21.15.60:0/4086641311 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3001acb0 con 0x7fbf3c1597e0 2023-07-13T22:24:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.201+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c0b0020 con 0x7fbf3c1597e0 2023-07-13T22:24:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.202+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c072890 con 0x7fbf3c1597e0 2023-07-13T22:24:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.203+0000 7fbf38ff9700 1 -- 172.21.15.60:0/4086641311 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf30022070 con 0x7fbf3c1597e0 2023-07-13T22:24:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.204+0000 7fbf38ff9700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf2806ace0 0x7fbf2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.204+0000 7fbf38ff9700 1 -- 172.21.15.60:0/4086641311 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbf3008a2a0 con 0x7fbf3c1597e0 2023-07-13T22:24:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.204+0000 7fbf3bfff700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf2806ace0 0x7fbf2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.205+0000 7fbf3bfff700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf2806ace0 0x7fbf2806d190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c000f60 tx=0x7fbf2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:02.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.208+0000 7fbf38ff9700 1 -- 172.21.15.60:0/4086641311 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbf300554d0 con 0x7fbf3c1597e0 2023-07-13T22:24:02.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.414+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf3c0cbfa0 con 0x7fbf3c1597e0 2023-07-13T22:24:02.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.416+0000 7fbf38ff9700 1 -- 172.21.15.60:0/4086641311 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18353 (secure 0 0 0) 0x7fbf30058af0 con 0x7fbf3c1597e0 2023-07-13T22:24:02.398 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:02.398 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":51,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:44.502069+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.21","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:02.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf2806ace0 msgr2=0x7fbf2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf2806ace0 0x7fbf2806d190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c000f60 tx=0x7fbf2c009040 comp rx=0 tx=0).stop 2023-07-13T22:24:02.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c1597e0 msgr2=0x7fbf3c157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c1597e0 0x7fbf3c157dd0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbf30014040 tx=0x7fbf30004750 comp rx=0 tx=0).stop 2023-07-13T22:24:02.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 shutdown_connections 2023-07-13T22:24:02.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf2806ace0 0x7fbf2806d190 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf3c15dc50 0x7fbf3c0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf3c06c7a0 0x7fbf3c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 --2- 172.21.15.60:0/4086641311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c1597e0 0x7fbf3c157dd0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.418+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 >> 172.21.15.60:0/4086641311 conn(0x7fbf3c0cb390 msgr2=0x7fbf3c06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:02.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.419+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 shutdown_connections 2023-07-13T22:24:02.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.419+0000 7fbf423ff700 1 -- 172.21.15.60:0/4086641311 wait complete. 2023-07-13T22:24:02.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:23:12.304337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:23:13.598583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:24:02.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:24:02.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.643+0000 7fedd6fff700 1 Processor -- start 2023-07-13T22:24:02.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.643+0000 7fedd6fff700 1 -- start start 2023-07-13T22:24:02.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedd6fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd00d1030 0x7fedd0160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedd6fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fedd0160e60 0x7fedd00ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedd6fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd00cd3a0 con 0x7fedd015ff70 2023-07-13T22:24:02.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedd6fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd00cd5f0 con 0x7fedd0160e60 2023-07-13T22:24:02.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedd6fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00cd8d0 con 0x7fedd00d1030 2023-07-13T22:24:02.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedd4d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fedd015ff70 0x7fedd00d0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39052/0 (socket says 172.21.15.60:39052) 2023-07-13T22:24:02.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedd4d9b700 1 -- 172.21.15.60:0/2260660389 learned_addr learned my addr 172.21.15.60:0/2260660389 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:02.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedd559c700 1 --2- 172.21.15.60:0/2260660389 >> v2:172.21.15.105:3300/0 conn(0x7fedd0160e60 0x7fedd00ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedcffff700 1 --2- 172.21.15.60:0/2260660389 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd00d1030 0x7fedd0160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedcf7fe700 1 -- 172.21.15.60:0/2260660389 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407186516 0 0) 0x7fedd00cd3a0 con 0x7fedd015ff70 2023-07-13T22:24:02.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.645+0000 7fedcf7fe700 1 -- 172.21.15.60:0/2260660389 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedc00035b0 con 0x7fedd015ff70 2023-07-13T22:24:02.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedcffff700 1 -- 172.21.15.60:0/2260660389 >> v2:172.21.15.105:3300/0 conn(0x7fedd0160e60 msgr2=0x7fedd00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:02.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedcffff700 1 --2- 172.21.15.60:0/2260660389 >> v2:172.21.15.105:3300/0 conn(0x7fedd0160e60 0x7fedd00ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedcf7fe700 1 -- 172.21.15.60:0/2260660389 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1489478376 0 0) 0x7fedc00035b0 con 0x7fedd015ff70 2023-07-13T22:24:02.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedcffff700 1 -- 172.21.15.60:0/2260660389 >> v1:172.21.15.60:6789/0 conn(0x7fedd015ff70 legacy=0x7fedd00d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedcffff700 1 -- 172.21.15.60:0/2260660389 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd00cdbb0 con 0x7fedd00d1030 2023-07-13T22:24:02.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedd559c700 1 --2- 172.21.15.60:0/2260660389 >> v2:172.21.15.105:3300/0 conn(0x7fedd0160e60 0x7fedd00ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:02.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedcffff700 1 --2- 172.21.15.60:0/2260660389 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd00d1030 0x7fedd0160920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009840 tx=0x7fedbc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3924c783d6baf612 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:02.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedcf7fe700 1 -- 172.21.15.60:0/2260660389 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc40017f0 con 0x7fedd00d1030 2023-07-13T22:24:02.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedcf7fe700 1 -- 172.21.15.60:0/2260660389 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc004030 con 0x7fedd00d1030 2023-07-13T22:24:02.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.646+0000 7fedcf7fe700 1 -- 172.21.15.60:0/2260660389 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0043c0 con 0x7fedd00d1030 2023-07-13T22:24:02.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.647+0000 7fedd6fff700 1 -- 172.21.15.60:0/2260660389 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd00d1030 msgr2=0x7fedd0160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.647+0000 7fedd6fff700 1 --2- 172.21.15.60:0/2260660389 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd00d1030 0x7fedd0160920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009840 tx=0x7fedbc009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:02.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.647+0000 7fedd6fff700 1 -- 172.21.15.60:0/2260660389 shutdown_connections 2023-07-13T22:24:02.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.647+0000 7fedd6fff700 1 --2- 172.21.15.60:0/2260660389 >> v2:172.21.15.105:3300/0 conn(0x7fedd0160e60 0x7fedd00ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.647+0000 7fedd6fff700 1 --2- 172.21.15.60:0/2260660389 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd00d1030 0x7fedd0160920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.647+0000 7fedd6fff700 1 -- 172.21.15.60:0/2260660389 >> 172.21.15.60:0/2260660389 conn(0x7fedd0071ed0 msgr2=0x7fedd009c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:02.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.647+0000 7fedd6fff700 1 -- 172.21.15.60:0/2260660389 shutdown_connections 2023-07-13T22:24:02.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.647+0000 7fedd6fff700 1 -- 172.21.15.60:0/2260660389 wait complete. 2023-07-13T22:24:02.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.649+0000 7fedd6fff700 1 Processor -- start 2023-07-13T22:24:02.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.649+0000 7fedd6fff700 1 -- start start 2023-07-13T22:24:02.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedd6fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedd00d1030 0x7fedd00b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedd6fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd015ff70 0x7fedd0151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedd6fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd0160e60 0x7fedd00b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedd6fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00b8cb0 con 0x7fedd015ff70 2023-07-13T22:24:02.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedd6fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00cafe0 con 0x7fedd00d1030 2023-07-13T22:24:02.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedd6fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00ca9d0 con 0x7fedd0160e60 2023-07-13T22:24:02.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedcffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd015ff70 0x7fedd0151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedcffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd015ff70 0x7fedd0151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59298/0 (socket says 172.21.15.60:59298) 2023-07-13T22:24:02.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedcffff700 1 -- 172.21.15.60:0/675720540 learned_addr learned my addr 172.21.15.60:0/675720540 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:02.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedd4d9b700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedd00d1030 0x7fedd00b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedcffff700 1 -- 172.21.15.60:0/675720540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd0160e60 msgr2=0x7fedd00b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedcffff700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd0160e60 0x7fedd00b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedcffff700 1 -- 172.21.15.60:0/675720540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedd00d1030 msgr2=0x7fedd00b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedcffff700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedd00d1030 0x7fedd00b83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedcffff700 1 -- 172.21.15.60:0/675720540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc8009a70 con 0x7fedd015ff70 2023-07-13T22:24:02.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.650+0000 7fedd4d9b700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedd00d1030 0x7fedd00b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:02.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.651+0000 7fedcffff700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd015ff70 0x7fedd0151360 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00a010 tx=0x7fedbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:02.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.651+0000 7fedcd7fa700 1 -- 172.21.15.60:0/675720540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc003e50 con 0x7fedd015ff70 2023-07-13T22:24:02.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.651+0000 7fedcd7fa700 1 -- 172.21.15.60:0/675720540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc01f070 con 0x7fedd015ff70 2023-07-13T22:24:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.651+0000 7fedcd7fa700 1 -- 172.21.15.60:0/675720540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc011870 con 0x7fedd015ff70 2023-07-13T22:24:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.651+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedbc0094a0 con 0x7fedd015ff70 2023-07-13T22:24:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.651+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd00cac70 con 0x7fedd015ff70 2023-07-13T22:24:02.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.652+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd0067a90 con 0x7fedd015ff70 2023-07-13T22:24:02.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.653+0000 7fedcd7fa700 1 -- 172.21.15.60:0/675720540 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fedbc0119d0 con 0x7fedd015ff70 2023-07-13T22:24:02.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.655+0000 7fedcd7fa700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb406ace0 0x7fedb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:02.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.655+0000 7fedcd7fa700 1 -- 172.21.15.60:0/675720540 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fedbc021070 con 0x7fedd015ff70 2023-07-13T22:24:02.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.655+0000 7fedd4d9b700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb406ace0 0x7fedb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:02.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.656+0000 7fedd4d9b700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb406ace0 0x7fedb406d190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fedc4000ef0 tx=0x7fedc4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:02.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.659+0000 7fedcd7fa700 1 -- 172.21.15.60:0/675720540 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fedbc055e70 con 0x7fedd015ff70 2023-07-13T22:24:02.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.865+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fedd00cb150 con 0x7fedd015ff70 2023-07-13T22:24:02.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.865+0000 7fedcd7fa700 1 -- 172.21.15.60:0/675720540 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7fedbc02b0a0 con 0x7fedd015ff70 2023-07-13T22:24:02.847 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:24:02.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.867+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb406ace0 msgr2=0x7fedb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.867+0000 7fedd6fff700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb406ace0 0x7fedb406d190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fedc4000ef0 tx=0x7fedc4009040 comp rx=0 tx=0).stop 2023-07-13T22:24:02.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd015ff70 msgr2=0x7fedd0151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:02.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd015ff70 0x7fedd0151360 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00a010 tx=0x7fedbc004750 comp rx=0 tx=0).stop 2023-07-13T22:24:02.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 shutdown_connections 2023-07-13T22:24:02.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedb406ace0 0x7fedb406d190 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedd0160e60 0x7fedd00b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedd00d1030 0x7fedd00b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 --2- 172.21.15.60:0/675720540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedd015ff70 0x7fedd0151360 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:02.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 >> 172.21.15.60:0/675720540 conn(0x7fedd0071ed0 msgr2=0x7fedd006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:02.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 shutdown_connections 2023-07-13T22:24:02.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:02.868+0000 7fedd6fff700 1 -- 172.21.15.60:0/675720540 wait complete. 2023-07-13T22:24:02.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:24:03.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.091+0000 7f51a83ff700 1 Processor -- start 2023-07-13T22:24:03.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.091+0000 7f51a83ff700 1 -- start start 2023-07-13T22:24:03.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.092+0000 7f51a83ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51a015ff70 0x7f51a00d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.092+0000 7f51a83ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51a00d0e60 0x7f51a0160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.092+0000 7f51a83ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a00cd3a0 con 0x7f51a0160e60 2023-07-13T22:24:03.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.092+0000 7f51a83ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a00d12e0 con 0x7f51a00d0e60 2023-07-13T22:24:03.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.092+0000 7f51a83ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a00cd5f0 con 0x7f51a015ff70 2023-07-13T22:24:03.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.092+0000 7f51a599a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51a00d0e60 0x7f51a0160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.092+0000 7f51a699c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f51a0160e60 0x7f51a00cf760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39060/0 (socket says 172.21.15.60:39060) 2023-07-13T22:24:03.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.092+0000 7f51a599a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51a00d0e60 0x7f51a0160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59312/0 (socket says 172.21.15.60:59312) 2023-07-13T22:24:03.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a699c700 1 -- 172.21.15.60:0/2773470816 learned_addr learned my addr 172.21.15.60:0/2773470816 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:03.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a619b700 1 --2- 172.21.15.60:0/2773470816 >> v2:172.21.15.105:3300/0 conn(0x7f51a015ff70 0x7f51a00d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a599a700 1 -- 172.21.15.60:0/2773470816 >> v2:172.21.15.105:3300/0 conn(0x7f51a015ff70 msgr2=0x7f51a00d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:03.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a599a700 1 --2- 172.21.15.60:0/2773470816 >> v2:172.21.15.105:3300/0 conn(0x7f51a015ff70 0x7f51a00d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a599a700 1 -- 172.21.15.60:0/2773470816 >> v1:172.21.15.60:6789/0 conn(0x7f51a0160e60 legacy=0x7f51a00cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a599a700 1 -- 172.21.15.60:0/2773470816 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00cd8d0 con 0x7f51a00d0e60 2023-07-13T22:24:03.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a5199700 1 -- 172.21.15.60:0/2773470816 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562690011 0 0) 0x7f51a00cd3a0 con 0x7f51a0160e60 2023-07-13T22:24:03.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a619b700 1 --2- 172.21.15.60:0/2773470816 >> v2:172.21.15.105:3300/0 conn(0x7f51a015ff70 0x7f51a00d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:03.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a599a700 1 --2- 172.21.15.60:0/2773470816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 0x7f51a0160920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5190009840 tx=0x7f5190009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f078f012e5d2e74b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:03.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a5199700 1 -- 172.21.15.60:0/2773470816 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190011830 con 0x7f51a00d0e60 2023-07-13T22:24:03.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a5199700 1 -- 172.21.15.60:0/2773470816 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5190011990 con 0x7f51a00d0e60 2023-07-13T22:24:03.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.093+0000 7f51a5199700 1 -- 172.21.15.60:0/2773470816 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190011d20 con 0x7f51a00d0e60 2023-07-13T22:24:03.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.094+0000 7f51a83ff700 1 -- 172.21.15.60:0/2773470816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 msgr2=0x7f51a0160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.094+0000 7f51a83ff700 1 --2- 172.21.15.60:0/2773470816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 0x7f51a0160920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5190009840 tx=0x7f5190009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:03.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.094+0000 7f51a83ff700 1 -- 172.21.15.60:0/2773470816 shutdown_connections 2023-07-13T22:24:03.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.094+0000 7f51a83ff700 1 --2- 172.21.15.60:0/2773470816 >> v2:172.21.15.105:3300/0 conn(0x7f51a015ff70 0x7f51a00d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.094+0000 7f51a83ff700 1 --2- 172.21.15.60:0/2773470816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 0x7f51a0160920 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.094+0000 7f51a83ff700 1 -- 172.21.15.60:0/2773470816 >> 172.21.15.60:0/2773470816 conn(0x7f51a0071ed0 msgr2=0x7f51a009c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:03.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.094+0000 7f51a83ff700 1 -- 172.21.15.60:0/2773470816 shutdown_connections 2023-07-13T22:24:03.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.094+0000 7f51a83ff700 1 -- 172.21.15.60:0/2773470816 wait complete. 2023-07-13T22:24:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.096+0000 7f51a83ff700 1 Processor -- start 2023-07-13T22:24:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.096+0000 7f51a83ff700 1 -- start start 2023-07-13T22:24:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a83ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 0x7f51a01f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a83ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a015ff70 0x7f51a015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a83ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a0160e60 0x7f51a01f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a83ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a015df30 con 0x7f51a00d0e60 2023-07-13T22:24:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a83ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a015e180 con 0x7f51a0160e60 2023-07-13T22:24:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a83ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a015e460 con 0x7f51a015ff70 2023-07-13T22:24:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a619b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 0x7f51a01f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a619b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 0x7f51a01f2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59326/0 (socket says 172.21.15.60:59326) 2023-07-13T22:24:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a619b700 1 -- 172.21.15.60:0/4270421101 learned_addr learned my addr 172.21.15.60:0/4270421101 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a699c700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a0160e60 0x7f51a01f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.097+0000 7f51a599a700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a015ff70 0x7f51a015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a619b700 1 -- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a015ff70 msgr2=0x7f51a015ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a619b700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a015ff70 0x7f51a015ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a619b700 1 -- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a0160e60 msgr2=0x7f51a01f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a619b700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a0160e60 0x7f51a01f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a619b700 1 -- 172.21.15.60:0/4270421101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900094a0 con 0x7f51a00d0e60 2023-07-13T22:24:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a699c700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a0160e60 0x7f51a01f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a599a700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a015ff70 0x7f51a015ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a619b700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 0x7f51a01f2b00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c7f0 tx=0x7f519c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f5196ffd700 1 -- 172.21.15.60:0/4270421101 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c005c20 con 0x7f51a00d0e60 2023-07-13T22:24:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f5196ffd700 1 -- 172.21.15.60:0/4270421101 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c005d80 con 0x7f51a00d0e60 2023-07-13T22:24:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a015e740 con 0x7f51a00d0e60 2023-07-13T22:24:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f5196ffd700 1 -- 172.21.15.60:0/4270421101 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c011820 con 0x7f51a00d0e60 2023-07-13T22:24:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.098+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a01f3040 con 0x7f51a00d0e60 2023-07-13T22:24:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.099+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a0067a90 con 0x7f51a00d0e60 2023-07-13T22:24:03.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.100+0000 7f5196ffd700 1 -- 172.21.15.60:0/4270421101 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f519c011980 con 0x7f51a00d0e60 2023-07-13T22:24:03.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.101+0000 7f5196ffd700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f518406ace0 0x7f518406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.102+0000 7f5196ffd700 1 -- 172.21.15.60:0/4270421101 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f519c0897f0 con 0x7f51a00d0e60 2023-07-13T22:24:03.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.102+0000 7f51a599a700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f518406ace0 0x7f518406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.102+0000 7f51a599a700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f518406ace0 0x7f518406d190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51a0155570 tx=0x7f5190008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.106+0000 7f5196ffd700 1 -- 172.21.15.60:0/4270421101 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f519c054850 con 0x7f51a00d0e60 2023-07-13T22:24:03.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.315+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51a0072890 con 0x7f51a00d0e60 2023-07-13T22:24:03.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.315+0000 7f5196ffd700 1 -- 172.21.15.60:0/4270421101 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f519c057e70 con 0x7f51a00d0e60 2023-07-13T22:24:03.297 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:24:03.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f518406ace0 msgr2=0x7f518406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f518406ace0 0x7f518406d190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51a0155570 tx=0x7f5190008030 comp rx=0 tx=0).stop 2023-07-13T22:24:03.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 msgr2=0x7f51a01f2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 0x7f51a01f2b00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c7f0 tx=0x7f519c00cbb0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 shutdown_connections 2023-07-13T22:24:03.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f518406ace0 0x7f518406d190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a00d0e60 0x7f51a01f2b00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a0160e60 0x7f51a01f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 --2- 172.21.15.60:0/4270421101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a015ff70 0x7f51a015ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 >> 172.21.15.60:0/4270421101 conn(0x7f51a0071ed0 msgr2=0x7f51a006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:03.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 shutdown_connections 2023-07-13T22:24:03.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.318+0000 7f51a83ff700 1 -- 172.21.15.60:0/4270421101 wait complete. 2023-07-13T22:24:03.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:24:03.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.541+0000 7fed18bff700 1 Processor -- start 2023-07-13T22:24:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.541+0000 7fed18bff700 1 -- start start 2023-07-13T22:24:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.542+0000 7fed18bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed1415ff70 0x7fed140d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed18bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fed14161030 0x7fed140ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed18bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed140cd3a0 con 0x7fed140d0e60 2023-07-13T22:24:03.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed18bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed140d12e0 con 0x7fed1415ff70 2023-07-13T22:24:03.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed18bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed14161470 con 0x7fed14161030 2023-07-13T22:24:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed1259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed1415ff70 0x7fed140d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed1259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed1415ff70 0x7fed140d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59328/0 (socket says 172.21.15.60:59328) 2023-07-13T22:24:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed1259c700 1 -- 172.21.15.60:0/2638260129 learned_addr learned my addr 172.21.15.60:0/2638260129 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed1259c700 1 -- 172.21.15.60:0/2638260129 >> v2:172.21.15.105:3300/0 conn(0x7fed14161030 msgr2=0x7fed140ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:03.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed12d9d700 1 --2- 172.21.15.60:0/2638260129 >> v2:172.21.15.105:3300/0 conn(0x7fed14161030 0x7fed140ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed1259c700 1 --2- 172.21.15.60:0/2638260129 >> v2:172.21.15.105:3300/0 conn(0x7fed14161030 0x7fed140ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed1259c700 1 -- 172.21.15.60:0/2638260129 >> v1:172.21.15.105:6789/0 conn(0x7fed140d0e60 legacy=0x7fed14160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:03.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed1259c700 1 -- 172.21.15.60:0/2638260129 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed140cd5f0 con 0x7fed1415ff70 2023-07-13T22:24:03.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.543+0000 7fed12d9d700 1 --2- 172.21.15.60:0/2638260129 >> v2:172.21.15.105:3300/0 conn(0x7fed14161030 0x7fed140ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:03.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.544+0000 7fed1259c700 1 --2- 172.21.15.60:0/2638260129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415ff70 0x7fed140d0920 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fed04006fd0 tx=0x7fed04009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c63a387b1b8c12 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:03.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.544+0000 7fed11d9b700 1 -- 172.21.15.60:0/2638260129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04005e30 con 0x7fed1415ff70 2023-07-13T22:24:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.544+0000 7fed11d9b700 1 -- 172.21.15.60:0/2638260129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04006030 con 0x7fed1415ff70 2023-07-13T22:24:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.544+0000 7fed11d9b700 1 -- 172.21.15.60:0/2638260129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed040063c0 con 0x7fed1415ff70 2023-07-13T22:24:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.544+0000 7fed18bff700 1 -- 172.21.15.60:0/2638260129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415ff70 msgr2=0x7fed140d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.544+0000 7fed18bff700 1 --2- 172.21.15.60:0/2638260129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415ff70 0x7fed140d0920 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fed04006fd0 tx=0x7fed04009bd0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.545+0000 7fed18bff700 1 -- 172.21.15.60:0/2638260129 shutdown_connections 2023-07-13T22:24:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.545+0000 7fed18bff700 1 --2- 172.21.15.60:0/2638260129 >> v2:172.21.15.105:3300/0 conn(0x7fed14161030 0x7fed140ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.545+0000 7fed18bff700 1 --2- 172.21.15.60:0/2638260129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415ff70 0x7fed140d0920 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.545+0000 7fed18bff700 1 -- 172.21.15.60:0/2638260129 >> 172.21.15.60:0/2638260129 conn(0x7fed14071ed0 msgr2=0x7fed1409c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:03.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.545+0000 7fed18bff700 1 -- 172.21.15.60:0/2638260129 shutdown_connections 2023-07-13T22:24:03.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.545+0000 7fed18bff700 1 -- 172.21.15.60:0/2638260129 wait complete. 2023-07-13T22:24:03.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.546+0000 7fed18bff700 1 Processor -- start 2023-07-13T22:24:03.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.546+0000 7fed18bff700 1 -- start start 2023-07-13T22:24:03.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.547+0000 7fed18bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140d0e60 0x7fed141f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.547+0000 7fed18bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed1415ff70 0x7fed140cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.547+0000 7fed18bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed14161030 0x7fed14152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.547+0000 7fed18bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1415f730 con 0x7fed140d0e60 2023-07-13T22:24:03.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.547+0000 7fed18bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1414fff0 con 0x7fed14161030 2023-07-13T22:24:03.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.547+0000 7fed18bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed141502d0 con 0x7fed1415ff70 2023-07-13T22:24:03.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed1259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140d0e60 0x7fed141f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed1259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140d0e60 0x7fed141f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59336/0 (socket says 172.21.15.60:59336) 2023-07-13T22:24:03.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed1259c700 1 -- 172.21.15.60:0/2639000604 learned_addr learned my addr 172.21.15.60:0/2639000604 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:03.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed12d9d700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed14161030 0x7fed14152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed0bfff700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed1415ff70 0x7fed140cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed1259c700 1 -- 172.21.15.60:0/2639000604 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed1415ff70 msgr2=0x7fed140cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed1259c700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed1415ff70 0x7fed140cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed1259c700 1 -- 172.21.15.60:0/2639000604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed14161030 msgr2=0x7fed14152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed1259c700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed14161030 0x7fed14152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed1259c700 1 -- 172.21.15.60:0/2639000604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04009500 con 0x7fed140d0e60 2023-07-13T22:24:03.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.548+0000 7fed1259c700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140d0e60 0x7fed141f2ba0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fed04009c10 tx=0x7fed04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:03.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.549+0000 7fed0affd700 1 -- 172.21.15.60:0/2639000604 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04025430 con 0x7fed140d0e60 2023-07-13T22:24:03.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.549+0000 7fed0affd700 1 -- 172.21.15.60:0/2639000604 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04020070 con 0x7fed140d0e60 2023-07-13T22:24:03.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.549+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed141f23a0 con 0x7fed140d0e60 2023-07-13T22:24:03.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.549+0000 7fed0affd700 1 -- 172.21.15.60:0/2639000604 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed040257d0 con 0x7fed140d0e60 2023-07-13T22:24:03.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.549+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed141f2050 con 0x7fed140d0e60 2023-07-13T22:24:03.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.550+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed14067a90 con 0x7fed140d0e60 2023-07-13T22:24:03.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.551+0000 7fed0affd700 1 -- 172.21.15.60:0/2639000604 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed04022030 con 0x7fed140d0e60 2023-07-13T22:24:03.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.552+0000 7fed0affd700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecf406ace0 0x7fecf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.552+0000 7fed0affd700 1 -- 172.21.15.60:0/2639000604 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fed0408abb0 con 0x7fed140d0e60 2023-07-13T22:24:03.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.552+0000 7fed0bfff700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecf406ace0 0x7fecf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.553+0000 7fed0bfff700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecf406ace0 0x7fecf406d190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed141f66b0 tx=0x7fecfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:03.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.556+0000 7fed0affd700 1 -- 172.21.15.60:0/2639000604 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fed04055cb0 con 0x7fed140d0e60 2023-07-13T22:24:03.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.761+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed14072890 con 0x7fed140d0e60 2023-07-13T22:24:03.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.761+0000 7fed0affd700 1 -- 172.21.15.60:0/2639000604 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed040592d0 con 0x7fed140d0e60 2023-07-13T22:24:03.744 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:24:03.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecf406ace0 msgr2=0x7fecf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecf406ace0 0x7fecf406d190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed141f66b0 tx=0x7fecfc009040 comp rx=0 tx=0).stop 2023-07-13T22:24:03.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140d0e60 msgr2=0x7fed141f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140d0e60 0x7fed141f2ba0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fed04009c10 tx=0x7fed04004750 comp rx=0 tx=0).stop 2023-07-13T22:24:03.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 shutdown_connections 2023-07-13T22:24:03.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecf406ace0 0x7fecf406d190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140d0e60 0x7fed141f2ba0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed14161030 0x7fed14152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 --2- 172.21.15.60:0/2639000604 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed1415ff70 0x7fed140cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 >> 172.21.15.60:0/2639000604 conn(0x7fed14071ed0 msgr2=0x7fed1406b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:03.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 shutdown_connections 2023-07-13T22:24:03.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.764+0000 7fed18bff700 1 -- 172.21.15.60:0/2639000604 wait complete. 2023-07-13T22:24:03.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:24:03.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.987+0000 7fd8f41ff700 1 Processor -- start 2023-07-13T22:24:03.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.988+0000 7fd8f41ff700 1 -- start start 2023-07-13T22:24:03.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.989+0000 7fd8f41ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8ec160ef0 0x7fd8ec0d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.989+0000 7fd8f41ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec0d14f0 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.989+0000 7fd8f41ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec0b27a0 con 0x7fd8ec146080 2023-07-13T22:24:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.989+0000 7fd8f41ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ec0d12d0 con 0x7fd8ec160ef0 2023-07-13T22:24:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.989+0000 7fd8f179a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8ec160ef0 0x7fd8ec0d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.989+0000 7fd8f279c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd8ec0d2e70 0x7fd8ec0b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39064/0 (socket says 172.21.15.60:39064) 2023-07-13T22:24:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.989+0000 7fd8f279c700 1 -- 172.21.15.60:0/1941974833 learned_addr learned my addr 172.21.15.60:0/1941974833 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.989+0000 7fd8f179a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8ec160ef0 0x7fd8ec0d2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59352/0 (socket says 172.21.15.60:59352) 2023-07-13T22:24:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.990+0000 7fd8f0f99700 1 -- 172.21.15.60:0/1941974833 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735432051 0 0) 0x7fd8ec0d14f0 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.990+0000 7fd8f0f99700 1 -- 172.21.15.60:0/1941974833 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8d80035b0 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.990+0000 7fd8f179a700 1 -- 172.21.15.60:0/1941974833 >> v1:172.21.15.105:6789/0 conn(0x7fd8ec146080 legacy=0x7fd8ec0d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.990+0000 7fd8f179a700 1 -- 172.21.15.60:0/1941974833 >> v1:172.21.15.60:6789/0 conn(0x7fd8ec0d2e70 legacy=0x7fd8ec0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.990+0000 7fd8f179a700 1 -- 172.21.15.60:0/1941974833 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec0b2a80 con 0x7fd8ec160ef0 2023-07-13T22:24:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.990+0000 7fd8f179a700 1 --2- 172.21.15.60:0/1941974833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec160ef0 0x7fd8ec0d2930 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009840 tx=0x7fd8dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2183f1e6352a4c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.990+0000 7fd8f0f99700 1 -- 172.21.15.60:0/1941974833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc011830 con 0x7fd8ec160ef0 2023-07-13T22:24:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.990+0000 7fd8f0f99700 1 -- 172.21.15.60:0/1941974833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc011990 con 0x7fd8ec160ef0 2023-07-13T22:24:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.990+0000 7fd8f0f99700 1 -- 172.21.15.60:0/1941974833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc011d20 con 0x7fd8ec160ef0 2023-07-13T22:24:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.991+0000 7fd8f41ff700 1 -- 172.21.15.60:0/1941974833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec160ef0 msgr2=0x7fd8ec0d2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.991+0000 7fd8f41ff700 1 --2- 172.21.15.60:0/1941974833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec160ef0 0x7fd8ec0d2930 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009840 tx=0x7fd8dc009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.991+0000 7fd8f41ff700 1 -- 172.21.15.60:0/1941974833 shutdown_connections 2023-07-13T22:24:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.991+0000 7fd8f41ff700 1 --2- 172.21.15.60:0/1941974833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec160ef0 0x7fd8ec0d2930 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.991+0000 7fd8f41ff700 1 -- 172.21.15.60:0/1941974833 >> 172.21.15.60:0/1941974833 conn(0x7fd8ec071ed0 msgr2=0x7fd8ec09c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.991+0000 7fd8f41ff700 1 -- 172.21.15.60:0/1941974833 shutdown_connections 2023-07-13T22:24:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.991+0000 7fd8f41ff700 1 -- 172.21.15.60:0/1941974833 wait complete. 2023-07-13T22:24:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.993+0000 7fd8f41ff700 1 Processor -- start 2023-07-13T22:24:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.994+0000 7fd8f41ff700 1 -- start start 2023-07-13T22:24:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f41ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec0d2e70 0x7fd8ec06af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f41ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8ec146080 0x7fd8ec15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f41ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8ec160ef0 0x7fd8ec1f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f41ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0b8c40 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f41ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0b67a0 con 0x7fd8ec146080 2023-07-13T22:24:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f41ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0b5b90 con 0x7fd8ec160ef0 2023-07-13T22:24:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f1f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec0d2e70 0x7fd8ec06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f1f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec0d2e70 0x7fd8ec06af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59364/0 (socket says 172.21.15.60:59364) 2023-07-13T22:24:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f1f9b700 1 -- 172.21.15.60:0/2350149394 learned_addr learned my addr 172.21.15.60:0/2350149394 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:03.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f179a700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8ec146080 0x7fd8ec15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.995+0000 7fd8f279c700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8ec160ef0 0x7fd8ec1f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8f1f9b700 1 -- 172.21.15.60:0/2350149394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8ec160ef0 msgr2=0x7fd8ec1f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8f1f9b700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8ec160ef0 0x7fd8ec1f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8f1f9b700 1 -- 172.21.15.60:0/2350149394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8ec146080 msgr2=0x7fd8ec15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8f1f9b700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8ec146080 0x7fd8ec15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8f1f9b700 1 -- 172.21.15.60:0/2350149394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0094a0 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8f179a700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8ec146080 0x7fd8ec15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8f1f9b700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec0d2e70 0x7fd8ec06af50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800c4d0 tx=0x7fd8e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8e27fc700 1 -- 172.21.15.60:0/2350149394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80052d0 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8e27fc700 1 -- 172.21.15.60:0/2350149394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e8005430 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8e27fc700 1 -- 172.21.15.60:0/2350149394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80038f0 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec0b5e70 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.996+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec0b6430 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.997+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec0b6a80 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.998+0000 7fd8e27fc700 1 -- 172.21.15.60:0/2350149394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd8e8003a50 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.999+0000 7fd8e27fc700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd8d006ace0 0x7fd8d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:03.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.999+0000 7fd8e27fc700 1 -- 172.21.15.60:0/2350149394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd8e8088a40 con 0x7fd8ec0d2e70 2023-07-13T22:24:03.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:03.999+0000 7fd8f179a700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd8d006ace0 0x7fd8d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:03.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.000+0000 7fd8f179a700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd8d006ace0 0x7fd8d006d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec161340 tx=0x7fd8dc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:03.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.004+0000 7fd8e27fc700 1 -- 172.21.15.60:0/2350149394 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd8e8053aa0 con 0x7fd8ec0d2e70 2023-07-13T22:24:04.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.209+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8ec072890 con 0x7fd8ec0d2e70 2023-07-13T22:24:04.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.209+0000 7fd8e27fc700 1 -- 172.21.15.60:0/2350149394 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8e8017020 con 0x7fd8ec0d2e70 2023-07-13T22:24:04.191 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:24:04.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd8d006ace0 msgr2=0x7fd8d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd8d006ace0 0x7fd8d006d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec161340 tx=0x7fd8dc008030 comp rx=0 tx=0).stop 2023-07-13T22:24:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec0d2e70 msgr2=0x7fd8ec06af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec0d2e70 0x7fd8ec06af50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800c4d0 tx=0x7fd8e800c890 comp rx=0 tx=0).stop 2023-07-13T22:24:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 shutdown_connections 2023-07-13T22:24:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd8d006ace0 0x7fd8d006d190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8ec0d2e70 0x7fd8ec06af50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8ec160ef0 0x7fd8ec1f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 --2- 172.21.15.60:0/2350149394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8ec146080 0x7fd8ec15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.211+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 >> 172.21.15.60:0/2350149394 conn(0x7fd8ec071ed0 msgr2=0x7fd8ec0b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:04.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.212+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 shutdown_connections 2023-07-13T22:24:04.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.212+0000 7fd8f41ff700 1 -- 172.21.15.60:0/2350149394 wait complete. 2023-07-13T22:24:04.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:04.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.431+0000 7fcf1adff700 1 Processor -- start 2023-07-13T22:24:04.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.432+0000 7fcf1adff700 1 -- start start 2023-07-13T22:24:04.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.433+0000 7fcf1adff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcf140d1040 0x7fcf140b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.433+0000 7fcf1adff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf140b1c10 0x7fcf140b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf1adff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf140d3cb0 con 0x7fcf1415c770 2023-07-13T22:24:04.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf1adff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf140b1800 con 0x7fcf140b1c10 2023-07-13T22:24:04.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf1adff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf140b19f0 con 0x7fcf140d1040 2023-07-13T22:24:04.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf1939c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf140b1c10 0x7fcf140b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf18b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcf1415c770 0x7fcf140d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39076/0 (socket says 172.21.15.60:39076) 2023-07-13T22:24:04.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf18b9b700 1 -- 172.21.15.60:0/1570012820 learned_addr learned my addr 172.21.15.60:0/1570012820 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:04.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf13fff700 1 --2- 172.21.15.60:0/1570012820 >> v2:172.21.15.105:3300/0 conn(0x7fcf140d1040 0x7fcf140b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf1939c700 1 -- 172.21.15.60:0/1570012820 >> v2:172.21.15.105:3300/0 conn(0x7fcf140d1040 msgr2=0x7fcf140b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:04.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf1939c700 1 --2- 172.21.15.60:0/1570012820 >> v2:172.21.15.105:3300/0 conn(0x7fcf140d1040 0x7fcf140b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf1939c700 1 -- 172.21.15.60:0/1570012820 >> v1:172.21.15.60:6789/0 conn(0x7fcf1415c770 legacy=0x7fcf140d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf1939c700 1 -- 172.21.15.60:0/1570012820 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf140b2290 con 0x7fcf140b1c10 2023-07-13T22:24:04.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.434+0000 7fcf137fe700 1 -- 172.21.15.60:0/1570012820 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122006966 0 0) 0x7fcf140d3cb0 con 0x7fcf1415c770 2023-07-13T22:24:04.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf1939c700 1 --2- 172.21.15.60:0/1570012820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140b1c10 0x7fcf140b3050 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcf1408b940 tx=0x7fcf0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4a4180c2ff11a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:04.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf137fe700 1 -- 172.21.15.60:0/1570012820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00e070 con 0x7fcf140b1c10 2023-07-13T22:24:04.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf137fe700 1 -- 172.21.15.60:0/1570012820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c00bbd0 con 0x7fcf140b1c10 2023-07-13T22:24:04.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf137fe700 1 -- 172.21.15.60:0/1570012820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c012430 con 0x7fcf140b1c10 2023-07-13T22:24:04.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf1adff700 1 -- 172.21.15.60:0/1570012820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140b1c10 msgr2=0x7fcf140b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf1adff700 1 --2- 172.21.15.60:0/1570012820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140b1c10 0x7fcf140b3050 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcf1408b940 tx=0x7fcf0c00b330 comp rx=0 tx=0).stop 2023-07-13T22:24:04.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf1adff700 1 -- 172.21.15.60:0/1570012820 shutdown_connections 2023-07-13T22:24:04.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf1adff700 1 --2- 172.21.15.60:0/1570012820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140b1c10 0x7fcf140b3050 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf1adff700 1 --2- 172.21.15.60:0/1570012820 >> v2:172.21.15.105:3300/0 conn(0x7fcf140d1040 0x7fcf140b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.435+0000 7fcf1adff700 1 -- 172.21.15.60:0/1570012820 >> 172.21.15.60:0/1570012820 conn(0x7fcf1409c070 msgr2=0x7fcf140d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:04.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.436+0000 7fcf1adff700 1 -- 172.21.15.60:0/1570012820 shutdown_connections 2023-07-13T22:24:04.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.436+0000 7fcf1adff700 1 -- 172.21.15.60:0/1570012820 wait complete. 2023-07-13T22:24:04.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.438+0000 7fcf1adff700 1 Processor -- start 2023-07-13T22:24:04.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.438+0000 7fcf1adff700 1 -- start start 2023-07-13T22:24:04.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.439+0000 7fcf1adff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf140b1c10 0x7fcf14156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.439+0000 7fcf1adff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d1040 0x7fcf14154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.439+0000 7fcf1adff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf1415c770 0x7fcf141e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.439+0000 7fcf1adff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14156c80 con 0x7fcf140d1040 2023-07-13T22:24:04.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.439+0000 7fcf1adff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140cb7f0 con 0x7fcf1415c770 2023-07-13T22:24:04.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.439+0000 7fcf1adff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140cbad0 con 0x7fcf140b1c10 2023-07-13T22:24:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf13fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d1040 0x7fcf14154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf13fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d1040 0x7fcf14154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59390/0 (socket says 172.21.15.60:59390) 2023-07-13T22:24:04.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf13fff700 1 -- 172.21.15.60:0/895920413 learned_addr learned my addr 172.21.15.60:0/895920413 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:04.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf18b9b700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf140b1c10 0x7fcf14156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf1939c700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf1415c770 0x7fcf141e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf13fff700 1 -- 172.21.15.60:0/895920413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf140b1c10 msgr2=0x7fcf14156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf13fff700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf140b1c10 0x7fcf14156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf13fff700 1 -- 172.21.15.60:0/895920413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf1415c770 msgr2=0x7fcf141e62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf13fff700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf1415c770 0x7fcf141e62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf13fff700 1 -- 172.21.15.60:0/895920413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c009a70 con 0x7fcf140d1040 2023-07-13T22:24:04.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf1939c700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf1415c770 0x7fcf141e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:04.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.440+0000 7fcf13fff700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d1040 0x7fcf14154180 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000e810 tx=0x7fcf0000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:04.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.441+0000 7fcf117fa700 1 -- 172.21.15.60:0/895920413 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00013610 con 0x7fcf140d1040 2023-07-13T22:24:04.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.441+0000 7fcf117fa700 1 -- 172.21.15.60:0/895920413 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf00013790 con 0x7fcf140d1040 2023-07-13T22:24:04.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.441+0000 7fcf117fa700 1 -- 172.21.15.60:0/895920413 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00013b40 con 0x7fcf140d1040 2023-07-13T22:24:04.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.441+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf140cbdb0 con 0x7fcf140d1040 2023-07-13T22:24:04.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.441+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf140cc370 con 0x7fcf140d1040 2023-07-13T22:24:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.442+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf140b1620 con 0x7fcf140d1040 2023-07-13T22:24:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.443+0000 7fcf117fa700 1 -- 172.21.15.60:0/895920413 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcf00013ca0 con 0x7fcf140d1040 2023-07-13T22:24:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.444+0000 7fcf117fa700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcef806ace0 0x7fcef806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.444+0000 7fcf117fa700 1 -- 172.21.15.60:0/895920413 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fcf00089150 con 0x7fcf140d1040 2023-07-13T22:24:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.444+0000 7fcf18b9b700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcef806ace0 0x7fcef806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.445+0000 7fcf18b9b700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcef806ace0 0x7fcef806d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcf080010e0 tx=0x7fcf08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.448+0000 7fcf117fa700 1 -- 172.21.15.60:0/895920413 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcf00057c50 con 0x7fcf140d1040 2023-07-13T22:24:04.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.702+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf14072890 con 0x7fcf140d1040 2023-07-13T22:24:04.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.703+0000 7fcf117fa700 1 -- 172.21.15.60:0/895920413 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fcf000577e0 con 0x7fcf140d1040 2023-07-13T22:24:04.685 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:04.685 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3226044033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3226044033},{"type":"v1","addr":"172.21.15.105:6834","nonce":3226044033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5599,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2171011214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":2171011214},{"type":"v1","addr":"172.21.15.60:6836","nonce":2171011214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1623228645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1623228645},{"type":"v1","addr":"172.21.15.60:6837","nonce":1623228645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:23:14.616210+0000","modified":"2023-07-13T22:23:18.125486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5608},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5608":{"gid":5608,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/3570367595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3570367595},{"type":"v1","addr":"172.21.15.105:6835","nonce":3570367595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-13T22:24:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.705+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcef806ace0 msgr2=0x7fcef806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.705+0000 7fcf1adff700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcef806ace0 0x7fcef806d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcf080010e0 tx=0x7fcf08009040 comp rx=0 tx=0).stop 2023-07-13T22:24:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.705+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d1040 msgr2=0x7fcf14154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.705+0000 7fcf1adff700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d1040 0x7fcf14154180 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000e810 tx=0x7fcf0000eb20 comp rx=0 tx=0).stop 2023-07-13T22:24:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.705+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 shutdown_connections 2023-07-13T22:24:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.705+0000 7fcf1adff700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcef806ace0 0x7fcef806d190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.706+0000 7fcf1adff700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf1415c770 0x7fcf141e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.706+0000 7fcf1adff700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf140b1c10 0x7fcf14156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.706+0000 7fcf1adff700 1 --2- 172.21.15.60:0/895920413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d1040 0x7fcf14154180 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.706+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 >> 172.21.15.60:0/895920413 conn(0x7fcf1409c070 msgr2=0x7fcf140af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:04.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.706+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 shutdown_connections 2023-07-13T22:24:04.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.706+0000 7fcf1adff700 1 -- 172.21.15.60:0/895920413 wait complete. 2023-07-13T22:24:04.690 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2023-07-13T22:24:04.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:04.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.926+0000 7f50c47ff700 1 Processor -- start 2023-07-13T22:24:04.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.926+0000 7f50c47ff700 1 -- start start 2023-07-13T22:24:04.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.927+0000 7f50c47ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50bc0b1c10 0x7f50bc0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.927+0000 7f50c47ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc0d3cb0 con 0x7f50bc15c770 2023-07-13T22:24:04.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.927+0000 7f50c47ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc0b19f0 con 0x7f50bc0d1040 2023-07-13T22:24:04.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.927+0000 7f50c47ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50bc0b1800 con 0x7f50bc0b1c10 2023-07-13T22:24:04.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.928+0000 7f50c259b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50bc15c770 0x7f50bc0d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36446/0 (socket says 172.21.15.60:36446) 2023-07-13T22:24:04.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.928+0000 7f50c259b700 1 -- 172.21.15.60:0/3531198227 learned_addr learned my addr 172.21.15.60:0/3531198227 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:04.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.928+0000 7f50c2d9c700 1 --2- 172.21.15.60:0/3531198227 >> v2:172.21.15.105:3300/0 conn(0x7f50bc0b1c10 0x7f50bc0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.928+0000 7f50c1599700 1 -- 172.21.15.60:0/3531198227 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587017284 0 0) 0x7f50bc0d3cb0 con 0x7f50bc15c770 2023-07-13T22:24:04.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.928+0000 7f50c1599700 1 -- 172.21.15.60:0/3531198227 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a80035b0 con 0x7f50bc15c770 2023-07-13T22:24:04.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.928+0000 7f50c1599700 1 -- 172.21.15.60:0/3531198227 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2866701466 0 0) 0x7f50a80035b0 con 0x7f50bc15c770 2023-07-13T22:24:04.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.928+0000 7f50c2d9c700 1 -- 172.21.15.60:0/3531198227 >> v1:172.21.15.105:6789/0 conn(0x7f50bc0d1040 legacy=0x7f50bc0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.928+0000 7f50c2d9c700 1 -- 172.21.15.60:0/3531198227 >> v1:172.21.15.60:6789/0 conn(0x7f50bc15c770 legacy=0x7f50bc0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.928+0000 7f50c2d9c700 1 -- 172.21.15.60:0/3531198227 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0b2290 con 0x7f50bc0b1c10 2023-07-13T22:24:04.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.929+0000 7f50c2d9c700 1 --2- 172.21.15.60:0/3531198227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc0b1c10 0x7f50bc0b12c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009d90 tx=0x7f50b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe53c974647dc52 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:04.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.929+0000 7f50c1599700 1 -- 172.21.15.60:0/3531198227 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400e070 con 0x7f50bc0b1c10 2023-07-13T22:24:04.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.929+0000 7f50c1599700 1 -- 172.21.15.60:0/3531198227 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b4003a20 con 0x7f50bc0b1c10 2023-07-13T22:24:04.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.929+0000 7f50c1599700 1 -- 172.21.15.60:0/3531198227 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80017f0 con 0x7f50bc0b1c10 2023-07-13T22:24:04.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.929+0000 7f50c47ff700 1 -- 172.21.15.60:0/3531198227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc0b1c10 msgr2=0x7f50bc0b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.929+0000 7f50c47ff700 1 --2- 172.21.15.60:0/3531198227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc0b1c10 0x7f50bc0b12c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009d90 tx=0x7f50b400b330 comp rx=0 tx=0).stop 2023-07-13T22:24:04.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.930+0000 7f50c47ff700 1 -- 172.21.15.60:0/3531198227 shutdown_connections 2023-07-13T22:24:04.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.930+0000 7f50c47ff700 1 --2- 172.21.15.60:0/3531198227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc0b1c10 0x7f50bc0b12c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.930+0000 7f50c47ff700 1 -- 172.21.15.60:0/3531198227 >> 172.21.15.60:0/3531198227 conn(0x7f50bc09c070 msgr2=0x7f50bc0d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:04.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.930+0000 7f50c47ff700 1 -- 172.21.15.60:0/3531198227 shutdown_connections 2023-07-13T22:24:04.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.930+0000 7f50c47ff700 1 -- 172.21.15.60:0/3531198227 wait complete. 2023-07-13T22:24:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.931+0000 7f50c47ff700 1 Processor -- start 2023-07-13T22:24:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.932+0000 7f50c47ff700 1 -- start start 2023-07-13T22:24:04.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c47ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50bc0b1c10 0x7f50bc153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c47ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50bc0d1040 0x7f50bc1f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c47ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc15c770 0x7f50bc0c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c47ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc15e290 con 0x7f50bc0b1c10 2023-07-13T22:24:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c47ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc15d680 con 0x7f50bc15c770 2023-07-13T22:24:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c47ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc15d960 con 0x7f50bc0d1040 2023-07-13T22:24:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c259b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50bc0b1c10 0x7f50bc153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c259b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50bc0b1c10 0x7f50bc153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37270/0 (socket says 172.21.15.60:37270) 2023-07-13T22:24:04.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c259b700 1 -- 172.21.15.60:0/1530694664 learned_addr learned my addr 172.21.15.60:0/1530694664 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:04.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c2d9c700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc15c770 0x7f50bc0c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c1d9a700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50bc0d1040 0x7f50bc1f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c259b700 1 -- 172.21.15.60:0/1530694664 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50bc0d1040 msgr2=0x7f50bc1f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c259b700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50bc0d1040 0x7f50bc1f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c259b700 1 -- 172.21.15.60:0/1530694664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc15c770 msgr2=0x7f50bc0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c259b700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc15c770 0x7f50bc0c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.933+0000 7f50c259b700 1 -- 172.21.15.60:0/1530694664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b4009a70 con 0x7f50bc0b1c10 2023-07-13T22:24:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.934+0000 7f50c259b700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50bc0b1c10 0x7f50bc153d00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c570 tx=0x7f50b800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.934+0000 7f50b2ffd700 1 -- 172.21.15.60:0/1530694664 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80074e0 con 0x7f50bc0b1c10 2023-07-13T22:24:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.934+0000 7f50b2ffd700 1 -- 172.21.15.60:0/1530694664 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b8007640 con 0x7f50bc0b1c10 2023-07-13T22:24:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.934+0000 7f50b2ffd700 1 -- 172.21.15.60:0/1530694664 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8005a70 con 0x7f50bc0b1c10 2023-07-13T22:24:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.934+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc15dc40 con 0x7f50bc0b1c10 2023-07-13T22:24:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.934+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc1f1db0 con 0x7f50bc0b1c10 2023-07-13T22:24:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.935+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc1f1f70 con 0x7f50bc0b1c10 2023-07-13T22:24:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.936+0000 7f50b2ffd700 1 -- 172.21.15.60:0/1530694664 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50b8005bd0 con 0x7f50bc0b1c10 2023-07-13T22:24:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.937+0000 7f50b2ffd700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50a006ace0 0x7f50a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.937+0000 7f50b2ffd700 1 -- 172.21.15.60:0/1530694664 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f50b8088bb0 con 0x7f50bc0b1c10 2023-07-13T22:24:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.937+0000 7f50c1d9a700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50a006ace0 0x7f50a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.938+0000 7f50c1d9a700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50a006ace0 0x7f50a006d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50bc157230 tx=0x7f50ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:04.941+0000 7f50b2ffd700 1 -- 172.21.15.60:0/1530694664 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f50b8053b80 con 0x7f50bc0b1c10 2023-07-13T22:24:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.149+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50bc072890 con 0x7f50bc0b1c10 2023-07-13T22:24:05.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.150+0000 7f50b2ffd700 1 -- 172.21.15.60:0/1530694664 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18353 (secure 0 0 0) 0x7f50b8017090 con 0x7f50bc0b1c10 2023-07-13T22:24:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:05.133 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":51,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:44.502069+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.21","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50a006ace0 msgr2=0x7f50a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50a006ace0 0x7f50a006d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50bc157230 tx=0x7f50ac009040 comp rx=0 tx=0).stop 2023-07-13T22:24:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50bc0b1c10 msgr2=0x7f50bc153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50bc0b1c10 0x7f50bc153d00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c570 tx=0x7f50b800c930 comp rx=0 tx=0).stop 2023-07-13T22:24:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 shutdown_connections 2023-07-13T22:24:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50a006ace0 0x7f50a006d190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50bc0b1c10 0x7f50bc153d00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50bc15c770 0x7f50bc0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 --2- 172.21.15.60:0/1530694664 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50bc0d1040 0x7f50bc1f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 >> 172.21.15.60:0/1530694664 conn(0x7f50bc09c070 msgr2=0x7f50bc0d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 shutdown_connections 2023-07-13T22:24:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.153+0000 7f50c47ff700 1 -- 172.21.15.60:0/1530694664 wait complete. 2023-07-13T22:24:05.153 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:24:05.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:05.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.374+0000 7fde075ff700 1 Processor -- start 2023-07-13T22:24:05.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.374+0000 7fde075ff700 1 -- start start 2023-07-13T22:24:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.376+0000 7fde075ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000d1040 0x7fde000b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.376+0000 7fde075ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000d3cb0 con 0x7fde0015c770 2023-07-13T22:24:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.376+0000 7fde075ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000b19f0 con 0x7fde000b1c10 2023-07-13T22:24:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.376+0000 7fde075ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000b1800 con 0x7fde000d1040 2023-07-13T22:24:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.376+0000 7fde0539b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fde0015c770 0x7fde000d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36456/0 (socket says 172.21.15.60:36456) 2023-07-13T22:24:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.376+0000 7fde0539b700 1 -- 172.21.15.60:0/3136728759 learned_addr learned my addr 172.21.15.60:0/3136728759 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:05.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.376+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886037313 0 0) 0x7fde000d3cb0 con 0x7fde0015c770 2023-07-13T22:24:05.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.376+0000 7fde04b9a700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000d1040 0x7fde000b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.376+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddec0035b0 con 0x7fde0015c770 2023-07-13T22:24:05.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3319476389 0 0) 0x7fddec0035b0 con 0x7fde0015c770 2023-07-13T22:24:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000d1040 msgr2=0x7fde000b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000d1040 0x7fde000b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 >> v1:172.21.15.105:6789/0 conn(0x7fde000b1c10 legacy=0x7fde000b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000b2290 con 0x7fde0015c770 2023-07-13T22:24:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fddfc0017f0 con 0x7fde0015c770 2023-07-13T22:24:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 >> v1:172.21.15.60:6789/0 conn(0x7fde0015c770 legacy=0x7fde000d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddec003a30 0x7fddec003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec004850 0x7fddec004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000d1040 0x7fddec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec005530 con 0x7fddec003a30 2023-07-13T22:24:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec0056a0 con 0x7fddec004850 2023-07-13T22:24:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec00bf50 con 0x7fde000d1040 2023-07-13T22:24:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fde0539b700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddec003a30 0x7fddec003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fde04b9a700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec004850 0x7fddec004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fde05b9c700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000d1040 0x7fddec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fde0539b700 1 -- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000d1040 msgr2=0x7fddec00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.377+0000 7fde0539b700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000d1040 0x7fddec00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fde0539b700 1 -- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec004850 msgr2=0x7fddec004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fde0539b700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec004850 0x7fddec004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fde0539b700 1 -- 172.21.15.60:0/3136728759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec0035b0 con 0x7fddec003a30 2023-07-13T22:24:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fde04b9a700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec004850 0x7fddec004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fde0539b700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddec003a30 0x7fddec003e40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c190 tx=0x7fddec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facade40ce63552d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00ccf0 con 0x7fddec003a30 2023-07-13T22:24:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc00ce70 con 0x7fddec003a30 2023-07-13T22:24:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fddf7fff700 1 -- 172.21.15.60:0/3136728759 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc005a50 con 0x7fddec003a30 2023-07-13T22:24:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fde075ff700 1 -- 172.21.15.60:0/3136728759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddec003a30 msgr2=0x7fddec003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fde075ff700 1 --2- 172.21.15.60:0/3136728759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddec003a30 0x7fddec003e40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c190 tx=0x7fddec005880 comp rx=0 tx=0).stop 2023-07-13T22:24:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.378+0000 7fde05b9c700 1 -- 172.21.15.60:0/3136728759 reap_dead start 2023-07-13T22:24:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.379+0000 7fde075ff700 1 -- 172.21.15.60:0/3136728759 shutdown_connections 2023-07-13T22:24:05.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.379+0000 7fde075ff700 1 -- 172.21.15.60:0/3136728759 >> 172.21.15.60:0/3136728759 conn(0x7fde0009c070 msgr2=0x7fde000d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:05.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.379+0000 7fde075ff700 1 -- 172.21.15.60:0/3136728759 shutdown_connections 2023-07-13T22:24:05.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.379+0000 7fde075ff700 1 -- 172.21.15.60:0/3136728759 wait complete. 2023-07-13T22:24:05.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.380+0000 7fde075ff700 1 Processor -- start 2023-07-13T22:24:05.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.381+0000 7fde075ff700 1 -- start start 2023-07-13T22:24:05.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.381+0000 7fde075ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec003a30 0x7fde00152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.381+0000 7fde075ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000c6880 0x7fde000c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde075ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde000c88b0 0x7fde000c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde075ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000cb1d0 con 0x7fde000c88b0 2023-07-13T22:24:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde075ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000c7230 con 0x7fddec003a30 2023-07-13T22:24:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde075ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000c96e0 con 0x7fde000c6880 2023-07-13T22:24:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde05b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde000c88b0 0x7fde000c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde05b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde000c88b0 0x7fde000c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37292/0 (socket says 172.21.15.60:37292) 2023-07-13T22:24:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde05b9c700 1 -- 172.21.15.60:0/3206056894 learned_addr learned my addr 172.21.15.60:0/3206056894 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde0539b700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec003a30 0x7fde00152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde04b9a700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000c6880 0x7fde000c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde05b9c700 1 -- 172.21.15.60:0/3206056894 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000c6880 msgr2=0x7fde000c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde05b9c700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000c6880 0x7fde000c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde05b9c700 1 -- 172.21.15.60:0/3206056894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec003a30 msgr2=0x7fde00152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde05b9c700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec003a30 0x7fde00152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde05b9c700 1 -- 172.21.15.60:0/3206056894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000ca830 con 0x7fde000c88b0 2023-07-13T22:24:05.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde04b9a700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000c6880 0x7fde000c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:05.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.382+0000 7fde05b9c700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde000c88b0 0x7fde000c8d20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fde00066f00 tx=0x7fddf800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:05.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.383+0000 7fddf5ffb700 1 -- 172.21.15.60:0/3206056894 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8003c10 con 0x7fde000c88b0 2023-07-13T22:24:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.383+0000 7fddf5ffb700 1 -- 172.21.15.60:0/3206056894 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf8003d70 con 0x7fde000c88b0 2023-07-13T22:24:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.383+0000 7fddf5ffb700 1 -- 172.21.15.60:0/3206056894 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8005310 con 0x7fde000c88b0 2023-07-13T22:24:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.383+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde000cab10 con 0x7fde000c88b0 2023-07-13T22:24:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.383+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde00153260 con 0x7fde000c88b0 2023-07-13T22:24:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.384+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde000cb310 con 0x7fde000c88b0 2023-07-13T22:24:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.385+0000 7fddf5ffb700 1 -- 172.21.15.60:0/3206056894 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fddf8005470 con 0x7fde000c88b0 2023-07-13T22:24:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.386+0000 7fddf5ffb700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdde406ace0 0x7fdde406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.386+0000 7fddf5ffb700 1 -- 172.21.15.60:0/3206056894 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fddf80893f0 con 0x7fde000c88b0 2023-07-13T22:24:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.386+0000 7fde0539b700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdde406ace0 0x7fdde406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.386+0000 7fde0539b700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdde406ace0 0x7fdde406d190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fddfc008fd0 tx=0x7fddfc007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.390+0000 7fddf5ffb700 1 -- 172.21.15.60:0/3206056894 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fde000cb310 con 0x7fde000c88b0 2023-07-13T22:24:05.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.642+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde000cb310 con 0x7fde000c88b0 2023-07-13T22:24:05.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.643+0000 7fddf5ffb700 1 -- 172.21.15.60:0/3206056894 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fde000cb310 con 0x7fde000c88b0 2023-07-13T22:24:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:05.625 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3226044033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3226044033},{"type":"v1","addr":"172.21.15.105:6834","nonce":3226044033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5599,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2171011214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":2171011214},{"type":"v1","addr":"172.21.15.60:6836","nonce":2171011214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1623228645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1623228645},{"type":"v1","addr":"172.21.15.60:6837","nonce":1623228645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:23:14.616210+0000","modified":"2023-07-13T22:23:18.125486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5608},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5608":{"gid":5608,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/3570367595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3570367595},{"type":"v1","addr":"172.21.15.105:6835","nonce":3570367595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-13T22:24:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdde406ace0 msgr2=0x7fdde406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdde406ace0 0x7fdde406d190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fddfc008fd0 tx=0x7fddfc007000 comp rx=0 tx=0).stop 2023-07-13T22:24:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde000c88b0 msgr2=0x7fde000c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde000c88b0 0x7fde000c8d20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fde00066f00 tx=0x7fddf800ed50 comp rx=0 tx=0).stop 2023-07-13T22:24:05.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 shutdown_connections 2023-07-13T22:24:05.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdde406ace0 0x7fdde406d190 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde000c88b0 0x7fde000c8d20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec003a30 0x7fde00152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 --2- 172.21.15.60:0/3206056894 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde000c6880 0x7fde000c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 >> 172.21.15.60:0/3206056894 conn(0x7fde0009c070 msgr2=0x7fde0015b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:05.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 shutdown_connections 2023-07-13T22:24:05.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.645+0000 7fde075ff700 1 -- 172.21.15.60:0/3206056894 wait complete. 2023-07-13T22:24:05.629 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2023-07-13T22:24:05.645 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:05.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.865+0000 7fa7885ff700 1 Processor -- start 2023-07-13T22:24:05.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.865+0000 7fa7885ff700 1 -- start start 2023-07-13T22:24:05.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.867+0000 7fa7885ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7801653c0 0x7fa7801677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.867+0000 7fa7885ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78006bb80 con 0x7fa78006beb0 2023-07-13T22:24:05.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.867+0000 7fa7885ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa780160250 con 0x7fa78006b020 2023-07-13T22:24:05.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.867+0000 7fa7885ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa780160040 con 0x7fa7801653c0 2023-07-13T22:24:05.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.867+0000 7fa78639b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa78006beb0 0x7fa78006c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36460/0 (socket says 172.21.15.60:36460) 2023-07-13T22:24:05.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.867+0000 7fa78639b700 1 -- 172.21.15.60:0/792564979 learned_addr learned my addr 172.21.15.60:0/792564979 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:05.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.867+0000 7fa786b9c700 1 --2- 172.21.15.60:0/792564979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7801653c0 0x7fa7801677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.867+0000 7fa785399700 1 -- 172.21.15.60:0/792564979 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995438291 0 0) 0x7fa78006bb80 con 0x7fa78006beb0 2023-07-13T22:24:05.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa785399700 1 -- 172.21.15.60:0/792564979 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa76c0035b0 con 0x7fa78006beb0 2023-07-13T22:24:05.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa785399700 1 -- 172.21.15.60:0/792564979 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588325377 0 0) 0x7fa780160250 con 0x7fa78006b020 2023-07-13T22:24:05.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa785399700 1 -- 172.21.15.60:0/792564979 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa78006bb80 con 0x7fa78006b020 2023-07-13T22:24:05.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa785399700 1 -- 172.21.15.60:0/792564979 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3671826942 0 0) 0x7fa76c0035b0 con 0x7fa78006beb0 2023-07-13T22:24:05.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa786b9c700 1 -- 172.21.15.60:0/792564979 >> v1:172.21.15.105:6789/0 conn(0x7fa78006b020 legacy=0x7fa78006b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa786b9c700 1 -- 172.21.15.60:0/792564979 >> v1:172.21.15.60:6789/0 conn(0x7fa78006beb0 legacy=0x7fa78006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa786b9c700 1 -- 172.21.15.60:0/792564979 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa780160530 con 0x7fa7801653c0 2023-07-13T22:24:05.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa786b9c700 1 --2- 172.21.15.60:0/792564979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7801653c0 0x7fa7801677a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa778009dc0 tx=0x7fa77800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e5b624e09bedcf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:05.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa785399700 1 -- 172.21.15.60:0/792564979 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77800e070 con 0x7fa7801653c0 2023-07-13T22:24:05.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa785399700 1 -- 172.21.15.60:0/792564979 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa778003a20 con 0x7fa7801653c0 2023-07-13T22:24:05.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.868+0000 7fa785399700 1 -- 172.21.15.60:0/792564979 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c001670 con 0x7fa7801653c0 2023-07-13T22:24:05.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.869+0000 7fa7885ff700 1 -- 172.21.15.60:0/792564979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7801653c0 msgr2=0x7fa7801677a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.869+0000 7fa7885ff700 1 --2- 172.21.15.60:0/792564979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7801653c0 0x7fa7801677a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa778009dc0 tx=0x7fa77800b330 comp rx=0 tx=0).stop 2023-07-13T22:24:05.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.869+0000 7fa7885ff700 1 -- 172.21.15.60:0/792564979 shutdown_connections 2023-07-13T22:24:05.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.869+0000 7fa7885ff700 1 --2- 172.21.15.60:0/792564979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7801653c0 0x7fa7801677a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.869+0000 7fa7885ff700 1 -- 172.21.15.60:0/792564979 >> 172.21.15.60:0/792564979 conn(0x7fa78006d890 msgr2=0x7fa78006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:05.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.869+0000 7fa7885ff700 1 -- 172.21.15.60:0/792564979 shutdown_connections 2023-07-13T22:24:05.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.869+0000 7fa7885ff700 1 -- 172.21.15.60:0/792564979 wait complete. 2023-07-13T22:24:05.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.871+0000 7fa7885ff700 1 Processor -- start 2023-07-13T22:24:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.871+0000 7fa7885ff700 1 -- start start 2023-07-13T22:24:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa7885ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa78006b020 0x7fa780152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa7885ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78006beb0 0x7fa78014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa7885ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7801653c0 0x7fa7800b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa7885ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7801530f0 con 0x7fa78006beb0 2023-07-13T22:24:05.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa7885ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7801533d0 con 0x7fa7801653c0 2023-07-13T22:24:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa7885ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7801536b0 con 0x7fa78006b020 2023-07-13T22:24:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa785b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78006beb0 0x7fa78014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa785b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78006beb0 0x7fa78014fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37302/0 (socket says 172.21.15.60:37302) 2023-07-13T22:24:05.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa785b9a700 1 -- 172.21.15.60:0/289610703 learned_addr learned my addr 172.21.15.60:0/289610703 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:05.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa78639b700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa78006b020 0x7fa780152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.872+0000 7fa786b9c700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7801653c0 0x7fa7800b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa785b9a700 1 -- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa78006b020 msgr2=0x7fa780152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa785b9a700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa78006b020 0x7fa780152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa785b9a700 1 -- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7801653c0 msgr2=0x7fa7800b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:05.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa785b9a700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7801653c0 0x7fa7800b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:05.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa785b9a700 1 -- 172.21.15.60:0/289610703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009a70 con 0x7fa78006beb0 2023-07-13T22:24:05.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa78639b700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa78006b020 0x7fa780152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa786b9c700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7801653c0 0x7fa7800b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa785b9a700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78006beb0 0x7fa78014fa90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa77000c700 tx=0x7fa77000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa776ffd700 1 -- 172.21.15.60:0/289610703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7700052d0 con 0x7fa78006beb0 2023-07-13T22:24:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa776ffd700 1 -- 172.21.15.60:0/289610703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa770005430 con 0x7fa78006beb0 2023-07-13T22:24:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa776ffd700 1 -- 172.21.15.60:0/289610703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa770003880 con 0x7fa78006beb0 2023-07-13T22:24:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7801f5ae0 con 0x7fa78006beb0 2023-07-13T22:24:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.873+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7801f6000 con 0x7fa78006beb0 2023-07-13T22:24:05.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.874+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa780153510 con 0x7fa78006beb0 2023-07-13T22:24:05.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.875+0000 7fa776ffd700 1 -- 172.21.15.60:0/289610703 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7700039e0 con 0x7fa78006beb0 2023-07-13T22:24:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.876+0000 7fa776ffd700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa76406ace0 0x7fa76406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.876+0000 7fa776ffd700 1 -- 172.21.15.60:0/289610703 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa770088940 con 0x7fa78006beb0 2023-07-13T22:24:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.876+0000 7fa78639b700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa76406ace0 0x7fa76406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.877+0000 7fa78639b700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa76406ace0 0x7fa76406d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa77c001860 tx=0x7fa77c0016d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:05.880+0000 7fa776ffd700 1 -- 172.21.15.60:0/289610703 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa780153510 con 0x7fa78006beb0 2023-07-13T22:24:06.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.132+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa780153510 con 0x7fa78006beb0 2023-07-13T22:24:06.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.133+0000 7fa776ffd700 1 -- 172.21.15.60:0/289610703 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fa780153510 con 0x7fa78006beb0 2023-07-13T22:24:06.115 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:06.116 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3226044033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3226044033},{"type":"v1","addr":"172.21.15.105:6834","nonce":3226044033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5599,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2171011214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":2171011214},{"type":"v1","addr":"172.21.15.60:6836","nonce":2171011214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1623228645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1623228645},{"type":"v1","addr":"172.21.15.60:6837","nonce":1623228645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:23:14.616210+0000","modified":"2023-07-13T22:23:18.125486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5608},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5608":{"gid":5608,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/3570367595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3570367595},{"type":"v1","addr":"172.21.15.105:6835","nonce":3570367595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-13T22:24:06.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa76406ace0 msgr2=0x7fa76406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa76406ace0 0x7fa76406d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa77c001860 tx=0x7fa77c0016d0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78006beb0 msgr2=0x7fa78014fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78006beb0 0x7fa78014fa90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa77000c700 tx=0x7fa77000cac0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 shutdown_connections 2023-07-13T22:24:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa76406ace0 0x7fa76406d190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7801653c0 0x7fa7800b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa78006b020 0x7fa780152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 --2- 172.21.15.60:0/289610703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa78006beb0 0x7fa78014fa90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.135+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 >> 172.21.15.60:0/289610703 conn(0x7fa78006d890 msgr2=0x7fa780066460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.136+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 shutdown_connections 2023-07-13T22:24:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.136+0000 7fa7885ff700 1 -- 172.21.15.60:0/289610703 wait complete. 2023-07-13T22:24:06.120 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2023-07-13T22:24:06.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:06.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.357+0000 7f8a4f1ff700 1 Processor -- start 2023-07-13T22:24:06.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.357+0000 7f8a4f1ff700 1 -- start start 2023-07-13T22:24:06.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.358+0000 7f8a4f1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8a4815c770 0x7f8a480b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.358+0000 7f8a4f1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a480b1c10 0x7f8a480d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.358+0000 7f8a4f1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a480d3cb0 con 0x7f8a480d0e70 2023-07-13T22:24:06.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.358+0000 7f8a4f1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a480b1800 con 0x7f8a4815c770 2023-07-13T22:24:06.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.358+0000 7f8a4f1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a480d12f0 con 0x7f8a480b1c10 2023-07-13T22:24:06.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.358+0000 7f8a4cf9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8a4815c770 0x7f8a480b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.358+0000 7f8a4cf9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8a4815c770 0x7f8a480b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37314/0 (socket says 172.21.15.60:37314) 2023-07-13T22:24:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.358+0000 7f8a4cf9b700 1 -- 172.21.15.60:0/1663245495 learned_addr learned my addr 172.21.15.60:0/1663245495 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.358+0000 7f8a47fff700 1 --2- 172.21.15.60:0/1663245495 >> v2:172.21.15.105:3300/0 conn(0x7f8a480b1c10 0x7f8a480d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.359+0000 7f8a4cf9b700 1 -- 172.21.15.60:0/1663245495 >> v2:172.21.15.105:3300/0 conn(0x7f8a480b1c10 msgr2=0x7f8a480d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.359+0000 7f8a4cf9b700 1 --2- 172.21.15.60:0/1663245495 >> v2:172.21.15.105:3300/0 conn(0x7f8a480b1c10 0x7f8a480d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.359+0000 7f8a4cf9b700 1 -- 172.21.15.60:0/1663245495 >> v1:172.21.15.105:6789/0 conn(0x7f8a480d0e70 legacy=0x7f8a480af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.359+0000 7f8a4cf9b700 1 -- 172.21.15.60:0/1663245495 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a480b19f0 con 0x7f8a4815c770 2023-07-13T22:24:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.359+0000 7f8a4cf9b700 1 --2- 172.21.15.60:0/1663245495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a4815c770 0x7f8a480b12c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8a4808b940 tx=0x7f8a3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3143ecfc03adeb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.359+0000 7f8a477fe700 1 -- 172.21.15.60:0/1663245495 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c005e30 con 0x7f8a4815c770 2023-07-13T22:24:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.359+0000 7f8a477fe700 1 -- 172.21.15.60:0/1663245495 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c004030 con 0x7f8a4815c770 2023-07-13T22:24:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.359+0000 7f8a477fe700 1 -- 172.21.15.60:0/1663245495 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0043e0 con 0x7f8a4815c770 2023-07-13T22:24:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.360+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1663245495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a4815c770 msgr2=0x7f8a480b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.360+0000 7f8a4f1ff700 1 --2- 172.21.15.60:0/1663245495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a4815c770 0x7f8a480b12c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8a4808b940 tx=0x7f8a3c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.360+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1663245495 shutdown_connections 2023-07-13T22:24:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.360+0000 7f8a4f1ff700 1 --2- 172.21.15.60:0/1663245495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a4815c770 0x7f8a480b12c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.360+0000 7f8a4f1ff700 1 --2- 172.21.15.60:0/1663245495 >> v2:172.21.15.105:3300/0 conn(0x7f8a480b1c10 0x7f8a480d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.360+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1663245495 >> 172.21.15.60:0/1663245495 conn(0x7f8a4809c070 msgr2=0x7f8a480d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.360+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1663245495 shutdown_connections 2023-07-13T22:24:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.360+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1663245495 wait complete. 2023-07-13T22:24:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.362+0000 7f8a4f1ff700 1 Processor -- start 2023-07-13T22:24:06.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.362+0000 7f8a4f1ff700 1 -- start start 2023-07-13T22:24:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.363+0000 7f8a4f1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a480b1c10 0x7f8a481ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.363+0000 7f8a4f1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a480d0e70 0x7f8a480c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.363+0000 7f8a4f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a4815c770 0x7f8a481f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.363+0000 7f8a4f1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a480cd5d0 con 0x7f8a480b1c10 2023-07-13T22:24:06.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.363+0000 7f8a4f1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a481f6210 con 0x7f8a480d0e70 2023-07-13T22:24:06.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.363+0000 7f8a4f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a480b8200 con 0x7f8a4815c770 2023-07-13T22:24:06.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4cf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a480b1c10 0x7f8a481ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4cf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a480b1c10 0x7f8a481ef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37320/0 (socket says 172.21.15.60:37320) 2023-07-13T22:24:06.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4cf9b700 1 -- 172.21.15.60:0/1824663761 learned_addr learned my addr 172.21.15.60:0/1824663761 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a47fff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a480d0e70 0x7f8a480c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4d79c700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a4815c770 0x7f8a481f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4cf9b700 1 -- 172.21.15.60:0/1824663761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a4815c770 msgr2=0x7f8a481f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4cf9b700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a4815c770 0x7f8a481f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4cf9b700 1 -- 172.21.15.60:0/1824663761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a480d0e70 msgr2=0x7f8a480c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4cf9b700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a480d0e70 0x7f8a480c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4cf9b700 1 -- 172.21.15.60:0/1824663761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c009500 con 0x7f8a480b1c10 2023-07-13T22:24:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a47fff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a480d0e70 0x7f8a480c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.364+0000 7f8a4cf9b700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a480b1c10 0x7f8a481ef3e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009c10 tx=0x7f8a3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.365+0000 7f8a457fa700 1 -- 172.21.15.60:0/1824663761 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c003d90 con 0x7f8a480b1c10 2023-07-13T22:24:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.365+0000 7f8a457fa700 1 -- 172.21.15.60:0/1824663761 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c01d430 con 0x7f8a480b1c10 2023-07-13T22:24:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.365+0000 7f8a457fa700 1 -- 172.21.15.60:0/1824663761 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c01d7e0 con 0x7f8a480b1c10 2023-07-13T22:24:06.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.365+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a480b84e0 con 0x7f8a480b1c10 2023-07-13T22:24:06.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.365+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a480b8aa0 con 0x7f8a480b1c10 2023-07-13T22:24:06.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.366+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a480cd780 con 0x7f8a480b1c10 2023-07-13T22:24:06.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.367+0000 7f8a457fa700 1 -- 172.21.15.60:0/1824663761 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a3c01d940 con 0x7f8a480b1c10 2023-07-13T22:24:06.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.368+0000 7f8a457fa700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8a2c06ace0 0x7f8a2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.368+0000 7f8a457fa700 1 -- 172.21.15.60:0/1824663761 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8a3c08a8b0 con 0x7f8a480b1c10 2023-07-13T22:24:06.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.368+0000 7f8a47fff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8a2c06ace0 0x7f8a2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.369+0000 7f8a47fff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8a2c06ace0 0x7f8a2c06d190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a481ef920 tx=0x7f8a34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:06.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.372+0000 7f8a457fa700 1 -- 172.21.15.60:0/1824663761 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8a480cd780 con 0x7f8a480b1c10 2023-07-13T22:24:06.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.577+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a48072890 con 0x7f8a480b1c10 2023-07-13T22:24:06.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.579+0000 7f8a457fa700 1 -- 172.21.15.60:0/1824663761 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18353 (secure 0 0 0) 0x7f8a3c055910 con 0x7f8a480b1c10 2023-07-13T22:24:06.561 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:06.561 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":51,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:23:44.502069+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:23:12.304337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-13T22:23:13.598583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.21","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8a2c06ace0 msgr2=0x7f8a2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8a2c06ace0 0x7f8a2c06d190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a481ef920 tx=0x7f8a34008040 comp rx=0 tx=0).stop 2023-07-13T22:24:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a480b1c10 msgr2=0x7f8a481ef3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a480b1c10 0x7f8a481ef3e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009c10 tx=0x7f8a3c004750 comp rx=0 tx=0).stop 2023-07-13T22:24:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 shutdown_connections 2023-07-13T22:24:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8a2c06ace0 0x7f8a2c06d190 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a480b1c10 0x7f8a481ef3e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a4815c770 0x7f8a481f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 --2- 172.21.15.60:0/1824663761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a480d0e70 0x7f8a480c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 >> 172.21.15.60:0/1824663761 conn(0x7f8a4809c070 msgr2=0x7f8a480b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:06.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 shutdown_connections 2023-07-13T22:24:06.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.581+0000 7f8a4f1ff700 1 -- 172.21.15.60:0/1824663761 wait complete. 2023-07-13T22:24:06.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:24:06.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.804+0000 7f47fa5ff700 1 Processor -- start 2023-07-13T22:24:06.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.804+0000 7f47fa5ff700 1 -- start start 2023-07-13T22:24:06.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.805+0000 7f47fa5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47f40b1b90 0x7f47f40b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.805+0000 7f47fa5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f40af890 con 0x7f47f40d0fa0 2023-07-13T22:24:06.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.805+0000 7f47fa5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f415ca80 con 0x7f47f415c6b0 2023-07-13T22:24:06.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.805+0000 7f47fa5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f40afae0 con 0x7f47f40b1b90 2023-07-13T22:24:06.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.806+0000 7f47f37fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f47f40d0fa0 0x7f47f40b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36474/0 (socket says 172.21.15.60:36474) 2023-07-13T22:24:06.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.806+0000 7f47f37fe700 1 -- 172.21.15.60:0/21069324 learned_addr learned my addr 172.21.15.60:0/21069324 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:06.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.806+0000 7f47f8b9c700 1 --2- 172.21.15.60:0/21069324 >> v2:172.21.15.60:3300/0 conn(0x7f47f40b1b90 0x7f47f40b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.806+0000 7f47f2ffd700 1 -- 172.21.15.60:0/21069324 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831919555 0 0) 0x7f47f40af890 con 0x7f47f40d0fa0 2023-07-13T22:24:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.806+0000 7f47f8b9c700 1 -- 172.21.15.60:0/21069324 >> v1:172.21.15.105:6789/0 conn(0x7f47f415c6b0 legacy=0x7f47f40d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.806+0000 7f47f8b9c700 1 -- 172.21.15.60:0/21069324 >> v1:172.21.15.60:6789/0 conn(0x7f47f40d0fa0 legacy=0x7f47f40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.806+0000 7f47f8b9c700 1 -- 172.21.15.60:0/21069324 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40afdc0 con 0x7f47f40b1b90 2023-07-13T22:24:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.806+0000 7f47f8b9c700 1 --2- 172.21.15.60:0/21069324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f40b1b90 0x7f47f40b2cb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009d90 tx=0x7f47ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d834b2df76d07b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47f2ffd700 1 -- 172.21.15.60:0/21069324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00e070 con 0x7f47f40b1b90 2023-07-13T22:24:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47f2ffd700 1 -- 172.21.15.60:0/21069324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec00bbd0 con 0x7f47f40b1b90 2023-07-13T22:24:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47f2ffd700 1 -- 172.21.15.60:0/21069324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec012430 con 0x7f47f40b1b90 2023-07-13T22:24:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47fa5ff700 1 -- 172.21.15.60:0/21069324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f40b1b90 msgr2=0x7f47f40b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47fa5ff700 1 --2- 172.21.15.60:0/21069324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f40b1b90 0x7f47f40b2cb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009d90 tx=0x7f47ec00b330 comp rx=0 tx=0).stop 2023-07-13T22:24:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47fa5ff700 1 -- 172.21.15.60:0/21069324 shutdown_connections 2023-07-13T22:24:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47fa5ff700 1 --2- 172.21.15.60:0/21069324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f40b1b90 0x7f47f40b2cb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47fa5ff700 1 -- 172.21.15.60:0/21069324 >> 172.21.15.60:0/21069324 conn(0x7f47f409c030 msgr2=0x7f47f40d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47fa5ff700 1 -- 172.21.15.60:0/21069324 shutdown_connections 2023-07-13T22:24:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.807+0000 7f47fa5ff700 1 -- 172.21.15.60:0/21069324 wait complete. 2023-07-13T22:24:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.809+0000 7f47fa5ff700 1 Processor -- start 2023-07-13T22:24:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.809+0000 7f47fa5ff700 1 -- start start 2023-07-13T22:24:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.810+0000 7f47fa5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f40b1b90 0x7f47f40c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.810+0000 7f47fa5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47f40d0fa0 0x7f47f4157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.810+0000 7f47fa5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f415c6b0 0x7f47f409e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.810+0000 7f47fa5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40c8fe0 con 0x7f47f415c6b0 2023-07-13T22:24:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.810+0000 7f47fa5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40c8d00 con 0x7f47f40b1b90 2023-07-13T22:24:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.810+0000 7f47fa5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40c7060 con 0x7f47f40d0fa0 2023-07-13T22:24:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.810+0000 7f47f8b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f415c6b0 0x7f47f409e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.810+0000 7f47f8b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f415c6b0 0x7f47f409e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37334/0 (socket says 172.21.15.60:37334) 2023-07-13T22:24:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.810+0000 7f47f8b9c700 1 -- 172.21.15.60:0/1751959288 learned_addr learned my addr 172.21.15.60:0/1751959288 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.811+0000 7f47f3fff700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f40b1b90 0x7f47f40c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.811+0000 7f47f37fe700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47f40d0fa0 0x7f47f4157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.811+0000 7f47f8b9c700 1 -- 172.21.15.60:0/1751959288 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47f40d0fa0 msgr2=0x7f47f4157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.811+0000 7f47f8b9c700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47f40d0fa0 0x7f47f4157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.811+0000 7f47f8b9c700 1 -- 172.21.15.60:0/1751959288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f40b1b90 msgr2=0x7f47f40c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.811+0000 7f47f8b9c700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f40b1b90 0x7f47f40c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.811+0000 7f47f8b9c700 1 -- 172.21.15.60:0/1751959288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec009a70 con 0x7f47f415c6b0 2023-07-13T22:24:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.811+0000 7f47f8b9c700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f415c6b0 0x7f47f409e7a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009d60 tx=0x7f47ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.811+0000 7f47f0ff9700 1 -- 172.21.15.60:0/1751959288 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec01b810 con 0x7f47f415c6b0 2023-07-13T22:24:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.812+0000 7f47f0ff9700 1 -- 172.21.15.60:0/1751959288 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec020070 con 0x7f47f415c6b0 2023-07-13T22:24:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.812+0000 7f47f0ff9700 1 -- 172.21.15.60:0/1751959288 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec01bbf0 con 0x7f47f415c6b0 2023-07-13T22:24:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.812+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f40c7340 con 0x7f47f415c6b0 2023-07-13T22:24:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.812+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f409db00 con 0x7f47f415c6b0 2023-07-13T22:24:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.813+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f40c7860 con 0x7f47f415c6b0 2023-07-13T22:24:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.814+0000 7f47f0ff9700 1 -- 172.21.15.60:0/1751959288 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47ec01bd50 con 0x7f47f415c6b0 2023-07-13T22:24:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.814+0000 7f47f0ff9700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47e006ace0 0x7f47e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.815+0000 7f47f0ff9700 1 -- 172.21.15.60:0/1751959288 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f47ec022070 con 0x7f47f415c6b0 2023-07-13T22:24:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.815+0000 7f47f3fff700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47e006ace0 0x7f47e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.815+0000 7f47f3fff700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47e006ace0 0x7f47e006d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f47e4000f60 tx=0x7f47e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:06.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:06.819+0000 7f47f0ff9700 1 -- 172.21.15.60:0/1751959288 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f47f40c7860 con 0x7f47f415c6b0 2023-07-13T22:24:07.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.070+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47f40c7860 con 0x7f47f415c6b0 2023-07-13T22:24:07.052 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:24:07.071+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:24:07.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.075+0000 7f47f0ff9700 1 -- 172.21.15.60:0/1751959288 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47f40c7860 con 0x7f47f415c6b0 2023-07-13T22:24:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47e006ace0 msgr2=0x7f47e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47e006ace0 0x7f47e006d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f47e4000f60 tx=0x7f47e4009040 comp rx=0 tx=0).stop 2023-07-13T22:24:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f415c6b0 msgr2=0x7f47f409e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f415c6b0 0x7f47f409e7a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009d60 tx=0x7f47ec004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 shutdown_connections 2023-07-13T22:24:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47e006ace0 0x7f47e006d190 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f415c6b0 0x7f47f409e7a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f40b1b90 0x7f47f40c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 --2- 172.21.15.60:0/1751959288 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47f40d0fa0 0x7f47f4157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 >> 172.21.15.60:0/1751959288 conn(0x7f47f409c030 msgr2=0x7f47f415b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 shutdown_connections 2023-07-13T22:24:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.078+0000 7f47fa5ff700 1 -- 172.21.15.60:0/1751959288 wait complete. 2023-07-13T22:24:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:24:07.078 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:24:07.131 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:24:07.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.302+0000 7f47d63ff700 1 Processor -- start 2023-07-13T22:24:07.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.303+0000 7f47d63ff700 1 -- start start 2023-07-13T22:24:07.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.304+0000 7f47d63ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47d015ff70 0x7f47d00d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:07.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.304+0000 7f47d63ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 0x7f47d0160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:07.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.304+0000 7f47d63ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d00d12b0 con 0x7f47d0160e60 2023-07-13T22:24:07.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.304+0000 7f47d63ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d00cd3a0 con 0x7f47d015ff70 2023-07-13T22:24:07.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.304+0000 7f47d63ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00cd5f0 con 0x7f47d00d0e60 2023-07-13T22:24:07.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.304+0000 7f47cffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47d015ff70 0x7f47d00d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:07.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.304+0000 7f47cffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47d015ff70 0x7f47d00d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37340/0 (socket says 172.21.15.60:37340) 2023-07-13T22:24:07.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.304+0000 7f47cffff700 1 -- 172.21.15.60:0/2869501873 learned_addr learned my addr 172.21.15.60:0/2869501873 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:07.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.304+0000 7f47cf7fe700 1 --2- 172.21.15.60:0/2869501873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 0x7f47d0160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:07.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.305+0000 7f47cffff700 1 -- 172.21.15.60:0/2869501873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 msgr2=0x7f47d0160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:07.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.305+0000 7f47cffff700 1 --2- 172.21.15.60:0/2869501873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 0x7f47d0160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.305+0000 7f47cffff700 1 -- 172.21.15.60:0/2869501873 >> v1:172.21.15.105:6789/0 conn(0x7f47d0160e60 legacy=0x7f47d00cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:07.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.305+0000 7f47cffff700 1 -- 172.21.15.60:0/2869501873 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00cd8d0 con 0x7f47d015ff70 2023-07-13T22:24:07.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.305+0000 7f47cffff700 1 --2- 172.21.15.60:0/2869501873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d015ff70 0x7f47d00d0920 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f47c0006fd0 tx=0x7f47c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bbeb503bbfa386 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:07.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.305+0000 7f47ceffd700 1 -- 172.21.15.60:0/2869501873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0005e30 con 0x7f47d015ff70 2023-07-13T22:24:07.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.305+0000 7f47ceffd700 1 -- 172.21.15.60:0/2869501873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c0004030 con 0x7f47d015ff70 2023-07-13T22:24:07.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.305+0000 7f47ceffd700 1 -- 172.21.15.60:0/2869501873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c00043e0 con 0x7f47d015ff70 2023-07-13T22:24:07.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.306+0000 7f47d63ff700 1 -- 172.21.15.60:0/2869501873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d015ff70 msgr2=0x7f47d00d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:07.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.306+0000 7f47d63ff700 1 --2- 172.21.15.60:0/2869501873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d015ff70 0x7f47d00d0920 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f47c0006fd0 tx=0x7f47c0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.306+0000 7f47d63ff700 1 -- 172.21.15.60:0/2869501873 shutdown_connections 2023-07-13T22:24:07.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.306+0000 7f47d63ff700 1 --2- 172.21.15.60:0/2869501873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d015ff70 0x7f47d00d0920 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.306+0000 7f47d63ff700 1 --2- 172.21.15.60:0/2869501873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 0x7f47d0160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.306+0000 7f47d63ff700 1 -- 172.21.15.60:0/2869501873 >> 172.21.15.60:0/2869501873 conn(0x7f47d0071ed0 msgr2=0x7f47d009c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:07.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.306+0000 7f47d63ff700 1 -- 172.21.15.60:0/2869501873 shutdown_connections 2023-07-13T22:24:07.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.306+0000 7f47d63ff700 1 -- 172.21.15.60:0/2869501873 wait complete. 2023-07-13T22:24:07.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.308+0000 7f47d63ff700 1 Processor -- start 2023-07-13T22:24:07.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.308+0000 7f47d63ff700 1 -- start start 2023-07-13T22:24:07.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d63ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 0x7f47d00b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:07.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d63ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d015ff70 0x7f47d00cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:07.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d63ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d0160e60 0x7f47d00cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:07.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d63ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00c9ca0 con 0x7f47d0160e60 2023-07-13T22:24:07.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d63ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0148730 con 0x7f47d015ff70 2023-07-13T22:24:07.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d63ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00b3680 con 0x7f47d00d0e60 2023-07-13T22:24:07.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d499c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d0160e60 0x7f47d00cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:07.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d499c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d0160e60 0x7f47d00cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37342/0 (socket says 172.21.15.60:37342) 2023-07-13T22:24:07.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d499c700 1 -- 172.21.15.60:0/1928192479 learned_addr learned my addr 172.21.15.60:0/1928192479 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:07.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47cffff700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 0x7f47d00b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47cf7fe700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d015ff70 0x7f47d00cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d499c700 1 -- 172.21.15.60:0/1928192479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 msgr2=0x7f47d00b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d499c700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 0x7f47d00b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d499c700 1 -- 172.21.15.60:0/1928192479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d015ff70 msgr2=0x7f47d00cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d499c700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d015ff70 0x7f47d00cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.309+0000 7f47d499c700 1 -- 172.21.15.60:0/1928192479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0009500 con 0x7f47d0160e60 2023-07-13T22:24:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.310+0000 7f47d499c700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d0160e60 0x7f47d00cc530 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f47c800cb50 tx=0x7f47c800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:07.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.310+0000 7f47ccff9700 1 -- 172.21.15.60:0/1928192479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8003c00 con 0x7f47d0160e60 2023-07-13T22:24:07.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.310+0000 7f47ccff9700 1 -- 172.21.15.60:0/1928192479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8003d60 con 0x7f47d0160e60 2023-07-13T22:24:07.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.310+0000 7f47ccff9700 1 -- 172.21.15.60:0/1928192479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800aa20 con 0x7f47d0160e60 2023-07-13T22:24:07.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.310+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d00c99c0 con 0x7f47d0160e60 2023-07-13T22:24:07.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.310+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d01483c0 con 0x7f47d0160e60 2023-07-13T22:24:07.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.311+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d0067a90 con 0x7f47d0160e60 2023-07-13T22:24:07.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.312+0000 7f47ccff9700 1 -- 172.21.15.60:0/1928192479 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47c800ab80 con 0x7f47d0160e60 2023-07-13T22:24:07.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.313+0000 7f47ccff9700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47bc06ace0 0x7f47bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:07.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.313+0000 7f47ccff9700 1 -- 172.21.15.60:0/1928192479 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f47c8089320 con 0x7f47d0160e60 2023-07-13T22:24:07.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.313+0000 7f47cffff700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47bc06ace0 0x7f47bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:07.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.314+0000 7f47cffff700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47bc06ace0 0x7f47bc06d190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009130 tx=0x7f47c0008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.317+0000 7f47ccff9700 1 -- 172.21.15.60:0/1928192479 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f47c8054330 con 0x7f47d0160e60 2023-07-13T22:24:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:07.576+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f47d0147f40 con 0x7f47d0160e60 2023-07-13T22:24:08.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.079+0000 7f47ccff9700 1 -- 172.21.15.60:0/1928192479 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f47d0147f40 con 0x7f47d0160e60 2023-07-13T22:24:08.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47bc06ace0 msgr2=0x7f47bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:08.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47bc06ace0 0x7f47bc06d190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009130 tx=0x7f47c0008070 comp rx=0 tx=0).stop 2023-07-13T22:24:08.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d0160e60 msgr2=0x7f47d00cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:08.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d0160e60 0x7f47d00cc530 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f47c800cb50 tx=0x7f47c800ce60 comp rx=0 tx=0).stop 2023-07-13T22:24:08.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 shutdown_connections 2023-07-13T22:24:08.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47bc06ace0 0x7f47bc06d190 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:08.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d0160e60 0x7f47d00cc530 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:08.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47d00d0e60 0x7f47d00b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:08.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 --2- 172.21.15.60:0/1928192479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47d015ff70 0x7f47d00cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:08.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 >> 172.21.15.60:0/1928192479 conn(0x7f47d0071ed0 msgr2=0x7f47d00cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:08.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 shutdown_connections 2023-07-13T22:24:08.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.082+0000 7f47d63ff700 1 -- 172.21.15.60:0/1928192479 wait complete. 2023-07-13T22:24:08.084 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:24:08.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.312+0000 7f10231ff700 1 Processor -- start 2023-07-13T22:24:08.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.312+0000 7f10231ff700 1 -- start start 2023-07-13T22:24:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.313+0000 7f10231ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f101c15c7d0 0x7f101c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.313+0000 7f10231ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0d0fc0 0x7f101c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.313+0000 7f10231ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c0b2300 con 0x7f101c0b1d10 2023-07-13T22:24:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.313+0000 7f10231ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101c0b17c0 con 0x7f101c15c7d0 2023-07-13T22:24:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.313+0000 7f10231ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0b25a0 con 0x7f101c0d0fc0 2023-07-13T22:24:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.313+0000 7f1020f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f101c15c7d0 0x7f101c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.313+0000 7f1020f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f101c15c7d0 0x7f101c0b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37346/0 (socket says 172.21.15.60:37346) 2023-07-13T22:24:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.313+0000 7f1020f9b700 1 -- 172.21.15.60:0/2006545472 learned_addr learned my addr 172.21.15.60:0/2006545472 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.313+0000 7f102179c700 1 --2- 172.21.15.60:0/2006545472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0d0fc0 0x7f101c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.314+0000 7f1020f9b700 1 -- 172.21.15.60:0/2006545472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0d0fc0 msgr2=0x7f101c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.314+0000 7f1020f9b700 1 --2- 172.21.15.60:0/2006545472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0d0fc0 0x7f101c0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.314+0000 7f1020f9b700 1 -- 172.21.15.60:0/2006545472 >> v1:172.21.15.105:6789/0 conn(0x7f101c0b1d10 legacy=0x7f101c0d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.314+0000 7f1020f9b700 1 -- 172.21.15.60:0/2006545472 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c0b2880 con 0x7f101c15c7d0 2023-07-13T22:24:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.314+0000 7f1020f9b700 1 --2- 172.21.15.60:0/2006545472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 0x7f101c0b1280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f10100098f0 tx=0x7f1010009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ff92fab01a67ea server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.314+0000 7f101b7fe700 1 -- 172.21.15.60:0/2006545472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010011830 con 0x7f101c15c7d0 2023-07-13T22:24:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.314+0000 7f101b7fe700 1 -- 172.21.15.60:0/2006545472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1010011990 con 0x7f101c15c7d0 2023-07-13T22:24:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.314+0000 7f101b7fe700 1 -- 172.21.15.60:0/2006545472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010011d20 con 0x7f101c15c7d0 2023-07-13T22:24:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.315+0000 7f10231ff700 1 -- 172.21.15.60:0/2006545472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 msgr2=0x7f101c0b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.315+0000 7f10231ff700 1 --2- 172.21.15.60:0/2006545472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 0x7f101c0b1280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f10100098f0 tx=0x7f1010009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.315+0000 7f10231ff700 1 -- 172.21.15.60:0/2006545472 shutdown_connections 2023-07-13T22:24:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.315+0000 7f10231ff700 1 --2- 172.21.15.60:0/2006545472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0d0fc0 0x7f101c0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.315+0000 7f10231ff700 1 --2- 172.21.15.60:0/2006545472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 0x7f101c0b1280 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.315+0000 7f10231ff700 1 -- 172.21.15.60:0/2006545472 >> 172.21.15.60:0/2006545472 conn(0x7f101c09c050 msgr2=0x7f101c0d2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.315+0000 7f10231ff700 1 -- 172.21.15.60:0/2006545472 shutdown_connections 2023-07-13T22:24:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.315+0000 7f10231ff700 1 -- 172.21.15.60:0/2006545472 wait complete. 2023-07-13T22:24:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.317+0000 7f10231ff700 1 Processor -- start 2023-07-13T22:24:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.317+0000 7f10231ff700 1 -- start start 2023-07-13T22:24:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f10231ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0b1d10 0x7f101c1f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f10231ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c0d0fc0 0x7f101c0a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f10231ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 0x7f101c09f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f10231ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c09ef30 con 0x7f101c15c7d0 2023-07-13T22:24:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f10231ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c09e660 con 0x7f101c0d0fc0 2023-07-13T22:24:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f10231ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c1f50c0 con 0x7f101c0b1d10 2023-07-13T22:24:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f102179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 0x7f101c09f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f102179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 0x7f101c09f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37356/0 (socket says 172.21.15.60:37356) 2023-07-13T22:24:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f102179c700 1 -- 172.21.15.60:0/3393685091 learned_addr learned my addr 172.21.15.60:0/3393685091 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f1020f9b700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0b1d10 0x7f101c1f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.318+0000 7f101bfff700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c0d0fc0 0x7f101c0a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f102179c700 1 -- 172.21.15.60:0/3393685091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0b1d10 msgr2=0x7f101c1f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f102179c700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0b1d10 0x7f101c1f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f102179c700 1 -- 172.21.15.60:0/3393685091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c0d0fc0 msgr2=0x7f101c0a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f102179c700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c0d0fc0 0x7f101c0a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f102179c700 1 -- 172.21.15.60:0/3393685091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010009550 con 0x7f101c15c7d0 2023-07-13T22:24:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f102179c700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 0x7f101c09f3c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f101400efc0 tx=0x7f101400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f10197fa700 1 -- 172.21.15.60:0/3393685091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400cc90 con 0x7f101c15c7d0 2023-07-13T22:24:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f10197fa700 1 -- 172.21.15.60:0/3393685091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101400f040 con 0x7f101c15c7d0 2023-07-13T22:24:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c09cf30 con 0x7f101c15c7d0 2023-07-13T22:24:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f10197fa700 1 -- 172.21.15.60:0/3393685091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140146c0 con 0x7f101c15c7d0 2023-07-13T22:24:08.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.319+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c09ebc0 con 0x7f101c15c7d0 2023-07-13T22:24:08.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.320+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c072890 con 0x7f101c15c7d0 2023-07-13T22:24:08.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.321+0000 7f10197fa700 1 -- 172.21.15.60:0/3393685091 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1014014820 con 0x7f101c15c7d0 2023-07-13T22:24:08.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.322+0000 7f10197fa700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100006ace0 0x7f100006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:08.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.323+0000 7f10197fa700 1 -- 172.21.15.60:0/3393685091 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f101408a1b0 con 0x7f101c15c7d0 2023-07-13T22:24:08.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.323+0000 7f1020f9b700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100006ace0 0x7f100006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:08.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.323+0000 7f1020f9b700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100006ace0 0x7f100006d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1010014040 tx=0x7f1010008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:08.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.326+0000 7f10197fa700 1 -- 172.21.15.60:0/3393685091 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1014055130 con 0x7f101c15c7d0 2023-07-13T22:24:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:08.537+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f101c0b35a0 con 0x7f101c15c7d0 2023-07-13T22:24:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.460+0000 7f10197fa700 1 -- 172.21.15.60:0/3393685091 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7f101c0b35a0 con 0x7f101c15c7d0 2023-07-13T22:24:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100006ace0 msgr2=0x7f100006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100006ace0 0x7f100006d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1010014040 tx=0x7f1010008030 comp rx=0 tx=0).stop 2023-07-13T22:24:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 msgr2=0x7f101c09f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 0x7f101c09f3c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f101400efc0 tx=0x7f101400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:24:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 shutdown_connections 2023-07-13T22:24:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100006ace0 0x7f100006d190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c15c7d0 0x7f101c09f3c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0b1d10 0x7f101c1f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 --2- 172.21.15.60:0/3393685091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c0d0fc0 0x7f101c0a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 >> 172.21.15.60:0/3393685091 conn(0x7f101c09c050 msgr2=0x7f101c06b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:09.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 shutdown_connections 2023-07-13T22:24:09.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.464+0000 7f10231ff700 1 -- 172.21.15.60:0/3393685091 wait complete. 2023-07-13T22:24:09.448 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:24:09.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:24:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.698+0000 7f6149dff700 1 Processor -- start 2023-07-13T22:24:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.698+0000 7f6149dff700 1 -- start start 2023-07-13T22:24:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.699+0000 7f6149dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61440b1d10 0x7f61440d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.699+0000 7f6149dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440b2300 con 0x7f614415c7d0 2023-07-13T22:24:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.699+0000 7f6149dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440b25a0 con 0x7f61440d0df0 2023-07-13T22:24:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.699+0000 7f6149dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61440d1290 con 0x7f61440b1d10 2023-07-13T22:24:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.700+0000 7f61437fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f614415c7d0 0x7f61440b1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36484/0 (socket says 172.21.15.60:36484) 2023-07-13T22:24:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.700+0000 7f61437fe700 1 -- 172.21.15.60:0/4257304412 learned_addr learned my addr 172.21.15.60:0/4257304412 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.700+0000 7f6142ffd700 1 --2- 172.21.15.60:0/4257304412 >> v2:172.21.15.105:3300/0 conn(0x7f61440b1d10 0x7f61440d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.700+0000 7f61427fc700 1 -- 172.21.15.60:0/4257304412 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996956535 0 0) 0x7f61440b2300 con 0x7f614415c7d0 2023-07-13T22:24:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.700+0000 7f61427fc700 1 -- 172.21.15.60:0/4257304412 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f612c0035b0 con 0x7f614415c7d0 2023-07-13T22:24:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.700+0000 7f61427fc700 1 -- 172.21.15.60:0/4257304412 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 156456045 0 0) 0x7f612c0035b0 con 0x7f614415c7d0 2023-07-13T22:24:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.700+0000 7f6142ffd700 1 -- 172.21.15.60:0/4257304412 >> v1:172.21.15.105:6789/0 conn(0x7f61440d0df0 legacy=0x7f61440b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.700+0000 7f6142ffd700 1 -- 172.21.15.60:0/4257304412 >> v1:172.21.15.60:6789/0 conn(0x7f614415c7d0 legacy=0x7f61440b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.700+0000 7f6142ffd700 1 -- 172.21.15.60:0/4257304412 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61440b2880 con 0x7f61440b1d10 2023-07-13T22:24:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.701+0000 7f6142ffd700 1 --2- 172.21.15.60:0/4257304412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61440b1d10 0x7f61440d08b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6138009840 tx=0x7f6138009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=277a7917155e7171 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.701+0000 7f61427fc700 1 -- 172.21.15.60:0/4257304412 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134001670 con 0x7f61440b1d10 2023-07-13T22:24:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.701+0000 7f61427fc700 1 -- 172.21.15.60:0/4257304412 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6138004030 con 0x7f61440b1d10 2023-07-13T22:24:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.701+0000 7f61427fc700 1 -- 172.21.15.60:0/4257304412 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61380043a0 con 0x7f61440b1d10 2023-07-13T22:24:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.701+0000 7f6149dff700 1 -- 172.21.15.60:0/4257304412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61440b1d10 msgr2=0x7f61440d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.701+0000 7f6149dff700 1 --2- 172.21.15.60:0/4257304412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61440b1d10 0x7f61440d08b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6138009840 tx=0x7f6138009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.701+0000 7f6149dff700 1 -- 172.21.15.60:0/4257304412 shutdown_connections 2023-07-13T22:24:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.701+0000 7f6149dff700 1 --2- 172.21.15.60:0/4257304412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61440b1d10 0x7f61440d08b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.702+0000 7f6149dff700 1 -- 172.21.15.60:0/4257304412 >> 172.21.15.60:0/4257304412 conn(0x7f614409c050 msgr2=0x7f61440d2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.702+0000 7f6149dff700 1 -- 172.21.15.60:0/4257304412 shutdown_connections 2023-07-13T22:24:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.702+0000 7f6149dff700 1 -- 172.21.15.60:0/4257304412 wait complete. 2023-07-13T22:24:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.703+0000 7f6149dff700 1 Processor -- start 2023-07-13T22:24:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.703+0000 7f6149dff700 1 -- start start 2023-07-13T22:24:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.704+0000 7f6149dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61440b1d10 0x7f61441578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.704+0000 7f6149dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61440d0df0 0x7f61440ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.704+0000 7f6149dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f614415c7d0 0x7f61440b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.704+0000 7f6149dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440cd030 con 0x7f61440b1d10 2023-07-13T22:24:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.704+0000 7f6149dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440b6050 con 0x7f61440d0df0 2023-07-13T22:24:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.704+0000 7f6149dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61440b6280 con 0x7f614415c7d0 2023-07-13T22:24:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f61437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61440b1d10 0x7f61441578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f61437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61440b1d10 0x7f61441578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37364/0 (socket says 172.21.15.60:37364) 2023-07-13T22:24:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f61437fe700 1 -- 172.21.15.60:0/1448496669 learned_addr learned my addr 172.21.15.60:0/1448496669 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f6142ffd700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61440d0df0 0x7f61440ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f6143fff700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f614415c7d0 0x7f61440b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f61437fe700 1 -- 172.21.15.60:0/1448496669 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f614415c7d0 msgr2=0x7f61440b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f61437fe700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f614415c7d0 0x7f61440b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f61437fe700 1 -- 172.21.15.60:0/1448496669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61440d0df0 msgr2=0x7f61440ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f61437fe700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61440d0df0 0x7f61440ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f61437fe700 1 -- 172.21.15.60:0/1448496669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61380094a0 con 0x7f61440b1d10 2023-07-13T22:24:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f6143fff700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f614415c7d0 0x7f61440b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.705+0000 7f61437fe700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61440b1d10 0x7f61441578a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f613400c520 tx=0x7f613400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.706+0000 7f612bfff700 1 -- 172.21.15.60:0/1448496669 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134004c80 con 0x7f61440b1d10 2023-07-13T22:24:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.706+0000 7f612bfff700 1 -- 172.21.15.60:0/1448496669 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6134004e00 con 0x7f61440b1d10 2023-07-13T22:24:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.706+0000 7f612bfff700 1 -- 172.21.15.60:0/1448496669 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134007310 con 0x7f61440b1d10 2023-07-13T22:24:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.706+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61440b6560 con 0x7f61440b1d10 2023-07-13T22:24:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.706+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61440b6b20 con 0x7f61440b1d10 2023-07-13T22:24:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.707+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6144072890 con 0x7f61440b1d10 2023-07-13T22:24:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.708+0000 7f612bfff700 1 -- 172.21.15.60:0/1448496669 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6134007470 con 0x7f61440b1d10 2023-07-13T22:24:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.709+0000 7f612bfff700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f613006ace0 0x7f613006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.709+0000 7f612bfff700 1 -- 172.21.15.60:0/1448496669 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f6134089130 con 0x7f61440b1d10 2023-07-13T22:24:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.710+0000 7f6142ffd700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f613006ace0 0x7f613006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.711+0000 7f6142ffd700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f613006ace0 0x7f613006d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6138009840 tx=0x7f6138008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:09.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.716+0000 7f612bfff700 1 -- 172.21.15.60:0/1448496669 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6134054260 con 0x7f61440b1d10 2023-07-13T22:24:09.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:09.923+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f61440672d0 con 0x7f61440b1d10 2023-07-13T22:24:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.577+0000 7f612bfff700 1 -- 172.21.15.60:0/1448496669 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6134057880 con 0x7f61440b1d10 2023-07-13T22:24:10.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f613006ace0 msgr2=0x7f613006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:10.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f613006ace0 0x7f613006d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6138009840 tx=0x7f6138008030 comp rx=0 tx=0).stop 2023-07-13T22:24:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61440b1d10 msgr2=0x7f61441578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61440b1d10 0x7f61441578a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f613400c520 tx=0x7f613400c830 comp rx=0 tx=0).stop 2023-07-13T22:24:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 shutdown_connections 2023-07-13T22:24:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f613006ace0 0x7f613006d190 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61440b1d10 0x7f61441578a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f614415c7d0 0x7f61440b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 --2- 172.21.15.60:0/1448496669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61440d0df0 0x7f61440ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.582+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 >> 172.21.15.60:0/1448496669 conn(0x7f614409c050 msgr2=0x7f61440b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.583+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 shutdown_connections 2023-07-13T22:24:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.583+0000 7f6149dff700 1 -- 172.21.15.60:0/1448496669 wait complete. 2023-07-13T22:24:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:24:10.587 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:24:10.591 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:24:10.713 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2023-07-13T22:24:10.713 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2023-07-13T22:24:10.714 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:24:10.714 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2023-07-13T22:24:10.714 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.714 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2023-07-13T22:24:10.714 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.714 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2023-07-13T22:24:10.714 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.715 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2023-07-13T22:24:10.715 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2023-07-13T22:24:10.715 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.715 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2023-07-13T22:24:10.715 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2023-07-13T22:24:10.715 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.715 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:24:10.715 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2023-07-13T22:24:10.716 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.716 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:24:10.716 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:24:10.716 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.716 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2023-07-13T22:24:10.716 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.716 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2023-07-13T22:24:10.717 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2023-07-13T22:24:10.717 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:24:10.717 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:24:10.717 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2023-07-13T22:24:10.717 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:24:10.717 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2023-07-13T22:24:10.717 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:24:10.717 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:24:10.718 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2023-07-13T22:24:10.718 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.718 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:24:10.718 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.718 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2023-07-13T22:24:10.718 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.718 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2023-07-13T22:24:10.718 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.719 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2023-07-13T22:24:10.719 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2023-07-13T22:24:10.719 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2023-07-13T22:24:10.719 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:24:10.719 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:24:10.719 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.719 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2023-07-13T22:24:10.719 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2023-07-13T22:24:10.720 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.720 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2023-07-13T22:24:10.720 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2023-07-13T22:24:10.720 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:24:10.720 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.720 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2023-07-13T22:24:10.720 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2023-07-13T22:24:10.721 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.721 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2023-07-13T22:24:10.721 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2023-07-13T22:24:10.721 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2023-07-13T22:24:10.721 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.721 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2023-07-13T22:24:10.721 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2023-07-13T22:24:10.721 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.722 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:24:10.722 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.722 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2023-07-13T22:24:10.722 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.722 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2023-07-13T22:24:10.722 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.722 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:24:10.722 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2023-07-13T22:24:10.723 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.723 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2023-07-13T22:24:10.723 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2023-07-13T22:24:10.723 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2023-07-13T22:24:10.723 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2023-07-13T22:24:10.723 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2023-07-13T22:24:10.723 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2023-07-13T22:24:10.723 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2023-07-13T22:24:10.724 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2023-07-13T22:24:10.724 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2023-07-13T22:24:10.724 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:24:10.724 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2023-07-13T22:24:10.724 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2023-07-13T22:24:10.724 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:24:10.724 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2023-07-13T22:24:10.725 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:24:10.725 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:24:10.725 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2023-07-13T22:24:10.725 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:24:10.725 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.725 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2023-07-13T22:24:10.725 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:24:10.725 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2023-07-13T22:24:10.726 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:24:10.726 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:24:10.726 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:24:10.726 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:24:10.726 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2023-07-13T22:24:10.726 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.726 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2023-07-13T22:24:10.726 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:24:10.727 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:24:10.727 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2023-07-13T22:24:10.727 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:24:10.727 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2023-07-13T22:24:10.727 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2023-07-13T22:24:10.727 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:24:10.727 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2023-07-13T22:24:10.727 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2023-07-13T22:24:10.728 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:24:10.728 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.728 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:24:10.728 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:24:10.728 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:24:10.728 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.728 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:24:10.728 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:24:10.729 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:24:10.729 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.729 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2023-07-13T22:24:10.729 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:24:10.729 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.729 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:24:10.729 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2023-07-13T22:24:10.729 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:10.730 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2023-07-13T22:24:10.730 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2023-07-13T22:24:10.730 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2023-07-13T22:24:10.730 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2023-07-13T22:24:10.730 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:24:10.730 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:24:10.730 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2023-07-13T22:24:10.731 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2023-07-13T22:24:10.731 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:24:10.731 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2023-07-13T22:24:10.731 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:24:10.731 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:24:10.731 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2023-07-13T22:24:10.731 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2023-07-13T22:24:10.731 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2023-07-13T22:24:10.732 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-07-13T22:24:10.732 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-07-13T22:24:10.732 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-07-13T22:24:10.732 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-07-13T22:24:10.732 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2023-07-13T22:24:10.732 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2023-07-13T22:24:10.732 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2023-07-13T22:24:10.732 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:24:10.733 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2023-07-13T22:24:10.733 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2023-07-13T22:24:10.733 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2023-07-13T22:24:10.733 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2023-07-13T22:24:10.733 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:24:10.733 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:24:10.733 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:24:10.734 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-07-13T22:24:10.734 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-07-13T22:24:10.734 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:24:10.734 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-07-13T22:24:10.734 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.735 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-07-13T22:24:10.735 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.735 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-07-13T22:24:10.735 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.735 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-07-13T22:24:10.735 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-07-13T22:24:10.735 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.735 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-07-13T22:24:10.736 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-07-13T22:24:10.736 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.736 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:24:10.736 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-07-13T22:24:10.736 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.736 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:24:10.736 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:24:10.737 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.737 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-07-13T22:24:10.737 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.737 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-07-13T22:24:10.737 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-07-13T22:24:10.737 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:24:10.737 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:24:10.737 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-07-13T22:24:10.738 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:24:10.738 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-07-13T22:24:10.738 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:24:10.738 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:24:10.738 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-07-13T22:24:10.738 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.738 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:24:10.738 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.739 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-07-13T22:24:10.739 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.739 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-07-13T22:24:10.739 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.739 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-07-13T22:24:10.739 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-07-13T22:24:10.739 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-07-13T22:24:10.739 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:24:10.740 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:24:10.740 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.740 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-07-13T22:24:10.740 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-07-13T22:24:10.740 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.740 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-07-13T22:24:10.740 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2023-07-13T22:24:10.741 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:24:10.741 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.741 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-07-13T22:24:10.741 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-07-13T22:24:10.741 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.741 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-07-13T22:24:10.741 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-07-13T22:24:10.741 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-07-13T22:24:10.742 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.742 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-07-13T22:24:10.742 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-07-13T22:24:10.742 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.742 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:24:10.742 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.742 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-07-13T22:24:10.742 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.743 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-07-13T22:24:10.743 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.743 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:24:10.743 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-07-13T22:24:10.743 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.743 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-07-13T22:24:10.743 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-07-13T22:24:10.743 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2023-07-13T22:24:10.744 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2023-07-13T22:24:10.744 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2023-07-13T22:24:10.744 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2023-07-13T22:24:10.744 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2023-07-13T22:24:10.744 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2023-07-13T22:24:10.744 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2023-07-13T22:24:10.744 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:24:10.744 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-07-13T22:24:10.745 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2023-07-13T22:24:10.745 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:24:10.745 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2023-07-13T22:24:10.745 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:24:10.745 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:24:10.745 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2023-07-13T22:24:10.745 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:24:10.745 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.746 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-07-13T22:24:10.746 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:24:10.746 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-07-13T22:24:10.746 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:24:10.746 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:24:10.746 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:24:10.746 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:24:10.746 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-07-13T22:24:10.747 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.747 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-07-13T22:24:10.747 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:24:10.747 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:24:10.747 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-07-13T22:24:10.747 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:24:10.747 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-07-13T22:24:10.747 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-07-13T22:24:10.748 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:24:10.748 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-07-13T22:24:10.748 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-07-13T22:24:10.748 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:24:10.748 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.748 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:24:10.748 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:24:10.749 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:24:10.749 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.749 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:24:10.749 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:24:10.749 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:24:10.749 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.749 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-07-13T22:24:10.749 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:24:10.750 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.750 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:24:10.750 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2023-07-13T22:24:10.750 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:24:10.750 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-07-13T22:24:10.750 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-07-13T22:24:10.750 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-07-13T22:24:10.750 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-07-13T22:24:10.751 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:24:10.751 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:24:10.751 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2023-07-13T22:24:10.751 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2023-07-13T22:24:10.751 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:24:10.751 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2023-07-13T22:24:10.751 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:24:10.751 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:24:10.752 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-07-13T22:24:10.752 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2023-07-13T22:24:10.752 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2023-07-13T22:24:10.752 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-07-13T22:24:10.752 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-07-13T22:24:10.752 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-07-13T22:24:10.752 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-07-13T22:24:10.752 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2023-07-13T22:24:10.753 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2023-07-13T22:24:10.753 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2023-07-13T22:24:10.753 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:24:10.753 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2023-07-13T22:24:10.753 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2023-07-13T22:24:10.753 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2023-07-13T22:24:10.753 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2023-07-13T22:24:10.753 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:24:10.754 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:24:10.754 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:24:10.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-13T22:24:10.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.977+0000 7f3ff35ff700 1 Processor -- start 2023-07-13T22:24:10.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.977+0000 7f3ff35ff700 1 -- start start 2023-07-13T22:24:10.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.978+0000 7f3ff35ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fec0cfd50 0x7f3fec0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:10.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.978+0000 7f3ff35ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fec0b1ac0 0x7f3fec0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:10.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff35ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fec0d1870 con 0x7f3fec0b8200 2023-07-13T22:24:10.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff35ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fec0cf370 con 0x7f3fec0cfd50 2023-07-13T22:24:10.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff35ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fec0d12b0 con 0x7f3fec0b1ac0 2023-07-13T22:24:10.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff0b9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fec0cfd50 0x7f3fec0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:10.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff1b9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fec0b1ac0 0x7f3fec0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:10.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff0b9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fec0cfd50 0x7f3fec0d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37374/0 (socket says 172.21.15.60:37374) 2023-07-13T22:24:10.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff0b9a700 1 -- 172.21.15.60:0/79153186 learned_addr learned my addr 172.21.15.60:0/79153186 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:10.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff0b9a700 1 -- 172.21.15.60:0/79153186 >> v2:172.21.15.105:3300/0 conn(0x7f3fec0b1ac0 msgr2=0x7f3fec0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:10.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff0b9a700 1 --2- 172.21.15.60:0/79153186 >> v2:172.21.15.105:3300/0 conn(0x7f3fec0b1ac0 0x7f3fec0d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff0b9a700 1 -- 172.21.15.60:0/79153186 >> v1:172.21.15.105:6789/0 conn(0x7f3fec0b8200 legacy=0x7f3fec0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:10.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.979+0000 7f3ff0b9a700 1 -- 172.21.15.60:0/79153186 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec0d1590 con 0x7f3fec0cfd50 2023-07-13T22:24:10.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3ff0b9a700 1 --2- 172.21.15.60:0/79153186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 0x7f3fec0d23c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0b8d70 tx=0x7f3fdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b3417fa057e483 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:10.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3fe3fff700 1 -- 172.21.15.60:0/79153186 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc011830 con 0x7f3fec0cfd50 2023-07-13T22:24:10.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3fe3fff700 1 -- 172.21.15.60:0/79153186 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc011990 con 0x7f3fec0cfd50 2023-07-13T22:24:10.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3fe3fff700 1 -- 172.21.15.60:0/79153186 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc011d20 con 0x7f3fec0cfd50 2023-07-13T22:24:10.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3ff35ff700 1 -- 172.21.15.60:0/79153186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 msgr2=0x7f3fec0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:10.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3ff35ff700 1 --2- 172.21.15.60:0/79153186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 0x7f3fec0d23c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0b8d70 tx=0x7f3fdc009bd0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3ff35ff700 1 -- 172.21.15.60:0/79153186 shutdown_connections 2023-07-13T22:24:10.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3ff35ff700 1 --2- 172.21.15.60:0/79153186 >> v2:172.21.15.105:3300/0 conn(0x7f3fec0b1ac0 0x7f3fec0d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3ff35ff700 1 --2- 172.21.15.60:0/79153186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 0x7f3fec0d23c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.980+0000 7f3ff35ff700 1 -- 172.21.15.60:0/79153186 >> 172.21.15.60:0/79153186 conn(0x7f3fec06b2f0 msgr2=0x7f3fec06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:10.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.981+0000 7f3ff35ff700 1 -- 172.21.15.60:0/79153186 shutdown_connections 2023-07-13T22:24:10.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.981+0000 7f3ff35ff700 1 -- 172.21.15.60:0/79153186 wait complete. 2023-07-13T22:24:10.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.982+0000 7f3ff35ff700 1 Processor -- start 2023-07-13T22:24:10.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.982+0000 7f3ff35ff700 1 -- start start 2023-07-13T22:24:10.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.983+0000 7f3ff35ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fec0b1ac0 0x7f3fec0cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:10.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.983+0000 7f3ff35ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fec0b8200 0x7f3fec14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:10.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.983+0000 7f3ff35ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 0x7f3fec14e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:10.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.983+0000 7f3ff35ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec15e180 con 0x7f3fec0cfd50 2023-07-13T22:24:10.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.983+0000 7f3ff35ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec15e460 con 0x7f3fec0b1ac0 2023-07-13T22:24:10.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.983+0000 7f3ff35ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0c9d00 con 0x7f3fec0b8200 2023-07-13T22:24:10.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff1b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 0x7f3fec14e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:10.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff1b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 0x7f3fec14e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37386/0 (socket says 172.21.15.60:37386) 2023-07-13T22:24:10.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff139b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fec0b1ac0 0x7f3fec0cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:10.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff1b9c700 1 -- 172.21.15.60:0/212778585 learned_addr learned my addr 172.21.15.60:0/212778585 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:10.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff0b9a700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fec0b8200 0x7f3fec14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:10.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff1b9c700 1 -- 172.21.15.60:0/212778585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fec0b8200 msgr2=0x7f3fec14dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:10.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff1b9c700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fec0b8200 0x7f3fec14dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff1b9c700 1 -- 172.21.15.60:0/212778585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fec0b1ac0 msgr2=0x7f3fec0cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:10.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff1b9c700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fec0b1ac0 0x7f3fec0cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:10.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff1b9c700 1 -- 172.21.15.60:0/212778585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc009550 con 0x7f3fec0cfd50 2023-07-13T22:24:10.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff139b700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fec0b1ac0 0x7f3fec0cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:10.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.984+0000 7f3ff1b9c700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 0x7f3fec14e200 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400efc0 tx=0x7f3fe400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:10.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.985+0000 7f3fe1ffb700 1 -- 172.21.15.60:0/212778585 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4009ea0 con 0x7f3fec0cfd50 2023-07-13T22:24:10.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.985+0000 7f3fe1ffb700 1 -- 172.21.15.60:0/212778585 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe400f040 con 0x7f3fec0cfd50 2023-07-13T22:24:10.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.985+0000 7f3fe1ffb700 1 -- 172.21.15.60:0/212778585 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400ced0 con 0x7f3fec0cfd50 2023-07-13T22:24:10.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.985+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec0c9fe0 con 0x7f3fec0cfd50 2023-07-13T22:24:10.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.985+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec0ccb90 con 0x7f3fec0cfd50 2023-07-13T22:24:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.986+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec0ccd80 con 0x7f3fec0cfd50 2023-07-13T22:24:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.987+0000 7f3fe1ffb700 1 -- 172.21.15.60:0/212778585 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3fe4014480 con 0x7f3fec0cfd50 2023-07-13T22:24:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.988+0000 7f3fe1ffb700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd006ace0 0x7f3fd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.988+0000 7f3fe1ffb700 1 -- 172.21.15.60:0/212778585 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3fe4089a00 con 0x7f3fec0cfd50 2023-07-13T22:24:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.988+0000 7f3ff139b700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd006ace0 0x7f3fd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.989+0000 7f3ff139b700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd006ace0 0x7f3fd006d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8007fd0 tx=0x7f3fe8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:10.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:10.992+0000 7f3fe1ffb700 1 -- 172.21.15.60:0/212778585 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3fe4054d30 con 0x7f3fec0cfd50 2023-07-13T22:24:11.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.206+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3fec15e620 con 0x7f3fec0cfd50 2023-07-13T22:24:11.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.210+0000 7f3fe1ffb700 1 -- 172.21.15.60:0/212778585 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3fe4058350 con 0x7f3fec0cfd50 2023-07-13T22:24:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.212+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd006ace0 msgr2=0x7f3fd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.212+0000 7f3ff35ff700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd006ace0 0x7f3fd006d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8007fd0 tx=0x7f3fe8009040 comp rx=0 tx=0).stop 2023-07-13T22:24:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.212+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 msgr2=0x7f3fec14e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.212+0000 7f3ff35ff700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 0x7f3fec14e200 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400efc0 tx=0x7f3fe400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.213+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 shutdown_connections 2023-07-13T22:24:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.213+0000 7f3ff35ff700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fd006ace0 0x7f3fd006d190 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.213+0000 7f3ff35ff700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fec0cfd50 0x7f3fec14e200 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.213+0000 7f3ff35ff700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fec0b1ac0 0x7f3fec0cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.213+0000 7f3ff35ff700 1 --2- 172.21.15.60:0/212778585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fec0b8200 0x7f3fec14dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.213+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 >> 172.21.15.60:0/212778585 conn(0x7f3fec06b2f0 msgr2=0x7f3fec0d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:11.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.213+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 shutdown_connections 2023-07-13T22:24:11.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.213+0000 7f3ff35ff700 1 -- 172.21.15.60:0/212778585 wait complete. 2023-07-13T22:24:11.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2023-07-13T22:24:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.442+0000 7f7900bff700 1 Processor -- start 2023-07-13T22:24:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.442+0000 7f7900bff700 1 -- start start 2023-07-13T22:24:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.443+0000 7f7900bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78fc0cd260 0x7f78fc0d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.443+0000 7f7900bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78fc0d0e20 0x7f78fc0d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.443+0000 7f7900bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc0d2410 con 0x7f78fc0d0a50 2023-07-13T22:24:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.443+0000 7f7900bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc15c900 con 0x7f78fc0d0e20 2023-07-13T22:24:11.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.443+0000 7f7900bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc0d2660 con 0x7f78fc0cd260 2023-07-13T22:24:11.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.444+0000 7f78fad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78fc0d0e20 0x7f78fc0d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.444+0000 7f78fad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78fc0d0e20 0x7f78fc0d1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37400/0 (socket says 172.21.15.60:37400) 2023-07-13T22:24:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.444+0000 7f78fad9d700 1 -- 172.21.15.60:0/451962494 learned_addr learned my addr 172.21.15.60:0/451962494 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.444+0000 7f78f9d9b700 1 --2- 172.21.15.60:0/451962494 >> v2:172.21.15.105:3300/0 conn(0x7f78fc0cd260 0x7f78fc0d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.444+0000 7f78fad9d700 1 -- 172.21.15.60:0/451962494 >> v2:172.21.15.105:3300/0 conn(0x7f78fc0cd260 msgr2=0x7f78fc0d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.444+0000 7f78fad9d700 1 --2- 172.21.15.60:0/451962494 >> v2:172.21.15.105:3300/0 conn(0x7f78fc0cd260 0x7f78fc0d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.444+0000 7f78fad9d700 1 -- 172.21.15.60:0/451962494 >> v1:172.21.15.105:6789/0 conn(0x7f78fc0d0a50 legacy=0x7f78fc0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.444+0000 7f78fad9d700 1 -- 172.21.15.60:0/451962494 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc0d2940 con 0x7f78fc0d0e20 2023-07-13T22:24:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.444+0000 7f78fad9d700 1 --2- 172.21.15.60:0/451962494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0e20 0x7f78fc0d1240 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f78fc072560 tx=0x7f78f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37dbaa138dc6315 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f78f959a700 1 -- 172.21.15.60:0/451962494 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f400e070 con 0x7f78fc0d0e20 2023-07-13T22:24:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f78f959a700 1 -- 172.21.15.60:0/451962494 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f400b910 con 0x7f78fc0d0e20 2023-07-13T22:24:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f78f959a700 1 -- 172.21.15.60:0/451962494 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f400bcc0 con 0x7f78fc0d0e20 2023-07-13T22:24:11.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f7900bff700 1 -- 172.21.15.60:0/451962494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0e20 msgr2=0x7f78fc0d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:11.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f7900bff700 1 --2- 172.21.15.60:0/451962494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0e20 0x7f78fc0d1240 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f78fc072560 tx=0x7f78f400b330 comp rx=0 tx=0).stop 2023-07-13T22:24:11.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f7900bff700 1 -- 172.21.15.60:0/451962494 shutdown_connections 2023-07-13T22:24:11.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f7900bff700 1 --2- 172.21.15.60:0/451962494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0e20 0x7f78fc0d1240 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f7900bff700 1 --2- 172.21.15.60:0/451962494 >> v2:172.21.15.105:3300/0 conn(0x7f78fc0cd260 0x7f78fc0d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f7900bff700 1 -- 172.21.15.60:0/451962494 >> 172.21.15.60:0/451962494 conn(0x7f78fc071600 msgr2=0x7f78fc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:11.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f7900bff700 1 -- 172.21.15.60:0/451962494 shutdown_connections 2023-07-13T22:24:11.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.445+0000 7f7900bff700 1 -- 172.21.15.60:0/451962494 wait complete. 2023-07-13T22:24:11.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.447+0000 7f7900bff700 1 Processor -- start 2023-07-13T22:24:11.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.447+0000 7f7900bff700 1 -- start start 2023-07-13T22:24:11.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.448+0000 7f7900bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc0cd260 0x7f78fc0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.448+0000 7f7900bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0a50 0x7f78fc159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.448+0000 7f7900bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78fc0d0e20 0x7f78fc15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.448+0000 7f7900bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c7900 con 0x7f78fc0d0a50 2023-07-13T22:24:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.448+0000 7f7900bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0cc8e0 con 0x7f78fc0cd260 2023-07-13T22:24:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.448+0000 7f7900bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c6ca0 con 0x7f78fc0d0e20 2023-07-13T22:24:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.448+0000 7f78f9d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0a50 0x7f78fc159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.448+0000 7f78f9d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0a50 0x7f78fc159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37412/0 (socket says 172.21.15.60:37412) 2023-07-13T22:24:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.448+0000 7f78f9d9b700 1 -- 172.21.15.60:0/312868363 learned_addr learned my addr 172.21.15.60:0/312868363 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78fa59c700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc0cd260 0x7f78fc0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78fad9d700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78fc0d0e20 0x7f78fc15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78f9d9b700 1 -- 172.21.15.60:0/312868363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78fc0d0e20 msgr2=0x7f78fc15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78f9d9b700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78fc0d0e20 0x7f78fc15ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78f9d9b700 1 -- 172.21.15.60:0/312868363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc0cd260 msgr2=0x7f78fc0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78f9d9b700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc0cd260 0x7f78fc0cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78f9d9b700 1 -- 172.21.15.60:0/312868363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4009b30 con 0x7f78fc0d0a50 2023-07-13T22:24:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78fad9d700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78fc0d0e20 0x7f78fc15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78f9d9b700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0a50 0x7f78fc159410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f78f000e9d0 tx=0x7f78f000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78eaffd700 1 -- 172.21.15.60:0/312868363 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0009730 con 0x7f78fc0d0a50 2023-07-13T22:24:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78eaffd700 1 -- 172.21.15.60:0/312868363 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f000c850 con 0x7f78fc0d0a50 2023-07-13T22:24:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc0c6f80 con 0x7f78fc0d0a50 2023-07-13T22:24:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.449+0000 7f78eaffd700 1 -- 172.21.15.60:0/312868363 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000cbe0 con 0x7f78fc0d0a50 2023-07-13T22:24:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.450+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0c7540 con 0x7f78fc0d0a50 2023-07-13T22:24:11.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.451+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc04e280 con 0x7f78fc0d0a50 2023-07-13T22:24:11.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.452+0000 7f78eaffd700 1 -- 172.21.15.60:0/312868363 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f78f000cd40 con 0x7f78fc0d0a50 2023-07-13T22:24:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.453+0000 7f78eaffd700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f78dc06ace0 0x7f78dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.453+0000 7f78eaffd700 1 -- 172.21.15.60:0/312868363 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f78f0089250 con 0x7f78fc0d0a50 2023-07-13T22:24:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.453+0000 7f78fa59c700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f78dc06ace0 0x7f78dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.453+0000 7f78fa59c700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f78dc06ace0 0x7f78dc06d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f78ec007fd0 tx=0x7f78ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:11.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.457+0000 7f78eaffd700 1 -- 172.21.15.60:0/312868363 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f78f0054610 con 0x7f78fc0d0a50 2023-07-13T22:24:11.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:11.749+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) v1 -- 0x7f78fc0c7a40 con 0x7f78fc0d0a50 2023-07-13T22:24:12.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.585+0000 7f78eaffd700 1 -- 172.21.15.60:0/312868363 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v188) v1 ==== 150+0+0 (secure 0 0 0) 0x7f78f0057c30 con 0x7f78fc0d0a50 2023-07-13T22:24:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f78dc06ace0 msgr2=0x7f78dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f78dc06ace0 0x7f78dc06d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f78ec007fd0 tx=0x7f78ec009040 comp rx=0 tx=0).stop 2023-07-13T22:24:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0a50 msgr2=0x7f78fc159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0a50 0x7f78fc159410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f78f000e9d0 tx=0x7f78f000ece0 comp rx=0 tx=0).stop 2023-07-13T22:24:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 shutdown_connections 2023-07-13T22:24:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f78dc06ace0 0x7f78dc06d190 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78fc0d0e20 0x7f78fc15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78fc0cd260 0x7f78fc0cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 --2- 172.21.15.60:0/312868363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78fc0d0a50 0x7f78fc159410 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 >> 172.21.15.60:0/312868363 conn(0x7f78fc071600 msgr2=0x7f78fc1573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:12.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 shutdown_connections 2023-07-13T22:24:12.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.588+0000 7f7900bff700 1 -- 172.21.15.60:0/312868363 wait complete. 2023-07-13T22:24:12.591 INFO:tasks.cephfs_test_runner:test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-13T22:24:12.591 INFO:tasks.cephfs_test_runner:Starting test: test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:24:12.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2023-07-13T22:24:12.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.818+0000 7f0a07bff700 1 Processor -- start 2023-07-13T22:24:12.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.818+0000 7f0a07bff700 1 -- start start 2023-07-13T22:24:12.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.819+0000 7f0a07bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0a000d0a50 0x7f0a000cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:12.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.819+0000 7f0a07bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a000d2410 con 0x7f0a000d0e20 2023-07-13T22:24:12.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.819+0000 7f0a07bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a000d2660 con 0x7f0a000cd260 2023-07-13T22:24:12.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.819+0000 7f0a07bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0015c900 con 0x7f0a000d0a50 2023-07-13T22:24:12.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.819+0000 7f0a0599b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0a000d0a50 0x7f0a000cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:12.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.819+0000 7f0a0619c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0a000d0e20 0x7f0a000b5a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36492/0 (socket says 172.21.15.60:36492) 2023-07-13T22:24:12.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.819+0000 7f0a0619c700 1 -- 172.21.15.60:0/4060583892 learned_addr learned my addr 172.21.15.60:0/4060583892 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:12.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.820+0000 7f0a0599b700 1 -- 172.21.15.60:0/4060583892 >> v1:172.21.15.105:6789/0 conn(0x7f0a000cd260 legacy=0x7f0a000d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:12.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.820+0000 7f0a0599b700 1 -- 172.21.15.60:0/4060583892 >> v1:172.21.15.60:6789/0 conn(0x7f0a000d0e20 legacy=0x7f0a000b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:12.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.820+0000 7f0a0599b700 1 -- 172.21.15.60:0/4060583892 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a000d2940 con 0x7f0a000d0a50 2023-07-13T22:24:12.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.820+0000 7f0a04999700 1 -- 172.21.15.60:0/4060583892 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323217810 0 0) 0x7f0a000d2410 con 0x7f0a000d0e20 2023-07-13T22:24:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.820+0000 7f0a0599b700 1 --2- 172.21.15.60:0/4060583892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000d0a50 0x7f0a000cf8a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009840 tx=0x7f09fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c582247ab04cd7b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.820+0000 7f0a04999700 1 -- 172.21.15.60:0/4060583892 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc011830 con 0x7f0a000d0a50 2023-07-13T22:24:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.821+0000 7f0a04999700 1 -- 172.21.15.60:0/4060583892 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc011990 con 0x7f0a000d0a50 2023-07-13T22:24:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.821+0000 7f0a04999700 1 -- 172.21.15.60:0/4060583892 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc011d20 con 0x7f0a000d0a50 2023-07-13T22:24:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.821+0000 7f0a07bff700 1 -- 172.21.15.60:0/4060583892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000d0a50 msgr2=0x7f0a000cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.821+0000 7f0a07bff700 1 --2- 172.21.15.60:0/4060583892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000d0a50 0x7f0a000cf8a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009840 tx=0x7f09fc009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.821+0000 7f0a07bff700 1 -- 172.21.15.60:0/4060583892 shutdown_connections 2023-07-13T22:24:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.821+0000 7f0a07bff700 1 --2- 172.21.15.60:0/4060583892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000d0a50 0x7f0a000cf8a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.821+0000 7f0a07bff700 1 -- 172.21.15.60:0/4060583892 >> 172.21.15.60:0/4060583892 conn(0x7f0a00071600 msgr2=0x7f0a000719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.821+0000 7f0a07bff700 1 -- 172.21.15.60:0/4060583892 shutdown_connections 2023-07-13T22:24:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.821+0000 7f0a07bff700 1 -- 172.21.15.60:0/4060583892 wait complete. 2023-07-13T22:24:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.824+0000 7f0a07bff700 1 Processor -- start 2023-07-13T22:24:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.824+0000 7f0a07bff700 1 -- start start 2023-07-13T22:24:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.825+0000 7f0a07bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000cd260 0x7f0a000cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.825+0000 7f0a07bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a000d0a50 0x7f0a0006b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.825+0000 7f0a07bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a000d0e20 0x7f0a0006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.825+0000 7f0a07bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000ca7b0 con 0x7f0a000cd260 2023-07-13T22:24:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.825+0000 7f0a07bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0015b4b0 con 0x7f0a000d0e20 2023-07-13T22:24:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.825+0000 7f0a07bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000cc3a0 con 0x7f0a000d0a50 2023-07-13T22:24:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.825+0000 7f0a0599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000cd260 0x7f0a000cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.825+0000 7f0a0599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000cd260 0x7f0a000cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37438/0 (socket says 172.21.15.60:37438) 2023-07-13T22:24:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.825+0000 7f0a0599b700 1 -- 172.21.15.60:0/3351694749 learned_addr learned my addr 172.21.15.60:0/3351694749 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0619c700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a000d0e20 0x7f0a0006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0519a700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a000d0a50 0x7f0a0006b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0599b700 1 -- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a000d0a50 msgr2=0x7f0a0006b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0599b700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a000d0a50 0x7f0a0006b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0599b700 1 -- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a000d0e20 msgr2=0x7f0a0006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0599b700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a000d0e20 0x7f0a0006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0599b700 1 -- 172.21.15.60:0/3351694749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc0094a0 con 0x7f0a000cd260 2023-07-13T22:24:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0619c700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a000d0e20 0x7f0a0006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0519a700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a000d0a50 0x7f0a0006b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a0599b700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000cd260 0x7f0a000cabd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0090b0 tx=0x7f09fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f09f67fc700 1 -- 172.21.15.60:0/3351694749 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc01a900 con 0x7f0a000cd260 2023-07-13T22:24:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f09f67fc700 1 -- 172.21.15.60:0/3351694749 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc01aa60 con 0x7f0a000cd260 2023-07-13T22:24:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f09f67fc700 1 -- 172.21.15.60:0/3351694749 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc01add0 con 0x7f0a000cd260 2023-07-13T22:24:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.826+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a000c9e10 con 0x7f0a000cd260 2023-07-13T22:24:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.827+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a000ca3d0 con 0x7f0a000cd260 2023-07-13T22:24:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.827+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a000caa20 con 0x7f0a000cd260 2023-07-13T22:24:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.828+0000 7f09f67fc700 1 -- 172.21.15.60:0/3351694749 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f09fc022070 con 0x7f0a000cd260 2023-07-13T22:24:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.829+0000 7f09f67fc700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f09e406ad10 0x7f09e406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.829+0000 7f09f67fc700 1 -- 172.21.15.60:0/3351694749 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f09fc08a3c0 con 0x7f0a000cd260 2023-07-13T22:24:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.830+0000 7f0a0519a700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f09e406ad10 0x7f09e406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.830+0000 7f0a0519a700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f09e406ad10 0x7f09e406d1c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a001580f0 tx=0x7f09f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:12.834+0000 7f09f67fc700 1 -- 172.21.15.60:0/3351694749 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f09fc055950 con 0x7f0a000cd260 2023-07-13T22:24:13.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.118+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) v1 -- 0x7f0a0015b930 con 0x7f0a000cd260 2023-07-13T22:24:13.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.589+0000 7f09f67fc700 1 -- 172.21.15.60:0/3351694749 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v189) v1 ==== 151+0+0 (secure 0 0 0) 0x7f09fc058f70 con 0x7f0a000cd260 2023-07-13T22:24:13.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f09e406ad10 msgr2=0x7f09e406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:13.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f09e406ad10 0x7f09e406d1c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a001580f0 tx=0x7f09f0009040 comp rx=0 tx=0).stop 2023-07-13T22:24:13.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000cd260 msgr2=0x7f0a000cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000cd260 0x7f0a000cabd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0090b0 tx=0x7f09fc004750 comp rx=0 tx=0).stop 2023-07-13T22:24:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 shutdown_connections 2023-07-13T22:24:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f09e406ad10 0x7f09e406d1c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a000cd260 0x7f0a000cabd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a000d0e20 0x7f0a0006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 --2- 172.21.15.60:0/3351694749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a000d0a50 0x7f0a0006b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 >> 172.21.15.60:0/3351694749 conn(0x7f0a00071600 msgr2=0x7f0a0015f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 shutdown_connections 2023-07-13T22:24:13.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.592+0000 7f0a07bff700 1 -- 172.21.15.60:0/3351694749 wait complete. 2023-07-13T22:24:13.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-13T22:24:13.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.819+0000 7f50863ff700 1 Processor -- start 2023-07-13T22:24:13.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.819+0000 7f50863ff700 1 -- start start 2023-07-13T22:24:13.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.820+0000 7f50863ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50800b8200 0x7f50800cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:13.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.820+0000 7f50863ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800cede0 0x7f50800cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:13.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.820+0000 7f50863ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50800d1870 con 0x7f50800b1ac0 2023-07-13T22:24:13.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.820+0000 7f50863ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50800cf370 con 0x7f50800b8200 2023-07-13T22:24:13.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.820+0000 7f50863ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50800d12b0 con 0x7f50800cede0 2023-07-13T22:24:13.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.820+0000 7f507ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50800b8200 0x7f50800cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:13.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.820+0000 7f507ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50800b8200 0x7f50800cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37450/0 (socket says 172.21.15.60:37450) 2023-07-13T22:24:13.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.820+0000 7f507ffff700 1 -- 172.21.15.60:0/1052537855 learned_addr learned my addr 172.21.15.60:0/1052537855 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:13.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.820+0000 7f507f7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50800b1ac0 0x7f508015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36494/0 (socket says 172.21.15.60:36494) 2023-07-13T22:24:13.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f508499c700 1 --2- 172.21.15.60:0/1052537855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800cede0 0x7f50800cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:13.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f507ffff700 1 -- 172.21.15.60:0/1052537855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800cede0 msgr2=0x7f50800cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:13.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f507ffff700 1 --2- 172.21.15.60:0/1052537855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800cede0 0x7f50800cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:13.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f507ffff700 1 -- 172.21.15.60:0/1052537855 >> v1:172.21.15.60:6789/0 conn(0x7f50800b1ac0 legacy=0x7f508015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:13.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f507ffff700 1 -- 172.21.15.60:0/1052537855 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800d1590 con 0x7f50800b8200 2023-07-13T22:24:13.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f507effd700 1 -- 172.21.15.60:0/1052537855 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264785071 0 0) 0x7f50800d1870 con 0x7f50800b1ac0 2023-07-13T22:24:13.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f507ffff700 1 --2- 172.21.15.60:0/1052537855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 0x7f50800cf640 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f50700098f0 tx=0x7f5070009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6a160ccbf51ea2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:13.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f507effd700 1 -- 172.21.15.60:0/1052537855 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507000b8a0 con 0x7f50800b8200 2023-07-13T22:24:13.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f507effd700 1 -- 172.21.15.60:0/1052537855 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507000ba00 con 0x7f50800b8200 2023-07-13T22:24:13.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.821+0000 7f507effd700 1 -- 172.21.15.60:0/1052537855 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507000bd90 con 0x7f50800b8200 2023-07-13T22:24:13.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.822+0000 7f50863ff700 1 -- 172.21.15.60:0/1052537855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 msgr2=0x7f50800cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:13.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.822+0000 7f50863ff700 1 --2- 172.21.15.60:0/1052537855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 0x7f50800cf640 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f50700098f0 tx=0x7f5070009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:13.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.822+0000 7f50863ff700 1 -- 172.21.15.60:0/1052537855 shutdown_connections 2023-07-13T22:24:13.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.822+0000 7f50863ff700 1 --2- 172.21.15.60:0/1052537855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800cede0 0x7f50800cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:13.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.822+0000 7f50863ff700 1 --2- 172.21.15.60:0/1052537855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 0x7f50800cf640 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:13.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.822+0000 7f50863ff700 1 -- 172.21.15.60:0/1052537855 >> 172.21.15.60:0/1052537855 conn(0x7f508006b2f0 msgr2=0x7f508006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:13.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.822+0000 7f50863ff700 1 -- 172.21.15.60:0/1052537855 shutdown_connections 2023-07-13T22:24:13.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.822+0000 7f50863ff700 1 -- 172.21.15.60:0/1052537855 wait complete. 2023-07-13T22:24:13.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.824+0000 7f50863ff700 1 Processor -- start 2023-07-13T22:24:13.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.824+0000 7f50863ff700 1 -- start start 2023-07-13T22:24:13.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f50863ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800b1ac0 0x7f50801e3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:13.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f50863ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 0x7f508014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:13.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f50863ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cede0 0x7f50801e1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:13.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f50863ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50801e2670 con 0x7f50800b8200 2023-07-13T22:24:13.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f50863ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50801e2950 con 0x7f50800cede0 2023-07-13T22:24:13.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f50863ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50801e2c30 con 0x7f50800b1ac0 2023-07-13T22:24:13.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f507f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 0x7f508014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:13.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f507f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 0x7f508014afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37458/0 (socket says 172.21.15.60:37458) 2023-07-13T22:24:13.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f507f7fe700 1 -- 172.21.15.60:0/1585026803 learned_addr learned my addr 172.21.15.60:0/1585026803 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:13.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f508499c700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cede0 0x7f50801e1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:13.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.825+0000 7f507ffff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800b1ac0 0x7f50801e3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:13.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507f7fe700 1 -- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800b1ac0 msgr2=0x7f50801e3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:13.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507f7fe700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800b1ac0 0x7f50801e3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:13.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507f7fe700 1 -- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cede0 msgr2=0x7f50801e1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:13.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507f7fe700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cede0 0x7f50801e1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:13.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507f7fe700 1 -- 172.21.15.60:0/1585026803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070009550 con 0x7f50800b8200 2023-07-13T22:24:13.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f508499c700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cede0 0x7f50801e1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:13.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507ffff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800b1ac0 0x7f50801e3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:13.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507f7fe700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 0x7f508014afc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f50740011e0 tx=0x7f507400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:13.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507cff9700 1 -- 172.21.15.60:0/1585026803 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074004c80 con 0x7f50800b8200 2023-07-13T22:24:13.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507cff9700 1 -- 172.21.15.60:0/1585026803 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5074004e00 con 0x7f50800b8200 2023-07-13T22:24:13.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f507cff9700 1 -- 172.21.15.60:0/1585026803 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074007310 con 0x7f50800b8200 2023-07-13T22:24:13.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50801e2f10 con 0x7f50800b8200 2023-07-13T22:24:13.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.826+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50801e3c50 con 0x7f50800b8200 2023-07-13T22:24:13.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.827+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508004e210 con 0x7f50800b8200 2023-07-13T22:24:13.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.828+0000 7f507cff9700 1 -- 172.21.15.60:0/1585026803 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5074007470 con 0x7f50800b8200 2023-07-13T22:24:13.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.829+0000 7f507cff9700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f506c06ace0 0x7f506c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:13.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.829+0000 7f507cff9700 1 -- 172.21.15.60:0/1585026803 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5074088f80 con 0x7f50800b8200 2023-07-13T22:24:13.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.830+0000 7f507ffff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f506c06ace0 0x7f506c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:13.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.830+0000 7f507ffff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f506c06ace0 0x7f506c06d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5070000c00 tx=0x7f5070008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:13.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:13.834+0000 7f507cff9700 1 -- 172.21.15.60:0/1585026803 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f50740542b0 con 0x7f50800b8200 2023-07-13T22:24:14.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.046+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f50800d07e0 con 0x7f50800b8200 2023-07-13T22:24:14.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.050+0000 7f507cff9700 1 -- 172.21.15.60:0/1585026803 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f50740578d0 con 0x7f50800b8200 2023-07-13T22:24:14.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f506c06ace0 msgr2=0x7f506c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:14.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f506c06ace0 0x7f506c06d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5070000c00 tx=0x7f5070008030 comp rx=0 tx=0).stop 2023-07-13T22:24:14.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 msgr2=0x7f508014afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:14.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 0x7f508014afc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f50740011e0 tx=0x7f507400cae0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 shutdown_connections 2023-07-13T22:24:14.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f506c06ace0 0x7f506c06d190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50800cede0 0x7f50801e1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50800b1ac0 0x7f50801e3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 --2- 172.21.15.60:0/1585026803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50800b8200 0x7f508014afc0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 >> 172.21.15.60:0/1585026803 conn(0x7f508006b2f0 msgr2=0x7f50800b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:14.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.052+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 shutdown_connections 2023-07-13T22:24:14.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.053+0000 7f50863ff700 1 -- 172.21.15.60:0/1585026803 wait complete. 2023-07-13T22:24:14.052 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:24:14.053 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:24:14.156 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:24:14.259 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:24:14.259 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:24:14.259 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:14.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:24:14.307 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:24:14.307 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:14.355 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:24:14.355 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:24:14.355 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:14.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:24:14.402 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:24:14.403 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:14.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:24:14.450 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-13T22:24:14.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:14.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.672+0000 7fdea61ff700 1 Processor -- start 2023-07-13T22:24:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.672+0000 7fdea61ff700 1 -- start start 2023-07-13T22:24:14.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.673+0000 7fdea61ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdea00d3750 0x7fdea00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:14.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.673+0000 7fdea61ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdea00cfad0 0x7fdea00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:14.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.673+0000 7fdea61ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea00cd950 con 0x7fdea00b8400 2023-07-13T22:24:14.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.673+0000 7fdea61ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea00cff50 con 0x7fdea00cfad0 2023-07-13T22:24:14.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.673+0000 7fdea61ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea00d0ac0 con 0x7fdea00d3750 2023-07-13T22:24:14.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.673+0000 7fde9ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdea00cfad0 0x7fdea00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:14.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.673+0000 7fde9ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdea00cfad0 0x7fdea00d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37472/0 (socket says 172.21.15.60:37472) 2023-07-13T22:24:14.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.673+0000 7fde9ffff700 1 -- 172.21.15.60:0/3889537932 learned_addr learned my addr 172.21.15.60:0/3889537932 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:14.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.673+0000 7fde9effd700 1 --2- 172.21.15.60:0/3889537932 >> v2:172.21.15.105:3300/0 conn(0x7fdea00d3750 0x7fdea00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:14.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.674+0000 7fde9ffff700 1 -- 172.21.15.60:0/3889537932 >> v2:172.21.15.105:3300/0 conn(0x7fdea00d3750 msgr2=0x7fdea00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:14.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.674+0000 7fde9ffff700 1 --2- 172.21.15.60:0/3889537932 >> v2:172.21.15.105:3300/0 conn(0x7fdea00d3750 0x7fdea00cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.674+0000 7fde9ffff700 1 -- 172.21.15.60:0/3889537932 >> v1:172.21.15.105:6789/0 conn(0x7fdea00b8400 legacy=0x7fdea00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.674+0000 7fde9ffff700 1 -- 172.21.15.60:0/3889537932 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea00b87d0 con 0x7fdea00cfad0 2023-07-13T22:24:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.674+0000 7fde9e7fc700 1 -- 172.21.15.60:0/3889537932 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810522025 0 0) 0x7fde90001110 con 0x7fdea00b8400 2023-07-13T22:24:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.674+0000 7fde9ffff700 1 --2- 172.21.15.60:0/3889537932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 0x7fdea00d0f50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdea00b8cf0 tx=0x7fde9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738e096395de6009 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.674+0000 7fde9e7fc700 1 -- 172.21.15.60:0/3889537932 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800e070 con 0x7fdea00cfad0 2023-07-13T22:24:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.674+0000 7fde9e7fc700 1 -- 172.21.15.60:0/3889537932 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9800b910 con 0x7fdea00cfad0 2023-07-13T22:24:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.674+0000 7fde9e7fc700 1 -- 172.21.15.60:0/3889537932 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800bcc0 con 0x7fdea00cfad0 2023-07-13T22:24:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.675+0000 7fdea61ff700 1 -- 172.21.15.60:0/3889537932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 msgr2=0x7fdea00d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.675+0000 7fdea61ff700 1 --2- 172.21.15.60:0/3889537932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 0x7fdea00d0f50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdea00b8cf0 tx=0x7fde9800b330 comp rx=0 tx=0).stop 2023-07-13T22:24:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.675+0000 7fdea61ff700 1 -- 172.21.15.60:0/3889537932 shutdown_connections 2023-07-13T22:24:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.675+0000 7fdea61ff700 1 --2- 172.21.15.60:0/3889537932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 0x7fdea00d0f50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.675+0000 7fdea61ff700 1 --2- 172.21.15.60:0/3889537932 >> v2:172.21.15.105:3300/0 conn(0x7fdea00d3750 0x7fdea00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.675+0000 7fdea61ff700 1 -- 172.21.15.60:0/3889537932 >> 172.21.15.60:0/3889537932 conn(0x7fdea006b390 msgr2=0x7fdea006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.675+0000 7fdea61ff700 1 -- 172.21.15.60:0/3889537932 shutdown_connections 2023-07-13T22:24:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.675+0000 7fdea61ff700 1 -- 172.21.15.60:0/3889537932 wait complete. 2023-07-13T22:24:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.677+0000 7fdea61ff700 1 Processor -- start 2023-07-13T22:24:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.678+0000 7fdea61ff700 1 -- start start 2023-07-13T22:24:14.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fdea61ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdea00b8400 0x7fdea0151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:14.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fdea61ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 0x7fdea015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:14.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fdea61ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea00d3750 0x7fdea00c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:14.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fdea61ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea015e9b0 con 0x7fdea00cfad0 2023-07-13T22:24:14.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fdea61ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea015dda0 con 0x7fdea00d3750 2023-07-13T22:24:14.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fdea61ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea015e080 con 0x7fdea00b8400 2023-07-13T22:24:14.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fde9effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 0x7fdea015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:14.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fde9effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 0x7fdea015ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37476/0 (socket says 172.21.15.60:37476) 2023-07-13T22:24:14.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fde9effd700 1 -- 172.21.15.60:0/1546593528 learned_addr learned my addr 172.21.15.60:0/1546593528 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fde9ffff700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea00d3750 0x7fdea00c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.679+0000 7fde9f7fe700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdea00b8400 0x7fdea0151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fde9effd700 1 -- 172.21.15.60:0/1546593528 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdea00b8400 msgr2=0x7fdea0151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fde9effd700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdea00b8400 0x7fdea0151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fde9effd700 1 -- 172.21.15.60:0/1546593528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea00d3750 msgr2=0x7fdea00c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fde9effd700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea00d3750 0x7fdea00c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fde9effd700 1 -- 172.21.15.60:0/1546593528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98009b30 con 0x7fdea00cfad0 2023-07-13T22:24:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fde9effd700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 0x7fdea015ee40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fde9400e980 tx=0x7fde9400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fde87fff700 1 -- 172.21.15.60:0/1546593528 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94009710 con 0x7fdea00cfad0 2023-07-13T22:24:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fde87fff700 1 -- 172.21.15.60:0/1546593528 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9400c690 con 0x7fdea00cfad0 2023-07-13T22:24:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fde87fff700 1 -- 172.21.15.60:0/1546593528 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400ca40 con 0x7fdea00cfad0 2023-07-13T22:24:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea015e360 con 0x7fdea00cfad0 2023-07-13T22:24:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.680+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea015f380 con 0x7fdea00cfad0 2023-07-13T22:24:14.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.681+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea015eb40 con 0x7fdea00cfad0 2023-07-13T22:24:14.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.682+0000 7fde87fff700 1 -- 172.21.15.60:0/1546593528 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde9400cba0 con 0x7fdea00cfad0 2023-07-13T22:24:14.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.684+0000 7fde87fff700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8c06ace0 0x7fde8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:14.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.684+0000 7fde87fff700 1 -- 172.21.15.60:0/1546593528 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fde940891e0 con 0x7fdea00cfad0 2023-07-13T22:24:14.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.684+0000 7fde9f7fe700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8c06ace0 0x7fde8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:14.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.685+0000 7fde9f7fe700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8c06ace0 0x7fde8c06d190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fde90000f60 tx=0x7fde90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.688+0000 7fde87fff700 1 -- 172.21.15.60:0/1546593528 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fde94057fa0 con 0x7fdea00cfad0 2023-07-13T22:24:14.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.947+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdea015aba0 con 0x7fdea00cfad0 2023-07-13T22:24:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.947+0000 7fde87fff700 1 -- 172.21.15.60:0/1546593528 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fde94021de0 con 0x7fdea00cfad0 2023-07-13T22:24:14.929 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:14.929 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3226044033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3226044033},{"type":"v1","addr":"172.21.15.105:6834","nonce":3226044033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/4188205098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":4188205098},{"type":"v1","addr":"172.21.15.105:6835","nonce":4188205098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5599,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2171011214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":2171011214},{"type":"v1","addr":"172.21.15.60:6836","nonce":2171011214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/1623228645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1623228645},{"type":"v1","addr":"172.21.15.60:6837","nonce":1623228645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-07-13T22:24:14.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8c06ace0 msgr2=0x7fde8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:14.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8c06ace0 0x7fde8c06d190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fde90000f60 tx=0x7fde90009040 comp rx=0 tx=0).stop 2023-07-13T22:24:14.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 msgr2=0x7fdea015ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:14.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 0x7fdea015ee40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fde9400e980 tx=0x7fde9400ec90 comp rx=0 tx=0).stop 2023-07-13T22:24:14.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 shutdown_connections 2023-07-13T22:24:14.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8c06ace0 0x7fde8c06d190 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdea00d3750 0x7fdea00c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdea00b8400 0x7fdea0151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 --2- 172.21.15.60:0/1546593528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea00cfad0 0x7fdea015ee40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 >> 172.21.15.60:0/1546593528 conn(0x7fdea006b390 msgr2=0x7fdea00d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 shutdown_connections 2023-07-13T22:24:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:14.950+0000 7fdea61ff700 1 -- 172.21.15.60:0/1546593528 wait complete. 2023-07-13T22:24:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2023-07-13T22:24:14.950 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:24:14.950 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-13T22:24:14.950 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:24:14.950 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:24:14.951 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:24:14.951 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-13T22:24:14.951 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:24:14.951 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:24:14.951 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:24:14.952 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-13T22:24:14.952 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:24:14.952 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:24:14.952 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:24:14.953 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-13T22:24:14.953 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:24:14.953 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:24:14.965 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:24:14.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:24:14.966 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:24:14.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:24:14.969 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:24:14.969 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:24:14.970 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:24:14.970 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:24:15.009 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:24:15.009 DEBUG:teuthology.parallel:result is None 2023-07-13T22:24:15.011 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:24:15.012 DEBUG:teuthology.parallel:result is None 2023-07-13T22:24:15.013 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:24:15.014 DEBUG:teuthology.parallel:result is None 2023-07-13T22:24:15.016 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:24:15.017 DEBUG:teuthology.parallel:result is None 2023-07-13T22:24:15.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-13T22:24:15.164 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:24:15.165 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:24:15.166 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:24:15.166 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:24:15.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.248+0000 7f4b661ff700 1 Processor -- start 2023-07-13T22:24:15.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.248+0000 7f4b661ff700 1 -- start start 2023-07-13T22:24:15.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.249+0000 7f4b661ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4b600b8400 0x7f4b600cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.249+0000 7f4b661ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b600cfca0 0x7f4b600d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.249+0000 7f4b661ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b600d0ac0 con 0x7f4b600d3750 2023-07-13T22:24:15.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.249+0000 7f4b661ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b600b87d0 con 0x7f4b600b8400 2023-07-13T22:24:15.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.249+0000 7f4b661ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b600b4eb0 con 0x7f4b600cfca0 2023-07-13T22:24:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.249+0000 7f4b5f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4b600b8400 0x7f4b600cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.249+0000 7f4b5f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4b600b8400 0x7f4b600cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57318/0 (socket says 172.21.15.60:57318) 2023-07-13T22:24:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.249+0000 7f4b5f7fe700 1 -- 172.21.15.60:0/1912077766 learned_addr learned my addr 172.21.15.60:0/1912077766 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5ffff700 1 --2- 172.21.15.60:0/1912077766 >> v2:172.21.15.105:3300/0 conn(0x7f4b600cfca0 0x7f4b600d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5f7fe700 1 -- 172.21.15.60:0/1912077766 >> v2:172.21.15.105:3300/0 conn(0x7f4b600cfca0 msgr2=0x7f4b600d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5f7fe700 1 --2- 172.21.15.60:0/1912077766 >> v2:172.21.15.105:3300/0 conn(0x7f4b600cfca0 0x7f4b600d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5f7fe700 1 -- 172.21.15.60:0/1912077766 >> v1:172.21.15.105:6789/0 conn(0x7f4b600d3750 legacy=0x7f4b600cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5f7fe700 1 -- 172.21.15.60:0/1912077766 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600b5190 con 0x7f4b600b8400 2023-07-13T22:24:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5ffff700 1 --2- 172.21.15.60:0/1912077766 >> v2:172.21.15.105:3300/0 conn(0x7f4b600cfca0 0x7f4b600d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5f7fe700 1 --2- 172.21.15.60:0/1912077766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600b8400 0x7f4b600cd240 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4b600d0070 tx=0x7f4b50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a12960bac4a8ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5e7fc700 1 -- 172.21.15.60:0/1912077766 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50004230 con 0x7f4b600b8400 2023-07-13T22:24:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5e7fc700 1 -- 172.21.15.60:0/1912077766 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b50004390 con 0x7f4b600b8400 2023-07-13T22:24:15.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.250+0000 7f4b5e7fc700 1 -- 172.21.15.60:0/1912077766 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b500036a0 con 0x7f4b600b8400 2023-07-13T22:24:15.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.251+0000 7f4b661ff700 1 -- 172.21.15.60:0/1912077766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600b8400 msgr2=0x7f4b600cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:15.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.251+0000 7f4b661ff700 1 --2- 172.21.15.60:0/1912077766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600b8400 0x7f4b600cd240 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4b600d0070 tx=0x7f4b50009bd0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.251+0000 7f4b661ff700 1 -- 172.21.15.60:0/1912077766 shutdown_connections 2023-07-13T22:24:15.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.251+0000 7f4b661ff700 1 --2- 172.21.15.60:0/1912077766 >> v2:172.21.15.105:3300/0 conn(0x7f4b600cfca0 0x7f4b600d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.251+0000 7f4b661ff700 1 --2- 172.21.15.60:0/1912077766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600b8400 0x7f4b600cd240 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.251+0000 7f4b661ff700 1 -- 172.21.15.60:0/1912077766 >> 172.21.15.60:0/1912077766 conn(0x7f4b6006b390 msgr2=0x7f4b6006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:15.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.251+0000 7f4b661ff700 1 -- 172.21.15.60:0/1912077766 shutdown_connections 2023-07-13T22:24:15.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.251+0000 7f4b661ff700 1 -- 172.21.15.60:0/1912077766 wait complete. 2023-07-13T22:24:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.253+0000 7f4b661ff700 1 Processor -- start 2023-07-13T22:24:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.253+0000 7f4b661ff700 1 -- start start 2023-07-13T22:24:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b661ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b600b8400 0x7f4b6015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b661ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600cfca0 0x7f4b6015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b661ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b600d3750 0x7f4b600b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b661ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601f1a00 con 0x7f4b600cfca0 2023-07-13T22:24:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b661ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6015b170 con 0x7f4b600d3750 2023-07-13T22:24:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b661ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6015d090 con 0x7f4b600b8400 2023-07-13T22:24:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600cfca0 0x7f4b6015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600cfca0 0x7f4b6015be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57334/0 (socket says 172.21.15.60:57334) 2023-07-13T22:24:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b5effd700 1 -- 172.21.15.60:0/372746006 learned_addr learned my addr 172.21.15.60:0/372746006 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b5ffff700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b600d3750 0x7f4b600b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.254+0000 7f4b5f7fe700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b600b8400 0x7f4b6015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b5effd700 1 -- 172.21.15.60:0/372746006 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b600b8400 msgr2=0x7f4b6015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b5effd700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b600b8400 0x7f4b6015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b5effd700 1 -- 172.21.15.60:0/372746006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b600d3750 msgr2=0x7f4b600b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b5effd700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b600d3750 0x7f4b600b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b5effd700 1 -- 172.21.15.60:0/372746006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50009550 con 0x7f4b600cfca0 2023-07-13T22:24:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b5effd700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600cfca0 0x7f4b6015be10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400c4a0 tx=0x7f4b5400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b47fff700 1 -- 172.21.15.60:0/372746006 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54011790 con 0x7f4b600cfca0 2023-07-13T22:24:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b47fff700 1 -- 172.21.15.60:0/372746006 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b540118f0 con 0x7f4b600cfca0 2023-07-13T22:24:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6015daa0 con 0x7f4b600cfca0 2023-07-13T22:24:15.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b47fff700 1 -- 172.21.15.60:0/372746006 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54011ca0 con 0x7f4b600cfca0 2023-07-13T22:24:15.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.255+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6015cad0 con 0x7f4b600cfca0 2023-07-13T22:24:15.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.256+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b600d1c20 con 0x7f4b600cfca0 2023-07-13T22:24:15.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.257+0000 7f4b47fff700 1 -- 172.21.15.60:0/372746006 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b5401c460 con 0x7f4b600cfca0 2023-07-13T22:24:15.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.259+0000 7f4b47fff700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b4c06ace0 0x7f4b4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.259+0000 7f4b47fff700 1 -- 172.21.15.60:0/372746006 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4b54088e70 con 0x7f4b600cfca0 2023-07-13T22:24:15.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.259+0000 7f4b5f7fe700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b4c06ace0 0x7f4b4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.260+0000 7f4b5f7fe700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b4c06ace0 0x7f4b4c06d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009130 tx=0x7f4b50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.263+0000 7f4b47fff700 1 -- 172.21.15.60:0/372746006 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4b54054230 con 0x7f4b600cfca0 2023-07-13T22:24:15.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.487+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4b6004e280 con 0x7f4b600cfca0 2023-07-13T22:24:15.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.600+0000 7f4b47fff700 1 -- 172.21.15.60:0/372746006 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4b54057850 con 0x7f4b600cfca0 2023-07-13T22:24:15.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b4c06ace0 msgr2=0x7f4b4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:15.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b4c06ace0 0x7f4b4c06d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009130 tx=0x7f4b50008030 comp rx=0 tx=0).stop 2023-07-13T22:24:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600cfca0 msgr2=0x7f4b6015be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600cfca0 0x7f4b6015be10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400c4a0 tx=0x7f4b5400c7b0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 shutdown_connections 2023-07-13T22:24:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b4c06ace0 0x7f4b4c06d190 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b600d3750 0x7f4b600b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b600b8400 0x7f4b6015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 --2- 172.21.15.60:0/372746006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b600cfca0 0x7f4b6015be10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 >> 172.21.15.60:0/372746006 conn(0x7f4b6006b390 msgr2=0x7f4b600d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:15.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.603+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 shutdown_connections 2023-07-13T22:24:15.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.604+0000 7f4b661ff700 1 -- 172.21.15.60:0/372746006 wait complete. 2023-07-13T22:24:15.588 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-07-13T22:24:15.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-13T22:24:15.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.829+0000 7f8ed0bff700 1 Processor -- start 2023-07-13T22:24:15.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.829+0000 7f8ed0bff700 1 -- start start 2023-07-13T22:24:15.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.830+0000 7f8ed0bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ecc0b8400 0x7f8ecc0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.830+0000 7f8ed0bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8ecc0cfca0 0x7f8ecc0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.830+0000 7f8ed0bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc0d0ac0 con 0x7f8ecc0d3750 2023-07-13T22:24:15.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.830+0000 7f8ed0bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b87d0 con 0x7f8ecc0cfca0 2023-07-13T22:24:15.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.830+0000 7f8ed0bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b4eb0 con 0x7f8ecc0b8400 2023-07-13T22:24:15.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.830+0000 7f8ecad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8ecc0cfca0 0x7f8ecc0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.830+0000 7f8ec3fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8ecc0d3750 0x7f8ecc0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34442/0 (socket says 172.21.15.60:34442) 2023-07-13T22:24:15.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.830+0000 7f8ec3fff700 1 -- 172.21.15.60:0/2481875508 learned_addr learned my addr 172.21.15.60:0/2481875508 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:15.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8eca59c700 1 --2- 172.21.15.60:0/2481875508 >> v2:172.21.15.105:3300/0 conn(0x7f8ecc0b8400 0x7f8ecc0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8ecad9d700 1 -- 172.21.15.60:0/2481875508 >> v2:172.21.15.105:3300/0 conn(0x7f8ecc0b8400 msgr2=0x7f8ecc0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:15.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8ecad9d700 1 --2- 172.21.15.60:0/2481875508 >> v2:172.21.15.105:3300/0 conn(0x7f8ecc0b8400 0x7f8ecc0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8ecad9d700 1 -- 172.21.15.60:0/2481875508 >> v1:172.21.15.60:6789/0 conn(0x7f8ecc0d3750 legacy=0x7f8ecc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:15.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8ecad9d700 1 -- 172.21.15.60:0/2481875508 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc0b5190 con 0x7f8ecc0cfca0 2023-07-13T22:24:15.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8ec9d9b700 1 -- 172.21.15.60:0/2481875508 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406207474 0 0) 0x7f8ecc0d0ac0 con 0x7f8ecc0d3750 2023-07-13T22:24:15.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8ecad9d700 1 --2- 172.21.15.60:0/2481875508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0cfca0 0x7f8ecc0d0f50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0cf560 tx=0x7f8ec400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1c6f92c3518b7d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:15.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8ec9d9b700 1 -- 172.21.15.60:0/2481875508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400e070 con 0x7f8ecc0cfca0 2023-07-13T22:24:15.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8ec9d9b700 1 -- 172.21.15.60:0/2481875508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec400bbd0 con 0x7f8ecc0cfca0 2023-07-13T22:24:15.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.831+0000 7f8ec9d9b700 1 -- 172.21.15.60:0/2481875508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4012430 con 0x7f8ecc0cfca0 2023-07-13T22:24:15.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.832+0000 7f8ed0bff700 1 -- 172.21.15.60:0/2481875508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0cfca0 msgr2=0x7f8ecc0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:15.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.832+0000 7f8ed0bff700 1 --2- 172.21.15.60:0/2481875508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0cfca0 0x7f8ecc0d0f50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0cf560 tx=0x7f8ec400b330 comp rx=0 tx=0).stop 2023-07-13T22:24:15.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.832+0000 7f8ed0bff700 1 -- 172.21.15.60:0/2481875508 shutdown_connections 2023-07-13T22:24:15.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.832+0000 7f8ed0bff700 1 --2- 172.21.15.60:0/2481875508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0cfca0 0x7f8ecc0d0f50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.832+0000 7f8ed0bff700 1 --2- 172.21.15.60:0/2481875508 >> v2:172.21.15.105:3300/0 conn(0x7f8ecc0b8400 0x7f8ecc0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.832+0000 7f8ed0bff700 1 -- 172.21.15.60:0/2481875508 >> 172.21.15.60:0/2481875508 conn(0x7f8ecc06b390 msgr2=0x7f8ecc06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:15.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.832+0000 7f8ed0bff700 1 -- 172.21.15.60:0/2481875508 shutdown_connections 2023-07-13T22:24:15.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.832+0000 7f8ed0bff700 1 -- 172.21.15.60:0/2481875508 wait complete. 2023-07-13T22:24:15.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.834+0000 7f8ed0bff700 1 Processor -- start 2023-07-13T22:24:15.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.835+0000 7f8ed0bff700 1 -- start start 2023-07-13T22:24:15.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ed0bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc0b8400 0x7f8ecc153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ed0bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ecc0cfca0 0x7f8ecc0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ed0bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0d3750 0x7f8ecc0b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ed0bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0c7760 con 0x7f8ecc0d3750 2023-07-13T22:24:15.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ed0bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc153ed0 con 0x7f8ecc0b8400 2023-07-13T22:24:15.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ed0bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0c6b50 con 0x7f8ecc0cfca0 2023-07-13T22:24:15.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ecad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0d3750 0x7f8ecc0b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ecad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0d3750 0x7f8ecc0b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57352/0 (socket says 172.21.15.60:57352) 2023-07-13T22:24:15.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ecad9d700 1 -- 172.21.15.60:0/1247826548 learned_addr learned my addr 172.21.15.60:0/1247826548 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:15.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ec3fff700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ecc0cfca0 0x7f8ecc0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8eca59c700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc0b8400 0x7f8ecc153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ecad9d700 1 -- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ecc0cfca0 msgr2=0x7f8ecc0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ecad9d700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ecc0cfca0 0x7f8ecc0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ecad9d700 1 -- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc0b8400 msgr2=0x7f8ecc153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ecad9d700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc0b8400 0x7f8ecc153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.836+0000 7f8ecad9d700 1 -- 172.21.15.60:0/1247826548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec4009a70 con 0x7f8ecc0d3750 2023-07-13T22:24:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.837+0000 7f8eca59c700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc0b8400 0x7f8ecc153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.837+0000 7f8ec3fff700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ecc0cfca0 0x7f8ecc0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:15.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.837+0000 7f8ecad9d700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0d3750 0x7f8ecc0b6fa0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4013000 tx=0x7f8ec4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.837+0000 7f8ec2ffd700 1 -- 172.21.15.60:0/1247826548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4012430 con 0x7f8ecc0d3750 2023-07-13T22:24:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.837+0000 7f8ec2ffd700 1 -- 172.21.15.60:0/1247826548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4020070 con 0x7f8ecc0d3750 2023-07-13T22:24:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.837+0000 7f8ec2ffd700 1 -- 172.21.15.60:0/1247826548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4004280 con 0x7f8ecc0d3750 2023-07-13T22:24:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.837+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc0c6e30 con 0x7f8ecc0d3750 2023-07-13T22:24:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.837+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc0c73f0 con 0x7f8ecc0d3750 2023-07-13T22:24:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.838+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc0c75b0 con 0x7f8ecc0d3750 2023-07-13T22:24:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.839+0000 7f8ec2ffd700 1 -- 172.21.15.60:0/1247826548 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8ec40043e0 con 0x7f8ecc0d3750 2023-07-13T22:24:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.840+0000 7f8ec2ffd700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8eac06ace0 0x7f8eac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.840+0000 7f8ec2ffd700 1 -- 172.21.15.60:0/1247826548 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8ec4022070 con 0x7f8ecc0d3750 2023-07-13T22:24:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.840+0000 7f8eca59c700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8eac06ace0 0x7f8eac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.841+0000 7f8eca59c700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8eac06ace0 0x7f8eac06d190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009820 tx=0x7f8ebc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:15.844+0000 7f8ec2ffd700 1 -- 172.21.15.60:0/1247826548 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8ec4056740 con 0x7f8ecc0d3750 2023-07-13T22:24:16.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.137+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8ecc0c78a0 con 0x7f8ecc0d3750 2023-07-13T22:24:16.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.137+0000 7f8ec2ffd700 1 -- 172.21.15.60:0/1247826548 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8ec4059d60 con 0x7f8ecc0d3750 2023-07-13T22:24:16.119 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:16.119 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:24:16.120 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-07-13T22:24:16.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.120 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-07-13T22:24:16.120 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw==", 2023-07-13T22:24:16.120 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:16.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:16.121 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:16.121 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.121 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-07-13T22:24:16.121 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g==", 2023-07-13T22:24:16.121 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:16.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:16.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:16.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.122 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.122 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-07-13T22:24:16.123 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA==", 2023-07-13T22:24:16.123 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:16.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:16.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:16.123 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.124 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-07-13T22:24:16.124 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ==", 2023-07-13T22:24:16.124 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:16.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:16.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:16.125 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.125 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.125 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-07-13T22:24:16.125 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ==", 2023-07-13T22:24:16.125 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:16.126 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:16.126 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.126 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-07-13T22:24:16.126 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw==", 2023-07-13T22:24:16.127 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:16.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:16.127 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.127 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-07-13T22:24:16.128 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q==", 2023-07-13T22:24:16.128 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.128 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:16.128 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:16.128 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.129 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-07-13T22:24:16.129 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ==", 2023-07-13T22:24:16.129 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:16.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:16.129 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.129 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.130 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-07-13T22:24:16.130 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg==", 2023-07-13T22:24:16.130 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:16.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:16.130 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.131 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-07-13T22:24:16.131 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA==", 2023-07-13T22:24:16.131 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.131 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:16.131 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:16.132 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.132 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.132 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-07-13T22:24:16.132 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw==", 2023-07-13T22:24:16.132 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.132 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:16.133 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:16.133 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.133 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.133 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-07-13T22:24:16.133 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ==", 2023-07-13T22:24:16.133 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.134 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:16.134 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:16.134 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.134 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.134 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-07-13T22:24:16.134 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w==", 2023-07-13T22:24:16.135 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.135 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:16.135 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2023-07-13T22:24:16.135 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-13T22:24:16.135 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.135 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-07-13T22:24:16.136 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw==", 2023-07-13T22:24:16.136 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.136 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:16.136 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:24:16.136 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:24:16.136 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:24:16.136 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.137 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-07-13T22:24:16.137 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg==", 2023-07-13T22:24:16.137 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.137 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:16.137 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:24:16.137 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:24:16.138 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:24:16.138 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.138 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-07-13T22:24:16.138 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q==", 2023-07-13T22:24:16.138 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.138 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:16.139 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:24:16.139 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:24:16.139 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:24:16.139 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.139 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-07-13T22:24:16.139 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBk7CgGBhAAaK88V4CEyrYfZx7wv+XG3g==", 2023-07-13T22:24:16.140 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.140 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:24:16.140 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:16.140 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:16.140 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.140 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.141 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-07-13T22:24:16.141 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ==", 2023-07-13T22:24:16.141 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.141 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:24:16.141 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:24:16.141 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.141 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:16.142 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:16.142 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-07-13T22:24:16.142 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ==", 2023-07-13T22:24:16.142 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:16.142 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:24:16.142 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:24:16.142 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:16.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:16.143 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:24:16.143 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:24:16.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.139+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8eac06ace0 msgr2=0x7f8eac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.139+0000 7f8ed0bff700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8eac06ace0 0x7f8eac06d190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009820 tx=0x7f8ebc008040 comp rx=0 tx=0).stop 2023-07-13T22:24:16.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0d3750 msgr2=0x7f8ecc0b6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0d3750 0x7f8ecc0b6fa0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4013000 tx=0x7f8ec4004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 shutdown_connections 2023-07-13T22:24:16.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8eac06ace0 0x7f8eac06d190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc0d3750 0x7f8ecc0b6fa0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc0b8400 0x7f8ecc153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 --2- 172.21.15.60:0/1247826548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ecc0cfca0 0x7f8ecc0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 >> 172.21.15.60:0/1247826548 conn(0x7f8ecc06b390 msgr2=0x7f8ecc0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:16.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 shutdown_connections 2023-07-13T22:24:16.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.140+0000 7f8ed0bff700 1 -- 172.21.15.60:0/1247826548 wait complete. 2023-07-13T22:24:16.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:16.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.367+0000 7fba147ff700 1 Processor -- start 2023-07-13T22:24:16.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.368+0000 7fba147ff700 1 -- start start 2023-07-13T22:24:16.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba147ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba0c0d35c0 0x7fba0c0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba147ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fba0c0d13f0 0x7fba0c15c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba147ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0c0d09d0 con 0x7fba0c0b8570 2023-07-13T22:24:16.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba147ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c0d3b30 con 0x7fba0c0d13f0 2023-07-13T22:24:16.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba147ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c0d1870 con 0x7fba0c0d35c0 2023-07-13T22:24:16.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba1259b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fba0c0b8570 0x7fba0c0ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34446/0 (socket says 172.21.15.60:34446) 2023-07-13T22:24:16.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba1259b700 1 -- 172.21.15.60:0/1208287043 learned_addr learned my addr 172.21.15.60:0/1208287043 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:16.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba11d9a700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba0c0d35c0 0x7fba0c0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba12d9c700 1 --2- 172.21.15.60:0/1208287043 >> v2:172.21.15.105:3300/0 conn(0x7fba0c0d13f0 0x7fba0c15c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028086438 0 0) 0x7fba0c0d09d0 con 0x7fba0c0b8570 2023-07-13T22:24:16.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.369+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9f80035b0 con 0x7fba0c0b8570 2023-07-13T22:24:16.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 640869166 0 0) 0x7fb9f80035b0 con 0x7fba0c0b8570 2023-07-13T22:24:16.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba0c0d35c0 msgr2=0x7fba0c0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:16.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba0c0d35c0 0x7fba0c0d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 >> v2:172.21.15.105:3300/0 conn(0x7fba0c0d13f0 msgr2=0x7fba0c15c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:16.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 --2- 172.21.15.60:0/1208287043 >> v2:172.21.15.105:3300/0 conn(0x7fba0c0d13f0 0x7fba0c15c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c0cda10 con 0x7fba0c0b8570 2023-07-13T22:24:16.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fba080017f0 con 0x7fba0c0b8570 2023-07-13T22:24:16.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 >> v1:172.21.15.60:6789/0 conn(0x7fba0c0b8570 legacy=0x7fba0c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f8003a00 0x7fb9f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba0c0d35c0 0x7fb9f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9f8004d90 0x7fb9f8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c0d3b30 con 0x7fb9f8004d90 2023-07-13T22:24:16.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c0d1870 con 0x7fb9f8003a00 2023-07-13T22:24:16.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f800ba10 con 0x7fba0c0d35c0 2023-07-13T22:24:16.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba12d9c700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9f8004d90 0x7fb9f8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba11d9a700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba0c0d35c0 0x7fb9f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.370+0000 7fba1259b700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f8003a00 0x7fb9f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.371+0000 7fba12d9c700 1 -- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba0c0d35c0 msgr2=0x7fb9f8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:16.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.371+0000 7fba12d9c700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba0c0d35c0 0x7fb9f8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.371+0000 7fba12d9c700 1 -- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f8003a00 msgr2=0x7fb9f8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:16.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.371+0000 7fba12d9c700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9f8003a00 0x7fb9f8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.371+0000 7fba12d9c700 1 -- 172.21.15.60:0/1208287043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f80035b0 con 0x7fb9f8004d90 2023-07-13T22:24:16.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.371+0000 7fba12d9c700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9f8004d90 0x7fb9f8005200 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fba04009fc0 tx=0x7fb9f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61de1991714a438b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:16.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.371+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0400ebd0 con 0x7fb9f8004d90 2023-07-13T22:24:16.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.371+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0400f040 con 0x7fb9f8004d90 2023-07-13T22:24:16.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.371+0000 7fba11599700 1 -- 172.21.15.60:0/1208287043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04014480 con 0x7fb9f8004d90 2023-07-13T22:24:16.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.372+0000 7fba147ff700 1 -- 172.21.15.60:0/1208287043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9f8004d90 msgr2=0x7fb9f8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.372+0000 7fba147ff700 1 --2- 172.21.15.60:0/1208287043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9f8004d90 0x7fb9f8005200 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fba04009fc0 tx=0x7fb9f8005880 comp rx=0 tx=0).stop 2023-07-13T22:24:16.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.372+0000 7fba12d9c700 1 -- 172.21.15.60:0/1208287043 reap_dead start 2023-07-13T22:24:16.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.372+0000 7fba147ff700 1 -- 172.21.15.60:0/1208287043 shutdown_connections 2023-07-13T22:24:16.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.372+0000 7fba147ff700 1 -- 172.21.15.60:0/1208287043 >> 172.21.15.60:0/1208287043 conn(0x7fba0c071600 msgr2=0x7fba0c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:16.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.372+0000 7fba147ff700 1 -- 172.21.15.60:0/1208287043 shutdown_connections 2023-07-13T22:24:16.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.372+0000 7fba147ff700 1 -- 172.21.15.60:0/1208287043 wait complete. 2023-07-13T22:24:16.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.374+0000 7fba147ff700 1 Processor -- start 2023-07-13T22:24:16.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.374+0000 7fba147ff700 1 -- start start 2023-07-13T22:24:16.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba147ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9f8004d90 0x7fba0c155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba147ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba0c0c9610 0x7fba0c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba147ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba0c1f1e40 0x7fba0c1f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba147ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c158b60 con 0x7fba0c0c9610 2023-07-13T22:24:16.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba147ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1f2820 con 0x7fba0c1f1e40 2023-07-13T22:24:16.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba147ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c157ee0 con 0x7fb9f8004d90 2023-07-13T22:24:16.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba11d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba0c0c9610 0x7fba0c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba11d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba0c0c9610 0x7fba0c1558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57356/0 (socket says 172.21.15.60:57356) 2023-07-13T22:24:16.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba11d9a700 1 -- 172.21.15.60:0/4120781306 learned_addr learned my addr 172.21.15.60:0/4120781306 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:16.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba12d9c700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba0c1f1e40 0x7fba0c1f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba1259b700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9f8004d90 0x7fba0c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba11d9a700 1 -- 172.21.15.60:0/4120781306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9f8004d90 msgr2=0x7fba0c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba11d9a700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9f8004d90 0x7fba0c155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba11d9a700 1 -- 172.21.15.60:0/4120781306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba0c1f1e40 msgr2=0x7fba0c1f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba11d9a700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba0c1f1e40 0x7fba0c1f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.375+0000 7fba11d9a700 1 -- 172.21.15.60:0/4120781306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c1581c0 con 0x7fba0c0c9610 2023-07-13T22:24:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.376+0000 7fba11d9a700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba0c0c9610 0x7fba0c1558d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc011aa0 tx=0x7fb9fc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:16.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.376+0000 7fba02ffd700 1 -- 172.21.15.60:0/4120781306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00f980 con 0x7fba0c0c9610 2023-07-13T22:24:16.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.376+0000 7fba02ffd700 1 -- 172.21.15.60:0/4120781306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00fae0 con 0x7fba0c0c9610 2023-07-13T22:24:16.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.376+0000 7fba02ffd700 1 -- 172.21.15.60:0/4120781306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00fe90 con 0x7fba0c0c9610 2023-07-13T22:24:16.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.376+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0c1584a0 con 0x7fba0c0c9610 2023-07-13T22:24:16.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.376+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0c155660 con 0x7fba0c0c9610 2023-07-13T22:24:16.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.377+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0c0cad80 con 0x7fba0c0c9610 2023-07-13T22:24:16.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.378+0000 7fba02ffd700 1 -- 172.21.15.60:0/4120781306 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb9fc014440 con 0x7fba0c0c9610 2023-07-13T22:24:16.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.379+0000 7fba02ffd700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb9f006ad10 0x7fb9f006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.379+0000 7fba02ffd700 1 -- 172.21.15.60:0/4120781306 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb9fc089980 con 0x7fba0c0c9610 2023-07-13T22:24:16.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.379+0000 7fba1259b700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb9f006ad10 0x7fb9f006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.380+0000 7fba1259b700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb9f006ad10 0x7fb9f006d1c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fba080011d0 tx=0x7fba08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:16.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.383+0000 7fba02ffd700 1 -- 172.21.15.60:0/4120781306 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fba0c0cad80 con 0x7fba0c0c9610 2023-07-13T22:24:16.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.588+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba0c0cad80 con 0x7fba0c0c9610 2023-07-13T22:24:16.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.589+0000 7fba02ffd700 1 -- 172.21.15.60:0/4120781306 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+14545 (secure 0 0 0) 0x7fba0c0cad80 con 0x7fba0c0c9610 2023-07-13T22:24:16.572 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:16.572 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":58,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:15.610499+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:16.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.591+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb9f006ad10 msgr2=0x7fb9f006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.591+0000 7fba147ff700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb9f006ad10 0x7fb9f006d1c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fba080011d0 tx=0x7fba08009040 comp rx=0 tx=0).stop 2023-07-13T22:24:16.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba0c0c9610 msgr2=0x7fba0c1558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba0c0c9610 0x7fba0c1558d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc011aa0 tx=0x7fb9fc011e60 comp rx=0 tx=0).stop 2023-07-13T22:24:16.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 shutdown_connections 2023-07-13T22:24:16.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb9f006ad10 0x7fb9f006d1c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba0c1f1e40 0x7fba0c1f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9f8004d90 0x7fba0c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 --2- 172.21.15.60:0/4120781306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba0c0c9610 0x7fba0c1558d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 >> 172.21.15.60:0/4120781306 conn(0x7fba0c071600 msgr2=0x7fba0c15fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:16.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 shutdown_connections 2023-07-13T22:24:16.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.592+0000 7fba147ff700 1 -- 172.21.15.60:0/4120781306 wait complete. 2023-07-13T22:24:16.595 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:24:16.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:24:16.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.819+0000 7f9b045ff700 1 Processor -- start 2023-07-13T22:24:16.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.819+0000 7f9b045ff700 1 -- start start 2023-07-13T22:24:16.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.820+0000 7f9b045ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0cfd50 0x7f9afc0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.820+0000 7f9b045ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc0d1870 con 0x7f9afc0b1ac0 2023-07-13T22:24:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.820+0000 7f9b045ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc0cf370 con 0x7f9afc0b8200 2023-07-13T22:24:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.820+0000 7f9b045ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0d12b0 con 0x7f9afc0cfd50 2023-07-13T22:24:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.820+0000 7f9b02b9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9afc0b1ac0 0x7f9afc0d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34462/0 (socket says 172.21.15.60:34462) 2023-07-13T22:24:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.820+0000 7f9b02b9c700 1 -- 172.21.15.60:0/2051003766 learned_addr learned my addr 172.21.15.60:0/2051003766 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01b9a700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0cfd50 0x7f9afc0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127696130 0 0) 0x7f9afc0d1870 con 0x7f9afc0b1ac0 2023-07-13T22:24:16.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae80035b0 con 0x7f9afc0b1ac0 2023-07-13T22:24:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 849022313 0 0) 0x7f9ae80035b0 con 0x7f9afc0b1ac0 2023-07-13T22:24:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0cfd50 msgr2=0x7f9afc0d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0cfd50 0x7f9afc0d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 >> v1:172.21.15.105:6789/0 conn(0x7f9afc0b8200 legacy=0x7f9afc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc0d1590 con 0x7f9afc0b1ac0 2023-07-13T22:24:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f9af40019f0 con 0x7f9afc0b1ac0 2023-07-13T22:24:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 >> v1:172.21.15.60:6789/0 conn(0x7f9afc0b1ac0 legacy=0x7f9afc0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0cfd50 0x7f9ae8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae8003f70 0x7f9ae8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9ae8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0d12b0 con 0x7f9ae8004d90 2023-07-13T22:24:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae800ba40 con 0x7f9ae8003f70 2023-07-13T22:24:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.821+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae800bc00 con 0x7f9afc0cfd50 2023-07-13T22:24:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b02b9c700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9ae8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b01b9a700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae8003f70 0x7f9ae8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b0239b700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0cfd50 0x7f9ae8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b02b9c700 1 -- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0cfd50 msgr2=0x7f9ae8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b02b9c700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0cfd50 0x7f9ae8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b02b9c700 1 -- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae8003f70 msgr2=0x7f9ae8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b02b9c700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae8003f70 0x7f9ae8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b02b9c700 1 -- 172.21.15.60:0/2051003766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80035b0 con 0x7f9ae8004d90 2023-07-13T22:24:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b02b9c700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9ae8005200 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0b1ee0 tx=0x7f9ae8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5564e0104dd29476 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40072d0 con 0x7f9ae8004d90 2023-07-13T22:24:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4007430 con 0x7f9ae8004d90 2023-07-13T22:24:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.822+0000 7f9b01399700 1 -- 172.21.15.60:0/2051003766 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40077e0 con 0x7f9ae8004d90 2023-07-13T22:24:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.823+0000 7f9b045ff700 1 -- 172.21.15.60:0/2051003766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 msgr2=0x7f9ae8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.823+0000 7f9b045ff700 1 --2- 172.21.15.60:0/2051003766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9ae8005200 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0b1ee0 tx=0x7f9ae8005880 comp rx=0 tx=0).stop 2023-07-13T22:24:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.823+0000 7f9b02b9c700 1 -- 172.21.15.60:0/2051003766 reap_dead start 2023-07-13T22:24:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.823+0000 7f9b045ff700 1 -- 172.21.15.60:0/2051003766 shutdown_connections 2023-07-13T22:24:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.823+0000 7f9b045ff700 1 -- 172.21.15.60:0/2051003766 >> 172.21.15.60:0/2051003766 conn(0x7f9afc06b2f0 msgr2=0x7f9afc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.823+0000 7f9b045ff700 1 -- 172.21.15.60:0/2051003766 shutdown_connections 2023-07-13T22:24:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.823+0000 7f9b045ff700 1 -- 172.21.15.60:0/2051003766 wait complete. 2023-07-13T22:24:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.825+0000 7f9b045ff700 1 Processor -- start 2023-07-13T22:24:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.826+0000 7f9b045ff700 1 -- start start 2023-07-13T22:24:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.827+0000 7f9b045ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9afc14ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.827+0000 7f9b045ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0ccbf0 0x7f9afc0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.827+0000 7f9b045ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0cb5e0 0x7f9afc0b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.827+0000 7f9b045ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0cbb50 con 0x7f9ae8004d90 2023-07-13T22:24:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.827+0000 7f9b045ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0cbe30 con 0x7f9afc0cb5e0 2023-07-13T22:24:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.827+0000 7f9b045ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0cc110 con 0x7f9afc0ccbf0 2023-07-13T22:24:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.827+0000 7f9b0239b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9afc14ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.827+0000 7f9b0239b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9afc14ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57374/0 (socket says 172.21.15.60:57374) 2023-07-13T22:24:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.827+0000 7f9b0239b700 1 -- 172.21.15.60:0/1832194447 learned_addr learned my addr 172.21.15.60:0/1832194447 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b02b9c700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0cb5e0 0x7f9afc0b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b01b9a700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0ccbf0 0x7f9afc0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b0239b700 1 -- 172.21.15.60:0/1832194447 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0ccbf0 msgr2=0x7f9afc0b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b0239b700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0ccbf0 0x7f9afc0b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b0239b700 1 -- 172.21.15.60:0/1832194447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0cb5e0 msgr2=0x7f9afc0b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b0239b700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0cb5e0 0x7f9afc0b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b0239b700 1 -- 172.21.15.60:0/1832194447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc0cc3f0 con 0x7f9ae8004d90 2023-07-13T22:24:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b0239b700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9afc14ceb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9af800f840 tx=0x7f9af800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9af2ffd700 1 -- 172.21.15.60:0/1832194447 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8005320 con 0x7f9ae8004d90 2023-07-13T22:24:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9af2ffd700 1 -- 172.21.15.60:0/1832194447 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80054a0 con 0x7f9ae8004d90 2023-07-13T22:24:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9af2ffd700 1 -- 172.21.15.60:0/1832194447 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8006aa0 con 0x7f9ae8004d90 2023-07-13T22:24:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc0cc6d0 con 0x7f9ae8004d90 2023-07-13T22:24:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.828+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc15e380 con 0x7f9ae8004d90 2023-07-13T22:24:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.829+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc0cc8c0 con 0x7f9ae8004d90 2023-07-13T22:24:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.830+0000 7f9af2ffd700 1 -- 172.21.15.60:0/1832194447 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9af8006c00 con 0x7f9ae8004d90 2023-07-13T22:24:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.831+0000 7f9af2ffd700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ae006ace0 0x7f9ae006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.831+0000 7f9af2ffd700 1 -- 172.21.15.60:0/1832194447 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9af80887c0 con 0x7f9ae8004d90 2023-07-13T22:24:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.832+0000 7f9b01b9a700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ae006ace0 0x7f9ae006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.832+0000 7f9b01b9a700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ae006ace0 0x7f9ae006d190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0cd020 tx=0x7f9aec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:16.835+0000 7f9af2ffd700 1 -- 172.21.15.60:0/1832194447 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9af8053b20 con 0x7f9ae8004d90 2023-07-13T22:24:17.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.042+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9afc04e210 con 0x7f9ae8004d90 2023-07-13T22:24:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.043+0000 7f9af2ffd700 1 -- 172.21.15.60:0/1832194447 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v58) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9af8057140 con 0x7f9ae8004d90 2023-07-13T22:24:17.025 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:24:17.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ae006ace0 msgr2=0x7f9ae006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ae006ace0 0x7f9ae006d190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0cd020 tx=0x7f9aec008040 comp rx=0 tx=0).stop 2023-07-13T22:24:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 msgr2=0x7f9afc14ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9afc14ceb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9af800f840 tx=0x7f9af800fc00 comp rx=0 tx=0).stop 2023-07-13T22:24:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 shutdown_connections 2023-07-13T22:24:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ae006ace0 0x7f9ae006d190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8004d90 0x7f9afc14ceb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9afc0cb5e0 0x7f9afc0b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 --2- 172.21.15.60:0/1832194447 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9afc0ccbf0 0x7f9afc0b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 >> 172.21.15.60:0/1832194447 conn(0x7f9afc06b2f0 msgr2=0x7f9afc0b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:17.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 shutdown_connections 2023-07-13T22:24:17.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.045+0000 7f9b045ff700 1 -- 172.21.15.60:0/1832194447 wait complete. 2023-07-13T22:24:17.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:24:17.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.269+0000 7f100f1ff700 1 Processor -- start 2023-07-13T22:24:17.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.270+0000 7f100f1ff700 1 -- start start 2023-07-13T22:24:17.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.271+0000 7f100f1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f10080b1ac0 0x7f10080d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.271+0000 7f100f1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10080d1870 con 0x7f10080cfd50 2023-07-13T22:24:17.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.271+0000 7f100f1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10080cf370 con 0x7f10080b8200 2023-07-13T22:24:17.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.271+0000 7f100f1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10080d12b0 con 0x7f10080b1ac0 2023-07-13T22:24:17.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.271+0000 7f1007fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f10080cfd50 0x7f10080d23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34470/0 (socket says 172.21.15.60:34470) 2023-07-13T22:24:17.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.271+0000 7f1007fff700 1 -- 172.21.15.60:0/2370085382 learned_addr learned my addr 172.21.15.60:0/2370085382 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:17.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.271+0000 7f100d79c700 1 --2- 172.21.15.60:0/2370085382 >> v2:172.21.15.60:3300/0 conn(0x7f10080b1ac0 0x7f10080d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.272+0000 7f100d79c700 1 -- 172.21.15.60:0/2370085382 >> v1:172.21.15.105:6789/0 conn(0x7f10080b8200 legacy=0x7f10080cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.272+0000 7f100d79c700 1 -- 172.21.15.60:0/2370085382 >> v1:172.21.15.60:6789/0 conn(0x7f10080cfd50 legacy=0x7f10080d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.272+0000 7f100d79c700 1 -- 172.21.15.60:0/2370085382 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10080d1590 con 0x7f10080b1ac0 2023-07-13T22:24:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.272+0000 7f10077fe700 1 -- 172.21.15.60:0/2370085382 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348213742 0 0) 0x7f10080d1870 con 0x7f10080cfd50 2023-07-13T22:24:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.272+0000 7f100d79c700 1 --2- 172.21.15.60:0/2370085382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 0x7f10080d2ad0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1000009d90 tx=0x7f100000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31fb3a7e01b596e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.272+0000 7f10077fe700 1 -- 172.21.15.60:0/2370085382 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000e070 con 0x7f10080b1ac0 2023-07-13T22:24:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.272+0000 7f10077fe700 1 -- 172.21.15.60:0/2370085382 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100000bb90 con 0x7f10080b1ac0 2023-07-13T22:24:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.272+0000 7f10077fe700 1 -- 172.21.15.60:0/2370085382 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000003a20 con 0x7f10080b1ac0 2023-07-13T22:24:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.273+0000 7f100f1ff700 1 -- 172.21.15.60:0/2370085382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 msgr2=0x7f10080d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.273+0000 7f100f1ff700 1 --2- 172.21.15.60:0/2370085382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 0x7f10080d2ad0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1000009d90 tx=0x7f100000b330 comp rx=0 tx=0).stop 2023-07-13T22:24:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.273+0000 7f100f1ff700 1 -- 172.21.15.60:0/2370085382 shutdown_connections 2023-07-13T22:24:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.273+0000 7f100f1ff700 1 --2- 172.21.15.60:0/2370085382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 0x7f10080d2ad0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.273+0000 7f100f1ff700 1 -- 172.21.15.60:0/2370085382 >> 172.21.15.60:0/2370085382 conn(0x7f100806b2f0 msgr2=0x7f100806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:17.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.273+0000 7f100f1ff700 1 -- 172.21.15.60:0/2370085382 shutdown_connections 2023-07-13T22:24:17.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.273+0000 7f100f1ff700 1 -- 172.21.15.60:0/2370085382 wait complete. 2023-07-13T22:24:17.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.274+0000 7f100f1ff700 1 Processor -- start 2023-07-13T22:24:17.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.275+0000 7f100f1ff700 1 -- start start 2023-07-13T22:24:17.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.275+0000 7f100f1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 0x7f10081f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.275+0000 7f100f1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10080b8200 0x7f1008149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.275+0000 7f100f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10080cfd50 0x7f1008149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.275+0000 7f100f1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1008147af0 con 0x7f10080b1ac0 2023-07-13T22:24:17.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.275+0000 7f100f1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1008147200 con 0x7f10080b8200 2023-07-13T22:24:17.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.275+0000 7f100f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10080c8ff0 con 0x7f10080cfd50 2023-07-13T22:24:17.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f100cf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 0x7f10081f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f100cf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 0x7f10081f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57394/0 (socket says 172.21.15.60:57394) 2023-07-13T22:24:17.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f100cf9b700 1 -- 172.21.15.60:0/3068823798 learned_addr learned my addr 172.21.15.60:0/3068823798 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:17.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f1007fff700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10080b8200 0x7f1008149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f100d79c700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10080cfd50 0x7f1008149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f100cf9b700 1 -- 172.21.15.60:0/3068823798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10080cfd50 msgr2=0x7f1008149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f100cf9b700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10080cfd50 0x7f1008149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f100cf9b700 1 -- 172.21.15.60:0/3068823798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10080b8200 msgr2=0x7f1008149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f100cf9b700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10080b8200 0x7f1008149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.276+0000 7f100cf9b700 1 -- 172.21.15.60:0/3068823798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000009a70 con 0x7f10080b1ac0 2023-07-13T22:24:17.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.277+0000 7f100cf9b700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 0x7f10081f51d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c4d0 tx=0x7f0ffc00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:17.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.277+0000 7f10057fa700 1 -- 172.21.15.60:0/3068823798 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc011790 con 0x7f10080b1ac0 2023-07-13T22:24:17.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.277+0000 7f10057fa700 1 -- 172.21.15.60:0/3068823798 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc0118f0 con 0x7f10080b1ac0 2023-07-13T22:24:17.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.277+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10081474a0 con 0x7f10080b1ac0 2023-07-13T22:24:17.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.277+0000 7f10057fa700 1 -- 172.21.15.60:0/3068823798 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc011ca0 con 0x7f10080b1ac0 2023-07-13T22:24:17.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.277+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10080ca210 con 0x7f10080b1ac0 2023-07-13T22:24:17.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.278+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100804e210 con 0x7f10080b1ac0 2023-07-13T22:24:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.279+0000 7f10057fa700 1 -- 172.21.15.60:0/3068823798 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ffc01c460 con 0x7f10080b1ac0 2023-07-13T22:24:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.280+0000 7f10057fa700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0fec06ace0 0x7f0fec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.280+0000 7f10057fa700 1 -- 172.21.15.60:0/3068823798 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0ffc088e90 con 0x7f10080b1ac0 2023-07-13T22:24:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.281+0000 7f1007fff700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0fec06ace0 0x7f0fec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.281+0000 7f1007fff700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0fec06ace0 0x7f0fec06d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10080c94d0 tx=0x7f0ff4007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:17.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.284+0000 7f10057fa700 1 -- 172.21.15.60:0/3068823798 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0ffc0541f0 con 0x7f10080b1ac0 2023-07-13T22:24:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.491+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10080d38b0 con 0x7f10080b1ac0 2023-07-13T22:24:17.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.491+0000 7f10057fa700 1 -- 172.21.15.60:0/3068823798 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ffc08a070 con 0x7f10080b1ac0 2023-07-13T22:24:17.474 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:24:17.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0fec06ace0 msgr2=0x7f0fec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0fec06ace0 0x7f0fec06d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10080c94d0 tx=0x7f0ff4007e70 comp rx=0 tx=0).stop 2023-07-13T22:24:17.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 msgr2=0x7f10081f51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 0x7f10081f51d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c4d0 tx=0x7f0ffc00c7e0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 shutdown_connections 2023-07-13T22:24:17.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0fec06ace0 0x7f0fec06d190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10080b1ac0 0x7f10081f51d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10080cfd50 0x7f1008149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 --2- 172.21.15.60:0/3068823798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10080b8200 0x7f1008149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 >> 172.21.15.60:0/3068823798 conn(0x7f100806b2f0 msgr2=0x7f100806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:17.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 shutdown_connections 2023-07-13T22:24:17.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.494+0000 7f100f1ff700 1 -- 172.21.15.60:0/3068823798 wait complete. 2023-07-13T22:24:17.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:24:17.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.717+0000 7fc75d5ff700 1 Processor -- start 2023-07-13T22:24:17.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.717+0000 7fc75d5ff700 1 -- start start 2023-07-13T22:24:17.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc75d5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc758154950 0x7fc758070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc75d5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc7580c9450 0x7fc75815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc75d5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc758070270 con 0x7fc75815a820 2023-07-13T22:24:17.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc75d5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75815b2e0 con 0x7fc7580c9450 2023-07-13T22:24:17.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc75d5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75815b5c0 con 0x7fc758154950 2023-07-13T22:24:17.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc74ffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc75815a820 0x7fc7580c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34476/0 (socket says 172.21.15.60:34476) 2023-07-13T22:24:17.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc74ffff700 1 -- 172.21.15.60:0/267690809 learned_addr learned my addr 172.21.15.60:0/267690809 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:17.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc756ffd700 1 --2- 172.21.15.60:0/267690809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc758154950 0x7fc758070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc7577fe700 1 --2- 172.21.15.60:0/267690809 >> v2:172.21.15.105:3300/0 conn(0x7fc7580c9450 0x7fc75815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.719+0000 7fc7567fc700 1 -- 172.21.15.60:0/267690809 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920002413 0 0) 0x7fc758070270 con 0x7fc75815a820 2023-07-13T22:24:17.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.720+0000 7fc7567fc700 1 -- 172.21.15.60:0/267690809 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7440035b0 con 0x7fc75815a820 2023-07-13T22:24:17.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.720+0000 7fc756ffd700 1 -- 172.21.15.60:0/267690809 >> v2:172.21.15.105:3300/0 conn(0x7fc7580c9450 msgr2=0x7fc75815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:17.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.720+0000 7fc756ffd700 1 --2- 172.21.15.60:0/267690809 >> v2:172.21.15.105:3300/0 conn(0x7fc7580c9450 0x7fc75815ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.720+0000 7fc756ffd700 1 -- 172.21.15.60:0/267690809 >> v1:172.21.15.60:6789/0 conn(0x7fc75815a820 legacy=0x7fc7580c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.720+0000 7fc756ffd700 1 -- 172.21.15.60:0/267690809 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7580704c0 con 0x7fc758154950 2023-07-13T22:24:17.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.720+0000 7fc756ffd700 1 --2- 172.21.15.60:0/267690809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc758154950 0x7fc758070cc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc748009840 tx=0x7fc748009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e026a35c934a6d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:17.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.720+0000 7fc7567fc700 1 -- 172.21.15.60:0/267690809 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748011830 con 0x7fc758154950 2023-07-13T22:24:17.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.720+0000 7fc7567fc700 1 -- 172.21.15.60:0/267690809 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748011990 con 0x7fc758154950 2023-07-13T22:24:17.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.720+0000 7fc7567fc700 1 -- 172.21.15.60:0/267690809 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748011d20 con 0x7fc758154950 2023-07-13T22:24:17.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.721+0000 7fc75d5ff700 1 -- 172.21.15.60:0/267690809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc758154950 msgr2=0x7fc758070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.721+0000 7fc75d5ff700 1 --2- 172.21.15.60:0/267690809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc758154950 0x7fc758070cc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc748009840 tx=0x7fc748009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:17.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.721+0000 7fc75d5ff700 1 -- 172.21.15.60:0/267690809 shutdown_connections 2023-07-13T22:24:17.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.721+0000 7fc75d5ff700 1 --2- 172.21.15.60:0/267690809 >> v2:172.21.15.105:3300/0 conn(0x7fc7580c9450 0x7fc75815ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.721+0000 7fc75d5ff700 1 --2- 172.21.15.60:0/267690809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc758154950 0x7fc758070cc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.721+0000 7fc75d5ff700 1 -- 172.21.15.60:0/267690809 >> 172.21.15.60:0/267690809 conn(0x7fc7580cf410 msgr2=0x7fc7580cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:17.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.721+0000 7fc75d5ff700 1 -- 172.21.15.60:0/267690809 shutdown_connections 2023-07-13T22:24:17.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.721+0000 7fc75d5ff700 1 -- 172.21.15.60:0/267690809 wait complete. 2023-07-13T22:24:17.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.723+0000 7fc75d5ff700 1 Processor -- start 2023-07-13T22:24:17.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.723+0000 7fc75d5ff700 1 -- start start 2023-07-13T22:24:17.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc75d5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7580c9450 0x7fc7580af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc75d5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc758154950 0x7fc7580cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc75d5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc75815a820 0x7fc75815bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc75d5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580afaa0 con 0x7fc7580c9450 2023-07-13T22:24:17.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc75d5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580afd80 con 0x7fc758154950 2023-07-13T22:24:17.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc75d5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580b0060 con 0x7fc75815a820 2023-07-13T22:24:17.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc756ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7580c9450 0x7fc7580af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc756ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7580c9450 0x7fc7580af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57402/0 (socket says 172.21.15.60:57402) 2023-07-13T22:24:17.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc756ffd700 1 -- 172.21.15.60:0/868913961 learned_addr learned my addr 172.21.15.60:0/868913961 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:17.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc756ffd700 1 -- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc75815a820 msgr2=0x7fc75815bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc7577fe700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc75815a820 0x7fc75815bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc74ffff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc758154950 0x7fc7580cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc756ffd700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc75815a820 0x7fc75815bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc756ffd700 1 -- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc758154950 msgr2=0x7fc7580cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.724+0000 7fc756ffd700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc758154950 0x7fc7580cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.725+0000 7fc756ffd700 1 -- 172.21.15.60:0/868913961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480094a0 con 0x7fc7580c9450 2023-07-13T22:24:17.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.725+0000 7fc7577fe700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc75815a820 0x7fc75815bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:17.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.725+0000 7fc74ffff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc758154950 0x7fc7580cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:17.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.725+0000 7fc756ffd700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7580c9450 0x7fc7580af4c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc7480090b0 tx=0x7fc748004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:17.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.725+0000 7fc74f7fe700 1 -- 172.21.15.60:0/868913961 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480036a0 con 0x7fc7580c9450 2023-07-13T22:24:17.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.725+0000 7fc74f7fe700 1 -- 172.21.15.60:0/868913961 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74801f070 con 0x7fc7580c9450 2023-07-13T22:24:17.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.725+0000 7fc74f7fe700 1 -- 172.21.15.60:0/868913961 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748003a40 con 0x7fc7580c9450 2023-07-13T22:24:17.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.725+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7580b0340 con 0x7fc7580c9450 2023-07-13T22:24:17.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.725+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75815c330 con 0x7fc7580c9450 2023-07-13T22:24:17.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.726+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7580b0500 con 0x7fc7580c9450 2023-07-13T22:24:17.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.727+0000 7fc74f7fe700 1 -- 172.21.15.60:0/868913961 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc748003ba0 con 0x7fc7580c9450 2023-07-13T22:24:17.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.728+0000 7fc74f7fe700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc73806ace0 0x7fc73806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:17.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.728+0000 7fc74f7fe700 1 -- 172.21.15.60:0/868913961 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc748021070 con 0x7fc7580c9450 2023-07-13T22:24:17.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.728+0000 7fc74ffff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc73806ace0 0x7fc73806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:17.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.729+0000 7fc74ffff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc73806ace0 0x7fc73806d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc75814bf40 tx=0x7fc740009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:17.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.732+0000 7fc74f7fe700 1 -- 172.21.15.60:0/868913961 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc748055dd0 con 0x7fc7580c9450 2023-07-13T22:24:17.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.943+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc75804e280 con 0x7fc7580c9450 2023-07-13T22:24:17.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.943+0000 7fc74f7fe700 1 -- 172.21.15.60:0/868913961 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc7480593f0 con 0x7fc7580c9450 2023-07-13T22:24:17.925 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:17.925 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:24:17.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:17.926 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:24:17.926 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:24:17.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:17.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:17.926 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:24:17.926 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:24:17.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:17.927 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:24:17.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.945+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc73806ace0 msgr2=0x7fc73806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.945+0000 7fc75d5ff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc73806ace0 0x7fc73806d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc75814bf40 tx=0x7fc740009040 comp rx=0 tx=0).stop 2023-07-13T22:24:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.945+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7580c9450 msgr2=0x7fc7580af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.945+0000 7fc75d5ff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7580c9450 0x7fc7580af4c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc7480090b0 tx=0x7fc748004750 comp rx=0 tx=0).stop 2023-07-13T22:24:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.946+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 shutdown_connections 2023-07-13T22:24:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.946+0000 7fc75d5ff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc73806ace0 0x7fc73806d190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.946+0000 7fc75d5ff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7580c9450 0x7fc7580af4c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.946+0000 7fc75d5ff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc75815a820 0x7fc75815bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.946+0000 7fc75d5ff700 1 --2- 172.21.15.60:0/868913961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc758154950 0x7fc7580cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.946+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 >> 172.21.15.60:0/868913961 conn(0x7fc7580cf410 msgr2=0x7fc7580c8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.946+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 shutdown_connections 2023-07-13T22:24:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:17.946+0000 7fc75d5ff700 1 -- 172.21.15.60:0/868913961 wait complete. 2023-07-13T22:24:17.946 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:24:17.947 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:24:18.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.177+0000 7f1e07fff700 1 Processor -- start 2023-07-13T22:24:18.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.177+0000 7f1e07fff700 1 -- start start 2023-07-13T22:24:18.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.178+0000 7f1e07fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 0x7f1e000cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e07fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e000cd890 con 0x7f1e000cfc00 2023-07-13T22:24:18.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e07fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e000b8640 con 0x7f1e000b8270 2023-07-13T22:24:18.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e07fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000cf380 con 0x7f1e000b1f50 2023-07-13T22:24:18.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e0559a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1e000cfc00 0x7f1e000d25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34488/0 (socket says 172.21.15.60:34488) 2023-07-13T22:24:18.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e0559a700 1 -- 172.21.15.60:0/3827797181 learned_addr learned my addr 172.21.15.60:0/3827797181 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:18.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e0659c700 1 --2- 172.21.15.60:0/3827797181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 0x7f1e000cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e04d99700 1 -- 172.21.15.60:0/3827797181 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208578260 0 0) 0x7f1e000cd890 con 0x7f1e000cfc00 2023-07-13T22:24:18.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e04d99700 1 -- 172.21.15.60:0/3827797181 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dec0035b0 con 0x7f1e000cfc00 2023-07-13T22:24:18.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e04d99700 1 -- 172.21.15.60:0/3827797181 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817743173 0 0) 0x7f1e000b8640 con 0x7f1e000b8270 2023-07-13T22:24:18.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e0659c700 1 -- 172.21.15.60:0/3827797181 >> v1:172.21.15.105:6789/0 conn(0x7f1e000b8270 legacy=0x7f1e000cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e0659c700 1 -- 172.21.15.60:0/3827797181 >> v1:172.21.15.60:6789/0 conn(0x7f1e000cfc00 legacy=0x7f1e000d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.179+0000 7f1e0659c700 1 -- 172.21.15.60:0/3827797181 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e000d1290 con 0x7f1e000b1f50 2023-07-13T22:24:18.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.180+0000 7f1e0659c700 1 --2- 172.21.15.60:0/3827797181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 0x7f1e000cecc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009dc0 tx=0x7f1df800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5dc19cc32d6e8f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:18.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.180+0000 7f1e04d99700 1 -- 172.21.15.60:0/3827797181 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df800e070 con 0x7f1e000b1f50 2023-07-13T22:24:18.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.180+0000 7f1e04d99700 1 -- 172.21.15.60:0/3827797181 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df800bb20 con 0x7f1e000b1f50 2023-07-13T22:24:18.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.180+0000 7f1e04d99700 1 -- 172.21.15.60:0/3827797181 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df800bed0 con 0x7f1e000b1f50 2023-07-13T22:24:18.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.181+0000 7f1e07fff700 1 -- 172.21.15.60:0/3827797181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 msgr2=0x7f1e000cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.181+0000 7f1e07fff700 1 --2- 172.21.15.60:0/3827797181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 0x7f1e000cecc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009dc0 tx=0x7f1df800b330 comp rx=0 tx=0).stop 2023-07-13T22:24:18.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.181+0000 7f1e07fff700 1 -- 172.21.15.60:0/3827797181 shutdown_connections 2023-07-13T22:24:18.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.181+0000 7f1e07fff700 1 --2- 172.21.15.60:0/3827797181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 0x7f1e000cecc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.181+0000 7f1e07fff700 1 -- 172.21.15.60:0/3827797181 >> 172.21.15.60:0/3827797181 conn(0x7f1e0006b330 msgr2=0x7f1e0006b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:18.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.181+0000 7f1e07fff700 1 -- 172.21.15.60:0/3827797181 shutdown_connections 2023-07-13T22:24:18.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.181+0000 7f1e07fff700 1 -- 172.21.15.60:0/3827797181 wait complete. 2023-07-13T22:24:18.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.182+0000 7f1e07fff700 1 Processor -- start 2023-07-13T22:24:18.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.183+0000 7f1e07fff700 1 -- start start 2023-07-13T22:24:18.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.183+0000 7f1e07fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 0x7f1e0015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e07fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e000b8270 0x7f1e0015e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e07fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e000cfc00 0x7f1e0015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e07fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e001f2780 con 0x7f1e000b8270 2023-07-13T22:24:18.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e07fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e001f2a60 con 0x7f1e000cfc00 2023-07-13T22:24:18.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e07fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000c9d60 con 0x7f1e000b1f50 2023-07-13T22:24:18.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e0559a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e000b8270 0x7f1e0015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e0559a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e000b8270 0x7f1e0015e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57406/0 (socket says 172.21.15.60:57406) 2023-07-13T22:24:18.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e0559a700 1 -- 172.21.15.60:0/4203715712 learned_addr learned my addr 172.21.15.60:0/4203715712 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:18.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e05d9b700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 0x7f1e0015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e0659c700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e000cfc00 0x7f1e0015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e0559a700 1 -- 172.21.15.60:0/4203715712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 msgr2=0x7f1e0015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e0559a700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 0x7f1e0015c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e0559a700 1 -- 172.21.15.60:0/4203715712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e000cfc00 msgr2=0x7f1e0015efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e0559a700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e000cfc00 0x7f1e0015efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.184+0000 7f1e0559a700 1 -- 172.21.15.60:0/4203715712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009a70 con 0x7f1e000b8270 2023-07-13T22:24:18.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.185+0000 7f1e0559a700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e000b8270 0x7f1e0015e640 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1df000c400 tx=0x7f1df000c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:18.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.185+0000 7f1df67fc700 1 -- 172.21.15.60:0/4203715712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0005d70 con 0x7f1e000b8270 2023-07-13T22:24:18.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.185+0000 7f1df67fc700 1 -- 172.21.15.60:0/4203715712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df000ce10 con 0x7f1e000b8270 2023-07-13T22:24:18.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.185+0000 7f1df67fc700 1 -- 172.21.15.60:0/4203715712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0007310 con 0x7f1e000b8270 2023-07-13T22:24:18.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.185+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e000ca040 con 0x7f1e000b8270 2023-07-13T22:24:18.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.185+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0015f520 con 0x7f1e000b8270 2023-07-13T22:24:18.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.186+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e000c9bc0 con 0x7f1e000b8270 2023-07-13T22:24:18.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.187+0000 7f1df67fc700 1 -- 172.21.15.60:0/4203715712 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1df0007470 con 0x7f1e000b8270 2023-07-13T22:24:18.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.188+0000 7f1df67fc700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1de406ace0 0x7f1de406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.188+0000 7f1df67fc700 1 -- 172.21.15.60:0/4203715712 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1df00884e0 con 0x7f1e000b8270 2023-07-13T22:24:18.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.188+0000 7f1e05d9b700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1de406ace0 0x7f1de406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.189+0000 7f1e05d9b700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1de406ace0 0x7f1de406d190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007fd0 tx=0x7f1dfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:18.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.193+0000 7f1df67fc700 1 -- 172.21.15.60:0/4203715712 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1e000c9bc0 con 0x7f1e000b8270 2023-07-13T22:24:18.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.401+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1e000c9bc0 con 0x7f1e000b8270 2023-07-13T22:24:18.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.612+0000 7f1df67fc700 1 -- 172.21.15.60:0/4203715712 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1e000c9bc0 con 0x7f1e000b8270 2023-07-13T22:24:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.615+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1de406ace0 msgr2=0x7f1de406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.615+0000 7f1e07fff700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1de406ace0 0x7f1de406d190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007fd0 tx=0x7f1dfc009040 comp rx=0 tx=0).stop 2023-07-13T22:24:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.615+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e000b8270 msgr2=0x7f1e0015e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.615+0000 7f1e07fff700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e000b8270 0x7f1e0015e640 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1df000c400 tx=0x7f1df000c710 comp rx=0 tx=0).stop 2023-07-13T22:24:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.615+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 shutdown_connections 2023-07-13T22:24:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.615+0000 7f1e07fff700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1de406ace0 0x7f1de406d190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.615+0000 7f1e07fff700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e000cfc00 0x7f1e0015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.615+0000 7f1e07fff700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e000b1f50 0x7f1e0015c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.615+0000 7f1e07fff700 1 --2- 172.21.15.60:0/4203715712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e000b8270 0x7f1e0015e640 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.616+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 >> 172.21.15.60:0/4203715712 conn(0x7f1e0006b330 msgr2=0x7f1e0015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.616+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 shutdown_connections 2023-07-13T22:24:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.616+0000 7f1e07fff700 1 -- 172.21.15.60:0/4203715712 wait complete. 2023-07-13T22:24:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:24:18.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:24:18.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.848+0000 7f8aca3ff700 1 Processor -- start 2023-07-13T22:24:18.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.848+0000 7f8aca3ff700 1 -- start start 2023-07-13T22:24:18.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.849+0000 7f8aca3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ac40cd6d0 0x7f8ac40d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.849+0000 7f8aca3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac40cdb20 con 0x7f8ac4159ea0 2023-07-13T22:24:18.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.849+0000 7f8aca3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac415d200 con 0x7f8ac40d37c0 2023-07-13T22:24:18.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.849+0000 7f8aca3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac40d2e60 con 0x7f8ac40cd6d0 2023-07-13T22:24:18.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.849+0000 7f8ac3fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8ac4159ea0 0x7f8ac415c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34496/0 (socket says 172.21.15.60:34496) 2023-07-13T22:24:18.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.849+0000 7f8ac3fff700 1 -- 172.21.15.60:0/2330365753 learned_addr learned my addr 172.21.15.60:0/2330365753 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:18.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac37fe700 1 --2- 172.21.15.60:0/2330365753 >> v2:172.21.15.105:3300/0 conn(0x7f8ac40cd6d0 0x7f8ac40d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396872568 0 0) 0x7f8ac40cdb20 con 0x7f8ac4159ea0 2023-07-13T22:24:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aac0035b0 con 0x7f8ac4159ea0 2023-07-13T22:24:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1255999305 0 0) 0x7f8aac0035b0 con 0x7f8ac4159ea0 2023-07-13T22:24:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 >> v2:172.21.15.105:3300/0 conn(0x7f8ac40cd6d0 msgr2=0x7f8ac40d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 --2- 172.21.15.60:0/2330365753 >> v2:172.21.15.105:3300/0 conn(0x7f8ac40cd6d0 0x7f8ac40d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 >> v1:172.21.15.105:6789/0 conn(0x7f8ac40d37c0 legacy=0x7f8ac40d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac415d4e0 con 0x7f8ac4159ea0 2023-07-13T22:24:18.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f8ab4001670 con 0x7f8ac4159ea0 2023-07-13T22:24:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 >> v1:172.21.15.60:6789/0 conn(0x7f8ac4159ea0 legacy=0x7f8ac415c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8aac003a00 0x7f8aac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aac004850 0x7f8aac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8aac005200 0x7f8aac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40d2e60 con 0x7f8aac004850 2023-07-13T22:24:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac00c080 con 0x7f8aac003a00 2023-07-13T22:24:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.850+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac00c240 con 0x7f8aac005200 2023-07-13T22:24:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac37fe700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aac004850 0x7f8aac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac3fff700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8aac003a00 0x7f8aac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac899c700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8aac005200 0x7f8aac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac37fe700 1 -- 172.21.15.60:0/2330365753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8aac005200 msgr2=0x7f8aac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac37fe700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8aac005200 0x7f8aac00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac37fe700 1 -- 172.21.15.60:0/2330365753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8aac003a00 msgr2=0x7f8aac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac37fe700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8aac003a00 0x7f8aac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac37fe700 1 -- 172.21.15.60:0/2330365753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0035b0 con 0x7f8aac004850 2023-07-13T22:24:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac899c700 1 -- 172.21.15.60:0/2330365753 reap_dead start 2023-07-13T22:24:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac3fff700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8aac003a00 0x7f8aac003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.851+0000 7f8ac37fe700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aac004850 0x7f8aac004cc0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009b60 tx=0x7f8aac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c564b71598d047ae server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800e6c0 con 0x7f8aac004850 2023-07-13T22:24:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab800e820 con 0x7f8aac004850 2023-07-13T22:24:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8ac2ffd700 1 -- 172.21.15.60:0/2330365753 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800eb90 con 0x7f8aac004850 2023-07-13T22:24:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2330365753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aac004850 msgr2=0x7f8aac004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8aca3ff700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aac004850 0x7f8aac004cc0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009b60 tx=0x7f8aac005880 comp rx=0 tx=0).stop 2023-07-13T22:24:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2330365753 shutdown_connections 2023-07-13T22:24:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8aca3ff700 1 --2- 172.21.15.60:0/2330365753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aac004850 0x7f8aac004cc0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2330365753 >> 172.21.15.60:0/2330365753 conn(0x7f8ac4071600 msgr2=0x7f8ac40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2330365753 shutdown_connections 2023-07-13T22:24:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.852+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2330365753 wait complete. 2023-07-13T22:24:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.855+0000 7f8aca3ff700 1 Processor -- start 2023-07-13T22:24:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.855+0000 7f8aca3ff700 1 -- start start 2023-07-13T22:24:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.856+0000 7f8aca3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8aac004850 0x7f8ac4154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.856+0000 7f8aca3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ac40c7530 0x7f8ac40d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.856+0000 7f8aca3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac40c79a0 0x7f8ac40d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.856+0000 7f8aca3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac41f1950 con 0x7f8ac40c7530 2023-07-13T22:24:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.856+0000 7f8aca3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40c8300 con 0x7f8ac40c79a0 2023-07-13T22:24:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.856+0000 7f8aca3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40c85e0 con 0x7f8aac004850 2023-07-13T22:24:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ac40c7530 0x7f8ac40d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ac40c7530 0x7f8ac40d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57428/0 (socket says 172.21.15.60:57428) 2023-07-13T22:24:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac37fe700 1 -- 172.21.15.60:0/2922768751 learned_addr learned my addr 172.21.15.60:0/2922768751 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac3fff700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8aac004850 0x7f8ac4154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac899c700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac40c79a0 0x7f8ac40d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac37fe700 1 -- 172.21.15.60:0/2922768751 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8aac004850 msgr2=0x7f8ac4154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac37fe700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8aac004850 0x7f8ac4154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac37fe700 1 -- 172.21.15.60:0/2922768751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac40c79a0 msgr2=0x7f8ac40d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac37fe700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac40c79a0 0x7f8ac40d17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac37fe700 1 -- 172.21.15.60:0/2922768751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40c88c0 con 0x7f8ac40c7530 2023-07-13T22:24:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac899c700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac40c79a0 0x7f8ac40d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.857+0000 7f8ac37fe700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ac40c7530 0x7f8ac40d12a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c050 tx=0x7f8ab80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.858+0000 7f8ac0ff9700 1 -- 172.21.15.60:0/2922768751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800e6c0 con 0x7f8ac40c7530 2023-07-13T22:24:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.858+0000 7f8ac0ff9700 1 -- 172.21.15.60:0/2922768751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab8003ad0 con 0x7f8ac40c7530 2023-07-13T22:24:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.858+0000 7f8ac0ff9700 1 -- 172.21.15.60:0/2922768751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8003e80 con 0x7f8ac40c7530 2023-07-13T22:24:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.858+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac40b6b40 con 0x7f8ac40c7530 2023-07-13T22:24:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.858+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac40c7eb0 con 0x7f8ac40c7530 2023-07-13T22:24:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.859+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac40b7190 con 0x7f8ac40c7530 2023-07-13T22:24:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.860+0000 7f8ac0ff9700 1 -- 172.21.15.60:0/2922768751 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8ab8013470 con 0x7f8ac40c7530 2023-07-13T22:24:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.861+0000 7f8ac0ff9700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8ab006ace0 0x7f8ab006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.861+0000 7f8ac0ff9700 1 -- 172.21.15.60:0/2922768751 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f8ab808ac60 con 0x7f8ac40c7530 2023-07-13T22:24:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.861+0000 7f8ac3fff700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8ab006ace0 0x7f8ab006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.862+0000 7f8ac3fff700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8ab006ace0 0x7f8ab006d190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4007fd0 tx=0x7f8ab4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:18.866+0000 7f8ac0ff9700 1 -- 172.21.15.60:0/2922768751 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8ab8056000 con 0x7f8ac40c7530 2023-07-13T22:24:19.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.073+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8ac4067a70 con 0x7f8ac40c7530 2023-07-13T22:24:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.617+0000 7f8ac0ff9700 1 -- 172.21.15.60:0/2922768751 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8ab80170d0 con 0x7f8ac40c7530 2023-07-13T22:24:19.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8ab006ace0 msgr2=0x7f8ab006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:19.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8ab006ace0 0x7f8ab006d190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4007fd0 tx=0x7f8ab4009040 comp rx=0 tx=0).stop 2023-07-13T22:24:19.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ac40c7530 msgr2=0x7f8ac40d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:19.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ac40c7530 0x7f8ac40d12a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c050 tx=0x7f8ab80047c0 comp rx=0 tx=0).stop 2023-07-13T22:24:19.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 shutdown_connections 2023-07-13T22:24:19.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8ab006ace0 0x7f8ab006d190 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4007fd0 tx=0x7f8ab4009040 comp rx=0 tx=0).stop 2023-07-13T22:24:19.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ac40c79a0 0x7f8ac40d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:19.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8aac004850 0x7f8ac4154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:19.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 --2- 172.21.15.60:0/2922768751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ac40c7530 0x7f8ac40d12a0 secure :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c050 tx=0x7f8ab80047c0 comp rx=0 tx=0).stop 2023-07-13T22:24:19.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.623+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 >> 172.21.15.60:0/2922768751 conn(0x7f8ac4071600 msgr2=0x7f8ac40cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.629+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 shutdown_connections 2023-07-13T22:24:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.629+0000 7f8aca3ff700 1 -- 172.21.15.60:0/2922768751 wait complete. 2023-07-13T22:24:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:24:19.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:24:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.868+0000 7f158b7ff700 1 Processor -- start 2023-07-13T22:24:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.868+0000 7f158b7ff700 1 -- start start 2023-07-13T22:24:19.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.869+0000 7f158b7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15840b8200 0x7f15840cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:19.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.869+0000 7f158b7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840cede0 0x7f15840cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:19.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.869+0000 7f158b7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840d1870 con 0x7f15840b1ac0 2023-07-13T22:24:19.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.869+0000 7f158b7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15840cf370 con 0x7f15840b8200 2023-07-13T22:24:19.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.869+0000 7f158b7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15840d12b0 con 0x7f15840cede0 2023-07-13T22:24:19.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f1588d9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f15840b1ac0 0x7f158415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34512/0 (socket says 172.21.15.60:34512) 2023-07-13T22:24:19.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f1588d9a700 1 -- 172.21.15.60:0/1554813958 learned_addr learned my addr 172.21.15.60:0/1554813958 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:19.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f1589d9c700 1 --2- 172.21.15.60:0/1554813958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840cede0 0x7f15840cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f158959b700 1 --2- 172.21.15.60:0/1554813958 >> v2:172.21.15.105:3300/0 conn(0x7f15840b8200 0x7f15840cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f157bfff700 1 -- 172.21.15.60:0/1554813958 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52656015 0 0) 0x7f15840d1870 con 0x7f15840b1ac0 2023-07-13T22:24:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f157bfff700 1 -- 172.21.15.60:0/1554813958 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15700035b0 con 0x7f15840b1ac0 2023-07-13T22:24:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f1589d9c700 1 -- 172.21.15.60:0/1554813958 >> v2:172.21.15.105:3300/0 conn(0x7f15840b8200 msgr2=0x7f15840cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f1589d9c700 1 --2- 172.21.15.60:0/1554813958 >> v2:172.21.15.105:3300/0 conn(0x7f15840b8200 0x7f15840cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f1589d9c700 1 -- 172.21.15.60:0/1554813958 >> v1:172.21.15.60:6789/0 conn(0x7f15840b1ac0 legacy=0x7f158415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f1589d9c700 1 -- 172.21.15.60:0/1554813958 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840d1590 con 0x7f15840cede0 2023-07-13T22:24:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.870+0000 7f158959b700 1 --2- 172.21.15.60:0/1554813958 >> v2:172.21.15.105:3300/0 conn(0x7f15840b8200 0x7f15840cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.871+0000 7f1589d9c700 1 --2- 172.21.15.60:0/1554813958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840cede0 0x7f15840cfb80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f157c009d90 tx=0x7f157c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53bf6137244d969 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.871+0000 7f157bfff700 1 -- 172.21.15.60:0/1554813958 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00e070 con 0x7f15840cede0 2023-07-13T22:24:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.871+0000 7f157bfff700 1 -- 172.21.15.60:0/1554813958 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00bb20 con 0x7f15840cede0 2023-07-13T22:24:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.871+0000 7f157bfff700 1 -- 172.21.15.60:0/1554813958 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c012430 con 0x7f15840cede0 2023-07-13T22:24:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.872+0000 7f158b7ff700 1 -- 172.21.15.60:0/1554813958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840cede0 msgr2=0x7f15840cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.872+0000 7f158b7ff700 1 --2- 172.21.15.60:0/1554813958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840cede0 0x7f15840cfb80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f157c009d90 tx=0x7f157c00b330 comp rx=0 tx=0).stop 2023-07-13T22:24:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.872+0000 7f158b7ff700 1 -- 172.21.15.60:0/1554813958 shutdown_connections 2023-07-13T22:24:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.872+0000 7f158b7ff700 1 --2- 172.21.15.60:0/1554813958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840cede0 0x7f15840cfb80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.872+0000 7f158b7ff700 1 --2- 172.21.15.60:0/1554813958 >> v2:172.21.15.105:3300/0 conn(0x7f15840b8200 0x7f15840cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.872+0000 7f158b7ff700 1 -- 172.21.15.60:0/1554813958 >> 172.21.15.60:0/1554813958 conn(0x7f158406b2f0 msgr2=0x7f158406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.872+0000 7f158b7ff700 1 -- 172.21.15.60:0/1554813958 shutdown_connections 2023-07-13T22:24:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.872+0000 7f158b7ff700 1 -- 172.21.15.60:0/1554813958 wait complete. 2023-07-13T22:24:19.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.873+0000 7f158b7ff700 1 Processor -- start 2023-07-13T22:24:19.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.874+0000 7f158b7ff700 1 -- start start 2023-07-13T22:24:19.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.874+0000 7f158b7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840b1ac0 0x7f15841f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:19.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.874+0000 7f158b7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15840b8200 0x7f15841e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:19.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f158b7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15840cede0 0x7f15841598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:19.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f158b7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15841f1c90 con 0x7f15840cede0 2023-07-13T22:24:19.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f158b7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840cb180 con 0x7f15840b8200 2023-07-13T22:24:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f158b7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158415a320 con 0x7f15840b1ac0 2023-07-13T22:24:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f1589d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15840cede0 0x7f15841598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f1589d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15840cede0 0x7f15841598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57434/0 (socket says 172.21.15.60:57434) 2023-07-13T22:24:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f1589d9c700 1 -- 172.21.15.60:0/2526974549 learned_addr learned my addr 172.21.15.60:0/2526974549 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f1588d9a700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15840b8200 0x7f15841e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f158959b700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840b1ac0 0x7f15841f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f1589d9c700 1 -- 172.21.15.60:0/2526974549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840b1ac0 msgr2=0x7f15841f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f1589d9c700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840b1ac0 0x7f15841f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f1589d9c700 1 -- 172.21.15.60:0/2526974549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15840b8200 msgr2=0x7f15841e3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f1589d9c700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15840b8200 0x7f15841e3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.875+0000 7f1589d9c700 1 -- 172.21.15.60:0/2526974549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15800094a0 con 0x7f15840cede0 2023-07-13T22:24:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.876+0000 7f1589d9c700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15840cede0 0x7f15841598d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f157c014040 tx=0x7f157c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.876+0000 7f1579ffb700 1 -- 172.21.15.60:0/2526974549 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0129f0 con 0x7f15840cede0 2023-07-13T22:24:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.876+0000 7f1579ffb700 1 -- 172.21.15.60:0/2526974549 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c020070 con 0x7f15840cede0 2023-07-13T22:24:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.876+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c009a70 con 0x7f15840cede0 2023-07-13T22:24:19.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.876+0000 7f1579ffb700 1 -- 172.21.15.60:0/2526974549 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c012db0 con 0x7f15840cede0 2023-07-13T22:24:19.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.876+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15840cab30 con 0x7f15840cede0 2023-07-13T22:24:19.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.877+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158404e210 con 0x7f15840cede0 2023-07-13T22:24:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.878+0000 7f1579ffb700 1 -- 172.21.15.60:0/2526974549 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f157c004040 con 0x7f15840cede0 2023-07-13T22:24:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.879+0000 7f1579ffb700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f156806ace0 0x7f156806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.879+0000 7f1579ffb700 1 -- 172.21.15.60:0/2526974549 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f157c022070 con 0x7f15840cede0 2023-07-13T22:24:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.879+0000 7f158959b700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f156806ace0 0x7f156806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.880+0000 7f158959b700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f156806ace0 0x7f156806d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15840ca760 tx=0x7f1580009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:19.884+0000 7f1579ffb700 1 -- 172.21.15.60:0/2526974549 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f157c056180 con 0x7f15840cede0 2023-07-13T22:24:20.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.138+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f15840cb2c0 con 0x7f15840cede0 2023-07-13T22:24:20.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.642+0000 7f1579ffb700 1 -- 172.21.15.60:0/2526974549 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v29) v1 ==== 173+0+0 (secure 0 0 0) 0x7f157c02c020 con 0x7f15840cede0 2023-07-13T22:24:20.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.654+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f156806ace0 msgr2=0x7f156806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:20.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.654+0000 7f158b7ff700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f156806ace0 0x7f156806d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15840ca760 tx=0x7f1580009210 comp rx=0 tx=0).stop 2023-07-13T22:24:20.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.654+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15840cede0 msgr2=0x7f15841598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:20.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.654+0000 7f158b7ff700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15840cede0 0x7f15841598d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f157c014040 tx=0x7f157c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.655+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 shutdown_connections 2023-07-13T22:24:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.655+0000 7f158b7ff700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f156806ace0 0x7f156806d190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.655+0000 7f158b7ff700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15840cede0 0x7f15841598d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.655+0000 7f158b7ff700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15840b1ac0 0x7f15841f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.655+0000 7f158b7ff700 1 --2- 172.21.15.60:0/2526974549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15840b8200 0x7f15841e3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.655+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 >> 172.21.15.60:0/2526974549 conn(0x7f158406b2f0 msgr2=0x7f158406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:20.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.658+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 shutdown_connections 2023-07-13T22:24:20.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.661+0000 7f158b7ff700 1 -- 172.21.15.60:0/2526974549 wait complete. 2023-07-13T22:24:20.646 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 9 and data pool 10 2023-07-13T22:24:20.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:20.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.893+0000 7f17a79ff700 1 Processor -- start 2023-07-13T22:24:20.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.893+0000 7f17a79ff700 1 -- start start 2023-07-13T22:24:20.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.894+0000 7f17a79ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00d26d0 0x7f17a00cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:20.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.894+0000 7f17a79ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17a00b5a00 0x7f17a015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:20.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.894+0000 7f17a79ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a00d14a0 con 0x7f17a00b2010 2023-07-13T22:24:20.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.894+0000 7f17a79ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a00d2570 con 0x7f17a00b5a00 2023-07-13T22:24:20.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.894+0000 7f17a79ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a00d1780 con 0x7f17a00d26d0 2023-07-13T22:24:20.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.895+0000 7f17a5f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17a00b5a00 0x7f17a015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:20.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.895+0000 7f17a579b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f17a00b2010 0x7f17a00cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34528/0 (socket says 172.21.15.60:34528) 2023-07-13T22:24:20.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.895+0000 7f17a579b700 1 -- 172.21.15.60:0/1188084025 learned_addr learned my addr 172.21.15.60:0/1188084025 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:20.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.895+0000 7f17a4f9a700 1 --2- 172.21.15.60:0/1188084025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00d26d0 0x7f17a00cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:20.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.895+0000 7f17a5f9c700 1 -- 172.21.15.60:0/1188084025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00d26d0 msgr2=0x7f17a00cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:20.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.895+0000 7f17a5f9c700 1 --2- 172.21.15.60:0/1188084025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00d26d0 0x7f17a00cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.895+0000 7f17a5f9c700 1 -- 172.21.15.60:0/1188084025 >> v1:172.21.15.60:6789/0 conn(0x7f17a00b2010 legacy=0x7f17a00cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:20.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.895+0000 7f17a5f9c700 1 -- 172.21.15.60:0/1188084025 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a00cd2a0 con 0x7f17a00b5a00 2023-07-13T22:24:20.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.895+0000 7f1797fff700 1 -- 172.21.15.60:0/1188084025 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512090815 0 0) 0x7f17a00d14a0 con 0x7f17a00b2010 2023-07-13T22:24:20.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f17a5f9c700 1 --2- 172.21.15.60:0/1188084025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 0x7f17a015fee0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1798009d90 tx=0x7f179800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f2ca1c8b9dc889 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:20.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f1797fff700 1 -- 172.21.15.60:0/1188084025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179800e070 con 0x7f17a00b5a00 2023-07-13T22:24:20.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f1797fff700 1 -- 172.21.15.60:0/1188084025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179800bbd0 con 0x7f17a00b5a00 2023-07-13T22:24:20.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f1797fff700 1 -- 172.21.15.60:0/1188084025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798012430 con 0x7f17a00b5a00 2023-07-13T22:24:20.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f17a79ff700 1 -- 172.21.15.60:0/1188084025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 msgr2=0x7f17a015fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:20.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f17a79ff700 1 --2- 172.21.15.60:0/1188084025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 0x7f17a015fee0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1798009d90 tx=0x7f179800b330 comp rx=0 tx=0).stop 2023-07-13T22:24:20.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f17a79ff700 1 -- 172.21.15.60:0/1188084025 shutdown_connections 2023-07-13T22:24:20.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f17a79ff700 1 --2- 172.21.15.60:0/1188084025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 0x7f17a015fee0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f17a79ff700 1 --2- 172.21.15.60:0/1188084025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00d26d0 0x7f17a00cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.896+0000 7f17a79ff700 1 -- 172.21.15.60:0/1188084025 >> 172.21.15.60:0/1188084025 conn(0x7f17a006b380 msgr2=0x7f17a006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:20.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.897+0000 7f17a79ff700 1 -- 172.21.15.60:0/1188084025 shutdown_connections 2023-07-13T22:24:20.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.897+0000 7f17a79ff700 1 -- 172.21.15.60:0/1188084025 wait complete. 2023-07-13T22:24:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.898+0000 7f17a79ff700 1 Processor -- start 2023-07-13T22:24:20.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.898+0000 7f17a79ff700 1 -- start start 2023-07-13T22:24:20.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.899+0000 7f17a79ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00b2010 0x7f17a01e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:20.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.899+0000 7f17a79ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 0x7f17a00c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:20.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.899+0000 7f17a79ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17a00d26d0 0x7f17a00c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:20.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.899+0000 7f17a79ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a00c7620 con 0x7f17a00b5a00 2023-07-13T22:24:20.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.899+0000 7f17a79ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a01f52c0 con 0x7f17a00d26d0 2023-07-13T22:24:20.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.899+0000 7f17a79ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a01f55a0 con 0x7f17a00b2010 2023-07-13T22:24:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a4f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 0x7f17a00c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a4f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 0x7f17a00c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57448/0 (socket says 172.21.15.60:57448) 2023-07-13T22:24:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a4f9a700 1 -- 172.21.15.60:0/1609091446 learned_addr learned my addr 172.21.15.60:0/1609091446 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a579b700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00b2010 0x7f17a01e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a5f9c700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17a00d26d0 0x7f17a00c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a4f9a700 1 -- 172.21.15.60:0/1609091446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00b2010 msgr2=0x7f17a01e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a4f9a700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00b2010 0x7f17a01e5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a4f9a700 1 -- 172.21.15.60:0/1609091446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17a00d26d0 msgr2=0x7f17a00c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a4f9a700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17a00d26d0 0x7f17a00c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a4f9a700 1 -- 172.21.15.60:0/1609091446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1798009a70 con 0x7f17a00b5a00 2023-07-13T22:24:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.900+0000 7f17a4f9a700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 0x7f17a00c8a70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f179000e810 tx=0x7f179000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.901+0000 7f1795ffb700 1 -- 172.21.15.60:0/1609091446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179000c070 con 0x7f17a00b5a00 2023-07-13T22:24:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.901+0000 7f1795ffb700 1 -- 172.21.15.60:0/1609091446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1790013610 con 0x7f17a00b5a00 2023-07-13T22:24:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.901+0000 7f1795ffb700 1 -- 172.21.15.60:0/1609091446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17900139c0 con 0x7f17a00b5a00 2023-07-13T22:24:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.901+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a01f5880 con 0x7f17a00b5a00 2023-07-13T22:24:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.901+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a00c94f0 con 0x7f17a00b5a00 2023-07-13T22:24:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.902+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a004e280 con 0x7f17a00b5a00 2023-07-13T22:24:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.903+0000 7f1795ffb700 1 -- 172.21.15.60:0/1609091446 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1790013b20 con 0x7f17a00b5a00 2023-07-13T22:24:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.904+0000 7f1795ffb700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f178406ace0 0x7f178406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.904+0000 7f1795ffb700 1 -- 172.21.15.60:0/1609091446 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1790089210 con 0x7f17a00b5a00 2023-07-13T22:24:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.904+0000 7f17a579b700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f178406ace0 0x7f178406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.904+0000 7f17a579b700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f178406ace0 0x7f178406d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f179c000f60 tx=0x7f179c007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:20.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:20.908+0000 7f1795ffb700 1 -- 172.21.15.60:0/1609091446 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1790054230 con 0x7f17a00b5a00 2023-07-13T22:24:21.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.118+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17a00b6700 con 0x7f17a00b5a00 2023-07-13T22:24:21.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.119+0000 7f1795ffb700 1 -- 172.21.15.60:0/1609091446 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18212 (secure 0 0 0) 0x7f1790057850 con 0x7f17a00b5a00 2023-07-13T22:24:21.102 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:21.102 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":61,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:20.139119+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:24:18.441889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-13T22:24:19.114200+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:21.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f178406ace0 msgr2=0x7f178406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f178406ace0 0x7f178406d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f179c000f60 tx=0x7f179c007e70 comp rx=0 tx=0).stop 2023-07-13T22:24:21.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 msgr2=0x7f17a00c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 0x7f17a00c8a70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f179000e810 tx=0x7f179000eb20 comp rx=0 tx=0).stop 2023-07-13T22:24:21.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 shutdown_connections 2023-07-13T22:24:21.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f178406ace0 0x7f178406d190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17a00d26d0 0x7f17a00c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17a00b2010 0x7f17a01e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 --2- 172.21.15.60:0/1609091446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17a00b5a00 0x7f17a00c8a70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 >> 172.21.15.60:0/1609091446 conn(0x7f17a006b380 msgr2=0x7f17a00d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 shutdown_connections 2023-07-13T22:24:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.122+0000 7f17a79ff700 1 -- 172.21.15.60:0/1609091446 wait complete. 2023-07-13T22:24:21.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.343+0000 7fa1e41ff700 1 Processor -- start 2023-07-13T22:24:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.343+0000 7fa1e41ff700 1 -- start start 2023-07-13T22:24:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.344+0000 7fa1e41ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1dc0d3750 0x7fa1dc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.344+0000 7fa1e41ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0cfad0 0x7fa1dc0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.344+0000 7fa1e41ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0cd950 con 0x7fa1dc0b8400 2023-07-13T22:24:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.344+0000 7fa1e41ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cff50 con 0x7fa1dc0d3750 2023-07-13T22:24:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.344+0000 7fa1e41ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0d0ac0 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.344+0000 7fa1e179a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1dc0d3750 0x7fa1dc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.345+0000 7fa1e179a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1dc0d3750 0x7fa1dc0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57460/0 (socket says 172.21.15.60:57460) 2023-07-13T22:24:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.345+0000 7fa1e179a700 1 -- 172.21.15.60:0/819011578 learned_addr learned my addr 172.21.15.60:0/819011578 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:21.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.345+0000 7fa1e179a700 1 -- 172.21.15.60:0/819011578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0cfad0 msgr2=0x7fa1dc0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.345+0000 7fa1e279c700 1 --2- 172.21.15.60:0/819011578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0cfad0 0x7fa1dc0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.345+0000 7fa1e179a700 1 --2- 172.21.15.60:0/819011578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0cfad0 0x7fa1dc0d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.345+0000 7fa1e179a700 1 -- 172.21.15.60:0/819011578 >> v1:172.21.15.105:6789/0 conn(0x7fa1dc0b8400 legacy=0x7fa1dc0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.345+0000 7fa1e179a700 1 -- 172.21.15.60:0/819011578 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0b87d0 con 0x7fa1dc0d3750 2023-07-13T22:24:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.345+0000 7fa1e279c700 1 --2- 172.21.15.60:0/819011578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0cfad0 0x7fa1dc0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.345+0000 7fa1e179a700 1 --2- 172.21.15.60:0/819011578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0d3750 0x7fa1dc0cf590 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0098f0 tx=0x7fa1cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ae92fb9baf80ca server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e0f99700 1 -- 172.21.15.60:0/819011578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc004230 con 0x7fa1dc0d3750 2023-07-13T22:24:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e0f99700 1 -- 172.21.15.60:0/819011578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc004390 con 0x7fa1dc0d3750 2023-07-13T22:24:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e0f99700 1 -- 172.21.15.60:0/819011578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0036a0 con 0x7fa1dc0d3750 2023-07-13T22:24:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e41ff700 1 -- 172.21.15.60:0/819011578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0d3750 msgr2=0x7fa1dc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e41ff700 1 --2- 172.21.15.60:0/819011578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0d3750 0x7fa1dc0cf590 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0098f0 tx=0x7fa1cc009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e41ff700 1 -- 172.21.15.60:0/819011578 shutdown_connections 2023-07-13T22:24:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e41ff700 1 --2- 172.21.15.60:0/819011578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0cfad0 0x7fa1dc0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e41ff700 1 --2- 172.21.15.60:0/819011578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0d3750 0x7fa1dc0cf590 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e41ff700 1 -- 172.21.15.60:0/819011578 >> 172.21.15.60:0/819011578 conn(0x7fa1dc06b390 msgr2=0x7fa1dc06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e41ff700 1 -- 172.21.15.60:0/819011578 shutdown_connections 2023-07-13T22:24:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.346+0000 7fa1e41ff700 1 -- 172.21.15.60:0/819011578 wait complete. 2023-07-13T22:24:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.348+0000 7fa1e41ff700 1 Processor -- start 2023-07-13T22:24:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.348+0000 7fa1e41ff700 1 -- start start 2023-07-13T22:24:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.349+0000 7fa1e41ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0b8400 0x7fa1dc1f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.349+0000 7fa1e41ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0cfad0 0x7fa1dc0b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.349+0000 7fa1e41ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1dc0d3750 0x7fa1dc0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.349+0000 7fa1e41ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0ccf00 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.349+0000 7fa1e41ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cafc0 con 0x7fa1dc0d3750 2023-07-13T22:24:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.349+0000 7fa1e41ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cb2a0 con 0x7fa1dc0b8400 2023-07-13T22:24:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.349+0000 7fa1e179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0cfad0 0x7fa1dc0b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.349+0000 7fa1e179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0cfad0 0x7fa1dc0b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57470/0 (socket says 172.21.15.60:57470) 2023-07-13T22:24:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.349+0000 7fa1e179a700 1 -- 172.21.15.60:0/1594550977 learned_addr learned my addr 172.21.15.60:0/1594550977 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1e279c700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1dc0d3750 0x7fa1dc0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1e1f9b700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0b8400 0x7fa1dc1f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1e179a700 1 -- 172.21.15.60:0/1594550977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0b8400 msgr2=0x7fa1dc1f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1e179a700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0b8400 0x7fa1dc1f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1e179a700 1 -- 172.21.15.60:0/1594550977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1dc0d3750 msgr2=0x7fa1dc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1e179a700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1dc0d3750 0x7fa1dc0ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1e179a700 1 -- 172.21.15.60:0/1594550977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc009550 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1e179a700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0cfad0 0x7fa1dc0b6da0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009130 tx=0x7fa1cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1d27fc700 1 -- 172.21.15.60:0/1594550977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc003cd0 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1d27fc700 1 -- 172.21.15.60:0/1594550977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc01f070 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1d27fc700 1 -- 172.21.15.60:0/1594550977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00b8a0 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.350+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc15e080 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.351+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc0cb580 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.352+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc15e5a0 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.353+0000 7fa1d27fc700 1 -- 172.21.15.60:0/1594550977 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa1cc021030 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.353+0000 7fa1d27fc700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa1c006ace0 0x7fa1c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.353+0000 7fa1d27fc700 1 -- 172.21.15.60:0/1594550977 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa1cc08a8a0 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.354+0000 7fa1e1f9b700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa1c006ace0 0x7fa1c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.354+0000 7fa1e1f9b700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa1c006ace0 0x7fa1c006d190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009960 tx=0x7fa1d80090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:21.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.358+0000 7fa1d27fc700 1 -- 172.21.15.60:0/1594550977 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa1dc15e5a0 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.567+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1dc0cd950 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.568+0000 7fa1d27fc700 1 -- 172.21.15.60:0/1594550977 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18212 (secure 0 0 0) 0x7fa1dc0cd950 con 0x7fa1dc0cfad0 2023-07-13T22:24:21.550 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:21.550 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":61,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:20.139119+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:24:18.441889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-13T22:24:19.114200+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa1c006ace0 msgr2=0x7fa1c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa1c006ace0 0x7fa1c006d190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009960 tx=0x7fa1d80090e0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0cfad0 msgr2=0x7fa1dc0b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0cfad0 0x7fa1dc0b6da0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009130 tx=0x7fa1cc004750 comp rx=0 tx=0).stop 2023-07-13T22:24:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 shutdown_connections 2023-07-13T22:24:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa1c006ace0 0x7fa1c006d190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1dc0d3750 0x7fa1dc0ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1dc0b8400 0x7fa1dc1f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 --2- 172.21.15.60:0/1594550977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1dc0cfad0 0x7fa1dc0b6da0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 >> 172.21.15.60:0/1594550977 conn(0x7fa1dc06b390 msgr2=0x7fa1dc0b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.570+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 shutdown_connections 2023-07-13T22:24:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.571+0000 7fa1e41ff700 1 -- 172.21.15.60:0/1594550977 wait complete. 2023-07-13T22:24:21.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:24:21.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.798+0000 7f61cadff700 1 Processor -- start 2023-07-13T22:24:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.798+0000 7f61cadff700 1 -- start start 2023-07-13T22:24:21.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.799+0000 7f61cadff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61c40b8200 0x7f61c40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.799+0000 7f61cadff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40cede0 0x7f61c40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.799+0000 7f61cadff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40d1870 con 0x7f61c40b1ac0 2023-07-13T22:24:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.799+0000 7f61cadff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c40cf370 con 0x7f61c40b8200 2023-07-13T22:24:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.799+0000 7f61cadff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40d12b0 con 0x7f61c40cede0 2023-07-13T22:24:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.799+0000 7f61c3fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f61c40b1ac0 0x7f61c415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34544/0 (socket says 172.21.15.60:34544) 2023-07-13T22:24:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.799+0000 7f61c3fff700 1 -- 172.21.15.60:0/537316150 learned_addr learned my addr 172.21.15.60:0/537316150 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.799+0000 7f61c8b9b700 1 --2- 172.21.15.60:0/537316150 >> v2:172.21.15.105:3300/0 conn(0x7f61c40b8200 0x7f61c40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.799+0000 7f61c939c700 1 --2- 172.21.15.60:0/537316150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40cede0 0x7f61c40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.800+0000 7f61c37fe700 1 -- 172.21.15.60:0/537316150 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157477121 0 0) 0x7f61c40d1870 con 0x7f61c40b1ac0 2023-07-13T22:24:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.800+0000 7f61c37fe700 1 -- 172.21.15.60:0/537316150 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b40035b0 con 0x7f61c40b1ac0 2023-07-13T22:24:21.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.800+0000 7f61c939c700 1 -- 172.21.15.60:0/537316150 >> v2:172.21.15.105:3300/0 conn(0x7f61c40b8200 msgr2=0x7f61c40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:21.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.800+0000 7f61c939c700 1 --2- 172.21.15.60:0/537316150 >> v2:172.21.15.105:3300/0 conn(0x7f61c40b8200 0x7f61c40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.800+0000 7f61c939c700 1 -- 172.21.15.60:0/537316150 >> v1:172.21.15.60:6789/0 conn(0x7f61c40b1ac0 legacy=0x7f61c415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.800+0000 7f61c939c700 1 -- 172.21.15.60:0/537316150 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40d1590 con 0x7f61c40cede0 2023-07-13T22:24:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.800+0000 7f61c8b9b700 1 --2- 172.21.15.60:0/537316150 >> v2:172.21.15.105:3300/0 conn(0x7f61c40b8200 0x7f61c40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.800+0000 7f61c939c700 1 --2- 172.21.15.60:0/537316150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40cede0 0x7f61c40cfb80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009d90 tx=0x7f61bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a2f506227d0b505 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61c37fe700 1 -- 172.21.15.60:0/537316150 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00e070 con 0x7f61c40cede0 2023-07-13T22:24:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61c37fe700 1 -- 172.21.15.60:0/537316150 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc00bb20 con 0x7f61c40cede0 2023-07-13T22:24:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61c37fe700 1 -- 172.21.15.60:0/537316150 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc012430 con 0x7f61c40cede0 2023-07-13T22:24:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61cadff700 1 -- 172.21.15.60:0/537316150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40cede0 msgr2=0x7f61c40cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61cadff700 1 --2- 172.21.15.60:0/537316150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40cede0 0x7f61c40cfb80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009d90 tx=0x7f61bc00b330 comp rx=0 tx=0).stop 2023-07-13T22:24:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61cadff700 1 -- 172.21.15.60:0/537316150 shutdown_connections 2023-07-13T22:24:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61cadff700 1 --2- 172.21.15.60:0/537316150 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40cede0 0x7f61c40cfb80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61cadff700 1 --2- 172.21.15.60:0/537316150 >> v2:172.21.15.105:3300/0 conn(0x7f61c40b8200 0x7f61c40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61cadff700 1 -- 172.21.15.60:0/537316150 >> 172.21.15.60:0/537316150 conn(0x7f61c406b2f0 msgr2=0x7f61c406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61cadff700 1 -- 172.21.15.60:0/537316150 shutdown_connections 2023-07-13T22:24:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.801+0000 7f61cadff700 1 -- 172.21.15.60:0/537316150 wait complete. 2023-07-13T22:24:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.802+0000 7f61cadff700 1 Processor -- start 2023-07-13T22:24:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.802+0000 7f61cadff700 1 -- start start 2023-07-13T22:24:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.803+0000 7f61cadff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40b1ac0 0x7f61c4159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.803+0000 7f61cadff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61c40b8200 0x7f61c40c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.803+0000 7f61cadff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cede0 0x7f61c40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.803+0000 7f61cadff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4158e20 con 0x7f61c40cede0 2023-07-13T22:24:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.803+0000 7f61cadff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4159100 con 0x7f61c40b8200 2023-07-13T22:24:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.803+0000 7f61cadff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c41593e0 con 0x7f61c40b1ac0 2023-07-13T22:24:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c939c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cede0 0x7f61c40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c939c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cede0 0x7f61c40cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57472/0 (socket says 172.21.15.60:57472) 2023-07-13T22:24:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c8b9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40b1ac0 0x7f61c4159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c939c700 1 -- 172.21.15.60:0/1805812351 learned_addr learned my addr 172.21.15.60:0/1805812351 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c3fff700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61c40b8200 0x7f61c40c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c939c700 1 -- 172.21.15.60:0/1805812351 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40b1ac0 msgr2=0x7f61c4159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c939c700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40b1ac0 0x7f61c4159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c939c700 1 -- 172.21.15.60:0/1805812351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61c40b8200 msgr2=0x7f61c40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c939c700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61c40b8200 0x7f61c40c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c939c700 1 -- 172.21.15.60:0/1805812351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b80094a0 con 0x7f61c40cede0 2023-07-13T22:24:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c939c700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cede0 0x7f61c40cc310 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f61bc014040 tx=0x7f61bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.804+0000 7f61c17fa700 1 -- 172.21.15.60:0/1805812351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0129f0 con 0x7f61c40cede0 2023-07-13T22:24:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.805+0000 7f61c17fa700 1 -- 172.21.15.60:0/1805812351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc020070 con 0x7f61c40cede0 2023-07-13T22:24:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.805+0000 7f61c17fa700 1 -- 172.21.15.60:0/1805812351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc012db0 con 0x7f61c40cede0 2023-07-13T22:24:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.805+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc009a70 con 0x7f61c40cede0 2023-07-13T22:24:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.805+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c41599a0 con 0x7f61c40cede0 2023-07-13T22:24:21.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.806+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c404e210 con 0x7f61c40cede0 2023-07-13T22:24:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.807+0000 7f61c17fa700 1 -- 172.21.15.60:0/1805812351 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61bc004040 con 0x7f61c40cede0 2023-07-13T22:24:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.808+0000 7f61c17fa700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61a806ace0 0x7f61a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.808+0000 7f61c17fa700 1 -- 172.21.15.60:0/1805812351 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61bc022070 con 0x7f61c40cede0 2023-07-13T22:24:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.808+0000 7f61c8b9b700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61a806ace0 0x7f61a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.809+0000 7f61c8b9b700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61a806ace0 0x7f61a806d190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009d70 tx=0x7f61b8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:21.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:21.812+0000 7f61c17fa700 1 -- 172.21.15.60:0/1805812351 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f61bc0560f0 con 0x7f61c40cede0 2023-07-13T22:24:22.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.064+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f61c40d3680 con 0x7f61c40cede0 2023-07-13T22:24:22.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.665+0000 7f61c17fa700 1 -- 172.21.15.60:0/1805812351 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f61bc059710 con 0x7f61c40cede0 2023-07-13T22:24:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61a806ace0 msgr2=0x7f61a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61a806ace0 0x7f61a806d190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009d70 tx=0x7f61b8009210 comp rx=0 tx=0).stop 2023-07-13T22:24:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cede0 msgr2=0x7f61c40cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cede0 0x7f61c40cc310 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f61bc014040 tx=0x7f61bc004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 shutdown_connections 2023-07-13T22:24:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61a806ace0 0x7f61a806d190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cede0 0x7f61c40cc310 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61c40b1ac0 0x7f61c4159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 --2- 172.21.15.60:0/1805812351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61c40b8200 0x7f61c40c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 >> 172.21.15.60:0/1805812351 conn(0x7f61c406b2f0 msgr2=0x7f61c406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 shutdown_connections 2023-07-13T22:24:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.668+0000 7f61cadff700 1 -- 172.21.15.60:0/1805812351 wait complete. 2023-07-13T22:24:22.672 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:24:22.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:24:22.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.902+0000 7f76aefff700 1 Processor -- start 2023-07-13T22:24:22.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.902+0000 7f76aefff700 1 -- start start 2023-07-13T22:24:22.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76aefff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f76a8159260 0x7f76a80ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:22.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76aefff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 0x7f76a80cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:22.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76aefff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76a80703e0 0x7f76a8070840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:22.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76aefff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a8070d80 con 0x7f76a80703e0 2023-07-13T22:24:22.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76aefff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a80cf640 con 0x7f76a8159260 2023-07-13T22:24:22.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76aefff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80ccf80 con 0x7f76a80cc410 2023-07-13T22:24:22.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76ad59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76a80703e0 0x7f76a8070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:22.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76ad59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76a80703e0 0x7f76a8070840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57480/0 (socket says 172.21.15.60:57480) 2023-07-13T22:24:22.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76ad59c700 1 -- 172.21.15.60:0/672745671 learned_addr learned my addr 172.21.15.60:0/672745671 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:22.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76a7fff700 1 --2- 172.21.15.60:0/672745671 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 0x7f76a80cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:22.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76acd9b700 1 --2- 172.21.15.60:0/672745671 >> v2:172.21.15.105:3300/0 conn(0x7f76a8159260 0x7f76a80ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:22.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76ad59c700 1 -- 172.21.15.60:0/672745671 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 msgr2=0x7f76a80cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:22.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76ad59c700 1 --2- 172.21.15.60:0/672745671 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 0x7f76a80cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76ad59c700 1 -- 172.21.15.60:0/672745671 >> v2:172.21.15.105:3300/0 conn(0x7f76a8159260 msgr2=0x7f76a80ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:22.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76ad59c700 1 --2- 172.21.15.60:0/672745671 >> v2:172.21.15.105:3300/0 conn(0x7f76a8159260 0x7f76a80ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.904+0000 7f76ad59c700 1 -- 172.21.15.60:0/672745671 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80c9b40 con 0x7f76a80703e0 2023-07-13T22:24:22.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.905+0000 7f76acd9b700 1 --2- 172.21.15.60:0/672745671 >> v2:172.21.15.105:3300/0 conn(0x7f76a8159260 0x7f76a80ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:22.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.905+0000 7f76a7fff700 1 --2- 172.21.15.60:0/672745671 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 0x7f76a80cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:22.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.905+0000 7f76ad59c700 1 --2- 172.21.15.60:0/672745671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 0x7f76a8070840 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f76a8071110 tx=0x7f76a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5609d0abe4c7b5c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:22.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.905+0000 7f76a77fe700 1 -- 172.21.15.60:0/672745671 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a000e070 con 0x7f76a80703e0 2023-07-13T22:24:22.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.905+0000 7f76a77fe700 1 -- 172.21.15.60:0/672745671 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a000b9c0 con 0x7f76a80703e0 2023-07-13T22:24:22.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.905+0000 7f76a77fe700 1 -- 172.21.15.60:0/672745671 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a000bd70 con 0x7f76a80703e0 2023-07-13T22:24:22.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.906+0000 7f76aefff700 1 -- 172.21.15.60:0/672745671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 msgr2=0x7f76a8070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:22.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.906+0000 7f76aefff700 1 --2- 172.21.15.60:0/672745671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 0x7f76a8070840 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f76a8071110 tx=0x7f76a000b330 comp rx=0 tx=0).stop 2023-07-13T22:24:22.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.906+0000 7f76aefff700 1 -- 172.21.15.60:0/672745671 shutdown_connections 2023-07-13T22:24:22.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.906+0000 7f76aefff700 1 --2- 172.21.15.60:0/672745671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 0x7f76a8070840 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.906+0000 7f76aefff700 1 --2- 172.21.15.60:0/672745671 >> v2:172.21.15.105:3300/0 conn(0x7f76a8159260 0x7f76a80ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.906+0000 7f76aefff700 1 --2- 172.21.15.60:0/672745671 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 0x7f76a80cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.906+0000 7f76aefff700 1 -- 172.21.15.60:0/672745671 >> 172.21.15.60:0/672745671 conn(0x7f76a80cbf50 msgr2=0x7f76a80ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:22.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.906+0000 7f76aefff700 1 -- 172.21.15.60:0/672745671 shutdown_connections 2023-07-13T22:24:22.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.906+0000 7f76aefff700 1 -- 172.21.15.60:0/672745671 wait complete. 2023-07-13T22:24:22.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.907+0000 7f76aefff700 1 Processor -- start 2023-07-13T22:24:22.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.908+0000 7f76aefff700 1 -- start start 2023-07-13T22:24:22.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76aefff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 0x7f76a80c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:22.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76aefff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 0x7f76a815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:22.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76aefff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a8159260 0x7f76a815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:22.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76aefff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a815b4d0 con 0x7f76a80703e0 2023-07-13T22:24:22.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76aefff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a815b7b0 con 0x7f76a8159260 2023-07-13T22:24:22.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76aefff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a815ba90 con 0x7f76a80cc410 2023-07-13T22:24:22.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76acd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 0x7f76a80c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:22.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76acd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 0x7f76a80c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57492/0 (socket says 172.21.15.60:57492) 2023-07-13T22:24:22.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76acd9b700 1 -- 172.21.15.60:0/4252142566 learned_addr learned my addr 172.21.15.60:0/4252142566 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:22.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76acd9b700 1 -- 172.21.15.60:0/4252142566 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 msgr2=0x7f76a815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:22.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76acd9b700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 0x7f76a815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76acd9b700 1 -- 172.21.15.60:0/4252142566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a8159260 msgr2=0x7f76a815ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:22.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76acd9b700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a8159260 0x7f76a815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:22.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76acd9b700 1 -- 172.21.15.60:0/4252142566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0009b30 con 0x7f76a80703e0 2023-07-13T22:24:22.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.909+0000 7f76acd9b700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 0x7f76a80c4640 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c750 tx=0x7f769c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:22.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.910+0000 7f76a57fa700 1 -- 172.21.15.60:0/4252142566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c011800 con 0x7f76a80703e0 2023-07-13T22:24:22.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.910+0000 7f76a57fa700 1 -- 172.21.15.60:0/4252142566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c011960 con 0x7f76a80703e0 2023-07-13T22:24:22.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.910+0000 7f76a57fa700 1 -- 172.21.15.60:0/4252142566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c011cd0 con 0x7f76a80703e0 2023-07-13T22:24:22.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.910+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a815bd70 con 0x7f76a80703e0 2023-07-13T22:24:22.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.910+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a80c4b80 con 0x7f76a80703e0 2023-07-13T22:24:22.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.911+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a815c290 con 0x7f76a80703e0 2023-07-13T22:24:22.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.912+0000 7f76a57fa700 1 -- 172.21.15.60:0/4252142566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f769c021350 con 0x7f76a80703e0 2023-07-13T22:24:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.913+0000 7f76a57fa700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f768c06ad10 0x7f768c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.913+0000 7f76a57fa700 1 -- 172.21.15.60:0/4252142566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f769c089470 con 0x7f76a80703e0 2023-07-13T22:24:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.913+0000 7f76a7fff700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f768c06ad10 0x7f768c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:22.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.914+0000 7f76a7fff700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f768c06ad10 0x7f768c06d1c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f76a80c58c0 tx=0x7f7694008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:22.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:22.917+0000 7f76a57fa700 1 -- 172.21.15.60:0/4252142566 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f76a815c290 con 0x7f76a80703e0 2023-07-13T22:24:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.170+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f76a815c290 con 0x7f76a80703e0 2023-07-13T22:24:23.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.667+0000 7f76a57fa700 1 -- 172.21.15.60:0/4252142566 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f76a815c290 con 0x7f76a80703e0 2023-07-13T22:24:23.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f768c06ad10 msgr2=0x7f768c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:23.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f768c06ad10 0x7f768c06d1c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f76a80c58c0 tx=0x7f7694008040 comp rx=0 tx=0).stop 2023-07-13T22:24:23.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 msgr2=0x7f76a80c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:23.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 0x7f76a80c4640 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c750 tx=0x7f769c00cb10 comp rx=0 tx=0).stop 2023-07-13T22:24:23.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 shutdown_connections 2023-07-13T22:24:23.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f768c06ad10 0x7f768c06d1c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:23.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76a80703e0 0x7f76a80c4640 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:23.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76a8159260 0x7f76a815ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:23.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 --2- 172.21.15.60:0/4252142566 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76a80cc410 0x7f76a815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:23.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 >> 172.21.15.60:0/4252142566 conn(0x7f76a80cbf50 msgr2=0x7f76a80b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:23.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.670+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 shutdown_connections 2023-07-13T22:24:23.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.671+0000 7f76aefff700 1 -- 172.21.15.60:0/4252142566 wait complete. 2023-07-13T22:24:23.674 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:24:23.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:23.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.896+0000 7f2921dff700 1 Processor -- start 2023-07-13T22:24:23.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.897+0000 7f2921dff700 1 -- start start 2023-07-13T22:24:23.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.898+0000 7f2921dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f291c0cfca0 0x7f291c167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:23.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.898+0000 7f2921dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291c0cd950 con 0x7f291c0b8400 2023-07-13T22:24:23.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.898+0000 7f2921dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291c0d0ac0 con 0x7f291c0d3750 2023-07-13T22:24:23.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.898+0000 7f2921dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291c0b87d0 con 0x7f291c0cfca0 2023-07-13T22:24:23.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.898+0000 7f291bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f291c0cfca0 0x7f291c167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:23.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.898+0000 7f291b7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f291c0b8400 0x7f291c0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34554/0 (socket says 172.21.15.60:34554) 2023-07-13T22:24:23.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.898+0000 7f291b7fe700 1 -- 172.21.15.60:0/2065501926 learned_addr learned my addr 172.21.15.60:0/2065501926 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:23.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.898+0000 7f291bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f291c0cfca0 0x7f291c167050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57504/0 (socket says 172.21.15.60:57504) 2023-07-13T22:24:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.899+0000 7f291bfff700 1 -- 172.21.15.60:0/2065501926 >> v1:172.21.15.105:6789/0 conn(0x7f291c0d3750 legacy=0x7f291c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.899+0000 7f291bfff700 1 -- 172.21.15.60:0/2065501926 >> v1:172.21.15.60:6789/0 conn(0x7f291c0b8400 legacy=0x7f291c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.899+0000 7f291bfff700 1 -- 172.21.15.60:0/2065501926 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0b4eb0 con 0x7f291c0cfca0 2023-07-13T22:24:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.899+0000 7f291affd700 1 -- 172.21.15.60:0/2065501926 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212672161 0 0) 0x7f290c001110 con 0x7f291c0b8400 2023-07-13T22:24:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.899+0000 7f291bfff700 1 --2- 172.21.15.60:0/2065501926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0cfca0 0x7f291c167050 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2914009d90 tx=0x7f291400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7909b78cf5bdfa28 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.899+0000 7f291affd700 1 -- 172.21.15.60:0/2065501926 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291400e070 con 0x7f291c0cfca0 2023-07-13T22:24:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.899+0000 7f291affd700 1 -- 172.21.15.60:0/2065501926 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291400bb90 con 0x7f291c0cfca0 2023-07-13T22:24:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.899+0000 7f291affd700 1 -- 172.21.15.60:0/2065501926 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914003a20 con 0x7f291c0cfca0 2023-07-13T22:24:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.900+0000 7f2921dff700 1 -- 172.21.15.60:0/2065501926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0cfca0 msgr2=0x7f291c167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.900+0000 7f2921dff700 1 --2- 172.21.15.60:0/2065501926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0cfca0 0x7f291c167050 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2914009d90 tx=0x7f291400b330 comp rx=0 tx=0).stop 2023-07-13T22:24:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.900+0000 7f2921dff700 1 -- 172.21.15.60:0/2065501926 shutdown_connections 2023-07-13T22:24:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.900+0000 7f2921dff700 1 --2- 172.21.15.60:0/2065501926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0cfca0 0x7f291c167050 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.900+0000 7f2921dff700 1 -- 172.21.15.60:0/2065501926 >> 172.21.15.60:0/2065501926 conn(0x7f291c06b390 msgr2=0x7f291c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.900+0000 7f2921dff700 1 -- 172.21.15.60:0/2065501926 shutdown_connections 2023-07-13T22:24:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.900+0000 7f2921dff700 1 -- 172.21.15.60:0/2065501926 wait complete. 2023-07-13T22:24:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.902+0000 7f2921dff700 1 Processor -- start 2023-07-13T22:24:23.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.903+0000 7f2921dff700 1 -- start start 2023-07-13T22:24:23.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f2921dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0b8400 0x7f291c150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:23.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f2921dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f291c0cfca0 0x7f291c1f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:23.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f2921dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0d3750 0x7f291c15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:23.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f2921dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c15e350 con 0x7f291c0d3750 2023-07-13T22:24:23.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f2921dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0c8780 con 0x7f291c0b8400 2023-07-13T22:24:23.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f2921dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0c7e50 con 0x7f291c0cfca0 2023-07-13T22:24:23.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f291bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0d3750 0x7f291c15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f291bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0d3750 0x7f291c15d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57508/0 (socket says 172.21.15.60:57508) 2023-07-13T22:24:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f291bfff700 1 -- 172.21.15.60:0/1527327251 learned_addr learned my addr 172.21.15.60:0/1527327251 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f291b7fe700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0b8400 0x7f291c150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.904+0000 7f2913fff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f291c0cfca0 0x7f291c1f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f291bfff700 1 -- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f291c0cfca0 msgr2=0x7f291c1f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f291bfff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f291c0cfca0 0x7f291c1f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f291bfff700 1 -- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0b8400 msgr2=0x7f291c150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f291bfff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0b8400 0x7f291c150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f291bfff700 1 -- 172.21.15.60:0/1527327251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2914009a70 con 0x7f291c0d3750 2023-07-13T22:24:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f2913fff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f291c0cfca0 0x7f291c1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f291b7fe700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0b8400 0x7f291c150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f291bfff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0d3750 0x7f291c15d890 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f291c06c720 tx=0x7f2914004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f2918ff9700 1 -- 172.21.15.60:0/1527327251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914012630 con 0x7f291c0d3750 2023-07-13T22:24:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f2918ff9700 1 -- 172.21.15.60:0/1527327251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29140127b0 con 0x7f291c0d3750 2023-07-13T22:24:23.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f2918ff9700 1 -- 172.21.15.60:0/1527327251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914012b60 con 0x7f291c0d3750 2023-07-13T22:24:23.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c0c8130 con 0x7f291c0d3750 2023-07-13T22:24:23.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.905+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c0c7610 con 0x7f291c0d3750 2023-07-13T22:24:23.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.906+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291c0ce7a0 con 0x7f291c0d3750 2023-07-13T22:24:23.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.907+0000 7f2918ff9700 1 -- 172.21.15.60:0/1527327251 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2914012cc0 con 0x7f291c0d3750 2023-07-13T22:24:23.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.908+0000 7f2918ff9700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28fc06ace0 0x7f28fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:23.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.908+0000 7f2918ff9700 1 -- 172.21.15.60:0/1527327251 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f291408b350 con 0x7f291c0d3750 2023-07-13T22:24:23.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.908+0000 7f291b7fe700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28fc06ace0 0x7f28fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:23.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.909+0000 7f291b7fe700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28fc06ace0 0x7f28fc06d190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f290c000f60 tx=0x7f290c007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:23.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:23.913+0000 7f2918ff9700 1 -- 172.21.15.60:0/1527327251 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f29140563f0 con 0x7f291c0d3750 2023-07-13T22:24:24.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.159+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f291c0c88c0 con 0x7f291c0d3750 2023-07-13T22:24:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.159+0000 7f2918ff9700 1 -- 172.21.15.60:0/1527327251 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f2914059a10 con 0x7f291c0d3750 2023-07-13T22:24:24.142 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:24.142 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/711733521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":711733521},{"type":"v1","addr":"172.21.15.105:6834","nonce":711733521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2135065566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":2135065566},{"type":"v1","addr":"172.21.15.105:6835","nonce":2135065566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6052,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4236704538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4236704538},{"type":"v1","addr":"172.21.15.60:6836","nonce":4236704538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:24:20.139470+0000","modified":"2023-07-13T22:24:23.665486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6058":{"gid":6058,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3036622731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3036622731},{"type":"v1","addr":"172.21.15.60:6837","nonce":3036622731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-13T22:24:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28fc06ace0 msgr2=0x7f28fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28fc06ace0 0x7f28fc06d190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f290c000f60 tx=0x7f290c007d30 comp rx=0 tx=0).stop 2023-07-13T22:24:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0d3750 msgr2=0x7f291c15d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0d3750 0x7f291c15d890 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f291c06c720 tx=0x7f2914004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 shutdown_connections 2023-07-13T22:24:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28fc06ace0 0x7f28fc06d190 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291c0d3750 0x7f291c15d890 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f291c0b8400 0x7f291c150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 --2- 172.21.15.60:0/1527327251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f291c0cfca0 0x7f291c1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 >> 172.21.15.60:0/1527327251 conn(0x7f291c06b390 msgr2=0x7f291c15a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 shutdown_connections 2023-07-13T22:24:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.162+0000 7f2921dff700 1 -- 172.21.15.60:0/1527327251 wait complete. 2023-07-13T22:24:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 33 2023-07-13T22:24:24.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.382+0000 7fa6517ff700 1 Processor -- start 2023-07-13T22:24:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.383+0000 7fa6517ff700 1 -- start start 2023-07-13T22:24:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.384+0000 7fa6517ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa64c0b12a0 0x7fa64c0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.384+0000 7fa6517ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa64c0cd760 0x7fa64c0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.384+0000 7fa6517ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c1609a0 con 0x7fa64c06bad0 2023-07-13T22:24:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.384+0000 7fa6517ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c06a5f0 con 0x7fa64c0cd760 2023-07-13T22:24:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.384+0000 7fa6517ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c160ed0 con 0x7fa64c0b12a0 2023-07-13T22:24:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.384+0000 7fa64b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa64c0cd760 0x7fa64c0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.384+0000 7fa64b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa64c0cd760 0x7fa64c0cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57524/0 (socket says 172.21.15.60:57524) 2023-07-13T22:24:24.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.384+0000 7fa64b7fe700 1 -- 172.21.15.60:0/3359074888 learned_addr learned my addr 172.21.15.60:0/3359074888 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:24.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.384+0000 7fa64affd700 1 --2- 172.21.15.60:0/3359074888 >> v2:172.21.15.105:3300/0 conn(0x7fa64c0b12a0 0x7fa64c0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.385+0000 7fa64b7fe700 1 -- 172.21.15.60:0/3359074888 >> v2:172.21.15.105:3300/0 conn(0x7fa64c0b12a0 msgr2=0x7fa64c0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:24.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.385+0000 7fa64b7fe700 1 --2- 172.21.15.60:0/3359074888 >> v2:172.21.15.105:3300/0 conn(0x7fa64c0b12a0 0x7fa64c0cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.385+0000 7fa64b7fe700 1 -- 172.21.15.60:0/3359074888 >> v1:172.21.15.105:6789/0 conn(0x7fa64c06bad0 legacy=0x7fa64c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.385+0000 7fa64b7fe700 1 -- 172.21.15.60:0/3359074888 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c15c860 con 0x7fa64c0cd760 2023-07-13T22:24:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.385+0000 7fa64affd700 1 --2- 172.21.15.60:0/3359074888 >> v2:172.21.15.105:3300/0 conn(0x7fa64c0b12a0 0x7fa64c0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:24.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.385+0000 7fa64b7fe700 1 --2- 172.21.15.60:0/3359074888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0cd760 0x7fa64c0cdb80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa64c06c110 tx=0x7fa64400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18fc22543e9c09b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.385+0000 7fa649ffb700 1 -- 172.21.15.60:0/3359074888 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400e070 con 0x7fa64c0cd760 2023-07-13T22:24:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.385+0000 7fa649ffb700 1 -- 172.21.15.60:0/3359074888 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa644003a20 con 0x7fa64c0cd760 2023-07-13T22:24:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.385+0000 7fa649ffb700 1 -- 172.21.15.60:0/3359074888 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644003dd0 con 0x7fa64c0cd760 2023-07-13T22:24:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.386+0000 7fa6517ff700 1 -- 172.21.15.60:0/3359074888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0cd760 msgr2=0x7fa64c0cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.386+0000 7fa6517ff700 1 --2- 172.21.15.60:0/3359074888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0cd760 0x7fa64c0cdb80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa64c06c110 tx=0x7fa64400b330 comp rx=0 tx=0).stop 2023-07-13T22:24:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.386+0000 7fa6517ff700 1 -- 172.21.15.60:0/3359074888 shutdown_connections 2023-07-13T22:24:24.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.386+0000 7fa6517ff700 1 --2- 172.21.15.60:0/3359074888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0cd760 0x7fa64c0cdb80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.386+0000 7fa6517ff700 1 --2- 172.21.15.60:0/3359074888 >> v2:172.21.15.105:3300/0 conn(0x7fa64c0b12a0 0x7fa64c0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.386+0000 7fa6517ff700 1 -- 172.21.15.60:0/3359074888 >> 172.21.15.60:0/3359074888 conn(0x7fa64c0cc2a0 msgr2=0x7fa64c0cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:24.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.386+0000 7fa6517ff700 1 -- 172.21.15.60:0/3359074888 shutdown_connections 2023-07-13T22:24:24.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.386+0000 7fa6517ff700 1 -- 172.21.15.60:0/3359074888 wait complete. 2023-07-13T22:24:24.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.388+0000 7fa6517ff700 1 Processor -- start 2023-07-13T22:24:24.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.388+0000 7fa6517ff700 1 -- start start 2023-07-13T22:24:24.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa6517ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c06bad0 0x7fa64c0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa6517ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0b12a0 0x7fa64c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa6517ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c0cd760 0x7fa64c0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa6517ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c7d50 con 0x7fa64c0b12a0 2023-07-13T22:24:24.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa6517ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c6e30 con 0x7fa64c06bad0 2023-07-13T22:24:24.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa6517ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c70d0 con 0x7fa64c0cd760 2023-07-13T22:24:24.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0b12a0 0x7fa64c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0b12a0 0x7fa64c159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57526/0 (socket says 172.21.15.60:57526) 2023-07-13T22:24:24.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64a7fc700 1 -- 172.21.15.60:0/308697155 learned_addr learned my addr 172.21.15.60:0/308697155 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:24.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64affd700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c06bad0 0x7fa64c0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64b7fe700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c0cd760 0x7fa64c0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64a7fc700 1 -- 172.21.15.60:0/308697155 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c0cd760 msgr2=0x7fa64c0c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64a7fc700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c0cd760 0x7fa64c0c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64a7fc700 1 -- 172.21.15.60:0/308697155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c06bad0 msgr2=0x7fa64c0c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64a7fc700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c06bad0 0x7fa64c0c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64a7fc700 1 -- 172.21.15.60:0/308697155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644009b30 con 0x7fa64c0b12a0 2023-07-13T22:24:24.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.389+0000 7fa64affd700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c06bad0 0x7fa64c0c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:24.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.390+0000 7fa64a7fc700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0b12a0 0x7fa64c159d40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa64000c4d0 tx=0x7fa64000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:24.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.390+0000 7fa6337fe700 1 -- 172.21.15.60:0/308697155 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640011790 con 0x7fa64c0b12a0 2023-07-13T22:24:24.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.390+0000 7fa6337fe700 1 -- 172.21.15.60:0/308697155 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6400118f0 con 0x7fa64c0b12a0 2023-07-13T22:24:24.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.390+0000 7fa6337fe700 1 -- 172.21.15.60:0/308697155 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640011ca0 con 0x7fa64c0b12a0 2023-07-13T22:24:24.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.390+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c0c73b0 con 0x7fa64c0b12a0 2023-07-13T22:24:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.390+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c0c7970 con 0x7fa64c0b12a0 2023-07-13T22:24:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.391+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c0c7e90 con 0x7fa64c0b12a0 2023-07-13T22:24:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.392+0000 7fa6337fe700 1 -- 172.21.15.60:0/308697155 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa64001c460 con 0x7fa64c0b12a0 2023-07-13T22:24:24.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.393+0000 7fa6337fe700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63806ace0 0x7fa63806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.393+0000 7fa6337fe700 1 -- 172.21.15.60:0/308697155 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa6400891a0 con 0x7fa64c0b12a0 2023-07-13T22:24:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.393+0000 7fa64affd700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63806ace0 0x7fa63806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.394+0000 7fa64affd700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63806ace0 0x7fa63806d190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009a50 tx=0x7fa63c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.398+0000 7fa6337fe700 1 -- 172.21.15.60:0/308697155 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa64c0c7e90 con 0x7fa64c0b12a0 2023-07-13T22:24:24.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.607+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa64c0c7e90 con 0x7fa64c0b12a0 2023-07-13T22:24:24.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.608+0000 7fa6337fe700 1 -- 172.21.15.60:0/308697155 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18199 (secure 0 0 0) 0x7fa64c0c7e90 con 0x7fa64c0b12a0 2023-07-13T22:24:24.592 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:24.592 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:21.653794+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:24:18.441889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-13T22:24:19.114200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63806ace0 msgr2=0x7fa63806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63806ace0 0x7fa63806d190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009a50 tx=0x7fa63c008040 comp rx=0 tx=0).stop 2023-07-13T22:24:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0b12a0 msgr2=0x7fa64c159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0b12a0 0x7fa64c159d40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa64000c4d0 tx=0x7fa64000c7e0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 shutdown_connections 2023-07-13T22:24:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63806ace0 0x7fa63806d190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c0cd760 0x7fa64c0c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c06bad0 0x7fa64c0c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 --2- 172.21.15.60:0/308697155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c0b12a0 0x7fa64c159d40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 >> 172.21.15.60:0/308697155 conn(0x7fa64c0cc2a0 msgr2=0x7fa64c0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 shutdown_connections 2023-07-13T22:24:24.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.611+0000 7fa6517ff700 1 -- 172.21.15.60:0/308697155 wait complete. 2023-07-13T22:24:24.613 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:24:24.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:24:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.835+0000 7fa0813ff700 1 Processor -- start 2023-07-13T22:24:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.835+0000 7fa0813ff700 1 -- start start 2023-07-13T22:24:24.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.836+0000 7fa0813ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa07c0b8400 0x7fa07c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.836+0000 7fa0813ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0d3750 0x7fa07c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.836+0000 7fa0813ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07c0cff20 con 0x7fa07c0cfad0 2023-07-13T22:24:24.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.836+0000 7fa0813ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07c0d0ac0 con 0x7fa07c0b8400 2023-07-13T22:24:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.836+0000 7fa0813ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0b87d0 con 0x7fa07c0d3750 2023-07-13T22:24:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa07a7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0d3750 0x7fa07c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa07affd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa07c0b8400 0x7fa07c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa07b7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa07c0cfad0 0x7fa07c15fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:41834/0 (socket says 172.21.15.60:41834) 2023-07-13T22:24:24.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa07b7fe700 1 -- 172.21.15.60:0/4083436692 learned_addr learned my addr 172.21.15.60:0/4083436692 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:24.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa07affd700 1 -- 172.21.15.60:0/4083436692 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0d3750 msgr2=0x7fa07c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:24.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa07affd700 1 --2- 172.21.15.60:0/4083436692 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0d3750 0x7fa07c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa07affd700 1 -- 172.21.15.60:0/4083436692 >> v1:172.21.15.105:6789/0 conn(0x7fa07c0cfad0 legacy=0x7fa07c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa07affd700 1 -- 172.21.15.60:0/4083436692 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c0b4eb0 con 0x7fa07c0b8400 2023-07-13T22:24:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa07a7fc700 1 --2- 172.21.15.60:0/4083436692 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0d3750 0x7fa07c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.837+0000 7fa079ffb700 1 -- 172.21.15.60:0/4083436692 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616574186 0 0) 0x7fa074001490 con 0x7fa07c0cfad0 2023-07-13T22:24:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.838+0000 7fa07affd700 1 --2- 172.21.15.60:0/4083436692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0b8400 0x7fa07c0cd240 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009840 tx=0x7fa06c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=848e730f361042d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:24.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.838+0000 7fa079ffb700 1 -- 172.21.15.60:0/4083436692 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c011830 con 0x7fa07c0b8400 2023-07-13T22:24:24.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.838+0000 7fa079ffb700 1 -- 172.21.15.60:0/4083436692 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c011990 con 0x7fa07c0b8400 2023-07-13T22:24:24.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.838+0000 7fa079ffb700 1 -- 172.21.15.60:0/4083436692 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c011d20 con 0x7fa07c0b8400 2023-07-13T22:24:24.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.838+0000 7fa0813ff700 1 -- 172.21.15.60:0/4083436692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0b8400 msgr2=0x7fa07c0cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.838+0000 7fa0813ff700 1 --2- 172.21.15.60:0/4083436692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0b8400 0x7fa07c0cd240 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009840 tx=0x7fa06c009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.839+0000 7fa0813ff700 1 -- 172.21.15.60:0/4083436692 shutdown_connections 2023-07-13T22:24:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.839+0000 7fa0813ff700 1 --2- 172.21.15.60:0/4083436692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0b8400 0x7fa07c0cd240 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.839+0000 7fa0813ff700 1 --2- 172.21.15.60:0/4083436692 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0d3750 0x7fa07c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.839+0000 7fa0813ff700 1 -- 172.21.15.60:0/4083436692 >> 172.21.15.60:0/4083436692 conn(0x7fa07c06b390 msgr2=0x7fa07c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.839+0000 7fa0813ff700 1 -- 172.21.15.60:0/4083436692 shutdown_connections 2023-07-13T22:24:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.839+0000 7fa0813ff700 1 -- 172.21.15.60:0/4083436692 wait complete. 2023-07-13T22:24:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.840+0000 7fa0813ff700 1 Processor -- start 2023-07-13T22:24:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.841+0000 7fa0813ff700 1 -- start start 2023-07-13T22:24:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa0813ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0b8400 0x7fa07c0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa0813ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa07c0cfad0 0x7fa07c154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa0813ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0d3750 0x7fa07c155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa0813ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c155f60 con 0x7fa07c0cfad0 2023-07-13T22:24:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa0813ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c154530 con 0x7fa07c0d3750 2023-07-13T22:24:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa0813ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1559a0 con 0x7fa07c0b8400 2023-07-13T22:24:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa07c0cfad0 0x7fa07c154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa07c0cfad0 0x7fa07c154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57532/0 (socket says 172.21.15.60:57532) 2023-07-13T22:24:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07a7fc700 1 -- 172.21.15.60:0/324527900 learned_addr learned my addr 172.21.15.60:0/324527900 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07affd700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0b8400 0x7fa07c0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07b7fe700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0d3750 0x7fa07c155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07a7fc700 1 -- 172.21.15.60:0/324527900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0b8400 msgr2=0x7fa07c0b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07a7fc700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0b8400 0x7fa07c0b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07a7fc700 1 -- 172.21.15.60:0/324527900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0d3750 msgr2=0x7fa07c155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:24.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07a7fc700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0d3750 0x7fa07c155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:24.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07a7fc700 1 -- 172.21.15.60:0/324527900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700094a0 con 0x7fa07c0cfad0 2023-07-13T22:24:24.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.842+0000 7fa07b7fe700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0d3750 0x7fa07c155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:24.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.843+0000 7fa07a7fc700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa07c0cfad0 0x7fa07c154e30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa07000e9b0 tx=0x7fa07000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:24.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.843+0000 7fa0637fe700 1 -- 172.21.15.60:0/324527900 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070009700 con 0x7fa07c0cfad0 2023-07-13T22:24:24.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.843+0000 7fa0637fe700 1 -- 172.21.15.60:0/324527900 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07000c850 con 0x7fa07c0cfad0 2023-07-13T22:24:24.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.843+0000 7fa0637fe700 1 -- 172.21.15.60:0/324527900 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07000cc00 con 0x7fa07c0cfad0 2023-07-13T22:24:24.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.843+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c0094a0 con 0x7fa07c0cfad0 2023-07-13T22:24:24.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.843+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c1562b0 con 0x7fa07c0cfad0 2023-07-13T22:24:24.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.844+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07c1560f0 con 0x7fa07c0cfad0 2023-07-13T22:24:24.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.845+0000 7fa0637fe700 1 -- 172.21.15.60:0/324527900 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa070013470 con 0x7fa07c0cfad0 2023-07-13T22:24:24.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.846+0000 7fa0637fe700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa06806ace0 0x7fa06806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:24.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.846+0000 7fa0637fe700 1 -- 172.21.15.60:0/324527900 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa070089210 con 0x7fa07c0cfad0 2023-07-13T22:24:24.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.846+0000 7fa07affd700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa06806ace0 0x7fa06806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.847+0000 7fa07affd700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa06806ace0 0x7fa06806d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0090b0 tx=0x7fa06c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:24.850+0000 7fa0637fe700 1 -- 172.21.15.60:0/324527900 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa070057cb0 con 0x7fa07c0cfad0 2023-07-13T22:24:25.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.051+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa07c0b5470 con 0x7fa06806ace0 2023-07-13T22:24:25.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.061+0000 7fa0637fe700 1 -- 172.21.15.60:0/324527900 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311483 (secure 0 0 0) 0x7fa06c011b80 con 0x7fa06806ace0 2023-07-13T22:24:25.047 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa06806ace0 msgr2=0x7fa06806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa06806ace0 0x7fa06806d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0090b0 tx=0x7fa06c01c040 comp rx=0 tx=0).stop 2023-07-13T22:24:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa07c0cfad0 msgr2=0x7fa07c154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa07c0cfad0 0x7fa07c154e30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa07000e9b0 tx=0x7fa07000ecc0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 shutdown_connections 2023-07-13T22:24:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa06806ace0 0x7fa06806d190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa07c0d3750 0x7fa07c155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa07c0b8400 0x7fa07c0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 --2- 172.21.15.60:0/324527900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa07c0cfad0 0x7fa07c154e30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.065+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 >> 172.21.15.60:0/324527900 conn(0x7fa07c06b390 msgr2=0x7fa07c0cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.066+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 shutdown_connections 2023-07-13T22:24:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.066+0000 7fa0813ff700 1 -- 172.21.15.60:0/324527900 wait complete. 2023-07-13T22:24:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:24:25.067 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-07-13T22:24:23.167324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255248,"kb_used_data":6864,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754719472,"statfs":{"total":773094113280,"available":772832739328,"internally_reserved":0,"allocated":7028736,"data_stored":3752928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.555703"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.671510+0000","last_change":"2023-07-13T22:24:20.669634+0000","last_active":"2023-07-13T22:24:20.671510+0000","last_peered":"2023-07-13T22:24:20.671510+0000","last_clean":"2023-07-13T22:24:20.671510+0000","last_became_active":"2023-07-13T22:24:20.669300+0000","last_became_peered":"2023-07-13T22:24:20.669300+0000","last_unstale":"2023-07-13T22:24:20.671510+0000","last_undegraded":"2023-07-13T22:24:20.671510+0000","last_fullsized":"2023-07-13T22:24:20.671510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:44.401182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.671423+0000","last_change":"2023-07-13T22:24:20.669097+0000","last_active":"2023-07-13T22:24:20.671423+0000","last_peered":"2023-07-13T22:24:20.671423+0000","last_clean":"2023-07-13T22:24:20.671423+0000","last_became_active":"2023-07-13T22:24:20.668747+0000","last_became_peered":"2023-07-13T22:24:20.668747+0000","last_unstale":"2023-07-13T22:24:20.671423+0000","last_undegraded":"2023-07-13T22:24:20.671423+0000","last_fullsized":"2023-07-13T22:24:20.671423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:03:49.921791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676591+0000","last_change":"2023-07-13T22:24:20.672092+0000","last_active":"2023-07-13T22:24:20.676591+0000","last_peered":"2023-07-13T22:24:20.676591+0000","last_clean":"2023-07-13T22:24:20.676591+0000","last_became_active":"2023-07-13T22:24:20.667557+0000","last_became_peered":"2023-07-13T22:24:20.667557+0000","last_unstale":"2023-07-13T22:24:20.676591+0000","last_undegraded":"2023-07-13T22:24:20.676591+0000","last_fullsized":"2023-07-13T22:24:20.676591+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:33:54.633734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675373+0000","last_change":"2023-07-13T22:24:20.659635+0000","last_active":"2023-07-13T22:24:20.675373+0000","last_peered":"2023-07-13T22:24:20.675373+0000","last_clean":"2023-07-13T22:24:20.675373+0000","last_became_active":"2023-07-13T22:24:20.659363+0000","last_became_peered":"2023-07-13T22:24:20.659363+0000","last_unstale":"2023-07-13T22:24:20.675373+0000","last_undegraded":"2023-07-13T22:24:20.675373+0000","last_fullsized":"2023-07-13T22:24:20.675373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:57:38.507855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676056+0000","last_change":"2023-07-13T22:24:20.674082+0000","last_active":"2023-07-13T22:24:20.676056+0000","last_peered":"2023-07-13T22:24:20.676056+0000","last_clean":"2023-07-13T22:24:20.676056+0000","last_became_active":"2023-07-13T22:24:20.673769+0000","last_became_peered":"2023-07-13T22:24:20.673769+0000","last_unstale":"2023-07-13T22:24:20.676056+0000","last_undegraded":"2023-07-13T22:24:20.676056+0000","last_fullsized":"2023-07-13T22:24:20.676056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:03:49.921791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675064+0000","last_change":"2023-07-13T22:24:20.673700+0000","last_active":"2023-07-13T22:24:20.675064+0000","last_peered":"2023-07-13T22:24:20.675064+0000","last_clean":"2023-07-13T22:24:20.675064+0000","last_became_active":"2023-07-13T22:24:20.673422+0000","last_became_peered":"2023-07-13T22:24:20.673422+0000","last_unstale":"2023-07-13T22:24:20.675064+0000","last_undegraded":"2023-07-13T22:24:20.675064+0000","last_fullsized":"2023-07-13T22:24:20.675064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:28.516245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664673+0000","last_change":"2023-07-13T22:24:20.660247+0000","last_active":"2023-07-13T22:24:20.664673+0000","last_peered":"2023-07-13T22:24:20.664673+0000","last_clean":"2023-07-13T22:24:20.664673+0000","last_became_active":"2023-07-13T22:24:20.659983+0000","last_became_peered":"2023-07-13T22:24:20.659983+0000","last_unstale":"2023-07-13T22:24:20.664673+0000","last_undegraded":"2023-07-13T22:24:20.664673+0000","last_fullsized":"2023-07-13T22:24:20.664673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:46:30.713669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.672771+0000","last_change":"2023-07-13T22:24:20.660171+0000","last_active":"2023-07-13T22:24:20.672771+0000","last_peered":"2023-07-13T22:24:20.672771+0000","last_clean":"2023-07-13T22:24:20.672771+0000","last_became_active":"2023-07-13T22:24:20.659867+0000","last_became_peered":"2023-07-13T22:24:20.659867+0000","last_unstale":"2023-07-13T22:24:20.672771+0000","last_undegraded":"2023-07-13T22:24:20.672771+0000","last_fullsized":"2023-07-13T22:24:20.672771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:21:41.046613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.679459+0000","last_change":"2023-07-13T22:24:20.675688+0000","last_active":"2023-07-13T22:24:20.679459+0000","last_peered":"2023-07-13T22:24:20.679459+0000","last_clean":"2023-07-13T22:24:20.679459+0000","last_became_active":"2023-07-13T22:24:20.675229+0000","last_became_peered":"2023-07-13T22:24:20.675229+0000","last_unstale":"2023-07-13T22:24:20.679459+0000","last_undegraded":"2023-07-13T22:24:20.679459+0000","last_fullsized":"2023-07-13T22:24:20.679459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:00:19.264666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676289+0000","last_change":"2023-07-13T22:24:20.670702+0000","last_active":"2023-07-13T22:24:20.676289+0000","last_peered":"2023-07-13T22:24:20.676289+0000","last_clean":"2023-07-13T22:24:20.676289+0000","last_became_active":"2023-07-13T22:24:20.670381+0000","last_became_peered":"2023-07-13T22:24:20.670381+0000","last_unstale":"2023-07-13T22:24:20.676289+0000","last_undegraded":"2023-07-13T22:24:20.676289+0000","last_fullsized":"2023-07-13T22:24:20.676289+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:13:42.472851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677840+0000","last_change":"2023-07-13T22:24:20.671721+0000","last_active":"2023-07-13T22:24:20.677840+0000","last_peered":"2023-07-13T22:24:20.677840+0000","last_clean":"2023-07-13T22:24:20.677840+0000","last_became_active":"2023-07-13T22:24:20.671382+0000","last_became_peered":"2023-07-13T22:24:20.671382+0000","last_unstale":"2023-07-13T22:24:20.677840+0000","last_undegraded":"2023-07-13T22:24:20.677840+0000","last_fullsized":"2023-07-13T22:24:20.677840+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:08:09.879746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673540+0000","last_change":"2023-07-13T22:24:20.661577+0000","last_active":"2023-07-13T22:24:20.673540+0000","last_peered":"2023-07-13T22:24:20.673540+0000","last_clean":"2023-07-13T22:24:20.673540+0000","last_became_active":"2023-07-13T22:24:20.661267+0000","last_became_peered":"2023-07-13T22:24:20.661267+0000","last_unstale":"2023-07-13T22:24:20.673540+0000","last_undegraded":"2023-07-13T22:24:20.673540+0000","last_fullsized":"2023-07-13T22:24:20.673540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:16.269860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.666703+0000","last_change":"2023-07-13T22:24:20.658980+0000","last_active":"2023-07-13T22:24:20.666703+0000","last_peered":"2023-07-13T22:24:20.666703+0000","last_clean":"2023-07-13T22:24:20.666703+0000","last_became_active":"2023-07-13T22:24:20.658680+0000","last_became_peered":"2023-07-13T22:24:20.658680+0000","last_unstale":"2023-07-13T22:24:20.666703+0000","last_undegraded":"2023-07-13T22:24:20.666703+0000","last_fullsized":"2023-07-13T22:24:20.666703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:01:08.735733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674430+0000","last_change":"2023-07-13T22:24:20.663756+0000","last_active":"2023-07-13T22:24:20.674430+0000","last_peered":"2023-07-13T22:24:20.674430+0000","last_clean":"2023-07-13T22:24:20.674430+0000","last_became_active":"2023-07-13T22:24:20.663423+0000","last_became_peered":"2023-07-13T22:24:20.663423+0000","last_unstale":"2023-07-13T22:24:20.674430+0000","last_undegraded":"2023-07-13T22:24:20.674430+0000","last_fullsized":"2023-07-13T22:24:20.674430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:53.689341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.678264+0000","last_change":"2023-07-13T22:24:20.674963+0000","last_active":"2023-07-13T22:24:20.678264+0000","last_peered":"2023-07-13T22:24:20.678264+0000","last_clean":"2023-07-13T22:24:20.678264+0000","last_became_active":"2023-07-13T22:24:20.674711+0000","last_became_peered":"2023-07-13T22:24:20.674711+0000","last_unstale":"2023-07-13T22:24:20.678264+0000","last_undegraded":"2023-07-13T22:24:20.678264+0000","last_fullsized":"2023-07-13T22:24:20.678264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:24:36.692640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.655782+0000","last_change":"2023-07-13T22:24:20.652882+0000","last_active":"2023-07-13T22:24:20.655782+0000","last_peered":"2023-07-13T22:24:20.655782+0000","last_clean":"2023-07-13T22:24:20.655782+0000","last_became_active":"2023-07-13T22:24:20.652579+0000","last_became_peered":"2023-07-13T22:24:20.652579+0000","last_unstale":"2023-07-13T22:24:20.655782+0000","last_undegraded":"2023-07-13T22:24:20.655782+0000","last_fullsized":"2023-07-13T22:24:20.655782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:16:54.497570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673481+0000","last_change":"2023-07-13T22:24:20.671527+0000","last_active":"2023-07-13T22:24:20.673481+0000","last_peered":"2023-07-13T22:24:20.673481+0000","last_clean":"2023-07-13T22:24:20.673481+0000","last_became_active":"2023-07-13T22:24:20.671256+0000","last_became_peered":"2023-07-13T22:24:20.671256+0000","last_unstale":"2023-07-13T22:24:20.673481+0000","last_undegraded":"2023-07-13T22:24:20.673481+0000","last_fullsized":"2023-07-13T22:24:20.673481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:36:21.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674916+0000","last_change":"2023-07-13T22:24:20.661021+0000","last_active":"2023-07-13T22:24:20.674916+0000","last_peered":"2023-07-13T22:24:20.674916+0000","last_clean":"2023-07-13T22:24:20.674916+0000","last_became_active":"2023-07-13T22:24:20.660628+0000","last_became_peered":"2023-07-13T22:24:20.660628+0000","last_unstale":"2023-07-13T22:24:20.674916+0000","last_undegraded":"2023-07-13T22:24:20.674916+0000","last_fullsized":"2023-07-13T22:24:20.674916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:28.516245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673744+0000","last_change":"2023-07-13T22:24:20.668034+0000","last_active":"2023-07-13T22:24:20.673744+0000","last_peered":"2023-07-13T22:24:20.673744+0000","last_clean":"2023-07-13T22:24:20.673744+0000","last_became_active":"2023-07-13T22:24:20.667711+0000","last_became_peered":"2023-07-13T22:24:20.667711+0000","last_unstale":"2023-07-13T22:24:20.673744+0000","last_undegraded":"2023-07-13T22:24:20.673744+0000","last_fullsized":"2023-07-13T22:24:20.673744+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:37:44.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673956+0000","last_change":"2023-07-13T22:24:20.669076+0000","last_active":"2023-07-13T22:24:20.673956+0000","last_peered":"2023-07-13T22:24:20.673956+0000","last_clean":"2023-07-13T22:24:20.673956+0000","last_became_active":"2023-07-13T22:24:20.668812+0000","last_became_peered":"2023-07-13T22:24:20.668812+0000","last_unstale":"2023-07-13T22:24:20.673956+0000","last_undegraded":"2023-07-13T22:24:20.673956+0000","last_fullsized":"2023-07-13T22:24:20.673956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:13:50.697622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673554+0000","last_change":"2023-07-13T22:24:20.665603+0000","last_active":"2023-07-13T22:24:20.673554+0000","last_peered":"2023-07-13T22:24:20.673554+0000","last_clean":"2023-07-13T22:24:20.673554+0000","last_became_active":"2023-07-13T22:24:20.665340+0000","last_became_peered":"2023-07-13T22:24:20.665340+0000","last_unstale":"2023-07-13T22:24:20.673554+0000","last_undegraded":"2023-07-13T22:24:20.673554+0000","last_fullsized":"2023-07-13T22:24:20.673554+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:46:36.043909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674886+0000","last_change":"2023-07-13T22:24:20.665186+0000","last_active":"2023-07-13T22:24:20.674886+0000","last_peered":"2023-07-13T22:24:20.674886+0000","last_clean":"2023-07-13T22:24:20.674886+0000","last_became_active":"2023-07-13T22:24:20.664898+0000","last_became_peered":"2023-07-13T22:24:20.664898+0000","last_unstale":"2023-07-13T22:24:20.674886+0000","last_undegraded":"2023-07-13T22:24:20.674886+0000","last_fullsized":"2023-07-13T22:24:20.674886+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:49:00.787861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664523+0000","last_change":"2023-07-13T22:24:20.663872+0000","last_active":"2023-07-13T22:24:20.664523+0000","last_peered":"2023-07-13T22:24:20.664523+0000","last_clean":"2023-07-13T22:24:20.664523+0000","last_became_active":"2023-07-13T22:24:20.663606+0000","last_became_peered":"2023-07-13T22:24:20.663606+0000","last_unstale":"2023-07-13T22:24:20.664523+0000","last_undegraded":"2023-07-13T22:24:20.664523+0000","last_fullsized":"2023-07-13T22:24:20.664523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:06:05.347740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676495+0000","last_change":"2023-07-13T22:24:20.671894+0000","last_active":"2023-07-13T22:24:20.676495+0000","last_peered":"2023-07-13T22:24:20.676495+0000","last_clean":"2023-07-13T22:24:20.676495+0000","last_became_active":"2023-07-13T22:24:20.669965+0000","last_became_peered":"2023-07-13T22:24:20.669965+0000","last_unstale":"2023-07-13T22:24:20.676495+0000","last_undegraded":"2023-07-13T22:24:20.676495+0000","last_fullsized":"2023-07-13T22:24:20.676495+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:06:17.569571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.660491+0000","last_change":"2023-07-13T22:24:20.655295+0000","last_active":"2023-07-13T22:24:20.660491+0000","last_peered":"2023-07-13T22:24:20.660491+0000","last_clean":"2023-07-13T22:24:20.660491+0000","last_became_active":"2023-07-13T22:24:20.654981+0000","last_became_peered":"2023-07-13T22:24:20.654981+0000","last_unstale":"2023-07-13T22:24:20.660491+0000","last_undegraded":"2023-07-13T22:24:20.660491+0000","last_fullsized":"2023-07-13T22:24:20.660491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:00:00.658423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.666790+0000","last_change":"2023-07-13T22:24:20.659246+0000","last_active":"2023-07-13T22:24:20.666790+0000","last_peered":"2023-07-13T22:24:20.666790+0000","last_clean":"2023-07-13T22:24:20.666790+0000","last_became_active":"2023-07-13T22:24:20.658269+0000","last_became_peered":"2023-07-13T22:24:20.658269+0000","last_unstale":"2023-07-13T22:24:20.666790+0000","last_undegraded":"2023-07-13T22:24:20.666790+0000","last_fullsized":"2023-07-13T22:24:20.666790+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:27:35.909871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.672033+0000","last_change":"2023-07-13T22:24:20.655579+0000","last_active":"2023-07-13T22:24:20.672033+0000","last_peered":"2023-07-13T22:24:20.672033+0000","last_clean":"2023-07-13T22:24:20.672033+0000","last_became_active":"2023-07-13T22:24:20.655319+0000","last_became_peered":"2023-07-13T22:24:20.655319+0000","last_unstale":"2023-07-13T22:24:20.672033+0000","last_undegraded":"2023-07-13T22:24:20.672033+0000","last_fullsized":"2023-07-13T22:24:20.672033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:55:25.391773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.672205+0000","last_change":"2023-07-13T22:24:20.669900+0000","last_active":"2023-07-13T22:24:20.672205+0000","last_peered":"2023-07-13T22:24:20.672205+0000","last_clean":"2023-07-13T22:24:20.672205+0000","last_became_active":"2023-07-13T22:24:20.669599+0000","last_became_peered":"2023-07-13T22:24:20.669599+0000","last_unstale":"2023-07-13T22:24:20.672205+0000","last_undegraded":"2023-07-13T22:24:20.672205+0000","last_fullsized":"2023-07-13T22:24:20.672205+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:35:03.770783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.679466+0000","last_change":"2023-07-13T22:24:20.675368+0000","last_active":"2023-07-13T22:24:20.679466+0000","last_peered":"2023-07-13T22:24:20.679466+0000","last_clean":"2023-07-13T22:24:20.679466+0000","last_became_active":"2023-07-13T22:24:20.671629+0000","last_became_peered":"2023-07-13T22:24:20.671629+0000","last_unstale":"2023-07-13T22:24:20.679466+0000","last_undegraded":"2023-07-13T22:24:20.679466+0000","last_fullsized":"2023-07-13T22:24:20.679466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:33:43.432649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674419+0000","last_change":"2023-07-13T22:24:20.664867+0000","last_active":"2023-07-13T22:24:20.674419+0000","last_peered":"2023-07-13T22:24:20.674419+0000","last_clean":"2023-07-13T22:24:20.674419+0000","last_became_active":"2023-07-13T22:24:20.664607+0000","last_became_peered":"2023-07-13T22:24:20.664607+0000","last_unstale":"2023-07-13T22:24:20.674419+0000","last_undegraded":"2023-07-13T22:24:20.674419+0000","last_fullsized":"2023-07-13T22:24:20.674419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:42:11.542303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665080+0000","last_change":"2023-07-13T22:24:20.652803+0000","last_active":"2023-07-13T22:24:20.665080+0000","last_peered":"2023-07-13T22:24:20.665080+0000","last_clean":"2023-07-13T22:24:20.665080+0000","last_became_active":"2023-07-13T22:24:20.652520+0000","last_became_peered":"2023-07-13T22:24:20.652520+0000","last_unstale":"2023-07-13T22:24:20.665080+0000","last_undegraded":"2023-07-13T22:24:20.665080+0000","last_fullsized":"2023-07-13T22:24:20.665080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:16.269860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665730+0000","last_change":"2023-07-13T22:24:20.660273+0000","last_active":"2023-07-13T22:24:20.665730+0000","last_peered":"2023-07-13T22:24:20.665730+0000","last_clean":"2023-07-13T22:24:20.665730+0000","last_became_active":"2023-07-13T22:24:20.659998+0000","last_became_peered":"2023-07-13T22:24:20.659998+0000","last_unstale":"2023-07-13T22:24:20.665730+0000","last_undegraded":"2023-07-13T22:24:20.665730+0000","last_fullsized":"2023-07-13T22:24:20.665730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:20:12.098671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675964+0000","last_change":"2023-07-13T22:24:20.672767+0000","last_active":"2023-07-13T22:24:20.675964+0000","last_peered":"2023-07-13T22:24:20.675964+0000","last_clean":"2023-07-13T22:24:20.675964+0000","last_became_active":"2023-07-13T22:24:20.670750+0000","last_became_peered":"2023-07-13T22:24:20.670750+0000","last_unstale":"2023-07-13T22:24:20.675964+0000","last_undegraded":"2023-07-13T22:24:20.675964+0000","last_fullsized":"2023-07-13T22:24:20.675964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:28:44.111936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677510+0000","last_change":"2023-07-13T22:24:19.634703+0000","last_active":"2023-07-13T22:24:20.677510+0000","last_peered":"2023-07-13T22:24:20.677510+0000","last_clean":"2023-07-13T22:24:20.677510+0000","last_became_active":"2023-07-13T22:24:19.634448+0000","last_became_peered":"2023-07-13T22:24:19.634448+0000","last_unstale":"2023-07-13T22:24:20.677510+0000","last_undegraded":"2023-07-13T22:24:20.677510+0000","last_fullsized":"2023-07-13T22:24:20.677510+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:45.573094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677818+0000","last_change":"2023-07-13T22:24:20.669989+0000","last_active":"2023-07-13T22:24:20.677818+0000","last_peered":"2023-07-13T22:24:20.677818+0000","last_clean":"2023-07-13T22:24:20.677818+0000","last_became_active":"2023-07-13T22:24:20.669649+0000","last_became_peered":"2023-07-13T22:24:20.669649+0000","last_unstale":"2023-07-13T22:24:20.677818+0000","last_undegraded":"2023-07-13T22:24:20.677818+0000","last_fullsized":"2023-07-13T22:24:20.677818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:23:28.971371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676936+0000","last_change":"2023-07-13T22:24:19.637226+0000","last_active":"2023-07-13T22:24:20.676936+0000","last_peered":"2023-07-13T22:24:20.676936+0000","last_clean":"2023-07-13T22:24:20.676936+0000","last_became_active":"2023-07-13T22:24:19.636945+0000","last_became_peered":"2023-07-13T22:24:19.636945+0000","last_unstale":"2023-07-13T22:24:20.676936+0000","last_undegraded":"2023-07-13T22:24:20.676936+0000","last_fullsized":"2023-07-13T22:24:20.676936+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:28:56.535031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.666279+0000","last_change":"2023-07-13T22:24:20.658243+0000","last_active":"2023-07-13T22:24:20.666279+0000","last_peered":"2023-07-13T22:24:20.666279+0000","last_clean":"2023-07-13T22:24:20.666279+0000","last_became_active":"2023-07-13T22:24:20.657989+0000","last_became_peered":"2023-07-13T22:24:20.657989+0000","last_unstale":"2023-07-13T22:24:20.666279+0000","last_undegraded":"2023-07-13T22:24:20.666279+0000","last_fullsized":"2023-07-13T22:24:20.666279+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:29:33.426585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675296+0000","last_change":"2023-07-13T22:24:19.638013+0000","last_active":"2023-07-13T22:24:20.675296+0000","last_peered":"2023-07-13T22:24:20.675296+0000","last_clean":"2023-07-13T22:24:20.675296+0000","last_became_active":"2023-07-13T22:24:19.637713+0000","last_became_peered":"2023-07-13T22:24:19.637713+0000","last_unstale":"2023-07-13T22:24:20.675296+0000","last_undegraded":"2023-07-13T22:24:20.675296+0000","last_fullsized":"2023-07-13T22:24:20.675296+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:03:54.517507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676197+0000","last_change":"2023-07-13T22:24:20.670847+0000","last_active":"2023-07-13T22:24:20.676197+0000","last_peered":"2023-07-13T22:24:20.676197+0000","last_clean":"2023-07-13T22:24:20.676197+0000","last_became_active":"2023-07-13T22:24:20.670534+0000","last_became_peered":"2023-07-13T22:24:20.670534+0000","last_unstale":"2023-07-13T22:24:20.676197+0000","last_undegraded":"2023-07-13T22:24:20.676197+0000","last_fullsized":"2023-07-13T22:24:20.676197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:01:57.690180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.672811+0000","last_change":"2023-07-13T22:24:19.633596+0000","last_active":"2023-07-13T22:24:20.672811+0000","last_peered":"2023-07-13T22:24:20.672811+0000","last_clean":"2023-07-13T22:24:20.672811+0000","last_became_active":"2023-07-13T22:24:19.633317+0000","last_became_peered":"2023-07-13T22:24:19.633317+0000","last_unstale":"2023-07-13T22:24:20.672811+0000","last_undegraded":"2023-07-13T22:24:20.672811+0000","last_fullsized":"2023-07-13T22:24:20.672811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:46:18.176621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664690+0000","last_change":"2023-07-13T22:24:20.662669+0000","last_active":"2023-07-13T22:24:20.664690+0000","last_peered":"2023-07-13T22:24:20.664690+0000","last_clean":"2023-07-13T22:24:20.664690+0000","last_became_active":"2023-07-13T22:24:20.662396+0000","last_became_peered":"2023-07-13T22:24:20.662396+0000","last_unstale":"2023-07-13T22:24:20.664690+0000","last_undegraded":"2023-07-13T22:24:20.664690+0000","last_fullsized":"2023-07-13T22:24:20.664690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:12:06.305028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.668020+0000","last_change":"2023-07-13T22:24:19.635730+0000","last_active":"2023-07-13T22:24:20.668020+0000","last_peered":"2023-07-13T22:24:20.668020+0000","last_clean":"2023-07-13T22:24:20.668020+0000","last_became_active":"2023-07-13T22:24:19.634831+0000","last_became_peered":"2023-07-13T22:24:19.634831+0000","last_unstale":"2023-07-13T22:24:20.668020+0000","last_undegraded":"2023-07-13T22:24:20.668020+0000","last_fullsized":"2023-07-13T22:24:20.668020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:27:37.954922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673420+0000","last_change":"2023-07-13T22:24:20.671314+0000","last_active":"2023-07-13T22:24:20.673420+0000","last_peered":"2023-07-13T22:24:20.673420+0000","last_clean":"2023-07-13T22:24:20.673420+0000","last_became_active":"2023-07-13T22:24:20.671050+0000","last_became_peered":"2023-07-13T22:24:20.671050+0000","last_unstale":"2023-07-13T22:24:20.673420+0000","last_undegraded":"2023-07-13T22:24:20.673420+0000","last_fullsized":"2023-07-13T22:24:20.673420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:18:33.656293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665625+0000","last_change":"2023-07-13T22:24:19.631973+0000","last_active":"2023-07-13T22:24:20.665625+0000","last_peered":"2023-07-13T22:24:20.665625+0000","last_clean":"2023-07-13T22:24:20.665625+0000","last_became_active":"2023-07-13T22:24:19.631250+0000","last_became_peered":"2023-07-13T22:24:19.631250+0000","last_unstale":"2023-07-13T22:24:20.665625+0000","last_undegraded":"2023-07-13T22:24:20.665625+0000","last_fullsized":"2023-07-13T22:24:20.665625+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:34:34.376728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.668939+0000","last_change":"2023-07-13T22:24:20.663336+0000","last_active":"2023-07-13T22:24:20.668939+0000","last_peered":"2023-07-13T22:24:20.668939+0000","last_clean":"2023-07-13T22:24:20.668939+0000","last_became_active":"2023-07-13T22:24:20.663010+0000","last_became_peered":"2023-07-13T22:24:20.663010+0000","last_unstale":"2023-07-13T22:24:20.668939+0000","last_undegraded":"2023-07-13T22:24:20.668939+0000","last_fullsized":"2023-07-13T22:24:20.668939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:06.435575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647121+0000","last_change":"2023-07-13T22:24:19.639911+0000","last_active":"2023-07-13T22:24:20.647121+0000","last_peered":"2023-07-13T22:24:20.647121+0000","last_clean":"2023-07-13T22:24:20.647121+0000","last_became_active":"2023-07-13T22:24:19.639573+0000","last_became_peered":"2023-07-13T22:24:19.639573+0000","last_unstale":"2023-07-13T22:24:20.647121+0000","last_undegraded":"2023-07-13T22:24:20.647121+0000","last_fullsized":"2023-07-13T22:24:20.647121+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:06:45.241977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.662722+0000","last_change":"2023-07-13T22:24:20.657551+0000","last_active":"2023-07-13T22:24:20.662722+0000","last_peered":"2023-07-13T22:24:20.662722+0000","last_clean":"2023-07-13T22:24:20.662722+0000","last_became_active":"2023-07-13T22:24:20.657243+0000","last_became_peered":"2023-07-13T22:24:20.657243+0000","last_unstale":"2023-07-13T22:24:20.662722+0000","last_undegraded":"2023-07-13T22:24:20.662722+0000","last_fullsized":"2023-07-13T22:24:20.662722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:16:30.944384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.658174+0000","last_change":"2023-07-13T22:24:19.635643+0000","last_active":"2023-07-13T22:24:20.658174+0000","last_peered":"2023-07-13T22:24:20.658174+0000","last_clean":"2023-07-13T22:24:20.658174+0000","last_became_active":"2023-07-13T22:24:19.635311+0000","last_became_peered":"2023-07-13T22:24:19.635311+0000","last_unstale":"2023-07-13T22:24:20.658174+0000","last_undegraded":"2023-07-13T22:24:20.658174+0000","last_fullsized":"2023-07-13T22:24:20.658174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:18:32.653836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.669478+0000","last_change":"2023-07-13T22:24:20.661322+0000","last_active":"2023-07-13T22:24:20.669478+0000","last_peered":"2023-07-13T22:24:20.669478+0000","last_clean":"2023-07-13T22:24:20.669478+0000","last_became_active":"2023-07-13T22:24:20.660938+0000","last_became_peered":"2023-07-13T22:24:20.660938+0000","last_unstale":"2023-07-13T22:24:20.669478+0000","last_undegraded":"2023-07-13T22:24:20.669478+0000","last_fullsized":"2023-07-13T22:24:20.669478+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:40:42.360427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.654024+0000","last_change":"2023-07-13T22:24:19.641304+0000","last_active":"2023-07-13T22:24:20.654024+0000","last_peered":"2023-07-13T22:24:20.654024+0000","last_clean":"2023-07-13T22:24:20.654024+0000","last_became_active":"2023-07-13T22:24:19.641040+0000","last_became_peered":"2023-07-13T22:24:19.641040+0000","last_unstale":"2023-07-13T22:24:20.654024+0000","last_undegraded":"2023-07-13T22:24:20.654024+0000","last_fullsized":"2023-07-13T22:24:20.654024+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:44:38.970375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.662137+0000","last_change":"2023-07-13T22:24:20.652838+0000","last_active":"2023-07-13T22:24:20.662137+0000","last_peered":"2023-07-13T22:24:20.662137+0000","last_clean":"2023-07-13T22:24:20.662137+0000","last_became_active":"2023-07-13T22:24:20.652541+0000","last_became_peered":"2023-07-13T22:24:20.652541+0000","last_unstale":"2023-07-13T22:24:20.662137+0000","last_undegraded":"2023-07-13T22:24:20.662137+0000","last_fullsized":"2023-07-13T22:24:20.662137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:58:45.677101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647667+0000","last_change":"2023-07-13T22:24:19.632268+0000","last_active":"2023-07-13T22:24:20.647667+0000","last_peered":"2023-07-13T22:24:20.647667+0000","last_clean":"2023-07-13T22:24:20.647667+0000","last_became_active":"2023-07-13T22:24:19.631926+0000","last_became_peered":"2023-07-13T22:24:19.631926+0000","last_unstale":"2023-07-13T22:24:20.647667+0000","last_undegraded":"2023-07-13T22:24:20.647667+0000","last_fullsized":"2023-07-13T22:24:20.647667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:00:51.354498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675230+0000","last_change":"2023-07-13T22:24:20.669725+0000","last_active":"2023-07-13T22:24:20.675230+0000","last_peered":"2023-07-13T22:24:20.675230+0000","last_clean":"2023-07-13T22:24:20.675230+0000","last_became_active":"2023-07-13T22:24:20.668270+0000","last_became_peered":"2023-07-13T22:24:20.668270+0000","last_unstale":"2023-07-13T22:24:20.675230+0000","last_undegraded":"2023-07-13T22:24:20.675230+0000","last_fullsized":"2023-07-13T22:24:20.675230+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:38:36.818193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.682300+0000","last_change":"2023-07-13T22:24:19.633522+0000","last_active":"2023-07-13T22:24:20.682300+0000","last_peered":"2023-07-13T22:24:20.682300+0000","last_clean":"2023-07-13T22:24:20.682300+0000","last_became_active":"2023-07-13T22:24:19.632838+0000","last_became_peered":"2023-07-13T22:24:19.632838+0000","last_unstale":"2023-07-13T22:24:20.682300+0000","last_undegraded":"2023-07-13T22:24:20.682300+0000","last_fullsized":"2023-07-13T22:24:20.682300+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:50:39.953564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673028+0000","last_change":"2023-07-13T22:24:20.654424+0000","last_active":"2023-07-13T22:24:20.673028+0000","last_peered":"2023-07-13T22:24:20.673028+0000","last_clean":"2023-07-13T22:24:20.673028+0000","last_became_active":"2023-07-13T22:24:20.653416+0000","last_became_peered":"2023-07-13T22:24:20.653416+0000","last_unstale":"2023-07-13T22:24:20.673028+0000","last_undegraded":"2023-07-13T22:24:20.673028+0000","last_fullsized":"2023-07-13T22:24:20.673028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:03.725412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.685245+0000","last_change":"2023-07-13T22:24:19.641018+0000","last_active":"2023-07-13T22:24:20.685245+0000","last_peered":"2023-07-13T22:24:20.685245+0000","last_clean":"2023-07-13T22:24:20.685245+0000","last_became_active":"2023-07-13T22:24:19.640570+0000","last_became_peered":"2023-07-13T22:24:19.640570+0000","last_unstale":"2023-07-13T22:24:20.685245+0000","last_undegraded":"2023-07-13T22:24:20.685245+0000","last_fullsized":"2023-07-13T22:24:20.685245+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:52:53.361933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.670679+0000","last_change":"2023-07-13T22:24:20.654840+0000","last_active":"2023-07-13T22:24:20.670679+0000","last_peered":"2023-07-13T22:24:20.670679+0000","last_clean":"2023-07-13T22:24:20.670679+0000","last_became_active":"2023-07-13T22:24:20.653388+0000","last_became_peered":"2023-07-13T22:24:20.653388+0000","last_unstale":"2023-07-13T22:24:20.670679+0000","last_undegraded":"2023-07-13T22:24:20.670679+0000","last_fullsized":"2023-07-13T22:24:20.670679+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:43:58.659929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.678197+0000","last_change":"2023-07-13T22:24:19.638081+0000","last_active":"2023-07-13T22:24:20.678197+0000","last_peered":"2023-07-13T22:24:20.678197+0000","last_clean":"2023-07-13T22:24:20.678197+0000","last_became_active":"2023-07-13T22:24:19.637797+0000","last_became_peered":"2023-07-13T22:24:19.637797+0000","last_unstale":"2023-07-13T22:24:20.678197+0000","last_undegraded":"2023-07-13T22:24:20.678197+0000","last_fullsized":"2023-07-13T22:24:20.678197+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:34:34.376728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674383+0000","last_change":"2023-07-13T22:24:20.663741+0000","last_active":"2023-07-13T22:24:20.674383+0000","last_peered":"2023-07-13T22:24:20.674383+0000","last_clean":"2023-07-13T22:24:20.674383+0000","last_became_active":"2023-07-13T22:24:20.663412+0000","last_became_peered":"2023-07-13T22:24:20.663412+0000","last_unstale":"2023-07-13T22:24:20.674383+0000","last_undegraded":"2023-07-13T22:24:20.674383+0000","last_fullsized":"2023-07-13T22:24:20.674383+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:37:36.477800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646138+0000","last_change":"2023-07-13T22:24:19.632013+0000","last_active":"2023-07-13T22:24:20.646138+0000","last_peered":"2023-07-13T22:24:20.646138+0000","last_clean":"2023-07-13T22:24:20.646138+0000","last_became_active":"2023-07-13T22:24:19.631753+0000","last_became_peered":"2023-07-13T22:24:19.631753+0000","last_unstale":"2023-07-13T22:24:20.646138+0000","last_undegraded":"2023-07-13T22:24:20.646138+0000","last_fullsized":"2023-07-13T22:24:20.646138+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:16.863810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.671691+0000","last_change":"2023-07-13T22:24:20.666281+0000","last_active":"2023-07-13T22:24:20.671691+0000","last_peered":"2023-07-13T22:24:20.671691+0000","last_clean":"2023-07-13T22:24:20.671691+0000","last_became_active":"2023-07-13T22:24:20.665958+0000","last_became_peered":"2023-07-13T22:24:20.665958+0000","last_unstale":"2023-07-13T22:24:20.671691+0000","last_undegraded":"2023-07-13T22:24:20.671691+0000","last_fullsized":"2023-07-13T22:24:20.671691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:35:03.770783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651218+0000","last_change":"2023-07-13T22:24:19.634708+0000","last_active":"2023-07-13T22:24:20.651218+0000","last_peered":"2023-07-13T22:24:20.651218+0000","last_clean":"2023-07-13T22:24:20.651218+0000","last_became_active":"2023-07-13T22:24:19.634441+0000","last_became_peered":"2023-07-13T22:24:19.634441+0000","last_unstale":"2023-07-13T22:24:20.651218+0000","last_undegraded":"2023-07-13T22:24:20.651218+0000","last_fullsized":"2023-07-13T22:24:20.651218+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:54:26.521094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.670353+0000","last_change":"2023-07-13T22:24:20.666631+0000","last_active":"2023-07-13T22:24:20.670353+0000","last_peered":"2023-07-13T22:24:20.670353+0000","last_clean":"2023-07-13T22:24:20.670353+0000","last_became_active":"2023-07-13T22:24:20.666340+0000","last_became_peered":"2023-07-13T22:24:20.666340+0000","last_unstale":"2023-07-13T22:24:20.670353+0000","last_undegraded":"2023-07-13T22:24:20.670353+0000","last_fullsized":"2023-07-13T22:24:20.670353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:06.435575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.678103+0000","last_change":"2023-07-13T22:24:19.636732+0000","last_active":"2023-07-13T22:24:20.678103+0000","last_peered":"2023-07-13T22:24:20.678103+0000","last_clean":"2023-07-13T22:24:20.678103+0000","last_became_active":"2023-07-13T22:24:19.636424+0000","last_became_peered":"2023-07-13T22:24:19.636424+0000","last_unstale":"2023-07-13T22:24:20.678103+0000","last_undegraded":"2023-07-13T22:24:20.678103+0000","last_fullsized":"2023-07-13T22:24:20.678103+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:25:01.024244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665791+0000","last_change":"2023-07-13T22:24:20.658997+0000","last_active":"2023-07-13T22:24:20.665791+0000","last_peered":"2023-07-13T22:24:20.665791+0000","last_clean":"2023-07-13T22:24:20.665791+0000","last_became_active":"2023-07-13T22:24:20.658720+0000","last_became_peered":"2023-07-13T22:24:20.658720+0000","last_unstale":"2023-07-13T22:24:20.665791+0000","last_undegraded":"2023-07-13T22:24:20.665791+0000","last_fullsized":"2023-07-13T22:24:20.665791+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:22:23.276499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677126+0000","last_change":"2023-07-13T22:24:19.633257+0000","last_active":"2023-07-13T22:24:20.677126+0000","last_peered":"2023-07-13T22:24:20.677126+0000","last_clean":"2023-07-13T22:24:20.677126+0000","last_became_active":"2023-07-13T22:24:19.632974+0000","last_became_peered":"2023-07-13T22:24:19.632974+0000","last_unstale":"2023-07-13T22:24:20.677126+0000","last_undegraded":"2023-07-13T22:24:20.677126+0000","last_fullsized":"2023-07-13T22:24:20.677126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:23:27.968915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664463+0000","last_change":"2023-07-13T22:24:20.659660+0000","last_active":"2023-07-13T22:24:20.664463+0000","last_peered":"2023-07-13T22:24:20.664463+0000","last_clean":"2023-07-13T22:24:20.664463+0000","last_became_active":"2023-07-13T22:24:20.659331+0000","last_became_peered":"2023-07-13T22:24:20.659331+0000","last_unstale":"2023-07-13T22:24:20.664463+0000","last_undegraded":"2023-07-13T22:24:20.664463+0000","last_fullsized":"2023-07-13T22:24:20.664463+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:35:26.938575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647190+0000","last_change":"2023-07-13T22:24:19.633234+0000","last_active":"2023-07-13T22:24:20.647190+0000","last_peered":"2023-07-13T22:24:20.647190+0000","last_clean":"2023-07-13T22:24:20.647190+0000","last_became_active":"2023-07-13T22:24:19.632900+0000","last_became_peered":"2023-07-13T22:24:19.632900+0000","last_unstale":"2023-07-13T22:24:20.647190+0000","last_undegraded":"2023-07-13T22:24:20.647190+0000","last_fullsized":"2023-07-13T22:24:20.647190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:57:37.505398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674722+0000","last_change":"2023-07-13T22:24:20.660619+0000","last_active":"2023-07-13T22:24:20.674722+0000","last_peered":"2023-07-13T22:24:20.674722+0000","last_clean":"2023-07-13T22:24:20.674722+0000","last_became_active":"2023-07-13T22:24:20.660361+0000","last_became_peered":"2023-07-13T22:24:20.660361+0000","last_unstale":"2023-07-13T22:24:20.674722+0000","last_undegraded":"2023-07-13T22:24:20.674722+0000","last_fullsized":"2023-07-13T22:24:20.674722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:04:29.444634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651212+0000","last_change":"2023-07-13T22:24:19.637527+0000","last_active":"2023-07-13T22:24:20.651212+0000","last_peered":"2023-07-13T22:24:20.651212+0000","last_clean":"2023-07-13T22:24:20.651212+0000","last_became_active":"2023-07-13T22:24:19.637206+0000","last_became_peered":"2023-07-13T22:24:19.637206+0000","last_unstale":"2023-07-13T22:24:20.651212+0000","last_undegraded":"2023-07-13T22:24:20.651212+0000","last_fullsized":"2023-07-13T22:24:20.651212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:32:56.744052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675163+0000","last_change":"2023-07-13T22:24:20.669324+0000","last_active":"2023-07-13T22:24:20.675163+0000","last_peered":"2023-07-13T22:24:20.675163+0000","last_clean":"2023-07-13T22:24:20.675163+0000","last_became_active":"2023-07-13T22:24:20.668923+0000","last_became_peered":"2023-07-13T22:24:20.668923+0000","last_unstale":"2023-07-13T22:24:20.675163+0000","last_undegraded":"2023-07-13T22:24:20.675163+0000","last_fullsized":"2023-07-13T22:24:20.675163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:27:40.319250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651705+0000","last_change":"2023-07-13T22:24:19.641241+0000","last_active":"2023-07-13T22:24:20.651705+0000","last_peered":"2023-07-13T22:24:20.651705+0000","last_clean":"2023-07-13T22:24:20.651705+0000","last_became_active":"2023-07-13T22:24:19.640989+0000","last_became_peered":"2023-07-13T22:24:19.640989+0000","last_unstale":"2023-07-13T22:24:20.651705+0000","last_undegraded":"2023-07-13T22:24:20.651705+0000","last_fullsized":"2023-07-13T22:24:20.651705+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:00:27.927042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675128+0000","last_change":"2023-07-13T22:24:20.671011+0000","last_active":"2023-07-13T22:24:20.675128+0000","last_peered":"2023-07-13T22:24:20.675128+0000","last_clean":"2023-07-13T22:24:20.675128+0000","last_became_active":"2023-07-13T22:24:20.670725+0000","last_became_peered":"2023-07-13T22:24:20.670725+0000","last_unstale":"2023-07-13T22:24:20.675128+0000","last_undegraded":"2023-07-13T22:24:20.675128+0000","last_fullsized":"2023-07-13T22:24:20.675128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:05:24.951741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.652377+0000","last_change":"2023-07-13T22:24:19.634141+0000","last_active":"2023-07-13T22:24:20.652377+0000","last_peered":"2023-07-13T22:24:20.652377+0000","last_clean":"2023-07-13T22:24:20.652377+0000","last_became_active":"2023-07-13T22:24:19.633831+0000","last_became_peered":"2023-07-13T22:24:19.633831+0000","last_unstale":"2023-07-13T22:24:20.652377+0000","last_undegraded":"2023-07-13T22:24:20.652377+0000","last_fullsized":"2023-07-13T22:24:20.652377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:10:27.068619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.670645+0000","last_change":"2023-07-13T22:24:20.656557+0000","last_active":"2023-07-13T22:24:20.670645+0000","last_peered":"2023-07-13T22:24:20.670645+0000","last_clean":"2023-07-13T22:24:20.670645+0000","last_became_active":"2023-07-13T22:24:20.656290+0000","last_became_peered":"2023-07-13T22:24:20.656290+0000","last_unstale":"2023-07-13T22:24:20.670645+0000","last_undegraded":"2023-07-13T22:24:20.670645+0000","last_fullsized":"2023-07-13T22:24:20.670645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:17.866267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647850+0000","last_change":"2023-07-13T22:24:19.631443+0000","last_active":"2023-07-13T22:24:20.647850+0000","last_peered":"2023-07-13T22:24:20.647850+0000","last_clean":"2023-07-13T22:24:20.647850+0000","last_became_active":"2023-07-13T22:24:19.631131+0000","last_became_peered":"2023-07-13T22:24:19.631131+0000","last_unstale":"2023-07-13T22:24:20.647850+0000","last_undegraded":"2023-07-13T22:24:20.647850+0000","last_fullsized":"2023-07-13T22:24:20.647850+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:33:42.430192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673145+0000","last_change":"2023-07-13T22:24:20.670685+0000","last_active":"2023-07-13T22:24:20.673145+0000","last_peered":"2023-07-13T22:24:20.673145+0000","last_clean":"2023-07-13T22:24:20.673145+0000","last_became_active":"2023-07-13T22:24:20.670340+0000","last_became_peered":"2023-07-13T22:24:20.670340+0000","last_unstale":"2023-07-13T22:24:20.673145+0000","last_undegraded":"2023-07-13T22:24:20.673145+0000","last_fullsized":"2023-07-13T22:24:20.673145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:09:41.848324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647340+0000","last_change":"2023-07-13T22:24:19.639638+0000","last_active":"2023-07-13T22:24:20.647340+0000","last_peered":"2023-07-13T22:24:20.647340+0000","last_clean":"2023-07-13T22:24:20.647340+0000","last_became_active":"2023-07-13T22:24:19.639258+0000","last_became_peered":"2023-07-13T22:24:19.639258+0000","last_unstale":"2023-07-13T22:24:20.647340+0000","last_undegraded":"2023-07-13T22:24:20.647340+0000","last_fullsized":"2023-07-13T22:24:20.647340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:11:17.164796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.656256+0000","last_change":"2023-07-13T22:24:20.652957+0000","last_active":"2023-07-13T22:24:20.656256+0000","last_peered":"2023-07-13T22:24:20.656256+0000","last_clean":"2023-07-13T22:24:20.656256+0000","last_became_active":"2023-07-13T22:24:20.652586+0000","last_became_peered":"2023-07-13T22:24:20.652586+0000","last_unstale":"2023-07-13T22:24:20.656256+0000","last_undegraded":"2023-07-13T22:24:20.656256+0000","last_fullsized":"2023-07-13T22:24:20.656256+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:59:12.441359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651382+0000","last_change":"2023-07-13T22:24:19.638493+0000","last_active":"2023-07-13T22:24:20.651382+0000","last_peered":"2023-07-13T22:24:20.651382+0000","last_clean":"2023-07-13T22:24:20.651382+0000","last_became_active":"2023-07-13T22:24:19.638176+0000","last_became_peered":"2023-07-13T22:24:19.638176+0000","last_unstale":"2023-07-13T22:24:20.651382+0000","last_undegraded":"2023-07-13T22:24:20.651382+0000","last_fullsized":"2023-07-13T22:24:20.651382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:34:49.979804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.678232+0000","last_change":"2023-07-13T22:24:20.671891+0000","last_active":"2023-07-13T22:24:20.678232+0000","last_peered":"2023-07-13T22:24:20.678232+0000","last_clean":"2023-07-13T22:24:20.678232+0000","last_became_active":"2023-07-13T22:24:20.671537+0000","last_became_peered":"2023-07-13T22:24:20.671537+0000","last_unstale":"2023-07-13T22:24:20.678232+0000","last_undegraded":"2023-07-13T22:24:20.678232+0000","last_fullsized":"2023-07-13T22:24:20.678232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:19:16.072624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.682396+0000","last_change":"2023-07-13T22:24:19.633687+0000","last_active":"2023-07-13T22:24:20.682396+0000","last_peered":"2023-07-13T22:24:20.682396+0000","last_clean":"2023-07-13T22:24:20.682396+0000","last_became_active":"2023-07-13T22:24:19.633257+0000","last_became_peered":"2023-07-13T22:24:19.633257+0000","last_unstale":"2023-07-13T22:24:20.682396+0000","last_undegraded":"2023-07-13T22:24:20.682396+0000","last_fullsized":"2023-07-13T22:24:20.682396+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:25:05.194710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":90,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646956+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:24:20.646956+0000","last_peered":"2023-07-13T22:24:20.646956+0000","last_clean":"2023-07-13T22:24:20.646956+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:24:20.646956+0000","last_undegraded":"2023-07-13T22:24:20.646956+0000","last_fullsized":"2023-07-13T22:24:20.646956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:16:39.194415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675182+0000","last_change":"2023-07-13T22:24:20.670008+0000","last_active":"2023-07-13T22:24:20.675182+0000","last_peered":"2023-07-13T22:24:20.675182+0000","last_clean":"2023-07-13T22:24:20.675182+0000","last_became_active":"2023-07-13T22:24:20.669670+0000","last_became_peered":"2023-07-13T22:24:20.669670+0000","last_unstale":"2023-07-13T22:24:20.675182+0000","last_undegraded":"2023-07-13T22:24:20.675182+0000","last_fullsized":"2023-07-13T22:24:20.675182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:40:42.360427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646831+0000","last_change":"2023-07-13T22:24:19.637537+0000","last_active":"2023-07-13T22:24:20.646831+0000","last_peered":"2023-07-13T22:24:20.646831+0000","last_clean":"2023-07-13T22:24:20.646831+0000","last_became_active":"2023-07-13T22:24:19.637204+0000","last_became_peered":"2023-07-13T22:24:19.637204+0000","last_unstale":"2023-07-13T22:24:20.646831+0000","last_undegraded":"2023-07-13T22:24:20.646831+0000","last_fullsized":"2023-07-13T22:24:20.646831+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:48:04.706795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646919+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:24:20.646919+0000","last_peered":"2023-07-13T22:24:20.646919+0000","last_clean":"2023-07-13T22:24:20.646919+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:24:20.646919+0000","last_undegraded":"2023-07-13T22:24:20.646919+0000","last_fullsized":"2023-07-13T22:24:20.646919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:29:12.003391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665703+0000","last_change":"2023-07-13T22:24:20.659090+0000","last_active":"2023-07-13T22:24:20.665703+0000","last_peered":"2023-07-13T22:24:20.665703+0000","last_clean":"2023-07-13T22:24:20.665703+0000","last_became_active":"2023-07-13T22:24:20.658783+0000","last_became_peered":"2023-07-13T22:24:20.658783+0000","last_unstale":"2023-07-13T22:24:20.665703+0000","last_undegraded":"2023-07-13T22:24:20.665703+0000","last_fullsized":"2023-07-13T22:24:20.665703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:55:25.391773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675803+0000","last_change":"2023-07-13T22:24:19.638375+0000","last_active":"2023-07-13T22:24:20.675803+0000","last_peered":"2023-07-13T22:24:20.675803+0000","last_clean":"2023-07-13T22:24:20.675803+0000","last_became_active":"2023-07-13T22:24:19.638041+0000","last_became_peered":"2023-07-13T22:24:19.638041+0000","last_unstale":"2023-07-13T22:24:20.675803+0000","last_undegraded":"2023-07-13T22:24:20.675803+0000","last_fullsized":"2023-07-13T22:24:20.675803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:00:51.354498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":92,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.650625+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:24:20.650625+0000","last_peered":"2023-07-13T22:24:20.650625+0000","last_clean":"2023-07-13T22:24:20.650625+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:24:20.650625+0000","last_undegraded":"2023-07-13T22:24:20.650625+0000","last_fullsized":"2023-07-13T22:24:20.650625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:22.528910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":236,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.650704+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:24:20.650704+0000","last_peered":"2023-07-13T22:24:20.650704+0000","last_clean":"2023-07-13T22:24:20.650704+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:24:20.650704+0000","last_undegraded":"2023-07-13T22:24:20.650704+0000","last_fullsized":"2023-07-13T22:24:20.650704+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:55:13.775565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664984+0000","last_change":"2023-07-13T22:24:20.656270+0000","last_active":"2023-07-13T22:24:20.664984+0000","last_peered":"2023-07-13T22:24:20.664984+0000","last_clean":"2023-07-13T22:24:20.664984+0000","last_became_active":"2023-07-13T22:24:20.655968+0000","last_became_peered":"2023-07-13T22:24:20.655968+0000","last_unstale":"2023-07-13T22:24:20.664984+0000","last_undegraded":"2023-07-13T22:24:20.664984+0000","last_fullsized":"2023-07-13T22:24:20.664984+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:11:46.990340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646291+0000","last_change":"2023-07-13T22:24:19.630466+0000","last_active":"2023-07-13T22:24:20.646291+0000","last_peered":"2023-07-13T22:24:20.646291+0000","last_clean":"2023-07-13T22:24:20.646291+0000","last_became_active":"2023-07-13T22:24:19.630164+0000","last_became_peered":"2023-07-13T22:24:19.630164+0000","last_unstale":"2023-07-13T22:24:20.646291+0000","last_undegraded":"2023-07-13T22:24:20.646291+0000","last_fullsized":"2023-07-13T22:24:20.646291+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:43:57.657473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":88,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651526+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:24:20.651526+0000","last_peered":"2023-07-13T22:24:20.651526+0000","last_clean":"2023-07-13T22:24:20.651526+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:24:20.651526+0000","last_undegraded":"2023-07-13T22:24:20.651526+0000","last_fullsized":"2023-07-13T22:24:20.651526+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:01:30.260860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675403+0000","last_change":"2023-07-13T22:24:20.671710+0000","last_active":"2023-07-13T22:24:20.675403+0000","last_peered":"2023-07-13T22:24:20.675403+0000","last_clean":"2023-07-13T22:24:20.675403+0000","last_became_active":"2023-07-13T22:24:20.671407+0000","last_became_peered":"2023-07-13T22:24:20.671407+0000","last_unstale":"2023-07-13T22:24:20.675403+0000","last_undegraded":"2023-07-13T22:24:20.675403+0000","last_fullsized":"2023-07-13T22:24:20.675403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:09:41.848324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646296+0000","last_change":"2023-07-13T22:24:19.635176+0000","last_active":"2023-07-13T22:24:20.646296+0000","last_peered":"2023-07-13T22:24:20.646296+0000","last_clean":"2023-07-13T22:24:20.646296+0000","last_became_active":"2023-07-13T22:24:19.634877+0000","last_became_peered":"2023-07-13T22:24:19.634877+0000","last_unstale":"2023-07-13T22:24:20.646296+0000","last_undegraded":"2023-07-13T22:24:20.646296+0000","last_fullsized":"2023-07-13T22:24:20.646296+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:17:20.831144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.650872+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:24:20.650872+0000","last_peered":"2023-07-13T22:24:20.650872+0000","last_clean":"2023-07-13T22:24:20.650872+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:24:20.650872+0000","last_undegraded":"2023-07-13T22:24:20.650872+0000","last_fullsized":"2023-07-13T22:24:20.650872+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:45:27.883604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.668914+0000","last_change":"2023-07-13T22:24:20.662676+0000","last_active":"2023-07-13T22:24:20.668914+0000","last_peered":"2023-07-13T22:24:20.668914+0000","last_clean":"2023-07-13T22:24:20.668914+0000","last_became_active":"2023-07-13T22:24:20.662205+0000","last_became_peered":"2023-07-13T22:24:20.662205+0000","last_unstale":"2023-07-13T22:24:20.668914+0000","last_undegraded":"2023-07-13T22:24:20.668914+0000","last_fullsized":"2023-07-13T22:24:20.668914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:47:51.868302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.649185+0000","last_change":"2023-07-13T22:24:19.635553+0000","last_active":"2023-07-13T22:24:20.649185+0000","last_peered":"2023-07-13T22:24:20.649185+0000","last_clean":"2023-07-13T22:24:20.649185+0000","last_became_active":"2023-07-13T22:24:19.635241+0000","last_became_peered":"2023-07-13T22:24:19.635241+0000","last_unstale":"2023-07-13T22:24:20.649185+0000","last_undegraded":"2023-07-13T22:24:20.649185+0000","last_fullsized":"2023-07-13T22:24:20.649185+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:56:46.919012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.644756+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:24:20.644756+0000","last_peered":"2023-07-13T22:24:20.644756+0000","last_clean":"2023-07-13T22:24:20.644756+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:24:20.644756+0000","last_undegraded":"2023-07-13T22:24:20.644756+0000","last_fullsized":"2023-07-13T22:24:20.644756+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:42:57.664224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677498+0000","last_change":"2023-07-13T22:24:20.674656+0000","last_active":"2023-07-13T22:24:20.677498+0000","last_peered":"2023-07-13T22:24:20.677498+0000","last_clean":"2023-07-13T22:24:20.677498+0000","last_became_active":"2023-07-13T22:24:20.674081+0000","last_became_peered":"2023-07-13T22:24:20.674081+0000","last_unstale":"2023-07-13T22:24:20.677498+0000","last_undegraded":"2023-07-13T22:24:20.677498+0000","last_fullsized":"2023-07-13T22:24:20.677498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:49:09.132236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677544+0000","last_change":"2023-07-13T22:24:19.640309+0000","last_active":"2023-07-13T22:24:20.677544+0000","last_peered":"2023-07-13T22:24:20.677544+0000","last_clean":"2023-07-13T22:24:20.677544+0000","last_became_active":"2023-07-13T22:24:19.639931+0000","last_became_peered":"2023-07-13T22:24:19.639931+0000","last_unstale":"2023-07-13T22:24:20.677544+0000","last_undegraded":"2023-07-13T22:24:20.677544+0000","last_fullsized":"2023-07-13T22:24:20.677544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:19:51.386379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.658867+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:24:20.658867+0000","last_peered":"2023-07-13T22:24:20.658867+0000","last_clean":"2023-07-13T22:24:20.658867+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:24:20.658867+0000","last_undegraded":"2023-07-13T22:24:20.658867+0000","last_fullsized":"2023-07-13T22:24:20.658867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:49:21.460134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.682214+0000","last_change":"2023-07-13T22:24:20.678000+0000","last_active":"2023-07-13T22:24:20.682214+0000","last_peered":"2023-07-13T22:24:20.682214+0000","last_clean":"2023-07-13T22:24:20.682214+0000","last_became_active":"2023-07-13T22:24:20.677184+0000","last_became_peered":"2023-07-13T22:24:20.677184+0000","last_unstale":"2023-07-13T22:24:20.682214+0000","last_undegraded":"2023-07-13T22:24:20.682214+0000","last_fullsized":"2023-07-13T22:24:20.682214+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:10:26.737746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673645+0000","last_change":"2023-07-13T22:24:19.631396+0000","last_active":"2023-07-13T22:24:20.673645+0000","last_peered":"2023-07-13T22:24:20.673645+0000","last_clean":"2023-07-13T22:24:20.673645+0000","last_became_active":"2023-07-13T22:24:19.631135+0000","last_became_peered":"2023-07-13T22:24:19.631135+0000","last_unstale":"2023-07-13T22:24:20.673645+0000","last_undegraded":"2023-07-13T22:24:20.673645+0000","last_fullsized":"2023-07-13T22:24:20.673645+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:25:01.024244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.658461+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:24:20.658461+0000","last_peered":"2023-07-13T22:24:20.658461+0000","last_clean":"2023-07-13T22:24:20.658461+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:24:20.658461+0000","last_undegraded":"2023-07-13T22:24:20.658461+0000","last_fullsized":"2023-07-13T22:24:20.658461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:31:18.078004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1314816,"data_stored":907481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.623},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.88,"5min":1.115,"15min":1.115},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86799999999999999,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77500000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.71099999999999997},"min":{"1min":0.40100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90300000000000002,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.496,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.81599999999999995},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94499999999999995,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.76100000000000001,"15min":0.81200000000000006},"min":{"1min":0.45000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.879,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.67800000000000005,"15min":0.68600000000000005},"min":{"1min":0.45000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95199999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76100000000000001,"15min":0.76500000000000001},"min":{"1min":0.57299999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95299999999999996,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88700000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0860000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93999999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1359999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.751}]},{"osd":6,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.56899999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.57399999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":737280,"data_stored":324275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.745},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.46100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85999999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.46200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85099999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66400000000000003,"15min":0.65900000000000003},"min":{"1min":0.55100000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.65500000000000003,"15min":0.65800000000000003},"min":{"1min":0.48999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81200000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65300000000000002,"15min":0.67300000000000004},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.439,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.877}]},{"osd":5,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78000000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92300000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.74299999999999999,"15min":0.76100000000000001},"min":{"1min":0.49299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.95099999999999996,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.53700000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88300000000000001,"5min":1.111,"15min":1.111},"last":0.65500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1314816,"data_stored":907481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":1.952,"15min":1.952},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64300000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.52800000000000002,"15min":0.56200000000000006},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61299999999999999,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.443},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58499999999999996,"15min":0.61899999999999999},"min":{"1min":0.39000000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.67100000000000004,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61599999999999999,"15min":0.63600000000000001},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76000000000000001,"5min":2.008,"15min":2.008},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68500000000000005,"5min":0.87,"15min":0.87},"last":0.496}]},{"osd":4,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79200000000000004,"15min":0.82699999999999996},"min":{"1min":0.59299999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91300000000000003,"5min":2.028,"15min":2.028},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77800000000000002,"15min":0.81299999999999994},"min":{"1min":0.55200000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84799999999999998,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.75,"15min":0.76800000000000002},"min":{"1min":0.625,"5min":0.44,"15min":0.44},"max":{"1min":0.97699999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.82999999999999996,"15min":0.873},"min":{"1min":0.56599999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.60599999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0169999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.84899999999999998,"15min":0.89700000000000002},"min":{"1min":0.55900000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.94899999999999995,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.91700000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80900000000000005,"15min":0.84099999999999997},"min":{"1min":0.52200000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96399999999999997,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.875},{"interface":"front","average":{"1min":0.745,"5min":0.80100000000000005,"15min":0.83399999999999996},"min":{"1min":0.63300000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.042,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":729088,"data_stored":323255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.60399999999999998,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.026,"5min":11.791,"15min":11.791},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.419,"5min":4.657,"15min":4.657},"last":0.497}]},{"osd":2,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63400000000000001,"15min":0.61199999999999999},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.012,"5min":4.5389999999999997,"15min":4.5389999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":1.0609999999999999,"15min":1.1419999999999999},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":13.398999999999999,"15min":13.398999999999999},"last":0.378}]},{"osd":3,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56499999999999995,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.37,"15min":0.37},"max":{"1min":0.89400000000000002,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58999999999999997,"15min":0.56599999999999995},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96099999999999997,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.438}]},{"osd":4,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":1.298,"15min":1.4059999999999999},"min":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.82499999999999996,"5min":13.667999999999999,"15min":13.667999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80200000000000005,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.083,"5min":1.776,"15min":1.776},"last":0.755}]},{"osd":5,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":1.3169999999999999,"15min":1.4179999999999999},"min":{"1min":0.61699999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88400000000000001,"5min":13.349,"15min":13.349},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.63400000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93600000000000005,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97599999999999998,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.752,"5min":1.2390000000000001,"15min":1.3899999999999999},"min":{"1min":0.64400000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.89800000000000002,"5min":11.679,"15min":11.679},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":1.327,"15min":1.4259999999999999},"min":{"1min":0.61699999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.94499999999999995,"5min":13.298,"15min":13.298},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.96899999999999997,"15min":0.91900000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.054,"5min":11.456,"15min":11.456},"last":0.84699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":716800,"data_stored":316127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.52500000000000002,"15min":0.57199999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61399999999999999,"5min":2.282,"15min":2.282},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46300000000000002,"15min":0.46999999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.46200000000000002,"15min":0.47399999999999998},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.58399999999999996,"15min":0.623},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65400000000000003,"5min":2.319,"15min":2.319},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.63200000000000001,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79000000000000004,"5min":2.351,"15min":2.351},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82699999999999996,"15min":0.86899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.95699999999999996,"5min":2.238,"15min":2.238},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.52000000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.518,"15min":1.518},"last":0.621}]},{"osd":5,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1299999999999999,"5min":1.325,"15min":1.325},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.83799999999999997,"15min":0.88200000000000001},"min":{"1min":0.59099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.0249999999999999,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.80000000000000004,"15min":0.81599999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91000000000000003,"5min":1.488,"15min":1.488},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.502,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.996,"5min":1.214,"15min":1.214},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.625,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.02,"5min":1.29,"15min":1.29},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81799999999999995,"15min":0.83999999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0860000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.65600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35860,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":733184,"data_stored":324241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53000000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.443,"15min":0.439},"min":{"1min":0.39000000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58699999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63700000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.56899999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.45700000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.72599999999999998,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.55300000000000005},"min":{"1min":0.44500000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69299999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.73799999999999999},"min":{"1min":0.51100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90900000000000003,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.63500000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95099999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90300000000000002,"5min":1.411,"15min":1.411},"last":1.032},{"interface":"front","average":{"1min":0.755,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.56499999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96899999999999997,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.55900000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93799999999999994,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.85799999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.88600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79200000000000004,"15min":0.81000000000000005},"min":{"1min":0.58999999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.93700000000000006,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.65400000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.085}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":745472,"data_stored":325805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.60599999999999998},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83499999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.495,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86199999999999999,"5min":1.298,"15min":1.298},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.51800000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97999999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82599999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.034,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.93999999999999995,"15min":1.0780000000000001},"min":{"1min":0.47499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86199999999999999,"5min":10.657999999999999,"15min":10.657999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.92700000000000005,"15min":1.0289999999999999},"min":{"1min":0.55000000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96699999999999997,"5min":8.75,"15min":8.75},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68400000000000005,"15min":0.69799999999999995},"min":{"1min":0.501,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.53800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0109999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.49399999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":1.173,"15min":1.173},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.73299999999999998},"min":{"1min":0.45100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.0509999999999999,"5min":1.099,"15min":1.099},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.92800000000000005,"5min":1.004,"15min":1.004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.54000000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.212,"15min":1.212},"last":0.88300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":737280,"data_stored":324263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81599999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.47899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.52900000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.093,"5min":0.73099999999999998,"15min":0.66900000000000004},"min":{"1min":0.36299999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":8.9930000000000003,"5min":8.9930000000000003,"15min":8.9930000000000003},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.626,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90900000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.80300000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.76000000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.52300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93999999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.78000000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.629,"15min":0.62},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67400000000000004,"15min":0.69699999999999995},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84899999999999998,"5min":0.995,"15min":0.995},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.502,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86699999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.58099999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88900000000000001,"5min":1.02,"15min":1.02},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.52900000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.97299999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:24:25.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:24:25.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.327+0000 7f7ce39ff700 1 Processor -- start 2023-07-13T22:24:25.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.327+0000 7f7ce39ff700 1 -- start start 2023-07-13T22:24:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce39ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 0x7f7cdc0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce39ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7cdc0d3750 0x7f7cdc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce39ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cdc0cff20 con 0x7f7cdc0cfad0 2023-07-13T22:24:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce39ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cdc0d0ac0 con 0x7f7cdc0d3750 2023-07-13T22:24:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce39ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0b87d0 con 0x7f7cdc0b8400 2023-07-13T22:24:25.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce0f9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7cdc0d3750 0x7f7cdc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce0f9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7cdc0d3750 0x7f7cdc0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49566/0 (socket says 172.21.15.60:49566) 2023-07-13T22:24:25.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce0f9a700 1 -- 172.21.15.60:0/1178364055 learned_addr learned my addr 172.21.15.60:0/1178364055 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:25.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce179b700 1 --2- 172.21.15.60:0/1178364055 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 0x7f7cdc0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce0f9a700 1 -- 172.21.15.60:0/1178364055 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 msgr2=0x7f7cdc0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:25.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce0f9a700 1 --2- 172.21.15.60:0/1178364055 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 0x7f7cdc0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce0f9a700 1 -- 172.21.15.60:0/1178364055 >> v1:172.21.15.105:6789/0 conn(0x7f7cdc0cfad0 legacy=0x7f7cdc15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.329+0000 7f7ce0f9a700 1 -- 172.21.15.60:0/1178364055 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc0b4eb0 con 0x7f7cdc0d3750 2023-07-13T22:24:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.330+0000 7f7ce0f9a700 1 --2- 172.21.15.60:0/1178364055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 0x7f7cdc0cf590 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0098f0 tx=0x7f7ccc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48edd4b148975f91 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.330+0000 7f7cd3fff700 1 -- 172.21.15.60:0/1178364055 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc011830 con 0x7f7cdc0d3750 2023-07-13T22:24:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.330+0000 7f7cd3fff700 1 -- 172.21.15.60:0/1178364055 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc011990 con 0x7f7cdc0d3750 2023-07-13T22:24:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.330+0000 7f7cd3fff700 1 -- 172.21.15.60:0/1178364055 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc011d20 con 0x7f7cdc0d3750 2023-07-13T22:24:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.330+0000 7f7ce39ff700 1 -- 172.21.15.60:0/1178364055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 msgr2=0x7f7cdc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.330+0000 7f7ce39ff700 1 --2- 172.21.15.60:0/1178364055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 0x7f7cdc0cf590 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0098f0 tx=0x7f7ccc009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.331+0000 7f7ce39ff700 1 -- 172.21.15.60:0/1178364055 shutdown_connections 2023-07-13T22:24:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.331+0000 7f7ce39ff700 1 --2- 172.21.15.60:0/1178364055 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 0x7f7cdc0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.331+0000 7f7ce39ff700 1 --2- 172.21.15.60:0/1178364055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 0x7f7cdc0cf590 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.331+0000 7f7ce39ff700 1 -- 172.21.15.60:0/1178364055 >> 172.21.15.60:0/1178364055 conn(0x7f7cdc06b390 msgr2=0x7f7cdc06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.331+0000 7f7ce39ff700 1 -- 172.21.15.60:0/1178364055 shutdown_connections 2023-07-13T22:24:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.331+0000 7f7ce39ff700 1 -- 172.21.15.60:0/1178364055 wait complete. 2023-07-13T22:24:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.333+0000 7f7ce39ff700 1 Processor -- start 2023-07-13T22:24:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.333+0000 7f7ce39ff700 1 -- start start 2023-07-13T22:24:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce39ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 0x7f7cdc0c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce39ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cdc0cfad0 0x7f7cdc15f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce39ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 0x7f7cdc0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce39ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0c90f0 con 0x7f7cdc0d3750 2023-07-13T22:24:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce39ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc15ec50 con 0x7f7cdc0cfad0 2023-07-13T22:24:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce39ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc15dc20 con 0x7f7cdc0b8400 2023-07-13T22:24:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce1f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 0x7f7cdc0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce1f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 0x7f7cdc0b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49570/0 (socket says 172.21.15.60:49570) 2023-07-13T22:24:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce1f9c700 1 -- 172.21.15.60:0/2377781990 learned_addr learned my addr 172.21.15.60:0/2377781990 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce179b700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 0x7f7cdc0c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce0f9a700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cdc0cfad0 0x7f7cdc15f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce1f9c700 1 -- 172.21.15.60:0/2377781990 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 msgr2=0x7f7cdc0c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce1f9c700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 0x7f7cdc0c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce1f9c700 1 -- 172.21.15.60:0/2377781990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cdc0cfad0 msgr2=0x7f7cdc15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce1f9c700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cdc0cfad0 0x7f7cdc15f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.334+0000 7f7ce1f9c700 1 -- 172.21.15.60:0/2377781990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc009550 con 0x7f7cdc0d3750 2023-07-13T22:24:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.335+0000 7f7ce1f9c700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 0x7f7cdc0b31e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400cb50 tx=0x7f7cd400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.335+0000 7f7cd1ffb700 1 -- 172.21.15.60:0/2377781990 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4003c00 con 0x7f7cdc0d3750 2023-07-13T22:24:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.335+0000 7f7cd1ffb700 1 -- 172.21.15.60:0/2377781990 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd4003d60 con 0x7f7cdc0d3750 2023-07-13T22:24:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.335+0000 7f7cd1ffb700 1 -- 172.21.15.60:0/2377781990 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd400a9c0 con 0x7f7cdc0d3750 2023-07-13T22:24:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.335+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc15d940 con 0x7f7cdc0d3750 2023-07-13T22:24:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.335+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc15e8e0 con 0x7f7cdc0d3750 2023-07-13T22:24:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.336+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc0b47e0 con 0x7f7cdc0d3750 2023-07-13T22:24:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.337+0000 7f7cd1ffb700 1 -- 172.21.15.60:0/2377781990 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7cd400ab20 con 0x7f7cdc0d3750 2023-07-13T22:24:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.338+0000 7f7cd1ffb700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cc006ace0 0x7f7cc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.338+0000 7f7cd1ffb700 1 -- 172.21.15.60:0/2377781990 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7cd4089350 con 0x7f7cdc0d3750 2023-07-13T22:24:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.338+0000 7f7ce179b700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cc006ace0 0x7f7cc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.339+0000 7f7ce179b700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cc006ace0 0x7f7cc006d190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009b40 tx=0x7f7cd8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:25.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.343+0000 7f7cd1ffb700 1 -- 172.21.15.60:0/2377781990 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7cd40542d0 con 0x7f7cdc0d3750 2023-07-13T22:24:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.543+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7cdc0b5470 con 0x7f7cc006ace0 2023-07-13T22:24:25.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.553+0000 7f7cd1ffb700 1 -- 172.21.15.60:0/2377781990 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311483 (secure 0 0 0) 0x7f7cd8011810 con 0x7f7cc006ace0 2023-07-13T22:24:25.539 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:25.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cc006ace0 msgr2=0x7f7cc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cc006ace0 0x7f7cc006d190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009b40 tx=0x7f7cd8009210 comp rx=0 tx=0).stop 2023-07-13T22:24:25.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 msgr2=0x7f7cdc0b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 0x7f7cdc0b31e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400cb50 tx=0x7f7cd400ce60 comp rx=0 tx=0).stop 2023-07-13T22:24:25.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 shutdown_connections 2023-07-13T22:24:25.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cc006ace0 0x7f7cc006d190 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc0d3750 0x7f7cdc0b31e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7cdc0b8400 0x7f7cdc0c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 --2- 172.21.15.60:0/2377781990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cdc0cfad0 0x7f7cdc15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 >> 172.21.15.60:0/2377781990 conn(0x7f7cdc06b390 msgr2=0x7f7cdc0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:25.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 shutdown_connections 2023-07-13T22:24:25.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.557+0000 7f7ce39ff700 1 -- 172.21.15.60:0/2377781990 wait complete. 2023-07-13T22:24:25.542 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:24:25.558 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-07-13T22:24:25.167635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255248,"kb_used_data":6864,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754719472,"statfs":{"total":773094113280,"available":772832739328,"internally_reserved":0,"allocated":7028736,"data_stored":3752928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000908"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.671510+0000","last_change":"2023-07-13T22:24:20.669634+0000","last_active":"2023-07-13T22:24:20.671510+0000","last_peered":"2023-07-13T22:24:20.671510+0000","last_clean":"2023-07-13T22:24:20.671510+0000","last_became_active":"2023-07-13T22:24:20.669300+0000","last_became_peered":"2023-07-13T22:24:20.669300+0000","last_unstale":"2023-07-13T22:24:20.671510+0000","last_undegraded":"2023-07-13T22:24:20.671510+0000","last_fullsized":"2023-07-13T22:24:20.671510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:44.401182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.671423+0000","last_change":"2023-07-13T22:24:20.669097+0000","last_active":"2023-07-13T22:24:20.671423+0000","last_peered":"2023-07-13T22:24:20.671423+0000","last_clean":"2023-07-13T22:24:20.671423+0000","last_became_active":"2023-07-13T22:24:20.668747+0000","last_became_peered":"2023-07-13T22:24:20.668747+0000","last_unstale":"2023-07-13T22:24:20.671423+0000","last_undegraded":"2023-07-13T22:24:20.671423+0000","last_fullsized":"2023-07-13T22:24:20.671423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:03:49.921791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676591+0000","last_change":"2023-07-13T22:24:20.672092+0000","last_active":"2023-07-13T22:24:20.676591+0000","last_peered":"2023-07-13T22:24:20.676591+0000","last_clean":"2023-07-13T22:24:20.676591+0000","last_became_active":"2023-07-13T22:24:20.667557+0000","last_became_peered":"2023-07-13T22:24:20.667557+0000","last_unstale":"2023-07-13T22:24:20.676591+0000","last_undegraded":"2023-07-13T22:24:20.676591+0000","last_fullsized":"2023-07-13T22:24:20.676591+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:33:54.633734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675373+0000","last_change":"2023-07-13T22:24:20.659635+0000","last_active":"2023-07-13T22:24:20.675373+0000","last_peered":"2023-07-13T22:24:20.675373+0000","last_clean":"2023-07-13T22:24:20.675373+0000","last_became_active":"2023-07-13T22:24:20.659363+0000","last_became_peered":"2023-07-13T22:24:20.659363+0000","last_unstale":"2023-07-13T22:24:20.675373+0000","last_undegraded":"2023-07-13T22:24:20.675373+0000","last_fullsized":"2023-07-13T22:24:20.675373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:57:38.507855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676056+0000","last_change":"2023-07-13T22:24:20.674082+0000","last_active":"2023-07-13T22:24:20.676056+0000","last_peered":"2023-07-13T22:24:20.676056+0000","last_clean":"2023-07-13T22:24:20.676056+0000","last_became_active":"2023-07-13T22:24:20.673769+0000","last_became_peered":"2023-07-13T22:24:20.673769+0000","last_unstale":"2023-07-13T22:24:20.676056+0000","last_undegraded":"2023-07-13T22:24:20.676056+0000","last_fullsized":"2023-07-13T22:24:20.676056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:03:49.921791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675064+0000","last_change":"2023-07-13T22:24:20.673700+0000","last_active":"2023-07-13T22:24:20.675064+0000","last_peered":"2023-07-13T22:24:20.675064+0000","last_clean":"2023-07-13T22:24:20.675064+0000","last_became_active":"2023-07-13T22:24:20.673422+0000","last_became_peered":"2023-07-13T22:24:20.673422+0000","last_unstale":"2023-07-13T22:24:20.675064+0000","last_undegraded":"2023-07-13T22:24:20.675064+0000","last_fullsized":"2023-07-13T22:24:20.675064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:28.516245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664673+0000","last_change":"2023-07-13T22:24:20.660247+0000","last_active":"2023-07-13T22:24:20.664673+0000","last_peered":"2023-07-13T22:24:20.664673+0000","last_clean":"2023-07-13T22:24:20.664673+0000","last_became_active":"2023-07-13T22:24:20.659983+0000","last_became_peered":"2023-07-13T22:24:20.659983+0000","last_unstale":"2023-07-13T22:24:20.664673+0000","last_undegraded":"2023-07-13T22:24:20.664673+0000","last_fullsized":"2023-07-13T22:24:20.664673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:46:30.713669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.672771+0000","last_change":"2023-07-13T22:24:20.660171+0000","last_active":"2023-07-13T22:24:20.672771+0000","last_peered":"2023-07-13T22:24:20.672771+0000","last_clean":"2023-07-13T22:24:20.672771+0000","last_became_active":"2023-07-13T22:24:20.659867+0000","last_became_peered":"2023-07-13T22:24:20.659867+0000","last_unstale":"2023-07-13T22:24:20.672771+0000","last_undegraded":"2023-07-13T22:24:20.672771+0000","last_fullsized":"2023-07-13T22:24:20.672771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:21:41.046613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.679459+0000","last_change":"2023-07-13T22:24:20.675688+0000","last_active":"2023-07-13T22:24:20.679459+0000","last_peered":"2023-07-13T22:24:20.679459+0000","last_clean":"2023-07-13T22:24:20.679459+0000","last_became_active":"2023-07-13T22:24:20.675229+0000","last_became_peered":"2023-07-13T22:24:20.675229+0000","last_unstale":"2023-07-13T22:24:20.679459+0000","last_undegraded":"2023-07-13T22:24:20.679459+0000","last_fullsized":"2023-07-13T22:24:20.679459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:00:19.264666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676289+0000","last_change":"2023-07-13T22:24:20.670702+0000","last_active":"2023-07-13T22:24:20.676289+0000","last_peered":"2023-07-13T22:24:20.676289+0000","last_clean":"2023-07-13T22:24:20.676289+0000","last_became_active":"2023-07-13T22:24:20.670381+0000","last_became_peered":"2023-07-13T22:24:20.670381+0000","last_unstale":"2023-07-13T22:24:20.676289+0000","last_undegraded":"2023-07-13T22:24:20.676289+0000","last_fullsized":"2023-07-13T22:24:20.676289+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:13:42.472851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677840+0000","last_change":"2023-07-13T22:24:20.671721+0000","last_active":"2023-07-13T22:24:20.677840+0000","last_peered":"2023-07-13T22:24:20.677840+0000","last_clean":"2023-07-13T22:24:20.677840+0000","last_became_active":"2023-07-13T22:24:20.671382+0000","last_became_peered":"2023-07-13T22:24:20.671382+0000","last_unstale":"2023-07-13T22:24:20.677840+0000","last_undegraded":"2023-07-13T22:24:20.677840+0000","last_fullsized":"2023-07-13T22:24:20.677840+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:08:09.879746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673540+0000","last_change":"2023-07-13T22:24:20.661577+0000","last_active":"2023-07-13T22:24:20.673540+0000","last_peered":"2023-07-13T22:24:20.673540+0000","last_clean":"2023-07-13T22:24:20.673540+0000","last_became_active":"2023-07-13T22:24:20.661267+0000","last_became_peered":"2023-07-13T22:24:20.661267+0000","last_unstale":"2023-07-13T22:24:20.673540+0000","last_undegraded":"2023-07-13T22:24:20.673540+0000","last_fullsized":"2023-07-13T22:24:20.673540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:16.269860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.666703+0000","last_change":"2023-07-13T22:24:20.658980+0000","last_active":"2023-07-13T22:24:20.666703+0000","last_peered":"2023-07-13T22:24:20.666703+0000","last_clean":"2023-07-13T22:24:20.666703+0000","last_became_active":"2023-07-13T22:24:20.658680+0000","last_became_peered":"2023-07-13T22:24:20.658680+0000","last_unstale":"2023-07-13T22:24:20.666703+0000","last_undegraded":"2023-07-13T22:24:20.666703+0000","last_fullsized":"2023-07-13T22:24:20.666703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:01:08.735733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674430+0000","last_change":"2023-07-13T22:24:20.663756+0000","last_active":"2023-07-13T22:24:20.674430+0000","last_peered":"2023-07-13T22:24:20.674430+0000","last_clean":"2023-07-13T22:24:20.674430+0000","last_became_active":"2023-07-13T22:24:20.663423+0000","last_became_peered":"2023-07-13T22:24:20.663423+0000","last_unstale":"2023-07-13T22:24:20.674430+0000","last_undegraded":"2023-07-13T22:24:20.674430+0000","last_fullsized":"2023-07-13T22:24:20.674430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:53.689341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.678264+0000","last_change":"2023-07-13T22:24:20.674963+0000","last_active":"2023-07-13T22:24:20.678264+0000","last_peered":"2023-07-13T22:24:20.678264+0000","last_clean":"2023-07-13T22:24:20.678264+0000","last_became_active":"2023-07-13T22:24:20.674711+0000","last_became_peered":"2023-07-13T22:24:20.674711+0000","last_unstale":"2023-07-13T22:24:20.678264+0000","last_undegraded":"2023-07-13T22:24:20.678264+0000","last_fullsized":"2023-07-13T22:24:20.678264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:24:36.692640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.655782+0000","last_change":"2023-07-13T22:24:20.652882+0000","last_active":"2023-07-13T22:24:20.655782+0000","last_peered":"2023-07-13T22:24:20.655782+0000","last_clean":"2023-07-13T22:24:20.655782+0000","last_became_active":"2023-07-13T22:24:20.652579+0000","last_became_peered":"2023-07-13T22:24:20.652579+0000","last_unstale":"2023-07-13T22:24:20.655782+0000","last_undegraded":"2023-07-13T22:24:20.655782+0000","last_fullsized":"2023-07-13T22:24:20.655782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:16:54.497570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673481+0000","last_change":"2023-07-13T22:24:20.671527+0000","last_active":"2023-07-13T22:24:20.673481+0000","last_peered":"2023-07-13T22:24:20.673481+0000","last_clean":"2023-07-13T22:24:20.673481+0000","last_became_active":"2023-07-13T22:24:20.671256+0000","last_became_peered":"2023-07-13T22:24:20.671256+0000","last_unstale":"2023-07-13T22:24:20.673481+0000","last_undegraded":"2023-07-13T22:24:20.673481+0000","last_fullsized":"2023-07-13T22:24:20.673481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:36:21.829255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674916+0000","last_change":"2023-07-13T22:24:20.661021+0000","last_active":"2023-07-13T22:24:20.674916+0000","last_peered":"2023-07-13T22:24:20.674916+0000","last_clean":"2023-07-13T22:24:20.674916+0000","last_became_active":"2023-07-13T22:24:20.660628+0000","last_became_peered":"2023-07-13T22:24:20.660628+0000","last_unstale":"2023-07-13T22:24:20.674916+0000","last_undegraded":"2023-07-13T22:24:20.674916+0000","last_fullsized":"2023-07-13T22:24:20.674916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:28.516245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673744+0000","last_change":"2023-07-13T22:24:20.668034+0000","last_active":"2023-07-13T22:24:20.673744+0000","last_peered":"2023-07-13T22:24:20.673744+0000","last_clean":"2023-07-13T22:24:20.673744+0000","last_became_active":"2023-07-13T22:24:20.667711+0000","last_became_peered":"2023-07-13T22:24:20.667711+0000","last_unstale":"2023-07-13T22:24:20.673744+0000","last_undegraded":"2023-07-13T22:24:20.673744+0000","last_fullsized":"2023-07-13T22:24:20.673744+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:37:44.224883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673956+0000","last_change":"2023-07-13T22:24:20.669076+0000","last_active":"2023-07-13T22:24:20.673956+0000","last_peered":"2023-07-13T22:24:20.673956+0000","last_clean":"2023-07-13T22:24:20.673956+0000","last_became_active":"2023-07-13T22:24:20.668812+0000","last_became_peered":"2023-07-13T22:24:20.668812+0000","last_unstale":"2023-07-13T22:24:20.673956+0000","last_undegraded":"2023-07-13T22:24:20.673956+0000","last_fullsized":"2023-07-13T22:24:20.673956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:13:50.697622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673554+0000","last_change":"2023-07-13T22:24:20.665603+0000","last_active":"2023-07-13T22:24:20.673554+0000","last_peered":"2023-07-13T22:24:20.673554+0000","last_clean":"2023-07-13T22:24:20.673554+0000","last_became_active":"2023-07-13T22:24:20.665340+0000","last_became_peered":"2023-07-13T22:24:20.665340+0000","last_unstale":"2023-07-13T22:24:20.673554+0000","last_undegraded":"2023-07-13T22:24:20.673554+0000","last_fullsized":"2023-07-13T22:24:20.673554+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:46:36.043909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674886+0000","last_change":"2023-07-13T22:24:20.665186+0000","last_active":"2023-07-13T22:24:20.674886+0000","last_peered":"2023-07-13T22:24:20.674886+0000","last_clean":"2023-07-13T22:24:20.674886+0000","last_became_active":"2023-07-13T22:24:20.664898+0000","last_became_peered":"2023-07-13T22:24:20.664898+0000","last_unstale":"2023-07-13T22:24:20.674886+0000","last_undegraded":"2023-07-13T22:24:20.674886+0000","last_fullsized":"2023-07-13T22:24:20.674886+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:49:00.787861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664523+0000","last_change":"2023-07-13T22:24:20.663872+0000","last_active":"2023-07-13T22:24:20.664523+0000","last_peered":"2023-07-13T22:24:20.664523+0000","last_clean":"2023-07-13T22:24:20.664523+0000","last_became_active":"2023-07-13T22:24:20.663606+0000","last_became_peered":"2023-07-13T22:24:20.663606+0000","last_unstale":"2023-07-13T22:24:20.664523+0000","last_undegraded":"2023-07-13T22:24:20.664523+0000","last_fullsized":"2023-07-13T22:24:20.664523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:06:05.347740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676495+0000","last_change":"2023-07-13T22:24:20.671894+0000","last_active":"2023-07-13T22:24:20.676495+0000","last_peered":"2023-07-13T22:24:20.676495+0000","last_clean":"2023-07-13T22:24:20.676495+0000","last_became_active":"2023-07-13T22:24:20.669965+0000","last_became_peered":"2023-07-13T22:24:20.669965+0000","last_unstale":"2023-07-13T22:24:20.676495+0000","last_undegraded":"2023-07-13T22:24:20.676495+0000","last_fullsized":"2023-07-13T22:24:20.676495+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:06:17.569571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.660491+0000","last_change":"2023-07-13T22:24:20.655295+0000","last_active":"2023-07-13T22:24:20.660491+0000","last_peered":"2023-07-13T22:24:20.660491+0000","last_clean":"2023-07-13T22:24:20.660491+0000","last_became_active":"2023-07-13T22:24:20.654981+0000","last_became_peered":"2023-07-13T22:24:20.654981+0000","last_unstale":"2023-07-13T22:24:20.660491+0000","last_undegraded":"2023-07-13T22:24:20.660491+0000","last_fullsized":"2023-07-13T22:24:20.660491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:00:00.658423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.666790+0000","last_change":"2023-07-13T22:24:20.659246+0000","last_active":"2023-07-13T22:24:20.666790+0000","last_peered":"2023-07-13T22:24:20.666790+0000","last_clean":"2023-07-13T22:24:20.666790+0000","last_became_active":"2023-07-13T22:24:20.658269+0000","last_became_peered":"2023-07-13T22:24:20.658269+0000","last_unstale":"2023-07-13T22:24:20.666790+0000","last_undegraded":"2023-07-13T22:24:20.666790+0000","last_fullsized":"2023-07-13T22:24:20.666790+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:27:35.909871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.672033+0000","last_change":"2023-07-13T22:24:20.655579+0000","last_active":"2023-07-13T22:24:20.672033+0000","last_peered":"2023-07-13T22:24:20.672033+0000","last_clean":"2023-07-13T22:24:20.672033+0000","last_became_active":"2023-07-13T22:24:20.655319+0000","last_became_peered":"2023-07-13T22:24:20.655319+0000","last_unstale":"2023-07-13T22:24:20.672033+0000","last_undegraded":"2023-07-13T22:24:20.672033+0000","last_fullsized":"2023-07-13T22:24:20.672033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:55:25.391773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.672205+0000","last_change":"2023-07-13T22:24:20.669900+0000","last_active":"2023-07-13T22:24:20.672205+0000","last_peered":"2023-07-13T22:24:20.672205+0000","last_clean":"2023-07-13T22:24:20.672205+0000","last_became_active":"2023-07-13T22:24:20.669599+0000","last_became_peered":"2023-07-13T22:24:20.669599+0000","last_unstale":"2023-07-13T22:24:20.672205+0000","last_undegraded":"2023-07-13T22:24:20.672205+0000","last_fullsized":"2023-07-13T22:24:20.672205+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:35:03.770783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.679466+0000","last_change":"2023-07-13T22:24:20.675368+0000","last_active":"2023-07-13T22:24:20.679466+0000","last_peered":"2023-07-13T22:24:20.679466+0000","last_clean":"2023-07-13T22:24:20.679466+0000","last_became_active":"2023-07-13T22:24:20.671629+0000","last_became_peered":"2023-07-13T22:24:20.671629+0000","last_unstale":"2023-07-13T22:24:20.679466+0000","last_undegraded":"2023-07-13T22:24:20.679466+0000","last_fullsized":"2023-07-13T22:24:20.679466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:33:43.432649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674419+0000","last_change":"2023-07-13T22:24:20.664867+0000","last_active":"2023-07-13T22:24:20.674419+0000","last_peered":"2023-07-13T22:24:20.674419+0000","last_clean":"2023-07-13T22:24:20.674419+0000","last_became_active":"2023-07-13T22:24:20.664607+0000","last_became_peered":"2023-07-13T22:24:20.664607+0000","last_unstale":"2023-07-13T22:24:20.674419+0000","last_undegraded":"2023-07-13T22:24:20.674419+0000","last_fullsized":"2023-07-13T22:24:20.674419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:42:11.542303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665080+0000","last_change":"2023-07-13T22:24:20.652803+0000","last_active":"2023-07-13T22:24:20.665080+0000","last_peered":"2023-07-13T22:24:20.665080+0000","last_clean":"2023-07-13T22:24:20.665080+0000","last_became_active":"2023-07-13T22:24:20.652520+0000","last_became_peered":"2023-07-13T22:24:20.652520+0000","last_unstale":"2023-07-13T22:24:20.665080+0000","last_undegraded":"2023-07-13T22:24:20.665080+0000","last_fullsized":"2023-07-13T22:24:20.665080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:16.269860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665730+0000","last_change":"2023-07-13T22:24:20.660273+0000","last_active":"2023-07-13T22:24:20.665730+0000","last_peered":"2023-07-13T22:24:20.665730+0000","last_clean":"2023-07-13T22:24:20.665730+0000","last_became_active":"2023-07-13T22:24:20.659998+0000","last_became_peered":"2023-07-13T22:24:20.659998+0000","last_unstale":"2023-07-13T22:24:20.665730+0000","last_undegraded":"2023-07-13T22:24:20.665730+0000","last_fullsized":"2023-07-13T22:24:20.665730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:20:12.098671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675964+0000","last_change":"2023-07-13T22:24:20.672767+0000","last_active":"2023-07-13T22:24:20.675964+0000","last_peered":"2023-07-13T22:24:20.675964+0000","last_clean":"2023-07-13T22:24:20.675964+0000","last_became_active":"2023-07-13T22:24:20.670750+0000","last_became_peered":"2023-07-13T22:24:20.670750+0000","last_unstale":"2023-07-13T22:24:20.675964+0000","last_undegraded":"2023-07-13T22:24:20.675964+0000","last_fullsized":"2023-07-13T22:24:20.675964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:28:44.111936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677510+0000","last_change":"2023-07-13T22:24:19.634703+0000","last_active":"2023-07-13T22:24:20.677510+0000","last_peered":"2023-07-13T22:24:20.677510+0000","last_clean":"2023-07-13T22:24:20.677510+0000","last_became_active":"2023-07-13T22:24:19.634448+0000","last_became_peered":"2023-07-13T22:24:19.634448+0000","last_unstale":"2023-07-13T22:24:20.677510+0000","last_undegraded":"2023-07-13T22:24:20.677510+0000","last_fullsized":"2023-07-13T22:24:20.677510+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:45.573094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677818+0000","last_change":"2023-07-13T22:24:20.669989+0000","last_active":"2023-07-13T22:24:20.677818+0000","last_peered":"2023-07-13T22:24:20.677818+0000","last_clean":"2023-07-13T22:24:20.677818+0000","last_became_active":"2023-07-13T22:24:20.669649+0000","last_became_peered":"2023-07-13T22:24:20.669649+0000","last_unstale":"2023-07-13T22:24:20.677818+0000","last_undegraded":"2023-07-13T22:24:20.677818+0000","last_fullsized":"2023-07-13T22:24:20.677818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:23:28.971371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676936+0000","last_change":"2023-07-13T22:24:19.637226+0000","last_active":"2023-07-13T22:24:20.676936+0000","last_peered":"2023-07-13T22:24:20.676936+0000","last_clean":"2023-07-13T22:24:20.676936+0000","last_became_active":"2023-07-13T22:24:19.636945+0000","last_became_peered":"2023-07-13T22:24:19.636945+0000","last_unstale":"2023-07-13T22:24:20.676936+0000","last_undegraded":"2023-07-13T22:24:20.676936+0000","last_fullsized":"2023-07-13T22:24:20.676936+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:28:56.535031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.666279+0000","last_change":"2023-07-13T22:24:20.658243+0000","last_active":"2023-07-13T22:24:20.666279+0000","last_peered":"2023-07-13T22:24:20.666279+0000","last_clean":"2023-07-13T22:24:20.666279+0000","last_became_active":"2023-07-13T22:24:20.657989+0000","last_became_peered":"2023-07-13T22:24:20.657989+0000","last_unstale":"2023-07-13T22:24:20.666279+0000","last_undegraded":"2023-07-13T22:24:20.666279+0000","last_fullsized":"2023-07-13T22:24:20.666279+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:29:33.426585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675296+0000","last_change":"2023-07-13T22:24:19.638013+0000","last_active":"2023-07-13T22:24:20.675296+0000","last_peered":"2023-07-13T22:24:20.675296+0000","last_clean":"2023-07-13T22:24:20.675296+0000","last_became_active":"2023-07-13T22:24:19.637713+0000","last_became_peered":"2023-07-13T22:24:19.637713+0000","last_unstale":"2023-07-13T22:24:20.675296+0000","last_undegraded":"2023-07-13T22:24:20.675296+0000","last_fullsized":"2023-07-13T22:24:20.675296+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:03:54.517507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.676197+0000","last_change":"2023-07-13T22:24:20.670847+0000","last_active":"2023-07-13T22:24:20.676197+0000","last_peered":"2023-07-13T22:24:20.676197+0000","last_clean":"2023-07-13T22:24:20.676197+0000","last_became_active":"2023-07-13T22:24:20.670534+0000","last_became_peered":"2023-07-13T22:24:20.670534+0000","last_unstale":"2023-07-13T22:24:20.676197+0000","last_undegraded":"2023-07-13T22:24:20.676197+0000","last_fullsized":"2023-07-13T22:24:20.676197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:01:57.690180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.672811+0000","last_change":"2023-07-13T22:24:19.633596+0000","last_active":"2023-07-13T22:24:20.672811+0000","last_peered":"2023-07-13T22:24:20.672811+0000","last_clean":"2023-07-13T22:24:20.672811+0000","last_became_active":"2023-07-13T22:24:19.633317+0000","last_became_peered":"2023-07-13T22:24:19.633317+0000","last_unstale":"2023-07-13T22:24:20.672811+0000","last_undegraded":"2023-07-13T22:24:20.672811+0000","last_fullsized":"2023-07-13T22:24:20.672811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:46:18.176621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664690+0000","last_change":"2023-07-13T22:24:20.662669+0000","last_active":"2023-07-13T22:24:20.664690+0000","last_peered":"2023-07-13T22:24:20.664690+0000","last_clean":"2023-07-13T22:24:20.664690+0000","last_became_active":"2023-07-13T22:24:20.662396+0000","last_became_peered":"2023-07-13T22:24:20.662396+0000","last_unstale":"2023-07-13T22:24:20.664690+0000","last_undegraded":"2023-07-13T22:24:20.664690+0000","last_fullsized":"2023-07-13T22:24:20.664690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:12:06.305028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.668020+0000","last_change":"2023-07-13T22:24:19.635730+0000","last_active":"2023-07-13T22:24:20.668020+0000","last_peered":"2023-07-13T22:24:20.668020+0000","last_clean":"2023-07-13T22:24:20.668020+0000","last_became_active":"2023-07-13T22:24:19.634831+0000","last_became_peered":"2023-07-13T22:24:19.634831+0000","last_unstale":"2023-07-13T22:24:20.668020+0000","last_undegraded":"2023-07-13T22:24:20.668020+0000","last_fullsized":"2023-07-13T22:24:20.668020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:27:37.954922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673420+0000","last_change":"2023-07-13T22:24:20.671314+0000","last_active":"2023-07-13T22:24:20.673420+0000","last_peered":"2023-07-13T22:24:20.673420+0000","last_clean":"2023-07-13T22:24:20.673420+0000","last_became_active":"2023-07-13T22:24:20.671050+0000","last_became_peered":"2023-07-13T22:24:20.671050+0000","last_unstale":"2023-07-13T22:24:20.673420+0000","last_undegraded":"2023-07-13T22:24:20.673420+0000","last_fullsized":"2023-07-13T22:24:20.673420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:18:33.656293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665625+0000","last_change":"2023-07-13T22:24:19.631973+0000","last_active":"2023-07-13T22:24:20.665625+0000","last_peered":"2023-07-13T22:24:20.665625+0000","last_clean":"2023-07-13T22:24:20.665625+0000","last_became_active":"2023-07-13T22:24:19.631250+0000","last_became_peered":"2023-07-13T22:24:19.631250+0000","last_unstale":"2023-07-13T22:24:20.665625+0000","last_undegraded":"2023-07-13T22:24:20.665625+0000","last_fullsized":"2023-07-13T22:24:20.665625+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:34:34.376728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.668939+0000","last_change":"2023-07-13T22:24:20.663336+0000","last_active":"2023-07-13T22:24:20.668939+0000","last_peered":"2023-07-13T22:24:20.668939+0000","last_clean":"2023-07-13T22:24:20.668939+0000","last_became_active":"2023-07-13T22:24:20.663010+0000","last_became_peered":"2023-07-13T22:24:20.663010+0000","last_unstale":"2023-07-13T22:24:20.668939+0000","last_undegraded":"2023-07-13T22:24:20.668939+0000","last_fullsized":"2023-07-13T22:24:20.668939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:06.435575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647121+0000","last_change":"2023-07-13T22:24:19.639911+0000","last_active":"2023-07-13T22:24:20.647121+0000","last_peered":"2023-07-13T22:24:20.647121+0000","last_clean":"2023-07-13T22:24:20.647121+0000","last_became_active":"2023-07-13T22:24:19.639573+0000","last_became_peered":"2023-07-13T22:24:19.639573+0000","last_unstale":"2023-07-13T22:24:20.647121+0000","last_undegraded":"2023-07-13T22:24:20.647121+0000","last_fullsized":"2023-07-13T22:24:20.647121+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:06:45.241977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.662722+0000","last_change":"2023-07-13T22:24:20.657551+0000","last_active":"2023-07-13T22:24:20.662722+0000","last_peered":"2023-07-13T22:24:20.662722+0000","last_clean":"2023-07-13T22:24:20.662722+0000","last_became_active":"2023-07-13T22:24:20.657243+0000","last_became_peered":"2023-07-13T22:24:20.657243+0000","last_unstale":"2023-07-13T22:24:20.662722+0000","last_undegraded":"2023-07-13T22:24:20.662722+0000","last_fullsized":"2023-07-13T22:24:20.662722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:16:30.944384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.658174+0000","last_change":"2023-07-13T22:24:19.635643+0000","last_active":"2023-07-13T22:24:20.658174+0000","last_peered":"2023-07-13T22:24:20.658174+0000","last_clean":"2023-07-13T22:24:20.658174+0000","last_became_active":"2023-07-13T22:24:19.635311+0000","last_became_peered":"2023-07-13T22:24:19.635311+0000","last_unstale":"2023-07-13T22:24:20.658174+0000","last_undegraded":"2023-07-13T22:24:20.658174+0000","last_fullsized":"2023-07-13T22:24:20.658174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:18:32.653836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.669478+0000","last_change":"2023-07-13T22:24:20.661322+0000","last_active":"2023-07-13T22:24:20.669478+0000","last_peered":"2023-07-13T22:24:20.669478+0000","last_clean":"2023-07-13T22:24:20.669478+0000","last_became_active":"2023-07-13T22:24:20.660938+0000","last_became_peered":"2023-07-13T22:24:20.660938+0000","last_unstale":"2023-07-13T22:24:20.669478+0000","last_undegraded":"2023-07-13T22:24:20.669478+0000","last_fullsized":"2023-07-13T22:24:20.669478+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:40:42.360427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.654024+0000","last_change":"2023-07-13T22:24:19.641304+0000","last_active":"2023-07-13T22:24:20.654024+0000","last_peered":"2023-07-13T22:24:20.654024+0000","last_clean":"2023-07-13T22:24:20.654024+0000","last_became_active":"2023-07-13T22:24:19.641040+0000","last_became_peered":"2023-07-13T22:24:19.641040+0000","last_unstale":"2023-07-13T22:24:20.654024+0000","last_undegraded":"2023-07-13T22:24:20.654024+0000","last_fullsized":"2023-07-13T22:24:20.654024+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:44:38.970375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.662137+0000","last_change":"2023-07-13T22:24:20.652838+0000","last_active":"2023-07-13T22:24:20.662137+0000","last_peered":"2023-07-13T22:24:20.662137+0000","last_clean":"2023-07-13T22:24:20.662137+0000","last_became_active":"2023-07-13T22:24:20.652541+0000","last_became_peered":"2023-07-13T22:24:20.652541+0000","last_unstale":"2023-07-13T22:24:20.662137+0000","last_undegraded":"2023-07-13T22:24:20.662137+0000","last_fullsized":"2023-07-13T22:24:20.662137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:58:45.677101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647667+0000","last_change":"2023-07-13T22:24:19.632268+0000","last_active":"2023-07-13T22:24:20.647667+0000","last_peered":"2023-07-13T22:24:20.647667+0000","last_clean":"2023-07-13T22:24:20.647667+0000","last_became_active":"2023-07-13T22:24:19.631926+0000","last_became_peered":"2023-07-13T22:24:19.631926+0000","last_unstale":"2023-07-13T22:24:20.647667+0000","last_undegraded":"2023-07-13T22:24:20.647667+0000","last_fullsized":"2023-07-13T22:24:20.647667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:00:51.354498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675230+0000","last_change":"2023-07-13T22:24:20.669725+0000","last_active":"2023-07-13T22:24:20.675230+0000","last_peered":"2023-07-13T22:24:20.675230+0000","last_clean":"2023-07-13T22:24:20.675230+0000","last_became_active":"2023-07-13T22:24:20.668270+0000","last_became_peered":"2023-07-13T22:24:20.668270+0000","last_unstale":"2023-07-13T22:24:20.675230+0000","last_undegraded":"2023-07-13T22:24:20.675230+0000","last_fullsized":"2023-07-13T22:24:20.675230+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:38:36.818193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.682300+0000","last_change":"2023-07-13T22:24:19.633522+0000","last_active":"2023-07-13T22:24:20.682300+0000","last_peered":"2023-07-13T22:24:20.682300+0000","last_clean":"2023-07-13T22:24:20.682300+0000","last_became_active":"2023-07-13T22:24:19.632838+0000","last_became_peered":"2023-07-13T22:24:19.632838+0000","last_unstale":"2023-07-13T22:24:20.682300+0000","last_undegraded":"2023-07-13T22:24:20.682300+0000","last_fullsized":"2023-07-13T22:24:20.682300+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:50:39.953564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673028+0000","last_change":"2023-07-13T22:24:20.654424+0000","last_active":"2023-07-13T22:24:20.673028+0000","last_peered":"2023-07-13T22:24:20.673028+0000","last_clean":"2023-07-13T22:24:20.673028+0000","last_became_active":"2023-07-13T22:24:20.653416+0000","last_became_peered":"2023-07-13T22:24:20.653416+0000","last_unstale":"2023-07-13T22:24:20.673028+0000","last_undegraded":"2023-07-13T22:24:20.673028+0000","last_fullsized":"2023-07-13T22:24:20.673028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:03.725412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.685245+0000","last_change":"2023-07-13T22:24:19.641018+0000","last_active":"2023-07-13T22:24:20.685245+0000","last_peered":"2023-07-13T22:24:20.685245+0000","last_clean":"2023-07-13T22:24:20.685245+0000","last_became_active":"2023-07-13T22:24:19.640570+0000","last_became_peered":"2023-07-13T22:24:19.640570+0000","last_unstale":"2023-07-13T22:24:20.685245+0000","last_undegraded":"2023-07-13T22:24:20.685245+0000","last_fullsized":"2023-07-13T22:24:20.685245+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:52:53.361933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.670679+0000","last_change":"2023-07-13T22:24:20.654840+0000","last_active":"2023-07-13T22:24:20.670679+0000","last_peered":"2023-07-13T22:24:20.670679+0000","last_clean":"2023-07-13T22:24:20.670679+0000","last_became_active":"2023-07-13T22:24:20.653388+0000","last_became_peered":"2023-07-13T22:24:20.653388+0000","last_unstale":"2023-07-13T22:24:20.670679+0000","last_undegraded":"2023-07-13T22:24:20.670679+0000","last_fullsized":"2023-07-13T22:24:20.670679+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:43:58.659929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.678197+0000","last_change":"2023-07-13T22:24:19.638081+0000","last_active":"2023-07-13T22:24:20.678197+0000","last_peered":"2023-07-13T22:24:20.678197+0000","last_clean":"2023-07-13T22:24:20.678197+0000","last_became_active":"2023-07-13T22:24:19.637797+0000","last_became_peered":"2023-07-13T22:24:19.637797+0000","last_unstale":"2023-07-13T22:24:20.678197+0000","last_undegraded":"2023-07-13T22:24:20.678197+0000","last_fullsized":"2023-07-13T22:24:20.678197+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:34:34.376728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674383+0000","last_change":"2023-07-13T22:24:20.663741+0000","last_active":"2023-07-13T22:24:20.674383+0000","last_peered":"2023-07-13T22:24:20.674383+0000","last_clean":"2023-07-13T22:24:20.674383+0000","last_became_active":"2023-07-13T22:24:20.663412+0000","last_became_peered":"2023-07-13T22:24:20.663412+0000","last_unstale":"2023-07-13T22:24:20.674383+0000","last_undegraded":"2023-07-13T22:24:20.674383+0000","last_fullsized":"2023-07-13T22:24:20.674383+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:37:36.477800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646138+0000","last_change":"2023-07-13T22:24:19.632013+0000","last_active":"2023-07-13T22:24:20.646138+0000","last_peered":"2023-07-13T22:24:20.646138+0000","last_clean":"2023-07-13T22:24:20.646138+0000","last_became_active":"2023-07-13T22:24:19.631753+0000","last_became_peered":"2023-07-13T22:24:19.631753+0000","last_unstale":"2023-07-13T22:24:20.646138+0000","last_undegraded":"2023-07-13T22:24:20.646138+0000","last_fullsized":"2023-07-13T22:24:20.646138+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:16.863810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.671691+0000","last_change":"2023-07-13T22:24:20.666281+0000","last_active":"2023-07-13T22:24:20.671691+0000","last_peered":"2023-07-13T22:24:20.671691+0000","last_clean":"2023-07-13T22:24:20.671691+0000","last_became_active":"2023-07-13T22:24:20.665958+0000","last_became_peered":"2023-07-13T22:24:20.665958+0000","last_unstale":"2023-07-13T22:24:20.671691+0000","last_undegraded":"2023-07-13T22:24:20.671691+0000","last_fullsized":"2023-07-13T22:24:20.671691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:35:03.770783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651218+0000","last_change":"2023-07-13T22:24:19.634708+0000","last_active":"2023-07-13T22:24:20.651218+0000","last_peered":"2023-07-13T22:24:20.651218+0000","last_clean":"2023-07-13T22:24:20.651218+0000","last_became_active":"2023-07-13T22:24:19.634441+0000","last_became_peered":"2023-07-13T22:24:19.634441+0000","last_unstale":"2023-07-13T22:24:20.651218+0000","last_undegraded":"2023-07-13T22:24:20.651218+0000","last_fullsized":"2023-07-13T22:24:20.651218+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:54:26.521094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.670353+0000","last_change":"2023-07-13T22:24:20.666631+0000","last_active":"2023-07-13T22:24:20.670353+0000","last_peered":"2023-07-13T22:24:20.670353+0000","last_clean":"2023-07-13T22:24:20.670353+0000","last_became_active":"2023-07-13T22:24:20.666340+0000","last_became_peered":"2023-07-13T22:24:20.666340+0000","last_unstale":"2023-07-13T22:24:20.670353+0000","last_undegraded":"2023-07-13T22:24:20.670353+0000","last_fullsized":"2023-07-13T22:24:20.670353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:06.435575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.678103+0000","last_change":"2023-07-13T22:24:19.636732+0000","last_active":"2023-07-13T22:24:20.678103+0000","last_peered":"2023-07-13T22:24:20.678103+0000","last_clean":"2023-07-13T22:24:20.678103+0000","last_became_active":"2023-07-13T22:24:19.636424+0000","last_became_peered":"2023-07-13T22:24:19.636424+0000","last_unstale":"2023-07-13T22:24:20.678103+0000","last_undegraded":"2023-07-13T22:24:20.678103+0000","last_fullsized":"2023-07-13T22:24:20.678103+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:25:01.024244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665791+0000","last_change":"2023-07-13T22:24:20.658997+0000","last_active":"2023-07-13T22:24:20.665791+0000","last_peered":"2023-07-13T22:24:20.665791+0000","last_clean":"2023-07-13T22:24:20.665791+0000","last_became_active":"2023-07-13T22:24:20.658720+0000","last_became_peered":"2023-07-13T22:24:20.658720+0000","last_unstale":"2023-07-13T22:24:20.665791+0000","last_undegraded":"2023-07-13T22:24:20.665791+0000","last_fullsized":"2023-07-13T22:24:20.665791+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:22:23.276499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677126+0000","last_change":"2023-07-13T22:24:19.633257+0000","last_active":"2023-07-13T22:24:20.677126+0000","last_peered":"2023-07-13T22:24:20.677126+0000","last_clean":"2023-07-13T22:24:20.677126+0000","last_became_active":"2023-07-13T22:24:19.632974+0000","last_became_peered":"2023-07-13T22:24:19.632974+0000","last_unstale":"2023-07-13T22:24:20.677126+0000","last_undegraded":"2023-07-13T22:24:20.677126+0000","last_fullsized":"2023-07-13T22:24:20.677126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:23:27.968915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664463+0000","last_change":"2023-07-13T22:24:20.659660+0000","last_active":"2023-07-13T22:24:20.664463+0000","last_peered":"2023-07-13T22:24:20.664463+0000","last_clean":"2023-07-13T22:24:20.664463+0000","last_became_active":"2023-07-13T22:24:20.659331+0000","last_became_peered":"2023-07-13T22:24:20.659331+0000","last_unstale":"2023-07-13T22:24:20.664463+0000","last_undegraded":"2023-07-13T22:24:20.664463+0000","last_fullsized":"2023-07-13T22:24:20.664463+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:35:26.938575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647190+0000","last_change":"2023-07-13T22:24:19.633234+0000","last_active":"2023-07-13T22:24:20.647190+0000","last_peered":"2023-07-13T22:24:20.647190+0000","last_clean":"2023-07-13T22:24:20.647190+0000","last_became_active":"2023-07-13T22:24:19.632900+0000","last_became_peered":"2023-07-13T22:24:19.632900+0000","last_unstale":"2023-07-13T22:24:20.647190+0000","last_undegraded":"2023-07-13T22:24:20.647190+0000","last_fullsized":"2023-07-13T22:24:20.647190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:57:37.505398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.674722+0000","last_change":"2023-07-13T22:24:20.660619+0000","last_active":"2023-07-13T22:24:20.674722+0000","last_peered":"2023-07-13T22:24:20.674722+0000","last_clean":"2023-07-13T22:24:20.674722+0000","last_became_active":"2023-07-13T22:24:20.660361+0000","last_became_peered":"2023-07-13T22:24:20.660361+0000","last_unstale":"2023-07-13T22:24:20.674722+0000","last_undegraded":"2023-07-13T22:24:20.674722+0000","last_fullsized":"2023-07-13T22:24:20.674722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:04:29.444634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651212+0000","last_change":"2023-07-13T22:24:19.637527+0000","last_active":"2023-07-13T22:24:20.651212+0000","last_peered":"2023-07-13T22:24:20.651212+0000","last_clean":"2023-07-13T22:24:20.651212+0000","last_became_active":"2023-07-13T22:24:19.637206+0000","last_became_peered":"2023-07-13T22:24:19.637206+0000","last_unstale":"2023-07-13T22:24:20.651212+0000","last_undegraded":"2023-07-13T22:24:20.651212+0000","last_fullsized":"2023-07-13T22:24:20.651212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:32:56.744052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675163+0000","last_change":"2023-07-13T22:24:20.669324+0000","last_active":"2023-07-13T22:24:20.675163+0000","last_peered":"2023-07-13T22:24:20.675163+0000","last_clean":"2023-07-13T22:24:20.675163+0000","last_became_active":"2023-07-13T22:24:20.668923+0000","last_became_peered":"2023-07-13T22:24:20.668923+0000","last_unstale":"2023-07-13T22:24:20.675163+0000","last_undegraded":"2023-07-13T22:24:20.675163+0000","last_fullsized":"2023-07-13T22:24:20.675163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:27:40.319250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651705+0000","last_change":"2023-07-13T22:24:19.641241+0000","last_active":"2023-07-13T22:24:20.651705+0000","last_peered":"2023-07-13T22:24:20.651705+0000","last_clean":"2023-07-13T22:24:20.651705+0000","last_became_active":"2023-07-13T22:24:19.640989+0000","last_became_peered":"2023-07-13T22:24:19.640989+0000","last_unstale":"2023-07-13T22:24:20.651705+0000","last_undegraded":"2023-07-13T22:24:20.651705+0000","last_fullsized":"2023-07-13T22:24:20.651705+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:00:27.927042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675128+0000","last_change":"2023-07-13T22:24:20.671011+0000","last_active":"2023-07-13T22:24:20.675128+0000","last_peered":"2023-07-13T22:24:20.675128+0000","last_clean":"2023-07-13T22:24:20.675128+0000","last_became_active":"2023-07-13T22:24:20.670725+0000","last_became_peered":"2023-07-13T22:24:20.670725+0000","last_unstale":"2023-07-13T22:24:20.675128+0000","last_undegraded":"2023-07-13T22:24:20.675128+0000","last_fullsized":"2023-07-13T22:24:20.675128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:05:24.951741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.652377+0000","last_change":"2023-07-13T22:24:19.634141+0000","last_active":"2023-07-13T22:24:20.652377+0000","last_peered":"2023-07-13T22:24:20.652377+0000","last_clean":"2023-07-13T22:24:20.652377+0000","last_became_active":"2023-07-13T22:24:19.633831+0000","last_became_peered":"2023-07-13T22:24:19.633831+0000","last_unstale":"2023-07-13T22:24:20.652377+0000","last_undegraded":"2023-07-13T22:24:20.652377+0000","last_fullsized":"2023-07-13T22:24:20.652377+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:10:27.068619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.670645+0000","last_change":"2023-07-13T22:24:20.656557+0000","last_active":"2023-07-13T22:24:20.670645+0000","last_peered":"2023-07-13T22:24:20.670645+0000","last_clean":"2023-07-13T22:24:20.670645+0000","last_became_active":"2023-07-13T22:24:20.656290+0000","last_became_peered":"2023-07-13T22:24:20.656290+0000","last_unstale":"2023-07-13T22:24:20.670645+0000","last_undegraded":"2023-07-13T22:24:20.670645+0000","last_fullsized":"2023-07-13T22:24:20.670645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:17.866267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647850+0000","last_change":"2023-07-13T22:24:19.631443+0000","last_active":"2023-07-13T22:24:20.647850+0000","last_peered":"2023-07-13T22:24:20.647850+0000","last_clean":"2023-07-13T22:24:20.647850+0000","last_became_active":"2023-07-13T22:24:19.631131+0000","last_became_peered":"2023-07-13T22:24:19.631131+0000","last_unstale":"2023-07-13T22:24:20.647850+0000","last_undegraded":"2023-07-13T22:24:20.647850+0000","last_fullsized":"2023-07-13T22:24:20.647850+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:33:42.430192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673145+0000","last_change":"2023-07-13T22:24:20.670685+0000","last_active":"2023-07-13T22:24:20.673145+0000","last_peered":"2023-07-13T22:24:20.673145+0000","last_clean":"2023-07-13T22:24:20.673145+0000","last_became_active":"2023-07-13T22:24:20.670340+0000","last_became_peered":"2023-07-13T22:24:20.670340+0000","last_unstale":"2023-07-13T22:24:20.673145+0000","last_undegraded":"2023-07-13T22:24:20.673145+0000","last_fullsized":"2023-07-13T22:24:20.673145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:09:41.848324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.647340+0000","last_change":"2023-07-13T22:24:19.639638+0000","last_active":"2023-07-13T22:24:20.647340+0000","last_peered":"2023-07-13T22:24:20.647340+0000","last_clean":"2023-07-13T22:24:20.647340+0000","last_became_active":"2023-07-13T22:24:19.639258+0000","last_became_peered":"2023-07-13T22:24:19.639258+0000","last_unstale":"2023-07-13T22:24:20.647340+0000","last_undegraded":"2023-07-13T22:24:20.647340+0000","last_fullsized":"2023-07-13T22:24:20.647340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:11:17.164796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.656256+0000","last_change":"2023-07-13T22:24:20.652957+0000","last_active":"2023-07-13T22:24:20.656256+0000","last_peered":"2023-07-13T22:24:20.656256+0000","last_clean":"2023-07-13T22:24:20.656256+0000","last_became_active":"2023-07-13T22:24:20.652586+0000","last_became_peered":"2023-07-13T22:24:20.652586+0000","last_unstale":"2023-07-13T22:24:20.656256+0000","last_undegraded":"2023-07-13T22:24:20.656256+0000","last_fullsized":"2023-07-13T22:24:20.656256+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:59:12.441359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651382+0000","last_change":"2023-07-13T22:24:19.638493+0000","last_active":"2023-07-13T22:24:20.651382+0000","last_peered":"2023-07-13T22:24:20.651382+0000","last_clean":"2023-07-13T22:24:20.651382+0000","last_became_active":"2023-07-13T22:24:19.638176+0000","last_became_peered":"2023-07-13T22:24:19.638176+0000","last_unstale":"2023-07-13T22:24:20.651382+0000","last_undegraded":"2023-07-13T22:24:20.651382+0000","last_fullsized":"2023-07-13T22:24:20.651382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:34:49.979804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.678232+0000","last_change":"2023-07-13T22:24:20.671891+0000","last_active":"2023-07-13T22:24:20.678232+0000","last_peered":"2023-07-13T22:24:20.678232+0000","last_clean":"2023-07-13T22:24:20.678232+0000","last_became_active":"2023-07-13T22:24:20.671537+0000","last_became_peered":"2023-07-13T22:24:20.671537+0000","last_unstale":"2023-07-13T22:24:20.678232+0000","last_undegraded":"2023-07-13T22:24:20.678232+0000","last_fullsized":"2023-07-13T22:24:20.678232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:19:16.072624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.682396+0000","last_change":"2023-07-13T22:24:19.633687+0000","last_active":"2023-07-13T22:24:20.682396+0000","last_peered":"2023-07-13T22:24:20.682396+0000","last_clean":"2023-07-13T22:24:20.682396+0000","last_became_active":"2023-07-13T22:24:19.633257+0000","last_became_peered":"2023-07-13T22:24:19.633257+0000","last_unstale":"2023-07-13T22:24:20.682396+0000","last_undegraded":"2023-07-13T22:24:20.682396+0000","last_fullsized":"2023-07-13T22:24:20.682396+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:25:05.194710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":90,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646956+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:24:20.646956+0000","last_peered":"2023-07-13T22:24:20.646956+0000","last_clean":"2023-07-13T22:24:20.646956+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:24:20.646956+0000","last_undegraded":"2023-07-13T22:24:20.646956+0000","last_fullsized":"2023-07-13T22:24:20.646956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:16:39.194415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675182+0000","last_change":"2023-07-13T22:24:20.670008+0000","last_active":"2023-07-13T22:24:20.675182+0000","last_peered":"2023-07-13T22:24:20.675182+0000","last_clean":"2023-07-13T22:24:20.675182+0000","last_became_active":"2023-07-13T22:24:20.669670+0000","last_became_peered":"2023-07-13T22:24:20.669670+0000","last_unstale":"2023-07-13T22:24:20.675182+0000","last_undegraded":"2023-07-13T22:24:20.675182+0000","last_fullsized":"2023-07-13T22:24:20.675182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:40:42.360427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646831+0000","last_change":"2023-07-13T22:24:19.637537+0000","last_active":"2023-07-13T22:24:20.646831+0000","last_peered":"2023-07-13T22:24:20.646831+0000","last_clean":"2023-07-13T22:24:20.646831+0000","last_became_active":"2023-07-13T22:24:19.637204+0000","last_became_peered":"2023-07-13T22:24:19.637204+0000","last_unstale":"2023-07-13T22:24:20.646831+0000","last_undegraded":"2023-07-13T22:24:20.646831+0000","last_fullsized":"2023-07-13T22:24:20.646831+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:48:04.706795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646919+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:24:20.646919+0000","last_peered":"2023-07-13T22:24:20.646919+0000","last_clean":"2023-07-13T22:24:20.646919+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:24:20.646919+0000","last_undegraded":"2023-07-13T22:24:20.646919+0000","last_fullsized":"2023-07-13T22:24:20.646919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:29:12.003391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.665703+0000","last_change":"2023-07-13T22:24:20.659090+0000","last_active":"2023-07-13T22:24:20.665703+0000","last_peered":"2023-07-13T22:24:20.665703+0000","last_clean":"2023-07-13T22:24:20.665703+0000","last_became_active":"2023-07-13T22:24:20.658783+0000","last_became_peered":"2023-07-13T22:24:20.658783+0000","last_unstale":"2023-07-13T22:24:20.665703+0000","last_undegraded":"2023-07-13T22:24:20.665703+0000","last_fullsized":"2023-07-13T22:24:20.665703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:55:25.391773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675803+0000","last_change":"2023-07-13T22:24:19.638375+0000","last_active":"2023-07-13T22:24:20.675803+0000","last_peered":"2023-07-13T22:24:20.675803+0000","last_clean":"2023-07-13T22:24:20.675803+0000","last_became_active":"2023-07-13T22:24:19.638041+0000","last_became_peered":"2023-07-13T22:24:19.638041+0000","last_unstale":"2023-07-13T22:24:20.675803+0000","last_undegraded":"2023-07-13T22:24:20.675803+0000","last_fullsized":"2023-07-13T22:24:20.675803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:00:51.354498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":92,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.650625+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:24:20.650625+0000","last_peered":"2023-07-13T22:24:20.650625+0000","last_clean":"2023-07-13T22:24:20.650625+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:24:20.650625+0000","last_undegraded":"2023-07-13T22:24:20.650625+0000","last_fullsized":"2023-07-13T22:24:20.650625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:22.528910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":236,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.650704+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:24:20.650704+0000","last_peered":"2023-07-13T22:24:20.650704+0000","last_clean":"2023-07-13T22:24:20.650704+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:24:20.650704+0000","last_undegraded":"2023-07-13T22:24:20.650704+0000","last_fullsized":"2023-07-13T22:24:20.650704+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:55:13.775565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.664984+0000","last_change":"2023-07-13T22:24:20.656270+0000","last_active":"2023-07-13T22:24:20.664984+0000","last_peered":"2023-07-13T22:24:20.664984+0000","last_clean":"2023-07-13T22:24:20.664984+0000","last_became_active":"2023-07-13T22:24:20.655968+0000","last_became_peered":"2023-07-13T22:24:20.655968+0000","last_unstale":"2023-07-13T22:24:20.664984+0000","last_undegraded":"2023-07-13T22:24:20.664984+0000","last_fullsized":"2023-07-13T22:24:20.664984+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:11:46.990340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646291+0000","last_change":"2023-07-13T22:24:19.630466+0000","last_active":"2023-07-13T22:24:20.646291+0000","last_peered":"2023-07-13T22:24:20.646291+0000","last_clean":"2023-07-13T22:24:20.646291+0000","last_became_active":"2023-07-13T22:24:19.630164+0000","last_became_peered":"2023-07-13T22:24:19.630164+0000","last_unstale":"2023-07-13T22:24:20.646291+0000","last_undegraded":"2023-07-13T22:24:20.646291+0000","last_fullsized":"2023-07-13T22:24:20.646291+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:43:57.657473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":88,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.651526+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:24:20.651526+0000","last_peered":"2023-07-13T22:24:20.651526+0000","last_clean":"2023-07-13T22:24:20.651526+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:24:20.651526+0000","last_undegraded":"2023-07-13T22:24:20.651526+0000","last_fullsized":"2023-07-13T22:24:20.651526+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:01:30.260860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.675403+0000","last_change":"2023-07-13T22:24:20.671710+0000","last_active":"2023-07-13T22:24:20.675403+0000","last_peered":"2023-07-13T22:24:20.675403+0000","last_clean":"2023-07-13T22:24:20.675403+0000","last_became_active":"2023-07-13T22:24:20.671407+0000","last_became_peered":"2023-07-13T22:24:20.671407+0000","last_unstale":"2023-07-13T22:24:20.675403+0000","last_undegraded":"2023-07-13T22:24:20.675403+0000","last_fullsized":"2023-07-13T22:24:20.675403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:09:41.848324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.646296+0000","last_change":"2023-07-13T22:24:19.635176+0000","last_active":"2023-07-13T22:24:20.646296+0000","last_peered":"2023-07-13T22:24:20.646296+0000","last_clean":"2023-07-13T22:24:20.646296+0000","last_became_active":"2023-07-13T22:24:19.634877+0000","last_became_peered":"2023-07-13T22:24:19.634877+0000","last_unstale":"2023-07-13T22:24:20.646296+0000","last_undegraded":"2023-07-13T22:24:20.646296+0000","last_fullsized":"2023-07-13T22:24:20.646296+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:17:20.831144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.650872+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:24:20.650872+0000","last_peered":"2023-07-13T22:24:20.650872+0000","last_clean":"2023-07-13T22:24:20.650872+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:24:20.650872+0000","last_undegraded":"2023-07-13T22:24:20.650872+0000","last_fullsized":"2023-07-13T22:24:20.650872+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:45:27.883604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.668914+0000","last_change":"2023-07-13T22:24:20.662676+0000","last_active":"2023-07-13T22:24:20.668914+0000","last_peered":"2023-07-13T22:24:20.668914+0000","last_clean":"2023-07-13T22:24:20.668914+0000","last_became_active":"2023-07-13T22:24:20.662205+0000","last_became_peered":"2023-07-13T22:24:20.662205+0000","last_unstale":"2023-07-13T22:24:20.668914+0000","last_undegraded":"2023-07-13T22:24:20.668914+0000","last_fullsized":"2023-07-13T22:24:20.668914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:47:51.868302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.649185+0000","last_change":"2023-07-13T22:24:19.635553+0000","last_active":"2023-07-13T22:24:20.649185+0000","last_peered":"2023-07-13T22:24:20.649185+0000","last_clean":"2023-07-13T22:24:20.649185+0000","last_became_active":"2023-07-13T22:24:19.635241+0000","last_became_peered":"2023-07-13T22:24:19.635241+0000","last_unstale":"2023-07-13T22:24:20.649185+0000","last_undegraded":"2023-07-13T22:24:20.649185+0000","last_fullsized":"2023-07-13T22:24:20.649185+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:56:46.919012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.644756+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:24:20.644756+0000","last_peered":"2023-07-13T22:24:20.644756+0000","last_clean":"2023-07-13T22:24:20.644756+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:24:20.644756+0000","last_undegraded":"2023-07-13T22:24:20.644756+0000","last_fullsized":"2023-07-13T22:24:20.644756+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:42:57.664224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677498+0000","last_change":"2023-07-13T22:24:20.674656+0000","last_active":"2023-07-13T22:24:20.677498+0000","last_peered":"2023-07-13T22:24:20.677498+0000","last_clean":"2023-07-13T22:24:20.677498+0000","last_became_active":"2023-07-13T22:24:20.674081+0000","last_became_peered":"2023-07-13T22:24:20.674081+0000","last_unstale":"2023-07-13T22:24:20.677498+0000","last_undegraded":"2023-07-13T22:24:20.677498+0000","last_fullsized":"2023-07-13T22:24:20.677498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:49:09.132236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.677544+0000","last_change":"2023-07-13T22:24:19.640309+0000","last_active":"2023-07-13T22:24:20.677544+0000","last_peered":"2023-07-13T22:24:20.677544+0000","last_clean":"2023-07-13T22:24:20.677544+0000","last_became_active":"2023-07-13T22:24:19.639931+0000","last_became_peered":"2023-07-13T22:24:19.639931+0000","last_unstale":"2023-07-13T22:24:20.677544+0000","last_undegraded":"2023-07-13T22:24:20.677544+0000","last_fullsized":"2023-07-13T22:24:20.677544+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:19:51.386379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.658867+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:24:20.658867+0000","last_peered":"2023-07-13T22:24:20.658867+0000","last_clean":"2023-07-13T22:24:20.658867+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:24:20.658867+0000","last_undegraded":"2023-07-13T22:24:20.658867+0000","last_fullsized":"2023-07-13T22:24:20.658867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:49:21.460134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.682214+0000","last_change":"2023-07-13T22:24:20.678000+0000","last_active":"2023-07-13T22:24:20.682214+0000","last_peered":"2023-07-13T22:24:20.682214+0000","last_clean":"2023-07-13T22:24:20.682214+0000","last_became_active":"2023-07-13T22:24:20.677184+0000","last_became_peered":"2023-07-13T22:24:20.677184+0000","last_unstale":"2023-07-13T22:24:20.682214+0000","last_undegraded":"2023-07-13T22:24:20.682214+0000","last_fullsized":"2023-07-13T22:24:20.682214+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:19.610349+0000","last_clean_scrub_stamp":"2023-07-13T22:24:19.610349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:10:26.737746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.673645+0000","last_change":"2023-07-13T22:24:19.631396+0000","last_active":"2023-07-13T22:24:20.673645+0000","last_peered":"2023-07-13T22:24:20.673645+0000","last_clean":"2023-07-13T22:24:20.673645+0000","last_became_active":"2023-07-13T22:24:19.631135+0000","last_became_peered":"2023-07-13T22:24:19.631135+0000","last_unstale":"2023-07-13T22:24:20.673645+0000","last_undegraded":"2023-07-13T22:24:20.673645+0000","last_fullsized":"2023-07-13T22:24:20.673645+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:24:18.607893+0000","last_clean_scrub_stamp":"2023-07-13T22:24:18.607893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:25:01.024244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-07-13T22:24:20.658461+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:24:20.658461+0000","last_peered":"2023-07-13T22:24:20.658461+0000","last_clean":"2023-07-13T22:24:20.658461+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:24:20.658461+0000","last_undegraded":"2023-07-13T22:24:20.658461+0000","last_fullsized":"2023-07-13T22:24:20.658461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:31:18.078004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1314816,"data_stored":907481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.624,"15min":0.623},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.88,"5min":1.115,"15min":1.115},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86799999999999999,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77500000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.67500000000000004,"15min":0.71099999999999997},"min":{"1min":0.40100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90300000000000002,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.496,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.81599999999999995},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94499999999999995,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.76100000000000001,"15min":0.81200000000000006},"min":{"1min":0.45000000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.879,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.67800000000000005,"15min":0.68600000000000005},"min":{"1min":0.45000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95199999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76100000000000001,"15min":0.76500000000000001},"min":{"1min":0.57299999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95299999999999996,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88700000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0860000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93999999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1359999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.751}]},{"osd":6,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.56899999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.57399999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":737280,"data_stored":324275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.745},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.46100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85999999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.46200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85099999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66400000000000003,"15min":0.65900000000000003},"min":{"1min":0.55100000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.65500000000000003,"15min":0.65800000000000003},"min":{"1min":0.48999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81200000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65300000000000002,"15min":0.67300000000000004},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.439,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.877}]},{"osd":5,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78000000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92300000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.74299999999999999,"15min":0.76100000000000001},"min":{"1min":0.49299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.95099999999999996,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.53700000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88300000000000001,"5min":1.111,"15min":1.111},"last":0.65500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1314816,"data_stored":907481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":1.952,"15min":1.952},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64300000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.52800000000000002,"15min":0.56200000000000006},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61299999999999999,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.443},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58499999999999996,"15min":0.61899999999999999},"min":{"1min":0.39000000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.67100000000000004,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61599999999999999,"15min":0.63600000000000001},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76000000000000001,"5min":2.008,"15min":2.008},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68500000000000005,"5min":0.87,"15min":0.87},"last":0.496}]},{"osd":4,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79200000000000004,"15min":0.82699999999999996},"min":{"1min":0.59299999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91300000000000003,"5min":2.028,"15min":2.028},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77800000000000002,"15min":0.81299999999999994},"min":{"1min":0.55200000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84799999999999998,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.75,"15min":0.76800000000000002},"min":{"1min":0.625,"5min":0.44,"15min":0.44},"max":{"1min":0.97699999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.82999999999999996,"15min":0.873},"min":{"1min":0.56599999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.60599999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0169999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.84899999999999998,"15min":0.89700000000000002},"min":{"1min":0.55900000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.94899999999999995,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.91700000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80900000000000005,"15min":0.84099999999999997},"min":{"1min":0.52200000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96399999999999997,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.875},{"interface":"front","average":{"1min":0.745,"5min":0.80100000000000005,"15min":0.83399999999999996},"min":{"1min":0.63300000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.042,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":729088,"data_stored":323255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.60399999999999998,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.026,"5min":11.791,"15min":11.791},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.419,"5min":4.657,"15min":4.657},"last":0.497}]},{"osd":2,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63400000000000001,"15min":0.61199999999999999},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.012,"5min":4.5389999999999997,"15min":4.5389999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":1.0609999999999999,"15min":1.1419999999999999},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81200000000000006,"5min":13.398999999999999,"15min":13.398999999999999},"last":0.378}]},{"osd":3,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56499999999999995,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.37,"15min":0.37},"max":{"1min":0.89400000000000002,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58999999999999997,"15min":0.56599999999999995},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96099999999999997,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.438}]},{"osd":4,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":1.298,"15min":1.4059999999999999},"min":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.82499999999999996,"5min":13.667999999999999,"15min":13.667999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80200000000000005,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.083,"5min":1.776,"15min":1.776},"last":0.755}]},{"osd":5,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":1.3169999999999999,"15min":1.4179999999999999},"min":{"1min":0.61699999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88400000000000001,"5min":13.349,"15min":13.349},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.89200000000000002,"15min":0.89700000000000002},"min":{"1min":0.63400000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93600000000000005,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.85399999999999998,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97599999999999998,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.752,"5min":1.2390000000000001,"15min":1.3899999999999999},"min":{"1min":0.64400000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.89800000000000002,"5min":11.679,"15min":11.679},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":1.327,"15min":1.4259999999999999},"min":{"1min":0.61699999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.94499999999999995,"5min":13.298,"15min":13.298},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.96899999999999997,"15min":0.91900000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.054,"5min":11.456,"15min":11.456},"last":0.84699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":716800,"data_stored":316127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.52500000000000002,"15min":0.57199999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61399999999999999,"5min":2.282,"15min":2.282},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46300000000000002,"15min":0.46999999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.46200000000000002,"15min":0.47399999999999998},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.58399999999999996,"15min":0.623},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65400000000000003,"5min":2.319,"15min":2.319},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.63200000000000001,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79000000000000004,"5min":2.351,"15min":2.351},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82699999999999996,"15min":0.86899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.95699999999999996,"5min":2.238,"15min":2.238},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.52000000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.518,"15min":1.518},"last":0.621}]},{"osd":5,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1299999999999999,"5min":1.325,"15min":1.325},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.83799999999999997,"15min":0.88200000000000001},"min":{"1min":0.59099999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.0249999999999999,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.80000000000000004,"15min":0.81599999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91000000000000003,"5min":1.488,"15min":1.488},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.502,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.996,"5min":1.214,"15min":1.214},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.625,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.02,"5min":1.29,"15min":1.29},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81799999999999995,"15min":0.83999999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0860000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.65600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35860,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335980,"statfs":{"total":96636764160,"available":96600043520,"internally_reserved":0,"allocated":733184,"data_stored":324241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53000000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.443,"15min":0.439},"min":{"1min":0.39000000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58699999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63700000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48099999999999998,"15min":0.48699999999999999},"min":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.56899999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.45700000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.72599999999999998,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.55300000000000005},"min":{"1min":0.44500000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69299999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.73799999999999999},"min":{"1min":0.51100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90900000000000003,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.63500000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95099999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90300000000000002,"5min":1.411,"15min":1.411},"last":1.032},{"interface":"front","average":{"1min":0.755,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.56499999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96899999999999997,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.55900000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93799999999999994,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.85799999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.88600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79200000000000004,"15min":0.81000000000000005},"min":{"1min":0.58999999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.93700000000000006,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.65400000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96799999999999997,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.085}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":745472,"data_stored":325805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.60599999999999998},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84499999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83499999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.495,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86199999999999999,"5min":1.298,"15min":1.298},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.51800000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97999999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82599999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.52600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.034,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.93999999999999995,"15min":1.0780000000000001},"min":{"1min":0.47499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86199999999999999,"5min":10.657999999999999,"15min":10.657999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.92700000000000005,"15min":1.0289999999999999},"min":{"1min":0.55000000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96699999999999997,"5min":8.75,"15min":8.75},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68400000000000005,"15min":0.69799999999999995},"min":{"1min":0.501,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.53800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0109999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.49399999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":1.173,"15min":1.173},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.73299999999999998},"min":{"1min":0.45100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.0509999999999999,"5min":1.099,"15min":1.099},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.92800000000000005,"5min":1.004,"15min":1.004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.54000000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.212,"15min":1.212},"last":0.88300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":737280,"data_stored":324263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81599999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.47899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.52900000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.093,"5min":0.73099999999999998,"15min":0.66900000000000004},"min":{"1min":0.36299999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":8.9930000000000003,"5min":8.9930000000000003,"15min":8.9930000000000003},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.626,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90900000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.80300000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.76000000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.52300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93999999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.78000000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.629,"15min":0.62},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67400000000000004,"15min":0.69699999999999995},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84899999999999998,"5min":0.995,"15min":0.995},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.502,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86699999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:23:50 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.58099999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88900000000000001,"5min":1.02,"15min":1.02},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.52900000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.97299999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:24:25.565 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:24:25.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-13T22:24:25.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:24:25.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:24:25.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.830+0000 7fea957ff700 1 Processor -- start 2023-07-13T22:24:25.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.830+0000 7fea957ff700 1 -- start start 2023-07-13T22:24:25.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.831+0000 7fea957ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fea900c8da0 0x7fea90070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.831+0000 7fea957ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea90070890 0x7fea900aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.831+0000 7fea957ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea900c94a0 con 0x7fea9015a530 2023-07-13T22:24:25.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.831+0000 7fea957ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9006de70 con 0x7fea90070890 2023-07-13T22:24:25.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.831+0000 7fea957ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9006e290 con 0x7fea900c8da0 2023-07-13T22:24:25.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.831+0000 7fea8f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea90070890 0x7fea900aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.831+0000 7fea8effd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fea9015a530 0x7fea900addb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41512/0 (socket says 172.21.15.60:41512) 2023-07-13T22:24:25.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.831+0000 7fea8f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea90070890 0x7fea900aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49584/0 (socket says 172.21.15.60:49584) 2023-07-13T22:24:25.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.831+0000 7fea8effd700 1 -- 172.21.15.60:0/3869589382 learned_addr learned my addr 172.21.15.60:0/3869589382 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:25.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8e7fc700 1 --2- 172.21.15.60:0/3869589382 >> v2:172.21.15.105:3300/0 conn(0x7fea900c8da0 0x7fea90070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8f7fe700 1 -- 172.21.15.60:0/3869589382 >> v2:172.21.15.105:3300/0 conn(0x7fea900c8da0 msgr2=0x7fea90070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8f7fe700 1 --2- 172.21.15.60:0/3869589382 >> v2:172.21.15.105:3300/0 conn(0x7fea900c8da0 0x7fea90070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8f7fe700 1 -- 172.21.15.60:0/3869589382 >> v1:172.21.15.60:6789/0 conn(0x7fea9015a530 legacy=0x7fea900addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8f7fe700 1 -- 172.21.15.60:0/3869589382 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900c91d0 con 0x7fea90070890 2023-07-13T22:24:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8dffb700 1 -- 172.21.15.60:0/3869589382 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405676631 0 0) 0x7fea900c94a0 con 0x7fea9015a530 2023-07-13T22:24:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8f7fe700 1 --2- 172.21.15.60:0/3869589382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 0x7fea900aebe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fea9006b7b0 tx=0x7fea8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebd2b4fafd9dce7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8dffb700 1 -- 172.21.15.60:0/3869589382 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800e070 con 0x7fea90070890 2023-07-13T22:24:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8dffb700 1 -- 172.21.15.60:0/3869589382 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8800bb20 con 0x7fea90070890 2023-07-13T22:24:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.832+0000 7fea8dffb700 1 -- 172.21.15.60:0/3869589382 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88012430 con 0x7fea90070890 2023-07-13T22:24:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.833+0000 7fea957ff700 1 -- 172.21.15.60:0/3869589382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 msgr2=0x7fea900aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.833+0000 7fea957ff700 1 --2- 172.21.15.60:0/3869589382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 0x7fea900aebe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fea9006b7b0 tx=0x7fea8800b330 comp rx=0 tx=0).stop 2023-07-13T22:24:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.833+0000 7fea957ff700 1 -- 172.21.15.60:0/3869589382 shutdown_connections 2023-07-13T22:24:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.833+0000 7fea957ff700 1 --2- 172.21.15.60:0/3869589382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 0x7fea900aebe0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.833+0000 7fea957ff700 1 --2- 172.21.15.60:0/3869589382 >> v2:172.21.15.105:3300/0 conn(0x7fea900c8da0 0x7fea90070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.833+0000 7fea957ff700 1 -- 172.21.15.60:0/3869589382 >> 172.21.15.60:0/3869589382 conn(0x7fea900cf3b0 msgr2=0x7fea900cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:25.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.833+0000 7fea957ff700 1 -- 172.21.15.60:0/3869589382 shutdown_connections 2023-07-13T22:24:25.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.833+0000 7fea957ff700 1 -- 172.21.15.60:0/3869589382 wait complete. 2023-07-13T22:24:25.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.835+0000 7fea957ff700 1 Processor -- start 2023-07-13T22:24:25.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.835+0000 7fea957ff700 1 -- start start 2023-07-13T22:24:25.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea957ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 0x7fea9015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea957ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900c8da0 0x7fea901f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea957ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea9015a530 0x7fea900c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea957ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901f4950 con 0x7fea90070890 2023-07-13T22:24:25.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea957ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900c8840 con 0x7fea900c8da0 2023-07-13T22:24:25.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea957ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9015af70 con 0x7fea9015a530 2023-07-13T22:24:25.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea8effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 0x7fea9015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea8effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 0x7fea9015a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49600/0 (socket says 172.21.15.60:49600) 2023-07-13T22:24:25.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea8effd700 1 -- 172.21.15.60:0/3610619334 learned_addr learned my addr 172.21.15.60:0/3610619334 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:25.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea8e7fc700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900c8da0 0x7fea901f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea8f7fe700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea9015a530 0x7fea900c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.836+0000 7fea8effd700 1 -- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea9015a530 msgr2=0x7fea900c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea8effd700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea9015a530 0x7fea900c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea8effd700 1 -- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900c8da0 msgr2=0x7fea901f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:25.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea8effd700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900c8da0 0x7fea901f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:25.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea8effd700 1 -- 172.21.15.60:0/3610619334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea88009a70 con 0x7fea90070890 2023-07-13T22:24:25.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea8f7fe700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea9015a530 0x7fea900c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:25.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea8e7fc700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900c8da0 0x7fea901f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea8effd700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 0x7fea9015a990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fea8000c550 tx=0x7fea8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea777fe700 1 -- 172.21.15.60:0/3610619334 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80004c80 con 0x7fea90070890 2023-07-13T22:24:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea777fe700 1 -- 172.21.15.60:0/3610619334 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea80004e00 con 0x7fea90070890 2023-07-13T22:24:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea777fe700 1 -- 172.21.15.60:0/3610619334 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80007310 con 0x7fea90070890 2023-07-13T22:24:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9015b250 con 0x7fea90070890 2023-07-13T22:24:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.837+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9015b810 con 0x7fea90070890 2023-07-13T22:24:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.838+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9004e210 con 0x7fea90070890 2023-07-13T22:24:25.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.839+0000 7fea777fe700 1 -- 172.21.15.60:0/3610619334 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea80007470 con 0x7fea90070890 2023-07-13T22:24:25.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.840+0000 7fea777fe700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fea7c06ace0 0x7fea7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:25.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.840+0000 7fea777fe700 1 -- 172.21.15.60:0/3610619334 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea80089230 con 0x7fea90070890 2023-07-13T22:24:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.840+0000 7fea8e7fc700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fea7c06ace0 0x7fea7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.841+0000 7fea8e7fc700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fea7c06ace0 0x7fea7c06d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fea9015da90 tx=0x7fea84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:25.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:25.844+0000 7fea777fe700 1 -- 172.21.15.60:0/3610619334 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fea80054240 con 0x7fea90070890 2023-07-13T22:24:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.136+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fea90158550 con 0x7fea90070890 2023-07-13T22:24:26.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.140+0000 7fea777fe700 1 -- 172.21.15.60:0/3610619334 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fea80057860 con 0x7fea90070890 2023-07-13T22:24:26.123 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:24:26.123 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:24:26.123 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-07-13T22:24:26.123 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-07-13T22:24:26.123 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-13T22:24:26.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.142+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fea7c06ace0 msgr2=0x7fea7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:26.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.142+0000 7fea957ff700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fea7c06ace0 0x7fea7c06d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fea9015da90 tx=0x7fea84008040 comp rx=0 tx=0).stop 2023-07-13T22:24:26.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.142+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 msgr2=0x7fea9015a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:26.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.142+0000 7fea957ff700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 0x7fea9015a990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fea8000c550 tx=0x7fea8000c910 comp rx=0 tx=0).stop 2023-07-13T22:24:26.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.143+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 shutdown_connections 2023-07-13T22:24:26.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.143+0000 7fea957ff700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fea7c06ace0 0x7fea7c06d190 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.143+0000 7fea957ff700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea90070890 0x7fea9015a990 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.143+0000 7fea957ff700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea9015a530 0x7fea900c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.143+0000 7fea957ff700 1 --2- 172.21.15.60:0/3610619334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900c8da0 0x7fea901f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.143+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 >> 172.21.15.60:0/3610619334 conn(0x7fea900cf3b0 msgr2=0x7fea90158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:26.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.143+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 shutdown_connections 2023-07-13T22:24:26.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.143+0000 7fea957ff700 1 -- 172.21.15.60:0/3610619334 wait complete. 2023-07-13T22:24:26.127 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-07-13T22:24:26.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:26.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.363+0000 7fb85ddff700 1 Processor -- start 2023-07-13T22:24:26.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.363+0000 7fb85ddff700 1 -- start start 2023-07-13T22:24:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb85ddff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8580cfca0 0x7fb858167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb85ddff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8580cd950 con 0x7fb8580b8400 2023-07-13T22:24:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb85ddff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8580d0ac0 con 0x7fb8580d3750 2023-07-13T22:24:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb85ddff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8580b87d0 con 0x7fb8580cfca0 2023-07-13T22:24:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb8577fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb8580b8400 0x7fb8580cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41526/0 (socket says 172.21.15.60:41526) 2023-07-13T22:24:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb8577fe700 1 -- 172.21.15.60:0/3084003133 learned_addr learned my addr 172.21.15.60:0/3084003133 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:26.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb857fff700 1 --2- 172.21.15.60:0/3084003133 >> v2:172.21.15.60:3300/0 conn(0x7fb8580cfca0 0x7fb858167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb856ffd700 1 -- 172.21.15.60:0/3084003133 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477875205 0 0) 0x7fb848001110 con 0x7fb8580b8400 2023-07-13T22:24:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb856ffd700 1 -- 172.21.15.60:0/3084003133 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8440035b0 con 0x7fb8580b8400 2023-07-13T22:24:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb857fff700 1 -- 172.21.15.60:0/3084003133 >> v1:172.21.15.105:6789/0 conn(0x7fb8580d3750 legacy=0x7fb8580cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.365+0000 7fb857fff700 1 -- 172.21.15.60:0/3084003133 >> v1:172.21.15.60:6789/0 conn(0x7fb8580b8400 legacy=0x7fb8580cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.366+0000 7fb857fff700 1 -- 172.21.15.60:0/3084003133 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580b4eb0 con 0x7fb8580cfca0 2023-07-13T22:24:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.366+0000 7fb857fff700 1 --2- 172.21.15.60:0/3084003133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580cfca0 0x7fb858167050 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb850009d90 tx=0x7fb85000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a9f5e2f1f7a73f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.366+0000 7fb856ffd700 1 -- 172.21.15.60:0/3084003133 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85000e070 con 0x7fb8580cfca0 2023-07-13T22:24:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.366+0000 7fb856ffd700 1 -- 172.21.15.60:0/3084003133 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85000bb20 con 0x7fb8580cfca0 2023-07-13T22:24:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.366+0000 7fb856ffd700 1 -- 172.21.15.60:0/3084003133 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850012430 con 0x7fb8580cfca0 2023-07-13T22:24:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.367+0000 7fb85ddff700 1 -- 172.21.15.60:0/3084003133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580cfca0 msgr2=0x7fb858167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.367+0000 7fb85ddff700 1 --2- 172.21.15.60:0/3084003133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580cfca0 0x7fb858167050 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb850009d90 tx=0x7fb85000b330 comp rx=0 tx=0).stop 2023-07-13T22:24:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.367+0000 7fb85ddff700 1 -- 172.21.15.60:0/3084003133 shutdown_connections 2023-07-13T22:24:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.367+0000 7fb85ddff700 1 --2- 172.21.15.60:0/3084003133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580cfca0 0x7fb858167050 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.367+0000 7fb85ddff700 1 -- 172.21.15.60:0/3084003133 >> 172.21.15.60:0/3084003133 conn(0x7fb85806b390 msgr2=0x7fb85806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.367+0000 7fb85ddff700 1 -- 172.21.15.60:0/3084003133 shutdown_connections 2023-07-13T22:24:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.367+0000 7fb85ddff700 1 -- 172.21.15.60:0/3084003133 wait complete. 2023-07-13T22:24:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.369+0000 7fb85ddff700 1 Processor -- start 2023-07-13T22:24:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.369+0000 7fb85ddff700 1 -- start start 2023-07-13T22:24:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb85ddff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580b8400 0x7fb858154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb85ddff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8580cfca0 0x7fb8580b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb85ddff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8580d3750 0x7fb85815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb85ddff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580b7730 con 0x7fb8580b8400 2023-07-13T22:24:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb85ddff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb858154b70 con 0x7fb8580d3750 2023-07-13T22:24:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb85ddff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580cb0f0 con 0x7fb8580cfca0 2023-07-13T22:24:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb8577fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580b8400 0x7fb858154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb8577fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580b8400 0x7fb858154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49620/0 (socket says 172.21.15.60:49620) 2023-07-13T22:24:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb8577fe700 1 -- 172.21.15.60:0/3896842928 learned_addr learned my addr 172.21.15.60:0/3896842928 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb857fff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8580d3750 0x7fb85815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:26.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb84ffff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8580cfca0 0x7fb8580b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb8577fe700 1 -- 172.21.15.60:0/3896842928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8580cfca0 msgr2=0x7fb8580b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:26.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb8577fe700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8580cfca0 0x7fb8580b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb8577fe700 1 -- 172.21.15.60:0/3896842928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8580d3750 msgr2=0x7fb85815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.370+0000 7fb8577fe700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8580d3750 0x7fb85815d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.371+0000 7fb8577fe700 1 -- 172.21.15.60:0/3896842928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb850009a70 con 0x7fb8580b8400 2023-07-13T22:24:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.371+0000 7fb8577fe700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580b8400 0x7fb858154630 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb848007fd0 tx=0x7fb84800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.371+0000 7fb854ff9700 1 -- 172.21.15.60:0/3896842928 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848005d70 con 0x7fb8580b8400 2023-07-13T22:24:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.371+0000 7fb854ff9700 1 -- 172.21.15.60:0/3896842928 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8480070d0 con 0x7fb8580b8400 2023-07-13T22:24:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.371+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8580b7450 con 0x7fb8580b8400 2023-07-13T22:24:26.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.371+0000 7fb854ff9700 1 -- 172.21.15.60:0/3896842928 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848007460 con 0x7fb8580b8400 2023-07-13T22:24:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.371+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8580cb670 con 0x7fb8580b8400 2023-07-13T22:24:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.372+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8580caf50 con 0x7fb8580b8400 2023-07-13T22:24:26.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.373+0000 7fb854ff9700 1 -- 172.21.15.60:0/3896842928 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8480075c0 con 0x7fb8580b8400 2023-07-13T22:24:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.374+0000 7fb854ff9700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb83806ace0 0x7fb83806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.374+0000 7fb854ff9700 1 -- 172.21.15.60:0/3896842928 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb848088830 con 0x7fb8580b8400 2023-07-13T22:24:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.374+0000 7fb84ffff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb83806ace0 0x7fb83806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:26.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.375+0000 7fb84ffff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb83806ace0 0x7fb83806d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb85815d090 tx=0x7fb840009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.378+0000 7fb854ff9700 1 -- 172.21.15.60:0/3896842928 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb8580caf50 con 0x7fb8580b8400 2023-07-13T22:24:26.592 INFO:tasks.ceph.mds.a.smithi060.stderr:2023-07-13T22:24:26.609+0000 7ffa22df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:24:26.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.628+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8580cd950 con 0x7fb8580b8400 2023-07-13T22:24:26.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.629+0000 7fb854ff9700 1 -- 172.21.15.60:0/3896842928 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fb8580cd950 con 0x7fb8580b8400 2023-07-13T22:24:26.611 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:26.612 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/711733521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":711733521},{"type":"v1","addr":"172.21.15.105:6834","nonce":711733521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2135065566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":2135065566},{"type":"v1","addr":"172.21.15.105:6835","nonce":2135065566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6052,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4236704538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4236704538},{"type":"v1","addr":"172.21.15.60:6836","nonce":4236704538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:24:20.139470+0000","modified":"2023-07-13T22:24:23.665486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6058":{"gid":6058,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3036622731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3036622731},{"type":"v1","addr":"172.21.15.60:6837","nonce":3036622731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-13T22:24:26.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb83806ace0 msgr2=0x7fb83806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:26.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb83806ace0 0x7fb83806d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb85815d090 tx=0x7fb840009040 comp rx=0 tx=0).stop 2023-07-13T22:24:26.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580b8400 msgr2=0x7fb858154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:26.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580b8400 0x7fb858154630 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb848007fd0 tx=0x7fb84800c7b0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 shutdown_connections 2023-07-13T22:24:26.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb83806ace0 0x7fb83806d190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8580b8400 0x7fb858154630 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8580d3750 0x7fb85815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 --2- 172.21.15.60:0/3896842928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8580cfca0 0x7fb8580b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:26.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 >> 172.21.15.60:0/3896842928 conn(0x7fb85806b390 msgr2=0x7fb8580cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:26.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 shutdown_connections 2023-07-13T22:24:26.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:26.631+0000 7fb85ddff700 1 -- 172.21.15.60:0/3896842928 wait complete. 2023-07-13T22:24:26.615 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 33 2023-07-13T22:24:27.033 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:24:20.139470+0000', 'modified': '2023-07-13T22:24:23.665486+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6058}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6058': {'gid': 6058, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6837/3036622731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6835', 'nonce': 3036622731}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 3036622731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:24:27.034 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:24:27.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-13T22:24:27.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.258+0000 7f2aabdff700 1 Processor -- start 2023-07-13T22:24:27.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.259+0000 7f2aabdff700 1 -- start start 2023-07-13T22:24:27.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aabdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2aa40b8400 0x7f2aa40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aabdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2aa40d3750 0x7f2aa40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aabdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40cfad0 0x7f2aa415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aabdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa40cff50 con 0x7f2aa40d3750 2023-07-13T22:24:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aabdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa40d0ac0 con 0x7f2aa40b8400 2023-07-13T22:24:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aabdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa40b87d0 con 0x7f2aa40cfad0 2023-07-13T22:24:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aa939a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2aa40d3750 0x7f2aa40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aa939a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2aa40d3750 0x7f2aa40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49632/0 (socket says 172.21.15.60:49632) 2023-07-13T22:24:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aa939a700 1 -- 172.21.15.60:0/833136488 learned_addr learned my addr 172.21.15.60:0/833136488 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/833136488 >> v2:172.21.15.105:3300/0 conn(0x7f2aa40b8400 0x7f2aa40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.260+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/833136488 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40cfad0 0x7f2aa415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.261+0000 7f2aa939a700 1 -- 172.21.15.60:0/833136488 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40cfad0 msgr2=0x7f2aa415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.261+0000 7f2aa939a700 1 --2- 172.21.15.60:0/833136488 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40cfad0 0x7f2aa415fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.261+0000 7f2aa939a700 1 -- 172.21.15.60:0/833136488 >> v2:172.21.15.105:3300/0 conn(0x7f2aa40b8400 msgr2=0x7f2aa40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.261+0000 7f2aa939a700 1 --2- 172.21.15.60:0/833136488 >> v2:172.21.15.105:3300/0 conn(0x7f2aa40b8400 0x7f2aa40cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.261+0000 7f2aa939a700 1 -- 172.21.15.60:0/833136488 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa40b4eb0 con 0x7f2aa40d3750 2023-07-13T22:24:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.261+0000 7f2aa939a700 1 --2- 172.21.15.60:0/833136488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40d3750 0x7f2aa40cf590 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40cf560 tx=0x7f2a94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f2b01a8847e31b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.261+0000 7f2aa8b99700 1 -- 172.21.15.60:0/833136488 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9400b870 con 0x7f2aa40d3750 2023-07-13T22:24:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.261+0000 7f2aa8b99700 1 -- 172.21.15.60:0/833136488 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9400b9d0 con 0x7f2aa40d3750 2023-07-13T22:24:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.261+0000 7f2aa8b99700 1 -- 172.21.15.60:0/833136488 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9400bd60 con 0x7f2aa40d3750 2023-07-13T22:24:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.262+0000 7f2aabdff700 1 -- 172.21.15.60:0/833136488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40d3750 msgr2=0x7f2aa40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.262+0000 7f2aabdff700 1 --2- 172.21.15.60:0/833136488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40d3750 0x7f2aa40cf590 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40cf560 tx=0x7f2a94009bd0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.262+0000 7f2aabdff700 1 -- 172.21.15.60:0/833136488 shutdown_connections 2023-07-13T22:24:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.262+0000 7f2aabdff700 1 --2- 172.21.15.60:0/833136488 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40cfad0 0x7f2aa415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.262+0000 7f2aabdff700 1 --2- 172.21.15.60:0/833136488 >> v2:172.21.15.105:3300/0 conn(0x7f2aa40b8400 0x7f2aa40cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.262+0000 7f2aabdff700 1 --2- 172.21.15.60:0/833136488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40d3750 0x7f2aa40cf590 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.262+0000 7f2aabdff700 1 -- 172.21.15.60:0/833136488 >> 172.21.15.60:0/833136488 conn(0x7f2aa406b390 msgr2=0x7f2aa406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.262+0000 7f2aabdff700 1 -- 172.21.15.60:0/833136488 shutdown_connections 2023-07-13T22:24:27.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.262+0000 7f2aabdff700 1 -- 172.21.15.60:0/833136488 wait complete. 2023-07-13T22:24:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.264+0000 7f2aabdff700 1 Processor -- start 2023-07-13T22:24:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.264+0000 7f2aabdff700 1 -- start start 2023-07-13T22:24:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aabdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa40b8400 0x7f2aa415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aabdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40cfad0 0x7f2aa4149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aabdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40d3750 0x7f2aa4154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aabdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa415c970 con 0x7f2aa40cfad0 2023-07-13T22:24:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aabdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa415cf30 con 0x7f2aa40b8400 2023-07-13T22:24:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aabdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa414a4d0 con 0x7f2aa40d3750 2023-07-13T22:24:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa939a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40cfad0 0x7f2aa4149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa939a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40cfad0 0x7f2aa4149f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49646/0 (socket says 172.21.15.60:49646) 2023-07-13T22:24:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa939a700 1 -- 172.21.15.60:0/1962899915 learned_addr learned my addr 172.21.15.60:0/1962899915 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa40b8400 0x7f2aa415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40d3750 0x7f2aa4154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa939a700 1 -- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40d3750 msgr2=0x7f2aa4154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa939a700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40d3750 0x7f2aa4154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa939a700 1 -- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa40b8400 msgr2=0x7f2aa415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa939a700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa40b8400 0x7f2aa415d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa939a700 1 -- 172.21.15.60:0/1962899915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94009550 con 0x7f2aa40cfad0 2023-07-13T22:24:27.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa40b8400 0x7f2aa415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:27.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.265+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40d3750 0x7f2aa4154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:27.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.266+0000 7f2aa939a700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40cfad0 0x7f2aa4149f90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2a94014040 tx=0x7f2a94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.266+0000 7f2a9a7fc700 1 -- 172.21.15.60:0/1962899915 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94004520 con 0x7f2aa40cfad0 2023-07-13T22:24:27.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.266+0000 7f2a9a7fc700 1 -- 172.21.15.60:0/1962899915 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9401e070 con 0x7f2aa40cfad0 2023-07-13T22:24:27.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.266+0000 7f2aabdff700 1 -- 172.21.15.60:0/1962899915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa41555f0 con 0x7f2aa40cfad0 2023-07-13T22:24:27.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.266+0000 7f2a9a7fc700 1 -- 172.21.15.60:0/1962899915 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940038c0 con 0x7f2aa40cfad0 2023-07-13T22:24:27.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.266+0000 7f2aabdff700 1 -- 172.21.15.60:0/1962899915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa415cbc0 con 0x7f2aa40cfad0 2023-07-13T22:24:27.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.268+0000 7f2a9a7fc700 1 -- 172.21.15.60:0/1962899915 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a94003a20 con 0x7f2aa40cfad0 2023-07-13T22:24:27.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.269+0000 7f2a9a7fc700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a8806ace0 0x7f2a8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.269+0000 7f2a9a7fc700 1 -- 172.21.15.60:0/1962899915 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a94020070 con 0x7f2aa40cfad0 2023-07-13T22:24:27.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.270+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a8806ace0 0x7f2a8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.270+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a8806ace0 0x7f2a8806d190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0007a50 tx=0x7f2aa0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.271+0000 7f2abf714b80 1 Processor -- start 2023-07-13T22:24:27.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.271+0000 7f2abf714b80 1 -- start start 2023-07-13T22:24:27.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460880 0x55e0d64905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6462dc0 0x55e0d6490b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d647b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2abf714b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d63daea0 con 0x55e0d6460c50 2023-07-13T22:24:27.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2abf714b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d634e600 con 0x55e0d6462dc0 2023-07-13T22:24:27.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2abf714b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0d63bd910 con 0x55e0d6460880 2023-07-13T22:24:27.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2aaa39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d647b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2aaa39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d647b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49654/0 (socket says 172.21.15.60:49654) 2023-07-13T22:24:27.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2aaa39c700 1 -- 172.21.15.60:0/1397812070 learned_addr learned my addr 172.21.15.60:0/1397812070 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:27.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.272+0000 7f2aa939a700 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6462dc0 0x55e0d6490b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460880 0x55e0d64905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2aaa39c700 1 -- 172.21.15.60:0/1397812070 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460880 msgr2=0x55e0d64905e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460880 0x55e0d64905e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2aaa39c700 1 -- 172.21.15.60:0/1397812070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6462dc0 msgr2=0x55e0d6490b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6462dc0 0x55e0d6490b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2aaa39c700 1 -- 172.21.15.60:0/1397812070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0d61aa790 con 0x55e0d6460c50 2023-07-13T22:24:27.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2aa939a700 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6462dc0 0x55e0d6490b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:27.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d647b120 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c007410 tx=0x7f2a9c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c75f0a1f476001 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2a877fe700 1 -- 172.21.15.60:0/1397812070 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c015670 con 0x55e0d6460c50 2023-07-13T22:24:27.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2a877fe700 1 -- 172.21.15.60:0/1397812070 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c00f050 con 0x55e0d6460c50 2023-07-13T22:24:27.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.273+0000 7f2a877fe700 1 -- 172.21.15.60:0/1397812070 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c015a10 con 0x55e0d6460c50 2023-07-13T22:24:27.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.274+0000 7f2abf714b80 1 -- 172.21.15.60:0/1397812070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 msgr2=0x55e0d647b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.274+0000 7f2abf714b80 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d647b120 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c007410 tx=0x7f2a9c007720 comp rx=0 tx=0).stop 2023-07-13T22:24:27.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.274+0000 7f2abf714b80 1 -- 172.21.15.60:0/1397812070 shutdown_connections 2023-07-13T22:24:27.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.274+0000 7f2abf714b80 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d647b120 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.274+0000 7f2abf714b80 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460880 0x55e0d64905e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.274+0000 7f2abf714b80 1 --2- 172.21.15.60:0/1397812070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6462dc0 0x55e0d6490b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.274+0000 7f2abf714b80 1 -- 172.21.15.60:0/1397812070 >> 172.21.15.60:0/1397812070 conn(0x55e0d645b3b0 msgr2=0x55e0d6466550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:27.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.274+0000 7f2abf714b80 1 -- 172.21.15.60:0/1397812070 shutdown_connections 2023-07-13T22:24:27.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.274+0000 7f2abf714b80 1 -- 172.21.15.60:0/1397812070 wait complete. 2023-07-13T22:24:27.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.275+0000 7f2abf714b80 1 -- start start 2023-07-13T22:24:27.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.275+0000 7f2abf714b80 1 Processor -- start 2023-07-13T22:24:27.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d6479050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460c50 0x55e0d64611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6462dc0 0x55e0d64616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2abf714b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d63b6be0 con 0x55e0d6462dc0 2023-07-13T22:24:27.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2abf714b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d63745b0 con 0x55e0d6460880 2023-07-13T22:24:27.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2abf714b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0d635a910 con 0x55e0d6460c50 2023-07-13T22:24:27.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2abf714b80 10 client.0 _ll_register_callbacks cb 0x55e0d6372b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:27.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2aaa39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6462dc0 0x55e0d64616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2aaa39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6462dc0 0x55e0d64616f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49656/0 (socket says 172.21.15.60:49656) 2023-07-13T22:24:27.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.276+0000 7f2aaa39c700 1 -- 172.21.15.60:0/3645540655 learned_addr learned my addr 172.21.15.60:0/3645540655 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:27.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2aa939a700 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460c50 0x55e0d64611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d6479050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2aaa39c700 1 -- 172.21.15.60:0/3645540655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460c50 msgr2=0x55e0d64611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460c50 0x55e0d64611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2aaa39c700 1 -- 172.21.15.60:0/3645540655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 msgr2=0x55e0d6479050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d6479050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2aaa39c700 1 -- 172.21.15.60:0/3645540655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c009c50 con 0x55e0d6462dc0 2023-07-13T22:24:27.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d6479050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:27.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6462dc0 0x55e0d64616f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c017040 tx=0x7f2a9c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83caf8c775fc52e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2a84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:24:27.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3645540655 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c029630 con 0x55e0d6462dc0 2023-07-13T22:24:27.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2abf714b80 1 -- 172.21.15.60:0/3645540655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0d64589c0 con 0x55e0d6462dc0 2023-07-13T22:24:27.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.277+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3645540655 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a9c024030 con 0x55e0d6462dc0 2023-07-13T22:24:27.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3645540655 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a9c009ab0 con 0x55e0d6462dc0 2023-07-13T22:24:27.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2abf714b80 10 client.6453 fetch_fsmap learned FSMap version 33 2023-07-13T22:24:27.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2abf714b80 1 -- 172.21.15.60:0/3645540655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55e0d62970a0 con 0x55e0d6462dc0 2023-07-13T22:24:27.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3645540655 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0d64589c0 con 0x55e0d6462dc0 2023-07-13T22:24:27.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3645540655 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2a9c029790 con 0x55e0d6462dc0 2023-07-13T22:24:27.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2abf714b80 10 client.6453 fetch_fsmap finished waiting for FSMap version 33 2023-07-13T22:24:27.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2abf714b80 10 client.6453 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:24:27.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2abf714b80 20 client.6453 populate_metadata read hostname 'smithi060' 2023-07-13T22:24:27.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] conn(0x55e0d6492ee0 0x55e0d64932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2abf714b80 4 client.6453 mds_command: new command op to 6058 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:24:27.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2abf714b80 1 -- 172.21.15.60:0/3645540655 --> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0d6435170 con 0x55e0d6492ee0 2023-07-13T22:24:27.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.278+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] conn(0x55e0d6492ee0 0x55e0d64932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.279+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] conn(0x55e0d6492ee0 0x55e0d64932b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3a352146075718 server_cookie=c4ab7e5c1b3ef4d1 in_seq=0 out_seq=0 2023-07-13T22:24:27.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.279+0000 7f2a84ff9700 10 client.6453 ms_handle_connect on v2:172.21.15.60:6835/3036622731 2023-07-13T22:24:27.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3645540655 <== mds.0 v2:172.21.15.60:6835/3036622731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e0d6435170 con 0x55e0d6492ee0 2023-07-13T22:24:27.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2a84ff9700 10 client.6453 handle_command_reply: tid=0 2023-07-13T22:24:27.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 client.6453 shutdown 2023-07-13T22:24:27.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 -- 172.21.15.60:0/3645540655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6462dc0 msgr2=0x55e0d64616f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6462dc0 0x55e0d64616f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c017040 tx=0x7f2a9c004b40 comp rx=0 tx=0).stop 2023-07-13T22:24:27.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 -- 172.21.15.60:0/3645540655 shutdown_connections 2023-07-13T22:24:27.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6462dc0 0x55e0d64616f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] conn(0x55e0d6492ee0 0x55e0d64932b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d6479050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3645540655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6460c50 0x55e0d64611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2a84ff9700 0 client.6453 ms_handle_reset on v2:172.21.15.60:6835/3036622731 2023-07-13T22:24:27.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 -- 172.21.15.60:0/3645540655 >> 172.21.15.60:0/3645540655 conn(0x55e0d645b3b0 msgr2=0x55e0d647be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:27.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 -- 172.21.15.60:0/3645540655 shutdown_connections 2023-07-13T22:24:27.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.280+0000 7f2abf714b80 1 -- 172.21.15.60:0/3645540655 wait complete. 2023-07-13T22:24:27.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.281+0000 7f2abf714b80 20 client.6453 trim_cache size 0 max 16384 2023-07-13T22:24:27.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.299+0000 7f2abf714b80 1 Processor -- start 2023-07-13T22:24:27.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.299+0000 7f2abf714b80 1 -- start start 2023-07-13T22:24:27.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.300+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d645f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.300+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d645fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.300+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.300+0000 7f2abf714b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d61ab390 con 0x55e0d6460c50 2023-07-13T22:24:27.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.300+0000 7f2abf714b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d64645b0 con 0x55e0d6460880 2023-07-13T22:24:27.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.300+0000 7f2abf714b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0d6467320 con 0x55e0d6462dc0 2023-07-13T22:24:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.300+0000 7f2aa939a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d645fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.300+0000 7f2aa939a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d645fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49662/0 (socket says 172.21.15.60:49662) 2023-07-13T22:24:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.300+0000 7f2aa939a700 1 -- 172.21.15.60:0/666638010 learned_addr learned my addr 172.21.15.60:0/666638010 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aa939a700 1 -- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 msgr2=0x55e0d64932b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d645f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aa939a700 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64932b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aa939a700 1 -- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 msgr2=0x55e0d645f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aa939a700 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d645f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aa939a700 1 -- 172.21.15.60:0/666638010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0d637cb10 con 0x55e0d6460c50 2023-07-13T22:24:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d645f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2aa939a700 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d645fb80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2a94059730 tx=0x7f2a94006df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99f81acc77da76 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2a87fff700 1 -- 172.21.15.60:0/666638010 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94061af0 con 0x55e0d6460c50 2023-07-13T22:24:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2a87fff700 1 -- 172.21.15.60:0/666638010 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a940578e0 con 0x55e0d6460c50 2023-07-13T22:24:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.301+0000 7f2a87fff700 1 -- 172.21.15.60:0/666638010 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94058c90 con 0x55e0d6460c50 2023-07-13T22:24:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.302+0000 7f2abf714b80 1 -- 172.21.15.60:0/666638010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 msgr2=0x55e0d645fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.302+0000 7f2abf714b80 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d645fb80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2a94059730 tx=0x7f2a94006df0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.302+0000 7f2abf714b80 1 -- 172.21.15.60:0/666638010 shutdown_connections 2023-07-13T22:24:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.302+0000 7f2abf714b80 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.302+0000 7f2abf714b80 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460880 0x55e0d645f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.302+0000 7f2abf714b80 1 --2- 172.21.15.60:0/666638010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460c50 0x55e0d645fb80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.302+0000 7f2abf714b80 1 -- 172.21.15.60:0/666638010 >> 172.21.15.60:0/666638010 conn(0x55e0d645b3b0 msgr2=0x55e0d6466520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:27.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.302+0000 7f2abf714b80 1 -- 172.21.15.60:0/666638010 shutdown_connections 2023-07-13T22:24:27.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.302+0000 7f2abf714b80 1 -- 172.21.15.60:0/666638010 wait complete. 2023-07-13T22:24:27.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.303+0000 7f2abf714b80 1 -- start start 2023-07-13T22:24:27.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.303+0000 7f2abf714b80 1 Processor -- start 2023-07-13T22:24:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460880 0x55e0d64acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460c50 0x55e0d6511a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2abf714b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2abf714b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d6511f60 con 0x55e0d6460880 2023-07-13T22:24:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2abf714b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d648d850 con 0x55e0d6460c50 2023-07-13T22:24:27.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2abf714b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0d6493340 con 0x55e0d6462dc0 2023-07-13T22:24:27.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2abf714b80 10 client.0 _ll_register_callbacks cb 0x7f2a9c0157a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:27.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa9b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460880 0x55e0d64acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa9b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460880 0x55e0d64acbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49670/0 (socket says 172.21.15.60:49670) 2023-07-13T22:24:27.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa9b9b700 1 -- 172.21.15.60:0/3747450489 learned_addr learned my addr 172.21.15.60:0/3747450489 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:27.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa939a700 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460c50 0x55e0d6511a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa9b9b700 1 -- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 msgr2=0x55e0d64a2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64a2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa9b9b700 1 -- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460c50 msgr2=0x55e0d6511a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460c50 0x55e0d6511a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa9b9b700 1 -- 172.21.15.60:0/3747450489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa415cbc0 con 0x55e0d6460880 2023-07-13T22:24:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aaa39c700 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.304+0000 7f2aa939a700 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460c50 0x55e0d6511a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.305+0000 7f2aa9b9b700 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460880 0x55e0d64acbb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0007fd0 tx=0x7f2aa00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2951155a6f5e2957 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.305+0000 7f2a84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:24:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.305+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3747450489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000fe60 con 0x55e0d6460880 2023-07-13T22:24:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.305+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3747450489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2aa0023440 con 0x55e0d6460880 2023-07-13T22:24:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.305+0000 7f2abf714b80 1 -- 172.21.15.60:0/3747450489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a9c00f050 con 0x55e0d6460880 2023-07-13T22:24:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.305+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3747450489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aa0023600 con 0x55e0d6460880 2023-07-13T22:24:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.305+0000 7f2abf714b80 10 client.6459 fetch_fsmap learned FSMap version 33 2023-07-13T22:24:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.305+0000 7f2abf714b80 1 -- 172.21.15.60:0/3747450489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55e0d620b1d0 con 0x55e0d6460880 2023-07-13T22:24:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3747450489 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c00f050 con 0x55e0d6460880 2023-07-13T22:24:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3747450489 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2aa0023760 con 0x55e0d6460880 2023-07-13T22:24:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2abf714b80 10 client.6459 fetch_fsmap finished waiting for FSMap version 33 2023-07-13T22:24:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2abf714b80 10 client.6459 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:24:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2abf714b80 20 client.6459 populate_metadata read hostname 'smithi060' 2023-07-13T22:24:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] conn(0x55e0d648c9a0 0x55e0d648cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2abf714b80 4 client.6459 mds_command: new command op to 6058 tid=0[{"prefix": "status"}] 2023-07-13T22:24:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2abf714b80 1 -- 172.21.15.60:0/3747450489 --> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e0d63afd20 con 0x55e0d648c9a0 2023-07-13T22:24:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2aa939a700 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] conn(0x55e0d648c9a0 0x55e0d648cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2a84ff9700 10 client.6459 ms_handle_connect on v2:172.21.15.60:6835/3036622731 2023-07-13T22:24:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.306+0000 7f2aa939a700 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] conn(0x55e0d648c9a0 0x55e0d648cd70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a8268f5a5a0dfd server_cookie=7a3921c210cbcdb1 in_seq=0 out_seq=0 2023-07-13T22:24:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2a84ff9700 1 -- 172.21.15.60:0/3747450489 <== mds.0 v2:172.21.15.60:6835/3036622731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e0d63afd20 con 0x55e0d648c9a0 2023-07-13T22:24:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2a84ff9700 10 client.6459 handle_command_reply: tid=0 2023-07-13T22:24:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 client.6459 shutdown 2023-07-13T22:24:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 -- 172.21.15.60:0/3747450489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460880 msgr2=0x55e0d64acbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460880 0x55e0d64acbb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0007fd0 tx=0x7f2aa00047c0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 -- 172.21.15.60:0/3747450489 shutdown_connections 2023-07-13T22:24:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d6460880 0x55e0d64acbb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.60:6835/3036622731,v1:172.21.15.60:6837/3036622731] conn(0x55e0d648c9a0 0x55e0d648cd70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55e0d6462dc0 0x55e0d64a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 --2- 172.21.15.60:0/3747450489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0d6460c50 0x55e0d6511a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2a84ff9700 0 client.6459 ms_handle_reset on v2:172.21.15.60:6835/3036622731 2023-07-13T22:24:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 -- 172.21.15.60:0/3747450489 >> 172.21.15.60:0/3747450489 conn(0x55e0d645b3b0 msgr2=0x55e0d6490980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 -- 172.21.15.60:0/3747450489 shutdown_connections 2023-07-13T22:24:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 1 -- 172.21.15.60:0/3747450489 wait complete. 2023-07-13T22:24:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.307+0000 7f2abf714b80 20 client.6459 trim_cache size 0 max 16384 2023-07-13T22:24:27.298 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:24:27.298 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:24:27.298 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:24:27.298 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6058, 2023-07-13T22:24:27.299 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:24:27.299 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:24:27.299 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:24:27.299 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.6672840449999997, 2023-07-13T22:24:27.299 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 33, 2023-07-13T22:24:27.299 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 62, 2023-07-13T22:24:27.299 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 62, 2023-07-13T22:24:27.299 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 12.119427349 2023-07-13T22:24:27.300 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:24:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 -- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a8806ace0 msgr2=0x7f2a8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a8806ace0 0x7f2a8806d190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0007a50 tx=0x7f2aa0008040 comp rx=0 tx=0).stop 2023-07-13T22:24:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 -- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40cfad0 msgr2=0x7f2aa4149f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40cfad0 0x7f2aa4149f90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2a94014040 tx=0x7f2a94004750 comp rx=0 tx=0).stop 2023-07-13T22:24:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 -- 172.21.15.60:0/1962899915 shutdown_connections 2023-07-13T22:24:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a8806ace0 0x7f2a8806d190 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aa40d3750 0x7f2aa4154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa40b8400 0x7f2aa415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 --2- 172.21.15.60:0/1962899915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa40cfad0 0x7f2aa4149f90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 -- 172.21.15.60:0/1962899915 >> 172.21.15.60:0/1962899915 conn(0x7f2aa406b390 msgr2=0x7f2aa40b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 -- 172.21.15.60:0/1962899915 shutdown_connections 2023-07-13T22:24:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.308+0000 7f2aabdff700 1 -- 172.21.15.60:0/1962899915 wait complete. 2023-07-13T22:24:27.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:24:27.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.527+0000 7f06de9ff700 1 Processor -- start 2023-07-13T22:24:27.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.528+0000 7f06de9ff700 1 -- start start 2023-07-13T22:24:27.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.529+0000 7f06de9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 0x7f06d80c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.529+0000 7f06de9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06d8070980 0x7f06d80c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.529+0000 7f06de9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d815d970 con 0x7f06d80c5040 2023-07-13T22:24:27.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.529+0000 7f06de9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d815cd60 con 0x7f06d8070980 2023-07-13T22:24:27.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.529+0000 7f06de9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d815d040 con 0x7f06d8159170 2023-07-13T22:24:27.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.529+0000 7f06d77fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f06d80c5040 0x7f06d8070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41528/0 (socket says 172.21.15.60:41528) 2023-07-13T22:24:27.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.529+0000 7f06d77fe700 1 -- 172.21.15.60:0/99150142 learned_addr learned my addr 172.21.15.60:0/99150142 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:27.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.529+0000 7f06d7fff700 1 --2- 172.21.15.60:0/99150142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 0x7f06d80c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.529+0000 7f06dcf9c700 1 --2- 172.21.15.60:0/99150142 >> v2:172.21.15.105:3300/0 conn(0x7f06d8070980 0x7f06d80c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.530+0000 7f06d6ffd700 1 -- 172.21.15.60:0/99150142 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903410469 0 0) 0x7f06d815d970 con 0x7f06d80c5040 2023-07-13T22:24:27.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.530+0000 7f06d6ffd700 1 -- 172.21.15.60:0/99150142 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06c00057c0 con 0x7f06d80c5040 2023-07-13T22:24:27.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.530+0000 7f06d7fff700 1 -- 172.21.15.60:0/99150142 >> v2:172.21.15.105:3300/0 conn(0x7f06d8070980 msgr2=0x7f06d80c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:27.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.530+0000 7f06d7fff700 1 --2- 172.21.15.60:0/99150142 >> v2:172.21.15.105:3300/0 conn(0x7f06d8070980 0x7f06d80c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.530+0000 7f06d7fff700 1 -- 172.21.15.60:0/99150142 >> v1:172.21.15.60:6789/0 conn(0x7f06d80c5040 legacy=0x7f06d8070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.530+0000 7f06d6ffd700 1 -- 172.21.15.60:0/99150142 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4080660495 0 0) 0x7f06c00057c0 con 0x7f06d80c5040 2023-07-13T22:24:27.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.530+0000 7f06d7fff700 1 -- 172.21.15.60:0/99150142 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d815d320 con 0x7f06d8159170 2023-07-13T22:24:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.530+0000 7f06dcf9c700 1 --2- 172.21.15.60:0/99150142 >> v2:172.21.15.105:3300/0 conn(0x7f06d8070980 0x7f06d80c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.530+0000 7f06d7fff700 1 --2- 172.21.15.60:0/99150142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 0x7f06d80c4b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009840 tx=0x7f06c8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0b4d0aafb0f4969 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06d6ffd700 1 -- 172.21.15.60:0/99150142 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc001670 con 0x7f06d8159170 2023-07-13T22:24:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06d6ffd700 1 -- 172.21.15.60:0/99150142 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8004030 con 0x7f06d8159170 2023-07-13T22:24:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06d6ffd700 1 -- 172.21.15.60:0/99150142 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80043c0 con 0x7f06d8159170 2023-07-13T22:24:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06de9ff700 1 -- 172.21.15.60:0/99150142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 msgr2=0x7f06d80c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06de9ff700 1 --2- 172.21.15.60:0/99150142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 0x7f06d80c4b00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009840 tx=0x7f06c8009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06de9ff700 1 -- 172.21.15.60:0/99150142 shutdown_connections 2023-07-13T22:24:27.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06de9ff700 1 --2- 172.21.15.60:0/99150142 >> v2:172.21.15.105:3300/0 conn(0x7f06d8070980 0x7f06d80c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06de9ff700 1 --2- 172.21.15.60:0/99150142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 0x7f06d80c4b00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06de9ff700 1 -- 172.21.15.60:0/99150142 >> 172.21.15.60:0/99150142 conn(0x7f06d80cf410 msgr2=0x7f06d80cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:27.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06de9ff700 1 -- 172.21.15.60:0/99150142 shutdown_connections 2023-07-13T22:24:27.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.531+0000 7f06de9ff700 1 -- 172.21.15.60:0/99150142 wait complete. 2023-07-13T22:24:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.533+0000 7f06de9ff700 1 Processor -- start 2023-07-13T22:24:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.533+0000 7f06de9ff700 1 -- start start 2023-07-13T22:24:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.534+0000 7f06de9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8070980 0x7f06d81ec470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.534+0000 7f06de9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d80c5040 0x7f06d80ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.534+0000 7f06de9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 0x7f06d81f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.534+0000 7f06de9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d81f33f0 con 0x7f06d8070980 2023-07-13T22:24:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.534+0000 7f06de9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80ac880 con 0x7f06d80c5040 2023-07-13T22:24:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.534+0000 7f06de9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80abfe0 con 0x7f06d8159170 2023-07-13T22:24:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8070980 0x7f06d81ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8070980 0x7f06d81ec470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49684/0 (socket says 172.21.15.60:49684) 2023-07-13T22:24:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d7fff700 1 -- 172.21.15.60:0/3082798837 learned_addr learned my addr 172.21.15.60:0/3082798837 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d77fe700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d80c5040 0x7f06d80ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06dcf9c700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 0x7f06d81f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d7fff700 1 -- 172.21.15.60:0/3082798837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 msgr2=0x7f06d81f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d7fff700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 0x7f06d81f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d7fff700 1 -- 172.21.15.60:0/3082798837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d80c5040 msgr2=0x7f06d80ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d7fff700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d80c5040 0x7f06d80ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d7fff700 1 -- 172.21.15.60:0/3082798837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0009a70 con 0x7f06d8070980 2023-07-13T22:24:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.535+0000 7f06d7fff700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8070980 0x7f06d81ec470 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f06c80090b0 tx=0x7f06c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.536+0000 7f06d4ff9700 1 -- 172.21.15.60:0/3082798837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8003cb0 con 0x7f06d8070980 2023-07-13T22:24:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.536+0000 7f06d4ff9700 1 -- 172.21.15.60:0/3082798837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c801f070 con 0x7f06d8070980 2023-07-13T22:24:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.536+0000 7f06d4ff9700 1 -- 172.21.15.60:0/3082798837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8011870 con 0x7f06d8070980 2023-07-13T22:24:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.536+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c80094a0 con 0x7f06d8070980 2023-07-13T22:24:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.536+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d80ac5a0 con 0x7f06d8070980 2023-07-13T22:24:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.537+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d80ad860 con 0x7f06d8070980 2023-07-13T22:24:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.538+0000 7f06d4ff9700 1 -- 172.21.15.60:0/3082798837 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f06c80119d0 con 0x7f06d8070980 2023-07-13T22:24:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.539+0000 7f06d4ff9700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c406ace0 0x7f06c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.539+0000 7f06d4ff9700 1 -- 172.21.15.60:0/3082798837 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06c8021070 con 0x7f06d8070980 2023-07-13T22:24:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.539+0000 7f06d77fe700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c406ace0 0x7f06c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.539+0000 7f06d77fe700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c406ace0 0x7f06c406d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f06cc000f60 tx=0x7f06cc001e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.543+0000 7f06d4ff9700 1 -- 172.21.15.60:0/3082798837 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f06c808ce60 con 0x7f06d8070980 2023-07-13T22:24:27.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.801+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06d8071440 con 0x7f06d8070980 2023-07-13T22:24:27.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.802+0000 7f06d4ff9700 1 -- 172.21.15.60:0/3082798837 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06c802b070 con 0x7f06d8070980 2023-07-13T22:24:27.785 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:24:27.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c406ace0 msgr2=0x7f06c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c406ace0 0x7f06c406d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f06cc000f60 tx=0x7f06cc001e00 comp rx=0 tx=0).stop 2023-07-13T22:24:27.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8070980 msgr2=0x7f06d81ec470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8070980 0x7f06d81ec470 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f06c80090b0 tx=0x7f06c8004750 comp rx=0 tx=0).stop 2023-07-13T22:24:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 shutdown_connections 2023-07-13T22:24:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c406ace0 0x7f06c406d190 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d8070980 0x7f06d81ec470 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d8159170 0x7f06d81f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 --2- 172.21.15.60:0/3082798837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d80c5040 0x7f06d80ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 >> 172.21.15.60:0/3082798837 conn(0x7f06d80cf410 msgr2=0x7f06d8154290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 shutdown_connections 2023-07-13T22:24:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:27.804+0000 7f06de9ff700 1 -- 172.21.15.60:0/3082798837 wait complete. 2023-07-13T22:24:27.805 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:24:27.806 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:24:27.806 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:24:27.806 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:24:27.806 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:24:27.807 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:24:27.807 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:24:27.807 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:24:27.807 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:24:27.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:27.808 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:24:27.855 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:24:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:24:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:24:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:24:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:27.856 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:24:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:24:27.856 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:24:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:24:27.857 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:24:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:24:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:24:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41506sec preferred_lft 41506sec 2023-07-13T22:24:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:24:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:27.858 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:24:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:24:27.858 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:24:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:24:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:24:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:27.859 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:24:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:24:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:24:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:27.859 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:24:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:24:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:24:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:27.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:27.861 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:24:27.930 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:24:27.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:27.933 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:24:27.982 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:24:27.983 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:24:27.983 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:24:27.983 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:24:27.983 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:24:27.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:27.984 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:24:27.985 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:24:27.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:24:27.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:24:27.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:24:27.986 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:24:28.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:28.159 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:24:28.206 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:24:28.206 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:24:28.207 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:24:28.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:28.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:24:28.279 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:24:28.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:24:28.280 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:24:28.326 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:24:28.326 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:24:28.326 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:24:28.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:24:28.524 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:24:28.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:24:28.594 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:24:28.597 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:24:28.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:28.598 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:24:28.645 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:24:28.646 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:24:28.650 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:24:28.722 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:24:28.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:24:28.794 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:24:28.796 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:24:28.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:28.797 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:24:28.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:24:28.846+0000 7ff03682c5c0 -1 init, newargv = 0x56422134de70 newargc=15 2023-07-13T22:24:28.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[137864]: starting ceph client 2023-07-13T22:24:28.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[137864]: starting fuse 2023-07-13T22:24:28.849 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:24:28.849 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:24:28.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:24:28.850 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:24:28.850 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:24:28.850 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:24:28.851 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:24:28.851 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:28.851 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:24:28.852 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:24:28.852 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:24:28.852 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:24:28.853 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:28.853 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:24:28.853 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:24:28.854 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:24:28.854 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:24:28.855 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:28.856 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:24:28.856 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:24:28.857 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:24:28.857 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:24:28.859 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:24:28.859 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:24:28.860 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:24:28.860 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:24:28.861 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:28.862 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:24:28.862 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:24:29.035 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.137864.asok 2023-07-13T22:24:29.212 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.137864.asok 2023-07-13T22:24:29.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:29.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.137864.asok status 2023-07-13T22:24:29.444 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:24:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:24:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:24:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:24:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:24:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:24:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:24:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "137864", 2023-07-13T22:24:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:24:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:24:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:24:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6474, 2023-07-13T22:24:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:24:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:24:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:24:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6474 2023-07-13T22:24:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:24:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:24:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:24:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4165840831 2023-07-13T22:24:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:24:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:24:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:24:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4165840831 2023-07-13T22:24:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6474 192.168.0.1:0/4165840831", 2023-07-13T22:24:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/4165840831", 2023-07-13T22:24:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:24:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 33, 2023-07-13T22:24:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 62, 2023-07-13T22:24:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:24:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:24:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:24:29.451 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:24:29.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:29.459 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:24:29.459 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:24:29.506 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:24:29.506 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:24:29.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:29.507 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:24:29.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:24:29.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.798+0000 7f034afff700 1 Processor -- start 2023-07-13T22:24:29.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.798+0000 7f034afff700 1 -- start start 2023-07-13T22:24:29.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.799+0000 7f034afff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03440b8570 0x7f03440ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.799+0000 7f034afff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03440d15c0 0x7f034415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.799+0000 7f034afff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03440d09d0 con 0x7f03440d35c0 2023-07-13T22:24:29.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.799+0000 7f034afff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03440d3b30 con 0x7f03440d15c0 2023-07-13T22:24:29.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.799+0000 7f034afff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03440cda10 con 0x7f03440b8570 2023-07-13T22:24:29.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f034959c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03440d15c0 0x7f034415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:29.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f0343fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f03440d35c0 0x7f03440d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41542/0 (socket says 172.21.15.60:41542) 2023-07-13T22:24:29.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f0343fff700 1 -- 172.21.15.60:0/2303813985 learned_addr learned my addr 172.21.15.60:0/2303813985 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:29.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f0348d9b700 1 --2- 172.21.15.60:0/2303813985 >> v2:172.21.15.105:3300/0 conn(0x7f03440b8570 0x7f03440ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:29.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f034959c700 1 -- 172.21.15.60:0/2303813985 >> v2:172.21.15.105:3300/0 conn(0x7f03440b8570 msgr2=0x7f03440ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:29.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f034959c700 1 --2- 172.21.15.60:0/2303813985 >> v2:172.21.15.105:3300/0 conn(0x7f03440b8570 0x7f03440ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f034959c700 1 -- 172.21.15.60:0/2303813985 >> v1:172.21.15.60:6789/0 conn(0x7f03440d35c0 legacy=0x7f03440d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:29.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f034959c700 1 -- 172.21.15.60:0/2303813985 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440cd450 con 0x7f03440d15c0 2023-07-13T22:24:29.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f03437fe700 1 -- 172.21.15.60:0/2303813985 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390957534 0 0) 0x7f03440d09d0 con 0x7f03440d35c0 2023-07-13T22:24:29.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.800+0000 7f034959c700 1 --2- 172.21.15.60:0/2303813985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d15c0 0x7f034415c920 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f034406c6c0 tx=0x7f033c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cafb21ad310de3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:29.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f03437fe700 1 -- 172.21.15.60:0/2303813985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c00e070 con 0x7f03440d15c0 2023-07-13T22:24:29.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f03437fe700 1 -- 172.21.15.60:0/2303813985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c00bb20 con 0x7f03440d15c0 2023-07-13T22:24:29.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f03437fe700 1 -- 172.21.15.60:0/2303813985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c012430 con 0x7f03440d15c0 2023-07-13T22:24:29.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f034afff700 1 -- 172.21.15.60:0/2303813985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d15c0 msgr2=0x7f034415c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:29.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f034afff700 1 --2- 172.21.15.60:0/2303813985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d15c0 0x7f034415c920 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f034406c6c0 tx=0x7f033c00b330 comp rx=0 tx=0).stop 2023-07-13T22:24:29.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f034afff700 1 -- 172.21.15.60:0/2303813985 shutdown_connections 2023-07-13T22:24:29.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f034afff700 1 --2- 172.21.15.60:0/2303813985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d15c0 0x7f034415c920 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f034afff700 1 --2- 172.21.15.60:0/2303813985 >> v2:172.21.15.105:3300/0 conn(0x7f03440b8570 0x7f03440ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f034afff700 1 -- 172.21.15.60:0/2303813985 >> 172.21.15.60:0/2303813985 conn(0x7f0344071600 msgr2=0x7f03440719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:29.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f034afff700 1 -- 172.21.15.60:0/2303813985 shutdown_connections 2023-07-13T22:24:29.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.801+0000 7f034afff700 1 -- 172.21.15.60:0/2303813985 wait complete. 2023-07-13T22:24:29.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.803+0000 7f034afff700 1 Processor -- start 2023-07-13T22:24:29.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.803+0000 7f034afff700 1 -- start start 2023-07-13T22:24:29.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.804+0000 7f034afff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03440b8570 0x7f034415f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.804+0000 7f034afff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03440d15c0 0x7f03441f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.804+0000 7f034afff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d35c0 0x7f03440cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.804+0000 7f034afff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034415f910 con 0x7f03440d35c0 2023-07-13T22:24:29.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.804+0000 7f034afff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03440b6f40 con 0x7f03440b8570 2023-07-13T22:24:29.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.804+0000 7f034afff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03440c8760 con 0x7f03440d15c0 2023-07-13T22:24:29.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.804+0000 7f034959c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d35c0 0x7f03440cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:29.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.804+0000 7f034959c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d35c0 0x7f03440cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49688/0 (socket says 172.21.15.60:49688) 2023-07-13T22:24:29.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f034959c700 1 -- 172.21.15.60:0/266526754 learned_addr learned my addr 172.21.15.60:0/266526754 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:29.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f0348d9b700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03440b8570 0x7f034415f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:29.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f0343fff700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03440d15c0 0x7f03441f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:29.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f034959c700 1 -- 172.21.15.60:0/266526754 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03440d15c0 msgr2=0x7f03441f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:29.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f034959c700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03440d15c0 0x7f03441f1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f034959c700 1 -- 172.21.15.60:0/266526754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03440b8570 msgr2=0x7f034415f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f034959c700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03440b8570 0x7f034415f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f034959c700 1 -- 172.21.15.60:0/266526754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c009a70 con 0x7f03440d35c0 2023-07-13T22:24:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f034959c700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d35c0 0x7f03440cb6f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f03440d3d60 tx=0x7f033c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f03417fa700 1 -- 172.21.15.60:0/266526754 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c012430 con 0x7f03440d35c0 2023-07-13T22:24:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.805+0000 7f03417fa700 1 -- 172.21.15.60:0/266526754 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c020070 con 0x7f03440d35c0 2023-07-13T22:24:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.806+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03440c8a40 con 0x7f03440d35c0 2023-07-13T22:24:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.806+0000 7f03417fa700 1 -- 172.21.15.60:0/266526754 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c004280 con 0x7f03440d35c0 2023-07-13T22:24:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.806+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03440c9000 con 0x7f03440d35c0 2023-07-13T22:24:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.807+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034404e280 con 0x7f03440d35c0 2023-07-13T22:24:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.808+0000 7f03417fa700 1 -- 172.21.15.60:0/266526754 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f033c0043e0 con 0x7f03440d35c0 2023-07-13T22:24:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.809+0000 7f03417fa700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f032806ace0 0x7f032806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.809+0000 7f03417fa700 1 -- 172.21.15.60:0/266526754 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f033c022070 con 0x7f03440d35c0 2023-07-13T22:24:29.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.809+0000 7f0348d9b700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f032806ace0 0x7f032806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:29.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.810+0000 7f0348d9b700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f032806ace0 0x7f032806d190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0338009bc0 tx=0x7f03380091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:29.814+0000 7f03417fa700 1 -- 172.21.15.60:0/266526754 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f033c0565c0 con 0x7f03440d35c0 2023-07-13T22:24:30.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.064+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0344160d90 con 0x7f03440d35c0 2023-07-13T22:24:30.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.065+0000 7f03417fa700 1 -- 172.21.15.60:0/266526754 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f033c0263e0 con 0x7f03440d35c0 2023-07-13T22:24:30.048 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:24:30.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f032806ace0 msgr2=0x7f032806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f032806ace0 0x7f032806d190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0338009bc0 tx=0x7f03380091c0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d35c0 msgr2=0x7f03440cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d35c0 0x7f03440cb6f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f03440d3d60 tx=0x7f033c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 shutdown_connections 2023-07-13T22:24:30.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f032806ace0 0x7f032806d190 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03440d35c0 0x7f03440cb6f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03440b8570 0x7f034415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 --2- 172.21.15.60:0/266526754 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03440d15c0 0x7f03441f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 >> 172.21.15.60:0/266526754 conn(0x7f0344071600 msgr2=0x7f03440d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:30.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 shutdown_connections 2023-07-13T22:24:30.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:30.067+0000 7f034afff700 1 -- 172.21.15.60:0/266526754 wait complete. 2023-07-13T22:24:30.068 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:24:30.068 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:24:30.068 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:24:30.069 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:24:30.069 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:24:30.069 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:24:30.069 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:24:30.070 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:24:30.070 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:24:30.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:30.071 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:24:30.123 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:24:30.123 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:24:30.123 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:24:30.123 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:30.123 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:24:30.124 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:30.124 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:24:30.124 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:30.124 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:24:30.124 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:24:30.124 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:30.124 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:24:30.125 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:24:30.125 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:30.125 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:24:30.125 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:30.125 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:24:30.125 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41501sec preferred_lft 41501sec 2023-07-13T22:24:30.125 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:24:30.125 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:30.126 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:24:30.126 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:24:30.126 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:24:30.126 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:30.126 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:24:30.126 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:30.126 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:24:30.127 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:24:30.127 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:24:30.127 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:30.127 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:24:30.127 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:24:30.127 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::14e3:bfff:fea8:bf57/64 scope link 2023-07-13T22:24:30.127 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:24:30.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:30.129 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:24:30.199 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:24:30.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:30.202 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:24:30.252 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:24:30.252 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:24:30.252 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:24:30.253 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:24:30.253 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:24:30.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:30.254 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:24:30.255 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:24:30.255 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:24:30.255 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:24:30.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:24:30.256 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:24:30.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:30.426 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:24:30.472 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:24:30.472 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:24:30.473 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:24:30.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:30.474 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:24:30.544 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:24:30.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:24:30.545 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:24:30.592 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:24:30.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:24:30.593 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:24:30.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:24:30.788 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:24:30.788 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:24:30.856 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:24:30.858 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:24:30.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:30.859 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:24:30.905 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:24:30.905 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:24:30.949 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:24:31.020 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:24:31.020 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:24:31.089 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:24:31.091 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:24:31.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:31.093 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:24:31.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:24:31.133+0000 7f061da095c0 -1 init, newargv = 0x55e660673f80 newargc=15 2023-07-13T22:24:31.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[122675]: starting ceph client 2023-07-13T22:24:31.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[122675]: starting fuse 2023-07-13T22:24:31.141 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:24:31.141 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:24:31.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:24:31.142 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:24:31.142 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:24:31.142 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:24:31.143 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:24:31.143 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:24:31.144 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:24:31.144 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:24:31.145 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:24:31.146 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:24:31.147 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:24:31.147 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:24:31.148 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:24:31.149 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:24:31.150 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:24:31.150 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:24:31.151 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:24:31.151 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:24:31.152 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:24:31.152 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:24:31.153 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:24:31.154 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:24:31.154 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:24:31.154 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:24:31.155 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:24:31.155 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:24:31.155 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:24:31.323 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.122675.asok 2023-07-13T22:24:31.501 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.122675.asok 2023-07-13T22:24:31.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:31.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.122675.asok status 2023-07-13T22:24:31.729 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:24:31.729 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:24:31.730 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:24:31.730 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:24:31.730 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:24:31.730 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:24:31.730 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:24:31.730 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "122675", 2023-07-13T22:24:31.730 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:24:31.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:24:31.731 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:24:31.731 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:24:31.731 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5717, 2023-07-13T22:24:31.731 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:24:31.731 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:24:31.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:24:31.731 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 5717 2023-07-13T22:24:31.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:24:31.732 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:24:31.732 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:24:31.732 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:24:31.732 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3413448787 2023-07-13T22:24:31.732 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:24:31.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:24:31.732 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:24:31.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:24:31.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:24:31.733 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3413448787 2023-07-13T22:24:31.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:24:31.733 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.5717 192.168.0.1:0/3413448787", 2023-07-13T22:24:31.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/3413448787", 2023-07-13T22:24:31.733 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:24:31.733 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 33, 2023-07-13T22:24:31.734 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 62, 2023-07-13T22:24:31.734 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:24:31.734 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:24:31.734 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:24:31.734 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:24:31.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:31.744 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:24:31.744 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:24:31.790 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:24:31.791 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:24:31.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:31.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:24:31.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-13T22:24:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.134+0000 7f13f3fff700 1 Processor -- start 2023-07-13T22:24:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.134+0000 7f13f3fff700 1 -- start start 2023-07-13T22:24:32.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b8200 0x7f13ec0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f3fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f13ec0b1ac0 0x7f13ec0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f3fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0d1870 con 0x7f13ec0cede0 2023-07-13T22:24:32.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f3fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec0cf370 con 0x7f13ec0b1ac0 2023-07-13T22:24:32.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f3fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0d12b0 con 0x7f13ec0b8200 2023-07-13T22:24:32.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f1d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b8200 0x7f13ec0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f159a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f13ec0b1ac0 0x7f13ec0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f159a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f13ec0b1ac0 0x7f13ec0cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:41752/0 (socket says 172.21.15.60:41752) 2023-07-13T22:24:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f159a700 1 -- 172.21.15.60:0/850698706 learned_addr learned my addr 172.21.15.60:0/850698706 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f1d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b8200 0x7f13ec0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:39588/0 (socket says 172.21.15.60:39588) 2023-07-13T22:24:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.136+0000 7f13f159a700 1 -- 172.21.15.60:0/850698706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b8200 msgr2=0x7f13ec0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.137+0000 7f13f159a700 1 --2- 172.21.15.60:0/850698706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b8200 0x7f13ec0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.137+0000 7f13f159a700 1 -- 172.21.15.60:0/850698706 >> v1:172.21.15.105:6789/0 conn(0x7f13ec0cede0 legacy=0x7f13ec15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.137+0000 7f13f159a700 1 -- 172.21.15.60:0/850698706 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0d1590 con 0x7f13ec0b1ac0 2023-07-13T22:24:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.137+0000 7f13f159a700 1 --2- 172.21.15.60:0/850698706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13ec0b1ac0 0x7f13ec0cfb80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0098f0 tx=0x7f13dc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ed3c138854b199 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.137+0000 7f13f0d99700 1 -- 172.21.15.60:0/850698706 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc011830 con 0x7f13ec0b1ac0 2023-07-13T22:24:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.137+0000 7f13f0d99700 1 -- 172.21.15.60:0/850698706 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc011990 con 0x7f13ec0b1ac0 2023-07-13T22:24:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.137+0000 7f13f0d99700 1 -- 172.21.15.60:0/850698706 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc011d20 con 0x7f13ec0b1ac0 2023-07-13T22:24:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.137+0000 7f13f3fff700 1 -- 172.21.15.60:0/850698706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13ec0b1ac0 msgr2=0x7f13ec0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.137+0000 7f13f3fff700 1 --2- 172.21.15.60:0/850698706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13ec0b1ac0 0x7f13ec0cfb80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f13dc0098f0 tx=0x7f13dc009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.138+0000 7f13f3fff700 1 -- 172.21.15.60:0/850698706 shutdown_connections 2023-07-13T22:24:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.138+0000 7f13f3fff700 1 --2- 172.21.15.60:0/850698706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b8200 0x7f13ec0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.138+0000 7f13f3fff700 1 --2- 172.21.15.60:0/850698706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13ec0b1ac0 0x7f13ec0cfb80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.138+0000 7f13f3fff700 1 -- 172.21.15.60:0/850698706 >> 172.21.15.60:0/850698706 conn(0x7f13ec06b2f0 msgr2=0x7f13ec06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.138+0000 7f13f3fff700 1 -- 172.21.15.60:0/850698706 shutdown_connections 2023-07-13T22:24:32.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.138+0000 7f13f3fff700 1 -- 172.21.15.60:0/850698706 wait complete. 2023-07-13T22:24:32.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.139+0000 7f13f3fff700 1 Processor -- start 2023-07-13T22:24:32.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.140+0000 7f13f3fff700 1 -- start start 2023-07-13T22:24:32.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.140+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b1ac0 0x7f13ec159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13ec0b8200 0x7f13ec1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13ec0cede0 0x7f13ec0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f3fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec1f5d70 con 0x7f13ec0cede0 2023-07-13T22:24:32.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f3fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec1f50f0 con 0x7f13ec0b8200 2023-07-13T22:24:32.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f3fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec1f53d0 con 0x7f13ec0b1ac0 2023-07-13T22:24:32.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13ec0cede0 0x7f13ec0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13ec0cede0 0x7f13ec0c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49704/0 (socket says 172.21.15.60:49704) 2023-07-13T22:24:32.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f259c700 1 -- 172.21.15.60:0/2849318258 learned_addr learned my addr 172.21.15.60:0/2849318258 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:32.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f1d9b700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b1ac0 0x7f13ec159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f159a700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13ec0b8200 0x7f13ec1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f259c700 1 -- 172.21.15.60:0/2849318258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b1ac0 msgr2=0x7f13ec159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f259c700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b1ac0 0x7f13ec159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f259c700 1 -- 172.21.15.60:0/2849318258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13ec0b8200 msgr2=0x7f13ec1531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f259c700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13ec0b8200 0x7f13ec1531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.141+0000 7f13f259c700 1 -- 172.21.15.60:0/2849318258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc009550 con 0x7f13ec0cede0 2023-07-13T22:24:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.142+0000 7f13f259c700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13ec0cede0 0x7f13ec0c8930 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f13e400cb50 tx=0x7f13e400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.142+0000 7f13e27fc700 1 -- 172.21.15.60:0/2849318258 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4003c00 con 0x7f13ec0cede0 2023-07-13T22:24:32.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.142+0000 7f13e27fc700 1 -- 172.21.15.60:0/2849318258 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e4003d60 con 0x7f13ec0cede0 2023-07-13T22:24:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.142+0000 7f13e27fc700 1 -- 172.21.15.60:0/2849318258 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e400aa20 con 0x7f13ec0cede0 2023-07-13T22:24:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.142+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec1f56b0 con 0x7f13ec0cede0 2023-07-13T22:24:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.142+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec15a200 con 0x7f13ec0cede0 2023-07-13T22:24:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.143+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec04e210 con 0x7f13ec0cede0 2023-07-13T22:24:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.144+0000 7f13e27fc700 1 -- 172.21.15.60:0/2849318258 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f13e400ab80 con 0x7f13ec0cede0 2023-07-13T22:24:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.145+0000 7f13e27fc700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13d006ace0 0x7f13d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.145+0000 7f13e27fc700 1 -- 172.21.15.60:0/2849318258 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13e40894a0 con 0x7f13ec0cede0 2023-07-13T22:24:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.145+0000 7f13f1d9b700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13d006ace0 0x7f13d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.146+0000 7f13f1d9b700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13d006ace0 0x7f13d006d190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009be0 tx=0x7f13e8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.150+0000 7f13e27fc700 1 -- 172.21.15.60:0/2849318258 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f13e4054540 con 0x7f13ec0cede0 2023-07-13T22:24:32.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.357+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7f13ec0b4f70 con 0x7f13ec0cede0 2023-07-13T22:24:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.357+0000 7f13e27fc700 1 -- 172.21.15.60:0/2849318258 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v18) v1 ==== 106+0+4 (secure 0 0 0) 0x7f13e4057b60 con 0x7f13ec0cede0 2023-07-13T22:24:32.340 INFO:teuthology.orchestra.run.smithi060.stdout:100 2023-07-13T22:24:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13d006ace0 msgr2=0x7f13d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13d006ace0 0x7f13d006d190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009be0 tx=0x7f13e8009210 comp rx=0 tx=0).stop 2023-07-13T22:24:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13ec0cede0 msgr2=0x7f13ec0c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13ec0cede0 0x7f13ec0c8930 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f13e400cb50 tx=0x7f13e400ce60 comp rx=0 tx=0).stop 2023-07-13T22:24:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 shutdown_connections 2023-07-13T22:24:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f13d006ace0 0x7f13d006d190 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13ec0cede0 0x7f13ec0c8930 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13ec0b1ac0 0x7f13ec159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 --2- 172.21.15.60:0/2849318258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13ec0b8200 0x7f13ec1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 >> 172.21.15.60:0/2849318258 conn(0x7f13ec06b2f0 msgr2=0x7f13ec0d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 shutdown_connections 2023-07-13T22:24:32.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.360+0000 7f13f3fff700 1 -- 172.21.15.60:0/2849318258 wait complete. 2023-07-13T22:24:32.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 200 2023-07-13T22:24:32.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.584+0000 7f5c2e5ff700 1 Processor -- start 2023-07-13T22:24:32.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.584+0000 7f5c2e5ff700 1 -- start start 2023-07-13T22:24:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.585+0000 7f5c2e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280cfd50 0x7f5c280d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.585+0000 7f5c2e5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c280d1870 con 0x7f5c280b1ac0 2023-07-13T22:24:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.585+0000 7f5c2e5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c280cf370 con 0x7f5c280b8200 2023-07-13T22:24:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.585+0000 7f5c2e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280d12b0 con 0x7f5c280cfd50 2023-07-13T22:24:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.585+0000 7f5c2cb9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5c280b1ac0 0x7f5c280d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41558/0 (socket says 172.21.15.60:41558) 2023-07-13T22:24:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.585+0000 7f5c2cb9c700 1 -- 172.21.15.60:0/330443020 learned_addr learned my addr 172.21.15.60:0/330443020 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:32.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.586+0000 7f5c277fe700 1 --2- 172.21.15.60:0/330443020 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280cfd50 0x7f5c280d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.586+0000 7f5c26ffd700 1 -- 172.21.15.60:0/330443020 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137964317 0 0) 0x7f5c280d1870 con 0x7f5c280b1ac0 2023-07-13T22:24:32.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.586+0000 7f5c26ffd700 1 -- 172.21.15.60:0/330443020 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c100035b0 con 0x7f5c280b1ac0 2023-07-13T22:24:32.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.586+0000 7f5c26ffd700 1 -- 172.21.15.60:0/330443020 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901849649 0 0) 0x7f5c280cf370 con 0x7f5c280b8200 2023-07-13T22:24:32.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.586+0000 7f5c26ffd700 1 -- 172.21.15.60:0/330443020 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c280d1870 con 0x7f5c280b8200 2023-07-13T22:24:32.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.586+0000 7f5c277fe700 1 -- 172.21.15.60:0/330443020 >> v1:172.21.15.105:6789/0 conn(0x7f5c280b8200 legacy=0x7f5c280cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.586+0000 7f5c277fe700 1 -- 172.21.15.60:0/330443020 >> v1:172.21.15.60:6789/0 conn(0x7f5c280b1ac0 legacy=0x7f5c280d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.586+0000 7f5c277fe700 1 -- 172.21.15.60:0/330443020 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280d1590 con 0x7f5c280cfd50 2023-07-13T22:24:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.586+0000 7f5c277fe700 1 --2- 172.21.15.60:0/330443020 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280cfd50 0x7f5c280d23c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0098a0 tx=0x7f5c1c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc85f2f68a0f2d52 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c26ffd700 1 -- 172.21.15.60:0/330443020 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c011830 con 0x7f5c280cfd50 2023-07-13T22:24:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c26ffd700 1 -- 172.21.15.60:0/330443020 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c011990 con 0x7f5c280cfd50 2023-07-13T22:24:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c26ffd700 1 -- 172.21.15.60:0/330443020 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c011d20 con 0x7f5c280cfd50 2023-07-13T22:24:32.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/330443020 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280cfd50 msgr2=0x7f5c280d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c2e5ff700 1 --2- 172.21.15.60:0/330443020 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280cfd50 0x7f5c280d23c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0098a0 tx=0x7f5c1c009bb0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/330443020 shutdown_connections 2023-07-13T22:24:32.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c2e5ff700 1 --2- 172.21.15.60:0/330443020 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280cfd50 0x7f5c280d23c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/330443020 >> 172.21.15.60:0/330443020 conn(0x7f5c2806b2f0 msgr2=0x7f5c2806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/330443020 shutdown_connections 2023-07-13T22:24:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.587+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/330443020 wait complete. 2023-07-13T22:24:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.589+0000 7f5c2e5ff700 1 Processor -- start 2023-07-13T22:24:32.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.589+0000 7f5c2e5ff700 1 -- start start 2023-07-13T22:24:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c2e5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c280b1ac0 0x7f5c28153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c2e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280b8200 0x7f5c2815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c2e5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280cfd50 0x7f5c28152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c2e5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280ca350 con 0x7f5c280b1ac0 2023-07-13T22:24:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c2e5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2815f470 con 0x7f5c280cfd50 2023-07-13T22:24:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c2e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28152f80 con 0x7f5c280b8200 2023-07-13T22:24:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c27fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c280b1ac0 0x7f5c28153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c27fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c280b1ac0 0x7f5c28153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49714/0 (socket says 172.21.15.60:49714) 2023-07-13T22:24:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c27fff700 1 -- 172.21.15.60:0/2277552179 learned_addr learned my addr 172.21.15.60:0/2277552179 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.590+0000 7f5c277fe700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280b8200 0x7f5c2815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c2cb9c700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280cfd50 0x7f5c28152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c27fff700 1 -- 172.21.15.60:0/2277552179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280b8200 msgr2=0x7f5c2815ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c27fff700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280b8200 0x7f5c2815ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c27fff700 1 -- 172.21.15.60:0/2277552179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280cfd50 msgr2=0x7f5c28152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c27fff700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280cfd50 0x7f5c28152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c27fff700 1 -- 172.21.15.60:0/2277552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c0094a0 con 0x7f5c280b1ac0 2023-07-13T22:24:32.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c27fff700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c280b1ac0 0x7f5c28153180 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800c520 tx=0x7f5c1800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c24ff9700 1 -- 172.21.15.60:0/2277552179 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180052d0 con 0x7f5c280b1ac0 2023-07-13T22:24:32.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c24ff9700 1 -- 172.21.15.60:0/2277552179 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c18005430 con 0x7f5c280b1ac0 2023-07-13T22:24:32.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c24ff9700 1 -- 172.21.15.60:0/2277552179 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180038c0 con 0x7f5c280b1ac0 2023-07-13T22:24:32.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.591+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2815f6d0 con 0x7f5c280b1ac0 2023-07-13T22:24:32.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.592+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c280c9d00 con 0x7f5c280b1ac0 2023-07-13T22:24:32.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.593+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2804e210 con 0x7f5c280b1ac0 2023-07-13T22:24:32.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.594+0000 7f5c24ff9700 1 -- 172.21.15.60:0/2277552179 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c18003a20 con 0x7f5c280b1ac0 2023-07-13T22:24:32.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.594+0000 7f5c24ff9700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c1406ace0 0x7f5c1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.595+0000 7f5c24ff9700 1 -- 172.21.15.60:0/2277552179 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c18088890 con 0x7f5c280b1ac0 2023-07-13T22:24:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.595+0000 7f5c277fe700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c1406ace0 0x7f5c1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.595+0000 7f5c277fe700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c1406ace0 0x7f5c1406d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c280d0140 tx=0x7f5c1c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.599+0000 7f5c24ff9700 1 -- 172.21.15.60:0/2277552179 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5c180539d0 con 0x7f5c280b1ac0 2023-07-13T22:24:32.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.806+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) v1 -- 0x7f5c280b4f70 con 0x7f5c280b1ac0 2023-07-13T22:24:32.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.809+0000 7f5c24ff9700 1 -- 172.21.15.60:0/2277552179 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v19)=0 v19) v1 ==== 123+0+0 (secure 0 0 0) 0x7f5c18056ff0 con 0x7f5c280b1ac0 2023-07-13T22:24:32.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c1406ace0 msgr2=0x7f5c1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c1406ace0 0x7f5c1406d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c280d0140 tx=0x7f5c1c01c040 comp rx=0 tx=0).stop 2023-07-13T22:24:32.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c280b1ac0 msgr2=0x7f5c28153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c280b1ac0 0x7f5c28153180 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800c520 tx=0x7f5c1800c830 comp rx=0 tx=0).stop 2023-07-13T22:24:32.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 shutdown_connections 2023-07-13T22:24:32.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c1406ace0 0x7f5c1406d190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c280b1ac0 0x7f5c28153180 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c280cfd50 0x7f5c28152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 --2- 172.21.15.60:0/2277552179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c280b8200 0x7f5c2815ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 >> 172.21.15.60:0/2277552179 conn(0x7f5c2806b2f0 msgr2=0x7f5c280b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 shutdown_connections 2023-07-13T22:24:32.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:32.812+0000 7f5c2e5ff700 1 -- 172.21.15.60:0/2277552179 wait complete. 2023-07-13T22:24:32.812 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:24:32.813 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:32.858 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:24:32.859 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:24:32.859 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2023-07-13T22:24:32.859 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:32.860 DEBUG:teuthology.orchestra.run.smithi060:> n = 600 2023-07-13T22:24:32.860 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2023-07-13T22:24:32.860 DEBUG:teuthology.orchestra.run.smithi060:> if 0 > 0: 2023-07-13T22:24:32.861 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2023-07-13T22:24:32.861 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/foo/" 2023-07-13T22:24:32.861 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:32.861 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2023-07-13T22:24:32.862 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2023-07-13T22:24:32.862 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2023-07-13T22:24:32.862 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:32.862 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-07-13T22:24:32.863 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-13T22:24:32.863 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:32.863 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2023-07-13T22:24:32.863 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-13T22:24:32.864 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2023-07-13T22:24:32.864 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2023-07-13T22:24:32.864 DEBUG:teuthology.orchestra.run.smithi060:> if True: 2023-07-13T22:24:32.865 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2023-07-13T22:24:32.865 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2023-07-13T22:24:32.865 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:24:32.865 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2023-07-13T22:24:32.866 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:24:32.866 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:24:32.866 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2023-07-13T22:24:32.866 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(0): 2023-07-13T22:24:32.867 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-13T22:24:32.867 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:24:32.867 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:24:32.867 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2023-07-13T22:24:32.868 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2023-07-13T22:24:32.868 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:24:36.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:24:36.401 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:24:36.401 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:24:36.402 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:24:36.402 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:24:36.402 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:36.402 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:24:36.403 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:24:36.403 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:24:36.403 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:24:36.404 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:36.404 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:24:36.404 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:24:36.404 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:24:36.405 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:24:36.405 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:36.405 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:24:36.406 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:24:36.406 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:24:36.406 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:24:36.406 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:24:36.407 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:24:36.407 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:24:36.407 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:24:36.407 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:24:36.408 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:24:36.408 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:24:36.587 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.137864.asok 2023-07-13T22:24:36.764 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.137864.asok 2023-07-13T22:24:36.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:36.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.137864.asok mds_sessions 2023-07-13T22:24:36.994 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:24:36.994 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6474, 2023-07-13T22:24:36.995 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:24:36.995 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:24:36.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:24:36.995 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6474 2023-07-13T22:24:36.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:36.995 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:24:36.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:24:36.996 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:24:36.996 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4165840831 2023-07-13T22:24:36.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:36.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:36.996 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6474 v1:192.168.0.1:0/4165840831", 2023-07-13T22:24:36.996 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "v1:192.168.0.1:0/4165840831", 2023-07-13T22:24:36.996 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions": [ 2023-07-13T22:24:36.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:36.997 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": 0, 2023-07-13T22:24:36.997 INFO:teuthology.orchestra.run.smithi060.stdout: "addrs": { 2023-07-13T22:24:36.997 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2023-07-13T22:24:36.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:36.997 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2023-07-13T22:24:36.997 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6835", 2023-07-13T22:24:36.998 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3036622731 2023-07-13T22:24:36.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:36.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:36.998 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:24:36.998 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6837", 2023-07-13T22:24:36.998 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3036622731 2023-07-13T22:24:36.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:36.999 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:24:36.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:37.000 INFO:teuthology.orchestra.run.smithi060.stdout: "seq": 1201, 2023-07-13T22:24:37.000 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_gen": 0, 2023-07-13T22:24:37.000 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_ttl": "2023-07-13T22:25:29.851237+0000", 2023-07-13T22:24:37.001 INFO:teuthology.orchestra.run.smithi060.stdout: "last_cap_renew_request": "2023-07-13T22:24:29.851237+0000", 2023-07-13T22:24:37.001 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_renew_seq": 2, 2023-07-13T22:24:37.001 INFO:teuthology.orchestra.run.smithi060.stdout: "num_caps": 100, 2023-07-13T22:24:37.001 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "open" 2023-07-13T22:24:37.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:37.001 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2023-07-13T22:24:37.001 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 33 2023-07-13T22:24:37.002 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:24:37.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.229+0000 7f0469fff700 1 Processor -- start 2023-07-13T22:24:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.229+0000 7f0469fff700 1 -- start start 2023-07-13T22:24:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.230+0000 7f0469fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640706f0 0x7f04640ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.230+0000 7f0469fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046415d970 con 0x7f0464159170 2023-07-13T22:24:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.230+0000 7f0469fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046415cd60 con 0x7f0464070270 2023-07-13T22:24:37.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.230+0000 7f0469fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046415d040 con 0x7f04640706f0 2023-07-13T22:24:37.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04637fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0464159170 0x7f04640c4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44004/0 (socket says 172.21.15.60:44004) 2023-07-13T22:24:37.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04637fe700 1 -- 172.21.15.60:0/1509632975 learned_addr learned my addr 172.21.15.60:0/1509632975 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f0463fff700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640706f0 0x7f04640ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748390020 0 0) 0x7f046415d970 con 0x7f0464159170 2023-07-13T22:24:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f044c0035b0 con 0x7f0464159170 2023-07-13T22:24:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990896993 0 0) 0x7f046415cd60 con 0x7f0464070270 2023-07-13T22:24:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f046415d970 con 0x7f0464070270 2023-07-13T22:24:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3714030420 0 0) 0x7f044c0035b0 con 0x7f0464159170 2023-07-13T22:24:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640706f0 msgr2=0x7f04640ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640706f0 0x7f04640ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 >> v1:172.21.15.105:6789/0 conn(0x7f0464070270 legacy=0x7f04640c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046415d320 con 0x7f0464159170 2023-07-13T22:24:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f0454001670 con 0x7f0464159170 2023-07-13T22:24:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.231+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 >> v1:172.21.15.60:6789/0 conn(0x7f0464159170 legacy=0x7f04640c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04627fc700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f044c003bc0 0x7f044c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04627fc700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640706f0 0x7f044c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04627fc700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c004d90 0x7f044c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046415d040 con 0x7f044c003bc0 2023-07-13T22:24:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c00bac0 con 0x7f044c004d90 2023-07-13T22:24:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c00bc30 con 0x7f04640706f0 2023-07-13T22:24:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04637fe700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f044c003bc0 0x7f044c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f0462ffd700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640706f0 0x7f044c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f0463fff700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c004d90 0x7f044c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04637fe700 1 -- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640706f0 msgr2=0x7f044c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04637fe700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640706f0 0x7f044c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04637fe700 1 -- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c004d90 msgr2=0x7f044c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04637fe700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c004d90 0x7f044c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.232+0000 7f04637fe700 1 -- 172.21.15.60:0/1509632975 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c0035b0 con 0x7f044c003bc0 2023-07-13T22:24:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f04637fe700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f044c003bc0 0x7f044c003fb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f045400c210 tx=0x7f044c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a61bd1ce41ed84 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045400ccf0 con 0x7f044c003bc0 2023-07-13T22:24:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045400ce70 con 0x7f044c003bc0 2023-07-13T22:24:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f04627fc700 1 -- 172.21.15.60:0/1509632975 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0454005b80 con 0x7f044c003bc0 2023-07-13T22:24:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f0469fff700 1 -- 172.21.15.60:0/1509632975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f044c003bc0 msgr2=0x7f044c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f0469fff700 1 --2- 172.21.15.60:0/1509632975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f044c003bc0 0x7f044c003fb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f045400c210 tx=0x7f044c005880 comp rx=0 tx=0).stop 2023-07-13T22:24:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f0463fff700 1 -- 172.21.15.60:0/1509632975 reap_dead start 2023-07-13T22:24:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f0469fff700 1 -- 172.21.15.60:0/1509632975 shutdown_connections 2023-07-13T22:24:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f0469fff700 1 -- 172.21.15.60:0/1509632975 >> 172.21.15.60:0/1509632975 conn(0x7f04640cf410 msgr2=0x7f04640cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:37.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.233+0000 7f0469fff700 1 -- 172.21.15.60:0/1509632975 shutdown_connections 2023-07-13T22:24:37.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.234+0000 7f0469fff700 1 -- 172.21.15.60:0/1509632975 wait complete. 2023-07-13T22:24:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.235+0000 7f0469fff700 1 Processor -- start 2023-07-13T22:24:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.235+0000 7f0469fff700 1 -- start start 2023-07-13T22:24:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.236+0000 7f0469fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c003bc0 0x7f046414be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.236+0000 7f0469fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640ca050 0x7f04640ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.236+0000 7f0469fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0464150570 0x7f04641509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.236+0000 7f0469fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0464152480 con 0x7f0464150570 2023-07-13T22:24:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.236+0000 7f0469fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0464151310 con 0x7f044c003bc0 2023-07-13T22:24:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.236+0000 7f0469fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04640caa00 con 0x7f04640ca050 2023-07-13T22:24:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0463fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0464150570 0x7f04641509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0463fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0464150570 0x7f04641509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52606/0 (socket says 172.21.15.60:52606) 2023-07-13T22:24:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0463fff700 1 -- 172.21.15.60:0/1392111340 learned_addr learned my addr 172.21.15.60:0/1392111340 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f04637fe700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c003bc0 0x7f046414be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0462ffd700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640ca050 0x7f04640ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0463fff700 1 -- 172.21.15.60:0/1392111340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640ca050 msgr2=0x7f04640ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0463fff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640ca050 0x7f04640ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0463fff700 1 -- 172.21.15.60:0/1392111340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c003bc0 msgr2=0x7f046414be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0463fff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c003bc0 0x7f046414be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0463fff700 1 -- 172.21.15.60:0/1392111340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04640c6720 con 0x7f0464150570 2023-07-13T22:24:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0462ffd700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640ca050 0x7f04640ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.237+0000 7f0463fff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0464150570 0x7f04641509f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f04640b0360 tx=0x7f045c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.238+0000 7f044bfff700 1 -- 172.21.15.60:0/1392111340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c00e040 con 0x7f0464150570 2023-07-13T22:24:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.238+0000 7f044bfff700 1 -- 172.21.15.60:0/1392111340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045c00f040 con 0x7f0464150570 2023-07-13T22:24:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.238+0000 7f044bfff700 1 -- 172.21.15.60:0/1392111340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c00fd80 con 0x7f0464150570 2023-07-13T22:24:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.238+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04640c5960 con 0x7f0464150570 2023-07-13T22:24:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.238+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04640c5f20 con 0x7f0464150570 2023-07-13T22:24:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.239+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04640ad860 con 0x7f0464150570 2023-07-13T22:24:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.240+0000 7f044bfff700 1 -- 172.21.15.60:0/1392111340 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f045c022460 con 0x7f0464150570 2023-07-13T22:24:37.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.241+0000 7f044bfff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006acd0 0x7f045006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:37.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.241+0000 7f044bfff700 1 -- 172.21.15.60:0/1392111340 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f045c089a30 con 0x7f0464150570 2023-07-13T22:24:37.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.241+0000 7f04637fe700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006acd0 0x7f045006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:37.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.242+0000 7f04637fe700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006acd0 0x7f045006d180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0454015040 tx=0x7f0454007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:37.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.246+0000 7f044bfff700 1 -- 172.21.15.60:0/1392111340 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f045c054a40 con 0x7f0464150570 2023-07-13T22:24:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.496+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0464071440 con 0x7f0464150570 2023-07-13T22:24:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.496+0000 7f044bfff700 1 -- 172.21.15.60:0/1392111340 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f045c058060 con 0x7f0464150570 2023-07-13T22:24:37.479 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:37.479 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/711733521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":711733521},{"type":"v1","addr":"172.21.15.105:6834","nonce":711733521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2135065566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":2135065566},{"type":"v1","addr":"172.21.15.105:6835","nonce":2135065566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6052,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4236704538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4236704538},{"type":"v1","addr":"172.21.15.60:6836","nonce":4236704538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:24:20.139470+0000","modified":"2023-07-13T22:24:23.665486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6058":{"gid":6058,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3036622731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3036622731},{"type":"v1","addr":"172.21.15.60:6837","nonce":3036622731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-13T22:24:37.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006acd0 msgr2=0x7f045006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006acd0 0x7f045006d180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0454015040 tx=0x7f0454007f90 comp rx=0 tx=0).stop 2023-07-13T22:24:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0464150570 msgr2=0x7f04641509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0464150570 0x7f04641509f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f04640b0360 tx=0x7f045c011f80 comp rx=0 tx=0).stop 2023-07-13T22:24:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 shutdown_connections 2023-07-13T22:24:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006acd0 0x7f045006d180 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0464150570 0x7f04641509f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f044c003bc0 0x7f046414be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 --2- 172.21.15.60:0/1392111340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04640ca050 0x7f04640ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 >> 172.21.15.60:0/1392111340 conn(0x7f04640cf410 msgr2=0x7f046406a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 shutdown_connections 2023-07-13T22:24:37.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:37.499+0000 7f0469fff700 1 -- 172.21.15.60:0/1392111340 wait complete. 2023-07-13T22:24:37.485 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 33 2023-07-13T22:24:37.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-07-13T22:24:37.740 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5717,"entity":{"name":{"type":"client","num":5717},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3413448787}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":6.6059273459999996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9633114724990777,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5717 192.168.0.1:0/3413448787","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"122675","root":"/"}},{"id":6474,"entity":{"name":{"type":"client","num":6474},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":4165840831}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":1133,"uptime":8.8929021929999994,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":488.46186879422021,"halflife":60},"release_caps":{"value":494.83339782731031,"halflife":60},"recall_caps_throttle":{"value":477.77416941812032,"halflife":1.5},"recall_caps_throttle2o":{"value":183.56901156563168,"halflife":0.5},"session_cache_liveness":{"value":1197.3581088995736,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1168,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6474 192.168.0.1:0/4165840831","completed_requests":[{"tid":1169,"created_ino":"0x1000000044c"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"137864","root":"/"}}] 2023-07-13T22:24:37.741 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "122675", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3413448787, "type": "any" }, "name": { "num": 5717, "type": "client" } }, "id": 5717, "inst": "client.5717 192.168.0.1:0/3413448787", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9633114724990777 }, "state": "open", "uptime": 6.605927346 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "137864", "root": "/" }, "completed_requests": [ { "created_ino": "0x1000000044c", "tid": 1169 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4165840831, "type": "any" }, "name": { "num": 6474, "type": "client" } }, "id": 6474, "inst": "client.6474 192.168.0.1:0/4165840831", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 1168, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 488.4618687942202 }, "recall_caps_throttle": { "halflife": 1.5, "value": 477.7741694181203 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 183.56901156563168 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 494.8333978273103 }, "request_load_avg": 1133, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1197.3581088995736 }, "state": "open", "uptime": 8.892902193 } ] 2023-07-13T22:24:37.741 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:24:37.742 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:24:37.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:24:37.838 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:24:37.948 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:24:37.949 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:24:37.949 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:37.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.257+0000 7f6a945ff700 1 Processor -- start 2023-07-13T22:24:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.257+0000 7f6a945ff700 1 -- start start 2023-07-13T22:24:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.258+0000 7f6a945ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b5a00 0x7f6a8c0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.258+0000 7f6a945ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6a8c0cf560 0x7f6a8c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.258+0000 7f6a945ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8c0cd860 con 0x7f6a8c0b1f80 2023-07-13T22:24:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.258+0000 7f6a945ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8c0b2350 con 0x7f6a8c0cf560 2023-07-13T22:24:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.258+0000 7f6a945ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0d2750 con 0x7f6a8c0b5a00 2023-07-13T22:24:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.259+0000 7f6a92b9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6a8c0cf560 0x7f6a8c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.259+0000 7f6a91b9a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b5a00 0x7f6a8c0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.259+0000 7f6a92b9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6a8c0cf560 0x7f6a8c0cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:41334/0 (socket says 172.21.15.60:41334) 2023-07-13T22:24:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.259+0000 7f6a92b9c700 1 -- 172.21.15.60:0/889051289 learned_addr learned my addr 172.21.15.60:0/889051289 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.259+0000 7f6a92b9c700 1 -- 172.21.15.60:0/889051289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b5a00 msgr2=0x7f6a8c0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.259+0000 7f6a92b9c700 1 --2- 172.21.15.60:0/889051289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b5a00 0x7f6a8c0ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.259+0000 7f6a92b9c700 1 -- 172.21.15.60:0/889051289 >> v1:172.21.15.105:6789/0 conn(0x7f6a8c0b1f80 legacy=0x7f6a8c0d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.259+0000 7f6a92b9c700 1 -- 172.21.15.60:0/889051289 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c0cff20 con 0x7f6a8c0cf560 2023-07-13T22:24:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a92b9c700 1 --2- 172.21.15.60:0/889051289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0cf560 0x7f6a8c0cf9e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009e50 tx=0x7f6a8400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1afe7274021ecedf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a91399700 1 -- 172.21.15.60:0/889051289 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400e070 con 0x7f6a8c0cf560 2023-07-13T22:24:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a91399700 1 -- 172.21.15.60:0/889051289 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8400b910 con 0x7f6a8c0cf560 2023-07-13T22:24:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a91399700 1 -- 172.21.15.60:0/889051289 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400bcc0 con 0x7f6a8c0cf560 2023-07-13T22:24:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a945ff700 1 -- 172.21.15.60:0/889051289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0cf560 msgr2=0x7f6a8c0cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a945ff700 1 --2- 172.21.15.60:0/889051289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0cf560 0x7f6a8c0cf9e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009e50 tx=0x7f6a8400b330 comp rx=0 tx=0).stop 2023-07-13T22:24:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a945ff700 1 -- 172.21.15.60:0/889051289 shutdown_connections 2023-07-13T22:24:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a945ff700 1 --2- 172.21.15.60:0/889051289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0cf560 0x7f6a8c0cf9e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a945ff700 1 --2- 172.21.15.60:0/889051289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b5a00 0x7f6a8c0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.260+0000 7f6a945ff700 1 -- 172.21.15.60:0/889051289 >> 172.21.15.60:0/889051289 conn(0x7f6a8c071600 msgr2=0x7f6a8c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.261+0000 7f6a945ff700 1 -- 172.21.15.60:0/889051289 shutdown_connections 2023-07-13T22:24:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.261+0000 7f6a945ff700 1 -- 172.21.15.60:0/889051289 wait complete. 2023-07-13T22:24:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.262+0000 7f6a945ff700 1 Processor -- start 2023-07-13T22:24:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.262+0000 7f6a945ff700 1 -- start start 2023-07-13T22:24:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.263+0000 7f6a945ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b1f80 0x7f6a8c155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.263+0000 7f6a945ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0b5a00 0x7f6a8c1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.263+0000 7f6a945ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8c0cf560 0x7f6a8c15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.263+0000 7f6a945ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0cd030 con 0x7f6a8c0cf560 2023-07-13T22:24:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.263+0000 7f6a945ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c158720 con 0x7f6a8c0b5a00 2023-07-13T22:24:38.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.263+0000 7f6a945ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c158a00 con 0x7f6a8c0b1f80 2023-07-13T22:24:38.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a91b9a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0b5a00 0x7f6a8c1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a92b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8c0cf560 0x7f6a8c15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a9239b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b1f80 0x7f6a8c155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a91b9a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0b5a00 0x7f6a8c1f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:41338/0 (socket says 172.21.15.60:41338) 2023-07-13T22:24:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a91b9a700 1 -- 172.21.15.60:0/727298526 learned_addr learned my addr 172.21.15.60:0/727298526 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a9239b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b1f80 0x7f6a8c155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:34456/0 (socket says 172.21.15.60:34456) 2023-07-13T22:24:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a92b9c700 1 -- 172.21.15.60:0/727298526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b1f80 msgr2=0x7f6a8c155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a92b9c700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b1f80 0x7f6a8c155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a92b9c700 1 -- 172.21.15.60:0/727298526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0b5a00 msgr2=0x7f6a8c1f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a92b9c700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0b5a00 0x7f6a8c1f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a92b9c700 1 -- 172.21.15.60:0/727298526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a84009b30 con 0x7f6a8c0cf560 2023-07-13T22:24:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.264+0000 7f6a92b9c700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8c0cf560 0x7f6a8c15a100 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6a84015000 tx=0x7f6a84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.265+0000 7f6a82ffd700 1 -- 172.21.15.60:0/727298526 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84004840 con 0x7f6a8c0cf560 2023-07-13T22:24:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.265+0000 7f6a82ffd700 1 -- 172.21.15.60:0/727298526 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a840099b0 con 0x7f6a8c0cf560 2023-07-13T22:24:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.265+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c158ce0 con 0x7f6a8c0cf560 2023-07-13T22:24:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.265+0000 7f6a82ffd700 1 -- 172.21.15.60:0/727298526 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8401c680 con 0x7f6a8c0cf560 2023-07-13T22:24:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.265+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c155b70 con 0x7f6a8c0cf560 2023-07-13T22:24:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.266+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c159200 con 0x7f6a8c0cf560 2023-07-13T22:24:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.267+0000 7f6a82ffd700 1 -- 172.21.15.60:0/727298526 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a8401c7e0 con 0x7f6a8c0cf560 2023-07-13T22:24:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.268+0000 7f6a82ffd700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a7006ace0 0x7f6a7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.269+0000 7f6a82ffd700 1 -- 172.21.15.60:0/727298526 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a8408ad60 con 0x7f6a8c0cf560 2023-07-13T22:24:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.269+0000 7f6a9239b700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a7006ace0 0x7f6a7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.269+0000 7f6a9239b700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a7006ace0 0x7f6a7006d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6a88007fd0 tx=0x7f6a88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:38.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.273+0000 7f6a82ffd700 1 -- 172.21.15.60:0/727298526 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6a8c159200 con 0x7f6a8c0cf560 2023-07-13T22:24:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.482+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a8c159200 con 0x7f6a8c0cf560 2023-07-13T22:24:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.483+0000 7f6a82ffd700 1 -- 172.21.15.60:0/727298526 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18199 (secure 0 0 0) 0x7f6a8c159200 con 0x7f6a8c0cf560 2023-07-13T22:24:38.466 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:38.467 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:21.653794+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:24:18.441889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-13T22:24:19.114200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a7006ace0 msgr2=0x7f6a7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a7006ace0 0x7f6a7006d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6a88007fd0 tx=0x7f6a88009040 comp rx=0 tx=0).stop 2023-07-13T22:24:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8c0cf560 msgr2=0x7f6a8c15a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8c0cf560 0x7f6a8c15a100 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6a84015000 tx=0x7f6a84004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 shutdown_connections 2023-07-13T22:24:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a7006ace0 0x7f6a7006d190 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8c0cf560 0x7f6a8c15a100 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a8c0b1f80 0x7f6a8c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 --2- 172.21.15.60:0/727298526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a8c0b5a00 0x7f6a8c1f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 >> 172.21.15.60:0/727298526 conn(0x7f6a8c071600 msgr2=0x7f6a8c0b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 shutdown_connections 2023-07-13T22:24:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.486+0000 7f6a945ff700 1 -- 172.21.15.60:0/727298526 wait complete. 2023-07-13T22:24:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:24:18.441889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:24:19.114200+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:24:38.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:24:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.713+0000 7f12c79ff700 1 Processor -- start 2023-07-13T22:24:38.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.714+0000 7f12c79ff700 1 -- start start 2023-07-13T22:24:38.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.715+0000 7f12c79ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 0x7f12c00d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.715+0000 7f12c79ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f12c00b1ac0 0x7f12c00d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.715+0000 7f12c79ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c00d1870 con 0x7f12c00b8200 2023-07-13T22:24:38.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.715+0000 7f12c79ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c00cf370 con 0x7f12c00b1ac0 2023-07-13T22:24:38.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.715+0000 7f12c79ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00d12b0 con 0x7f12c00cfd50 2023-07-13T22:24:38.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12c5f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f12c00b1ac0 0x7f12c00d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12c5f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f12c00b1ac0 0x7f12c00d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52622/0 (socket says 172.21.15.60:52622) 2023-07-13T22:24:38.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12c4f9a700 1 --2- 172.21.15.60:0/852322900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 0x7f12c00d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12c579b700 1 --1- 172.21.15.60:0/852322900 >> v1:172.21.15.105:6789/0 conn(0x7f12c00b8200 0x7f12c00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:48062/0 (socket says 172.21.15.60:48062) 2023-07-13T22:24:38.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12c5f9c700 1 -- 172.21.15.60:0/852322900 learned_addr learned my addr 172.21.15.60:0/852322900 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:38.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12c5f9c700 1 -- 172.21.15.60:0/852322900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 msgr2=0x7f12c00d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:38.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12c5f9c700 1 --2- 172.21.15.60:0/852322900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 0x7f12c00d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12c5f9c700 1 -- 172.21.15.60:0/852322900 >> v1:172.21.15.105:6789/0 conn(0x7f12c00b8200 legacy=0x7f12c00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12c5f9c700 1 -- 172.21.15.60:0/852322900 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c00d1590 con 0x7f12c00b1ac0 2023-07-13T22:24:38.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.716+0000 7f12b7fff700 1 -- 172.21.15.60:0/852322900 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931773339 0 0) 0x7f12c00d1870 con 0x7f12c00b8200 2023-07-13T22:24:38.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12c5f9c700 1 --2- 172.21.15.60:0/852322900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b1ac0 0x7f12c00d2900 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009e80 tx=0x7f12b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6586f7fd6dc2c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:38.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12b7fff700 1 -- 172.21.15.60:0/852322900 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b800e070 con 0x7f12c00b1ac0 2023-07-13T22:24:38.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12b7fff700 1 -- 172.21.15.60:0/852322900 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b800b8d0 con 0x7f12c00b1ac0 2023-07-13T22:24:38.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12b7fff700 1 -- 172.21.15.60:0/852322900 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b800bc80 con 0x7f12c00b1ac0 2023-07-13T22:24:38.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12c79ff700 1 -- 172.21.15.60:0/852322900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b1ac0 msgr2=0x7f12c00d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12c79ff700 1 --2- 172.21.15.60:0/852322900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b1ac0 0x7f12c00d2900 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009e80 tx=0x7f12b800b330 comp rx=0 tx=0).stop 2023-07-13T22:24:38.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12c79ff700 1 -- 172.21.15.60:0/852322900 shutdown_connections 2023-07-13T22:24:38.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12c79ff700 1 --2- 172.21.15.60:0/852322900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b1ac0 0x7f12c00d2900 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12c79ff700 1 --2- 172.21.15.60:0/852322900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 0x7f12c00d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12c79ff700 1 -- 172.21.15.60:0/852322900 >> 172.21.15.60:0/852322900 conn(0x7f12c006b2f0 msgr2=0x7f12c006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:38.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.717+0000 7f12c79ff700 1 -- 172.21.15.60:0/852322900 shutdown_connections 2023-07-13T22:24:38.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.718+0000 7f12c79ff700 1 -- 172.21.15.60:0/852322900 wait complete. 2023-07-13T22:24:38.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.719+0000 7f12c79ff700 1 Processor -- start 2023-07-13T22:24:38.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.719+0000 7f12c79ff700 1 -- start start 2023-07-13T22:24:38.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.720+0000 7f12c79ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c00b1ac0 0x7f12c015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.720+0000 7f12c79ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b8200 0x7f12c015f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.720+0000 7f12c79ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 0x7f12c0155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.720+0000 7f12c79ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00b72a0 con 0x7f12c00b8200 2023-07-13T22:24:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.720+0000 7f12c79ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c015d970 con 0x7f12c00b1ac0 2023-07-13T22:24:38.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.720+0000 7f12c79ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c00b6900 con 0x7f12c00cfd50 2023-07-13T22:24:38.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c4f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b8200 0x7f12c015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c4f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b8200 0x7f12c015f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52632/0 (socket says 172.21.15.60:52632) 2023-07-13T22:24:38.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c4f9a700 1 -- 172.21.15.60:0/3582555033 learned_addr learned my addr 172.21.15.60:0/3582555033 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:38.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c5f9c700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 0x7f12c0155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c579b700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c00b1ac0 0x7f12c015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c4f9a700 1 -- 172.21.15.60:0/3582555033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 msgr2=0x7f12c0155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c4f9a700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 0x7f12c0155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c4f9a700 1 -- 172.21.15.60:0/3582555033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c00b1ac0 msgr2=0x7f12c015cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c4f9a700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c00b1ac0 0x7f12c015cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c4f9a700 1 -- 172.21.15.60:0/3582555033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b8009b30 con 0x7f12c00b8200 2023-07-13T22:24:38.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c579b700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c00b1ac0 0x7f12c015cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:38.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.721+0000 7f12c4f9a700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b8200 0x7f12c015f320 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f12b000e840 tx=0x7f12b000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:38.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.722+0000 7f12b5ffb700 1 -- 172.21.15.60:0/3582555033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0009590 con 0x7f12c00b8200 2023-07-13T22:24:38.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.722+0000 7f12b5ffb700 1 -- 172.21.15.60:0/3582555033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b0013680 con 0x7f12c00b8200 2023-07-13T22:24:38.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.722+0000 7f12b5ffb700 1 -- 172.21.15.60:0/3582555033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0013a30 con 0x7f12c00b8200 2023-07-13T22:24:38.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.722+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c00b6be0 con 0x7f12c00b8200 2023-07-13T22:24:38.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.722+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c015d010 con 0x7f12c00b8200 2023-07-13T22:24:38.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.723+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c00b73e0 con 0x7f12c00b8200 2023-07-13T22:24:38.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.724+0000 7f12b5ffb700 1 -- 172.21.15.60:0/3582555033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f12b0013b90 con 0x7f12c00b8200 2023-07-13T22:24:38.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.725+0000 7f12b5ffb700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12a406ace0 0x7f12a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:38.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.725+0000 7f12b5ffb700 1 -- 172.21.15.60:0/3582555033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12b0089320 con 0x7f12c00b8200 2023-07-13T22:24:38.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.725+0000 7f12c579b700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12a406ace0 0x7f12a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:38.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.726+0000 7f12c579b700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12a406ace0 0x7f12a406d190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000f60 tx=0x7f12bc007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:38.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.730+0000 7f12b5ffb700 1 -- 172.21.15.60:0/3582555033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f12c00b73e0 con 0x7f12c00b8200 2023-07-13T22:24:38.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.934+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f12c00b73e0 con 0x7f12c00b8200 2023-07-13T22:24:38.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.935+0000 7f12b5ffb700 1 -- 172.21.15.60:0/3582555033 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7f12c00b73e0 con 0x7f12c00b8200 2023-07-13T22:24:38.918 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:24:38.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12a406ace0 msgr2=0x7f12a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12a406ace0 0x7f12a406d190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000f60 tx=0x7f12bc007ca0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b8200 msgr2=0x7f12c015f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:38.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b8200 0x7f12c015f320 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f12b000e840 tx=0x7f12b000eb50 comp rx=0 tx=0).stop 2023-07-13T22:24:38.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 shutdown_connections 2023-07-13T22:24:38.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12a406ace0 0x7f12a406d190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12c00cfd50 0x7f12c0155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12c00b1ac0 0x7f12c015cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 --2- 172.21.15.60:0/3582555033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12c00b8200 0x7f12c015f320 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:38.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.937+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 >> 172.21.15.60:0/3582555033 conn(0x7f12c006b2f0 msgr2=0x7f12c00d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:38.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.938+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 shutdown_connections 2023-07-13T22:24:38.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:38.938+0000 7f12c79ff700 1 -- 172.21.15.60:0/3582555033 wait complete. 2023-07-13T22:24:38.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:24:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.160+0000 7faccc1ff700 1 Processor -- start 2023-07-13T22:24:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.161+0000 7faccc1ff700 1 -- start start 2023-07-13T22:24:39.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.162+0000 7faccc1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7facc40add00 0x7facc406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.162+0000 7faccc1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc40c8e90 con 0x7facc406a5e0 2023-07-13T22:24:39.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.162+0000 7faccc1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc40ae0d0 con 0x7facc40ca9f0 2023-07-13T22:24:39.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.162+0000 7faccc1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc406d3b0 con 0x7facc40add00 2023-07-13T22:24:39.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.162+0000 7facca79c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7facc406a5e0 0x7facc40b1690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44020/0 (socket says 172.21.15.60:44020) 2023-07-13T22:24:39.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.162+0000 7facca79c700 1 -- 172.21.15.60:0/1677068086 learned_addr learned my addr 172.21.15.60:0/1677068086 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:39.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.162+0000 7facc9f9b700 1 --2- 172.21.15.60:0/1677068086 >> v2:172.21.15.105:3300/0 conn(0x7facc40add00 0x7facc406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.163+0000 7facc8f99700 1 -- 172.21.15.60:0/1677068086 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096646572 0 0) 0x7facc40c8e90 con 0x7facc406a5e0 2023-07-13T22:24:39.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.163+0000 7facc8f99700 1 -- 172.21.15.60:0/1677068086 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb00035b0 con 0x7facc406a5e0 2023-07-13T22:24:39.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.163+0000 7facc9f9b700 1 -- 172.21.15.60:0/1677068086 >> v1:172.21.15.105:6789/0 conn(0x7facc40ca9f0 legacy=0x7facc4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.163+0000 7facc8f99700 1 -- 172.21.15.60:0/1677068086 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1836660263 0 0) 0x7facb00035b0 con 0x7facc406a5e0 2023-07-13T22:24:39.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.163+0000 7facc9f9b700 1 -- 172.21.15.60:0/1677068086 >> v1:172.21.15.60:6789/0 conn(0x7facc406a5e0 legacy=0x7facc40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.163+0000 7facc9f9b700 1 -- 172.21.15.60:0/1677068086 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc40c90e0 con 0x7facc40add00 2023-07-13T22:24:39.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.163+0000 7facc9f9b700 1 --2- 172.21.15.60:0/1677068086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc40add00 0x7facc406ce70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7facc0009840 tx=0x7facc0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e4d04290b668c5c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:39.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7facc8f99700 1 -- 172.21.15.60:0/1677068086 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc0019f0 con 0x7facc40add00 2023-07-13T22:24:39.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7facc8f99700 1 -- 172.21.15.60:0/1677068086 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc0004030 con 0x7facc40add00 2023-07-13T22:24:39.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7facc8f99700 1 -- 172.21.15.60:0/1677068086 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc00043c0 con 0x7facc40add00 2023-07-13T22:24:39.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7faccc1ff700 1 -- 172.21.15.60:0/1677068086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc40add00 msgr2=0x7facc406ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7faccc1ff700 1 --2- 172.21.15.60:0/1677068086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc40add00 0x7facc406ce70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7facc0009840 tx=0x7facc0009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:39.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7faccc1ff700 1 -- 172.21.15.60:0/1677068086 shutdown_connections 2023-07-13T22:24:39.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7faccc1ff700 1 --2- 172.21.15.60:0/1677068086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc40add00 0x7facc406ce70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7faccc1ff700 1 -- 172.21.15.60:0/1677068086 >> 172.21.15.60:0/1677068086 conn(0x7facc40cbf50 msgr2=0x7facc40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:39.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7faccc1ff700 1 -- 172.21.15.60:0/1677068086 shutdown_connections 2023-07-13T22:24:39.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.164+0000 7faccc1ff700 1 -- 172.21.15.60:0/1677068086 wait complete. 2023-07-13T22:24:39.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.166+0000 7faccc1ff700 1 Processor -- start 2023-07-13T22:24:39.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.166+0000 7faccc1ff700 1 -- start start 2023-07-13T22:24:39.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.167+0000 7faccc1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc406a5e0 0x7facc4158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.167+0000 7faccc1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40add00 0x7facc40c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.167+0000 7faccc1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc40ca9f0 0x7facc415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.167+0000 7faccc1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c3ee0 con 0x7facc40ca9f0 2023-07-13T22:24:39.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.167+0000 7faccc1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c3260 con 0x7facc406a5e0 2023-07-13T22:24:39.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.167+0000 7faccc1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c3540 con 0x7facc40add00 2023-07-13T22:24:39.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.167+0000 7facca79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc40ca9f0 0x7facc415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.167+0000 7facca79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc40ca9f0 0x7facc415cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52646/0 (socket says 172.21.15.60:52646) 2023-07-13T22:24:39.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.167+0000 7facca79c700 1 -- 172.21.15.60:0/4147662300 learned_addr learned my addr 172.21.15.60:0/4147662300 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:39.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facc9f9b700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc406a5e0 0x7facc4158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facc979a700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40add00 0x7facc40c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facca79c700 1 -- 172.21.15.60:0/4147662300 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40add00 msgr2=0x7facc40c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facca79c700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40add00 0x7facc40c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facca79c700 1 -- 172.21.15.60:0/4147662300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc406a5e0 msgr2=0x7facc4158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facca79c700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc406a5e0 0x7facc4158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facca79c700 1 -- 172.21.15.60:0/4147662300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00094a0 con 0x7facc40ca9f0 2023-07-13T22:24:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facc979a700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40add00 0x7facc40c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facca79c700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc40ca9f0 0x7facc415cb60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7facbc00cca0 tx=0x7facbc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facba7fc700 1 -- 172.21.15.60:0/4147662300 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00abe0 con 0x7facc40ca9f0 2023-07-13T22:24:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facba7fc700 1 -- 172.21.15.60:0/4147662300 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc00ad40 con 0x7facc40ca9f0 2023-07-13T22:24:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7facba7fc700 1 -- 172.21.15.60:0/4147662300 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc007310 con 0x7facc40ca9f0 2023-07-13T22:24:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.168+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc40c3820 con 0x7facc40ca9f0 2023-07-13T22:24:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.169+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc4158790 con 0x7facc40ca9f0 2023-07-13T22:24:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.170+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc40c4020 con 0x7facc40ca9f0 2023-07-13T22:24:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.171+0000 7facba7fc700 1 -- 172.21.15.60:0/4147662300 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7facbc007470 con 0x7facc40ca9f0 2023-07-13T22:24:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.172+0000 7facba7fc700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.172+0000 7facba7fc700 1 -- 172.21.15.60:0/4147662300 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7facbc089250 con 0x7facc40ca9f0 2023-07-13T22:24:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.172+0000 7facc9f9b700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.173+0000 7facc9f9b700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7facc0009b90 tx=0x7facc0008000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.176+0000 7facba7fc700 1 -- 172.21.15.60:0/4147662300 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7facbc0541d0 con 0x7facc40ca9f0 2023-07-13T22:24:39.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.380+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facc404e210 con 0x7facc40ca9f0 2023-07-13T22:24:39.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.380+0000 7facba7fc700 1 -- 172.21.15.60:0/4147662300 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7facbc0577f0 con 0x7facc40ca9f0 2023-07-13T22:24:39.363 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:24:39.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.382+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 msgr2=0x7faca806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.382+0000 7faccc1ff700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7facc0009b90 tx=0x7facc0008000 comp rx=0 tx=0).stop 2023-07-13T22:24:39.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.382+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc40ca9f0 msgr2=0x7facc415cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.382+0000 7faccc1ff700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc40ca9f0 0x7facc415cb60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7facbc00cca0 tx=0x7facbc00a060 comp rx=0 tx=0).stop 2023-07-13T22:24:39.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.382+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 shutdown_connections 2023-07-13T22:24:39.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.383+0000 7faccc1ff700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.383+0000 7faccc1ff700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc40ca9f0 0x7facc415cb60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.383+0000 7faccc1ff700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc406a5e0 0x7facc4158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.383+0000 7faccc1ff700 1 --2- 172.21.15.60:0/4147662300 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40add00 0x7facc40c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.383+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 >> 172.21.15.60:0/4147662300 conn(0x7facc40cbf50 msgr2=0x7facc40734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:39.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.383+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 shutdown_connections 2023-07-13T22:24:39.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.383+0000 7faccc1ff700 1 -- 172.21.15.60:0/4147662300 wait complete. 2023-07-13T22:24:39.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:24:39.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.606+0000 7f40645ff700 1 Processor -- start 2023-07-13T22:24:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.606+0000 7f40645ff700 1 -- start start 2023-07-13T22:24:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.607+0000 7f40645ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f405c070af0 0x7f405c06c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.607+0000 7f40645ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405c0cf640 con 0x7f405c0ca900 2023-07-13T22:24:39.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.607+0000 7f40645ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405c0ccf80 con 0x7f405c159260 2023-07-13T22:24:39.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.607+0000 7f40645ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405c0cada0 con 0x7f405c070af0 2023-07-13T22:24:39.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.607+0000 7f4061b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f405c0ca900 0x7f405c0703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44036/0 (socket says 172.21.15.60:44036) 2023-07-13T22:24:39.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.607+0000 7f4061b9a700 1 -- 172.21.15.60:0/3047627668 learned_addr learned my addr 172.21.15.60:0/3047627668 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:39.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.607+0000 7f4062b9c700 1 --2- 172.21.15.60:0/3047627668 >> v2:172.21.15.105:3300/0 conn(0x7f405c070af0 0x7f405c06c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.608+0000 7f4061399700 1 -- 172.21.15.60:0/3047627668 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465296786 0 0) 0x7f405c0cf640 con 0x7f405c0ca900 2023-07-13T22:24:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.608+0000 7f4061399700 1 -- 172.21.15.60:0/3047627668 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40480035b0 con 0x7f405c0ca900 2023-07-13T22:24:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.608+0000 7f4062b9c700 1 -- 172.21.15.60:0/3047627668 >> v1:172.21.15.105:6789/0 conn(0x7f405c159260 legacy=0x7f405c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.608+0000 7f4062b9c700 1 -- 172.21.15.60:0/3047627668 >> v1:172.21.15.60:6789/0 conn(0x7f405c0ca900 legacy=0x7f405c0703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.608+0000 7f4061399700 1 -- 172.21.15.60:0/3047627668 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 248262494 0 0) 0x7f40480035b0 con 0x7f405c0ca900 2023-07-13T22:24:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.608+0000 7f4062b9c700 1 -- 172.21.15.60:0/3047627668 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c0c9b40 con 0x7f405c070af0 2023-07-13T22:24:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.609+0000 7f4062b9c700 1 --2- 172.21.15.60:0/3047627668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c070af0 0x7f405c06c440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4054009d90 tx=0x7f405400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b13b8db6b59b6c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.609+0000 7f4061399700 1 -- 172.21.15.60:0/3047627668 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400e070 con 0x7f405c070af0 2023-07-13T22:24:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.609+0000 7f4061399700 1 -- 172.21.15.60:0/3047627668 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054003a20 con 0x7f405c070af0 2023-07-13T22:24:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.609+0000 7f4061399700 1 -- 172.21.15.60:0/3047627668 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c001670 con 0x7f405c070af0 2023-07-13T22:24:39.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.610+0000 7f40645ff700 1 -- 172.21.15.60:0/3047627668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c070af0 msgr2=0x7f405c06c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.610+0000 7f40645ff700 1 --2- 172.21.15.60:0/3047627668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c070af0 0x7f405c06c440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4054009d90 tx=0x7f405400b330 comp rx=0 tx=0).stop 2023-07-13T22:24:39.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.610+0000 7f40645ff700 1 -- 172.21.15.60:0/3047627668 shutdown_connections 2023-07-13T22:24:39.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.610+0000 7f40645ff700 1 --2- 172.21.15.60:0/3047627668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c070af0 0x7f405c06c440 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.610+0000 7f40645ff700 1 -- 172.21.15.60:0/3047627668 >> 172.21.15.60:0/3047627668 conn(0x7f405c0cbf50 msgr2=0x7f405c0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.610+0000 7f40645ff700 1 -- 172.21.15.60:0/3047627668 shutdown_connections 2023-07-13T22:24:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.610+0000 7f40645ff700 1 -- 172.21.15.60:0/3047627668 wait complete. 2023-07-13T22:24:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.611+0000 7f40645ff700 1 Processor -- start 2023-07-13T22:24:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.612+0000 7f40645ff700 1 -- start start 2023-07-13T22:24:39.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f40645ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c070af0 0x7f405c0c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f40645ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f405c0ca900 0x7f405c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f40645ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c159260 0x7f405c0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f40645ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c15b4d0 con 0x7f405c159260 2023-07-13T22:24:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f40645ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c0c5e50 con 0x7f405c070af0 2023-07-13T22:24:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f40645ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405c154590 con 0x7f405c0ca900 2023-07-13T22:24:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f4062b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c159260 0x7f405c0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f4062b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c159260 0x7f405c0b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52652/0 (socket says 172.21.15.60:52652) 2023-07-13T22:24:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f4062b9c700 1 -- 172.21.15.60:0/1022214725 learned_addr learned my addr 172.21.15.60:0/1022214725 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f406239b700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c070af0 0x7f405c0c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f4061b9a700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f405c0ca900 0x7f405c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f4062b9c700 1 -- 172.21.15.60:0/1022214725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f405c0ca900 msgr2=0x7f405c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f4062b9c700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f405c0ca900 0x7f405c154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f4062b9c700 1 -- 172.21.15.60:0/1022214725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c070af0 msgr2=0x7f405c0c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f4062b9c700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c070af0 0x7f405c0c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.613+0000 7f4062b9c700 1 -- 172.21.15.60:0/1022214725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4054009a70 con 0x7f405c159260 2023-07-13T22:24:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.614+0000 7f4062b9c700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c159260 0x7f405c0b32d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f405c06c200 tx=0x7f4054004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:39.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.614+0000 7f4052ffd700 1 -- 172.21.15.60:0/1022214725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40540046b0 con 0x7f405c159260 2023-07-13T22:24:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.614+0000 7f4052ffd700 1 -- 172.21.15.60:0/1022214725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054004810 con 0x7f405c159260 2023-07-13T22:24:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.614+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c15a850 con 0x7f405c159260 2023-07-13T22:24:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.614+0000 7f4052ffd700 1 -- 172.21.15.60:0/1022214725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054012670 con 0x7f405c159260 2023-07-13T22:24:39.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.614+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405c15ae10 con 0x7f405c159260 2023-07-13T22:24:39.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.615+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405c15b610 con 0x7f405c159260 2023-07-13T22:24:39.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.616+0000 7f4052ffd700 1 -- 172.21.15.60:0/1022214725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f40540127d0 con 0x7f405c159260 2023-07-13T22:24:39.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.617+0000 7f4052ffd700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f404006ace0 0x7f404006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:39.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.617+0000 7f4052ffd700 1 -- 172.21.15.60:0/1022214725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f405408abb0 con 0x7f405c159260 2023-07-13T22:24:39.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.617+0000 7f406239b700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f404006ace0 0x7f404006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.618+0000 7f406239b700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f404006ace0 0x7f404006d190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f40580016c0 tx=0x7f4058009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:39.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.621+0000 7f4052ffd700 1 -- 172.21.15.60:0/1022214725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4054055bd0 con 0x7f405c159260 2023-07-13T22:24:39.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.826+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f405c04e210 con 0x7f405c159260 2023-07-13T22:24:39.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.827+0000 7f4052ffd700 1 -- 172.21.15.60:0/1022214725 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f40540591f0 con 0x7f405c159260 2023-07-13T22:24:39.810 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:24:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f404006ace0 msgr2=0x7f404006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f404006ace0 0x7f404006d190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f40580016c0 tx=0x7f4058009040 comp rx=0 tx=0).stop 2023-07-13T22:24:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c159260 msgr2=0x7f405c0b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c159260 0x7f405c0b32d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f405c06c200 tx=0x7f4054004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 shutdown_connections 2023-07-13T22:24:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f404006ace0 0x7f404006d190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c159260 0x7f405c0b32d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c070af0 0x7f405c0c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 --2- 172.21.15.60:0/1022214725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f405c0ca900 0x7f405c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 >> 172.21.15.60:0/1022214725 conn(0x7f405c0cbf50 msgr2=0x7f405c0b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 shutdown_connections 2023-07-13T22:24:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:39.829+0000 7f40645ff700 1 -- 172.21.15.60:0/1022214725 wait complete. 2023-07-13T22:24:39.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:24:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.053+0000 7fb586fff700 1 Processor -- start 2023-07-13T22:24:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.053+0000 7fb586fff700 1 -- start start 2023-07-13T22:24:40.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.054+0000 7fb586fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800b8200 0x7fb5800cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.054+0000 7fb586fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5800cede0 0x7fb5800cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.054+0000 7fb586fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5800d1870 con 0x7fb5800b1ac0 2023-07-13T22:24:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.054+0000 7fb586fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5800cf370 con 0x7fb5800cede0 2023-07-13T22:24:40.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.054+0000 7fb586fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5800d12b0 con 0x7fb5800b8200 2023-07-13T22:24:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb584d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800b8200 0x7fb5800cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb58559c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5800cede0 0x7fb5800cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb58559c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5800cede0 0x7fb5800cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:41380/0 (socket says 172.21.15.60:41380) 2023-07-13T22:24:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb58559c700 1 -- 172.21.15.60:0/2047090280 learned_addr learned my addr 172.21.15.60:0/2047090280 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb584d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800b8200 0x7fb5800cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:34510/0 (socket says 172.21.15.60:34510) 2023-07-13T22:24:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb584d9b700 1 -- 172.21.15.60:0/2047090280 >> v2:172.21.15.105:3300/0 conn(0x7fb5800cede0 msgr2=0x7fb5800cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb584d9b700 1 --2- 172.21.15.60:0/2047090280 >> v2:172.21.15.105:3300/0 conn(0x7fb5800cede0 0x7fb5800cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb584d9b700 1 -- 172.21.15.60:0/2047090280 >> v1:172.21.15.105:6789/0 conn(0x7fb5800b1ac0 legacy=0x7fb58015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb584d9b700 1 -- 172.21.15.60:0/2047090280 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5800d1590 con 0x7fb5800b8200 2023-07-13T22:24:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb58559c700 1 --2- 172.21.15.60:0/2047090280 >> v2:172.21.15.105:3300/0 conn(0x7fb5800cede0 0x7fb5800cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:40.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.055+0000 7fb584d9b700 1 --2- 172.21.15.60:0/2047090280 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800b8200 0x7fb5800cf640 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb574009840 tx=0x7fb574009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1864bce899f587a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:40.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb57f7fe700 1 -- 172.21.15.60:0/2047090280 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574011830 con 0x7fb5800b8200 2023-07-13T22:24:40.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb57f7fe700 1 -- 172.21.15.60:0/2047090280 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb574011990 con 0x7fb5800b8200 2023-07-13T22:24:40.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb57f7fe700 1 -- 172.21.15.60:0/2047090280 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574011d20 con 0x7fb5800b8200 2023-07-13T22:24:40.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb586fff700 1 -- 172.21.15.60:0/2047090280 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800b8200 msgr2=0x7fb5800cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb586fff700 1 --2- 172.21.15.60:0/2047090280 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800b8200 0x7fb5800cf640 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb574009840 tx=0x7fb574009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb586fff700 1 -- 172.21.15.60:0/2047090280 shutdown_connections 2023-07-13T22:24:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb586fff700 1 --2- 172.21.15.60:0/2047090280 >> v2:172.21.15.105:3300/0 conn(0x7fb5800cede0 0x7fb5800cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb586fff700 1 --2- 172.21.15.60:0/2047090280 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800b8200 0x7fb5800cf640 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb586fff700 1 -- 172.21.15.60:0/2047090280 >> 172.21.15.60:0/2047090280 conn(0x7fb58006b2f0 msgr2=0x7fb58006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb586fff700 1 -- 172.21.15.60:0/2047090280 shutdown_connections 2023-07-13T22:24:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.056+0000 7fb586fff700 1 -- 172.21.15.60:0/2047090280 wait complete. 2023-07-13T22:24:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.058+0000 7fb586fff700 1 Processor -- start 2023-07-13T22:24:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.058+0000 7fb586fff700 1 -- start start 2023-07-13T22:24:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.059+0000 7fb586fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5800b1ac0 0x7fb5800c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.059+0000 7fb586fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5800b8200 0x7fb5801e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.059+0000 7fb586fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800cede0 0x7fb5801e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.059+0000 7fb586fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5801e6eb0 con 0x7fb5800b8200 2023-07-13T22:24:40.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.059+0000 7fb586fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5801e7cf0 con 0x7fb5800b1ac0 2023-07-13T22:24:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.059+0000 7fb586fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5801e70e0 con 0x7fb5800cede0 2023-07-13T22:24:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.059+0000 7fb57ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5800b8200 0x7fb5801e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb584d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5800b1ac0 0x7fb5800c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb58559c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800cede0 0x7fb5801e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb584d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5800b1ac0 0x7fb5800c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:41392/0 (socket says 172.21.15.60:41392) 2023-07-13T22:24:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb584d9b700 1 -- 172.21.15.60:0/2298903025 learned_addr learned my addr 172.21.15.60:0/2298903025 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:40.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb57ffff700 1 -- 172.21.15.60:0/2298903025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800cede0 msgr2=0x7fb5801e4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb57ffff700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800cede0 0x7fb5801e4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb57ffff700 1 -- 172.21.15.60:0/2298903025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5800b1ac0 msgr2=0x7fb5800c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb57ffff700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5800b1ac0 0x7fb5800c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb57ffff700 1 -- 172.21.15.60:0/2298903025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578009b30 con 0x7fb5800b8200 2023-07-13T22:24:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb57ffff700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5800b8200 0x7fb5801e6970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb56c000f60 tx=0x7fb56c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.060+0000 7fb57d7fa700 1 -- 172.21.15.60:0/2298903025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0052d0 con 0x7fb5800b8200 2023-07-13T22:24:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.061+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5740094a0 con 0x7fb5800b8200 2023-07-13T22:24:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.061+0000 7fb57d7fa700 1 -- 172.21.15.60:0/2298903025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c005430 con 0x7fb5800b8200 2023-07-13T22:24:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.061+0000 7fb57d7fa700 1 -- 172.21.15.60:0/2298903025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0038c0 con 0x7fb5800b8200 2023-07-13T22:24:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.061+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5801e76a0 con 0x7fb5800b8200 2023-07-13T22:24:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.062+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5801e7e80 con 0x7fb5800b8200 2023-07-13T22:24:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.063+0000 7fb57d7fa700 1 -- 172.21.15.60:0/2298903025 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb56c003a20 con 0x7fb5800b8200 2023-07-13T22:24:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.065+0000 7fb57d7fa700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb56406ace0 0x7fb56406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.065+0000 7fb57d7fa700 1 -- 172.21.15.60:0/2298903025 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb56c088800 con 0x7fb5800b8200 2023-07-13T22:24:40.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.065+0000 7fb584d9b700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb56406ace0 0x7fb56406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.066+0000 7fb584d9b700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb56406ace0 0x7fb56406d190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb574009b90 tx=0x7fb574008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:40.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.069+0000 7fb57d7fa700 1 -- 172.21.15.60:0/2298903025 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb56c053830 con 0x7fb5800b8200 2023-07-13T22:24:40.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.279+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb58004e210 con 0x7fb5800b8200 2023-07-13T22:24:40.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.280+0000 7fb57d7fa700 1 -- 172.21.15.60:0/2298903025 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb56c056e50 con 0x7fb5800b8200 2023-07-13T22:24:40.263 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:24:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb56406ace0 msgr2=0x7fb56406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb56406ace0 0x7fb56406d190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb574009b90 tx=0x7fb574008030 comp rx=0 tx=0).stop 2023-07-13T22:24:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5800b8200 msgr2=0x7fb5801e6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5800b8200 0x7fb5801e6970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb56c000f60 tx=0x7fb56c00c730 comp rx=0 tx=0).stop 2023-07-13T22:24:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 shutdown_connections 2023-07-13T22:24:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb56406ace0 0x7fb56406d190 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5800cede0 0x7fb5801e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5800b1ac0 0x7fb5800c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 --2- 172.21.15.60:0/2298903025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5800b8200 0x7fb5801e6970 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.282+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 >> 172.21.15.60:0/2298903025 conn(0x7fb58006b2f0 msgr2=0x7fb5800b67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.283+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 shutdown_connections 2023-07-13T22:24:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.283+0000 7fb586fff700 1 -- 172.21.15.60:0/2298903025 wait complete. 2023-07-13T22:24:40.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:24:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.505+0000 7f26dd5ff700 1 Processor -- start 2023-07-13T22:24:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.505+0000 7f26dd5ff700 1 -- start start 2023-07-13T22:24:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.506+0000 7f26dd5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26d815a820 0x7f26d815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26dd5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d80c8d40 0x7f26d80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26dd5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d8070270 con 0x7f26d8154950 2023-07-13T22:24:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26dd5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d80713d0 con 0x7f26d815a820 2023-07-13T22:24:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26dd5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80c96a0 con 0x7f26d80c8d40 2023-07-13T22:24:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26d67fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26d815a820 0x7f26d815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26d67fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26d815a820 0x7f26d815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52674/0 (socket says 172.21.15.60:52674) 2023-07-13T22:24:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26d67fc700 1 -- 172.21.15.60:0/4229818321 learned_addr learned my addr 172.21.15.60:0/4229818321 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26d77fe700 1 --2- 172.21.15.60:0/4229818321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d80c8d40 0x7f26d80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26d67fc700 1 -- 172.21.15.60:0/4229818321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d80c8d40 msgr2=0x7f26d80c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26d67fc700 1 --2- 172.21.15.60:0/4229818321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d80c8d40 0x7f26d80c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26d67fc700 1 -- 172.21.15.60:0/4229818321 >> v1:172.21.15.60:6789/0 conn(0x7f26d8154950 legacy=0x7f26d8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.507+0000 7f26d67fc700 1 -- 172.21.15.60:0/4229818321 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d815b2e0 con 0x7f26d815a820 2023-07-13T22:24:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26d67fc700 1 --2- 172.21.15.60:0/4229818321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d815a820 0x7f26d815ac70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0098f0 tx=0x7f26cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec700830ac96f42e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26d5ffb700 1 -- 172.21.15.60:0/4229818321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00b8a0 con 0x7f26d815a820 2023-07-13T22:24:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26d5ffb700 1 -- 172.21.15.60:0/4229818321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc00ba00 con 0x7f26d815a820 2023-07-13T22:24:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26d5ffb700 1 -- 172.21.15.60:0/4229818321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00bd90 con 0x7f26d815a820 2023-07-13T22:24:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26dd5ff700 1 -- 172.21.15.60:0/4229818321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d815a820 msgr2=0x7f26d815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26dd5ff700 1 --2- 172.21.15.60:0/4229818321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d815a820 0x7f26d815ac70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0098f0 tx=0x7f26cc009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26dd5ff700 1 -- 172.21.15.60:0/4229818321 shutdown_connections 2023-07-13T22:24:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26dd5ff700 1 --2- 172.21.15.60:0/4229818321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d80c8d40 0x7f26d80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26dd5ff700 1 --2- 172.21.15.60:0/4229818321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d815a820 0x7f26d815ac70 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26dd5ff700 1 -- 172.21.15.60:0/4229818321 >> 172.21.15.60:0/4229818321 conn(0x7f26d80cf410 msgr2=0x7f26d80cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.508+0000 7f26dd5ff700 1 -- 172.21.15.60:0/4229818321 shutdown_connections 2023-07-13T22:24:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.509+0000 7f26dd5ff700 1 -- 172.21.15.60:0/4229818321 wait complete. 2023-07-13T22:24:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.510+0000 7f26dd5ff700 1 Processor -- start 2023-07-13T22:24:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.510+0000 7f26dd5ff700 1 -- start start 2023-07-13T22:24:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.511+0000 7f26dd5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d80c8d40 0x7f26d80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.511+0000 7f26dd5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d8154950 0x7f26d8141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.511+0000 7f26dd5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26d815a820 0x7f26d81593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26dd5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80b0010 con 0x7f26d80c8d40 2023-07-13T22:24:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26dd5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81411b0 con 0x7f26d815a820 2023-07-13T22:24:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26dd5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d80b0c10 con 0x7f26d8154950 2023-07-13T22:24:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d80c8d40 0x7f26d80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d80c8d40 0x7f26d80b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52684/0 (socket says 172.21.15.60:52684) 2023-07-13T22:24:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d6ffd700 1 -- 172.21.15.60:0/2048701422 learned_addr learned my addr 172.21.15.60:0/2048701422 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d67fc700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d8154950 0x7f26d8141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d77fe700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26d815a820 0x7f26d81593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d6ffd700 1 -- 172.21.15.60:0/2048701422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d8154950 msgr2=0x7f26d8141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d6ffd700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d8154950 0x7f26d8141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d6ffd700 1 -- 172.21.15.60:0/2048701422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26d815a820 msgr2=0x7f26d81593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d6ffd700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26d815a820 0x7f26d81593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d6ffd700 1 -- 172.21.15.60:0/2048701422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc009550 con 0x7f26d80c8d40 2023-07-13T22:24:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d77fe700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26d815a820 0x7f26d81593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.512+0000 7f26d6ffd700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d80c8d40 0x7f26d80b0530 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c4e0 tx=0x7f26c800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.513+0000 7f26bf7fe700 1 -- 172.21.15.60:0/2048701422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80052d0 con 0x7f26d80c8d40 2023-07-13T22:24:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.513+0000 7f26bf7fe700 1 -- 172.21.15.60:0/2048701422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c8005430 con 0x7f26d80c8d40 2023-07-13T22:24:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.513+0000 7f26bf7fe700 1 -- 172.21.15.60:0/2048701422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80038c0 con 0x7f26d80c8d40 2023-07-13T22:24:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.513+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d80b0ef0 con 0x7f26d80c8d40 2023-07-13T22:24:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.513+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d80b11d0 con 0x7f26d80c8d40 2023-07-13T22:24:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.514+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d804e280 con 0x7f26d80c8d40 2023-07-13T22:24:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.515+0000 7f26bf7fe700 1 -- 172.21.15.60:0/2048701422 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26c8003a20 con 0x7f26d80c8d40 2023-07-13T22:24:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.516+0000 7f26bf7fe700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26c406ace0 0x7f26c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.516+0000 7f26bf7fe700 1 -- 172.21.15.60:0/2048701422 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f26c8088a40 con 0x7f26d80c8d40 2023-07-13T22:24:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.516+0000 7f26d67fc700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26c406ace0 0x7f26c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.517+0000 7f26d67fc700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26c406ace0 0x7f26c406d190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26cc014040 tx=0x7f26cc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.520+0000 7f26bf7fe700 1 -- 172.21.15.60:0/2048701422 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f26c8053a50 con 0x7f26d80c8d40 2023-07-13T22:24:40.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.743+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26d80cbc20 con 0x7f26d80c8d40 2023-07-13T22:24:40.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.744+0000 7f26bf7fe700 1 -- 172.21.15.60:0/2048701422 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f26c8057070 con 0x7f26d80c8d40 2023-07-13T22:24:40.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26c406ace0 msgr2=0x7f26c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26c406ace0 0x7f26c406d190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26cc014040 tx=0x7f26cc008030 comp rx=0 tx=0).stop 2023-07-13T22:24:40.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d80c8d40 msgr2=0x7f26d80b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:40.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d80c8d40 0x7f26d80b0530 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c4e0 tx=0x7f26c800c8a0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 shutdown_connections 2023-07-13T22:24:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26c406ace0 0x7f26c406d190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d80c8d40 0x7f26d80b0530 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26d815a820 0x7f26d81593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 --2- 172.21.15.60:0/2048701422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26d8154950 0x7f26d8141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.746+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 >> 172.21.15.60:0/2048701422 conn(0x7f26d80cf410 msgr2=0x7f26d81527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.747+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 shutdown_connections 2023-07-13T22:24:40.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:40.747+0000 7f26dd5ff700 1 -- 172.21.15.60:0/2048701422 wait complete. 2023-07-13T22:24:40.731 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:24:40.747 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:24:40.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:40.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:24:40.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[137864]: fuse finished with error 0 and tester_r 0 2023-07-13T22:24:40.819 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:24:40.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:40.820 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:24:41.011 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:24:41.012 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:41.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:41.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.283+0000 7fceb4bff700 1 Processor -- start 2023-07-13T22:24:41.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.284+0000 7fceb4bff700 1 -- start start 2023-07-13T22:24:41.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.285+0000 7fceb4bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fceb00b8570 0x7fceb00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.285+0000 7fceb4bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fceb00d15c0 0x7fceb015c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.285+0000 7fceb4bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb00d09d0 con 0x7fceb00d35c0 2023-07-13T22:24:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.285+0000 7fceb4bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb00d3b30 con 0x7fceb00b8570 2023-07-13T22:24:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.285+0000 7fceb4bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb00cda10 con 0x7fceb00d15c0 2023-07-13T22:24:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.285+0000 7fceae59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fceb00b8570 0x7fceb00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.285+0000 7fcea7fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fceb00d35c0 0x7fceb00d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44062/0 (socket says 172.21.15.60:44062) 2023-07-13T22:24:41.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.285+0000 7fcea7fff700 1 -- 172.21.15.60:0/688208471 learned_addr learned my addr 172.21.15.60:0/688208471 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.285+0000 7fceaed9d700 1 --2- 172.21.15.60:0/688208471 >> v2:172.21.15.105:3300/0 conn(0x7fceb00d15c0 0x7fceb015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.286+0000 7fceae59c700 1 -- 172.21.15.60:0/688208471 >> v2:172.21.15.105:3300/0 conn(0x7fceb00d15c0 msgr2=0x7fceb015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.286+0000 7fceae59c700 1 --2- 172.21.15.60:0/688208471 >> v2:172.21.15.105:3300/0 conn(0x7fceb00d15c0 0x7fceb015c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.286+0000 7fceae59c700 1 -- 172.21.15.60:0/688208471 >> v1:172.21.15.60:6789/0 conn(0x7fceb00d35c0 legacy=0x7fceb00d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.286+0000 7fceae59c700 1 -- 172.21.15.60:0/688208471 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00cd450 con 0x7fceb00b8570 2023-07-13T22:24:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.286+0000 7fceadd9b700 1 -- 172.21.15.60:0/688208471 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614789283 0 0) 0x7fceb00d09d0 con 0x7fceb00d35c0 2023-07-13T22:24:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.286+0000 7fceae59c700 1 --2- 172.21.15.60:0/688208471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00b8570 0x7fceb00ceeb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b2670 tx=0x7fcea0009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce99eb067be9d04 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.286+0000 7fceadd9b700 1 -- 172.21.15.60:0/688208471 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0005e30 con 0x7fceb00b8570 2023-07-13T22:24:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.286+0000 7fceadd9b700 1 -- 172.21.15.60:0/688208471 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea0006030 con 0x7fceb00b8570 2023-07-13T22:24:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.286+0000 7fceadd9b700 1 -- 172.21.15.60:0/688208471 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea00063c0 con 0x7fceb00b8570 2023-07-13T22:24:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.287+0000 7fceb4bff700 1 -- 172.21.15.60:0/688208471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00b8570 msgr2=0x7fceb00ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.287+0000 7fceb4bff700 1 --2- 172.21.15.60:0/688208471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00b8570 0x7fceb00ceeb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b2670 tx=0x7fcea0009b00 comp rx=0 tx=0).stop 2023-07-13T22:24:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.287+0000 7fceb4bff700 1 -- 172.21.15.60:0/688208471 shutdown_connections 2023-07-13T22:24:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.287+0000 7fceb4bff700 1 --2- 172.21.15.60:0/688208471 >> v2:172.21.15.105:3300/0 conn(0x7fceb00d15c0 0x7fceb015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.287+0000 7fceb4bff700 1 --2- 172.21.15.60:0/688208471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00b8570 0x7fceb00ceeb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.287+0000 7fceb4bff700 1 -- 172.21.15.60:0/688208471 >> 172.21.15.60:0/688208471 conn(0x7fceb0071600 msgr2=0x7fceb00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.287+0000 7fceb4bff700 1 -- 172.21.15.60:0/688208471 shutdown_connections 2023-07-13T22:24:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.287+0000 7fceb4bff700 1 -- 172.21.15.60:0/688208471 wait complete. 2023-07-13T22:24:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.289+0000 7fceb4bff700 1 Processor -- start 2023-07-13T22:24:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.289+0000 7fceb4bff700 1 -- start start 2023-07-13T22:24:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceb4bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fceb00b8570 0x7fceb01f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceb4bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb00d15c0 0x7fceb00caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceb4bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00d35c0 0x7fceb00cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceb4bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb00cc6b0 con 0x7fceb00d35c0 2023-07-13T22:24:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceb4bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb01f4f20 con 0x7fceb00d15c0 2023-07-13T22:24:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceb4bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb00ccad0 con 0x7fceb00b8570 2023-07-13T22:24:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceaed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00d35c0 0x7fceb00cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceaed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00d35c0 0x7fceb00cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52692/0 (socket says 172.21.15.60:52692) 2023-07-13T22:24:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceaed9d700 1 -- 172.21.15.60:0/1480216612 learned_addr learned my addr 172.21.15.60:0/1480216612 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceae59c700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fceb00b8570 0x7fceb01f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fcea7fff700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb00d15c0 0x7fceb00caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceaed9d700 1 -- 172.21.15.60:0/1480216612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fceb00b8570 msgr2=0x7fceb01f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceaed9d700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fceb00b8570 0x7fceb01f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceaed9d700 1 -- 172.21.15.60:0/1480216612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb00d15c0 msgr2=0x7fceb00caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceaed9d700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb00d15c0 0x7fceb00caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.290+0000 7fceaed9d700 1 -- 172.21.15.60:0/1480216612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0009430 con 0x7fceb00d35c0 2023-07-13T22:24:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.291+0000 7fceaed9d700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00d35c0 0x7fceb00cb420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcea8007e40 tx=0x7fcea800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.291+0000 7fcea6ffd700 1 -- 172.21.15.60:0/1480216612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea800ed30 con 0x7fceb00d35c0 2023-07-13T22:24:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.291+0000 7fcea6ffd700 1 -- 172.21.15.60:0/1480216612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea8014420 con 0x7fceb00d35c0 2023-07-13T22:24:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.291+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb00cba30 con 0x7fceb00d35c0 2023-07-13T22:24:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.291+0000 7fcea6ffd700 1 -- 172.21.15.60:0/1480216612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea80147d0 con 0x7fceb00d35c0 2023-07-13T22:24:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.291+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb00cbff0 con 0x7fceb00d35c0 2023-07-13T22:24:41.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.292+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb004e280 con 0x7fceb00d35c0 2023-07-13T22:24:41.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.293+0000 7fcea6ffd700 1 -- 172.21.15.60:0/1480216612 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcea8014930 con 0x7fceb00d35c0 2023-07-13T22:24:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.294+0000 7fcea6ffd700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce9006ace0 0x7fce9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.294+0000 7fcea6ffd700 1 -- 172.21.15.60:0/1480216612 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcea8089b30 con 0x7fceb00d35c0 2023-07-13T22:24:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.294+0000 7fceae59c700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce9006ace0 0x7fce9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.295+0000 7fceae59c700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce9006ace0 0x7fce9006d190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcea0000c00 tx=0x7fcea0008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:41.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.298+0000 7fcea6ffd700 1 -- 172.21.15.60:0/1480216612 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcea8054bd0 con 0x7fceb00d35c0 2023-07-13T22:24:41.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.507+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fceb0065ee0 con 0x7fceb00d35c0 2023-07-13T22:24:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.508+0000 7fcea6ffd700 1 -- 172.21.15.60:0/1480216612 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18199 (secure 0 0 0) 0x7fcea80581f0 con 0x7fceb00d35c0 2023-07-13T22:24:41.491 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:41.492 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:21.653794+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:24:18.441889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-13T22:24:19.114200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce9006ace0 msgr2=0x7fce9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce9006ace0 0x7fce9006d190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcea0000c00 tx=0x7fcea0008070 comp rx=0 tx=0).stop 2023-07-13T22:24:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00d35c0 msgr2=0x7fceb00cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00d35c0 0x7fceb00cb420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcea8007e40 tx=0x7fcea800e3c0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 shutdown_connections 2023-07-13T22:24:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce9006ace0 0x7fce9006d190 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fceb00d35c0 0x7fceb00cb420 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fceb00b8570 0x7fceb01f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 --2- 172.21.15.60:0/1480216612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb00d15c0 0x7fceb00caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 >> 172.21.15.60:0/1480216612 conn(0x7fceb0071600 msgr2=0x7fceb00d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 shutdown_connections 2023-07-13T22:24:41.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.511+0000 7fceb4bff700 1 -- 172.21.15.60:0/1480216612 wait complete. 2023-07-13T22:24:41.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:24:18.441889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:24:19.114200+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:24:41.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:24:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.737+0000 7f21451ff700 1 Processor -- start 2023-07-13T22:24:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.737+0000 7f21451ff700 1 -- start start 2023-07-13T22:24:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.739+0000 7f21451ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2140070af0 0x7f214006c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.739+0000 7f21451ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21400cf640 con 0x7f21400ca900 2023-07-13T22:24:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.739+0000 7f21451ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21400ccf80 con 0x7f2140159260 2023-07-13T22:24:41.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.739+0000 7f21451ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21400cada0 con 0x7f2140070af0 2023-07-13T22:24:41.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.739+0000 7f213e59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f21400ca900 0x7f21400703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44064/0 (socket says 172.21.15.60:44064) 2023-07-13T22:24:41.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.739+0000 7f213e59c700 1 -- 172.21.15.60:0/450255176 learned_addr learned my addr 172.21.15.60:0/450255176 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:41.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.739+0000 7f213f59e700 1 --2- 172.21.15.60:0/450255176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2140070af0 0x7f214006c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.739+0000 7f213dd9b700 1 -- 172.21.15.60:0/450255176 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681699560 0 0) 0x7f21400cf640 con 0x7f21400ca900 2023-07-13T22:24:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.739+0000 7f213dd9b700 1 -- 172.21.15.60:0/450255176 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21280035b0 con 0x7f21400ca900 2023-07-13T22:24:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.740+0000 7f213dd9b700 1 -- 172.21.15.60:0/450255176 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446731486 0 0) 0x7f21400ccf80 con 0x7f2140159260 2023-07-13T22:24:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.740+0000 7f213dd9b700 1 -- 172.21.15.60:0/450255176 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21400cf640 con 0x7f2140159260 2023-07-13T22:24:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.740+0000 7f213f59e700 1 -- 172.21.15.60:0/450255176 >> v1:172.21.15.105:6789/0 conn(0x7f2140159260 legacy=0x7f21400cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.740+0000 7f213f59e700 1 -- 172.21.15.60:0/450255176 >> v1:172.21.15.60:6789/0 conn(0x7f21400ca900 legacy=0x7f21400703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.740+0000 7f213f59e700 1 -- 172.21.15.60:0/450255176 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400c9b40 con 0x7f2140070af0 2023-07-13T22:24:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.740+0000 7f213f59e700 1 --2- 172.21.15.60:0/450255176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2140070af0 0x7f214006c440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2138009dc0 tx=0x7f213800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8388e8ea94c5a5c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.740+0000 7f213dd9b700 1 -- 172.21.15.60:0/450255176 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213800e070 con 0x7f2140070af0 2023-07-13T22:24:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.740+0000 7f213dd9b700 1 -- 172.21.15.60:0/450255176 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213800bb20 con 0x7f2140070af0 2023-07-13T22:24:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.740+0000 7f213dd9b700 1 -- 172.21.15.60:0/450255176 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2138012430 con 0x7f2140070af0 2023-07-13T22:24:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.741+0000 7f21451ff700 1 -- 172.21.15.60:0/450255176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2140070af0 msgr2=0x7f214006c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.741+0000 7f21451ff700 1 --2- 172.21.15.60:0/450255176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2140070af0 0x7f214006c440 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2138009dc0 tx=0x7f213800b330 comp rx=0 tx=0).stop 2023-07-13T22:24:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.741+0000 7f21451ff700 1 -- 172.21.15.60:0/450255176 shutdown_connections 2023-07-13T22:24:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.741+0000 7f21451ff700 1 --2- 172.21.15.60:0/450255176 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2140070af0 0x7f214006c440 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.741+0000 7f21451ff700 1 -- 172.21.15.60:0/450255176 >> 172.21.15.60:0/450255176 conn(0x7f21400cbf50 msgr2=0x7f21400ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.741+0000 7f21451ff700 1 -- 172.21.15.60:0/450255176 shutdown_connections 2023-07-13T22:24:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.741+0000 7f21451ff700 1 -- 172.21.15.60:0/450255176 wait complete. 2023-07-13T22:24:41.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.743+0000 7f21451ff700 1 Processor -- start 2023-07-13T22:24:41.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.743+0000 7f21451ff700 1 -- start start 2023-07-13T22:24:41.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f21451ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2140070af0 0x7f2140153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f21451ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21400ca900 0x7f21400af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f21451ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2140159260 0x7f21400b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f21451ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21400afe80 con 0x7f2140159260 2023-07-13T22:24:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f21451ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21400b03d0 con 0x7f2140070af0 2023-07-13T22:24:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f21451ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2140154400 con 0x7f21400ca900 2023-07-13T22:24:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f213f59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2140159260 0x7f21400b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f213f59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2140159260 0x7f21400b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52698/0 (socket says 172.21.15.60:52698) 2023-07-13T22:24:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f213f59e700 1 -- 172.21.15.60:0/1188492144 learned_addr learned my addr 172.21.15.60:0/1188492144 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f213e59c700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21400ca900 0x7f21400af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.744+0000 7f213ed9d700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2140070af0 0x7f2140153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f213f59e700 1 -- 172.21.15.60:0/1188492144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21400ca900 msgr2=0x7f21400af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f213f59e700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21400ca900 0x7f21400af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f213f59e700 1 -- 172.21.15.60:0/1188492144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2140070af0 msgr2=0x7f2140153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f213f59e700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2140070af0 0x7f2140153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f213f59e700 1 -- 172.21.15.60:0/1188492144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2138009a70 con 0x7f2140159260 2023-07-13T22:24:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f213f59e700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2140159260 0x7f21400b0860 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f213800b370 tx=0x7f2138004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f21277fe700 1 -- 172.21.15.60:0/1188492144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213801b810 con 0x7f2140159260 2023-07-13T22:24:41.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f21277fe700 1 -- 172.21.15.60:0/1188492144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2138020070 con 0x7f2140159260 2023-07-13T22:24:41.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21400c8800 con 0x7f2140159260 2023-07-13T22:24:41.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f21277fe700 1 -- 172.21.15.60:0/1188492144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213801bbf0 con 0x7f2140159260 2023-07-13T22:24:41.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.745+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21400c51e0 con 0x7f2140159260 2023-07-13T22:24:41.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.746+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21400b0580 con 0x7f2140159260 2023-07-13T22:24:41.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.747+0000 7f21277fe700 1 -- 172.21.15.60:0/1188492144 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f213801bd50 con 0x7f2140159260 2023-07-13T22:24:41.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.748+0000 7f21277fe700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f212c06ace0 0x7f212c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:41.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.748+0000 7f21277fe700 1 -- 172.21.15.60:0/1188492144 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2138022070 con 0x7f2140159260 2023-07-13T22:24:41.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.748+0000 7f213ed9d700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f212c06ace0 0x7f212c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:41.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.749+0000 7f213ed9d700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f212c06ace0 0x7f212c06d190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f21300011e0 tx=0x7f2130009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:41.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.752+0000 7f21277fe700 1 -- 172.21.15.60:0/1188492144 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f21400b0580 con 0x7f2140159260 2023-07-13T22:24:41.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.958+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f21400b0540 con 0x7f2140159260 2023-07-13T22:24:41.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.958+0000 7f21277fe700 1 -- 172.21.15.60:0/1188492144 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2138056ba0 con 0x7f2140159260 2023-07-13T22:24:41.941 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:24:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f212c06ace0 msgr2=0x7f212c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f212c06ace0 0x7f212c06d190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f21300011e0 tx=0x7f2130009040 comp rx=0 tx=0).stop 2023-07-13T22:24:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2140159260 msgr2=0x7f21400b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2140159260 0x7f21400b0860 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f213800b370 tx=0x7f2138004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 shutdown_connections 2023-07-13T22:24:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f212c06ace0 0x7f212c06d190 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2140159260 0x7f21400b0860 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2140070af0 0x7f2140153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 --2- 172.21.15.60:0/1188492144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21400ca900 0x7f21400af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:41.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 >> 172.21.15.60:0/1188492144 conn(0x7f21400cbf50 msgr2=0x7f21400b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:41.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 shutdown_connections 2023-07-13T22:24:41.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:41.961+0000 7f21451ff700 1 -- 172.21.15.60:0/1188492144 wait complete. 2023-07-13T22:24:41.962 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:24:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.185+0000 7f3fb71ff700 1 Processor -- start 2023-07-13T22:24:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.185+0000 7f3fb71ff700 1 -- start start 2023-07-13T22:24:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.186+0000 7f3fb71ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00b8200 0x7f3fb00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.186+0000 7f3fb71ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fb00b1ac0 0x7f3fb00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.186+0000 7f3fb71ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb00d1870 con 0x7f3fb00cede0 2023-07-13T22:24:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3fb71ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb00cf370 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3fb71ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00d12b0 con 0x7f3fb00b8200 2023-07-13T22:24:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3fb579c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3fb00cede0 0x7f3fb015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44076/0 (socket says 172.21.15.60:44076) 2023-07-13T22:24:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3fb579c700 1 -- 172.21.15.60:0/2981918678 learned_addr learned my addr 172.21.15.60:0/2981918678 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:42.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3fb4f9b700 1 --2- 172.21.15.60:0/2981918678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00b8200 0x7f3fb00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3faffff700 1 --2- 172.21.15.60:0/2981918678 >> v2:172.21.15.105:3300/0 conn(0x7f3fb00b1ac0 0x7f3fb00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3faf7fe700 1 -- 172.21.15.60:0/2981918678 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994452292 0 0) 0x7f3fb00d1870 con 0x7f3fb00cede0 2023-07-13T22:24:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3faf7fe700 1 -- 172.21.15.60:0/2981918678 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa00035b0 con 0x7f3fb00cede0 2023-07-13T22:24:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3fb4f9b700 1 -- 172.21.15.60:0/2981918678 >> v2:172.21.15.105:3300/0 conn(0x7f3fb00b1ac0 msgr2=0x7f3fb00cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3fb4f9b700 1 --2- 172.21.15.60:0/2981918678 >> v2:172.21.15.105:3300/0 conn(0x7f3fb00b1ac0 0x7f3fb00cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3fb4f9b700 1 -- 172.21.15.60:0/2981918678 >> v1:172.21.15.60:6789/0 conn(0x7f3fb00cede0 legacy=0x7f3fb015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.187+0000 7f3fb4f9b700 1 -- 172.21.15.60:0/2981918678 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00d1590 con 0x7f3fb00b8200 2023-07-13T22:24:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.188+0000 7f3fb4f9b700 1 --2- 172.21.15.60:0/2981918678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00b8200 0x7f3fb00cf640 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009840 tx=0x7f3fa4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b24f2d298f020e96 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.188+0000 7f3faf7fe700 1 -- 172.21.15.60:0/2981918678 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4011830 con 0x7f3fb00b8200 2023-07-13T22:24:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.188+0000 7f3faf7fe700 1 -- 172.21.15.60:0/2981918678 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa4011990 con 0x7f3fb00b8200 2023-07-13T22:24:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.188+0000 7f3faf7fe700 1 -- 172.21.15.60:0/2981918678 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4011d20 con 0x7f3fb00b8200 2023-07-13T22:24:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.189+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2981918678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00b8200 msgr2=0x7f3fb00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.189+0000 7f3fb71ff700 1 --2- 172.21.15.60:0/2981918678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00b8200 0x7f3fb00cf640 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009840 tx=0x7f3fa4009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.189+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2981918678 shutdown_connections 2023-07-13T22:24:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.189+0000 7f3fb71ff700 1 --2- 172.21.15.60:0/2981918678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00b8200 0x7f3fb00cf640 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.189+0000 7f3fb71ff700 1 --2- 172.21.15.60:0/2981918678 >> v2:172.21.15.105:3300/0 conn(0x7f3fb00b1ac0 0x7f3fb00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.189+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2981918678 >> 172.21.15.60:0/2981918678 conn(0x7f3fb006b2f0 msgr2=0x7f3fb006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.189+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2981918678 shutdown_connections 2023-07-13T22:24:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.189+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2981918678 wait complete. 2023-07-13T22:24:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.190+0000 7f3fb71ff700 1 Processor -- start 2023-07-13T22:24:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.191+0000 7f3fb71ff700 1 -- start start 2023-07-13T22:24:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb71ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fb00b1ac0 0x7f3fb00cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb71ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b8200 0x7f3fb01f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb71ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00cede0 0x7f3fb01f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb71ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00c78a0 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb71ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00c7b20 con 0x7f3fb00b8200 2023-07-13T22:24:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb71ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00cc1b0 con 0x7f3fb00cede0 2023-07-13T22:24:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb4f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fb00b1ac0 0x7f3fb00cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb4f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fb00b1ac0 0x7f3fb00cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52706/0 (socket says 172.21.15.60:52706) 2023-07-13T22:24:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb4f9b700 1 -- 172.21.15.60:0/2395513097 learned_addr learned my addr 172.21.15.60:0/2395513097 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3faffff700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b8200 0x7f3fb01f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb579c700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00cede0 0x7f3fb01f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb4f9b700 1 -- 172.21.15.60:0/2395513097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00cede0 msgr2=0x7f3fb01f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb4f9b700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00cede0 0x7f3fb01f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb4f9b700 1 -- 172.21.15.60:0/2395513097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b8200 msgr2=0x7f3fb01f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb4f9b700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b8200 0x7f3fb01f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb4f9b700 1 -- 172.21.15.60:0/2395513097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa40094a0 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.192+0000 7f3fb4f9b700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fb00b1ac0 0x7f3fb00cb760 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40090b0 tx=0x7f3fa4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.193+0000 7f3fad7fa700 1 -- 172.21.15.60:0/2395513097 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40036a0 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.193+0000 7f3fad7fa700 1 -- 172.21.15.60:0/2395513097 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa401f070 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.193+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb00cc490 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.193+0000 7f3fad7fa700 1 -- 172.21.15.60:0/2395513097 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4003a40 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.193+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb014b710 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.194+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb004e210 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.195+0000 7f3fad7fa700 1 -- 172.21.15.60:0/2395513097 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3fa4003ba0 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.197+0000 7f3fad7fa700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f9406ace0 0x7f3f9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.197+0000 7f3fad7fa700 1 -- 172.21.15.60:0/2395513097 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3fa4021070 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.197+0000 7f3faffff700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f9406ace0 0x7f3f9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.198+0000 7f3faffff700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f9406ace0 0x7f3f9406d190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009b70 tx=0x7f3f9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.201+0000 7f3fad7fa700 1 -- 172.21.15.60:0/2395513097 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3fa4055dc0 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.405+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fb00cc680 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.406+0000 7f3fad7fa700 1 -- 172.21.15.60:0/2395513097 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fa40150a0 con 0x7f3fb00b1ac0 2023-07-13T22:24:42.388 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:24:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f9406ace0 msgr2=0x7f3f9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f9406ace0 0x7f3f9406d190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009b70 tx=0x7f3f9c008040 comp rx=0 tx=0).stop 2023-07-13T22:24:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fb00b1ac0 msgr2=0x7f3fb00cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fb00b1ac0 0x7f3fb00cb760 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40090b0 tx=0x7f3fa4004750 comp rx=0 tx=0).stop 2023-07-13T22:24:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 shutdown_connections 2023-07-13T22:24:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3f9406ace0 0x7f3f9406d190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fb00b1ac0 0x7f3fb00cb760 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fb00cede0 0x7f3fb01f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 --2- 172.21.15.60:0/2395513097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b8200 0x7f3fb01f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 >> 172.21.15.60:0/2395513097 conn(0x7f3fb006b2f0 msgr2=0x7f3fb00b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 shutdown_connections 2023-07-13T22:24:42.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.408+0000 7f3fb71ff700 1 -- 172.21.15.60:0/2395513097 wait complete. 2023-07-13T22:24:42.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:24:42.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.633+0000 7f4e7ffff700 1 Processor -- start 2023-07-13T22:24:42.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.634+0000 7f4e7ffff700 1 -- start start 2023-07-13T22:24:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.635+0000 7f4e7ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e780b8200 0x7f4e780cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.635+0000 7f4e7ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b1ac0 0x7f4e780cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.635+0000 7f4e7ffff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e780d1870 con 0x7f4e780cede0 2023-07-13T22:24:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.635+0000 7f4e7ffff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e780cf370 con 0x7f4e780b8200 2023-07-13T22:24:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.635+0000 7f4e7ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780d12b0 con 0x7f4e780b1ac0 2023-07-13T22:24:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.635+0000 7f4e7dd9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e780b8200 0x7f4e780cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.635+0000 7f4e7dd9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e780b8200 0x7f4e780cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52710/0 (socket says 172.21.15.60:52710) 2023-07-13T22:24:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.635+0000 7f4e7dd9b700 1 -- 172.21.15.60:0/2345799422 learned_addr learned my addr 172.21.15.60:0/2345799422 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:42.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.635+0000 7f4e7d59a700 1 --2- 172.21.15.60:0/2345799422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b1ac0 0x7f4e780cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.636+0000 7f4e7dd9b700 1 -- 172.21.15.60:0/2345799422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b1ac0 msgr2=0x7f4e780cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:42.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.636+0000 7f4e7dd9b700 1 --2- 172.21.15.60:0/2345799422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b1ac0 0x7f4e780cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.636+0000 7f4e7dd9b700 1 -- 172.21.15.60:0/2345799422 >> v1:172.21.15.105:6789/0 conn(0x7f4e780cede0 legacy=0x7f4e7815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:42.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.636+0000 7f4e7dd9b700 1 -- 172.21.15.60:0/2345799422 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e780d1590 con 0x7f4e780b8200 2023-07-13T22:24:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.636+0000 7f4e7dd9b700 1 --2- 172.21.15.60:0/2345799422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b8200 0x7f4e780cf640 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4e740098f0 tx=0x7f4e74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3f6ed8401049b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.636+0000 7f4e7cd99700 1 -- 172.21.15.60:0/2345799422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400b8a0 con 0x7f4e780b8200 2023-07-13T22:24:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.636+0000 7f4e7cd99700 1 -- 172.21.15.60:0/2345799422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7400ba00 con 0x7f4e780b8200 2023-07-13T22:24:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.636+0000 7f4e7cd99700 1 -- 172.21.15.60:0/2345799422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400bd90 con 0x7f4e780b8200 2023-07-13T22:24:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.637+0000 7f4e7ffff700 1 -- 172.21.15.60:0/2345799422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b8200 msgr2=0x7f4e780cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.637+0000 7f4e7ffff700 1 --2- 172.21.15.60:0/2345799422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b8200 0x7f4e780cf640 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4e740098f0 tx=0x7f4e74009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.637+0000 7f4e7ffff700 1 -- 172.21.15.60:0/2345799422 shutdown_connections 2023-07-13T22:24:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.637+0000 7f4e7ffff700 1 --2- 172.21.15.60:0/2345799422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b8200 0x7f4e780cf640 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.637+0000 7f4e7ffff700 1 --2- 172.21.15.60:0/2345799422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b1ac0 0x7f4e780cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.637+0000 7f4e7ffff700 1 -- 172.21.15.60:0/2345799422 >> 172.21.15.60:0/2345799422 conn(0x7f4e7806b2f0 msgr2=0x7f4e7806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.637+0000 7f4e7ffff700 1 -- 172.21.15.60:0/2345799422 shutdown_connections 2023-07-13T22:24:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.637+0000 7f4e7ffff700 1 -- 172.21.15.60:0/2345799422 wait complete. 2023-07-13T22:24:42.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.639+0000 7f4e7ffff700 1 Processor -- start 2023-07-13T22:24:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.639+0000 7f4e7ffff700 1 -- start start 2023-07-13T22:24:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b1ac0 0x7f4e780c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b8200 0x7f4e781e2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e780cede0 0x7f4e78148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78147a60 con 0x7f4e780b1ac0 2023-07-13T22:24:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7ffff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781488c0 con 0x7f4e780cede0 2023-07-13T22:24:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78147cb0 con 0x7f4e780b8200 2023-07-13T22:24:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b1ac0 0x7f4e780c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b1ac0 0x7f4e780c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52714/0 (socket says 172.21.15.60:52714) 2023-07-13T22:24:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7dd9b700 1 -- 172.21.15.60:0/1045305295 learned_addr learned my addr 172.21.15.60:0/1045305295 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7dd9b700 1 -- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b8200 msgr2=0x7f4e781e2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7d59a700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b8200 0x7f4e781e2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7e59c700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e780cede0 0x7f4e78148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7dd9b700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b8200 0x7f4e781e2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7dd9b700 1 -- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e780cede0 msgr2=0x7f4e78148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7dd9b700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e780cede0 0x7f4e78148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7dd9b700 1 -- 172.21.15.60:0/1045305295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e74009550 con 0x7f4e780b1ac0 2023-07-13T22:24:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.640+0000 7f4e7d59a700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b8200 0x7f4e781e2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.641+0000 7f4e7e59c700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e780cede0 0x7f4e78148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.641+0000 7f4e7dd9b700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b1ac0 0x7f4e780c5d20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4e74000c00 tx=0x7f4e74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:42.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.641+0000 7f4e6e7fc700 1 -- 172.21.15.60:0/1045305295 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74004480 con 0x7f4e780b1ac0 2023-07-13T22:24:42.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.641+0000 7f4e6e7fc700 1 -- 172.21.15.60:0/1045305295 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7401f070 con 0x7f4e780b1ac0 2023-07-13T22:24:42.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.641+0000 7f4e6e7fc700 1 -- 172.21.15.60:0/1045305295 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e740038c0 con 0x7f4e780b1ac0 2023-07-13T22:24:42.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.641+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e78147f90 con 0x7f4e780b1ac0 2023-07-13T22:24:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.641+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e78148550 con 0x7f4e780b1ac0 2023-07-13T22:24:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.642+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e78148ba0 con 0x7f4e780b1ac0 2023-07-13T22:24:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.643+0000 7f4e6e7fc700 1 -- 172.21.15.60:0/1045305295 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e74021030 con 0x7f4e780b1ac0 2023-07-13T22:24:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.644+0000 7f4e6e7fc700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e5c06ace0 0x7f4e5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.644+0000 7f4e6e7fc700 1 -- 172.21.15.60:0/1045305295 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e7408aa00 con 0x7f4e780b1ac0 2023-07-13T22:24:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.644+0000 7f4e7d59a700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e5c06ace0 0x7f4e5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:42.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.645+0000 7f4e7d59a700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e5c06ace0 0x7f4e5c06d190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e780b4f10 tx=0x7f4e68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:42.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.648+0000 7f4e6e7fc700 1 -- 172.21.15.60:0/1045305295 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4e74055aa0 con 0x7f4e780b1ac0 2023-07-13T22:24:42.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.855+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e781486e0 con 0x7f4e780b1ac0 2023-07-13T22:24:42.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.855+0000 7f4e6e7fc700 1 -- 172.21.15.60:0/1045305295 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e740590c0 con 0x7f4e780b1ac0 2023-07-13T22:24:42.838 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:24:42.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.857+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e5c06ace0 msgr2=0x7f4e5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e5c06ace0 0x7f4e5c06d190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e780b4f10 tx=0x7f4e68008040 comp rx=0 tx=0).stop 2023-07-13T22:24:42.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b1ac0 msgr2=0x7f4e780c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:42.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b1ac0 0x7f4e780c5d20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4e74000c00 tx=0x7f4e74004750 comp rx=0 tx=0).stop 2023-07-13T22:24:42.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 shutdown_connections 2023-07-13T22:24:42.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e5c06ace0 0x7f4e5c06d190 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e780b1ac0 0x7f4e780c5d20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e780cede0 0x7f4e78148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 --2- 172.21.15.60:0/1045305295 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e780b8200 0x7f4e781e2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:42.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 >> 172.21.15.60:0/1045305295 conn(0x7f4e7806b2f0 msgr2=0x7f4e780b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:42.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 shutdown_connections 2023-07-13T22:24:42.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:42.858+0000 7f4e7ffff700 1 -- 172.21.15.60:0/1045305295 wait complete. 2023-07-13T22:24:42.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:24:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.083+0000 7fbf6a5ff700 1 Processor -- start 2023-07-13T22:24:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.083+0000 7fbf6a5ff700 1 -- start start 2023-07-13T22:24:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.085+0000 7fbf6a5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf640b1ac0 0x7fbf640d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.085+0000 7fbf6a5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf640d1870 con 0x7fbf640cfd50 2023-07-13T22:24:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.085+0000 7fbf6a5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf640cf370 con 0x7fbf640b8200 2023-07-13T22:24:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.085+0000 7fbf6a5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf640d12b0 con 0x7fbf640b1ac0 2023-07-13T22:24:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.085+0000 7fbf637fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbf640cfd50 0x7fbf640d23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44088/0 (socket says 172.21.15.60:44088) 2023-07-13T22:24:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.085+0000 7fbf637fe700 1 -- 172.21.15.60:0/2929199819 learned_addr learned my addr 172.21.15.60:0/2929199819 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.085+0000 7fbf68b9c700 1 --2- 172.21.15.60:0/2929199819 >> v2:172.21.15.105:3300/0 conn(0x7fbf640b1ac0 0x7fbf640d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.085+0000 7fbf62ffd700 1 -- 172.21.15.60:0/2929199819 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992334504 0 0) 0x7fbf640d1870 con 0x7fbf640cfd50 2023-07-13T22:24:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.085+0000 7fbf62ffd700 1 -- 172.21.15.60:0/2929199819 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf4c0035b0 con 0x7fbf640cfd50 2023-07-13T22:24:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.086+0000 7fbf62ffd700 1 -- 172.21.15.60:0/2929199819 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1214870186 0 0) 0x7fbf4c0035b0 con 0x7fbf640cfd50 2023-07-13T22:24:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.086+0000 7fbf68b9c700 1 -- 172.21.15.60:0/2929199819 >> v1:172.21.15.105:6789/0 conn(0x7fbf640b8200 legacy=0x7fbf640cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.086+0000 7fbf68b9c700 1 -- 172.21.15.60:0/2929199819 >> v1:172.21.15.60:6789/0 conn(0x7fbf640cfd50 legacy=0x7fbf640d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.086+0000 7fbf68b9c700 1 -- 172.21.15.60:0/2929199819 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640d1590 con 0x7fbf640b1ac0 2023-07-13T22:24:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.086+0000 7fbf68b9c700 1 --2- 172.21.15.60:0/2929199819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b1ac0 0x7fbf640d2ad0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009d90 tx=0x7fbf5c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea28557a5369e596 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.086+0000 7fbf62ffd700 1 -- 172.21.15.60:0/2929199819 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00e070 con 0x7fbf640b1ac0 2023-07-13T22:24:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.086+0000 7fbf62ffd700 1 -- 172.21.15.60:0/2929199819 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c003a20 con 0x7fbf640b1ac0 2023-07-13T22:24:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.086+0000 7fbf62ffd700 1 -- 172.21.15.60:0/2929199819 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58001670 con 0x7fbf640b1ac0 2023-07-13T22:24:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.087+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/2929199819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b1ac0 msgr2=0x7fbf640d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.087+0000 7fbf6a5ff700 1 --2- 172.21.15.60:0/2929199819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b1ac0 0x7fbf640d2ad0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009d90 tx=0x7fbf5c00b330 comp rx=0 tx=0).stop 2023-07-13T22:24:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.087+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/2929199819 shutdown_connections 2023-07-13T22:24:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.087+0000 7fbf6a5ff700 1 --2- 172.21.15.60:0/2929199819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b1ac0 0x7fbf640d2ad0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.087+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/2929199819 >> 172.21.15.60:0/2929199819 conn(0x7fbf6406b2f0 msgr2=0x7fbf6406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.087+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/2929199819 shutdown_connections 2023-07-13T22:24:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.087+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/2929199819 wait complete. 2023-07-13T22:24:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.089+0000 7fbf6a5ff700 1 Processor -- start 2023-07-13T22:24:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.089+0000 7fbf6a5ff700 1 -- start start 2023-07-13T22:24:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf6a5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf640b1ac0 0x7fbf64156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf6a5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b8200 0x7fbf640b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf6a5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf640cfd50 0x7fbf640b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf6a5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cad70 con 0x7fbf640cfd50 2023-07-13T22:24:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf6a5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cbc50 con 0x7fbf640b8200 2023-07-13T22:24:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf6a5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cbf30 con 0x7fbf640b1ac0 2023-07-13T22:24:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf68b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf640cfd50 0x7fbf640b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf68b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf640cfd50 0x7fbf640b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52716/0 (socket says 172.21.15.60:52716) 2023-07-13T22:24:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf68b9c700 1 -- 172.21.15.60:0/3597138575 learned_addr learned my addr 172.21.15.60:0/3597138575 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf637fe700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b8200 0x7fbf640b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf63fff700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf640b1ac0 0x7fbf64156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf68b9c700 1 -- 172.21.15.60:0/3597138575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf640b1ac0 msgr2=0x7fbf64156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf68b9c700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf640b1ac0 0x7fbf64156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf68b9c700 1 -- 172.21.15.60:0/3597138575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b8200 msgr2=0x7fbf640b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf68b9c700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b8200 0x7fbf640b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.090+0000 7fbf68b9c700 1 -- 172.21.15.60:0/3597138575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c009a70 con 0x7fbf640cfd50 2023-07-13T22:24:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.091+0000 7fbf637fe700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b8200 0x7fbf640b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.091+0000 7fbf68b9c700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf640cfd50 0x7fbf640b3c20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbf6406bf10 tx=0x7fbf5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.091+0000 7fbf60ff9700 1 -- 172.21.15.60:0/3597138575 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c004720 con 0x7fbf640cfd50 2023-07-13T22:24:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.091+0000 7fbf60ff9700 1 -- 172.21.15.60:0/3597138575 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c004880 con 0x7fbf640cfd50 2023-07-13T22:24:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.091+0000 7fbf60ff9700 1 -- 172.21.15.60:0/3597138575 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c012650 con 0x7fbf640cfd50 2023-07-13T22:24:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.091+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf640cc210 con 0x7fbf640cfd50 2023-07-13T22:24:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.091+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf640cc7d0 con 0x7fbf640cfd50 2023-07-13T22:24:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.092+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6404e210 con 0x7fbf640cfd50 2023-07-13T22:24:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.093+0000 7fbf60ff9700 1 -- 172.21.15.60:0/3597138575 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf5c0127b0 con 0x7fbf640cfd50 2023-07-13T22:24:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.094+0000 7fbf60ff9700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5006ace0 0x7fbf5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.094+0000 7fbf60ff9700 1 -- 172.21.15.60:0/3597138575 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf5c08af10 con 0x7fbf640cfd50 2023-07-13T22:24:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.095+0000 7fbf63fff700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5006ace0 0x7fbf5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.095+0000 7fbf63fff700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5006ace0 0x7fbf5006d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000f60 tx=0x7fbf54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.098+0000 7fbf60ff9700 1 -- 172.21.15.60:0/3597138575 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbf5c055fb0 con 0x7fbf640cfd50 2023-07-13T22:24:43.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.305+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf640b4f70 con 0x7fbf640cfd50 2023-07-13T22:24:43.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.306+0000 7fbf60ff9700 1 -- 172.21.15.60:0/3597138575 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf5c0595d0 con 0x7fbf640cfd50 2023-07-13T22:24:43.288 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:24:43.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5006ace0 msgr2=0x7fbf5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5006ace0 0x7fbf5006d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000f60 tx=0x7fbf54009040 comp rx=0 tx=0).stop 2023-07-13T22:24:43.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf640cfd50 msgr2=0x7fbf640b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf640cfd50 0x7fbf640b3c20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbf6406bf10 tx=0x7fbf5c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 shutdown_connections 2023-07-13T22:24:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf5006ace0 0x7fbf5006d190 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf640cfd50 0x7fbf640b3c20 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf640b1ac0 0x7fbf64156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 --2- 172.21.15.60:0/3597138575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf640b8200 0x7fbf640b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 >> 172.21.15.60:0/3597138575 conn(0x7fbf6406b2f0 msgr2=0x7fbf640b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 shutdown_connections 2023-07-13T22:24:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.308+0000 7fbf6a5ff700 1 -- 172.21.15.60:0/3597138575 wait complete. 2023-07-13T22:24:43.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:24:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.531+0000 7f38dcdff700 1 Processor -- start 2023-07-13T22:24:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.532+0000 7f38dcdff700 1 -- start start 2023-07-13T22:24:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.533+0000 7f38dcdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38d815a820 0x7f38d815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.533+0000 7f38dcdff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d8070270 con 0x7f38d8154950 2023-07-13T22:24:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.533+0000 7f38dcdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d815b2e0 con 0x7f38d80c8d40 2023-07-13T22:24:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.533+0000 7f38dcdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d80713d0 con 0x7f38d815a820 2023-07-13T22:24:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.533+0000 7f38cffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38d815a820 0x7f38d815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.533+0000 7f38d659c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f38d8154950 0x7f38d8070cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44094/0 (socket says 172.21.15.60:44094) 2023-07-13T22:24:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.533+0000 7f38cffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38d815a820 0x7f38d815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52724/0 (socket says 172.21.15.60:52724) 2023-07-13T22:24:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.533+0000 7f38d659c700 1 -- 172.21.15.60:0/4271828720 learned_addr learned my addr 172.21.15.60:0/4271828720 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.534+0000 7f38cffff700 1 -- 172.21.15.60:0/4271828720 >> v1:172.21.15.105:6789/0 conn(0x7f38d80c8d40 legacy=0x7f38d80c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.534+0000 7f38cffff700 1 -- 172.21.15.60:0/4271828720 >> v1:172.21.15.60:6789/0 conn(0x7f38d8154950 legacy=0x7f38d8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.534+0000 7f38cffff700 1 -- 172.21.15.60:0/4271828720 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d815b5c0 con 0x7f38d815a820 2023-07-13T22:24:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.534+0000 7f38d5d9b700 1 -- 172.21.15.60:0/4271828720 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783064566 0 0) 0x7f38d8070270 con 0x7f38d8154950 2023-07-13T22:24:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.534+0000 7f38cffff700 1 --2- 172.21.15.60:0/4271828720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d815a820 0x7f38d815ac70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009840 tx=0x7f38c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfac36e887fc072 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.534+0000 7f38d5d9b700 1 -- 172.21.15.60:0/4271828720 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0011830 con 0x7f38d815a820 2023-07-13T22:24:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.534+0000 7f38d5d9b700 1 -- 172.21.15.60:0/4271828720 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c0011990 con 0x7f38d815a820 2023-07-13T22:24:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.534+0000 7f38d5d9b700 1 -- 172.21.15.60:0/4271828720 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0011d20 con 0x7f38d815a820 2023-07-13T22:24:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.535+0000 7f38dcdff700 1 -- 172.21.15.60:0/4271828720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d815a820 msgr2=0x7f38d815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.535+0000 7f38dcdff700 1 --2- 172.21.15.60:0/4271828720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d815a820 0x7f38d815ac70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009840 tx=0x7f38c0009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.535+0000 7f38dcdff700 1 -- 172.21.15.60:0/4271828720 shutdown_connections 2023-07-13T22:24:43.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.535+0000 7f38dcdff700 1 --2- 172.21.15.60:0/4271828720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d815a820 0x7f38d815ac70 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.535+0000 7f38dcdff700 1 -- 172.21.15.60:0/4271828720 >> 172.21.15.60:0/4271828720 conn(0x7f38d80cf410 msgr2=0x7f38d80cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:43.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.535+0000 7f38dcdff700 1 -- 172.21.15.60:0/4271828720 shutdown_connections 2023-07-13T22:24:43.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.535+0000 7f38dcdff700 1 -- 172.21.15.60:0/4271828720 wait complete. 2023-07-13T22:24:43.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.537+0000 7f38dcdff700 1 Processor -- start 2023-07-13T22:24:43.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.537+0000 7f38dcdff700 1 -- start start 2023-07-13T22:24:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38dcdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d80c8d40 0x7f38d80b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38dcdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8154950 0x7f38d814f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38dcdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38d815a820 0x7f38d8153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38dcdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d8150450 con 0x7f38d80c8d40 2023-07-13T22:24:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38dcdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80cdd30 con 0x7f38d8154950 2023-07-13T22:24:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38dcdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80ce010 con 0x7f38d815a820 2023-07-13T22:24:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38d659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d80c8d40 0x7f38d80b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38d659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d80c8d40 0x7f38d80b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52738/0 (socket says 172.21.15.60:52738) 2023-07-13T22:24:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38d659c700 1 -- 172.21.15.60:0/2299483432 learned_addr learned my addr 172.21.15.60:0/2299483432 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38cffff700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8154950 0x7f38d814f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38d659c700 1 -- 172.21.15.60:0/2299483432 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38d815a820 msgr2=0x7f38d8153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38d659c700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38d815a820 0x7f38d8153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38d659c700 1 -- 172.21.15.60:0/2299483432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8154950 msgr2=0x7f38d814f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38d659c700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8154950 0x7f38d814f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.538+0000 7f38d659c700 1 -- 172.21.15.60:0/2299483432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00094a0 con 0x7f38d80c8d40 2023-07-13T22:24:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.539+0000 7f38d659c700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d80c8d40 0x7f38d80b3e80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c520 tx=0x7f38c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.539+0000 7f38ceffd700 1 -- 172.21.15.60:0/2299483432 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8004c80 con 0x7f38d80c8d40 2023-07-13T22:24:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.539+0000 7f38ceffd700 1 -- 172.21.15.60:0/2299483432 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8004e00 con 0x7f38d80c8d40 2023-07-13T22:24:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.539+0000 7f38ceffd700 1 -- 172.21.15.60:0/2299483432 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8007310 con 0x7f38d80c8d40 2023-07-13T22:24:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.539+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d80b1770 con 0x7f38d80c8d40 2023-07-13T22:24:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.539+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d80b1d30 con 0x7f38d80c8d40 2023-07-13T22:24:43.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.540+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d80b1f20 con 0x7f38d80c8d40 2023-07-13T22:24:43.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.541+0000 7f38ceffd700 1 -- 172.21.15.60:0/2299483432 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38c8007470 con 0x7f38d80c8d40 2023-07-13T22:24:43.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.542+0000 7f38ceffd700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38b806ace0 0x7f38b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:43.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.542+0000 7f38ceffd700 1 -- 172.21.15.60:0/2299483432 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38c80892a0 con 0x7f38d80c8d40 2023-07-13T22:24:43.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.542+0000 7f38cffff700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38b806ace0 0x7f38b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:43.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.543+0000 7f38cffff700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38b806ace0 0x7f38b806d190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f38d81e6a30 tx=0x7f38c001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:43.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.546+0000 7f38ceffd700 1 -- 172.21.15.60:0/2299483432 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f38c80542b0 con 0x7f38d80c8d40 2023-07-13T22:24:43.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.772+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38d80cbc20 con 0x7f38d80c8d40 2023-07-13T22:24:43.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.772+0000 7f38ceffd700 1 -- 172.21.15.60:0/2299483432 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f38c80578d0 con 0x7f38d80c8d40 2023-07-13T22:24:43.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38b806ace0 msgr2=0x7f38b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38b806ace0 0x7f38b806d190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f38d81e6a30 tx=0x7f38c001c040 comp rx=0 tx=0).stop 2023-07-13T22:24:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d80c8d40 msgr2=0x7f38d80b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d80c8d40 0x7f38d80b3e80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c520 tx=0x7f38c800c8e0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 shutdown_connections 2023-07-13T22:24:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f38b806ace0 0x7f38b806d190 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38d80c8d40 0x7f38d80b3e80 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38d815a820 0x7f38d8153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 --2- 172.21.15.60:0/2299483432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8154950 0x7f38d814f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 >> 172.21.15.60:0/2299483432 conn(0x7f38d80cf410 msgr2=0x7f38d8152ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 shutdown_connections 2023-07-13T22:24:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:43.775+0000 7f38dcdff700 1 -- 172.21.15.60:0/2299483432 wait complete. 2023-07-13T22:24:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:24:43.775 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:24:43.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:43.776 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:24:43.849 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[122675]: fuse finished with error 0 and tester_r 0 2023-07-13T22:24:43.853 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:24:43.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:43.854 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:24:43.914 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:24:43.915 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:43.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:24:43.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:24:43.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:43.963 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:24:44.048 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-13T22:24:44.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:24:44.049 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:24:44.049 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:44.096 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:24:44.096 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:24:44.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:24:44.097 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:24:44.181 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-13T22:24:44.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:24:44.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:44.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.404+0000 7f1915fff700 1 Processor -- start 2023-07-13T22:24:44.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.404+0000 7f1915fff700 1 -- start start 2023-07-13T22:24:44.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.405+0000 7f1915fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100d3750 0x7f19100cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.405+0000 7f1915fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19100cd950 con 0x7f19100cfad0 2023-07-13T22:24:44.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.405+0000 7f1915fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19100cff20 con 0x7f19100b8400 2023-07-13T22:24:44.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.405+0000 7f1915fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19100d0ac0 con 0x7f19100d3750 2023-07-13T22:24:44.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190ffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f19100cfad0 0x7f19100d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44100/0 (socket says 172.21.15.60:44100) 2023-07-13T22:24:44.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190ffff700 1 -- 172.21.15.60:0/2625967949 learned_addr learned my addr 172.21.15.60:0/2625967949 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:44.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190effd700 1 --2- 172.21.15.60:0/2625967949 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100d3750 0x7f19100cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190e7fc700 1 -- 172.21.15.60:0/2625967949 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860903054 0 0) 0x7f1908001490 con 0x7f19100cfad0 2023-07-13T22:24:44.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190e7fc700 1 -- 172.21.15.60:0/2625967949 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f80035b0 con 0x7f19100cfad0 2023-07-13T22:24:44.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190e7fc700 1 -- 172.21.15.60:0/2625967949 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99474948 0 0) 0x7f1900001110 con 0x7f19100b8400 2023-07-13T22:24:44.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190e7fc700 1 -- 172.21.15.60:0/2625967949 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1908001490 con 0x7f19100b8400 2023-07-13T22:24:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190effd700 1 -- 172.21.15.60:0/2625967949 >> v1:172.21.15.105:6789/0 conn(0x7f19100b8400 legacy=0x7f19100cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190effd700 1 -- 172.21.15.60:0/2625967949 >> v1:172.21.15.60:6789/0 conn(0x7f19100cfad0 legacy=0x7f19100d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.406+0000 7f190effd700 1 -- 172.21.15.60:0/2625967949 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100b87d0 con 0x7f19100d3750 2023-07-13T22:24:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.407+0000 7f190effd700 1 --2- 172.21.15.60:0/2625967949 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100d3750 0x7f19100cf590 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1904009870 tx=0x7f1904009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597bd4a63b9e0e04 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.407+0000 7f190e7fc700 1 -- 172.21.15.60:0/2625967949 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904011830 con 0x7f19100d3750 2023-07-13T22:24:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.407+0000 7f190e7fc700 1 -- 172.21.15.60:0/2625967949 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1904011990 con 0x7f19100d3750 2023-07-13T22:24:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.407+0000 7f190e7fc700 1 -- 172.21.15.60:0/2625967949 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904011d00 con 0x7f19100d3750 2023-07-13T22:24:44.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.408+0000 7f1915fff700 1 -- 172.21.15.60:0/2625967949 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100d3750 msgr2=0x7f19100cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.408+0000 7f1915fff700 1 --2- 172.21.15.60:0/2625967949 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100d3750 0x7f19100cf590 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1904009870 tx=0x7f1904009b80 comp rx=0 tx=0).stop 2023-07-13T22:24:44.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.408+0000 7f1915fff700 1 -- 172.21.15.60:0/2625967949 shutdown_connections 2023-07-13T22:24:44.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.408+0000 7f1915fff700 1 --2- 172.21.15.60:0/2625967949 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100d3750 0x7f19100cf590 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.408+0000 7f1915fff700 1 -- 172.21.15.60:0/2625967949 >> 172.21.15.60:0/2625967949 conn(0x7f191006b390 msgr2=0x7f191006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:44.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.408+0000 7f1915fff700 1 -- 172.21.15.60:0/2625967949 shutdown_connections 2023-07-13T22:24:44.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.408+0000 7f1915fff700 1 -- 172.21.15.60:0/2625967949 wait complete. 2023-07-13T22:24:44.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.409+0000 7f1915fff700 1 Processor -- start 2023-07-13T22:24:44.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.410+0000 7f1915fff700 1 -- start start 2023-07-13T22:24:44.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.410+0000 7f1915fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19100b8400 0x7f19100cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f1915fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100cfad0 0x7f19100c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f1915fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19100d3750 0x7f19100b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f1915fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19101e7950 con 0x7f19100d3750 2023-07-13T22:24:44.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f1915fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19101e7020 con 0x7f19100b8400 2023-07-13T22:24:44.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f1915fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19101e7300 con 0x7f19100cfad0 2023-07-13T22:24:44.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19100d3750 0x7f19100b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19100b8400 0x7f19100cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19100d3750 0x7f19100b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52752/0 (socket says 172.21.15.60:52752) 2023-07-13T22:24:44.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190ffff700 1 -- 172.21.15.60:0/3531201107 learned_addr learned my addr 172.21.15.60:0/3531201107 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:44.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190effd700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100cfad0 0x7f19100c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190ffff700 1 -- 172.21.15.60:0/3531201107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100cfad0 msgr2=0x7f19100c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190ffff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100cfad0 0x7f19100c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190ffff700 1 -- 172.21.15.60:0/3531201107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19100b8400 msgr2=0x7f19100cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190ffff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19100b8400 0x7f19100cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.411+0000 7f190ffff700 1 -- 172.21.15.60:0/3531201107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19040094a0 con 0x7f19100d3750 2023-07-13T22:24:44.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.412+0000 7f190ffff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19100d3750 0x7f19100b6c70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f190800cc30 tx=0x7f190800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:44.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.412+0000 7f18f7fff700 1 -- 172.21.15.60:0/3531201107 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190800ab00 con 0x7f19100d3750 2023-07-13T22:24:44.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.412+0000 7f18f7fff700 1 -- 172.21.15.60:0/3531201107 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190800ac60 con 0x7f19100d3750 2023-07-13T22:24:44.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.412+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19101e75e0 con 0x7f19100d3750 2023-07-13T22:24:44.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.412+0000 7f18f7fff700 1 -- 172.21.15.60:0/3531201107 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908012400 con 0x7f19100d3750 2023-07-13T22:24:44.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.412+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1910151320 con 0x7f19100d3750 2023-07-13T22:24:44.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.413+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19101e7ac0 con 0x7f19100d3750 2023-07-13T22:24:44.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.414+0000 7f18f7fff700 1 -- 172.21.15.60:0/3531201107 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f190801b3b0 con 0x7f19100d3750 2023-07-13T22:24:44.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.415+0000 7f18f7fff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f18fc06ace0 0x7f18fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.415+0000 7f18f7fff700 1 -- 172.21.15.60:0/3531201107 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1908089200 con 0x7f19100d3750 2023-07-13T22:24:44.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.415+0000 7f190f7fe700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f18fc06ace0 0x7f18fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.416+0000 7f190f7fe700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f18fc06ace0 0x7f18fc06d190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19000013b0 tx=0x7f1900009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:44.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.419+0000 7f18f7fff700 1 -- 172.21.15.60:0/3531201107 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1908054180 con 0x7f19100d3750 2023-07-13T22:24:44.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.683+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19100cd950 con 0x7f19100d3750 2023-07-13T22:24:44.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.683+0000 7f18f7fff700 1 -- 172.21.15.60:0/3531201107 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f19100cd950 con 0x7f19100d3750 2023-07-13T22:24:44.666 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:44.666 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/711733521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":711733521},{"type":"v1","addr":"172.21.15.105:6834","nonce":711733521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2135065566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":2135065566},{"type":"v1","addr":"172.21.15.105:6835","nonce":2135065566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6052,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4236704538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4236704538},{"type":"v1","addr":"172.21.15.60:6836","nonce":4236704538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:24:20.139470+0000","modified":"2023-07-13T22:24:23.665486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6058":{"gid":6058,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3036622731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3036622731},{"type":"v1","addr":"172.21.15.60:6837","nonce":3036622731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-13T22:24:44.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f18fc06ace0 msgr2=0x7f18fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f18fc06ace0 0x7f18fc06d190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19000013b0 tx=0x7f1900009040 comp rx=0 tx=0).stop 2023-07-13T22:24:44.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19100d3750 msgr2=0x7f19100b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19100d3750 0x7f19100b6c70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f190800cc30 tx=0x7f190800cf40 comp rx=0 tx=0).stop 2023-07-13T22:24:44.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 shutdown_connections 2023-07-13T22:24:44.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f18fc06ace0 0x7f18fc06d190 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19100d3750 0x7f19100b6c70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19100b8400 0x7f19100cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 --2- 172.21.15.60:0/3531201107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19100cfad0 0x7f19100c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 >> 172.21.15.60:0/3531201107 conn(0x7f191006b390 msgr2=0x7f19100732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:44.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 shutdown_connections 2023-07-13T22:24:44.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.686+0000 7f1915fff700 1 -- 172.21.15.60:0/3531201107 wait complete. 2023-07-13T22:24:44.671 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 33 2023-07-13T22:24:44.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:44.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.908+0000 7fc1cd5ff700 1 Processor -- start 2023-07-13T22:24:44.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.908+0000 7fc1cd5ff700 1 -- start start 2023-07-13T22:24:44.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.909+0000 7fc1cd5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1c80d15c0 0x7fc1c8167030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.909+0000 7fc1cd5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c80d09d0 con 0x7fc1c80b8570 2023-07-13T22:24:44.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.909+0000 7fc1cd5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c80cda10 con 0x7fc1c80d35c0 2023-07-13T22:24:44.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.909+0000 7fc1cd5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c80d3b30 con 0x7fc1c80d15c0 2023-07-13T22:24:44.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.909+0000 7fc1c77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1c80d15c0 0x7fc1c8167030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.909+0000 7fc1c77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1c80d15c0 0x7fc1c8167030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47504/0 (socket says 172.21.15.60:47504) 2023-07-13T22:24:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.909+0000 7fc1c77fe700 1 -- 172.21.15.60:0/44662324 learned_addr learned my addr 172.21.15.60:0/44662324 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.909+0000 7fc1c6ffd700 1 --1- 172.21.15.60:0/44662324 >> v1:172.21.15.60:6789/0 conn(0x7fc1c80b8570 0x7fc1c80ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59824/0 (socket says 172.21.15.60:59824) 2023-07-13T22:24:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.910+0000 7fc1c77fe700 1 -- 172.21.15.60:0/44662324 >> v1:172.21.15.105:6789/0 conn(0x7fc1c80d35c0 legacy=0x7fc1c80d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.910+0000 7fc1c77fe700 1 -- 172.21.15.60:0/44662324 >> v1:172.21.15.60:6789/0 conn(0x7fc1c80b8570 legacy=0x7fc1c80ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.910+0000 7fc1c77fe700 1 -- 172.21.15.60:0/44662324 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80cd450 con 0x7fc1c80d15c0 2023-07-13T22:24:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.910+0000 7fc1c77fe700 1 --2- 172.21.15.60:0/44662324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d15c0 0x7fc1c8167030 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009e50 tx=0x7fc1c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125cbf73c4c80805 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.910+0000 7fc1c5ffb700 1 -- 172.21.15.60:0/44662324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000e070 con 0x7fc1c80d15c0 2023-07-13T22:24:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.910+0000 7fc1c5ffb700 1 -- 172.21.15.60:0/44662324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c000b9c0 con 0x7fc1c80d15c0 2023-07-13T22:24:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.910+0000 7fc1c5ffb700 1 -- 172.21.15.60:0/44662324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000bd70 con 0x7fc1c80d15c0 2023-07-13T22:24:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.911+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/44662324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d15c0 msgr2=0x7fc1c8167030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.911+0000 7fc1cd5ff700 1 --2- 172.21.15.60:0/44662324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d15c0 0x7fc1c8167030 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009e50 tx=0x7fc1c000b330 comp rx=0 tx=0).stop 2023-07-13T22:24:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.911+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/44662324 shutdown_connections 2023-07-13T22:24:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.911+0000 7fc1cd5ff700 1 --2- 172.21.15.60:0/44662324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d15c0 0x7fc1c8167030 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.911+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/44662324 >> 172.21.15.60:0/44662324 conn(0x7fc1c8071600 msgr2=0x7fc1c80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:44.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.911+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/44662324 shutdown_connections 2023-07-13T22:24:44.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.911+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/44662324 wait complete. 2023-07-13T22:24:44.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.913+0000 7fc1cd5ff700 1 Processor -- start 2023-07-13T22:24:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.913+0000 7fc1cd5ff700 1 -- start start 2023-07-13T22:24:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1cd5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80b8570 0x7fc1c80a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1cd5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80d15c0 0x7fc1c81f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1cd5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d35c0 0x7fc1c815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1cd5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80a9c40 con 0x7fc1c80d35c0 2023-07-13T22:24:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1cd5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80cccb0 con 0x7fc1c80b8570 2023-07-13T22:24:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1cd5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80a9960 con 0x7fc1c80d15c0 2023-07-13T22:24:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1c77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d35c0 0x7fc1c815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1c77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d35c0 0x7fc1c815ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47508/0 (socket says 172.21.15.60:47508) 2023-07-13T22:24:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1c77fe700 1 -- 172.21.15.60:0/3094781614 learned_addr learned my addr 172.21.15.60:0/3094781614 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1c6ffd700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80b8570 0x7fc1c80a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.914+0000 7fc1c67fc700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80d15c0 0x7fc1c81f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1c77fe700 1 -- 172.21.15.60:0/3094781614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80d15c0 msgr2=0x7fc1c81f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1c77fe700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80d15c0 0x7fc1c81f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1c77fe700 1 -- 172.21.15.60:0/3094781614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80b8570 msgr2=0x7fc1c80a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1c77fe700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80b8570 0x7fc1c80a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1c77fe700 1 -- 172.21.15.60:0/3094781614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c0009b30 con 0x7fc1c80d35c0 2023-07-13T22:24:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1c77fe700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d35c0 0x7fc1c815ec40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80d3d00 tx=0x7fc1c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1af7fe700 1 -- 172.21.15.60:0/3094781614 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0004800 con 0x7fc1c80d35c0 2023-07-13T22:24:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1af7fe700 1 -- 172.21.15.60:0/3094781614 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c0004960 con 0x7fc1c80d35c0 2023-07-13T22:24:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1af7fe700 1 -- 172.21.15.60:0/3094781614 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c001c680 con 0x7fc1c80d35c0 2023-07-13T22:24:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.915+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c80cb650 con 0x7fc1c80d35c0 2023-07-13T22:24:44.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.916+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c80cbc10 con 0x7fc1c80d35c0 2023-07-13T22:24:44.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.916+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c80cb4b0 con 0x7fc1c80d35c0 2023-07-13T22:24:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.917+0000 7fc1af7fe700 1 -- 172.21.15.60:0/3094781614 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc1c001c7e0 con 0x7fc1c80d35c0 2023-07-13T22:24:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.918+0000 7fc1af7fe700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1b406ace0 0x7fc1b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.918+0000 7fc1af7fe700 1 -- 172.21.15.60:0/3094781614 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc1c008b240 con 0x7fc1c80d35c0 2023-07-13T22:24:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.919+0000 7fc1c6ffd700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1b406ace0 0x7fc1b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.919+0000 7fc1c6ffd700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1b406ace0 0x7fc1b406d190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8001020 tx=0x7fc1b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:44.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:44.923+0000 7fc1af7fe700 1 -- 172.21.15.60:0/3094781614 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc1c80cb4b0 con 0x7fc1c80d35c0 2023-07-13T22:24:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.131+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1c80cb4b0 con 0x7fc1c80d35c0 2023-07-13T22:24:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.132+0000 7fc1af7fe700 1 -- 172.21.15.60:0/3094781614 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+18439 (secure 0 0 0) 0x7fc1c80cb4b0 con 0x7fc1c80d35c0 2023-07-13T22:24:45.115 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:45.115 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":63,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:44.480091+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:24:18.441889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-13T22:24:19.114200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.11","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.134+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1b406ace0 msgr2=0x7fc1b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.134+0000 7fc1cd5ff700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1b406ace0 0x7fc1b406d190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8001020 tx=0x7fc1b8009040 comp rx=0 tx=0).stop 2023-07-13T22:24:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.134+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d35c0 msgr2=0x7fc1c815ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.134+0000 7fc1cd5ff700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d35c0 0x7fc1c815ec40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80d3d00 tx=0x7fc1c0004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.135+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 shutdown_connections 2023-07-13T22:24:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.135+0000 7fc1cd5ff700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1b406ace0 0x7fc1b406d190 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.135+0000 7fc1cd5ff700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c80d35c0 0x7fc1c815ec40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.135+0000 7fc1cd5ff700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c80b8570 0x7fc1c80a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.135+0000 7fc1cd5ff700 1 --2- 172.21.15.60:0/3094781614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1c80d15c0 0x7fc1c81f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.135+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 >> 172.21.15.60:0/3094781614 conn(0x7fc1c8071600 msgr2=0x7fc1c80d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.135+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 shutdown_connections 2023-07-13T22:24:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.135+0000 7fc1cd5ff700 1 -- 172.21.15.60:0/3094781614 wait complete. 2023-07-13T22:24:45.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:24:18.441889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:24:19.114200+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:24:45.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:24:45.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.360+0000 7fc6f9bff700 1 Processor -- start 2023-07-13T22:24:45.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.360+0000 7fc6f9bff700 1 -- start start 2023-07-13T22:24:45.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.361+0000 7fc6f9bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 0x7fc6f40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.361+0000 7fc6f9bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f40d1870 con 0x7fc6f40cede0 2023-07-13T22:24:45.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.361+0000 7fc6f9bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f4160d80 con 0x7fc6f40b1ac0 2023-07-13T22:24:45.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.361+0000 7fc6f9bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40cf370 con 0x7fc6f40b8200 2023-07-13T22:24:45.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.362+0000 7fc6f3fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc6f40cede0 0x7fc6f4160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59834/0 (socket says 172.21.15.60:59834) 2023-07-13T22:24:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.362+0000 7fc6f3fff700 1 -- 172.21.15.60:0/3741697357 learned_addr learned my addr 172.21.15.60:0/3741697357 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.362+0000 7fc6f37fe700 1 --2- 172.21.15.60:0/3741697357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 0x7fc6f40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.362+0000 7fc6f27fc700 1 -- 172.21.15.60:0/3741697357 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302497531 0 0) 0x7fc6f40d1870 con 0x7fc6f40cede0 2023-07-13T22:24:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.362+0000 7fc6f27fc700 1 -- 172.21.15.60:0/3741697357 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6dc0035b0 con 0x7fc6f40cede0 2023-07-13T22:24:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.362+0000 7fc6f27fc700 1 -- 172.21.15.60:0/3741697357 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171021608 0 0) 0x7fc6f4160d80 con 0x7fc6f40b1ac0 2023-07-13T22:24:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.362+0000 7fc6f37fe700 1 -- 172.21.15.60:0/3741697357 >> v1:172.21.15.105:6789/0 conn(0x7fc6f40b1ac0 legacy=0x7fc6f415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.362+0000 7fc6f37fe700 1 -- 172.21.15.60:0/3741697357 >> v1:172.21.15.60:6789/0 conn(0x7fc6f40cede0 legacy=0x7fc6f4160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.362+0000 7fc6f37fe700 1 -- 172.21.15.60:0/3741697357 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40d12b0 con 0x7fc6f40b8200 2023-07-13T22:24:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f37fe700 1 --2- 172.21.15.60:0/3741697357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 0x7fc6f40cf640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009870 tx=0x7fc6e4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644a67f890b7e673 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f27fc700 1 -- 172.21.15.60:0/3741697357 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4011830 con 0x7fc6f40b8200 2023-07-13T22:24:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f27fc700 1 -- 172.21.15.60:0/3741697357 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e4011990 con 0x7fc6f40b8200 2023-07-13T22:24:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f27fc700 1 -- 172.21.15.60:0/3741697357 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4011d20 con 0x7fc6f40b8200 2023-07-13T22:24:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f9bff700 1 -- 172.21.15.60:0/3741697357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 msgr2=0x7fc6f40cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f9bff700 1 --2- 172.21.15.60:0/3741697357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 0x7fc6f40cf640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009870 tx=0x7fc6e4009b80 comp rx=0 tx=0).stop 2023-07-13T22:24:45.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f9bff700 1 -- 172.21.15.60:0/3741697357 shutdown_connections 2023-07-13T22:24:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f9bff700 1 --2- 172.21.15.60:0/3741697357 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 0x7fc6f40cf640 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f9bff700 1 -- 172.21.15.60:0/3741697357 >> 172.21.15.60:0/3741697357 conn(0x7fc6f406b2f0 msgr2=0x7fc6f406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f9bff700 1 -- 172.21.15.60:0/3741697357 shutdown_connections 2023-07-13T22:24:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.363+0000 7fc6f9bff700 1 -- 172.21.15.60:0/3741697357 wait complete. 2023-07-13T22:24:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.365+0000 7fc6f9bff700 1 Processor -- start 2023-07-13T22:24:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.365+0000 7fc6f9bff700 1 -- start start 2023-07-13T22:24:45.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.366+0000 7fc6f9bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f40c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.366+0000 7fc6f9bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b8200 0x7fc6f415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.366+0000 7fc6f9bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 0x7fc6f415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.366+0000 7fc6f9bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f415cb90 con 0x7fc6f40b8200 2023-07-13T22:24:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.366+0000 7fc6f9bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f415ce70 con 0x7fc6f40b1ac0 2023-07-13T22:24:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.366+0000 7fc6f9bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f415d150 con 0x7fc6f40cede0 2023-07-13T22:24:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b8200 0x7fc6f415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f37fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f40c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b8200 0x7fc6f415ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47514/0 (socket says 172.21.15.60:47514) 2023-07-13T22:24:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f2ffd700 1 -- 172.21.15.60:0/1461643353 learned_addr learned my addr 172.21.15.60:0/1461643353 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f3fff700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 0x7fc6f415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f2ffd700 1 -- 172.21.15.60:0/1461643353 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 msgr2=0x7fc6f415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f2ffd700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 0x7fc6f415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f2ffd700 1 -- 172.21.15.60:0/1461643353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b1ac0 msgr2=0x7fc6f40c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f2ffd700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f40c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f2ffd700 1 -- 172.21.15.60:0/1461643353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e40094a0 con 0x7fc6f40b8200 2023-07-13T22:24:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.367+0000 7fc6f2ffd700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b8200 0x7fc6f415ea00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80011e0 tx=0x7fc6e800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.368+0000 7fc6dbfff700 1 -- 172.21.15.60:0/1461643353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80054e0 con 0x7fc6f40b8200 2023-07-13T22:24:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.368+0000 7fc6dbfff700 1 -- 172.21.15.60:0/1461643353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e8003680 con 0x7fc6f40b8200 2023-07-13T22:24:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.368+0000 7fc6dbfff700 1 -- 172.21.15.60:0/1461643353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8003a30 con 0x7fc6f40b8200 2023-07-13T22:24:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.368+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f415d430 con 0x7fc6f40b8200 2023-07-13T22:24:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.368+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f40c91b0 con 0x7fc6f40b8200 2023-07-13T22:24:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.369+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f415d950 con 0x7fc6f40b8200 2023-07-13T22:24:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.370+0000 7fc6dbfff700 1 -- 172.21.15.60:0/1461643353 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6e8003b90 con 0x7fc6f40b8200 2023-07-13T22:24:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.371+0000 7fc6dbfff700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6e006ace0 0x7fc6e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.371+0000 7fc6dbfff700 1 -- 172.21.15.60:0/1461643353 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc6e8088950 con 0x7fc6f40b8200 2023-07-13T22:24:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.371+0000 7fc6f37fe700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6e006ace0 0x7fc6e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.371+0000 7fc6f37fe700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6e006ace0 0x7fc6e006d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40097c0 tx=0x7fc6e401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:45.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.375+0000 7fc6dbfff700 1 -- 172.21.15.60:0/1461643353 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc6f415d950 con 0x7fc6f40b8200 2023-07-13T22:24:45.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.595+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc6f415d950 con 0x7fc6f40b8200 2023-07-13T22:24:45.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.596+0000 7fc6dbfff700 1 -- 172.21.15.60:0/1461643353 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc6f415d950 con 0x7fc6f40b8200 2023-07-13T22:24:45.579 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:24:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6e006ace0 msgr2=0x7fc6e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6e006ace0 0x7fc6e006d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40097c0 tx=0x7fc6e401c040 comp rx=0 tx=0).stop 2023-07-13T22:24:45.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b8200 msgr2=0x7fc6f415ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b8200 0x7fc6f415ea00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80011e0 tx=0x7fc6e800ca30 comp rx=0 tx=0).stop 2023-07-13T22:24:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 shutdown_connections 2023-07-13T22:24:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6e006ace0 0x7fc6e006d190 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 0x7fc6f415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f40c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 --2- 172.21.15.60:0/1461643353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b8200 0x7fc6f415ea00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 >> 172.21.15.60:0/1461643353 conn(0x7fc6f406b2f0 msgr2=0x7fc6f4071600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 shutdown_connections 2023-07-13T22:24:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.598+0000 7fc6f9bff700 1 -- 172.21.15.60:0/1461643353 wait complete. 2023-07-13T22:24:45.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:24:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.821+0000 7f2fe51ff700 1 Processor -- start 2023-07-13T22:24:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.821+0000 7f2fe51ff700 1 -- start start 2023-07-13T22:24:45.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.822+0000 7f2fe51ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2fe00cfd50 0x7f2fe00d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.822+0000 7f2fe51ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe00d1870 con 0x7f2fe00b8200 2023-07-13T22:24:45.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.822+0000 7f2fe51ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe00cf370 con 0x7f2fe00b1ac0 2023-07-13T22:24:45.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.822+0000 7f2fe51ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe00d12b0 con 0x7f2fe00cfd50 2023-07-13T22:24:45.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.822+0000 7f2fded9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2fe00b8200 0x7f2fe00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59846/0 (socket says 172.21.15.60:59846) 2023-07-13T22:24:45.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.822+0000 7f2fded9d700 1 -- 172.21.15.60:0/1079200668 learned_addr learned my addr 172.21.15.60:0/1079200668 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:45.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.822+0000 7f2fde59c700 1 --2- 172.21.15.60:0/1079200668 >> v2:172.21.15.105:3300/0 conn(0x7f2fe00cfd50 0x7f2fe00d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570390329 0 0) 0x7f2fe00d1870 con 0x7f2fe00b8200 2023-07-13T22:24:45.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc80035b0 con 0x7f2fe00b8200 2023-07-13T22:24:45.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3228396843 0 0) 0x7f2fc80035b0 con 0x7f2fe00b8200 2023-07-13T22:24:45.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 >> v2:172.21.15.105:3300/0 conn(0x7f2fe00cfd50 msgr2=0x7f2fe00d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:45.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 --2- 172.21.15.60:0/1079200668 >> v2:172.21.15.105:3300/0 conn(0x7f2fe00cfd50 0x7f2fe00d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 >> v1:172.21.15.105:6789/0 conn(0x7f2fe00b1ac0 legacy=0x7f2fe00d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00d1590 con 0x7f2fe00b8200 2023-07-13T22:24:45.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f2fd0001670 con 0x7f2fe00b8200 2023-07-13T22:24:45.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 >> v1:172.21.15.60:6789/0 conn(0x7f2fe00b8200 legacy=0x7f2fe00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8003a50 0x7f2fc8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fc8004850 0x7f2fc8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc8005200 0x7f2fc800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00d12b0 con 0x7f2fc8005200 2023-07-13T22:24:45.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc800c0e0 con 0x7f2fc8003a50 2023-07-13T22:24:45.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc800c2a0 con 0x7f2fc8004850 2023-07-13T22:24:45.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.823+0000 7f2fdf59e700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc8005200 0x7f2fc800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fde59c700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fc8004850 0x7f2fc8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fded9d700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8003a50 0x7f2fc8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fdf59e700 1 -- 172.21.15.60:0/1079200668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fc8004850 msgr2=0x7f2fc8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:45.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fdf59e700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fc8004850 0x7f2fc8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fdf59e700 1 -- 172.21.15.60:0/1079200668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8003a50 msgr2=0x7f2fc8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:45.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fdf59e700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8003a50 0x7f2fc8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fdf59e700 1 -- 172.21.15.60:0/1079200668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc80035b0 con 0x7f2fc8005200 2023-07-13T22:24:45.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fded9d700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8003a50 0x7f2fc8003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:45.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fdf59e700 1 -- 172.21.15.60:0/1079200668 reap_dead start 2023-07-13T22:24:45.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fdf59e700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc8005200 0x7f2fc800ba10 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c160 tx=0x7f2fc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a41a1962cd0103 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:45.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8001db0 con 0x7f2fc8005200 2023-07-13T22:24:45.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd800cca0 con 0x7f2fc8005200 2023-07-13T22:24:45.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.824+0000 7f2fddd9b700 1 -- 172.21.15.60:0/1079200668 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8003c20 con 0x7f2fc8005200 2023-07-13T22:24:45.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.825+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1079200668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc8005200 msgr2=0x7f2fc800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.825+0000 7f2fe51ff700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc8005200 0x7f2fc800ba10 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c160 tx=0x7f2fc8005880 comp rx=0 tx=0).stop 2023-07-13T22:24:45.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.825+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1079200668 shutdown_connections 2023-07-13T22:24:45.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.825+0000 7f2fe51ff700 1 --2- 172.21.15.60:0/1079200668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc8005200 0x7f2fc800ba10 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.825+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1079200668 >> 172.21.15.60:0/1079200668 conn(0x7f2fe006b2f0 msgr2=0x7f2fe006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:45.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.825+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1079200668 shutdown_connections 2023-07-13T22:24:45.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.825+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1079200668 wait complete. 2023-07-13T22:24:45.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.827+0000 7f2fe51ff700 1 Processor -- start 2023-07-13T22:24:45.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.827+0000 7f2fe51ff700 1 -- start start 2023-07-13T22:24:45.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fe51ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8005200 0x7f2fe00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fe51ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fe01f1a80 0x7f2fe01f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fe51ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fe015f190 0x7f2fe015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fe51ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe01f2af0 con 0x7f2fe01f1a80 2023-07-13T22:24:45.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fe51ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b6350 con 0x7f2fc8005200 2023-07-13T22:24:45.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fe51ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00ccd50 con 0x7f2fe015f190 2023-07-13T22:24:45.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fde59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fe01f1a80 0x7f2fe01f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fde59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fe01f1a80 0x7f2fe01f1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47528/0 (socket says 172.21.15.60:47528) 2023-07-13T22:24:45.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fde59c700 1 -- 172.21.15.60:0/1935273327 learned_addr learned my addr 172.21.15.60:0/1935273327 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:45.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fded9d700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8005200 0x7f2fe00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.828+0000 7f2fdf59e700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fe015f190 0x7f2fe015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fde59c700 1 -- 172.21.15.60:0/1935273327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fe015f190 msgr2=0x7f2fe015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fde59c700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fe015f190 0x7f2fe015f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fde59c700 1 -- 172.21.15.60:0/1935273327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8005200 msgr2=0x7f2fe00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fde59c700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8005200 0x7f2fe00cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fde59c700 1 -- 172.21.15.60:0/1935273327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe01f2530 con 0x7f2fe01f1a80 2023-07-13T22:24:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fde59c700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fe01f1a80 0x7f2fe01f1ef0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4011940 tx=0x7f2fd4011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fc77fe700 1 -- 172.21.15.60:0/1935273327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4004240 con 0x7f2fe01f1a80 2023-07-13T22:24:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fc77fe700 1 -- 172.21.15.60:0/1935273327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd40043a0 con 0x7f2fe01f1a80 2023-07-13T22:24:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fc77fe700 1 -- 172.21.15.60:0/1935273327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400f950 con 0x7f2fe01f1a80 2023-07-13T22:24:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe01f2810 con 0x7f2fe01f1a80 2023-07-13T22:24:45.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.829+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe015fb70 con 0x7f2fe01f1a80 2023-07-13T22:24:45.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.830+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe00b5e40 con 0x7f2fe01f1a80 2023-07-13T22:24:45.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.831+0000 7f2fc77fe700 1 -- 172.21.15.60:0/1935273327 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2fd400fab0 con 0x7f2fe01f1a80 2023-07-13T22:24:45.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.832+0000 7f2fc77fe700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fcc06ace0 0x7f2fcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:45.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.832+0000 7f2fc77fe700 1 -- 172.21.15.60:0/1935273327 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2fd4089b20 con 0x7f2fe01f1a80 2023-07-13T22:24:45.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.833+0000 7f2fded9d700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fcc06ace0 0x7f2fcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:45.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.833+0000 7f2fded9d700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fcc06ace0 0x7f2fcc06d190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00cac10 tx=0x7f2fd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:45.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:45.836+0000 7f2fc77fe700 1 -- 172.21.15.60:0/1935273327 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2fe00b5e40 con 0x7f2fe01f1a80 2023-07-13T22:24:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.045+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2fe00b5e40 con 0x7f2fe01f1a80 2023-07-13T22:24:46.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.045+0000 7f2fc77fe700 1 -- 172.21.15.60:0/1935273327 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2fd4023020 con 0x7f2fe01f1a80 2023-07-13T22:24:46.028 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:24:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fcc06ace0 msgr2=0x7f2fcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fcc06ace0 0x7f2fcc06d190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00cac10 tx=0x7f2fd0009040 comp rx=0 tx=0).stop 2023-07-13T22:24:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fe01f1a80 msgr2=0x7f2fe01f1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fe01f1a80 0x7f2fe01f1ef0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4011940 tx=0x7f2fd4011d00 comp rx=0 tx=0).stop 2023-07-13T22:24:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 shutdown_connections 2023-07-13T22:24:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fcc06ace0 0x7f2fcc06d190 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fe015f190 0x7f2fe015f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc8005200 0x7f2fe00cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 --2- 172.21.15.60:0/1935273327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fe01f1a80 0x7f2fe01f1ef0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 >> 172.21.15.60:0/1935273327 conn(0x7f2fe006b2f0 msgr2=0x7f2fe006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:46.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 shutdown_connections 2023-07-13T22:24:46.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.048+0000 7f2fe51ff700 1 -- 172.21.15.60:0/1935273327 wait complete. 2023-07-13T22:24:46.049 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:24:46.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.272+0000 7f97825ff700 1 Processor -- start 2023-07-13T22:24:46.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.273+0000 7f97825ff700 1 -- start start 2023-07-13T22:24:46.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f97825ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f977c0b8200 0x7f977c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f97825ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0cede0 0x7f977c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f97825ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c0d1870 con 0x7f977c0b1ac0 2023-07-13T22:24:46.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f97825ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c0cf370 con 0x7f977c0b8200 2023-07-13T22:24:46.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f97825ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0d12b0 con 0x7f977c0cede0 2023-07-13T22:24:46.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f977bfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f977c0b8200 0x7f977c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f9780b9c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0cede0 0x7f977c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f9780b9c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0cede0 0x7f977c0cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:46770/0 (socket says 172.21.15.60:46770) 2023-07-13T22:24:46.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f9780b9c700 1 -- 172.21.15.60:0/1826712088 learned_addr learned my addr 172.21.15.60:0/1826712088 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:46.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.274+0000 7f977bfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f977c0b8200 0x7f977c0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:43840/0 (socket says 172.21.15.60:43840) 2023-07-13T22:24:46.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.275+0000 7f977bfff700 1 -- 172.21.15.60:0/1826712088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0cede0 msgr2=0x7f977c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:46.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.275+0000 7f977bfff700 1 --2- 172.21.15.60:0/1826712088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0cede0 0x7f977c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.275+0000 7f977bfff700 1 -- 172.21.15.60:0/1826712088 >> v1:172.21.15.105:6789/0 conn(0x7f977c0b1ac0 legacy=0x7f977c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.275+0000 7f977bfff700 1 -- 172.21.15.60:0/1826712088 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c0d1590 con 0x7f977c0b8200 2023-07-13T22:24:46.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.275+0000 7f9780b9c700 1 --2- 172.21.15.60:0/1826712088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0cede0 0x7f977c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:46.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.275+0000 7f977affd700 1 -- 172.21.15.60:0/1826712088 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130054230 0 0) 0x7f977c0d1870 con 0x7f977c0b1ac0 2023-07-13T22:24:46.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.275+0000 7f977bfff700 1 --2- 172.21.15.60:0/1826712088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f977c0b8200 0x7f977c0cf640 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f976c009840 tx=0x7f976c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a77c406aff004e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:46.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.275+0000 7f977affd700 1 -- 172.21.15.60:0/1826712088 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c011830 con 0x7f977c0b8200 2023-07-13T22:24:46.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f977affd700 1 -- 172.21.15.60:0/1826712088 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c011990 con 0x7f977c0b8200 2023-07-13T22:24:46.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f977affd700 1 -- 172.21.15.60:0/1826712088 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c011d20 con 0x7f977c0b8200 2023-07-13T22:24:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f97825ff700 1 -- 172.21.15.60:0/1826712088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f977c0b8200 msgr2=0x7f977c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f97825ff700 1 --2- 172.21.15.60:0/1826712088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f977c0b8200 0x7f977c0cf640 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f976c009840 tx=0x7f976c009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f97825ff700 1 -- 172.21.15.60:0/1826712088 shutdown_connections 2023-07-13T22:24:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f97825ff700 1 --2- 172.21.15.60:0/1826712088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0cede0 0x7f977c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f97825ff700 1 --2- 172.21.15.60:0/1826712088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f977c0b8200 0x7f977c0cf640 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f97825ff700 1 -- 172.21.15.60:0/1826712088 >> 172.21.15.60:0/1826712088 conn(0x7f977c06b2f0 msgr2=0x7f977c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f97825ff700 1 -- 172.21.15.60:0/1826712088 shutdown_connections 2023-07-13T22:24:46.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.276+0000 7f97825ff700 1 -- 172.21.15.60:0/1826712088 wait complete. 2023-07-13T22:24:46.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.278+0000 7f97825ff700 1 Processor -- start 2023-07-13T22:24:46.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.278+0000 7f97825ff700 1 -- start start 2023-07-13T22:24:46.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.279+0000 7f97825ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f977c0b1ac0 0x7f977c0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.279+0000 7f97825ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0b8200 0x7f977c0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.279+0000 7f97825ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f977c0cede0 0x7f977c149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.279+0000 7f97825ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c14bba0 con 0x7f977c0b1ac0 2023-07-13T22:24:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.279+0000 7f97825ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c14af90 con 0x7f977c0cede0 2023-07-13T22:24:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.279+0000 7f97825ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c14b270 con 0x7f977c0b8200 2023-07-13T22:24:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.279+0000 7f977bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f977c0b1ac0 0x7f977c0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.279+0000 7f977bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f977c0b1ac0 0x7f977c0c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47542/0 (socket says 172.21.15.60:47542) 2023-07-13T22:24:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.279+0000 7f977bfff700 1 -- 172.21.15.60:0/3315684536 learned_addr learned my addr 172.21.15.60:0/3315684536 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:46.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f977b7fe700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0b8200 0x7f977c0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f9780b9c700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f977c0cede0 0x7f977c149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f977bfff700 1 -- 172.21.15.60:0/3315684536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0b8200 msgr2=0x7f977c0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f977bfff700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0b8200 0x7f977c0c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f977bfff700 1 -- 172.21.15.60:0/3315684536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f977c0cede0 msgr2=0x7f977c149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f977bfff700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f977c0cede0 0x7f977c149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f977bfff700 1 -- 172.21.15.60:0/3315684536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774009b30 con 0x7f977c0b1ac0 2023-07-13T22:24:46.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f977bfff700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f977c0b1ac0 0x7f977c0c8770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f976c009fd0 tx=0x7f976c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:46.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f9778ff9700 1 -- 172.21.15.60:0/3315684536 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c0036a0 con 0x7f977c0b1ac0 2023-07-13T22:24:46.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f9778ff9700 1 -- 172.21.15.60:0/3315684536 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c01f070 con 0x7f977c0b1ac0 2023-07-13T22:24:46.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976c0094a0 con 0x7f977c0b1ac0 2023-07-13T22:24:46.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.280+0000 7f9778ff9700 1 -- 172.21.15.60:0/3315684536 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c003a60 con 0x7f977c0b1ac0 2023-07-13T22:24:46.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.281+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c14b830 con 0x7f977c0b1ac0 2023-07-13T22:24:46.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.282+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977c04e210 con 0x7f977c0b1ac0 2023-07-13T22:24:46.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.283+0000 7f9778ff9700 1 -- 172.21.15.60:0/3315684536 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f976c003bc0 con 0x7f977c0b1ac0 2023-07-13T22:24:46.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.283+0000 7f9778ff9700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f976806ace0 0x7f976806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.284+0000 7f9778ff9700 1 -- 172.21.15.60:0/3315684536 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f976c021070 con 0x7f977c0b1ac0 2023-07-13T22:24:46.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.284+0000 7f977b7fe700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f976806ace0 0x7f976806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.284+0000 7f977b7fe700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f976806ace0 0x7f976806d190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f977c0c62f0 tx=0x7f9770009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:46.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.288+0000 7f9778ff9700 1 -- 172.21.15.60:0/3315684536 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f976c055d30 con 0x7f977c0b1ac0 2023-07-13T22:24:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.491+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f977c0b4f70 con 0x7f977c0b1ac0 2023-07-13T22:24:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.492+0000 7f9778ff9700 1 -- 172.21.15.60:0/3315684536 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7f976c059350 con 0x7f977c0b1ac0 2023-07-13T22:24:46.474 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:24:46.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f976806ace0 msgr2=0x7f976806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f976806ace0 0x7f976806d190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f977c0c62f0 tx=0x7f9770009040 comp rx=0 tx=0).stop 2023-07-13T22:24:46.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f977c0b1ac0 msgr2=0x7f977c0c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f977c0b1ac0 0x7f977c0c8770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f976c009fd0 tx=0x7f976c004750 comp rx=0 tx=0).stop 2023-07-13T22:24:46.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 shutdown_connections 2023-07-13T22:24:46.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f976806ace0 0x7f976806d190 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f977c0b1ac0 0x7f977c0c8770 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f977c0cede0 0x7f977c149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 --2- 172.21.15.60:0/3315684536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f977c0b8200 0x7f977c0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 >> 172.21.15.60:0/3315684536 conn(0x7f977c06b2f0 msgr2=0x7f977c0d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.494+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 shutdown_connections 2023-07-13T22:24:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.495+0000 7f97825ff700 1 -- 172.21.15.60:0/3315684536 wait complete. 2023-07-13T22:24:46.495 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:24:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.718+0000 7f24589ff700 1 Processor -- start 2023-07-13T22:24:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.718+0000 7f24589ff700 1 -- start start 2023-07-13T22:24:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.719+0000 7f24589ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f24540b8200 0x7f24540cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.719+0000 7f24589ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24540b1ac0 0x7f24540cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.719+0000 7f24589ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24540d1870 con 0x7f24540cede0 2023-07-13T22:24:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.719+0000 7f24589ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24540cf370 con 0x7f24540b1ac0 2023-07-13T22:24:46.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.719+0000 7f24589ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24540d12b0 con 0x7f24540b8200 2023-07-13T22:24:46.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f2451d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24540b1ac0 0x7f24540cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f2451d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24540b1ac0 0x7f24540cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47554/0 (socket says 172.21.15.60:47554) 2023-07-13T22:24:46.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f2451d9b700 1 -- 172.21.15.60:0/337099550 learned_addr learned my addr 172.21.15.60:0/337099550 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:46.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f245259c700 1 --2- 172.21.15.60:0/337099550 >> v2:172.21.15.105:3300/0 conn(0x7f24540b8200 0x7f24540cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f2451d9b700 1 -- 172.21.15.60:0/337099550 >> v2:172.21.15.105:3300/0 conn(0x7f24540b8200 msgr2=0x7f24540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f2451d9b700 1 --2- 172.21.15.60:0/337099550 >> v2:172.21.15.105:3300/0 conn(0x7f24540b8200 0x7f24540cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f2451d9b700 1 -- 172.21.15.60:0/337099550 >> v1:172.21.15.60:6789/0 conn(0x7f24540cede0 legacy=0x7f245415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f2451d9b700 1 -- 172.21.15.60:0/337099550 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24540d1590 con 0x7f24540b1ac0 2023-07-13T22:24:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f245159a700 1 -- 172.21.15.60:0/337099550 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052550357 0 0) 0x7f24540d1870 con 0x7f24540cede0 2023-07-13T22:24:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.720+0000 7f2451d9b700 1 --2- 172.21.15.60:0/337099550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540b1ac0 0x7f24540cfb80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f24540b2660 tx=0x7f2448009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8eed1028a00734a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.721+0000 7f245159a700 1 -- 172.21.15.60:0/337099550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244800b870 con 0x7f24540b1ac0 2023-07-13T22:24:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.721+0000 7f245159a700 1 -- 172.21.15.60:0/337099550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244800b9d0 con 0x7f24540b1ac0 2023-07-13T22:24:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.721+0000 7f245159a700 1 -- 172.21.15.60:0/337099550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244800bd60 con 0x7f24540b1ac0 2023-07-13T22:24:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.721+0000 7f24589ff700 1 -- 172.21.15.60:0/337099550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540b1ac0 msgr2=0x7f24540cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.721+0000 7f24589ff700 1 --2- 172.21.15.60:0/337099550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540b1ac0 0x7f24540cfb80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f24540b2660 tx=0x7f2448009bd0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.721+0000 7f24589ff700 1 -- 172.21.15.60:0/337099550 shutdown_connections 2023-07-13T22:24:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.721+0000 7f24589ff700 1 --2- 172.21.15.60:0/337099550 >> v2:172.21.15.105:3300/0 conn(0x7f24540b8200 0x7f24540cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.721+0000 7f24589ff700 1 --2- 172.21.15.60:0/337099550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540b1ac0 0x7f24540cfb80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.721+0000 7f24589ff700 1 -- 172.21.15.60:0/337099550 >> 172.21.15.60:0/337099550 conn(0x7f245406b2f0 msgr2=0x7f245406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:46.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.722+0000 7f24589ff700 1 -- 172.21.15.60:0/337099550 shutdown_connections 2023-07-13T22:24:46.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.722+0000 7f24589ff700 1 -- 172.21.15.60:0/337099550 wait complete. 2023-07-13T22:24:46.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.723+0000 7f24589ff700 1 Processor -- start 2023-07-13T22:24:46.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.723+0000 7f24589ff700 1 -- start start 2023-07-13T22:24:46.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.724+0000 7f24589ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24540b1ac0 0x7f2454158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.724+0000 7f24589ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540b8200 0x7f2454159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.724+0000 7f24589ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540cede0 0x7f24540cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.724+0000 7f24589ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245415b550 con 0x7f24540cede0 2023-07-13T22:24:46.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f24589ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245415d5c0 con 0x7f24540b8200 2023-07-13T22:24:46.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f24589ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2454159680 con 0x7f24540b1ac0 2023-07-13T22:24:46.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2452d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540cede0 0x7f24540cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2452d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540cede0 0x7f24540cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47566/0 (socket says 172.21.15.60:47566) 2023-07-13T22:24:46.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2452d9d700 1 -- 172.21.15.60:0/3389459494 learned_addr learned my addr 172.21.15.60:0/3389459494 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:46.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f245259c700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24540b1ac0 0x7f2454158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2451d9b700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540b8200 0x7f2454159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2452d9d700 1 -- 172.21.15.60:0/3389459494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24540b1ac0 msgr2=0x7f2454158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2452d9d700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24540b1ac0 0x7f2454158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2452d9d700 1 -- 172.21.15.60:0/3389459494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540b8200 msgr2=0x7f2454159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2452d9d700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540b8200 0x7f2454159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2452d9d700 1 -- 172.21.15.60:0/3389459494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448009550 con 0x7f24540cede0 2023-07-13T22:24:46.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2451d9b700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540b8200 0x7f2454159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:46.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.725+0000 7f2452d9d700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540cede0 0x7f24540cbc70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f244c001560 tx=0x7f244c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:46.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.726+0000 7f2442ffd700 1 -- 172.21.15.60:0/3389459494 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c003e30 con 0x7f24540cede0 2023-07-13T22:24:46.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.726+0000 7f2442ffd700 1 -- 172.21.15.60:0/3389459494 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c005070 con 0x7f24540cede0 2023-07-13T22:24:46.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.726+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24540cc6c0 con 0x7f24540cede0 2023-07-13T22:24:46.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.726+0000 7f2442ffd700 1 -- 172.21.15.60:0/3389459494 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c005420 con 0x7f24540cede0 2023-07-13T22:24:46.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.726+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245415cf70 con 0x7f24540cede0 2023-07-13T22:24:46.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.727+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24540b53c0 con 0x7f24540cede0 2023-07-13T22:24:46.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.728+0000 7f2442ffd700 1 -- 172.21.15.60:0/3389459494 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f244c005580 con 0x7f24540cede0 2023-07-13T22:24:46.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.729+0000 7f2442ffd700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f243406ace0 0x7f243406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.729+0000 7f2442ffd700 1 -- 172.21.15.60:0/3389459494 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f244c088d20 con 0x7f24540cede0 2023-07-13T22:24:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.729+0000 7f245259c700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f243406ace0 0x7f243406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.730+0000 7f245259c700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f243406ace0 0x7f243406d190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2444009930 tx=0x7f2444008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:46.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.733+0000 7f2442ffd700 1 -- 172.21.15.60:0/3389459494 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f244c08f080 con 0x7f24540cede0 2023-07-13T22:24:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.936+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f245404e210 con 0x7f24540cede0 2023-07-13T22:24:46.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.936+0000 7f2442ffd700 1 -- 172.21.15.60:0/3389459494 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7f244c0170a0 con 0x7f24540cede0 2023-07-13T22:24:46.919 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:24:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f243406ace0 msgr2=0x7f243406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f243406ace0 0x7f243406d190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2444009930 tx=0x7f2444008040 comp rx=0 tx=0).stop 2023-07-13T22:24:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540cede0 msgr2=0x7f24540cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540cede0 0x7f24540cbc70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f244c001560 tx=0x7f244c00cc00 comp rx=0 tx=0).stop 2023-07-13T22:24:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 shutdown_connections 2023-07-13T22:24:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f243406ace0 0x7f243406d190 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24540cede0 0x7f24540cbc70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24540b1ac0 0x7f2454158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 --2- 172.21.15.60:0/3389459494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24540b8200 0x7f2454159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 >> 172.21.15.60:0/3389459494 conn(0x7f245406b2f0 msgr2=0x7f24540b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 shutdown_connections 2023-07-13T22:24:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:46.939+0000 7f24589ff700 1 -- 172.21.15.60:0/3389459494 wait complete. 2023-07-13T22:24:46.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.160+0000 7fa6621ff700 1 Processor -- start 2023-07-13T22:24:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.160+0000 7fa6621ff700 1 -- start start 2023-07-13T22:24:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.161+0000 7fa6621ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0b3050 0x7fa65c06afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.161+0000 7fa6621ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c06b8e0 con 0x7fa65c0b24b0 2023-07-13T22:24:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.161+0000 7fa6621ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c06bbc0 con 0x7fa65c15c6e0 2023-07-13T22:24:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.161+0000 7fa6621ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c15cab0 con 0x7fa65c0b3050 2023-07-13T22:24:47.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa65c0b24b0 0x7fa65c0cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59864/0 (socket says 172.21.15.60:59864) 2023-07-13T22:24:47.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65bfff700 1 -- 172.21.15.60:0/1162754029 learned_addr learned my addr 172.21.15.60:0/1162754029 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:47.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65affd700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0b3050 0x7fa65c06afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166892140 0 0) 0x7fa65c06b8e0 con 0x7fa65c0b24b0 2023-07-13T22:24:47.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6440035b0 con 0x7fa65c0b24b0 2023-07-13T22:24:47.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 608686618 0 0) 0x7fa6440035b0 con 0x7fa65c0b24b0 2023-07-13T22:24:47.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0b3050 msgr2=0x7fa65c06afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:47.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65a7fc700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0b3050 0x7fa65c06afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 >> v1:172.21.15.105:6789/0 conn(0x7fa65c15c6e0 legacy=0x7fa65c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c06bea0 con 0x7fa65c0b24b0 2023-07-13T22:24:47.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fa6540019f0 con 0x7fa65c0b24b0 2023-07-13T22:24:47.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.162+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 >> v1:172.21.15.60:6789/0 conn(0x7fa65c0b24b0 legacy=0x7fa65c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65a7fc700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa644003a30 0x7fa644003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65a7fc700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0b3050 0x7fa644004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65a7fc700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa644005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c15cab0 con 0x7fa644004d90 2023-07-13T22:24:47.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64400baf0 con 0x7fa644003a30 2023-07-13T22:24:47.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64400bcb0 con 0x7fa65c0b3050 2023-07-13T22:24:47.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65bfff700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa644005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65b7fe700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa644003a30 0x7fa644003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65affd700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0b3050 0x7fa644004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65bfff700 1 -- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0b3050 msgr2=0x7fa644004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65bfff700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0b3050 0x7fa644004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65bfff700 1 -- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa644003a30 msgr2=0x7fa644003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65bfff700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa644003a30 0x7fa644003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65bfff700 1 -- 172.21.15.60:0/1162754029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440035b0 con 0x7fa644004d90 2023-07-13T22:24:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.163+0000 7fa65b7fe700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa644003a30 0x7fa644003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:47.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa65bfff700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa644005200 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa65400a760 tx=0x7fa644005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f7c0add73a273d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400a250 con 0x7fa644004d90 2023-07-13T22:24:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6540072d0 con 0x7fa644004d90 2023-07-13T22:24:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa65a7fc700 1 -- 172.21.15.60:0/1162754029 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654007680 con 0x7fa644004d90 2023-07-13T22:24:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa6621ff700 1 -- 172.21.15.60:0/1162754029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 msgr2=0x7fa644005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa6621ff700 1 --2- 172.21.15.60:0/1162754029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa644005200 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa65400a760 tx=0x7fa644005880 comp rx=0 tx=0).stop 2023-07-13T22:24:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa65bfff700 1 -- 172.21.15.60:0/1162754029 reap_dead start 2023-07-13T22:24:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa6621ff700 1 -- 172.21.15.60:0/1162754029 shutdown_connections 2023-07-13T22:24:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa6621ff700 1 -- 172.21.15.60:0/1162754029 >> 172.21.15.60:0/1162754029 conn(0x7fa65c06e680 msgr2=0x7fa65c06ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:47.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa6621ff700 1 -- 172.21.15.60:0/1162754029 shutdown_connections 2023-07-13T22:24:47.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.164+0000 7fa6621ff700 1 -- 172.21.15.60:0/1162754029 wait complete. 2023-07-13T22:24:47.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.166+0000 7fa6621ff700 1 Processor -- start 2023-07-13T22:24:47.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.166+0000 7fa6621ff700 1 -- start start 2023-07-13T22:24:47.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.167+0000 7fa6621ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa65c150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.167+0000 7fa6621ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa65c0c9cd0 0x7fa65c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.167+0000 7fa6621ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0c6a80 0x7fa65c0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.167+0000 7fa6621ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c0c47c0 con 0x7fa644004d90 2023-07-13T22:24:47.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.167+0000 7fa6621ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c0c7430 con 0x7fa65c0c9cd0 2023-07-13T22:24:47.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.167+0000 7fa6621ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c1513f0 con 0x7fa65c0c6a80 2023-07-13T22:24:47.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.167+0000 7fa65b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa65c150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.167+0000 7fa65b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa65c150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47574/0 (socket says 172.21.15.60:47574) 2023-07-13T22:24:47.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa65b7fe700 1 -- 172.21.15.60:0/2764739305 learned_addr learned my addr 172.21.15.60:0/2764739305 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:47.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa65b7fe700 1 -- 172.21.15.60:0/2764739305 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0c6a80 msgr2=0x7fa65c0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa65b7fe700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0c6a80 0x7fa65c0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa65b7fe700 1 -- 172.21.15.60:0/2764739305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa65c0c9cd0 msgr2=0x7fa65c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa65affd700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa65c0c9cd0 0x7fa65c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa65b7fe700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa65c0c9cd0 0x7fa65c15dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa65b7fe700 1 -- 172.21.15.60:0/2764739305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c15e4f0 con 0x7fa644004d90 2023-07-13T22:24:47.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa65affd700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa65c0c9cd0 0x7fa65c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:47.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa65b7fe700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa65c150eb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00e760 tx=0x7fa64c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:47.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa643fff700 1 -- 172.21.15.60:0/2764739305 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c0052e0 con 0x7fa644004d90 2023-07-13T22:24:47.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa643fff700 1 -- 172.21.15.60:0/2764739305 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c005440 con 0x7fa644004d90 2023-07-13T22:24:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa643fff700 1 -- 172.21.15.60:0/2764739305 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c003880 con 0x7fa644004d90 2023-07-13T22:24:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.168+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65c0ca290 con 0x7fa644004d90 2023-07-13T22:24:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.169+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c0ca850 con 0x7fa644004d90 2023-07-13T22:24:47.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.170+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65c070a00 con 0x7fa644004d90 2023-07-13T22:24:47.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.171+0000 7fa643fff700 1 -- 172.21.15.60:0/2764739305 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa64c0039e0 con 0x7fa644004d90 2023-07-13T22:24:47.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.171+0000 7fa643fff700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa64806ad10 0x7fa64806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.172+0000 7fa643fff700 1 -- 172.21.15.60:0/2764739305 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa64c088a10 con 0x7fa644004d90 2023-07-13T22:24:47.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.172+0000 7fa65affd700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa64806ad10 0x7fa64806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.172+0000 7fa65affd700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa64806ad10 0x7fa64806d1c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa65c15f960 tx=0x7fa650008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.176+0000 7fa643fff700 1 -- 172.21.15.60:0/2764739305 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa64c0539a0 con 0x7fa644004d90 2023-07-13T22:24:47.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.426+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa65c065fa0 con 0x7fa644004d90 2023-07-13T22:24:47.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.426+0000 7fa643fff700 1 -- 172.21.15.60:0/2764739305 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fa64c019020 con 0x7fa644004d90 2023-07-13T22:24:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:47.409 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/711733521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":711733521},{"type":"v1","addr":"172.21.15.105:6834","nonce":711733521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2135065566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":2135065566},{"type":"v1","addr":"172.21.15.105:6835","nonce":2135065566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6052,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4236704538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4236704538},{"type":"v1","addr":"172.21.15.60:6836","nonce":4236704538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:24:20.139470+0000","modified":"2023-07-13T22:24:23.665486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6058":{"gid":6058,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3036622731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3036622731},{"type":"v1","addr":"172.21.15.60:6837","nonce":3036622731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-13T22:24:47.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa64806ad10 msgr2=0x7fa64806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa64806ad10 0x7fa64806d1c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa65c15f960 tx=0x7fa650008040 comp rx=0 tx=0).stop 2023-07-13T22:24:47.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 msgr2=0x7fa65c150eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa65c150eb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00e760 tx=0x7fa64c00eb20 comp rx=0 tx=0).stop 2023-07-13T22:24:47.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 shutdown_connections 2023-07-13T22:24:47.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa64806ad10 0x7fa64806d1c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644004d90 0x7fa65c150eb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa65c0c6a80 0x7fa65c0c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 --2- 172.21.15.60:0/2764739305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa65c0c9cd0 0x7fa65c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 >> 172.21.15.60:0/2764739305 conn(0x7fa65c06e680 msgr2=0x7fa65c0b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 shutdown_connections 2023-07-13T22:24:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.429+0000 7fa6621ff700 1 -- 172.21.15.60:0/2764739305 wait complete. 2023-07-13T22:24:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 33 2023-07-13T22:24:47.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:47.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.652+0000 7f5102dff700 1 Processor -- start 2023-07-13T22:24:47.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.652+0000 7f5102dff700 1 -- start start 2023-07-13T22:24:47.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f5102dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50fc159170 0x7f50fc0c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f5102dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc0c5040 0x7f50fc070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f5102dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50fc15d970 con 0x7f50fc0707b0 2023-07-13T22:24:47.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f5102dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50fc15cd60 con 0x7f50fc159170 2023-07-13T22:24:47.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f5102dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc15d040 con 0x7f50fc0c5040 2023-07-13T22:24:47.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f510139c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50fc0707b0 0x7f50fc0cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59878/0 (socket says 172.21.15.60:59878) 2023-07-13T22:24:47.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f510139c700 1 -- 172.21.15.60:0/497025824 learned_addr learned my addr 172.21.15.60:0/497025824 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f50fbfff700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc0c5040 0x7f50fc070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f5100b9b700 1 --2- 172.21.15.60:0/497025824 >> v2:172.21.15.105:3300/0 conn(0x7f50fc159170 0x7f50fc0c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607549288 0 0) 0x7f50fc15d970 con 0x7f50fc0707b0 2023-07-13T22:24:47.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.654+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50ec0035b0 con 0x7f50fc0707b0 2023-07-13T22:24:47.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 630507857 0 0) 0x7f50ec0035b0 con 0x7f50fc0707b0 2023-07-13T22:24:47.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc0c5040 msgr2=0x7f50fc070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:47.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc0c5040 0x7f50fc070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 >> v2:172.21.15.105:3300/0 conn(0x7f50fc159170 msgr2=0x7f50fc0c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:47.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 --2- 172.21.15.60:0/497025824 >> v2:172.21.15.105:3300/0 conn(0x7f50fc159170 0x7f50fc0c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc070c10 con 0x7f50fc0707b0 2023-07-13T22:24:47.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f50f40019f0 con 0x7f50fc0707b0 2023-07-13T22:24:47.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 >> v1:172.21.15.60:6789/0 conn(0x7f50fc0707b0 legacy=0x7f50fc0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50ec003a00 0x7f50ec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc0c5040 0x7f50ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50ec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc15cd60 con 0x7f50ec004d90 2023-07-13T22:24:47.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc15d040 con 0x7f50ec003a00 2023-07-13T22:24:47.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec00dc20 con 0x7f50fc0c5040 2023-07-13T22:24:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f510139c700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50ec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f5100b9b700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50ec003a00 0x7f50ec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.655+0000 7f50fbfff700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc0c5040 0x7f50ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.656+0000 7f510139c700 1 -- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc0c5040 msgr2=0x7f50ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.656+0000 7f510139c700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc0c5040 0x7f50ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.656+0000 7f510139c700 1 -- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50ec003a00 msgr2=0x7f50ec003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.656+0000 7f510139c700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50ec003a00 0x7f50ec003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.656+0000 7f510139c700 1 -- 172.21.15.60:0/497025824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec0035b0 con 0x7f50ec004d90 2023-07-13T22:24:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.656+0000 7f510139c700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50ec005200 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f50f400a790 tx=0x7f50ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910c7ab450683601 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.656+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f400a250 con 0x7f50ec004d90 2023-07-13T22:24:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.656+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f40072d0 con 0x7f50ec004d90 2023-07-13T22:24:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.656+0000 7f50fb7fe700 1 -- 172.21.15.60:0/497025824 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4007680 con 0x7f50ec004d90 2023-07-13T22:24:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.657+0000 7f5102dff700 1 -- 172.21.15.60:0/497025824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 msgr2=0x7f50ec005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.657+0000 7f5102dff700 1 --2- 172.21.15.60:0/497025824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50ec005200 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f50f400a790 tx=0x7f50ec005880 comp rx=0 tx=0).stop 2023-07-13T22:24:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.657+0000 7f510139c700 1 -- 172.21.15.60:0/497025824 reap_dead start 2023-07-13T22:24:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.657+0000 7f5102dff700 1 -- 172.21.15.60:0/497025824 shutdown_connections 2023-07-13T22:24:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.657+0000 7f5102dff700 1 -- 172.21.15.60:0/497025824 >> 172.21.15.60:0/497025824 conn(0x7f50fc0cf410 msgr2=0x7f50fc0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.657+0000 7f5102dff700 1 -- 172.21.15.60:0/497025824 shutdown_connections 2023-07-13T22:24:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.657+0000 7f5102dff700 1 -- 172.21.15.60:0/497025824 wait complete. 2023-07-13T22:24:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.658+0000 7f5102dff700 1 Processor -- start 2023-07-13T22:24:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.659+0000 7f5102dff700 1 -- start start 2023-07-13T22:24:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5102dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50fc1f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5102dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc15b5c0 0x7f50fc15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5102dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50fc1f5660 0x7f50fc1f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5102dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1f54a0 con 0x7f50ec004d90 2023-07-13T22:24:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5102dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1f6010 con 0x7f50fc1f5660 2023-07-13T22:24:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5102dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1f64e0 con 0x7f50fc15b5c0 2023-07-13T22:24:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5100b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50fc1f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5100b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50fc1f4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47588/0 (socket says 172.21.15.60:47588) 2023-07-13T22:24:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5100b9b700 1 -- 172.21.15.60:0/2382531236 learned_addr learned my addr 172.21.15.60:0/2382531236 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5100b9b700 1 -- 172.21.15.60:0/2382531236 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc15b5c0 msgr2=0x7f50fc15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5100b9b700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc15b5c0 0x7f50fc15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5100b9b700 1 -- 172.21.15.60:0/2382531236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50fc1f5660 msgr2=0x7f50fc1f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f510139c700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50fc1f5660 0x7f50fc1f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5100b9b700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50fc1f5660 0x7f50fc1f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f5100b9b700 1 -- 172.21.15.60:0/2382531236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc15a300 con 0x7f50ec004d90 2023-07-13T22:24:47.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.660+0000 7f510139c700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50fc1f5660 0x7f50fc1f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:47.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.661+0000 7f5100b9b700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50fc1f4f60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50f0011ad0 tx=0x7f50f0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:47.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.661+0000 7f50f97fa700 1 -- 172.21.15.60:0/2382531236 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000f9a0 con 0x7f50ec004d90 2023-07-13T22:24:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.661+0000 7f50f97fa700 1 -- 172.21.15.60:0/2382531236 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f000fb20 con 0x7f50ec004d90 2023-07-13T22:24:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.661+0000 7f50f97fa700 1 -- 172.21.15.60:0/2382531236 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000fed0 con 0x7f50ec004d90 2023-07-13T22:24:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.661+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc15a5e0 con 0x7f50ec004d90 2023-07-13T22:24:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.661+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc15aba0 con 0x7f50ec004d90 2023-07-13T22:24:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.662+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc0ad860 con 0x7f50ec004d90 2023-07-13T22:24:47.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.663+0000 7f50f97fa700 1 -- 172.21.15.60:0/2382531236 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50f0014440 con 0x7f50ec004d90 2023-07-13T22:24:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.664+0000 7f50f97fa700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50e006ace0 0x7f50e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.664+0000 7f50f97fa700 1 -- 172.21.15.60:0/2382531236 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f50f0089d40 con 0x7f50ec004d90 2023-07-13T22:24:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.664+0000 7f50fbfff700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50e006ace0 0x7f50e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.665+0000 7f50fbfff700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50e006ace0 0x7f50e006d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009b50 tx=0x7f50e8009670 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.668+0000 7f50f97fa700 1 -- 172.21.15.60:0/2382531236 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f50f0058810 con 0x7f50ec004d90 2023-07-13T22:24:47.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.874+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50fc071440 con 0x7f50ec004d90 2023-07-13T22:24:47.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.875+0000 7f50f97fa700 1 -- 172.21.15.60:0/2382531236 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+18439 (secure 0 0 0) 0x7f50f0022090 con 0x7f50ec004d90 2023-07-13T22:24:47.858 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:47.858 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":64,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:45.486987+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:24:18.441889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-13T22:24:19.114200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.11","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:47.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.877+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50e006ace0 msgr2=0x7f50e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.877+0000 7f5102dff700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50e006ace0 0x7f50e006d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009b50 tx=0x7f50e8009670 comp rx=0 tx=0).stop 2023-07-13T22:24:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.877+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 msgr2=0x7f50fc1f4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.877+0000 7f5102dff700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50fc1f4f60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50f0011ad0 tx=0x7f50f0011e90 comp rx=0 tx=0).stop 2023-07-13T22:24:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.878+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 shutdown_connections 2023-07-13T22:24:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.878+0000 7f5102dff700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50e006ace0 0x7f50e006d190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.878+0000 7f5102dff700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50ec004d90 0x7f50fc1f4f60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.878+0000 7f5102dff700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50fc1f5660 0x7f50fc1f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.878+0000 7f5102dff700 1 --2- 172.21.15.60:0/2382531236 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50fc15b5c0 0x7f50fc15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.878+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 >> 172.21.15.60:0/2382531236 conn(0x7f50fc0cf410 msgr2=0x7f50fc0ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:47.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.878+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 shutdown_connections 2023-07-13T22:24:47.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:47.878+0000 7f5102dff700 1 -- 172.21.15.60:0/2382531236 wait complete. 2023-07-13T22:24:47.879 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:24:47.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:48.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.099+0000 7f2448dff700 1 Processor -- start 2023-07-13T22:24:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.099+0000 7f2448dff700 1 -- start start 2023-07-13T22:24:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.100+0000 7f2448dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440b8400 0x7f24440cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.100+0000 7f2448dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24440cfca0 0x7f24440d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.100+0000 7f2448dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24440d0ac0 con 0x7f24440d3750 2023-07-13T22:24:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.100+0000 7f2448dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24440b87d0 con 0x7f24440cfca0 2023-07-13T22:24:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.100+0000 7f2448dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440b4eb0 con 0x7f24440b8400 2023-07-13T22:24:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f2442d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24440cfca0 0x7f24440d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f2442d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24440cfca0 0x7f24440d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47590/0 (socket says 172.21.15.60:47590) 2023-07-13T22:24:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f2442d9d700 1 -- 172.21.15.60:0/1096487120 learned_addr learned my addr 172.21.15.60:0/1096487120 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f244259c700 1 --2- 172.21.15.60:0/1096487120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440b8400 0x7f24440cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f2442d9d700 1 -- 172.21.15.60:0/1096487120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440b8400 msgr2=0x7f24440cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f2442d9d700 1 --2- 172.21.15.60:0/1096487120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440b8400 0x7f24440cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f2442d9d700 1 -- 172.21.15.60:0/1096487120 >> v1:172.21.15.105:6789/0 conn(0x7f24440d3750 legacy=0x7f24440cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f2442d9d700 1 -- 172.21.15.60:0/1096487120 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440b5190 con 0x7f24440cfca0 2023-07-13T22:24:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f244159a700 1 -- 172.21.15.60:0/1096487120 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408360206 0 0) 0x7f24440d0ac0 con 0x7f24440d3750 2023-07-13T22:24:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.101+0000 7f2442d9d700 1 --2- 172.21.15.60:0/1096487120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440cfca0 0x7f24440d0f50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f243c009e50 tx=0x7f243c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cc6a1e09645b9b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f244159a700 1 -- 172.21.15.60:0/1096487120 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00e070 con 0x7f24440cfca0 2023-07-13T22:24:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f244159a700 1 -- 172.21.15.60:0/1096487120 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c00b910 con 0x7f24440cfca0 2023-07-13T22:24:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f244159a700 1 -- 172.21.15.60:0/1096487120 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00bcc0 con 0x7f24440cfca0 2023-07-13T22:24:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f2448dff700 1 -- 172.21.15.60:0/1096487120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440cfca0 msgr2=0x7f24440d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f2448dff700 1 --2- 172.21.15.60:0/1096487120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440cfca0 0x7f24440d0f50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f243c009e50 tx=0x7f243c00b330 comp rx=0 tx=0).stop 2023-07-13T22:24:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f2448dff700 1 -- 172.21.15.60:0/1096487120 shutdown_connections 2023-07-13T22:24:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f2448dff700 1 --2- 172.21.15.60:0/1096487120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440cfca0 0x7f24440d0f50 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f2448dff700 1 --2- 172.21.15.60:0/1096487120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440b8400 0x7f24440cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f2448dff700 1 -- 172.21.15.60:0/1096487120 >> 172.21.15.60:0/1096487120 conn(0x7f244406b390 msgr2=0x7f244406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f2448dff700 1 -- 172.21.15.60:0/1096487120 shutdown_connections 2023-07-13T22:24:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.102+0000 7f2448dff700 1 -- 172.21.15.60:0/1096487120 wait complete. 2023-07-13T22:24:48.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.104+0000 7f2448dff700 1 Processor -- start 2023-07-13T22:24:48.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.104+0000 7f2448dff700 1 -- start start 2023-07-13T22:24:48.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.105+0000 7f2448dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440b8400 0x7f24440c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.105+0000 7f2448dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24440cfca0 0x7f24440c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.105+0000 7f2448dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440d3750 0x7f2444158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.105+0000 7f2448dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cacf0 con 0x7f24440b8400 2023-07-13T22:24:48.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.105+0000 7f2448dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c6bf0 con 0x7f24440cfca0 2023-07-13T22:24:48.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.105+0000 7f2448dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440caf00 con 0x7f24440d3750 2023-07-13T22:24:48.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.105+0000 7f244259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440b8400 0x7f24440c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.105+0000 7f244259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440b8400 0x7f24440c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47598/0 (socket says 172.21.15.60:47598) 2023-07-13T22:24:48.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.105+0000 7f244259c700 1 -- 172.21.15.60:0/694685408 learned_addr learned my addr 172.21.15.60:0/694685408 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:48.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f2442d9d700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440d3750 0x7f2444158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f2441d9b700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24440cfca0 0x7f24440c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f244259c700 1 -- 172.21.15.60:0/694685408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440d3750 msgr2=0x7f2444158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f244259c700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440d3750 0x7f2444158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f244259c700 1 -- 172.21.15.60:0/694685408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24440cfca0 msgr2=0x7f24440c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f244259c700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24440cfca0 0x7f24440c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f244259c700 1 -- 172.21.15.60:0/694685408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c009b30 con 0x7f24440b8400 2023-07-13T22:24:48.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f2442d9d700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440d3750 0x7f2444158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:48.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f244259c700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440b8400 0x7f24440c7270 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f243400e870 tx=0x7f243400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:48.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f2432ffd700 1 -- 172.21.15.60:0/694685408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434009590 con 0x7f24440b8400 2023-07-13T22:24:48.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f2432ffd700 1 -- 172.21.15.60:0/694685408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243400ee70 con 0x7f24440b8400 2023-07-13T22:24:48.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24440cb1e0 con 0x7f24440b8400 2023-07-13T22:24:48.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f2432ffd700 1 -- 172.21.15.60:0/694685408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243400ca50 con 0x7f24440b8400 2023-07-13T22:24:48.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.106+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24440cb7a0 con 0x7f24440b8400 2023-07-13T22:24:48.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.107+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24440b4a50 con 0x7f24440b8400 2023-07-13T22:24:48.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.108+0000 7f2432ffd700 1 -- 172.21.15.60:0/694685408 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f243400cbb0 con 0x7f24440b8400 2023-07-13T22:24:48.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.109+0000 7f2432ffd700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f242406ace0 0x7f242406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.109+0000 7f2432ffd700 1 -- 172.21.15.60:0/694685408 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2434089530 con 0x7f24440b8400 2023-07-13T22:24:48.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.110+0000 7f2441d9b700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f242406ace0 0x7f242406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.110+0000 7f2441d9b700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f242406ace0 0x7f242406d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f24441596d0 tx=0x7f2438009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:48.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.114+0000 7f2432ffd700 1 -- 172.21.15.60:0/694685408 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2434057fd0 con 0x7f24440b8400 2023-07-13T22:24:48.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.363+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24440d0750 con 0x7f24440b8400 2023-07-13T22:24:48.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.363+0000 7f2432ffd700 1 -- 172.21.15.60:0/694685408 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f243400ce60 con 0x7f24440b8400 2023-07-13T22:24:48.346 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:48.346 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/711733521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":711733521},{"type":"v1","addr":"172.21.15.105:6834","nonce":711733521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2135065566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":2135065566},{"type":"v1","addr":"172.21.15.105:6835","nonce":2135065566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6052,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4236704538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4236704538},{"type":"v1","addr":"172.21.15.60:6836","nonce":4236704538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:24:20.139470+0000","modified":"2023-07-13T22:24:23.665486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6058":{"gid":6058,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3036622731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3036622731},{"type":"v1","addr":"172.21.15.60:6837","nonce":3036622731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-13T22:24:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f242406ace0 msgr2=0x7f242406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f242406ace0 0x7f242406d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f24441596d0 tx=0x7f2438009040 comp rx=0 tx=0).stop 2023-07-13T22:24:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440b8400 msgr2=0x7f24440c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440b8400 0x7f24440c7270 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f243400e870 tx=0x7f243400ec30 comp rx=0 tx=0).stop 2023-07-13T22:24:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 shutdown_connections 2023-07-13T22:24:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f242406ace0 0x7f242406d190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24440b8400 0x7f24440c7270 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24440d3750 0x7f2444158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 --2- 172.21.15.60:0/694685408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24440cfca0 0x7f24440c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 >> 172.21.15.60:0/694685408 conn(0x7f244406b390 msgr2=0x7f24440d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:48.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 shutdown_connections 2023-07-13T22:24:48.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.366+0000 7f2448dff700 1 -- 172.21.15.60:0/694685408 wait complete. 2023-07-13T22:24:48.350 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 33 2023-07-13T22:24:48.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.588+0000 7f71649ff700 1 Processor -- start 2023-07-13T22:24:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.588+0000 7f71649ff700 1 -- start start 2023-07-13T22:24:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.589+0000 7f71649ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c6d0 0x7f71600b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.589+0000 7f71649ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71600b46c0 0x7f71600b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.589+0000 7f71649ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71600b5060 con 0x7f716015c300 2023-07-13T22:24:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.589+0000 7f71649ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71600d18a0 con 0x7f71600b46c0 2023-07-13T22:24:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.589+0000 7f71649ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71600cf500 con 0x7f716015c6d0 2023-07-13T22:24:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.590+0000 7f715ed9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71600b46c0 0x7f71600b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.590+0000 7f715dd9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c6d0 0x7f71600b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.590+0000 7f715ed9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71600b46c0 0x7f71600b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:43944/0 (socket says 172.21.15.60:43944) 2023-07-13T22:24:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.590+0000 7f715ed9d700 1 -- 172.21.15.60:0/1677900988 learned_addr learned my addr 172.21.15.60:0/1677900988 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.590+0000 7f715dd9b700 1 -- 172.21.15.60:0/1677900988 >> v2:172.21.15.105:3300/0 conn(0x7f71600b46c0 msgr2=0x7f71600b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.590+0000 7f715dd9b700 1 --2- 172.21.15.60:0/1677900988 >> v2:172.21.15.105:3300/0 conn(0x7f71600b46c0 0x7f71600b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.590+0000 7f715dd9b700 1 -- 172.21.15.60:0/1677900988 >> v1:172.21.15.105:6789/0 conn(0x7f716015c300 legacy=0x7f71600cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.590+0000 7f715dd9b700 1 -- 172.21.15.60:0/1677900988 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71600d15c0 con 0x7f716015c6d0 2023-07-13T22:24:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.591+0000 7f715dd9b700 1 --2- 172.21.15.60:0/1677900988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c6d0 0x7f71600b5a90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7154009840 tx=0x7f7154009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c54f648aaa0eb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:48.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.591+0000 7f715d59a700 1 -- 172.21.15.60:0/1677900988 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154011830 con 0x7f716015c6d0 2023-07-13T22:24:48.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.591+0000 7f715d59a700 1 -- 172.21.15.60:0/1677900988 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7154011990 con 0x7f716015c6d0 2023-07-13T22:24:48.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.591+0000 7f715d59a700 1 -- 172.21.15.60:0/1677900988 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154011d20 con 0x7f716015c6d0 2023-07-13T22:24:48.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.592+0000 7f71649ff700 1 -- 172.21.15.60:0/1677900988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c6d0 msgr2=0x7f71600b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.592+0000 7f71649ff700 1 --2- 172.21.15.60:0/1677900988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c6d0 0x7f71600b5a90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7154009840 tx=0x7f7154009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.592+0000 7f71649ff700 1 -- 172.21.15.60:0/1677900988 shutdown_connections 2023-07-13T22:24:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.592+0000 7f71649ff700 1 --2- 172.21.15.60:0/1677900988 >> v2:172.21.15.105:3300/0 conn(0x7f71600b46c0 0x7f71600b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.592+0000 7f71649ff700 1 --2- 172.21.15.60:0/1677900988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c6d0 0x7f71600b5a90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.592+0000 7f71649ff700 1 -- 172.21.15.60:0/1677900988 >> 172.21.15.60:0/1677900988 conn(0x7f71600cf6c0 msgr2=0x7f71600cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.592+0000 7f71649ff700 1 -- 172.21.15.60:0/1677900988 shutdown_connections 2023-07-13T22:24:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.592+0000 7f71649ff700 1 -- 172.21.15.60:0/1677900988 wait complete. 2023-07-13T22:24:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.593+0000 7f71649ff700 1 Processor -- start 2023-07-13T22:24:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.594+0000 7f71649ff700 1 -- start start 2023-07-13T22:24:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.594+0000 7f71649ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71600b46c0 0x7f716015d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.594+0000 7f71649ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c300 0x7f71600b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f71649ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f716015c6d0 0x7f71600b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f71649ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71600cbc80 con 0x7f716015c6d0 2023-07-13T22:24:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f71649ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71600cad60 con 0x7f71600b46c0 2023-07-13T22:24:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f71649ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716015da50 con 0x7f716015c300 2023-07-13T22:24:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f716015c6d0 0x7f71600b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f716015c6d0 0x7f71600b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47606/0 (socket says 172.21.15.60:47606) 2023-07-13T22:24:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715ed9d700 1 -- 172.21.15.60:0/724659847 learned_addr learned my addr 172.21.15.60:0/724659847 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715dd9b700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c300 0x7f71600b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715e59c700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71600b46c0 0x7f716015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715ed9d700 1 -- 172.21.15.60:0/724659847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c300 msgr2=0x7f71600b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715ed9d700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c300 0x7f71600b7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715ed9d700 1 -- 172.21.15.60:0/724659847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71600b46c0 msgr2=0x7f716015d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715ed9d700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71600b46c0 0x7f716015d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.595+0000 7f715ed9d700 1 -- 172.21.15.60:0/724659847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71540094a0 con 0x7f716015c6d0 2023-07-13T22:24:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.596+0000 7f715ed9d700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f716015c6d0 0x7f71600b7760 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f715800efc0 tx=0x7f715800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.596+0000 7f714effd700 1 -- 172.21.15.60:0/724659847 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158003c20 con 0x7f716015c6d0 2023-07-13T22:24:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.596+0000 7f714effd700 1 -- 172.21.15.60:0/724659847 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715800f040 con 0x7f716015c6d0 2023-07-13T22:24:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.596+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71600cb000 con 0x7f716015c6d0 2023-07-13T22:24:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.596+0000 7f714effd700 1 -- 172.21.15.60:0/724659847 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715800c660 con 0x7f716015c6d0 2023-07-13T22:24:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.596+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71600cb5c0 con 0x7f716015c6d0 2023-07-13T22:24:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.597+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71600cbdc0 con 0x7f716015c6d0 2023-07-13T22:24:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.598+0000 7f714effd700 1 -- 172.21.15.60:0/724659847 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f715800c7c0 con 0x7f716015c6d0 2023-07-13T22:24:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.599+0000 7f714effd700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f714006ace0 0x7f714006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.599+0000 7f714effd700 1 -- 172.21.15.60:0/724659847 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f71580897d0 con 0x7f716015c6d0 2023-07-13T22:24:48.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.599+0000 7f715e59c700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f714006ace0 0x7f714006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:48.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.600+0000 7f715e59c700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f714006ace0 0x7f714006d190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f71600d0ba0 tx=0x7f7150009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.604+0000 7f714effd700 1 -- 172.21.15.60:0/724659847 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f71600cbdc0 con 0x7f716015c6d0 2023-07-13T22:24:48.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.860+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71600cbdc0 con 0x7f716015c6d0 2023-07-13T22:24:48.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.861+0000 7f714effd700 1 -- 172.21.15.60:0/724659847 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f71600cbdc0 con 0x7f716015c6d0 2023-07-13T22:24:48.844 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:48.844 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/711733521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":711733521},{"type":"v1","addr":"172.21.15.105:6834","nonce":711733521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2135065566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":2135065566},{"type":"v1","addr":"172.21.15.105:6835","nonce":2135065566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6052,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4236704538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4236704538},{"type":"v1","addr":"172.21.15.60:6836","nonce":4236704538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:24:20.139470+0000","modified":"2023-07-13T22:24:23.665486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6058":{"gid":6058,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3036622731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3036622731},{"type":"v1","addr":"172.21.15.60:6837","nonce":3036622731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-13T22:24:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f714006ace0 msgr2=0x7f714006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f714006ace0 0x7f714006d190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f71600d0ba0 tx=0x7f7150009040 comp rx=0 tx=0).stop 2023-07-13T22:24:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f716015c6d0 msgr2=0x7f71600b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f716015c6d0 0x7f71600b7760 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f715800efc0 tx=0x7f715800c0c0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 shutdown_connections 2023-07-13T22:24:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f714006ace0 0x7f714006d190 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f716015c6d0 0x7f71600b7760 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71600b46c0 0x7f716015d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 --2- 172.21.15.60:0/724659847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f716015c300 0x7f71600b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 >> 172.21.15.60:0/724659847 conn(0x7f71600cf6c0 msgr2=0x7f71600d2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 shutdown_connections 2023-07-13T22:24:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:48.864+0000 7f71649ff700 1 -- 172.21.15.60:0/724659847 wait complete. 2023-07-13T22:24:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 33 2023-07-13T22:24:48.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:49.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.085+0000 7f2ac43ff700 1 Processor -- start 2023-07-13T22:24:49.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.085+0000 7f2ac43ff700 1 -- start start 2023-07-13T22:24:49.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.087+0000 7f2ac43ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0d3750 0x7f2abc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.087+0000 7f2ac43ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc0cd950 con 0x7f2abc0cfad0 2023-07-13T22:24:49.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.087+0000 7f2ac43ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc0cff20 con 0x7f2abc0b8400 2023-07-13T22:24:49.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.087+0000 7f2ac43ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0d0ac0 con 0x7f2abc0d3750 2023-07-13T22:24:49.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.087+0000 7f2ac299c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2abc0cfad0 0x7f2abc0d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59884/0 (socket says 172.21.15.60:59884) 2023-07-13T22:24:49.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.087+0000 7f2ac299c700 1 -- 172.21.15.60:0/1658771083 learned_addr learned my addr 172.21.15.60:0/1658771083 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:49.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.087+0000 7f2ac199a700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0d3750 0x7f2abc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.087+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345077460 0 0) 0x7f2ab4001490 con 0x7f2abc0cfad0 2023-07-13T22:24:49.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa80035b0 con 0x7f2abc0cfad0 2023-07-13T22:24:49.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116494434 0 0) 0x7f2ab8001110 con 0x7f2abc0b8400 2023-07-13T22:24:49.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ab4001490 con 0x7f2abc0b8400 2023-07-13T22:24:49.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1045630394 0 0) 0x7f2aa80035b0 con 0x7f2abc0cfad0 2023-07-13T22:24:49.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0d3750 msgr2=0x7f2abc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:49.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0d3750 0x7f2abc0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 >> v1:172.21.15.105:6789/0 conn(0x7f2abc0b8400 legacy=0x7f2abc0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc0b87d0 con 0x7f2abc0cfad0 2023-07-13T22:24:49.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f2ab4001b80 con 0x7f2abc0cfad0 2023-07-13T22:24:49.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 >> v1:172.21.15.60:6789/0 conn(0x7f2abc0cfad0 legacy=0x7f2abc0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2aa8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0d3750 0x7f2aa8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa8004d90 0x7f2aa8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0d0ac0 con 0x7f2aa8003bc0 2023-07-13T22:24:49.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa800bb10 con 0x7f2aa8004d90 2023-07-13T22:24:49.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa800bcd0 con 0x7f2abc0d3750 2023-07-13T22:24:49.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac219b700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2aa8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac299c700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa8004d90 0x7f2aa8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.088+0000 7f2ac199a700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0d3750 0x7f2aa8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.089+0000 7f2ac219b700 1 -- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0d3750 msgr2=0x7f2aa8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.089+0000 7f2ac219b700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0d3750 0x7f2aa8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.089+0000 7f2ac219b700 1 -- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa8004d90 msgr2=0x7f2aa8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:49.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.089+0000 7f2ac219b700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2aa8004d90 0x7f2aa8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.089+0000 7f2ac219b700 1 -- 172.21.15.60:0/1658771083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa80035b0 con 0x7f2aa8003bc0 2023-07-13T22:24:49.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.089+0000 7f2ac219b700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2aa8003fb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8001970 tx=0x7f2aa8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9339d3bca6130c8c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:49.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.089+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8001600 con 0x7f2aa8003bc0 2023-07-13T22:24:49.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.089+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab80052d0 con 0x7f2aa8003bc0 2023-07-13T22:24:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.089+0000 7f2ac1199700 1 -- 172.21.15.60:0/1658771083 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8003680 con 0x7f2aa8003bc0 2023-07-13T22:24:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.090+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1658771083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 msgr2=0x7f2aa8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.090+0000 7f2ac43ff700 1 --2- 172.21.15.60:0/1658771083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2aa8003fb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8001970 tx=0x7f2aa8005880 comp rx=0 tx=0).stop 2023-07-13T22:24:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.090+0000 7f2ac299c700 1 -- 172.21.15.60:0/1658771083 reap_dead start 2023-07-13T22:24:49.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.090+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1658771083 shutdown_connections 2023-07-13T22:24:49.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.090+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1658771083 >> 172.21.15.60:0/1658771083 conn(0x7f2abc06b390 msgr2=0x7f2abc06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:49.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.090+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1658771083 shutdown_connections 2023-07-13T22:24:49.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.090+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1658771083 wait complete. 2023-07-13T22:24:49.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.092+0000 7f2ac43ff700 1 Processor -- start 2023-07-13T22:24:49.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.092+0000 7f2ac43ff700 1 -- start start 2023-07-13T22:24:49.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac43ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2abc1f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac43ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0cc780 0x7f2abc1e7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac43ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2abc1e79c0 0x7f2abc158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac43ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc1f37f0 con 0x7f2aa8003bc0 2023-07-13T22:24:49.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac43ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc1f2be0 con 0x7f2abc1e79c0 2023-07-13T22:24:49.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac43ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc1f2ec0 con 0x7f2abc0cc780 2023-07-13T22:24:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac219b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2abc1f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac219b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2abc1f39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47624/0 (socket says 172.21.15.60:47624) 2023-07-13T22:24:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac219b700 1 -- 172.21.15.60:0/1557496598 learned_addr learned my addr 172.21.15.60:0/1557496598 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac299c700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2abc1e79c0 0x7f2abc158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac199a700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0cc780 0x7f2abc1e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac219b700 1 -- 172.21.15.60:0/1557496598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0cc780 msgr2=0x7f2abc1e7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac219b700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0cc780 0x7f2abc1e7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac219b700 1 -- 172.21.15.60:0/1557496598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2abc1e79c0 msgr2=0x7f2abc158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac219b700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2abc1e79c0 0x7f2abc158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac219b700 1 -- 172.21.15.60:0/1557496598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc1f31a0 con 0x7f2aa8003bc0 2023-07-13T22:24:49.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.093+0000 7f2ac299c700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2abc1e79c0 0x7f2abc158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:49.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.094+0000 7f2ac219b700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2abc1f39c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c3c0 tx=0x7f2ab80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:49.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.094+0000 7f2ab2ffd700 1 -- 172.21.15.60:0/1557496598 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8006a70 con 0x7f2aa8003bc0 2023-07-13T22:24:49.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.094+0000 7f2ab2ffd700 1 -- 172.21.15.60:0/1557496598 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab8006bd0 con 0x7f2aa8003bc0 2023-07-13T22:24:49.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.094+0000 7f2ab2ffd700 1 -- 172.21.15.60:0/1557496598 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8007040 con 0x7f2aa8003bc0 2023-07-13T22:24:49.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.094+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc1f3480 con 0x7f2aa8003bc0 2023-07-13T22:24:49.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.094+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc1f2980 con 0x7f2aa8003bc0 2023-07-13T22:24:49.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.095+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc1e5e60 con 0x7f2aa8003bc0 2023-07-13T22:24:49.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.096+0000 7f2ab2ffd700 1 -- 172.21.15.60:0/1557496598 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ab80071a0 con 0x7f2aa8003bc0 2023-07-13T22:24:49.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.098+0000 7f2ab2ffd700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2aa006ace0 0x7f2aa006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.098+0000 7f2ab2ffd700 1 -- 172.21.15.60:0/1557496598 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2ab808a2e0 con 0x7f2aa8003bc0 2023-07-13T22:24:49.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.098+0000 7f2ac199a700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2aa006ace0 0x7f2aa006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.098+0000 7f2ac199a700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2aa006ace0 0x7f2aa006d190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2abc15aee0 tx=0x7f2aac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:49.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.102+0000 7f2ab2ffd700 1 -- 172.21.15.60:0/1557496598 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2ab80552f0 con 0x7f2aa8003bc0 2023-07-13T22:24:49.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.313+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2abc0cff20 con 0x7f2aa8003bc0 2023-07-13T22:24:49.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.314+0000 7f2ab2ffd700 1 -- 172.21.15.60:0/1557496598 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+18439 (secure 0 0 0) 0x7f2abc0cff20 con 0x7f2aa8003bc0 2023-07-13T22:24:49.297 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:49.298 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":64,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:45.486987+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:24:18.441889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-13T22:24:19.114200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.11","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:49.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.316+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2aa006ace0 msgr2=0x7f2aa006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.316+0000 7f2ac43ff700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2aa006ace0 0x7f2aa006d190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2abc15aee0 tx=0x7f2aac008040 comp rx=0 tx=0).stop 2023-07-13T22:24:49.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.316+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 msgr2=0x7f2abc1f39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.316+0000 7f2ac43ff700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2abc1f39c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c3c0 tx=0x7f2ab80014a0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.317+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 shutdown_connections 2023-07-13T22:24:49.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.317+0000 7f2ac43ff700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2aa006ace0 0x7f2aa006d190 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.317+0000 7f2ac43ff700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2aa8003bc0 0x7f2abc1f39c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.317+0000 7f2ac43ff700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2abc1e79c0 0x7f2abc158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.317+0000 7f2ac43ff700 1 --2- 172.21.15.60:0/1557496598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2abc0cc780 0x7f2abc1e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.317+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 >> 172.21.15.60:0/1557496598 conn(0x7f2abc06b390 msgr2=0x7f2abc15b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:49.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.317+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 shutdown_connections 2023-07-13T22:24:49.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.317+0000 7f2ac43ff700 1 -- 172.21.15.60:0/1557496598 wait complete. 2023-07-13T22:24:49.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:24:49.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.542+0000 7f2b89fff700 1 Processor -- start 2023-07-13T22:24:49.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.542+0000 7f2b89fff700 1 -- start start 2023-07-13T22:24:49.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.543+0000 7f2b89fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840b8400 0x7f2b840cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.543+0000 7f2b89fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2b840cfca0 0x7f2b840d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.543+0000 7f2b89fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b840d0ac0 con 0x7f2b840d3750 2023-07-13T22:24:49.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.543+0000 7f2b89fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b840b87d0 con 0x7f2b840cfca0 2023-07-13T22:24:49.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.543+0000 7f2b89fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840b4eb0 con 0x7f2b840b8400 2023-07-13T22:24:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.544+0000 7f2b837fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840b8400 0x7f2b840cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.544+0000 7f2b83fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2b840cfca0 0x7f2b840d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.544+0000 7f2b83fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2b840cfca0 0x7f2b840d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:43966/0 (socket says 172.21.15.60:43966) 2023-07-13T22:24:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.544+0000 7f2b83fff700 1 -- 172.21.15.60:0/1953744423 learned_addr learned my addr 172.21.15.60:0/1953744423 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.544+0000 7f2b837fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840b8400 0x7f2b840cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:46934/0 (socket says 172.21.15.60:46934) 2023-07-13T22:24:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.545+0000 7f2b837fe700 1 -- 172.21.15.60:0/1953744423 >> v2:172.21.15.105:3300/0 conn(0x7f2b840cfca0 msgr2=0x7f2b840d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.545+0000 7f2b837fe700 1 --2- 172.21.15.60:0/1953744423 >> v2:172.21.15.105:3300/0 conn(0x7f2b840cfca0 0x7f2b840d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.545+0000 7f2b837fe700 1 -- 172.21.15.60:0/1953744423 >> v1:172.21.15.105:6789/0 conn(0x7f2b840d3750 legacy=0x7f2b840cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.545+0000 7f2b837fe700 1 -- 172.21.15.60:0/1953744423 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b840b5190 con 0x7f2b840b8400 2023-07-13T22:24:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.545+0000 7f2b83fff700 1 --2- 172.21.15.60:0/1953744423 >> v2:172.21.15.105:3300/0 conn(0x7f2b840cfca0 0x7f2b840d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.545+0000 7f2b837fe700 1 --2- 172.21.15.60:0/1953744423 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840b8400 0x7f2b840cd240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009840 tx=0x7f2b74009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b043124b45ce388 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.545+0000 7f2b827fc700 1 -- 172.21.15.60:0/1953744423 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74011830 con 0x7f2b840b8400 2023-07-13T22:24:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.545+0000 7f2b827fc700 1 -- 172.21.15.60:0/1953744423 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b74011990 con 0x7f2b840b8400 2023-07-13T22:24:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.545+0000 7f2b827fc700 1 -- 172.21.15.60:0/1953744423 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74011d20 con 0x7f2b840b8400 2023-07-13T22:24:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.546+0000 7f2b89fff700 1 -- 172.21.15.60:0/1953744423 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840b8400 msgr2=0x7f2b840cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.546+0000 7f2b89fff700 1 --2- 172.21.15.60:0/1953744423 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840b8400 0x7f2b840cd240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009840 tx=0x7f2b74009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.546+0000 7f2b89fff700 1 -- 172.21.15.60:0/1953744423 shutdown_connections 2023-07-13T22:24:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.546+0000 7f2b89fff700 1 --2- 172.21.15.60:0/1953744423 >> v2:172.21.15.105:3300/0 conn(0x7f2b840cfca0 0x7f2b840d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.546+0000 7f2b89fff700 1 --2- 172.21.15.60:0/1953744423 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840b8400 0x7f2b840cd240 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.546+0000 7f2b89fff700 1 -- 172.21.15.60:0/1953744423 >> 172.21.15.60:0/1953744423 conn(0x7f2b8406b390 msgr2=0x7f2b8406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.546+0000 7f2b89fff700 1 -- 172.21.15.60:0/1953744423 shutdown_connections 2023-07-13T22:24:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.546+0000 7f2b89fff700 1 -- 172.21.15.60:0/1953744423 wait complete. 2023-07-13T22:24:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.548+0000 7f2b89fff700 1 Processor -- start 2023-07-13T22:24:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.548+0000 7f2b89fff700 1 -- start start 2023-07-13T22:24:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b89fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b840b8400 0x7f2b8415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b89fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840cfca0 0x7f2b841491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b89fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b840d3750 0x7f2b840b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b89fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8415e6c0 con 0x7f2b840d3750 2023-07-13T22:24:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b89fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8415e9a0 con 0x7f2b840b8400 2023-07-13T22:24:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b89fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8415ec80 con 0x7f2b840cfca0 2023-07-13T22:24:49.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b83fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b840d3750 0x7f2b840b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b83fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b840d3750 0x7f2b840b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47634/0 (socket says 172.21.15.60:47634) 2023-07-13T22:24:49.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b83fff700 1 -- 172.21.15.60:0/4066441662 learned_addr learned my addr 172.21.15.60:0/4066441662 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:49.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b82ffd700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840cfca0 0x7f2b841491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.549+0000 7f2b837fe700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b840b8400 0x7f2b8415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b83fff700 1 -- 172.21.15.60:0/4066441662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840cfca0 msgr2=0x7f2b841491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b83fff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840cfca0 0x7f2b841491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b83fff700 1 -- 172.21.15.60:0/4066441662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b840b8400 msgr2=0x7f2b8415dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b83fff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b840b8400 0x7f2b8415dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b83fff700 1 -- 172.21.15.60:0/4066441662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c009b30 con 0x7f2b840d3750 2023-07-13T22:24:49.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b83fff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b840d3750 0x7f2b840b6920 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00ef90 tx=0x7f2b7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:49.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b6bfff700 1 -- 172.21.15.60:0/4066441662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c018070 con 0x7f2b840d3750 2023-07-13T22:24:49.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b6bfff700 1 -- 172.21.15.60:0/4066441662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c00f040 con 0x7f2b840d3750 2023-07-13T22:24:49.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b6bfff700 1 -- 172.21.15.60:0/4066441662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c003c60 con 0x7f2b840d3750 2023-07-13T22:24:49.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.550+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b740094a0 con 0x7f2b840d3750 2023-07-13T22:24:49.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.551+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b841496e0 con 0x7f2b840d3750 2023-07-13T22:24:49.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.551+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8415cbd0 con 0x7f2b840d3750 2023-07-13T22:24:49.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.552+0000 7f2b6bfff700 1 -- 172.21.15.60:0/4066441662 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b7c00c880 con 0x7f2b840d3750 2023-07-13T22:24:49.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.554+0000 7f2b6bfff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2b7006ace0 0x7f2b7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:49.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.554+0000 7f2b6bfff700 1 -- 172.21.15.60:0/4066441662 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2b7c0895b0 con 0x7f2b840d3750 2023-07-13T22:24:49.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.554+0000 7f2b837fe700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2b7006ace0 0x7f2b7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:49.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.555+0000 7f2b837fe700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2b7006ace0 0x7f2b7006d190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009fd0 tx=0x7f2b7401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:49.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.558+0000 7f2b6bfff700 1 -- 172.21.15.60:0/4066441662 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2b7c0545c0 con 0x7f2b840d3750 2023-07-13T22:24:49.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.810+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2b8415f1a0 con 0x7f2b840d3750 2023-07-13T22:24:49.793 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:24:49.810+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:24:49.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.815+0000 7f2b6bfff700 1 -- 172.21.15.60:0/4066441662 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2b8415f1a0 con 0x7f2b840d3750 2023-07-13T22:24:49.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.818+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2b7006ace0 msgr2=0x7f2b7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.818+0000 7f2b89fff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2b7006ace0 0x7f2b7006d190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009fd0 tx=0x7f2b7401c040 comp rx=0 tx=0).stop 2023-07-13T22:24:49.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.818+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b840d3750 msgr2=0x7f2b840b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:49.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.818+0000 7f2b89fff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b840d3750 0x7f2b840b6920 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00ef90 tx=0x7f2b7c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.818+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 shutdown_connections 2023-07-13T22:24:49.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.819+0000 7f2b89fff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2b7006ace0 0x7f2b7006d190 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.819+0000 7f2b89fff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b840d3750 0x7f2b840b6920 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.819+0000 7f2b89fff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b840b8400 0x7f2b8415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.819+0000 7f2b89fff700 1 --2- 172.21.15.60:0/4066441662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b840cfca0 0x7f2b841491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:49.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.819+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 >> 172.21.15.60:0/4066441662 conn(0x7f2b8406b390 msgr2=0x7f2b8415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:49.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.819+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 shutdown_connections 2023-07-13T22:24:49.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:49.819+0000 7f2b89fff700 1 -- 172.21.15.60:0/4066441662 wait complete. 2023-07-13T22:24:49.803 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:24:49.828 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:24:49.885 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:24:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.056+0000 7fcfca5ff700 1 Processor -- start 2023-07-13T22:24:50.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.056+0000 7fcfca5ff700 1 -- start start 2023-07-13T22:24:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.057+0000 7fcfca5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcfc415c530 0x7fcfc40b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.057+0000 7fcfca5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcfc40cfbc0 0x7fcfc4165eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.057+0000 7fcfca5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc40cffe0 con 0x7fcfc40d11c0 2023-07-13T22:24:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.057+0000 7fcfca5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc40cefd0 con 0x7fcfc40cfbc0 2023-07-13T22:24:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.057+0000 7fcfca5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc40d1780 con 0x7fcfc415c530 2023-07-13T22:24:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.057+0000 7fcfc8b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcfc40cfbc0 0x7fcfc4165eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc37fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcfc40d11c0 0x7fcfc40cf4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59912/0 (socket says 172.21.15.60:59912) 2023-07-13T22:24:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc37fe700 1 -- 172.21.15.60:0/107635656 learned_addr learned my addr 172.21.15.60:0/107635656 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc3fff700 1 --2- 172.21.15.60:0/107635656 >> v2:172.21.15.105:3300/0 conn(0x7fcfc415c530 0x7fcfc40b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc2ffd700 1 -- 172.21.15.60:0/107635656 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037836384 0 0) 0x7fcfc40cffe0 con 0x7fcfc40d11c0 2023-07-13T22:24:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc2ffd700 1 -- 172.21.15.60:0/107635656 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfac0035b0 con 0x7fcfc40d11c0 2023-07-13T22:24:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc8b9c700 1 -- 172.21.15.60:0/107635656 >> v2:172.21.15.105:3300/0 conn(0x7fcfc415c530 msgr2=0x7fcfc40b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc8b9c700 1 --2- 172.21.15.60:0/107635656 >> v2:172.21.15.105:3300/0 conn(0x7fcfc415c530 0x7fcfc40b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc8b9c700 1 -- 172.21.15.60:0/107635656 >> v1:172.21.15.60:6789/0 conn(0x7fcfc40d11c0 legacy=0x7fcfc40cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc8b9c700 1 -- 172.21.15.60:0/107635656 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc40cd2d0 con 0x7fcfc40cfbc0 2023-07-13T22:24:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.058+0000 7fcfc8b9c700 1 --2- 172.21.15.60:0/107635656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40cfbc0 0x7fcfc4165eb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40d0760 tx=0x7fcfbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b2c906208ed78e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfc2ffd700 1 -- 172.21.15.60:0/107635656 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00e070 con 0x7fcfc40cfbc0 2023-07-13T22:24:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfc2ffd700 1 -- 172.21.15.60:0/107635656 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfbc00bb20 con 0x7fcfc40cfbc0 2023-07-13T22:24:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfc2ffd700 1 -- 172.21.15.60:0/107635656 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc012430 con 0x7fcfc40cfbc0 2023-07-13T22:24:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfca5ff700 1 -- 172.21.15.60:0/107635656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40cfbc0 msgr2=0x7fcfc4165eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfca5ff700 1 --2- 172.21.15.60:0/107635656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40cfbc0 0x7fcfc4165eb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40d0760 tx=0x7fcfbc00b330 comp rx=0 tx=0).stop 2023-07-13T22:24:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfca5ff700 1 -- 172.21.15.60:0/107635656 shutdown_connections 2023-07-13T22:24:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfca5ff700 1 --2- 172.21.15.60:0/107635656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40cfbc0 0x7fcfc4165eb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfca5ff700 1 --2- 172.21.15.60:0/107635656 >> v2:172.21.15.105:3300/0 conn(0x7fcfc415c530 0x7fcfc40b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfca5ff700 1 -- 172.21.15.60:0/107635656 >> 172.21.15.60:0/107635656 conn(0x7fcfc406b400 msgr2=0x7fcfc406b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfca5ff700 1 -- 172.21.15.60:0/107635656 shutdown_connections 2023-07-13T22:24:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.059+0000 7fcfca5ff700 1 -- 172.21.15.60:0/107635656 wait complete. 2023-07-13T22:24:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.061+0000 7fcfca5ff700 1 Processor -- start 2023-07-13T22:24:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.061+0000 7fcfca5ff700 1 -- start start 2023-07-13T22:24:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.062+0000 7fcfca5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcfc40cfbc0 0x7fcfc4156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.062+0000 7fcfca5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40d11c0 0x7fcfc415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.062+0000 7fcfca5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfc4157590 0x7fcfc415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.062+0000 7fcfca5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc4159ce0 con 0x7fcfc40d11c0 2023-07-13T22:24:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.062+0000 7fcfca5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc41e6320 con 0x7fcfc4157590 2023-07-13T22:24:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.062+0000 7fcfca5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc41e6600 con 0x7fcfc40cfbc0 2023-07-13T22:24:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40d11c0 0x7fcfc415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:50.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40d11c0 0x7fcfc415b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47670/0 (socket says 172.21.15.60:47670) 2023-07-13T22:24:50.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc37fe700 1 -- 172.21.15.60:0/4290504457 learned_addr learned my addr 172.21.15.60:0/4290504457 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:50.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc3fff700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcfc40cfbc0 0x7fcfc4156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:50.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc8b9c700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfc4157590 0x7fcfc415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:50.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc37fe700 1 -- 172.21.15.60:0/4290504457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcfc40cfbc0 msgr2=0x7fcfc4156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:50.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc37fe700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcfc40cfbc0 0x7fcfc4156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:50.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc37fe700 1 -- 172.21.15.60:0/4290504457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfc4157590 msgr2=0x7fcfc415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:50.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc37fe700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfc4157590 0x7fcfc415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:50.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc37fe700 1 -- 172.21.15.60:0/4290504457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc009a70 con 0x7fcfc40d11c0 2023-07-13T22:24:50.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc37fe700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40d11c0 0x7fcfc415b4c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800c700 tx=0x7fcfb800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:50.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc0ff9700 1 -- 172.21.15.60:0/4290504457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8005d70 con 0x7fcfc40d11c0 2023-07-13T22:24:50.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc0ff9700 1 -- 172.21.15.60:0/4290504457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb80070d0 con 0x7fcfc40d11c0 2023-07-13T22:24:50.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.063+0000 7fcfc0ff9700 1 -- 172.21.15.60:0/4290504457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8007460 con 0x7fcfc40d11c0 2023-07-13T22:24:50.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.064+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc41e68e0 con 0x7fcfc40d11c0 2023-07-13T22:24:50.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.064+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc415a210 con 0x7fcfc40d11c0 2023-07-13T22:24:50.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.065+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc415a430 con 0x7fcfc40d11c0 2023-07-13T22:24:50.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.066+0000 7fcfc0ff9700 1 -- 172.21.15.60:0/4290504457 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcfb80075c0 con 0x7fcfc40d11c0 2023-07-13T22:24:50.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.066+0000 7fcfc0ff9700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcfb006ace0 0x7fcfb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:50.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.067+0000 7fcfc0ff9700 1 -- 172.21.15.60:0/4290504457 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fcfb8088a60 con 0x7fcfc40d11c0 2023-07-13T22:24:50.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.067+0000 7fcfc3fff700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcfb006ace0 0x7fcfb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:50.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.067+0000 7fcfc3fff700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcfb006ace0 0x7fcfb006d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcfc415baf0 tx=0x7fcfb4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:50.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.071+0000 7fcfc0ff9700 1 -- 172.21.15.60:0/4290504457 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcfb8053980 con 0x7fcfc40d11c0 2023-07-13T22:24:50.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.321+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcfc404e210 con 0x7fcfc40d11c0 2023-07-13T22:24:50.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.816+0000 7fcfc0ff9700 1 -- 172.21.15.60:0/4290504457 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcfb8056fa0 con 0x7fcfc40d11c0 2023-07-13T22:24:50.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcfb006ace0 msgr2=0x7fcfb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:50.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcfb006ace0 0x7fcfb006d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcfc415baf0 tx=0x7fcfb4009210 comp rx=0 tx=0).stop 2023-07-13T22:24:50.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40d11c0 msgr2=0x7fcfc415b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:50.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40d11c0 0x7fcfc415b4c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800c700 tx=0x7fcfb800ca10 comp rx=0 tx=0).stop 2023-07-13T22:24:50.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 shutdown_connections 2023-07-13T22:24:50.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcfb006ace0 0x7fcfb006d190 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:50.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcfc4157590 0x7fcfc415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:50.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcfc40cfbc0 0x7fcfc4156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:50.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 --2- 172.21.15.60:0/4290504457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfc40d11c0 0x7fcfc415b4c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:50.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 >> 172.21.15.60:0/4290504457 conn(0x7fcfc406b400 msgr2=0x7fcfc40d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:50.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 shutdown_connections 2023-07-13T22:24:50.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:50.820+0000 7fcfca5ff700 1 -- 172.21.15.60:0/4290504457 wait complete. 2023-07-13T22:24:50.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:24:51.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.053+0000 7fa6987ff700 1 Processor -- start 2023-07-13T22:24:51.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.053+0000 7fa6987ff700 1 -- start start 2023-07-13T22:24:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.054+0000 7fa6987ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa6900d0890 0x7fa6900d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.054+0000 7fa6987ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6900cdb20 con 0x7fa6900cd6d0 2023-07-13T22:24:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.054+0000 7fa6987ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69015d200 con 0x7fa690159ea0 2023-07-13T22:24:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.054+0000 7fa6987ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6900d2e60 con 0x7fa6900d0890 2023-07-13T22:24:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.054+0000 7fa696d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa6900d0890 0x7fa6900d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.054+0000 7fa695d9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa6900cd6d0 0x7fa6900d3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59924/0 (socket says 172.21.15.60:59924) 2023-07-13T22:24:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.054+0000 7fa695d9a700 1 -- 172.21.15.60:0/3921925888 learned_addr learned my addr 172.21.15.60:0/3921925888 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:51.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.055+0000 7fa696d9c700 1 -- 172.21.15.60:0/3921925888 >> v1:172.21.15.105:6789/0 conn(0x7fa690159ea0 legacy=0x7fa69015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:51.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.055+0000 7fa696d9c700 1 -- 172.21.15.60:0/3921925888 >> v1:172.21.15.60:6789/0 conn(0x7fa6900cd6d0 legacy=0x7fa6900d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:51.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.055+0000 7fa696d9c700 1 -- 172.21.15.60:0/3921925888 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69015d4e0 con 0x7fa6900d0890 2023-07-13T22:24:51.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.055+0000 7fa695599700 1 -- 172.21.15.60:0/3921925888 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025188134 0 0) 0x7fa6900cdb20 con 0x7fa6900cd6d0 2023-07-13T22:24:51.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.055+0000 7fa696d9c700 1 --2- 172.21.15.60:0/3921925888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 0x7fa6900d0cc0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa688009d90 tx=0x7fa68800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5439382ba049dc10 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:51.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.055+0000 7fa695599700 1 -- 172.21.15.60:0/3921925888 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800e070 con 0x7fa6900d0890 2023-07-13T22:24:51.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.055+0000 7fa695599700 1 -- 172.21.15.60:0/3921925888 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68800bbd0 con 0x7fa6900d0890 2023-07-13T22:24:51.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.055+0000 7fa695599700 1 -- 172.21.15.60:0/3921925888 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688012430 con 0x7fa6900d0890 2023-07-13T22:24:51.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.056+0000 7fa6987ff700 1 -- 172.21.15.60:0/3921925888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 msgr2=0x7fa6900d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.056+0000 7fa6987ff700 1 --2- 172.21.15.60:0/3921925888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 0x7fa6900d0cc0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa688009d90 tx=0x7fa68800b330 comp rx=0 tx=0).stop 2023-07-13T22:24:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.056+0000 7fa6987ff700 1 -- 172.21.15.60:0/3921925888 shutdown_connections 2023-07-13T22:24:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.056+0000 7fa6987ff700 1 --2- 172.21.15.60:0/3921925888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 0x7fa6900d0cc0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.056+0000 7fa6987ff700 1 -- 172.21.15.60:0/3921925888 >> 172.21.15.60:0/3921925888 conn(0x7fa690071600 msgr2=0x7fa6900719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.056+0000 7fa6987ff700 1 -- 172.21.15.60:0/3921925888 shutdown_connections 2023-07-13T22:24:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.056+0000 7fa6987ff700 1 -- 172.21.15.60:0/3921925888 wait complete. 2023-07-13T22:24:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.058+0000 7fa6987ff700 1 Processor -- start 2023-07-13T22:24:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.058+0000 7fa6987ff700 1 -- start start 2023-07-13T22:24:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa6987ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6900cd6d0 0x7fa6901f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa6987ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 0x7fa6900cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa6987ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa690159ea0 0x7fa6900b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa6987ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6900b8680 con 0x7fa6900d0890 2023-07-13T22:24:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa6987ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6901f6230 con 0x7fa690159ea0 2023-07-13T22:24:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa6987ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6900b5f90 con 0x7fa6900cd6d0 2023-07-13T22:24:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa695d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 0x7fa6900cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa695d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 0x7fa6900cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47680/0 (socket says 172.21.15.60:47680) 2023-07-13T22:24:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa695d9a700 1 -- 172.21.15.60:0/4161918968 learned_addr learned my addr 172.21.15.60:0/4161918968 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa69659b700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6900cd6d0 0x7fa6901f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa696d9c700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa690159ea0 0x7fa6900b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.059+0000 7fa695d9a700 1 -- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6900cd6d0 msgr2=0x7fa6901f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa695d9a700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6900cd6d0 0x7fa6901f5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa695d9a700 1 -- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa690159ea0 msgr2=0x7fa6900b7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa695d9a700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa690159ea0 0x7fa6900b7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa695d9a700 1 -- 172.21.15.60:0/4161918968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688009a70 con 0x7fa6900d0890 2023-07-13T22:24:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa69659b700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6900cd6d0 0x7fa6901f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa696d9c700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa690159ea0 0x7fa6900b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa695d9a700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 0x7fa6900cc000 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c520 tx=0x7fa68000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:51.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa686ffd700 1 -- 172.21.15.60:0/4161918968 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680004c80 con 0x7fa6900d0890 2023-07-13T22:24:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa686ffd700 1 -- 172.21.15.60:0/4161918968 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa680004e00 con 0x7fa6900d0890 2023-07-13T22:24:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa686ffd700 1 -- 172.21.15.60:0/4161918968 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680007310 con 0x7fa6900d0890 2023-07-13T22:24:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6900b6270 con 0x7fa6900d0890 2023-07-13T22:24:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.060+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6900b6830 con 0x7fa6900d0890 2023-07-13T22:24:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.061+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa690067a70 con 0x7fa6900d0890 2023-07-13T22:24:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.062+0000 7fa686ffd700 1 -- 172.21.15.60:0/4161918968 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa680007470 con 0x7fa6900d0890 2023-07-13T22:24:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.063+0000 7fa686ffd700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa67406ace0 0x7fa67406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.063+0000 7fa686ffd700 1 -- 172.21.15.60:0/4161918968 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fa680089300 con 0x7fa6900d0890 2023-07-13T22:24:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.063+0000 7fa69659b700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa67406ace0 0x7fa67406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:51.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.064+0000 7fa69659b700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa67406ace0 0x7fa67406d190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa68c000f60 tx=0x7fa68c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:51.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.067+0000 7fa686ffd700 1 -- 172.21.15.60:0/4161918968 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa6800542b0 con 0x7fa6900d0890 2023-07-13T22:24:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.278+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa69004e210 con 0x7fa6900d0890 2023-07-13T22:24:51.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.822+0000 7fa686ffd700 1 -- 172.21.15.60:0/4161918968 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa6800578d0 con 0x7fa6900d0890 2023-07-13T22:24:51.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.826+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa67406ace0 msgr2=0x7fa67406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:51.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.826+0000 7fa6987ff700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa67406ace0 0x7fa67406d190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa68c000f60 tx=0x7fa68c009040 comp rx=0 tx=0).stop 2023-07-13T22:24:51.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.826+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 msgr2=0x7fa6900cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:51.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.826+0000 7fa6987ff700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 0x7fa6900cc000 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c520 tx=0x7fa68000c8e0 comp rx=0 tx=0).stop 2023-07-13T22:24:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.827+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 shutdown_connections 2023-07-13T22:24:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.827+0000 7fa6987ff700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa67406ace0 0x7fa67406d190 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.827+0000 7fa6987ff700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa690159ea0 0x7fa6900b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.827+0000 7fa6987ff700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6900cd6d0 0x7fa6901f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.827+0000 7fa6987ff700 1 --2- 172.21.15.60:0/4161918968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6900d0890 0x7fa6900cc000 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.827+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 >> 172.21.15.60:0/4161918968 conn(0x7fa690071600 msgr2=0x7fa69006b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.827+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 shutdown_connections 2023-07-13T22:24:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:51.827+0000 7fa6987ff700 1 -- 172.21.15.60:0/4161918968 wait complete. 2023-07-13T22:24:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:24:51.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:24:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.059+0000 7f0473dff700 1 Processor -- start 2023-07-13T22:24:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.060+0000 7f0473dff700 1 -- start start 2023-07-13T22:24:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.061+0000 7f0473dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f046c0b1990 0x7f046c1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.061+0000 7f0473dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046c069ed0 con 0x7f046c15c320 2023-07-13T22:24:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.061+0000 7f0473dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046c0cd7b0 con 0x7f046c06a990 2023-07-13T22:24:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.061+0000 7f0473dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046c0cda90 con 0x7f046c0b1990 2023-07-13T22:24:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.061+0000 7f0471b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f046c15c320 0x7f046c0b1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59936/0 (socket says 172.21.15.60:59936) 2023-07-13T22:24:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.061+0000 7f0471b9b700 1 -- 172.21.15.60:0/2031622168 learned_addr learned my addr 172.21.15.60:0/2031622168 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.061+0000 7f047139a700 1 --2- 172.21.15.60:0/2031622168 >> v2:172.21.15.105:3300/0 conn(0x7f046c0b1990 0x7f046c1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.062+0000 7f0470b99700 1 -- 172.21.15.60:0/2031622168 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215718627 0 0) 0x7f046c069ed0 con 0x7f046c15c320 2023-07-13T22:24:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.062+0000 7f0470b99700 1 -- 172.21.15.60:0/2031622168 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04580035b0 con 0x7f046c15c320 2023-07-13T22:24:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.062+0000 7f0470b99700 1 -- 172.21.15.60:0/2031622168 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990306098 0 0) 0x7f046c0cd7b0 con 0x7f046c06a990 2023-07-13T22:24:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.062+0000 7f0470b99700 1 -- 172.21.15.60:0/2031622168 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f046c069ed0 con 0x7f046c06a990 2023-07-13T22:24:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.062+0000 7f047139a700 1 -- 172.21.15.60:0/2031622168 >> v1:172.21.15.105:6789/0 conn(0x7f046c06a990 legacy=0x7f046c06add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.062+0000 7f047139a700 1 -- 172.21.15.60:0/2031622168 >> v1:172.21.15.60:6789/0 conn(0x7f046c15c320 legacy=0x7f046c0b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.062+0000 7f047139a700 1 -- 172.21.15.60:0/2031622168 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c06a190 con 0x7f046c0b1990 2023-07-13T22:24:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.062+0000 7f047139a700 1 --2- 172.21.15.60:0/2031622168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c0b1990 0x7f046c1609b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f045c009840 tx=0x7f045c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2312a83b2f7b7954 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.062+0000 7f0470b99700 1 -- 172.21.15.60:0/2031622168 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c011830 con 0x7f046c0b1990 2023-07-13T22:24:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.063+0000 7f0470b99700 1 -- 172.21.15.60:0/2031622168 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045c011990 con 0x7f046c0b1990 2023-07-13T22:24:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.063+0000 7f0470b99700 1 -- 172.21.15.60:0/2031622168 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c011d20 con 0x7f046c0b1990 2023-07-13T22:24:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.063+0000 7f0473dff700 1 -- 172.21.15.60:0/2031622168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c0b1990 msgr2=0x7f046c1609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.063+0000 7f0473dff700 1 --2- 172.21.15.60:0/2031622168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c0b1990 0x7f046c1609b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f045c009840 tx=0x7f045c009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:52.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.063+0000 7f0473dff700 1 -- 172.21.15.60:0/2031622168 shutdown_connections 2023-07-13T22:24:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.063+0000 7f0473dff700 1 --2- 172.21.15.60:0/2031622168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c0b1990 0x7f046c1609b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.063+0000 7f0473dff700 1 -- 172.21.15.60:0/2031622168 >> 172.21.15.60:0/2031622168 conn(0x7f046c0cc280 msgr2=0x7f046c0cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.063+0000 7f0473dff700 1 -- 172.21.15.60:0/2031622168 shutdown_connections 2023-07-13T22:24:52.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.063+0000 7f0473dff700 1 -- 172.21.15.60:0/2031622168 wait complete. 2023-07-13T22:24:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.065+0000 7f0473dff700 1 Processor -- start 2023-07-13T22:24:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.065+0000 7f0473dff700 1 -- start start 2023-07-13T22:24:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.066+0000 7f0473dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f046c06a990 0x7f046c14cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.066+0000 7f0473dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f046c0b1990 0x7f046c0b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.066+0000 7f0473dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c15c320 0x7f046c0c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.066+0000 7f0473dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c14d1b0 con 0x7f046c06a990 2023-07-13T22:24:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.066+0000 7f0473dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c0b51e0 con 0x7f046c15c320 2023-07-13T22:24:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.066+0000 7f0473dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046c0b4140 con 0x7f046c0b1990 2023-07-13T22:24:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.066+0000 7f0471b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f046c06a990 0x7f046c14cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:52.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.066+0000 7f0471b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f046c06a990 0x7f046c14cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47688/0 (socket says 172.21.15.60:47688) 2023-07-13T22:24:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.066+0000 7f0471b9b700 1 -- 172.21.15.60:0/685050037 learned_addr learned my addr 172.21.15.60:0/685050037 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f047139a700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f046c0b1990 0x7f046c0b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f047239c700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c15c320 0x7f046c0c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f0471b9b700 1 -- 172.21.15.60:0/685050037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f046c0b1990 msgr2=0x7f046c0b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f0471b9b700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f046c0b1990 0x7f046c0b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:52.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f0471b9b700 1 -- 172.21.15.60:0/685050037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c15c320 msgr2=0x7f046c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f0471b9b700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c15c320 0x7f046c0c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f0471b9b700 1 -- 172.21.15.60:0/685050037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c0094a0 con 0x7f046c06a990 2023-07-13T22:24:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f047239c700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c15c320 0x7f046c0c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:52.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f0471b9b700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f046c06a990 0x7f046c14cc70 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0468001eb0 tx=0x7f046800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:52.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f04627fc700 1 -- 172.21.15.60:0/685050037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468005d70 con 0x7f046c06a990 2023-07-13T22:24:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f04627fc700 1 -- 172.21.15.60:0/685050037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04680070d0 con 0x7f046c06a990 2023-07-13T22:24:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046c0b4420 con 0x7f046c06a990 2023-07-13T22:24:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f04627fc700 1 -- 172.21.15.60:0/685050037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468007460 con 0x7f046c06a990 2023-07-13T22:24:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.067+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046c0b49e0 con 0x7f046c06a990 2023-07-13T22:24:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.068+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046c04e210 con 0x7f046c06a990 2023-07-13T22:24:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.069+0000 7f04627fc700 1 -- 172.21.15.60:0/685050037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f04680075c0 con 0x7f046c06a990 2023-07-13T22:24:52.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.070+0000 7f04627fc700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006ace0 0x7f045006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.070+0000 7f04627fc700 1 -- 172.21.15.60:0/685050037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5826+0+0 (secure 0 0 0) 0x7f0468088920 con 0x7f046c06a990 2023-07-13T22:24:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.071+0000 7f047139a700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006ace0 0x7f045006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:52.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.071+0000 7f047139a700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006ace0 0x7f045006d190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f046c071410 tx=0x7f045c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:52.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.075+0000 7f04627fc700 1 -- 172.21.15.60:0/685050037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0468053a10 con 0x7f046c06a990 2023-07-13T22:24:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.280+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f046c0b5030 con 0x7f046c06a990 2023-07-13T22:24:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.824+0000 7f04627fc700 1 -- 172.21.15.60:0/685050037 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0468057030 con 0x7f046c06a990 2023-07-13T22:24:52.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.828+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006ace0 msgr2=0x7f045006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.828+0000 7f0473dff700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006ace0 0x7f045006d190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f046c071410 tx=0x7f045c01c040 comp rx=0 tx=0).stop 2023-07-13T22:24:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.828+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f046c06a990 msgr2=0x7f046c14cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.828+0000 7f0473dff700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f046c06a990 0x7f046c14cc70 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0468001eb0 tx=0x7f046800c7b0 comp rx=0 tx=0).stop 2023-07-13T22:24:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.829+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 shutdown_connections 2023-07-13T22:24:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.829+0000 7f0473dff700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f045006ace0 0x7f045006d190 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.829+0000 7f0473dff700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f046c06a990 0x7f046c14cc70 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.829+0000 7f0473dff700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f046c15c320 0x7f046c0c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.829+0000 7f0473dff700 1 --2- 172.21.15.60:0/685050037 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f046c0b1990 0x7f046c0b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.829+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 >> 172.21.15.60:0/685050037 conn(0x7f046c0cc280 msgr2=0x7f046c1585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.829+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 shutdown_connections 2023-07-13T22:24:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:52.829+0000 7f0473dff700 1 -- 172.21.15.60:0/685050037 wait complete. 2023-07-13T22:24:52.814 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:24:52.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2023-07-13T22:24:53.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.063+0000 7fdb193ff700 1 Processor -- start 2023-07-13T22:24:53.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.063+0000 7fdb193ff700 1 -- start start 2023-07-13T22:24:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.064+0000 7fdb193ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb1415ccc0 0x7fdb140ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.064+0000 7fdb193ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb140b0f60 con 0x7fdb1415d090 2023-07-13T22:24:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.064+0000 7fdb193ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb140cae40 con 0x7fdb14159310 2023-07-13T22:24:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.064+0000 7fdb193ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140ae760 con 0x7fdb1415ccc0 2023-07-13T22:24:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.064+0000 7fdb127fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdb1415d090 0x7fdb1415d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59950/0 (socket says 172.21.15.60:59950) 2023-07-13T22:24:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.064+0000 7fdb127fc700 1 -- 172.21.15.60:0/2026213455 learned_addr learned my addr 172.21.15.60:0/2026213455 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:53.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb12ffd700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb1415ccc0 0x7fdb140ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697240431 0 0) 0x7fdb140b0f60 con 0x7fdb1415d090 2023-07-13T22:24:53.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdafc0035b0 con 0x7fdb1415d090 2023-07-13T22:24:53.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049197945 0 0) 0x7fdb140cae40 con 0x7fdb14159310 2023-07-13T22:24:53.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb140b0f60 con 0x7fdb14159310 2023-07-13T22:24:53.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2512196509 0 0) 0x7fdafc0035b0 con 0x7fdb1415d090 2023-07-13T22:24:53.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb1415ccc0 msgr2=0x7fdb140ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:53.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb1415ccc0 0x7fdb140ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 >> v1:172.21.15.105:6789/0 conn(0x7fdb14159310 legacy=0x7fdb14159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb140ca880 con 0x7fdb1415d090 2023-07-13T22:24:53.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fdb08001670 con 0x7fdb1415d090 2023-07-13T22:24:53.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 >> v1:172.21.15.60:6789/0 conn(0x7fdb1415d090 legacy=0x7fdb1415d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdafc003bc0 0x7fdafc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.065+0000 7fdb11ffb700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb1415ccc0 0x7fdafc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb11ffb700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc004d90 0x7fdafc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140ae760 con 0x7fdafc003bc0 2023-07-13T22:24:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc00dd20 con 0x7fdafc004d90 2023-07-13T22:24:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc00dee0 con 0x7fdb1415ccc0 2023-07-13T22:24:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb12ffd700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdafc003bc0 0x7fdafc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb127fc700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb1415ccc0 0x7fdafc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb137fe700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc004d90 0x7fdafc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb12ffd700 1 -- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb1415ccc0 msgr2=0x7fdafc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb12ffd700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb1415ccc0 0x7fdafc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb12ffd700 1 -- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc004d90 msgr2=0x7fdafc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb12ffd700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc004d90 0x7fdafc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.066+0000 7fdb12ffd700 1 -- 172.21.15.60:0/2026213455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc0035b0 con 0x7fdafc003bc0 2023-07-13T22:24:53.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb12ffd700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdafc003bc0 0x7fdafc003fb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009b60 tx=0x7fdafc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fb6b926cbac1ba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:53.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400e6c0 con 0x7fdafc003bc0 2023-07-13T22:24:53.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0400e820 con 0x7fdafc003bc0 2023-07-13T22:24:53.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb11ffb700 1 -- 172.21.15.60:0/2026213455 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400eb90 con 0x7fdafc003bc0 2023-07-13T22:24:53.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb193ff700 1 -- 172.21.15.60:0/2026213455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdafc003bc0 msgr2=0x7fdafc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb193ff700 1 --2- 172.21.15.60:0/2026213455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdafc003bc0 0x7fdafc003fb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009b60 tx=0x7fdafc005880 comp rx=0 tx=0).stop 2023-07-13T22:24:53.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb137fe700 1 -- 172.21.15.60:0/2026213455 reap_dead start 2023-07-13T22:24:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb193ff700 1 -- 172.21.15.60:0/2026213455 shutdown_connections 2023-07-13T22:24:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb193ff700 1 -- 172.21.15.60:0/2026213455 >> 172.21.15.60:0/2026213455 conn(0x7fdb140cf340 msgr2=0x7fdb140cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb193ff700 1 -- 172.21.15.60:0/2026213455 shutdown_connections 2023-07-13T22:24:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.067+0000 7fdb193ff700 1 -- 172.21.15.60:0/2026213455 wait complete. 2023-07-13T22:24:53.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.069+0000 7fdb193ff700 1 Processor -- start 2023-07-13T22:24:53.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.069+0000 7fdb193ff700 1 -- start start 2023-07-13T22:24:53.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.070+0000 7fdb193ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc003bc0 0x7fdb14143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.070+0000 7fdb193ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb14151a30 0x7fdb140b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.070+0000 7fdb193ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb140b2bf0 0x7fdb140afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.070+0000 7fdb193ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140b36f0 con 0x7fdb140b2bf0 2023-07-13T22:24:53.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.070+0000 7fdb193ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb14150fe0 con 0x7fdafc003bc0 2023-07-13T22:24:53.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.070+0000 7fdb193ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb141515a0 con 0x7fdb14151a30 2023-07-13T22:24:53.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.070+0000 7fdb137fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb140b2bf0 0x7fdb140afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.070+0000 7fdb137fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb140b2bf0 0x7fdb140afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47702/0 (socket says 172.21.15.60:47702) 2023-07-13T22:24:53.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb137fe700 1 -- 172.21.15.60:0/2506342409 learned_addr learned my addr 172.21.15.60:0/2506342409 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:53.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb12ffd700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc003bc0 0x7fdb14143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb127fc700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb14151a30 0x7fdb140b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb137fe700 1 -- 172.21.15.60:0/2506342409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb14151a30 msgr2=0x7fdb140b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb137fe700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb14151a30 0x7fdb140b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb137fe700 1 -- 172.21.15.60:0/2506342409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc003bc0 msgr2=0x7fdb14143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb137fe700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc003bc0 0x7fdb14143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb137fe700 1 -- 172.21.15.60:0/2506342409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb140b3130 con 0x7fdb140b2bf0 2023-07-13T22:24:53.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb137fe700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb140b2bf0 0x7fdb140afae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdb140af2d0 tx=0x7fdb0c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdafb7fe700 1 -- 172.21.15.60:0/2506342409 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c003c10 con 0x7fdb140b2bf0 2023-07-13T22:24:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdafb7fe700 1 -- 172.21.15.60:0/2506342409 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c003d70 con 0x7fdb140b2bf0 2023-07-13T22:24:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb140b3410 con 0x7fdb140b2bf0 2023-07-13T22:24:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.071+0000 7fdafb7fe700 1 -- 172.21.15.60:0/2506342409 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00dac0 con 0x7fdb140b2bf0 2023-07-13T22:24:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.072+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb14150740 con 0x7fdb140b2bf0 2023-07-13T22:24:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.073+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1404e210 con 0x7fdb140b2bf0 2023-07-13T22:24:53.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.073+0000 7fdafb7fe700 1 -- 172.21.15.60:0/2506342409 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb0c00dc20 con 0x7fdb140b2bf0 2023-07-13T22:24:53.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.075+0000 7fdafb7fe700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0006ace0 0x7fdb0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.075+0000 7fdafb7fe700 1 -- 172.21.15.60:0/2506342409 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdb0c0893b0 con 0x7fdb140b2bf0 2023-07-13T22:24:53.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.075+0000 7fdb12ffd700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0006ace0 0x7fdb0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.076+0000 7fdb12ffd700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0006ace0 0x7fdb0006d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdb040090b0 tx=0x7fdb0401f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:53.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.079+0000 7fdafb7fe700 1 -- 172.21.15.60:0/2506342409 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdb0c0546e0 con 0x7fdb140b2bf0 2023-07-13T22:24:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.298+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) v1 -- 0x7fdb140ae540 con 0x7fdb140b2bf0 2023-07-13T22:24:53.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.302+0000 7fdafb7fe700 1 -- 172.21.15.60:0/2506342409 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v20) v1 ==== 106+0+0 (secure 0 0 0) 0x7fdb0c057d00 con 0x7fdb140b2bf0 2023-07-13T22:24:53.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0006ace0 msgr2=0x7fdb0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0006ace0 0x7fdb0006d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdb040090b0 tx=0x7fdb0401f040 comp rx=0 tx=0).stop 2023-07-13T22:24:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb140b2bf0 msgr2=0x7fdb140afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb140b2bf0 0x7fdb140afae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdb140af2d0 tx=0x7fdb0c00d430 comp rx=0 tx=0).stop 2023-07-13T22:24:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 shutdown_connections 2023-07-13T22:24:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0006ace0 0x7fdb0006d190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb140b2bf0 0x7fdb140afae0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdafc003bc0 0x7fdb14143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 --2- 172.21.15.60:0/2506342409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb14151a30 0x7fdb140b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 >> 172.21.15.60:0/2506342409 conn(0x7fdb140cf340 msgr2=0x7fdb140c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 shutdown_connections 2023-07-13T22:24:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.305+0000 7fdb193ff700 1 -- 172.21.15.60:0/2506342409 wait complete. 2023-07-13T22:24:53.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-13T22:24:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.529+0000 7f28e27ff700 1 Processor -- start 2023-07-13T22:24:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.529+0000 7f28e27ff700 1 -- start start 2023-07-13T22:24:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.530+0000 7f28e27ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28dc160970 0x7f28dc0cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.530+0000 7f28e27ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc0d1040 con 0x7f28dc0d14d0 2023-07-13T22:24:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.530+0000 7f28e27ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc160de0 con 0x7f28dc0b45b0 2023-07-13T22:24:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.530+0000 7f28e27ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28dc0cd630 con 0x7f28dc160970 2023-07-13T22:24:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.530+0000 7f28db7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28dc160970 0x7f28dc0cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.531+0000 7f28e0d9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f28dc0d14d0 0x7f28dc0d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59964/0 (socket says 172.21.15.60:59964) 2023-07-13T22:24:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.531+0000 7f28db7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28dc160970 0x7f28dc0cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47704/0 (socket says 172.21.15.60:47704) 2023-07-13T22:24:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.531+0000 7f28e0d9c700 1 -- 172.21.15.60:0/3584462375 learned_addr learned my addr 172.21.15.60:0/3584462375 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.531+0000 7f28db7fe700 1 -- 172.21.15.60:0/3584462375 >> v1:172.21.15.105:6789/0 conn(0x7f28dc0b45b0 legacy=0x7f28dc0b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.531+0000 7f28db7fe700 1 -- 172.21.15.60:0/3584462375 >> v1:172.21.15.60:6789/0 conn(0x7f28dc0d14d0 legacy=0x7f28dc0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.531+0000 7f28db7fe700 1 -- 172.21.15.60:0/3584462375 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc0cd840 con 0x7f28dc160970 2023-07-13T22:24:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.531+0000 7f28db7fe700 1 --2- 172.21.15.60:0/3584462375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 0x7f28dc0cd0f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009840 tx=0x7f28d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f909efde141036 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:53.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.531+0000 7f28daffd700 1 -- 172.21.15.60:0/3584462375 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0011830 con 0x7f28dc160970 2023-07-13T22:24:53.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.531+0000 7f28daffd700 1 -- 172.21.15.60:0/3584462375 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d0011990 con 0x7f28dc160970 2023-07-13T22:24:53.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.532+0000 7f28daffd700 1 -- 172.21.15.60:0/3584462375 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0011d20 con 0x7f28dc160970 2023-07-13T22:24:53.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.532+0000 7f28e27ff700 1 -- 172.21.15.60:0/3584462375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 msgr2=0x7f28dc0cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.532+0000 7f28e27ff700 1 --2- 172.21.15.60:0/3584462375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 0x7f28dc0cd0f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009840 tx=0x7f28d0009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:53.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.532+0000 7f28e27ff700 1 -- 172.21.15.60:0/3584462375 shutdown_connections 2023-07-13T22:24:53.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.532+0000 7f28e27ff700 1 --2- 172.21.15.60:0/3584462375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 0x7f28dc0cd0f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.532+0000 7f28e27ff700 1 -- 172.21.15.60:0/3584462375 >> 172.21.15.60:0/3584462375 conn(0x7f28dc072240 msgr2=0x7f28dc0d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:53.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.532+0000 7f28e27ff700 1 -- 172.21.15.60:0/3584462375 shutdown_connections 2023-07-13T22:24:53.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.532+0000 7f28e27ff700 1 -- 172.21.15.60:0/3584462375 wait complete. 2023-07-13T22:24:53.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.534+0000 7f28e27ff700 1 Processor -- start 2023-07-13T22:24:53.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.534+0000 7f28e27ff700 1 -- start start 2023-07-13T22:24:53.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.535+0000 7f28e27ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28dc0b45b0 0x7f28dc0c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.535+0000 7f28e27ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28dc0d14d0 0x7f28dc0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.535+0000 7f28e27ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 0x7f28dc0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.535+0000 7f28e27ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0cbd50 con 0x7f28dc160970 2023-07-13T22:24:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.535+0000 7f28e27ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0c6cb0 con 0x7f28dc0b45b0 2023-07-13T22:24:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.535+0000 7f28e27ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0c6f90 con 0x7f28dc0d14d0 2023-07-13T22:24:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.535+0000 7f28e0d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 0x7f28dc0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.535+0000 7f28e0d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 0x7f28dc0c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47712/0 (socket says 172.21.15.60:47712) 2023-07-13T22:24:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.535+0000 7f28e0d9c700 1 -- 172.21.15.60:0/1483020322 learned_addr learned my addr 172.21.15.60:0/1483020322 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28dbfff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28dc0b45b0 0x7f28dc0c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28e0d9c700 1 -- 172.21.15.60:0/1483020322 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28dc0d14d0 msgr2=0x7f28dc0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28e0d9c700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28dc0d14d0 0x7f28dc0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28e0d9c700 1 -- 172.21.15.60:0/1483020322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28dc0b45b0 msgr2=0x7f28dc0c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28e0d9c700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28dc0b45b0 0x7f28dc0c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28e0d9c700 1 -- 172.21.15.60:0/1483020322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d00094a0 con 0x7f28dc160970 2023-07-13T22:24:53.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28dbfff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28dc0b45b0 0x7f28dc0c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:53.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28e0d9c700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 0x7f28dc0c7d50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f28d400cbc0 tx=0x7f28d400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:53.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28d8ff9700 1 -- 172.21.15.60:0/1483020322 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d400ab50 con 0x7f28dc160970 2023-07-13T22:24:53.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28d8ff9700 1 -- 172.21.15.60:0/1483020322 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d400acb0 con 0x7f28dc160970 2023-07-13T22:24:53.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28d8ff9700 1 -- 172.21.15.60:0/1483020322 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4012600 con 0x7f28dc160970 2023-07-13T22:24:53.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc0c7270 con 0x7f28dc160970 2023-07-13T22:24:53.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.536+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc0c7830 con 0x7f28dc160970 2023-07-13T22:24:53.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.537+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc0c7a20 con 0x7f28dc160970 2023-07-13T22:24:53.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.538+0000 7f28d8ff9700 1 -- 172.21.15.60:0/1483020322 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28d401b3b0 con 0x7f28dc160970 2023-07-13T22:24:53.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.539+0000 7f28d8ff9700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28c806ace0 0x7f28c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.539+0000 7f28d8ff9700 1 -- 172.21.15.60:0/1483020322 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f28d4088df0 con 0x7f28dc160970 2023-07-13T22:24:53.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.540+0000 7f28dbfff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28c806ace0 0x7f28c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.540+0000 7f28dbfff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28c806ace0 0x7f28c806d190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000f60 tx=0x7f28cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:53.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.544+0000 7f28d8ff9700 1 -- 172.21.15.60:0/1483020322 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f28d4054260 con 0x7f28dc160970 2023-07-13T22:24:53.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.748+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f28dc04e210 con 0x7f28dc160970 2023-07-13T22:24:53.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.752+0000 7f28d8ff9700 1 -- 172.21.15.60:0/1483020322 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f28d4057880 con 0x7f28dc160970 2023-07-13T22:24:53.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28c806ace0 msgr2=0x7f28c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28c806ace0 0x7f28c806d190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000f60 tx=0x7f28cc009040 comp rx=0 tx=0).stop 2023-07-13T22:24:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 msgr2=0x7f28dc0c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 0x7f28dc0c7d50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f28d400cbc0 tx=0x7f28d400cf80 comp rx=0 tx=0).stop 2023-07-13T22:24:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 shutdown_connections 2023-07-13T22:24:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28c806ace0 0x7f28c806d190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc160970 0x7f28dc0c7d50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28dc0b45b0 0x7f28dc0c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 --2- 172.21.15.60:0/1483020322 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28dc0d14d0 0x7f28dc0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 >> 172.21.15.60:0/1483020322 conn(0x7f28dc072240 msgr2=0x7f28dc0d18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 shutdown_connections 2023-07-13T22:24:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.755+0000 7f28e27ff700 1 -- 172.21.15.60:0/1483020322 wait complete. 2023-07-13T22:24:53.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2023-07-13T22:24:53.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.976+0000 7febfd1ff700 1 Processor -- start 2023-07-13T22:24:53.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.976+0000 7febfd1ff700 1 -- start start 2023-07-13T22:24:53.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febfd1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7febf80d0a50 0x7febf80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febfd1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0e20 0x7febf80d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febfd1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf80d2410 con 0x7febf80cd260 2023-07-13T22:24:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febfd1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf815c900 con 0x7febf80d0a50 2023-07-13T22:24:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febfd1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf80d2660 con 0x7febf80d0e20 2023-07-13T22:24:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febf6d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7febf80d0a50 0x7febf80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febf6d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7febf80d0a50 0x7febf80cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47728/0 (socket says 172.21.15.60:47728) 2023-07-13T22:24:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febf6d9d700 1 -- 172.21.15.60:0/3938471846 learned_addr learned my addr 172.21.15.60:0/3938471846 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febf759e700 1 --2- 172.21.15.60:0/3938471846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0e20 0x7febf80d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febf6d9d700 1 -- 172.21.15.60:0/3938471846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0e20 msgr2=0x7febf80d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febf6d9d700 1 --2- 172.21.15.60:0/3938471846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0e20 0x7febf80d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febf6d9d700 1 -- 172.21.15.60:0/3938471846 >> v1:172.21.15.105:6789/0 conn(0x7febf80cd260 legacy=0x7febf80d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.978+0000 7febf6d9d700 1 -- 172.21.15.60:0/3938471846 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf80d2940 con 0x7febf80d0a50 2023-07-13T22:24:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.979+0000 7febf6d9d700 1 --2- 172.21.15.60:0/3938471846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0a50 0x7febf80cf8a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7febe80098f0 tx=0x7febe8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7089dcdd141d714 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.979+0000 7febf5d9b700 1 -- 172.21.15.60:0/3938471846 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800b8a0 con 0x7febf80d0a50 2023-07-13T22:24:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.979+0000 7febf5d9b700 1 -- 172.21.15.60:0/3938471846 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe800ba00 con 0x7febf80d0a50 2023-07-13T22:24:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.979+0000 7febf5d9b700 1 -- 172.21.15.60:0/3938471846 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800bd90 con 0x7febf80d0a50 2023-07-13T22:24:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.979+0000 7febfd1ff700 1 -- 172.21.15.60:0/3938471846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0a50 msgr2=0x7febf80cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.979+0000 7febfd1ff700 1 --2- 172.21.15.60:0/3938471846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0a50 0x7febf80cf8a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7febe80098f0 tx=0x7febe8009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.980+0000 7febfd1ff700 1 -- 172.21.15.60:0/3938471846 shutdown_connections 2023-07-13T22:24:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.980+0000 7febfd1ff700 1 --2- 172.21.15.60:0/3938471846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0e20 0x7febf80d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.980+0000 7febfd1ff700 1 --2- 172.21.15.60:0/3938471846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0a50 0x7febf80cf8a0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.980+0000 7febfd1ff700 1 -- 172.21.15.60:0/3938471846 >> 172.21.15.60:0/3938471846 conn(0x7febf8071600 msgr2=0x7febf80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.980+0000 7febfd1ff700 1 -- 172.21.15.60:0/3938471846 shutdown_connections 2023-07-13T22:24:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.980+0000 7febfd1ff700 1 -- 172.21.15.60:0/3938471846 wait complete. 2023-07-13T22:24:53.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.982+0000 7febfd1ff700 1 Processor -- start 2023-07-13T22:24:53.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.982+0000 7febfd1ff700 1 -- start start 2023-07-13T22:24:53.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.983+0000 7febfd1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf80cd260 0x7febf81f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.983+0000 7febfd1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0a50 0x7febf80c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febfd1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0e20 0x7febf815af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febfd1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf80c75f0 con 0x7febf80d0e20 2023-07-13T22:24:53.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febfd1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf815b460 con 0x7febf80cd260 2023-07-13T22:24:53.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febfd1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf80c6700 con 0x7febf80d0a50 2023-07-13T22:24:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf759e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0e20 0x7febf815af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf759e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0e20 0x7febf815af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47742/0 (socket says 172.21.15.60:47742) 2023-07-13T22:24:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf6d9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf80cd260 0x7febf81f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf759e700 1 -- 172.21.15.60:0/2283230675 learned_addr learned my addr 172.21.15.60:0/2283230675 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf659c700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0a50 0x7febf80c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf759e700 1 -- 172.21.15.60:0/2283230675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0a50 msgr2=0x7febf80c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf759e700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0a50 0x7febf80c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf759e700 1 -- 172.21.15.60:0/2283230675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf80cd260 msgr2=0x7febf81f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf759e700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf80cd260 0x7febf81f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.984+0000 7febf759e700 1 -- 172.21.15.60:0/2283230675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8009550 con 0x7febf80d0e20 2023-07-13T22:24:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.985+0000 7febf759e700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0e20 0x7febf815af20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7febf0009fd0 tx=0x7febf000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.985+0000 7febdf7fe700 1 -- 172.21.15.60:0/2283230675 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0007ea0 con 0x7febf80d0e20 2023-07-13T22:24:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.985+0000 7febdf7fe700 1 -- 172.21.15.60:0/2283230675 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf000ecd0 con 0x7febf80d0e20 2023-07-13T22:24:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.985+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf80c69c0 con 0x7febf80d0e20 2023-07-13T22:24:53.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.985+0000 7febdf7fe700 1 -- 172.21.15.60:0/2283230675 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0014660 con 0x7febf80d0e20 2023-07-13T22:24:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.985+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf815a440 con 0x7febf80d0e20 2023-07-13T22:24:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.986+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf804e280 con 0x7febf80d0e20 2023-07-13T22:24:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.987+0000 7febdf7fe700 1 -- 172.21.15.60:0/2283230675 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7febf00147c0 con 0x7febf80d0e20 2023-07-13T22:24:53.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.988+0000 7febdf7fe700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7febe406ace0 0x7febe406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:53.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.988+0000 7febdf7fe700 1 -- 172.21.15.60:0/2283230675 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5296+0+0 (secure 0 0 0) 0x7febf00896e0 con 0x7febf80d0e20 2023-07-13T22:24:53.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.988+0000 7febf6d9d700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7febe406ace0 0x7febe406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:53.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.989+0000 7febf6d9d700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7febe406ace0 0x7febe406d190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7febe8009c40 tx=0x7febe8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:53.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:53.992+0000 7febdf7fe700 1 -- 172.21.15.60:0/2283230675 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7febf0058530 con 0x7febf80d0e20 2023-07-13T22:24:54.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.283+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) v1 -- 0x7febf815a220 con 0x7febf80d0e20 2023-07-13T22:24:54.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.831+0000 7febdf7fe700 1 -- 172.21.15.60:0/2283230675 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v226) v1 ==== 148+0+0 (secure 0 0 0) 0x7febf00580c0 con 0x7febf80d0e20 2023-07-13T22:24:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7febe406ace0 msgr2=0x7febe406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7febe406ace0 0x7febe406d190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7febe8009c40 tx=0x7febe8008030 comp rx=0 tx=0).stop 2023-07-13T22:24:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0e20 msgr2=0x7febf815af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0e20 0x7febf815af20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7febf0009fd0 tx=0x7febf000e3c0 comp rx=0 tx=0).stop 2023-07-13T22:24:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 shutdown_connections 2023-07-13T22:24:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7febe406ace0 0x7febe406d190 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf80d0e20 0x7febf815af20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf80cd260 0x7febf81f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 --2- 172.21.15.60:0/2283230675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febf80d0a50 0x7febf80c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 >> 172.21.15.60:0/2283230675 conn(0x7febf8071600 msgr2=0x7febf80733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 shutdown_connections 2023-07-13T22:24:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:54.834+0000 7febfd1ff700 1 -- 172.21.15.60:0/2283230675 wait complete. 2023-07-13T22:24:54.838 INFO:tasks.cephfs_test_runner:test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-13T22:24:54.839 INFO:tasks.cephfs_test_runner:Starting test: test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:24:54.839 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2023-07-13T22:24:55.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.065+0000 7f9f6a3ff700 1 Processor -- start 2023-07-13T22:24:55.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.065+0000 7f9f6a3ff700 1 -- start start 2023-07-13T22:24:55.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.066+0000 7f9f6a3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f640d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:55.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.066+0000 7f9f6a3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f640d2410 con 0x7f9f640d0e20 2023-07-13T22:24:55.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.066+0000 7f9f6a3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f640d2660 con 0x7f9f640d0a50 2023-07-13T22:24:55.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.066+0000 7f9f6a3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6415c900 con 0x7f9f640cd260 2023-07-13T22:24:55.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f6899c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9f640d0e20 0x7f9f640b5a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51896/0 (socket says 172.21.15.60:51896) 2023-07-13T22:24:55.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f6899c700 1 -- 172.21.15.60:0/2533444693 learned_addr learned my addr 172.21.15.60:0/2533444693 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:55.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f637fe700 1 --2- 172.21.15.60:0/2533444693 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f640d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:55.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f62ffd700 1 -- 172.21.15.60:0/2533444693 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035011958 0 0) 0x7f9f640d2410 con 0x7f9f640d0e20 2023-07-13T22:24:55.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f62ffd700 1 -- 172.21.15.60:0/2533444693 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f4c0035b0 con 0x7f9f640d0e20 2023-07-13T22:24:55.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f62ffd700 1 -- 172.21.15.60:0/2533444693 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553737063 0 0) 0x7f9f640d2660 con 0x7f9f640d0a50 2023-07-13T22:24:55.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f62ffd700 1 -- 172.21.15.60:0/2533444693 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f640d2410 con 0x7f9f640d0a50 2023-07-13T22:24:55.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f637fe700 1 -- 172.21.15.60:0/2533444693 >> v1:172.21.15.105:6789/0 conn(0x7f9f640d0a50 legacy=0x7f9f640cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:55.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f637fe700 1 -- 172.21.15.60:0/2533444693 >> v1:172.21.15.60:6789/0 conn(0x7f9f640d0e20 legacy=0x7f9f640b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:55.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f62ffd700 1 -- 172.21.15.60:0/2533444693 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3930646926 0 0) 0x7f9f4c0035b0 con 0x7f9f640d0e20 2023-07-13T22:24:55.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.067+0000 7f9f637fe700 1 -- 172.21.15.60:0/2533444693 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640d2940 con 0x7f9f640cd260 2023-07-13T22:24:55.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.068+0000 7f9f637fe700 1 --2- 172.21.15.60:0/2533444693 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f640d3420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009870 tx=0x7f9f58009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d79b6fc2e0092b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:55.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.068+0000 7f9f62ffd700 1 -- 172.21.15.60:0/2533444693 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0019f0 con 0x7f9f640cd260 2023-07-13T22:24:55.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.068+0000 7f9f62ffd700 1 -- 172.21.15.60:0/2533444693 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f58004030 con 0x7f9f640cd260 2023-07-13T22:24:55.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.068+0000 7f9f62ffd700 1 -- 172.21.15.60:0/2533444693 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f580043c0 con 0x7f9f640cd260 2023-07-13T22:24:55.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.069+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/2533444693 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 msgr2=0x7f9f640d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:55.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.069+0000 7f9f6a3ff700 1 --2- 172.21.15.60:0/2533444693 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f640d3420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009870 tx=0x7f9f58009b80 comp rx=0 tx=0).stop 2023-07-13T22:24:55.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.069+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/2533444693 shutdown_connections 2023-07-13T22:24:55.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.069+0000 7f9f6a3ff700 1 --2- 172.21.15.60:0/2533444693 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f640d3420 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.069+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/2533444693 >> 172.21.15.60:0/2533444693 conn(0x7f9f64071600 msgr2=0x7f9f640719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.069+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/2533444693 shutdown_connections 2023-07-13T22:24:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.069+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/2533444693 wait complete. 2023-07-13T22:24:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.070+0000 7f9f6a3ff700 1 Processor -- start 2023-07-13T22:24:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.071+0000 7f9f6a3ff700 1 -- start start 2023-07-13T22:24:55.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.071+0000 7f9f6a3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f6414b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:55.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.071+0000 7f9f6a3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f640d0a50 0x7f9f640c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:55.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.071+0000 7f9f6a3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f640d0e20 0x7f9f640ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:55.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6a3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6414c320 con 0x7f9f640d0e20 2023-07-13T22:24:55.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6a3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640cc660 con 0x7f9f640d0a50 2023-07-13T22:24:55.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6a3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640cc380 con 0x7f9f640cd260 2023-07-13T22:24:55.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6899c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f640d0e20 0x7f9f640ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6899c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f640d0e20 0x7f9f640ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41098/0 (socket says 172.21.15.60:41098) 2023-07-13T22:24:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6899c700 1 -- 172.21.15.60:0/490323747 learned_addr learned my addr 172.21.15.60:0/490323747 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f637fe700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f640d0a50 0x7f9f640c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f63fff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f6414b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6899c700 1 -- 172.21.15.60:0/490323747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 msgr2=0x7f9f6414b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6899c700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f6414b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6899c700 1 -- 172.21.15.60:0/490323747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f640d0a50 msgr2=0x7f9f640c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6899c700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f640d0a50 0x7f9f640c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:55.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f6899c700 1 -- 172.21.15.60:0/490323747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580094a0 con 0x7f9f640d0e20 2023-07-13T22:24:55.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.072+0000 7f9f63fff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f6414b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:55.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.073+0000 7f9f6899c700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f640d0e20 0x7f9f640ca220 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00cbc0 tx=0x7f9f5c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:55.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.073+0000 7f9f60ff9700 1 -- 172.21.15.60:0/490323747 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00ab00 con 0x7f9f640d0e20 2023-07-13T22:24:55.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.073+0000 7f9f60ff9700 1 -- 172.21.15.60:0/490323747 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c00ac60 con 0x7f9f640d0e20 2023-07-13T22:24:55.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.073+0000 7f9f60ff9700 1 -- 172.21.15.60:0/490323747 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0070d0 con 0x7f9f640d0e20 2023-07-13T22:24:55.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.073+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6414bcd0 con 0x7f9f640d0e20 2023-07-13T22:24:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.073+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6414b930 con 0x7f9f640d0e20 2023-07-13T22:24:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.074+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f64071f00 con 0x7f9f640d0e20 2023-07-13T22:24:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.075+0000 7f9f60ff9700 1 -- 172.21.15.60:0/490323747 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f5c007230 con 0x7f9f640d0e20 2023-07-13T22:24:55.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.076+0000 7f9f60ff9700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f5006ace0 0x7f9f5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:55.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.076+0000 7f9f60ff9700 1 -- 172.21.15.60:0/490323747 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9f5c088cc0 con 0x7f9f640d0e20 2023-07-13T22:24:55.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.076+0000 7f9f63fff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f5006ace0 0x7f9f5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:55.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.077+0000 7f9f63fff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f5006ace0 0x7f9f5006d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9f540011e0 tx=0x7f9f54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:55.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.081+0000 7f9f60ff9700 1 -- 172.21.15.60:0/490323747 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9f5c08f080 con 0x7f9f640d0e20 2023-07-13T22:24:55.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.366+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) v1 -- 0x7f9f6404e280 con 0x7f9f640d0e20 2023-07-13T22:24:55.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:24:55.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:24:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.835+0000 7f9f60ff9700 1 -- 172.21.15.60:0/490323747 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v227) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9f5c0180e0 con 0x7f9f640d0e20 2023-07-13T22:24:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f5006ace0 msgr2=0x7f9f5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f5006ace0 0x7f9f5006d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9f540011e0 tx=0x7f9f54009040 comp rx=0 tx=0).stop 2023-07-13T22:24:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f640d0e20 msgr2=0x7f9f640ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f640d0e20 0x7f9f640ca220 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00cbc0 tx=0x7f9f5c00ced0 comp rx=0 tx=0).stop 2023-07-13T22:24:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 shutdown_connections 2023-07-13T22:24:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9f5006ace0 0x7f9f5006d190 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f640d0e20 0x7f9f640ca220 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9f640cd260 0x7f9f6414b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 --2- 172.21.15.60:0/490323747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f640d0a50 0x7f9f640c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 >> 172.21.15.60:0/490323747 conn(0x7f9f64071600 msgr2=0x7f9f640cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 shutdown_connections 2023-07-13T22:24:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:55.837+0000 7f9f6a3ff700 1 -- 172.21.15.60:0/490323747 wait complete. 2023-07-13T22:24:55.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-13T22:24:56.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.064+0000 7f8397dff700 1 Processor -- start 2023-07-13T22:24:56.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.065+0000 7f8397dff700 1 -- start start 2023-07-13T22:24:56.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f8397dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8390159260 0x7f83900ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f8397dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83900cc410 0x7f83900cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f8397dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900703e0 0x7f8390070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f8397dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8390070d60 con 0x7f83900cc410 2023-07-13T22:24:56.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f8397dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83900cf640 con 0x7f8390159260 2023-07-13T22:24:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f8397dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83900ccf80 con 0x7f83900703e0 2023-07-13T22:24:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f839539a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83900cc410 0x7f83900cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f839539a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83900cc410 0x7f83900cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41112/0 (socket says 172.21.15.60:41112) 2023-07-13T22:24:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f839539a700 1 -- 172.21.15.60:0/4012212427 learned_addr learned my addr 172.21.15.60:0/4012212427 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f8395b9b700 1 --2- 172.21.15.60:0/4012212427 >> v2:172.21.15.105:3300/0 conn(0x7f8390159260 0x7f83900ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.066+0000 7f839639c700 1 --2- 172.21.15.60:0/4012212427 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900703e0 0x7f8390070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f839539a700 1 -- 172.21.15.60:0/4012212427 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900703e0 msgr2=0x7f8390070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f839539a700 1 --2- 172.21.15.60:0/4012212427 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900703e0 0x7f8390070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f839539a700 1 -- 172.21.15.60:0/4012212427 >> v2:172.21.15.105:3300/0 conn(0x7f8390159260 msgr2=0x7f83900ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f839539a700 1 --2- 172.21.15.60:0/4012212427 >> v2:172.21.15.105:3300/0 conn(0x7f8390159260 0x7f83900ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f839539a700 1 -- 172.21.15.60:0/4012212427 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83900c9b40 con 0x7f83900cc410 2023-07-13T22:24:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f8395b9b700 1 --2- 172.21.15.60:0/4012212427 >> v2:172.21.15.105:3300/0 conn(0x7f8390159260 0x7f83900ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:56.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f839539a700 1 --2- 172.21.15.60:0/4012212427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900cc410 0x7f83900cc8b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8390071110 tx=0x7f838000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a3d0fe15521df server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:56.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f8394b99700 1 -- 172.21.15.60:0/4012212427 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838000dd30 con 0x7f83900cc410 2023-07-13T22:24:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f8394b99700 1 -- 172.21.15.60:0/4012212427 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838000de90 con 0x7f83900cc410 2023-07-13T22:24:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.067+0000 7f8394b99700 1 -- 172.21.15.60:0/4012212427 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380010690 con 0x7f83900cc410 2023-07-13T22:24:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.068+0000 7f8397dff700 1 -- 172.21.15.60:0/4012212427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900cc410 msgr2=0x7f83900cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.068+0000 7f8397dff700 1 --2- 172.21.15.60:0/4012212427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900cc410 0x7f83900cc8b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8390071110 tx=0x7f838000be70 comp rx=0 tx=0).stop 2023-07-13T22:24:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.068+0000 7f8397dff700 1 -- 172.21.15.60:0/4012212427 shutdown_connections 2023-07-13T22:24:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.068+0000 7f8397dff700 1 --2- 172.21.15.60:0/4012212427 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900703e0 0x7f8390070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.068+0000 7f8397dff700 1 --2- 172.21.15.60:0/4012212427 >> v2:172.21.15.105:3300/0 conn(0x7f8390159260 0x7f83900ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.068+0000 7f8397dff700 1 --2- 172.21.15.60:0/4012212427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900cc410 0x7f83900cc8b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.068+0000 7f8397dff700 1 -- 172.21.15.60:0/4012212427 >> 172.21.15.60:0/4012212427 conn(0x7f83900cbf50 msgr2=0x7f83900ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.068+0000 7f8397dff700 1 -- 172.21.15.60:0/4012212427 shutdown_connections 2023-07-13T22:24:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.068+0000 7f8397dff700 1 -- 172.21.15.60:0/4012212427 wait complete. 2023-07-13T22:24:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.070+0000 7f8397dff700 1 Processor -- start 2023-07-13T22:24:56.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.070+0000 7f8397dff700 1 -- start start 2023-07-13T22:24:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8397dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900703e0 0x7f8390153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8397dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900cc410 0x7f83900c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8397dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8390159260 0x7f8390158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8397dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900c6b00 con 0x7f83900703e0 2023-07-13T22:24:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8397dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900af3c0 con 0x7f8390159260 2023-07-13T22:24:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8397dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83900af6a0 con 0x7f83900cc410 2023-07-13T22:24:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8395b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900703e0 0x7f8390153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8395b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900703e0 0x7f8390153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41120/0 (socket says 172.21.15.60:41120) 2023-07-13T22:24:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8395b9b700 1 -- 172.21.15.60:0/1241310393 learned_addr learned my addr 172.21.15.60:0/1241310393 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f839539a700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900cc410 0x7f83900c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f839639c700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8390159260 0x7f8390158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8395b9b700 1 -- 172.21.15.60:0/1241310393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900cc410 msgr2=0x7f83900c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8395b9b700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900cc410 0x7f83900c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8395b9b700 1 -- 172.21.15.60:0/1241310393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8390159260 msgr2=0x7f8390158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8395b9b700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8390159260 0x7f8390158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f8395b9b700 1 -- 172.21.15.60:0/1241310393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838000b7f0 con 0x7f83900703e0 2023-07-13T22:24:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.071+0000 7f839639c700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8390159260 0x7f8390158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.072+0000 7f8395b9b700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900703e0 0x7f8390153c00 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e810 tx=0x7f838c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.072+0000 7f83867fc700 1 -- 172.21.15.60:0/1241310393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c013610 con 0x7f83900703e0 2023-07-13T22:24:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.072+0000 7f83867fc700 1 -- 172.21.15.60:0/1241310393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c013770 con 0x7f83900703e0 2023-07-13T22:24:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.072+0000 7f83867fc700 1 -- 172.21.15.60:0/1241310393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c013b20 con 0x7f83900703e0 2023-07-13T22:24:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.072+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83900af980 con 0x7f83900703e0 2023-07-13T22:24:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.072+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83900c66d0 con 0x7f83900703e0 2023-07-13T22:24:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.073+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83900af220 con 0x7f83900703e0 2023-07-13T22:24:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.074+0000 7f83867fc700 1 -- 172.21.15.60:0/1241310393 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f838c013c80 con 0x7f83900703e0 2023-07-13T22:24:56.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.075+0000 7f83867fc700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f837406ace0 0x7f837406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.075+0000 7f83867fc700 1 -- 172.21.15.60:0/1241310393 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f838c088dd0 con 0x7f83900703e0 2023-07-13T22:24:56.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.075+0000 7f839539a700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f837406ace0 0x7f837406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.076+0000 7f839539a700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f837406ace0 0x7f837406d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f838000beb0 tx=0x7f838001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:56.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.079+0000 7f83867fc700 1 -- 172.21.15.60:0/1241310393 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f83900af220 con 0x7f83900703e0 2023-07-13T22:24:56.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.287+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f83900af220 con 0x7f83900703e0 2023-07-13T22:24:56.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.291+0000 7f83867fc700 1 -- 172.21.15.60:0/1241310393 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f83900af220 con 0x7f83900703e0 2023-07-13T22:24:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f837406ace0 msgr2=0x7f837406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f837406ace0 0x7f837406d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f838000beb0 tx=0x7f838001c040 comp rx=0 tx=0).stop 2023-07-13T22:24:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900703e0 msgr2=0x7f8390153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900703e0 0x7f8390153c00 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e810 tx=0x7f838c00eb20 comp rx=0 tx=0).stop 2023-07-13T22:24:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 shutdown_connections 2023-07-13T22:24:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f837406ace0 0x7f837406d190 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83900703e0 0x7f8390153c00 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8390159260 0x7f8390158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 --2- 172.21.15.60:0/1241310393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83900cc410 0x7f83900c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 >> 172.21.15.60:0/1241310393 conn(0x7f83900cbf50 msgr2=0x7f83900b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 shutdown_connections 2023-07-13T22:24:56.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.294+0000 7f8397dff700 1 -- 172.21.15.60:0/1241310393 wait complete. 2023-07-13T22:24:56.294 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:24:56.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:24:56.396 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:24:56.503 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:24:56.503 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:24:56.504 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:56.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:24:56.550 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:24:56.550 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:56.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:24:56.598 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:24:56.598 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:56.644 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:24:56.644 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:24:56.645 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:24:56.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:24:56.691 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-13T22:24:56.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.911+0000 7fe3e73ff700 1 Processor -- start 2023-07-13T22:24:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.911+0000 7fe3e73ff700 1 -- start start 2023-07-13T22:24:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.912+0000 7fe3e73ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3e00b8570 0x7fe3e00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.912+0000 7fe3e73ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 0x7fe3e00d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.912+0000 7fe3e73ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e00d09d0 con 0x7fe3e00cf3f0 2023-07-13T22:24:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.912+0000 7fe3e73ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e00d3b30 con 0x7fe3e00b8570 2023-07-13T22:24:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.912+0000 7fe3e73ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00cda10 con 0x7fe3e00d35c0 2023-07-13T22:24:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.912+0000 7fe3e519b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3e00b8570 0x7fe3e00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.913+0000 7fe3e519b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3e00b8570 0x7fe3e00ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41126/0 (socket says 172.21.15.60:41126) 2023-07-13T22:24:56.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.913+0000 7fe3e519b700 1 -- 172.21.15.60:0/3695024091 learned_addr learned my addr 172.21.15.60:0/3695024091 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:56.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.913+0000 7fe3e499a700 1 --2- 172.21.15.60:0/3695024091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 0x7fe3e00d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.913+0000 7fe3e519b700 1 -- 172.21.15.60:0/3695024091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 msgr2=0x7fe3e00d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:56.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.913+0000 7fe3e519b700 1 --2- 172.21.15.60:0/3695024091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 0x7fe3e00d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.913+0000 7fe3e519b700 1 -- 172.21.15.60:0/3695024091 >> v1:172.21.15.105:6789/0 conn(0x7fe3e00cf3f0 legacy=0x7fe3e015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:56.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.913+0000 7fe3e519b700 1 -- 172.21.15.60:0/3695024091 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00cd450 con 0x7fe3e00b8570 2023-07-13T22:24:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.913+0000 7fe3e499a700 1 --2- 172.21.15.60:0/3695024091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 0x7fe3e00d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.913+0000 7fe3e519b700 1 --2- 172.21.15.60:0/3695024091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 0x7fe3e00ceeb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc006fd0 tx=0x7fe3dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12160fca8475843d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3d7fff700 1 -- 172.21.15.60:0/3695024091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc005e30 con 0x7fe3e00b8570 2023-07-13T22:24:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3d7fff700 1 -- 172.21.15.60:0/3695024091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc004030 con 0x7fe3e00b8570 2023-07-13T22:24:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3d7fff700 1 -- 172.21.15.60:0/3695024091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0043e0 con 0x7fe3e00b8570 2023-07-13T22:24:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3695024091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 msgr2=0x7fe3e00ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3e73ff700 1 --2- 172.21.15.60:0/3695024091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 0x7fe3e00ceeb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc006fd0 tx=0x7fe3dc009bd0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3695024091 shutdown_connections 2023-07-13T22:24:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3e73ff700 1 --2- 172.21.15.60:0/3695024091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 0x7fe3e00ceeb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3e73ff700 1 --2- 172.21.15.60:0/3695024091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 0x7fe3e00d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3695024091 >> 172.21.15.60:0/3695024091 conn(0x7fe3e0071600 msgr2=0x7fe3e00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3695024091 shutdown_connections 2023-07-13T22:24:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.914+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3695024091 wait complete. 2023-07-13T22:24:56.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.916+0000 7fe3e73ff700 1 Processor -- start 2023-07-13T22:24:56.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.916+0000 7fe3e73ff700 1 -- start start 2023-07-13T22:24:56.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.917+0000 7fe3e73ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 0x7fe3e00b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.917+0000 7fe3e73ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00cf3f0 0x7fe3e0147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.917+0000 7fe3e73ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 0x7fe3e00cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.917+0000 7fe3e73ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e0148190 con 0x7fe3e00b8570 2023-07-13T22:24:56.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.917+0000 7fe3e73ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00b2d50 con 0x7fe3e00cf3f0 2023-07-13T22:24:56.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.917+0000 7fe3e73ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00b3030 con 0x7fe3e00d35c0 2023-07-13T22:24:56.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.917+0000 7fe3e519b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 0x7fe3e00b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.917+0000 7fe3e519b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 0x7fe3e00b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41136/0 (socket says 172.21.15.60:41136) 2023-07-13T22:24:56.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.917+0000 7fe3e519b700 1 -- 172.21.15.60:0/3192892078 learned_addr learned my addr 172.21.15.60:0/3192892078 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:56.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e499a700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00cf3f0 0x7fe3e0147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e599c700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 0x7fe3e00cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e519b700 1 -- 172.21.15.60:0/3192892078 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 msgr2=0x7fe3e00cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e519b700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 0x7fe3e00cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e519b700 1 -- 172.21.15.60:0/3192892078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00cf3f0 msgr2=0x7fe3e0147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e519b700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00cf3f0 0x7fe3e0147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e519b700 1 -- 172.21.15.60:0/3192892078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc009500 con 0x7fe3e00b8570 2023-07-13T22:24:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e499a700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00cf3f0 0x7fe3e0147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e519b700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 0x7fe3e00b26c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009130 tx=0x7fe3dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3d5ffb700 1 -- 172.21.15.60:0/3192892078 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc003e10 con 0x7fe3e00b8570 2023-07-13T22:24:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3d5ffb700 1 -- 172.21.15.60:0/3192892078 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc01d430 con 0x7fe3e00b8570 2023-07-13T22:24:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3d5ffb700 1 -- 172.21.15.60:0/3192892078 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc01d7e0 con 0x7fe3e00b8570 2023-07-13T22:24:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.918+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e00b3310 con 0x7fe3e00b8570 2023-07-13T22:24:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.919+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e00b38d0 con 0x7fe3e00b8570 2023-07-13T22:24:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.920+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e004e280 con 0x7fe3e00b8570 2023-07-13T22:24:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.921+0000 7fe3d5ffb700 1 -- 172.21.15.60:0/3192892078 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3dc01d940 con 0x7fe3e00b8570 2023-07-13T22:24:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.921+0000 7fe3d5ffb700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3c406ace0 0x7fe3c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.921+0000 7fe3d5ffb700 1 -- 172.21.15.60:0/3192892078 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe3dc08a6f0 con 0x7fe3e00b8570 2023-07-13T22:24:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.922+0000 7fe3e499a700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3c406ace0 0x7fe3c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.922+0000 7fe3e499a700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3c406ace0 0x7fe3c406d190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00b4ad0 tx=0x7fe3d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:56.926+0000 7fe3d5ffb700 1 -- 172.21.15.60:0/3192892078 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe3dc055bd0 con 0x7fe3e00b8570 2023-07-13T22:24:57.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.177+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3e00b7a50 con 0x7fe3e00b8570 2023-07-13T22:24:57.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.177+0000 7fe3d5ffb700 1 -- 172.21.15.60:0/3192892078 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe3dc016de0 con 0x7fe3e00b8570 2023-07-13T22:24:57.160 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:57.160 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/711733521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":711733521},{"type":"v1","addr":"172.21.15.105:6834","nonce":711733521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5609,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2135065566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":2135065566},{"type":"v1","addr":"172.21.15.105:6835","nonce":2135065566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6052,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/4236704538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4236704538},{"type":"v1","addr":"172.21.15.60:6836","nonce":4236704538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6654,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2913656796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":2913656796},{"type":"v1","addr":"172.21.15.60:6837","nonce":2913656796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[]} 2023-07-13T22:24:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3c406ace0 msgr2=0x7fe3c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3c406ace0 0x7fe3c406d190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00b4ad0 tx=0x7fe3d0008040 comp rx=0 tx=0).stop 2023-07-13T22:24:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 msgr2=0x7fe3e00b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:57.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 0x7fe3e00b26c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009130 tx=0x7fe3dc004750 comp rx=0 tx=0).stop 2023-07-13T22:24:57.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 shutdown_connections 2023-07-13T22:24:57.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3c406ace0 0x7fe3c406d190 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3e00b8570 0x7fe3e00b26c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3e00d35c0 0x7fe3e00cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 --2- 172.21.15.60:0/3192892078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3e00cf3f0 0x7fe3e0147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 >> 172.21.15.60:0/3192892078 conn(0x7fe3e0071600 msgr2=0x7fe3e00d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 shutdown_connections 2023-07-13T22:24:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.180+0000 7fe3e73ff700 1 -- 172.21.15.60:0/3192892078 wait complete. 2023-07-13T22:24:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 35 2023-07-13T22:24:57.180 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:24:57.181 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-13T22:24:57.181 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:24:57.181 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:24:57.181 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:24:57.182 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-13T22:24:57.182 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:24:57.182 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:24:57.182 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:24:57.183 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-13T22:24:57.183 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:24:57.183 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:24:57.183 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:24:57.183 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-13T22:24:57.184 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:24:57.184 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:24:57.194 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:24:57.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:24:57.196 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:24:57.196 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:24:57.198 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:24:57.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:24:57.199 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:24:57.199 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:24:57.239 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:24:57.239 DEBUG:teuthology.parallel:result is None 2023-07-13T22:24:57.241 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:24:57.241 DEBUG:teuthology.parallel:result is None 2023-07-13T22:24:57.242 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:24:57.243 DEBUG:teuthology.parallel:result is None 2023-07-13T22:24:57.246 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:24:57.246 DEBUG:teuthology.parallel:result is None 2023-07-13T22:24:57.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-13T22:24:57.387 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:24:57.388 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:24:57.392 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:24:57.393 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:24:57.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.482+0000 7f687ddff700 1 Processor -- start 2023-07-13T22:24:57.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.482+0000 7f687ddff700 1 -- start start 2023-07-13T22:24:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.483+0000 7f687ddff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f687806f970 0x7f6878071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.483+0000 7f687ddff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687806eed0 con 0x7f68780ced20 2023-07-13T22:24:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.483+0000 7f687ddff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6878069ed0 con 0x7f687815c710 2023-07-13T22:24:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.483+0000 7f687ddff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687806fd40 con 0x7f687806f970 2023-07-13T22:24:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.483+0000 7f6877fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f687806f970 0x7f6878071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:57.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.483+0000 7f686ffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f68780ced20 0x7f68781600c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51912/0 (socket says 172.21.15.60:51912) 2023-07-13T22:24:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.483+0000 7f686ffff700 1 -- 172.21.15.60:0/2708357150 learned_addr learned my addr 172.21.15.60:0/2708357150 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6876ffd700 1 -- 172.21.15.60:0/2708357150 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485600898 0 0) 0x7f687806eed0 con 0x7f68780ced20 2023-07-13T22:24:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6876ffd700 1 -- 172.21.15.60:0/2708357150 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68640035b0 con 0x7f68780ced20 2023-07-13T22:24:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6876ffd700 1 -- 172.21.15.60:0/2708357150 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315956135 0 0) 0x7f6878069ed0 con 0x7f687815c710 2023-07-13T22:24:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6877fff700 1 -- 172.21.15.60:0/2708357150 >> v1:172.21.15.105:6789/0 conn(0x7f687815c710 legacy=0x7f68780ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6877fff700 1 -- 172.21.15.60:0/2708357150 >> v1:172.21.15.60:6789/0 conn(0x7f68780ced20 legacy=0x7f68781600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6877fff700 1 -- 172.21.15.60:0/2708357150 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687806a1b0 con 0x7f687806f970 2023-07-13T22:24:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6877fff700 1 --2- 172.21.15.60:0/2708357150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687806f970 0x7f6878071e60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6870009d90 tx=0x7f687000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409b10f906839bd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6876ffd700 1 -- 172.21.15.60:0/2708357150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000e070 con 0x7f687806f970 2023-07-13T22:24:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6876ffd700 1 -- 172.21.15.60:0/2708357150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687000bb20 con 0x7f687806f970 2023-07-13T22:24:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.484+0000 7f6876ffd700 1 -- 172.21.15.60:0/2708357150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000bed0 con 0x7f687806f970 2023-07-13T22:24:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.485+0000 7f687ddff700 1 -- 172.21.15.60:0/2708357150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687806f970 msgr2=0x7f6878071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.485+0000 7f687ddff700 1 --2- 172.21.15.60:0/2708357150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687806f970 0x7f6878071e60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6870009d90 tx=0x7f687000b330 comp rx=0 tx=0).stop 2023-07-13T22:24:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.485+0000 7f687ddff700 1 -- 172.21.15.60:0/2708357150 shutdown_connections 2023-07-13T22:24:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.485+0000 7f687ddff700 1 --2- 172.21.15.60:0/2708357150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687806f970 0x7f6878071e60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.485+0000 7f687ddff700 1 -- 172.21.15.60:0/2708357150 >> 172.21.15.60:0/2708357150 conn(0x7f68780cc2a0 msgr2=0x7f68780cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.485+0000 7f687ddff700 1 -- 172.21.15.60:0/2708357150 shutdown_connections 2023-07-13T22:24:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.485+0000 7f687ddff700 1 -- 172.21.15.60:0/2708357150 wait complete. 2023-07-13T22:24:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.488+0000 7f687ddff700 1 Processor -- start 2023-07-13T22:24:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.488+0000 7f687ddff700 1 -- start start 2023-07-13T22:24:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f687ddff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f687806f970 0x7f68780ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f687ddff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68780ced20 0x7f68781f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f687ddff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f687815c710 0x7f68781f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f687ddff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68780ae970 con 0x7f68780ced20 2023-07-13T22:24:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f687ddff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68781f1b70 con 0x7f687815c710 2023-07-13T22:24:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f687ddff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687815fbc0 con 0x7f687806f970 2023-07-13T22:24:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f686ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68780ced20 0x7f68781f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f686ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68780ced20 0x7f68781f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41198/0 (socket says 172.21.15.60:41198) 2023-07-13T22:24:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f686ffff700 1 -- 172.21.15.60:0/3603812516 learned_addr learned my addr 172.21.15.60:0/3603812516 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f6877fff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f687815c710 0x7f68781f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:57.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.489+0000 7f68777fe700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f687806f970 0x7f68780ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f686ffff700 1 -- 172.21.15.60:0/3603812516 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f687806f970 msgr2=0x7f68780ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f686ffff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f687806f970 0x7f68780ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f686ffff700 1 -- 172.21.15.60:0/3603812516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f687815c710 msgr2=0x7f68781f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f686ffff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f687815c710 0x7f68781f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f686ffff700 1 -- 172.21.15.60:0/3603812516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6870009a70 con 0x7f68780ced20 2023-07-13T22:24:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f68777fe700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f687806f970 0x7f68780ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f686ffff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68780ced20 0x7f68781f1630 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6860001eb0 tx=0x7f686000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f6874ff9700 1 -- 172.21.15.60:0/3603812516 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860005d70 con 0x7f68780ced20 2023-07-13T22:24:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f6874ff9700 1 -- 172.21.15.60:0/3603812516 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68600070d0 con 0x7f68780ced20 2023-07-13T22:24:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68780c4c70 con 0x7f68780ced20 2023-07-13T22:24:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f6874ff9700 1 -- 172.21.15.60:0/3603812516 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860007480 con 0x7f68780ced20 2023-07-13T22:24:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.490+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68780c5230 con 0x7f68780ced20 2023-07-13T22:24:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.491+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6878071c40 con 0x7f68780ced20 2023-07-13T22:24:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.492+0000 7f6874ff9700 1 -- 172.21.15.60:0/3603812516 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f68600075e0 con 0x7f68780ced20 2023-07-13T22:24:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.493+0000 7f6874ff9700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f685806ace0 0x7f685806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.493+0000 7f6874ff9700 1 -- 172.21.15.60:0/3603812516 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6860088680 con 0x7f68780ced20 2023-07-13T22:24:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.493+0000 7f68777fe700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f685806ace0 0x7f685806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.494+0000 7f68777fe700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f685806ace0 0x7f685806d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6868000f60 tx=0x7f6868009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.497+0000 7f6874ff9700 1 -- 172.21.15.60:0/3603812516 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6860053a70 con 0x7f68780ced20 2023-07-13T22:24:57.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.718+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f68780718b0 con 0x7f68780ced20 2023-07-13T22:24:57.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.846+0000 7f6874ff9700 1 -- 172.21.15.60:0/3603812516 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v69) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6860057090 con 0x7f68780ced20 2023-07-13T22:24:57.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.848+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f685806ace0 msgr2=0x7f685806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:57.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.848+0000 7f687ddff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f685806ace0 0x7f685806d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6868000f60 tx=0x7f6868009040 comp rx=0 tx=0).stop 2023-07-13T22:24:57.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.848+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68780ced20 msgr2=0x7f68781f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:57.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.848+0000 7f687ddff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68780ced20 0x7f68781f1630 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6860001eb0 tx=0x7f686000c7b0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.849+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 shutdown_connections 2023-07-13T22:24:57.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.849+0000 7f687ddff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f685806ace0 0x7f685806d190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.849+0000 7f687ddff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f687815c710 0x7f68781f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.849+0000 7f687ddff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f687806f970 0x7f68780ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.849+0000 7f687ddff700 1 --2- 172.21.15.60:0/3603812516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68780ced20 0x7f68781f1630 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:57.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.849+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 >> 172.21.15.60:0/3603812516 conn(0x7f68780cc2a0 msgr2=0x7f68780b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:57.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.849+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 shutdown_connections 2023-07-13T22:24:57.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:57.849+0000 7f687ddff700 1 -- 172.21.15.60:0/3603812516 wait complete. 2023-07-13T22:24:57.833 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-07-13T22:24:57.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-13T22:24:58.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.071+0000 7eff7e1ff700 1 Processor -- start 2023-07-13T22:24:58.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.071+0000 7eff7e1ff700 1 -- start start 2023-07-13T22:24:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.072+0000 7eff7e1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780b8400 0x7eff780cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.072+0000 7eff7e1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff780cfca0 0x7eff780d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff7e1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff780d0ac0 con 0x7eff780d3750 2023-07-13T22:24:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff7e1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780b87d0 con 0x7eff780cfca0 2023-07-13T22:24:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff7e1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff780b4eb0 con 0x7eff780b8400 2023-07-13T22:24:58.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff77fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff780cfca0 0x7eff780d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff77fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff780cfca0 0x7eff780d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41212/0 (socket says 172.21.15.60:41212) 2023-07-13T22:24:58.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff77fff700 1 -- 172.21.15.60:0/4150361760 learned_addr learned my addr 172.21.15.60:0/4150361760 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:58.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff777fe700 1 --2- 172.21.15.60:0/4150361760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780b8400 0x7eff780cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff77fff700 1 -- 172.21.15.60:0/4150361760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780b8400 msgr2=0x7eff780cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:58.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff77fff700 1 --2- 172.21.15.60:0/4150361760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780b8400 0x7eff780cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff77fff700 1 -- 172.21.15.60:0/4150361760 >> v1:172.21.15.105:6789/0 conn(0x7eff780d3750 legacy=0x7eff780cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff77fff700 1 -- 172.21.15.60:0/4150361760 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780b5190 con 0x7eff780cfca0 2023-07-13T22:24:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.073+0000 7eff777fe700 1 --2- 172.21.15.60:0/4150361760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780b8400 0x7eff780cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.074+0000 7eff77fff700 1 --2- 172.21.15.60:0/4150361760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780cfca0 0x7eff780d0f50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7eff70009e50 tx=0x7eff7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d80df28f974e9f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.074+0000 7eff76ffd700 1 -- 172.21.15.60:0/4150361760 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7000e070 con 0x7eff780cfca0 2023-07-13T22:24:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.074+0000 7eff76ffd700 1 -- 172.21.15.60:0/4150361760 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7000b9c0 con 0x7eff780cfca0 2023-07-13T22:24:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.074+0000 7eff76ffd700 1 -- 172.21.15.60:0/4150361760 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7000bd70 con 0x7eff780cfca0 2023-07-13T22:24:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.074+0000 7eff7e1ff700 1 -- 172.21.15.60:0/4150361760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780cfca0 msgr2=0x7eff780d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.074+0000 7eff7e1ff700 1 --2- 172.21.15.60:0/4150361760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780cfca0 0x7eff780d0f50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7eff70009e50 tx=0x7eff7000b330 comp rx=0 tx=0).stop 2023-07-13T22:24:58.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.075+0000 7eff7e1ff700 1 -- 172.21.15.60:0/4150361760 shutdown_connections 2023-07-13T22:24:58.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.075+0000 7eff7e1ff700 1 --2- 172.21.15.60:0/4150361760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780cfca0 0x7eff780d0f50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.075+0000 7eff7e1ff700 1 --2- 172.21.15.60:0/4150361760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780b8400 0x7eff780cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.075+0000 7eff7e1ff700 1 -- 172.21.15.60:0/4150361760 >> 172.21.15.60:0/4150361760 conn(0x7eff7806b390 msgr2=0x7eff7806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:58.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.075+0000 7eff7e1ff700 1 -- 172.21.15.60:0/4150361760 shutdown_connections 2023-07-13T22:24:58.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.075+0000 7eff7e1ff700 1 -- 172.21.15.60:0/4150361760 wait complete. 2023-07-13T22:24:58.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.076+0000 7eff7e1ff700 1 Processor -- start 2023-07-13T22:24:58.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.076+0000 7eff7e1ff700 1 -- start start 2023-07-13T22:24:58.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.077+0000 7eff7e1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff780b8400 0x7eff78159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.077+0000 7eff7e1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780cfca0 0x7eff780c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.077+0000 7eff7e1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780d3750 0x7eff780c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.077+0000 7eff7e1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815da00 con 0x7eff780d3750 2023-07-13T22:24:58.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.077+0000 7eff7e1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815dce0 con 0x7eff780b8400 2023-07-13T22:24:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.077+0000 7eff7e1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815dfc0 con 0x7eff780cfca0 2023-07-13T22:24:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff77fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780d3750 0x7eff780c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff77fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780d3750 0x7eff780c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41220/0 (socket says 172.21.15.60:41220) 2023-07-13T22:24:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff77fff700 1 -- 172.21.15.60:0/3391062705 learned_addr learned my addr 172.21.15.60:0/3391062705 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff777fe700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff780b8400 0x7eff78159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff6ffff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780cfca0 0x7eff780c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff77fff700 1 -- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780cfca0 msgr2=0x7eff780c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff77fff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780cfca0 0x7eff780c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff77fff700 1 -- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff780b8400 msgr2=0x7eff78159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff77fff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff780b8400 0x7eff78159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff77fff700 1 -- 172.21.15.60:0/3391062705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70009b30 con 0x7eff780d3750 2023-07-13T22:24:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff777fe700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff780b8400 0x7eff78159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff6ffff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780cfca0 0x7eff780c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.078+0000 7eff77fff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780d3750 0x7eff780c9580 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7eff70012000 tx=0x7eff70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.079+0000 7eff74ff9700 1 -- 172.21.15.60:0/3391062705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700099b0 con 0x7eff780d3750 2023-07-13T22:24:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.079+0000 7eff74ff9700 1 -- 172.21.15.60:0/3391062705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff700092e0 con 0x7eff780d3750 2023-07-13T22:24:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.079+0000 7eff74ff9700 1 -- 172.21.15.60:0/3391062705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7001c640 con 0x7eff780d3750 2023-07-13T22:24:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.079+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7815e2a0 con 0x7eff780d3750 2023-07-13T22:24:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.079+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7815ccd0 con 0x7eff780d3750 2023-07-13T22:24:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.080+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff780c6650 con 0x7eff780d3750 2023-07-13T22:24:58.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.081+0000 7eff74ff9700 1 -- 172.21.15.60:0/3391062705 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff7001c7a0 con 0x7eff780d3750 2023-07-13T22:24:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.082+0000 7eff74ff9700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff5806ace0 0x7eff5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.082+0000 7eff74ff9700 1 -- 172.21.15.60:0/3391062705 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5210+0+0 (secure 0 0 0) 0x7eff7008abf0 con 0x7eff780d3750 2023-07-13T22:24:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.082+0000 7eff777fe700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff5806ace0 0x7eff5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.082+0000 7eff777fe700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff5806ace0 0x7eff5806d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7eff780c9c00 tx=0x7eff68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.086+0000 7eff74ff9700 1 -- 172.21.15.60:0/3391062705 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7eff70056090 con 0x7eff780d3750 2023-07-13T22:24:58.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.370+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7eff7815a910 con 0x7eff780d3750 2023-07-13T22:24:58.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.371+0000 7eff74ff9700 1 -- 172.21.15.60:0/3391062705 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7eff700596b0 con 0x7eff780d3750 2023-07-13T22:24:58.353 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:58.354 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:24:58.354 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-07-13T22:24:58.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.354 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-07-13T22:24:58.354 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw==", 2023-07-13T22:24:58.354 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:58.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:58.355 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:58.355 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.355 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-07-13T22:24:58.355 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g==", 2023-07-13T22:24:58.355 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:58.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:58.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:58.356 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.357 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-07-13T22:24:58.357 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA==", 2023-07-13T22:24:58.357 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:58.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:58.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:58.357 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.358 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-07-13T22:24:58.358 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ==", 2023-07-13T22:24:58.358 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:58.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:58.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:58.359 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.359 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-07-13T22:24:58.359 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ==", 2023-07-13T22:24:58.359 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.359 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:58.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:58.360 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.360 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-07-13T22:24:58.360 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw==", 2023-07-13T22:24:58.360 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:58.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:58.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.361 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-07-13T22:24:58.361 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q==", 2023-07-13T22:24:58.362 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.362 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:58.362 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:58.362 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.362 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.362 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-07-13T22:24:58.363 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ==", 2023-07-13T22:24:58.363 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.363 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:58.363 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:58.363 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.363 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.364 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-07-13T22:24:58.364 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg==", 2023-07-13T22:24:58.364 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.364 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:58.364 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:58.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.364 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.365 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-07-13T22:24:58.365 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA==", 2023-07-13T22:24:58.365 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.365 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:58.365 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:58.365 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.366 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-07-13T22:24:58.366 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw==", 2023-07-13T22:24:58.366 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.366 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:58.366 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:58.367 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.367 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.367 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-07-13T22:24:58.367 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ==", 2023-07-13T22:24:58.367 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.367 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:24:58.368 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:24:58.368 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.368 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.368 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-07-13T22:24:58.368 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w==", 2023-07-13T22:24:58.369 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.369 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:58.369 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2023-07-13T22:24:58.369 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-13T22:24:58.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.369 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-07-13T22:24:58.370 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw==", 2023-07-13T22:24:58.370 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.370 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:58.370 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:24:58.370 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:24:58.370 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:24:58.370 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.371 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-07-13T22:24:58.371 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg==", 2023-07-13T22:24:58.371 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.371 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:58.371 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:24:58.371 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:24:58.371 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:24:58.372 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.372 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-07-13T22:24:58.372 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q==", 2023-07-13T22:24:58.372 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.372 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:24:58.372 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:24:58.373 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:24:58.373 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:24:58.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.373 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-07-13T22:24:58.373 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBk7CgGBhAAaK88V4CEyrYfZx7wv+XG3g==", 2023-07-13T22:24:58.373 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.374 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:24:58.374 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:24:58.374 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:24:58.374 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.374 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.374 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-07-13T22:24:58.375 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ==", 2023-07-13T22:24:58.375 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.375 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:24:58.375 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:24:58.375 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.375 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:24:58.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:24:58.376 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-07-13T22:24:58.376 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ==", 2023-07-13T22:24:58.376 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:24:58.376 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:24:58.376 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:24:58.376 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:24:58.376 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.376 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:24:58.377 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:24:58.377 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:24:58.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff5806ace0 msgr2=0x7eff5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff5806ace0 0x7eff5806d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7eff780c9c00 tx=0x7eff68008040 comp rx=0 tx=0).stop 2023-07-13T22:24:58.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780d3750 msgr2=0x7eff780c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780d3750 0x7eff780c9580 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7eff70012000 tx=0x7eff70004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 shutdown_connections 2023-07-13T22:24:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff5806ace0 0x7eff5806d190 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff780d3750 0x7eff780c9580 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff780b8400 0x7eff78159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 --2- 172.21.15.60:0/3391062705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff780cfca0 0x7eff780c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 >> 172.21.15.60:0/3391062705 conn(0x7eff7806b390 msgr2=0x7eff780b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 shutdown_connections 2023-07-13T22:24:58.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.373+0000 7eff7e1ff700 1 -- 172.21.15.60:0/3391062705 wait complete. 2023-07-13T22:24:58.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:24:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.599+0000 7fc96cbff700 1 Processor -- start 2023-07-13T22:24:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.600+0000 7fc96cbff700 1 -- start start 2023-07-13T22:24:58.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.601+0000 7fc96cbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 0x7fc9680cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.601+0000 7fc96cbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc9680d3750 0x7fc9680cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.601+0000 7fc96cbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9680cff20 con 0x7fc9680cfad0 2023-07-13T22:24:58.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.601+0000 7fc96cbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9680d0ac0 con 0x7fc9680d3750 2023-07-13T22:24:58.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.601+0000 7fc96cbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680b87d0 con 0x7fc9680b8400 2023-07-13T22:24:58.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.601+0000 7fc95ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc9680d3750 0x7fc9680cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.601+0000 7fc95ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc9680d3750 0x7fc9680cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41226/0 (socket says 172.21.15.60:41226) 2023-07-13T22:24:58.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.601+0000 7fc95ffff700 1 -- 172.21.15.60:0/2604627432 learned_addr learned my addr 172.21.15.60:0/2604627432 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:58.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.601+0000 7fc96659c700 1 --2- 172.21.15.60:0/2604627432 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 0x7fc9680cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.602+0000 7fc95ffff700 1 -- 172.21.15.60:0/2604627432 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 msgr2=0x7fc9680cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:58.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.602+0000 7fc95ffff700 1 --2- 172.21.15.60:0/2604627432 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 0x7fc9680cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.602+0000 7fc95ffff700 1 -- 172.21.15.60:0/2604627432 >> v1:172.21.15.105:6789/0 conn(0x7fc9680cfad0 legacy=0x7fc96815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.602+0000 7fc95ffff700 1 -- 172.21.15.60:0/2604627432 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680b4eb0 con 0x7fc9680d3750 2023-07-13T22:24:58.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.602+0000 7fc95ffff700 1 --2- 172.21.15.60:0/2604627432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680d3750 0x7fc9680cf590 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc9500098f0 tx=0x7fc950009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d09f61d1dd2ace1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.602+0000 7fc965d9b700 1 -- 172.21.15.60:0/2604627432 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950011830 con 0x7fc9680d3750 2023-07-13T22:24:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.602+0000 7fc965d9b700 1 -- 172.21.15.60:0/2604627432 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950011990 con 0x7fc9680d3750 2023-07-13T22:24:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.602+0000 7fc965d9b700 1 -- 172.21.15.60:0/2604627432 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950011d20 con 0x7fc9680d3750 2023-07-13T22:24:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.603+0000 7fc96cbff700 1 -- 172.21.15.60:0/2604627432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680d3750 msgr2=0x7fc9680cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.603+0000 7fc96cbff700 1 --2- 172.21.15.60:0/2604627432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680d3750 0x7fc9680cf590 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc9500098f0 tx=0x7fc950009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.603+0000 7fc96cbff700 1 -- 172.21.15.60:0/2604627432 shutdown_connections 2023-07-13T22:24:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.603+0000 7fc96cbff700 1 --2- 172.21.15.60:0/2604627432 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 0x7fc9680cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.603+0000 7fc96cbff700 1 --2- 172.21.15.60:0/2604627432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680d3750 0x7fc9680cf590 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.603+0000 7fc96cbff700 1 -- 172.21.15.60:0/2604627432 >> 172.21.15.60:0/2604627432 conn(0x7fc96806b390 msgr2=0x7fc96806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:58.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.603+0000 7fc96cbff700 1 -- 172.21.15.60:0/2604627432 shutdown_connections 2023-07-13T22:24:58.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.603+0000 7fc96cbff700 1 -- 172.21.15.60:0/2604627432 wait complete. 2023-07-13T22:24:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.605+0000 7fc96cbff700 1 Processor -- start 2023-07-13T22:24:58.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.605+0000 7fc96cbff700 1 -- start start 2023-07-13T22:24:58.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc96cbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 0x7fc9680c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc96cbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680cfad0 0x7fc96815f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc96cbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9680d3750 0x7fc9680b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc96cbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680c90f0 con 0x7fc9680cfad0 2023-07-13T22:24:58.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc96cbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96815ec50 con 0x7fc9680d3750 2023-07-13T22:24:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc96cbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96815dc20 con 0x7fc9680b8400 2023-07-13T22:24:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc95ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680cfad0 0x7fc96815f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc95ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680cfad0 0x7fc96815f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41236/0 (socket says 172.21.15.60:41236) 2023-07-13T22:24:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc95ffff700 1 -- 172.21.15.60:0/3527721756 learned_addr learned my addr 172.21.15.60:0/3527721756 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc96659c700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 0x7fc9680c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc966d9d700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9680d3750 0x7fc9680b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc95ffff700 1 -- 172.21.15.60:0/3527721756 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 msgr2=0x7fc9680c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc95ffff700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 0x7fc9680c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc95ffff700 1 -- 172.21.15.60:0/3527721756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9680d3750 msgr2=0x7fc9680b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc95ffff700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9680d3750 0x7fc9680b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.606+0000 7fc95ffff700 1 -- 172.21.15.60:0/3527721756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950009550 con 0x7fc9680cfad0 2023-07-13T22:24:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.607+0000 7fc95ffff700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680cfad0 0x7fc96815f0e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc950009c40 tx=0x7fc950004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.607+0000 7fc95effd700 1 -- 172.21.15.60:0/3527721756 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950004590 con 0x7fc9680cfad0 2023-07-13T22:24:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.607+0000 7fc95effd700 1 -- 172.21.15.60:0/3527721756 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95001e070 con 0x7fc9680cfad0 2023-07-13T22:24:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.607+0000 7fc95effd700 1 -- 172.21.15.60:0/3527721756 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500038e0 con 0x7fc9680cfad0 2023-07-13T22:24:58.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.607+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96815d940 con 0x7fc9680cfad0 2023-07-13T22:24:58.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.607+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96815e8e0 con 0x7fc9680cfad0 2023-07-13T22:24:58.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.608+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96815ea70 con 0x7fc9680cfad0 2023-07-13T22:24:58.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.609+0000 7fc95effd700 1 -- 172.21.15.60:0/3527721756 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc950003a40 con 0x7fc9680cfad0 2023-07-13T22:24:58.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.611+0000 7fc95effd700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc94806ace0 0x7fc94806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:58.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.611+0000 7fc95effd700 1 -- 172.21.15.60:0/3527721756 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc95008a0a0 con 0x7fc9680cfad0 2023-07-13T22:24:58.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.611+0000 7fc96659c700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc94806ace0 0x7fc94806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:58.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.611+0000 7fc96659c700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc94806ace0 0x7fc94806d190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc958009b40 tx=0x7fc958009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:58.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.615+0000 7fc95effd700 1 -- 172.21.15.60:0/3527721756 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc950055490 con 0x7fc9680cfad0 2023-07-13T22:24:58.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.825+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9680b47e0 con 0x7fc9680cfad0 2023-07-13T22:24:58.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.826+0000 7fc95effd700 1 -- 172.21.15.60:0/3527721756 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+14546 (secure 0 0 0) 0x7fc950058ab0 con 0x7fc9680cfad0 2023-07-13T22:24:58.808 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:24:58.809 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:24:57.851743+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:24:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc94806ace0 msgr2=0x7fc94806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc94806ace0 0x7fc94806d190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc958009b40 tx=0x7fc958009210 comp rx=0 tx=0).stop 2023-07-13T22:24:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680cfad0 msgr2=0x7fc96815f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680cfad0 0x7fc96815f0e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc950009c40 tx=0x7fc950004750 comp rx=0 tx=0).stop 2023-07-13T22:24:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 shutdown_connections 2023-07-13T22:24:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc94806ace0 0x7fc94806d190 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9680d3750 0x7fc9680b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9680b8400 0x7fc9680c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 --2- 172.21.15.60:0/3527721756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680cfad0 0x7fc96815f0e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 >> 172.21.15.60:0/3527721756 conn(0x7fc96806b390 msgr2=0x7fc9680d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 shutdown_connections 2023-07-13T22:24:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:58.828+0000 7fc96cbff700 1 -- 172.21.15.60:0/3527721756 wait complete. 2023-07-13T22:24:58.829 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:24:58.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:24:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.056+0000 7fc72b9ff700 1 Processor -- start 2023-07-13T22:24:59.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.056+0000 7fc72b9ff700 1 -- start start 2023-07-13T22:24:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.057+0000 7fc72b9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc7240cfd50 0x7fc7240d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.057+0000 7fc72b9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b1ac0 0x7fc7240d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.057+0000 7fc72b9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240d1870 con 0x7fc7240b8200 2023-07-13T22:24:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.057+0000 7fc72b9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240cf370 con 0x7fc7240cfd50 2023-07-13T22:24:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.057+0000 7fc72b9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d12b0 con 0x7fc7240b1ac0 2023-07-13T22:24:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc72979b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc7240b8200 0x7fc7240cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51920/0 (socket says 172.21.15.60:51920) 2023-07-13T22:24:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc72979b700 1 -- 172.21.15.60:0/1550486538 learned_addr learned my addr 172.21.15.60:0/1550486538 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc729f9c700 1 --2- 172.21.15.60:0/1550486538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b1ac0 0x7fc7240d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc728f9a700 1 --2- 172.21.15.60:0/1550486538 >> v2:172.21.15.105:3300/0 conn(0x7fc7240cfd50 0x7fc7240d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc71bfff700 1 -- 172.21.15.60:0/1550486538 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078731990 0 0) 0x7fc7240d1870 con 0x7fc7240b8200 2023-07-13T22:24:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc71bfff700 1 -- 172.21.15.60:0/1550486538 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7100035b0 con 0x7fc7240b8200 2023-07-13T22:24:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc729f9c700 1 -- 172.21.15.60:0/1550486538 >> v2:172.21.15.105:3300/0 conn(0x7fc7240cfd50 msgr2=0x7fc7240d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc729f9c700 1 --2- 172.21.15.60:0/1550486538 >> v2:172.21.15.105:3300/0 conn(0x7fc7240cfd50 0x7fc7240d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc729f9c700 1 -- 172.21.15.60:0/1550486538 >> v1:172.21.15.60:6789/0 conn(0x7fc7240b8200 legacy=0x7fc7240cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc71bfff700 1 -- 172.21.15.60:0/1550486538 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1012765485 0 0) 0x7fc7100035b0 con 0x7fc7240b8200 2023-07-13T22:24:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.058+0000 7fc729f9c700 1 -- 172.21.15.60:0/1550486538 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240d1590 con 0x7fc7240b1ac0 2023-07-13T22:24:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.059+0000 7fc729f9c700 1 --2- 172.21.15.60:0/1550486538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b1ac0 0x7fc7240d2900 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009d90 tx=0x7fc71c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75606d2345ba7768 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.059+0000 7fc71bfff700 1 -- 172.21.15.60:0/1550486538 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00e070 con 0x7fc7240b1ac0 2023-07-13T22:24:59.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.059+0000 7fc71bfff700 1 -- 172.21.15.60:0/1550486538 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c00bb20 con 0x7fc7240b1ac0 2023-07-13T22:24:59.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.059+0000 7fc71bfff700 1 -- 172.21.15.60:0/1550486538 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00be90 con 0x7fc7240b1ac0 2023-07-13T22:24:59.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.059+0000 7fc72b9ff700 1 -- 172.21.15.60:0/1550486538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b1ac0 msgr2=0x7fc7240d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.059+0000 7fc72b9ff700 1 --2- 172.21.15.60:0/1550486538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b1ac0 0x7fc7240d2900 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009d90 tx=0x7fc71c00b330 comp rx=0 tx=0).stop 2023-07-13T22:24:59.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.060+0000 7fc72b9ff700 1 -- 172.21.15.60:0/1550486538 shutdown_connections 2023-07-13T22:24:59.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.060+0000 7fc72b9ff700 1 --2- 172.21.15.60:0/1550486538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b1ac0 0x7fc7240d2900 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.060+0000 7fc72b9ff700 1 --2- 172.21.15.60:0/1550486538 >> v2:172.21.15.105:3300/0 conn(0x7fc7240cfd50 0x7fc7240d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.060+0000 7fc72b9ff700 1 -- 172.21.15.60:0/1550486538 >> 172.21.15.60:0/1550486538 conn(0x7fc72406b2f0 msgr2=0x7fc72406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:59.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.060+0000 7fc72b9ff700 1 -- 172.21.15.60:0/1550486538 shutdown_connections 2023-07-13T22:24:59.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.060+0000 7fc72b9ff700 1 -- 172.21.15.60:0/1550486538 wait complete. 2023-07-13T22:24:59.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.061+0000 7fc72b9ff700 1 Processor -- start 2023-07-13T22:24:59.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.062+0000 7fc72b9ff700 1 -- start start 2023-07-13T22:24:59.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.062+0000 7fc72b9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7240b1ac0 0x7fc7241f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.062+0000 7fc72b9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b8200 0x7fc72415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.062+0000 7fc72b9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7240cfd50 0x7fc72415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.062+0000 7fc72b9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72415f9a0 con 0x7fc7240b1ac0 2023-07-13T22:24:59.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.062+0000 7fc72b9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72415ca60 con 0x7fc7240cfd50 2023-07-13T22:24:59.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.062+0000 7fc72b9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72415cc90 con 0x7fc7240b8200 2023-07-13T22:24:59.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc72979b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7240b1ac0 0x7fc7241f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc72979b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7240b1ac0 0x7fc7241f26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41246/0 (socket says 172.21.15.60:41246) 2023-07-13T22:24:59.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc72979b700 1 -- 172.21.15.60:0/3277225668 learned_addr learned my addr 172.21.15.60:0/3277225668 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:59.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc728f9a700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b8200 0x7fc72415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc729f9c700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7240cfd50 0x7fc72415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc72979b700 1 -- 172.21.15.60:0/3277225668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b8200 msgr2=0x7fc72415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc72979b700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b8200 0x7fc72415f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc72979b700 1 -- 172.21.15.60:0/3277225668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7240cfd50 msgr2=0x7fc72415ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc72979b700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7240cfd50 0x7fc72415ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc72979b700 1 -- 172.21.15.60:0/3277225668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c009a70 con 0x7fc7240b1ac0 2023-07-13T22:24:59.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.063+0000 7fc72979b700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7240b1ac0 0x7fc7241f26a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc72000c780 tx=0x7fc72000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:59.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.064+0000 7fc719ffb700 1 -- 172.21.15.60:0/3277225668 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720005d70 con 0x7fc7240b1ac0 2023-07-13T22:24:59.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.064+0000 7fc719ffb700 1 -- 172.21.15.60:0/3277225668 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7200070d0 con 0x7fc7240b1ac0 2023-07-13T22:24:59.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.064+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72415cf70 con 0x7fc7240b1ac0 2023-07-13T22:24:59.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.064+0000 7fc719ffb700 1 -- 172.21.15.60:0/3277225668 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720007480 con 0x7fc7240b1ac0 2023-07-13T22:24:59.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.064+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72415d530 con 0x7fc7240b1ac0 2023-07-13T22:24:59.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.065+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72415d750 con 0x7fc7240b1ac0 2023-07-13T22:24:59.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.066+0000 7fc719ffb700 1 -- 172.21.15.60:0/3277225668 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc7200075e0 con 0x7fc7240b1ac0 2023-07-13T22:24:59.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.067+0000 7fc719ffb700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc70806ace0 0x7fc70806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.067+0000 7fc719ffb700 1 -- 172.21.15.60:0/3277225668 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc7200886e0 con 0x7fc7240b1ac0 2023-07-13T22:24:59.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.067+0000 7fc728f9a700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc70806ace0 0x7fc70806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.068+0000 7fc728f9a700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc70806ace0 0x7fc70806d190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc7240b0dc0 tx=0x7fc714009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:59.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.071+0000 7fc719ffb700 1 -- 172.21.15.60:0/3277225668 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc720053a50 con 0x7fc7240b1ac0 2023-07-13T22:24:59.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.278+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc72404e210 con 0x7fc7240b1ac0 2023-07-13T22:24:59.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.278+0000 7fc719ffb700 1 -- 172.21.15.60:0/3277225668 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc720057070 con 0x7fc7240b1ac0 2023-07-13T22:24:59.261 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:24:59.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.280+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc70806ace0 msgr2=0x7fc70806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc70806ace0 0x7fc70806d190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc7240b0dc0 tx=0x7fc714009210 comp rx=0 tx=0).stop 2023-07-13T22:24:59.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7240b1ac0 msgr2=0x7fc7241f26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7240b1ac0 0x7fc7241f26a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc72000c780 tx=0x7fc72000ca90 comp rx=0 tx=0).stop 2023-07-13T22:24:59.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 shutdown_connections 2023-07-13T22:24:59.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc70806ace0 0x7fc70806d190 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7240b1ac0 0x7fc7241f26a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7240cfd50 0x7fc72415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 --2- 172.21.15.60:0/3277225668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7240b8200 0x7fc72415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 >> 172.21.15.60:0/3277225668 conn(0x7fc72406b2f0 msgr2=0x7fc7240b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:59.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 shutdown_connections 2023-07-13T22:24:59.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.281+0000 7fc72b9ff700 1 -- 172.21.15.60:0/3277225668 wait complete. 2023-07-13T22:24:59.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:24:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.504+0000 7f2eae3ff700 1 Processor -- start 2023-07-13T22:24:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.505+0000 7f2eae3ff700 1 -- start start 2023-07-13T22:24:59.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.506+0000 7f2eae3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 0x7f2ea80703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.506+0000 7f2eae3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea8070d80 con 0x7f2ea8070920 2023-07-13T22:24:59.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.506+0000 7f2eae3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea80cad70 con 0x7f2ea8159260 2023-07-13T22:24:59.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.506+0000 7f2eae3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80cf640 con 0x7f2ea80ca900 2023-07-13T22:24:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.506+0000 7f2eac99c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2ea8070920 0x7f2ea806c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51932/0 (socket says 172.21.15.60:51932) 2023-07-13T22:24:59.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.506+0000 7f2eac99c700 1 -- 172.21.15.60:0/2381528049 learned_addr learned my addr 172.21.15.60:0/2381528049 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:59.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.506+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/2381528049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 0x7f2ea80703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/2381528049 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267741030 0 0) 0x7f2ea8070d80 con 0x7f2ea8070920 2023-07-13T22:24:59.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/2381528049 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e900035b0 con 0x7f2ea8070920 2023-07-13T22:24:59.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/2381528049 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771841884 0 0) 0x7f2ea80cad70 con 0x7f2ea8159260 2023-07-13T22:24:59.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/2381528049 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea8070d80 con 0x7f2ea8159260 2023-07-13T22:24:59.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea77fe700 1 -- 172.21.15.60:0/2381528049 >> v1:172.21.15.105:6789/0 conn(0x7f2ea8159260 legacy=0x7f2ea80cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea77fe700 1 -- 172.21.15.60:0/2381528049 >> v1:172.21.15.60:6789/0 conn(0x7f2ea8070920 legacy=0x7f2ea806c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea77fe700 1 -- 172.21.15.60:0/2381528049 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80ccf80 con 0x7f2ea80ca900 2023-07-13T22:24:59.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/2381528049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 0x7f2ea80703e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009870 tx=0x7f2e9c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e000d173ab6e5dcb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:59.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/2381528049 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c011830 con 0x7f2ea80ca900 2023-07-13T22:24:59.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/2381528049 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c011990 con 0x7f2ea80ca900 2023-07-13T22:24:59.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.507+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/2381528049 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c011d20 con 0x7f2ea80ca900 2023-07-13T22:24:59.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.508+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2381528049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 msgr2=0x7f2ea80703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.508+0000 7f2eae3ff700 1 --2- 172.21.15.60:0/2381528049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 0x7f2ea80703e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009870 tx=0x7f2e9c009b80 comp rx=0 tx=0).stop 2023-07-13T22:24:59.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.508+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2381528049 shutdown_connections 2023-07-13T22:24:59.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.508+0000 7f2eae3ff700 1 --2- 172.21.15.60:0/2381528049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 0x7f2ea80703e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.508+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2381528049 >> 172.21.15.60:0/2381528049 conn(0x7f2ea80cbf50 msgr2=0x7f2ea80ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:59.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.508+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2381528049 shutdown_connections 2023-07-13T22:24:59.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.508+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2381528049 wait complete. 2023-07-13T22:24:59.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.510+0000 7f2eae3ff700 1 Processor -- start 2023-07-13T22:24:59.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.511+0000 7f2eae3ff700 1 -- start start 2023-07-13T22:24:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2eae3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea8070920 0x7f2ea8151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2eae3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 0x7f2ea815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2eae3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea8159260 0x7f2ea81f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2eae3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea8159630 con 0x7f2ea8070920 2023-07-13T22:24:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2eae3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81522c0 con 0x7f2ea8159260 2023-07-13T22:24:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2eae3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81f6030 con 0x7f2ea80ca900 2023-07-13T22:24:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2ea7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea8070920 0x7f2ea8151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2ea7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea8070920 0x7f2ea8151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41254/0 (socket says 172.21.15.60:41254) 2023-07-13T22:24:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2ea7fff700 1 -- 172.21.15.60:0/2630883630 learned_addr learned my addr 172.21.15.60:0/2630883630 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 0x7f2ea815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.512+0000 7f2eac99c700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea8159260 0x7f2ea81f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2ea7fff700 1 -- 172.21.15.60:0/2630883630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 msgr2=0x7f2ea815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2ea7fff700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 0x7f2ea815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2ea7fff700 1 -- 172.21.15.60:0/2630883630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea8159260 msgr2=0x7f2ea81f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2ea7fff700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea8159260 0x7f2ea81f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2ea7fff700 1 -- 172.21.15.60:0/2630883630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0094a0 con 0x7f2ea8070920 2023-07-13T22:24:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2ea7fff700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea8070920 0x7f2ea8151d80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800c530 tx=0x7f2e9800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2ea4ff9700 1 -- 172.21.15.60:0/2630883630 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800cea0 con 0x7f2ea8070920 2023-07-13T22:24:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2ea4ff9700 1 -- 172.21.15.60:0/2630883630 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e980052f0 con 0x7f2ea8070920 2023-07-13T22:24:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea81f6310 con 0x7f2ea8070920 2023-07-13T22:24:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2ea4ff9700 1 -- 172.21.15.60:0/2630883630 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98003680 con 0x7f2ea8070920 2023-07-13T22:24:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.513+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea8159b60 con 0x7f2ea8070920 2023-07-13T22:24:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.514+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea81f6550 con 0x7f2ea8070920 2023-07-13T22:24:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.515+0000 7f2ea4ff9700 1 -- 172.21.15.60:0/2630883630 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2e980037e0 con 0x7f2ea8070920 2023-07-13T22:24:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.516+0000 7f2ea4ff9700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.516+0000 7f2ea4ff9700 1 -- 172.21.15.60:0/2630883630 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2e98088550 con 0x7f2ea8070920 2023-07-13T22:24:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.517+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.517+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0097c0 tx=0x7f2e9c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.520+0000 7f2ea4ff9700 1 -- 172.21.15.60:0/2630883630 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2e980538b0 con 0x7f2ea8070920 2023-07-13T22:24:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.728+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ea804e210 con 0x7f2ea8070920 2023-07-13T22:24:59.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.728+0000 7f2ea4ff9700 1 -- 172.21.15.60:0/2630883630 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e98056ed0 con 0x7f2ea8070920 2023-07-13T22:24:59.711 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:24:59.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 msgr2=0x7f2e9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0097c0 tx=0x7f2e9c01c040 comp rx=0 tx=0).stop 2023-07-13T22:24:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea8070920 msgr2=0x7f2ea8151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea8070920 0x7f2ea8151d80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800c530 tx=0x7f2e9800c840 comp rx=0 tx=0).stop 2023-07-13T22:24:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 shutdown_connections 2023-07-13T22:24:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea8070920 0x7f2ea8151d80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea8159260 0x7f2ea81f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 --2- 172.21.15.60:0/2630883630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80ca900 0x7f2ea815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 >> 172.21.15.60:0/2630883630 conn(0x7f2ea80cbf50 msgr2=0x7f2ea80ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:59.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 shutdown_connections 2023-07-13T22:24:59.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.731+0000 7f2eae3ff700 1 -- 172.21.15.60:0/2630883630 wait complete. 2023-07-13T22:24:59.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:24:59.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.953+0000 7f7d15dff700 1 Processor -- start 2023-07-13T22:24:59.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.953+0000 7f7d15dff700 1 -- start start 2023-07-13T22:24:59.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.954+0000 7f7d15dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d100d3750 0x7f7d100cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.954+0000 7f7d15dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100cfad0 0x7f7d100d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.954+0000 7f7d15dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d100cd950 con 0x7f7d100b8400 2023-07-13T22:24:59.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.954+0000 7f7d15dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d100cff50 con 0x7f7d100d3750 2023-07-13T22:24:59.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.954+0000 7f7d15dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100d0ac0 con 0x7f7d100cfad0 2023-07-13T22:24:59.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.954+0000 7f7d0effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d100d3750 0x7f7d100cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.954+0000 7f7d0f7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7d100b8400 0x7f7d100cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51942/0 (socket says 172.21.15.60:51942) 2023-07-13T22:24:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.954+0000 7f7d0f7fe700 1 -- 172.21.15.60:0/1241284837 learned_addr learned my addr 172.21.15.60:0/1241284837 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:24:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.954+0000 7f7d0ffff700 1 --2- 172.21.15.60:0/1241284837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100cfad0 0x7f7d100d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.955+0000 7f7d0effd700 1 -- 172.21.15.60:0/1241284837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100cfad0 msgr2=0x7f7d100d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.955+0000 7f7d0effd700 1 --2- 172.21.15.60:0/1241284837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100cfad0 0x7f7d100d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.955+0000 7f7d0effd700 1 -- 172.21.15.60:0/1241284837 >> v1:172.21.15.60:6789/0 conn(0x7f7d100b8400 legacy=0x7f7d100cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.955+0000 7f7d0effd700 1 -- 172.21.15.60:0/1241284837 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d100b87d0 con 0x7f7d100d3750 2023-07-13T22:24:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.955+0000 7f7d0e7fc700 1 -- 172.21.15.60:0/1241284837 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865367740 0 0) 0x7f7d00001110 con 0x7f7d100b8400 2023-07-13T22:24:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.955+0000 7f7d0effd700 1 --2- 172.21.15.60:0/1241284837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100d3750 0x7f7d100cf590 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009840 tx=0x7f7d04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b9cf9846f27b8d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.955+0000 7f7d0e7fc700 1 -- 172.21.15.60:0/1241284837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04011830 con 0x7f7d100d3750 2023-07-13T22:24:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.955+0000 7f7d0e7fc700 1 -- 172.21.15.60:0/1241284837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d04011990 con 0x7f7d100d3750 2023-07-13T22:24:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.955+0000 7f7d0e7fc700 1 -- 172.21.15.60:0/1241284837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04011d20 con 0x7f7d100d3750 2023-07-13T22:24:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.956+0000 7f7d15dff700 1 -- 172.21.15.60:0/1241284837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100d3750 msgr2=0x7f7d100cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.956+0000 7f7d15dff700 1 --2- 172.21.15.60:0/1241284837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100d3750 0x7f7d100cf590 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009840 tx=0x7f7d04009b50 comp rx=0 tx=0).stop 2023-07-13T22:24:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.956+0000 7f7d15dff700 1 -- 172.21.15.60:0/1241284837 shutdown_connections 2023-07-13T22:24:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.956+0000 7f7d15dff700 1 --2- 172.21.15.60:0/1241284837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100cfad0 0x7f7d100d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.956+0000 7f7d15dff700 1 --2- 172.21.15.60:0/1241284837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100d3750 0x7f7d100cf590 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.956+0000 7f7d15dff700 1 -- 172.21.15.60:0/1241284837 >> 172.21.15.60:0/1241284837 conn(0x7f7d1006b390 msgr2=0x7f7d1006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.956+0000 7f7d15dff700 1 -- 172.21.15.60:0/1241284837 shutdown_connections 2023-07-13T22:24:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.956+0000 7f7d15dff700 1 -- 172.21.15.60:0/1241284837 wait complete. 2023-07-13T22:24:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.958+0000 7f7d15dff700 1 Processor -- start 2023-07-13T22:24:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.958+0000 7f7d15dff700 1 -- start start 2023-07-13T22:24:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d15dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d100b8400 0x7f7d10152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d15dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100cfad0 0x7f7d10153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d15dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100d3750 0x7f7d10148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d15dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d101488c0 con 0x7f7d100cfad0 2023-07-13T22:24:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d15dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100c84e0 con 0x7f7d100b8400 2023-07-13T22:24:59.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d15dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10152850 con 0x7f7d100d3750 2023-07-13T22:24:59.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100cfad0 0x7f7d10153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100cfad0 0x7f7d10153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41274/0 (socket says 172.21.15.60:41274) 2023-07-13T22:24:59.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0effd700 1 -- 172.21.15.60:0/3572472997 learned_addr learned my addr 172.21.15.60:0/3572472997 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:24:59.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0ffff700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100d3750 0x7f7d10148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0f7fe700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d100b8400 0x7f7d10152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0effd700 1 -- 172.21.15.60:0/3572472997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100d3750 msgr2=0x7f7d10148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0effd700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100d3750 0x7f7d10148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0effd700 1 -- 172.21.15.60:0/3572472997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d100b8400 msgr2=0x7f7d10152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:59.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0effd700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d100b8400 0x7f7d10152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:59.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.959+0000 7f7d0effd700 1 -- 172.21.15.60:0/3572472997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040094a0 con 0x7f7d100cfad0 2023-07-13T22:24:59.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.960+0000 7f7d0effd700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100cfad0 0x7f7d10153220 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009fd0 tx=0x7f7d04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:59.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.960+0000 7f7cf7fff700 1 -- 172.21.15.60:0/3572472997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d040038a0 con 0x7f7d100cfad0 2023-07-13T22:24:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.960+0000 7f7cf7fff700 1 -- 172.21.15.60:0/3572472997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0401f070 con 0x7f7d100cfad0 2023-07-13T22:24:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.960+0000 7f7cf7fff700 1 -- 172.21.15.60:0/3572472997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04003c40 con 0x7f7d100cfad0 2023-07-13T22:24:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.960+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d10151c40 con 0x7f7d100cfad0 2023-07-13T22:24:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.960+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d10152200 con 0x7f7d100cfad0 2023-07-13T22:24:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.961+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d10152990 con 0x7f7d100cfad0 2023-07-13T22:24:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.962+0000 7f7cf7fff700 1 -- 172.21.15.60:0/3572472997 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d04026480 con 0x7f7d100cfad0 2023-07-13T22:24:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.964+0000 7f7cf7fff700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cfc06ace0 0x7f7cfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.964+0000 7f7cf7fff700 1 -- 172.21.15.60:0/3572472997 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7d04021070 con 0x7f7d100cfad0 2023-07-13T22:24:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.964+0000 7f7d0f7fe700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cfc06ace0 0x7f7cfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.964+0000 7f7d0f7fe700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cfc06ace0 0x7f7cfc06d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f60 tx=0x7f7d00007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:59.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:24:59.968+0000 7f7cf7fff700 1 -- 172.21.15.60:0/3572472997 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7d04055e70 con 0x7f7d100cfad0 2023-07-13T22:25:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.176+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7d100d1ad0 con 0x7f7d100cfad0 2023-07-13T22:25:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.177+0000 7f7cf7fff700 1 -- 172.21.15.60:0/3572472997 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7d0402b090 con 0x7f7d100cfad0 2023-07-13T22:25:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:00.160 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:25:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:25:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:25:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:25:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:25:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:25:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:25:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:25:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:25:00.161 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:25:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cfc06ace0 msgr2=0x7f7cfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cfc06ace0 0x7f7cfc06d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f60 tx=0x7f7d00007c90 comp rx=0 tx=0).stop 2023-07-13T22:25:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100cfad0 msgr2=0x7f7d10153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100cfad0 0x7f7d10153220 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009fd0 tx=0x7f7d04004750 comp rx=0 tx=0).stop 2023-07-13T22:25:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 shutdown_connections 2023-07-13T22:25:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7cfc06ace0 0x7f7cfc06d190 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d100d3750 0x7f7d10148a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d100b8400 0x7f7d10152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 --2- 172.21.15.60:0/3572472997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d100cfad0 0x7f7d10153220 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 >> 172.21.15.60:0/3572472997 conn(0x7f7d1006b390 msgr2=0x7f7d100d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 shutdown_connections 2023-07-13T22:25:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.179+0000 7f7d15dff700 1 -- 172.21.15.60:0/3572472997 wait complete. 2023-07-13T22:25:00.180 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:25:00.180 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:25:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.404+0000 7efc96fff700 1 Processor -- start 2023-07-13T22:25:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.404+0000 7efc96fff700 1 -- start start 2023-07-13T22:25:00.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.406+0000 7efc96fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc900c8da0 0x7efc900cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:00.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.406+0000 7efc96fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900ce0f0 0x7efc9006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:00.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.406+0000 7efc96fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc9015cab0 con 0x7efc9015c6e0 2023-07-13T22:25:00.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.406+0000 7efc96fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc900cafd0 con 0x7efc900c8da0 2023-07-13T22:25:00.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.406+0000 7efc96fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc900ce570 con 0x7efc900ce0f0 2023-07-13T22:25:00.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.406+0000 7efc8ffff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc900c8da0 0x7efc900cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:00.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.406+0000 7efc9559c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900ce0f0 0x7efc9006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:00.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.406+0000 7efc94d9b700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efc9015c6e0 0x7efc9015d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:55332/0 (socket says 172.21.15.60:55332) 2023-07-13T22:25:00.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.406+0000 7efc94d9b700 1 -- 172.21.15.60:0/3448657636 learned_addr learned my addr 172.21.15.60:0/3448657636 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:00.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.407+0000 7efc9559c700 1 -- 172.21.15.60:0/3448657636 >> v2:172.21.15.105:3300/0 conn(0x7efc900c8da0 msgr2=0x7efc900cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:00.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.407+0000 7efc9559c700 1 --2- 172.21.15.60:0/3448657636 >> v2:172.21.15.105:3300/0 conn(0x7efc900c8da0 0x7efc900cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.407+0000 7efc9559c700 1 -- 172.21.15.60:0/3448657636 >> v1:172.21.15.105:6789/0 conn(0x7efc9015c6e0 legacy=0x7efc9015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:00.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.407+0000 7efc9559c700 1 -- 172.21.15.60:0/3448657636 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc900ca820 con 0x7efc900ce0f0 2023-07-13T22:25:00.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.407+0000 7efc9559c700 1 --2- 172.21.15.60:0/3448657636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900ce0f0 0x7efc9006cac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc88009d90 tx=0x7efc8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269b779db726d6ed server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:00.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.407+0000 7efc8f7fe700 1 -- 172.21.15.60:0/3448657636 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8800bd20 con 0x7efc900ce0f0 2023-07-13T22:25:00.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.407+0000 7efc8f7fe700 1 -- 172.21.15.60:0/3448657636 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8800be80 con 0x7efc900ce0f0 2023-07-13T22:25:00.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.407+0000 7efc8f7fe700 1 -- 172.21.15.60:0/3448657636 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc880116a0 con 0x7efc900ce0f0 2023-07-13T22:25:00.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.408+0000 7efc96fff700 1 -- 172.21.15.60:0/3448657636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900ce0f0 msgr2=0x7efc9006cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:00.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.408+0000 7efc96fff700 1 --2- 172.21.15.60:0/3448657636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900ce0f0 0x7efc9006cac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc88009d90 tx=0x7efc8800b330 comp rx=0 tx=0).stop 2023-07-13T22:25:00.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.408+0000 7efc96fff700 1 -- 172.21.15.60:0/3448657636 shutdown_connections 2023-07-13T22:25:00.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.408+0000 7efc96fff700 1 --2- 172.21.15.60:0/3448657636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900ce0f0 0x7efc9006cac0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.408+0000 7efc96fff700 1 --2- 172.21.15.60:0/3448657636 >> v2:172.21.15.105:3300/0 conn(0x7efc900c8da0 0x7efc900cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.408+0000 7efc96fff700 1 -- 172.21.15.60:0/3448657636 >> 172.21.15.60:0/3448657636 conn(0x7efc900cf3b0 msgr2=0x7efc900cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.408+0000 7efc96fff700 1 -- 172.21.15.60:0/3448657636 shutdown_connections 2023-07-13T22:25:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.408+0000 7efc96fff700 1 -- 172.21.15.60:0/3448657636 wait complete. 2023-07-13T22:25:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.410+0000 7efc96fff700 1 Processor -- start 2023-07-13T22:25:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.410+0000 7efc96fff700 1 -- start start 2023-07-13T22:25:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc96fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900c8da0 0x7efc900c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc96fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc900ce0f0 0x7efc9014bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc96fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9015c6e0 0x7efc900cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc96fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900aeb00 con 0x7efc9015c6e0 2023-07-13T22:25:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc96fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900c8210 con 0x7efc900ce0f0 2023-07-13T22:25:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc96fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc900c84f0 con 0x7efc900c8da0 2023-07-13T22:25:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc9559c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9015c6e0 0x7efc900cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc94d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900c8da0 0x7efc900c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc8ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc900ce0f0 0x7efc9014bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc94d9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900c8da0 0x7efc900c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:55082/0 (socket says 172.21.15.60:55082) 2023-07-13T22:25:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc94d9b700 1 -- 172.21.15.60:0/3985862340 learned_addr learned my addr 172.21.15.60:0/3985862340 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.411+0000 7efc9559c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9015c6e0 0x7efc900cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41290/0 (socket says 172.21.15.60:41290) 2023-07-13T22:25:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc94d9b700 1 -- 172.21.15.60:0/3985862340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc900ce0f0 msgr2=0x7efc9014bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc94d9b700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc900ce0f0 0x7efc9014bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc94d9b700 1 -- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9015c6e0 msgr2=0x7efc900cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc94d9b700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9015c6e0 0x7efc900cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc94d9b700 1 -- 172.21.15.60:0/3985862340 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc88009a70 con 0x7efc900c8da0 2023-07-13T22:25:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc9559c700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9015c6e0 0x7efc900cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc94d9b700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900c8da0 0x7efc900c7ba0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c480 tx=0x7efc8400c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc8d7fa700 1 -- 172.21.15.60:0/3985862340 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84005d70 con 0x7efc900c8da0 2023-07-13T22:25:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc8d7fa700 1 -- 172.21.15.60:0/3985862340 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8400ce90 con 0x7efc900c8da0 2023-07-13T22:25:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc8d7fa700 1 -- 172.21.15.60:0/3985862340 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc840072d0 con 0x7efc900c8da0 2023-07-13T22:25:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc900aed50 con 0x7efc900c8da0 2023-07-13T22:25:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.412+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9014c080 con 0x7efc900c8da0 2023-07-13T22:25:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.413+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9004e210 con 0x7efc900c8da0 2023-07-13T22:25:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.414+0000 7efc8d7fa700 1 -- 172.21.15.60:0/3985862340 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc84007430 con 0x7efc900c8da0 2023-07-13T22:25:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.416+0000 7efc8d7fa700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc7406ace0 0x7efc7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.416+0000 7efc8d7fa700 1 -- 172.21.15.60:0/3985862340 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efc84088350 con 0x7efc900c8da0 2023-07-13T22:25:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.416+0000 7efc8ffff700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc7406ace0 0x7efc7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.416+0000 7efc8ffff700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc7406ace0 0x7efc7406d190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efc9006d450 tx=0x7efc7c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.421+0000 7efc8d7fa700 1 -- 172.21.15.60:0/3985862340 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efc840537f0 con 0x7efc900c8da0 2023-07-13T22:25:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.629+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efc900c8630 con 0x7efc900c8da0 2023-07-13T22:25:00.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.866+0000 7efc8d7fa700 1 -- 172.21.15.60:0/3985862340 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) v1 ==== 138+0+0 (secure 0 0 0) 0x7efc900c8630 con 0x7efc900c8da0 2023-07-13T22:25:00.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc7406ace0 msgr2=0x7efc7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:00.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc7406ace0 0x7efc7406d190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efc9006d450 tx=0x7efc7c009210 comp rx=0 tx=0).stop 2023-07-13T22:25:00.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900c8da0 msgr2=0x7efc900c7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:00.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900c8da0 0x7efc900c7ba0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c480 tx=0x7efc8400c790 comp rx=0 tx=0).stop 2023-07-13T22:25:00.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 shutdown_connections 2023-07-13T22:25:00.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc7406ace0 0x7efc7406d190 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc9015c6e0 0x7efc900cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc900c8da0 0x7efc900c7ba0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 --2- 172.21.15.60:0/3985862340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc900ce0f0 0x7efc9014bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:00.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 >> 172.21.15.60:0/3985862340 conn(0x7efc900cf3b0 msgr2=0x7efc9015d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:00.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 shutdown_connections 2023-07-13T22:25:00.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:00.871+0000 7efc96fff700 1 -- 172.21.15.60:0/3985862340 wait complete. 2023-07-13T22:25:00.856 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:25:00.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:25:01.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.100+0000 7f03a7dff700 1 Processor -- start 2023-07-13T22:25:01.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.101+0000 7f03a7dff700 1 -- start start 2023-07-13T22:25:01.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.102+0000 7f03a7dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03a00b8270 0x7f03a00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:01.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.102+0000 7f03a7dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a00cd890 con 0x7f03a00cfc00 2023-07-13T22:25:01.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.102+0000 7f03a7dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a00b8640 con 0x7f03a00b1f50 2023-07-13T22:25:01.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.102+0000 7f03a7dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a00cf380 con 0x7f03a00b8270 2023-07-13T22:25:01.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.102+0000 7f03a639c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f03a00cfc00 0x7f03a00d25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51954/0 (socket says 172.21.15.60:51954) 2023-07-13T22:25:01.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.102+0000 7f03a639c700 1 -- 172.21.15.60:0/4053413574 learned_addr learned my addr 172.21.15.60:0/4053413574 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.102+0000 7f03a5b9b700 1 --2- 172.21.15.60:0/4053413574 >> v2:172.21.15.105:3300/0 conn(0x7f03a00b8270 0x7f03a00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.103+0000 7f03a4b99700 1 -- 172.21.15.60:0/4053413574 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367415475 0 0) 0x7f03a00cd890 con 0x7f03a00cfc00 2023-07-13T22:25:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.103+0000 7f03a4b99700 1 -- 172.21.15.60:0/4053413574 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f038c0035b0 con 0x7f03a00cfc00 2023-07-13T22:25:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.103+0000 7f03a4b99700 1 -- 172.21.15.60:0/4053413574 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190135921 0 0) 0x7f03a00b8640 con 0x7f03a00b1f50 2023-07-13T22:25:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.103+0000 7f03a4b99700 1 -- 172.21.15.60:0/4053413574 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a00cd890 con 0x7f03a00b1f50 2023-07-13T22:25:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.103+0000 7f03a5b9b700 1 -- 172.21.15.60:0/4053413574 >> v1:172.21.15.105:6789/0 conn(0x7f03a00b1f50 legacy=0x7f03a00cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.103+0000 7f03a5b9b700 1 -- 172.21.15.60:0/4053413574 >> v1:172.21.15.60:6789/0 conn(0x7f03a00cfc00 legacy=0x7f03a00d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.103+0000 7f03a5b9b700 1 -- 172.21.15.60:0/4053413574 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a00d1290 con 0x7f03a00b8270 2023-07-13T22:25:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.103+0000 7f03a5b9b700 1 --2- 172.21.15.60:0/4053413574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 0x7f03a00cecc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f039c009840 tx=0x7f039c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d9bd0444a9fae9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a4b99700 1 -- 172.21.15.60:0/4053413574 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c011830 con 0x7f03a00b8270 2023-07-13T22:25:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a4b99700 1 -- 172.21.15.60:0/4053413574 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c011990 con 0x7f03a00b8270 2023-07-13T22:25:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a4b99700 1 -- 172.21.15.60:0/4053413574 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c011d00 con 0x7f03a00b8270 2023-07-13T22:25:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a7dff700 1 -- 172.21.15.60:0/4053413574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 msgr2=0x7f03a00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a7dff700 1 --2- 172.21.15.60:0/4053413574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 0x7f03a00cecc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f039c009840 tx=0x7f039c009b50 comp rx=0 tx=0).stop 2023-07-13T22:25:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a7dff700 1 -- 172.21.15.60:0/4053413574 shutdown_connections 2023-07-13T22:25:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a7dff700 1 --2- 172.21.15.60:0/4053413574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 0x7f03a00cecc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a7dff700 1 -- 172.21.15.60:0/4053413574 >> 172.21.15.60:0/4053413574 conn(0x7f03a006b330 msgr2=0x7f03a006b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a7dff700 1 -- 172.21.15.60:0/4053413574 shutdown_connections 2023-07-13T22:25:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.104+0000 7f03a7dff700 1 -- 172.21.15.60:0/4053413574 wait complete. 2023-07-13T22:25:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.106+0000 7f03a7dff700 1 Processor -- start 2023-07-13T22:25:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.106+0000 7f03a7dff700 1 -- start start 2023-07-13T22:25:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.107+0000 7f03a7dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a00b1f50 0x7f03a015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.107+0000 7f03a7dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 0x7f03a00c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.107+0000 7f03a7dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03a00cfc00 0x7f03a00c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.107+0000 7f03a7dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a01f30a0 con 0x7f03a00b1f50 2023-07-13T22:25:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.107+0000 7f03a7dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a015af50 con 0x7f03a00b8270 2023-07-13T22:25:01.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.107+0000 7f03a7dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a00c71f0 con 0x7f03a00cfc00 2023-07-13T22:25:01.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.107+0000 7f03a5b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a00b1f50 0x7f03a015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:01.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a5b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a00b1f50 0x7f03a015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41302/0 (socket says 172.21.15.60:41302) 2023-07-13T22:25:01.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a5b9b700 1 -- 172.21.15.60:0/2754693045 learned_addr learned my addr 172.21.15.60:0/2754693045 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:01.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a539a700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 0x7f03a00c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:01.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a639c700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03a00cfc00 0x7f03a00c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:01.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a5b9b700 1 -- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03a00cfc00 msgr2=0x7f03a00c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a5b9b700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03a00cfc00 0x7f03a00c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a5b9b700 1 -- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 msgr2=0x7f03a00c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a5b9b700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 0x7f03a00c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a5b9b700 1 -- 172.21.15.60:0/2754693045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0094a0 con 0x7f03a00b1f50 2023-07-13T22:25:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a539a700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 0x7f03a00c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a639c700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03a00cfc00 0x7f03a00c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03a5b9b700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a00b1f50 0x7f03a015a090 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f039c009fd0 tx=0x7f039c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03967fc700 1 -- 172.21.15.60:0/2754693045 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0045d0 con 0x7f03a00b1f50 2023-07-13T22:25:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03967fc700 1 -- 172.21.15.60:0/2754693045 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c01f070 con 0x7f03a00b1f50 2023-07-13T22:25:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.108+0000 7f03967fc700 1 -- 172.21.15.60:0/2754693045 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0038c0 con 0x7f03a00b1f50 2023-07-13T22:25:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.109+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a00c74d0 con 0x7f03a00b1f50 2023-07-13T22:25:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.109+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a00c7a90 con 0x7f03a00b1f50 2023-07-13T22:25:01.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.110+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a004e210 con 0x7f03a00b1f50 2023-07-13T22:25:01.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.111+0000 7f03967fc700 1 -- 172.21.15.60:0/2754693045 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f039c003a20 con 0x7f03a00b1f50 2023-07-13T22:25:01.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.112+0000 7f03967fc700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f038406ace0 0x7f038406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:01.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.112+0000 7f03967fc700 1 -- 172.21.15.60:0/2754693045 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f039c021070 con 0x7f03a00b1f50 2023-07-13T22:25:01.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.112+0000 7f03a539a700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f038406ace0 0x7f038406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:01.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.113+0000 7f03a539a700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f038406ace0 0x7f038406d190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f03a015b170 tx=0x7f0390009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.116+0000 7f03967fc700 1 -- 172.21.15.60:0/2754693045 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f039c056350 con 0x7f03a00b1f50 2023-07-13T22:25:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.323+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f03a00c7c70 con 0x7f03a00b1f50 2023-07-13T22:25:01.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.871+0000 7f03967fc700 1 -- 172.21.15.60:0/2754693045 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) v1 ==== 170+0+0 (secure 0 0 0) 0x7f039c0150d0 con 0x7f03a00b1f50 2023-07-13T22:25:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.879+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f038406ace0 msgr2=0x7f038406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.879+0000 7f03a7dff700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f038406ace0 0x7f038406d190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f03a015b170 tx=0x7f0390009040 comp rx=0 tx=0).stop 2023-07-13T22:25:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.879+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a00b1f50 msgr2=0x7f03a015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.879+0000 7f03a7dff700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a00b1f50 0x7f03a015a090 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f039c009fd0 tx=0x7f039c004750 comp rx=0 tx=0).stop 2023-07-13T22:25:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.883+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 shutdown_connections 2023-07-13T22:25:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.883+0000 7f03a7dff700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f038406ace0 0x7f038406d190 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.883+0000 7f03a7dff700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03a00b1f50 0x7f03a015a090 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.883+0000 7f03a7dff700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03a00cfc00 0x7f03a00c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.883+0000 7f03a7dff700 1 --2- 172.21.15.60:0/2754693045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a00b8270 0x7f03a00c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.883+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 >> 172.21.15.60:0/2754693045 conn(0x7f03a006b330 msgr2=0x7f03a00b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.888+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 shutdown_connections 2023-07-13T22:25:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:01.889+0000 7f03a7dff700 1 -- 172.21.15.60:0/2754693045 wait complete. 2023-07-13T22:25:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:25:01.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:25:02.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.121+0000 7f5061fff700 1 Processor -- start 2023-07-13T22:25:02.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.122+0000 7f5061fff700 1 -- start start 2023-07-13T22:25:02.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f5061fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f505c0b8200 0x7f505c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:02.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f5061fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b1ac0 0x7f505c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:02.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f5061fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f505c0cede0 0x7f505c15ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:02.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f5061fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0d1870 con 0x7f505c0b8200 2023-07-13T22:25:02.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f5061fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0cf370 con 0x7f505c0cede0 2023-07-13T22:25:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f5061fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0d12b0 con 0x7f505c0b1ac0 2023-07-13T22:25:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f505b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f505c0b8200 0x7f505c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f505b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f505c0b8200 0x7f505c0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41318/0 (socket says 172.21.15.60:41318) 2023-07-13T22:25:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f505b7fe700 1 -- 172.21.15.60:0/2007626026 learned_addr learned my addr 172.21.15.60:0/2007626026 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f505bfff700 1 --2- 172.21.15.60:0/2007626026 >> v2:172.21.15.105:3300/0 conn(0x7f505c0cede0 0x7f505c15ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:02.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.123+0000 7f505affd700 1 --2- 172.21.15.60:0/2007626026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b1ac0 0x7f505c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:02.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505b7fe700 1 -- 172.21.15.60:0/2007626026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b1ac0 msgr2=0x7f505c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:02.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505b7fe700 1 --2- 172.21.15.60:0/2007626026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b1ac0 0x7f505c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505b7fe700 1 -- 172.21.15.60:0/2007626026 >> v2:172.21.15.105:3300/0 conn(0x7f505c0cede0 msgr2=0x7f505c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:02.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505b7fe700 1 --2- 172.21.15.60:0/2007626026 >> v2:172.21.15.105:3300/0 conn(0x7f505c0cede0 0x7f505c15ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505b7fe700 1 -- 172.21.15.60:0/2007626026 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c0d1590 con 0x7f505c0b8200 2023-07-13T22:25:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505bfff700 1 --2- 172.21.15.60:0/2007626026 >> v2:172.21.15.105:3300/0 conn(0x7f505c0cede0 0x7f505c15ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505b7fe700 1 --2- 172.21.15.60:0/2007626026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b8200 0x7f505c0cf640 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f505c0b8d70 tx=0x7f504c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f5560231f1abe0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505a7fc700 1 -- 172.21.15.60:0/2007626026 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011830 con 0x7f505c0b8200 2023-07-13T22:25:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505a7fc700 1 -- 172.21.15.60:0/2007626026 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c011990 con 0x7f505c0b8200 2023-07-13T22:25:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.124+0000 7f505a7fc700 1 -- 172.21.15.60:0/2007626026 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011d20 con 0x7f505c0b8200 2023-07-13T22:25:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.125+0000 7f5061fff700 1 -- 172.21.15.60:0/2007626026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b8200 msgr2=0x7f505c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.125+0000 7f5061fff700 1 --2- 172.21.15.60:0/2007626026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b8200 0x7f505c0cf640 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f505c0b8d70 tx=0x7f504c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.125+0000 7f5061fff700 1 -- 172.21.15.60:0/2007626026 shutdown_connections 2023-07-13T22:25:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.125+0000 7f5061fff700 1 --2- 172.21.15.60:0/2007626026 >> v2:172.21.15.105:3300/0 conn(0x7f505c0cede0 0x7f505c15ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.125+0000 7f5061fff700 1 --2- 172.21.15.60:0/2007626026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b8200 0x7f505c0cf640 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.125+0000 7f5061fff700 1 --2- 172.21.15.60:0/2007626026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b1ac0 0x7f505c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.125+0000 7f5061fff700 1 -- 172.21.15.60:0/2007626026 >> 172.21.15.60:0/2007626026 conn(0x7f505c06b2f0 msgr2=0x7f505c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.125+0000 7f5061fff700 1 -- 172.21.15.60:0/2007626026 shutdown_connections 2023-07-13T22:25:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.125+0000 7f5061fff700 1 -- 172.21.15.60:0/2007626026 wait complete. 2023-07-13T22:25:02.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.127+0000 7f5061fff700 1 Processor -- start 2023-07-13T22:25:02.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.127+0000 7f5061fff700 1 -- start start 2023-07-13T22:25:02.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f5061fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b1ac0 0x7f505c1e5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:02.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f5061fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b8200 0x7f505c0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:02.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f5061fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505c0cede0 0x7f505c1f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:02.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f5061fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c1f3c20 con 0x7f505c0b1ac0 2023-07-13T22:25:02.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f5061fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c1e4d50 con 0x7f505c0cede0 2023-07-13T22:25:02.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f5061fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0cb2d0 con 0x7f505c0b8200 2023-07-13T22:25:02.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b1ac0 0x7f505c1e5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:02.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b1ac0 0x7f505c1e5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41332/0 (socket says 172.21.15.60:41332) 2023-07-13T22:25:02.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505b7fe700 1 -- 172.21.15.60:0/1480114934 learned_addr learned my addr 172.21.15.60:0/1480114934 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:02.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505bfff700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505c0cede0 0x7f505c1f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:02.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505b7fe700 1 -- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b8200 msgr2=0x7f505c0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:02.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505affd700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b8200 0x7f505c0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:02.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505b7fe700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b8200 0x7f505c0cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505b7fe700 1 -- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505c0cede0 msgr2=0x7f505c1f7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:02.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505b7fe700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505c0cede0 0x7f505c1f7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505b7fe700 1 -- 172.21.15.60:0/1480114934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c009550 con 0x7f505c0b1ac0 2023-07-13T22:25:02.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505bfff700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505c0cede0 0x7f505c1f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:02.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.128+0000 7f505affd700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b8200 0x7f505c0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:02.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.129+0000 7f505b7fe700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b1ac0 0x7f505c1e5170 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f504c000c00 tx=0x7f504c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:02.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.129+0000 7f5043fff700 1 -- 172.21.15.60:0/1480114934 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c004380 con 0x7f505c0b1ac0 2023-07-13T22:25:02.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.129+0000 7f5043fff700 1 -- 172.21.15.60:0/1480114934 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c01f070 con 0x7f505c0b1ac0 2023-07-13T22:25:02.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.129+0000 7f5043fff700 1 -- 172.21.15.60:0/1480114934 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c0036a0 con 0x7f505c0b1ac0 2023-07-13T22:25:02.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.129+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c15a9f0 con 0x7f505c0b1ac0 2023-07-13T22:25:02.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.129+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c1e43b0 con 0x7f505c0b1ac0 2023-07-13T22:25:02.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.130+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c04e210 con 0x7f505c0b1ac0 2023-07-13T22:25:02.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.131+0000 7f5043fff700 1 -- 172.21.15.60:0/1480114934 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f504c021030 con 0x7f505c0b1ac0 2023-07-13T22:25:02.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.132+0000 7f5043fff700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f504806ace0 0x7f504806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:02.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.132+0000 7f5043fff700 1 -- 172.21.15.60:0/1480114934 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f504c08a720 con 0x7f505c0b1ac0 2023-07-13T22:25:02.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.132+0000 7f505affd700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f504806ace0 0x7f504806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:02.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.133+0000 7f505affd700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f504806ace0 0x7f504806d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f505c1f7270 tx=0x7f5050008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:02.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.137+0000 7f5043fff700 1 -- 172.21.15.60:0/1480114934 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f504c055820 con 0x7f505c0b1ac0 2023-07-13T22:25:02.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.390+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f505c15bb70 con 0x7f505c0b1ac0 2023-07-13T22:25:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.877+0000 7f5043fff700 1 -- 172.21.15.60:0/1480114934 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7f505c15bb70 con 0x7f505c0b1ac0 2023-07-13T22:25:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.883+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f504806ace0 msgr2=0x7f504806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.884+0000 7f5061fff700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f504806ace0 0x7f504806d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f505c1f7270 tx=0x7f5050008040 comp rx=0 tx=0).stop 2023-07-13T22:25:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.884+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b1ac0 msgr2=0x7f505c1e5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.884+0000 7f5061fff700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b1ac0 0x7f505c1e5170 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f504c000c00 tx=0x7f504c004750 comp rx=0 tx=0).stop 2023-07-13T22:25:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.888+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 shutdown_connections 2023-07-13T22:25:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.888+0000 7f5061fff700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f504806ace0 0x7f504806d190 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.888+0000 7f5061fff700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505c0b1ac0 0x7f505c1e5170 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.888+0000 7f5061fff700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f505c0cede0 0x7f505c1f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.888+0000 7f5061fff700 1 --2- 172.21.15.60:0/1480114934 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f505c0b8200 0x7f505c0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.888+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 >> 172.21.15.60:0/1480114934 conn(0x7f505c06b2f0 msgr2=0x7f505c15c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.891+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 shutdown_connections 2023-07-13T22:25:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:02.892+0000 7f5061fff700 1 -- 172.21.15.60:0/1480114934 wait complete. 2023-07-13T22:25:02.890 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 11 and data pool 12 2023-07-13T22:25:02.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:25:03.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.129+0000 7fb7e55ff700 1 Processor -- start 2023-07-13T22:25:03.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.130+0000 7fb7e55ff700 1 -- start start 2023-07-13T22:25:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7e55ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00b8400 0x7fb7e00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7e55ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7e00cfca0 0x7fb7e00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7e55ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e00d0ac0 con 0x7fb7e00d3750 2023-07-13T22:25:03.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7e55ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e00b87d0 con 0x7fb7e00cfca0 2023-07-13T22:25:03.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7e55ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00b4eb0 con 0x7fb7e00b8400 2023-07-13T22:25:03.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7df7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7e00cfca0 0x7fb7e00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7de7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb7e00d3750 0x7fb7e00cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51970/0 (socket says 172.21.15.60:51970) 2023-07-13T22:25:03.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7de7fc700 1 -- 172.21.15.60:0/3755115893 learned_addr learned my addr 172.21.15.60:0/3755115893 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:03.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7df7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7e00cfca0 0x7fb7e00d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41342/0 (socket says 172.21.15.60:41342) 2023-07-13T22:25:03.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.131+0000 7fb7deffd700 1 --2- 172.21.15.60:0/3755115893 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00b8400 0x7fb7e00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.132+0000 7fb7df7fe700 1 -- 172.21.15.60:0/3755115893 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00b8400 msgr2=0x7fb7e00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.132+0000 7fb7df7fe700 1 --2- 172.21.15.60:0/3755115893 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00b8400 0x7fb7e00cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.132+0000 7fb7df7fe700 1 -- 172.21.15.60:0/3755115893 >> v1:172.21.15.60:6789/0 conn(0x7fb7e00d3750 legacy=0x7fb7e00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.132+0000 7fb7df7fe700 1 -- 172.21.15.60:0/3755115893 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e00b5190 con 0x7fb7e00cfca0 2023-07-13T22:25:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.132+0000 7fb7df7fe700 1 --2- 172.21.15.60:0/3755115893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00cfca0 0x7fb7e00d0f50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466d16762e082f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.132+0000 7fb7ddffb700 1 -- 172.21.15.60:0/3755115893 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d800e070 con 0x7fb7e00cfca0 2023-07-13T22:25:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.132+0000 7fb7ddffb700 1 -- 172.21.15.60:0/3755115893 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d800bb20 con 0x7fb7e00cfca0 2023-07-13T22:25:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.132+0000 7fb7ddffb700 1 -- 172.21.15.60:0/3755115893 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8012430 con 0x7fb7e00cfca0 2023-07-13T22:25:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.133+0000 7fb7e55ff700 1 -- 172.21.15.60:0/3755115893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00cfca0 msgr2=0x7fb7e00d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.133+0000 7fb7e55ff700 1 --2- 172.21.15.60:0/3755115893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00cfca0 0x7fb7e00d0f50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d800b330 comp rx=0 tx=0).stop 2023-07-13T22:25:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.133+0000 7fb7e55ff700 1 -- 172.21.15.60:0/3755115893 shutdown_connections 2023-07-13T22:25:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.133+0000 7fb7e55ff700 1 --2- 172.21.15.60:0/3755115893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00cfca0 0x7fb7e00d0f50 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.133+0000 7fb7e55ff700 1 --2- 172.21.15.60:0/3755115893 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00b8400 0x7fb7e00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.133+0000 7fb7e55ff700 1 -- 172.21.15.60:0/3755115893 >> 172.21.15.60:0/3755115893 conn(0x7fb7e006b390 msgr2=0x7fb7e006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:03.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.133+0000 7fb7e55ff700 1 -- 172.21.15.60:0/3755115893 shutdown_connections 2023-07-13T22:25:03.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.133+0000 7fb7e55ff700 1 -- 172.21.15.60:0/3755115893 wait complete. 2023-07-13T22:25:03.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.134+0000 7fb7e55ff700 1 Processor -- start 2023-07-13T22:25:03.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.135+0000 7fb7e55ff700 1 -- start start 2023-07-13T22:25:03.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.135+0000 7fb7e55ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00b8400 0x7fb7e0155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.135+0000 7fb7e55ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00cfca0 0x7fb7e014ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.135+0000 7fb7e55ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e00d3750 0x7fb7e00cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7e55ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00c8e10 con 0x7fb7e00b8400 2023-07-13T22:25:03.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7e55ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00c8fa0 con 0x7fb7e00d3750 2023-07-13T22:25:03.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7e55ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00c94d0 con 0x7fb7e00cfca0 2023-07-13T22:25:03.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7deffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00b8400 0x7fb7e0155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7deffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00b8400 0x7fb7e0155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41350/0 (socket says 172.21.15.60:41350) 2023-07-13T22:25:03.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7deffd700 1 -- 172.21.15.60:0/4076560421 learned_addr learned my addr 172.21.15.60:0/4076560421 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:03.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7de7fc700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00cfca0 0x7fb7e014ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7df7fe700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e00d3750 0x7fb7e00cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7deffd700 1 -- 172.21.15.60:0/4076560421 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00cfca0 msgr2=0x7fb7e014ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7deffd700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00cfca0 0x7fb7e014ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7deffd700 1 -- 172.21.15.60:0/4076560421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e00d3750 msgr2=0x7fb7e00cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7deffd700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e00d3750 0x7fb7e00cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7deffd700 1 -- 172.21.15.60:0/4076560421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d8009a70 con 0x7fb7e00b8400 2023-07-13T22:25:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7de7fc700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00cfca0 0x7fb7e014ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.136+0000 7fb7deffd700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00b8400 0x7fb7e0155300 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000e8a0 tx=0x7fb7d000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.137+0000 7fb7c77fe700 1 -- 172.21.15.60:0/4076560421 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0009590 con 0x7fb7e00b8400 2023-07-13T22:25:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.137+0000 7fb7c77fe700 1 -- 172.21.15.60:0/4076560421 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d000eea0 con 0x7fb7e00b8400 2023-07-13T22:25:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.137+0000 7fb7c77fe700 1 -- 172.21.15.60:0/4076560421 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000ca50 con 0x7fb7e00b8400 2023-07-13T22:25:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.137+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e00c91f0 con 0x7fb7e00b8400 2023-07-13T22:25:03.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.137+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e00c74d0 con 0x7fb7e00b8400 2023-07-13T22:25:03.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.138+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e00cd740 con 0x7fb7e00b8400 2023-07-13T22:25:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.139+0000 7fb7c77fe700 1 -- 172.21.15.60:0/4076560421 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7d000cbb0 con 0x7fb7e00b8400 2023-07-13T22:25:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.140+0000 7fb7c77fe700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7cc06ace0 0x7fb7cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.140+0000 7fb7c77fe700 1 -- 172.21.15.60:0/4076560421 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7d0089450 con 0x7fb7e00b8400 2023-07-13T22:25:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.140+0000 7fb7de7fc700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7cc06ace0 0x7fb7cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.141+0000 7fb7de7fc700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7cc06ace0 0x7fb7cc06d190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00cb1d0 tx=0x7fb7d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:03.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.144+0000 7fb7c77fe700 1 -- 172.21.15.60:0/4076560421 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb7d0057f80 con 0x7fb7e00b8400 2023-07-13T22:25:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.358+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7e00b2190 con 0x7fb7e00b8400 2023-07-13T22:25:03.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.359+0000 7fb7c77fe700 1 -- 172.21.15.60:0/4076560421 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18273 (secure 0 0 0) 0x7fb7d0057b10 con 0x7fb7e00b8400 2023-07-13T22:25:03.342 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:03.342 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":74,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:25:02.392358+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:25:00.672661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-13T22:25:01.364844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:25:03.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7cc06ace0 msgr2=0x7fb7cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7cc06ace0 0x7fb7cc06d190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00cb1d0 tx=0x7fb7d4009040 comp rx=0 tx=0).stop 2023-07-13T22:25:03.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00b8400 msgr2=0x7fb7e0155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00b8400 0x7fb7e0155300 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000e8a0 tx=0x7fb7d000ec60 comp rx=0 tx=0).stop 2023-07-13T22:25:03.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 shutdown_connections 2023-07-13T22:25:03.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7cc06ace0 0x7fb7cc06d190 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7e00b8400 0x7fb7e0155300 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e00d3750 0x7fb7e00cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 --2- 172.21.15.60:0/4076560421 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e00cfca0 0x7fb7e014ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 >> 172.21.15.60:0/4076560421 conn(0x7fb7e006b390 msgr2=0x7fb7e00b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 shutdown_connections 2023-07-13T22:25:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.362+0000 7fb7e55ff700 1 -- 172.21.15.60:0/4076560421 wait complete. 2023-07-13T22:25:03.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:25:03.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.584+0000 7f82bf1ff700 1 Processor -- start 2023-07-13T22:25:03.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.584+0000 7f82bf1ff700 1 -- start start 2023-07-13T22:25:03.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.585+0000 7f82bf1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f82b8159170 0x7f82b80c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.585+0000 7f82bf1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 0x7f82b8070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.585+0000 7f82bf1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82b80707b0 0x7f82b80cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.585+0000 7f82bf1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b815d970 con 0x7f82b80707b0 2023-07-13T22:25:03.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.585+0000 7f82bf1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b8070c10 con 0x7f82b8159170 2023-07-13T22:25:03.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.585+0000 7f82bf1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b815cd60 con 0x7f82b80c5040 2023-07-13T22:25:03.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bd79c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82b80707b0 0x7f82b80cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bd79c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82b80707b0 0x7f82b80cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41362/0 (socket says 172.21.15.60:41362) 2023-07-13T22:25:03.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bd79c700 1 -- 172.21.15.60:0/1339683059 learned_addr learned my addr 172.21.15.60:0/1339683059 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:03.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bcf9b700 1 --2- 172.21.15.60:0/1339683059 >> v2:172.21.15.105:3300/0 conn(0x7f82b8159170 0x7f82b80c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82b7fff700 1 --2- 172.21.15.60:0/1339683059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 0x7f82b8070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bd79c700 1 -- 172.21.15.60:0/1339683059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 msgr2=0x7f82b8070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:03.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bd79c700 1 --2- 172.21.15.60:0/1339683059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 0x7f82b8070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bd79c700 1 -- 172.21.15.60:0/1339683059 >> v2:172.21.15.105:3300/0 conn(0x7f82b8159170 msgr2=0x7f82b80c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:03.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bd79c700 1 --2- 172.21.15.60:0/1339683059 >> v2:172.21.15.105:3300/0 conn(0x7f82b8159170 0x7f82b80c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bd79c700 1 -- 172.21.15.60:0/1339683059 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b815d040 con 0x7f82b80707b0 2023-07-13T22:25:03.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.586+0000 7f82bd79c700 1 --2- 172.21.15.60:0/1339683059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 0x7f82b80cc2c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f82b8067080 tx=0x7f82b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c11af7fa9efb3e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:03.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82b77fe700 1 -- 172.21.15.60:0/1339683059 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000e070 con 0x7f82b80707b0 2023-07-13T22:25:03.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82b77fe700 1 -- 172.21.15.60:0/1339683059 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b000b9c0 con 0x7f82b80707b0 2023-07-13T22:25:03.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82b77fe700 1 -- 172.21.15.60:0/1339683059 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000bd70 con 0x7f82b80707b0 2023-07-13T22:25:03.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82bf1ff700 1 -- 172.21.15.60:0/1339683059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 msgr2=0x7f82b80cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/1339683059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 0x7f82b80cc2c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f82b8067080 tx=0x7f82b000b330 comp rx=0 tx=0).stop 2023-07-13T22:25:03.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82bf1ff700 1 -- 172.21.15.60:0/1339683059 shutdown_connections 2023-07-13T22:25:03.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/1339683059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 0x7f82b80cc2c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/1339683059 >> v2:172.21.15.105:3300/0 conn(0x7f82b8159170 0x7f82b80c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/1339683059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 0x7f82b8070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82bf1ff700 1 -- 172.21.15.60:0/1339683059 >> 172.21.15.60:0/1339683059 conn(0x7f82b80cf410 msgr2=0x7f82b80cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:03.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82bf1ff700 1 -- 172.21.15.60:0/1339683059 shutdown_connections 2023-07-13T22:25:03.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.587+0000 7f82bf1ff700 1 -- 172.21.15.60:0/1339683059 wait complete. 2023-07-13T22:25:03.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.589+0000 7f82bf1ff700 1 Processor -- start 2023-07-13T22:25:03.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.589+0000 7f82bf1ff700 1 -- start start 2023-07-13T22:25:03.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.590+0000 7f82bf1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 0x7f82b81f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.590+0000 7f82bf1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 0x7f82b814d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.590+0000 7f82bf1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b8159170 0x7f82b80af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.590+0000 7f82bf1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b814dd20 con 0x7f82b80707b0 2023-07-13T22:25:03.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.590+0000 7f82bf1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80afc50 con 0x7f82b8159170 2023-07-13T22:25:03.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.590+0000 7f82bf1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80aff30 con 0x7f82b80c5040 2023-07-13T22:25:03.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.590+0000 7f82bcf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 0x7f82b81f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.590+0000 7f82bcf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 0x7f82b81f2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41372/0 (socket says 172.21.15.60:41372) 2023-07-13T22:25:03.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.590+0000 7f82bcf9b700 1 -- 172.21.15.60:0/4034841505 learned_addr learned my addr 172.21.15.60:0/4034841505 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:03.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82b7fff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 0x7f82b814d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82bd79c700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b8159170 0x7f82b80af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82bcf9b700 1 -- 172.21.15.60:0/4034841505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 msgr2=0x7f82b814d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82bcf9b700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 0x7f82b814d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82bcf9b700 1 -- 172.21.15.60:0/4034841505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b8159170 msgr2=0x7f82b80af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82bcf9b700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b8159170 0x7f82b80af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82bcf9b700 1 -- 172.21.15.60:0/4034841505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b0009b30 con 0x7f82b80707b0 2023-07-13T22:25:03.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82b7fff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 0x7f82b814d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:03.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82bcf9b700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 0x7f82b81f2b20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e810 tx=0x7f82ac00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:03.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82b57fa700 1 -- 172.21.15.60:0/4034841505 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac013610 con 0x7f82b80707b0 2023-07-13T22:25:03.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82b57fa700 1 -- 172.21.15.60:0/4034841505 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac013770 con 0x7f82b80707b0 2023-07-13T22:25:03.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82b57fa700 1 -- 172.21.15.60:0/4034841505 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac013b20 con 0x7f82b80707b0 2023-07-13T22:25:03.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.591+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b81f2320 con 0x7f82b80707b0 2023-07-13T22:25:03.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.592+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b81f2010 con 0x7f82b80707b0 2023-07-13T22:25:03.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.593+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b80ad860 con 0x7f82b80707b0 2023-07-13T22:25:03.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.593+0000 7f82b57fa700 1 -- 172.21.15.60:0/4034841505 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82ac013c80 con 0x7f82b80707b0 2023-07-13T22:25:03.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.595+0000 7f82b57fa700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829c06ace0 0x7f829c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:03.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.595+0000 7f82b57fa700 1 -- 172.21.15.60:0/4034841505 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82ac089050 con 0x7f82b80707b0 2023-07-13T22:25:03.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.596+0000 7f82b7fff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829c06ace0 0x7f829c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:03.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.596+0000 7f82b7fff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829c06ace0 0x7f829c06d190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f82a4007a50 tx=0x7f82a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:03.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.600+0000 7f82b57fa700 1 -- 172.21.15.60:0/4034841505 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f82ac057ba0 con 0x7f82b80707b0 2023-07-13T22:25:03.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.807+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82b8071440 con 0x7f82b80707b0 2023-07-13T22:25:03.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.808+0000 7f82b57fa700 1 -- 172.21.15.60:0/4034841505 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18273 (secure 0 0 0) 0x7f82ac057730 con 0x7f82b80707b0 2023-07-13T22:25:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:03.791 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":74,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:25:02.392358+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:25:00.672661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-13T22:25:01.364844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:25:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829c06ace0 msgr2=0x7f829c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829c06ace0 0x7f829c06d190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f82a4007a50 tx=0x7f82a4008040 comp rx=0 tx=0).stop 2023-07-13T22:25:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 msgr2=0x7f82b81f2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 0x7f82b81f2b20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e810 tx=0x7f82ac00eb20 comp rx=0 tx=0).stop 2023-07-13T22:25:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 shutdown_connections 2023-07-13T22:25:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829c06ace0 0x7f829c06d190 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b80707b0 0x7f82b81f2b20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b8159170 0x7f82b80af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 --2- 172.21.15.60:0/4034841505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c5040 0x7f82b814d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 >> 172.21.15.60:0/4034841505 conn(0x7f82b80cf410 msgr2=0x7f82b80ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 shutdown_connections 2023-07-13T22:25:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:03.811+0000 7f82bf1ff700 1 -- 172.21.15.60:0/4034841505 wait complete. 2023-07-13T22:25:03.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:25:03.838 INFO:tasks.ceph.mds.c.smithi060.stderr:2023-07-13T22:25:03.856+0000 7fb5f03f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:25:04.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.039+0000 7fd6fddff700 1 Processor -- start 2023-07-13T22:25:04.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.040+0000 7fd6fddff700 1 -- start start 2023-07-13T22:25:04.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6fddff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd6f80b8200 0x7fd6f80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6fddff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80b1ac0 0x7fd6f80cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6fddff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6f80cede0 0x7fd6f815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6fddff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f80d1870 con 0x7fd6f80b8200 2023-07-13T22:25:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6fddff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f80cf370 con 0x7fd6f80cede0 2023-07-13T22:25:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6fddff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80d12b0 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6f77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd6f80b8200 0x7fd6f80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6f77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd6f80b8200 0x7fd6f80cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41378/0 (socket says 172.21.15.60:41378) 2023-07-13T22:25:04.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6f77fe700 1 -- 172.21.15.60:0/1599229159 learned_addr learned my addr 172.21.15.60:0/1599229159 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:04.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6f7fff700 1 --2- 172.21.15.60:0/1599229159 >> v2:172.21.15.105:3300/0 conn(0x7fd6f80cede0 0x7fd6f815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:04.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.041+0000 7fd6f6ffd700 1 --2- 172.21.15.60:0/1599229159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80b1ac0 0x7fd6f80cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:04.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f77fe700 1 -- 172.21.15.60:0/1599229159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80b1ac0 msgr2=0x7fd6f80cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:04.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f77fe700 1 --2- 172.21.15.60:0/1599229159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80b1ac0 0x7fd6f80cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f77fe700 1 -- 172.21.15.60:0/1599229159 >> v2:172.21.15.105:3300/0 conn(0x7fd6f80cede0 msgr2=0x7fd6f815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:04.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f77fe700 1 --2- 172.21.15.60:0/1599229159 >> v2:172.21.15.105:3300/0 conn(0x7fd6f80cede0 0x7fd6f815ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f77fe700 1 -- 172.21.15.60:0/1599229159 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f80d1590 con 0x7fd6f80b8200 2023-07-13T22:25:04.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f6ffd700 1 --2- 172.21.15.60:0/1599229159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80b1ac0 0x7fd6f80cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:04.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f77fe700 1 --2- 172.21.15.60:0/1599229159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b8200 0x7fd6f80cf640 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80b8d70 tx=0x7fd6e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89030e80ab504aba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:04.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f67fc700 1 -- 172.21.15.60:0/1599229159 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8011830 con 0x7fd6f80b8200 2023-07-13T22:25:04.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f67fc700 1 -- 172.21.15.60:0/1599229159 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e8011990 con 0x7fd6f80b8200 2023-07-13T22:25:04.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.042+0000 7fd6f67fc700 1 -- 172.21.15.60:0/1599229159 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8011d20 con 0x7fd6f80b8200 2023-07-13T22:25:04.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.043+0000 7fd6fddff700 1 -- 172.21.15.60:0/1599229159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b8200 msgr2=0x7fd6f80cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:04.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.043+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1599229159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b8200 0x7fd6f80cf640 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80b8d70 tx=0x7fd6e8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.043+0000 7fd6fddff700 1 -- 172.21.15.60:0/1599229159 shutdown_connections 2023-07-13T22:25:04.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.043+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1599229159 >> v2:172.21.15.105:3300/0 conn(0x7fd6f80cede0 0x7fd6f815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.043+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1599229159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b8200 0x7fd6f80cf640 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.043+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1599229159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80b1ac0 0x7fd6f80cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.043+0000 7fd6fddff700 1 -- 172.21.15.60:0/1599229159 >> 172.21.15.60:0/1599229159 conn(0x7fd6f806b2f0 msgr2=0x7fd6f806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:04.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.043+0000 7fd6fddff700 1 -- 172.21.15.60:0/1599229159 shutdown_connections 2023-07-13T22:25:04.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.043+0000 7fd6fddff700 1 -- 172.21.15.60:0/1599229159 wait complete. 2023-07-13T22:25:04.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.046+0000 7fd6fddff700 1 Processor -- start 2023-07-13T22:25:04.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.046+0000 7fd6fddff700 1 -- start start 2023-07-13T22:25:04.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6fddff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b1ac0 0x7fd6f80c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:04.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6fddff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6f80b8200 0x7fd6f81449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:04.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6fddff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80cede0 0x7fd6f81f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6fddff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80c9a50 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6fddff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8144f00 con 0x7fd6f80b8200 2023-07-13T22:25:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6fddff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80c8dd0 con 0x7fd6f80cede0 2023-07-13T22:25:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6f77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b1ac0 0x7fd6f80c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6f77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b1ac0 0x7fd6f80c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41394/0 (socket says 172.21.15.60:41394) 2023-07-13T22:25:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6f77fe700 1 -- 172.21.15.60:0/1370585220 learned_addr learned my addr 172.21.15.60:0/1370585220 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.047+0000 7fd6f6ffd700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6f80b8200 0x7fd6f81449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6f7fff700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80cede0 0x7fd6f81f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6f77fe700 1 -- 172.21.15.60:0/1370585220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80cede0 msgr2=0x7fd6f81f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6f77fe700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80cede0 0x7fd6f81f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6f77fe700 1 -- 172.21.15.60:0/1370585220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6f80b8200 msgr2=0x7fd6f81449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6f77fe700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6f80b8200 0x7fd6f81449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6f77fe700 1 -- 172.21.15.60:0/1370585220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8009550 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6f7fff700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80cede0 0x7fd6f81f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6f77fe700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b1ac0 0x7fd6f80c8730 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8000c00 tx=0x7fd6e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6dffff700 1 -- 172.21.15.60:0/1370585220 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8004300 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6dffff700 1 -- 172.21.15.60:0/1370585220 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e801f070 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f80c90b0 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.048+0000 7fd6dffff700 1 -- 172.21.15.60:0/1370585220 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80036a0 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.049+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f80c9670 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.049+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f8066700 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.050+0000 7fd6dffff700 1 -- 172.21.15.60:0/1370585220 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd6e8021030 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.051+0000 7fd6dffff700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd6e406ace0 0x7fd6e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:04.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.051+0000 7fd6dffff700 1 -- 172.21.15.60:0/1370585220 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd6e808a700 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.052+0000 7fd6f6ffd700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd6e406ace0 0x7fd6e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:04.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.052+0000 7fd6f6ffd700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd6e406ace0 0x7fd6e406d190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd6f81e6230 tx=0x7fd6ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.056+0000 7fd6dffff700 1 -- 172.21.15.60:0/1370585220 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd6e80558c0 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.308+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd6f804e210 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.897+0000 7fd6dffff700 1 -- 172.21.15.60:0/1370585220 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd6e8058ee0 con 0x7fd6f80b1ac0 2023-07-13T22:25:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.900+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd6e406ace0 msgr2=0x7fd6e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.900+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd6e406ace0 0x7fd6e406d190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd6f81e6230 tx=0x7fd6ec008040 comp rx=0 tx=0).stop 2023-07-13T22:25:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.900+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b1ac0 msgr2=0x7fd6f80c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.900+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b1ac0 0x7fd6f80c8730 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8000c00 tx=0x7fd6e8004750 comp rx=0 tx=0).stop 2023-07-13T22:25:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.901+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 shutdown_connections 2023-07-13T22:25:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.901+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd6e406ace0 0x7fd6e406d190 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.901+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6f80b1ac0 0x7fd6f80c8730 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.901+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6f80cede0 0x7fd6f81f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.901+0000 7fd6fddff700 1 --2- 172.21.15.60:0/1370585220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6f80b8200 0x7fd6f81449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.901+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 >> 172.21.15.60:0/1370585220 conn(0x7fd6f806b2f0 msgr2=0x7fd6f80d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.901+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 shutdown_connections 2023-07-13T22:25:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:04.901+0000 7fd6fddff700 1 -- 172.21.15.60:0/1370585220 wait complete. 2023-07-13T22:25:04.905 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:25:04.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:25:05.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.173+0000 7f7fc9dff700 1 Processor -- start 2023-07-13T22:25:05.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.173+0000 7f7fc9dff700 1 -- start start 2023-07-13T22:25:05.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc9dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7fc40cfd50 0x7f7fc40d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:05.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc9dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7fc40b1ac0 0x7f7fc40d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:05.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc9dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc40d1870 con 0x7f7fc40b8200 2023-07-13T22:25:05.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc9dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc40cf370 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc9dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc40d12b0 con 0x7f7fc40cfd50 2023-07-13T22:25:05.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc37fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7fc40b8200 0x7f7fc40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48946/0 (socket says 172.21.15.60:48946) 2023-07-13T22:25:05.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc37fe700 1 -- 172.21.15.60:0/3445749095 learned_addr learned my addr 172.21.15.60:0/3445749095 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:05.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc3fff700 1 --2- 172.21.15.60:0/3445749095 >> v2:172.21.15.60:3300/0 conn(0x7f7fc40b1ac0 0x7f7fc40d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:05.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fbbfff700 1 --2- 172.21.15.60:0/3445749095 >> v2:172.21.15.105:3300/0 conn(0x7f7fc40cfd50 0x7f7fc40d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:05.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc2ffd700 1 -- 172.21.15.60:0/3445749095 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708546152 0 0) 0x7f7fc40d1870 con 0x7f7fc40b8200 2023-07-13T22:25:05.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.175+0000 7f7fc2ffd700 1 -- 172.21.15.60:0/3445749095 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fb00035b0 con 0x7f7fc40b8200 2023-07-13T22:25:05.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.176+0000 7f7fc3fff700 1 -- 172.21.15.60:0/3445749095 >> v2:172.21.15.105:3300/0 conn(0x7f7fc40cfd50 msgr2=0x7f7fc40d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:05.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.176+0000 7f7fc3fff700 1 --2- 172.21.15.60:0/3445749095 >> v2:172.21.15.105:3300/0 conn(0x7f7fc40cfd50 0x7f7fc40d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:05.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.176+0000 7f7fc3fff700 1 -- 172.21.15.60:0/3445749095 >> v1:172.21.15.60:6789/0 conn(0x7f7fc40b8200 legacy=0x7f7fc40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:05.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.176+0000 7f7fc3fff700 1 -- 172.21.15.60:0/3445749095 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40d1590 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.176+0000 7f7fbbfff700 1 --2- 172.21.15.60:0/3445749095 >> v2:172.21.15.105:3300/0 conn(0x7f7fc40cfd50 0x7f7fc40d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:05.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.176+0000 7f7fc3fff700 1 --2- 172.21.15.60:0/3445749095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 0x7f7fc40d2900 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40d07b0 tx=0x7f7fbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5bc21fc6e0c53 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.176+0000 7f7fc2ffd700 1 -- 172.21.15.60:0/3445749095 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00e070 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.176+0000 7f7fc2ffd700 1 -- 172.21.15.60:0/3445749095 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc00bb20 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.176+0000 7f7fc2ffd700 1 -- 172.21.15.60:0/3445749095 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00be90 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.177+0000 7f7fc9dff700 1 -- 172.21.15.60:0/3445749095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 msgr2=0x7f7fc40d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.177+0000 7f7fc9dff700 1 --2- 172.21.15.60:0/3445749095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 0x7f7fc40d2900 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40d07b0 tx=0x7f7fbc00b330 comp rx=0 tx=0).stop 2023-07-13T22:25:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.177+0000 7f7fc9dff700 1 -- 172.21.15.60:0/3445749095 shutdown_connections 2023-07-13T22:25:05.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.177+0000 7f7fc9dff700 1 --2- 172.21.15.60:0/3445749095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 0x7f7fc40d2900 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:05.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.177+0000 7f7fc9dff700 1 --2- 172.21.15.60:0/3445749095 >> v2:172.21.15.105:3300/0 conn(0x7f7fc40cfd50 0x7f7fc40d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:05.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.177+0000 7f7fc9dff700 1 -- 172.21.15.60:0/3445749095 >> 172.21.15.60:0/3445749095 conn(0x7f7fc406b2f0 msgr2=0x7f7fc406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:05.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.177+0000 7f7fc9dff700 1 -- 172.21.15.60:0/3445749095 shutdown_connections 2023-07-13T22:25:05.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.177+0000 7f7fc9dff700 1 -- 172.21.15.60:0/3445749095 wait complete. 2023-07-13T22:25:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.179+0000 7f7fc9dff700 1 Processor -- start 2023-07-13T22:25:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.179+0000 7f7fc9dff700 1 -- start start 2023-07-13T22:25:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc9dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 0x7f7fc40ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc9dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fc40b8200 0x7f7fc4158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc9dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fc40cfd50 0x7f7fc41e8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc9dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40ca380 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc9dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40c9d00 con 0x7f7fc40cfd50 2023-07-13T22:25:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc9dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40cace0 con 0x7f7fc40b8200 2023-07-13T22:25:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 0x7f7fc40ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 0x7f7fc40ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41970/0 (socket says 172.21.15.60:41970) 2023-07-13T22:25:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc37fe700 1 -- 172.21.15.60:0/1855217974 learned_addr learned my addr 172.21.15.60:0/1855217974 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fbbfff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fc40b8200 0x7f7fc4158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc3fff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fc40cfd50 0x7f7fc41e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc37fe700 1 -- 172.21.15.60:0/1855217974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fc40b8200 msgr2=0x7f7fc4158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc37fe700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fc40b8200 0x7f7fc4158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc37fe700 1 -- 172.21.15.60:0/1855217974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fc40cfd50 msgr2=0x7f7fc41e8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc37fe700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fc40cfd50 0x7f7fc41e8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.180+0000 7f7fc37fe700 1 -- 172.21.15.60:0/1855217974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac0094a0 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.181+0000 7f7fc37fe700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 0x7f7fc40ca7a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400c4d0 tx=0x7f7fb400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.181+0000 7f7fc0ff9700 1 -- 172.21.15.60:0/1855217974 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4005d70 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.181+0000 7f7fc0ff9700 1 -- 172.21.15.60:0/1855217974 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb40070d0 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.181+0000 7f7fc0ff9700 1 -- 172.21.15.60:0/1855217974 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4007440 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.181+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc009a70 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.181+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc41e1e10 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.182+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc404e210 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.183+0000 7f7fc0ff9700 1 -- 172.21.15.60:0/1855217974 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7fb40075a0 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.184+0000 7f7fc0ff9700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7fa406ace0 0x7f7fa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.184+0000 7f7fc0ff9700 1 -- 172.21.15.60:0/1855217974 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7fb4088940 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.184+0000 7f7fbbfff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7fa406ace0 0x7f7fa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:05.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.185+0000 7f7fbbfff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7fa406ace0 0x7f7fa406d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7fc406ba00 tx=0x7f7fac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.189+0000 7f7fc0ff9700 1 -- 172.21.15.60:0/1855217974 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7fb4053a90 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.441+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7fc40b4f70 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.899+0000 7f7fc0ff9700 1 -- 172.21.15.60:0/1855217974 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7fb40170c0 con 0x7f7fc40b1ac0 2023-07-13T22:25:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7fa406ace0 msgr2=0x7f7fa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7fa406ace0 0x7f7fa406d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7fc406ba00 tx=0x7f7fac008040 comp rx=0 tx=0).stop 2023-07-13T22:25:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 msgr2=0x7f7fc40ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 0x7f7fc40ca7a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400c4d0 tx=0x7f7fb400c890 comp rx=0 tx=0).stop 2023-07-13T22:25:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 shutdown_connections 2023-07-13T22:25:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7fa406ace0 0x7f7fa406d190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:05.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fc40b1ac0 0x7f7fc40ca7a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:05.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fc40cfd50 0x7f7fc41e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:05.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 --2- 172.21.15.60:0/1855217974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fc40b8200 0x7f7fc4158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:05.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 >> 172.21.15.60:0/1855217974 conn(0x7f7fc406b2f0 msgr2=0x7f7fc40b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:05.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.902+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 shutdown_connections 2023-07-13T22:25:05.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:05.903+0000 7f7fc9dff700 1 -- 172.21.15.60:0/1855217974 wait complete. 2023-07-13T22:25:05.905 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:25:05.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:25:06.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.128+0000 7fce7e3ff700 1 Processor -- start 2023-07-13T22:25:06.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.128+0000 7fce7e3ff700 1 -- start start 2023-07-13T22:25:06.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.129+0000 7fce7e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 0x7fce780ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce7e3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fce780d35c0 0x7fce780d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce7e3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce780d09d0 con 0x7fce780cf3f0 2023-07-13T22:25:06.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce7e3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce780d3b30 con 0x7fce780d35c0 2023-07-13T22:25:06.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce7e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780cda10 con 0x7fce780b8570 2023-07-13T22:25:06.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce777fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fce780d35c0 0x7fce780d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce7c99c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fce780cf3f0 0x7fce7815fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48960/0 (socket says 172.21.15.60:48960) 2023-07-13T22:25:06.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce777fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fce780d35c0 0x7fce780d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41972/0 (socket says 172.21.15.60:41972) 2023-07-13T22:25:06.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce7c99c700 1 -- 172.21.15.60:0/1331379639 learned_addr learned my addr 172.21.15.60:0/1331379639 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:06.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce77fff700 1 --2- 172.21.15.60:0/1331379639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 0x7fce780ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce777fe700 1 -- 172.21.15.60:0/1331379639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 msgr2=0x7fce780ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:06.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce777fe700 1 --2- 172.21.15.60:0/1331379639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 0x7fce780ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce777fe700 1 -- 172.21.15.60:0/1331379639 >> v1:172.21.15.60:6789/0 conn(0x7fce780cf3f0 legacy=0x7fce7815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce777fe700 1 -- 172.21.15.60:0/1331379639 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780cd450 con 0x7fce780d35c0 2023-07-13T22:25:06.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.130+0000 7fce76ffd700 1 -- 172.21.15.60:0/1331379639 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770563163 0 0) 0x7fce780d09d0 con 0x7fce780cf3f0 2023-07-13T22:25:06.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.131+0000 7fce777fe700 1 --2- 172.21.15.60:0/1331379639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780d35c0 0x7fce780d0eb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009840 tx=0x7fce6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3358f0ccdaef3a3a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:06.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.131+0000 7fce76ffd700 1 -- 172.21.15.60:0/1331379639 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c011830 con 0x7fce780d35c0 2023-07-13T22:25:06.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.131+0000 7fce76ffd700 1 -- 172.21.15.60:0/1331379639 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c011990 con 0x7fce780d35c0 2023-07-13T22:25:06.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.131+0000 7fce76ffd700 1 -- 172.21.15.60:0/1331379639 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c011d20 con 0x7fce780d35c0 2023-07-13T22:25:06.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.131+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1331379639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780d35c0 msgr2=0x7fce780d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.131+0000 7fce7e3ff700 1 --2- 172.21.15.60:0/1331379639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780d35c0 0x7fce780d0eb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009840 tx=0x7fce6c009b50 comp rx=0 tx=0).stop 2023-07-13T22:25:06.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.132+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1331379639 shutdown_connections 2023-07-13T22:25:06.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.132+0000 7fce7e3ff700 1 --2- 172.21.15.60:0/1331379639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 0x7fce780ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.132+0000 7fce7e3ff700 1 --2- 172.21.15.60:0/1331379639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780d35c0 0x7fce780d0eb0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.132+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1331379639 >> 172.21.15.60:0/1331379639 conn(0x7fce78071600 msgr2=0x7fce780719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:06.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.132+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1331379639 shutdown_connections 2023-07-13T22:25:06.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.132+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1331379639 wait complete. 2023-07-13T22:25:06.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.133+0000 7fce7e3ff700 1 Processor -- start 2023-07-13T22:25:06.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.134+0000 7fce7e3ff700 1 -- start start 2023-07-13T22:25:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce7e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 0x7fce780cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce7e3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780cf3f0 0x7fce780c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce7e3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce780d35c0 0x7fce78155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce7e3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780c7a10 con 0x7fce780cf3f0 2023-07-13T22:25:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce7e3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780ca580 con 0x7fce780d35c0 2023-07-13T22:25:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce7e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780ca860 con 0x7fce780b8570 2023-07-13T22:25:06.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce777fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780cf3f0 0x7fce780c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce777fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780cf3f0 0x7fce780c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41978/0 (socket says 172.21.15.60:41978) 2023-07-13T22:25:06.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce777fe700 1 -- 172.21.15.60:0/1624404296 learned_addr learned my addr 172.21.15.60:0/1624404296 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:06.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce77fff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 0x7fce780cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce7c99c700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce780d35c0 0x7fce78155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce777fe700 1 -- 172.21.15.60:0/1624404296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 msgr2=0x7fce780cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce777fe700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 0x7fce780cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce777fe700 1 -- 172.21.15.60:0/1624404296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce780d35c0 msgr2=0x7fce78155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce777fe700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce780d35c0 0x7fce78155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce777fe700 1 -- 172.21.15.60:0/1624404296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0094a0 con 0x7fce780cf3f0 2023-07-13T22:25:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.135+0000 7fce77fff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 0x7fce780cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.136+0000 7fce777fe700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780cf3f0 0x7fce780c9d30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009b90 tx=0x7fce6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.136+0000 7fce74ff9700 1 -- 172.21.15.60:0/1624404296 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0038a0 con 0x7fce780cf3f0 2023-07-13T22:25:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.136+0000 7fce74ff9700 1 -- 172.21.15.60:0/1624404296 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c01e070 con 0x7fce780cf3f0 2023-07-13T22:25:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.136+0000 7fce74ff9700 1 -- 172.21.15.60:0/1624404296 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c003c20 con 0x7fce780cf3f0 2023-07-13T22:25:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.136+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce780cab40 con 0x7fce780cf3f0 2023-07-13T22:25:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.136+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce780ca270 con 0x7fce780cf3f0 2023-07-13T22:25:06.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.137+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7804e280 con 0x7fce780cf3f0 2023-07-13T22:25:06.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.138+0000 7fce74ff9700 1 -- 172.21.15.60:0/1624404296 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fce6c02d450 con 0x7fce780cf3f0 2023-07-13T22:25:06.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.139+0000 7fce74ff9700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6406ace0 0x7fce6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.139+0000 7fce74ff9700 1 -- 172.21.15.60:0/1624404296 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fce6c08a4c0 con 0x7fce780cf3f0 2023-07-13T22:25:06.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.139+0000 7fce77fff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6406ace0 0x7fce6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.140+0000 7fce77fff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6406ace0 0x7fce6406d190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fce68009c10 tx=0x7fce68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.143+0000 7fce74ff9700 1 -- 172.21.15.60:0/1624404296 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fce6c055560 con 0x7fce780cf3f0 2023-07-13T22:25:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.400+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce780736f0 con 0x7fce780cf3f0 2023-07-13T22:25:06.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.401+0000 7fce74ff9700 1 -- 172.21.15.60:0/1624404296 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fce6c058b80 con 0x7fce780cf3f0 2023-07-13T22:25:06.383 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:06.384 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1603865496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1603865496},{"type":"v1","addr":"172.21.15.105:6834","nonce":1603865496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6735,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1777842000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1777842000},{"type":"v1","addr":"172.21.15.105:6835","nonce":1777842000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6738,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2613179178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2613179178},{"type":"v1","addr":"172.21.15.60:6836","nonce":2613179178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:25:02.392864+0000","modified":"2023-07-13T22:25:05.896853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3617406711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3617406711},{"type":"v1","addr":"172.21.15.60:6837","nonce":3617406711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-13T22:25:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6406ace0 msgr2=0x7fce6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6406ace0 0x7fce6406d190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fce68009c10 tx=0x7fce68008040 comp rx=0 tx=0).stop 2023-07-13T22:25:06.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780cf3f0 msgr2=0x7fce780c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780cf3f0 0x7fce780c9d30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009b90 tx=0x7fce6c004750 comp rx=0 tx=0).stop 2023-07-13T22:25:06.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 shutdown_connections 2023-07-13T22:25:06.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce6406ace0 0x7fce6406d190 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce780d35c0 0x7fce78155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce780b8570 0x7fce780cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 --2- 172.21.15.60:0/1624404296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce780cf3f0 0x7fce780c9d30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 >> 172.21.15.60:0/1624404296 conn(0x7fce78071600 msgr2=0x7fce780cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:06.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 shutdown_connections 2023-07-13T22:25:06.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.403+0000 7fce7e3ff700 1 -- 172.21.15.60:0/1624404296 wait complete. 2023-07-13T22:25:06.387 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 41 2023-07-13T22:25:06.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:25:06.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.623+0000 7fdee21ff700 1 Processor -- start 2023-07-13T22:25:06.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.624+0000 7fdee21ff700 1 -- start start 2023-07-13T22:25:06.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdee21ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 0x7fdedc0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdee21ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdedc0cd760 0x7fdedc0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdee21ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdedc1609a0 con 0x7fdedc06bad0 2023-07-13T22:25:06.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdee21ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdedc06a5f0 con 0x7fdedc0cd760 2023-07-13T22:25:06.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdee21ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc160ed0 con 0x7fdedc0b12a0 2023-07-13T22:25:06.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdedbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdedc0cd760 0x7fdedc0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdedbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdedc0cd760 0x7fdedc0cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41980/0 (socket says 172.21.15.60:41980) 2023-07-13T22:25:06.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdedbfff700 1 -- 172.21.15.60:0/189470106 learned_addr learned my addr 172.21.15.60:0/189470106 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:06.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdedaffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdedc06bad0 0x7fdedc0b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48964/0 (socket says 172.21.15.60:48964) 2023-07-13T22:25:06.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdedb7fe700 1 --2- 172.21.15.60:0/189470106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 0x7fdedc0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdedbfff700 1 -- 172.21.15.60:0/189470106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 msgr2=0x7fdedc0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:06.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.625+0000 7fdedbfff700 1 --2- 172.21.15.60:0/189470106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 0x7fdedc0cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.626+0000 7fdedbfff700 1 -- 172.21.15.60:0/189470106 >> v1:172.21.15.60:6789/0 conn(0x7fdedc06bad0 legacy=0x7fdedc0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.626+0000 7fdedbfff700 1 -- 172.21.15.60:0/189470106 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc06bf40 con 0x7fdedc0cd760 2023-07-13T22:25:06.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.626+0000 7fdedbfff700 1 --2- 172.21.15.60:0/189470106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc0cd760 0x7fdedc0cdbe0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fded4009e80 tx=0x7fded400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfec964fab51873f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:06.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.626+0000 7fdeda7fc700 1 -- 172.21.15.60:0/189470106 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400e070 con 0x7fdedc0cd760 2023-07-13T22:25:06.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.626+0000 7fdeda7fc700 1 -- 172.21.15.60:0/189470106 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded400b980 con 0x7fdedc0cd760 2023-07-13T22:25:06.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.626+0000 7fdeda7fc700 1 -- 172.21.15.60:0/189470106 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400bd30 con 0x7fdedc0cd760 2023-07-13T22:25:06.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.627+0000 7fdee21ff700 1 -- 172.21.15.60:0/189470106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc0cd760 msgr2=0x7fdedc0cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.627+0000 7fdee21ff700 1 --2- 172.21.15.60:0/189470106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc0cd760 0x7fdedc0cdbe0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fded4009e80 tx=0x7fded400b330 comp rx=0 tx=0).stop 2023-07-13T22:25:06.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.627+0000 7fdee21ff700 1 -- 172.21.15.60:0/189470106 shutdown_connections 2023-07-13T22:25:06.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.627+0000 7fdee21ff700 1 --2- 172.21.15.60:0/189470106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc0cd760 0x7fdedc0cdbe0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.627+0000 7fdee21ff700 1 --2- 172.21.15.60:0/189470106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 0x7fdedc0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.627+0000 7fdee21ff700 1 -- 172.21.15.60:0/189470106 >> 172.21.15.60:0/189470106 conn(0x7fdedc0cc2a0 msgr2=0x7fdedc0cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:06.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.627+0000 7fdee21ff700 1 -- 172.21.15.60:0/189470106 shutdown_connections 2023-07-13T22:25:06.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.627+0000 7fdee21ff700 1 -- 172.21.15.60:0/189470106 wait complete. 2023-07-13T22:25:06.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.628+0000 7fdee21ff700 1 Processor -- start 2023-07-13T22:25:06.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.629+0000 7fdee21ff700 1 -- start start 2023-07-13T22:25:06.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdee21ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc06bad0 0x7fdedc0ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdee21ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 0x7fdedc1e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdee21ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdedc0cd760 0x7fdedc0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdee21ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc15d7d0 con 0x7fdedc06bad0 2023-07-13T22:25:06.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdee21ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0acc70 con 0x7fdedc0cd760 2023-07-13T22:25:06.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdee21ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0acf50 con 0x7fdedc0b12a0 2023-07-13T22:25:06.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc06bad0 0x7fdedc0ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc06bad0 0x7fdedc0ac300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41992/0 (socket says 172.21.15.60:41992) 2023-07-13T22:25:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedb7fe700 1 -- 172.21.15.60:0/3629267121 learned_addr learned my addr 172.21.15.60:0/3629267121 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedaffd700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 0x7fdedc1e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedb7fe700 1 -- 172.21.15.60:0/3629267121 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 msgr2=0x7fdedc1e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedb7fe700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 0x7fdedc1e7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedb7fe700 1 -- 172.21.15.60:0/3629267121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdedc0cd760 msgr2=0x7fdedc0adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedb7fe700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdedc0cd760 0x7fdedc0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedb7fe700 1 -- 172.21.15.60:0/3629267121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded4009b30 con 0x7fdedc06bad0 2023-07-13T22:25:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.630+0000 7fdedaffd700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 0x7fdedc1e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.631+0000 7fdedb7fe700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc06bad0 0x7fdedc0ac300 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e810 tx=0x7fdecc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.631+0000 7fdec3fff700 1 -- 172.21.15.60:0/3629267121 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc013610 con 0x7fdedc06bad0 2023-07-13T22:25:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.631+0000 7fdec3fff700 1 -- 172.21.15.60:0/3629267121 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc013770 con 0x7fdedc06bad0 2023-07-13T22:25:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.631+0000 7fdec3fff700 1 -- 172.21.15.60:0/3629267121 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc013b20 con 0x7fdedc06bad0 2023-07-13T22:25:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.631+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdedc0ad230 con 0x7fdedc06bad0 2023-07-13T22:25:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.631+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdedc0ac840 con 0x7fdedc06bad0 2023-07-13T22:25:06.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.632+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdedc15c860 con 0x7fdedc06bad0 2023-07-13T22:25:06.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.633+0000 7fdec3fff700 1 -- 172.21.15.60:0/3629267121 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdecc013c80 con 0x7fdedc06bad0 2023-07-13T22:25:06.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.634+0000 7fdec3fff700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec806ace0 0x7fdec806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:06.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.634+0000 7fdec3fff700 1 -- 172.21.15.60:0/3629267121 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdecc089250 con 0x7fdedc06bad0 2023-07-13T22:25:06.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.634+0000 7fdedaffd700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec806ace0 0x7fdec806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:06.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.635+0000 7fdedaffd700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec806ace0 0x7fdec806d190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdedc15ab80 tx=0x7fded0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:06.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.638+0000 7fdec3fff700 1 -- 172.21.15.60:0/3629267121 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdedc15c860 con 0x7fdedc06bad0 2023-07-13T22:25:06.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.845+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdedc15dab0 con 0x7fdedc06bad0 2023-07-13T22:25:06.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.846+0000 7fdec3fff700 1 -- 172.21.15.60:0/3629267121 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18260 (secure 0 0 0) 0x7fdecc054270 con 0x7fdedc06bad0 2023-07-13T22:25:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:06.829 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":75,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:25:03.885230+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:25:00.672661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-13T22:25:01.364844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:25:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.848+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec806ace0 msgr2=0x7fdec806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.848+0000 7fdee21ff700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec806ace0 0x7fdec806d190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdedc15ab80 tx=0x7fded0009040 comp rx=0 tx=0).stop 2023-07-13T22:25:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.848+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc06bad0 msgr2=0x7fdedc0ac300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:06.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.848+0000 7fdee21ff700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc06bad0 0x7fdedc0ac300 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e810 tx=0x7fdecc00ebd0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.849+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 shutdown_connections 2023-07-13T22:25:06.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.849+0000 7fdee21ff700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdec806ace0 0x7fdec806d190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.849+0000 7fdee21ff700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdedc06bad0 0x7fdedc0ac300 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.849+0000 7fdee21ff700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdedc0cd760 0x7fdedc0adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.849+0000 7fdee21ff700 1 --2- 172.21.15.60:0/3629267121 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdedc0b12a0 0x7fdedc1e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.849+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 >> 172.21.15.60:0/3629267121 conn(0x7fdedc0cc2a0 msgr2=0x7fdedc0cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.849+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 shutdown_connections 2023-07-13T22:25:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:06.849+0000 7fdee21ff700 1 -- 172.21.15.60:0/3629267121 wait complete. 2023-07-13T22:25:06.849 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:25:06.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:25:07.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.069+0000 7f4b4fdff700 1 Processor -- start 2023-07-13T22:25:07.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.070+0000 7f4b4fdff700 1 -- start start 2023-07-13T22:25:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.071+0000 7f4b4fdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4b480b8400 0x7f4b480cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.071+0000 7f4b4fdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b480d3750 0x7f4b480cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.071+0000 7f4b4fdff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b480d0ac0 con 0x7f4b480cfad0 2023-07-13T22:25:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.071+0000 7f4b4fdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b480cff50 con 0x7f4b480b8400 2023-07-13T22:25:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.071+0000 7f4b4fdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b480b87d0 con 0x7f4b480d3750 2023-07-13T22:25:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.071+0000 7f4b4db9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4b480b8400 0x7f4b480cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.071+0000 7f4b4e39c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4b480cfad0 0x7f4b4815fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48974/0 (socket says 172.21.15.60:48974) 2023-07-13T22:25:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.071+0000 7f4b4e39c700 1 -- 172.21.15.60:0/872507807 learned_addr learned my addr 172.21.15.60:0/872507807 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.071+0000 7f4b4d39a700 1 --2- 172.21.15.60:0/872507807 >> v2:172.21.15.105:3300/0 conn(0x7f4b480d3750 0x7f4b480cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.072+0000 7f4b4cb99700 1 -- 172.21.15.60:0/872507807 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708229548 0 0) 0x7f4b480d0ac0 con 0x7f4b480cfad0 2023-07-13T22:25:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.072+0000 7f4b4db9b700 1 -- 172.21.15.60:0/872507807 >> v2:172.21.15.105:3300/0 conn(0x7f4b480d3750 msgr2=0x7f4b480cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.072+0000 7f4b4db9b700 1 --2- 172.21.15.60:0/872507807 >> v2:172.21.15.105:3300/0 conn(0x7f4b480d3750 0x7f4b480cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.072+0000 7f4b4db9b700 1 -- 172.21.15.60:0/872507807 >> v1:172.21.15.60:6789/0 conn(0x7f4b480cfad0 legacy=0x7f4b4815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.072+0000 7f4b4db9b700 1 -- 172.21.15.60:0/872507807 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b480b4eb0 con 0x7f4b480b8400 2023-07-13T22:25:07.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.072+0000 7f4b4db9b700 1 --2- 172.21.15.60:0/872507807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480b8400 0x7f4b480cd240 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4b480b8cf0 tx=0x7f4b44009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f219915790c79443 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.072+0000 7f4b4cb99700 1 -- 172.21.15.60:0/872507807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44011830 con 0x7f4b480b8400 2023-07-13T22:25:07.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.072+0000 7f4b4cb99700 1 -- 172.21.15.60:0/872507807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b44011990 con 0x7f4b480b8400 2023-07-13T22:25:07.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.072+0000 7f4b4cb99700 1 -- 172.21.15.60:0/872507807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44011d20 con 0x7f4b480b8400 2023-07-13T22:25:07.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.073+0000 7f4b4fdff700 1 -- 172.21.15.60:0/872507807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480b8400 msgr2=0x7f4b480cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.073+0000 7f4b4fdff700 1 --2- 172.21.15.60:0/872507807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480b8400 0x7f4b480cd240 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4b480b8cf0 tx=0x7f4b44009b20 comp rx=0 tx=0).stop 2023-07-13T22:25:07.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.073+0000 7f4b4fdff700 1 -- 172.21.15.60:0/872507807 shutdown_connections 2023-07-13T22:25:07.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.073+0000 7f4b4fdff700 1 --2- 172.21.15.60:0/872507807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480b8400 0x7f4b480cd240 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.073+0000 7f4b4fdff700 1 --2- 172.21.15.60:0/872507807 >> v2:172.21.15.105:3300/0 conn(0x7f4b480d3750 0x7f4b480cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.073+0000 7f4b4fdff700 1 -- 172.21.15.60:0/872507807 >> 172.21.15.60:0/872507807 conn(0x7f4b4806b390 msgr2=0x7f4b4806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:07.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.073+0000 7f4b4fdff700 1 -- 172.21.15.60:0/872507807 shutdown_connections 2023-07-13T22:25:07.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.073+0000 7f4b4fdff700 1 -- 172.21.15.60:0/872507807 wait complete. 2023-07-13T22:25:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.075+0000 7f4b4fdff700 1 Processor -- start 2023-07-13T22:25:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.075+0000 7f4b4fdff700 1 -- start start 2023-07-13T22:25:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4fdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b480b8400 0x7f4b480cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4fdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b480cfad0 0x7f4b481541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4fdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480d3750 0x7f4b480b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4fdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b480ca030 con 0x7f4b480d3750 2023-07-13T22:25:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4fdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b480ca310 con 0x7f4b480cfad0 2023-07-13T22:25:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4fdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b480ca5f0 con 0x7f4b480b8400 2023-07-13T22:25:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4e39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480d3750 0x7f4b480b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4e39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480d3750 0x7f4b480b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42002/0 (socket says 172.21.15.60:42002) 2023-07-13T22:25:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4e39c700 1 -- 172.21.15.60:0/2774198913 learned_addr learned my addr 172.21.15.60:0/2774198913 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4d39a700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b480cfad0 0x7f4b481541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4db9b700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b480b8400 0x7f4b480cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4e39c700 1 -- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b480b8400 msgr2=0x7f4b480cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4e39c700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b480b8400 0x7f4b480cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4e39c700 1 -- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b480cfad0 msgr2=0x7f4b481541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4e39c700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b480cfad0 0x7f4b481541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4e39c700 1 -- 172.21.15.60:0/2774198913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b440094a0 con 0x7f4b480d3750 2023-07-13T22:25:07.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4d39a700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b480cfad0 0x7f4b481541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:07.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.076+0000 7f4b4db9b700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b480b8400 0x7f4b480cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:07.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.077+0000 7f4b4e39c700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480d3750 0x7f4b480b6a00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000cba0 tx=0x7f4b4000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.077+0000 7f4b3e7fc700 1 -- 172.21.15.60:0/2774198913 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000ab80 con 0x7f4b480d3750 2023-07-13T22:25:07.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.077+0000 7f4b3e7fc700 1 -- 172.21.15.60:0/2774198913 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4000ad00 con 0x7f4b480d3750 2023-07-13T22:25:07.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.077+0000 7f4b3e7fc700 1 -- 172.21.15.60:0/2774198913 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40007310 con 0x7f4b480d3750 2023-07-13T22:25:07.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.077+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b480ca8d0 con 0x7f4b480d3750 2023-07-13T22:25:07.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.077+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b480c9c50 con 0x7f4b480d3750 2023-07-13T22:25:07.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.078+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b480b3af0 con 0x7f4b480d3750 2023-07-13T22:25:07.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.079+0000 7f4b3e7fc700 1 -- 172.21.15.60:0/2774198913 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b40007470 con 0x7f4b480d3750 2023-07-13T22:25:07.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.081+0000 7f4b3e7fc700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2c06ace0 0x7f4b2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.081+0000 7f4b3e7fc700 1 -- 172.21.15.60:0/2774198913 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4b40088f50 con 0x7f4b480d3750 2023-07-13T22:25:07.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.081+0000 7f4b4db9b700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2c06ace0 0x7f4b2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.082+0000 7f4b4db9b700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2c06ace0 0x7f4b2c06d190 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009b60 tx=0x7f4b44008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.085+0000 7f4b3e7fc700 1 -- 172.21.15.60:0/2774198913 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4b40053ff0 con 0x7f4b480d3750 2023-07-13T22:25:07.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.288+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b480ca480 con 0x7f4b2c06ace0 2023-07-13T22:25:07.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.297+0000 7f4b3e7fc700 1 -- 172.21.15.60:0/2774198913 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311541 (secure 0 0 0) 0x7f4b480ca480 con 0x7f4b2c06ace0 2023-07-13T22:25:07.283 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:07.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2c06ace0 msgr2=0x7f4b2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2c06ace0 0x7f4b2c06d190 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009b60 tx=0x7f4b44008030 comp rx=0 tx=0).stop 2023-07-13T22:25:07.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480d3750 msgr2=0x7f4b480b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480d3750 0x7f4b480b6a00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000cba0 tx=0x7f4b4000cf60 comp rx=0 tx=0).stop 2023-07-13T22:25:07.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 shutdown_connections 2023-07-13T22:25:07.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2c06ace0 0x7f4b2c06d190 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b480d3750 0x7f4b480b6a00 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b480b8400 0x7f4b480cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 --2- 172.21.15.60:0/2774198913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b480cfad0 0x7f4b481541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.301+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 >> 172.21.15.60:0/2774198913 conn(0x7f4b4806b390 msgr2=0x7f4b480b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:07.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.302+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 shutdown_connections 2023-07-13T22:25:07.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.302+0000 7f4b4fdff700 1 -- 172.21.15.60:0/2774198913 wait complete. 2023-07-13T22:25:07.286 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:25:07.302 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2023-07-13T22:25:07.175603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277216,"kb_used_data":8352,"kb_used_omap":0,"kb_used_meta":268800,"kb_avail":754697504,"statfs":{"total":773094113280,"available":772810244096,"internally_reserved":0,"allocated":8552448,"data_stored":4477184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":275251200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000925"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893253+0000","last_change":"2023-07-13T22:25:02.892413+0000","last_active":"2023-07-13T22:25:03.893253+0000","last_peered":"2023-07-13T22:25:03.893253+0000","last_clean":"2023-07-13T22:25:03.893253+0000","last_became_active":"2023-07-13T22:25:02.892159+0000","last_became_peered":"2023-07-13T22:25:02.892159+0000","last_unstale":"2023-07-13T22:25:03.893253+0000","last_undegraded":"2023-07-13T22:25:03.893253+0000","last_fullsized":"2023-07-13T22:25:03.893253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:16:31.837939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016469+0000","last_change":"2023-07-13T22:25:02.910258+0000","last_active":"2023-07-13T22:25:04.016469+0000","last_peered":"2023-07-13T22:25:04.016469+0000","last_clean":"2023-07-13T22:25:04.016469+0000","last_became_active":"2023-07-13T22:25:02.909979+0000","last_became_peered":"2023-07-13T22:25:02.909979+0000","last_unstale":"2023-07-13T22:25:04.016469+0000","last_undegraded":"2023-07-13T22:25:04.016469+0000","last_fullsized":"2023-07-13T22:25:04.016469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:07:50.443338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661693+0000","last_change":"2023-07-13T22:25:02.904222+0000","last_active":"2023-07-13T22:25:04.661693+0000","last_peered":"2023-07-13T22:25:04.661693+0000","last_clean":"2023-07-13T22:25:04.661693+0000","last_became_active":"2023-07-13T22:25:02.903890+0000","last_became_peered":"2023-07-13T22:25:02.903890+0000","last_unstale":"2023-07-13T22:25:04.661693+0000","last_undegraded":"2023-07-13T22:25:04.661693+0000","last_fullsized":"2023-07-13T22:25:04.661693+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:52:46.539330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661257+0000","last_change":"2023-07-13T22:25:02.883184+0000","last_active":"2023-07-13T22:25:04.661257+0000","last_peered":"2023-07-13T22:25:04.661257+0000","last_clean":"2023-07-13T22:25:04.661257+0000","last_became_active":"2023-07-13T22:25:02.882874+0000","last_became_peered":"2023-07-13T22:25:02.882874+0000","last_unstale":"2023-07-13T22:25:04.661257+0000","last_undegraded":"2023-07-13T22:25:04.661257+0000","last_fullsized":"2023-07-13T22:25:04.661257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:00:48.053942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661447+0000","last_change":"2023-07-13T22:25:02.889343+0000","last_active":"2023-07-13T22:25:04.661447+0000","last_peered":"2023-07-13T22:25:04.661447+0000","last_clean":"2023-07-13T22:25:04.661447+0000","last_became_active":"2023-07-13T22:25:02.889018+0000","last_became_peered":"2023-07-13T22:25:02.889018+0000","last_unstale":"2023-07-13T22:25:04.661447+0000","last_undegraded":"2023-07-13T22:25:04.661447+0000","last_fullsized":"2023-07-13T22:25:04.661447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:01.720916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892321+0000","last_change":"2023-07-13T22:25:02.895444+0000","last_active":"2023-07-13T22:25:03.892321+0000","last_peered":"2023-07-13T22:25:03.892321+0000","last_clean":"2023-07-13T22:25:03.892321+0000","last_became_active":"2023-07-13T22:25:02.895173+0000","last_became_peered":"2023-07-13T22:25:02.895173+0000","last_unstale":"2023-07-13T22:25:03.892321+0000","last_undegraded":"2023-07-13T22:25:03.892321+0000","last_fullsized":"2023-07-13T22:25:03.892321+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:47:31.063839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894600+0000","last_change":"2023-07-13T22:25:02.903143+0000","last_active":"2023-07-13T22:25:03.894600+0000","last_peered":"2023-07-13T22:25:03.894600+0000","last_clean":"2023-07-13T22:25:03.894600+0000","last_became_active":"2023-07-13T22:25:02.902447+0000","last_became_peered":"2023-07-13T22:25:02.902447+0000","last_unstale":"2023-07-13T22:25:03.894600+0000","last_undegraded":"2023-07-13T22:25:03.894600+0000","last_fullsized":"2023-07-13T22:25:03.894600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:52:46.539330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.660939+0000","last_change":"2023-07-13T22:25:02.889701+0000","last_active":"2023-07-13T22:25:04.660939+0000","last_peered":"2023-07-13T22:25:04.660939+0000","last_clean":"2023-07-13T22:25:04.660939+0000","last_became_active":"2023-07-13T22:25:02.889413+0000","last_became_peered":"2023-07-13T22:25:02.889413+0000","last_unstale":"2023-07-13T22:25:04.660939+0000","last_undegraded":"2023-07-13T22:25:04.660939+0000","last_fullsized":"2023-07-13T22:25:04.660939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:54:19.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571818+0000","last_change":"2023-07-13T22:25:02.891513+0000","last_active":"2023-07-13T22:25:04.571818+0000","last_peered":"2023-07-13T22:25:04.571818+0000","last_clean":"2023-07-13T22:25:04.571818+0000","last_became_active":"2023-07-13T22:25:02.891212+0000","last_became_peered":"2023-07-13T22:25:02.891212+0000","last_unstale":"2023-07-13T22:25:04.571818+0000","last_undegraded":"2023-07-13T22:25:04.571818+0000","last_fullsized":"2023-07-13T22:25:04.571818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:04:20.883177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661307+0000","last_change":"2023-07-13T22:25:02.897232+0000","last_active":"2023-07-13T22:25:04.661307+0000","last_peered":"2023-07-13T22:25:04.661307+0000","last_clean":"2023-07-13T22:25:04.661307+0000","last_became_active":"2023-07-13T22:25:02.896937+0000","last_became_peered":"2023-07-13T22:25:02.896937+0000","last_unstale":"2023-07-13T22:25:04.661307+0000","last_undegraded":"2023-07-13T22:25:04.661307+0000","last_fullsized":"2023-07-13T22:25:04.661307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:00.750773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016612+0000","last_change":"2023-07-13T22:25:02.904784+0000","last_active":"2023-07-13T22:25:04.016612+0000","last_peered":"2023-07-13T22:25:04.016612+0000","last_clean":"2023-07-13T22:25:04.016612+0000","last_became_active":"2023-07-13T22:25:02.904132+0000","last_became_peered":"2023-07-13T22:25:02.904132+0000","last_unstale":"2023-07-13T22:25:04.016612+0000","last_undegraded":"2023-07-13T22:25:04.016612+0000","last_fullsized":"2023-07-13T22:25:04.016612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:32.116765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661400+0000","last_change":"2023-07-13T22:25:02.900398+0000","last_active":"2023-07-13T22:25:04.661400+0000","last_peered":"2023-07-13T22:25:04.661400+0000","last_clean":"2023-07-13T22:25:04.661400+0000","last_became_active":"2023-07-13T22:25:02.899872+0000","last_became_peered":"2023-07-13T22:25:02.899872+0000","last_unstale":"2023-07-13T22:25:04.661400+0000","last_undegraded":"2023-07-13T22:25:04.661400+0000","last_fullsized":"2023-07-13T22:25:04.661400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:47:29.590272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661663+0000","last_change":"2023-07-13T22:25:02.899591+0000","last_active":"2023-07-13T22:25:04.661663+0000","last_peered":"2023-07-13T22:25:04.661663+0000","last_clean":"2023-07-13T22:25:04.661663+0000","last_became_active":"2023-07-13T22:25:02.896723+0000","last_became_peered":"2023-07-13T22:25:02.896723+0000","last_unstale":"2023-07-13T22:25:04.661663+0000","last_undegraded":"2023-07-13T22:25:04.661663+0000","last_fullsized":"2023-07-13T22:25:04.661663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:26:13.609262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892281+0000","last_change":"2023-07-13T22:25:02.895780+0000","last_active":"2023-07-13T22:25:03.892281+0000","last_peered":"2023-07-13T22:25:03.892281+0000","last_clean":"2023-07-13T22:25:03.892281+0000","last_became_active":"2023-07-13T22:25:02.895508+0000","last_became_peered":"2023-07-13T22:25:02.895508+0000","last_unstale":"2023-07-13T22:25:03.892281+0000","last_undegraded":"2023-07-13T22:25:03.892281+0000","last_fullsized":"2023-07-13T22:25:03.892281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:01:59.663749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016199+0000","last_change":"2023-07-13T22:25:02.904113+0000","last_active":"2023-07-13T22:25:04.016199+0000","last_peered":"2023-07-13T22:25:04.016199+0000","last_clean":"2023-07-13T22:25:04.016199+0000","last_became_active":"2023-07-13T22:25:02.903754+0000","last_became_peered":"2023-07-13T22:25:02.903754+0000","last_unstale":"2023-07-13T22:25:04.016199+0000","last_undegraded":"2023-07-13T22:25:04.016199+0000","last_fullsized":"2023-07-13T22:25:04.016199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:29:46.913646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892720+0000","last_change":"2023-07-13T22:25:02.889397+0000","last_active":"2023-07-13T22:25:03.892720+0000","last_peered":"2023-07-13T22:25:03.892720+0000","last_clean":"2023-07-13T22:25:03.892720+0000","last_became_active":"2023-07-13T22:25:02.889139+0000","last_became_peered":"2023-07-13T22:25:02.889139+0000","last_unstale":"2023-07-13T22:25:03.892720+0000","last_undegraded":"2023-07-13T22:25:03.892720+0000","last_fullsized":"2023-07-13T22:25:03.892720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:40:48.503052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661930+0000","last_change":"2023-07-13T22:25:02.895041+0000","last_active":"2023-07-13T22:25:04.661930+0000","last_peered":"2023-07-13T22:25:04.661930+0000","last_clean":"2023-07-13T22:25:04.661930+0000","last_became_active":"2023-07-13T22:25:02.894751+0000","last_became_peered":"2023-07-13T22:25:02.894751+0000","last_unstale":"2023-07-13T22:25:04.661930+0000","last_undegraded":"2023-07-13T22:25:04.661930+0000","last_fullsized":"2023-07-13T22:25:04.661930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:25:50.676534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661538+0000","last_change":"2023-07-13T22:25:02.884921+0000","last_active":"2023-07-13T22:25:04.661538+0000","last_peered":"2023-07-13T22:25:04.661538+0000","last_clean":"2023-07-13T22:25:04.661538+0000","last_became_active":"2023-07-13T22:25:02.884656+0000","last_became_peered":"2023-07-13T22:25:02.884656+0000","last_unstale":"2023-07-13T22:25:04.661538+0000","last_undegraded":"2023-07-13T22:25:04.661538+0000","last_fullsized":"2023-07-13T22:25:04.661538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:33:11.983924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891862+0000","last_change":"2023-07-13T22:25:02.902049+0000","last_active":"2023-07-13T22:25:03.891862+0000","last_peered":"2023-07-13T22:25:03.891862+0000","last_clean":"2023-07-13T22:25:03.891862+0000","last_became_active":"2023-07-13T22:25:02.901749+0000","last_became_peered":"2023-07-13T22:25:02.901749+0000","last_unstale":"2023-07-13T22:25:03.891862+0000","last_undegraded":"2023-07-13T22:25:03.891862+0000","last_fullsized":"2023-07-13T22:25:03.891862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:17:07.084128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.662040+0000","last_change":"2023-07-13T22:25:02.895975+0000","last_active":"2023-07-13T22:25:04.662040+0000","last_peered":"2023-07-13T22:25:04.662040+0000","last_clean":"2023-07-13T22:25:04.662040+0000","last_became_active":"2023-07-13T22:25:02.895355+0000","last_became_peered":"2023-07-13T22:25:02.895355+0000","last_unstale":"2023-07-13T22:25:04.662040+0000","last_undegraded":"2023-07-13T22:25:04.662040+0000","last_fullsized":"2023-07-13T22:25:04.662040+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:29:26.365541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571399+0000","last_change":"2023-07-13T22:25:02.880929+0000","last_active":"2023-07-13T22:25:04.571399+0000","last_peered":"2023-07-13T22:25:04.571399+0000","last_clean":"2023-07-13T22:25:04.571399+0000","last_became_active":"2023-07-13T22:25:02.880647+0000","last_became_peered":"2023-07-13T22:25:02.880647+0000","last_unstale":"2023-07-13T22:25:04.571399+0000","last_undegraded":"2023-07-13T22:25:04.571399+0000","last_fullsized":"2023-07-13T22:25:04.571399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:20:33.736576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661823+0000","last_change":"2023-07-13T22:25:02.899960+0000","last_active":"2023-07-13T22:25:04.661823+0000","last_peered":"2023-07-13T22:25:04.661823+0000","last_clean":"2023-07-13T22:25:04.661823+0000","last_became_active":"2023-07-13T22:25:02.899634+0000","last_became_peered":"2023-07-13T22:25:02.899634+0000","last_unstale":"2023-07-13T22:25:04.661823+0000","last_undegraded":"2023-07-13T22:25:04.661823+0000","last_fullsized":"2023-07-13T22:25:04.661823+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:51:40.469761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571252+0000","last_change":"2023-07-13T22:25:02.881432+0000","last_active":"2023-07-13T22:25:04.571252+0000","last_peered":"2023-07-13T22:25:04.571252+0000","last_clean":"2023-07-13T22:25:04.571252+0000","last_became_active":"2023-07-13T22:25:02.881134+0000","last_became_peered":"2023-07-13T22:25:02.881134+0000","last_unstale":"2023-07-13T22:25:04.571252+0000","last_undegraded":"2023-07-13T22:25:04.571252+0000","last_fullsized":"2023-07-13T22:25:04.571252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:43:02.034043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016705+0000","last_change":"2023-07-13T22:25:02.901388+0000","last_active":"2023-07-13T22:25:04.016705+0000","last_peered":"2023-07-13T22:25:04.016705+0000","last_clean":"2023-07-13T22:25:04.016705+0000","last_became_active":"2023-07-13T22:25:02.898081+0000","last_became_peered":"2023-07-13T22:25:02.898081+0000","last_unstale":"2023-07-13T22:25:04.016705+0000","last_undegraded":"2023-07-13T22:25:04.016705+0000","last_fullsized":"2023-07-13T22:25:04.016705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:11:40.375640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016740+0000","last_change":"2023-07-13T22:25:02.899295+0000","last_active":"2023-07-13T22:25:04.016740+0000","last_peered":"2023-07-13T22:25:04.016740+0000","last_clean":"2023-07-13T22:25:04.016740+0000","last_became_active":"2023-07-13T22:25:02.899012+0000","last_became_peered":"2023-07-13T22:25:02.899012+0000","last_unstale":"2023-07-13T22:25:04.016740+0000","last_undegraded":"2023-07-13T22:25:04.016740+0000","last_fullsized":"2023-07-13T22:25:04.016740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:04:20.883177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661408+0000","last_change":"2023-07-13T22:25:02.903993+0000","last_active":"2023-07-13T22:25:04.661408+0000","last_peered":"2023-07-13T22:25:04.661408+0000","last_clean":"2023-07-13T22:25:04.661408+0000","last_became_active":"2023-07-13T22:25:02.903699+0000","last_became_peered":"2023-07-13T22:25:02.903699+0000","last_unstale":"2023-07-13T22:25:04.661408+0000","last_undegraded":"2023-07-13T22:25:04.661408+0000","last_fullsized":"2023-07-13T22:25:04.661408+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:47:22.989608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892876+0000","last_change":"2023-07-13T22:25:02.889401+0000","last_active":"2023-07-13T22:25:03.892876+0000","last_peered":"2023-07-13T22:25:03.892876+0000","last_clean":"2023-07-13T22:25:03.892876+0000","last_became_active":"2023-07-13T22:25:02.889112+0000","last_became_peered":"2023-07-13T22:25:02.889112+0000","last_unstale":"2023-07-13T22:25:03.892876+0000","last_undegraded":"2023-07-13T22:25:03.892876+0000","last_fullsized":"2023-07-13T22:25:03.892876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:03:34.152998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661823+0000","last_change":"2023-07-13T22:25:02.896184+0000","last_active":"2023-07-13T22:25:04.661823+0000","last_peered":"2023-07-13T22:25:04.661823+0000","last_clean":"2023-07-13T22:25:04.661823+0000","last_became_active":"2023-07-13T22:25:02.895708+0000","last_became_peered":"2023-07-13T22:25:02.895708+0000","last_unstale":"2023-07-13T22:25:04.661823+0000","last_undegraded":"2023-07-13T22:25:04.661823+0000","last_fullsized":"2023-07-13T22:25:04.661823+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:04:21.759427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016506+0000","last_change":"2023-07-13T22:25:02.903294+0000","last_active":"2023-07-13T22:25:04.016506+0000","last_peered":"2023-07-13T22:25:04.016506+0000","last_clean":"2023-07-13T22:25:04.016506+0000","last_became_active":"2023-07-13T22:25:02.903014+0000","last_became_peered":"2023-07-13T22:25:02.903014+0000","last_unstale":"2023-07-13T22:25:04.016506+0000","last_undegraded":"2023-07-13T22:25:04.016506+0000","last_fullsized":"2023-07-13T22:25:04.016506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:03:22.729119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892963+0000","last_change":"2023-07-13T22:25:02.903557+0000","last_active":"2023-07-13T22:25:03.892963+0000","last_peered":"2023-07-13T22:25:03.892963+0000","last_clean":"2023-07-13T22:25:03.892963+0000","last_became_active":"2023-07-13T22:25:02.903173+0000","last_became_peered":"2023-07-13T22:25:02.903173+0000","last_unstale":"2023-07-13T22:25:03.892963+0000","last_undegraded":"2023-07-13T22:25:03.892963+0000","last_fullsized":"2023-07-13T22:25:03.892963+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:48:11.388663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661024+0000","last_change":"2023-07-13T22:25:02.897146+0000","last_active":"2023-07-13T22:25:04.661024+0000","last_peered":"2023-07-13T22:25:04.661024+0000","last_clean":"2023-07-13T22:25:04.661024+0000","last_became_active":"2023-07-13T22:25:02.896847+0000","last_became_peered":"2023-07-13T22:25:02.896847+0000","last_unstale":"2023-07-13T22:25:04.661024+0000","last_undegraded":"2023-07-13T22:25:04.661024+0000","last_fullsized":"2023-07-13T22:25:04.661024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:54:06.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893298+0000","last_change":"2023-07-13T22:25:02.893025+0000","last_active":"2023-07-13T22:25:03.893298+0000","last_peered":"2023-07-13T22:25:03.893298+0000","last_clean":"2023-07-13T22:25:03.893298+0000","last_became_active":"2023-07-13T22:25:02.892740+0000","last_became_peered":"2023-07-13T22:25:02.892740+0000","last_unstale":"2023-07-13T22:25:03.893298+0000","last_undegraded":"2023-07-13T22:25:03.893298+0000","last_fullsized":"2023-07-13T22:25:03.893298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:06:54.924557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661026+0000","last_change":"2023-07-13T22:25:02.884197+0000","last_active":"2023-07-13T22:25:04.661026+0000","last_peered":"2023-07-13T22:25:04.661026+0000","last_clean":"2023-07-13T22:25:04.661026+0000","last_became_active":"2023-07-13T22:25:02.883922+0000","last_became_peered":"2023-07-13T22:25:02.883922+0000","last_unstale":"2023-07-13T22:25:04.661026+0000","last_undegraded":"2023-07-13T22:25:04.661026+0000","last_fullsized":"2023-07-13T22:25:04.661026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:17:55.114100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661573+0000","last_change":"2023-07-13T22:25:01.891747+0000","last_active":"2023-07-13T22:25:04.661573+0000","last_peered":"2023-07-13T22:25:04.661573+0000","last_clean":"2023-07-13T22:25:04.661573+0000","last_became_active":"2023-07-13T22:25:01.891483+0000","last_became_peered":"2023-07-13T22:25:01.891483+0000","last_unstale":"2023-07-13T22:25:04.661573+0000","last_undegraded":"2023-07-13T22:25:04.661573+0000","last_fullsized":"2023-07-13T22:25:04.661573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:13:38.340670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016316+0000","last_change":"2023-07-13T22:25:02.900528+0000","last_active":"2023-07-13T22:25:04.016316+0000","last_peered":"2023-07-13T22:25:04.016316+0000","last_clean":"2023-07-13T22:25:04.016316+0000","last_became_active":"2023-07-13T22:25:02.900212+0000","last_became_peered":"2023-07-13T22:25:02.900212+0000","last_unstale":"2023-07-13T22:25:04.016316+0000","last_undegraded":"2023-07-13T22:25:04.016316+0000","last_fullsized":"2023-07-13T22:25:04.016316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:31:15.060624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893124+0000","last_change":"2023-07-13T22:25:01.897643+0000","last_active":"2023-07-13T22:25:03.893124+0000","last_peered":"2023-07-13T22:25:03.893124+0000","last_clean":"2023-07-13T22:25:03.893124+0000","last_became_active":"2023-07-13T22:25:01.896751+0000","last_became_peered":"2023-07-13T22:25:01.896751+0000","last_unstale":"2023-07-13T22:25:03.893124+0000","last_undegraded":"2023-07-13T22:25:03.893124+0000","last_fullsized":"2023-07-13T22:25:03.893124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:46:44.734121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571368+0000","last_change":"2023-07-13T22:25:02.898265+0000","last_active":"2023-07-13T22:25:04.571368+0000","last_peered":"2023-07-13T22:25:04.571368+0000","last_clean":"2023-07-13T22:25:04.571368+0000","last_became_active":"2023-07-13T22:25:02.897431+0000","last_became_peered":"2023-07-13T22:25:02.897431+0000","last_unstale":"2023-07-13T22:25:04.571368+0000","last_undegraded":"2023-07-13T22:25:04.571368+0000","last_fullsized":"2023-07-13T22:25:04.571368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:03:22.729119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893148+0000","last_change":"2023-07-13T22:25:01.895148+0000","last_active":"2023-07-13T22:25:03.893148+0000","last_peered":"2023-07-13T22:25:03.893148+0000","last_clean":"2023-07-13T22:25:03.893148+0000","last_became_active":"2023-07-13T22:25:01.894871+0000","last_became_peered":"2023-07-13T22:25:01.894871+0000","last_unstale":"2023-07-13T22:25:03.893148+0000","last_undegraded":"2023-07-13T22:25:03.893148+0000","last_fullsized":"2023-07-13T22:25:03.893148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:58:26.743673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571530+0000","last_change":"2023-07-13T22:25:02.880732+0000","last_active":"2023-07-13T22:25:04.571530+0000","last_peered":"2023-07-13T22:25:04.571530+0000","last_clean":"2023-07-13T22:25:04.571530+0000","last_became_active":"2023-07-13T22:25:02.880166+0000","last_became_peered":"2023-07-13T22:25:02.880166+0000","last_unstale":"2023-07-13T22:25:04.571530+0000","last_undegraded":"2023-07-13T22:25:04.571530+0000","last_fullsized":"2023-07-13T22:25:04.571530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:42:55.921674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894293+0000","last_change":"2023-07-13T22:25:01.896257+0000","last_active":"2023-07-13T22:25:03.894293+0000","last_peered":"2023-07-13T22:25:03.894293+0000","last_clean":"2023-07-13T22:25:03.894293+0000","last_became_active":"2023-07-13T22:25:01.895995+0000","last_became_peered":"2023-07-13T22:25:01.895995+0000","last_unstale":"2023-07-13T22:25:03.894293+0000","last_undegraded":"2023-07-13T22:25:03.894293+0000","last_fullsized":"2023-07-13T22:25:03.894293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:19:49.849385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016584+0000","last_change":"2023-07-13T22:25:02.910492+0000","last_active":"2023-07-13T22:25:04.016584+0000","last_peered":"2023-07-13T22:25:04.016584+0000","last_clean":"2023-07-13T22:25:04.016584+0000","last_became_active":"2023-07-13T22:25:02.909942+0000","last_became_peered":"2023-07-13T22:25:02.909942+0000","last_unstale":"2023-07-13T22:25:04.016584+0000","last_undegraded":"2023-07-13T22:25:04.016584+0000","last_fullsized":"2023-07-13T22:25:04.016584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:42:55.921674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571514+0000","last_change":"2023-07-13T22:25:01.892113+0000","last_active":"2023-07-13T22:25:04.571514+0000","last_peered":"2023-07-13T22:25:04.571514+0000","last_clean":"2023-07-13T22:25:04.571514+0000","last_became_active":"2023-07-13T22:25:01.891830+0000","last_became_peered":"2023-07-13T22:25:01.891830+0000","last_unstale":"2023-07-13T22:25:04.571514+0000","last_undegraded":"2023-07-13T22:25:04.571514+0000","last_fullsized":"2023-07-13T22:25:04.571514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:31.110289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016439+0000","last_change":"2023-07-13T22:25:02.904938+0000","last_active":"2023-07-13T22:25:04.016439+0000","last_peered":"2023-07-13T22:25:04.016439+0000","last_clean":"2023-07-13T22:25:04.016439+0000","last_became_active":"2023-07-13T22:25:02.904549+0000","last_became_peered":"2023-07-13T22:25:02.904549+0000","last_unstale":"2023-07-13T22:25:04.016439+0000","last_undegraded":"2023-07-13T22:25:04.016439+0000","last_fullsized":"2023-07-13T22:25:04.016439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:43:02.034043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661129+0000","last_change":"2023-07-13T22:25:01.893789+0000","last_active":"2023-07-13T22:25:04.661129+0000","last_peered":"2023-07-13T22:25:04.661129+0000","last_clean":"2023-07-13T22:25:04.661129+0000","last_became_active":"2023-07-13T22:25:01.893145+0000","last_became_peered":"2023-07-13T22:25:01.893145+0000","last_unstale":"2023-07-13T22:25:04.661129+0000","last_undegraded":"2023-07-13T22:25:04.661129+0000","last_fullsized":"2023-07-13T22:25:04.661129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:03:41.445998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571803+0000","last_change":"2023-07-13T22:25:02.891454+0000","last_active":"2023-07-13T22:25:04.571803+0000","last_peered":"2023-07-13T22:25:04.571803+0000","last_clean":"2023-07-13T22:25:04.571803+0000","last_became_active":"2023-07-13T22:25:02.891167+0000","last_became_peered":"2023-07-13T22:25:02.891167+0000","last_unstale":"2023-07-13T22:25:04.571803+0000","last_undegraded":"2023-07-13T22:25:04.571803+0000","last_fullsized":"2023-07-13T22:25:04.571803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:11:40.375640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016107+0000","last_change":"2023-07-13T22:25:01.894353+0000","last_active":"2023-07-13T22:25:04.016107+0000","last_peered":"2023-07-13T22:25:04.016107+0000","last_clean":"2023-07-13T22:25:04.016107+0000","last_became_active":"2023-07-13T22:25:01.894012+0000","last_became_peered":"2023-07-13T22:25:01.894012+0000","last_unstale":"2023-07-13T22:25:04.016107+0000","last_undegraded":"2023-07-13T22:25:04.016107+0000","last_fullsized":"2023-07-13T22:25:04.016107+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:54:31.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894840+0000","last_change":"2023-07-13T22:25:02.909278+0000","last_active":"2023-07-13T22:25:03.894840+0000","last_peered":"2023-07-13T22:25:03.894840+0000","last_clean":"2023-07-13T22:25:03.894840+0000","last_became_active":"2023-07-13T22:25:02.908914+0000","last_became_peered":"2023-07-13T22:25:02.908914+0000","last_unstale":"2023-07-13T22:25:03.894840+0000","last_undegraded":"2023-07-13T22:25:03.894840+0000","last_fullsized":"2023-07-13T22:25:03.894840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:47:53.889830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661165+0000","last_change":"2023-07-13T22:25:01.887959+0000","last_active":"2023-07-13T22:25:04.661165+0000","last_peered":"2023-07-13T22:25:04.661165+0000","last_clean":"2023-07-13T22:25:04.661165+0000","last_became_active":"2023-07-13T22:25:01.887696+0000","last_became_peered":"2023-07-13T22:25:01.887696+0000","last_unstale":"2023-07-13T22:25:04.661165+0000","last_undegraded":"2023-07-13T22:25:04.661165+0000","last_fullsized":"2023-07-13T22:25:04.661165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:49:34.461411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016137+0000","last_change":"2023-07-13T22:25:02.901715+0000","last_active":"2023-07-13T22:25:04.016137+0000","last_peered":"2023-07-13T22:25:04.016137+0000","last_clean":"2023-07-13T22:25:04.016137+0000","last_became_active":"2023-07-13T22:25:02.898738+0000","last_became_peered":"2023-07-13T22:25:02.898738+0000","last_unstale":"2023-07-13T22:25:04.016137+0000","last_undegraded":"2023-07-13T22:25:04.016137+0000","last_fullsized":"2023-07-13T22:25:04.016137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:45:14.519486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893399+0000","last_change":"2023-07-13T22:25:01.897430+0000","last_active":"2023-07-13T22:25:03.893399+0000","last_peered":"2023-07-13T22:25:03.893399+0000","last_clean":"2023-07-13T22:25:03.893399+0000","last_became_active":"2023-07-13T22:25:01.897100+0000","last_became_peered":"2023-07-13T22:25:01.897100+0000","last_unstale":"2023-07-13T22:25:03.893399+0000","last_undegraded":"2023-07-13T22:25:03.893399+0000","last_fullsized":"2023-07-13T22:25:03.893399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:42:45.904497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661302+0000","last_change":"2023-07-13T22:25:02.881783+0000","last_active":"2023-07-13T22:25:04.661302+0000","last_peered":"2023-07-13T22:25:04.661302+0000","last_clean":"2023-07-13T22:25:04.661302+0000","last_became_active":"2023-07-13T22:25:02.881486+0000","last_became_peered":"2023-07-13T22:25:02.881486+0000","last_unstale":"2023-07-13T22:25:04.661302+0000","last_undegraded":"2023-07-13T22:25:04.661302+0000","last_fullsized":"2023-07-13T22:25:04.661302+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:17:08.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661442+0000","last_change":"2023-07-13T22:25:01.892695+0000","last_active":"2023-07-13T22:25:04.661442+0000","last_peered":"2023-07-13T22:25:04.661442+0000","last_clean":"2023-07-13T22:25:04.661442+0000","last_became_active":"2023-07-13T22:25:01.892443+0000","last_became_peered":"2023-07-13T22:25:01.892443+0000","last_unstale":"2023-07-13T22:25:04.661442+0000","last_undegraded":"2023-07-13T22:25:04.661442+0000","last_fullsized":"2023-07-13T22:25:04.661442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:07:41.239307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892180+0000","last_change":"2023-07-13T22:25:02.892080+0000","last_active":"2023-07-13T22:25:03.892180+0000","last_peered":"2023-07-13T22:25:03.892180+0000","last_clean":"2023-07-13T22:25:03.892180+0000","last_became_active":"2023-07-13T22:25:02.891504+0000","last_became_peered":"2023-07-13T22:25:02.891504+0000","last_unstale":"2023-07-13T22:25:03.892180+0000","last_undegraded":"2023-07-13T22:25:03.892180+0000","last_fullsized":"2023-07-13T22:25:03.892180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:24:42.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661925+0000","last_change":"2023-07-13T22:25:01.885599+0000","last_active":"2023-07-13T22:25:04.661925+0000","last_peered":"2023-07-13T22:25:04.661925+0000","last_clean":"2023-07-13T22:25:04.661925+0000","last_became_active":"2023-07-13T22:25:01.885199+0000","last_became_peered":"2023-07-13T22:25:01.885199+0000","last_unstale":"2023-07-13T22:25:04.661925+0000","last_undegraded":"2023-07-13T22:25:04.661925+0000","last_fullsized":"2023-07-13T22:25:04.661925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:34:54.913732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661425+0000","last_change":"2023-07-13T22:25:02.904741+0000","last_active":"2023-07-13T22:25:04.661425+0000","last_peered":"2023-07-13T22:25:04.661425+0000","last_clean":"2023-07-13T22:25:04.661425+0000","last_became_active":"2023-07-13T22:25:02.904477+0000","last_became_peered":"2023-07-13T22:25:02.904477+0000","last_unstale":"2023-07-13T22:25:04.661425+0000","last_undegraded":"2023-07-13T22:25:04.661425+0000","last_fullsized":"2023-07-13T22:25:04.661425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:38:22.743598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894453+0000","last_change":"2023-07-13T22:25:01.891393+0000","last_active":"2023-07-13T22:25:03.894453+0000","last_peered":"2023-07-13T22:25:03.894453+0000","last_clean":"2023-07-13T22:25:03.894453+0000","last_became_active":"2023-07-13T22:25:01.891110+0000","last_became_peered":"2023-07-13T22:25:01.891110+0000","last_unstale":"2023-07-13T22:25:03.894453+0000","last_undegraded":"2023-07-13T22:25:03.894453+0000","last_fullsized":"2023-07-13T22:25:03.894453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:13:55.614160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661468+0000","last_change":"2023-07-13T22:25:02.886611+0000","last_active":"2023-07-13T22:25:04.661468+0000","last_peered":"2023-07-13T22:25:04.661468+0000","last_clean":"2023-07-13T22:25:04.661468+0000","last_became_active":"2023-07-13T22:25:02.886347+0000","last_became_peered":"2023-07-13T22:25:02.886347+0000","last_unstale":"2023-07-13T22:25:04.661468+0000","last_undegraded":"2023-07-13T22:25:04.661468+0000","last_fullsized":"2023-07-13T22:25:04.661468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:13:05.407426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661129+0000","last_change":"2023-07-13T22:25:01.891579+0000","last_active":"2023-07-13T22:25:04.661129+0000","last_peered":"2023-07-13T22:25:04.661129+0000","last_clean":"2023-07-13T22:25:04.661129+0000","last_became_active":"2023-07-13T22:25:01.891284+0000","last_became_peered":"2023-07-13T22:25:01.891284+0000","last_unstale":"2023-07-13T22:25:04.661129+0000","last_undegraded":"2023-07-13T22:25:04.661129+0000","last_fullsized":"2023-07-13T22:25:04.661129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:35:04.767631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894235+0000","last_change":"2023-07-13T22:25:02.898624+0000","last_active":"2023-07-13T22:25:03.894235+0000","last_peered":"2023-07-13T22:25:03.894235+0000","last_clean":"2023-07-13T22:25:03.894235+0000","last_became_active":"2023-07-13T22:25:02.898357+0000","last_became_peered":"2023-07-13T22:25:02.898357+0000","last_unstale":"2023-07-13T22:25:03.894235+0000","last_undegraded":"2023-07-13T22:25:03.894235+0000","last_fullsized":"2023-07-13T22:25:03.894235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:17:55.114100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892794+0000","last_change":"2023-07-13T22:25:01.889495+0000","last_active":"2023-07-13T22:25:03.892794+0000","last_peered":"2023-07-13T22:25:03.892794+0000","last_clean":"2023-07-13T22:25:03.892794+0000","last_became_active":"2023-07-13T22:25:01.889121+0000","last_became_peered":"2023-07-13T22:25:01.889121+0000","last_unstale":"2023-07-13T22:25:03.892794+0000","last_undegraded":"2023-07-13T22:25:03.892794+0000","last_fullsized":"2023-07-13T22:25:03.892794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:36:53.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894650+0000","last_change":"2023-07-13T22:25:02.894652+0000","last_active":"2023-07-13T22:25:03.894650+0000","last_peered":"2023-07-13T22:25:03.894650+0000","last_clean":"2023-07-13T22:25:03.894650+0000","last_became_active":"2023-07-13T22:25:02.894360+0000","last_became_peered":"2023-07-13T22:25:02.894360+0000","last_unstale":"2023-07-13T22:25:03.894650+0000","last_undegraded":"2023-07-13T22:25:03.894650+0000","last_fullsized":"2023-07-13T22:25:03.894650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:54:36.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661334+0000","last_change":"2023-07-13T22:25:01.887450+0000","last_active":"2023-07-13T22:25:04.661334+0000","last_peered":"2023-07-13T22:25:04.661334+0000","last_clean":"2023-07-13T22:25:04.661334+0000","last_became_active":"2023-07-13T22:25:01.887182+0000","last_became_peered":"2023-07-13T22:25:01.887182+0000","last_unstale":"2023-07-13T22:25:04.661334+0000","last_undegraded":"2023-07-13T22:25:04.661334+0000","last_fullsized":"2023-07-13T22:25:04.661334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:40:30.826920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894940+0000","last_change":"2023-07-13T22:25:02.904910+0000","last_active":"2023-07-13T22:25:03.894940+0000","last_peered":"2023-07-13T22:25:03.894940+0000","last_clean":"2023-07-13T22:25:03.894940+0000","last_became_active":"2023-07-13T22:25:02.904593+0000","last_became_peered":"2023-07-13T22:25:02.904593+0000","last_unstale":"2023-07-13T22:25:03.894940+0000","last_undegraded":"2023-07-13T22:25:03.894940+0000","last_fullsized":"2023-07-13T22:25:03.894940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:08:49.001753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016065+0000","last_change":"2023-07-13T22:25:01.896800+0000","last_active":"2023-07-13T22:25:04.016065+0000","last_peered":"2023-07-13T22:25:04.016065+0000","last_clean":"2023-07-13T22:25:04.016065+0000","last_became_active":"2023-07-13T22:25:01.896421+0000","last_became_peered":"2023-07-13T22:25:01.896421+0000","last_unstale":"2023-07-13T22:25:04.016065+0000","last_undegraded":"2023-07-13T22:25:04.016065+0000","last_fullsized":"2023-07-13T22:25:04.016065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:52:28.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571290+0000","last_change":"2023-07-13T22:25:02.895020+0000","last_active":"2023-07-13T22:25:04.571290+0000","last_peered":"2023-07-13T22:25:04.571290+0000","last_clean":"2023-07-13T22:25:04.571290+0000","last_became_active":"2023-07-13T22:25:02.894733+0000","last_became_peered":"2023-07-13T22:25:02.894733+0000","last_unstale":"2023-07-13T22:25:04.571290+0000","last_undegraded":"2023-07-13T22:25:04.571290+0000","last_fullsized":"2023-07-13T22:25:04.571290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:07:50.443338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661338+0000","last_change":"2023-07-13T22:25:01.886175+0000","last_active":"2023-07-13T22:25:04.661338+0000","last_peered":"2023-07-13T22:25:04.661338+0000","last_clean":"2023-07-13T22:25:04.661338+0000","last_became_active":"2023-07-13T22:25:01.885875+0000","last_became_peered":"2023-07-13T22:25:01.885875+0000","last_unstale":"2023-07-13T22:25:04.661338+0000","last_undegraded":"2023-07-13T22:25:04.661338+0000","last_fullsized":"2023-07-13T22:25:04.661338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:45:36.294115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894132+0000","last_change":"2023-07-13T22:25:02.904352+0000","last_active":"2023-07-13T22:25:03.894132+0000","last_peered":"2023-07-13T22:25:03.894132+0000","last_clean":"2023-07-13T22:25:03.894132+0000","last_became_active":"2023-07-13T22:25:02.904054+0000","last_became_peered":"2023-07-13T22:25:02.904054+0000","last_unstale":"2023-07-13T22:25:03.894132+0000","last_undegraded":"2023-07-13T22:25:03.894132+0000","last_fullsized":"2023-07-13T22:25:03.894132+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:08.913386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016730+0000","last_change":"2023-07-13T22:25:01.894640+0000","last_active":"2023-07-13T22:25:04.016730+0000","last_peered":"2023-07-13T22:25:04.016730+0000","last_clean":"2023-07-13T22:25:04.016730+0000","last_became_active":"2023-07-13T22:25:01.894228+0000","last_became_peered":"2023-07-13T22:25:01.894228+0000","last_unstale":"2023-07-13T22:25:04.016730+0000","last_undegraded":"2023-07-13T22:25:04.016730+0000","last_fullsized":"2023-07-13T22:25:04.016730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:06:57.166270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571407+0000","last_change":"2023-07-13T22:25:02.890599+0000","last_active":"2023-07-13T22:25:04.571407+0000","last_peered":"2023-07-13T22:25:04.571407+0000","last_clean":"2023-07-13T22:25:04.571407+0000","last_became_active":"2023-07-13T22:25:02.890318+0000","last_became_peered":"2023-07-13T22:25:02.890318+0000","last_unstale":"2023-07-13T22:25:04.571407+0000","last_undegraded":"2023-07-13T22:25:04.571407+0000","last_fullsized":"2023-07-13T22:25:04.571407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:21.200021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893162+0000","last_change":"2023-07-13T22:25:01.890328+0000","last_active":"2023-07-13T22:25:03.893162+0000","last_peered":"2023-07-13T22:25:03.893162+0000","last_clean":"2023-07-13T22:25:03.893162+0000","last_became_active":"2023-07-13T22:25:01.890053+0000","last_became_peered":"2023-07-13T22:25:01.890053+0000","last_unstale":"2023-07-13T22:25:03.893162+0000","last_undegraded":"2023-07-13T22:25:03.893162+0000","last_fullsized":"2023-07-13T22:25:03.893162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:58:44.813230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016532+0000","last_change":"2023-07-13T22:25:02.898303+0000","last_active":"2023-07-13T22:25:04.016532+0000","last_peered":"2023-07-13T22:25:04.016532+0000","last_clean":"2023-07-13T22:25:04.016532+0000","last_became_active":"2023-07-13T22:25:02.898039+0000","last_became_peered":"2023-07-13T22:25:02.898039+0000","last_unstale":"2023-07-13T22:25:04.016532+0000","last_undegraded":"2023-07-13T22:25:04.016532+0000","last_fullsized":"2023-07-13T22:25:04.016532+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:20:33.736576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892968+0000","last_change":"2023-07-13T22:25:01.897511+0000","last_active":"2023-07-13T22:25:03.892968+0000","last_peered":"2023-07-13T22:25:03.892968+0000","last_clean":"2023-07-13T22:25:03.892968+0000","last_became_active":"2023-07-13T22:25:01.897181+0000","last_became_peered":"2023-07-13T22:25:01.897181+0000","last_unstale":"2023-07-13T22:25:03.892968+0000","last_undegraded":"2023-07-13T22:25:03.892968+0000","last_fullsized":"2023-07-13T22:25:03.892968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:46.290911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894534+0000","last_change":"2023-07-13T22:25:02.896731+0000","last_active":"2023-07-13T22:25:03.894534+0000","last_peered":"2023-07-13T22:25:03.894534+0000","last_clean":"2023-07-13T22:25:03.894534+0000","last_became_active":"2023-07-13T22:25:02.896138+0000","last_became_peered":"2023-07-13T22:25:02.896138+0000","last_unstale":"2023-07-13T22:25:03.894534+0000","last_undegraded":"2023-07-13T22:25:03.894534+0000","last_fullsized":"2023-07-13T22:25:03.894534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:54:06.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894331+0000","last_change":"2023-07-13T22:25:01.896045+0000","last_active":"2023-07-13T22:25:03.894331+0000","last_peered":"2023-07-13T22:25:03.894331+0000","last_clean":"2023-07-13T22:25:03.894331+0000","last_became_active":"2023-07-13T22:25:01.895768+0000","last_became_peered":"2023-07-13T22:25:01.895768+0000","last_unstale":"2023-07-13T22:25:03.894331+0000","last_undegraded":"2023-07-13T22:25:03.894331+0000","last_fullsized":"2023-07-13T22:25:03.894331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:20:24.658129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894594+0000","last_change":"2023-07-13T22:25:02.903163+0000","last_active":"2023-07-13T22:25:03.894594+0000","last_peered":"2023-07-13T22:25:03.894594+0000","last_clean":"2023-07-13T22:25:03.894594+0000","last_became_active":"2023-07-13T22:25:02.902447+0000","last_became_peered":"2023-07-13T22:25:02.902447+0000","last_unstale":"2023-07-13T22:25:03.894594+0000","last_undegraded":"2023-07-13T22:25:03.894594+0000","last_fullsized":"2023-07-13T22:25:03.894594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:33:11.983924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661926+0000","last_change":"2023-07-13T22:25:01.886901+0000","last_active":"2023-07-13T22:25:04.661926+0000","last_peered":"2023-07-13T22:25:04.661926+0000","last_clean":"2023-07-13T22:25:04.661926+0000","last_became_active":"2023-07-13T22:25:01.886620+0000","last_became_peered":"2023-07-13T22:25:01.886620+0000","last_unstale":"2023-07-13T22:25:04.661926+0000","last_undegraded":"2023-07-13T22:25:04.661926+0000","last_fullsized":"2023-07-13T22:25:04.661926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:17:25.586012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661732+0000","last_change":"2023-07-13T22:25:02.904314+0000","last_active":"2023-07-13T22:25:04.661732+0000","last_peered":"2023-07-13T22:25:04.661732+0000","last_clean":"2023-07-13T22:25:04.661732+0000","last_became_active":"2023-07-13T22:25:02.904038+0000","last_became_peered":"2023-07-13T22:25:02.904038+0000","last_unstale":"2023-07-13T22:25:04.661732+0000","last_undegraded":"2023-07-13T22:25:04.661732+0000","last_fullsized":"2023-07-13T22:25:04.661732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:54:36.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894426+0000","last_change":"2023-07-13T22:25:01.895277+0000","last_active":"2023-07-13T22:25:03.894426+0000","last_peered":"2023-07-13T22:25:03.894426+0000","last_clean":"2023-07-13T22:25:03.894426+0000","last_became_active":"2023-07-13T22:25:01.895006+0000","last_became_peered":"2023-07-13T22:25:01.895006+0000","last_unstale":"2023-07-13T22:25:03.894426+0000","last_undegraded":"2023-07-13T22:25:03.894426+0000","last_fullsized":"2023-07-13T22:25:03.894426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:38:21.737122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661504+0000","last_change":"2023-07-13T22:25:02.888475+0000","last_active":"2023-07-13T22:25:04.661504+0000","last_peered":"2023-07-13T22:25:04.661504+0000","last_clean":"2023-07-13T22:25:04.661504+0000","last_became_active":"2023-07-13T22:25:02.888077+0000","last_became_peered":"2023-07-13T22:25:02.888077+0000","last_unstale":"2023-07-13T22:25:04.661504+0000","last_undegraded":"2023-07-13T22:25:04.661504+0000","last_fullsized":"2023-07-13T22:25:04.661504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:13:56.620635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.660878+0000","last_change":"2023-07-13T22:25:01.885772+0000","last_active":"2023-07-13T22:25:04.660878+0000","last_peered":"2023-07-13T22:25:04.660878+0000","last_clean":"2023-07-13T22:25:04.660878+0000","last_became_active":"2023-07-13T22:25:01.885461+0000","last_became_peered":"2023-07-13T22:25:01.885461+0000","last_unstale":"2023-07-13T22:25:04.660878+0000","last_undegraded":"2023-07-13T22:25:04.660878+0000","last_fullsized":"2023-07-13T22:25:04.660878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:07.906911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891685+0000","last_change":"2023-07-13T22:25:02.911564+0000","last_active":"2023-07-13T22:25:03.891685+0000","last_peered":"2023-07-13T22:25:03.891685+0000","last_clean":"2023-07-13T22:25:03.891685+0000","last_became_active":"2023-07-13T22:25:02.911264+0000","last_became_peered":"2023-07-13T22:25:02.911264+0000","last_unstale":"2023-07-13T22:25:03.891685+0000","last_undegraded":"2023-07-13T22:25:03.891685+0000","last_fullsized":"2023-07-13T22:25:03.891685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:07:15.256799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893353+0000","last_change":"2023-07-13T22:25:01.891948+0000","last_active":"2023-07-13T22:25:03.893353+0000","last_peered":"2023-07-13T22:25:03.893353+0000","last_clean":"2023-07-13T22:25:03.893353+0000","last_became_active":"2023-07-13T22:25:01.891595+0000","last_became_peered":"2023-07-13T22:25:01.891595+0000","last_unstale":"2023-07-13T22:25:03.893353+0000","last_undegraded":"2023-07-13T22:25:03.893353+0000","last_fullsized":"2023-07-13T22:25:03.893353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:45:57.113798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571564+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:25:04.571564+0000","last_peered":"2023-07-13T22:25:04.571564+0000","last_clean":"2023-07-13T22:25:04.571564+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:25:04.571564+0000","last_undegraded":"2023-07-13T22:25:04.571564+0000","last_fullsized":"2023-07-13T22:25:04.571564+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:53:12.722752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661540+0000","last_change":"2023-07-13T22:25:02.894621+0000","last_active":"2023-07-13T22:25:04.661540+0000","last_peered":"2023-07-13T22:25:04.661540+0000","last_clean":"2023-07-13T22:25:04.661540+0000","last_became_active":"2023-07-13T22:25:02.894328+0000","last_became_peered":"2023-07-13T22:25:02.894328+0000","last_unstale":"2023-07-13T22:25:04.661540+0000","last_undegraded":"2023-07-13T22:25:04.661540+0000","last_fullsized":"2023-07-13T22:25:04.661540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:21:11.546254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571840+0000","last_change":"2023-07-13T22:25:01.885197+0000","last_active":"2023-07-13T22:25:04.571840+0000","last_peered":"2023-07-13T22:25:04.571840+0000","last_clean":"2023-07-13T22:25:04.571840+0000","last_became_active":"2023-07-13T22:25:01.884915+0000","last_became_peered":"2023-07-13T22:25:01.884915+0000","last_unstale":"2023-07-13T22:25:04.571840+0000","last_undegraded":"2023-07-13T22:25:04.571840+0000","last_fullsized":"2023-07-13T22:25:04.571840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:06:57.166270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891962+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:25:03.891962+0000","last_peered":"2023-07-13T22:25:03.891962+0000","last_clean":"2023-07-13T22:25:03.891962+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:25:03.891962+0000","last_undegraded":"2023-07-13T22:25:03.891962+0000","last_fullsized":"2023-07-13T22:25:03.891962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:58:03.089682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016518+0000","last_change":"2023-07-13T22:25:02.902508+0000","last_active":"2023-07-13T22:25:04.016518+0000","last_peered":"2023-07-13T22:25:04.016518+0000","last_clean":"2023-07-13T22:25:04.016518+0000","last_became_active":"2023-07-13T22:25:02.902144+0000","last_became_peered":"2023-07-13T22:25:02.902144+0000","last_unstale":"2023-07-13T22:25:04.016518+0000","last_undegraded":"2023-07-13T22:25:04.016518+0000","last_fullsized":"2023-07-13T22:25:04.016518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:21.200021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891743+0000","last_change":"2023-07-13T22:25:01.893555+0000","last_active":"2023-07-13T22:25:03.891743+0000","last_peered":"2023-07-13T22:25:03.891743+0000","last_clean":"2023-07-13T22:25:03.891743+0000","last_became_active":"2023-07-13T22:25:01.893227+0000","last_became_peered":"2023-07-13T22:25:01.893227+0000","last_unstale":"2023-07-13T22:25:03.891743+0000","last_undegraded":"2023-07-13T22:25:03.891743+0000","last_fullsized":"2023-07-13T22:25:03.891743+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:37:08.777840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661231+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:25:04.661231+0000","last_peered":"2023-07-13T22:25:04.661231+0000","last_clean":"2023-07-13T22:25:04.661231+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:25:04.661231+0000","last_undegraded":"2023-07-13T22:25:04.661231+0000","last_fullsized":"2023-07-13T22:25:04.661231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:10.506743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893243+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:25:03.893243+0000","last_peered":"2023-07-13T22:25:03.893243+0000","last_clean":"2023-07-13T22:25:03.893243+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:25:03.893243+0000","last_undegraded":"2023-07-13T22:25:03.893243+0000","last_fullsized":"2023-07-13T22:25:03.893243+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:35:17.610266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894900+0000","last_change":"2023-07-13T22:25:02.895995+0000","last_active":"2023-07-13T22:25:03.894900+0000","last_peered":"2023-07-13T22:25:03.894900+0000","last_clean":"2023-07-13T22:25:03.894900+0000","last_became_active":"2023-07-13T22:25:02.895724+0000","last_became_peered":"2023-07-13T22:25:02.895724+0000","last_unstale":"2023-07-13T22:25:03.894900+0000","last_undegraded":"2023-07-13T22:25:03.894900+0000","last_fullsized":"2023-07-13T22:25:03.894900+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:35:33.390458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891785+0000","last_change":"2023-07-13T22:25:01.884525+0000","last_active":"2023-07-13T22:25:03.891785+0000","last_peered":"2023-07-13T22:25:03.891785+0000","last_clean":"2023-07-13T22:25:03.891785+0000","last_became_active":"2023-07-13T22:25:01.884186+0000","last_became_peered":"2023-07-13T22:25:01.884186+0000","last_unstale":"2023-07-13T22:25:03.891785+0000","last_undegraded":"2023-07-13T22:25:03.891785+0000","last_fullsized":"2023-07-13T22:25:03.891785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:12:18.097405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893075+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:25:03.893075+0000","last_peered":"2023-07-13T22:25:03.893075+0000","last_clean":"2023-07-13T22:25:03.893075+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:25:03.893075+0000","last_undegraded":"2023-07-13T22:25:03.893075+0000","last_fullsized":"2023-07-13T22:25:03.893075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:23:01.801448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893029+0000","last_change":"2023-07-13T22:25:02.896598+0000","last_active":"2023-07-13T22:25:03.893029+0000","last_peered":"2023-07-13T22:25:03.893029+0000","last_clean":"2023-07-13T22:25:03.893029+0000","last_became_active":"2023-07-13T22:25:02.896299+0000","last_became_peered":"2023-07-13T22:25:02.896299+0000","last_unstale":"2023-07-13T22:25:03.893029+0000","last_undegraded":"2023-07-13T22:25:03.893029+0000","last_fullsized":"2023-07-13T22:25:03.893029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:31:31.326824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892077+0000","last_change":"2023-07-13T22:25:01.892891+0000","last_active":"2023-07-13T22:25:03.892077+0000","last_peered":"2023-07-13T22:25:03.892077+0000","last_clean":"2023-07-13T22:25:03.892077+0000","last_became_active":"2023-07-13T22:25:01.892598+0000","last_became_peered":"2023-07-13T22:25:01.892598+0000","last_unstale":"2023-07-13T22:25:03.892077+0000","last_undegraded":"2023-07-13T22:25:03.892077+0000","last_fullsized":"2023-07-13T22:25:03.892077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:43:54.995576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893423+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:25:03.893423+0000","last_peered":"2023-07-13T22:25:03.893423+0000","last_clean":"2023-07-13T22:25:03.893423+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:25:03.893423+0000","last_undegraded":"2023-07-13T22:25:03.893423+0000","last_fullsized":"2023-07-13T22:25:03.893423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:48:04.740879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016628+0000","last_change":"2023-07-13T22:25:02.899939+0000","last_active":"2023-07-13T22:25:04.016628+0000","last_peered":"2023-07-13T22:25:04.016628+0000","last_clean":"2023-07-13T22:25:04.016628+0000","last_became_active":"2023-07-13T22:25:02.899512+0000","last_became_peered":"2023-07-13T22:25:02.899512+0000","last_unstale":"2023-07-13T22:25:04.016628+0000","last_undegraded":"2023-07-13T22:25:04.016628+0000","last_fullsized":"2023-07-13T22:25:04.016628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:57:27.880613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891952+0000","last_change":"2023-07-13T22:25:01.888706+0000","last_active":"2023-07-13T22:25:03.891952+0000","last_peered":"2023-07-13T22:25:03.891952+0000","last_clean":"2023-07-13T22:25:03.891952+0000","last_became_active":"2023-07-13T22:25:01.888403+0000","last_became_peered":"2023-07-13T22:25:01.888403+0000","last_unstale":"2023-07-13T22:25:03.891952+0000","last_undegraded":"2023-07-13T22:25:03.891952+0000","last_fullsized":"2023-07-13T22:25:03.891952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:39:32.541068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.660898+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:25:04.660898+0000","last_peered":"2023-07-13T22:25:04.660898+0000","last_clean":"2023-07-13T22:25:04.660898+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:25:04.660898+0000","last_undegraded":"2023-07-13T22:25:04.660898+0000","last_fullsized":"2023-07-13T22:25:04.660898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:01:16.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891958+0000","last_change":"2023-07-13T22:25:02.902133+0000","last_active":"2023-07-13T22:25:03.891958+0000","last_peered":"2023-07-13T22:25:03.891958+0000","last_clean":"2023-07-13T22:25:03.891958+0000","last_became_active":"2023-07-13T22:25:02.901319+0000","last_became_peered":"2023-07-13T22:25:02.901319+0000","last_unstale":"2023-07-13T22:25:03.891958+0000","last_undegraded":"2023-07-13T22:25:03.891958+0000","last_fullsized":"2023-07-13T22:25:03.891958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:12:46.956883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661239+0000","last_change":"2023-07-13T22:25:01.886358+0000","last_active":"2023-07-13T22:25:04.661239+0000","last_peered":"2023-07-13T22:25:04.661239+0000","last_clean":"2023-07-13T22:25:04.661239+0000","last_became_active":"2023-07-13T22:25:01.885833+0000","last_became_peered":"2023-07-13T22:25:01.885833+0000","last_unstale":"2023-07-13T22:25:04.661239+0000","last_undegraded":"2023-07-13T22:25:04.661239+0000","last_fullsized":"2023-07-13T22:25:04.661239+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:19:49.849385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894682+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:25:03.894682+0000","last_peered":"2023-07-13T22:25:03.894682+0000","last_clean":"2023-07-13T22:25:03.894682+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:25:03.894682+0000","last_undegraded":"2023-07-13T22:25:03.894682+0000","last_fullsized":"2023-07-13T22:25:03.894682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:23:41.104604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661932+0000","last_change":"2023-07-13T22:25:02.908309+0000","last_active":"2023-07-13T22:25:04.661932+0000","last_peered":"2023-07-13T22:25:04.661932+0000","last_clean":"2023-07-13T22:25:04.661932+0000","last_became_active":"2023-07-13T22:25:02.908036+0000","last_became_peered":"2023-07-13T22:25:02.908036+0000","last_unstale":"2023-07-13T22:25:04.661932+0000","last_undegraded":"2023-07-13T22:25:04.661932+0000","last_fullsized":"2023-07-13T22:25:04.661932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:43:29.685005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.662073+0000","last_change":"2023-07-13T22:25:01.888841+0000","last_active":"2023-07-13T22:25:04.662073+0000","last_peered":"2023-07-13T22:25:04.662073+0000","last_clean":"2023-07-13T22:25:04.662073+0000","last_became_active":"2023-07-13T22:25:01.888503+0000","last_became_peered":"2023-07-13T22:25:01.888503+0000","last_unstale":"2023-07-13T22:25:04.662073+0000","last_undegraded":"2023-07-13T22:25:04.662073+0000","last_fullsized":"2023-07-13T22:25:04.662073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:48:33.115431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894993+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:25:03.894993+0000","last_peered":"2023-07-13T22:25:03.894993+0000","last_clean":"2023-07-13T22:25:03.894993+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:25:03.894993+0000","last_undegraded":"2023-07-13T22:25:03.894993+0000","last_fullsized":"2023-07-13T22:25:03.894993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:28:08.944929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36620,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335220,"statfs":{"total":96636764160,"available":96599265280,"internally_reserved":0,"allocated":1511424,"data_stored":1002102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.89900000000000002,"5min":1.115,"15min":1.115},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88200000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64500000000000002,"15min":0.70099999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94799999999999995,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84299999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.74399999999999999,"15min":0.80400000000000005},"min":{"1min":0.50900000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92500000000000004,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.748},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.72499999999999998,"15min":0.80000000000000004},"min":{"1min":0.45500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93000000000000005,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.62}]},{"osd":3,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.74299999999999999,"15min":0.75900000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.83999999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.65400000000000003},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81299999999999994,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91400000000000003,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83899999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.5},{"interface":"front","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.79400000000000004},"min":{"1min":0.57799999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.022,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.437,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88900000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":1.042,"15min":1.042},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":925696,"data_stored":413276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89900000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82199999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66100000000000003,"15min":0.68799999999999994},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84199999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.38100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.88300000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76000000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65300000000000002,"15min":0.64600000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.86199999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.67100000000000004},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86899999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67900000000000005,"15min":0.70099999999999996},"min":{"1min":0.40699999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94199999999999995,"5min":0.998,"15min":0.998},"last":0.87}]},{"osd":5,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94199999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.58199999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.95399999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.96999999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91900000000000004,"5min":1.111,"15min":1.111},"last":0.91700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36624,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335216,"statfs":{"total":96636764160,"available":96599261184,"internally_reserved":0,"allocated":1515520,"data_stored":1002192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.504},"min":{"1min":0.313,"5min":0.249,"15min":0.249},"max":{"1min":0.71599999999999997,"5min":1.952,"15min":1.952},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.48699999999999999,"15min":0.51900000000000002},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58499999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.499,"15min":0.55300000000000005},"min":{"1min":0.34000000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.52300000000000002,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.437},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.61099999999999999},"min":{"1min":0.42799999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65900000000000003,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59599999999999997,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82499999999999996,"5min":2.008,"15min":2.008},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51900000000000002,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64600000000000002,"5min":0.87,"15min":0.87},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.76400000000000001,"15min":0.81799999999999995},"min":{"1min":0.59499999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89300000000000002,"5min":2.028,"15min":2.028},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.755,"15min":0.80500000000000005},"min":{"1min":0.61199999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.87,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.624}]},{"osd":5,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.74099999999999999,"15min":0.76500000000000001},"min":{"1min":0.55400000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.92900000000000005,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79600000000000004,"15min":0.86199999999999999},"min":{"1min":0.54400000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88900000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76900000000000002,"15min":0.80600000000000005},"min":{"1min":0.57199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98299999999999998,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.80600000000000005,"15min":0.88300000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91700000000000004,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.83499999999999996},"min":{"1min":0.65400000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.92000000000000004,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78100000000000003,"15min":0.82699999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36032,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335808,"statfs":{"total":96636764160,"available":96599867392,"internally_reserved":0,"allocated":909312,"data_stored":410726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.58399999999999996,"15min":0.52800000000000002},"min":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.51100000000000001,"5min":11.791,"15min":11.791},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.33900000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90700000000000003,"5min":4.657,"15min":4.657},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.625,"15min":0.60899999999999999},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.746,"5min":4.5389999999999997,"15min":4.5389999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.92600000000000005,"15min":1.097},"min":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89200000000000002,"5min":13.398999999999999,"15min":13.398999999999999},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.54200000000000004,"15min":0.57999999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60799999999999998,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58399999999999996,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":1.139,"15min":1.353},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89400000000000002,"5min":13.667999999999999,"15min":13.667999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77500000000000002,"15min":0.82299999999999995},"min":{"1min":0.51400000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84299999999999997,"5min":1.776,"15min":1.776},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":1.157,"15min":1.365},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85299999999999998,"5min":13.349,"15min":13.349},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.85899999999999999,"15min":0.88600000000000001},"min":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.97199999999999998,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.52800000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.85499999999999998,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.0900000000000001,"15min":1.3400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92900000000000005,"5min":11.679,"15min":11.679},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":1.169,"15min":1.373},"min":{"1min":0.53800000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.83399999999999996,"5min":13.298,"15min":13.298},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.93500000000000005,"15min":0.90700000000000003},"min":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84699999999999998,"5min":11.456,"15min":11.456},"last":0.76800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":921600,"data_stored":411468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48899999999999999,"15min":0.56000000000000005},"min":{"1min":0.252,"5min":0.249,"15min":0.249},"max":{"1min":0.75,"5min":2.282,"15min":2.282},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.46600000000000003},"min":{"1min":0.30399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.57199999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.55200000000000005,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66600000000000004,"5min":2.319,"15min":2.319},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.56699999999999995},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.88100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60299999999999998,"15min":0.65900000000000003},"min":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93600000000000005,"5min":2.351,"15min":2.351},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.79100000000000004,"15min":0.85699999999999998},"min":{"1min":0.56999999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.88700000000000001,"5min":2.238,"15min":2.238},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.47499999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0349999999999999,"5min":1.518,"15min":1.518},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.76900000000000002,"15min":0.78000000000000003},"min":{"1min":0.58099999999999996,"5min":0.505,"15min":0.505},"max":{"1min":0.90600000000000003,"5min":1.325,"15min":1.325},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.80500000000000005,"15min":0.871},"min":{"1min":0.59699999999999998,"5min":0.495,"15min":0.495},"max":{"1min":0.90700000000000003,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.81200000000000006},"min":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1040000000000001,"5min":1.488,"15min":1.488},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.76500000000000001,"15min":0.78800000000000003},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86099999999999999,"5min":1.214,"15min":1.214},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77500000000000002,"15min":0.78600000000000003},"min":{"1min":0.64400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0640000000000001,"5min":1.29,"15min":1.29},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.79700000000000004,"15min":0.83299999999999996},"min":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95099999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.89600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36040,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335800,"statfs":{"total":96636764160,"available":96599859200,"internally_reserved":0,"allocated":917504,"data_stored":412294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41199999999999998,"15min":0.42399999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.59799999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44500000000000001,"15min":0.439},"min":{"1min":0.33300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.38}]},{"osd":1,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.623,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.55300000000000005},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72499999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1699999999999999,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73399999999999999,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.117,"5min":1.411,"15min":1.411},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.78300000000000003},"min":{"1min":0.51800000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1779999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.034,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.046,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77900000000000003,"15min":0.80600000000000005},"min":{"1min":0.51900000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.1299999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.085,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":933888,"data_stored":413380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79400000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69099999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.442,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81899999999999995,"5min":1.298,"15min":1.298},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.47999999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.77400000000000002,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69799999999999995,"15min":0.72699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.873,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69899999999999995,"15min":0.70799999999999996},"min":{"1min":0.52700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.78300000000000003,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.83599999999999997,"15min":1.0429999999999999},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79100000000000004,"5min":10.657999999999999,"15min":10.657999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.84799999999999998,"15min":1.002},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97099999999999997,"5min":8.75,"15min":8.75},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.50800000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84099999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.94499999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.45600000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":1.173,"15min":1.173},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.72899999999999998},"min":{"1min":0.51600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.84699999999999998,"5min":1.099,"15min":1.099},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.51300000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.82199999999999995,"5min":1.004,"15min":1.004},"last":0.873},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.497,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94799999999999995,"5min":1.212,"15min":1.212},"last":0.71099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36040,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335800,"statfs":{"total":96636764160,"available":96599859200,"internally_reserved":0,"allocated":917504,"data_stored":411746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.60499999999999998},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86299999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89000000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65800000000000003,"15min":0.64200000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.72099999999999997,"15min":0.66500000000000004},"min":{"1min":0.42799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":8.9930000000000003,"15min":8.9930000000000003},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.78500000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84699999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68400000000000005,"15min":0.67200000000000004},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84199999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88,"5min":0.995,"15min":0.995},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.495,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93200000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81799999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.72999999999999998},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":1.02,"15min":1.02},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.59099999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.92400000000000004,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.87}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:25:07.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:25:07.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.563+0000 7fa758bff700 1 Processor -- start 2023-07-13T22:25:07.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.564+0000 7fa758bff700 1 -- start start 2023-07-13T22:25:07.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.565+0000 7fa758bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7540d35c0 0x7fa7540d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.565+0000 7fa758bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7540d13f0 0x7fa75415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.565+0000 7fa758bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7540d09d0 con 0x7fa7540b8570 2023-07-13T22:25:07.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.565+0000 7fa758bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7540d3b30 con 0x7fa7540d13f0 2023-07-13T22:25:07.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.565+0000 7fa758bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7540cda10 con 0x7fa7540d35c0 2023-07-13T22:25:07.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.565+0000 7fa752d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7540d13f0 0x7fa75415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.565+0000 7fa752d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7540d13f0 0x7fa75415c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42004/0 (socket says 172.21.15.60:42004) 2023-07-13T22:25:07.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.565+0000 7fa752d9d700 1 -- 172.21.15.60:0/1671153977 learned_addr learned my addr 172.21.15.60:0/1671153977 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:07.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.565+0000 7fa751d9b700 1 --2- 172.21.15.60:0/1671153977 >> v2:172.21.15.105:3300/0 conn(0x7fa7540d35c0 0x7fa7540d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.566+0000 7fa752d9d700 1 -- 172.21.15.60:0/1671153977 >> v2:172.21.15.105:3300/0 conn(0x7fa7540d35c0 msgr2=0x7fa7540d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.566+0000 7fa752d9d700 1 --2- 172.21.15.60:0/1671153977 >> v2:172.21.15.105:3300/0 conn(0x7fa7540d35c0 0x7fa7540d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.566+0000 7fa752d9d700 1 -- 172.21.15.60:0/1671153977 >> v1:172.21.15.105:6789/0 conn(0x7fa7540b8570 legacy=0x7fa7540ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.566+0000 7fa752d9d700 1 -- 172.21.15.60:0/1671153977 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7540cd450 con 0x7fa7540d13f0 2023-07-13T22:25:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.566+0000 7fa751d9b700 1 --2- 172.21.15.60:0/1671153977 >> v2:172.21.15.105:3300/0 conn(0x7fa7540d35c0 0x7fa7540d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.566+0000 7fa752d9d700 1 --2- 172.21.15.60:0/1671153977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540d13f0 0x7fa75415c920 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa7540d07e0 tx=0x7fa74c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3348e2c31f461ba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.566+0000 7fa75159a700 1 -- 172.21.15.60:0/1671153977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00e070 con 0x7fa7540d13f0 2023-07-13T22:25:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.566+0000 7fa75159a700 1 -- 172.21.15.60:0/1671153977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c003a20 con 0x7fa7540d13f0 2023-07-13T22:25:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.566+0000 7fa75159a700 1 -- 172.21.15.60:0/1671153977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c003dd0 con 0x7fa7540d13f0 2023-07-13T22:25:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.567+0000 7fa758bff700 1 -- 172.21.15.60:0/1671153977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540d13f0 msgr2=0x7fa75415c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.567+0000 7fa758bff700 1 --2- 172.21.15.60:0/1671153977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540d13f0 0x7fa75415c920 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa7540d07e0 tx=0x7fa74c00b330 comp rx=0 tx=0).stop 2023-07-13T22:25:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.567+0000 7fa758bff700 1 -- 172.21.15.60:0/1671153977 shutdown_connections 2023-07-13T22:25:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.567+0000 7fa758bff700 1 --2- 172.21.15.60:0/1671153977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540d13f0 0x7fa75415c920 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.567+0000 7fa758bff700 1 --2- 172.21.15.60:0/1671153977 >> v2:172.21.15.105:3300/0 conn(0x7fa7540d35c0 0x7fa7540d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.567+0000 7fa758bff700 1 -- 172.21.15.60:0/1671153977 >> 172.21.15.60:0/1671153977 conn(0x7fa754071600 msgr2=0x7fa7540719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:07.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.567+0000 7fa758bff700 1 -- 172.21.15.60:0/1671153977 shutdown_connections 2023-07-13T22:25:07.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.567+0000 7fa758bff700 1 -- 172.21.15.60:0/1671153977 wait complete. 2023-07-13T22:25:07.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.568+0000 7fa758bff700 1 Processor -- start 2023-07-13T22:25:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.569+0000 7fa758bff700 1 -- start start 2023-07-13T22:25:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.569+0000 7fa758bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540b8570 0x7fa7540aab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.569+0000 7fa758bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7540d13f0 0x7fa75415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa758bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7540d35c0 0x7fa7540c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa758bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540a9dc0 con 0x7fa7540b8570 2023-07-13T22:25:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa758bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540aa0a0 con 0x7fa7540d35c0 2023-07-13T22:25:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa758bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540aa380 con 0x7fa7540d13f0 2023-07-13T22:25:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa75259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540b8570 0x7fa7540aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa75259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540b8570 0x7fa7540aab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42006/0 (socket says 172.21.15.60:42006) 2023-07-13T22:25:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa75259c700 1 -- 172.21.15.60:0/3163223725 learned_addr learned my addr 172.21.15.60:0/3163223725 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa751d9b700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7540d13f0 0x7fa75415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa752d9d700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7540d35c0 0x7fa7540c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa75259c700 1 -- 172.21.15.60:0/3163223725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7540d13f0 msgr2=0x7fa75415aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa75259c700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7540d13f0 0x7fa75415aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa75259c700 1 -- 172.21.15.60:0/3163223725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7540d35c0 msgr2=0x7fa7540c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa75259c700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7540d35c0 0x7fa7540c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa75259c700 1 -- 172.21.15.60:0/3163223725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c009b30 con 0x7fa7540b8570 2023-07-13T22:25:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.570+0000 7fa75259c700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540b8570 0x7fa7540aab80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa74400c4b0 tx=0x7fa74400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.571+0000 7fa742ffd700 1 -- 172.21.15.60:0/3163223725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744011790 con 0x7fa7540b8570 2023-07-13T22:25:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.571+0000 7fa742ffd700 1 -- 172.21.15.60:0/3163223725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7440118f0 con 0x7fa7540b8570 2023-07-13T22:25:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.571+0000 7fa742ffd700 1 -- 172.21.15.60:0/3163223725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744011ca0 con 0x7fa7540b8570 2023-07-13T22:25:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.571+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7540aa660 con 0x7fa7540b8570 2023-07-13T22:25:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.571+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7540c8370 con 0x7fa7540b8570 2023-07-13T22:25:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.572+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7540aa880 con 0x7fa7540b8570 2023-07-13T22:25:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.573+0000 7fa742ffd700 1 -- 172.21.15.60:0/3163223725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa744012420 con 0x7fa7540b8570 2023-07-13T22:25:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.574+0000 7fa742ffd700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa73406ace0 0x7fa73406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.574+0000 7fa742ffd700 1 -- 172.21.15.60:0/3163223725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa744088ff0 con 0x7fa7540b8570 2023-07-13T22:25:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.574+0000 7fa751d9b700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa73406ace0 0x7fa73406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.575+0000 7fa751d9b700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa73406ace0 0x7fa73406d190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa748006fd0 tx=0x7fa748008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.578+0000 7fa742ffd700 1 -- 172.21.15.60:0/3163223725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa7440bb330 con 0x7fa7540b8570 2023-07-13T22:25:07.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.780+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7540d1830 con 0x7fa73406ace0 2023-07-13T22:25:07.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.789+0000 7fa742ffd700 1 -- 172.21.15.60:0/3163223725 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311541 (secure 0 0 0) 0x7fa74800a720 con 0x7fa73406ace0 2023-07-13T22:25:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:07.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa73406ace0 msgr2=0x7fa73406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa73406ace0 0x7fa73406d190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa748006fd0 tx=0x7fa748008040 comp rx=0 tx=0).stop 2023-07-13T22:25:07.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540b8570 msgr2=0x7fa7540aab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540b8570 0x7fa7540aab80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa74400c4b0 tx=0x7fa74400c870 comp rx=0 tx=0).stop 2023-07-13T22:25:07.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 shutdown_connections 2023-07-13T22:25:07.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa73406ace0 0x7fa73406d190 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7540b8570 0x7fa7540aab80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7540d35c0 0x7fa7540c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 --2- 172.21.15.60:0/3163223725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7540d13f0 0x7fa75415aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 >> 172.21.15.60:0/3163223725 conn(0x7fa754071600 msgr2=0x7fa754073510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:07.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 shutdown_connections 2023-07-13T22:25:07.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:07.794+0000 7fa758bff700 1 -- 172.21.15.60:0/3163223725 wait complete. 2023-07-13T22:25:07.780 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:25:07.794 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2023-07-13T22:25:07.175603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277216,"kb_used_data":8352,"kb_used_omap":0,"kb_used_meta":268800,"kb_avail":754697504,"statfs":{"total":773094113280,"available":772810244096,"internally_reserved":0,"allocated":8552448,"data_stored":4477184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":275251200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000925"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893253+0000","last_change":"2023-07-13T22:25:02.892413+0000","last_active":"2023-07-13T22:25:03.893253+0000","last_peered":"2023-07-13T22:25:03.893253+0000","last_clean":"2023-07-13T22:25:03.893253+0000","last_became_active":"2023-07-13T22:25:02.892159+0000","last_became_peered":"2023-07-13T22:25:02.892159+0000","last_unstale":"2023-07-13T22:25:03.893253+0000","last_undegraded":"2023-07-13T22:25:03.893253+0000","last_fullsized":"2023-07-13T22:25:03.893253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:16:31.837939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016469+0000","last_change":"2023-07-13T22:25:02.910258+0000","last_active":"2023-07-13T22:25:04.016469+0000","last_peered":"2023-07-13T22:25:04.016469+0000","last_clean":"2023-07-13T22:25:04.016469+0000","last_became_active":"2023-07-13T22:25:02.909979+0000","last_became_peered":"2023-07-13T22:25:02.909979+0000","last_unstale":"2023-07-13T22:25:04.016469+0000","last_undegraded":"2023-07-13T22:25:04.016469+0000","last_fullsized":"2023-07-13T22:25:04.016469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:07:50.443338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661693+0000","last_change":"2023-07-13T22:25:02.904222+0000","last_active":"2023-07-13T22:25:04.661693+0000","last_peered":"2023-07-13T22:25:04.661693+0000","last_clean":"2023-07-13T22:25:04.661693+0000","last_became_active":"2023-07-13T22:25:02.903890+0000","last_became_peered":"2023-07-13T22:25:02.903890+0000","last_unstale":"2023-07-13T22:25:04.661693+0000","last_undegraded":"2023-07-13T22:25:04.661693+0000","last_fullsized":"2023-07-13T22:25:04.661693+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:52:46.539330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661257+0000","last_change":"2023-07-13T22:25:02.883184+0000","last_active":"2023-07-13T22:25:04.661257+0000","last_peered":"2023-07-13T22:25:04.661257+0000","last_clean":"2023-07-13T22:25:04.661257+0000","last_became_active":"2023-07-13T22:25:02.882874+0000","last_became_peered":"2023-07-13T22:25:02.882874+0000","last_unstale":"2023-07-13T22:25:04.661257+0000","last_undegraded":"2023-07-13T22:25:04.661257+0000","last_fullsized":"2023-07-13T22:25:04.661257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:00:48.053942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661447+0000","last_change":"2023-07-13T22:25:02.889343+0000","last_active":"2023-07-13T22:25:04.661447+0000","last_peered":"2023-07-13T22:25:04.661447+0000","last_clean":"2023-07-13T22:25:04.661447+0000","last_became_active":"2023-07-13T22:25:02.889018+0000","last_became_peered":"2023-07-13T22:25:02.889018+0000","last_unstale":"2023-07-13T22:25:04.661447+0000","last_undegraded":"2023-07-13T22:25:04.661447+0000","last_fullsized":"2023-07-13T22:25:04.661447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:01.720916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892321+0000","last_change":"2023-07-13T22:25:02.895444+0000","last_active":"2023-07-13T22:25:03.892321+0000","last_peered":"2023-07-13T22:25:03.892321+0000","last_clean":"2023-07-13T22:25:03.892321+0000","last_became_active":"2023-07-13T22:25:02.895173+0000","last_became_peered":"2023-07-13T22:25:02.895173+0000","last_unstale":"2023-07-13T22:25:03.892321+0000","last_undegraded":"2023-07-13T22:25:03.892321+0000","last_fullsized":"2023-07-13T22:25:03.892321+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:47:31.063839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894600+0000","last_change":"2023-07-13T22:25:02.903143+0000","last_active":"2023-07-13T22:25:03.894600+0000","last_peered":"2023-07-13T22:25:03.894600+0000","last_clean":"2023-07-13T22:25:03.894600+0000","last_became_active":"2023-07-13T22:25:02.902447+0000","last_became_peered":"2023-07-13T22:25:02.902447+0000","last_unstale":"2023-07-13T22:25:03.894600+0000","last_undegraded":"2023-07-13T22:25:03.894600+0000","last_fullsized":"2023-07-13T22:25:03.894600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:52:46.539330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.660939+0000","last_change":"2023-07-13T22:25:02.889701+0000","last_active":"2023-07-13T22:25:04.660939+0000","last_peered":"2023-07-13T22:25:04.660939+0000","last_clean":"2023-07-13T22:25:04.660939+0000","last_became_active":"2023-07-13T22:25:02.889413+0000","last_became_peered":"2023-07-13T22:25:02.889413+0000","last_unstale":"2023-07-13T22:25:04.660939+0000","last_undegraded":"2023-07-13T22:25:04.660939+0000","last_fullsized":"2023-07-13T22:25:04.660939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:54:19.510027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571818+0000","last_change":"2023-07-13T22:25:02.891513+0000","last_active":"2023-07-13T22:25:04.571818+0000","last_peered":"2023-07-13T22:25:04.571818+0000","last_clean":"2023-07-13T22:25:04.571818+0000","last_became_active":"2023-07-13T22:25:02.891212+0000","last_became_peered":"2023-07-13T22:25:02.891212+0000","last_unstale":"2023-07-13T22:25:04.571818+0000","last_undegraded":"2023-07-13T22:25:04.571818+0000","last_fullsized":"2023-07-13T22:25:04.571818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:04:20.883177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661307+0000","last_change":"2023-07-13T22:25:02.897232+0000","last_active":"2023-07-13T22:25:04.661307+0000","last_peered":"2023-07-13T22:25:04.661307+0000","last_clean":"2023-07-13T22:25:04.661307+0000","last_became_active":"2023-07-13T22:25:02.896937+0000","last_became_peered":"2023-07-13T22:25:02.896937+0000","last_unstale":"2023-07-13T22:25:04.661307+0000","last_undegraded":"2023-07-13T22:25:04.661307+0000","last_fullsized":"2023-07-13T22:25:04.661307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:40:00.750773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016612+0000","last_change":"2023-07-13T22:25:02.904784+0000","last_active":"2023-07-13T22:25:04.016612+0000","last_peered":"2023-07-13T22:25:04.016612+0000","last_clean":"2023-07-13T22:25:04.016612+0000","last_became_active":"2023-07-13T22:25:02.904132+0000","last_became_peered":"2023-07-13T22:25:02.904132+0000","last_unstale":"2023-07-13T22:25:04.016612+0000","last_undegraded":"2023-07-13T22:25:04.016612+0000","last_fullsized":"2023-07-13T22:25:04.016612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:32.116765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661400+0000","last_change":"2023-07-13T22:25:02.900398+0000","last_active":"2023-07-13T22:25:04.661400+0000","last_peered":"2023-07-13T22:25:04.661400+0000","last_clean":"2023-07-13T22:25:04.661400+0000","last_became_active":"2023-07-13T22:25:02.899872+0000","last_became_peered":"2023-07-13T22:25:02.899872+0000","last_unstale":"2023-07-13T22:25:04.661400+0000","last_undegraded":"2023-07-13T22:25:04.661400+0000","last_fullsized":"2023-07-13T22:25:04.661400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:47:29.590272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661663+0000","last_change":"2023-07-13T22:25:02.899591+0000","last_active":"2023-07-13T22:25:04.661663+0000","last_peered":"2023-07-13T22:25:04.661663+0000","last_clean":"2023-07-13T22:25:04.661663+0000","last_became_active":"2023-07-13T22:25:02.896723+0000","last_became_peered":"2023-07-13T22:25:02.896723+0000","last_unstale":"2023-07-13T22:25:04.661663+0000","last_undegraded":"2023-07-13T22:25:04.661663+0000","last_fullsized":"2023-07-13T22:25:04.661663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:26:13.609262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892281+0000","last_change":"2023-07-13T22:25:02.895780+0000","last_active":"2023-07-13T22:25:03.892281+0000","last_peered":"2023-07-13T22:25:03.892281+0000","last_clean":"2023-07-13T22:25:03.892281+0000","last_became_active":"2023-07-13T22:25:02.895508+0000","last_became_peered":"2023-07-13T22:25:02.895508+0000","last_unstale":"2023-07-13T22:25:03.892281+0000","last_undegraded":"2023-07-13T22:25:03.892281+0000","last_fullsized":"2023-07-13T22:25:03.892281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:01:59.663749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016199+0000","last_change":"2023-07-13T22:25:02.904113+0000","last_active":"2023-07-13T22:25:04.016199+0000","last_peered":"2023-07-13T22:25:04.016199+0000","last_clean":"2023-07-13T22:25:04.016199+0000","last_became_active":"2023-07-13T22:25:02.903754+0000","last_became_peered":"2023-07-13T22:25:02.903754+0000","last_unstale":"2023-07-13T22:25:04.016199+0000","last_undegraded":"2023-07-13T22:25:04.016199+0000","last_fullsized":"2023-07-13T22:25:04.016199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:29:46.913646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892720+0000","last_change":"2023-07-13T22:25:02.889397+0000","last_active":"2023-07-13T22:25:03.892720+0000","last_peered":"2023-07-13T22:25:03.892720+0000","last_clean":"2023-07-13T22:25:03.892720+0000","last_became_active":"2023-07-13T22:25:02.889139+0000","last_became_peered":"2023-07-13T22:25:02.889139+0000","last_unstale":"2023-07-13T22:25:03.892720+0000","last_undegraded":"2023-07-13T22:25:03.892720+0000","last_fullsized":"2023-07-13T22:25:03.892720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:40:48.503052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661930+0000","last_change":"2023-07-13T22:25:02.895041+0000","last_active":"2023-07-13T22:25:04.661930+0000","last_peered":"2023-07-13T22:25:04.661930+0000","last_clean":"2023-07-13T22:25:04.661930+0000","last_became_active":"2023-07-13T22:25:02.894751+0000","last_became_peered":"2023-07-13T22:25:02.894751+0000","last_unstale":"2023-07-13T22:25:04.661930+0000","last_undegraded":"2023-07-13T22:25:04.661930+0000","last_fullsized":"2023-07-13T22:25:04.661930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:25:50.676534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661538+0000","last_change":"2023-07-13T22:25:02.884921+0000","last_active":"2023-07-13T22:25:04.661538+0000","last_peered":"2023-07-13T22:25:04.661538+0000","last_clean":"2023-07-13T22:25:04.661538+0000","last_became_active":"2023-07-13T22:25:02.884656+0000","last_became_peered":"2023-07-13T22:25:02.884656+0000","last_unstale":"2023-07-13T22:25:04.661538+0000","last_undegraded":"2023-07-13T22:25:04.661538+0000","last_fullsized":"2023-07-13T22:25:04.661538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:33:11.983924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891862+0000","last_change":"2023-07-13T22:25:02.902049+0000","last_active":"2023-07-13T22:25:03.891862+0000","last_peered":"2023-07-13T22:25:03.891862+0000","last_clean":"2023-07-13T22:25:03.891862+0000","last_became_active":"2023-07-13T22:25:02.901749+0000","last_became_peered":"2023-07-13T22:25:02.901749+0000","last_unstale":"2023-07-13T22:25:03.891862+0000","last_undegraded":"2023-07-13T22:25:03.891862+0000","last_fullsized":"2023-07-13T22:25:03.891862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:17:07.084128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.662040+0000","last_change":"2023-07-13T22:25:02.895975+0000","last_active":"2023-07-13T22:25:04.662040+0000","last_peered":"2023-07-13T22:25:04.662040+0000","last_clean":"2023-07-13T22:25:04.662040+0000","last_became_active":"2023-07-13T22:25:02.895355+0000","last_became_peered":"2023-07-13T22:25:02.895355+0000","last_unstale":"2023-07-13T22:25:04.662040+0000","last_undegraded":"2023-07-13T22:25:04.662040+0000","last_fullsized":"2023-07-13T22:25:04.662040+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:29:26.365541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571399+0000","last_change":"2023-07-13T22:25:02.880929+0000","last_active":"2023-07-13T22:25:04.571399+0000","last_peered":"2023-07-13T22:25:04.571399+0000","last_clean":"2023-07-13T22:25:04.571399+0000","last_became_active":"2023-07-13T22:25:02.880647+0000","last_became_peered":"2023-07-13T22:25:02.880647+0000","last_unstale":"2023-07-13T22:25:04.571399+0000","last_undegraded":"2023-07-13T22:25:04.571399+0000","last_fullsized":"2023-07-13T22:25:04.571399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:20:33.736576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661823+0000","last_change":"2023-07-13T22:25:02.899960+0000","last_active":"2023-07-13T22:25:04.661823+0000","last_peered":"2023-07-13T22:25:04.661823+0000","last_clean":"2023-07-13T22:25:04.661823+0000","last_became_active":"2023-07-13T22:25:02.899634+0000","last_became_peered":"2023-07-13T22:25:02.899634+0000","last_unstale":"2023-07-13T22:25:04.661823+0000","last_undegraded":"2023-07-13T22:25:04.661823+0000","last_fullsized":"2023-07-13T22:25:04.661823+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:51:40.469761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571252+0000","last_change":"2023-07-13T22:25:02.881432+0000","last_active":"2023-07-13T22:25:04.571252+0000","last_peered":"2023-07-13T22:25:04.571252+0000","last_clean":"2023-07-13T22:25:04.571252+0000","last_became_active":"2023-07-13T22:25:02.881134+0000","last_became_peered":"2023-07-13T22:25:02.881134+0000","last_unstale":"2023-07-13T22:25:04.571252+0000","last_undegraded":"2023-07-13T22:25:04.571252+0000","last_fullsized":"2023-07-13T22:25:04.571252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:43:02.034043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016705+0000","last_change":"2023-07-13T22:25:02.901388+0000","last_active":"2023-07-13T22:25:04.016705+0000","last_peered":"2023-07-13T22:25:04.016705+0000","last_clean":"2023-07-13T22:25:04.016705+0000","last_became_active":"2023-07-13T22:25:02.898081+0000","last_became_peered":"2023-07-13T22:25:02.898081+0000","last_unstale":"2023-07-13T22:25:04.016705+0000","last_undegraded":"2023-07-13T22:25:04.016705+0000","last_fullsized":"2023-07-13T22:25:04.016705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:11:40.375640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016740+0000","last_change":"2023-07-13T22:25:02.899295+0000","last_active":"2023-07-13T22:25:04.016740+0000","last_peered":"2023-07-13T22:25:04.016740+0000","last_clean":"2023-07-13T22:25:04.016740+0000","last_became_active":"2023-07-13T22:25:02.899012+0000","last_became_peered":"2023-07-13T22:25:02.899012+0000","last_unstale":"2023-07-13T22:25:04.016740+0000","last_undegraded":"2023-07-13T22:25:04.016740+0000","last_fullsized":"2023-07-13T22:25:04.016740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:04:20.883177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661408+0000","last_change":"2023-07-13T22:25:02.903993+0000","last_active":"2023-07-13T22:25:04.661408+0000","last_peered":"2023-07-13T22:25:04.661408+0000","last_clean":"2023-07-13T22:25:04.661408+0000","last_became_active":"2023-07-13T22:25:02.903699+0000","last_became_peered":"2023-07-13T22:25:02.903699+0000","last_unstale":"2023-07-13T22:25:04.661408+0000","last_undegraded":"2023-07-13T22:25:04.661408+0000","last_fullsized":"2023-07-13T22:25:04.661408+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:47:22.989608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892876+0000","last_change":"2023-07-13T22:25:02.889401+0000","last_active":"2023-07-13T22:25:03.892876+0000","last_peered":"2023-07-13T22:25:03.892876+0000","last_clean":"2023-07-13T22:25:03.892876+0000","last_became_active":"2023-07-13T22:25:02.889112+0000","last_became_peered":"2023-07-13T22:25:02.889112+0000","last_unstale":"2023-07-13T22:25:03.892876+0000","last_undegraded":"2023-07-13T22:25:03.892876+0000","last_fullsized":"2023-07-13T22:25:03.892876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:03:34.152998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661823+0000","last_change":"2023-07-13T22:25:02.896184+0000","last_active":"2023-07-13T22:25:04.661823+0000","last_peered":"2023-07-13T22:25:04.661823+0000","last_clean":"2023-07-13T22:25:04.661823+0000","last_became_active":"2023-07-13T22:25:02.895708+0000","last_became_peered":"2023-07-13T22:25:02.895708+0000","last_unstale":"2023-07-13T22:25:04.661823+0000","last_undegraded":"2023-07-13T22:25:04.661823+0000","last_fullsized":"2023-07-13T22:25:04.661823+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:04:21.759427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016506+0000","last_change":"2023-07-13T22:25:02.903294+0000","last_active":"2023-07-13T22:25:04.016506+0000","last_peered":"2023-07-13T22:25:04.016506+0000","last_clean":"2023-07-13T22:25:04.016506+0000","last_became_active":"2023-07-13T22:25:02.903014+0000","last_became_peered":"2023-07-13T22:25:02.903014+0000","last_unstale":"2023-07-13T22:25:04.016506+0000","last_undegraded":"2023-07-13T22:25:04.016506+0000","last_fullsized":"2023-07-13T22:25:04.016506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:03:22.729119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892963+0000","last_change":"2023-07-13T22:25:02.903557+0000","last_active":"2023-07-13T22:25:03.892963+0000","last_peered":"2023-07-13T22:25:03.892963+0000","last_clean":"2023-07-13T22:25:03.892963+0000","last_became_active":"2023-07-13T22:25:02.903173+0000","last_became_peered":"2023-07-13T22:25:02.903173+0000","last_unstale":"2023-07-13T22:25:03.892963+0000","last_undegraded":"2023-07-13T22:25:03.892963+0000","last_fullsized":"2023-07-13T22:25:03.892963+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:48:11.388663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661024+0000","last_change":"2023-07-13T22:25:02.897146+0000","last_active":"2023-07-13T22:25:04.661024+0000","last_peered":"2023-07-13T22:25:04.661024+0000","last_clean":"2023-07-13T22:25:04.661024+0000","last_became_active":"2023-07-13T22:25:02.896847+0000","last_became_peered":"2023-07-13T22:25:02.896847+0000","last_unstale":"2023-07-13T22:25:04.661024+0000","last_undegraded":"2023-07-13T22:25:04.661024+0000","last_fullsized":"2023-07-13T22:25:04.661024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:54:06.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893298+0000","last_change":"2023-07-13T22:25:02.893025+0000","last_active":"2023-07-13T22:25:03.893298+0000","last_peered":"2023-07-13T22:25:03.893298+0000","last_clean":"2023-07-13T22:25:03.893298+0000","last_became_active":"2023-07-13T22:25:02.892740+0000","last_became_peered":"2023-07-13T22:25:02.892740+0000","last_unstale":"2023-07-13T22:25:03.893298+0000","last_undegraded":"2023-07-13T22:25:03.893298+0000","last_fullsized":"2023-07-13T22:25:03.893298+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:06:54.924557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661026+0000","last_change":"2023-07-13T22:25:02.884197+0000","last_active":"2023-07-13T22:25:04.661026+0000","last_peered":"2023-07-13T22:25:04.661026+0000","last_clean":"2023-07-13T22:25:04.661026+0000","last_became_active":"2023-07-13T22:25:02.883922+0000","last_became_peered":"2023-07-13T22:25:02.883922+0000","last_unstale":"2023-07-13T22:25:04.661026+0000","last_undegraded":"2023-07-13T22:25:04.661026+0000","last_fullsized":"2023-07-13T22:25:04.661026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:17:55.114100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661573+0000","last_change":"2023-07-13T22:25:01.891747+0000","last_active":"2023-07-13T22:25:04.661573+0000","last_peered":"2023-07-13T22:25:04.661573+0000","last_clean":"2023-07-13T22:25:04.661573+0000","last_became_active":"2023-07-13T22:25:01.891483+0000","last_became_peered":"2023-07-13T22:25:01.891483+0000","last_unstale":"2023-07-13T22:25:04.661573+0000","last_undegraded":"2023-07-13T22:25:04.661573+0000","last_fullsized":"2023-07-13T22:25:04.661573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:13:38.340670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016316+0000","last_change":"2023-07-13T22:25:02.900528+0000","last_active":"2023-07-13T22:25:04.016316+0000","last_peered":"2023-07-13T22:25:04.016316+0000","last_clean":"2023-07-13T22:25:04.016316+0000","last_became_active":"2023-07-13T22:25:02.900212+0000","last_became_peered":"2023-07-13T22:25:02.900212+0000","last_unstale":"2023-07-13T22:25:04.016316+0000","last_undegraded":"2023-07-13T22:25:04.016316+0000","last_fullsized":"2023-07-13T22:25:04.016316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:31:15.060624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893124+0000","last_change":"2023-07-13T22:25:01.897643+0000","last_active":"2023-07-13T22:25:03.893124+0000","last_peered":"2023-07-13T22:25:03.893124+0000","last_clean":"2023-07-13T22:25:03.893124+0000","last_became_active":"2023-07-13T22:25:01.896751+0000","last_became_peered":"2023-07-13T22:25:01.896751+0000","last_unstale":"2023-07-13T22:25:03.893124+0000","last_undegraded":"2023-07-13T22:25:03.893124+0000","last_fullsized":"2023-07-13T22:25:03.893124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:46:44.734121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571368+0000","last_change":"2023-07-13T22:25:02.898265+0000","last_active":"2023-07-13T22:25:04.571368+0000","last_peered":"2023-07-13T22:25:04.571368+0000","last_clean":"2023-07-13T22:25:04.571368+0000","last_became_active":"2023-07-13T22:25:02.897431+0000","last_became_peered":"2023-07-13T22:25:02.897431+0000","last_unstale":"2023-07-13T22:25:04.571368+0000","last_undegraded":"2023-07-13T22:25:04.571368+0000","last_fullsized":"2023-07-13T22:25:04.571368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:03:22.729119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893148+0000","last_change":"2023-07-13T22:25:01.895148+0000","last_active":"2023-07-13T22:25:03.893148+0000","last_peered":"2023-07-13T22:25:03.893148+0000","last_clean":"2023-07-13T22:25:03.893148+0000","last_became_active":"2023-07-13T22:25:01.894871+0000","last_became_peered":"2023-07-13T22:25:01.894871+0000","last_unstale":"2023-07-13T22:25:03.893148+0000","last_undegraded":"2023-07-13T22:25:03.893148+0000","last_fullsized":"2023-07-13T22:25:03.893148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:58:26.743673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571530+0000","last_change":"2023-07-13T22:25:02.880732+0000","last_active":"2023-07-13T22:25:04.571530+0000","last_peered":"2023-07-13T22:25:04.571530+0000","last_clean":"2023-07-13T22:25:04.571530+0000","last_became_active":"2023-07-13T22:25:02.880166+0000","last_became_peered":"2023-07-13T22:25:02.880166+0000","last_unstale":"2023-07-13T22:25:04.571530+0000","last_undegraded":"2023-07-13T22:25:04.571530+0000","last_fullsized":"2023-07-13T22:25:04.571530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:42:55.921674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894293+0000","last_change":"2023-07-13T22:25:01.896257+0000","last_active":"2023-07-13T22:25:03.894293+0000","last_peered":"2023-07-13T22:25:03.894293+0000","last_clean":"2023-07-13T22:25:03.894293+0000","last_became_active":"2023-07-13T22:25:01.895995+0000","last_became_peered":"2023-07-13T22:25:01.895995+0000","last_unstale":"2023-07-13T22:25:03.894293+0000","last_undegraded":"2023-07-13T22:25:03.894293+0000","last_fullsized":"2023-07-13T22:25:03.894293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:19:49.849385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016584+0000","last_change":"2023-07-13T22:25:02.910492+0000","last_active":"2023-07-13T22:25:04.016584+0000","last_peered":"2023-07-13T22:25:04.016584+0000","last_clean":"2023-07-13T22:25:04.016584+0000","last_became_active":"2023-07-13T22:25:02.909942+0000","last_became_peered":"2023-07-13T22:25:02.909942+0000","last_unstale":"2023-07-13T22:25:04.016584+0000","last_undegraded":"2023-07-13T22:25:04.016584+0000","last_fullsized":"2023-07-13T22:25:04.016584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:42:55.921674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571514+0000","last_change":"2023-07-13T22:25:01.892113+0000","last_active":"2023-07-13T22:25:04.571514+0000","last_peered":"2023-07-13T22:25:04.571514+0000","last_clean":"2023-07-13T22:25:04.571514+0000","last_became_active":"2023-07-13T22:25:01.891830+0000","last_became_peered":"2023-07-13T22:25:01.891830+0000","last_unstale":"2023-07-13T22:25:04.571514+0000","last_undegraded":"2023-07-13T22:25:04.571514+0000","last_fullsized":"2023-07-13T22:25:04.571514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:31.110289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016439+0000","last_change":"2023-07-13T22:25:02.904938+0000","last_active":"2023-07-13T22:25:04.016439+0000","last_peered":"2023-07-13T22:25:04.016439+0000","last_clean":"2023-07-13T22:25:04.016439+0000","last_became_active":"2023-07-13T22:25:02.904549+0000","last_became_peered":"2023-07-13T22:25:02.904549+0000","last_unstale":"2023-07-13T22:25:04.016439+0000","last_undegraded":"2023-07-13T22:25:04.016439+0000","last_fullsized":"2023-07-13T22:25:04.016439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:43:02.034043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661129+0000","last_change":"2023-07-13T22:25:01.893789+0000","last_active":"2023-07-13T22:25:04.661129+0000","last_peered":"2023-07-13T22:25:04.661129+0000","last_clean":"2023-07-13T22:25:04.661129+0000","last_became_active":"2023-07-13T22:25:01.893145+0000","last_became_peered":"2023-07-13T22:25:01.893145+0000","last_unstale":"2023-07-13T22:25:04.661129+0000","last_undegraded":"2023-07-13T22:25:04.661129+0000","last_fullsized":"2023-07-13T22:25:04.661129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:03:41.445998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571803+0000","last_change":"2023-07-13T22:25:02.891454+0000","last_active":"2023-07-13T22:25:04.571803+0000","last_peered":"2023-07-13T22:25:04.571803+0000","last_clean":"2023-07-13T22:25:04.571803+0000","last_became_active":"2023-07-13T22:25:02.891167+0000","last_became_peered":"2023-07-13T22:25:02.891167+0000","last_unstale":"2023-07-13T22:25:04.571803+0000","last_undegraded":"2023-07-13T22:25:04.571803+0000","last_fullsized":"2023-07-13T22:25:04.571803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:11:40.375640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016107+0000","last_change":"2023-07-13T22:25:01.894353+0000","last_active":"2023-07-13T22:25:04.016107+0000","last_peered":"2023-07-13T22:25:04.016107+0000","last_clean":"2023-07-13T22:25:04.016107+0000","last_became_active":"2023-07-13T22:25:01.894012+0000","last_became_peered":"2023-07-13T22:25:01.894012+0000","last_unstale":"2023-07-13T22:25:04.016107+0000","last_undegraded":"2023-07-13T22:25:04.016107+0000","last_fullsized":"2023-07-13T22:25:04.016107+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:54:31.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894840+0000","last_change":"2023-07-13T22:25:02.909278+0000","last_active":"2023-07-13T22:25:03.894840+0000","last_peered":"2023-07-13T22:25:03.894840+0000","last_clean":"2023-07-13T22:25:03.894840+0000","last_became_active":"2023-07-13T22:25:02.908914+0000","last_became_peered":"2023-07-13T22:25:02.908914+0000","last_unstale":"2023-07-13T22:25:03.894840+0000","last_undegraded":"2023-07-13T22:25:03.894840+0000","last_fullsized":"2023-07-13T22:25:03.894840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:47:53.889830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661165+0000","last_change":"2023-07-13T22:25:01.887959+0000","last_active":"2023-07-13T22:25:04.661165+0000","last_peered":"2023-07-13T22:25:04.661165+0000","last_clean":"2023-07-13T22:25:04.661165+0000","last_became_active":"2023-07-13T22:25:01.887696+0000","last_became_peered":"2023-07-13T22:25:01.887696+0000","last_unstale":"2023-07-13T22:25:04.661165+0000","last_undegraded":"2023-07-13T22:25:04.661165+0000","last_fullsized":"2023-07-13T22:25:04.661165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:49:34.461411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016137+0000","last_change":"2023-07-13T22:25:02.901715+0000","last_active":"2023-07-13T22:25:04.016137+0000","last_peered":"2023-07-13T22:25:04.016137+0000","last_clean":"2023-07-13T22:25:04.016137+0000","last_became_active":"2023-07-13T22:25:02.898738+0000","last_became_peered":"2023-07-13T22:25:02.898738+0000","last_unstale":"2023-07-13T22:25:04.016137+0000","last_undegraded":"2023-07-13T22:25:04.016137+0000","last_fullsized":"2023-07-13T22:25:04.016137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:45:14.519486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893399+0000","last_change":"2023-07-13T22:25:01.897430+0000","last_active":"2023-07-13T22:25:03.893399+0000","last_peered":"2023-07-13T22:25:03.893399+0000","last_clean":"2023-07-13T22:25:03.893399+0000","last_became_active":"2023-07-13T22:25:01.897100+0000","last_became_peered":"2023-07-13T22:25:01.897100+0000","last_unstale":"2023-07-13T22:25:03.893399+0000","last_undegraded":"2023-07-13T22:25:03.893399+0000","last_fullsized":"2023-07-13T22:25:03.893399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:42:45.904497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661302+0000","last_change":"2023-07-13T22:25:02.881783+0000","last_active":"2023-07-13T22:25:04.661302+0000","last_peered":"2023-07-13T22:25:04.661302+0000","last_clean":"2023-07-13T22:25:04.661302+0000","last_became_active":"2023-07-13T22:25:02.881486+0000","last_became_peered":"2023-07-13T22:25:02.881486+0000","last_unstale":"2023-07-13T22:25:04.661302+0000","last_undegraded":"2023-07-13T22:25:04.661302+0000","last_fullsized":"2023-07-13T22:25:04.661302+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:17:08.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661442+0000","last_change":"2023-07-13T22:25:01.892695+0000","last_active":"2023-07-13T22:25:04.661442+0000","last_peered":"2023-07-13T22:25:04.661442+0000","last_clean":"2023-07-13T22:25:04.661442+0000","last_became_active":"2023-07-13T22:25:01.892443+0000","last_became_peered":"2023-07-13T22:25:01.892443+0000","last_unstale":"2023-07-13T22:25:04.661442+0000","last_undegraded":"2023-07-13T22:25:04.661442+0000","last_fullsized":"2023-07-13T22:25:04.661442+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:07:41.239307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892180+0000","last_change":"2023-07-13T22:25:02.892080+0000","last_active":"2023-07-13T22:25:03.892180+0000","last_peered":"2023-07-13T22:25:03.892180+0000","last_clean":"2023-07-13T22:25:03.892180+0000","last_became_active":"2023-07-13T22:25:02.891504+0000","last_became_peered":"2023-07-13T22:25:02.891504+0000","last_unstale":"2023-07-13T22:25:03.892180+0000","last_undegraded":"2023-07-13T22:25:03.892180+0000","last_fullsized":"2023-07-13T22:25:03.892180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:24:42.048927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661925+0000","last_change":"2023-07-13T22:25:01.885599+0000","last_active":"2023-07-13T22:25:04.661925+0000","last_peered":"2023-07-13T22:25:04.661925+0000","last_clean":"2023-07-13T22:25:04.661925+0000","last_became_active":"2023-07-13T22:25:01.885199+0000","last_became_peered":"2023-07-13T22:25:01.885199+0000","last_unstale":"2023-07-13T22:25:04.661925+0000","last_undegraded":"2023-07-13T22:25:04.661925+0000","last_fullsized":"2023-07-13T22:25:04.661925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:34:54.913732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661425+0000","last_change":"2023-07-13T22:25:02.904741+0000","last_active":"2023-07-13T22:25:04.661425+0000","last_peered":"2023-07-13T22:25:04.661425+0000","last_clean":"2023-07-13T22:25:04.661425+0000","last_became_active":"2023-07-13T22:25:02.904477+0000","last_became_peered":"2023-07-13T22:25:02.904477+0000","last_unstale":"2023-07-13T22:25:04.661425+0000","last_undegraded":"2023-07-13T22:25:04.661425+0000","last_fullsized":"2023-07-13T22:25:04.661425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:38:22.743598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894453+0000","last_change":"2023-07-13T22:25:01.891393+0000","last_active":"2023-07-13T22:25:03.894453+0000","last_peered":"2023-07-13T22:25:03.894453+0000","last_clean":"2023-07-13T22:25:03.894453+0000","last_became_active":"2023-07-13T22:25:01.891110+0000","last_became_peered":"2023-07-13T22:25:01.891110+0000","last_unstale":"2023-07-13T22:25:03.894453+0000","last_undegraded":"2023-07-13T22:25:03.894453+0000","last_fullsized":"2023-07-13T22:25:03.894453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:13:55.614160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661468+0000","last_change":"2023-07-13T22:25:02.886611+0000","last_active":"2023-07-13T22:25:04.661468+0000","last_peered":"2023-07-13T22:25:04.661468+0000","last_clean":"2023-07-13T22:25:04.661468+0000","last_became_active":"2023-07-13T22:25:02.886347+0000","last_became_peered":"2023-07-13T22:25:02.886347+0000","last_unstale":"2023-07-13T22:25:04.661468+0000","last_undegraded":"2023-07-13T22:25:04.661468+0000","last_fullsized":"2023-07-13T22:25:04.661468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:13:05.407426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661129+0000","last_change":"2023-07-13T22:25:01.891579+0000","last_active":"2023-07-13T22:25:04.661129+0000","last_peered":"2023-07-13T22:25:04.661129+0000","last_clean":"2023-07-13T22:25:04.661129+0000","last_became_active":"2023-07-13T22:25:01.891284+0000","last_became_peered":"2023-07-13T22:25:01.891284+0000","last_unstale":"2023-07-13T22:25:04.661129+0000","last_undegraded":"2023-07-13T22:25:04.661129+0000","last_fullsized":"2023-07-13T22:25:04.661129+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:35:04.767631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894235+0000","last_change":"2023-07-13T22:25:02.898624+0000","last_active":"2023-07-13T22:25:03.894235+0000","last_peered":"2023-07-13T22:25:03.894235+0000","last_clean":"2023-07-13T22:25:03.894235+0000","last_became_active":"2023-07-13T22:25:02.898357+0000","last_became_peered":"2023-07-13T22:25:02.898357+0000","last_unstale":"2023-07-13T22:25:03.894235+0000","last_undegraded":"2023-07-13T22:25:03.894235+0000","last_fullsized":"2023-07-13T22:25:03.894235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:17:55.114100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892794+0000","last_change":"2023-07-13T22:25:01.889495+0000","last_active":"2023-07-13T22:25:03.892794+0000","last_peered":"2023-07-13T22:25:03.892794+0000","last_clean":"2023-07-13T22:25:03.892794+0000","last_became_active":"2023-07-13T22:25:01.889121+0000","last_became_peered":"2023-07-13T22:25:01.889121+0000","last_unstale":"2023-07-13T22:25:03.892794+0000","last_undegraded":"2023-07-13T22:25:03.892794+0000","last_fullsized":"2023-07-13T22:25:03.892794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:36:53.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894650+0000","last_change":"2023-07-13T22:25:02.894652+0000","last_active":"2023-07-13T22:25:03.894650+0000","last_peered":"2023-07-13T22:25:03.894650+0000","last_clean":"2023-07-13T22:25:03.894650+0000","last_became_active":"2023-07-13T22:25:02.894360+0000","last_became_peered":"2023-07-13T22:25:02.894360+0000","last_unstale":"2023-07-13T22:25:03.894650+0000","last_undegraded":"2023-07-13T22:25:03.894650+0000","last_fullsized":"2023-07-13T22:25:03.894650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:54:36.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661334+0000","last_change":"2023-07-13T22:25:01.887450+0000","last_active":"2023-07-13T22:25:04.661334+0000","last_peered":"2023-07-13T22:25:04.661334+0000","last_clean":"2023-07-13T22:25:04.661334+0000","last_became_active":"2023-07-13T22:25:01.887182+0000","last_became_peered":"2023-07-13T22:25:01.887182+0000","last_unstale":"2023-07-13T22:25:04.661334+0000","last_undegraded":"2023-07-13T22:25:04.661334+0000","last_fullsized":"2023-07-13T22:25:04.661334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:40:30.826920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894940+0000","last_change":"2023-07-13T22:25:02.904910+0000","last_active":"2023-07-13T22:25:03.894940+0000","last_peered":"2023-07-13T22:25:03.894940+0000","last_clean":"2023-07-13T22:25:03.894940+0000","last_became_active":"2023-07-13T22:25:02.904593+0000","last_became_peered":"2023-07-13T22:25:02.904593+0000","last_unstale":"2023-07-13T22:25:03.894940+0000","last_undegraded":"2023-07-13T22:25:03.894940+0000","last_fullsized":"2023-07-13T22:25:03.894940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:08:49.001753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016065+0000","last_change":"2023-07-13T22:25:01.896800+0000","last_active":"2023-07-13T22:25:04.016065+0000","last_peered":"2023-07-13T22:25:04.016065+0000","last_clean":"2023-07-13T22:25:04.016065+0000","last_became_active":"2023-07-13T22:25:01.896421+0000","last_became_peered":"2023-07-13T22:25:01.896421+0000","last_unstale":"2023-07-13T22:25:04.016065+0000","last_undegraded":"2023-07-13T22:25:04.016065+0000","last_fullsized":"2023-07-13T22:25:04.016065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:52:28.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571290+0000","last_change":"2023-07-13T22:25:02.895020+0000","last_active":"2023-07-13T22:25:04.571290+0000","last_peered":"2023-07-13T22:25:04.571290+0000","last_clean":"2023-07-13T22:25:04.571290+0000","last_became_active":"2023-07-13T22:25:02.894733+0000","last_became_peered":"2023-07-13T22:25:02.894733+0000","last_unstale":"2023-07-13T22:25:04.571290+0000","last_undegraded":"2023-07-13T22:25:04.571290+0000","last_fullsized":"2023-07-13T22:25:04.571290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:07:50.443338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661338+0000","last_change":"2023-07-13T22:25:01.886175+0000","last_active":"2023-07-13T22:25:04.661338+0000","last_peered":"2023-07-13T22:25:04.661338+0000","last_clean":"2023-07-13T22:25:04.661338+0000","last_became_active":"2023-07-13T22:25:01.885875+0000","last_became_peered":"2023-07-13T22:25:01.885875+0000","last_unstale":"2023-07-13T22:25:04.661338+0000","last_undegraded":"2023-07-13T22:25:04.661338+0000","last_fullsized":"2023-07-13T22:25:04.661338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:45:36.294115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894132+0000","last_change":"2023-07-13T22:25:02.904352+0000","last_active":"2023-07-13T22:25:03.894132+0000","last_peered":"2023-07-13T22:25:03.894132+0000","last_clean":"2023-07-13T22:25:03.894132+0000","last_became_active":"2023-07-13T22:25:02.904054+0000","last_became_peered":"2023-07-13T22:25:02.904054+0000","last_unstale":"2023-07-13T22:25:03.894132+0000","last_undegraded":"2023-07-13T22:25:03.894132+0000","last_fullsized":"2023-07-13T22:25:03.894132+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:08.913386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016730+0000","last_change":"2023-07-13T22:25:01.894640+0000","last_active":"2023-07-13T22:25:04.016730+0000","last_peered":"2023-07-13T22:25:04.016730+0000","last_clean":"2023-07-13T22:25:04.016730+0000","last_became_active":"2023-07-13T22:25:01.894228+0000","last_became_peered":"2023-07-13T22:25:01.894228+0000","last_unstale":"2023-07-13T22:25:04.016730+0000","last_undegraded":"2023-07-13T22:25:04.016730+0000","last_fullsized":"2023-07-13T22:25:04.016730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:06:57.166270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571407+0000","last_change":"2023-07-13T22:25:02.890599+0000","last_active":"2023-07-13T22:25:04.571407+0000","last_peered":"2023-07-13T22:25:04.571407+0000","last_clean":"2023-07-13T22:25:04.571407+0000","last_became_active":"2023-07-13T22:25:02.890318+0000","last_became_peered":"2023-07-13T22:25:02.890318+0000","last_unstale":"2023-07-13T22:25:04.571407+0000","last_undegraded":"2023-07-13T22:25:04.571407+0000","last_fullsized":"2023-07-13T22:25:04.571407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:21.200021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893162+0000","last_change":"2023-07-13T22:25:01.890328+0000","last_active":"2023-07-13T22:25:03.893162+0000","last_peered":"2023-07-13T22:25:03.893162+0000","last_clean":"2023-07-13T22:25:03.893162+0000","last_became_active":"2023-07-13T22:25:01.890053+0000","last_became_peered":"2023-07-13T22:25:01.890053+0000","last_unstale":"2023-07-13T22:25:03.893162+0000","last_undegraded":"2023-07-13T22:25:03.893162+0000","last_fullsized":"2023-07-13T22:25:03.893162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:58:44.813230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016532+0000","last_change":"2023-07-13T22:25:02.898303+0000","last_active":"2023-07-13T22:25:04.016532+0000","last_peered":"2023-07-13T22:25:04.016532+0000","last_clean":"2023-07-13T22:25:04.016532+0000","last_became_active":"2023-07-13T22:25:02.898039+0000","last_became_peered":"2023-07-13T22:25:02.898039+0000","last_unstale":"2023-07-13T22:25:04.016532+0000","last_undegraded":"2023-07-13T22:25:04.016532+0000","last_fullsized":"2023-07-13T22:25:04.016532+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:20:33.736576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892968+0000","last_change":"2023-07-13T22:25:01.897511+0000","last_active":"2023-07-13T22:25:03.892968+0000","last_peered":"2023-07-13T22:25:03.892968+0000","last_clean":"2023-07-13T22:25:03.892968+0000","last_became_active":"2023-07-13T22:25:01.897181+0000","last_became_peered":"2023-07-13T22:25:01.897181+0000","last_unstale":"2023-07-13T22:25:03.892968+0000","last_undegraded":"2023-07-13T22:25:03.892968+0000","last_fullsized":"2023-07-13T22:25:03.892968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:46.290911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894534+0000","last_change":"2023-07-13T22:25:02.896731+0000","last_active":"2023-07-13T22:25:03.894534+0000","last_peered":"2023-07-13T22:25:03.894534+0000","last_clean":"2023-07-13T22:25:03.894534+0000","last_became_active":"2023-07-13T22:25:02.896138+0000","last_became_peered":"2023-07-13T22:25:02.896138+0000","last_unstale":"2023-07-13T22:25:03.894534+0000","last_undegraded":"2023-07-13T22:25:03.894534+0000","last_fullsized":"2023-07-13T22:25:03.894534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:54:06.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894331+0000","last_change":"2023-07-13T22:25:01.896045+0000","last_active":"2023-07-13T22:25:03.894331+0000","last_peered":"2023-07-13T22:25:03.894331+0000","last_clean":"2023-07-13T22:25:03.894331+0000","last_became_active":"2023-07-13T22:25:01.895768+0000","last_became_peered":"2023-07-13T22:25:01.895768+0000","last_unstale":"2023-07-13T22:25:03.894331+0000","last_undegraded":"2023-07-13T22:25:03.894331+0000","last_fullsized":"2023-07-13T22:25:03.894331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:20:24.658129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894594+0000","last_change":"2023-07-13T22:25:02.903163+0000","last_active":"2023-07-13T22:25:03.894594+0000","last_peered":"2023-07-13T22:25:03.894594+0000","last_clean":"2023-07-13T22:25:03.894594+0000","last_became_active":"2023-07-13T22:25:02.902447+0000","last_became_peered":"2023-07-13T22:25:02.902447+0000","last_unstale":"2023-07-13T22:25:03.894594+0000","last_undegraded":"2023-07-13T22:25:03.894594+0000","last_fullsized":"2023-07-13T22:25:03.894594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:33:11.983924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661926+0000","last_change":"2023-07-13T22:25:01.886901+0000","last_active":"2023-07-13T22:25:04.661926+0000","last_peered":"2023-07-13T22:25:04.661926+0000","last_clean":"2023-07-13T22:25:04.661926+0000","last_became_active":"2023-07-13T22:25:01.886620+0000","last_became_peered":"2023-07-13T22:25:01.886620+0000","last_unstale":"2023-07-13T22:25:04.661926+0000","last_undegraded":"2023-07-13T22:25:04.661926+0000","last_fullsized":"2023-07-13T22:25:04.661926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:17:25.586012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661732+0000","last_change":"2023-07-13T22:25:02.904314+0000","last_active":"2023-07-13T22:25:04.661732+0000","last_peered":"2023-07-13T22:25:04.661732+0000","last_clean":"2023-07-13T22:25:04.661732+0000","last_became_active":"2023-07-13T22:25:02.904038+0000","last_became_peered":"2023-07-13T22:25:02.904038+0000","last_unstale":"2023-07-13T22:25:04.661732+0000","last_undegraded":"2023-07-13T22:25:04.661732+0000","last_fullsized":"2023-07-13T22:25:04.661732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:54:36.464401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894426+0000","last_change":"2023-07-13T22:25:01.895277+0000","last_active":"2023-07-13T22:25:03.894426+0000","last_peered":"2023-07-13T22:25:03.894426+0000","last_clean":"2023-07-13T22:25:03.894426+0000","last_became_active":"2023-07-13T22:25:01.895006+0000","last_became_peered":"2023-07-13T22:25:01.895006+0000","last_unstale":"2023-07-13T22:25:03.894426+0000","last_undegraded":"2023-07-13T22:25:03.894426+0000","last_fullsized":"2023-07-13T22:25:03.894426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:38:21.737122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661504+0000","last_change":"2023-07-13T22:25:02.888475+0000","last_active":"2023-07-13T22:25:04.661504+0000","last_peered":"2023-07-13T22:25:04.661504+0000","last_clean":"2023-07-13T22:25:04.661504+0000","last_became_active":"2023-07-13T22:25:02.888077+0000","last_became_peered":"2023-07-13T22:25:02.888077+0000","last_unstale":"2023-07-13T22:25:04.661504+0000","last_undegraded":"2023-07-13T22:25:04.661504+0000","last_fullsized":"2023-07-13T22:25:04.661504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:13:56.620635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.660878+0000","last_change":"2023-07-13T22:25:01.885772+0000","last_active":"2023-07-13T22:25:04.660878+0000","last_peered":"2023-07-13T22:25:04.660878+0000","last_clean":"2023-07-13T22:25:04.660878+0000","last_became_active":"2023-07-13T22:25:01.885461+0000","last_became_peered":"2023-07-13T22:25:01.885461+0000","last_unstale":"2023-07-13T22:25:04.660878+0000","last_undegraded":"2023-07-13T22:25:04.660878+0000","last_fullsized":"2023-07-13T22:25:04.660878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:07.906911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891685+0000","last_change":"2023-07-13T22:25:02.911564+0000","last_active":"2023-07-13T22:25:03.891685+0000","last_peered":"2023-07-13T22:25:03.891685+0000","last_clean":"2023-07-13T22:25:03.891685+0000","last_became_active":"2023-07-13T22:25:02.911264+0000","last_became_peered":"2023-07-13T22:25:02.911264+0000","last_unstale":"2023-07-13T22:25:03.891685+0000","last_undegraded":"2023-07-13T22:25:03.891685+0000","last_fullsized":"2023-07-13T22:25:03.891685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:07:15.256799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893353+0000","last_change":"2023-07-13T22:25:01.891948+0000","last_active":"2023-07-13T22:25:03.893353+0000","last_peered":"2023-07-13T22:25:03.893353+0000","last_clean":"2023-07-13T22:25:03.893353+0000","last_became_active":"2023-07-13T22:25:01.891595+0000","last_became_peered":"2023-07-13T22:25:01.891595+0000","last_unstale":"2023-07-13T22:25:03.893353+0000","last_undegraded":"2023-07-13T22:25:03.893353+0000","last_fullsized":"2023-07-13T22:25:03.893353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:45:57.113798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571564+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:25:04.571564+0000","last_peered":"2023-07-13T22:25:04.571564+0000","last_clean":"2023-07-13T22:25:04.571564+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:25:04.571564+0000","last_undegraded":"2023-07-13T22:25:04.571564+0000","last_fullsized":"2023-07-13T22:25:04.571564+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:53:12.722752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661540+0000","last_change":"2023-07-13T22:25:02.894621+0000","last_active":"2023-07-13T22:25:04.661540+0000","last_peered":"2023-07-13T22:25:04.661540+0000","last_clean":"2023-07-13T22:25:04.661540+0000","last_became_active":"2023-07-13T22:25:02.894328+0000","last_became_peered":"2023-07-13T22:25:02.894328+0000","last_unstale":"2023-07-13T22:25:04.661540+0000","last_undegraded":"2023-07-13T22:25:04.661540+0000","last_fullsized":"2023-07-13T22:25:04.661540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:21:11.546254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.571840+0000","last_change":"2023-07-13T22:25:01.885197+0000","last_active":"2023-07-13T22:25:04.571840+0000","last_peered":"2023-07-13T22:25:04.571840+0000","last_clean":"2023-07-13T22:25:04.571840+0000","last_became_active":"2023-07-13T22:25:01.884915+0000","last_became_peered":"2023-07-13T22:25:01.884915+0000","last_unstale":"2023-07-13T22:25:04.571840+0000","last_undegraded":"2023-07-13T22:25:04.571840+0000","last_fullsized":"2023-07-13T22:25:04.571840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:06:57.166270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891962+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:25:03.891962+0000","last_peered":"2023-07-13T22:25:03.891962+0000","last_clean":"2023-07-13T22:25:03.891962+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:25:03.891962+0000","last_undegraded":"2023-07-13T22:25:03.891962+0000","last_fullsized":"2023-07-13T22:25:03.891962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:58:03.089682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016518+0000","last_change":"2023-07-13T22:25:02.902508+0000","last_active":"2023-07-13T22:25:04.016518+0000","last_peered":"2023-07-13T22:25:04.016518+0000","last_clean":"2023-07-13T22:25:04.016518+0000","last_became_active":"2023-07-13T22:25:02.902144+0000","last_became_peered":"2023-07-13T22:25:02.902144+0000","last_unstale":"2023-07-13T22:25:04.016518+0000","last_undegraded":"2023-07-13T22:25:04.016518+0000","last_fullsized":"2023-07-13T22:25:04.016518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:21.200021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891743+0000","last_change":"2023-07-13T22:25:01.893555+0000","last_active":"2023-07-13T22:25:03.891743+0000","last_peered":"2023-07-13T22:25:03.891743+0000","last_clean":"2023-07-13T22:25:03.891743+0000","last_became_active":"2023-07-13T22:25:01.893227+0000","last_became_peered":"2023-07-13T22:25:01.893227+0000","last_unstale":"2023-07-13T22:25:03.891743+0000","last_undegraded":"2023-07-13T22:25:03.891743+0000","last_fullsized":"2023-07-13T22:25:03.891743+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:37:08.777840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661231+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:25:04.661231+0000","last_peered":"2023-07-13T22:25:04.661231+0000","last_clean":"2023-07-13T22:25:04.661231+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:25:04.661231+0000","last_undegraded":"2023-07-13T22:25:04.661231+0000","last_fullsized":"2023-07-13T22:25:04.661231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:16:10.506743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893243+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:25:03.893243+0000","last_peered":"2023-07-13T22:25:03.893243+0000","last_clean":"2023-07-13T22:25:03.893243+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:25:03.893243+0000","last_undegraded":"2023-07-13T22:25:03.893243+0000","last_fullsized":"2023-07-13T22:25:03.893243+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:35:17.610266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894900+0000","last_change":"2023-07-13T22:25:02.895995+0000","last_active":"2023-07-13T22:25:03.894900+0000","last_peered":"2023-07-13T22:25:03.894900+0000","last_clean":"2023-07-13T22:25:03.894900+0000","last_became_active":"2023-07-13T22:25:02.895724+0000","last_became_peered":"2023-07-13T22:25:02.895724+0000","last_unstale":"2023-07-13T22:25:03.894900+0000","last_undegraded":"2023-07-13T22:25:03.894900+0000","last_fullsized":"2023-07-13T22:25:03.894900+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:35:33.390458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891785+0000","last_change":"2023-07-13T22:25:01.884525+0000","last_active":"2023-07-13T22:25:03.891785+0000","last_peered":"2023-07-13T22:25:03.891785+0000","last_clean":"2023-07-13T22:25:03.891785+0000","last_became_active":"2023-07-13T22:25:01.884186+0000","last_became_peered":"2023-07-13T22:25:01.884186+0000","last_unstale":"2023-07-13T22:25:03.891785+0000","last_undegraded":"2023-07-13T22:25:03.891785+0000","last_fullsized":"2023-07-13T22:25:03.891785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:12:18.097405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893075+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:25:03.893075+0000","last_peered":"2023-07-13T22:25:03.893075+0000","last_clean":"2023-07-13T22:25:03.893075+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:25:03.893075+0000","last_undegraded":"2023-07-13T22:25:03.893075+0000","last_fullsized":"2023-07-13T22:25:03.893075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:23:01.801448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893029+0000","last_change":"2023-07-13T22:25:02.896598+0000","last_active":"2023-07-13T22:25:03.893029+0000","last_peered":"2023-07-13T22:25:03.893029+0000","last_clean":"2023-07-13T22:25:03.893029+0000","last_became_active":"2023-07-13T22:25:02.896299+0000","last_became_peered":"2023-07-13T22:25:02.896299+0000","last_unstale":"2023-07-13T22:25:03.893029+0000","last_undegraded":"2023-07-13T22:25:03.893029+0000","last_fullsized":"2023-07-13T22:25:03.893029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:31:31.326824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.892077+0000","last_change":"2023-07-13T22:25:01.892891+0000","last_active":"2023-07-13T22:25:03.892077+0000","last_peered":"2023-07-13T22:25:03.892077+0000","last_clean":"2023-07-13T22:25:03.892077+0000","last_became_active":"2023-07-13T22:25:01.892598+0000","last_became_peered":"2023-07-13T22:25:01.892598+0000","last_unstale":"2023-07-13T22:25:03.892077+0000","last_undegraded":"2023-07-13T22:25:03.892077+0000","last_fullsized":"2023-07-13T22:25:03.892077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:43:54.995576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.893423+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:25:03.893423+0000","last_peered":"2023-07-13T22:25:03.893423+0000","last_clean":"2023-07-13T22:25:03.893423+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:25:03.893423+0000","last_undegraded":"2023-07-13T22:25:03.893423+0000","last_fullsized":"2023-07-13T22:25:03.893423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:48:04.740879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.016628+0000","last_change":"2023-07-13T22:25:02.899939+0000","last_active":"2023-07-13T22:25:04.016628+0000","last_peered":"2023-07-13T22:25:04.016628+0000","last_clean":"2023-07-13T22:25:04.016628+0000","last_became_active":"2023-07-13T22:25:02.899512+0000","last_became_peered":"2023-07-13T22:25:02.899512+0000","last_unstale":"2023-07-13T22:25:04.016628+0000","last_undegraded":"2023-07-13T22:25:04.016628+0000","last_fullsized":"2023-07-13T22:25:04.016628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:57:27.880613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891952+0000","last_change":"2023-07-13T22:25:01.888706+0000","last_active":"2023-07-13T22:25:03.891952+0000","last_peered":"2023-07-13T22:25:03.891952+0000","last_clean":"2023-07-13T22:25:03.891952+0000","last_became_active":"2023-07-13T22:25:01.888403+0000","last_became_peered":"2023-07-13T22:25:01.888403+0000","last_unstale":"2023-07-13T22:25:03.891952+0000","last_undegraded":"2023-07-13T22:25:03.891952+0000","last_fullsized":"2023-07-13T22:25:03.891952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:39:32.541068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.660898+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:25:04.660898+0000","last_peered":"2023-07-13T22:25:04.660898+0000","last_clean":"2023-07-13T22:25:04.660898+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:25:04.660898+0000","last_undegraded":"2023-07-13T22:25:04.660898+0000","last_fullsized":"2023-07-13T22:25:04.660898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:01:16.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.891958+0000","last_change":"2023-07-13T22:25:02.902133+0000","last_active":"2023-07-13T22:25:03.891958+0000","last_peered":"2023-07-13T22:25:03.891958+0000","last_clean":"2023-07-13T22:25:03.891958+0000","last_became_active":"2023-07-13T22:25:02.901319+0000","last_became_peered":"2023-07-13T22:25:02.901319+0000","last_unstale":"2023-07-13T22:25:03.891958+0000","last_undegraded":"2023-07-13T22:25:03.891958+0000","last_fullsized":"2023-07-13T22:25:03.891958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:12:46.956883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661239+0000","last_change":"2023-07-13T22:25:01.886358+0000","last_active":"2023-07-13T22:25:04.661239+0000","last_peered":"2023-07-13T22:25:04.661239+0000","last_clean":"2023-07-13T22:25:04.661239+0000","last_became_active":"2023-07-13T22:25:01.885833+0000","last_became_peered":"2023-07-13T22:25:01.885833+0000","last_unstale":"2023-07-13T22:25:04.661239+0000","last_undegraded":"2023-07-13T22:25:04.661239+0000","last_fullsized":"2023-07-13T22:25:04.661239+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:19:49.849385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894682+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:25:03.894682+0000","last_peered":"2023-07-13T22:25:03.894682+0000","last_clean":"2023-07-13T22:25:03.894682+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:25:03.894682+0000","last_undegraded":"2023-07-13T22:25:03.894682+0000","last_fullsized":"2023-07-13T22:25:03.894682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:23:41.104604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.661932+0000","last_change":"2023-07-13T22:25:02.908309+0000","last_active":"2023-07-13T22:25:04.661932+0000","last_peered":"2023-07-13T22:25:04.661932+0000","last_clean":"2023-07-13T22:25:04.661932+0000","last_became_active":"2023-07-13T22:25:02.908036+0000","last_became_peered":"2023-07-13T22:25:02.908036+0000","last_unstale":"2023-07-13T22:25:04.661932+0000","last_undegraded":"2023-07-13T22:25:04.661932+0000","last_fullsized":"2023-07-13T22:25:04.661932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:01.863954+0000","last_clean_scrub_stamp":"2023-07-13T22:25:01.863954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:43:29.685005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:04.662073+0000","last_change":"2023-07-13T22:25:01.888841+0000","last_active":"2023-07-13T22:25:04.662073+0000","last_peered":"2023-07-13T22:25:04.662073+0000","last_clean":"2023-07-13T22:25:04.662073+0000","last_became_active":"2023-07-13T22:25:01.888503+0000","last_became_peered":"2023-07-13T22:25:01.888503+0000","last_unstale":"2023-07-13T22:25:04.662073+0000","last_undegraded":"2023-07-13T22:25:04.662073+0000","last_fullsized":"2023-07-13T22:25:04.662073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:25:00.857479+0000","last_clean_scrub_stamp":"2023-07-13T22:25:00.857479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:48:33.115431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-07-13T22:25:03.894993+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:25:03.894993+0000","last_peered":"2023-07-13T22:25:03.894993+0000","last_clean":"2023-07-13T22:25:03.894993+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:25:03.894993+0000","last_undegraded":"2023-07-13T22:25:03.894993+0000","last_fullsized":"2023-07-13T22:25:03.894993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:28:08.944929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36620,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335220,"statfs":{"total":96636764160,"available":96599265280,"internally_reserved":0,"allocated":1511424,"data_stored":1002102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.89900000000000002,"5min":1.115,"15min":1.115},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88200000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64500000000000002,"15min":0.70099999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94799999999999995,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84299999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.74399999999999999,"15min":0.80400000000000005},"min":{"1min":0.50900000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92500000000000004,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.748},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.72499999999999998,"15min":0.80000000000000004},"min":{"1min":0.45500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93000000000000005,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.62}]},{"osd":3,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.74299999999999999,"15min":0.75900000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.83999999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.65400000000000003},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81299999999999994,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91400000000000003,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65300000000000002,"15min":0.66900000000000004},"min":{"1min":0.38200000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83899999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.5},{"interface":"front","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.79400000000000004},"min":{"1min":0.57799999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.022,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.437,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88900000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":1.042,"15min":1.042},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":925696,"data_stored":413276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.33100000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89900000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82199999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66100000000000003,"15min":0.68799999999999994},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84199999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.38100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.88300000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76000000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65300000000000002,"15min":0.64600000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71299999999999997,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.86199999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.67100000000000004},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86899999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67900000000000005,"15min":0.70099999999999996},"min":{"1min":0.40699999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94199999999999995,"5min":0.998,"15min":0.998},"last":0.87}]},{"osd":5,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94199999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.58199999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.95399999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.96999999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91900000000000004,"5min":1.111,"15min":1.111},"last":0.91700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36624,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335216,"statfs":{"total":96636764160,"available":96599261184,"internally_reserved":0,"allocated":1515520,"data_stored":1002192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46800000000000003,"15min":0.504},"min":{"1min":0.313,"5min":0.249,"15min":0.249},"max":{"1min":0.71599999999999997,"5min":1.952,"15min":1.952},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.48699999999999999,"15min":0.51900000000000002},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58499999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.499,"15min":0.55300000000000005},"min":{"1min":0.34000000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.52300000000000002,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.437},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.61099999999999999},"min":{"1min":0.42799999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65900000000000003,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.59599999999999997,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82499999999999996,"5min":2.008,"15min":2.008},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51900000000000002,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64600000000000002,"5min":0.87,"15min":0.87},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.76400000000000001,"15min":0.81799999999999995},"min":{"1min":0.59499999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89300000000000002,"5min":2.028,"15min":2.028},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.755,"15min":0.80500000000000005},"min":{"1min":0.61199999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.87,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.624}]},{"osd":5,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.74099999999999999,"15min":0.76500000000000001},"min":{"1min":0.55400000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.92900000000000005,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79600000000000004,"15min":0.86199999999999999},"min":{"1min":0.54400000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.88900000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76900000000000002,"15min":0.80600000000000005},"min":{"1min":0.57199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98299999999999998,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.80600000000000005,"15min":0.88300000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91700000000000004,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:24:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.83499999999999996},"min":{"1min":0.65400000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.92000000000000004,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78100000000000003,"15min":0.82699999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36032,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335808,"statfs":{"total":96636764160,"available":96599867392,"internally_reserved":0,"allocated":909312,"data_stored":410726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.58399999999999996,"15min":0.52800000000000002},"min":{"1min":0.30099999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.51100000000000001,"5min":11.791,"15min":11.791},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.33900000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90700000000000003,"5min":4.657,"15min":4.657},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.625,"15min":0.60899999999999999},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.746,"5min":4.5389999999999997,"15min":4.5389999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.92600000000000005,"15min":1.097},"min":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89200000000000002,"5min":13.398999999999999,"15min":13.398999999999999},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.54200000000000004,"15min":0.57999999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60799999999999998,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58399999999999996,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":1.139,"15min":1.353},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89400000000000002,"5min":13.667999999999999,"15min":13.667999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77500000000000002,"15min":0.82299999999999995},"min":{"1min":0.51400000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84299999999999997,"5min":1.776,"15min":1.776},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":1.157,"15min":1.365},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85299999999999998,"5min":13.349,"15min":13.349},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.85899999999999999,"15min":0.88600000000000001},"min":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.97199999999999998,"5min":4.7649999999999997,"15min":4.7649999999999997},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.82499999999999996,"15min":0.84299999999999997},"min":{"1min":0.52800000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.85499999999999998,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.0900000000000001,"15min":1.3400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92900000000000005,"5min":11.679,"15min":11.679},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":1.169,"15min":1.373},"min":{"1min":0.53800000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.83399999999999996,"5min":13.298,"15min":13.298},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.93500000000000005,"15min":0.90700000000000003},"min":{"1min":0.55000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84699999999999998,"5min":11.456,"15min":11.456},"last":0.76800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":921600,"data_stored":411468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48899999999999999,"15min":0.56000000000000005},"min":{"1min":0.252,"5min":0.249,"15min":0.249},"max":{"1min":0.75,"5min":2.282,"15min":2.282},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45200000000000001,"15min":0.46600000000000003},"min":{"1min":0.30399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.57199999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.55200000000000005,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66600000000000004,"5min":2.319,"15min":2.319},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.56699999999999995},"min":{"1min":0.33900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.88100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60299999999999998,"15min":0.65900000000000003},"min":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93600000000000005,"5min":2.351,"15min":2.351},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.79100000000000004,"15min":0.85699999999999998},"min":{"1min":0.56999999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.88700000000000001,"5min":2.238,"15min":2.238},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.47499999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0349999999999999,"5min":1.518,"15min":1.518},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.76900000000000002,"15min":0.78000000000000003},"min":{"1min":0.58099999999999996,"5min":0.505,"15min":0.505},"max":{"1min":0.90600000000000003,"5min":1.325,"15min":1.325},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.80500000000000005,"15min":0.871},"min":{"1min":0.59699999999999998,"5min":0.495,"15min":0.495},"max":{"1min":0.90700000000000003,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.81200000000000006},"min":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1040000000000001,"5min":1.488,"15min":1.488},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.76500000000000001,"15min":0.78800000000000003},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86099999999999999,"5min":1.214,"15min":1.214},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77500000000000002,"15min":0.78600000000000003},"min":{"1min":0.64400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0640000000000001,"5min":1.29,"15min":1.29},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.79700000000000004,"15min":0.83299999999999996},"min":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95099999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.89600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36040,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335800,"statfs":{"total":96636764160,"available":96599859200,"internally_reserved":0,"allocated":917504,"data_stored":412294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41199999999999998,"15min":0.42399999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.59799999999999998,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44500000000000001,"15min":0.439},"min":{"1min":0.33300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.38}]},{"osd":1,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.623,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48799999999999999},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.55300000000000005},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72499999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1699999999999999,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73399999999999999,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.117,"5min":1.411,"15min":1.411},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.78300000000000003},"min":{"1min":0.51800000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1779999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.034,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.046,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77900000000000003,"15min":0.80600000000000005},"min":{"1min":0.51900000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.1299999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77200000000000002,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.085,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":933888,"data_stored":413380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79400000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69099999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.442,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81899999999999995,"5min":1.298,"15min":1.298},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.47999999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.77400000000000002,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69799999999999995,"15min":0.72699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.873,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69899999999999995,"15min":0.70799999999999996},"min":{"1min":0.52700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.78300000000000003,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.83599999999999997,"15min":1.0429999999999999},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79100000000000004,"5min":10.657999999999999,"15min":10.657999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.84799999999999998,"15min":1.002},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97099999999999997,"5min":8.75,"15min":8.75},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.50800000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84099999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.94499999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.45600000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":1.173,"15min":1.173},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.72899999999999998},"min":{"1min":0.51600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.84699999999999998,"5min":1.099,"15min":1.099},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:24:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.68600000000000005},"min":{"1min":0.51300000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.82199999999999995,"5min":1.004,"15min":1.004},"last":0.873},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.497,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94799999999999995,"5min":1.212,"15min":1.212},"last":0.71099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36040,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335800,"statfs":{"total":96636764160,"available":96599859200,"internally_reserved":0,"allocated":917504,"data_stored":411746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.60499999999999998},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86299999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89000000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65800000000000003,"15min":0.64200000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.72099999999999997,"15min":0.66500000000000004},"min":{"1min":0.42799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":8.9930000000000003,"15min":8.9930000000000003},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.78500000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84699999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68400000000000005,"15min":0.67200000000000004},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89600000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84199999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88,"5min":0.995,"15min":0.995},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.495,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93200000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81799999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.72999999999999998},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":1.02,"15min":1.02},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.59099999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.92400000000000004,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.87}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:25:07.801 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:25:07.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-13T22:25:08.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.062+0000 7f7f929ff700 1 Processor -- start 2023-07-13T22:25:08.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.062+0000 7f7f929ff700 1 -- start start 2023-07-13T22:25:08.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.063+0000 7f7f929ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7f8c15a530 0x7f7f8c0c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.063+0000 7f7f929ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c0addb0 0x7f7f8c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.063+0000 7f7f929ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c070ce0 con 0x7f7f8c070890 2023-07-13T22:25:08.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.063+0000 7f7f929ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c06de70 con 0x7f7f8c15a530 2023-07-13T22:25:08.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.063+0000 7f7f929ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06e290 con 0x7f7f8c0addb0 2023-07-13T22:25:08.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f90f9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7f8c070890 0x7f7f8c06c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48980/0 (socket says 172.21.15.60:48980) 2023-07-13T22:25:08.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f90f9c700 1 -- 172.21.15.60:0/956868304 learned_addr learned my addr 172.21.15.60:0/956868304 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:08.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8bfff700 1 --2- 172.21.15.60:0/956868304 >> v2:172.21.15.105:3300/0 conn(0x7f7f8c15a530 0x7f7f8c0c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8b7fe700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c0addb0 0x7f7f8c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715850657 0 0) 0x7f7f84001490 con 0x7f7f8c070890 2023-07-13T22:25:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f740035b0 con 0x7f7f8c070890 2023-07-13T22:25:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2813651607 0 0) 0x7f7f740035b0 con 0x7f7f8c070890 2023-07-13T22:25:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c0addb0 msgr2=0x7f7f8c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8affd700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c0addb0 0x7f7f8c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 >> v2:172.21.15.105:3300/0 conn(0x7f7f8c15a530 msgr2=0x7f7f8c0c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8affd700 1 --2- 172.21.15.60:0/956868304 >> v2:172.21.15.105:3300/0 conn(0x7f7f8c15a530 0x7f7f8c0c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.064+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0c94a0 con 0x7f7f8c070890 2023-07-13T22:25:08.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f7f84001b80 con 0x7f7f8c070890 2023-07-13T22:25:08.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 >> v1:172.21.15.60:6789/0 conn(0x7f7f8c070890 legacy=0x7f7f8c06c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8affd700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 0x7f7f74003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8affd700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f74004850 0x7f7f74004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8affd700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c0addb0 0x7f7f74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06de70 con 0x7f7f74004850 2023-07-13T22:25:08.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06e290 con 0x7f7f74003a00 2023-07-13T22:25:08.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7400baf0 con 0x7f7f8c0addb0 2023-07-13T22:25:08.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8bfff700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 0x7f7f74003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f90f9c700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c0addb0 0x7f7f74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8b7fe700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f74004850 0x7f7f74004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8bfff700 1 -- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c0addb0 msgr2=0x7f7f74005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8bfff700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c0addb0 0x7f7f74005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8bfff700 1 -- 172.21.15.60:0/956868304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f74004850 msgr2=0x7f7f74004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8bfff700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f74004850 0x7f7f74004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.065+0000 7f7f8bfff700 1 -- 172.21.15.60:0/956868304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f740035b0 con 0x7f7f74003a00 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.066+0000 7f7f8bfff700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 0x7f7f74003e10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009b60 tx=0x7f7f74005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5225689d635a30e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.066+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c00e6c0 con 0x7f7f74003a00 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.066+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c00e820 con 0x7f7f74003a00 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.066+0000 7f7f8affd700 1 -- 172.21.15.60:0/956868304 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c00eb90 con 0x7f7f74003a00 2023-07-13T22:25:08.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.067+0000 7f7f929ff700 1 -- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 msgr2=0x7f7f74003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.067+0000 7f7f929ff700 1 --2- 172.21.15.60:0/956868304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 0x7f7f74003e10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009b60 tx=0x7f7f74005880 comp rx=0 tx=0).stop 2023-07-13T22:25:08.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.067+0000 7f7f90f9c700 1 -- 172.21.15.60:0/956868304 reap_dead start 2023-07-13T22:25:08.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.067+0000 7f7f929ff700 1 -- 172.21.15.60:0/956868304 shutdown_connections 2023-07-13T22:25:08.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.067+0000 7f7f929ff700 1 -- 172.21.15.60:0/956868304 >> 172.21.15.60:0/956868304 conn(0x7f7f8c0cf3b0 msgr2=0x7f7f8c0cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:08.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.067+0000 7f7f929ff700 1 -- 172.21.15.60:0/956868304 shutdown_connections 2023-07-13T22:25:08.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.067+0000 7f7f929ff700 1 -- 172.21.15.60:0/956868304 wait complete. 2023-07-13T22:25:08.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.069+0000 7f7f929ff700 1 Processor -- start 2023-07-13T22:25:08.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.070+0000 7f7f929ff700 1 -- start start 2023-07-13T22:25:08.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f929ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 0x7f7f8c144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f929ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c1f56c0 0x7f7f8c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f929ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f8c1f5b30 0x7f7f8c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f929ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c144b40 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f929ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c15b9e0 con 0x7f7f74003a00 2023-07-13T22:25:08.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f929ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c15bc30 con 0x7f7f8c1f56c0 2023-07-13T22:25:08.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f90f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f8c1f5b30 0x7f7f8c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f90f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f8c1f5b30 0x7f7f8c15c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42036/0 (socket says 172.21.15.60:42036) 2023-07-13T22:25:08.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f90f9c700 1 -- 172.21.15.60:0/1783434620 learned_addr learned my addr 172.21.15.60:0/1783434620 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:08.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f8bfff700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 0x7f7f8c144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f8b7fe700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c1f56c0 0x7f7f8c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f90f9c700 1 -- 172.21.15.60:0/1783434620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c1f56c0 msgr2=0x7f7f8c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f90f9c700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c1f56c0 0x7f7f8c0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f90f9c700 1 -- 172.21.15.60:0/1783434620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 msgr2=0x7f7f8c144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f90f9c700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 0x7f7f8c144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f90f9c700 1 -- 172.21.15.60:0/1783434620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c15bf10 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.071+0000 7f7f8b7fe700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c1f56c0 0x7f7f8c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:08.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.072+0000 7f7f90f9c700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f8c1f5b30 0x7f7f8c15c9f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c06f750 tx=0x7f7f84007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:08.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.072+0000 7f7f88ff9700 1 -- 172.21.15.60:0/1783434620 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400ae90 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.072+0000 7f7f88ff9700 1 -- 172.21.15.60:0/1783434620 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84007a50 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.072+0000 7f7f88ff9700 1 -- 172.21.15.60:0/1783434620 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84013630 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.072+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c15c1f0 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.072+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c1f51c0 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.073+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c15c840 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.074+0000 7f7f88ff9700 1 -- 172.21.15.60:0/1783434620 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f84013790 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.075+0000 7f7f88ff9700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f7806ad10 0x7f7f7806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.075+0000 7f7f88ff9700 1 -- 172.21.15.60:0/1783434620 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f84089d70 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.075+0000 7f7f8bfff700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f7806ad10 0x7f7f7806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.076+0000 7f7f8bfff700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f7806ad10 0x7f7f7806d1c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0090b0 tx=0x7f7f7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:08.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.079+0000 7f7f88ff9700 1 -- 172.21.15.60:0/1783434620 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7f84054e10 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.371+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7f8c070ce0 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.376+0000 7f7f88ff9700 1 -- 172.21.15.60:0/1783434620 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7f8c070ce0 con 0x7f7f8c1f5b30 2023-07-13T22:25:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:25:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:25:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-07-13T22:25:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-07-13T22:25:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-13T22:25:08.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.378+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f7806ad10 msgr2=0x7f7f7806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.378+0000 7f7f929ff700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f7806ad10 0x7f7f7806d1c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0090b0 tx=0x7f7f7c008030 comp rx=0 tx=0).stop 2023-07-13T22:25:08.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.378+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f8c1f5b30 msgr2=0x7f7f8c15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.378+0000 7f7f929ff700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f8c1f5b30 0x7f7f8c15c9f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c06f750 tx=0x7f7f84007340 comp rx=0 tx=0).stop 2023-07-13T22:25:08.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.379+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 shutdown_connections 2023-07-13T22:25:08.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.379+0000 7f7f929ff700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f7806ad10 0x7f7f7806d1c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.379+0000 7f7f929ff700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f8c1f5b30 0x7f7f8c15c9f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.379+0000 7f7f929ff700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f74003a00 0x7f7f8c144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.379+0000 7f7f929ff700 1 --2- 172.21.15.60:0/1783434620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f8c1f56c0 0x7f7f8c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.379+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 >> 172.21.15.60:0/1783434620 conn(0x7f7f8c0cf3b0 msgr2=0x7f7f8c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:08.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.379+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 shutdown_connections 2023-07-13T22:25:08.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.379+0000 7f7f929ff700 1 -- 172.21.15.60:0/1783434620 wait complete. 2023-07-13T22:25:08.363 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-07-13T22:25:08.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:25:08.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.600+0000 7f106e9ff700 1 Processor -- start 2023-07-13T22:25:08.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.600+0000 7f106e9ff700 1 -- start start 2023-07-13T22:25:08.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.601+0000 7f106e9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10680cefa0 0x7f106806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.601+0000 7f106e9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f10680cd760 0x7f10680cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.601+0000 7f106e9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10680cf3f0 con 0x7f10680b12a0 2023-07-13T22:25:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.601+0000 7f106e9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106806a5f0 con 0x7f10680cd760 2023-07-13T22:25:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.601+0000 7f106e9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1068160ed0 con 0x7f10680cefa0 2023-07-13T22:25:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f1067fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f10680b12a0 0x7f10680b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48994/0 (socket says 172.21.15.60:48994) 2023-07-13T22:25:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f1067fff700 1 -- 172.21.15.60:0/3167428461 learned_addr learned my addr 172.21.15.60:0/3167428461 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f106cf9c700 1 --2- 172.21.15.60:0/3167428461 >> v2:172.21.15.105:3300/0 conn(0x7f10680cd760 0x7f10680cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f105ffff700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10680cefa0 0x7f106806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424651874 0 0) 0x7f10680cf3f0 con 0x7f10680b12a0 2023-07-13T22:25:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10540035b0 con 0x7f10680b12a0 2023-07-13T22:25:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 767999015 0 0) 0x7f10540035b0 con 0x7f10680b12a0 2023-07-13T22:25:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10680cefa0 msgr2=0x7f106806bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f10677fe700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10680cefa0 0x7f106806bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 >> v2:172.21.15.105:3300/0 conn(0x7f10680cd760 msgr2=0x7f10680cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f10677fe700 1 --2- 172.21.15.60:0/3167428461 >> v2:172.21.15.105:3300/0 conn(0x7f10680cd760 0x7f10680cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.602+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10681609a0 con 0x7f10680b12a0 2023-07-13T22:25:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f1058001670 con 0x7f10680b12a0 2023-07-13T22:25:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 >> v1:172.21.15.60:6789/0 conn(0x7f10680b12a0 legacy=0x7f10680b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f10677fe700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1054003a30 0x7f1054003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f10677fe700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10680cefa0 0x7f1054004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f10677fe700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f1054005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106806a5f0 con 0x7f1054004d90 2023-07-13T22:25:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1068160ed0 con 0x7f1054003a30 2023-07-13T22:25:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105400ba60 con 0x7f10680cefa0 2023-07-13T22:25:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f106cf9c700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f1054005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f105ffff700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10680cefa0 0x7f1054004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f1067fff700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1054003a30 0x7f1054003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f106cf9c700 1 -- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10680cefa0 msgr2=0x7f1054004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f106cf9c700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10680cefa0 0x7f1054004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f106cf9c700 1 -- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1054003a30 msgr2=0x7f1054003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f106cf9c700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1054003a30 0x7f1054003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f106cf9c700 1 -- 172.21.15.60:0/3167428461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10540035b0 con 0x7f1054004d90 2023-07-13T22:25:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.603+0000 7f105ffff700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10680cefa0 0x7f1054004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.604+0000 7f106cf9c700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f1054005200 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1060009fc0 tx=0x7f1054005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f49d515336e393 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.604+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106000ec30 con 0x7f1054004d90 2023-07-13T22:25:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.604+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106000f040 con 0x7f1054004d90 2023-07-13T22:25:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.604+0000 7f10677fe700 1 -- 172.21.15.60:0/3167428461 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060014480 con 0x7f1054004d90 2023-07-13T22:25:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.604+0000 7f106e9ff700 1 -- 172.21.15.60:0/3167428461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 msgr2=0x7f1054005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.604+0000 7f106e9ff700 1 --2- 172.21.15.60:0/3167428461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f1054005200 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1060009fc0 tx=0x7f1054005880 comp rx=0 tx=0).stop 2023-07-13T22:25:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.604+0000 7f106cf9c700 1 -- 172.21.15.60:0/3167428461 reap_dead start 2023-07-13T22:25:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.604+0000 7f106e9ff700 1 -- 172.21.15.60:0/3167428461 shutdown_connections 2023-07-13T22:25:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.604+0000 7f106e9ff700 1 -- 172.21.15.60:0/3167428461 >> 172.21.15.60:0/3167428461 conn(0x7f10680cc2a0 msgr2=0x7f10680cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.605+0000 7f106e9ff700 1 -- 172.21.15.60:0/3167428461 shutdown_connections 2023-07-13T22:25:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.605+0000 7f106e9ff700 1 -- 172.21.15.60:0/3167428461 wait complete. 2023-07-13T22:25:08.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.606+0000 7f106e9ff700 1 Processor -- start 2023-07-13T22:25:08.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.606+0000 7f106e9ff700 1 -- start start 2023-07-13T22:25:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.607+0000 7f106e9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f10680c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.607+0000 7f106e9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10681e7440 0x7f10681e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.607+0000 7f106e9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106815c010 0x7f106815e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f106e9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681fa610 con 0x7f1054004d90 2023-07-13T22:25:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f106e9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681f9740 con 0x7f106815c010 2023-07-13T22:25:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f106e9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10681e6d00 con 0x7f10681e7440 2023-07-13T22:25:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f1067fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f10680c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f1067fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f10680c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42052/0 (socket says 172.21.15.60:42052) 2023-07-13T22:25:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f1067fff700 1 -- 172.21.15.60:0/1104635494 learned_addr learned my addr 172.21.15.60:0/1104635494 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f106cf9c700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106815c010 0x7f106815e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f105ffff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10681e7440 0x7f10681e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f1067fff700 1 -- 172.21.15.60:0/1104635494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10681e7440 msgr2=0x7f10681e67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f1067fff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10681e7440 0x7f10681e67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f1067fff700 1 -- 172.21.15.60:0/1104635494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106815c010 msgr2=0x7f106815e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f1067fff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106815c010 0x7f106815e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f1067fff700 1 -- 172.21.15.60:0/1104635494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10681f9990 con 0x7f1054004d90 2023-07-13T22:25:08.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.608+0000 7f1067fff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f10680c3250 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f105800e7b0 tx=0x7f105800eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:08.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.609+0000 7f10657fa700 1 -- 172.21.15.60:0/1104635494 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058012590 con 0x7f1054004d90 2023-07-13T22:25:08.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.609+0000 7f10657fa700 1 -- 172.21.15.60:0/1104635494 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1058012710 con 0x7f1054004d90 2023-07-13T22:25:08.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.609+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10681f9c70 con 0x7f1054004d90 2023-07-13T22:25:08.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.609+0000 7f10657fa700 1 -- 172.21.15.60:0/1104635494 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058012ac0 con 0x7f1054004d90 2023-07-13T22:25:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.609+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10681fa230 con 0x7f1054004d90 2023-07-13T22:25:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.610+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10680af010 con 0x7f1054004d90 2023-07-13T22:25:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.611+0000 7f10657fa700 1 -- 172.21.15.60:0/1104635494 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1058012c20 con 0x7f1054004d90 2023-07-13T22:25:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.612+0000 7f10657fa700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f104806ace0 0x7f104806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.612+0000 7f10657fa700 1 -- 172.21.15.60:0/1104635494 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f10580892a0 con 0x7f1054004d90 2023-07-13T22:25:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.612+0000 7f105ffff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f104806ace0 0x7f104806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.613+0000 7f105ffff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f104806ace0 0x7f104806d190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f106815f0f0 tx=0x7f1050008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:08.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.616+0000 7f10657fa700 1 -- 172.21.15.60:0/1104635494 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f10580542b0 con 0x7f1054004d90 2023-07-13T22:25:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.868+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1068071c90 con 0x7f1054004d90 2023-07-13T22:25:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.869+0000 7f10657fa700 1 -- 172.21.15.60:0/1104635494 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f1058012ed0 con 0x7f1054004d90 2023-07-13T22:25:08.852 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:08.852 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1603865496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1603865496},{"type":"v1","addr":"172.21.15.105:6834","nonce":1603865496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6735,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1777842000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1777842000},{"type":"v1","addr":"172.21.15.105:6835","nonce":1777842000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6738,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2613179178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2613179178},{"type":"v1","addr":"172.21.15.60:6836","nonce":2613179178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:25:02.392864+0000","modified":"2023-07-13T22:25:05.896853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3617406711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3617406711},{"type":"v1","addr":"172.21.15.60:6837","nonce":3617406711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-13T22:25:08.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.871+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f104806ace0 msgr2=0x7f104806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.871+0000 7f106e9ff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f104806ace0 0x7f104806d190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f106815f0f0 tx=0x7f1050008040 comp rx=0 tx=0).stop 2023-07-13T22:25:08.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.871+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 msgr2=0x7f10680c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.871+0000 7f106e9ff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f10680c3250 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f105800e7b0 tx=0x7f105800eb70 comp rx=0 tx=0).stop 2023-07-13T22:25:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.872+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 shutdown_connections 2023-07-13T22:25:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.872+0000 7f106e9ff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f104806ace0 0x7f104806d190 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.872+0000 7f106e9ff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1054004d90 0x7f10680c3250 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.872+0000 7f106e9ff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106815c010 0x7f106815e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.872+0000 7f106e9ff700 1 --2- 172.21.15.60:0/1104635494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10681e7440 0x7f10681e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.872+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 >> 172.21.15.60:0/1104635494 conn(0x7f10680cc2a0 msgr2=0x7f10680b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.872+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 shutdown_connections 2023-07-13T22:25:08.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:08.872+0000 7f106e9ff700 1 -- 172.21.15.60:0/1104635494 wait complete. 2023-07-13T22:25:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 41 2023-07-13T22:25:09.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:25:02.392864+0000', 'modified': '2023-07-13T22:25:05.896853+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6741}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6741': {'gid': 6741, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6837/3617406711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6835', 'nonce': 3617406711}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 3617406711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:25:09.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:25:09.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-13T22:25:09.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.497+0000 7effd8dff700 1 Processor -- start 2023-07-13T22:25:09.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.498+0000 7effd8dff700 1 -- start start 2023-07-13T22:25:09.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.499+0000 7effd8dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd4159170 0x7effd40c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.499+0000 7effd8dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7effd40c5040 0x7effd4070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.499+0000 7effd8dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd415d970 con 0x7effd40707b0 2023-07-13T22:25:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.499+0000 7effd8dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd4070bd0 con 0x7effd40c5040 2023-07-13T22:25:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.499+0000 7effd8dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd415cd60 con 0x7effd4159170 2023-07-13T22:25:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.499+0000 7effd2d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7effd40707b0 0x7effd40cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49006/0 (socket says 172.21.15.60:49006) 2023-07-13T22:25:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.499+0000 7effd2d9d700 1 -- 172.21.15.60:0/418528698 learned_addr learned my addr 172.21.15.60:0/418528698 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:09.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effcbfff700 1 --2- 172.21.15.60:0/418528698 >> v2:172.21.15.105:3300/0 conn(0x7effd40c5040 0x7effd4070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252072090 0 0) 0x7effd415d970 con 0x7effd40707b0 2023-07-13T22:25:09.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd259c700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd4159170 0x7effd40c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc00035b0 con 0x7effd40707b0 2023-07-13T22:25:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1707919242 0 0) 0x7effc00035b0 con 0x7effd40707b0 2023-07-13T22:25:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd4159170 msgr2=0x7effd40c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd4159170 0x7effd40c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 >> v2:172.21.15.105:3300/0 conn(0x7effd40c5040 msgr2=0x7effd4070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 --2- 172.21.15.60:0/418528698 >> v2:172.21.15.105:3300/0 conn(0x7effd40c5040 0x7effd4070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd415d040 con 0x7effd40707b0 2023-07-13T22:25:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7effcc0019f0 con 0x7effd40707b0 2023-07-13T22:25:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 >> v1:172.21.15.60:6789/0 conn(0x7effd40707b0 legacy=0x7effd40cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc0003a00 0x7effc0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0004850 0x7effc0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd4159170 0x7effc0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd4070bd0 con 0x7effc0004850 2023-07-13T22:25:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd415cd60 con 0x7effc0003a00 2023-07-13T22:25:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc000dc20 con 0x7effd4159170 2023-07-13T22:25:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.500+0000 7effcbfff700 1 --2- 172.21.15.60:0/418528698 >> v2:172.21.15.105:3300/0 conn(0x7effd40c5040 0x7effd4070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effd259c700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc0003a00 0x7effc0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effcbfff700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0004850 0x7effc0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effd2d9d700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd4159170 0x7effc0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effd259c700 1 -- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd4159170 msgr2=0x7effc0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effd259c700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd4159170 0x7effc0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effd259c700 1 -- 172.21.15.60:0/418528698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0004850 msgr2=0x7effc0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effd259c700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0004850 0x7effc0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effd259c700 1 -- 172.21.15.60:0/418528698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc00035b0 con 0x7effc0003a00 2023-07-13T22:25:09.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effcbfff700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0004850 0x7effc0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:09.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.501+0000 7effd259c700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc0003a00 0x7effc0003e10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7effc4009b90 tx=0x7effc0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf6ec890a6ab5088 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400e6c0 con 0x7effc0003a00 2023-07-13T22:25:09.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc400e820 con 0x7effc0003a00 2023-07-13T22:25:09.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd1d9b700 1 -- 172.21.15.60:0/418528698 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400eb90 con 0x7effc0003a00 2023-07-13T22:25:09.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd8dff700 1 -- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc0003a00 msgr2=0x7effc0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd8dff700 1 --2- 172.21.15.60:0/418528698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc0003a00 0x7effc0003e10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7effc4009b90 tx=0x7effc0005880 comp rx=0 tx=0).stop 2023-07-13T22:25:09.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd2d9d700 1 -- 172.21.15.60:0/418528698 reap_dead start 2023-07-13T22:25:09.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd8dff700 1 -- 172.21.15.60:0/418528698 shutdown_connections 2023-07-13T22:25:09.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd8dff700 1 -- 172.21.15.60:0/418528698 >> 172.21.15.60:0/418528698 conn(0x7effd40cf410 msgr2=0x7effd40cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:09.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd8dff700 1 -- 172.21.15.60:0/418528698 shutdown_connections 2023-07-13T22:25:09.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.502+0000 7effd8dff700 1 -- 172.21.15.60:0/418528698 wait complete. 2023-07-13T22:25:09.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.504+0000 7effd8dff700 1 Processor -- start 2023-07-13T22:25:09.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.504+0000 7effd8dff700 1 -- start start 2023-07-13T22:25:09.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.505+0000 7effd8dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0003a00 0x7effd4151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd8dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd406d6d0 0x7effd406db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd8dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd40c5a60 0x7effd40c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd8dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41f2060 con 0x7effc0003a00 2023-07-13T22:25:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd8dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41f2ec0 con 0x7effd406d6d0 2023-07-13T22:25:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd8dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd406e060 con 0x7effd40c5a60 2023-07-13T22:25:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0003a00 0x7effd4151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0003a00 0x7effd4151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42078/0 (socket says 172.21.15.60:42078) 2023-07-13T22:25:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd259c700 1 -- 172.21.15.60:0/3322878527 learned_addr learned my addr 172.21.15.60:0/3322878527 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effcbfff700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd406d6d0 0x7effd406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd2d9d700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd40c5a60 0x7effd40c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd259c700 1 -- 172.21.15.60:0/3322878527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd40c5a60 msgr2=0x7effd40c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd259c700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd40c5a60 0x7effd40c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd259c700 1 -- 172.21.15.60:0/3322878527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd406d6d0 msgr2=0x7effd406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd259c700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd406d6d0 0x7effd406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd259c700 1 -- 172.21.15.60:0/3322878527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd4150240 con 0x7effc0003a00 2023-07-13T22:25:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.506+0000 7effd2d9d700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd40c5a60 0x7effd40c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.507+0000 7effd259c700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0003a00 0x7effd4151c60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7effc0004740 tx=0x7effc40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.507+0000 7effcaffd700 1 -- 172.21.15.60:0/3322878527 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4021070 con 0x7effc0003a00 2023-07-13T22:25:09.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.507+0000 7effcaffd700 1 -- 172.21.15.60:0/3322878527 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc40045d0 con 0x7effc0003a00 2023-07-13T22:25:09.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.507+0000 7effcaffd700 1 -- 172.21.15.60:0/3322878527 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400e6c0 con 0x7effc0003a00 2023-07-13T22:25:09.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.507+0000 7effd8dff700 1 -- 172.21.15.60:0/3322878527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd41f2240 con 0x7effc0003a00 2023-07-13T22:25:09.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.507+0000 7effd8dff700 1 -- 172.21.15.60:0/3322878527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd41f2800 con 0x7effc0003a00 2023-07-13T22:25:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.509+0000 7effcaffd700 1 -- 172.21.15.60:0/3322878527 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effc4025070 con 0x7effc0003a00 2023-07-13T22:25:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.510+0000 7effcaffd700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effb406ace0 0x7effb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.510+0000 7effcaffd700 1 -- 172.21.15.60:0/3322878527 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effc408a7b0 con 0x7effc0003a00 2023-07-13T22:25:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.510+0000 7effcbfff700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effb406ace0 0x7effb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.511+0000 7effcbfff700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effb406ace0 0x7effb406d190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7effbc009df0 tx=0x7effbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.512+0000 7effec799b80 1 Processor -- start 2023-07-13T22:25:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.512+0000 7effec799b80 1 -- start start 2023-07-13T22:25:09.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effec799b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 0x5619eeb895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effec799b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb5bdc0 0x5619eeb89b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effec799b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 0x5619eeb74120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effec799b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619eead3ea0 con 0x5619eeb5bdc0 2023-07-13T22:25:09.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effec799b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619eea47600 con 0x5619eeb59c50 2023-07-13T22:25:09.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effec799b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619eeab6910 con 0x5619eeb59880 2023-07-13T22:25:09.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effcbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb5bdc0 0x5619eeb89b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effcbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb5bdc0 0x5619eeb89b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42092/0 (socket says 172.21.15.60:42092) 2023-07-13T22:25:09.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effcbfff700 1 -- 172.21.15.60:0/952900958 learned_addr learned my addr 172.21.15.60:0/952900958 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:09.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.513+0000 7effd259c700 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 0x5619eeb895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effd2d9d700 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 0x5619eeb74120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effcbfff700 1 -- 172.21.15.60:0/952900958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 msgr2=0x5619eeb895e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effcbfff700 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 0x5619eeb895e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effcbfff700 1 -- 172.21.15.60:0/952900958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 msgr2=0x5619eeb74120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effcbfff700 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 0x5619eeb74120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effcbfff700 1 -- 172.21.15.60:0/952900958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619ee8a3790 con 0x5619eeb5bdc0 2023-07-13T22:25:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effd2d9d700 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 0x5619eeb74120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effcbfff700 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb5bdc0 0x5619eeb89b20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7effbc014f90 tx=0x7effbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af58546ae2da1e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effb3fff700 1 -- 172.21.15.60:0/952900958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc016df0 con 0x5619eeb5bdc0 2023-07-13T22:25:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effb3fff700 1 -- 172.21.15.60:0/952900958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc023450 con 0x5619eeb5bdc0 2023-07-13T22:25:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.514+0000 7effb3fff700 1 -- 172.21.15.60:0/952900958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc0237e0 con 0x5619eeb5bdc0 2023-07-13T22:25:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.515+0000 7effec799b80 1 -- 172.21.15.60:0/952900958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb5bdc0 msgr2=0x5619eeb89b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.515+0000 7effec799b80 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb5bdc0 0x5619eeb89b20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7effbc014f90 tx=0x7effbc017000 comp rx=0 tx=0).stop 2023-07-13T22:25:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.515+0000 7effec799b80 1 -- 172.21.15.60:0/952900958 shutdown_connections 2023-07-13T22:25:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.515+0000 7effec799b80 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 0x5619eeb74120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.515+0000 7effec799b80 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 0x5619eeb895e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.515+0000 7effec799b80 1 --2- 172.21.15.60:0/952900958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb5bdc0 0x5619eeb89b20 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.515+0000 7effec799b80 1 -- 172.21.15.60:0/952900958 >> 172.21.15.60:0/952900958 conn(0x5619eeb543b0 msgr2=0x5619eeb5f550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.515+0000 7effec799b80 1 -- 172.21.15.60:0/952900958 shutdown_connections 2023-07-13T22:25:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.515+0000 7effec799b80 1 -- 172.21.15.60:0/952900958 wait complete. 2023-07-13T22:25:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.516+0000 7effec799b80 1 -- start start 2023-07-13T22:25:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.516+0000 7effec799b80 1 Processor -- start 2023-07-13T22:25:09.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.517+0000 7effec799b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59880 0x5619eeb850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.517+0000 7effec799b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 0x5619eeb85660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.517+0000 7effec799b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 0x5619eeb5a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.517+0000 7effec799b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619eeaafbe0 con 0x5619eeb59880 2023-07-13T22:25:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.517+0000 7effec799b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619eea6d5b0 con 0x5619eeb59c50 2023-07-13T22:25:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.517+0000 7effec799b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619eea53910 con 0x5619eeb5bdc0 2023-07-13T22:25:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.517+0000 7effec799b80 10 client.0 _ll_register_callbacks cb 0x5619eea6bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59880 0x5619eeb850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59880 0x5619eeb850b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42094/0 (socket says 172.21.15.60:42094) 2023-07-13T22:25:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd259c700 1 -- 172.21.15.60:0/4033694099 learned_addr learned my addr 172.21.15.60:0/4033694099 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd2d9d700 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 0x5619eeb5a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effcbfff700 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 0x5619eeb85660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd259c700 1 -- 172.21.15.60:0/4033694099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 msgr2=0x5619eeb5a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd259c700 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 0x5619eeb5a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd259c700 1 -- 172.21.15.60:0/4033694099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 msgr2=0x5619eeb85660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd259c700 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 0x5619eeb85660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd259c700 1 -- 172.21.15.60:0/4033694099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc014c10 con 0x5619eeb59880 2023-07-13T22:25:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd2d9d700 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 0x5619eeb5a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effd259c700 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59880 0x5619eeb850b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7effc40595c0 tx=0x7effc4061a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f4182400ec2e3c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:25:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effb17fa700 1 -- 172.21.15.60:0/4033694099 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40614f0 con 0x5619eeb59880 2023-07-13T22:25:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effb17fa700 1 -- 172.21.15.60:0/4033694099 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effc4061650 con 0x5619eeb59880 2023-07-13T22:25:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.518+0000 7effec799b80 1 -- 172.21.15.60:0/4033694099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619eeb519c0 con 0x5619eeb59880 2023-07-13T22:25:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effb17fa700 1 -- 172.21.15.60:0/4033694099 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7effc4061bd0 con 0x5619eeb59880 2023-07-13T22:25:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effec799b80 10 client.6885 fetch_fsmap learned FSMap version 41 2023-07-13T22:25:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effec799b80 1 -- 172.21.15.60:0/4033694099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x5619ee9900a0 con 0x5619eeb59880 2023-07-13T22:25:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effb17fa700 1 -- 172.21.15.60:0/4033694099 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4065020 con 0x5619eeb59880 2023-07-13T22:25:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effb17fa700 1 -- 172.21.15.60:0/4033694099 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 41) v1 ==== 3114+0+0 (secure 0 0 0) 0x7effc405d070 con 0x5619eeb59880 2023-07-13T22:25:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effec799b80 10 client.6885 fetch_fsmap finished waiting for FSMap version 41 2023-07-13T22:25:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effec799b80 10 client.6885 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-13T22:25:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effec799b80 20 client.6885 populate_metadata read hostname 'smithi060' 2023-07-13T22:25:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effec799b80 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] conn(0x5619eeb72a30 0x5619eeb72e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effec799b80 4 client.6885 mds_command: new command op to 6741 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effec799b80 1 -- 172.21.15.60:0/4033694099 --> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619eeb2e170 con 0x5619eeb72a30 2023-07-13T22:25:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.519+0000 7effcbfff700 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] conn(0x5619eeb72a30 0x5619eeb72e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.520+0000 7effcbfff700 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] conn(0x5619eeb72a30 0x5619eeb72e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d88eae5d9c85ae7 server_cookie=c66f3cd9c84e3f67 in_seq=0 out_seq=0 2023-07-13T22:25:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.520+0000 7effb17fa700 10 client.6885 ms_handle_connect on v2:172.21.15.60:6835/3617406711 2023-07-13T22:25:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.520+0000 7effb17fa700 1 -- 172.21.15.60:0/4033694099 <== mds.0 v2:172.21.15.60:6835/3617406711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5619eeb2e170 con 0x5619eeb72a30 2023-07-13T22:25:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effb17fa700 10 client.6885 handle_command_reply: tid=0 2023-07-13T22:25:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 client.6885 shutdown 2023-07-13T22:25:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 -- 172.21.15.60:0/4033694099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59880 msgr2=0x5619eeb850b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59880 0x5619eeb850b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7effc40595c0 tx=0x7effc4061a30 comp rx=0 tx=0).stop 2023-07-13T22:25:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 -- 172.21.15.60:0/4033694099 shutdown_connections 2023-07-13T22:25:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] conn(0x5619eeb72a30 0x5619eeb72e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59880 0x5619eeb850b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 0x5619eeb5a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 --2- 172.21.15.60:0/4033694099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59c50 0x5619eeb85660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effb17fa700 0 client.6885 ms_handle_reset on v2:172.21.15.60:6835/3617406711 2023-07-13T22:25:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 -- 172.21.15.60:0/4033694099 >> 172.21.15.60:0/4033694099 conn(0x5619eeb543b0 msgr2=0x5619eeb5ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 -- 172.21.15.60:0/4033694099 shutdown_connections 2023-07-13T22:25:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 1 -- 172.21.15.60:0/4033694099 wait complete. 2023-07-13T22:25:09.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.521+0000 7effec799b80 20 client.6885 trim_cache size 0 max 16384 2023-07-13T22:25:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.540+0000 7effec799b80 1 Processor -- start 2023-07-13T22:25:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.540+0000 7effec799b80 1 -- start start 2023-07-13T22:25:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.540+0000 7effec799b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59880 0x5619eeb5f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.540+0000 7effec799b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeb5c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.540+0000 7effec799b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 0x5619eeb5cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effec799b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619ee8a4390 con 0x5619eeb59c50 2023-07-13T22:25:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effec799b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619eeb5a020 con 0x5619eeb59880 2023-07-13T22:25:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effec799b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619eeb85970 con 0x5619eeb5bdc0 2023-07-13T22:25:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effcbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeb5c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effcbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeb5c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42104/0 (socket says 172.21.15.60:42104) 2023-07-13T22:25:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effcbfff700 1 -- 172.21.15.60:0/2098133239 learned_addr learned my addr 172.21.15.60:0/2098133239 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effd259c700 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59880 0x5619eeb5f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effd2d9d700 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 0x5619eeb5cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effcbfff700 1 -- 172.21.15.60:0/2098133239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 msgr2=0x5619eeb5cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effcbfff700 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 0x5619eeb5cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effcbfff700 1 -- 172.21.15.60:0/2098133239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59880 msgr2=0x5619eeb5f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effcbfff700 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59880 0x5619eeb5f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effcbfff700 1 -- 172.21.15.60:0/2098133239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619eeb8aa20 con 0x5619eeb59c50 2023-07-13T22:25:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effd259c700 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59880 0x5619eeb5f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.541+0000 7effcbfff700 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeb5c650 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7effbc014f90 tx=0x7effbc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de3796f4ae6f21b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effc8ff9700 1 -- 172.21.15.60:0/2098133239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc032070 con 0x5619eeb59c50 2023-07-13T22:25:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effc8ff9700 1 -- 172.21.15.60:0/2098133239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc034310 con 0x5619eeb59c50 2023-07-13T22:25:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effc8ff9700 1 -- 172.21.15.60:0/2098133239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc024030 con 0x5619eeb59c50 2023-07-13T22:25:09.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effec799b80 1 -- 172.21.15.60:0/2098133239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 msgr2=0x5619eeb5c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effec799b80 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeb5c650 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7effbc014f90 tx=0x7effbc01b000 comp rx=0 tx=0).stop 2023-07-13T22:25:09.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effec799b80 1 -- 172.21.15.60:0/2098133239 shutdown_connections 2023-07-13T22:25:09.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effec799b80 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb5bdc0 0x5619eeb5cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effec799b80 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb59880 0x5619eeb5f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effec799b80 1 --2- 172.21.15.60:0/2098133239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeb5c650 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effec799b80 1 -- 172.21.15.60:0/2098133239 >> 172.21.15.60:0/2098133239 conn(0x5619eeb543b0 msgr2=0x5619eeb5ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effec799b80 1 -- 172.21.15.60:0/2098133239 shutdown_connections 2023-07-13T22:25:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.542+0000 7effec799b80 1 -- 172.21.15.60:0/2098133239 wait complete. 2023-07-13T22:25:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.543+0000 7effec799b80 1 -- start start 2023-07-13T22:25:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.543+0000 7effec799b80 1 Processor -- start 2023-07-13T22:25:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effec799b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 0x5619eeb8bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effec799b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeba7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effec799b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb5bdc0 0x5619eeb5c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effec799b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619eec059c0 con 0x5619eeb59c50 2023-07-13T22:25:09.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effec799b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619eeb81dd0 con 0x5619eeb5bdc0 2023-07-13T22:25:09.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effec799b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619eec0f2c0 con 0x5619eeb59880 2023-07-13T22:25:09.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effec799b80 10 client.0 _ll_register_callbacks cb 0x7effc405d5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:09.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effcbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeba7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effcbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeba7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42120/0 (socket says 172.21.15.60:42120) 2023-07-13T22:25:09.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effd259c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 0x5619eeb8bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.544+0000 7effcbfff700 1 -- 172.21.15.60:0/1136172876 learned_addr learned my addr 172.21.15.60:0/1136172876 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:09.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effd2d9d700 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb5bdc0 0x5619eeb5c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effd259c700 1 -- 172.21.15.60:0/1136172876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb5bdc0 msgr2=0x5619eeb5c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effd259c700 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb5bdc0 0x5619eeb5c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effd259c700 1 -- 172.21.15.60:0/1136172876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 msgr2=0x5619eeba7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effd259c700 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeba7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effd259c700 1 -- 172.21.15.60:0/1136172876 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619ee8a3790 con 0x5619eeb59880 2023-07-13T22:25:09.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effcbfff700 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeba7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:09.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effd259c700 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 0x5619eeb8bce0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7effc4061450 tx=0x7effc40594b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0c12f94933870b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.105:3301/0 2023-07-13T22:25:09.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effb17fa700 1 -- 172.21.15.60:0/1136172876 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4061bd0 con 0x5619eeb59880 2023-07-13T22:25:09.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effb17fa700 1 -- 172.21.15.60:0/1136172876 <== mon.2 v2:172.21.15.105:3301/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effc4066080 con 0x5619eeb59880 2023-07-13T22:25:09.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.545+0000 7effec799b80 1 -- 172.21.15.60:0/1136172876 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effbc023450 con 0x5619eeb59880 2023-07-13T22:25:09.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effb17fa700 1 -- 172.21.15.60:0/1136172876 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7effc4058f90 con 0x5619eeb59880 2023-07-13T22:25:09.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effec799b80 10 client.5999 fetch_fsmap learned FSMap version 41 2023-07-13T22:25:09.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effec799b80 1 -- 172.21.15.60:0/1136172876 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x5619eea75b10 con 0x5619eeb59880 2023-07-13T22:25:09.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effb17fa700 1 -- 172.21.15.60:0/1136172876 <== mon.2 v2:172.21.15.105:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc023450 con 0x5619eeb59880 2023-07-13T22:25:09.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effb17fa700 1 -- 172.21.15.60:0/1136172876 <== mon.2 v2:172.21.15.105:3301/0 5 ==== fsmap(e 41) v1 ==== 3114+0+0 (secure 0 0 0) 0x7effc40614f0 con 0x5619eeb59880 2023-07-13T22:25:09.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effec799b80 10 client.5999 fetch_fsmap finished waiting for FSMap version 41 2023-07-13T22:25:09.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effec799b80 10 client.5999 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-13T22:25:09.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effec799b80 20 client.5999 populate_metadata read hostname 'smithi060' 2023-07-13T22:25:09.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effec799b80 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] conn(0x5619eeb73c50 0x5619eeb74020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effec799b80 4 client.5999 mds_command: new command op to 6741 tid=0[{"prefix": "status"}] 2023-07-13T22:25:09.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effec799b80 1 -- 172.21.15.60:0/1136172876 --> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5619eeaa8d20 con 0x5619eeb73c50 2023-07-13T22:25:09.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.546+0000 7effcbfff700 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] conn(0x5619eeb73c50 0x5619eeb74020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effcbfff700 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] conn(0x5619eeb73c50 0x5619eeb74020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5e75737f69d4ad server_cookie=fa2dc339bc24ef86 in_seq=0 out_seq=0 2023-07-13T22:25:09.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effb17fa700 10 client.5999 ms_handle_connect on v2:172.21.15.60:6835/3617406711 2023-07-13T22:25:09.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effb17fa700 1 -- 172.21.15.60:0/1136172876 <== mds.0 v2:172.21.15.60:6835/3617406711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5619eeaa8d20 con 0x5619eeb73c50 2023-07-13T22:25:09.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effb17fa700 10 client.5999 handle_command_reply: tid=0 2023-07-13T22:25:09.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 client.5999 shutdown 2023-07-13T22:25:09.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 -- 172.21.15.60:0/1136172876 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 msgr2=0x5619eeb8bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 0x5619eeb8bce0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7effc4061450 tx=0x7effc40594b0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 -- 172.21.15.60:0/1136172876 shutdown_connections 2023-07-13T22:25:09.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.60:6835/3617406711,v1:172.21.15.60:6837/3617406711] conn(0x5619eeb73c50 0x5619eeb74020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619eeb5bdc0 0x5619eeb5c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5619eeb59880 0x5619eeb8bce0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effb17fa700 0 client.5999 ms_handle_reset on v2:172.21.15.60:6835/3617406711 2023-07-13T22:25:09.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 --2- 172.21.15.60:0/1136172876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619eeb59c50 0x5619eeba7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 -- 172.21.15.60:0/1136172876 >> 172.21.15.60:0/1136172876 conn(0x5619eeb543b0 msgr2=0x5619eeb60a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:09.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.547+0000 7effec799b80 1 -- 172.21.15.60:0/1136172876 shutdown_connections 2023-07-13T22:25:09.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effec799b80 1 -- 172.21.15.60:0/1136172876 wait complete. 2023-07-13T22:25:09.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effec799b80 20 client.5999 trim_cache size 0 max 16384 2023-07-13T22:25:09.539 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:25:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:25:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:25:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6741, 2023-07-13T22:25:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:25:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:25:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:25:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.6752064950000003, 2023-07-13T22:25:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 41, 2023-07-13T22:25:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 75, 2023-07-13T22:25:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 75, 2023-07-13T22:25:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 12.130592570999999 2023-07-13T22:25:09.540 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:25:09.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 -- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effb406ace0 msgr2=0x7effb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effb406ace0 0x7effb406d190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7effbc009df0 tx=0x7effbc008040 comp rx=0 tx=0).stop 2023-07-13T22:25:09.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 -- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0003a00 msgr2=0x7effd4151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0003a00 0x7effd4151c60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7effc0004740 tx=0x7effc40047c0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 -- 172.21.15.60:0/3322878527 shutdown_connections 2023-07-13T22:25:09.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effb406ace0 0x7effb406d190 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc0003a00 0x7effd4151c60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effd40c5a60 0x7effd40c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 --2- 172.21.15.60:0/3322878527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effd406d6d0 0x7effd406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 -- 172.21.15.60:0/3322878527 >> 172.21.15.60:0/3322878527 conn(0x7effd40cf410 msgr2=0x7effd40cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:09.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 -- 172.21.15.60:0/3322878527 shutdown_connections 2023-07-13T22:25:09.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.548+0000 7effd8dff700 1 -- 172.21.15.60:0/3322878527 wait complete. 2023-07-13T22:25:09.548 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:25:09.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.768+0000 7ff74f9ff700 1 Processor -- start 2023-07-13T22:25:09.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.768+0000 7ff74f9ff700 1 -- start start 2023-07-13T22:25:09.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.769+0000 7ff74f9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff74815c300 0x7ff7480b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.769+0000 7ff74f9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7480b46c0 0x7ff7480b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.769+0000 7ff74f9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7480d18a0 con 0x7ff74815c6d0 2023-07-13T22:25:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.769+0000 7ff74f9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7480cf500 con 0x7ff7480b46c0 2023-07-13T22:25:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.769+0000 7ff74f9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7480b4fd0 con 0x7ff74815c300 2023-07-13T22:25:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.769+0000 7ff74cf9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff74815c6d0 0x7ff7480cedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49016/0 (socket says 172.21.15.60:49016) 2023-07-13T22:25:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.769+0000 7ff74cf9a700 1 -- 172.21.15.60:0/1151699463 learned_addr learned my addr 172.21.15.60:0/1151699463 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:09.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.769+0000 7ff74df9c700 1 --2- 172.21.15.60:0/1151699463 >> v2:172.21.15.60:3300/0 conn(0x7ff7480b46c0 0x7ff7480b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.769+0000 7ff74d79b700 1 --2- 172.21.15.60:0/1151699463 >> v2:172.21.15.105:3300/0 conn(0x7ff74815c300 0x7ff7480b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.770+0000 7ff73ffff700 1 -- 172.21.15.60:0/1151699463 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283662072 0 0) 0x7ff7480d18a0 con 0x7ff74815c6d0 2023-07-13T22:25:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.770+0000 7ff73ffff700 1 -- 172.21.15.60:0/1151699463 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7340035b0 con 0x7ff74815c6d0 2023-07-13T22:25:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.770+0000 7ff74d79b700 1 -- 172.21.15.60:0/1151699463 >> v2:172.21.15.60:3300/0 conn(0x7ff7480b46c0 msgr2=0x7ff7480b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.770+0000 7ff74d79b700 1 --2- 172.21.15.60:0/1151699463 >> v2:172.21.15.60:3300/0 conn(0x7ff7480b46c0 0x7ff7480b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.770+0000 7ff74d79b700 1 -- 172.21.15.60:0/1151699463 >> v1:172.21.15.60:6789/0 conn(0x7ff74815c6d0 legacy=0x7ff7480cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.770+0000 7ff74d79b700 1 -- 172.21.15.60:0/1151699463 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480d15c0 con 0x7ff74815c300 2023-07-13T22:25:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.770+0000 7ff74d79b700 1 --2- 172.21.15.60:0/1151699463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff74815c300 0x7ff7480b5a90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff7480d2ba0 tx=0x7ff744009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e220e60c347fb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff73ffff700 1 -- 172.21.15.60:0/1151699463 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744011830 con 0x7ff74815c300 2023-07-13T22:25:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff73ffff700 1 -- 172.21.15.60:0/1151699463 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff744011990 con 0x7ff74815c300 2023-07-13T22:25:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff73ffff700 1 -- 172.21.15.60:0/1151699463 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744011d20 con 0x7ff74815c300 2023-07-13T22:25:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff74f9ff700 1 -- 172.21.15.60:0/1151699463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff74815c300 msgr2=0x7ff7480b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff74f9ff700 1 --2- 172.21.15.60:0/1151699463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff74815c300 0x7ff7480b5a90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff7480d2ba0 tx=0x7ff744009b20 comp rx=0 tx=0).stop 2023-07-13T22:25:09.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff74f9ff700 1 -- 172.21.15.60:0/1151699463 shutdown_connections 2023-07-13T22:25:09.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff74f9ff700 1 --2- 172.21.15.60:0/1151699463 >> v2:172.21.15.60:3300/0 conn(0x7ff7480b46c0 0x7ff7480b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff74f9ff700 1 --2- 172.21.15.60:0/1151699463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff74815c300 0x7ff7480b5a90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff74f9ff700 1 -- 172.21.15.60:0/1151699463 >> 172.21.15.60:0/1151699463 conn(0x7ff7480cf6c0 msgr2=0x7ff7480cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:09.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff74f9ff700 1 -- 172.21.15.60:0/1151699463 shutdown_connections 2023-07-13T22:25:09.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.771+0000 7ff74f9ff700 1 -- 172.21.15.60:0/1151699463 wait complete. 2023-07-13T22:25:09.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.773+0000 7ff74f9ff700 1 Processor -- start 2023-07-13T22:25:09.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.773+0000 7ff74f9ff700 1 -- start start 2023-07-13T22:25:09.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74f9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7480b46c0 0x7ff7481f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74f9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74815c300 0x7ff748148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74f9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff74815c6d0 0x7ff7480c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74f9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff748149c20 con 0x7ff74815c300 2023-07-13T22:25:09.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74f9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff748149150 con 0x7ff74815c6d0 2023-07-13T22:25:09.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74f9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480cbb80 con 0x7ff7480b46c0 2023-07-13T22:25:09.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74cf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74815c300 0x7ff748148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74cf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74815c300 0x7ff748148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42136/0 (socket says 172.21.15.60:42136) 2023-07-13T22:25:09.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74cf9a700 1 -- 172.21.15.60:0/2697043027 learned_addr learned my addr 172.21.15.60:0/2697043027 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:09.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74df9c700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff74815c6d0 0x7ff7480c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.774+0000 7ff74d79b700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7480b46c0 0x7ff7481f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff74cf9a700 1 -- 172.21.15.60:0/2697043027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7480b46c0 msgr2=0x7ff7481f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff74cf9a700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7480b46c0 0x7ff7481f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff74cf9a700 1 -- 172.21.15.60:0/2697043027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff74815c6d0 msgr2=0x7ff7480c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff74cf9a700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff74815c6d0 0x7ff7480c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff74cf9a700 1 -- 172.21.15.60:0/2697043027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440094a0 con 0x7ff74815c300 2023-07-13T22:25:09.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff74cf9a700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74815c300 0x7ff748148c10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c700 tx=0x7ff73800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff73dffb700 1 -- 172.21.15.60:0/2697043027 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738005d70 con 0x7ff74815c300 2023-07-13T22:25:09.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff73dffb700 1 -- 172.21.15.60:0/2697043027 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7380070d0 con 0x7ff74815c300 2023-07-13T22:25:09.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff73dffb700 1 -- 172.21.15.60:0/2697043027 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738007460 con 0x7ff74815c300 2023-07-13T22:25:09.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff748149380 con 0x7ff74815c300 2023-07-13T22:25:09.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.775+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7480cbe20 con 0x7ff74815c300 2023-07-13T22:25:09.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.776+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7480cd3a0 con 0x7ff74815c300 2023-07-13T22:25:09.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.777+0000 7ff73dffb700 1 -- 172.21.15.60:0/2697043027 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff7380075c0 con 0x7ff74815c300 2023-07-13T22:25:09.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.778+0000 7ff73dffb700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72c06ace0 0x7ff72c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.778+0000 7ff73dffb700 1 -- 172.21.15.60:0/2697043027 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff738088a00 con 0x7ff74815c300 2023-07-13T22:25:09.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.778+0000 7ff74d79b700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72c06ace0 0x7ff72c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.779+0000 7ff74d79b700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72c06ace0 0x7ff72c06d190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff7440090b0 tx=0x7ff74401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:09.782+0000 7ff73dffb700 1 -- 172.21.15.60:0/2697043027 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff7480cd3a0 con 0x7ff74815c300 2023-07-13T22:25:10.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.048+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff7480cd3a0 con 0x7ff74815c300 2023-07-13T22:25:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.048+0000 7ff73dffb700 1 -- 172.21.15.60:0/2697043027 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff7480cd3a0 con 0x7ff74815c300 2023-07-13T22:25:10.031 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:25:10.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72c06ace0 msgr2=0x7ff72c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72c06ace0 0x7ff72c06d190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff7440090b0 tx=0x7ff74401c040 comp rx=0 tx=0).stop 2023-07-13T22:25:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74815c300 msgr2=0x7ff748148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74815c300 0x7ff748148c10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c700 tx=0x7ff73800ca10 comp rx=0 tx=0).stop 2023-07-13T22:25:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 shutdown_connections 2023-07-13T22:25:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72c06ace0 0x7ff72c06d190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff74815c6d0 0x7ff7480c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7480b46c0 0x7ff7481f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 --2- 172.21.15.60:0/2697043027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff74815c300 0x7ff748148c10 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 >> 172.21.15.60:0/2697043027 conn(0x7ff7480cf6c0 msgr2=0x7ff7480d2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 shutdown_connections 2023-07-13T22:25:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:10.051+0000 7ff74f9ff700 1 -- 172.21.15.60:0/2697043027 wait complete. 2023-07-13T22:25:10.052 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:25:10.052 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:25:10.052 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:25:10.052 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:25:10.053 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:25:10.053 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:25:10.053 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:25:10.053 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:25:10.054 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:25:10.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:10.055 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:25:10.101 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:25:10.102 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:25:10.102 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:25:10.102 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:10.102 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:25:10.102 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:10.102 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:25:10.103 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:10.103 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:25:10.103 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:25:10.103 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:10.103 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:25:10.103 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:25:10.103 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:10.104 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:25:10.104 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:25:10.104 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41463sec preferred_lft 41463sec 2023-07-13T22:25:10.105 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:25:10.105 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:10.106 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:25:10.106 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:10.106 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:25:10.106 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:25:10.107 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:10.107 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:25:10.107 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:10.108 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:25:10.108 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:10.108 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:25:10.108 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:25:10.108 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:25:10.109 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:10.109 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:25:10.109 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:25:10.109 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:25:10.110 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:10.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:10.112 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:25:10.181 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:25:10.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:10.184 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:25:10.233 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:25:10.234 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:25:10.234 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:25:10.234 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:25:10.234 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:25:10.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:10.235 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:25:10.235 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:25:10.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:25:10.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:25:10.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:25:10.237 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:25:10.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:10.410 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:25:10.456 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:25:10.456 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:25:10.457 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:25:10.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:10.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:25:10.529 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:25:10.530 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:25:10.530 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:25:10.577 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:25:10.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:25:10.578 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:25:10.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:25:10.772 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:25:10.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:25:10.842 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:25:10.845 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:25:10.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:10.846 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:25:10.894 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:25:10.895 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:25:10.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:25:10.977 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:25:10.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:25:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:25:11.051 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:25:11.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:11.052 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:25:11.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:25:11.096+0000 7f1ae19485c0 -1 init, newargv = 0x5587a40c0110 newargc=15 2023-07-13T22:25:11.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[141735]: starting ceph client 2023-07-13T22:25:11.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[141735]: starting fuse 2023-07-13T22:25:11.105 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:25:11.105 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:25:11.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:25:11.106 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:25:11.106 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:25:11.107 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:25:11.107 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:25:11.107 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:11.108 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:25:11.108 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:25:11.108 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:25:11.109 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:25:11.109 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:11.109 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:25:11.109 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:25:11.110 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:25:11.110 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:25:11.110 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:11.111 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:25:11.111 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:25:11.111 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:25:11.111 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:25:11.112 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:25:11.112 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:25:11.112 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:25:11.113 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:25:11.113 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:11.113 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:25:11.114 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:25:11.292 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.141735.asok 2023-07-13T22:25:11.469 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.141735.asok 2023-07-13T22:25:11.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:11.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.141735.asok status 2023-07-13T22:25:11.702 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:25:11.702 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:25:11.702 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:25:11.703 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:25:11.703 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:25:11.703 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:25:11.703 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:25:11.703 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "141735", 2023-07-13T22:25:11.703 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:25:11.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:25:11.704 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:25:11.704 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:25:11.704 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6906, 2023-07-13T22:25:11.704 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:25:11.704 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:25:11.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:25:11.704 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6906 2023-07-13T22:25:11.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:25:11.705 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:25:11.705 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:25:11.705 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:25:11.705 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1059118983 2023-07-13T22:25:11.705 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:25:11.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:25:11.705 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:25:11.706 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:25:11.706 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:25:11.706 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1059118983 2023-07-13T22:25:11.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:25:11.706 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6906 192.168.0.1:0/1059118983", 2023-07-13T22:25:11.706 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/1059118983", 2023-07-13T22:25:11.706 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:25:11.706 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 41, 2023-07-13T22:25:11.707 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 75, 2023-07-13T22:25:11.707 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:25:11.707 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:25:11.707 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:25:11.707 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:25:11.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:11.717 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:25:11.717 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:25:11.764 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:25:11.764 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:25:11.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:11.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:25:11.836 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:25:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.058+0000 7f4e5dfff700 1 Processor -- start 2023-07-13T22:25:12.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.058+0000 7f4e5dfff700 1 -- start start 2023-07-13T22:25:12.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.059+0000 7f4e5dfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 0x7f4e5806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.059+0000 7f4e5dfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e580b24b0 0x7f4e580b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.059+0000 7f4e5dfff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5806b8e0 con 0x7f4e580b3050 2023-07-13T22:25:12.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e5dfff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5806b6f0 con 0x7f4e580b24b0 2023-07-13T22:25:12.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e5dfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5815cab0 con 0x7f4e5815c6e0 2023-07-13T22:25:12.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e57fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e580b24b0 0x7f4e580b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e57fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e580b24b0 0x7f4e580b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42152/0 (socket says 172.21.15.60:42152) 2023-07-13T22:25:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e57fff700 1 -- 172.21.15.60:0/990689693 learned_addr learned my addr 172.21.15.60:0/990689693 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e56ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4e580b3050 0x7f4e5806afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49020/0 (socket says 172.21.15.60:49020) 2023-07-13T22:25:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e577fe700 1 --2- 172.21.15.60:0/990689693 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 0x7f4e5806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e57fff700 1 -- 172.21.15.60:0/990689693 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 msgr2=0x7f4e5806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e57fff700 1 --2- 172.21.15.60:0/990689693 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 0x7f4e5806c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e57fff700 1 -- 172.21.15.60:0/990689693 >> v1:172.21.15.60:6789/0 conn(0x7f4e580b3050 legacy=0x7f4e5806afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e57fff700 1 -- 172.21.15.60:0/990689693 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5806bbc0 con 0x7f4e580b24b0 2023-07-13T22:25:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.060+0000 7f4e57fff700 1 --2- 172.21.15.60:0/990689693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 0x7f4e580b2930 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009e50 tx=0x7f4e5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47909a0d2ea27ec3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e567fc700 1 -- 172.21.15.60:0/990689693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000e070 con 0x7f4e580b24b0 2023-07-13T22:25:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e567fc700 1 -- 172.21.15.60:0/990689693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5000b9c0 con 0x7f4e580b24b0 2023-07-13T22:25:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e567fc700 1 -- 172.21.15.60:0/990689693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000bd70 con 0x7f4e580b24b0 2023-07-13T22:25:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e5dfff700 1 -- 172.21.15.60:0/990689693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 msgr2=0x7f4e580b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e5dfff700 1 --2- 172.21.15.60:0/990689693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 0x7f4e580b2930 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009e50 tx=0x7f4e5000b330 comp rx=0 tx=0).stop 2023-07-13T22:25:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e5dfff700 1 -- 172.21.15.60:0/990689693 shutdown_connections 2023-07-13T22:25:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e5dfff700 1 --2- 172.21.15.60:0/990689693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 0x7f4e580b2930 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e5dfff700 1 --2- 172.21.15.60:0/990689693 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 0x7f4e5806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e5dfff700 1 -- 172.21.15.60:0/990689693 >> 172.21.15.60:0/990689693 conn(0x7f4e5806e680 msgr2=0x7f4e5806ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e5dfff700 1 -- 172.21.15.60:0/990689693 shutdown_connections 2023-07-13T22:25:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.061+0000 7f4e5dfff700 1 -- 172.21.15.60:0/990689693 wait complete. 2023-07-13T22:25:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.063+0000 7f4e5dfff700 1 Processor -- start 2023-07-13T22:25:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.063+0000 7f4e5dfff700 1 -- start start 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.064+0000 7f4e5dfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 0x7f4e58147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.064+0000 7f4e5dfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e580b3050 0x7f4e580c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.064+0000 7f4e5dfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 0x7f4e580c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.064+0000 7f4e5dfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5815dce0 con 0x7f4e580b24b0 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.064+0000 7f4e5dfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58147570 con 0x7f4e580b3050 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.064+0000 7f4e5dfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58147880 con 0x7f4e5815c6e0 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.064+0000 7f4e577fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 0x7f4e58147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.064+0000 7f4e577fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 0x7f4e58147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42162/0 (socket says 172.21.15.60:42162) 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.064+0000 7f4e577fe700 1 -- 172.21.15.60:0/2974965587 learned_addr learned my addr 172.21.15.60:0/2974965587 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e56ffd700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e580b3050 0x7f4e580c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e57fff700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 0x7f4e580c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e577fe700 1 -- 172.21.15.60:0/2974965587 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 msgr2=0x7f4e580c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e577fe700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 0x7f4e580c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e577fe700 1 -- 172.21.15.60:0/2974965587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e580b3050 msgr2=0x7f4e580c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e577fe700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e580b3050 0x7f4e580c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e577fe700 1 -- 172.21.15.60:0/2974965587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e50009b30 con 0x7f4e580b24b0 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e577fe700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 0x7f4e58147030 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800d800 tx=0x7f4e4800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e3ffff700 1 -- 172.21.15.60:0/2974965587 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480125f0 con 0x7f4e580b24b0 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e3ffff700 1 -- 172.21.15.60:0/2974965587 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48012750 con 0x7f4e580b24b0 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e580ca4a0 con 0x7f4e580b24b0 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.065+0000 7f4e3ffff700 1 -- 172.21.15.60:0/2974965587 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48012b00 con 0x7f4e580b24b0 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.066+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5815df80 con 0x7f4e580b24b0 2023-07-13T22:25:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.067+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5815e2f0 con 0x7f4e580b24b0 2023-07-13T22:25:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.068+0000 7f4e3ffff700 1 -- 172.21.15.60:0/2974965587 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e48012c60 con 0x7f4e580b24b0 2023-07-13T22:25:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.069+0000 7f4e3ffff700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ace0 0x7f4e4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.069+0000 7f4e3ffff700 1 -- 172.21.15.60:0/2974965587 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e48089910 con 0x7f4e580b24b0 2023-07-13T22:25:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.069+0000 7f4e56ffd700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ace0 0x7f4e4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.070+0000 7f4e56ffd700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ace0 0x7f4e4406d190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c000f60 tx=0x7f4e4c007d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.074+0000 7f4e3ffff700 1 -- 172.21.15.60:0/2974965587 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4e48054920 con 0x7f4e580b24b0 2023-07-13T22:25:12.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.329+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4e5815e110 con 0x7f4e580b24b0 2023-07-13T22:25:12.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.329+0000 7f4e3ffff700 1 -- 172.21.15.60:0/2974965587 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4e48057f40 con 0x7f4e580b24b0 2023-07-13T22:25:12.312 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:25:12.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ace0 msgr2=0x7f4e4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ace0 0x7f4e4406d190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c000f60 tx=0x7f4e4c007d10 comp rx=0 tx=0).stop 2023-07-13T22:25:12.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 msgr2=0x7f4e58147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 0x7f4e58147030 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800d800 tx=0x7f4e4800dbc0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 shutdown_connections 2023-07-13T22:25:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ace0 0x7f4e4406d190 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580b24b0 0x7f4e58147030 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e5815c6e0 0x7f4e580c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 --2- 172.21.15.60:0/2974965587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e580b3050 0x7f4e580c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 >> 172.21.15.60:0/2974965587 conn(0x7f4e5806e680 msgr2=0x7f4e580d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 shutdown_connections 2023-07-13T22:25:12.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:12.332+0000 7f4e5dfff700 1 -- 172.21.15.60:0/2974965587 wait complete. 2023-07-13T22:25:12.332 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:25:12.333 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:25:12.333 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:25:12.334 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:25:12.334 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:25:12.335 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:25:12.336 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:25:12.336 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:25:12.337 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:25:12.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:12.338 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:25:12.391 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:25:12.391 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:25:12.391 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:25:12.391 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:12.391 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:25:12.391 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:12.392 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:25:12.392 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:12.392 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:25:12.392 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:25:12.392 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:12.392 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:25:12.392 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:25:12.393 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:12.393 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:25:12.393 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:12.393 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:25:12.393 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41458sec preferred_lft 41458sec 2023-07-13T22:25:12.393 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:25:12.393 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:12.394 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:25:12.394 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:25:12.394 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:25:12.394 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:12.394 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:25:12.394 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:12.394 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:25:12.394 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:25:12.395 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:25:12.395 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:12.395 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:25:12.395 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:25:12.395 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::14e3:bfff:fea8:bf57/64 scope link 2023-07-13T22:25:12.395 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:25:12.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:12.397 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:25:12.466 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:25:12.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:12.469 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:25:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:25:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:25:12.518 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:25:12.519 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:25:12.519 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:25:12.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:12.520 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:25:12.520 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:25:12.521 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:25:12.521 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:25:12.521 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:25:12.521 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:25:12.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:12.692 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:25:12.738 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:25:12.738 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:25:12.739 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:25:12.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:12.739 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:25:12.850 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:25:12.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:25:12.851 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:25:12.897 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:25:12.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:25:12.897 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:25:12.982 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:25:13.091 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:25:13.092 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:25:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:25:13.162 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:25:13.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:13.163 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:25:13.209 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:25:13.210 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:25:13.254 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:25:13.324 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:25:13.324 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:25:13.393 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:25:13.395 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:25:13.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:13.396 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:25:13.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:25:13.441+0000 7f0f8c1155c0 -1 init, newargv = 0x55d46cc75f50 newargc=15 2023-07-13T22:25:13.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[123518]: starting ceph client 2023-07-13T22:25:13.431 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[123518]: starting fuse 2023-07-13T22:25:13.448 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:25:13.448 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:25:13.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:25:13.449 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:25:13.449 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:25:13.450 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:25:13.450 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:25:13.450 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:25:13.451 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:25:13.451 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:25:13.451 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:25:13.451 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:25:13.452 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:25:13.452 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:25:13.452 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:25:13.453 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:25:13.453 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:25:13.453 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:25:13.453 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:25:13.454 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:25:13.454 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:25:13.454 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:25:13.455 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:25:13.455 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:25:13.455 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:25:13.455 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:25:13.456 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:25:13.456 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:25:13.456 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:25:13.629 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.123518.asok 2023-07-13T22:25:13.807 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.123518.asok 2023-07-13T22:25:13.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:13.808 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.123518.asok status 2023-07-13T22:25:14.033 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:25:14.033 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:25:14.033 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:25:14.033 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:25:14.033 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:25:14.033 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:25:14.034 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:25:14.034 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "123518", 2023-07-13T22:25:14.034 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:25:14.034 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:25:14.034 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:25:14.034 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:25:14.034 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6017, 2023-07-13T22:25:14.035 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:25:14.035 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:25:14.035 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:25:14.035 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 6017 2023-07-13T22:25:14.035 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:25:14.035 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:25:14.035 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:25:14.035 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:25:14.036 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3775652031 2023-07-13T22:25:14.036 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:25:14.036 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:25:14.036 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:25:14.036 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:25:14.036 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:25:14.036 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3775652031 2023-07-13T22:25:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:25:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.6017 192.168.0.1:0/3775652031", 2023-07-13T22:25:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/3775652031", 2023-07-13T22:25:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:25:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 41, 2023-07-13T22:25:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 75, 2023-07-13T22:25:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:25:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:25:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:25:14.038 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:25:14.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:14.047 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:25:14.048 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:25:14.094 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:25:14.095 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:25:14.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:14.095 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:25:14.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-07-13T22:25:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.439+0000 7f9673fff700 1 Processor -- start 2023-07-13T22:25:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.439+0000 7f9673fff700 1 -- start start 2023-07-13T22:25:14.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.440+0000 7f9673fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f966c0cfd50 0x7f966c0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.440+0000 7f9673fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966c0d1870 con 0x7f966c0b1ac0 2023-07-13T22:25:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.440+0000 7f9673fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966c0cf370 con 0x7f966c0b8200 2023-07-13T22:25:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.440+0000 7f9673fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f966c0d12b0 con 0x7f966c0cfd50 2023-07-13T22:25:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.440+0000 7f967159a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f966c0cfd50 0x7f966c0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.440+0000 7f967159a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f966c0cfd50 0x7f966c0d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42172/0 (socket says 172.21.15.60:42172) 2023-07-13T22:25:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.440+0000 7f967159a700 1 -- 172.21.15.60:0/538605 learned_addr learned my addr 172.21.15.60:0/538605 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.440+0000 7f967259c700 1 --1- 172.21.15.60:0/538605 >> v1:172.21.15.60:6789/0 conn(0x7f966c0b1ac0 0x7f966c0d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49036/0 (socket says 172.21.15.60:49036) 2023-07-13T22:25:14.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.441+0000 7f967159a700 1 -- 172.21.15.60:0/538605 >> v1:172.21.15.105:6789/0 conn(0x7f966c0b8200 legacy=0x7f966c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.441+0000 7f967159a700 1 -- 172.21.15.60:0/538605 >> v1:172.21.15.60:6789/0 conn(0x7f966c0b1ac0 legacy=0x7f966c0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.441+0000 7f967159a700 1 -- 172.21.15.60:0/538605 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966c0d1590 con 0x7f966c0cfd50 2023-07-13T22:25:14.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.441+0000 7f967159a700 1 --2- 172.21.15.60:0/538605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0cfd50 0x7f966c0d23c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f965c0098f0 tx=0x7f965c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c751c148f38d970b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.441+0000 7f9670d99700 1 -- 172.21.15.60:0/538605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00b8a0 con 0x7f966c0cfd50 2023-07-13T22:25:14.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.441+0000 7f9670d99700 1 -- 172.21.15.60:0/538605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c00ba00 con 0x7f966c0cfd50 2023-07-13T22:25:14.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.441+0000 7f9670d99700 1 -- 172.21.15.60:0/538605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00bd90 con 0x7f966c0cfd50 2023-07-13T22:25:14.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.442+0000 7f9673fff700 1 -- 172.21.15.60:0/538605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0cfd50 msgr2=0x7f966c0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.442+0000 7f9673fff700 1 --2- 172.21.15.60:0/538605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0cfd50 0x7f966c0d23c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f965c0098f0 tx=0x7f965c009c00 comp rx=0 tx=0).stop 2023-07-13T22:25:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.442+0000 7f9673fff700 1 -- 172.21.15.60:0/538605 shutdown_connections 2023-07-13T22:25:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.442+0000 7f9673fff700 1 --2- 172.21.15.60:0/538605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0cfd50 0x7f966c0d23c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.442+0000 7f9673fff700 1 -- 172.21.15.60:0/538605 >> 172.21.15.60:0/538605 conn(0x7f966c06b2f0 msgr2=0x7f966c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.442+0000 7f9673fff700 1 -- 172.21.15.60:0/538605 shutdown_connections 2023-07-13T22:25:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.442+0000 7f9673fff700 1 -- 172.21.15.60:0/538605 wait complete. 2023-07-13T22:25:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.444+0000 7f9673fff700 1 Processor -- start 2023-07-13T22:25:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.444+0000 7f9673fff700 1 -- start start 2023-07-13T22:25:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.445+0000 7f9673fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0b1ac0 0x7f966c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.445+0000 7f9673fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f966c0b8200 0x7f966c0c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.445+0000 7f9673fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966c0cfd50 0x7f966c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.445+0000 7f9673fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c0c7950 con 0x7f966c0b1ac0 2023-07-13T22:25:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.445+0000 7f9673fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c0c76b0 con 0x7f966c0cfd50 2023-07-13T22:25:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.445+0000 7f9673fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966c1571a0 con 0x7f966c0b8200 2023-07-13T22:25:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.445+0000 7f9671d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0b1ac0 0x7f966c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.445+0000 7f9671d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0b1ac0 0x7f966c0ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42174/0 (socket says 172.21.15.60:42174) 2023-07-13T22:25:14.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.445+0000 7f9671d9b700 1 -- 172.21.15.60:0/629964795 learned_addr learned my addr 172.21.15.60:0/629964795 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:14.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f9671d9b700 1 -- 172.21.15.60:0/629964795 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f966c0b8200 msgr2=0x7f966c0c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f9671d9b700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f966c0b8200 0x7f966c0c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f9671d9b700 1 -- 172.21.15.60:0/629964795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966c0cfd50 msgr2=0x7f966c0c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f967259c700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966c0cfd50 0x7f966c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f9671d9b700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966c0cfd50 0x7f966c0c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f9671d9b700 1 -- 172.21.15.60:0/629964795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c009550 con 0x7f966c0b1ac0 2023-07-13T22:25:14.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f967259c700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966c0cfd50 0x7f966c0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:14.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f9671d9b700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0b1ac0 0x7f966c0ccd20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f966800c4e0 tx=0x7f966800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f96627fc700 1 -- 172.21.15.60:0/629964795 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680052d0 con 0x7f966c0b1ac0 2023-07-13T22:25:14.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f96627fc700 1 -- 172.21.15.60:0/629964795 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668005430 con 0x7f966c0b1ac0 2023-07-13T22:25:14.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966c157480 con 0x7f966c0b1ac0 2023-07-13T22:25:14.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f96627fc700 1 -- 172.21.15.60:0/629964795 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680038c0 con 0x7f966c0b1ac0 2023-07-13T22:25:14.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.446+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966c157a40 con 0x7f966c0b1ac0 2023-07-13T22:25:14.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.447+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966c04e210 con 0x7f966c0b1ac0 2023-07-13T22:25:14.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.448+0000 7f96627fc700 1 -- 172.21.15.60:0/629964795 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9668003a20 con 0x7f966c0b1ac0 2023-07-13T22:25:14.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.449+0000 7f96627fc700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f965006ace0 0x7f965006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.449+0000 7f96627fc700 1 -- 172.21.15.60:0/629964795 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9668088a40 con 0x7f966c0b1ac0 2023-07-13T22:25:14.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.450+0000 7f967159a700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f965006ace0 0x7f965006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.450+0000 7f967159a700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f965006ace0 0x7f965006d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f966c0d0140 tx=0x7f965c010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.454+0000 7f96627fc700 1 -- 172.21.15.60:0/629964795 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9668053a50 con 0x7f966c0b1ac0 2023-07-13T22:25:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.662+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f966c0d0910 con 0x7f966c0b1ac0 2023-07-13T22:25:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.666+0000 7f96627fc700 1 -- 172.21.15.60:0/629964795 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v23)=0 v23) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9668057070 con 0x7f966c0b1ac0 2023-07-13T22:25:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f965006ace0 msgr2=0x7f965006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f965006ace0 0x7f965006d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f966c0d0140 tx=0x7f965c010000 comp rx=0 tx=0).stop 2023-07-13T22:25:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0b1ac0 msgr2=0x7f966c0ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0b1ac0 0x7f966c0ccd20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f966800c4e0 tx=0x7f966800c8a0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 shutdown_connections 2023-07-13T22:25:14.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f965006ace0 0x7f965006d190 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f966c0b1ac0 0x7f966c0ccd20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f966c0cfd50 0x7f966c0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 --2- 172.21.15.60:0/629964795 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f966c0b8200 0x7f966c0c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 >> 172.21.15.60:0/629964795 conn(0x7f966c06b2f0 msgr2=0x7f966c0d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 shutdown_connections 2023-07-13T22:25:14.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.669+0000 7f9673fff700 1 -- 172.21.15.60:0/629964795 wait complete. 2023-07-13T22:25:14.669 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2023-07-13T22:25:14.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.893+0000 7f6fddbff700 1 Processor -- start 2023-07-13T22:25:14.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.893+0000 7f6fddbff700 1 -- start start 2023-07-13T22:25:14.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.894+0000 7f6fddbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6fd815c790 0x7f6fd80ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.894+0000 7f6fddbff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd80cf6d0 con 0x7f6fd80ac490 2023-07-13T22:25:14.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fddbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd80ac900 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fddbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd80acbe0 con 0x7f6fd815c790 2023-07-13T22:25:14.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fd77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6fd815c790 0x7f6fd80ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fd77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6fd815c790 0x7f6fd80ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59790/0 (socket says 172.21.15.60:59790) 2023-07-13T22:25:14.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fd77fe700 1 -- 172.21.15.60:0/2079963931 learned_addr learned my addr 172.21.15.60:0/2079963931 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:14.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fd77fe700 1 -- 172.21.15.60:0/2079963931 >> v1:172.21.15.105:6789/0 conn(0x7f6fd80ad9a0 legacy=0x7f6fd80adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fd77fe700 1 -- 172.21.15.60:0/2079963931 >> v1:172.21.15.60:6789/0 conn(0x7f6fd80ac490 legacy=0x7f6fd80c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fd77fe700 1 -- 172.21.15.60:0/2079963931 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd80acec0 con 0x7f6fd815c790 2023-07-13T22:25:14.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fd67fc700 1 -- 172.21.15.60:0/2079963931 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763902473 0 0) 0x7f6fd80cf6d0 con 0x7f6fd80ac490 2023-07-13T22:25:14.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.895+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/2079963931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd815c790 0x7f6fd80ce4b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8006fd0 tx=0x7f6fc8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d119011c3e7941 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fd67fc700 1 -- 172.21.15.60:0/2079963931 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8005e30 con 0x7f6fd815c790 2023-07-13T22:25:14.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fd67fc700 1 -- 172.21.15.60:0/2079963931 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc8004030 con 0x7f6fd815c790 2023-07-13T22:25:14.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fd67fc700 1 -- 172.21.15.60:0/2079963931 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc80043e0 con 0x7f6fd815c790 2023-07-13T22:25:14.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fddbff700 1 -- 172.21.15.60:0/2079963931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd815c790 msgr2=0x7f6fd80ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fddbff700 1 --2- 172.21.15.60:0/2079963931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd815c790 0x7f6fd80ce4b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8006fd0 tx=0x7f6fc8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fddbff700 1 -- 172.21.15.60:0/2079963931 shutdown_connections 2023-07-13T22:25:14.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fddbff700 1 --2- 172.21.15.60:0/2079963931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd815c790 0x7f6fd80ce4b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fddbff700 1 -- 172.21.15.60:0/2079963931 >> 172.21.15.60:0/2079963931 conn(0x7f6fd806b900 msgr2=0x7f6fd80cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:14.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fddbff700 1 -- 172.21.15.60:0/2079963931 shutdown_connections 2023-07-13T22:25:14.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.896+0000 7f6fddbff700 1 -- 172.21.15.60:0/2079963931 wait complete. 2023-07-13T22:25:14.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.898+0000 7f6fddbff700 1 Processor -- start 2023-07-13T22:25:14.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.898+0000 7f6fddbff700 1 -- start start 2023-07-13T22:25:14.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.899+0000 7f6fddbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80ac490 0x7f6fd80c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.899+0000 7f6fddbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80ad9a0 0x7f6fd81f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.899+0000 7f6fddbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd815c790 0x7f6fd81f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.899+0000 7f6fddbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80c7fa0 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.899+0000 7f6fddbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80c8130 con 0x7f6fd80ac490 2023-07-13T22:25:14.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.899+0000 7f6fddbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80c5580 con 0x7f6fd815c790 2023-07-13T22:25:14.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.899+0000 7f6fd6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80ad9a0 0x7f6fd81f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.899+0000 7f6fd6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80ad9a0 0x7f6fd81f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59802/0 (socket says 172.21.15.60:59802) 2023-07-13T22:25:14.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fd6ffd700 1 -- 172.21.15.60:0/4135289684 learned_addr learned my addr 172.21.15.60:0/4135289684 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:14.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80ac490 0x7f6fd80c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fd7fff700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd815c790 0x7f6fd81f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fd6ffd700 1 -- 172.21.15.60:0/4135289684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd815c790 msgr2=0x7f6fd81f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fd6ffd700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd815c790 0x7f6fd81f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fd6ffd700 1 -- 172.21.15.60:0/4135289684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80ac490 msgr2=0x7f6fd80c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fd6ffd700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80ac490 0x7f6fd80c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fd6ffd700 1 -- 172.21.15.60:0/4135289684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8009500 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fd6ffd700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80ad9a0 0x7f6fd81f5af0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00c520 tx=0x7f6fcc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fbffff700 1 -- 172.21.15.60:0/4135289684 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0054e0 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fbffff700 1 -- 172.21.15.60:0/4135289684 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc003680 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.900+0000 7f6fbffff700 1 -- 172.21.15.60:0/4135289684 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0039f0 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.901+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd80c5860 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.901+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd80c5e20 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.902+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd804e210 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.903+0000 7f6fbffff700 1 -- 172.21.15.60:0/4135289684 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6fcc003b50 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.903+0000 7f6fbffff700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.904+0000 7f6fbffff700 1 -- 172.21.15.60:0/4135289684 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6fcc088750 con 0x7f6fd80ad9a0 2023-07-13T22:25:14.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.904+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.904+0000 7f6fd77fe700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009c10 tx=0x7f6fc8008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:14.908+0000 7f6fbffff700 1 -- 172.21.15.60:0/4135289684 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6fcc0538a0 con 0x7f6fd80ad9a0 2023-07-13T22:25:15.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.118+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7f6fd80c8270 con 0x7f6fd80ad9a0 2023-07-13T22:25:15.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.122+0000 7f6fbffff700 1 -- 172.21.15.60:0/4135289684 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v24)=0 v24) v1 ==== 119+0+0 (secure 0 0 0) 0x7f6fcc056ec0 con 0x7f6fd80ad9a0 2023-07-13T22:25:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.124+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 msgr2=0x7f6fc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.124+0000 7f6fddbff700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009c10 tx=0x7f6fc8008070 comp rx=0 tx=0).stop 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.124+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80ad9a0 msgr2=0x7f6fd81f5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.124+0000 7f6fddbff700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80ad9a0 0x7f6fd81f5af0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00c520 tx=0x7f6fcc00c830 comp rx=0 tx=0).stop 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.125+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 shutdown_connections 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.125+0000 7f6fddbff700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fc406ace0 0x7f6fc406d190 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.125+0000 7f6fddbff700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd815c790 0x7f6fd81f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.125+0000 7f6fddbff700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd80ac490 0x7f6fd80c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.125+0000 7f6fddbff700 1 --2- 172.21.15.60:0/4135289684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd80ad9a0 0x7f6fd81f5af0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.125+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 >> 172.21.15.60:0/4135289684 conn(0x7f6fd806b900 msgr2=0x7f6fd80ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.125+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 shutdown_connections 2023-07-13T22:25:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.125+0000 7f6fddbff700 1 -- 172.21.15.60:0/4135289684 wait complete. 2023-07-13T22:25:15.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2023-07-13T22:25:15.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.349+0000 7f765bdff700 1 Processor -- start 2023-07-13T22:25:15.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.350+0000 7f765bdff700 1 -- start start 2023-07-13T22:25:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.351+0000 7f765bdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 0x7f76540d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.351+0000 7f765bdff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76540d1870 con 0x7f76540b8200 2023-07-13T22:25:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.351+0000 7f765bdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76540cf370 con 0x7f76540b1ac0 2023-07-13T22:25:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.351+0000 7f765bdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76540d12b0 con 0x7f76540cfd50 2023-07-13T22:25:15.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.351+0000 7f7659b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f76540b8200 0x7f76540cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41044/0 (socket says 172.21.15.60:41044) 2023-07-13T22:25:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.351+0000 7f7659b9b700 1 -- 172.21.15.60:0/4046338927 learned_addr learned my addr 172.21.15.60:0/4046338927 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.351+0000 7f765939a700 1 --2- 172.21.15.60:0/4046338927 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 0x7f76540d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f7658b99700 1 -- 172.21.15.60:0/4046338927 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36713736 0 0) 0x7f76540d1870 con 0x7f76540b8200 2023-07-13T22:25:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f7658b99700 1 -- 172.21.15.60:0/4046338927 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76400035b0 con 0x7f76540b8200 2023-07-13T22:25:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f7658b99700 1 -- 172.21.15.60:0/4046338927 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888133389 0 0) 0x7f76540cf370 con 0x7f76540b1ac0 2023-07-13T22:25:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f7658b99700 1 -- 172.21.15.60:0/4046338927 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76540d1870 con 0x7f76540b1ac0 2023-07-13T22:25:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f765939a700 1 -- 172.21.15.60:0/4046338927 >> v1:172.21.15.105:6789/0 conn(0x7f76540b1ac0 legacy=0x7f76540d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f765939a700 1 -- 172.21.15.60:0/4046338927 >> v1:172.21.15.60:6789/0 conn(0x7f76540b8200 legacy=0x7f76540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f7658b99700 1 -- 172.21.15.60:0/4046338927 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2948402868 0 0) 0x7f76400035b0 con 0x7f76540b8200 2023-07-13T22:25:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f765939a700 1 -- 172.21.15.60:0/4046338927 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540d1590 con 0x7f76540cfd50 2023-07-13T22:25:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f765939a700 1 --2- 172.21.15.60:0/4046338927 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 0x7f76540d23c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f76440098a0 tx=0x7f7644009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d0a9666d03d2d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f7658b99700 1 -- 172.21.15.60:0/4046338927 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650001670 con 0x7f76540cfd50 2023-07-13T22:25:15.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f7658b99700 1 -- 172.21.15.60:0/4046338927 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644004030 con 0x7f76540cfd50 2023-07-13T22:25:15.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.352+0000 7f7658b99700 1 -- 172.21.15.60:0/4046338927 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76440043c0 con 0x7f76540cfd50 2023-07-13T22:25:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.353+0000 7f765bdff700 1 -- 172.21.15.60:0/4046338927 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 msgr2=0x7f76540d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.353+0000 7f765bdff700 1 --2- 172.21.15.60:0/4046338927 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 0x7f76540d23c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f76440098a0 tx=0x7f7644009bb0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.353+0000 7f765bdff700 1 -- 172.21.15.60:0/4046338927 shutdown_connections 2023-07-13T22:25:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.353+0000 7f765bdff700 1 --2- 172.21.15.60:0/4046338927 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 0x7f76540d23c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.353+0000 7f765bdff700 1 -- 172.21.15.60:0/4046338927 >> 172.21.15.60:0/4046338927 conn(0x7f765406b2f0 msgr2=0x7f765406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.353+0000 7f765bdff700 1 -- 172.21.15.60:0/4046338927 shutdown_connections 2023-07-13T22:25:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.353+0000 7f765bdff700 1 -- 172.21.15.60:0/4046338927 wait complete. 2023-07-13T22:25:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.355+0000 7f765bdff700 1 Processor -- start 2023-07-13T22:25:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.355+0000 7f765bdff700 1 -- start start 2023-07-13T22:25:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f765bdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76540b1ac0 0x7f7654152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f765bdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76540b8200 0x7f765415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f765bdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 0x7f76540b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f765bdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7654156e00 con 0x7f76540b1ac0 2023-07-13T22:25:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f765bdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7654153860 con 0x7f76540b8200 2023-07-13T22:25:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f765bdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7654156280 con 0x7f76540cfd50 2023-07-13T22:25:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f7659b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76540b1ac0 0x7f7654152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f7659b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76540b1ac0 0x7f7654152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59806/0 (socket says 172.21.15.60:59806) 2023-07-13T22:25:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f7659b9b700 1 -- 172.21.15.60:0/814592236 learned_addr learned my addr 172.21.15.60:0/814592236 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.356+0000 7f765939a700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76540b8200 0x7f765415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f765a39c700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 0x7f76540b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f7659b9b700 1 -- 172.21.15.60:0/814592236 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 msgr2=0x7f76540b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f7659b9b700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 0x7f76540b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f7659b9b700 1 -- 172.21.15.60:0/814592236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76540b8200 msgr2=0x7f765415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f7659b9b700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76540b8200 0x7f765415eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f7659b9b700 1 -- 172.21.15.60:0/814592236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76440094a0 con 0x7f76540b1ac0 2023-07-13T22:25:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f7659b9b700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76540b1ac0 0x7f7654152ad0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f765000c780 tx=0x7f765000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f764a7fc700 1 -- 172.21.15.60:0/814592236 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650004d40 con 0x7f76540b1ac0 2023-07-13T22:25:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f764a7fc700 1 -- 172.21.15.60:0/814592236 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76500070d0 con 0x7f76540b1ac0 2023-07-13T22:25:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7654156560 con 0x7f76540b1ac0 2023-07-13T22:25:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.357+0000 7f764a7fc700 1 -- 172.21.15.60:0/814592236 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650004ea0 con 0x7f76540b1ac0 2023-07-13T22:25:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.358+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7654156b20 con 0x7f76540b1ac0 2023-07-13T22:25:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.358+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765404e210 con 0x7f76540b1ac0 2023-07-13T22:25:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.359+0000 7f764a7fc700 1 -- 172.21.15.60:0/814592236 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7650007440 con 0x7f76540b1ac0 2023-07-13T22:25:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.360+0000 7f764a7fc700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763806ace0 0x7f763806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.360+0000 7f764a7fc700 1 -- 172.21.15.60:0/814592236 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7650089330 con 0x7f76540b1ac0 2023-07-13T22:25:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.361+0000 7f765939a700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763806ace0 0x7f763806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.361+0000 7f765939a700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763806ace0 0x7f763806d190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f76540d0140 tx=0x7f7644008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.364+0000 7f764a7fc700 1 -- 172.21.15.60:0/814592236 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7650054340 con 0x7f76540b1ac0 2023-07-13T22:25:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.570+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7f76540b4f70 con 0x7f76540b1ac0 2023-07-13T22:25:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.574+0000 7f764a7fc700 1 -- 172.21.15.60:0/814592236 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v25)=0 v25) v1 ==== 128+0+0 (secure 0 0 0) 0x7f7650057960 con 0x7f76540b1ac0 2023-07-13T22:25:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763806ace0 msgr2=0x7f763806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763806ace0 0x7f763806d190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f76540d0140 tx=0x7f7644008030 comp rx=0 tx=0).stop 2023-07-13T22:25:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76540b1ac0 msgr2=0x7f7654152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76540b1ac0 0x7f7654152ad0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f765000c780 tx=0x7f765000ca90 comp rx=0 tx=0).stop 2023-07-13T22:25:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 shutdown_connections 2023-07-13T22:25:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763806ace0 0x7f763806d190 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76540b1ac0 0x7f7654152ad0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76540cfd50 0x7f76540b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 --2- 172.21.15.60:0/814592236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76540b8200 0x7f765415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 >> 172.21.15.60:0/814592236 conn(0x7f765406b2f0 msgr2=0x7f76540732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 shutdown_connections 2023-07-13T22:25:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.577+0000 7f765bdff700 1 -- 172.21.15.60:0/814592236 wait complete. 2023-07-13T22:25:15.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 800 2023-07-13T22:25:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.803+0000 7f687a7ff700 1 Processor -- start 2023-07-13T22:25:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.804+0000 7f687a7ff700 1 -- start start 2023-07-13T22:25:15.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f687a7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f68740add00 0x7f687406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f687a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 0x7f6874069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f687a7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f687406a410 0x7f68740703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f687a7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687406d3b0 con 0x7f687406a410 2023-07-13T22:25:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f687a7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687406a890 con 0x7f68740add00 2023-07-13T22:25:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f687a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68740c8e90 con 0x7f68740ca9f0 2023-07-13T22:25:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f6878d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f687406a410 0x7f68740703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f6878d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f687406a410 0x7f68740703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59822/0 (socket says 172.21.15.60:59822) 2023-07-13T22:25:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f6878d9c700 1 -- 172.21.15.60:0/331598299 learned_addr learned my addr 172.21.15.60:0/331598299 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f6878d9c700 1 -- 172.21.15.60:0/331598299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 msgr2=0x7f6874069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f68737fe700 1 --2- 172.21.15.60:0/331598299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 0x7f6874069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f6873fff700 1 --2- 172.21.15.60:0/331598299 >> v2:172.21.15.105:3300/0 conn(0x7f68740add00 0x7f687406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f6878d9c700 1 --2- 172.21.15.60:0/331598299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 0x7f6874069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f6878d9c700 1 -- 172.21.15.60:0/331598299 >> v2:172.21.15.105:3300/0 conn(0x7f68740add00 msgr2=0x7f687406ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.805+0000 7f6878d9c700 1 --2- 172.21.15.60:0/331598299 >> v2:172.21.15.105:3300/0 conn(0x7f68740add00 0x7f687406ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.806+0000 7f6878d9c700 1 -- 172.21.15.60:0/331598299 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68740ae0d0 con 0x7f687406a410 2023-07-13T22:25:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.806+0000 7f68737fe700 1 --2- 172.21.15.60:0/331598299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 0x7f6874069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.806+0000 7f6873fff700 1 --2- 172.21.15.60:0/331598299 >> v2:172.21.15.105:3300/0 conn(0x7f68740add00 0x7f687406ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.806+0000 7f6878d9c700 1 --2- 172.21.15.60:0/331598299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 0x7f68740703e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f68740cb220 tx=0x7f686c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125da99b3cddedf1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.806+0000 7f6872ffd700 1 -- 172.21.15.60:0/331598299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c00e070 con 0x7f687406a410 2023-07-13T22:25:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.806+0000 7f6872ffd700 1 -- 172.21.15.60:0/331598299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c00b9c0 con 0x7f687406a410 2023-07-13T22:25:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.806+0000 7f6872ffd700 1 -- 172.21.15.60:0/331598299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c00bd70 con 0x7f687406a410 2023-07-13T22:25:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.806+0000 7f687a7ff700 1 -- 172.21.15.60:0/331598299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 msgr2=0x7f68740703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.806+0000 7f687a7ff700 1 --2- 172.21.15.60:0/331598299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 0x7f68740703e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f68740cb220 tx=0x7f686c00b330 comp rx=0 tx=0).stop 2023-07-13T22:25:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.807+0000 7f687a7ff700 1 -- 172.21.15.60:0/331598299 shutdown_connections 2023-07-13T22:25:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.807+0000 7f687a7ff700 1 --2- 172.21.15.60:0/331598299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 0x7f68740703e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.807+0000 7f687a7ff700 1 --2- 172.21.15.60:0/331598299 >> v2:172.21.15.105:3300/0 conn(0x7f68740add00 0x7f687406ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.807+0000 7f687a7ff700 1 --2- 172.21.15.60:0/331598299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 0x7f6874069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.807+0000 7f687a7ff700 1 -- 172.21.15.60:0/331598299 >> 172.21.15.60:0/331598299 conn(0x7f68740cbf50 msgr2=0x7f68740ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.807+0000 7f687a7ff700 1 -- 172.21.15.60:0/331598299 shutdown_connections 2023-07-13T22:25:15.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.807+0000 7f687a7ff700 1 -- 172.21.15.60:0/331598299 wait complete. 2023-07-13T22:25:15.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.808+0000 7f687a7ff700 1 Processor -- start 2023-07-13T22:25:15.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.809+0000 7f687a7ff700 1 -- start start 2023-07-13T22:25:15.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.809+0000 7f687a7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 0x7f687414c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f687a7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68740add00 0x7f6874149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f687a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 0x7f68740c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f687a7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68740c7750 con 0x7f687406a410 2023-07-13T22:25:15.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f687a7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687414a010 con 0x7f68740add00 2023-07-13T22:25:15.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f687a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687414c990 con 0x7f68740ca9f0 2023-07-13T22:25:15.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f6873fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 0x7f687414c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f6873fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 0x7f687414c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59828/0 (socket says 172.21.15.60:59828) 2023-07-13T22:25:15.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f6873fff700 1 -- 172.21.15.60:0/2193843598 learned_addr learned my addr 172.21.15.60:0/2193843598 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:15.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f6873fff700 1 -- 172.21.15.60:0/2193843598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 msgr2=0x7f68740c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f6873fff700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 0x7f68740c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f6873fff700 1 -- 172.21.15.60:0/2193843598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68740add00 msgr2=0x7f6874149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f6873fff700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68740add00 0x7f6874149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f6873fff700 1 -- 172.21.15.60:0/2193843598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c009b30 con 0x7f687406a410 2023-07-13T22:25:15.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.810+0000 7f6873fff700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 0x7f687414c350 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f686400c750 tx=0x7f686400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.811+0000 7f6870ff9700 1 -- 172.21.15.60:0/2193843598 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864011800 con 0x7f687406a410 2023-07-13T22:25:15.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.811+0000 7f6870ff9700 1 -- 172.21.15.60:0/2193843598 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864011960 con 0x7f687406a410 2023-07-13T22:25:15.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.811+0000 7f6870ff9700 1 -- 172.21.15.60:0/2193843598 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864011cd0 con 0x7f687406a410 2023-07-13T22:25:15.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.811+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687414cc70 con 0x7f687406a410 2023-07-13T22:25:15.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.811+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687414d230 con 0x7f687406a410 2023-07-13T22:25:15.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.812+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68740c75b0 con 0x7f687406a410 2023-07-13T22:25:15.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.813+0000 7f6870ff9700 1 -- 172.21.15.60:0/2193843598 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6864021350 con 0x7f687406a410 2023-07-13T22:25:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.814+0000 7f6870ff9700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f686006ace0 0x7f686006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.814+0000 7f6870ff9700 1 -- 172.21.15.60:0/2193843598 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f68640894f0 con 0x7f687406a410 2023-07-13T22:25:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.814+0000 7f68737fe700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f686006ace0 0x7f686006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.815+0000 7f68737fe700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f686006ace0 0x7f686006d190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6868006ba0 tx=0x7f6868008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:15.819+0000 7f6870ff9700 1 -- 172.21.15.60:0/2193843598 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f68740c75b0 con 0x7f687406a410 2023-07-13T22:25:16.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.025+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7f68740c75b0 con 0x7f687406a410 2023-07-13T22:25:16.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.029+0000 7f6870ff9700 1 -- 172.21.15.60:0/2193843598 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v26)=0 v26) v1 ==== 124+0+0 (secure 0 0 0) 0x7f68740c75b0 con 0x7f687406a410 2023-07-13T22:25:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f686006ace0 msgr2=0x7f686006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f686006ace0 0x7f686006d190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6868006ba0 tx=0x7f6868008040 comp rx=0 tx=0).stop 2023-07-13T22:25:16.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 msgr2=0x7f687414c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 0x7f687414c350 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f686400c750 tx=0x7f686400cb10 comp rx=0 tx=0).stop 2023-07-13T22:25:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 shutdown_connections 2023-07-13T22:25:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f686006ace0 0x7f686006d190 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f687406a410 0x7f687414c350 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68740ca9f0 0x7f68740c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 --2- 172.21.15.60:0/2193843598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68740add00 0x7f6874149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 >> 172.21.15.60:0/2193843598 conn(0x7f68740cbf50 msgr2=0x7f68741e4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 shutdown_connections 2023-07-13T22:25:16.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.032+0000 7f687a7ff700 1 -- 172.21.15.60:0/2193843598 wait complete. 2023-07-13T22:25:16.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-13T22:25:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.256+0000 7fbab4dff700 1 Processor -- start 2023-07-13T22:25:16.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.256+0000 7fbab4dff700 1 -- start start 2023-07-13T22:25:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbab4dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab015c530 0x7fbab00b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbab4dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbab00cfbc0 0x7fbab0165eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbab4dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab00cefd0 con 0x7fbab00d11c0 2023-07-13T22:25:16.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbab4dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab00d0020 con 0x7fbab00cfbc0 2023-07-13T22:25:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbab4dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab00d1780 con 0x7fbab015c530 2023-07-13T22:25:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbaa7fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbab00d11c0 0x7fbab00cf4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41046/0 (socket says 172.21.15.60:41046) 2023-07-13T22:25:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbaa7fff700 1 -- 172.21.15.60:0/4026083361 learned_addr learned my addr 172.21.15.60:0/4026083361 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbaaed9d700 1 --2- 172.21.15.60:0/4026083361 >> v2:172.21.15.60:3300/0 conn(0x7fbab00cfbc0 0x7fbab0165eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbaae59c700 1 --2- 172.21.15.60:0/4026083361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab015c530 0x7fbab00b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbaadd9b700 1 -- 172.21.15.60:0/4026083361 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182691163 0 0) 0x7fbab00cefd0 con 0x7fbab00d11c0 2023-07-13T22:25:16.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.258+0000 7fbaadd9b700 1 -- 172.21.15.60:0/4026083361 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba9c0035b0 con 0x7fbab00d11c0 2023-07-13T22:25:16.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.259+0000 7fbaaed9d700 1 -- 172.21.15.60:0/4026083361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab015c530 msgr2=0x7fbab00b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:16.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.259+0000 7fbaaed9d700 1 --2- 172.21.15.60:0/4026083361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab015c530 0x7fbab00b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.259+0000 7fbaaed9d700 1 -- 172.21.15.60:0/4026083361 >> v1:172.21.15.60:6789/0 conn(0x7fbab00d11c0 legacy=0x7fbab00cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.259+0000 7fbaaed9d700 1 -- 172.21.15.60:0/4026083361 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab00cd2d0 con 0x7fbab00cfbc0 2023-07-13T22:25:16.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.259+0000 7fbaaed9d700 1 --2- 172.21.15.60:0/4026083361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00cfbc0 0x7fbab0165eb0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009d90 tx=0x7fbaa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a704e699cbb2c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:16.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.259+0000 7fbaadd9b700 1 -- 172.21.15.60:0/4026083361 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800e070 con 0x7fbab00cfbc0 2023-07-13T22:25:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.259+0000 7fbaadd9b700 1 -- 172.21.15.60:0/4026083361 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa800bb20 con 0x7fbab00cfbc0 2023-07-13T22:25:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.259+0000 7fbaadd9b700 1 -- 172.21.15.60:0/4026083361 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8012430 con 0x7fbab00cfbc0 2023-07-13T22:25:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.260+0000 7fbab4dff700 1 -- 172.21.15.60:0/4026083361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00cfbc0 msgr2=0x7fbab0165eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.260+0000 7fbab4dff700 1 --2- 172.21.15.60:0/4026083361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00cfbc0 0x7fbab0165eb0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009d90 tx=0x7fbaa800b330 comp rx=0 tx=0).stop 2023-07-13T22:25:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.260+0000 7fbab4dff700 1 -- 172.21.15.60:0/4026083361 shutdown_connections 2023-07-13T22:25:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.260+0000 7fbab4dff700 1 --2- 172.21.15.60:0/4026083361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00cfbc0 0x7fbab0165eb0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.260+0000 7fbab4dff700 1 --2- 172.21.15.60:0/4026083361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab015c530 0x7fbab00b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.260+0000 7fbab4dff700 1 -- 172.21.15.60:0/4026083361 >> 172.21.15.60:0/4026083361 conn(0x7fbab006b400 msgr2=0x7fbab006b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:16.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.260+0000 7fbab4dff700 1 -- 172.21.15.60:0/4026083361 shutdown_connections 2023-07-13T22:25:16.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.260+0000 7fbab4dff700 1 -- 172.21.15.60:0/4026083361 wait complete. 2023-07-13T22:25:16.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.261+0000 7fbab4dff700 1 Processor -- start 2023-07-13T22:25:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.262+0000 7fbab4dff700 1 -- start start 2023-07-13T22:25:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbab4dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab00cfbc0 0x7fbab01eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbab4dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00d11c0 0x7fbab015e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbab4dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab015e820 0x7fbab01f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbab4dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab01f2680 con 0x7fbab00d11c0 2023-07-13T22:25:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbab4dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab01f2960 con 0x7fbab015e820 2023-07-13T22:25:16.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbab4dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab015dae0 con 0x7fbab00cfbc0 2023-07-13T22:25:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaa7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00d11c0 0x7fbab015e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaa7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00d11c0 0x7fbab015e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59834/0 (socket says 172.21.15.60:59834) 2023-07-13T22:25:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaa7fff700 1 -- 172.21.15.60:0/1848572679 learned_addr learned my addr 172.21.15.60:0/1848572679 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaae59c700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab00cfbc0 0x7fbab01eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaa7fff700 1 -- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab00cfbc0 msgr2=0x7fbab01eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaaed9d700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab015e820 0x7fbab01f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaa7fff700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab00cfbc0 0x7fbab01eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaa7fff700 1 -- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab015e820 msgr2=0x7fbab01f1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaa7fff700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab015e820 0x7fbab01f1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaa7fff700 1 -- 172.21.15.60:0/1848572679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa8009a70 con 0x7fbab00d11c0 2023-07-13T22:25:16.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaae59c700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab00cfbc0 0x7fbab01eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:16.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.263+0000 7fbaaed9d700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab015e820 0x7fbab01f1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:16.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.264+0000 7fbaa7fff700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00d11c0 0x7fbab015e2e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fba9800c700 tx=0x7fba9800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:16.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.264+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/1848572679 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98005d70 con 0x7fbab00d11c0 2023-07-13T22:25:16.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.264+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/1848572679 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba980070d0 con 0x7fbab00d11c0 2023-07-13T22:25:16.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.264+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab015ddc0 con 0x7fbab00d11c0 2023-07-13T22:25:16.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.264+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/1848572679 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98007460 con 0x7fbab00d11c0 2023-07-13T22:25:16.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.264+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab01ef470 con 0x7fbab00d11c0 2023-07-13T22:25:16.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.265+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab015dfe0 con 0x7fbab00d11c0 2023-07-13T22:25:16.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.266+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/1848572679 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba980075c0 con 0x7fbab00d11c0 2023-07-13T22:25:16.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.267+0000 7fbaa6ffd700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba9006ace0 0x7fba9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.267+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/1848572679 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba98088b30 con 0x7fbab00d11c0 2023-07-13T22:25:16.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.267+0000 7fbaae59c700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba9006ace0 0x7fba9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.268+0000 7fbaae59c700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba9006ace0 0x7fba9006d190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009b60 tx=0x7fbaa0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:16.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.271+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/1848572679 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fba98053b50 con 0x7fbab00d11c0 2023-07-13T22:25:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.480+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7fbab004e210 con 0x7fbab00d11c0 2023-07-13T22:25:16.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.481+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/1848572679 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v26) v1 ==== 106+0+4 (secure 0 0 0) 0x7fba98057170 con 0x7fbab00d11c0 2023-07-13T22:25:16.463 INFO:teuthology.orchestra.run.smithi060.stdout:100 2023-07-13T22:25:16.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba9006ace0 msgr2=0x7fba9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba9006ace0 0x7fba9006d190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009b60 tx=0x7fbaa0008040 comp rx=0 tx=0).stop 2023-07-13T22:25:16.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00d11c0 msgr2=0x7fbab015e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00d11c0 0x7fbab015e2e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fba9800c700 tx=0x7fba9800cac0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 shutdown_connections 2023-07-13T22:25:16.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba9006ace0 0x7fba9006d190 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab015e820 0x7fbab01f1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbab00cfbc0 0x7fbab01eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 --2- 172.21.15.60:0/1848572679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbab00d11c0 0x7fbab015e2e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 >> 172.21.15.60:0/1848572679 conn(0x7fbab006b400 msgr2=0x7fbab015a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:16.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 shutdown_connections 2023-07-13T22:25:16.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.483+0000 7fbab4dff700 1 -- 172.21.15.60:0/1848572679 wait complete. 2023-07-13T22:25:16.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-07-13T22:25:16.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.707+0000 7f1b9dbff700 1 Processor -- start 2023-07-13T22:25:16.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.707+0000 7f1b9dbff700 1 -- start start 2023-07-13T22:25:16.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.708+0000 7f1b9dbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1b9815c530 0x7f1b980b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.708+0000 7f1b9dbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1b980cfbc0 0x7f1b98165eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.708+0000 7f1b9dbff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b980cffe0 con 0x7f1b980d11c0 2023-07-13T22:25:16.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.708+0000 7f1b9dbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980cefd0 con 0x7f1b980cfbc0 2023-07-13T22:25:16.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b9dbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980d1780 con 0x7f1b9815c530 2023-07-13T22:25:16.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b96ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1b980d11c0 0x7f1b980cf4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41062/0 (socket says 172.21.15.60:41062) 2023-07-13T22:25:16.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b96ffd700 1 -- 172.21.15.60:0/2265779523 learned_addr learned my addr 172.21.15.60:0/2265779523 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:16.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b97fff700 1 --2- 172.21.15.60:0/2265779523 >> v2:172.21.15.60:3300/0 conn(0x7f1b980cfbc0 0x7f1b98165eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b977fe700 1 --2- 172.21.15.60:0/2265779523 >> v2:172.21.15.105:3300/0 conn(0x7f1b9815c530 0x7f1b980b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b97fff700 1 -- 172.21.15.60:0/2265779523 >> v2:172.21.15.105:3300/0 conn(0x7f1b9815c530 msgr2=0x7f1b980b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b97fff700 1 --2- 172.21.15.60:0/2265779523 >> v2:172.21.15.105:3300/0 conn(0x7f1b9815c530 0x7f1b980b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b97fff700 1 -- 172.21.15.60:0/2265779523 >> v1:172.21.15.60:6789/0 conn(0x7f1b980d11c0 legacy=0x7f1b980cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b97fff700 1 -- 172.21.15.60:0/2265779523 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b980cd2d0 con 0x7f1b980cfbc0 2023-07-13T22:25:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.709+0000 7f1b967fc700 1 -- 172.21.15.60:0/2265779523 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82085571 0 0) 0x7f1b980cffe0 con 0x7f1b980d11c0 2023-07-13T22:25:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.710+0000 7f1b97fff700 1 --2- 172.21.15.60:0/2265779523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980cfbc0 0x7f1b98165eb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1b980d0760 tx=0x7f1b9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891078060c301bdf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.710+0000 7f1b967fc700 1 -- 172.21.15.60:0/2265779523 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000e070 con 0x7f1b980cfbc0 2023-07-13T22:25:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.710+0000 7f1b967fc700 1 -- 172.21.15.60:0/2265779523 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9000bb20 con 0x7f1b980cfbc0 2023-07-13T22:25:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.710+0000 7f1b967fc700 1 -- 172.21.15.60:0/2265779523 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90012430 con 0x7f1b980cfbc0 2023-07-13T22:25:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.710+0000 7f1b9dbff700 1 -- 172.21.15.60:0/2265779523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980cfbc0 msgr2=0x7f1b98165eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.710+0000 7f1b9dbff700 1 --2- 172.21.15.60:0/2265779523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980cfbc0 0x7f1b98165eb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1b980d0760 tx=0x7f1b9000b330 comp rx=0 tx=0).stop 2023-07-13T22:25:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.710+0000 7f1b9dbff700 1 -- 172.21.15.60:0/2265779523 shutdown_connections 2023-07-13T22:25:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.710+0000 7f1b9dbff700 1 --2- 172.21.15.60:0/2265779523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980cfbc0 0x7f1b98165eb0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.711+0000 7f1b9dbff700 1 --2- 172.21.15.60:0/2265779523 >> v2:172.21.15.105:3300/0 conn(0x7f1b9815c530 0x7f1b980b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.711+0000 7f1b9dbff700 1 -- 172.21.15.60:0/2265779523 >> 172.21.15.60:0/2265779523 conn(0x7f1b9806b400 msgr2=0x7f1b9806b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.711+0000 7f1b9dbff700 1 -- 172.21.15.60:0/2265779523 shutdown_connections 2023-07-13T22:25:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.711+0000 7f1b9dbff700 1 -- 172.21.15.60:0/2265779523 wait complete. 2023-07-13T22:25:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.712+0000 7f1b9dbff700 1 Processor -- start 2023-07-13T22:25:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.712+0000 7f1b9dbff700 1 -- start start 2023-07-13T22:25:16.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.713+0000 7f1b9dbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b980cfbc0 0x7f1b9814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.713+0000 7f1b9dbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980d11c0 0x7f1b9815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.713+0000 7f1b9dbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b9815e800 0x7f1b9815c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.713+0000 7f1b9dbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9815f1a0 con 0x7f1b980d11c0 2023-07-13T22:25:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.713+0000 7f1b9dbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9815d8b0 con 0x7f1b980cfbc0 2023-07-13T22:25:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.713+0000 7f1b9dbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9815db90 con 0x7f1b9815e800 2023-07-13T22:25:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b96ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980d11c0 0x7f1b9815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b96ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980d11c0 0x7f1b9815e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59844/0 (socket says 172.21.15.60:59844) 2023-07-13T22:25:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b96ffd700 1 -- 172.21.15.60:0/4221290924 learned_addr learned my addr 172.21.15.60:0/4221290924 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b97fff700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b9815e800 0x7f1b9815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b977fe700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b980cfbc0 0x7f1b9814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b96ffd700 1 -- 172.21.15.60:0/4221290924 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b9815e800 msgr2=0x7f1b9815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b96ffd700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b9815e800 0x7f1b9815c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b96ffd700 1 -- 172.21.15.60:0/4221290924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b980cfbc0 msgr2=0x7f1b9814e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b96ffd700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b980cfbc0 0x7f1b9814e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b96ffd700 1 -- 172.21.15.60:0/4221290924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90009a70 con 0x7f1b980d11c0 2023-07-13T22:25:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.714+0000 7f1b96ffd700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980d11c0 0x7f1b9815e2c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0011e0 tx=0x7f1b8c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.715+0000 7f1b7ffff700 1 -- 172.21.15.60:0/4221290924 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c004c80 con 0x7f1b980d11c0 2023-07-13T22:25:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.715+0000 7f1b7ffff700 1 -- 172.21.15.60:0/4221290924 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c004e00 con 0x7f1b980d11c0 2023-07-13T22:25:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.715+0000 7f1b7ffff700 1 -- 172.21.15.60:0/4221290924 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c007310 con 0x7f1b980d11c0 2023-07-13T22:25:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.715+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9815edc0 con 0x7f1b980d11c0 2023-07-13T22:25:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.715+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9815f5c0 con 0x7f1b980d11c0 2023-07-13T22:25:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.716+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9815f2e0 con 0x7f1b980d11c0 2023-07-13T22:25:16.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.717+0000 7f1b7ffff700 1 -- 172.21.15.60:0/4221290924 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b8c007470 con 0x7f1b980d11c0 2023-07-13T22:25:16.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.718+0000 7f1b7ffff700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b8406ace0 0x7f1b8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:16.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.718+0000 7f1b7ffff700 1 -- 172.21.15.60:0/4221290924 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b8c0890c0 con 0x7f1b980d11c0 2023-07-13T22:25:16.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.718+0000 7f1b977fe700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b8406ace0 0x7f1b8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:16.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.719+0000 7f1b977fe700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b8406ace0 0x7f1b8406d190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1b980c9290 tx=0x7f1b88009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:16.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.722+0000 7f1b7ffff700 1 -- 172.21.15.60:0/4221290924 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1b9815f2e0 con 0x7f1b980d11c0 2023-07-13T22:25:16.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.931+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7f1b9815f2e0 con 0x7f1b980d11c0 2023-07-13T22:25:16.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.931+0000 7f1b7ffff700 1 -- 172.21.15.60:0/4221290924 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v26) v1 ==== 112+0+10 (secure 0 0 0) 0x7f1b9815f2e0 con 0x7f1b980d11c0 2023-07-13T22:25:16.914 INFO:teuthology.orchestra.run.smithi060.stdout:60.000000 2023-07-13T22:25:16.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b8406ace0 msgr2=0x7f1b8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b8406ace0 0x7f1b8406d190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1b980c9290 tx=0x7f1b88009210 comp rx=0 tx=0).stop 2023-07-13T22:25:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980d11c0 msgr2=0x7f1b9815e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980d11c0 0x7f1b9815e2c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0011e0 tx=0x7f1b8c00ca30 comp rx=0 tx=0).stop 2023-07-13T22:25:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 shutdown_connections 2023-07-13T22:25:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b8406ace0 0x7f1b8406d190 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b9815e800 0x7f1b9815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b980cfbc0 0x7f1b9814e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 --2- 172.21.15.60:0/4221290924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b980d11c0 0x7f1b9815e2c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 >> 172.21.15.60:0/4221290924 conn(0x7f1b9806b400 msgr2=0x7f1b9815a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 shutdown_connections 2023-07-13T22:25:16.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:16.934+0000 7f1b9dbff700 1 -- 172.21.15.60:0/4221290924 wait complete. 2023-07-13T22:25:16.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:25:16.935 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:25:16.935 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:25:16.935 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:25:16.936 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:25:16.936 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:16.936 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:25:16.936 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:25:16.937 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:25:16.937 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:25:16.937 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:16.938 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:25:16.938 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:25:16.938 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:25:16.939 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:25:16.939 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:16.939 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:25:16.939 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:25:16.940 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:25:16.940 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:25:16.940 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:25:16.941 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:25:16.941 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:25:16.941 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:25:16.942 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:16.942 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:25:16.942 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:25:17.088 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.141735.asok 2023-07-13T22:25:17.265 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.141735.asok 2023-07-13T22:25:17.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:25:17.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.141735.asok mds_sessions 2023-07-13T22:25:17.496 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:25:17.496 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6906, 2023-07-13T22:25:17.496 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:25:17.496 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:25:17.497 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:25:17.497 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6906 2023-07-13T22:25:17.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:25:17.497 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:25:17.497 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:25:17.497 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:25:17.497 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1059118983 2023-07-13T22:25:17.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:25:17.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:25:17.498 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6906 v1:192.168.0.1:0/1059118983", 2023-07-13T22:25:17.498 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "v1:192.168.0.1:0/1059118983", 2023-07-13T22:25:17.498 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions": [ 2023-07-13T22:25:17.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:25:17.498 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": 0, 2023-07-13T22:25:17.498 INFO:teuthology.orchestra.run.smithi060.stdout: "addrs": { 2023-07-13T22:25:17.498 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2023-07-13T22:25:17.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:25:17.499 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2023-07-13T22:25:17.499 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6835", 2023-07-13T22:25:17.499 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3617406711 2023-07-13T22:25:17.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:25:17.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:25:17.499 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:25:17.499 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6837", 2023-07-13T22:25:17.500 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3617406711 2023-07-13T22:25:17.500 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:25:17.500 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:25:17.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:25:17.500 INFO:teuthology.orchestra.run.smithi060.stdout: "seq": 0, 2023-07-13T22:25:17.500 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_gen": 0, 2023-07-13T22:25:17.500 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_ttl": "2023-07-13T22:26:12.101221+0000", 2023-07-13T22:25:17.501 INFO:teuthology.orchestra.run.smithi060.stdout: "last_cap_renew_request": "2023-07-13T22:25:12.101221+0000", 2023-07-13T22:25:17.501 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_renew_seq": 2, 2023-07-13T22:25:17.501 INFO:teuthology.orchestra.run.smithi060.stdout: "num_caps": 1, 2023-07-13T22:25:17.501 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "open" 2023-07-13T22:25:17.501 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:25:17.501 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2023-07-13T22:25:17.501 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 41 2023-07-13T22:25:17.501 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:25:17.510 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:25:17.510 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:25:17.556 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:25:17.556 DEBUG:teuthology.orchestra.run.smithi060:> import sys 2023-07-13T22:25:17.557 DEBUG:teuthology.orchestra.run.smithi060:> import time 2023-07-13T22:25:17.557 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:25:17.557 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:17.558 DEBUG:teuthology.orchestra.run.smithi060:> n = 400 2023-07-13T22:25:17.558 DEBUG:teuthology.orchestra.run.smithi060:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-07-13T22:25:17.558 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:17.558 DEBUG:teuthology.orchestra.run.smithi060:> if not os.path.exists(abs_path): 2023-07-13T22:25:17.559 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(abs_path) 2023-07-13T22:25:17.559 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:17.559 DEBUG:teuthology.orchestra.run.smithi060:> handles = [] 2023-07-13T22:25:17.559 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(0, n): 2023-07-13T22:25:17.560 DEBUG:teuthology.orchestra.run.smithi060:> fname = "file_"+str(i) 2023-07-13T22:25:17.560 DEBUG:teuthology.orchestra.run.smithi060:> path = os.path.join(abs_path, fname) 2023-07-13T22:25:17.560 DEBUG:teuthology.orchestra.run.smithi060:> handles.append(open(path, '"'"'w'"'"')) 2023-07-13T22:25:17.560 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:25:17.561 DEBUG:teuthology.orchestra.run.smithi060:> while True: 2023-07-13T22:25:17.561 DEBUG:teuthology.orchestra.run.smithi060:> time.sleep(1) 2023-07-13T22:25:17.561 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:25:17.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:25:17.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.828+0000 7effc4fff700 1 Processor -- start 2023-07-13T22:25:17.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.828+0000 7effc4fff700 1 -- start start 2023-07-13T22:25:17.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.830+0000 7effc4fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7effc00c9a10 0x7effc006a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:17.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.830+0000 7effc4fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc006a860 con 0x7effc0158870 2023-07-13T22:25:17.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.830+0000 7effc4fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc00c3fc0 con 0x7effc0069ed0 2023-07-13T22:25:17.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.830+0000 7effc4fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc00c42a0 con 0x7effc00c9a10 2023-07-13T22:25:17.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.830+0000 7effbe59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7effc0158870 0x7effc00c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41076/0 (socket says 172.21.15.60:41076) 2023-07-13T22:25:17.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.830+0000 7effbe59c700 1 -- 172.21.15.60:0/1028433136 learned_addr learned my addr 172.21.15.60:0/1028433136 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:17.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.830+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794849732 0 0) 0x7effb0001110 con 0x7effc0158870 2023-07-13T22:25:17.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.830+0000 7effbed9d700 1 --2- 172.21.15.60:0/1028433136 >> v2:172.21.15.105:3300/0 conn(0x7effc00c9a10 0x7effc006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:17.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.830+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa80035b0 con 0x7effc0158870 2023-07-13T22:25:17.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651233435 0 0) 0x7effc00c3fc0 con 0x7effc0069ed0 2023-07-13T22:25:17.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effb0001110 con 0x7effc0069ed0 2023-07-13T22:25:17.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3289025650 0 0) 0x7effa80035b0 con 0x7effc0158870 2023-07-13T22:25:17.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 >> v2:172.21.15.105:3300/0 conn(0x7effc00c9a10 msgr2=0x7effc006a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:17.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 --2- 172.21.15.60:0/1028433136 >> v2:172.21.15.105:3300/0 conn(0x7effc00c9a10 0x7effc006a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 >> v1:172.21.15.105:6789/0 conn(0x7effc0069ed0 legacy=0x7effc00c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc00c4580 con 0x7effc0158870 2023-07-13T22:25:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7effb0001780 con 0x7effc0158870 2023-07-13T22:25:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 >> v1:172.21.15.60:6789/0 conn(0x7effc0158870 legacy=0x7effc00c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effa8003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa8004850 0x7effa8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa8005250 0x7effa80059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc00c42a0 con 0x7effa8003be0 2023-07-13T22:25:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa8006150 con 0x7effa8005250 2023-07-13T22:25:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa8006310 con 0x7effa8004850 2023-07-13T22:25:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.831+0000 7effbe59c700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effa8003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbed9d700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa8005250 0x7effa80059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbdd9b700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa8004850 0x7effa8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbe59c700 1 -- 172.21.15.60:0/1028433136 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa8004850 msgr2=0x7effa8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbe59c700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa8004850 0x7effa8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:17.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbe59c700 1 -- 172.21.15.60:0/1028433136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa8005250 msgr2=0x7effa80059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:17.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbe59c700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa8005250 0x7effa80059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:17.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbe59c700 1 -- 172.21.15.60:0/1028433136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80035b0 con 0x7effa8003be0 2023-07-13T22:25:17.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbed9d700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa8005250 0x7effa80059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:17.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbed9d700 1 -- 172.21.15.60:0/1028433136 reap_dead start 2023-07-13T22:25:17.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbe59c700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effa8003fd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7effb0007fb0 tx=0x7effa8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405821e632d3c715 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:17.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.832+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0007c10 con 0x7effa8003be0 2023-07-13T22:25:17.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.833+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb000cc50 con 0x7effa8003be0 2023-07-13T22:25:17.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.833+0000 7effbd59a700 1 -- 172.21.15.60:0/1028433136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0005960 con 0x7effa8003be0 2023-07-13T22:25:17.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.833+0000 7effc4fff700 1 -- 172.21.15.60:0/1028433136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 msgr2=0x7effa8003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:17.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.833+0000 7effc4fff700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effa8003fd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7effb0007fb0 tx=0x7effa8005880 comp rx=0 tx=0).stop 2023-07-13T22:25:17.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.833+0000 7effc4fff700 1 -- 172.21.15.60:0/1028433136 shutdown_connections 2023-07-13T22:25:17.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.833+0000 7effc4fff700 1 --2- 172.21.15.60:0/1028433136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effa8003fd0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:17.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.833+0000 7effc4fff700 1 -- 172.21.15.60:0/1028433136 >> 172.21.15.60:0/1028433136 conn(0x7effc00cf410 msgr2=0x7effc00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:17.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.833+0000 7effc4fff700 1 -- 172.21.15.60:0/1028433136 shutdown_connections 2023-07-13T22:25:17.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.833+0000 7effc4fff700 1 -- 172.21.15.60:0/1028433136 wait complete. 2023-07-13T22:25:17.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.836+0000 7effc4fff700 1 Processor -- start 2023-07-13T22:25:17.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.836+0000 7effc4fff700 1 -- start start 2023-07-13T22:25:17.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effc4fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effc014c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:17.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effc4fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effc00b16d0 0x7effc00b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:17.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effc4fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc00b0e60 0x7effc00c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:17.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effc4fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc00b12e0 con 0x7effa8003be0 2023-07-13T22:25:17.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effc4fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc01559b0 con 0x7effc00b0e60 2023-07-13T22:25:17.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effc4fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc0155c90 con 0x7effc00b16d0 2023-07-13T22:25:17.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effbe59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effc014c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:17.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effbe59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effc014c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59860/0 (socket says 172.21.15.60:59860) 2023-07-13T22:25:17.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effbe59c700 1 -- 172.21.15.60:0/559331966 learned_addr learned my addr 172.21.15.60:0/559331966 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:17.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.837+0000 7effbdd9b700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effc00b16d0 0x7effc00b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:17.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effbed9d700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc00b0e60 0x7effc00c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:17.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effbe59c700 1 -- 172.21.15.60:0/559331966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effc00b16d0 msgr2=0x7effc00b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:17.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effbe59c700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effc00b16d0 0x7effc00b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:17.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effbe59c700 1 -- 172.21.15.60:0/559331966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc00b0e60 msgr2=0x7effc00c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:17.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effbe59c700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc00b0e60 0x7effc00c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:17.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effbe59c700 1 -- 172.21.15.60:0/559331966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc0155f70 con 0x7effa8003be0 2023-07-13T22:25:17.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effbe59c700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effc014c7d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7effa8006120 tx=0x7effb0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:17.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effaeffd700 1 -- 172.21.15.60:0/559331966 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0001540 con 0x7effa8003be0 2023-07-13T22:25:17.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effaeffd700 1 -- 172.21.15.60:0/559331966 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb00016a0 con 0x7effa8003be0 2023-07-13T22:25:17.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.838+0000 7effaeffd700 1 -- 172.21.15.60:0/559331966 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0007310 con 0x7effa8003be0 2023-07-13T22:25:17.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.839+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc0156250 con 0x7effa8003be0 2023-07-13T22:25:17.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.839+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc00b1b40 con 0x7effa8003be0 2023-07-13T22:25:17.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.840+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc00b1260 con 0x7effa8003be0 2023-07-13T22:25:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.841+0000 7effaeffd700 1 -- 172.21.15.60:0/559331966 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effb0007470 con 0x7effa8003be0 2023-07-13T22:25:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.842+0000 7effaeffd700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effa006ace0 0x7effa006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.842+0000 7effaeffd700 1 -- 172.21.15.60:0/559331966 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effb008a340 con 0x7effa8003be0 2023-07-13T22:25:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.842+0000 7effbdd9b700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effa006ace0 0x7effa006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.843+0000 7effbdd9b700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effa006ace0 0x7effa006d190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7effb4007fd0 tx=0x7effb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:17.847+0000 7effaeffd700 1 -- 172.21.15.60:0/559331966 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7effb0055360 con 0x7effa8003be0 2023-07-13T22:25:18.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.217+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effc006a860 con 0x7effa8003be0 2023-07-13T22:25:18.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.218+0000 7effaeffd700 1 -- 172.21.15.60:0/559331966 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4717 (secure 0 0 0) 0x7effc006a860 con 0x7effa8003be0 2023-07-13T22:25:18.200 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:18.200 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1603865496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1603865496},{"type":"v1","addr":"172.21.15.105:6834","nonce":1603865496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6735,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1777842000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1777842000},{"type":"v1","addr":"172.21.15.105:6835","nonce":1777842000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6738,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2613179178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2613179178},{"type":"v1","addr":"172.21.15.60:6836","nonce":2613179178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:25:02.392864+0000","modified":"2023-07-13T22:25:05.896853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/3617406711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3617406711},{"type":"v1","addr":"172.21.15.60:6837","nonce":3617406711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-13T22:25:18.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effa006ace0 msgr2=0x7effa006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:18.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effa006ace0 0x7effa006d190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7effb4007fd0 tx=0x7effb4009040 comp rx=0 tx=0).stop 2023-07-13T22:25:18.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 msgr2=0x7effc014c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:18.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effc014c7d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7effa8006120 tx=0x7effb0001a40 comp rx=0 tx=0).stop 2023-07-13T22:25:18.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 shutdown_connections 2023-07-13T22:25:18.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7effa006ace0 0x7effa006d190 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:18.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8003be0 0x7effc014c7d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:18.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc00b0e60 0x7effc00c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:18.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 --2- 172.21.15.60:0/559331966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effc00b16d0 0x7effc00b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:18.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.220+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 >> 172.21.15.60:0/559331966 conn(0x7effc00cf410 msgr2=0x7effc014e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:18.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.221+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 shutdown_connections 2023-07-13T22:25:18.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:18.221+0000 7effc4fff700 1 -- 172.21.15.60:0/559331966 wait complete. 2023-07-13T22:25:18.205 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 41 2023-07-13T22:25:18.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-13T22:25:18.487 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":6017,"entity":{"name":{"type":"client","num":6017},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3775652031}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":5.0409458540000003,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.984946230996163,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6017 192.168.0.1:0/3775652031","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"123518","root":"/"}},{"id":6906,"entity":{"name":{"type":"client","num":6906},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1059118983}},"state":"open","num_leases":1,"num_caps":374,"request_load_avg":374,"uptime":7.3869206539999999,"requests_in_flight":372,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":749.33290028213185,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6906 192.168.0.1:0/1059118983","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"141735","root":"/"}}] 2023-07-13T22:25:18.488 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "123518", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3775652031, "type": "any" }, "name": { "num": 6017, "type": "client" } }, "id": 6017, "inst": "client.6017 192.168.0.1:0/3775652031", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.984946230996163 }, "state": "open", "uptime": 5.040945854 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "141735", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1059118983, "type": "any" }, "name": { "num": 6906, "type": "client" } }, "id": 6906, "inst": "client.6906 192.168.0.1:0/1059118983", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 374, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 374, "requests_in_flight": 372, "session_cache_liveness": { "halflife": 300, "value": 749.3329002821318 }, "state": "open", "uptime": 7.386920654 } ] 2023-07-13T22:25:18.488 DEBUG:tasks.ceph_test_case:wait_until_equal: 374 != 402, waiting (timeout=600)... 2023-07-13T22:25:23.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:25:23.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.722+0000 7faa04fff700 1 Processor -- start 2023-07-13T22:25:23.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.722+0000 7faa04fff700 1 -- start start 2023-07-13T22:25:23.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.723+0000 7faa04fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa000b8570 0x7faa000ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:23.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.723+0000 7faa04fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa000d35c0 0x7faa000d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:23.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.723+0000 7faa04fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa000d09d0 con 0x7faa000cf3f0 2023-07-13T22:25:23.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.723+0000 7faa04fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa000d3b30 con 0x7faa000b8570 2023-07-13T22:25:23.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.723+0000 7faa04fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa000cda10 con 0x7faa000d35c0 2023-07-13T22:25:23.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.723+0000 7fa9fe59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa000b8570 0x7faa000ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:23.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.723+0000 7fa9fed9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faa000cf3f0 0x7faa0015fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41086/0 (socket says 172.21.15.60:41086) 2023-07-13T22:25:23.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.723+0000 7fa9fed9d700 1 -- 172.21.15.60:0/3358503999 learned_addr learned my addr 172.21.15.60:0/3358503999 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:23.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9f7fff700 1 --2- 172.21.15.60:0/3358503999 >> v2:172.21.15.105:3300/0 conn(0x7faa000d35c0 0x7faa000d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:23.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9fe59c700 1 -- 172.21.15.60:0/3358503999 >> v2:172.21.15.105:3300/0 conn(0x7faa000d35c0 msgr2=0x7faa000d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:23.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9fe59c700 1 --2- 172.21.15.60:0/3358503999 >> v2:172.21.15.105:3300/0 conn(0x7faa000d35c0 0x7faa000d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:23.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9fe59c700 1 -- 172.21.15.60:0/3358503999 >> v1:172.21.15.60:6789/0 conn(0x7faa000cf3f0 legacy=0x7faa0015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:23.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9fe59c700 1 -- 172.21.15.60:0/3358503999 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa000cd450 con 0x7faa000b8570 2023-07-13T22:25:23.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9fdd9b700 1 -- 172.21.15.60:0/3358503999 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516363163 0 0) 0x7faa000d09d0 con 0x7faa000cf3f0 2023-07-13T22:25:23.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9f7fff700 1 --2- 172.21.15.60:0/3358503999 >> v2:172.21.15.105:3300/0 conn(0x7faa000d35c0 0x7faa000d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:23.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9fe59c700 1 --2- 172.21.15.60:0/3358503999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000b8570 0x7faa000ceeb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faa000d07e0 tx=0x7fa9f0009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfb6a6eb226b337 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:23.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9fdd9b700 1 -- 172.21.15.60:0/3358503999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0011830 con 0x7faa000b8570 2023-07-13T22:25:23.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9fdd9b700 1 -- 172.21.15.60:0/3358503999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f0011990 con 0x7faa000b8570 2023-07-13T22:25:23.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.724+0000 7fa9fdd9b700 1 -- 172.21.15.60:0/3358503999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0011d40 con 0x7faa000b8570 2023-07-13T22:25:23.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.725+0000 7faa04fff700 1 -- 172.21.15.60:0/3358503999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000b8570 msgr2=0x7faa000ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:23.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.725+0000 7faa04fff700 1 --2- 172.21.15.60:0/3358503999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000b8570 0x7faa000ceeb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faa000d07e0 tx=0x7fa9f0009b10 comp rx=0 tx=0).stop 2023-07-13T22:25:23.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.725+0000 7faa04fff700 1 -- 172.21.15.60:0/3358503999 shutdown_connections 2023-07-13T22:25:23.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.725+0000 7faa04fff700 1 --2- 172.21.15.60:0/3358503999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000b8570 0x7faa000ceeb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:23.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.725+0000 7faa04fff700 1 --2- 172.21.15.60:0/3358503999 >> v2:172.21.15.105:3300/0 conn(0x7faa000d35c0 0x7faa000d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:23.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.725+0000 7faa04fff700 1 -- 172.21.15.60:0/3358503999 >> 172.21.15.60:0/3358503999 conn(0x7faa00071600 msgr2=0x7faa000719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:23.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.725+0000 7faa04fff700 1 -- 172.21.15.60:0/3358503999 shutdown_connections 2023-07-13T22:25:23.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.725+0000 7faa04fff700 1 -- 172.21.15.60:0/3358503999 wait complete. 2023-07-13T22:25:23.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.728+0000 7faa04fff700 1 Processor -- start 2023-07-13T22:25:23.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.728+0000 7faa04fff700 1 -- start start 2023-07-13T22:25:23.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.729+0000 7faa04fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa000b8570 0x7faa000c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:23.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.729+0000 7faa04fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa000cf3f0 0x7faa00151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:23.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.729+0000 7faa04fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000d35c0 0x7faa00155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:23.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.729+0000 7faa04fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00157e00 con 0x7faa000d35c0 2023-07-13T22:25:23.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.729+0000 7faa04fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00151f90 con 0x7faa000cf3f0 2023-07-13T22:25:23.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.729+0000 7faa04fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa000c95e0 con 0x7faa000b8570 2023-07-13T22:25:23.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.729+0000 7fa9fed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000d35c0 0x7faa00155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:23.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.729+0000 7fa9fed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000d35c0 0x7faa00155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59880/0 (socket says 172.21.15.60:59880) 2023-07-13T22:25:23.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.729+0000 7fa9fed9d700 1 -- 172.21.15.60:0/457113320 learned_addr learned my addr 172.21.15.60:0/457113320 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:23.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9fe59c700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa000b8570 0x7faa000c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:23.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9f7fff700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa000cf3f0 0x7faa00151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:23.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9fed9d700 1 -- 172.21.15.60:0/457113320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa000b8570 msgr2=0x7faa000c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:23.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9fed9d700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa000b8570 0x7faa000c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:23.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9fed9d700 1 -- 172.21.15.60:0/457113320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa000cf3f0 msgr2=0x7faa00151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:23.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9fed9d700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa000cf3f0 0x7faa00151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:23.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9fed9d700 1 -- 172.21.15.60:0/457113320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0009440 con 0x7faa000d35c0 2023-07-13T22:25:23.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9fed9d700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000d35c0 0x7faa00155c90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8002230 tx=0x7fa9f800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:23.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9f6ffd700 1 -- 172.21.15.60:0/457113320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800aac0 con 0x7faa000d35c0 2023-07-13T22:25:23.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9f6ffd700 1 -- 172.21.15.60:0/457113320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f800ac40 con 0x7faa000d35c0 2023-07-13T22:25:23.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7fa9f6ffd700 1 -- 172.21.15.60:0/457113320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80070d0 con 0x7faa000d35c0 2023-07-13T22:25:23.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.730+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa000c98c0 con 0x7faa000d35c0 2023-07-13T22:25:23.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.731+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa00157780 con 0x7faa000d35c0 2023-07-13T22:25:23.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.731+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0004e280 con 0x7faa000d35c0 2023-07-13T22:25:23.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.733+0000 7fa9f6ffd700 1 -- 172.21.15.60:0/457113320 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa9f8007250 con 0x7faa000d35c0 2023-07-13T22:25:23.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.733+0000 7fa9f6ffd700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9e006ace0 0x7fa9e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:23.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.734+0000 7fa9f6ffd700 1 -- 172.21.15.60:0/457113320 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa9f8088fe0 con 0x7faa000d35c0 2023-07-13T22:25:23.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.734+0000 7fa9fe59c700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9e006ace0 0x7fa9e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:23.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.734+0000 7fa9fe59c700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9e006ace0 0x7fa9e006d190 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00090b0 tx=0x7fa9f001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:23.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.738+0000 7fa9f6ffd700 1 -- 172.21.15.60:0/457113320 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa9f8053ff0 con 0x7faa000d35c0 2023-07-13T22:25:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.990+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa00157f40 con 0x7faa000d35c0 2023-07-13T22:25:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.990+0000 7fa9f6ffd700 1 -- 172.21.15.60:0/457113320 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fa9f8018090 con 0x7faa000d35c0 2023-07-13T22:25:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:23.973 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1603865496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1603865496},{"type":"v1","addr":"172.21.15.105:6834","nonce":1603865496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6735,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1777842000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1777842000},{"type":"v1","addr":"172.21.15.105:6835","nonce":1777842000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6738,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2613179178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2613179178},{"type":"v1","addr":"172.21.15.60:6836","nonce":2613179178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:25:02.392864+0000","modified":"2023-07-13T22:25:23.235357+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6837/3617406711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3617406711},{"type":"v1","addr":"172.21.15.60:6837","nonce":3617406711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-13T22:25:23.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9e006ace0 msgr2=0x7fa9e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:23.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9e006ace0 0x7fa9e006d190 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00090b0 tx=0x7fa9f001c040 comp rx=0 tx=0).stop 2023-07-13T22:25:23.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000d35c0 msgr2=0x7faa00155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:23.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000d35c0 0x7faa00155c90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8002230 tx=0x7fa9f800cf50 comp rx=0 tx=0).stop 2023-07-13T22:25:23.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 shutdown_connections 2023-07-13T22:25:23.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9e006ace0 0x7fa9e006d190 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:23.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa000d35c0 0x7faa00155c90 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:23.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa000b8570 0x7faa000c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:23.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 --2- 172.21.15.60:0/457113320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa000cf3f0 0x7faa00151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:23.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 >> 172.21.15.60:0/457113320 conn(0x7faa00071600 msgr2=0x7faa000732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:23.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 shutdown_connections 2023-07-13T22:25:23.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:23.993+0000 7faa04fff700 1 -- 172.21.15.60:0/457113320 wait complete. 2023-07-13T22:25:23.978 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 43 2023-07-13T22:25:23.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-13T22:25:24.220 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":6017,"entity":{"name":{"type":"client","num":6017},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3775652031}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":10.776884243,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9456482209812234,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6017 192.168.0.1:0/3775652031","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"123518","root":"/"}},{"id":6906,"entity":{"name":{"type":"client","num":6906},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1059118983}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":377,"uptime":13.122859043,"requests_in_flight":0,"num_completed_requests":400,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1161.7298142868826,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":436.71333233886389,"halflife":1.5},"recall_caps_throttle2o":{"value":171.00471449644931,"halflife":0.5},"session_cache_liveness":{"value":794.73330305691104,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6906 192.168.0.1:0/1059118983","completed_requests":[{"tid":5,"created_ino":"0x10000000001"},{"tid":6,"created_ino":"0x100000001f6"},{"tid":7,"created_ino":"0x100000001f7"},{"tid":8,"created_ino":"0x100000001f8"},{"tid":9,"created_ino":"0x100000001f9"},{"tid":10,"created_ino":"0x100000001fa"},{"tid":11,"created_ino":"0x100000001fb"},{"tid":12,"created_ino":"0x100000001fc"},{"tid":13,"created_ino":"0x100000001fd"},{"tid":14,"created_ino":"0x100000001fe"},{"tid":15,"created_ino":"0x100000001ff"},{"tid":16,"created_ino":"0x10000000200"},{"tid":17,"created_ino":"0x10000000201"},{"tid":18,"created_ino":"0x10000000202"},{"tid":19,"created_ino":"0x10000000203"},{"tid":20,"created_ino":"0x10000000204"},{"tid":21,"created_ino":"0x10000000205"},{"tid":22,"created_ino":"0x10000000206"},{"tid":23,"created_ino":"0x10000000207"},{"tid":24,"created_ino":"0x10000000208"},{"tid":25,"created_ino":"0x10000000209"},{"tid":26,"created_ino":"0x1000000020a"},{"tid":27,"created_ino":"0x1000000020b"},{"tid":28,"created_ino":"0x1000000020c"},{"tid":29,"created_ino":"0x1000000020d"},{"tid":30,"created_ino":"0x1000000020e"},{"tid":31,"created_ino":"0x1000000020f"},{"tid":32,"created_ino":"0x10000000210"},{"tid":33,"created_ino":"0x10000000211"},{"tid":34,"created_ino":"0x10000000212"},{"tid":35,"created_ino":"0x10000000213"},{"tid":36,"created_ino":"0x10000000214"},{"tid":37,"created_ino":"0x10000000215"},{"tid":38,"created_ino":"0x10000000216"},{"tid":39,"created_ino":"0x10000000217"},{"tid":40,"created_ino":"0x10000000218"},{"tid":41,"created_ino":"0x10000000219"},{"tid":42,"created_ino":"0x1000000021a"},{"tid":43,"created_ino":"0x1000000021b"},{"tid":44,"created_ino":"0x1000000021c"},{"tid":45,"created_ino":"0x1000000021d"},{"tid":46,"created_ino":"0x1000000021e"},{"tid":47,"created_ino":"0x1000000021f"},{"tid":48,"created_ino":"0x10000000220"},{"tid":49,"created_ino":"0x10000000221"},{"tid":50,"created_ino":"0x10000000222"},{"tid":51,"created_ino":"0x10000000223"},{"tid":52,"created_ino":"0x10000000224"},{"tid":53,"created_ino":"0x10000000225"},{"tid":54,"created_ino":"0x10000000226"},{"tid":55,"created_ino":"0x10000000227"},{"tid":56,"created_ino":"0x10000000228"},{"tid":57,"created_ino":"0x10000000229"},{"tid":58,"created_ino":"0x1000000022a"},{"tid":59,"created_ino":"0x1000000022b"},{"tid":60,"created_ino":"0x1000000022c"},{"tid":61,"created_ino":"0x1000000022d"},{"tid":62,"created_ino":"0x1000000022e"},{"tid":63,"created_ino":"0x1000000022f"},{"tid":64,"created_ino":"0x10000000230"},{"tid":65,"created_ino":"0x10000000231"},{"tid":66,"created_ino":"0x10000000232"},{"tid":67,"created_ino":"0x10000000233"},{"tid":68,"created_ino":"0x10000000234"},{"tid":69,"created_ino":"0x10000000235"},{"tid":70,"created_ino":"0x10000000236"},{"tid":71,"created_ino":"0x10000000237"},{"tid":72,"created_ino":"0x10000000238"},{"tid":73,"created_ino":"0x10000000239"},{"tid":74,"created_ino":"0x1000000023a"},{"tid":75,"created_ino":"0x1000000023b"},{"tid":76,"created_ino":"0x1000000023c"},{"tid":77,"created_ino":"0x1000000023d"},{"tid":78,"created_ino":"0x1000000023e"},{"tid":79,"created_ino":"0x1000000023f"},{"tid":80,"created_ino":"0x10000000240"},{"tid":81,"created_ino":"0x10000000241"},{"tid":82,"created_ino":"0x10000000242"},{"tid":83,"created_ino":"0x10000000243"},{"tid":84,"created_ino":"0x10000000244"},{"tid":85,"created_ino":"0x10000000245"},{"tid":86,"created_ino":"0x10000000246"},{"tid":87,"created_ino":"0x10000000247"},{"tid":88,"created_ino":"0x10000000248"},{"tid":89,"created_ino":"0x10000000249"},{"tid":90,"created_ino":"0x1000000024a"},{"tid":91,"created_ino":"0x1000000024b"},{"tid":92,"created_ino":"0x1000000024c"},{"tid":93,"created_ino":"0x1000000024d"},{"tid":94,"created_ino":"0x1000000024e"},{"tid":95,"created_ino":"0x1000000024f"},{"tid":96,"created_ino":"0x10000000250"},{"tid":97,"created_ino":"0x10000000251"},{"tid":98,"created_ino":"0x10000000252"},{"tid":99,"created_ino":"0x10000000253"},{"tid":100,"created_ino":"0x10000000254"},{"tid":101,"created_ino":"0x10000000255"},{"tid":102,"created_ino":"0x10000000256"},{"tid":103,"created_ino":"0x10000000257"},{"tid":104,"created_ino":"0x10000000258"},{"tid":105,"created_ino":"0x10000000259"},{"tid":106,"created_ino":"0x1000000025a"},{"tid":107,"created_ino":"0x1000000025b"},{"tid":108,"created_ino":"0x1000000025c"},{"tid":109,"created_ino":"0x1000000025d"},{"tid":110,"created_ino":"0x1000000025e"},{"tid":111,"created_ino":"0x1000000025f"},{"tid":112,"created_ino":"0x10000000260"},{"tid":113,"created_ino":"0x10000000261"},{"tid":114,"created_ino":"0x10000000262"},{"tid":115,"created_ino":"0x10000000263"},{"tid":116,"created_ino":"0x10000000264"},{"tid":117,"created_ino":"0x10000000265"},{"tid":118,"created_ino":"0x10000000266"},{"tid":119,"created_ino":"0x10000000267"},{"tid":120,"created_ino":"0x10000000268"},{"tid":121,"created_ino":"0x10000000269"},{"tid":122,"created_ino":"0x1000000026a"},{"tid":123,"created_ino":"0x1000000026b"},{"tid":124,"created_ino":"0x1000000026c"},{"tid":125,"created_ino":"0x1000000026d"},{"tid":126,"created_ino":"0x1000000026e"},{"tid":127,"created_ino":"0x1000000026f"},{"tid":128,"created_ino":"0x10000000270"},{"tid":129,"created_ino":"0x10000000271"},{"tid":130,"created_ino":"0x10000000272"},{"tid":131,"created_ino":"0x10000000273"},{"tid":132,"created_ino":"0x10000000274"},{"tid":133,"created_ino":"0x10000000275"},{"tid":134,"created_ino":"0x10000000276"},{"tid":135,"created_ino":"0x10000000277"},{"tid":136,"created_ino":"0x10000000278"},{"tid":137,"created_ino":"0x10000000279"},{"tid":138,"created_ino":"0x1000000027a"},{"tid":139,"created_ino":"0x1000000027b"},{"tid":140,"created_ino":"0x1000000027c"},{"tid":141,"created_ino":"0x1000000027d"},{"tid":142,"created_ino":"0x1000000027e"},{"tid":143,"created_ino":"0x1000000027f"},{"tid":144,"created_ino":"0x10000000280"},{"tid":145,"created_ino":"0x10000000281"},{"tid":146,"created_ino":"0x10000000282"},{"tid":147,"created_ino":"0x10000000283"},{"tid":148,"created_ino":"0x10000000284"},{"tid":149,"created_ino":"0x10000000285"},{"tid":150,"created_ino":"0x10000000286"},{"tid":151,"created_ino":"0x10000000287"},{"tid":152,"created_ino":"0x10000000288"},{"tid":153,"created_ino":"0x10000000289"},{"tid":154,"created_ino":"0x1000000028a"},{"tid":155,"created_ino":"0x1000000028b"},{"tid":156,"created_ino":"0x1000000028c"},{"tid":157,"created_ino":"0x1000000028d"},{"tid":158,"created_ino":"0x1000000028e"},{"tid":159,"created_ino":"0x1000000028f"},{"tid":160,"created_ino":"0x10000000290"},{"tid":161,"created_ino":"0x10000000291"},{"tid":162,"created_ino":"0x10000000292"},{"tid":163,"created_ino":"0x10000000293"},{"tid":164,"created_ino":"0x10000000294"},{"tid":165,"created_ino":"0x10000000295"},{"tid":166,"created_ino":"0x10000000296"},{"tid":167,"created_ino":"0x10000000297"},{"tid":168,"created_ino":"0x10000000298"},{"tid":169,"created_ino":"0x10000000299"},{"tid":170,"created_ino":"0x1000000029a"},{"tid":171,"created_ino":"0x1000000029b"},{"tid":172,"created_ino":"0x1000000029c"},{"tid":173,"created_ino":"0x1000000029d"},{"tid":174,"created_ino":"0x1000000029e"},{"tid":175,"created_ino":"0x1000000029f"},{"tid":176,"created_ino":"0x100000002a0"},{"tid":177,"created_ino":"0x100000002a1"},{"tid":178,"created_ino":"0x100000002a2"},{"tid":179,"created_ino":"0x100000002a3"},{"tid":180,"created_ino":"0x100000002a4"},{"tid":181,"created_ino":"0x100000002a5"},{"tid":182,"created_ino":"0x100000002a6"},{"tid":183,"created_ino":"0x100000002a7"},{"tid":184,"created_ino":"0x100000002a8"},{"tid":185,"created_ino":"0x100000002a9"},{"tid":186,"created_ino":"0x100000002aa"},{"tid":187,"created_ino":"0x100000002ab"},{"tid":188,"created_ino":"0x100000002ac"},{"tid":189,"created_ino":"0x100000002ad"},{"tid":190,"created_ino":"0x100000002ae"},{"tid":191,"created_ino":"0x100000002af"},{"tid":192,"created_ino":"0x100000002b0"},{"tid":193,"created_ino":"0x100000002b1"},{"tid":194,"created_ino":"0x100000002b2"},{"tid":195,"created_ino":"0x100000002b3"},{"tid":196,"created_ino":"0x100000002b4"},{"tid":197,"created_ino":"0x100000002b5"},{"tid":198,"created_ino":"0x100000002b6"},{"tid":199,"created_ino":"0x100000002b7"},{"tid":200,"created_ino":"0x100000002b8"},{"tid":201,"created_ino":"0x100000002b9"},{"tid":202,"created_ino":"0x100000002ba"},{"tid":203,"created_ino":"0x100000002bb"},{"tid":204,"created_ino":"0x100000002bc"},{"tid":205,"created_ino":"0x100000002bd"},{"tid":206,"created_ino":"0x100000002be"},{"tid":207,"created_ino":"0x100000002bf"},{"tid":208,"created_ino":"0x100000002c0"},{"tid":209,"created_ino":"0x100000002c1"},{"tid":210,"created_ino":"0x100000002c2"},{"tid":211,"created_ino":"0x100000002c3"},{"tid":212,"created_ino":"0x100000002c4"},{"tid":213,"created_ino":"0x100000002c5"},{"tid":214,"created_ino":"0x100000002c6"},{"tid":215,"created_ino":"0x100000002c7"},{"tid":216,"created_ino":"0x100000002c8"},{"tid":217,"created_ino":"0x100000002c9"},{"tid":218,"created_ino":"0x100000002ca"},{"tid":219,"created_ino":"0x100000002cb"},{"tid":220,"created_ino":"0x100000002cc"},{"tid":221,"created_ino":"0x100000002cd"},{"tid":222,"created_ino":"0x100000002ce"},{"tid":223,"created_ino":"0x100000002cf"},{"tid":224,"created_ino":"0x100000002d0"},{"tid":225,"created_ino":"0x100000002d1"},{"tid":226,"created_ino":"0x100000002d2"},{"tid":227,"created_ino":"0x100000002d3"},{"tid":228,"created_ino":"0x100000002d4"},{"tid":229,"created_ino":"0x100000002d5"},{"tid":230,"created_ino":"0x100000002d6"},{"tid":231,"created_ino":"0x100000002d7"},{"tid":232,"created_ino":"0x100000002d8"},{"tid":233,"created_ino":"0x100000002d9"},{"tid":234,"created_ino":"0x100000002da"},{"tid":235,"created_ino":"0x100000002db"},{"tid":236,"created_ino":"0x100000002dc"},{"tid":237,"created_ino":"0x100000002dd"},{"tid":238,"created_ino":"0x100000002de"},{"tid":239,"created_ino":"0x100000002df"},{"tid":240,"created_ino":"0x100000002e0"},{"tid":241,"created_ino":"0x100000002e1"},{"tid":242,"created_ino":"0x100000002e2"},{"tid":243,"created_ino":"0x100000002e3"},{"tid":244,"created_ino":"0x100000002e4"},{"tid":245,"created_ino":"0x100000002e5"},{"tid":246,"created_ino":"0x100000002e6"},{"tid":247,"created_ino":"0x100000002e7"},{"tid":248,"created_ino":"0x100000002e8"},{"tid":249,"created_ino":"0x100000002e9"},{"tid":250,"created_ino":"0x100000002ea"},{"tid":251,"created_ino":"0x100000002eb"},{"tid":252,"created_ino":"0x100000002ec"},{"tid":253,"created_ino":"0x100000002ed"},{"tid":254,"created_ino":"0x100000002ee"},{"tid":255,"created_ino":"0x100000002ef"},{"tid":256,"created_ino":"0x100000002f0"},{"tid":257,"created_ino":"0x100000002f1"},{"tid":258,"created_ino":"0x100000002f2"},{"tid":259,"created_ino":"0x100000002f3"},{"tid":260,"created_ino":"0x100000002f4"},{"tid":261,"created_ino":"0x100000002f5"},{"tid":262,"created_ino":"0x100000002f6"},{"tid":263,"created_ino":"0x100000002f7"},{"tid":264,"created_ino":"0x100000002f8"},{"tid":265,"created_ino":"0x100000002f9"},{"tid":266,"created_ino":"0x100000002fa"},{"tid":267,"created_ino":"0x100000002fb"},{"tid":268,"created_ino":"0x100000002fc"},{"tid":269,"created_ino":"0x100000002fd"},{"tid":270,"created_ino":"0x100000002fe"},{"tid":271,"created_ino":"0x100000002ff"},{"tid":272,"created_ino":"0x10000000300"},{"tid":273,"created_ino":"0x10000000301"},{"tid":274,"created_ino":"0x10000000302"},{"tid":275,"created_ino":"0x10000000303"},{"tid":276,"created_ino":"0x10000000304"},{"tid":277,"created_ino":"0x10000000305"},{"tid":278,"created_ino":"0x10000000306"},{"tid":279,"created_ino":"0x10000000307"},{"tid":280,"created_ino":"0x10000000308"},{"tid":281,"created_ino":"0x10000000309"},{"tid":282,"created_ino":"0x1000000030a"},{"tid":283,"created_ino":"0x1000000030b"},{"tid":284,"created_ino":"0x1000000030c"},{"tid":285,"created_ino":"0x1000000030d"},{"tid":286,"created_ino":"0x1000000030e"},{"tid":287,"created_ino":"0x1000000030f"},{"tid":288,"created_ino":"0x10000000310"},{"tid":289,"created_ino":"0x10000000311"},{"tid":290,"created_ino":"0x10000000312"},{"tid":291,"created_ino":"0x10000000313"},{"tid":292,"created_ino":"0x10000000314"},{"tid":293,"created_ino":"0x10000000315"},{"tid":294,"created_ino":"0x10000000316"},{"tid":295,"created_ino":"0x10000000317"},{"tid":296,"created_ino":"0x10000000318"},{"tid":297,"created_ino":"0x10000000319"},{"tid":298,"created_ino":"0x1000000031a"},{"tid":299,"created_ino":"0x1000000031b"},{"tid":300,"created_ino":"0x1000000031c"},{"tid":301,"created_ino":"0x1000000031d"},{"tid":302,"created_ino":"0x1000000031e"},{"tid":303,"created_ino":"0x1000000031f"},{"tid":304,"created_ino":"0x10000000320"},{"tid":305,"created_ino":"0x10000000321"},{"tid":306,"created_ino":"0x10000000322"},{"tid":307,"created_ino":"0x10000000323"},{"tid":308,"created_ino":"0x10000000324"},{"tid":309,"created_ino":"0x10000000325"},{"tid":310,"created_ino":"0x10000000326"},{"tid":311,"created_ino":"0x10000000327"},{"tid":312,"created_ino":"0x10000000328"},{"tid":313,"created_ino":"0x10000000329"},{"tid":314,"created_ino":"0x1000000032a"},{"tid":315,"created_ino":"0x1000000032b"},{"tid":316,"created_ino":"0x1000000032c"},{"tid":317,"created_ino":"0x1000000032d"},{"tid":318,"created_ino":"0x1000000032e"},{"tid":319,"created_ino":"0x1000000032f"},{"tid":320,"created_ino":"0x10000000330"},{"tid":321,"created_ino":"0x10000000331"},{"tid":322,"created_ino":"0x10000000332"},{"tid":323,"created_ino":"0x10000000333"},{"tid":324,"created_ino":"0x10000000334"},{"tid":325,"created_ino":"0x10000000335"},{"tid":326,"created_ino":"0x10000000336"},{"tid":327,"created_ino":"0x10000000337"},{"tid":328,"created_ino":"0x10000000338"},{"tid":329,"created_ino":"0x10000000339"},{"tid":330,"created_ino":"0x1000000033a"},{"tid":331,"created_ino":"0x1000000033b"},{"tid":332,"created_ino":"0x1000000033c"},{"tid":333,"created_ino":"0x1000000033d"},{"tid":334,"created_ino":"0x1000000033e"},{"tid":335,"created_ino":"0x1000000033f"},{"tid":336,"created_ino":"0x10000000340"},{"tid":337,"created_ino":"0x10000000341"},{"tid":338,"created_ino":"0x10000000342"},{"tid":339,"created_ino":"0x10000000343"},{"tid":340,"created_ino":"0x10000000344"},{"tid":341,"created_ino":"0x10000000345"},{"tid":342,"created_ino":"0x10000000346"},{"tid":343,"created_ino":"0x10000000347"},{"tid":344,"created_ino":"0x10000000348"},{"tid":345,"created_ino":"0x10000000349"},{"tid":346,"created_ino":"0x1000000034a"},{"tid":347,"created_ino":"0x1000000034b"},{"tid":348,"created_ino":"0x1000000034c"},{"tid":349,"created_ino":"0x1000000034d"},{"tid":350,"created_ino":"0x1000000034e"},{"tid":351,"created_ino":"0x1000000034f"},{"tid":352,"created_ino":"0x10000000350"},{"tid":353,"created_ino":"0x10000000351"},{"tid":354,"created_ino":"0x10000000352"},{"tid":355,"created_ino":"0x10000000353"},{"tid":356,"created_ino":"0x10000000354"},{"tid":357,"created_ino":"0x10000000355"},{"tid":358,"created_ino":"0x10000000356"},{"tid":359,"created_ino":"0x10000000357"},{"tid":360,"created_ino":"0x10000000358"},{"tid":361,"created_ino":"0x10000000359"},{"tid":362,"created_ino":"0x1000000035a"},{"tid":363,"created_ino":"0x1000000035b"},{"tid":364,"created_ino":"0x1000000035c"},{"tid":365,"created_ino":"0x1000000035d"},{"tid":366,"created_ino":"0x1000000035e"},{"tid":367,"created_ino":"0x1000000035f"},{"tid":368,"created_ino":"0x10000000360"},{"tid":369,"created_ino":"0x10000000361"},{"tid":370,"created_ino":"0x10000000362"},{"tid":371,"created_ino":"0x10000000363"},{"tid":372,"created_ino":"0x10000000364"},{"tid":373,"created_ino":"0x10000000365"},{"tid":374,"created_ino":"0x10000000366"},{"tid":375,"created_ino":"0x10000000367"},{"tid":376,"created_ino":"0x10000000368"},{"tid":377,"created_ino":"0x10000000369"},{"tid":378,"created_ino":"0x1000000036a"},{"tid":379,"created_ino":"0x1000000036b"},{"tid":380,"created_ino":"0x1000000036c"},{"tid":381,"created_ino":"0x1000000036d"},{"tid":382,"created_ino":"0x1000000036e"},{"tid":383,"created_ino":"0x1000000036f"},{"tid":384,"created_ino":"0x10000000370"},{"tid":385,"created_ino":"0x10000000371"},{"tid":386,"created_ino":"0x10000000372"},{"tid":387,"created_ino":"0x10000000373"},{"tid":388,"created_ino":"0x10000000374"},{"tid":389,"created_ino":"0x10000000375"},{"tid":390,"created_ino":"0x10000000376"},{"tid":391,"created_ino":"0x10000000377"},{"tid":392,"created_ino":"0x10000000378"},{"tid":393,"created_ino":"0x10000000379"},{"tid":394,"created_ino":"0x1000000037a"},{"tid":395,"created_ino":"0x1000000037b"},{"tid":396,"created_ino":"0x1000000037c"},{"tid":397,"created_ino":"0x1000000037d"},{"tid":398,"created_ino":"0x1000000037e"},{"tid":399,"created_ino":"0x1000000037f"},{"tid":400,"created_ino":"0x10000000380"},{"tid":401,"created_ino":"0x10000000381"},{"tid":402,"created_ino":"0x10000000382"},{"tid":403,"created_ino":"0x10000000383"},{"tid":404,"created_ino":"0x10000000384"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000385","length":602}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"141735","root":"/"}}] 2023-07-13T22:25:24.223 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "123518", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3775652031, "type": "any" }, "name": { "num": 6017, "type": "client" } }, "id": 6017, "inst": "client.6017 192.168.0.1:0/3775652031", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9456482209812234 }, "state": "open", "uptime": 10.776884243 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "141735", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 5 }, { "created_ino": "0x100000001f6", "tid": 6 }, { "created_ino": "0x100000001f7", "tid": 7 }, { "created_ino": "0x100000001f8", "tid": 8 }, { "created_ino": "0x100000001f9", "tid": 9 }, { "created_ino": "0x100000001fa", "tid": 10 }, { "created_ino": "0x100000001fb", "tid": 11 }, { "created_ino": "0x100000001fc", "tid": 12 }, { "created_ino": "0x100000001fd", "tid": 13 }, { "created_ino": "0x100000001fe", "tid": 14 }, { "created_ino": "0x100000001ff", "tid": 15 }, { "created_ino": "0x10000000200", "tid": 16 }, { "created_ino": "0x10000000201", "tid": 17 }, { "created_ino": "0x10000000202", "tid": 18 }, { "created_ino": "0x10000000203", "tid": 19 }, { "created_ino": "0x10000000204", "tid": 20 }, { "created_ino": "0x10000000205", "tid": 21 }, { "created_ino": "0x10000000206", "tid": 22 }, { "created_ino": "0x10000000207", "tid": 23 }, { "created_ino": "0x10000000208", "tid": 24 }, { "created_ino": "0x10000000209", "tid": 25 }, { "created_ino": "0x1000000020a", "tid": 26 }, { "created_ino": "0x1000000020b", "tid": 27 }, { "created_ino": "0x1000000020c", "tid": 28 }, { "created_ino": "0x1000000020d", "tid": 29 }, { "created_ino": "0x1000000020e", "tid": 30 }, { "created_ino": "0x1000000020f", "tid": 31 }, { "created_ino": "0x10000000210", "tid": 32 }, { "created_ino": "0x10000000211", "tid": 33 }, { "created_ino": "0x10000000212", "tid": 34 }, { "created_ino": "0x10000000213", "tid": 35 }, { "created_ino": "0x10000000214", "tid": 36 }, { "created_ino": "0x10000000215", "tid": 37 }, { "created_ino": "0x10000000216", "tid": 38 }, { "created_ino": "0x10000000217", "tid": 39 }, { "created_ino": "0x10000000218", "tid": 40 }, { "created_ino": "0x10000000219", "tid": 41 }, { "created_ino": "0x1000000021a", "tid": 42 }, { "created_ino": "0x1000000021b", "tid": 43 }, { "created_ino": "0x1000000021c", "tid": 44 }, { "created_ino": "0x1000000021d", "tid": 45 }, { "created_ino": "0x1000000021e", "tid": 46 }, { "created_ino": "0x1000000021f", "tid": 47 }, { "created_ino": "0x10000000220", "tid": 48 }, { "created_ino": "0x10000000221", "tid": 49 }, { "created_ino": "0x10000000222", "tid": 50 }, { "created_ino": "0x10000000223", "tid": 51 }, { "created_ino": "0x10000000224", "tid": 52 }, { "created_ino": "0x10000000225", "tid": 53 }, { "created_ino": "0x10000000226", "tid": 54 }, { "created_ino": "0x10000000227", "tid": 55 }, { "created_ino": "0x10000000228", "tid": 56 }, { "created_ino": "0x10000000229", "tid": 57 }, { "created_ino": "0x1000000022a", "tid": 58 }, { "created_ino": "0x1000000022b", "tid": 59 }, { "created_ino": "0x1000000022c", "tid": 60 }, { "created_ino": "0x1000000022d", "tid": 61 }, { "created_ino": "0x1000000022e", "tid": 62 }, { "created_ino": "0x1000000022f", "tid": 63 }, { "created_ino": "0x10000000230", "tid": 64 }, { "created_ino": "0x10000000231", "tid": 65 }, { "created_ino": "0x10000000232", "tid": 66 }, { "created_ino": "0x10000000233", "tid": 67 }, { "created_ino": "0x10000000234", "tid": 68 }, { "created_ino": "0x10000000235", "tid": 69 }, { "created_ino": "0x10000000236", "tid": 70 }, { "created_ino": "0x10000000237", "tid": 71 }, { "created_ino": "0x10000000238", "tid": 72 }, { "created_ino": "0x10000000239", "tid": 73 }, { "created_ino": "0x1000000023a", "tid": 74 }, { "created_ino": "0x1000000023b", "tid": 75 }, { "created_ino": "0x1000000023c", "tid": 76 }, { "created_ino": "0x1000000023d", "tid": 77 }, { "created_ino": "0x1000000023e", "tid": 78 }, { "created_ino": "0x1000000023f", "tid": 79 }, { "created_ino": "0x10000000240", "tid": 80 }, { "created_ino": "0x10000000241", "tid": 81 }, { "created_ino": "0x10000000242", "tid": 82 }, { "created_ino": "0x10000000243", "tid": 83 }, { "created_ino": "0x10000000244", "tid": 84 }, { "created_ino": "0x10000000245", "tid": 85 }, { "created_ino": "0x10000000246", "tid": 86 }, { "created_ino": "0x10000000247", "tid": 87 }, { "created_ino": "0x10000000248", "tid": 88 }, { "created_ino": "0x10000000249", "tid": 89 }, { "created_ino": "0x1000000024a", "tid": 90 }, { "created_ino": "0x1000000024b", "tid": 91 }, { "created_ino": "0x1000000024c", "tid": 92 }, { "created_ino": "0x1000000024d", "tid": 93 }, { "created_ino": "0x1000000024e", "tid": 94 }, { "created_ino": "0x1000000024f", "tid": 95 }, { "created_ino": "0x10000000250", "tid": 96 }, { "created_ino": "0x10000000251", "tid": 97 }, { "created_ino": "0x10000000252", "tid": 98 }, { "created_ino": "0x10000000253", "tid": 99 }, { "created_ino": "0x10000000254", "tid": 100 }, { "created_ino": "0x10000000255", "tid": 101 }, { "created_ino": "0x10000000256", "tid": 102 }, { "created_ino": "0x10000000257", "tid": 103 }, { "created_ino": "0x10000000258", "tid": 104 }, { "created_ino": "0x10000000259", "tid": 105 }, { "created_ino": "0x1000000025a", "tid": 106 }, { "created_ino": "0x1000000025b", "tid": 107 }, { "created_ino": "0x1000000025c", "tid": 108 }, { "created_ino": "0x1000000025d", "tid": 109 }, { "created_ino": "0x1000000025e", "tid": 110 }, { "created_ino": "0x1000000025f", "tid": 111 }, { "created_ino": "0x10000000260", "tid": 112 }, { "created_ino": "0x10000000261", "tid": 113 }, { "created_ino": "0x10000000262", "tid": 114 }, { "created_ino": "0x10000000263", "tid": 115 }, { "created_ino": "0x10000000264", "tid": 116 }, { "created_ino": "0x10000000265", "tid": 117 }, { "created_ino": "0x10000000266", "tid": 118 }, { "created_ino": "0x10000000267", "tid": 119 }, { "created_ino": "0x10000000268", "tid": 120 }, { "created_ino": "0x10000000269", "tid": 121 }, { "created_ino": "0x1000000026a", "tid": 122 }, { "created_ino": "0x1000000026b", "tid": 123 }, { "created_ino": "0x1000000026c", "tid": 124 }, { "created_ino": "0x1000000026d", "tid": 125 }, { "created_ino": "0x1000000026e", "tid": 126 }, { "created_ino": "0x1000000026f", "tid": 127 }, { "created_ino": "0x10000000270", "tid": 128 }, { "created_ino": "0x10000000271", "tid": 129 }, { "created_ino": "0x10000000272", "tid": 130 }, { "created_ino": "0x10000000273", "tid": 131 }, { "created_ino": "0x10000000274", "tid": 132 }, { "created_ino": "0x10000000275", "tid": 133 }, { "created_ino": "0x10000000276", "tid": 134 }, { "created_ino": "0x10000000277", "tid": 135 }, { "created_ino": "0x10000000278", "tid": 136 }, { "created_ino": "0x10000000279", "tid": 137 }, { "created_ino": "0x1000000027a", "tid": 138 }, { "created_ino": "0x1000000027b", "tid": 139 }, { "created_ino": "0x1000000027c", "tid": 140 }, { "created_ino": "0x1000000027d", "tid": 141 }, { "created_ino": "0x1000000027e", "tid": 142 }, { "created_ino": "0x1000000027f", "tid": 143 }, { "created_ino": "0x10000000280", "tid": 144 }, { "created_ino": "0x10000000281", "tid": 145 }, { "created_ino": "0x10000000282", "tid": 146 }, { "created_ino": "0x10000000283", "tid": 147 }, { "created_ino": "0x10000000284", "tid": 148 }, { "created_ino": "0x10000000285", "tid": 149 }, { "created_ino": "0x10000000286", "tid": 150 }, { "created_ino": "0x10000000287", "tid": 151 }, { "created_ino": "0x10000000288", "tid": 152 }, { "created_ino": "0x10000000289", "tid": 153 }, { "created_ino": "0x1000000028a", "tid": 154 }, { "created_ino": "0x1000000028b", "tid": 155 }, { "created_ino": "0x1000000028c", "tid": 156 }, { "created_ino": "0x1000000028d", "tid": 157 }, { "created_ino": "0x1000000028e", "tid": 158 }, { "created_ino": "0x1000000028f", "tid": 159 }, { "created_ino": "0x10000000290", "tid": 160 }, { "created_ino": "0x10000000291", "tid": 161 }, { "created_ino": "0x10000000292", "tid": 162 }, { "created_ino": "0x10000000293", "tid": 163 }, { "created_ino": "0x10000000294", "tid": 164 }, { "created_ino": "0x10000000295", "tid": 165 }, { "created_ino": "0x10000000296", "tid": 166 }, { "created_ino": "0x10000000297", "tid": 167 }, { "created_ino": "0x10000000298", "tid": 168 }, { "created_ino": "0x10000000299", "tid": 169 }, { "created_ino": "0x1000000029a", "tid": 170 }, { "created_ino": "0x1000000029b", "tid": 171 }, { "created_ino": "0x1000000029c", "tid": 172 }, { "created_ino": "0x1000000029d", "tid": 173 }, { "created_ino": "0x1000000029e", "tid": 174 }, { "created_ino": "0x1000000029f", "tid": 175 }, { "created_ino": "0x100000002a0", "tid": 176 }, { "created_ino": "0x100000002a1", "tid": 177 }, { "created_ino": "0x100000002a2", "tid": 178 }, { "created_ino": "0x100000002a3", "tid": 179 }, { "created_ino": "0x100000002a4", "tid": 180 }, { "created_ino": "0x100000002a5", "tid": 181 }, { "created_ino": "0x100000002a6", "tid": 182 }, { "created_ino": "0x100000002a7", "tid": 183 }, { "created_ino": "0x100000002a8", "tid": 184 }, { "created_ino": "0x100000002a9", "tid": 185 }, { "created_ino": "0x100000002aa", "tid": 186 }, { "created_ino": "0x100000002ab", "tid": 187 }, { "created_ino": "0x100000002ac", "tid": 188 }, { "created_ino": "0x100000002ad", "tid": 189 }, { "created_ino": "0x100000002ae", "tid": 190 }, { "created_ino": "0x100000002af", "tid": 191 }, { "created_ino": "0x100000002b0", "tid": 192 }, { "created_ino": "0x100000002b1", "tid": 193 }, { "created_ino": "0x100000002b2", "tid": 194 }, { "created_ino": "0x100000002b3", "tid": 195 }, { "created_ino": "0x100000002b4", "tid": 196 }, { "created_ino": "0x100000002b5", "tid": 197 }, { "created_ino": "0x100000002b6", "tid": 198 }, { "created_ino": "0x100000002b7", "tid": 199 }, { "created_ino": "0x100000002b8", "tid": 200 }, { "created_ino": "0x100000002b9", "tid": 201 }, { "created_ino": "0x100000002ba", "tid": 202 }, { "created_ino": "0x100000002bb", "tid": 203 }, { "created_ino": "0x100000002bc", "tid": 204 }, { "created_ino": "0x100000002bd", "tid": 205 }, { "created_ino": "0x100000002be", "tid": 206 }, { "created_ino": "0x100000002bf", "tid": 207 }, { "created_ino": "0x100000002c0", "tid": 208 }, { "created_ino": "0x100000002c1", "tid": 209 }, { "created_ino": "0x100000002c2", "tid": 210 }, { "created_ino": "0x100000002c3", "tid": 211 }, { "created_ino": "0x100000002c4", "tid": 212 }, { "created_ino": "0x100000002c5", "tid": 213 }, { "created_ino": "0x100000002c6", "tid": 214 }, { "created_ino": "0x100000002c7", "tid": 215 }, { "created_ino": "0x100000002c8", "tid": 216 }, { "created_ino": "0x100000002c9", "tid": 217 }, { "created_ino": "0x100000002ca", "tid": 218 }, { "created_ino": "0x100000002cb", "tid": 219 }, { "created_ino": "0x100000002cc", "tid": 220 }, { "created_ino": "0x100000002cd", "tid": 221 }, { "created_ino": "0x100000002ce", "tid": 222 }, { "created_ino": "0x100000002cf", "tid": 223 }, { "created_ino": "0x100000002d0", "tid": 224 }, { "created_ino": "0x100000002d1", "tid": 225 }, { "created_ino": "0x100000002d2", "tid": 226 }, { "created_ino": "0x100000002d3", "tid": 227 }, { "created_ino": "0x100000002d4", "tid": 228 }, { "created_ino": "0x100000002d5", "tid": 229 }, { "created_ino": "0x100000002d6", "tid": 230 }, { "created_ino": "0x100000002d7", "tid": 231 }, { "created_ino": "0x100000002d8", "tid": 232 }, { "created_ino": "0x100000002d9", "tid": 233 }, { "created_ino": "0x100000002da", "tid": 234 }, { "created_ino": "0x100000002db", "tid": 235 }, { "created_ino": "0x100000002dc", "tid": 236 }, { "created_ino": "0x100000002dd", "tid": 237 }, { "created_ino": "0x100000002de", "tid": 238 }, { "created_ino": "0x100000002df", "tid": 239 }, { "created_ino": "0x100000002e0", "tid": 240 }, { "created_ino": "0x100000002e1", "tid": 241 }, { "created_ino": "0x100000002e2", "tid": 242 }, { "created_ino": "0x100000002e3", "tid": 243 }, { "created_ino": "0x100000002e4", "tid": 244 }, { "created_ino": "0x100000002e5", "tid": 245 }, { "created_ino": "0x100000002e6", "tid": 246 }, { "created_ino": "0x100000002e7", "tid": 247 }, { "created_ino": "0x100000002e8", "tid": 248 }, { "created_ino": "0x100000002e9", "tid": 249 }, { "created_ino": "0x100000002ea", "tid": 250 }, { "created_ino": "0x100000002eb", "tid": 251 }, { "created_ino": "0x100000002ec", "tid": 252 }, { "created_ino": "0x100000002ed", "tid": 253 }, { "created_ino": "0x100000002ee", "tid": 254 }, { "created_ino": "0x100000002ef", "tid": 255 }, { "created_ino": "0x100000002f0", "tid": 256 }, { "created_ino": "0x100000002f1", "tid": 257 }, { "created_ino": "0x100000002f2", "tid": 258 }, { "created_ino": "0x100000002f3", "tid": 259 }, { "created_ino": "0x100000002f4", "tid": 260 }, { "created_ino": "0x100000002f5", "tid": 261 }, { "created_ino": "0x100000002f6", "tid": 262 }, { "created_ino": "0x100000002f7", "tid": 263 }, { "created_ino": "0x100000002f8", "tid": 264 }, { "created_ino": "0x100000002f9", "tid": 265 }, { "created_ino": "0x100000002fa", "tid": 266 }, { "created_ino": "0x100000002fb", "tid": 267 }, { "created_ino": "0x100000002fc", "tid": 268 }, { "created_ino": "0x100000002fd", "tid": 269 }, { "created_ino": "0x100000002fe", "tid": 270 }, { "created_ino": "0x100000002ff", "tid": 271 }, { "created_ino": "0x10000000300", "tid": 272 }, { "created_ino": "0x10000000301", "tid": 273 }, { "created_ino": "0x10000000302", "tid": 274 }, { "created_ino": "0x10000000303", "tid": 275 }, { "created_ino": "0x10000000304", "tid": 276 }, { "created_ino": "0x10000000305", "tid": 277 }, { "created_ino": "0x10000000306", "tid": 278 }, { "created_ino": "0x10000000307", "tid": 279 }, { "created_ino": "0x10000000308", "tid": 280 }, { "created_ino": "0x10000000309", "tid": 281 }, { "created_ino": "0x1000000030a", "tid": 282 }, { "created_ino": "0x1000000030b", "tid": 283 }, { "created_ino": "0x1000000030c", "tid": 284 }, { "created_ino": "0x1000000030d", "tid": 285 }, { "created_ino": "0x1000000030e", "tid": 286 }, { "created_ino": "0x1000000030f", "tid": 287 }, { "created_ino": "0x10000000310", "tid": 288 }, { "created_ino": "0x10000000311", "tid": 289 }, { "created_ino": "0x10000000312", "tid": 290 }, { "created_ino": "0x10000000313", "tid": 291 }, { "created_ino": "0x10000000314", "tid": 292 }, { "created_ino": "0x10000000315", "tid": 293 }, { "created_ino": "0x10000000316", "tid": 294 }, { "created_ino": "0x10000000317", "tid": 295 }, { "created_ino": "0x10000000318", "tid": 296 }, { "created_ino": "0x10000000319", "tid": 297 }, { "created_ino": "0x1000000031a", "tid": 298 }, { "created_ino": "0x1000000031b", "tid": 299 }, { "created_ino": "0x1000000031c", "tid": 300 }, { "created_ino": "0x1000000031d", "tid": 301 }, { "created_ino": "0x1000000031e", "tid": 302 }, { "created_ino": "0x1000000031f", "tid": 303 }, { "created_ino": "0x10000000320", "tid": 304 }, { "created_ino": "0x10000000321", "tid": 305 }, { "created_ino": "0x10000000322", "tid": 306 }, { "created_ino": "0x10000000323", "tid": 307 }, { "created_ino": "0x10000000324", "tid": 308 }, { "created_ino": "0x10000000325", "tid": 309 }, { "created_ino": "0x10000000326", "tid": 310 }, { "created_ino": "0x10000000327", "tid": 311 }, { "created_ino": "0x10000000328", "tid": 312 }, { "created_ino": "0x10000000329", "tid": 313 }, { "created_ino": "0x1000000032a", "tid": 314 }, { "created_ino": "0x1000000032b", "tid": 315 }, { "created_ino": "0x1000000032c", "tid": 316 }, { "created_ino": "0x1000000032d", "tid": 317 }, { "created_ino": "0x1000000032e", "tid": 318 }, { "created_ino": "0x1000000032f", "tid": 319 }, { "created_ino": "0x10000000330", "tid": 320 }, { "created_ino": "0x10000000331", "tid": 321 }, { "created_ino": "0x10000000332", "tid": 322 }, { "created_ino": "0x10000000333", "tid": 323 }, { "created_ino": "0x10000000334", "tid": 324 }, { "created_ino": "0x10000000335", "tid": 325 }, { "created_ino": "0x10000000336", "tid": 326 }, { "created_ino": "0x10000000337", "tid": 327 }, { "created_ino": "0x10000000338", "tid": 328 }, { "created_ino": "0x10000000339", "tid": 329 }, { "created_ino": "0x1000000033a", "tid": 330 }, { "created_ino": "0x1000000033b", "tid": 331 }, { "created_ino": "0x1000000033c", "tid": 332 }, { "created_ino": "0x1000000033d", "tid": 333 }, { "created_ino": "0x1000000033e", "tid": 334 }, { "created_ino": "0x1000000033f", "tid": 335 }, { "created_ino": "0x10000000340", "tid": 336 }, { "created_ino": "0x10000000341", "tid": 337 }, { "created_ino": "0x10000000342", "tid": 338 }, { "created_ino": "0x10000000343", "tid": 339 }, { "created_ino": "0x10000000344", "tid": 340 }, { "created_ino": "0x10000000345", "tid": 341 }, { "created_ino": "0x10000000346", "tid": 342 }, { "created_ino": "0x10000000347", "tid": 343 }, { "created_ino": "0x10000000348", "tid": 344 }, { "created_ino": "0x10000000349", "tid": 345 }, { "created_ino": "0x1000000034a", "tid": 346 }, { "created_ino": "0x1000000034b", "tid": 347 }, { "created_ino": "0x1000000034c", "tid": 348 }, { "created_ino": "0x1000000034d", "tid": 349 }, { "created_ino": "0x1000000034e", "tid": 350 }, { "created_ino": "0x1000000034f", "tid": 351 }, { "created_ino": "0x10000000350", "tid": 352 }, { "created_ino": "0x10000000351", "tid": 353 }, { "created_ino": "0x10000000352", "tid": 354 }, { "created_ino": "0x10000000353", "tid": 355 }, { "created_ino": "0x10000000354", "tid": 356 }, { "created_ino": "0x10000000355", "tid": 357 }, { "created_ino": "0x10000000356", "tid": 358 }, { "created_ino": "0x10000000357", "tid": 359 }, { "created_ino": "0x10000000358", "tid": 360 }, { "created_ino": "0x10000000359", "tid": 361 }, { "created_ino": "0x1000000035a", "tid": 362 }, { "created_ino": "0x1000000035b", "tid": 363 }, { "created_ino": "0x1000000035c", "tid": 364 }, { "created_ino": "0x1000000035d", "tid": 365 }, { "created_ino": "0x1000000035e", "tid": 366 }, { "created_ino": "0x1000000035f", "tid": 367 }, { "created_ino": "0x10000000360", "tid": 368 }, { "created_ino": "0x10000000361", "tid": 369 }, { "created_ino": "0x10000000362", "tid": 370 }, { "created_ino": "0x10000000363", "tid": 371 }, { "created_ino": "0x10000000364", "tid": 372 }, { "created_ino": "0x10000000365", "tid": 373 }, { "created_ino": "0x10000000366", "tid": 374 }, { "created_ino": "0x10000000367", "tid": 375 }, { "created_ino": "0x10000000368", "tid": 376 }, { "created_ino": "0x10000000369", "tid": 377 }, { "created_ino": "0x1000000036a", "tid": 378 }, { "created_ino": "0x1000000036b", "tid": 379 }, { "created_ino": "0x1000000036c", "tid": 380 }, { "created_ino": "0x1000000036d", "tid": 381 }, { "created_ino": "0x1000000036e", "tid": 382 }, { "created_ino": "0x1000000036f", "tid": 383 }, { "created_ino": "0x10000000370", "tid": 384 }, { "created_ino": "0x10000000371", "tid": 385 }, { "created_ino": "0x10000000372", "tid": 386 }, { "created_ino": "0x10000000373", "tid": 387 }, { "created_ino": "0x10000000374", "tid": 388 }, { "created_ino": "0x10000000375", "tid": 389 }, { "created_ino": "0x10000000376", "tid": 390 }, { "created_ino": "0x10000000377", "tid": 391 }, { "created_ino": "0x10000000378", "tid": 392 }, { "created_ino": "0x10000000379", "tid": 393 }, { "created_ino": "0x1000000037a", "tid": 394 }, { "created_ino": "0x1000000037b", "tid": 395 }, { "created_ino": "0x1000000037c", "tid": 396 }, { "created_ino": "0x1000000037d", "tid": 397 }, { "created_ino": "0x1000000037e", "tid": 398 }, { "created_ino": "0x1000000037f", "tid": 399 }, { "created_ino": "0x10000000380", "tid": 400 }, { "created_ino": "0x10000000381", "tid": 401 }, { "created_ino": "0x10000000382", "tid": 402 }, { "created_ino": "0x10000000383", "tid": 403 }, { "created_ino": "0x10000000384", "tid": 404 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1059118983, "type": "any" }, "name": { "num": 6906, "type": "client" } }, "id": 6906, "inst": "client.6906 192.168.0.1:0/1059118983", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 400, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 602, "start": "0x10000000385" } ], "recall_caps": { "halflife": 60, "value": 1161.7298142868826 }, "recall_caps_throttle": { "halflife": 1.5, "value": 436.7133323388639 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 171.0047144964493 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 377, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 794.733303056911 }, "state": "open", "uptime": 13.122859043 } ] 2023-07-13T22:25:24.223 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-07-13T22:25:24.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:25:24.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.444+0000 7f56d87ff700 1 Processor -- start 2023-07-13T22:25:24.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.444+0000 7f56d87ff700 1 -- start start 2023-07-13T22:25:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.445+0000 7f56d87ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0a50 0x7f56d00cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.445+0000 7f56d87ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56d00cd260 0x7f56d00d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.445+0000 7f56d87ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56d00d0e20 0x7f56d00d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.445+0000 7f56d87ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d015c900 con 0x7f56d00d0e20 2023-07-13T22:25:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.445+0000 7f56d87ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d00d2410 con 0x7f56d00cd260 2023-07-13T22:25:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d87ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00d2660 con 0x7f56d00d0a50 2023-07-13T22:25:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d6d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56d00d0e20 0x7f56d00d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d6d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56d00d0e20 0x7f56d00d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59886/0 (socket says 172.21.15.60:59886) 2023-07-13T22:25:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d6d9c700 1 -- 172.21.15.60:0/476095982 learned_addr learned my addr 172.21.15.60:0/476095982 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d5d9a700 1 --2- 172.21.15.60:0/476095982 >> v2:172.21.15.105:3300/0 conn(0x7f56d00cd260 0x7f56d00d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d659b700 1 --2- 172.21.15.60:0/476095982 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0a50 0x7f56d00cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d6d9c700 1 -- 172.21.15.60:0/476095982 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0a50 msgr2=0x7f56d00cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d6d9c700 1 --2- 172.21.15.60:0/476095982 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0a50 0x7f56d00cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d6d9c700 1 -- 172.21.15.60:0/476095982 >> v2:172.21.15.105:3300/0 conn(0x7f56d00cd260 msgr2=0x7f56d00d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d6d9c700 1 --2- 172.21.15.60:0/476095982 >> v2:172.21.15.105:3300/0 conn(0x7f56d00cd260 0x7f56d00d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.446+0000 7f56d6d9c700 1 -- 172.21.15.60:0/476095982 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d00d2940 con 0x7f56d00d0e20 2023-07-13T22:25:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d6d9c700 1 --2- 172.21.15.60:0/476095982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0e20 0x7f56d00d1280 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f56d00d3d10 tx=0x7f56c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77983f917ac508c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d5599700 1 -- 172.21.15.60:0/476095982 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c800e070 con 0x7f56d00d0e20 2023-07-13T22:25:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d5599700 1 -- 172.21.15.60:0/476095982 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c800b9c0 con 0x7f56d00d0e20 2023-07-13T22:25:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d5599700 1 -- 172.21.15.60:0/476095982 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c800bd70 con 0x7f56d00d0e20 2023-07-13T22:25:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d87ff700 1 -- 172.21.15.60:0/476095982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0e20 msgr2=0x7f56d00d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d87ff700 1 --2- 172.21.15.60:0/476095982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0e20 0x7f56d00d1280 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f56d00d3d10 tx=0x7f56c800b330 comp rx=0 tx=0).stop 2023-07-13T22:25:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d87ff700 1 -- 172.21.15.60:0/476095982 shutdown_connections 2023-07-13T22:25:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d87ff700 1 --2- 172.21.15.60:0/476095982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0e20 0x7f56d00d1280 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d87ff700 1 --2- 172.21.15.60:0/476095982 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0a50 0x7f56d00cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d87ff700 1 --2- 172.21.15.60:0/476095982 >> v2:172.21.15.105:3300/0 conn(0x7f56d00cd260 0x7f56d00d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d87ff700 1 -- 172.21.15.60:0/476095982 >> 172.21.15.60:0/476095982 conn(0x7f56d0071600 msgr2=0x7f56d00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:24.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d87ff700 1 -- 172.21.15.60:0/476095982 shutdown_connections 2023-07-13T22:25:24.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.447+0000 7f56d87ff700 1 -- 172.21.15.60:0/476095982 wait complete. 2023-07-13T22:25:24.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.449+0000 7f56d87ff700 1 Processor -- start 2023-07-13T22:25:24.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.449+0000 7f56d87ff700 1 -- start start 2023-07-13T22:25:24.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.450+0000 7f56d87ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00cd260 0x7f56d0150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.450+0000 7f56d87ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0a50 0x7f56d00c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.450+0000 7f56d87ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0e20 0x7f56d00cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.450+0000 7f56d87ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d015cb00 con 0x7f56d00d0a50 2023-07-13T22:25:24.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.450+0000 7f56d87ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00c7920 con 0x7f56d00cd260 2023-07-13T22:25:24.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.450+0000 7f56d87ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00c7c00 con 0x7f56d00d0e20 2023-07-13T22:25:24.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.450+0000 7f56d5d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0a50 0x7f56d00c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d5d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0a50 0x7f56d00c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59902/0 (socket says 172.21.15.60:59902) 2023-07-13T22:25:24.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d5d9a700 1 -- 172.21.15.60:0/2880288420 learned_addr learned my addr 172.21.15.60:0/2880288420 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:24.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d6d9c700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0e20 0x7f56d00cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d659b700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00cd260 0x7f56d0150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d5d9a700 1 -- 172.21.15.60:0/2880288420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0e20 msgr2=0x7f56d00cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:24.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d5d9a700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0e20 0x7f56d00cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d5d9a700 1 -- 172.21.15.60:0/2880288420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00cd260 msgr2=0x7f56d0150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:24.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d5d9a700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00cd260 0x7f56d0150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d5d9a700 1 -- 172.21.15.60:0/2880288420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c8009b30 con 0x7f56d00d0a50 2023-07-13T22:25:24.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d5d9a700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0a50 0x7f56d00c8dd0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f56c000d800 tx=0x7f56c000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:24.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56c6ffd700 1 -- 172.21.15.60:0/2880288420 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c00125f0 con 0x7f56d00d0a50 2023-07-13T22:25:24.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56c6ffd700 1 -- 172.21.15.60:0/2880288420 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c0012750 con 0x7f56d00d0a50 2023-07-13T22:25:24.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d00c7ee0 con 0x7f56d00d0a50 2023-07-13T22:25:24.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.451+0000 7f56c6ffd700 1 -- 172.21.15.60:0/2880288420 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0012b00 con 0x7f56d00d0a50 2023-07-13T22:25:24.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.452+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d00c81c0 con 0x7f56d00d0a50 2023-07-13T22:25:24.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.452+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d004e280 con 0x7f56d00d0a50 2023-07-13T22:25:24.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.454+0000 7f56c6ffd700 1 -- 172.21.15.60:0/2880288420 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56c0012c60 con 0x7f56d00d0a50 2023-07-13T22:25:24.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.455+0000 7f56c6ffd700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56b406ace0 0x7f56b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.455+0000 7f56c6ffd700 1 -- 172.21.15.60:0/2880288420 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f56c0089510 con 0x7f56d00d0a50 2023-07-13T22:25:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.455+0000 7f56d659b700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56b406ace0 0x7f56b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.456+0000 7f56d659b700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56b406ace0 0x7f56b406d190 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f56cc006b20 tx=0x7f56cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.459+0000 7f56c6ffd700 1 -- 172.21.15.60:0/2880288420 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f56c0054520 con 0x7f56d00d0a50 2023-07-13T22:25:24.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.712+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f56d00b6710 con 0x7f56d00d0a50 2023-07-13T22:25:24.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.712+0000 7f56c6ffd700 1 -- 172.21.15.60:0/2880288420 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f56c0057b40 con 0x7f56d00d0a50 2023-07-13T22:25:24.693 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:24.694 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:25:24.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56b406ace0 msgr2=0x7f56b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56b406ace0 0x7f56b406d190 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f56cc006b20 tx=0x7f56cc008040 comp rx=0 tx=0).stop 2023-07-13T22:25:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0a50 msgr2=0x7f56d00c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0a50 0x7f56d00c8dd0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f56c000d800 tx=0x7f56c000db10 comp rx=0 tx=0).stop 2023-07-13T22:25:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 shutdown_connections 2023-07-13T22:25:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56b406ace0 0x7f56b406d190 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d00d0e20 0x7f56d00cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d00cd260 0x7f56d0150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 --2- 172.21.15.60:0/2880288420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d00d0a50 0x7f56d00c8dd0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 >> 172.21.15.60:0/2880288420 conn(0x7f56d0071600 msgr2=0x7f56d00b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.714+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 shutdown_connections 2023-07-13T22:25:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.715+0000 7f56d87ff700 1 -- 172.21.15.60:0/2880288420 wait complete. 2023-07-13T22:25:24.713 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:25:24.714 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-07-13T22:25:24.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:25:24.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.935+0000 7f50dd3ff700 1 Processor -- start 2023-07-13T22:25:24.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.935+0000 7f50dd3ff700 1 -- start start 2023-07-13T22:25:24.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.936+0000 7f50dd3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50d80d0a50 0x7f50d80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.936+0000 7f50dd3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50d80cd260 0x7f50d80d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.936+0000 7f50dd3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d80d2410 con 0x7f50d80d0e20 2023-07-13T22:25:24.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.936+0000 7f50dd3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d815c900 con 0x7f50d80cd260 2023-07-13T22:25:24.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.936+0000 7f50dd3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d80d2660 con 0x7f50d80d0a50 2023-07-13T22:25:24.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.937+0000 7f50cffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50d80cd260 0x7f50d80d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.937+0000 7f50d77fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50d80d0e20 0x7f50d80b5a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55074/0 (socket says 172.21.15.60:55074) 2023-07-13T22:25:24.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.937+0000 7f50d77fe700 1 -- 172.21.15.60:0/435636259 learned_addr learned my addr 172.21.15.60:0/435636259 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:24.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.937+0000 7f50d6ffd700 1 --2- 172.21.15.60:0/435636259 >> v2:172.21.15.105:3300/0 conn(0x7f50d80d0a50 0x7f50d80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.937+0000 7f50cffff700 1 -- 172.21.15.60:0/435636259 >> v2:172.21.15.105:3300/0 conn(0x7f50d80d0a50 msgr2=0x7f50d80cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:24.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.937+0000 7f50cffff700 1 --2- 172.21.15.60:0/435636259 >> v2:172.21.15.105:3300/0 conn(0x7f50d80d0a50 0x7f50d80cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.937+0000 7f50cffff700 1 -- 172.21.15.60:0/435636259 >> v1:172.21.15.60:6789/0 conn(0x7f50d80d0e20 legacy=0x7f50d80b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:24.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.937+0000 7f50cffff700 1 -- 172.21.15.60:0/435636259 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80d2940 con 0x7f50d80cd260 2023-07-13T22:25:24.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.937+0000 7f50d67fc700 1 -- 172.21.15.60:0/435636259 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450831760 0 0) 0x7f50d80d2410 con 0x7f50d80d0e20 2023-07-13T22:25:24.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50cffff700 1 --2- 172.21.15.60:0/435636259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80cd260 0x7f50d80d3420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f50d808be90 tx=0x7f50c0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b39a1e69c82c1d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:24.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50d67fc700 1 -- 172.21.15.60:0/435636259 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0011830 con 0x7f50d80cd260 2023-07-13T22:25:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50d67fc700 1 -- 172.21.15.60:0/435636259 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0011990 con 0x7f50d80cd260 2023-07-13T22:25:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50d67fc700 1 -- 172.21.15.60:0/435636259 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0011d20 con 0x7f50d80cd260 2023-07-13T22:25:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50dd3ff700 1 -- 172.21.15.60:0/435636259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80cd260 msgr2=0x7f50d80d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50dd3ff700 1 --2- 172.21.15.60:0/435636259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80cd260 0x7f50d80d3420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f50d808be90 tx=0x7f50c0009b20 comp rx=0 tx=0).stop 2023-07-13T22:25:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50dd3ff700 1 -- 172.21.15.60:0/435636259 shutdown_connections 2023-07-13T22:25:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50dd3ff700 1 --2- 172.21.15.60:0/435636259 >> v2:172.21.15.105:3300/0 conn(0x7f50d80d0a50 0x7f50d80cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50dd3ff700 1 --2- 172.21.15.60:0/435636259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80cd260 0x7f50d80d3420 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50dd3ff700 1 -- 172.21.15.60:0/435636259 >> 172.21.15.60:0/435636259 conn(0x7f50d8071600 msgr2=0x7f50d80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.938+0000 7f50dd3ff700 1 -- 172.21.15.60:0/435636259 shutdown_connections 2023-07-13T22:25:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.939+0000 7f50dd3ff700 1 -- 172.21.15.60:0/435636259 wait complete. 2023-07-13T22:25:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.941+0000 7f50dd3ff700 1 Processor -- start 2023-07-13T22:25:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.941+0000 7f50dd3ff700 1 -- start start 2023-07-13T22:25:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.942+0000 7f50dd3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d80cd260 0x7f50d815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.942+0000 7f50dd3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80d0a50 0x7f50d80c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.942+0000 7f50dd3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d80d0e20 0x7f50d80ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.942+0000 7f50dd3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80cb720 con 0x7f50d80d0a50 2023-07-13T22:25:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.942+0000 7f50dd3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d815f5c0 con 0x7f50d80cd260 2023-07-13T22:25:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.942+0000 7f50dd3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80cb380 con 0x7f50d80d0e20 2023-07-13T22:25:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50cffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80d0a50 0x7f50d80c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50cffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80d0a50 0x7f50d80c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60048/0 (socket says 172.21.15.60:60048) 2023-07-13T22:25:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50cffff700 1 -- 172.21.15.60:0/2332013958 learned_addr learned my addr 172.21.15.60:0/2332013958 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50d6ffd700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d80cd260 0x7f50d815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50d77fe700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d80d0e20 0x7f50d80ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50cffff700 1 -- 172.21.15.60:0/2332013958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d80d0e20 msgr2=0x7f50d80ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50cffff700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d80d0e20 0x7f50d80ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50cffff700 1 -- 172.21.15.60:0/2332013958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d80cd260 msgr2=0x7f50d815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50cffff700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d80cd260 0x7f50d815d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50cffff700 1 -- 172.21.15.60:0/2332013958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00094a0 con 0x7f50d80d0a50 2023-07-13T22:25:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50d77fe700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d80d0e20 0x7f50d80ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.943+0000 7f50cffff700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80d0a50 0x7f50d80c9ce0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f50c00090b0 tx=0x7f50c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:24.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.944+0000 7f50cf7fe700 1 -- 172.21.15.60:0/2332013958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c00038a0 con 0x7f50d80d0a50 2023-07-13T22:25:24.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.944+0000 7f50cf7fe700 1 -- 172.21.15.60:0/2332013958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c001f070 con 0x7f50d80d0a50 2023-07-13T22:25:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.944+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d80ca830 con 0x7f50d80d0a50 2023-07-13T22:25:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.944+0000 7f50cf7fe700 1 -- 172.21.15.60:0/2332013958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0003c20 con 0x7f50d80d0a50 2023-07-13T22:25:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.944+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d80cadf0 con 0x7f50d80d0a50 2023-07-13T22:25:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.945+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d804e280 con 0x7f50d80d0a50 2023-07-13T22:25:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.946+0000 7f50cf7fe700 1 -- 172.21.15.60:0/2332013958 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50c0026480 con 0x7f50d80d0a50 2023-07-13T22:25:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.947+0000 7f50cf7fe700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b806ace0 0x7f50b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.947+0000 7f50cf7fe700 1 -- 172.21.15.60:0/2332013958 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f50c0021070 con 0x7f50d80d0a50 2023-07-13T22:25:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.947+0000 7f50d6ffd700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b806ace0 0x7f50b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.948+0000 7f50d6ffd700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b806ace0 0x7f50b806d190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009930 tx=0x7f50c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:24.951+0000 7f50cf7fe700 1 -- 172.21.15.60:0/2332013958 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f50c0055f70 con 0x7f50d80d0a50 2023-07-13T22:25:25.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.203+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f50d8156ff0 con 0x7f50d80d0a50 2023-07-13T22:25:25.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.204+0000 7f50cf7fe700 1 -- 172.21.15.60:0/2332013958 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f50c0059590 con 0x7f50d80d0a50 2023-07-13T22:25:25.185 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:25.185 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:25:25.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b806ace0 msgr2=0x7f50b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:25.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b806ace0 0x7f50b806d190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009930 tx=0x7f50c8008040 comp rx=0 tx=0).stop 2023-07-13T22:25:25.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80d0a50 msgr2=0x7f50d80c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:25.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80d0a50 0x7f50d80c9ce0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f50c00090b0 tx=0x7f50c0004750 comp rx=0 tx=0).stop 2023-07-13T22:25:25.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 shutdown_connections 2023-07-13T22:25:25.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b806ace0 0x7f50b806d190 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:25.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d80d0e20 0x7f50d80ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:25.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d80cd260 0x7f50d815d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:25.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 --2- 172.21.15.60:0/2332013958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d80d0a50 0x7f50d80c9ce0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:25.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 >> 172.21.15.60:0/2332013958 conn(0x7f50d8071600 msgr2=0x7f50d815bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:25.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 shutdown_connections 2023-07-13T22:25:25.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:25.206+0000 7f50dd3ff700 1 -- 172.21.15.60:0/2332013958 wait complete. 2023-07-13T22:25:25.205 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:25:25.205 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:25:25.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:25:25.885 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:25:30.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:25:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.439+0000 7f895cdff700 1 Processor -- start 2023-07-13T22:25:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.439+0000 7f895cdff700 1 -- start start 2023-07-13T22:25:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.440+0000 7f895cdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89580d0a50 0x7f89580cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.440+0000 7f895cdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f89580d0e20 0x7f89580d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.440+0000 7f895cdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89580d2410 con 0x7f89580cd260 2023-07-13T22:25:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.440+0000 7f895cdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895815c900 con 0x7f89580d0e20 2023-07-13T22:25:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.440+0000 7f895cdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89580d2660 con 0x7f89580d0a50 2023-07-13T22:25:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.440+0000 7f8956d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f89580d0e20 0x7f89580d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f8956d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f89580d0e20 0x7f89580d1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60056/0 (socket says 172.21.15.60:60056) 2023-07-13T22:25:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f8956d9d700 1 -- 172.21.15.60:0/207650752 learned_addr learned my addr 172.21.15.60:0/207650752 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f895659c700 1 --2- 172.21.15.60:0/207650752 >> v2:172.21.15.105:3300/0 conn(0x7f89580d0a50 0x7f89580cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f8956d9d700 1 -- 172.21.15.60:0/207650752 >> v2:172.21.15.105:3300/0 conn(0x7f89580d0a50 msgr2=0x7f89580cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f8956d9d700 1 --2- 172.21.15.60:0/207650752 >> v2:172.21.15.105:3300/0 conn(0x7f89580d0a50 0x7f89580cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f8956d9d700 1 -- 172.21.15.60:0/207650752 >> v1:172.21.15.105:6789/0 conn(0x7f89580cd260 legacy=0x7f89580d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f8956d9d700 1 -- 172.21.15.60:0/207650752 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580d2940 con 0x7f89580d0e20 2023-07-13T22:25:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f8956d9d700 1 --2- 172.21.15.60:0/207650752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0e20 0x7f89580d1240 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f895808be90 tx=0x7f895000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97750e72ae6a4da server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f895559a700 1 -- 172.21.15.60:0/207650752 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000e070 con 0x7f89580d0e20 2023-07-13T22:25:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.441+0000 7f895559a700 1 -- 172.21.15.60:0/207650752 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8950003a20 con 0x7f89580d0e20 2023-07-13T22:25:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.442+0000 7f895559a700 1 -- 172.21.15.60:0/207650752 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950003dd0 con 0x7f89580d0e20 2023-07-13T22:25:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.442+0000 7f895cdff700 1 -- 172.21.15.60:0/207650752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0e20 msgr2=0x7f89580d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.442+0000 7f895cdff700 1 --2- 172.21.15.60:0/207650752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0e20 0x7f89580d1240 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f895808be90 tx=0x7f895000b330 comp rx=0 tx=0).stop 2023-07-13T22:25:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.442+0000 7f895cdff700 1 -- 172.21.15.60:0/207650752 shutdown_connections 2023-07-13T22:25:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.442+0000 7f895cdff700 1 --2- 172.21.15.60:0/207650752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0e20 0x7f89580d1240 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.442+0000 7f895cdff700 1 --2- 172.21.15.60:0/207650752 >> v2:172.21.15.105:3300/0 conn(0x7f89580d0a50 0x7f89580cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.442+0000 7f895cdff700 1 -- 172.21.15.60:0/207650752 >> 172.21.15.60:0/207650752 conn(0x7f8958071600 msgr2=0x7f89580719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.442+0000 7f895cdff700 1 -- 172.21.15.60:0/207650752 shutdown_connections 2023-07-13T22:25:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.442+0000 7f895cdff700 1 -- 172.21.15.60:0/207650752 wait complete. 2023-07-13T22:25:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.444+0000 7f895cdff700 1 Processor -- start 2023-07-13T22:25:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.444+0000 7f895cdff700 1 -- start start 2023-07-13T22:25:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.445+0000 7f895cdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89580cd260 0x7f895815cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.445+0000 7f895cdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0a50 0x7f89580cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.445+0000 7f895cdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89580d0e20 0x7f895814a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.445+0000 7f895cdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89580c7660 con 0x7f89580d0a50 2023-07-13T22:25:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.445+0000 7f895cdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895814b490 con 0x7f89580d0e20 2023-07-13T22:25:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.445+0000 7f895cdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89580c6700 con 0x7f89580cd260 2023-07-13T22:25:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.445+0000 7f8955d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0a50 0x7f89580cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.445+0000 7f8955d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0a50 0x7f89580cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60060/0 (socket says 172.21.15.60:60060) 2023-07-13T22:25:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.445+0000 7f8955d9b700 1 -- 172.21.15.60:0/1491844731 learned_addr learned my addr 172.21.15.60:0/1491844731 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8956d9d700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89580d0e20 0x7f895814a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f895659c700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89580cd260 0x7f895815cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8955d9b700 1 -- 172.21.15.60:0/1491844731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89580cd260 msgr2=0x7f895815cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8955d9b700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89580cd260 0x7f895815cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:30.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8955d9b700 1 -- 172.21.15.60:0/1491844731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89580d0e20 msgr2=0x7f895814a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8955d9b700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89580d0e20 0x7f895814a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8955d9b700 1 -- 172.21.15.60:0/1491844731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009b30 con 0x7f89580d0a50 2023-07-13T22:25:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8955d9b700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0a50 0x7f89580cb9b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f894c00c4d0 tx=0x7f894c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8946ffd700 1 -- 172.21.15.60:0/1491844731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c011790 con 0x7f89580d0a50 2023-07-13T22:25:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8946ffd700 1 -- 172.21.15.60:0/1491844731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c0118f0 con 0x7f89580d0a50 2023-07-13T22:25:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f8946ffd700 1 -- 172.21.15.60:0/1491844731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c011ca0 con 0x7f89580d0a50 2023-07-13T22:25:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89580c69e0 con 0x7f89580d0a50 2023-07-13T22:25:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.446+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89580c6fa0 con 0x7f89580d0a50 2023-07-13T22:25:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.447+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89580c77a0 con 0x7f89580d0a50 2023-07-13T22:25:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.448+0000 7f8946ffd700 1 -- 172.21.15.60:0/1491844731 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f894c01c460 con 0x7f89580d0a50 2023-07-13T22:25:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.449+0000 7f8946ffd700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f893806ace0 0x7f893806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.450+0000 7f8946ffd700 1 -- 172.21.15.60:0/1491844731 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f894c0892d0 con 0x7f89580d0a50 2023-07-13T22:25:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.450+0000 7f895659c700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f893806ace0 0x7f893806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.450+0000 7f895659c700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f893806ace0 0x7f893806d190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8948009930 tx=0x7f8948008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.454+0000 7f8946ffd700 1 -- 172.21.15.60:0/1491844731 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f89580c77a0 con 0x7f89580d0a50 2023-07-13T22:25:30.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.704+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f89580c77a0 con 0x7f89580d0a50 2023-07-13T22:25:30.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.704+0000 7f8946ffd700 1 -- 172.21.15.60:0/1491844731 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f89580c77a0 con 0x7f89580d0a50 2023-07-13T22:25:30.685 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:30.686 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:25:30.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.706+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f893806ace0 msgr2=0x7f893806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:30.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.706+0000 7f895cdff700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f893806ace0 0x7f893806d190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8948009930 tx=0x7f8948008040 comp rx=0 tx=0).stop 2023-07-13T22:25:30.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.706+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0a50 msgr2=0x7f89580cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.706+0000 7f895cdff700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0a50 0x7f89580cb9b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f894c00c4d0 tx=0x7f894c00c890 comp rx=0 tx=0).stop 2023-07-13T22:25:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.707+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 shutdown_connections 2023-07-13T22:25:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.707+0000 7f895cdff700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f893806ace0 0x7f893806d190 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.707+0000 7f895cdff700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89580d0e20 0x7f895814a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.707+0000 7f895cdff700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89580cd260 0x7f895815cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.707+0000 7f895cdff700 1 --2- 172.21.15.60:0/1491844731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89580d0a50 0x7f89580cb9b0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.707+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 >> 172.21.15.60:0/1491844731 conn(0x7f8958071600 msgr2=0x7f89581592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.707+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 shutdown_connections 2023-07-13T22:25:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:30.707+0000 7f895cdff700 1 -- 172.21.15.60:0/1491844731 wait complete. 2023-07-13T22:25:30.705 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:25:30.706 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:25:35.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:25:35.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.942+0000 7fa02e5ff700 1 Processor -- start 2023-07-13T22:25:35.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.942+0000 7fa02e5ff700 1 -- start start 2023-07-13T22:25:35.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.943+0000 7fa02e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa028071e30 0x7fa028160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:35.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.943+0000 7fa02e5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa0280b12a0 0x7fa0280b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:35.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.943+0000 7fa02e5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02806a070 con 0x7fa028072200 2023-07-13T22:25:35.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.943+0000 7fa02e5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0280b1c60 con 0x7fa0280b12a0 2023-07-13T22:25:35.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.943+0000 7fa02e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa028160ff0 con 0x7fa028071e30 2023-07-13T22:25:35.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa0277fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa028072200 0x7fa0280cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40332/0 (socket says 172.21.15.60:40332) 2023-07-13T22:25:35.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa0277fe700 1 -- 172.21.15.60:0/3295108312 learned_addr learned my addr 172.21.15.60:0/3295108312 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:35.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa027fff700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa028071e30 0x7fa028160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:35.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa02cb9c700 1 --2- 172.21.15.60:0/3295108312 >> v2:172.21.15.105:3300/0 conn(0x7fa0280b12a0 0x7fa0280b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:35.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804297005 0 0) 0x7fa02806a070 con 0x7fa028072200 2023-07-13T22:25:35.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0100035b0 con 0x7fa028072200 2023-07-13T22:25:35.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 203220064 0 0) 0x7fa0100035b0 con 0x7fa028072200 2023-07-13T22:25:35.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa028071e30 msgr2=0x7fa028160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:35.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa026ffd700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa028071e30 0x7fa028160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:35.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 >> v2:172.21.15.105:3300/0 conn(0x7fa0280b12a0 msgr2=0x7fa0280b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:35.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa026ffd700 1 --2- 172.21.15.60:0/3295108312 >> v2:172.21.15.105:3300/0 conn(0x7fa0280b12a0 0x7fa0280b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:35.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028160750 con 0x7fa028072200 2023-07-13T22:25:35.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.944+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fa01c001670 con 0x7fa028072200 2023-07-13T22:25:35.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 >> v1:172.21.15.60:6789/0 conn(0x7fa028072200 legacy=0x7fa0280cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:35.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa026ffd700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa010003a00 0x7fa010003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:35.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa026ffd700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa010004430 0x7fa0100048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:35.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa026ffd700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa01000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:35.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280b1c60 con 0x7fa010004f10 2023-07-13T22:25:35.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028160ff0 con 0x7fa010003a00 2023-07-13T22:25:35.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01000bf50 con 0x7fa010004430 2023-07-13T22:25:35.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa027fff700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa028071e30 0x7fa028160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:35.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa02cb9c700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa01000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:35.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa0277fe700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa010004430 0x7fa0100048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:35.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa027fff700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa010003a00 0x7fa010003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:35.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa02cb9c700 1 -- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa010004430 msgr2=0x7fa0100048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:35.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa02cb9c700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa010004430 0x7fa0100048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:35.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa02cb9c700 1 -- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa010003a00 msgr2=0x7fa010003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:35.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa02cb9c700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa010003a00 0x7fa010003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:35.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.945+0000 7fa02cb9c700 1 -- 172.21.15.60:0/3295108312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0100035b0 con 0x7fa010004f10 2023-07-13T22:25:35.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.946+0000 7fa02cb9c700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa01000ba10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa020009fc0 tx=0x7fa010005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2282334919e1d4cc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:35.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.946+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000ec30 con 0x7fa010004f10 2023-07-13T22:25:35.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.946+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02000f040 con 0x7fa010004f10 2023-07-13T22:25:35.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.946+0000 7fa026ffd700 1 -- 172.21.15.60:0/3295108312 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020014480 con 0x7fa010004f10 2023-07-13T22:25:35.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.946+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3295108312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 msgr2=0x7fa01000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:35.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.946+0000 7fa02e5ff700 1 --2- 172.21.15.60:0/3295108312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa01000ba10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa020009fc0 tx=0x7fa010005880 comp rx=0 tx=0).stop 2023-07-13T22:25:35.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.946+0000 7fa02cb9c700 1 -- 172.21.15.60:0/3295108312 reap_dead start 2023-07-13T22:25:35.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.946+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3295108312 shutdown_connections 2023-07-13T22:25:35.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.946+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3295108312 >> 172.21.15.60:0/3295108312 conn(0x7fa0280cc2a0 msgr2=0x7fa0280cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:35.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.947+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3295108312 shutdown_connections 2023-07-13T22:25:35.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.947+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3295108312 wait complete. 2023-07-13T22:25:35.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.948+0000 7fa02e5ff700 1 Processor -- start 2023-07-13T22:25:35.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.948+0000 7fa02e5ff700 1 -- start start 2023-07-13T22:25:35.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.949+0000 7fa02e5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa0280ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:35.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.949+0000 7fa02e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa028154540 0x7fa0281e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:35.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.949+0000 7fa02e5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0281e6920 0x7fa02815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:35.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.949+0000 7fa02e5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0281e6e30 con 0x7fa010004f10 2023-07-13T22:25:35.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.949+0000 7fa02e5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0281e7110 con 0x7fa0281e6920 2023-07-13T22:25:35.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.949+0000 7fa02e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280b4220 con 0x7fa028154540 2023-07-13T22:25:35.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.950+0000 7fa027fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa0280ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:35.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.950+0000 7fa027fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa0280ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39862/0 (socket says 172.21.15.60:39862) 2023-07-13T22:25:35.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.950+0000 7fa027fff700 1 -- 172.21.15.60:0/3779172778 learned_addr learned my addr 172.21.15.60:0/3779172778 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:35.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.950+0000 7fa027fff700 1 -- 172.21.15.60:0/3779172778 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa028154540 msgr2=0x7fa0281e63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:35.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.950+0000 7fa027fff700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa028154540 0x7fa0281e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:35.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.950+0000 7fa027fff700 1 -- 172.21.15.60:0/3779172778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0281e6920 msgr2=0x7fa02815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:35.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.950+0000 7fa027fff700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0281e6920 0x7fa02815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:35.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.950+0000 7fa027fff700 1 -- 172.21.15.60:0/3779172778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0280b4500 con 0x7fa010004f10 2023-07-13T22:25:35.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.950+0000 7fa027fff700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa0280ab8f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa018011c00 tx=0x7fa018009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:35.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.951+0000 7fa024ff9700 1 -- 172.21.15.60:0/3779172778 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800faa0 con 0x7fa010004f10 2023-07-13T22:25:35.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.951+0000 7fa024ff9700 1 -- 172.21.15.60:0/3779172778 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01800fc00 con 0x7fa010004f10 2023-07-13T22:25:35.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.951+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0280abe30 con 0x7fa010004f10 2023-07-13T22:25:35.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.951+0000 7fa024ff9700 1 -- 172.21.15.60:0/3779172778 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018014440 con 0x7fa010004f10 2023-07-13T22:25:35.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.951+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0280b3980 con 0x7fa010004f10 2023-07-13T22:25:35.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.952+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0280b4690 con 0x7fa010004f10 2023-07-13T22:25:35.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.953+0000 7fa024ff9700 1 -- 172.21.15.60:0/3779172778 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa0180145a0 con 0x7fa010004f10 2023-07-13T22:25:35.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.954+0000 7fa024ff9700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01406ace0 0x7fa01406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:35.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.954+0000 7fa024ff9700 1 -- 172.21.15.60:0/3779172778 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa018089d10 con 0x7fa010004f10 2023-07-13T22:25:35.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.954+0000 7fa0277fe700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01406ace0 0x7fa01406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:35.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.955+0000 7fa0277fe700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01406ace0 0x7fa01406d190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0011e0 tx=0x7fa01c001170 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:35.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:35.958+0000 7fa024ff9700 1 -- 172.21.15.60:0/3779172778 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa01808f020 con 0x7fa010004f10 2023-07-13T22:25:36.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.209+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa0280b4870 con 0x7fa010004f10 2023-07-13T22:25:36.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.209+0000 7fa024ff9700 1 -- 172.21.15.60:0/3779172778 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fa018054db0 con 0x7fa010004f10 2023-07-13T22:25:36.190 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:36.191 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:25:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01406ace0 msgr2=0x7fa01406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01406ace0 0x7fa01406d190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0011e0 tx=0x7fa01c001170 comp rx=0 tx=0).stop 2023-07-13T22:25:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 msgr2=0x7fa0280ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa0280ab8f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa018011c00 tx=0x7fa018009ec0 comp rx=0 tx=0).stop 2023-07-13T22:25:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 shutdown_connections 2023-07-13T22:25:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa01406ace0 0x7fa01406d190 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa010004f10 0x7fa0280ab8f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0281e6920 0x7fa02815ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 --2- 172.21.15.60:0/3779172778 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa028154540 0x7fa0281e63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.211+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 >> 172.21.15.60:0/3779172778 conn(0x7fa0280cc2a0 msgr2=0x7fa02806a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.212+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 shutdown_connections 2023-07-13T22:25:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:36.212+0000 7fa02e5ff700 1 -- 172.21.15.60:0/3779172778 wait complete. 2023-07-13T22:25:36.211 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:25:36.211 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:25:41.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:25:41.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.444+0000 7f1fec9ff700 1 Processor -- start 2023-07-13T22:25:41.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.445+0000 7f1fec9ff700 1 -- start start 2023-07-13T22:25:41.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.446+0000 7f1fec9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fe80d0a50 0x7f1fe80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:41.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.446+0000 7f1fec9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe80d2410 con 0x7f1fe80cd260 2023-07-13T22:25:41.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.446+0000 7f1fec9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe80d2660 con 0x7f1fe80d0e20 2023-07-13T22:25:41.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.446+0000 7f1fec9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe815c900 con 0x7f1fe80d0a50 2023-07-13T22:25:41.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.446+0000 7f1fe659c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fe80d0a50 0x7f1fe80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:41.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.446+0000 7f1fdffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1fe80cd260 0x7f1fe80d3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40338/0 (socket says 172.21.15.60:40338) 2023-07-13T22:25:41.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.446+0000 7f1fe659c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fe80d0a50 0x7f1fe80cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39876/0 (socket says 172.21.15.60:39876) 2023-07-13T22:25:41.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.446+0000 7f1fdffff700 1 -- 172.21.15.60:0/3359276331 learned_addr learned my addr 172.21.15.60:0/3359276331 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:41.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.447+0000 7f1fe659c700 1 -- 172.21.15.60:0/3359276331 >> v1:172.21.15.105:6789/0 conn(0x7f1fe80d0e20 legacy=0x7f1fe80b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:41.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.447+0000 7f1fe659c700 1 -- 172.21.15.60:0/3359276331 >> v1:172.21.15.60:6789/0 conn(0x7f1fe80cd260 legacy=0x7f1fe80d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:41.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.447+0000 7f1fe659c700 1 -- 172.21.15.60:0/3359276331 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80d2940 con 0x7f1fe80d0a50 2023-07-13T22:25:41.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.447+0000 7f1fe5d9b700 1 -- 172.21.15.60:0/3359276331 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205141189 0 0) 0x7f1fe80d2410 con 0x7f1fe80cd260 2023-07-13T22:25:41.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.447+0000 7f1fe659c700 1 --2- 172.21.15.60:0/3359276331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 0x7f1fe80cf8a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009840 tx=0x7f1fd8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752b37ad7bfda115 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:41.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.447+0000 7f1fe5d9b700 1 -- 172.21.15.60:0/3359276331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011830 con 0x7f1fe80d0a50 2023-07-13T22:25:41.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.447+0000 7f1fe5d9b700 1 -- 172.21.15.60:0/3359276331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8011990 con 0x7f1fe80d0a50 2023-07-13T22:25:41.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.447+0000 7f1fe5d9b700 1 -- 172.21.15.60:0/3359276331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011d20 con 0x7f1fe80d0a50 2023-07-13T22:25:41.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.448+0000 7f1fec9ff700 1 -- 172.21.15.60:0/3359276331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 msgr2=0x7f1fe80cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:41.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.448+0000 7f1fec9ff700 1 --2- 172.21.15.60:0/3359276331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 0x7f1fe80cf8a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009840 tx=0x7f1fd8009b50 comp rx=0 tx=0).stop 2023-07-13T22:25:41.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.448+0000 7f1fec9ff700 1 -- 172.21.15.60:0/3359276331 shutdown_connections 2023-07-13T22:25:41.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.448+0000 7f1fec9ff700 1 --2- 172.21.15.60:0/3359276331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 0x7f1fe80cf8a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:41.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.448+0000 7f1fec9ff700 1 -- 172.21.15.60:0/3359276331 >> 172.21.15.60:0/3359276331 conn(0x7f1fe8071600 msgr2=0x7f1fe80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:41.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.448+0000 7f1fec9ff700 1 -- 172.21.15.60:0/3359276331 shutdown_connections 2023-07-13T22:25:41.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.448+0000 7f1fec9ff700 1 -- 172.21.15.60:0/3359276331 wait complete. 2023-07-13T22:25:41.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.449+0000 7f1fec9ff700 1 Processor -- start 2023-07-13T22:25:41.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.450+0000 7f1fec9ff700 1 -- start start 2023-07-13T22:25:41.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fec9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1fe80cd260 0x7f1fe81565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:41.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fec9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 0x7f1fe80c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:41.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fec9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1fe80d0e20 0x7f1fe815d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:41.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fec9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c98a0 con 0x7f1fe80d0a50 2023-07-13T22:25:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fec9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8156b00 con 0x7f1fe80cd260 2023-07-13T22:25:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fec9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c6c40 con 0x7f1fe80d0e20 2023-07-13T22:25:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fdffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 0x7f1fe80c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fdffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 0x7f1fe80c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39880/0 (socket says 172.21.15.60:39880) 2023-07-13T22:25:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fdffff700 1 -- 172.21.15.60:0/4018695393 learned_addr learned my addr 172.21.15.60:0/4018695393 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fe659c700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1fe80cd260 0x7f1fe81565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fe6d9d700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1fe80d0e20 0x7f1fe815d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fdffff700 1 -- 172.21.15.60:0/4018695393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1fe80d0e20 msgr2=0x7f1fe815d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:41.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fdffff700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1fe80d0e20 0x7f1fe815d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:41.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fdffff700 1 -- 172.21.15.60:0/4018695393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1fe80cd260 msgr2=0x7f1fe81565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:41.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fdffff700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1fe80cd260 0x7f1fe81565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:41.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fdffff700 1 -- 172.21.15.60:0/4018695393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80094a0 con 0x7f1fe80d0a50 2023-07-13T22:25:41.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.451+0000 7f1fe6d9d700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1fe80d0e20 0x7f1fe815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:41.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.452+0000 7f1fdffff700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 0x7f1fe80c6700 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c550 tx=0x7f1fd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:41.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.452+0000 7f1fdeffd700 1 -- 172.21.15.60:0/4018695393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0004c80 con 0x7f1fe80d0a50 2023-07-13T22:25:41.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.452+0000 7f1fdeffd700 1 -- 172.21.15.60:0/4018695393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0004e00 con 0x7f1fe80d0a50 2023-07-13T22:25:41.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.452+0000 7f1fdeffd700 1 -- 172.21.15.60:0/4018695393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0007310 con 0x7f1fe80d0a50 2023-07-13T22:25:41.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.452+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe81f1b30 con 0x7f1fe80d0a50 2023-07-13T22:25:41.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.452+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe81f2090 con 0x7f1fe80d0a50 2023-07-13T22:25:41.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.453+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe804e280 con 0x7f1fe80d0a50 2023-07-13T22:25:41.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.454+0000 7f1fdeffd700 1 -- 172.21.15.60:0/4018695393 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1fd0007470 con 0x7f1fe80d0a50 2023-07-13T22:25:41.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.455+0000 7f1fdeffd700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1fc806ace0 0x7f1fc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:41.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.455+0000 7f1fdeffd700 1 -- 172.21.15.60:0/4018695393 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1fd0089240 con 0x7f1fe80d0a50 2023-07-13T22:25:41.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.455+0000 7f1fe659c700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1fc806ace0 0x7f1fc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:41.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.456+0000 7f1fe659c700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1fc806ace0 0x7f1fc806d190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80ccee0 tx=0x7f1fd801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:41.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.459+0000 7f1fdeffd700 1 -- 172.21.15.60:0/4018695393 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1fd0054250 con 0x7f1fe80d0a50 2023-07-13T22:25:41.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.706+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1fe815f1a0 con 0x7f1fe80d0a50 2023-07-13T22:25:41.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.706+0000 7f1fdeffd700 1 -- 172.21.15.60:0/4018695393 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f1fd0057870 con 0x7f1fe80d0a50 2023-07-13T22:25:41.688 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:41.688 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:25:41.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1fc806ace0 msgr2=0x7f1fc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:41.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1fc806ace0 0x7f1fc806d190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80ccee0 tx=0x7f1fd801c040 comp rx=0 tx=0).stop 2023-07-13T22:25:41.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 msgr2=0x7f1fe80c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:41.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 0x7f1fe80c6700 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c550 tx=0x7f1fd000c910 comp rx=0 tx=0).stop 2023-07-13T22:25:41.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 shutdown_connections 2023-07-13T22:25:41.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1fc806ace0 0x7f1fc806d190 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:41.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1fe80d0e20 0x7f1fe815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:41.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1fe80cd260 0x7f1fe81565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:41.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 --2- 172.21.15.60:0/4018695393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fe80d0a50 0x7f1fe80c6700 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:41.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 >> 172.21.15.60:0/4018695393 conn(0x7f1fe8071600 msgr2=0x7f1fe815f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:41.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 shutdown_connections 2023-07-13T22:25:41.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:41.709+0000 7f1fec9ff700 1 -- 172.21.15.60:0/4018695393 wait complete. 2023-07-13T22:25:41.707 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:25:41.708 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:25:46.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:25:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.941+0000 7f18687ff700 1 Processor -- start 2023-07-13T22:25:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.942+0000 7f18687ff700 1 -- start start 2023-07-13T22:25:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f18687ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1860072200 0x7f1860160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f18687ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18600b12a0 0x7f18600b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f18687ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600b1c40 con 0x7f1860071e30 2023-07-13T22:25:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f18687ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186006a070 con 0x7f1860072200 2023-07-13T22:25:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f18687ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1860160ff0 con 0x7f18600b12a0 2023-07-13T22:25:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f1865d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1860072200 0x7f1860160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f186659b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1860071e30 0x7f18600cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35942/0 (socket says 172.21.15.60:35942) 2023-07-13T22:25:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f186659b700 1 -- 172.21.15.60:0/3847629228 learned_addr learned my addr 172.21.15.60:0/3847629228 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:25:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f1865d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1860072200 0x7f1860160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60538/0 (socket says 172.21.15.60:60538) 2023-07-13T22:25:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.943+0000 7f1866d9c700 1 --2- 172.21.15.60:0/3847629228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18600b12a0 0x7f18600b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1865d9a700 1 -- 172.21.15.60:0/3847629228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18600b12a0 msgr2=0x7f18600b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1865d9a700 1 --2- 172.21.15.60:0/3847629228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18600b12a0 0x7f18600b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1865d9a700 1 -- 172.21.15.60:0/3847629228 >> v1:172.21.15.60:6789/0 conn(0x7f1860071e30 legacy=0x7f18600cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1865d9a700 1 -- 172.21.15.60:0/3847629228 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860160750 con 0x7f1860072200 2023-07-13T22:25:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1866d9c700 1 --2- 172.21.15.60:0/3847629228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18600b12a0 0x7f18600b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:46.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1865599700 1 -- 172.21.15.60:0/3847629228 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692945305 0 0) 0x7f18600b1c40 con 0x7f1860071e30 2023-07-13T22:25:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1865d9a700 1 --2- 172.21.15.60:0/3847629228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 0x7f1860160090 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1850009840 tx=0x7f1850009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd25ca3a436bd7be server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1865599700 1 -- 172.21.15.60:0/3847629228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850011830 con 0x7f1860072200 2023-07-13T22:25:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1865599700 1 -- 172.21.15.60:0/3847629228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1850011990 con 0x7f1860072200 2023-07-13T22:25:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.944+0000 7f1865599700 1 -- 172.21.15.60:0/3847629228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850011d20 con 0x7f1860072200 2023-07-13T22:25:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.945+0000 7f18687ff700 1 -- 172.21.15.60:0/3847629228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 msgr2=0x7f1860160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.945+0000 7f18687ff700 1 --2- 172.21.15.60:0/3847629228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 0x7f1860160090 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f1850009840 tx=0x7f1850009b50 comp rx=0 tx=0).stop 2023-07-13T22:25:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.945+0000 7f18687ff700 1 -- 172.21.15.60:0/3847629228 shutdown_connections 2023-07-13T22:25:46.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.945+0000 7f18687ff700 1 --2- 172.21.15.60:0/3847629228 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18600b12a0 0x7f18600b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.945+0000 7f18687ff700 1 --2- 172.21.15.60:0/3847629228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 0x7f1860160090 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.945+0000 7f18687ff700 1 -- 172.21.15.60:0/3847629228 >> 172.21.15.60:0/3847629228 conn(0x7f18600cc2a0 msgr2=0x7f18600cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.945+0000 7f18687ff700 1 -- 172.21.15.60:0/3847629228 shutdown_connections 2023-07-13T22:25:46.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.945+0000 7f18687ff700 1 -- 172.21.15.60:0/3847629228 wait complete. 2023-07-13T22:25:46.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.947+0000 7f18687ff700 1 Processor -- start 2023-07-13T22:25:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.947+0000 7f18687ff700 1 -- start start 2023-07-13T22:25:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f18687ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1860071e30 0x7f1860154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f18687ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 0x7f186015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f18687ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18600b12a0 0x7f18601582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f18687ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600b48f0 con 0x7f1860072200 2023-07-13T22:25:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f18687ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1860157e30 con 0x7f18600b12a0 2023-07-13T22:25:46.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f18687ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1860155320 con 0x7f1860071e30 2023-07-13T22:25:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f1865d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 0x7f186015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f1865d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 0x7f186015db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60552/0 (socket says 172.21.15.60:60552) 2023-07-13T22:25:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f1865d9a700 1 -- 172.21.15.60:0/3050031961 learned_addr learned my addr 172.21.15.60:0/3050031961 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f1865d9a700 1 -- 172.21.15.60:0/3050031961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1860071e30 msgr2=0x7f1860154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f1865d9a700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1860071e30 0x7f1860154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f1865d9a700 1 -- 172.21.15.60:0/3050031961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18600b12a0 msgr2=0x7f18601582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f1865d9a700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18600b12a0 0x7f18601582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.948+0000 7f1865d9a700 1 -- 172.21.15.60:0/3050031961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18500094a0 con 0x7f1860072200 2023-07-13T22:25:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.949+0000 7f1865d9a700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 0x7f186015db20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f1850000c00 tx=0x7f1850004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.949+0000 7f1856ffd700 1 -- 172.21.15.60:0/3050031961 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18500045f0 con 0x7f1860072200 2023-07-13T22:25:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.949+0000 7f1856ffd700 1 -- 172.21.15.60:0/3050031961 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185001f070 con 0x7f1860072200 2023-07-13T22:25:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.949+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1860157500 con 0x7f1860072200 2023-07-13T22:25:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.949+0000 7f1856ffd700 1 -- 172.21.15.60:0/3050031961 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850003ae0 con 0x7f1860072200 2023-07-13T22:25:46.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.949+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1860157ac0 con 0x7f1860072200 2023-07-13T22:25:46.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.950+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186004e280 con 0x7f1860072200 2023-07-13T22:25:46.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.951+0000 7f1856ffd700 1 -- 172.21.15.60:0/3050031961 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1850003c40 con 0x7f1860072200 2023-07-13T22:25:46.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.952+0000 7f1856ffd700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f184406ace0 0x7f184406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.952+0000 7f1856ffd700 1 -- 172.21.15.60:0/3050031961 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1850021070 con 0x7f1860072200 2023-07-13T22:25:46.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.952+0000 7f186659b700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f184406ace0 0x7f184406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.953+0000 7f186659b700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f184406ace0 0x7f184406d190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f185c001610 tx=0x7f185c0079f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:46.957+0000 7f1856ffd700 1 -- 172.21.15.60:0/3050031961 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1850055fd0 con 0x7f1860072200 2023-07-13T22:25:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.206+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1860157fa0 con 0x7f1860072200 2023-07-13T22:25:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.206+0000 7f1856ffd700 1 -- 172.21.15.60:0/3050031961 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f1850015070 con 0x7f1860072200 2023-07-13T22:25:47.188 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:47.188 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:25:47.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f184406ace0 msgr2=0x7f184406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f184406ace0 0x7f184406d190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f185c001610 tx=0x7f185c0079f0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 msgr2=0x7f186015db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 0x7f186015db20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f1850000c00 tx=0x7f1850004750 comp rx=0 tx=0).stop 2023-07-13T22:25:47.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 shutdown_connections 2023-07-13T22:25:47.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f184406ace0 0x7f184406d190 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18600b12a0 0x7f18601582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1860071e30 0x7f1860154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 --2- 172.21.15.60:0/3050031961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1860072200 0x7f186015db20 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 >> 172.21.15.60:0/3050031961 conn(0x7f18600cc2a0 msgr2=0x7f186006eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 shutdown_connections 2023-07-13T22:25:47.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:47.209+0000 7f18687ff700 1 -- 172.21.15.60:0/3050031961 wait complete. 2023-07-13T22:25:47.208 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:25:47.209 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:25:52.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.441+0000 7f9aef3ff700 1 Processor -- start 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.441+0000 7f9aef3ff700 1 -- start start 2023-07-13T22:25:52.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aef3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 0x7f9ae8160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:52.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aef3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ae8072200 0x7f9ae80cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aef3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9ae80b12a0 0x7f9ae80b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aef3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae80b1c40 con 0x7f9ae8072200 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aef3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae806a070 con 0x7f9ae80b12a0 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aef3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8160ff0 con 0x7f9ae8071e30 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aec99a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ae8072200 0x7f9ae80cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aec99a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ae8072200 0x7f9ae80cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60556/0 (socket says 172.21.15.60:60556) 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aec99a700 1 -- 172.21.15.60:0/3862494146 learned_addr learned my addr 172.21.15.60:0/3862494146 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aed19b700 1 --2- 172.21.15.60:0/3862494146 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 0x7f9ae8160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aed99c700 1 --2- 172.21.15.60:0/3862494146 >> v2:172.21.15.105:3300/0 conn(0x7f9ae80b12a0 0x7f9ae80b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aec99a700 1 -- 172.21.15.60:0/3862494146 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 msgr2=0x7f9ae8160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aec99a700 1 --2- 172.21.15.60:0/3862494146 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 0x7f9ae8160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aec99a700 1 -- 172.21.15.60:0/3862494146 >> v2:172.21.15.105:3300/0 conn(0x7f9ae80b12a0 msgr2=0x7f9ae80b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aec99a700 1 --2- 172.21.15.60:0/3862494146 >> v2:172.21.15.105:3300/0 conn(0x7f9ae80b12a0 0x7f9ae80b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.443+0000 7f9aec99a700 1 -- 172.21.15.60:0/3862494146 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80cdcf0 con 0x7f9ae8072200 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.444+0000 7f9aec99a700 1 --2- 172.21.15.60:0/3862494146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 0x7f9ae80cd7b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80cc250 tx=0x7f9ad8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2e6cbf87121e79 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.444+0000 7f9adffff700 1 -- 172.21.15.60:0/3862494146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8011830 con 0x7f9ae8072200 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.444+0000 7f9adffff700 1 -- 172.21.15.60:0/3862494146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad8011990 con 0x7f9ae8072200 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.444+0000 7f9adffff700 1 -- 172.21.15.60:0/3862494146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8011d20 con 0x7f9ae8072200 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.445+0000 7f9aef3ff700 1 -- 172.21.15.60:0/3862494146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 msgr2=0x7f9ae80cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.445+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/3862494146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 0x7f9ae80cd7b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80cc250 tx=0x7f9ad8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.445+0000 7f9aef3ff700 1 -- 172.21.15.60:0/3862494146 shutdown_connections 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.445+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/3862494146 >> v2:172.21.15.105:3300/0 conn(0x7f9ae80b12a0 0x7f9ae80b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.445+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/3862494146 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 0x7f9ae8160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.445+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/3862494146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 0x7f9ae80cd7b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.445+0000 7f9aef3ff700 1 -- 172.21.15.60:0/3862494146 >> 172.21.15.60:0/3862494146 conn(0x7f9ae80cc2a0 msgr2=0x7f9ae80cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.445+0000 7f9aef3ff700 1 -- 172.21.15.60:0/3862494146 shutdown_connections 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.445+0000 7f9aef3ff700 1 -- 172.21.15.60:0/3862494146 wait complete. 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.446+0000 7f9aef3ff700 1 Processor -- start 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.447+0000 7f9aef3ff700 1 -- start start 2023-07-13T22:25:52.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aef3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 0x7f9ae815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aef3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 0x7f9ae815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aef3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80b12a0 0x7f9ae8150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aef3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae815e560 con 0x7f9ae8072200 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aef3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae815dfa0 con 0x7f9ae80b12a0 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aef3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae815e280 con 0x7f9ae8071e30 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aec99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 0x7f9ae815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aec99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 0x7f9ae815d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60560/0 (socket says 172.21.15.60:60560) 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aec99a700 1 -- 172.21.15.60:0/806428336 learned_addr learned my addr 172.21.15.60:0/806428336 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aed19b700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 0x7f9ae815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aed99c700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80b12a0 0x7f9ae8150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aec99a700 1 -- 172.21.15.60:0/806428336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 msgr2=0x7f9ae815ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aec99a700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 0x7f9ae815ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aec99a700 1 -- 172.21.15.60:0/806428336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80b12a0 msgr2=0x7f9ae8150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aec99a700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80b12a0 0x7f9ae8150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aec99a700 1 -- 172.21.15.60:0/806428336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad8009550 con 0x7f9ae8072200 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.448+0000 7f9aed99c700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80b12a0 0x7f9ae8150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.449+0000 7f9aec99a700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 0x7f9ae815d520 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009c10 tx=0x7f9ad8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.449+0000 7f9addffb700 1 -- 172.21.15.60:0/806428336 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8004450 con 0x7f9ae8072200 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.449+0000 7f9addffb700 1 -- 172.21.15.60:0/806428336 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad801e070 con 0x7f9ae8072200 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.449+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae80aba00 con 0x7f9ae8072200 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.449+0000 7f9addffb700 1 -- 172.21.15.60:0/806428336 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad80038e0 con 0x7f9ae8072200 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.449+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae8150ed0 con 0x7f9ae8072200 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.450+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae80abf20 con 0x7f9ae8072200 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.451+0000 7f9addffb700 1 -- 172.21.15.60:0/806428336 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ad8003a40 con 0x7f9ae8072200 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.452+0000 7f9addffb700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9acc06ace0 0x7f9acc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.452+0000 7f9addffb700 1 -- 172.21.15.60:0/806428336 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ad808a260 con 0x7f9ae8072200 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.453+0000 7f9aed19b700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9acc06ace0 0x7f9acc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.453+0000 7f9aed19b700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9acc06ace0 0x7f9acc06d190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009b40 tx=0x7f9ae4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.457+0000 7f9addffb700 1 -- 172.21.15.60:0/806428336 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9ae80abf20 con 0x7f9ae8072200 2023-07-13T22:25:52.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.709+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9ae80abf20 con 0x7f9ae8072200 2023-07-13T22:25:52.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.710+0000 7f9addffb700 1 -- 172.21.15.60:0/806428336 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f9ae80abf20 con 0x7f9ae8072200 2023-07-13T22:25:52.692 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:52.692 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9acc06ace0 msgr2=0x7f9acc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9acc06ace0 0x7f9acc06d190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009b40 tx=0x7f9ae4009210 comp rx=0 tx=0).stop 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 msgr2=0x7f9ae815d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 0x7f9ae815d520 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009c10 tx=0x7f9ad8004750 comp rx=0 tx=0).stop 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 shutdown_connections 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9acc06ace0 0x7f9acc06d190 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80b12a0 0x7f9ae8150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ae8071e30 0x7f9ae815ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 --2- 172.21.15.60:0/806428336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ae8072200 0x7f9ae815d520 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 >> 172.21.15.60:0/806428336 conn(0x7f9ae80cc2a0 msgr2=0x7f9ae80b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:52.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 shutdown_connections 2023-07-13T22:25:52.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:52.712+0000 7f9aef3ff700 1 -- 172.21.15.60:0/806428336 wait complete. 2023-07-13T22:25:52.711 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:25:52.711 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:25:56.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:25:56.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:25:57.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:25:57.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.946+0000 7f7e5b5ff700 1 Processor -- start 2023-07-13T22:25:57.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.946+0000 7f7e5b5ff700 1 -- start start 2023-07-13T22:25:57.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.947+0000 7f7e5b5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7e540d09e0 0x7f7e540cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:57.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.947+0000 7f7e5b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540d0db0 0x7f7e540d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.947+0000 7f7e5b5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e540d2ed0 con 0x7f7e540cd1f0 2023-07-13T22:25:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e5b5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e540d3a70 con 0x7f7e540d09e0 2023-07-13T22:25:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e5b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540cd960 con 0x7f7e540d0db0 2023-07-13T22:25:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e5939b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7e540d09e0 0x7f7e540cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e5939b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7e540d09e0 0x7f7e540cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44018/0 (socket says 172.21.15.60:44018) 2023-07-13T22:25:57.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e5939b700 1 -- 172.21.15.60:0/61636155 learned_addr learned my addr 172.21.15.60:0/61636155 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:57.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e59b9c700 1 --2- 172.21.15.60:0/61636155 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540d0db0 0x7f7e540d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:57.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e5939b700 1 -- 172.21.15.60:0/61636155 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540d0db0 msgr2=0x7f7e540d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:57.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e5939b700 1 --2- 172.21.15.60:0/61636155 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540d0db0 0x7f7e540d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:57.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e5939b700 1 -- 172.21.15.60:0/61636155 >> v1:172.21.15.105:6789/0 conn(0x7f7e540cd1f0 legacy=0x7f7e540d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:57.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.948+0000 7f7e5939b700 1 -- 172.21.15.60:0/61636155 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e540cd680 con 0x7f7e540d09e0 2023-07-13T22:25:57.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e5939b700 1 --2- 172.21.15.60:0/61636155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d09e0 0x7f7e540cf900 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7e500098f0 tx=0x7f7e50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24a6385da71beee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e4bfff700 1 -- 172.21.15.60:0/61636155 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000b8a0 con 0x7f7e540d09e0 2023-07-13T22:25:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e4bfff700 1 -- 172.21.15.60:0/61636155 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5000ba00 con 0x7f7e540d09e0 2023-07-13T22:25:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e4bfff700 1 -- 172.21.15.60:0/61636155 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000bd90 con 0x7f7e540d09e0 2023-07-13T22:25:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/61636155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d09e0 msgr2=0x7f7e540cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e5b5ff700 1 --2- 172.21.15.60:0/61636155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d09e0 0x7f7e540cf900 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7e500098f0 tx=0x7f7e50009c00 comp rx=0 tx=0).stop 2023-07-13T22:25:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/61636155 shutdown_connections 2023-07-13T22:25:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e5b5ff700 1 --2- 172.21.15.60:0/61636155 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540d0db0 0x7f7e540d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e5b5ff700 1 --2- 172.21.15.60:0/61636155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d09e0 0x7f7e540cf900 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.949+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/61636155 >> 172.21.15.60:0/61636155 conn(0x7f7e54071600 msgr2=0x7f7e540719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.950+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/61636155 shutdown_connections 2023-07-13T22:25:57.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.950+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/61636155 wait complete. 2023-07-13T22:25:57.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.951+0000 7f7e5b5ff700 1 Processor -- start 2023-07-13T22:25:57.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.951+0000 7f7e5b5ff700 1 -- start start 2023-07-13T22:25:57.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.952+0000 7f7e5b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540cd1f0 0x7f7e540c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:57.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.952+0000 7f7e5b5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e540d09e0 0x7f7e54158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.952+0000 7f7e5b5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d0db0 0x7f7e541e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.952+0000 7f7e5b5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5415e430 con 0x7f7e540d0db0 2023-07-13T22:25:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.952+0000 7f7e5b5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5415db00 con 0x7f7e540d09e0 2023-07-13T22:25:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.952+0000 7f7e5b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5415dde0 con 0x7f7e540cd1f0 2023-07-13T22:25:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e59b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d0db0 0x7f7e541e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e59b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d0db0 0x7f7e541e82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44022/0 (socket says 172.21.15.60:44022) 2023-07-13T22:25:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e59b9c700 1 -- 172.21.15.60:0/4251183106 learned_addr learned my addr 172.21.15.60:0/4251183106 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:25:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e5939b700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540cd1f0 0x7f7e540c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e58b9a700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e540d09e0 0x7f7e54158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e59b9c700 1 -- 172.21.15.60:0/4251183106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540cd1f0 msgr2=0x7f7e540c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e59b9c700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540cd1f0 0x7f7e540c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e59b9c700 1 -- 172.21.15.60:0/4251183106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e540d09e0 msgr2=0x7f7e54158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e59b9c700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e540d09e0 0x7f7e54158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e59b9c700 1 -- 172.21.15.60:0/4251183106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e50009550 con 0x7f7e540d0db0 2023-07-13T22:25:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.953+0000 7f7e59b9c700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d0db0 0x7f7e541e82b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c007e40 tx=0x7f7e4c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.954+0000 7f7e49ffb700 1 -- 172.21.15.60:0/4251183106 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00ed30 con 0x7f7e540d0db0 2023-07-13T22:25:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.954+0000 7f7e49ffb700 1 -- 172.21.15.60:0/4251183106 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c014420 con 0x7f7e540d0db0 2023-07-13T22:25:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.954+0000 7f7e49ffb700 1 -- 172.21.15.60:0/4251183106 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0147d0 con 0x7f7e540d0db0 2023-07-13T22:25:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.954+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5415e0c0 con 0x7f7e540d0db0 2023-07-13T22:25:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.954+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e541e87f0 con 0x7f7e540d0db0 2023-07-13T22:25:57.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.955+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5415e5a0 con 0x7f7e540d0db0 2023-07-13T22:25:57.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.956+0000 7f7e49ffb700 1 -- 172.21.15.60:0/4251183106 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e4c014930 con 0x7f7e540d0db0 2023-07-13T22:25:57.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.957+0000 7f7e49ffb700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e3806ace0 0x7f7e3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.957+0000 7f7e49ffb700 1 -- 172.21.15.60:0/4251183106 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e4c089c10 con 0x7f7e540d0db0 2023-07-13T22:25:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.957+0000 7f7e5939b700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e3806ace0 0x7f7e3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.957+0000 7f7e5939b700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e3806ace0 0x7f7e3806d190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009c40 tx=0x7f7e50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:57.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:57.961+0000 7f7e49ffb700 1 -- 172.21.15.60:0/4251183106 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7e4c054cd0 con 0x7f7e540d0db0 2023-07-13T22:25:58.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.212+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7e5404e280 con 0x7f7e540d0db0 2023-07-13T22:25:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.212+0000 7f7e49ffb700 1 -- 172.21.15.60:0/4251183106 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f7e4c0582f0 con 0x7f7e540d0db0 2023-07-13T22:25:58.194 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:25:58.195 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:25:58.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e3806ace0 msgr2=0x7f7e3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:58.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e3806ace0 0x7f7e3806d190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009c40 tx=0x7f7e50008030 comp rx=0 tx=0).stop 2023-07-13T22:25:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d0db0 msgr2=0x7f7e541e82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d0db0 0x7f7e541e82b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c007e40 tx=0x7f7e4c00e3c0 comp rx=0 tx=0).stop 2023-07-13T22:25:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 shutdown_connections 2023-07-13T22:25:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e3806ace0 0x7f7e3806d190 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e540d0db0 0x7f7e541e82b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e540cd1f0 0x7f7e540c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 --2- 172.21.15.60:0/4251183106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e540d09e0 0x7f7e54158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.214+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 >> 172.21.15.60:0/4251183106 conn(0x7f7e54071600 msgr2=0x7f7e540b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.215+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 shutdown_connections 2023-07-13T22:25:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:25:58.215+0000 7f7e5b5ff700 1 -- 172.21.15.60:0/4251183106 wait complete. 2023-07-13T22:25:58.215 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:25:58.216 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:03.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:03.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.447+0000 7f7a8d5ff700 1 Processor -- start 2023-07-13T22:26:03.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.447+0000 7f7a8d5ff700 1 -- start start 2023-07-13T22:26:03.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.448+0000 7f7a8d5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a880cd260 0x7f7a880d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a8d5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a880d0e20 0x7f7a880d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a8d5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a880d2410 con 0x7f7a880d0a50 2023-07-13T22:26:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a8d5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8815c900 con 0x7f7a880cd260 2023-07-13T22:26:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a8d5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a880d2660 con 0x7f7a880d0e20 2023-07-13T22:26:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a867fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a880cd260 0x7f7a880d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a867fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a880cd260 0x7f7a880d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44034/0 (socket says 172.21.15.60:44034) 2023-07-13T22:26:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a86ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7a880d0a50 0x7f7a880cf8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41130/0 (socket says 172.21.15.60:41130) 2023-07-13T22:26:03.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a867fc700 1 -- 172.21.15.60:0/3326093025 learned_addr learned my addr 172.21.15.60:0/3326093025 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:03.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a877fe700 1 --2- 172.21.15.60:0/3326093025 >> v2:172.21.15.105:3300/0 conn(0x7f7a880d0e20 0x7f7a880d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:03.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a867fc700 1 -- 172.21.15.60:0/3326093025 >> v2:172.21.15.105:3300/0 conn(0x7f7a880d0e20 msgr2=0x7f7a880d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:26:03.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a867fc700 1 --2- 172.21.15.60:0/3326093025 >> v2:172.21.15.105:3300/0 conn(0x7f7a880d0e20 0x7f7a880d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a867fc700 1 -- 172.21.15.60:0/3326093025 >> v1:172.21.15.60:6789/0 conn(0x7f7a880d0a50 legacy=0x7f7a880cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:03.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a85ffb700 1 -- 172.21.15.60:0/3326093025 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357657168 0 0) 0x7f7a880d2410 con 0x7f7a880d0a50 2023-07-13T22:26:03.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.449+0000 7f7a867fc700 1 -- 172.21.15.60:0/3326093025 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a880d2940 con 0x7f7a880cd260 2023-07-13T22:26:03.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.450+0000 7f7a867fc700 1 --2- 172.21.15.60:0/3326093025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd260 0x7f7a880d3420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7a880b8ce0 tx=0x7f7a7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ba5a4de61469d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:03.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.450+0000 7f7a85ffb700 1 -- 172.21.15.60:0/3326093025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00b870 con 0x7f7a880cd260 2023-07-13T22:26:03.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.450+0000 7f7a85ffb700 1 -- 172.21.15.60:0/3326093025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c00b9d0 con 0x7f7a880cd260 2023-07-13T22:26:03.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.450+0000 7f7a85ffb700 1 -- 172.21.15.60:0/3326093025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00bd60 con 0x7f7a880cd260 2023-07-13T22:26:03.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.450+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3326093025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd260 msgr2=0x7f7a880d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:03.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.450+0000 7f7a8d5ff700 1 --2- 172.21.15.60:0/3326093025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd260 0x7f7a880d3420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7a880b8ce0 tx=0x7f7a7c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.450+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3326093025 shutdown_connections 2023-07-13T22:26:03.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.451+0000 7f7a8d5ff700 1 --2- 172.21.15.60:0/3326093025 >> v2:172.21.15.105:3300/0 conn(0x7f7a880d0e20 0x7f7a880d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.451+0000 7f7a8d5ff700 1 --2- 172.21.15.60:0/3326093025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd260 0x7f7a880d3420 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.451+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3326093025 >> 172.21.15.60:0/3326093025 conn(0x7f7a88071600 msgr2=0x7f7a880719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:03.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.451+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3326093025 shutdown_connections 2023-07-13T22:26:03.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.451+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3326093025 wait complete. 2023-07-13T22:26:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.452+0000 7f7a8d5ff700 1 Processor -- start 2023-07-13T22:26:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.452+0000 7f7a8d5ff700 1 -- start start 2023-07-13T22:26:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.453+0000 7f7a8d5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a880cd260 0x7f7a881f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.453+0000 7f7a8d5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a880d0a50 0x7f7a880cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.453+0000 7f7a8d5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880d0e20 0x7f7a881f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.453+0000 7f7a8d5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a881f2b70 con 0x7f7a880d0e20 2023-07-13T22:26:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.453+0000 7f7a8d5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880cc380 con 0x7f7a880d0a50 2023-07-13T22:26:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.453+0000 7f7a8d5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880cc660 con 0x7f7a880cd260 2023-07-13T22:26:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880d0e20 0x7f7a881f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880d0e20 0x7f7a881f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44036/0 (socket says 172.21.15.60:44036) 2023-07-13T22:26:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a877fe700 1 -- 172.21.15.60:0/3208596479 learned_addr learned my addr 172.21.15.60:0/3208596479 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:03.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a86ffd700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a880cd260 0x7f7a881f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:03.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a867fc700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a880d0a50 0x7f7a880cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:03.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a877fe700 1 -- 172.21.15.60:0/3208596479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a880cd260 msgr2=0x7f7a881f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:03.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a877fe700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a880cd260 0x7f7a881f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a877fe700 1 -- 172.21.15.60:0/3208596479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a880d0a50 msgr2=0x7f7a880cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:03.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a877fe700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a880d0a50 0x7f7a880cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a877fe700 1 -- 172.21.15.60:0/3208596479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c009550 con 0x7f7a880d0e20 2023-07-13T22:26:03.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.454+0000 7f7a877fe700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880d0e20 0x7f7a881f51c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7a80007e60 tx=0x7f7a8000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:03.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.455+0000 7f7a6f7fe700 1 -- 172.21.15.60:0/3208596479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000ed30 con 0x7f7a880d0e20 2023-07-13T22:26:03.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.455+0000 7f7a6f7fe700 1 -- 172.21.15.60:0/3208596479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a80014420 con 0x7f7a880d0e20 2023-07-13T22:26:03.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.455+0000 7f7a6f7fe700 1 -- 172.21.15.60:0/3208596479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800147d0 con 0x7f7a880d0e20 2023-07-13T22:26:03.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.455+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a880cc940 con 0x7f7a880d0e20 2023-07-13T22:26:03.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.455+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a880cbec0 con 0x7f7a880d0e20 2023-07-13T22:26:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.456+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a880cc0e0 con 0x7f7a880d0e20 2023-07-13T22:26:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.457+0000 7f7a6f7fe700 1 -- 172.21.15.60:0/3208596479 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a80014930 con 0x7f7a880d0e20 2023-07-13T22:26:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.458+0000 7f7a6f7fe700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7406ace0 0x7f7a7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.458+0000 7f7a6f7fe700 1 -- 172.21.15.60:0/3208596479 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a80089b10 con 0x7f7a880d0e20 2023-07-13T22:26:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.458+0000 7f7a86ffd700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7406ace0 0x7f7a7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.459+0000 7f7a86ffd700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7406ace0 0x7f7a7406d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7a78007fd0 tx=0x7f7a78007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:03.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.462+0000 7f7a6f7fe700 1 -- 172.21.15.60:0/3208596479 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7a80054c60 con 0x7f7a880d0e20 2023-07-13T22:26:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.716+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7a8804e280 con 0x7f7a880d0e20 2023-07-13T22:26:03.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.716+0000 7f7a6f7fe700 1 -- 172.21.15.60:0/3208596479 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f7a80058280 con 0x7f7a880d0e20 2023-07-13T22:26:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:03.698 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:03.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.718+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7406ace0 msgr2=0x7f7a7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:03.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.718+0000 7f7a8d5ff700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7406ace0 0x7f7a7406d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7a78007fd0 tx=0x7f7a78007de0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880d0e20 msgr2=0x7f7a881f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:03.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880d0e20 0x7f7a881f51c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7a80007e60 tx=0x7f7a8000e3c0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 shutdown_connections 2023-07-13T22:26:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7406ace0 0x7f7a7406d190 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880d0e20 0x7f7a881f51c0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a880cd260 0x7f7a881f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 --2- 172.21.15.60:0/3208596479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a880d0a50 0x7f7a880cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 >> 172.21.15.60:0/3208596479 conn(0x7f7a88071600 msgr2=0x7f7a8806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 shutdown_connections 2023-07-13T22:26:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:03.719+0000 7f7a8d5ff700 1 -- 172.21.15.60:0/3208596479 wait complete. 2023-07-13T22:26:03.718 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:03.719 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:08.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:08.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.953+0000 7fedabfff700 1 Processor -- start 2023-07-13T22:26:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.953+0000 7fedabfff700 1 -- start start 2023-07-13T22:26:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.954+0000 7fedabfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fedac0cd6d0 0x7fedac160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.954+0000 7fedabfff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac0cf680 con 0x7fedac0b1d60 2023-07-13T22:26:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.954+0000 7fedabfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac160810 con 0x7fedac15fca0 2023-07-13T22:26:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.954+0000 7fedabfff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedac160af0 con 0x7fedac0cd6d0 2023-07-13T22:26:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.954+0000 7fedaaffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fedac0b1d60 0x7fedac0d30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47876/0 (socket says 172.21.15.60:47876) 2023-07-13T22:26:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.954+0000 7fedaaffd700 1 -- 172.21.15.60:0/3087748093 learned_addr learned my addr 172.21.15.60:0/3087748093 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:26:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.954+0000 7fedab7fe700 1 --2- 172.21.15.60:0/3087748093 >> v2:172.21.15.105:3300/0 conn(0x7fedac0cd6d0 0x7fedac160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.955+0000 7fedaa7fc700 1 -- 172.21.15.60:0/3087748093 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261677946 0 0) 0x7fedac0cf680 con 0x7fedac0b1d60 2023-07-13T22:26:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.955+0000 7fedaa7fc700 1 -- 172.21.15.60:0/3087748093 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed980035b0 con 0x7fedac0b1d60 2023-07-13T22:26:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.955+0000 7fedab7fe700 1 -- 172.21.15.60:0/3087748093 >> v1:172.21.15.105:6789/0 conn(0x7fedac15fca0 legacy=0x7fedac0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.955+0000 7fedab7fe700 1 -- 172.21.15.60:0/3087748093 >> v1:172.21.15.60:6789/0 conn(0x7fedac0b1d60 legacy=0x7fedac0d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.955+0000 7fedab7fe700 1 -- 172.21.15.60:0/3087748093 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac0d28d0 con 0x7fedac0cd6d0 2023-07-13T22:26:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.955+0000 7fedab7fe700 1 --2- 172.21.15.60:0/3087748093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedac0cd6d0 0x7fedac160120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feda4009d90 tx=0x7feda400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cbd416ba73092f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedaa7fc700 1 -- 172.21.15.60:0/3087748093 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda400e070 con 0x7fedac0cd6d0 2023-07-13T22:26:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedaa7fc700 1 -- 172.21.15.60:0/3087748093 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda400bb20 con 0x7fedac0cd6d0 2023-07-13T22:26:08.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedaa7fc700 1 -- 172.21.15.60:0/3087748093 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda400be90 con 0x7fedac0cd6d0 2023-07-13T22:26:08.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedabfff700 1 -- 172.21.15.60:0/3087748093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedac0cd6d0 msgr2=0x7fedac160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:08.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedabfff700 1 --2- 172.21.15.60:0/3087748093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedac0cd6d0 0x7fedac160120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feda4009d90 tx=0x7feda400b330 comp rx=0 tx=0).stop 2023-07-13T22:26:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedabfff700 1 -- 172.21.15.60:0/3087748093 shutdown_connections 2023-07-13T22:26:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedabfff700 1 --2- 172.21.15.60:0/3087748093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedac0cd6d0 0x7fedac160120 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedabfff700 1 -- 172.21.15.60:0/3087748093 >> 172.21.15.60:0/3087748093 conn(0x7fedac071f20 msgr2=0x7fedac09bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedabfff700 1 -- 172.21.15.60:0/3087748093 shutdown_connections 2023-07-13T22:26:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.956+0000 7fedabfff700 1 -- 172.21.15.60:0/3087748093 wait complete. 2023-07-13T22:26:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.958+0000 7fedabfff700 1 Processor -- start 2023-07-13T22:26:08.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.958+0000 7fedabfff700 1 -- start start 2023-07-13T22:26:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.959+0000 7fedabfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedac0b1d60 0x7fedac1f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.959+0000 7fedabfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedac0cd6d0 0x7fedac15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.959+0000 7fedabfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedac15fca0 0x7fedac15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.959+0000 7fedabfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac149590 con 0x7fedac0cd6d0 2023-07-13T22:26:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.959+0000 7fedabfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac1f1820 con 0x7fedac15fca0 2023-07-13T22:26:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.959+0000 7fedabfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac1497e0 con 0x7fedac0b1d60 2023-07-13T22:26:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.959+0000 7feda3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedac0cd6d0 0x7fedac15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.959+0000 7feda3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedac0cd6d0 0x7fedac15ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39362/0 (socket says 172.21.15.60:39362) 2023-07-13T22:26:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.959+0000 7feda3fff700 1 -- 172.21.15.60:0/3438337261 learned_addr learned my addr 172.21.15.60:0/3438337261 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7fedaaffd700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedac0b1d60 0x7fedac1f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7fedab7fe700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedac15fca0 0x7fedac15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7feda3fff700 1 -- 172.21.15.60:0/3438337261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedac0b1d60 msgr2=0x7fedac1f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7feda3fff700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedac0b1d60 0x7fedac1f12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7feda3fff700 1 -- 172.21.15.60:0/3438337261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedac15fca0 msgr2=0x7fedac15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7feda3fff700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedac15fca0 0x7fedac15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:08.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7feda3fff700 1 -- 172.21.15.60:0/3438337261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda4009a70 con 0x7fedac0cd6d0 2023-07-13T22:26:08.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7feda3fff700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedac0cd6d0 0x7fedac15ac60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c430 tx=0x7fed9400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:08.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7feda8ff9700 1 -- 172.21.15.60:0/3438337261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed940050d0 con 0x7fedac0cd6d0 2023-07-13T22:26:08.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7feda8ff9700 1 -- 172.21.15.60:0/3438337261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed94005250 con 0x7fedac0cd6d0 2023-07-13T22:26:08.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac149ac0 con 0x7fedac0cd6d0 2023-07-13T22:26:08.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.960+0000 7feda8ff9700 1 -- 172.21.15.60:0/3438337261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94006860 con 0x7fedac0cd6d0 2023-07-13T22:26:08.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.961+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac14a080 con 0x7fedac0cd6d0 2023-07-13T22:26:08.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.962+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac04e210 con 0x7fedac0cd6d0 2023-07-13T22:26:08.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.963+0000 7feda8ff9700 1 -- 172.21.15.60:0/3438337261 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed94003680 con 0x7fedac0cd6d0 2023-07-13T22:26:08.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.963+0000 7feda8ff9700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed8c06ace0 0x7fed8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:08.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.964+0000 7feda8ff9700 1 -- 172.21.15.60:0/3438337261 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fed94088660 con 0x7fedac0cd6d0 2023-07-13T22:26:08.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.964+0000 7fedaaffd700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed8c06ace0 0x7fed8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:08.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.964+0000 7fedaaffd700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed8c06ace0 0x7fed8c06d190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fed9c007fd0 tx=0x7fed9c007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:08.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:08.968+0000 7feda8ff9700 1 -- 172.21.15.60:0/3438337261 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fed94053670 con 0x7fedac0cd6d0 2023-07-13T22:26:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.218+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fedac08caa0 con 0x7fedac0cd6d0 2023-07-13T22:26:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.218+0000 7feda8ff9700 1 -- 172.21.15.60:0/3438337261 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fed94056c90 con 0x7fedac0cd6d0 2023-07-13T22:26:09.200 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:09.200 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.220+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed8c06ace0 msgr2=0x7fed8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.220+0000 7fedabfff700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed8c06ace0 0x7fed8c06d190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fed9c007fd0 tx=0x7fed9c007dc0 comp rx=0 tx=0).stop 2023-07-13T22:26:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedac0cd6d0 msgr2=0x7fedac15ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedac0cd6d0 0x7fedac15ac60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c430 tx=0x7fed9400c740 comp rx=0 tx=0).stop 2023-07-13T22:26:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 shutdown_connections 2023-07-13T22:26:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed8c06ace0 0x7fed8c06d190 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedac15fca0 0x7fedac15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedac0b1d60 0x7fedac1f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 --2- 172.21.15.60:0/3438337261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedac0cd6d0 0x7fedac15ac60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 >> 172.21.15.60:0/3438337261 conn(0x7fedac071f20 msgr2=0x7fedac06c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 shutdown_connections 2023-07-13T22:26:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:09.221+0000 7fedabfff700 1 -- 172.21.15.60:0/3438337261 wait complete. 2023-07-13T22:26:09.220 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:09.220 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:14.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:14.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.452+0000 7f8bfbfff700 1 Processor -- start 2023-07-13T22:26:14.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.452+0000 7f8bfbfff700 1 -- start start 2023-07-13T22:26:14.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.453+0000 7f8bfbfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8bf415c9d0 0x7f8bf40cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:14.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.453+0000 7f8bfbfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf40c9610 0x7f8bf40c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:14.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.453+0000 7f8bfbfff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf40c9f70 con 0x7f8bf415bef0 2023-07-13T22:26:14.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.453+0000 7f8bfbfff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf415ce80 con 0x7f8bf415c9d0 2023-07-13T22:26:14.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.453+0000 7f8bfbfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf415d160 con 0x7f8bf40c9610 2023-07-13T22:26:14.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.454+0000 7f8bf9d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8bf415bef0 0x7f8bf415c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47886/0 (socket says 172.21.15.60:47886) 2023-07-13T22:26:14.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.454+0000 7f8bf9d9b700 1 -- 172.21.15.60:0/3745156608 learned_addr learned my addr 172.21.15.60:0/3745156608 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:26:14.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.454+0000 7f8bf959a700 1 --2- 172.21.15.60:0/3745156608 >> v2:172.21.15.105:3300/0 conn(0x7f8bf415c9d0 0x7f8bf40cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:14.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.454+0000 7f8bfa59c700 1 --2- 172.21.15.60:0/3745156608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf40c9610 0x7f8bf40c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:14.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.454+0000 7f8bf8d99700 1 -- 172.21.15.60:0/3745156608 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611294036 0 0) 0x7f8bf40c9f70 con 0x7f8bf415bef0 2023-07-13T22:26:14.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.454+0000 7f8bf8d99700 1 -- 172.21.15.60:0/3745156608 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be00035b0 con 0x7f8bf415bef0 2023-07-13T22:26:14.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.454+0000 7f8bf8d99700 1 -- 172.21.15.60:0/3745156608 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975479325 0 0) 0x7f8be00035b0 con 0x7f8bf415bef0 2023-07-13T22:26:14.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.454+0000 7f8bf959a700 1 -- 172.21.15.60:0/3745156608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf40c9610 msgr2=0x7f8bf40c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:26:14.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.454+0000 7f8bf959a700 1 --2- 172.21.15.60:0/3745156608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf40c9610 0x7f8bf40c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.455+0000 7f8bf959a700 1 -- 172.21.15.60:0/3745156608 >> v1:172.21.15.60:6789/0 conn(0x7f8bf415bef0 legacy=0x7f8bf415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:14.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.455+0000 7f8bf959a700 1 -- 172.21.15.60:0/3745156608 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf406c7e0 con 0x7f8bf415c9d0 2023-07-13T22:26:14.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.455+0000 7f8bfa59c700 1 --2- 172.21.15.60:0/3745156608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf40c9610 0x7f8bf40c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:26:14.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.455+0000 7f8bf959a700 1 --2- 172.21.15.60:0/3745156608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf415c9d0 0x7f8bf40cc860 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009840 tx=0x7f8be4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a34b56e2413f7804 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:14.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.455+0000 7f8bf8d99700 1 -- 172.21.15.60:0/3745156608 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00017f0 con 0x7f8bf415c9d0 2023-07-13T22:26:14.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.455+0000 7f8bf8d99700 1 -- 172.21.15.60:0/3745156608 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4004030 con 0x7f8bf415c9d0 2023-07-13T22:26:14.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.455+0000 7f8bf8d99700 1 -- 172.21.15.60:0/3745156608 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be40043c0 con 0x7f8bf415c9d0 2023-07-13T22:26:14.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.456+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3745156608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf415c9d0 msgr2=0x7f8bf40cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:14.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.456+0000 7f8bfbfff700 1 --2- 172.21.15.60:0/3745156608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf415c9d0 0x7f8bf40cc860 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009840 tx=0x7f8be4009b50 comp rx=0 tx=0).stop 2023-07-13T22:26:14.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.456+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3745156608 shutdown_connections 2023-07-13T22:26:14.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.456+0000 7f8bfbfff700 1 --2- 172.21.15.60:0/3745156608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf40c9610 0x7f8bf40c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.456+0000 7f8bfbfff700 1 --2- 172.21.15.60:0/3745156608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf415c9d0 0x7f8bf40cc860 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.456+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3745156608 >> 172.21.15.60:0/3745156608 conn(0x7f8bf40cbe70 msgr2=0x7f8bf40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:14.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.456+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3745156608 shutdown_connections 2023-07-13T22:26:14.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.456+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3745156608 wait complete. 2023-07-13T22:26:14.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.457+0000 7f8bfbfff700 1 Processor -- start 2023-07-13T22:26:14.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.458+0000 7f8bfbfff700 1 -- start start 2023-07-13T22:26:14.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bfbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9610 0x7f8bf4144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:14.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bfbfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf415bef0 0x7f8bf41595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:14.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bfbfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf415c9d0 0x7f8bf40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:14.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bfbfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4159ec0 con 0x7f8bf40c9610 2023-07-13T22:26:14.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bfbfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf415a1a0 con 0x7f8bf415c9d0 2023-07-13T22:26:14.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bfbfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf415a480 con 0x7f8bf415bef0 2023-07-13T22:26:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bf9d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9610 0x7f8bf4144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bf9d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9610 0x7f8bf4144580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39368/0 (socket says 172.21.15.60:39368) 2023-07-13T22:26:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bf9d9b700 1 -- 172.21.15.60:0/3084380237 learned_addr learned my addr 172.21.15.60:0/3084380237 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bf9d9b700 1 -- 172.21.15.60:0/3084380237 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf415bef0 msgr2=0x7f8bf41595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bf9d9b700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf415bef0 0x7f8bf41595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bf9d9b700 1 -- 172.21.15.60:0/3084380237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf415c9d0 msgr2=0x7f8bf40b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bf9d9b700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf415c9d0 0x7f8bf40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.459+0000 7f8bf9d9b700 1 -- 172.21.15.60:0/3084380237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec009a70 con 0x7f8bf40c9610 2023-07-13T22:26:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.460+0000 7f8bf9d9b700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9610 0x7f8bf4144580 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c520 tx=0x7f8bf000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.460+0000 7f8bea7fc700 1 -- 172.21.15.60:0/3084380237 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00074e0 con 0x7f8bf40c9610 2023-07-13T22:26:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.460+0000 7f8bea7fc700 1 -- 172.21.15.60:0/3084380237 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf0007640 con 0x7f8bf40c9610 2023-07-13T22:26:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.460+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be40094a0 con 0x7f8bf40c9610 2023-07-13T22:26:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.460+0000 7f8bea7fc700 1 -- 172.21.15.60:0/3084380237 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0005a70 con 0x7f8bf40c9610 2023-07-13T22:26:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.460+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf415aa40 con 0x7f8bf40c9610 2023-07-13T22:26:14.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.461+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf404e280 con 0x7f8bf40c9610 2023-07-13T22:26:14.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.462+0000 7f8bea7fc700 1 -- 172.21.15.60:0/3084380237 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8bf0005bd0 con 0x7f8bf40c9610 2023-07-13T22:26:14.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.463+0000 7f8bea7fc700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:14.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.463+0000 7f8bea7fc700 1 -- 172.21.15.60:0/3084380237 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8bf0088b10 con 0x7f8bf40c9610 2023-07-13T22:26:14.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.463+0000 7f8bf959a700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:14.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.464+0000 7f8bf959a700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009840 tx=0x7f8be4010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.468+0000 7f8bea7fc700 1 -- 172.21.15.60:0/3084380237 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8bf0053b20 con 0x7f8bf40c9610 2023-07-13T22:26:14.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.720+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8bf406cda0 con 0x7f8bf40c9610 2023-07-13T22:26:14.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.720+0000 7f8bea7fc700 1 -- 172.21.15.60:0/3084380237 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f8bf00170c0 con 0x7f8bf40c9610 2023-07-13T22:26:14.702 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:14.702 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:14.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.722+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 msgr2=0x7f8bd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:14.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.722+0000 7f8bfbfff700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009840 tx=0x7f8be4010000 comp rx=0 tx=0).stop 2023-07-13T22:26:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.722+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9610 msgr2=0x7f8bf4144580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.722+0000 7f8bfbfff700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9610 0x7f8bf4144580 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c520 tx=0x7f8bf000c8e0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.723+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 shutdown_connections 2023-07-13T22:26:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.723+0000 7f8bfbfff700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.723+0000 7f8bfbfff700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9610 0x7f8bf4144580 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.723+0000 7f8bfbfff700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf415c9d0 0x7f8bf40b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.723+0000 7f8bfbfff700 1 --2- 172.21.15.60:0/3084380237 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf415bef0 0x7f8bf41595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:14.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.723+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 >> 172.21.15.60:0/3084380237 conn(0x7f8bf40cbe70 msgr2=0x7f8bf40b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:14.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.723+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 shutdown_connections 2023-07-13T22:26:14.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:14.723+0000 7f8bfbfff700 1 -- 172.21.15.60:0/3084380237 wait complete. 2023-07-13T22:26:14.722 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:14.723 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:19.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.958+0000 7fb06a1ff700 1 Processor -- start 2023-07-13T22:26:19.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.958+0000 7fb06a1ff700 1 -- start start 2023-07-13T22:26:19.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.959+0000 7fb06a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0a50 0x7fb0640cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:19.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.959+0000 7fb06a1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0640cd260 0x7fb0640d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:19.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.959+0000 7fb06a1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0640d2410 con 0x7fb0640d0e20 2023-07-13T22:26:19.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.959+0000 7fb06a1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06415c900 con 0x7fb0640cd260 2023-07-13T22:26:19.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.959+0000 7fb06a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640d2660 con 0x7fb0640d0a50 2023-07-13T22:26:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.960+0000 7fb062ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0640cd260 0x7fb0640d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.960+0000 7fb062ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0640cd260 0x7fb0640d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44592/0 (socket says 172.21.15.60:44592) 2023-07-13T22:26:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.960+0000 7fb062ffd700 1 -- 172.21.15.60:0/1593882452 learned_addr learned my addr 172.21.15.60:0/1593882452 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:19.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.960+0000 7fb0637fe700 1 --2- 172.21.15.60:0/1593882452 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0a50 0x7fb0640cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:19.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.960+0000 7fb062ffd700 1 -- 172.21.15.60:0/1593882452 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0a50 msgr2=0x7fb0640cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:26:19.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.960+0000 7fb062ffd700 1 --2- 172.21.15.60:0/1593882452 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0a50 0x7fb0640cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:19.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.960+0000 7fb062ffd700 1 -- 172.21.15.60:0/1593882452 >> v1:172.21.15.105:6789/0 conn(0x7fb0640d0e20 legacy=0x7fb0640b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:19.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.960+0000 7fb062ffd700 1 -- 172.21.15.60:0/1593882452 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0640d2940 con 0x7fb0640cd260 2023-07-13T22:26:19.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.960+0000 7fb062ffd700 1 --2- 172.21.15.60:0/1593882452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640cd260 0x7fb0640d3420 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb0580098f0 tx=0x7fb058009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8b0b00bcd63363 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:19.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb0627fc700 1 -- 172.21.15.60:0/1593882452 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058011830 con 0x7fb0640cd260 2023-07-13T22:26:19.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb0627fc700 1 -- 172.21.15.60:0/1593882452 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb058011990 con 0x7fb0640cd260 2023-07-13T22:26:19.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb0627fc700 1 -- 172.21.15.60:0/1593882452 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058011d20 con 0x7fb0640cd260 2023-07-13T22:26:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb06a1ff700 1 -- 172.21.15.60:0/1593882452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640cd260 msgr2=0x7fb0640d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb06a1ff700 1 --2- 172.21.15.60:0/1593882452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640cd260 0x7fb0640d3420 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb0580098f0 tx=0x7fb058009c00 comp rx=0 tx=0).stop 2023-07-13T22:26:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb06a1ff700 1 -- 172.21.15.60:0/1593882452 shutdown_connections 2023-07-13T22:26:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb06a1ff700 1 --2- 172.21.15.60:0/1593882452 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0a50 0x7fb0640cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:19.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb06a1ff700 1 --2- 172.21.15.60:0/1593882452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640cd260 0x7fb0640d3420 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:19.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb06a1ff700 1 -- 172.21.15.60:0/1593882452 >> 172.21.15.60:0/1593882452 conn(0x7fb064071600 msgr2=0x7fb0640719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:19.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb06a1ff700 1 -- 172.21.15.60:0/1593882452 shutdown_connections 2023-07-13T22:26:19.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.961+0000 7fb06a1ff700 1 -- 172.21.15.60:0/1593882452 wait complete. 2023-07-13T22:26:19.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.963+0000 7fb06a1ff700 1 Processor -- start 2023-07-13T22:26:19.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.963+0000 7fb06a1ff700 1 -- start start 2023-07-13T22:26:19.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.964+0000 7fb06a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0640cd260 0x7fb0640c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:19.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.964+0000 7fb06a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640d0a50 0x7fb0640c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:19.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.964+0000 7fb06a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0e20 0x7fb064157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:19.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.964+0000 7fb06a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb064159440 con 0x7fb0640d0a50 2023-07-13T22:26:19.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.964+0000 7fb06a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640c77e0 con 0x7fb0640cd260 2023-07-13T22:26:19.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.964+0000 7fb06a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640c8020 con 0x7fb0640d0e20 2023-07-13T22:26:19.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.964+0000 7fb062ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640d0a50 0x7fb0640c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:19.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.964+0000 7fb0637fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0640cd260 0x7fb0640c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:19.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb062ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640d0a50 0x7fb0640c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44602/0 (socket says 172.21.15.60:44602) 2023-07-13T22:26:19.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb062ffd700 1 -- 172.21.15.60:0/2948089270 learned_addr learned my addr 172.21.15.60:0/2948089270 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:19.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb063fff700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0e20 0x7fb064157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:19.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb062ffd700 1 -- 172.21.15.60:0/2948089270 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0e20 msgr2=0x7fb064157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb062ffd700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0e20 0x7fb064157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb062ffd700 1 -- 172.21.15.60:0/2948089270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0640cd260 msgr2=0x7fb0640c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb062ffd700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0640cd260 0x7fb0640c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb062ffd700 1 -- 172.21.15.60:0/2948089270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058009550 con 0x7fb0640d0a50 2023-07-13T22:26:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb062ffd700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640d0a50 0x7fb0640c72a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb058000c00 tx=0x7fb058004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.965+0000 7fb04bfff700 1 -- 172.21.15.60:0/2948089270 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058004400 con 0x7fb0640d0a50 2023-07-13T22:26:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.966+0000 7fb04bfff700 1 -- 172.21.15.60:0/2948089270 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05801f070 con 0x7fb0640d0a50 2023-07-13T22:26:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.966+0000 7fb04bfff700 1 -- 172.21.15.60:0/2948089270 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580038e0 con 0x7fb0640d0a50 2023-07-13T22:26:19.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.966+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0640c8300 con 0x7fb0640d0a50 2023-07-13T22:26:19.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.966+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0640c85e0 con 0x7fb0640d0a50 2023-07-13T22:26:19.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.967+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb064159580 con 0x7fb0640d0a50 2023-07-13T22:26:19.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.968+0000 7fb04bfff700 1 -- 172.21.15.60:0/2948089270 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb058021030 con 0x7fb0640d0a50 2023-07-13T22:26:19.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.968+0000 7fb04bfff700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb05006ace0 0x7fb05006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:19.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.969+0000 7fb04bfff700 1 -- 172.21.15.60:0/2948089270 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb05808a900 con 0x7fb0640d0a50 2023-07-13T22:26:19.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.969+0000 7fb0637fe700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb05006ace0 0x7fb05006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:19.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.969+0000 7fb0637fe700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb05006ace0 0x7fb05006d190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb054009be0 tx=0x7fb054009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:19.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:19.973+0000 7fb04bfff700 1 -- 172.21.15.60:0/2948089270 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb064159580 con 0x7fb0640d0a50 2023-07-13T22:26:20.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.225+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb064159580 con 0x7fb0640d0a50 2023-07-13T22:26:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.226+0000 7fb04bfff700 1 -- 172.21.15.60:0/2948089270 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fb064159580 con 0x7fb0640d0a50 2023-07-13T22:26:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:20.208 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:20.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb05006ace0 msgr2=0x7fb05006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:20.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb05006ace0 0x7fb05006d190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb054009be0 tx=0x7fb054009210 comp rx=0 tx=0).stop 2023-07-13T22:26:20.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640d0a50 msgr2=0x7fb0640c72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:20.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640d0a50 0x7fb0640c72a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb058000c00 tx=0x7fb058004750 comp rx=0 tx=0).stop 2023-07-13T22:26:20.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 shutdown_connections 2023-07-13T22:26:20.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb05006ace0 0x7fb05006d190 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:20.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0640d0e20 0x7fb064157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0640cd260 0x7fb0640c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 --2- 172.21.15.60:0/2948089270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0640d0a50 0x7fb0640c72a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 >> 172.21.15.60:0/2948089270 conn(0x7fb064071600 msgr2=0x7fb0640734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 shutdown_connections 2023-07-13T22:26:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:20.228+0000 7fb06a1ff700 1 -- 172.21.15.60:0/2948089270 wait complete. 2023-07-13T22:26:20.228 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:20.228 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:25.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:25.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.460+0000 7f234f3ff700 1 Processor -- start 2023-07-13T22:26:25.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.461+0000 7f234f3ff700 1 -- start start 2023-07-13T22:26:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.462+0000 7f234f3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f23480d0a50 0x7f23480cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.462+0000 7f234f3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 0x7f23480d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.462+0000 7f234f3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23480d2410 con 0x7f23480d0e20 2023-07-13T22:26:25.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.462+0000 7f234f3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234815c900 con 0x7f23480d0a50 2023-07-13T22:26:25.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.462+0000 7f234f3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480d2660 con 0x7f23480cd260 2023-07-13T22:26:25.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.462+0000 7f234d19b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f23480d0a50 0x7f23480cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.462+0000 7f234d19b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f23480d0a50 0x7f23480cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56550/0 (socket says 172.21.15.60:56550) 2023-07-13T22:26:25.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.462+0000 7f234d19b700 1 -- 172.21.15.60:0/176379208 learned_addr learned my addr 172.21.15.60:0/176379208 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:25.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.462+0000 7f234c99a700 1 --2- 172.21.15.60:0/176379208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 0x7f23480d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.463+0000 7f234d19b700 1 -- 172.21.15.60:0/176379208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 msgr2=0x7f23480d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:26:25.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.463+0000 7f234d19b700 1 --2- 172.21.15.60:0/176379208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 0x7f23480d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.463+0000 7f234d19b700 1 -- 172.21.15.60:0/176379208 >> v1:172.21.15.105:6789/0 conn(0x7f23480d0e20 legacy=0x7f23480b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.463+0000 7f234d19b700 1 -- 172.21.15.60:0/176379208 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23480d2940 con 0x7f23480d0a50 2023-07-13T22:26:25.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.463+0000 7f234d19b700 1 --2- 172.21.15.60:0/176379208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 0x7f23480cf8a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f23440098f0 tx=0x7f2344009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f26339d8d773477 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:25.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.463+0000 7f233ffff700 1 -- 172.21.15.60:0/176379208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344011830 con 0x7f23480d0a50 2023-07-13T22:26:25.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.463+0000 7f233ffff700 1 -- 172.21.15.60:0/176379208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2344011990 con 0x7f23480d0a50 2023-07-13T22:26:25.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.463+0000 7f233ffff700 1 -- 172.21.15.60:0/176379208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344011d20 con 0x7f23480d0a50 2023-07-13T22:26:25.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.464+0000 7f234f3ff700 1 -- 172.21.15.60:0/176379208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 msgr2=0x7f23480cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.464+0000 7f234f3ff700 1 --2- 172.21.15.60:0/176379208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 0x7f23480cf8a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f23440098f0 tx=0x7f2344009c00 comp rx=0 tx=0).stop 2023-07-13T22:26:25.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.464+0000 7f234f3ff700 1 -- 172.21.15.60:0/176379208 shutdown_connections 2023-07-13T22:26:25.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.464+0000 7f234f3ff700 1 --2- 172.21.15.60:0/176379208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 0x7f23480cf8a0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.464+0000 7f234f3ff700 1 --2- 172.21.15.60:0/176379208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 0x7f23480d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.464+0000 7f234f3ff700 1 -- 172.21.15.60:0/176379208 >> 172.21.15.60:0/176379208 conn(0x7f2348071600 msgr2=0x7f23480719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:25.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.464+0000 7f234f3ff700 1 -- 172.21.15.60:0/176379208 shutdown_connections 2023-07-13T22:26:25.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.464+0000 7f234f3ff700 1 -- 172.21.15.60:0/176379208 wait complete. 2023-07-13T22:26:25.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.465+0000 7f234f3ff700 1 Processor -- start 2023-07-13T22:26:25.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.466+0000 7f234f3ff700 1 -- start start 2023-07-13T22:26:25.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234f3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 0x7f2348157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234f3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 0x7f23480c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234f3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23480d0e20 0x7f23481f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234f3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23480c9ce0 con 0x7f23480d0a50 2023-07-13T22:26:25.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234f3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23480c9f30 con 0x7f23480d0e20 2023-07-13T22:26:25.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234f3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480ca210 con 0x7f23480cd260 2023-07-13T22:26:25.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 0x7f23480c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 0x7f23480c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56552/0 (socket says 172.21.15.60:56552) 2023-07-13T22:26:25.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234c99a700 1 -- 172.21.15.60:0/1098047425 learned_addr learned my addr 172.21.15.60:0/1098047425 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:25.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234d19b700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 0x7f2348157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234d99c700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23480d0e20 0x7f23481f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234c99a700 1 -- 172.21.15.60:0/1098047425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 msgr2=0x7f2348157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234c99a700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 0x7f2348157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234c99a700 1 -- 172.21.15.60:0/1098047425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23480d0e20 msgr2=0x7f23481f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234c99a700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23480d0e20 0x7f23481f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.467+0000 7f234c99a700 1 -- 172.21.15.60:0/1098047425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2344009550 con 0x7f23480d0a50 2023-07-13T22:26:25.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.468+0000 7f234c99a700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 0x7f23480c8b40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f233800e9b0 tx=0x7f233800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:25.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.468+0000 7f233dffb700 1 -- 172.21.15.60:0/1098047425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338009710 con 0x7f23480d0a50 2023-07-13T22:26:25.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.468+0000 7f233dffb700 1 -- 172.21.15.60:0/1098047425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233800c850 con 0x7f23480d0a50 2023-07-13T22:26:25.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.468+0000 7f233dffb700 1 -- 172.21.15.60:0/1098047425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233800cbc0 con 0x7f23480d0a50 2023-07-13T22:26:25.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.468+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23480ca4f0 con 0x7f23480d0a50 2023-07-13T22:26:25.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.468+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2348157ec0 con 0x7f23480d0a50 2023-07-13T22:26:25.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.469+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234804e280 con 0x7f23480d0a50 2023-07-13T22:26:25.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.470+0000 7f233dffb700 1 -- 172.21.15.60:0/1098047425 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f233800cd20 con 0x7f23480d0a50 2023-07-13T22:26:25.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.471+0000 7f233dffb700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f232c06ace0 0x7f232c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.471+0000 7f233dffb700 1 -- 172.21.15.60:0/1098047425 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2338089520 con 0x7f23480d0a50 2023-07-13T22:26:25.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.471+0000 7f234d19b700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f232c06ace0 0x7f232c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.472+0000 7f234d19b700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f232c06ace0 0x7f232c06d190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2344009c40 tx=0x7f2344008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:25.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.475+0000 7f233dffb700 1 -- 172.21.15.60:0/1098047425 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f23380545c0 con 0x7f23480d0a50 2023-07-13T22:26:25.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.727+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2348156e50 con 0x7f23480d0a50 2023-07-13T22:26:25.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.727+0000 7f233dffb700 1 -- 172.21.15.60:0/1098047425 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f2338057be0 con 0x7f23480d0a50 2023-07-13T22:26:25.709 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:25.709 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:25.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.729+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f232c06ace0 msgr2=0x7f232c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.729+0000 7f234f3ff700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f232c06ace0 0x7f232c06d190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2344009c40 tx=0x7f2344008030 comp rx=0 tx=0).stop 2023-07-13T22:26:25.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.729+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 msgr2=0x7f23480c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.729+0000 7f234f3ff700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 0x7f23480c8b40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f233800e9b0 tx=0x7f233800ecc0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.730+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 shutdown_connections 2023-07-13T22:26:25.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.730+0000 7f234f3ff700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f232c06ace0 0x7f232c06d190 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.730+0000 7f234f3ff700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23480d0e20 0x7f23481f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.730+0000 7f234f3ff700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23480cd260 0x7f2348157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.730+0000 7f234f3ff700 1 --2- 172.21.15.60:0/1098047425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23480d0a50 0x7f23480c8b40 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.730+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 >> 172.21.15.60:0/1098047425 conn(0x7f2348071600 msgr2=0x7f23480cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:25.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.730+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 shutdown_connections 2023-07-13T22:26:25.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:25.730+0000 7f234f3ff700 1 -- 172.21.15.60:0/1098047425 wait complete. 2023-07-13T22:26:25.729 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:25.729 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:26.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:26:26.166 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:26:30.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:30.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.965+0000 7f4233bff700 1 Processor -- start 2023-07-13T22:26:30.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.965+0000 7f4233bff700 1 -- start start 2023-07-13T22:26:30.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.966+0000 7f4233bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c0cfcf0 0x7f422c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.966+0000 7f4233bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f422c0cd790 0x7f422c15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.966+0000 7f4233bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c0d16b0 con 0x7f422c15d160 2023-07-13T22:26:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.966+0000 7f4233bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c0cdc00 con 0x7f422c0cd790 2023-07-13T22:26:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.966+0000 7f4233bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0d0d80 con 0x7f422c0cfcf0 2023-07-13T22:26:30.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.967+0000 7f423119a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f422c0cd790 0x7f422c15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:30.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.967+0000 7f423119a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f422c0cd790 0x7f422c15cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56560/0 (socket says 172.21.15.60:56560) 2023-07-13T22:26:30.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.967+0000 7f423119a700 1 -- 172.21.15.60:0/2410988613 learned_addr learned my addr 172.21.15.60:0/2410988613 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:30.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.967+0000 7f423199b700 1 --2- 172.21.15.60:0/2410988613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c0cfcf0 0x7f422c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:30.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.967+0000 7f423119a700 1 -- 172.21.15.60:0/2410988613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c0cfcf0 msgr2=0x7f422c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:26:30.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.967+0000 7f423119a700 1 --2- 172.21.15.60:0/2410988613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c0cfcf0 0x7f422c0cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.967+0000 7f423119a700 1 -- 172.21.15.60:0/2410988613 >> v1:172.21.15.105:6789/0 conn(0x7f422c15d160 legacy=0x7f422c1600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.967+0000 7f423119a700 1 -- 172.21.15.60:0/2410988613 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0d1060 con 0x7f422c0cd790 2023-07-13T22:26:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.967+0000 7f423199b700 1 --2- 172.21.15.60:0/2410988613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c0cfcf0 0x7f422c0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:26:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f423119a700 1 --2- 172.21.15.60:0/2410988613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 0x7f422c15cc20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f421c0098f0 tx=0x7f421c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed363dcf53545be server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f4230999700 1 -- 172.21.15.60:0/2410988613 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c00b8a0 con 0x7f422c0cd790 2023-07-13T22:26:30.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f4230999700 1 -- 172.21.15.60:0/2410988613 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c00ba00 con 0x7f422c0cd790 2023-07-13T22:26:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f4230999700 1 -- 172.21.15.60:0/2410988613 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c00bd90 con 0x7f422c0cd790 2023-07-13T22:26:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f4233bff700 1 -- 172.21.15.60:0/2410988613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 msgr2=0x7f422c15cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f4233bff700 1 --2- 172.21.15.60:0/2410988613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 0x7f422c15cc20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f421c0098f0 tx=0x7f421c009c00 comp rx=0 tx=0).stop 2023-07-13T22:26:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f4233bff700 1 -- 172.21.15.60:0/2410988613 shutdown_connections 2023-07-13T22:26:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f4233bff700 1 --2- 172.21.15.60:0/2410988613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c0cfcf0 0x7f422c0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f4233bff700 1 --2- 172.21.15.60:0/2410988613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 0x7f422c15cc20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.968+0000 7f4233bff700 1 -- 172.21.15.60:0/2410988613 >> 172.21.15.60:0/2410988613 conn(0x7f422c06b310 msgr2=0x7f422c06b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:30.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.969+0000 7f4233bff700 1 -- 172.21.15.60:0/2410988613 shutdown_connections 2023-07-13T22:26:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.969+0000 7f4233bff700 1 -- 172.21.15.60:0/2410988613 wait complete. 2023-07-13T22:26:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.970+0000 7f4233bff700 1 Processor -- start 2023-07-13T22:26:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.970+0000 7f4233bff700 1 -- start start 2023-07-13T22:26:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.971+0000 7f4233bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 0x7f422c0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.971+0000 7f4233bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0cfcf0 0x7f422c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.971+0000 7f4233bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c15d160 0x7f422c0c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.971+0000 7f4233bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0ccbc0 con 0x7f422c0cd790 2023-07-13T22:26:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.971+0000 7f4233bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0c5e80 con 0x7f422c0cfcf0 2023-07-13T22:26:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.971+0000 7f4233bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c14d680 con 0x7f422c15d160 2023-07-13T22:26:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 0x7f422c0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 0x7f422c0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56562/0 (socket says 172.21.15.60:56562) 2023-07-13T22:26:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423199b700 1 -- 172.21.15.60:0/2626824843 learned_addr learned my addr 172.21.15.60:0/2626824843 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423119a700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0cfcf0 0x7f422c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423219c700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c15d160 0x7f422c0c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423199b700 1 -- 172.21.15.60:0/2626824843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c15d160 msgr2=0x7f422c0c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423199b700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c15d160 0x7f422c0c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423199b700 1 -- 172.21.15.60:0/2626824843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0cfcf0 msgr2=0x7f422c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423199b700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0cfcf0 0x7f422c0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423199b700 1 -- 172.21.15.60:0/2626824843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c009550 con 0x7f422c0cd790 2023-07-13T22:26:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423219c700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c15d160 0x7f422c0c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:26:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.972+0000 7f423199b700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 0x7f422c0cc680 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f422800e810 tx=0x7f422800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.973+0000 7f42227fc700 1 -- 172.21.15.60:0/2626824843 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228004230 con 0x7f422c0cd790 2023-07-13T22:26:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.973+0000 7f42227fc700 1 -- 172.21.15.60:0/2626824843 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42280043b0 con 0x7f422c0cd790 2023-07-13T22:26:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.973+0000 7f42227fc700 1 -- 172.21.15.60:0/2626824843 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42280038e0 con 0x7f422c0cd790 2023-07-13T22:26:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.973+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c0c7180 con 0x7f422c0cd790 2023-07-13T22:26:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.973+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c0c7740 con 0x7f422c0cd790 2023-07-13T22:26:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.974+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c0b8c10 con 0x7f422c0cd790 2023-07-13T22:26:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.975+0000 7f42227fc700 1 -- 172.21.15.60:0/2626824843 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4228003a40 con 0x7f422c0cd790 2023-07-13T22:26:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.976+0000 7f42227fc700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f421006ace0 0x7f421006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.976+0000 7f42227fc700 1 -- 172.21.15.60:0/2626824843 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4228088fd0 con 0x7f422c0cd790 2023-07-13T22:26:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.976+0000 7f423119a700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f421006ace0 0x7f421006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.977+0000 7f423119a700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f421006ace0 0x7f421006d190 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f421c000c00 tx=0x7f421c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:30.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:30.981+0000 7f42227fc700 1 -- 172.21.15.60:0/2626824843 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f42280579f0 con 0x7f422c0cd790 2023-07-13T22:26:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.237+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f422c04e280 con 0x7f422c0cd790 2023-07-13T22:26:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.237+0000 7f42227fc700 1 -- 172.21.15.60:0/2626824843 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f42280094c0 con 0x7f422c0cd790 2023-07-13T22:26:31.219 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:31.219 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:31.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f421006ace0 msgr2=0x7f421006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:31.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f421006ace0 0x7f421006d190 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f421c000c00 tx=0x7f421c008030 comp rx=0 tx=0).stop 2023-07-13T22:26:31.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 msgr2=0x7f422c0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:31.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 0x7f422c0cc680 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f422800e810 tx=0x7f422800eb20 comp rx=0 tx=0).stop 2023-07-13T22:26:31.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 shutdown_connections 2023-07-13T22:26:31.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f421006ace0 0x7f421006d190 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:31.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f422c0cd790 0x7f422c0cc680 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:31.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f422c15d160 0x7f422c0c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:31.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 --2- 172.21.15.60:0/2626824843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f422c0cfcf0 0x7f422c0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:31.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 >> 172.21.15.60:0/2626824843 conn(0x7f422c06b310 msgr2=0x7f422c0b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:31.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 shutdown_connections 2023-07-13T22:26:31.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:31.240+0000 7f4233bff700 1 -- 172.21.15.60:0/2626824843 wait complete. 2023-07-13T22:26:31.239 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:31.239 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:36.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:36.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.472+0000 7fcdd9bff700 1 Processor -- start 2023-07-13T22:26:36.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.472+0000 7fcdd9bff700 1 -- start start 2023-07-13T22:26:36.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.473+0000 7fcdd9bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdd4159510 0x7fcdd40c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:36.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.473+0000 7fcdd9bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd40c9510 0x7fcdd40703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:36.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.473+0000 7fcdd9bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd41598e0 con 0x7fcdd4070920 2023-07-13T22:26:36.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.473+0000 7fcdd9bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd4070da0 con 0x7fcdd4159510 2023-07-13T22:26:36.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.473+0000 7fcdd9bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd40ca8d0 con 0x7fcdd40c9510 2023-07-13T22:26:36.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.474+0000 7fcdd37fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdd4159510 0x7fcdd40c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:36.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.474+0000 7fcdd37fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdd4159510 0x7fcdd40c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52192/0 (socket says 172.21.15.60:52192) 2023-07-13T22:26:36.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.474+0000 7fcdd37fe700 1 -- 172.21.15.60:0/2450795025 learned_addr learned my addr 172.21.15.60:0/2450795025 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:36.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.474+0000 7fcdd2ffd700 1 --2- 172.21.15.60:0/2450795025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd40c9510 0x7fcdd40703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:36.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.474+0000 7fcdd37fe700 1 -- 172.21.15.60:0/2450795025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd40c9510 msgr2=0x7fcdd40703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:26:36.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.474+0000 7fcdd37fe700 1 --2- 172.21.15.60:0/2450795025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd40c9510 0x7fcdd40703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.474+0000 7fcdd37fe700 1 -- 172.21.15.60:0/2450795025 >> v1:172.21.15.105:6789/0 conn(0x7fcdd4070920 legacy=0x7fcdd415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:36.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.474+0000 7fcdd37fe700 1 -- 172.21.15.60:0/2450795025 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40cd340 con 0x7fcdd4159510 2023-07-13T22:26:36.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.474+0000 7fcdd37fe700 1 --2- 172.21.15.60:0/2450795025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 0x7fcdd40c8fd0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40098e0 tx=0x7fcdc4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4356b56cd610021 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:36.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd27fc700 1 -- 172.21.15.60:0/2450795025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400b8a0 con 0x7fcdd4159510 2023-07-13T22:26:36.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd27fc700 1 -- 172.21.15.60:0/2450795025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc400ba00 con 0x7fcdd4159510 2023-07-13T22:26:36.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd27fc700 1 -- 172.21.15.60:0/2450795025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400bdb0 con 0x7fcdd4159510 2023-07-13T22:26:36.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd9bff700 1 -- 172.21.15.60:0/2450795025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 msgr2=0x7fcdd40c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:36.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd9bff700 1 --2- 172.21.15.60:0/2450795025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 0x7fcdd40c8fd0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40098e0 tx=0x7fcdc4009bf0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd9bff700 1 -- 172.21.15.60:0/2450795025 shutdown_connections 2023-07-13T22:26:36.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd9bff700 1 --2- 172.21.15.60:0/2450795025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 0x7fcdd40c8fd0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd9bff700 1 --2- 172.21.15.60:0/2450795025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd40c9510 0x7fcdd40703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd9bff700 1 -- 172.21.15.60:0/2450795025 >> 172.21.15.60:0/2450795025 conn(0x7fcdd40cbe70 msgr2=0x7fcdd40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:36.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd9bff700 1 -- 172.21.15.60:0/2450795025 shutdown_connections 2023-07-13T22:26:36.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.475+0000 7fcdd9bff700 1 -- 172.21.15.60:0/2450795025 wait complete. 2023-07-13T22:26:36.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.477+0000 7fcdd9bff700 1 Processor -- start 2023-07-13T22:26:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.477+0000 7fcdd9bff700 1 -- start start 2023-07-13T22:26:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.478+0000 7fcdd9bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd4070920 0x7fcdd415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.478+0000 7fcdd9bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd40c9510 0x7fcdd41e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.478+0000 7fcdd9bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 0x7fcdd4154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.478+0000 7fcdd9bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd415b060 con 0x7fcdd4159510 2023-07-13T22:26:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.478+0000 7fcdd9bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd40b2540 con 0x7fcdd40c9510 2023-07-13T22:26:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.478+0000 7fcdd9bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd41e44f0 con 0x7fcdd4070920 2023-07-13T22:26:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.478+0000 7fcdd3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 0x7fcdd4154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 0x7fcdd4154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52208/0 (socket says 172.21.15.60:52208) 2023-07-13T22:26:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd3fff700 1 -- 172.21.15.60:0/3123014466 learned_addr learned my addr 172.21.15.60:0/3123014466 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd37fe700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd4070920 0x7fcdd415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd2ffd700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd40c9510 0x7fcdd41e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd3fff700 1 -- 172.21.15.60:0/3123014466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd4070920 msgr2=0x7fcdd415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd3fff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd4070920 0x7fcdd415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd3fff700 1 -- 172.21.15.60:0/3123014466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd40c9510 msgr2=0x7fcdd41e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:36.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd3fff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd40c9510 0x7fcdd41e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd3fff700 1 -- 172.21.15.60:0/3123014466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40094f0 con 0x7fcdd4159510 2023-07-13T22:26:36.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdd3fff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 0x7fcdd4154d60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00cc80 tx=0x7fcdcc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:36.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.479+0000 7fcdbbfff700 1 -- 172.21.15.60:0/3123014466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc003c00 con 0x7fcdd4159510 2023-07-13T22:26:36.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.480+0000 7fcdbbfff700 1 -- 172.21.15.60:0/3123014466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc003d60 con 0x7fcdd4159510 2023-07-13T22:26:36.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.480+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd41552a0 con 0x7fcdd4159510 2023-07-13T22:26:36.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.480+0000 7fcdbbfff700 1 -- 172.21.15.60:0/3123014466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00aaa0 con 0x7fcdd4159510 2023-07-13T22:26:36.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.480+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd415aa10 con 0x7fcdd4159510 2023-07-13T22:26:36.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.481+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd404e280 con 0x7fcdd4159510 2023-07-13T22:26:36.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.482+0000 7fcdbbfff700 1 -- 172.21.15.60:0/3123014466 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcdcc00ac00 con 0x7fcdd4159510 2023-07-13T22:26:36.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.483+0000 7fcdbbfff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcdc006ace0 0x7fcdc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:36.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.483+0000 7fcdbbfff700 1 -- 172.21.15.60:0/3123014466 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcdcc0894b0 con 0x7fcdd4159510 2023-07-13T22:26:36.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.483+0000 7fcdd37fe700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcdc006ace0 0x7fcdc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:36.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.484+0000 7fcdd37fe700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcdc006ace0 0x7fcdc006d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009130 tx=0x7fcdc4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:36.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.488+0000 7fcdbbfff700 1 -- 172.21.15.60:0/3123014466 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcdcc054560 con 0x7fcdd4159510 2023-07-13T22:26:36.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.737+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcdd406db20 con 0x7fcdd4159510 2023-07-13T22:26:36.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.738+0000 7fcdbbfff700 1 -- 172.21.15.60:0/3123014466 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fcdcc057b80 con 0x7fcdd4159510 2023-07-13T22:26:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:36.719 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:36.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcdc006ace0 msgr2=0x7fcdc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:36.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcdc006ace0 0x7fcdc006d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009130 tx=0x7fcdc4008030 comp rx=0 tx=0).stop 2023-07-13T22:26:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 msgr2=0x7fcdd4154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 0x7fcdd4154d60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00cc80 tx=0x7fcdcc00cf90 comp rx=0 tx=0).stop 2023-07-13T22:26:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 shutdown_connections 2023-07-13T22:26:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcdc006ace0 0x7fcdc006d190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdd4159510 0x7fcdd4154d60 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdd4070920 0x7fcdd415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 --2- 172.21.15.60:0/3123014466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd40c9510 0x7fcdd41e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 >> 172.21.15.60:0/3123014466 conn(0x7fcdd40cbe70 msgr2=0x7fcdd40abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 shutdown_connections 2023-07-13T22:26:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:36.740+0000 7fcdd9bff700 1 -- 172.21.15.60:0/3123014466 wait complete. 2023-07-13T22:26:36.739 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:36.740 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:41.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.973+0000 7fc05f7ff700 1 Processor -- start 2023-07-13T22:26:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.973+0000 7fc05f7ff700 1 -- start start 2023-07-13T22:26:41.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.975+0000 7fc05f7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc0580d0a50 0x7fc0580cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:41.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.975+0000 7fc05f7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0580d2410 con 0x7fc0580d0e20 2023-07-13T22:26:41.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.975+0000 7fc05f7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0580d2660 con 0x7fc0580cd260 2023-07-13T22:26:41.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.975+0000 7fc05f7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc05815c900 con 0x7fc0580d0a50 2023-07-13T22:26:41.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.975+0000 7fc05d59b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc0580d0a50 0x7fc0580cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:41.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.975+0000 7fc05d59b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc0580d0a50 0x7fc0580cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52212/0 (socket says 172.21.15.60:52212) 2023-07-13T22:26:41.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.975+0000 7fc05d59b700 1 -- 172.21.15.60:0/899369716 learned_addr learned my addr 172.21.15.60:0/899369716 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:41.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.975+0000 7fc05dd9c700 1 --1- 172.21.15.60:0/899369716 >> v1:172.21.15.60:6789/0 conn(0x7fc0580d0e20 0x7fc0580b5a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46324/0 (socket says 172.21.15.60:46324) 2023-07-13T22:26:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.976+0000 7fc05d59b700 1 -- 172.21.15.60:0/899369716 >> v1:172.21.15.105:6789/0 conn(0x7fc0580cd260 legacy=0x7fc0580d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.976+0000 7fc05d59b700 1 -- 172.21.15.60:0/899369716 >> v1:172.21.15.60:6789/0 conn(0x7fc0580d0e20 legacy=0x7fc0580b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.976+0000 7fc04ffff700 1 -- 172.21.15.60:0/899369716 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903598299 0 0) 0x7fc0580d2410 con 0x7fc0580d0e20 2023-07-13T22:26:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.976+0000 7fc05d59b700 1 -- 172.21.15.60:0/899369716 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0580d2940 con 0x7fc0580d0a50 2023-07-13T22:26:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.976+0000 7fc05d59b700 1 --2- 172.21.15.60:0/899369716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580d0a50 0x7fc0580cf8a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc0540098f0 tx=0x7fc054009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1c59ddf6305dc6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.976+0000 7fc04ffff700 1 -- 172.21.15.60:0/899369716 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05400b8a0 con 0x7fc0580d0a50 2023-07-13T22:26:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.976+0000 7fc04ffff700 1 -- 172.21.15.60:0/899369716 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05400ba00 con 0x7fc0580d0a50 2023-07-13T22:26:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.976+0000 7fc04ffff700 1 -- 172.21.15.60:0/899369716 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05400bd90 con 0x7fc0580d0a50 2023-07-13T22:26:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.977+0000 7fc05f7ff700 1 -- 172.21.15.60:0/899369716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580d0a50 msgr2=0x7fc0580cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.977+0000 7fc05f7ff700 1 --2- 172.21.15.60:0/899369716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580d0a50 0x7fc0580cf8a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc0540098f0 tx=0x7fc054009c00 comp rx=0 tx=0).stop 2023-07-13T22:26:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.977+0000 7fc05f7ff700 1 -- 172.21.15.60:0/899369716 shutdown_connections 2023-07-13T22:26:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.977+0000 7fc05f7ff700 1 --2- 172.21.15.60:0/899369716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580d0a50 0x7fc0580cf8a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.977+0000 7fc05f7ff700 1 -- 172.21.15.60:0/899369716 >> 172.21.15.60:0/899369716 conn(0x7fc058071600 msgr2=0x7fc0580719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.977+0000 7fc05f7ff700 1 -- 172.21.15.60:0/899369716 shutdown_connections 2023-07-13T22:26:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.977+0000 7fc05f7ff700 1 -- 172.21.15.60:0/899369716 wait complete. 2023-07-13T22:26:41.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.979+0000 7fc05f7ff700 1 Processor -- start 2023-07-13T22:26:41.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.979+0000 7fc05f7ff700 1 -- start start 2023-07-13T22:26:41.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.980+0000 7fc05f7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580cd260 0x7fc0581551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:41.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.980+0000 7fc05f7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0580d0a50 0x7fc0580cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:41.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.980+0000 7fc05f7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0580d0e20 0x7fc0580ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:41.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.980+0000 7fc05f7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580c8770 con 0x7fc0580cd260 2023-07-13T22:26:41.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.980+0000 7fc05f7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580c8a30 con 0x7fc0580d0a50 2023-07-13T22:26:41.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.980+0000 7fc05f7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580c8d10 con 0x7fc0580d0e20 2023-07-13T22:26:41.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.980+0000 7fc05d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580cd260 0x7fc0581551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:41.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.980+0000 7fc05d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580cd260 0x7fc0581551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52224/0 (socket says 172.21.15.60:52224) 2023-07-13T22:26:41.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.980+0000 7fc05d59b700 1 -- 172.21.15.60:0/1495841971 learned_addr learned my addr 172.21.15.60:0/1495841971 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05cd9a700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0580d0a50 0x7fc0580cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05dd9c700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0580d0e20 0x7fc0580ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05d59b700 1 -- 172.21.15.60:0/1495841971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0580d0e20 msgr2=0x7fc0580ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05d59b700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0580d0e20 0x7fc0580ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05d59b700 1 -- 172.21.15.60:0/1495841971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0580d0a50 msgr2=0x7fc0580cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05d59b700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0580d0a50 0x7fc0580cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05d59b700 1 -- 172.21.15.60:0/1495841971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc054009550 con 0x7fc0580cd260 2023-07-13T22:26:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05dd9c700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0580d0e20 0x7fc0580ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:26:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05d59b700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580cd260 0x7fc0581551b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc05806c710 tx=0x7fc054004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc04dffb700 1 -- 172.21.15.60:0/1495841971 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054011a70 con 0x7fc0580cd260 2023-07-13T22:26:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc04dffb700 1 -- 172.21.15.60:0/1495841971 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05401f070 con 0x7fc0580cd260 2023-07-13T22:26:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc05f7ff700 1 -- 172.21.15.60:0/1495841971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0580c8ff0 con 0x7fc0580cd260 2023-07-13T22:26:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.981+0000 7fc04dffb700 1 -- 172.21.15.60:0/1495841971 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054011dd0 con 0x7fc0580cd260 2023-07-13T22:26:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.982+0000 7fc05f7ff700 1 -- 172.21.15.60:0/1495841971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0580c79d0 con 0x7fc0580cd260 2023-07-13T22:26:41.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.983+0000 7fc0437fe700 1 -- 172.21.15.60:0/1495841971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0300052f0 con 0x7fc0580cd260 2023-07-13T22:26:41.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.983+0000 7fc04dffb700 1 -- 172.21.15.60:0/1495841971 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc05401a430 con 0x7fc0580cd260 2023-07-13T22:26:41.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.984+0000 7fc04dffb700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03c06ace0 0x7fc03c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:41.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.985+0000 7fc04dffb700 1 -- 172.21.15.60:0/1495841971 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc054021070 con 0x7fc0580cd260 2023-07-13T22:26:41.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.985+0000 7fc05cd9a700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03c06ace0 0x7fc03c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:41.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.985+0000 7fc05cd9a700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03c06ace0 0x7fc03c06d190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc0580b4cc0 tx=0x7fc048009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:41.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:41.989+0000 7fc04dffb700 1 -- 172.21.15.60:0/1495841971 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc054056050 con 0x7fc0580cd260 2023-07-13T22:26:42.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.241+0000 7fc0437fe700 1 -- 172.21.15.60:0/1495841971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc030005b20 con 0x7fc0580cd260 2023-07-13T22:26:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.242+0000 7fc04dffb700 1 -- 172.21.15.60:0/1495841971 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fc0540150c0 con 0x7fc0580cd260 2023-07-13T22:26:42.223 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:42.224 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 -- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03c06ace0 msgr2=0x7fc03c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:42.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03c06ace0 0x7fc03c06d190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc0580b4cc0 tx=0x7fc048009040 comp rx=0 tx=0).stop 2023-07-13T22:26:42.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 -- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580cd260 msgr2=0x7fc0581551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:42.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580cd260 0x7fc0581551b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc05806c710 tx=0x7fc054004750 comp rx=0 tx=0).stop 2023-07-13T22:26:42.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 -- 172.21.15.60:0/1495841971 shutdown_connections 2023-07-13T22:26:42.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc03c06ace0 0x7fc03c06d190 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:42.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0580cd260 0x7fc0581551b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:42.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0580d0e20 0x7fc0580ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:42.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 --2- 172.21.15.60:0/1495841971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0580d0a50 0x7fc0580cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:42.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 -- 172.21.15.60:0/1495841971 >> 172.21.15.60:0/1495841971 conn(0x7fc058071600 msgr2=0x7fc0580b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:42.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 -- 172.21.15.60:0/1495841971 shutdown_connections 2023-07-13T22:26:42.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:42.244+0000 7fc0437fe700 1 -- 172.21.15.60:0/1495841971 wait complete. 2023-07-13T22:26:42.244 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:42.244 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:47.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.478+0000 7fe036bff700 1 Processor -- start 2023-07-13T22:26:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.478+0000 7fe036bff700 1 -- start start 2023-07-13T22:26:47.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.479+0000 7fe036bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0300cd260 0x7fe0300d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:47.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.479+0000 7fe036bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0300d2410 con 0x7fe0300d0e20 2023-07-13T22:26:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.479+0000 7fe036bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0300d2660 con 0x7fe0300d0a50 2023-07-13T22:26:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.479+0000 7fe036bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03015c900 con 0x7fe0300cd260 2023-07-13T22:26:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.479+0000 7fe02ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0300cd260 0x7fe0300d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.479+0000 7fe02ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0300cd260 0x7fe0300d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42004/0 (socket says 172.21.15.60:42004) 2023-07-13T22:26:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.479+0000 7fe02ffff700 1 -- 172.21.15.60:0/2395277252 learned_addr learned my addr 172.21.15.60:0/2395277252 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.479+0000 7fe03519c700 1 --1- 172.21.15.60:0/2395277252 >> v1:172.21.15.60:6789/0 conn(0x7fe0300d0e20 0x7fe0300b5a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49506/0 (socket says 172.21.15.60:49506) 2023-07-13T22:26:47.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.480+0000 7fe02ffff700 1 -- 172.21.15.60:0/2395277252 >> v1:172.21.15.105:6789/0 conn(0x7fe0300d0a50 legacy=0x7fe0300cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:47.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.480+0000 7fe02ffff700 1 -- 172.21.15.60:0/2395277252 >> v1:172.21.15.60:6789/0 conn(0x7fe0300d0e20 legacy=0x7fe0300b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:47.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.480+0000 7fe02ffff700 1 -- 172.21.15.60:0/2395277252 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300d2940 con 0x7fe0300cd260 2023-07-13T22:26:47.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.480+0000 7fe02f7fe700 1 -- 172.21.15.60:0/2395277252 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177425302 0 0) 0x7fe0300d2410 con 0x7fe0300d0e20 2023-07-13T22:26:47.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.480+0000 7fe02ffff700 1 --2- 172.21.15.60:0/2395277252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300cd260 0x7fe0300d3420 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0098f0 tx=0x7fe01c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22674464c9db217 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:47.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.480+0000 7fe02f7fe700 1 -- 172.21.15.60:0/2395277252 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00b8a0 con 0x7fe0300cd260 2023-07-13T22:26:47.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.480+0000 7fe02f7fe700 1 -- 172.21.15.60:0/2395277252 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c00ba00 con 0x7fe0300cd260 2023-07-13T22:26:47.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.480+0000 7fe02f7fe700 1 -- 172.21.15.60:0/2395277252 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00bd90 con 0x7fe0300cd260 2023-07-13T22:26:47.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.481+0000 7fe036bff700 1 -- 172.21.15.60:0/2395277252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300cd260 msgr2=0x7fe0300d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:47.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.481+0000 7fe036bff700 1 --2- 172.21.15.60:0/2395277252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300cd260 0x7fe0300d3420 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0098f0 tx=0x7fe01c009c00 comp rx=0 tx=0).stop 2023-07-13T22:26:47.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.481+0000 7fe036bff700 1 -- 172.21.15.60:0/2395277252 shutdown_connections 2023-07-13T22:26:47.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.481+0000 7fe036bff700 1 --2- 172.21.15.60:0/2395277252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300cd260 0x7fe0300d3420 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:47.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.481+0000 7fe036bff700 1 -- 172.21.15.60:0/2395277252 >> 172.21.15.60:0/2395277252 conn(0x7fe030071600 msgr2=0x7fe0300719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:47.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.481+0000 7fe036bff700 1 -- 172.21.15.60:0/2395277252 shutdown_connections 2023-07-13T22:26:47.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.481+0000 7fe036bff700 1 -- 172.21.15.60:0/2395277252 wait complete. 2023-07-13T22:26:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.483+0000 7fe036bff700 1 Processor -- start 2023-07-13T22:26:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.483+0000 7fe036bff700 1 -- start start 2023-07-13T22:26:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.484+0000 7fe036bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0300cd260 0x7fe0300c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.484+0000 7fe036bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0300d0a50 0x7fe0300cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.484+0000 7fe036bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300d0e20 0x7fe0301f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:47.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.484+0000 7fe036bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c95f0 con 0x7fe0300d0e20 2023-07-13T22:26:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.484+0000 7fe036bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c9d20 con 0x7fe0300cd260 2023-07-13T22:26:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.484+0000 7fe036bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0301535b0 con 0x7fe0300d0a50 2023-07-13T22:26:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.484+0000 7fe03519c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300d0e20 0x7fe0301f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.484+0000 7fe03519c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300d0e20 0x7fe0301f5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42006/0 (socket says 172.21.15.60:42006) 2023-07-13T22:26:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.484+0000 7fe03519c700 1 -- 172.21.15.60:0/976367490 learned_addr learned my addr 172.21.15.60:0/976367490 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe03499b700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0300cd260 0x7fe0300c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe02ffff700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0300d0a50 0x7fe0300cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe03519c700 1 -- 172.21.15.60:0/976367490 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0300d0a50 msgr2=0x7fe0300cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:47.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe03519c700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0300d0a50 0x7fe0300cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:47.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe03519c700 1 -- 172.21.15.60:0/976367490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0300cd260 msgr2=0x7fe0300c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:47.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe03519c700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0300cd260 0x7fe0300c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe03519c700 1 -- 172.21.15.60:0/976367490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c009550 con 0x7fe0300d0e20 2023-07-13T22:26:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe03519c700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300d0e20 0x7fe0301f5380 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe02800cba0 tx=0x7fe02800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe02d7fa700 1 -- 172.21.15.60:0/976367490 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02800ab80 con 0x7fe0300d0e20 2023-07-13T22:26:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe02d7fa700 1 -- 172.21.15.60:0/976367490 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02800ad00 con 0x7fe0300d0e20 2023-07-13T22:26:47.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0300c9030 con 0x7fe0300d0e20 2023-07-13T22:26:47.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe02d7fa700 1 -- 172.21.15.60:0/976367490 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028007310 con 0x7fe0300d0e20 2023-07-13T22:26:47.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.485+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030153240 con 0x7fe0300d0e20 2023-07-13T22:26:47.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.486+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0301530a0 con 0x7fe0300d0e20 2023-07-13T22:26:47.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.487+0000 7fe02d7fa700 1 -- 172.21.15.60:0/976367490 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe028007470 con 0x7fe0300d0e20 2023-07-13T22:26:47.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.488+0000 7fe02d7fa700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe01406ace0 0x7fe01406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:47.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.489+0000 7fe02d7fa700 1 -- 172.21.15.60:0/976367490 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0280893e0 con 0x7fe0300d0e20 2023-07-13T22:26:47.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.489+0000 7fe03499b700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe01406ace0 0x7fe01406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:47.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.489+0000 7fe03499b700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe01406ace0 0x7fe01406d190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe024000f60 tx=0x7fe024009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:47.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.493+0000 7fe02d7fa700 1 -- 172.21.15.60:0/976367490 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe0301530a0 con 0x7fe0300d0e20 2023-07-13T22:26:47.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.745+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe0301530a0 con 0x7fe0300d0e20 2023-07-13T22:26:47.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.745+0000 7fe02d7fa700 1 -- 172.21.15.60:0/976367490 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fe0301530a0 con 0x7fe0300d0e20 2023-07-13T22:26:47.727 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:47.727 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:47.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe01406ace0 msgr2=0x7fe01406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:47.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe01406ace0 0x7fe01406d190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe024000f60 tx=0x7fe024009040 comp rx=0 tx=0).stop 2023-07-13T22:26:47.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300d0e20 msgr2=0x7fe0301f5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:47.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300d0e20 0x7fe0301f5380 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe02800cba0 tx=0x7fe02800cf60 comp rx=0 tx=0).stop 2023-07-13T22:26:47.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 shutdown_connections 2023-07-13T22:26:47.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe01406ace0 0x7fe01406d190 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:47.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0300d0e20 0x7fe0301f5380 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:47.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0300cd260 0x7fe0300c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:47.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 --2- 172.21.15.60:0/976367490 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0300d0a50 0x7fe0300cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:47.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 >> 172.21.15.60:0/976367490 conn(0x7fe030071600 msgr2=0x7fe030156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:47.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 shutdown_connections 2023-07-13T22:26:47.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:47.748+0000 7fe036bff700 1 -- 172.21.15.60:0/976367490 wait complete. 2023-07-13T22:26:47.747 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:47.747 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:52.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.981+0000 7f07affff700 1 Processor -- start 2023-07-13T22:26:52.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.982+0000 7f07affff700 1 -- start start 2023-07-13T22:26:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07affff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07a80d0a50 0x7f07a80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07affff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80cd260 0x7f07a80d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07affff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f07a80d0e20 0x7f07a80d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07affff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a815c900 con 0x7f07a80d0a50 2023-07-13T22:26:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07affff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a80d2410 con 0x7f07a80d0e20 2023-07-13T22:26:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07affff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80d2660 con 0x7f07a80cd260 2023-07-13T22:26:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07add9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07a80d0a50 0x7f07a80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07add9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07a80d0a50 0x7f07a80cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42018/0 (socket says 172.21.15.60:42018) 2023-07-13T22:26:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07add9b700 1 -- 172.21.15.60:0/349491352 learned_addr learned my addr 172.21.15.60:0/349491352 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07add9b700 1 -- 172.21.15.60:0/349491352 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80cd260 msgr2=0x7f07a80d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:26:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07add9b700 1 --2- 172.21.15.60:0/349491352 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80cd260 0x7f07a80d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07add9b700 1 -- 172.21.15.60:0/349491352 >> v2:172.21.15.105:3300/0 conn(0x7f07a80d0e20 msgr2=0x7f07a80d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:26:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07add9b700 1 --2- 172.21.15.60:0/349491352 >> v2:172.21.15.105:3300/0 conn(0x7f07a80d0e20 0x7f07a80d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.983+0000 7f07add9b700 1 -- 172.21.15.60:0/349491352 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a80d2940 con 0x7f07a80d0a50 2023-07-13T22:26:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.984+0000 7f07add9b700 1 --2- 172.21.15.60:0/349491352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0a50 0x7f07a80cf8a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f07a8072560 tx=0x7f07a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad93efe9608a953c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.984+0000 7f07acd99700 1 -- 172.21.15.60:0/349491352 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400b870 con 0x7f07a80d0a50 2023-07-13T22:26:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.984+0000 7f07acd99700 1 -- 172.21.15.60:0/349491352 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a400b9d0 con 0x7f07a80d0a50 2023-07-13T22:26:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.984+0000 7f07acd99700 1 -- 172.21.15.60:0/349491352 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400bd60 con 0x7f07a80d0a50 2023-07-13T22:26:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.984+0000 7f07affff700 1 -- 172.21.15.60:0/349491352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0a50 msgr2=0x7f07a80cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.984+0000 7f07affff700 1 --2- 172.21.15.60:0/349491352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0a50 0x7f07a80cf8a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f07a8072560 tx=0x7f07a4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:26:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.985+0000 7f07affff700 1 -- 172.21.15.60:0/349491352 shutdown_connections 2023-07-13T22:26:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.985+0000 7f07affff700 1 --2- 172.21.15.60:0/349491352 >> v2:172.21.15.105:3300/0 conn(0x7f07a80d0e20 0x7f07a80d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.985+0000 7f07affff700 1 --2- 172.21.15.60:0/349491352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0a50 0x7f07a80cf8a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.985+0000 7f07affff700 1 --2- 172.21.15.60:0/349491352 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80cd260 0x7f07a80d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:52.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.985+0000 7f07affff700 1 -- 172.21.15.60:0/349491352 >> 172.21.15.60:0/349491352 conn(0x7f07a8071600 msgr2=0x7f07a80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.985+0000 7f07affff700 1 -- 172.21.15.60:0/349491352 shutdown_connections 2023-07-13T22:26:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.985+0000 7f07affff700 1 -- 172.21.15.60:0/349491352 wait complete. 2023-07-13T22:26:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.986+0000 7f07affff700 1 Processor -- start 2023-07-13T22:26:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.987+0000 7f07affff700 1 -- start start 2023-07-13T22:26:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.987+0000 7f07affff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a80cd260 0x7f07a80cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.987+0000 7f07affff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80d0a50 0x7f07a81f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07affff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0e20 0x7f07a8158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07affff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80cb580 con 0x7f07a80d0e20 2023-07-13T22:26:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07affff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a815bef0 con 0x7f07a80cd260 2023-07-13T22:26:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07affff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80ccd70 con 0x7f07a80d0a50 2023-07-13T22:26:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0e20 0x7f07a8158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0e20 0x7f07a8158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42032/0 (socket says 172.21.15.60:42032) 2023-07-13T22:26:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ae59c700 1 -- 172.21.15.60:0/664815566 learned_addr learned my addr 172.21.15.60:0/664815566 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07add9b700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a80cd260 0x7f07a80cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ad59a700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80d0a50 0x7f07a81f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ae59c700 1 -- 172.21.15.60:0/664815566 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80d0a50 msgr2=0x7f07a81f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ae59c700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80d0a50 0x7f07a81f1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ae59c700 1 -- 172.21.15.60:0/664815566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a80cd260 msgr2=0x7f07a80cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ae59c700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a80cd260 0x7f07a80cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ae59c700 1 -- 172.21.15.60:0/664815566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a4009550 con 0x7f07a80d0e20 2023-07-13T22:26:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.988+0000 7f07ad59a700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80d0a50 0x7f07a81f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:26:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.989+0000 7f07ae59c700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0e20 0x7f07a8158d60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f07a000b9e0 tx=0x7f07a000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.989+0000 7f079e7fc700 1 -- 172.21.15.60:0/664815566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000c930 con 0x7f07a80d0e20 2023-07-13T22:26:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.989+0000 7f079e7fc700 1 -- 172.21.15.60:0/664815566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a000cad0 con 0x7f07a80d0e20 2023-07-13T22:26:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.989+0000 7f079e7fc700 1 -- 172.21.15.60:0/664815566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000ce80 con 0x7f07a80d0e20 2023-07-13T22:26:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.989+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a80cb790 con 0x7f07a80d0e20 2023-07-13T22:26:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.989+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a80cbd50 con 0x7f07a80d0e20 2023-07-13T22:26:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.990+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a8066700 con 0x7f07a80d0e20 2023-07-13T22:26:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.991+0000 7f079e7fc700 1 -- 172.21.15.60:0/664815566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f07a0014440 con 0x7f07a80d0e20 2023-07-13T22:26:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.992+0000 7f079e7fc700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f078c06ace0 0x7f078c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.992+0000 7f079e7fc700 1 -- 172.21.15.60:0/664815566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f07a0006e40 con 0x7f07a80d0e20 2023-07-13T22:26:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.992+0000 7f07add9b700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f078c06ace0 0x7f078c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.993+0000 7f07add9b700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f078c06ace0 0x7f078c06d190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07a4000c00 tx=0x7f07a4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:52.996+0000 7f079e7fc700 1 -- 172.21.15.60:0/664815566 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f07a0054110 con 0x7f07a80d0e20 2023-07-13T22:26:53.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.244+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f07a804e280 con 0x7f07a80d0e20 2023-07-13T22:26:53.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.245+0000 7f079e7fc700 1 -- 172.21.15.60:0/664815566 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f07a00190c0 con 0x7f07a80d0e20 2023-07-13T22:26:53.226 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:53.227 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:53.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f078c06ace0 msgr2=0x7f078c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:53.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f078c06ace0 0x7f078c06d190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07a4000c00 tx=0x7f07a4008030 comp rx=0 tx=0).stop 2023-07-13T22:26:53.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0e20 msgr2=0x7f07a8158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:53.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0e20 0x7f07a8158d60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f07a000b9e0 tx=0x7f07a000bcf0 comp rx=0 tx=0).stop 2023-07-13T22:26:53.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 shutdown_connections 2023-07-13T22:26:53.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f078c06ace0 0x7f078c06d190 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:53.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07a80d0e20 0x7f07a8158d60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:53.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07a80cd260 0x7f07a80cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:53.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 --2- 172.21.15.60:0/664815566 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07a80d0a50 0x7f07a81f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:53.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 >> 172.21.15.60:0/664815566 conn(0x7f07a8071600 msgr2=0x7f07a80cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:53.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 shutdown_connections 2023-07-13T22:26:53.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:53.247+0000 7f07affff700 1 -- 172.21.15.60:0/664815566 wait complete. 2023-07-13T22:26:53.246 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:53.247 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:26:56.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:26:56.589 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:26:58.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:26:58.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.481+0000 7f5a77bff700 1 Processor -- start 2023-07-13T22:26:58.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.481+0000 7f5a77bff700 1 -- start start 2023-07-13T22:26:58.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.482+0000 7f5a77bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a7015a390 0x7f5a700b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:58.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.482+0000 7f5a77bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a700cf1f0 0x7f5a7006c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:58.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.482+0000 7f5a77bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a700cf640 con 0x7f5a700ca520 2023-07-13T22:26:58.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.482+0000 7f5a77bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a700cab10 con 0x7f5a700cf1f0 2023-07-13T22:26:58.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.482+0000 7f5a77bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700cadf0 con 0x7f5a7015a390 2023-07-13T22:26:58.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.482+0000 7f5a7619c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a700cf1f0 0x7f5a7006c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:58.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.482+0000 7f5a7619c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a700cf1f0 0x7f5a7006c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42266/0 (socket says 172.21.15.60:42266) 2023-07-13T22:26:58.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.482+0000 7f5a7619c700 1 -- 172.21.15.60:0/3546652462 learned_addr learned my addr 172.21.15.60:0/3546652462 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a7599b700 1 --2- 172.21.15.60:0/3546652462 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a7015a390 0x7f5a700b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a7619c700 1 -- 172.21.15.60:0/3546652462 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a7015a390 msgr2=0x7f5a700b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:26:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a7619c700 1 --2- 172.21.15.60:0/3546652462 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a7015a390 0x7f5a700b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a7619c700 1 -- 172.21.15.60:0/3546652462 >> v1:172.21.15.105:6789/0 conn(0x7f5a700ca520 legacy=0x7f5a70070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a7619c700 1 -- 172.21.15.60:0/3546652462 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7006e030 con 0x7f5a700cf1f0 2023-07-13T22:26:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a7599b700 1 --2- 172.21.15.60:0/3546652462 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a7015a390 0x7f5a700b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:26:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a7619c700 1 --2- 172.21.15.60:0/3546652462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 0x7f5a7006c3e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009e50 tx=0x7f5a6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7575c72d392ad8f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:58.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a74999700 1 -- 172.21.15.60:0/3546652462 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800e070 con 0x7f5a700cf1f0 2023-07-13T22:26:58.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a74999700 1 -- 172.21.15.60:0/3546652462 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6800b9c0 con 0x7f5a700cf1f0 2023-07-13T22:26:58.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.483+0000 7f5a74999700 1 -- 172.21.15.60:0/3546652462 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800bd70 con 0x7f5a700cf1f0 2023-07-13T22:26:58.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.484+0000 7f5a77bff700 1 -- 172.21.15.60:0/3546652462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 msgr2=0x7f5a7006c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:58.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.484+0000 7f5a77bff700 1 --2- 172.21.15.60:0/3546652462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 0x7f5a7006c3e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009e50 tx=0x7f5a6800b330 comp rx=0 tx=0).stop 2023-07-13T22:26:58.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.484+0000 7f5a77bff700 1 -- 172.21.15.60:0/3546652462 shutdown_connections 2023-07-13T22:26:58.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.484+0000 7f5a77bff700 1 --2- 172.21.15.60:0/3546652462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 0x7f5a7006c3e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.484+0000 7f5a77bff700 1 --2- 172.21.15.60:0/3546652462 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a7015a390 0x7f5a700b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.484+0000 7f5a77bff700 1 -- 172.21.15.60:0/3546652462 >> 172.21.15.60:0/3546652462 conn(0x7f5a700cbe70 msgr2=0x7f5a700ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:58.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.484+0000 7f5a77bff700 1 -- 172.21.15.60:0/3546652462 shutdown_connections 2023-07-13T22:26:58.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.484+0000 7f5a77bff700 1 -- 172.21.15.60:0/3546652462 wait complete. 2023-07-13T22:26:58.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.486+0000 7f5a77bff700 1 Processor -- start 2023-07-13T22:26:58.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.486+0000 7f5a77bff700 1 -- start start 2023-07-13T22:26:58.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a77bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a700ca520 0x7f5a70147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:58.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a77bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 0x7f5a700c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:58.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a77bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a7015a390 0x7f5a700cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:58.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a77bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700c4730 con 0x7f5a700cf1f0 2023-07-13T22:26:58.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a77bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700c5100 con 0x7f5a7015a390 2023-07-13T22:26:58.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a77bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700c3b20 con 0x7f5a700ca520 2023-07-13T22:26:58.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a7519a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 0x7f5a700c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:58.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a7519a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 0x7f5a700c4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42282/0 (socket says 172.21.15.60:42282) 2023-07-13T22:26:58.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a7519a700 1 -- 172.21.15.60:0/816835916 learned_addr learned my addr 172.21.15.60:0/816835916 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:26:58.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a7599b700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a700ca520 0x7f5a70147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:58.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.487+0000 7f5a7619c700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a7015a390 0x7f5a700cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:58.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a7519a700 1 -- 172.21.15.60:0/816835916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a700ca520 msgr2=0x7f5a70147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:58.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a7519a700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a700ca520 0x7f5a70147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a7519a700 1 -- 172.21.15.60:0/816835916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a7015a390 msgr2=0x7f5a700cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:58.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a7519a700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a7015a390 0x7f5a700cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a7519a700 1 -- 172.21.15.60:0/816835916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68009b30 con 0x7f5a700cf1f0 2023-07-13T22:26:58.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a7519a700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 0x7f5a700c4bc0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000c4e0 tx=0x7f5a6000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:58.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a667fc700 1 -- 172.21.15.60:0/816835916 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600052d0 con 0x7f5a700cf1f0 2023-07-13T22:26:58.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a667fc700 1 -- 172.21.15.60:0/816835916 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60005430 con 0x7f5a700cf1f0 2023-07-13T22:26:58.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a667fc700 1 -- 172.21.15.60:0/816835916 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600038c0 con 0x7f5a700cf1f0 2023-07-13T22:26:58.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a700c3e00 con 0x7f5a700cf1f0 2023-07-13T22:26:58.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.488+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a700c43c0 con 0x7f5a700cf1f0 2023-07-13T22:26:58.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.489+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a70155f50 con 0x7f5a700cf1f0 2023-07-13T22:26:58.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.490+0000 7f5a667fc700 1 -- 172.21.15.60:0/816835916 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a60003a20 con 0x7f5a700cf1f0 2023-07-13T22:26:58.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.491+0000 7f5a667fc700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a5406ace0 0x7f5a5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:58.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.491+0000 7f5a667fc700 1 -- 172.21.15.60:0/816835916 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a60088880 con 0x7f5a700cf1f0 2023-07-13T22:26:58.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.491+0000 7f5a7599b700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a5406ace0 0x7f5a5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:58.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.492+0000 7f5a7599b700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a5406ace0 0x7f5a5406d190 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009930 tx=0x7f5a6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.496+0000 7f5a667fc700 1 -- 172.21.15.60:0/816835916 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5a60053890 con 0x7f5a700cf1f0 2023-07-13T22:26:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.749+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5a7004e280 con 0x7f5a700cf1f0 2023-07-13T22:26:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.749+0000 7f5a667fc700 1 -- 172.21.15.60:0/816835916 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f5a60056eb0 con 0x7f5a700cf1f0 2023-07-13T22:26:58.730 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:26:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:26:58.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a5406ace0 msgr2=0x7f5a5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:58.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a5406ace0 0x7f5a5406d190 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009930 tx=0x7f5a6c008040 comp rx=0 tx=0).stop 2023-07-13T22:26:58.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 msgr2=0x7f5a700c4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:58.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 0x7f5a700c4bc0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000c4e0 tx=0x7f5a6000c7f0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 shutdown_connections 2023-07-13T22:26:58.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a5406ace0 0x7f5a5406d190 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a7015a390 0x7f5a700cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a700ca520 0x7f5a70147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 --2- 172.21.15.60:0/816835916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a700cf1f0 0x7f5a700c4bc0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:58.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 >> 172.21.15.60:0/816835916 conn(0x7f5a700cbe70 msgr2=0x7f5a70154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:58.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 shutdown_connections 2023-07-13T22:26:58.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:26:58.752+0000 7f5a77bff700 1 -- 172.21.15.60:0/816835916 wait complete. 2023-07-13T22:26:58.749 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:26:58.750 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:27:03.751 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:27:03.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.985+0000 7f2353fff700 1 Processor -- start 2023-07-13T22:27:03.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.985+0000 7f2353fff700 1 -- start start 2023-07-13T22:27:03.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.986+0000 7f2353fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f234c0cd260 0x7f234c0d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:03.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.986+0000 7f2353fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234c0d2410 con 0x7f234c0d0a50 2023-07-13T22:27:03.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.986+0000 7f2353fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234c0d2660 con 0x7f234c0d0e20 2023-07-13T22:27:03.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.986+0000 7f2353fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234c15c900 con 0x7f234c0cd260 2023-07-13T22:27:03.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.987+0000 7f235159a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f234c0cd260 0x7f234c0d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:03.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.987+0000 7f2351d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f234c0d0a50 0x7f234c0cf8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46158/0 (socket says 172.21.15.60:46158) 2023-07-13T22:27:03.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.987+0000 7f2351d9b700 1 -- 172.21.15.60:0/4247747574 learned_addr learned my addr 172.21.15.60:0/4247747574 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:03.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.987+0000 7f235159a700 1 -- 172.21.15.60:0/4247747574 >> v1:172.21.15.105:6789/0 conn(0x7f234c0d0e20 legacy=0x7f234c0b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:03.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.987+0000 7f235159a700 1 -- 172.21.15.60:0/4247747574 >> v1:172.21.15.60:6789/0 conn(0x7f234c0d0a50 legacy=0x7f234c0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:03.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.987+0000 7f235159a700 1 -- 172.21.15.60:0/4247747574 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c0d2940 con 0x7f234c0cd260 2023-07-13T22:27:03.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.987+0000 7f2350d99700 1 -- 172.21.15.60:0/4247747574 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268858337 0 0) 0x7f234c0d2410 con 0x7f234c0d0a50 2023-07-13T22:27:03.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.987+0000 7f235159a700 1 --2- 172.21.15.60:0/4247747574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0cd260 0x7f234c0d3420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f233c009840 tx=0x7f233c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209a2a99505d3ff7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:03.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.988+0000 7f2350d99700 1 -- 172.21.15.60:0/4247747574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c011830 con 0x7f234c0cd260 2023-07-13T22:27:03.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.988+0000 7f2350d99700 1 -- 172.21.15.60:0/4247747574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c011990 con 0x7f234c0cd260 2023-07-13T22:27:03.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.988+0000 7f2350d99700 1 -- 172.21.15.60:0/4247747574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c011d20 con 0x7f234c0cd260 2023-07-13T22:27:03.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.988+0000 7f2353fff700 1 -- 172.21.15.60:0/4247747574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0cd260 msgr2=0x7f234c0d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:03.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.988+0000 7f2353fff700 1 --2- 172.21.15.60:0/4247747574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0cd260 0x7f234c0d3420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f233c009840 tx=0x7f233c009b50 comp rx=0 tx=0).stop 2023-07-13T22:27:03.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.988+0000 7f2353fff700 1 -- 172.21.15.60:0/4247747574 shutdown_connections 2023-07-13T22:27:03.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.988+0000 7f2353fff700 1 --2- 172.21.15.60:0/4247747574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0cd260 0x7f234c0d3420 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:03.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.988+0000 7f2353fff700 1 -- 172.21.15.60:0/4247747574 >> 172.21.15.60:0/4247747574 conn(0x7f234c071600 msgr2=0x7f234c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:03.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.988+0000 7f2353fff700 1 -- 172.21.15.60:0/4247747574 shutdown_connections 2023-07-13T22:27:03.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.989+0000 7f2353fff700 1 -- 172.21.15.60:0/4247747574 wait complete. 2023-07-13T22:27:03.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.990+0000 7f2353fff700 1 Processor -- start 2023-07-13T22:27:03.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.990+0000 7f2353fff700 1 -- start start 2023-07-13T22:27:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.991+0000 7f2353fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f234c0cd260 0x7f234c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.991+0000 7f2353fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f234c0d0a50 0x7f234c0c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.991+0000 7f2353fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0d0e20 0x7f234c1f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.991+0000 7f2353fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c154a00 con 0x7f234c0d0e20 2023-07-13T22:27:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.991+0000 7f2353fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c0c8e70 con 0x7f234c0d0a50 2023-07-13T22:27:03.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.991+0000 7f2353fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234c0c9150 con 0x7f234c0cd260 2023-07-13T22:27:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0d0e20 0x7f234c1f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0d0e20 0x7f234c1f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42312/0 (socket says 172.21.15.60:42312) 2023-07-13T22:27:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235259c700 1 -- 172.21.15.60:0/3491641203 learned_addr learned my addr 172.21.15.60:0/3491641203 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f2351d9b700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f234c0cd260 0x7f234c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235159a700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f234c0d0a50 0x7f234c0c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235259c700 1 -- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f234c0cd260 msgr2=0x7f234c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:03.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235259c700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f234c0cd260 0x7f234c0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235259c700 1 -- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f234c0d0a50 msgr2=0x7f234c0c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235259c700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f234c0d0a50 0x7f234c0c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235259c700 1 -- 172.21.15.60:0/3491641203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c0094a0 con 0x7f234c0d0e20 2023-07-13T22:27:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f235159a700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f234c0d0a50 0x7f234c0c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.992+0000 7f2351d9b700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f234c0cd260 0x7f234c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.993+0000 7f235259c700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0d0e20 0x7f234c1f2f00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f234400cb50 tx=0x7f234400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.993+0000 7f23427fc700 1 -- 172.21.15.60:0/3491641203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344003c00 con 0x7f234c0d0e20 2023-07-13T22:27:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.993+0000 7f23427fc700 1 -- 172.21.15.60:0/3491641203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2344003d60 con 0x7f234c0d0e20 2023-07-13T22:27:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.993+0000 7f23427fc700 1 -- 172.21.15.60:0/3491641203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234400ab70 con 0x7f234c0d0e20 2023-07-13T22:27:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.993+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c0c9430 con 0x7f234c0d0e20 2023-07-13T22:27:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.993+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c1f1e50 con 0x7f234c0d0e20 2023-07-13T22:27:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.994+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c04e280 con 0x7f234c0d0e20 2023-07-13T22:27:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.995+0000 7f23427fc700 1 -- 172.21.15.60:0/3491641203 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f234400acd0 con 0x7f234c0d0e20 2023-07-13T22:27:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.996+0000 7f23427fc700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f233006ace0 0x7f233006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.996+0000 7f23427fc700 1 -- 172.21.15.60:0/3491641203 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2344089560 con 0x7f234c0d0e20 2023-07-13T22:27:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.996+0000 7f2351d9b700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f233006ace0 0x7f233006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:03.997+0000 7f2351d9b700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f233006ace0 0x7f233006d190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2348000f60 tx=0x7f2348007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.000+0000 7f23427fc700 1 -- 172.21.15.60:0/3491641203 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2344054570 con 0x7f234c0d0e20 2023-07-13T22:27:04.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.250+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f234c0cf1e0 con 0x7f234c0d0e20 2023-07-13T22:27:04.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.250+0000 7f23427fc700 1 -- 172.21.15.60:0/3491641203 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f2344057b90 con 0x7f234c0d0e20 2023-07-13T22:27:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:04.230 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:27:04.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.252+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f233006ace0 msgr2=0x7f233006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.252+0000 7f2353fff700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f233006ace0 0x7f233006d190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2348000f60 tx=0x7f2348007d00 comp rx=0 tx=0).stop 2023-07-13T22:27:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.252+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0d0e20 msgr2=0x7f234c1f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.252+0000 7f2353fff700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0d0e20 0x7f234c1f2f00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f234400cb50 tx=0x7f234400cf10 comp rx=0 tx=0).stop 2023-07-13T22:27:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.252+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 shutdown_connections 2023-07-13T22:27:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.252+0000 7f2353fff700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f233006ace0 0x7f233006d190 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.252+0000 7f2353fff700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f234c0d0e20 0x7f234c1f2f00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.252+0000 7f2353fff700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f234c0cd260 0x7f234c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.252+0000 7f2353fff700 1 --2- 172.21.15.60:0/3491641203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f234c0d0a50 0x7f234c0c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.253+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 >> 172.21.15.60:0/3491641203 conn(0x7f234c071600 msgr2=0x7f234c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.253+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 shutdown_connections 2023-07-13T22:27:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:04.253+0000 7f2353fff700 1 -- 172.21.15.60:0/3491641203 wait complete. 2023-07-13T22:27:04.250 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:27:04.251 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:27:09.252 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:27:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.485+0000 7f53135ff700 1 Processor -- start 2023-07-13T22:27:09.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.486+0000 7f53135ff700 1 -- start start 2023-07-13T22:27:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f53135ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f530c159510 0x7f530c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f53135ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f530c070af0 0x7f530c0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f53135ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c1598e0 con 0x7f530c0c9510 2023-07-13T22:27:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f53135ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c0ca8d0 con 0x7f530c070af0 2023-07-13T22:27:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f53135ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c06d290 con 0x7f530c159510 2023-07-13T22:27:09.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f5310b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f530c0c9510 0x7f530c0703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48478/0 (socket says 172.21.15.60:48478) 2023-07-13T22:27:09.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f5310b9a700 1 -- 172.21.15.60:0/3844297399 learned_addr learned my addr 172.21.15.60:0/3844297399 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:09.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f5311b9c700 1 --2- 172.21.15.60:0/3844297399 >> v2:172.21.15.60:3300/0 conn(0x7f530c070af0 0x7f530c0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:09.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f531139b700 1 --2- 172.21.15.60:0/3844297399 >> v2:172.21.15.105:3300/0 conn(0x7f530c159510 0x7f530c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f5303fff700 1 -- 172.21.15.60:0/3844297399 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556154645 0 0) 0x7f530c1598e0 con 0x7f530c0c9510 2023-07-13T22:27:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.487+0000 7f5303fff700 1 -- 172.21.15.60:0/3844297399 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f80057c0 con 0x7f530c0c9510 2023-07-13T22:27:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.488+0000 7f5311b9c700 1 -- 172.21.15.60:0/3844297399 >> v2:172.21.15.105:3300/0 conn(0x7f530c159510 msgr2=0x7f530c0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.488+0000 7f5311b9c700 1 --2- 172.21.15.60:0/3844297399 >> v2:172.21.15.105:3300/0 conn(0x7f530c159510 0x7f530c0c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.488+0000 7f5311b9c700 1 -- 172.21.15.60:0/3844297399 >> v1:172.21.15.60:6789/0 conn(0x7f530c0c9510 legacy=0x7f530c0703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.488+0000 7f5311b9c700 1 -- 172.21.15.60:0/3844297399 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c06d570 con 0x7f530c070af0 2023-07-13T22:27:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.488+0000 7f531139b700 1 --2- 172.21.15.60:0/3844297399 >> v2:172.21.15.105:3300/0 conn(0x7f530c159510 0x7f530c0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.488+0000 7f5311b9c700 1 --2- 172.21.15.60:0/3844297399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 0x7f530c0cc330 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f530c06c250 tx=0x7f530400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2661b0119556843a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.488+0000 7f5303fff700 1 -- 172.21.15.60:0/3844297399 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530400e070 con 0x7f530c070af0 2023-07-13T22:27:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.488+0000 7f5303fff700 1 -- 172.21.15.60:0/3844297399 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530400bb20 con 0x7f530c070af0 2023-07-13T22:27:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.488+0000 7f5303fff700 1 -- 172.21.15.60:0/3844297399 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304012430 con 0x7f530c070af0 2023-07-13T22:27:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.489+0000 7f53135ff700 1 -- 172.21.15.60:0/3844297399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 msgr2=0x7f530c0cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.489+0000 7f53135ff700 1 --2- 172.21.15.60:0/3844297399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 0x7f530c0cc330 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f530c06c250 tx=0x7f530400b330 comp rx=0 tx=0).stop 2023-07-13T22:27:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.489+0000 7f53135ff700 1 -- 172.21.15.60:0/3844297399 shutdown_connections 2023-07-13T22:27:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.489+0000 7f53135ff700 1 --2- 172.21.15.60:0/3844297399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 0x7f530c0cc330 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.489+0000 7f53135ff700 1 --2- 172.21.15.60:0/3844297399 >> v2:172.21.15.105:3300/0 conn(0x7f530c159510 0x7f530c0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:09.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.489+0000 7f53135ff700 1 -- 172.21.15.60:0/3844297399 >> 172.21.15.60:0/3844297399 conn(0x7f530c0cbe70 msgr2=0x7f530c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:09.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.489+0000 7f53135ff700 1 -- 172.21.15.60:0/3844297399 shutdown_connections 2023-07-13T22:27:09.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.489+0000 7f53135ff700 1 -- 172.21.15.60:0/3844297399 wait complete. 2023-07-13T22:27:09.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.491+0000 7f53135ff700 1 Processor -- start 2023-07-13T22:27:09.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.491+0000 7f53135ff700 1 -- start start 2023-07-13T22:27:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f53135ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 0x7f530c1f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f53135ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f530c0c9510 0x7f530c0ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f53135ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f530c159510 0x7f530c157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f53135ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c1f2bd0 con 0x7f530c070af0 2023-07-13T22:27:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f53135ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c1f2eb0 con 0x7f530c159510 2023-07-13T22:27:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f53135ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c1f3190 con 0x7f530c0c9510 2023-07-13T22:27:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f531139b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 0x7f530c1f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f531139b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 0x7f530c1f22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34036/0 (socket says 172.21.15.60:34036) 2023-07-13T22:27:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f531139b700 1 -- 172.21.15.60:0/3163811772 learned_addr learned my addr 172.21.15.60:0/3163811772 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f5311b9c700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f530c159510 0x7f530c157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f5310b9a700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f530c0c9510 0x7f530c0ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f531139b700 1 -- 172.21.15.60:0/3163811772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f530c0c9510 msgr2=0x7f530c0ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f531139b700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f530c0c9510 0x7f530c0ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f531139b700 1 -- 172.21.15.60:0/3163811772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f530c159510 msgr2=0x7f530c157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f531139b700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f530c159510 0x7f530c157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:09.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f531139b700 1 -- 172.21.15.60:0/3163811772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5304009a70 con 0x7f530c070af0 2023-07-13T22:27:09.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.493+0000 7f5310b9a700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f530c0c9510 0x7f530c0ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:09.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.494+0000 7f531139b700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 0x7f530c1f22e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f530800e810 tx=0x7f530800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.494+0000 7f5301ffb700 1 -- 172.21.15.60:0/3163811772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308005c20 con 0x7f530c070af0 2023-07-13T22:27:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.494+0000 7f5301ffb700 1 -- 172.21.15.60:0/3163811772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5308005d80 con 0x7f530c070af0 2023-07-13T22:27:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.494+0000 7f5301ffb700 1 -- 172.21.15.60:0/3163811772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308006270 con 0x7f530c070af0 2023-07-13T22:27:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.494+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c0ab990 con 0x7f530c070af0 2023-07-13T22:27:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.494+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c1f2820 con 0x7f530c070af0 2023-07-13T22:27:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.495+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530c0abeb0 con 0x7f530c070af0 2023-07-13T22:27:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.496+0000 7f5301ffb700 1 -- 172.21.15.60:0/3163811772 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53080063d0 con 0x7f530c070af0 2023-07-13T22:27:09.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.497+0000 7f5301ffb700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52f006ace0 0x7f52f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:09.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.498+0000 7f5301ffb700 1 -- 172.21.15.60:0/3163811772 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5308088e30 con 0x7f530c070af0 2023-07-13T22:27:09.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.498+0000 7f5310b9a700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52f006ace0 0x7f52f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:09.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.498+0000 7f5310b9a700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52f006ace0 0x7f52f006d190 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f530c158a00 tx=0x7f52fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.502+0000 7f5301ffb700 1 -- 172.21.15.60:0/3163811772 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f530c0abeb0 con 0x7f530c070af0 2023-07-13T22:27:09.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.753+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f530c04e280 con 0x7f530c070af0 2023-07-13T22:27:09.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.754+0000 7f5301ffb700 1 -- 172.21.15.60:0/3163811772 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f53080200c0 con 0x7f530c070af0 2023-07-13T22:27:09.734 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:09.734 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:27:09.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52f006ace0 msgr2=0x7f52f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:09.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52f006ace0 0x7f52f006d190 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f530c158a00 tx=0x7f52fc009040 comp rx=0 tx=0).stop 2023-07-13T22:27:09.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 msgr2=0x7f530c1f22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:09.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 0x7f530c1f22e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f530800e810 tx=0x7f530800eb20 comp rx=0 tx=0).stop 2023-07-13T22:27:09.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 shutdown_connections 2023-07-13T22:27:09.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52f006ace0 0x7f52f006d190 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:09.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f530c070af0 0x7f530c1f22e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:09.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f530c159510 0x7f530c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:09.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 --2- 172.21.15.60:0/3163811772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f530c0c9510 0x7f530c0ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:09.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 >> 172.21.15.60:0/3163811772 conn(0x7f530c0cbe70 msgr2=0x7f530c1529a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:09.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.756+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 shutdown_connections 2023-07-13T22:27:09.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:09.757+0000 7f53135ff700 1 -- 172.21.15.60:0/3163811772 wait complete. 2023-07-13T22:27:09.754 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:27:09.754 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:27:14.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:27:14.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.989+0000 7fb828fff700 1 Processor -- start 2023-07-13T22:27:14.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.989+0000 7fb828fff700 1 -- start start 2023-07-13T22:27:14.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.990+0000 7fb828fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb824159510 0x7fb8240c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:14.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.990+0000 7fb828fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb824070af0 0x7fb8240cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb828fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8241598e0 con 0x7fb8240c9510 2023-07-13T22:27:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb828fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8240ca8d0 con 0x7fb824070af0 2023-07-13T22:27:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb828fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82406d290 con 0x7fb824159510 2023-07-13T22:27:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb822d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb824070af0 0x7fb8240cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb822d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb824070af0 0x7fb8240cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44788/0 (socket says 172.21.15.60:44788) 2023-07-13T22:27:14.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb822d9d700 1 -- 172.21.15.60:0/334217177 learned_addr learned my addr 172.21.15.60:0/334217177 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:14.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb82259c700 1 --2- 172.21.15.60:0/334217177 >> v2:172.21.15.105:3300/0 conn(0x7fb824159510 0x7fb8240c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:14.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb822d9d700 1 -- 172.21.15.60:0/334217177 >> v2:172.21.15.105:3300/0 conn(0x7fb824159510 msgr2=0x7fb8240c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:14.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb822d9d700 1 --2- 172.21.15.60:0/334217177 >> v2:172.21.15.105:3300/0 conn(0x7fb824159510 0x7fb8240c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:14.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb822d9d700 1 -- 172.21.15.60:0/334217177 >> v1:172.21.15.105:6789/0 conn(0x7fb8240c9510 legacy=0x7fb8240703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:14.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb822d9d700 1 -- 172.21.15.60:0/334217177 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82406d570 con 0x7fb824070af0 2023-07-13T22:27:14.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.991+0000 7fb822d9d700 1 --2- 172.21.15.60:0/334217177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb824070af0 0x7fb8240cc330 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb82406c250 tx=0x7fb81c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fec2939335cc9ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:14.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb82159a700 1 -- 172.21.15.60:0/334217177 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00bb10 con 0x7fb824070af0 2023-07-13T22:27:14.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb82159a700 1 -- 172.21.15.60:0/334217177 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c00bc70 con 0x7fb824070af0 2023-07-13T22:27:14.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb82159a700 1 -- 172.21.15.60:0/334217177 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c011460 con 0x7fb824070af0 2023-07-13T22:27:14.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb828fff700 1 -- 172.21.15.60:0/334217177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb824070af0 msgr2=0x7fb8240cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:14.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb828fff700 1 --2- 172.21.15.60:0/334217177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb824070af0 0x7fb8240cc330 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb82406c250 tx=0x7fb81c00b330 comp rx=0 tx=0).stop 2023-07-13T22:27:14.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb828fff700 1 -- 172.21.15.60:0/334217177 shutdown_connections 2023-07-13T22:27:14.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb828fff700 1 --2- 172.21.15.60:0/334217177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb824070af0 0x7fb8240cc330 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:14.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb828fff700 1 --2- 172.21.15.60:0/334217177 >> v2:172.21.15.105:3300/0 conn(0x7fb824159510 0x7fb8240c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:14.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb828fff700 1 -- 172.21.15.60:0/334217177 >> 172.21.15.60:0/334217177 conn(0x7fb8240cbe70 msgr2=0x7fb8240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:14.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.992+0000 7fb828fff700 1 -- 172.21.15.60:0/334217177 shutdown_connections 2023-07-13T22:27:14.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.993+0000 7fb828fff700 1 -- 172.21.15.60:0/334217177 wait complete. 2023-07-13T22:27:14.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.995+0000 7fb828fff700 1 Processor -- start 2023-07-13T22:27:14.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.995+0000 7fb828fff700 1 -- start start 2023-07-13T22:27:14.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb828fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb824070af0 0x7fb824150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:14.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb828fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8240c9510 0x7fb8240ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:14.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb828fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824159510 0x7fb8241e2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:14.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb828fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240ad270 con 0x7fb8240c9510 2023-07-13T22:27:14.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb828fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240ac1d0 con 0x7fb824159510 2023-07-13T22:27:14.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb828fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240ac4b0 con 0x7fb824070af0 2023-07-13T22:27:14.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb821d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8240c9510 0x7fb8240ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:14.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb821d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8240c9510 0x7fb8240ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44804/0 (socket says 172.21.15.60:44804) 2023-07-13T22:27:14.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb821d9b700 1 -- 172.21.15.60:0/3775945437 learned_addr learned my addr 172.21.15.60:0/3775945437 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:14.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb822d9d700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824159510 0x7fb8241e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:14.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb82259c700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb824070af0 0x7fb824150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:14.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb821d9b700 1 -- 172.21.15.60:0/3775945437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb824070af0 msgr2=0x7fb824150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:14.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.996+0000 7fb821d9b700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb824070af0 0x7fb824150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:14.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.997+0000 7fb821d9b700 1 -- 172.21.15.60:0/3775945437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824159510 msgr2=0x7fb8241e2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:14.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.997+0000 7fb821d9b700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824159510 0x7fb8241e2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:14.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.997+0000 7fb821d9b700 1 -- 172.21.15.60:0/3775945437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c009b30 con 0x7fb8240c9510 2023-07-13T22:27:14.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.997+0000 7fb821d9b700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8240c9510 0x7fb8240ab8f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c4e0 tx=0x7fb81800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:14.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.997+0000 7fb812ffd700 1 -- 172.21.15.60:0/3775945437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180052d0 con 0x7fb8240c9510 2023-07-13T22:27:14.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.997+0000 7fb812ffd700 1 -- 172.21.15.60:0/3775945437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818005430 con 0x7fb8240c9510 2023-07-13T22:27:14.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.997+0000 7fb812ffd700 1 -- 172.21.15.60:0/3775945437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180038c0 con 0x7fb8240c9510 2023-07-13T22:27:14.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.997+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8240ac790 con 0x7fb8240c9510 2023-07-13T22:27:14.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.997+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8240acd50 con 0x7fb8240c9510 2023-07-13T22:27:14.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.998+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82404e280 con 0x7fb8240c9510 2023-07-13T22:27:14.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:14.999+0000 7fb812ffd700 1 -- 172.21.15.60:0/3775945437 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb818003a20 con 0x7fb8240c9510 2023-07-13T22:27:14.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.000+0000 7fb812ffd700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80406ace0 0x7fb80406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:14.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.000+0000 7fb812ffd700 1 -- 172.21.15.60:0/3775945437 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb8180888c0 con 0x7fb8240c9510 2023-07-13T22:27:14.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.001+0000 7fb82259c700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80406ace0 0x7fb80406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:14.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.002+0000 7fb82259c700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80406ace0 0x7fb80406d190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb814009b20 tx=0x7fb8140091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.005+0000 7fb812ffd700 1 -- 172.21.15.60:0/3775945437 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb8180538d0 con 0x7fb8240c9510 2023-07-13T22:27:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.254+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb8240cf3c0 con 0x7fb8240c9510 2023-07-13T22:27:15.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.255+0000 7fb812ffd700 1 -- 172.21.15.60:0/3775945437 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fb818056ef0 con 0x7fb8240c9510 2023-07-13T22:27:15.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:15.235 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:27:15.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80406ace0 msgr2=0x7fb80406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:15.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80406ace0 0x7fb80406d190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb814009b20 tx=0x7fb8140091c0 comp rx=0 tx=0).stop 2023-07-13T22:27:15.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8240c9510 msgr2=0x7fb8240ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8240c9510 0x7fb8240ab8f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c4e0 tx=0x7fb81800c7f0 comp rx=0 tx=0).stop 2023-07-13T22:27:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 shutdown_connections 2023-07-13T22:27:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb80406ace0 0x7fb80406d190 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824159510 0x7fb8241e2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb824070af0 0x7fb824150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 --2- 172.21.15.60:0/3775945437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8240c9510 0x7fb8240ab8f0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 >> 172.21.15.60:0/3775945437 conn(0x7fb8240cbe70 msgr2=0x7fb824153580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:15.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 shutdown_connections 2023-07-13T22:27:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:15.257+0000 7fb828fff700 1 -- 172.21.15.60:0/3775945437 wait complete. 2023-07-13T22:27:15.255 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:27:15.255 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:27:20.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:27:20.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.490+0000 7f98687ff700 1 Processor -- start 2023-07-13T22:27:20.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.490+0000 7f98687ff700 1 -- start start 2023-07-13T22:27:20.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.491+0000 7f98687ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98600d0a50 0x7f98600cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:20.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.491+0000 7f98687ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 0x7f98600d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:20.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.491+0000 7f98687ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98600d0e20 0x7f98600d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:20.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.491+0000 7f98687ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986015c900 con 0x7f98600d0e20 2023-07-13T22:27:20.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.491+0000 7f98687ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98600d2410 con 0x7f98600d0a50 2023-07-13T22:27:20.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.491+0000 7f98687ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98600d2660 con 0x7f98600cd260 2023-07-13T22:27:20.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9866d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98600d0e20 0x7f98600d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:20.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9866d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98600d0e20 0x7f98600d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44818/0 (socket says 172.21.15.60:44818) 2023-07-13T22:27:20.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9866d9c700 1 -- 172.21.15.60:0/336255430 learned_addr learned my addr 172.21.15.60:0/336255430 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:20.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f986659b700 1 --2- 172.21.15.60:0/336255430 >> v2:172.21.15.105:3300/0 conn(0x7f98600d0a50 0x7f98600cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:20.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9865d9a700 1 --2- 172.21.15.60:0/336255430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 0x7f98600d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:20.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9866d9c700 1 -- 172.21.15.60:0/336255430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 msgr2=0x7f98600d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:20.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9866d9c700 1 --2- 172.21.15.60:0/336255430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 0x7f98600d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9866d9c700 1 -- 172.21.15.60:0/336255430 >> v2:172.21.15.105:3300/0 conn(0x7f98600d0a50 msgr2=0x7f98600cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:20.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9866d9c700 1 --2- 172.21.15.60:0/336255430 >> v2:172.21.15.105:3300/0 conn(0x7f98600d0a50 0x7f98600cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9866d9c700 1 -- 172.21.15.60:0/336255430 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600d2940 con 0x7f98600d0e20 2023-07-13T22:27:20.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f986659b700 1 --2- 172.21.15.60:0/336255430 >> v2:172.21.15.105:3300/0 conn(0x7f98600d0a50 0x7f98600cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:20.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9865d9a700 1 --2- 172.21.15.60:0/336255430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 0x7f98600d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:20.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.492+0000 7f9866d9c700 1 --2- 172.21.15.60:0/336255430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 0x7f98600d1280 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f986008be90 tx=0x7f985800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9b2d030ce4e10b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:20.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f9865599700 1 -- 172.21.15.60:0/336255430 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985800e070 con 0x7f98600d0e20 2023-07-13T22:27:20.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f9865599700 1 -- 172.21.15.60:0/336255430 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985800b9c0 con 0x7f98600d0e20 2023-07-13T22:27:20.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f9865599700 1 -- 172.21.15.60:0/336255430 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985800bd70 con 0x7f98600d0e20 2023-07-13T22:27:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f98687ff700 1 -- 172.21.15.60:0/336255430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 msgr2=0x7f98600d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f98687ff700 1 --2- 172.21.15.60:0/336255430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 0x7f98600d1280 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f986008be90 tx=0x7f985800b330 comp rx=0 tx=0).stop 2023-07-13T22:27:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f98687ff700 1 -- 172.21.15.60:0/336255430 shutdown_connections 2023-07-13T22:27:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f98687ff700 1 --2- 172.21.15.60:0/336255430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 0x7f98600d1280 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f98687ff700 1 --2- 172.21.15.60:0/336255430 >> v2:172.21.15.105:3300/0 conn(0x7f98600d0a50 0x7f98600cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f98687ff700 1 --2- 172.21.15.60:0/336255430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 0x7f98600d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f98687ff700 1 -- 172.21.15.60:0/336255430 >> 172.21.15.60:0/336255430 conn(0x7f9860071600 msgr2=0x7f98600719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.493+0000 7f98687ff700 1 -- 172.21.15.60:0/336255430 shutdown_connections 2023-07-13T22:27:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.494+0000 7f98687ff700 1 -- 172.21.15.60:0/336255430 wait complete. 2023-07-13T22:27:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.495+0000 7f98687ff700 1 Processor -- start 2023-07-13T22:27:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.495+0000 7f98687ff700 1 -- start start 2023-07-13T22:27:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.496+0000 7f98687ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 0x7f986014c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.496+0000 7f98687ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98600d0a50 0x7f98600b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.496+0000 7f98687ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 0x7f98601f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.496+0000 7f98687ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986015cd40 con 0x7f98600d0e20 2023-07-13T22:27:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.496+0000 7f98687ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98601f5830 con 0x7f98600d0a50 2023-07-13T22:27:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.496+0000 7f98687ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98601f5b10 con 0x7f98600cd260 2023-07-13T22:27:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9866d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 0x7f98601f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9866d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 0x7f98601f4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44834/0 (socket says 172.21.15.60:44834) 2023-07-13T22:27:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9866d9c700 1 -- 172.21.15.60:0/3655253941 learned_addr learned my addr 172.21.15.60:0/3655253941 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f986659b700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 0x7f986014c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9865d9a700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98600d0a50 0x7f98600b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9866d9c700 1 -- 172.21.15.60:0/3655253941 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 msgr2=0x7f986014c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:20.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9866d9c700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 0x7f986014c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9866d9c700 1 -- 172.21.15.60:0/3655253941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98600d0a50 msgr2=0x7f98600b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:20.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9866d9c700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98600d0a50 0x7f98600b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9866d9c700 1 -- 172.21.15.60:0/3655253941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858009b30 con 0x7f98600d0e20 2023-07-13T22:27:20.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9866d9c700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 0x7f98601f4cc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f985800b370 tx=0x7f9858004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:20.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9856ffd700 1 -- 172.21.15.60:0/3655253941 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858004310 con 0x7f98600d0e20 2023-07-13T22:27:20.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9856ffd700 1 -- 172.21.15.60:0/3655253941 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9858004470 con 0x7f98600d0e20 2023-07-13T22:27:20.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.497+0000 7f9856ffd700 1 -- 172.21.15.60:0/3655253941 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858004820 con 0x7f98600d0e20 2023-07-13T22:27:20.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.498+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986015e670 con 0x7f98600d0e20 2023-07-13T22:27:20.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.498+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986015e370 con 0x7f98600d0e20 2023-07-13T22:27:20.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.499+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986004e280 con 0x7f98600d0e20 2023-07-13T22:27:20.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.500+0000 7f9856ffd700 1 -- 172.21.15.60:0/3655253941 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f985801c440 con 0x7f98600d0e20 2023-07-13T22:27:20.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.501+0000 7f9856ffd700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984406ad10 0x7f984406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:20.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.501+0000 7f9856ffd700 1 -- 172.21.15.60:0/3655253941 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f985808aac0 con 0x7f98600d0e20 2023-07-13T22:27:20.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.501+0000 7f986659b700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984406ad10 0x7f984406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:20.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.501+0000 7f986659b700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984406ad10 0x7f984406d1c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f985c006fd0 tx=0x7f985c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:20.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.505+0000 7f9856ffd700 1 -- 172.21.15.60:0/3655253941 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9858055b60 con 0x7f98600d0e20 2023-07-13T22:27:20.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.759+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f986015ba30 con 0x7f98600d0e20 2023-07-13T22:27:20.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.759+0000 7f9856ffd700 1 -- 172.21.15.60:0/3655253941 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f98580200c0 con 0x7f98600d0e20 2023-07-13T22:27:20.739 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:20.739 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:27:20.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.761+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984406ad10 msgr2=0x7f984406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:20.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.761+0000 7f98687ff700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984406ad10 0x7f984406d1c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f985c006fd0 tx=0x7f985c008040 comp rx=0 tx=0).stop 2023-07-13T22:27:20.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.761+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 msgr2=0x7f98601f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:20.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.761+0000 7f98687ff700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 0x7f98601f4cc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f985800b370 tx=0x7f9858004ad0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.761+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 shutdown_connections 2023-07-13T22:27:20.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.761+0000 7f98687ff700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f984406ad10 0x7f984406d1c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.761+0000 7f98687ff700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98600d0e20 0x7f98601f4cc0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.761+0000 7f98687ff700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98600cd260 0x7f986014c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.761+0000 7f98687ff700 1 --2- 172.21.15.60:0/3655253941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98600d0a50 0x7f98600b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:20.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.762+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 >> 172.21.15.60:0/3655253941 conn(0x7f9860071600 msgr2=0x7f986015bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:20.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.762+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 shutdown_connections 2023-07-13T22:27:20.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:20.762+0000 7f98687ff700 1 -- 172.21.15.60:0/3655253941 wait complete. 2023-07-13T22:27:20.759 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:27:20.759 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:27:25.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:27:25.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.994+0000 7f3454dff700 1 Processor -- start 2023-07-13T22:27:25.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.994+0000 7f3454dff700 1 -- start start 2023-07-13T22:27:25.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.995+0000 7f3454dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500cdb20 0x7f34500ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:25.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.995+0000 7f3454dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34500b2bc0 0x7f34501663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:25.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.995+0000 7f3454dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345015fe20 con 0x7f34500d0320 2023-07-13T22:27:25.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.995+0000 7f3454dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34500cdef0 con 0x7f34500b2bc0 2023-07-13T22:27:25.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.995+0000 7f3454dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34500ceb40 con 0x7f34500cdb20 2023-07-13T22:27:25.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.996+0000 7f344ed9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34500b2bc0 0x7f34501663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:25.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.996+0000 7f344e59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500cdb20 0x7f34500ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:25.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.996+0000 7f344ed9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34500b2bc0 0x7f34501663c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:47652/0 (socket says 172.21.15.60:47652) 2023-07-13T22:27:25.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.996+0000 7f344ed9d700 1 -- 172.21.15.60:0/1445197378 learned_addr learned my addr 172.21.15.60:0/1445197378 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:25.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.996+0000 7f344e59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500cdb20 0x7f34500ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:54330/0 (socket says 172.21.15.60:54330) 2023-07-13T22:27:25.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.996+0000 7f344ed9d700 1 -- 172.21.15.60:0/1445197378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500cdb20 msgr2=0x7f34500ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:25.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.996+0000 7f344ed9d700 1 --2- 172.21.15.60:0/1445197378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500cdb20 0x7f34500ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:25.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.996+0000 7f344ed9d700 1 -- 172.21.15.60:0/1445197378 >> v1:172.21.15.105:6789/0 conn(0x7f34500d0320 legacy=0x7f34500b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:25.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.996+0000 7f344ed9d700 1 -- 172.21.15.60:0/1445197378 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345015cb50 con 0x7f34500b2bc0 2023-07-13T22:27:25.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.997+0000 7f344ed9d700 1 --2- 172.21.15.60:0/1445197378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500b2bc0 0x7f34501663c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3448009e50 tx=0x7f344800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c1ff0bc996dc324 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:25.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.997+0000 7f344d59a700 1 -- 172.21.15.60:0/1445197378 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800e070 con 0x7f34500b2bc0 2023-07-13T22:27:25.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.997+0000 7f344d59a700 1 -- 172.21.15.60:0/1445197378 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344800b910 con 0x7f34500b2bc0 2023-07-13T22:27:25.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.997+0000 7f344d59a700 1 -- 172.21.15.60:0/1445197378 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800bcc0 con 0x7f34500b2bc0 2023-07-13T22:27:25.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.998+0000 7f3454dff700 1 -- 172.21.15.60:0/1445197378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500b2bc0 msgr2=0x7f34501663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:25.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.998+0000 7f3454dff700 1 --2- 172.21.15.60:0/1445197378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500b2bc0 0x7f34501663c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3448009e50 tx=0x7f344800b330 comp rx=0 tx=0).stop 2023-07-13T22:27:25.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.998+0000 7f3454dff700 1 -- 172.21.15.60:0/1445197378 shutdown_connections 2023-07-13T22:27:25.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.998+0000 7f3454dff700 1 --2- 172.21.15.60:0/1445197378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500b2bc0 0x7f34501663c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:25.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.998+0000 7f3454dff700 1 --2- 172.21.15.60:0/1445197378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500cdb20 0x7f34500ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:25.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.998+0000 7f3454dff700 1 -- 172.21.15.60:0/1445197378 >> 172.21.15.60:0/1445197378 conn(0x7f345006e660 msgr2=0x7f345006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:25.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.998+0000 7f3454dff700 1 -- 172.21.15.60:0/1445197378 shutdown_connections 2023-07-13T22:27:25.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:25.998+0000 7f3454dff700 1 -- 172.21.15.60:0/1445197378 wait complete. 2023-07-13T22:27:25.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.000+0000 7f3454dff700 1 Processor -- start 2023-07-13T22:27:25.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.001+0000 7f3454dff700 1 -- start start 2023-07-13T22:27:25.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f3454dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500b2bc0 0x7f34500b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:25.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f3454dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500cdb20 0x7f34501f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:25.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f3454dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34500d0320 0x7f3450158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:25.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f3454dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3450155550 con 0x7f34500d0320 2023-07-13T22:27:25.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f3454dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34501545f0 con 0x7f34500cdb20 2023-07-13T22:27:25.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f3454dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34501548d0 con 0x7f34500b2bc0 2023-07-13T22:27:25.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f344ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34500d0320 0x7f3450158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:25.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f344ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34500d0320 0x7f3450158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42316/0 (socket says 172.21.15.60:42316) 2023-07-13T22:27:25.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f344ed9d700 1 -- 172.21.15.60:0/2373028259 learned_addr learned my addr 172.21.15.60:0/2373028259 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:25.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f344dd9b700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500cdb20 0x7f34501f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:25.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.002+0000 7f344e59c700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500b2bc0 0x7f34500b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:25.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.003+0000 7f344ed9d700 1 -- 172.21.15.60:0/2373028259 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500b2bc0 msgr2=0x7f34500b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:25.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.003+0000 7f344ed9d700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500b2bc0 0x7f34500b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:25.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.003+0000 7f344ed9d700 1 -- 172.21.15.60:0/2373028259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500cdb20 msgr2=0x7f34501f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:25.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.003+0000 7f344ed9d700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500cdb20 0x7f34501f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:25.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.003+0000 7f344ed9d700 1 -- 172.21.15.60:0/2373028259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448009b30 con 0x7f34500d0320 2023-07-13T22:27:25.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.003+0000 7f344dd9b700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500cdb20 0x7f34501f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:25.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.003+0000 7f344ed9d700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34500d0320 0x7f3450158130 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3448012000 tx=0x7f3448004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:25.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.003+0000 7f343effd700 1 -- 172.21.15.60:0/2373028259 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448004800 con 0x7f34500d0320 2023-07-13T22:27:25.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.004+0000 7f343effd700 1 -- 172.21.15.60:0/2373028259 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3448004960 con 0x7f34500d0320 2023-07-13T22:27:25.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.004+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3450154bb0 con 0x7f34500d0320 2023-07-13T22:27:25.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.004+0000 7f343effd700 1 -- 172.21.15.60:0/2373028259 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344801b670 con 0x7f34500d0320 2023-07-13T22:27:25.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.004+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3450155170 con 0x7f34500d0320 2023-07-13T22:27:25.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.005+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345004e280 con 0x7f34500d0320 2023-07-13T22:27:25.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.006+0000 7f343effd700 1 -- 172.21.15.60:0/2373028259 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f344801b7d0 con 0x7f34500d0320 2023-07-13T22:27:25.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.007+0000 7f343effd700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f343006ace0 0x7f343006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:25.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.007+0000 7f343effd700 1 -- 172.21.15.60:0/2373028259 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f344808a980 con 0x7f34500d0320 2023-07-13T22:27:25.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.007+0000 7f344e59c700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f343006ace0 0x7f343006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:25.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.008+0000 7f344e59c700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f343006ace0 0x7f343006d190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3440009be0 tx=0x7f3440009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.012+0000 7f343effd700 1 -- 172.21.15.60:0/2373028259 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3448055a20 con 0x7f34500d0320 2023-07-13T22:27:26.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.262+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f345006c650 con 0x7f34500d0320 2023-07-13T22:27:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.263+0000 7f343effd700 1 -- 172.21.15.60:0/2373028259 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f3448059040 con 0x7f34500d0320 2023-07-13T22:27:26.243 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:26.243 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:27:26.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f343006ace0 msgr2=0x7f343006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:26.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f343006ace0 0x7f343006d190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3440009be0 tx=0x7f3440009210 comp rx=0 tx=0).stop 2023-07-13T22:27:26.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34500d0320 msgr2=0x7f3450158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:26.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34500d0320 0x7f3450158130 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3448012000 tx=0x7f3448004ad0 comp rx=0 tx=0).stop 2023-07-13T22:27:26.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 shutdown_connections 2023-07-13T22:27:26.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f343006ace0 0x7f343006d190 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:26.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34500d0320 0x7f3450158130 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:26.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34500b2bc0 0x7f34500b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:26.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 --2- 172.21.15.60:0/2373028259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34500cdb20 0x7f34501f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:26.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 >> 172.21.15.60:0/2373028259 conn(0x7f345006e660 msgr2=0x7f3450145740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:26.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 shutdown_connections 2023-07-13T22:27:26.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:26.265+0000 7f3454dff700 1 -- 172.21.15.60:0/2373028259 wait complete. 2023-07-13T22:27:26.263 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:27:26.263 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:27:26.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:27:26.711 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:27:31.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:27:31.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.498+0000 7efda89ff700 1 Processor -- start 2023-07-13T22:27:31.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.498+0000 7efda89ff700 1 -- start start 2023-07-13T22:27:31.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.499+0000 7efda89ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efda40cdb20 0x7efda40ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:31.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.499+0000 7efda89ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda415fe20 con 0x7efda40b2bc0 2023-07-13T22:27:31.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.499+0000 7efda89ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda415cb50 con 0x7efda40d0320 2023-07-13T22:27:31.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.499+0000 7efda89ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda40cdef0 con 0x7efda40cdb20 2023-07-13T22:27:31.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.500+0000 7efda259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efda40cdb20 0x7efda40ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:31.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.500+0000 7efda259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efda40cdb20 0x7efda40ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42318/0 (socket says 172.21.15.60:42318) 2023-07-13T22:27:31.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.500+0000 7efda2d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efda40b2bc0 0x7efda41663c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58168/0 (socket says 172.21.15.60:58168) 2023-07-13T22:27:31.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.500+0000 7efda259c700 1 -- 172.21.15.60:0/3655404974 learned_addr learned my addr 172.21.15.60:0/3655404974 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:31.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.500+0000 7efda259c700 1 -- 172.21.15.60:0/3655404974 >> v1:172.21.15.105:6789/0 conn(0x7efda40d0320 legacy=0x7efda40b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:31.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.500+0000 7efda259c700 1 -- 172.21.15.60:0/3655404974 >> v1:172.21.15.60:6789/0 conn(0x7efda40b2bc0 legacy=0x7efda41663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:31.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.500+0000 7efda259c700 1 -- 172.21.15.60:0/3655404974 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda415cdc0 con 0x7efda40cdb20 2023-07-13T22:27:31.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.500+0000 7efda159a700 1 -- 172.21.15.60:0/3655404974 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982543672 0 0) 0x7efd9c001490 con 0x7efda40b2bc0 2023-07-13T22:27:31.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.500+0000 7efda259c700 1 --2- 172.21.15.60:0/3655404974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40cdb20 0x7efda40ce600 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7efd940098f0 tx=0x7efd94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6909601e28e6951 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:31.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda159a700 1 -- 172.21.15.60:0/3655404974 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94004230 con 0x7efda40cdb20 2023-07-13T22:27:31.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda159a700 1 -- 172.21.15.60:0/3655404974 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd94004390 con 0x7efda40cdb20 2023-07-13T22:27:31.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda159a700 1 -- 172.21.15.60:0/3655404974 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd940036a0 con 0x7efda40cdb20 2023-07-13T22:27:31.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda89ff700 1 -- 172.21.15.60:0/3655404974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40cdb20 msgr2=0x7efda40ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:31.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda89ff700 1 --2- 172.21.15.60:0/3655404974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40cdb20 0x7efda40ce600 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7efd940098f0 tx=0x7efd94009c00 comp rx=0 tx=0).stop 2023-07-13T22:27:31.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda89ff700 1 -- 172.21.15.60:0/3655404974 shutdown_connections 2023-07-13T22:27:31.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda89ff700 1 --2- 172.21.15.60:0/3655404974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40cdb20 0x7efda40ce600 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:31.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda89ff700 1 -- 172.21.15.60:0/3655404974 >> 172.21.15.60:0/3655404974 conn(0x7efda406e660 msgr2=0x7efda406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:31.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda89ff700 1 -- 172.21.15.60:0/3655404974 shutdown_connections 2023-07-13T22:27:31.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.501+0000 7efda89ff700 1 -- 172.21.15.60:0/3655404974 wait complete. 2023-07-13T22:27:31.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.503+0000 7efda89ff700 1 Processor -- start 2023-07-13T22:27:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.503+0000 7efda89ff700 1 -- start start 2023-07-13T22:27:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.504+0000 7efda89ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efda40b2bc0 0x7efda40c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.504+0000 7efda89ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda40cdb20 0x7efda41f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.504+0000 7efda89ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40d0320 0x7efda41f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.504+0000 7efda89ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40c68b0 con 0x7efda40d0320 2023-07-13T22:27:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.504+0000 7efda89ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40ad450 con 0x7efda40b2bc0 2023-07-13T22:27:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.504+0000 7efda89ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda40ad6f0 con 0x7efda40cdb20 2023-07-13T22:27:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.504+0000 7efda2d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40d0320 0x7efda41f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.504+0000 7efda2d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40d0320 0x7efda41f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42320/0 (socket says 172.21.15.60:42320) 2023-07-13T22:27:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.504+0000 7efda2d9d700 1 -- 172.21.15.60:0/1762952036 learned_addr learned my addr 172.21.15.60:0/1762952036 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efda259c700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efda40b2bc0 0x7efda40c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efda1d9b700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda40cdb20 0x7efda41f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efda2d9d700 1 -- 172.21.15.60:0/1762952036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda40cdb20 msgr2=0x7efda41f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:31.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efda2d9d700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda40cdb20 0x7efda41f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:31.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efda2d9d700 1 -- 172.21.15.60:0/1762952036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efda40b2bc0 msgr2=0x7efda40c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efda2d9d700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efda40b2bc0 0x7efda40c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efda2d9d700 1 -- 172.21.15.60:0/1762952036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd94009550 con 0x7efda40d0320 2023-07-13T22:27:31.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efda2d9d700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40d0320 0x7efda41f5f00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00cb60 tx=0x7efd9c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:31.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efd92ffd700 1 -- 172.21.15.60:0/1762952036 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00a9e0 con 0x7efda40d0320 2023-07-13T22:27:31.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efd92ffd700 1 -- 172.21.15.60:0/1762952036 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c00ab40 con 0x7efda40d0320 2023-07-13T22:27:31.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efd92ffd700 1 -- 172.21.15.60:0/1762952036 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0070d0 con 0x7efda40d0320 2023-07-13T22:27:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.505+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda4156900 con 0x7efda40d0320 2023-07-13T22:27:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.506+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda40abcd0 con 0x7efda40d0320 2023-07-13T22:27:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.507+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda40d0780 con 0x7efda40d0320 2023-07-13T22:27:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.508+0000 7efd92ffd700 1 -- 172.21.15.60:0/1762952036 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efd9c007230 con 0x7efda40d0320 2023-07-13T22:27:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.508+0000 7efd92ffd700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efd8406ace0 0x7efd8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.509+0000 7efd92ffd700 1 -- 172.21.15.60:0/1762952036 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efd9c088d60 con 0x7efda40d0320 2023-07-13T22:27:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.509+0000 7efda259c700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efd8406ace0 0x7efd8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.509+0000 7efda259c700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efd8406ace0 0x7efd8406d190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd94009130 tx=0x7efd94008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.513+0000 7efd92ffd700 1 -- 172.21.15.60:0/1762952036 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efda40d0780 con 0x7efda40d0320 2023-07-13T22:27:31.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.762+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efda415fe20 con 0x7efda40d0320 2023-07-13T22:27:31.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.763+0000 7efd92ffd700 1 -- 172.21.15.60:0/1762952036 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7efda415fe20 con 0x7efda40d0320 2023-07-13T22:27:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:31.743 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:27:31.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efd8406ace0 msgr2=0x7efd8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:31.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efd8406ace0 0x7efd8406d190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd94009130 tx=0x7efd94008030 comp rx=0 tx=0).stop 2023-07-13T22:27:31.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40d0320 msgr2=0x7efda41f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:31.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40d0320 0x7efda41f5f00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00cb60 tx=0x7efd9c00ce70 comp rx=0 tx=0).stop 2023-07-13T22:27:31.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 shutdown_connections 2023-07-13T22:27:31.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efd8406ace0 0x7efd8406d190 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:31.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda40d0320 0x7efda41f5f00 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:31.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efda40b2bc0 0x7efda40c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:31.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 --2- 172.21.15.60:0/1762952036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda40cdb20 0x7efda41f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:31.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 >> 172.21.15.60:0/1762952036 conn(0x7efda406e660 msgr2=0x7efda40b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:31.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 shutdown_connections 2023-07-13T22:27:31.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:31.765+0000 7efda89ff700 1 -- 172.21.15.60:0/1762952036 wait complete. 2023-07-13T22:27:31.763 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:27:31.763 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:27:36.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:27:36.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.997+0000 7fd65f9ff700 1 Processor -- start 2023-07-13T22:27:36.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.997+0000 7fd65f9ff700 1 -- start start 2023-07-13T22:27:36.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.998+0000 7fd65f9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658159510 0x7fd6580c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:36.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.998+0000 7fd65f9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6581598e0 con 0x7fd6580c9510 2023-07-13T22:27:36.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.998+0000 7fd65f9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6580ca8d0 con 0x7fd658070af0 2023-07-13T22:27:36.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.998+0000 7fd65f9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65806d290 con 0x7fd658159510 2023-07-13T22:27:36.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd65cf9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd6580c9510 0x7fd6580703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58546/0 (socket says 172.21.15.60:58546) 2023-07-13T22:27:36.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd65cf9a700 1 -- 172.21.15.60:0/846648745 learned_addr learned my addr 172.21.15.60:0/846648745 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:36.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd65d79b700 1 --2- 172.21.15.60:0/846648745 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658159510 0x7fd6580c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:36.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd64ffff700 1 -- 172.21.15.60:0/846648745 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809085860 0 0) 0x7fd6581598e0 con 0x7fd6580c9510 2023-07-13T22:27:36.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd64ffff700 1 -- 172.21.15.60:0/846648745 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6440035b0 con 0x7fd6580c9510 2023-07-13T22:27:36.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd64ffff700 1 -- 172.21.15.60:0/846648745 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758751791 0 0) 0x7fd6580ca8d0 con 0x7fd658070af0 2023-07-13T22:27:36.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd64ffff700 1 -- 172.21.15.60:0/846648745 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6581598e0 con 0x7fd658070af0 2023-07-13T22:27:36.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd65d79b700 1 -- 172.21.15.60:0/846648745 >> v1:172.21.15.105:6789/0 conn(0x7fd658070af0 legacy=0x7fd6580cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:36.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd64ffff700 1 -- 172.21.15.60:0/846648745 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2142446985 0 0) 0x7fd6440035b0 con 0x7fd6580c9510 2023-07-13T22:27:36.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd65d79b700 1 -- 172.21.15.60:0/846648745 >> v1:172.21.15.60:6789/0 conn(0x7fd6580c9510 legacy=0x7fd6580703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:36.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:36.999+0000 7fd65d79b700 1 -- 172.21.15.60:0/846648745 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65806d570 con 0x7fd658159510 2023-07-13T22:27:36.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.000+0000 7fd65d79b700 1 --2- 172.21.15.60:0/846648745 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658159510 0x7fd6580c8fd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd654009850 tx=0x7fd654009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5768eeea76986f8b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:36.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.000+0000 7fd64ffff700 1 -- 172.21.15.60:0/846648745 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648001670 con 0x7fd658159510 2023-07-13T22:27:36.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.000+0000 7fd64ffff700 1 -- 172.21.15.60:0/846648745 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654005c30 con 0x7fd658159510 2023-07-13T22:27:36.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.000+0000 7fd64ffff700 1 -- 172.21.15.60:0/846648745 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654004030 con 0x7fd658159510 2023-07-13T22:27:36.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.001+0000 7fd65f9ff700 1 -- 172.21.15.60:0/846648745 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658159510 msgr2=0x7fd6580c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:36.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.001+0000 7fd65f9ff700 1 --2- 172.21.15.60:0/846648745 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658159510 0x7fd6580c8fd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd654009850 tx=0x7fd654009b60 comp rx=0 tx=0).stop 2023-07-13T22:27:36.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.001+0000 7fd65f9ff700 1 -- 172.21.15.60:0/846648745 shutdown_connections 2023-07-13T22:27:36.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.001+0000 7fd65f9ff700 1 --2- 172.21.15.60:0/846648745 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658159510 0x7fd6580c8fd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:36.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.001+0000 7fd65f9ff700 1 -- 172.21.15.60:0/846648745 >> 172.21.15.60:0/846648745 conn(0x7fd6580cbe70 msgr2=0x7fd6580ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:36.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.001+0000 7fd65f9ff700 1 -- 172.21.15.60:0/846648745 shutdown_connections 2023-07-13T22:27:36.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.001+0000 7fd65f9ff700 1 -- 172.21.15.60:0/846648745 wait complete. 2023-07-13T22:27:36.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.002+0000 7fd65f9ff700 1 Processor -- start 2023-07-13T22:27:36.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.003+0000 7fd65f9ff700 1 -- start start 2023-07-13T22:27:36.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65f9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658070af0 0x7fd6581f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:36.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65f9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6580c9510 0x7fd658151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:36.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65f9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd658159510 0x7fd658155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:36.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65f9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581efd80 con 0x7fd6580c9510 2023-07-13T22:27:36.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65f9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581f08a0 con 0x7fd658159510 2023-07-13T22:27:36.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65f9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581f0d30 con 0x7fd658070af0 2023-07-13T22:27:36.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65cf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6580c9510 0x7fd658151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:36.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65cf9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6580c9510 0x7fd658151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56604/0 (socket says 172.21.15.60:56604) 2023-07-13T22:27:36.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65cf9a700 1 -- 172.21.15.60:0/4008483632 learned_addr learned my addr 172.21.15.60:0/4008483632 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:36.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65d79b700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658070af0 0x7fd6581f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:36.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65df9c700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd658159510 0x7fd658155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:36.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65cf9a700 1 -- 172.21.15.60:0/4008483632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658070af0 msgr2=0x7fd6581f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:36.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65cf9a700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658070af0 0x7fd6581f3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65cf9a700 1 -- 172.21.15.60:0/4008483632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd658159510 msgr2=0x7fd658155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65cf9a700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd658159510 0x7fd658155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65cf9a700 1 -- 172.21.15.60:0/4008483632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd654009430 con 0x7fd6580c9510 2023-07-13T22:27:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.004+0000 7fd65df9c700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd658159510 0x7fd658155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.005+0000 7fd65cf9a700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6580c9510 0x7fd658151b80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c7a0 tx=0x7fd64800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.005+0000 7fd64dffb700 1 -- 172.21.15.60:0/4008483632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648007560 con 0x7fd6580c9510 2023-07-13T22:27:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.005+0000 7fd64dffb700 1 -- 172.21.15.60:0/4008483632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6480076c0 con 0x7fd6580c9510 2023-07-13T22:27:36.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.005+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd658155670 con 0x7fd6580c9510 2023-07-13T22:27:36.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.005+0000 7fd64dffb700 1 -- 172.21.15.60:0/4008483632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648004a70 con 0x7fd6580c9510 2023-07-13T22:27:36.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.005+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6581f0250 con 0x7fd6580c9510 2023-07-13T22:27:36.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.006+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6581f0a50 con 0x7fd6580c9510 2023-07-13T22:27:36.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.007+0000 7fd64dffb700 1 -- 172.21.15.60:0/4008483632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd648004bd0 con 0x7fd6580c9510 2023-07-13T22:27:36.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.008+0000 7fd64dffb700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd63c06ace0 0x7fd63c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:36.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.008+0000 7fd64dffb700 1 -- 172.21.15.60:0/4008483632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd648088a80 con 0x7fd6580c9510 2023-07-13T22:27:36.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.009+0000 7fd65d79b700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd63c06ace0 0x7fd63c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:36.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.009+0000 7fd65d79b700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd63c06ace0 0x7fd63c06d190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd6540097a0 tx=0x7fd654008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:36.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.013+0000 7fd64dffb700 1 -- 172.21.15.60:0/4008483632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd6581f0a50 con 0x7fd6580c9510 2023-07-13T22:27:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.266+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd6581f0a50 con 0x7fd6580c9510 2023-07-13T22:27:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.266+0000 7fd64dffb700 1 -- 172.21.15.60:0/4008483632 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fd6581f0a50 con 0x7fd6580c9510 2023-07-13T22:27:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:37.248 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:27:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.268+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd63c06ace0 msgr2=0x7fd63c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.268+0000 7fd65f9ff700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd63c06ace0 0x7fd63c06d190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd6540097a0 tx=0x7fd654008070 comp rx=0 tx=0).stop 2023-07-13T22:27:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.268+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6580c9510 msgr2=0x7fd658151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.268+0000 7fd65f9ff700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6580c9510 0x7fd658151b80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c7a0 tx=0x7fd64800cb60 comp rx=0 tx=0).stop 2023-07-13T22:27:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.268+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 shutdown_connections 2023-07-13T22:27:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.269+0000 7fd65f9ff700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd63c06ace0 0x7fd63c06d190 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.269+0000 7fd65f9ff700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd658159510 0x7fd658155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.269+0000 7fd65f9ff700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd658070af0 0x7fd6581f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.269+0000 7fd65f9ff700 1 --2- 172.21.15.60:0/4008483632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6580c9510 0x7fd658151b80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.269+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 >> 172.21.15.60:0/4008483632 conn(0x7fd6580cbe70 msgr2=0x7fd65815c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.269+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 shutdown_connections 2023-07-13T22:27:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.269+0000 7fd65f9ff700 1 -- 172.21.15.60:0/4008483632 wait complete. 2023-07-13T22:27:37.267 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-13T22:27:37.267 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:27:37.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:27:37.370 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:27:37.477 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:27:37.478 INFO:tasks.cephfs.mount:Terminating background process 2023-07-13T22:27:37.490 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:27:37.490 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:27:37.576 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:27:37.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.838+0000 7f64e7fff700 1 Processor -- start 2023-07-13T22:27:37.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.838+0000 7f64e7fff700 1 -- start start 2023-07-13T22:27:37.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e7fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f64e0159170 0x7f64e00c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:37.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64e00c5040 0x7f64e0070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:37.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00707b0 0x7f64e00cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:37.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e7fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e015d970 con 0x7f64e00c5040 2023-07-13T22:27:37.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e7fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e0070bf0 con 0x7f64e0159170 2023-07-13T22:27:37.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e7fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e015cd60 con 0x7f64e00707b0 2023-07-13T22:27:37.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e559a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64e00c5040 0x7f64e0070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:37.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e559a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64e00c5040 0x7f64e0070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56606/0 (socket says 172.21.15.60:56606) 2023-07-13T22:27:37.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e559a700 1 -- 172.21.15.60:0/3472761399 learned_addr learned my addr 172.21.15.60:0/3472761399 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:37.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e5d9b700 1 --2- 172.21.15.60:0/3472761399 >> v2:172.21.15.105:3300/0 conn(0x7f64e0159170 0x7f64e00c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e659c700 1 --2- 172.21.15.60:0/3472761399 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00707b0 0x7f64e00cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e559a700 1 -- 172.21.15.60:0/3472761399 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00707b0 msgr2=0x7f64e00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e559a700 1 --2- 172.21.15.60:0/3472761399 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00707b0 0x7f64e00cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e559a700 1 -- 172.21.15.60:0/3472761399 >> v2:172.21.15.105:3300/0 conn(0x7f64e0159170 msgr2=0x7f64e00c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e559a700 1 --2- 172.21.15.60:0/3472761399 >> v2:172.21.15.105:3300/0 conn(0x7f64e0159170 0x7f64e00c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.840+0000 7f64e559a700 1 -- 172.21.15.60:0/3472761399 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e015d040 con 0x7f64e00c5040 2023-07-13T22:27:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.841+0000 7f64e559a700 1 --2- 172.21.15.60:0/3472761399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00c5040 0x7f64e0070270 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f64e0067080 tx=0x7f64d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06f6561c117da9f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:37.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.841+0000 7f64e4d99700 1 -- 172.21.15.60:0/3472761399 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d000b870 con 0x7f64e00c5040 2023-07-13T22:27:37.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.841+0000 7f64e4d99700 1 -- 172.21.15.60:0/3472761399 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d000b9d0 con 0x7f64e00c5040 2023-07-13T22:27:37.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.841+0000 7f64e4d99700 1 -- 172.21.15.60:0/3472761399 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d000bd60 con 0x7f64e00c5040 2023-07-13T22:27:37.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.841+0000 7f64e7fff700 1 -- 172.21.15.60:0/3472761399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00c5040 msgr2=0x7f64e0070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:37.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.841+0000 7f64e7fff700 1 --2- 172.21.15.60:0/3472761399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00c5040 0x7f64e0070270 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f64e0067080 tx=0x7f64d0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.842+0000 7f64e7fff700 1 -- 172.21.15.60:0/3472761399 shutdown_connections 2023-07-13T22:27:37.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.842+0000 7f64e7fff700 1 --2- 172.21.15.60:0/3472761399 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00707b0 0x7f64e00cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.842+0000 7f64e7fff700 1 --2- 172.21.15.60:0/3472761399 >> v2:172.21.15.105:3300/0 conn(0x7f64e0159170 0x7f64e00c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.842+0000 7f64e7fff700 1 --2- 172.21.15.60:0/3472761399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00c5040 0x7f64e0070270 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.842+0000 7f64e7fff700 1 -- 172.21.15.60:0/3472761399 >> 172.21.15.60:0/3472761399 conn(0x7f64e00cf410 msgr2=0x7f64e00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:37.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.842+0000 7f64e7fff700 1 -- 172.21.15.60:0/3472761399 shutdown_connections 2023-07-13T22:27:37.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.842+0000 7f64e7fff700 1 -- 172.21.15.60:0/3472761399 wait complete. 2023-07-13T22:27:37.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.843+0000 7f64e7fff700 1 Processor -- start 2023-07-13T22:27:37.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.843+0000 7f64e7fff700 1 -- start start 2023-07-13T22:27:37.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.844+0000 7f64e7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00707b0 0x7f64e00b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:37.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.844+0000 7f64e7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00c5040 0x7f64e014c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:37.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.844+0000 7f64e7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e0159170 0x7f64e01577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:37.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.844+0000 7f64e7fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0156420 con 0x7f64e00707b0 2023-07-13T22:27:37.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.844+0000 7f64e7fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00b20e0 con 0x7f64e0159170 2023-07-13T22:27:37.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.844+0000 7f64e7fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00b2660 con 0x7f64e00c5040 2023-07-13T22:27:37.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00707b0 0x7f64e00b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:37.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00707b0 0x7f64e00b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56620/0 (socket says 172.21.15.60:56620) 2023-07-13T22:27:37.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e5d9b700 1 -- 172.21.15.60:0/1140428451 learned_addr learned my addr 172.21.15.60:0/1140428451 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e559a700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00c5040 0x7f64e014c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e659c700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e0159170 0x7f64e01577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e5d9b700 1 -- 172.21.15.60:0/1140428451 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00c5040 msgr2=0x7f64e014c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e5d9b700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00c5040 0x7f64e014c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e5d9b700 1 -- 172.21.15.60:0/1140428451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e0159170 msgr2=0x7f64e01577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e5d9b700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e0159170 0x7f64e01577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e5d9b700 1 -- 172.21.15.60:0/1140428451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0009550 con 0x7f64e00707b0 2023-07-13T22:27:37.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.845+0000 7f64e5d9b700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00707b0 0x7f64e00b2af0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00e810 tx=0x7f64dc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:37.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.846+0000 7f64d67fc700 1 -- 172.21.15.60:0/1140428451 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00c070 con 0x7f64e00707b0 2023-07-13T22:27:37.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.846+0000 7f64d67fc700 1 -- 172.21.15.60:0/1140428451 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc013610 con 0x7f64e00707b0 2023-07-13T22:27:37.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.846+0000 7f64d67fc700 1 -- 172.21.15.60:0/1140428451 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0139c0 con 0x7f64e00707b0 2023-07-13T22:27:37.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.846+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e00b3030 con 0x7f64e00707b0 2023-07-13T22:27:37.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.846+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e0155e60 con 0x7f64e00707b0 2023-07-13T22:27:37.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.847+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e00ad860 con 0x7f64e00707b0 2023-07-13T22:27:37.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.848+0000 7f64d67fc700 1 -- 172.21.15.60:0/1140428451 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f64dc013b20 con 0x7f64e00707b0 2023-07-13T22:27:37.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.849+0000 7f64d67fc700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c406ace0 0x7f64c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:37.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.849+0000 7f64d67fc700 1 -- 172.21.15.60:0/1140428451 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64dc089220 con 0x7f64e00707b0 2023-07-13T22:27:37.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.849+0000 7f64e559a700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c406ace0 0x7f64c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:37.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.850+0000 7f64e559a700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c406ace0 0x7f64c406d190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64e00c5480 tx=0x7f64d0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:37.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:37.853+0000 7f64d67fc700 1 -- 172.21.15.60:0/1140428451 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f64dc054240 con 0x7f64e00707b0 2023-07-13T22:27:38.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.063+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64e0071440 con 0x7f64e00707b0 2023-07-13T22:27:38.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.064+0000 7f64d67fc700 1 -- 172.21.15.60:0/1140428451 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18260 (secure 0 0 0) 0x7f64dc057860 con 0x7f64e00707b0 2023-07-13T22:27:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:38.045 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":75,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:25:03.885230+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:25:00.672661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-13T22:25:01.364844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:27:38.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c406ace0 msgr2=0x7f64c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c406ace0 0x7f64c406d190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64e00c5480 tx=0x7f64d0008030 comp rx=0 tx=0).stop 2023-07-13T22:27:38.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00707b0 msgr2=0x7f64e00b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00707b0 0x7f64e00b2af0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00e810 tx=0x7f64dc00eb20 comp rx=0 tx=0).stop 2023-07-13T22:27:38.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 shutdown_connections 2023-07-13T22:27:38.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f64c406ace0 0x7f64c406d190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64e00707b0 0x7f64e00b2af0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64e0159170 0x7f64e01577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 --2- 172.21.15.60:0/1140428451 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f64e00c5040 0x7f64e014c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 >> 172.21.15.60:0/1140428451 conn(0x7f64e00cf410 msgr2=0x7f64e01583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:38.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 shutdown_connections 2023-07-13T22:27:38.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.067+0000 7f64e7fff700 1 -- 172.21.15.60:0/1140428451 wait complete. 2023-07-13T22:27:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:25:00.672661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:25:01.364844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:27:38.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:27:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.290+0000 7feefc5ff700 1 Processor -- start 2023-07-13T22:27:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.291+0000 7feefc5ff700 1 -- start start 2023-07-13T22:27:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.292+0000 7feefc5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feef406ce70 0x7feef40ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.292+0000 7feefc5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef40c8e90 con 0x7feef406a5e0 2023-07-13T22:27:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.292+0000 7feefc5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef40caf30 con 0x7feef40add00 2023-07-13T22:27:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.292+0000 7feefc5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef40ae0d0 con 0x7feef406ce70 2023-07-13T22:27:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.292+0000 7feef9b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feef406a5e0 0x7feef40b1690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58554/0 (socket says 172.21.15.60:58554) 2023-07-13T22:27:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.292+0000 7feef9b9a700 1 -- 172.21.15.60:0/2789997688 learned_addr learned my addr 172.21.15.60:0/2789997688 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.292+0000 7feefab9c700 1 --2- 172.21.15.60:0/2789997688 >> v2:172.21.15.105:3300/0 conn(0x7feef406ce70 0x7feef40ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.292+0000 7feef9399700 1 -- 172.21.15.60:0/2789997688 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041567162 0 0) 0x7feef40c8e90 con 0x7feef406a5e0 2023-07-13T22:27:38.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.293+0000 7feef9399700 1 -- 172.21.15.60:0/2789997688 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee00035b0 con 0x7feef406a5e0 2023-07-13T22:27:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.293+0000 7feefab9c700 1 -- 172.21.15.60:0/2789997688 >> v1:172.21.15.105:6789/0 conn(0x7feef40add00 legacy=0x7feef4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.293+0000 7feefab9c700 1 -- 172.21.15.60:0/2789997688 >> v1:172.21.15.60:6789/0 conn(0x7feef406a5e0 legacy=0x7feef40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.293+0000 7feef9399700 1 -- 172.21.15.60:0/2789997688 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3655790228 0 0) 0x7feee00035b0 con 0x7feef406a5e0 2023-07-13T22:27:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.293+0000 7feefab9c700 1 -- 172.21.15.60:0/2789997688 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef40c90e0 con 0x7feef406ce70 2023-07-13T22:27:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.293+0000 7feefab9c700 1 --2- 172.21.15.60:0/2789997688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef406ce70 0x7feef40ca9f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feeec009d90 tx=0x7feeec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ce4132e4fe654f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.293+0000 7feef9399700 1 -- 172.21.15.60:0/2789997688 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec00e070 con 0x7feef406ce70 2023-07-13T22:27:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.293+0000 7feef9399700 1 -- 172.21.15.60:0/2789997688 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec00bb20 con 0x7feef406ce70 2023-07-13T22:27:38.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.293+0000 7feef9399700 1 -- 172.21.15.60:0/2789997688 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec00bed0 con 0x7feef406ce70 2023-07-13T22:27:38.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.294+0000 7feefc5ff700 1 -- 172.21.15.60:0/2789997688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef406ce70 msgr2=0x7feef40ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.294+0000 7feefc5ff700 1 --2- 172.21.15.60:0/2789997688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef406ce70 0x7feef40ca9f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feeec009d90 tx=0x7feeec00b330 comp rx=0 tx=0).stop 2023-07-13T22:27:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.294+0000 7feefc5ff700 1 -- 172.21.15.60:0/2789997688 shutdown_connections 2023-07-13T22:27:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.294+0000 7feefc5ff700 1 --2- 172.21.15.60:0/2789997688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef406ce70 0x7feef40ca9f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.294+0000 7feefc5ff700 1 -- 172.21.15.60:0/2789997688 >> 172.21.15.60:0/2789997688 conn(0x7feef40cbf50 msgr2=0x7feef40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.294+0000 7feefc5ff700 1 -- 172.21.15.60:0/2789997688 shutdown_connections 2023-07-13T22:27:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.294+0000 7feefc5ff700 1 -- 172.21.15.60:0/2789997688 wait complete. 2023-07-13T22:27:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.296+0000 7feefc5ff700 1 Processor -- start 2023-07-13T22:27:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.296+0000 7feefc5ff700 1 -- start start 2023-07-13T22:27:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefc5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feef406a5e0 0x7feef40b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefc5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef406ce70 0x7feef40c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefc5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feef40add00 0x7feef40b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefc5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef40b1170 con 0x7feef406a5e0 2023-07-13T22:27:38.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefc5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef40b17a0 con 0x7feef406ce70 2023-07-13T22:27:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefc5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef40b1a80 con 0x7feef40add00 2023-07-13T22:27:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefa39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feef406a5e0 0x7feef40b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefa39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feef406a5e0 0x7feef40b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56624/0 (socket says 172.21.15.60:56624) 2023-07-13T22:27:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefa39b700 1 -- 172.21.15.60:0/877428690 learned_addr learned my addr 172.21.15.60:0/877428690 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feefab9c700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feef40add00 0x7feef40b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.297+0000 7feef9b9a700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef406ce70 0x7feef40c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feefa39b700 1 -- 172.21.15.60:0/877428690 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feef40add00 msgr2=0x7feef40b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feefa39b700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feef40add00 0x7feef40b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feefa39b700 1 -- 172.21.15.60:0/877428690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef406ce70 msgr2=0x7feef40c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feefa39b700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef406ce70 0x7feef40c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feefa39b700 1 -- 172.21.15.60:0/877428690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec009a70 con 0x7feef406a5e0 2023-07-13T22:27:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feefa39b700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feef406a5e0 0x7feef40b0860 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7feef000c450 tx=0x7feef000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feeeaffd700 1 -- 172.21.15.60:0/877428690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef00052d0 con 0x7feef406a5e0 2023-07-13T22:27:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feeeaffd700 1 -- 172.21.15.60:0/877428690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0005430 con 0x7feef406a5e0 2023-07-13T22:27:38.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef40b1d60 con 0x7feef406a5e0 2023-07-13T22:27:38.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feeeaffd700 1 -- 172.21.15.60:0/877428690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef00038c0 con 0x7feef406a5e0 2023-07-13T22:27:38.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.298+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef40b0da0 con 0x7feef406a5e0 2023-07-13T22:27:38.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.299+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef40b2280 con 0x7feef406a5e0 2023-07-13T22:27:38.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.300+0000 7feeeaffd700 1 -- 172.21.15.60:0/877428690 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feef0003a20 con 0x7feef406a5e0 2023-07-13T22:27:38.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.301+0000 7feeeaffd700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feed806ace0 0x7feed806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.301+0000 7feeeaffd700 1 -- 172.21.15.60:0/877428690 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feef0088830 con 0x7feef406a5e0 2023-07-13T22:27:38.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.302+0000 7feef9b9a700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feed806ace0 0x7feed806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.302+0000 7feef9b9a700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feed806ace0 0x7feed806d190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feef40c8e70 tx=0x7feee4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.306+0000 7feeeaffd700 1 -- 172.21.15.60:0/877428690 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7feef40b2280 con 0x7feef406a5e0 2023-07-13T22:27:38.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.509+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feef404e210 con 0x7feef406a5e0 2023-07-13T22:27:38.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.510+0000 7feeeaffd700 1 -- 172.21.15.60:0/877428690 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7feef0053850 con 0x7feef406a5e0 2023-07-13T22:27:38.491 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:27:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feed806ace0 msgr2=0x7feed806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feed806ace0 0x7feed806d190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feef40c8e70 tx=0x7feee4009040 comp rx=0 tx=0).stop 2023-07-13T22:27:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feef406a5e0 msgr2=0x7feef40b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feef406a5e0 0x7feef40b0860 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7feef000c450 tx=0x7feef000c760 comp rx=0 tx=0).stop 2023-07-13T22:27:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 shutdown_connections 2023-07-13T22:27:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feed806ace0 0x7feed806d190 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feef406a5e0 0x7feef40b0860 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feef40add00 0x7feef40b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 --2- 172.21.15.60:0/877428690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef406ce70 0x7feef40c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 >> 172.21.15.60:0/877428690 conn(0x7feef40cbf50 msgr2=0x7feef4069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 shutdown_connections 2023-07-13T22:27:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.512+0000 7feefc5ff700 1 -- 172.21.15.60:0/877428690 wait complete. 2023-07-13T22:27:38.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:27:38.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.738+0000 7fb7c95ff700 1 Processor -- start 2023-07-13T22:27:38.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.738+0000 7fb7c95ff700 1 -- start start 2023-07-13T22:27:38.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.740+0000 7fb7c95ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb7c40b1ac0 0x7fb7c40d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.740+0000 7fb7c95ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c40d1870 con 0x7fb7c40cfd50 2023-07-13T22:27:38.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.740+0000 7fb7c95ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c40cf370 con 0x7fb7c40b8200 2023-07-13T22:27:38.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.740+0000 7fb7c95ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c40d12b0 con 0x7fb7c40b1ac0 2023-07-13T22:27:38.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.740+0000 7fb7bbfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb7c40cfd50 0x7fb7c40d23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58570/0 (socket says 172.21.15.60:58570) 2023-07-13T22:27:38.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.740+0000 7fb7bbfff700 1 -- 172.21.15.60:0/1739332738 learned_addr learned my addr 172.21.15.60:0/1739332738 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:38.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.740+0000 7fb7c37fe700 1 --2- 172.21.15.60:0/1739332738 >> v2:172.21.15.105:3300/0 conn(0x7fb7c40b1ac0 0x7fb7c40d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.740+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690223641 0 0) 0x7fb7c40d1870 con 0x7fb7c40cfd50 2023-07-13T22:27:38.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.740+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7b00035b0 con 0x7fb7c40cfd50 2023-07-13T22:27:38.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297834852 0 0) 0x7fb7c40cf370 con 0x7fb7c40b8200 2023-07-13T22:27:38.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c40d1870 con 0x7fb7c40b8200 2023-07-13T22:27:38.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2579921700 0 0) 0x7fb7b00035b0 con 0x7fb7c40cfd50 2023-07-13T22:27:38.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 >> v2:172.21.15.105:3300/0 conn(0x7fb7c40b1ac0 msgr2=0x7fb7c40d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:38.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 --2- 172.21.15.60:0/1739332738 >> v2:172.21.15.105:3300/0 conn(0x7fb7c40b1ac0 0x7fb7c40d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 >> v1:172.21.15.105:6789/0 conn(0x7fb7c40b8200 legacy=0x7fb7c40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c40d1590 con 0x7fb7c40cfd50 2023-07-13T22:27:38.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fb7ac001670 con 0x7fb7c40cfd50 2023-07-13T22:27:38.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c37fe700 1 --2- 172.21.15.60:0/1739332738 >> v2:172.21.15.105:3300/0 conn(0x7fb7c40b1ac0 0x7fb7c40d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:38.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 >> v1:172.21.15.60:6789/0 conn(0x7fb7c40cfd50 legacy=0x7fb7c40d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b0003b90 0x7fb7b0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b0004850 0x7fb7b0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b0005200 0x7fb7b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c40d12b0 con 0x7fb7b0003b90 2023-07-13T22:27:38.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b000c060 con 0x7fb7b0004850 2023-07-13T22:27:38.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b000c220 con 0x7fb7b0005200 2023-07-13T22:27:38.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c2ffd700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b0003b90 0x7fb7b0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7bbfff700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b0004850 0x7fb7b0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c37fe700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b0005200 0x7fb7b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c2ffd700 1 -- 172.21.15.60:0/1739332738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b0005200 msgr2=0x7fb7b000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:38.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.741+0000 7fb7c2ffd700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b0005200 0x7fb7b000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c2ffd700 1 -- 172.21.15.60:0/1739332738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b0004850 msgr2=0x7fb7b0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:38.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c2ffd700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b0004850 0x7fb7b0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c2ffd700 1 -- 172.21.15.60:0/1739332738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00035b0 con 0x7fb7b0003b90 2023-07-13T22:27:38.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7bbfff700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b0004850 0x7fb7b0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:38.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c37fe700 1 -- 172.21.15.60:0/1739332738 reap_dead start 2023-07-13T22:27:38.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c2ffd700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b0003b90 0x7fb7b0003f80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4001940 tx=0x7fb7b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b81ecc5516e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:38.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4001430 con 0x7fb7b0003b90 2023-07-13T22:27:38.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b4001d70 con 0x7fb7b0003b90 2023-07-13T22:27:38.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c27fc700 1 -- 172.21.15.60:0/1739332738 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4005510 con 0x7fb7b0003b90 2023-07-13T22:27:38.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1739332738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b0003b90 msgr2=0x7fb7b0003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.742+0000 7fb7c95ff700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b0003b90 0x7fb7b0003f80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4001940 tx=0x7fb7b0005880 comp rx=0 tx=0).stop 2023-07-13T22:27:38.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.743+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1739332738 shutdown_connections 2023-07-13T22:27:38.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.743+0000 7fb7c95ff700 1 --2- 172.21.15.60:0/1739332738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b0003b90 0x7fb7b0003f80 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.743+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1739332738 >> 172.21.15.60:0/1739332738 conn(0x7fb7c406b2f0 msgr2=0x7fb7c406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:38.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.743+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1739332738 shutdown_connections 2023-07-13T22:27:38.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.743+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1739332738 wait complete. 2023-07-13T22:27:38.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.744+0000 7fb7c95ff700 1 Processor -- start 2023-07-13T22:27:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.745+0000 7fb7c95ff700 1 -- start start 2023-07-13T22:27:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.745+0000 7fb7c95ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b0003b90 0x7fb7c40c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.745+0000 7fb7c95ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7c40ca6b0 0x7fb7c41f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.745+0000 7fb7c95ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7c415a110 0x7fb7c415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.745+0000 7fb7c95ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415aad0 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.745+0000 7fb7c95ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c41f5ff0 con 0x7fb7b0003b90 2023-07-13T22:27:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7c95ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c41f53e0 con 0x7fb7c415a110 2023-07-13T22:27:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7bbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7c40ca6b0 0x7fb7c41f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7bbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7c40ca6b0 0x7fb7c41f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56636/0 (socket says 172.21.15.60:56636) 2023-07-13T22:27:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7bbfff700 1 -- 172.21.15.60:0/1265299027 learned_addr learned my addr 172.21.15.60:0/1265299027 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7bbfff700 1 -- 172.21.15.60:0/1265299027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7c415a110 msgr2=0x7fb7c415a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7bbfff700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7c415a110 0x7fb7c415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7bbfff700 1 -- 172.21.15.60:0/1265299027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b0003b90 msgr2=0x7fb7c40c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7bbfff700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b0003b90 0x7fb7c40c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7bbfff700 1 -- 172.21.15.60:0/1265299027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c41f56c0 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.746+0000 7fb7bbfff700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7c40ca6b0 0x7fb7c41f4b40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00eaf0 tx=0x7fb7ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.747+0000 7fb7bb7fe700 1 -- 172.21.15.60:0/1265299027 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00c520 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.747+0000 7fb7bb7fe700 1 -- 172.21.15.60:0/1265299027 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac00c680 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.747+0000 7fb7bb7fe700 1 -- 172.21.15.60:0/1265299027 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00c9f0 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.747+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c41f59a0 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.747+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c40ca240 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.748+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c404e210 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.749+0000 7fb7bb7fe700 1 -- 172.21.15.60:0/1265299027 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7ac00cb50 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.750+0000 7fb7bb7fe700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a406ace0 0x7fb7a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.750+0000 7fb7bb7fe700 1 -- 172.21.15.60:0/1265299027 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7ac089260 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.750+0000 7fb7c2ffd700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a406ace0 0x7fb7a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.751+0000 7fb7c2ffd700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a406ace0 0x7fb7a406d190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4005040 tx=0x7fb7b4009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.755+0000 7fb7bb7fe700 1 -- 172.21.15.60:0/1265299027 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb7ac054330 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.963+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7c41f6130 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.964+0000 7fb7bb7fe700 1 -- 172.21.15.60:0/1265299027 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7ac057950 con 0x7fb7c40ca6b0 2023-07-13T22:27:38.945 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:27:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a406ace0 msgr2=0x7fb7a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a406ace0 0x7fb7a406d190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4005040 tx=0x7fb7b4009030 comp rx=0 tx=0).stop 2023-07-13T22:27:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7c40ca6b0 msgr2=0x7fb7c41f4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7c40ca6b0 0x7fb7c41f4b40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00eaf0 tx=0x7fb7ac00eeb0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 shutdown_connections 2023-07-13T22:27:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a406ace0 0x7fb7a406d190 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7c415a110 0x7fb7c415a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b0003b90 0x7fb7c40c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 --2- 172.21.15.60:0/1265299027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7c40ca6b0 0x7fb7c41f4b40 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 >> 172.21.15.60:0/1265299027 conn(0x7fb7c406b2f0 msgr2=0x7fb7c40b7480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.966+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 shutdown_connections 2023-07-13T22:27:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:38.967+0000 7fb7c95ff700 1 -- 172.21.15.60:0/1265299027 wait complete. 2023-07-13T22:27:38.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:27:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.195+0000 7f703adff700 1 Processor -- start 2023-07-13T22:27:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.195+0000 7f703adff700 1 -- start start 2023-07-13T22:27:39.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.196+0000 7f703adff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f703415c790 0x7f70340ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.197+0000 7f703adff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70340ac490 0x7f70340c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.197+0000 7f703adff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70340cf6d0 con 0x7f70340ca3a0 2023-07-13T22:27:39.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.197+0000 7f703adff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70340ac900 con 0x7f70340ac490 2023-07-13T22:27:39.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.197+0000 7f703adff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70340acbe0 con 0x7f703415c790 2023-07-13T22:27:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.197+0000 7f7038b9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f703415c790 0x7f70340ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.197+0000 7f7033fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70340ac490 0x7f70340c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.197+0000 7f7038b9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f703415c790 0x7f70340ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:40572/0 (socket says 172.21.15.60:40572) 2023-07-13T22:27:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.197+0000 7f7038b9b700 1 -- 172.21.15.60:0/985859527 learned_addr learned my addr 172.21.15.60:0/985859527 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.198+0000 7f7038b9b700 1 -- 172.21.15.60:0/985859527 >> v2:172.21.15.105:3300/0 conn(0x7f70340ac490 msgr2=0x7f70340c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.198+0000 7f7038b9b700 1 --2- 172.21.15.60:0/985859527 >> v2:172.21.15.105:3300/0 conn(0x7f70340ac490 0x7f70340c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.198+0000 7f7038b9b700 1 -- 172.21.15.60:0/985859527 >> v1:172.21.15.105:6789/0 conn(0x7f70340ca3a0 legacy=0x7f70340ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.198+0000 7f7038b9b700 1 -- 172.21.15.60:0/985859527 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70340acec0 con 0x7f703415c790 2023-07-13T22:27:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.198+0000 7f7033fff700 1 --2- 172.21.15.60:0/985859527 >> v2:172.21.15.105:3300/0 conn(0x7f70340ac490 0x7f70340c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.198+0000 7f7038b9b700 1 --2- 172.21.15.60:0/985859527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f703415c790 0x7f70340ce4b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7028006fd0 tx=0x7f7028009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b357723816bc7156 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.198+0000 7f70337fe700 1 -- 172.21.15.60:0/985859527 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028005e30 con 0x7f703415c790 2023-07-13T22:27:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.198+0000 7f70337fe700 1 -- 172.21.15.60:0/985859527 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7028004030 con 0x7f703415c790 2023-07-13T22:27:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.198+0000 7f70337fe700 1 -- 172.21.15.60:0/985859527 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70280043e0 con 0x7f703415c790 2023-07-13T22:27:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.199+0000 7f703adff700 1 -- 172.21.15.60:0/985859527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f703415c790 msgr2=0x7f70340ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.199+0000 7f703adff700 1 --2- 172.21.15.60:0/985859527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f703415c790 0x7f70340ce4b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7028006fd0 tx=0x7f7028009bd0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.199+0000 7f703adff700 1 -- 172.21.15.60:0/985859527 shutdown_connections 2023-07-13T22:27:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.199+0000 7f703adff700 1 --2- 172.21.15.60:0/985859527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f703415c790 0x7f70340ce4b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.199+0000 7f703adff700 1 --2- 172.21.15.60:0/985859527 >> v2:172.21.15.105:3300/0 conn(0x7f70340ac490 0x7f70340c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.199+0000 7f703adff700 1 -- 172.21.15.60:0/985859527 >> 172.21.15.60:0/985859527 conn(0x7f703406b900 msgr2=0x7f70340cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.199+0000 7f703adff700 1 -- 172.21.15.60:0/985859527 shutdown_connections 2023-07-13T22:27:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.199+0000 7f703adff700 1 -- 172.21.15.60:0/985859527 wait complete. 2023-07-13T22:27:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.201+0000 7f703adff700 1 Processor -- start 2023-07-13T22:27:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.201+0000 7f703adff700 1 -- start start 2023-07-13T22:27:39.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f703adff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70340ac490 0x7f70341e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f703adff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70340ca3a0 0x7f703415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f703adff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703415c790 0x7f703409b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f703adff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703415b460 con 0x7f703415c790 2023-07-13T22:27:39.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f703adff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70341e4d50 con 0x7f70340ca3a0 2023-07-13T22:27:39.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f703adff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70341e8b00 con 0x7f70340ac490 2023-07-13T22:27:39.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f703939c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703415c790 0x7f703409b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f703939c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703415c790 0x7f703409b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56652/0 (socket says 172.21.15.60:56652) 2023-07-13T22:27:39.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f703939c700 1 -- 172.21.15.60:0/1535469866 learned_addr learned my addr 172.21.15.60:0/1535469866 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:39.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f7038b9b700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70340ac490 0x7f70341e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.202+0000 7f7033fff700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70340ca3a0 0x7f703415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f703939c700 1 -- 172.21.15.60:0/1535469866 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70340ac490 msgr2=0x7f70341e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f703939c700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70340ac490 0x7f70341e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f703939c700 1 -- 172.21.15.60:0/1535469866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70340ca3a0 msgr2=0x7f703415a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f703939c700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70340ca3a0 0x7f703415a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f703939c700 1 -- 172.21.15.60:0/1535469866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70200094a0 con 0x7f703415c790 2023-07-13T22:27:39.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f703939c700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703415c790 0x7f703409b340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f702c00cc50 tx=0x7f702c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:39.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f70317fa700 1 -- 172.21.15.60:0/1535469866 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c003c00 con 0x7f703415c790 2023-07-13T22:27:39.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f70317fa700 1 -- 172.21.15.60:0/1535469866 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c003d60 con 0x7f703415c790 2023-07-13T22:27:39.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f70317fa700 1 -- 172.21.15.60:0/1535469866 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c00aab0 con 0x7f703415c790 2023-07-13T22:27:39.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7028009500 con 0x7f703415c790 2023-07-13T22:27:39.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.203+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703409a980 con 0x7f703415c790 2023-07-13T22:27:39.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.204+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703409ab10 con 0x7f703415c790 2023-07-13T22:27:39.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.205+0000 7f70317fa700 1 -- 172.21.15.60:0/1535469866 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f702c00ac10 con 0x7f703415c790 2023-07-13T22:27:39.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.207+0000 7f70317fa700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f701806ace0 0x7f701806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.207+0000 7f70317fa700 1 -- 172.21.15.60:0/1535469866 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f702c089460 con 0x7f703415c790 2023-07-13T22:27:39.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.207+0000 7f7038b9b700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f701806ace0 0x7f701806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.207+0000 7f7038b9b700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f701806ace0 0x7f701806d190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7028006fd0 tx=0x7f7028008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:39.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.211+0000 7f70317fa700 1 -- 172.21.15.60:0/1535469866 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f702c054470 con 0x7f703415c790 2023-07-13T22:27:39.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.417+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f703404e210 con 0x7f703415c790 2023-07-13T22:27:39.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.418+0000 7f70317fa700 1 -- 172.21.15.60:0/1535469866 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f702c057a90 con 0x7f703415c790 2023-07-13T22:27:39.399 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:27:39.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f701806ace0 msgr2=0x7f701806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f701806ace0 0x7f701806d190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7028006fd0 tx=0x7f7028008070 comp rx=0 tx=0).stop 2023-07-13T22:27:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703415c790 msgr2=0x7f703409b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703415c790 0x7f703409b340 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f702c00cc50 tx=0x7f702c00cf60 comp rx=0 tx=0).stop 2023-07-13T22:27:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 shutdown_connections 2023-07-13T22:27:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f701806ace0 0x7f701806d190 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703415c790 0x7f703409b340 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70340ac490 0x7f70341e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 --2- 172.21.15.60:0/1535469866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70340ca3a0 0x7f703415a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.420+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 >> 172.21.15.60:0/1535469866 conn(0x7f703406b900 msgr2=0x7f70340ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:39.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.421+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 shutdown_connections 2023-07-13T22:27:39.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.421+0000 7f703adff700 1 -- 172.21.15.60:0/1535469866 wait complete. 2023-07-13T22:27:39.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:27:39.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.647+0000 7f1a2d5ff700 1 Processor -- start 2023-07-13T22:27:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.647+0000 7f1a2d5ff700 1 -- start start 2023-07-13T22:27:39.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.648+0000 7f1a2d5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1a28160970 0x7f1a280cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.648+0000 7f1a2d5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a280d1040 con 0x7f1a280b45b0 2023-07-13T22:27:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.648+0000 7f1a2d5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a28160de0 con 0x7f1a280d14d0 2023-07-13T22:27:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.648+0000 7f1a2d5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a280cd630 con 0x7f1a28160970 2023-07-13T22:27:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.648+0000 7f1a26ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1a280b45b0 0x7f1a280b4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58572/0 (socket says 172.21.15.60:58572) 2023-07-13T22:27:39.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.648+0000 7f1a26ffd700 1 -- 172.21.15.60:0/1853100806 learned_addr learned my addr 172.21.15.60:0/1853100806 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.648+0000 7f1a267fc700 1 --2- 172.21.15.60:0/1853100806 >> v2:172.21.15.105:3300/0 conn(0x7f1a28160970 0x7f1a280cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.649+0000 7f1a25ffb700 1 -- 172.21.15.60:0/1853100806 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31590703 0 0) 0x7f1a280d1040 con 0x7f1a280b45b0 2023-07-13T22:27:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.649+0000 7f1a25ffb700 1 -- 172.21.15.60:0/1853100806 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a100035b0 con 0x7f1a280b45b0 2023-07-13T22:27:39.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.649+0000 7f1a267fc700 1 -- 172.21.15.60:0/1853100806 >> v1:172.21.15.105:6789/0 conn(0x7f1a280d14d0 legacy=0x7f1a280d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.649+0000 7f1a267fc700 1 -- 172.21.15.60:0/1853100806 >> v1:172.21.15.60:6789/0 conn(0x7f1a280b45b0 legacy=0x7f1a280b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.649+0000 7f1a267fc700 1 -- 172.21.15.60:0/1853100806 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280cd840 con 0x7f1a28160970 2023-07-13T22:27:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.649+0000 7f1a267fc700 1 --2- 172.21.15.60:0/1853100806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a28160970 0x7f1a280cd0f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009840 tx=0x7f1a1c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=526603d60574f8fa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.649+0000 7f1a25ffb700 1 -- 172.21.15.60:0/1853100806 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011830 con 0x7f1a28160970 2023-07-13T22:27:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.650+0000 7f1a25ffb700 1 -- 172.21.15.60:0/1853100806 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c011990 con 0x7f1a28160970 2023-07-13T22:27:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.650+0000 7f1a25ffb700 1 -- 172.21.15.60:0/1853100806 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011d20 con 0x7f1a28160970 2023-07-13T22:27:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.650+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/1853100806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a28160970 msgr2=0x7f1a280cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.650+0000 7f1a2d5ff700 1 --2- 172.21.15.60:0/1853100806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a28160970 0x7f1a280cd0f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009840 tx=0x7f1a1c009b50 comp rx=0 tx=0).stop 2023-07-13T22:27:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.650+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/1853100806 shutdown_connections 2023-07-13T22:27:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.650+0000 7f1a2d5ff700 1 --2- 172.21.15.60:0/1853100806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a28160970 0x7f1a280cd0f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.650+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/1853100806 >> 172.21.15.60:0/1853100806 conn(0x7f1a28072240 msgr2=0x7f1a280d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.650+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/1853100806 shutdown_connections 2023-07-13T22:27:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.650+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/1853100806 wait complete. 2023-07-13T22:27:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.652+0000 7f1a2d5ff700 1 Processor -- start 2023-07-13T22:27:39.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.652+0000 7f1a2d5ff700 1 -- start start 2023-07-13T22:27:39.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a2d5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1a280b45b0 0x7f1a281f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a2d5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a280d14d0 0x7f1a280ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a2d5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a28160970 0x7f1a28155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a2d5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cba90 con 0x7f1a28160970 2023-07-13T22:27:39.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a2d5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cae80 con 0x7f1a280d14d0 2023-07-13T22:27:39.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a2d5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cb160 con 0x7f1a280b45b0 2023-07-13T22:27:39.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a277fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a28160970 0x7f1a28155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a277fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a28160970 0x7f1a28155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56654/0 (socket says 172.21.15.60:56654) 2023-07-13T22:27:39.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a277fe700 1 -- 172.21.15.60:0/2432905817 learned_addr learned my addr 172.21.15.60:0/2432905817 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:39.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a26ffd700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1a280b45b0 0x7f1a281f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.653+0000 7f1a267fc700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a280d14d0 0x7f1a280ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a277fe700 1 -- 172.21.15.60:0/2432905817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1a280b45b0 msgr2=0x7f1a281f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a277fe700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1a280b45b0 0x7f1a281f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a277fe700 1 -- 172.21.15.60:0/2432905817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a280d14d0 msgr2=0x7f1a280ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a277fe700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a280d14d0 0x7f1a280ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a277fe700 1 -- 172.21.15.60:0/2432905817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c0094a0 con 0x7f1a28160970 2023-07-13T22:27:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a277fe700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a28160970 0x7f1a28155aa0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000cc50 tx=0x7f1a2000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a0f7fe700 1 -- 172.21.15.60:0/2432905817 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20003c00 con 0x7f1a28160970 2023-07-13T22:27:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a0f7fe700 1 -- 172.21.15.60:0/2432905817 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20003d60 con 0x7f1a28160970 2023-07-13T22:27:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a280cb440 con 0x7f1a28160970 2023-07-13T22:27:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a0f7fe700 1 -- 172.21.15.60:0/2432905817 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000aab0 con 0x7f1a28160970 2023-07-13T22:27:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.654+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a280caac0 con 0x7f1a28160970 2023-07-13T22:27:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.655+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2804e210 con 0x7f1a28160970 2023-07-13T22:27:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.656+0000 7f1a0f7fe700 1 -- 172.21.15.60:0/2432905817 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a2000ac10 con 0x7f1a28160970 2023-07-13T22:27:39.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.657+0000 7f1a0f7fe700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1a1406ace0 0x7f1a1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:39.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.657+0000 7f1a0f7fe700 1 -- 172.21.15.60:0/2432905817 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a200894b0 con 0x7f1a28160970 2023-07-13T22:27:39.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.657+0000 7f1a26ffd700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1a1406ace0 0x7f1a1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:39.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.658+0000 7f1a26ffd700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1a1406ace0 0x7f1a1406d190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1a18007fd0 tx=0x7f1a18007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:39.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.661+0000 7f1a0f7fe700 1 -- 172.21.15.60:0/2432905817 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1a200544c0 con 0x7f1a28160970 2023-07-13T22:27:39.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.871+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a280b2270 con 0x7f1a28160970 2023-07-13T22:27:39.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.871+0000 7f1a0f7fe700 1 -- 172.21.15.60:0/2432905817 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a20057ae0 con 0x7f1a28160970 2023-07-13T22:27:39.852 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:27:39.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.873+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1a1406ace0 msgr2=0x7f1a1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.873+0000 7f1a2d5ff700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1a1406ace0 0x7f1a1406d190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1a18007fd0 tx=0x7f1a18007dc0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.873+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a28160970 msgr2=0x7f1a28155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:39.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.873+0000 7f1a2d5ff700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a28160970 0x7f1a28155aa0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000cc50 tx=0x7f1a2000cf60 comp rx=0 tx=0).stop 2023-07-13T22:27:39.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.874+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 shutdown_connections 2023-07-13T22:27:39.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.874+0000 7f1a2d5ff700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1a1406ace0 0x7f1a1406d190 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.874+0000 7f1a2d5ff700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a28160970 0x7f1a28155aa0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.874+0000 7f1a2d5ff700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1a280b45b0 0x7f1a281f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.874+0000 7f1a2d5ff700 1 --2- 172.21.15.60:0/2432905817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a280d14d0 0x7f1a280ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:39.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.874+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 >> 172.21.15.60:0/2432905817 conn(0x7f1a28072240 msgr2=0x7f1a280b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:39.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.874+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 shutdown_connections 2023-07-13T22:27:39.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:39.874+0000 7f1a2d5ff700 1 -- 172.21.15.60:0/2432905817 wait complete. 2023-07-13T22:27:39.872 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:27:40.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.098+0000 7fbe0c1ff700 1 Processor -- start 2023-07-13T22:27:40.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.098+0000 7fbe0c1ff700 1 -- start start 2023-07-13T22:27:40.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.099+0000 7fbe0c1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe040b2010 0x7fbe040cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.099+0000 7fbe0c1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe040d14a0 con 0x7fbe040b5a00 2023-07-13T22:27:40.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.099+0000 7fbe0c1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe040d1780 con 0x7fbe040d26d0 2023-07-13T22:27:40.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.099+0000 7fbe0c1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe040d2570 con 0x7fbe040b2010 2023-07-13T22:27:40.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.099+0000 7fbe0a79c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbe040b5a00 0x7fbe0415fee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58582/0 (socket says 172.21.15.60:58582) 2023-07-13T22:27:40.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.099+0000 7fbe0a79c700 1 -- 172.21.15.60:0/999875377 learned_addr learned my addr 172.21.15.60:0/999875377 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:40.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.099+0000 7fbe09f9b700 1 --2- 172.21.15.60:0/999875377 >> v2:172.21.15.105:3300/0 conn(0x7fbe040b2010 0x7fbe040cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:40.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.100+0000 7fbe08f99700 1 -- 172.21.15.60:0/999875377 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967797394 0 0) 0x7fbe040d14a0 con 0x7fbe040b5a00 2023-07-13T22:27:40.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.100+0000 7fbe08f99700 1 -- 172.21.15.60:0/999875377 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf00035b0 con 0x7fbe040b5a00 2023-07-13T22:27:40.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.100+0000 7fbe09f9b700 1 -- 172.21.15.60:0/999875377 >> v1:172.21.15.105:6789/0 conn(0x7fbe040d26d0 legacy=0x7fbe040cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.100+0000 7fbe08f99700 1 -- 172.21.15.60:0/999875377 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207705772 0 0) 0x7fbe040d1780 con 0x7fbe040d26d0 2023-07-13T22:27:40.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.100+0000 7fbe09f9b700 1 -- 172.21.15.60:0/999875377 >> v1:172.21.15.60:6789/0 conn(0x7fbe040b5a00 legacy=0x7fbe0415fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.100+0000 7fbe09f9b700 1 -- 172.21.15.60:0/999875377 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe040cd2a0 con 0x7fbe040b2010 2023-07-13T22:27:40.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.100+0000 7fbe09f9b700 1 --2- 172.21.15.60:0/999875377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b2010 0x7fbe040cf1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009840 tx=0x7fbe00009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e35d9ceaa2be57 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:40.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe08f99700 1 -- 172.21.15.60:0/999875377 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00011830 con 0x7fbe040b2010 2023-07-13T22:27:40.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe08f99700 1 -- 172.21.15.60:0/999875377 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe00011990 con 0x7fbe040b2010 2023-07-13T22:27:40.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe08f99700 1 -- 172.21.15.60:0/999875377 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00011d20 con 0x7fbe040b2010 2023-07-13T22:27:40.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/999875377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b2010 msgr2=0x7fbe040cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe0c1ff700 1 --2- 172.21.15.60:0/999875377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b2010 0x7fbe040cf1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009840 tx=0x7fbe00009b50 comp rx=0 tx=0).stop 2023-07-13T22:27:40.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/999875377 shutdown_connections 2023-07-13T22:27:40.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe0c1ff700 1 --2- 172.21.15.60:0/999875377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b2010 0x7fbe040cf1a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/999875377 >> 172.21.15.60:0/999875377 conn(0x7fbe0406b380 msgr2=0x7fbe0406b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:40.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/999875377 shutdown_connections 2023-07-13T22:27:40.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.101+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/999875377 wait complete. 2023-07-13T22:27:40.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.103+0000 7fbe0c1ff700 1 Processor -- start 2023-07-13T22:27:40.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.103+0000 7fbe0c1ff700 1 -- start start 2023-07-13T22:27:40.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.104+0000 7fbe0c1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe040b2010 0x7fbe0415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.104+0000 7fbe0c1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b5a00 0x7fbe040c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.104+0000 7fbe0c1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe040d26d0 0x7fbe041568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.104+0000 7fbe0c1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0415f480 con 0x7fbe040d26d0 2023-07-13T22:27:40.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.104+0000 7fbe0c1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040ca2a0 con 0x7fbe040b5a00 2023-07-13T22:27:40.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.104+0000 7fbe0c1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040cc4d0 con 0x7fbe040b2010 2023-07-13T22:27:40.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.104+0000 7fbe0a79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe040d26d0 0x7fbe041568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:40.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.104+0000 7fbe0a79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe040d26d0 0x7fbe041568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56656/0 (socket says 172.21.15.60:56656) 2023-07-13T22:27:40.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.104+0000 7fbe0a79c700 1 -- 172.21.15.60:0/1368720170 learned_addr learned my addr 172.21.15.60:0/1368720170 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:40.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe09f9b700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe040b2010 0x7fbe0415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:40.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe0979a700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b5a00 0x7fbe040c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:40.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe0a79c700 1 -- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe040b2010 msgr2=0x7fbe0415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe0a79c700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe040b2010 0x7fbe0415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe0a79c700 1 -- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b5a00 msgr2=0x7fbe040c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe0a79c700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b5a00 0x7fbe040c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe0a79c700 1 -- 172.21.15.60:0/1368720170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000094a0 con 0x7fbe040d26d0 2023-07-13T22:27:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe09f9b700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe040b2010 0x7fbe0415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe0979a700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b5a00 0x7fbe040c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe0a79c700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe040d26d0 0x7fbe041568f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00cb40 tx=0x7fbdfc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbdfa7fc700 1 -- 172.21.15.60:0/1368720170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc00ab50 con 0x7fbe040d26d0 2023-07-13T22:27:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbdfa7fc700 1 -- 172.21.15.60:0/1368720170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc00acb0 con 0x7fbe040d26d0 2023-07-13T22:27:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbdfa7fc700 1 -- 172.21.15.60:0/1368720170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc012620 con 0x7fbe040d26d0 2023-07-13T22:27:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.105+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe040cc7b0 con 0x7fbe040d26d0 2023-07-13T22:27:40.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.106+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe04156e30 con 0x7fbe040d26d0 2023-07-13T22:27:40.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.106+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe040b82e0 con 0x7fbe040d26d0 2023-07-13T22:27:40.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.108+0000 7fbdfa7fc700 1 -- 172.21.15.60:0/1368720170 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbdfc01b3b0 con 0x7fbe040d26d0 2023-07-13T22:27:40.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.108+0000 7fbdfa7fc700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbde806ace0 0x7fbde806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.108+0000 7fbdfa7fc700 1 -- 172.21.15.60:0/1368720170 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbdfc0892b0 con 0x7fbe040d26d0 2023-07-13T22:27:40.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.109+0000 7fbe09f9b700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbde806ace0 0x7fbde806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:40.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.109+0000 7fbe09f9b700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbde806ace0 0x7fbde806d190 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbe0415ea70 tx=0x7fbe0001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:40.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.113+0000 7fbdfa7fc700 1 -- 172.21.15.60:0/1368720170 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbdfc054350 con 0x7fbe040d26d0 2023-07-13T22:27:40.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.337+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbe040b8000 con 0x7fbe040d26d0 2023-07-13T22:27:40.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.337+0000 7fbdfa7fc700 1 -- 172.21.15.60:0/1368720170 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbdfc057970 con 0x7fbe040d26d0 2023-07-13T22:27:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.339+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbde806ace0 msgr2=0x7fbde806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.339+0000 7fbe0c1ff700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbde806ace0 0x7fbde806d190 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbe0415ea70 tx=0x7fbe0001c040 comp rx=0 tx=0).stop 2023-07-13T22:27:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.339+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe040d26d0 msgr2=0x7fbe041568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.339+0000 7fbe0c1ff700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe040d26d0 0x7fbe041568f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00cb40 tx=0x7fbdfc00cf00 comp rx=0 tx=0).stop 2023-07-13T22:27:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.340+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 shutdown_connections 2023-07-13T22:27:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.340+0000 7fbe0c1ff700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbde806ace0 0x7fbde806d190 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.340+0000 7fbe0c1ff700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe040d26d0 0x7fbe041568f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.340+0000 7fbe0c1ff700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe040b2010 0x7fbe0415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.340+0000 7fbe0c1ff700 1 --2- 172.21.15.60:0/1368720170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe040b5a00 0x7fbe040c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.340+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 >> 172.21.15.60:0/1368720170 conn(0x7fbe0406b380 msgr2=0x7fbe0415b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.340+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 shutdown_connections 2023-07-13T22:27:40.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.340+0000 7fbe0c1ff700 1 -- 172.21.15.60:0/1368720170 wait complete. 2023-07-13T22:27:40.322 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:27:40.338 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:27:40.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:27:40.338 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:27:40.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[141735]: fuse finished with error 0 and tester_r 0 2023-07-13T22:27:40.410 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:27:40.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:27:40.411 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:27:40.586 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:27:40.586 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:27:40.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:27:40.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.858+0000 7fa8c8dff700 1 Processor -- start 2023-07-13T22:27:40.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.859+0000 7fa8c8dff700 1 -- start start 2023-07-13T22:27:40.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.860+0000 7fa8c8dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa8c40b8570 0x7fa8c40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.860+0000 7fa8c8dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8c40d35c0 0x7fa8c40d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.860+0000 7fa8c8dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40d09d0 con 0x7fa8c40cf3f0 2023-07-13T22:27:40.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.860+0000 7fa8c8dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c40d3b30 con 0x7fa8c40b8570 2023-07-13T22:27:40.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.860+0000 7fa8c8dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c40cda10 con 0x7fa8c40d35c0 2023-07-13T22:27:40.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.860+0000 7fa8c259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa8c40b8570 0x7fa8c40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:40.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.860+0000 7fa8c259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa8c40b8570 0x7fa8c40ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56670/0 (socket says 172.21.15.60:56670) 2023-07-13T22:27:40.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.860+0000 7fa8c259c700 1 -- 172.21.15.60:0/1393300507 learned_addr learned my addr 172.21.15.60:0/1393300507 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:40.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.860+0000 7fa8bbfff700 1 --2- 172.21.15.60:0/1393300507 >> v2:172.21.15.105:3300/0 conn(0x7fa8c40d35c0 0x7fa8c40d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.861+0000 7fa8c259c700 1 -- 172.21.15.60:0/1393300507 >> v2:172.21.15.105:3300/0 conn(0x7fa8c40d35c0 msgr2=0x7fa8c40d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.861+0000 7fa8c259c700 1 --2- 172.21.15.60:0/1393300507 >> v2:172.21.15.105:3300/0 conn(0x7fa8c40d35c0 0x7fa8c40d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.861+0000 7fa8c259c700 1 -- 172.21.15.60:0/1393300507 >> v1:172.21.15.105:6789/0 conn(0x7fa8c40cf3f0 legacy=0x7fa8c415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.861+0000 7fa8c259c700 1 -- 172.21.15.60:0/1393300507 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40cd450 con 0x7fa8c40b8570 2023-07-13T22:27:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.861+0000 7fa8c259c700 1 --2- 172.21.15.60:0/1393300507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 0x7fa8c40ceeb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40d07e0 tx=0x7fa8b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced522e25a79619f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.861+0000 7fa8c1d9b700 1 -- 172.21.15.60:0/1393300507 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4005e30 con 0x7fa8c40b8570 2023-07-13T22:27:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.861+0000 7fa8c1d9b700 1 -- 172.21.15.60:0/1393300507 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b4006030 con 0x7fa8c40b8570 2023-07-13T22:27:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.861+0000 7fa8c1d9b700 1 -- 172.21.15.60:0/1393300507 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40063e0 con 0x7fa8c40b8570 2023-07-13T22:27:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.862+0000 7fa8c8dff700 1 -- 172.21.15.60:0/1393300507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 msgr2=0x7fa8c40ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.862+0000 7fa8c8dff700 1 --2- 172.21.15.60:0/1393300507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 0x7fa8c40ceeb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40d07e0 tx=0x7fa8b4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.862+0000 7fa8c8dff700 1 -- 172.21.15.60:0/1393300507 shutdown_connections 2023-07-13T22:27:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.862+0000 7fa8c8dff700 1 --2- 172.21.15.60:0/1393300507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 0x7fa8c40ceeb0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.862+0000 7fa8c8dff700 1 --2- 172.21.15.60:0/1393300507 >> v2:172.21.15.105:3300/0 conn(0x7fa8c40d35c0 0x7fa8c40d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.862+0000 7fa8c8dff700 1 -- 172.21.15.60:0/1393300507 >> 172.21.15.60:0/1393300507 conn(0x7fa8c4071600 msgr2=0x7fa8c40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.862+0000 7fa8c8dff700 1 -- 172.21.15.60:0/1393300507 shutdown_connections 2023-07-13T22:27:40.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.862+0000 7fa8c8dff700 1 -- 172.21.15.60:0/1393300507 wait complete. 2023-07-13T22:27:40.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.864+0000 7fa8c8dff700 1 Processor -- start 2023-07-13T22:27:40.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.864+0000 7fa8c8dff700 1 -- start start 2023-07-13T22:27:40.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c8dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 0x7fa8c40c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c8dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8c40cf3f0 0x7fa8c40caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c8dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8c40d35c0 0x7fa8c4159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c8dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40c82e0 con 0x7fa8c40b8570 2023-07-13T22:27:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c8dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40cb020 con 0x7fa8c40cf3f0 2023-07-13T22:27:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c8dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40cc780 con 0x7fa8c40d35c0 2023-07-13T22:27:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 0x7fa8c40c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 0x7fa8c40c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56684/0 (socket says 172.21.15.60:56684) 2023-07-13T22:27:40.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c259c700 1 -- 172.21.15.60:0/3711306023 learned_addr learned my addr 172.21.15.60:0/3711306023 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:40.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c259c700 1 -- 172.21.15.60:0/3711306023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8c40d35c0 msgr2=0x7fa8c4159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c259c700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8c40d35c0 0x7fa8c4159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c259c700 1 -- 172.21.15.60:0/3711306023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8c40cf3f0 msgr2=0x7fa8c40caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:40.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c259c700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8c40cf3f0 0x7fa8c40caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:40.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c259c700 1 -- 172.21.15.60:0/3711306023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b4009500 con 0x7fa8c40b8570 2023-07-13T22:27:40.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.865+0000 7fa8c259c700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 0x7fa8c40c6cc0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4006000 tx=0x7fa8b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:40.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.866+0000 7fa8baffd700 1 -- 172.21.15.60:0/3711306023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4004250 con 0x7fa8c40b8570 2023-07-13T22:27:40.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.866+0000 7fa8baffd700 1 -- 172.21.15.60:0/3711306023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b40043f0 con 0x7fa8c40b8570 2023-07-13T22:27:40.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.866+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c40c7380 con 0x7fa8c40b8570 2023-07-13T22:27:40.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.866+0000 7fa8baffd700 1 -- 172.21.15.60:0/3711306023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40038e0 con 0x7fa8c40b8570 2023-07-13T22:27:40.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.866+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c40c7940 con 0x7fa8c40b8570 2023-07-13T22:27:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.867+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c404e280 con 0x7fa8c40b8570 2023-07-13T22:27:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.868+0000 7fa8baffd700 1 -- 172.21.15.60:0/3711306023 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa8b4022070 con 0x7fa8c40b8570 2023-07-13T22:27:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.869+0000 7fa8baffd700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8a406ad10 0x7fa8a406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.869+0000 7fa8baffd700 1 -- 172.21.15.60:0/3711306023 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8b408ad30 con 0x7fa8c40b8570 2023-07-13T22:27:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.869+0000 7fa8bbfff700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8a406ad10 0x7fa8a406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:40.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.870+0000 7fa8bbfff700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8a406ad10 0x7fa8a406d1c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40c66b0 tx=0x7fa8ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:40.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:40.873+0000 7fa8baffd700 1 -- 172.21.15.60:0/3711306023 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa8b4055dd0 con 0x7fa8c40b8570 2023-07-13T22:27:41.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.083+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c4073530 con 0x7fa8c40b8570 2023-07-13T22:27:41.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.084+0000 7fa8baffd700 1 -- 172.21.15.60:0/3711306023 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18260 (secure 0 0 0) 0x7fa8b40593f0 con 0x7fa8c40b8570 2023-07-13T22:27:41.065 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:41.065 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":75,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:25:03.885230+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:25:00.672661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-13T22:25:01.364844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:27:41.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8a406ad10 msgr2=0x7fa8a406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8a406ad10 0x7fa8a406d1c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40c66b0 tx=0x7fa8ac008040 comp rx=0 tx=0).stop 2023-07-13T22:27:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 msgr2=0x7fa8c40c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 0x7fa8c40c6cc0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4006000 tx=0x7fa8b4004750 comp rx=0 tx=0).stop 2023-07-13T22:27:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 shutdown_connections 2023-07-13T22:27:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa8a406ad10 0x7fa8a406d1c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c40b8570 0x7fa8c40c6cc0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8c40d35c0 0x7fa8c4159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 --2- 172.21.15.60:0/3711306023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8c40cf3f0 0x7fa8c40caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 >> 172.21.15.60:0/3711306023 conn(0x7fa8c4071600 msgr2=0x7fa8c4160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:41.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 shutdown_connections 2023-07-13T22:27:41.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.087+0000 7fa8c8dff700 1 -- 172.21.15.60:0/3711306023 wait complete. 2023-07-13T22:27:41.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:25:00.672661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:25:01.364844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:27:41.087 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:27:41.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.314+0000 7f94e8bff700 1 Processor -- start 2023-07-13T22:27:41.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.314+0000 7f94e8bff700 1 -- start start 2023-07-13T22:27:41.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.315+0000 7f94e8bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f94e4159260 0x7f94e40ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.315+0000 7f94e8bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f94e4070af0 0x7f94e40cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.315+0000 7f94e8bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e40cf640 con 0x7f94e40cc410 2023-07-13T22:27:41.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.315+0000 7f94e8bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e40ccf80 con 0x7f94e4070af0 2023-07-13T22:27:41.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.315+0000 7f94e8bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e40c9b40 con 0x7f94e4159260 2023-07-13T22:27:41.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.315+0000 7f94e2d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f94e4070af0 0x7f94e40cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:41.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.315+0000 7f94e2d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f94e4070af0 0x7f94e40cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56690/0 (socket says 172.21.15.60:56690) 2023-07-13T22:27:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e2d9d700 1 -- 172.21.15.60:0/549733784 learned_addr learned my addr 172.21.15.60:0/549733784 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e259c700 1 --2- 172.21.15.60:0/549733784 >> v2:172.21.15.105:3300/0 conn(0x7f94e4159260 0x7f94e40ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e2d9d700 1 -- 172.21.15.60:0/549733784 >> v2:172.21.15.105:3300/0 conn(0x7f94e4159260 msgr2=0x7f94e40ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e2d9d700 1 --2- 172.21.15.60:0/549733784 >> v2:172.21.15.105:3300/0 conn(0x7f94e4159260 0x7f94e40ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e2d9d700 1 -- 172.21.15.60:0/549733784 >> v1:172.21.15.105:6789/0 conn(0x7f94e40cc410 legacy=0x7f94e40703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e2d9d700 1 -- 172.21.15.60:0/549733784 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40c9e20 con 0x7f94e4070af0 2023-07-13T22:27:41.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e2d9d700 1 --2- 172.21.15.60:0/549733784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4070af0 0x7f94e40cc8b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f94e4071110 tx=0x7f94dc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94828641b872414 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:41.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e1d9b700 1 -- 172.21.15.60:0/549733784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc010070 con 0x7f94e4070af0 2023-07-13T22:27:41.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e1d9b700 1 -- 172.21.15.60:0/549733784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc00dce0 con 0x7f94e4070af0 2023-07-13T22:27:41.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.316+0000 7f94e1d9b700 1 -- 172.21.15.60:0/549733784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc00b310 con 0x7f94e4070af0 2023-07-13T22:27:41.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.317+0000 7f94e8bff700 1 -- 172.21.15.60:0/549733784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4070af0 msgr2=0x7f94e40cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.317+0000 7f94e8bff700 1 --2- 172.21.15.60:0/549733784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4070af0 0x7f94e40cc8b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f94e4071110 tx=0x7f94dc00d430 comp rx=0 tx=0).stop 2023-07-13T22:27:41.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.317+0000 7f94e8bff700 1 -- 172.21.15.60:0/549733784 shutdown_connections 2023-07-13T22:27:41.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.317+0000 7f94e8bff700 1 --2- 172.21.15.60:0/549733784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4070af0 0x7f94e40cc8b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.317+0000 7f94e8bff700 1 --2- 172.21.15.60:0/549733784 >> v2:172.21.15.105:3300/0 conn(0x7f94e4159260 0x7f94e40ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.317+0000 7f94e8bff700 1 -- 172.21.15.60:0/549733784 >> 172.21.15.60:0/549733784 conn(0x7f94e40cbf50 msgr2=0x7f94e40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:41.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.317+0000 7f94e8bff700 1 -- 172.21.15.60:0/549733784 shutdown_connections 2023-07-13T22:27:41.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.317+0000 7f94e8bff700 1 -- 172.21.15.60:0/549733784 wait complete. 2023-07-13T22:27:41.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.319+0000 7f94e8bff700 1 Processor -- start 2023-07-13T22:27:41.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.319+0000 7f94e8bff700 1 -- start start 2023-07-13T22:27:41.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e8bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4070af0 0x7f94e40b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94e40cc410 0x7f94e40c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e8bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4159260 0x7f94e40ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e8bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b0180 con 0x7f94e4159260 2023-07-13T22:27:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e8bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b3090 con 0x7f94e4070af0 2023-07-13T22:27:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b32f0 con 0x7f94e40cc410 2023-07-13T22:27:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e2d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4159260 0x7f94e40ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e2d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4159260 0x7f94e40ae160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56694/0 (socket says 172.21.15.60:56694) 2023-07-13T22:27:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e2d9d700 1 -- 172.21.15.60:0/3107801555 learned_addr learned my addr 172.21.15.60:0/3107801555 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:41.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.320+0000 7f94e259c700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4070af0 0x7f94e40b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:41.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94e2d9d700 1 -- 172.21.15.60:0/3107801555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94e40cc410 msgr2=0x7f94e40c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94dbfff700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94e40cc410 0x7f94e40c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:41.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94e2d9d700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94e40cc410 0x7f94e40c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94e2d9d700 1 -- 172.21.15.60:0/3107801555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4070af0 msgr2=0x7f94e40b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94e2d9d700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4070af0 0x7f94e40b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94e2d9d700 1 -- 172.21.15.60:0/3107801555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc00bd40 con 0x7f94e4159260 2023-07-13T22:27:41.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94dbfff700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94e40cc410 0x7f94e40c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:41.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94e2d9d700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4159260 0x7f94e40ae160 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f94e406c200 tx=0x7f94dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:41.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94daffd700 1 -- 172.21.15.60:0/3107801555 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc009a10 con 0x7f94e4159260 2023-07-13T22:27:41.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94daffd700 1 -- 172.21.15.60:0/3107801555 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc009b90 con 0x7f94e4159260 2023-07-13T22:27:41.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94daffd700 1 -- 172.21.15.60:0/3107801555 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc004040 con 0x7f94e4159260 2023-07-13T22:27:41.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e40b35d0 con 0x7f94e4159260 2023-07-13T22:27:41.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.321+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e40b3b90 con 0x7f94e4159260 2023-07-13T22:27:41.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.322+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e40ca0e0 con 0x7f94e4159260 2023-07-13T22:27:41.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.323+0000 7f94daffd700 1 -- 172.21.15.60:0/3107801555 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f94dc0041a0 con 0x7f94e4159260 2023-07-13T22:27:41.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.324+0000 7f94daffd700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f94c406ace0 0x7f94c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.324+0000 7f94daffd700 1 -- 172.21.15.60:0/3107801555 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94dc08aaf0 con 0x7f94e4159260 2023-07-13T22:27:41.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.325+0000 7f94e259c700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f94c406ace0 0x7f94c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:41.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.325+0000 7f94e259c700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f94c406ace0 0x7f94c406d190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009ad0 tx=0x7f94d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:41.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.329+0000 7f94daffd700 1 -- 172.21.15.60:0/3107801555 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f94e40ca0e0 con 0x7f94e4159260 2023-07-13T22:27:41.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.537+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f94e40ca0e0 con 0x7f94e4159260 2023-07-13T22:27:41.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.537+0000 7f94daffd700 1 -- 172.21.15.60:0/3107801555 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7f94e40ca0e0 con 0x7f94e4159260 2023-07-13T22:27:41.519 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:27:41.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f94c406ace0 msgr2=0x7f94c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f94c406ace0 0x7f94c406d190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009ad0 tx=0x7f94d4008040 comp rx=0 tx=0).stop 2023-07-13T22:27:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4159260 msgr2=0x7f94e40ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4159260 0x7f94e40ae160 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f94e406c200 tx=0x7f94dc004ad0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 shutdown_connections 2023-07-13T22:27:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f94c406ace0 0x7f94c406d190 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94e4159260 0x7f94e40ae160 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94e4070af0 0x7f94e40b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 --2- 172.21.15.60:0/3107801555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94e40cc410 0x7f94e40c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 >> 172.21.15.60:0/3107801555 conn(0x7f94e40cbf50 msgr2=0x7f94e406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:41.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 shutdown_connections 2023-07-13T22:27:41.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.540+0000 7f94e8bff700 1 -- 172.21.15.60:0/3107801555 wait complete. 2023-07-13T22:27:41.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:27:41.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.772+0000 7fb4f83ff700 1 Processor -- start 2023-07-13T22:27:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.773+0000 7fb4f83ff700 1 -- start start 2023-07-13T22:27:41.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.774+0000 7fb4f83ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb4f00cfd50 0x7fb4f00d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.774+0000 7fb4f83ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f00d1870 con 0x7fb4f00b8200 2023-07-13T22:27:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.774+0000 7fb4f83ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f00cf370 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.774+0000 7fb4f83ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f00d12b0 con 0x7fb4f00cfd50 2023-07-13T22:27:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.774+0000 7fb4f599a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb4f00cfd50 0x7fb4f00d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.774+0000 7fb4f619b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb4f00b8200 0x7fb4f00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58584/0 (socket says 172.21.15.60:58584) 2023-07-13T22:27:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.774+0000 7fb4f619b700 1 -- 172.21.15.60:0/3233345771 learned_addr learned my addr 172.21.15.60:0/3233345771 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:41.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.775+0000 7fb4f599a700 1 -- 172.21.15.60:0/3233345771 >> v1:172.21.15.105:6789/0 conn(0x7fb4f00b1ac0 legacy=0x7fb4f00d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.775+0000 7fb4f599a700 1 -- 172.21.15.60:0/3233345771 >> v1:172.21.15.60:6789/0 conn(0x7fb4f00b8200 legacy=0x7fb4f00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.775+0000 7fb4f599a700 1 -- 172.21.15.60:0/3233345771 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f00d1590 con 0x7fb4f00cfd50 2023-07-13T22:27:41.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.775+0000 7fb4f599a700 1 --2- 172.21.15.60:0/3233345771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00cfd50 0x7fb4f00d23c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009840 tx=0x7fb4e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85f6fb9e1c5e833 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:41.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.775+0000 7fb4f5199700 1 -- 172.21.15.60:0/3233345771 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0011830 con 0x7fb4f00cfd50 2023-07-13T22:27:41.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.775+0000 7fb4f5199700 1 -- 172.21.15.60:0/3233345771 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e0011990 con 0x7fb4f00cfd50 2023-07-13T22:27:41.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.775+0000 7fb4f5199700 1 -- 172.21.15.60:0/3233345771 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0011d20 con 0x7fb4f00cfd50 2023-07-13T22:27:41.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.776+0000 7fb4f83ff700 1 -- 172.21.15.60:0/3233345771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00cfd50 msgr2=0x7fb4f00d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.776+0000 7fb4f83ff700 1 --2- 172.21.15.60:0/3233345771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00cfd50 0x7fb4f00d23c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009840 tx=0x7fb4e0009b50 comp rx=0 tx=0).stop 2023-07-13T22:27:41.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.776+0000 7fb4f83ff700 1 -- 172.21.15.60:0/3233345771 shutdown_connections 2023-07-13T22:27:41.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.776+0000 7fb4f83ff700 1 --2- 172.21.15.60:0/3233345771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00cfd50 0x7fb4f00d23c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.776+0000 7fb4f83ff700 1 -- 172.21.15.60:0/3233345771 >> 172.21.15.60:0/3233345771 conn(0x7fb4f006b2f0 msgr2=0x7fb4f006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:41.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.776+0000 7fb4f83ff700 1 -- 172.21.15.60:0/3233345771 shutdown_connections 2023-07-13T22:27:41.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.776+0000 7fb4f83ff700 1 -- 172.21.15.60:0/3233345771 wait complete. 2023-07-13T22:27:41.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.778+0000 7fb4f83ff700 1 Processor -- start 2023-07-13T22:27:41.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.778+0000 7fb4f83ff700 1 -- start start 2023-07-13T22:27:41.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f83ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00b1ac0 0x7fb4f0149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f83ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4f00b8200 0x7fb4f00ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f83ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f00cfd50 0x7fb4f01f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f83ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f01f4730 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f83ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f00ca4e0 con 0x7fb4f00cfd50 2023-07-13T22:27:41.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f83ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f00ca200 con 0x7fb4f00b8200 2023-07-13T22:27:41.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f619b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00b1ac0 0x7fb4f0149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:41.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f619b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00b1ac0 0x7fb4f0149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56714/0 (socket says 172.21.15.60:56714) 2023-07-13T22:27:41.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f619b700 1 -- 172.21.15.60:0/2189489604 learned_addr learned my addr 172.21.15.60:0/2189489604 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:41.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f619b700 1 -- 172.21.15.60:0/2189489604 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4f00b8200 msgr2=0x7fb4f00ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f619b700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4f00b8200 0x7fb4f00ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f619b700 1 -- 172.21.15.60:0/2189489604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f00cfd50 msgr2=0x7fb4f01f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f619b700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f00cfd50 0x7fb4f01f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.779+0000 7fb4f619b700 1 -- 172.21.15.60:0/2189489604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e00094a0 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.780+0000 7fb4f619b700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00b1ac0 0x7fb4f0149250 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c520 tx=0x7fb4ec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.780+0000 7fb4e6ffd700 1 -- 172.21.15.60:0/2189489604 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec005d70 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.780+0000 7fb4e6ffd700 1 -- 172.21.15.60:0/2189489604 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0070d0 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.780+0000 7fb4e6ffd700 1 -- 172.21.15.60:0/2189489604 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec007440 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.780+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f01f4070 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.780+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f01564c0 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.781+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f01f4870 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.782+0000 7fb4e6ffd700 1 -- 172.21.15.60:0/2189489604 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb4ec0075a0 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.783+0000 7fb4e6ffd700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb4d406ace0 0x7fb4d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:41.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.783+0000 7fb4e6ffd700 1 -- 172.21.15.60:0/2189489604 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4ec0888c0 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.783+0000 7fb4f599a700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb4d406ace0 0x7fb4d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:41.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.784+0000 7fb4f599a700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb4d406ace0 0x7fb4d406d190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00d0140 tx=0x7fb4e0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:41.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.787+0000 7fb4e6ffd700 1 -- 172.21.15.60:0/2189489604 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb4f01f4870 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.995+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4f01f4870 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.995+0000 7fb4e6ffd700 1 -- 172.21.15.60:0/2189489604 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4f01f4870 con 0x7fb4f00b1ac0 2023-07-13T22:27:41.976 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:27:41.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb4d406ace0 msgr2=0x7fb4d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb4d406ace0 0x7fb4d406d190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00d0140 tx=0x7fb4e0010000 comp rx=0 tx=0).stop 2023-07-13T22:27:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00b1ac0 msgr2=0x7fb4f0149250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00b1ac0 0x7fb4f0149250 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c520 tx=0x7fb4ec00c8e0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 shutdown_connections 2023-07-13T22:27:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb4d406ace0 0x7fb4d406d190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4f00b1ac0 0x7fb4f0149250 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f00cfd50 0x7fb4f01f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 --2- 172.21.15.60:0/2189489604 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4f00b8200 0x7fb4f00ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 >> 172.21.15.60:0/2189489604 conn(0x7fb4f006b2f0 msgr2=0x7fb4f00d2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 shutdown_connections 2023-07-13T22:27:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:41.998+0000 7fb4f83ff700 1 -- 172.21.15.60:0/2189489604 wait complete. 2023-07-13T22:27:41.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:27:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.221+0000 7f9a8e5ff700 1 Processor -- start 2023-07-13T22:27:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.221+0000 7f9a8e5ff700 1 -- start start 2023-07-13T22:27:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.222+0000 7f9a8e5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a880cfd50 0x7f9a880d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.222+0000 7f9a8e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880b1ac0 0x7f9a880d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.222+0000 7f9a8e5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a880d1870 con 0x7f9a880b8200 2023-07-13T22:27:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.222+0000 7f9a8e5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a880cf370 con 0x7f9a880cfd50 2023-07-13T22:27:42.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.222+0000 7f9a8e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880d12b0 con 0x7f9a880b1ac0 2023-07-13T22:27:42.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.223+0000 7f9a877fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a880cfd50 0x7f9a880d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.223+0000 7f9a877fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a880cfd50 0x7f9a880d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56726/0 (socket says 172.21.15.60:56726) 2023-07-13T22:27:42.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.223+0000 7f9a877fe700 1 -- 172.21.15.60:0/4015583203 learned_addr learned my addr 172.21.15.60:0/4015583203 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:42.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.223+0000 7f9a8cb9c700 1 --2- 172.21.15.60:0/4015583203 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880b1ac0 0x7f9a880d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.223+0000 7f9a877fe700 1 -- 172.21.15.60:0/4015583203 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880b1ac0 msgr2=0x7f9a880d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.223+0000 7f9a877fe700 1 --2- 172.21.15.60:0/4015583203 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880b1ac0 0x7f9a880d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.223+0000 7f9a877fe700 1 -- 172.21.15.60:0/4015583203 >> v1:172.21.15.105:6789/0 conn(0x7f9a880b8200 legacy=0x7f9a880cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.223+0000 7f9a877fe700 1 -- 172.21.15.60:0/4015583203 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a880d1590 con 0x7f9a880cfd50 2023-07-13T22:27:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a877fe700 1 --2- 172.21.15.60:0/4015583203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880cfd50 0x7f9a880d23c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0098f0 tx=0x7f9a7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f0bae18739cf07 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a86ffd700 1 -- 172.21.15.60:0/4015583203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00b8a0 con 0x7f9a880cfd50 2023-07-13T22:27:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a86ffd700 1 -- 172.21.15.60:0/4015583203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c00ba00 con 0x7f9a880cfd50 2023-07-13T22:27:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a86ffd700 1 -- 172.21.15.60:0/4015583203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00bd90 con 0x7f9a880cfd50 2023-07-13T22:27:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/4015583203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880cfd50 msgr2=0x7f9a880d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a8e5ff700 1 --2- 172.21.15.60:0/4015583203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880cfd50 0x7f9a880d23c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0098f0 tx=0x7f9a7c009c00 comp rx=0 tx=0).stop 2023-07-13T22:27:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/4015583203 shutdown_connections 2023-07-13T22:27:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a8e5ff700 1 --2- 172.21.15.60:0/4015583203 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880b1ac0 0x7f9a880d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a8e5ff700 1 --2- 172.21.15.60:0/4015583203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880cfd50 0x7f9a880d23c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/4015583203 >> 172.21.15.60:0/4015583203 conn(0x7f9a8806b2f0 msgr2=0x7f9a8806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.224+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/4015583203 shutdown_connections 2023-07-13T22:27:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.225+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/4015583203 wait complete. 2023-07-13T22:27:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.226+0000 7f9a8e5ff700 1 Processor -- start 2023-07-13T22:27:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.226+0000 7f9a8e5ff700 1 -- start start 2023-07-13T22:27:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.227+0000 7f9a8e5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a880b1ac0 0x7f9a880c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.227+0000 7f9a8e5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880b8200 0x7f9a8815e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.227+0000 7f9a8e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880cfd50 0x7f9a8815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.227+0000 7f9a8e5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880c6ca0 con 0x7f9a880b8200 2023-07-13T22:27:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.227+0000 7f9a8e5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880c6f70 con 0x7f9a880b1ac0 2023-07-13T22:27:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.227+0000 7f9a8e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880c7250 con 0x7f9a880cfd50 2023-07-13T22:27:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.227+0000 7f9a877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880b8200 0x7f9a8815e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880b8200 0x7f9a8815e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56742/0 (socket says 172.21.15.60:56742) 2023-07-13T22:27:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a877fe700 1 -- 172.21.15.60:0/2282169966 learned_addr learned my addr 172.21.15.60:0/2282169966 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a87fff700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a880b1ac0 0x7f9a880c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a8cb9c700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880cfd50 0x7f9a8815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a877fe700 1 -- 172.21.15.60:0/2282169966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880cfd50 msgr2=0x7f9a8815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a877fe700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880cfd50 0x7f9a8815ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a877fe700 1 -- 172.21.15.60:0/2282169966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a880b1ac0 msgr2=0x7f9a880c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a877fe700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a880b1ac0 0x7f9a880c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a877fe700 1 -- 172.21.15.60:0/2282169966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c009550 con 0x7f9a880b8200 2023-07-13T22:27:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a8cb9c700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880cfd50 0x7f9a8815ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a877fe700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880b8200 0x7f9a8815e9a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c014040 tx=0x7f9a7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.228+0000 7f9a84ff9700 1 -- 172.21.15.60:0/2282169966 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c004380 con 0x7f9a880b8200 2023-07-13T22:27:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.229+0000 7f9a84ff9700 1 -- 172.21.15.60:0/2282169966 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c01f070 con 0x7f9a880b8200 2023-07-13T22:27:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.229+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a880c7530 con 0x7f9a880b8200 2023-07-13T22:27:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.229+0000 7f9a84ff9700 1 -- 172.21.15.60:0/2282169966 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c0036a0 con 0x7f9a880b8200 2023-07-13T22:27:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.229+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a880c7af0 con 0x7f9a880b8200 2023-07-13T22:27:42.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.230+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a880c7ce0 con 0x7f9a880b8200 2023-07-13T22:27:42.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.231+0000 7f9a84ff9700 1 -- 172.21.15.60:0/2282169966 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a7c021030 con 0x7f9a880b8200 2023-07-13T22:27:42.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.232+0000 7f9a84ff9700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7406ace0 0x7f9a7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.232+0000 7f9a84ff9700 1 -- 172.21.15.60:0/2282169966 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a7c08a970 con 0x7f9a880b8200 2023-07-13T22:27:42.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.232+0000 7f9a87fff700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7406ace0 0x7f9a7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.233+0000 7f9a87fff700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7406ace0 0x7f9a7406d190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9a78007fd0 tx=0x7f9a78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:42.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.236+0000 7f9a84ff9700 1 -- 172.21.15.60:0/2282169966 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9a7c055b30 con 0x7f9a880b8200 2023-07-13T22:27:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.442+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a8804e210 con 0x7f9a880b8200 2023-07-13T22:27:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.443+0000 7f9a84ff9700 1 -- 172.21.15.60:0/2282169966 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a7c059150 con 0x7f9a880b8200 2023-07-13T22:27:42.423 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:27:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7406ace0 msgr2=0x7f9a7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7406ace0 0x7f9a7406d190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9a78007fd0 tx=0x7f9a78009040 comp rx=0 tx=0).stop 2023-07-13T22:27:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880b8200 msgr2=0x7f9a8815e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880b8200 0x7f9a8815e9a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c014040 tx=0x7f9a7c004750 comp rx=0 tx=0).stop 2023-07-13T22:27:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 shutdown_connections 2023-07-13T22:27:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a7406ace0 0x7f9a7406d190 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a880cfd50 0x7f9a8815ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a880b1ac0 0x7f9a880c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 --2- 172.21.15.60:0/2282169966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a880b8200 0x7f9a8815e9a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 >> 172.21.15.60:0/2282169966 conn(0x7f9a8806b2f0 msgr2=0x7f9a880cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 shutdown_connections 2023-07-13T22:27:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.445+0000 7f9a8e5ff700 1 -- 172.21.15.60:0/2282169966 wait complete. 2023-07-13T22:27:42.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:27:42.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.670+0000 7fa579bff700 1 Processor -- start 2023-07-13T22:27:42.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.670+0000 7fa579bff700 1 -- start start 2023-07-13T22:27:42.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.672+0000 7fa579bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5740b1ac0 0x7fa5740d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.672+0000 7fa579bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740d1870 con 0x7fa5740cfd50 2023-07-13T22:27:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.672+0000 7fa579bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740cf370 con 0x7fa5740b8200 2023-07-13T22:27:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.672+0000 7fa579bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5740d12b0 con 0x7fa5740b1ac0 2023-07-13T22:27:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.672+0000 7fa572ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa5740cfd50 0x7fa5740d23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58598/0 (socket says 172.21.15.60:58598) 2023-07-13T22:27:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.672+0000 7fa572ffd700 1 -- 172.21.15.60:0/1672771699 learned_addr learned my addr 172.21.15.60:0/1672771699 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.672+0000 7fa573fff700 1 --2- 172.21.15.60:0/1672771699 >> v2:172.21.15.60:3300/0 conn(0x7fa5740b1ac0 0x7fa5740d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.673+0000 7fa5727fc700 1 -- 172.21.15.60:0/1672771699 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240238010 0 0) 0x7fa5740d1870 con 0x7fa5740cfd50 2023-07-13T22:27:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.673+0000 7fa5727fc700 1 -- 172.21.15.60:0/1672771699 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa55c0035b0 con 0x7fa5740cfd50 2023-07-13T22:27:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.673+0000 7fa573fff700 1 -- 172.21.15.60:0/1672771699 >> v1:172.21.15.105:6789/0 conn(0x7fa5740b8200 legacy=0x7fa5740cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.673+0000 7fa573fff700 1 -- 172.21.15.60:0/1672771699 >> v1:172.21.15.60:6789/0 conn(0x7fa5740cfd50 legacy=0x7fa5740d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.673+0000 7fa573fff700 1 -- 172.21.15.60:0/1672771699 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740d1590 con 0x7fa5740b1ac0 2023-07-13T22:27:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.673+0000 7fa573fff700 1 --2- 172.21.15.60:0/1672771699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740b1ac0 0x7fa5740d2ad0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009d90 tx=0x7fa56c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debe37a8397ac52d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.673+0000 7fa5727fc700 1 -- 172.21.15.60:0/1672771699 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00e070 con 0x7fa5740b1ac0 2023-07-13T22:27:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.673+0000 7fa5727fc700 1 -- 172.21.15.60:0/1672771699 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c00bb20 con 0x7fa5740b1ac0 2023-07-13T22:27:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.673+0000 7fa5727fc700 1 -- 172.21.15.60:0/1672771699 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c012430 con 0x7fa5740b1ac0 2023-07-13T22:27:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.674+0000 7fa579bff700 1 -- 172.21.15.60:0/1672771699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740b1ac0 msgr2=0x7fa5740d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.674+0000 7fa579bff700 1 --2- 172.21.15.60:0/1672771699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740b1ac0 0x7fa5740d2ad0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009d90 tx=0x7fa56c00b330 comp rx=0 tx=0).stop 2023-07-13T22:27:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.674+0000 7fa579bff700 1 -- 172.21.15.60:0/1672771699 shutdown_connections 2023-07-13T22:27:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.674+0000 7fa579bff700 1 --2- 172.21.15.60:0/1672771699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740b1ac0 0x7fa5740d2ad0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.674+0000 7fa579bff700 1 -- 172.21.15.60:0/1672771699 >> 172.21.15.60:0/1672771699 conn(0x7fa57406b2f0 msgr2=0x7fa57406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.674+0000 7fa579bff700 1 -- 172.21.15.60:0/1672771699 shutdown_connections 2023-07-13T22:27:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.674+0000 7fa579bff700 1 -- 172.21.15.60:0/1672771699 wait complete. 2023-07-13T22:27:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.676+0000 7fa579bff700 1 Processor -- start 2023-07-13T22:27:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.677+0000 7fa579bff700 1 -- start start 2023-07-13T22:27:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa579bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5740b1ac0 0x7fa5741e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa579bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5740b8200 0x7fa5740b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa579bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740cfd50 0x7fa5741e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa579bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa574156940 con 0x7fa5740cfd50 2023-07-13T22:27:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa579bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741e6190 con 0x7fa5740b1ac0 2023-07-13T22:27:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa579bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740b6ed0 con 0x7fa5740b8200 2023-07-13T22:27:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa573fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740cfd50 0x7fa5741e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa573fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740cfd50 0x7fa5741e6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56752/0 (socket says 172.21.15.60:56752) 2023-07-13T22:27:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa573fff700 1 -- 172.21.15.60:0/151220428 learned_addr learned my addr 172.21.15.60:0/151220428 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa5737fe700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5740b1ac0 0x7fa5741e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.678+0000 7fa572ffd700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5740b8200 0x7fa5740b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa573fff700 1 -- 172.21.15.60:0/151220428 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5740b8200 msgr2=0x7fa5740b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa573fff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5740b8200 0x7fa5740b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa573fff700 1 -- 172.21.15.60:0/151220428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5740b1ac0 msgr2=0x7fa5741e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa573fff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5740b1ac0 0x7fa5741e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa573fff700 1 -- 172.21.15.60:0/151220428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c009a70 con 0x7fa5740cfd50 2023-07-13T22:27:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa573fff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740cfd50 0x7fa5741e6970 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa57406bf10 tx=0x7fa56c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa55bfff700 1 -- 172.21.15.60:0/151220428 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c012430 con 0x7fa5740cfd50 2023-07-13T22:27:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa55bfff700 1 -- 172.21.15.60:0/151220428 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c020070 con 0x7fa5740cfd50 2023-07-13T22:27:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5741e6eb0 con 0x7fa5740cfd50 2023-07-13T22:27:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa55bfff700 1 -- 172.21.15.60:0/151220428 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c004280 con 0x7fa5740cfd50 2023-07-13T22:27:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.679+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5740b7440 con 0x7fa5740cfd50 2023-07-13T22:27:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.681+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57404e210 con 0x7fa5740cfd50 2023-07-13T22:27:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.682+0000 7fa55bfff700 1 -- 172.21.15.60:0/151220428 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa56c0043e0 con 0x7fa5740cfd50 2023-07-13T22:27:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.682+0000 7fa55bfff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa56006ace0 0x7fa56006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.683+0000 7fa55bfff700 1 -- 172.21.15.60:0/151220428 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa56c022070 con 0x7fa5740cfd50 2023-07-13T22:27:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.683+0000 7fa5737fe700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa56006ace0 0x7fa56006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.683+0000 7fa5737fe700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa56006ace0 0x7fa56006d190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa564007fd0 tx=0x7fa564009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:42.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.687+0000 7fa55bfff700 1 -- 172.21.15.60:0/151220428 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa56c056140 con 0x7fa5740cfd50 2023-07-13T22:27:42.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.893+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5740b4f70 con 0x7fa5740cfd50 2023-07-13T22:27:42.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.893+0000 7fa55bfff700 1 -- 172.21.15.60:0/151220428 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa56c02c0a0 con 0x7fa5740cfd50 2023-07-13T22:27:42.874 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:27:42.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa56006ace0 msgr2=0x7fa56006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa56006ace0 0x7fa56006d190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa564007fd0 tx=0x7fa564009040 comp rx=0 tx=0).stop 2023-07-13T22:27:42.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740cfd50 msgr2=0x7fa5741e6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:42.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740cfd50 0x7fa5741e6970 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa57406bf10 tx=0x7fa56c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 shutdown_connections 2023-07-13T22:27:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa56006ace0 0x7fa56006d190 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5740cfd50 0x7fa5741e6970 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5740b1ac0 0x7fa5741e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 --2- 172.21.15.60:0/151220428 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5740b8200 0x7fa5740b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 >> 172.21.15.60:0/151220428 conn(0x7fa57406b2f0 msgr2=0x7fa5740c8770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 shutdown_connections 2023-07-13T22:27:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:42.896+0000 7fa579bff700 1 -- 172.21.15.60:0/151220428 wait complete. 2023-07-13T22:27:42.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:27:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.118+0000 7f9a68fff700 1 Processor -- start 2023-07-13T22:27:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.119+0000 7f9a68fff700 1 -- start start 2023-07-13T22:27:43.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.120+0000 7f9a68fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a640d3750 0x7f9a640cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.120+0000 7f9a68fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a640cfad0 0x7f9a640d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.120+0000 7f9a68fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a640cd950 con 0x7f9a640b8400 2023-07-13T22:27:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.120+0000 7f9a68fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a640cff50 con 0x7f9a640cfad0 2023-07-13T22:27:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.120+0000 7f9a68fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a640d0ac0 con 0x7f9a640d3750 2023-07-13T22:27:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.120+0000 7f9a62d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a640cfad0 0x7f9a640d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.120+0000 7f9a62d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a640cfad0 0x7f9a640d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56758/0 (socket says 172.21.15.60:56758) 2023-07-13T22:27:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.120+0000 7f9a62d9d700 1 -- 172.21.15.60:0/2243221146 learned_addr learned my addr 172.21.15.60:0/2243221146 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.120+0000 7f9a5bfff700 1 --2- 172.21.15.60:0/2243221146 >> v2:172.21.15.105:3300/0 conn(0x7f9a640d3750 0x7f9a640cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.121+0000 7f9a62d9d700 1 -- 172.21.15.60:0/2243221146 >> v2:172.21.15.105:3300/0 conn(0x7f9a640d3750 msgr2=0x7f9a640cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.121+0000 7f9a62d9d700 1 --2- 172.21.15.60:0/2243221146 >> v2:172.21.15.105:3300/0 conn(0x7f9a640d3750 0x7f9a640cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.121+0000 7f9a62d9d700 1 -- 172.21.15.60:0/2243221146 >> v1:172.21.15.105:6789/0 conn(0x7f9a640b8400 legacy=0x7f9a640cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.121+0000 7f9a62d9d700 1 -- 172.21.15.60:0/2243221146 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640b87d0 con 0x7f9a640cfad0 2023-07-13T22:27:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.121+0000 7f9a62d9d700 1 --2- 172.21.15.60:0/2243221146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 0x7f9a640d0f50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9a640b8cf0 tx=0x7f9a5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a3a3ed0a325e8d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.121+0000 7f9a61d9b700 1 -- 172.21.15.60:0/2243221146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c00e070 con 0x7f9a640cfad0 2023-07-13T22:27:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.121+0000 7f9a61d9b700 1 -- 172.21.15.60:0/2243221146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c00b910 con 0x7f9a640cfad0 2023-07-13T22:27:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.121+0000 7f9a61d9b700 1 -- 172.21.15.60:0/2243221146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c00bcc0 con 0x7f9a640cfad0 2023-07-13T22:27:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.122+0000 7f9a68fff700 1 -- 172.21.15.60:0/2243221146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 msgr2=0x7f9a640d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.122+0000 7f9a68fff700 1 --2- 172.21.15.60:0/2243221146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 0x7f9a640d0f50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9a640b8cf0 tx=0x7f9a5c00b330 comp rx=0 tx=0).stop 2023-07-13T22:27:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.122+0000 7f9a68fff700 1 -- 172.21.15.60:0/2243221146 shutdown_connections 2023-07-13T22:27:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.122+0000 7f9a68fff700 1 --2- 172.21.15.60:0/2243221146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 0x7f9a640d0f50 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.122+0000 7f9a68fff700 1 --2- 172.21.15.60:0/2243221146 >> v2:172.21.15.105:3300/0 conn(0x7f9a640d3750 0x7f9a640cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.122+0000 7f9a68fff700 1 -- 172.21.15.60:0/2243221146 >> 172.21.15.60:0/2243221146 conn(0x7f9a6406b390 msgr2=0x7f9a6406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.122+0000 7f9a68fff700 1 -- 172.21.15.60:0/2243221146 shutdown_connections 2023-07-13T22:27:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.122+0000 7f9a68fff700 1 -- 172.21.15.60:0/2243221146 wait complete. 2023-07-13T22:27:43.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.124+0000 7f9a68fff700 1 Processor -- start 2023-07-13T22:27:43.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.124+0000 7f9a68fff700 1 -- start start 2023-07-13T22:27:43.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a68fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a640b8400 0x7f9a641e83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:43.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a68fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 0x7f9a640b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a68fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a640d3750 0x7f9a640b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a68fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a64150b60 con 0x7f9a640cfad0 2023-07-13T22:27:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a68fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6415d2a0 con 0x7f9a640d3750 2023-07-13T22:27:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a68fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6415eb30 con 0x7f9a640b8400 2023-07-13T22:27:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a5bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 0x7f9a640b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a5bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 0x7f9a640b6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56774/0 (socket says 172.21.15.60:56774) 2023-07-13T22:27:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a5bfff700 1 -- 172.21.15.60:0/4010365061 learned_addr learned my addr 172.21.15.60:0/4010365061 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a62d9d700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a640d3750 0x7f9a640b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a6259c700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a640b8400 0x7f9a641e83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a5bfff700 1 -- 172.21.15.60:0/4010365061 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a640b8400 msgr2=0x7f9a641e83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a5bfff700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a640b8400 0x7f9a641e83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a5bfff700 1 -- 172.21.15.60:0/4010365061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a640d3750 msgr2=0x7f9a640b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a5bfff700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a640d3750 0x7f9a640b7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.125+0000 7f9a5bfff700 1 -- 172.21.15.60:0/4010365061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c009b30 con 0x7f9a640cfad0 2023-07-13T22:27:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.126+0000 7f9a5bfff700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 0x7f9a640b6f10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9b0 tx=0x7f9a4c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.126+0000 7f9a5affd700 1 -- 172.21.15.60:0/4010365061 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0096d0 con 0x7f9a640cfad0 2023-07-13T22:27:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.126+0000 7f9a5affd700 1 -- 172.21.15.60:0/4010365061 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c00ee50 con 0x7f9a640cfad0 2023-07-13T22:27:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.126+0000 7f9a5affd700 1 -- 172.21.15.60:0/4010365061 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00ca70 con 0x7f9a640cfad0 2023-07-13T22:27:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.126+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6415c970 con 0x7f9a640cfad0 2023-07-13T22:27:43.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.126+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6415cf30 con 0x7f9a640cfad0 2023-07-13T22:27:43.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.127+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6415d430 con 0x7f9a640cfad0 2023-07-13T22:27:43.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.128+0000 7f9a5affd700 1 -- 172.21.15.60:0/4010365061 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a4c00cbd0 con 0x7f9a640cfad0 2023-07-13T22:27:43.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.129+0000 7f9a5affd700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a4406ace0 0x7f9a4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:43.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.129+0000 7f9a5affd700 1 -- 172.21.15.60:0/4010365061 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a4c089490 con 0x7f9a640cfad0 2023-07-13T22:27:43.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.129+0000 7f9a6259c700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a4406ace0 0x7f9a4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:43.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.130+0000 7f9a6259c700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a4406ace0 0x7f9a4406d190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9a54007fd0 tx=0x7f9a54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:43.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.133+0000 7f9a5affd700 1 -- 172.21.15.60:0/4010365061 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9a4c054530 con 0x7f9a640cfad0 2023-07-13T22:27:43.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.354+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9a640b42d0 con 0x7f9a640cfad0 2023-07-13T22:27:43.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.354+0000 7f9a5affd700 1 -- 172.21.15.60:0/4010365061 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9a4c057b50 con 0x7f9a640cfad0 2023-07-13T22:27:43.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a4406ace0 msgr2=0x7f9a4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:43.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a4406ace0 0x7f9a4406d190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9a54007fd0 tx=0x7f9a54009040 comp rx=0 tx=0).stop 2023-07-13T22:27:43.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 msgr2=0x7f9a640b6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:43.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 0x7f9a640b6f10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9b0 tx=0x7f9a4c00ecc0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 shutdown_connections 2023-07-13T22:27:43.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9a4406ace0 0x7f9a4406d190 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a640d3750 0x7f9a640b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a640b8400 0x7f9a641e83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 --2- 172.21.15.60:0/4010365061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a640cfad0 0x7f9a640b6f10 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:43.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 >> 172.21.15.60:0/4010365061 conn(0x7f9a6406b390 msgr2=0x7f9a640d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:43.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 shutdown_connections 2023-07-13T22:27:43.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:43.357+0000 7f9a68fff700 1 -- 172.21.15.60:0/4010365061 wait complete. 2023-07-13T22:27:43.339 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:27:43.355 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:27:43.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:27:43.356 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:27:43.430 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[123518]: fuse finished with error 0 and tester_r 0 2023-07-13T22:27:43.433 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:27:43.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:27:43.434 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:27:43.596 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:27:43.596 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:27:43.643 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:27:43.669 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:27:43.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:27:43.670 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:27:43.729 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-13T22:27:43.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:27:43.731 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:27:43.731 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:27:43.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:27:43.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:27:43.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:27:43.779 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:27:43.863 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-13T22:27:43.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:27:43.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:27:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.087+0000 7fb016bff700 1 Processor -- start 2023-07-13T22:27:44.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.087+0000 7fb016bff700 1 -- start start 2023-07-13T22:27:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.088+0000 7fb016bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0100d35c0 0x7fb0100d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.088+0000 7fb016bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d13f0 0x7fb01015c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.088+0000 7fb016bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0100d09d0 con 0x7fb0100b8570 2023-07-13T22:27:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.088+0000 7fb016bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0100d3b30 con 0x7fb0100d35c0 2023-07-13T22:27:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.088+0000 7fb016bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100d1870 con 0x7fb0100d13f0 2023-07-13T22:27:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb00ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0100d35c0 0x7fb0100d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb00ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0100d35c0 0x7fb0100d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56780/0 (socket says 172.21.15.60:56780) 2023-07-13T22:27:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb00ffff700 1 -- 172.21.15.60:0/3176533138 learned_addr learned my addr 172.21.15.60:0/3176533138 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb01499b700 1 --1- 172.21.15.60:0/3176533138 >> v1:172.21.15.60:6789/0 conn(0x7fb0100b8570 0x7fb0100ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58602/0 (socket says 172.21.15.60:58602) 2023-07-13T22:27:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb01519c700 1 --2- 172.21.15.60:0/3176533138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d13f0 0x7fb01015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb00ffff700 1 -- 172.21.15.60:0/3176533138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d13f0 msgr2=0x7fb01015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb00ffff700 1 --2- 172.21.15.60:0/3176533138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d13f0 0x7fb01015c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb00ffff700 1 -- 172.21.15.60:0/3176533138 >> v1:172.21.15.60:6789/0 conn(0x7fb0100b8570 legacy=0x7fb0100ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb00ffff700 1 -- 172.21.15.60:0/3176533138 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100cda10 con 0x7fb0100d35c0 2023-07-13T22:27:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb00f7fe700 1 -- 172.21.15.60:0/3176533138 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063995755 0 0) 0x7fb0100d09d0 con 0x7fb0100b8570 2023-07-13T22:27:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.089+0000 7fb00ffff700 1 --2- 172.21.15.60:0/3176533138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d35c0 0x7fb0100d0eb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7faffc0098f0 tx=0x7faffc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91b2873678f96f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb00f7fe700 1 -- 172.21.15.60:0/3176533138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc011830 con 0x7fb0100d35c0 2023-07-13T22:27:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb00f7fe700 1 -- 172.21.15.60:0/3176533138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc011990 con 0x7fb0100d35c0 2023-07-13T22:27:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb00f7fe700 1 -- 172.21.15.60:0/3176533138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc011d20 con 0x7fb0100d35c0 2023-07-13T22:27:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb016bff700 1 -- 172.21.15.60:0/3176533138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d35c0 msgr2=0x7fb0100d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb016bff700 1 --2- 172.21.15.60:0/3176533138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d35c0 0x7fb0100d0eb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7faffc0098f0 tx=0x7faffc009c00 comp rx=0 tx=0).stop 2023-07-13T22:27:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb016bff700 1 -- 172.21.15.60:0/3176533138 shutdown_connections 2023-07-13T22:27:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb016bff700 1 --2- 172.21.15.60:0/3176533138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d13f0 0x7fb01015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb016bff700 1 --2- 172.21.15.60:0/3176533138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d35c0 0x7fb0100d0eb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb016bff700 1 -- 172.21.15.60:0/3176533138 >> 172.21.15.60:0/3176533138 conn(0x7fb010071600 msgr2=0x7fb0100719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb016bff700 1 -- 172.21.15.60:0/3176533138 shutdown_connections 2023-07-13T22:27:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.090+0000 7fb016bff700 1 -- 172.21.15.60:0/3176533138 wait complete. 2023-07-13T22:27:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.092+0000 7fb016bff700 1 Processor -- start 2023-07-13T22:27:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.092+0000 7fb016bff700 1 -- start start 2023-07-13T22:27:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.093+0000 7fb016bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0100b8570 0x7fb0101f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.093+0000 7fb016bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d13f0 0x7fb0100ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.093+0000 7fb016bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d35c0 0x7fb0100c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.093+0000 7fb016bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c98a0 con 0x7fb0100d13f0 2023-07-13T22:27:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.093+0000 7fb016bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0101f19c0 con 0x7fb0100b8570 2023-07-13T22:27:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.093+0000 7fb016bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c95c0 con 0x7fb0100d35c0 2023-07-13T22:27:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d13f0 0x7fb0100ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d13f0 0x7fb0100ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56790/0 (socket says 172.21.15.60:56790) 2023-07-13T22:27:44.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00ffff700 1 -- 172.21.15.60:0/1959477244 learned_addr learned my addr 172.21.15.60:0/1959477244 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:44.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb01499b700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0100b8570 0x7fb0101f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb01519c700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d35c0 0x7fb0100c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00ffff700 1 -- 172.21.15.60:0/1959477244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d35c0 msgr2=0x7fb0100c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00ffff700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d35c0 0x7fb0100c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00ffff700 1 -- 172.21.15.60:0/1959477244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0100b8570 msgr2=0x7fb0101f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00ffff700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0100b8570 0x7fb0101f1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00ffff700 1 -- 172.21.15.60:0/1959477244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc009550 con 0x7fb0100d13f0 2023-07-13T22:27:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb01499b700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0100b8570 0x7fb0101f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00ffff700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d13f0 0x7fb0100ca9b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7faffc014040 tx=0x7faffc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.094+0000 7fb00d7fa700 1 -- 172.21.15.60:0/1959477244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc004400 con 0x7fb0100d13f0 2023-07-13T22:27:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.095+0000 7fb00d7fa700 1 -- 172.21.15.60:0/1959477244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc01f070 con 0x7fb0100d13f0 2023-07-13T22:27:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.095+0000 7fb00d7fa700 1 -- 172.21.15.60:0/1959477244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0038a0 con 0x7fb0100d13f0 2023-07-13T22:27:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.095+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0100c9ed0 con 0x7fb0100d13f0 2023-07-13T22:27:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.095+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0100ca490 con 0x7fb0100d13f0 2023-07-13T22:27:44.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.096+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0100c9d10 con 0x7fb0100d13f0 2023-07-13T22:27:44.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.097+0000 7fb00d7fa700 1 -- 172.21.15.60:0/1959477244 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faffc021030 con 0x7fb0100d13f0 2023-07-13T22:27:44.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.098+0000 7fb00d7fa700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faff406ace0 0x7faff406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.098+0000 7fb00d7fa700 1 -- 172.21.15.60:0/1959477244 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faffc08a7e0 con 0x7fb0100d13f0 2023-07-13T22:27:44.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.098+0000 7fb01499b700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faff406ace0 0x7faff406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.099+0000 7fb01499b700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faff406ace0 0x7faff406d190 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb004001860 tx=0x7fb004009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:44.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.102+0000 7fb00d7fa700 1 -- 172.21.15.60:0/1959477244 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7faffc055920 con 0x7fb0100d13f0 2023-07-13T22:27:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.351+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb01004e280 con 0x7fb0100d13f0 2023-07-13T22:27:44.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.351+0000 7fb00d7fa700 1 -- 172.21.15.60:0/1959477244 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4718 (secure 0 0 0) 0x7faffc058f40 con 0x7fb0100d13f0 2023-07-13T22:27:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:44.332 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1603865496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1603865496},{"type":"v1","addr":"172.21.15.105:6834","nonce":1603865496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6735,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1777842000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1777842000},{"type":"v1","addr":"172.21.15.105:6835","nonce":1777842000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6738,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2613179178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2613179178},{"type":"v1","addr":"172.21.15.60:6836","nonce":2613179178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:25:02.392864+0000","modified":"2023-07-13T22:27:43.547746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":43,"addr":"172.21.15.60:6837/3617406711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3617406711},{"type":"v1","addr":"172.21.15.60:6837","nonce":3617406711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-13T22:27:44.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faff406ace0 msgr2=0x7faff406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faff406ace0 0x7faff406d190 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb004001860 tx=0x7fb004009040 comp rx=0 tx=0).stop 2023-07-13T22:27:44.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d13f0 msgr2=0x7fb0100ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d13f0 0x7fb0100ca9b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7faffc014040 tx=0x7faffc004750 comp rx=0 tx=0).stop 2023-07-13T22:27:44.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 shutdown_connections 2023-07-13T22:27:44.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faff406ace0 0x7faff406d190 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0100d35c0 0x7fb0100c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0100b8570 0x7fb0101f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 --2- 172.21.15.60:0/1959477244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0100d13f0 0x7fb0100ca9b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 >> 172.21.15.60:0/1959477244 conn(0x7fb010071600 msgr2=0x7fb010160570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:44.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 shutdown_connections 2023-07-13T22:27:44.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.354+0000 7fb016bff700 1 -- 172.21.15.60:0/1959477244 wait complete. 2023-07-13T22:27:44.337 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2023-07-13T22:27:44.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:27:44.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.578+0000 7f4dfdbff700 1 Processor -- start 2023-07-13T22:27:44.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.578+0000 7f4dfdbff700 1 -- start start 2023-07-13T22:27:44.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.579+0000 7f4dfdbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4df80d15c0 0x7f4df8167030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.579+0000 7f4dfdbff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df80d09d0 con 0x7f4df80b8570 2023-07-13T22:27:44.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.579+0000 7f4dfdbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df80cda10 con 0x7f4df80d35c0 2023-07-13T22:27:44.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.579+0000 7f4dfdbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df80d3b30 con 0x7f4df80d15c0 2023-07-13T22:27:44.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.579+0000 7f4df77fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4df80b8570 0x7f4df80ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58616/0 (socket says 172.21.15.60:58616) 2023-07-13T22:27:44.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.579+0000 7f4df77fe700 1 -- 172.21.15.60:0/3720717439 learned_addr learned my addr 172.21.15.60:0/3720717439 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:44.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.580+0000 7f4df7fff700 1 --2- 172.21.15.60:0/3720717439 >> v2:172.21.15.105:3300/0 conn(0x7f4df80d15c0 0x7f4df8167030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.580+0000 7f4df67fc700 1 -- 172.21.15.60:0/3720717439 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105544221 0 0) 0x7f4df80d09d0 con 0x7f4df80b8570 2023-07-13T22:27:44.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.580+0000 7f4df67fc700 1 -- 172.21.15.60:0/3720717439 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4de00035b0 con 0x7f4df80b8570 2023-07-13T22:27:44.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.580+0000 7f4df7fff700 1 -- 172.21.15.60:0/3720717439 >> v1:172.21.15.105:6789/0 conn(0x7f4df80d35c0 legacy=0x7f4df80d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.580+0000 7f4df67fc700 1 -- 172.21.15.60:0/3720717439 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2040228316 0 0) 0x7f4de00035b0 con 0x7f4df80b8570 2023-07-13T22:27:44.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.580+0000 7f4df7fff700 1 -- 172.21.15.60:0/3720717439 >> v1:172.21.15.60:6789/0 conn(0x7f4df80b8570 legacy=0x7f4df80ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.580+0000 7f4df7fff700 1 -- 172.21.15.60:0/3720717439 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80cd450 con 0x7f4df80d15c0 2023-07-13T22:27:44.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.580+0000 7f4df7fff700 1 --2- 172.21.15.60:0/3720717439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 0x7f4df8167030 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009d90 tx=0x7f4df000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7b77be8bf757a1d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:44.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4df67fc700 1 -- 172.21.15.60:0/3720717439 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df000e070 con 0x7f4df80d15c0 2023-07-13T22:27:44.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4df67fc700 1 -- 172.21.15.60:0/3720717439 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df0003a20 con 0x7f4df80d15c0 2023-07-13T22:27:44.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4df67fc700 1 -- 172.21.15.60:0/3720717439 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80017f0 con 0x7f4df80d15c0 2023-07-13T22:27:44.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4dfdbff700 1 -- 172.21.15.60:0/3720717439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 msgr2=0x7f4df8167030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4dfdbff700 1 --2- 172.21.15.60:0/3720717439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 0x7f4df8167030 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009d90 tx=0x7f4df000b330 comp rx=0 tx=0).stop 2023-07-13T22:27:44.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4dfdbff700 1 -- 172.21.15.60:0/3720717439 shutdown_connections 2023-07-13T22:27:44.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4dfdbff700 1 --2- 172.21.15.60:0/3720717439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 0x7f4df8167030 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4dfdbff700 1 -- 172.21.15.60:0/3720717439 >> 172.21.15.60:0/3720717439 conn(0x7f4df8071600 msgr2=0x7f4df80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:44.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4dfdbff700 1 -- 172.21.15.60:0/3720717439 shutdown_connections 2023-07-13T22:27:44.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.581+0000 7f4dfdbff700 1 -- 172.21.15.60:0/3720717439 wait complete. 2023-07-13T22:27:44.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.583+0000 7f4dfdbff700 1 Processor -- start 2023-07-13T22:27:44.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.583+0000 7f4dfdbff700 1 -- start start 2023-07-13T22:27:44.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4dfdbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df80b8570 0x7f4df814d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4dfdbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 0x7f4df80c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4dfdbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4df80d35c0 0x7f4df80c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4dfdbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80cbd10 con 0x7f4df80b8570 2023-07-13T22:27:44.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4dfdbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80cac20 con 0x7f4df80d15c0 2023-07-13T22:27:44.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4dfdbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80cb650 con 0x7f4df80d35c0 2023-07-13T22:27:44.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4df77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df80b8570 0x7f4df814d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4df77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df80b8570 0x7f4df814d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56804/0 (socket says 172.21.15.60:56804) 2023-07-13T22:27:44.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4df77fe700 1 -- 172.21.15.60:0/579504048 learned_addr learned my addr 172.21.15.60:0/579504048 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:44.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4df7fff700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4df80d35c0 0x7f4df80c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.584+0000 7f4df6ffd700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 0x7f4df80c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4df77fe700 1 -- 172.21.15.60:0/579504048 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4df80d35c0 msgr2=0x7f4df80c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4df77fe700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4df80d35c0 0x7f4df80c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4df77fe700 1 -- 172.21.15.60:0/579504048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 msgr2=0x7f4df80c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4df77fe700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 0x7f4df80c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4df77fe700 1 -- 172.21.15.60:0/579504048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df0009a70 con 0x7f4df80b8570 2023-07-13T22:27:44.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4df6ffd700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 0x7f4df80c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:44.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4df77fe700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df80b8570 0x7f4df814d130 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c750 tx=0x7f4de800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:44.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4ddffff700 1 -- 172.21.15.60:0/579504048 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80074e0 con 0x7f4df80b8570 2023-07-13T22:27:44.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4ddffff700 1 -- 172.21.15.60:0/579504048 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de8007660 con 0x7f4df80b8570 2023-07-13T22:27:44.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df80cb930 con 0x7f4df80b8570 2023-07-13T22:27:44.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4ddffff700 1 -- 172.21.15.60:0/579504048 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8005a70 con 0x7f4df80b8570 2023-07-13T22:27:44.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.585+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df814d670 con 0x7f4df80b8570 2023-07-13T22:27:44.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.586+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df80b7830 con 0x7f4df80b8570 2023-07-13T22:27:44.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.587+0000 7f4ddffff700 1 -- 172.21.15.60:0/579504048 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4de8005bd0 con 0x7f4df80b8570 2023-07-13T22:27:44.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.589+0000 7f4ddffff700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4de406ace0 0x7f4de406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:44.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.589+0000 7f4ddffff700 1 -- 172.21.15.60:0/579504048 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4de8088a00 con 0x7f4df80b8570 2023-07-13T22:27:44.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.589+0000 7f4df6ffd700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4de406ace0 0x7f4de406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:44.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.590+0000 7f4df6ffd700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4de406ace0 0x7f4de406d190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4dec000f60 tx=0x7f4dec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:44.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.593+0000 7f4ddffff700 1 -- 172.21.15.60:0/579504048 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4de8053aa0 con 0x7f4df80b8570 2023-07-13T22:27:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.803+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4df80b7550 con 0x7f4df80b8570 2023-07-13T22:27:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.804+0000 7f4ddffff700 1 -- 172.21.15.60:0/579504048 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18260 (secure 0 0 0) 0x7f4de8017020 con 0x7f4df80b8570 2023-07-13T22:27:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:44.786 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":75,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:25:03.885230+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:25:00.672661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-13T22:25:01.364844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:27:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4de406ace0 msgr2=0x7f4de406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4de406ace0 0x7f4de406d190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4dec000f60 tx=0x7f4dec009040 comp rx=0 tx=0).stop 2023-07-13T22:27:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df80b8570 msgr2=0x7f4df814d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df80b8570 0x7f4df814d130 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c750 tx=0x7f4de800ca60 comp rx=0 tx=0).stop 2023-07-13T22:27:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 shutdown_connections 2023-07-13T22:27:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4de406ace0 0x7f4de406d190 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4df80b8570 0x7f4df814d130 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4df80d35c0 0x7f4df80c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 --2- 172.21.15.60:0/579504048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4df80d15c0 0x7f4df80c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 >> 172.21.15.60:0/579504048 conn(0x7f4df8071600 msgr2=0x7f4df80d1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 shutdown_connections 2023-07-13T22:27:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:44.807+0000 7f4dfdbff700 1 -- 172.21.15.60:0/579504048 wait complete. 2023-07-13T22:27:44.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:25:00.672661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:25:01.364844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:27:44.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:27:45.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.032+0000 7f4d90fff700 1 Processor -- start 2023-07-13T22:27:45.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.032+0000 7f4d90fff700 1 -- start start 2023-07-13T22:27:45.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.033+0000 7f4d90fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.033+0000 7f4d90fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d8c0cd280 0x7f4d8c0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.033+0000 7f4d90fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8c0d2430 con 0x7f4d8c0d0e40 2023-07-13T22:27:45.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.033+0000 7f4d90fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8c15c920 con 0x7f4d8c0cd280 2023-07-13T22:27:45.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.033+0000 7f4d90fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c0d2680 con 0x7f4d8c0d0a70 2023-07-13T22:27:45.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d83fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d8c0cd280 0x7f4d8c0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d8a59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d8ad9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4d8c0d0e40 0x7f4d8c0b5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38268/0 (socket says 172.21.15.60:38268) 2023-07-13T22:27:45.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d83fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d8c0cd280 0x7f4d8c0d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49012/0 (socket says 172.21.15.60:49012) 2023-07-13T22:27:45.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d8ad9d700 1 -- 172.21.15.60:0/4206134733 learned_addr learned my addr 172.21.15.60:0/4206134733 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:45.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d83fff700 1 -- 172.21.15.60:0/4206134733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 msgr2=0x7f4d8c0cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:45.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d83fff700 1 --2- 172.21.15.60:0/4206134733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c0cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d83fff700 1 -- 172.21.15.60:0/4206134733 >> v1:172.21.15.60:6789/0 conn(0x7f4d8c0d0e40 legacy=0x7f4d8c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d89d9b700 1 -- 172.21.15.60:0/4206134733 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739554309 0 0) 0x7f4d8c0d2430 con 0x7f4d8c0d0e40 2023-07-13T22:27:45.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d83fff700 1 -- 172.21.15.60:0/4206134733 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c0d2960 con 0x7f4d8c0cd280 2023-07-13T22:27:45.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.034+0000 7f4d8a59c700 1 --2- 172.21.15.60:0/4206134733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c0cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:45.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d83fff700 1 --2- 172.21.15.60:0/4206134733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 0x7f4d8c0d3440 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009840 tx=0x7f4d74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7038c2c1fc134d6f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:45.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d89d9b700 1 -- 172.21.15.60:0/4206134733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74011830 con 0x7f4d8c0cd280 2023-07-13T22:27:45.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d89d9b700 1 -- 172.21.15.60:0/4206134733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d74011990 con 0x7f4d8c0cd280 2023-07-13T22:27:45.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d89d9b700 1 -- 172.21.15.60:0/4206134733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74011d20 con 0x7f4d8c0cd280 2023-07-13T22:27:45.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d90fff700 1 -- 172.21.15.60:0/4206134733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 msgr2=0x7f4d8c0d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d90fff700 1 --2- 172.21.15.60:0/4206134733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 0x7f4d8c0d3440 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009840 tx=0x7f4d74009b50 comp rx=0 tx=0).stop 2023-07-13T22:27:45.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d90fff700 1 -- 172.21.15.60:0/4206134733 shutdown_connections 2023-07-13T22:27:45.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d90fff700 1 --2- 172.21.15.60:0/4206134733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c0cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d90fff700 1 --2- 172.21.15.60:0/4206134733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 0x7f4d8c0d3440 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.035+0000 7f4d90fff700 1 -- 172.21.15.60:0/4206134733 >> 172.21.15.60:0/4206134733 conn(0x7f4d8c071600 msgr2=0x7f4d8c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:45.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.036+0000 7f4d90fff700 1 -- 172.21.15.60:0/4206134733 shutdown_connections 2023-07-13T22:27:45.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.036+0000 7f4d90fff700 1 -- 172.21.15.60:0/4206134733 wait complete. 2023-07-13T22:27:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.037+0000 7f4d90fff700 1 Processor -- start 2023-07-13T22:27:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.037+0000 7f4d90fff700 1 -- start start 2023-07-13T22:27:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.038+0000 7f4d90fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 0x7f4d8c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.038+0000 7f4d90fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.038+0000 7f4d90fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8c0d0e40 0x7f4d8c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.038+0000 7f4d90fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c0c7070 con 0x7f4d8c0cd280 2023-07-13T22:27:45.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d90fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c151640 con 0x7f4d8c0d0e40 2023-07-13T22:27:45.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d90fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c15d960 con 0x7f4d8c0d0a70 2023-07-13T22:27:45.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 0x7f4d8c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 0x7f4d8c0cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49026/0 (socket says 172.21.15.60:49026) 2023-07-13T22:27:45.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8a59c700 1 -- 172.21.15.60:0/2317870327 learned_addr learned my addr 172.21.15.60:0/2317870327 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:45.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d83fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8ad9d700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8c0d0e40 0x7f4d8c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8a59c700 1 -- 172.21.15.60:0/2317870327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 msgr2=0x7f4d8c151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8a59c700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8a59c700 1 -- 172.21.15.60:0/2317870327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8c0d0e40 msgr2=0x7f4d8c15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8a59c700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8c0d0e40 0x7f4d8c15d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8a59c700 1 -- 172.21.15.60:0/2317870327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c0094a0 con 0x7f4d8c0cd280 2023-07-13T22:27:45.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d83fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:45.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.039+0000 7f4d8a59c700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 0x7f4d8c0cbfa0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00ebe0 tx=0x7f4d7c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:45.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.040+0000 7f4d82ffd700 1 -- 172.21.15.60:0/2317870327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00c910 con 0x7f4d8c0cd280 2023-07-13T22:27:45.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.040+0000 7f4d82ffd700 1 -- 172.21.15.60:0/2317870327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c00ca70 con 0x7f4d8c0cd280 2023-07-13T22:27:45.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.040+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d740094a0 con 0x7f4d8c0cd280 2023-07-13T22:27:45.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.040+0000 7f4d82ffd700 1 -- 172.21.15.60:0/2317870327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00cde0 con 0x7f4d8c0cd280 2023-07-13T22:27:45.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.040+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8c15def0 con 0x7f4d8c0cd280 2023-07-13T22:27:45.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.041+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8c0c7380 con 0x7f4d8c0cd280 2023-07-13T22:27:45.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.042+0000 7f4d82ffd700 1 -- 172.21.15.60:0/2317870327 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d7c013470 con 0x7f4d8c0cd280 2023-07-13T22:27:45.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.043+0000 7f4d82ffd700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6c06ace0 0x7f4d6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.043+0000 7f4d82ffd700 1 -- 172.21.15.60:0/2317870327 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d7c0897c0 con 0x7f4d8c0cd280 2023-07-13T22:27:45.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.043+0000 7f4d83fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6c06ace0 0x7f4d6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.044+0000 7f4d83fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6c06ace0 0x7f4d6c06d190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4d74000c00 tx=0x7f4d74008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:45.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.047+0000 7f4d82ffd700 1 -- 172.21.15.60:0/2317870327 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4d7c0547d0 con 0x7f4d8c0cd280 2023-07-13T22:27:45.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.251+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d8c067a70 con 0x7f4d8c0cd280 2023-07-13T22:27:45.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.251+0000 7f4d82ffd700 1 -- 172.21.15.60:0/2317870327 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d7c057df0 con 0x7f4d8c0cd280 2023-07-13T22:27:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:27:45.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6c06ace0 msgr2=0x7f4d6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6c06ace0 0x7f4d6c06d190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4d74000c00 tx=0x7f4d74008030 comp rx=0 tx=0).stop 2023-07-13T22:27:45.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 msgr2=0x7f4d8c0cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 0x7f4d8c0cbfa0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00ebe0 tx=0x7f4d7c009ec0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 shutdown_connections 2023-07-13T22:27:45.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d6c06ace0 0x7f4d6c06d190 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d8c0cd280 0x7f4d8c0cbfa0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8c0d0e40 0x7f4d8c15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 --2- 172.21.15.60:0/2317870327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d8c0d0a70 0x7f4d8c151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 >> 172.21.15.60:0/2317870327 conn(0x7f4d8c071600 msgr2=0x7f4d8c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:45.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 shutdown_connections 2023-07-13T22:27:45.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.254+0000 7f4d90fff700 1 -- 172.21.15.60:0/2317870327 wait complete. 2023-07-13T22:27:45.252 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:27:45.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.477+0000 7f35e45ff700 1 Processor -- start 2023-07-13T22:27:45.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.478+0000 7f35e45ff700 1 -- start start 2023-07-13T22:27:45.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e45ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc158930 0x7f35dc0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e45ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35dc06a5e0 0x7f35dc15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e45ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc06d030 con 0x7f35dc15c4a0 2023-07-13T22:27:45.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e45ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc0c9620 con 0x7f35dc06a5e0 2023-07-13T22:27:45.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e45ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc06ca70 con 0x7f35dc158930 2023-07-13T22:27:45.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e2b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35dc06a5e0 0x7f35dc15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e1b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f35dc15c4a0 0x7f35dc069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38280/0 (socket says 172.21.15.60:38280) 2023-07-13T22:27:45.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e1b9a700 1 -- 172.21.15.60:0/422384282 learned_addr learned my addr 172.21.15.60:0/422384282 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:45.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e239b700 1 --2- 172.21.15.60:0/422384282 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc158930 0x7f35dc0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e2b9c700 1 -- 172.21.15.60:0/422384282 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc158930 msgr2=0x7f35dc0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:45.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e2b9c700 1 --2- 172.21.15.60:0/422384282 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc158930 0x7f35dc0c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.479+0000 7f35e2b9c700 1 -- 172.21.15.60:0/422384282 >> v1:172.21.15.60:6789/0 conn(0x7f35dc15c4a0 legacy=0x7f35dc069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.480+0000 7f35e2b9c700 1 -- 172.21.15.60:0/422384282 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc0adc20 con 0x7f35dc06a5e0 2023-07-13T22:27:45.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.480+0000 7f35e1399700 1 -- 172.21.15.60:0/422384282 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465814029 0 0) 0x7f35dc06d030 con 0x7f35dc15c4a0 2023-07-13T22:27:45.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.480+0000 7f35e2b9c700 1 --2- 172.21.15.60:0/422384282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc06a5e0 0x7f35dc15c910 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009d90 tx=0x7f35d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155a8b3e016d517a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:45.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.480+0000 7f35e1399700 1 -- 172.21.15.60:0/422384282 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d400e070 con 0x7f35dc06a5e0 2023-07-13T22:27:45.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.480+0000 7f35e1399700 1 -- 172.21.15.60:0/422384282 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d400bbd0 con 0x7f35dc06a5e0 2023-07-13T22:27:45.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.480+0000 7f35e1399700 1 -- 172.21.15.60:0/422384282 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4012430 con 0x7f35dc06a5e0 2023-07-13T22:27:45.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.481+0000 7f35e45ff700 1 -- 172.21.15.60:0/422384282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc06a5e0 msgr2=0x7f35dc15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.481+0000 7f35e45ff700 1 --2- 172.21.15.60:0/422384282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc06a5e0 0x7f35dc15c910 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009d90 tx=0x7f35d400b330 comp rx=0 tx=0).stop 2023-07-13T22:27:45.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.481+0000 7f35e45ff700 1 -- 172.21.15.60:0/422384282 shutdown_connections 2023-07-13T22:27:45.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.481+0000 7f35e45ff700 1 --2- 172.21.15.60:0/422384282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc06a5e0 0x7f35dc15c910 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.481+0000 7f35e45ff700 1 --2- 172.21.15.60:0/422384282 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc158930 0x7f35dc0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.481+0000 7f35e45ff700 1 -- 172.21.15.60:0/422384282 >> 172.21.15.60:0/422384282 conn(0x7f35dc0cbf50 msgr2=0x7f35dc0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:45.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.481+0000 7f35e45ff700 1 -- 172.21.15.60:0/422384282 shutdown_connections 2023-07-13T22:27:45.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.481+0000 7f35e45ff700 1 -- 172.21.15.60:0/422384282 wait complete. 2023-07-13T22:27:45.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.482+0000 7f35e45ff700 1 Processor -- start 2023-07-13T22:27:45.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.483+0000 7f35e45ff700 1 -- start start 2023-07-13T22:27:45.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e45ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc06a5e0 0x7f35dc1674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e45ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc158930 0x7f35dc14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e45ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc15c4a0 0x7f35dc1f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e45ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc14ea10 con 0x7f35dc158930 2023-07-13T22:27:45.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e45ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc1f26c0 con 0x7f35dc06a5e0 2023-07-13T22:27:45.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e45ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc1f29a0 con 0x7f35dc15c4a0 2023-07-13T22:27:45.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e1b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc158930 0x7f35dc14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e1b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc158930 0x7f35dc14e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49032/0 (socket says 172.21.15.60:49032) 2023-07-13T22:27:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e1b9a700 1 -- 172.21.15.60:0/2103497019 learned_addr learned my addr 172.21.15.60:0/2103497019 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e239b700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc06a5e0 0x7f35dc1674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e2b9c700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc15c4a0 0x7f35dc1f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e1b9a700 1 -- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc15c4a0 msgr2=0x7f35dc1f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e1b9a700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc15c4a0 0x7f35dc1f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e1b9a700 1 -- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc06a5e0 msgr2=0x7f35dc1674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e1b9a700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc06a5e0 0x7f35dc1674b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e1b9a700 1 -- 172.21.15.60:0/2103497019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4009a70 con 0x7f35dc158930 2023-07-13T22:27:45.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e239b700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc06a5e0 0x7f35dc1674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:45.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.484+0000 7f35e2b9c700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc15c4a0 0x7f35dc1f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:45.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.485+0000 7f35e1b9a700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc158930 0x7f35dc14e4d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f35cc005910 tx=0x7f35cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:45.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.485+0000 7f35d2ffd700 1 -- 172.21.15.60:0/2103497019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc005d70 con 0x7f35dc158930 2023-07-13T22:27:45.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.485+0000 7f35d2ffd700 1 -- 172.21.15.60:0/2103497019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc0070d0 con 0x7f35dc158930 2023-07-13T22:27:45.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.485+0000 7f35d2ffd700 1 -- 172.21.15.60:0/2103497019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc007460 con 0x7f35dc158930 2023-07-13T22:27:45.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.485+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc1f2c80 con 0x7f35dc158930 2023-07-13T22:27:45.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.485+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc0c73e0 con 0x7f35dc158930 2023-07-13T22:27:45.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.486+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc04e210 con 0x7f35dc158930 2023-07-13T22:27:45.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.487+0000 7f35d2ffd700 1 -- 172.21.15.60:0/2103497019 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35cc0075c0 con 0x7f35dc158930 2023-07-13T22:27:45.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.488+0000 7f35d2ffd700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35c006ace0 0x7f35c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.489+0000 7f35d2ffd700 1 -- 172.21.15.60:0/2103497019 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35cc088b00 con 0x7f35dc158930 2023-07-13T22:27:45.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.489+0000 7f35e239b700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35c006ace0 0x7f35c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.489+0000 7f35e239b700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35c006ace0 0x7f35c006d190 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009820 tx=0x7f35d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:45.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.493+0000 7f35d2ffd700 1 -- 172.21.15.60:0/2103497019 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f35cc08ae60 con 0x7f35dc158930 2023-07-13T22:27:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.699+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35dc0b47c0 con 0x7f35dc158930 2023-07-13T22:27:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.700+0000 7f35d2ffd700 1 -- 172.21.15.60:0/2103497019 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35cc053b10 con 0x7f35dc158930 2023-07-13T22:27:45.681 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:27:45.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35c006ace0 msgr2=0x7f35c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35c006ace0 0x7f35c006d190 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009820 tx=0x7f35d8008040 comp rx=0 tx=0).stop 2023-07-13T22:27:45.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc158930 msgr2=0x7f35dc14e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc158930 0x7f35dc14e4d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f35cc005910 tx=0x7f35cc00c8b0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 shutdown_connections 2023-07-13T22:27:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35c006ace0 0x7f35c006d190 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35dc15c4a0 0x7f35dc1f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc06a5e0 0x7f35dc1674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 --2- 172.21.15.60:0/2103497019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35dc158930 0x7f35dc14e4d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 >> 172.21.15.60:0/2103497019 conn(0x7f35dc0cbf50 msgr2=0x7f35dc070390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:45.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 shutdown_connections 2023-07-13T22:27:45.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.702+0000 7f35e45ff700 1 -- 172.21.15.60:0/2103497019 wait complete. 2023-07-13T22:27:45.701 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:27:45.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.925+0000 7f2a27dff700 1 Processor -- start 2023-07-13T22:27:45.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.926+0000 7f2a27dff700 1 -- start start 2023-07-13T22:27:45.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a27dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a200cfd50 0x7f2a200d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a27dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b1ac0 0x7f2a200d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a27dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a200d1870 con 0x7f2a200b8200 2023-07-13T22:27:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a27dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a200cf370 con 0x7f2a200cfd50 2023-07-13T22:27:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a27dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a200d12b0 con 0x7f2a200b1ac0 2023-07-13T22:27:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a2539a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a200cfd50 0x7f2a200d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a25b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2a200b8200 0x7f2a200cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38296/0 (socket says 172.21.15.60:38296) 2023-07-13T22:27:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a25b9b700 1 -- 172.21.15.60:0/1684937137 learned_addr learned my addr 172.21.15.60:0/1684937137 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:45.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a2639c700 1 --2- 172.21.15.60:0/1684937137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b1ac0 0x7f2a200d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a2539a700 1 -- 172.21.15.60:0/1684937137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b1ac0 msgr2=0x7f2a200d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:45.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.927+0000 7f2a2539a700 1 --2- 172.21.15.60:0/1684937137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b1ac0 0x7f2a200d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.928+0000 7f2a2539a700 1 -- 172.21.15.60:0/1684937137 >> v1:172.21.15.60:6789/0 conn(0x7f2a200b8200 legacy=0x7f2a200cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.928+0000 7f2a2539a700 1 -- 172.21.15.60:0/1684937137 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a200d1590 con 0x7f2a200cfd50 2023-07-13T22:27:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.928+0000 7f2a2639c700 1 --2- 172.21.15.60:0/1684937137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b1ac0 0x7f2a200d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.928+0000 7f2a24b99700 1 -- 172.21.15.60:0/1684937137 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626176099 0 0) 0x7f2a200d1870 con 0x7f2a200b8200 2023-07-13T22:27:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.928+0000 7f2a2539a700 1 --2- 172.21.15.60:0/1684937137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200cfd50 0x7f2a200d23c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009840 tx=0x7f2a10009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb00f7778cec7c0d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.928+0000 7f2a24b99700 1 -- 172.21.15.60:0/1684937137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10011830 con 0x7f2a200cfd50 2023-07-13T22:27:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.928+0000 7f2a24b99700 1 -- 172.21.15.60:0/1684937137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a10011990 con 0x7f2a200cfd50 2023-07-13T22:27:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.928+0000 7f2a24b99700 1 -- 172.21.15.60:0/1684937137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10011d20 con 0x7f2a200cfd50 2023-07-13T22:27:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.929+0000 7f2a27dff700 1 -- 172.21.15.60:0/1684937137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200cfd50 msgr2=0x7f2a200d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.929+0000 7f2a27dff700 1 --2- 172.21.15.60:0/1684937137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200cfd50 0x7f2a200d23c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009840 tx=0x7f2a10009b50 comp rx=0 tx=0).stop 2023-07-13T22:27:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.929+0000 7f2a27dff700 1 -- 172.21.15.60:0/1684937137 shutdown_connections 2023-07-13T22:27:45.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.929+0000 7f2a27dff700 1 --2- 172.21.15.60:0/1684937137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b1ac0 0x7f2a200d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.929+0000 7f2a27dff700 1 --2- 172.21.15.60:0/1684937137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200cfd50 0x7f2a200d23c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.929+0000 7f2a27dff700 1 -- 172.21.15.60:0/1684937137 >> 172.21.15.60:0/1684937137 conn(0x7f2a2006b2f0 msgr2=0x7f2a2006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:45.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.929+0000 7f2a27dff700 1 -- 172.21.15.60:0/1684937137 shutdown_connections 2023-07-13T22:27:45.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.929+0000 7f2a27dff700 1 -- 172.21.15.60:0/1684937137 wait complete. 2023-07-13T22:27:45.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.930+0000 7f2a27dff700 1 Processor -- start 2023-07-13T22:27:45.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.931+0000 7f2a27dff700 1 -- start start 2023-07-13T22:27:45.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.931+0000 7f2a27dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200b1ac0 0x7f2a2014df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a27dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b8200 0x7f2a2015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a27dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a200cfd50 0x7f2a200c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a27dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a200c8160 con 0x7f2a200b1ac0 2023-07-13T22:27:45.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a27dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a200c7550 con 0x7f2a200cfd50 2023-07-13T22:27:45.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a27dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a200c7830 con 0x7f2a200b8200 2023-07-13T22:27:45.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a25b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200b1ac0 0x7f2a2014df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a25b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200b1ac0 0x7f2a2014df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49056/0 (socket says 172.21.15.60:49056) 2023-07-13T22:27:45.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a25b9b700 1 -- 172.21.15.60:0/4229690879 learned_addr learned my addr 172.21.15.60:0/4229690879 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:45.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a2539a700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b8200 0x7f2a2015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a2639c700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a200cfd50 0x7f2a200c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a25b9b700 1 -- 172.21.15.60:0/4229690879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b8200 msgr2=0x7f2a2015f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a25b9b700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b8200 0x7f2a2015f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a25b9b700 1 -- 172.21.15.60:0/4229690879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a200cfd50 msgr2=0x7f2a200c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a25b9b700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a200cfd50 0x7f2a200c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a25b9b700 1 -- 172.21.15.60:0/4229690879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a100094a0 con 0x7f2a200b1ac0 2023-07-13T22:27:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.932+0000 7f2a2639c700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a200cfd50 0x7f2a200c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.933+0000 7f2a25b9b700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200b1ac0 0x7f2a2014df50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0011e0 tx=0x7f2a1c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.933+0000 7f2a167fc700 1 -- 172.21.15.60:0/4229690879 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c004c80 con 0x7f2a200b1ac0 2023-07-13T22:27:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.933+0000 7f2a167fc700 1 -- 172.21.15.60:0/4229690879 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c004de0 con 0x7f2a200b1ac0 2023-07-13T22:27:45.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.933+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a200c7b10 con 0x7f2a200b1ac0 2023-07-13T22:27:45.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.933+0000 7f2a167fc700 1 -- 172.21.15.60:0/4229690879 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c007310 con 0x7f2a200b1ac0 2023-07-13T22:27:45.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.933+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2014e490 con 0x7f2a200b1ac0 2023-07-13T22:27:45.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.934+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a200cd370 con 0x7f2a200b1ac0 2023-07-13T22:27:45.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.935+0000 7f2a167fc700 1 -- 172.21.15.60:0/4229690879 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a1c007470 con 0x7f2a200b1ac0 2023-07-13T22:27:45.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.936+0000 7f2a167fc700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a0406ace0 0x7f2a0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:45.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.936+0000 7f2a167fc700 1 -- 172.21.15.60:0/4229690879 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a1c089270 con 0x7f2a200b1ac0 2023-07-13T22:27:45.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.936+0000 7f2a2539a700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a0406ace0 0x7f2a0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:45.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.937+0000 7f2a2539a700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a0406ace0 0x7f2a0406d190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2a200d0140 tx=0x7f2a10008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:45.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:45.941+0000 7f2a167fc700 1 -- 172.21.15.60:0/4229690879 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2a200cd370 con 0x7f2a200b1ac0 2023-07-13T22:27:46.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.144+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a200cd370 con 0x7f2a200b1ac0 2023-07-13T22:27:46.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.145+0000 7f2a167fc700 1 -- 172.21.15.60:0/4229690879 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a200cd370 con 0x7f2a200b1ac0 2023-07-13T22:27:46.126 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:27:46.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a0406ace0 msgr2=0x7f2a0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a0406ace0 0x7f2a0406d190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2a200d0140 tx=0x7f2a10008030 comp rx=0 tx=0).stop 2023-07-13T22:27:46.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200b1ac0 msgr2=0x7f2a2014df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200b1ac0 0x7f2a2014df50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0011e0 tx=0x7f2a1c00ca30 comp rx=0 tx=0).stop 2023-07-13T22:27:46.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 shutdown_connections 2023-07-13T22:27:46.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2a0406ace0 0x7f2a0406d190 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a200b1ac0 0x7f2a2014df50 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a200cfd50 0x7f2a200c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 --2- 172.21.15.60:0/4229690879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2a200b8200 0x7f2a2015f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 >> 172.21.15.60:0/4229690879 conn(0x7f2a2006b2f0 msgr2=0x7f2a200d2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:46.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 shutdown_connections 2023-07-13T22:27:46.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.147+0000 7f2a27dff700 1 -- 172.21.15.60:0/4229690879 wait complete. 2023-07-13T22:27:46.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:27:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.371+0000 7fd0185ff700 1 Processor -- start 2023-07-13T22:27:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.371+0000 7fd0185ff700 1 -- start start 2023-07-13T22:27:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.372+0000 7fd0185ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd0100add00 0x7fd01006ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.372+0000 7fd0185ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0100c8e90 con 0x7fd0100ca9f0 2023-07-13T22:27:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.372+0000 7fd0185ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0100ae0d0 con 0x7fd01006a5e0 2023-07-13T22:27:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.372+0000 7fd0185ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01006d3b0 con 0x7fd0100add00 2023-07-13T22:27:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.373+0000 7fd015b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd0100ca9f0 0x7fd010069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38312/0 (socket says 172.21.15.60:38312) 2023-07-13T22:27:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.373+0000 7fd015b9a700 1 -- 172.21.15.60:0/1841732076 learned_addr learned my addr 172.21.15.60:0/1841732076 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.373+0000 7fd01639b700 1 --2- 172.21.15.60:0/1841732076 >> v2:172.21.15.105:3300/0 conn(0x7fd0100add00 0x7fd01006ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.373+0000 7fd015399700 1 -- 172.21.15.60:0/1841732076 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579332848 0 0) 0x7fd0100c8e90 con 0x7fd0100ca9f0 2023-07-13T22:27:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.373+0000 7fd015399700 1 -- 172.21.15.60:0/1841732076 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcffc0035b0 con 0x7fd0100ca9f0 2023-07-13T22:27:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.373+0000 7fd01639b700 1 -- 172.21.15.60:0/1841732076 >> v1:172.21.15.105:6789/0 conn(0x7fd01006a5e0 legacy=0x7fd0100b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.373+0000 7fd01639b700 1 -- 172.21.15.60:0/1841732076 >> v1:172.21.15.60:6789/0 conn(0x7fd0100ca9f0 legacy=0x7fd010069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.373+0000 7fd01639b700 1 -- 172.21.15.60:0/1841732076 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0100c90e0 con 0x7fd0100add00 2023-07-13T22:27:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd01639b700 1 --2- 172.21.15.60:0/1841732076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0100add00 0x7fd01006ce70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009840 tx=0x7fd00c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15635a1e05d5be43 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd015399700 1 -- 172.21.15.60:0/1841732076 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c011830 con 0x7fd0100add00 2023-07-13T22:27:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd015399700 1 -- 172.21.15.60:0/1841732076 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c011990 con 0x7fd0100add00 2023-07-13T22:27:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd015399700 1 -- 172.21.15.60:0/1841732076 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c011d20 con 0x7fd0100add00 2023-07-13T22:27:46.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd0185ff700 1 -- 172.21.15.60:0/1841732076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0100add00 msgr2=0x7fd01006ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd0185ff700 1 --2- 172.21.15.60:0/1841732076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0100add00 0x7fd01006ce70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009840 tx=0x7fd00c009b50 comp rx=0 tx=0).stop 2023-07-13T22:27:46.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd0185ff700 1 -- 172.21.15.60:0/1841732076 shutdown_connections 2023-07-13T22:27:46.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd0185ff700 1 --2- 172.21.15.60:0/1841732076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0100add00 0x7fd01006ce70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd0185ff700 1 -- 172.21.15.60:0/1841732076 >> 172.21.15.60:0/1841732076 conn(0x7fd0100cbf50 msgr2=0x7fd0100ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:46.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.374+0000 7fd0185ff700 1 -- 172.21.15.60:0/1841732076 shutdown_connections 2023-07-13T22:27:46.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.375+0000 7fd0185ff700 1 -- 172.21.15.60:0/1841732076 wait complete. 2023-07-13T22:27:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.376+0000 7fd0185ff700 1 Processor -- start 2023-07-13T22:27:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.376+0000 7fd0185ff700 1 -- start start 2023-07-13T22:27:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.377+0000 7fd0185ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd01006a5e0 0x7fd010152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.377+0000 7fd0185ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0100add00 0x7fd0100c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.377+0000 7fd0185ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0100ca9f0 0x7fd01014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.377+0000 7fd0185ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100c7980 con 0x7fd0100ca9f0 2023-07-13T22:27:46.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.377+0000 7fd0185ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100c82e0 con 0x7fd0100add00 2023-07-13T22:27:46.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.377+0000 7fd0185ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0100c6d00 con 0x7fd01006a5e0 2023-07-13T22:27:46.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd016b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0100ca9f0 0x7fd01014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd016b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0100ca9f0 0x7fd01014fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49060/0 (socket says 172.21.15.60:49060) 2023-07-13T22:27:46.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd016b9c700 1 -- 172.21.15.60:0/224267983 learned_addr learned my addr 172.21.15.60:0/224267983 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:46.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd01639b700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd01006a5e0 0x7fd010152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd015b9a700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0100add00 0x7fd0100c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd01639b700 1 -- 172.21.15.60:0/224267983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0100add00 msgr2=0x7fd0100c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd01639b700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0100add00 0x7fd0100c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd01639b700 1 -- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0100ca9f0 msgr2=0x7fd01014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd01639b700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0100ca9f0 0x7fd01014fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd01639b700 1 -- 172.21.15.60:0/224267983 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c0094a0 con 0x7fd01006a5e0 2023-07-13T22:27:46.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.378+0000 7fd016b9c700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0100ca9f0 0x7fd01014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:46.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.379+0000 7fd01639b700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd01006a5e0 0x7fd010152f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd010071690 tx=0x7fd00c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.379+0000 7fd006ffd700 1 -- 172.21.15.60:0/224267983 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0036a0 con 0x7fd01006a5e0 2023-07-13T22:27:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.379+0000 7fd006ffd700 1 -- 172.21.15.60:0/224267983 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c01f070 con 0x7fd01006a5e0 2023-07-13T22:27:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.379+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0100c6fe0 con 0x7fd01006a5e0 2023-07-13T22:27:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.379+0000 7fd006ffd700 1 -- 172.21.15.60:0/224267983 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c003a20 con 0x7fd01006a5e0 2023-07-13T22:27:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.379+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0100c75a0 con 0x7fd01006a5e0 2023-07-13T22:27:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.380+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01004e210 con 0x7fd01006a5e0 2023-07-13T22:27:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.381+0000 7fd006ffd700 1 -- 172.21.15.60:0/224267983 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd00c003b80 con 0x7fd01006a5e0 2023-07-13T22:27:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.382+0000 7fd006ffd700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcff406ace0 0x7fcff406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.382+0000 7fd006ffd700 1 -- 172.21.15.60:0/224267983 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd00c021070 con 0x7fd01006a5e0 2023-07-13T22:27:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.382+0000 7fd015b9a700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcff406ace0 0x7fcff406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.383+0000 7fd015b9a700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcff406ace0 0x7fcff406d190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd01015b460 tx=0x7fd000007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:46.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.387+0000 7fd006ffd700 1 -- 172.21.15.60:0/224267983 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd00c055ec0 con 0x7fd01006a5e0 2023-07-13T22:27:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.592+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0100b3ad0 con 0x7fd01006a5e0 2023-07-13T22:27:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.593+0000 7fd006ffd700 1 -- 172.21.15.60:0/224267983 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd00c02b0a0 con 0x7fd01006a5e0 2023-07-13T22:27:46.573 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:27:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcff406ace0 msgr2=0x7fcff406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcff406ace0 0x7fcff406d190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd01015b460 tx=0x7fd000007dc0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd01006a5e0 msgr2=0x7fd010152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd01006a5e0 0x7fd010152f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd010071690 tx=0x7fd00c004750 comp rx=0 tx=0).stop 2023-07-13T22:27:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 shutdown_connections 2023-07-13T22:27:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcff406ace0 0x7fcff406d190 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0100ca9f0 0x7fd01014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd01006a5e0 0x7fd010152f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 --2- 172.21.15.60:0/224267983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0100add00 0x7fd0100c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 >> 172.21.15.60:0/224267983 conn(0x7fd0100cbf50 msgr2=0x7fd01015d420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 shutdown_connections 2023-07-13T22:27:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.595+0000 7fd0185ff700 1 -- 172.21.15.60:0/224267983 wait complete. 2023-07-13T22:27:46.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:27:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.816+0000 7f1f291ff700 1 Processor -- start 2023-07-13T22:27:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.816+0000 7f1f291ff700 1 -- start start 2023-07-13T22:27:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f291ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f24070270 0x7f1f240706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f291ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f240c9c50 0x7f1f240ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f291ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2415d970 con 0x7f1f24159170 2023-07-13T22:27:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f291ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f24070c00 con 0x7f1f240c9c50 2023-07-13T22:27:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f291ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2415cd60 con 0x7f1f24070270 2023-07-13T22:27:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f22d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1f24159170 0x7f1f240c4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38328/0 (socket says 172.21.15.60:38328) 2023-07-13T22:27:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f22d9d700 1 -- 172.21.15.60:0/1093104750 learned_addr learned my addr 172.21.15.60:0/1093104750 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f2359e700 1 --2- 172.21.15.60:0/1093104750 >> v2:172.21.15.105:3300/0 conn(0x7f1f240c9c50 0x7f1f240ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f2259c700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f24070270 0x7f1f240706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657854230 0 0) 0x7f1f2415d970 con 0x7f1f24159170 2023-07-13T22:27:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.818+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f0c0057c0 con 0x7f1f24159170 2023-07-13T22:27:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3803505691 0 0) 0x7f1f0c0057c0 con 0x7f1f24159170 2023-07-13T22:27:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f24070270 msgr2=0x7f1f240706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f24070270 0x7f1f240706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 >> v2:172.21.15.105:3300/0 conn(0x7f1f240c9c50 msgr2=0x7f1f240ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 --2- 172.21.15.60:0/1093104750 >> v2:172.21.15.105:3300/0 conn(0x7f1f240c9c50 0x7f1f240ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2415d040 con 0x7f1f24159170 2023-07-13T22:27:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f1f14001670 con 0x7f1f24159170 2023-07-13T22:27:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f2359e700 1 --2- 172.21.15.60:0/1093104750 >> v2:172.21.15.105:3300/0 conn(0x7f1f240c9c50 0x7f1f240ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 >> v1:172.21.15.60:6789/0 conn(0x7f1f24159170 legacy=0x7f1f240c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c005c10 0x7f1f0c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c006640 0x7f1f0c006b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f24070270 0x7f1f0c0071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f24070c00 con 0x7f1f0c005c10 2023-07-13T22:27:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2415cd60 con 0x7f1f0c006640 2023-07-13T22:27:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c00dc20 con 0x7f1f24070270 2023-07-13T22:27:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f22d9d700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c005c10 0x7f1f0c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f2259c700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c006640 0x7f1f0c006b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.819+0000 7f1f2359e700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f24070270 0x7f1f0c0071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.820+0000 7f1f22d9d700 1 -- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f24070270 msgr2=0x7f1f0c0071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.820+0000 7f1f22d9d700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f24070270 0x7f1f0c0071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.820+0000 7f1f22d9d700 1 -- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c006640 msgr2=0x7f1f0c006b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.820+0000 7f1f22d9d700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c006640 0x7f1f0c006b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.820+0000 7f1f22d9d700 1 -- 172.21.15.60:0/1093104750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0057c0 con 0x7f1f0c005c10 2023-07-13T22:27:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.820+0000 7f1f22d9d700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c005c10 0x7f1f0c006020 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400c320 tx=0x7f1f0c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21749234528384a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.820+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1400ce60 con 0x7f1f0c005c10 2023-07-13T22:27:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.820+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14005960 con 0x7f1f0c005c10 2023-07-13T22:27:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.820+0000 7f1f21d9b700 1 -- 172.21.15.60:0/1093104750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14005cf0 con 0x7f1f0c005c10 2023-07-13T22:27:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.821+0000 7f1f291ff700 1 -- 172.21.15.60:0/1093104750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c005c10 msgr2=0x7f1f0c006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.821+0000 7f1f291ff700 1 --2- 172.21.15.60:0/1093104750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c005c10 0x7f1f0c006020 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400c320 tx=0x7f1f0c007a90 comp rx=0 tx=0).stop 2023-07-13T22:27:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.821+0000 7f1f2359e700 1 -- 172.21.15.60:0/1093104750 reap_dead start 2023-07-13T22:27:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.821+0000 7f1f291ff700 1 -- 172.21.15.60:0/1093104750 shutdown_connections 2023-07-13T22:27:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.821+0000 7f1f291ff700 1 -- 172.21.15.60:0/1093104750 >> 172.21.15.60:0/1093104750 conn(0x7f1f240cf410 msgr2=0x7f1f240cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.821+0000 7f1f291ff700 1 -- 172.21.15.60:0/1093104750 shutdown_connections 2023-07-13T22:27:46.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.821+0000 7f1f291ff700 1 -- 172.21.15.60:0/1093104750 wait complete. 2023-07-13T22:27:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.823+0000 7f1f291ff700 1 Processor -- start 2023-07-13T22:27:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.823+0000 7f1f291ff700 1 -- start start 2023-07-13T22:27:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f291ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c005c10 0x7f1f24156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f291ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2414bec0 0x7f1f2406c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f291ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f240af270 0x7f1f240af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f291ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f240b1f30 con 0x7f1f2414bec0 2023-07-13T22:27:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f291ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2414c710 con 0x7f1f240af270 2023-07-13T22:27:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f291ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f241571e0 con 0x7f1f0c005c10 2023-07-13T22:27:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2414bec0 0x7f1f2406c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2414bec0 0x7f1f2406c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49072/0 (socket says 172.21.15.60:49072) 2023-07-13T22:27:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2259c700 1 -- 172.21.15.60:0/1132655873 learned_addr learned my addr 172.21.15.60:0/1132655873 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f22d9d700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c005c10 0x7f1f24156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2359e700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f240af270 0x7f1f240af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2259c700 1 -- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c005c10 msgr2=0x7f1f24156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2259c700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c005c10 0x7f1f24156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2259c700 1 -- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f240af270 msgr2=0x7f1f240af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2259c700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f240af270 0x7f1f240af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2259c700 1 -- 172.21.15.60:0/1132655873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f240b1870 con 0x7f1f2414bec0 2023-07-13T22:27:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f22d9d700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c005c10 0x7f1f24156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.824+0000 7f1f2359e700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f240af270 0x7f1f240af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.825+0000 7f1f2259c700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2414bec0 0x7f1f2406c650 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1f180119a0 tx=0x7f1f18011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.825+0000 7f1f0b7fe700 1 -- 172.21.15.60:0/1132655873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1800f940 con 0x7f1f2414bec0 2023-07-13T22:27:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.825+0000 7f1f0b7fe700 1 -- 172.21.15.60:0/1132655873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1800faa0 con 0x7f1f2414bec0 2023-07-13T22:27:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.825+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f240b1b50 con 0x7f1f2414bec0 2023-07-13T22:27:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.825+0000 7f1f0b7fe700 1 -- 172.21.15.60:0/1132655873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1800fe30 con 0x7f1f2414bec0 2023-07-13T22:27:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.825+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2406cd70 con 0x7f1f2414bec0 2023-07-13T22:27:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.826+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f240b16d0 con 0x7f1f2414bec0 2023-07-13T22:27:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.827+0000 7f1f0b7fe700 1 -- 172.21.15.60:0/1132655873 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f1801d400 con 0x7f1f2414bec0 2023-07-13T22:27:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.828+0000 7f1f0b7fe700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f1006ace0 0x7f1f1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.828+0000 7f1f0b7fe700 1 -- 172.21.15.60:0/1132655873 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f18089aa0 con 0x7f1f2414bec0 2023-07-13T22:27:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.828+0000 7f1f22d9d700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f1006ace0 0x7f1f1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.829+0000 7f1f22d9d700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f1006ace0 0x7f1f1006d190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1f14008fd0 tx=0x7f1f1400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:46.832+0000 7f1f0b7fe700 1 -- 172.21.15.60:0/1132655873 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1f240b16d0 con 0x7f1f2414bec0 2023-07-13T22:27:47.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.080+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f2406cd70 con 0x7f1f2414bec0 2023-07-13T22:27:47.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.081+0000 7f1f0b7fe700 1 -- 172.21.15.60:0/1132655873 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4718 (secure 0 0 0) 0x7f1f2406cd70 con 0x7f1f2414bec0 2023-07-13T22:27:47.062 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:47.062 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1603865496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1603865496},{"type":"v1","addr":"172.21.15.105:6834","nonce":1603865496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6735,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1777842000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1777842000},{"type":"v1","addr":"172.21.15.105:6835","nonce":1777842000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6738,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2613179178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2613179178},{"type":"v1","addr":"172.21.15.60:6836","nonce":2613179178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:25:02.392864+0000","modified":"2023-07-13T22:27:43.547746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":43,"addr":"172.21.15.60:6837/3617406711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3617406711},{"type":"v1","addr":"172.21.15.60:6837","nonce":3617406711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-13T22:27:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f1006ace0 msgr2=0x7f1f1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f1006ace0 0x7f1f1006d190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1f14008fd0 tx=0x7f1f1400a000 comp rx=0 tx=0).stop 2023-07-13T22:27:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2414bec0 msgr2=0x7f1f2406c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2414bec0 0x7f1f2406c650 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1f180119a0 tx=0x7f1f18011d60 comp rx=0 tx=0).stop 2023-07-13T22:27:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 shutdown_connections 2023-07-13T22:27:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1f1006ace0 0x7f1f1006d190 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f240af270 0x7f1f240af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c005c10 0x7f1f24156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 --2- 172.21.15.60:0/1132655873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2414bec0 0x7f1f2406c650 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 >> 172.21.15.60:0/1132655873 conn(0x7f1f240cf410 msgr2=0x7f1f24154550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 shutdown_connections 2023-07-13T22:27:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.083+0000 7f1f291ff700 1 -- 172.21.15.60:0/1132655873 wait complete. 2023-07-13T22:27:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2023-07-13T22:27:47.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:27:47.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.304+0000 7f9060fff700 1 Processor -- start 2023-07-13T22:27:47.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.305+0000 7f9060fff700 1 -- start start 2023-07-13T22:27:47.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.306+0000 7f9060fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f905c159170 0x7f905c0c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.306+0000 7f9060fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f905c070980 0x7f905c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.306+0000 7f9060fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905c15d970 con 0x7f905c0c5040 2023-07-13T22:27:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.306+0000 7f9060fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c15cd60 con 0x7f905c159170 2023-07-13T22:27:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.306+0000 7f9060fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c15d040 con 0x7f905c070980 2023-07-13T22:27:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.306+0000 7f905a59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f905c159170 0x7f905c0c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.306+0000 7f905a59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f905c159170 0x7f905c0c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49076/0 (socket says 172.21.15.60:49076) 2023-07-13T22:27:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.306+0000 7f905a59c700 1 -- 172.21.15.60:0/2208671577 learned_addr learned my addr 172.21.15.60:0/2208671577 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.306+0000 7f905ad9d700 1 --2- 172.21.15.60:0/2208671577 >> v2:172.21.15.105:3300/0 conn(0x7f905c070980 0x7f905c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.307+0000 7f905a59c700 1 -- 172.21.15.60:0/2208671577 >> v2:172.21.15.105:3300/0 conn(0x7f905c070980 msgr2=0x7f905c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:47.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.307+0000 7f905a59c700 1 --2- 172.21.15.60:0/2208671577 >> v2:172.21.15.105:3300/0 conn(0x7f905c070980 0x7f905c0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.307+0000 7f905a59c700 1 -- 172.21.15.60:0/2208671577 >> v1:172.21.15.105:6789/0 conn(0x7f905c0c5040 legacy=0x7f905c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.307+0000 7f905a59c700 1 -- 172.21.15.60:0/2208671577 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c15d320 con 0x7f905c159170 2023-07-13T22:27:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.307+0000 7f905ad9d700 1 --2- 172.21.15.60:0/2208671577 >> v2:172.21.15.105:3300/0 conn(0x7f905c070980 0x7f905c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.307+0000 7f905a59c700 1 --2- 172.21.15.60:0/2208671577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c159170 0x7f905c0c4b00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f905c06e4b0 tx=0x7f904c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35abf3a90164a434 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.307+0000 7f9059d9b700 1 -- 172.21.15.60:0/2208671577 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00b870 con 0x7f905c159170 2023-07-13T22:27:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.307+0000 7f9059d9b700 1 -- 172.21.15.60:0/2208671577 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c00b9d0 con 0x7f905c159170 2023-07-13T22:27:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.307+0000 7f9059d9b700 1 -- 172.21.15.60:0/2208671577 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00bd60 con 0x7f905c159170 2023-07-13T22:27:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.308+0000 7f9060fff700 1 -- 172.21.15.60:0/2208671577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c159170 msgr2=0x7f905c0c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.308+0000 7f9060fff700 1 --2- 172.21.15.60:0/2208671577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c159170 0x7f905c0c4b00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f905c06e4b0 tx=0x7f904c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.308+0000 7f9060fff700 1 -- 172.21.15.60:0/2208671577 shutdown_connections 2023-07-13T22:27:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.308+0000 7f9060fff700 1 --2- 172.21.15.60:0/2208671577 >> v2:172.21.15.105:3300/0 conn(0x7f905c070980 0x7f905c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.308+0000 7f9060fff700 1 --2- 172.21.15.60:0/2208671577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c159170 0x7f905c0c4b00 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.308+0000 7f9060fff700 1 -- 172.21.15.60:0/2208671577 >> 172.21.15.60:0/2208671577 conn(0x7f905c0cf410 msgr2=0x7f905c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.308+0000 7f9060fff700 1 -- 172.21.15.60:0/2208671577 shutdown_connections 2023-07-13T22:27:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.308+0000 7f9060fff700 1 -- 172.21.15.60:0/2208671577 wait complete. 2023-07-13T22:27:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.310+0000 7f9060fff700 1 Processor -- start 2023-07-13T22:27:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.310+0000 7f9060fff700 1 -- start start 2023-07-13T22:27:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f9060fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c070980 0x7f905c144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f9060fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0c5040 0x7f905c150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f9060fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905c159170 0x7f905c150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f9060fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c140a70 con 0x7f905c070980 2023-07-13T22:27:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f9060fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c1599f0 con 0x7f905c0c5040 2023-07-13T22:27:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f9060fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905c1514f0 con 0x7f905c159170 2023-07-13T22:27:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c070980 0x7f905c144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c070980 0x7f905c144620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49080/0 (socket says 172.21.15.60:49080) 2023-07-13T22:27:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905a59c700 1 -- 172.21.15.60:0/4259797160 learned_addr learned my addr 172.21.15.60:0/4259797160 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f9053fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0c5040 0x7f905c150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905a59c700 1 -- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905c159170 msgr2=0x7f905c150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905ad9d700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905c159170 0x7f905c150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905a59c700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905c159170 0x7f905c150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905a59c700 1 -- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0c5040 msgr2=0x7f905c150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905a59c700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0c5040 0x7f905c150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905a59c700 1 -- 172.21.15.60:0/4259797160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c009550 con 0x7f905c070980 2023-07-13T22:27:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f905ad9d700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905c159170 0x7f905c150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.312+0000 7f9053fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0c5040 0x7f905c150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.313+0000 7f905a59c700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c070980 0x7f905c144620 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f904c000c00 tx=0x7f904c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.313+0000 7f9052ffd700 1 -- 172.21.15.60:0/4259797160 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c004540 con 0x7f905c070980 2023-07-13T22:27:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.313+0000 7f9052ffd700 1 -- 172.21.15.60:0/4259797160 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c0036a0 con 0x7f905c070980 2023-07-13T22:27:47.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.313+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c144b60 con 0x7f905c070980 2023-07-13T22:27:47.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.313+0000 7f9052ffd700 1 -- 172.21.15.60:0/4259797160 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c003a30 con 0x7f905c070980 2023-07-13T22:27:47.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.313+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c144de0 con 0x7f905c070980 2023-07-13T22:27:47.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.314+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905c0ad860 con 0x7f905c070980 2023-07-13T22:27:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.315+0000 7f9052ffd700 1 -- 172.21.15.60:0/4259797160 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f904c021070 con 0x7f905c070980 2023-07-13T22:27:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.316+0000 7f9052ffd700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f903c06ace0 0x7f903c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.316+0000 7f9052ffd700 1 -- 172.21.15.60:0/4259797160 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f904c08ac50 con 0x7f905c070980 2023-07-13T22:27:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.316+0000 7f9053fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f903c06ace0 0x7f903c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.317+0000 7f9053fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f903c06ace0 0x7f903c06d190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f905c14f240 tx=0x7f9044009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.320+0000 7f9052ffd700 1 -- 172.21.15.60:0/4259797160 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f904c055e10 con 0x7f905c070980 2023-07-13T22:27:47.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.528+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f905c071440 con 0x7f905c070980 2023-07-13T22:27:47.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.529+0000 7f9052ffd700 1 -- 172.21.15.60:0/4259797160 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18260 (secure 0 0 0) 0x7f904c015090 con 0x7f905c070980 2023-07-13T22:27:47.510 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:47.511 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":75,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:25:03.885230+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:25:00.672661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-13T22:25:01.364844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:27:47.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f903c06ace0 msgr2=0x7f903c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f903c06ace0 0x7f903c06d190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f905c14f240 tx=0x7f9044009040 comp rx=0 tx=0).stop 2023-07-13T22:27:47.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c070980 msgr2=0x7f905c144620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c070980 0x7f905c144620 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f904c000c00 tx=0x7f904c004750 comp rx=0 tx=0).stop 2023-07-13T22:27:47.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 shutdown_connections 2023-07-13T22:27:47.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f903c06ace0 0x7f903c06d190 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905c070980 0x7f905c144620 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905c159170 0x7f905c150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 --2- 172.21.15.60:0/4259797160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f905c0c5040 0x7f905c150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.531+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 >> 172.21.15.60:0/4259797160 conn(0x7f905c0cf410 msgr2=0x7f905c0cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:47.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.532+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 shutdown_connections 2023-07-13T22:27:47.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.532+0000 7f9060fff700 1 -- 172.21.15.60:0/4259797160 wait complete. 2023-07-13T22:27:47.530 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:27:47.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:27:47.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.755+0000 7f32025ff700 1 Processor -- start 2023-07-13T22:27:47.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.755+0000 7f32025ff700 1 -- start start 2023-07-13T22:27:47.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.756+0000 7f32025ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0b8400 0x7f31fc0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.756+0000 7f32025ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f31fc0cfca0 0x7f31fc0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.756+0000 7f32025ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc0d0ac0 con 0x7f31fc0d3750 2023-07-13T22:27:47.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.756+0000 7f32025ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc0b87d0 con 0x7f31fc0cfca0 2023-07-13T22:27:47.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.756+0000 7f32025ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0b4eb0 con 0x7f31fc0b8400 2023-07-13T22:27:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.756+0000 7f3200b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f31fc0cfca0 0x7f31fc0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.756+0000 7f3200b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f31fc0cfca0 0x7f31fc0d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49088/0 (socket says 172.21.15.60:49088) 2023-07-13T22:27:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.756+0000 7f3200b9c700 1 -- 172.21.15.60:0/973392641 learned_addr learned my addr 172.21.15.60:0/973392641 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.757+0000 7f3200b9c700 1 -- 172.21.15.60:0/973392641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0b8400 msgr2=0x7f31fc0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.757+0000 7f3200b9c700 1 --2- 172.21.15.60:0/973392641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0b8400 0x7f31fc0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.757+0000 7f3200b9c700 1 -- 172.21.15.60:0/973392641 >> v1:172.21.15.105:6789/0 conn(0x7f31fc0d3750 legacy=0x7f31fc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.757+0000 7f3200b9c700 1 -- 172.21.15.60:0/973392641 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0b5190 con 0x7f31fc0cfca0 2023-07-13T22:27:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.757+0000 7f3200b9c700 1 --2- 172.21.15.60:0/973392641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0cfca0 0x7f31fc0d0f50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f31f4009e50 tx=0x7f31f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbe996175980c8e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.757+0000 7f31faffd700 1 -- 172.21.15.60:0/973392641 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f400e070 con 0x7f31fc0cfca0 2023-07-13T22:27:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.757+0000 7f31faffd700 1 -- 172.21.15.60:0/973392641 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f4003a20 con 0x7f31fc0cfca0 2023-07-13T22:27:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.757+0000 7f31faffd700 1 -- 172.21.15.60:0/973392641 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4003dd0 con 0x7f31fc0cfca0 2023-07-13T22:27:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.758+0000 7f32025ff700 1 -- 172.21.15.60:0/973392641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0cfca0 msgr2=0x7f31fc0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.758+0000 7f32025ff700 1 --2- 172.21.15.60:0/973392641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0cfca0 0x7f31fc0d0f50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f31f4009e50 tx=0x7f31f400b330 comp rx=0 tx=0).stop 2023-07-13T22:27:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.758+0000 7f32025ff700 1 -- 172.21.15.60:0/973392641 shutdown_connections 2023-07-13T22:27:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.758+0000 7f32025ff700 1 --2- 172.21.15.60:0/973392641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0cfca0 0x7f31fc0d0f50 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.758+0000 7f32025ff700 1 --2- 172.21.15.60:0/973392641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0b8400 0x7f31fc0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.758+0000 7f32025ff700 1 -- 172.21.15.60:0/973392641 >> 172.21.15.60:0/973392641 conn(0x7f31fc06b390 msgr2=0x7f31fc06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.758+0000 7f32025ff700 1 -- 172.21.15.60:0/973392641 shutdown_connections 2023-07-13T22:27:47.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.758+0000 7f32025ff700 1 -- 172.21.15.60:0/973392641 wait complete. 2023-07-13T22:27:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.760+0000 7f32025ff700 1 Processor -- start 2023-07-13T22:27:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.760+0000 7f32025ff700 1 -- start start 2023-07-13T22:27:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f32025ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0b8400 0x7f31fc0b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f32025ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0cfca0 0x7f31fc0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f32025ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31fc0d3750 0x7f31fc1f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f32025ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1f2000 con 0x7f31fc0b8400 2023-07-13T22:27:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f32025ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1f5a10 con 0x7f31fc0d3750 2023-07-13T22:27:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f32025ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1f5cf0 con 0x7f31fc0cfca0 2023-07-13T22:27:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0b8400 0x7f31fc0b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0b8400 0x7f31fc0b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49098/0 (socket says 172.21.15.60:49098) 2023-07-13T22:27:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fbfff700 1 -- 172.21.15.60:0/1698658456 learned_addr learned my addr 172.21.15.60:0/1698658456 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f3200b9c700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31fc0d3750 0x7f31fc1f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fb7fe700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0cfca0 0x7f31fc0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fbfff700 1 -- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0cfca0 msgr2=0x7f31fc0c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fbfff700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0cfca0 0x7f31fc0c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fbfff700 1 -- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31fc0d3750 msgr2=0x7f31fc1f2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fbfff700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31fc0d3750 0x7f31fc1f2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fbfff700 1 -- 172.21.15.60:0/1698658456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f4009b30 con 0x7f31fc0b8400 2023-07-13T22:27:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f3200b9c700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31fc0d3750 0x7f31fc1f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.761+0000 7f31fb7fe700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0cfca0 0x7f31fc0c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.762+0000 7f31fbfff700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0b8400 0x7f31fc0b6ed0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00b4a0 tx=0x7f31ec00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.762+0000 7f31f8ff9700 1 -- 172.21.15.60:0/1698658456 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec011830 con 0x7f31fc0b8400 2023-07-13T22:27:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.762+0000 7f31f8ff9700 1 -- 172.21.15.60:0/1698658456 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec011990 con 0x7f31fc0b8400 2023-07-13T22:27:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.762+0000 7f31f8ff9700 1 -- 172.21.15.60:0/1698658456 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec011d40 con 0x7f31fc0b8400 2023-07-13T22:27:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.762+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc1f5fd0 con 0x7f31fc0b8400 2023-07-13T22:27:47.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.762+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc0cc9b0 con 0x7f31fc0b8400 2023-07-13T22:27:47.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.763+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31fc15b4a0 con 0x7f31fc0b8400 2023-07-13T22:27:47.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.764+0000 7f31f8ff9700 1 -- 172.21.15.60:0/1698658456 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f31ec01a400 con 0x7f31fc0b8400 2023-07-13T22:27:47.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.765+0000 7f31f8ff9700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31e806ace0 0x7f31e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:47.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.765+0000 7f31f8ff9700 1 -- 172.21.15.60:0/1698658456 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31ec089510 con 0x7f31fc0b8400 2023-07-13T22:27:47.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.765+0000 7f31fb7fe700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31e806ace0 0x7f31e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:47.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.766+0000 7f31fb7fe700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31e806ace0 0x7f31e806d190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f31fc154d70 tx=0x7f31f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:47.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:47.769+0000 7f31f8ff9700 1 -- 172.21.15.60:0/1698658456 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f31ec054530 con 0x7f31fc0b8400 2023-07-13T22:27:48.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.021+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31fc15b1c0 con 0x7f31fc0b8400 2023-07-13T22:27:48.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.022+0000 7f31f8ff9700 1 -- 172.21.15.60:0/1698658456 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4718 (secure 0 0 0) 0x7f31ec057b50 con 0x7f31fc0b8400 2023-07-13T22:27:48.002 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:48.003 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1603865496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1603865496},{"type":"v1","addr":"172.21.15.105:6834","nonce":1603865496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6735,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1777842000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1777842000},{"type":"v1","addr":"172.21.15.105:6835","nonce":1777842000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6738,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2613179178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2613179178},{"type":"v1","addr":"172.21.15.60:6836","nonce":2613179178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:25:02.392864+0000","modified":"2023-07-13T22:27:43.547746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":43,"addr":"172.21.15.60:6837/3617406711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3617406711},{"type":"v1","addr":"172.21.15.60:6837","nonce":3617406711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-13T22:27:48.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31e806ace0 msgr2=0x7f31e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31e806ace0 0x7f31e806d190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f31fc154d70 tx=0x7f31f0009040 comp rx=0 tx=0).stop 2023-07-13T22:27:48.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0b8400 msgr2=0x7f31fc0b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0b8400 0x7f31fc0b6ed0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00b4a0 tx=0x7f31ec00b860 comp rx=0 tx=0).stop 2023-07-13T22:27:48.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 shutdown_connections 2023-07-13T22:27:48.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31e806ace0 0x7f31e806d190 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc0b8400 0x7f31fc0b6ed0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31fc0d3750 0x7f31fc1f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 --2- 172.21.15.60:0/1698658456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31fc0cfca0 0x7f31fc0c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 >> 172.21.15.60:0/1698658456 conn(0x7f31fc06b390 msgr2=0x7f31fc0b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:48.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 shutdown_connections 2023-07-13T22:27:48.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.024+0000 7f32025ff700 1 -- 172.21.15.60:0/1698658456 wait complete. 2023-07-13T22:27:48.007 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2023-07-13T22:27:48.023 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:27:48.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.246+0000 7fda6bdff700 1 Processor -- start 2023-07-13T22:27:48.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.246+0000 7fda6bdff700 1 -- start start 2023-07-13T22:27:48.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.247+0000 7fda6bdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fda640b8400 0x7fda640cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.247+0000 7fda6bdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda640d3750 0x7fda640cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.247+0000 7fda6bdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 0x7fda6415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.247+0000 7fda6bdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda640cff50 con 0x7fda640d3750 2023-07-13T22:27:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.247+0000 7fda6bdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda640d0ac0 con 0x7fda640b8400 2023-07-13T22:27:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.247+0000 7fda6bdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda640b87d0 con 0x7fda640cfad0 2023-07-13T22:27:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.247+0000 7fda6939a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda640d3750 0x7fda640cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.247+0000 7fda6939a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda640d3750 0x7fda640cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49104/0 (socket says 172.21.15.60:49104) 2023-07-13T22:27:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.247+0000 7fda6939a700 1 -- 172.21.15.60:0/2855899450 learned_addr learned my addr 172.21.15.60:0/2855899450 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda69b9b700 1 --2- 172.21.15.60:0/2855899450 >> v2:172.21.15.105:3300/0 conn(0x7fda640b8400 0x7fda640cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda6a39c700 1 --2- 172.21.15.60:0/2855899450 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 0x7fda6415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda6939a700 1 -- 172.21.15.60:0/2855899450 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 msgr2=0x7fda6415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda6939a700 1 --2- 172.21.15.60:0/2855899450 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 0x7fda6415fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda6939a700 1 -- 172.21.15.60:0/2855899450 >> v2:172.21.15.105:3300/0 conn(0x7fda640b8400 msgr2=0x7fda640cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda6939a700 1 --2- 172.21.15.60:0/2855899450 >> v2:172.21.15.105:3300/0 conn(0x7fda640b8400 0x7fda640cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda6939a700 1 -- 172.21.15.60:0/2855899450 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda640b4eb0 con 0x7fda640d3750 2023-07-13T22:27:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda6939a700 1 --2- 172.21.15.60:0/2855899450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 0x7fda640cf590 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fda640cf560 tx=0x7fda54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7854e8b83ba37282 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda68b99700 1 -- 172.21.15.60:0/2855899450 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5400b870 con 0x7fda640d3750 2023-07-13T22:27:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda68b99700 1 -- 172.21.15.60:0/2855899450 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5400b9d0 con 0x7fda640d3750 2023-07-13T22:27:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.248+0000 7fda68b99700 1 -- 172.21.15.60:0/2855899450 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5400bd60 con 0x7fda640d3750 2023-07-13T22:27:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.249+0000 7fda6bdff700 1 -- 172.21.15.60:0/2855899450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 msgr2=0x7fda640cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.249+0000 7fda6bdff700 1 --2- 172.21.15.60:0/2855899450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 0x7fda640cf590 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fda640cf560 tx=0x7fda54009bd0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.249+0000 7fda6bdff700 1 -- 172.21.15.60:0/2855899450 shutdown_connections 2023-07-13T22:27:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.249+0000 7fda6bdff700 1 --2- 172.21.15.60:0/2855899450 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 0x7fda6415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.249+0000 7fda6bdff700 1 --2- 172.21.15.60:0/2855899450 >> v2:172.21.15.105:3300/0 conn(0x7fda640b8400 0x7fda640cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.249+0000 7fda6bdff700 1 --2- 172.21.15.60:0/2855899450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 0x7fda640cf590 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.249+0000 7fda6bdff700 1 -- 172.21.15.60:0/2855899450 >> 172.21.15.60:0/2855899450 conn(0x7fda6406b390 msgr2=0x7fda6406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.249+0000 7fda6bdff700 1 -- 172.21.15.60:0/2855899450 shutdown_connections 2023-07-13T22:27:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.249+0000 7fda6bdff700 1 -- 172.21.15.60:0/2855899450 wait complete. 2023-07-13T22:27:48.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.251+0000 7fda6bdff700 1 Processor -- start 2023-07-13T22:27:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.251+0000 7fda6bdff700 1 -- start start 2023-07-13T22:27:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6bdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda640b8400 0x7fda64151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6bdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 0x7fda6415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6bdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 0x7fda640d1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6bdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda641515b0 con 0x7fda640d3750 2023-07-13T22:27:48.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6bdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6415ca10 con 0x7fda640b8400 2023-07-13T22:27:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6bdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6415ccb0 con 0x7fda640cfad0 2023-07-13T22:27:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6a39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 0x7fda640d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6a39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 0x7fda640d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49106/0 (socket says 172.21.15.60:49106) 2023-07-13T22:27:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6a39c700 1 -- 172.21.15.60:0/3098805790 learned_addr learned my addr 172.21.15.60:0/3098805790 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda69b9b700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda640b8400 0x7fda64151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.252+0000 7fda6939a700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 0x7fda6415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda6a39c700 1 -- 172.21.15.60:0/3098805790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 msgr2=0x7fda6415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda6a39c700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 0x7fda6415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda6a39c700 1 -- 172.21.15.60:0/3098805790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda640b8400 msgr2=0x7fda64151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda6a39c700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda640b8400 0x7fda64151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda6a39c700 1 -- 172.21.15.60:0/3098805790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54009550 con 0x7fda640d3750 2023-07-13T22:27:48.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda6a39c700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 0x7fda640d1050 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007e60 tx=0x7fda5c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:48.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda5a7fc700 1 -- 172.21.15.60:0/3098805790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00ed30 con 0x7fda640d3750 2023-07-13T22:27:48.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda5a7fc700 1 -- 172.21.15.60:0/3098805790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c014420 con 0x7fda640d3750 2023-07-13T22:27:48.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6415cf90 con 0x7fda640d3750 2023-07-13T22:27:48.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.253+0000 7fda5a7fc700 1 -- 172.21.15.60:0/3098805790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0147d0 con 0x7fda640d3750 2023-07-13T22:27:48.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.254+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6415e1d0 con 0x7fda640d3750 2023-07-13T22:27:48.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.255+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6415d5c0 con 0x7fda640d3750 2023-07-13T22:27:48.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.255+0000 7fda5a7fc700 1 -- 172.21.15.60:0/3098805790 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda5c014930 con 0x7fda640d3750 2023-07-13T22:27:48.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.256+0000 7fda5a7fc700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda4806ace0 0x7fda4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.256+0000 7fda5a7fc700 1 -- 172.21.15.60:0/3098805790 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda5c089a80 con 0x7fda640d3750 2023-07-13T22:27:48.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.257+0000 7fda69b9b700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda4806ace0 0x7fda4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.257+0000 7fda69b9b700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda4806ace0 0x7fda4806d190 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fda640b4c10 tx=0x7fda60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:48.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.261+0000 7fda5a7fc700 1 -- 172.21.15.60:0/3098805790 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fda5c054a90 con 0x7fda640d3750 2023-07-13T22:27:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.510+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda6415d790 con 0x7fda640d3750 2023-07-13T22:27:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.511+0000 7fda5a7fc700 1 -- 172.21.15.60:0/3098805790 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4718 (secure 0 0 0) 0x7fda5c0580b0 con 0x7fda640d3750 2023-07-13T22:27:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:48.492 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1603865496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1603865496},{"type":"v1","addr":"172.21.15.105:6834","nonce":1603865496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6735,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1777842000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1777842000},{"type":"v1","addr":"172.21.15.105:6835","nonce":1777842000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6738,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2613179178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2613179178},{"type":"v1","addr":"172.21.15.60:6836","nonce":2613179178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:25:02.392864+0000","modified":"2023-07-13T22:27:43.547746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":43,"addr":"172.21.15.60:6837/3617406711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":3617406711},{"type":"v1","addr":"172.21.15.60:6837","nonce":3617406711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-13T22:27:48.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda4806ace0 msgr2=0x7fda4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda4806ace0 0x7fda4806d190 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fda640b4c10 tx=0x7fda60008040 comp rx=0 tx=0).stop 2023-07-13T22:27:48.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 msgr2=0x7fda640d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 0x7fda640d1050 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007e60 tx=0x7fda5c00e3c0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 shutdown_connections 2023-07-13T22:27:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda4806ace0 0x7fda4806d190 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda640d3750 0x7fda640d1050 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda640b8400 0x7fda64151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 --2- 172.21.15.60:0/3098805790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda640cfad0 0x7fda6415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.513+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 >> 172.21.15.60:0/3098805790 conn(0x7fda6406b390 msgr2=0x7fda640b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:48.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.514+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 shutdown_connections 2023-07-13T22:27:48.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.514+0000 7fda6bdff700 1 -- 172.21.15.60:0/3098805790 wait complete. 2023-07-13T22:27:48.496 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2023-07-13T22:27:48.512 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:27:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.734+0000 7f4e197ff700 1 Processor -- start 2023-07-13T22:27:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.735+0000 7f4e197ff700 1 -- start start 2023-07-13T22:27:48.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.736+0000 7f4e197ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140b8400 0x7f4e140cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.736+0000 7f4e197ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e140d0ac0 con 0x7f4e140cfca0 2023-07-13T22:27:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.736+0000 7f4e197ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e140b87d0 con 0x7f4e140d3750 2023-07-13T22:27:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.736+0000 7f4e197ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140b4eb0 con 0x7f4e140b8400 2023-07-13T22:27:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.736+0000 7f4e137fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4e140cfca0 0x7f4e140d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38334/0 (socket says 172.21.15.60:38334) 2023-07-13T22:27:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.736+0000 7f4e137fe700 1 -- 172.21.15.60:0/5059327 learned_addr learned my addr 172.21.15.60:0/5059327 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:48.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.736+0000 7f4e12ffd700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140b8400 0x7f4e140cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.736+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238433627 0 0) 0x7f4e140d0ac0 con 0x7f4e140cfca0 2023-07-13T22:27:48.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dfc0035b0 con 0x7f4e140cfca0 2023-07-13T22:27:48.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864380903 0 0) 0x7f4e140b87d0 con 0x7f4e140d3750 2023-07-13T22:27:48.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e140d0ac0 con 0x7f4e140d3750 2023-07-13T22:27:48.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2129646556 0 0) 0x7f4dfc0035b0 con 0x7f4e140cfca0 2023-07-13T22:27:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140b8400 msgr2=0x7f4e140cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140b8400 0x7f4e140cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 >> v1:172.21.15.105:6789/0 conn(0x7f4e140d3750 legacy=0x7f4e140cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e140b5190 con 0x7f4e140cfca0 2023-07-13T22:27:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f4e0c0019f0 con 0x7f4e140cfca0 2023-07-13T22:27:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 >> v1:172.21.15.60:6789/0 conn(0x7f4e140cfca0 legacy=0x7f4e140d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4dfc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dfc004850 0x7f4dfc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dfc005200 0x7f4dfc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140b4eb0 con 0x7f4dfc003bc0 2023-07-13T22:27:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc00c060 con 0x7f4dfc004850 2023-07-13T22:27:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc00c220 con 0x7f4dfc005200 2023-07-13T22:27:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e12ffd700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140b8400 0x7f4e140cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e12ffd700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4dfc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e127fc700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dfc004850 0x7f4dfc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.737+0000 7f4e137fe700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dfc005200 0x7f4dfc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.738+0000 7f4e12ffd700 1 -- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dfc005200 msgr2=0x7f4dfc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.738+0000 7f4e12ffd700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dfc005200 0x7f4dfc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.738+0000 7f4e12ffd700 1 -- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dfc004850 msgr2=0x7f4dfc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.738+0000 7f4e12ffd700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dfc004850 0x7f4dfc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.738+0000 7f4e12ffd700 1 -- 172.21.15.60:0/5059327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc0035b0 con 0x7f4dfc003bc0 2023-07-13T22:27:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.738+0000 7f4e12ffd700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4dfc003fb0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4e14072650 tx=0x7f4dfc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d01b6584303322 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.738+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04004030 con 0x7f4dfc003bc0 2023-07-13T22:27:48.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.738+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e04004190 con 0x7f4dfc003bc0 2023-07-13T22:27:48.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.738+0000 7f4e11ffb700 1 -- 172.21.15.60:0/5059327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04004540 con 0x7f4dfc003bc0 2023-07-13T22:27:48.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.739+0000 7f4e197ff700 1 -- 172.21.15.60:0/5059327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 msgr2=0x7f4dfc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.739+0000 7f4e197ff700 1 --2- 172.21.15.60:0/5059327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4dfc003fb0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4e14072650 tx=0x7f4dfc005880 comp rx=0 tx=0).stop 2023-07-13T22:27:48.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.739+0000 7f4e137fe700 1 -- 172.21.15.60:0/5059327 reap_dead start 2023-07-13T22:27:48.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.739+0000 7f4e197ff700 1 -- 172.21.15.60:0/5059327 shutdown_connections 2023-07-13T22:27:48.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.739+0000 7f4e197ff700 1 -- 172.21.15.60:0/5059327 >> 172.21.15.60:0/5059327 conn(0x7f4e1406b390 msgr2=0x7f4e1406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:48.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.739+0000 7f4e197ff700 1 -- 172.21.15.60:0/5059327 shutdown_connections 2023-07-13T22:27:48.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.739+0000 7f4e197ff700 1 -- 172.21.15.60:0/5059327 wait complete. 2023-07-13T22:27:48.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.741+0000 7f4e197ff700 1 Processor -- start 2023-07-13T22:27:48.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.741+0000 7f4e197ff700 1 -- start start 2023-07-13T22:27:48.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e197ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4e140d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e197ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e141e5d30 0x7f4e14148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e197ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140c8880 0x7f4e140c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e197ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e141e6e40 con 0x7f4dfc003bc0 2023-07-13T22:27:48.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e197ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140c9260 con 0x7f4e141e5d30 2023-07-13T22:27:48.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e197ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e141494c0 con 0x7f4e140c8880 2023-07-13T22:27:48.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e12ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4e140d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e12ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4e140d21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49138/0 (socket says 172.21.15.60:49138) 2023-07-13T22:27:48.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e12ffd700 1 -- 172.21.15.60:0/1877530137 learned_addr learned my addr 172.21.15.60:0/1877530137 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:48.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e127fc700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e141e5d30 0x7f4e14148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e137fe700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140c8880 0x7f4e140c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e12ffd700 1 -- 172.21.15.60:0/1877530137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140c8880 msgr2=0x7f4e140c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e12ffd700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140c8880 0x7f4e140c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e12ffd700 1 -- 172.21.15.60:0/1877530137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e141e5d30 msgr2=0x7f4e14148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e12ffd700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e141e5d30 0x7f4e14148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e12ffd700 1 -- 172.21.15.60:0/1877530137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e141e61c0 con 0x7f4dfc003bc0 2023-07-13T22:27:48.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.742+0000 7f4e127fc700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e141e5d30 0x7f4e14148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:48.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.743+0000 7f4e12ffd700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4e140d21c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400e4b0 tx=0x7f4e040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:48.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.743+0000 7f4dfb7fe700 1 -- 172.21.15.60:0/1877530137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040038a0 con 0x7f4dfc003bc0 2023-07-13T22:27:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.743+0000 7f4dfb7fe700 1 -- 172.21.15.60:0/1877530137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e04003a00 con 0x7f4dfc003bc0 2023-07-13T22:27:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.743+0000 7f4dfb7fe700 1 -- 172.21.15.60:0/1877530137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04003d70 con 0x7f4dfc003bc0 2023-07-13T22:27:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.743+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e141e64a0 con 0x7f4dfc003bc0 2023-07-13T22:27:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.743+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e141e6a60 con 0x7f4dfc003bc0 2023-07-13T22:27:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.744+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e141e6c20 con 0x7f4dfc003bc0 2023-07-13T22:27:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.745+0000 7f4dfb7fe700 1 -- 172.21.15.60:0/1877530137 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e04013470 con 0x7f4dfc003bc0 2023-07-13T22:27:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.746+0000 7f4dfb7fe700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e0006ace0 0x7f4e0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:48.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.746+0000 7f4dfb7fe700 1 -- 172.21.15.60:0/1877530137 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e0408ad20 con 0x7f4dfc003bc0 2023-07-13T22:27:48.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.746+0000 7f4e127fc700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e0006ace0 0x7f4e0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:48.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.747+0000 7f4e127fc700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e0006ace0 0x7f4e0006d190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4e140c9490 tx=0x7f4e08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:48.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.751+0000 7f4dfb7fe700 1 -- 172.21.15.60:0/1877530137 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4e04055dc0 con 0x7f4dfc003bc0 2023-07-13T22:27:48.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.958+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e14073850 con 0x7f4dfc003bc0 2023-07-13T22:27:48.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.959+0000 7f4dfb7fe700 1 -- 172.21.15.60:0/1877530137 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18260 (secure 0 0 0) 0x7f4e040593e0 con 0x7f4dfc003bc0 2023-07-13T22:27:48.940 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:48.940 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":75,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:25:03.885230+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:25:00.672661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-13T22:25:01.364844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:27:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.961+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e0006ace0 msgr2=0x7f4e0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.961+0000 7f4e197ff700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e0006ace0 0x7f4e0006d190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4e140c9490 tx=0x7f4e08009040 comp rx=0 tx=0).stop 2023-07-13T22:27:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.961+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 msgr2=0x7f4e140d21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:48.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.961+0000 7f4e197ff700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4e140d21c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400e4b0 tx=0x7f4e040047c0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.962+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 shutdown_connections 2023-07-13T22:27:48.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.962+0000 7f4e197ff700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e0006ace0 0x7f4e0006d190 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.962+0000 7f4e197ff700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dfc003bc0 0x7f4e140d21c0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.962+0000 7f4e197ff700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e140c8880 0x7f4e140c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.962+0000 7f4e197ff700 1 --2- 172.21.15.60:0/1877530137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e141e5d30 0x7f4e14148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:48.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.962+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 >> 172.21.15.60:0/1877530137 conn(0x7f4e1406b390 msgr2=0x7f4e140b7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:48.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.962+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 shutdown_connections 2023-07-13T22:27:48.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:48.962+0000 7f4e197ff700 1 -- 172.21.15.60:0/1877530137 wait complete. 2023-07-13T22:27:48.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:27:49.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.185+0000 7f0d3adff700 1 Processor -- start 2023-07-13T22:27:49.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.185+0000 7f0d3adff700 1 -- start start 2023-07-13T22:27:49.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.186+0000 7f0d3adff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340b8570 0x7f0d340ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.186+0000 7f0d3adff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d340d35c0 0x7f0d340d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.186+0000 7f0d3adff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0d340cf3f0 0x7f0d340d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.186+0000 7f0d3adff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d340d3b30 con 0x7f0d340d35c0 2023-07-13T22:27:49.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.186+0000 7f0d3adff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d340d1930 con 0x7f0d340cf3f0 2023-07-13T22:27:49.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.186+0000 7f0d3adff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340d09d0 con 0x7f0d340b8570 2023-07-13T22:27:49.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.186+0000 7f0d33fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d340d35c0 0x7f0d340d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.186+0000 7f0d33fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d340d35c0 0x7f0d340d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49146/0 (socket says 172.21.15.60:49146) 2023-07-13T22:27:49.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.186+0000 7f0d33fff700 1 -- 172.21.15.60:0/1436710470 learned_addr learned my addr 172.21.15.60:0/1436710470 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:49.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d38b9b700 1 --2- 172.21.15.60:0/1436710470 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340b8570 0x7f0d340ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d3939c700 1 --2- 172.21.15.60:0/1436710470 >> v2:172.21.15.105:3300/0 conn(0x7f0d340cf3f0 0x7f0d340d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d33fff700 1 -- 172.21.15.60:0/1436710470 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340b8570 msgr2=0x7f0d340ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:49.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d33fff700 1 --2- 172.21.15.60:0/1436710470 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340b8570 0x7f0d340ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d33fff700 1 -- 172.21.15.60:0/1436710470 >> v2:172.21.15.105:3300/0 conn(0x7f0d340cf3f0 msgr2=0x7f0d340d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:49.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d33fff700 1 --2- 172.21.15.60:0/1436710470 >> v2:172.21.15.105:3300/0 conn(0x7f0d340cf3f0 0x7f0d340d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d33fff700 1 -- 172.21.15.60:0/1436710470 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340cda10 con 0x7f0d340d35c0 2023-07-13T22:27:49.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d3939c700 1 --2- 172.21.15.60:0/1436710470 >> v2:172.21.15.105:3300/0 conn(0x7f0d340cf3f0 0x7f0d340d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:49.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d33fff700 1 --2- 172.21.15.60:0/1436710470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340d35c0 0x7f0d340d0eb0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0d340cd240 tx=0x7f0d20009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de61950fb77fbb39 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:49.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d337fe700 1 -- 172.21.15.60:0/1436710470 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20011830 con 0x7f0d340d35c0 2023-07-13T22:27:49.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d337fe700 1 -- 172.21.15.60:0/1436710470 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d20011990 con 0x7f0d340d35c0 2023-07-13T22:27:49.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.187+0000 7f0d337fe700 1 -- 172.21.15.60:0/1436710470 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20011d20 con 0x7f0d340d35c0 2023-07-13T22:27:49.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.188+0000 7f0d3adff700 1 -- 172.21.15.60:0/1436710470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340d35c0 msgr2=0x7f0d340d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:49.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.188+0000 7f0d3adff700 1 --2- 172.21.15.60:0/1436710470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340d35c0 0x7f0d340d0eb0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0d340cd240 tx=0x7f0d20009bd0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.188+0000 7f0d3adff700 1 -- 172.21.15.60:0/1436710470 shutdown_connections 2023-07-13T22:27:49.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.188+0000 7f0d3adff700 1 --2- 172.21.15.60:0/1436710470 >> v2:172.21.15.105:3300/0 conn(0x7f0d340cf3f0 0x7f0d340d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.188+0000 7f0d3adff700 1 --2- 172.21.15.60:0/1436710470 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340b8570 0x7f0d340ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.188+0000 7f0d3adff700 1 --2- 172.21.15.60:0/1436710470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340d35c0 0x7f0d340d0eb0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.188+0000 7f0d3adff700 1 -- 172.21.15.60:0/1436710470 >> 172.21.15.60:0/1436710470 conn(0x7f0d34071600 msgr2=0x7f0d340719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:49.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.188+0000 7f0d3adff700 1 -- 172.21.15.60:0/1436710470 shutdown_connections 2023-07-13T22:27:49.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.188+0000 7f0d3adff700 1 -- 172.21.15.60:0/1436710470 wait complete. 2023-07-13T22:27:49.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.190+0000 7f0d3adff700 1 Processor -- start 2023-07-13T22:27:49.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.190+0000 7f0d3adff700 1 -- start start 2023-07-13T22:27:49.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d3adff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340b8570 0x7f0d340cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d3adff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d340cf3f0 0x7f0d34153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d3adff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340d35c0 0x7f0d341536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d3adff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340ccb30 con 0x7f0d340b8570 2023-07-13T22:27:49.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d3adff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340ca8e0 con 0x7f0d340cf3f0 2023-07-13T22:27:49.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d3adff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340cabc0 con 0x7f0d340d35c0 2023-07-13T22:27:49.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d38b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340b8570 0x7f0d340cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d38b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340b8570 0x7f0d340cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49156/0 (socket says 172.21.15.60:49156) 2023-07-13T22:27:49.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d38b9b700 1 -- 172.21.15.60:0/14716953 learned_addr learned my addr 172.21.15.60:0/14716953 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:49.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d33fff700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d340cf3f0 0x7f0d34153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d3939c700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340d35c0 0x7f0d341536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d38b9b700 1 -- 172.21.15.60:0/14716953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340d35c0 msgr2=0x7f0d341536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:49.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d38b9b700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340d35c0 0x7f0d341536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d38b9b700 1 -- 172.21.15.60:0/14716953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d340cf3f0 msgr2=0x7f0d34153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:49.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d38b9b700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d340cf3f0 0x7f0d34153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d38b9b700 1 -- 172.21.15.60:0/14716953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20009550 con 0x7f0d340b8570 2023-07-13T22:27:49.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.191+0000 7f0d3939c700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340d35c0 0x7f0d341536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:49.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.192+0000 7f0d38b9b700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340b8570 0x7f0d340cc550 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800e870 tx=0x7f0d2800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:49.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.192+0000 7f0d317fa700 1 -- 172.21.15.60:0/14716953 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800ee70 con 0x7f0d340b8570 2023-07-13T22:27:49.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.192+0000 7f0d317fa700 1 -- 172.21.15.60:0/14716953 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d28005a20 con 0x7f0d340b8570 2023-07-13T22:27:49.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.192+0000 7f0d317fa700 1 -- 172.21.15.60:0/14716953 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28005db0 con 0x7f0d340b8570 2023-07-13T22:27:49.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.192+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d340caea0 con 0x7f0d340b8570 2023-07-13T22:27:49.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.192+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d340c79d0 con 0x7f0d340b8570 2023-07-13T22:27:49.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.193+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d340ca740 con 0x7f0d340b8570 2023-07-13T22:27:49.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.194+0000 7f0d317fa700 1 -- 172.21.15.60:0/14716953 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d2800c850 con 0x7f0d340b8570 2023-07-13T22:27:49.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.195+0000 7f0d317fa700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d1806ace0 0x7f0d1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.195+0000 7f0d317fa700 1 -- 172.21.15.60:0/14716953 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d28089620 con 0x7f0d340b8570 2023-07-13T22:27:49.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.196+0000 7f0d33fff700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d1806ace0 0x7f0d1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.196+0000 7f0d33fff700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d1806ace0 0x7f0d1806d190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009c10 tx=0x7f0d20008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:49.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.199+0000 7f0d317fa700 1 -- 172.21.15.60:0/14716953 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0d340ca740 con 0x7f0d340b8570 2023-07-13T22:27:49.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.451+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0d340ca740 con 0x7f0d340b8570 2023-07-13T22:27:49.432 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:27:49.451+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:27:49.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.456+0000 7f0d317fa700 1 -- 172.21.15.60:0/14716953 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0d340ca740 con 0x7f0d340b8570 2023-07-13T22:27:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d1806ace0 msgr2=0x7f0d1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d1806ace0 0x7f0d1806d190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009c10 tx=0x7f0d20008030 comp rx=0 tx=0).stop 2023-07-13T22:27:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340b8570 msgr2=0x7f0d340cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340b8570 0x7f0d340cc550 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800e870 tx=0x7f0d2800ec30 comp rx=0 tx=0).stop 2023-07-13T22:27:49.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 shutdown_connections 2023-07-13T22:27:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d1806ace0 0x7f0d1806d190 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d340b8570 0x7f0d340cc550 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d340d35c0 0x7f0d341536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 --2- 172.21.15.60:0/14716953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d340cf3f0 0x7f0d34153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 >> 172.21.15.60:0/14716953 conn(0x7f0d34071600 msgr2=0x7f0d34159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 shutdown_connections 2023-07-13T22:27:49.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.459+0000 7f0d3adff700 1 -- 172.21.15.60:0/14716953 wait complete. 2023-07-13T22:27:49.442 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:27:49.467 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:27:49.517 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:27:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.693+0000 7f2cf91ff700 1 Processor -- start 2023-07-13T22:27:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.693+0000 7f2cf91ff700 1 -- start start 2023-07-13T22:27:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.694+0000 7f2cf91ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cf40ac490 0x7f2cf40ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.694+0000 7f2cf91ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2cf40ad9a0 0x7f2cf40ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.694+0000 7f2cf91ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf40cf6d0 con 0x7f2cf415c790 2023-07-13T22:27:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.695+0000 7f2cf91ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf40ae340 con 0x7f2cf40ad9a0 2023-07-13T22:27:49.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.695+0000 7f2cf91ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40ac900 con 0x7f2cf40ac490 2023-07-13T22:27:49.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.695+0000 7f2cf2d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2cf415c790 0x7f2cf40c9e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38348/0 (socket says 172.21.15.60:38348) 2023-07-13T22:27:49.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.695+0000 7f2cf2d9d700 1 -- 172.21.15.60:0/3376476789 learned_addr learned my addr 172.21.15.60:0/3376476789 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:49.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.695+0000 7f2cf359e700 1 --2- 172.21.15.60:0/3376476789 >> v2:172.21.15.105:3300/0 conn(0x7f2cf40ad9a0 0x7f2cf40ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.695+0000 7f2cf259c700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cf40ac490 0x7f2cf40ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.695+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230760094 0 0) 0x7f2cf40cf6d0 con 0x7f2cf415c790 2023-07-13T22:27:49.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.695+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cdc0035b0 con 0x7f2cf415c790 2023-07-13T22:27:49.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.695+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243818060 0 0) 0x7f2cdc0035b0 con 0x7f2cf415c790 2023-07-13T22:27:49.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cf40ac490 msgr2=0x7f2cf40ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cf40ac490 0x7f2cf40ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 >> v2:172.21.15.105:3300/0 conn(0x7f2cf40ad9a0 msgr2=0x7f2cf40ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 --2- 172.21.15.60:0/3376476789 >> v2:172.21.15.105:3300/0 conn(0x7f2cf40ad9a0 0x7f2cf40ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40acbe0 con 0x7f2cf415c790 2023-07-13T22:27:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f2ce40017f0 con 0x7f2cf415c790 2023-07-13T22:27:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 >> v1:172.21.15.60:6789/0 conn(0x7f2cf415c790 legacy=0x7f2cf40c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cdc003a00 0x7f2cdc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cf40ac490 0x7f2cdc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc004d90 0x7f2cdc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40ae340 con 0x7f2cdc004d90 2023-07-13T22:27:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40ac900 con 0x7f2cdc003a00 2023-07-13T22:27:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc00ba10 con 0x7f2cf40ac490 2023-07-13T22:27:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf359e700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc004d90 0x7f2cdc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf2d9d700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cdc003a00 0x7f2cdc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf259c700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cf40ac490 0x7f2cdc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf359e700 1 -- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cf40ac490 msgr2=0x7f2cdc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf359e700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cf40ac490 0x7f2cdc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf359e700 1 -- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cdc003a00 msgr2=0x7f2cdc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf359e700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cdc003a00 0x7f2cdc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.696+0000 7f2cf359e700 1 -- 172.21.15.60:0/3376476789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0035b0 con 0x7f2cdc004d90 2023-07-13T22:27:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.697+0000 7f2cf359e700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc004d90 0x7f2cdc005200 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00e170 tx=0x7f2cdc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16a7b5065e4ec02 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.697+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec00ec30 con 0x7f2cdc004d90 2023-07-13T22:27:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.697+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec00f040 con 0x7f2cdc004d90 2023-07-13T22:27:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.697+0000 7f2cf1d9b700 1 -- 172.21.15.60:0/3376476789 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec014480 con 0x7f2cdc004d90 2023-07-13T22:27:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.697+0000 7f2cf91ff700 1 -- 172.21.15.60:0/3376476789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc004d90 msgr2=0x7f2cdc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.697+0000 7f2cf91ff700 1 --2- 172.21.15.60:0/3376476789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc004d90 0x7f2cdc005200 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00e170 tx=0x7f2cdc005880 comp rx=0 tx=0).stop 2023-07-13T22:27:49.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.697+0000 7f2cf359e700 1 -- 172.21.15.60:0/3376476789 reap_dead start 2023-07-13T22:27:49.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.697+0000 7f2cf91ff700 1 -- 172.21.15.60:0/3376476789 shutdown_connections 2023-07-13T22:27:49.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.697+0000 7f2cf91ff700 1 -- 172.21.15.60:0/3376476789 >> 172.21.15.60:0/3376476789 conn(0x7f2cf406b900 msgr2=0x7f2cf40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:49.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.698+0000 7f2cf91ff700 1 -- 172.21.15.60:0/3376476789 shutdown_connections 2023-07-13T22:27:49.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.698+0000 7f2cf91ff700 1 -- 172.21.15.60:0/3376476789 wait complete. 2023-07-13T22:27:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.699+0000 7f2cf91ff700 1 Processor -- start 2023-07-13T22:27:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.699+0000 7f2cf91ff700 1 -- start start 2023-07-13T22:27:49.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.700+0000 7f2cf91ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc004d90 0x7f2cf415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.700+0000 7f2cf91ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf41f1f20 0x7f2cf40c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.700+0000 7f2cf91ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cf40c7830 0x7f2cf406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.700+0000 7f2cf91ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf41f2fb0 con 0x7f2cf41f1f20 2023-07-13T22:27:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.700+0000 7f2cf91ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf41f23a0 con 0x7f2cf40c7830 2023-07-13T22:27:49.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.700+0000 7f2cf91ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf41f2680 con 0x7f2cdc004d90 2023-07-13T22:27:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf41f1f20 0x7f2cf40c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf41f1f20 0x7f2cf40c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49200/0 (socket says 172.21.15.60:49200) 2023-07-13T22:27:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf259c700 1 -- 172.21.15.60:0/432507908 learned_addr learned my addr 172.21.15.60:0/432507908 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf2d9d700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc004d90 0x7f2cf415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf359e700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cf40c7830 0x7f2cf406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf259c700 1 -- 172.21.15.60:0/432507908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc004d90 msgr2=0x7f2cf415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf259c700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc004d90 0x7f2cf415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf259c700 1 -- 172.21.15.60:0/432507908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cf40c7830 msgr2=0x7f2cf406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf259c700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cf40c7830 0x7f2cf406e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf259c700 1 -- 172.21.15.60:0/432507908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf41f2960 con 0x7f2cf41f1f20 2023-07-13T22:27:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cf259c700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf41f1f20 0x7f2cf40c72f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8011aa0 tx=0x7f2ce8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cdb7fe700 1 -- 172.21.15.60:0/432507908 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800f980 con 0x7f2cf41f1f20 2023-07-13T22:27:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cdb7fe700 1 -- 172.21.15.60:0/432507908 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce800fae0 con 0x7f2cf41f1f20 2023-07-13T22:27:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.701+0000 7f2cdb7fe700 1 -- 172.21.15.60:0/432507908 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800fe90 con 0x7f2cf41f1f20 2023-07-13T22:27:49.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.702+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf41f2c40 con 0x7f2cf41f1f20 2023-07-13T22:27:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.702+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf406ed00 con 0x7f2cf41f1f20 2023-07-13T22:27:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.703+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf41f3120 con 0x7f2cf41f1f20 2023-07-13T22:27:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.704+0000 7f2cdb7fe700 1 -- 172.21.15.60:0/432507908 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ce8014440 con 0x7f2cf41f1f20 2023-07-13T22:27:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.705+0000 7f2cdb7fe700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ce006ace0 0x7f2ce006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.705+0000 7f2cdb7fe700 1 -- 172.21.15.60:0/432507908 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f2ce8089d70 con 0x7f2cf41f1f20 2023-07-13T22:27:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.705+0000 7f2cf2d9d700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ce006ace0 0x7f2ce006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.706+0000 7f2cf2d9d700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ce006ace0 0x7f2ce006d190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40011d0 tx=0x7f2ce4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.709+0000 7f2cdb7fe700 1 -- 172.21.15.60:0/432507908 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2ce8058870 con 0x7f2cf41f1f20 2023-07-13T22:27:49.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:49.961+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2cf406ef50 con 0x7f2cf41f1f20 2023-07-13T22:27:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.457+0000 7f2cdb7fe700 1 -- 172.21.15.60:0/432507908 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2ce8058400 con 0x7f2cf41f1f20 2023-07-13T22:27:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ce006ace0 msgr2=0x7f2ce006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ce006ace0 0x7f2ce006d190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40011d0 tx=0x7f2ce4009040 comp rx=0 tx=0).stop 2023-07-13T22:27:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf41f1f20 msgr2=0x7f2cf40c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf41f1f20 0x7f2cf40c72f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8011aa0 tx=0x7f2ce8011e60 comp rx=0 tx=0).stop 2023-07-13T22:27:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 shutdown_connections 2023-07-13T22:27:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ce006ace0 0x7f2ce006d190 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cf40c7830 0x7f2cf406e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc004d90 0x7f2cf415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 --2- 172.21.15.60:0/432507908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf41f1f20 0x7f2cf40c72f0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 >> 172.21.15.60:0/432507908 conn(0x7f2cf406b900 msgr2=0x7f2cf40b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 shutdown_connections 2023-07-13T22:27:50.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.460+0000 7f2cf91ff700 1 -- 172.21.15.60:0/432507908 wait complete. 2023-07-13T22:27:50.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:27:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.691+0000 7fc8b5bff700 1 Processor -- start 2023-07-13T22:27:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.691+0000 7fc8b5bff700 1 -- start start 2023-07-13T22:27:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.692+0000 7fc8b5bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc8b00b45f0 0x7fc8b00b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.692+0000 7fc8b5bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b0160020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.692+0000 7fc8b5bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b00d0a70 con 0x7fc8b006b7a0 2023-07-13T22:27:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.692+0000 7fc8b5bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b015c920 con 0x7fc8b00b45f0 2023-07-13T22:27:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.692+0000 7fc8b5bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b00b4f00 con 0x7fc8b006b330 2023-07-13T22:27:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8af7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc8b00b45f0 0x7fc8b00b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:50.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8af7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc8b00b45f0 0x7fc8b00b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49208/0 (socket says 172.21.15.60:49208) 2023-07-13T22:27:50.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8af7fe700 1 -- 172.21.15.60:0/3071600943 learned_addr learned my addr 172.21.15.60:0/3071600943 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8aeffd700 1 --2- 172.21.15.60:0/3071600943 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b0160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8af7fe700 1 -- 172.21.15.60:0/3071600943 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 msgr2=0x7fc8b0160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8af7fe700 1 --2- 172.21.15.60:0/3071600943 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b0160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:50.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8af7fe700 1 -- 172.21.15.60:0/3071600943 >> v1:172.21.15.105:6789/0 conn(0x7fc8b006b7a0 legacy=0x7fc8b0160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8af7fe700 1 -- 172.21.15.60:0/3071600943 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b006bd10 con 0x7fc8b00b45f0 2023-07-13T22:27:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8aeffd700 1 --2- 172.21.15.60:0/3071600943 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b0160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8af7fe700 1 --2- 172.21.15.60:0/3071600943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b00b45f0 0x7fc8b00b49c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00098f0 tx=0x7fc8a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af33c8e224c77ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.693+0000 7fc8ae7fc700 1 -- 172.21.15.60:0/3071600943 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0004230 con 0x7fc8b00b45f0 2023-07-13T22:27:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8ae7fc700 1 -- 172.21.15.60:0/3071600943 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a0004390 con 0x7fc8b00b45f0 2023-07-13T22:27:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8ae7fc700 1 -- 172.21.15.60:0/3071600943 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a00036a0 con 0x7fc8b00b45f0 2023-07-13T22:27:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8b5bff700 1 -- 172.21.15.60:0/3071600943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b00b45f0 msgr2=0x7fc8b00b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8b5bff700 1 --2- 172.21.15.60:0/3071600943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b00b45f0 0x7fc8b00b49c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00098f0 tx=0x7fc8a0009c00 comp rx=0 tx=0).stop 2023-07-13T22:27:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8b5bff700 1 -- 172.21.15.60:0/3071600943 shutdown_connections 2023-07-13T22:27:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8b5bff700 1 --2- 172.21.15.60:0/3071600943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b00b45f0 0x7fc8b00b49c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8b5bff700 1 --2- 172.21.15.60:0/3071600943 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b0160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8b5bff700 1 -- 172.21.15.60:0/3071600943 >> 172.21.15.60:0/3071600943 conn(0x7fc8b00cf5f0 msgr2=0x7fc8b00cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8b5bff700 1 -- 172.21.15.60:0/3071600943 shutdown_connections 2023-07-13T22:27:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.694+0000 7fc8b5bff700 1 -- 172.21.15.60:0/3071600943 wait complete. 2023-07-13T22:27:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.696+0000 7fc8b5bff700 1 Processor -- start 2023-07-13T22:27:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.696+0000 7fc8b5bff700 1 -- start start 2023-07-13T22:27:50.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.697+0000 7fc8b5bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:50.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.697+0000 7fc8b5bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b006b7a0 0x7fc8b015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:50.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.697+0000 7fc8b5bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8b00b45f0 0x7fc8b015f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:50.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.697+0000 7fc8b5bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b015f1d0 con 0x7fc8b006b7a0 2023-07-13T22:27:50.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.697+0000 7fc8b5bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b015e320 con 0x7fc8b00b45f0 2023-07-13T22:27:50.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.697+0000 7fc8b5bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b015e5c0 con 0x7fc8b006b330 2023-07-13T22:27:50.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.697+0000 7fc8aeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b006b7a0 0x7fc8b015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:50.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.697+0000 7fc8aeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b006b7a0 0x7fc8b015dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49224/0 (socket says 172.21.15.60:49224) 2023-07-13T22:27:50.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.697+0000 7fc8aeffd700 1 -- 172.21.15.60:0/1088884779 learned_addr learned my addr 172.21.15.60:0/1088884779 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:50.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8af7fe700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:50.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8aeffd700 1 -- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 msgr2=0x7fc8b015cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:50.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8aeffd700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b015cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:50.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8affff700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8b00b45f0 0x7fc8b015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:50.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8aeffd700 1 -- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8b00b45f0 msgr2=0x7fc8b015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:50.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8aeffd700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8b00b45f0 0x7fc8b015f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:50.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8aeffd700 1 -- 172.21.15.60:0/1088884779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a0009550 con 0x7fc8b006b7a0 2023-07-13T22:27:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8af7fe700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8affff700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8b00b45f0 0x7fc8b015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8aeffd700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b006b7a0 0x7fc8b015dde0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c750 tx=0x7fc8a400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc897fff700 1 -- 172.21.15.60:0/1088884779 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4011800 con 0x7fc8b006b7a0 2023-07-13T22:27:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc897fff700 1 -- 172.21.15.60:0/1088884779 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a4011960 con 0x7fc8b006b7a0 2023-07-13T22:27:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc897fff700 1 -- 172.21.15.60:0/1088884779 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4011cd0 con 0x7fc8b006b7a0 2023-07-13T22:27:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.698+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b015e8a0 con 0x7fc8b006b7a0 2023-07-13T22:27:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.699+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b015ee60 con 0x7fc8b006b7a0 2023-07-13T22:27:50.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.700+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b00c9a90 con 0x7fc8b006b7a0 2023-07-13T22:27:50.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.700+0000 7fc897fff700 1 -- 172.21.15.60:0/1088884779 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc8a4021350 con 0x7fc8b006b7a0 2023-07-13T22:27:50.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.701+0000 7fc897fff700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc89c06ace0 0x7fc89c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:50.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.701+0000 7fc897fff700 1 -- 172.21.15.60:0/1088884779 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fc8a4089560 con 0x7fc8b006b7a0 2023-07-13T22:27:50.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.702+0000 7fc8af7fe700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc89c06ace0 0x7fc89c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:50.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.702+0000 7fc8af7fe700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc89c06ace0 0x7fc89c06d190 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc8b015d290 tx=0x7fc8a0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:50.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.706+0000 7fc897fff700 1 -- 172.21.15.60:0/1088884779 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc8a4057fa0 con 0x7fc8b006b7a0 2023-07-13T22:27:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:50.916+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8b004e210 con 0x7fc8b006b7a0 2023-07-13T22:27:51.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.587+0000 7fc897fff700 1 -- 172.21.15.60:0/1088884779 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc8a4057b30 con 0x7fc8b006b7a0 2023-07-13T22:27:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.591+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc89c06ace0 msgr2=0x7fc89c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.591+0000 7fc8b5bff700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc89c06ace0 0x7fc89c06d190 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc8b015d290 tx=0x7fc8a0008030 comp rx=0 tx=0).stop 2023-07-13T22:27:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.591+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b006b7a0 msgr2=0x7fc8b015dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.591+0000 7fc8b5bff700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b006b7a0 0x7fc8b015dde0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c750 tx=0x7fc8a400cb10 comp rx=0 tx=0).stop 2023-07-13T22:27:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.592+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 shutdown_connections 2023-07-13T22:27:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.592+0000 7fc8b5bff700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc89c06ace0 0x7fc89c06d190 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.592+0000 7fc8b5bff700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8b00b45f0 0x7fc8b015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.592+0000 7fc8b5bff700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc8b006b330 0x7fc8b015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.592+0000 7fc8b5bff700 1 --2- 172.21.15.60:0/1088884779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b006b7a0 0x7fc8b015dde0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.592+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 >> 172.21.15.60:0/1088884779 conn(0x7fc8b00cf5f0 msgr2=0x7fc8b00b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.592+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 shutdown_connections 2023-07-13T22:27:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.592+0000 7fc8b5bff700 1 -- 172.21.15.60:0/1088884779 wait complete. 2023-07-13T22:27:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:27:51.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:27:51.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.825+0000 7f26fcbff700 1 Processor -- start 2023-07-13T22:27:51.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.825+0000 7f26fcbff700 1 -- start start 2023-07-13T22:27:51.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.826+0000 7f26fcbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 0x7f26f80c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:51.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.826+0000 7f26fcbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26f80addb0 0x7f26f8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:51.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.826+0000 7f26fcbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26f8070890 0x7f26f806c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:51.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.826+0000 7f26fcbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f806de70 con 0x7f26f80addb0 2023-07-13T22:27:51.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.826+0000 7f26fcbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f806e290 con 0x7f26f8070890 2023-07-13T22:27:51.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.826+0000 7f26fcbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f8070ce0 con 0x7f26f815a530 2023-07-13T22:27:51.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.826+0000 7f26f5d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26f80addb0 0x7f26f8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:51.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f5d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26f80addb0 0x7f26f8070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49230/0 (socket says 172.21.15.60:49230) 2023-07-13T22:27:51.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f5d9b700 1 -- 172.21.15.60:0/4230115245 learned_addr learned my addr 172.21.15.60:0/4230115245 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:51.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f659c700 1 --2- 172.21.15.60:0/4230115245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 0x7f26f80c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:51.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f5d9b700 1 -- 172.21.15.60:0/4230115245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 msgr2=0x7f26f80c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f6d9d700 1 --2- 172.21.15.60:0/4230115245 >> v2:172.21.15.105:3300/0 conn(0x7f26f8070890 0x7f26f806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f5d9b700 1 --2- 172.21.15.60:0/4230115245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 0x7f26f80c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f5d9b700 1 -- 172.21.15.60:0/4230115245 >> v2:172.21.15.105:3300/0 conn(0x7f26f8070890 msgr2=0x7f26f806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f5d9b700 1 --2- 172.21.15.60:0/4230115245 >> v2:172.21.15.105:3300/0 conn(0x7f26f8070890 0x7f26f806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f5d9b700 1 -- 172.21.15.60:0/4230115245 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f80c94a0 con 0x7f26f80addb0 2023-07-13T22:27:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f659c700 1 --2- 172.21.15.60:0/4230115245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 0x7f26f80c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f6d9d700 1 --2- 172.21.15.60:0/4230115245 >> v2:172.21.15.105:3300/0 conn(0x7f26f8070890 0x7f26f806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:51.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f5d9b700 1 --2- 172.21.15.60:0/4230115245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f80addb0 0x7f26f8070350 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f26f80712c0 tx=0x7f26ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1b7b3bae02cbe9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.827+0000 7f26f559a700 1 -- 172.21.15.60:0/4230115245 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00b870 con 0x7f26f80addb0 2023-07-13T22:27:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26f559a700 1 -- 172.21.15.60:0/4230115245 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec00b9d0 con 0x7f26f80addb0 2023-07-13T22:27:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26f559a700 1 -- 172.21.15.60:0/4230115245 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00bd60 con 0x7f26f80addb0 2023-07-13T22:27:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26fcbff700 1 -- 172.21.15.60:0/4230115245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f80addb0 msgr2=0x7f26f8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26fcbff700 1 --2- 172.21.15.60:0/4230115245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f80addb0 0x7f26f8070350 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f26f80712c0 tx=0x7f26ec009bd0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26fcbff700 1 -- 172.21.15.60:0/4230115245 shutdown_connections 2023-07-13T22:27:51.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26fcbff700 1 --2- 172.21.15.60:0/4230115245 >> v2:172.21.15.105:3300/0 conn(0x7f26f8070890 0x7f26f806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26fcbff700 1 --2- 172.21.15.60:0/4230115245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 0x7f26f80c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26fcbff700 1 --2- 172.21.15.60:0/4230115245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f80addb0 0x7f26f8070350 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26fcbff700 1 -- 172.21.15.60:0/4230115245 >> 172.21.15.60:0/4230115245 conn(0x7f26f80cf3b0 msgr2=0x7f26f80cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:51.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26fcbff700 1 -- 172.21.15.60:0/4230115245 shutdown_connections 2023-07-13T22:27:51.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.828+0000 7f26fcbff700 1 -- 172.21.15.60:0/4230115245 wait complete. 2023-07-13T22:27:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.831+0000 7f26fcbff700 1 Processor -- start 2023-07-13T22:27:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.831+0000 7f26fcbff700 1 -- start start 2023-07-13T22:27:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.832+0000 7f26fcbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f8070890 0x7f26f80c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.832+0000 7f26fcbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80addb0 0x7f26f8140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.832+0000 7f26fcbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 0x7f26f80b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.832+0000 7f26fcbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81406d0 con 0x7f26f8070890 2023-07-13T22:27:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.832+0000 7f26fcbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80cc450 con 0x7f26f80addb0 2023-07-13T22:27:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.832+0000 7f26fcbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80cc730 con 0x7f26f815a530 2023-07-13T22:27:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.832+0000 7f26f659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f8070890 0x7f26f80c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.832+0000 7f26f659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f8070890 0x7f26f80c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49236/0 (socket says 172.21.15.60:49236) 2023-07-13T22:27:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.832+0000 7f26f659c700 1 -- 172.21.15.60:0/1235886170 learned_addr learned my addr 172.21.15.60:0/1235886170 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26f5d9b700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80addb0 0x7f26f8140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26f6d9d700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 0x7f26f80b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26f659c700 1 -- 172.21.15.60:0/1235886170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 msgr2=0x7f26f80b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26f659c700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 0x7f26f80b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26f659c700 1 -- 172.21.15.60:0/1235886170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80addb0 msgr2=0x7f26f8140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26f659c700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80addb0 0x7f26f8140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26f659c700 1 -- 172.21.15.60:0/1235886170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec009550 con 0x7f26f8070890 2023-07-13T22:27:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26f5d9b700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80addb0 0x7f26f8140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26f659c700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f8070890 0x7f26f80c5820 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c750 tx=0x7f26e800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26e6ffd700 1 -- 172.21.15.60:0/1235886170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8011800 con 0x7f26f8070890 2023-07-13T22:27:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26e6ffd700 1 -- 172.21.15.60:0/1235886170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e8011960 con 0x7f26f8070890 2023-07-13T22:27:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26e6ffd700 1 -- 172.21.15.60:0/1235886170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8011cd0 con 0x7f26f8070890 2023-07-13T22:27:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f80cca10 con 0x7f26f8070890 2023-07-13T22:27:51.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.833+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f80ccfd0 con 0x7f26f8070890 2023-07-13T22:27:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.834+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f804e210 con 0x7f26f8070890 2023-07-13T22:27:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.835+0000 7f26e6ffd700 1 -- 172.21.15.60:0/1235886170 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26e8021350 con 0x7f26f8070890 2023-07-13T22:27:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.836+0000 7f26e6ffd700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26d806ace0 0x7f26d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.836+0000 7f26e6ffd700 1 -- 172.21.15.60:0/1235886170 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f26e8089330 con 0x7f26f8070890 2023-07-13T22:27:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.837+0000 7f26f5d9b700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26d806ace0 0x7f26d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.837+0000 7f26f5d9b700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26d806ace0 0x7f26d806d190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f26ec014040 tx=0x7f26ec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:51.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:51.840+0000 7f26e6ffd700 1 -- 172.21.15.60:0/1235886170 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f26e8054530 con 0x7f26f8070890 2023-07-13T22:27:52.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.048+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f26f80cd1c0 con 0x7f26f8070890 2023-07-13T22:27:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.590+0000 7f26e6ffd700 1 -- 172.21.15.60:0/1235886170 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 169+0+0 (secure 0 0 0) 0x7f26e8057b50 con 0x7f26f8070890 2023-07-13T22:27:52.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26d806ace0 msgr2=0x7f26d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:52.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26d806ace0 0x7f26d806d190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f26ec014040 tx=0x7f26ec008030 comp rx=0 tx=0).stop 2023-07-13T22:27:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f8070890 msgr2=0x7f26f80c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f8070890 0x7f26f80c5820 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c750 tx=0x7f26e800cb10 comp rx=0 tx=0).stop 2023-07-13T22:27:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 shutdown_connections 2023-07-13T22:27:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f26d806ace0 0x7f26d806d190 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26f8070890 0x7f26f80c5820 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26f815a530 0x7f26f80b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 --2- 172.21.15.60:0/1235886170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f80addb0 0x7f26f8140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.594+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 >> 172.21.15.60:0/1235886170 conn(0x7f26f80cf3b0 msgr2=0x7f26f80aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.595+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 shutdown_connections 2023-07-13T22:27:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.595+0000 7f26fcbff700 1 -- 172.21.15.60:0/1235886170 wait complete. 2023-07-13T22:27:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:27:52.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-07-13T22:27:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.821+0000 7f55003ff700 1 Processor -- start 2023-07-13T22:27:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.822+0000 7f55003ff700 1 -- start start 2023-07-13T22:27:52.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.823+0000 7f55003ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f54f80c90e0 0x7f54f8069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.823+0000 7f55003ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f806a410 0x7f54f806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.823+0000 7f55003ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f806d030 con 0x7f54f8158930 2023-07-13T22:27:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.823+0000 7f55003ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f806a890 con 0x7f54f80c90e0 2023-07-13T22:27:52.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f55003ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f806ca70 con 0x7f54f806a410 2023-07-13T22:27:52.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fd99a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f54f80c90e0 0x7f54f8069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:52.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fd99a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f54f80c90e0 0x7f54f8069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49238/0 (socket says 172.21.15.60:49238) 2023-07-13T22:27:52.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fd99a700 1 -- 172.21.15.60:0/3525530016 learned_addr learned my addr 172.21.15.60:0/3525530016 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:52.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fe19b700 1 --1- 172.21.15.60:0/3525530016 >> v1:172.21.15.60:6789/0 conn(0x7f54f8158930 0x7f54f815c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38354/0 (socket says 172.21.15.60:38354) 2023-07-13T22:27:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fe99c700 1 --2- 172.21.15.60:0/3525530016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f806a410 0x7f54f806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fd99a700 1 -- 172.21.15.60:0/3525530016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f806a410 msgr2=0x7f54f806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fd99a700 1 --2- 172.21.15.60:0/3525530016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f806a410 0x7f54f806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fd99a700 1 -- 172.21.15.60:0/3525530016 >> v1:172.21.15.60:6789/0 conn(0x7f54f8158930 legacy=0x7f54f815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fd199700 1 -- 172.21.15.60:0/3525530016 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239187187 0 0) 0x7f54f806d030 con 0x7f54f8158930 2023-07-13T22:27:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.824+0000 7f54fd99a700 1 -- 172.21.15.60:0/3525530016 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80adc20 con 0x7f54f80c90e0 2023-07-13T22:27:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f54fd99a700 1 --2- 172.21.15.60:0/3525530016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f80c90e0 0x7f54f8069ed0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f54e800bb90 tx=0x7f54e800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce291c14a1485b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f54fd199700 1 -- 172.21.15.60:0/3525530016 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800dd50 con 0x7f54f80c90e0 2023-07-13T22:27:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f54fd199700 1 -- 172.21.15.60:0/3525530016 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e8010470 con 0x7f54f80c90e0 2023-07-13T22:27:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f54fd199700 1 -- 172.21.15.60:0/3525530016 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8010800 con 0x7f54f80c90e0 2023-07-13T22:27:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f55003ff700 1 -- 172.21.15.60:0/3525530016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f80c90e0 msgr2=0x7f54f8069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f55003ff700 1 --2- 172.21.15.60:0/3525530016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f80c90e0 0x7f54f8069ed0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f54e800bb90 tx=0x7f54e800bea0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f55003ff700 1 -- 172.21.15.60:0/3525530016 shutdown_connections 2023-07-13T22:27:52.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f55003ff700 1 --2- 172.21.15.60:0/3525530016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f806a410 0x7f54f806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f55003ff700 1 --2- 172.21.15.60:0/3525530016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f80c90e0 0x7f54f8069ed0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.825+0000 7f55003ff700 1 -- 172.21.15.60:0/3525530016 >> 172.21.15.60:0/3525530016 conn(0x7f54f80cbf50 msgr2=0x7f54f80ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:52.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.826+0000 7f55003ff700 1 -- 172.21.15.60:0/3525530016 shutdown_connections 2023-07-13T22:27:52.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.826+0000 7f55003ff700 1 -- 172.21.15.60:0/3525530016 wait complete. 2023-07-13T22:27:52.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.827+0000 7f55003ff700 1 Processor -- start 2023-07-13T22:27:52.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.827+0000 7f55003ff700 1 -- start start 2023-07-13T22:27:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.828+0000 7f55003ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f806a410 0x7f54f80c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.828+0000 7f55003ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f80c90e0 0x7f54f80ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.828+0000 7f55003ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f8158930 0x7f54f80b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:52.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.828+0000 7f55003ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80c8540 con 0x7f54f806a410 2023-07-13T22:27:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.828+0000 7f55003ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80ac1f0 con 0x7f54f80c90e0 2023-07-13T22:27:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.828+0000 7f55003ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80ac4d0 con 0x7f54f8158930 2023-07-13T22:27:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f806a410 0x7f54f80c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f806a410 0x7f54f80c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49240/0 (socket says 172.21.15.60:49240) 2023-07-13T22:27:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe19b700 1 -- 172.21.15.60:0/3346328198 learned_addr learned my addr 172.21.15.60:0/3346328198 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fd99a700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f80c90e0 0x7f54f80ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:52.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe99c700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f8158930 0x7f54f80b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:52.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe19b700 1 -- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f8158930 msgr2=0x7f54f80b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:52.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe19b700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f8158930 0x7f54f80b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe19b700 1 -- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f80c90e0 msgr2=0x7f54f80ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:52.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe19b700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f80c90e0 0x7f54f80ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:52.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe19b700 1 -- 172.21.15.60:0/3346328198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e800b7f0 con 0x7f54f806a410 2023-07-13T22:27:52.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe99c700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f8158930 0x7f54f80b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:52.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fd99a700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f80c90e0 0x7f54f80ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:52.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54fe19b700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f806a410 0x7f54f80c8000 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f54f400c550 tx=0x7f54f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:52.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54eeffd700 1 -- 172.21.15.60:0/3346328198 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4004c80 con 0x7f54f806a410 2023-07-13T22:27:52.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54eeffd700 1 -- 172.21.15.60:0/3346328198 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f4004e00 con 0x7f54f806a410 2023-07-13T22:27:52.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.829+0000 7f54eeffd700 1 -- 172.21.15.60:0/3346328198 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4007310 con 0x7f54f806a410 2023-07-13T22:27:52.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.830+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f80ac7b0 con 0x7f54f806a410 2023-07-13T22:27:52.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.830+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f80abe30 con 0x7f54f806a410 2023-07-13T22:27:52.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.831+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f814d950 con 0x7f54f806a410 2023-07-13T22:27:52.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.831+0000 7f54eeffd700 1 -- 172.21.15.60:0/3346328198 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54f4007470 con 0x7f54f806a410 2023-07-13T22:27:52.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.832+0000 7f54eeffd700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f54dc06ace0 0x7f54dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:52.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.832+0000 7f54eeffd700 1 -- 172.21.15.60:0/3346328198 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f54f4088eb0 con 0x7f54f806a410 2023-07-13T22:27:52.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.833+0000 7f54fd99a700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f54dc06ace0 0x7f54dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:52.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.833+0000 7f54fd99a700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f54dc06ace0 0x7f54dc06d190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f54f80b1030 tx=0x7f54e8013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:52.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:52.837+0000 7f54eeffd700 1 -- 172.21.15.60:0/3346328198 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f54f4054270 con 0x7f54f806a410 2023-07-13T22:27:53.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.052+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7f54f80ac9a0 con 0x7f54f806a410 2023-07-13T22:27:53.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.056+0000 7f54eeffd700 1 -- 172.21.15.60:0/3346328198 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v27) v1 ==== 102+0+0 (secure 0 0 0) 0x7f54f4057890 con 0x7f54f806a410 2023-07-13T22:27:53.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f54dc06ace0 msgr2=0x7f54dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f54dc06ace0 0x7f54dc06d190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f54f80b1030 tx=0x7f54e8013000 comp rx=0 tx=0).stop 2023-07-13T22:27:53.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f806a410 msgr2=0x7f54f80c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f806a410 0x7f54f80c8000 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f54f400c550 tx=0x7f54f400c910 comp rx=0 tx=0).stop 2023-07-13T22:27:53.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 shutdown_connections 2023-07-13T22:27:53.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f54dc06ace0 0x7f54dc06d190 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f806a410 0x7f54f80c8000 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54f8158930 0x7f54f80b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 --2- 172.21.15.60:0/3346328198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f80c90e0 0x7f54f80ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 >> 172.21.15.60:0/3346328198 conn(0x7f54f80cbf50 msgr2=0x7f54f81569f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:53.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 shutdown_connections 2023-07-13T22:27:53.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.059+0000 7f55003ff700 1 -- 172.21.15.60:0/3346328198 wait complete. 2023-07-13T22:27:53.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-13T22:27:53.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.282+0000 7f7f83bff700 1 Processor -- start 2023-07-13T22:27:53.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.282+0000 7f7f83bff700 1 -- start start 2023-07-13T22:27:53.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.283+0000 7f7f83bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f7c0cfd50 0x7f7f7c0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f83bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f83bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c0d1870 con 0x7f7f7c0b8200 2023-07-13T22:27:53.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f83bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c0cf370 con 0x7f7f7c0cfd50 2023-07-13T22:27:53.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f83bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0d12b0 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f8119a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f7c0cfd50 0x7f7f7c0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f8119a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f7c0cfd50 0x7f7f7c0d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49246/0 (socket says 172.21.15.60:49246) 2023-07-13T22:27:53.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f8119a700 1 -- 172.21.15.60:0/3857864113 learned_addr learned my addr 172.21.15.60:0/3857864113 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:53.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f8219c700 1 --2- 172.21.15.60:0/3857864113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f8119a700 1 -- 172.21.15.60:0/3857864113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b1ac0 msgr2=0x7f7f7c0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f8119a700 1 --2- 172.21.15.60:0/3857864113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f8119a700 1 -- 172.21.15.60:0/3857864113 >> v1:172.21.15.105:6789/0 conn(0x7f7f7c0b8200 legacy=0x7f7f7c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.284+0000 7f7f8119a700 1 -- 172.21.15.60:0/3857864113 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c0d1590 con 0x7f7f7c0cfd50 2023-07-13T22:27:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f8119a700 1 --2- 172.21.15.60:0/3857864113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0cfd50 0x7f7f7c0d23c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0098f0 tx=0x7f7f6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38359f51b007c0a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f80999700 1 -- 172.21.15.60:0/3857864113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011830 con 0x7f7f7c0cfd50 2023-07-13T22:27:53.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f80999700 1 -- 172.21.15.60:0/3857864113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c011990 con 0x7f7f7c0cfd50 2023-07-13T22:27:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f80999700 1 -- 172.21.15.60:0/3857864113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011d20 con 0x7f7f7c0cfd50 2023-07-13T22:27:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f83bff700 1 -- 172.21.15.60:0/3857864113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0cfd50 msgr2=0x7f7f7c0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f83bff700 1 --2- 172.21.15.60:0/3857864113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0cfd50 0x7f7f7c0d23c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0098f0 tx=0x7f7f6c009c00 comp rx=0 tx=0).stop 2023-07-13T22:27:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f83bff700 1 -- 172.21.15.60:0/3857864113 shutdown_connections 2023-07-13T22:27:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f83bff700 1 --2- 172.21.15.60:0/3857864113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f83bff700 1 --2- 172.21.15.60:0/3857864113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0cfd50 0x7f7f7c0d23c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f83bff700 1 -- 172.21.15.60:0/3857864113 >> 172.21.15.60:0/3857864113 conn(0x7f7f7c06b2f0 msgr2=0x7f7f7c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.285+0000 7f7f83bff700 1 -- 172.21.15.60:0/3857864113 shutdown_connections 2023-07-13T22:27:53.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.286+0000 7f7f83bff700 1 -- 172.21.15.60:0/3857864113 wait complete. 2023-07-13T22:27:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.287+0000 7f7f83bff700 1 Processor -- start 2023-07-13T22:27:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.287+0000 7f7f83bff700 1 -- start start 2023-07-13T22:27:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.288+0000 7f7f83bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.288+0000 7f7f83bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b8200 0x7f7f7c15e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.288+0000 7f7f83bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f7c0cfd50 0x7f7f7c15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.288+0000 7f7f83bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c6ca0 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.288+0000 7f7f83bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c6f70 con 0x7f7f7c0cfd50 2023-07-13T22:27:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f83bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c7250 con 0x7f7f7c0b8200 2023-07-13T22:27:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49252/0 (socket says 172.21.15.60:49252) 2023-07-13T22:27:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8199b700 1 -- 172.21.15.60:0/1619076501 learned_addr learned my addr 172.21.15.60:0/1619076501 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:53.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8119a700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b8200 0x7f7f7c15e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8219c700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f7c0cfd50 0x7f7f7c15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8199b700 1 -- 172.21.15.60:0/1619076501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b8200 msgr2=0x7f7f7c15e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8199b700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b8200 0x7f7f7c15e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8199b700 1 -- 172.21.15.60:0/1619076501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f7c0cfd50 msgr2=0x7f7f7c15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8199b700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f7c0cfd50 0x7f7f7c15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8199b700 1 -- 172.21.15.60:0/1619076501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c009550 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8119a700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b8200 0x7f7f7c15e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:53.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.289+0000 7f7f8199b700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0c6760 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800c4e0 tx=0x7f7f7800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:53.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.290+0000 7f7f727fc700 1 -- 172.21.15.60:0/1619076501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780052d0 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.290+0000 7f7f727fc700 1 -- 172.21.15.60:0/1619076501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78005430 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.290+0000 7f7f727fc700 1 -- 172.21.15.60:0/1619076501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780038c0 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.290+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c0c7530 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.290+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c0c7af0 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.291+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c04e210 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.292+0000 7f7f727fc700 1 -- 172.21.15.60:0/1619076501 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f78003a20 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.293+0000 7f7f727fc700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f6006ace0 0x7f7f6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.293+0000 7f7f727fc700 1 -- 172.21.15.60:0/1619076501 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7f78088690 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.293+0000 7f7f8119a700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f6006ace0 0x7f7f6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.294+0000 7f7f8119a700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f6006ace0 0x7f7f6006d190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0d0140 tx=0x7f7f6c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.297+0000 7f7f727fc700 1 -- 172.21.15.60:0/1619076501 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7f78053a50 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.505+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7f7c0b4f70 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.508+0000 7f7f727fc700 1 -- 172.21.15.60:0/1619076501 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7f78057070 con 0x7f7f7c0b1ac0 2023-07-13T22:27:53.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f6006ace0 msgr2=0x7f7f6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f6006ace0 0x7f7f6006d190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0d0140 tx=0x7f7f6c008030 comp rx=0 tx=0).stop 2023-07-13T22:27:53.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0b1ac0 msgr2=0x7f7f7c0c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0c6760 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800c4e0 tx=0x7f7f7800c8a0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 shutdown_connections 2023-07-13T22:27:53.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f6006ace0 0x7f7f6006d190 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f7c0b1ac0 0x7f7f7c0c6760 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f7c0cfd50 0x7f7f7c15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 --2- 172.21.15.60:0/1619076501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f7c0b8200 0x7f7f7c15e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 >> 172.21.15.60:0/1619076501 conn(0x7f7f7c06b2f0 msgr2=0x7f7f7c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:53.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 shutdown_connections 2023-07-13T22:27:53.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.511+0000 7f7f83bff700 1 -- 172.21.15.60:0/1619076501 wait complete. 2023-07-13T22:27:53.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-07-13T22:27:53.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.736+0000 7fcd9a3ff700 1 Processor -- start 2023-07-13T22:27:53.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.736+0000 7fcd9a3ff700 1 -- start start 2023-07-13T22:27:53.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.737+0000 7fcd9a3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 0x7fcd940ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.737+0000 7fcd9a3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcd940cc410 0x7fcd940cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.737+0000 7fcd9a3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd940703e0 0x7fcd94070840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.737+0000 7fcd9a3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd94070d80 con 0x7fcd940703e0 2023-07-13T22:27:53.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.737+0000 7fcd9a3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940cf640 con 0x7fcd940cc410 2023-07-13T22:27:53.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.737+0000 7fcd9a3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940ccf80 con 0x7fcd94159260 2023-07-13T22:27:53.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd9899c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd940703e0 0x7fcd94070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd9899c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd940703e0 0x7fcd94070840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49262/0 (socket says 172.21.15.60:49262) 2023-07-13T22:27:53.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd9899c700 1 -- 172.21.15.60:0/3361656402 learned_addr learned my addr 172.21.15.60:0/3361656402 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:53.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd93fff700 1 --2- 172.21.15.60:0/3361656402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 0x7fcd940ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd937fe700 1 --2- 172.21.15.60:0/3361656402 >> v2:172.21.15.105:3300/0 conn(0x7fcd940cc410 0x7fcd940cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd9899c700 1 -- 172.21.15.60:0/3361656402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 msgr2=0x7fcd940ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd9899c700 1 --2- 172.21.15.60:0/3361656402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 0x7fcd940ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd9899c700 1 -- 172.21.15.60:0/3361656402 >> v2:172.21.15.105:3300/0 conn(0x7fcd940cc410 msgr2=0x7fcd940cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd9899c700 1 --2- 172.21.15.60:0/3361656402 >> v2:172.21.15.105:3300/0 conn(0x7fcd940cc410 0x7fcd940cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd9899c700 1 -- 172.21.15.60:0/3361656402 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd940c9b40 con 0x7fcd940703e0 2023-07-13T22:27:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.738+0000 7fcd9899c700 1 --2- 172.21.15.60:0/3361656402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940703e0 0x7fcd94070840 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fcd9406f200 tx=0x7fcd8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a471961da8bcf1e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd92ffd700 1 -- 172.21.15.60:0/3361656402 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00e070 con 0x7fcd940703e0 2023-07-13T22:27:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd92ffd700 1 -- 172.21.15.60:0/3361656402 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00b9c0 con 0x7fcd940703e0 2023-07-13T22:27:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd92ffd700 1 -- 172.21.15.60:0/3361656402 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00bd70 con 0x7fcd940703e0 2023-07-13T22:27:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/3361656402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940703e0 msgr2=0x7fcd94070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/3361656402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940703e0 0x7fcd94070840 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fcd9406f200 tx=0x7fcd8c00b330 comp rx=0 tx=0).stop 2023-07-13T22:27:53.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/3361656402 shutdown_connections 2023-07-13T22:27:53.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/3361656402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940703e0 0x7fcd94070840 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/3361656402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 0x7fcd940ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/3361656402 >> v2:172.21.15.105:3300/0 conn(0x7fcd940cc410 0x7fcd940cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/3361656402 >> 172.21.15.60:0/3361656402 conn(0x7fcd940cbf50 msgr2=0x7fcd940ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:53.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/3361656402 shutdown_connections 2023-07-13T22:27:53.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.739+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/3361656402 wait complete. 2023-07-13T22:27:53.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.741+0000 7fcd9a3ff700 1 Processor -- start 2023-07-13T22:27:53.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.741+0000 7fcd9a3ff700 1 -- start start 2023-07-13T22:27:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.742+0000 7fcd9a3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd940703e0 0x7fcd940c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.742+0000 7fcd9a3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940cc410 0x7fcd94150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.742+0000 7fcd9a3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 0x7fcd9415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.742+0000 7fcd9a3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940aeea0 con 0x7fcd940cc410 2023-07-13T22:27:53.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.742+0000 7fcd9a3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940ae290 con 0x7fcd940703e0 2023-07-13T22:27:53.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.742+0000 7fcd9a3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940ae570 con 0x7fcd94159260 2023-07-13T22:27:53.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.742+0000 7fcd937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940cc410 0x7fcd94150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940cc410 0x7fcd94150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49264/0 (socket says 172.21.15.60:49264) 2023-07-13T22:27:53.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd937fe700 1 -- 172.21.15.60:0/2069597100 learned_addr learned my addr 172.21.15.60:0/2069597100 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:53.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd93fff700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd940703e0 0x7fcd940c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd9899c700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 0x7fcd9415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd937fe700 1 -- 172.21.15.60:0/2069597100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 msgr2=0x7fcd9415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd937fe700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 0x7fcd9415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd937fe700 1 -- 172.21.15.60:0/2069597100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd940703e0 msgr2=0x7fcd940c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd937fe700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd940703e0 0x7fcd940c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd937fe700 1 -- 172.21.15.60:0/2069597100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c009b30 con 0x7fcd940cc410 2023-07-13T22:27:53.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd937fe700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940cc410 0x7fcd94150b40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800e810 tx=0x7fcd8800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:53.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd90ff9700 1 -- 172.21.15.60:0/2069597100 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800c070 con 0x7fcd940cc410 2023-07-13T22:27:53.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd90ff9700 1 -- 172.21.15.60:0/2069597100 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd88013610 con 0x7fcd940cc410 2023-07-13T22:27:53.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd940ae850 con 0x7fcd940cc410 2023-07-13T22:27:53.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.743+0000 7fcd90ff9700 1 -- 172.21.15.60:0/2069597100 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd880139c0 con 0x7fcd940cc410 2023-07-13T22:27:53.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.744+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9415b560 con 0x7fcd940cc410 2023-07-13T22:27:53.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.745+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd940af050 con 0x7fcd940cc410 2023-07-13T22:27:53.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.746+0000 7fcd90ff9700 1 -- 172.21.15.60:0/2069597100 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd88013b20 con 0x7fcd940cc410 2023-07-13T22:27:53.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.746+0000 7fcd90ff9700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8006ace0 0x7fcd8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:53.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.747+0000 7fcd90ff9700 1 -- 172.21.15.60:0/2069597100 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcd88088ef0 con 0x7fcd940cc410 2023-07-13T22:27:53.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.747+0000 7fcd93fff700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8006ace0 0x7fcd8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:53.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.747+0000 7fcd93fff700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8006ace0 0x7fcd8006d190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcd84006b20 tx=0x7fcd84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:53.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.751+0000 7fcd90ff9700 1 -- 172.21.15.60:0/2069597100 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcd940af050 con 0x7fcd940cc410 2023-07-13T22:27:53.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.958+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7fcd9404e210 con 0x7fcd940cc410 2023-07-13T22:27:53.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.962+0000 7fcd90ff9700 1 -- 172.21.15.60:0/2069597100 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v29) v1 ==== 105+0+0 (secure 0 0 0) 0x7fcd88057d40 con 0x7fcd940cc410 2023-07-13T22:27:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8006ace0 msgr2=0x7fcd8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8006ace0 0x7fcd8006d190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcd84006b20 tx=0x7fcd84008040 comp rx=0 tx=0).stop 2023-07-13T22:27:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940cc410 msgr2=0x7fcd94150b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940cc410 0x7fcd94150b40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800e810 tx=0x7fcd8800eb20 comp rx=0 tx=0).stop 2023-07-13T22:27:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 shutdown_connections 2023-07-13T22:27:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd8006ace0 0x7fcd8006d190 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd94159260 0x7fcd9415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd940703e0 0x7fcd940c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 --2- 172.21.15.60:0/2069597100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd940cc410 0x7fcd94150b40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 >> 172.21.15.60:0/2069597100 conn(0x7fcd940cbf50 msgr2=0x7fcd940b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 shutdown_connections 2023-07-13T22:27:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:53.965+0000 7fcd9a3ff700 1 -- 172.21.15.60:0/2069597100 wait complete. 2023-07-13T22:27:53.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-07-13T22:27:54.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.187+0000 7f617f5ff700 1 Processor -- start 2023-07-13T22:27:54.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.188+0000 7f617f5ff700 1 -- start start 2023-07-13T22:27:54.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.189+0000 7f617f5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61780ac490 0x7f61780ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.189+0000 7f617f5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780cf6d0 con 0x7f61780ad9a0 2023-07-13T22:27:54.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.189+0000 7f617f5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780ac900 con 0x7f617815c790 2023-07-13T22:27:54.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.189+0000 7f617f5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61780acbe0 con 0x7f61780ac490 2023-07-13T22:27:54.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.189+0000 7f617db9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f61780ad9a0 0x7f61780adda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38364/0 (socket says 172.21.15.60:38364) 2023-07-13T22:27:54.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.189+0000 7f617db9c700 1 -- 172.21.15.60:0/2912430713 learned_addr learned my addr 172.21.15.60:0/2912430713 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:54.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.189+0000 7f617cb9a700 1 --2- 172.21.15.60:0/2912430713 >> v2:172.21.15.105:3300/0 conn(0x7f61780ac490 0x7f61780ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.190+0000 7f616ffff700 1 -- 172.21.15.60:0/2912430713 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380472093 0 0) 0x7f61780cf6d0 con 0x7f61780ad9a0 2023-07-13T22:27:54.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.190+0000 7f616ffff700 1 -- 172.21.15.60:0/2912430713 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61640035b0 con 0x7f61780ad9a0 2023-07-13T22:27:54.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.190+0000 7f616ffff700 1 -- 172.21.15.60:0/2912430713 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 592582284 0 0) 0x7f61640035b0 con 0x7f61780ad9a0 2023-07-13T22:27:54.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.190+0000 7f617cb9a700 1 -- 172.21.15.60:0/2912430713 >> v1:172.21.15.105:6789/0 conn(0x7f617815c790 legacy=0x7f61780c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.190+0000 7f617cb9a700 1 -- 172.21.15.60:0/2912430713 >> v1:172.21.15.60:6789/0 conn(0x7f61780ad9a0 legacy=0x7f61780adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.190+0000 7f617cb9a700 1 -- 172.21.15.60:0/2912430713 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780acec0 con 0x7f61780ac490 2023-07-13T22:27:54.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.190+0000 7f617cb9a700 1 --2- 172.21.15.60:0/2912430713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61780ac490 0x7f61780ce4b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6168009840 tx=0x7f6168009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=472e139e275cd548 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:54.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f616ffff700 1 -- 172.21.15.60:0/2912430713 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700019f0 con 0x7f61780ac490 2023-07-13T22:27:54.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f616ffff700 1 -- 172.21.15.60:0/2912430713 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6168004030 con 0x7f61780ac490 2023-07-13T22:27:54.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f616ffff700 1 -- 172.21.15.60:0/2912430713 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680043c0 con 0x7f61780ac490 2023-07-13T22:27:54.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f617f5ff700 1 -- 172.21.15.60:0/2912430713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61780ac490 msgr2=0x7f61780ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f617f5ff700 1 --2- 172.21.15.60:0/2912430713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61780ac490 0x7f61780ce4b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6168009840 tx=0x7f6168009b50 comp rx=0 tx=0).stop 2023-07-13T22:27:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f617f5ff700 1 -- 172.21.15.60:0/2912430713 shutdown_connections 2023-07-13T22:27:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f617f5ff700 1 --2- 172.21.15.60:0/2912430713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61780ac490 0x7f61780ce4b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f617f5ff700 1 -- 172.21.15.60:0/2912430713 >> 172.21.15.60:0/2912430713 conn(0x7f617806b900 msgr2=0x7f61780cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f617f5ff700 1 -- 172.21.15.60:0/2912430713 shutdown_connections 2023-07-13T22:27:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.191+0000 7f617f5ff700 1 -- 172.21.15.60:0/2912430713 wait complete. 2023-07-13T22:27:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.193+0000 7f617f5ff700 1 Processor -- start 2023-07-13T22:27:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.193+0000 7f617f5ff700 1 -- start start 2023-07-13T22:27:54.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.194+0000 7f617f5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61780ac490 0x7f61780b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.194+0000 7f617f5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61780ad9a0 0x7f617806d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.194+0000 7f617f5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f617815c790 0x7f61781f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.194+0000 7f617f5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61780b37e0 con 0x7f61780ad9a0 2023-07-13T22:27:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.194+0000 7f617f5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617815ae80 con 0x7f617815c790 2023-07-13T22:27:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.194+0000 7f617f5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61780b3220 con 0x7f61780ac490 2023-07-13T22:27:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.194+0000 7f617cb9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61780ad9a0 0x7f617806d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.194+0000 7f617cb9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61780ad9a0 0x7f617806d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49272/0 (socket says 172.21.15.60:49272) 2023-07-13T22:27:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.194+0000 7f617cb9a700 1 -- 172.21.15.60:0/3266457002 learned_addr learned my addr 172.21.15.60:0/3266457002 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.195+0000 7f617d39b700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61780ac490 0x7f61780b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.195+0000 7f617db9c700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f617815c790 0x7f61781f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.195+0000 7f617cb9a700 1 -- 172.21.15.60:0/3266457002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61780ac490 msgr2=0x7f61780b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.195+0000 7f617cb9a700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61780ac490 0x7f61780b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.195+0000 7f617cb9a700 1 -- 172.21.15.60:0/3266457002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f617815c790 msgr2=0x7f61781f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.195+0000 7f617cb9a700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f617815c790 0x7f61781f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.195+0000 7f617cb9a700 1 -- 172.21.15.60:0/3266457002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61680094a0 con 0x7f61780ad9a0 2023-07-13T22:27:54.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.195+0000 7f617cb9a700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61780ad9a0 0x7f617806d800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6168009b90 tx=0x7f6168004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:54.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.195+0000 7f616dffb700 1 -- 172.21.15.60:0/3266457002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168011630 con 0x7f61780ad9a0 2023-07-13T22:27:54.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.196+0000 7f616dffb700 1 -- 172.21.15.60:0/3266457002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616801f070 con 0x7f61780ad9a0 2023-07-13T22:27:54.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.196+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61780b3500 con 0x7f61780ad9a0 2023-07-13T22:27:54.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.196+0000 7f616dffb700 1 -- 172.21.15.60:0/3266457002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680119d0 con 0x7f61780ad9a0 2023-07-13T22:27:54.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.196+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617815df80 con 0x7f61780ad9a0 2023-07-13T22:27:54.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.197+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617804e210 con 0x7f61780ad9a0 2023-07-13T22:27:54.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.198+0000 7f616dffb700 1 -- 172.21.15.60:0/3266457002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6168011b30 con 0x7f61780ad9a0 2023-07-13T22:27:54.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.199+0000 7f616dffb700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f615c06ace0 0x7f615c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.199+0000 7f616dffb700 1 -- 172.21.15.60:0/3266457002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6168021070 con 0x7f61780ad9a0 2023-07-13T22:27:54.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.199+0000 7f617d39b700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f615c06ace0 0x7f615c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.200+0000 7f617d39b700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f615c06ace0 0x7f615c06d190 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f61740017b0 tx=0x7f6174009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:54.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.203+0000 7f616dffb700 1 -- 172.21.15.60:0/3266457002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6168055f20 con 0x7f61780ad9a0 2023-07-13T22:27:54.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.409+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7f61780720e0 con 0x7f61780ad9a0 2023-07-13T22:27:54.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.413+0000 7f616dffb700 1 -- 172.21.15.60:0/3266457002 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v30) v1 ==== 111+0+0 (secure 0 0 0) 0x7f616802b0b0 con 0x7f61780ad9a0 2023-07-13T22:27:54.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.415+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f615c06ace0 msgr2=0x7f615c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.415+0000 7f617f5ff700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f615c06ace0 0x7f615c06d190 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f61740017b0 tx=0x7f6174009040 comp rx=0 tx=0).stop 2023-07-13T22:27:54.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.415+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61780ad9a0 msgr2=0x7f617806d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.415+0000 7f617f5ff700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61780ad9a0 0x7f617806d800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6168009b90 tx=0x7f6168004750 comp rx=0 tx=0).stop 2023-07-13T22:27:54.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.416+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 shutdown_connections 2023-07-13T22:27:54.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.416+0000 7f617f5ff700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f615c06ace0 0x7f615c06d190 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.416+0000 7f617f5ff700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f617815c790 0x7f61781f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.416+0000 7f617f5ff700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61780ac490 0x7f61780b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.416+0000 7f617f5ff700 1 --2- 172.21.15.60:0/3266457002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61780ad9a0 0x7f617806d800 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.416+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 >> 172.21.15.60:0/3266457002 conn(0x7f617806b900 msgr2=0x7f61780cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:54.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.416+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 shutdown_connections 2023-07-13T22:27:54.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.416+0000 7f617f5ff700 1 -- 172.21.15.60:0/3266457002 wait complete. 2023-07-13T22:27:54.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-07-13T22:27:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.640+0000 7fbd361ff700 1 Processor -- start 2023-07-13T22:27:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.640+0000 7fbd361ff700 1 -- start start 2023-07-13T22:27:54.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.641+0000 7fbd361ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd3006a5e0 0x7fbd3006ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.641+0000 7fbd361ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd300ca9f0 0x7fbd300b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.641+0000 7fbd361ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd300c8e90 con 0x7fbd300add00 2023-07-13T22:27:54.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.641+0000 7fbd361ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3006d3b0 con 0x7fbd3006a5e0 2023-07-13T22:27:54.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.641+0000 7fbd361ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd300ae0d0 con 0x7fbd300ca9f0 2023-07-13T22:27:54.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.641+0000 7fbd2effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd3006a5e0 0x7fbd3006ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.641+0000 7fbd2effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd3006a5e0 0x7fbd3006ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49280/0 (socket says 172.21.15.60:49280) 2023-07-13T22:27:54.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.641+0000 7fbd2effd700 1 -- 172.21.15.60:0/2681715791 learned_addr learned my addr 172.21.15.60:0/2681715791 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:54.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2ffff700 1 --2- 172.21.15.60:0/2681715791 >> v2:172.21.15.105:3300/0 conn(0x7fbd300ca9f0 0x7fbd300b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2effd700 1 -- 172.21.15.60:0/2681715791 >> v2:172.21.15.105:3300/0 conn(0x7fbd300ca9f0 msgr2=0x7fbd300b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:54.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2effd700 1 --2- 172.21.15.60:0/2681715791 >> v2:172.21.15.105:3300/0 conn(0x7fbd300ca9f0 0x7fbd300b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2effd700 1 -- 172.21.15.60:0/2681715791 >> v1:172.21.15.105:6789/0 conn(0x7fbd300add00 legacy=0x7fbd30069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:54.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2effd700 1 -- 172.21.15.60:0/2681715791 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd300c90e0 con 0x7fbd3006a5e0 2023-07-13T22:27:54.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2ffff700 1 --2- 172.21.15.60:0/2681715791 >> v2:172.21.15.105:3300/0 conn(0x7fbd300ca9f0 0x7fbd300b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:54.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2effd700 1 --2- 172.21.15.60:0/2681715791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3006a5e0 0x7fbd3006ce70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbd3006dad0 tx=0x7fbd24009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b719ebdc3132cc3f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:54.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2e7fc700 1 -- 172.21.15.60:0/2681715791 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24004230 con 0x7fbd3006a5e0 2023-07-13T22:27:54.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2e7fc700 1 -- 172.21.15.60:0/2681715791 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd24004390 con 0x7fbd3006a5e0 2023-07-13T22:27:54.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.642+0000 7fbd2e7fc700 1 -- 172.21.15.60:0/2681715791 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd240036a0 con 0x7fbd3006a5e0 2023-07-13T22:27:54.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.643+0000 7fbd361ff700 1 -- 172.21.15.60:0/2681715791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3006a5e0 msgr2=0x7fbd3006ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.643+0000 7fbd361ff700 1 --2- 172.21.15.60:0/2681715791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3006a5e0 0x7fbd3006ce70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbd3006dad0 tx=0x7fbd24009bd0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.643+0000 7fbd361ff700 1 -- 172.21.15.60:0/2681715791 shutdown_connections 2023-07-13T22:27:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.643+0000 7fbd361ff700 1 --2- 172.21.15.60:0/2681715791 >> v2:172.21.15.105:3300/0 conn(0x7fbd300ca9f0 0x7fbd300b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.643+0000 7fbd361ff700 1 --2- 172.21.15.60:0/2681715791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3006a5e0 0x7fbd3006ce70 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.643+0000 7fbd361ff700 1 -- 172.21.15.60:0/2681715791 >> 172.21.15.60:0/2681715791 conn(0x7fbd300cbf50 msgr2=0x7fbd300ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.643+0000 7fbd361ff700 1 -- 172.21.15.60:0/2681715791 shutdown_connections 2023-07-13T22:27:54.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.643+0000 7fbd361ff700 1 -- 172.21.15.60:0/2681715791 wait complete. 2023-07-13T22:27:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.645+0000 7fbd361ff700 1 Processor -- start 2023-07-13T22:27:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.645+0000 7fbd361ff700 1 -- start start 2023-07-13T22:27:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd361ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd3006a5e0 0x7fbd30156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd361ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd300add00 0x7fbd300b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd361ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd300ca9f0 0x7fbd300b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd361ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300b12e0 con 0x7fbd300add00 2023-07-13T22:27:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd361ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300ad400 con 0x7fbd300ca9f0 2023-07-13T22:27:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd361ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300b1000 con 0x7fbd3006a5e0 2023-07-13T22:27:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd2effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd300add00 0x7fbd300b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd2effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd300add00 0x7fbd300b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49294/0 (socket says 172.21.15.60:49294) 2023-07-13T22:27:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd2effd700 1 -- 172.21.15.60:0/1584864768 learned_addr learned my addr 172.21.15.60:0/1584864768 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:54.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd2ffff700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd300ca9f0 0x7fbd300b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.646+0000 7fbd2f7fe700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd3006a5e0 0x7fbd30156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd2effd700 1 -- 172.21.15.60:0/1584864768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd3006a5e0 msgr2=0x7fbd30156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd2effd700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd3006a5e0 0x7fbd30156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd2effd700 1 -- 172.21.15.60:0/1584864768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd300ca9f0 msgr2=0x7fbd300b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd2effd700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd300ca9f0 0x7fbd300b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd2effd700 1 -- 172.21.15.60:0/1584864768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd24009550 con 0x7fbd300add00 2023-07-13T22:27:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd2ffff700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd300ca9f0 0x7fbd300b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd2effd700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd300add00 0x7fbd300b2d60 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbd30071d20 tx=0x7fbd24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd17fff700 1 -- 172.21.15.60:0/1584864768 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24003d80 con 0x7fbd300add00 2023-07-13T22:27:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd17fff700 1 -- 172.21.15.60:0/1584864768 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2401f070 con 0x7fbd300add00 2023-07-13T22:27:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd17fff700 1 -- 172.21.15.60:0/1584864768 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24011870 con 0x7fbd300add00 2023-07-13T22:27:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd300cc5b0 con 0x7fbd300add00 2023-07-13T22:27:54.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.647+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd300ccb70 con 0x7fbd300add00 2023-07-13T22:27:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.648+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd300cc410 con 0x7fbd300add00 2023-07-13T22:27:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.649+0000 7fbd17fff700 1 -- 172.21.15.60:0/1584864768 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbd24021030 con 0x7fbd300add00 2023-07-13T22:27:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.650+0000 7fbd17fff700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd1c06ace0 0x7fbd1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:54.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.650+0000 7fbd17fff700 1 -- 172.21.15.60:0/1584864768 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbd2408a870 con 0x7fbd300add00 2023-07-13T22:27:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.650+0000 7fbd2f7fe700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd1c06ace0 0x7fbd1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:54.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.651+0000 7fbd2f7fe700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd1c06ace0 0x7fbd1c06d190 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbd20001730 tx=0x7fbd20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:54.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.654+0000 7fbd17fff700 1 -- 172.21.15.60:0/1584864768 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbd300cc410 con 0x7fbd300add00 2023-07-13T22:27:54.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.863+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7fbd300cc410 con 0x7fbd300add00 2023-07-13T22:27:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.869+0000 7fbd17fff700 1 -- 172.21.15.60:0/1584864768 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v31) v1 ==== 107+0+0 (secure 0 0 0) 0x7fbd300cc410 con 0x7fbd300add00 2023-07-13T22:27:54.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.871+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd1c06ace0 msgr2=0x7fbd1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.871+0000 7fbd361ff700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd1c06ace0 0x7fbd1c06d190 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbd20001730 tx=0x7fbd20009040 comp rx=0 tx=0).stop 2023-07-13T22:27:54.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.871+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd300add00 msgr2=0x7fbd300b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:54.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.871+0000 7fbd361ff700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd300add00 0x7fbd300b2d60 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbd30071d20 tx=0x7fbd24004750 comp rx=0 tx=0).stop 2023-07-13T22:27:54.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.872+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 shutdown_connections 2023-07-13T22:27:54.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.872+0000 7fbd361ff700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd1c06ace0 0x7fbd1c06d190 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.872+0000 7fbd361ff700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd300ca9f0 0x7fbd300b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.872+0000 7fbd361ff700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd3006a5e0 0x7fbd30156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.872+0000 7fbd361ff700 1 --2- 172.21.15.60:0/1584864768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd300add00 0x7fbd300b2d60 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.872+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 >> 172.21.15.60:0/1584864768 conn(0x7fbd300cbf50 msgr2=0x7fbd3006cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.872+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 shutdown_connections 2023-07-13T22:27:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:54.872+0000 7fbd361ff700 1 -- 172.21.15.60:0/1584864768 wait complete. 2023-07-13T22:27:54.870 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2023-07-13T22:27:55.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.092+0000 7f38a43ff700 1 Processor -- start 2023-07-13T22:27:55.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.092+0000 7f38a43ff700 1 -- start start 2023-07-13T22:27:55.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a43ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f389c0cd260 0x7f389c0d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a43ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0d0e20 0x7f389c0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a43ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c0d2410 con 0x7f389c0d0a50 2023-07-13T22:27:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a43ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389c15c900 con 0x7f389c0cd260 2023-07-13T22:27:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a43ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0d2660 con 0x7f389c0d0e20 2023-07-13T22:27:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a199a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f389c0cd260 0x7f389c0d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a199a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f389c0cd260 0x7f389c0d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40742/0 (socket says 172.21.15.60:40742) 2023-07-13T22:27:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a199a700 1 -- 172.21.15.60:0/3412504028 learned_addr learned my addr 172.21.15.60:0/3412504028 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a299c700 1 --2- 172.21.15.60:0/3412504028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0d0e20 0x7f389c0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.094+0000 7f38a199a700 1 -- 172.21.15.60:0/3412504028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0d0e20 msgr2=0x7f389c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:55.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.095+0000 7f38a199a700 1 --2- 172.21.15.60:0/3412504028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0d0e20 0x7f389c0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.095+0000 7f38a199a700 1 -- 172.21.15.60:0/3412504028 >> v1:172.21.15.105:6789/0 conn(0x7f389c0d0a50 legacy=0x7f389c0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.095+0000 7f38a199a700 1 -- 172.21.15.60:0/3412504028 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c0d2940 con 0x7f389c0cd260 2023-07-13T22:27:55.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.095+0000 7f38a1199700 1 -- 172.21.15.60:0/3412504028 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421528301 0 0) 0x7f389c0d2410 con 0x7f389c0d0a50 2023-07-13T22:27:55.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.095+0000 7f38a199a700 1 --2- 172.21.15.60:0/3412504028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0cd260 0x7f389c0d3420 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f388c0098f0 tx=0x7f388c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83b10457a690ec1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.095+0000 7f38a1199700 1 -- 172.21.15.60:0/3412504028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c011830 con 0x7f389c0cd260 2023-07-13T22:27:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.095+0000 7f38a1199700 1 -- 172.21.15.60:0/3412504028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c011990 con 0x7f389c0cd260 2023-07-13T22:27:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.095+0000 7f38a1199700 1 -- 172.21.15.60:0/3412504028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c011d20 con 0x7f389c0cd260 2023-07-13T22:27:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.096+0000 7f38a43ff700 1 -- 172.21.15.60:0/3412504028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0cd260 msgr2=0x7f389c0d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.096+0000 7f38a43ff700 1 --2- 172.21.15.60:0/3412504028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0cd260 0x7f389c0d3420 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f388c0098f0 tx=0x7f388c009c00 comp rx=0 tx=0).stop 2023-07-13T22:27:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.096+0000 7f38a43ff700 1 -- 172.21.15.60:0/3412504028 shutdown_connections 2023-07-13T22:27:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.096+0000 7f38a43ff700 1 --2- 172.21.15.60:0/3412504028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0d0e20 0x7f389c0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.096+0000 7f38a43ff700 1 --2- 172.21.15.60:0/3412504028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0cd260 0x7f389c0d3420 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.096+0000 7f38a43ff700 1 -- 172.21.15.60:0/3412504028 >> 172.21.15.60:0/3412504028 conn(0x7f389c071600 msgr2=0x7f389c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.096+0000 7f38a43ff700 1 -- 172.21.15.60:0/3412504028 shutdown_connections 2023-07-13T22:27:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.096+0000 7f38a43ff700 1 -- 172.21.15.60:0/3412504028 wait complete. 2023-07-13T22:27:55.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.097+0000 7f38a43ff700 1 Processor -- start 2023-07-13T22:27:55.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.098+0000 7f38a43ff700 1 -- start start 2023-07-13T22:27:55.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.098+0000 7f38a43ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0cd260 0x7f389c0c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.098+0000 7f38a43ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f389c0d0a50 0x7f389c14e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a43ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0d0e20 0x7f389c0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a43ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0c8720 con 0x7f389c0d0e20 2023-07-13T22:27:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a43ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0c90f0 con 0x7f389c0d0a50 2023-07-13T22:27:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a43ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0c7b10 con 0x7f389c0cd260 2023-07-13T22:27:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a299c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0d0e20 0x7f389c0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a299c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0d0e20 0x7f389c0cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40754/0 (socket says 172.21.15.60:40754) 2023-07-13T22:27:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a299c700 1 -- 172.21.15.60:0/2222982213 learned_addr learned my addr 172.21.15.60:0/2222982213 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a199a700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f389c0d0a50 0x7f389c14e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a219b700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0cd260 0x7f389c0c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a299c700 1 -- 172.21.15.60:0/2222982213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0cd260 msgr2=0x7f389c0c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a299c700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0cd260 0x7f389c0c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a299c700 1 -- 172.21.15.60:0/2222982213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f389c0d0a50 msgr2=0x7f389c14e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a299c700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f389c0d0a50 0x7f389c14e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a299c700 1 -- 172.21.15.60:0/2222982213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c009550 con 0x7f389c0d0e20 2023-07-13T22:27:55.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.099+0000 7f38a219b700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0cd260 0x7f389c0c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:27:55.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.100+0000 7f38a299c700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0d0e20 0x7f389c0cb730 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f389400efc0 tx=0x7f389400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:55.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.100+0000 7f3892ffd700 1 -- 172.21.15.60:0/2222982213 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894009b20 con 0x7f389c0d0e20 2023-07-13T22:27:55.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.100+0000 7f3892ffd700 1 -- 172.21.15.60:0/2222982213 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389400f040 con 0x7f389c0d0e20 2023-07-13T22:27:55.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.100+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c0c7df0 con 0x7f389c0d0e20 2023-07-13T22:27:55.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.100+0000 7f3892ffd700 1 -- 172.21.15.60:0/2222982213 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894009e50 con 0x7f389c0d0e20 2023-07-13T22:27:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.100+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c0c83b0 con 0x7f389c0d0e20 2023-07-13T22:27:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.101+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c0cde50 con 0x7f389c0d0e20 2023-07-13T22:27:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.102+0000 7f3892ffd700 1 -- 172.21.15.60:0/2222982213 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3894014480 con 0x7f389c0d0e20 2023-07-13T22:27:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.103+0000 7f3892ffd700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388006ace0 0x7f388006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.103+0000 7f3892ffd700 1 -- 172.21.15.60:0/2222982213 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3894089d30 con 0x7f389c0d0e20 2023-07-13T22:27:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.103+0000 7f38a219b700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388006ace0 0x7f388006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.104+0000 7f38a219b700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388006ace0 0x7f388006d190 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3898000f60 tx=0x7f3898009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:55.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.107+0000 7f3892ffd700 1 -- 172.21.15.60:0/2222982213 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f389408f020 con 0x7f389c0d0e20 2023-07-13T22:27:55.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.393+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) v1 -- 0x7f389c04e280 con 0x7f389c0d0e20 2023-07-13T22:27:55.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.603+0000 7f3892ffd700 1 -- 172.21.15.60:0/2222982213 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v349) v1 ==== 160+0+0 (secure 0 0 0) 0x7f38940180e0 con 0x7f389c0d0e20 2023-07-13T22:27:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388006ace0 msgr2=0x7f388006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388006ace0 0x7f388006d190 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3898000f60 tx=0x7f3898009040 comp rx=0 tx=0).stop 2023-07-13T22:27:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0d0e20 msgr2=0x7f389c0cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0d0e20 0x7f389c0cb730 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f389400efc0 tx=0x7f389400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 shutdown_connections 2023-07-13T22:27:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388006ace0 0x7f388006d190 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389c0d0e20 0x7f389c0cb730 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f389c0cd260 0x7f389c0c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 --2- 172.21.15.60:0/2222982213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f389c0d0a50 0x7f389c14e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 >> 172.21.15.60:0/2222982213 conn(0x7f389c071600 msgr2=0x7f389c159110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 shutdown_connections 2023-07-13T22:27:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.606+0000 7f38a43ff700 1 -- 172.21.15.60:0/2222982213 wait complete. 2023-07-13T22:27:55.606 INFO:tasks.cephfs_test_runner:test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-13T22:27:55.606 INFO:tasks.cephfs_test_runner:Starting test: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:27:55.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2023-07-13T22:27:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.830+0000 7f6a3a9ff700 1 Processor -- start 2023-07-13T22:27:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.830+0000 7f6a3a9ff700 1 -- start start 2023-07-13T22:27:55.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.831+0000 7f6a3a9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6a340b2bc0 0x7f6a340ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.831+0000 7f6a3a9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340d0320 0x7f6a341663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.831+0000 7f6a3a9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3415fe20 con 0x7f6a340cdb20 2023-07-13T22:27:55.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.831+0000 7f6a3a9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340cdef0 con 0x7f6a340b2bc0 2023-07-13T22:27:55.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a3a9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340ceb40 con 0x7f6a340d0320 2023-07-13T22:27:55.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a33fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6a340cdb20 0x7f6a340b24b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36212/0 (socket says 172.21.15.60:36212) 2023-07-13T22:27:55.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a33fff700 1 -- 172.21.15.60:0/3186838323 learned_addr learned my addr 172.21.15.60:0/3186838323 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:55.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a337fe700 1 --2- 172.21.15.60:0/3186838323 >> v2:172.21.15.105:3300/0 conn(0x7f6a340b2bc0 0x7f6a340ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a38f9c700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340d0320 0x7f6a341663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729515985 0 0) 0x7f6a24001110 con 0x7f6a340cdb20 2023-07-13T22:27:55.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a1c0035b0 con 0x7f6a340cdb20 2023-07-13T22:27:55.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 859193833 0 0) 0x7f6a1c0035b0 con 0x7f6a340cdb20 2023-07-13T22:27:55.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340d0320 msgr2=0x7f6a341663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:55.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a32ffd700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340d0320 0x7f6a341663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 >> v2:172.21.15.105:3300/0 conn(0x7f6a340b2bc0 msgr2=0x7f6a340ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:55.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a32ffd700 1 --2- 172.21.15.60:0/3186838323 >> v2:172.21.15.105:3300/0 conn(0x7f6a340b2bc0 0x7f6a340ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.832+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3415cb50 con 0x7f6a340cdb20 2023-07-13T22:27:55.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f6a24001800 con 0x7f6a340cdb20 2023-07-13T22:27:55.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a337fe700 1 --2- 172.21.15.60:0/3186838323 >> v2:172.21.15.105:3300/0 conn(0x7f6a340b2bc0 0x7f6a340ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 >> v1:172.21.15.60:6789/0 conn(0x7f6a340cdb20 legacy=0x7f6a340b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a32ffd700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340d0320 0x7f6a1c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a32ffd700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a1c003f80 0x7f6a1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a32ffd700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a1c004d90 0x7f6a1c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340cdef0 con 0x7f6a1c004d90 2023-07-13T22:27:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340ceb40 con 0x7f6a1c003f80 2023-07-13T22:27:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c00ba10 con 0x7f6a340d0320 2023-07-13T22:27:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a38f9c700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a1c004d90 0x7f6a1c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a33fff700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340d0320 0x7f6a1c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a337fe700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a1c003f80 0x7f6a1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a38f9c700 1 -- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340d0320 msgr2=0x7f6a1c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a38f9c700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340d0320 0x7f6a1c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a38f9c700 1 -- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a1c003f80 msgr2=0x7f6a1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a38f9c700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a1c003f80 0x7f6a1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.833+0000 7f6a38f9c700 1 -- 172.21.15.60:0/3186838323 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c0035b0 con 0x7f6a1c004d90 2023-07-13T22:27:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.834+0000 7f6a38f9c700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a1c004d90 0x7f6a1c005200 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009fc0 tx=0x7f6a1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ddb2a192821063 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:55.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.834+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00ec30 con 0x7f6a1c004d90 2023-07-13T22:27:55.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.834+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c00f040 con 0x7f6a1c004d90 2023-07-13T22:27:55.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.834+0000 7f6a32ffd700 1 -- 172.21.15.60:0/3186838323 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c014480 con 0x7f6a1c004d90 2023-07-13T22:27:55.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.834+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3186838323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a1c004d90 msgr2=0x7f6a1c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.834+0000 7f6a3a9ff700 1 --2- 172.21.15.60:0/3186838323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a1c004d90 0x7f6a1c005200 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009fc0 tx=0x7f6a1c005880 comp rx=0 tx=0).stop 2023-07-13T22:27:55.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.834+0000 7f6a38f9c700 1 -- 172.21.15.60:0/3186838323 reap_dead start 2023-07-13T22:27:55.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.835+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3186838323 shutdown_connections 2023-07-13T22:27:55.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.835+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3186838323 >> 172.21.15.60:0/3186838323 conn(0x7f6a3406e660 msgr2=0x7f6a3406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.835+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3186838323 shutdown_connections 2023-07-13T22:27:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.835+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3186838323 wait complete. 2023-07-13T22:27:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.836+0000 7f6a3a9ff700 1 Processor -- start 2023-07-13T22:27:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.837+0000 7f6a3a9ff700 1 -- start start 2023-07-13T22:27:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.837+0000 7f6a3a9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a1c004d90 0x7f6a340c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.837+0000 7f6a3a9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a340c8f20 0x7f6a340c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.837+0000 7f6a3a9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340cab30 0x7f6a340cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.837+0000 7f6a3a9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340b28b0 con 0x7f6a340c8f20 2023-07-13T22:27:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.837+0000 7f6a3a9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341e7ff0 con 0x7f6a1c004d90 2023-07-13T22:27:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.837+0000 7f6a3a9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340c9a90 con 0x7f6a340cab30 2023-07-13T22:27:55.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a337fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a340c8f20 0x7f6a340c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a337fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a340c8f20 0x7f6a340c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40772/0 (socket says 172.21.15.60:40772) 2023-07-13T22:27:55.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a337fe700 1 -- 172.21.15.60:0/3365275254 learned_addr learned my addr 172.21.15.60:0/3365275254 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:55.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a33fff700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a1c004d90 0x7f6a340c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a38f9c700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340cab30 0x7f6a340cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a337fe700 1 -- 172.21.15.60:0/3365275254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340cab30 msgr2=0x7f6a340cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a337fe700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340cab30 0x7f6a340cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a337fe700 1 -- 172.21.15.60:0/3365275254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a1c004d90 msgr2=0x7f6a340c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:55.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a337fe700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a1c004d90 0x7f6a340c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:55.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a337fe700 1 -- 172.21.15.60:0/3365275254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340c9d70 con 0x7f6a340c8f20 2023-07-13T22:27:55.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.838+0000 7f6a337fe700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a340c8f20 0x7f6a340c9390 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6a28011cd0 tx=0x7f6a28009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:55.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.839+0000 7f6a30ff9700 1 -- 172.21.15.60:0/3365275254 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800fa00 con 0x7f6a340c8f20 2023-07-13T22:27:55.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.839+0000 7f6a30ff9700 1 -- 172.21.15.60:0/3365275254 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2800fb60 con 0x7f6a340c8f20 2023-07-13T22:27:55.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.839+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a340ca050 con 0x7f6a340c8f20 2023-07-13T22:27:55.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.839+0000 7f6a30ff9700 1 -- 172.21.15.60:0/3365275254 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28004030 con 0x7f6a340c8f20 2023-07-13T22:27:55.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.839+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a340ca610 con 0x7f6a340c8f20 2023-07-13T22:27:55.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.840+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3404e280 con 0x7f6a340c8f20 2023-07-13T22:27:55.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.841+0000 7f6a30ff9700 1 -- 172.21.15.60:0/3365275254 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a28004190 con 0x7f6a340c8f20 2023-07-13T22:27:55.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.842+0000 7f6a30ff9700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a2006ace0 0x7f6a2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:55.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.842+0000 7f6a30ff9700 1 -- 172.21.15.60:0/3365275254 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6a280897a0 con 0x7f6a340c8f20 2023-07-13T22:27:55.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.842+0000 7f6a33fff700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a2006ace0 0x7f6a2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:55.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.843+0000 7f6a33fff700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a2006ace0 0x7f6a2006d190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6a24001780 tx=0x7f6a24007b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:55.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:55.846+0000 7f6a30ff9700 1 -- 172.21.15.60:0/3365275254 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6a28054ad0 con 0x7f6a340c8f20 2023-07-13T22:27:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.137+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) v1 -- 0x7f6a340b69b0 con 0x7f6a340c8f20 2023-07-13T22:27:56.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.606+0000 7f6a30ff9700 1 -- 172.21.15.60:0/3365275254 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v350) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6a280220e0 con 0x7f6a340c8f20 2023-07-13T22:27:56.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a2006ace0 msgr2=0x7f6a2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:56.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a2006ace0 0x7f6a2006d190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6a24001780 tx=0x7f6a24007b00 comp rx=0 tx=0).stop 2023-07-13T22:27:56.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a340c8f20 msgr2=0x7f6a340c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:56.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a340c8f20 0x7f6a340c9390 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6a28011cd0 tx=0x7f6a28009ef0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 shutdown_connections 2023-07-13T22:27:56.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a2006ace0 0x7f6a2006d190 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a340cab30 0x7f6a340cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a1c004d90 0x7f6a340c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 --2- 172.21.15.60:0/3365275254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a340c8f20 0x7f6a340c9390 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 >> 172.21.15.60:0/3365275254 conn(0x7f6a3406e660 msgr2=0x7f6a3406b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:56.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 shutdown_connections 2023-07-13T22:27:56.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.609+0000 7f6a3a9ff700 1 -- 172.21.15.60:0/3365275254 wait complete. 2023-07-13T22:27:56.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-13T22:27:56.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.839+0000 7fca589ff700 1 Processor -- start 2023-07-13T22:27:56.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.839+0000 7fca589ff700 1 -- start start 2023-07-13T22:27:56.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.840+0000 7fca589ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540b8200 0x7fca540cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:56.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.840+0000 7fca589ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca540d1870 con 0x7fca540cede0 2023-07-13T22:27:56.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.840+0000 7fca589ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca54160d80 con 0x7fca540b1ac0 2023-07-13T22:27:56.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.840+0000 7fca589ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca540cf370 con 0x7fca540b8200 2023-07-13T22:27:56.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.840+0000 7fca52d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fca540cede0 0x7fca54160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36218/0 (socket says 172.21.15.60:36218) 2023-07-13T22:27:56.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.840+0000 7fca52d9d700 1 -- 172.21.15.60:0/1224505889 learned_addr learned my addr 172.21.15.60:0/1224505889 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:56.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca5259c700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540b8200 0x7fca540cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:56.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014367720 0 0) 0x7fca540d1870 con 0x7fca540cede0 2023-07-13T22:27:56.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca400035b0 con 0x7fca540cede0 2023-07-13T22:27:56.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37377709 0 0) 0x7fca54160d80 con 0x7fca540b1ac0 2023-07-13T22:27:56.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca540d1870 con 0x7fca540b1ac0 2023-07-13T22:27:56.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 342482613 0 0) 0x7fca400035b0 con 0x7fca540cede0 2023-07-13T22:27:56.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540b8200 msgr2=0x7fca540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:56.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540b8200 0x7fca540cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 >> v1:172.21.15.105:6789/0 conn(0x7fca540b1ac0 legacy=0x7fca5415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540d12b0 con 0x7fca540cede0 2023-07-13T22:27:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fca4c0019f0 con 0x7fca540cede0 2023-07-13T22:27:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 >> v1:172.21.15.60:6789/0 conn(0x7fca540cede0 legacy=0x7fca54160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:56.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540b8200 0x7fca40003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:56.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca40004170 0x7fca40004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:56.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca40004d90 0x7fca40005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:56.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca540cf370 con 0x7fca40004d90 2023-07-13T22:27:56.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4000bb20 con 0x7fca40004170 2023-07-13T22:27:56.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.841+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4000bce0 con 0x7fca540b8200 2023-07-13T22:27:56.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca52d9d700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca40004d90 0x7fca40005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:56.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca5259c700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540b8200 0x7fca40003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:56.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca4bfff700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca40004170 0x7fca40004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:56.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca52d9d700 1 -- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540b8200 msgr2=0x7fca40003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:56.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca52d9d700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540b8200 0x7fca40003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca52d9d700 1 -- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca40004170 msgr2=0x7fca40004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:56.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca52d9d700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca40004170 0x7fca40004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca52d9d700 1 -- 172.21.15.60:0/1224505889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400035b0 con 0x7fca40004d90 2023-07-13T22:27:56.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca52d9d700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca40004d90 0x7fca40005200 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00a6e0 tx=0x7fca40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adb70fa0e3ef377 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:56.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.842+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c0072d0 con 0x7fca40004d90 2023-07-13T22:27:56.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.843+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c007430 con 0x7fca40004d90 2023-07-13T22:27:56.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.843+0000 7fca51d9b700 1 -- 172.21.15.60:0/1224505889 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c0077e0 con 0x7fca40004d90 2023-07-13T22:27:56.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.843+0000 7fca589ff700 1 -- 172.21.15.60:0/1224505889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca40004d90 msgr2=0x7fca40005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:56.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.843+0000 7fca589ff700 1 --2- 172.21.15.60:0/1224505889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca40004d90 0x7fca40005200 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00a6e0 tx=0x7fca40005880 comp rx=0 tx=0).stop 2023-07-13T22:27:56.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.843+0000 7fca52d9d700 1 -- 172.21.15.60:0/1224505889 reap_dead start 2023-07-13T22:27:56.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.843+0000 7fca589ff700 1 -- 172.21.15.60:0/1224505889 shutdown_connections 2023-07-13T22:27:56.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.843+0000 7fca589ff700 1 -- 172.21.15.60:0/1224505889 >> 172.21.15.60:0/1224505889 conn(0x7fca5406b2f0 msgr2=0x7fca5406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:56.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.843+0000 7fca589ff700 1 -- 172.21.15.60:0/1224505889 shutdown_connections 2023-07-13T22:27:56.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.843+0000 7fca589ff700 1 -- 172.21.15.60:0/1224505889 wait complete. 2023-07-13T22:27:56.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.845+0000 7fca589ff700 1 Processor -- start 2023-07-13T22:27:56.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.845+0000 7fca589ff700 1 -- start start 2023-07-13T22:27:56.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.846+0000 7fca589ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca40004d90 0x7fca540c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:56.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.846+0000 7fca589ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540c9380 0x7fca540ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:56.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.846+0000 7fca589ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca540c77b0 0x7fca540b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:56.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.846+0000 7fca589ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca540ca330 con 0x7fca540c77b0 2023-07-13T22:27:56.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.846+0000 7fca589ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca540c9d00 con 0x7fca40004d90 2023-07-13T22:27:56.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.846+0000 7fca589ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca540c8190 con 0x7fca540c9380 2023-07-13T22:27:56.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.846+0000 7fca52d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca540c77b0 0x7fca540b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:56.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.846+0000 7fca52d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca540c77b0 0x7fca540b6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40786/0 (socket says 172.21.15.60:40786) 2023-07-13T22:27:56.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.846+0000 7fca52d9d700 1 -- 172.21.15.60:0/1021991869 learned_addr learned my addr 172.21.15.60:0/1021991869 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:56.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca5259c700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca40004d90 0x7fca540c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:56.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca4bfff700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540c9380 0x7fca540ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:56.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca52d9d700 1 -- 172.21.15.60:0/1021991869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540c9380 msgr2=0x7fca540ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:56.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca52d9d700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540c9380 0x7fca540ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca52d9d700 1 -- 172.21.15.60:0/1021991869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca40004d90 msgr2=0x7fca540c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:56.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca52d9d700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca40004d90 0x7fca540c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:56.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca52d9d700 1 -- 172.21.15.60:0/1021991869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540c8470 con 0x7fca540c77b0 2023-07-13T22:27:56.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca52d9d700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca540c77b0 0x7fca540b6d10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fca4c000f80 tx=0x7fca4c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:56.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca4affd700 1 -- 172.21.15.60:0/1021991869 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c025070 con 0x7fca540c77b0 2023-07-13T22:27:56.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca4affd700 1 -- 172.21.15.60:0/1021991869 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c001400 con 0x7fca540c77b0 2023-07-13T22:27:56.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca4affd700 1 -- 172.21.15.60:0/1021991869 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c0077e0 con 0x7fca540c77b0 2023-07-13T22:27:56.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca540c9f50 con 0x7fca540c77b0 2023-07-13T22:27:56.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.847+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca540cac90 con 0x7fca540c77b0 2023-07-13T22:27:56.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.848+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca540ca470 con 0x7fca540c77b0 2023-07-13T22:27:56.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.849+0000 7fca4affd700 1 -- 172.21.15.60:0/1021991869 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca4c01a070 con 0x7fca540c77b0 2023-07-13T22:27:56.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.850+0000 7fca4affd700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca3406ace0 0x7fca3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:56.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.850+0000 7fca4affd700 1 -- 172.21.15.60:0/1021991869 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fca4c08a5e0 con 0x7fca540c77b0 2023-07-13T22:27:56.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.850+0000 7fca5259c700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca3406ace0 0x7fca3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:56.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.851+0000 7fca5259c700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca3406ace0 0x7fca3406d190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fca540737e0 tx=0x7fca44009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:56.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:56.854+0000 7fca4affd700 1 -- 172.21.15.60:0/1021991869 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fca4c0559a0 con 0x7fca540c77b0 2023-07-13T22:27:56.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:27:56.874 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:27:57.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.065+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fca5404e210 con 0x7fca540c77b0 2023-07-13T22:27:57.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.070+0000 7fca4affd700 1 -- 172.21.15.60:0/1021991869 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fca4c058fc0 con 0x7fca540c77b0 2023-07-13T22:27:57.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.072+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca3406ace0 msgr2=0x7fca3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.072+0000 7fca589ff700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca3406ace0 0x7fca3406d190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fca540737e0 tx=0x7fca44009210 comp rx=0 tx=0).stop 2023-07-13T22:27:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.072+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca540c77b0 msgr2=0x7fca540b6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.072+0000 7fca589ff700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca540c77b0 0x7fca540b6d10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fca4c000f80 tx=0x7fca4c001cb0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.073+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 shutdown_connections 2023-07-13T22:27:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.073+0000 7fca589ff700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca3406ace0 0x7fca3406d190 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.073+0000 7fca589ff700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca540c77b0 0x7fca540b6d10 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.073+0000 7fca589ff700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca40004d90 0x7fca540c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.073+0000 7fca589ff700 1 --2- 172.21.15.60:0/1021991869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca540c9380 0x7fca540ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.073+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 >> 172.21.15.60:0/1021991869 conn(0x7fca5406b2f0 msgr2=0x7fca540b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.073+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 shutdown_connections 2023-07-13T22:27:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.073+0000 7fca589ff700 1 -- 172.21.15.60:0/1021991869 wait complete. 2023-07-13T22:27:57.071 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:27:57.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:27:57.177 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:27:57.291 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:27:57.292 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:27:57.292 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:27:57.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:27:57.338 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:27:57.339 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:27:57.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:27:57.386 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:27:57.387 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:27:57.433 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:27:57.433 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:27:57.434 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:27:57.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:27:57.481 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-13T22:27:57.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:27:57.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.704+0000 7fb8985ff700 1 Processor -- start 2023-07-13T22:27:57.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.704+0000 7fb8985ff700 1 -- start start 2023-07-13T22:27:57.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.705+0000 7fb8985ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8900b8570 0x7fb8900ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:57.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.705+0000 7fb8985ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d15c0 0x7fb89015c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:57.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.705+0000 7fb8985ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8900d09d0 con 0x7fb8900d35c0 2023-07-13T22:27:57.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.705+0000 7fb8985ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8900d3b30 con 0x7fb8900b8570 2023-07-13T22:27:57.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.705+0000 7fb8985ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900cda10 con 0x7fb8900d15c0 2023-07-13T22:27:57.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.705+0000 7fb895b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb8900d35c0 0x7fb8900d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36234/0 (socket says 172.21.15.60:36234) 2023-07-13T22:27:57.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.705+0000 7fb895b9a700 1 -- 172.21.15.60:0/1565261689 learned_addr learned my addr 172.21.15.60:0/1565261689 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:57.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.705+0000 7fb896b9c700 1 --2- 172.21.15.60:0/1565261689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d15c0 0x7fb89015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.705+0000 7fb89639b700 1 --2- 172.21.15.60:0/1565261689 >> v2:172.21.15.105:3300/0 conn(0x7fb8900b8570 0x7fb8900ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb895399700 1 -- 172.21.15.60:0/1565261689 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611516272 0 0) 0x7fb8900d09d0 con 0x7fb8900d35c0 2023-07-13T22:27:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb895399700 1 -- 172.21.15.60:0/1565261689 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb87c0035b0 con 0x7fb8900d35c0 2023-07-13T22:27:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb89639b700 1 -- 172.21.15.60:0/1565261689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d15c0 msgr2=0x7fb89015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb89639b700 1 --2- 172.21.15.60:0/1565261689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d15c0 0x7fb89015c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb89639b700 1 -- 172.21.15.60:0/1565261689 >> v1:172.21.15.60:6789/0 conn(0x7fb8900d35c0 legacy=0x7fb8900d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb89639b700 1 -- 172.21.15.60:0/1565261689 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900cd450 con 0x7fb8900b8570 2023-07-13T22:27:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb896b9c700 1 --2- 172.21.15.60:0/1565261689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d15c0 0x7fb89015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb89639b700 1 --2- 172.21.15.60:0/1565261689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8900b8570 0x7fb8900ceeb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009820 tx=0x7fb88c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64c78670bb7a1da3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb895399700 1 -- 172.21.15.60:0/1565261689 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c005e30 con 0x7fb8900b8570 2023-07-13T22:27:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.706+0000 7fb895399700 1 -- 172.21.15.60:0/1565261689 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c006030 con 0x7fb8900b8570 2023-07-13T22:27:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.707+0000 7fb895399700 1 -- 172.21.15.60:0/1565261689 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c0063c0 con 0x7fb8900b8570 2023-07-13T22:27:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.707+0000 7fb8985ff700 1 -- 172.21.15.60:0/1565261689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8900b8570 msgr2=0x7fb8900ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.707+0000 7fb8985ff700 1 --2- 172.21.15.60:0/1565261689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8900b8570 0x7fb8900ceeb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009820 tx=0x7fb88c009b30 comp rx=0 tx=0).stop 2023-07-13T22:27:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.707+0000 7fb8985ff700 1 -- 172.21.15.60:0/1565261689 shutdown_connections 2023-07-13T22:27:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.707+0000 7fb8985ff700 1 --2- 172.21.15.60:0/1565261689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d15c0 0x7fb89015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.707+0000 7fb8985ff700 1 --2- 172.21.15.60:0/1565261689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8900b8570 0x7fb8900ceeb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.707+0000 7fb8985ff700 1 -- 172.21.15.60:0/1565261689 >> 172.21.15.60:0/1565261689 conn(0x7fb890071600 msgr2=0x7fb8900719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.707+0000 7fb8985ff700 1 -- 172.21.15.60:0/1565261689 shutdown_connections 2023-07-13T22:27:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.707+0000 7fb8985ff700 1 -- 172.21.15.60:0/1565261689 wait complete. 2023-07-13T22:27:57.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.709+0000 7fb8985ff700 1 Processor -- start 2023-07-13T22:27:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.709+0000 7fb8985ff700 1 -- start start 2023-07-13T22:27:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb8985ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8900b8570 0x7fb8901f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb8985ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8900d15c0 0x7fb8900b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb8985ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d35c0 0x7fb89015e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb8985ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900c7380 con 0x7fb8900b8570 2023-07-13T22:27:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb8985ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900c6790 con 0x7fb8900d15c0 2023-07-13T22:27:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb8985ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8901f1950 con 0x7fb8900d35c0 2023-07-13T22:27:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb89639b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8900b8570 0x7fb8901f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb89639b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8900b8570 0x7fb8901f1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40788/0 (socket says 172.21.15.60:40788) 2023-07-13T22:27:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb89639b700 1 -- 172.21.15.60:0/2701346284 learned_addr learned my addr 172.21.15.60:0/2701346284 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb895b9a700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8900d15c0 0x7fb8900b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.710+0000 7fb896b9c700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d35c0 0x7fb89015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb89639b700 1 -- 172.21.15.60:0/2701346284 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d35c0 msgr2=0x7fb89015e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb89639b700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d35c0 0x7fb89015e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb89639b700 1 -- 172.21.15.60:0/2701346284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8900d15c0 msgr2=0x7fb8900b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:57.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb89639b700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8900d15c0 0x7fb8900b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb89639b700 1 -- 172.21.15.60:0/2701346284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888009a70 con 0x7fb8900b8570 2023-07-13T22:27:57.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb89639b700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8900b8570 0x7fb8901f1410 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb8900d3d60 tx=0x7fb88c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:57.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb886ffd700 1 -- 172.21.15.60:0/2701346284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c0061a0 con 0x7fb8900b8570 2023-07-13T22:27:57.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb886ffd700 1 -- 172.21.15.60:0/2701346284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c020070 con 0x7fb8900b8570 2023-07-13T22:27:57.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb886ffd700 1 -- 172.21.15.60:0/2701346284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c006560 con 0x7fb8900b8570 2023-07-13T22:27:57.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.711+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c009430 con 0x7fb8900b8570 2023-07-13T22:27:57.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.712+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8900c6cc0 con 0x7fb8900b8570 2023-07-13T22:27:57.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.713+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89004e280 con 0x7fb8900b8570 2023-07-13T22:27:57.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.714+0000 7fb886ffd700 1 -- 172.21.15.60:0/2701346284 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb88c022030 con 0x7fb8900b8570 2023-07-13T22:27:57.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.715+0000 7fb886ffd700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87406ace0 0x7fb87406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:57.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.715+0000 7fb886ffd700 1 -- 172.21.15.60:0/2701346284 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb88c08a5a0 con 0x7fb8900b8570 2023-07-13T22:27:57.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.715+0000 7fb895b9a700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87406ace0 0x7fb87406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:57.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.715+0000 7fb895b9a700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87406ace0 0x7fb87406d190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb8901f4fd0 tx=0x7fb880007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:57.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.719+0000 7fb886ffd700 1 -- 172.21.15.60:0/2701346284 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb88c055a70 con 0x7fb8900b8570 2023-07-13T22:27:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.971+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb890160a50 con 0x7fb8900b8570 2023-07-13T22:27:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.971+0000 7fb886ffd700 1 -- 172.21.15.60:0/2701346284 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fb88c059090 con 0x7fb8900b8570 2023-07-13T22:27:57.952 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:27:57.952 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1603865496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1603865496},{"type":"v1","addr":"172.21.15.105:6834","nonce":1603865496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6735,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1777842000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1777842000},{"type":"v1","addr":"172.21.15.105:6835","nonce":1777842000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6738,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2613179178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2613179178},{"type":"v1","addr":"172.21.15.60:6836","nonce":2613179178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2091377893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":2091377893},{"type":"v1","addr":"172.21.15.60:6837","nonce":2091377893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[]} 2023-07-13T22:27:57.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.973+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87406ace0 msgr2=0x7fb87406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:57.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.973+0000 7fb8985ff700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87406ace0 0x7fb87406d190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb8901f4fd0 tx=0x7fb880007dc0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.973+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8900b8570 msgr2=0x7fb8901f1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:57.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.974+0000 7fb8985ff700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8900b8570 0x7fb8901f1410 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb8900d3d60 tx=0x7fb88c004750 comp rx=0 tx=0).stop 2023-07-13T22:27:57.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.974+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 shutdown_connections 2023-07-13T22:27:57.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.974+0000 7fb8985ff700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87406ace0 0x7fb87406d190 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.974+0000 7fb8985ff700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8900b8570 0x7fb8901f1410 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.974+0000 7fb8985ff700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8900d35c0 0x7fb89015e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.974+0000 7fb8985ff700 1 --2- 172.21.15.60:0/2701346284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8900d15c0 0x7fb8900b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:57.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.974+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 >> 172.21.15.60:0/2701346284 conn(0x7fb890071600 msgr2=0x7fb89015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:57.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.974+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 shutdown_connections 2023-07-13T22:27:57.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:57.974+0000 7fb8985ff700 1 -- 172.21.15.60:0/2701346284 wait complete. 2023-07-13T22:27:57.956 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2023-07-13T22:27:57.972 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:27:57.973 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-13T22:27:57.973 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:27:57.974 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:27:57.974 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:27:57.974 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-13T22:27:57.975 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:27:57.975 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:27:57.975 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:27:57.975 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-13T22:27:57.976 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:27:57.976 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:27:57.976 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:27:57.976 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-13T22:27:57.977 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:27:57.977 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:27:57.987 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:27:57.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:27:57.989 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:27:57.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:27:57.992 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:27:57.992 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:27:57.994 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:27:57.994 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:27:58.033 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:27:58.034 DEBUG:teuthology.parallel:result is None 2023-07-13T22:27:58.037 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:27:58.037 DEBUG:teuthology.parallel:result is None 2023-07-13T22:27:58.038 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:27:58.038 DEBUG:teuthology.parallel:result is None 2023-07-13T22:27:58.041 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:27:58.042 DEBUG:teuthology.parallel:result is None 2023-07-13T22:27:58.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-13T22:27:58.189 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:27:58.190 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:27:58.190 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:27:58.191 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:27:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.277+0000 7fe8d31ff700 1 Processor -- start 2023-07-13T22:27:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.277+0000 7fe8d31ff700 1 -- start start 2023-07-13T22:27:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.278+0000 7fe8d31ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe8cc0b8570 0x7fe8cc0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.278+0000 7fe8d31ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0d35c0 0x7fe8cc0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8d31ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8cc0d09d0 con 0x7fe8cc0cf3f0 2023-07-13T22:27:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8d31ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8cc0d3b30 con 0x7fe8cc0b8570 2023-07-13T22:27:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8d31ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc0cda10 con 0x7fe8cc0d35c0 2023-07-13T22:27:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8d179c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe8cc0cf3f0 0x7fe8cc15fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36258/0 (socket says 172.21.15.60:36258) 2023-07-13T22:27:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8d179c700 1 -- 172.21.15.60:0/2804467903 learned_addr learned my addr 172.21.15.60:0/2804467903 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:27:58.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8cbfff700 1 --2- 172.21.15.60:0/2804467903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0d35c0 0x7fe8cc0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:58.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8d0f9b700 1 --2- 172.21.15.60:0/2804467903 >> v2:172.21.15.105:3300/0 conn(0x7fe8cc0b8570 0x7fe8cc0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:58.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8cb7fe700 1 -- 172.21.15.60:0/2804467903 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316538071 0 0) 0x7fe8cc0d09d0 con 0x7fe8cc0cf3f0 2023-07-13T22:27:58.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8cb7fe700 1 -- 172.21.15.60:0/2804467903 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8bc0035b0 con 0x7fe8cc0cf3f0 2023-07-13T22:27:58.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8cbfff700 1 -- 172.21.15.60:0/2804467903 >> v2:172.21.15.105:3300/0 conn(0x7fe8cc0b8570 msgr2=0x7fe8cc0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:58.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8cbfff700 1 --2- 172.21.15.60:0/2804467903 >> v2:172.21.15.105:3300/0 conn(0x7fe8cc0b8570 0x7fe8cc0ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:58.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8cbfff700 1 -- 172.21.15.60:0/2804467903 >> v1:172.21.15.60:6789/0 conn(0x7fe8cc0cf3f0 legacy=0x7fe8cc15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:58.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8cbfff700 1 -- 172.21.15.60:0/2804467903 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc0cd450 con 0x7fe8cc0d35c0 2023-07-13T22:27:58.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.279+0000 7fe8d0f9b700 1 --2- 172.21.15.60:0/2804467903 >> v2:172.21.15.105:3300/0 conn(0x7fe8cc0b8570 0x7fe8cc0ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:58.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.280+0000 7fe8cbfff700 1 --2- 172.21.15.60:0/2804467903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0d35c0 0x7fe8cc0d0eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009840 tx=0x7fe8b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a35db37e2bfc532 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.280+0000 7fe8cb7fe700 1 -- 172.21.15.60:0/2804467903 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8011830 con 0x7fe8cc0d35c0 2023-07-13T22:27:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.280+0000 7fe8cb7fe700 1 -- 172.21.15.60:0/2804467903 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8011990 con 0x7fe8cc0d35c0 2023-07-13T22:27:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.280+0000 7fe8cb7fe700 1 -- 172.21.15.60:0/2804467903 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8011d00 con 0x7fe8cc0d35c0 2023-07-13T22:27:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.281+0000 7fe8d31ff700 1 -- 172.21.15.60:0/2804467903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0d35c0 msgr2=0x7fe8cc0d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.281+0000 7fe8d31ff700 1 --2- 172.21.15.60:0/2804467903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0d35c0 0x7fe8cc0d0eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009840 tx=0x7fe8b8009b50 comp rx=0 tx=0).stop 2023-07-13T22:27:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.281+0000 7fe8d31ff700 1 -- 172.21.15.60:0/2804467903 shutdown_connections 2023-07-13T22:27:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.281+0000 7fe8d31ff700 1 --2- 172.21.15.60:0/2804467903 >> v2:172.21.15.105:3300/0 conn(0x7fe8cc0b8570 0x7fe8cc0ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.281+0000 7fe8d31ff700 1 --2- 172.21.15.60:0/2804467903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0d35c0 0x7fe8cc0d0eb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.281+0000 7fe8d31ff700 1 -- 172.21.15.60:0/2804467903 >> 172.21.15.60:0/2804467903 conn(0x7fe8cc071600 msgr2=0x7fe8cc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.281+0000 7fe8d31ff700 1 -- 172.21.15.60:0/2804467903 shutdown_connections 2023-07-13T22:27:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.281+0000 7fe8d31ff700 1 -- 172.21.15.60:0/2804467903 wait complete. 2023-07-13T22:27:58.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.282+0000 7fe8d31ff700 1 Processor -- start 2023-07-13T22:27:58.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.283+0000 7fe8d31ff700 1 -- start start 2023-07-13T22:27:58.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d31ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0b8570 0x7fe8cc148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:58.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d31ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8cc0cf3f0 0x7fe8cc0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:58.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d31ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8cc0d35c0 0x7fe8cc157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:58.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d31ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc0c7160 con 0x7fe8cc0cf3f0 2023-07-13T22:27:58.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d31ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc156820 con 0x7fe8cc0d35c0 2023-07-13T22:27:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d31ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc0c90a0 con 0x7fe8cc0b8570 2023-07-13T22:27:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d0f9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0b8570 0x7fe8cc148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8cbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8cc0cf3f0 0x7fe8cc0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d0f9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0b8570 0x7fe8cc148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:40970/0 (socket says 172.21.15.60:40970) 2023-07-13T22:27:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d0f9b700 1 -- 172.21.15.60:0/354610168 learned_addr learned my addr 172.21.15.60:0/354610168 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.284+0000 7fe8d179c700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8cc0d35c0 0x7fe8cc157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8d0f9b700 1 -- 172.21.15.60:0/354610168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8cc0d35c0 msgr2=0x7fe8cc157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8d0f9b700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8cc0d35c0 0x7fe8cc157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:58.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8d0f9b700 1 -- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8cc0cf3f0 msgr2=0x7fe8cc0c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:58.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8d0f9b700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8cc0cf3f0 0x7fe8cc0c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:58.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8d0f9b700 1 -- 172.21.15.60:0/354610168 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c0009430 con 0x7fe8cc0b8570 2023-07-13T22:27:58.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8cbfff700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8cc0cf3f0 0x7fe8cc0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:27:58.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8d0f9b700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0b8570 0x7fe8cc148f30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000ebe0 tx=0x7fe8c000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:58.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8c97fa700 1 -- 172.21.15.60:0/354610168 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0005a20 con 0x7fe8cc0b8570 2023-07-13T22:27:58.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8c97fa700 1 -- 172.21.15.60:0/354610168 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c0005b80 con 0x7fe8cc0b8570 2023-07-13T22:27:58.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8c97fa700 1 -- 172.21.15.60:0/354610168 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c000c850 con 0x7fe8cc0b8570 2023-07-13T22:27:58.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b80094a0 con 0x7fe8cc0b8570 2023-07-13T22:27:58.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.285+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc0c95d0 con 0x7fe8cc0b8570 2023-07-13T22:27:58.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.286+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8cc04e280 con 0x7fe8cc0b8570 2023-07-13T22:27:58.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.287+0000 7fe8c97fa700 1 -- 172.21.15.60:0/354610168 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8c000c9b0 con 0x7fe8cc0b8570 2023-07-13T22:27:58.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.288+0000 7fe8c97fa700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe8b006ace0 0x7fe8b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:58.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.288+0000 7fe8c97fa700 1 -- 172.21.15.60:0/354610168 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe8c0089410 con 0x7fe8cc0b8570 2023-07-13T22:27:58.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.289+0000 7fe8cbfff700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe8b006ace0 0x7fe8b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:58.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.289+0000 7fe8cbfff700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe8b006ace0 0x7fe8b006d190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8000c00 tx=0x7fe8b801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:58.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.293+0000 7fe8c97fa700 1 -- 172.21.15.60:0/354610168 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe8c0054740 con 0x7fe8cc0b8570 2023-07-13T22:27:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:58.519+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe8cc0d2180 con 0x7fe8cc0b8570 2023-07-13T22:27:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.531+0000 7fe8c97fa700 1 -- 172.21.15.60:0/354610168 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe8c0057d60 con 0x7fe8cc0b8570 2023-07-13T22:27:59.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe8b006ace0 msgr2=0x7fe8b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:59.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe8b006ace0 0x7fe8b006d190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8000c00 tx=0x7fe8b801c040 comp rx=0 tx=0).stop 2023-07-13T22:27:59.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0b8570 msgr2=0x7fe8cc148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:59.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0b8570 0x7fe8cc148f30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000ebe0 tx=0x7fe8c000eef0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 shutdown_connections 2023-07-13T22:27:59.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe8b006ace0 0x7fe8b006d190 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8cc0d35c0 0x7fe8cc157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8cc0b8570 0x7fe8cc148f30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 --2- 172.21.15.60:0/354610168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8cc0cf3f0 0x7fe8cc0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.534+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 >> 172.21.15.60:0/354610168 conn(0x7fe8cc071600 msgr2=0x7fe8cc15e110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:59.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.535+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 shutdown_connections 2023-07-13T22:27:59.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.535+0000 7fe8d31ff700 1 -- 172.21.15.60:0/354610168 wait complete. 2023-07-13T22:27:59.519 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-07-13T22:27:59.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-13T22:27:59.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.765+0000 7f52887ff700 1 Processor -- start 2023-07-13T22:27:59.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.765+0000 7f52887ff700 1 -- start start 2023-07-13T22:27:59.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.767+0000 7f52887ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f528015a820 0x7f528015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:59.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.767+0000 7f52887ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f52800c8d40 0x7f52800c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:59.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.767+0000 7f52887ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5280070270 con 0x7f5280154950 2023-07-13T22:27:59.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.767+0000 7f52887ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52800713d0 con 0x7f528015a820 2023-07-13T22:27:59.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.767+0000 7f52887ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52800c96a0 con 0x7f52800c8d40 2023-07-13T22:27:59.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.767+0000 7f5285d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f528015a820 0x7f528015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:59.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.767+0000 7f5285d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f528015a820 0x7f528015ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40864/0 (socket says 172.21.15.60:40864) 2023-07-13T22:27:59.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.767+0000 7f5285d9a700 1 -- 172.21.15.60:0/3020752069 learned_addr learned my addr 172.21.15.60:0/3020752069 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:59.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.767+0000 7f5286d9c700 1 --2- 172.21.15.60:0/3020752069 >> v2:172.21.15.105:3300/0 conn(0x7f52800c8d40 0x7f52800c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:59.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.768+0000 7f5285d9a700 1 -- 172.21.15.60:0/3020752069 >> v2:172.21.15.105:3300/0 conn(0x7f52800c8d40 msgr2=0x7f52800c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:27:59.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.768+0000 7f5285d9a700 1 --2- 172.21.15.60:0/3020752069 >> v2:172.21.15.105:3300/0 conn(0x7f52800c8d40 0x7f52800c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.768+0000 7f5285d9a700 1 -- 172.21.15.60:0/3020752069 >> v1:172.21.15.105:6789/0 conn(0x7f5280154950 legacy=0x7f5280070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.768+0000 7f5285d9a700 1 -- 172.21.15.60:0/3020752069 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528015b2e0 con 0x7f528015a820 2023-07-13T22:27:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.768+0000 7f5285d9a700 1 --2- 172.21.15.60:0/3020752069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528015a820 0x7f528015ac70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f528006b060 tx=0x7f5270009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dcc31a89a9bce5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.768+0000 7f5285599700 1 -- 172.21.15.60:0/3020752069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270011830 con 0x7f528015a820 2023-07-13T22:27:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.768+0000 7f5285599700 1 -- 172.21.15.60:0/3020752069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5270011990 con 0x7f528015a820 2023-07-13T22:27:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.768+0000 7f5285599700 1 -- 172.21.15.60:0/3020752069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270011d20 con 0x7f528015a820 2023-07-13T22:27:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.769+0000 7f52887ff700 1 -- 172.21.15.60:0/3020752069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528015a820 msgr2=0x7f528015ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.769+0000 7f52887ff700 1 --2- 172.21.15.60:0/3020752069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528015a820 0x7f528015ac70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f528006b060 tx=0x7f5270009bd0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.769+0000 7f52887ff700 1 -- 172.21.15.60:0/3020752069 shutdown_connections 2023-07-13T22:27:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.769+0000 7f52887ff700 1 --2- 172.21.15.60:0/3020752069 >> v2:172.21.15.105:3300/0 conn(0x7f52800c8d40 0x7f52800c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.769+0000 7f52887ff700 1 --2- 172.21.15.60:0/3020752069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528015a820 0x7f528015ac70 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.769+0000 7f52887ff700 1 -- 172.21.15.60:0/3020752069 >> 172.21.15.60:0/3020752069 conn(0x7f52800cf410 msgr2=0x7f52800cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:27:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.769+0000 7f52887ff700 1 -- 172.21.15.60:0/3020752069 shutdown_connections 2023-07-13T22:27:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.769+0000 7f52887ff700 1 -- 172.21.15.60:0/3020752069 wait complete. 2023-07-13T22:27:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.771+0000 7f52887ff700 1 Processor -- start 2023-07-13T22:27:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.771+0000 7f52887ff700 1 -- start start 2023-07-13T22:27:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f52887ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52800c8d40 0x7f528013f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f52887ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5280154950 0x7f52800c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f52887ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528015a820 0x7f52800c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f52887ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52800c8360 con 0x7f5280154950 2023-07-13T22:27:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f52887ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52800c7da0 con 0x7f528015a820 2023-07-13T22:27:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f52887ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52800c8080 con 0x7f52800c8d40 2023-07-13T22:27:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5285d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5280154950 0x7f52800c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5285d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5280154950 0x7f52800c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40870/0 (socket says 172.21.15.60:40870) 2023-07-13T22:27:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5285d9a700 1 -- 172.21.15.60:0/1695618153 learned_addr learned my addr 172.21.15.60:0/1695618153 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:27:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f528659b700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52800c8d40 0x7f528013f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5286d9c700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528015a820 0x7f52800c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5285d9a700 1 -- 172.21.15.60:0/1695618153 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52800c8d40 msgr2=0x7f528013f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5285d9a700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52800c8d40 0x7f528013f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5285d9a700 1 -- 172.21.15.60:0/1695618153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528015a820 msgr2=0x7f52800c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:27:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5285d9a700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528015a820 0x7f52800c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:27:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5285d9a700 1 -- 172.21.15.60:0/1695618153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5270009550 con 0x7f5280154950 2023-07-13T22:27:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.772+0000 7f5286d9c700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528015a820 0x7f52800c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:27:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.773+0000 7f5285d9a700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5280154950 0x7f52800c6d00 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5270009c10 tx=0x7f5270004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.773+0000 7f5276ffd700 1 -- 172.21.15.60:0/1695618153 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52700036a0 con 0x7f5280154950 2023-07-13T22:27:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.773+0000 7f5276ffd700 1 -- 172.21.15.60:0/1695618153 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527001e070 con 0x7f5280154950 2023-07-13T22:27:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.773+0000 7f5276ffd700 1 -- 172.21.15.60:0/1695618153 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270003a60 con 0x7f5280154950 2023-07-13T22:27:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.773+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52800ae4e0 con 0x7f5280154950 2023-07-13T22:27:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.773+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528013f5c0 con 0x7f5280154950 2023-07-13T22:27:59.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.774+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52800aea00 con 0x7f5280154950 2023-07-13T22:27:59.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.775+0000 7f5276ffd700 1 -- 172.21.15.60:0/1695618153 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5270003bc0 con 0x7f5280154950 2023-07-13T22:27:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.776+0000 7f5276ffd700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f526406ace0 0x7f526406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:27:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.776+0000 7f5276ffd700 1 -- 172.21.15.60:0/1695618153 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f527008a0d0 con 0x7f5280154950 2023-07-13T22:27:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.776+0000 7f528659b700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f526406ace0 0x7f526406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:27:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.777+0000 7f528659b700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f526406ace0 0x7f526406d190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f527c000f60 tx=0x7f527c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:27:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:27:59.780+0000 7f5276ffd700 1 -- 172.21.15.60:0/1695618153 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f52800aea00 con 0x7f5280154950 2023-07-13T22:28:00.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.062+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f52800aea00 con 0x7f5280154950 2023-07-13T22:28:00.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.062+0000 7f5276ffd700 1 -- 172.21.15.60:0/1695618153 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f52800aea00 con 0x7f5280154950 2023-07-13T22:28:00.043 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:00.044 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:28:00.044 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-07-13T22:28:00.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.044 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-07-13T22:28:00.044 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw==", 2023-07-13T22:28:00.044 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.045 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:00.045 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:00.045 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:00.045 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-07-13T22:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g==", 2023-07-13T22:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.047 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-07-13T22:28:00.047 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA==", 2023-07-13T22:28:00.047 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.047 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:00.047 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:00.047 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:00.048 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.048 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.048 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-07-13T22:28:00.048 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ==", 2023-07-13T22:28:00.048 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.049 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:00.049 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:00.049 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:00.049 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.049 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.049 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-07-13T22:28:00.050 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ==", 2023-07-13T22:28:00.050 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.050 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:00.050 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:00.050 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.050 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.051 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-07-13T22:28:00.051 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw==", 2023-07-13T22:28:00.051 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.051 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:00.051 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:00.051 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.051 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.052 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-07-13T22:28:00.052 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q==", 2023-07-13T22:28:00.052 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.052 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:00.052 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:00.052 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.053 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.053 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-07-13T22:28:00.053 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ==", 2023-07-13T22:28:00.053 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.053 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:00.053 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:00.054 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.054 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.054 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-07-13T22:28:00.054 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg==", 2023-07-13T22:28:00.054 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.054 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:00.055 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:00.055 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.055 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-07-13T22:28:00.055 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA==", 2023-07-13T22:28:00.056 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.056 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:00.056 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:00.056 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.056 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.056 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-07-13T22:28:00.057 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw==", 2023-07-13T22:28:00.057 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.057 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:00.057 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:00.057 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.058 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-07-13T22:28:00.058 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ==", 2023-07-13T22:28:00.058 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.058 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:00.058 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:00.058 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.059 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-07-13T22:28:00.059 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w==", 2023-07-13T22:28:00.059 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.059 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:00.059 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2023-07-13T22:28:00.059 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-13T22:28:00.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.060 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-07-13T22:28:00.060 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw==", 2023-07-13T22:28:00.060 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.060 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:00.061 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:28:00.061 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:28:00.061 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:28:00.061 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.061 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-07-13T22:28:00.061 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg==", 2023-07-13T22:28:00.062 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.062 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:00.062 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:28:00.062 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:28:00.062 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:28:00.062 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.063 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-07-13T22:28:00.063 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q==", 2023-07-13T22:28:00.063 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.063 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:00.063 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:28:00.063 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:28:00.063 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:28:00.063 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.064 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-07-13T22:28:00.064 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBk7CgGBhAAaK88V4CEyrYfZx7wv+XG3g==", 2023-07-13T22:28:00.064 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.064 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:28:00.064 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:00.064 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:00.065 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.065 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.065 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-07-13T22:28:00.065 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ==", 2023-07-13T22:28:00.065 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.066 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:28:00.066 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:28:00.066 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:00.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:00.066 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-07-13T22:28:00.066 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ==", 2023-07-13T22:28:00.067 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:00.067 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:28:00.067 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:28:00.067 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:00.067 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.067 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:00.067 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:28:00.067 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:28:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.064+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f526406ace0 msgr2=0x7f526406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.064+0000 7f52887ff700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f526406ace0 0x7f526406d190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f527c000f60 tx=0x7f527c009040 comp rx=0 tx=0).stop 2023-07-13T22:28:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.064+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5280154950 msgr2=0x7f52800c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.064+0000 7f52887ff700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5280154950 0x7f52800c6d00 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5270009c10 tx=0x7f5270004750 comp rx=0 tx=0).stop 2023-07-13T22:28:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.065+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 shutdown_connections 2023-07-13T22:28:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.065+0000 7f52887ff700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f526406ace0 0x7f526406d190 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.065+0000 7f52887ff700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528015a820 0x7f52800c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.065+0000 7f52887ff700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52800c8d40 0x7f528013f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.065+0000 7f52887ff700 1 --2- 172.21.15.60:0/1695618153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5280154950 0x7f52800c6d00 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.065+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 >> 172.21.15.60:0/1695618153 conn(0x7f52800cf410 msgr2=0x7f528006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:00.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.065+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 shutdown_connections 2023-07-13T22:28:00.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.065+0000 7f52887ff700 1 -- 172.21.15.60:0/1695618153 wait complete. 2023-07-13T22:28:00.071 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:00.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.293+0000 7fcd3f1ff700 1 Processor -- start 2023-07-13T22:28:00.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.293+0000 7fcd3f1ff700 1 -- start start 2023-07-13T22:28:00.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.294+0000 7fcd3f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd380c9450 0x7fcd3815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.294+0000 7fcd3f1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd38070270 con 0x7fcd38154950 2023-07-13T22:28:00.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.294+0000 7fcd3f1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3815b2e0 con 0x7fcd3815a820 2023-07-13T22:28:00.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.294+0000 7fcd3f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380713d0 con 0x7fcd380c9450 2023-07-13T22:28:00.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd3cf9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcd38154950 0x7fcd38070cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36272/0 (socket says 172.21.15.60:36272) 2023-07-13T22:28:00.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd3cf9b700 1 -- 172.21.15.60:0/1702881407 learned_addr learned my addr 172.21.15.60:0/1702881407 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:00.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd3d79c700 1 --2- 172.21.15.60:0/1702881407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd380c9450 0x7fcd3815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd377fe700 1 -- 172.21.15.60:0/1702881407 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086841611 0 0) 0x7fcd38070270 con 0x7fcd38154950 2023-07-13T22:28:00.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd377fe700 1 -- 172.21.15.60:0/1702881407 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd280035b0 con 0x7fcd38154950 2023-07-13T22:28:00.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd377fe700 1 -- 172.21.15.60:0/1702881407 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376066577 0 0) 0x7fcd3815b2e0 con 0x7fcd3815a820 2023-07-13T22:28:00.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd377fe700 1 -- 172.21.15.60:0/1702881407 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd38070270 con 0x7fcd3815a820 2023-07-13T22:28:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd3d79c700 1 -- 172.21.15.60:0/1702881407 >> v1:172.21.15.105:6789/0 conn(0x7fcd3815a820 legacy=0x7fcd380c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd3d79c700 1 -- 172.21.15.60:0/1702881407 >> v1:172.21.15.60:6789/0 conn(0x7fcd38154950 legacy=0x7fcd38070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd377fe700 1 -- 172.21.15.60:0/1702881407 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2596644044 0 0) 0x7fcd280035b0 con 0x7fcd38154950 2023-07-13T22:28:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.295+0000 7fcd3d79c700 1 -- 172.21.15.60:0/1702881407 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3815b5c0 con 0x7fcd380c9450 2023-07-13T22:28:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.296+0000 7fcd3d79c700 1 --2- 172.21.15.60:0/1702881407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd380c9450 0x7fcd3815ac70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009dc0 tx=0x7fcd3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0fd50700999c44 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.296+0000 7fcd377fe700 1 -- 172.21.15.60:0/1702881407 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000e070 con 0x7fcd380c9450 2023-07-13T22:28:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.296+0000 7fcd377fe700 1 -- 172.21.15.60:0/1702881407 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd30003a20 con 0x7fcd380c9450 2023-07-13T22:28:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.296+0000 7fcd377fe700 1 -- 172.21.15.60:0/1702881407 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c001670 con 0x7fcd380c9450 2023-07-13T22:28:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.296+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1702881407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd380c9450 msgr2=0x7fcd3815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.296+0000 7fcd3f1ff700 1 --2- 172.21.15.60:0/1702881407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd380c9450 0x7fcd3815ac70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009dc0 tx=0x7fcd3000b330 comp rx=0 tx=0).stop 2023-07-13T22:28:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.297+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1702881407 shutdown_connections 2023-07-13T22:28:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.297+0000 7fcd3f1ff700 1 --2- 172.21.15.60:0/1702881407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd380c9450 0x7fcd3815ac70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.297+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1702881407 >> 172.21.15.60:0/1702881407 conn(0x7fcd380cf410 msgr2=0x7fcd380cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:00.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.297+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1702881407 shutdown_connections 2023-07-13T22:28:00.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.297+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1702881407 wait complete. 2023-07-13T22:28:00.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.298+0000 7fcd3f1ff700 1 Processor -- start 2023-07-13T22:28:00.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.299+0000 7fcd3f1ff700 1 -- start start 2023-07-13T22:28:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.299+0000 7fcd3f1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd380c9450 0x7fcd381f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.299+0000 7fcd3f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd38154950 0x7fcd380b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.299+0000 7fcd3f1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd3815a820 0x7fcd381ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3f1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3814fd90 con 0x7fcd3815a820 2023-07-13T22:28:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3f1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd381f5560 con 0x7fcd380c9450 2023-07-13T22:28:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380c7b40 con 0x7fcd38154950 2023-07-13T22:28:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3d79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd3815a820 0x7fcd381ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3d79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd3815a820 0x7fcd381ef4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40884/0 (socket says 172.21.15.60:40884) 2023-07-13T22:28:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3d79c700 1 -- 172.21.15.60:0/1470931536 learned_addr learned my addr 172.21.15.60:0/1470931536 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3cf9b700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd380c9450 0x7fcd381f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd37fff700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd38154950 0x7fcd380b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3d79c700 1 -- 172.21.15.60:0/1470931536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd38154950 msgr2=0x7fcd380b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3d79c700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd38154950 0x7fcd380b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3d79c700 1 -- 172.21.15.60:0/1470931536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd380c9450 msgr2=0x7fcd381f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3d79c700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd380c9450 0x7fcd381f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3d79c700 1 -- 172.21.15.60:0/1470931536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30009a70 con 0x7fcd3815a820 2023-07-13T22:28:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.300+0000 7fcd3d79c700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd3815a820 0x7fcd381ef4c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009d90 tx=0x7fcd30004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.301+0000 7fcd357fa700 1 -- 172.21.15.60:0/1470931536 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd300046f0 con 0x7fcd3815a820 2023-07-13T22:28:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.301+0000 7fcd357fa700 1 -- 172.21.15.60:0/1470931536 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd30004850 con 0x7fcd3815a820 2023-07-13T22:28:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.301+0000 7fcd357fa700 1 -- 172.21.15.60:0/1470931536 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3001d670 con 0x7fcd3815a820 2023-07-13T22:28:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.301+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd380c7e20 con 0x7fcd3815a820 2023-07-13T22:28:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.301+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd381f5b00 con 0x7fcd3815a820 2023-07-13T22:28:00.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.302+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd380c7fe0 con 0x7fcd3815a820 2023-07-13T22:28:00.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.303+0000 7fcd357fa700 1 -- 172.21.15.60:0/1470931536 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd3001d7d0 con 0x7fcd3815a820 2023-07-13T22:28:00.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.304+0000 7fcd357fa700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd1c06ace0 0x7fcd1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.304+0000 7fcd357fa700 1 -- 172.21.15.60:0/1470931536 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcd3008af00 con 0x7fcd3815a820 2023-07-13T22:28:00.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.304+0000 7fcd3cf9b700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd1c06ace0 0x7fcd1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.305+0000 7fcd3cf9b700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd1c06ace0 0x7fcd1c06d190 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c000f60 tx=0x7fcd2c001c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:00.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.308+0000 7fcd357fa700 1 -- 172.21.15.60:0/1470931536 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcd300562f0 con 0x7fcd3815a820 2023-07-13T22:28:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.515+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd381f5cf0 con 0x7fcd3815a820 2023-07-13T22:28:00.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.515+0000 7fcd357fa700 1 -- 172.21.15.60:0/1470931536 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+14546 (secure 0 0 0) 0x7fcd30059910 con 0x7fcd3815a820 2023-07-13T22:28:00.496 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:00.496 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":83,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:27:59.525624+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:00.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd1c06ace0 msgr2=0x7fcd1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd1c06ace0 0x7fcd1c06d190 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c000f60 tx=0x7fcd2c001c00 comp rx=0 tx=0).stop 2023-07-13T22:28:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd3815a820 msgr2=0x7fcd381ef4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd3815a820 0x7fcd381ef4c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009d90 tx=0x7fcd30004ad0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 shutdown_connections 2023-07-13T22:28:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd1c06ace0 0x7fcd1c06d190 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd3815a820 0x7fcd381ef4c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd380c9450 0x7fcd381f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 --2- 172.21.15.60:0/1470931536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd38154950 0x7fcd380b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 >> 172.21.15.60:0/1470931536 conn(0x7fcd380cf410 msgr2=0x7fcd380c9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:00.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 shutdown_connections 2023-07-13T22:28:00.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.518+0000 7fcd3f1ff700 1 -- 172.21.15.60:0/1470931536 wait complete. 2023-07-13T22:28:00.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:28:00.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:28:00.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.744+0000 7f75f47ff700 1 Processor -- start 2023-07-13T22:28:00.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.744+0000 7f75f47ff700 1 -- start start 2023-07-13T22:28:00.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.745+0000 7f75f47ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f75ec0b8200 0x7f75ec0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.745+0000 7f75f47ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0cede0 0x7f75ec0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.745+0000 7f75f47ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec0d1870 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.745+0000 7f75f47ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ec0cf370 con 0x7f75ec0b8200 2023-07-13T22:28:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.745+0000 7f75f47ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0d12b0 con 0x7f75ec0cede0 2023-07-13T22:28:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.745+0000 7f75f259b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f75ec0b8200 0x7f75ec0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.745+0000 7f75f259b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f75ec0b8200 0x7f75ec0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40900/0 (socket says 172.21.15.60:40900) 2023-07-13T22:28:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.745+0000 7f75f259b700 1 -- 172.21.15.60:0/24336265 learned_addr learned my addr 172.21.15.60:0/24336265 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.745+0000 7f75f2d9c700 1 --2- 172.21.15.60:0/24336265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0cede0 0x7f75ec0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.746+0000 7f75f259b700 1 -- 172.21.15.60:0/24336265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0cede0 msgr2=0x7f75ec0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.746+0000 7f75f259b700 1 --2- 172.21.15.60:0/24336265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0cede0 0x7f75ec0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.746+0000 7f75f259b700 1 -- 172.21.15.60:0/24336265 >> v1:172.21.15.105:6789/0 conn(0x7f75ec0b1ac0 legacy=0x7f75ec15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.746+0000 7f75f259b700 1 -- 172.21.15.60:0/24336265 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec0d1590 con 0x7f75ec0b8200 2023-07-13T22:28:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.746+0000 7f75f259b700 1 --2- 172.21.15.60:0/24336265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b8200 0x7f75ec0cf640 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f75e80098f0 tx=0x7f75e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b0742f9e794ac8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.746+0000 7f75f1599700 1 -- 172.21.15.60:0/24336265 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800b8a0 con 0x7f75ec0b8200 2023-07-13T22:28:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.746+0000 7f75f1599700 1 -- 172.21.15.60:0/24336265 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e800ba00 con 0x7f75ec0b8200 2023-07-13T22:28:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.746+0000 7f75f1599700 1 -- 172.21.15.60:0/24336265 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800bd90 con 0x7f75ec0b8200 2023-07-13T22:28:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.747+0000 7f75f47ff700 1 -- 172.21.15.60:0/24336265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b8200 msgr2=0x7f75ec0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.747+0000 7f75f47ff700 1 --2- 172.21.15.60:0/24336265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b8200 0x7f75ec0cf640 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f75e80098f0 tx=0x7f75e8009c00 comp rx=0 tx=0).stop 2023-07-13T22:28:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.747+0000 7f75f47ff700 1 -- 172.21.15.60:0/24336265 shutdown_connections 2023-07-13T22:28:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.747+0000 7f75f47ff700 1 --2- 172.21.15.60:0/24336265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0cede0 0x7f75ec0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.747+0000 7f75f47ff700 1 --2- 172.21.15.60:0/24336265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b8200 0x7f75ec0cf640 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.747+0000 7f75f47ff700 1 -- 172.21.15.60:0/24336265 >> 172.21.15.60:0/24336265 conn(0x7f75ec06b2f0 msgr2=0x7f75ec06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.747+0000 7f75f47ff700 1 -- 172.21.15.60:0/24336265 shutdown_connections 2023-07-13T22:28:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.747+0000 7f75f47ff700 1 -- 172.21.15.60:0/24336265 wait complete. 2023-07-13T22:28:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.749+0000 7f75f47ff700 1 Processor -- start 2023-07-13T22:28:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.749+0000 7f75f47ff700 1 -- start start 2023-07-13T22:28:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f47ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b1ac0 0x7f75ec1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f47ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0b8200 0x7f75ec157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f47ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75ec0cede0 0x7f75ec0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f47ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0c8de0 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f47ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec158090 con 0x7f75ec0cede0 2023-07-13T22:28:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f47ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0c8440 con 0x7f75ec0b8200 2023-07-13T22:28:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f259b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b1ac0 0x7f75ec1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f259b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b1ac0 0x7f75ec1565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40906/0 (socket says 172.21.15.60:40906) 2023-07-13T22:28:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f259b700 1 -- 172.21.15.60:0/3341271557 learned_addr learned my addr 172.21.15.60:0/3341271557 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f1d9a700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0b8200 0x7f75ec157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f2d9c700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75ec0cede0 0x7f75ec0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f259b700 1 -- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0b8200 msgr2=0x7f75ec157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f259b700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0b8200 0x7f75ec157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f259b700 1 -- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75ec0cede0 msgr2=0x7f75ec0c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f259b700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75ec0cede0 0x7f75ec0c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f259b700 1 -- 172.21.15.60:0/3341271557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8009550 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f1d9a700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0b8200 0x7f75ec157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.750+0000 7f75f2d9c700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75ec0cede0 0x7f75ec0c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:00.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.751+0000 7f75f259b700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b1ac0 0x7f75ec1565a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f75e8000c00 tx=0x7f75e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.751+0000 7f75e2ffd700 1 -- 172.21.15.60:0/3341271557 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8004480 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.751+0000 7f75e2ffd700 1 -- 172.21.15.60:0/3341271557 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e801f070 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.751+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec0c8720 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.751+0000 7f75e2ffd700 1 -- 172.21.15.60:0/3341271557 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e80038c0 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.751+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec156ae0 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.752+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec04e210 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.753+0000 7f75e2ffd700 1 -- 172.21.15.60:0/3341271557 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f75e8021030 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.754+0000 7f75e2ffd700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f75d006ace0 0x7f75d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.754+0000 7f75e2ffd700 1 -- 172.21.15.60:0/3341271557 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f75e808a710 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.754+0000 7f75f1d9a700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f75d006ace0 0x7f75d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.755+0000 7f75f1d9a700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f75d006ace0 0x7f75d006d190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f75dc000f60 tx=0x7f75dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.758+0000 7f75e2ffd700 1 -- 172.21.15.60:0/3341271557 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f75e8055c20 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.965+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f75ec0b4f70 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.966+0000 7f75e2ffd700 1 -- 172.21.15.60:0/3341271557 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v83) v1 ==== 92+0+10 (secure 0 0 0) 0x7f75e8059240 con 0x7f75ec0b1ac0 2023-07-13T22:28:00.947 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:28:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f75d006ace0 msgr2=0x7f75d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f75d006ace0 0x7f75d006d190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f75dc000f60 tx=0x7f75dc009040 comp rx=0 tx=0).stop 2023-07-13T22:28:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b1ac0 msgr2=0x7f75ec1565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b1ac0 0x7f75ec1565a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f75e8000c00 tx=0x7f75e8004750 comp rx=0 tx=0).stop 2023-07-13T22:28:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 shutdown_connections 2023-07-13T22:28:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f75d006ace0 0x7f75d006d190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec0b1ac0 0x7f75ec1565a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75ec0cede0 0x7f75ec0c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 --2- 172.21.15.60:0/3341271557 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75ec0b8200 0x7f75ec157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:00.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 >> 172.21.15.60:0/3341271557 conn(0x7f75ec06b2f0 msgr2=0x7f75ec0d2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:00.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 shutdown_connections 2023-07-13T22:28:00.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:00.968+0000 7f75f47ff700 1 -- 172.21.15.60:0/3341271557 wait complete. 2023-07-13T22:28:00.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:28:01.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.193+0000 7f73925ff700 1 Processor -- start 2023-07-13T22:28:01.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.194+0000 7f73925ff700 1 -- start start 2023-07-13T22:28:01.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f73925ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f738c0b8200 0x7f738c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f73925ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c0d1870 con 0x7f738c0b1ac0 2023-07-13T22:28:01.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f73925ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c160d80 con 0x7f738c0cede0 2023-07-13T22:28:01.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f73925ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c0cf370 con 0x7f738c0b8200 2023-07-13T22:28:01.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f738b7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f738c0b1ac0 0x7f738c15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36288/0 (socket says 172.21.15.60:36288) 2023-07-13T22:28:01.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f738b7fe700 1 -- 172.21.15.60:0/1914526402 learned_addr learned my addr 172.21.15.60:0/1914526402 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f738bfff700 1 --2- 172.21.15.60:0/1914526402 >> v2:172.21.15.60:3300/0 conn(0x7f738c0b8200 0x7f738c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f738bfff700 1 -- 172.21.15.60:0/1914526402 >> v1:172.21.15.105:6789/0 conn(0x7f738c0cede0 legacy=0x7f738c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f738bfff700 1 -- 172.21.15.60:0/1914526402 >> v1:172.21.15.60:6789/0 conn(0x7f738c0b1ac0 legacy=0x7f738c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.195+0000 7f738bfff700 1 -- 172.21.15.60:0/1914526402 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c0d12b0 con 0x7f738c0b8200 2023-07-13T22:28:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.196+0000 7f738bfff700 1 --2- 172.21.15.60:0/1914526402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0b8200 0x7f738c0cf640 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f737c009840 tx=0x7f737c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cb61d2a7d9ac84 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.196+0000 7f738affd700 1 -- 172.21.15.60:0/1914526402 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c011830 con 0x7f738c0b8200 2023-07-13T22:28:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.196+0000 7f738affd700 1 -- 172.21.15.60:0/1914526402 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c011990 con 0x7f738c0b8200 2023-07-13T22:28:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.196+0000 7f738affd700 1 -- 172.21.15.60:0/1914526402 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c011d20 con 0x7f738c0b8200 2023-07-13T22:28:01.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.196+0000 7f73925ff700 1 -- 172.21.15.60:0/1914526402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0b8200 msgr2=0x7f738c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.196+0000 7f73925ff700 1 --2- 172.21.15.60:0/1914526402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0b8200 0x7f738c0cf640 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f737c009840 tx=0x7f737c009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:01.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.197+0000 7f73925ff700 1 -- 172.21.15.60:0/1914526402 shutdown_connections 2023-07-13T22:28:01.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.197+0000 7f73925ff700 1 --2- 172.21.15.60:0/1914526402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0b8200 0x7f738c0cf640 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.197+0000 7f73925ff700 1 -- 172.21.15.60:0/1914526402 >> 172.21.15.60:0/1914526402 conn(0x7f738c06b2f0 msgr2=0x7f738c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:01.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.197+0000 7f73925ff700 1 -- 172.21.15.60:0/1914526402 shutdown_connections 2023-07-13T22:28:01.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.197+0000 7f73925ff700 1 -- 172.21.15.60:0/1914526402 wait complete. 2023-07-13T22:28:01.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.198+0000 7f73925ff700 1 Processor -- start 2023-07-13T22:28:01.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.199+0000 7f73925ff700 1 -- start start 2023-07-13T22:28:01.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f73925ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f738c0b1ac0 0x7f738c154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f73925ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f738c0b8200 0x7f738c0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f73925ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0cede0 0x7f738c0c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f73925ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0caab0 con 0x7f738c0cede0 2023-07-13T22:28:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f73925ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0c9ea0 con 0x7f738c0b1ac0 2023-07-13T22:28:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f73925ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0ca180 con 0x7f738c0b8200 2023-07-13T22:28:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f7390b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0cede0 0x7f738c0c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f7390b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0cede0 0x7f738c0c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40926/0 (socket says 172.21.15.60:40926) 2023-07-13T22:28:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f7390b9c700 1 -- 172.21.15.60:0/3221285332 learned_addr learned my addr 172.21.15.60:0/3221285332 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:01.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f738bfff700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f738c0b1ac0 0x7f738c154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f738b7fe700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f738c0b8200 0x7f738c0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f7390b9c700 1 -- 172.21.15.60:0/3221285332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f738c0b8200 msgr2=0x7f738c0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f7390b9c700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f738c0b8200 0x7f738c0cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f7390b9c700 1 -- 172.21.15.60:0/3221285332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f738c0b1ac0 msgr2=0x7f738c154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f7390b9c700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f738c0b1ac0 0x7f738c154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.200+0000 7f7390b9c700 1 -- 172.21.15.60:0/3221285332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c0094a0 con 0x7f738c0cede0 2023-07-13T22:28:01.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.201+0000 7f7390b9c700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0cede0 0x7f738c0c8d00 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f738400cb50 tx=0x7f738400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:01.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.201+0000 7f7388ff9700 1 -- 172.21.15.60:0/3221285332 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384003c00 con 0x7f738c0cede0 2023-07-13T22:28:01.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.201+0000 7f7388ff9700 1 -- 172.21.15.60:0/3221285332 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7384003d60 con 0x7f738c0cede0 2023-07-13T22:28:01.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.201+0000 7f7388ff9700 1 -- 172.21.15.60:0/3221285332 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400aa20 con 0x7f738c0cede0 2023-07-13T22:28:01.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.201+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c0ca460 con 0x7f738c0cede0 2023-07-13T22:28:01.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.201+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c154740 con 0x7f738c0cede0 2023-07-13T22:28:01.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.202+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738c0cac60 con 0x7f738c0cede0 2023-07-13T22:28:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.203+0000 7f7388ff9700 1 -- 172.21.15.60:0/3221285332 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f738400ab80 con 0x7f738c0cede0 2023-07-13T22:28:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.204+0000 7f7388ff9700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f737806ace0 0x7f737806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.204+0000 7f7388ff9700 1 -- 172.21.15.60:0/3221285332 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7384089150 con 0x7f738c0cede0 2023-07-13T22:28:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.204+0000 7f738bfff700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f737806ace0 0x7f737806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.205+0000 7f738bfff700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f737806ace0 0x7f737806d190 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f737c0090b0 tx=0x7f737c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:01.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.208+0000 7f7388ff9700 1 -- 172.21.15.60:0/3221285332 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f738c0cac60 con 0x7f738c0cede0 2023-07-13T22:28:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.414+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f738c0cae40 con 0x7f738c0cede0 2023-07-13T22:28:01.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.414+0000 7f7388ff9700 1 -- 172.21.15.60:0/3221285332 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7384057f40 con 0x7f738c0cede0 2023-07-13T22:28:01.395 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:28:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f737806ace0 msgr2=0x7f737806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f737806ace0 0x7f737806d190 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f737c0090b0 tx=0x7f737c008030 comp rx=0 tx=0).stop 2023-07-13T22:28:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0cede0 msgr2=0x7f738c0c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0cede0 0x7f738c0c8d00 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f738400cb50 tx=0x7f738400ce60 comp rx=0 tx=0).stop 2023-07-13T22:28:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 shutdown_connections 2023-07-13T22:28:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f737806ace0 0x7f737806d190 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f738c0cede0 0x7f738c0c8d00 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f738c0b1ac0 0x7f738c154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 --2- 172.21.15.60:0/3221285332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f738c0b8200 0x7f738c0cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 >> 172.21.15.60:0/3221285332 conn(0x7f738c06b2f0 msgr2=0x7f738c071600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 shutdown_connections 2023-07-13T22:28:01.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.417+0000 7f73925ff700 1 -- 172.21.15.60:0/3221285332 wait complete. 2023-07-13T22:28:01.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:28:01.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.639+0000 7f4df0bff700 1 Processor -- start 2023-07-13T22:28:01.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.639+0000 7f4df0bff700 1 -- start start 2023-07-13T22:28:01.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.640+0000 7f4df0bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0d3750 0x7f4dec0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.640+0000 7f4df0bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dec0cd950 con 0x7f4dec0cfad0 2023-07-13T22:28:01.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.640+0000 7f4df0bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dec0cff20 con 0x7f4dec0b8400 2023-07-13T22:28:01.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.640+0000 7f4df0bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0d0ac0 con 0x7f4dec0d3750 2023-07-13T22:28:01.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4dea59c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4dec0b8400 0x7f4dec0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:35068/0 (socket says 172.21.15.60:35068) 2023-07-13T22:28:01.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4dead9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4dec0cfad0 0x7f4dec0d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36294/0 (socket says 172.21.15.60:36294) 2023-07-13T22:28:01.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4dea59c700 1 -- 172.21.15.60:0/1507156883 learned_addr learned my addr 172.21.15.60:0/1507156883 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:01.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4de9d9b700 1 --2- 172.21.15.60:0/1507156883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0d3750 0x7f4dec0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4de959a700 1 -- 172.21.15.60:0/1507156883 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059026086 0 0) 0x7f4de4001490 con 0x7f4dec0cfad0 2023-07-13T22:28:01.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4de959a700 1 -- 172.21.15.60:0/1507156883 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd40035b0 con 0x7f4dec0cfad0 2023-07-13T22:28:01.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4de959a700 1 -- 172.21.15.60:0/1507156883 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139028462 0 0) 0x7f4ddc001110 con 0x7f4dec0b8400 2023-07-13T22:28:01.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4de959a700 1 -- 172.21.15.60:0/1507156883 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4de4001490 con 0x7f4dec0b8400 2023-07-13T22:28:01.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4de9d9b700 1 -- 172.21.15.60:0/1507156883 >> v1:172.21.15.105:6789/0 conn(0x7f4dec0b8400 legacy=0x7f4dec0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4de9d9b700 1 -- 172.21.15.60:0/1507156883 >> v1:172.21.15.60:6789/0 conn(0x7f4dec0cfad0 legacy=0x7f4dec0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.641+0000 7f4de9d9b700 1 -- 172.21.15.60:0/1507156883 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec0b87d0 con 0x7f4dec0d3750 2023-07-13T22:28:01.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.642+0000 7f4de9d9b700 1 --2- 172.21.15.60:0/1507156883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0d3750 0x7f4dec0cf590 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009870 tx=0x7f4de0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=137e0d6ba34e3938 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:01.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.642+0000 7f4de959a700 1 -- 172.21.15.60:0/1507156883 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0011830 con 0x7f4dec0d3750 2023-07-13T22:28:01.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.642+0000 7f4de959a700 1 -- 172.21.15.60:0/1507156883 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de0011990 con 0x7f4dec0d3750 2023-07-13T22:28:01.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.642+0000 7f4de959a700 1 -- 172.21.15.60:0/1507156883 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0011d00 con 0x7f4dec0d3750 2023-07-13T22:28:01.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.642+0000 7f4df0bff700 1 -- 172.21.15.60:0/1507156883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0d3750 msgr2=0x7f4dec0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.642+0000 7f4df0bff700 1 --2- 172.21.15.60:0/1507156883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0d3750 0x7f4dec0cf590 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009870 tx=0x7f4de0009b80 comp rx=0 tx=0).stop 2023-07-13T22:28:01.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.642+0000 7f4df0bff700 1 -- 172.21.15.60:0/1507156883 shutdown_connections 2023-07-13T22:28:01.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.642+0000 7f4df0bff700 1 --2- 172.21.15.60:0/1507156883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0d3750 0x7f4dec0cf590 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.642+0000 7f4df0bff700 1 -- 172.21.15.60:0/1507156883 >> 172.21.15.60:0/1507156883 conn(0x7f4dec06b390 msgr2=0x7f4dec06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:01.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.643+0000 7f4df0bff700 1 -- 172.21.15.60:0/1507156883 shutdown_connections 2023-07-13T22:28:01.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.643+0000 7f4df0bff700 1 -- 172.21.15.60:0/1507156883 wait complete. 2023-07-13T22:28:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.644+0000 7f4df0bff700 1 Processor -- start 2023-07-13T22:28:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.644+0000 7f4df0bff700 1 -- start start 2023-07-13T22:28:01.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.645+0000 7f4df0bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0b8400 0x7f4dec150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.645+0000 7f4df0bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dec0cfad0 0x7f4dec0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.645+0000 7f4df0bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dec0d3750 0x7f4dec15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.645+0000 7f4df0bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec15e580 con 0x7f4dec0cfad0 2023-07-13T22:28:01.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.645+0000 7f4df0bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0c7c40 con 0x7f4dec0d3750 2023-07-13T22:28:01.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.645+0000 7f4df0bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec15df30 con 0x7f4dec0b8400 2023-07-13T22:28:01.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.645+0000 7f4de9d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dec0cfad0 0x7f4dec0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4de9d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dec0cfad0 0x7f4dec0c7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40940/0 (socket says 172.21.15.60:40940) 2023-07-13T22:28:01.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4de9d9b700 1 -- 172.21.15.60:0/3482410546 learned_addr learned my addr 172.21.15.60:0/3482410546 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:01.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4dea59c700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0b8400 0x7f4dec150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4dead9d700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dec0d3750 0x7f4dec15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4de9d9b700 1 -- 172.21.15.60:0/3482410546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0b8400 msgr2=0x7f4dec150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4de9d9b700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0b8400 0x7f4dec150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4de9d9b700 1 -- 172.21.15.60:0/3482410546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dec0d3750 msgr2=0x7f4dec15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4de9d9b700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dec0d3750 0x7f4dec15abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4de9d9b700 1 -- 172.21.15.60:0/3482410546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de00094a0 con 0x7f4dec0cfad0 2023-07-13T22:28:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.646+0000 7f4de9d9b700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dec0cfad0 0x7f4dec0c7260 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4de00097c0 tx=0x7f4de0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.647+0000 7f4ddaffd700 1 -- 172.21.15.60:0/3482410546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de00044d0 con 0x7f4dec0cfad0 2023-07-13T22:28:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.647+0000 7f4ddaffd700 1 -- 172.21.15.60:0/3482410546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de001f070 con 0x7f4dec0cfad0 2023-07-13T22:28:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.647+0000 7f4ddaffd700 1 -- 172.21.15.60:0/3482410546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de00038e0 con 0x7f4dec0cfad0 2023-07-13T22:28:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.647+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec15e210 con 0x7f4dec0cfad0 2023-07-13T22:28:01.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.647+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec0c7f20 con 0x7f4dec0cfad0 2023-07-13T22:28:01.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.648+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec15e6c0 con 0x7f4dec0cfad0 2023-07-13T22:28:01.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.649+0000 7f4ddaffd700 1 -- 172.21.15.60:0/3482410546 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4de0003a40 con 0x7f4dec0cfad0 2023-07-13T22:28:01.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.650+0000 7f4ddaffd700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4dcc06ace0 0x7f4dcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:01.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.650+0000 7f4ddaffd700 1 -- 172.21.15.60:0/3482410546 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4de0021070 con 0x7f4dec0cfad0 2023-07-13T22:28:01.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.650+0000 7f4dea59c700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4dcc06ace0 0x7f4dcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:01.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.651+0000 7f4dea59c700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4dcc06ace0 0x7f4dcc06d190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0012b0 tx=0x7f4ddc007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.654+0000 7f4ddaffd700 1 -- 172.21.15.60:0/3482410546 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4de0055c70 con 0x7f4dec0cfad0 2023-07-13T22:28:01.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.866+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4dec15c540 con 0x7f4dec0cfad0 2023-07-13T22:28:01.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.866+0000 7f4ddaffd700 1 -- 172.21.15.60:0/3482410546 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4de0015090 con 0x7f4dec0cfad0 2023-07-13T22:28:01.847 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:01.847 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:28:01.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:01.847 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:28:01.848 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:28:01.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:01.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:01.848 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:28:01.848 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:28:01.848 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:01.848 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:28:01.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.868+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4dcc06ace0 msgr2=0x7f4dcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.868+0000 7f4df0bff700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4dcc06ace0 0x7f4dcc06d190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0012b0 tx=0x7f4ddc007c00 comp rx=0 tx=0).stop 2023-07-13T22:28:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.868+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dec0cfad0 msgr2=0x7f4dec0c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.868+0000 7f4df0bff700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dec0cfad0 0x7f4dec0c7260 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4de00097c0 tx=0x7f4de0004750 comp rx=0 tx=0).stop 2023-07-13T22:28:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.869+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 shutdown_connections 2023-07-13T22:28:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.869+0000 7f4df0bff700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4dcc06ace0 0x7f4dcc06d190 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.869+0000 7f4df0bff700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dec0d3750 0x7f4dec15abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.869+0000 7f4df0bff700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4dec0b8400 0x7f4dec150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.869+0000 7f4df0bff700 1 --2- 172.21.15.60:0/3482410546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4dec0cfad0 0x7f4dec0c7260 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.869+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 >> 172.21.15.60:0/3482410546 conn(0x7f4dec06b390 msgr2=0x7f4dec0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.869+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 shutdown_connections 2023-07-13T22:28:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:01.869+0000 7f4df0bff700 1 -- 172.21.15.60:0/3482410546 wait complete. 2023-07-13T22:28:01.867 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:28:01.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:28:02.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.095+0000 7f5ea15ff700 1 Processor -- start 2023-07-13T22:28:02.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.095+0000 7f5ea15ff700 1 -- start start 2023-07-13T22:28:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.096+0000 7f5ea15ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e9c159ea0 0x7f5e9c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.096+0000 7f5ea15ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c15c9b0 0x7f5e9c0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.096+0000 7f5ea15ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5e9c0d37c0 0x7f5e9c0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.096+0000 7f5ea15ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9c0d2e60 con 0x7f5e9c159ea0 2023-07-13T22:28:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.096+0000 7f5ea15ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9c0d2b50 con 0x7f5e9c0d37c0 2023-07-13T22:28:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.096+0000 7f5ea15ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c15d200 con 0x7f5e9c15c9b0 2023-07-13T22:28:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9affd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e9c159ea0 0x7f5e9c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9affd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e9c159ea0 0x7f5e9c0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40950/0 (socket says 172.21.15.60:40950) 2023-07-13T22:28:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9affd700 1 -- 172.21.15.60:0/1300832766 learned_addr learned my addr 172.21.15.60:0/1300832766 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9b7fe700 1 --2- 172.21.15.60:0/1300832766 >> v2:172.21.15.105:3300/0 conn(0x7f5e9c0d37c0 0x7f5e9c0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e93fff700 1 --2- 172.21.15.60:0/1300832766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c15c9b0 0x7f5e9c0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9affd700 1 -- 172.21.15.60:0/1300832766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c15c9b0 msgr2=0x7f5e9c0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9affd700 1 --2- 172.21.15.60:0/1300832766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c15c9b0 0x7f5e9c0d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9affd700 1 -- 172.21.15.60:0/1300832766 >> v2:172.21.15.105:3300/0 conn(0x7f5e9c0d37c0 msgr2=0x7f5e9c0d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9affd700 1 --2- 172.21.15.60:0/1300832766 >> v2:172.21.15.105:3300/0 conn(0x7f5e9c0d37c0 0x7f5e9c0d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9affd700 1 -- 172.21.15.60:0/1300832766 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c15d4e0 con 0x7f5e9c159ea0 2023-07-13T22:28:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.097+0000 7f5e9b7fe700 1 --2- 172.21.15.60:0/1300832766 >> v2:172.21.15.105:3300/0 conn(0x7f5e9c0d37c0 0x7f5e9c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.098+0000 7f5e9affd700 1 --2- 172.21.15.60:0/1300832766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 0x7f5e9c0cd6d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0cd2a0 tx=0x7f5e8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6eb908b2b1297c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.098+0000 7f5e9a7fc700 1 -- 172.21.15.60:0/1300832766 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c00b870 con 0x7f5e9c159ea0 2023-07-13T22:28:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.098+0000 7f5e9a7fc700 1 -- 172.21.15.60:0/1300832766 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8c00b9d0 con 0x7f5e9c159ea0 2023-07-13T22:28:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.098+0000 7f5e9a7fc700 1 -- 172.21.15.60:0/1300832766 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c00bd60 con 0x7f5e9c159ea0 2023-07-13T22:28:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.098+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1300832766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 msgr2=0x7f5e9c0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.098+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1300832766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 0x7f5e9c0cd6d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0cd2a0 tx=0x7f5e8c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:28:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.098+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1300832766 shutdown_connections 2023-07-13T22:28:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.099+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1300832766 >> v2:172.21.15.105:3300/0 conn(0x7f5e9c0d37c0 0x7f5e9c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.099+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1300832766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 0x7f5e9c0cd6d0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.099+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1300832766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c15c9b0 0x7f5e9c0d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:02.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.099+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1300832766 >> 172.21.15.60:0/1300832766 conn(0x7f5e9c071600 msgr2=0x7f5e9c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:02.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.099+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1300832766 shutdown_connections 2023-07-13T22:28:02.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.099+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1300832766 wait complete. 2023-07-13T22:28:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.100+0000 7f5ea15ff700 1 Processor -- start 2023-07-13T22:28:02.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.100+0000 7f5ea15ff700 1 -- start start 2023-07-13T22:28:02.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.101+0000 7f5ea15ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c0d37c0 0x7f5e9c1493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:02.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.101+0000 7f5ea15ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 0x7f5e9c0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:02.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.101+0000 7f5ea15ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e9c15c9b0 0x7f5e9c0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:02.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.101+0000 7f5ea15ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c159570 con 0x7f5e9c159ea0 2023-07-13T22:28:02.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.101+0000 7f5ea15ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c149920 con 0x7f5e9c15c9b0 2023-07-13T22:28:02.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.101+0000 7f5ea15ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c0c91a0 con 0x7f5e9c0d37c0 2023-07-13T22:28:02.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e93fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 0x7f5e9c0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:02.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e93fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 0x7f5e9c0c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40952/0 (socket says 172.21.15.60:40952) 2023-07-13T22:28:02.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e93fff700 1 -- 172.21.15.60:0/1019707638 learned_addr learned my addr 172.21.15.60:0/1019707638 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:02.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e9affd700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c0d37c0 0x7f5e9c1493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:02.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e9b7fe700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e9c15c9b0 0x7f5e9c0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:02.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e93fff700 1 -- 172.21.15.60:0/1019707638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c0d37c0 msgr2=0x7f5e9c1493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:02.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e93fff700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c0d37c0 0x7f5e9c1493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:02.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e93fff700 1 -- 172.21.15.60:0/1019707638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e9c15c9b0 msgr2=0x7f5e9c0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e93fff700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e9c15c9b0 0x7f5e9c0cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e93fff700 1 -- 172.21.15.60:0/1019707638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c009550 con 0x7f5e9c159ea0 2023-07-13T22:28:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.102+0000 7f5e93fff700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 0x7f5e9c0c8c60 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400d830 tx=0x7f5e8400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.103+0000 7f5e937fe700 1 -- 172.21.15.60:0/1019707638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e840125f0 con 0x7f5e9c159ea0 2023-07-13T22:28:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.103+0000 7f5e937fe700 1 -- 172.21.15.60:0/1019707638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e84012750 con 0x7f5e9c159ea0 2023-07-13T22:28:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.103+0000 7f5e937fe700 1 -- 172.21.15.60:0/1019707638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84012b00 con 0x7f5e9c159ea0 2023-07-13T22:28:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.103+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9c0c7bc0 con 0x7f5e9c159ea0 2023-07-13T22:28:02.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.103+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9c0c8180 con 0x7f5e9c159ea0 2023-07-13T22:28:02.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.104+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9c0c8ab0 con 0x7f5e9c159ea0 2023-07-13T22:28:02.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.105+0000 7f5e937fe700 1 -- 172.21.15.60:0/1019707638 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e84012c60 con 0x7f5e9c159ea0 2023-07-13T22:28:02.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.106+0000 7f5e937fe700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5e7c06ace0 0x7f5e7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:02.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.106+0000 7f5e937fe700 1 -- 172.21.15.60:0/1019707638 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5e840891b0 con 0x7f5e9c159ea0 2023-07-13T22:28:02.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.106+0000 7f5e9affd700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5e7c06ace0 0x7f5e7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:02.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.106+0000 7f5e9affd700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5e7c06ace0 0x7f5e7c06d190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000c00 tx=0x7f5e8c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:02.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.110+0000 7f5e937fe700 1 -- 172.21.15.60:0/1019707638 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5e84054510 con 0x7f5e9c159ea0 2023-07-13T22:28:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:02.318+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5e9c067a70 con 0x7f5e9c159ea0 2023-07-13T22:28:03.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.094+0000 7f5e937fe700 1 -- 172.21.15.60:0/1019707638 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5e84057b30 con 0x7f5e9c159ea0 2023-07-13T22:28:03.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.099+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5e7c06ace0 msgr2=0x7f5e7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:03.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.099+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5e7c06ace0 0x7f5e7c06d190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000c00 tx=0x7f5e8c008030 comp rx=0 tx=0).stop 2023-07-13T22:28:03.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.099+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 msgr2=0x7f5e9c0c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:03.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.099+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 0x7f5e9c0c8c60 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400d830 tx=0x7f5e8400db40 comp rx=0 tx=0).stop 2023-07-13T22:28:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.101+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 shutdown_connections 2023-07-13T22:28:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.101+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5e7c06ace0 0x7f5e7c06d190 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.101+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e9c15c9b0 0x7f5e9c0cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.101+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e9c0d37c0 0x7f5e9c1493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.101+0000 7f5ea15ff700 1 --2- 172.21.15.60:0/1019707638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9c159ea0 0x7f5e9c0c8c60 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.101+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 >> 172.21.15.60:0/1019707638 conn(0x7f5e9c071600 msgr2=0x7f5e9c15a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.101+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 shutdown_connections 2023-07-13T22:28:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.102+0000 7f5ea15ff700 1 -- 172.21.15.60:0/1019707638 wait complete. 2023-07-13T22:28:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:28:03.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:28:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.339+0000 7fef399ff700 1 Processor -- start 2023-07-13T22:28:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.339+0000 7fef399ff700 1 -- start start 2023-07-13T22:28:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.340+0000 7fef399ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fef340cd6d0 0x7fef340d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.340+0000 7fef399ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef340d37c0 0x7fef340d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.340+0000 7fef399ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef340cdb20 con 0x7fef34159ea0 2023-07-13T22:28:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.340+0000 7fef399ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef340d2e60 con 0x7fef340d37c0 2023-07-13T22:28:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.340+0000 7fef399ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef340d2b50 con 0x7fef340cd6d0 2023-07-13T22:28:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.340+0000 7fef337fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef340d37c0 0x7fef340d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.340+0000 7fef337fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef340d37c0 0x7fef340d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40954/0 (socket says 172.21.15.60:40954) 2023-07-13T22:28:03.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.340+0000 7fef337fe700 1 -- 172.21.15.60:0/3329550691 learned_addr learned my addr 172.21.15.60:0/3329550691 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:03.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.341+0000 7fef327fc700 1 --2- 172.21.15.60:0/3329550691 >> v2:172.21.15.105:3300/0 conn(0x7fef340cd6d0 0x7fef340d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:03.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.341+0000 7fef337fe700 1 -- 172.21.15.60:0/3329550691 >> v2:172.21.15.105:3300/0 conn(0x7fef340cd6d0 msgr2=0x7fef340d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:03.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.341+0000 7fef337fe700 1 --2- 172.21.15.60:0/3329550691 >> v2:172.21.15.105:3300/0 conn(0x7fef340cd6d0 0x7fef340d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:03.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.341+0000 7fef337fe700 1 -- 172.21.15.60:0/3329550691 >> v1:172.21.15.105:6789/0 conn(0x7fef34159ea0 legacy=0x7fef3415c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:03.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.341+0000 7fef337fe700 1 -- 172.21.15.60:0/3329550691 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3415d200 con 0x7fef340d37c0 2023-07-13T22:28:03.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.341+0000 7fef337fe700 1 --2- 172.21.15.60:0/3329550691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340d37c0 0x7fef340d0890 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fef340d2910 tx=0x7fef2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852c4e842afed40 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.341+0000 7fef31ffb700 1 -- 172.21.15.60:0/3329550691 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00e070 con 0x7fef340d37c0 2023-07-13T22:28:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.341+0000 7fef31ffb700 1 -- 172.21.15.60:0/3329550691 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c00b910 con 0x7fef340d37c0 2023-07-13T22:28:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.341+0000 7fef31ffb700 1 -- 172.21.15.60:0/3329550691 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00bcc0 con 0x7fef340d37c0 2023-07-13T22:28:03.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.342+0000 7fef399ff700 1 -- 172.21.15.60:0/3329550691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340d37c0 msgr2=0x7fef340d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:03.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.342+0000 7fef399ff700 1 --2- 172.21.15.60:0/3329550691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340d37c0 0x7fef340d0890 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fef340d2910 tx=0x7fef2c00b330 comp rx=0 tx=0).stop 2023-07-13T22:28:03.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.342+0000 7fef399ff700 1 -- 172.21.15.60:0/3329550691 shutdown_connections 2023-07-13T22:28:03.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.342+0000 7fef399ff700 1 --2- 172.21.15.60:0/3329550691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340d37c0 0x7fef340d0890 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:03.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.342+0000 7fef399ff700 1 --2- 172.21.15.60:0/3329550691 >> v2:172.21.15.105:3300/0 conn(0x7fef340cd6d0 0x7fef340d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:03.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.342+0000 7fef399ff700 1 -- 172.21.15.60:0/3329550691 >> 172.21.15.60:0/3329550691 conn(0x7fef34071600 msgr2=0x7fef340719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:03.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.342+0000 7fef399ff700 1 -- 172.21.15.60:0/3329550691 shutdown_connections 2023-07-13T22:28:03.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.342+0000 7fef399ff700 1 -- 172.21.15.60:0/3329550691 wait complete. 2023-07-13T22:28:03.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.344+0000 7fef399ff700 1 Processor -- start 2023-07-13T22:28:03.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.344+0000 7fef399ff700 1 -- start start 2023-07-13T22:28:03.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef399ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340cd6d0 0x7fef3414f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:03.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef399ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef340d37c0 0x7fef340c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:03.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef399ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef34159ea0 0x7fef340c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:03.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef399ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3414f5d0 con 0x7fef340cd6d0 2023-07-13T22:28:03.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef399ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340c9ab0 con 0x7fef340d37c0 2023-07-13T22:28:03.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef399ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef340c97d0 con 0x7fef34159ea0 2023-07-13T22:28:03.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef32ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340cd6d0 0x7fef3414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:03.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef32ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340cd6d0 0x7fef3414f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40970/0 (socket says 172.21.15.60:40970) 2023-07-13T22:28:03.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef32ffd700 1 -- 172.21.15.60:0/3590717703 learned_addr learned my addr 172.21.15.60:0/3590717703 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:03.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.345+0000 7fef327fc700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef340d37c0 0x7fef340c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:03.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef337fe700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef34159ea0 0x7fef340c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:03.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef32ffd700 1 -- 172.21.15.60:0/3590717703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef34159ea0 msgr2=0x7fef340c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:03.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef32ffd700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef34159ea0 0x7fef340c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:03.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef32ffd700 1 -- 172.21.15.60:0/3590717703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef340d37c0 msgr2=0x7fef340c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:03.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef32ffd700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef340d37c0 0x7fef340c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:03.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef32ffd700 1 -- 172.21.15.60:0/3590717703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c009b30 con 0x7fef340cd6d0 2023-07-13T22:28:03.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef337fe700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef34159ea0 0x7fef340c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:03.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef32ffd700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340cd6d0 0x7fef3414f090 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fef2400c450 tx=0x7fef2400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:03.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef1bfff700 1 -- 172.21.15.60:0/3590717703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef240052d0 con 0x7fef340cd6d0 2023-07-13T22:28:03.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef1bfff700 1 -- 172.21.15.60:0/3590717703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef24005430 con 0x7fef340cd6d0 2023-07-13T22:28:03.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef340b6850 con 0x7fef340cd6d0 2023-07-13T22:28:03.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef1bfff700 1 -- 172.21.15.60:0/3590717703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef240038c0 con 0x7fef340cd6d0 2023-07-13T22:28:03.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.346+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef340b6e10 con 0x7fef340cd6d0 2023-07-13T22:28:03.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.347+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef340b6fd0 con 0x7fef340cd6d0 2023-07-13T22:28:03.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.348+0000 7fef1bfff700 1 -- 172.21.15.60:0/3590717703 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef24003a20 con 0x7fef340cd6d0 2023-07-13T22:28:03.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.349+0000 7fef1bfff700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2006ace0 0x7fef2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:03.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.349+0000 7fef1bfff700 1 -- 172.21.15.60:0/3590717703 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fef24088640 con 0x7fef340cd6d0 2023-07-13T22:28:03.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.350+0000 7fef327fc700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2006ace0 0x7fef2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:03.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.350+0000 7fef327fc700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2006ace0 0x7fef2006d190 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef341f5930 tx=0x7fef28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:03.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.355+0000 7fef1bfff700 1 -- 172.21.15.60:0/3590717703 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fef24053830 con 0x7fef340cd6d0 2023-07-13T22:28:03.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:03.567+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fef34067a70 con 0x7fef340cd6d0 2023-07-13T22:28:04.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.101+0000 7fef1bfff700 1 -- 172.21.15.60:0/3590717703 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7fef24056e50 con 0x7fef340cd6d0 2023-07-13T22:28:04.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.107+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2006ace0 msgr2=0x7fef2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:04.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.107+0000 7fef399ff700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2006ace0 0x7fef2006d190 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef341f5930 tx=0x7fef28008040 comp rx=0 tx=0).stop 2023-07-13T22:28:04.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.107+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340cd6d0 msgr2=0x7fef3414f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:04.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.107+0000 7fef399ff700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340cd6d0 0x7fef3414f090 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fef2400c450 tx=0x7fef2400c810 comp rx=0 tx=0).stop 2023-07-13T22:28:04.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.113+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 shutdown_connections 2023-07-13T22:28:04.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.113+0000 7fef399ff700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef2006ace0 0x7fef2006d190 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.113+0000 7fef399ff700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef340cd6d0 0x7fef3414f090 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.113+0000 7fef399ff700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef34159ea0 0x7fef340c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.113+0000 7fef399ff700 1 --2- 172.21.15.60:0/3590717703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef340d37c0 0x7fef340c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.114+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 >> 172.21.15.60:0/3590717703 conn(0x7fef34071600 msgr2=0x7fef3415e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.116+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 shutdown_connections 2023-07-13T22:28:04.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.116+0000 7fef399ff700 1 -- 172.21.15.60:0/3590717703 wait complete. 2023-07-13T22:28:04.102 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:28:04.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:28:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.348+0000 7f8e9f7ff700 1 Processor -- start 2023-07-13T22:28:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.348+0000 7f8e9f7ff700 1 -- start start 2023-07-13T22:28:04.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.350+0000 7f8e9f7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e980b8200 0x7f8e980cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:04.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.350+0000 7f8e9f7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e980d1870 con 0x7f8e980b1ac0 2023-07-13T22:28:04.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.350+0000 7f8e9f7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e98160d80 con 0x7f8e980cede0 2023-07-13T22:28:04.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.350+0000 7f8e9f7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e980cf370 con 0x7f8e980b8200 2023-07-13T22:28:04.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.350+0000 7f8e9cd9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8e980b1ac0 0x7f8e9815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36302/0 (socket says 172.21.15.60:36302) 2023-07-13T22:28:04.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.350+0000 7f8e9cd9a700 1 -- 172.21.15.60:0/3092429705 learned_addr learned my addr 172.21.15.60:0/3092429705 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.350+0000 7f8e9d59b700 1 --2- 172.21.15.60:0/3092429705 >> v2:172.21.15.105:3300/0 conn(0x7f8e980b8200 0x7f8e980cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.350+0000 7f8e8ffff700 1 -- 172.21.15.60:0/3092429705 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343673315 0 0) 0x7f8e980d1870 con 0x7f8e980b1ac0 2023-07-13T22:28:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.350+0000 7f8e8ffff700 1 -- 172.21.15.60:0/3092429705 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e840035b0 con 0x7f8e980b1ac0 2023-07-13T22:28:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.351+0000 7f8e9d59b700 1 -- 172.21.15.60:0/3092429705 >> v1:172.21.15.105:6789/0 conn(0x7f8e980cede0 legacy=0x7f8e98160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.351+0000 7f8e9d59b700 1 -- 172.21.15.60:0/3092429705 >> v1:172.21.15.60:6789/0 conn(0x7f8e980b1ac0 legacy=0x7f8e9815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.351+0000 7f8e8ffff700 1 -- 172.21.15.60:0/3092429705 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296147638 0 0) 0x7f8e98160d80 con 0x7f8e980cede0 2023-07-13T22:28:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.351+0000 7f8e9d59b700 1 -- 172.21.15.60:0/3092429705 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e980d12b0 con 0x7f8e980b8200 2023-07-13T22:28:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.351+0000 7f8e9d59b700 1 --2- 172.21.15.60:0/3092429705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980b8200 0x7f8e980cf640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009840 tx=0x7f8e94009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2512cccf56752e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.351+0000 7f8e8ffff700 1 -- 172.21.15.60:0/3092429705 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94011830 con 0x7f8e980b8200 2023-07-13T22:28:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.351+0000 7f8e8ffff700 1 -- 172.21.15.60:0/3092429705 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e94011990 con 0x7f8e980b8200 2023-07-13T22:28:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.351+0000 7f8e8ffff700 1 -- 172.21.15.60:0/3092429705 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94011d20 con 0x7f8e980b8200 2023-07-13T22:28:04.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.352+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/3092429705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980b8200 msgr2=0x7f8e980cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:04.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.352+0000 7f8e9f7ff700 1 --2- 172.21.15.60:0/3092429705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980b8200 0x7f8e980cf640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009840 tx=0x7f8e94009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:04.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.352+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/3092429705 shutdown_connections 2023-07-13T22:28:04.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.352+0000 7f8e9f7ff700 1 --2- 172.21.15.60:0/3092429705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980b8200 0x7f8e980cf640 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:04.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.352+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/3092429705 >> 172.21.15.60:0/3092429705 conn(0x7f8e9806b2f0 msgr2=0x7f8e9806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:04.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.352+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/3092429705 shutdown_connections 2023-07-13T22:28:04.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.352+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/3092429705 wait complete. 2023-07-13T22:28:04.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.354+0000 7f8e9f7ff700 1 Processor -- start 2023-07-13T22:28:04.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.354+0000 7f8e9f7ff700 1 -- start start 2023-07-13T22:28:04.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9f7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980b1ac0 0x7f8e981555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:04.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9f7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e980b8200 0x7f8e980cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:04.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9f7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980cede0 0x7f8e980cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:04.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9f7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e98157520 con 0x7f8e980b1ac0 2023-07-13T22:28:04.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9f7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e98155af0 con 0x7f8e980cede0 2023-07-13T22:28:04.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9f7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e980c9fa0 con 0x7f8e980b8200 2023-07-13T22:28:04.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980b1ac0 0x7f8e981555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:04.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980b1ac0 0x7f8e981555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40986/0 (socket says 172.21.15.60:40986) 2023-07-13T22:28:04.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9d59b700 1 -- 172.21.15.60:0/1206975003 learned_addr learned my addr 172.21.15.60:0/1206975003 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:04.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9dd9c700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980cede0 0x7f8e980cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:04.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9cd9a700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e980b8200 0x7f8e980cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:04.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.355+0000 7f8e9d59b700 1 -- 172.21.15.60:0/1206975003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e980b8200 msgr2=0x7f8e980cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:04.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e9d59b700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e980b8200 0x7f8e980cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:04.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e9d59b700 1 -- 172.21.15.60:0/1206975003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980cede0 msgr2=0x7f8e980cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:04.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e9d59b700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980cede0 0x7f8e980cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:04.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e9d59b700 1 -- 172.21.15.60:0/1206975003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e940094a0 con 0x7f8e980b1ac0 2023-07-13T22:28:04.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e9cd9a700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e980b8200 0x7f8e980cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e9d59b700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980b1ac0 0x7f8e981555b0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009fd0 tx=0x7f8e94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1206975003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9401ab20 con 0x7f8e980b1ac0 2023-07-13T22:28:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1206975003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9401ac80 con 0x7f8e980b1ac0 2023-07-13T22:28:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1206975003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e940243f0 con 0x7f8e980b1ac0 2023-07-13T22:28:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e980c86a0 con 0x7f8e980b1ac0 2023-07-13T22:28:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.356+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e980c8920 con 0x7f8e980b1ac0 2023-07-13T22:28:04.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.357+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e98157660 con 0x7f8e980b1ac0 2023-07-13T22:28:04.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.358+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1206975003 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e94024550 con 0x7f8e980b1ac0 2023-07-13T22:28:04.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.359+0000 7f8e8dffb700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:04.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.359+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1206975003 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f8e9408b060 con 0x7f8e980b1ac0 2023-07-13T22:28:04.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.360+0000 7f8e9cd9a700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:04.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.360+0000 7f8e9cd9a700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8e980cb5f0 tx=0x7f8e88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:04.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.363+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1206975003 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8e940561e0 con 0x7f8e980b1ac0 2023-07-13T22:28:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:04.616+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8e98157660 con 0x7f8e980b1ac0 2023-07-13T22:28:05.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.105+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1206975003 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v48) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8e940320c0 con 0x7f8e980b1ac0 2023-07-13T22:28:05.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.110+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 msgr2=0x7f8e7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.110+0000 7f8e9f7ff700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8e980cb5f0 tx=0x7f8e88009040 comp rx=0 tx=0).stop 2023-07-13T22:28:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.110+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980b1ac0 msgr2=0x7f8e981555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.110+0000 7f8e9f7ff700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980b1ac0 0x7f8e981555b0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009fd0 tx=0x7f8e94004750 comp rx=0 tx=0).stop 2023-07-13T22:28:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.112+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 shutdown_connections 2023-07-13T22:28:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.112+0000 7f8e9f7ff700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.112+0000 7f8e9f7ff700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980b1ac0 0x7f8e981555b0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.112+0000 7f8e9f7ff700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e980cede0 0x7f8e980cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.112+0000 7f8e9f7ff700 1 --2- 172.21.15.60:0/1206975003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e980b8200 0x7f8e980cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.112+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 >> 172.21.15.60:0/1206975003 conn(0x7f8e9806b2f0 msgr2=0x7f8e9815be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:05.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.113+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 shutdown_connections 2023-07-13T22:28:05.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.113+0000 7f8e9f7ff700 1 -- 172.21.15.60:0/1206975003 wait complete. 2023-07-13T22:28:05.101 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 13 and data pool 14 2023-07-13T22:28:05.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:05.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.357+0000 7fad215ff700 1 Processor -- start 2023-07-13T22:28:05.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.357+0000 7fad215ff700 1 -- start start 2023-07-13T22:28:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.358+0000 7fad215ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0b8400 0x7fad1c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.358+0000 7fad215ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fad1c0cfca0 0x7fad1c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.358+0000 7fad215ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c0d0ac0 con 0x7fad1c0d3750 2023-07-13T22:28:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.358+0000 7fad215ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c0b87d0 con 0x7fad1c0cfca0 2023-07-13T22:28:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.358+0000 7fad215ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0b4eb0 con 0x7fad1c0b8400 2023-07-13T22:28:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.358+0000 7fad1b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fad1c0cfca0 0x7fad1c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.358+0000 7fad1a7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fad1c0d3750 0x7fad1c0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53702/0 (socket says 172.21.15.60:53702) 2023-07-13T22:28:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.358+0000 7fad1a7fc700 1 -- 172.21.15.60:0/2308103283 learned_addr learned my addr 172.21.15.60:0/2308103283 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.358+0000 7fad1affd700 1 --2- 172.21.15.60:0/2308103283 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0b8400 0x7fad1c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.359+0000 7fad1b7fe700 1 -- 172.21.15.60:0/2308103283 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0b8400 msgr2=0x7fad1c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.359+0000 7fad1b7fe700 1 --2- 172.21.15.60:0/2308103283 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0b8400 0x7fad1c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.359+0000 7fad1b7fe700 1 -- 172.21.15.60:0/2308103283 >> v1:172.21.15.60:6789/0 conn(0x7fad1c0d3750 legacy=0x7fad1c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.359+0000 7fad1b7fe700 1 -- 172.21.15.60:0/2308103283 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c0b5190 con 0x7fad1c0cfca0 2023-07-13T22:28:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.359+0000 7fad19ffb700 1 -- 172.21.15.60:0/2308103283 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009740373 0 0) 0x7fad1c0d0ac0 con 0x7fad1c0d3750 2023-07-13T22:28:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.359+0000 7fad1b7fe700 1 --2- 172.21.15.60:0/2308103283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0cfca0 0x7fad1c0d0f50 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fad14009d90 tx=0x7fad1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d944f8691928906 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.359+0000 7fad19ffb700 1 -- 172.21.15.60:0/2308103283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1400e070 con 0x7fad1c0cfca0 2023-07-13T22:28:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.359+0000 7fad19ffb700 1 -- 172.21.15.60:0/2308103283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1400bb20 con 0x7fad1c0cfca0 2023-07-13T22:28:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.359+0000 7fad19ffb700 1 -- 172.21.15.60:0/2308103283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14012430 con 0x7fad1c0cfca0 2023-07-13T22:28:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.360+0000 7fad215ff700 1 -- 172.21.15.60:0/2308103283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0cfca0 msgr2=0x7fad1c0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.360+0000 7fad215ff700 1 --2- 172.21.15.60:0/2308103283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0cfca0 0x7fad1c0d0f50 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fad14009d90 tx=0x7fad1400b330 comp rx=0 tx=0).stop 2023-07-13T22:28:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.360+0000 7fad215ff700 1 -- 172.21.15.60:0/2308103283 shutdown_connections 2023-07-13T22:28:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.360+0000 7fad215ff700 1 --2- 172.21.15.60:0/2308103283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0cfca0 0x7fad1c0d0f50 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.360+0000 7fad215ff700 1 --2- 172.21.15.60:0/2308103283 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0b8400 0x7fad1c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.360+0000 7fad215ff700 1 -- 172.21.15.60:0/2308103283 >> 172.21.15.60:0/2308103283 conn(0x7fad1c06b390 msgr2=0x7fad1c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.360+0000 7fad215ff700 1 -- 172.21.15.60:0/2308103283 shutdown_connections 2023-07-13T22:28:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.360+0000 7fad215ff700 1 -- 172.21.15.60:0/2308103283 wait complete. 2023-07-13T22:28:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.362+0000 7fad215ff700 1 Processor -- start 2023-07-13T22:28:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.362+0000 7fad215ff700 1 -- start start 2023-07-13T22:28:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad215ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0b8400 0x7fad1c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad215ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad1c0cfca0 0x7fad1c15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad215ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0d3750 0x7fad1c14e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad215ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0c66b0 con 0x7fad1c0b8400 2023-07-13T22:28:05.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad215ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0cbbb0 con 0x7fad1c0cfca0 2023-07-13T22:28:05.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad215ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0c6900 con 0x7fad1c0d3750 2023-07-13T22:28:05.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0b8400 0x7fad1c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0b8400 0x7fad1c0b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60862/0 (socket says 172.21.15.60:60862) 2023-07-13T22:28:05.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1affd700 1 -- 172.21.15.60:0/1461005569 learned_addr learned my addr 172.21.15.60:0/1461005569 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:05.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1b7fe700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0d3750 0x7fad1c14e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1a7fc700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad1c0cfca0 0x7fad1c15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1affd700 1 -- 172.21.15.60:0/1461005569 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0d3750 msgr2=0x7fad1c14e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1affd700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0d3750 0x7fad1c14e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1affd700 1 -- 172.21.15.60:0/1461005569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad1c0cfca0 msgr2=0x7fad1c15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1affd700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad1c0cfca0 0x7fad1c15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1affd700 1 -- 172.21.15.60:0/1461005569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14009a70 con 0x7fad1c0b8400 2023-07-13T22:28:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.363+0000 7fad1b7fe700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0d3750 0x7fad1c14e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.364+0000 7fad1affd700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0b8400 0x7fad1c0b6820 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00e870 tx=0x7fad0c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.364+0000 7fad037fe700 1 -- 172.21.15.60:0/1461005569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c009590 con 0x7fad1c0b8400 2023-07-13T22:28:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.364+0000 7fad037fe700 1 -- 172.21.15.60:0/1461005569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c00ee70 con 0x7fad1c0b8400 2023-07-13T22:28:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.364+0000 7fad037fe700 1 -- 172.21.15.60:0/1461005569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00ca50 con 0x7fad1c0b8400 2023-07-13T22:28:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.364+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c0c6be0 con 0x7fad1c0b8400 2023-07-13T22:28:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.364+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c0b6d60 con 0x7fad1c0b8400 2023-07-13T22:28:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.365+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c0c7230 con 0x7fad1c0b8400 2023-07-13T22:28:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.366+0000 7fad037fe700 1 -- 172.21.15.60:0/1461005569 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad0c00cbb0 con 0x7fad1c0b8400 2023-07-13T22:28:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.367+0000 7fad037fe700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad0806ace0 0x7fad0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.367+0000 7fad037fe700 1 -- 172.21.15.60:0/1461005569 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad0c089460 con 0x7fad1c0b8400 2023-07-13T22:28:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.367+0000 7fad1a7fc700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad0806ace0 0x7fad0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.368+0000 7fad1a7fc700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad0806ace0 0x7fad0806d190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad1c15ca90 tx=0x7fad10007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:05.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.372+0000 7fad037fe700 1 -- 172.21.15.60:0/1461005569 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fad0c057f90 con 0x7fad1c0b8400 2023-07-13T22:28:05.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.576+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad1c0cee40 con 0x7fad1c0b8400 2023-07-13T22:28:05.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.578+0000 7fad037fe700 1 -- 172.21.15.60:0/1461005569 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18213 (secure 0 0 0) 0x7fad0c00ce60 con 0x7fad1c0b8400 2023-07-13T22:28:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:05.559 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:04.617924+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:02.360339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:03.609373+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:05.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad0806ace0 msgr2=0x7fad0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad0806ace0 0x7fad0806d190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad1c15ca90 tx=0x7fad10007be0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0b8400 msgr2=0x7fad1c0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0b8400 0x7fad1c0b6820 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00e870 tx=0x7fad0c00ec30 comp rx=0 tx=0).stop 2023-07-13T22:28:05.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 shutdown_connections 2023-07-13T22:28:05.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad0806ace0 0x7fad0806d190 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad1c0b8400 0x7fad1c0b6820 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad1c0d3750 0x7fad1c14e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 --2- 172.21.15.60:0/1461005569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad1c0cfca0 0x7fad1c15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 >> 172.21.15.60:0/1461005569 conn(0x7fad1c06b390 msgr2=0x7fad1c0b7810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:05.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 shutdown_connections 2023-07-13T22:28:05.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.580+0000 7fad215ff700 1 -- 172.21.15.60:0/1461005569 wait complete. 2023-07-13T22:28:05.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:05.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.801+0000 7fd4db5ff700 1 Processor -- start 2023-07-13T22:28:05.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.801+0000 7fd4db5ff700 1 -- start start 2023-07-13T22:28:05.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.802+0000 7fd4db5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4159170 0x7fd4d40c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.802+0000 7fd4db5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4d4070980 0x7fd4d40c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.802+0000 7fd4db5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d415d970 con 0x7fd4d40c5040 2023-07-13T22:28:05.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.802+0000 7fd4db5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d415cd60 con 0x7fd4d4070980 2023-07-13T22:28:05.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.802+0000 7fd4db5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415d040 con 0x7fd4d4159170 2023-07-13T22:28:05.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.803+0000 7fd4d9b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4d4070980 0x7fd4d40c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.803+0000 7fd4d9b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4d4070980 0x7fd4d40c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60864/0 (socket says 172.21.15.60:60864) 2023-07-13T22:28:05.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.803+0000 7fd4d9b9c700 1 -- 172.21.15.60:0/3882428166 learned_addr learned my addr 172.21.15.60:0/3882428166 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:05.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.803+0000 7fd4d939b700 1 --2- 172.21.15.60:0/3882428166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4159170 0x7fd4d40c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.803+0000 7fd4d9b9c700 1 -- 172.21.15.60:0/3882428166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4159170 msgr2=0x7fd4d40c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:05.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.803+0000 7fd4d9b9c700 1 --2- 172.21.15.60:0/3882428166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4159170 0x7fd4d40c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.803+0000 7fd4d9b9c700 1 -- 172.21.15.60:0/3882428166 >> v1:172.21.15.105:6789/0 conn(0x7fd4d40c5040 legacy=0x7fd4d4070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.803+0000 7fd4d9b9c700 1 -- 172.21.15.60:0/3882428166 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d415d320 con 0x7fd4d4070980 2023-07-13T22:28:05.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.803+0000 7fd4d9b9c700 1 --2- 172.21.15.60:0/3882428166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d4070980 0x7fd4d40c9c50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009e50 tx=0x7fd4cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65037b327b0e23f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:05.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.804+0000 7fd4cbfff700 1 -- 172.21.15.60:0/3882428166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00e070 con 0x7fd4d4070980 2023-07-13T22:28:05.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.804+0000 7fd4cbfff700 1 -- 172.21.15.60:0/3882428166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc00b910 con 0x7fd4d4070980 2023-07-13T22:28:05.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.804+0000 7fd4cbfff700 1 -- 172.21.15.60:0/3882428166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00bcc0 con 0x7fd4d4070980 2023-07-13T22:28:05.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.804+0000 7fd4db5ff700 1 -- 172.21.15.60:0/3882428166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d4070980 msgr2=0x7fd4d40c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.804+0000 7fd4db5ff700 1 --2- 172.21.15.60:0/3882428166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d4070980 0x7fd4d40c9c50 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009e50 tx=0x7fd4cc00b330 comp rx=0 tx=0).stop 2023-07-13T22:28:05.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.804+0000 7fd4db5ff700 1 -- 172.21.15.60:0/3882428166 shutdown_connections 2023-07-13T22:28:05.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.804+0000 7fd4db5ff700 1 --2- 172.21.15.60:0/3882428166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d4070980 0x7fd4d40c9c50 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.804+0000 7fd4db5ff700 1 --2- 172.21.15.60:0/3882428166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4159170 0x7fd4d40c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.804+0000 7fd4db5ff700 1 -- 172.21.15.60:0/3882428166 >> 172.21.15.60:0/3882428166 conn(0x7fd4d40cf410 msgr2=0x7fd4d40cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:05.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.805+0000 7fd4db5ff700 1 -- 172.21.15.60:0/3882428166 shutdown_connections 2023-07-13T22:28:05.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.805+0000 7fd4db5ff700 1 -- 172.21.15.60:0/3882428166 wait complete. 2023-07-13T22:28:05.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.806+0000 7fd4db5ff700 1 Processor -- start 2023-07-13T22:28:05.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.806+0000 7fd4db5ff700 1 -- start start 2023-07-13T22:28:05.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.807+0000 7fd4db5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4070980 0x7fd4d415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.807+0000 7fd4db5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d40c5040 0x7fd4d40b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.807+0000 7fd4db5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4d4159170 0x7fd4d414abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.807+0000 7fd4db5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40b1c10 con 0x7fd4d40c5040 2023-07-13T22:28:05.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.807+0000 7fd4db5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40b4610 con 0x7fd4d4159170 2023-07-13T22:28:05.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.807+0000 7fd4db5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d414b130 con 0x7fd4d4070980 2023-07-13T22:28:05.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d8b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d40c5040 0x7fd4d40b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d939b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4070980 0x7fd4d415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d8b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d40c5040 0x7fd4d40b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60880/0 (socket says 172.21.15.60:60880) 2023-07-13T22:28:05.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d9b9c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4d4159170 0x7fd4d414abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d8b9a700 1 -- 172.21.15.60:0/4252586053 learned_addr learned my addr 172.21.15.60:0/4252586053 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:05.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d8b9a700 1 -- 172.21.15.60:0/4252586053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4070980 msgr2=0x7fd4d415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d8b9a700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4070980 0x7fd4d415b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d8b9a700 1 -- 172.21.15.60:0/4252586053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4d4159170 msgr2=0x7fd4d414abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:05.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d8b9a700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4d4159170 0x7fd4d414abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:05.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d8b9a700 1 -- 172.21.15.60:0/4252586053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc009b30 con 0x7fd4d40c5040 2023-07-13T22:28:05.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.808+0000 7fd4d8b9a700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d40c5040 0x7fd4d40b16d0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c510 tx=0x7fd4c400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:05.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.809+0000 7fd4c9ffb700 1 -- 172.21.15.60:0/4252586053 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40052d0 con 0x7fd4d40c5040 2023-07-13T22:28:05.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.809+0000 7fd4c9ffb700 1 -- 172.21.15.60:0/4252586053 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4005430 con 0x7fd4d40c5040 2023-07-13T22:28:05.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.809+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d415bf50 con 0x7fd4d40c5040 2023-07-13T22:28:05.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.809+0000 7fd4c9ffb700 1 -- 172.21.15.60:0/4252586053 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40038c0 con 0x7fd4d40c5040 2023-07-13T22:28:05.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.809+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d40b3fc0 con 0x7fd4d40c5040 2023-07-13T22:28:05.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.810+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d41587f0 con 0x7fd4d40c5040 2023-07-13T22:28:05.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.811+0000 7fd4c9ffb700 1 -- 172.21.15.60:0/4252586053 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd4c4003a20 con 0x7fd4d40c5040 2023-07-13T22:28:05.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.812+0000 7fd4c9ffb700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4b806ace0 0x7fd4b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:05.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.812+0000 7fd4c9ffb700 1 -- 172.21.15.60:0/4252586053 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4c4088840 con 0x7fd4d40c5040 2023-07-13T22:28:05.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.813+0000 7fd4d939b700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4b806ace0 0x7fd4b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:05.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.813+0000 7fd4d939b700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4b806ace0 0x7fd4b806d190 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009b40 tx=0x7fd4d0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:05.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:05.817+0000 7fd4c9ffb700 1 -- 172.21.15.60:0/4252586053 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd4c40538e0 con 0x7fd4d40c5040 2023-07-13T22:28:06.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.023+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4d40ad860 con 0x7fd4d40c5040 2023-07-13T22:28:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.025+0000 7fd4c9ffb700 1 -- 172.21.15.60:0/4252586053 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18213 (secure 0 0 0) 0x7fd4c4056f00 con 0x7fd4d40c5040 2023-07-13T22:28:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:06.006 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:04.617924+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:02.360339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:03.609373+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:06.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4b806ace0 msgr2=0x7fd4b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4b806ace0 0x7fd4b806d190 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009b40 tx=0x7fd4d0009210 comp rx=0 tx=0).stop 2023-07-13T22:28:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d40c5040 msgr2=0x7fd4d40b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d40c5040 0x7fd4d40b16d0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c510 tx=0x7fd4c400c820 comp rx=0 tx=0).stop 2023-07-13T22:28:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 shutdown_connections 2023-07-13T22:28:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4b806ace0 0x7fd4b806d190 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4d4159170 0x7fd4d414abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:06.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4d4070980 0x7fd4d415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:06.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 --2- 172.21.15.60:0/4252586053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4d40c5040 0x7fd4d40b16d0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:06.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 >> 172.21.15.60:0/4252586053 conn(0x7fd4d40cf410 msgr2=0x7fd4d4070270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:06.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 shutdown_connections 2023-07-13T22:28:06.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.027+0000 7fd4db5ff700 1 -- 172.21.15.60:0/4252586053 wait complete. 2023-07-13T22:28:06.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:28:06.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.256+0000 7f4eb75ff700 1 Processor -- start 2023-07-13T22:28:06.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.256+0000 7f4eb75ff700 1 -- start start 2023-07-13T22:28:06.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.257+0000 7f4eb75ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4eb00b8200 0x7f4eb00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:06.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.257+0000 7f4eb75ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4eb00cede0 0x7f4eb00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:06.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb75ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb00d1870 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb75ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb00cf370 con 0x7f4eb00cede0 2023-07-13T22:28:06.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb75ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb00d12b0 con 0x7f4eb00b8200 2023-07-13T22:28:06.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb4b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4eb00b1ac0 0x7f4eb015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53704/0 (socket says 172.21.15.60:53704) 2023-07-13T22:28:06.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb4b9a700 1 -- 172.21.15.60:0/1330733625 learned_addr learned my addr 172.21.15.60:0/1330733625 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:06.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb5b9c700 1 --2- 172.21.15.60:0/1330733625 >> v2:172.21.15.60:3300/0 conn(0x7f4eb00cede0 0x7f4eb00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:06.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb539b700 1 --2- 172.21.15.60:0/1330733625 >> v2:172.21.15.105:3300/0 conn(0x7f4eb00b8200 0x7f4eb00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:06.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb5b9c700 1 -- 172.21.15.60:0/1330733625 >> v2:172.21.15.105:3300/0 conn(0x7f4eb00b8200 msgr2=0x7f4eb00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:06.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb5b9c700 1 --2- 172.21.15.60:0/1330733625 >> v2:172.21.15.105:3300/0 conn(0x7f4eb00b8200 0x7f4eb00cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:06.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb5b9c700 1 -- 172.21.15.60:0/1330733625 >> v1:172.21.15.60:6789/0 conn(0x7f4eb00b1ac0 legacy=0x7f4eb015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:06.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.258+0000 7f4eb5b9c700 1 -- 172.21.15.60:0/1330733625 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb00d1590 con 0x7f4eb00cede0 2023-07-13T22:28:06.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.259+0000 7f4eb5b9c700 1 --2- 172.21.15.60:0/1330733625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00cede0 0x7f4eb00cfb80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00b2660 tx=0x7f4ea800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbef2cd13715fd9e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:06.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.259+0000 7f4ea7fff700 1 -- 172.21.15.60:0/1330733625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea800e070 con 0x7f4eb00cede0 2023-07-13T22:28:06.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.259+0000 7f4ea7fff700 1 -- 172.21.15.60:0/1330733625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea800bbd0 con 0x7f4eb00cede0 2023-07-13T22:28:06.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.259+0000 7f4ea7fff700 1 -- 172.21.15.60:0/1330733625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8012430 con 0x7f4eb00cede0 2023-07-13T22:28:06.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.259+0000 7f4eb75ff700 1 -- 172.21.15.60:0/1330733625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00cede0 msgr2=0x7f4eb00cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:06.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.259+0000 7f4eb75ff700 1 --2- 172.21.15.60:0/1330733625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00cede0 0x7f4eb00cfb80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00b2660 tx=0x7f4ea800b330 comp rx=0 tx=0).stop 2023-07-13T22:28:06.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.260+0000 7f4eb75ff700 1 -- 172.21.15.60:0/1330733625 shutdown_connections 2023-07-13T22:28:06.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.260+0000 7f4eb75ff700 1 --2- 172.21.15.60:0/1330733625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00cede0 0x7f4eb00cfb80 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:06.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.260+0000 7f4eb75ff700 1 --2- 172.21.15.60:0/1330733625 >> v2:172.21.15.105:3300/0 conn(0x7f4eb00b8200 0x7f4eb00cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:06.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.260+0000 7f4eb75ff700 1 -- 172.21.15.60:0/1330733625 >> 172.21.15.60:0/1330733625 conn(0x7f4eb006b2f0 msgr2=0x7f4eb006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:06.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.260+0000 7f4eb75ff700 1 -- 172.21.15.60:0/1330733625 shutdown_connections 2023-07-13T22:28:06.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.260+0000 7f4eb75ff700 1 -- 172.21.15.60:0/1330733625 wait complete. 2023-07-13T22:28:06.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.261+0000 7f4eb75ff700 1 Processor -- start 2023-07-13T22:28:06.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.261+0000 7f4eb75ff700 1 -- start start 2023-07-13T22:28:06.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.262+0000 7f4eb75ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00b1ac0 0x7f4eb0155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:06.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.262+0000 7f4eb75ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4eb00b8200 0x7f4eb0159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:06.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.262+0000 7f4eb75ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb00cede0 0x7f4eb00cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:06.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.262+0000 7f4eb75ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb01567e0 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.262+0000 7f4eb75ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb00cc3b0 con 0x7f4eb00cede0 2023-07-13T22:28:06.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.262+0000 7f4eb75ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb00cb7a0 con 0x7f4eb00b8200 2023-07-13T22:28:06.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb539b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00b1ac0 0x7f4eb0155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:06.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb539b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00b1ac0 0x7f4eb0155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60892/0 (socket says 172.21.15.60:60892) 2023-07-13T22:28:06.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb539b700 1 -- 172.21.15.60:0/4214820801 learned_addr learned my addr 172.21.15.60:0/4214820801 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:06.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb539b700 1 -- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4eb00b8200 msgr2=0x7f4eb0159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:06.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb4b9a700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4eb00b8200 0x7f4eb0159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:06.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb5b9c700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb00cede0 0x7f4eb00cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:06.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb539b700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4eb00b8200 0x7f4eb0159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:06.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb539b700 1 -- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb00cede0 msgr2=0x7f4eb00cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:06.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb539b700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb00cede0 0x7f4eb00cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:06.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb539b700 1 -- 172.21.15.60:0/4214820801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea8009a70 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb4b9a700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4eb00b8200 0x7f4eb0159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:06.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb5b9c700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb00cede0 0x7f4eb00cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:06.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.263+0000 7f4eb539b700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00b1ac0 0x7f4eb0155b90 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00e840 tx=0x7f4eac00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:06.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.264+0000 7f4ea5ffb700 1 -- 172.21.15.60:0/4214820801 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac013610 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.264+0000 7f4ea5ffb700 1 -- 172.21.15.60:0/4214820801 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac013770 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.264+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb00cba80 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.264+0000 7f4ea5ffb700 1 -- 172.21.15.60:0/4214820801 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac013b20 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.264+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb00cc040 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.265+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb004e210 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.266+0000 7f4ea5ffb700 1 -- 172.21.15.60:0/4214820801 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4eac013c80 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.267+0000 7f4ea5ffb700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e9406ace0 0x7f4e9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:06.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.267+0000 7f4ea5ffb700 1 -- 172.21.15.60:0/4214820801 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4eac0891e0 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.268+0000 7f4eb4b9a700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e9406ace0 0x7f4e9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:06.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.268+0000 7f4eb4b9a700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e9406ace0 0x7f4e9406d190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0159490 tx=0x7f4ea0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:06.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.272+0000 7f4ea5ffb700 1 -- 172.21.15.60:0/4214820801 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4eac054200 con 0x7f4eb00b1ac0 2023-07-13T22:28:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:06.522+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4eb00b4a70 con 0x7f4eb00b1ac0 2023-07-13T22:28:07.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.127+0000 7f4ea5ffb700 1 -- 172.21.15.60:0/4214820801 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4eac057820 con 0x7f4eb00b1ac0 2023-07-13T22:28:07.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e9406ace0 msgr2=0x7f4e9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:07.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e9406ace0 0x7f4e9406d190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0159490 tx=0x7f4ea0009040 comp rx=0 tx=0).stop 2023-07-13T22:28:07.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00b1ac0 msgr2=0x7f4eb0155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:07.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00b1ac0 0x7f4eb0155b90 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00e840 tx=0x7f4eac00ec00 comp rx=0 tx=0).stop 2023-07-13T22:28:07.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 shutdown_connections 2023-07-13T22:28:07.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e9406ace0 0x7f4e9406d190 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:07.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb00b1ac0 0x7f4eb0155b90 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:07.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4eb00cede0 0x7f4eb00cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:07.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 --2- 172.21.15.60:0/4214820801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4eb00b8200 0x7f4eb0159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:07.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 >> 172.21.15.60:0/4214820801 conn(0x7f4eb006b2f0 msgr2=0x7f4eb00cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:07.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 shutdown_connections 2023-07-13T22:28:07.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.130+0000 7f4eb75ff700 1 -- 172.21.15.60:0/4214820801 wait complete. 2023-07-13T22:28:07.130 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:28:07.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:28:07.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.358+0000 7f7becdff700 1 Processor -- start 2023-07-13T22:28:07.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.358+0000 7f7becdff700 1 -- start start 2023-07-13T22:28:07.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.359+0000 7f7becdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7be80b8200 0x7f7be80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:07.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.359+0000 7f7becdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80cede0 0x7f7be80cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:07.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.359+0000 7f7becdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be80d1870 con 0x7f7be80b1ac0 2023-07-13T22:28:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.359+0000 7f7becdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be80cf370 con 0x7f7be80b8200 2023-07-13T22:28:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.359+0000 7f7becdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be80d12b0 con 0x7f7be80cede0 2023-07-13T22:28:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be659c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7be80b8200 0x7f7be80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:07.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be6d9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80cede0 0x7f7be80cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:07.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be6d9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80cede0 0x7f7be80cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:55932/0 (socket says 172.21.15.60:55932) 2023-07-13T22:28:07.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be6d9d700 1 -- 172.21.15.60:0/160758715 learned_addr learned my addr 172.21.15.60:0/160758715 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:07.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be659c700 1 -- 172.21.15.60:0/160758715 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80cede0 msgr2=0x7f7be80cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:07.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be659c700 1 --2- 172.21.15.60:0/160758715 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80cede0 0x7f7be80cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:07.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be659c700 1 -- 172.21.15.60:0/160758715 >> v1:172.21.15.105:6789/0 conn(0x7f7be80b1ac0 legacy=0x7f7be815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:07.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be659c700 1 -- 172.21.15.60:0/160758715 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be80d1590 con 0x7f7be80b8200 2023-07-13T22:28:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be5d9b700 1 -- 172.21.15.60:0/160758715 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713523204 0 0) 0x7f7be80d1870 con 0x7f7be80b1ac0 2023-07-13T22:28:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be6d9d700 1 --2- 172.21.15.60:0/160758715 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80cede0 0x7f7be80cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:28:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.360+0000 7f7be659c700 1 --2- 172.21.15.60:0/160758715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80b8200 0x7f7be80cf640 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009840 tx=0x7f7bd8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84de81f6c4d50896 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.361+0000 7f7be5d9b700 1 -- 172.21.15.60:0/160758715 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8011830 con 0x7f7be80b8200 2023-07-13T22:28:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.361+0000 7f7be5d9b700 1 -- 172.21.15.60:0/160758715 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd8011990 con 0x7f7be80b8200 2023-07-13T22:28:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.361+0000 7f7be5d9b700 1 -- 172.21.15.60:0/160758715 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8011d20 con 0x7f7be80b8200 2023-07-13T22:28:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.361+0000 7f7becdff700 1 -- 172.21.15.60:0/160758715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80b8200 msgr2=0x7f7be80cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.361+0000 7f7becdff700 1 --2- 172.21.15.60:0/160758715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80b8200 0x7f7be80cf640 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009840 tx=0x7f7bd8009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.361+0000 7f7becdff700 1 -- 172.21.15.60:0/160758715 shutdown_connections 2023-07-13T22:28:07.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.361+0000 7f7becdff700 1 --2- 172.21.15.60:0/160758715 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80cede0 0x7f7be80cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:07.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.361+0000 7f7becdff700 1 --2- 172.21.15.60:0/160758715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80b8200 0x7f7be80cf640 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:07.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.361+0000 7f7becdff700 1 -- 172.21.15.60:0/160758715 >> 172.21.15.60:0/160758715 conn(0x7f7be806b2f0 msgr2=0x7f7be806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:07.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.362+0000 7f7becdff700 1 -- 172.21.15.60:0/160758715 shutdown_connections 2023-07-13T22:28:07.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.362+0000 7f7becdff700 1 -- 172.21.15.60:0/160758715 wait complete. 2023-07-13T22:28:07.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.363+0000 7f7becdff700 1 Processor -- start 2023-07-13T22:28:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.363+0000 7f7becdff700 1 -- start start 2023-07-13T22:28:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.364+0000 7f7becdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7be80b1ac0 0x7f7be8156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.364+0000 7f7becdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80b8200 0x7f7be81e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.364+0000 7f7becdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80cede0 0x7f7be80ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.364+0000 7f7becdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be80b2ee0 con 0x7f7be80b1ac0 2023-07-13T22:28:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.364+0000 7f7becdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be80cb310 con 0x7f7be80cede0 2023-07-13T22:28:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.364+0000 7f7becdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be80cb5f0 con 0x7f7be80b8200 2023-07-13T22:28:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7be80b1ac0 0x7f7be8156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7be80b1ac0 0x7f7be8156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60894/0 (socket says 172.21.15.60:60894) 2023-07-13T22:28:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be659c700 1 -- 172.21.15.60:0/4053652848 learned_addr learned my addr 172.21.15.60:0/4053652848 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7bdffff700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80b8200 0x7f7be81e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be6d9d700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80cede0 0x7f7be80ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be659c700 1 -- 172.21.15.60:0/4053652848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80b8200 msgr2=0x7f7be81e63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be659c700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80b8200 0x7f7be81e63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be659c700 1 -- 172.21.15.60:0/4053652848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80cede0 msgr2=0x7f7be80ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be659c700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80cede0 0x7f7be80ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be659c700 1 -- 172.21.15.60:0/4053652848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be0009b30 con 0x7f7be80b1ac0 2023-07-13T22:28:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be6d9d700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80cede0 0x7f7be80ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.365+0000 7f7be659c700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7be80b1ac0 0x7f7be8156b40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009fd0 tx=0x7f7bd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:07.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.366+0000 7f7bdeffd700 1 -- 172.21.15.60:0/4053652848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80038a0 con 0x7f7be80b1ac0 2023-07-13T22:28:07.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.366+0000 7f7bdeffd700 1 -- 172.21.15.60:0/4053652848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd801f070 con 0x7f7be80b1ac0 2023-07-13T22:28:07.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.366+0000 7f7bdeffd700 1 -- 172.21.15.60:0/4053652848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8003c40 con 0x7f7be80b1ac0 2023-07-13T22:28:07.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.366+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd80094a0 con 0x7f7be80b1ac0 2023-07-13T22:28:07.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.366+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be80cbbb0 con 0x7f7be80b1ac0 2023-07-13T22:28:07.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.367+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be804e210 con 0x7f7be80b1ac0 2023-07-13T22:28:07.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.368+0000 7f7bdeffd700 1 -- 172.21.15.60:0/4053652848 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7bd8026480 con 0x7f7be80b1ac0 2023-07-13T22:28:07.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.369+0000 7f7bdeffd700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7bc806ace0 0x7f7bc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:07.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.369+0000 7f7bdeffd700 1 -- 172.21.15.60:0/4053652848 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7bd8021070 con 0x7f7be80b1ac0 2023-07-13T22:28:07.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.369+0000 7f7bdffff700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7bc806ace0 0x7f7bc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:07.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.370+0000 7f7bdffff700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7bc806ace0 0x7f7bc806d190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7be80b1770 tx=0x7f7bd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:07.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.374+0000 7f7bdeffd700 1 -- 172.21.15.60:0/4053652848 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7bd8055f00 con 0x7f7be80b1ac0 2023-07-13T22:28:07.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:07.630+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7be80d37f0 con 0x7f7be80b1ac0 2023-07-13T22:28:08.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.137+0000 7f7bdeffd700 1 -- 172.21.15.60:0/4053652848 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7bd80150c0 con 0x7f7be80b1ac0 2023-07-13T22:28:08.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7bc806ace0 msgr2=0x7f7bc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7bc806ace0 0x7f7bc806d190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7be80b1770 tx=0x7f7bd0009040 comp rx=0 tx=0).stop 2023-07-13T22:28:08.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7be80b1ac0 msgr2=0x7f7be8156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7be80b1ac0 0x7f7be8156b40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009fd0 tx=0x7f7bd8004750 comp rx=0 tx=0).stop 2023-07-13T22:28:08.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 shutdown_connections 2023-07-13T22:28:08.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7bc806ace0 0x7f7bc806d190 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7be80b1ac0 0x7f7be8156b40 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7be80cede0 0x7f7be80ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 --2- 172.21.15.60:0/4053652848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7be80b8200 0x7f7be81e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 >> 172.21.15.60:0/4053652848 conn(0x7f7be806b2f0 msgr2=0x7f7be80b74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:08.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 shutdown_connections 2023-07-13T22:28:08.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.140+0000 7f7becdff700 1 -- 172.21.15.60:0/4053652848 wait complete. 2023-07-13T22:28:08.141 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:28:08.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:28:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.367+0000 7fa859bff700 1 Processor -- start 2023-07-13T22:28:08.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.367+0000 7fa859bff700 1 -- start start 2023-07-13T22:28:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.368+0000 7fa859bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 0x7fa85415c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.368+0000 7fa859bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85406edd0 con 0x7fa854070c50 2023-07-13T22:28:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.368+0000 7fa859bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85406b290 con 0x7fa8540ce7d0 2023-07-13T22:28:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.368+0000 7fa859bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540cec70 con 0x7fa85406e990 2023-07-13T22:28:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa8537fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa854070c50 0x7fa854071020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53706/0 (socket says 172.21.15.60:53706) 2023-07-13T22:28:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa8537fe700 1 -- 172.21.15.60:0/471996703 learned_addr learned my addr 172.21.15.60:0/471996703 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:08.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa853fff700 1 --2- 172.21.15.60:0/471996703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 0x7fa85415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa8527fc700 1 -- 172.21.15.60:0/471996703 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121821343 0 0) 0x7fa85406edd0 con 0x7fa854070c50 2023-07-13T22:28:08.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa8527fc700 1 -- 172.21.15.60:0/471996703 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa83c0035b0 con 0x7fa854070c50 2023-07-13T22:28:08.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa8527fc700 1 -- 172.21.15.60:0/471996703 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802589426 0 0) 0x7fa85406b290 con 0x7fa8540ce7d0 2023-07-13T22:28:08.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa8527fc700 1 -- 172.21.15.60:0/471996703 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa85406edd0 con 0x7fa8540ce7d0 2023-07-13T22:28:08.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa8527fc700 1 -- 172.21.15.60:0/471996703 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2781137654 0 0) 0x7fa83c0035b0 con 0x7fa854070c50 2023-07-13T22:28:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa853fff700 1 -- 172.21.15.60:0/471996703 >> v1:172.21.15.105:6789/0 conn(0x7fa8540ce7d0 legacy=0x7fa85415ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa853fff700 1 -- 172.21.15.60:0/471996703 >> v1:172.21.15.60:6789/0 conn(0x7fa854070c50 legacy=0x7fa854071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.369+0000 7fa853fff700 1 -- 172.21.15.60:0/471996703 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540cd330 con 0x7fa85406e990 2023-07-13T22:28:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.370+0000 7fa853fff700 1 --2- 172.21.15.60:0/471996703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 0x7fa85415c700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009dc0 tx=0x7fa84c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=664803546a86e516 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.370+0000 7fa8527fc700 1 -- 172.21.15.60:0/471996703 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00e070 con 0x7fa85406e990 2023-07-13T22:28:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.370+0000 7fa8527fc700 1 -- 172.21.15.60:0/471996703 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c003a20 con 0x7fa85406e990 2023-07-13T22:28:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.370+0000 7fa8527fc700 1 -- 172.21.15.60:0/471996703 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844001670 con 0x7fa85406e990 2023-07-13T22:28:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.371+0000 7fa859bff700 1 -- 172.21.15.60:0/471996703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 msgr2=0x7fa85415c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.371+0000 7fa859bff700 1 --2- 172.21.15.60:0/471996703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 0x7fa85415c700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009dc0 tx=0x7fa84c00b330 comp rx=0 tx=0).stop 2023-07-13T22:28:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.371+0000 7fa859bff700 1 -- 172.21.15.60:0/471996703 shutdown_connections 2023-07-13T22:28:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.371+0000 7fa859bff700 1 --2- 172.21.15.60:0/471996703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 0x7fa85415c700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.371+0000 7fa859bff700 1 -- 172.21.15.60:0/471996703 >> 172.21.15.60:0/471996703 conn(0x7fa8540b1820 msgr2=0x7fa8540cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.371+0000 7fa859bff700 1 -- 172.21.15.60:0/471996703 shutdown_connections 2023-07-13T22:28:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.371+0000 7fa859bff700 1 -- 172.21.15.60:0/471996703 wait complete. 2023-07-13T22:28:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.372+0000 7fa859bff700 1 Processor -- start 2023-07-13T22:28:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.373+0000 7fa859bff700 1 -- start start 2023-07-13T22:28:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.373+0000 7fa859bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 0x7fa85414c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa859bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa854070c50 0x7fa8541e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa859bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8540ce7d0 0x7fa8541e6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa859bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540abca0 con 0x7fa854070c50 2023-07-13T22:28:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa859bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8541e7660 con 0x7fa8540ce7d0 2023-07-13T22:28:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa859bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85414c810 con 0x7fa85406e990 2023-07-13T22:28:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa852ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa854070c50 0x7fa8541e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa852ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa854070c50 0x7fa8541e5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60906/0 (socket says 172.21.15.60:60906) 2023-07-13T22:28:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa852ffd700 1 -- 172.21.15.60:0/2816803790 learned_addr learned my addr 172.21.15.60:0/2816803790 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa8537fe700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 0x7fa85414c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa853fff700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8540ce7d0 0x7fa8541e6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa852ffd700 1 -- 172.21.15.60:0/2816803790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 msgr2=0x7fa85414c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa852ffd700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 0x7fa85414c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa852ffd700 1 -- 172.21.15.60:0/2816803790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8540ce7d0 msgr2=0x7fa8541e6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa852ffd700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8540ce7d0 0x7fa8541e6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.374+0000 7fa852ffd700 1 -- 172.21.15.60:0/2816803790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c009a70 con 0x7fa854070c50 2023-07-13T22:28:08.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.375+0000 7fa852ffd700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa854070c50 0x7fa8541e5e40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c520 tx=0x7fa84800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:08.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.375+0000 7fa83bfff700 1 -- 172.21.15.60:0/2816803790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480052d0 con 0x7fa854070c50 2023-07-13T22:28:08.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.375+0000 7fa83bfff700 1 -- 172.21.15.60:0/2816803790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa848005430 con 0x7fa854070c50 2023-07-13T22:28:08.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.375+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8541e6a50 con 0x7fa854070c50 2023-07-13T22:28:08.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.375+0000 7fa83bfff700 1 -- 172.21.15.60:0/2816803790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480038c0 con 0x7fa854070c50 2023-07-13T22:28:08.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.375+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8541e7010 con 0x7fa854070c50 2023-07-13T22:28:08.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.376+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8540714e0 con 0x7fa854070c50 2023-07-13T22:28:08.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.377+0000 7fa83bfff700 1 -- 172.21.15.60:0/2816803790 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa848003a20 con 0x7fa854070c50 2023-07-13T22:28:08.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.378+0000 7fa83bfff700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006ace0 0x7fa84006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.378+0000 7fa83bfff700 1 -- 172.21.15.60:0/2816803790 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa848088ad0 con 0x7fa854070c50 2023-07-13T22:28:08.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.378+0000 7fa8537fe700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006ace0 0x7fa84006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.379+0000 7fa8537fe700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006ace0 0x7fa84006d190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa844005990 tx=0x7fa844001b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:08.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.382+0000 7fa83bfff700 1 -- 172.21.15.60:0/2816803790 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa848053ae0 con 0x7fa854070c50 2023-07-13T22:28:08.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.632+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8540cf9c0 con 0x7fa854070c50 2023-07-13T22:28:08.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.632+0000 7fa83bfff700 1 -- 172.21.15.60:0/2816803790 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fa848057100 con 0x7fa854070c50 2023-07-13T22:28:08.613 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:08.613 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2395854295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2395854295},{"type":"v1","addr":"172.21.15.105:6834","nonce":2395854295}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1048261776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1048261776},{"type":"v1","addr":"172.21.15.105:6835","nonce":1048261776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/1639899719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1639899719},{"type":"v1","addr":"172.21.15.60:6836","nonce":1639899719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:04.618240+0000","modified":"2023-07-13T22:28:08.127584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7434":{"gid":7434,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/1512512599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":1512512599},{"type":"v1","addr":"172.21.15.60:6837","nonce":1512512599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-13T22:28:08.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006ace0 msgr2=0x7fa84006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006ace0 0x7fa84006d190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa844005990 tx=0x7fa844001b80 comp rx=0 tx=0).stop 2023-07-13T22:28:08.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa854070c50 msgr2=0x7fa8541e5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa854070c50 0x7fa8541e5e40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c520 tx=0x7fa84800c8e0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 shutdown_connections 2023-07-13T22:28:08.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa84006ace0 0x7fa84006d190 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8540ce7d0 0x7fa8541e6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa85406e990 0x7fa85414c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 --2- 172.21.15.60:0/2816803790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa854070c50 0x7fa8541e5e40 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 >> 172.21.15.60:0/2816803790 conn(0x7fa8540b1820 msgr2=0x7fa8540b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:08.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 shutdown_connections 2023-07-13T22:28:08.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.635+0000 7fa859bff700 1 -- 172.21.15.60:0/2816803790 wait complete. 2023-07-13T22:28:08.617 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 52 2023-07-13T22:28:08.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:08.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.855+0000 7f3da95ff700 1 Processor -- start 2023-07-13T22:28:08.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.855+0000 7f3da95ff700 1 -- start start 2023-07-13T22:28:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.856+0000 7f3da95ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3da40d35c0 0x7f3da40d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.856+0000 7f3da95ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 0x7f3da415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.856+0000 7f3da95ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da40d09d0 con 0x7f3da40b8570 2023-07-13T22:28:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.856+0000 7f3da95ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da40d3b30 con 0x7f3da40d35c0 2023-07-13T22:28:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.856+0000 7f3da95ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40d1870 con 0x7f3da40d13f0 2023-07-13T22:28:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da2ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3da40b8570 0x7f3da40ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53718/0 (socket says 172.21.15.60:53718) 2023-07-13T22:28:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da2ffd700 1 -- 172.21.15.60:0/3191711902 learned_addr learned my addr 172.21.15.60:0/3191711902 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:08.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da27fc700 1 --2- 172.21.15.60:0/3191711902 >> v2:172.21.15.105:3300/0 conn(0x7f3da40d35c0 0x7f3da40d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da37fe700 1 --2- 172.21.15.60:0/3191711902 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 0x7f3da415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da1ffb700 1 -- 172.21.15.60:0/3191711902 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469618556 0 0) 0x7f3da40d09d0 con 0x7f3da40b8570 2023-07-13T22:28:08.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da1ffb700 1 -- 172.21.15.60:0/3191711902 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d8c0035b0 con 0x7f3da40b8570 2023-07-13T22:28:08.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da27fc700 1 -- 172.21.15.60:0/3191711902 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 msgr2=0x7f3da415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:08.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da27fc700 1 --2- 172.21.15.60:0/3191711902 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 0x7f3da415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da27fc700 1 -- 172.21.15.60:0/3191711902 >> v1:172.21.15.60:6789/0 conn(0x7f3da40b8570 legacy=0x7f3da40ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da27fc700 1 -- 172.21.15.60:0/3191711902 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da40cda10 con 0x7f3da40d35c0 2023-07-13T22:28:08.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.857+0000 7f3da37fe700 1 --2- 172.21.15.60:0/3191711902 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 0x7f3da415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:28:08.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.858+0000 7f3da27fc700 1 --2- 172.21.15.60:0/3191711902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3da40d35c0 0x7f3da40d0eb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009840 tx=0x7f3d98009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f79a505389b93f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:08.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.858+0000 7f3da1ffb700 1 -- 172.21.15.60:0/3191711902 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98011830 con 0x7f3da40d35c0 2023-07-13T22:28:08.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.858+0000 7f3da1ffb700 1 -- 172.21.15.60:0/3191711902 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d98011990 con 0x7f3da40d35c0 2023-07-13T22:28:08.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.858+0000 7f3da1ffb700 1 -- 172.21.15.60:0/3191711902 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98011d20 con 0x7f3da40d35c0 2023-07-13T22:28:08.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.858+0000 7f3da95ff700 1 -- 172.21.15.60:0/3191711902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3da40d35c0 msgr2=0x7f3da40d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.858+0000 7f3da95ff700 1 --2- 172.21.15.60:0/3191711902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3da40d35c0 0x7f3da40d0eb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009840 tx=0x7f3d98009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:08.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.859+0000 7f3da95ff700 1 -- 172.21.15.60:0/3191711902 shutdown_connections 2023-07-13T22:28:08.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.859+0000 7f3da95ff700 1 --2- 172.21.15.60:0/3191711902 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 0x7f3da415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.859+0000 7f3da95ff700 1 --2- 172.21.15.60:0/3191711902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3da40d35c0 0x7f3da40d0eb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.859+0000 7f3da95ff700 1 -- 172.21.15.60:0/3191711902 >> 172.21.15.60:0/3191711902 conn(0x7f3da4071600 msgr2=0x7f3da40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:08.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.859+0000 7f3da95ff700 1 -- 172.21.15.60:0/3191711902 shutdown_connections 2023-07-13T22:28:08.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.859+0000 7f3da95ff700 1 -- 172.21.15.60:0/3191711902 wait complete. 2023-07-13T22:28:08.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.861+0000 7f3da95ff700 1 Processor -- start 2023-07-13T22:28:08.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.861+0000 7f3da95ff700 1 -- start start 2023-07-13T22:28:08.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.862+0000 7f3da95ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3da40b8570 0x7f3da40c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.862+0000 7f3da95ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 0x7f3da4155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.862+0000 7f3da95ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3da40d35c0 0x7f3da40cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.862+0000 7f3da95ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4158f70 con 0x7f3da40d35c0 2023-07-13T22:28:08.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.862+0000 7f3da95ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4159180 con 0x7f3da40b8570 2023-07-13T22:28:08.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da95ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4159460 con 0x7f3da40d13f0 2023-07-13T22:28:08.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3da40d35c0 0x7f3da40cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3da40d35c0 0x7f3da40cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60920/0 (socket says 172.21.15.60:60920) 2023-07-13T22:28:08.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da37fe700 1 -- 172.21.15.60:0/3287368800 learned_addr learned my addr 172.21.15.60:0/3287368800 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da2ffd700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3da40b8570 0x7f3da40c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da27fc700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 0x7f3da4155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da37fe700 1 -- 172.21.15.60:0/3287368800 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 msgr2=0x7f3da4155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da37fe700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 0x7f3da4155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da37fe700 1 -- 172.21.15.60:0/3287368800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3da40b8570 msgr2=0x7f3da40c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da37fe700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3da40b8570 0x7f3da40c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.863+0000 7f3da37fe700 1 -- 172.21.15.60:0/3287368800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c009a70 con 0x7f3da40d35c0 2023-07-13T22:28:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.864+0000 7f3da37fe700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3da40d35c0 0x7f3da40cb250 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00efc0 tx=0x7f3d9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.864+0000 7f3d8b7fe700 1 -- 172.21.15.60:0/3287368800 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00ca90 con 0x7f3da40d35c0 2023-07-13T22:28:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.864+0000 7f3d8b7fe700 1 -- 172.21.15.60:0/3287368800 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c00f040 con 0x7f3da40d35c0 2023-07-13T22:28:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.864+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d980094a0 con 0x7f3da40d35c0 2023-07-13T22:28:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.864+0000 7f3d8b7fe700 1 -- 172.21.15.60:0/3287368800 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00ce10 con 0x7f3da40d35c0 2023-07-13T22:28:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.864+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da4159a20 con 0x7f3da40d35c0 2023-07-13T22:28:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.865+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da404e280 con 0x7f3da40d35c0 2023-07-13T22:28:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.866+0000 7f3d8b7fe700 1 -- 172.21.15.60:0/3287368800 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d9c014480 con 0x7f3da40d35c0 2023-07-13T22:28:08.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.867+0000 7f3d8b7fe700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3d9006ace0 0x7f3d9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:08.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.867+0000 7f3d8b7fe700 1 -- 172.21.15.60:0/3287368800 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d9c089bd0 con 0x7f3da40d35c0 2023-07-13T22:28:08.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.867+0000 7f3da2ffd700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3d9006ace0 0x7f3d9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:08.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.868+0000 7f3da2ffd700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3d9006ace0 0x7f3d9006d190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3d940010e0 tx=0x7f3d94007c70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:08.872+0000 7f3d8b7fe700 1 -- 172.21.15.60:0/3287368800 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3d9c058670 con 0x7f3da40d35c0 2023-07-13T22:28:09.060 INFO:tasks.ceph.mds.a.smithi060.stderr:2023-07-13T22:28:09.080+0000 7f27987f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:28:09.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.082+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3da415f1a0 con 0x7f3da40d35c0 2023-07-13T22:28:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.083+0000 7f3d8b7fe700 1 -- 172.21.15.60:0/3287368800 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18200 (secure 0 0 0) 0x7f3d9c009d70 con 0x7f3da40d35c0 2023-07-13T22:28:09.064 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:09.064 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:06.113196+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:02.360339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:03.609373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3d9006ace0 msgr2=0x7f3d9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3d9006ace0 0x7f3d9006d190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3d940010e0 tx=0x7f3d94007c70 comp rx=0 tx=0).stop 2023-07-13T22:28:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3da40d35c0 msgr2=0x7f3da40cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3da40d35c0 0x7f3da40cb250 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00efc0 tx=0x7f3d9c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 shutdown_connections 2023-07-13T22:28:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3d9006ace0 0x7f3d9006d190 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3da40d35c0 0x7f3da40cb250 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3da40b8570 0x7f3da40c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 --2- 172.21.15.60:0/3287368800 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3da40d13f0 0x7f3da4155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 >> 172.21.15.60:0/3287368800 conn(0x7f3da4071600 msgr2=0x7f3da4160570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 shutdown_connections 2023-07-13T22:28:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.086+0000 7f3da95ff700 1 -- 172.21.15.60:0/3287368800 wait complete. 2023-07-13T22:28:09.084 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:28:09.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:28:09.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.307+0000 7f5427dff700 1 Processor -- start 2023-07-13T22:28:09.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.307+0000 7f5427dff700 1 -- start start 2023-07-13T22:28:09.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.308+0000 7f5427dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f542015c6d0 0x7f54200b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.308+0000 7f5427dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54200d18a0 con 0x7f542015c300 2023-07-13T22:28:09.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.308+0000 7f5427dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54200cf500 con 0x7f54200b46c0 2023-07-13T22:28:09.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.308+0000 7f5427dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200d15c0 con 0x7f542015c6d0 2023-07-13T22:28:09.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.309+0000 7f5425b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f542015c300 0x7f54200cedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53730/0 (socket says 172.21.15.60:53730) 2023-07-13T22:28:09.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.309+0000 7f5425b9b700 1 -- 172.21.15.60:0/2083905471 learned_addr learned my addr 172.21.15.60:0/2083905471 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.309+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452148686 0 0) 0x7f54200d18a0 con 0x7f542015c300 2023-07-13T22:28:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.309+0000 7f542539a700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f542015c6d0 0x7f54200b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.309+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f540c0035b0 con 0x7f542015c300 2023-07-13T22:28:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.309+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3912783269 0 0) 0x7f540c0035b0 con 0x7f542015c300 2023-07-13T22:28:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.309+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f542015c6d0 msgr2=0x7f54200b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.309+0000 7f5424b99700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f542015c6d0 0x7f54200b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.309+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 >> v1:172.21.15.105:6789/0 conn(0x7f54200b46c0 legacy=0x7f542006b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200cd3c0 con 0x7f542015c300 2023-07-13T22:28:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f541c001670 con 0x7f542015c300 2023-07-13T22:28:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 >> v1:172.21.15.60:6789/0 conn(0x7f542015c300 legacy=0x7f54200cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5424b99700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f542015c6d0 0x7f540c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5424b99700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c003f70 0x7f540c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5424b99700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c004d90 0x7f540c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200d15c0 con 0x7f540c004d90 2023-07-13T22:28:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c00bb50 con 0x7f540c003f70 2023-07-13T22:28:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c00bd10 con 0x7f542015c6d0 2023-07-13T22:28:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f5425b9b700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f542015c6d0 0x7f540c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f542639c700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c004d90 0x7f540c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f542539a700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c003f70 0x7f540c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f542639c700 1 -- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f542015c6d0 msgr2=0x7f540c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f542639c700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f542015c6d0 0x7f540c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f542639c700 1 -- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c003f70 msgr2=0x7f540c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f542639c700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c003f70 0x7f540c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.310+0000 7f542639c700 1 -- 172.21.15.60:0/2083905471 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0035b0 con 0x7f540c004d90 2023-07-13T22:28:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.311+0000 7f542639c700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c004d90 0x7f540c005200 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f541800c260 tx=0x7f540c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb1df3baea69d82 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.311+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418001e70 con 0x7f540c004d90 2023-07-13T22:28:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.311+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541800ce20 con 0x7f540c004d90 2023-07-13T22:28:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.311+0000 7f5424b99700 1 -- 172.21.15.60:0/2083905471 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418012660 con 0x7f540c004d90 2023-07-13T22:28:09.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.311+0000 7f5427dff700 1 -- 172.21.15.60:0/2083905471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c004d90 msgr2=0x7f540c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.311+0000 7f5427dff700 1 --2- 172.21.15.60:0/2083905471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f540c004d90 0x7f540c005200 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f541800c260 tx=0x7f540c005880 comp rx=0 tx=0).stop 2023-07-13T22:28:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.311+0000 7f542639c700 1 -- 172.21.15.60:0/2083905471 reap_dead start 2023-07-13T22:28:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.312+0000 7f5427dff700 1 -- 172.21.15.60:0/2083905471 shutdown_connections 2023-07-13T22:28:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.312+0000 7f5427dff700 1 -- 172.21.15.60:0/2083905471 >> 172.21.15.60:0/2083905471 conn(0x7f54200cf6c0 msgr2=0x7f54200cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.312+0000 7f5427dff700 1 -- 172.21.15.60:0/2083905471 shutdown_connections 2023-07-13T22:28:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.312+0000 7f5427dff700 1 -- 172.21.15.60:0/2083905471 wait complete. 2023-07-13T22:28:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.313+0000 7f5427dff700 1 Processor -- start 2023-07-13T22:28:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.314+0000 7f5427dff700 1 -- start start 2023-07-13T22:28:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.314+0000 7f5427dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c004d90 0x7f542014c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.314+0000 7f5427dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54201574c0 0x7f542015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.314+0000 7f5427dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54200cc440 0x7f542006b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.314+0000 7f5427dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200c7940 con 0x7f54200cc440 2023-07-13T22:28:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.314+0000 7f5427dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200c7c00 con 0x7f540c004d90 2023-07-13T22:28:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f5427dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200c7ee0 con 0x7f54201574c0 2023-07-13T22:28:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542639c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54200cc440 0x7f542006b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542639c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54200cc440 0x7f542006b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60950/0 (socket says 172.21.15.60:60950) 2023-07-13T22:28:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542639c700 1 -- 172.21.15.60:0/1084195002 learned_addr learned my addr 172.21.15.60:0/1084195002 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f5425b9b700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c004d90 0x7f542014c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542539a700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54201574c0 0x7f542015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542639c700 1 -- 172.21.15.60:0/1084195002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54201574c0 msgr2=0x7f542015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542639c700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54201574c0 0x7f542015cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542639c700 1 -- 172.21.15.60:0/1084195002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c004d90 msgr2=0x7f542014c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542639c700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c004d90 0x7f542014c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542639c700 1 -- 172.21.15.60:0/1084195002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200c81c0 con 0x7f54200cc440 2023-07-13T22:28:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f5425b9b700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c004d90 0x7f542014c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.315+0000 7f542639c700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54200cc440 0x7f542006b4a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5418013000 tx=0x7f54180015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.316+0000 7f54167fc700 1 -- 172.21.15.60:0/1084195002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180050c0 con 0x7f54200cc440 2023-07-13T22:28:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.316+0000 7f54167fc700 1 -- 172.21.15.60:0/1084195002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418005220 con 0x7f54200cc440 2023-07-13T22:28:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.316+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54200c84a0 con 0x7f54200cc440 2023-07-13T22:28:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.316+0000 7f54167fc700 1 -- 172.21.15.60:0/1084195002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180055b0 con 0x7f54200cc440 2023-07-13T22:28:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.316+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542015d250 con 0x7f54200cc440 2023-07-13T22:28:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.317+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54200cde60 con 0x7f54200cc440 2023-07-13T22:28:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.318+0000 7f54167fc700 1 -- 172.21.15.60:0/1084195002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5418023070 con 0x7f54200cc440 2023-07-13T22:28:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.319+0000 7f54167fc700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f540406ad10 0x7f540406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.319+0000 7f54167fc700 1 -- 172.21.15.60:0/1084195002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f541808b060 con 0x7f54200cc440 2023-07-13T22:28:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.319+0000 7f5425b9b700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f540406ad10 0x7f540406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.320+0000 7f5425b9b700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f540406ad10 0x7f540406d1c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f541c007c30 tx=0x7f541c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:09.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.323+0000 7f54167fc700 1 -- 172.21.15.60:0/1084195002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5418056220 con 0x7f54200cc440 2023-07-13T22:28:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.526+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54200b55b0 con 0x7f540406ad10 2023-07-13T22:28:09.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.536+0000 7f54167fc700 1 -- 172.21.15.60:0/1084195002 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311409 (secure 0 0 0) 0x7f541c0015b0 con 0x7f540406ad10 2023-07-13T22:28:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f540406ad10 msgr2=0x7f540406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f540406ad10 0x7f540406d1c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f541c007c30 tx=0x7f541c009040 comp rx=0 tx=0).stop 2023-07-13T22:28:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54200cc440 msgr2=0x7f542006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54200cc440 0x7f542006b4a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5418013000 tx=0x7f54180015b0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 shutdown_connections 2023-07-13T22:28:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f540406ad10 0x7f540406d1c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54200cc440 0x7f542006b4a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f540c004d90 0x7f542014c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 --2- 172.21.15.60:0/1084195002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54201574c0 0x7f542015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 >> 172.21.15.60:0/1084195002 conn(0x7f54200cf6c0 msgr2=0x7f54200b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 shutdown_connections 2023-07-13T22:28:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.541+0000 7f5427dff700 1 -- 172.21.15.60:0/1084195002 wait complete. 2023-07-13T22:28:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:28:09.539 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2023-07-13T22:28:09.221784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278336,"kb_used_data":9472,"kb_used_omap":0,"kb_used_meta":268800,"kb_avail":754696384,"statfs":{"total":773094113280,"available":772809097216,"internally_reserved":0,"allocated":9699328,"data_stored":5005968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":275251200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001135"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.121062+0000","last_change":"2023-07-13T22:28:05.134097+0000","last_active":"2023-07-13T22:28:06.121062+0000","last_peered":"2023-07-13T22:28:06.121062+0000","last_clean":"2023-07-13T22:28:06.121062+0000","last_became_active":"2023-07-13T22:28:05.133812+0000","last_became_peered":"2023-07-13T22:28:05.133812+0000","last_unstale":"2023-07-13T22:28:06.121062+0000","last_undegraded":"2023-07-13T22:28:06.121062+0000","last_fullsized":"2023-07-13T22:28:06.121062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:34:47.472127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.135771+0000","last_change":"2023-07-13T22:28:05.134664+0000","last_active":"2023-07-13T22:28:05.135771+0000","last_peered":"2023-07-13T22:28:05.135771+0000","last_clean":"2023-07-13T22:28:05.135771+0000","last_became_active":"2023-07-13T22:28:05.134434+0000","last_became_peered":"2023-07-13T22:28:05.134434+0000","last_unstale":"2023-07-13T22:28:05.135771+0000","last_undegraded":"2023-07-13T22:28:05.135771+0000","last_fullsized":"2023-07-13T22:28:05.135771+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:03:51.828510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120849+0000","last_change":"2023-07-13T22:28:05.130472+0000","last_active":"2023-07-13T22:28:06.120849+0000","last_peered":"2023-07-13T22:28:06.120849+0000","last_clean":"2023-07-13T22:28:06.120849+0000","last_became_active":"2023-07-13T22:28:05.130175+0000","last_became_peered":"2023-07-13T22:28:05.130175+0000","last_unstale":"2023-07-13T22:28:06.120849+0000","last_undegraded":"2023-07-13T22:28:06.120849+0000","last_fullsized":"2023-07-13T22:28:06.120849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:49:21.735034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148352+0000","last_change":"2023-07-13T22:28:05.140817+0000","last_active":"2023-07-13T22:28:05.148352+0000","last_peered":"2023-07-13T22:28:05.148352+0000","last_clean":"2023-07-13T22:28:05.148352+0000","last_became_active":"2023-07-13T22:28:05.140026+0000","last_became_peered":"2023-07-13T22:28:05.140026+0000","last_unstale":"2023-07-13T22:28:05.148352+0000","last_undegraded":"2023-07-13T22:28:05.148352+0000","last_fullsized":"2023-07-13T22:28:05.148352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:16:57.958444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124213+0000","last_change":"2023-07-13T22:28:05.125775+0000","last_active":"2023-07-13T22:28:06.124213+0000","last_peered":"2023-07-13T22:28:06.124213+0000","last_clean":"2023-07-13T22:28:06.124213+0000","last_became_active":"2023-07-13T22:28:05.122355+0000","last_became_peered":"2023-07-13T22:28:05.122355+0000","last_unstale":"2023-07-13T22:28:06.124213+0000","last_undegraded":"2023-07-13T22:28:06.124213+0000","last_fullsized":"2023-07-13T22:28:06.124213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:34:48.309427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148761+0000","last_change":"2023-07-13T22:28:05.144223+0000","last_active":"2023-07-13T22:28:05.148761+0000","last_peered":"2023-07-13T22:28:05.148761+0000","last_clean":"2023-07-13T22:28:05.148761+0000","last_became_active":"2023-07-13T22:28:05.143588+0000","last_became_peered":"2023-07-13T22:28:05.143588+0000","last_unstale":"2023-07-13T22:28:05.148761+0000","last_undegraded":"2023-07-13T22:28:05.148761+0000","last_fullsized":"2023-07-13T22:28:05.148761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:30:21.714147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123366+0000","last_change":"2023-07-13T22:28:05.132165+0000","last_active":"2023-07-13T22:28:06.123366+0000","last_peered":"2023-07-13T22:28:06.123366+0000","last_clean":"2023-07-13T22:28:06.123366+0000","last_became_active":"2023-07-13T22:28:05.131792+0000","last_became_peered":"2023-07-13T22:28:05.131792+0000","last_unstale":"2023-07-13T22:28:06.123366+0000","last_undegraded":"2023-07-13T22:28:06.123366+0000","last_fullsized":"2023-07-13T22:28:06.123366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:31:33.102873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.143141+0000","last_change":"2023-07-13T22:28:05.131662+0000","last_active":"2023-07-13T22:28:05.143141+0000","last_peered":"2023-07-13T22:28:05.143141+0000","last_clean":"2023-07-13T22:28:05.143141+0000","last_became_active":"2023-07-13T22:28:05.131414+0000","last_became_peered":"2023-07-13T22:28:05.131414+0000","last_unstale":"2023-07-13T22:28:05.143141+0000","last_undegraded":"2023-07-13T22:28:05.143141+0000","last_fullsized":"2023-07-13T22:28:05.143141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:15:48.515310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123575+0000","last_change":"2023-07-13T22:28:05.125450+0000","last_active":"2023-07-13T22:28:06.123575+0000","last_peered":"2023-07-13T22:28:06.123575+0000","last_clean":"2023-07-13T22:28:06.123575+0000","last_became_active":"2023-07-13T22:28:05.125113+0000","last_became_peered":"2023-07-13T22:28:05.125113+0000","last_unstale":"2023-07-13T22:28:06.123575+0000","last_undegraded":"2023-07-13T22:28:06.123575+0000","last_fullsized":"2023-07-13T22:28:06.123575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:30:55.850197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.147805+0000","last_change":"2023-07-13T22:28:05.141771+0000","last_active":"2023-07-13T22:28:05.147805+0000","last_peered":"2023-07-13T22:28:05.147805+0000","last_clean":"2023-07-13T22:28:05.147805+0000","last_became_active":"2023-07-13T22:28:05.141447+0000","last_became_peered":"2023-07-13T22:28:05.141447+0000","last_unstale":"2023-07-13T22:28:05.147805+0000","last_undegraded":"2023-07-13T22:28:05.147805+0000","last_fullsized":"2023-07-13T22:28:05.147805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:50:02.888373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124894+0000","last_change":"2023-07-13T22:28:05.128026+0000","last_active":"2023-07-13T22:28:06.124894+0000","last_peered":"2023-07-13T22:28:06.124894+0000","last_clean":"2023-07-13T22:28:06.124894+0000","last_became_active":"2023-07-13T22:28:05.127549+0000","last_became_peered":"2023-07-13T22:28:05.127549+0000","last_unstale":"2023-07-13T22:28:06.124894+0000","last_undegraded":"2023-07-13T22:28:06.124894+0000","last_fullsized":"2023-07-13T22:28:06.124894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:32:42.267626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.140457+0000","last_change":"2023-07-13T22:28:05.135103+0000","last_active":"2023-07-13T22:28:05.140457+0000","last_peered":"2023-07-13T22:28:05.140457+0000","last_clean":"2023-07-13T22:28:05.140457+0000","last_became_active":"2023-07-13T22:28:05.134690+0000","last_became_peered":"2023-07-13T22:28:05.134690+0000","last_unstale":"2023-07-13T22:28:05.140457+0000","last_undegraded":"2023-07-13T22:28:05.140457+0000","last_fullsized":"2023-07-13T22:28:05.140457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:02:10.808735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.139143+0000","last_change":"2023-07-13T22:28:05.134778+0000","last_active":"2023-07-13T22:28:05.139143+0000","last_peered":"2023-07-13T22:28:05.139143+0000","last_clean":"2023-07-13T22:28:05.139143+0000","last_became_active":"2023-07-13T22:28:05.134498+0000","last_became_peered":"2023-07-13T22:28:05.134498+0000","last_unstale":"2023-07-13T22:28:05.139143+0000","last_undegraded":"2023-07-13T22:28:05.139143+0000","last_fullsized":"2023-07-13T22:28:05.139143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:59.410349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123799+0000","last_change":"2023-07-13T22:28:05.136940+0000","last_active":"2023-07-13T22:28:06.123799+0000","last_peered":"2023-07-13T22:28:06.123799+0000","last_clean":"2023-07-13T22:28:06.123799+0000","last_became_active":"2023-07-13T22:28:05.136666+0000","last_became_peered":"2023-07-13T22:28:05.136666+0000","last_unstale":"2023-07-13T22:28:06.123799+0000","last_undegraded":"2023-07-13T22:28:06.123799+0000","last_fullsized":"2023-07-13T22:28:06.123799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:40:30.225003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148424+0000","last_change":"2023-07-13T22:28:05.141803+0000","last_active":"2023-07-13T22:28:05.148424+0000","last_peered":"2023-07-13T22:28:05.148424+0000","last_clean":"2023-07-13T22:28:05.148424+0000","last_became_active":"2023-07-13T22:28:05.141480+0000","last_became_peered":"2023-07-13T22:28:05.141480+0000","last_unstale":"2023-07-13T22:28:05.148424+0000","last_undegraded":"2023-07-13T22:28:05.148424+0000","last_fullsized":"2023-07-13T22:28:05.148424+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:30:17.292002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148500+0000","last_change":"2023-07-13T22:28:05.138725+0000","last_active":"2023-07-13T22:28:05.148500+0000","last_peered":"2023-07-13T22:28:05.148500+0000","last_clean":"2023-07-13T22:28:05.148500+0000","last_became_active":"2023-07-13T22:28:05.138011+0000","last_became_peered":"2023-07-13T22:28:05.138011+0000","last_unstale":"2023-07-13T22:28:05.148500+0000","last_undegraded":"2023-07-13T22:28:05.148500+0000","last_fullsized":"2023-07-13T22:28:05.148500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:11.314965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148732+0000","last_change":"2023-07-13T22:28:05.141932+0000","last_active":"2023-07-13T22:28:05.148732+0000","last_peered":"2023-07-13T22:28:05.148732+0000","last_clean":"2023-07-13T22:28:05.148732+0000","last_became_active":"2023-07-13T22:28:05.141649+0000","last_became_peered":"2023-07-13T22:28:05.141649+0000","last_unstale":"2023-07-13T22:28:05.148732+0000","last_undegraded":"2023-07-13T22:28:05.148732+0000","last_fullsized":"2023-07-13T22:28:05.148732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:58:09.166380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.121208+0000","last_change":"2023-07-13T22:28:05.132456+0000","last_active":"2023-07-13T22:28:06.121208+0000","last_peered":"2023-07-13T22:28:06.121208+0000","last_clean":"2023-07-13T22:28:06.121208+0000","last_became_active":"2023-07-13T22:28:05.131979+0000","last_became_peered":"2023-07-13T22:28:05.131979+0000","last_unstale":"2023-07-13T22:28:06.121208+0000","last_undegraded":"2023-07-13T22:28:06.121208+0000","last_fullsized":"2023-07-13T22:28:06.121208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:27:07.969392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124584+0000","last_change":"2023-07-13T22:28:05.131909+0000","last_active":"2023-07-13T22:28:06.124584+0000","last_peered":"2023-07-13T22:28:06.124584+0000","last_clean":"2023-07-13T22:28:06.124584+0000","last_became_active":"2023-07-13T22:28:05.131555+0000","last_became_peered":"2023-07-13T22:28:05.131555+0000","last_unstale":"2023-07-13T22:28:06.124584+0000","last_undegraded":"2023-07-13T22:28:06.124584+0000","last_fullsized":"2023-07-13T22:28:06.124584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:08:35.046700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148391+0000","last_change":"2023-07-13T22:28:05.141090+0000","last_active":"2023-07-13T22:28:05.148391+0000","last_peered":"2023-07-13T22:28:05.148391+0000","last_clean":"2023-07-13T22:28:05.148391+0000","last_became_active":"2023-07-13T22:28:05.140529+0000","last_became_peered":"2023-07-13T22:28:05.140529+0000","last_unstale":"2023-07-13T22:28:05.148391+0000","last_undegraded":"2023-07-13T22:28:05.148391+0000","last_fullsized":"2023-07-13T22:28:05.148391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:02:11.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124206+0000","last_change":"2023-07-13T22:28:05.113249+0000","last_active":"2023-07-13T22:28:06.124206+0000","last_peered":"2023-07-13T22:28:06.124206+0000","last_clean":"2023-07-13T22:28:06.124206+0000","last_became_active":"2023-07-13T22:28:05.112862+0000","last_became_peered":"2023-07-13T22:28:05.112862+0000","last_unstale":"2023-07-13T22:28:06.124206+0000","last_undegraded":"2023-07-13T22:28:06.124206+0000","last_fullsized":"2023-07-13T22:28:06.124206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:56:23.463078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123949+0000","last_change":"2023-07-13T22:28:05.136898+0000","last_active":"2023-07-13T22:28:06.123949+0000","last_peered":"2023-07-13T22:28:06.123949+0000","last_clean":"2023-07-13T22:28:06.123949+0000","last_became_active":"2023-07-13T22:28:05.136605+0000","last_became_peered":"2023-07-13T22:28:05.136605+0000","last_unstale":"2023-07-13T22:28:06.123949+0000","last_undegraded":"2023-07-13T22:28:06.123949+0000","last_fullsized":"2023-07-13T22:28:06.123949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:33:57.059183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.137763+0000","last_change":"2023-07-13T22:28:05.136943+0000","last_active":"2023-07-13T22:28:05.137763+0000","last_peered":"2023-07-13T22:28:05.137763+0000","last_clean":"2023-07-13T22:28:05.137763+0000","last_became_active":"2023-07-13T22:28:05.136593+0000","last_became_peered":"2023-07-13T22:28:05.136593+0000","last_unstale":"2023-07-13T22:28:05.137763+0000","last_undegraded":"2023-07-13T22:28:05.137763+0000","last_fullsized":"2023-07-13T22:28:05.137763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:14:41.780622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120734+0000","last_change":"2023-07-13T22:28:05.131146+0000","last_active":"2023-07-13T22:28:06.120734+0000","last_peered":"2023-07-13T22:28:06.120734+0000","last_clean":"2023-07-13T22:28:06.120734+0000","last_became_active":"2023-07-13T22:28:05.130854+0000","last_became_peered":"2023-07-13T22:28:05.130854+0000","last_unstale":"2023-07-13T22:28:06.120734+0000","last_undegraded":"2023-07-13T22:28:06.120734+0000","last_fullsized":"2023-07-13T22:28:06.120734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:34:20.015490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.130674+0000","last_change":"2023-07-13T22:28:05.129899+0000","last_active":"2023-07-13T22:28:05.130674+0000","last_peered":"2023-07-13T22:28:05.130674+0000","last_clean":"2023-07-13T22:28:05.130674+0000","last_became_active":"2023-07-13T22:28:05.129555+0000","last_became_peered":"2023-07-13T22:28:05.129555+0000","last_unstale":"2023-07-13T22:28:05.130674+0000","last_undegraded":"2023-07-13T22:28:05.130674+0000","last_fullsized":"2023-07-13T22:28:05.130674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:54.170522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123885+0000","last_change":"2023-07-13T22:28:05.140369+0000","last_active":"2023-07-13T22:28:06.123885+0000","last_peered":"2023-07-13T22:28:06.123885+0000","last_clean":"2023-07-13T22:28:06.123885+0000","last_became_active":"2023-07-13T22:28:05.140005+0000","last_became_peered":"2023-07-13T22:28:05.140005+0000","last_unstale":"2023-07-13T22:28:06.123885+0000","last_undegraded":"2023-07-13T22:28:06.123885+0000","last_fullsized":"2023-07-13T22:28:06.123885+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:14:18.498632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.133833+0000","last_change":"2023-07-13T22:28:05.125174+0000","last_active":"2023-07-13T22:28:05.133833+0000","last_peered":"2023-07-13T22:28:05.133833+0000","last_clean":"2023-07-13T22:28:05.133833+0000","last_became_active":"2023-07-13T22:28:05.124145+0000","last_became_peered":"2023-07-13T22:28:05.124145+0000","last_unstale":"2023-07-13T22:28:05.133833+0000","last_undegraded":"2023-07-13T22:28:05.133833+0000","last_fullsized":"2023-07-13T22:28:05.133833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:07:09.664684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.139788+0000","last_change":"2023-07-13T22:28:05.132304+0000","last_active":"2023-07-13T22:28:05.139788+0000","last_peered":"2023-07-13T22:28:05.139788+0000","last_clean":"2023-07-13T22:28:05.139788+0000","last_became_active":"2023-07-13T22:28:05.131985+0000","last_became_peered":"2023-07-13T22:28:05.131985+0000","last_unstale":"2023-07-13T22:28:05.139788+0000","last_undegraded":"2023-07-13T22:28:05.139788+0000","last_fullsized":"2023-07-13T22:28:05.139788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:24:35.465349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123614+0000","last_change":"2023-07-13T22:28:05.116319+0000","last_active":"2023-07-13T22:28:06.123614+0000","last_peered":"2023-07-13T22:28:06.123614+0000","last_clean":"2023-07-13T22:28:06.123614+0000","last_became_active":"2023-07-13T22:28:05.116001+0000","last_became_peered":"2023-07-13T22:28:05.116001+0000","last_unstale":"2023-07-13T22:28:06.123614+0000","last_undegraded":"2023-07-13T22:28:06.123614+0000","last_fullsized":"2023-07-13T22:28:06.123614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:41:26.305788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124141+0000","last_change":"2023-07-13T22:28:05.125706+0000","last_active":"2023-07-13T22:28:06.124141+0000","last_peered":"2023-07-13T22:28:06.124141+0000","last_clean":"2023-07-13T22:28:06.124141+0000","last_became_active":"2023-07-13T22:28:05.124797+0000","last_became_peered":"2023-07-13T22:28:05.124797+0000","last_unstale":"2023-07-13T22:28:06.124141+0000","last_undegraded":"2023-07-13T22:28:06.124141+0000","last_fullsized":"2023-07-13T22:28:06.124141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:49:23.912113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.142878+0000","last_change":"2023-07-13T22:28:05.141555+0000","last_active":"2023-07-13T22:28:05.142878+0000","last_peered":"2023-07-13T22:28:05.142878+0000","last_clean":"2023-07-13T22:28:05.142878+0000","last_became_active":"2023-07-13T22:28:05.141252+0000","last_became_peered":"2023-07-13T22:28:05.141252+0000","last_unstale":"2023-07-13T22:28:05.142878+0000","last_undegraded":"2023-07-13T22:28:05.142878+0000","last_fullsized":"2023-07-13T22:28:05.142878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:25:43.756134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148523+0000","last_change":"2023-07-13T22:28:05.138929+0000","last_active":"2023-07-13T22:28:05.148523+0000","last_peered":"2023-07-13T22:28:05.148523+0000","last_clean":"2023-07-13T22:28:05.148523+0000","last_became_active":"2023-07-13T22:28:05.138459+0000","last_became_peered":"2023-07-13T22:28:05.138459+0000","last_unstale":"2023-07-13T22:28:05.148523+0000","last_undegraded":"2023-07-13T22:28:05.148523+0000","last_fullsized":"2023-07-13T22:28:05.148523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:38:27.482861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.132402+0000","last_change":"2023-07-13T22:28:05.128710+0000","last_active":"2023-07-13T22:28:05.132402+0000","last_peered":"2023-07-13T22:28:05.132402+0000","last_clean":"2023-07-13T22:28:05.132402+0000","last_became_active":"2023-07-13T22:28:05.128292+0000","last_became_peered":"2023-07-13T22:28:05.128292+0000","last_unstale":"2023-07-13T22:28:05.132402+0000","last_undegraded":"2023-07-13T22:28:05.132402+0000","last_fullsized":"2023-07-13T22:28:05.132402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:53:33.685855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"86'4","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.128241+0000","last_change":"2023-07-13T22:28:04.124865+0000","last_active":"2023-07-13T22:28:05.128241+0000","last_peered":"2023-07-13T22:28:05.128241+0000","last_clean":"2023-07-13T22:28:05.128241+0000","last_became_active":"2023-07-13T22:28:04.124456+0000","last_became_peered":"2023-07-13T22:28:04.124456+0000","last_unstale":"2023-07-13T22:28:05.128241+0000","last_undegraded":"2023-07-13T22:28:05.128241+0000","last_fullsized":"2023-07-13T22:28:05.128241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:07:41.535542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148560+0000","last_change":"2023-07-13T22:28:05.139945+0000","last_active":"2023-07-13T22:28:05.148560+0000","last_peered":"2023-07-13T22:28:05.148560+0000","last_clean":"2023-07-13T22:28:05.148560+0000","last_became_active":"2023-07-13T22:28:05.139688+0000","last_became_peered":"2023-07-13T22:28:05.139688+0000","last_unstale":"2023-07-13T22:28:05.148560+0000","last_undegraded":"2023-07-13T22:28:05.148560+0000","last_fullsized":"2023-07-13T22:28:05.148560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:43:26.500224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"86'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124016+0000","last_change":"2023-07-13T22:28:04.122784+0000","last_active":"2023-07-13T22:28:06.124016+0000","last_peered":"2023-07-13T22:28:06.124016+0000","last_clean":"2023-07-13T22:28:06.124016+0000","last_became_active":"2023-07-13T22:28:04.122526+0000","last_became_peered":"2023-07-13T22:28:04.122526+0000","last_unstale":"2023-07-13T22:28:06.124016+0000","last_undegraded":"2023-07-13T22:28:06.124016+0000","last_fullsized":"2023-07-13T22:28:06.124016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:30:52.863628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123581+0000","last_change":"2023-07-13T22:28:05.140647+0000","last_active":"2023-07-13T22:28:06.123581+0000","last_peered":"2023-07-13T22:28:06.123581+0000","last_clean":"2023-07-13T22:28:06.123581+0000","last_became_active":"2023-07-13T22:28:05.140359+0000","last_became_peered":"2023-07-13T22:28:05.140359+0000","last_unstale":"2023-07-13T22:28:06.123581+0000","last_undegraded":"2023-07-13T22:28:06.123581+0000","last_fullsized":"2023-07-13T22:28:06.123581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:22:43.078094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"86'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123954+0000","last_change":"2023-07-13T22:28:04.122417+0000","last_active":"2023-07-13T22:28:06.123954+0000","last_peered":"2023-07-13T22:28:06.123954+0000","last_clean":"2023-07-13T22:28:06.123954+0000","last_became_active":"2023-07-13T22:28:04.122170+0000","last_became_peered":"2023-07-13T22:28:04.122170+0000","last_unstale":"2023-07-13T22:28:06.123954+0000","last_undegraded":"2023-07-13T22:28:06.123954+0000","last_fullsized":"2023-07-13T22:28:06.123954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:07:21.817229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.147735+0000","last_change":"2023-07-13T22:28:05.131812+0000","last_active":"2023-07-13T22:28:05.147735+0000","last_peered":"2023-07-13T22:28:05.147735+0000","last_clean":"2023-07-13T22:28:05.147735+0000","last_became_active":"2023-07-13T22:28:05.131533+0000","last_became_peered":"2023-07-13T22:28:05.131533+0000","last_unstale":"2023-07-13T22:28:05.147735+0000","last_undegraded":"2023-07-13T22:28:05.147735+0000","last_fullsized":"2023-07-13T22:28:05.147735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:39:09.576155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"86'2","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.146172+0000","last_change":"2023-07-13T22:28:04.125078+0000","last_active":"2023-07-13T22:28:05.146172+0000","last_peered":"2023-07-13T22:28:05.146172+0000","last_clean":"2023-07-13T22:28:05.146172+0000","last_became_active":"2023-07-13T22:28:04.124530+0000","last_became_peered":"2023-07-13T22:28:04.124530+0000","last_unstale":"2023-07-13T22:28:05.146172+0000","last_undegraded":"2023-07-13T22:28:05.146172+0000","last_fullsized":"2023-07-13T22:28:05.146172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:37.461212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148620+0000","last_change":"2023-07-13T22:28:05.144236+0000","last_active":"2023-07-13T22:28:05.148620+0000","last_peered":"2023-07-13T22:28:05.148620+0000","last_clean":"2023-07-13T22:28:05.148620+0000","last_became_active":"2023-07-13T22:28:05.143719+0000","last_became_peered":"2023-07-13T22:28:05.143719+0000","last_unstale":"2023-07-13T22:28:05.148620+0000","last_undegraded":"2023-07-13T22:28:05.148620+0000","last_fullsized":"2023-07-13T22:28:05.148620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:14:41.780622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"86'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124502+0000","last_change":"2023-07-13T22:28:04.116193+0000","last_active":"2023-07-13T22:28:06.124502+0000","last_peered":"2023-07-13T22:28:06.124502+0000","last_clean":"2023-07-13T22:28:06.124502+0000","last_became_active":"2023-07-13T22:28:04.115912+0000","last_became_peered":"2023-07-13T22:28:04.115912+0000","last_unstale":"2023-07-13T22:28:06.124502+0000","last_undegraded":"2023-07-13T22:28:06.124502+0000","last_fullsized":"2023-07-13T22:28:06.124502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:58:40.075593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.130162+0000","last_change":"2023-07-13T22:28:05.127334+0000","last_active":"2023-07-13T22:28:05.130162+0000","last_peered":"2023-07-13T22:28:05.130162+0000","last_clean":"2023-07-13T22:28:05.130162+0000","last_became_active":"2023-07-13T22:28:05.126970+0000","last_became_peered":"2023-07-13T22:28:05.126970+0000","last_unstale":"2023-07-13T22:28:05.130162+0000","last_undegraded":"2023-07-13T22:28:05.130162+0000","last_fullsized":"2023-07-13T22:28:05.130162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:25:19.873884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.140419+0000","last_change":"2023-07-13T22:28:04.113585+0000","last_active":"2023-07-13T22:28:05.140419+0000","last_peered":"2023-07-13T22:28:05.140419+0000","last_clean":"2023-07-13T22:28:05.140419+0000","last_became_active":"2023-07-13T22:28:04.113181+0000","last_became_peered":"2023-07-13T22:28:04.113181+0000","last_unstale":"2023-07-13T22:28:05.140419+0000","last_undegraded":"2023-07-13T22:28:05.140419+0000","last_fullsized":"2023-07-13T22:28:05.140419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:31:45.581771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148785+0000","last_change":"2023-07-13T22:28:05.144347+0000","last_active":"2023-07-13T22:28:05.148785+0000","last_peered":"2023-07-13T22:28:05.148785+0000","last_clean":"2023-07-13T22:28:05.148785+0000","last_became_active":"2023-07-13T22:28:05.143940+0000","last_became_peered":"2023-07-13T22:28:05.143940+0000","last_unstale":"2023-07-13T22:28:05.148785+0000","last_undegraded":"2023-07-13T22:28:05.148785+0000","last_fullsized":"2023-07-13T22:28:05.148785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:48:12.036002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120873+0000","last_change":"2023-07-13T22:28:04.123606+0000","last_active":"2023-07-13T22:28:06.120873+0000","last_peered":"2023-07-13T22:28:06.120873+0000","last_clean":"2023-07-13T22:28:06.120873+0000","last_became_active":"2023-07-13T22:28:04.122467+0000","last_became_peered":"2023-07-13T22:28:04.122467+0000","last_unstale":"2023-07-13T22:28:06.120873+0000","last_undegraded":"2023-07-13T22:28:06.120873+0000","last_fullsized":"2023-07-13T22:28:06.120873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:00:29.399267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.139120+0000","last_change":"2023-07-13T22:28:05.130768+0000","last_active":"2023-07-13T22:28:05.139120+0000","last_peered":"2023-07-13T22:28:05.139120+0000","last_clean":"2023-07-13T22:28:05.139120+0000","last_became_active":"2023-07-13T22:28:05.130496+0000","last_became_peered":"2023-07-13T22:28:05.130496+0000","last_unstale":"2023-07-13T22:28:05.139120+0000","last_undegraded":"2023-07-13T22:28:05.139120+0000","last_fullsized":"2023-07-13T22:28:05.139120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:49:26.925461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123109+0000","last_change":"2023-07-13T22:28:04.121114+0000","last_active":"2023-07-13T22:28:06.123109+0000","last_peered":"2023-07-13T22:28:06.123109+0000","last_clean":"2023-07-13T22:28:06.123109+0000","last_became_active":"2023-07-13T22:28:04.120764+0000","last_became_peered":"2023-07-13T22:28:04.120764+0000","last_unstale":"2023-07-13T22:28:06.123109+0000","last_undegraded":"2023-07-13T22:28:06.123109+0000","last_fullsized":"2023-07-13T22:28:06.123109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:34:12.381021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123603+0000","last_change":"2023-07-13T22:28:05.135296+0000","last_active":"2023-07-13T22:28:06.123603+0000","last_peered":"2023-07-13T22:28:06.123603+0000","last_clean":"2023-07-13T22:28:06.123603+0000","last_became_active":"2023-07-13T22:28:05.134903+0000","last_became_peered":"2023-07-13T22:28:05.134903+0000","last_unstale":"2023-07-13T22:28:06.123603+0000","last_undegraded":"2023-07-13T22:28:06.123603+0000","last_fullsized":"2023-07-13T22:28:06.123603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:51:58.047835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117509+0000","last_change":"2023-07-13T22:28:04.116318+0000","last_active":"2023-07-13T22:28:05.117509+0000","last_peered":"2023-07-13T22:28:05.117509+0000","last_clean":"2023-07-13T22:28:05.117509+0000","last_became_active":"2023-07-13T22:28:04.116032+0000","last_became_peered":"2023-07-13T22:28:04.116032+0000","last_unstale":"2023-07-13T22:28:05.117509+0000","last_undegraded":"2023-07-13T22:28:05.117509+0000","last_fullsized":"2023-07-13T22:28:05.117509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:04:54.440821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123833+0000","last_change":"2023-07-13T22:28:05.116979+0000","last_active":"2023-07-13T22:28:06.123833+0000","last_peered":"2023-07-13T22:28:06.123833+0000","last_clean":"2023-07-13T22:28:06.123833+0000","last_became_active":"2023-07-13T22:28:05.116711+0000","last_became_peered":"2023-07-13T22:28:05.116711+0000","last_unstale":"2023-07-13T22:28:06.123833+0000","last_undegraded":"2023-07-13T22:28:06.123833+0000","last_fullsized":"2023-07-13T22:28:06.123833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:22:52.742728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.121246+0000","last_change":"2023-07-13T22:28:04.121322+0000","last_active":"2023-07-13T22:28:06.121246+0000","last_peered":"2023-07-13T22:28:06.121246+0000","last_clean":"2023-07-13T22:28:06.121246+0000","last_became_active":"2023-07-13T22:28:04.121033+0000","last_became_peered":"2023-07-13T22:28:04.121033+0000","last_unstale":"2023-07-13T22:28:06.121246+0000","last_undegraded":"2023-07-13T22:28:06.121246+0000","last_fullsized":"2023-07-13T22:28:06.121246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:57:55.847128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120964+0000","last_change":"2023-07-13T22:28:05.132267+0000","last_active":"2023-07-13T22:28:06.120964+0000","last_peered":"2023-07-13T22:28:06.120964+0000","last_clean":"2023-07-13T22:28:06.120964+0000","last_became_active":"2023-07-13T22:28:05.131658+0000","last_became_peered":"2023-07-13T22:28:05.131658+0000","last_unstale":"2023-07-13T22:28:06.120964+0000","last_undegraded":"2023-07-13T22:28:06.120964+0000","last_fullsized":"2023-07-13T22:28:06.120964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:50:24.483224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"86'3","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124760+0000","last_change":"2023-07-13T22:28:04.120443+0000","last_active":"2023-07-13T22:28:06.124760+0000","last_peered":"2023-07-13T22:28:06.124760+0000","last_clean":"2023-07-13T22:28:06.124760+0000","last_became_active":"2023-07-13T22:28:04.120096+0000","last_became_peered":"2023-07-13T22:28:04.120096+0000","last_unstale":"2023-07-13T22:28:06.124760+0000","last_undegraded":"2023-07-13T22:28:06.124760+0000","last_fullsized":"2023-07-13T22:28:06.124760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:03:05.434656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.140502+0000","last_change":"2023-07-13T22:28:05.131997+0000","last_active":"2023-07-13T22:28:05.140502+0000","last_peered":"2023-07-13T22:28:05.140502+0000","last_clean":"2023-07-13T22:28:05.140502+0000","last_became_active":"2023-07-13T22:28:05.131742+0000","last_became_peered":"2023-07-13T22:28:05.131742+0000","last_unstale":"2023-07-13T22:28:05.140502+0000","last_undegraded":"2023-07-13T22:28:05.140502+0000","last_fullsized":"2023-07-13T22:28:05.140502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:39:44.607563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"86'6","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.150872+0000","last_change":"2023-07-13T22:28:04.125990+0000","last_active":"2023-07-13T22:28:05.150872+0000","last_peered":"2023-07-13T22:28:05.150872+0000","last_clean":"2023-07-13T22:28:05.150872+0000","last_became_active":"2023-07-13T22:28:04.125639+0000","last_became_peered":"2023-07-13T22:28:04.125639+0000","last_unstale":"2023-07-13T22:28:05.150872+0000","last_undegraded":"2023-07-13T22:28:05.150872+0000","last_fullsized":"2023-07-13T22:28:05.150872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:19:30.990209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148601+0000","last_change":"2023-07-13T22:28:05.144071+0000","last_active":"2023-07-13T22:28:05.148601+0000","last_peered":"2023-07-13T22:28:05.148601+0000","last_clean":"2023-07-13T22:28:05.148601+0000","last_became_active":"2023-07-13T22:28:05.143311+0000","last_became_peered":"2023-07-13T22:28:05.143311+0000","last_unstale":"2023-07-13T22:28:05.148601+0000","last_undegraded":"2023-07-13T22:28:05.148601+0000","last_fullsized":"2023-07-13T22:28:05.148601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:52:25.475987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"86'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123779+0000","last_change":"2023-07-13T22:28:04.121584+0000","last_active":"2023-07-13T22:28:06.123779+0000","last_peered":"2023-07-13T22:28:06.123779+0000","last_clean":"2023-07-13T22:28:06.123779+0000","last_became_active":"2023-07-13T22:28:04.121300+0000","last_became_peered":"2023-07-13T22:28:04.121300+0000","last_unstale":"2023-07-13T22:28:06.123779+0000","last_undegraded":"2023-07-13T22:28:06.123779+0000","last_fullsized":"2023-07-13T22:28:06.123779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:55:31.018514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123954+0000","last_change":"2023-07-13T22:28:05.120159+0000","last_active":"2023-07-13T22:28:06.123954+0000","last_peered":"2023-07-13T22:28:06.123954+0000","last_clean":"2023-07-13T22:28:06.123954+0000","last_became_active":"2023-07-13T22:28:05.119840+0000","last_became_peered":"2023-07-13T22:28:05.119840+0000","last_unstale":"2023-07-13T22:28:06.123954+0000","last_undegraded":"2023-07-13T22:28:06.123954+0000","last_fullsized":"2023-07-13T22:28:06.123954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:02:51.152705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.137152+0000","last_change":"2023-07-13T22:28:04.123344+0000","last_active":"2023-07-13T22:28:05.137152+0000","last_peered":"2023-07-13T22:28:05.137152+0000","last_clean":"2023-07-13T22:28:05.137152+0000","last_became_active":"2023-07-13T22:28:04.122982+0000","last_became_peered":"2023-07-13T22:28:04.122982+0000","last_unstale":"2023-07-13T22:28:05.137152+0000","last_undegraded":"2023-07-13T22:28:05.137152+0000","last_fullsized":"2023-07-13T22:28:05.137152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:34:57.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.146642+0000","last_change":"2023-07-13T22:28:05.141551+0000","last_active":"2023-07-13T22:28:05.146642+0000","last_peered":"2023-07-13T22:28:05.146642+0000","last_clean":"2023-07-13T22:28:05.146642+0000","last_became_active":"2023-07-13T22:28:05.141228+0000","last_became_peered":"2023-07-13T22:28:05.141228+0000","last_unstale":"2023-07-13T22:28:05.146642+0000","last_undegraded":"2023-07-13T22:28:05.146642+0000","last_fullsized":"2023-07-13T22:28:05.146642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:24:56.271819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.114947+0000","last_change":"2023-07-13T22:28:04.125608+0000","last_active":"2023-07-13T22:28:05.114947+0000","last_peered":"2023-07-13T22:28:05.114947+0000","last_clean":"2023-07-13T22:28:05.114947+0000","last_became_active":"2023-07-13T22:28:04.125269+0000","last_became_peered":"2023-07-13T22:28:04.125269+0000","last_unstale":"2023-07-13T22:28:05.114947+0000","last_undegraded":"2023-07-13T22:28:05.114947+0000","last_fullsized":"2023-07-13T22:28:05.114947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:24:32.672612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117143+0000","last_change":"2023-07-13T22:28:05.115291+0000","last_active":"2023-07-13T22:28:05.117143+0000","last_peered":"2023-07-13T22:28:05.117143+0000","last_clean":"2023-07-13T22:28:05.117143+0000","last_became_active":"2023-07-13T22:28:05.114976+0000","last_became_peered":"2023-07-13T22:28:05.114976+0000","last_unstale":"2023-07-13T22:28:05.117143+0000","last_undegraded":"2023-07-13T22:28:05.117143+0000","last_fullsized":"2023-07-13T22:28:05.117143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:56:01.267865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.143027+0000","last_change":"2023-07-13T22:28:04.114326+0000","last_active":"2023-07-13T22:28:05.143027+0000","last_peered":"2023-07-13T22:28:05.143027+0000","last_clean":"2023-07-13T22:28:05.143027+0000","last_became_active":"2023-07-13T22:28:04.114014+0000","last_became_peered":"2023-07-13T22:28:04.114014+0000","last_unstale":"2023-07-13T22:28:05.143027+0000","last_undegraded":"2023-07-13T22:28:05.143027+0000","last_fullsized":"2023-07-13T22:28:05.143027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:37.461212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117841+0000","last_change":"2023-07-13T22:28:05.115809+0000","last_active":"2023-07-13T22:28:05.117841+0000","last_peered":"2023-07-13T22:28:05.117841+0000","last_clean":"2023-07-13T22:28:05.117841+0000","last_became_active":"2023-07-13T22:28:05.115493+0000","last_became_peered":"2023-07-13T22:28:05.115493+0000","last_unstale":"2023-07-13T22:28:05.117841+0000","last_undegraded":"2023-07-13T22:28:05.117841+0000","last_fullsized":"2023-07-13T22:28:05.117841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:31:46.589384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"86'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124123+0000","last_change":"2023-07-13T22:28:04.125778+0000","last_active":"2023-07-13T22:28:06.124123+0000","last_peered":"2023-07-13T22:28:06.124123+0000","last_clean":"2023-07-13T22:28:06.124123+0000","last_became_active":"2023-07-13T22:28:04.125454+0000","last_became_peered":"2023-07-13T22:28:04.125454+0000","last_unstale":"2023-07-13T22:28:06.124123+0000","last_undegraded":"2023-07-13T22:28:06.124123+0000","last_fullsized":"2023-07-13T22:28:06.124123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:26.332363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.147787+0000","last_change":"2023-07-13T22:28:05.131796+0000","last_active":"2023-07-13T22:28:05.147787+0000","last_peered":"2023-07-13T22:28:05.147787+0000","last_clean":"2023-07-13T22:28:05.147787+0000","last_became_active":"2023-07-13T22:28:05.131475+0000","last_became_peered":"2023-07-13T22:28:05.131475+0000","last_unstale":"2023-07-13T22:28:05.147787+0000","last_undegraded":"2023-07-13T22:28:05.147787+0000","last_fullsized":"2023-07-13T22:28:05.147787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:36:09.746449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124120+0000","last_change":"2023-07-13T22:28:04.117826+0000","last_active":"2023-07-13T22:28:06.124120+0000","last_peered":"2023-07-13T22:28:06.124120+0000","last_clean":"2023-07-13T22:28:06.124120+0000","last_became_active":"2023-07-13T22:28:04.117550+0000","last_became_peered":"2023-07-13T22:28:04.117550+0000","last_unstale":"2023-07-13T22:28:06.124120+0000","last_undegraded":"2023-07-13T22:28:06.124120+0000","last_fullsized":"2023-07-13T22:28:06.124120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:42:28.594229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148684+0000","last_change":"2023-07-13T22:28:05.142881+0000","last_active":"2023-07-13T22:28:05.148684+0000","last_peered":"2023-07-13T22:28:05.148684+0000","last_clean":"2023-07-13T22:28:05.148684+0000","last_became_active":"2023-07-13T22:28:05.142589+0000","last_became_peered":"2023-07-13T22:28:05.142589+0000","last_unstale":"2023-07-13T22:28:05.148684+0000","last_undegraded":"2023-07-13T22:28:05.148684+0000","last_fullsized":"2023-07-13T22:28:05.148684+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:20:22.801603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124741+0000","last_change":"2023-07-13T22:28:04.120397+0000","last_active":"2023-07-13T22:28:06.124741+0000","last_peered":"2023-07-13T22:28:06.124741+0000","last_clean":"2023-07-13T22:28:06.124741+0000","last_became_active":"2023-07-13T22:28:04.119625+0000","last_became_peered":"2023-07-13T22:28:04.119625+0000","last_unstale":"2023-07-13T22:28:06.124741+0000","last_undegraded":"2023-07-13T22:28:06.124741+0000","last_fullsized":"2023-07-13T22:28:06.124741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:14:02.481798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.128166+0000","last_change":"2023-07-13T22:28:05.121440+0000","last_active":"2023-07-13T22:28:05.128166+0000","last_peered":"2023-07-13T22:28:05.128166+0000","last_clean":"2023-07-13T22:28:05.128166+0000","last_became_active":"2023-07-13T22:28:05.121166+0000","last_became_peered":"2023-07-13T22:28:05.121166+0000","last_unstale":"2023-07-13T22:28:05.128166+0000","last_undegraded":"2023-07-13T22:28:05.128166+0000","last_fullsized":"2023-07-13T22:28:05.128166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:19:31.997822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.111694+0000","last_change":"2023-07-13T22:28:04.125151+0000","last_active":"2023-07-13T22:28:05.111694+0000","last_peered":"2023-07-13T22:28:05.111694+0000","last_clean":"2023-07-13T22:28:05.111694+0000","last_became_active":"2023-07-13T22:28:04.124580+0000","last_became_peered":"2023-07-13T22:28:04.124580+0000","last_unstale":"2023-07-13T22:28:05.111694+0000","last_undegraded":"2023-07-13T22:28:05.111694+0000","last_fullsized":"2023-07-13T22:28:05.111694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:23.537202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.134051+0000","last_change":"2023-07-13T22:28:05.125659+0000","last_active":"2023-07-13T22:28:05.134051+0000","last_peered":"2023-07-13T22:28:05.134051+0000","last_clean":"2023-07-13T22:28:05.134051+0000","last_became_active":"2023-07-13T22:28:05.125342+0000","last_became_peered":"2023-07-13T22:28:05.125342+0000","last_unstale":"2023-07-13T22:28:05.134051+0000","last_undegraded":"2023-07-13T22:28:05.134051+0000","last_fullsized":"2023-07-13T22:28:05.134051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:30:43.561195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.142817+0000","last_change":"2023-07-13T22:28:04.121524+0000","last_active":"2023-07-13T22:28:05.142817+0000","last_peered":"2023-07-13T22:28:05.142817+0000","last_clean":"2023-07-13T22:28:05.142817+0000","last_became_active":"2023-07-13T22:28:04.120610+0000","last_became_peered":"2023-07-13T22:28:04.120610+0000","last_unstale":"2023-07-13T22:28:05.142817+0000","last_undegraded":"2023-07-13T22:28:05.142817+0000","last_fullsized":"2023-07-13T22:28:05.142817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:51:35.242168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120733+0000","last_change":"2023-07-13T22:28:05.135910+0000","last_active":"2023-07-13T22:28:06.120733+0000","last_peered":"2023-07-13T22:28:06.120733+0000","last_clean":"2023-07-13T22:28:06.120733+0000","last_became_active":"2023-07-13T22:28:05.135188+0000","last_became_peered":"2023-07-13T22:28:05.135188+0000","last_unstale":"2023-07-13T22:28:06.120733+0000","last_undegraded":"2023-07-13T22:28:06.120733+0000","last_fullsized":"2023-07-13T22:28:06.120733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:55:52.233253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.113862+0000","last_change":"2023-07-13T22:28:04.117486+0000","last_active":"2023-07-13T22:28:05.113862+0000","last_peered":"2023-07-13T22:28:05.113862+0000","last_clean":"2023-07-13T22:28:05.113862+0000","last_became_active":"2023-07-13T22:28:04.117179+0000","last_became_peered":"2023-07-13T22:28:04.117179+0000","last_unstale":"2023-07-13T22:28:05.113862+0000","last_undegraded":"2023-07-13T22:28:05.113862+0000","last_fullsized":"2023-07-13T22:28:05.113862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:51:35.242168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.137761+0000","last_change":"2023-07-13T22:28:05.136950+0000","last_active":"2023-07-13T22:28:05.137761+0000","last_peered":"2023-07-13T22:28:05.137761+0000","last_clean":"2023-07-13T22:28:05.137761+0000","last_became_active":"2023-07-13T22:28:05.136654+0000","last_became_peered":"2023-07-13T22:28:05.136654+0000","last_unstale":"2023-07-13T22:28:05.137761+0000","last_undegraded":"2023-07-13T22:28:05.137761+0000","last_fullsized":"2023-07-13T22:28:05.137761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:21:18.024141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.112080+0000","last_change":"2023-07-13T22:28:04.123161+0000","last_active":"2023-07-13T22:28:05.112080+0000","last_peered":"2023-07-13T22:28:05.112080+0000","last_clean":"2023-07-13T22:28:05.112080+0000","last_became_active":"2023-07-13T22:28:04.121907+0000","last_became_peered":"2023-07-13T22:28:04.121907+0000","last_unstale":"2023-07-13T22:28:05.112080+0000","last_undegraded":"2023-07-13T22:28:05.112080+0000","last_fullsized":"2023-07-13T22:28:05.112080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:44:38.459035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148949+0000","last_change":"2023-07-13T22:28:05.143433+0000","last_active":"2023-07-13T22:28:05.148949+0000","last_peered":"2023-07-13T22:28:05.148949+0000","last_clean":"2023-07-13T22:28:05.148949+0000","last_became_active":"2023-07-13T22:28:05.143118+0000","last_became_peered":"2023-07-13T22:28:05.143118+0000","last_unstale":"2023-07-13T22:28:05.148949+0000","last_undegraded":"2023-07-13T22:28:05.148949+0000","last_fullsized":"2023-07-13T22:28:05.148949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:07:42.543155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123367+0000","last_change":"2023-07-13T22:28:04.118891+0000","last_active":"2023-07-13T22:28:06.123367+0000","last_peered":"2023-07-13T22:28:06.123367+0000","last_clean":"2023-07-13T22:28:06.123367+0000","last_became_active":"2023-07-13T22:28:04.118621+0000","last_became_peered":"2023-07-13T22:28:04.118621+0000","last_unstale":"2023-07-13T22:28:06.123367+0000","last_undegraded":"2023-07-13T22:28:06.123367+0000","last_fullsized":"2023-07-13T22:28:06.123367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:06:24.628924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.142895+0000","last_change":"2023-07-13T22:28:05.141582+0000","last_active":"2023-07-13T22:28:05.142895+0000","last_peered":"2023-07-13T22:28:05.142895+0000","last_clean":"2023-07-13T22:28:05.142895+0000","last_became_active":"2023-07-13T22:28:05.141187+0000","last_became_peered":"2023-07-13T22:28:05.141187+0000","last_unstale":"2023-07-13T22:28:05.142895+0000","last_undegraded":"2023-07-13T22:28:05.142895+0000","last_fullsized":"2023-07-13T22:28:05.142895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:38:27.482861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117810+0000","last_change":"2023-07-13T22:28:04.119820+0000","last_active":"2023-07-13T22:28:05.117810+0000","last_peered":"2023-07-13T22:28:05.117810+0000","last_clean":"2023-07-13T22:28:05.117810+0000","last_became_active":"2023-07-13T22:28:04.119536+0000","last_became_peered":"2023-07-13T22:28:04.119536+0000","last_unstale":"2023-07-13T22:28:05.117810+0000","last_undegraded":"2023-07-13T22:28:05.117810+0000","last_fullsized":"2023-07-13T22:28:05.117810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:48:14.544857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124343+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:28:06.124343+0000","last_peered":"2023-07-13T22:28:06.124343+0000","last_clean":"2023-07-13T22:28:06.124343+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:28:06.124343+0000","last_undegraded":"2023-07-13T22:28:06.124343+0000","last_fullsized":"2023-07-13T22:28:06.124343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:53:12.006344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148927+0000","last_change":"2023-07-13T22:28:05.142840+0000","last_active":"2023-07-13T22:28:05.148927+0000","last_peered":"2023-07-13T22:28:05.148927+0000","last_clean":"2023-07-13T22:28:05.148927+0000","last_became_active":"2023-07-13T22:28:05.136746+0000","last_became_peered":"2023-07-13T22:28:05.136746+0000","last_unstale":"2023-07-13T22:28:05.148927+0000","last_undegraded":"2023-07-13T22:28:05.148927+0000","last_fullsized":"2023-07-13T22:28:05.148927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:30.946948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.105118+0000","last_change":"2023-07-13T22:28:04.117787+0000","last_active":"2023-07-13T22:28:05.105118+0000","last_peered":"2023-07-13T22:28:05.105118+0000","last_clean":"2023-07-13T22:28:05.105118+0000","last_became_active":"2023-07-13T22:28:04.117533+0000","last_became_peered":"2023-07-13T22:28:04.117533+0000","last_unstale":"2023-07-13T22:28:05.105118+0000","last_undegraded":"2023-07-13T22:28:05.105118+0000","last_fullsized":"2023-07-13T22:28:05.105118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:34:33.528642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":130,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.121046+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:28:06.121046+0000","last_peered":"2023-07-13T22:28:06.121046+0000","last_clean":"2023-07-13T22:28:06.121046+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:28:06.121046+0000","last_undegraded":"2023-07-13T22:28:06.121046+0000","last_fullsized":"2023-07-13T22:28:06.121046+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:33:38.927032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.147636+0000","last_change":"2023-07-13T22:28:05.128837+0000","last_active":"2023-07-13T22:28:05.147636+0000","last_peered":"2023-07-13T22:28:05.147636+0000","last_clean":"2023-07-13T22:28:05.147636+0000","last_became_active":"2023-07-13T22:28:05.128516+0000","last_became_peered":"2023-07-13T22:28:05.128516+0000","last_unstale":"2023-07-13T22:28:05.147636+0000","last_undegraded":"2023-07-13T22:28:05.147636+0000","last_fullsized":"2023-07-13T22:28:05.147636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:03:51.828510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"86'1","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.129119+0000","last_change":"2023-07-13T22:28:04.122417+0000","last_active":"2023-07-13T22:28:05.129119+0000","last_peered":"2023-07-13T22:28:05.129119+0000","last_clean":"2023-07-13T22:28:05.129119+0000","last_became_active":"2023-07-13T22:28:04.122144+0000","last_became_peered":"2023-07-13T22:28:04.122144+0000","last_unstale":"2023-07-13T22:28:05.129119+0000","last_undegraded":"2023-07-13T22:28:05.129119+0000","last_fullsized":"2023-07-13T22:28:05.129119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:29.939335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117332+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:28:05.117332+0000","last_peered":"2023-07-13T22:28:05.117332+0000","last_clean":"2023-07-13T22:28:05.117332+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:28:05.117332+0000","last_undegraded":"2023-07-13T22:28:05.117332+0000","last_fullsized":"2023-07-13T22:28:05.117332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:46:47.107876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":280,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123229+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:28:06.123229+0000","last_peered":"2023-07-13T22:28:06.123229+0000","last_clean":"2023-07-13T22:28:06.123229+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:28:06.123229+0000","last_undegraded":"2023-07-13T22:28:06.123229+0000","last_fullsized":"2023-07-13T22:28:06.123229+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:07:42.856173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.128167+0000","last_change":"2023-07-13T22:28:05.122705+0000","last_active":"2023-07-13T22:28:05.128167+0000","last_peered":"2023-07-13T22:28:05.128167+0000","last_clean":"2023-07-13T22:28:05.128167+0000","last_became_active":"2023-07-13T22:28:05.122406+0000","last_became_peered":"2023-07-13T22:28:05.122406+0000","last_unstale":"2023-07-13T22:28:05.128167+0000","last_undegraded":"2023-07-13T22:28:05.128167+0000","last_fullsized":"2023-07-13T22:28:05.128167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:54:04.495802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"86'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123719+0000","last_change":"2023-07-13T22:28:04.120990+0000","last_active":"2023-07-13T22:28:06.123719+0000","last_peered":"2023-07-13T22:28:06.123719+0000","last_clean":"2023-07-13T22:28:06.123719+0000","last_became_active":"2023-07-13T22:28:04.120714+0000","last_became_peered":"2023-07-13T22:28:04.120714+0000","last_unstale":"2023-07-13T22:28:06.123719+0000","last_undegraded":"2023-07-13T22:28:06.123719+0000","last_fullsized":"2023-07-13T22:28:06.123719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:25:18.484423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123452+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:28:06.123452+0000","last_peered":"2023-07-13T22:28:06.123452+0000","last_clean":"2023-07-13T22:28:06.123452+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:28:06.123452+0000","last_undegraded":"2023-07-13T22:28:06.123452+0000","last_fullsized":"2023-07-13T22:28:06.123452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:48:19.989497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123991+0000","last_change":"2023-07-13T22:28:05.132005+0000","last_active":"2023-07-13T22:28:06.123991+0000","last_peered":"2023-07-13T22:28:06.123991+0000","last_clean":"2023-07-13T22:28:06.123991+0000","last_became_active":"2023-07-13T22:28:05.131702+0000","last_became_peered":"2023-07-13T22:28:05.131702+0000","last_unstale":"2023-07-13T22:28:06.123991+0000","last_undegraded":"2023-07-13T22:28:06.123991+0000","last_fullsized":"2023-07-13T22:28:06.123991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:57:44.701616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.122110+0000","last_change":"2023-07-13T22:28:04.120075+0000","last_active":"2023-07-13T22:28:05.122110+0000","last_peered":"2023-07-13T22:28:05.122110+0000","last_clean":"2023-07-13T22:28:05.122110+0000","last_became_active":"2023-07-13T22:28:04.119722+0000","last_became_peered":"2023-07-13T22:28:04.119722+0000","last_unstale":"2023-07-13T22:28:05.122110+0000","last_undegraded":"2023-07-13T22:28:05.122110+0000","last_fullsized":"2023-07-13T22:28:05.122110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:21:17.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123366+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:28:06.123366+0000","last_peered":"2023-07-13T22:28:06.123366+0000","last_clean":"2023-07-13T22:28:06.123366+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:28:06.123366+0000","last_undegraded":"2023-07-13T22:28:06.123366+0000","last_fullsized":"2023-07-13T22:28:06.123366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:57.880488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123754+0000","last_change":"2023-07-13T22:28:05.119032+0000","last_active":"2023-07-13T22:28:06.123754+0000","last_peered":"2023-07-13T22:28:06.123754+0000","last_clean":"2023-07-13T22:28:06.123754+0000","last_became_active":"2023-07-13T22:28:05.118688+0000","last_became_peered":"2023-07-13T22:28:05.118688+0000","last_unstale":"2023-07-13T22:28:06.123754+0000","last_undegraded":"2023-07-13T22:28:06.123754+0000","last_fullsized":"2023-07-13T22:28:06.123754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:05:35.372840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"86'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124532+0000","last_change":"2023-07-13T22:28:04.117910+0000","last_active":"2023-07-13T22:28:06.124532+0000","last_peered":"2023-07-13T22:28:06.124532+0000","last_clean":"2023-07-13T22:28:06.124532+0000","last_became_active":"2023-07-13T22:28:04.117618+0000","last_became_peered":"2023-07-13T22:28:04.117618+0000","last_unstale":"2023-07-13T22:28:06.124532+0000","last_undegraded":"2023-07-13T22:28:06.124532+0000","last_fullsized":"2023-07-13T22:28:06.124532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:54:55.243797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117924+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:28:05.117924+0000","last_peered":"2023-07-13T22:28:05.117924+0000","last_clean":"2023-07-13T22:28:05.117924+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:28:05.117924+0000","last_undegraded":"2023-07-13T22:28:05.117924+0000","last_fullsized":"2023-07-13T22:28:05.117924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:05:04.054792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.138922+0000","last_change":"2023-07-13T22:28:05.137879+0000","last_active":"2023-07-13T22:28:05.138922+0000","last_peered":"2023-07-13T22:28:05.138922+0000","last_clean":"2023-07-13T22:28:05.138922+0000","last_became_active":"2023-07-13T22:28:05.137568+0000","last_became_peered":"2023-07-13T22:28:05.137568+0000","last_unstale":"2023-07-13T22:28:05.138922+0000","last_undegraded":"2023-07-13T22:28:05.138922+0000","last_fullsized":"2023-07-13T22:28:05.138922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:01.584028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.105306+0000","last_change":"2023-07-13T22:28:04.116113+0000","last_active":"2023-07-13T22:28:05.105306+0000","last_peered":"2023-07-13T22:28:05.105306+0000","last_clean":"2023-07-13T22:28:05.105306+0000","last_became_active":"2023-07-13T22:28:04.115841+0000","last_became_peered":"2023-07-13T22:28:04.115841+0000","last_unstale":"2023-07-13T22:28:05.105306+0000","last_undegraded":"2023-07-13T22:28:05.105306+0000","last_fullsized":"2023-07-13T22:28:05.105306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:57:36.598845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":126,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.111956+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:28:05.111956+0000","last_peered":"2023-07-13T22:28:05.111956+0000","last_clean":"2023-07-13T22:28:05.111956+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:28:05.111956+0000","last_undegraded":"2023-07-13T22:28:05.111956+0000","last_fullsized":"2023-07-13T22:28:05.111956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:14:13.540273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148810+0000","last_change":"2023-07-13T22:28:05.145439+0000","last_active":"2023-07-13T22:28:05.148810+0000","last_peered":"2023-07-13T22:28:05.148810+0000","last_clean":"2023-07-13T22:28:05.148810+0000","last_became_active":"2023-07-13T22:28:05.145105+0000","last_became_peered":"2023-07-13T22:28:05.145105+0000","last_unstale":"2023-07-13T22:28:05.148810+0000","last_undegraded":"2023-07-13T22:28:05.148810+0000","last_fullsized":"2023-07-13T22:28:05.148810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:36:14.284898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"86'2","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.138679+0000","last_change":"2023-07-13T22:28:04.118114+0000","last_active":"2023-07-13T22:28:05.138679+0000","last_peered":"2023-07-13T22:28:05.138679+0000","last_clean":"2023-07-13T22:28:05.138679+0000","last_became_active":"2023-07-13T22:28:04.117849+0000","last_became_peered":"2023-07-13T22:28:04.117849+0000","last_unstale":"2023-07-13T22:28:05.138679+0000","last_undegraded":"2023-07-13T22:28:05.138679+0000","last_fullsized":"2023-07-13T22:28:05.138679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:34:57.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":126,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.114030+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:28:05.114030+0000","last_peered":"2023-07-13T22:28:05.114030+0000","last_clean":"2023-07-13T22:28:05.114030+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:28:05.114030+0000","last_undegraded":"2023-07-13T22:28:05.114030+0000","last_fullsized":"2023-07-13T22:28:05.114030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:46:16.297930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1658880,"data_stored":1068234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.311,"15min":0.311},"max":{"1min":0.98699999999999999,"5min":1.0780000000000001,"15min":1.115},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69799999999999995,"15min":0.67200000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.30099999999999999},"max":{"1min":0.97199999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66400000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63800000000000001,"15min":0.68799999999999994},"min":{"1min":0.42399999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94799999999999995,"15min":2.2610000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67300000000000004,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.34799999999999998,"15min":0.34300000000000003},"max":{"1min":0.873,"5min":1.0329999999999999,"15min":1.0609999999999999},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.78400000000000003},"min":{"1min":0.45600000000000002,"5min":0.443,"15min":0.40600000000000003},"max":{"1min":0.95099999999999996,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70299999999999996,"15min":0.77700000000000002},"min":{"1min":0.495,"5min":0.42999999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":1.107,"15min":3.4710000000000001},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69899999999999995,"15min":0.68799999999999994},"min":{"1min":0.501,"5min":0.45000000000000001,"15min":0.34599999999999997},"max":{"1min":0.98199999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.75800000000000001},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.01,"5min":1.024,"15min":1.1399999999999999},"last":0.755}]},{"osd":4,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.64200000000000002},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90100000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91500000000000004,"5min":1.0860000000000001,"15min":1.1850000000000001},"last":0.82999999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.66200000000000003},"min":{"1min":0.503,"5min":0.38200000000000001,"15min":0.35099999999999998},"max":{"1min":0.91500000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.79000000000000004},"min":{"1min":0.59899999999999998,"5min":0.498,"15min":0.48899999999999999},"max":{"1min":0.96499999999999997,"5min":1.1359999999999999,"15min":1.2110000000000001},"last":0.79500000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69099999999999995,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.437,"15min":0.39500000000000002},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.72199999999999998},"min":{"1min":0.47199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80000000000000004,"5min":0.98799999999999999,"15min":1.042},"last":0.63}]}]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1077248,"data_stored":479978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67100000000000004,"15min":0.628},"min":{"1min":0.42699999999999999,"5min":0.33100000000000002,"15min":0.28799999999999998},"max":{"1min":1.002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68999999999999995,"15min":0.628},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.33500000000000002},"max":{"1min":0.89700000000000002,"5min":0.97399999999999998,"15min":0.98899999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.505,"5min":0.39400000000000002,"15min":0.33900000000000002},"max":{"1min":0.94099999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73399999999999999,"15min":0.70299999999999996},"min":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.438},"max":{"1min":0.96899999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.67100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.91800000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69199999999999995,"15min":0.67000000000000004},"min":{"1min":0.56200000000000006,"5min":0.45600000000000002,"15min":0.35599999999999998},"max":{"1min":0.871,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70899999999999996,"15min":0.66300000000000003},"min":{"1min":0.56100000000000005,"5min":0.37,"15min":0.37},"max":{"1min":0.91600000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73599999999999999,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.95599999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.94499999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65500000000000003,"15min":0.68799999999999994},"min":{"1min":0.52200000000000002,"5min":0.40200000000000002,"15min":0.30399999999999999},"max":{"1min":0.997,"5min":0.998,"15min":0.998},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86899999999999999,"5min":0.94199999999999995,"15min":0.95999999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.73799999999999999,"15min":0.75800000000000001},"min":{"1min":0.54200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97399999999999998,"5min":1.05,"15min":1.05},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70699999999999996,"15min":0.745},"min":{"1min":0.46700000000000003,"5min":0.45600000000000002,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96999999999999997,"15min":1.1890000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":1.111},"last":0.48999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1654784,"data_stored":1068248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45000000000000001,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.249,"15min":0.249},"max":{"1min":0.70199999999999996,"5min":0.71599999999999997,"15min":1.952},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.496,"15min":0.51700000000000002},"min":{"1min":0.30599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":1.1060000000000001},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.29899999999999999,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":2.1200000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.35099999999999998},"max":{"1min":0.67400000000000004,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":2.008},"last":0.433},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53000000000000003,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.79000000000000004,"5min":1.054,"15min":1.054},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73899999999999999,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.57199999999999995,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":0.98199999999999998,"15min":2.028},"last":0.749},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.46400000000000002},"max":{"1min":0.94199999999999995,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.98899999999999999,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78000000000000003,"15min":0.84399999999999997},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.006,"5min":1.095,"15min":2.0899999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78600000000000003,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.55100000000000005,"15min":0.52600000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.2230000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.85799999999999998},"min":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0489999999999999,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79400000000000004,"15min":0.82699999999999996},"min":{"1min":0.68700000000000006,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":0.99199999999999999,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.59099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":1077248,"data_stored":479094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.52500000000000002},"min":{"1min":0.33500000000000002,"5min":0.30099999999999999,"15min":0.28499999999999998},"max":{"1min":0.84999999999999998,"5min":1.026,"15min":11.791},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.53400000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.874,"5min":1.419,"15min":4.657},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.432,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":1.1459999999999999,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52600000000000002,"15min":0.96699999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":1.276,"15min":13.398999999999999},"last":0.434}]},{"osd":3,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72999999999999998,"5min":0.89400000000000002,"15min":1.8089999999999999},"last":0.379},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56499999999999995,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.997,"15min":4.4530000000000003},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.747,"15min":1.218},"min":{"1min":0.44800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76000000000000001,"15min":0.80800000000000005},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.41299999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.776},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.78400000000000003,"15min":1.238},"min":{"1min":0.60399999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.073,"5min":1.113,"15min":13.349},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.82699999999999996,"15min":0.88},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.302,"15min":4.7649999999999997},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80000000000000004,"15min":0.83899999999999997},"min":{"1min":0.64800000000000002,"5min":0.52800000000000002,"15min":0.49099999999999999},"max":{"1min":1.212,"5min":1.212,"15min":4.5039999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":1.2090000000000001},"min":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1259999999999999,"5min":1.2330000000000001,"15min":11.679},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79200000000000004,"15min":1.2430000000000001},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.101,"5min":1.1759999999999999,"15min":13.298},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80100000000000005,"15min":0.89300000000000002},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.151,"5min":1.151,"15min":11.456},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":1052672,"data_stored":476824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44,"15min":0.53300000000000003},"min":{"1min":0.307,"5min":0.249,"15min":0.249},"max":{"1min":0.78300000000000003,"5min":1.1779999999999999,"15min":2.282},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.34799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64600000000000002,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.47399999999999998},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.64800000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51900000000000002,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":2.319},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":1.01,"15min":1.01},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.64300000000000002},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.38700000000000001},"max":{"1min":0.72899999999999998,"5min":0.95599999999999996,"15min":2.351},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.76200000000000001,"15min":0.83899999999999997},"min":{"1min":0.64000000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.165,"15min":2.238},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.58299999999999996,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":1.274,"15min":1.518},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.57099999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.997,"5min":1.1299999999999999,"15min":1.325},"last":0.748},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.80300000000000005,"15min":0.85799999999999998},"min":{"1min":0.66700000000000004,"5min":0.53500000000000003,"15min":0.495},"max":{"1min":0.97999999999999998,"5min":1.2490000000000001,"15min":2.1909999999999998},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.71099999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.026,"5min":1.1040000000000001,"15min":1.488},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.61599999999999999,"5min":0.502,"15min":0.48199999999999998},"max":{"1min":0.96499999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81599999999999995,"15min":0.79600000000000004},"min":{"1min":0.72299999999999998,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":0.98199999999999998,"5min":1.29,"15min":1.29},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.69899999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.046,"5min":1.254,"15min":1.254},"last":0.64900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36176,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335664,"statfs":{"total":96636764160,"available":96599719936,"internally_reserved":0,"allocated":1056768,"data_stored":477810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.41999999999999998,"15min":0.42399999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.48299999999999998,"15min":0.45300000000000001},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.27200000000000002},"max":{"1min":0.71099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.35599999999999998,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.378,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.71899999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73599999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56599999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.45800000000000002,"15min":0.44700000000000001},"max":{"1min":1.075,"5min":1.1699999999999999,"15min":1.2250000000000001},"last":0.752},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.014,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.96899999999999997,"5min":1.1919999999999999,"15min":1.411},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.998,"5min":1.1779999999999999,"15min":1.5469999999999999},"last":0.95299999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.52500000000000002,"15min":0.51800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.52100000000000002},"max":{"1min":0.93500000000000005,"5min":1.046,"15min":1.1240000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.80100000000000005},"min":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.498},"max":{"1min":1.0740000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78200000000000003,"15min":0.79100000000000004},"min":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.47199999999999998},"max":{"1min":0.93400000000000005,"5min":1.153,"15min":1.1950000000000001},"last":0.93400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":1064960,"data_stored":477970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.97299999999999998,"15min":1.0349999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86599999999999999,"5min":0.997,"15min":0.997},"last":0.84699999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":1.0489999999999999,"15min":1.298},"last":0.75},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.69499999999999995},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.438},"max":{"1min":0.85499999999999998,"5min":1.0049999999999999,"15min":1.3109999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.48199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.1200000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.71899999999999997},"min":{"1min":0.61799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94599999999999995,"5min":1.089,"15min":1.1839999999999999},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.95699999999999996},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.35899999999999999},"max":{"1min":0.83299999999999996,"5min":0.94999999999999996,"15min":10.657999999999999},"last":1.234},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72899999999999998,"15min":0.93600000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":8.75},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.46200000000000002,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":0.90800000000000003,"5min":1.042,"15min":1.042},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94899999999999995,"5min":1.0109999999999999,"15min":1.0429999999999999},"last":1.1759999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93000000000000005,"5min":1.0189999999999999,"15min":1.173},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.72299999999999998},"min":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97199999999999998,"5min":1.0509999999999999,"15min":1.099},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.004},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.753,"15min":0.77200000000000002},"min":{"1min":0.53200000000000003,"5min":0.497,"15min":0.47899999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.212},"last":1.0429999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36176,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335664,"statfs":{"total":96636764160,"available":96599719936,"internally_reserved":0,"allocated":1056768,"data_stored":477810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65800000000000003,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.33700000000000002},"max":{"1min":0.82499999999999996,"5min":1.042,"15min":1.042},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.64000000000000001},"min":{"1min":0.53500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.998,"15min":0.998},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.36299999999999999,"15min":0.33100000000000002},"max":{"1min":0.86799999999999999,"5min":8.9930000000000003,"15min":8.9930000000000003},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.63900000000000001},"min":{"1min":0.57299999999999995,"5min":0.38400000000000001,"15min":0.35899999999999999},"max":{"1min":0.86599999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.67500000000000004},"min":{"1min":0.57799999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72799999999999998,"15min":0.69499999999999995},"min":{"1min":0.57399999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.96599999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.68799999999999994},"min":{"1min":0.57599999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89500000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.623}]},{"osd":4,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81299999999999994,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90900000000000003,"5min":0.93500000000000005,"15min":0.995},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66300000000000003,"15min":0.66600000000000004},"min":{"1min":0.44,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.85399999999999998,"5min":0.98099999999999998,"15min":0.98499999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71999999999999997,"15min":0.72699999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.997,"15min":1.02},"last":0.754},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.76300000000000001},"min":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0669999999999999},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:28:09.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:28:09.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.802+0000 7fa238dff700 1 Processor -- start 2023-07-13T22:28:09.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.802+0000 7fa238dff700 1 -- start start 2023-07-13T22:28:09.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.803+0000 7fa238dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa2340b8570 0x7fa2340ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa238dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2340d35c0 0x7fa2340d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa238dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2340d09d0 con 0x7fa2340cf3f0 2023-07-13T22:28:09.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa238dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340d3b30 con 0x7fa2340b8570 2023-07-13T22:28:09.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa238dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340cda10 con 0x7fa2340d35c0 2023-07-13T22:28:09.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa232d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa2340cf3f0 0x7fa23415fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53736/0 (socket says 172.21.15.60:53736) 2023-07-13T22:28:09.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa232d9d700 1 -- 172.21.15.60:0/1275192158 learned_addr learned my addr 172.21.15.60:0/1275192158 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:09.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa23259c700 1 --2- 172.21.15.60:0/1275192158 >> v2:172.21.15.105:3300/0 conn(0x7fa2340b8570 0x7fa2340ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa231d9b700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2340d35c0 0x7fa2340d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232678696 0 0) 0x7fa2340d09d0 con 0x7fa2340cf3f0 2023-07-13T22:28:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.804+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa21c0035b0 con 0x7fa2340cf3f0 2023-07-13T22:28:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 149144025 0 0) 0x7fa21c0035b0 con 0x7fa2340cf3f0 2023-07-13T22:28:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2340d35c0 msgr2=0x7fa2340d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2340d35c0 0x7fa2340d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 >> v2:172.21.15.105:3300/0 conn(0x7fa2340b8570 msgr2=0x7fa2340ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 --2- 172.21.15.60:0/1275192158 >> v2:172.21.15.105:3300/0 conn(0x7fa2340b8570 0x7fa2340ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340cd450 con 0x7fa2340cf3f0 2023-07-13T22:28:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fa22c0019f0 con 0x7fa2340cf3f0 2023-07-13T22:28:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 >> v1:172.21.15.60:6789/0 conn(0x7fa2340cf3f0 legacy=0x7fa23415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2340d35c0 0x7fa21c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa21c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21c004d90 0x7fa21c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340d3b30 con 0x7fa21c003fd0 2023-07-13T22:28:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340cda10 con 0x7fa21c004d90 2023-07-13T22:28:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c00ba90 con 0x7fa2340d35c0 2023-07-13T22:28:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23259c700 1 --2- 172.21.15.60:0/1275192158 >> v2:172.21.15.105:3300/0 conn(0x7fa2340b8570 0x7fa2340ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:28:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa231d9b700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa21c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa232d9d700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21c004d90 0x7fa21c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.805+0000 7fa23259c700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2340d35c0 0x7fa21c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa231d9b700 1 -- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2340d35c0 msgr2=0x7fa21c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa231d9b700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2340d35c0 0x7fa21c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa231d9b700 1 -- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21c004d90 msgr2=0x7fa21c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa231d9b700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa21c004d90 0x7fa21c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa231d9b700 1 -- 172.21.15.60:0/1275192158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0035b0 con 0x7fa21c003fd0 2023-07-13T22:28:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa23259c700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2340d35c0 0x7fa21c003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa231d9b700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa21c004850 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa228009b60 tx=0x7fa21c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860e33210446ff97 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800e6c0 con 0x7fa21c003fd0 2023-07-13T22:28:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22800e820 con 0x7fa21c003fd0 2023-07-13T22:28:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.806+0000 7fa23159a700 1 -- 172.21.15.60:0/1275192158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800eb90 con 0x7fa21c003fd0 2023-07-13T22:28:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.807+0000 7fa238dff700 1 -- 172.21.15.60:0/1275192158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 msgr2=0x7fa21c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.807+0000 7fa238dff700 1 --2- 172.21.15.60:0/1275192158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa21c004850 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa228009b60 tx=0x7fa21c005880 comp rx=0 tx=0).stop 2023-07-13T22:28:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.807+0000 7fa232d9d700 1 -- 172.21.15.60:0/1275192158 reap_dead start 2023-07-13T22:28:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.807+0000 7fa238dff700 1 -- 172.21.15.60:0/1275192158 shutdown_connections 2023-07-13T22:28:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.807+0000 7fa238dff700 1 -- 172.21.15.60:0/1275192158 >> 172.21.15.60:0/1275192158 conn(0x7fa234071600 msgr2=0x7fa2340719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.807+0000 7fa238dff700 1 -- 172.21.15.60:0/1275192158 shutdown_connections 2023-07-13T22:28:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.807+0000 7fa238dff700 1 -- 172.21.15.60:0/1275192158 wait complete. 2023-07-13T22:28:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.809+0000 7fa238dff700 1 Processor -- start 2023-07-13T22:28:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.810+0000 7fa238dff700 1 -- start start 2023-07-13T22:28:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa238dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa234149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa238dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2341f1d30 0x7fa2341f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa238dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7230 0x7fa2340c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa238dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23414ae80 con 0x7fa21c003fd0 2023-07-13T22:28:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa238dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23414a310 con 0x7fa2340c7230 2023-07-13T22:28:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa238dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23414a570 con 0x7fa2341f1d30 2023-07-13T22:28:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa23259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa234149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa23259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa234149dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60970/0 (socket says 172.21.15.60:60970) 2023-07-13T22:28:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa23259c700 1 -- 172.21.15.60:0/2302295507 learned_addr learned my addr 172.21.15.60:0/2302295507 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa231d9b700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2341f1d30 0x7fa2341f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa232d9d700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7230 0x7fa2340c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa23259c700 1 -- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2341f1d30 msgr2=0x7fa2341f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa23259c700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2341f1d30 0x7fa2341f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa23259c700 1 -- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7230 msgr2=0x7fa2340c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa23259c700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7230 0x7fa2340c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa23259c700 1 -- 172.21.15.60:0/2302295507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2341f2eb0 con 0x7fa21c003fd0 2023-07-13T22:28:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa231d9b700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2341f1d30 0x7fa2341f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.811+0000 7fa232d9d700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7230 0x7fa2340c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.812+0000 7fa23259c700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa234149dd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa224011860 tx=0x7fa224011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.812+0000 7fa222ffd700 1 -- 172.21.15.60:0/2302295507 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400fae0 con 0x7fa21c003fd0 2023-07-13T22:28:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.812+0000 7fa222ffd700 1 -- 172.21.15.60:0/2302295507 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22400fc40 con 0x7fa21c003fd0 2023-07-13T22:28:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.812+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2341f26e0 con 0x7fa21c003fd0 2023-07-13T22:28:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.812+0000 7fa222ffd700 1 -- 172.21.15.60:0/2302295507 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224011e60 con 0x7fa21c003fd0 2023-07-13T22:28:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.812+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23414a7c0 con 0x7fa21c003fd0 2023-07-13T22:28:09.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.813+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2340cea90 con 0x7fa21c003fd0 2023-07-13T22:28:09.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.814+0000 7fa222ffd700 1 -- 172.21.15.60:0/2302295507 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa224015460 con 0x7fa21c003fd0 2023-07-13T22:28:09.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.815+0000 7fa222ffd700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa21406ace0 0x7fa21406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:09.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.815+0000 7fa222ffd700 1 -- 172.21.15.60:0/2302295507 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa224089e40 con 0x7fa21c003fd0 2023-07-13T22:28:09.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.815+0000 7fa231d9b700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa21406ace0 0x7fa21406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:09.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.816+0000 7fa231d9b700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa21406ace0 0x7fa21406d190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa228000c00 tx=0x7fa228008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:09.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:09.819+0000 7fa222ffd700 1 -- 172.21.15.60:0/2302295507 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa224054ee0 con 0x7fa21c003fd0 2023-07-13T22:28:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.017+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2340b5000 con 0x7fa21406ace0 2023-07-13T22:28:10.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.026+0000 7fa222ffd700 1 -- 172.21.15.60:0/2302295507 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311409 (secure 0 0 0) 0x7fa22800e990 con 0x7fa21406ace0 2023-07-13T22:28:10.010 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:10.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa21406ace0 msgr2=0x7fa21406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa21406ace0 0x7fa21406d190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa228000c00 tx=0x7fa228008030 comp rx=0 tx=0).stop 2023-07-13T22:28:10.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 msgr2=0x7fa234149dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa234149dd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa224011860 tx=0x7fa224011c20 comp rx=0 tx=0).stop 2023-07-13T22:28:10.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 shutdown_connections 2023-07-13T22:28:10.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa21406ace0 0x7fa21406d190 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c003fd0 0x7fa234149dd0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2340c7230 0x7fa2340c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 --2- 172.21.15.60:0/2302295507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2341f1d30 0x7fa2341f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 >> 172.21.15.60:0/2302295507 conn(0x7fa234071600 msgr2=0x7fa2340cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:10.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 shutdown_connections 2023-07-13T22:28:10.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.031+0000 7fa238dff700 1 -- 172.21.15.60:0/2302295507 wait complete. 2023-07-13T22:28:10.014 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:28:10.029 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2023-07-13T22:28:09.221784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278336,"kb_used_data":9472,"kb_used_omap":0,"kb_used_meta":268800,"kb_avail":754696384,"statfs":{"total":773094113280,"available":772809097216,"internally_reserved":0,"allocated":9699328,"data_stored":5005968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":275251200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001135"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.121062+0000","last_change":"2023-07-13T22:28:05.134097+0000","last_active":"2023-07-13T22:28:06.121062+0000","last_peered":"2023-07-13T22:28:06.121062+0000","last_clean":"2023-07-13T22:28:06.121062+0000","last_became_active":"2023-07-13T22:28:05.133812+0000","last_became_peered":"2023-07-13T22:28:05.133812+0000","last_unstale":"2023-07-13T22:28:06.121062+0000","last_undegraded":"2023-07-13T22:28:06.121062+0000","last_fullsized":"2023-07-13T22:28:06.121062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:34:47.472127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.135771+0000","last_change":"2023-07-13T22:28:05.134664+0000","last_active":"2023-07-13T22:28:05.135771+0000","last_peered":"2023-07-13T22:28:05.135771+0000","last_clean":"2023-07-13T22:28:05.135771+0000","last_became_active":"2023-07-13T22:28:05.134434+0000","last_became_peered":"2023-07-13T22:28:05.134434+0000","last_unstale":"2023-07-13T22:28:05.135771+0000","last_undegraded":"2023-07-13T22:28:05.135771+0000","last_fullsized":"2023-07-13T22:28:05.135771+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:03:51.828510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120849+0000","last_change":"2023-07-13T22:28:05.130472+0000","last_active":"2023-07-13T22:28:06.120849+0000","last_peered":"2023-07-13T22:28:06.120849+0000","last_clean":"2023-07-13T22:28:06.120849+0000","last_became_active":"2023-07-13T22:28:05.130175+0000","last_became_peered":"2023-07-13T22:28:05.130175+0000","last_unstale":"2023-07-13T22:28:06.120849+0000","last_undegraded":"2023-07-13T22:28:06.120849+0000","last_fullsized":"2023-07-13T22:28:06.120849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:49:21.735034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148352+0000","last_change":"2023-07-13T22:28:05.140817+0000","last_active":"2023-07-13T22:28:05.148352+0000","last_peered":"2023-07-13T22:28:05.148352+0000","last_clean":"2023-07-13T22:28:05.148352+0000","last_became_active":"2023-07-13T22:28:05.140026+0000","last_became_peered":"2023-07-13T22:28:05.140026+0000","last_unstale":"2023-07-13T22:28:05.148352+0000","last_undegraded":"2023-07-13T22:28:05.148352+0000","last_fullsized":"2023-07-13T22:28:05.148352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:16:57.958444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124213+0000","last_change":"2023-07-13T22:28:05.125775+0000","last_active":"2023-07-13T22:28:06.124213+0000","last_peered":"2023-07-13T22:28:06.124213+0000","last_clean":"2023-07-13T22:28:06.124213+0000","last_became_active":"2023-07-13T22:28:05.122355+0000","last_became_peered":"2023-07-13T22:28:05.122355+0000","last_unstale":"2023-07-13T22:28:06.124213+0000","last_undegraded":"2023-07-13T22:28:06.124213+0000","last_fullsized":"2023-07-13T22:28:06.124213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:34:48.309427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148761+0000","last_change":"2023-07-13T22:28:05.144223+0000","last_active":"2023-07-13T22:28:05.148761+0000","last_peered":"2023-07-13T22:28:05.148761+0000","last_clean":"2023-07-13T22:28:05.148761+0000","last_became_active":"2023-07-13T22:28:05.143588+0000","last_became_peered":"2023-07-13T22:28:05.143588+0000","last_unstale":"2023-07-13T22:28:05.148761+0000","last_undegraded":"2023-07-13T22:28:05.148761+0000","last_fullsized":"2023-07-13T22:28:05.148761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:30:21.714147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123366+0000","last_change":"2023-07-13T22:28:05.132165+0000","last_active":"2023-07-13T22:28:06.123366+0000","last_peered":"2023-07-13T22:28:06.123366+0000","last_clean":"2023-07-13T22:28:06.123366+0000","last_became_active":"2023-07-13T22:28:05.131792+0000","last_became_peered":"2023-07-13T22:28:05.131792+0000","last_unstale":"2023-07-13T22:28:06.123366+0000","last_undegraded":"2023-07-13T22:28:06.123366+0000","last_fullsized":"2023-07-13T22:28:06.123366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:31:33.102873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.143141+0000","last_change":"2023-07-13T22:28:05.131662+0000","last_active":"2023-07-13T22:28:05.143141+0000","last_peered":"2023-07-13T22:28:05.143141+0000","last_clean":"2023-07-13T22:28:05.143141+0000","last_became_active":"2023-07-13T22:28:05.131414+0000","last_became_peered":"2023-07-13T22:28:05.131414+0000","last_unstale":"2023-07-13T22:28:05.143141+0000","last_undegraded":"2023-07-13T22:28:05.143141+0000","last_fullsized":"2023-07-13T22:28:05.143141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:15:48.515310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123575+0000","last_change":"2023-07-13T22:28:05.125450+0000","last_active":"2023-07-13T22:28:06.123575+0000","last_peered":"2023-07-13T22:28:06.123575+0000","last_clean":"2023-07-13T22:28:06.123575+0000","last_became_active":"2023-07-13T22:28:05.125113+0000","last_became_peered":"2023-07-13T22:28:05.125113+0000","last_unstale":"2023-07-13T22:28:06.123575+0000","last_undegraded":"2023-07-13T22:28:06.123575+0000","last_fullsized":"2023-07-13T22:28:06.123575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:30:55.850197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.147805+0000","last_change":"2023-07-13T22:28:05.141771+0000","last_active":"2023-07-13T22:28:05.147805+0000","last_peered":"2023-07-13T22:28:05.147805+0000","last_clean":"2023-07-13T22:28:05.147805+0000","last_became_active":"2023-07-13T22:28:05.141447+0000","last_became_peered":"2023-07-13T22:28:05.141447+0000","last_unstale":"2023-07-13T22:28:05.147805+0000","last_undegraded":"2023-07-13T22:28:05.147805+0000","last_fullsized":"2023-07-13T22:28:05.147805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:50:02.888373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124894+0000","last_change":"2023-07-13T22:28:05.128026+0000","last_active":"2023-07-13T22:28:06.124894+0000","last_peered":"2023-07-13T22:28:06.124894+0000","last_clean":"2023-07-13T22:28:06.124894+0000","last_became_active":"2023-07-13T22:28:05.127549+0000","last_became_peered":"2023-07-13T22:28:05.127549+0000","last_unstale":"2023-07-13T22:28:06.124894+0000","last_undegraded":"2023-07-13T22:28:06.124894+0000","last_fullsized":"2023-07-13T22:28:06.124894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:32:42.267626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.140457+0000","last_change":"2023-07-13T22:28:05.135103+0000","last_active":"2023-07-13T22:28:05.140457+0000","last_peered":"2023-07-13T22:28:05.140457+0000","last_clean":"2023-07-13T22:28:05.140457+0000","last_became_active":"2023-07-13T22:28:05.134690+0000","last_became_peered":"2023-07-13T22:28:05.134690+0000","last_unstale":"2023-07-13T22:28:05.140457+0000","last_undegraded":"2023-07-13T22:28:05.140457+0000","last_fullsized":"2023-07-13T22:28:05.140457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:02:10.808735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.139143+0000","last_change":"2023-07-13T22:28:05.134778+0000","last_active":"2023-07-13T22:28:05.139143+0000","last_peered":"2023-07-13T22:28:05.139143+0000","last_clean":"2023-07-13T22:28:05.139143+0000","last_became_active":"2023-07-13T22:28:05.134498+0000","last_became_peered":"2023-07-13T22:28:05.134498+0000","last_unstale":"2023-07-13T22:28:05.139143+0000","last_undegraded":"2023-07-13T22:28:05.139143+0000","last_fullsized":"2023-07-13T22:28:05.139143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:59.410349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123799+0000","last_change":"2023-07-13T22:28:05.136940+0000","last_active":"2023-07-13T22:28:06.123799+0000","last_peered":"2023-07-13T22:28:06.123799+0000","last_clean":"2023-07-13T22:28:06.123799+0000","last_became_active":"2023-07-13T22:28:05.136666+0000","last_became_peered":"2023-07-13T22:28:05.136666+0000","last_unstale":"2023-07-13T22:28:06.123799+0000","last_undegraded":"2023-07-13T22:28:06.123799+0000","last_fullsized":"2023-07-13T22:28:06.123799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:40:30.225003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148424+0000","last_change":"2023-07-13T22:28:05.141803+0000","last_active":"2023-07-13T22:28:05.148424+0000","last_peered":"2023-07-13T22:28:05.148424+0000","last_clean":"2023-07-13T22:28:05.148424+0000","last_became_active":"2023-07-13T22:28:05.141480+0000","last_became_peered":"2023-07-13T22:28:05.141480+0000","last_unstale":"2023-07-13T22:28:05.148424+0000","last_undegraded":"2023-07-13T22:28:05.148424+0000","last_fullsized":"2023-07-13T22:28:05.148424+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:30:17.292002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148500+0000","last_change":"2023-07-13T22:28:05.138725+0000","last_active":"2023-07-13T22:28:05.148500+0000","last_peered":"2023-07-13T22:28:05.148500+0000","last_clean":"2023-07-13T22:28:05.148500+0000","last_became_active":"2023-07-13T22:28:05.138011+0000","last_became_peered":"2023-07-13T22:28:05.138011+0000","last_unstale":"2023-07-13T22:28:05.148500+0000","last_undegraded":"2023-07-13T22:28:05.148500+0000","last_fullsized":"2023-07-13T22:28:05.148500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:11.314965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148732+0000","last_change":"2023-07-13T22:28:05.141932+0000","last_active":"2023-07-13T22:28:05.148732+0000","last_peered":"2023-07-13T22:28:05.148732+0000","last_clean":"2023-07-13T22:28:05.148732+0000","last_became_active":"2023-07-13T22:28:05.141649+0000","last_became_peered":"2023-07-13T22:28:05.141649+0000","last_unstale":"2023-07-13T22:28:05.148732+0000","last_undegraded":"2023-07-13T22:28:05.148732+0000","last_fullsized":"2023-07-13T22:28:05.148732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:58:09.166380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.121208+0000","last_change":"2023-07-13T22:28:05.132456+0000","last_active":"2023-07-13T22:28:06.121208+0000","last_peered":"2023-07-13T22:28:06.121208+0000","last_clean":"2023-07-13T22:28:06.121208+0000","last_became_active":"2023-07-13T22:28:05.131979+0000","last_became_peered":"2023-07-13T22:28:05.131979+0000","last_unstale":"2023-07-13T22:28:06.121208+0000","last_undegraded":"2023-07-13T22:28:06.121208+0000","last_fullsized":"2023-07-13T22:28:06.121208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:27:07.969392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124584+0000","last_change":"2023-07-13T22:28:05.131909+0000","last_active":"2023-07-13T22:28:06.124584+0000","last_peered":"2023-07-13T22:28:06.124584+0000","last_clean":"2023-07-13T22:28:06.124584+0000","last_became_active":"2023-07-13T22:28:05.131555+0000","last_became_peered":"2023-07-13T22:28:05.131555+0000","last_unstale":"2023-07-13T22:28:06.124584+0000","last_undegraded":"2023-07-13T22:28:06.124584+0000","last_fullsized":"2023-07-13T22:28:06.124584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:08:35.046700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148391+0000","last_change":"2023-07-13T22:28:05.141090+0000","last_active":"2023-07-13T22:28:05.148391+0000","last_peered":"2023-07-13T22:28:05.148391+0000","last_clean":"2023-07-13T22:28:05.148391+0000","last_became_active":"2023-07-13T22:28:05.140529+0000","last_became_peered":"2023-07-13T22:28:05.140529+0000","last_unstale":"2023-07-13T22:28:05.148391+0000","last_undegraded":"2023-07-13T22:28:05.148391+0000","last_fullsized":"2023-07-13T22:28:05.148391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:02:11.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124206+0000","last_change":"2023-07-13T22:28:05.113249+0000","last_active":"2023-07-13T22:28:06.124206+0000","last_peered":"2023-07-13T22:28:06.124206+0000","last_clean":"2023-07-13T22:28:06.124206+0000","last_became_active":"2023-07-13T22:28:05.112862+0000","last_became_peered":"2023-07-13T22:28:05.112862+0000","last_unstale":"2023-07-13T22:28:06.124206+0000","last_undegraded":"2023-07-13T22:28:06.124206+0000","last_fullsized":"2023-07-13T22:28:06.124206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:56:23.463078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123949+0000","last_change":"2023-07-13T22:28:05.136898+0000","last_active":"2023-07-13T22:28:06.123949+0000","last_peered":"2023-07-13T22:28:06.123949+0000","last_clean":"2023-07-13T22:28:06.123949+0000","last_became_active":"2023-07-13T22:28:05.136605+0000","last_became_peered":"2023-07-13T22:28:05.136605+0000","last_unstale":"2023-07-13T22:28:06.123949+0000","last_undegraded":"2023-07-13T22:28:06.123949+0000","last_fullsized":"2023-07-13T22:28:06.123949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:33:57.059183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.137763+0000","last_change":"2023-07-13T22:28:05.136943+0000","last_active":"2023-07-13T22:28:05.137763+0000","last_peered":"2023-07-13T22:28:05.137763+0000","last_clean":"2023-07-13T22:28:05.137763+0000","last_became_active":"2023-07-13T22:28:05.136593+0000","last_became_peered":"2023-07-13T22:28:05.136593+0000","last_unstale":"2023-07-13T22:28:05.137763+0000","last_undegraded":"2023-07-13T22:28:05.137763+0000","last_fullsized":"2023-07-13T22:28:05.137763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:14:41.780622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120734+0000","last_change":"2023-07-13T22:28:05.131146+0000","last_active":"2023-07-13T22:28:06.120734+0000","last_peered":"2023-07-13T22:28:06.120734+0000","last_clean":"2023-07-13T22:28:06.120734+0000","last_became_active":"2023-07-13T22:28:05.130854+0000","last_became_peered":"2023-07-13T22:28:05.130854+0000","last_unstale":"2023-07-13T22:28:06.120734+0000","last_undegraded":"2023-07-13T22:28:06.120734+0000","last_fullsized":"2023-07-13T22:28:06.120734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:34:20.015490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.130674+0000","last_change":"2023-07-13T22:28:05.129899+0000","last_active":"2023-07-13T22:28:05.130674+0000","last_peered":"2023-07-13T22:28:05.130674+0000","last_clean":"2023-07-13T22:28:05.130674+0000","last_became_active":"2023-07-13T22:28:05.129555+0000","last_became_peered":"2023-07-13T22:28:05.129555+0000","last_unstale":"2023-07-13T22:28:05.130674+0000","last_undegraded":"2023-07-13T22:28:05.130674+0000","last_fullsized":"2023-07-13T22:28:05.130674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:54.170522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123885+0000","last_change":"2023-07-13T22:28:05.140369+0000","last_active":"2023-07-13T22:28:06.123885+0000","last_peered":"2023-07-13T22:28:06.123885+0000","last_clean":"2023-07-13T22:28:06.123885+0000","last_became_active":"2023-07-13T22:28:05.140005+0000","last_became_peered":"2023-07-13T22:28:05.140005+0000","last_unstale":"2023-07-13T22:28:06.123885+0000","last_undegraded":"2023-07-13T22:28:06.123885+0000","last_fullsized":"2023-07-13T22:28:06.123885+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:14:18.498632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.133833+0000","last_change":"2023-07-13T22:28:05.125174+0000","last_active":"2023-07-13T22:28:05.133833+0000","last_peered":"2023-07-13T22:28:05.133833+0000","last_clean":"2023-07-13T22:28:05.133833+0000","last_became_active":"2023-07-13T22:28:05.124145+0000","last_became_peered":"2023-07-13T22:28:05.124145+0000","last_unstale":"2023-07-13T22:28:05.133833+0000","last_undegraded":"2023-07-13T22:28:05.133833+0000","last_fullsized":"2023-07-13T22:28:05.133833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:07:09.664684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.139788+0000","last_change":"2023-07-13T22:28:05.132304+0000","last_active":"2023-07-13T22:28:05.139788+0000","last_peered":"2023-07-13T22:28:05.139788+0000","last_clean":"2023-07-13T22:28:05.139788+0000","last_became_active":"2023-07-13T22:28:05.131985+0000","last_became_peered":"2023-07-13T22:28:05.131985+0000","last_unstale":"2023-07-13T22:28:05.139788+0000","last_undegraded":"2023-07-13T22:28:05.139788+0000","last_fullsized":"2023-07-13T22:28:05.139788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:24:35.465349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123614+0000","last_change":"2023-07-13T22:28:05.116319+0000","last_active":"2023-07-13T22:28:06.123614+0000","last_peered":"2023-07-13T22:28:06.123614+0000","last_clean":"2023-07-13T22:28:06.123614+0000","last_became_active":"2023-07-13T22:28:05.116001+0000","last_became_peered":"2023-07-13T22:28:05.116001+0000","last_unstale":"2023-07-13T22:28:06.123614+0000","last_undegraded":"2023-07-13T22:28:06.123614+0000","last_fullsized":"2023-07-13T22:28:06.123614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:41:26.305788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124141+0000","last_change":"2023-07-13T22:28:05.125706+0000","last_active":"2023-07-13T22:28:06.124141+0000","last_peered":"2023-07-13T22:28:06.124141+0000","last_clean":"2023-07-13T22:28:06.124141+0000","last_became_active":"2023-07-13T22:28:05.124797+0000","last_became_peered":"2023-07-13T22:28:05.124797+0000","last_unstale":"2023-07-13T22:28:06.124141+0000","last_undegraded":"2023-07-13T22:28:06.124141+0000","last_fullsized":"2023-07-13T22:28:06.124141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:49:23.912113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.142878+0000","last_change":"2023-07-13T22:28:05.141555+0000","last_active":"2023-07-13T22:28:05.142878+0000","last_peered":"2023-07-13T22:28:05.142878+0000","last_clean":"2023-07-13T22:28:05.142878+0000","last_became_active":"2023-07-13T22:28:05.141252+0000","last_became_peered":"2023-07-13T22:28:05.141252+0000","last_unstale":"2023-07-13T22:28:05.142878+0000","last_undegraded":"2023-07-13T22:28:05.142878+0000","last_fullsized":"2023-07-13T22:28:05.142878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:25:43.756134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148523+0000","last_change":"2023-07-13T22:28:05.138929+0000","last_active":"2023-07-13T22:28:05.148523+0000","last_peered":"2023-07-13T22:28:05.148523+0000","last_clean":"2023-07-13T22:28:05.148523+0000","last_became_active":"2023-07-13T22:28:05.138459+0000","last_became_peered":"2023-07-13T22:28:05.138459+0000","last_unstale":"2023-07-13T22:28:05.148523+0000","last_undegraded":"2023-07-13T22:28:05.148523+0000","last_fullsized":"2023-07-13T22:28:05.148523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:38:27.482861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.132402+0000","last_change":"2023-07-13T22:28:05.128710+0000","last_active":"2023-07-13T22:28:05.132402+0000","last_peered":"2023-07-13T22:28:05.132402+0000","last_clean":"2023-07-13T22:28:05.132402+0000","last_became_active":"2023-07-13T22:28:05.128292+0000","last_became_peered":"2023-07-13T22:28:05.128292+0000","last_unstale":"2023-07-13T22:28:05.132402+0000","last_undegraded":"2023-07-13T22:28:05.132402+0000","last_fullsized":"2023-07-13T22:28:05.132402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:53:33.685855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"86'4","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.128241+0000","last_change":"2023-07-13T22:28:04.124865+0000","last_active":"2023-07-13T22:28:05.128241+0000","last_peered":"2023-07-13T22:28:05.128241+0000","last_clean":"2023-07-13T22:28:05.128241+0000","last_became_active":"2023-07-13T22:28:04.124456+0000","last_became_peered":"2023-07-13T22:28:04.124456+0000","last_unstale":"2023-07-13T22:28:05.128241+0000","last_undegraded":"2023-07-13T22:28:05.128241+0000","last_fullsized":"2023-07-13T22:28:05.128241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:07:41.535542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148560+0000","last_change":"2023-07-13T22:28:05.139945+0000","last_active":"2023-07-13T22:28:05.148560+0000","last_peered":"2023-07-13T22:28:05.148560+0000","last_clean":"2023-07-13T22:28:05.148560+0000","last_became_active":"2023-07-13T22:28:05.139688+0000","last_became_peered":"2023-07-13T22:28:05.139688+0000","last_unstale":"2023-07-13T22:28:05.148560+0000","last_undegraded":"2023-07-13T22:28:05.148560+0000","last_fullsized":"2023-07-13T22:28:05.148560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:43:26.500224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"86'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124016+0000","last_change":"2023-07-13T22:28:04.122784+0000","last_active":"2023-07-13T22:28:06.124016+0000","last_peered":"2023-07-13T22:28:06.124016+0000","last_clean":"2023-07-13T22:28:06.124016+0000","last_became_active":"2023-07-13T22:28:04.122526+0000","last_became_peered":"2023-07-13T22:28:04.122526+0000","last_unstale":"2023-07-13T22:28:06.124016+0000","last_undegraded":"2023-07-13T22:28:06.124016+0000","last_fullsized":"2023-07-13T22:28:06.124016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:30:52.863628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123581+0000","last_change":"2023-07-13T22:28:05.140647+0000","last_active":"2023-07-13T22:28:06.123581+0000","last_peered":"2023-07-13T22:28:06.123581+0000","last_clean":"2023-07-13T22:28:06.123581+0000","last_became_active":"2023-07-13T22:28:05.140359+0000","last_became_peered":"2023-07-13T22:28:05.140359+0000","last_unstale":"2023-07-13T22:28:06.123581+0000","last_undegraded":"2023-07-13T22:28:06.123581+0000","last_fullsized":"2023-07-13T22:28:06.123581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:22:43.078094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"86'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123954+0000","last_change":"2023-07-13T22:28:04.122417+0000","last_active":"2023-07-13T22:28:06.123954+0000","last_peered":"2023-07-13T22:28:06.123954+0000","last_clean":"2023-07-13T22:28:06.123954+0000","last_became_active":"2023-07-13T22:28:04.122170+0000","last_became_peered":"2023-07-13T22:28:04.122170+0000","last_unstale":"2023-07-13T22:28:06.123954+0000","last_undegraded":"2023-07-13T22:28:06.123954+0000","last_fullsized":"2023-07-13T22:28:06.123954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:07:21.817229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.147735+0000","last_change":"2023-07-13T22:28:05.131812+0000","last_active":"2023-07-13T22:28:05.147735+0000","last_peered":"2023-07-13T22:28:05.147735+0000","last_clean":"2023-07-13T22:28:05.147735+0000","last_became_active":"2023-07-13T22:28:05.131533+0000","last_became_peered":"2023-07-13T22:28:05.131533+0000","last_unstale":"2023-07-13T22:28:05.147735+0000","last_undegraded":"2023-07-13T22:28:05.147735+0000","last_fullsized":"2023-07-13T22:28:05.147735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:39:09.576155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"86'2","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.146172+0000","last_change":"2023-07-13T22:28:04.125078+0000","last_active":"2023-07-13T22:28:05.146172+0000","last_peered":"2023-07-13T22:28:05.146172+0000","last_clean":"2023-07-13T22:28:05.146172+0000","last_became_active":"2023-07-13T22:28:04.124530+0000","last_became_peered":"2023-07-13T22:28:04.124530+0000","last_unstale":"2023-07-13T22:28:05.146172+0000","last_undegraded":"2023-07-13T22:28:05.146172+0000","last_fullsized":"2023-07-13T22:28:05.146172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:37.461212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148620+0000","last_change":"2023-07-13T22:28:05.144236+0000","last_active":"2023-07-13T22:28:05.148620+0000","last_peered":"2023-07-13T22:28:05.148620+0000","last_clean":"2023-07-13T22:28:05.148620+0000","last_became_active":"2023-07-13T22:28:05.143719+0000","last_became_peered":"2023-07-13T22:28:05.143719+0000","last_unstale":"2023-07-13T22:28:05.148620+0000","last_undegraded":"2023-07-13T22:28:05.148620+0000","last_fullsized":"2023-07-13T22:28:05.148620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:14:41.780622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"86'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124502+0000","last_change":"2023-07-13T22:28:04.116193+0000","last_active":"2023-07-13T22:28:06.124502+0000","last_peered":"2023-07-13T22:28:06.124502+0000","last_clean":"2023-07-13T22:28:06.124502+0000","last_became_active":"2023-07-13T22:28:04.115912+0000","last_became_peered":"2023-07-13T22:28:04.115912+0000","last_unstale":"2023-07-13T22:28:06.124502+0000","last_undegraded":"2023-07-13T22:28:06.124502+0000","last_fullsized":"2023-07-13T22:28:06.124502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:58:40.075593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.130162+0000","last_change":"2023-07-13T22:28:05.127334+0000","last_active":"2023-07-13T22:28:05.130162+0000","last_peered":"2023-07-13T22:28:05.130162+0000","last_clean":"2023-07-13T22:28:05.130162+0000","last_became_active":"2023-07-13T22:28:05.126970+0000","last_became_peered":"2023-07-13T22:28:05.126970+0000","last_unstale":"2023-07-13T22:28:05.130162+0000","last_undegraded":"2023-07-13T22:28:05.130162+0000","last_fullsized":"2023-07-13T22:28:05.130162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:25:19.873884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.140419+0000","last_change":"2023-07-13T22:28:04.113585+0000","last_active":"2023-07-13T22:28:05.140419+0000","last_peered":"2023-07-13T22:28:05.140419+0000","last_clean":"2023-07-13T22:28:05.140419+0000","last_became_active":"2023-07-13T22:28:04.113181+0000","last_became_peered":"2023-07-13T22:28:04.113181+0000","last_unstale":"2023-07-13T22:28:05.140419+0000","last_undegraded":"2023-07-13T22:28:05.140419+0000","last_fullsized":"2023-07-13T22:28:05.140419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:31:45.581771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148785+0000","last_change":"2023-07-13T22:28:05.144347+0000","last_active":"2023-07-13T22:28:05.148785+0000","last_peered":"2023-07-13T22:28:05.148785+0000","last_clean":"2023-07-13T22:28:05.148785+0000","last_became_active":"2023-07-13T22:28:05.143940+0000","last_became_peered":"2023-07-13T22:28:05.143940+0000","last_unstale":"2023-07-13T22:28:05.148785+0000","last_undegraded":"2023-07-13T22:28:05.148785+0000","last_fullsized":"2023-07-13T22:28:05.148785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:48:12.036002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120873+0000","last_change":"2023-07-13T22:28:04.123606+0000","last_active":"2023-07-13T22:28:06.120873+0000","last_peered":"2023-07-13T22:28:06.120873+0000","last_clean":"2023-07-13T22:28:06.120873+0000","last_became_active":"2023-07-13T22:28:04.122467+0000","last_became_peered":"2023-07-13T22:28:04.122467+0000","last_unstale":"2023-07-13T22:28:06.120873+0000","last_undegraded":"2023-07-13T22:28:06.120873+0000","last_fullsized":"2023-07-13T22:28:06.120873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:00:29.399267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.139120+0000","last_change":"2023-07-13T22:28:05.130768+0000","last_active":"2023-07-13T22:28:05.139120+0000","last_peered":"2023-07-13T22:28:05.139120+0000","last_clean":"2023-07-13T22:28:05.139120+0000","last_became_active":"2023-07-13T22:28:05.130496+0000","last_became_peered":"2023-07-13T22:28:05.130496+0000","last_unstale":"2023-07-13T22:28:05.139120+0000","last_undegraded":"2023-07-13T22:28:05.139120+0000","last_fullsized":"2023-07-13T22:28:05.139120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:49:26.925461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123109+0000","last_change":"2023-07-13T22:28:04.121114+0000","last_active":"2023-07-13T22:28:06.123109+0000","last_peered":"2023-07-13T22:28:06.123109+0000","last_clean":"2023-07-13T22:28:06.123109+0000","last_became_active":"2023-07-13T22:28:04.120764+0000","last_became_peered":"2023-07-13T22:28:04.120764+0000","last_unstale":"2023-07-13T22:28:06.123109+0000","last_undegraded":"2023-07-13T22:28:06.123109+0000","last_fullsized":"2023-07-13T22:28:06.123109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:34:12.381021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123603+0000","last_change":"2023-07-13T22:28:05.135296+0000","last_active":"2023-07-13T22:28:06.123603+0000","last_peered":"2023-07-13T22:28:06.123603+0000","last_clean":"2023-07-13T22:28:06.123603+0000","last_became_active":"2023-07-13T22:28:05.134903+0000","last_became_peered":"2023-07-13T22:28:05.134903+0000","last_unstale":"2023-07-13T22:28:06.123603+0000","last_undegraded":"2023-07-13T22:28:06.123603+0000","last_fullsized":"2023-07-13T22:28:06.123603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:51:58.047835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117509+0000","last_change":"2023-07-13T22:28:04.116318+0000","last_active":"2023-07-13T22:28:05.117509+0000","last_peered":"2023-07-13T22:28:05.117509+0000","last_clean":"2023-07-13T22:28:05.117509+0000","last_became_active":"2023-07-13T22:28:04.116032+0000","last_became_peered":"2023-07-13T22:28:04.116032+0000","last_unstale":"2023-07-13T22:28:05.117509+0000","last_undegraded":"2023-07-13T22:28:05.117509+0000","last_fullsized":"2023-07-13T22:28:05.117509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:04:54.440821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123833+0000","last_change":"2023-07-13T22:28:05.116979+0000","last_active":"2023-07-13T22:28:06.123833+0000","last_peered":"2023-07-13T22:28:06.123833+0000","last_clean":"2023-07-13T22:28:06.123833+0000","last_became_active":"2023-07-13T22:28:05.116711+0000","last_became_peered":"2023-07-13T22:28:05.116711+0000","last_unstale":"2023-07-13T22:28:06.123833+0000","last_undegraded":"2023-07-13T22:28:06.123833+0000","last_fullsized":"2023-07-13T22:28:06.123833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:22:52.742728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.121246+0000","last_change":"2023-07-13T22:28:04.121322+0000","last_active":"2023-07-13T22:28:06.121246+0000","last_peered":"2023-07-13T22:28:06.121246+0000","last_clean":"2023-07-13T22:28:06.121246+0000","last_became_active":"2023-07-13T22:28:04.121033+0000","last_became_peered":"2023-07-13T22:28:04.121033+0000","last_unstale":"2023-07-13T22:28:06.121246+0000","last_undegraded":"2023-07-13T22:28:06.121246+0000","last_fullsized":"2023-07-13T22:28:06.121246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:57:55.847128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120964+0000","last_change":"2023-07-13T22:28:05.132267+0000","last_active":"2023-07-13T22:28:06.120964+0000","last_peered":"2023-07-13T22:28:06.120964+0000","last_clean":"2023-07-13T22:28:06.120964+0000","last_became_active":"2023-07-13T22:28:05.131658+0000","last_became_peered":"2023-07-13T22:28:05.131658+0000","last_unstale":"2023-07-13T22:28:06.120964+0000","last_undegraded":"2023-07-13T22:28:06.120964+0000","last_fullsized":"2023-07-13T22:28:06.120964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:50:24.483224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"86'3","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124760+0000","last_change":"2023-07-13T22:28:04.120443+0000","last_active":"2023-07-13T22:28:06.124760+0000","last_peered":"2023-07-13T22:28:06.124760+0000","last_clean":"2023-07-13T22:28:06.124760+0000","last_became_active":"2023-07-13T22:28:04.120096+0000","last_became_peered":"2023-07-13T22:28:04.120096+0000","last_unstale":"2023-07-13T22:28:06.124760+0000","last_undegraded":"2023-07-13T22:28:06.124760+0000","last_fullsized":"2023-07-13T22:28:06.124760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:03:05.434656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.140502+0000","last_change":"2023-07-13T22:28:05.131997+0000","last_active":"2023-07-13T22:28:05.140502+0000","last_peered":"2023-07-13T22:28:05.140502+0000","last_clean":"2023-07-13T22:28:05.140502+0000","last_became_active":"2023-07-13T22:28:05.131742+0000","last_became_peered":"2023-07-13T22:28:05.131742+0000","last_unstale":"2023-07-13T22:28:05.140502+0000","last_undegraded":"2023-07-13T22:28:05.140502+0000","last_fullsized":"2023-07-13T22:28:05.140502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:39:44.607563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"86'6","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.150872+0000","last_change":"2023-07-13T22:28:04.125990+0000","last_active":"2023-07-13T22:28:05.150872+0000","last_peered":"2023-07-13T22:28:05.150872+0000","last_clean":"2023-07-13T22:28:05.150872+0000","last_became_active":"2023-07-13T22:28:04.125639+0000","last_became_peered":"2023-07-13T22:28:04.125639+0000","last_unstale":"2023-07-13T22:28:05.150872+0000","last_undegraded":"2023-07-13T22:28:05.150872+0000","last_fullsized":"2023-07-13T22:28:05.150872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:19:30.990209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148601+0000","last_change":"2023-07-13T22:28:05.144071+0000","last_active":"2023-07-13T22:28:05.148601+0000","last_peered":"2023-07-13T22:28:05.148601+0000","last_clean":"2023-07-13T22:28:05.148601+0000","last_became_active":"2023-07-13T22:28:05.143311+0000","last_became_peered":"2023-07-13T22:28:05.143311+0000","last_unstale":"2023-07-13T22:28:05.148601+0000","last_undegraded":"2023-07-13T22:28:05.148601+0000","last_fullsized":"2023-07-13T22:28:05.148601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:52:25.475987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"86'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123779+0000","last_change":"2023-07-13T22:28:04.121584+0000","last_active":"2023-07-13T22:28:06.123779+0000","last_peered":"2023-07-13T22:28:06.123779+0000","last_clean":"2023-07-13T22:28:06.123779+0000","last_became_active":"2023-07-13T22:28:04.121300+0000","last_became_peered":"2023-07-13T22:28:04.121300+0000","last_unstale":"2023-07-13T22:28:06.123779+0000","last_undegraded":"2023-07-13T22:28:06.123779+0000","last_fullsized":"2023-07-13T22:28:06.123779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:55:31.018514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123954+0000","last_change":"2023-07-13T22:28:05.120159+0000","last_active":"2023-07-13T22:28:06.123954+0000","last_peered":"2023-07-13T22:28:06.123954+0000","last_clean":"2023-07-13T22:28:06.123954+0000","last_became_active":"2023-07-13T22:28:05.119840+0000","last_became_peered":"2023-07-13T22:28:05.119840+0000","last_unstale":"2023-07-13T22:28:06.123954+0000","last_undegraded":"2023-07-13T22:28:06.123954+0000","last_fullsized":"2023-07-13T22:28:06.123954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:02:51.152705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.137152+0000","last_change":"2023-07-13T22:28:04.123344+0000","last_active":"2023-07-13T22:28:05.137152+0000","last_peered":"2023-07-13T22:28:05.137152+0000","last_clean":"2023-07-13T22:28:05.137152+0000","last_became_active":"2023-07-13T22:28:04.122982+0000","last_became_peered":"2023-07-13T22:28:04.122982+0000","last_unstale":"2023-07-13T22:28:05.137152+0000","last_undegraded":"2023-07-13T22:28:05.137152+0000","last_fullsized":"2023-07-13T22:28:05.137152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:34:57.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.146642+0000","last_change":"2023-07-13T22:28:05.141551+0000","last_active":"2023-07-13T22:28:05.146642+0000","last_peered":"2023-07-13T22:28:05.146642+0000","last_clean":"2023-07-13T22:28:05.146642+0000","last_became_active":"2023-07-13T22:28:05.141228+0000","last_became_peered":"2023-07-13T22:28:05.141228+0000","last_unstale":"2023-07-13T22:28:05.146642+0000","last_undegraded":"2023-07-13T22:28:05.146642+0000","last_fullsized":"2023-07-13T22:28:05.146642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:24:56.271819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.114947+0000","last_change":"2023-07-13T22:28:04.125608+0000","last_active":"2023-07-13T22:28:05.114947+0000","last_peered":"2023-07-13T22:28:05.114947+0000","last_clean":"2023-07-13T22:28:05.114947+0000","last_became_active":"2023-07-13T22:28:04.125269+0000","last_became_peered":"2023-07-13T22:28:04.125269+0000","last_unstale":"2023-07-13T22:28:05.114947+0000","last_undegraded":"2023-07-13T22:28:05.114947+0000","last_fullsized":"2023-07-13T22:28:05.114947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:24:32.672612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117143+0000","last_change":"2023-07-13T22:28:05.115291+0000","last_active":"2023-07-13T22:28:05.117143+0000","last_peered":"2023-07-13T22:28:05.117143+0000","last_clean":"2023-07-13T22:28:05.117143+0000","last_became_active":"2023-07-13T22:28:05.114976+0000","last_became_peered":"2023-07-13T22:28:05.114976+0000","last_unstale":"2023-07-13T22:28:05.117143+0000","last_undegraded":"2023-07-13T22:28:05.117143+0000","last_fullsized":"2023-07-13T22:28:05.117143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:56:01.267865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.143027+0000","last_change":"2023-07-13T22:28:04.114326+0000","last_active":"2023-07-13T22:28:05.143027+0000","last_peered":"2023-07-13T22:28:05.143027+0000","last_clean":"2023-07-13T22:28:05.143027+0000","last_became_active":"2023-07-13T22:28:04.114014+0000","last_became_peered":"2023-07-13T22:28:04.114014+0000","last_unstale":"2023-07-13T22:28:05.143027+0000","last_undegraded":"2023-07-13T22:28:05.143027+0000","last_fullsized":"2023-07-13T22:28:05.143027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:37:37.461212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117841+0000","last_change":"2023-07-13T22:28:05.115809+0000","last_active":"2023-07-13T22:28:05.117841+0000","last_peered":"2023-07-13T22:28:05.117841+0000","last_clean":"2023-07-13T22:28:05.117841+0000","last_became_active":"2023-07-13T22:28:05.115493+0000","last_became_peered":"2023-07-13T22:28:05.115493+0000","last_unstale":"2023-07-13T22:28:05.117841+0000","last_undegraded":"2023-07-13T22:28:05.117841+0000","last_fullsized":"2023-07-13T22:28:05.117841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:31:46.589384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"86'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124123+0000","last_change":"2023-07-13T22:28:04.125778+0000","last_active":"2023-07-13T22:28:06.124123+0000","last_peered":"2023-07-13T22:28:06.124123+0000","last_clean":"2023-07-13T22:28:06.124123+0000","last_became_active":"2023-07-13T22:28:04.125454+0000","last_became_peered":"2023-07-13T22:28:04.125454+0000","last_unstale":"2023-07-13T22:28:06.124123+0000","last_undegraded":"2023-07-13T22:28:06.124123+0000","last_fullsized":"2023-07-13T22:28:06.124123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:26.332363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.147787+0000","last_change":"2023-07-13T22:28:05.131796+0000","last_active":"2023-07-13T22:28:05.147787+0000","last_peered":"2023-07-13T22:28:05.147787+0000","last_clean":"2023-07-13T22:28:05.147787+0000","last_became_active":"2023-07-13T22:28:05.131475+0000","last_became_peered":"2023-07-13T22:28:05.131475+0000","last_unstale":"2023-07-13T22:28:05.147787+0000","last_undegraded":"2023-07-13T22:28:05.147787+0000","last_fullsized":"2023-07-13T22:28:05.147787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:36:09.746449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124120+0000","last_change":"2023-07-13T22:28:04.117826+0000","last_active":"2023-07-13T22:28:06.124120+0000","last_peered":"2023-07-13T22:28:06.124120+0000","last_clean":"2023-07-13T22:28:06.124120+0000","last_became_active":"2023-07-13T22:28:04.117550+0000","last_became_peered":"2023-07-13T22:28:04.117550+0000","last_unstale":"2023-07-13T22:28:06.124120+0000","last_undegraded":"2023-07-13T22:28:06.124120+0000","last_fullsized":"2023-07-13T22:28:06.124120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:42:28.594229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148684+0000","last_change":"2023-07-13T22:28:05.142881+0000","last_active":"2023-07-13T22:28:05.148684+0000","last_peered":"2023-07-13T22:28:05.148684+0000","last_clean":"2023-07-13T22:28:05.148684+0000","last_became_active":"2023-07-13T22:28:05.142589+0000","last_became_peered":"2023-07-13T22:28:05.142589+0000","last_unstale":"2023-07-13T22:28:05.148684+0000","last_undegraded":"2023-07-13T22:28:05.148684+0000","last_fullsized":"2023-07-13T22:28:05.148684+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:20:22.801603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124741+0000","last_change":"2023-07-13T22:28:04.120397+0000","last_active":"2023-07-13T22:28:06.124741+0000","last_peered":"2023-07-13T22:28:06.124741+0000","last_clean":"2023-07-13T22:28:06.124741+0000","last_became_active":"2023-07-13T22:28:04.119625+0000","last_became_peered":"2023-07-13T22:28:04.119625+0000","last_unstale":"2023-07-13T22:28:06.124741+0000","last_undegraded":"2023-07-13T22:28:06.124741+0000","last_fullsized":"2023-07-13T22:28:06.124741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:14:02.481798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.128166+0000","last_change":"2023-07-13T22:28:05.121440+0000","last_active":"2023-07-13T22:28:05.128166+0000","last_peered":"2023-07-13T22:28:05.128166+0000","last_clean":"2023-07-13T22:28:05.128166+0000","last_became_active":"2023-07-13T22:28:05.121166+0000","last_became_peered":"2023-07-13T22:28:05.121166+0000","last_unstale":"2023-07-13T22:28:05.128166+0000","last_undegraded":"2023-07-13T22:28:05.128166+0000","last_fullsized":"2023-07-13T22:28:05.128166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:19:31.997822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.111694+0000","last_change":"2023-07-13T22:28:04.125151+0000","last_active":"2023-07-13T22:28:05.111694+0000","last_peered":"2023-07-13T22:28:05.111694+0000","last_clean":"2023-07-13T22:28:05.111694+0000","last_became_active":"2023-07-13T22:28:04.124580+0000","last_became_peered":"2023-07-13T22:28:04.124580+0000","last_unstale":"2023-07-13T22:28:05.111694+0000","last_undegraded":"2023-07-13T22:28:05.111694+0000","last_fullsized":"2023-07-13T22:28:05.111694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:23.537202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.134051+0000","last_change":"2023-07-13T22:28:05.125659+0000","last_active":"2023-07-13T22:28:05.134051+0000","last_peered":"2023-07-13T22:28:05.134051+0000","last_clean":"2023-07-13T22:28:05.134051+0000","last_became_active":"2023-07-13T22:28:05.125342+0000","last_became_peered":"2023-07-13T22:28:05.125342+0000","last_unstale":"2023-07-13T22:28:05.134051+0000","last_undegraded":"2023-07-13T22:28:05.134051+0000","last_fullsized":"2023-07-13T22:28:05.134051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:30:43.561195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.142817+0000","last_change":"2023-07-13T22:28:04.121524+0000","last_active":"2023-07-13T22:28:05.142817+0000","last_peered":"2023-07-13T22:28:05.142817+0000","last_clean":"2023-07-13T22:28:05.142817+0000","last_became_active":"2023-07-13T22:28:04.120610+0000","last_became_peered":"2023-07-13T22:28:04.120610+0000","last_unstale":"2023-07-13T22:28:05.142817+0000","last_undegraded":"2023-07-13T22:28:05.142817+0000","last_fullsized":"2023-07-13T22:28:05.142817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:51:35.242168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.120733+0000","last_change":"2023-07-13T22:28:05.135910+0000","last_active":"2023-07-13T22:28:06.120733+0000","last_peered":"2023-07-13T22:28:06.120733+0000","last_clean":"2023-07-13T22:28:06.120733+0000","last_became_active":"2023-07-13T22:28:05.135188+0000","last_became_peered":"2023-07-13T22:28:05.135188+0000","last_unstale":"2023-07-13T22:28:06.120733+0000","last_undegraded":"2023-07-13T22:28:06.120733+0000","last_fullsized":"2023-07-13T22:28:06.120733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:55:52.233253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.113862+0000","last_change":"2023-07-13T22:28:04.117486+0000","last_active":"2023-07-13T22:28:05.113862+0000","last_peered":"2023-07-13T22:28:05.113862+0000","last_clean":"2023-07-13T22:28:05.113862+0000","last_became_active":"2023-07-13T22:28:04.117179+0000","last_became_peered":"2023-07-13T22:28:04.117179+0000","last_unstale":"2023-07-13T22:28:05.113862+0000","last_undegraded":"2023-07-13T22:28:05.113862+0000","last_fullsized":"2023-07-13T22:28:05.113862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:51:35.242168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.137761+0000","last_change":"2023-07-13T22:28:05.136950+0000","last_active":"2023-07-13T22:28:05.137761+0000","last_peered":"2023-07-13T22:28:05.137761+0000","last_clean":"2023-07-13T22:28:05.137761+0000","last_became_active":"2023-07-13T22:28:05.136654+0000","last_became_peered":"2023-07-13T22:28:05.136654+0000","last_unstale":"2023-07-13T22:28:05.137761+0000","last_undegraded":"2023-07-13T22:28:05.137761+0000","last_fullsized":"2023-07-13T22:28:05.137761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:21:18.024141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.112080+0000","last_change":"2023-07-13T22:28:04.123161+0000","last_active":"2023-07-13T22:28:05.112080+0000","last_peered":"2023-07-13T22:28:05.112080+0000","last_clean":"2023-07-13T22:28:05.112080+0000","last_became_active":"2023-07-13T22:28:04.121907+0000","last_became_peered":"2023-07-13T22:28:04.121907+0000","last_unstale":"2023-07-13T22:28:05.112080+0000","last_undegraded":"2023-07-13T22:28:05.112080+0000","last_fullsized":"2023-07-13T22:28:05.112080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:44:38.459035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148949+0000","last_change":"2023-07-13T22:28:05.143433+0000","last_active":"2023-07-13T22:28:05.148949+0000","last_peered":"2023-07-13T22:28:05.148949+0000","last_clean":"2023-07-13T22:28:05.148949+0000","last_became_active":"2023-07-13T22:28:05.143118+0000","last_became_peered":"2023-07-13T22:28:05.143118+0000","last_unstale":"2023-07-13T22:28:05.148949+0000","last_undegraded":"2023-07-13T22:28:05.148949+0000","last_fullsized":"2023-07-13T22:28:05.148949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:07:42.543155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123367+0000","last_change":"2023-07-13T22:28:04.118891+0000","last_active":"2023-07-13T22:28:06.123367+0000","last_peered":"2023-07-13T22:28:06.123367+0000","last_clean":"2023-07-13T22:28:06.123367+0000","last_became_active":"2023-07-13T22:28:04.118621+0000","last_became_peered":"2023-07-13T22:28:04.118621+0000","last_unstale":"2023-07-13T22:28:06.123367+0000","last_undegraded":"2023-07-13T22:28:06.123367+0000","last_fullsized":"2023-07-13T22:28:06.123367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:06:24.628924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.142895+0000","last_change":"2023-07-13T22:28:05.141582+0000","last_active":"2023-07-13T22:28:05.142895+0000","last_peered":"2023-07-13T22:28:05.142895+0000","last_clean":"2023-07-13T22:28:05.142895+0000","last_became_active":"2023-07-13T22:28:05.141187+0000","last_became_peered":"2023-07-13T22:28:05.141187+0000","last_unstale":"2023-07-13T22:28:05.142895+0000","last_undegraded":"2023-07-13T22:28:05.142895+0000","last_fullsized":"2023-07-13T22:28:05.142895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:38:27.482861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117810+0000","last_change":"2023-07-13T22:28:04.119820+0000","last_active":"2023-07-13T22:28:05.117810+0000","last_peered":"2023-07-13T22:28:05.117810+0000","last_clean":"2023-07-13T22:28:05.117810+0000","last_became_active":"2023-07-13T22:28:04.119536+0000","last_became_peered":"2023-07-13T22:28:04.119536+0000","last_unstale":"2023-07-13T22:28:05.117810+0000","last_undegraded":"2023-07-13T22:28:05.117810+0000","last_fullsized":"2023-07-13T22:28:05.117810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:48:14.544857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124343+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:28:06.124343+0000","last_peered":"2023-07-13T22:28:06.124343+0000","last_clean":"2023-07-13T22:28:06.124343+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:28:06.124343+0000","last_undegraded":"2023-07-13T22:28:06.124343+0000","last_fullsized":"2023-07-13T22:28:06.124343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:53:12.006344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148927+0000","last_change":"2023-07-13T22:28:05.142840+0000","last_active":"2023-07-13T22:28:05.148927+0000","last_peered":"2023-07-13T22:28:05.148927+0000","last_clean":"2023-07-13T22:28:05.148927+0000","last_became_active":"2023-07-13T22:28:05.136746+0000","last_became_peered":"2023-07-13T22:28:05.136746+0000","last_unstale":"2023-07-13T22:28:05.148927+0000","last_undegraded":"2023-07-13T22:28:05.148927+0000","last_fullsized":"2023-07-13T22:28:05.148927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:30.946948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.105118+0000","last_change":"2023-07-13T22:28:04.117787+0000","last_active":"2023-07-13T22:28:05.105118+0000","last_peered":"2023-07-13T22:28:05.105118+0000","last_clean":"2023-07-13T22:28:05.105118+0000","last_became_active":"2023-07-13T22:28:04.117533+0000","last_became_peered":"2023-07-13T22:28:04.117533+0000","last_unstale":"2023-07-13T22:28:05.105118+0000","last_undegraded":"2023-07-13T22:28:05.105118+0000","last_fullsized":"2023-07-13T22:28:05.105118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:34:33.528642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":130,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.121046+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:28:06.121046+0000","last_peered":"2023-07-13T22:28:06.121046+0000","last_clean":"2023-07-13T22:28:06.121046+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:28:06.121046+0000","last_undegraded":"2023-07-13T22:28:06.121046+0000","last_fullsized":"2023-07-13T22:28:06.121046+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:33:38.927032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.147636+0000","last_change":"2023-07-13T22:28:05.128837+0000","last_active":"2023-07-13T22:28:05.147636+0000","last_peered":"2023-07-13T22:28:05.147636+0000","last_clean":"2023-07-13T22:28:05.147636+0000","last_became_active":"2023-07-13T22:28:05.128516+0000","last_became_peered":"2023-07-13T22:28:05.128516+0000","last_unstale":"2023-07-13T22:28:05.147636+0000","last_undegraded":"2023-07-13T22:28:05.147636+0000","last_fullsized":"2023-07-13T22:28:05.147636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:03:51.828510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"86'1","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.129119+0000","last_change":"2023-07-13T22:28:04.122417+0000","last_active":"2023-07-13T22:28:05.129119+0000","last_peered":"2023-07-13T22:28:05.129119+0000","last_clean":"2023-07-13T22:28:05.129119+0000","last_became_active":"2023-07-13T22:28:04.122144+0000","last_became_peered":"2023-07-13T22:28:04.122144+0000","last_unstale":"2023-07-13T22:28:05.129119+0000","last_undegraded":"2023-07-13T22:28:05.129119+0000","last_fullsized":"2023-07-13T22:28:05.129119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:29.939335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117332+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:28:05.117332+0000","last_peered":"2023-07-13T22:28:05.117332+0000","last_clean":"2023-07-13T22:28:05.117332+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:28:05.117332+0000","last_undegraded":"2023-07-13T22:28:05.117332+0000","last_fullsized":"2023-07-13T22:28:05.117332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:46:47.107876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":280,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123229+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:28:06.123229+0000","last_peered":"2023-07-13T22:28:06.123229+0000","last_clean":"2023-07-13T22:28:06.123229+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:28:06.123229+0000","last_undegraded":"2023-07-13T22:28:06.123229+0000","last_fullsized":"2023-07-13T22:28:06.123229+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:07:42.856173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.128167+0000","last_change":"2023-07-13T22:28:05.122705+0000","last_active":"2023-07-13T22:28:05.128167+0000","last_peered":"2023-07-13T22:28:05.128167+0000","last_clean":"2023-07-13T22:28:05.128167+0000","last_became_active":"2023-07-13T22:28:05.122406+0000","last_became_peered":"2023-07-13T22:28:05.122406+0000","last_unstale":"2023-07-13T22:28:05.128167+0000","last_undegraded":"2023-07-13T22:28:05.128167+0000","last_fullsized":"2023-07-13T22:28:05.128167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:54:04.495802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"86'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123719+0000","last_change":"2023-07-13T22:28:04.120990+0000","last_active":"2023-07-13T22:28:06.123719+0000","last_peered":"2023-07-13T22:28:06.123719+0000","last_clean":"2023-07-13T22:28:06.123719+0000","last_became_active":"2023-07-13T22:28:04.120714+0000","last_became_peered":"2023-07-13T22:28:04.120714+0000","last_unstale":"2023-07-13T22:28:06.123719+0000","last_undegraded":"2023-07-13T22:28:06.123719+0000","last_fullsized":"2023-07-13T22:28:06.123719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:25:18.484423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123452+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:28:06.123452+0000","last_peered":"2023-07-13T22:28:06.123452+0000","last_clean":"2023-07-13T22:28:06.123452+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:28:06.123452+0000","last_undegraded":"2023-07-13T22:28:06.123452+0000","last_fullsized":"2023-07-13T22:28:06.123452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:48:19.989497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123991+0000","last_change":"2023-07-13T22:28:05.132005+0000","last_active":"2023-07-13T22:28:06.123991+0000","last_peered":"2023-07-13T22:28:06.123991+0000","last_clean":"2023-07-13T22:28:06.123991+0000","last_became_active":"2023-07-13T22:28:05.131702+0000","last_became_peered":"2023-07-13T22:28:05.131702+0000","last_unstale":"2023-07-13T22:28:06.123991+0000","last_undegraded":"2023-07-13T22:28:06.123991+0000","last_fullsized":"2023-07-13T22:28:06.123991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:57:44.701616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.122110+0000","last_change":"2023-07-13T22:28:04.120075+0000","last_active":"2023-07-13T22:28:05.122110+0000","last_peered":"2023-07-13T22:28:05.122110+0000","last_clean":"2023-07-13T22:28:05.122110+0000","last_became_active":"2023-07-13T22:28:04.119722+0000","last_became_peered":"2023-07-13T22:28:04.119722+0000","last_unstale":"2023-07-13T22:28:05.122110+0000","last_undegraded":"2023-07-13T22:28:05.122110+0000","last_fullsized":"2023-07-13T22:28:05.122110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:21:17.016528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123366+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:28:06.123366+0000","last_peered":"2023-07-13T22:28:06.123366+0000","last_clean":"2023-07-13T22:28:06.123366+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:28:06.123366+0000","last_undegraded":"2023-07-13T22:28:06.123366+0000","last_fullsized":"2023-07-13T22:28:06.123366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:38:57.880488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.123754+0000","last_change":"2023-07-13T22:28:05.119032+0000","last_active":"2023-07-13T22:28:06.123754+0000","last_peered":"2023-07-13T22:28:06.123754+0000","last_clean":"2023-07-13T22:28:06.123754+0000","last_became_active":"2023-07-13T22:28:05.118688+0000","last_became_peered":"2023-07-13T22:28:05.118688+0000","last_unstale":"2023-07-13T22:28:06.123754+0000","last_undegraded":"2023-07-13T22:28:06.123754+0000","last_fullsized":"2023-07-13T22:28:06.123754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:05:35.372840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"86'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-13T22:28:06.124532+0000","last_change":"2023-07-13T22:28:04.117910+0000","last_active":"2023-07-13T22:28:06.124532+0000","last_peered":"2023-07-13T22:28:06.124532+0000","last_clean":"2023-07-13T22:28:06.124532+0000","last_became_active":"2023-07-13T22:28:04.117618+0000","last_became_peered":"2023-07-13T22:28:04.117618+0000","last_unstale":"2023-07-13T22:28:06.124532+0000","last_undegraded":"2023-07-13T22:28:06.124532+0000","last_fullsized":"2023-07-13T22:28:06.124532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:54:55.243797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.117924+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:28:05.117924+0000","last_peered":"2023-07-13T22:28:05.117924+0000","last_clean":"2023-07-13T22:28:05.117924+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:28:05.117924+0000","last_undegraded":"2023-07-13T22:28:05.117924+0000","last_fullsized":"2023-07-13T22:28:05.117924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:05:04.054792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.138922+0000","last_change":"2023-07-13T22:28:05.137879+0000","last_active":"2023-07-13T22:28:05.138922+0000","last_peered":"2023-07-13T22:28:05.138922+0000","last_clean":"2023-07-13T22:28:05.138922+0000","last_became_active":"2023-07-13T22:28:05.137568+0000","last_became_peered":"2023-07-13T22:28:05.137568+0000","last_unstale":"2023-07-13T22:28:05.138922+0000","last_undegraded":"2023-07-13T22:28:05.138922+0000","last_fullsized":"2023-07-13T22:28:05.138922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:01.584028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.105306+0000","last_change":"2023-07-13T22:28:04.116113+0000","last_active":"2023-07-13T22:28:05.105306+0000","last_peered":"2023-07-13T22:28:05.105306+0000","last_clean":"2023-07-13T22:28:05.105306+0000","last_became_active":"2023-07-13T22:28:04.115841+0000","last_became_peered":"2023-07-13T22:28:04.115841+0000","last_unstale":"2023-07-13T22:28:05.105306+0000","last_undegraded":"2023-07-13T22:28:05.105306+0000","last_fullsized":"2023-07-13T22:28:05.105306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:57:36.598845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":126,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.111956+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:28:05.111956+0000","last_peered":"2023-07-13T22:28:05.111956+0000","last_clean":"2023-07-13T22:28:05.111956+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:28:05.111956+0000","last_undegraded":"2023-07-13T22:28:05.111956+0000","last_fullsized":"2023-07-13T22:28:05.111956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:14:13.540273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.148810+0000","last_change":"2023-07-13T22:28:05.145439+0000","last_active":"2023-07-13T22:28:05.148810+0000","last_peered":"2023-07-13T22:28:05.148810+0000","last_clean":"2023-07-13T22:28:05.148810+0000","last_became_active":"2023-07-13T22:28:05.145105+0000","last_became_peered":"2023-07-13T22:28:05.145105+0000","last_unstale":"2023-07-13T22:28:05.148810+0000","last_undegraded":"2023-07-13T22:28:05.148810+0000","last_fullsized":"2023-07-13T22:28:05.148810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:04.094019+0000","last_clean_scrub_stamp":"2023-07-13T22:28:04.094019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:36:14.284898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"86'2","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.138679+0000","last_change":"2023-07-13T22:28:04.118114+0000","last_active":"2023-07-13T22:28:05.138679+0000","last_peered":"2023-07-13T22:28:05.138679+0000","last_clean":"2023-07-13T22:28:05.138679+0000","last_became_active":"2023-07-13T22:28:04.117849+0000","last_became_peered":"2023-07-13T22:28:04.117849+0000","last_unstale":"2023-07-13T22:28:05.138679+0000","last_undegraded":"2023-07-13T22:28:05.138679+0000","last_fullsized":"2023-07-13T22:28:05.138679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:03.086406+0000","last_clean_scrub_stamp":"2023-07-13T22:28:03.086406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:34:57.846578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":126,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-13T22:28:05.114030+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:28:05.114030+0000","last_peered":"2023-07-13T22:28:05.114030+0000","last_clean":"2023-07-13T22:28:05.114030+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:28:05.114030+0000","last_undegraded":"2023-07-13T22:28:05.114030+0000","last_fullsized":"2023-07-13T22:28:05.114030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:46:16.297930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1658880,"data_stored":1068234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.311,"15min":0.311},"max":{"1min":0.98699999999999999,"5min":1.0780000000000001,"15min":1.115},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69799999999999995,"15min":0.67200000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.30099999999999999},"max":{"1min":0.97199999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66400000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63800000000000001,"15min":0.68799999999999994},"min":{"1min":0.42399999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94799999999999995,"15min":2.2610000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67300000000000004,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.34799999999999998,"15min":0.34300000000000003},"max":{"1min":0.873,"5min":1.0329999999999999,"15min":1.0609999999999999},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.78400000000000003},"min":{"1min":0.45600000000000002,"5min":0.443,"15min":0.40600000000000003},"max":{"1min":0.95099999999999996,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70299999999999996,"15min":0.77700000000000002},"min":{"1min":0.495,"5min":0.42999999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":1.107,"15min":3.4710000000000001},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69899999999999995,"15min":0.68799999999999994},"min":{"1min":0.501,"5min":0.45000000000000001,"15min":0.34599999999999997},"max":{"1min":0.98199999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.75800000000000001},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.01,"5min":1.024,"15min":1.1399999999999999},"last":0.755}]},{"osd":4,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.64200000000000002},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90100000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91500000000000004,"5min":1.0860000000000001,"15min":1.1850000000000001},"last":0.82999999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.66200000000000003},"min":{"1min":0.503,"5min":0.38200000000000001,"15min":0.35099999999999998},"max":{"1min":0.91500000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.79000000000000004},"min":{"1min":0.59899999999999998,"5min":0.498,"15min":0.48899999999999999},"max":{"1min":0.96499999999999997,"5min":1.1359999999999999,"15min":1.2110000000000001},"last":0.79500000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69099999999999995,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.437,"15min":0.39500000000000002},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.72199999999999998},"min":{"1min":0.47199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80000000000000004,"5min":0.98799999999999999,"15min":1.042},"last":0.63}]}]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":1077248,"data_stored":479978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67100000000000004,"15min":0.628},"min":{"1min":0.42699999999999999,"5min":0.33100000000000002,"15min":0.28799999999999998},"max":{"1min":1.002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68999999999999995,"15min":0.628},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.33500000000000002},"max":{"1min":0.89700000000000002,"5min":0.97399999999999998,"15min":0.98899999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.505,"5min":0.39400000000000002,"15min":0.33900000000000002},"max":{"1min":0.94099999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73399999999999999,"15min":0.70299999999999996},"min":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.438},"max":{"1min":0.96899999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.67100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.91800000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69199999999999995,"15min":0.67000000000000004},"min":{"1min":0.56200000000000006,"5min":0.45600000000000002,"15min":0.35599999999999998},"max":{"1min":0.871,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70899999999999996,"15min":0.66300000000000003},"min":{"1min":0.56100000000000005,"5min":0.37,"15min":0.37},"max":{"1min":0.91600000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73599999999999999,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.95599999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.94499999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65500000000000003,"15min":0.68799999999999994},"min":{"1min":0.52200000000000002,"5min":0.40200000000000002,"15min":0.30399999999999999},"max":{"1min":0.997,"5min":0.998,"15min":0.998},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86899999999999999,"5min":0.94199999999999995,"15min":0.95999999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.73799999999999999,"15min":0.75800000000000001},"min":{"1min":0.54200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97399999999999998,"5min":1.05,"15min":1.05},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70699999999999996,"15min":0.745},"min":{"1min":0.46700000000000003,"5min":0.45600000000000002,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96999999999999997,"15min":1.1890000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":1.111},"last":0.48999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1654784,"data_stored":1068248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45000000000000001,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.249,"15min":0.249},"max":{"1min":0.70199999999999996,"5min":0.71599999999999997,"15min":1.952},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.496,"15min":0.51700000000000002},"min":{"1min":0.30599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":1.1060000000000001},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.29899999999999999,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":2.1200000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.35099999999999998},"max":{"1min":0.67400000000000004,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":2.008},"last":0.433},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53000000000000003,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.79000000000000004,"5min":1.054,"15min":1.054},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73899999999999999,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.57199999999999995,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":0.98199999999999998,"15min":2.028},"last":0.749},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.46400000000000002},"max":{"1min":0.94199999999999995,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.98899999999999999,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78000000000000003,"15min":0.84399999999999997},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.006,"5min":1.095,"15min":2.0899999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78600000000000003,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.55100000000000005,"15min":0.52600000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.2230000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.85799999999999998},"min":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0489999999999999,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79400000000000004,"15min":0.82699999999999996},"min":{"1min":0.68700000000000006,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":0.99199999999999999,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.59099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":1077248,"data_stored":479094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.52500000000000002},"min":{"1min":0.33500000000000002,"5min":0.30099999999999999,"15min":0.28499999999999998},"max":{"1min":0.84999999999999998,"5min":1.026,"15min":11.791},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.53400000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.874,"5min":1.419,"15min":4.657},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.432,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":1.1459999999999999,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52600000000000002,"15min":0.96699999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":1.276,"15min":13.398999999999999},"last":0.434}]},{"osd":3,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72999999999999998,"5min":0.89400000000000002,"15min":1.8089999999999999},"last":0.379},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56499999999999995,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.997,"15min":4.4530000000000003},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.747,"15min":1.218},"min":{"1min":0.44800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76000000000000001,"15min":0.80800000000000005},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.41299999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.776},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.78400000000000003,"15min":1.238},"min":{"1min":0.60399999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.073,"5min":1.113,"15min":13.349},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.82699999999999996,"15min":0.88},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.302,"15min":4.7649999999999997},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80000000000000004,"15min":0.83899999999999997},"min":{"1min":0.64800000000000002,"5min":0.52800000000000002,"15min":0.49099999999999999},"max":{"1min":1.212,"5min":1.212,"15min":4.5039999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":1.2090000000000001},"min":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1259999999999999,"5min":1.2330000000000001,"15min":11.679},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79200000000000004,"15min":1.2430000000000001},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.101,"5min":1.1759999999999999,"15min":13.298},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80100000000000005,"15min":0.89300000000000002},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.151,"5min":1.151,"15min":11.456},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":1052672,"data_stored":476824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44,"15min":0.53300000000000003},"min":{"1min":0.307,"5min":0.249,"15min":0.249},"max":{"1min":0.78300000000000003,"5min":1.1779999999999999,"15min":2.282},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.34799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64600000000000002,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.47399999999999998},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.64800000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51900000000000002,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":2.319},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":1.01,"15min":1.01},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.64300000000000002},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.38700000000000001},"max":{"1min":0.72899999999999998,"5min":0.95599999999999996,"15min":2.351},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.76200000000000001,"15min":0.83899999999999997},"min":{"1min":0.64000000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.165,"15min":2.238},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.58299999999999996,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":1.274,"15min":1.518},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.57099999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.997,"5min":1.1299999999999999,"15min":1.325},"last":0.748},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.80300000000000005,"15min":0.85799999999999998},"min":{"1min":0.66700000000000004,"5min":0.53500000000000003,"15min":0.495},"max":{"1min":0.97999999999999998,"5min":1.2490000000000001,"15min":2.1909999999999998},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.71099999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.026,"5min":1.1040000000000001,"15min":1.488},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.61599999999999999,"5min":0.502,"15min":0.48199999999999998},"max":{"1min":0.96499999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81599999999999995,"15min":0.79600000000000004},"min":{"1min":0.72299999999999998,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":0.98199999999999998,"5min":1.29,"15min":1.29},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.69899999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.046,"5min":1.254,"15min":1.254},"last":0.64900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36176,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335664,"statfs":{"total":96636764160,"available":96599719936,"internally_reserved":0,"allocated":1056768,"data_stored":477810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.41999999999999998,"15min":0.42399999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.48299999999999998,"15min":0.45300000000000001},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.27200000000000002},"max":{"1min":0.71099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.35599999999999998,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.378,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.71899999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73599999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56599999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.45800000000000002,"15min":0.44700000000000001},"max":{"1min":1.075,"5min":1.1699999999999999,"15min":1.2250000000000001},"last":0.752},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.014,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.96899999999999997,"5min":1.1919999999999999,"15min":1.411},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.998,"5min":1.1779999999999999,"15min":1.5469999999999999},"last":0.95299999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.52500000000000002,"15min":0.51800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.52100000000000002},"max":{"1min":0.93500000000000005,"5min":1.046,"15min":1.1240000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.80100000000000005},"min":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.498},"max":{"1min":1.0740000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78200000000000003,"15min":0.79100000000000004},"min":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.47199999999999998},"max":{"1min":0.93400000000000005,"5min":1.153,"15min":1.1950000000000001},"last":0.93400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":1064960,"data_stored":477970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.97299999999999998,"15min":1.0349999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86599999999999999,"5min":0.997,"15min":0.997},"last":0.84699999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":1.0489999999999999,"15min":1.298},"last":0.75},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.69499999999999995},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.438},"max":{"1min":0.85499999999999998,"5min":1.0049999999999999,"15min":1.3109999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.48199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.1200000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.71899999999999997},"min":{"1min":0.61799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94599999999999995,"5min":1.089,"15min":1.1839999999999999},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.95699999999999996},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.35899999999999999},"max":{"1min":0.83299999999999996,"5min":0.94999999999999996,"15min":10.657999999999999},"last":1.234},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72899999999999998,"15min":0.93600000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":8.75},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.46200000000000002,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":0.90800000000000003,"5min":1.042,"15min":1.042},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94899999999999995,"5min":1.0109999999999999,"15min":1.0429999999999999},"last":1.1759999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93000000000000005,"5min":1.0189999999999999,"15min":1.173},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.72299999999999998},"min":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97199999999999998,"5min":1.0509999999999999,"15min":1.099},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.004},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.753,"15min":0.77200000000000002},"min":{"1min":0.53200000000000003,"5min":0.497,"15min":0.47899999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.212},"last":1.0429999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36176,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335664,"statfs":{"total":96636764160,"available":96599719936,"internally_reserved":0,"allocated":1056768,"data_stored":477810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65800000000000003,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.33700000000000002},"max":{"1min":0.82499999999999996,"5min":1.042,"15min":1.042},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.64000000000000001},"min":{"1min":0.53500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.998,"15min":0.998},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.36299999999999999,"15min":0.33100000000000002},"max":{"1min":0.86799999999999999,"5min":8.9930000000000003,"15min":8.9930000000000003},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.63900000000000001},"min":{"1min":0.57299999999999995,"5min":0.38400000000000001,"15min":0.35899999999999999},"max":{"1min":0.86599999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.67500000000000004},"min":{"1min":0.57799999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72799999999999998,"15min":0.69499999999999995},"min":{"1min":0.57399999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.96599999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.68799999999999994},"min":{"1min":0.57599999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89500000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.623}]},{"osd":4,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81299999999999994,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90900000000000003,"5min":0.93500000000000005,"15min":0.995},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66300000000000003,"15min":0.66600000000000004},"min":{"1min":0.44,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.85399999999999998,"5min":0.98099999999999998,"15min":0.98499999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71999999999999997,"15min":0.72699999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.997,"15min":1.02},"last":0.754},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.76300000000000001},"min":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0669999999999999},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:28:10.036 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:28:10.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-13T22:28:10.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.300+0000 7fbaad7ff700 1 Processor -- start 2023-07-13T22:28:10.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.300+0000 7fbaad7ff700 1 -- start start 2023-07-13T22:28:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.301+0000 7fbaad7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa80c8da0 0x7fbaa8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.301+0000 7fbaad7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa8070ce0 con 0x7fbaa8070890 2023-07-13T22:28:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.301+0000 7fbaad7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa80c94a0 con 0x7fbaa815a530 2023-07-13T22:28:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.301+0000 7fbaad7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa806de70 con 0x7fbaa80c8da0 2023-07-13T22:28:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.301+0000 7fbaa77fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbaa8070890 0x7fbaa80aebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53740/0 (socket says 172.21.15.60:53740) 2023-07-13T22:28:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.301+0000 7fbaa77fe700 1 -- 172.21.15.60:0/2889758942 learned_addr learned my addr 172.21.15.60:0/2889758942 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.302+0000 7fba9ffff700 1 --2- 172.21.15.60:0/2889758942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa80c8da0 0x7fbaa8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.302+0000 7fbaa67fc700 1 -- 172.21.15.60:0/2889758942 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333332510 0 0) 0x7fbaa0001490 con 0x7fbaa8070890 2023-07-13T22:28:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.302+0000 7fbaa67fc700 1 -- 172.21.15.60:0/2889758942 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba940035b0 con 0x7fbaa8070890 2023-07-13T22:28:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.302+0000 7fbaa67fc700 1 -- 172.21.15.60:0/2889758942 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372591416 0 0) 0x7fbaa80c94a0 con 0x7fbaa815a530 2023-07-13T22:28:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.302+0000 7fbaa67fc700 1 -- 172.21.15.60:0/2889758942 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaa0001490 con 0x7fbaa815a530 2023-07-13T22:28:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.302+0000 7fba9ffff700 1 -- 172.21.15.60:0/2889758942 >> v1:172.21.15.105:6789/0 conn(0x7fbaa815a530 legacy=0x7fbaa80addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.302+0000 7fba9ffff700 1 -- 172.21.15.60:0/2889758942 >> v1:172.21.15.60:6789/0 conn(0x7fbaa8070890 legacy=0x7fbaa80aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.302+0000 7fba9ffff700 1 -- 172.21.15.60:0/2889758942 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa80c91d0 con 0x7fbaa80c8da0 2023-07-13T22:28:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.303+0000 7fba9ffff700 1 --2- 172.21.15.60:0/2889758942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa80c8da0 0x7fbaa8070350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba90009870 tx=0x7fba90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfceb25bd708083e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.303+0000 7fbaa67fc700 1 -- 172.21.15.60:0/2889758942 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90011830 con 0x7fbaa80c8da0 2023-07-13T22:28:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.303+0000 7fbaa67fc700 1 -- 172.21.15.60:0/2889758942 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba90011990 con 0x7fbaa80c8da0 2023-07-13T22:28:10.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.303+0000 7fbaa67fc700 1 -- 172.21.15.60:0/2889758942 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90011d20 con 0x7fbaa80c8da0 2023-07-13T22:28:10.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.303+0000 7fbaad7ff700 1 -- 172.21.15.60:0/2889758942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa80c8da0 msgr2=0x7fbaa8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.303+0000 7fbaad7ff700 1 --2- 172.21.15.60:0/2889758942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa80c8da0 0x7fbaa8070350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba90009870 tx=0x7fba90009b80 comp rx=0 tx=0).stop 2023-07-13T22:28:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.303+0000 7fbaad7ff700 1 -- 172.21.15.60:0/2889758942 shutdown_connections 2023-07-13T22:28:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.303+0000 7fbaad7ff700 1 --2- 172.21.15.60:0/2889758942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa80c8da0 0x7fbaa8070350 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.303+0000 7fbaad7ff700 1 -- 172.21.15.60:0/2889758942 >> 172.21.15.60:0/2889758942 conn(0x7fbaa80cf3b0 msgr2=0x7fbaa80cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.304+0000 7fbaad7ff700 1 -- 172.21.15.60:0/2889758942 shutdown_connections 2023-07-13T22:28:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.304+0000 7fbaad7ff700 1 -- 172.21.15.60:0/2889758942 wait complete. 2023-07-13T22:28:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.305+0000 7fbaad7ff700 1 Processor -- start 2023-07-13T22:28:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.305+0000 7fbaad7ff700 1 -- start start 2023-07-13T22:28:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.306+0000 7fbaad7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaa8070890 0x7fbaa814d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.306+0000 7fbaad7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaa80c8da0 0x7fbaa80b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.306+0000 7fbaad7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa815a530 0x7fbaa80aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.306+0000 7fbaad7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa80ce150 con 0x7fbaa8070890 2023-07-13T22:28:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.306+0000 7fbaad7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa80b1800 con 0x7fbaa80c8da0 2023-07-13T22:28:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.306+0000 7fbaad7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa80b1ae0 con 0x7fbaa815a530 2023-07-13T22:28:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fbaa6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaa8070890 0x7fbaa814d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fbaa6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaa8070890 0x7fbaa814d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60972/0 (socket says 172.21.15.60:60972) 2023-07-13T22:28:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/4130139554 learned_addr learned my addr 172.21.15.60:0/4130139554 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fba9ffff700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaa80c8da0 0x7fbaa80b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fbaa77fe700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa815a530 0x7fbaa80aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/4130139554 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa815a530 msgr2=0x7fbaa80aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fbaa6ffd700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa815a530 0x7fbaa80aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/4130139554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaa80c8da0 msgr2=0x7fbaa80b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fbaa6ffd700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaa80c8da0 0x7fbaa80b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.307+0000 7fbaa6ffd700 1 -- 172.21.15.60:0/4130139554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba900094a0 con 0x7fbaa8070890 2023-07-13T22:28:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.308+0000 7fbaa6ffd700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaa8070890 0x7fbaa814d360 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fba9800c520 tx=0x7fba9800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.308+0000 7fba9f7fe700 1 -- 172.21.15.60:0/4130139554 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba980052d0 con 0x7fbaa8070890 2023-07-13T22:28:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.308+0000 7fba9f7fe700 1 -- 172.21.15.60:0/4130139554 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba98005430 con 0x7fbaa8070890 2023-07-13T22:28:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.308+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa80b1dc0 con 0x7fbaa8070890 2023-07-13T22:28:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.308+0000 7fba9f7fe700 1 -- 172.21.15.60:0/4130139554 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba980038c0 con 0x7fbaa8070890 2023-07-13T22:28:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.308+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa814e620 con 0x7fbaa8070890 2023-07-13T22:28:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.309+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa80b1660 con 0x7fbaa8070890 2023-07-13T22:28:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.310+0000 7fba9f7fe700 1 -- 172.21.15.60:0/4130139554 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba98003a20 con 0x7fbaa8070890 2023-07-13T22:28:10.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.311+0000 7fba9f7fe700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba8806ace0 0x7fba8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.312+0000 7fba9f7fe700 1 -- 172.21.15.60:0/4130139554 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba98088910 con 0x7fbaa8070890 2023-07-13T22:28:10.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.312+0000 7fba9ffff700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba8806ace0 0x7fba8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.312+0000 7fba9ffff700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba8806ace0 0x7fba8806d190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fba900097c0 tx=0x7fba9001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.316+0000 7fba9f7fe700 1 -- 172.21.15.60:0/4130139554 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbaa80b1660 con 0x7fbaa8070890 2023-07-13T22:28:10.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.601+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbaa80b1660 con 0x7fbaa8070890 2023-07-13T22:28:10.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.606+0000 7fba9f7fe700 1 -- 172.21.15.60:0/4130139554 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fba980170b0 con 0x7fbaa8070890 2023-07-13T22:28:10.586 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:28:10.586 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:28:10.586 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-07-13T22:28:10.586 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-07-13T22:28:10.587 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-13T22:28:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba8806ace0 msgr2=0x7fba8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba8806ace0 0x7fba8806d190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fba900097c0 tx=0x7fba9001c040 comp rx=0 tx=0).stop 2023-07-13T22:28:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaa8070890 msgr2=0x7fbaa814d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaa8070890 0x7fbaa814d360 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fba9800c520 tx=0x7fba9800c830 comp rx=0 tx=0).stop 2023-07-13T22:28:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 shutdown_connections 2023-07-13T22:28:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba8806ace0 0x7fba8806d190 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaa8070890 0x7fbaa814d360 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbaa815a530 0x7fbaa80aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 --2- 172.21.15.60:0/4130139554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbaa80c8da0 0x7fbaa80b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 >> 172.21.15.60:0/4130139554 conn(0x7fbaa80cf3b0 msgr2=0x7fbaa80c3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.608+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 shutdown_connections 2023-07-13T22:28:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.609+0000 7fbaad7ff700 1 -- 172.21.15.60:0/4130139554 wait complete. 2023-07-13T22:28:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-07-13T22:28:10.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:28:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.830+0000 7fd66e3ff700 1 Processor -- start 2023-07-13T22:28:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.830+0000 7fd66e3ff700 1 -- start start 2023-07-13T22:28:10.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.831+0000 7fd66e3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd6680b8400 0x7fd6680cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.831+0000 7fd66e3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6680d0ac0 con 0x7fd6680cfca0 2023-07-13T22:28:10.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.831+0000 7fd66e3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6680b87d0 con 0x7fd6680d3750 2023-07-13T22:28:10.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.831+0000 7fd66e3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6680b4eb0 con 0x7fd6680b8400 2023-07-13T22:28:10.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.831+0000 7fd66c99c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd6680cfca0 0x7fd6680d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53754/0 (socket says 172.21.15.60:53754) 2023-07-13T22:28:10.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.831+0000 7fd66c99c700 1 -- 172.21.15.60:0/1162449874 learned_addr learned my addr 172.21.15.60:0/1162449874 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.831+0000 7fd667fff700 1 --2- 172.21.15.60:0/1162449874 >> v2:172.21.15.105:3300/0 conn(0x7fd6680b8400 0x7fd6680cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.832+0000 7fd666ffd700 1 -- 172.21.15.60:0/1162449874 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901341116 0 0) 0x7fd6680d0ac0 con 0x7fd6680cfca0 2023-07-13T22:28:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.832+0000 7fd666ffd700 1 -- 172.21.15.60:0/1162449874 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6500035b0 con 0x7fd6680cfca0 2023-07-13T22:28:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.832+0000 7fd667fff700 1 -- 172.21.15.60:0/1162449874 >> v1:172.21.15.105:6789/0 conn(0x7fd6680d3750 legacy=0x7fd6680cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.832+0000 7fd667fff700 1 -- 172.21.15.60:0/1162449874 >> v1:172.21.15.60:6789/0 conn(0x7fd6680cfca0 legacy=0x7fd6680d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.832+0000 7fd667fff700 1 -- 172.21.15.60:0/1162449874 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680b5190 con 0x7fd6680b8400 2023-07-13T22:28:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.832+0000 7fd667fff700 1 --2- 172.21.15.60:0/1162449874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6680b8400 0x7fd6680cd240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd658009840 tx=0x7fd658009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd71d13a80fb4084 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:10.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd666ffd700 1 -- 172.21.15.60:0/1162449874 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658011830 con 0x7fd6680b8400 2023-07-13T22:28:10.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd666ffd700 1 -- 172.21.15.60:0/1162449874 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658011990 con 0x7fd6680b8400 2023-07-13T22:28:10.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd666ffd700 1 -- 172.21.15.60:0/1162449874 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658011d20 con 0x7fd6680b8400 2023-07-13T22:28:10.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd66e3ff700 1 -- 172.21.15.60:0/1162449874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6680b8400 msgr2=0x7fd6680cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd66e3ff700 1 --2- 172.21.15.60:0/1162449874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6680b8400 0x7fd6680cd240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd658009840 tx=0x7fd658009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:10.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd66e3ff700 1 -- 172.21.15.60:0/1162449874 shutdown_connections 2023-07-13T22:28:10.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd66e3ff700 1 --2- 172.21.15.60:0/1162449874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6680b8400 0x7fd6680cd240 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd66e3ff700 1 -- 172.21.15.60:0/1162449874 >> 172.21.15.60:0/1162449874 conn(0x7fd66806b390 msgr2=0x7fd66806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:10.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd66e3ff700 1 -- 172.21.15.60:0/1162449874 shutdown_connections 2023-07-13T22:28:10.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.833+0000 7fd66e3ff700 1 -- 172.21.15.60:0/1162449874 wait complete. 2023-07-13T22:28:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.835+0000 7fd66e3ff700 1 Processor -- start 2023-07-13T22:28:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.835+0000 7fd66e3ff700 1 -- start start 2023-07-13T22:28:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd66e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6680b8400 0x7fd66815e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd66e3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680cfca0 0x7fd66814bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd66e3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6680d3750 0x7fd6680d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd66e3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66815ef20 con 0x7fd6680cfca0 2023-07-13T22:28:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd66e3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681f2a70 con 0x7fd6680d3750 2023-07-13T22:28:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd66e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66815df00 con 0x7fd6680b8400 2023-07-13T22:28:10.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd6677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680cfca0 0x7fd66814bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd6677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680cfca0 0x7fd66814bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60984/0 (socket says 172.21.15.60:60984) 2023-07-13T22:28:10.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd6677fe700 1 -- 172.21.15.60:0/3916452817 learned_addr learned my addr 172.21.15.60:0/3916452817 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:10.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd667fff700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6680b8400 0x7fd66815e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.836+0000 7fd66c99c700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6680d3750 0x7fd6680d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd6677fe700 1 -- 172.21.15.60:0/3916452817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6680b8400 msgr2=0x7fd66815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd6677fe700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6680b8400 0x7fd66815e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd6677fe700 1 -- 172.21.15.60:0/3916452817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6680d3750 msgr2=0x7fd6680d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:10.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd6677fe700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6680d3750 0x7fd6680d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:10.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd6677fe700 1 -- 172.21.15.60:0/3916452817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6580094a0 con 0x7fd6680cfca0 2023-07-13T22:28:10.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd6677fe700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680cfca0 0x7fd66814bf20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c420 tx=0x7fd65c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:10.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd664ff9700 1 -- 172.21.15.60:0/3916452817 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0052d0 con 0x7fd6680cfca0 2023-07-13T22:28:10.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd664ff9700 1 -- 172.21.15.60:0/3916452817 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c005430 con 0x7fd6680cfca0 2023-07-13T22:28:10.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd664ff9700 1 -- 172.21.15.60:0/3916452817 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0038c0 con 0x7fd6680cfca0 2023-07-13T22:28:10.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.837+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66815e1e0 con 0x7fd6680cfca0 2023-07-13T22:28:10.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.838+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66814c460 con 0x7fd6680cfca0 2023-07-13T22:28:10.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.839+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66815e700 con 0x7fd6680cfca0 2023-07-13T22:28:10.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.839+0000 7fd664ff9700 1 -- 172.21.15.60:0/3916452817 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd65c003a20 con 0x7fd6680cfca0 2023-07-13T22:28:10.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.840+0000 7fd664ff9700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd65406ace0 0x7fd65406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:10.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.841+0000 7fd664ff9700 1 -- 172.21.15.60:0/3916452817 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd65c0887a0 con 0x7fd6680cfca0 2023-07-13T22:28:10.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.841+0000 7fd667fff700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd65406ace0 0x7fd65406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:10.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.841+0000 7fd667fff700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd65406ace0 0x7fd65406d190 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd6580090b0 tx=0x7fd65801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:10.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:10.845+0000 7fd664ff9700 1 -- 172.21.15.60:0/3916452817 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd66815e700 con 0x7fd6680cfca0 2023-07-13T22:28:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.095+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd66815e700 con 0x7fd6680cfca0 2023-07-13T22:28:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.096+0000 7fd664ff9700 1 -- 172.21.15.60:0/3916452817 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fd66815e700 con 0x7fd6680cfca0 2023-07-13T22:28:11.076 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:11.077 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2395854295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2395854295},{"type":"v1","addr":"172.21.15.105:6834","nonce":2395854295}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1048261776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1048261776},{"type":"v1","addr":"172.21.15.105:6835","nonce":1048261776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/1639899719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1639899719},{"type":"v1","addr":"172.21.15.60:6836","nonce":1639899719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:04.618240+0000","modified":"2023-07-13T22:28:08.127584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7434":{"gid":7434,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/1512512599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":1512512599},{"type":"v1","addr":"172.21.15.60:6837","nonce":1512512599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-13T22:28:11.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd65406ace0 msgr2=0x7fd65406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd65406ace0 0x7fd65406d190 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd6580090b0 tx=0x7fd65801c040 comp rx=0 tx=0).stop 2023-07-13T22:28:11.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680cfca0 msgr2=0x7fd66814bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680cfca0 0x7fd66814bf20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c420 tx=0x7fd65c00c730 comp rx=0 tx=0).stop 2023-07-13T22:28:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 shutdown_connections 2023-07-13T22:28:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd65406ace0 0x7fd65406d190 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6680d3750 0x7fd6680d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6680b8400 0x7fd66815e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 --2- 172.21.15.60:0/3916452817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680cfca0 0x7fd66814bf20 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 >> 172.21.15.60:0/3916452817 conn(0x7fd66806b390 msgr2=0x7fd66815a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 shutdown_connections 2023-07-13T22:28:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.099+0000 7fd66e3ff700 1 -- 172.21.15.60:0/3916452817 wait complete. 2023-07-13T22:28:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 52 2023-07-13T22:28:11.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:28:04.618240+0000', 'modified': '2023-07-13T22:28:08.127584+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7434': {'gid': 7434, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6837/1512512599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6835', 'nonce': 1512512599}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 1512512599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:28:11.500 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:28:11.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-13T22:28:11.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.724+0000 7f1d841ff700 1 Processor -- start 2023-07-13T22:28:11.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.724+0000 7f1d841ff700 1 -- start start 2023-07-13T22:28:11.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.725+0000 7f1d841ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d7c0d3750 0x7f1d7c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.725+0000 7f1d841ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 0x7f1d7c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.725+0000 7f1d841ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c0cd950 con 0x7f1d7c0b8400 2023-07-13T22:28:11.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.725+0000 7f1d841ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c0cff50 con 0x7f1d7c0d3750 2023-07-13T22:28:11.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.725+0000 7f1d841ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0d0ac0 con 0x7f1d7c0cfad0 2023-07-13T22:28:11.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d81f9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1d7c0b8400 0x7f1d7c0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53766/0 (socket says 172.21.15.60:53766) 2023-07-13T22:28:11.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d81f9b700 1 -- 172.21.15.60:0/3228504559 learned_addr learned my addr 172.21.15.60:0/3228504559 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:11.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d8179a700 1 --2- 172.21.15.60:0/3228504559 >> v2:172.21.15.60:3300/0 conn(0x7f1d7c0d3750 0x7f1d7c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d8279c700 1 --2- 172.21.15.60:0/3228504559 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 0x7f1d7c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d80f99700 1 -- 172.21.15.60:0/3228504559 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283240767 0 0) 0x7f1d78001110 con 0x7f1d7c0b8400 2023-07-13T22:28:11.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d80f99700 1 -- 172.21.15.60:0/3228504559 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d680035b0 con 0x7f1d7c0b8400 2023-07-13T22:28:11.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d8179a700 1 -- 172.21.15.60:0/3228504559 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 msgr2=0x7f1d7c0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d8179a700 1 --2- 172.21.15.60:0/3228504559 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 0x7f1d7c0d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d8179a700 1 -- 172.21.15.60:0/3228504559 >> v1:172.21.15.60:6789/0 conn(0x7f1d7c0b8400 legacy=0x7f1d7c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.726+0000 7f1d8179a700 1 -- 172.21.15.60:0/3228504559 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0b87d0 con 0x7f1d7c0d3750 2023-07-13T22:28:11.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.727+0000 7f1d8179a700 1 --2- 172.21.15.60:0/3228504559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0d3750 0x7f1d7c0cf590 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009840 tx=0x7f1d6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4c3c0d5ac7a77a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.727+0000 7f1d80f99700 1 -- 172.21.15.60:0/3228504559 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c011830 con 0x7f1d7c0d3750 2023-07-13T22:28:11.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.727+0000 7f1d80f99700 1 -- 172.21.15.60:0/3228504559 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c011990 con 0x7f1d7c0d3750 2023-07-13T22:28:11.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.727+0000 7f1d80f99700 1 -- 172.21.15.60:0/3228504559 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c011d20 con 0x7f1d7c0d3750 2023-07-13T22:28:11.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.727+0000 7f1d841ff700 1 -- 172.21.15.60:0/3228504559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0d3750 msgr2=0x7f1d7c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.727+0000 7f1d841ff700 1 --2- 172.21.15.60:0/3228504559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0d3750 0x7f1d7c0cf590 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009840 tx=0x7f1d6c009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:11.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.728+0000 7f1d841ff700 1 -- 172.21.15.60:0/3228504559 shutdown_connections 2023-07-13T22:28:11.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.728+0000 7f1d841ff700 1 --2- 172.21.15.60:0/3228504559 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 0x7f1d7c0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.728+0000 7f1d841ff700 1 --2- 172.21.15.60:0/3228504559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0d3750 0x7f1d7c0cf590 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.728+0000 7f1d841ff700 1 -- 172.21.15.60:0/3228504559 >> 172.21.15.60:0/3228504559 conn(0x7f1d7c06b390 msgr2=0x7f1d7c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:11.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.728+0000 7f1d841ff700 1 -- 172.21.15.60:0/3228504559 shutdown_connections 2023-07-13T22:28:11.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.728+0000 7f1d841ff700 1 -- 172.21.15.60:0/3228504559 wait complete. 2023-07-13T22:28:11.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.730+0000 7f1d841ff700 1 Processor -- start 2023-07-13T22:28:11.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.730+0000 7f1d841ff700 1 -- start start 2023-07-13T22:28:11.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.731+0000 7f1d841ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0b8400 0x7f1d7c0c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.731+0000 7f1d841ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 0x7f1d7c150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d841ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d7c0d3750 0x7f1d7c0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d841ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0c7600 con 0x7f1d7c0b8400 2023-07-13T22:28:11.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d841ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0c8750 con 0x7f1d7c0d3750 2023-07-13T22:28:11.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d841ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0c8a30 con 0x7f1d7c0cfad0 2023-07-13T22:28:11.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d81f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0b8400 0x7f1d7c0c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d81f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0b8400 0x7f1d7c0c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60992/0 (socket says 172.21.15.60:60992) 2023-07-13T22:28:11.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d81f9b700 1 -- 172.21.15.60:0/2099587742 learned_addr learned my addr 172.21.15.60:0/2099587742 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:11.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d8179a700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 0x7f1d7c150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d8279c700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d7c0d3750 0x7f1d7c0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d81f9b700 1 -- 172.21.15.60:0/2099587742 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 msgr2=0x7f1d7c150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 0x7f1d7c150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d81f9b700 1 -- 172.21.15.60:0/2099587742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d7c0d3750 msgr2=0x7f1d7c0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d7c0d3750 0x7f1d7c0b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d81f9b700 1 -- 172.21.15.60:0/2099587742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c0094a0 con 0x7f1d7c0b8400 2023-07-13T22:28:11.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.732+0000 7f1d8179a700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 0x7f1d7c150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:11.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.733+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0b8400 0x7f1d7c0c70c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1d78007fd0 tx=0x7f1d7800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.733+0000 7f1d727fc700 1 -- 172.21.15.60:0/2099587742 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78005d70 con 0x7f1d7c0b8400 2023-07-13T22:28:11.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.733+0000 7f1d727fc700 1 -- 172.21.15.60:0/2099587742 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d780070d0 con 0x7f1d7c0b8400 2023-07-13T22:28:11.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.733+0000 7f1d841ff700 1 -- 172.21.15.60:0/2099587742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c0c8d10 con 0x7f1d7c0b8400 2023-07-13T22:28:11.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.733+0000 7f1d727fc700 1 -- 172.21.15.60:0/2099587742 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78007460 con 0x7f1d7c0b8400 2023-07-13T22:28:11.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.733+0000 7f1d841ff700 1 -- 172.21.15.60:0/2099587742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c1514b0 con 0x7f1d7c0b8400 2023-07-13T22:28:11.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.735+0000 7f1d727fc700 1 -- 172.21.15.60:0/2099587742 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d780075c0 con 0x7f1d7c0b8400 2023-07-13T22:28:11.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.736+0000 7f1d727fc700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1d6006ace0 0x7f1d6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.736+0000 7f1d727fc700 1 -- 172.21.15.60:0/2099587742 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d780888f0 con 0x7f1d7c0b8400 2023-07-13T22:28:11.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.736+0000 7f1d8179a700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1d6006ace0 0x7f1d6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.737+0000 7f1d8179a700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1d6006ace0 0x7f1d6006d190 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0d3b60 tx=0x7f1d6c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.738+0000 7f1d97b4fb80 1 Processor -- start 2023-07-13T22:28:11.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.738+0000 7f1d97b4fb80 1 -- start start 2023-07-13T22:28:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd439dc0 0x5594bd467b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd437c50 0x5594bd452120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bd3b1ea0 con 0x5594bd437880 2023-07-13T22:28:11.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bd325600 con 0x5594bd437c50 2023-07-13T22:28:11.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594bd394910 con 0x5594bd439dc0 2023-07-13T22:28:11.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d81f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d81f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4675e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60996/0 (socket says 172.21.15.60:60996) 2023-07-13T22:28:11.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d81f9b700 1 -- 172.21.15.60:0/4057290547 learned_addr learned my addr 172.21.15.60:0/4057290547 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:11.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d81f9b700 1 -- 172.21.15.60:0/4057290547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd439dc0 msgr2=0x5594bd467b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/4057290547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd439dc0 0x5594bd467b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d81f9b700 1 -- 172.21.15.60:0/4057290547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd437c50 msgr2=0x5594bd452120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/4057290547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd437c50 0x5594bd452120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.739+0000 7f1d81f9b700 1 -- 172.21.15.60:0/4057290547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594bd181790 con 0x5594bd437880 2023-07-13T22:28:11.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.740+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/4057290547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4675e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1d780572c0 tx=0x7f1d78007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536a3e79faafd96d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.740+0000 7f1d5f7fe700 1 -- 172.21.15.60:0/4057290547 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7805f910 con 0x5594bd437880 2023-07-13T22:28:11.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.740+0000 7f1d5f7fe700 1 -- 172.21.15.60:0/4057290547 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78055960 con 0x5594bd437880 2023-07-13T22:28:11.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.740+0000 7f1d5f7fe700 1 -- 172.21.15.60:0/4057290547 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7805f680 con 0x5594bd437880 2023-07-13T22:28:11.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.740+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4057290547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 msgr2=0x5594bd4675e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.741+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4057290547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4675e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1d780572c0 tx=0x7f1d78007e30 comp rx=0 tx=0).stop 2023-07-13T22:28:11.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.741+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4057290547 shutdown_connections 2023-07-13T22:28:11.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.741+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4057290547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4675e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.741+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4057290547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd437c50 0x5594bd452120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.741+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4057290547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd439dc0 0x5594bd467b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.741+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4057290547 >> 172.21.15.60:0/4057290547 conn(0x5594bd4323b0 msgr2=0x5594bd43d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:11.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.741+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4057290547 shutdown_connections 2023-07-13T22:28:11.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.741+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4057290547 wait complete. 2023-07-13T22:28:11.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.742+0000 7f1d97b4fb80 1 -- start start 2023-07-13T22:28:11.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.742+0000 7f1d97b4fb80 1 Processor -- start 2023-07-13T22:28:11.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 0x5594bd494990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd46b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 0x5594bd495110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bd38dbe0 con 0x5594bd437c50 2023-07-13T22:28:11.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bd34b5b0 con 0x5594bd439dc0 2023-07-13T22:28:11.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594bd331910 con 0x5594bd437880 2023-07-13T22:28:11.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d97b4fb80 10 client.0 _ll_register_callbacks cb 0x5594bd349b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:28:11.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d8179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd46b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d8179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd46b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32768/0 (socket says 172.21.15.60:32768) 2023-07-13T22:28:11.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d8179a700 1 -- 172.21.15.60:0/4112672667 learned_addr learned my addr 172.21.15.60:0/4112672667 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:11.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 0x5594bd494990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.743+0000 7f1d8279c700 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 0x5594bd495110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d8179a700 1 -- 172.21.15.60:0/4112672667 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 msgr2=0x5594bd494990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d8179a700 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 0x5594bd494990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d8179a700 1 -- 172.21.15.60:0/4112672667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 msgr2=0x5594bd495110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d8179a700 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 0x5594bd495110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d8179a700 1 -- 172.21.15.60:0/4112672667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c1514b0 con 0x5594bd437c50 2023-07-13T22:28:11.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d8179a700 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd46b310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c06c720 tx=0x7f1d6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a195b0ea4ee72c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:28:11.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d5cff9700 1 -- 172.21.15.60:0/4112672667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c02b070 con 0x5594bd437c50 2023-07-13T22:28:11.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4112672667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594bd42f9c0 con 0x5594bd437c50 2023-07-13T22:28:11.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d5cff9700 1 -- 172.21.15.60:0/4112672667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d6c0038b0 con 0x5594bd437c50 2023-07-13T22:28:11.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d5cff9700 1 -- 172.21.15.60:0/4112672667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d6c01f070 con 0x5594bd437c50 2023-07-13T22:28:11.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d97b4fb80 10 client.7569 fetch_fsmap learned FSMap version 52 2023-07-13T22:28:11.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.744+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4112672667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x5594bd26e0a0 con 0x5594bd437c50 2023-07-13T22:28:11.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.745+0000 7f1d5cff9700 1 -- 172.21.15.60:0/4112672667 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594bd42f9c0 con 0x5594bd437c50 2023-07-13T22:28:11.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.745+0000 7f1d5cff9700 1 -- 172.21.15.60:0/4112672667 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 52) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1d6c003a70 con 0x5594bd437c50 2023-07-13T22:28:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.745+0000 7f1d97b4fb80 10 client.7569 fetch_fsmap finished waiting for FSMap version 52 2023-07-13T22:28:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.745+0000 7f1d97b4fb80 10 client.7569 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:28:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.745+0000 7f1d97b4fb80 20 client.7569 populate_metadata read hostname 'smithi060' 2023-07-13T22:28:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.745+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] conn(0x5594bd47ecf0 0x5594bd47f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.745+0000 7f1d97b4fb80 4 client.7569 mds_command: new command op to 7434 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:28:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.745+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4112672667 --> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594bd40c170 con 0x5594bd47ecf0 2023-07-13T22:28:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.745+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] conn(0x5594bd47ecf0 0x5594bd47f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.746+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] conn(0x5594bd47ecf0 0x5594bd47f0c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd621e13f3c3bd3b server_cookie=7531280187b01563 in_seq=0 out_seq=0 2023-07-13T22:28:11.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.746+0000 7f1d5cff9700 10 client.7569 ms_handle_connect on v2:172.21.15.60:6835/1512512599 2023-07-13T22:28:11.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.746+0000 7f1d5cff9700 1 -- 172.21.15.60:0/4112672667 <== mds.0 v2:172.21.15.60:6835/1512512599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5594bd40c170 con 0x5594bd47ecf0 2023-07-13T22:28:11.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.746+0000 7f1d5cff9700 10 client.7569 handle_command_reply: tid=0 2023-07-13T22:28:11.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.746+0000 7f1d97b4fb80 1 client.7569 shutdown 2023-07-13T22:28:11.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4112672667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 msgr2=0x5594bd46b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd46b310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c06c720 tx=0x7f1d6c004750 comp rx=0 tx=0).stop 2023-07-13T22:28:11.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4112672667 shutdown_connections 2023-07-13T22:28:11.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] conn(0x5594bd47ecf0 0x5594bd47f0c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 0x5594bd495110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 0x5594bd494990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d5cff9700 0 client.7569 ms_handle_reset on v2:172.21.15.60:6835/1512512599 2023-07-13T22:28:11.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/4112672667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd46b310 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4112672667 >> 172.21.15.60:0/4112672667 conn(0x5594bd4323b0 msgr2=0x5594bd43cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:11.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4112672667 shutdown_connections 2023-07-13T22:28:11.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/4112672667 wait complete. 2023-07-13T22:28:11.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.747+0000 7f1d97b4fb80 20 client.7569 trim_cache size 0 max 16384 2023-07-13T22:28:11.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.765+0000 7f1d97b4fb80 1 Processor -- start 2023-07-13T22:28:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.766+0000 7f1d97b4fb80 1 -- start start 2023-07-13T22:28:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.766+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 0x5594bd47fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.766+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd4947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.766+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 0x5594bd494ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.766+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bd437630 con 0x5594bd437c50 2023-07-13T22:28:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.766+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bd182390 con 0x5594bd439dc0 2023-07-13T22:28:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.766+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594bd484ff0 con 0x5594bd437880 2023-07-13T22:28:11.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd4947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8179a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd4947a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32784/0 (socket says 172.21.15.60:32784) 2023-07-13T22:28:11.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8179a700 1 -- 172.21.15.60:0/2930081744 learned_addr learned my addr 172.21.15.60:0/2930081744 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:11.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 0x5594bd47fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8179a700 1 -- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 msgr2=0x5594bd47fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8179a700 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 0x5594bd47fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8279c700 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 0x5594bd494ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8179a700 1 -- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 msgr2=0x5594bd494ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8179a700 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 0x5594bd494ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8179a700 1 -- 172.21.15.60:0/2930081744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594bd353b10 con 0x5594bd437c50 2023-07-13T22:28:11.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 0x5594bd47fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:11.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8279c700 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 0x5594bd494ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:11.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.767+0000 7f1d8179a700 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd4947a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c004790 tx=0x7f1d6c02b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fcd9b64c82b91d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d5ffff700 1 -- 172.21.15.60:0/2930081744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c01f050 con 0x5594bd437c50 2023-07-13T22:28:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d5ffff700 1 -- 172.21.15.60:0/2930081744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c0038b0 con 0x5594bd437c50 2023-07-13T22:28:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d5ffff700 1 -- 172.21.15.60:0/2930081744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c02f410 con 0x5594bd437c50 2023-07-13T22:28:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/2930081744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 msgr2=0x5594bd4947a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd4947a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c004790 tx=0x7f1d6c02b000 comp rx=0 tx=0).stop 2023-07-13T22:28:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/2930081744 shutdown_connections 2023-07-13T22:28:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd439dc0 0x5594bd494ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd437880 0x5594bd47fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/2930081744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437c50 0x5594bd4947a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/2930081744 >> 172.21.15.60:0/2930081744 conn(0x5594bd4323b0 msgr2=0x5594bd433920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:11.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/2930081744 shutdown_connections 2023-07-13T22:28:11.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.768+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/2930081744 wait complete. 2023-07-13T22:28:11.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.769+0000 7f1d97b4fb80 1 -- start start 2023-07-13T22:28:11.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.769+0000 7f1d97b4fb80 1 Processor -- start 2023-07-13T22:28:11.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4ed150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd437c50 0x5594bd4e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d97b4fb80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd439dc0 0x5594bd4e9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bd491ec0 con 0x5594bd437880 2023-07-13T22:28:11.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594bd4e9ea0 con 0x5594bd437c50 2023-07-13T22:28:11.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d97b4fb80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594bd4916b0 con 0x5594bd439dc0 2023-07-13T22:28:11.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d97b4fb80 10 client.0 _ll_register_callbacks cb 0x7f1d6c011c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:28:11.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d81f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d81f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4ed150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32796/0 (socket says 172.21.15.60:32796) 2023-07-13T22:28:11.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.770+0000 7f1d81f9b700 1 -- 172.21.15.60:0/307561998 learned_addr learned my addr 172.21.15.60:0/307561998 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:11.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d81f9b700 1 -- 172.21.15.60:0/307561998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd439dc0 msgr2=0x5594bd4e9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd439dc0 0x5594bd4e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d81f9b700 1 -- 172.21.15.60:0/307561998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd437c50 msgr2=0x5594bd4e90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd437c50 0x5594bd4e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d81f9b700 1 -- 172.21.15.60:0/307561998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594bd26e0a0 con 0x5594bd437880 2023-07-13T22:28:11.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d81f9b700 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4ed150 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1d78001270 tx=0x7f1d7801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404f8d9fcc78d72b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:28:11.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d5cff9700 1 -- 172.21.15.60:0/307561998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78086040 con 0x5594bd437880 2023-07-13T22:28:11.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d5cff9700 1 -- 172.21.15.60:0/307561998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d7805f680 con 0x5594bd437880 2023-07-13T22:28:11.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/307561998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d78055960 con 0x5594bd437880 2023-07-13T22:28:11.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d5cff9700 1 -- 172.21.15.60:0/307561998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d7805b0b0 con 0x5594bd437880 2023-07-13T22:28:11.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d97b4fb80 10 client.7575 fetch_fsmap learned FSMap version 52 2023-07-13T22:28:11.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.771+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/307561998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x5594bd1e21d0 con 0x5594bd437880 2023-07-13T22:28:11.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d5cff9700 1 -- 172.21.15.60:0/307561998 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78055960 con 0x5594bd437880 2023-07-13T22:28:11.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d5cff9700 1 -- 172.21.15.60:0/307561998 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 52) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1d7805b210 con 0x5594bd437880 2023-07-13T22:28:11.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d97b4fb80 10 client.7575 fetch_fsmap finished waiting for FSMap version 52 2023-07-13T22:28:11.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d97b4fb80 10 client.7575 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:28:11.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d97b4fb80 20 client.7575 populate_metadata read hostname 'smithi060' 2023-07-13T22:28:11.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] conn(0x5594bd4cf1e0 0x5594bd4947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d97b4fb80 4 client.7575 mds_command: new command op to 7434 tid=0[{"prefix": "status"}] 2023-07-13T22:28:11.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/307561998 --> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5594bd386d20 con 0x5594bd4cf1e0 2023-07-13T22:28:11.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d8179a700 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] conn(0x5594bd4cf1e0 0x5594bd4947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d8179a700 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] conn(0x5594bd4cf1e0 0x5594bd4947a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b9090edf1f619e9 server_cookie=ddc4d0185164248d in_seq=0 out_seq=0 2023-07-13T22:28:11.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.772+0000 7f1d5cff9700 10 client.7575 ms_handle_connect on v2:172.21.15.60:6835/1512512599 2023-07-13T22:28:11.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d5cff9700 1 -- 172.21.15.60:0/307561998 <== mds.0 v2:172.21.15.60:6835/1512512599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5594bd386d20 con 0x5594bd4cf1e0 2023-07-13T22:28:11.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d5cff9700 10 client.7575 handle_command_reply: tid=0 2023-07-13T22:28:11.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 client.7575 shutdown 2023-07-13T22:28:11.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/307561998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 msgr2=0x5594bd4ed150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4ed150 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1d78001270 tx=0x7f1d7801d000 comp rx=0 tx=0).stop 2023-07-13T22:28:11.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/307561998 shutdown_connections 2023-07-13T22:28:11.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.60:6835/1512512599,v1:172.21.15.60:6837/1512512599] conn(0x5594bd4cf1e0 0x5594bd4947a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594bd437880 0x5594bd4ed150 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5594bd439dc0 0x5594bd4e9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d5cff9700 0 client.7575 ms_handle_reset on v2:172.21.15.60:6835/1512512599 2023-07-13T22:28:11.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 --2- 172.21.15.60:0/307561998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5594bd437c50 0x5594bd4e90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/307561998 >> 172.21.15.60:0/307561998 conn(0x5594bd4323b0 msgr2=0x5594bd4681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:11.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/307561998 shutdown_connections 2023-07-13T22:28:11.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 1 -- 172.21.15.60:0/307561998 wait complete. 2023-07-13T22:28:11.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.773+0000 7f1d97b4fb80 20 client.7575 trim_cache size 0 max 16384 2023-07-13T22:28:11.762 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:28:11.762 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:28:11.762 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:28:11.762 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7434, 2023-07-13T22:28:11.762 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:28:11.762 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:28:11.763 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:28:11.763 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.6715735489999997, 2023-07-13T22:28:11.763 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 52, 2023-07-13T22:28:11.763 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2023-07-13T22:28:11.763 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2023-07-13T22:28:11.763 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 13.557831572 2023-07-13T22:28:11.763 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:28:11.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 -- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1d6006ace0 msgr2=0x7f1d6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1d6006ace0 0x7f1d6006d190 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0d3b60 tx=0x7f1d6c008030 comp rx=0 tx=0).stop 2023-07-13T22:28:11.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 -- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0b8400 msgr2=0x7f1d7c0c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0b8400 0x7f1d7c0c70c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1d78007fd0 tx=0x7f1d7800c860 comp rx=0 tx=0).stop 2023-07-13T22:28:11.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 -- 172.21.15.60:0/2099587742 shutdown_connections 2023-07-13T22:28:11.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1d6006ace0 0x7f1d6006d190 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d7c0b8400 0x7f1d7c0c70c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d7c0d3750 0x7f1d7c0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 --2- 172.21.15.60:0/2099587742 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d7c0cfad0 0x7f1d7c150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 -- 172.21.15.60:0/2099587742 >> 172.21.15.60:0/2099587742 conn(0x7f1d7c06b390 msgr2=0x7f1d7c0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:11.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 -- 172.21.15.60:0/2099587742 shutdown_connections 2023-07-13T22:28:11.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.774+0000 7f1d841ff700 1 -- 172.21.15.60:0/2099587742 wait complete. 2023-07-13T22:28:11.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:28:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.995+0000 7fa6ccdff700 1 Processor -- start 2023-07-13T22:28:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.996+0000 7fa6ccdff700 1 -- start start 2023-07-13T22:28:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6ccdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6c80b8400 0x7fa6c80cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6ccdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 0x7fa6c80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6ccdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c80cff20 con 0x7fa6c80cfad0 2023-07-13T22:28:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6ccdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c80d0ac0 con 0x7fa6c80b8400 2023-07-13T22:28:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6ccdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80b87d0 con 0x7fa6c80d3750 2023-07-13T22:28:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6c659c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6c80b8400 0x7fa6c80cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6bffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 0x7fa6c80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6c659c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6c80b8400 0x7fa6c80cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:41870/0 (socket says 172.21.15.60:41870) 2023-07-13T22:28:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6c659c700 1 -- 172.21.15.60:0/3858388103 learned_addr learned my addr 172.21.15.60:0/3858388103 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.997+0000 7fa6c6d9d700 1 --1- 172.21.15.60:0/3858388103 >> v1:172.21.15.105:6789/0 conn(0x7fa6c80cfad0 0x7fa6c815fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:46250/0 (socket says 172.21.15.60:46250) 2023-07-13T22:28:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.998+0000 7fa6c659c700 1 -- 172.21.15.60:0/3858388103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 msgr2=0x7fa6c80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.998+0000 7fa6c659c700 1 --2- 172.21.15.60:0/3858388103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 0x7fa6c80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.998+0000 7fa6c659c700 1 -- 172.21.15.60:0/3858388103 >> v1:172.21.15.105:6789/0 conn(0x7fa6c80cfad0 legacy=0x7fa6c815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.998+0000 7fa6c5d9b700 1 -- 172.21.15.60:0/3858388103 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589008346 0 0) 0x7fa6c0001490 con 0x7fa6c80cfad0 2023-07-13T22:28:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.998+0000 7fa6c659c700 1 -- 172.21.15.60:0/3858388103 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c80b4eb0 con 0x7fa6c80b8400 2023-07-13T22:28:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.998+0000 7fa6c659c700 1 --2- 172.21.15.60:0/3858388103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c80b8400 0x7fa6c80cd240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80098f0 tx=0x7fa6b8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb2cdbc40f8b51e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.998+0000 7fa6c5d9b700 1 -- 172.21.15.60:0/3858388103 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8011830 con 0x7fa6c80b8400 2023-07-13T22:28:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.998+0000 7fa6c5d9b700 1 -- 172.21.15.60:0/3858388103 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b8011990 con 0x7fa6c80b8400 2023-07-13T22:28:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.998+0000 7fa6c5d9b700 1 -- 172.21.15.60:0/3858388103 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8011d20 con 0x7fa6c80b8400 2023-07-13T22:28:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.999+0000 7fa6ccdff700 1 -- 172.21.15.60:0/3858388103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c80b8400 msgr2=0x7fa6c80cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.999+0000 7fa6ccdff700 1 --2- 172.21.15.60:0/3858388103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c80b8400 0x7fa6c80cd240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80098f0 tx=0x7fa6b8009c00 comp rx=0 tx=0).stop 2023-07-13T22:28:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.999+0000 7fa6ccdff700 1 -- 172.21.15.60:0/3858388103 shutdown_connections 2023-07-13T22:28:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.999+0000 7fa6ccdff700 1 --2- 172.21.15.60:0/3858388103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c80b8400 0x7fa6c80cd240 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.999+0000 7fa6ccdff700 1 --2- 172.21.15.60:0/3858388103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 0x7fa6c80cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.999+0000 7fa6ccdff700 1 -- 172.21.15.60:0/3858388103 >> 172.21.15.60:0/3858388103 conn(0x7fa6c806b390 msgr2=0x7fa6c806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:11.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.999+0000 7fa6ccdff700 1 -- 172.21.15.60:0/3858388103 shutdown_connections 2023-07-13T22:28:11.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:11.999+0000 7fa6ccdff700 1 -- 172.21.15.60:0/3858388103 wait complete. 2023-07-13T22:28:11.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.001+0000 7fa6ccdff700 1 Processor -- start 2023-07-13T22:28:11.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.001+0000 7fa6ccdff700 1 -- start start 2023-07-13T22:28:11.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6ccdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c80b8400 0x7fa6c80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6ccdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c80cfad0 0x7fa6c8154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6ccdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 0x7fa6c8155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6ccdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8155f60 con 0x7fa6c80cfad0 2023-07-13T22:28:11.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6ccdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8154530 con 0x7fa6c80b8400 2023-07-13T22:28:11.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6ccdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c81559a0 con 0x7fa6c80d3750 2023-07-13T22:28:11.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6bffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c80cfad0 0x7fa6c8154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6bffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c80cfad0 0x7fa6c8154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32810/0 (socket says 172.21.15.60:32810) 2023-07-13T22:28:11.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6bffff700 1 -- 172.21.15.60:0/1451603336 learned_addr learned my addr 172.21.15.60:0/1451603336 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:11.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.002+0000 7fa6c659c700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c80b8400 0x7fa6c80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6c6d9d700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 0x7fa6c8155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6bffff700 1 -- 172.21.15.60:0/1451603336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 msgr2=0x7fa6c8155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6bffff700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 0x7fa6c8155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6bffff700 1 -- 172.21.15.60:0/1451603336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c80b8400 msgr2=0x7fa6c80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:11.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6bffff700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c80b8400 0x7fa6c80b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:11.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6bffff700 1 -- 172.21.15.60:0/1451603336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b8009550 con 0x7fa6c80cfad0 2023-07-13T22:28:11.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6bffff700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c80cfad0 0x7fa6c8154e30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000e840 tx=0x7fa6b000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6beffd700 1 -- 172.21.15.60:0/1451603336 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0009590 con 0x7fa6c80cfad0 2023-07-13T22:28:11.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6beffd700 1 -- 172.21.15.60:0/1451603336 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b0013680 con 0x7fa6c80cfad0 2023-07-13T22:28:11.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c8155c80 con 0x7fa6c80cfad0 2023-07-13T22:28:11.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6beffd700 1 -- 172.21.15.60:0/1451603336 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0013a30 con 0x7fa6c80cfad0 2023-07-13T22:28:11.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.003+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c8156590 con 0x7fa6c80cfad0 2023-07-13T22:28:11.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.004+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8156750 con 0x7fa6c80cfad0 2023-07-13T22:28:11.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.005+0000 7fa6beffd700 1 -- 172.21.15.60:0/1451603336 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa6b0013b90 con 0x7fa6c80cfad0 2023-07-13T22:28:11.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.006+0000 7fa6beffd700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6a806ace0 0x7fa6a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:11.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.006+0000 7fa6beffd700 1 -- 172.21.15.60:0/1451603336 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa6b00893b0 con 0x7fa6c80cfad0 2023-07-13T22:28:11.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.007+0000 7fa6c659c700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6a806ace0 0x7fa6a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:11.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.007+0000 7fa6c659c700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6a806ace0 0x7fa6a806d190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8014040 tx=0x7fa6b8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:11.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.011+0000 7fa6beffd700 1 -- 172.21.15.60:0/1451603336 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa6b0057e50 con 0x7fa6c80cfad0 2023-07-13T22:28:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.266+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa6c81560f0 con 0x7fa6c80cfad0 2023-07-13T22:28:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.266+0000 7fa6beffd700 1 -- 172.21.15.60:0/1451603336 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa6b00579e0 con 0x7fa6c80cfad0 2023-07-13T22:28:12.247 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:28:12.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.268+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6a806ace0 msgr2=0x7fa6a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:12.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.268+0000 7fa6ccdff700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6a806ace0 0x7fa6a806d190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8014040 tx=0x7fa6b8008030 comp rx=0 tx=0).stop 2023-07-13T22:28:12.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.268+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c80cfad0 msgr2=0x7fa6c8154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:12.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.268+0000 7fa6ccdff700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c80cfad0 0x7fa6c8154e30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000e840 tx=0x7fa6b000eb50 comp rx=0 tx=0).stop 2023-07-13T22:28:12.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.269+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 shutdown_connections 2023-07-13T22:28:12.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.269+0000 7fa6ccdff700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6a806ace0 0x7fa6a806d190 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.269+0000 7fa6ccdff700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6c80d3750 0x7fa6c8155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.269+0000 7fa6ccdff700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6c80b8400 0x7fa6c80b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.269+0000 7fa6ccdff700 1 --2- 172.21.15.60:0/1451603336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c80cfad0 0x7fa6c8154e30 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.269+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 >> 172.21.15.60:0/1451603336 conn(0x7fa6c806b390 msgr2=0x7fa6c80cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.269+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 shutdown_connections 2023-07-13T22:28:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:12.269+0000 7fa6ccdff700 1 -- 172.21.15.60:0/1451603336 wait complete. 2023-07-13T22:28:12.267 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:28:12.267 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:28:12.268 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:28:12.268 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:28:12.268 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:12.269 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:28:12.269 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:28:12.269 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:28:12.269 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:28:12.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:12.270 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:28:12.317 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:28:12.317 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:28:12.317 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:28:12.317 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:12.317 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:28:12.318 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:12.318 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:28:12.318 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:12.318 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:28:12.318 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:28:12.319 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:12.319 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:28:12.319 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:28:12.319 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:12.319 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:28:12.319 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:28:12.319 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41281sec preferred_lft 41281sec 2023-07-13T22:28:12.320 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:28:12.320 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:12.320 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:28:12.320 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:12.320 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:28:12.320 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:28:12.320 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:28:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:28:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:12.321 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:28:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:28:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:28:12.321 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:12.322 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:28:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:28:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:28:12.322 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:12.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:12.324 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:28:12.431 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:28:12.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:12.434 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:28:12.483 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:28:12.483 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:28:12.483 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:28:12.483 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:28:12.484 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:28:12.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:12.486 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:28:12.486 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:28:12.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:28:12.487 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:28:12.487 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:28:12.487 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:28:12.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:12.661 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:28:12.708 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:28:12.709 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:28:12.709 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:28:12.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:12.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:28:12.821 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:28:12.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:28:12.821 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:12.867 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:28:12.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:28:12.868 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:12.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:28:13.065 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:28:13.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:28:13.137 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:28:13.140 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:28:13.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:13.141 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:28:13.187 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:28:13.188 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:28:13.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:28:13.267 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:28:13.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:28:13.340 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:28:13.342 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:28:13.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:13.343 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:28:13.372 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:28:13.392+0000 7f227d6e25c0 -1 init, newargv = 0x55633cb25c00 newargc=15 2023-07-13T22:28:13.372 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[147642]: starting ceph client 2023-07-13T22:28:13.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[147642]: starting fuse 2023-07-13T22:28:13.397 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:28:13.398 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:28:13.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:28:13.399 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:28:13.399 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:28:13.399 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:28:13.400 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:28:13.400 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:13.400 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:28:13.401 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:28:13.401 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:28:13.401 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:28:13.402 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:13.402 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:28:13.402 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:28:13.403 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:28:13.403 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:28:13.403 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:13.403 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:28:13.404 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:28:13.404 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:28:13.404 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:28:13.405 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:28:13.405 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:28:13.405 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:28:13.406 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:28:13.406 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:13.406 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:28:13.407 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:28:13.586 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.147642.asok 2023-07-13T22:28:13.763 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.147642.asok 2023-07-13T22:28:13.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:13.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.147642.asok status 2023-07-13T22:28:13.995 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:28:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:28:13.995 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:28:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:28:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:28:13.996 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:28:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:28:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "147642", 2023-07-13T22:28:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:28:13.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:28:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:28:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7584, 2023-07-13T22:28:13.998 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:28:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:28:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:28:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 7584 2023-07-13T22:28:13.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:28:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:28:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:28:14.000 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 435172672 2023-07-13T22:28:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:14.001 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:28:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:28:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:28:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 435172672 2023-07-13T22:28:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.7584 192.168.0.1:0/435172672", 2023-07-13T22:28:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/435172672", 2023-07-13T22:28:14.002 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:28:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 52, 2023-07-13T22:28:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 87, 2023-07-13T22:28:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:28:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:28:14.003 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:28:14.003 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:28:14.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:14.010 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:28:14.011 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:14.059 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:28:14.059 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:14.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:14.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:14.131 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:28:14.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.354+0000 7fe9e25ff700 1 Processor -- start 2023-07-13T22:28:14.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.354+0000 7fe9e25ff700 1 -- start start 2023-07-13T22:28:14.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.356+0000 7fe9e25ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe9dc158870 0x7fe9dc0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:14.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.356+0000 7fe9e25ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9dc0c52c0 0x7fe9dc069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:14.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.356+0000 7fe9e25ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9dc06a830 con 0x7fe9dc06a410 2023-07-13T22:28:14.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.356+0000 7fe9e25ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc0c3fc0 con 0x7fe9dc0c52c0 2023-07-13T22:28:14.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.356+0000 7fe9e25ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc0c42a0 con 0x7fe9dc158870 2023-07-13T22:28:14.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.356+0000 7fe9db7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9dc0c52c0 0x7fe9dc069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:14.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.356+0000 7fe9db7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9dc0c52c0 0x7fe9dc069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32816/0 (socket says 172.21.15.60:32816) 2023-07-13T22:28:14.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.356+0000 7fe9db7fe700 1 -- 172.21.15.60:0/1739343591 learned_addr learned my addr 172.21.15.60:0/1739343591 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:14.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.356+0000 7fe9dbfff700 1 --2- 172.21.15.60:0/1739343591 >> v2:172.21.15.105:3300/0 conn(0x7fe9dc158870 0x7fe9dc0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:14.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.357+0000 7fe9db7fe700 1 -- 172.21.15.60:0/1739343591 >> v2:172.21.15.105:3300/0 conn(0x7fe9dc158870 msgr2=0x7fe9dc0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:14.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.357+0000 7fe9db7fe700 1 --2- 172.21.15.60:0/1739343591 >> v2:172.21.15.105:3300/0 conn(0x7fe9dc158870 0x7fe9dc0c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.357+0000 7fe9db7fe700 1 -- 172.21.15.60:0/1739343591 >> v1:172.21.15.105:6789/0 conn(0x7fe9dc06a410 legacy=0x7fe9dc0c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:14.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.357+0000 7fe9db7fe700 1 -- 172.21.15.60:0/1739343591 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc0c4580 con 0x7fe9dc0c52c0 2023-07-13T22:28:14.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.357+0000 7fe9dbfff700 1 --2- 172.21.15.60:0/1739343591 >> v2:172.21.15.105:3300/0 conn(0x7fe9dc158870 0x7fe9dc0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:14.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.357+0000 7fe9db7fe700 1 --2- 172.21.15.60:0/1739343591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc0c52c0 0x7fe9dc069ed0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc06fe20 tx=0x7fe9d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf1828e7da27f4f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:14.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.357+0000 7fe9daffd700 1 -- 172.21.15.60:0/1739343591 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d000b870 con 0x7fe9dc0c52c0 2023-07-13T22:28:14.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.357+0000 7fe9daffd700 1 -- 172.21.15.60:0/1739343591 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d000b9d0 con 0x7fe9dc0c52c0 2023-07-13T22:28:14.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.357+0000 7fe9daffd700 1 -- 172.21.15.60:0/1739343591 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d000bd60 con 0x7fe9dc0c52c0 2023-07-13T22:28:14.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.358+0000 7fe9e25ff700 1 -- 172.21.15.60:0/1739343591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc0c52c0 msgr2=0x7fe9dc069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:14.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.358+0000 7fe9e25ff700 1 --2- 172.21.15.60:0/1739343591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc0c52c0 0x7fe9dc069ed0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc06fe20 tx=0x7fe9d0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.358+0000 7fe9e25ff700 1 -- 172.21.15.60:0/1739343591 shutdown_connections 2023-07-13T22:28:14.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.358+0000 7fe9e25ff700 1 --2- 172.21.15.60:0/1739343591 >> v2:172.21.15.105:3300/0 conn(0x7fe9dc158870 0x7fe9dc0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.358+0000 7fe9e25ff700 1 --2- 172.21.15.60:0/1739343591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc0c52c0 0x7fe9dc069ed0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.358+0000 7fe9e25ff700 1 -- 172.21.15.60:0/1739343591 >> 172.21.15.60:0/1739343591 conn(0x7fe9dc0cf410 msgr2=0x7fe9dc0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:14.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.358+0000 7fe9e25ff700 1 -- 172.21.15.60:0/1739343591 shutdown_connections 2023-07-13T22:28:14.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.358+0000 7fe9e25ff700 1 -- 172.21.15.60:0/1739343591 wait complete. 2023-07-13T22:28:14.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.360+0000 7fe9e25ff700 1 Processor -- start 2023-07-13T22:28:14.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.360+0000 7fe9e25ff700 1 -- start start 2023-07-13T22:28:14.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9e25ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9dc06a410 0x7fe9dc0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:14.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9e25ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9dc0c52c0 0x7fe9dc1f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:14.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9e25ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc158870 0x7fe9dc1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:14.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9e25ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0c6710 con 0x7fe9dc158870 2023-07-13T22:28:14.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9e25ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0c57c0 con 0x7fe9dc06a410 2023-07-13T22:28:14.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9e25ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0c5aa0 con 0x7fe9dc0c52c0 2023-07-13T22:28:14.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9e0b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc158870 0x7fe9dc1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:14.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9dbfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9dc06a410 0x7fe9dc0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:14.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9e0b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc158870 0x7fe9dc1f5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32828/0 (socket says 172.21.15.60:32828) 2023-07-13T22:28:14.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9e0b9c700 1 -- 172.21.15.60:0/3618452400 learned_addr learned my addr 172.21.15.60:0/3618452400 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:14.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.361+0000 7fe9db7fe700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9dc0c52c0 0x7fe9dc1f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:14.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9e0b9c700 1 -- 172.21.15.60:0/3618452400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9dc0c52c0 msgr2=0x7fe9dc1f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:14.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9e0b9c700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9dc0c52c0 0x7fe9dc1f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9e0b9c700 1 -- 172.21.15.60:0/3618452400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9dc06a410 msgr2=0x7fe9dc0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:14.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9e0b9c700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9dc06a410 0x7fe9dc0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9e0b9c700 1 -- 172.21.15.60:0/3618452400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d0009550 con 0x7fe9dc158870 2023-07-13T22:28:14.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9e0b9c700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc158870 0x7fe9dc1f5e40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400cc50 tx=0x7fe9d400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:14.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9d8ff9700 1 -- 172.21.15.60:0/3618452400 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4003c00 con 0x7fe9dc158870 2023-07-13T22:28:14.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9d8ff9700 1 -- 172.21.15.60:0/3618452400 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d4003d60 con 0x7fe9dc158870 2023-07-13T22:28:14.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9d8ff9700 1 -- 172.21.15.60:0/3618452400 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400aab0 con 0x7fe9dc158870 2023-07-13T22:28:14.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc0c5d80 con 0x7fe9dc158870 2023-07-13T22:28:14.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.362+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc141b60 con 0x7fe9dc158870 2023-07-13T22:28:14.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.363+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc0c8330 con 0x7fe9dc158870 2023-07-13T22:28:14.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.364+0000 7fe9d8ff9700 1 -- 172.21.15.60:0/3618452400 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe9d400ac10 con 0x7fe9dc158870 2023-07-13T22:28:14.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.365+0000 7fe9d8ff9700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe9c806ace0 0x7fe9c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:14.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.365+0000 7fe9d8ff9700 1 -- 172.21.15.60:0/3618452400 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9d40894b0 con 0x7fe9dc158870 2023-07-13T22:28:14.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.366+0000 7fe9dbfff700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe9c806ace0 0x7fe9c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:14.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.366+0000 7fe9dbfff700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe9c806ace0 0x7fe9c806d190 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc006f10 tx=0x7fe9cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:14.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.370+0000 7fe9d8ff9700 1 -- 172.21.15.60:0/3618452400 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe9dc0c8330 con 0x7fe9dc158870 2023-07-13T22:28:14.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.623+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe9dc0c8330 con 0x7fe9dc158870 2023-07-13T22:28:14.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.624+0000 7fe9d8ff9700 1 -- 172.21.15.60:0/3618452400 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe9dc0c8330 con 0x7fe9dc158870 2023-07-13T22:28:14.604 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:28:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe9c806ace0 msgr2=0x7fe9c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe9c806ace0 0x7fe9c806d190 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc006f10 tx=0x7fe9cc008040 comp rx=0 tx=0).stop 2023-07-13T22:28:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc158870 msgr2=0x7fe9dc1f5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc158870 0x7fe9dc1f5e40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400cc50 tx=0x7fe9d400cf60 comp rx=0 tx=0).stop 2023-07-13T22:28:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 shutdown_connections 2023-07-13T22:28:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe9c806ace0 0x7fe9c806d190 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9dc158870 0x7fe9dc1f5e40 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9dc06a410 0x7fe9dc0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 --2- 172.21.15.60:0/3618452400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9dc0c52c0 0x7fe9dc1f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 >> 172.21.15.60:0/3618452400 conn(0x7fe9dc0cf410 msgr2=0x7fe9dc06e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 shutdown_connections 2023-07-13T22:28:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:14.626+0000 7fe9e25ff700 1 -- 172.21.15.60:0/3618452400 wait complete. 2023-07-13T22:28:14.625 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:28:14.625 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:28:14.626 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:28:14.627 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:28:14.627 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:28:14.628 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:28:14.628 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:28:14.629 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:28:14.630 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:28:14.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:14.631 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:28:14.685 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:28:14.685 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:28:14.686 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:28:14.686 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:14.686 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:28:14.687 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:14.687 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:28:14.687 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:14.687 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:28:14.688 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:28:14.688 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:14.688 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:28:14.688 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:28:14.689 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:14.689 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:28:14.689 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:14.690 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:28:14.690 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41276sec preferred_lft 41276sec 2023-07-13T22:28:14.690 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:28:14.690 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:14.691 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:28:14.691 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:14.691 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:28:14.692 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:14.692 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:28:14.692 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:14.692 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:28:14.693 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:28:14.693 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:28:14.694 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:14.694 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:28:14.694 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:28:14.694 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::14e3:bfff:fea8:bf57/64 scope link 2023-07-13T22:28:14.695 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:14.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:14.695 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:28:14.765 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:28:14.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:14.769 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:28:14.817 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:28:14.818 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:28:14.818 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:28:14.818 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:28:14.818 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:28:14.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:14.820 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:28:14.820 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:28:14.820 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:28:14.821 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:28:14.821 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:28:14.821 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:28:14.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:14.991 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:28:15.038 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:28:15.038 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:28:15.039 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:28:15.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:15.041 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:28:15.111 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:28:15.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:28:15.112 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:28:15.158 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:28:15.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:28:15.159 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:28:15.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:28:15.353 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:28:15.354 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:28:15.423 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:28:15.425 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:28:15.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:15.428 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:28:15.474 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:28:15.475 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:28:15.520 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:28:15.591 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:28:15.591 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:28:15.660 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:28:15.662 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:28:15.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:15.663 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:28:15.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:28:15.705+0000 7f112962a5c0 -1 init, newargv = 0x5556f7a06f00 newargc=15 2023-07-13T22:28:15.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[124502]: starting ceph client 2023-07-13T22:28:15.693 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[124502]: starting fuse 2023-07-13T22:28:15.714 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:28:15.715 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:28:15.715 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:28:15.716 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:28:15.716 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:28:15.716 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:28:15.717 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:28:15.717 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:28:15.717 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:28:15.718 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:28:15.718 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:28:15.718 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:28:15.719 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:28:15.719 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:28:15.719 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:28:15.719 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:28:15.720 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:28:15.720 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:28:15.720 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:28:15.721 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:28:15.721 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:28:15.721 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:28:15.722 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:28:15.722 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:28:15.722 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:28:15.723 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:28:15.723 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:28:15.723 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:28:15.724 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:28:15.898 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.124502.asok 2023-07-13T22:28:16.076 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.124502.asok 2023-07-13T22:28:16.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:16.077 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.124502.asok status 2023-07-13T22:28:16.299 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:28:16.299 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:28:16.300 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:28:16.300 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:28:16.300 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:28:16.301 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:28:16.302 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:28:16.302 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "124502", 2023-07-13T22:28:16.302 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:28:16.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:28:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:28:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:28:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7117, 2023-07-13T22:28:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:28:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:28:16.304 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:28:16.304 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 7117 2023-07-13T22:28:16.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:28:16.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:28:16.305 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:28:16.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:28:16.305 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2299108023 2023-07-13T22:28:16.306 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:28:16.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:28:16.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:28:16.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:28:16.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:28:16.307 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2299108023 2023-07-13T22:28:16.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:28:16.307 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.7117 192.168.0.1:0/2299108023", 2023-07-13T22:28:16.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/2299108023", 2023-07-13T22:28:16.307 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:28:16.307 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 52, 2023-07-13T22:28:16.307 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 87, 2023-07-13T22:28:16.307 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:28:16.308 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:28:16.308 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:28:16.308 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:28:16.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:16.313 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:28:16.313 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:28:16.360 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:28:16.360 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:28:16.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:16.361 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:28:16.431 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:28:16.437 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:28:16.556 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-07-13T22:28:16.557 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-07-13T22:28:16.557 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:28:16.557 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-07-13T22:28:16.557 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.557 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-07-13T22:28:16.558 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.558 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-07-13T22:28:16.558 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.558 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-07-13T22:28:16.558 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-07-13T22:28:16.558 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.558 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-07-13T22:28:16.558 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-07-13T22:28:16.559 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.559 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:28:16.559 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-07-13T22:28:16.559 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.559 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:28:16.559 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:28:16.559 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.559 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-07-13T22:28:16.560 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.560 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-07-13T22:28:16.560 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-07-13T22:28:16.560 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:28:16.560 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:28:16.560 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-07-13T22:28:16.560 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:28:16.561 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-07-13T22:28:16.561 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:28:16.561 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:28:16.561 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-07-13T22:28:16.561 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.561 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:28:16.561 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.561 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-07-13T22:28:16.562 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.562 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-07-13T22:28:16.562 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.562 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-07-13T22:28:16.562 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-07-13T22:28:16.562 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-07-13T22:28:16.562 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:28:16.563 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:28:16.563 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.563 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-07-13T22:28:16.563 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-07-13T22:28:16.563 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.563 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-07-13T22:28:16.563 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2023-07-13T22:28:16.563 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:28:16.564 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.564 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-07-13T22:28:16.564 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-07-13T22:28:16.564 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.564 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-07-13T22:28:16.564 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-07-13T22:28:16.564 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-07-13T22:28:16.564 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.565 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-07-13T22:28:16.565 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-07-13T22:28:16.565 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.565 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:28:16.565 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.565 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-07-13T22:28:16.565 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.565 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-07-13T22:28:16.566 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.566 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:28:16.566 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-07-13T22:28:16.566 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.566 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-07-13T22:28:16.566 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-07-13T22:28:16.566 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2023-07-13T22:28:16.566 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2023-07-13T22:28:16.567 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2023-07-13T22:28:16.567 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2023-07-13T22:28:16.567 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2023-07-13T22:28:16.567 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2023-07-13T22:28:16.567 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2023-07-13T22:28:16.567 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:16.567 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-07-13T22:28:16.567 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2023-07-13T22:28:16.568 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:28:16.568 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2023-07-13T22:28:16.568 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:28:16.568 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:28:16.568 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2023-07-13T22:28:16.568 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:28:16.568 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.568 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-07-13T22:28:16.569 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:16.569 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-07-13T22:28:16.569 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:28:16.569 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:28:16.569 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:28:16.569 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:28:16.569 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-07-13T22:28:16.569 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.570 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-07-13T22:28:16.570 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:16.570 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:28:16.570 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-07-13T22:28:16.570 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:28:16.570 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-07-13T22:28:16.570 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-07-13T22:28:16.570 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:28:16.571 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-07-13T22:28:16.571 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-07-13T22:28:16.571 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:28:16.571 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.571 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:28:16.571 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:28:16.571 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:28:16.571 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.572 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:28:16.572 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:28:16.572 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:28:16.572 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.572 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-07-13T22:28:16.572 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:28:16.572 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.572 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:28:16.573 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2023-07-13T22:28:16.573 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:16.573 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-07-13T22:28:16.573 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-07-13T22:28:16.573 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-07-13T22:28:16.573 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-07-13T22:28:16.573 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:28:16.573 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:28:16.574 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2023-07-13T22:28:16.574 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2023-07-13T22:28:16.574 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:16.574 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2023-07-13T22:28:16.574 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:28:16.574 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:28:16.574 INFO:teuthology.orchestra.run.smithi105.stdout: client inject fixed oldest tid = true 2023-07-13T22:28:16.574 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-07-13T22:28:16.575 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2023-07-13T22:28:16.575 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2023-07-13T22:28:16.575 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-07-13T22:28:16.575 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-07-13T22:28:16.575 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-07-13T22:28:16.575 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-07-13T22:28:16.575 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2023-07-13T22:28:16.575 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2023-07-13T22:28:16.575 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2023-07-13T22:28:16.576 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:16.576 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2023-07-13T22:28:16.576 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2023-07-13T22:28:16.576 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2023-07-13T22:28:16.576 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2023-07-13T22:28:16.576 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:28:16.576 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:28:16.577 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:28:16.577 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2023-07-13T22:28:16.577 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2023-07-13T22:28:16.577 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:28:16.578 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2023-07-13T22:28:16.578 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.578 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2023-07-13T22:28:16.578 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.578 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2023-07-13T22:28:16.578 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.578 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2023-07-13T22:28:16.578 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2023-07-13T22:28:16.579 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.579 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2023-07-13T22:28:16.579 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2023-07-13T22:28:16.579 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.579 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:28:16.579 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2023-07-13T22:28:16.579 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.579 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:28:16.580 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:28:16.580 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.580 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2023-07-13T22:28:16.580 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.580 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2023-07-13T22:28:16.580 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2023-07-13T22:28:16.580 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:28:16.580 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:28:16.581 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2023-07-13T22:28:16.581 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:28:16.581 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2023-07-13T22:28:16.581 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:28:16.581 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:28:16.581 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2023-07-13T22:28:16.581 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.581 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:28:16.582 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.582 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2023-07-13T22:28:16.582 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.582 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2023-07-13T22:28:16.582 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.582 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2023-07-13T22:28:16.582 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2023-07-13T22:28:16.582 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2023-07-13T22:28:16.583 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:28:16.583 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:28:16.583 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.583 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2023-07-13T22:28:16.583 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2023-07-13T22:28:16.583 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.583 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2023-07-13T22:28:16.584 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2023-07-13T22:28:16.584 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:28:16.584 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.584 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2023-07-13T22:28:16.584 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2023-07-13T22:28:16.584 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.584 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2023-07-13T22:28:16.584 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2023-07-13T22:28:16.585 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2023-07-13T22:28:16.585 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.585 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2023-07-13T22:28:16.585 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2023-07-13T22:28:16.585 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.585 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:28:16.585 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.585 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2023-07-13T22:28:16.586 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.586 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2023-07-13T22:28:16.586 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.586 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:28:16.586 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2023-07-13T22:28:16.586 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.586 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2023-07-13T22:28:16.586 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2023-07-13T22:28:16.587 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2023-07-13T22:28:16.587 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2023-07-13T22:28:16.587 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2023-07-13T22:28:16.587 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2023-07-13T22:28:16.587 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2023-07-13T22:28:16.587 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2023-07-13T22:28:16.587 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2023-07-13T22:28:16.587 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:16.588 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2023-07-13T22:28:16.588 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2023-07-13T22:28:16.588 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:28:16.588 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2023-07-13T22:28:16.588 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:28:16.588 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:28:16.588 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2023-07-13T22:28:16.588 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:28:16.589 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.589 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2023-07-13T22:28:16.589 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:16.589 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2023-07-13T22:28:16.589 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:28:16.589 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:28:16.589 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:28:16.589 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:28:16.590 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2023-07-13T22:28:16.590 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.590 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2023-07-13T22:28:16.590 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:16.590 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:28:16.590 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2023-07-13T22:28:16.590 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:28:16.590 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2023-07-13T22:28:16.591 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2023-07-13T22:28:16.591 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:28:16.591 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2023-07-13T22:28:16.591 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2023-07-13T22:28:16.591 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:28:16.591 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.591 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:28:16.591 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:28:16.592 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:28:16.592 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.592 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:28:16.592 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:28:16.592 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:28:16.592 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.592 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2023-07-13T22:28:16.593 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:28:16.593 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.593 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:28:16.593 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2023-07-13T22:28:16.593 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:16.593 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2023-07-13T22:28:16.593 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2023-07-13T22:28:16.593 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2023-07-13T22:28:16.594 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2023-07-13T22:28:16.594 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:28:16.594 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:28:16.594 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2023-07-13T22:28:16.594 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2023-07-13T22:28:16.594 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:16.594 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2023-07-13T22:28:16.594 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:28:16.595 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:28:16.595 INFO:teuthology.orchestra.run.smithi060.stdout: client inject fixed oldest tid = true 2023-07-13T22:28:16.595 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2023-07-13T22:28:16.595 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2023-07-13T22:28:16.595 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2023-07-13T22:28:16.595 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-07-13T22:28:16.595 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-07-13T22:28:16.595 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-07-13T22:28:16.596 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-07-13T22:28:16.596 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2023-07-13T22:28:16.596 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2023-07-13T22:28:16.596 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2023-07-13T22:28:16.596 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:16.596 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2023-07-13T22:28:16.596 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2023-07-13T22:28:16.596 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2023-07-13T22:28:16.597 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2023-07-13T22:28:16.597 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:28:16.597 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:28:16.597 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:28:16.598 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:28:16.599 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:16.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.909+0000 7fd2043ff700 1 Processor -- start 2023-07-13T22:28:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.909+0000 7fd2043ff700 1 -- start start 2023-07-13T22:28:16.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.910+0000 7fd2043ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0b80 0x7fd1fc0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:16.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.910+0000 7fd2043ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1fc0d0f50 0x7fd1fc0d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.910+0000 7fd2043ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1fc0cf8e0 0x7fd1fc0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.910+0000 7fd2043ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc0b23f0 con 0x7fd1fc0d0f50 2023-07-13T22:28:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.910+0000 7fd2043ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc0b4bb0 con 0x7fd1fc0cf8e0 2023-07-13T22:28:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.910+0000 7fd2043ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc0ceeb0 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.910+0000 7fd20199a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1fc0d0f50 0x7fd1fc0d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.910+0000 7fd20199a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1fc0d0f50 0x7fd1fc0d2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59998/0 (socket says 172.21.15.60:59998) 2023-07-13T22:28:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.910+0000 7fd20199a700 1 -- 172.21.15.60:0/3214017337 learned_addr learned my addr 172.21.15.60:0/3214017337 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd20299c700 1 --2- 172.21.15.60:0/3214017337 >> v2:172.21.15.105:3300/0 conn(0x7fd1fc0cf8e0 0x7fd1fc0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd20219b700 1 --2- 172.21.15.60:0/3214017337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0b80 0x7fd1fc0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd20199a700 1 -- 172.21.15.60:0/3214017337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0b80 msgr2=0x7fd1fc0b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd20199a700 1 --2- 172.21.15.60:0/3214017337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0b80 0x7fd1fc0b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd20199a700 1 -- 172.21.15.60:0/3214017337 >> v2:172.21.15.105:3300/0 conn(0x7fd1fc0cf8e0 msgr2=0x7fd1fc0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd20199a700 1 --2- 172.21.15.60:0/3214017337 >> v2:172.21.15.105:3300/0 conn(0x7fd1fc0cf8e0 0x7fd1fc0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd20199a700 1 -- 172.21.15.60:0/3214017337 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc0b5380 con 0x7fd1fc0d0f50 2023-07-13T22:28:16.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd20199a700 1 --2- 172.21.15.60:0/3214017337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0f50 0x7fd1fc0d2f90 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0d1db0 tx=0x7fd1ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188fe7c73d347104 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:16.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd201199700 1 -- 172.21.15.60:0/3214017337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec011830 con 0x7fd1fc0d0f50 2023-07-13T22:28:16.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd201199700 1 -- 172.21.15.60:0/3214017337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec011990 con 0x7fd1fc0d0f50 2023-07-13T22:28:16.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.911+0000 7fd201199700 1 -- 172.21.15.60:0/3214017337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec011d20 con 0x7fd1fc0d0f50 2023-07-13T22:28:16.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.912+0000 7fd2043ff700 1 -- 172.21.15.60:0/3214017337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0f50 msgr2=0x7fd1fc0d2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:16.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.912+0000 7fd2043ff700 1 --2- 172.21.15.60:0/3214017337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0f50 0x7fd1fc0d2f90 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0d1db0 tx=0x7fd1ec009bd0 comp rx=0 tx=0).stop 2023-07-13T22:28:16.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.912+0000 7fd2043ff700 1 -- 172.21.15.60:0/3214017337 shutdown_connections 2023-07-13T22:28:16.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.912+0000 7fd2043ff700 1 --2- 172.21.15.60:0/3214017337 >> v2:172.21.15.105:3300/0 conn(0x7fd1fc0cf8e0 0x7fd1fc0cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.912+0000 7fd2043ff700 1 --2- 172.21.15.60:0/3214017337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0b80 0x7fd1fc0b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.912+0000 7fd2043ff700 1 --2- 172.21.15.60:0/3214017337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0f50 0x7fd1fc0d2f90 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.912+0000 7fd2043ff700 1 -- 172.21.15.60:0/3214017337 >> 172.21.15.60:0/3214017337 conn(0x7fd1fc071660 msgr2=0x7fd1fc071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.912+0000 7fd2043ff700 1 -- 172.21.15.60:0/3214017337 shutdown_connections 2023-07-13T22:28:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.912+0000 7fd2043ff700 1 -- 172.21.15.60:0/3214017337 wait complete. 2023-07-13T22:28:16.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.914+0000 7fd2043ff700 1 Processor -- start 2023-07-13T22:28:16.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.914+0000 7fd2043ff700 1 -- start start 2023-07-13T22:28:16.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd2043ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1fc0cf8e0 0x7fd1fc154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:16.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd2043ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0b80 0x7fd1fc158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:16.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd2043ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0f50 0x7fd1fc0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:16.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd2043ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc1551a0 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd2043ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc158ba0 con 0x7fd1fc0cf8e0 2023-07-13T22:28:16.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd2043ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc1575c0 con 0x7fd1fc0d0f50 2023-07-13T22:28:16.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd20199a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0b80 0x7fd1fc158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:16.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd20199a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0b80 0x7fd1fc158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60014/0 (socket says 172.21.15.60:60014) 2023-07-13T22:28:16.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd20199a700 1 -- 172.21.15.60:0/1930450864 learned_addr learned my addr 172.21.15.60:0/1930450864 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:16.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd20219b700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1fc0cf8e0 0x7fd1fc154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:16.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.915+0000 7fd20299c700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0f50 0x7fd1fc0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd20199a700 1 -- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0f50 msgr2=0x7fd1fc0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd20199a700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0f50 0x7fd1fc0cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd20199a700 1 -- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1fc0cf8e0 msgr2=0x7fd1fc154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd20199a700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1fc0cf8e0 0x7fd1fc154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd20199a700 1 -- 172.21.15.60:0/1930450864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec009550 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd20219b700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1fc0cf8e0 0x7fd1fc154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd20299c700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0f50 0x7fd1fc0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd20199a700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0b80 0x7fd1fc158660 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec000c00 tx=0x7fd1ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd1f2ffd700 1 -- 172.21.15.60:0/1930450864 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec004380 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd1f2ffd700 1 -- 172.21.15.60:0/1930450864 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec01f070 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc1578a0 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd1f2ffd700 1 -- 172.21.15.60:0/1930450864 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0036a0 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.916+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1fc157e60 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.917+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1fc0741e0 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.918+0000 7fd1f2ffd700 1 -- 172.21.15.60:0/1930450864 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd1ec021030 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.919+0000 7fd1f2ffd700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1e006ace0 0x7fd1e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.919+0000 7fd1f2ffd700 1 -- 172.21.15.60:0/1930450864 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1ec08a980 con 0x7fd1fc0d0b80 2023-07-13T22:28:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.919+0000 7fd20219b700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1e006ace0 0x7fd1e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.920+0000 7fd20219b700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1e006ace0 0x7fd1e006d190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc159080 tx=0x7fd1f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:16.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:16.923+0000 7fd1f2ffd700 1 -- 172.21.15.60:0/1930450864 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd1ec055a20 con 0x7fd1fc0d0b80 2023-07-13T22:28:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.133+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1fc0b83f0 con 0x7fd1fc0d0b80 2023-07-13T22:28:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.134+0000 7fd1f2ffd700 1 -- 172.21.15.60:0/1930450864 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18200 (secure 0 0 0) 0x7fd1ec059040 con 0x7fd1fc0d0b80 2023-07-13T22:28:17.115 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:17.115 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:06.113196+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:02.360339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:03.609373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:17.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.136+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1e006ace0 msgr2=0x7fd1e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.136+0000 7fd2043ff700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1e006ace0 0x7fd1e006d190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc159080 tx=0x7fd1f8008040 comp rx=0 tx=0).stop 2023-07-13T22:28:17.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.136+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0b80 msgr2=0x7fd1fc158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.136+0000 7fd2043ff700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0b80 0x7fd1fc158660 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec000c00 tx=0x7fd1ec004750 comp rx=0 tx=0).stop 2023-07-13T22:28:17.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.137+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 shutdown_connections 2023-07-13T22:28:17.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.137+0000 7fd2043ff700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1e006ace0 0x7fd1e006d190 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.137+0000 7fd2043ff700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1fc0d0f50 0x7fd1fc0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.137+0000 7fd2043ff700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1fc0cf8e0 0x7fd1fc154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.137+0000 7fd2043ff700 1 --2- 172.21.15.60:0/1930450864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1fc0d0b80 0x7fd1fc158660 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.137+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 >> 172.21.15.60:0/1930450864 conn(0x7fd1fc071660 msgr2=0x7fd1fc0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.137+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 shutdown_connections 2023-07-13T22:28:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.137+0000 7fd2043ff700 1 -- 172.21.15.60:0/1930450864 wait complete. 2023-07-13T22:28:17.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:28:02.360339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:28:03.609373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:28:17.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:28:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.362+0000 7f65c7bff700 1 Processor -- start 2023-07-13T22:28:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.362+0000 7f65c7bff700 1 -- start start 2023-07-13T22:28:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.363+0000 7f65c7bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 0x7f65c0160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c7bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65c00b4e50 0x7f65c00d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c7bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c0161020 con 0x7f65c00cf280 2023-07-13T22:28:17.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c7bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c00cd310 con 0x7f65c00b4e50 2023-07-13T22:28:17.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c7bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c015c9f0 con 0x7f65c015c620 2023-07-13T22:28:17.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c619c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65c00b4e50 0x7f65c00d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c619c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65c00b4e50 0x7f65c00d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60018/0 (socket says 172.21.15.60:60018) 2023-07-13T22:28:17.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c619c700 1 -- 172.21.15.60:0/1945990755 learned_addr learned my addr 172.21.15.60:0/1945990755 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c599b700 1 --2- 172.21.15.60:0/1945990755 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 0x7f65c0160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c619c700 1 -- 172.21.15.60:0/1945990755 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 msgr2=0x7f65c0160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c619c700 1 --2- 172.21.15.60:0/1945990755 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 0x7f65c0160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c619c700 1 -- 172.21.15.60:0/1945990755 >> v1:172.21.15.105:6789/0 conn(0x7f65c00cf280 legacy=0x7f65c00b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.364+0000 7f65c619c700 1 -- 172.21.15.60:0/1945990755 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c00cd560 con 0x7f65c00b4e50 2023-07-13T22:28:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.365+0000 7f65c619c700 1 --2- 172.21.15.60:0/1945990755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 0x7f65c00d11c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009e50 tx=0x7f65b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afe42a9e99eadab server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.365+0000 7f65c4999700 1 -- 172.21.15.60:0/1945990755 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800e070 con 0x7f65c00b4e50 2023-07-13T22:28:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.365+0000 7f65c4999700 1 -- 172.21.15.60:0/1945990755 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b800b910 con 0x7f65c00b4e50 2023-07-13T22:28:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.365+0000 7f65c4999700 1 -- 172.21.15.60:0/1945990755 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800bcc0 con 0x7f65c00b4e50 2023-07-13T22:28:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.365+0000 7f65c7bff700 1 -- 172.21.15.60:0/1945990755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 msgr2=0x7f65c00d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.365+0000 7f65c7bff700 1 --2- 172.21.15.60:0/1945990755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 0x7f65c00d11c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009e50 tx=0x7f65b800b330 comp rx=0 tx=0).stop 2023-07-13T22:28:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.366+0000 7f65c7bff700 1 -- 172.21.15.60:0/1945990755 shutdown_connections 2023-07-13T22:28:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.366+0000 7f65c7bff700 1 --2- 172.21.15.60:0/1945990755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 0x7f65c00d11c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.366+0000 7f65c7bff700 1 --2- 172.21.15.60:0/1945990755 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 0x7f65c0160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.366+0000 7f65c7bff700 1 -- 172.21.15.60:0/1945990755 >> 172.21.15.60:0/1945990755 conn(0x7f65c006b210 msgr2=0x7f65c00cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.366+0000 7f65c7bff700 1 -- 172.21.15.60:0/1945990755 shutdown_connections 2023-07-13T22:28:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.366+0000 7f65c7bff700 1 -- 172.21.15.60:0/1945990755 wait complete. 2023-07-13T22:28:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.368+0000 7f65c7bff700 1 Processor -- start 2023-07-13T22:28:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.368+0000 7f65c7bff700 1 -- start start 2023-07-13T22:28:17.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.369+0000 7f65c7bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 0x7f65c00b8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.369+0000 7f65c7bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c00cf280 0x7f65c01522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.369+0000 7f65c7bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 0x7f65c015dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c7bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c00b8a80 con 0x7f65c00b4e50 2023-07-13T22:28:17.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c7bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c00b72d0 con 0x7f65c00cf280 2023-07-13T22:28:17.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c7bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c015f8d0 con 0x7f65c015c620 2023-07-13T22:28:17.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 0x7f65c00b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 0x7f65c00b8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60024/0 (socket says 172.21.15.60:60024) 2023-07-13T22:28:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c599b700 1 -- 172.21.15.60:0/4051655337 learned_addr learned my addr 172.21.15.60:0/4051655337 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c619c700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 0x7f65c015dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c519a700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c00cf280 0x7f65c01522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c599b700 1 -- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 msgr2=0x7f65c015dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c599b700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 0x7f65c015dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c599b700 1 -- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c00cf280 msgr2=0x7f65c01522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c599b700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c00cf280 0x7f65c01522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c599b700 1 -- 172.21.15.60:0/4051655337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b8009b30 con 0x7f65c00b4e50 2023-07-13T22:28:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c619c700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 0x7f65c015dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c519a700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c00cf280 0x7f65c01522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.370+0000 7f65c599b700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 0x7f65c00b8540 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00e870 tx=0x7f65bc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.371+0000 7f65b67fc700 1 -- 172.21.15.60:0/4051655337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc009590 con 0x7f65c00b4e50 2023-07-13T22:28:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.371+0000 7f65b67fc700 1 -- 172.21.15.60:0/4051655337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc00ee70 con 0x7f65c00b4e50 2023-07-13T22:28:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.371+0000 7f65b67fc700 1 -- 172.21.15.60:0/4051655337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00ca50 con 0x7f65c00b4e50 2023-07-13T22:28:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.371+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c00b6c80 con 0x7f65c00b4e50 2023-07-13T22:28:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.371+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c00b7760 con 0x7f65c00b4e50 2023-07-13T22:28:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.372+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c00b75b0 con 0x7f65c00b4e50 2023-07-13T22:28:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.373+0000 7f65b67fc700 1 -- 172.21.15.60:0/4051655337 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65bc00cbb0 con 0x7f65c00b4e50 2023-07-13T22:28:17.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.374+0000 7f65b67fc700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a406ace0 0x7f65a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.374+0000 7f65b67fc700 1 -- 172.21.15.60:0/4051655337 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65bc0894a0 con 0x7f65c00b4e50 2023-07-13T22:28:17.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.374+0000 7f65c519a700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a406ace0 0x7f65a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.375+0000 7f65c519a700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a406ace0 0x7f65a406d190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f65c00b8260 tx=0x7f65b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:17.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.379+0000 7f65b67fc700 1 -- 172.21.15.60:0/4051655337 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f65bc057fd0 con 0x7f65c00b4e50 2023-07-13T22:28:17.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.583+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65c00741e0 con 0x7f65c00b4e50 2023-07-13T22:28:17.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.584+0000 7f65b67fc700 1 -- 172.21.15.60:0/4051655337 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65bc00ce60 con 0x7f65c00b4e50 2023-07-13T22:28:17.564 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:28:17.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a406ace0 msgr2=0x7f65a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a406ace0 0x7f65a406d190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f65c00b8260 tx=0x7f65b0009040 comp rx=0 tx=0).stop 2023-07-13T22:28:17.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 msgr2=0x7f65c00b8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 0x7f65c00b8540 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00e870 tx=0x7f65bc00ec30 comp rx=0 tx=0).stop 2023-07-13T22:28:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 shutdown_connections 2023-07-13T22:28:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f65a406ace0 0x7f65a406d190 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c00b4e50 0x7f65c00b8540 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65c015c620 0x7f65c015dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 --2- 172.21.15.60:0/4051655337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c00cf280 0x7f65c01522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 >> 172.21.15.60:0/4051655337 conn(0x7f65c006b210 msgr2=0x7f65c015b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:17.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 shutdown_connections 2023-07-13T22:28:17.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.586+0000 7f65c7bff700 1 -- 172.21.15.60:0/4051655337 wait complete. 2023-07-13T22:28:17.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:28:17.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.810+0000 7f7d4ffff700 1 Processor -- start 2023-07-13T22:28:17.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.811+0000 7f7d4ffff700 1 -- start start 2023-07-13T22:28:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.812+0000 7f7d4ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d48160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.812+0000 7f7d4ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d480b4e50 0x7f7d480d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.812+0000 7f7d4ffff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d48161020 con 0x7f7d480cf280 2023-07-13T22:28:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.812+0000 7f7d4ffff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d480cd310 con 0x7f7d480b4e50 2023-07-13T22:28:17.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.812+0000 7f7d4ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4815c9f0 con 0x7f7d4815c620 2023-07-13T22:28:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.812+0000 7f7d4e59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d480b4e50 0x7f7d480d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.812+0000 7f7d4d59a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7d480cf280 0x7f7d480b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38684/0 (socket says 172.21.15.60:38684) 2023-07-13T22:28:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.812+0000 7f7d4d59a700 1 -- 172.21.15.60:0/3961153393 learned_addr learned my addr 172.21.15.60:0/3961153393 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.812+0000 7f7d4dd9b700 1 --2- 172.21.15.60:0/3961153393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d48160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.813+0000 7f7d4e59c700 1 -- 172.21.15.60:0/3961153393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 msgr2=0x7f7d48160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.813+0000 7f7d4e59c700 1 --2- 172.21.15.60:0/3961153393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d48160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.813+0000 7f7d4e59c700 1 -- 172.21.15.60:0/3961153393 >> v1:172.21.15.60:6789/0 conn(0x7f7d480cf280 legacy=0x7f7d480b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.813+0000 7f7d4e59c700 1 -- 172.21.15.60:0/3961153393 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480cd560 con 0x7f7d480b4e50 2023-07-13T22:28:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.813+0000 7f7d4dd9b700 1 --2- 172.21.15.60:0/3961153393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d48160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.813+0000 7f7d4e59c700 1 --2- 172.21.15.60:0/3961153393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480b4e50 0x7f7d480d11c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009d90 tx=0x7f7d4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9081dafe3b0778 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.813+0000 7f7d4cd99700 1 -- 172.21.15.60:0/3961153393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000e070 con 0x7f7d480b4e50 2023-07-13T22:28:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.813+0000 7f7d4cd99700 1 -- 172.21.15.60:0/3961153393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4000bbd0 con 0x7f7d480b4e50 2023-07-13T22:28:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.813+0000 7f7d4cd99700 1 -- 172.21.15.60:0/3961153393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40012430 con 0x7f7d480b4e50 2023-07-13T22:28:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.814+0000 7f7d4ffff700 1 -- 172.21.15.60:0/3961153393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480b4e50 msgr2=0x7f7d480d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.814+0000 7f7d4ffff700 1 --2- 172.21.15.60:0/3961153393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480b4e50 0x7f7d480d11c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009d90 tx=0x7f7d4000b330 comp rx=0 tx=0).stop 2023-07-13T22:28:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.814+0000 7f7d4ffff700 1 -- 172.21.15.60:0/3961153393 shutdown_connections 2023-07-13T22:28:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.814+0000 7f7d4ffff700 1 --2- 172.21.15.60:0/3961153393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480b4e50 0x7f7d480d11c0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.814+0000 7f7d4ffff700 1 --2- 172.21.15.60:0/3961153393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d48160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.814+0000 7f7d4ffff700 1 -- 172.21.15.60:0/3961153393 >> 172.21.15.60:0/3961153393 conn(0x7f7d4806b210 msgr2=0x7f7d480cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.814+0000 7f7d4ffff700 1 -- 172.21.15.60:0/3961153393 shutdown_connections 2023-07-13T22:28:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.814+0000 7f7d4ffff700 1 -- 172.21.15.60:0/3961153393 wait complete. 2023-07-13T22:28:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.815+0000 7f7d4ffff700 1 Processor -- start 2023-07-13T22:28:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.816+0000 7f7d4ffff700 1 -- start start 2023-07-13T22:28:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d480b4e50 0x7f7d48153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480cf280 0x7f7d4815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d4815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d48153840 con 0x7f7d480cf280 2023-07-13T22:28:17.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4ffff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480b6e60 con 0x7f7d480b4e50 2023-07-13T22:28:17.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480c6e50 con 0x7f7d4815c620 2023-07-13T22:28:17.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4dd9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d480b4e50 0x7f7d48153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4d59a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480cf280 0x7f7d4815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4dd9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d480b4e50 0x7f7d48153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:36758/0 (socket says 172.21.15.60:36758) 2023-07-13T22:28:17.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4dd9b700 1 -- 172.21.15.60:0/1630568845 learned_addr learned my addr 172.21.15.60:0/1630568845 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:17.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4e59c700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d4815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4d59a700 1 -- 172.21.15.60:0/1630568845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 msgr2=0x7f7d4815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4d59a700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d4815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4d59a700 1 -- 172.21.15.60:0/1630568845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d480b4e50 msgr2=0x7f7d48153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:17.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4d59a700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d480b4e50 0x7f7d48153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:17.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4d59a700 1 -- 172.21.15.60:0/1630568845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d40009a70 con 0x7f7d480cf280 2023-07-13T22:28:17.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.817+0000 7f7d4e59c700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d4815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:17.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.818+0000 7f7d4d59a700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480cf280 0x7f7d4815cb60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c500 tx=0x7f7d3800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:17.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.818+0000 7f7d3e7fc700 1 -- 172.21.15.60:0/1630568845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38005d70 con 0x7f7d480cf280 2023-07-13T22:28:17.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.818+0000 7f7d3e7fc700 1 -- 172.21.15.60:0/1630568845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d380070d0 con 0x7f7d480cf280 2023-07-13T22:28:17.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.818+0000 7f7d3e7fc700 1 -- 172.21.15.60:0/1630568845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38007440 con 0x7f7d480cf280 2023-07-13T22:28:17.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.818+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d480c70f0 con 0x7f7d480cf280 2023-07-13T22:28:17.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.818+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d480c76b0 con 0x7f7d480cf280 2023-07-13T22:28:17.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.819+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d480741e0 con 0x7f7d480cf280 2023-07-13T22:28:17.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.820+0000 7f7d3e7fc700 1 -- 172.21.15.60:0/1630568845 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d380075a0 con 0x7f7d480cf280 2023-07-13T22:28:17.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.821+0000 7f7d3e7fc700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2c06ace0 0x7f7d2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:17.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.821+0000 7f7d3e7fc700 1 -- 172.21.15.60:0/1630568845 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d380885a0 con 0x7f7d480cf280 2023-07-13T22:28:17.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.821+0000 7f7d4dd9b700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2c06ace0 0x7f7d2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:17.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.822+0000 7f7d4dd9b700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2c06ace0 0x7f7d2c06d190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7d44004470 tx=0x7f7d44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:17.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:17.825+0000 7f7d3e7fc700 1 -- 172.21.15.60:0/1630568845 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7d380536f0 con 0x7f7d480cf280 2023-07-13T22:28:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.029+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d48161020 con 0x7f7d480cf280 2023-07-13T22:28:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.030+0000 7f7d3e7fc700 1 -- 172.21.15.60:0/1630568845 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d48161020 con 0x7f7d480cf280 2023-07-13T22:28:18.010 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:28:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2c06ace0 msgr2=0x7f7d2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2c06ace0 0x7f7d2c06d190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7d44004470 tx=0x7f7d44008040 comp rx=0 tx=0).stop 2023-07-13T22:28:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480cf280 msgr2=0x7f7d4815cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480cf280 0x7f7d4815cb60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c500 tx=0x7f7d3800c810 comp rx=0 tx=0).stop 2023-07-13T22:28:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 shutdown_connections 2023-07-13T22:28:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d2c06ace0 0x7f7d2c06d190 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d4815c620 0x7f7d4815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d480b4e50 0x7f7d48153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 --2- 172.21.15.60:0/1630568845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d480cf280 0x7f7d4815cb60 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 >> 172.21.15.60:0/1630568845 conn(0x7f7d4806b210 msgr2=0x7f7d480d1730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 shutdown_connections 2023-07-13T22:28:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.032+0000 7f7d4ffff700 1 -- 172.21.15.60:0/1630568845 wait complete. 2023-07-13T22:28:18.030 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:28:18.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.257+0000 7fb1907ff700 1 Processor -- start 2023-07-13T22:28:18.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.257+0000 7fb1907ff700 1 -- start start 2023-07-13T22:28:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.258+0000 7fb1907ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 0x7fb188160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.258+0000 7fb1907ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb188161020 con 0x7fb1880b4e50 2023-07-13T22:28:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.258+0000 7fb1907ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1880cd310 con 0x7fb1880cf280 2023-07-13T22:28:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.258+0000 7fb1907ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18815c9f0 con 0x7fb18815c620 2023-07-13T22:28:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.259+0000 7fb18ed9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb1880b4e50 0x7fb1880d11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38692/0 (socket says 172.21.15.60:38692) 2023-07-13T22:28:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.259+0000 7fb18ed9c700 1 -- 172.21.15.60:0/1122113279 learned_addr learned my addr 172.21.15.60:0/1122113279 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.259+0000 7fb18e59b700 1 --2- 172.21.15.60:0/1122113279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 0x7fb188160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.259+0000 7fb18d599700 1 -- 172.21.15.60:0/1122113279 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185760115 0 0) 0x7fb1800014e0 con 0x7fb1880b4e50 2023-07-13T22:28:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.259+0000 7fb18d599700 1 -- 172.21.15.60:0/1122113279 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1740035b0 con 0x7fb1880b4e50 2023-07-13T22:28:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.259+0000 7fb18d599700 1 -- 172.21.15.60:0/1122113279 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380763267 0 0) 0x7fb1880cd310 con 0x7fb1880cf280 2023-07-13T22:28:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.259+0000 7fb18d599700 1 -- 172.21.15.60:0/1122113279 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1800014e0 con 0x7fb1880cf280 2023-07-13T22:28:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.259+0000 7fb18e59b700 1 -- 172.21.15.60:0/1122113279 >> v1:172.21.15.105:6789/0 conn(0x7fb1880cf280 legacy=0x7fb1880b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.259+0000 7fb18e59b700 1 -- 172.21.15.60:0/1122113279 >> v1:172.21.15.60:6789/0 conn(0x7fb1880b4e50 legacy=0x7fb1880d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.260+0000 7fb18e59b700 1 -- 172.21.15.60:0/1122113279 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1880cd560 con 0x7fb18815c620 2023-07-13T22:28:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.260+0000 7fb18e59b700 1 --2- 172.21.15.60:0/1122113279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 0x7fb188160ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb184009870 tx=0x7fb184009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883a683044350527 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:18.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.260+0000 7fb18d599700 1 -- 172.21.15.60:0/1122113279 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184011830 con 0x7fb18815c620 2023-07-13T22:28:18.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.260+0000 7fb18d599700 1 -- 172.21.15.60:0/1122113279 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb184011990 con 0x7fb18815c620 2023-07-13T22:28:18.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.260+0000 7fb18d599700 1 -- 172.21.15.60:0/1122113279 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184011d20 con 0x7fb18815c620 2023-07-13T22:28:18.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.261+0000 7fb1907ff700 1 -- 172.21.15.60:0/1122113279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 msgr2=0x7fb188160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.261+0000 7fb1907ff700 1 --2- 172.21.15.60:0/1122113279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 0x7fb188160ae0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb184009870 tx=0x7fb184009b80 comp rx=0 tx=0).stop 2023-07-13T22:28:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.261+0000 7fb1907ff700 1 -- 172.21.15.60:0/1122113279 shutdown_connections 2023-07-13T22:28:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.261+0000 7fb1907ff700 1 --2- 172.21.15.60:0/1122113279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 0x7fb188160ae0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.261+0000 7fb1907ff700 1 -- 172.21.15.60:0/1122113279 >> 172.21.15.60:0/1122113279 conn(0x7fb18806b210 msgr2=0x7fb1880cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.261+0000 7fb1907ff700 1 -- 172.21.15.60:0/1122113279 shutdown_connections 2023-07-13T22:28:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.261+0000 7fb1907ff700 1 -- 172.21.15.60:0/1122113279 wait complete. 2023-07-13T22:28:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.263+0000 7fb1907ff700 1 Processor -- start 2023-07-13T22:28:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.263+0000 7fb1907ff700 1 -- start start 2023-07-13T22:28:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb1907ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1880b4e50 0x7fb188158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb1907ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1880cf280 0x7fb188158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb1907ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 0x7fb188159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb1907ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb188158aa0 con 0x7fb1880cf280 2023-07-13T22:28:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb1907ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1880cab00 con 0x7fb1880b4e50 2023-07-13T22:28:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb1907ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1880c9f30 con 0x7fb18815c620 2023-07-13T22:28:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18dd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1880cf280 0x7fb188158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18dd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1880cf280 0x7fb188158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60054/0 (socket says 172.21.15.60:60054) 2023-07-13T22:28:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18dd9a700 1 -- 172.21.15.60:0/2447370509 learned_addr learned my addr 172.21.15.60:0/2447370509 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18e59b700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1880b4e50 0x7fb188158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18ed9c700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 0x7fb188159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18dd9a700 1 -- 172.21.15.60:0/2447370509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 msgr2=0x7fb188159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18dd9a700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 0x7fb188159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18dd9a700 1 -- 172.21.15.60:0/2447370509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1880b4e50 msgr2=0x7fb188158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18dd9a700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1880b4e50 0x7fb188158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18dd9a700 1 -- 172.21.15.60:0/2447370509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1840094a0 con 0x7fb1880cf280 2023-07-13T22:28:18.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.264+0000 7fb18e59b700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1880b4e50 0x7fb188158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.265+0000 7fb18dd9a700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1880cf280 0x7fb188158c20 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c520 tx=0x7fb17800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.265+0000 7fb17effd700 1 -- 172.21.15.60:0/2447370509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780052d0 con 0x7fb1880cf280 2023-07-13T22:28:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.265+0000 7fb17effd700 1 -- 172.21.15.60:0/2447370509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb178005430 con 0x7fb1880cf280 2023-07-13T22:28:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.265+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1880ca1d0 con 0x7fb1880cf280 2023-07-13T22:28:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.265+0000 7fb17effd700 1 -- 172.21.15.60:0/2447370509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780038c0 con 0x7fb1880cf280 2023-07-13T22:28:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.265+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1880ca790 con 0x7fb1880cf280 2023-07-13T22:28:18.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.266+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1880cae90 con 0x7fb1880cf280 2023-07-13T22:28:18.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.267+0000 7fb17effd700 1 -- 172.21.15.60:0/2447370509 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb178003a20 con 0x7fb1880cf280 2023-07-13T22:28:18.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.268+0000 7fb17effd700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb16c06ace0 0x7fb16c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.268+0000 7fb17effd700 1 -- 172.21.15.60:0/2447370509 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb178088b00 con 0x7fb1880cf280 2023-07-13T22:28:18.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.268+0000 7fb18e59b700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb16c06ace0 0x7fb16c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.269+0000 7fb18e59b700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb16c06ace0 0x7fb16c06d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb1840090b0 tx=0x7fb18401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.272+0000 7fb17effd700 1 -- 172.21.15.60:0/2447370509 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb17808ae60 con 0x7fb1880cf280 2023-07-13T22:28:18.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.476+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1880741e0 con 0x7fb1880cf280 2023-07-13T22:28:18.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.477+0000 7fb17effd700 1 -- 172.21.15.60:0/2447370509 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb178053b10 con 0x7fb1880cf280 2023-07-13T22:28:18.457 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:28:18.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb16c06ace0 msgr2=0x7fb16c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb16c06ace0 0x7fb16c06d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb1840090b0 tx=0x7fb18401c040 comp rx=0 tx=0).stop 2023-07-13T22:28:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1880cf280 msgr2=0x7fb188158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1880cf280 0x7fb188158c20 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c520 tx=0x7fb17800c8e0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 shutdown_connections 2023-07-13T22:28:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb16c06ace0 0x7fb16c06d190 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb18815c620 0x7fb188159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1880b4e50 0x7fb188158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 --2- 172.21.15.60:0/2447370509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1880cf280 0x7fb188158c20 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 >> 172.21.15.60:0/2447370509 conn(0x7fb18806b210 msgr2=0x7fb18806be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:18.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 shutdown_connections 2023-07-13T22:28:18.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.479+0000 7fb1907ff700 1 -- 172.21.15.60:0/2447370509 wait complete. 2023-07-13T22:28:18.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:28:18.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.704+0000 7ffb603ff700 1 Processor -- start 2023-07-13T22:28:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.705+0000 7ffb603ff700 1 -- start start 2023-07-13T22:28:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.706+0000 7ffb603ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffb580b4e50 0x7ffb58160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.706+0000 7ffb603ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb58161020 con 0x7ffb5815c620 2023-07-13T22:28:18.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.706+0000 7ffb603ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb580cd310 con 0x7ffb580cf280 2023-07-13T22:28:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.706+0000 7ffb603ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5815c9f0 con 0x7ffb580b4e50 2023-07-13T22:28:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.706+0000 7ffb5e19b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffb5815c620 0x7ffb580b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38706/0 (socket says 172.21.15.60:38706) 2023-07-13T22:28:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.706+0000 7ffb5e19b700 1 -- 172.21.15.60:0/3751256926 learned_addr learned my addr 172.21.15.60:0/3751256926 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.706+0000 7ffb5d99a700 1 --2- 172.21.15.60:0/3751256926 >> v2:172.21.15.105:3300/0 conn(0x7ffb580b4e50 0x7ffb58160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.706+0000 7ffb5d199700 1 -- 172.21.15.60:0/3751256926 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982602895 0 0) 0x7ffb54001110 con 0x7ffb5815c620 2023-07-13T22:28:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.707+0000 7ffb5d199700 1 -- 172.21.15.60:0/3751256926 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb440035b0 con 0x7ffb5815c620 2023-07-13T22:28:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.707+0000 7ffb5d99a700 1 -- 172.21.15.60:0/3751256926 >> v1:172.21.15.105:6789/0 conn(0x7ffb580cf280 legacy=0x7ffb580d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.707+0000 7ffb5d99a700 1 -- 172.21.15.60:0/3751256926 >> v1:172.21.15.60:6789/0 conn(0x7ffb5815c620 legacy=0x7ffb580b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.707+0000 7ffb5d99a700 1 -- 172.21.15.60:0/3751256926 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb580cd560 con 0x7ffb580b4e50 2023-07-13T22:28:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.707+0000 7ffb5d99a700 1 --2- 172.21.15.60:0/3751256926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 0x7ffb58160ae0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009840 tx=0x7ffb48009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae3f387eb846124 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.707+0000 7ffb5d199700 1 -- 172.21.15.60:0/3751256926 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48011830 con 0x7ffb580b4e50 2023-07-13T22:28:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.707+0000 7ffb5d199700 1 -- 172.21.15.60:0/3751256926 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48011990 con 0x7ffb580b4e50 2023-07-13T22:28:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.707+0000 7ffb5d199700 1 -- 172.21.15.60:0/3751256926 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48011d20 con 0x7ffb580b4e50 2023-07-13T22:28:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.708+0000 7ffb603ff700 1 -- 172.21.15.60:0/3751256926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 msgr2=0x7ffb58160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.708+0000 7ffb603ff700 1 --2- 172.21.15.60:0/3751256926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 0x7ffb58160ae0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009840 tx=0x7ffb48009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.708+0000 7ffb603ff700 1 -- 172.21.15.60:0/3751256926 shutdown_connections 2023-07-13T22:28:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.708+0000 7ffb603ff700 1 --2- 172.21.15.60:0/3751256926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 0x7ffb58160ae0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.708+0000 7ffb603ff700 1 -- 172.21.15.60:0/3751256926 >> 172.21.15.60:0/3751256926 conn(0x7ffb5806b210 msgr2=0x7ffb580cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.708+0000 7ffb603ff700 1 -- 172.21.15.60:0/3751256926 shutdown_connections 2023-07-13T22:28:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.708+0000 7ffb603ff700 1 -- 172.21.15.60:0/3751256926 wait complete. 2023-07-13T22:28:18.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.710+0000 7ffb603ff700 1 Processor -- start 2023-07-13T22:28:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.710+0000 7ffb603ff700 1 -- start start 2023-07-13T22:28:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb603ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 0x7ffb580b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb603ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb580cf280 0x7ffb58142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb603ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb5815c620 0x7ffb5815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb603ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5815cea0 con 0x7ffb5815c620 2023-07-13T22:28:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb603ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5815ec80 con 0x7ffb580b4e50 2023-07-13T22:28:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb603ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb581f2b30 con 0x7ffb580cf280 2023-07-13T22:28:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb5e99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb5815c620 0x7ffb5815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb5e99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb5815c620 0x7ffb5815a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60058/0 (socket says 172.21.15.60:60058) 2023-07-13T22:28:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb5e99c700 1 -- 172.21.15.60:0/2398513246 learned_addr learned my addr 172.21.15.60:0/2398513246 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb5e19b700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 0x7ffb580b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.711+0000 7ffb5d99a700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb580cf280 0x7ffb58142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb5e99c700 1 -- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb580cf280 msgr2=0x7ffb58142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb5e99c700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb580cf280 0x7ffb58142aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb5e99c700 1 -- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 msgr2=0x7ffb580b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb5e99c700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 0x7ffb580b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb5e99c700 1 -- 172.21.15.60:0/2398513246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb480094a0 con 0x7ffb5815c620 2023-07-13T22:28:18.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb5e19b700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 0x7ffb580b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:18.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb5d99a700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb580cf280 0x7ffb58142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:18.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb5e99c700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb5815c620 0x7ffb5815a6b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000cbe0 tx=0x7ffb5000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:18.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb4effd700 1 -- 172.21.15.60:0/2398513246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50003c50 con 0x7ffb5815c620 2023-07-13T22:28:18.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb4effd700 1 -- 172.21.15.60:0/2398513246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb50003db0 con 0x7ffb5815c620 2023-07-13T22:28:18.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb4effd700 1 -- 172.21.15.60:0/2398513246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000abe0 con 0x7ffb5815c620 2023-07-13T22:28:18.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5815e630 con 0x7ffb5815c620 2023-07-13T22:28:18.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.712+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb581f2e10 con 0x7ffb5815c620 2023-07-13T22:28:18.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.713+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5815e490 con 0x7ffb5815c620 2023-07-13T22:28:18.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.714+0000 7ffb4effd700 1 -- 172.21.15.60:0/2398513246 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb5000ad40 con 0x7ffb5815c620 2023-07-13T22:28:18.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.715+0000 7ffb4effd700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb3c06ace0 0x7ffb3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:18.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.715+0000 7ffb4effd700 1 -- 172.21.15.60:0/2398513246 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb50089710 con 0x7ffb5815c620 2023-07-13T22:28:18.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.716+0000 7ffb5e19b700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb3c06ace0 0x7ffb3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:18.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.716+0000 7ffb5e19b700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb3c06ace0 0x7ffb3c06d190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ffb54001270 tx=0x7ffb54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:18.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.720+0000 7ffb4effd700 1 -- 172.21.15.60:0/2398513246 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffb5815e490 con 0x7ffb5815c620 2023-07-13T22:28:18.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.925+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb5815e490 con 0x7ffb5815c620 2023-07-13T22:28:18.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.926+0000 7ffb4effd700 1 -- 172.21.15.60:0/2398513246 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb5815e490 con 0x7ffb5815c620 2023-07-13T22:28:18.906 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:28:18.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb3c06ace0 msgr2=0x7ffb3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb3c06ace0 0x7ffb3c06d190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ffb54001270 tx=0x7ffb54009040 comp rx=0 tx=0).stop 2023-07-13T22:28:18.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb5815c620 msgr2=0x7ffb5815a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:18.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb5815c620 0x7ffb5815a6b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000cbe0 tx=0x7ffb5000a060 comp rx=0 tx=0).stop 2023-07-13T22:28:18.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 shutdown_connections 2023-07-13T22:28:18.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb3c06ace0 0x7ffb3c06d190 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb5815c620 0x7ffb5815a6b0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb580b4e50 0x7ffb580b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 --2- 172.21.15.60:0/2398513246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb580cf280 0x7ffb58142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:18.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 >> 172.21.15.60:0/2398513246 conn(0x7ffb5806b210 msgr2=0x7ffb5815b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:18.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.928+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 shutdown_connections 2023-07-13T22:28:18.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:18.929+0000 7ffb603ff700 1 -- 172.21.15.60:0/2398513246 wait complete. 2023-07-13T22:28:18.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:28:19.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.151+0000 7f8db05ff700 1 Processor -- start 2023-07-13T22:28:19.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.151+0000 7f8db05ff700 1 -- start start 2023-07-13T22:28:19.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.152+0000 7f8db05ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8da80b5ad0 0x7f8da80d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.152+0000 7f8db05ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8da80cf8e0 0x7f8da80cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8db05ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da80ceeb0 con 0x7f8da80d0b80 2023-07-13T22:28:19.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8db05ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da80b23f0 con 0x7f8da80cf8e0 2023-07-13T22:28:19.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8db05ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80b4bb0 con 0x7f8da80b5ad0 2023-07-13T22:28:19.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8dae39b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8da80d0b80 0x7f8da80d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38710/0 (socket says 172.21.15.60:38710) 2023-07-13T22:28:19.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8dae39b700 1 -- 172.21.15.60:0/771721049 learned_addr learned my addr 172.21.15.60:0/771721049 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8daeb9c700 1 --2- 172.21.15.60:0/771721049 >> v2:172.21.15.105:3300/0 conn(0x7f8da80cf8e0 0x7f8da80cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8dadb9a700 1 --2- 172.21.15.60:0/771721049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8da80b5ad0 0x7f8da80d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8dad399700 1 -- 172.21.15.60:0/771721049 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133122760 0 0) 0x7f8da80ceeb0 con 0x7f8da80d0b80 2023-07-13T22:28:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8dad399700 1 -- 172.21.15.60:0/771721049 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d940035b0 con 0x7f8da80d0b80 2023-07-13T22:28:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8daeb9c700 1 -- 172.21.15.60:0/771721049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8da80b5ad0 msgr2=0x7f8da80d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:19.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8daeb9c700 1 --2- 172.21.15.60:0/771721049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8da80b5ad0 0x7f8da80d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8daeb9c700 1 -- 172.21.15.60:0/771721049 >> v1:172.21.15.60:6789/0 conn(0x7f8da80d0b80 legacy=0x7f8da80d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.153+0000 7f8daeb9c700 1 -- 172.21.15.60:0/771721049 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80b5f40 con 0x7f8da80cf8e0 2023-07-13T22:28:19.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.154+0000 7f8daeb9c700 1 --2- 172.21.15.60:0/771721049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8da80cf8e0 0x7f8da80cfd60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009d90 tx=0x7f8da000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2398487cb71d514c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:19.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.154+0000 7f8dad399700 1 -- 172.21.15.60:0/771721049 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da000e070 con 0x7f8da80cf8e0 2023-07-13T22:28:19.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.154+0000 7f8dad399700 1 -- 172.21.15.60:0/771721049 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da000bb20 con 0x7f8da80cf8e0 2023-07-13T22:28:19.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.154+0000 7f8dad399700 1 -- 172.21.15.60:0/771721049 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0012430 con 0x7f8da80cf8e0 2023-07-13T22:28:19.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.155+0000 7f8db05ff700 1 -- 172.21.15.60:0/771721049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8da80cf8e0 msgr2=0x7f8da80cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.155+0000 7f8db05ff700 1 --2- 172.21.15.60:0/771721049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8da80cf8e0 0x7f8da80cfd60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009d90 tx=0x7f8da000b330 comp rx=0 tx=0).stop 2023-07-13T22:28:19.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.155+0000 7f8db05ff700 1 -- 172.21.15.60:0/771721049 shutdown_connections 2023-07-13T22:28:19.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.155+0000 7f8db05ff700 1 --2- 172.21.15.60:0/771721049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8da80cf8e0 0x7f8da80cfd60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.155+0000 7f8db05ff700 1 --2- 172.21.15.60:0/771721049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8da80b5ad0 0x7f8da80d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.155+0000 7f8db05ff700 1 -- 172.21.15.60:0/771721049 >> 172.21.15.60:0/771721049 conn(0x7f8da8071660 msgr2=0x7f8da8071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:19.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.155+0000 7f8db05ff700 1 -- 172.21.15.60:0/771721049 shutdown_connections 2023-07-13T22:28:19.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.155+0000 7f8db05ff700 1 -- 172.21.15.60:0/771721049 wait complete. 2023-07-13T22:28:19.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.157+0000 7f8db05ff700 1 Processor -- start 2023-07-13T22:28:19.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.157+0000 7f8db05ff700 1 -- start start 2023-07-13T22:28:19.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8db05ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da80b5ad0 0x7f8da8155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8db05ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8da80cf8e0 0x7f8da80c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8db05ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8da80d0b80 0x7f8da814bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8db05ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81564a0 con 0x7f8da80b5ad0 2023-07-13T22:28:19.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8db05ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80c9800 con 0x7f8da80d0b80 2023-07-13T22:28:19.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8db05ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da814c4c0 con 0x7f8da80cf8e0 2023-07-13T22:28:19.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8dae39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da80b5ad0 0x7f8da8155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:19.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8dae39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da80b5ad0 0x7f8da8155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60066/0 (socket says 172.21.15.60:60066) 2023-07-13T22:28:19.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8dae39b700 1 -- 172.21.15.60:0/2927285179 learned_addr learned my addr 172.21.15.60:0/2927285179 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:19.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8dae39b700 1 -- 172.21.15.60:0/2927285179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8da80cf8e0 msgr2=0x7f8da80c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8dae39b700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8da80cf8e0 0x7f8da80c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8dae39b700 1 -- 172.21.15.60:0/2927285179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8da80d0b80 msgr2=0x7f8da814bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8dae39b700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8da80d0b80 0x7f8da814bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.158+0000 7f8dae39b700 1 -- 172.21.15.60:0/2927285179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da0009a70 con 0x7f8da80b5ad0 2023-07-13T22:28:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.159+0000 7f8dae39b700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da80b5ad0 0x7f8da8155f60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8da400c700 tx=0x7f8da400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.159+0000 7f8d9effd700 1 -- 172.21.15.60:0/2927285179 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4005d70 con 0x7f8da80b5ad0 2023-07-13T22:28:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.159+0000 7f8d9effd700 1 -- 172.21.15.60:0/2927285179 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da40070d0 con 0x7f8da80b5ad0 2023-07-13T22:28:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.159+0000 7f8d9effd700 1 -- 172.21.15.60:0/2927285179 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4007460 con 0x7f8da80b5ad0 2023-07-13T22:28:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.159+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da8158600 con 0x7f8da80b5ad0 2023-07-13T22:28:19.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.159+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da8158bc0 con 0x7f8da80b5ad0 2023-07-13T22:28:19.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.160+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da80c69f0 con 0x7f8da80b5ad0 2023-07-13T22:28:19.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.161+0000 7f8d9effd700 1 -- 172.21.15.60:0/2927285179 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8da40075c0 con 0x7f8da80b5ad0 2023-07-13T22:28:19.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.162+0000 7f8d9effd700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d8c06ace0 0x7f8d8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.162+0000 7f8d9effd700 1 -- 172.21.15.60:0/2927285179 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8da4088b30 con 0x7f8da80b5ad0 2023-07-13T22:28:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.162+0000 7f8dadb9a700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d8c06ace0 0x7f8d8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.163+0000 7f8dadb9a700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d8c06ace0 0x7f8d8c06d190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8d98006b20 tx=0x7f8d98006ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.166+0000 7f8d9effd700 1 -- 172.21.15.60:0/2927285179 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8da4053b50 con 0x7f8da80b5ad0 2023-07-13T22:28:19.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.389+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8da80741e0 con 0x7f8da80b5ad0 2023-07-13T22:28:19.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.390+0000 7f8d9effd700 1 -- 172.21.15.60:0/2927285179 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8da4057170 con 0x7f8da80b5ad0 2023-07-13T22:28:19.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d8c06ace0 msgr2=0x7f8d8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d8c06ace0 0x7f8d8c06d190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8d98006b20 tx=0x7f8d98006ab0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da80b5ad0 msgr2=0x7f8da8155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da80b5ad0 0x7f8da8155f60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8da400c700 tx=0x7f8da400cac0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 shutdown_connections 2023-07-13T22:28:19.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8d8c06ace0 0x7f8d8c06d190 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8da80b5ad0 0x7f8da8155f60 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8da80d0b80 0x7f8da814bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 --2- 172.21.15.60:0/2927285179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8da80cf8e0 0x7f8da80c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 >> 172.21.15.60:0/2927285179 conn(0x7f8da8071660 msgr2=0x7f8da806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:19.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 shutdown_connections 2023-07-13T22:28:19.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.392+0000 7f8db05ff700 1 -- 172.21.15.60:0/2927285179 wait complete. 2023-07-13T22:28:19.374 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:28:19.390 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:28:19.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:19.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:19.458 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[147642]: fuse finished with error 0 and tester_r 0 2023-07-13T22:28:19.463 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:28:19.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:19.464 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:28:19.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:28:19.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.780+0000 7fe7355ff700 1 Processor -- start 2023-07-13T22:28:19.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.780+0000 7fe7355ff700 1 -- start start 2023-07-13T22:28:19.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.781+0000 7fe7355ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe7300cf8e0 0x7fe7300cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.781+0000 7fe7355ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7300ceeb0 con 0x7fe7300b5ad0 2023-07-13T22:28:19.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.781+0000 7fe7355ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7300b5f40 con 0x7fe7300d0b80 2023-07-13T22:28:19.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.781+0000 7fe7355ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7300b23f0 con 0x7fe7300cf8e0 2023-07-13T22:28:19.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.782+0000 7fe72e7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe7300b5ad0 0x7fe7300d2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38712/0 (socket says 172.21.15.60:38712) 2023-07-13T22:28:19.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.782+0000 7fe72e7fc700 1 -- 172.21.15.60:0/2932414956 learned_addr learned my addr 172.21.15.60:0/2932414956 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:19.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.782+0000 7fe72f7fe700 1 --2- 172.21.15.60:0/2932414956 >> v2:172.21.15.60:3300/0 conn(0x7fe7300cf8e0 0x7fe7300cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:19.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.782+0000 7fe72dffb700 1 -- 172.21.15.60:0/2932414956 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702295728 0 0) 0x7fe7300ceeb0 con 0x7fe7300b5ad0 2023-07-13T22:28:19.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.782+0000 7fe72dffb700 1 -- 172.21.15.60:0/2932414956 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7180035b0 con 0x7fe7300b5ad0 2023-07-13T22:28:19.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.782+0000 7fe72dffb700 1 -- 172.21.15.60:0/2932414956 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019240160 0 0) 0x7fe7300b5f40 con 0x7fe7300d0b80 2023-07-13T22:28:19.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.782+0000 7fe72f7fe700 1 -- 172.21.15.60:0/2932414956 >> v1:172.21.15.105:6789/0 conn(0x7fe7300d0b80 legacy=0x7fe7300d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.782+0000 7fe72f7fe700 1 -- 172.21.15.60:0/2932414956 >> v1:172.21.15.60:6789/0 conn(0x7fe7300b5ad0 legacy=0x7fe7300d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.782+0000 7fe72f7fe700 1 -- 172.21.15.60:0/2932414956 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7300b5380 con 0x7fe7300cf8e0 2023-07-13T22:28:19.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe72f7fe700 1 --2- 172.21.15.60:0/2932414956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300cf8e0 0x7fe7300cfd60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe728009d90 tx=0x7fe72800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73110de2fd0c333 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:19.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe72dffb700 1 -- 172.21.15.60:0/2932414956 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72800e070 con 0x7fe7300cf8e0 2023-07-13T22:28:19.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe72dffb700 1 -- 172.21.15.60:0/2932414956 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72800bb20 con 0x7fe7300cf8e0 2023-07-13T22:28:19.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe72dffb700 1 -- 172.21.15.60:0/2932414956 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72800bed0 con 0x7fe7300cf8e0 2023-07-13T22:28:19.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe7355ff700 1 -- 172.21.15.60:0/2932414956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300cf8e0 msgr2=0x7fe7300cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe7355ff700 1 --2- 172.21.15.60:0/2932414956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300cf8e0 0x7fe7300cfd60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe728009d90 tx=0x7fe72800b330 comp rx=0 tx=0).stop 2023-07-13T22:28:19.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe7355ff700 1 -- 172.21.15.60:0/2932414956 shutdown_connections 2023-07-13T22:28:19.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe7355ff700 1 --2- 172.21.15.60:0/2932414956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300cf8e0 0x7fe7300cfd60 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe7355ff700 1 -- 172.21.15.60:0/2932414956 >> 172.21.15.60:0/2932414956 conn(0x7fe730071660 msgr2=0x7fe730071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:19.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.783+0000 7fe7355ff700 1 -- 172.21.15.60:0/2932414956 shutdown_connections 2023-07-13T22:28:19.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.784+0000 7fe7355ff700 1 -- 172.21.15.60:0/2932414956 wait complete. 2023-07-13T22:28:19.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.785+0000 7fe7355ff700 1 Processor -- start 2023-07-13T22:28:19.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.785+0000 7fe7355ff700 1 -- start start 2023-07-13T22:28:19.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.786+0000 7fe7355ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7300b5ad0 0x7fe73015a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.786+0000 7fe7355ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7300cf8e0 0x7fe7300cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.786+0000 7fe7355ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300d0b80 0x7fe7301f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.786+0000 7fe7355ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300cce10 con 0x7fe7300d0b80 2023-07-13T22:28:19.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.786+0000 7fe7355ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe730157db0 con 0x7fe7300cf8e0 2023-07-13T22:28:19.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.786+0000 7fe7355ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe730158040 con 0x7fe7300b5ad0 2023-07-13T22:28:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.786+0000 7fe72f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300d0b80 0x7fe7301f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.786+0000 7fe72f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300d0b80 0x7fe7301f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60076/0 (socket says 172.21.15.60:60076) 2023-07-13T22:28:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.786+0000 7fe72f7fe700 1 -- 172.21.15.60:0/3324796532 learned_addr learned my addr 172.21.15.60:0/3324796532 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72effd700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7300b5ad0 0x7fe73015a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72e7fc700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7300cf8e0 0x7fe7300cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72f7fe700 1 -- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7300b5ad0 msgr2=0x7fe73015a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72f7fe700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7300b5ad0 0x7fe73015a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72f7fe700 1 -- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7300cf8e0 msgr2=0x7fe7300cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72f7fe700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7300cf8e0 0x7fe7300cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72f7fe700 1 -- 172.21.15.60:0/3324796532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe728009a70 con 0x7fe7300d0b80 2023-07-13T22:28:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72e7fc700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7300cf8e0 0x7fe7300cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72effd700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7300b5ad0 0x7fe73015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe72f7fe700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300d0b80 0x7fe7301f56d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe728014040 tx=0x7fe728004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe7177fe700 1 -- 172.21.15.60:0/3324796532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72801b810 con 0x7fe7300d0b80 2023-07-13T22:28:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe7177fe700 1 -- 172.21.15.60:0/3324796532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe728020070 con 0x7fe7300d0b80 2023-07-13T22:28:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe7177fe700 1 -- 172.21.15.60:0/3324796532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72801bbb0 con 0x7fe7300d0b80 2023-07-13T22:28:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.787+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe730158320 con 0x7fe7300d0b80 2023-07-13T22:28:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.788+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7301588e0 con 0x7fe7300d0b80 2023-07-13T22:28:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.789+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7300741e0 con 0x7fe7300d0b80 2023-07-13T22:28:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.790+0000 7fe7177fe700 1 -- 172.21.15.60:0/3324796532 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe72801bd10 con 0x7fe7300d0b80 2023-07-13T22:28:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.790+0000 7fe7177fe700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe71c06ace0 0x7fe71c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.791+0000 7fe7177fe700 1 -- 172.21.15.60:0/3324796532 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe728022070 con 0x7fe7300d0b80 2023-07-13T22:28:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.791+0000 7fe72effd700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe71c06ace0 0x7fe71c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.791+0000 7fe72effd700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe71c06ace0 0x7fe71c06d190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe720000f60 tx=0x7fe720009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:19.795+0000 7fe7177fe700 1 -- 172.21.15.60:0/3324796532 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe72805a080 con 0x7fe7300d0b80 2023-07-13T22:28:20.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.046+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe730158aa0 con 0x7fe7300d0b80 2023-07-13T22:28:20.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.047+0000 7fe7177fe700 1 -- 172.21.15.60:0/3324796532 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe728059c10 con 0x7fe7300d0b80 2023-07-13T22:28:20.027 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:28:20.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe71c06ace0 msgr2=0x7fe71c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:20.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe71c06ace0 0x7fe71c06d190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe720000f60 tx=0x7fe720009040 comp rx=0 tx=0).stop 2023-07-13T22:28:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300d0b80 msgr2=0x7fe7301f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300d0b80 0x7fe7301f56d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe728014040 tx=0x7fe728004ad0 comp rx=0 tx=0).stop 2023-07-13T22:28:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 shutdown_connections 2023-07-13T22:28:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe71c06ace0 0x7fe71c06d190 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7300d0b80 0x7fe7301f56d0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7300b5ad0 0x7fe73015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 --2- 172.21.15.60:0/3324796532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7300cf8e0 0x7fe7300cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 >> 172.21.15.60:0/3324796532 conn(0x7fe730071660 msgr2=0x7fe7300cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 shutdown_connections 2023-07-13T22:28:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:20.049+0000 7fe7355ff700 1 -- 172.21.15.60:0/3324796532 wait complete. 2023-07-13T22:28:20.047 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:28:20.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:28:20.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:28:20.048 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:28:20.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:20.048 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:28:20.049 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:28:20.049 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:28:20.049 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:28:20.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:20.050 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:28:20.099 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:28:20.099 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:28:20.099 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:28:20.100 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:20.100 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:28:20.100 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:20.101 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:28:20.101 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:20.101 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:28:20.101 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:28:20.102 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:20.102 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:28:20.102 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:28:20.103 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:20.103 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:28:20.103 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:28:20.103 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41273sec preferred_lft 41273sec 2023-07-13T22:28:20.104 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:28:20.104 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:20.104 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:28:20.104 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:20.105 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:28:20.105 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:28:20.105 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:28:20.105 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:28:20.106 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:20.106 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:28:20.106 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:20.106 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:28:20.107 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:28:20.107 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:28:20.107 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:20.107 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:28:20.108 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:28:20.108 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:28:20.108 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:28:20.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:20.109 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:28:20.178 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:28:20.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:20.182 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:28:20.231 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:28:20.232 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:28:20.232 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:28:20.232 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:28:20.232 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:28:20.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:20.233 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:28:20.234 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:28:20.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:28:20.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:28:20.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:28:20.236 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:28:20.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:20.408 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:28:20.455 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:28:20.455 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:28:20.456 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:28:20.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:20.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:28:20.529 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:28:20.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:28:20.529 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:20.576 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:28:20.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:28:20.576 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:20.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:28:20.771 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:28:20.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:28:20.841 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:28:20.843 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:28:20.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:20.844 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:28:20.891 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:28:20.892 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:28:20.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:28:20.967 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:28:20.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:28:21.039 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:28:21.041 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:28:21.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:21.043 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:28:21.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:28:21.090+0000 7fa3a0bd05c0 -1 init, newargv = 0x55eb98b17e10 newargc=15 2023-07-13T22:28:21.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[148646]: starting ceph client 2023-07-13T22:28:21.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[148646]: starting fuse 2023-07-13T22:28:21.097 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:28:21.097 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:28:21.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:28:21.098 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:28:21.098 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:28:21.099 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:28:21.099 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:28:21.099 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:21.100 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:28:21.100 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:28:21.100 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:28:21.101 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:28:21.101 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:21.101 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:28:21.101 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:28:21.102 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:28:21.102 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:28:21.102 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:21.103 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:28:21.103 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:28:21.103 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:28:21.103 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:28:21.104 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:28:21.104 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:28:21.104 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:28:21.105 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:28:21.105 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:21.105 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:28:21.106 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:28:21.286 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.148646.asok 2023-07-13T22:28:21.463 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.148646.asok 2023-07-13T22:28:21.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:21.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.148646.asok status 2023-07-13T22:28:21.695 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:28:21.695 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:28:21.696 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:28:21.696 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:28:21.696 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:28:21.696 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:28:21.696 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:28:21.696 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "148646", 2023-07-13T22:28:21.696 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:28:21.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:21.697 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:28:21.697 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:28:21.697 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7650, 2023-07-13T22:28:21.697 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:28:21.697 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:28:21.697 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:28:21.697 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 7650 2023-07-13T22:28:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:28:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:28:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:28:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 763246751 2023-07-13T22:28:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:28:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:28:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:28:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 763246751 2023-07-13T22:28:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.7650 192.168.0.1:0/763246751", 2023-07-13T22:28:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/763246751", 2023-07-13T22:28:21.699 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:28:21.700 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 52, 2023-07-13T22:28:21.700 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 87, 2023-07-13T22:28:21.700 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:28:21.700 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:28:21.700 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:28:21.700 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:28:21.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:21.709 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:28:21.710 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:21.756 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:28:21.757 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:21.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:21.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:21.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:28:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.051+0000 7fb7c3dff700 1 Processor -- start 2023-07-13T22:28:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.052+0000 7fb7c3dff700 1 -- start start 2023-07-13T22:28:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c3dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7bc0d0b80 0x7fb7bc0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c3dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0d0f50 0x7fb7bc0d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c3dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7bc0ceeb0 con 0x7fb7bc0cf8e0 2023-07-13T22:28:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c3dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7bc0b23f0 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c3dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0b4bb0 con 0x7fb7bc0d0f50 2023-07-13T22:28:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c1b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7bc0d0b80 0x7fb7bc0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c1b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7bc0d0b80 0x7fb7bc0b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60086/0 (socket says 172.21.15.60:60086) 2023-07-13T22:28:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c1b9b700 1 -- 172.21.15.60:0/2891735464 learned_addr learned my addr 172.21.15.60:0/2891735464 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c139a700 1 --2- 172.21.15.60:0/2891735464 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0d0f50 0x7fb7bc0d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c1b9b700 1 -- 172.21.15.60:0/2891735464 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0d0f50 msgr2=0x7fb7bc0d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c1b9b700 1 --2- 172.21.15.60:0/2891735464 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0d0f50 0x7fb7bc0d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c1b9b700 1 -- 172.21.15.60:0/2891735464 >> v1:172.21.15.105:6789/0 conn(0x7fb7bc0cf8e0 legacy=0x7fb7bc0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.053+0000 7fb7c1b9b700 1 -- 172.21.15.60:0/2891735464 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc0b5380 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.054+0000 7fb7c1b9b700 1 --2- 172.21.15.60:0/2891735464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 0x7fb7bc0b5ad0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80098f0 tx=0x7fb7b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f868a9195e178e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.054+0000 7fb7c0b99700 1 -- 172.21.15.60:0/2891735464 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8004230 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.054+0000 7fb7c0b99700 1 -- 172.21.15.60:0/2891735464 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8004390 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.054+0000 7fb7c0b99700 1 -- 172.21.15.60:0/2891735464 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80036a0 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.054+0000 7fb7c3dff700 1 -- 172.21.15.60:0/2891735464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 msgr2=0x7fb7bc0b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.054+0000 7fb7c3dff700 1 --2- 172.21.15.60:0/2891735464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 0x7fb7bc0b5ad0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80098f0 tx=0x7fb7b8009c00 comp rx=0 tx=0).stop 2023-07-13T22:28:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.055+0000 7fb7c3dff700 1 -- 172.21.15.60:0/2891735464 shutdown_connections 2023-07-13T22:28:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.055+0000 7fb7c3dff700 1 --2- 172.21.15.60:0/2891735464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 0x7fb7bc0b5ad0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.055+0000 7fb7c3dff700 1 --2- 172.21.15.60:0/2891735464 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0d0f50 0x7fb7bc0d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.055+0000 7fb7c3dff700 1 -- 172.21.15.60:0/2891735464 >> 172.21.15.60:0/2891735464 conn(0x7fb7bc071660 msgr2=0x7fb7bc071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.055+0000 7fb7c3dff700 1 -- 172.21.15.60:0/2891735464 shutdown_connections 2023-07-13T22:28:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.055+0000 7fb7c3dff700 1 -- 172.21.15.60:0/2891735464 wait complete. 2023-07-13T22:28:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.056+0000 7fb7c3dff700 1 Processor -- start 2023-07-13T22:28:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.057+0000 7fb7c3dff700 1 -- start start 2023-07-13T22:28:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.057+0000 7fb7c3dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0cf8e0 0x7fb7bc0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.057+0000 7fb7c3dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 0x7fb7bc1f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.057+0000 7fb7c3dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7bc0d0f50 0x7fb7bc15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.057+0000 7fb7c3dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc1e5e50 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.057+0000 7fb7c3dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc1e4df0 con 0x7fb7bc0d0f50 2023-07-13T22:28:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.057+0000 7fb7c3dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc1e5090 con 0x7fb7bc0cf8e0 2023-07-13T22:28:22.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c139a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 0x7fb7bc1f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:22.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c139a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 0x7fb7bc1f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60092/0 (socket says 172.21.15.60:60092) 2023-07-13T22:28:22.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c139a700 1 -- 172.21.15.60:0/3151031249 learned_addr learned my addr 172.21.15.60:0/3151031249 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:22.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c1b9b700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0cf8e0 0x7fb7bc0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:22.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c239c700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7bc0d0f50 0x7fb7bc15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:22.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c139a700 1 -- 172.21.15.60:0/3151031249 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0cf8e0 msgr2=0x7fb7bc0c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:22.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c139a700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0cf8e0 0x7fb7bc0c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:22.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c139a700 1 -- 172.21.15.60:0/3151031249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7bc0d0f50 msgr2=0x7fb7bc15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:22.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c139a700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7bc0d0f50 0x7fb7bc15b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:22.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c139a700 1 -- 172.21.15.60:0/3151031249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b8009550 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c239c700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7bc0d0f50 0x7fb7bc15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:22.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.058+0000 7fb7c139a700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 0x7fb7bc1f2490 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00d800 tx=0x7fb7ac00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:22.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.059+0000 7fb7b27fc700 1 -- 172.21.15.60:0/3151031249 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0125f0 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.059+0000 7fb7b27fc700 1 -- 172.21.15.60:0/3151031249 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac012750 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.059+0000 7fb7b27fc700 1 -- 172.21.15.60:0/3151031249 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac012b00 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.059+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc1e5370 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.059+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc1e5930 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.060+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7bc1e5af0 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.061+0000 7fb7b27fc700 1 -- 172.21.15.60:0/3151031249 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7ac012c60 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.062+0000 7fb7b27fc700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a006ace0 0x7fb7a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:22.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.062+0000 7fb7b27fc700 1 -- 172.21.15.60:0/3151031249 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7ac089860 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.062+0000 7fb7c1b9b700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a006ace0 0x7fb7a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:22.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.063+0000 7fb7c1b9b700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a006ace0 0x7fb7a006d190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009130 tx=0x7fb7b8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:22.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.066+0000 7fb7b27fc700 1 -- 172.21.15.60:0/3151031249 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb7ac054870 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.316+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7bc0741e0 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.316+0000 7fb7b27fc700 1 -- 172.21.15.60:0/3151031249 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fb7ac057e90 con 0x7fb7bc0d0b80 2023-07-13T22:28:22.296 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:22.297 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2395854295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2395854295},{"type":"v1","addr":"172.21.15.105:6834","nonce":2395854295}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1048261776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1048261776},{"type":"v1","addr":"172.21.15.105:6835","nonce":1048261776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/1639899719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1639899719},{"type":"v1","addr":"172.21.15.60:6836","nonce":1639899719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:04.618240+0000","modified":"2023-07-13T22:28:08.127584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7434":{"gid":7434,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/1512512599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":1512512599},{"type":"v1","addr":"172.21.15.60:6837","nonce":1512512599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-13T22:28:22.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a006ace0 msgr2=0x7fb7a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:22.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a006ace0 0x7fb7a006d190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009130 tx=0x7fb7b8008030 comp rx=0 tx=0).stop 2023-07-13T22:28:22.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 msgr2=0x7fb7bc1f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:22.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 0x7fb7bc1f2490 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00d800 tx=0x7fb7ac00db10 comp rx=0 tx=0).stop 2023-07-13T22:28:22.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 shutdown_connections 2023-07-13T22:28:22.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb7a006ace0 0x7fb7a006d190 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:22.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7bc0d0f50 0x7fb7bc15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:22.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7bc0cf8e0 0x7fb7bc0c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:22.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 --2- 172.21.15.60:0/3151031249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7bc0d0b80 0x7fb7bc1f2490 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:22.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 >> 172.21.15.60:0/3151031249 conn(0x7fb7bc071660 msgr2=0x7fb7bc0b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:22.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 shutdown_connections 2023-07-13T22:28:22.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:22.319+0000 7fb7c3dff700 1 -- 172.21.15.60:0/3151031249 wait complete. 2023-07-13T22:28:22.301 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 52 2023-07-13T22:28:22.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_max_completed_requests 1000 2023-07-13T22:28:22.539 INFO:teuthology.orchestra.run.smithi060.stdout:{"success":"mds_max_completed_requests = '1000' (not observed, change may require restart) "} 2023-07-13T22:28:22.540 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_max_completed_requests = '1000' (not observed, change may require restart) " } 2023-07-13T22:28:22.540 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:28:22.540 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:22.587 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:28:22.588 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:28:22.588 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2023-07-13T22:28:22.588 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:22.589 DEBUG:teuthology.orchestra.run.smithi060:> n = 1100 2023-07-13T22:28:22.589 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2023-07-13T22:28:22.589 DEBUG:teuthology.orchestra.run.smithi060:> if 0 > 0: 2023-07-13T22:28:22.589 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2023-07-13T22:28:22.590 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file1" 2023-07-13T22:28:22.590 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:22.590 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2023-07-13T22:28:22.591 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2023-07-13T22:28:22.591 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2023-07-13T22:28:22.591 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:22.591 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-07-13T22:28:22.592 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-13T22:28:22.592 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:22.592 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2023-07-13T22:28:22.592 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-13T22:28:22.593 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2023-07-13T22:28:22.593 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2023-07-13T22:28:22.593 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:28:22.594 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2023-07-13T22:28:22.594 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2023-07-13T22:28:22.594 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:28:22.594 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2023-07-13T22:28:22.595 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:28:22.595 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:28:22.595 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2023-07-13T22:28:22.595 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(0): 2023-07-13T22:28:22.596 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-13T22:28:22.596 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:28:22.596 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:28:22.596 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2023-07-13T22:28:22.597 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2023-07-13T22:28:22.597 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:28:25.730 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:28:25.730 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:25.816 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:28:25.817 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:28:25.817 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2023-07-13T22:28:25.817 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:25.817 DEBUG:teuthology.orchestra.run.smithi060:> n = 5 2023-07-13T22:28:25.818 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2023-07-13T22:28:25.818 DEBUG:teuthology.orchestra.run.smithi060:> if 0 > 0: 2023-07-13T22:28:25.818 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2023-07-13T22:28:25.818 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2023-07-13T22:28:25.819 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:25.819 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2023-07-13T22:28:25.819 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2023-07-13T22:28:25.819 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2023-07-13T22:28:25.820 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:25.820 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-07-13T22:28:25.820 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-13T22:28:25.821 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:28:25.821 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2023-07-13T22:28:25.821 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-13T22:28:25.821 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2023-07-13T22:28:25.822 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2023-07-13T22:28:25.822 DEBUG:teuthology.orchestra.run.smithi060:> if True: 2023-07-13T22:28:25.822 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2023-07-13T22:28:25.822 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2023-07-13T22:28:25.823 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:28:25.823 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2023-07-13T22:28:25.823 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:28:25.824 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:28:25.824 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2023-07-13T22:28:25.824 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(0): 2023-07-13T22:28:25.824 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-13T22:28:25.825 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-07-13T22:28:25.825 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2023-07-13T22:28:25.825 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2023-07-13T22:28:25.825 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2023-07-13T22:28:25.826 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:28:26.155 INFO:tasks.ceph_test_case:waiting 100s for health warning matching MDS_CLIENT_OLDEST_TID 2023-07-13T22:28:26.155 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-07-13T22:28:26.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.418+0000 7f9322fff700 1 Processor -- start 2023-07-13T22:28:26.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.418+0000 7f9322fff700 1 -- start start 2023-07-13T22:28:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f9322fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f931c0b5ad0 0x7f931c0d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f9322fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f931c0cf8e0 0x7f931c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f9322fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c0ceeb0 con 0x7f931c0d0b80 2023-07-13T22:28:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f9322fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c0b23f0 con 0x7f931c0cf8e0 2023-07-13T22:28:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f9322fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c0b4bb0 con 0x7f931c0b5ad0 2023-07-13T22:28:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f932159c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f931c0cf8e0 0x7f931c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f932159c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f931c0cf8e0 0x7f931c0cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45804/0 (socket says 172.21.15.60:45804) 2023-07-13T22:28:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f932159c700 1 -- 172.21.15.60:0/3774148783 learned_addr learned my addr 172.21.15.60:0/3774148783 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f9320d9b700 1 --1- 172.21.15.60:0/3774148783 >> v1:172.21.15.60:6789/0 conn(0x7f931c0d0b80 0x7f931c0d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53932/0 (socket says 172.21.15.60:53932) 2023-07-13T22:28:26.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.419+0000 7f931bfff700 1 --2- 172.21.15.60:0/3774148783 >> v2:172.21.15.105:3300/0 conn(0x7f931c0b5ad0 0x7f931c0d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:26.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.420+0000 7f932159c700 1 -- 172.21.15.60:0/3774148783 >> v2:172.21.15.105:3300/0 conn(0x7f931c0b5ad0 msgr2=0x7f931c0d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:26.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.420+0000 7f932159c700 1 --2- 172.21.15.60:0/3774148783 >> v2:172.21.15.105:3300/0 conn(0x7f931c0b5ad0 0x7f931c0d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.420+0000 7f932159c700 1 -- 172.21.15.60:0/3774148783 >> v1:172.21.15.60:6789/0 conn(0x7f931c0d0b80 legacy=0x7f931c0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:26.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.420+0000 7f932159c700 1 -- 172.21.15.60:0/3774148783 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c0b5f40 con 0x7f931c0cf8e0 2023-07-13T22:28:26.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.420+0000 7f931b7fe700 1 -- 172.21.15.60:0/3774148783 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441190232 0 0) 0x7f931c0ceeb0 con 0x7f931c0d0b80 2023-07-13T22:28:26.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.420+0000 7f932159c700 1 --2- 172.21.15.60:0/3774148783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0cf8e0 0x7f931c0cfd60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f931c0d0930 tx=0x7f931400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e0cc17bb6e60c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:26.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.420+0000 7f931b7fe700 1 -- 172.21.15.60:0/3774148783 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400e070 con 0x7f931c0cf8e0 2023-07-13T22:28:26.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.420+0000 7f931b7fe700 1 -- 172.21.15.60:0/3774148783 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931400b910 con 0x7f931c0cf8e0 2023-07-13T22:28:26.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.420+0000 7f931b7fe700 1 -- 172.21.15.60:0/3774148783 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400bcc0 con 0x7f931c0cf8e0 2023-07-13T22:28:26.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.421+0000 7f9322fff700 1 -- 172.21.15.60:0/3774148783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0cf8e0 msgr2=0x7f931c0cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:26.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.421+0000 7f9322fff700 1 --2- 172.21.15.60:0/3774148783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0cf8e0 0x7f931c0cfd60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f931c0d0930 tx=0x7f931400b330 comp rx=0 tx=0).stop 2023-07-13T22:28:26.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.421+0000 7f9322fff700 1 -- 172.21.15.60:0/3774148783 shutdown_connections 2023-07-13T22:28:26.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.421+0000 7f9322fff700 1 --2- 172.21.15.60:0/3774148783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0cf8e0 0x7f931c0cfd60 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.421+0000 7f9322fff700 1 --2- 172.21.15.60:0/3774148783 >> v2:172.21.15.105:3300/0 conn(0x7f931c0b5ad0 0x7f931c0d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.421+0000 7f9322fff700 1 -- 172.21.15.60:0/3774148783 >> 172.21.15.60:0/3774148783 conn(0x7f931c071660 msgr2=0x7f931c071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:26.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.421+0000 7f9322fff700 1 -- 172.21.15.60:0/3774148783 shutdown_connections 2023-07-13T22:28:26.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.421+0000 7f9322fff700 1 -- 172.21.15.60:0/3774148783 wait complete. 2023-07-13T22:28:26.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.423+0000 7f9322fff700 1 Processor -- start 2023-07-13T22:28:26.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.423+0000 7f9322fff700 1 -- start start 2023-07-13T22:28:26.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f9322fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f931c0b5ad0 0x7f931c0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:26.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f9322fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f931c0cf8e0 0x7f931c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:26.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f9322fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0d0b80 0x7f931c14bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:26.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f9322fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c1555c0 con 0x7f931c0d0b80 2023-07-13T22:28:26.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f9322fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c15c260 con 0x7f931c0b5ad0 2023-07-13T22:28:26.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f9322fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c15c500 con 0x7f931c0cf8e0 2023-07-13T22:28:26.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f932159c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0d0b80 0x7f931c14bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:26.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f932159c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0d0b80 0x7f931c14bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45808/0 (socket says 172.21.15.60:45808) 2023-07-13T22:28:26.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f932159c700 1 -- 172.21.15.60:0/313242629 learned_addr learned my addr 172.21.15.60:0/313242629 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:26.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f9320d9b700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f931c0b5ad0 0x7f931c0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:26.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.424+0000 7f931bfff700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f931c0cf8e0 0x7f931c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:26.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f932159c700 1 -- 172.21.15.60:0/313242629 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f931c0cf8e0 msgr2=0x7f931c0b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:26.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f932159c700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f931c0cf8e0 0x7f931c0b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f932159c700 1 -- 172.21.15.60:0/313242629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f931c0b5ad0 msgr2=0x7f931c0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:26.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f932159c700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f931c0b5ad0 0x7f931c0cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f932159c700 1 -- 172.21.15.60:0/313242629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009b30 con 0x7f931c0d0b80 2023-07-13T22:28:26.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f932159c700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0d0b80 0x7f931c14bca0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f931c0d0a50 tx=0x7f9314004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:26.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f93197fa700 1 -- 172.21.15.60:0/313242629 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314004840 con 0x7f931c0d0b80 2023-07-13T22:28:26.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f93197fa700 1 -- 172.21.15.60:0/313242629 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93140099b0 con 0x7f931c0d0b80 2023-07-13T22:28:26.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c15c7e0 con 0x7f931c0d0b80 2023-07-13T22:28:26.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f93197fa700 1 -- 172.21.15.60:0/313242629 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931401c680 con 0x7f931c0d0b80 2023-07-13T22:28:26.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.425+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c159b10 con 0x7f931c0d0b80 2023-07-13T22:28:26.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.426+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c0c9f00 con 0x7f931c0d0b80 2023-07-13T22:28:26.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.427+0000 7f93197fa700 1 -- 172.21.15.60:0/313242629 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f931401c7e0 con 0x7f931c0d0b80 2023-07-13T22:28:26.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.429+0000 7f93197fa700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930006ace0 0x7f930006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:26.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.429+0000 7f93197fa700 1 -- 172.21.15.60:0/313242629 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f931408ad40 con 0x7f931c0d0b80 2023-07-13T22:28:26.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.429+0000 7f9320d9b700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930006ace0 0x7f930006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:26.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.430+0000 7f9320d9b700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930006ace0 0x7f930006d190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9310007fd0 tx=0x7f9310007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:26.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.434+0000 7f93197fa700 1 -- 172.21.15.60:0/313242629 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f931c0c9f00 con 0x7f931c0d0b80 2023-07-13T22:28:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.727+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f931c0c9f00 con 0x7f931c0d0b80 2023-07-13T22:28:26.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.728+0000 7f93197fa700 1 -- 172.21.15.60:0/313242629 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+46 (secure 0 0 0) 0x7f931c0c9f00 con 0x7f931c0d0b80 2023-07-13T22:28:26.708 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:26.708 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:28:26.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930006ace0 msgr2=0x7f930006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:26.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930006ace0 0x7f930006d190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9310007fd0 tx=0x7f9310007de0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0d0b80 msgr2=0x7f931c14bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:26.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0d0b80 0x7f931c14bca0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f931c0d0a50 tx=0x7f9314004ad0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 shutdown_connections 2023-07-13T22:28:26.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930006ace0 0x7f930006d190 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c0d0b80 0x7f931c14bca0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f931c0b5ad0 0x7f931c0cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 --2- 172.21.15.60:0/313242629 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f931c0cf8e0 0x7f931c0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:26.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 >> 172.21.15.60:0/313242629 conn(0x7f931c071660 msgr2=0x7f931c0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:26.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 shutdown_connections 2023-07-13T22:28:26.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:26.730+0000 7f9322fff700 1 -- 172.21.15.60:0/313242629 wait complete. 2023-07-13T22:28:26.728 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:28:26.729 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-07-13T22:28:26.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:28:26.989 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:28:31.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-07-13T22:28:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.953+0000 7f1f119ff700 1 Processor -- start 2023-07-13T22:28:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.953+0000 7f1f119ff700 1 -- start start 2023-07-13T22:28:31.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.954+0000 7f1f119ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f0c0d0b80 0x7f1f0c0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:31.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.954+0000 7f1f119ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f0c0cf8e0 0x7f1f0c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:31.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.954+0000 7f1f119ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c0ceeb0 con 0x7f1f0c0d0f50 2023-07-13T22:28:31.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.954+0000 7f1f119ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0b23f0 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.954+0000 7f1f119ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0b4bb0 con 0x7f1f0c0cf8e0 2023-07-13T22:28:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.954+0000 7f1f0affd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f0c0d0b80 0x7f1f0c0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.954+0000 7f1f0affd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f0c0d0b80 0x7f1f0c0b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45816/0 (socket says 172.21.15.60:45816) 2023-07-13T22:28:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.954+0000 7f1f0affd700 1 -- 172.21.15.60:0/3222769688 learned_addr learned my addr 172.21.15.60:0/3222769688 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.954+0000 7f1f03fff700 1 --1- 172.21.15.60:0/3222769688 >> v1:172.21.15.60:6789/0 conn(0x7f1f0c0d0f50 0x7f1f0c0d2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53936/0 (socket says 172.21.15.60:53936) 2023-07-13T22:28:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.955+0000 7f1f0b7fe700 1 --2- 172.21.15.60:0/3222769688 >> v2:172.21.15.105:3300/0 conn(0x7f1f0c0cf8e0 0x7f1f0c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.955+0000 7f1f0affd700 1 -- 172.21.15.60:0/3222769688 >> v2:172.21.15.105:3300/0 conn(0x7f1f0c0cf8e0 msgr2=0x7f1f0c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.955+0000 7f1f0affd700 1 --2- 172.21.15.60:0/3222769688 >> v2:172.21.15.105:3300/0 conn(0x7f1f0c0cf8e0 0x7f1f0c0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.955+0000 7f1f0affd700 1 -- 172.21.15.60:0/3222769688 >> v1:172.21.15.60:6789/0 conn(0x7f1f0c0d0f50 legacy=0x7f1f0c0d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.955+0000 7f1f0affd700 1 -- 172.21.15.60:0/3222769688 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0b5380 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.955+0000 7f1f0affd700 1 --2- 172.21.15.60:0/3222769688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 0x7f1f0c0b5ad0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c06a2d0 tx=0x7f1efc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d220ff3da20d8f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.955+0000 7f1f0a7fc700 1 -- 172.21.15.60:0/3222769688 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00b870 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.955+0000 7f1f0a7fc700 1 -- 172.21.15.60:0/3222769688 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc00b9d0 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.955+0000 7f1f0a7fc700 1 -- 172.21.15.60:0/3222769688 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00bd60 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.956+0000 7f1f119ff700 1 -- 172.21.15.60:0/3222769688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 msgr2=0x7f1f0c0b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.956+0000 7f1f119ff700 1 --2- 172.21.15.60:0/3222769688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 0x7f1f0c0b5ad0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c06a2d0 tx=0x7f1efc009bd0 comp rx=0 tx=0).stop 2023-07-13T22:28:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.956+0000 7f1f119ff700 1 -- 172.21.15.60:0/3222769688 shutdown_connections 2023-07-13T22:28:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.956+0000 7f1f119ff700 1 --2- 172.21.15.60:0/3222769688 >> v2:172.21.15.105:3300/0 conn(0x7f1f0c0cf8e0 0x7f1f0c0cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.956+0000 7f1f119ff700 1 --2- 172.21.15.60:0/3222769688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 0x7f1f0c0b5ad0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.956+0000 7f1f119ff700 1 -- 172.21.15.60:0/3222769688 >> 172.21.15.60:0/3222769688 conn(0x7f1f0c071660 msgr2=0x7f1f0c071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.956+0000 7f1f119ff700 1 -- 172.21.15.60:0/3222769688 shutdown_connections 2023-07-13T22:28:31.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.956+0000 7f1f119ff700 1 -- 172.21.15.60:0/3222769688 wait complete. 2023-07-13T22:28:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.958+0000 7f1f119ff700 1 Processor -- start 2023-07-13T22:28:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.958+0000 7f1f119ff700 1 -- start start 2023-07-13T22:28:31.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f119ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c0cf8e0 0x7f1f0c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f119ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 0x7f1f0c0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f119ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c0d0f50 0x7f1f0c0b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f119ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c15c680 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f119ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1590b0 con 0x7f1f0c0d0f50 2023-07-13T22:28:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f119ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c15ba00 con 0x7f1f0c0cf8e0 2023-07-13T22:28:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f03fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 0x7f1f0c0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f03fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 0x7f1f0c0cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45824/0 (socket says 172.21.15.60:45824) 2023-07-13T22:28:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f03fff700 1 -- 172.21.15.60:0/2517687190 learned_addr learned my addr 172.21.15.60:0/2517687190 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f0affd700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c0cf8e0 0x7f1f0c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.959+0000 7f1f0b7fe700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c0d0f50 0x7f1f0c0b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f03fff700 1 -- 172.21.15.60:0/2517687190 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c0cf8e0 msgr2=0x7f1f0c158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f03fff700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c0cf8e0 0x7f1f0c158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f03fff700 1 -- 172.21.15.60:0/2517687190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c0d0f50 msgr2=0x7f1f0c0b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f03fff700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c0d0f50 0x7f1f0c0b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:31.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f03fff700 1 -- 172.21.15.60:0/2517687190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc009550 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f0b7fe700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c0d0f50 0x7f1f0c0b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f03fff700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 0x7f1f0c0cc3a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400c520 tx=0x7f1ef400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f08ff9700 1 -- 172.21.15.60:0/2517687190 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4005cb0 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f08ff9700 1 -- 172.21.15.60:0/2517687190 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4005e10 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c15bce0 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f08ff9700 1 -- 172.21.15.60:0/2517687190 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40072d0 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.960+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c15c2a0 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.961+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c15c7c0 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.962+0000 7f1f08ff9700 1 -- 172.21.15.60:0/2517687190 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ef4007430 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.963+0000 7f1f08ff9700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1eec06ace0 0x7f1eec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.963+0000 7f1f08ff9700 1 -- 172.21.15.60:0/2517687190 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ef4088850 con 0x7f1f0c0d0b80 2023-07-13T22:28:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.963+0000 7f1f0affd700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1eec06ace0 0x7f1eec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:31.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.964+0000 7f1f0affd700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1eec06ace0 0x7f1eec06d190 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1efc000c00 tx=0x7f1efc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:31.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:31.967+0000 7f1f08ff9700 1 -- 172.21.15.60:0/2517687190 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1f0c15c7c0 con 0x7f1f0c0d0b80 2023-07-13T22:28:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.260+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f1f0c15c7c0 con 0x7f1f0c0d0b80 2023-07-13T22:28:32.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.260+0000 7f1f08ff9700 1 -- 172.21.15.60:0/2517687190 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+322 (secure 0 0 0) 0x7f1f0c15c7c0 con 0x7f1f0c0d0b80 2023-07-13T22:28:32.241 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:32.241 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_OLDEST_TID":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to advance oldest client/flush tid","count":1},"detail":[{"message":"mds.a(mds.0): Client smithi060:0 failing to advance its oldest client/flush tid. client_id: 7650"}],"muted":false}},"mutes":[]} 2023-07-13T22:28:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1eec06ace0 msgr2=0x7f1eec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1eec06ace0 0x7f1eec06d190 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1efc000c00 tx=0x7f1efc008030 comp rx=0 tx=0).stop 2023-07-13T22:28:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 msgr2=0x7f1f0c0cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 0x7f1f0c0cc3a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400c520 tx=0x7f1ef400c830 comp rx=0 tx=0).stop 2023-07-13T22:28:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 shutdown_connections 2023-07-13T22:28:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1eec06ace0 0x7f1eec06d190 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f0c0d0f50 0x7f1f0c0b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f0c0cf8e0 0x7f1f0c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 --2- 172.21.15.60:0/2517687190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0b80 0x7f1f0c0cc3a0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 >> 172.21.15.60:0/2517687190 conn(0x7f1f0c071660 msgr2=0x7f1f0c0cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.262+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 shutdown_connections 2023-07-13T22:28:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.263+0000 7f1f119ff700 1 -- 172.21.15.60:0/2517687190 wait complete. 2023-07-13T22:28:32.260 DEBUG:tasks.ceph_test_case:detail_strings: ['mds.a(mds.0): Client smithi060:0 failing to advance its oldest client/flush tid. client_id: 7650'] 2023-07-13T22:28:32.261 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-07-13T22:28:32.261 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:28:32.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:28:32.365 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:28:32.472 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:28:32.473 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:28:32.473 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:32.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:32.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.786+0000 7f0923dff700 1 Processor -- start 2023-07-13T22:28:32.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.786+0000 7f0923dff700 1 -- start start 2023-07-13T22:28:32.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.787+0000 7f0923dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 0x7f091c15d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:32.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.787+0000 7f0923dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c0cc4a0 con 0x7f091c1594f0 2023-07-13T22:28:32.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.787+0000 7f0923dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c0cc6f0 con 0x7f091c0c9840 2023-07-13T22:28:32.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.787+0000 7f0923dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0cc9d0 con 0x7f091c0b0ed0 2023-07-13T22:28:32.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.787+0000 7f0921b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f091c1594f0 0x7f091c0c9130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53950/0 (socket says 172.21.15.60:53950) 2023-07-13T22:28:32.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.787+0000 7f0921b9b700 1 -- 172.21.15.60:0/2324012746 learned_addr learned my addr 172.21.15.60:0/2324012746 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:32.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.788+0000 7f092239c700 1 --2- 172.21.15.60:0/2324012746 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 0x7f091c15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:32.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.788+0000 7f0920b99700 1 -- 172.21.15.60:0/2324012746 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475170032 0 0) 0x7f091c0cc4a0 con 0x7f091c1594f0 2023-07-13T22:28:32.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.788+0000 7f0920b99700 1 -- 172.21.15.60:0/2324012746 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09080035b0 con 0x7f091c1594f0 2023-07-13T22:28:32.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.788+0000 7f0920b99700 1 -- 172.21.15.60:0/2324012746 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879157914 0 0) 0x7f091c0cc6f0 con 0x7f091c0c9840 2023-07-13T22:28:32.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.788+0000 7f0920b99700 1 -- 172.21.15.60:0/2324012746 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f091c0cc4a0 con 0x7f091c0c9840 2023-07-13T22:28:32.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.788+0000 7f092239c700 1 -- 172.21.15.60:0/2324012746 >> v1:172.21.15.105:6789/0 conn(0x7f091c0c9840 legacy=0x7f091c15ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:32.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.788+0000 7f092239c700 1 -- 172.21.15.60:0/2324012746 >> v1:172.21.15.60:6789/0 conn(0x7f091c1594f0 legacy=0x7f091c0c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:32.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.788+0000 7f092239c700 1 -- 172.21.15.60:0/2324012746 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0cccb0 con 0x7f091c0b0ed0 2023-07-13T22:28:32.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.788+0000 7f092239c700 1 --2- 172.21.15.60:0/2324012746 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 0x7f091c15d570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0914009dc0 tx=0x7f091400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e669592d03198c7d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:32.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0920b99700 1 -- 172.21.15.60:0/2324012746 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091400e070 con 0x7f091c0b0ed0 2023-07-13T22:28:32.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0920b99700 1 -- 172.21.15.60:0/2324012746 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091400bb20 con 0x7f091c0b0ed0 2023-07-13T22:28:32.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0920b99700 1 -- 172.21.15.60:0/2324012746 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914012430 con 0x7f091c0b0ed0 2023-07-13T22:28:32.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0923dff700 1 -- 172.21.15.60:0/2324012746 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 msgr2=0x7f091c15d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:32.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0923dff700 1 --2- 172.21.15.60:0/2324012746 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 0x7f091c15d570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0914009dc0 tx=0x7f091400b330 comp rx=0 tx=0).stop 2023-07-13T22:28:32.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0923dff700 1 -- 172.21.15.60:0/2324012746 shutdown_connections 2023-07-13T22:28:32.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0923dff700 1 --2- 172.21.15.60:0/2324012746 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 0x7f091c15d570 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0923dff700 1 -- 172.21.15.60:0/2324012746 >> 172.21.15.60:0/2324012746 conn(0x7f091c0ce960 msgr2=0x7f091c0ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:32.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0923dff700 1 -- 172.21.15.60:0/2324012746 shutdown_connections 2023-07-13T22:28:32.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.789+0000 7f0923dff700 1 -- 172.21.15.60:0/2324012746 wait complete. 2023-07-13T22:28:32.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.792+0000 7f0923dff700 1 Processor -- start 2023-07-13T22:28:32.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.792+0000 7f0923dff700 1 -- start start 2023-07-13T22:28:32.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.793+0000 7f0923dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 0x7f091c1f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:32.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.793+0000 7f0923dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0c9840 0x7f091c0c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:32.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.793+0000 7f0923dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091c1594f0 0x7f091c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:32.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.793+0000 7f0923dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c1f3490 con 0x7f091c1594f0 2023-07-13T22:28:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.793+0000 7f0923dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0acdf0 con 0x7f091c0c9840 2023-07-13T22:28:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.793+0000 7f0923dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0ad0d0 con 0x7f091c0b0ed0 2023-07-13T22:28:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092239c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091c1594f0 0x7f091c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092239c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091c1594f0 0x7f091c06ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45826/0 (socket says 172.21.15.60:45826) 2023-07-13T22:28:32.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092239c700 1 -- 172.21.15.60:0/462746812 learned_addr learned my addr 172.21.15.60:0/462746812 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:32.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f0921b9b700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 0x7f091c1f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:32.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092139a700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0c9840 0x7f091c0c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092239c700 1 -- 172.21.15.60:0/462746812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 msgr2=0x7f091c1f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092239c700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 0x7f091c1f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092239c700 1 -- 172.21.15.60:0/462746812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0c9840 msgr2=0x7f091c0c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092239c700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0c9840 0x7f091c0c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092239c700 1 -- 172.21.15.60:0/462746812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914009a70 con 0x7f091c1594f0 2023-07-13T22:28:32.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092139a700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0c9840 0x7f091c0c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:32.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.794+0000 7f092239c700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091c1594f0 0x7f091c06ba30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f091400b370 tx=0x7f0914004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:32.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.795+0000 7f09127fc700 1 -- 172.21.15.60:0/462746812 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091401b810 con 0x7f091c1594f0 2023-07-13T22:28:32.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.795+0000 7f09127fc700 1 -- 172.21.15.60:0/462746812 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914020070 con 0x7f091c1594f0 2023-07-13T22:28:32.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.795+0000 7f09127fc700 1 -- 172.21.15.60:0/462746812 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091401bbf0 con 0x7f091c1594f0 2023-07-13T22:28:32.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.795+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c0ad3b0 con 0x7f091c1594f0 2023-07-13T22:28:32.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.795+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c0ad970 con 0x7f091c1594f0 2023-07-13T22:28:32.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.796+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c066b20 con 0x7f091c1594f0 2023-07-13T22:28:32.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.797+0000 7f09127fc700 1 -- 172.21.15.60:0/462746812 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f091401bd50 con 0x7f091c1594f0 2023-07-13T22:28:32.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.798+0000 7f09127fc700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f090006ace0 0x7f090006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:32.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.798+0000 7f09127fc700 1 -- 172.21.15.60:0/462746812 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0914022070 con 0x7f091c1594f0 2023-07-13T22:28:32.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.798+0000 7f0921b9b700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f090006ace0 0x7f090006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:32.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.798+0000 7f0921b9b700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f090006ace0 0x7f090006d190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0918007fd0 tx=0x7f0918007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:32.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:32.802+0000 7f09127fc700 1 -- 172.21.15.60:0/462746812 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0914056900 con 0x7f091c1594f0 2023-07-13T22:28:32.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.009+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f091c0741e0 con 0x7f091c1594f0 2023-07-13T22:28:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.010+0000 7f09127fc700 1 -- 172.21.15.60:0/462746812 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18200 (secure 0 0 0) 0x7f0914059f20 con 0x7f091c1594f0 2023-07-13T22:28:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:32.991 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:06.113196+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:02.360339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:03.609373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f090006ace0 msgr2=0x7f090006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f090006ace0 0x7f090006d190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0918007fd0 tx=0x7f0918007e00 comp rx=0 tx=0).stop 2023-07-13T22:28:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091c1594f0 msgr2=0x7f091c06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091c1594f0 0x7f091c06ba30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f091400b370 tx=0x7f0914004ad0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 shutdown_connections 2023-07-13T22:28:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f090006ace0 0x7f090006d190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091c1594f0 0x7f091c06ba30 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091c0b0ed0 0x7f091c1f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 --2- 172.21.15.60:0/462746812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f091c0c9840 0x7f091c0c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 >> 172.21.15.60:0/462746812 conn(0x7f091c0ce960 msgr2=0x7f091c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 shutdown_connections 2023-07-13T22:28:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.013+0000 7f0923dff700 1 -- 172.21.15.60:0/462746812 wait complete. 2023-07-13T22:28:33.011 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:28:02.360339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:28:03.609373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:28:33.012 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:28:33.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.242+0000 7f5ff05ff700 1 Processor -- start 2023-07-13T22:28:33.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.242+0000 7f5ff05ff700 1 -- start start 2023-07-13T22:28:33.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.243+0000 7f5ff05ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5fe806a640 0x7f5fe80aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.243+0000 7f5ff05ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fe80ae9e0 0x7f5fe80ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.243+0000 7f5ff05ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe81602f0 con 0x7f5fe80cf330 2023-07-13T22:28:33.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.243+0000 7f5ff05ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe80ceb80 con 0x7f5fe80ae9e0 2023-07-13T22:28:33.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5ff05ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe8160590 con 0x7f5fe806a640 2023-07-13T22:28:33.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5feeb9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fe80ae9e0 0x7f5fe80ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5fee39b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5fe80cf330 0x7f5fe8069f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53956/0 (socket says 172.21.15.60:53956) 2023-07-13T22:28:33.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5feeb9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fe80ae9e0 0x7f5fe80ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45836/0 (socket says 172.21.15.60:45836) 2023-07-13T22:28:33.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5fee39b700 1 -- 172.21.15.60:0/3914272225 learned_addr learned my addr 172.21.15.60:0/3914272225 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:33.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5fedb9a700 1 --2- 172.21.15.60:0/3914272225 >> v2:172.21.15.105:3300/0 conn(0x7f5fe806a640 0x7f5fe80aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5feeb9c700 1 -- 172.21.15.60:0/3914272225 >> v2:172.21.15.105:3300/0 conn(0x7f5fe806a640 msgr2=0x7f5fe80aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:33.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5feeb9c700 1 --2- 172.21.15.60:0/3914272225 >> v2:172.21.15.105:3300/0 conn(0x7f5fe806a640 0x7f5fe80aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5feeb9c700 1 -- 172.21.15.60:0/3914272225 >> v1:172.21.15.60:6789/0 conn(0x7f5fe80cf330 legacy=0x7f5fe8069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.244+0000 7f5feeb9c700 1 -- 172.21.15.60:0/3914272225 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe8160870 con 0x7f5fe80ae9e0 2023-07-13T22:28:33.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.245+0000 7f5feeb9c700 1 --2- 172.21.15.60:0/3914272225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80ae9e0 0x7f5fe80ce640 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80d09c0 tx=0x7f5fe000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367ce1da27acd3da server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:33.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.245+0000 7f5fed399700 1 -- 172.21.15.60:0/3914272225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000e070 con 0x7f5fe80ae9e0 2023-07-13T22:28:33.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.245+0000 7f5fed399700 1 -- 172.21.15.60:0/3914272225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe000bb20 con 0x7f5fe80ae9e0 2023-07-13T22:28:33.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.245+0000 7f5fed399700 1 -- 172.21.15.60:0/3914272225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0012430 con 0x7f5fe80ae9e0 2023-07-13T22:28:33.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.245+0000 7f5ff05ff700 1 -- 172.21.15.60:0/3914272225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80ae9e0 msgr2=0x7f5fe80ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.245+0000 7f5ff05ff700 1 --2- 172.21.15.60:0/3914272225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80ae9e0 0x7f5fe80ce640 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80d09c0 tx=0x7f5fe000b330 comp rx=0 tx=0).stop 2023-07-13T22:28:33.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.245+0000 7f5ff05ff700 1 -- 172.21.15.60:0/3914272225 shutdown_connections 2023-07-13T22:28:33.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.246+0000 7f5ff05ff700 1 --2- 172.21.15.60:0/3914272225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80ae9e0 0x7f5fe80ce640 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.246+0000 7f5ff05ff700 1 --2- 172.21.15.60:0/3914272225 >> v2:172.21.15.105:3300/0 conn(0x7f5fe806a640 0x7f5fe80aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.246+0000 7f5ff05ff700 1 -- 172.21.15.60:0/3914272225 >> 172.21.15.60:0/3914272225 conn(0x7f5fe806bcd0 msgr2=0x7f5fe80cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:33.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.246+0000 7f5ff05ff700 1 -- 172.21.15.60:0/3914272225 shutdown_connections 2023-07-13T22:28:33.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.246+0000 7f5ff05ff700 1 -- 172.21.15.60:0/3914272225 wait complete. 2023-07-13T22:28:33.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.247+0000 7f5ff05ff700 1 Processor -- start 2023-07-13T22:28:33.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.248+0000 7f5ff05ff700 1 -- start start 2023-07-13T22:28:33.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.248+0000 7f5ff05ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe806a640 0x7f5fe81500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.248+0000 7f5ff05ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fe80ae9e0 0x7f5fe80b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5ff05ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80cf330 0x7f5fe80b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5ff05ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8150610 con 0x7f5fe80cf330 2023-07-13T22:28:33.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5ff05ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81f5360 con 0x7f5fe806a640 2023-07-13T22:28:33.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5ff05ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81f5640 con 0x7f5fe80ae9e0 2023-07-13T22:28:33.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5feeb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80cf330 0x7f5fe80b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5fee39b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe806a640 0x7f5fe81500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5feeb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80cf330 0x7f5fe80b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45850/0 (socket says 172.21.15.60:45850) 2023-07-13T22:28:33.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5feeb9c700 1 -- 172.21.15.60:0/850649983 learned_addr learned my addr 172.21.15.60:0/850649983 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:33.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5fedb9a700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fe80ae9e0 0x7f5fe80b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5feeb9c700 1 -- 172.21.15.60:0/850649983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fe80ae9e0 msgr2=0x7f5fe80b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5feeb9c700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fe80ae9e0 0x7f5fe80b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5feeb9c700 1 -- 172.21.15.60:0/850649983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe806a640 msgr2=0x7f5fe81500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5feeb9c700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe806a640 0x7f5fe81500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.249+0000 7f5feeb9c700 1 -- 172.21.15.60:0/850649983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0009a70 con 0x7f5fe80cf330 2023-07-13T22:28:33.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.250+0000 7f5feeb9c700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80cf330 0x7f5fe80b06f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0014040 tx=0x7f5fe0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:33.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.250+0000 7f5fdeffd700 1 -- 172.21.15.60:0/850649983 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0012430 con 0x7f5fe80cf330 2023-07-13T22:28:33.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.250+0000 7f5fdeffd700 1 -- 172.21.15.60:0/850649983 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe0020070 con 0x7f5fe80cf330 2023-07-13T22:28:33.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.250+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe81f5920 con 0x7f5fe80cf330 2023-07-13T22:28:33.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.250+0000 7f5fdeffd700 1 -- 172.21.15.60:0/850649983 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00098f0 con 0x7f5fe80cf330 2023-07-13T22:28:33.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.250+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe81f1cd0 con 0x7f5fe80cf330 2023-07-13T22:28:33.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.251+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe8066b20 con 0x7f5fe80cf330 2023-07-13T22:28:33.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.252+0000 7f5fdeffd700 1 -- 172.21.15.60:0/850649983 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5fe0004040 con 0x7f5fe80cf330 2023-07-13T22:28:33.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.253+0000 7f5fdeffd700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fcc06ace0 0x7f5fcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.253+0000 7f5fdeffd700 1 -- 172.21.15.60:0/850649983 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5fe0022070 con 0x7f5fe80cf330 2023-07-13T22:28:33.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.253+0000 7f5fee39b700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fcc06ace0 0x7f5fcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.254+0000 7f5fee39b700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fcc06ace0 0x7f5fcc06d190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4005910 tx=0x7f5fe4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:33.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.257+0000 7f5fdeffd700 1 -- 172.21.15.60:0/850649983 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5fe0056130 con 0x7f5fe80cf330 2023-07-13T22:28:33.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.464+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5fe81f6120 con 0x7f5fe80cf330 2023-07-13T22:28:33.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.465+0000 7f5fdeffd700 1 -- 172.21.15.60:0/850649983 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5fe81f6120 con 0x7f5fe80cf330 2023-07-13T22:28:33.445 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:28:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fcc06ace0 msgr2=0x7f5fcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fcc06ace0 0x7f5fcc06d190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4005910 tx=0x7f5fe4009040 comp rx=0 tx=0).stop 2023-07-13T22:28:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80cf330 msgr2=0x7f5fe80b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80cf330 0x7f5fe80b06f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0014040 tx=0x7f5fe0004ad0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 shutdown_connections 2023-07-13T22:28:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5fcc06ace0 0x7f5fcc06d190 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe80cf330 0x7f5fe80b06f0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe806a640 0x7f5fe81500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 --2- 172.21.15.60:0/850649983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fe80ae9e0 0x7f5fe80b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 >> 172.21.15.60:0/850649983 conn(0x7f5fe806bcd0 msgr2=0x7f5fe80ae190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 shutdown_connections 2023-07-13T22:28:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.467+0000 7f5ff05ff700 1 -- 172.21.15.60:0/850649983 wait complete. 2023-07-13T22:28:33.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:28:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.693+0000 7ffb725ff700 1 Processor -- start 2023-07-13T22:28:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.693+0000 7ffb725ff700 1 -- start start 2023-07-13T22:28:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.694+0000 7ffb725ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 0x7ffb6c160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.694+0000 7ffb725ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb6c0cf280 0x7ffb6c0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.694+0000 7ffb725ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c0b50e0 con 0x7ffb6c0b4c80 2023-07-13T22:28:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.694+0000 7ffb725ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c161020 con 0x7ffb6c0cf280 2023-07-13T22:28:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.694+0000 7ffb725ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0cd310 con 0x7ffb6c15c620 2023-07-13T22:28:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb6b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb6c0cf280 0x7ffb6c0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb70b9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffb6c0b4c80 0x7ffb6c06be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53964/0 (socket says 172.21.15.60:53964) 2023-07-13T22:28:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb6b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb6c0cf280 0x7ffb6c0b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45866/0 (socket says 172.21.15.60:45866) 2023-07-13T22:28:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb70b9c700 1 -- 172.21.15.60:0/3628986743 learned_addr learned my addr 172.21.15.60:0/3628986743 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb6bfff700 1 --2- 172.21.15.60:0/3628986743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 0x7ffb6c160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb6b7fe700 1 -- 172.21.15.60:0/3628986743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 msgr2=0x7ffb6c160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb6b7fe700 1 --2- 172.21.15.60:0/3628986743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 0x7ffb6c160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb6b7fe700 1 -- 172.21.15.60:0/3628986743 >> v1:172.21.15.60:6789/0 conn(0x7ffb6c0b4c80 legacy=0x7ffb6c06be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb6b7fe700 1 -- 172.21.15.60:0/3628986743 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c15c9f0 con 0x7ffb6c0cf280 2023-07-13T22:28:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb6affd700 1 -- 172.21.15.60:0/3628986743 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038708396 0 0) 0x7ffb6c0b50e0 con 0x7ffb6c0b4c80 2023-07-13T22:28:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.695+0000 7ffb6b7fe700 1 --2- 172.21.15.60:0/3628986743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 0x7ffb6c0b4740 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009840 tx=0x7ffb60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f67cf33d311e67 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb6affd700 1 -- 172.21.15.60:0/3628986743 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60011830 con 0x7ffb6c0cf280 2023-07-13T22:28:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb6affd700 1 -- 172.21.15.60:0/3628986743 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb60011990 con 0x7ffb6c0cf280 2023-07-13T22:28:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb6affd700 1 -- 172.21.15.60:0/3628986743 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60011d20 con 0x7ffb6c0cf280 2023-07-13T22:28:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb725ff700 1 -- 172.21.15.60:0/3628986743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 msgr2=0x7ffb6c0b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb725ff700 1 --2- 172.21.15.60:0/3628986743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 0x7ffb6c0b4740 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009840 tx=0x7ffb60009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb725ff700 1 -- 172.21.15.60:0/3628986743 shutdown_connections 2023-07-13T22:28:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb725ff700 1 --2- 172.21.15.60:0/3628986743 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 0x7ffb6c160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb725ff700 1 --2- 172.21.15.60:0/3628986743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 0x7ffb6c0b4740 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb725ff700 1 -- 172.21.15.60:0/3628986743 >> 172.21.15.60:0/3628986743 conn(0x7ffb6c06b210 msgr2=0x7ffb6c0cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb725ff700 1 -- 172.21.15.60:0/3628986743 shutdown_connections 2023-07-13T22:28:33.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.696+0000 7ffb725ff700 1 -- 172.21.15.60:0/3628986743 wait complete. 2023-07-13T22:28:33.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.698+0000 7ffb725ff700 1 Processor -- start 2023-07-13T22:28:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.698+0000 7ffb725ff700 1 -- start start 2023-07-13T22:28:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.699+0000 7ffb725ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb6c0b4c80 0x7ffb6c0b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.699+0000 7ffb725ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 0x7ffb6c0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.699+0000 7ffb725ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 0x7ffb6c0b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.699+0000 7ffb725ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0b76f0 con 0x7ffb6c0cf280 2023-07-13T22:28:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.699+0000 7ffb725ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0cc220 con 0x7ffb6c0b4c80 2023-07-13T22:28:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.699+0000 7ffb725ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0b6b70 con 0x7ffb6c15c620 2023-07-13T22:28:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.699+0000 7ffb6b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 0x7ffb6c0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.699+0000 7ffb6b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 0x7ffb6c0cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45868/0 (socket says 172.21.15.60:45868) 2023-07-13T22:28:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.699+0000 7ffb6b7fe700 1 -- 172.21.15.60:0/2498807578 learned_addr learned my addr 172.21.15.60:0/2498807578 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb6bfff700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb6c0b4c80 0x7ffb6c0b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb70b9c700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 0x7ffb6c0b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb6b7fe700 1 -- 172.21.15.60:0/2498807578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 msgr2=0x7ffb6c0b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb6b7fe700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 0x7ffb6c0b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb6b7fe700 1 -- 172.21.15.60:0/2498807578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb6c0b4c80 msgr2=0x7ffb6c0b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb6b7fe700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb6c0b4c80 0x7ffb6c0b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb6b7fe700 1 -- 172.21.15.60:0/2498807578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600094a0 con 0x7ffb6c0cf280 2023-07-13T22:28:33.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb70b9c700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 0x7ffb6c0b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:33.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb6b7fe700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 0x7ffb6c0cbce0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ffb600090b0 tx=0x7ffb60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:33.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb68ff9700 1 -- 172.21.15.60:0/2498807578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600036a0 con 0x7ffb6c0cf280 2023-07-13T22:28:33.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb68ff9700 1 -- 172.21.15.60:0/2498807578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6001f070 con 0x7ffb6c0cf280 2023-07-13T22:28:33.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb68ff9700 1 -- 172.21.15.60:0/2498807578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60003a00 con 0x7ffb6c0cf280 2023-07-13T22:28:33.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.700+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c0b6e50 con 0x7ffb6c0cf280 2023-07-13T22:28:33.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.701+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c0b7410 con 0x7ffb6c0cf280 2023-07-13T22:28:33.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.702+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c0b2e30 con 0x7ffb6c0cf280 2023-07-13T22:28:33.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.703+0000 7ffb68ff9700 1 -- 172.21.15.60:0/2498807578 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb60003b60 con 0x7ffb6c0cf280 2023-07-13T22:28:33.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.704+0000 7ffb68ff9700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb5806ace0 0x7ffb5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:33.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.704+0000 7ffb68ff9700 1 -- 172.21.15.60:0/2498807578 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb60021070 con 0x7ffb6c0cf280 2023-07-13T22:28:33.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.704+0000 7ffb6bfff700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb5806ace0 0x7ffb5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:33.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.705+0000 7ffb6bfff700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb5806ace0 0x7ffb5806d190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c006b20 tx=0x7ffb5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:33.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.709+0000 7ffb68ff9700 1 -- 172.21.15.60:0/2498807578 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffb60055f80 con 0x7ffb6c0cf280 2023-07-13T22:28:33.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.914+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb6c0741e0 con 0x7ffb6c0cf280 2023-07-13T22:28:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.915+0000 7ffb68ff9700 1 -- 172.21.15.60:0/2498807578 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb6005de60 con 0x7ffb6c0cf280 2023-07-13T22:28:33.895 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:28:33.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb5806ace0 msgr2=0x7ffb5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb5806ace0 0x7ffb5806d190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c006b20 tx=0x7ffb5c008040 comp rx=0 tx=0).stop 2023-07-13T22:28:33.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 msgr2=0x7ffb6c0cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 0x7ffb6c0cbce0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ffb600090b0 tx=0x7ffb60004750 comp rx=0 tx=0).stop 2023-07-13T22:28:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 shutdown_connections 2023-07-13T22:28:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffb5806ace0 0x7ffb5806d190 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb6c15c620 0x7ffb6c0b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb6c0b4c80 0x7ffb6c0b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 --2- 172.21.15.60:0/2498807578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb6c0cf280 0x7ffb6c0cbce0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 >> 172.21.15.60:0/2498807578 conn(0x7ffb6c06b210 msgr2=0x7ffb6c0b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 shutdown_connections 2023-07-13T22:28:33.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:33.917+0000 7ffb725ff700 1 -- 172.21.15.60:0/2498807578 wait complete. 2023-07-13T22:28:33.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:28:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.141+0000 7f7afd3ff700 1 Processor -- start 2023-07-13T22:28:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.142+0000 7f7afd3ff700 1 -- start start 2023-07-13T22:28:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7afd3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7af815c620 0x7f7af8160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7afd3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80cf280 0x7f7af80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7afd3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7af80b4c80 0x7f7af806be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7afd3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af80b50e0 con 0x7f7af80b4c80 2023-07-13T22:28:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7afd3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af8161020 con 0x7f7af815c620 2023-07-13T22:28:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7afd3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80cd310 con 0x7f7af80cf280 2023-07-13T22:28:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7af77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7af80b4c80 0x7f7af806be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7af77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7af80b4c80 0x7f7af806be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45876/0 (socket says 172.21.15.60:45876) 2023-07-13T22:28:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7af77fe700 1 -- 172.21.15.60:0/586317340 learned_addr learned my addr 172.21.15.60:0/586317340 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7aeffff700 1 --2- 172.21.15.60:0/586317340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80cf280 0x7f7af80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.143+0000 7f7af6ffd700 1 --2- 172.21.15.60:0/586317340 >> v2:172.21.15.105:3300/0 conn(0x7f7af815c620 0x7f7af8160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af77fe700 1 -- 172.21.15.60:0/586317340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80cf280 msgr2=0x7f7af80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af77fe700 1 --2- 172.21.15.60:0/586317340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80cf280 0x7f7af80b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af77fe700 1 -- 172.21.15.60:0/586317340 >> v2:172.21.15.105:3300/0 conn(0x7f7af815c620 msgr2=0x7f7af8160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af77fe700 1 --2- 172.21.15.60:0/586317340 >> v2:172.21.15.105:3300/0 conn(0x7f7af815c620 0x7f7af8160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af77fe700 1 -- 172.21.15.60:0/586317340 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af815c9f0 con 0x7f7af80b4c80 2023-07-13T22:28:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7aeffff700 1 --2- 172.21.15.60:0/586317340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80cf280 0x7f7af80b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af6ffd700 1 --2- 172.21.15.60:0/586317340 >> v2:172.21.15.105:3300/0 conn(0x7f7af815c620 0x7f7af8160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af77fe700 1 --2- 172.21.15.60:0/586317340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80b4c80 0x7f7af806be90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7af8072640 tx=0x7f7af000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3594b57d15dc2764 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af67fc700 1 -- 172.21.15.60:0/586317340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af000e070 con 0x7f7af80b4c80 2023-07-13T22:28:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af67fc700 1 -- 172.21.15.60:0/586317340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af000b9c0 con 0x7f7af80b4c80 2023-07-13T22:28:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.144+0000 7f7af67fc700 1 -- 172.21.15.60:0/586317340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af000bd70 con 0x7f7af80b4c80 2023-07-13T22:28:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.145+0000 7f7afd3ff700 1 -- 172.21.15.60:0/586317340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80b4c80 msgr2=0x7f7af806be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.145+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/586317340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80b4c80 0x7f7af806be90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7af8072640 tx=0x7f7af000b330 comp rx=0 tx=0).stop 2023-07-13T22:28:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.145+0000 7f7afd3ff700 1 -- 172.21.15.60:0/586317340 shutdown_connections 2023-07-13T22:28:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.145+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/586317340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80b4c80 0x7f7af806be90 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.145+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/586317340 >> v2:172.21.15.105:3300/0 conn(0x7f7af815c620 0x7f7af8160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.145+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/586317340 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80cf280 0x7f7af80b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.145+0000 7f7afd3ff700 1 -- 172.21.15.60:0/586317340 >> 172.21.15.60:0/586317340 conn(0x7f7af806b210 msgr2=0x7f7af80cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.145+0000 7f7afd3ff700 1 -- 172.21.15.60:0/586317340 shutdown_connections 2023-07-13T22:28:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.145+0000 7f7afd3ff700 1 -- 172.21.15.60:0/586317340 wait complete. 2023-07-13T22:28:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.147+0000 7f7afd3ff700 1 Processor -- start 2023-07-13T22:28:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.147+0000 7f7afd3ff700 1 -- start start 2023-07-13T22:28:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7afd3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80b4c80 0x7f7af815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7afd3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80cf280 0x7f7af80ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7afd3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7af815c620 0x7f7af815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7afd3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b6e80 con 0x7f7af80cf280 2023-07-13T22:28:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7afd3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b7160 con 0x7f7af815c620 2023-07-13T22:28:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7afd3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b7440 con 0x7f7af80b4c80 2023-07-13T22:28:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80cf280 0x7f7af80ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80cf280 0x7f7af80ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45890/0 (socket says 172.21.15.60:45890) 2023-07-13T22:28:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7aeffff700 1 -- 172.21.15.60:0/1811680611 learned_addr learned my addr 172.21.15.60:0/1811680611 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7aeffff700 1 -- 172.21.15.60:0/1811680611 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80b4c80 msgr2=0x7f7af815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7aeffff700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80b4c80 0x7f7af815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7aeffff700 1 -- 172.21.15.60:0/1811680611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7af815c620 msgr2=0x7f7af815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7aeffff700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7af815c620 0x7f7af815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.148+0000 7f7aeffff700 1 -- 172.21.15.60:0/1811680611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af0009b30 con 0x7f7af80cf280 2023-07-13T22:28:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.149+0000 7f7aeffff700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80cf280 0x7f7af80ca7a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e810 tx=0x7f7ae000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.149+0000 7f7aef7fe700 1 -- 172.21.15.60:0/1811680611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0004230 con 0x7f7af80cf280 2023-07-13T22:28:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.149+0000 7f7aef7fe700 1 -- 172.21.15.60:0/1811680611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae0004390 con 0x7f7af80cf280 2023-07-13T22:28:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.149+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af80b7720 con 0x7f7af80cf280 2023-07-13T22:28:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.149+0000 7f7aef7fe700 1 -- 172.21.15.60:0/1811680611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae00036a0 con 0x7f7af80cf280 2023-07-13T22:28:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.149+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af815d680 con 0x7f7af80cf280 2023-07-13T22:28:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.150+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af80741e0 con 0x7f7af80cf280 2023-07-13T22:28:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.151+0000 7f7aef7fe700 1 -- 172.21.15.60:0/1811680611 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ae0003800 con 0x7f7af80cf280 2023-07-13T22:28:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.152+0000 7f7aef7fe700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ad806ace0 0x7f7ad806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.152+0000 7f7aef7fe700 1 -- 172.21.15.60:0/1811680611 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ae0088ff0 con 0x7f7af80cf280 2023-07-13T22:28:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.152+0000 7f7af6ffd700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ad806ace0 0x7f7ad806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.153+0000 7f7af6ffd700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ad806ace0 0x7f7ad806d190 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8006fd0 tx=0x7f7ae8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.156+0000 7f7aef7fe700 1 -- 172.21.15.60:0/1811680611 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7ae0054040 con 0x7f7af80cf280 2023-07-13T22:28:34.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.363+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7af80d00f0 con 0x7f7af80cf280 2023-07-13T22:28:34.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.364+0000 7f7aef7fe700 1 -- 172.21.15.60:0/1811680611 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ae0057660 con 0x7f7af80cf280 2023-07-13T22:28:34.344 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:28:34.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ad806ace0 msgr2=0x7f7ad806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ad806ace0 0x7f7ad806d190 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8006fd0 tx=0x7f7ae8008040 comp rx=0 tx=0).stop 2023-07-13T22:28:34.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80cf280 msgr2=0x7f7af80ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80cf280 0x7f7af80ca7a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e810 tx=0x7f7ae000ebd0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 shutdown_connections 2023-07-13T22:28:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7ad806ace0 0x7f7ad806d190 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7af815c620 0x7f7af815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7af80b4c80 0x7f7af815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 --2- 172.21.15.60:0/1811680611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af80cf280 0x7f7af80ca7a0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 >> 172.21.15.60:0/1811680611 conn(0x7f7af806b210 msgr2=0x7f7af815b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 shutdown_connections 2023-07-13T22:28:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.366+0000 7f7afd3ff700 1 -- 172.21.15.60:0/1811680611 wait complete. 2023-07-13T22:28:34.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:28:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.590+0000 7fb0c37ff700 1 Processor -- start 2023-07-13T22:28:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.590+0000 7fb0c37ff700 1 -- start start 2023-07-13T22:28:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.591+0000 7fb0c37ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0bc0cf860 0x7fb0bc06b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.591+0000 7fb0c37ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc0d0be0 con 0x7fb0bc06b3b0 2023-07-13T22:28:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.591+0000 7fb0c37ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc0cb9b0 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.591+0000 7fb0c37ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc0cfd00 con 0x7fb0bc0cf860 2023-07-13T22:28:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.591+0000 7fb0c0d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0bc0cf860 0x7fb0bc06b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.592+0000 7fb0c0d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0bc0cf860 0x7fb0bc06b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45904/0 (socket says 172.21.15.60:45904) 2023-07-13T22:28:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.592+0000 7fb0c0d9a700 1 -- 172.21.15.60:0/3503901650 learned_addr learned my addr 172.21.15.60:0/3503901650 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.592+0000 7fb0c159b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb0bc06b3b0 0x7fb0bc0cb2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53980/0 (socket says 172.21.15.60:53980) 2023-07-13T22:28:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.592+0000 7fb0c0d9a700 1 -- 172.21.15.60:0/3503901650 >> v1:172.21.15.105:6789/0 conn(0x7fb0bc06bcc0 legacy=0x7fb0bc06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.592+0000 7fb0c0d9a700 1 -- 172.21.15.60:0/3503901650 >> v1:172.21.15.60:6789/0 conn(0x7fb0bc06b3b0 legacy=0x7fb0bc0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.592+0000 7fb0c0d9a700 1 -- 172.21.15.60:0/3503901650 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0cffd0 con 0x7fb0bc0cf860 2023-07-13T22:28:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.592+0000 7fb0b3fff700 1 -- 172.21.15.60:0/3503901650 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132596857 0 0) 0x7fb0bc0d0be0 con 0x7fb0bc06b3b0 2023-07-13T22:28:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.592+0000 7fb0c0d9a700 1 --2- 172.21.15.60:0/3503901650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc0cf860 0x7fb0bc06b780 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0098f0 tx=0x7fb0ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2a40a1174d2a08 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0b3fff700 1 -- 172.21.15.60:0/3503901650 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00b8a0 con 0x7fb0bc0cf860 2023-07-13T22:28:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0b3fff700 1 -- 172.21.15.60:0/3503901650 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac00ba00 con 0x7fb0bc0cf860 2023-07-13T22:28:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0b3fff700 1 -- 172.21.15.60:0/3503901650 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00bd90 con 0x7fb0bc0cf860 2023-07-13T22:28:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0c37ff700 1 -- 172.21.15.60:0/3503901650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc0cf860 msgr2=0x7fb0bc06b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0c37ff700 1 --2- 172.21.15.60:0/3503901650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc0cf860 0x7fb0bc06b780 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0098f0 tx=0x7fb0ac009c00 comp rx=0 tx=0).stop 2023-07-13T22:28:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0c37ff700 1 -- 172.21.15.60:0/3503901650 shutdown_connections 2023-07-13T22:28:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0c37ff700 1 --2- 172.21.15.60:0/3503901650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc0cf860 0x7fb0bc06b780 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0c37ff700 1 -- 172.21.15.60:0/3503901650 >> 172.21.15.60:0/3503901650 conn(0x7fb0bc06dd00 msgr2=0x7fb0bc06e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0c37ff700 1 -- 172.21.15.60:0/3503901650 shutdown_connections 2023-07-13T22:28:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.593+0000 7fb0c37ff700 1 -- 172.21.15.60:0/3503901650 wait complete. 2023-07-13T22:28:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.595+0000 7fb0c37ff700 1 Processor -- start 2023-07-13T22:28:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.595+0000 7fb0c37ff700 1 -- start start 2023-07-13T22:28:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.596+0000 7fb0c37ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0bc06b3b0 0x7fb0bc0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.596+0000 7fb0c37ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc06bcc0 0x7fb0bc0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.596+0000 7fb0c37ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0bc0cf860 0x7fb0bc159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.596+0000 7fb0c37ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0c7100 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.596+0000 7fb0c37ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0af2c0 con 0x7fb0bc0cf860 2023-07-13T22:28:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.596+0000 7fb0c37ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0af5a0 con 0x7fb0bc06b3b0 2023-07-13T22:28:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.596+0000 7fb0c0d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc06bcc0 0x7fb0bc0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.596+0000 7fb0c0d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc06bcc0 0x7fb0bc0c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45914/0 (socket says 172.21.15.60:45914) 2023-07-13T22:28:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.596+0000 7fb0c0d9a700 1 -- 172.21.15.60:0/1978138637 learned_addr learned my addr 172.21.15.60:0/1978138637 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0c159b700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0bc06b3b0 0x7fb0bc0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0c1d9c700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0bc0cf860 0x7fb0bc159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0c0d9a700 1 -- 172.21.15.60:0/1978138637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0bc06b3b0 msgr2=0x7fb0bc0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0c0d9a700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0bc06b3b0 0x7fb0bc0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0c0d9a700 1 -- 172.21.15.60:0/1978138637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0bc0cf860 msgr2=0x7fb0bc159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0c0d9a700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0bc0cf860 0x7fb0bc159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0c0d9a700 1 -- 172.21.15.60:0/1978138637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac009550 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0c0d9a700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc06bcc0 0x7fb0bc0c49d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac014040 tx=0x7fb0ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0b1ffb700 1 -- 172.21.15.60:0/1978138637 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac011a70 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0b1ffb700 1 -- 172.21.15.60:0/1978138637 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac01e070 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.597+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc0ad260 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.598+0000 7fb0b1ffb700 1 -- 172.21.15.60:0/1978138637 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac011dd0 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.598+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc0ad820 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.599+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc0b6a30 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.599+0000 7fb0b1ffb700 1 -- 172.21.15.60:0/1978138637 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0ac02d450 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.600+0000 7fb0b1ffb700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb0a006ace0 0x7fb0a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:34.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.600+0000 7fb0b1ffb700 1 -- 172.21.15.60:0/1978138637 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0ac08a620 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.601+0000 7fb0c159b700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb0a006ace0 0x7fb0a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.601+0000 7fb0c159b700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb0a006ace0 0x7fb0a006d190 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8007fd0 tx=0x7fb0b8007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:34.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.605+0000 7fb0b1ffb700 1 -- 172.21.15.60:0/1978138637 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb0ac0556c0 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.813+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0bc0ada10 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.814+0000 7fb0b1ffb700 1 -- 172.21.15.60:0/1978138637 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0ac058ce0 con 0x7fb0bc06bcc0 2023-07-13T22:28:34.794 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:28:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb0a006ace0 msgr2=0x7fb0a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb0a006ace0 0x7fb0a006d190 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8007fd0 tx=0x7fb0b8007c20 comp rx=0 tx=0).stop 2023-07-13T22:28:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc06bcc0 msgr2=0x7fb0bc0c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc06bcc0 0x7fb0bc0c49d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac014040 tx=0x7fb0ac004750 comp rx=0 tx=0).stop 2023-07-13T22:28:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 shutdown_connections 2023-07-13T22:28:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb0a006ace0 0x7fb0a006d190 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0bc0cf860 0x7fb0bc159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb0bc06b3b0 0x7fb0bc0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 --2- 172.21.15.60:0/1978138637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0bc06bcc0 0x7fb0bc0c49d0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 >> 172.21.15.60:0/1978138637 conn(0x7fb0bc06dd00 msgr2=0x7fb0bc0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:34.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 shutdown_connections 2023-07-13T22:28:34.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:34.816+0000 7fb0c37ff700 1 -- 172.21.15.60:0/1978138637 wait complete. 2023-07-13T22:28:34.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:28:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.037+0000 7f34a1dff700 1 Processor -- start 2023-07-13T22:28:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.037+0000 7f34a1dff700 1 -- start start 2023-07-13T22:28:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.038+0000 7f34a1dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f349c071110 0x7f349c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.038+0000 7f34a1dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c06c140 0x7f349c15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.038+0000 7f34a1dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349c0ce560 con 0x7f349c0ad900 2023-07-13T22:28:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.038+0000 7f34a1dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c0c8fc0 con 0x7f349c071110 2023-07-13T22:28:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.038+0000 7f34a1dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0c92a0 con 0x7f349c06c140 2023-07-13T22:28:35.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.038+0000 7f349b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f349c071110 0x7f349c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.038+0000 7f349b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f349c071110 0x7f349c0caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39526/0 (socket says 172.21.15.60:39526) 2023-07-13T22:28:35.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.038+0000 7f349b7fe700 1 -- 172.21.15.60:0/1767770505 learned_addr learned my addr 172.21.15.60:0/1767770505 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:35.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.039+0000 7f349bfff700 1 --2- 172.21.15.60:0/1767770505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c06c140 0x7f349c15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.039+0000 7f349b7fe700 1 -- 172.21.15.60:0/1767770505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c06c140 msgr2=0x7f349c15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:35.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.039+0000 7f349b7fe700 1 --2- 172.21.15.60:0/1767770505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c06c140 0x7f349c15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.039+0000 7f349b7fe700 1 -- 172.21.15.60:0/1767770505 >> v1:172.21.15.105:6789/0 conn(0x7f349c0ad900 legacy=0x7f349c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.039+0000 7f349b7fe700 1 -- 172.21.15.60:0/1767770505 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c0ce1f0 con 0x7f349c071110 2023-07-13T22:28:35.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.039+0000 7f349b7fe700 1 --2- 172.21.15.60:0/1767770505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c071110 0x7f349c0caa80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f348c0098f0 tx=0x7f348c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb7a3074385beb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:35.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.039+0000 7f349a7fc700 1 -- 172.21.15.60:0/1767770505 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c00b8a0 con 0x7f349c071110 2023-07-13T22:28:35.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.039+0000 7f349a7fc700 1 -- 172.21.15.60:0/1767770505 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c00ba00 con 0x7f349c071110 2023-07-13T22:28:35.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.039+0000 7f349a7fc700 1 -- 172.21.15.60:0/1767770505 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c00bd90 con 0x7f349c071110 2023-07-13T22:28:35.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.040+0000 7f34a1dff700 1 -- 172.21.15.60:0/1767770505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c071110 msgr2=0x7f349c0caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.040+0000 7f34a1dff700 1 --2- 172.21.15.60:0/1767770505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c071110 0x7f349c0caa80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f348c0098f0 tx=0x7f348c009c00 comp rx=0 tx=0).stop 2023-07-13T22:28:35.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.040+0000 7f34a1dff700 1 -- 172.21.15.60:0/1767770505 shutdown_connections 2023-07-13T22:28:35.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.040+0000 7f34a1dff700 1 --2- 172.21.15.60:0/1767770505 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c06c140 0x7f349c15c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.040+0000 7f34a1dff700 1 --2- 172.21.15.60:0/1767770505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c071110 0x7f349c0caa80 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.040+0000 7f34a1dff700 1 -- 172.21.15.60:0/1767770505 >> 172.21.15.60:0/1767770505 conn(0x7f349c0ce960 msgr2=0x7f349c0ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:35.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.040+0000 7f34a1dff700 1 -- 172.21.15.60:0/1767770505 shutdown_connections 2023-07-13T22:28:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.040+0000 7f34a1dff700 1 -- 172.21.15.60:0/1767770505 wait complete. 2023-07-13T22:28:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.042+0000 7f34a1dff700 1 Processor -- start 2023-07-13T22:28:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.042+0000 7f34a1dff700 1 -- start start 2023-07-13T22:28:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f34a1dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c06c140 0x7f349c151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f34a1dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f349c071110 0x7f349c1f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f34a1dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c1f3020 0x7f349c0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f34a1dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c1f2300 con 0x7f349c06c140 2023-07-13T22:28:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f34a1dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c14e1a0 con 0x7f349c071110 2023-07-13T22:28:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f34a1dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c14e480 con 0x7f349c1f3020 2023-07-13T22:28:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c06c140 0x7f349c151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c06c140 0x7f349c151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39530/0 (socket says 172.21.15.60:39530) 2023-07-13T22:28:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349b7fe700 1 -- 172.21.15.60:0/1566081584 learned_addr learned my addr 172.21.15.60:0/1566081584 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349affd700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f349c071110 0x7f349c1f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349bfff700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c1f3020 0x7f349c0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349b7fe700 1 -- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c1f3020 msgr2=0x7f349c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349b7fe700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c1f3020 0x7f349c0c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349b7fe700 1 -- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f349c071110 msgr2=0x7f349c1f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349b7fe700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f349c071110 0x7f349c1f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349b7fe700 1 -- 172.21.15.60:0/1566081584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c009550 con 0x7f349c06c140 2023-07-13T22:28:35.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349bfff700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c1f3020 0x7f349c0c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:35.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.043+0000 7f349affd700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f349c071110 0x7f349c1f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:35.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.044+0000 7f349b7fe700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c06c140 0x7f349c151860 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f348c000c00 tx=0x7f348c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:35.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.044+0000 7f3483fff700 1 -- 172.21.15.60:0/1566081584 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c004480 con 0x7f349c06c140 2023-07-13T22:28:35.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.044+0000 7f3483fff700 1 -- 172.21.15.60:0/1566081584 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c01f070 con 0x7f349c06c140 2023-07-13T22:28:35.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.044+0000 7f3483fff700 1 -- 172.21.15.60:0/1566081584 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c0038c0 con 0x7f349c06c140 2023-07-13T22:28:35.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.044+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349c14e760 con 0x7f349c06c140 2023-07-13T22:28:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.044+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349c151da0 con 0x7f349c06c140 2023-07-13T22:28:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.045+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349c066b20 con 0x7f349c06c140 2023-07-13T22:28:35.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.046+0000 7f3483fff700 1 -- 172.21.15.60:0/1566081584 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f348c021030 con 0x7f349c06c140 2023-07-13T22:28:35.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.047+0000 7f3483fff700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f348806ace0 0x7f348806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.047+0000 7f3483fff700 1 -- 172.21.15.60:0/1566081584 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f348c08aa40 con 0x7f349c06c140 2023-07-13T22:28:35.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.047+0000 7f349affd700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f348806ace0 0x7f348806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.048+0000 7f349affd700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f348806ace0 0x7f348806d190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f349c0cbaf0 tx=0x7f3490009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:35.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.051+0000 7f3483fff700 1 -- 172.21.15.60:0/1566081584 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f348c055c00 con 0x7f349c06c140 2023-07-13T22:28:35.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.273+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f349c0741e0 con 0x7f349c06c140 2023-07-13T22:28:35.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.273+0000 7f3483fff700 1 -- 172.21.15.60:0/1566081584 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) v1 ==== 80+0+0 (secure 0 0 0) 0x7f348c059220 con 0x7f349c06c140 2023-07-13T22:28:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.275+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f348806ace0 msgr2=0x7f348806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.275+0000 7f34a1dff700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f348806ace0 0x7f348806d190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f349c0cbaf0 tx=0x7f3490009040 comp rx=0 tx=0).stop 2023-07-13T22:28:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.275+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c06c140 msgr2=0x7f349c151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.275+0000 7f34a1dff700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c06c140 0x7f349c151860 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f348c000c00 tx=0x7f348c004750 comp rx=0 tx=0).stop 2023-07-13T22:28:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.276+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 shutdown_connections 2023-07-13T22:28:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.276+0000 7f34a1dff700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f348806ace0 0x7f348806d190 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.276+0000 7f34a1dff700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f349c06c140 0x7f349c151860 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.276+0000 7f34a1dff700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f349c1f3020 0x7f349c0c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.276+0000 7f34a1dff700 1 --2- 172.21.15.60:0/1566081584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f349c071110 0x7f349c1f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.276+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 >> 172.21.15.60:0/1566081584 conn(0x7f349c0ce960 msgr2=0x7f349c153280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.276+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 shutdown_connections 2023-07-13T22:28:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.276+0000 7f34a1dff700 1 -- 172.21.15.60:0/1566081584 wait complete. 2023-07-13T22:28:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:28:35.273 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:28:35.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:35.274 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:28:35.341 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[148646]: fuse finished with error 0 and tester_r 0 2023-07-13T22:28:35.353 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:28:35.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:35.354 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:28:35.517 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:28:35.518 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:35.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:35.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.792+0000 7ff2411ff700 1 Processor -- start 2023-07-13T22:28:35.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.792+0000 7ff2411ff700 1 -- start start 2023-07-13T22:28:35.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff2411ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff23c0d0b80 0x7ff23c0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff2411ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0f50 0x7ff23c0d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff2411ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c0ceeb0 con 0x7ff23c0cf8e0 2023-07-13T22:28:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff2411ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c0b23f0 con 0x7ff23c0d0b80 2023-07-13T22:28:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff2411ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0b4bb0 con 0x7ff23c0d0f50 2023-07-13T22:28:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff23b59e700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff23c0cf8e0 0x7ff23c0cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60352/0 (socket says 172.21.15.60:60352) 2023-07-13T22:28:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff23b59e700 1 -- 172.21.15.60:0/4293037782 learned_addr learned my addr 172.21.15.60:0/4293037782 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:35.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff23ad9d700 1 --2- 172.21.15.60:0/4293037782 >> v2:172.21.15.60:3300/0 conn(0x7ff23c0d0b80 0x7ff23c0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff233fff700 1 --2- 172.21.15.60:0/4293037782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0f50 0x7ff23c0d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff23a59c700 1 -- 172.21.15.60:0/4293037782 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706830297 0 0) 0x7ff23c0ceeb0 con 0x7ff23c0cf8e0 2023-07-13T22:28:35.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff23a59c700 1 -- 172.21.15.60:0/4293037782 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2280035b0 con 0x7ff23c0cf8e0 2023-07-13T22:28:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff23ad9d700 1 -- 172.21.15.60:0/4293037782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0f50 msgr2=0x7ff23c0d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff23ad9d700 1 --2- 172.21.15.60:0/4293037782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0f50 0x7ff23c0d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff23ad9d700 1 -- 172.21.15.60:0/4293037782 >> v1:172.21.15.60:6789/0 conn(0x7ff23c0cf8e0 legacy=0x7ff23c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.793+0000 7ff23ad9d700 1 -- 172.21.15.60:0/4293037782 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0b5380 con 0x7ff23c0d0b80 2023-07-13T22:28:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.794+0000 7ff23ad9d700 1 --2- 172.21.15.60:0/4293037782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0b80 0x7ff23c0b5ad0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009840 tx=0x7ff22c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bcfd05601ebf37 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.794+0000 7ff23a59c700 1 -- 172.21.15.60:0/4293037782 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c011830 con 0x7ff23c0d0b80 2023-07-13T22:28:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.794+0000 7ff23a59c700 1 -- 172.21.15.60:0/4293037782 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c011990 con 0x7ff23c0d0b80 2023-07-13T22:28:35.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.794+0000 7ff23a59c700 1 -- 172.21.15.60:0/4293037782 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c011d20 con 0x7ff23c0d0b80 2023-07-13T22:28:35.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.794+0000 7ff2411ff700 1 -- 172.21.15.60:0/4293037782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0b80 msgr2=0x7ff23c0b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.794+0000 7ff2411ff700 1 --2- 172.21.15.60:0/4293037782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0b80 0x7ff23c0b5ad0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009840 tx=0x7ff22c009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:35.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.795+0000 7ff2411ff700 1 -- 172.21.15.60:0/4293037782 shutdown_connections 2023-07-13T22:28:35.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.795+0000 7ff2411ff700 1 --2- 172.21.15.60:0/4293037782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0b80 0x7ff23c0b5ad0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.795+0000 7ff2411ff700 1 --2- 172.21.15.60:0/4293037782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0f50 0x7ff23c0d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.795+0000 7ff2411ff700 1 -- 172.21.15.60:0/4293037782 >> 172.21.15.60:0/4293037782 conn(0x7ff23c071660 msgr2=0x7ff23c071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.795+0000 7ff2411ff700 1 -- 172.21.15.60:0/4293037782 shutdown_connections 2023-07-13T22:28:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.795+0000 7ff2411ff700 1 -- 172.21.15.60:0/4293037782 wait complete. 2023-07-13T22:28:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.796+0000 7ff2411ff700 1 Processor -- start 2023-07-13T22:28:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.796+0000 7ff2411ff700 1 -- start start 2023-07-13T22:28:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.797+0000 7ff2411ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0cf8e0 0x7ff23c144a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.797+0000 7ff2411ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0b80 0x7ff23c0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.797+0000 7ff2411ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0f50 0x7ff23c0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.797+0000 7ff2411ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0cd1a0 con 0x7ff23c0d0f50 2023-07-13T22:28:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.797+0000 7ff2411ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c15a720 con 0x7ff23c0cf8e0 2023-07-13T22:28:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.797+0000 7ff2411ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0ccec0 con 0x7ff23c0d0b80 2023-07-13T22:28:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23b59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0f50 0x7ff23c0ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23b59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0f50 0x7ff23c0ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39548/0 (socket says 172.21.15.60:39548) 2023-07-13T22:28:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23b59e700 1 -- 172.21.15.60:0/1780262922 learned_addr learned my addr 172.21.15.60:0/1780262922 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23ad9d700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0cf8e0 0x7ff23c144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff233fff700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0b80 0x7ff23c0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23b59e700 1 -- 172.21.15.60:0/1780262922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0b80 msgr2=0x7ff23c0c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23b59e700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0b80 0x7ff23c0c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23b59e700 1 -- 172.21.15.60:0/1780262922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0cf8e0 msgr2=0x7ff23c144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23b59e700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0cf8e0 0x7ff23c144a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23b59e700 1 -- 172.21.15.60:0/1780262922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c0094a0 con 0x7ff23c0d0f50 2023-07-13T22:28:35.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.798+0000 7ff23b59e700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0f50 0x7ff23c0ca200 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff23400cb40 tx=0x7ff23400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:35.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.799+0000 7ff2337fe700 1 -- 172.21.15.60:0/1780262922 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23400a9e0 con 0x7ff23c0d0f50 2023-07-13T22:28:35.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.799+0000 7ff2337fe700 1 -- 172.21.15.60:0/1780262922 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23400ab40 con 0x7ff23c0d0f50 2023-07-13T22:28:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.799+0000 7ff2337fe700 1 -- 172.21.15.60:0/1780262922 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234012400 con 0x7ff23c0d0f50 2023-07-13T22:28:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.799+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c15a9c0 con 0x7ff23c0d0f50 2023-07-13T22:28:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.799+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c145180 con 0x7ff23c0d0f50 2023-07-13T22:28:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.800+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c15ad10 con 0x7ff23c0d0f50 2023-07-13T22:28:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.801+0000 7ff2337fe700 1 -- 172.21.15.60:0/1780262922 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff23401b3b0 con 0x7ff23c0d0f50 2023-07-13T22:28:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.803+0000 7ff2337fe700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff21c06ace0 0x7ff21c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.803+0000 7ff2337fe700 1 -- 172.21.15.60:0/1780262922 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff234088f60 con 0x7ff23c0d0f50 2023-07-13T22:28:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.803+0000 7ff23ad9d700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff21c06ace0 0x7ff21c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:35.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.803+0000 7ff23ad9d700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff21c06ace0 0x7ff21c06d190 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff22c000c00 tx=0x7ff22c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:35.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:35.807+0000 7ff2337fe700 1 -- 172.21.15.60:0/1780262922 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff2340540b0 con 0x7ff23c0d0f50 2023-07-13T22:28:35.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.013+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff23c15aee0 con 0x7ff23c0d0f50 2023-07-13T22:28:35.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.014+0000 7ff2337fe700 1 -- 172.21.15.60:0/1780262922 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18200 (secure 0 0 0) 0x7ff2340576d0 con 0x7ff23c0d0f50 2023-07-13T22:28:35.995 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:35.995 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:06.113196+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:02.360339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:03.609373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:35.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff21c06ace0 msgr2=0x7ff21c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff21c06ace0 0x7ff21c06d190 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff22c000c00 tx=0x7ff22c008030 comp rx=0 tx=0).stop 2023-07-13T22:28:35.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0f50 msgr2=0x7ff23c0ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:35.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0f50 0x7ff23c0ca200 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff23400cb40 tx=0x7ff23400ce50 comp rx=0 tx=0).stop 2023-07-13T22:28:35.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 shutdown_connections 2023-07-13T22:28:35.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff21c06ace0 0x7ff21c06d190 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff23c0d0f50 0x7ff23c0ca200 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0cf8e0 0x7ff23c144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 --2- 172.21.15.60:0/1780262922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff23c0d0b80 0x7ff23c0c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:35.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 >> 172.21.15.60:0/1780262922 conn(0x7ff23c071660 msgr2=0x7ff23c071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:35.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 shutdown_connections 2023-07-13T22:28:35.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.017+0000 7ff2411ff700 1 -- 172.21.15.60:0/1780262922 wait complete. 2023-07-13T22:28:36.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:28:02.360339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:28:03.609373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:28:36.016 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:28:36.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.245+0000 7fcc091ff700 1 Processor -- start 2023-07-13T22:28:36.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.245+0000 7fcc091ff700 1 -- start start 2023-07-13T22:28:36.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.246+0000 7fcc091ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcc040c9070 0x7fcc040718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.246+0000 7fcc091ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcc040cb220 0x7fcc040cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.246+0000 7fcc091ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc040ca1f0 con 0x7fcc0415a5c0 2023-07-13T22:28:36.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.246+0000 7fcc091ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc04071e30 con 0x7fcc040c9070 2023-07-13T22:28:36.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.246+0000 7fcc091ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc04070ca0 con 0x7fcc040cb220 2023-07-13T22:28:36.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.246+0000 7fcc0259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcc040c9070 0x7fcc040718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.246+0000 7fcc02d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcc0415a5c0 0x7fcc040ca680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60366/0 (socket says 172.21.15.60:60366) 2023-07-13T22:28:36.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.246+0000 7fcc02d9d700 1 -- 172.21.15.60:0/1932759977 learned_addr learned my addr 172.21.15.60:0/1932759977 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc0359e700 1 --2- 172.21.15.60:0/1932759977 >> v2:172.21.15.105:3300/0 conn(0x7fcc040cb220 0x7fcc040cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc0259c700 1 -- 172.21.15.60:0/1932759977 >> v2:172.21.15.105:3300/0 conn(0x7fcc040cb220 msgr2=0x7fcc040cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc0259c700 1 --2- 172.21.15.60:0/1932759977 >> v2:172.21.15.105:3300/0 conn(0x7fcc040cb220 0x7fcc040cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc0259c700 1 -- 172.21.15.60:0/1932759977 >> v1:172.21.15.60:6789/0 conn(0x7fcc0415a5c0 legacy=0x7fcc040ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc0259c700 1 -- 172.21.15.60:0/1932759977 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040c95e0 con 0x7fcc040c9070 2023-07-13T22:28:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc01d9b700 1 -- 172.21.15.60:0/1932759977 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389582923 0 0) 0x7fcc040ca1f0 con 0x7fcc0415a5c0 2023-07-13T22:28:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc0259c700 1 --2- 172.21.15.60:0/1932759977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040c9070 0x7fcc040718f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcc0406fa30 tx=0x7fcbf8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3465952123ff0c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc01d9b700 1 -- 172.21.15.60:0/1932759977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8011830 con 0x7fcc040c9070 2023-07-13T22:28:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc01d9b700 1 -- 172.21.15.60:0/1932759977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf8011990 con 0x7fcc040c9070 2023-07-13T22:28:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.247+0000 7fcc01d9b700 1 -- 172.21.15.60:0/1932759977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8011d20 con 0x7fcc040c9070 2023-07-13T22:28:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.248+0000 7fcc091ff700 1 -- 172.21.15.60:0/1932759977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040c9070 msgr2=0x7fcc040718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.248+0000 7fcc091ff700 1 --2- 172.21.15.60:0/1932759977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040c9070 0x7fcc040718f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcc0406fa30 tx=0x7fcbf8009b20 comp rx=0 tx=0).stop 2023-07-13T22:28:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.248+0000 7fcc091ff700 1 -- 172.21.15.60:0/1932759977 shutdown_connections 2023-07-13T22:28:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.248+0000 7fcc091ff700 1 --2- 172.21.15.60:0/1932759977 >> v2:172.21.15.105:3300/0 conn(0x7fcc040cb220 0x7fcc040cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.248+0000 7fcc091ff700 1 --2- 172.21.15.60:0/1932759977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040c9070 0x7fcc040718f0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.248+0000 7fcc091ff700 1 -- 172.21.15.60:0/1932759977 >> 172.21.15.60:0/1932759977 conn(0x7fcc04069f30 msgr2=0x7fcc0406a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.248+0000 7fcc091ff700 1 -- 172.21.15.60:0/1932759977 shutdown_connections 2023-07-13T22:28:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.248+0000 7fcc091ff700 1 -- 172.21.15.60:0/1932759977 wait complete. 2023-07-13T22:28:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.250+0000 7fcc091ff700 1 Processor -- start 2023-07-13T22:28:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.250+0000 7fcc091ff700 1 -- start start 2023-07-13T22:28:36.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc091ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc040c9070 0x7fcc04155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc091ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040cb220 0x7fcc040ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc091ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc0415a5c0 0x7fcc041e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc091ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c53d0 con 0x7fcc040cb220 2023-07-13T22:28:36.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc091ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c4750 con 0x7fcc0415a5c0 2023-07-13T22:28:36.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc091ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c4a30 con 0x7fcc040c9070 2023-07-13T22:28:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc0259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040cb220 0x7fcc040ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc0259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040cb220 0x7fcc040ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39576/0 (socket says 172.21.15.60:39576) 2023-07-13T22:28:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc0259c700 1 -- 172.21.15.60:0/4037127623 learned_addr learned my addr 172.21.15.60:0/4037127623 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc0359e700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc0415a5c0 0x7fcc041e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.251+0000 7fcc02d9d700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc040c9070 0x7fcc04155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc0259c700 1 -- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc040c9070 msgr2=0x7fcc04155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc0259c700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc040c9070 0x7fcc04155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc0259c700 1 -- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc0415a5c0 msgr2=0x7fcc041e5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc0259c700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc0415a5c0 0x7fcc041e5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc0259c700 1 -- 172.21.15.60:0/4037127623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf80094a0 con 0x7fcc040cb220 2023-07-13T22:28:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc0359e700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc0415a5c0 0x7fcc041e5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc02d9d700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc040c9070 0x7fcc04155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc0259c700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040cb220 0x7fcc040ce1c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80090b0 tx=0x7fcbf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcbeb7fe700 1 -- 172.21.15.60:0/4037127623 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf80038a0 con 0x7fcc040cb220 2023-07-13T22:28:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcbeb7fe700 1 -- 172.21.15.60:0/4037127623 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf801f070 con 0x7fcc040cb220 2023-07-13T22:28:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc040c4d10 con 0x7fcc040cb220 2023-07-13T22:28:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcbeb7fe700 1 -- 172.21.15.60:0/4037127623 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8003c20 con 0x7fcc040cb220 2023-07-13T22:28:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.252+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc04156620 con 0x7fcc040cb220 2023-07-13T22:28:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.254+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc04066b20 con 0x7fcc040cb220 2023-07-13T22:28:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.254+0000 7fcbeb7fe700 1 -- 172.21.15.60:0/4037127623 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcbf8026480 con 0x7fcc040cb220 2023-07-13T22:28:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.255+0000 7fcbeb7fe700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcbf006ace0 0x7fcbf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.255+0000 7fcbeb7fe700 1 -- 172.21.15.60:0/4037127623 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbf8021070 con 0x7fcc040cb220 2023-07-13T22:28:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.256+0000 7fcc02d9d700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcbf006ace0 0x7fcbf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.256+0000 7fcc02d9d700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcbf006ace0 0x7fcbf006d190 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4000f60 tx=0x7fcbf4007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:36.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.260+0000 7fcbeb7fe700 1 -- 172.21.15.60:0/4037127623 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcbf8055fa0 con 0x7fcc040cb220 2023-07-13T22:28:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.463+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcc040c5510 con 0x7fcc040cb220 2023-07-13T22:28:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.464+0000 7fcbeb7fe700 1 -- 172.21.15.60:0/4037127623 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcc040c5510 con 0x7fcc040cb220 2023-07-13T22:28:36.443 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:28:36.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcbf006ace0 msgr2=0x7fcbf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcbf006ace0 0x7fcbf006d190 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4000f60 tx=0x7fcbf4007d30 comp rx=0 tx=0).stop 2023-07-13T22:28:36.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040cb220 msgr2=0x7fcc040ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040cb220 0x7fcc040ce1c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80090b0 tx=0x7fcbf8004750 comp rx=0 tx=0).stop 2023-07-13T22:28:36.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 shutdown_connections 2023-07-13T22:28:36.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcbf006ace0 0x7fcbf006d190 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc0415a5c0 0x7fcc041e5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc040c9070 0x7fcc04155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 --2- 172.21.15.60:0/4037127623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc040cb220 0x7fcc040ce1c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 >> 172.21.15.60:0/4037127623 conn(0x7fcc04069f30 msgr2=0x7fcc04070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:36.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.466+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 shutdown_connections 2023-07-13T22:28:36.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.467+0000 7fcc091ff700 1 -- 172.21.15.60:0/4037127623 wait complete. 2023-07-13T22:28:36.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:28:36.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.691+0000 7fd1877ff700 1 Processor -- start 2023-07-13T22:28:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.691+0000 7fd1877ff700 1 -- start start 2023-07-13T22:28:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.692+0000 7fd1877ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1800b4e50 0x7fd180160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.692+0000 7fd1877ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1800cf280 0x7fd1800d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.692+0000 7fd1877ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd180161020 con 0x7fd18015c620 2023-07-13T22:28:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.692+0000 7fd1877ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1800cd310 con 0x7fd1800b4e50 2023-07-13T22:28:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.692+0000 7fd1877ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18015c9f0 con 0x7fd1800cf280 2023-07-13T22:28:36.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.692+0000 7fd184d9a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1800b4e50 0x7fd180160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.692+0000 7fd185d9c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1800cf280 0x7fd1800d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd18559b700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd18015c620 0x7fd1800b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:55852/0 (socket says 172.21.15.60:55852) 2023-07-13T22:28:36.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd184d9a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1800b4e50 0x7fd180160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:44278/0 (socket says 172.21.15.60:44278) 2023-07-13T22:28:36.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd184d9a700 1 -- 172.21.15.60:0/4237580812 learned_addr learned my addr 172.21.15.60:0/4237580812 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:36.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd184d9a700 1 -- 172.21.15.60:0/4237580812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1800cf280 msgr2=0x7fd1800d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:36.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd184d9a700 1 --2- 172.21.15.60:0/4237580812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1800cf280 0x7fd1800d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd184d9a700 1 -- 172.21.15.60:0/4237580812 >> v1:172.21.15.105:6789/0 conn(0x7fd18015c620 legacy=0x7fd1800b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd184d9a700 1 -- 172.21.15.60:0/4237580812 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800cd560 con 0x7fd1800b4e50 2023-07-13T22:28:36.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd177fff700 1 -- 172.21.15.60:0/4237580812 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430057610 0 0) 0x7fd17c001110 con 0x7fd18015c620 2023-07-13T22:28:36.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd185d9c700 1 --2- 172.21.15.60:0/4237580812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1800cf280 0x7fd1800d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:28:36.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.693+0000 7fd184d9a700 1 --2- 172.21.15.60:0/4237580812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 0x7fd180160ae0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd1700098f0 tx=0x7fd170009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7c32ce47e815cb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:36.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd177fff700 1 -- 172.21.15.60:0/4237580812 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170011830 con 0x7fd1800b4e50 2023-07-13T22:28:36.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd177fff700 1 -- 172.21.15.60:0/4237580812 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170011990 con 0x7fd1800b4e50 2023-07-13T22:28:36.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd177fff700 1 -- 172.21.15.60:0/4237580812 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170011d20 con 0x7fd1800b4e50 2023-07-13T22:28:36.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd1877ff700 1 -- 172.21.15.60:0/4237580812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 msgr2=0x7fd180160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd1877ff700 1 --2- 172.21.15.60:0/4237580812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 0x7fd180160ae0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd1700098f0 tx=0x7fd170009c00 comp rx=0 tx=0).stop 2023-07-13T22:28:36.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd1877ff700 1 -- 172.21.15.60:0/4237580812 shutdown_connections 2023-07-13T22:28:36.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd1877ff700 1 --2- 172.21.15.60:0/4237580812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1800cf280 0x7fd1800d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd1877ff700 1 --2- 172.21.15.60:0/4237580812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 0x7fd180160ae0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd1877ff700 1 -- 172.21.15.60:0/4237580812 >> 172.21.15.60:0/4237580812 conn(0x7fd18006b210 msgr2=0x7fd1800cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd1877ff700 1 -- 172.21.15.60:0/4237580812 shutdown_connections 2023-07-13T22:28:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.694+0000 7fd1877ff700 1 -- 172.21.15.60:0/4237580812 wait complete. 2023-07-13T22:28:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.696+0000 7fd1877ff700 1 Processor -- start 2023-07-13T22:28:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.696+0000 7fd1877ff700 1 -- start start 2023-07-13T22:28:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd1877ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 0x7fd18015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd1877ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1800cf280 0x7fd180158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd1877ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18015c620 0x7fd18015e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd1877ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800cb6d0 con 0x7fd1800cf280 2023-07-13T22:28:36.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd1877ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18015f2d0 con 0x7fd1800b4e50 2023-07-13T22:28:36.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd1877ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18015f5b0 con 0x7fd18015c620 2023-07-13T22:28:36.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd18559b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 0x7fd18015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd184d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1800cf280 0x7fd180158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd18559b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 0x7fd18015cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:44290/0 (socket says 172.21.15.60:44290) 2023-07-13T22:28:36.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.697+0000 7fd18559b700 1 -- 172.21.15.60:0/3150556049 learned_addr learned my addr 172.21.15.60:0/3150556049 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:36.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd185d9c700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18015c620 0x7fd18015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd184d9a700 1 -- 172.21.15.60:0/3150556049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18015c620 msgr2=0x7fd18015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd184d9a700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18015c620 0x7fd18015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd184d9a700 1 -- 172.21.15.60:0/3150556049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 msgr2=0x7fd18015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd184d9a700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 0x7fd18015cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd184d9a700 1 -- 172.21.15.60:0/3150556049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd178009a70 con 0x7fd1800cf280 2023-07-13T22:28:36.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd184d9a700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1800cf280 0x7fd180158540 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd170016000 tx=0x7fd170004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:36.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd175ffb700 1 -- 172.21.15.60:0/3150556049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1700044d0 con 0x7fd1800cf280 2023-07-13T22:28:36.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd175ffb700 1 -- 172.21.15.60:0/3150556049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17001e070 con 0x7fd1800cf280 2023-07-13T22:28:36.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd170009550 con 0x7fd1800cf280 2023-07-13T22:28:36.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.698+0000 7fd175ffb700 1 -- 172.21.15.60:0/3150556049 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1700038e0 con 0x7fd1800cf280 2023-07-13T22:28:36.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.699+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18015e990 con 0x7fd1800cf280 2023-07-13T22:28:36.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.700+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18015fad0 con 0x7fd1800cf280 2023-07-13T22:28:36.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.701+0000 7fd175ffb700 1 -- 172.21.15.60:0/3150556049 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd170003a40 con 0x7fd1800cf280 2023-07-13T22:28:36.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.701+0000 7fd175ffb700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16406ace0 0x7fd16406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:36.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.702+0000 7fd175ffb700 1 -- 172.21.15.60:0/3150556049 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd17008a2b0 con 0x7fd1800cf280 2023-07-13T22:28:36.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.702+0000 7fd18559b700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16406ace0 0x7fd16406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:36.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.702+0000 7fd18559b700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16406ace0 0x7fd16406d190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd17c007fd0 tx=0x7fd17c007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:36.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.706+0000 7fd175ffb700 1 -- 172.21.15.60:0/3150556049 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd170055350 con 0x7fd1800cf280 2023-07-13T22:28:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.909+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1800741e0 con 0x7fd1800cf280 2023-07-13T22:28:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.909+0000 7fd175ffb700 1 -- 172.21.15.60:0/3150556049 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd170058970 con 0x7fd1800cf280 2023-07-13T22:28:36.889 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:28:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.911+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16406ace0 msgr2=0x7fd16406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.911+0000 7fd1877ff700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16406ace0 0x7fd16406d190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd17c007fd0 tx=0x7fd17c007ce0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.911+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1800cf280 msgr2=0x7fd180158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.911+0000 7fd1877ff700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1800cf280 0x7fd180158540 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd170016000 tx=0x7fd170004750 comp rx=0 tx=0).stop 2023-07-13T22:28:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.911+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 shutdown_connections 2023-07-13T22:28:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.912+0000 7fd1877ff700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16406ace0 0x7fd16406d190 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.912+0000 7fd1877ff700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18015c620 0x7fd18015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.912+0000 7fd1877ff700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1800b4e50 0x7fd18015cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.912+0000 7fd1877ff700 1 --2- 172.21.15.60:0/3150556049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1800cf280 0x7fd180158540 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.912+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 >> 172.21.15.60:0/3150556049 conn(0x7fd18006b210 msgr2=0x7fd18006be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.912+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 shutdown_connections 2023-07-13T22:28:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:36.912+0000 7fd1877ff700 1 -- 172.21.15.60:0/3150556049 wait complete. 2023-07-13T22:28:36.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:28:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.137+0000 7f25e75ff700 1 Processor -- start 2023-07-13T22:28:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.138+0000 7f25e75ff700 1 -- start start 2023-07-13T22:28:37.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.139+0000 7f25e75ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f25e015a5c0 0x7f25e00718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.139+0000 7f25e75ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25e00cb220 0x7f25e00cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.139+0000 7f25e75ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e00ca1f0 con 0x7f25e00c9070 2023-07-13T22:28:37.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.139+0000 7f25e75ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e0071e30 con 0x7f25e00cb220 2023-07-13T22:28:37.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.139+0000 7f25e75ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e0070ca0 con 0x7f25e015a5c0 2023-07-13T22:28:37.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.139+0000 7f25e5b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25e00cb220 0x7f25e00cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.140+0000 7f25e5b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25e00cb220 0x7f25e00cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39580/0 (socket says 172.21.15.60:39580) 2023-07-13T22:28:37.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.140+0000 7f25e5b9c700 1 -- 172.21.15.60:0/2721562331 learned_addr learned my addr 172.21.15.60:0/2721562331 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:37.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.140+0000 7f25e539b700 1 --2- 172.21.15.60:0/2721562331 >> v2:172.21.15.105:3300/0 conn(0x7f25e015a5c0 0x7f25e00718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.140+0000 7f25e5b9c700 1 -- 172.21.15.60:0/2721562331 >> v2:172.21.15.105:3300/0 conn(0x7f25e015a5c0 msgr2=0x7f25e00718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:37.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.140+0000 7f25e5b9c700 1 --2- 172.21.15.60:0/2721562331 >> v2:172.21.15.105:3300/0 conn(0x7f25e015a5c0 0x7f25e00718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.140+0000 7f25e5b9c700 1 -- 172.21.15.60:0/2721562331 >> v1:172.21.15.105:6789/0 conn(0x7f25e00c9070 legacy=0x7f25e00ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.140+0000 7f25e5b9c700 1 -- 172.21.15.60:0/2721562331 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00c95e0 con 0x7f25e00cb220 2023-07-13T22:28:37.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.141+0000 7f25e5b9c700 1 --2- 172.21.15.60:0/2721562331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 0x7f25e00cb660 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f25e006f020 tx=0x7f25d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0adbfd95b693853 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:37.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.141+0000 7f25d7fff700 1 -- 172.21.15.60:0/2721562331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800e070 con 0x7f25e00cb220 2023-07-13T22:28:37.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.141+0000 7f25d7fff700 1 -- 172.21.15.60:0/2721562331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d800b910 con 0x7f25e00cb220 2023-07-13T22:28:37.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.141+0000 7f25d7fff700 1 -- 172.21.15.60:0/2721562331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800bcc0 con 0x7f25e00cb220 2023-07-13T22:28:37.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.142+0000 7f25e75ff700 1 -- 172.21.15.60:0/2721562331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 msgr2=0x7f25e00cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.142+0000 7f25e75ff700 1 --2- 172.21.15.60:0/2721562331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 0x7f25e00cb660 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f25e006f020 tx=0x7f25d800b330 comp rx=0 tx=0).stop 2023-07-13T22:28:37.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.142+0000 7f25e75ff700 1 -- 172.21.15.60:0/2721562331 shutdown_connections 2023-07-13T22:28:37.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.142+0000 7f25e75ff700 1 --2- 172.21.15.60:0/2721562331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 0x7f25e00cb660 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.142+0000 7f25e75ff700 1 --2- 172.21.15.60:0/2721562331 >> v2:172.21.15.105:3300/0 conn(0x7f25e015a5c0 0x7f25e00718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.142+0000 7f25e75ff700 1 -- 172.21.15.60:0/2721562331 >> 172.21.15.60:0/2721562331 conn(0x7f25e0069f30 msgr2=0x7f25e006a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:37.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.142+0000 7f25e75ff700 1 -- 172.21.15.60:0/2721562331 shutdown_connections 2023-07-13T22:28:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.142+0000 7f25e75ff700 1 -- 172.21.15.60:0/2721562331 wait complete. 2023-07-13T22:28:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.144+0000 7f25e75ff700 1 Processor -- start 2023-07-13T22:28:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.144+0000 7f25e75ff700 1 -- start start 2023-07-13T22:28:37.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.145+0000 7f25e75ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e00c9070 0x7f25e01f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.145+0000 7f25e75ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 0x7f25e00c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.145+0000 7f25e75ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e015a5c0 0x7f25e00c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.145+0000 7f25e75ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00c5030 con 0x7f25e00cb220 2023-07-13T22:28:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.145+0000 7f25e75ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e01f1b90 con 0x7f25e015a5c0 2023-07-13T22:28:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.145+0000 7f25e75ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e015b7c0 con 0x7f25e00c9070 2023-07-13T22:28:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e4b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 0x7f25e00c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e4b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 0x7f25e00c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39582/0 (socket says 172.21.15.60:39582) 2023-07-13T22:28:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e4b9a700 1 -- 172.21.15.60:0/1406689492 learned_addr learned my addr 172.21.15.60:0/1406689492 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e539b700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e00c9070 0x7f25e01f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e5b9c700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e015a5c0 0x7f25e00c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e4b9a700 1 -- 172.21.15.60:0/1406689492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e00c9070 msgr2=0x7f25e01f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e4b9a700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e00c9070 0x7f25e01f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e4b9a700 1 -- 172.21.15.60:0/1406689492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e015a5c0 msgr2=0x7f25e00c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e4b9a700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e015a5c0 0x7f25e00c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.146+0000 7f25e4b9a700 1 -- 172.21.15.60:0/1406689492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d8009b30 con 0x7f25e00cb220 2023-07-13T22:28:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.147+0000 7f25e4b9a700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 0x7f25e00c3fb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c510 tx=0x7f25d000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.147+0000 7f25d5ffb700 1 -- 172.21.15.60:0/1406689492 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d00052d0 con 0x7f25e00cb220 2023-07-13T22:28:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.147+0000 7f25d5ffb700 1 -- 172.21.15.60:0/1406689492 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d0005430 con 0x7f25e00cb220 2023-07-13T22:28:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.147+0000 7f25d5ffb700 1 -- 172.21.15.60:0/1406689492 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d00038c0 con 0x7f25e00cb220 2023-07-13T22:28:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.147+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e00c4a70 con 0x7f25e00cb220 2023-07-13T22:28:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.147+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e015b450 con 0x7f25e00cb220 2023-07-13T22:28:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.148+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e0066b20 con 0x7f25e00cb220 2023-07-13T22:28:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.149+0000 7f25d5ffb700 1 -- 172.21.15.60:0/1406689492 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f25d0003a20 con 0x7f25e00cb220 2023-07-13T22:28:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.150+0000 7f25d5ffb700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c406ace0 0x7f25c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.150+0000 7f25d5ffb700 1 -- 172.21.15.60:0/1406689492 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25d0088800 con 0x7f25e00cb220 2023-07-13T22:28:37.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.150+0000 7f25e539b700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c406ace0 0x7f25c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.151+0000 7f25e539b700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c406ace0 0x7f25c406d190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f25e00b0890 tx=0x7f25dc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:37.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.154+0000 7f25d5ffb700 1 -- 172.21.15.60:0/1406689492 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f25d0053810 con 0x7f25e00cb220 2023-07-13T22:28:37.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.361+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25e015b290 con 0x7f25e00cb220 2023-07-13T22:28:37.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.361+0000 7f25d5ffb700 1 -- 172.21.15.60:0/1406689492 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25d0056e30 con 0x7f25e00cb220 2023-07-13T22:28:37.341 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:28:37.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c406ace0 msgr2=0x7f25c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c406ace0 0x7f25c406d190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f25e00b0890 tx=0x7f25dc009210 comp rx=0 tx=0).stop 2023-07-13T22:28:37.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 msgr2=0x7f25e00c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 0x7f25e00c3fb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c510 tx=0x7f25d000c820 comp rx=0 tx=0).stop 2023-07-13T22:28:37.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 shutdown_connections 2023-07-13T22:28:37.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c406ace0 0x7f25c406d190 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e015a5c0 0x7f25e00c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e00c9070 0x7f25e01f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 --2- 172.21.15.60:0/1406689492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e00cb220 0x7f25e00c3fb0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 >> 172.21.15.60:0/1406689492 conn(0x7f25e0069f30 msgr2=0x7f25e0070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:37.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 shutdown_connections 2023-07-13T22:28:37.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.364+0000 7f25e75ff700 1 -- 172.21.15.60:0/1406689492 wait complete. 2023-07-13T22:28:37.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:28:37.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.589+0000 7f2eadfff700 1 Processor -- start 2023-07-13T22:28:37.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.589+0000 7f2eadfff700 1 -- start start 2023-07-13T22:28:37.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.590+0000 7f2eadfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2ea815c620 0x7f2ea8160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.590+0000 7f2eadfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80b4e50 0x7f2ea80d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.590+0000 7f2eadfff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea8161020 con 0x7f2ea80cf280 2023-07-13T22:28:37.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.590+0000 7f2eadfff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea80cd310 con 0x7f2ea815c620 2023-07-13T22:28:37.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.590+0000 7f2eadfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea815c9f0 con 0x7f2ea80b4e50 2023-07-13T22:28:37.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea6ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2ea80cf280 0x7f2ea80b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60374/0 (socket says 172.21.15.60:60374) 2023-07-13T22:28:37.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/1252686841 learned_addr learned my addr 172.21.15.60:0/1252686841 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:37.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/1252686841 >> v2:172.21.15.105:3300/0 conn(0x7f2ea815c620 0x7f2ea8160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea7fff700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80b4e50 0x7f2ea80d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035569145 0 0) 0x7f2e9c001110 con 0x7f2ea80cf280 2023-07-13T22:28:37.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e900035b0 con 0x7f2ea80cf280 2023-07-13T22:28:37.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3495391975 0 0) 0x7f2e900035b0 con 0x7f2ea80cf280 2023-07-13T22:28:37.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80b4e50 msgr2=0x7f2ea80d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:37.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80b4e50 0x7f2ea80d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 >> v2:172.21.15.105:3300/0 conn(0x7f2ea815c620 msgr2=0x7f2ea8160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:37.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 --2- 172.21.15.60:0/1252686841 >> v2:172.21.15.105:3300/0 conn(0x7f2ea815c620 0x7f2ea8160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80cd560 con 0x7f2ea80cf280 2023-07-13T22:28:37.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f2e9c001800 con 0x7f2ea80cf280 2023-07-13T22:28:37.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 >> v1:172.21.15.60:6789/0 conn(0x7f2ea80cf280 legacy=0x7f2ea80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e90003a00 0x7f2e90003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.591+0000 7f2ea67fc700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80b4e50 0x7f2e90004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea67fc700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e90004d90 0x7f2e90005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80cd310 con 0x7f2e90003a00 2023-07-13T22:28:37.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea815c9f0 con 0x7f2e90004d90 2023-07-13T22:28:37.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9000ba10 con 0x7f2ea80b4e50 2023-07-13T22:28:37.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e90003a00 0x7f2e90003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea6ffd700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80b4e50 0x7f2e90004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea7fff700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e90004d90 0x7f2e90005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea77fe700 1 -- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80b4e50 msgr2=0x7f2e90004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:37.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2ea80b4e50 0x7f2e90004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea77fe700 1 -- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e90004d90 msgr2=0x7f2e90005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:37.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e90004d90 0x7f2e90005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea77fe700 1 -- 172.21.15.60:0/1252686841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e900035b0 con 0x7f2e90003a00 2023-07-13T22:28:37.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.592+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e90003a00 0x7f2e90003e10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009b60 tx=0x7f2e90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93af89618445ebb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800e6c0 con 0x7f2e90003a00 2023-07-13T22:28:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9800e820 con 0x7f2e90003a00 2023-07-13T22:28:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2ea67fc700 1 -- 172.21.15.60:0/1252686841 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800eb90 con 0x7f2e90003a00 2023-07-13T22:28:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2eadfff700 1 -- 172.21.15.60:0/1252686841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e90003a00 msgr2=0x7f2e90003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2eadfff700 1 --2- 172.21.15.60:0/1252686841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e90003a00 0x7f2e90003e10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009b60 tx=0x7f2e90005880 comp rx=0 tx=0).stop 2023-07-13T22:28:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2ea7fff700 1 -- 172.21.15.60:0/1252686841 reap_dead start 2023-07-13T22:28:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2eadfff700 1 -- 172.21.15.60:0/1252686841 shutdown_connections 2023-07-13T22:28:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2eadfff700 1 -- 172.21.15.60:0/1252686841 >> 172.21.15.60:0/1252686841 conn(0x7f2ea806b210 msgr2=0x7f2ea80cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2eadfff700 1 -- 172.21.15.60:0/1252686841 shutdown_connections 2023-07-13T22:28:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.593+0000 7f2eadfff700 1 -- 172.21.15.60:0/1252686841 wait complete. 2023-07-13T22:28:37.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.595+0000 7f2eadfff700 1 Processor -- start 2023-07-13T22:28:37.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.595+0000 7f2eadfff700 1 -- start start 2023-07-13T22:28:37.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.596+0000 7f2eadfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e90003a00 0x7f2ea81e6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.596+0000 7f2eadfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea80b5af0 0x7f2ea815f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.596+0000 7f2eadfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea806c130 0x7f2ea806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.596+0000 7f2eadfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81e7d60 con 0x7f2ea80b5af0 2023-07-13T22:28:37.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.596+0000 7f2eadfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea806cb10 con 0x7f2ea806c130 2023-07-13T22:28:37.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.596+0000 7f2eadfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81e72b0 con 0x7f2e90003a00 2023-07-13T22:28:37.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.596+0000 7f2ea6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea80b5af0 0x7f2ea815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea80b5af0 0x7f2ea815f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39586/0 (socket says 172.21.15.60:39586) 2023-07-13T22:28:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea77fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e90003a00 0x7f2ea81e6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/1223317124 learned_addr learned my addr 172.21.15.60:0/1223317124 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea7fff700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea806c130 0x7f2ea806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/1223317124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e90003a00 msgr2=0x7f2ea81e6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea6ffd700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e90003a00 0x7f2ea81e6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/1223317124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea806c130 msgr2=0x7f2ea806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea6ffd700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea806c130 0x7f2ea806c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea6ffd700 1 -- 172.21.15.60:0/1223317124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80b7430 con 0x7f2ea80b5af0 2023-07-13T22:28:37.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2ea6ffd700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea80b5af0 0x7f2ea815f440 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c001c30 tx=0x7f2e9c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:37.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2e8ffff700 1 -- 172.21.15.60:0/1223317124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00f5e0 con 0x7f2ea80b5af0 2023-07-13T22:28:37.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2e8ffff700 1 -- 172.21.15.60:0/1223317124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c00f780 con 0x7f2ea80b5af0 2023-07-13T22:28:37.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.597+0000 7f2e8ffff700 1 -- 172.21.15.60:0/1223317124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00fb30 con 0x7f2ea80b5af0 2023-07-13T22:28:37.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.598+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea81e74c0 con 0x7f2ea80b5af0 2023-07-13T22:28:37.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.598+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea81e7a80 con 0x7f2ea80b5af0 2023-07-13T22:28:37.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.599+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea81e66c0 con 0x7f2ea80b5af0 2023-07-13T22:28:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.600+0000 7f2e8ffff700 1 -- 172.21.15.60:0/1223317124 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2e9c00fc90 con 0x7f2ea80b5af0 2023-07-13T22:28:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.601+0000 7f2e8ffff700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.601+0000 7f2e8ffff700 1 -- 172.21.15.60:0/1223317124 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e9c089230 con 0x7f2ea80b5af0 2023-07-13T22:28:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.601+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.601+0000 7f2ea77fe700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2e980090b0 tx=0x7f2e9801f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.605+0000 7f2e8ffff700 1 -- 172.21.15.60:0/1223317124 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2e9c0542d0 con 0x7f2ea80b5af0 2023-07-13T22:28:37.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.815+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ea80741e0 con 0x7f2ea80b5af0 2023-07-13T22:28:37.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.816+0000 7f2e8ffff700 1 -- 172.21.15.60:0/1223317124 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e9c0578f0 con 0x7f2ea80b5af0 2023-07-13T22:28:37.795 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:28:37.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 msgr2=0x7f2e9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2e980090b0 tx=0x7f2e9801f040 comp rx=0 tx=0).stop 2023-07-13T22:28:37.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea80b5af0 msgr2=0x7f2ea815f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:37.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea80b5af0 0x7f2ea815f440 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c001c30 tx=0x7f2e9c0075f0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 shutdown_connections 2023-07-13T22:28:37.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2e9406ace0 0x7f2e9406d190 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ea806c130 0x7f2ea806c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e90003a00 0x7f2ea81e6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 --2- 172.21.15.60:0/1223317124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea80b5af0 0x7f2ea815f440 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:37.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 >> 172.21.15.60:0/1223317124 conn(0x7f2ea806b210 msgr2=0x7f2ea815b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:37.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 shutdown_connections 2023-07-13T22:28:37.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:37.818+0000 7f2eadfff700 1 -- 172.21.15.60:0/1223317124 wait complete. 2023-07-13T22:28:37.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:28:38.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.040+0000 7faa4c9ff700 1 Processor -- start 2023-07-13T22:28:38.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.041+0000 7faa4c9ff700 1 -- start start 2023-07-13T22:28:38.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.042+0000 7faa4c9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa4806b750 0x7faa480cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.042+0000 7faa4c9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa480b0240 con 0x7faa480cc600 2023-07-13T22:28:38.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.042+0000 7faa4c9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa480b0ae0 con 0x7faa480d11e0 2023-07-13T22:28:38.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.042+0000 7faa4c9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4815cc30 con 0x7faa4806b750 2023-07-13T22:28:38.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.042+0000 7faa46d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faa480cc600 0x7faa480b0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60390/0 (socket says 172.21.15.60:60390) 2023-07-13T22:28:38.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.042+0000 7faa46d9d700 1 -- 172.21.15.60:0/2040781640 learned_addr learned my addr 172.21.15.60:0/2040781640 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.042+0000 7faa3ffff700 1 --2- 172.21.15.60:0/2040781640 >> v2:172.21.15.105:3300/0 conn(0x7faa4806b750 0x7faa480cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.042+0000 7faa45d9b700 1 -- 172.21.15.60:0/2040781640 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120575875 0 0) 0x7faa480b0240 con 0x7faa480cc600 2023-07-13T22:28:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.042+0000 7faa45d9b700 1 -- 172.21.15.60:0/2040781640 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa340035b0 con 0x7faa480cc600 2023-07-13T22:28:38.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.043+0000 7faa45d9b700 1 -- 172.21.15.60:0/2040781640 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3860115789 0 0) 0x7faa340035b0 con 0x7faa480cc600 2023-07-13T22:28:38.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.043+0000 7faa3ffff700 1 -- 172.21.15.60:0/2040781640 >> v1:172.21.15.105:6789/0 conn(0x7faa480d11e0 legacy=0x7faa4806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.043+0000 7faa3ffff700 1 -- 172.21.15.60:0/2040781640 >> v1:172.21.15.60:6789/0 conn(0x7faa480cc600 legacy=0x7faa480b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.043+0000 7faa3ffff700 1 -- 172.21.15.60:0/2040781640 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480cce30 con 0x7faa4806b750 2023-07-13T22:28:38.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.043+0000 7faa3ffff700 1 --2- 172.21.15.60:0/2040781640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa4806b750 0x7faa480cc0c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7faa30009840 tx=0x7faa30009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d051fa4a2ead1d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:38.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.043+0000 7faa45d9b700 1 -- 172.21.15.60:0/2040781640 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa400019f0 con 0x7faa4806b750 2023-07-13T22:28:38.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.043+0000 7faa45d9b700 1 -- 172.21.15.60:0/2040781640 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa30004030 con 0x7faa4806b750 2023-07-13T22:28:38.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.043+0000 7faa45d9b700 1 -- 172.21.15.60:0/2040781640 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300043c0 con 0x7faa4806b750 2023-07-13T22:28:38.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.044+0000 7faa4c9ff700 1 -- 172.21.15.60:0/2040781640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa4806b750 msgr2=0x7faa480cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.044+0000 7faa4c9ff700 1 --2- 172.21.15.60:0/2040781640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa4806b750 0x7faa480cc0c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7faa30009840 tx=0x7faa30009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:38.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.044+0000 7faa4c9ff700 1 -- 172.21.15.60:0/2040781640 shutdown_connections 2023-07-13T22:28:38.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.044+0000 7faa4c9ff700 1 --2- 172.21.15.60:0/2040781640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa4806b750 0x7faa480cc0c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.044+0000 7faa4c9ff700 1 -- 172.21.15.60:0/2040781640 >> 172.21.15.60:0/2040781640 conn(0x7faa480999d0 msgr2=0x7faa48099dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:38.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.044+0000 7faa4c9ff700 1 -- 172.21.15.60:0/2040781640 shutdown_connections 2023-07-13T22:28:38.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.044+0000 7faa4c9ff700 1 -- 172.21.15.60:0/2040781640 wait complete. 2023-07-13T22:28:38.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.046+0000 7faa4c9ff700 1 Processor -- start 2023-07-13T22:28:38.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.046+0000 7faa4c9ff700 1 -- start start 2023-07-13T22:28:38.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4c9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4806b750 0x7faa480b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4c9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa480cc600 0x7faa481f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4c9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa480d11e0 0x7faa4815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4c9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480b4890 con 0x7faa4806b750 2023-07-13T22:28:38.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4c9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480b39c0 con 0x7faa480d11e0 2023-07-13T22:28:38.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4c9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa481f29b0 con 0x7faa480cc600 2023-07-13T22:28:38.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4806b750 0x7faa480b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4806b750 0x7faa480b6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39590/0 (socket says 172.21.15.60:39590) 2023-07-13T22:28:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4659c700 1 -- 172.21.15.60:0/1664096676 learned_addr learned my addr 172.21.15.60:0/1664096676 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa46d9d700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa480d11e0 0x7faa4815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa3ffff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa480cc600 0x7faa481f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4659c700 1 -- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa480cc600 msgr2=0x7faa481f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4659c700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa480cc600 0x7faa481f1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4659c700 1 -- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa480d11e0 msgr2=0x7faa4815ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.047+0000 7faa4659c700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa480d11e0 0x7faa4815ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.048+0000 7faa4659c700 1 -- 172.21.15.60:0/1664096676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa300094a0 con 0x7faa4806b750 2023-07-13T22:28:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.048+0000 7faa3ffff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa480cc600 0x7faa481f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.048+0000 7faa46d9d700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa480d11e0 0x7faa4815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.048+0000 7faa4659c700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4806b750 0x7faa480b6360 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7faa3800c500 tx=0x7faa3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.048+0000 7faa3effd700 1 -- 172.21.15.60:0/1664096676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380052d0 con 0x7faa4806b750 2023-07-13T22:28:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.048+0000 7faa3effd700 1 -- 172.21.15.60:0/1664096676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa38005430 con 0x7faa4806b750 2023-07-13T22:28:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.048+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa480b3c10 con 0x7faa4806b750 2023-07-13T22:28:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.048+0000 7faa3effd700 1 -- 172.21.15.60:0/1664096676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380038f0 con 0x7faa4806b750 2023-07-13T22:28:38.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.048+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa480b41d0 con 0x7faa4806b750 2023-07-13T22:28:38.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.049+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa480b49d0 con 0x7faa4806b750 2023-07-13T22:28:38.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.050+0000 7faa3effd700 1 -- 172.21.15.60:0/1664096676 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa38003a50 con 0x7faa4806b750 2023-07-13T22:28:38.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.051+0000 7faa3effd700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2806ace0 0x7faa2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.051+0000 7faa3effd700 1 -- 172.21.15.60:0/1664096676 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa38088990 con 0x7faa4806b750 2023-07-13T22:28:38.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.051+0000 7faa3ffff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2806ace0 0x7faa2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.052+0000 7faa3ffff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2806ace0 0x7faa2806d190 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faa4806bb40 tx=0x7faa30008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:38.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.055+0000 7faa3effd700 1 -- 172.21.15.60:0/1664096676 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7faa38053a30 con 0x7faa4806b750 2023-07-13T22:28:38.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.279+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa480741e0 con 0x7faa4806b750 2023-07-13T22:28:38.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.279+0000 7faa3effd700 1 -- 172.21.15.60:0/1664096676 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) v1 ==== 80+0+0 (secure 0 0 0) 0x7faa38057050 con 0x7faa4806b750 2023-07-13T22:28:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2806ace0 msgr2=0x7faa2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2806ace0 0x7faa2806d190 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faa4806bb40 tx=0x7faa30008030 comp rx=0 tx=0).stop 2023-07-13T22:28:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4806b750 msgr2=0x7faa480b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4806b750 0x7faa480b6360 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7faa3800c500 tx=0x7faa3800c8c0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 shutdown_connections 2023-07-13T22:28:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2806ace0 0x7faa2806d190 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4806b750 0x7faa480b6360 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa480d11e0 0x7faa4815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 --2- 172.21.15.60:0/1664096676 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa480cc600 0x7faa481f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 >> 172.21.15.60:0/1664096676 conn(0x7faa480999d0 msgr2=0x7faa480af790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 shutdown_connections 2023-07-13T22:28:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.282+0000 7faa4c9ff700 1 -- 172.21.15.60:0/1664096676 wait complete. 2023-07-13T22:28:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:28:38.279 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:28:38.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:38.280 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:28:38.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[124502]: fuse finished with error 0 and tester_r 0 2023-07-13T22:28:38.356 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:28:38.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:38.357 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:28:38.501 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:28:38.501 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:38.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:28:38.549 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:28:38.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:38.550 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:28:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-13T22:28:38.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:28:38.636 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:28:38.636 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:38.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:28:38.683 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:28:38.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:28:38.684 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:28:38.768 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-13T22:28:38.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:28:38.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:28:38.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.994+0000 7f676c3ff700 1 Processor -- start 2023-07-13T22:28:38.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.994+0000 7f676c3ff700 1 -- start start 2023-07-13T22:28:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.995+0000 7f676c3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67640ca140 0x7f6764070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.995+0000 7f676c3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ae0d0 0x7f676415e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.995+0000 7f676c3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676415a760 con 0x7f67640ca510 2023-07-13T22:28:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.995+0000 7f676c3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6764070e60 con 0x7f67640ca140 2023-07-13T22:28:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.995+0000 7f676c3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67640b12f0 con 0x7f67640ae0d0 2023-07-13T22:28:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.996+0000 7f676999a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f67640ca510 0x7f67640ad9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60404/0 (socket says 172.21.15.60:60404) 2023-07-13T22:28:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.996+0000 7f676999a700 1 -- 172.21.15.60:0/283362316 learned_addr learned my addr 172.21.15.60:0/283362316 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:38.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.996+0000 7f676a19b700 1 --2- 172.21.15.60:0/283362316 >> v2:172.21.15.60:3300/0 conn(0x7f67640ca140 0x7f6764070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.996+0000 7f676a99c700 1 --2- 172.21.15.60:0/283362316 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ae0d0 0x7f676415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.996+0000 7f6769199700 1 -- 172.21.15.60:0/283362316 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170043435 0 0) 0x7f676415a760 con 0x7f67640ca510 2023-07-13T22:28:38.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.996+0000 7f676a19b700 1 -- 172.21.15.60:0/283362316 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ae0d0 msgr2=0x7f676415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:38.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.996+0000 7f676a19b700 1 --2- 172.21.15.60:0/283362316 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ae0d0 0x7f676415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.996+0000 7f676a19b700 1 -- 172.21.15.60:0/283362316 >> v1:172.21.15.60:6789/0 conn(0x7f67640ca510 legacy=0x7f67640ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.996+0000 7f676a19b700 1 -- 172.21.15.60:0/283362316 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764071ce0 con 0x7f67640ca140 2023-07-13T22:28:38.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f676a19b700 1 --2- 172.21.15.60:0/283362316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ca140 0x7f6764070920 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6760009840 tx=0x7f6760009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eace7e798a6c7e9c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:38.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f6769199700 1 -- 172.21.15.60:0/283362316 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760011830 con 0x7f67640ca140 2023-07-13T22:28:38.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f6769199700 1 -- 172.21.15.60:0/283362316 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6760011990 con 0x7f67640ca140 2023-07-13T22:28:38.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f6769199700 1 -- 172.21.15.60:0/283362316 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760011d20 con 0x7f67640ca140 2023-07-13T22:28:38.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f676c3ff700 1 -- 172.21.15.60:0/283362316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ca140 msgr2=0x7f6764070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f676c3ff700 1 --2- 172.21.15.60:0/283362316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ca140 0x7f6764070920 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6760009840 tx=0x7f6760009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:38.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f676c3ff700 1 -- 172.21.15.60:0/283362316 shutdown_connections 2023-07-13T22:28:38.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f676c3ff700 1 --2- 172.21.15.60:0/283362316 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ae0d0 0x7f676415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f676c3ff700 1 --2- 172.21.15.60:0/283362316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ca140 0x7f6764070920 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.997+0000 7f676c3ff700 1 -- 172.21.15.60:0/283362316 >> 172.21.15.60:0/283362316 conn(0x7f67640ce960 msgr2=0x7f67640ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:38.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.998+0000 7f676c3ff700 1 -- 172.21.15.60:0/283362316 shutdown_connections 2023-07-13T22:28:38.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:38.998+0000 7f676c3ff700 1 -- 172.21.15.60:0/283362316 wait complete. 2023-07-13T22:28:38.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.000+0000 7f676c3ff700 1 Processor -- start 2023-07-13T22:28:38.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.000+0000 7f676c3ff700 1 -- start start 2023-07-13T22:28:38.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.001+0000 7f676c3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ae0d0 0x7f67640c2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.001+0000 7f676c3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ca140 0x7f67641f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.001+0000 7f676c3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640ca510 0x7f67641f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.001+0000 7f676c3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676415bcb0 con 0x7f67640ae0d0 2023-07-13T22:28:38.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.001+0000 7f676c3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676415b960 con 0x7f67640ca510 2023-07-13T22:28:38.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.001+0000 7f676c3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676415cb20 con 0x7f67640ca140 2023-07-13T22:28:38.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ae0d0 0x7f67640c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ae0d0 0x7f67640c2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39608/0 (socket says 172.21.15.60:39608) 2023-07-13T22:28:38.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a19b700 1 -- 172.21.15.60:0/3367039987 learned_addr learned my addr 172.21.15.60:0/3367039987 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:38.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676999a700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ca140 0x7f67641f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a99c700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640ca510 0x7f67641f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a19b700 1 -- 172.21.15.60:0/3367039987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ca140 msgr2=0x7f67641f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a19b700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ca140 0x7f67641f29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a19b700 1 -- 172.21.15.60:0/3367039987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640ca510 msgr2=0x7f67641f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:38.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a19b700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640ca510 0x7f67641f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:38.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a19b700 1 -- 172.21.15.60:0/3367039987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600094a0 con 0x7f67640ae0d0 2023-07-13T22:28:38.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a99c700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640ca510 0x7f67641f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:38.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.002+0000 7f676a19b700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ae0d0 0x7f67640c2400 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6760009fd0 tx=0x7f6760004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:38.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.003+0000 7f675affd700 1 -- 172.21.15.60:0/3367039987 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600036a0 con 0x7f67640ae0d0 2023-07-13T22:28:38.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.003+0000 7f675affd700 1 -- 172.21.15.60:0/3367039987 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676001f070 con 0x7f67640ae0d0 2023-07-13T22:28:38.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.003+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676415cde0 con 0x7f67640ae0d0 2023-07-13T22:28:38.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.003+0000 7f675affd700 1 -- 172.21.15.60:0/3367039987 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760003a20 con 0x7f67640ae0d0 2023-07-13T22:28:38.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.003+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67641f1cd0 con 0x7f67640ae0d0 2023-07-13T22:28:38.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.004+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676415d300 con 0x7f67640ae0d0 2023-07-13T22:28:38.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.005+0000 7f675affd700 1 -- 172.21.15.60:0/3367039987 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6760003b80 con 0x7f67640ae0d0 2023-07-13T22:28:38.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.006+0000 7f675affd700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f674806ace0 0x7f674806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:38.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.006+0000 7f675affd700 1 -- 172.21.15.60:0/3367039987 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6760021070 con 0x7f67640ae0d0 2023-07-13T22:28:38.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.006+0000 7f676999a700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f674806ace0 0x7f674806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:38.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.006+0000 7f676999a700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f674806ace0 0x7f674806d190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f676415cab0 tx=0x7f6754007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:38.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.010+0000 7f675affd700 1 -- 172.21.15.60:0/3367039987 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6760055f30 con 0x7f67640ae0d0 2023-07-13T22:28:39.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.265+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6764066b20 con 0x7f67640ae0d0 2023-07-13T22:28:39.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.266+0000 7f675affd700 1 -- 172.21.15.60:0/3367039987 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f676002b090 con 0x7f67640ae0d0 2023-07-13T22:28:39.245 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:39.246 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2395854295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2395854295},{"type":"v1","addr":"172.21.15.105:6834","nonce":2395854295}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1048261776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1048261776},{"type":"v1","addr":"172.21.15.105:6835","nonce":1048261776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/1639899719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1639899719},{"type":"v1","addr":"172.21.15.60:6836","nonce":1639899719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:04.618240+0000","modified":"2023-07-13T22:28:29.557259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7434":{"gid":7434,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":9,"addr":"172.21.15.60:6837/1512512599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":1512512599},{"type":"v1","addr":"172.21.15.60:6837","nonce":1512512599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-13T22:28:39.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.268+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f674806ace0 msgr2=0x7f674806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.268+0000 7f676c3ff700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f674806ace0 0x7f674806d190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f676415cab0 tx=0x7f6754007c90 comp rx=0 tx=0).stop 2023-07-13T22:28:39.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.268+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ae0d0 msgr2=0x7f67640c2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.268+0000 7f676c3ff700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ae0d0 0x7f67640c2400 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6760009fd0 tx=0x7f6760004750 comp rx=0 tx=0).stop 2023-07-13T22:28:39.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.269+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 shutdown_connections 2023-07-13T22:28:39.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.269+0000 7f676c3ff700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f674806ace0 0x7f674806d190 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.269+0000 7f676c3ff700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640ae0d0 0x7f67640c2400 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.269+0000 7f676c3ff700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67640ca510 0x7f67641f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.269+0000 7f676c3ff700 1 --2- 172.21.15.60:0/3367039987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67640ca140 0x7f67641f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.269+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 >> 172.21.15.60:0/3367039987 conn(0x7f67640ce960 msgr2=0x7f67640b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:39.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.269+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 shutdown_connections 2023-07-13T22:28:39.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.269+0000 7f676c3ff700 1 -- 172.21.15.60:0/3367039987 wait complete. 2023-07-13T22:28:39.249 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 53 2023-07-13T22:28:39.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:39.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.490+0000 7fbfadbff700 1 Processor -- start 2023-07-13T22:28:39.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.490+0000 7fbfadbff700 1 -- start start 2023-07-13T22:28:39.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.491+0000 7fbfadbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbfa80d0b80 0x7fbfa80b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfadbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 0x7fbfa80cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfadbff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa80ceeb0 con 0x7fbfa80d0f50 2023-07-13T22:28:39.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfadbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa80b23f0 con 0x7fbfa80d0b80 2023-07-13T22:28:39.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfadbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa80b4bb0 con 0x7fbfa80cf8e0 2023-07-13T22:28:39.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa6ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbfa80d0f50 0x7fbfa80d2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60410/0 (socket says 172.21.15.60:60410) 2023-07-13T22:28:39.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa6ffd700 1 -- 172.21.15.60:0/806775460 learned_addr learned my addr 172.21.15.60:0/806775460 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:39.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa77fe700 1 --2- 172.21.15.60:0/806775460 >> v2:172.21.15.60:3300/0 conn(0x7fbfa80d0b80 0x7fbfa80b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa7fff700 1 --2- 172.21.15.60:0/806775460 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 0x7fbfa80cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa67fc700 1 -- 172.21.15.60:0/806775460 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771702522 0 0) 0x7fbfa80ceeb0 con 0x7fbfa80d0f50 2023-07-13T22:28:39.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa67fc700 1 -- 172.21.15.60:0/806775460 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf900035b0 con 0x7fbfa80d0f50 2023-07-13T22:28:39.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa77fe700 1 -- 172.21.15.60:0/806775460 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 msgr2=0x7fbfa80cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:39.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa77fe700 1 --2- 172.21.15.60:0/806775460 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 0x7fbfa80cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa77fe700 1 -- 172.21.15.60:0/806775460 >> v1:172.21.15.60:6789/0 conn(0x7fbfa80d0f50 legacy=0x7fbfa80d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.492+0000 7fbfa77fe700 1 -- 172.21.15.60:0/806775460 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa80b5380 con 0x7fbfa80d0b80 2023-07-13T22:28:39.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.493+0000 7fbfa77fe700 1 --2- 172.21.15.60:0/806775460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0b80 0x7fbfa80b5ad0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009840 tx=0x7fbf98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd95fa230c5d486c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:39.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.493+0000 7fbfa67fc700 1 -- 172.21.15.60:0/806775460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98011830 con 0x7fbfa80d0b80 2023-07-13T22:28:39.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.493+0000 7fbfa67fc700 1 -- 172.21.15.60:0/806775460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf98011990 con 0x7fbfa80d0b80 2023-07-13T22:28:39.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.493+0000 7fbfa67fc700 1 -- 172.21.15.60:0/806775460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98011d00 con 0x7fbfa80d0b80 2023-07-13T22:28:39.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.494+0000 7fbfadbff700 1 -- 172.21.15.60:0/806775460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0b80 msgr2=0x7fbfa80b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.494+0000 7fbfadbff700 1 --2- 172.21.15.60:0/806775460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0b80 0x7fbfa80b5ad0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009840 tx=0x7fbf98009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:39.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.494+0000 7fbfadbff700 1 -- 172.21.15.60:0/806775460 shutdown_connections 2023-07-13T22:28:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.494+0000 7fbfadbff700 1 --2- 172.21.15.60:0/806775460 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 0x7fbfa80cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.494+0000 7fbfadbff700 1 --2- 172.21.15.60:0/806775460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0b80 0x7fbfa80b5ad0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.494+0000 7fbfadbff700 1 -- 172.21.15.60:0/806775460 >> 172.21.15.60:0/806775460 conn(0x7fbfa8071660 msgr2=0x7fbfa8071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.494+0000 7fbfadbff700 1 -- 172.21.15.60:0/806775460 shutdown_connections 2023-07-13T22:28:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.494+0000 7fbfadbff700 1 -- 172.21.15.60:0/806775460 wait complete. 2023-07-13T22:28:39.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.495+0000 7fbfadbff700 1 Processor -- start 2023-07-13T22:28:39.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.496+0000 7fbfadbff700 1 -- start start 2023-07-13T22:28:39.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.496+0000 7fbfadbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 0x7fbfa81561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.496+0000 7fbfadbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa80d0b80 0x7fbfa8156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfadbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0f50 0x7fbfa80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfadbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa8158350 con 0x7fbfa80d0f50 2023-07-13T22:28:39.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfadbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa8157470 con 0x7fbfa80d0b80 2023-07-13T22:28:39.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfadbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa8157750 con 0x7fbfa80cf8e0 2023-07-13T22:28:39.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0f50 0x7fbfa80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0f50 0x7fbfa80b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39632/0 (socket says 172.21.15.60:39632) 2023-07-13T22:28:39.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa7fff700 1 -- 172.21.15.60:0/1795925371 learned_addr learned my addr 172.21.15.60:0/1795925371 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:39.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa77fe700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 0x7fbfa81561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa6ffd700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa80d0b80 0x7fbfa8156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa7fff700 1 -- 172.21.15.60:0/1795925371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 msgr2=0x7fbfa81561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa7fff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 0x7fbfa81561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa7fff700 1 -- 172.21.15.60:0/1795925371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa80d0b80 msgr2=0x7fbfa8156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa7fff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa80d0b80 0x7fbfa8156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.497+0000 7fbfa7fff700 1 -- 172.21.15.60:0/1795925371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf980094a0 con 0x7fbfa80d0f50 2023-07-13T22:28:39.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.498+0000 7fbfa7fff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0f50 0x7fbfa80b2d50 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0007e60 tx=0x7fbfa000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:39.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.498+0000 7fbf8ffff700 1 -- 172.21.15.60:0/1795925371 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000ed30 con 0x7fbfa80d0f50 2023-07-13T22:28:39.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.498+0000 7fbf8ffff700 1 -- 172.21.15.60:0/1795925371 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa0014420 con 0x7fbfa80d0f50 2023-07-13T22:28:39.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.498+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa81585c0 con 0x7fbfa80d0f50 2023-07-13T22:28:39.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.498+0000 7fbf8ffff700 1 -- 172.21.15.60:0/1795925371 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa00147d0 con 0x7fbfa80d0f50 2023-07-13T22:28:39.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.498+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa81e4ab0 con 0x7fbfa80d0f50 2023-07-13T22:28:39.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.499+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa80741e0 con 0x7fbfa80d0f50 2023-07-13T22:28:39.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.500+0000 7fbf8ffff700 1 -- 172.21.15.60:0/1795925371 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbfa0014930 con 0x7fbfa80d0f50 2023-07-13T22:28:39.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.501+0000 7fbf8ffff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf9406ace0 0x7fbf9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.501+0000 7fbf8ffff700 1 -- 172.21.15.60:0/1795925371 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfa0089b10 con 0x7fbfa80d0f50 2023-07-13T22:28:39.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.501+0000 7fbfa77fe700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf9406ace0 0x7fbf9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.502+0000 7fbfa77fe700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf9406ace0 0x7fbf9406d190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009fd0 tx=0x7fbf98008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:39.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.505+0000 7fbf8ffff700 1 -- 172.21.15.60:0/1795925371 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbfa0054c60 con 0x7fbfa80d0f50 2023-07-13T22:28:39.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.714+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfa80d3670 con 0x7fbfa80d0f50 2023-07-13T22:28:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.716+0000 7fbf8ffff700 1 -- 172.21.15.60:0/1795925371 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18200 (secure 0 0 0) 0x7fbfa0058280 con 0x7fbfa80d0f50 2023-07-13T22:28:39.695 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:39.695 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:06.113196+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:02.360339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:03.609373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf9406ace0 msgr2=0x7fbf9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf9406ace0 0x7fbf9406d190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009fd0 tx=0x7fbf98008030 comp rx=0 tx=0).stop 2023-07-13T22:28:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0f50 msgr2=0x7fbfa80b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0f50 0x7fbfa80b2d50 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0007e60 tx=0x7fbfa000e3c0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 shutdown_connections 2023-07-13T22:28:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbf9406ace0 0x7fbf9406d190 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfa80d0f50 0x7fbfa80b2d50 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfa80cf8e0 0x7fbfa81561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 --2- 172.21.15.60:0/1795925371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa80d0b80 0x7fbfa8156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 >> 172.21.15.60:0/1795925371 conn(0x7fbfa8071660 msgr2=0x7fbfa8071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:39.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 shutdown_connections 2023-07-13T22:28:39.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.718+0000 7fbfadbff700 1 -- 172.21.15.60:0/1795925371 wait complete. 2023-07-13T22:28:39.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:28:02.360339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:28:03.609373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:28:39.716 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:28:39.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.944+0000 7f52cfdff700 1 Processor -- start 2023-07-13T22:28:39.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.944+0000 7f52cfdff700 1 -- start start 2023-07-13T22:28:39.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.945+0000 7f52cfdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f52c815c620 0x7f52c8160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.945+0000 7f52cfdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f52c80cf280 0x7f52c80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.945+0000 7f52cfdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c80b4c80 0x7f52c806be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.945+0000 7f52cfdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c80b5120 con 0x7f52c80cf280 2023-07-13T22:28:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.945+0000 7f52cfdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c8161020 con 0x7f52c815c620 2023-07-13T22:28:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.945+0000 7f52cfdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80cd310 con 0x7f52c80b4c80 2023-07-13T22:28:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.945+0000 7f52cd39a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f52c80cf280 0x7f52c80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.945+0000 7f52cd39a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f52c80cf280 0x7f52c80b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39648/0 (socket says 172.21.15.60:39648) 2023-07-13T22:28:39.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.945+0000 7f52cd39a700 1 -- 172.21.15.60:0/767507827 learned_addr learned my addr 172.21.15.60:0/767507827 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:39.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.946+0000 7f52cd39a700 1 -- 172.21.15.60:0/767507827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c80b4c80 msgr2=0x7f52c806be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:39.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.946+0000 7f52cd39a700 1 --2- 172.21.15.60:0/767507827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c80b4c80 0x7f52c806be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.946+0000 7f52cd39a700 1 -- 172.21.15.60:0/767507827 >> v2:172.21.15.105:3300/0 conn(0x7f52c815c620 msgr2=0x7f52c8160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:39.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.946+0000 7f52cd39a700 1 --2- 172.21.15.60:0/767507827 >> v2:172.21.15.105:3300/0 conn(0x7f52c815c620 0x7f52c8160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.946+0000 7f52cd39a700 1 -- 172.21.15.60:0/767507827 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c815c9f0 con 0x7f52c80cf280 2023-07-13T22:28:39.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.946+0000 7f52cd39a700 1 --2- 172.21.15.60:0/767507827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80cf280 0x7f52c80b4740 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f52c8072640 tx=0x7f52b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32c6e3f8746286d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:39.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.946+0000 7f52ccb99700 1 -- 172.21.15.60:0/767507827 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b800b870 con 0x7f52c80cf280 2023-07-13T22:28:39.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.946+0000 7f52ccb99700 1 -- 172.21.15.60:0/767507827 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b800b9d0 con 0x7f52c80cf280 2023-07-13T22:28:39.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.946+0000 7f52ccb99700 1 -- 172.21.15.60:0/767507827 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b800bd60 con 0x7f52c80cf280 2023-07-13T22:28:39.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.947+0000 7f52cfdff700 1 -- 172.21.15.60:0/767507827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80cf280 msgr2=0x7f52c80b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.947+0000 7f52cfdff700 1 --2- 172.21.15.60:0/767507827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80cf280 0x7f52c80b4740 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f52c8072640 tx=0x7f52b8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.947+0000 7f52cfdff700 1 -- 172.21.15.60:0/767507827 shutdown_connections 2023-07-13T22:28:39.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.947+0000 7f52cfdff700 1 --2- 172.21.15.60:0/767507827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c80b4c80 0x7f52c806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.947+0000 7f52cfdff700 1 --2- 172.21.15.60:0/767507827 >> v2:172.21.15.105:3300/0 conn(0x7f52c815c620 0x7f52c8160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.947+0000 7f52cfdff700 1 --2- 172.21.15.60:0/767507827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80cf280 0x7f52c80b4740 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.947+0000 7f52cfdff700 1 -- 172.21.15.60:0/767507827 >> 172.21.15.60:0/767507827 conn(0x7f52c806b210 msgr2=0x7f52c80cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:39.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.947+0000 7f52cfdff700 1 -- 172.21.15.60:0/767507827 shutdown_connections 2023-07-13T22:28:39.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.947+0000 7f52cfdff700 1 -- 172.21.15.60:0/767507827 wait complete. 2023-07-13T22:28:39.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.949+0000 7f52cfdff700 1 Processor -- start 2023-07-13T22:28:39.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.949+0000 7f52cfdff700 1 -- start start 2023-07-13T22:28:39.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cfdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80b4c80 0x7f52c81592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cfdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c80cf280 0x7f52c80c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cfdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c815c620 0x7f52c815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cfdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c81f3520 con 0x7f52c80b4c80 2023-07-13T22:28:39.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cfdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80c7be0 con 0x7f52c80cf280 2023-07-13T22:28:39.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cfdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80c7ec0 con 0x7f52c815c620 2023-07-13T22:28:39.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cdb9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80b4c80 0x7f52c81592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cdb9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80b4c80 0x7f52c81592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39660/0 (socket says 172.21.15.60:39660) 2023-07-13T22:28:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cdb9b700 1 -- 172.21.15.60:0/851121168 learned_addr learned my addr 172.21.15.60:0/851121168 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cd39a700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c80cf280 0x7f52c80c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52ce39c700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c815c620 0x7f52c815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cdb9b700 1 -- 172.21.15.60:0/851121168 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c815c620 msgr2=0x7f52c815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cdb9b700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c815c620 0x7f52c815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cdb9b700 1 -- 172.21.15.60:0/851121168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c80cf280 msgr2=0x7f52c80c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:39.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cdb9b700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c80cf280 0x7f52c80c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:39.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cdb9b700 1 -- 172.21.15.60:0/851121168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b8009550 con 0x7f52c80b4c80 2023-07-13T22:28:39.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.950+0000 7f52cd39a700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c80cf280 0x7f52c80c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:39.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.951+0000 7f52cdb9b700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80b4c80 0x7f52c81592c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f52c400b530 tx=0x7f52c400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:39.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.951+0000 7f52be7fc700 1 -- 172.21.15.60:0/851121168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4011830 con 0x7f52c80b4c80 2023-07-13T22:28:39.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.951+0000 7f52be7fc700 1 -- 172.21.15.60:0/851121168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c40119b0 con 0x7f52c80b4c80 2023-07-13T22:28:39.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.951+0000 7f52be7fc700 1 -- 172.21.15.60:0/851121168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4011d60 con 0x7f52c80b4c80 2023-07-13T22:28:39.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.951+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c80c81a0 con 0x7f52c80b4c80 2023-07-13T22:28:39.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.951+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c80c77b0 con 0x7f52c80b4c80 2023-07-13T22:28:39.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.952+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c80c79a0 con 0x7f52c80b4c80 2023-07-13T22:28:39.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.953+0000 7f52be7fc700 1 -- 172.21.15.60:0/851121168 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52c401a400 con 0x7f52c80b4c80 2023-07-13T22:28:39.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.954+0000 7f52be7fc700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52ac06ace0 0x7f52ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:39.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.954+0000 7f52be7fc700 1 -- 172.21.15.60:0/851121168 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52c40895e0 con 0x7f52c80b4c80 2023-07-13T22:28:39.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.954+0000 7f52cd39a700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52ac06ace0 0x7f52ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:39.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.955+0000 7f52cd39a700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52ac06ace0 0x7f52ac06d190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f52b8000c00 tx=0x7f52b8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:39.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:39.958+0000 7f52be7fc700 1 -- 172.21.15.60:0/851121168 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f52c80c79a0 con 0x7f52c80b4c80 2023-07-13T22:28:40.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.163+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f52c80c79a0 con 0x7f52c80b4c80 2023-07-13T22:28:40.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.164+0000 7f52be7fc700 1 -- 172.21.15.60:0/851121168 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7f52c80c79a0 con 0x7f52c80b4c80 2023-07-13T22:28:40.143 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:28:40.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52ac06ace0 msgr2=0x7f52ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52ac06ace0 0x7f52ac06d190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f52b8000c00 tx=0x7f52b8008030 comp rx=0 tx=0).stop 2023-07-13T22:28:40.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80b4c80 msgr2=0x7f52c81592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80b4c80 0x7f52c81592c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f52c400b530 tx=0x7f52c400b8f0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 shutdown_connections 2023-07-13T22:28:40.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f52ac06ace0 0x7f52ac06d190 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52c80b4c80 0x7f52c81592c0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52c815c620 0x7f52c815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 --2- 172.21.15.60:0/851121168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52c80cf280 0x7f52c80c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 >> 172.21.15.60:0/851121168 conn(0x7f52c806b210 msgr2=0x7f52c8160850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:40.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 shutdown_connections 2023-07-13T22:28:40.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.166+0000 7f52cfdff700 1 -- 172.21.15.60:0/851121168 wait complete. 2023-07-13T22:28:40.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:28:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.390+0000 7fd9d8bff700 1 Processor -- start 2023-07-13T22:28:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.390+0000 7fd9d8bff700 1 -- start start 2023-07-13T22:28:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.391+0000 7fd9d8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40b4e50 0x7fd9d4160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.391+0000 7fd9d8bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd9d40cf280 0x7fd9d40d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.391+0000 7fd9d8bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d4161020 con 0x7fd9d415c620 2023-07-13T22:28:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.391+0000 7fd9d8bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40cd310 con 0x7fd9d40cf280 2023-07-13T22:28:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.391+0000 7fd9d8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415c9f0 con 0x7fd9d40b4e50 2023-07-13T22:28:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d2d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd9d40cf280 0x7fd9d40d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d2d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd9d40cf280 0x7fd9d40d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39674/0 (socket says 172.21.15.60:39674) 2023-07-13T22:28:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d2d9d700 1 -- 172.21.15.60:0/135710279 learned_addr learned my addr 172.21.15.60:0/135710279 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d1d9b700 1 --2- 172.21.15.60:0/135710279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40b4e50 0x7fd9d4160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d2d9d700 1 -- 172.21.15.60:0/135710279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40b4e50 msgr2=0x7fd9d4160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d2d9d700 1 --2- 172.21.15.60:0/135710279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40b4e50 0x7fd9d4160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d2d9d700 1 -- 172.21.15.60:0/135710279 >> v1:172.21.15.105:6789/0 conn(0x7fd9d415c620 legacy=0x7fd9d40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d2d9d700 1 -- 172.21.15.60:0/135710279 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40cd560 con 0x7fd9d40cf280 2023-07-13T22:28:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d159a700 1 -- 172.21.15.60:0/135710279 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635468846 0 0) 0x7fd9c4001110 con 0x7fd9d415c620 2023-07-13T22:28:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.392+0000 7fd9d2d9d700 1 --2- 172.21.15.60:0/135710279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40cf280 0x7fd9d40d11c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009e50 tx=0x7fd9cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ef46b096577603 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d159a700 1 -- 172.21.15.60:0/135710279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00e070 con 0x7fd9d40cf280 2023-07-13T22:28:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d159a700 1 -- 172.21.15.60:0/135710279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc00b910 con 0x7fd9d40cf280 2023-07-13T22:28:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d159a700 1 -- 172.21.15.60:0/135710279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00bcc0 con 0x7fd9d40cf280 2023-07-13T22:28:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d8bff700 1 -- 172.21.15.60:0/135710279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40cf280 msgr2=0x7fd9d40d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d8bff700 1 --2- 172.21.15.60:0/135710279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40cf280 0x7fd9d40d11c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009e50 tx=0x7fd9cc00b330 comp rx=0 tx=0).stop 2023-07-13T22:28:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d8bff700 1 -- 172.21.15.60:0/135710279 shutdown_connections 2023-07-13T22:28:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d8bff700 1 --2- 172.21.15.60:0/135710279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40cf280 0x7fd9d40d11c0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d8bff700 1 --2- 172.21.15.60:0/135710279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40b4e50 0x7fd9d4160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d8bff700 1 -- 172.21.15.60:0/135710279 >> 172.21.15.60:0/135710279 conn(0x7fd9d406b210 msgr2=0x7fd9d40cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d8bff700 1 -- 172.21.15.60:0/135710279 shutdown_connections 2023-07-13T22:28:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.393+0000 7fd9d8bff700 1 -- 172.21.15.60:0/135710279 wait complete. 2023-07-13T22:28:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.395+0000 7fd9d8bff700 1 Processor -- start 2023-07-13T22:28:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.395+0000 7fd9d8bff700 1 -- start start 2023-07-13T22:28:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.396+0000 7fd9d8bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40b4e50 0x7fd9d415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.396+0000 7fd9d8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40cf280 0x7fd9d40c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.396+0000 7fd9d8bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d415c620 0x7fd9d41f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.396+0000 7fd9d8bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40c6cf0 con 0x7fd9d40b4e50 2023-07-13T22:28:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.396+0000 7fd9d8bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40c6fd0 con 0x7fd9d415c620 2023-07-13T22:28:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.396+0000 7fd9d8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40c72b0 con 0x7fd9d40cf280 2023-07-13T22:28:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.396+0000 7fd9d259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40b4e50 0x7fd9d415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.396+0000 7fd9d259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40b4e50 0x7fd9d415dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39682/0 (socket says 172.21.15.60:39682) 2023-07-13T22:28:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.396+0000 7fd9d259c700 1 -- 172.21.15.60:0/2088478110 learned_addr learned my addr 172.21.15.60:0/2088478110 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9d1d9b700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40cf280 0x7fd9d40c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9d2d9d700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d415c620 0x7fd9d41f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9d259c700 1 -- 172.21.15.60:0/2088478110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40cf280 msgr2=0x7fd9d40c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9d259c700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40cf280 0x7fd9d40c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9d259c700 1 -- 172.21.15.60:0/2088478110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d415c620 msgr2=0x7fd9d41f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9d259c700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d415c620 0x7fd9d41f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9d259c700 1 -- 172.21.15.60:0/2088478110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc009b30 con 0x7fd9d40b4e50 2023-07-13T22:28:40.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9d259c700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40b4e50 0x7fd9d415dfe0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c750 tx=0x7fd9c400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:40.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9c2ffd700 1 -- 172.21.15.60:0/2088478110 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40054e0 con 0x7fd9d40b4e50 2023-07-13T22:28:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9c2ffd700 1 -- 172.21.15.60:0/2088478110 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4003680 con 0x7fd9d40b4e50 2023-07-13T22:28:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d40c7590 con 0x7fd9d40b4e50 2023-07-13T22:28:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.397+0000 7fd9c2ffd700 1 -- 172.21.15.60:0/2088478110 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4003a30 con 0x7fd9d40b4e50 2023-07-13T22:28:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.398+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d415e520 con 0x7fd9d40b4e50 2023-07-13T22:28:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.399+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d40741e0 con 0x7fd9d40b4e50 2023-07-13T22:28:40.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.400+0000 7fd9c2ffd700 1 -- 172.21.15.60:0/2088478110 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd9c4003b90 con 0x7fd9d40b4e50 2023-07-13T22:28:40.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.401+0000 7fd9c2ffd700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9b406ace0 0x7fd9b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.401+0000 7fd9c2ffd700 1 -- 172.21.15.60:0/2088478110 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd9c40887e0 con 0x7fd9d40b4e50 2023-07-13T22:28:40.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.401+0000 7fd9d1d9b700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9b406ace0 0x7fd9b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.402+0000 7fd9d1d9b700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9b406ace0 0x7fd9b406d190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40c8490 tx=0x7fd9c8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:40.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.405+0000 7fd9c2ffd700 1 -- 172.21.15.60:0/2088478110 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd9c4057280 con 0x7fd9d40b4e50 2023-07-13T22:28:40.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.613+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9d4161020 con 0x7fd9d40b4e50 2023-07-13T22:28:40.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.613+0000 7fd9c2ffd700 1 -- 172.21.15.60:0/2088478110 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9d4161020 con 0x7fd9d40b4e50 2023-07-13T22:28:40.593 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:28:40.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9b406ace0 msgr2=0x7fd9b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9b406ace0 0x7fd9b406d190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40c8490 tx=0x7fd9c8009210 comp rx=0 tx=0).stop 2023-07-13T22:28:40.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40b4e50 msgr2=0x7fd9d415dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40b4e50 0x7fd9d415dfe0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c750 tx=0x7fd9c400ca60 comp rx=0 tx=0).stop 2023-07-13T22:28:40.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 shutdown_connections 2023-07-13T22:28:40.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9b406ace0 0x7fd9b406d190 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d40b4e50 0x7fd9d415dfe0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d415c620 0x7fd9d41f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 --2- 172.21.15.60:0/2088478110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9d40cf280 0x7fd9d40c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 >> 172.21.15.60:0/2088478110 conn(0x7fd9d406b210 msgr2=0x7fd9d406c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:40.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 shutdown_connections 2023-07-13T22:28:40.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.616+0000 7fd9d8bff700 1 -- 172.21.15.60:0/2088478110 wait complete. 2023-07-13T22:28:40.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:28:40.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.839+0000 7f2fc7bff700 1 Processor -- start 2023-07-13T22:28:40.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.839+0000 7f2fc7bff700 1 -- start start 2023-07-13T22:28:40.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc7bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2fc00cf280 0x7f2fc0160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc7bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc0161020 con 0x7f2fc015c620 2023-07-13T22:28:40.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc7bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc00cd310 con 0x7f2fc00b4e50 2023-07-13T22:28:40.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc7bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc015c9f0 con 0x7f2fc00cf280 2023-07-13T22:28:40.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc619c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2fc00cf280 0x7f2fc0160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc599b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2fc015c620 0x7f2fc00b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60416/0 (socket says 172.21.15.60:60416) 2023-07-13T22:28:40.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc599b700 1 -- 172.21.15.60:0/1022386594 learned_addr learned my addr 172.21.15.60:0/1022386594 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc619c700 1 -- 172.21.15.60:0/1022386594 >> v1:172.21.15.105:6789/0 conn(0x7f2fc00b4e50 legacy=0x7f2fc00d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc619c700 1 -- 172.21.15.60:0/1022386594 >> v1:172.21.15.60:6789/0 conn(0x7f2fc015c620 legacy=0x7f2fc00b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.841+0000 7f2fc619c700 1 -- 172.21.15.60:0/1022386594 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00cd560 con 0x7f2fc00cf280 2023-07-13T22:28:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.842+0000 7f2fc619c700 1 --2- 172.21.15.60:0/1022386594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc00cf280 0x7f2fc0160ae0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009d90 tx=0x7f2fb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc854fa3f653086 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.842+0000 7f2fc4999700 1 -- 172.21.15.60:0/1022386594 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb800e070 con 0x7f2fc00cf280 2023-07-13T22:28:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.842+0000 7f2fc4999700 1 -- 172.21.15.60:0/1022386594 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb800bbd0 con 0x7f2fc00cf280 2023-07-13T22:28:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.842+0000 7f2fc4999700 1 -- 172.21.15.60:0/1022386594 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8012430 con 0x7f2fc00cf280 2023-07-13T22:28:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.842+0000 7f2fc7bff700 1 -- 172.21.15.60:0/1022386594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc00cf280 msgr2=0x7f2fc0160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.842+0000 7f2fc7bff700 1 --2- 172.21.15.60:0/1022386594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc00cf280 0x7f2fc0160ae0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009d90 tx=0x7f2fb800b330 comp rx=0 tx=0).stop 2023-07-13T22:28:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.842+0000 7f2fc7bff700 1 -- 172.21.15.60:0/1022386594 shutdown_connections 2023-07-13T22:28:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.842+0000 7f2fc7bff700 1 --2- 172.21.15.60:0/1022386594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc00cf280 0x7f2fc0160ae0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.842+0000 7f2fc7bff700 1 -- 172.21.15.60:0/1022386594 >> 172.21.15.60:0/1022386594 conn(0x7f2fc006b210 msgr2=0x7f2fc00cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.843+0000 7f2fc7bff700 1 -- 172.21.15.60:0/1022386594 shutdown_connections 2023-07-13T22:28:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.843+0000 7f2fc7bff700 1 -- 172.21.15.60:0/1022386594 wait complete. 2023-07-13T22:28:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.845+0000 7f2fc7bff700 1 Processor -- start 2023-07-13T22:28:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.845+0000 7f2fc7bff700 1 -- start start 2023-07-13T22:28:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.846+0000 7f2fc7bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fc00b4e50 0x7f2fc00c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.846+0000 7f2fc7bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc00cf280 0x7f2fc015f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.846+0000 7f2fc7bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc015c620 0x7f2fc00c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc7bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0150850 con 0x7f2fc015c620 2023-07-13T22:28:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc7bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00b7f20 con 0x7f2fc00cf280 2023-07-13T22:28:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc7bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00c9490 con 0x7f2fc00b4e50 2023-07-13T22:28:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc015c620 0x7f2fc00c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc015c620 0x7f2fc00c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39692/0 (socket says 172.21.15.60:39692) 2023-07-13T22:28:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc619c700 1 -- 172.21.15.60:0/501696624 learned_addr learned my addr 172.21.15.60:0/501696624 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc599b700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fc00b4e50 0x7f2fc00c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc519a700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc00cf280 0x7f2fc015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc619c700 1 -- 172.21.15.60:0/501696624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fc00b4e50 msgr2=0x7f2fc00c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc619c700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fc00b4e50 0x7f2fc00c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc619c700 1 -- 172.21.15.60:0/501696624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc00cf280 msgr2=0x7f2fc015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:40.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc619c700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc00cf280 0x7f2fc015f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:40.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.847+0000 7f2fc619c700 1 -- 172.21.15.60:0/501696624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8009a70 con 0x7f2fc015c620 2023-07-13T22:28:40.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.848+0000 7f2fc619c700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc015c620 0x7f2fc00c67b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b370 tx=0x7f2fb8004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:40.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.848+0000 7f2fb67fc700 1 -- 172.21.15.60:0/501696624 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8012430 con 0x7f2fc015c620 2023-07-13T22:28:40.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.848+0000 7f2fb67fc700 1 -- 172.21.15.60:0/501696624 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb8020070 con 0x7f2fc015c620 2023-07-13T22:28:40.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.848+0000 7f2fb67fc700 1 -- 172.21.15.60:0/501696624 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80098f0 con 0x7f2fc015c620 2023-07-13T22:28:40.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.848+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc00c9760 con 0x7f2fc015c620 2023-07-13T22:28:40.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.848+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc00b78d0 con 0x7f2fc015c620 2023-07-13T22:28:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.849+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc00741e0 con 0x7f2fc015c620 2023-07-13T22:28:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.850+0000 7f2fb67fc700 1 -- 172.21.15.60:0/501696624 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2fb8004040 con 0x7f2fc015c620 2023-07-13T22:28:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.851+0000 7f2fb67fc700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fa406ace0 0x7f2fa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.851+0000 7f2fb67fc700 1 -- 172.21.15.60:0/501696624 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2fb8022070 con 0x7f2fc015c620 2023-07-13T22:28:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.851+0000 7f2fc599b700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fa406ace0 0x7f2fa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.852+0000 7f2fc599b700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fa406ace0 0x7f2fa406d190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc005910 tx=0x7f2fbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:40.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:40.855+0000 7f2fb67fc700 1 -- 172.21.15.60:0/501696624 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2fb8056340 con 0x7f2fc015c620 2023-07-13T22:28:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.059+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fc00d00f0 con 0x7f2fc015c620 2023-07-13T22:28:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.060+0000 7f2fb67fc700 1 -- 172.21.15.60:0/501696624 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fb802c020 con 0x7f2fc015c620 2023-07-13T22:28:41.039 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:28:41.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.062+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fa406ace0 msgr2=0x7f2fa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.062+0000 7f2fc7bff700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fa406ace0 0x7f2fa406d190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc005910 tx=0x7f2fbc009040 comp rx=0 tx=0).stop 2023-07-13T22:28:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.062+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc015c620 msgr2=0x7f2fc00c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.062+0000 7f2fc7bff700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc015c620 0x7f2fc00c67b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b370 tx=0x7f2fb8004b20 comp rx=0 tx=0).stop 2023-07-13T22:28:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.062+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 shutdown_connections 2023-07-13T22:28:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.062+0000 7f2fc7bff700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2fa406ace0 0x7f2fa406d190 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.063+0000 7f2fc7bff700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fc015c620 0x7f2fc00c67b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.063+0000 7f2fc7bff700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2fc00b4e50 0x7f2fc00c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.063+0000 7f2fc7bff700 1 --2- 172.21.15.60:0/501696624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fc00cf280 0x7f2fc015f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.063+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 >> 172.21.15.60:0/501696624 conn(0x7f2fc006b210 msgr2=0x7f2fc00d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:41.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.063+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 shutdown_connections 2023-07-13T22:28:41.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.063+0000 7f2fc7bff700 1 -- 172.21.15.60:0/501696624 wait complete. 2023-07-13T22:28:41.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:28:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.289+0000 7fcdccbff700 1 Processor -- start 2023-07-13T22:28:41.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.290+0000 7fcdccbff700 1 -- start start 2023-07-13T22:28:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdccbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcdc815c620 0x7fcdc8160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdccbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80cf280 0x7fcdc80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdccbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdc80b4c80 0x7fcdc806be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdccbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc80b50e0 con 0x7fcdc80b4c80 2023-07-13T22:28:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdccbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc8161020 con 0x7fcdc815c620 2023-07-13T22:28:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdccbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80cd310 con 0x7fcdc80cf280 2023-07-13T22:28:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdc6d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdc80b4c80 0x7fcdc806be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdc6d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdc80b4c80 0x7fcdc806be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39704/0 (socket says 172.21.15.60:39704) 2023-07-13T22:28:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdc6d9d700 1 -- 172.21.15.60:0/1066387464 learned_addr learned my addr 172.21.15.60:0/1066387464 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdc5d9b700 1 --2- 172.21.15.60:0/1066387464 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80cf280 0x7fcdc80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.291+0000 7fcdc659c700 1 --2- 172.21.15.60:0/1066387464 >> v2:172.21.15.105:3300/0 conn(0x7fcdc815c620 0x7fcdc8160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.292+0000 7fcdc6d9d700 1 -- 172.21.15.60:0/1066387464 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80cf280 msgr2=0x7fcdc80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.292+0000 7fcdc6d9d700 1 --2- 172.21.15.60:0/1066387464 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80cf280 0x7fcdc80b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.292+0000 7fcdc6d9d700 1 -- 172.21.15.60:0/1066387464 >> v2:172.21.15.105:3300/0 conn(0x7fcdc815c620 msgr2=0x7fcdc8160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.292+0000 7fcdc6d9d700 1 --2- 172.21.15.60:0/1066387464 >> v2:172.21.15.105:3300/0 conn(0x7fcdc815c620 0x7fcdc8160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.292+0000 7fcdc6d9d700 1 -- 172.21.15.60:0/1066387464 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc815c9f0 con 0x7fcdc80b4c80 2023-07-13T22:28:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.292+0000 7fcdc6d9d700 1 --2- 172.21.15.60:0/1066387464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80b4c80 0x7fcdc806be90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8072640 tx=0x7fcdc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40159a5d943dbfdc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.292+0000 7fcdc559a700 1 -- 172.21.15.60:0/1066387464 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc000e070 con 0x7fcdc80b4c80 2023-07-13T22:28:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.292+0000 7fcdc559a700 1 -- 172.21.15.60:0/1066387464 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc000b9c0 con 0x7fcdc80b4c80 2023-07-13T22:28:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.292+0000 7fcdc559a700 1 -- 172.21.15.60:0/1066387464 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc000bd70 con 0x7fcdc80b4c80 2023-07-13T22:28:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.293+0000 7fcdccbff700 1 -- 172.21.15.60:0/1066387464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80b4c80 msgr2=0x7fcdc806be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.293+0000 7fcdccbff700 1 --2- 172.21.15.60:0/1066387464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80b4c80 0x7fcdc806be90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8072640 tx=0x7fcdc000b330 comp rx=0 tx=0).stop 2023-07-13T22:28:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.293+0000 7fcdccbff700 1 -- 172.21.15.60:0/1066387464 shutdown_connections 2023-07-13T22:28:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.293+0000 7fcdccbff700 1 --2- 172.21.15.60:0/1066387464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80b4c80 0x7fcdc806be90 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.293+0000 7fcdccbff700 1 --2- 172.21.15.60:0/1066387464 >> v2:172.21.15.105:3300/0 conn(0x7fcdc815c620 0x7fcdc8160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.293+0000 7fcdccbff700 1 --2- 172.21.15.60:0/1066387464 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80cf280 0x7fcdc80b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.293+0000 7fcdccbff700 1 -- 172.21.15.60:0/1066387464 >> 172.21.15.60:0/1066387464 conn(0x7fcdc806b210 msgr2=0x7fcdc80cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.293+0000 7fcdccbff700 1 -- 172.21.15.60:0/1066387464 shutdown_connections 2023-07-13T22:28:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.293+0000 7fcdccbff700 1 -- 172.21.15.60:0/1066387464 wait complete. 2023-07-13T22:28:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.295+0000 7fcdccbff700 1 Processor -- start 2023-07-13T22:28:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.295+0000 7fcdccbff700 1 -- start start 2023-07-13T22:28:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdccbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80b4c80 0x7fcdc80c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdccbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80cf280 0x7fcdc8156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdccbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc815c620 0x7fcdc815d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdccbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80cb0e0 con 0x7fcdc80cf280 2023-07-13T22:28:41.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdccbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc815faf0 con 0x7fcdc815c620 2023-07-13T22:28:41.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdccbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc815f810 con 0x7fcdc80b4c80 2023-07-13T22:28:41.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdc5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80cf280 0x7fcdc8156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdc5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80cf280 0x7fcdc8156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39712/0 (socket says 172.21.15.60:39712) 2023-07-13T22:28:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdc5d9b700 1 -- 172.21.15.60:0/18910433 learned_addr learned my addr 172.21.15.60:0/18910433 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdc659c700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80b4c80 0x7fcdc80c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.296+0000 7fcdc6d9d700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc815c620 0x7fcdc815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdc5d9b700 1 -- 172.21.15.60:0/18910433 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80b4c80 msgr2=0x7fcdc80c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdc5d9b700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80b4c80 0x7fcdc80c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdc5d9b700 1 -- 172.21.15.60:0/18910433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc815c620 msgr2=0x7fcdc815d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdc5d9b700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc815c620 0x7fcdc815d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdc5d9b700 1 -- 172.21.15.60:0/18910433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc0009b30 con 0x7fcdc80cf280 2023-07-13T22:28:41.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdc6d9d700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc815c620 0x7fcdc815d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:41.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdc5d9b700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80cf280 0x7fcdc8156ee0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00e810 tx=0x7fcdbc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:41.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdb6ffd700 1 -- 172.21.15.60:0/18910433 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc013610 con 0x7fcdc80cf280 2023-07-13T22:28:41.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdb6ffd700 1 -- 172.21.15.60:0/18910433 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc013770 con 0x7fcdc80cf280 2023-07-13T22:28:41.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdb6ffd700 1 -- 172.21.15.60:0/18910433 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc013b20 con 0x7fcdc80cf280 2023-07-13T22:28:41.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc80c8420 con 0x7fcdc80cf280 2023-07-13T22:28:41.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.297+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc80c89e0 con 0x7fcdc80cf280 2023-07-13T22:28:41.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.298+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc80741e0 con 0x7fcdc80cf280 2023-07-13T22:28:41.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.299+0000 7fcdb6ffd700 1 -- 172.21.15.60:0/18910433 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcdbc013c80 con 0x7fcdc80cf280 2023-07-13T22:28:41.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.300+0000 7fcdb6ffd700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcda806ace0 0x7fcda806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.300+0000 7fcdb6ffd700 1 -- 172.21.15.60:0/18910433 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcdbc089100 con 0x7fcdc80cf280 2023-07-13T22:28:41.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.301+0000 7fcdc659c700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcda806ace0 0x7fcda806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.301+0000 7fcdc659c700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcda806ace0 0x7fcda806d190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80b8cf0 tx=0x7fcdb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:41.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.304+0000 7fcdb6ffd700 1 -- 172.21.15.60:0/18910433 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcdbc057bb0 con 0x7fcdc80cf280 2023-07-13T22:28:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.511+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcdc806cf30 con 0x7fcdc80cf280 2023-07-13T22:28:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.511+0000 7fcdb6ffd700 1 -- 172.21.15.60:0/18910433 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcdbc057740 con 0x7fcdc80cf280 2023-07-13T22:28:41.491 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:28:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcda806ace0 msgr2=0x7fcda806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcda806ace0 0x7fcda806d190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80b8cf0 tx=0x7fcdb8008040 comp rx=0 tx=0).stop 2023-07-13T22:28:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80cf280 msgr2=0x7fcdc8156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80cf280 0x7fcdc8156ee0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00e810 tx=0x7fcdbc00eb20 comp rx=0 tx=0).stop 2023-07-13T22:28:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 shutdown_connections 2023-07-13T22:28:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcda806ace0 0x7fcda806d190 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc815c620 0x7fcdc815d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcdc80b4c80 0x7fcdc80c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 --2- 172.21.15.60:0/18910433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80cf280 0x7fcdc8156ee0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.513+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 >> 172.21.15.60:0/18910433 conn(0x7fcdc806b210 msgr2=0x7fcdc806be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.514+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 shutdown_connections 2023-07-13T22:28:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.514+0000 7fcdccbff700 1 -- 172.21.15.60:0/18910433 wait complete. 2023-07-13T22:28:41.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:28:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.735+0000 7fcd4adff700 1 Processor -- start 2023-07-13T22:28:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.735+0000 7fcd4adff700 1 -- start start 2023-07-13T22:28:41.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd4adff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd440d0b80 0x7fcd440b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd4adff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcd440cf8e0 0x7fcd440cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd4adff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd440ceeb0 con 0x7fcd440d0f50 2023-07-13T22:28:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd4adff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd440b23f0 con 0x7fcd440d0b80 2023-07-13T22:28:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd4adff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd440b4bb0 con 0x7fcd440cf8e0 2023-07-13T22:28:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd48b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd440d0b80 0x7fcd440b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd43fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcd440d0f50 0x7fcd440d2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60422/0 (socket says 172.21.15.60:60422) 2023-07-13T22:28:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd43fff700 1 -- 172.21.15.60:0/1542888830 learned_addr learned my addr 172.21.15.60:0/1542888830 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd48b9b700 1 -- 172.21.15.60:0/1542888830 >> v2:172.21.15.105:3300/0 conn(0x7fcd440cf8e0 msgr2=0x7fcd440cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd48b9b700 1 --2- 172.21.15.60:0/1542888830 >> v2:172.21.15.105:3300/0 conn(0x7fcd440cf8e0 0x7fcd440cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd48b9b700 1 -- 172.21.15.60:0/1542888830 >> v1:172.21.15.60:6789/0 conn(0x7fcd440d0f50 legacy=0x7fcd440d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd48b9b700 1 -- 172.21.15.60:0/1542888830 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440b5380 con 0x7fcd440d0b80 2023-07-13T22:28:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.737+0000 7fcd437fe700 1 -- 172.21.15.60:0/1542888830 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513154663 0 0) 0x7fcd440ceeb0 con 0x7fcd440d0f50 2023-07-13T22:28:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.738+0000 7fcd48b9b700 1 --2- 172.21.15.60:0/1542888830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 0x7fcd440b5ad0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcd4406a2d0 tx=0x7fcd38009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769c304c35df60e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.738+0000 7fcd437fe700 1 -- 172.21.15.60:0/1542888830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38011830 con 0x7fcd440d0b80 2023-07-13T22:28:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.738+0000 7fcd437fe700 1 -- 172.21.15.60:0/1542888830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd38011990 con 0x7fcd440d0b80 2023-07-13T22:28:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.738+0000 7fcd437fe700 1 -- 172.21.15.60:0/1542888830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38011d20 con 0x7fcd440d0b80 2023-07-13T22:28:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.738+0000 7fcd4adff700 1 -- 172.21.15.60:0/1542888830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 msgr2=0x7fcd440b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.738+0000 7fcd4adff700 1 --2- 172.21.15.60:0/1542888830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 0x7fcd440b5ad0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcd4406a2d0 tx=0x7fcd38009b20 comp rx=0 tx=0).stop 2023-07-13T22:28:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.738+0000 7fcd4adff700 1 -- 172.21.15.60:0/1542888830 shutdown_connections 2023-07-13T22:28:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.738+0000 7fcd4adff700 1 --2- 172.21.15.60:0/1542888830 >> v2:172.21.15.105:3300/0 conn(0x7fcd440cf8e0 0x7fcd440cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.738+0000 7fcd4adff700 1 --2- 172.21.15.60:0/1542888830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 0x7fcd440b5ad0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.739+0000 7fcd4adff700 1 -- 172.21.15.60:0/1542888830 >> 172.21.15.60:0/1542888830 conn(0x7fcd44071660 msgr2=0x7fcd44071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.739+0000 7fcd4adff700 1 -- 172.21.15.60:0/1542888830 shutdown_connections 2023-07-13T22:28:41.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.739+0000 7fcd4adff700 1 -- 172.21.15.60:0/1542888830 wait complete. 2023-07-13T22:28:41.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.741+0000 7fcd4adff700 1 Processor -- start 2023-07-13T22:28:41.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.741+0000 7fcd4adff700 1 -- start start 2023-07-13T22:28:41.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.742+0000 7fcd4adff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd440cf8e0 0x7fcd440ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.742+0000 7fcd4adff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 0x7fcd440cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.742+0000 7fcd4adff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd440d0f50 0x7fcd4415de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.742+0000 7fcd4adff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd44155590 con 0x7fcd440d0b80 2023-07-13T22:28:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.742+0000 7fcd4adff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440cc500 con 0x7fcd440cf8e0 2023-07-13T22:28:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd4adff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440cc750 con 0x7fcd440d0f50 2023-07-13T22:28:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd43fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 0x7fcd440cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd43fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 0x7fcd440cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39734/0 (socket says 172.21.15.60:39734) 2023-07-13T22:28:41.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd43fff700 1 -- 172.21.15.60:0/2034127926 learned_addr learned my addr 172.21.15.60:0/2034127926 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd48b9b700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd440cf8e0 0x7fcd440ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd4939c700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd440d0f50 0x7fcd4415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd43fff700 1 -- 172.21.15.60:0/2034127926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd440d0f50 msgr2=0x7fcd4415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd43fff700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd440d0f50 0x7fcd4415de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd43fff700 1 -- 172.21.15.60:0/2034127926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd440cf8e0 msgr2=0x7fcd440ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd43fff700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd440cf8e0 0x7fcd440ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.743+0000 7fcd43fff700 1 -- 172.21.15.60:0/2034127926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd380094a0 con 0x7fcd440d0b80 2023-07-13T22:28:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.744+0000 7fcd43fff700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 0x7fcd440cbfc0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcd300011e0 tx=0x7fcd3000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:41.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.744+0000 7fcd417fa700 1 -- 172.21.15.60:0/2034127926 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30004c80 con 0x7fcd440d0b80 2023-07-13T22:28:41.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.744+0000 7fcd417fa700 1 -- 172.21.15.60:0/2034127926 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd30004e00 con 0x7fcd440d0b80 2023-07-13T22:28:41.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.744+0000 7fcd417fa700 1 -- 172.21.15.60:0/2034127926 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30007310 con 0x7fcd440d0b80 2023-07-13T22:28:41.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.744+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd440cca30 con 0x7fcd440d0b80 2023-07-13T22:28:41.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.744+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd440ccff0 con 0x7fcd440d0b80 2023-07-13T22:28:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.745+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd440d36a0 con 0x7fcd440d0b80 2023-07-13T22:28:41.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.746+0000 7fcd417fa700 1 -- 172.21.15.60:0/2034127926 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd30007470 con 0x7fcd440d0b80 2023-07-13T22:28:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.747+0000 7fcd417fa700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2806ace0 0x7fcd2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.747+0000 7fcd417fa700 1 -- 172.21.15.60:0/2034127926 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd300890c0 con 0x7fcd440d0b80 2023-07-13T22:28:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.747+0000 7fcd48b9b700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2806ace0 0x7fcd2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:41.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.748+0000 7fcd48b9b700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2806ace0 0x7fcd2806d190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcd38000c00 tx=0x7fcd38008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:41.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:41.751+0000 7fcd417fa700 1 -- 172.21.15.60:0/2034127926 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcd30054160 con 0x7fcd440d0b80 2023-07-13T22:28:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.001+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd440741e0 con 0x7fcd440d0b80 2023-07-13T22:28:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.001+0000 7fcd417fa700 1 -- 172.21.15.60:0/2034127926 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4718 (secure 0 0 0) 0x7fcd30057780 con 0x7fcd440d0b80 2023-07-13T22:28:41.981 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:41.981 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2395854295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2395854295},{"type":"v1","addr":"172.21.15.105:6834","nonce":2395854295}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1048261776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1048261776},{"type":"v1","addr":"172.21.15.105:6835","nonce":1048261776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/1639899719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1639899719},{"type":"v1","addr":"172.21.15.60:6836","nonce":1639899719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:04.618240+0000","modified":"2023-07-13T22:28:41.646873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7434":{"gid":7434,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":12,"addr":"172.21.15.60:6837/1512512599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":1512512599},{"type":"v1","addr":"172.21.15.60:6837","nonce":1512512599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-13T22:28:41.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2806ace0 msgr2=0x7fcd2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2806ace0 0x7fcd2806d190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcd38000c00 tx=0x7fcd38008030 comp rx=0 tx=0).stop 2023-07-13T22:28:41.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 msgr2=0x7fcd440cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:41.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 0x7fcd440cbfc0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcd300011e0 tx=0x7fcd3000ca30 comp rx=0 tx=0).stop 2023-07-13T22:28:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 shutdown_connections 2023-07-13T22:28:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fcd2806ace0 0x7fcd2806d190 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd440d0f50 0x7fcd4415de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd440cf8e0 0x7fcd440ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 --2- 172.21.15.60:0/2034127926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd440d0b80 0x7fcd440cbfc0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 >> 172.21.15.60:0/2034127926 conn(0x7fcd44071660 msgr2=0x7fcd440b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 shutdown_connections 2023-07-13T22:28:41.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.004+0000 7fcd4adff700 1 -- 172.21.15.60:0/2034127926 wait complete. 2023-07-13T22:28:41.985 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 54 2023-07-13T22:28:42.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:42.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.226+0000 7fa7a55ff700 1 Processor -- start 2023-07-13T22:28:42.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.226+0000 7fa7a55ff700 1 -- start start 2023-07-13T22:28:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.227+0000 7fa7a55ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7a00b5ad0 0x7fa7a00d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.227+0000 7fa7a55ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7a00cf8e0 0x7fa7a00cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.227+0000 7fa7a55ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a00ceeb0 con 0x7fa7a00d0b80 2023-07-13T22:28:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.227+0000 7fa7a55ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a00b23f0 con 0x7fa7a00b5ad0 2023-07-13T22:28:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.227+0000 7fa7a55ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a00b4bb0 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.228+0000 7fa79e7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7a00b5ad0 0x7fa7a00d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.228+0000 7fa79e7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7a00b5ad0 0x7fa7a00d2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39740/0 (socket says 172.21.15.60:39740) 2023-07-13T22:28:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.228+0000 7fa79e7fc700 1 -- 172.21.15.60:0/3802944478 learned_addr learned my addr 172.21.15.60:0/3802944478 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.228+0000 7fa79e7fc700 1 -- 172.21.15.60:0/3802944478 >> v2:172.21.15.105:3300/0 conn(0x7fa7a00cf8e0 msgr2=0x7fa7a00cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.228+0000 7fa79e7fc700 1 --2- 172.21.15.60:0/3802944478 >> v2:172.21.15.105:3300/0 conn(0x7fa7a00cf8e0 0x7fa7a00cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.228+0000 7fa79e7fc700 1 -- 172.21.15.60:0/3802944478 >> v1:172.21.15.105:6789/0 conn(0x7fa7a00d0b80 legacy=0x7fa7a00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.228+0000 7fa79e7fc700 1 -- 172.21.15.60:0/3802944478 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a00b5f40 con 0x7fa7a00b5ad0 2023-07-13T22:28:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.228+0000 7fa79e7fc700 1 --2- 172.21.15.60:0/3802944478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00b5ad0 0x7fa7a00d2f90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa7a006a2a0 tx=0x7fa794009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b67d4850dc5e865 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.228+0000 7fa79dffb700 1 -- 172.21.15.60:0/3802944478 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794004230 con 0x7fa7a00b5ad0 2023-07-13T22:28:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa79dffb700 1 -- 172.21.15.60:0/3802944478 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa794004390 con 0x7fa7a00b5ad0 2023-07-13T22:28:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa79dffb700 1 -- 172.21.15.60:0/3802944478 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7940036a0 con 0x7fa7a00b5ad0 2023-07-13T22:28:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa7a55ff700 1 -- 172.21.15.60:0/3802944478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00b5ad0 msgr2=0x7fa7a00d2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa7a55ff700 1 --2- 172.21.15.60:0/3802944478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00b5ad0 0x7fa7a00d2f90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa7a006a2a0 tx=0x7fa794009bd0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa7a55ff700 1 -- 172.21.15.60:0/3802944478 shutdown_connections 2023-07-13T22:28:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa7a55ff700 1 --2- 172.21.15.60:0/3802944478 >> v2:172.21.15.105:3300/0 conn(0x7fa7a00cf8e0 0x7fa7a00cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa7a55ff700 1 --2- 172.21.15.60:0/3802944478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00b5ad0 0x7fa7a00d2f90 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa7a55ff700 1 -- 172.21.15.60:0/3802944478 >> 172.21.15.60:0/3802944478 conn(0x7fa7a0071660 msgr2=0x7fa7a0071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa7a55ff700 1 -- 172.21.15.60:0/3802944478 shutdown_connections 2023-07-13T22:28:42.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.229+0000 7fa7a55ff700 1 -- 172.21.15.60:0/3802944478 wait complete. 2023-07-13T22:28:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.231+0000 7fa7a55ff700 1 Processor -- start 2023-07-13T22:28:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.231+0000 7fa7a55ff700 1 -- start start 2023-07-13T22:28:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.232+0000 7fa7a55ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7a00b5ad0 0x7fa7a0145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.232+0000 7fa7a55ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00cf8e0 0x7fa7a01f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.232+0000 7fa7a55ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7a00d0b80 0x7fa7a015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.232+0000 7fa7a55ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0145f80 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.232+0000 7fa7a55ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0144ff0 con 0x7fa7a00b5ad0 2023-07-13T22:28:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.232+0000 7fa7a55ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a01455b0 con 0x7fa7a00d0b80 2023-07-13T22:28:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.232+0000 7fa79e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00cf8e0 0x7fa7a01f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.232+0000 7fa79e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00cf8e0 0x7fa7a01f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39742/0 (socket says 172.21.15.60:39742) 2023-07-13T22:28:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79e7fc700 1 -- 172.21.15.60:0/2694947289 learned_addr learned my addr 172.21.15.60:0/2694947289 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79effd700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7a00b5ad0 0x7fa7a0145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79f7fe700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7a00d0b80 0x7fa7a015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79e7fc700 1 -- 172.21.15.60:0/2694947289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7a00d0b80 msgr2=0x7fa7a015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79e7fc700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7a00d0b80 0x7fa7a015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79e7fc700 1 -- 172.21.15.60:0/2694947289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7a00b5ad0 msgr2=0x7fa7a0145a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79e7fc700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7a00b5ad0 0x7fa7a0145a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79e7fc700 1 -- 172.21.15.60:0/2694947289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa794009550 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79f7fe700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7a00d0b80 0x7fa7a015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:42.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa79e7fc700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00cf8e0 0x7fa7a01f1d40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00d0a50 tx=0x7fa794004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:42.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa7877fe700 1 -- 172.21.15.60:0/2694947289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794011630 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa7877fe700 1 -- 172.21.15.60:0/2694947289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa794011790 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.233+0000 7fa7877fe700 1 -- 172.21.15.60:0/2694947289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794011b40 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.234+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a01f27f0 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.234+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a0145240 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.235+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a0145720 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.235+0000 7fa7877fe700 1 -- 172.21.15.60:0/2694947289 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa794011ca0 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.236+0000 7fa7877fe700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa78c06ad10 0x7fa78c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.236+0000 7fa7877fe700 1 -- 172.21.15.60:0/2694947289 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa79408ab40 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.237+0000 7fa79effd700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa78c06ad10 0x7fa78c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.237+0000 7fa79effd700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa78c06ad10 0x7fa78c06d1c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa7a015c180 tx=0x7fa790009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:42.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.241+0000 7fa7877fe700 1 -- 172.21.15.60:0/2694947289 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa794055be0 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.448+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7a00741e0 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.450+0000 7fa7877fe700 1 -- 172.21.15.60:0/2694947289 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18200 (secure 0 0 0) 0x7fa79402c3a0 con 0x7fa7a00cf8e0 2023-07-13T22:28:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:42.429 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:06.113196+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:02.360339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:03.609373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:42.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa78c06ad10 msgr2=0x7fa78c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa78c06ad10 0x7fa78c06d1c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa7a015c180 tx=0x7fa790009040 comp rx=0 tx=0).stop 2023-07-13T22:28:42.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00cf8e0 msgr2=0x7fa7a01f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00cf8e0 0x7fa7a01f1d40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00d0a50 tx=0x7fa794004750 comp rx=0 tx=0).stop 2023-07-13T22:28:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 shutdown_connections 2023-07-13T22:28:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa78c06ad10 0x7fa78c06d1c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7a00d0b80 0x7fa7a015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7a00b5ad0 0x7fa7a0145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 --2- 172.21.15.60:0/2694947289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a00cf8e0 0x7fa7a01f1d40 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 >> 172.21.15.60:0/2694947289 conn(0x7fa7a0071660 msgr2=0x7fa7a00b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 shutdown_connections 2023-07-13T22:28:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.452+0000 7fa7a55ff700 1 -- 172.21.15.60:0/2694947289 wait complete. 2023-07-13T22:28:42.449 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:28:42.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:28:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.675+0000 7f06405ff700 1 Processor -- start 2023-07-13T22:28:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.676+0000 7f06405ff700 1 -- start start 2023-07-13T22:28:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.677+0000 7f06405ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06380d0b80 0x7f06380b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.677+0000 7f06405ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380cf8e0 0x7f06380cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.677+0000 7f06405ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06380ceeb0 con 0x7f06380d0f50 2023-07-13T22:28:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.677+0000 7f06405ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06380b23f0 con 0x7f06380d0b80 2023-07-13T22:28:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.677+0000 7f06405ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06380b4bb0 con 0x7f06380cf8e0 2023-07-13T22:28:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.677+0000 7f063eb9c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380cf8e0 0x7f06380cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.677+0000 7f063e39b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06380d0b80 0x7f06380b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.677+0000 7f063eb9c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380cf8e0 0x7f06380cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:50438/0 (socket says 172.21.15.60:50438) 2023-07-13T22:28:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.677+0000 7f063eb9c700 1 -- 172.21.15.60:0/1192562584 learned_addr learned my addr 172.21.15.60:0/1192562584 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.678+0000 7f063eb9c700 1 -- 172.21.15.60:0/1192562584 >> v2:172.21.15.105:3300/0 conn(0x7f06380d0b80 msgr2=0x7f06380b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.678+0000 7f063eb9c700 1 --2- 172.21.15.60:0/1192562584 >> v2:172.21.15.105:3300/0 conn(0x7f06380d0b80 0x7f06380b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.678+0000 7f063eb9c700 1 -- 172.21.15.60:0/1192562584 >> v1:172.21.15.105:6789/0 conn(0x7f06380d0f50 legacy=0x7f06380d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.678+0000 7f063eb9c700 1 -- 172.21.15.60:0/1192562584 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380b5380 con 0x7f06380cf8e0 2023-07-13T22:28:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.678+0000 7f063eb9c700 1 --2- 172.21.15.60:0/1192562584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380cf8e0 0x7f06380cfd60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0630009e50 tx=0x7f063000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1349b0524c2023 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.678+0000 7f063d399700 1 -- 172.21.15.60:0/1192562584 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000e070 con 0x7f06380cf8e0 2023-07-13T22:28:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.678+0000 7f063d399700 1 -- 172.21.15.60:0/1192562584 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063000b910 con 0x7f06380cf8e0 2023-07-13T22:28:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.678+0000 7f063d399700 1 -- 172.21.15.60:0/1192562584 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000bcc0 con 0x7f06380cf8e0 2023-07-13T22:28:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.679+0000 7f06405ff700 1 -- 172.21.15.60:0/1192562584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380cf8e0 msgr2=0x7f06380cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.679+0000 7f06405ff700 1 --2- 172.21.15.60:0/1192562584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380cf8e0 0x7f06380cfd60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0630009e50 tx=0x7f063000b330 comp rx=0 tx=0).stop 2023-07-13T22:28:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.679+0000 7f06405ff700 1 -- 172.21.15.60:0/1192562584 shutdown_connections 2023-07-13T22:28:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.679+0000 7f06405ff700 1 --2- 172.21.15.60:0/1192562584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380cf8e0 0x7f06380cfd60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.679+0000 7f06405ff700 1 --2- 172.21.15.60:0/1192562584 >> v2:172.21.15.105:3300/0 conn(0x7f06380d0b80 0x7f06380b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.679+0000 7f06405ff700 1 -- 172.21.15.60:0/1192562584 >> 172.21.15.60:0/1192562584 conn(0x7f0638071660 msgr2=0x7f0638071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.679+0000 7f06405ff700 1 -- 172.21.15.60:0/1192562584 shutdown_connections 2023-07-13T22:28:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.679+0000 7f06405ff700 1 -- 172.21.15.60:0/1192562584 wait complete. 2023-07-13T22:28:42.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.681+0000 7f06405ff700 1 Processor -- start 2023-07-13T22:28:42.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.681+0000 7f06405ff700 1 -- start start 2023-07-13T22:28:42.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f06405ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06380cf8e0 0x7f0638159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f06405ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380d0b80 0x7f0638154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f06405ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06380d0f50 0x7f063815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f06405ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638155af0 con 0x7f06380cf8e0 2023-07-13T22:28:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f06405ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638156f80 con 0x7f06380d0f50 2023-07-13T22:28:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f06405ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06381571d0 con 0x7f06380d0b80 2023-07-13T22:28:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f063e39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06380cf8e0 0x7f0638159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f063e39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06380cf8e0 0x7f0638159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39748/0 (socket says 172.21.15.60:39748) 2023-07-13T22:28:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f063e39b700 1 -- 172.21.15.60:0/3743138660 learned_addr learned my addr 172.21.15.60:0/3743138660 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f063db9a700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380d0b80 0x7f0638154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.682+0000 7f063eb9c700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06380d0f50 0x7f063815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f063e39b700 1 -- 172.21.15.60:0/3743138660 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380d0b80 msgr2=0x7f0638154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f063e39b700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380d0b80 0x7f0638154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f063e39b700 1 -- 172.21.15.60:0/3743138660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06380d0f50 msgr2=0x7f063815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f063e39b700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06380d0f50 0x7f063815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f063e39b700 1 -- 172.21.15.60:0/3743138660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630009b30 con 0x7f06380cf8e0 2023-07-13T22:28:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f063e39b700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06380cf8e0 0x7f0638159ea0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f063400e9e0 tx=0x7f063400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f062effd700 1 -- 172.21.15.60:0/3743138660 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634009710 con 0x7f06380cf8e0 2023-07-13T22:28:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f062effd700 1 -- 172.21.15.60:0/3743138660 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063400ee80 con 0x7f06380cf8e0 2023-07-13T22:28:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f062effd700 1 -- 172.21.15.60:0/3743138660 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400ca50 con 0x7f06380cf8e0 2023-07-13T22:28:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06381574b0 con 0x7f06380cf8e0 2023-07-13T22:28:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.683+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0638157a70 con 0x7f06380cf8e0 2023-07-13T22:28:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.684+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0638157c90 con 0x7f06380cf8e0 2023-07-13T22:28:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.685+0000 7f062effd700 1 -- 172.21.15.60:0/3743138660 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f063400cbb0 con 0x7f06380cf8e0 2023-07-13T22:28:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.686+0000 7f062effd700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f061c06ace0 0x7f061c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.686+0000 7f062effd700 1 -- 172.21.15.60:0/3743138660 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06340895e0 con 0x7f06380cf8e0 2023-07-13T22:28:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.687+0000 7f063db9a700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f061c06ace0 0x7f061c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.687+0000 7f063db9a700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f061c06ace0 0x7f061c06d190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f06380b30a0 tx=0x7f0628009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.691+0000 7f062effd700 1 -- 172.21.15.60:0/3743138660 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0634057f70 con 0x7f06380cf8e0 2023-07-13T22:28:42.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.940+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06380741e0 con 0x7f06380cf8e0 2023-07-13T22:28:42.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.940+0000 7f062effd700 1 -- 172.21.15.60:0/3743138660 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4718 (secure 0 0 0) 0x7f0634057b00 con 0x7f06380cf8e0 2023-07-13T22:28:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:42.920 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2395854295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2395854295},{"type":"v1","addr":"172.21.15.105:6834","nonce":2395854295}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1048261776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1048261776},{"type":"v1","addr":"172.21.15.105:6835","nonce":1048261776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/1639899719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1639899719},{"type":"v1","addr":"172.21.15.60:6836","nonce":1639899719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:04.618240+0000","modified":"2023-07-13T22:28:41.646873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7434":{"gid":7434,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":12,"addr":"172.21.15.60:6837/1512512599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":1512512599},{"type":"v1","addr":"172.21.15.60:6837","nonce":1512512599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-13T22:28:42.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f061c06ace0 msgr2=0x7f061c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f061c06ace0 0x7f061c06d190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f06380b30a0 tx=0x7f0628009040 comp rx=0 tx=0).stop 2023-07-13T22:28:42.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06380cf8e0 msgr2=0x7f0638159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:42.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06380cf8e0 0x7f0638159ea0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f063400e9e0 tx=0x7f063400ecf0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 shutdown_connections 2023-07-13T22:28:42.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f061c06ace0 0x7f061c06d190 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06380cf8e0 0x7f0638159ea0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06380d0f50 0x7f063815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 --2- 172.21.15.60:0/3743138660 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06380d0b80 0x7f0638154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:42.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 >> 172.21.15.60:0/3743138660 conn(0x7f0638071660 msgr2=0x7f06380b80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:42.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 shutdown_connections 2023-07-13T22:28:42.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:42.943+0000 7f06405ff700 1 -- 172.21.15.60:0/3743138660 wait complete. 2023-07-13T22:28:42.924 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 54 2023-07-13T22:28:42.941 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:28:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.165+0000 7fd15b5ff700 1 Processor -- start 2023-07-13T22:28:43.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.166+0000 7fd15b5ff700 1 -- start start 2023-07-13T22:28:43.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.167+0000 7fd15b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0b80 0x7fd1540b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.167+0000 7fd15b5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1540cf8e0 0x7fd1540cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.167+0000 7fd15b5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1540ceeb0 con 0x7fd1540d0f50 2023-07-13T22:28:43.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.167+0000 7fd15b5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1540b23f0 con 0x7fd1540cf8e0 2023-07-13T22:28:43.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.167+0000 7fd15b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540b4bb0 con 0x7fd1540d0b80 2023-07-13T22:28:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.167+0000 7fd159b9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1540cf8e0 0x7fd1540cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.167+0000 7fd15939b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0b80 0x7fd1540b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.167+0000 7fd159b9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1540cf8e0 0x7fd1540cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:44442/0 (socket says 172.21.15.60:44442) 2023-07-13T22:28:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.167+0000 7fd159b9c700 1 -- 172.21.15.60:0/910236758 learned_addr learned my addr 172.21.15.60:0/910236758 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.168+0000 7fd159b9c700 1 -- 172.21.15.60:0/910236758 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0b80 msgr2=0x7fd1540b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.168+0000 7fd159b9c700 1 --2- 172.21.15.60:0/910236758 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0b80 0x7fd1540b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.168+0000 7fd159b9c700 1 -- 172.21.15.60:0/910236758 >> v1:172.21.15.105:6789/0 conn(0x7fd1540d0f50 legacy=0x7fd1540d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.168+0000 7fd159b9c700 1 -- 172.21.15.60:0/910236758 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540b5380 con 0x7fd1540cf8e0 2023-07-13T22:28:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.168+0000 7fd159b9c700 1 --2- 172.21.15.60:0/910236758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540cf8e0 0x7fd1540cfd60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009e50 tx=0x7fd14c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d93f02920a9bd06 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.168+0000 7fd14bfff700 1 -- 172.21.15.60:0/910236758 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00e070 con 0x7fd1540cf8e0 2023-07-13T22:28:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.168+0000 7fd14bfff700 1 -- 172.21.15.60:0/910236758 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c00b910 con 0x7fd1540cf8e0 2023-07-13T22:28:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.169+0000 7fd14bfff700 1 -- 172.21.15.60:0/910236758 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00bcc0 con 0x7fd1540cf8e0 2023-07-13T22:28:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.169+0000 7fd15b5ff700 1 -- 172.21.15.60:0/910236758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540cf8e0 msgr2=0x7fd1540cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.169+0000 7fd15b5ff700 1 --2- 172.21.15.60:0/910236758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540cf8e0 0x7fd1540cfd60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009e50 tx=0x7fd14c00b330 comp rx=0 tx=0).stop 2023-07-13T22:28:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.169+0000 7fd15b5ff700 1 -- 172.21.15.60:0/910236758 shutdown_connections 2023-07-13T22:28:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.169+0000 7fd15b5ff700 1 --2- 172.21.15.60:0/910236758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540cf8e0 0x7fd1540cfd60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.169+0000 7fd15b5ff700 1 --2- 172.21.15.60:0/910236758 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0b80 0x7fd1540b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.169+0000 7fd15b5ff700 1 -- 172.21.15.60:0/910236758 >> 172.21.15.60:0/910236758 conn(0x7fd154071660 msgr2=0x7fd154071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.169+0000 7fd15b5ff700 1 -- 172.21.15.60:0/910236758 shutdown_connections 2023-07-13T22:28:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.169+0000 7fd15b5ff700 1 -- 172.21.15.60:0/910236758 wait complete. 2023-07-13T22:28:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.171+0000 7fd15b5ff700 1 Processor -- start 2023-07-13T22:28:43.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.171+0000 7fd15b5ff700 1 -- start start 2023-07-13T22:28:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd15b5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540cf8e0 0x7fd154154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd15b5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1540d0b80 0x7fd1541f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd15b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0f50 0x7fd154158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd15b5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540c9060 con 0x7fd1540d0b80 2023-07-13T22:28:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd15b5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540cb1b0 con 0x7fd1540cf8e0 2023-07-13T22:28:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd15b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540cb490 con 0x7fd1540d0f50 2023-07-13T22:28:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd158b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1540d0b80 0x7fd1541f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd158b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1540d0b80 0x7fd1541f2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39754/0 (socket says 172.21.15.60:39754) 2023-07-13T22:28:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd158b9a700 1 -- 172.21.15.60:0/625222362 learned_addr learned my addr 172.21.15.60:0/625222362 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd159b9c700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0f50 0x7fd154158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.172+0000 7fd15939b700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540cf8e0 0x7fd154154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd158b9a700 1 -- 172.21.15.60:0/625222362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0f50 msgr2=0x7fd154158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd158b9a700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0f50 0x7fd154158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd158b9a700 1 -- 172.21.15.60:0/625222362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540cf8e0 msgr2=0x7fd154154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd158b9a700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540cf8e0 0x7fd154154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd158b9a700 1 -- 172.21.15.60:0/625222362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c009b30 con 0x7fd1540d0b80 2023-07-13T22:28:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd158b9a700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1540d0b80 0x7fd1541f2140 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd14400c450 tx=0x7fd14400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd149ffb700 1 -- 172.21.15.60:0/625222362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440052d0 con 0x7fd1540d0b80 2023-07-13T22:28:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd149ffb700 1 -- 172.21.15.60:0/625222362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144005430 con 0x7fd1540d0b80 2023-07-13T22:28:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd149ffb700 1 -- 172.21.15.60:0/625222362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440038c0 con 0x7fd1540d0b80 2023-07-13T22:28:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1540cb770 con 0x7fd1540d0b80 2023-07-13T22:28:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.173+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1541553c0 con 0x7fd1540d0b80 2023-07-13T22:28:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.174+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1540cbc90 con 0x7fd1540d0b80 2023-07-13T22:28:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.175+0000 7fd149ffb700 1 -- 172.21.15.60:0/625222362 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd144003a20 con 0x7fd1540d0b80 2023-07-13T22:28:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.176+0000 7fd149ffb700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd13806ace0 0x7fd13806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.176+0000 7fd149ffb700 1 -- 172.21.15.60:0/625222362 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd144088840 con 0x7fd1540d0b80 2023-07-13T22:28:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.176+0000 7fd15939b700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd13806ace0 0x7fd13806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.177+0000 7fd15939b700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd13806ace0 0x7fd13806d190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd1540b5eb0 tx=0x7fd150009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:43.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.180+0000 7fd149ffb700 1 -- 172.21.15.60:0/625222362 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd1540cbc90 con 0x7fd1540d0b80 2023-07-13T22:28:43.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.434+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1540cbc90 con 0x7fd1540d0b80 2023-07-13T22:28:43.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.435+0000 7fd149ffb700 1 -- 172.21.15.60:0/625222362 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4718 (secure 0 0 0) 0x7fd1540cbc90 con 0x7fd1540d0b80 2023-07-13T22:28:43.414 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:43.414 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2395854295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2395854295},{"type":"v1","addr":"172.21.15.105:6834","nonce":2395854295}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1048261776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1048261776},{"type":"v1","addr":"172.21.15.105:6835","nonce":1048261776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/1639899719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1639899719},{"type":"v1","addr":"172.21.15.60:6836","nonce":1639899719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:04.618240+0000","modified":"2023-07-13T22:28:41.646873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7434":{"gid":7434,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":12,"addr":"172.21.15.60:6837/1512512599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":1512512599},{"type":"v1","addr":"172.21.15.60:6837","nonce":1512512599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-13T22:28:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd13806ace0 msgr2=0x7fd13806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd13806ace0 0x7fd13806d190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd1540b5eb0 tx=0x7fd150009210 comp rx=0 tx=0).stop 2023-07-13T22:28:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1540d0b80 msgr2=0x7fd1541f2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1540d0b80 0x7fd1541f2140 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd14400c450 tx=0x7fd14400c760 comp rx=0 tx=0).stop 2023-07-13T22:28:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 shutdown_connections 2023-07-13T22:28:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd13806ace0 0x7fd13806d190 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1540d0f50 0x7fd154158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1540cf8e0 0x7fd154154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 --2- 172.21.15.60:0/625222362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1540d0b80 0x7fd1541f2140 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 >> 172.21.15.60:0/625222362 conn(0x7fd154071660 msgr2=0x7fd1540b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 shutdown_connections 2023-07-13T22:28:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.437+0000 7fd15b5ff700 1 -- 172.21.15.60:0/625222362 wait complete. 2023-07-13T22:28:43.418 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 54 2023-07-13T22:28:43.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.660+0000 7feaff3ff700 1 Processor -- start 2023-07-13T22:28:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.660+0000 7feaff3ff700 1 -- start start 2023-07-13T22:28:43.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.661+0000 7feaff3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feaf80b5ad0 0x7feaf80d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.661+0000 7feaff3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feaf80cf8e0 0x7feaf80cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.661+0000 7feaff3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf80ceeb0 con 0x7feaf80d0b80 2023-07-13T22:28:43.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.661+0000 7feaff3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf80b23f0 con 0x7feaf80b5ad0 2023-07-13T22:28:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.661+0000 7feaff3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf80b4bb0 con 0x7feaf80cf8e0 2023-07-13T22:28:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.662+0000 7feafc99a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feaf80b5ad0 0x7feaf80d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.662+0000 7feafc99a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feaf80b5ad0 0x7feaf80d2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39762/0 (socket says 172.21.15.60:39762) 2023-07-13T22:28:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.662+0000 7feafc99a700 1 -- 172.21.15.60:0/4289446667 learned_addr learned my addr 172.21.15.60:0/4289446667 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.662+0000 7feafd99c700 1 --2- 172.21.15.60:0/4289446667 >> v2:172.21.15.105:3300/0 conn(0x7feaf80cf8e0 0x7feaf80cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.662+0000 7feafc99a700 1 -- 172.21.15.60:0/4289446667 >> v2:172.21.15.105:3300/0 conn(0x7feaf80cf8e0 msgr2=0x7feaf80cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.662+0000 7feafc99a700 1 --2- 172.21.15.60:0/4289446667 >> v2:172.21.15.105:3300/0 conn(0x7feaf80cf8e0 0x7feaf80cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.662+0000 7feafc99a700 1 -- 172.21.15.60:0/4289446667 >> v1:172.21.15.105:6789/0 conn(0x7feaf80d0b80 legacy=0x7feaf80d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.662+0000 7feafc99a700 1 -- 172.21.15.60:0/4289446667 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80b5f40 con 0x7feaf80b5ad0 2023-07-13T22:28:43.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.662+0000 7feafc99a700 1 --2- 172.21.15.60:0/4289446667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80b5ad0 0x7feaf80d2f90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7feaf806a2a0 tx=0x7feae8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8db345d1b2c8b48 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:43.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaeffff700 1 -- 172.21.15.60:0/4289446667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8011830 con 0x7feaf80b5ad0 2023-07-13T22:28:43.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaeffff700 1 -- 172.21.15.60:0/4289446667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae8011990 con 0x7feaf80b5ad0 2023-07-13T22:28:43.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaeffff700 1 -- 172.21.15.60:0/4289446667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8011d20 con 0x7feaf80b5ad0 2023-07-13T22:28:43.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaff3ff700 1 -- 172.21.15.60:0/4289446667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80b5ad0 msgr2=0x7feaf80d2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaff3ff700 1 --2- 172.21.15.60:0/4289446667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80b5ad0 0x7feaf80d2f90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7feaf806a2a0 tx=0x7feae8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaff3ff700 1 -- 172.21.15.60:0/4289446667 shutdown_connections 2023-07-13T22:28:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaff3ff700 1 --2- 172.21.15.60:0/4289446667 >> v2:172.21.15.105:3300/0 conn(0x7feaf80cf8e0 0x7feaf80cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaff3ff700 1 --2- 172.21.15.60:0/4289446667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80b5ad0 0x7feaf80d2f90 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaff3ff700 1 -- 172.21.15.60:0/4289446667 >> 172.21.15.60:0/4289446667 conn(0x7feaf8071660 msgr2=0x7feaf8071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaff3ff700 1 -- 172.21.15.60:0/4289446667 shutdown_connections 2023-07-13T22:28:43.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.663+0000 7feaff3ff700 1 -- 172.21.15.60:0/4289446667 wait complete. 2023-07-13T22:28:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.665+0000 7feaff3ff700 1 Processor -- start 2023-07-13T22:28:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.665+0000 7feaff3ff700 1 -- start start 2023-07-13T22:28:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.666+0000 7feaff3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf80b5ad0 0x7feaf80c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.666+0000 7feaff3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feaf80cf8e0 0x7feaf815af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.666+0000 7feaff3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80d0b80 0x7feaf8146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.666+0000 7feaff3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8157f60 con 0x7feaf80d0b80 2023-07-13T22:28:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.666+0000 7feaff3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8157350 con 0x7feaf80b5ad0 2023-07-13T22:28:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.666+0000 7feaff3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8157630 con 0x7feaf80cf8e0 2023-07-13T22:28:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.666+0000 7feafd99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80d0b80 0x7feaf8146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.666+0000 7feafd99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80d0b80 0x7feaf8146870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39776/0 (socket says 172.21.15.60:39776) 2023-07-13T22:28:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.666+0000 7feafd99c700 1 -- 172.21.15.60:0/3962461117 learned_addr learned my addr 172.21.15.60:0/3962461117 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feafd19b700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf80b5ad0 0x7feaf80c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feafc99a700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feaf80cf8e0 0x7feaf815af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feafd99c700 1 -- 172.21.15.60:0/3962461117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feaf80cf8e0 msgr2=0x7feaf815af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feafd99c700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feaf80cf8e0 0x7feaf815af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feafd99c700 1 -- 172.21.15.60:0/3962461117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf80b5ad0 msgr2=0x7feaf80c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feafd99c700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf80b5ad0 0x7feaf80c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feafd99c700 1 -- 172.21.15.60:0/3962461117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009550 con 0x7feaf80d0b80 2023-07-13T22:28:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feafd99c700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80d0b80 0x7feaf8146870 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7feaf000efc0 tx=0x7feaf000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feaedffb700 1 -- 172.21.15.60:0/3962461117 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf000cd40 con 0x7feaf80d0b80 2023-07-13T22:28:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feaedffb700 1 -- 172.21.15.60:0/3962461117 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf000f040 con 0x7feaf80d0b80 2023-07-13T22:28:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf8157910 con 0x7feaf80d0b80 2023-07-13T22:28:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feaedffb700 1 -- 172.21.15.60:0/3962461117 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00146c0 con 0x7feaf80d0b80 2023-07-13T22:28:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.667+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf80c90e0 con 0x7feaf80d0b80 2023-07-13T22:28:43.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.669+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf80741e0 con 0x7feaf80d0b80 2023-07-13T22:28:43.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.669+0000 7feaedffb700 1 -- 172.21.15.60:0/3962461117 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feaf0014820 con 0x7feaf80d0b80 2023-07-13T22:28:43.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.670+0000 7feaedffb700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feadc06ace0 0x7feadc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:43.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.670+0000 7feaedffb700 1 -- 172.21.15.60:0/3962461117 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feaf0089ec0 con 0x7feaf80d0b80 2023-07-13T22:28:43.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.671+0000 7feafd19b700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feadc06ace0 0x7feadc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:43.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.671+0000 7feafd19b700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feadc06ace0 0x7feadc06d190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f60 tx=0x7feaf4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:43.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.675+0000 7feaedffb700 1 -- 172.21.15.60:0/3962461117 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7feaf0058960 con 0x7feaf80d0b80 2023-07-13T22:28:43.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.883+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaf81580a0 con 0x7feaf80d0b80 2023-07-13T22:28:43.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.885+0000 7feaedffb700 1 -- 172.21.15.60:0/3962461117 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18200 (secure 0 0 0) 0x7feaf000cea0 con 0x7feaf80d0b80 2023-07-13T22:28:43.864 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:43.864 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:06.113196+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:02.360339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:03.609373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:43.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feadc06ace0 msgr2=0x7feadc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feadc06ace0 0x7feadc06d190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f60 tx=0x7feaf4009040 comp rx=0 tx=0).stop 2023-07-13T22:28:43.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80d0b80 msgr2=0x7feaf8146870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:43.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80d0b80 0x7feaf8146870 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7feaf000efc0 tx=0x7feaf000c3a0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 shutdown_connections 2023-07-13T22:28:43.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feadc06ace0 0x7feadc06d190 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feaf80d0b80 0x7feaf8146870 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf80b5ad0 0x7feaf80c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 --2- 172.21.15.60:0/3962461117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feaf80cf8e0 0x7feaf815af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:43.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 >> 172.21.15.60:0/3962461117 conn(0x7feaf8071660 msgr2=0x7feaf80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:43.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 shutdown_connections 2023-07-13T22:28:43.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:43.887+0000 7feaff3ff700 1 -- 172.21.15.60:0/3962461117 wait complete. 2023-07-13T22:28:43.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:28:44.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.110+0000 7f81383ff700 1 Processor -- start 2023-07-13T22:28:44.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.110+0000 7f81383ff700 1 -- start start 2023-07-13T22:28:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.111+0000 7f81383ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f81300b5ad0 0x7f81300d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.111+0000 7f81383ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81300ceeb0 con 0x7f81300d0b80 2023-07-13T22:28:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.111+0000 7f81383ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81300b5f40 con 0x7f81300cf8e0 2023-07-13T22:28:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.111+0000 7f81383ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81300b23f0 con 0x7f81300b5ad0 2023-07-13T22:28:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.111+0000 7f813619b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f81300d0b80 0x7f81300d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60428/0 (socket says 172.21.15.60:60428) 2023-07-13T22:28:44.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.111+0000 7f813619b700 1 -- 172.21.15.60:0/1139396684 learned_addr learned my addr 172.21.15.60:0/1139396684 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.111+0000 7f813599a700 1 --2- 172.21.15.60:0/1139396684 >> v2:172.21.15.105:3300/0 conn(0x7f81300b5ad0 0x7f81300d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.112+0000 7f8135199700 1 -- 172.21.15.60:0/1139396684 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675151300 0 0) 0x7f81300ceeb0 con 0x7f81300d0b80 2023-07-13T22:28:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.112+0000 7f8135199700 1 -- 172.21.15.60:0/1139396684 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811c0035b0 con 0x7f81300d0b80 2023-07-13T22:28:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.112+0000 7f813599a700 1 -- 172.21.15.60:0/1139396684 >> v1:172.21.15.105:6789/0 conn(0x7f81300cf8e0 legacy=0x7f81300cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.112+0000 7f813599a700 1 -- 172.21.15.60:0/1139396684 >> v1:172.21.15.60:6789/0 conn(0x7f81300d0b80 legacy=0x7f81300d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.112+0000 7f813599a700 1 -- 172.21.15.60:0/1139396684 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300b5380 con 0x7f81300b5ad0 2023-07-13T22:28:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.112+0000 7f813599a700 1 --2- 172.21.15.60:0/1139396684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81300b5ad0 0x7f81300d2f90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8120009840 tx=0x7f8120009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2194b030cb58f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.112+0000 7f8135199700 1 -- 172.21.15.60:0/1139396684 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120011830 con 0x7f81300b5ad0 2023-07-13T22:28:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.112+0000 7f8135199700 1 -- 172.21.15.60:0/1139396684 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8120011990 con 0x7f81300b5ad0 2023-07-13T22:28:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.113+0000 7f8135199700 1 -- 172.21.15.60:0/1139396684 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120011d20 con 0x7f81300b5ad0 2023-07-13T22:28:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.113+0000 7f81383ff700 1 -- 172.21.15.60:0/1139396684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81300b5ad0 msgr2=0x7f81300d2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.113+0000 7f81383ff700 1 --2- 172.21.15.60:0/1139396684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81300b5ad0 0x7f81300d2f90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8120009840 tx=0x7f8120009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:44.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.113+0000 7f81383ff700 1 -- 172.21.15.60:0/1139396684 shutdown_connections 2023-07-13T22:28:44.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.113+0000 7f81383ff700 1 --2- 172.21.15.60:0/1139396684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81300b5ad0 0x7f81300d2f90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.113+0000 7f81383ff700 1 -- 172.21.15.60:0/1139396684 >> 172.21.15.60:0/1139396684 conn(0x7f8130071660 msgr2=0x7f8130071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:44.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.113+0000 7f81383ff700 1 -- 172.21.15.60:0/1139396684 shutdown_connections 2023-07-13T22:28:44.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.113+0000 7f81383ff700 1 -- 172.21.15.60:0/1139396684 wait complete. 2023-07-13T22:28:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.115+0000 7f81383ff700 1 Processor -- start 2023-07-13T22:28:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.115+0000 7f81383ff700 1 -- start start 2023-07-13T22:28:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.116+0000 7f81383ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81300b5ad0 0x7f8130154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.116+0000 7f81383ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81300cf8e0 0x7f81300cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.116+0000 7f81383ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81300d0b80 0x7f8130153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.116+0000 7f81383ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130154920 con 0x7f81300b5ad0 2023-07-13T22:28:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.116+0000 7f81383ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130156330 con 0x7f81300cf8e0 2023-07-13T22:28:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.116+0000 7f81383ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8130156610 con 0x7f81300d0b80 2023-07-13T22:28:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813619b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81300b5ad0 0x7f8130154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813619b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81300b5ad0 0x7f8130154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39782/0 (socket says 172.21.15.60:39782) 2023-07-13T22:28:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813619b700 1 -- 172.21.15.60:0/4269499277 learned_addr learned my addr 172.21.15.60:0/4269499277 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813599a700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81300cf8e0 0x7f81300cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813699c700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81300d0b80 0x7f8130153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813619b700 1 -- 172.21.15.60:0/4269499277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81300d0b80 msgr2=0x7f8130153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813619b700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81300d0b80 0x7f8130153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813619b700 1 -- 172.21.15.60:0/4269499277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81300cf8e0 msgr2=0x7f81300cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813619b700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81300cf8e0 0x7f81300cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813619b700 1 -- 172.21.15.60:0/4269499277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200094a0 con 0x7f81300b5ad0 2023-07-13T22:28:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813699c700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81300d0b80 0x7f8130153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.117+0000 7f813619b700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81300b5ad0 0x7f8130154020 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c700 tx=0x7f812c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.118+0000 7f8126ffd700 1 -- 172.21.15.60:0/4269499277 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c005d70 con 0x7f81300b5ad0 2023-07-13T22:28:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.118+0000 7f8126ffd700 1 -- 172.21.15.60:0/4269499277 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812c0070d0 con 0x7f81300b5ad0 2023-07-13T22:28:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.118+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81301568f0 con 0x7f81300b5ad0 2023-07-13T22:28:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.118+0000 7f8126ffd700 1 -- 172.21.15.60:0/4269499277 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c007460 con 0x7f81300b5ad0 2023-07-13T22:28:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.118+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8130156eb0 con 0x7f81300b5ad0 2023-07-13T22:28:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.119+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81300741e0 con 0x7f81300b5ad0 2023-07-13T22:28:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.120+0000 7f8126ffd700 1 -- 172.21.15.60:0/4269499277 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f812c0075c0 con 0x7f81300b5ad0 2023-07-13T22:28:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.121+0000 7f8126ffd700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f811406ad30 0x7f811406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.121+0000 7f8126ffd700 1 -- 172.21.15.60:0/4269499277 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f812c088930 con 0x7f81300b5ad0 2023-07-13T22:28:44.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.122+0000 7f813599a700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f811406ad30 0x7f811406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.122+0000 7f813599a700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f811406ad30 0x7f811406d1e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f81200090b0 tx=0x7f812001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.126+0000 7f8126ffd700 1 -- 172.21.15.60:0/4269499277 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f812c053b50 con 0x7f81300b5ad0 2023-07-13T22:28:44.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.377+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8130159540 con 0x7f81300b5ad0 2023-07-13T22:28:44.356 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:28:44.378+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:28:44.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.382+0000 7f8126ffd700 1 -- 172.21.15.60:0/4269499277 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f812c057170 con 0x7f81300b5ad0 2023-07-13T22:28:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f811406ad30 msgr2=0x7f811406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f811406ad30 0x7f811406d1e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f81200090b0 tx=0x7f812001c040 comp rx=0 tx=0).stop 2023-07-13T22:28:44.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81300b5ad0 msgr2=0x7f8130154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81300b5ad0 0x7f8130154020 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c700 tx=0x7f812c00ca10 comp rx=0 tx=0).stop 2023-07-13T22:28:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 shutdown_connections 2023-07-13T22:28:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f811406ad30 0x7f811406d1e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81300b5ad0 0x7f8130154020 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f81300d0b80 0x7f8130153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 --2- 172.21.15.60:0/4269499277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81300cf8e0 0x7f81300cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 >> 172.21.15.60:0/4269499277 conn(0x7f8130071660 msgr2=0x7f81300b7770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 shutdown_connections 2023-07-13T22:28:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.385+0000 7f81383ff700 1 -- 172.21.15.60:0/4269499277 wait complete. 2023-07-13T22:28:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:28:44.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:28:44.437 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:28:44.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.617+0000 7fd513fff700 1 Processor -- start 2023-07-13T22:28:44.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.617+0000 7fd513fff700 1 -- start start 2023-07-13T22:28:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.618+0000 7fd513fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd50c15c620 0x7fd50c160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.618+0000 7fd513fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd50c0b4e50 0x7fd50c0d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.618+0000 7fd513fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c161020 con 0x7fd50c0cf280 2023-07-13T22:28:44.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.618+0000 7fd513fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c0cd310 con 0x7fd50c15c620 2023-07-13T22:28:44.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.618+0000 7fd513fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c15c9f0 con 0x7fd50c0b4e50 2023-07-13T22:28:44.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.618+0000 7fd51159a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd50c0cf280 0x7fd50c0b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60440/0 (socket says 172.21.15.60:60440) 2023-07-13T22:28:44.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd51159a700 1 -- 172.21.15.60:0/1615837209 learned_addr learned my addr 172.21.15.60:0/1615837209 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd51259c700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd50c0b4e50 0x7fd50c0d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd511d9b700 1 --2- 172.21.15.60:0/1615837209 >> v2:172.21.15.105:3300/0 conn(0x7fd50c15c620 0x7fd50c160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829878183 0 0) 0x7fd4fc001110 con 0x7fd50c0cf280 2023-07-13T22:28:44.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4f80035b0 con 0x7fd50c0cf280 2023-07-13T22:28:44.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3845122560 0 0) 0x7fd4f80035b0 con 0x7fd50c0cf280 2023-07-13T22:28:44.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd50c0b4e50 msgr2=0x7fd50c0d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:44.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd50c0b4e50 0x7fd50c0d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 >> v2:172.21.15.105:3300/0 conn(0x7fd50c15c620 msgr2=0x7fd50c160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:44.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 --2- 172.21.15.60:0/1615837209 >> v2:172.21.15.105:3300/0 conn(0x7fd50c15c620 0x7fd50c160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0cd560 con 0x7fd50c0cf280 2023-07-13T22:28:44.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fd4fc001800 con 0x7fd50c0cf280 2023-07-13T22:28:44.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 >> v1:172.21.15.60:6789/0 conn(0x7fd50c0cf280 legacy=0x7fd50c0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.619+0000 7fd510d99700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8003a00 0x7fd4f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd510d99700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8004850 0x7fd4f8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd510d99700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd50c0b4e50 0x7fd4f8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0cd310 con 0x7fd4f8003a00 2023-07-13T22:28:44.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c15c9f0 con 0x7fd4f8004850 2023-07-13T22:28:44.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f800ba40 con 0x7fd50c0b4e50 2023-07-13T22:28:44.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd511d9b700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8003a00 0x7fd4f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd51259c700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd50c0b4e50 0x7fd4f8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd51159a700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8004850 0x7fd4f8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd511d9b700 1 -- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd50c0b4e50 msgr2=0x7fd4f8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:44.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd511d9b700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd50c0b4e50 0x7fd4f8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd511d9b700 1 -- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8004850 msgr2=0x7fd4f8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:44.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd511d9b700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8004850 0x7fd4f8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd511d9b700 1 -- 172.21.15.60:0/1615837209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f80035b0 con 0x7fd4f8003a00 2023-07-13T22:28:44.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.620+0000 7fd511d9b700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8003a00 0x7fd4f8003e10 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd508009b60 tx=0x7fd4f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc792062f6cad5a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:44.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50800e6c0 con 0x7fd4f8003a00 2023-07-13T22:28:44.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50800e820 con 0x7fd4f8003a00 2023-07-13T22:28:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd510d99700 1 -- 172.21.15.60:0/1615837209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50800eb90 con 0x7fd4f8003a00 2023-07-13T22:28:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd513fff700 1 -- 172.21.15.60:0/1615837209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8003a00 msgr2=0x7fd4f8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd513fff700 1 --2- 172.21.15.60:0/1615837209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8003a00 0x7fd4f8003e10 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd508009b60 tx=0x7fd4f8005880 comp rx=0 tx=0).stop 2023-07-13T22:28:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd51259c700 1 -- 172.21.15.60:0/1615837209 reap_dead start 2023-07-13T22:28:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd513fff700 1 -- 172.21.15.60:0/1615837209 shutdown_connections 2023-07-13T22:28:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd513fff700 1 -- 172.21.15.60:0/1615837209 >> 172.21.15.60:0/1615837209 conn(0x7fd50c06b210 msgr2=0x7fd50c0cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd513fff700 1 -- 172.21.15.60:0/1615837209 shutdown_connections 2023-07-13T22:28:44.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.621+0000 7fd513fff700 1 -- 172.21.15.60:0/1615837209 wait complete. 2023-07-13T22:28:44.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.623+0000 7fd513fff700 1 Processor -- start 2023-07-13T22:28:44.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.623+0000 7fd513fff700 1 -- start start 2023-07-13T22:28:44.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.624+0000 7fd513fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4f8003a00 0x7fd50c0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.624+0000 7fd513fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c0c67f0 0x7fd50c0c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.624+0000 7fd513fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c147ff0 0x7fd50c148460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.624+0000 7fd513fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c1e6f20 con 0x7fd50c0c67f0 2023-07-13T22:28:44.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.624+0000 7fd513fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0c71a0 con 0x7fd50c147ff0 2023-07-13T22:28:44.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.624+0000 7fd513fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0c8eb0 con 0x7fd4f8003a00 2023-07-13T22:28:44.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.624+0000 7fd51159a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c0c67f0 0x7fd50c0c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.624+0000 7fd51159a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c0c67f0 0x7fd50c0c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39816/0 (socket says 172.21.15.60:39816) 2023-07-13T22:28:44.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.624+0000 7fd51159a700 1 -- 172.21.15.60:0/1260762776 learned_addr learned my addr 172.21.15.60:0/1260762776 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:44.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd51259c700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c147ff0 0x7fd50c148460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd51159a700 1 -- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4f8003a00 msgr2=0x7fd50c0c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd511d9b700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4f8003a00 0x7fd50c0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd51159a700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4f8003a00 0x7fd50c0c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd51159a700 1 -- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c147ff0 msgr2=0x7fd50c148460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd51159a700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c147ff0 0x7fd50c148460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd51159a700 1 -- 172.21.15.60:0/1260762776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0c9190 con 0x7fd50c0c67f0 2023-07-13T22:28:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd51259c700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c147ff0 0x7fd50c148460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd511d9b700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4f8003a00 0x7fd50c0c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd51159a700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c0c67f0 0x7fd50c0c6c60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00e7b0 tx=0x7fd4fc00eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:44.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd5027fc700 1 -- 172.21.15.60:0/1260762776 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc012590 con 0x7fd50c0c67f0 2023-07-13T22:28:44.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd5027fc700 1 -- 172.21.15.60:0/1260762776 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc012710 con 0x7fd50c0c67f0 2023-07-13T22:28:44.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c0c9470 con 0x7fd50c0c67f0 2023-07-13T22:28:44.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd5027fc700 1 -- 172.21.15.60:0/1260762776 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc012ac0 con 0x7fd50c0c67f0 2023-07-13T22:28:44.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.625+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c1489a0 con 0x7fd50c0c67f0 2023-07-13T22:28:44.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.627+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c1e7060 con 0x7fd50c0c67f0 2023-07-13T22:28:44.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.627+0000 7fd5027fc700 1 -- 172.21.15.60:0/1260762776 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd4fc012c20 con 0x7fd50c0c67f0 2023-07-13T22:28:44.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.629+0000 7fd5027fc700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4f006ace0 0x7fd4f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:44.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.629+0000 7fd5027fc700 1 -- 172.21.15.60:0/1260762776 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fd4fc089240 con 0x7fd50c0c67f0 2023-07-13T22:28:44.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.629+0000 7fd511d9b700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4f006ace0 0x7fd4f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:44.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.630+0000 7fd511d9b700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4f006ace0 0x7fd4f006d190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd50800a010 tx=0x7fd508008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:44.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.633+0000 7fd5027fc700 1 -- 172.21.15.60:0/1260762776 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd50c1e7060 con 0x7fd50c0c67f0 2023-07-13T22:28:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:44.886+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd50c161020 con 0x7fd50c0c67f0 2023-07-13T22:28:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.383+0000 7fd5027fc700 1 -- 172.21.15.60:0/1260762776 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd50c161020 con 0x7fd50c0c67f0 2023-07-13T22:28:45.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.386+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4f006ace0 msgr2=0x7fd4f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.386+0000 7fd513fff700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4f006ace0 0x7fd4f006d190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd50800a010 tx=0x7fd508008030 comp rx=0 tx=0).stop 2023-07-13T22:28:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.386+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c0c67f0 msgr2=0x7fd50c0c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.386+0000 7fd513fff700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c0c67f0 0x7fd50c0c6c60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00e7b0 tx=0x7fd4fc00eb70 comp rx=0 tx=0).stop 2023-07-13T22:28:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.387+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 shutdown_connections 2023-07-13T22:28:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.387+0000 7fd513fff700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd4f006ace0 0x7fd4f006d190 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.387+0000 7fd513fff700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd50c147ff0 0x7fd50c148460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.387+0000 7fd513fff700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd4f8003a00 0x7fd50c0c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:45.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.387+0000 7fd513fff700 1 --2- 172.21.15.60:0/1260762776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd50c0c67f0 0x7fd50c0c6c60 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:45.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.387+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 >> 172.21.15.60:0/1260762776 conn(0x7fd50c06b210 msgr2=0x7fd50c15b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:45.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.387+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 shutdown_connections 2023-07-13T22:28:45.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.387+0000 7fd513fff700 1 -- 172.21.15.60:0/1260762776 wait complete. 2023-07-13T22:28:45.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:28:45.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.620+0000 7f890a3ff700 1 Processor -- start 2023-07-13T22:28:45.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.620+0000 7f890a3ff700 1 -- start start 2023-07-13T22:28:45.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.621+0000 7f890a3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89040cd530 0x7f89040ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:45.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.621+0000 7f890a3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f89040b2e00 0x7f89040b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:45.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.621+0000 7f890a3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89040d2ce0 con 0x7f89040d0ab0 2023-07-13T22:28:45.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.621+0000 7f890a3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89040ceb30 con 0x7f89040b2e00 2023-07-13T22:28:45.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.621+0000 7f890a3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89040cee10 con 0x7f89040cd530 2023-07-13T22:28:45.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.622+0000 7f8903fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f89040d0ab0 0x7f89040d0e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43042/0 (socket says 172.21.15.60:43042) 2023-07-13T22:28:45.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.622+0000 7f8903fff700 1 -- 172.21.15.60:0/2428098478 learned_addr learned my addr 172.21.15.60:0/2428098478 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:45.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.622+0000 7f890899c700 1 --2- 172.21.15.60:0/2428098478 >> v2:172.21.15.60:3300/0 conn(0x7f89040b2e00 0x7f89040b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:45.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.622+0000 7f89037fe700 1 --2- 172.21.15.60:0/2428098478 >> v2:172.21.15.105:3300/0 conn(0x7f89040cd530 0x7f89040ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:45.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.622+0000 7f8902ffd700 1 -- 172.21.15.60:0/2428098478 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902839388 0 0) 0x7f89040d2ce0 con 0x7f89040d0ab0 2023-07-13T22:28:45.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.622+0000 7f890899c700 1 -- 172.21.15.60:0/2428098478 >> v2:172.21.15.105:3300/0 conn(0x7f89040cd530 msgr2=0x7f89040ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:45.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.622+0000 7f890899c700 1 --2- 172.21.15.60:0/2428098478 >> v2:172.21.15.105:3300/0 conn(0x7f89040cd530 0x7f89040ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:45.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.622+0000 7f890899c700 1 -- 172.21.15.60:0/2428098478 >> v1:172.21.15.60:6789/0 conn(0x7f89040d0ab0 legacy=0x7f89040d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:45.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.622+0000 7f890899c700 1 -- 172.21.15.60:0/2428098478 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89040cf0f0 con 0x7f89040b2e00 2023-07-13T22:28:45.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f890899c700 1 --2- 172.21.15.60:0/2428098478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040b2e00 0x7f89040b3280 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f890408ba40 tx=0x7f88fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a55cdda05246bae server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:45.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f8902ffd700 1 -- 172.21.15.60:0/2428098478 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00e070 con 0x7f89040b2e00 2023-07-13T22:28:45.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f8902ffd700 1 -- 172.21.15.60:0/2428098478 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88fc00bb20 con 0x7f89040b2e00 2023-07-13T22:28:45.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f8902ffd700 1 -- 172.21.15.60:0/2428098478 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc012430 con 0x7f89040b2e00 2023-07-13T22:28:45.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f890a3ff700 1 -- 172.21.15.60:0/2428098478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040b2e00 msgr2=0x7f89040b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:45.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f890a3ff700 1 --2- 172.21.15.60:0/2428098478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040b2e00 0x7f89040b3280 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f890408ba40 tx=0x7f88fc00b330 comp rx=0 tx=0).stop 2023-07-13T22:28:45.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f890a3ff700 1 -- 172.21.15.60:0/2428098478 shutdown_connections 2023-07-13T22:28:45.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f890a3ff700 1 --2- 172.21.15.60:0/2428098478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040b2e00 0x7f89040b3280 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:45.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f890a3ff700 1 --2- 172.21.15.60:0/2428098478 >> v2:172.21.15.105:3300/0 conn(0x7f89040cd530 0x7f89040ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:45.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f890a3ff700 1 -- 172.21.15.60:0/2428098478 >> 172.21.15.60:0/2428098478 conn(0x7f890409c170 msgr2=0x7f89040d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:45.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.623+0000 7f890a3ff700 1 -- 172.21.15.60:0/2428098478 shutdown_connections 2023-07-13T22:28:45.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.624+0000 7f890a3ff700 1 -- 172.21.15.60:0/2428098478 wait complete. 2023-07-13T22:28:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.625+0000 7f890a3ff700 1 Processor -- start 2023-07-13T22:28:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.625+0000 7f890a3ff700 1 -- start start 2023-07-13T22:28:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.626+0000 7f890a3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89040b2e00 0x7f89041592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.626+0000 7f890a3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89040cd530 0x7f89040cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.626+0000 7f890a3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040d0ab0 0x7f89040b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:45.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.626+0000 7f890a3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89040cb080 con 0x7f89040d0ab0 2023-07-13T22:28:45.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.626+0000 7f890a3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89040ca400 con 0x7f89040b2e00 2023-07-13T22:28:45.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.626+0000 7f890a3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89040ca6e0 con 0x7f89040cd530 2023-07-13T22:28:45.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f890899c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040d0ab0 0x7f89040b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:45.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f890899c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040d0ab0 0x7f89040b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47984/0 (socket says 172.21.15.60:47984) 2023-07-13T22:28:45.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f890899c700 1 -- 172.21.15.60:0/3175701524 learned_addr learned my addr 172.21.15.60:0/3175701524 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:45.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f8903fff700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89040b2e00 0x7f89041592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:45.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f89037fe700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89040cd530 0x7f89040cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:45.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f890899c700 1 -- 172.21.15.60:0/3175701524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89040cd530 msgr2=0x7f89040cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:45.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f890899c700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89040cd530 0x7f89040cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:45.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f890899c700 1 -- 172.21.15.60:0/3175701524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89040b2e00 msgr2=0x7f89041592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:45.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f890899c700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89040b2e00 0x7f89041592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:45.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f890899c700 1 -- 172.21.15.60:0/3175701524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc009a70 con 0x7f89040d0ab0 2023-07-13T22:28:45.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f89037fe700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89040cd530 0x7f89040cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:45.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f890899c700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040d0ab0 0x7f89040b4220 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f88fc014040 tx=0x7f88fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:45.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f8900ff9700 1 -- 172.21.15.60:0/3175701524 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc012430 con 0x7f89040d0ab0 2023-07-13T22:28:45.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f8900ff9700 1 -- 172.21.15.60:0/3175701524 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88fc020070 con 0x7f89040d0ab0 2023-07-13T22:28:45.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.627+0000 7f8900ff9700 1 -- 172.21.15.60:0/3175701524 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc004280 con 0x7f89040d0ab0 2023-07-13T22:28:45.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.628+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89040ca9c0 con 0x7f89040d0ab0 2023-07-13T22:28:45.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.628+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89040c9fb0 con 0x7f89040d0ab0 2023-07-13T22:28:45.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.629+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8904159940 con 0x7f89040d0ab0 2023-07-13T22:28:45.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.630+0000 7f8900ff9700 1 -- 172.21.15.60:0/3175701524 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f88fc0043e0 con 0x7f89040d0ab0 2023-07-13T22:28:45.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.631+0000 7f8900ff9700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88f006ace0 0x7f88f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:45.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.631+0000 7f8900ff9700 1 -- 172.21.15.60:0/3175701524 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f88fc022070 con 0x7f89040d0ab0 2023-07-13T22:28:45.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.631+0000 7f8903fff700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88f006ace0 0x7f88f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:45.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.632+0000 7f8903fff700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88f006ace0 0x7f88f006d190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007fd0 tx=0x7f88f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:45.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.635+0000 7f8900ff9700 1 -- 172.21.15.60:0/3175701524 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8904159940 con 0x7f89040d0ab0 2023-07-13T22:28:45.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:45.845+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8904159940 con 0x7f89040d0ab0 2023-07-13T22:28:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.394+0000 7f8900ff9700 1 -- 172.21.15.60:0/3175701524 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8904159940 con 0x7f89040d0ab0 2023-07-13T22:28:46.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88f006ace0 msgr2=0x7f88f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88f006ace0 0x7f88f006d190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007fd0 tx=0x7f88f4009040 comp rx=0 tx=0).stop 2023-07-13T22:28:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040d0ab0 msgr2=0x7f89040b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040d0ab0 0x7f89040b4220 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f88fc014040 tx=0x7f88fc004ad0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 shutdown_connections 2023-07-13T22:28:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88f006ace0 0x7f88f006d190 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89040d0ab0 0x7f89040b4220 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89040b2e00 0x7f89041592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 --2- 172.21.15.60:0/3175701524 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89040cd530 0x7f89040cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.401+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 >> 172.21.15.60:0/3175701524 conn(0x7f890409c170 msgr2=0x7f890415b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:46.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.402+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 shutdown_connections 2023-07-13T22:28:46.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.402+0000 7f890a3ff700 1 -- 172.21.15.60:0/3175701524 wait complete. 2023-07-13T22:28:46.384 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:28:46.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:28:46.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.644+0000 7ff480fff700 1 Processor -- start 2023-07-13T22:28:46.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.645+0000 7ff480fff700 1 -- start start 2023-07-13T22:28:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.646+0000 7ff480fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0d0ab0 0x7ff47c0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.646+0000 7ff480fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff47c0cd530 0x7ff47c0d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.646+0000 7ff480fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47c0d2ce0 con 0x7ff47c0cf870 2023-07-13T22:28:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.646+0000 7ff480fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47c0d13c0 con 0x7ff47c0cd530 2023-07-13T22:28:46.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.646+0000 7ff480fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0ceb30 con 0x7ff47c0d0ab0 2023-07-13T22:28:46.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.646+0000 7ff473fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff47c0cd530 0x7ff47c0d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:46.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.646+0000 7ff473fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff47c0cd530 0x7ff47c0d0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47986/0 (socket says 172.21.15.60:47986) 2023-07-13T22:28:46.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.646+0000 7ff473fff700 1 -- 172.21.15.60:0/2044858619 learned_addr learned my addr 172.21.15.60:0/2044858619 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.647+0000 7ff47a59c700 1 --2- 172.21.15.60:0/2044858619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0d0ab0 0x7ff47c0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.647+0000 7ff473fff700 1 -- 172.21.15.60:0/2044858619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0d0ab0 msgr2=0x7ff47c0ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.647+0000 7ff473fff700 1 --2- 172.21.15.60:0/2044858619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0d0ab0 0x7ff47c0ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.647+0000 7ff473fff700 1 -- 172.21.15.60:0/2044858619 >> v1:172.21.15.105:6789/0 conn(0x7ff47c0cf870 legacy=0x7ff47c0cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:46.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.647+0000 7ff473fff700 1 -- 172.21.15.60:0/2044858619 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c0cee10 con 0x7ff47c0cd530 2023-07-13T22:28:46.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.647+0000 7ff473fff700 1 --2- 172.21.15.60:0/2044858619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 0x7ff47c0d0e80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff4640098f0 tx=0x7ff464009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749947b0a3ffca5d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:46.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.647+0000 7ff479d9b700 1 -- 172.21.15.60:0/2044858619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400b8a0 con 0x7ff47c0cd530 2023-07-13T22:28:46.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.647+0000 7ff479d9b700 1 -- 172.21.15.60:0/2044858619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46400ba00 con 0x7ff47c0cd530 2023-07-13T22:28:46.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.647+0000 7ff479d9b700 1 -- 172.21.15.60:0/2044858619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400bd90 con 0x7ff47c0cd530 2023-07-13T22:28:46.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.648+0000 7ff480fff700 1 -- 172.21.15.60:0/2044858619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 msgr2=0x7ff47c0d0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:46.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.648+0000 7ff480fff700 1 --2- 172.21.15.60:0/2044858619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 0x7ff47c0d0e80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff4640098f0 tx=0x7ff464009c00 comp rx=0 tx=0).stop 2023-07-13T22:28:46.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.648+0000 7ff480fff700 1 -- 172.21.15.60:0/2044858619 shutdown_connections 2023-07-13T22:28:46.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.648+0000 7ff480fff700 1 --2- 172.21.15.60:0/2044858619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0d0ab0 0x7ff47c0ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.648+0000 7ff480fff700 1 --2- 172.21.15.60:0/2044858619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 0x7ff47c0d0e80 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.648+0000 7ff480fff700 1 -- 172.21.15.60:0/2044858619 >> 172.21.15.60:0/2044858619 conn(0x7ff47c09c170 msgr2=0x7ff47c0d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.648+0000 7ff480fff700 1 -- 172.21.15.60:0/2044858619 shutdown_connections 2023-07-13T22:28:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.648+0000 7ff480fff700 1 -- 172.21.15.60:0/2044858619 wait complete. 2023-07-13T22:28:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.650+0000 7ff480fff700 1 Processor -- start 2023-07-13T22:28:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.650+0000 7ff480fff700 1 -- start start 2023-07-13T22:28:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff480fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 0x7ff47c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff480fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0cf870 0x7ff47c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff480fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff47c0d0ab0 0x7ff47c0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff480fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1585b0 con 0x7ff47c0cd530 2023-07-13T22:28:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff480fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c157930 con 0x7ff47c0d0ab0 2023-07-13T22:28:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff480fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c157c10 con 0x7ff47c0cf870 2023-07-13T22:28:46.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff47a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 0x7ff47c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff47a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 0x7ff47c0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48002/0 (socket says 172.21.15.60:48002) 2023-07-13T22:28:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff47a59c700 1 -- 172.21.15.60:0/2928197987 learned_addr learned my addr 172.21.15.60:0/2928197987 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff473fff700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0cf870 0x7ff47c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.651+0000 7ff47ad9d700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff47c0d0ab0 0x7ff47c0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff47a59c700 1 -- 172.21.15.60:0/2928197987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0cf870 msgr2=0x7ff47c0cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff47a59c700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0cf870 0x7ff47c0cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff47a59c700 1 -- 172.21.15.60:0/2928197987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff47c0d0ab0 msgr2=0x7ff47c0b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:46.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff47a59c700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff47c0d0ab0 0x7ff47c0b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff47a59c700 1 -- 172.21.15.60:0/2928197987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464009550 con 0x7ff47c0cd530 2023-07-13T22:28:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff47a59c700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 0x7ff47c0c9760 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00d7d0 tx=0x7ff46c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff472ffd700 1 -- 172.21.15.60:0/2928197987 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c017070 con 0x7ff47c0cd530 2023-07-13T22:28:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff472ffd700 1 -- 172.21.15.60:0/2928197987 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c005a20 con 0x7ff47c0cd530 2023-07-13T22:28:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c157ef0 con 0x7ff47c0cd530 2023-07-13T22:28:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff472ffd700 1 -- 172.21.15.60:0/2928197987 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c005bd0 con 0x7ff47c0cd530 2023-07-13T22:28:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.652+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c0c9ca0 con 0x7ff47c0cd530 2023-07-13T22:28:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.653+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c04e210 con 0x7ff47c0cd530 2023-07-13T22:28:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.654+0000 7ff472ffd700 1 -- 172.21.15.60:0/2928197987 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff46c005d30 con 0x7ff47c0cd530 2023-07-13T22:28:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.655+0000 7ff472ffd700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff45c06ace0 0x7ff45c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.655+0000 7ff472ffd700 1 -- 172.21.15.60:0/2928197987 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5826+0+0 (secure 0 0 0) 0x7ff46c088e60 con 0x7ff47c0cd530 2023-07-13T22:28:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.655+0000 7ff473fff700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff45c06ace0 0x7ff45c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.656+0000 7ff473fff700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff45c06ace0 0x7ff45c06d190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff464009c40 tx=0x7ff464008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:46.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.659+0000 7ff472ffd700 1 -- 172.21.15.60:0/2928197987 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff46c053f50 con 0x7ff47c0cd530 2023-07-13T22:28:46.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:46.864+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff47c0afb90 con 0x7ff47c0cd530 2023-07-13T22:28:47.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.674+0000 7ff472ffd700 1 -- 172.21.15.60:0/2928197987 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff46c057570 con 0x7ff47c0cd530 2023-07-13T22:28:47.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.679+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff45c06ace0 msgr2=0x7ff45c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:47.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.679+0000 7ff480fff700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff45c06ace0 0x7ff45c06d190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff464009c40 tx=0x7ff464008030 comp rx=0 tx=0).stop 2023-07-13T22:28:47.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.679+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 msgr2=0x7ff47c0c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:47.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.679+0000 7ff480fff700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 0x7ff47c0c9760 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00d7d0 tx=0x7ff46c00dae0 comp rx=0 tx=0).stop 2023-07-13T22:28:47.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.682+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 shutdown_connections 2023-07-13T22:28:47.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.682+0000 7ff480fff700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff45c06ace0 0x7ff45c06d190 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.682+0000 7ff480fff700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff47c0cd530 0x7ff47c0c9760 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.682+0000 7ff480fff700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff47c0d0ab0 0x7ff47c0b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.682+0000 7ff480fff700 1 --2- 172.21.15.60:0/2928197987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47c0cf870 0x7ff47c0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:47.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.682+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 >> 172.21.15.60:0/2928197987 conn(0x7ff47c09c170 msgr2=0x7ff47c0cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:47.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.684+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 shutdown_connections 2023-07-13T22:28:47.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:47.685+0000 7ff480fff700 1 -- 172.21.15.60:0/2928197987 wait complete. 2023-07-13T22:28:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:28:47.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:28:47.696 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:28:47.827 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2023-07-13T22:28:47.827 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2023-07-13T22:28:47.827 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:28:47.827 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2023-07-13T22:28:47.827 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.828 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2023-07-13T22:28:47.828 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.828 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2023-07-13T22:28:47.828 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.828 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2023-07-13T22:28:47.828 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2023-07-13T22:28:47.828 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.829 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2023-07-13T22:28:47.829 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2023-07-13T22:28:47.829 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.829 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:28:47.829 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2023-07-13T22:28:47.829 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.829 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:28:47.829 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:28:47.830 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.830 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2023-07-13T22:28:47.830 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2023-07-13T22:28:47.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2023-07-13T22:28:47.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:28:47.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:28:47.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2023-07-13T22:28:47.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:28:47.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2023-07-13T22:28:47.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:28:47.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:28:47.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2023-07-13T22:28:47.831 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.831 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:28:47.831 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.832 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2023-07-13T22:28:47.832 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.832 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2023-07-13T22:28:47.832 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.832 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2023-07-13T22:28:47.832 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2023-07-13T22:28:47.833 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2023-07-13T22:28:47.833 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:28:47.833 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:28:47.833 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.833 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2023-07-13T22:28:47.833 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2023-07-13T22:28:47.833 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.834 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2023-07-13T22:28:47.834 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2023-07-13T22:28:47.834 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:28:47.834 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.834 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2023-07-13T22:28:47.834 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2023-07-13T22:28:47.834 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.834 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2023-07-13T22:28:47.835 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2023-07-13T22:28:47.835 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2023-07-13T22:28:47.835 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.835 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2023-07-13T22:28:47.835 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2023-07-13T22:28:47.835 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.835 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:28:47.835 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.836 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2023-07-13T22:28:47.836 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.836 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2023-07-13T22:28:47.836 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.836 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:28:47.836 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2023-07-13T22:28:47.836 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.836 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2023-07-13T22:28:47.837 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2023-07-13T22:28:47.837 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2023-07-13T22:28:47.837 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2023-07-13T22:28:47.837 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2023-07-13T22:28:47.837 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2023-07-13T22:28:47.837 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2023-07-13T22:28:47.837 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2023-07-13T22:28:47.837 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2023-07-13T22:28:47.838 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:47.838 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2023-07-13T22:28:47.838 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2023-07-13T22:28:47.838 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:28:47.838 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2023-07-13T22:28:47.838 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:28:47.838 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:28:47.838 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2023-07-13T22:28:47.839 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:28:47.839 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.839 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2023-07-13T22:28:47.839 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:47.839 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2023-07-13T22:28:47.839 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:28:47.839 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:28:47.839 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:28:47.840 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:28:47.840 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2023-07-13T22:28:47.840 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.840 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2023-07-13T22:28:47.840 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:47.840 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:28:47.840 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2023-07-13T22:28:47.840 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:28:47.841 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2023-07-13T22:28:47.841 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2023-07-13T22:28:47.841 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:28:47.841 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2023-07-13T22:28:47.841 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2023-07-13T22:28:47.841 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:28:47.841 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.841 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:28:47.842 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:28:47.842 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:28:47.842 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.842 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:28:47.842 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:28:47.842 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:28:47.842 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.842 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2023-07-13T22:28:47.843 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:28:47.843 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.843 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:28:47.843 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2023-07-13T22:28:47.843 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:47.843 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2023-07-13T22:28:47.843 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2023-07-13T22:28:47.844 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2023-07-13T22:28:47.844 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2023-07-13T22:28:47.844 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:28:47.844 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:28:47.844 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2023-07-13T22:28:47.844 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2023-07-13T22:28:47.845 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:47.845 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2023-07-13T22:28:47.845 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:28:47.845 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:28:47.845 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2023-07-13T22:28:47.845 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2023-07-13T22:28:47.845 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2023-07-13T22:28:47.845 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-07-13T22:28:47.846 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-07-13T22:28:47.846 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-07-13T22:28:47.846 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-07-13T22:28:47.846 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2023-07-13T22:28:47.846 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2023-07-13T22:28:47.846 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2023-07-13T22:28:47.846 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:28:47.846 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2023-07-13T22:28:47.847 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2023-07-13T22:28:47.847 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2023-07-13T22:28:47.847 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2023-07-13T22:28:47.847 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:28:47.847 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:28:47.847 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:28:47.848 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-07-13T22:28:47.848 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-07-13T22:28:47.848 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:28:47.848 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-07-13T22:28:47.848 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.848 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-07-13T22:28:47.849 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.849 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-07-13T22:28:47.849 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.849 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-07-13T22:28:47.849 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-07-13T22:28:47.849 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.849 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-07-13T22:28:47.849 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-07-13T22:28:47.850 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.850 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:28:47.850 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-07-13T22:28:47.850 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.850 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:28:47.850 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:28:47.850 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.850 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-07-13T22:28:47.851 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.851 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-07-13T22:28:47.851 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-07-13T22:28:47.851 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:28:47.851 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:28:47.851 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-07-13T22:28:47.851 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:28:47.851 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-07-13T22:28:47.852 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:28:47.852 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:28:47.852 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-07-13T22:28:47.852 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.852 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:28:47.852 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.852 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-07-13T22:28:47.853 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.853 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-07-13T22:28:47.854 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.854 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-07-13T22:28:47.854 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-07-13T22:28:47.854 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-07-13T22:28:47.854 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:28:47.855 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:28:47.855 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.855 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-07-13T22:28:47.855 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-07-13T22:28:47.855 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.855 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-07-13T22:28:47.855 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2023-07-13T22:28:47.855 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:28:47.856 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.856 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-07-13T22:28:47.856 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-07-13T22:28:47.856 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.856 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-07-13T22:28:47.857 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-07-13T22:28:47.857 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-07-13T22:28:47.857 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.857 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-07-13T22:28:47.858 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-07-13T22:28:47.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.858 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:28:47.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.859 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-07-13T22:28:47.859 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.859 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-07-13T22:28:47.859 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.859 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:28:47.859 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-07-13T22:28:47.859 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.859 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-07-13T22:28:47.860 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-07-13T22:28:47.860 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2023-07-13T22:28:47.860 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2023-07-13T22:28:47.860 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2023-07-13T22:28:47.860 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2023-07-13T22:28:47.860 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2023-07-13T22:28:47.860 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2023-07-13T22:28:47.861 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2023-07-13T22:28:47.861 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:47.861 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-07-13T22:28:47.861 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2023-07-13T22:28:47.861 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:28:47.861 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2023-07-13T22:28:47.861 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:28:47.861 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:28:47.862 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2023-07-13T22:28:47.862 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:28:47.862 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.862 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-07-13T22:28:47.862 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:47.862 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-07-13T22:28:47.862 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:28:47.862 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:28:47.863 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:28:47.863 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:28:47.863 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-07-13T22:28:47.863 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.863 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-07-13T22:28:47.863 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:47.863 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:28:47.863 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-07-13T22:28:47.864 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:28:47.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-07-13T22:28:47.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-07-13T22:28:47.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:28:47.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-07-13T22:28:47.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-07-13T22:28:47.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:28:47.865 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.865 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:28:47.865 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:28:47.865 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:28:47.865 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.865 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:28:47.865 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:28:47.865 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:28:47.866 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.866 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-07-13T22:28:47.866 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:28:47.866 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.866 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:28:47.866 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2023-07-13T22:28:47.866 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:28:47.866 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-07-13T22:28:47.867 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-07-13T22:28:47.867 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-07-13T22:28:47.867 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-07-13T22:28:47.867 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:28:47.867 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:28:47.867 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2023-07-13T22:28:47.867 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2023-07-13T22:28:47.868 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:47.868 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2023-07-13T22:28:47.868 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:28:47.868 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:28:47.868 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-07-13T22:28:47.868 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2023-07-13T22:28:47.868 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2023-07-13T22:28:47.868 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-07-13T22:28:47.869 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-07-13T22:28:47.869 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-07-13T22:28:47.869 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-07-13T22:28:47.869 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2023-07-13T22:28:47.869 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2023-07-13T22:28:47.869 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2023-07-13T22:28:47.869 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:28:47.869 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2023-07-13T22:28:47.870 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2023-07-13T22:28:47.870 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2023-07-13T22:28:47.870 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2023-07-13T22:28:47.870 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:28:47.870 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:28:47.870 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:28:47.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-13T22:28:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.097+0000 7f57ed3ff700 1 Processor -- start 2023-07-13T22:28:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.097+0000 7f57ed3ff700 1 -- start start 2023-07-13T22:28:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57ed3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57ed3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f57e80caf80 0x7f57e80ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57ed3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e806bf40 con 0x7f57e806c8f0 2023-07-13T22:28:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57ed3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e806b9f0 con 0x7f57e80caf80 2023-07-13T22:28:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57ed3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e806c750 con 0x7f57e815c460 2023-07-13T22:28:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57e67fc700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f57e80caf80 0x7f57e80ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57e6ffd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57e67fc700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f57e80caf80 0x7f57e80ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:55034/0 (socket says 172.21.15.60:55034) 2023-07-13T22:28:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57e6ffd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e8070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:46450/0 (socket says 172.21.15.60:46450) 2023-07-13T22:28:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.099+0000 7f57e67fc700 1 -- 172.21.15.60:0/2958750973 learned_addr learned my addr 172.21.15.60:0/2958750973 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e6ffd700 1 -- 172.21.15.60:0/2958750973 >> v2:172.21.15.105:3300/0 conn(0x7f57e80caf80 msgr2=0x7f57e80ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e6ffd700 1 --2- 172.21.15.60:0/2958750973 >> v2:172.21.15.105:3300/0 conn(0x7f57e80caf80 0x7f57e80ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e6ffd700 1 -- 172.21.15.60:0/2958750973 >> v1:172.21.15.105:6789/0 conn(0x7f57e806c8f0 legacy=0x7f57e815fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e6ffd700 1 -- 172.21.15.60:0/2958750973 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e80cb3f0 con 0x7f57e815c460 2023-07-13T22:28:48.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e67fc700 1 --2- 172.21.15.60:0/2958750973 >> v2:172.21.15.105:3300/0 conn(0x7f57e80caf80 0x7f57e80ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:28:48.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e5ffb700 1 -- 172.21.15.60:0/2958750973 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077786807 0 0) 0x7f57e806bf40 con 0x7f57e806c8f0 2023-07-13T22:28:48.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e6ffd700 1 --2- 172.21.15.60:0/2958750973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e8070b30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009840 tx=0x7f57d8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49766ef2d72d484 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:48.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e5ffb700 1 -- 172.21.15.60:0/2958750973 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8011830 con 0x7f57e815c460 2023-07-13T22:28:48.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e5ffb700 1 -- 172.21.15.60:0/2958750973 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d8011990 con 0x7f57e815c460 2023-07-13T22:28:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.100+0000 7f57e5ffb700 1 -- 172.21.15.60:0/2958750973 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8011d20 con 0x7f57e815c460 2023-07-13T22:28:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.101+0000 7f57ed3ff700 1 -- 172.21.15.60:0/2958750973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 msgr2=0x7f57e8070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.101+0000 7f57ed3ff700 1 --2- 172.21.15.60:0/2958750973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e8070b30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009840 tx=0x7f57d8009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.101+0000 7f57ed3ff700 1 -- 172.21.15.60:0/2958750973 shutdown_connections 2023-07-13T22:28:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.101+0000 7f57ed3ff700 1 --2- 172.21.15.60:0/2958750973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e8070b30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.101+0000 7f57ed3ff700 1 --2- 172.21.15.60:0/2958750973 >> v2:172.21.15.105:3300/0 conn(0x7f57e80caf80 0x7f57e80ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.101+0000 7f57ed3ff700 1 -- 172.21.15.60:0/2958750973 >> 172.21.15.60:0/2958750973 conn(0x7f57e806e580 msgr2=0x7f57e806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.101+0000 7f57ed3ff700 1 -- 172.21.15.60:0/2958750973 shutdown_connections 2023-07-13T22:28:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.101+0000 7f57ed3ff700 1 -- 172.21.15.60:0/2958750973 wait complete. 2023-07-13T22:28:48.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.103+0000 7f57ed3ff700 1 Processor -- start 2023-07-13T22:28:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.103+0000 7f57ed3ff700 1 -- start start 2023-07-13T22:28:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57ed3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e806c8f0 0x7f57e80b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57ed3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80caf80 0x7f57e8146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57ed3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e80c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57ed3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80c7380 con 0x7f57e80caf80 2023-07-13T22:28:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57ed3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80c6ae0 con 0x7f57e806c8f0 2023-07-13T22:28:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57ed3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80c6dc0 con 0x7f57e815c460 2023-07-13T22:28:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80caf80 0x7f57e8146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80caf80 0x7f57e8146d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48018/0 (socket says 172.21.15.60:48018) 2023-07-13T22:28:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e67fc700 1 -- 172.21.15.60:0/3730871486 learned_addr learned my addr 172.21.15.60:0/3730871486 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e6ffd700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e806c8f0 0x7f57e80b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e77fe700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e80c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e67fc700 1 -- 172.21.15.60:0/3730871486 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 msgr2=0x7f57e80c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e67fc700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e80c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e67fc700 1 -- 172.21.15.60:0/3730871486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e806c8f0 msgr2=0x7f57e80b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e67fc700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e806c8f0 0x7f57e80b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.104+0000 7f57e67fc700 1 -- 172.21.15.60:0/3730871486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc009550 con 0x7f57e80caf80 2023-07-13T22:28:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.105+0000 7f57e67fc700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80caf80 0x7f57e8146d30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00ebe0 tx=0x7f57dc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.105+0000 7f57cf7fe700 1 -- 172.21.15.60:0/3730871486 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00c850 con 0x7f57e80caf80 2023-07-13T22:28:48.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.105+0000 7f57cf7fe700 1 -- 172.21.15.60:0/3730871486 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc00c9b0 con 0x7f57e80caf80 2023-07-13T22:28:48.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.105+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d80094a0 con 0x7f57e80caf80 2023-07-13T22:28:48.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.105+0000 7f57cf7fe700 1 -- 172.21.15.60:0/3730871486 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00cd40 con 0x7f57e80caf80 2023-07-13T22:28:48.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.105+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e80b3af0 con 0x7f57e80caf80 2023-07-13T22:28:48.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.106+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e80b3e60 con 0x7f57e80caf80 2023-07-13T22:28:48.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.107+0000 7f57cf7fe700 1 -- 172.21.15.60:0/3730871486 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57dc013470 con 0x7f57e80caf80 2023-07-13T22:28:48.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.108+0000 7f57cf7fe700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57d406ace0 0x7f57d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.108+0000 7f57cf7fe700 1 -- 172.21.15.60:0/3730871486 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f57dc089320 con 0x7f57e80caf80 2023-07-13T22:28:48.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.108+0000 7f57e6ffd700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57d406ace0 0x7f57d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.109+0000 7f57e6ffd700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57d406ace0 0x7f57d406d190 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f57d80090b0 tx=0x7f57d801d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:48.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.112+0000 7f57cf7fe700 1 -- 172.21.15.60:0/3730871486 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f57dc054650 con 0x7f57e80caf80 2023-07-13T22:28:48.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.316+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f57e80b3950 con 0x7f57e80caf80 2023-07-13T22:28:48.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.320+0000 7f57cf7fe700 1 -- 172.21.15.60:0/3730871486 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57e80b3950 con 0x7f57e80caf80 2023-07-13T22:28:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57d406ace0 msgr2=0x7f57d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57d406ace0 0x7f57d406d190 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f57d80090b0 tx=0x7f57d801d040 comp rx=0 tx=0).stop 2023-07-13T22:28:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80caf80 msgr2=0x7f57e8146d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80caf80 0x7f57e8146d30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00ebe0 tx=0x7f57dc00eef0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 shutdown_connections 2023-07-13T22:28:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f57d406ace0 0x7f57d406d190 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f57e815c460 0x7f57e80c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57e806c8f0 0x7f57e80b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 --2- 172.21.15.60:0/3730871486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80caf80 0x7f57e8146d30 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 >> 172.21.15.60:0/3730871486 conn(0x7f57e806e580 msgr2=0x7f57e80cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 shutdown_connections 2023-07-13T22:28:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.322+0000 7f57ed3ff700 1 -- 172.21.15.60:0/3730871486 wait complete. 2023-07-13T22:28:48.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2023-07-13T22:28:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.544+0000 7fe2d91ff700 1 Processor -- start 2023-07-13T22:28:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.545+0000 7fe2d91ff700 1 -- start start 2023-07-13T22:28:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.546+0000 7fe2d91ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2d40d09e0 0x7fe2d40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.546+0000 7fe2d91ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2d40cd1f0 0x7fe2d40d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.546+0000 7fe2d91ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d40d2ed0 con 0x7fe2d40d0db0 2023-07-13T22:28:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.546+0000 7fe2d91ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d40cd960 con 0x7fe2d40d09e0 2023-07-13T22:28:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.546+0000 7fe2d91ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d40cd680 con 0x7fe2d40cd1f0 2023-07-13T22:28:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.546+0000 7fe2d2d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2d40d09e0 0x7fe2d40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.546+0000 7fe2d2d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2d40d09e0 0x7fe2d40cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48026/0 (socket says 172.21.15.60:48026) 2023-07-13T22:28:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.546+0000 7fe2d2d9d700 1 -- 172.21.15.60:0/2191530740 learned_addr learned my addr 172.21.15.60:0/2191530740 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.546+0000 7fe2d259c700 1 --2- 172.21.15.60:0/2191530740 >> v2:172.21.15.105:3300/0 conn(0x7fe2d40cd1f0 0x7fe2d40d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.547+0000 7fe2d2d9d700 1 -- 172.21.15.60:0/2191530740 >> v2:172.21.15.105:3300/0 conn(0x7fe2d40cd1f0 msgr2=0x7fe2d40d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.547+0000 7fe2d2d9d700 1 --2- 172.21.15.60:0/2191530740 >> v2:172.21.15.105:3300/0 conn(0x7fe2d40cd1f0 0x7fe2d40d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.547+0000 7fe2d2d9d700 1 -- 172.21.15.60:0/2191530740 >> v1:172.21.15.105:6789/0 conn(0x7fe2d40d0db0 legacy=0x7fe2d40b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.547+0000 7fe2d2d9d700 1 -- 172.21.15.60:0/2191530740 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d40d2880 con 0x7fe2d40d09e0 2023-07-13T22:28:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.547+0000 7fe2d2d9d700 1 --2- 172.21.15.60:0/2191530740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 0x7fe2d40cf900 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40b25a0 tx=0x7fe2c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de80d13a3c5c695 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.547+0000 7fe2d1d9b700 1 -- 172.21.15.60:0/2191530740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4011830 con 0x7fe2d40d09e0 2023-07-13T22:28:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.547+0000 7fe2d1d9b700 1 -- 172.21.15.60:0/2191530740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c4011990 con 0x7fe2d40d09e0 2023-07-13T22:28:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.547+0000 7fe2d1d9b700 1 -- 172.21.15.60:0/2191530740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4011d20 con 0x7fe2d40d09e0 2023-07-13T22:28:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.548+0000 7fe2d91ff700 1 -- 172.21.15.60:0/2191530740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 msgr2=0x7fe2d40cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.548+0000 7fe2d91ff700 1 --2- 172.21.15.60:0/2191530740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 0x7fe2d40cf900 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40b25a0 tx=0x7fe2c4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.548+0000 7fe2d91ff700 1 -- 172.21.15.60:0/2191530740 shutdown_connections 2023-07-13T22:28:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.548+0000 7fe2d91ff700 1 --2- 172.21.15.60:0/2191530740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 0x7fe2d40cf900 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.548+0000 7fe2d91ff700 1 --2- 172.21.15.60:0/2191530740 >> v2:172.21.15.105:3300/0 conn(0x7fe2d40cd1f0 0x7fe2d40d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.548+0000 7fe2d91ff700 1 -- 172.21.15.60:0/2191530740 >> 172.21.15.60:0/2191530740 conn(0x7fe2d4071600 msgr2=0x7fe2d40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.548+0000 7fe2d91ff700 1 -- 172.21.15.60:0/2191530740 shutdown_connections 2023-07-13T22:28:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.548+0000 7fe2d91ff700 1 -- 172.21.15.60:0/2191530740 wait complete. 2023-07-13T22:28:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.550+0000 7fe2d91ff700 1 Processor -- start 2023-07-13T22:28:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.550+0000 7fe2d91ff700 1 -- start start 2023-07-13T22:28:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d91ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2d40cd1f0 0x7fe2d40ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d91ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 0x7fe2d4159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d91ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2d40d0db0 0x7fe2d41f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d91ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40cbc70 con 0x7fe2d40d09e0 2023-07-13T22:28:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d91ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40cb0d0 con 0x7fe2d40d0db0 2023-07-13T22:28:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d91ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40cb340 con 0x7fe2d40cd1f0 2023-07-13T22:28:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 0x7fe2d4159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 0x7fe2d4159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48034/0 (socket says 172.21.15.60:48034) 2023-07-13T22:28:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d259c700 1 -- 172.21.15.60:0/1307350129 learned_addr learned my addr 172.21.15.60:0/1307350129 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d359e700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2d40d0db0 0x7fe2d41f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d2d9d700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2d40cd1f0 0x7fe2d40ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d259c700 1 -- 172.21.15.60:0/1307350129 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2d40cd1f0 msgr2=0x7fe2d40ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d259c700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2d40cd1f0 0x7fe2d40ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d259c700 1 -- 172.21.15.60:0/1307350129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2d40d0db0 msgr2=0x7fe2d41f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.551+0000 7fe2d259c700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2d40d0db0 0x7fe2d41f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.552+0000 7fe2d259c700 1 -- 172.21.15.60:0/1307350129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c4009550 con 0x7fe2d40d09e0 2023-07-13T22:28:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.552+0000 7fe2d259c700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 0x7fe2d4159f80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800e9e0 tx=0x7fe2c800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.552+0000 7fe2bb7fe700 1 -- 172.21.15.60:0/1307350129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8009710 con 0x7fe2d40d09e0 2023-07-13T22:28:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.552+0000 7fe2bb7fe700 1 -- 172.21.15.60:0/1307350129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c800ee80 con 0x7fe2d40d09e0 2023-07-13T22:28:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.552+0000 7fe2bb7fe700 1 -- 172.21.15.60:0/1307350129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800ca50 con 0x7fe2d40d09e0 2023-07-13T22:28:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.552+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d40cffc0 con 0x7fe2d40d09e0 2023-07-13T22:28:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.552+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d40cb900 con 0x7fe2d40d09e0 2023-07-13T22:28:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.553+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d40cbe20 con 0x7fe2d40d09e0 2023-07-13T22:28:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.554+0000 7fe2bb7fe700 1 -- 172.21.15.60:0/1307350129 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe2c800cbb0 con 0x7fe2d40d09e0 2023-07-13T22:28:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.555+0000 7fe2bb7fe700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2c006ace0 0x7fe2c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.555+0000 7fe2bb7fe700 1 -- 172.21.15.60:0/1307350129 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe2c8089220 con 0x7fe2d40d09e0 2023-07-13T22:28:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.555+0000 7fe2d2d9d700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2c006ace0 0x7fe2c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.556+0000 7fe2d2d9d700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2c006ace0 0x7fe2c006d190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009c10 tx=0x7fe2c4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.560+0000 7fe2bb7fe700 1 -- 172.21.15.60:0/1307350129 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe2d40cbe20 con 0x7fe2d40d09e0 2023-07-13T22:28:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:48.851+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) v1 -- 0x7fe2d40cbe20 con 0x7fe2d40d09e0 2023-07-13T22:28:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.682+0000 7fe2bb7fe700 1 -- 172.21.15.60:0/1307350129 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v398) v1 ==== 150+0+0 (secure 0 0 0) 0x7fe2d40cbe20 con 0x7fe2d40d09e0 2023-07-13T22:28:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2c006ace0 msgr2=0x7fe2c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2c006ace0 0x7fe2c006d190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009c10 tx=0x7fe2c4008030 comp rx=0 tx=0).stop 2023-07-13T22:28:49.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 msgr2=0x7fe2d4159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 0x7fe2d4159f80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800e9e0 tx=0x7fe2c800ecf0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 shutdown_connections 2023-07-13T22:28:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2c006ace0 0x7fe2c006d190 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2d40d0db0 0x7fe2d41f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2d40cd1f0 0x7fe2d40ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 --2- 172.21.15.60:0/1307350129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2d40d09e0 0x7fe2d4159f80 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.685+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 >> 172.21.15.60:0/1307350129 conn(0x7fe2d4071600 msgr2=0x7fe2d40b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.686+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 shutdown_connections 2023-07-13T22:28:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.686+0000 7fe2d91ff700 1 -- 172.21.15.60:0/1307350129 wait complete. 2023-07-13T22:28:49.686 INFO:tasks.cephfs_test_runner:test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-13T22:28:49.686 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:28:49.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2023-07-13T22:28:49.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.915+0000 7ff6e21ff700 1 Processor -- start 2023-07-13T22:28:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.915+0000 7ff6e21ff700 1 -- start start 2023-07-13T22:28:49.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.916+0000 7ff6e21ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc0703e0 0x7ff6dc070850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:49.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.916+0000 7ff6e21ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6dc0cc330 0x7ff6dc0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:49.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.916+0000 7ff6e21ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6dc1598e0 con 0x7ff6dc159510 2023-07-13T22:28:49.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.916+0000 7ff6e21ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc070d90 con 0x7ff6dc0cc330 2023-07-13T22:28:49.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.916+0000 7ff6e21ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0ca8d0 con 0x7ff6dc0703e0 2023-07-13T22:28:49.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.916+0000 7ff6dbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6dc0cc330 0x7ff6dc0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:49.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.916+0000 7ff6dbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6dc0cc330 0x7ff6dc0cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48048/0 (socket says 172.21.15.60:48048) 2023-07-13T22:28:49.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.916+0000 7ff6dbfff700 1 -- 172.21.15.60:0/182478071 learned_addr learned my addr 172.21.15.60:0/182478071 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:49.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.916+0000 7ff6daffd700 1 --2- 172.21.15.60:0/182478071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc0703e0 0x7ff6dc070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:49.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.917+0000 7ff6dbfff700 1 -- 172.21.15.60:0/182478071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc0703e0 msgr2=0x7ff6dc070850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:49.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.917+0000 7ff6dbfff700 1 --2- 172.21.15.60:0/182478071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc0703e0 0x7ff6dc070850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.917+0000 7ff6dbfff700 1 -- 172.21.15.60:0/182478071 >> v1:172.21.15.105:6789/0 conn(0x7ff6dc159510 legacy=0x7ff6dc0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:49.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.917+0000 7ff6dbfff700 1 -- 172.21.15.60:0/182478071 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc06d290 con 0x7ff6dc0cc330 2023-07-13T22:28:49.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.917+0000 7ff6da7fc700 1 -- 172.21.15.60:0/182478071 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746011327 0 0) 0x7ff6dc1598e0 con 0x7ff6dc159510 2023-07-13T22:28:49.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.917+0000 7ff6dbfff700 1 --2- 172.21.15.60:0/182478071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0cc330 0x7ff6dc0cc7b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009e50 tx=0x7ff6d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0e41ab51bc7cc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:49.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.917+0000 7ff6da7fc700 1 -- 172.21.15.60:0/182478071 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400e070 con 0x7ff6dc0cc330 2023-07-13T22:28:49.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.917+0000 7ff6da7fc700 1 -- 172.21.15.60:0/182478071 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d400b910 con 0x7ff6dc0cc330 2023-07-13T22:28:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.918+0000 7ff6da7fc700 1 -- 172.21.15.60:0/182478071 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400bcc0 con 0x7ff6dc0cc330 2023-07-13T22:28:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.918+0000 7ff6e21ff700 1 -- 172.21.15.60:0/182478071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0cc330 msgr2=0x7ff6dc0cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.918+0000 7ff6e21ff700 1 --2- 172.21.15.60:0/182478071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0cc330 0x7ff6dc0cc7b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009e50 tx=0x7ff6d400b330 comp rx=0 tx=0).stop 2023-07-13T22:28:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.918+0000 7ff6e21ff700 1 -- 172.21.15.60:0/182478071 shutdown_connections 2023-07-13T22:28:49.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.918+0000 7ff6e21ff700 1 --2- 172.21.15.60:0/182478071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0cc330 0x7ff6dc0cc7b0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.918+0000 7ff6e21ff700 1 --2- 172.21.15.60:0/182478071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc0703e0 0x7ff6dc070850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.918+0000 7ff6e21ff700 1 -- 172.21.15.60:0/182478071 >> 172.21.15.60:0/182478071 conn(0x7ff6dc0cbe70 msgr2=0x7ff6dc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:49.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.918+0000 7ff6e21ff700 1 -- 172.21.15.60:0/182478071 shutdown_connections 2023-07-13T22:28:49.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.918+0000 7ff6e21ff700 1 -- 172.21.15.60:0/182478071 wait complete. 2023-07-13T22:28:49.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.920+0000 7ff6e21ff700 1 Processor -- start 2023-07-13T22:28:49.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.920+0000 7ff6e21ff700 1 -- start start 2023-07-13T22:28:49.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.921+0000 7ff6e21ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0703e0 0x7ff6dc1f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:49.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.921+0000 7ff6e21ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc0cc330 0x7ff6dc0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:49.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.921+0000 7ff6e21ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc159510 0x7ff6dc0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:49.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.921+0000 7ff6e21ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc1f5a70 con 0x7ff6dc0703e0 2023-07-13T22:28:49.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.921+0000 7ff6e21ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0b1590 con 0x7ff6dc0cc330 2023-07-13T22:28:49.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.921+0000 7ff6e21ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc1f5cf0 con 0x7ff6dc159510 2023-07-13T22:28:49.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.921+0000 7ff6db7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0703e0 0x7ff6dc1f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6db7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0703e0 0x7ff6dc1f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48062/0 (socket says 172.21.15.60:48062) 2023-07-13T22:28:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6db7fe700 1 -- 172.21.15.60:0/1736485272 learned_addr learned my addr 172.21.15.60:0/1736485272 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6daffd700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc0cc330 0x7ff6dc0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6dbfff700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc159510 0x7ff6dc0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6db7fe700 1 -- 172.21.15.60:0/1736485272 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc159510 msgr2=0x7ff6dc0ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6db7fe700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc159510 0x7ff6dc0ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6db7fe700 1 -- 172.21.15.60:0/1736485272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc0cc330 msgr2=0x7ff6dc0b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:49.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6db7fe700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc0cc330 0x7ff6dc0b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:49.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6db7fe700 1 -- 172.21.15.60:0/1736485272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4009b30 con 0x7ff6dc0703e0 2023-07-13T22:28:49.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6db7fe700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0703e0 0x7ff6dc1f5530 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00c4a0 tx=0x7ff6cc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:49.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6c3fff700 1 -- 172.21.15.60:0/1736485272 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0054e0 con 0x7ff6dc0703e0 2023-07-13T22:28:49.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6c3fff700 1 -- 172.21.15.60:0/1736485272 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc0036d0 con 0x7ff6dc0703e0 2023-07-13T22:28:49.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc1f5fd0 con 0x7ff6dc0703e0 2023-07-13T22:28:49.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.922+0000 7ff6c3fff700 1 -- 172.21.15.60:0/1736485272 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc003a80 con 0x7ff6dc0703e0 2023-07-13T22:28:49.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.923+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc0b1830 con 0x7ff6dc0703e0 2023-07-13T22:28:49.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.924+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc04e280 con 0x7ff6dc0703e0 2023-07-13T22:28:49.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.924+0000 7ff6c3fff700 1 -- 172.21.15.60:0/1736485272 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff6cc003be0 con 0x7ff6dc0703e0 2023-07-13T22:28:49.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.925+0000 7ff6c3fff700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6c806ace0 0x7ff6c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:49.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.926+0000 7ff6c3fff700 1 -- 172.21.15.60:0/1736485272 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff6cc088530 con 0x7ff6dc0703e0 2023-07-13T22:28:49.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.926+0000 7ff6daffd700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6c806ace0 0x7ff6c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:49.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.926+0000 7ff6daffd700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6c806ace0 0x7ff6c806d190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005c50 tx=0x7ff6d000b750 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:49.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:49.930+0000 7ff6c3fff700 1 -- 172.21.15.60:0/1736485272 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff6cc0538f0 con 0x7ff6dc0703e0 2023-07-13T22:28:50.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.220+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) v1 -- 0x7ff6dc0b1a20 con 0x7ff6dc0703e0 2023-07-13T22:28:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.684+0000 7ff6c3fff700 1 -- 172.21.15.60:0/1736485272 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v399) v1 ==== 146+0+0 (secure 0 0 0) 0x7ff6cc056f10 con 0x7ff6dc0703e0 2023-07-13T22:28:50.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6c806ace0 msgr2=0x7ff6c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:50.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6c806ace0 0x7ff6c806d190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005c50 tx=0x7ff6d000b750 comp rx=0 tx=0).stop 2023-07-13T22:28:50.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0703e0 msgr2=0x7ff6dc1f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:50.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0703e0 0x7ff6dc1f5530 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00c4a0 tx=0x7ff6cc00c7b0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 shutdown_connections 2023-07-13T22:28:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6c806ace0 0x7ff6c806d190 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6dc0703e0 0x7ff6dc1f5530 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6dc159510 0x7ff6dc0ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 --2- 172.21.15.60:0/1736485272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc0cc330 0x7ff6dc0b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 >> 172.21.15.60:0/1736485272 conn(0x7ff6dc0cbe70 msgr2=0x7ff6dc0cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 shutdown_connections 2023-07-13T22:28:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.688+0000 7ff6e21ff700 1 -- 172.21.15.60:0/1736485272 wait complete. 2023-07-13T22:28:50.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-13T22:28:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.920+0000 7ff3f13ff700 1 Processor -- start 2023-07-13T22:28:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.920+0000 7ff3f13ff700 1 -- start start 2023-07-13T22:28:50.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.921+0000 7ff3f13ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff3ec0cfd50 0x7ff3ec0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:50.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.921+0000 7ff3f13ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3ec0b1ac0 0x7ff3ec0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:50.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.921+0000 7ff3f13ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec0d1870 con 0x7ff3ec0b8200 2023-07-13T22:28:50.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.921+0000 7ff3f13ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cf370 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.921+0000 7ff3f13ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec0d12b0 con 0x7ff3ec0b1ac0 2023-07-13T22:28:50.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.922+0000 7ff3ea7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff3ec0cfd50 0x7ff3ec0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:50.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.922+0000 7ff3ea7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff3ec0cfd50 0x7ff3ec0d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48072/0 (socket says 172.21.15.60:48072) 2023-07-13T22:28:50.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.922+0000 7ff3ea7fc700 1 -- 172.21.15.60:0/953426193 learned_addr learned my addr 172.21.15.60:0/953426193 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:50.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.922+0000 7ff3eb7fe700 1 --2- 172.21.15.60:0/953426193 >> v2:172.21.15.105:3300/0 conn(0x7ff3ec0b1ac0 0x7ff3ec0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:50.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.922+0000 7ff3ea7fc700 1 -- 172.21.15.60:0/953426193 >> v2:172.21.15.105:3300/0 conn(0x7ff3ec0b1ac0 msgr2=0x7ff3ec0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:50.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.922+0000 7ff3ea7fc700 1 --2- 172.21.15.60:0/953426193 >> v2:172.21.15.105:3300/0 conn(0x7ff3ec0b1ac0 0x7ff3ec0d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.922+0000 7ff3ea7fc700 1 -- 172.21.15.60:0/953426193 >> v1:172.21.15.105:6789/0 conn(0x7ff3ec0b8200 legacy=0x7ff3ec0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:50.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.922+0000 7ff3ea7fc700 1 -- 172.21.15.60:0/953426193 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0d1590 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.922+0000 7ff3ea7fc700 1 --2- 172.21.15.60:0/953426193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 0x7ff3ec0d23c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0b8d70 tx=0x7ff3e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1ca55f9ceabdf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:50.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3e9ffb700 1 -- 172.21.15.60:0/953426193 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0011830 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3e9ffb700 1 -- 172.21.15.60:0/953426193 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e0011990 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3e9ffb700 1 -- 172.21.15.60:0/953426193 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0011d20 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3f13ff700 1 -- 172.21.15.60:0/953426193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 msgr2=0x7ff3ec0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3f13ff700 1 --2- 172.21.15.60:0/953426193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 0x7ff3ec0d23c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0b8d70 tx=0x7ff3e0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3f13ff700 1 -- 172.21.15.60:0/953426193 shutdown_connections 2023-07-13T22:28:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3f13ff700 1 --2- 172.21.15.60:0/953426193 >> v2:172.21.15.105:3300/0 conn(0x7ff3ec0b1ac0 0x7ff3ec0d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3f13ff700 1 --2- 172.21.15.60:0/953426193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 0x7ff3ec0d23c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3f13ff700 1 -- 172.21.15.60:0/953426193 >> 172.21.15.60:0/953426193 conn(0x7ff3ec06b2f0 msgr2=0x7ff3ec06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3f13ff700 1 -- 172.21.15.60:0/953426193 shutdown_connections 2023-07-13T22:28:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.923+0000 7ff3f13ff700 1 -- 172.21.15.60:0/953426193 wait complete. 2023-07-13T22:28:50.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.925+0000 7ff3f13ff700 1 Processor -- start 2023-07-13T22:28:50.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.925+0000 7ff3f13ff700 1 -- start start 2023-07-13T22:28:50.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.926+0000 7ff3f13ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ec0b1ac0 0x7ff3ec0cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:50.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.926+0000 7ff3f13ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3ec0b8200 0x7ff3ec14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:50.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.926+0000 7ff3f13ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 0x7ff3ec14e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:50.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.926+0000 7ff3f13ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec15e180 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.926+0000 7ff3f13ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec15e460 con 0x7ff3ec0b1ac0 2023-07-13T22:28:50.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.926+0000 7ff3f13ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0c9d00 con 0x7ff3ec0b8200 2023-07-13T22:28:50.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.926+0000 7ff3eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 0x7ff3ec14e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:50.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.926+0000 7ff3eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 0x7ff3ec14e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48074/0 (socket says 172.21.15.60:48074) 2023-07-13T22:28:50.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.926+0000 7ff3eb7fe700 1 -- 172.21.15.60:0/2137484143 learned_addr learned my addr 172.21.15.60:0/2137484143 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:50.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3ea7fc700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3ec0b8200 0x7ff3ec14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:50.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3eaffd700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ec0b1ac0 0x7ff3ec0cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:50.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3eb7fe700 1 -- 172.21.15.60:0/2137484143 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3ec0b8200 msgr2=0x7ff3ec14dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:50.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3eb7fe700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3ec0b8200 0x7ff3ec14dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3eb7fe700 1 -- 172.21.15.60:0/2137484143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ec0b1ac0 msgr2=0x7ff3ec0cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:50.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3eb7fe700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ec0b1ac0 0x7ff3ec0cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:50.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3eb7fe700 1 -- 172.21.15.60:0/2137484143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e0009550 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3eb7fe700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 0x7ff3ec14e200 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400efc0 tx=0x7ff3e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:50.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3d37fe700 1 -- 172.21.15.60:0/2137484143 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4009ea0 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3d37fe700 1 -- 172.21.15.60:0/2137484143 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e400f040 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3d37fe700 1 -- 172.21.15.60:0/2137484143 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e400ced0 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.927+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec0c9fe0 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.928+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec0ccb90 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.929+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec0ccd80 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.930+0000 7ff3d37fe700 1 -- 172.21.15.60:0/2137484143 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff3e4014480 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.931+0000 7ff3d37fe700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3d806ace0 0x7ff3d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:50.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.931+0000 7ff3d37fe700 1 -- 172.21.15.60:0/2137484143 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff3e4089a00 con 0x7ff3ec0cfd50 2023-07-13T22:28:50.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.931+0000 7ff3eaffd700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3d806ace0 0x7ff3d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:50.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.932+0000 7ff3eaffd700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3d806ace0 0x7ff3d806d190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc007fd0 tx=0x7ff3dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:50.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:50.935+0000 7ff3d37fe700 1 -- 172.21.15.60:0/2137484143 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff3e4054d30 con 0x7ff3ec0cfd50 2023-07-13T22:28:51.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.144+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff3ec15e620 con 0x7ff3ec0cfd50 2023-07-13T22:28:51.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.148+0000 7ff3d37fe700 1 -- 172.21.15.60:0/2137484143 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff3e40180c0 con 0x7ff3ec0cfd50 2023-07-13T22:28:51.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3d806ace0 msgr2=0x7ff3d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:51.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3d806ace0 0x7ff3d806d190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc007fd0 tx=0x7ff3dc009040 comp rx=0 tx=0).stop 2023-07-13T22:28:51.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 msgr2=0x7ff3ec14e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:51.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 0x7ff3ec14e200 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400efc0 tx=0x7ff3e400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 shutdown_connections 2023-07-13T22:28:51.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff3d806ace0 0x7ff3d806d190 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3ec0cfd50 0x7ff3ec14e200 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ec0b1ac0 0x7ff3ec0cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 --2- 172.21.15.60:0/2137484143 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3ec0b8200 0x7ff3ec14dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 >> 172.21.15.60:0/2137484143 conn(0x7ff3ec06b2f0 msgr2=0x7ff3ec0d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:51.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 shutdown_connections 2023-07-13T22:28:51.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.150+0000 7ff3f13ff700 1 -- 172.21.15.60:0/2137484143 wait complete. 2023-07-13T22:28:51.148 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:28:51.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:28:51.255 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:28:51.374 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:28:51.374 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:28:51.375 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:51.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:28:51.423 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:28:51.423 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:51.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:28:51.471 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:28:51.471 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:51.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:28:51.518 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:28:51.518 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:28:51.564 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:28:51.565 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-13T22:28:51.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:28:51.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.789+0000 7f0cc7bff700 1 Processor -- start 2023-07-13T22:28:51.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.789+0000 7f0cc7bff700 1 -- start start 2023-07-13T22:28:51.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc7bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 0x7f0cc00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:51.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc7bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0cc00d3750 0x7f0cc00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:51.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc7bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc00cff20 con 0x7f0cc00cfad0 2023-07-13T22:28:51.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc7bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc00d0ac0 con 0x7f0cc00d3750 2023-07-13T22:28:51.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc7bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc00b87d0 con 0x7f0cc00b8400 2023-07-13T22:28:51.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc619c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0cc00cfad0 0x7f0cc015fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43054/0 (socket says 172.21.15.60:43054) 2023-07-13T22:28:51.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc619c700 1 -- 172.21.15.60:0/830581031 learned_addr learned my addr 172.21.15.60:0/830581031 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:51.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc519a700 1 --2- 172.21.15.60:0/830581031 >> v2:172.21.15.105:3300/0 conn(0x7f0cc00d3750 0x7f0cc00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:51.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc599b700 1 --2- 172.21.15.60:0/830581031 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 0x7f0cc00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:51.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc4999700 1 -- 172.21.15.60:0/830581031 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297223888 0 0) 0x7f0cb8001490 con 0x7f0cc00cfad0 2023-07-13T22:28:51.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.791+0000 7f0cc4999700 1 -- 172.21.15.60:0/830581031 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cac0035b0 con 0x7f0cc00cfad0 2023-07-13T22:28:51.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.792+0000 7f0cc519a700 1 -- 172.21.15.60:0/830581031 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 msgr2=0x7f0cc00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:51.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.792+0000 7f0cc519a700 1 --2- 172.21.15.60:0/830581031 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 0x7f0cc00cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.792+0000 7f0cc519a700 1 -- 172.21.15.60:0/830581031 >> v1:172.21.15.60:6789/0 conn(0x7f0cc00cfad0 legacy=0x7f0cc015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:51.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.792+0000 7f0cc519a700 1 -- 172.21.15.60:0/830581031 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc00b4eb0 con 0x7f0cc00d3750 2023-07-13T22:28:51.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.792+0000 7f0cc599b700 1 --2- 172.21.15.60:0/830581031 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 0x7f0cc00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:28:51.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.792+0000 7f0cc519a700 1 --2- 172.21.15.60:0/830581031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cc00d3750 0x7f0cc00cf590 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009840 tx=0x7f0cb0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68f9e0a53631129e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:51.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.792+0000 7f0cc4999700 1 -- 172.21.15.60:0/830581031 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0011830 con 0x7f0cc00d3750 2023-07-13T22:28:51.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.792+0000 7f0cc4999700 1 -- 172.21.15.60:0/830581031 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0011990 con 0x7f0cc00d3750 2023-07-13T22:28:51.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.792+0000 7f0cc4999700 1 -- 172.21.15.60:0/830581031 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0011d20 con 0x7f0cc00d3750 2023-07-13T22:28:51.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.793+0000 7f0cc7bff700 1 -- 172.21.15.60:0/830581031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cc00d3750 msgr2=0x7f0cc00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:51.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.793+0000 7f0cc7bff700 1 --2- 172.21.15.60:0/830581031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cc00d3750 0x7f0cc00cf590 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009840 tx=0x7f0cb0009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.793+0000 7f0cc7bff700 1 -- 172.21.15.60:0/830581031 shutdown_connections 2023-07-13T22:28:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.793+0000 7f0cc7bff700 1 --2- 172.21.15.60:0/830581031 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 0x7f0cc00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.793+0000 7f0cc7bff700 1 --2- 172.21.15.60:0/830581031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cc00d3750 0x7f0cc00cf590 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.793+0000 7f0cc7bff700 1 -- 172.21.15.60:0/830581031 >> 172.21.15.60:0/830581031 conn(0x7f0cc006b390 msgr2=0x7f0cc006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.793+0000 7f0cc7bff700 1 -- 172.21.15.60:0/830581031 shutdown_connections 2023-07-13T22:28:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.793+0000 7f0cc7bff700 1 -- 172.21.15.60:0/830581031 wait complete. 2023-07-13T22:28:51.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.794+0000 7f0cc7bff700 1 Processor -- start 2023-07-13T22:28:51.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.795+0000 7f0cc7bff700 1 -- start start 2023-07-13T22:28:51.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc7bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 0x7f0cc0159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:51.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc7bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cc00cfad0 0x7f0cc0150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:51.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc7bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cc00d3750 0x7f0cc0151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:51.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc7bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc01f2b00 con 0x7f0cc00d3750 2023-07-13T22:28:51.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc7bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc015e310 con 0x7f0cc00cfad0 2023-07-13T22:28:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc7bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc01f2160 con 0x7f0cc00b8400 2023-07-13T22:28:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cc00d3750 0x7f0cc0151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cc00d3750 0x7f0cc0151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48080/0 (socket says 172.21.15.60:48080) 2023-07-13T22:28:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc619c700 1 -- 172.21.15.60:0/873697712 learned_addr learned my addr 172.21.15.60:0/873697712 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc519a700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cc00cfad0 0x7f0cc0150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc599b700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 0x7f0cc0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc619c700 1 -- 172.21.15.60:0/873697712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 msgr2=0x7f0cc0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc619c700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 0x7f0cc0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc619c700 1 -- 172.21.15.60:0/873697712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cc00cfad0 msgr2=0x7f0cc0150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc619c700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cc00cfad0 0x7f0cc0150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.796+0000 7f0cc619c700 1 -- 172.21.15.60:0/873697712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc0094a0 con 0x7f0cc00d3750 2023-07-13T22:28:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.797+0000 7f0cc619c700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cc00d3750 0x7f0cc0151110 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800cae0 tx=0x7f0cb800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.797+0000 7f0cb67fc700 1 -- 172.21.15.60:0/873697712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb800aa80 con 0x7f0cc00d3750 2023-07-13T22:28:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.797+0000 7f0cb67fc700 1 -- 172.21.15.60:0/873697712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb800abe0 con 0x7f0cc00d3750 2023-07-13T22:28:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.797+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb00094a0 con 0x7f0cc00d3750 2023-07-13T22:28:51.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.797+0000 7f0cb67fc700 1 -- 172.21.15.60:0/873697712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8012400 con 0x7f0cc00d3750 2023-07-13T22:28:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.797+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc01f2720 con 0x7f0cc00d3750 2023-07-13T22:28:51.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.798+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc01f2c40 con 0x7f0cc00d3750 2023-07-13T22:28:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.799+0000 7f0cb67fc700 1 -- 172.21.15.60:0/873697712 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0cb801b3b0 con 0x7f0cc00d3750 2023-07-13T22:28:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.800+0000 7f0cb67fc700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ca406ace0 0x7f0ca406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.800+0000 7f0cb67fc700 1 -- 172.21.15.60:0/873697712 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0cb8088d30 con 0x7f0cc00d3750 2023-07-13T22:28:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.800+0000 7f0cc599b700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ca406ace0 0x7f0ca406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.801+0000 7f0cc599b700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ca406ace0 0x7f0ca406d190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0cc015cc90 tx=0x7f0cbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:51.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:51.804+0000 7f0cb67fc700 1 -- 172.21.15.60:0/873697712 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0cc01f2c40 con 0x7f0cc00d3750 2023-07-13T22:28:52.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.057+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cc00cff20 con 0x7f0cc00d3750 2023-07-13T22:28:52.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.058+0000 7f0cb67fc700 1 -- 172.21.15.60:0/873697712 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f0cb8017de0 con 0x7f0cc00d3750 2023-07-13T22:28:52.037 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:52.037 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2395854295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2395854295},{"type":"v1","addr":"172.21.15.105:6834","nonce":2395854295}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1048261776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1048261776},{"type":"v1","addr":"172.21.15.105:6835","nonce":1048261776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7431,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/1639899719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1639899719},{"type":"v1","addr":"172.21.15.60:6836","nonce":1639899719}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/190744701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":190744701},{"type":"v1","addr":"172.21.15.60:6837","nonce":190744701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[]} 2023-07-13T22:28:52.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ca406ace0 msgr2=0x7f0ca406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ca406ace0 0x7f0ca406d190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0cc015cc90 tx=0x7f0cbc008040 comp rx=0 tx=0).stop 2023-07-13T22:28:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cc00d3750 msgr2=0x7f0cc0151110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cc00d3750 0x7f0cc0151110 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800cae0 tx=0x7f0cb800cea0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 shutdown_connections 2023-07-13T22:28:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ca406ace0 0x7f0ca406d190 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cc00d3750 0x7f0cc0151110 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0cc00b8400 0x7f0cc0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 --2- 172.21.15.60:0/873697712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cc00cfad0 0x7f0cc0150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 >> 172.21.15.60:0/873697712 conn(0x7f0cc006b390 msgr2=0x7f0cc00d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 shutdown_connections 2023-07-13T22:28:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.060+0000 7f0cc7bff700 1 -- 172.21.15.60:0/873697712 wait complete. 2023-07-13T22:28:52.041 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 56 2023-07-13T22:28:52.058 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:28:52.058 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-13T22:28:52.058 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:28:52.058 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:28:52.059 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:28:52.059 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-13T22:28:52.059 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:28:52.059 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:28:52.060 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:28:52.060 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-13T22:28:52.060 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:28:52.060 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:28:52.061 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:28:52.061 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-13T22:28:52.061 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:28:52.062 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:28:52.074 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:28:52.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:28:52.075 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:28:52.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:28:52.076 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:28:52.077 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:28:52.078 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:28:52.078 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:28:52.119 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:28:52.119 DEBUG:teuthology.parallel:result is None 2023-07-13T22:28:52.121 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:28:52.122 DEBUG:teuthology.parallel:result is None 2023-07-13T22:28:52.122 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:28:52.123 DEBUG:teuthology.parallel:result is None 2023-07-13T22:28:52.124 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:28:52.125 DEBUG:teuthology.parallel:result is None 2023-07-13T22:28:52.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-13T22:28:52.267 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:28:52.268 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:28:52.274 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:28:52.278 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:28:52.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.362+0000 7f1021fff700 1 Processor -- start 2023-07-13T22:28:52.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.362+0000 7f1021fff700 1 -- start start 2023-07-13T22:28:52.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.363+0000 7f1021fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f101c159170 0x7f101c0c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:52.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.363+0000 7f1021fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f101c070980 0x7f101c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:52.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f1021fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c15d970 con 0x7f101c0c5040 2023-07-13T22:28:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f1021fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101c15cd60 con 0x7f101c159170 2023-07-13T22:28:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f1021fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101c15d040 con 0x7f101c070980 2023-07-13T22:28:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101affd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f101c0c5040 0x7f101c070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43082/0 (socket says 172.21.15.60:43082) 2023-07-13T22:28:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101affd700 1 -- 172.21.15.60:0/3720551176 learned_addr learned my addr 172.21.15.60:0/3720551176 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:52.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101b7fe700 1 --2- 172.21.15.60:0/3720551176 >> v2:172.21.15.60:3300/0 conn(0x7f101c159170 0x7f101c0c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:52.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101bfff700 1 --2- 172.21.15.60:0/3720551176 >> v2:172.21.15.105:3300/0 conn(0x7f101c070980 0x7f101c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:52.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101a7fc700 1 -- 172.21.15.60:0/3720551176 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829539839 0 0) 0x7f101c15d970 con 0x7f101c0c5040 2023-07-13T22:28:52.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101a7fc700 1 -- 172.21.15.60:0/3720551176 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10040057c0 con 0x7f101c0c5040 2023-07-13T22:28:52.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101b7fe700 1 -- 172.21.15.60:0/3720551176 >> v2:172.21.15.105:3300/0 conn(0x7f101c070980 msgr2=0x7f101c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:52.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101b7fe700 1 --2- 172.21.15.60:0/3720551176 >> v2:172.21.15.105:3300/0 conn(0x7f101c070980 0x7f101c0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101b7fe700 1 -- 172.21.15.60:0/3720551176 >> v1:172.21.15.60:6789/0 conn(0x7f101c0c5040 legacy=0x7f101c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:52.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.364+0000 7f101b7fe700 1 -- 172.21.15.60:0/3720551176 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c15d320 con 0x7f101c159170 2023-07-13T22:28:52.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.365+0000 7f101b7fe700 1 --2- 172.21.15.60:0/3720551176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 0x7f101c0c4b00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f101c06e4b0 tx=0x7f100c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652d83e509a7ff39 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:52.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.365+0000 7f101a7fc700 1 -- 172.21.15.60:0/3720551176 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c011830 con 0x7f101c159170 2023-07-13T22:28:52.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.365+0000 7f101a7fc700 1 -- 172.21.15.60:0/3720551176 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c011990 con 0x7f101c159170 2023-07-13T22:28:52.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.365+0000 7f101a7fc700 1 -- 172.21.15.60:0/3720551176 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c011d20 con 0x7f101c159170 2023-07-13T22:28:52.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.365+0000 7f1021fff700 1 -- 172.21.15.60:0/3720551176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 msgr2=0x7f101c0c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:52.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.365+0000 7f1021fff700 1 --2- 172.21.15.60:0/3720551176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 0x7f101c0c4b00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f101c06e4b0 tx=0x7f100c009b20 comp rx=0 tx=0).stop 2023-07-13T22:28:52.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.366+0000 7f1021fff700 1 -- 172.21.15.60:0/3720551176 shutdown_connections 2023-07-13T22:28:52.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.366+0000 7f1021fff700 1 --2- 172.21.15.60:0/3720551176 >> v2:172.21.15.105:3300/0 conn(0x7f101c070980 0x7f101c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.366+0000 7f1021fff700 1 --2- 172.21.15.60:0/3720551176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 0x7f101c0c4b00 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.366+0000 7f1021fff700 1 -- 172.21.15.60:0/3720551176 >> 172.21.15.60:0/3720551176 conn(0x7f101c0cf410 msgr2=0x7f101c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:52.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.366+0000 7f1021fff700 1 -- 172.21.15.60:0/3720551176 shutdown_connections 2023-07-13T22:28:52.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.366+0000 7f1021fff700 1 -- 172.21.15.60:0/3720551176 wait complete. 2023-07-13T22:28:52.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.367+0000 7f1021fff700 1 Processor -- start 2023-07-13T22:28:52.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.368+0000 7f1021fff700 1 -- start start 2023-07-13T22:28:52.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.368+0000 7f1021fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c070980 0x7f101c14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:52.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.368+0000 7f1021fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0c5040 0x7f101c0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:52.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f1021fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 0x7f101c0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:52.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f1021fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c14cbe0 con 0x7f101c159170 2023-07-13T22:28:52.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f1021fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c150300 con 0x7f101c070980 2023-07-13T22:28:52.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f1021fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0b23f0 con 0x7f101c0c5040 2023-07-13T22:28:52.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 0x7f101c0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:52.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 0x7f101c0cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48126/0 (socket says 172.21.15.60:48126) 2023-07-13T22:28:52.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101bfff700 1 -- 172.21.15.60:0/3807340827 learned_addr learned my addr 172.21.15.60:0/3807340827 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:52.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101b7fe700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c070980 0x7f101c14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:52.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101affd700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0c5040 0x7f101c0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:52.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101bfff700 1 -- 172.21.15.60:0/3807340827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0c5040 msgr2=0x7f101c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:52.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101bfff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0c5040 0x7f101c0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101bfff700 1 -- 172.21.15.60:0/3807340827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c070980 msgr2=0x7f101c14c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:52.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101bfff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c070980 0x7f101c14c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:52.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.369+0000 7f101bfff700 1 -- 172.21.15.60:0/3807340827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c0094a0 con 0x7f101c159170 2023-07-13T22:28:52.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.370+0000 7f101bfff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 0x7f101c0cd040 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f1014009d30 tx=0x7f101400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:52.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.370+0000 7f1003fff700 1 -- 172.21.15.60:0/3807340827 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400cd40 con 0x7f101c159170 2023-07-13T22:28:52.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.370+0000 7f1003fff700 1 -- 172.21.15.60:0/3807340827 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101400f040 con 0x7f101c159170 2023-07-13T22:28:52.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.370+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c0b26d0 con 0x7f101c159170 2023-07-13T22:28:52.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.370+0000 7f1003fff700 1 -- 172.21.15.60:0/3807340827 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140146c0 con 0x7f101c159170 2023-07-13T22:28:52.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.370+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c150830 con 0x7f101c159170 2023-07-13T22:28:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.371+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c150a50 con 0x7f101c159170 2023-07-13T22:28:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.372+0000 7f1003fff700 1 -- 172.21.15.60:0/3807340827 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1014014820 con 0x7f101c159170 2023-07-13T22:28:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.373+0000 7f1003fff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100806ace0 0x7f100806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.373+0000 7f1003fff700 1 -- 172.21.15.60:0/3807340827 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1014089950 con 0x7f101c159170 2023-07-13T22:28:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.373+0000 7f101b7fe700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100806ace0 0x7f100806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.374+0000 7f101b7fe700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100806ace0 0x7f100806d190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f100c0090b0 tx=0x7f100c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:52.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.378+0000 7f1003fff700 1 -- 172.21.15.60:0/3807340827 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1014054d10 con 0x7f101c159170 2023-07-13T22:28:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:52.597+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f101c0ad860 con 0x7f101c159170 2023-07-13T22:28:53.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.709+0000 7f1003fff700 1 -- 172.21.15.60:0/3807340827 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v96) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1014018080 con 0x7f101c159170 2023-07-13T22:28:53.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100806ace0 msgr2=0x7f100806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:53.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100806ace0 0x7f100806d190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f100c0090b0 tx=0x7f100c008030 comp rx=0 tx=0).stop 2023-07-13T22:28:53.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 msgr2=0x7f101c0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:53.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 0x7f101c0cd040 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f1014009d30 tx=0x7f101400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 shutdown_connections 2023-07-13T22:28:53.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f100806ace0 0x7f100806d190 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101c159170 0x7f101c0cd040 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c070980 0x7f101c14c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 --2- 172.21.15.60:0/3807340827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c0c5040 0x7f101c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.712+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 >> 172.21.15.60:0/3807340827 conn(0x7f101c0cf410 msgr2=0x7f101c157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:53.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.713+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 shutdown_connections 2023-07-13T22:28:53.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.713+0000 7f1021fff700 1 -- 172.21.15.60:0/3807340827 wait complete. 2023-07-13T22:28:53.694 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-07-13T22:28:53.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-13T22:28:53.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.941+0000 7f7f2bbff700 1 Processor -- start 2023-07-13T22:28:53.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.941+0000 7f7f2bbff700 1 -- start start 2023-07-13T22:28:53.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.942+0000 7f7f2bbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d35c0 0x7f7f240d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:53.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.942+0000 7f7f2bbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f240d13f0 0x7f7f2415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:53.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.942+0000 7f7f2bbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f240d09d0 con 0x7f7f240b8570 2023-07-13T22:28:53.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.942+0000 7f7f2bbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f240d3b30 con 0x7f7f240d13f0 2023-07-13T22:28:53.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.942+0000 7f7f2bbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240d1870 con 0x7f7f240d35c0 2023-07-13T22:28:53.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2a19c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f240d13f0 0x7f7f2415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:53.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2a19c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f240d13f0 0x7f7f2415c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48140/0 (socket says 172.21.15.60:48140) 2023-07-13T22:28:53.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2a19c700 1 -- 172.21.15.60:0/2555500703 learned_addr learned my addr 172.21.15.60:0/2555500703 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:53.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2919a700 1 --2- 172.21.15.60:0/2555500703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d35c0 0x7f7f240d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:53.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2a19c700 1 -- 172.21.15.60:0/2555500703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d35c0 msgr2=0x7f7f240d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:53.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2a19c700 1 --2- 172.21.15.60:0/2555500703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d35c0 0x7f7f240d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2a19c700 1 -- 172.21.15.60:0/2555500703 >> v1:172.21.15.105:6789/0 conn(0x7f7f240b8570 legacy=0x7f7f240ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:53.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2a19c700 1 -- 172.21.15.60:0/2555500703 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f240cda10 con 0x7f7f240d13f0 2023-07-13T22:28:53.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2919a700 1 --2- 172.21.15.60:0/2555500703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d35c0 0x7f7f240d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:53.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.943+0000 7f7f2a19c700 1 --2- 172.21.15.60:0/2555500703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240d13f0 0x7f7f2415c920 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009e50 tx=0x7f7f1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b931f8b71aba550d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:53.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.944+0000 7f7f28999700 1 -- 172.21.15.60:0/2555500703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00e070 con 0x7f7f240d13f0 2023-07-13T22:28:53.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.944+0000 7f7f28999700 1 -- 172.21.15.60:0/2555500703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c003a20 con 0x7f7f240d13f0 2023-07-13T22:28:53.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.944+0000 7f7f28999700 1 -- 172.21.15.60:0/2555500703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c003dd0 con 0x7f7f240d13f0 2023-07-13T22:28:53.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.944+0000 7f7f2bbff700 1 -- 172.21.15.60:0/2555500703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240d13f0 msgr2=0x7f7f2415c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:53.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.944+0000 7f7f2bbff700 1 --2- 172.21.15.60:0/2555500703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240d13f0 0x7f7f2415c920 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009e50 tx=0x7f7f1c00b330 comp rx=0 tx=0).stop 2023-07-13T22:28:53.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.944+0000 7f7f2bbff700 1 -- 172.21.15.60:0/2555500703 shutdown_connections 2023-07-13T22:28:53.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.944+0000 7f7f2bbff700 1 --2- 172.21.15.60:0/2555500703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240d13f0 0x7f7f2415c920 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.944+0000 7f7f2bbff700 1 --2- 172.21.15.60:0/2555500703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d35c0 0x7f7f240d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.944+0000 7f7f2bbff700 1 -- 172.21.15.60:0/2555500703 >> 172.21.15.60:0/2555500703 conn(0x7f7f24071600 msgr2=0x7f7f240719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:53.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.945+0000 7f7f2bbff700 1 -- 172.21.15.60:0/2555500703 shutdown_connections 2023-07-13T22:28:53.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.945+0000 7f7f2bbff700 1 -- 172.21.15.60:0/2555500703 wait complete. 2023-07-13T22:28:53.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.946+0000 7f7f2bbff700 1 Processor -- start 2023-07-13T22:28:53.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.946+0000 7f7f2bbff700 1 -- start start 2023-07-13T22:28:53.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.947+0000 7f7f2bbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240b8570 0x7f7f240b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:53.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.947+0000 7f7f2bbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d13f0 0x7f7f240c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:53.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.947+0000 7f7f2bbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f240d35c0 0x7f7f241f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:53.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.947+0000 7f7f2bbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240c9e60 con 0x7f7f240b8570 2023-07-13T22:28:53.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.947+0000 7f7f2bbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240ca140 con 0x7f7f240d35c0 2023-07-13T22:28:53.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.947+0000 7f7f2bbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240ca420 con 0x7f7f240d13f0 2023-07-13T22:28:53.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2999b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240b8570 0x7f7f240b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:53.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2999b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240b8570 0x7f7f240b6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48148/0 (socket says 172.21.15.60:48148) 2023-07-13T22:28:53.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2999b700 1 -- 172.21.15.60:0/3850723690 learned_addr learned my addr 172.21.15.60:0/3850723690 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:53.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2919a700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d13f0 0x7f7f240c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:53.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2a19c700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f240d35c0 0x7f7f241f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:53.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2999b700 1 -- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d13f0 msgr2=0x7f7f240c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:53.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2999b700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d13f0 0x7f7f240c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2999b700 1 -- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f240d35c0 msgr2=0x7f7f241f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:53.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2999b700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f240d35c0 0x7f7f241f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:53.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2999b700 1 -- 172.21.15.60:0/3850723690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c009b30 con 0x7f7f240b8570 2023-07-13T22:28:53.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2919a700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d13f0 0x7f7f240c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:53.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2a19c700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f240d35c0 0x7f7f241f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:53.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2999b700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240b8570 0x7f7f240b6d50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c520 tx=0x7f7f2000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:53.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f1a7fc700 1 -- 172.21.15.60:0/3850723690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f200054e0 con 0x7f7f240b8570 2023-07-13T22:28:53.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f1a7fc700 1 -- 172.21.15.60:0/3850723690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f200036d0 con 0x7f7f240b8570 2023-07-13T22:28:53.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f1a7fc700 1 -- 172.21.15.60:0/3850723690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20003a80 con 0x7f7f240b8570 2023-07-13T22:28:53.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.948+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f240ca700 con 0x7f7f240b8570 2023-07-13T22:28:53.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.949+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f240b2370 con 0x7f7f240b8570 2023-07-13T22:28:53.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.950+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2404e280 con 0x7f7f240b8570 2023-07-13T22:28:53.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.951+0000 7f7f1a7fc700 1 -- 172.21.15.60:0/3850723690 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f20003be0 con 0x7f7f240b8570 2023-07-13T22:28:53.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.951+0000 7f7f1a7fc700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f0806ace0 0x7f7f0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:53.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.952+0000 7f7f1a7fc700 1 -- 172.21.15.60:0/3850723690 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7f20088300 con 0x7f7f240b8570 2023-07-13T22:28:53.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.952+0000 7f7f2919a700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f0806ace0 0x7f7f0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:53.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.952+0000 7f7f2919a700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f0806ace0 0x7f7f0806d190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7f240c7340 tx=0x7f7f14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:53.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:53.956+0000 7f7f1a7fc700 1 -- 172.21.15.60:0/3850723690 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7f200536f0 con 0x7f7f240b8570 2023-07-13T22:28:54.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.241+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7f24073810 con 0x7f7f240b8570 2023-07-13T22:28:54.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.241+0000 7f7f1a7fc700 1 -- 172.21.15.60:0/3850723690 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7f20056d10 con 0x7f7f240b8570 2023-07-13T22:28:54.221 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:54.221 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:28:54.221 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-07-13T22:28:54.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.221 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-07-13T22:28:54.221 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw==", 2023-07-13T22:28:54.221 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.222 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:54.222 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:54.222 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:54.222 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.222 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.223 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-07-13T22:28:54.223 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g==", 2023-07-13T22:28:54.223 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:54.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:54.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:54.224 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.224 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-07-13T22:28:54.224 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA==", 2023-07-13T22:28:54.224 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.225 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:54.225 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:54.225 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:54.225 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.225 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-07-13T22:28:54.226 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ==", 2023-07-13T22:28:54.226 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.226 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:54.226 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:54.226 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:54.226 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.226 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.227 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-07-13T22:28:54.227 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ==", 2023-07-13T22:28:54.227 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.227 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:54.227 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:54.228 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.228 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.228 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-07-13T22:28:54.228 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw==", 2023-07-13T22:28:54.229 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.229 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:54.229 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:54.229 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.229 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-07-13T22:28:54.230 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q==", 2023-07-13T22:28:54.230 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.230 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:54.230 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:54.230 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.230 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-07-13T22:28:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ==", 2023-07-13T22:28:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-07-13T22:28:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg==", 2023-07-13T22:28:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.234 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-07-13T22:28:54.234 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA==", 2023-07-13T22:28:54.234 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.234 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:54.234 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:54.234 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.235 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.235 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-07-13T22:28:54.235 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw==", 2023-07-13T22:28:54.235 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.235 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:54.235 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:54.236 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.236 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-07-13T22:28:54.236 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ==", 2023-07-13T22:28:54.236 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.237 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:28:54.237 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:28:54.237 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.237 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.237 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-07-13T22:28:54.237 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w==", 2023-07-13T22:28:54.238 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.238 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:54.238 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2023-07-13T22:28:54.238 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-13T22:28:54.238 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.238 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-07-13T22:28:54.239 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw==", 2023-07-13T22:28:54.239 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.239 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:54.239 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:28:54.239 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:28:54.239 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:28:54.239 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.240 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-07-13T22:28:54.240 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg==", 2023-07-13T22:28:54.240 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.240 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:54.240 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:28:54.241 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:28:54.241 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:28:54.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.241 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-07-13T22:28:54.241 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q==", 2023-07-13T22:28:54.241 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.242 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:28:54.242 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:28:54.242 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:28:54.242 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:28:54.242 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.242 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-07-13T22:28:54.243 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBk7CgGBhAAaK88V4CEyrYfZx7wv+XG3g==", 2023-07-13T22:28:54.243 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.243 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:28:54.243 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:28:54.243 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:28:54.243 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.244 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-07-13T22:28:54.244 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ==", 2023-07-13T22:28:54.244 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.244 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:28:54.245 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:28:54.245 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:54.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:54.245 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-07-13T22:28:54.246 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ==", 2023-07-13T22:28:54.246 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:28:54.246 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:28:54.246 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:28:54.246 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:28:54.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:54.246 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:28:54.247 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:28:54.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.243+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f0806ace0 msgr2=0x7f7f0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.243+0000 7f7f2bbff700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f0806ace0 0x7f7f0806d190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7f240c7340 tx=0x7f7f14008040 comp rx=0 tx=0).stop 2023-07-13T22:28:54.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.243+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240b8570 msgr2=0x7f7f240b6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.243+0000 7f7f2bbff700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240b8570 0x7f7f240b6d50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c520 tx=0x7f7f2000c8e0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.243+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 shutdown_connections 2023-07-13T22:28:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.244+0000 7f7f2bbff700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7f0806ace0 0x7f7f0806d190 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.244+0000 7f7f2bbff700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240b8570 0x7f7f240b6d50 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.244+0000 7f7f2bbff700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f240d35c0 0x7f7f241f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.244+0000 7f7f2bbff700 1 --2- 172.21.15.60:0/3850723690 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7f240d13f0 0x7f7f240c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.244+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 >> 172.21.15.60:0/3850723690 conn(0x7f7f24071600 msgr2=0x7f7f2415de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.244+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 shutdown_connections 2023-07-13T22:28:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.244+0000 7f7f2bbff700 1 -- 172.21.15.60:0/3850723690 wait complete. 2023-07-13T22:28:54.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.473+0000 7ff469fff700 1 Processor -- start 2023-07-13T22:28:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.473+0000 7ff469fff700 1 -- start start 2023-07-13T22:28:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff469fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640b8400 0x7ff4640cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff469fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4640d0ac0 con 0x7ff4640cfca0 2023-07-13T22:28:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff469fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4640b87d0 con 0x7ff4640d3750 2023-07-13T22:28:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff469fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640b4eb0 con 0x7ff4640b8400 2023-07-13T22:28:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff463fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff4640cfca0 0x7ff4640d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43084/0 (socket says 172.21.15.60:43084) 2023-07-13T22:28:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff463fff700 1 -- 172.21.15.60:0/2447840675 learned_addr learned my addr 172.21.15.60:0/2447840675 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff4637fe700 1 --2- 172.21.15.60:0/2447840675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640b8400 0x7ff4640cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff462ffd700 1 -- 172.21.15.60:0/2447840675 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174807886 0 0) 0x7ff4640d0ac0 con 0x7ff4640cfca0 2023-07-13T22:28:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff462ffd700 1 -- 172.21.15.60:0/2447840675 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4500035b0 con 0x7ff4640cfca0 2023-07-13T22:28:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.475+0000 7ff462ffd700 1 -- 172.21.15.60:0/2447840675 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942380680 0 0) 0x7ff4640b87d0 con 0x7ff4640d3750 2023-07-13T22:28:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.476+0000 7ff462ffd700 1 -- 172.21.15.60:0/2447840675 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4640d0ac0 con 0x7ff4640d3750 2023-07-13T22:28:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.476+0000 7ff4637fe700 1 -- 172.21.15.60:0/2447840675 >> v1:172.21.15.105:6789/0 conn(0x7ff4640d3750 legacy=0x7ff4640cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.476+0000 7ff4637fe700 1 -- 172.21.15.60:0/2447840675 >> v1:172.21.15.60:6789/0 conn(0x7ff4640cfca0 legacy=0x7ff4640d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.476+0000 7ff4637fe700 1 -- 172.21.15.60:0/2447840675 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4640b5190 con 0x7ff4640b8400 2023-07-13T22:28:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.476+0000 7ff4637fe700 1 --2- 172.21.15.60:0/2447840675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640b8400 0x7ff4640cd240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff454009870 tx=0x7ff454009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f3e38371ee6168 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.476+0000 7ff462ffd700 1 -- 172.21.15.60:0/2447840675 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454011830 con 0x7ff4640b8400 2023-07-13T22:28:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.476+0000 7ff462ffd700 1 -- 172.21.15.60:0/2447840675 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454011990 con 0x7ff4640b8400 2023-07-13T22:28:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.476+0000 7ff462ffd700 1 -- 172.21.15.60:0/2447840675 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454011d20 con 0x7ff4640b8400 2023-07-13T22:28:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.477+0000 7ff469fff700 1 -- 172.21.15.60:0/2447840675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640b8400 msgr2=0x7ff4640cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.477+0000 7ff469fff700 1 --2- 172.21.15.60:0/2447840675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640b8400 0x7ff4640cd240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff454009870 tx=0x7ff454009b80 comp rx=0 tx=0).stop 2023-07-13T22:28:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.477+0000 7ff469fff700 1 -- 172.21.15.60:0/2447840675 shutdown_connections 2023-07-13T22:28:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.477+0000 7ff469fff700 1 --2- 172.21.15.60:0/2447840675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640b8400 0x7ff4640cd240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.477+0000 7ff469fff700 1 -- 172.21.15.60:0/2447840675 >> 172.21.15.60:0/2447840675 conn(0x7ff46406b390 msgr2=0x7ff46406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.477+0000 7ff469fff700 1 -- 172.21.15.60:0/2447840675 shutdown_connections 2023-07-13T22:28:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.477+0000 7ff469fff700 1 -- 172.21.15.60:0/2447840675 wait complete. 2023-07-13T22:28:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.479+0000 7ff469fff700 1 Processor -- start 2023-07-13T22:28:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.479+0000 7ff469fff700 1 -- start start 2023-07-13T22:28:54.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff469fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4640b8400 0x7ff46415d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff469fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4640cfca0 0x7ff4640c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff469fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640d3750 0x7ff464149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff469fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4640ca190 con 0x7ff4640b8400 2023-07-13T22:28:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff469fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4641f5960 con 0x7ff4640cfca0 2023-07-13T22:28:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff469fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4641f5c40 con 0x7ff4640d3750 2023-07-13T22:28:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff4637fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4640b8400 0x7ff46415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff4637fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4640b8400 0x7ff46415d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48158/0 (socket says 172.21.15.60:48158) 2023-07-13T22:28:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff4637fe700 1 -- 172.21.15.60:0/2967501756 learned_addr learned my addr 172.21.15.60:0/2967501756 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff463fff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640d3750 0x7ff464149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff45bfff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4640cfca0 0x7ff4640c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff4637fe700 1 -- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640d3750 msgr2=0x7ff464149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.480+0000 7ff4637fe700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640d3750 0x7ff464149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff4637fe700 1 -- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4640cfca0 msgr2=0x7ff4640c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff4637fe700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4640cfca0 0x7ff4640c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff4637fe700 1 -- 172.21.15.60:0/2967501756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4540094a0 con 0x7ff4640b8400 2023-07-13T22:28:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff463fff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640d3750 0x7ff464149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff45bfff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4640cfca0 0x7ff4640c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff4637fe700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4640b8400 0x7ff46415d410 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff4540090b0 tx=0x7ff454004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff460ff9700 1 -- 172.21.15.60:0/2967501756 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45401a860 con 0x7ff4640b8400 2023-07-13T22:28:54.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff460ff9700 1 -- 172.21.15.60:0/2967501756 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45401a9c0 con 0x7ff4640b8400 2023-07-13T22:28:54.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff460ff9700 1 -- 172.21.15.60:0/2967501756 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45401ad30 con 0x7ff4640b8400 2023-07-13T22:28:54.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4641f5f20 con 0x7ff4640b8400 2023-07-13T22:28:54.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.481+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4641f64e0 con 0x7ff4640b8400 2023-07-13T22:28:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.482+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4641f66c0 con 0x7ff4640b8400 2023-07-13T22:28:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.483+0000 7ff460ff9700 1 -- 172.21.15.60:0/2967501756 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff454022070 con 0x7ff4640b8400 2023-07-13T22:28:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.484+0000 7ff460ff9700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff44406ace0 0x7ff44406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.484+0000 7ff460ff9700 1 -- 172.21.15.60:0/2967501756 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff454089e80 con 0x7ff4640b8400 2023-07-13T22:28:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.484+0000 7ff45bfff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff44406ace0 0x7ff44406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.485+0000 7ff45bfff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff44406ace0 0x7ff44406d190 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff46415e070 tx=0x7ff44c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.489+0000 7ff460ff9700 1 -- 172.21.15.60:0/2967501756 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff454055320 con 0x7ff4640b8400 2023-07-13T22:28:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.694+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4640b23c0 con 0x7ff4640b8400 2023-07-13T22:28:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.696+0000 7ff460ff9700 1 -- 172.21.15.60:0/2967501756 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+14546 (secure 0 0 0) 0x7ff454029020 con 0x7ff4640b8400 2023-07-13T22:28:54.674 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:54.675 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":96,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:53.705668+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff44406ace0 msgr2=0x7ff44406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff44406ace0 0x7ff44406d190 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff46415e070 tx=0x7ff44c009040 comp rx=0 tx=0).stop 2023-07-13T22:28:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4640b8400 msgr2=0x7ff46415d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4640b8400 0x7ff46415d410 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff4540090b0 tx=0x7ff454004750 comp rx=0 tx=0).stop 2023-07-13T22:28:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 shutdown_connections 2023-07-13T22:28:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff44406ace0 0x7ff44406d190 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4640b8400 0x7ff46415d410 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4640d3750 0x7ff464149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 --2- 172.21.15.60:0/2967501756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4640cfca0 0x7ff4640c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 >> 172.21.15.60:0/2967501756 conn(0x7ff46406b390 msgr2=0x7ff4640b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:54.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 shutdown_connections 2023-07-13T22:28:54.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.698+0000 7ff469fff700 1 -- 172.21.15.60:0/2967501756 wait complete. 2023-07-13T22:28:54.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:28:54.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:28:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.924+0000 7f13385ff700 1 Processor -- start 2023-07-13T22:28:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.925+0000 7f13385ff700 1 -- start start 2023-07-13T22:28:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.926+0000 7f13385ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f13300b8200 0x7f13300cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.926+0000 7f13385ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13300b1ac0 0x7f13300cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.926+0000 7f13385ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13300d1870 con 0x7f13300cede0 2023-07-13T22:28:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.926+0000 7f13385ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13300cf370 con 0x7f13300b1ac0 2023-07-13T22:28:54.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.926+0000 7f13385ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13300d12b0 con 0x7f13300b8200 2023-07-13T22:28:54.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f1335b9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13300b1ac0 0x7f13300cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f1336b9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f13300cede0 0x7f133015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50036/0 (socket says 172.21.15.60:50036) 2023-07-13T22:28:54.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f1336b9c700 1 -- 172.21.15.60:0/1105039717 learned_addr learned my addr 172.21.15.60:0/1105039717 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:54.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f133639b700 1 --2- 172.21.15.60:0/1105039717 >> v2:172.21.15.105:3300/0 conn(0x7f13300b8200 0x7f13300cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f1335b9a700 1 -- 172.21.15.60:0/1105039717 >> v2:172.21.15.105:3300/0 conn(0x7f13300b8200 msgr2=0x7f13300cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:54.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f1335b9a700 1 --2- 172.21.15.60:0/1105039717 >> v2:172.21.15.105:3300/0 conn(0x7f13300b8200 0x7f13300cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f1335b9a700 1 -- 172.21.15.60:0/1105039717 >> v1:172.21.15.60:6789/0 conn(0x7f13300cede0 legacy=0x7f133015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f1335b9a700 1 -- 172.21.15.60:0/1105039717 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13300d1590 con 0x7f13300b1ac0 2023-07-13T22:28:54.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f1335399700 1 -- 172.21.15.60:0/1105039717 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713251039 0 0) 0x7f13300d1870 con 0x7f13300cede0 2023-07-13T22:28:54.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.927+0000 7f1335b9a700 1 --2- 172.21.15.60:0/1105039717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b1ac0 0x7f13300cfb80 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f13300b2660 tx=0x7f1320009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cca1c08befa2a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:54.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f1335399700 1 -- 172.21.15.60:0/1105039717 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320011830 con 0x7f13300b1ac0 2023-07-13T22:28:54.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f1335399700 1 -- 172.21.15.60:0/1105039717 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1320011990 con 0x7f13300b1ac0 2023-07-13T22:28:54.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f1335399700 1 -- 172.21.15.60:0/1105039717 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320011d20 con 0x7f13300b1ac0 2023-07-13T22:28:54.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f13385ff700 1 -- 172.21.15.60:0/1105039717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b1ac0 msgr2=0x7f13300cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f13385ff700 1 --2- 172.21.15.60:0/1105039717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b1ac0 0x7f13300cfb80 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f13300b2660 tx=0x7f1320009b20 comp rx=0 tx=0).stop 2023-07-13T22:28:54.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f13385ff700 1 -- 172.21.15.60:0/1105039717 shutdown_connections 2023-07-13T22:28:54.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f13385ff700 1 --2- 172.21.15.60:0/1105039717 >> v2:172.21.15.105:3300/0 conn(0x7f13300b8200 0x7f13300cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f13385ff700 1 --2- 172.21.15.60:0/1105039717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b1ac0 0x7f13300cfb80 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f13385ff700 1 -- 172.21.15.60:0/1105039717 >> 172.21.15.60:0/1105039717 conn(0x7f133006b2f0 msgr2=0x7f133006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:54.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f13385ff700 1 -- 172.21.15.60:0/1105039717 shutdown_connections 2023-07-13T22:28:54.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.928+0000 7f13385ff700 1 -- 172.21.15.60:0/1105039717 wait complete. 2023-07-13T22:28:54.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.930+0000 7f13385ff700 1 Processor -- start 2023-07-13T22:28:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.930+0000 7f13385ff700 1 -- start start 2023-07-13T22:28:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.931+0000 7f13385ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13300b1ac0 0x7f1330158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.931+0000 7f13385ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b8200 0x7f1330159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.931+0000 7f13385ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13300cede0 0x7f13300cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.931+0000 7f13385ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133015b550 con 0x7f13300b8200 2023-07-13T22:28:54.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.931+0000 7f13385ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133015d5c0 con 0x7f13300b1ac0 2023-07-13T22:28:54.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.931+0000 7f13385ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1330159680 con 0x7f13300cede0 2023-07-13T22:28:54.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.931+0000 7f1335b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b8200 0x7f1330159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.931+0000 7f1335b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b8200 0x7f1330159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57586/0 (socket says 172.21.15.60:57586) 2023-07-13T22:28:54.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.931+0000 7f1335b9a700 1 -- 172.21.15.60:0/3916191609 learned_addr learned my addr 172.21.15.60:0/3916191609 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:54.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f133639b700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13300b1ac0 0x7f1330158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1335b9a700 1 -- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13300cede0 msgr2=0x7f13300cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1336b9c700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13300cede0 0x7f13300cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1335b9a700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13300cede0 0x7f13300cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1335b9a700 1 -- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13300b1ac0 msgr2=0x7f1330158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:54.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1335b9a700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13300b1ac0 0x7f1330158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:54.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1335b9a700 1 -- 172.21.15.60:0/3916191609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13200094a0 con 0x7f13300b8200 2023-07-13T22:28:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f133639b700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13300b1ac0 0x7f1330158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1336b9c700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13300cede0 0x7f13300cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1335b9a700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b8200 0x7f1330159140 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f1320014040 tx=0x7f1320004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1326ffd700 1 -- 172.21.15.60:0/3916191609 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13200038a0 con 0x7f13300b8200 2023-07-13T22:28:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1326ffd700 1 -- 172.21.15.60:0/3916191609 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132001e070 con 0x7f13300b8200 2023-07-13T22:28:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13300cc6c0 con 0x7f13300b8200 2023-07-13T22:28:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.932+0000 7f1326ffd700 1 -- 172.21.15.60:0/3916191609 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320003c20 con 0x7f13300b8200 2023-07-13T22:28:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.933+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133015cf70 con 0x7f13300b8200 2023-07-13T22:28:54.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.933+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133015d750 con 0x7f13300b8200 2023-07-13T22:28:54.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.934+0000 7f1326ffd700 1 -- 172.21.15.60:0/3916191609 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f132002d450 con 0x7f13300b8200 2023-07-13T22:28:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.935+0000 7f1326ffd700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f131406ace0 0x7f131406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.935+0000 7f1326ffd700 1 -- 172.21.15.60:0/3916191609 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f132008a100 con 0x7f13300b8200 2023-07-13T22:28:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.936+0000 7f133639b700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f131406ace0 0x7f131406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.936+0000 7f133639b700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f131406ace0 0x7f131406d190 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f132c009b60 tx=0x7f132c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:54.939+0000 7f1326ffd700 1 -- 172.21.15.60:0/3916191609 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f13200554f0 con 0x7f13300b8200 2023-07-13T22:28:55.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.145+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f13300b53c0 con 0x7f13300b8200 2023-07-13T22:28:55.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.146+0000 7f1326ffd700 1 -- 172.21.15.60:0/3916191609 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v96) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1320058b10 con 0x7f13300b8200 2023-07-13T22:28:55.125 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:28:55.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f131406ace0 msgr2=0x7f131406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f131406ace0 0x7f131406d190 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f132c009b60 tx=0x7f132c008040 comp rx=0 tx=0).stop 2023-07-13T22:28:55.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b8200 msgr2=0x7f1330159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b8200 0x7f1330159140 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f1320014040 tx=0x7f1320004750 comp rx=0 tx=0).stop 2023-07-13T22:28:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 shutdown_connections 2023-07-13T22:28:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f131406ace0 0x7f131406d190 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13300cede0 0x7f13300cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13300b1ac0 0x7f1330158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 --2- 172.21.15.60:0/3916191609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13300b8200 0x7f1330159140 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 >> 172.21.15.60:0/3916191609 conn(0x7f133006b2f0 msgr2=0x7f13300b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:55.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 shutdown_connections 2023-07-13T22:28:55.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.148+0000 7f13385ff700 1 -- 172.21.15.60:0/3916191609 wait complete. 2023-07-13T22:28:55.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:28:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.373+0000 7fd4631ff700 1 Processor -- start 2023-07-13T22:28:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.373+0000 7fd4631ff700 1 -- start start 2023-07-13T22:28:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.374+0000 7fd4631ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd45c15c460 0x7fd45c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.374+0000 7fd4631ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c0caf80 0x7fd45c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.374+0000 7fd4631ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd45c06c8f0 0x7fd45c15fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.374+0000 7fd4631ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c06b9f0 con 0x7fd45c06c8f0 2023-07-13T22:28:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.374+0000 7fd4631ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c06c750 con 0x7fd45c15c460 2023-07-13T22:28:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.374+0000 7fd4631ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c06bf40 con 0x7fd45c0caf80 2023-07-13T22:28:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.374+0000 7fd46179c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd45c06c8f0 0x7fd45c15fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.374+0000 7fd46179c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd45c06c8f0 0x7fd45c15fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57594/0 (socket says 172.21.15.60:57594) 2023-07-13T22:28:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.374+0000 7fd46179c700 1 -- 172.21.15.60:0/2551233956 learned_addr learned my addr 172.21.15.60:0/2551233956 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd460f9b700 1 --2- 172.21.15.60:0/2551233956 >> v2:172.21.15.105:3300/0 conn(0x7fd45c15c460 0x7fd45c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd45bfff700 1 --2- 172.21.15.60:0/2551233956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c0caf80 0x7fd45c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd46179c700 1 -- 172.21.15.60:0/2551233956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c0caf80 msgr2=0x7fd45c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:55.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd46179c700 1 --2- 172.21.15.60:0/2551233956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c0caf80 0x7fd45c0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd46179c700 1 -- 172.21.15.60:0/2551233956 >> v2:172.21.15.105:3300/0 conn(0x7fd45c15c460 msgr2=0x7fd45c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:55.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd46179c700 1 --2- 172.21.15.60:0/2551233956 >> v2:172.21.15.105:3300/0 conn(0x7fd45c15c460 0x7fd45c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd46179c700 1 -- 172.21.15.60:0/2551233956 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c0cb3f0 con 0x7fd45c06c8f0 2023-07-13T22:28:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd45bfff700 1 --2- 172.21.15.60:0/2551233956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c0caf80 0x7fd45c0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd46179c700 1 --2- 172.21.15.60:0/2551233956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c06c8f0 0x7fd45c15fdd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd45c072360 tx=0x7fd45400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10c34d86c94d2e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd45b7fe700 1 -- 172.21.15.60:0/2551233956 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45400e070 con 0x7fd45c06c8f0 2023-07-13T22:28:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd45b7fe700 1 -- 172.21.15.60:0/2551233956 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45400b910 con 0x7fd45c06c8f0 2023-07-13T22:28:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.375+0000 7fd45b7fe700 1 -- 172.21.15.60:0/2551233956 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45400bcc0 con 0x7fd45c06c8f0 2023-07-13T22:28:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.376+0000 7fd4631ff700 1 -- 172.21.15.60:0/2551233956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c06c8f0 msgr2=0x7fd45c15fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.376+0000 7fd4631ff700 1 --2- 172.21.15.60:0/2551233956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c06c8f0 0x7fd45c15fdd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd45c072360 tx=0x7fd45400b330 comp rx=0 tx=0).stop 2023-07-13T22:28:55.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.376+0000 7fd4631ff700 1 -- 172.21.15.60:0/2551233956 shutdown_connections 2023-07-13T22:28:55.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.376+0000 7fd4631ff700 1 --2- 172.21.15.60:0/2551233956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c06c8f0 0x7fd45c15fdd0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.376+0000 7fd4631ff700 1 --2- 172.21.15.60:0/2551233956 >> v2:172.21.15.105:3300/0 conn(0x7fd45c15c460 0x7fd45c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.376+0000 7fd4631ff700 1 --2- 172.21.15.60:0/2551233956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c0caf80 0x7fd45c0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.376+0000 7fd4631ff700 1 -- 172.21.15.60:0/2551233956 >> 172.21.15.60:0/2551233956 conn(0x7fd45c06e580 msgr2=0x7fd45c06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:55.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.376+0000 7fd4631ff700 1 -- 172.21.15.60:0/2551233956 shutdown_connections 2023-07-13T22:28:55.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.376+0000 7fd4631ff700 1 -- 172.21.15.60:0/2551233956 wait complete. 2023-07-13T22:28:55.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.378+0000 7fd4631ff700 1 Processor -- start 2023-07-13T22:28:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.378+0000 7fd4631ff700 1 -- start start 2023-07-13T22:28:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.379+0000 7fd4631ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c06c8f0 0x7fd45c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.379+0000 7fd4631ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c0caf80 0x7fd45c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.379+0000 7fd4631ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45c15c460 0x7fd45c0b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.379+0000 7fd4631ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c15f4a0 con 0x7fd45c0caf80 2023-07-13T22:28:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.379+0000 7fd4631ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c158bf0 con 0x7fd45c15c460 2023-07-13T22:28:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.379+0000 7fd4631ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c158850 con 0x7fd45c06c8f0 2023-07-13T22:28:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.379+0000 7fd45bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c0caf80 0x7fd45c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.379+0000 7fd45bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c0caf80 0x7fd45c15e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57598/0 (socket says 172.21.15.60:57598) 2023-07-13T22:28:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.379+0000 7fd45bfff700 1 -- 172.21.15.60:0/14239277 learned_addr learned my addr 172.21.15.60:0/14239277 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd46179c700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45c15c460 0x7fd45c0b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd460f9b700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c06c8f0 0x7fd45c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd45bfff700 1 -- 172.21.15.60:0/14239277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c06c8f0 msgr2=0x7fd45c157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd45bfff700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c06c8f0 0x7fd45c157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd45bfff700 1 -- 172.21.15.60:0/14239277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45c15c460 msgr2=0x7fd45c0b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd45bfff700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45c15c460 0x7fd45c0b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd45bfff700 1 -- 172.21.15.60:0/14239277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454009b30 con 0x7fd45c0caf80 2023-07-13T22:28:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd46179c700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45c15c460 0x7fd45c0b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd45bfff700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c0caf80 0x7fd45c15e240 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd44800e810 tx=0x7fd44800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd4597fa700 1 -- 172.21.15.60:0/14239277 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448013610 con 0x7fd45c0caf80 2023-07-13T22:28:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd4597fa700 1 -- 172.21.15.60:0/14239277 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd448013770 con 0x7fd45c0caf80 2023-07-13T22:28:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c15ec00 con 0x7fd45c0caf80 2023-07-13T22:28:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd4597fa700 1 -- 172.21.15.60:0/14239277 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448013b20 con 0x7fd45c0caf80 2023-07-13T22:28:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.380+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c15f1c0 con 0x7fd45c0caf80 2023-07-13T22:28:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.381+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c04e210 con 0x7fd45c0caf80 2023-07-13T22:28:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.382+0000 7fd4597fa700 1 -- 172.21.15.60:0/14239277 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd448013c80 con 0x7fd45c0caf80 2023-07-13T22:28:55.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.384+0000 7fd4597fa700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd44006ace0 0x7fd44006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.384+0000 7fd4597fa700 1 -- 172.21.15.60:0/14239277 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd448088e00 con 0x7fd45c0caf80 2023-07-13T22:28:55.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.384+0000 7fd460f9b700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd44006ace0 0x7fd44006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.385+0000 7fd460f9b700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd44006ace0 0x7fd44006d190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd450009af0 tx=0x7fd450008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:55.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.388+0000 7fd4597fa700 1 -- 172.21.15.60:0/14239277 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd448057c80 con 0x7fd45c0caf80 2023-07-13T22:28:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.593+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd45c0b5030 con 0x7fd45c0caf80 2023-07-13T22:28:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.594+0000 7fd4597fa700 1 -- 172.21.15.60:0/14239277 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4480210a0 con 0x7fd45c0caf80 2023-07-13T22:28:55.573 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:28:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd44006ace0 msgr2=0x7fd44006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd44006ace0 0x7fd44006d190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd450009af0 tx=0x7fd450008040 comp rx=0 tx=0).stop 2023-07-13T22:28:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c0caf80 msgr2=0x7fd45c15e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c0caf80 0x7fd45c15e240 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd44800e810 tx=0x7fd44800ebd0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 shutdown_connections 2023-07-13T22:28:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd44006ace0 0x7fd44006d190 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45c15c460 0x7fd45c0b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd45c06c8f0 0x7fd45c157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 --2- 172.21.15.60:0/14239277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd45c0caf80 0x7fd45c15e240 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.596+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 >> 172.21.15.60:0/14239277 conn(0x7fd45c06e580 msgr2=0x7fd45c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:55.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.597+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 shutdown_connections 2023-07-13T22:28:55.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.597+0000 7fd4631ff700 1 -- 172.21.15.60:0/14239277 wait complete. 2023-07-13T22:28:55.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:28:55.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.818+0000 7fa8907ff700 1 Processor -- start 2023-07-13T22:28:55.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.818+0000 7fa8907ff700 1 -- start start 2023-07-13T22:28:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.819+0000 7fa8907ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa8880b8400 0x7fa8880cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.819+0000 7fa8907ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880d3750 0x7fa8880cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.819+0000 7fa8907ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa8880cfad0 0x7fa88815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.819+0000 7fa8907ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8880cff20 con 0x7fa8880cfad0 2023-07-13T22:28:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.819+0000 7fa8907ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8880d0ac0 con 0x7fa8880b8400 2023-07-13T22:28:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.819+0000 7fa8907ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880b87d0 con 0x7fa8880d3750 2023-07-13T22:28:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88ed9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa8880cfad0 0x7fa88815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88ed9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa8880cfad0 0x7fa88815fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57600/0 (socket says 172.21.15.60:57600) 2023-07-13T22:28:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88ed9c700 1 -- 172.21.15.60:0/2591334772 learned_addr learned my addr 172.21.15.60:0/2591334772 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88dd9a700 1 --2- 172.21.15.60:0/2591334772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880d3750 0x7fa8880cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88e59b700 1 --2- 172.21.15.60:0/2591334772 >> v2:172.21.15.105:3300/0 conn(0x7fa8880b8400 0x7fa8880cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88ed9c700 1 -- 172.21.15.60:0/2591334772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880d3750 msgr2=0x7fa8880cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88ed9c700 1 --2- 172.21.15.60:0/2591334772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880d3750 0x7fa8880cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88ed9c700 1 -- 172.21.15.60:0/2591334772 >> v2:172.21.15.105:3300/0 conn(0x7fa8880b8400 msgr2=0x7fa8880cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88ed9c700 1 --2- 172.21.15.60:0/2591334772 >> v2:172.21.15.105:3300/0 conn(0x7fa8880b8400 0x7fa8880cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88ed9c700 1 -- 172.21.15.60:0/2591334772 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880b4eb0 con 0x7fa8880cfad0 2023-07-13T22:28:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.820+0000 7fa88ed9c700 1 --2- 172.21.15.60:0/2591334772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880cfad0 0x7fa88815fef0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa8880cf560 tx=0x7fa88000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba6017670bd745c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa88d599700 1 -- 172.21.15.60:0/2591334772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88000e070 con 0x7fa8880cfad0 2023-07-13T22:28:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa88d599700 1 -- 172.21.15.60:0/2591334772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88000b9c0 con 0x7fa8880cfad0 2023-07-13T22:28:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa88d599700 1 -- 172.21.15.60:0/2591334772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88000bd70 con 0x7fa8880cfad0 2023-07-13T22:28:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa8907ff700 1 -- 172.21.15.60:0/2591334772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880cfad0 msgr2=0x7fa88815fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2591334772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880cfad0 0x7fa88815fef0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa8880cf560 tx=0x7fa88000b330 comp rx=0 tx=0).stop 2023-07-13T22:28:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa8907ff700 1 -- 172.21.15.60:0/2591334772 shutdown_connections 2023-07-13T22:28:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2591334772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880cfad0 0x7fa88815fef0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2591334772 >> v2:172.21.15.105:3300/0 conn(0x7fa8880b8400 0x7fa8880cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2591334772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880d3750 0x7fa8880cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa8907ff700 1 -- 172.21.15.60:0/2591334772 >> 172.21.15.60:0/2591334772 conn(0x7fa88806b390 msgr2=0x7fa88806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa8907ff700 1 -- 172.21.15.60:0/2591334772 shutdown_connections 2023-07-13T22:28:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.821+0000 7fa8907ff700 1 -- 172.21.15.60:0/2591334772 wait complete. 2023-07-13T22:28:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.823+0000 7fa8907ff700 1 Processor -- start 2023-07-13T22:28:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.823+0000 7fa8907ff700 1 -- start start 2023-07-13T22:28:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.824+0000 7fa8907ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880b8400 0x7fa8880c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.824+0000 7fa8907ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880cfad0 0x7fa88814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.824+0000 7fa8907ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8880d3750 0x7fa88815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.824+0000 7fa8907ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880c8840 con 0x7fa8880b8400 2023-07-13T22:28:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.824+0000 7fa8907ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880c7bc0 con 0x7fa8880d3750 2023-07-13T22:28:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.824+0000 7fa8907ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8880c7ea0 con 0x7fa8880cfad0 2023-07-13T22:28:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.824+0000 7fa88e59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880b8400 0x7fa8880c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88e59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880b8400 0x7fa8880c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57610/0 (socket says 172.21.15.60:57610) 2023-07-13T22:28:55.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88e59b700 1 -- 172.21.15.60:0/2148760167 learned_addr learned my addr 172.21.15.60:0/2148760167 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:55.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88dd9a700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880cfad0 0x7fa88814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88ed9c700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8880d3750 0x7fa88815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88e59b700 1 -- 172.21.15.60:0/2148760167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880cfad0 msgr2=0x7fa88814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88e59b700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880cfad0 0x7fa88814ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88e59b700 1 -- 172.21.15.60:0/2148760167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8880d3750 msgr2=0x7fa88815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:55.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88e59b700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8880d3750 0x7fa88815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:55.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88e59b700 1 -- 172.21.15.60:0/2148760167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880009b30 con 0x7fa8880b8400 2023-07-13T22:28:55.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa88e59b700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880b8400 0x7fa8880c70c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa88400d800 tx=0x7fa88400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:55.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.825+0000 7fa87effd700 1 -- 172.21.15.60:0/2148760167 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8840125f0 con 0x7fa8880b8400 2023-07-13T22:28:55.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.826+0000 7fa87effd700 1 -- 172.21.15.60:0/2148760167 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa884012750 con 0x7fa8880b8400 2023-07-13T22:28:55.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.826+0000 7fa87effd700 1 -- 172.21.15.60:0/2148760167 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884012b00 con 0x7fa8880b8400 2023-07-13T22:28:55.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.826+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8880c8180 con 0x7fa8880b8400 2023-07-13T22:28:55.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.826+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8880cc970 con 0x7fa8880b8400 2023-07-13T22:28:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.827+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8880ce740 con 0x7fa8880b8400 2023-07-13T22:28:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.828+0000 7fa87effd700 1 -- 172.21.15.60:0/2148760167 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa884012c60 con 0x7fa8880b8400 2023-07-13T22:28:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.829+0000 7fa87effd700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa86c06ace0 0x7fa86c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.829+0000 7fa87effd700 1 -- 172.21.15.60:0/2148760167 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa884089520 con 0x7fa8880b8400 2023-07-13T22:28:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.829+0000 7fa88dd9a700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa86c06ace0 0x7fa86c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.830+0000 7fa88dd9a700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa86c06ace0 0x7fa86c06d190 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa88815ee40 tx=0x7fa878008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:55.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:55.833+0000 7fa87effd700 1 -- 172.21.15.60:0/2148760167 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa884054920 con 0x7fa8880b8400 2023-07-13T22:28:56.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.041+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa8880c8ab0 con 0x7fa8880b8400 2023-07-13T22:28:56.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.041+0000 7fa87effd700 1 -- 172.21.15.60:0/2148760167 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa884057f40 con 0x7fa8880b8400 2023-07-13T22:28:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:56.021 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:28:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:28:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:28:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:28:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:28:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:28:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:28:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:28:56.022 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:28:56.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.043+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa86c06ace0 msgr2=0x7fa86c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.043+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa86c06ace0 0x7fa86c06d190 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa88815ee40 tx=0x7fa878008040 comp rx=0 tx=0).stop 2023-07-13T22:28:56.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.043+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880b8400 msgr2=0x7fa8880c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.043+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880b8400 0x7fa8880c70c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa88400d800 tx=0x7fa88400dbc0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.044+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 shutdown_connections 2023-07-13T22:28:56.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.044+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa86c06ace0 0x7fa86c06d190 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.044+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8880b8400 0x7fa8880c70c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.044+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8880d3750 0x7fa88815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.044+0000 7fa8907ff700 1 --2- 172.21.15.60:0/2148760167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8880cfad0 0x7fa88814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.044+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 >> 172.21.15.60:0/2148760167 conn(0x7fa88806b390 msgr2=0x7fa8880b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:56.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.044+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 shutdown_connections 2023-07-13T22:28:56.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.044+0000 7fa8907ff700 1 -- 172.21.15.60:0/2148760167 wait complete. 2023-07-13T22:28:56.040 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:28:56.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:28:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.270+0000 7fbb733ff700 1 Processor -- start 2023-07-13T22:28:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.270+0000 7fbb733ff700 1 -- start start 2023-07-13T22:28:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.271+0000 7fbb733ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 0x7fbb6c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.271+0000 7fbb733ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c15cab0 con 0x7fbb6c0ce2c0 2023-07-13T22:28:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.271+0000 7fbb733ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c0ca820 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.271+0000 7fbb733ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0cafd0 con 0x7fbb6c15c6e0 2023-07-13T22:28:56.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.271+0000 7fbb7199c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbb6c0ce2c0 0x7fbb6c06cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50052/0 (socket says 172.21.15.60:50052) 2023-07-13T22:28:56.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.271+0000 7fbb7199c700 1 -- 172.21.15.60:0/3261962345 learned_addr learned my addr 172.21.15.60:0/3261962345 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:56.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.272+0000 7fbb7119b700 1 --2- 172.21.15.60:0/3261962345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 0x7fbb6c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.272+0000 7fbb63fff700 1 -- 172.21.15.60:0/3261962345 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571913841 0 0) 0x7fbb6c15cab0 con 0x7fbb6c0ce2c0 2023-07-13T22:28:56.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.272+0000 7fbb63fff700 1 -- 172.21.15.60:0/3261962345 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb580035b0 con 0x7fbb6c0ce2c0 2023-07-13T22:28:56.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.272+0000 7fbb63fff700 1 -- 172.21.15.60:0/3261962345 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255273873 0 0) 0x7fbb6c0ca820 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.272+0000 7fbb63fff700 1 -- 172.21.15.60:0/3261962345 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb6c15cab0 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.272+0000 7fbb7119b700 1 -- 172.21.15.60:0/3261962345 >> v1:172.21.15.105:6789/0 conn(0x7fbb6c0c8da0 legacy=0x7fbb6c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.272+0000 7fbb7119b700 1 -- 172.21.15.60:0/3261962345 >> v1:172.21.15.60:6789/0 conn(0x7fbb6c0ce2c0 legacy=0x7fbb6c06cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.272+0000 7fbb7119b700 1 -- 172.21.15.60:0/3261962345 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c0caa50 con 0x7fbb6c15c6e0 2023-07-13T22:28:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.272+0000 7fbb7119b700 1 --2- 172.21.15.60:0/3261962345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 0x7fbb6c15d6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009870 tx=0x7fbb68009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5de4f9434fbefeec server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.273+0000 7fbb63fff700 1 -- 172.21.15.60:0/3261962345 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68011830 con 0x7fbb6c15c6e0 2023-07-13T22:28:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.273+0000 7fbb63fff700 1 -- 172.21.15.60:0/3261962345 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb68011990 con 0x7fbb6c15c6e0 2023-07-13T22:28:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.273+0000 7fbb63fff700 1 -- 172.21.15.60:0/3261962345 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68011d00 con 0x7fbb6c15c6e0 2023-07-13T22:28:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.273+0000 7fbb733ff700 1 -- 172.21.15.60:0/3261962345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 msgr2=0x7fbb6c15d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.273+0000 7fbb733ff700 1 --2- 172.21.15.60:0/3261962345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 0x7fbb6c15d6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009870 tx=0x7fbb68009b80 comp rx=0 tx=0).stop 2023-07-13T22:28:56.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.273+0000 7fbb733ff700 1 -- 172.21.15.60:0/3261962345 shutdown_connections 2023-07-13T22:28:56.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.273+0000 7fbb733ff700 1 --2- 172.21.15.60:0/3261962345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 0x7fbb6c15d6c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.273+0000 7fbb733ff700 1 -- 172.21.15.60:0/3261962345 >> 172.21.15.60:0/3261962345 conn(0x7fbb6c0cf3b0 msgr2=0x7fbb6c0cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:56.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.273+0000 7fbb733ff700 1 -- 172.21.15.60:0/3261962345 shutdown_connections 2023-07-13T22:28:56.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.274+0000 7fbb733ff700 1 -- 172.21.15.60:0/3261962345 wait complete. 2023-07-13T22:28:56.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.275+0000 7fbb733ff700 1 Processor -- start 2023-07-13T22:28:56.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.275+0000 7fbb733ff700 1 -- start start 2023-07-13T22:28:56.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.276+0000 7fbb733ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb6c0c8da0 0x7fbb6c0b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.276+0000 7fbb733ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb6c0ce2c0 0x7fbb6c14cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.276+0000 7fbb733ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 0x7fbb6c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.276+0000 7fbb733ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0b1810 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.276+0000 7fbb733ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0b1ab0 con 0x7fbb6c0ce2c0 2023-07-13T22:28:56.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.276+0000 7fbb733ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0b1d90 con 0x7fbb6c15c6e0 2023-07-13T22:28:56.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.276+0000 7fbb7119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb6c0c8da0 0x7fbb6c0b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb6c0c8da0 0x7fbb6c0b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57612/0 (socket says 172.21.15.60:57612) 2023-07-13T22:28:56.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7119b700 1 -- 172.21.15.60:0/3038636047 learned_addr learned my addr 172.21.15.60:0/3038636047 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:56.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7199c700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 0x7fbb6c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7099a700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb6c0ce2c0 0x7fbb6c14cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7119b700 1 -- 172.21.15.60:0/3038636047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 msgr2=0x7fbb6c0b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7119b700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 0x7fbb6c0b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7119b700 1 -- 172.21.15.60:0/3038636047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb6c0ce2c0 msgr2=0x7fbb6c14cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7119b700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb6c0ce2c0 0x7fbb6c14cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7119b700 1 -- 172.21.15.60:0/3038636047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb680094a0 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb7119b700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb6c0c8da0 0x7fbb6c0b2b50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009bc0 tx=0x7fbb68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:56.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.277+0000 7fbb61ffb700 1 -- 172.21.15.60:0/3038636047 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6801a860 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.278+0000 7fbb61ffb700 1 -- 172.21.15.60:0/3038636047 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6801a9c0 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.278+0000 7fbb61ffb700 1 -- 172.21.15.60:0/3038636047 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6801ad30 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.278+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c0b2070 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.278+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c0b2630 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.279+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c0b29a0 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.280+0000 7fbb61ffb700 1 -- 172.21.15.60:0/3038636047 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb68022070 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.280+0000 7fbb61ffb700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb5006ad30 0x7fbb5006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.281+0000 7fbb61ffb700 1 -- 172.21.15.60:0/3038636047 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbb6808a220 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.281+0000 7fbb7099a700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb5006ad30 0x7fbb5006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.281+0000 7fbb7099a700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb5006ad30 0x7fbb5006d1e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c14d9c0 tx=0x7fbb5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:56.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.286+0000 7fbb61ffb700 1 -- 172.21.15.60:0/3038636047 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbb68055860 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.493+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbb6c0b27c0 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.724+0000 7fbb61ffb700 1 -- 172.21.15.60:0/3038636047 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbb68058e80 con 0x7fbb6c0c8da0 2023-07-13T22:28:56.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.727+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb5006ad30 msgr2=0x7fbb5006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.727+0000 7fbb733ff700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb5006ad30 0x7fbb5006d1e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c14d9c0 tx=0x7fbb5c009040 comp rx=0 tx=0).stop 2023-07-13T22:28:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.727+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb6c0c8da0 msgr2=0x7fbb6c0b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.727+0000 7fbb733ff700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb6c0c8da0 0x7fbb6c0b2b50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009bc0 tx=0x7fbb68004750 comp rx=0 tx=0).stop 2023-07-13T22:28:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.728+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 shutdown_connections 2023-07-13T22:28:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.728+0000 7fbb733ff700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb5006ad30 0x7fbb5006d1e0 secure :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c14d9c0 tx=0x7fbb5c009040 comp rx=0 tx=0).stop 2023-07-13T22:28:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.728+0000 7fbb733ff700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb6c0c8da0 0x7fbb6c0b2b50 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.728+0000 7fbb733ff700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb6c15c6e0 0x7fbb6c0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.728+0000 7fbb733ff700 1 --2- 172.21.15.60:0/3038636047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb6c0ce2c0 0x7fbb6c14cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.728+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 >> 172.21.15.60:0/3038636047 conn(0x7fbb6c0cf3b0 msgr2=0x7fbb6c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.729+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 shutdown_connections 2023-07-13T22:28:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.729+0000 7fbb733ff700 1 -- 172.21.15.60:0/3038636047 wait complete. 2023-07-13T22:28:56.711 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:28:56.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:28:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.960+0000 7f43c55ff700 1 Processor -- start 2023-07-13T22:28:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.960+0000 7f43c55ff700 1 -- start start 2023-07-13T22:28:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.961+0000 7f43c55ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c00d0890 0x7f43c00d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.961+0000 7f43c55ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c00cdb20 con 0x7f43c00cd6d0 2023-07-13T22:28:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.961+0000 7f43c55ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c015d200 con 0x7f43c0159ea0 2023-07-13T22:28:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.961+0000 7f43c55ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c00d2e60 con 0x7f43c00d0890 2023-07-13T22:28:56.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.962+0000 7f43be7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43c00cd6d0 0x7f43c00d3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50064/0 (socket says 172.21.15.60:50064) 2023-07-13T22:28:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.962+0000 7f43be7fc700 1 -- 172.21.15.60:0/953516908 learned_addr learned my addr 172.21.15.60:0/953516908 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.962+0000 7f43bf7fe700 1 --2- 172.21.15.60:0/953516908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c00d0890 0x7f43c00d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.962+0000 7f43bdffb700 1 -- 172.21.15.60:0/953516908 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690712807 0 0) 0x7f43c00cdb20 con 0x7f43c00cd6d0 2023-07-13T22:28:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.962+0000 7f43bdffb700 1 -- 172.21.15.60:0/953516908 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a80035b0 con 0x7f43c00cd6d0 2023-07-13T22:28:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.962+0000 7f43bdffb700 1 -- 172.21.15.60:0/953516908 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603694820 0 0) 0x7f43c015d200 con 0x7f43c0159ea0 2023-07-13T22:28:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.962+0000 7f43bdffb700 1 -- 172.21.15.60:0/953516908 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43c00cdb20 con 0x7f43c0159ea0 2023-07-13T22:28:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.963+0000 7f43bf7fe700 1 -- 172.21.15.60:0/953516908 >> v1:172.21.15.105:6789/0 conn(0x7f43c0159ea0 legacy=0x7f43c015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.963+0000 7f43bdffb700 1 -- 172.21.15.60:0/953516908 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 267516348 0 0) 0x7f43a80035b0 con 0x7f43c00cd6d0 2023-07-13T22:28:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.963+0000 7f43bf7fe700 1 -- 172.21.15.60:0/953516908 >> v1:172.21.15.60:6789/0 conn(0x7f43c00cd6d0 legacy=0x7f43c00d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.963+0000 7f43bf7fe700 1 -- 172.21.15.60:0/953516908 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c015d4e0 con 0x7f43c00d0890 2023-07-13T22:28:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.963+0000 7f43bf7fe700 1 --2- 172.21.15.60:0/953516908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c00d0890 0x7f43c00d0cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009dc0 tx=0x7f43b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86cb14d4625ed716 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.963+0000 7f43bdffb700 1 -- 172.21.15.60:0/953516908 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b800e070 con 0x7f43c00d0890 2023-07-13T22:28:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.963+0000 7f43bdffb700 1 -- 172.21.15.60:0/953516908 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b8003a20 con 0x7f43c00d0890 2023-07-13T22:28:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.963+0000 7f43bdffb700 1 -- 172.21.15.60:0/953516908 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4001670 con 0x7f43c00d0890 2023-07-13T22:28:56.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.964+0000 7f43c55ff700 1 -- 172.21.15.60:0/953516908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c00d0890 msgr2=0x7f43c00d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.964+0000 7f43c55ff700 1 --2- 172.21.15.60:0/953516908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c00d0890 0x7f43c00d0cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009dc0 tx=0x7f43b800b330 comp rx=0 tx=0).stop 2023-07-13T22:28:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.964+0000 7f43c55ff700 1 -- 172.21.15.60:0/953516908 shutdown_connections 2023-07-13T22:28:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.964+0000 7f43c55ff700 1 --2- 172.21.15.60:0/953516908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c00d0890 0x7f43c00d0cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.964+0000 7f43c55ff700 1 -- 172.21.15.60:0/953516908 >> 172.21.15.60:0/953516908 conn(0x7f43c0071600 msgr2=0x7f43c00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.964+0000 7f43c55ff700 1 -- 172.21.15.60:0/953516908 shutdown_connections 2023-07-13T22:28:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.964+0000 7f43c55ff700 1 -- 172.21.15.60:0/953516908 wait complete. 2023-07-13T22:28:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.966+0000 7f43c55ff700 1 Processor -- start 2023-07-13T22:28:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.966+0000 7f43c55ff700 1 -- start start 2023-07-13T22:28:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43c55ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c00cd6d0 0x7f43c0153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43c55ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c00d0890 0x7f43c0153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43c55ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c0159ea0 0x7f43c00ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43c55ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0153a50 con 0x7f43c00cd6d0 2023-07-13T22:28:56.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43c55ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c00c88e0 con 0x7f43c00d0890 2023-07-13T22:28:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43c55ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c00c8600 con 0x7f43c0159ea0 2023-07-13T22:28:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43beffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c00cd6d0 0x7f43c0153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43beffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c00cd6d0 0x7f43c0153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57620/0 (socket says 172.21.15.60:57620) 2023-07-13T22:28:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43beffd700 1 -- 172.21.15.60:0/4196440589 learned_addr learned my addr 172.21.15.60:0/4196440589 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43be7fc700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c00d0890 0x7f43c0153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.967+0000 7f43bf7fe700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c0159ea0 0x7f43c00ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43beffd700 1 -- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c0159ea0 msgr2=0x7f43c00ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43beffd700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c0159ea0 0x7f43c00ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43beffd700 1 -- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c00d0890 msgr2=0x7f43c0153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43beffd700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c00d0890 0x7f43c0153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43beffd700 1 -- 172.21.15.60:0/4196440589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8009a70 con 0x7f43c00cd6d0 2023-07-13T22:28:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43bf7fe700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c0159ea0 0x7f43c00ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43be7fc700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c00d0890 0x7f43c0153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43beffd700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c00cd6d0 0x7f43c0153c10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c500 tx=0x7f43b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43a77fe700 1 -- 172.21.15.60:0/4196440589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b00052d0 con 0x7f43c00cd6d0 2023-07-13T22:28:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43a77fe700 1 -- 172.21.15.60:0/4196440589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b0005450 con 0x7f43c00cd6d0 2023-07-13T22:28:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43a77fe700 1 -- 172.21.15.60:0/4196440589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b00038c0 con 0x7f43c00cd6d0 2023-07-13T22:28:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c00c9ec0 con 0x7f43c00cd6d0 2023-07-13T22:28:56.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.968+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c00ca480 con 0x7f43c00cd6d0 2023-07-13T22:28:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.969+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c00ca640 con 0x7f43c00cd6d0 2023-07-13T22:28:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.971+0000 7f43a77fe700 1 -- 172.21.15.60:0/4196440589 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f43b0003a20 con 0x7f43c00cd6d0 2023-07-13T22:28:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.972+0000 7f43a77fe700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f43ac06ace0 0x7f43ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.972+0000 7f43a77fe700 1 -- 172.21.15.60:0/4196440589 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f43b0088c10 con 0x7f43c00cd6d0 2023-07-13T22:28:56.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.972+0000 7f43be7fc700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f43ac06ace0 0x7f43ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:56.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.972+0000 7f43be7fc700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f43ac06ace0 0x7f43ac06d190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f43b4001eb0 tx=0x7f43b4001ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:56.975+0000 7f43a77fe700 1 -- 172.21.15.60:0/4196440589 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f43b0053f10 con 0x7f43c00cd6d0 2023-07-13T22:28:57.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:28:57.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:28:57.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.184+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f43c0067a70 con 0x7f43c00cd6d0 2023-07-13T22:28:57.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.732+0000 7f43a77fe700 1 -- 172.21.15.60:0/4196440589 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) v1 ==== 170+0+0 (secure 0 0 0) 0x7f43b0057530 con 0x7f43c00cd6d0 2023-07-13T22:28:57.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.745+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f43ac06ace0 msgr2=0x7f43ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:57.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.745+0000 7f43c55ff700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f43ac06ace0 0x7f43ac06d190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f43b4001eb0 tx=0x7f43b4001ca0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.745+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c00cd6d0 msgr2=0x7f43c0153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:57.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.745+0000 7f43c55ff700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c00cd6d0 0x7f43c0153c10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c500 tx=0x7f43b000c8c0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.746+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 shutdown_connections 2023-07-13T22:28:57.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.746+0000 7f43c55ff700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f43ac06ace0 0x7f43ac06d190 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.746+0000 7f43c55ff700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c00cd6d0 0x7f43c0153c10 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.746+0000 7f43c55ff700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43c0159ea0 0x7f43c00ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.746+0000 7f43c55ff700 1 --2- 172.21.15.60:0/4196440589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c00d0890 0x7f43c0153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.746+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 >> 172.21.15.60:0/4196440589 conn(0x7f43c0071600 msgr2=0x7f43c015efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:57.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.750+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 shutdown_connections 2023-07-13T22:28:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.751+0000 7f43c55ff700 1 -- 172.21.15.60:0/4196440589 wait complete. 2023-07-13T22:28:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:28:57.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:28:57.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.984+0000 7f94337ff700 1 Processor -- start 2023-07-13T22:28:57.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.985+0000 7f94337ff700 1 -- start start 2023-07-13T22:28:57.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.986+0000 7f94337ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c15c790 0x7f942c0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:57.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.986+0000 7f94337ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f942c0ad9a0 0x7f942c0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:57.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.986+0000 7f94337ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c0cf6d0 con 0x7f942c0ac490 2023-07-13T22:28:57.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.986+0000 7f94337ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0ae340 con 0x7f942c0ad9a0 2023-07-13T22:28:57.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.986+0000 7f94337ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0ac900 con 0x7f942c15c790 2023-07-13T22:28:57.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.986+0000 7f9430d9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f942c0ac490 0x7f942c0c9e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50070/0 (socket says 172.21.15.60:50070) 2023-07-13T22:28:57.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.986+0000 7f9430d9a700 1 -- 172.21.15.60:0/4014195172 learned_addr learned my addr 172.21.15.60:0/4014195172 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:57.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.986+0000 7f9431d9c700 1 --2- 172.21.15.60:0/4014195172 >> v2:172.21.15.60:3300/0 conn(0x7f942c0ad9a0 0x7f942c0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:57.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.986+0000 7f943159b700 1 --2- 172.21.15.60:0/4014195172 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c15c790 0x7f942c0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:57.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9423fff700 1 -- 172.21.15.60:0/4014195172 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935318602 0 0) 0x7f942c0cf6d0 con 0x7f942c0ac490 2023-07-13T22:28:57.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9423fff700 1 -- 172.21.15.60:0/4014195172 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94180035b0 con 0x7f942c0ac490 2023-07-13T22:28:57.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9431d9c700 1 -- 172.21.15.60:0/4014195172 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c15c790 msgr2=0x7f942c0ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:57.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9431d9c700 1 --2- 172.21.15.60:0/4014195172 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c15c790 0x7f942c0ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9431d9c700 1 -- 172.21.15.60:0/4014195172 >> v1:172.21.15.60:6789/0 conn(0x7f942c0ac490 legacy=0x7f942c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9431d9c700 1 -- 172.21.15.60:0/4014195172 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0acbe0 con 0x7f942c0ad9a0 2023-07-13T22:28:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9431d9c700 1 --2- 172.21.15.60:0/4014195172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c0ad9a0 0x7f942c0ade00 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9424009d90 tx=0x7f942400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe71db11d00dc99 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9423fff700 1 -- 172.21.15.60:0/4014195172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942400e070 con 0x7f942c0ad9a0 2023-07-13T22:28:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9423fff700 1 -- 172.21.15.60:0/4014195172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942400bb20 con 0x7f942c0ad9a0 2023-07-13T22:28:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.987+0000 7f9423fff700 1 -- 172.21.15.60:0/4014195172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424012430 con 0x7f942c0ad9a0 2023-07-13T22:28:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.988+0000 7f94337ff700 1 -- 172.21.15.60:0/4014195172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c0ad9a0 msgr2=0x7f942c0ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.988+0000 7f94337ff700 1 --2- 172.21.15.60:0/4014195172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c0ad9a0 0x7f942c0ade00 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9424009d90 tx=0x7f942400b330 comp rx=0 tx=0).stop 2023-07-13T22:28:57.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.988+0000 7f94337ff700 1 -- 172.21.15.60:0/4014195172 shutdown_connections 2023-07-13T22:28:57.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.988+0000 7f94337ff700 1 --2- 172.21.15.60:0/4014195172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c0ad9a0 0x7f942c0ade00 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.988+0000 7f94337ff700 1 --2- 172.21.15.60:0/4014195172 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c15c790 0x7f942c0ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.988+0000 7f94337ff700 1 -- 172.21.15.60:0/4014195172 >> 172.21.15.60:0/4014195172 conn(0x7f942c06b900 msgr2=0x7f942c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:57.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.988+0000 7f94337ff700 1 -- 172.21.15.60:0/4014195172 shutdown_connections 2023-07-13T22:28:57.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.988+0000 7f94337ff700 1 -- 172.21.15.60:0/4014195172 wait complete. 2023-07-13T22:28:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.990+0000 7f94337ff700 1 Processor -- start 2023-07-13T22:28:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.990+0000 7f94337ff700 1 -- start start 2023-07-13T22:28:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f94337ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f942c0ac490 0x7f942c15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:57.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f94337ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c0ad9a0 0x7f942c15a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:57.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f94337ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c15c790 0x7f942c09acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:57.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f94337ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c09d460 con 0x7f942c15c790 2023-07-13T22:28:57.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f94337ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15efc0 con 0x7f942c0ac490 2023-07-13T22:28:57.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f94337ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c09cea0 con 0x7f942c0ad9a0 2023-07-13T22:28:57.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f9431d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c15c790 0x7f942c09acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:57.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f9431d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c15c790 0x7f942c09acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57632/0 (socket says 172.21.15.60:57632) 2023-07-13T22:28:57.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f9431d9c700 1 -- 172.21.15.60:0/2583684109 learned_addr learned my addr 172.21.15.60:0/2583684109 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:57.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f9430d9a700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c0ad9a0 0x7f942c15a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:57.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f943159b700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f942c0ac490 0x7f942c15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:57.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.991+0000 7f9431d9c700 1 -- 172.21.15.60:0/2583684109 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c0ad9a0 msgr2=0x7f942c15a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:57.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f9431d9c700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c0ad9a0 0x7f942c15a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f9431d9c700 1 -- 172.21.15.60:0/2583684109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f942c0ac490 msgr2=0x7f942c15a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:57.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f9431d9c700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f942c0ac490 0x7f942c15a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f9431d9c700 1 -- 172.21.15.60:0/2583684109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424009a70 con 0x7f942c15c790 2023-07-13T22:28:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f9431d9c700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c15c790 0x7f942c09acf0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9424014040 tx=0x7f9424004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f9421ffb700 1 -- 172.21.15.60:0/2583684109 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240129f0 con 0x7f942c15c790 2023-07-13T22:28:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f9421ffb700 1 -- 172.21.15.60:0/2583684109 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9424020070 con 0x7f942c15c790 2023-07-13T22:28:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f9421ffb700 1 -- 172.21.15.60:0/2583684109 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424012db0 con 0x7f942c15c790 2023-07-13T22:28:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c09d180 con 0x7f942c15c790 2023-07-13T22:28:57.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.992+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c15a5b0 con 0x7f942c15c790 2023-07-13T22:28:57.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.993+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c15a740 con 0x7f942c15c790 2023-07-13T22:28:57.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.994+0000 7f9421ffb700 1 -- 172.21.15.60:0/2583684109 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9424004040 con 0x7f942c15c790 2023-07-13T22:28:57.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.995+0000 7f9421ffb700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f941006ace0 0x7f941006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:57.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.995+0000 7f9421ffb700 1 -- 172.21.15.60:0/2583684109 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f9424022070 con 0x7f942c15c790 2023-07-13T22:28:57.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.995+0000 7f943159b700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f941006ace0 0x7f941006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:57.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:57.996+0000 7f943159b700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f941006ace0 0x7f941006d190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f942c0ae990 tx=0x7f9428008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:57.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.000+0000 7f9421ffb700 1 -- 172.21.15.60:0/2583684109 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f942c15a740 con 0x7f942c15c790 2023-07-13T22:28:58.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.253+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f942c15ee20 con 0x7f942c15c790 2023-07-13T22:28:58.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.733+0000 7f9421ffb700 1 -- 172.21.15.60:0/2583684109 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7f942c15ee20 con 0x7f942c15c790 2023-07-13T22:28:58.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.737+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f941006ace0 msgr2=0x7f941006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:58.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.737+0000 7f94337ff700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f941006ace0 0x7f941006d190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f942c0ae990 tx=0x7f9428008040 comp rx=0 tx=0).stop 2023-07-13T22:28:58.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.737+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c15c790 msgr2=0x7f942c09acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:58.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.737+0000 7f94337ff700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c15c790 0x7f942c09acf0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9424014040 tx=0x7f9424004ad0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.738+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 shutdown_connections 2023-07-13T22:28:58.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.738+0000 7f94337ff700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f941006ace0 0x7f941006d190 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.738+0000 7f94337ff700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f942c15c790 0x7f942c09acf0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.738+0000 7f94337ff700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f942c0ac490 0x7f942c15a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.738+0000 7f94337ff700 1 --2- 172.21.15.60:0/2583684109 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f942c0ad9a0 0x7f942c15a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.738+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 >> 172.21.15.60:0/2583684109 conn(0x7f942c06b900 msgr2=0x7f942c06e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.740+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 shutdown_connections 2023-07-13T22:28:58.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.740+0000 7f94337ff700 1 -- 172.21.15.60:0/2583684109 wait complete. 2023-07-13T22:28:58.722 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 15 and data pool 16 2023-07-13T22:28:58.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:58.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.967+0000 7f2371fff700 1 Processor -- start 2023-07-13T22:28:58.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.967+0000 7f2371fff700 1 -- start start 2023-07-13T22:28:58.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.969+0000 7f2371fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f236c0b8400 0x7f236c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:58.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.969+0000 7f2371fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f236c0d3750 0x7f236c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:58.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.969+0000 7f2371fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c0d0ac0 con 0x7f236c0cfad0 2023-07-13T22:28:58.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.969+0000 7f2371fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0cff50 con 0x7f236c0d3750 2023-07-13T22:28:58.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.969+0000 7f2371fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0b87d0 con 0x7f236c0b8400 2023-07-13T22:28:58.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.969+0000 7f236affd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f236c0d3750 0x7f236c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:58.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.969+0000 7f236bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f236c0cfad0 0x7f236c15fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50080/0 (socket says 172.21.15.60:50080) 2023-07-13T22:28:58.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.969+0000 7f236bfff700 1 -- 172.21.15.60:0/1309719180 learned_addr learned my addr 172.21.15.60:0/1309719180 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:58.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236affd700 1 -- 172.21.15.60:0/1309719180 >> v2:172.21.15.105:3300/0 conn(0x7f236c0b8400 msgr2=0x7f236c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:58.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236b7fe700 1 --2- 172.21.15.60:0/1309719180 >> v2:172.21.15.105:3300/0 conn(0x7f236c0b8400 0x7f236c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:58.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236affd700 1 --2- 172.21.15.60:0/1309719180 >> v2:172.21.15.105:3300/0 conn(0x7f236c0b8400 0x7f236c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236affd700 1 -- 172.21.15.60:0/1309719180 >> v1:172.21.15.60:6789/0 conn(0x7f236c0cfad0 legacy=0x7f236c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:58.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236affd700 1 -- 172.21.15.60:0/1309719180 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0b4eb0 con 0x7f236c0d3750 2023-07-13T22:28:58.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236b7fe700 1 --2- 172.21.15.60:0/1309719180 >> v2:172.21.15.105:3300/0 conn(0x7f236c0b8400 0x7f236c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:28:58.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236a7fc700 1 -- 172.21.15.60:0/1309719180 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271141000 0 0) 0x7f236c0d0ac0 con 0x7f236c0cfad0 2023-07-13T22:28:58.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236affd700 1 --2- 172.21.15.60:0/1309719180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0d3750 0x7f236c0cf590 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f236c0cf560 tx=0x7f2360009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8738529ae8cee0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:58.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236a7fc700 1 -- 172.21.15.60:0/1309719180 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360011830 con 0x7f236c0d3750 2023-07-13T22:28:58.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236a7fc700 1 -- 172.21.15.60:0/1309719180 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360011990 con 0x7f236c0d3750 2023-07-13T22:28:58.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.970+0000 7f236a7fc700 1 -- 172.21.15.60:0/1309719180 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360011d20 con 0x7f236c0d3750 2023-07-13T22:28:58.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.971+0000 7f2371fff700 1 -- 172.21.15.60:0/1309719180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0d3750 msgr2=0x7f236c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:58.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.971+0000 7f2371fff700 1 --2- 172.21.15.60:0/1309719180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0d3750 0x7f236c0cf590 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f236c0cf560 tx=0x7f2360009b20 comp rx=0 tx=0).stop 2023-07-13T22:28:58.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.971+0000 7f2371fff700 1 -- 172.21.15.60:0/1309719180 shutdown_connections 2023-07-13T22:28:58.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.971+0000 7f2371fff700 1 --2- 172.21.15.60:0/1309719180 >> v2:172.21.15.105:3300/0 conn(0x7f236c0b8400 0x7f236c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.971+0000 7f2371fff700 1 --2- 172.21.15.60:0/1309719180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0d3750 0x7f236c0cf590 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.971+0000 7f2371fff700 1 -- 172.21.15.60:0/1309719180 >> 172.21.15.60:0/1309719180 conn(0x7f236c06b390 msgr2=0x7f236c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:58.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.971+0000 7f2371fff700 1 -- 172.21.15.60:0/1309719180 shutdown_connections 2023-07-13T22:28:58.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.971+0000 7f2371fff700 1 -- 172.21.15.60:0/1309719180 wait complete. 2023-07-13T22:28:58.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.973+0000 7f2371fff700 1 Processor -- start 2023-07-13T22:28:58.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.973+0000 7f2371fff700 1 -- start start 2023-07-13T22:28:58.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f2371fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8400 0x7f236c149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:58.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f2371fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfad0 0x7f236c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f2371fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0d3750 0x7f236c0c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f2371fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15e9c0 con 0x7f236c0b8400 2023-07-13T22:28:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f2371fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15dd40 con 0x7f236c0cfad0 2023-07-13T22:28:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f2371fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15e020 con 0x7f236c0d3750 2023-07-13T22:28:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f236b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8400 0x7f236c149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:58.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f236b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8400 0x7f236c149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57648/0 (socket says 172.21.15.60:57648) 2023-07-13T22:28:58.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f236b7fe700 1 -- 172.21.15.60:0/881413640 learned_addr learned my addr 172.21.15.60:0/881413640 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:58.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f236bfff700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0d3750 0x7f236c0c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:58.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.974+0000 7f236affd700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfad0 0x7f236c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:58.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f236b7fe700 1 -- 172.21.15.60:0/881413640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0d3750 msgr2=0x7f236c0c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:58.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f236b7fe700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0d3750 0x7f236c0c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f236b7fe700 1 -- 172.21.15.60:0/881413640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfad0 msgr2=0x7f236c154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:58.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f236b7fe700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfad0 0x7f236c154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:58.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f236b7fe700 1 -- 172.21.15.60:0/881413640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23600094a0 con 0x7f236c0b8400 2023-07-13T22:28:58.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f236b7fe700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8400 0x7f236c149db0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c7f0 tx=0x7f235c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:58.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f2353fff700 1 -- 172.21.15.60:0/881413640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c011800 con 0x7f236c0b8400 2023-07-13T22:28:58.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f2353fff700 1 -- 172.21.15.60:0/881413640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c011960 con 0x7f236c0b8400 2023-07-13T22:28:58.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f2353fff700 1 -- 172.21.15.60:0/881413640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c011d10 con 0x7f236c0b8400 2023-07-13T22:28:58.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c15e300 con 0x7f236c0b8400 2023-07-13T22:28:58.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.975+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c1540b0 con 0x7f236c0b8400 2023-07-13T22:28:58.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.976+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c073590 con 0x7f236c0b8400 2023-07-13T22:28:58.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.977+0000 7f2353fff700 1 -- 172.21.15.60:0/881413640 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f235c018360 con 0x7f236c0b8400 2023-07-13T22:28:58.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.978+0000 7f2353fff700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235806ace0 0x7f235806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:58.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.978+0000 7f2353fff700 1 -- 172.21.15.60:0/881413640 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f235c0895b0 con 0x7f236c0b8400 2023-07-13T22:28:58.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.978+0000 7f236affd700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235806ace0 0x7f235806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:58.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.979+0000 7f236affd700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235806ace0 0x7f235806d190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f236c0d3b60 tx=0x7f236001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:58.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:58.983+0000 7f2353fff700 1 -- 172.21.15.60:0/881413640 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f235c054530 con 0x7f236c0b8400 2023-07-13T22:28:59.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.191+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f236c0b8a00 con 0x7f236c0b8400 2023-07-13T22:28:59.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.193+0000 7f2353fff700 1 -- 172.21.15.60:0/881413640 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+18201 (secure 0 0 0) 0x7f235c057b50 con 0x7f236c0b8400 2023-07-13T22:28:59.172 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:59.172 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:58.254212+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:56.533994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:57.226035+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:59.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235806ace0 msgr2=0x7f235806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235806ace0 0x7f235806d190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f236c0d3b60 tx=0x7f236001c040 comp rx=0 tx=0).stop 2023-07-13T22:28:59.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8400 msgr2=0x7f236c149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8400 0x7f236c149db0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c7f0 tx=0x7f235c00cbb0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 shutdown_connections 2023-07-13T22:28:59.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235806ace0 0x7f235806d190 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8400 0x7f236c149db0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0d3750 0x7f236c0c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 --2- 172.21.15.60:0/881413640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfad0 0x7f236c154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 >> 172.21.15.60:0/881413640 conn(0x7f236c06b390 msgr2=0x7f236c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 shutdown_connections 2023-07-13T22:28:59.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.195+0000 7f2371fff700 1 -- 172.21.15.60:0/881413640 wait complete. 2023-07-13T22:28:59.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:28:59.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.417+0000 7fb4953ff700 1 Processor -- start 2023-07-13T22:28:59.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.417+0000 7fb4953ff700 1 -- start start 2023-07-13T22:28:59.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.418+0000 7fb4953ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb4900d3750 0x7fb4900cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.418+0000 7fb4953ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900cd950 con 0x7fb4900b8400 2023-07-13T22:28:59.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.418+0000 7fb4953ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900d0ac0 con 0x7fb4900cfad0 2023-07-13T22:28:59.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.418+0000 7fb4953ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4900cff50 con 0x7fb4900d3750 2023-07-13T22:28:59.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.418+0000 7fb48effd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb4900b8400 0x7fb4900cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50082/0 (socket says 172.21.15.60:50082) 2023-07-13T22:28:59.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.418+0000 7fb48effd700 1 -- 172.21.15.60:0/2247908006 learned_addr learned my addr 172.21.15.60:0/2247908006 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:28:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.419+0000 7fb487fff700 1 --2- 172.21.15.60:0/2247908006 >> v2:172.21.15.105:3300/0 conn(0x7fb4900d3750 0x7fb4900cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.419+0000 7fb48e7fc700 1 -- 172.21.15.60:0/2247908006 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184253378 0 0) 0x7fb480001110 con 0x7fb4900b8400 2023-07-13T22:28:59.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.419+0000 7fb48e7fc700 1 -- 172.21.15.60:0/2247908006 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb47c0035b0 con 0x7fb4900b8400 2023-07-13T22:28:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.419+0000 7fb48e7fc700 1 -- 172.21.15.60:0/2247908006 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777226944 0 0) 0x7fb4900d0ac0 con 0x7fb4900cfad0 2023-07-13T22:28:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.419+0000 7fb48e7fc700 1 -- 172.21.15.60:0/2247908006 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb480001110 con 0x7fb4900cfad0 2023-07-13T22:28:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.419+0000 7fb487fff700 1 -- 172.21.15.60:0/2247908006 >> v1:172.21.15.105:6789/0 conn(0x7fb4900cfad0 legacy=0x7fb4900d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.419+0000 7fb487fff700 1 -- 172.21.15.60:0/2247908006 >> v1:172.21.15.60:6789/0 conn(0x7fb4900b8400 legacy=0x7fb4900cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.419+0000 7fb487fff700 1 -- 172.21.15.60:0/2247908006 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900b87d0 con 0x7fb4900d3750 2023-07-13T22:28:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.419+0000 7fb487fff700 1 --2- 172.21.15.60:0/2247908006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4900d3750 0x7fb4900cf590 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb478009840 tx=0x7fb478009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86898c34c683ba38 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:59.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb48e7fc700 1 -- 172.21.15.60:0/2247908006 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478011830 con 0x7fb4900d3750 2023-07-13T22:28:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb48e7fc700 1 -- 172.21.15.60:0/2247908006 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb478011990 con 0x7fb4900d3750 2023-07-13T22:28:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb48e7fc700 1 -- 172.21.15.60:0/2247908006 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478011d00 con 0x7fb4900d3750 2023-07-13T22:28:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb4953ff700 1 -- 172.21.15.60:0/2247908006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4900d3750 msgr2=0x7fb4900cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb4953ff700 1 --2- 172.21.15.60:0/2247908006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4900d3750 0x7fb4900cf590 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb478009840 tx=0x7fb478009b50 comp rx=0 tx=0).stop 2023-07-13T22:28:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb4953ff700 1 -- 172.21.15.60:0/2247908006 shutdown_connections 2023-07-13T22:28:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb4953ff700 1 --2- 172.21.15.60:0/2247908006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4900d3750 0x7fb4900cf590 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb4953ff700 1 -- 172.21.15.60:0/2247908006 >> 172.21.15.60:0/2247908006 conn(0x7fb49006b390 msgr2=0x7fb49006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:59.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb4953ff700 1 -- 172.21.15.60:0/2247908006 shutdown_connections 2023-07-13T22:28:59.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.420+0000 7fb4953ff700 1 -- 172.21.15.60:0/2247908006 wait complete. 2023-07-13T22:28:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.422+0000 7fb4953ff700 1 Processor -- start 2023-07-13T22:28:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.422+0000 7fb4953ff700 1 -- start start 2023-07-13T22:28:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.423+0000 7fb4953ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4900b8400 0x7fb4901f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.423+0000 7fb4953ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4900cfad0 0x7fb490157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.423+0000 7fb4953ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4900d3750 0x7fb4900ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.423+0000 7fb4953ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4901ef6f0 con 0x7fb4900cfad0 2023-07-13T22:28:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.423+0000 7fb4953ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900caba0 con 0x7fb4900d3750 2023-07-13T22:28:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.423+0000 7fb4953ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4901f23a0 con 0x7fb4900b8400 2023-07-13T22:28:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.423+0000 7fb487fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4900cfad0 0x7fb490157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.423+0000 7fb487fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4900cfad0 0x7fb490157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57664/0 (socket says 172.21.15.60:57664) 2023-07-13T22:28:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.423+0000 7fb487fff700 1 -- 172.21.15.60:0/3874801075 learned_addr learned my addr 172.21.15.60:0/3874801075 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb48f7fe700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4900d3750 0x7fb4900ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb48effd700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4900b8400 0x7fb4901f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb487fff700 1 -- 172.21.15.60:0/3874801075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4900b8400 msgr2=0x7fb4901f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb487fff700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4900b8400 0x7fb4901f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb487fff700 1 -- 172.21.15.60:0/3874801075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4900d3750 msgr2=0x7fb4900ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb487fff700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4900d3750 0x7fb4900ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb487fff700 1 -- 172.21.15.60:0/3874801075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780094a0 con 0x7fb4900cfad0 2023-07-13T22:28:59.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb487fff700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4900cfad0 0x7fb490157b40 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fb478009fd0 tx=0x7fb478004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:59.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb4877fe700 1 -- 172.21.15.60:0/3874801075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780045c0 con 0x7fb4900cfad0 2023-07-13T22:28:59.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb4877fe700 1 -- 172.21.15.60:0/3874801075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47801e070 con 0x7fb4900cfad0 2023-07-13T22:28:59.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4900cadd0 con 0x7fb4900cfad0 2023-07-13T22:28:59.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb4877fe700 1 -- 172.21.15.60:0/3874801075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478003900 con 0x7fb4900cfad0 2023-07-13T22:28:59.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.424+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4901f2030 con 0x7fb4900cfad0 2023-07-13T22:28:59.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.425+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4901f2510 con 0x7fb4900cfad0 2023-07-13T22:28:59.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.426+0000 7fb4877fe700 1 -- 172.21.15.60:0/3874801075 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb478003a60 con 0x7fb4900cfad0 2023-07-13T22:28:59.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.427+0000 7fb4877fe700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb47006ace0 0x7fb47006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.427+0000 7fb4877fe700 1 -- 172.21.15.60:0/3874801075 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb47808a410 con 0x7fb4900cfad0 2023-07-13T22:28:59.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.427+0000 7fb48effd700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb47006ace0 0x7fb47006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.428+0000 7fb48effd700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb47006ace0 0x7fb47006d190 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb480000f60 tx=0x7fb480009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:59.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.432+0000 7fb4877fe700 1 -- 172.21.15.60:0/3874801075 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb4780554b0 con 0x7fb4900cfad0 2023-07-13T22:28:59.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.641+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb490160120 con 0x7fb4900cfad0 2023-07-13T22:28:59.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.642+0000 7fb4877fe700 1 -- 172.21.15.60:0/3874801075 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+18190 (secure 0 0 0) 0x7fb478058ad0 con 0x7fb4900cfad0 2023-07-13T22:28:59.623 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:28:59.623 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":100,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:59.528113+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:56.533994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:57.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:28:59.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb47006ace0 msgr2=0x7fb47006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb47006ace0 0x7fb47006d190 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb480000f60 tx=0x7fb480009040 comp rx=0 tx=0).stop 2023-07-13T22:28:59.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4900cfad0 msgr2=0x7fb490157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4900cfad0 0x7fb490157b40 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fb478009fd0 tx=0x7fb478004750 comp rx=0 tx=0).stop 2023-07-13T22:28:59.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 shutdown_connections 2023-07-13T22:28:59.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb47006ace0 0x7fb47006d190 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4900d3750 0x7fb4900ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4900b8400 0x7fb4901f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 --2- 172.21.15.60:0/3874801075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4900cfad0 0x7fb490157b40 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 >> 172.21.15.60:0/3874801075 conn(0x7fb49006b390 msgr2=0x7fb490160640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:59.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 shutdown_connections 2023-07-13T22:28:59.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.645+0000 7fb4953ff700 1 -- 172.21.15.60:0/3874801075 wait complete. 2023-07-13T22:28:59.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:28:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.873+0000 7f5f933ff700 1 Processor -- start 2023-07-13T22:28:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.873+0000 7f5f933ff700 1 -- start start 2023-07-13T22:28:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f933ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f933ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f8c0caf80 0x7f5f8c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f933ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f8c06c8f0 0x7f5f8c15fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f933ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c06b9f0 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f933ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c06c750 con 0x7f5f8c0caf80 2023-07-13T22:28:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f933ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c06bf40 con 0x7f5f8c15c460 2023-07-13T22:28:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f9199c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f8c06c8f0 0x7f5f8c15fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f9199c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f8c06c8f0 0x7f5f8c15fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57670/0 (socket says 172.21.15.60:57670) 2023-07-13T22:28:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f9199c700 1 -- 172.21.15.60:0/1850758816 learned_addr learned my addr 172.21.15.60:0/1850758816 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f9119b700 1 --2- 172.21.15.60:0/1850758816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.874+0000 7f5f9099a700 1 --2- 172.21.15.60:0/1850758816 >> v2:172.21.15.105:3300/0 conn(0x7f5f8c0caf80 0x7f5f8c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f9199c700 1 -- 172.21.15.60:0/1850758816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 msgr2=0x7f5f8c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f9199c700 1 --2- 172.21.15.60:0/1850758816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f9199c700 1 -- 172.21.15.60:0/1850758816 >> v2:172.21.15.105:3300/0 conn(0x7f5f8c0caf80 msgr2=0x7f5f8c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:28:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f9199c700 1 --2- 172.21.15.60:0/1850758816 >> v2:172.21.15.105:3300/0 conn(0x7f5f8c0caf80 0x7f5f8c0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f9199c700 1 -- 172.21.15.60:0/1850758816 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c0cb3f0 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f9099a700 1 --2- 172.21.15.60:0/1850758816 >> v2:172.21.15.105:3300/0 conn(0x7f5f8c0caf80 0x7f5f8c0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f9119b700 1 --2- 172.21.15.60:0/1850758816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f9199c700 1 --2- 172.21.15.60:0/1850758816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 0x7f5f8c15fdd0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0b34f0 tx=0x7f5f8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab51c5785ddb416 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f83fff700 1 -- 172.21.15.60:0/1850758816 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400e070 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f83fff700 1 -- 172.21.15.60:0/1850758816 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8400b9c0 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.875+0000 7f5f83fff700 1 -- 172.21.15.60:0/1850758816 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400bd70 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.876+0000 7f5f933ff700 1 -- 172.21.15.60:0/1850758816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 msgr2=0x7f5f8c15fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.876+0000 7f5f933ff700 1 --2- 172.21.15.60:0/1850758816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 0x7f5f8c15fdd0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0b34f0 tx=0x7f5f8400b330 comp rx=0 tx=0).stop 2023-07-13T22:28:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.876+0000 7f5f933ff700 1 -- 172.21.15.60:0/1850758816 shutdown_connections 2023-07-13T22:28:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.876+0000 7f5f933ff700 1 --2- 172.21.15.60:0/1850758816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 0x7f5f8c15fdd0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.876+0000 7f5f933ff700 1 --2- 172.21.15.60:0/1850758816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.876+0000 7f5f933ff700 1 --2- 172.21.15.60:0/1850758816 >> v2:172.21.15.105:3300/0 conn(0x7f5f8c0caf80 0x7f5f8c0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.876+0000 7f5f933ff700 1 -- 172.21.15.60:0/1850758816 >> 172.21.15.60:0/1850758816 conn(0x7f5f8c06e580 msgr2=0x7f5f8c06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:28:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.876+0000 7f5f933ff700 1 -- 172.21.15.60:0/1850758816 shutdown_connections 2023-07-13T22:28:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.876+0000 7f5f933ff700 1 -- 172.21.15.60:0/1850758816 wait complete. 2023-07-13T22:28:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.878+0000 7f5f933ff700 1 Processor -- start 2023-07-13T22:28:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.878+0000 7f5f933ff700 1 -- start start 2023-07-13T22:28:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f933ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 0x7f5f8c155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f933ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c0caf80 0x7f5f8c15e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f933ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f933ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0b60a0 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f933ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0b5790 con 0x7f5f8c0caf80 2023-07-13T22:28:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f933ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c15ed60 con 0x7f5f8c15c460 2023-07-13T22:28:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f9119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 0x7f5f8c155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f9119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 0x7f5f8c155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57684/0 (socket says 172.21.15.60:57684) 2023-07-13T22:28:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f9119b700 1 -- 172.21.15.60:0/4059825796 learned_addr learned my addr 172.21.15.60:0/4059825796 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:28:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f9199c700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.879+0000 7f5f9099a700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c0caf80 0x7f5f8c15e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f9119b700 1 -- 172.21.15.60:0/4059825796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 msgr2=0x7f5f8c0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f9119b700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c0b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f9119b700 1 -- 172.21.15.60:0/4059825796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c0caf80 msgr2=0x7f5f8c15e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:28:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f9119b700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c0caf80 0x7f5f8c15e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:28:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f9119b700 1 -- 172.21.15.60:0/4059825796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f84009b30 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f9199c700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:28:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f9119b700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 0x7f5f8c155530 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800e810 tx=0x7f5f8800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f81ffb700 1 -- 172.21.15.60:0/4059825796 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88013610 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f81ffb700 1 -- 172.21.15.60:0/4059825796 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f88013770 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c0b59e0 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f81ffb700 1 -- 172.21.15.60:0/4059825796 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88013b20 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.880+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c155a70 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.881+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c0b61e0 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.882+0000 7f5f81ffb700 1 -- 172.21.15.60:0/4059825796 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f88013c80 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.884+0000 7f5f81ffb700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f7006ace0 0x7f5f7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:28:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.884+0000 7f5f81ffb700 1 -- 172.21.15.60:0/4059825796 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f88089090 con 0x7f5f8c06c8f0 2023-07-13T22:28:59.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.884+0000 7f5f9099a700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f7006ace0 0x7f5f7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:28:59.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.884+0000 7f5f9099a700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f7006ace0 0x7f5f7006d190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0b2870 tx=0x7f5f7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:28:59.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:28:59.888+0000 7f5f81ffb700 1 -- 172.21.15.60:0/4059825796 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5f8c0b61e0 con 0x7f5f8c06c8f0 2023-07-13T22:29:00.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.140+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5f8c0b61e0 con 0x7f5f8c06c8f0 2023-07-13T22:29:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.747+0000 7f5f81ffb700 1 -- 172.21.15.60:0/4059825796 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5f8c0b61e0 con 0x7f5f8c06c8f0 2023-07-13T22:29:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.750+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f7006ace0 msgr2=0x7f5f7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.750+0000 7f5f933ff700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f7006ace0 0x7f5f7006d190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0b2870 tx=0x7f5f7c008040 comp rx=0 tx=0).stop 2023-07-13T22:29:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.750+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 msgr2=0x7f5f8c155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.750+0000 7f5f933ff700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 0x7f5f8c155530 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800e810 tx=0x7f5f8800ebd0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.751+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 shutdown_connections 2023-07-13T22:29:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.751+0000 7f5f933ff700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f7006ace0 0x7f5f7006d190 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.751+0000 7f5f933ff700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f8c06c8f0 0x7f5f8c155530 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.751+0000 7f5f933ff700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f8c15c460 0x7f5f8c0b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.751+0000 7f5f933ff700 1 --2- 172.21.15.60:0/4059825796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f8c0caf80 0x7f5f8c15e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.751+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 >> 172.21.15.60:0/4059825796 conn(0x7f5f8c06e580 msgr2=0x7f5f8c157150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.751+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 shutdown_connections 2023-07-13T22:29:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.751+0000 7f5f933ff700 1 -- 172.21.15.60:0/4059825796 wait complete. 2023-07-13T22:29:00.751 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:29:00.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:29:00.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.982+0000 7f3b66fff700 1 Processor -- start 2023-07-13T22:29:00.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.983+0000 7f3b66fff700 1 -- start start 2023-07-13T22:29:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b66fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b600d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b66fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3b600b1ac0 0x7f3b600d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:00.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b66fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b600d1870 con 0x7f3b600b8200 2023-07-13T22:29:00.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b66fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b600cf370 con 0x7f3b600b1ac0 2023-07-13T22:29:00.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b66fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600d12b0 con 0x7f3b600cfd50 2023-07-13T22:29:00.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b5ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b600d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:00.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b6559c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3b600b1ac0 0x7f3b600d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:00.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b5ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b600d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:46576/0 (socket says 172.21.15.60:46576) 2023-07-13T22:29:00.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b5ffff700 1 -- 172.21.15.60:0/925914948 learned_addr learned my addr 172.21.15.60:0/925914948 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:00.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.984+0000 7f3b6559c700 1 --2- 172.21.15.60:0/925914948 >> v2:172.21.15.105:3300/0 conn(0x7f3b600b1ac0 0x7f3b600d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:59142/0 (socket says 172.21.15.60:59142) 2023-07-13T22:29:00.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.985+0000 7f3b6559c700 1 -- 172.21.15.60:0/925914948 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 msgr2=0x7f3b600d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:00.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.985+0000 7f3b6559c700 1 --2- 172.21.15.60:0/925914948 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b600d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.985+0000 7f3b6559c700 1 -- 172.21.15.60:0/925914948 >> v1:172.21.15.105:6789/0 conn(0x7f3b600b8200 legacy=0x7f3b600cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:00.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.985+0000 7f3b6559c700 1 -- 172.21.15.60:0/925914948 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b600d1590 con 0x7f3b600b1ac0 2023-07-13T22:29:00.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.985+0000 7f3b5ffff700 1 --2- 172.21.15.60:0/925914948 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b600d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:00.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.985+0000 7f3b6559c700 1 --2- 172.21.15.60:0/925914948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b600b1ac0 0x7f3b600d2900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009dc0 tx=0x7f3b5800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94cc0da5edfa4828 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:00.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.985+0000 7f3b5f7fe700 1 -- 172.21.15.60:0/925914948 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800e070 con 0x7f3b600b1ac0 2023-07-13T22:29:00.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.985+0000 7f3b5f7fe700 1 -- 172.21.15.60:0/925914948 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5800bae0 con 0x7f3b600b1ac0 2023-07-13T22:29:00.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.986+0000 7f3b5f7fe700 1 -- 172.21.15.60:0/925914948 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800be90 con 0x7f3b600b1ac0 2023-07-13T22:29:00.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.986+0000 7f3b66fff700 1 -- 172.21.15.60:0/925914948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b600b1ac0 msgr2=0x7f3b600d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.986+0000 7f3b66fff700 1 --2- 172.21.15.60:0/925914948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b600b1ac0 0x7f3b600d2900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009dc0 tx=0x7f3b5800b330 comp rx=0 tx=0).stop 2023-07-13T22:29:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.986+0000 7f3b66fff700 1 -- 172.21.15.60:0/925914948 shutdown_connections 2023-07-13T22:29:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.986+0000 7f3b66fff700 1 --2- 172.21.15.60:0/925914948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b600b1ac0 0x7f3b600d2900 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.986+0000 7f3b66fff700 1 --2- 172.21.15.60:0/925914948 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b600d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.986+0000 7f3b66fff700 1 -- 172.21.15.60:0/925914948 >> 172.21.15.60:0/925914948 conn(0x7f3b6006b2f0 msgr2=0x7f3b6006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.986+0000 7f3b66fff700 1 -- 172.21.15.60:0/925914948 shutdown_connections 2023-07-13T22:29:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.986+0000 7f3b66fff700 1 -- 172.21.15.60:0/925914948 wait complete. 2023-07-13T22:29:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.988+0000 7f3b66fff700 1 Processor -- start 2023-07-13T22:29:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.988+0000 7f3b66fff700 1 -- start start 2023-07-13T22:29:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b66fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b600b1ac0 0x7f3b600cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b66fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600b8200 0x7f3b601f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:00.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b66fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b601e8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:00.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b66fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600ca9b0 con 0x7f3b600b8200 2023-07-13T22:29:00.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b66fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b601f2af0 con 0x7f3b600b1ac0 2023-07-13T22:29:00.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b66fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600c9da0 con 0x7f3b600cfd50 2023-07-13T22:29:00.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b5ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600b8200 0x7f3b601f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:00.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b5ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600b8200 0x7f3b601f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57696/0 (socket says 172.21.15.60:57696) 2023-07-13T22:29:00.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b5ffff700 1 -- 172.21.15.60:0/2356920840 learned_addr learned my addr 172.21.15.60:0/2356920840 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:00.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b6559c700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b601e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.989+0000 7f3b64d9b700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b600b1ac0 0x7f3b600cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b5ffff700 1 -- 172.21.15.60:0/2356920840 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 msgr2=0x7f3b601e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b5ffff700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b601e8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b5ffff700 1 -- 172.21.15.60:0/2356920840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b600b1ac0 msgr2=0x7f3b600cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b5ffff700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b600b1ac0 0x7f3b600cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b5ffff700 1 -- 172.21.15.60:0/2356920840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c009550 con 0x7f3b600b8200 2023-07-13T22:29:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b5ffff700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600b8200 0x7f3b601f25b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00e980 tx=0x7f3b4c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b5d7fa700 1 -- 172.21.15.60:0/2356920840 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00c690 con 0x7f3b600b8200 2023-07-13T22:29:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b5d7fa700 1 -- 172.21.15.60:0/2356920840 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c00c7f0 con 0x7f3b600b8200 2023-07-13T22:29:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b5d7fa700 1 -- 172.21.15.60:0/2356920840 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00cb60 con 0x7f3b600b8200 2023-07-13T22:29:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b58009a70 con 0x7f3b600b8200 2023-07-13T22:29:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.990+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b600ca360 con 0x7f3b600b8200 2023-07-13T22:29:00.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.991+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6004e210 con 0x7f3b600b8200 2023-07-13T22:29:00.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.992+0000 7f3b5d7fa700 1 -- 172.21.15.60:0/2356920840 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b4c00ccc0 con 0x7f3b600b8200 2023-07-13T22:29:00.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.994+0000 7f3b5d7fa700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b4406ace0 0x7f3b4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:00.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.994+0000 7f3b5d7fa700 1 -- 172.21.15.60:0/2356920840 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3b4c0897c0 con 0x7f3b600b8200 2023-07-13T22:29:00.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.994+0000 7f3b64d9b700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b4406ace0 0x7f3b4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:00.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.994+0000 7f3b64d9b700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b4406ace0 0x7f3b4406d190 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3b600b1a50 tx=0x7f3b54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:00.997+0000 7f3b5d7fa700 1 -- 172.21.15.60:0/2356920840 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3b4c054860 con 0x7f3b600b8200 2023-07-13T22:29:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.253+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3b600b4f70 con 0x7f3b600b8200 2023-07-13T22:29:01.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.753+0000 7f3b5d7fa700 1 -- 172.21.15.60:0/2356920840 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3b4c057e80 con 0x7f3b600b8200 2023-07-13T22:29:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b4406ace0 msgr2=0x7f3b4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b4406ace0 0x7f3b4406d190 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3b600b1a50 tx=0x7f3b54009040 comp rx=0 tx=0).stop 2023-07-13T22:29:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600b8200 msgr2=0x7f3b601f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600b8200 0x7f3b601f25b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00e980 tx=0x7f3b4c00ec90 comp rx=0 tx=0).stop 2023-07-13T22:29:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 shutdown_connections 2023-07-13T22:29:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3b4406ace0 0x7f3b4406d190 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b600cfd50 0x7f3b601e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b600b1ac0 0x7f3b600cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 --2- 172.21.15.60:0/2356920840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600b8200 0x7f3b601f25b0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 >> 172.21.15.60:0/2356920840 conn(0x7f3b6006b2f0 msgr2=0x7f3b600d2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 shutdown_connections 2023-07-13T22:29:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.756+0000 7f3b66fff700 1 -- 172.21.15.60:0/2356920840 wait complete. 2023-07-13T22:29:01.754 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:29:01.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.981+0000 7f06755ff700 1 Processor -- start 2023-07-13T22:29:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.981+0000 7f06755ff700 1 -- start start 2023-07-13T22:29:01.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.982+0000 7f06755ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0670154950 0x7f0670070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:01.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.982+0000 7f06755ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f067015a820 0x7f067015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:01.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.982+0000 7f06755ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0670070270 con 0x7f06700c8d40 2023-07-13T22:29:01.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.982+0000 7f06755ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067015b2e0 con 0x7f0670154950 2023-07-13T22:29:01.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.982+0000 7f06755ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067015b5c0 con 0x7f067015a820 2023-07-13T22:29:01.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.982+0000 7f066effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0670154950 0x7f0670070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:01.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.982+0000 7f066effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0670154950 0x7f0670070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57702/0 (socket says 172.21.15.60:57702) 2023-07-13T22:29:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.982+0000 7f066effd700 1 -- 172.21.15.60:0/1585326601 learned_addr learned my addr 172.21.15.60:0/1585326601 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.983+0000 7f066effd700 1 -- 172.21.15.60:0/1585326601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f067015a820 msgr2=0x7f067015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.983+0000 7f066effd700 1 --2- 172.21.15.60:0/1585326601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f067015a820 0x7f067015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.983+0000 7f066effd700 1 -- 172.21.15.60:0/1585326601 >> v1:172.21.15.105:6789/0 conn(0x7f06700c8d40 legacy=0x7f06700c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.983+0000 7f066effd700 1 -- 172.21.15.60:0/1585326601 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06700704c0 con 0x7f0670154950 2023-07-13T22:29:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.983+0000 7f066effd700 1 --2- 172.21.15.60:0/1585326601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 0x7f0670070cc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f06600098f0 tx=0x7f0660009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474babfbd4e2b8ed server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.983+0000 7f066dffb700 1 -- 172.21.15.60:0/1585326601 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660004230 con 0x7f0670154950 2023-07-13T22:29:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.983+0000 7f066dffb700 1 -- 172.21.15.60:0/1585326601 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0660004390 con 0x7f0670154950 2023-07-13T22:29:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.983+0000 7f066dffb700 1 -- 172.21.15.60:0/1585326601 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06600036a0 con 0x7f0670154950 2023-07-13T22:29:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.984+0000 7f06755ff700 1 -- 172.21.15.60:0/1585326601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 msgr2=0x7f0670070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.984+0000 7f06755ff700 1 --2- 172.21.15.60:0/1585326601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 0x7f0670070cc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f06600098f0 tx=0x7f0660009c00 comp rx=0 tx=0).stop 2023-07-13T22:29:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.984+0000 7f06755ff700 1 -- 172.21.15.60:0/1585326601 shutdown_connections 2023-07-13T22:29:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.984+0000 7f06755ff700 1 --2- 172.21.15.60:0/1585326601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 0x7f0670070cc0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.984+0000 7f06755ff700 1 --2- 172.21.15.60:0/1585326601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f067015a820 0x7f067015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.984+0000 7f06755ff700 1 -- 172.21.15.60:0/1585326601 >> 172.21.15.60:0/1585326601 conn(0x7f06700cf410 msgr2=0x7f06700cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:01.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.984+0000 7f06755ff700 1 -- 172.21.15.60:0/1585326601 shutdown_connections 2023-07-13T22:29:01.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.984+0000 7f06755ff700 1 -- 172.21.15.60:0/1585326601 wait complete. 2023-07-13T22:29:01.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.986+0000 7f06755ff700 1 Processor -- start 2023-07-13T22:29:01.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.987+0000 7f06755ff700 1 -- start start 2023-07-13T22:29:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f06755ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06700c8d40 0x7f067014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f06755ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 0x7f06700cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f06755ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f067015a820 0x7f06701e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f06755ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06700c7d70 con 0x7f0670154950 2023-07-13T22:29:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f06755ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06700c8050 con 0x7f067015a820 2023-07-13T22:29:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f06755ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06700c58d0 con 0x7f06700c8d40 2023-07-13T22:29:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f066e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 0x7f06700cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f066e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 0x7f06700cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57714/0 (socket says 172.21.15.60:57714) 2023-07-13T22:29:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f066e7fc700 1 -- 172.21.15.60:0/3482684279 learned_addr learned my addr 172.21.15.60:0/3482684279 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f066effd700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06700c8d40 0x7f067014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.988+0000 7f066f7fe700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f067015a820 0x7f06701e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f066e7fc700 1 -- 172.21.15.60:0/3482684279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06700c8d40 msgr2=0x7f067014f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f066e7fc700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06700c8d40 0x7f067014f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f066e7fc700 1 -- 172.21.15.60:0/3482684279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f067015a820 msgr2=0x7f06701e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f066e7fc700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f067015a820 0x7f06701e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f066e7fc700 1 -- 172.21.15.60:0/3482684279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660009550 con 0x7f0670154950 2023-07-13T22:29:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f066e7fc700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 0x7f06700cdb80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f066400b4a0 tx=0x7f066400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f06577fe700 1 -- 172.21.15.60:0/3482684279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664011830 con 0x7f0670154950 2023-07-13T22:29:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f06577fe700 1 -- 172.21.15.60:0/3482684279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0664011990 con 0x7f0670154950 2023-07-13T22:29:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f06577fe700 1 -- 172.21.15.60:0/3482684279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664011d40 con 0x7f0670154950 2023-07-13T22:29:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06700c5bb0 con 0x7f0670154950 2023-07-13T22:29:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.989+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067014f550 con 0x7f0670154950 2023-07-13T22:29:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.990+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06700cbc20 con 0x7f0670154950 2023-07-13T22:29:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.991+0000 7f06577fe700 1 -- 172.21.15.60:0/3482684279 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f066401a400 con 0x7f0670154950 2023-07-13T22:29:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.992+0000 7f06577fe700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f065c06ace0 0x7f065c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.992+0000 7f06577fe700 1 -- 172.21.15.60:0/3482684279 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06640893e0 con 0x7f0670154950 2023-07-13T22:29:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.992+0000 7f066effd700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f065c06ace0 0x7f065c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.993+0000 7f066effd700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f065c06ace0 0x7f065c06d190 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0660009130 tx=0x7f0660008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:01.997+0000 7f06577fe700 1 -- 172.21.15.60:0/3482684279 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0664054400 con 0x7f0670154950 2023-07-13T22:29:02.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.251+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f067006e100 con 0x7f0670154950 2023-07-13T22:29:02.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.251+0000 7f06577fe700 1 -- 172.21.15.60:0/3482684279 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f0664057a20 con 0x7f0670154950 2023-07-13T22:29:02.230 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:02.230 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:58.254590+0000","modified":"2023-07-13T22:29:01.746898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4078186433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":4078186433},{"type":"v1","addr":"172.21.15.60:6837","nonce":4078186433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-13T22:29:02.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f065c06ace0 msgr2=0x7f065c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f065c06ace0 0x7f065c06d190 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0660009130 tx=0x7f0660008030 comp rx=0 tx=0).stop 2023-07-13T22:29:02.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 msgr2=0x7f06700cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 0x7f06700cdb80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f066400b4a0 tx=0x7f066400b7b0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 shutdown_connections 2023-07-13T22:29:02.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f065c06ace0 0x7f065c06d190 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f067015a820 0x7f06701e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06700c8d40 0x7f067014f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 --2- 172.21.15.60:0/3482684279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0670154950 0x7f06700cdb80 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 >> 172.21.15.60:0/3482684279 conn(0x7f06700cf410 msgr2=0x7f06700cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 shutdown_connections 2023-07-13T22:29:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.254+0000 7f06755ff700 1 -- 172.21.15.60:0/3482684279 wait complete. 2023-07-13T22:29:02.236 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2023-07-13T22:29:02.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.476+0000 7f06da5ff700 1 Processor -- start 2023-07-13T22:29:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.476+0000 7f06da5ff700 1 -- start start 2023-07-13T22:29:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.477+0000 7f06da5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06d40b8570 0x7f06d40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.477+0000 7f06da5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06d40d35c0 0x7f06d40d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06da5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40cf3f0 0x7f06d40d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06da5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d40d3b30 con 0x7f06d40b8570 2023-07-13T22:29:02.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06da5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d40d1930 con 0x7f06d40d35c0 2023-07-13T22:29:02.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06da5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40d09d0 con 0x7f06d40cf3f0 2023-07-13T22:29:02.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06d40b8570 0x7f06d40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06d40b8570 0x7f06d40ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57720/0 (socket says 172.21.15.60:57720) 2023-07-13T22:29:02.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d3fff700 1 -- 172.21.15.60:0/167251727 learned_addr learned my addr 172.21.15.60:0/167251727 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:02.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d37fe700 1 --2- 172.21.15.60:0/167251727 >> v2:172.21.15.105:3300/0 conn(0x7f06d40d35c0 0x7f06d40d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d8b9c700 1 --2- 172.21.15.60:0/167251727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40cf3f0 0x7f06d40d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d3fff700 1 -- 172.21.15.60:0/167251727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40cf3f0 msgr2=0x7f06d40d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:02.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d3fff700 1 --2- 172.21.15.60:0/167251727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40cf3f0 0x7f06d40d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d3fff700 1 -- 172.21.15.60:0/167251727 >> v2:172.21.15.105:3300/0 conn(0x7f06d40d35c0 msgr2=0x7f06d40d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:02.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d3fff700 1 --2- 172.21.15.60:0/167251727 >> v2:172.21.15.105:3300/0 conn(0x7f06d40d35c0 0x7f06d40d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d3fff700 1 -- 172.21.15.60:0/167251727 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d40cda10 con 0x7f06d40b8570 2023-07-13T22:29:02.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d37fe700 1 --2- 172.21.15.60:0/167251727 >> v2:172.21.15.105:3300/0 conn(0x7f06d40d35c0 0x7f06d40d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:02.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.478+0000 7f06d8b9c700 1 --2- 172.21.15.60:0/167251727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40cf3f0 0x7f06d40d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:02.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06d3fff700 1 --2- 172.21.15.60:0/167251727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 0x7f06d40ceeb0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f06d40d07e0 tx=0x7f06c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d354b350ee41b1ad server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:02.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06d2ffd700 1 -- 172.21.15.60:0/167251727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4005e30 con 0x7f06d40b8570 2023-07-13T22:29:02.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06d2ffd700 1 -- 172.21.15.60:0/167251727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4004030 con 0x7f06d40b8570 2023-07-13T22:29:02.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06d2ffd700 1 -- 172.21.15.60:0/167251727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40043e0 con 0x7f06d40b8570 2023-07-13T22:29:02.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06da5ff700 1 -- 172.21.15.60:0/167251727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 msgr2=0x7f06d40ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06da5ff700 1 --2- 172.21.15.60:0/167251727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 0x7f06d40ceeb0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f06d40d07e0 tx=0x7f06c4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06da5ff700 1 -- 172.21.15.60:0/167251727 shutdown_connections 2023-07-13T22:29:02.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06da5ff700 1 --2- 172.21.15.60:0/167251727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40cf3f0 0x7f06d40d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06da5ff700 1 --2- 172.21.15.60:0/167251727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 0x7f06d40ceeb0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06da5ff700 1 --2- 172.21.15.60:0/167251727 >> v2:172.21.15.105:3300/0 conn(0x7f06d40d35c0 0x7f06d40d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.479+0000 7f06da5ff700 1 -- 172.21.15.60:0/167251727 >> 172.21.15.60:0/167251727 conn(0x7f06d4071600 msgr2=0x7f06d40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:02.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.480+0000 7f06da5ff700 1 -- 172.21.15.60:0/167251727 shutdown_connections 2023-07-13T22:29:02.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.480+0000 7f06da5ff700 1 -- 172.21.15.60:0/167251727 wait complete. 2023-07-13T22:29:02.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.481+0000 7f06da5ff700 1 Processor -- start 2023-07-13T22:29:02.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.481+0000 7f06da5ff700 1 -- start start 2023-07-13T22:29:02.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.482+0000 7f06da5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 0x7f06d415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.482+0000 7f06da5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d40cf3f0 0x7f06d40c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.482+0000 7f06da5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40d35c0 0x7f06d40c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.482+0000 7f06da5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40c7f10 con 0x7f06d40b8570 2023-07-13T22:29:02.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.482+0000 7f06da5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40c7290 con 0x7f06d40cf3f0 2023-07-13T22:29:02.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.482+0000 7f06da5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40c7570 con 0x7f06d40d35c0 2023-07-13T22:29:02.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 0x7f06d415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 0x7f06d415b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57730/0 (socket says 172.21.15.60:57730) 2023-07-13T22:29:02.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d3fff700 1 -- 172.21.15.60:0/669811786 learned_addr learned my addr 172.21.15.60:0/669811786 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:02.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d37fe700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d40cf3f0 0x7f06d40c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d8b9c700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40d35c0 0x7f06d40c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d3fff700 1 -- 172.21.15.60:0/669811786 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40d35c0 msgr2=0x7f06d40c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d3fff700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40d35c0 0x7f06d40c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d3fff700 1 -- 172.21.15.60:0/669811786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d40cf3f0 msgr2=0x7f06d40c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d3fff700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d40cf3f0 0x7f06d40c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d3fff700 1 -- 172.21.15.60:0/669811786 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c4009500 con 0x7f06d40b8570 2023-07-13T22:29:02.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.483+0000 7f06d3fff700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 0x7f06d415b030 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009c10 tx=0x7f06c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:02.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.484+0000 7f06d0ff9700 1 -- 172.21.15.60:0/669811786 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4003c70 con 0x7f06d40b8570 2023-07-13T22:29:02.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.484+0000 7f06d0ff9700 1 -- 172.21.15.60:0/669811786 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4003dd0 con 0x7f06d40b8570 2023-07-13T22:29:02.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.484+0000 7f06d0ff9700 1 -- 172.21.15.60:0/669811786 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4011660 con 0x7f06d40b8570 2023-07-13T22:29:02.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.484+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d40c7850 con 0x7f06d40b8570 2023-07-13T22:29:02.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.484+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d415b570 con 0x7f06d40b8570 2023-07-13T22:29:02.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.485+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d404e280 con 0x7f06d40b8570 2023-07-13T22:29:02.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.486+0000 7f06d0ff9700 1 -- 172.21.15.60:0/669811786 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f06c40117c0 con 0x7f06d40b8570 2023-07-13T22:29:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.487+0000 7f06d0ff9700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c006ace0 0x7f06c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.487+0000 7f06d0ff9700 1 -- 172.21.15.60:0/669811786 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06c408a4e0 con 0x7f06d40b8570 2023-07-13T22:29:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.487+0000 7f06d37fe700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c006ace0 0x7f06c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.488+0000 7f06d37fe700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c006ace0 0x7f06c006d190 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f06d41e64a0 tx=0x7f06c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.491+0000 7f06d0ff9700 1 -- 172.21.15.60:0/669811786 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f06c4055620 con 0x7f06d40b8570 2023-07-13T22:29:02.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.699+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06d4065ee0 con 0x7f06d40b8570 2023-07-13T22:29:02.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.700+0000 7f06d0ff9700 1 -- 172.21.15.60:0/669811786 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+18190 (secure 0 0 0) 0x7f06c4058c40 con 0x7f06d40b8570 2023-07-13T22:29:02.679 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:02.679 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":100,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:59.528113+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:56.533994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:57.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:02.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c006ace0 msgr2=0x7f06c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c006ace0 0x7f06c006d190 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f06d41e64a0 tx=0x7f06c8008040 comp rx=0 tx=0).stop 2023-07-13T22:29:02.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 msgr2=0x7f06d415b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 0x7f06d415b030 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009c10 tx=0x7f06c4004750 comp rx=0 tx=0).stop 2023-07-13T22:29:02.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 shutdown_connections 2023-07-13T22:29:02.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06c006ace0 0x7f06c006d190 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06d40b8570 0x7f06d415b030 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d40d35c0 0x7f06d40c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 --2- 172.21.15.60:0/669811786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d40cf3f0 0x7f06d40c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 >> 172.21.15.60:0/669811786 conn(0x7f06d4071600 msgr2=0x7f06d415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:02.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 shutdown_connections 2023-07-13T22:29:02.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.703+0000 7f06da5ff700 1 -- 172.21.15.60:0/669811786 wait complete. 2023-07-13T22:29:02.700 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:29:02.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:29:02.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.927+0000 7f639a3ff700 1 Processor -- start 2023-07-13T22:29:02.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.927+0000 7f639a3ff700 1 -- start start 2023-07-13T22:29:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.928+0000 7f639a3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f639415c6e0 0x7f639406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.928+0000 7f639a3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63940b24b0 0x7f63940b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.928+0000 7f639a3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639406b8e0 con 0x7f63940b3050 2023-07-13T22:29:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.928+0000 7f639a3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639406b6f0 con 0x7f63940b24b0 2023-07-13T22:29:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.928+0000 7f639a3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639415cab0 con 0x7f639415c6e0 2023-07-13T22:29:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.928+0000 7f639899c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63940b24b0 0x7f63940b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.928+0000 7f63937fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f63940b3050 0x7f639406afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50092/0 (socket says 172.21.15.60:50092) 2023-07-13T22:29:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f63937fe700 1 -- 172.21.15.60:0/2157249952 learned_addr learned my addr 172.21.15.60:0/2157249952 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f6393fff700 1 --2- 172.21.15.60:0/2157249952 >> v2:172.21.15.105:3300/0 conn(0x7f639415c6e0 0x7f639406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f639899c700 1 -- 172.21.15.60:0/2157249952 >> v2:172.21.15.105:3300/0 conn(0x7f639415c6e0 msgr2=0x7f639406c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:02.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f639899c700 1 --2- 172.21.15.60:0/2157249952 >> v2:172.21.15.105:3300/0 conn(0x7f639415c6e0 0x7f639406c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f639899c700 1 -- 172.21.15.60:0/2157249952 >> v1:172.21.15.60:6789/0 conn(0x7f63940b3050 legacy=0x7f639406afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f639899c700 1 -- 172.21.15.60:0/2157249952 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639406bbc0 con 0x7f63940b24b0 2023-07-13T22:29:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f6392ffd700 1 -- 172.21.15.60:0/2157249952 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767707173 0 0) 0x7f639406b8e0 con 0x7f63940b3050 2023-07-13T22:29:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f639899c700 1 --2- 172.21.15.60:0/2157249952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b24b0 0x7f63940b2930 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f63940ce0f0 tx=0x7f638c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60e0a4c2f56957d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f6392ffd700 1 -- 172.21.15.60:0/2157249952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00e070 con 0x7f63940b24b0 2023-07-13T22:29:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f6392ffd700 1 -- 172.21.15.60:0/2157249952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c00bb20 con 0x7f63940b24b0 2023-07-13T22:29:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.929+0000 7f6392ffd700 1 -- 172.21.15.60:0/2157249952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c012430 con 0x7f63940b24b0 2023-07-13T22:29:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.930+0000 7f639a3ff700 1 -- 172.21.15.60:0/2157249952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b24b0 msgr2=0x7f63940b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.930+0000 7f639a3ff700 1 --2- 172.21.15.60:0/2157249952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b24b0 0x7f63940b2930 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f63940ce0f0 tx=0x7f638c00b330 comp rx=0 tx=0).stop 2023-07-13T22:29:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.930+0000 7f639a3ff700 1 -- 172.21.15.60:0/2157249952 shutdown_connections 2023-07-13T22:29:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.930+0000 7f639a3ff700 1 --2- 172.21.15.60:0/2157249952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b24b0 0x7f63940b2930 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.930+0000 7f639a3ff700 1 --2- 172.21.15.60:0/2157249952 >> v2:172.21.15.105:3300/0 conn(0x7f639415c6e0 0x7f639406c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.930+0000 7f639a3ff700 1 -- 172.21.15.60:0/2157249952 >> 172.21.15.60:0/2157249952 conn(0x7f639406e680 msgr2=0x7f639406ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.930+0000 7f639a3ff700 1 -- 172.21.15.60:0/2157249952 shutdown_connections 2023-07-13T22:29:02.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.930+0000 7f639a3ff700 1 -- 172.21.15.60:0/2157249952 wait complete. 2023-07-13T22:29:02.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.933+0000 7f639a3ff700 1 Processor -- start 2023-07-13T22:29:02.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.933+0000 7f639a3ff700 1 -- start start 2023-07-13T22:29:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f639a3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63940b24b0 0x7f63940b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f639a3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b3050 0x7f63940c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f639a3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639415c6e0 0x7f6394157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f639a3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940c4ff0 con 0x7f63940b3050 2023-07-13T22:29:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f639a3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940c4690 con 0x7f639415c6e0 2023-07-13T22:29:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f639a3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63940c4930 con 0x7f63940b24b0 2023-07-13T22:29:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f63937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b3050 0x7f63940c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f63937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b3050 0x7f63940c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57746/0 (socket says 172.21.15.60:57746) 2023-07-13T22:29:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f63937fe700 1 -- 172.21.15.60:0/3799680265 learned_addr learned my addr 172.21.15.60:0/3799680265 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f6393fff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63940b24b0 0x7f63940b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f639899c700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639415c6e0 0x7f6394157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f63937fe700 1 -- 172.21.15.60:0/3799680265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63940b24b0 msgr2=0x7f63940b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f63937fe700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63940b24b0 0x7f63940b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f63937fe700 1 -- 172.21.15.60:0/3799680265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639415c6e0 msgr2=0x7f6394157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f63937fe700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639415c6e0 0x7f6394157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:02.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f63937fe700 1 -- 172.21.15.60:0/3799680265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c009a70 con 0x7f63940b3050 2023-07-13T22:29:02.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.934+0000 7f6393fff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63940b24b0 0x7f63940b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:02.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.935+0000 7f63937fe700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b3050 0x7f63940c6c50 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f63880011e0 tx=0x7f638800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:02.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.935+0000 7f6390ff9700 1 -- 172.21.15.60:0/3799680265 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388004c80 con 0x7f63940b3050 2023-07-13T22:29:02.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.935+0000 7f6390ff9700 1 -- 172.21.15.60:0/3799680265 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6388004e00 con 0x7f63940b3050 2023-07-13T22:29:02.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.935+0000 7f6390ff9700 1 -- 172.21.15.60:0/3799680265 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388007310 con 0x7f63940b3050 2023-07-13T22:29:02.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.935+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63940c4c10 con 0x7f63940b3050 2023-07-13T22:29:02.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.935+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63940b4b90 con 0x7f63940b3050 2023-07-13T22:29:02.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.936+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63940c5130 con 0x7f63940b3050 2023-07-13T22:29:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.937+0000 7f6390ff9700 1 -- 172.21.15.60:0/3799680265 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6388007470 con 0x7f63940b3050 2023-07-13T22:29:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.938+0000 7f6390ff9700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f638006ace0 0x7f638006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.938+0000 7f6390ff9700 1 -- 172.21.15.60:0/3799680265 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6388089210 con 0x7f63940b3050 2023-07-13T22:29:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.938+0000 7f6393fff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f638006ace0 0x7f638006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.939+0000 7f6393fff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f638006ace0 0x7f638006d190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f639406b220 tx=0x7f6384008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:02.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:02.942+0000 7f6390ff9700 1 -- 172.21.15.60:0/3799680265 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f63940c5130 con 0x7f63940b3050 2023-07-13T22:29:03.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.143+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f639406c180 con 0x7f638006ace0 2023-07-13T22:29:03.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.152+0000 7f6390ff9700 1 -- 172.21.15.60:0/3799680265 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296690 (secure 0 0 0) 0x7f6384011780 con 0x7f638006ace0 2023-07-13T22:29:03.136 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:03.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f638006ace0 msgr2=0x7f638006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f638006ace0 0x7f638006d190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f639406b220 tx=0x7f6384008040 comp rx=0 tx=0).stop 2023-07-13T22:29:03.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b3050 msgr2=0x7f63940c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b3050 0x7f63940c6c50 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f63880011e0 tx=0x7f638800cae0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 shutdown_connections 2023-07-13T22:29:03.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f638006ace0 0x7f638006d190 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639415c6e0 0x7f6394157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63940b24b0 0x7f63940b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 --2- 172.21.15.60:0/3799680265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63940b3050 0x7f63940c6c50 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 >> 172.21.15.60:0/3799680265 conn(0x7f639406e680 msgr2=0x7f63940ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.156+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 shutdown_connections 2023-07-13T22:29:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.157+0000 7f639a3ff700 1 -- 172.21.15.60:0/3799680265 wait complete. 2023-07-13T22:29:03.141 INFO:tasks.ceph.mds.c.smithi060.stderr:2023-07-13T22:29:03.157+0000 7fc7c57f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:29:03.141 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:29:03.154 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2023-07-13T22:29:01.232911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283344,"kb_used_data":10384,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754691376,"statfs":{"total":773094113280,"available":772803969024,"internally_reserved":0,"allocated":10633216,"data_stored":5457156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.505442"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":158,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-07-13T22:28:53.710394+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:28:53.710394+0000","last_peered":"2023-07-13T22:28:53.710394+0000","last_clean":"2023-07-13T22:28:53.710394+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:28:53.710394+0000","last_undegraded":"2023-07-13T22:28:53.710394+0000","last_fullsized":"2023-07-13T22:28:53.710394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:48:19.989497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":146,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-07-13T22:28:56.196480+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:28:56.196480+0000","last_peered":"2023-07-13T22:28:56.196480+0000","last_clean":"2023-07-13T22:28:56.196480+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:28:56.196480+0000","last_undegraded":"2023-07-13T22:28:56.196480+0000","last_fullsized":"2023-07-13T22:28:56.196480+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:45:41.832305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":158,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-07-13T22:28:53.709629+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:28:53.709629+0000","last_peered":"2023-07-13T22:28:53.709629+0000","last_clean":"2023-07-13T22:28:53.709629+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:28:53.709629+0000","last_undegraded":"2023-07-13T22:28:53.709629+0000","last_fullsized":"2023-07-13T22:28:53.709629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:57:41.253254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":295,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-07-13T22:28:52.752641+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:28:52.752641+0000","last_peered":"2023-07-13T22:28:52.752641+0000","last_clean":"2023-07-13T22:28:52.752641+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:28:52.752641+0000","last_undegraded":"2023-07-13T22:28:52.752641+0000","last_fullsized":"2023-07-13T22:28:52.752641+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:46:46.292702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":146,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-07-13T22:28:52.752673+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:28:52.752673+0000","last_peered":"2023-07-13T22:28:52.752673+0000","last_clean":"2023-07-13T22:28:52.752673+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:28:52.752673+0000","last_undegraded":"2023-07-13T22:28:52.752673+0000","last_fullsized":"2023-07-13T22:28:52.752673+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:18:56.273225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":147,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-07-13T22:28:52.752701+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:28:52.752701+0000","last_peered":"2023-07-13T22:28:52.752701+0000","last_clean":"2023-07-13T22:28:52.752701+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:28:52.752701+0000","last_undegraded":"2023-07-13T22:28:52.752701+0000","last_fullsized":"2023-07-13T22:28:52.752701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:41:33.606096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":164,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-07-13T22:28:53.710394+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:28:53.710394+0000","last_peered":"2023-07-13T22:28:53.710394+0000","last_clean":"2023-07-13T22:28:53.710394+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:28:53.710394+0000","last_undegraded":"2023-07-13T22:28:53.710394+0000","last_fullsized":"2023-07-13T22:28:53.710394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:37:04.147650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":140,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-07-13T22:28:52.752329+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:28:52.752329+0000","last_peered":"2023-07-13T22:28:52.752329+0000","last_clean":"2023-07-13T22:28:52.752329+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:28:52.752329+0000","last_undegraded":"2023-07-13T22:28:52.752329+0000","last_fullsized":"2023-07-13T22:28:52.752329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:29:51.814410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:57.723219+0000","last_change":"2023-07-13T22:28:57.723219+0000","last_active":"2023-07-13T22:28:57.723219+0000","last_peered":"2023-07-13T22:28:57.723219+0000","last_clean":"2023-07-13T22:28:57.723219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:57.723219+0000","last_undegraded":"2023-07-13T22:28:57.723219+0000","last_fullsized":"2023-07-13T22:28:57.723219+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-13T22:28:56.714828+0000","last_change":"2023-07-13T22:28:56.714828+0000","last_active":"2023-07-13T22:28:56.714828+0000","last_peered":"2023-07-13T22:28:56.714828+0000","last_clean":"2023-07-13T22:28:56.714828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-13T22:28:56.714828+0000","last_undegraded":"2023-07-13T22:28:56.714828+0000","last_fullsized":"2023-07-13T22:28:56.714828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":140,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-07-13T22:28:52.752343+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:28:52.752343+0000","last_peered":"2023-07-13T22:28:52.752343+0000","last_clean":"2023-07-13T22:28:52.752343+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:28:52.752343+0000","last_undegraded":"2023-07-13T22:28:52.752343+0000","last_fullsized":"2023-07-13T22:28:52.752343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:32:58.136970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1769472,"data_stored":1122427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.311,"15min":0.311},"max":{"1min":0.98699999999999999,"5min":1.0780000000000001,"15min":1.115},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69799999999999995,"15min":0.67200000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.30099999999999999},"max":{"1min":0.97199999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68400000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63800000000000001,"15min":0.68799999999999994},"min":{"1min":0.42399999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94799999999999995,"15min":2.2610000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67300000000000004,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.34799999999999998,"15min":0.34300000000000003},"max":{"1min":0.873,"5min":1.0329999999999999,"15min":1.0609999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.78400000000000003},"min":{"1min":0.45600000000000002,"5min":0.443,"15min":0.40600000000000003},"max":{"1min":0.95099999999999996,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70299999999999996,"15min":0.77700000000000002},"min":{"1min":0.495,"5min":0.42999999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":1.107,"15min":3.4710000000000001},"last":0.627}]},{"osd":3,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69899999999999995,"15min":0.68799999999999994},"min":{"1min":0.501,"5min":0.45000000000000001,"15min":0.34599999999999997},"max":{"1min":0.98199999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.75800000000000001},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.01,"5min":1.024,"15min":1.1399999999999999},"last":0.74199999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.64200000000000002},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90100000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.496},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91500000000000004,"5min":1.0860000000000001,"15min":1.1850000000000001},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.66200000000000003},"min":{"1min":0.503,"5min":0.38200000000000001,"15min":0.35099999999999998},"max":{"1min":0.91500000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.79000000000000004},"min":{"1min":0.59899999999999998,"5min":0.498,"15min":0.48899999999999999},"max":{"1min":0.96499999999999997,"5min":1.1359999999999999,"15min":1.2110000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69099999999999995,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.437,"15min":0.39500000000000002},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.72199999999999998},"min":{"1min":0.47199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80000000000000004,"5min":0.98799999999999999,"15min":1.042},"last":0.53900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574948,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1175552,"data_stored":532059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67100000000000004,"15min":0.628},"min":{"1min":0.42699999999999999,"5min":0.33100000000000002,"15min":0.28799999999999998},"max":{"1min":1.002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68999999999999995,"15min":0.628},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.33500000000000002},"max":{"1min":0.89700000000000002,"5min":0.97399999999999998,"15min":0.98899999999999999},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.505,"5min":0.39400000000000002,"15min":0.33900000000000002},"max":{"1min":0.94099999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73399999999999999,"15min":0.70299999999999996},"min":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.438},"max":{"1min":0.96899999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.67100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.91800000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.878},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69199999999999995,"15min":0.67000000000000004},"min":{"1min":0.56200000000000006,"5min":0.45600000000000002,"15min":0.35599999999999998},"max":{"1min":0.871,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.625}]},{"osd":3,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70899999999999996,"15min":0.66300000000000003},"min":{"1min":0.56100000000000005,"5min":0.37,"15min":0.37},"max":{"1min":0.91600000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73599999999999999,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.95599999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.94499999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65500000000000003,"15min":0.68799999999999994},"min":{"1min":0.52200000000000002,"5min":0.40200000000000002,"15min":0.30399999999999999},"max":{"1min":0.997,"5min":0.998,"15min":0.998},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86899999999999999,"5min":0.94199999999999995,"15min":0.95999999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.73799999999999999,"15min":0.75800000000000001},"min":{"1min":0.54200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97399999999999998,"5min":1.05,"15min":1.05},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70699999999999996,"15min":0.745},"min":{"1min":0.46700000000000003,"5min":0.45600000000000002,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96999999999999997,"15min":1.1890000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":1.111},"last":0.82399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36884,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334956,"statfs":{"total":96636764160,"available":96598994944,"internally_reserved":0,"allocated":1781760,"data_stored":1127907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45000000000000001,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.249,"15min":0.249},"max":{"1min":0.70199999999999996,"5min":0.71599999999999997,"15min":1.952},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.496,"15min":0.51700000000000002},"min":{"1min":0.30599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":1.1060000000000001},"last":0.79900000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.29899999999999999,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":2.1200000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.35099999999999998},"max":{"1min":0.67400000000000004,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":2.008},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53000000000000003,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.79000000000000004,"5min":1.054,"15min":1.054},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73899999999999999,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.57199999999999995,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":0.98199999999999998,"15min":2.028},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.46400000000000002},"max":{"1min":0.94199999999999995,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.626}]},{"osd":5,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.98899999999999999,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78000000000000003,"15min":0.84399999999999997},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.006,"5min":1.095,"15min":2.0899999999999999},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78600000000000003,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.55100000000000005,"15min":0.52600000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.2230000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.85799999999999998},"min":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0489999999999999,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.86799999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79400000000000004,"15min":0.82699999999999996},"min":{"1min":0.68700000000000006,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":0.99199999999999999,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.89200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36288,"kb_used_data":1144,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335552,"statfs":{"total":96636764160,"available":96599605248,"internally_reserved":0,"allocated":1171456,"data_stored":531073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.52500000000000002},"min":{"1min":0.33500000000000002,"5min":0.30099999999999999,"15min":0.28499999999999998},"max":{"1min":0.84999999999999998,"5min":1.026,"15min":11.791},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.53400000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.874,"5min":1.419,"15min":4.657},"last":0.75900000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.432,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":1.1459999999999999,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52600000000000002,"15min":0.96699999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":1.276,"15min":13.398999999999999},"last":0.376}]},{"osd":3,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72999999999999998,"5min":0.89400000000000002,"15min":1.8089999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56499999999999995,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.997,"15min":4.4530000000000003},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.747,"15min":1.218},"min":{"1min":0.44800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76000000000000001,"15min":0.80800000000000005},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.41299999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.776},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.78400000000000003,"15min":1.238},"min":{"1min":0.60399999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.073,"5min":1.113,"15min":13.349},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.82699999999999996,"15min":0.88},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.302,"15min":4.7649999999999997},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80000000000000004,"15min":0.83899999999999997},"min":{"1min":0.64800000000000002,"5min":0.52800000000000002,"15min":0.49099999999999999},"max":{"1min":1.212,"5min":1.212,"15min":4.5039999999999996},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":1.2090000000000001},"min":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1259999999999999,"5min":1.2330000000000001,"15min":11.679},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79200000000000004,"15min":1.2430000000000001},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.101,"5min":1.1759999999999999,"15min":13.298},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80100000000000005,"15min":0.89300000000000002},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.151,"5min":1.151,"15min":11.456},"last":0.64500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1183744,"data_stored":536553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.44,"15min":0.53300000000000003},"min":{"1min":0.307,"5min":0.249,"15min":0.249},"max":{"1min":0.78300000000000003,"5min":1.1779999999999999,"15min":2.282},"last":0.311},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.34799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64600000000000002,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.47399999999999998},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.64800000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51900000000000002,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":2.319},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.45100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":1.01,"15min":1.01},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.64300000000000002},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.38700000000000001},"max":{"1min":0.72899999999999998,"5min":0.95599999999999996,"15min":2.351},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.76200000000000001,"15min":0.83899999999999997},"min":{"1min":0.64000000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.165,"15min":2.238},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.58299999999999996,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":1.274,"15min":1.518},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.57099999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.997,"5min":1.1299999999999999,"15min":1.325},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.80300000000000005,"15min":0.85799999999999998},"min":{"1min":0.66700000000000004,"5min":0.53500000000000003,"15min":0.495},"max":{"1min":0.97999999999999998,"5min":1.2490000000000001,"15min":2.1909999999999998},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.71099999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.026,"5min":1.1040000000000001,"15min":1.488},"last":0.746},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.61599999999999999,"5min":0.502,"15min":0.48199999999999998},"max":{"1min":0.96499999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.81599999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81599999999999995,"15min":0.79600000000000004},"min":{"1min":0.72299999999999998,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":0.98199999999999998,"5min":1.29,"15min":1.29},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.69899999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.046,"5min":1.254,"15min":1.254},"last":0.85199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36292,"kb_used_data":1148,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":1175552,"data_stored":532059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.41999999999999998,"15min":0.42399999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.48299999999999998,"15min":0.45300000000000001},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.27200000000000002},"max":{"1min":0.71099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.35599999999999998,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.378,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.71899999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73599999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.45800000000000002,"15min":0.44700000000000001},"max":{"1min":1.075,"5min":1.1699999999999999,"15min":1.2250000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.014,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.96899999999999997,"5min":1.1919999999999999,"15min":1.411},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.998,"5min":1.1779999999999999,"15min":1.5469999999999999},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.52500000000000002,"15min":0.51800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.52100000000000002},"max":{"1min":0.93500000000000005,"5min":1.046,"15min":1.1240000000000001},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.80100000000000005},"min":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.498},"max":{"1min":1.0740000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78200000000000003,"15min":0.79100000000000004},"min":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.47199999999999998},"max":{"1min":0.93400000000000005,"5min":1.153,"15min":1.1950000000000001},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":1187840,"data_stored":537539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.97299999999999998,"15min":1.0349999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86599999999999999,"5min":0.997,"15min":0.997},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":1.0489999999999999,"15min":1.298},"last":0.627},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.69499999999999995},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.438},"max":{"1min":0.85499999999999998,"5min":1.0049999999999999,"15min":1.3109999999999999},"last":0.504}]},{"osd":2,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.48199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.1200000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.71899999999999997},"min":{"1min":0.61799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94599999999999995,"5min":1.089,"15min":1.1839999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.95699999999999996},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.35899999999999999},"max":{"1min":0.83299999999999996,"5min":0.94999999999999996,"15min":10.657999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72899999999999998,"15min":0.93600000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":8.75},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.46200000000000002,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":0.90800000000000003,"5min":1.042,"15min":1.042},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94899999999999995,"5min":1.0109999999999999,"15min":1.0429999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93000000000000005,"5min":1.0189999999999999,"15min":1.173},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.72299999999999998},"min":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97199999999999998,"5min":1.0509999999999999,"15min":1.099},"last":0.76300000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.004},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.753,"15min":0.77200000000000002},"min":{"1min":0.53200000000000003,"5min":0.497,"15min":0.47899999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.212},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":1187840,"data_stored":537539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65800000000000003,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.33700000000000002},"max":{"1min":0.82499999999999996,"5min":1.042,"15min":1.042},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.64000000000000001},"min":{"1min":0.53500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.998,"15min":0.998},"last":0.81499999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89600000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.36299999999999999,"15min":0.33100000000000002},"max":{"1min":0.86799999999999999,"5min":8.9930000000000003,"15min":8.9930000000000003},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.63900000000000001},"min":{"1min":0.57299999999999995,"5min":0.38400000000000001,"15min":0.35899999999999999},"max":{"1min":0.86599999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.626},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.67500000000000004},"min":{"1min":0.57799999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72799999999999998,"15min":0.69499999999999995},"min":{"1min":0.57399999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.96599999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.68799999999999994},"min":{"1min":0.57599999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89500000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81299999999999994,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90900000000000003,"5min":0.93500000000000005,"15min":0.995},"last":0.878}]},{"osd":6,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66300000000000003,"15min":0.66600000000000004},"min":{"1min":0.44,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":0.85399999999999998,"5min":0.98099999999999998,"15min":0.98499999999999999},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71999999999999997,"15min":0.72699999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.997,"15min":1.02},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.76300000000000001},"min":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.503},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0669999999999999},"last":0.80200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:29:03.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:29:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.419+0000 7fbe3abff700 1 Processor -- start 2023-07-13T22:29:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.419+0000 7fbe3abff700 1 -- start start 2023-07-13T22:29:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.420+0000 7fbe3abff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe340d35c0 0x7fbe340d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.420+0000 7fbe3abff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbe340d13f0 0x7fbe3415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.420+0000 7fbe3abff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe340d09d0 con 0x7fbe340b8570 2023-07-13T22:29:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.420+0000 7fbe3abff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe340d3b30 con 0x7fbe340d13f0 2023-07-13T22:29:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.420+0000 7fbe3abff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe340cda10 con 0x7fbe340d35c0 2023-07-13T22:29:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe3899b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbe340b8570 0x7fbe340ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50094/0 (socket says 172.21.15.60:50094) 2023-07-13T22:29:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe3899b700 1 -- 172.21.15.60:0/199171484 learned_addr learned my addr 172.21.15.60:0/199171484 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe3919c700 1 --2- 172.21.15.60:0/199171484 >> v2:172.21.15.60:3300/0 conn(0x7fbe340d13f0 0x7fbe3415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe33fff700 1 --2- 172.21.15.60:0/199171484 >> v2:172.21.15.105:3300/0 conn(0x7fbe340d35c0 0x7fbe340d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe337fe700 1 -- 172.21.15.60:0/199171484 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650626822 0 0) 0x7fbe340d09d0 con 0x7fbe340b8570 2023-07-13T22:29:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe337fe700 1 -- 172.21.15.60:0/199171484 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe240035b0 con 0x7fbe340b8570 2023-07-13T22:29:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe3919c700 1 -- 172.21.15.60:0/199171484 >> v2:172.21.15.105:3300/0 conn(0x7fbe340d35c0 msgr2=0x7fbe340d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe3919c700 1 --2- 172.21.15.60:0/199171484 >> v2:172.21.15.105:3300/0 conn(0x7fbe340d35c0 0x7fbe340d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe3919c700 1 -- 172.21.15.60:0/199171484 >> v1:172.21.15.60:6789/0 conn(0x7fbe340b8570 legacy=0x7fbe340ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe3919c700 1 -- 172.21.15.60:0/199171484 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340cd450 con 0x7fbe340d13f0 2023-07-13T22:29:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.421+0000 7fbe3919c700 1 --2- 172.21.15.60:0/199171484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d13f0 0x7fbe3415c920 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fbe340d07e0 tx=0x7fbe2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4122b3499955d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe337fe700 1 -- 172.21.15.60:0/199171484 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00e070 con 0x7fbe340d13f0 2023-07-13T22:29:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe337fe700 1 -- 172.21.15.60:0/199171484 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00bb20 con 0x7fbe340d13f0 2023-07-13T22:29:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe337fe700 1 -- 172.21.15.60:0/199171484 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c012430 con 0x7fbe340d13f0 2023-07-13T22:29:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe3abff700 1 -- 172.21.15.60:0/199171484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d13f0 msgr2=0x7fbe3415c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe3abff700 1 --2- 172.21.15.60:0/199171484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d13f0 0x7fbe3415c920 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fbe340d07e0 tx=0x7fbe2c00b330 comp rx=0 tx=0).stop 2023-07-13T22:29:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe3abff700 1 -- 172.21.15.60:0/199171484 shutdown_connections 2023-07-13T22:29:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe3abff700 1 --2- 172.21.15.60:0/199171484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d13f0 0x7fbe3415c920 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe3abff700 1 --2- 172.21.15.60:0/199171484 >> v2:172.21.15.105:3300/0 conn(0x7fbe340d35c0 0x7fbe340d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe3abff700 1 -- 172.21.15.60:0/199171484 >> 172.21.15.60:0/199171484 conn(0x7fbe34071600 msgr2=0x7fbe340719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.422+0000 7fbe3abff700 1 -- 172.21.15.60:0/199171484 shutdown_connections 2023-07-13T22:29:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.423+0000 7fbe3abff700 1 -- 172.21.15.60:0/199171484 wait complete. 2023-07-13T22:29:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.425+0000 7fbe3abff700 1 Processor -- start 2023-07-13T22:29:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.425+0000 7fbe3abff700 1 -- start start 2023-07-13T22:29:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.426+0000 7fbe3abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8570 0x7fbe341551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.426+0000 7fbe3abff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340d13f0 0x7fbe34155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.426+0000 7fbe3abff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d35c0 0x7fbe3415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.426+0000 7fbe3abff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340c77e0 con 0x7fbe340d35c0 2023-07-13T22:29:03.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.426+0000 7fbe3abff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe341556f0 con 0x7fbe340d13f0 2023-07-13T22:29:03.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.426+0000 7fbe3abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe34156560 con 0x7fbe340b8570 2023-07-13T22:29:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3919c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d35c0 0x7fbe3415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3919c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d35c0 0x7fbe3415db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57766/0 (socket says 172.21.15.60:57766) 2023-07-13T22:29:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3919c700 1 -- 172.21.15.60:0/3354060858 learned_addr learned my addr 172.21.15.60:0/3354060858 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe33fff700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340d13f0 0x7fbe34155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3899b700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8570 0x7fbe341551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3919c700 1 -- 172.21.15.60:0/3354060858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8570 msgr2=0x7fbe341551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3919c700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8570 0x7fbe341551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3919c700 1 -- 172.21.15.60:0/3354060858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340d13f0 msgr2=0x7fbe34155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3919c700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340d13f0 0x7fbe34155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3919c700 1 -- 172.21.15.60:0/3354060858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c009a70 con 0x7fbe340d35c0 2023-07-13T22:29:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.427+0000 7fbe3919c700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d35c0 0x7fbe3415db30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c014040 tx=0x7fbe2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.428+0000 7fbe317fa700 1 -- 172.21.15.60:0/3354060858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c012430 con 0x7fbe340d35c0 2023-07-13T22:29:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.428+0000 7fbe317fa700 1 -- 172.21.15.60:0/3354060858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c020070 con 0x7fbe340d35c0 2023-07-13T22:29:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.428+0000 7fbe317fa700 1 -- 172.21.15.60:0/3354060858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c0098f0 con 0x7fbe340d35c0 2023-07-13T22:29:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.428+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe34156840 con 0x7fbe340d35c0 2023-07-13T22:29:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.428+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe34156e00 con 0x7fbe340d35c0 2023-07-13T22:29:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.429+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe340c7920 con 0x7fbe340d35c0 2023-07-13T22:29:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.430+0000 7fbe317fa700 1 -- 172.21.15.60:0/3354060858 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe2c004040 con 0x7fbe340d35c0 2023-07-13T22:29:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.431+0000 7fbe317fa700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe1806ace0 0x7fbe1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.431+0000 7fbe317fa700 1 -- 172.21.15.60:0/3354060858 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe2c022070 con 0x7fbe340d35c0 2023-07-13T22:29:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.431+0000 7fbe3899b700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe1806ace0 0x7fbe1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.432+0000 7fbe3899b700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe1806ace0 0x7fbe1806d190 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbe280010e0 tx=0x7fbe28007c70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:03.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.435+0000 7fbe317fa700 1 -- 172.21.15.60:0/3354060858 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbe340c7920 con 0x7fbe340d35c0 2023-07-13T22:29:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.634+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe340d1830 con 0x7fbe1806ace0 2023-07-13T22:29:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.644+0000 7fbe317fa700 1 -- 172.21.15.60:0/3354060858 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311668 (secure 0 0 0) 0x7fbe28005830 con 0x7fbe1806ace0 2023-07-13T22:29:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:03.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe1806ace0 msgr2=0x7fbe1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe1806ace0 0x7fbe1806d190 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbe280010e0 tx=0x7fbe28007c70 comp rx=0 tx=0).stop 2023-07-13T22:29:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d35c0 msgr2=0x7fbe3415db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d35c0 0x7fbe3415db30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c014040 tx=0x7fbe2c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 shutdown_connections 2023-07-13T22:29:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbe1806ace0 0x7fbe1806d190 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe340d35c0 0x7fbe3415db30 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe340b8570 0x7fbe341551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 --2- 172.21.15.60:0/3354060858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe340d13f0 0x7fbe34155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 >> 172.21.15.60:0/3354060858 conn(0x7fbe34071600 msgr2=0x7fbe340ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 shutdown_connections 2023-07-13T22:29:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.648+0000 7fbe3abff700 1 -- 172.21.15.60:0/3354060858 wait complete. 2023-07-13T22:29:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:29:03.645 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2023-07-13T22:29:03.233297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283840,"kb_used_data":10880,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754690880,"statfs":{"total":773094113280,"available":772803461120,"internally_reserved":0,"allocated":11141120,"data_stored":5696432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000978"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715853+0000","last_change":"2023-07-13T22:28:58.553912+0000","last_active":"2023-07-13T22:28:59.715853+0000","last_peered":"2023-07-13T22:28:59.715853+0000","last_clean":"2023-07-13T22:28:59.715853+0000","last_became_active":"2023-07-13T22:28:58.553620+0000","last_became_peered":"2023-07-13T22:28:58.553620+0000","last_unstale":"2023-07-13T22:28:59.715853+0000","last_undegraded":"2023-07-13T22:28:59.715853+0000","last_fullsized":"2023-07-13T22:28:59.715853+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:05:53.160908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716321+0000","last_change":"2023-07-13T22:28:58.552494+0000","last_active":"2023-07-13T22:28:59.716321+0000","last_peered":"2023-07-13T22:28:59.716321+0000","last_clean":"2023-07-13T22:28:59.716321+0000","last_became_active":"2023-07-13T22:28:58.552084+0000","last_became_peered":"2023-07-13T22:28:58.552084+0000","last_unstale":"2023-07-13T22:28:59.716321+0000","last_undegraded":"2023-07-13T22:28:59.716321+0000","last_fullsized":"2023-07-13T22:28:59.716321+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:46.371928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715674+0000","last_change":"2023-07-13T22:28:58.552410+0000","last_active":"2023-07-13T22:28:59.715674+0000","last_peered":"2023-07-13T22:28:59.715674+0000","last_clean":"2023-07-13T22:28:59.715674+0000","last_became_active":"2023-07-13T22:28:58.552130+0000","last_became_peered":"2023-07-13T22:28:58.552130+0000","last_unstale":"2023-07-13T22:28:59.715674+0000","last_undegraded":"2023-07-13T22:28:59.715674+0000","last_fullsized":"2023-07-13T22:28:59.715674+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:49:17.780600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716537+0000","last_change":"2023-07-13T22:28:58.539253+0000","last_active":"2023-07-13T22:28:59.716537+0000","last_peered":"2023-07-13T22:28:59.716537+0000","last_clean":"2023-07-13T22:28:59.716537+0000","last_became_active":"2023-07-13T22:28:58.538985+0000","last_became_peered":"2023-07-13T22:28:58.538985+0000","last_unstale":"2023-07-13T22:28:59.716537+0000","last_undegraded":"2023-07-13T22:28:59.716537+0000","last_fullsized":"2023-07-13T22:28:59.716537+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:19:09.018008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715293+0000","last_change":"2023-07-13T22:28:58.543526+0000","last_active":"2023-07-13T22:28:59.715293+0000","last_peered":"2023-07-13T22:28:59.715293+0000","last_clean":"2023-07-13T22:28:59.715293+0000","last_became_active":"2023-07-13T22:28:58.543019+0000","last_became_peered":"2023-07-13T22:28:58.543019+0000","last_unstale":"2023-07-13T22:28:59.715293+0000","last_undegraded":"2023-07-13T22:28:59.715293+0000","last_fullsized":"2023-07-13T22:28:59.715293+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:31:11.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716210+0000","last_change":"2023-07-13T22:28:58.548347+0000","last_active":"2023-07-13T22:28:59.716210+0000","last_peered":"2023-07-13T22:28:59.716210+0000","last_clean":"2023-07-13T22:28:59.716210+0000","last_became_active":"2023-07-13T22:28:58.548050+0000","last_became_peered":"2023-07-13T22:28:58.548050+0000","last_unstale":"2023-07-13T22:28:59.716210+0000","last_undegraded":"2023-07-13T22:28:59.716210+0000","last_fullsized":"2023-07-13T22:28:59.716210+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:42:19.934988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715811+0000","last_change":"2023-07-13T22:28:58.548993+0000","last_active":"2023-07-13T22:28:59.715811+0000","last_peered":"2023-07-13T22:28:59.715811+0000","last_clean":"2023-07-13T22:28:59.715811+0000","last_became_active":"2023-07-13T22:28:58.548652+0000","last_became_peered":"2023-07-13T22:28:58.548652+0000","last_unstale":"2023-07-13T22:28:59.715811+0000","last_undegraded":"2023-07-13T22:28:59.715811+0000","last_fullsized":"2023-07-13T22:28:59.715811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:52:42.656632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.531770+0000","last_change":"2023-07-13T22:28:58.547977+0000","last_active":"2023-07-13T22:28:59.531770+0000","last_peered":"2023-07-13T22:28:59.531770+0000","last_clean":"2023-07-13T22:28:59.531770+0000","last_became_active":"2023-07-13T22:28:58.547678+0000","last_became_peered":"2023-07-13T22:28:58.547678+0000","last_unstale":"2023-07-13T22:28:59.531770+0000","last_undegraded":"2023-07-13T22:28:59.531770+0000","last_fullsized":"2023-07-13T22:28:59.531770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:49:10.737715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715927+0000","last_change":"2023-07-13T22:28:58.548451+0000","last_active":"2023-07-13T22:28:59.715927+0000","last_peered":"2023-07-13T22:28:59.715927+0000","last_clean":"2023-07-13T22:28:59.715927+0000","last_became_active":"2023-07-13T22:28:58.547192+0000","last_became_peered":"2023-07-13T22:28:58.547192+0000","last_unstale":"2023-07-13T22:28:59.715927+0000","last_undegraded":"2023-07-13T22:28:59.715927+0000","last_fullsized":"2023-07-13T22:28:59.715927+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:15:52.870579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716125+0000","last_change":"2023-07-13T22:28:58.545465+0000","last_active":"2023-07-13T22:28:59.716125+0000","last_peered":"2023-07-13T22:28:59.716125+0000","last_clean":"2023-07-13T22:28:59.716125+0000","last_became_active":"2023-07-13T22:28:58.544088+0000","last_became_peered":"2023-07-13T22:28:58.544088+0000","last_unstale":"2023-07-13T22:28:59.716125+0000","last_undegraded":"2023-07-13T22:28:59.716125+0000","last_fullsized":"2023-07-13T22:28:59.716125+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:11:03.487516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536612+0000","last_change":"2023-07-13T22:28:58.544697+0000","last_active":"2023-07-13T22:28:59.536612+0000","last_peered":"2023-07-13T22:28:59.536612+0000","last_clean":"2023-07-13T22:28:59.536612+0000","last_became_active":"2023-07-13T22:28:58.544418+0000","last_became_peered":"2023-07-13T22:28:58.544418+0000","last_unstale":"2023-07-13T22:28:59.536612+0000","last_undegraded":"2023-07-13T22:28:59.536612+0000","last_fullsized":"2023-07-13T22:28:59.536612+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:02:15.932454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.535914+0000","last_change":"2023-07-13T22:28:58.541908+0000","last_active":"2023-07-13T22:28:59.535914+0000","last_peered":"2023-07-13T22:28:59.535914+0000","last_clean":"2023-07-13T22:28:59.535914+0000","last_became_active":"2023-07-13T22:28:58.541459+0000","last_became_peered":"2023-07-13T22:28:58.541459+0000","last_unstale":"2023-07-13T22:28:59.535914+0000","last_undegraded":"2023-07-13T22:28:59.535914+0000","last_fullsized":"2023-07-13T22:28:59.535914+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:27:21.494226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715803+0000","last_change":"2023-07-13T22:28:58.537045+0000","last_active":"2023-07-13T22:28:59.715803+0000","last_peered":"2023-07-13T22:28:59.715803+0000","last_clean":"2023-07-13T22:28:59.715803+0000","last_became_active":"2023-07-13T22:28:58.536748+0000","last_became_peered":"2023-07-13T22:28:58.536748+0000","last_unstale":"2023-07-13T22:28:59.715803+0000","last_undegraded":"2023-07-13T22:28:59.715803+0000","last_fullsized":"2023-07-13T22:28:59.715803+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:22:00.478301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715647+0000","last_change":"2023-07-13T22:28:58.547142+0000","last_active":"2023-07-13T22:28:59.715647+0000","last_peered":"2023-07-13T22:28:59.715647+0000","last_clean":"2023-07-13T22:28:59.715647+0000","last_became_active":"2023-07-13T22:28:58.546848+0000","last_became_peered":"2023-07-13T22:28:58.546848+0000","last_unstale":"2023-07-13T22:28:59.715647+0000","last_undegraded":"2023-07-13T22:28:59.715647+0000","last_fullsized":"2023-07-13T22:28:59.715647+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:56.014087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715572+0000","last_change":"2023-07-13T22:28:58.552514+0000","last_active":"2023-07-13T22:28:59.715572+0000","last_peered":"2023-07-13T22:28:59.715572+0000","last_clean":"2023-07-13T22:28:59.715572+0000","last_became_active":"2023-07-13T22:28:58.551968+0000","last_became_peered":"2023-07-13T22:28:58.551968+0000","last_unstale":"2023-07-13T22:28:59.715572+0000","last_undegraded":"2023-07-13T22:28:59.715572+0000","last_fullsized":"2023-07-13T22:28:59.715572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:05:02.287544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715326+0000","last_change":"2023-07-13T22:28:58.541838+0000","last_active":"2023-07-13T22:28:59.715326+0000","last_peered":"2023-07-13T22:28:59.715326+0000","last_clean":"2023-07-13T22:28:59.715326+0000","last_became_active":"2023-07-13T22:28:58.541508+0000","last_became_peered":"2023-07-13T22:28:58.541508+0000","last_unstale":"2023-07-13T22:28:59.715326+0000","last_undegraded":"2023-07-13T22:28:59.715326+0000","last_fullsized":"2023-07-13T22:28:59.715326+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:51:22.448630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715893+0000","last_change":"2023-07-13T22:28:58.552911+0000","last_active":"2023-07-13T22:28:59.715893+0000","last_peered":"2023-07-13T22:28:59.715893+0000","last_clean":"2023-07-13T22:28:59.715893+0000","last_became_active":"2023-07-13T22:28:58.552425+0000","last_became_peered":"2023-07-13T22:28:58.552425+0000","last_unstale":"2023-07-13T22:28:59.715893+0000","last_undegraded":"2023-07-13T22:28:59.715893+0000","last_fullsized":"2023-07-13T22:28:59.715893+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:03:41.678511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715989+0000","last_change":"2023-07-13T22:28:58.549494+0000","last_active":"2023-07-13T22:28:59.715989+0000","last_peered":"2023-07-13T22:28:59.715989+0000","last_clean":"2023-07-13T22:28:59.715989+0000","last_became_active":"2023-07-13T22:28:58.549231+0000","last_became_peered":"2023-07-13T22:28:58.549231+0000","last_unstale":"2023-07-13T22:28:59.715989+0000","last_undegraded":"2023-07-13T22:28:59.715989+0000","last_fullsized":"2023-07-13T22:28:59.715989+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:19.265737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716231+0000","last_change":"2023-07-13T22:28:58.553411+0000","last_active":"2023-07-13T22:28:59.716231+0000","last_peered":"2023-07-13T22:28:59.716231+0000","last_clean":"2023-07-13T22:28:59.716231+0000","last_became_active":"2023-07-13T22:28:58.553145+0000","last_became_peered":"2023-07-13T22:28:58.553145+0000","last_unstale":"2023-07-13T22:28:59.716231+0000","last_undegraded":"2023-07-13T22:28:59.716231+0000","last_fullsized":"2023-07-13T22:28:59.716231+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:39:11.370144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536085+0000","last_change":"2023-07-13T22:28:58.543086+0000","last_active":"2023-07-13T22:28:59.536085+0000","last_peered":"2023-07-13T22:28:59.536085+0000","last_clean":"2023-07-13T22:28:59.536085+0000","last_became_active":"2023-07-13T22:28:58.542827+0000","last_became_peered":"2023-07-13T22:28:58.542827+0000","last_unstale":"2023-07-13T22:28:59.536085+0000","last_undegraded":"2023-07-13T22:28:59.536085+0000","last_fullsized":"2023-07-13T22:28:59.536085+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:22:00.478301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532099+0000","last_change":"2023-07-13T22:28:58.539131+0000","last_active":"2023-07-13T22:28:59.532099+0000","last_peered":"2023-07-13T22:28:59.532099+0000","last_clean":"2023-07-13T22:28:59.532099+0000","last_became_active":"2023-07-13T22:28:58.538878+0000","last_became_peered":"2023-07-13T22:28:58.538878+0000","last_unstale":"2023-07-13T22:28:59.532099+0000","last_undegraded":"2023-07-13T22:28:59.532099+0000","last_fullsized":"2023-07-13T22:28:59.532099+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:21:21.668239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715859+0000","last_change":"2023-07-13T22:28:58.543072+0000","last_active":"2023-07-13T22:28:59.715859+0000","last_peered":"2023-07-13T22:28:59.715859+0000","last_clean":"2023-07-13T22:28:59.715859+0000","last_became_active":"2023-07-13T22:28:58.542503+0000","last_became_peered":"2023-07-13T22:28:58.542503+0000","last_unstale":"2023-07-13T22:28:59.715859+0000","last_undegraded":"2023-07-13T22:28:59.715859+0000","last_fullsized":"2023-07-13T22:28:59.715859+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:32:16.498638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715494+0000","last_change":"2023-07-13T22:28:58.552261+0000","last_active":"2023-07-13T22:28:59.715494+0000","last_peered":"2023-07-13T22:28:59.715494+0000","last_clean":"2023-07-13T22:28:59.715494+0000","last_became_active":"2023-07-13T22:28:58.551990+0000","last_became_peered":"2023-07-13T22:28:58.551990+0000","last_unstale":"2023-07-13T22:28:59.715494+0000","last_undegraded":"2023-07-13T22:28:59.715494+0000","last_fullsized":"2023-07-13T22:28:59.715494+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:40:16.842515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.714952+0000","last_change":"2023-07-13T22:28:58.556163+0000","last_active":"2023-07-13T22:28:59.714952+0000","last_peered":"2023-07-13T22:28:59.714952+0000","last_clean":"2023-07-13T22:28:59.714952+0000","last_became_active":"2023-07-13T22:28:58.555566+0000","last_became_peered":"2023-07-13T22:28:58.555566+0000","last_unstale":"2023-07-13T22:28:59.714952+0000","last_undegraded":"2023-07-13T22:28:59.714952+0000","last_fullsized":"2023-07-13T22:28:59.714952+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:47:17.723610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536344+0000","last_change":"2023-07-13T22:28:58.544683+0000","last_active":"2023-07-13T22:28:59.536344+0000","last_peered":"2023-07-13T22:28:59.536344+0000","last_clean":"2023-07-13T22:28:59.536344+0000","last_became_active":"2023-07-13T22:28:58.544357+0000","last_became_peered":"2023-07-13T22:28:58.544357+0000","last_unstale":"2023-07-13T22:28:59.536344+0000","last_undegraded":"2023-07-13T22:28:59.536344+0000","last_fullsized":"2023-07-13T22:28:59.536344+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:49:28.410430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536309+0000","last_change":"2023-07-13T22:28:58.550587+0000","last_active":"2023-07-13T22:28:59.536309+0000","last_peered":"2023-07-13T22:28:59.536309+0000","last_clean":"2023-07-13T22:28:59.536309+0000","last_became_active":"2023-07-13T22:28:58.550273+0000","last_became_peered":"2023-07-13T22:28:58.550273+0000","last_unstale":"2023-07-13T22:28:59.536309+0000","last_undegraded":"2023-07-13T22:28:59.536309+0000","last_fullsized":"2023-07-13T22:28:59.536309+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:55:44.302181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536760+0000","last_change":"2023-07-13T22:28:58.551185+0000","last_active":"2023-07-13T22:28:59.536760+0000","last_peered":"2023-07-13T22:28:59.536760+0000","last_clean":"2023-07-13T22:28:59.536760+0000","last_became_active":"2023-07-13T22:28:58.550859+0000","last_became_peered":"2023-07-13T22:28:58.550859+0000","last_unstale":"2023-07-13T22:28:59.536760+0000","last_undegraded":"2023-07-13T22:28:59.536760+0000","last_fullsized":"2023-07-13T22:28:59.536760+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:52:42.656632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716378+0000","last_change":"2023-07-13T22:28:58.545806+0000","last_active":"2023-07-13T22:28:59.716378+0000","last_peered":"2023-07-13T22:28:59.716378+0000","last_clean":"2023-07-13T22:28:59.716378+0000","last_became_active":"2023-07-13T22:28:58.545519+0000","last_became_peered":"2023-07-13T22:28:58.545519+0000","last_unstale":"2023-07-13T22:28:59.716378+0000","last_undegraded":"2023-07-13T22:28:59.716378+0000","last_fullsized":"2023-07-13T22:28:59.716378+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:21:20.969177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532302+0000","last_change":"2023-07-13T22:28:58.545809+0000","last_active":"2023-07-13T22:28:59.532302+0000","last_peered":"2023-07-13T22:28:59.532302+0000","last_clean":"2023-07-13T22:28:59.532302+0000","last_became_active":"2023-07-13T22:28:58.545520+0000","last_became_peered":"2023-07-13T22:28:58.545520+0000","last_unstale":"2023-07-13T22:28:59.532302+0000","last_undegraded":"2023-07-13T22:28:59.532302+0000","last_fullsized":"2023-07-13T22:28:59.532302+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:08:18.836615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536680+0000","last_change":"2023-07-13T22:28:58.545216+0000","last_active":"2023-07-13T22:28:59.536680+0000","last_peered":"2023-07-13T22:28:59.536680+0000","last_clean":"2023-07-13T22:28:59.536680+0000","last_became_active":"2023-07-13T22:28:58.544930+0000","last_became_peered":"2023-07-13T22:28:58.544930+0000","last_unstale":"2023-07-13T22:28:59.536680+0000","last_undegraded":"2023-07-13T22:28:59.536680+0000","last_fullsized":"2023-07-13T22:28:59.536680+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:44:51.798335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716345+0000","last_change":"2023-07-13T22:28:58.554972+0000","last_active":"2023-07-13T22:28:59.716345+0000","last_peered":"2023-07-13T22:28:59.716345+0000","last_clean":"2023-07-13T22:28:59.716345+0000","last_became_active":"2023-07-13T22:28:58.554700+0000","last_became_peered":"2023-07-13T22:28:58.554700+0000","last_unstale":"2023-07-13T22:28:59.716345+0000","last_undegraded":"2023-07-13T22:28:59.716345+0000","last_fullsized":"2023-07-13T22:28:59.716345+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:23:36.707294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.531841+0000","last_change":"2023-07-13T22:28:58.545894+0000","last_active":"2023-07-13T22:28:59.531841+0000","last_peered":"2023-07-13T22:28:59.531841+0000","last_clean":"2023-07-13T22:28:59.531841+0000","last_became_active":"2023-07-13T22:28:58.545346+0000","last_became_peered":"2023-07-13T22:28:58.545346+0000","last_unstale":"2023-07-13T22:28:59.531841+0000","last_undegraded":"2023-07-13T22:28:59.531841+0000","last_fullsized":"2023-07-13T22:28:59.531841+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:43:08.672327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715516+0000","last_change":"2023-07-13T22:28:58.554848+0000","last_active":"2023-07-13T22:28:59.715516+0000","last_peered":"2023-07-13T22:28:59.715516+0000","last_clean":"2023-07-13T22:28:59.715516+0000","last_became_active":"2023-07-13T22:28:58.554354+0000","last_became_peered":"2023-07-13T22:28:58.554354+0000","last_unstale":"2023-07-13T22:28:59.715516+0000","last_undegraded":"2023-07-13T22:28:59.715516+0000","last_fullsized":"2023-07-13T22:28:59.715516+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:42:27.548103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"99'4","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716209+0000","last_change":"2023-07-13T22:28:57.748310+0000","last_active":"2023-07-13T22:28:59.716209+0000","last_peered":"2023-07-13T22:28:59.716209+0000","last_clean":"2023-07-13T22:28:59.716209+0000","last_became_active":"2023-07-13T22:28:57.748010+0000","last_became_peered":"2023-07-13T22:28:57.748010+0000","last_unstale":"2023-07-13T22:28:59.716209+0000","last_undegraded":"2023-07-13T22:28:59.716209+0000","last_fullsized":"2023-07-13T22:28:59.716209+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:55:43.293790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532247+0000","last_change":"2023-07-13T22:28:58.538438+0000","last_active":"2023-07-13T22:28:59.532247+0000","last_peered":"2023-07-13T22:28:59.532247+0000","last_clean":"2023-07-13T22:28:59.532247+0000","last_became_active":"2023-07-13T22:28:58.538149+0000","last_became_peered":"2023-07-13T22:28:58.538149+0000","last_unstale":"2023-07-13T22:28:59.532247+0000","last_undegraded":"2023-07-13T22:28:59.532247+0000","last_fullsized":"2023-07-13T22:28:59.532247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:47:59.121441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532295+0000","last_change":"2023-07-13T22:28:57.749797+0000","last_active":"2023-07-13T22:28:59.532295+0000","last_peered":"2023-07-13T22:28:59.532295+0000","last_clean":"2023-07-13T22:28:59.532295+0000","last_became_active":"2023-07-13T22:28:57.749471+0000","last_became_peered":"2023-07-13T22:28:57.749471+0000","last_unstale":"2023-07-13T22:28:59.532295+0000","last_undegraded":"2023-07-13T22:28:59.532295+0000","last_fullsized":"2023-07-13T22:28:59.532295+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:06:03.252206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536938+0000","last_change":"2023-07-13T22:28:58.545249+0000","last_active":"2023-07-13T22:28:59.536938+0000","last_peered":"2023-07-13T22:28:59.536938+0000","last_clean":"2023-07-13T22:28:59.536938+0000","last_became_active":"2023-07-13T22:28:58.544967+0000","last_became_peered":"2023-07-13T22:28:58.544967+0000","last_unstale":"2023-07-13T22:28:59.536938+0000","last_undegraded":"2023-07-13T22:28:59.536938+0000","last_fullsized":"2023-07-13T22:28:59.536938+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:07:50.190158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536409+0000","last_change":"2023-07-13T22:28:57.751001+0000","last_active":"2023-07-13T22:28:59.536409+0000","last_peered":"2023-07-13T22:28:59.536409+0000","last_clean":"2023-07-13T22:28:59.536409+0000","last_became_active":"2023-07-13T22:28:57.750656+0000","last_became_peered":"2023-07-13T22:28:57.750656+0000","last_unstale":"2023-07-13T22:28:59.536409+0000","last_undegraded":"2023-07-13T22:28:59.536409+0000","last_fullsized":"2023-07-13T22:28:59.536409+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:27.543987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.537258+0000","last_change":"2023-07-13T22:28:58.538510+0000","last_active":"2023-07-13T22:28:59.537258+0000","last_peered":"2023-07-13T22:28:59.537258+0000","last_clean":"2023-07-13T22:28:59.537258+0000","last_became_active":"2023-07-13T22:28:58.538250+0000","last_became_peered":"2023-07-13T22:28:58.538250+0000","last_unstale":"2023-07-13T22:28:59.537258+0000","last_undegraded":"2023-07-13T22:28:59.537258+0000","last_fullsized":"2023-07-13T22:28:59.537258+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:56:31.006623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716206+0000","last_change":"2023-07-13T22:28:57.747981+0000","last_active":"2023-07-13T22:28:59.716206+0000","last_peered":"2023-07-13T22:28:59.716206+0000","last_clean":"2023-07-13T22:28:59.716206+0000","last_became_active":"2023-07-13T22:28:57.747718+0000","last_became_peered":"2023-07-13T22:28:57.747718+0000","last_unstale":"2023-07-13T22:28:59.716206+0000","last_undegraded":"2023-07-13T22:28:59.716206+0000","last_fullsized":"2023-07-13T22:28:59.716206+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:19:10.815922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716668+0000","last_change":"2023-07-13T22:28:58.537072+0000","last_active":"2023-07-13T22:28:59.716668+0000","last_peered":"2023-07-13T22:28:59.716668+0000","last_clean":"2023-07-13T22:28:59.716668+0000","last_became_active":"2023-07-13T22:28:58.536807+0000","last_became_peered":"2023-07-13T22:28:58.536807+0000","last_unstale":"2023-07-13T22:28:59.716668+0000","last_undegraded":"2023-07-13T22:28:59.716668+0000","last_fullsized":"2023-07-13T22:28:59.716668+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:07:50.190158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716417+0000","last_change":"2023-07-13T22:28:57.748166+0000","last_active":"2023-07-13T22:28:59.716417+0000","last_peered":"2023-07-13T22:28:59.716417+0000","last_clean":"2023-07-13T22:28:59.716417+0000","last_became_active":"2023-07-13T22:28:57.747879+0000","last_became_peered":"2023-07-13T22:28:57.747879+0000","last_unstale":"2023-07-13T22:28:59.716417+0000","last_undegraded":"2023-07-13T22:28:59.716417+0000","last_fullsized":"2023-07-13T22:28:59.716417+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:43:38.103175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716024+0000","last_change":"2023-07-13T22:28:58.554841+0000","last_active":"2023-07-13T22:28:59.716024+0000","last_peered":"2023-07-13T22:28:59.716024+0000","last_clean":"2023-07-13T22:28:59.716024+0000","last_became_active":"2023-07-13T22:28:58.554547+0000","last_became_peered":"2023-07-13T22:28:58.554547+0000","last_unstale":"2023-07-13T22:28:59.716024+0000","last_undegraded":"2023-07-13T22:28:59.716024+0000","last_fullsized":"2023-07-13T22:28:59.716024+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:21:35.914234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.537249+0000","last_change":"2023-07-13T22:28:57.748465+0000","last_active":"2023-07-13T22:28:59.537249+0000","last_peered":"2023-07-13T22:28:59.537249+0000","last_clean":"2023-07-13T22:28:59.537249+0000","last_became_active":"2023-07-13T22:28:57.748180+0000","last_became_peered":"2023-07-13T22:28:57.748180+0000","last_unstale":"2023-07-13T22:28:59.537249+0000","last_undegraded":"2023-07-13T22:28:59.537249+0000","last_fullsized":"2023-07-13T22:28:59.537249+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:47:07.889223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716209+0000","last_change":"2023-07-13T22:28:58.552569+0000","last_active":"2023-07-13T22:28:59.716209+0000","last_peered":"2023-07-13T22:28:59.716209+0000","last_clean":"2023-07-13T22:28:59.716209+0000","last_became_active":"2023-07-13T22:28:58.551627+0000","last_became_peered":"2023-07-13T22:28:58.551627+0000","last_unstale":"2023-07-13T22:28:59.716209+0000","last_undegraded":"2023-07-13T22:28:59.716209+0000","last_fullsized":"2023-07-13T22:28:59.716209+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:26:13.121237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536045+0000","last_change":"2023-07-13T22:28:57.745164+0000","last_active":"2023-07-13T22:28:59.536045+0000","last_peered":"2023-07-13T22:28:59.536045+0000","last_clean":"2023-07-13T22:28:59.536045+0000","last_became_active":"2023-07-13T22:28:57.744870+0000","last_became_peered":"2023-07-13T22:28:57.744870+0000","last_unstale":"2023-07-13T22:28:59.536045+0000","last_undegraded":"2023-07-13T22:28:59.536045+0000","last_fullsized":"2023-07-13T22:28:59.536045+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:58:27.895772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716355+0000","last_change":"2023-07-13T22:28:58.554952+0000","last_active":"2023-07-13T22:28:59.716355+0000","last_peered":"2023-07-13T22:28:59.716355+0000","last_clean":"2023-07-13T22:28:59.716355+0000","last_became_active":"2023-07-13T22:28:58.554701+0000","last_became_peered":"2023-07-13T22:28:58.554701+0000","last_unstale":"2023-07-13T22:28:59.716355+0000","last_undegraded":"2023-07-13T22:28:59.716355+0000","last_fullsized":"2023-07-13T22:28:59.716355+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:28.552377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716269+0000","last_change":"2023-07-13T22:28:57.748720+0000","last_active":"2023-07-13T22:28:59.716269+0000","last_peered":"2023-07-13T22:28:59.716269+0000","last_clean":"2023-07-13T22:28:59.716269+0000","last_became_active":"2023-07-13T22:28:57.748479+0000","last_became_peered":"2023-07-13T22:28:57.748479+0000","last_unstale":"2023-07-13T22:28:59.716269+0000","last_undegraded":"2023-07-13T22:28:59.716269+0000","last_fullsized":"2023-07-13T22:28:59.716269+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:49:27.402040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715376+0000","last_change":"2023-07-13T22:28:58.551146+0000","last_active":"2023-07-13T22:28:59.715376+0000","last_peered":"2023-07-13T22:28:59.715376+0000","last_clean":"2023-07-13T22:28:59.715376+0000","last_became_active":"2023-07-13T22:28:58.550826+0000","last_became_peered":"2023-07-13T22:28:58.550826+0000","last_unstale":"2023-07-13T22:28:59.715376+0000","last_undegraded":"2023-07-13T22:28:59.715376+0000","last_fullsized":"2023-07-13T22:28:59.715376+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:18:46.349332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536444+0000","last_change":"2023-07-13T22:28:57.746760+0000","last_active":"2023-07-13T22:28:59.536444+0000","last_peered":"2023-07-13T22:28:59.536444+0000","last_clean":"2023-07-13T22:28:59.536444+0000","last_became_active":"2023-07-13T22:28:57.746294+0000","last_became_peered":"2023-07-13T22:28:57.746294+0000","last_unstale":"2023-07-13T22:28:59.536444+0000","last_undegraded":"2023-07-13T22:28:59.536444+0000","last_fullsized":"2023-07-13T22:28:59.536444+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:02.024613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715175+0000","last_change":"2023-07-13T22:28:58.540305+0000","last_active":"2023-07-13T22:28:59.715175+0000","last_peered":"2023-07-13T22:28:59.715175+0000","last_clean":"2023-07-13T22:28:59.715175+0000","last_became_active":"2023-07-13T22:28:58.540048+0000","last_became_peered":"2023-07-13T22:28:58.540048+0000","last_unstale":"2023-07-13T22:28:59.715175+0000","last_undegraded":"2023-07-13T22:28:59.715175+0000","last_fullsized":"2023-07-13T22:28:59.715175+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:15:21.716284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715134+0000","last_change":"2023-07-13T22:28:57.752044+0000","last_active":"2023-07-13T22:28:59.715134+0000","last_peered":"2023-07-13T22:28:59.715134+0000","last_clean":"2023-07-13T22:28:59.715134+0000","last_became_active":"2023-07-13T22:28:57.751728+0000","last_became_peered":"2023-07-13T22:28:57.751728+0000","last_unstale":"2023-07-13T22:28:59.715134+0000","last_undegraded":"2023-07-13T22:28:59.715134+0000","last_fullsized":"2023-07-13T22:28:59.715134+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:53:51.841040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.531883+0000","last_change":"2023-07-13T22:28:58.540564+0000","last_active":"2023-07-13T22:28:59.531883+0000","last_peered":"2023-07-13T22:28:59.531883+0000","last_clean":"2023-07-13T22:28:59.531883+0000","last_became_active":"2023-07-13T22:28:58.540259+0000","last_became_peered":"2023-07-13T22:28:58.540259+0000","last_unstale":"2023-07-13T22:28:59.531883+0000","last_undegraded":"2023-07-13T22:28:59.531883+0000","last_fullsized":"2023-07-13T22:28:59.531883+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:24:36.668248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"99'3","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715503+0000","last_change":"2023-07-13T22:28:57.752212+0000","last_active":"2023-07-13T22:28:59.715503+0000","last_peered":"2023-07-13T22:28:59.715503+0000","last_clean":"2023-07-13T22:28:59.715503+0000","last_became_active":"2023-07-13T22:28:57.751862+0000","last_became_peered":"2023-07-13T22:28:57.751862+0000","last_unstale":"2023-07-13T22:28:59.715503+0000","last_undegraded":"2023-07-13T22:28:59.715503+0000","last_fullsized":"2023-07-13T22:28:59.715503+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:03:03.990850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715568+0000","last_change":"2023-07-13T22:28:58.544002+0000","last_active":"2023-07-13T22:28:59.715568+0000","last_peered":"2023-07-13T22:28:59.715568+0000","last_clean":"2023-07-13T22:28:59.715568+0000","last_became_active":"2023-07-13T22:28:58.543663+0000","last_became_peered":"2023-07-13T22:28:58.543663+0000","last_unstale":"2023-07-13T22:28:59.715568+0000","last_undegraded":"2023-07-13T22:28:59.715568+0000","last_fullsized":"2023-07-13T22:28:59.715568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:37:30.207709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"99'6","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716362+0000","last_change":"2023-07-13T22:28:57.753724+0000","last_active":"2023-07-13T22:28:59.716362+0000","last_peered":"2023-07-13T22:28:59.716362+0000","last_clean":"2023-07-13T22:28:59.716362+0000","last_became_active":"2023-07-13T22:28:57.753441+0000","last_became_peered":"2023-07-13T22:28:57.753441+0000","last_unstale":"2023-07-13T22:28:59.716362+0000","last_undegraded":"2023-07-13T22:28:59.716362+0000","last_fullsized":"2023-07-13T22:28:59.716362+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:58:37.322426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536219+0000","last_change":"2023-07-13T22:28:58.554499+0000","last_active":"2023-07-13T22:28:59.536219+0000","last_peered":"2023-07-13T22:28:59.536219+0000","last_clean":"2023-07-13T22:28:59.536219+0000","last_became_active":"2023-07-13T22:28:58.554152+0000","last_became_peered":"2023-07-13T22:28:58.554152+0000","last_unstale":"2023-07-13T22:28:59.536219+0000","last_undegraded":"2023-07-13T22:28:59.536219+0000","last_fullsized":"2023-07-13T22:28:59.536219+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:11:03.487516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715773+0000","last_change":"2023-07-13T22:28:57.751210+0000","last_active":"2023-07-13T22:28:59.715773+0000","last_peered":"2023-07-13T22:28:59.715773+0000","last_clean":"2023-07-13T22:28:59.715773+0000","last_became_active":"2023-07-13T22:28:57.750907+0000","last_became_peered":"2023-07-13T22:28:57.750907+0000","last_unstale":"2023-07-13T22:28:59.715773+0000","last_undegraded":"2023-07-13T22:28:59.715773+0000","last_fullsized":"2023-07-13T22:28:59.715773+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:08:05.080959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715806+0000","last_change":"2023-07-13T22:28:58.546202+0000","last_active":"2023-07-13T22:28:59.715806+0000","last_peered":"2023-07-13T22:28:59.715806+0000","last_clean":"2023-07-13T22:28:59.715806+0000","last_became_active":"2023-07-13T22:28:58.545888+0000","last_became_peered":"2023-07-13T22:28:58.545888+0000","last_unstale":"2023-07-13T22:28:59.715806+0000","last_undegraded":"2023-07-13T22:28:59.715806+0000","last_fullsized":"2023-07-13T22:28:59.715806+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:02:15.932454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716555+0000","last_change":"2023-07-13T22:28:57.747068+0000","last_active":"2023-07-13T22:28:59.716555+0000","last_peered":"2023-07-13T22:28:59.716555+0000","last_clean":"2023-07-13T22:28:59.716555+0000","last_became_active":"2023-07-13T22:28:57.746623+0000","last_became_peered":"2023-07-13T22:28:57.746623+0000","last_unstale":"2023-07-13T22:28:59.716555+0000","last_undegraded":"2023-07-13T22:28:59.716555+0000","last_fullsized":"2023-07-13T22:28:59.716555+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:44:50.789944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532134+0000","last_change":"2023-07-13T22:28:58.545034+0000","last_active":"2023-07-13T22:28:59.532134+0000","last_peered":"2023-07-13T22:28:59.532134+0000","last_clean":"2023-07-13T22:28:59.532134+0000","last_became_active":"2023-07-13T22:28:58.544747+0000","last_became_peered":"2023-07-13T22:28:58.544747+0000","last_unstale":"2023-07-13T22:28:59.532134+0000","last_undegraded":"2023-07-13T22:28:59.532134+0000","last_fullsized":"2023-07-13T22:28:59.532134+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:21:22.001418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.531838+0000","last_change":"2023-07-13T22:28:57.746092+0000","last_active":"2023-07-13T22:28:59.531838+0000","last_peered":"2023-07-13T22:28:59.531838+0000","last_clean":"2023-07-13T22:28:59.531838+0000","last_became_active":"2023-07-13T22:28:57.745720+0000","last_became_peered":"2023-07-13T22:28:57.745720+0000","last_unstale":"2023-07-13T22:28:59.531838+0000","last_undegraded":"2023-07-13T22:28:59.531838+0000","last_fullsized":"2023-07-13T22:28:59.531838+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:30:34.594906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715993+0000","last_change":"2023-07-13T22:28:58.552447+0000","last_active":"2023-07-13T22:28:59.715993+0000","last_peered":"2023-07-13T22:28:59.715993+0000","last_clean":"2023-07-13T22:28:59.715993+0000","last_became_active":"2023-07-13T22:28:58.552042+0000","last_became_peered":"2023-07-13T22:28:58.552042+0000","last_unstale":"2023-07-13T22:28:59.715993+0000","last_undegraded":"2023-07-13T22:28:59.715993+0000","last_fullsized":"2023-07-13T22:28:59.715993+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:42:50.657112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"99'8","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.535796+0000","last_change":"2023-07-13T22:28:57.746003+0000","last_active":"2023-07-13T22:28:59.535796+0000","last_peered":"2023-07-13T22:28:59.535796+0000","last_clean":"2023-07-13T22:28:59.535796+0000","last_became_active":"2023-07-13T22:28:57.745703+0000","last_became_peered":"2023-07-13T22:28:57.745703+0000","last_unstale":"2023-07-13T22:28:59.535796+0000","last_undegraded":"2023-07-13T22:28:59.535796+0000","last_fullsized":"2023-07-13T22:28:59.535796+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:30:54.612388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715850+0000","last_change":"2023-07-13T22:28:58.550536+0000","last_active":"2023-07-13T22:28:59.715850+0000","last_peered":"2023-07-13T22:28:59.715850+0000","last_clean":"2023-07-13T22:28:59.715850+0000","last_became_active":"2023-07-13T22:28:58.550179+0000","last_became_peered":"2023-07-13T22:28:58.550179+0000","last_unstale":"2023-07-13T22:28:59.715850+0000","last_undegraded":"2023-07-13T22:28:59.715850+0000","last_fullsized":"2023-07-13T22:28:59.715850+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:52:36.556018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716354+0000","last_change":"2023-07-13T22:28:57.750619+0000","last_active":"2023-07-13T22:28:59.716354+0000","last_peered":"2023-07-13T22:28:59.716354+0000","last_clean":"2023-07-13T22:28:59.716354+0000","last_became_active":"2023-07-13T22:28:57.750317+0000","last_became_peered":"2023-07-13T22:28:57.750317+0000","last_unstale":"2023-07-13T22:28:59.716354+0000","last_undegraded":"2023-07-13T22:28:59.716354+0000","last_fullsized":"2023-07-13T22:28:59.716354+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:52:41.648242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.535966+0000","last_change":"2023-07-13T22:28:58.544480+0000","last_active":"2023-07-13T22:28:59.535966+0000","last_peered":"2023-07-13T22:28:59.535966+0000","last_clean":"2023-07-13T22:28:59.535966+0000","last_became_active":"2023-07-13T22:28:58.544208+0000","last_became_peered":"2023-07-13T22:28:58.544208+0000","last_unstale":"2023-07-13T22:28:59.535966+0000","last_undegraded":"2023-07-13T22:28:59.535966+0000","last_fullsized":"2023-07-13T22:28:59.535966+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:44:17.691898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715665+0000","last_change":"2023-07-13T22:28:57.740233+0000","last_active":"2023-07-13T22:28:59.715665+0000","last_peered":"2023-07-13T22:28:59.715665+0000","last_clean":"2023-07-13T22:28:59.715665+0000","last_became_active":"2023-07-13T22:28:57.739916+0000","last_became_peered":"2023-07-13T22:28:57.739916+0000","last_unstale":"2023-07-13T22:28:59.715665+0000","last_undegraded":"2023-07-13T22:28:59.715665+0000","last_fullsized":"2023-07-13T22:28:59.715665+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:12.720742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716485+0000","last_change":"2023-07-13T22:28:58.544899+0000","last_active":"2023-07-13T22:28:59.716485+0000","last_peered":"2023-07-13T22:28:59.716485+0000","last_clean":"2023-07-13T22:28:59.716485+0000","last_became_active":"2023-07-13T22:28:58.544631+0000","last_became_peered":"2023-07-13T22:28:58.544631+0000","last_unstale":"2023-07-13T22:28:59.716485+0000","last_undegraded":"2023-07-13T22:28:59.716485+0000","last_fullsized":"2023-07-13T22:28:59.716485+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:03.033004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715983+0000","last_change":"2023-07-13T22:28:57.753316+0000","last_active":"2023-07-13T22:28:59.715983+0000","last_peered":"2023-07-13T22:28:59.715983+0000","last_clean":"2023-07-13T22:28:59.715983+0000","last_became_active":"2023-07-13T22:28:57.753032+0000","last_became_peered":"2023-07-13T22:28:57.753032+0000","last_unstale":"2023-07-13T22:28:59.715983+0000","last_undegraded":"2023-07-13T22:28:59.715983+0000","last_fullsized":"2023-07-13T22:28:59.715983+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:32:25.723682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.537169+0000","last_change":"2023-07-13T22:28:58.540795+0000","last_active":"2023-07-13T22:28:59.537169+0000","last_peered":"2023-07-13T22:28:59.537169+0000","last_clean":"2023-07-13T22:28:59.537169+0000","last_became_active":"2023-07-13T22:28:58.540507+0000","last_became_peered":"2023-07-13T22:28:58.540507+0000","last_unstale":"2023-07-13T22:28:59.537169+0000","last_undegraded":"2023-07-13T22:28:59.537169+0000","last_fullsized":"2023-07-13T22:28:59.537169+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:19.265737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716252+0000","last_change":"2023-07-13T22:28:57.747162+0000","last_active":"2023-07-13T22:28:59.716252+0000","last_peered":"2023-07-13T22:28:59.716252+0000","last_clean":"2023-07-13T22:28:59.716252+0000","last_became_active":"2023-07-13T22:28:57.746899+0000","last_became_peered":"2023-07-13T22:28:57.746899+0000","last_unstale":"2023-07-13T22:28:59.716252+0000","last_undegraded":"2023-07-13T22:28:59.716252+0000","last_fullsized":"2023-07-13T22:28:59.716252+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:02:14.924064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532115+0000","last_change":"2023-07-13T22:28:58.544966+0000","last_active":"2023-07-13T22:28:59.532115+0000","last_peered":"2023-07-13T22:28:59.532115+0000","last_clean":"2023-07-13T22:28:59.532115+0000","last_became_active":"2023-07-13T22:28:58.544676+0000","last_became_peered":"2023-07-13T22:28:58.544676+0000","last_unstale":"2023-07-13T22:28:59.532115+0000","last_undegraded":"2023-07-13T22:28:59.532115+0000","last_fullsized":"2023-07-13T22:28:59.532115+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:15:20.808301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536211+0000","last_change":"2023-07-13T22:28:57.747689+0000","last_active":"2023-07-13T22:28:59.536211+0000","last_peered":"2023-07-13T22:28:59.536211+0000","last_clean":"2023-07-13T22:28:59.536211+0000","last_became_active":"2023-07-13T22:28:57.747416+0000","last_became_peered":"2023-07-13T22:28:57.747416+0000","last_unstale":"2023-07-13T22:28:59.536211+0000","last_undegraded":"2023-07-13T22:28:59.536211+0000","last_fullsized":"2023-07-13T22:28:59.536211+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:57:38.895257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532182+0000","last_change":"2023-07-13T22:28:58.547382+0000","last_active":"2023-07-13T22:28:59.532182+0000","last_peered":"2023-07-13T22:28:59.532182+0000","last_clean":"2023-07-13T22:28:59.532182+0000","last_became_active":"2023-07-13T22:28:58.547113+0000","last_became_peered":"2023-07-13T22:28:58.547113+0000","last_unstale":"2023-07-13T22:28:59.532182+0000","last_undegraded":"2023-07-13T22:28:59.532182+0000","last_fullsized":"2023-07-13T22:28:59.532182+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:04:23.788857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532006+0000","last_change":"2023-07-13T22:28:57.748907+0000","last_active":"2023-07-13T22:28:59.532006+0000","last_peered":"2023-07-13T22:28:59.532006+0000","last_clean":"2023-07-13T22:28:59.532006+0000","last_became_active":"2023-07-13T22:28:57.748591+0000","last_became_peered":"2023-07-13T22:28:57.748591+0000","last_unstale":"2023-07-13T22:28:59.532006+0000","last_undegraded":"2023-07-13T22:28:59.532006+0000","last_fullsized":"2023-07-13T22:28:59.532006+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:05:00.622595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716342+0000","last_change":"2023-07-13T22:28:58.549202+0000","last_active":"2023-07-13T22:28:59.716342+0000","last_peered":"2023-07-13T22:28:59.716342+0000","last_clean":"2023-07-13T22:28:59.716342+0000","last_became_active":"2023-07-13T22:28:58.548849+0000","last_became_peered":"2023-07-13T22:28:58.548849+0000","last_unstale":"2023-07-13T22:28:59.716342+0000","last_undegraded":"2023-07-13T22:28:59.716342+0000","last_fullsized":"2023-07-13T22:28:59.716342+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:21:58.085927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716262+0000","last_change":"2023-07-13T22:28:57.746948+0000","last_active":"2023-07-13T22:28:59.716262+0000","last_peered":"2023-07-13T22:28:59.716262+0000","last_clean":"2023-07-13T22:28:59.716262+0000","last_became_active":"2023-07-13T22:28:57.746673+0000","last_became_peered":"2023-07-13T22:28:57.746673+0000","last_unstale":"2023-07-13T22:28:59.716262+0000","last_undegraded":"2023-07-13T22:28:59.716262+0000","last_fullsized":"2023-07-13T22:28:59.716262+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:56:24.646937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.537001+0000","last_change":"2023-07-13T22:28:58.546094+0000","last_active":"2023-07-13T22:28:59.537001+0000","last_peered":"2023-07-13T22:28:59.537001+0000","last_clean":"2023-07-13T22:28:59.537001+0000","last_became_active":"2023-07-13T22:28:58.545803+0000","last_became_peered":"2023-07-13T22:28:58.545803+0000","last_unstale":"2023-07-13T22:28:59.537001+0000","last_undegraded":"2023-07-13T22:28:59.537001+0000","last_fullsized":"2023-07-13T22:28:59.537001+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:15:52.870579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532066+0000","last_change":"2023-07-13T22:28:57.747795+0000","last_active":"2023-07-13T22:28:59.532066+0000","last_peered":"2023-07-13T22:28:59.532066+0000","last_clean":"2023-07-13T22:28:59.532066+0000","last_became_active":"2023-07-13T22:28:57.747501+0000","last_became_peered":"2023-07-13T22:28:57.747501+0000","last_unstale":"2023-07-13T22:28:59.532066+0000","last_undegraded":"2023-07-13T22:28:59.532066+0000","last_fullsized":"2023-07-13T22:28:59.532066+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:04:03.469593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715676+0000","last_change":"2023-07-13T22:28:58.549256+0000","last_active":"2023-07-13T22:28:59.715676+0000","last_peered":"2023-07-13T22:28:59.715676+0000","last_clean":"2023-07-13T22:28:59.715676+0000","last_became_active":"2023-07-13T22:28:58.548982+0000","last_became_peered":"2023-07-13T22:28:58.548982+0000","last_unstale":"2023-07-13T22:28:59.715676+0000","last_undegraded":"2023-07-13T22:28:59.715676+0000","last_fullsized":"2023-07-13T22:28:59.715676+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:48:49.326985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532433+0000","last_change":"2023-07-13T22:28:57.753424+0000","last_active":"2023-07-13T22:28:59.532433+0000","last_peered":"2023-07-13T22:28:59.532433+0000","last_clean":"2023-07-13T22:28:59.532433+0000","last_became_active":"2023-07-13T22:28:57.753153+0000","last_became_peered":"2023-07-13T22:28:57.753153+0000","last_unstale":"2023-07-13T22:28:59.532433+0000","last_undegraded":"2023-07-13T22:28:59.532433+0000","last_fullsized":"2023-07-13T22:28:59.532433+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:47:34.670280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":166,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536490+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:28:59.536490+0000","last_peered":"2023-07-13T22:28:59.536490+0000","last_clean":"2023-07-13T22:28:59.536490+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:28:59.536490+0000","last_undegraded":"2023-07-13T22:28:59.536490+0000","last_fullsized":"2023-07-13T22:28:59.536490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:10:58.000882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716357+0000","last_change":"2023-07-13T22:28:58.553890+0000","last_active":"2023-07-13T22:28:59.716357+0000","last_peered":"2023-07-13T22:28:59.716357+0000","last_clean":"2023-07-13T22:28:59.716357+0000","last_became_active":"2023-07-13T22:28:58.553631+0000","last_became_peered":"2023-07-13T22:28:58.553631+0000","last_unstale":"2023-07-13T22:28:59.716357+0000","last_undegraded":"2023-07-13T22:28:59.716357+0000","last_fullsized":"2023-07-13T22:28:59.716357+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:27:26.611945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532200+0000","last_change":"2023-07-13T22:28:57.751571+0000","last_active":"2023-07-13T22:28:59.532200+0000","last_peered":"2023-07-13T22:28:59.532200+0000","last_clean":"2023-07-13T22:28:59.532200+0000","last_became_active":"2023-07-13T22:28:57.751247+0000","last_became_peered":"2023-07-13T22:28:57.751247+0000","last_unstale":"2023-07-13T22:28:59.532200+0000","last_undegraded":"2023-07-13T22:28:59.532200+0000","last_fullsized":"2023-07-13T22:28:59.532200+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:41:26.895079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":154,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716658+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:28:59.716658+0000","last_peered":"2023-07-13T22:28:59.716658+0000","last_clean":"2023-07-13T22:28:59.716658+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:28:59.716658+0000","last_undegraded":"2023-07-13T22:28:59.716658+0000","last_fullsized":"2023-07-13T22:28:59.716658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:29:05.296164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716358+0000","last_change":"2023-07-13T22:28:58.553743+0000","last_active":"2023-07-13T22:28:59.716358+0000","last_peered":"2023-07-13T22:28:59.716358+0000","last_clean":"2023-07-13T22:28:59.716358+0000","last_became_active":"2023-07-13T22:28:58.553385+0000","last_became_peered":"2023-07-13T22:28:58.553385+0000","last_unstale":"2023-07-13T22:28:59.716358+0000","last_undegraded":"2023-07-13T22:28:59.716358+0000","last_fullsized":"2023-07-13T22:28:59.716358+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:56:31.006623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715643+0000","last_change":"2023-07-13T22:28:57.751093+0000","last_active":"2023-07-13T22:28:59.715643+0000","last_peered":"2023-07-13T22:28:59.715643+0000","last_clean":"2023-07-13T22:28:59.715643+0000","last_became_active":"2023-07-13T22:28:57.750826+0000","last_became_peered":"2023-07-13T22:28:57.750826+0000","last_unstale":"2023-07-13T22:28:59.715643+0000","last_undegraded":"2023-07-13T22:28:59.715643+0000","last_fullsized":"2023-07-13T22:28:59.715643+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:36.190491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":166,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715858+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:28:59.715858+0000","last_peered":"2023-07-13T22:28:59.715858+0000","last_clean":"2023-07-13T22:28:59.715858+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:28:59.715858+0000","last_undegraded":"2023-07-13T22:28:59.715858+0000","last_fullsized":"2023-07-13T22:28:59.715858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:37:50.540016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":305,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715245+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:28:59.715245+0000","last_peered":"2023-07-13T22:28:59.715245+0000","last_clean":"2023-07-13T22:28:59.715245+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:28:59.715245+0000","last_undegraded":"2023-07-13T22:28:59.715245+0000","last_fullsized":"2023-07-13T22:28:59.715245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:25:28.818805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715438+0000","last_change":"2023-07-13T22:28:58.550817+0000","last_active":"2023-07-13T22:28:59.715438+0000","last_peered":"2023-07-13T22:28:59.715438+0000","last_clean":"2023-07-13T22:28:59.715438+0000","last_became_active":"2023-07-13T22:28:58.550541+0000","last_became_peered":"2023-07-13T22:28:58.550541+0000","last_unstale":"2023-07-13T22:28:59.715438+0000","last_undegraded":"2023-07-13T22:28:59.715438+0000","last_fullsized":"2023-07-13T22:28:59.715438+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:59:29.154486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536362+0000","last_change":"2023-07-13T22:28:57.747870+0000","last_active":"2023-07-13T22:28:59.536362+0000","last_peered":"2023-07-13T22:28:59.536362+0000","last_clean":"2023-07-13T22:28:59.536362+0000","last_became_active":"2023-07-13T22:28:57.747531+0000","last_became_peered":"2023-07-13T22:28:57.747531+0000","last_unstale":"2023-07-13T22:28:59.536362+0000","last_undegraded":"2023-07-13T22:28:59.536362+0000","last_fullsized":"2023-07-13T22:28:59.536362+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:19:10.815922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":156,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715782+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:28:59.715782+0000","last_peered":"2023-07-13T22:28:59.715782+0000","last_clean":"2023-07-13T22:28:59.715782+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:28:59.715782+0000","last_undegraded":"2023-07-13T22:28:59.715782+0000","last_fullsized":"2023-07-13T22:28:59.715782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:25:41.712766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.537108+0000","last_change":"2023-07-13T22:28:58.548050+0000","last_active":"2023-07-13T22:28:59.537108+0000","last_peered":"2023-07-13T22:28:59.537108+0000","last_clean":"2023-07-13T22:28:59.537108+0000","last_became_active":"2023-07-13T22:28:58.547793+0000","last_became_peered":"2023-07-13T22:28:58.547793+0000","last_unstale":"2023-07-13T22:28:59.537108+0000","last_undegraded":"2023-07-13T22:28:59.537108+0000","last_fullsized":"2023-07-13T22:28:59.537108+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:32:26.732073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532373+0000","last_change":"2023-07-13T22:28:57.754001+0000","last_active":"2023-07-13T22:28:59.532373+0000","last_peered":"2023-07-13T22:28:59.532373+0000","last_clean":"2023-07-13T22:28:59.532373+0000","last_became_active":"2023-07-13T22:28:57.753722+0000","last_became_peered":"2023-07-13T22:28:57.753722+0000","last_unstale":"2023-07-13T22:28:59.532373+0000","last_undegraded":"2023-07-13T22:28:59.532373+0000","last_fullsized":"2023-07-13T22:28:59.532373+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:12:30.687872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":157,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715434+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:28:59.715434+0000","last_peered":"2023-07-13T22:28:59.715434+0000","last_clean":"2023-07-13T22:28:59.715434+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:28:59.715434+0000","last_undegraded":"2023-07-13T22:28:59.715434+0000","last_fullsized":"2023-07-13T22:28:59.715434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:14:55.781456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.531924+0000","last_change":"2023-07-13T22:28:58.552035+0000","last_active":"2023-07-13T22:28:59.531924+0000","last_peered":"2023-07-13T22:28:59.531924+0000","last_clean":"2023-07-13T22:28:59.531924+0000","last_became_active":"2023-07-13T22:28:58.551756+0000","last_became_peered":"2023-07-13T22:28:58.551756+0000","last_unstale":"2023-07-13T22:28:59.531924+0000","last_undegraded":"2023-07-13T22:28:59.531924+0000","last_fullsized":"2023-07-13T22:28:59.531924+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:35:37.888525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716454+0000","last_change":"2023-07-13T22:28:57.752483+0000","last_active":"2023-07-13T22:28:59.716454+0000","last_peered":"2023-07-13T22:28:59.716454+0000","last_clean":"2023-07-13T22:28:59.716454+0000","last_became_active":"2023-07-13T22:28:57.752187+0000","last_became_peered":"2023-07-13T22:28:57.752187+0000","last_unstale":"2023-07-13T22:28:59.716454+0000","last_undegraded":"2023-07-13T22:28:59.716454+0000","last_fullsized":"2023-07-13T22:28:59.716454+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:43.872003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":172,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.536757+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:28:59.536757+0000","last_peered":"2023-07-13T22:28:59.536757+0000","last_clean":"2023-07-13T22:28:59.536757+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:28:59.536757+0000","last_undegraded":"2023-07-13T22:28:59.536757+0000","last_fullsized":"2023-07-13T22:28:59.536757+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:26:56.000709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.532311+0000","last_change":"2023-07-13T22:28:58.547981+0000","last_active":"2023-07-13T22:28:59.532311+0000","last_peered":"2023-07-13T22:28:59.532311+0000","last_clean":"2023-07-13T22:28:59.532311+0000","last_became_active":"2023-07-13T22:28:58.547701+0000","last_became_peered":"2023-07-13T22:28:58.547701+0000","last_unstale":"2023-07-13T22:28:59.532311+0000","last_undegraded":"2023-07-13T22:28:59.532311+0000","last_fullsized":"2023-07-13T22:28:59.532311+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:35:33.966738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.531982+0000","last_change":"2023-07-13T22:28:57.753462+0000","last_active":"2023-07-13T22:28:59.531982+0000","last_peered":"2023-07-13T22:28:59.531982+0000","last_clean":"2023-07-13T22:28:59.531982+0000","last_became_active":"2023-07-13T22:28:57.753051+0000","last_became_peered":"2023-07-13T22:28:57.753051+0000","last_unstale":"2023-07-13T22:28:59.531982+0000","last_undegraded":"2023-07-13T22:28:59.531982+0000","last_fullsized":"2023-07-13T22:28:59.531982+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:23:14.888175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":150,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715776+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:28:59.715776+0000","last_peered":"2023-07-13T22:28:59.715776+0000","last_clean":"2023-07-13T22:28:59.715776+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:28:59.715776+0000","last_undegraded":"2023-07-13T22:28:59.715776+0000","last_fullsized":"2023-07-13T22:28:59.715776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:26:56.000709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.715927+0000","last_change":"2023-07-13T22:28:58.551907+0000","last_active":"2023-07-13T22:28:59.715927+0000","last_peered":"2023-07-13T22:28:59.715927+0000","last_clean":"2023-07-13T22:28:59.715927+0000","last_became_active":"2023-07-13T22:28:58.550993+0000","last_became_peered":"2023-07-13T22:28:58.550993+0000","last_unstale":"2023-07-13T22:28:59.715927+0000","last_undegraded":"2023-07-13T22:28:59.715927+0000","last_fullsized":"2023-07-13T22:28:59.715927+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:57.723219+0000","last_clean_scrub_stamp":"2023-07-13T22:28:57.723219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:57:39.903647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716094+0000","last_change":"2023-07-13T22:28:57.747514+0000","last_active":"2023-07-13T22:28:59.716094+0000","last_peered":"2023-07-13T22:28:59.716094+0000","last_clean":"2023-07-13T22:28:59.716094+0000","last_became_active":"2023-07-13T22:28:57.747244+0000","last_became_peered":"2023-07-13T22:28:57.747244+0000","last_unstale":"2023-07-13T22:28:59.716094+0000","last_undegraded":"2023-07-13T22:28:59.716094+0000","last_fullsized":"2023-07-13T22:28:59.716094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:28:56.714828+0000","last_clean_scrub_stamp":"2023-07-13T22:28:56.714828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:47:07.889223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":150,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-07-13T22:28:59.716299+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:28:59.716299+0000","last_peered":"2023-07-13T22:28:59.716299+0000","last_clean":"2023-07-13T22:28:59.716299+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:28:59.716299+0000","last_undegraded":"2023-07-13T22:28:59.716299+0000","last_fullsized":"2023-07-13T22:28:59.716299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:00:05.436917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36940,"kb_used_data":1796,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334900,"statfs":{"total":96636764160,"available":96598937600,"internally_reserved":0,"allocated":1839104,"data_stored":1154646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67200000000000004,"15min":0.63900000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.874,"5min":1.0780000000000001,"15min":1.115},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.67300000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.94199999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.42899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.86299999999999999,"5min":0.94799999999999995,"15min":2.2610000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34300000000000003},"max":{"1min":1.0149999999999999,"5min":1.0309999999999999,"15min":1.0609999999999999},"last":0.754}]},{"osd":2,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70699999999999996,"15min":0.77400000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69999999999999996,"15min":0.76600000000000001},"min":{"1min":0.442,"5min":0.42999999999999999,"15min":0.36499999999999999},"max":{"1min":1.006,"5min":1.107,"15min":3.4710000000000001},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.34599999999999997},"max":{"1min":0.90500000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.747,"15min":0.75800000000000001},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92200000000000004,"5min":1.024,"15min":1.1399999999999999},"last":0.83799999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.82399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.73899999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.1850000000000001},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.626,"15min":0.65600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0149999999999999,"5min":1.022,"15min":1.2110000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.67700000000000005,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.93500000000000005,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.875},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69099999999999995,"15min":0.71799999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94399999999999995,"5min":0.98799999999999999,"15min":1.042},"last":0.86099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1257472,"data_stored":566286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67100000000000004,"15min":0.628},"min":{"1min":0.42699999999999999,"5min":0.33100000000000002,"15min":0.28799999999999998},"max":{"1min":1.002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68999999999999995,"15min":0.628},"min":{"1min":0.47199999999999998,"5min":0.45100000000000001,"15min":0.33500000000000002},"max":{"1min":0.89700000000000002,"5min":0.97399999999999998,"15min":0.98899999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.505,"5min":0.39400000000000002,"15min":0.33900000000000002},"max":{"1min":0.94099999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73399999999999999,"15min":0.70299999999999996},"min":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.438},"max":{"1min":0.96899999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.67100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.91800000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69199999999999995,"15min":0.67000000000000004},"min":{"1min":0.56200000000000006,"5min":0.45600000000000002,"15min":0.35599999999999998},"max":{"1min":0.871,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.498}]},{"osd":3,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70899999999999996,"15min":0.66300000000000003},"min":{"1min":0.56100000000000005,"5min":0.37,"15min":0.37},"max":{"1min":0.91600000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73599999999999999,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.95599999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.94499999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65500000000000003,"15min":0.68799999999999994},"min":{"1min":0.52200000000000002,"5min":0.40200000000000002,"15min":0.30399999999999999},"max":{"1min":0.997,"5min":0.998,"15min":0.998},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86899999999999999,"5min":0.94199999999999995,"15min":0.95999999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.73799999999999999,"15min":0.75800000000000001},"min":{"1min":0.54200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97399999999999998,"5min":1.05,"15min":1.05},"last":0.78600000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70699999999999996,"15min":0.745},"min":{"1min":0.46700000000000003,"5min":0.45600000000000002,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96999999999999997,"15min":1.1890000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.92300000000000004,"5min":0.95799999999999996,"15min":1.111},"last":0.69899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36936,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334904,"statfs":{"total":96636764160,"available":96598941696,"internally_reserved":0,"allocated":1835008,"data_stored":1154520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45000000000000001,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.249,"15min":0.249},"max":{"1min":0.70199999999999996,"5min":0.71599999999999997,"15min":1.952},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.496,"15min":0.51700000000000002},"min":{"1min":0.30599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":1.1060000000000001},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.29899999999999999,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":2.1200000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.35099999999999998},"max":{"1min":0.67400000000000004,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.439}]},{"osd":3,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":0.83899999999999997,"15min":2.008},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53000000000000003,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.79000000000000004,"5min":1.054,"15min":1.054},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73899999999999999,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.57199999999999995,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":0.98199999999999998,"15min":2.028},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.59899999999999998,"5min":0.55200000000000005,"15min":0.46400000000000002},"max":{"1min":0.94199999999999995,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.98899999999999999,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78000000000000003,"15min":0.84399999999999997},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.006,"5min":1.095,"15min":2.0899999999999999},"last":0.873}]},{"osd":6,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78600000000000003,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.55100000000000005,"15min":0.52600000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.2230000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.85799999999999998},"min":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0489999999999999,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.747}]},{"osd":7,"last update":"Thu Jul 13 22:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79400000000000004,"15min":0.82699999999999996},"min":{"1min":0.68700000000000006,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":0.99199999999999999,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78300000000000003,"15min":0.81899999999999995},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.84899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1220,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1249280,"data_stored":564786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.52500000000000002},"min":{"1min":0.33500000000000002,"5min":0.30099999999999999,"15min":0.28499999999999998},"max":{"1min":0.84999999999999998,"5min":1.026,"15min":11.791},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.53400000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.874,"5min":1.419,"15min":4.657},"last":0.317}]},{"osd":2,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.432,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":1.1459999999999999,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52600000000000002,"15min":0.96699999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":1.276,"15min":13.398999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72999999999999998,"5min":0.89400000000000002,"15min":1.8089999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56499999999999995,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.997,"15min":4.4530000000000003},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.747,"15min":1.218},"min":{"1min":0.44800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76000000000000001,"15min":0.80800000000000005},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.41299999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.776},"last":0.628}]},{"osd":5,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.78400000000000003,"15min":1.238},"min":{"1min":0.60399999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.073,"5min":1.113,"15min":13.349},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.82699999999999996,"15min":0.88},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.302,"15min":4.7649999999999997},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80000000000000004,"15min":0.83899999999999997},"min":{"1min":0.64800000000000002,"5min":0.52800000000000002,"15min":0.49099999999999999},"max":{"1min":1.212,"5min":1.212,"15min":4.5039999999999996},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":1.2090000000000001},"min":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1259999999999999,"5min":1.2330000000000001,"15min":11.679},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79200000000000004,"15min":1.2430000000000001},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.101,"5min":1.1759999999999999,"15min":13.298},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80100000000000005,"15min":0.89300000000000002},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.151,"5min":1.151,"15min":11.456},"last":0.749}]}]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1236992,"data_stored":563166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.434,"15min":0.51800000000000002},"min":{"1min":0.27600000000000002,"5min":0.252,"15min":0.249},"max":{"1min":0.67700000000000005,"5min":1.1779999999999999,"15min":2.282},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.497,"15min":0.47799999999999998},"min":{"1min":0.32700000000000001,"5min":0.30399999999999999,"15min":0.29099999999999998},"max":{"1min":0.58999999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.26200000000000001},"max":{"1min":0.54200000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.58299999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73899999999999999,"5min":0.86899999999999999,"15min":2.319},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.33900000000000002,"15min":0.311},"max":{"1min":0.64600000000000002,"5min":1.01,"15min":1.01},"last":0.432},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57799999999999996,"15min":0.63300000000000001},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.38700000000000001},"max":{"1min":0.754,"5min":0.95599999999999996,"15min":2.351},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77000000000000002,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95599999999999996,"5min":1.165,"15min":2.238},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78200000000000003,"15min":0.78100000000000003},"min":{"1min":0.53700000000000003,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":1.0269999999999999,"5min":1.274,"15min":1.518},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.61299999999999999,"5min":0.55800000000000005,"15min":0.505},"max":{"1min":1.0109999999999999,"5min":1.099,"15min":1.325},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80400000000000005,"15min":0.84899999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":0.98299999999999998,"5min":1.2490000000000001,"15min":2.1909999999999998},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.56200000000000006,"5min":0.52900000000000003,"15min":0.51700000000000002},"max":{"1min":0.95599999999999996,"5min":1.1040000000000001,"15min":1.488},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.79200000000000004},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.48199999999999998},"max":{"1min":1.0409999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81499999999999995,"15min":0.79500000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":1.0549999999999999,"5min":1.29,"15min":1.29},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81000000000000005,"15min":0.82599999999999996},"min":{"1min":0.57599999999999996,"5min":0.54800000000000004,"15min":0.54400000000000004},"max":{"1min":1.0329999999999999,"5min":1.254,"15min":1.254},"last":0.59099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36356,"kb_used_data":1212,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335484,"statfs":{"total":96636764160,"available":96599535616,"internally_reserved":0,"allocated":1241088,"data_stored":564700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.41999999999999998,"15min":0.42399999999999999},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.66900000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.48299999999999998,"15min":0.45300000000000001},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.27200000000000002},"max":{"1min":0.71099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.5}]},{"osd":1,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.35599999999999998,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.378,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.71899999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73599999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.45800000000000002,"15min":0.44700000000000001},"max":{"1min":1.075,"5min":1.1699999999999999,"15min":1.2250000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.014,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.96899999999999997,"5min":1.1919999999999999,"15min":1.411},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.998,"5min":1.1779999999999999,"15min":1.5469999999999999},"last":0.90500000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.52500000000000002,"15min":0.51800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.52100000000000002},"max":{"1min":0.93500000000000005,"5min":1.046,"15min":1.1240000000000001},"last":0.79700000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.80100000000000005},"min":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.498},"max":{"1min":1.0740000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78200000000000003,"15min":0.79100000000000004},"min":{"1min":0.59199999999999997,"5min":0.55600000000000005,"15min":0.47199999999999998},"max":{"1min":0.93400000000000005,"5min":1.153,"15min":1.1950000000000001},"last":0.88}]}]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36360,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335480,"statfs":{"total":96636764160,"available":96599531520,"internally_reserved":0,"allocated":1245184,"data_stored":564210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.97299999999999998,"15min":1.0349999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.39400000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86599999999999999,"5min":0.997,"15min":0.997},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":1.0489999999999999,"15min":1.298},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.69499999999999995},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.438},"max":{"1min":0.85499999999999998,"5min":1.0049999999999999,"15min":1.3109999999999999},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.48199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.1200000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.71899999999999997},"min":{"1min":0.61799999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94599999999999995,"5min":1.089,"15min":1.1839999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68899999999999995,"15min":0.95699999999999996},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.35899999999999999},"max":{"1min":0.83299999999999996,"5min":0.94999999999999996,"15min":10.657999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72899999999999998,"15min":0.93600000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":8.75},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.46200000000000002,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":0.90800000000000003,"5min":1.042,"15min":1.042},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71199999999999997,"15min":0.73199999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94899999999999995,"5min":1.0109999999999999,"15min":1.0429999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93000000000000005,"5min":1.0189999999999999,"15min":1.173},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.72299999999999998},"min":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97199999999999998,"5min":1.0509999999999999,"15min":1.099},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.004},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.753,"15min":0.77200000000000002},"min":{"1min":0.53200000000000003,"5min":0.497,"15min":0.47899999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.212},"last":1.0600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36352,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335488,"statfs":{"total":96636764160,"available":96599539712,"internally_reserved":0,"allocated":1236992,"data_stored":564118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.65500000000000003,"15min":0.623},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.752,"5min":1.042,"15min":1.042},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.67600000000000005,"15min":0.63600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81499999999999995,"5min":0.998,"15min":0.998},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69999999999999996,"15min":0.66300000000000003},"min":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.66000000000000003,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84699999999999998,"5min":0.94499999999999995,"15min":8.9930000000000003},"last":0.58899999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.67700000000000005,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.35899999999999999},"max":{"1min":0.74199999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70099999999999996,"15min":0.67700000000000005},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90900000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71799999999999997,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":0.96599999999999997},"last":0.751},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72199999999999998,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.66900000000000004,"15min":0.63800000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92900000000000005,"5min":0.93500000000000005,"15min":0.995},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.495,"15min":0.42299999999999999},"max":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.44700000000000001,"15min":0.41199999999999998},"max":{"1min":0.99299999999999999,"5min":0.997,"15min":1.02},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77800000000000002,"15min":0.76500000000000001},"min":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0289999999999999,"15min":1.0669999999999999},"last":0.77200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:29:03.651 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:29:03.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-13T22:29:03.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.918+0000 7f905e1ff700 1 Processor -- start 2023-07-13T22:29:03.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.918+0000 7f905e1ff700 1 -- start start 2023-07-13T22:29:03.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.919+0000 7f905e1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9058159ea0 0x7f90580cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.919+0000 7f905e1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905815c9b0 0x7f90580d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.919+0000 7f905e1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905815d200 con 0x7f90580d37c0 2023-07-13T22:29:03.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.919+0000 7f905e1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90580d2e60 con 0x7f9058159ea0 2023-07-13T22:29:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.919+0000 7f905e1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580d2b50 con 0x7f905815c9b0 2023-07-13T22:29:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f90577fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9058159ea0 0x7f90580cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f9057fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f90580d37c0 0x7f90580d0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50100/0 (socket says 172.21.15.60:50100) 2023-07-13T22:29:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f9057fff700 1 -- 172.21.15.60:0/801937189 learned_addr learned my addr 172.21.15.60:0/801937189 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:03.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f9056ffd700 1 --2- 172.21.15.60:0/801937189 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905815c9b0 0x7f90580d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f90577fe700 1 -- 172.21.15.60:0/801937189 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905815c9b0 msgr2=0x7f90580d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:03.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f90577fe700 1 --2- 172.21.15.60:0/801937189 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905815c9b0 0x7f90580d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f90577fe700 1 -- 172.21.15.60:0/801937189 >> v1:172.21.15.60:6789/0 conn(0x7f90580d37c0 legacy=0x7f90580d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f90577fe700 1 -- 172.21.15.60:0/801937189 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905815d4e0 con 0x7f9058159ea0 2023-07-13T22:29:03.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f9056ffd700 1 --2- 172.21.15.60:0/801937189 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905815c9b0 0x7f90580d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:03.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f90567fc700 1 -- 172.21.15.60:0/801937189 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762203089 0 0) 0x7f905815d200 con 0x7f90580d37c0 2023-07-13T22:29:03.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.920+0000 7f90577fe700 1 --2- 172.21.15.60:0/801937189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9058159ea0 0x7f90580cd6d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f9048009840 tx=0x7f9048009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ed336385e08e87 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:03.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f90567fc700 1 -- 172.21.15.60:0/801937189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048011830 con 0x7f9058159ea0 2023-07-13T22:29:03.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f90567fc700 1 -- 172.21.15.60:0/801937189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048011990 con 0x7f9058159ea0 2023-07-13T22:29:03.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f90567fc700 1 -- 172.21.15.60:0/801937189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048011d20 con 0x7f9058159ea0 2023-07-13T22:29:03.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f905e1ff700 1 -- 172.21.15.60:0/801937189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9058159ea0 msgr2=0x7f90580cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f905e1ff700 1 --2- 172.21.15.60:0/801937189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9058159ea0 0x7f90580cd6d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f9048009840 tx=0x7f9048009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:03.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f905e1ff700 1 -- 172.21.15.60:0/801937189 shutdown_connections 2023-07-13T22:29:03.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f905e1ff700 1 --2- 172.21.15.60:0/801937189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9058159ea0 0x7f90580cd6d0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f905e1ff700 1 --2- 172.21.15.60:0/801937189 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f905815c9b0 0x7f90580d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f905e1ff700 1 -- 172.21.15.60:0/801937189 >> 172.21.15.60:0/801937189 conn(0x7f9058071600 msgr2=0x7f90580719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:03.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f905e1ff700 1 -- 172.21.15.60:0/801937189 shutdown_connections 2023-07-13T22:29:03.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.921+0000 7f905e1ff700 1 -- 172.21.15.60:0/801937189 wait complete. 2023-07-13T22:29:03.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.923+0000 7f905e1ff700 1 Processor -- start 2023-07-13T22:29:03.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.923+0000 7f905e1ff700 1 -- start start 2023-07-13T22:29:03.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f905e1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90580d37c0 0x7f90580cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f905e1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9058159ea0 0x7f90580c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f905e1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905815c9b0 0x7f9058148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f905e1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c7840 con 0x7f905815c9b0 2023-07-13T22:29:03.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f905e1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c7a90 con 0x7f90580d37c0 2023-07-13T22:29:03.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f905e1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c7d70 con 0x7f9058159ea0 2023-07-13T22:29:03.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f9057fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905815c9b0 0x7f9058148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f9057fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905815c9b0 0x7f9058148d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57788/0 (socket says 172.21.15.60:57788) 2023-07-13T22:29:03.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f9057fff700 1 -- 172.21.15.60:0/540425144 learned_addr learned my addr 172.21.15.60:0/540425144 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:03.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.924+0000 7f90577fe700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90580d37c0 0x7f90580cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f9056ffd700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9058159ea0 0x7f90580c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f9057fff700 1 -- 172.21.15.60:0/540425144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9058159ea0 msgr2=0x7f90580c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f9057fff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9058159ea0 0x7f90580c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f9057fff700 1 -- 172.21.15.60:0/540425144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90580d37c0 msgr2=0x7f90580cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:03.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f9057fff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90580d37c0 0x7f90580cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:03.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f9057fff700 1 -- 172.21.15.60:0/540425144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480094a0 con 0x7f905815c9b0 2023-07-13T22:29:03.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f9057fff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905815c9b0 0x7f9058148d90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9050002230 tx=0x7f905000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:03.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f903ffff700 1 -- 172.21.15.60:0/540425144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905000aac0 con 0x7f905815c9b0 2023-07-13T22:29:03.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f903ffff700 1 -- 172.21.15.60:0/540425144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905000ac40 con 0x7f905815c9b0 2023-07-13T22:29:03.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90580c8050 con 0x7f905815c9b0 2023-07-13T22:29:03.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.925+0000 7f903ffff700 1 -- 172.21.15.60:0/540425144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90500070d0 con 0x7f905815c9b0 2023-07-13T22:29:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.926+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90580c8610 con 0x7f905815c9b0 2023-07-13T22:29:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.926+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90580c8980 con 0x7f905815c9b0 2023-07-13T22:29:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.927+0000 7f903ffff700 1 -- 172.21.15.60:0/540425144 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9050007250 con 0x7f905815c9b0 2023-07-13T22:29:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.928+0000 7f903ffff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f904406ace0 0x7f904406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.928+0000 7f903ffff700 1 -- 172.21.15.60:0/540425144 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9050089390 con 0x7f905815c9b0 2023-07-13T22:29:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.929+0000 7f90577fe700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f904406ace0 0x7f904406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:03.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.929+0000 7f90577fe700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f904406ace0 0x7f904406d190 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f90480090b0 tx=0x7f9048008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:03.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:03.933+0000 7f903ffff700 1 -- 172.21.15.60:0/540425144 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9050054310 con 0x7f905815c9b0 2023-07-13T22:29:04.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.221+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9058067a70 con 0x7f905815c9b0 2023-07-13T22:29:04.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.225+0000 7f903ffff700 1 -- 172.21.15.60:0/540425144 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f90500180e0 con 0x7f905815c9b0 2023-07-13T22:29:04.204 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:29:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:29:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-07-13T22:29:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-07-13T22:29:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-13T22:29:04.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f904406ace0 msgr2=0x7f904406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:04.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f904406ace0 0x7f904406d190 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f90480090b0 tx=0x7f9048008030 comp rx=0 tx=0).stop 2023-07-13T22:29:04.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905815c9b0 msgr2=0x7f9058148d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:04.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905815c9b0 0x7f9058148d90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9050002230 tx=0x7f905000cf50 comp rx=0 tx=0).stop 2023-07-13T22:29:04.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 shutdown_connections 2023-07-13T22:29:04.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f904406ace0 0x7f904406d190 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f905815c9b0 0x7f9058148d90 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90580d37c0 0x7f90580cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 --2- 172.21.15.60:0/540425144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9058159ea0 0x7f90580c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 >> 172.21.15.60:0/540425144 conn(0x7f9058071600 msgr2=0x7f905815b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:04.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 shutdown_connections 2023-07-13T22:29:04.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.228+0000 7f905e1ff700 1 -- 172.21.15.60:0/540425144 wait complete. 2023-07-13T22:29:04.208 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-07-13T22:29:04.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.449+0000 7f7544bff700 1 Processor -- start 2023-07-13T22:29:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.449+0000 7f7544bff700 1 -- start start 2023-07-13T22:29:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.450+0000 7f7544bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75400b8570 0x7f75400ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:04.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.450+0000 7f7544bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75400d09d0 con 0x7f75400d15c0 2023-07-13T22:29:04.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.450+0000 7f7544bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75400cda10 con 0x7f75400d35c0 2023-07-13T22:29:04.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.450+0000 7f7544bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75400d3b30 con 0x7f75400b8570 2023-07-13T22:29:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.450+0000 7f753ed9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f75400d15c0 0x7f754015c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50114/0 (socket says 172.21.15.60:50114) 2023-07-13T22:29:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.450+0000 7f753ed9d700 1 -- 172.21.15.60:0/535160988 learned_addr learned my addr 172.21.15.60:0/535160988 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:04.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328892133 0 0) 0x7f75400d09d0 con 0x7f75400d15c0 2023-07-13T22:29:04.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752c0035b0 con 0x7f75400d15c0 2023-07-13T22:29:04.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753e59c700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75400b8570 0x7f75400ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:04.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1619182871 0 0) 0x7f752c0035b0 con 0x7f75400d15c0 2023-07-13T22:29:04.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75400b8570 msgr2=0x7f75400ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:04.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75400b8570 0x7f75400ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 >> v1:172.21.15.105:6789/0 conn(0x7f75400d35c0 legacy=0x7f75400d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:04.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400cd450 con 0x7f75400d15c0 2023-07-13T22:29:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f75380019f0 con 0x7f75400d15c0 2023-07-13T22:29:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 >> v1:172.21.15.60:6789/0 conn(0x7f75400d15c0 legacy=0x7f754015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c003a30 0x7f752c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75400b8570 0x7f752c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f752c004d90 0x7f752c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400d3b30 con 0x7f752c004d90 2023-07-13T22:29:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c00baf0 con 0x7f752c003a30 2023-07-13T22:29:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.451+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c00bcb0 con 0x7f75400b8570 2023-07-13T22:29:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f753ed9d700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f752c004d90 0x7f752c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f753e59c700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c003a30 0x7f752c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f7537fff700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75400b8570 0x7f752c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f753ed9d700 1 -- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75400b8570 msgr2=0x7f752c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f753ed9d700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75400b8570 0x7f752c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f753ed9d700 1 -- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c003a30 msgr2=0x7f752c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f753ed9d700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c003a30 0x7f752c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f753ed9d700 1 -- 172.21.15.60:0/535160988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c0035b0 con 0x7f752c004d90 2023-07-13T22:29:04.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f753ed9d700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f752c004d90 0x7f752c005200 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f75400d1a40 tx=0x7f752c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a394fee28af84a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:04.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.452+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753800a250 con 0x7f752c004d90 2023-07-13T22:29:04.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.453+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75380072f0 con 0x7f752c004d90 2023-07-13T22:29:04.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.453+0000 7f753dd9b700 1 -- 172.21.15.60:0/535160988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75380076a0 con 0x7f752c004d90 2023-07-13T22:29:04.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.453+0000 7f7544bff700 1 -- 172.21.15.60:0/535160988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f752c004d90 msgr2=0x7f752c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:04.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.453+0000 7f7544bff700 1 --2- 172.21.15.60:0/535160988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f752c004d90 0x7f752c005200 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f75400d1a40 tx=0x7f752c005880 comp rx=0 tx=0).stop 2023-07-13T22:29:04.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.453+0000 7f753ed9d700 1 -- 172.21.15.60:0/535160988 reap_dead start 2023-07-13T22:29:04.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.453+0000 7f7544bff700 1 -- 172.21.15.60:0/535160988 shutdown_connections 2023-07-13T22:29:04.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.453+0000 7f7544bff700 1 -- 172.21.15.60:0/535160988 >> 172.21.15.60:0/535160988 conn(0x7f7540071600 msgr2=0x7f75400719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:04.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.453+0000 7f7544bff700 1 -- 172.21.15.60:0/535160988 shutdown_connections 2023-07-13T22:29:04.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.453+0000 7f7544bff700 1 -- 172.21.15.60:0/535160988 wait complete. 2023-07-13T22:29:04.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.455+0000 7f7544bff700 1 Processor -- start 2023-07-13T22:29:04.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.455+0000 7f7544bff700 1 -- start start 2023-07-13T22:29:04.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.456+0000 7f7544bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c004d90 0x7f75400c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:04.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.456+0000 7f7544bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7540159550 0x7f75400cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:04.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.456+0000 7f7544bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f754014be50 0x7f754014c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:04.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.456+0000 7f7544bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7540159df0 con 0x7f7540159550 2023-07-13T22:29:04.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.456+0000 7f7544bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400ca410 con 0x7f752c004d90 2023-07-13T22:29:04.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.456+0000 7f7544bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7540149990 con 0x7f754014be50 2023-07-13T22:29:04.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.456+0000 7f7537fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7540159550 0x7f75400cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:04.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.456+0000 7f7537fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7540159550 0x7f75400cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57802/0 (socket says 172.21.15.60:57802) 2023-07-13T22:29:04.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.456+0000 7f7537fff700 1 -- 172.21.15.60:0/2690242257 learned_addr learned my addr 172.21.15.60:0/2690242257 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:04.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f753ed9d700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f754014be50 0x7f754014c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:04.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f753e59c700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c004d90 0x7f75400c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:04.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7537fff700 1 -- 172.21.15.60:0/2690242257 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f754014be50 msgr2=0x7f754014c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:04.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7537fff700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f754014be50 0x7f754014c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7537fff700 1 -- 172.21.15.60:0/2690242257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c004d90 msgr2=0x7f75400c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:04.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7537fff700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c004d90 0x7f75400c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7537fff700 1 -- 172.21.15.60:0/2690242257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7540149c70 con 0x7f7540159550 2023-07-13T22:29:04.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f753e59c700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c004d90 0x7f75400c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:04.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7537fff700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7540159550 0x7f75400cf610 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f752800f7f0 tx=0x7f752800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:04.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7536ffd700 1 -- 172.21.15.60:0/2690242257 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75280052e0 con 0x7f7540159550 2023-07-13T22:29:04.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7536ffd700 1 -- 172.21.15.60:0/2690242257 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7528005440 con 0x7f7540159550 2023-07-13T22:29:04.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7540149f50 con 0x7f7540159550 2023-07-13T22:29:04.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7536ffd700 1 -- 172.21.15.60:0/2690242257 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75280038c0 con 0x7f7540159550 2023-07-13T22:29:04.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.457+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75400cfb50 con 0x7f7540159550 2023-07-13T22:29:04.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.458+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754014a470 con 0x7f7540159550 2023-07-13T22:29:04.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.459+0000 7f7536ffd700 1 -- 172.21.15.60:0/2690242257 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7528003a20 con 0x7f7540159550 2023-07-13T22:29:04.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.460+0000 7f7536ffd700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f752006ad10 0x7f752006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:04.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.460+0000 7f7536ffd700 1 -- 172.21.15.60:0/2690242257 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f75280893f0 con 0x7f7540159550 2023-07-13T22:29:04.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.461+0000 7f753e59c700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f752006ad10 0x7f752006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:04.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.461+0000 7f753e59c700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f752006ad10 0x7f752006d1c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7530009b30 tx=0x7f7530008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:04.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.465+0000 7f7536ffd700 1 -- 172.21.15.60:0/2690242257 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f754014a470 con 0x7f7540159550 2023-07-13T22:29:04.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.719+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f754014a470 con 0x7f7540159550 2023-07-13T22:29:04.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.719+0000 7f7536ffd700 1 -- 172.21.15.60:0/2690242257 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f754014a470 con 0x7f7540159550 2023-07-13T22:29:04.698 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:04.699 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:58.254590+0000","modified":"2023-07-13T22:29:01.746898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4078186433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":4078186433},{"type":"v1","addr":"172.21.15.60:6837","nonce":4078186433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-13T22:29:04.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f752006ad10 msgr2=0x7f752006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:04.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f752006ad10 0x7f752006d1c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7530009b30 tx=0x7f7530008040 comp rx=0 tx=0).stop 2023-07-13T22:29:04.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7540159550 msgr2=0x7f75400cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:04.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7540159550 0x7f75400cf610 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f752800f7f0 tx=0x7f752800fbb0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 shutdown_connections 2023-07-13T22:29:04.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f752006ad10 0x7f752006d1c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f754014be50 0x7f754014c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f752c004d90 0x7f75400c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 --2- 172.21.15.60:0/2690242257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7540159550 0x7f75400cf610 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:04.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.721+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 >> 172.21.15.60:0/2690242257 conn(0x7f7540071600 msgr2=0x7f754015bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:04.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.722+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 shutdown_connections 2023-07-13T22:29:04.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:04.722+0000 7f7544bff700 1 -- 172.21.15.60:0/2690242257 wait complete. 2023-07-13T22:29:04.704 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2023-07-13T22:29:05.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:28:58.254590+0000', 'modified': '2023-07-13T22:29:01.746898+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7920}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7920': {'gid': 7920, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6837/4078186433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6836', 'nonce': 4078186433}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 4078186433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:29:05.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:29:05.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-13T22:29:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.353+0000 7fc604dff700 1 Processor -- start 2023-07-13T22:29:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.353+0000 7fc604dff700 1 -- start start 2023-07-13T22:29:05.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.354+0000 7fc604dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6000cfca0 0x7fc600167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc604dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6000cd950 con 0x7fc6000d3750 2023-07-13T22:29:05.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc604dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6000d0ac0 con 0x7fc6000b8400 2023-07-13T22:29:05.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc604dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000b87d0 con 0x7fc6000cfca0 2023-07-13T22:29:05.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fdd9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc6000d3750 0x7fc6000cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49170/0 (socket says 172.21.15.60:49170) 2023-07-13T22:29:05.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fdd9b700 1 -- 172.21.15.60:0/31825019 learned_addr learned my addr 172.21.15.60:0/31825019 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:05.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6000cfca0 0x7fc600167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280570012 0 0) 0x7fc5f4001110 con 0x7fc6000d3750 2023-07-13T22:29:05.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5e80035b0 con 0x7fc6000d3750 2023-07-13T22:29:05.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574726551 0 0) 0x7fc6000d0ac0 con 0x7fc6000b8400 2023-07-13T22:29:05.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f4001110 con 0x7fc6000b8400 2023-07-13T22:29:05.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 106703219 0 0) 0x7fc5e80035b0 con 0x7fc6000d3750 2023-07-13T22:29:05.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6000cfca0 msgr2=0x7fc600167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.355+0000 7fc5fd59a700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6000cfca0 0x7fc600167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 >> v1:172.21.15.105:6789/0 conn(0x7fc6000b8400 legacy=0x7fc6000cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000b4eb0 con 0x7fc6000d3750 2023-07-13T22:29:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fc5f4001800 con 0x7fc6000d3750 2023-07-13T22:29:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 >> v1:172.21.15.60:6789/0 conn(0x7fc6000d3750 legacy=0x7fc6000cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6000cfca0 0x7fc5e8003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5e8004170 0x7fc5e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc5e8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000b87d0 con 0x7fc5e8004d90 2023-07-13T22:29:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e800bba0 con 0x7fc5e8004170 2023-07-13T22:29:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e800bd60 con 0x7fc6000cfca0 2023-07-13T22:29:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5e8004170 0x7fc5e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc5e8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6000cfca0 0x7fc5e8003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fed9d700 1 -- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6000cfca0 msgr2=0x7fc5e8003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6000cfca0 0x7fc5e8003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fed9d700 1 -- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5e8004170 msgr2=0x7fc5e8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5e8004170 0x7fc5e8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fed9d700 1 -- 172.21.15.60:0/31825019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80035b0 con 0x7fc5e8004d90 2023-07-13T22:29:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.356+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5e8004170 0x7fc5e8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.357+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc5e8005200 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009f90 tx=0x7fc5e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6224e3d13f68c3c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.357+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800ec30 con 0x7fc5e8004d90 2023-07-13T22:29:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.357+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f800f040 con 0x7fc5e8004d90 2023-07-13T22:29:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.357+0000 7fc5fd59a700 1 -- 172.21.15.60:0/31825019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8014480 con 0x7fc5e8004d90 2023-07-13T22:29:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.357+0000 7fc604dff700 1 -- 172.21.15.60:0/31825019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 msgr2=0x7fc5e8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.357+0000 7fc604dff700 1 --2- 172.21.15.60:0/31825019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc5e8005200 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009f90 tx=0x7fc5e8005880 comp rx=0 tx=0).stop 2023-07-13T22:29:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.357+0000 7fc5fed9d700 1 -- 172.21.15.60:0/31825019 reap_dead start 2023-07-13T22:29:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.358+0000 7fc604dff700 1 -- 172.21.15.60:0/31825019 shutdown_connections 2023-07-13T22:29:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.358+0000 7fc604dff700 1 -- 172.21.15.60:0/31825019 >> 172.21.15.60:0/31825019 conn(0x7fc60006b390 msgr2=0x7fc60006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.358+0000 7fc604dff700 1 -- 172.21.15.60:0/31825019 shutdown_connections 2023-07-13T22:29:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.358+0000 7fc604dff700 1 -- 172.21.15.60:0/31825019 wait complete. 2023-07-13T22:29:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.359+0000 7fc604dff700 1 Processor -- start 2023-07-13T22:29:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.360+0000 7fc604dff700 1 -- start start 2023-07-13T22:29:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.360+0000 7fc604dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc60014a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.360+0000 7fc604dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc60015ea50 0x7fc6000c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.360+0000 7fc604dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6000b35d0 0x7fc6000b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.360+0000 7fc604dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60014a540 con 0x7fc5e8004d90 2023-07-13T22:29:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.360+0000 7fc604dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000b3f90 con 0x7fc6000b35d0 2023-07-13T22:29:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.360+0000 7fc604dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000c91a0 con 0x7fc60015ea50 2023-07-13T22:29:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fe59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc60014a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fe59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc60014a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40880/0 (socket says 172.21.15.60:40880) 2023-07-13T22:29:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fe59c700 1 -- 172.21.15.60:0/2104002981 learned_addr learned my addr 172.21.15.60:0/2104002981 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6000b35d0 0x7fc6000b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc60015ea50 0x7fc6000c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fe59c700 1 -- 172.21.15.60:0/2104002981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc60015ea50 msgr2=0x7fc6000c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc60015ea50 0x7fc6000c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fe59c700 1 -- 172.21.15.60:0/2104002981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6000b35d0 msgr2=0x7fc6000b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6000b35d0 0x7fc6000b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fe59c700 1 -- 172.21.15.60:0/2104002981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000c9630 con 0x7fc5e8004d90 2023-07-13T22:29:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.361+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc60014a000 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000f8b0 tx=0x7fc5f000fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.362+0000 7fc5eeffd700 1 -- 172.21.15.60:0/2104002981 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00052e0 con 0x7fc5e8004d90 2023-07-13T22:29:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.362+0000 7fc5eeffd700 1 -- 172.21.15.60:0/2104002981 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f0005440 con 0x7fc5e8004d90 2023-07-13T22:29:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.362+0000 7fc604dff700 1 -- 172.21.15.60:0/2104002981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6000c8590 con 0x7fc5e8004d90 2023-07-13T22:29:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.362+0000 7fc5eeffd700 1 -- 172.21.15.60:0/2104002981 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00038c0 con 0x7fc5e8004d90 2023-07-13T22:29:05.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.362+0000 7fc604dff700 1 -- 172.21.15.60:0/2104002981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6000c8b50 con 0x7fc5e8004d90 2023-07-13T22:29:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.364+0000 7fc5eeffd700 1 -- 172.21.15.60:0/2104002981 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc5f0003a20 con 0x7fc5e8004d90 2023-07-13T22:29:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.365+0000 7fc5eeffd700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5e006ace0 0x7fc5e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.365+0000 7fc5eeffd700 1 -- 172.21.15.60:0/2104002981 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5f0089300 con 0x7fc5e8004d90 2023-07-13T22:29:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.365+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5e006ace0 0x7fc5e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.366+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5e006ace0 0x7fc5e006d190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc60015ee20 tx=0x7fc5f4000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.367+0000 7fc6186a9b80 1 Processor -- start 2023-07-13T22:29:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.367+0000 7fc6186a9b80 1 -- start start 2023-07-13T22:29:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 0x562eb6f625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f62b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 0x562eb6f4d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6eacea0 con 0x562eb6f34dc0 2023-07-13T22:29:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6e20600 con 0x562eb6f32c50 2023-07-13T22:29:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eb6e8f910 con 0x562eb6f32880 2023-07-13T22:29:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fdd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f62b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fdd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f62b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40886/0 (socket says 172.21.15.60:40886) 2023-07-13T22:29:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fdd9b700 1 -- 172.21.15.60:0/2283115102 learned_addr learned my addr 172.21.15.60:0/2283115102 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/2283115102 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 0x562eb6f625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/2283115102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 0x562eb6f4d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fdd9b700 1 -- 172.21.15.60:0/2283115102 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 msgr2=0x562eb6f625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/2283115102 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 0x562eb6f625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fdd9b700 1 -- 172.21.15.60:0/2283115102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 msgr2=0x562eb6f4d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/2283115102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 0x562eb6f4d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.368+0000 7fc5fdd9b700 1 -- 172.21.15.60:0/2283115102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eb6c7c790 con 0x562eb6f34dc0 2023-07-13T22:29:05.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.369+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/2283115102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f62b20 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40072e0 tx=0x7fc5f400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4b84127e6b6a89 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.369+0000 7fc5dffff700 1 -- 172.21.15.60:0/2283115102 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f401a040 con 0x562eb6f34dc0 2023-07-13T22:29:05.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.369+0000 7fc5dffff700 1 -- 172.21.15.60:0/2283115102 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f4015750 con 0x562eb6f34dc0 2023-07-13T22:29:05.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.369+0000 7fc5dffff700 1 -- 172.21.15.60:0/2283115102 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4015ac0 con 0x562eb6f34dc0 2023-07-13T22:29:05.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.370+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2283115102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 msgr2=0x562eb6f62b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.370+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2283115102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f62b20 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40072e0 tx=0x7fc5f400c000 comp rx=0 tx=0).stop 2023-07-13T22:29:05.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.370+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2283115102 shutdown_connections 2023-07-13T22:29:05.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.370+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2283115102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 0x562eb6f4d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.370+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2283115102 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 0x562eb6f625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.370+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2283115102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f62b20 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.370+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2283115102 >> 172.21.15.60:0/2283115102 conn(0x562eb6f2d3b0 msgr2=0x562eb6f38550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:05.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.370+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2283115102 shutdown_connections 2023-07-13T22:29:05.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.370+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2283115102 wait complete. 2023-07-13T22:29:05.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.371+0000 7fc6186a9b80 1 -- start start 2023-07-13T22:29:05.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.371+0000 7fc6186a9b80 1 Processor -- start 2023-07-13T22:29:05.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 0x562eb6f4dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32c50 0x562eb6f65ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f34dc0 0x562eb6f796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6e88be0 con 0x562eb6f32c50 2023-07-13T22:29:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6e465b0 con 0x562eb6f34dc0 2023-07-13T22:29:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eb6e2c910 con 0x562eb6f32880 2023-07-13T22:29:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc6186a9b80 10 client.0 _ll_register_callbacks cb 0x562eb6e44b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:29:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc5fdd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32c50 0x562eb6f65ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc5fdd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32c50 0x562eb6f65ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40894/0 (socket says 172.21.15.60:40894) 2023-07-13T22:29:05.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc5fdd9b700 1 -- 172.21.15.60:0/2122229641 learned_addr learned my addr 172.21.15.60:0/2122229641 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:05.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 0x562eb6f4dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.372+0000 7fc5fdd9b700 1 -- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 msgr2=0x562eb6f4dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f34dc0 0x562eb6f796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 0x562eb6f4dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5fdd9b700 1 -- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f34dc0 msgr2=0x562eb6f796a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f34dc0 0x562eb6f796a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5fdd9b700 1 -- 172.21.15.60:0/2122229641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6000b4eb0 con 0x562eb6f32c50 2023-07-13T22:29:05.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 0x562eb6f4dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:05.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f34dc0 0x562eb6f796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:05.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32c50 0x562eb6f65ee0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40010e0 tx=0x7fc5f4001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8be23d7f3515340 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:29:05.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/2122229641 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f402c900 con 0x562eb6f32c50 2023-07-13T22:29:05.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2122229641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eb6f2a9c0 con 0x562eb6f32c50 2023-07-13T22:29:05.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/2122229641 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5f402ca60 con 0x562eb6f32c50 2023-07-13T22:29:05.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/2122229641 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5f402cc20 con 0x562eb6f32c50 2023-07-13T22:29:05.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc6186a9b80 10 client.8064 fetch_fsmap learned FSMap version 62 2023-07-13T22:29:05.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.373+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2122229641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x562eb6d690a0 con 0x562eb6f32c50 2023-07-13T22:29:05.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.374+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/2122229641 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562eb6f2a9c0 con 0x562eb6f32c50 2023-07-13T22:29:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.374+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/2122229641 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc5f4020040 con 0x562eb6f32c50 2023-07-13T22:29:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.374+0000 7fc6186a9b80 10 client.8064 fetch_fsmap finished waiting for FSMap version 62 2023-07-13T22:29:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.374+0000 7fc6186a9b80 10 client.8064 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-13T22:29:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.374+0000 7fc6186a9b80 20 client.8064 populate_metadata read hostname 'smithi060' 2023-07-13T22:29:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.374+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] conn(0x562eb6f64950 0x562eb6f64d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.374+0000 7fc6186a9b80 4 client.8064 mds_command: new command op to 7920 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:29:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.374+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2122229641 --> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eb6f07170 con 0x562eb6f64950 2023-07-13T22:29:05.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.374+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] conn(0x562eb6f64950 0x562eb6f64d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.375+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] conn(0x562eb6f64950 0x562eb6f64d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e56c5986c33f16b server_cookie=9309eca094f6ea43 in_seq=0 out_seq=0 2023-07-13T22:29:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.375+0000 7fc5dd7fa700 10 client.8064 ms_handle_connect on v2:172.21.15.60:6836/4078186433 2023-07-13T22:29:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.375+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/2122229641 <== mds.0 v2:172.21.15.60:6836/4078186433 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562eb6f07170 con 0x562eb6f64950 2023-07-13T22:29:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.375+0000 7fc5dd7fa700 10 client.8064 handle_command_reply: tid=0 2023-07-13T22:29:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.375+0000 7fc6186a9b80 1 client.8064 shutdown 2023-07-13T22:29:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2122229641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32c50 msgr2=0x562eb6f65ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32c50 0x562eb6f65ee0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40010e0 tx=0x7fc5f4001ac0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2122229641 shutdown_connections 2023-07-13T22:29:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] conn(0x562eb6f64950 0x562eb6f64d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f34dc0 0x562eb6f796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32880 0x562eb6f4dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc5dd7fa700 0 client.8064 ms_handle_reset on v2:172.21.15.60:6836/4078186433 2023-07-13T22:29:05.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/2122229641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32c50 0x562eb6f65ee0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2122229641 >> 172.21.15.60:0/2122229641 conn(0x562eb6f2d3b0 msgr2=0x562eb6f62b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2122229641 shutdown_connections 2023-07-13T22:29:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 1 -- 172.21.15.60:0/2122229641 wait complete. 2023-07-13T22:29:05.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.376+0000 7fc6186a9b80 20 client.8064 trim_cache size 0 max 16384 2023-07-13T22:29:05.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.395+0000 7fc6186a9b80 1 Processor -- start 2023-07-13T22:29:05.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.395+0000 7fc6186a9b80 1 -- start start 2023-07-13T22:29:05.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.396+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32880 0x562eb6f80990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.396+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 0x562eb6f66090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.396+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f34dc0 0x562eb6f8b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.396+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6c7d390 con 0x562eb6f32880 2023-07-13T22:29:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.396+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6f31960 con 0x562eb6f32c50 2023-07-13T22:29:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.396+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eb6f36af0 con 0x562eb6f34dc0 2023-07-13T22:29:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.396+0000 7fc5fe59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32880 0x562eb6f80990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.396+0000 7fc5fe59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32880 0x562eb6f80990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40908/0 (socket says 172.21.15.60:40908) 2023-07-13T22:29:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.396+0000 7fc5fe59c700 1 -- 172.21.15.60:0/559245540 learned_addr learned my addr 172.21.15.60:0/559245540 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 0x562eb6f66090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fe59c700 1 -- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f34dc0 msgr2=0x562eb6f8b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f34dc0 0x562eb6f8b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f34dc0 0x562eb6f8b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fe59c700 1 -- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 msgr2=0x562eb6f66090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 0x562eb6f66090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fe59c700 1 -- 172.21.15.60:0/559245540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eb6e4eb10 con 0x562eb6f32880 2023-07-13T22:29:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 0x562eb6f66090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f34dc0 0x562eb6f8b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32880 0x562eb6f80990 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0057cd0 tx=0x7fc5f001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95c1e4e58260b94 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5ecff9700 1 -- 172.21.15.60:0/559245540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0073070 con 0x562eb6f32880 2023-07-13T22:29:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5ecff9700 1 -- 172.21.15.60:0/559245540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f0056110 con 0x562eb6f32880 2023-07-13T22:29:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.397+0000 7fc5ecff9700 1 -- 172.21.15.60:0/559245540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0086070 con 0x562eb6f32880 2023-07-13T22:29:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 -- 172.21.15.60:0/559245540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32880 msgr2=0x562eb6f80990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32880 0x562eb6f80990 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0057cd0 tx=0x7fc5f001f040 comp rx=0 tx=0).stop 2023-07-13T22:29:05.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 -- 172.21.15.60:0/559245540 shutdown_connections 2023-07-13T22:29:05.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f32880 0x562eb6f80990 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f34dc0 0x562eb6f8b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/559245540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32c50 0x562eb6f66090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 -- 172.21.15.60:0/559245540 >> 172.21.15.60:0/559245540 conn(0x562eb6f2d3b0 msgr2=0x562eb6f38520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:05.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 -- 172.21.15.60:0/559245540 shutdown_connections 2023-07-13T22:29:05.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 -- 172.21.15.60:0/559245540 wait complete. 2023-07-13T22:29:05.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.398+0000 7fc6186a9b80 1 -- start start 2023-07-13T22:29:05.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.399+0000 7fc6186a9b80 1 Processor -- start 2023-07-13T22:29:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32880 0x562eb6fdaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32c50 0x562eb6f64d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc6186a9b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6fe52a0 con 0x562eb6f34dc0 2023-07-13T22:29:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb6f33800 con 0x562eb6f32880 2023-07-13T22:29:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc6186a9b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eb6fdaf60 con 0x562eb6f32c50 2023-07-13T22:29:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc6186a9b80 10 client.0 _ll_register_callbacks cb 0x7fc5f4026070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:29:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f65260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40922/0 (socket says 172.21.15.60:40922) 2023-07-13T22:29:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fed9d700 1 -- 172.21.15.60:0/3124379753 learned_addr learned my addr 172.21.15.60:0/3124379753 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32880 0x562eb6fdaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fdd9b700 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32c50 0x562eb6f64d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fed9d700 1 -- 172.21.15.60:0/3124379753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32c50 msgr2=0x562eb6f64d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32c50 0x562eb6f64d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fed9d700 1 -- 172.21.15.60:0/3124379753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32880 msgr2=0x562eb6fdaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32880 0x562eb6fdaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fed9d700 1 -- 172.21.15.60:0/3124379753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6000c8b50 con 0x562eb6f34dc0 2023-07-13T22:29:05.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.400+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32880 0x562eb6fdaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:05.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc5fed9d700 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f65260 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8002230 tx=0x7fc5f8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb0873276392cc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc5dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:29:05.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/3124379753 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8022070 con 0x562eb6f34dc0 2023-07-13T22:29:05.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc6186a9b80 1 -- 172.21.15.60:0/3124379753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5f4015750 con 0x562eb6f34dc0 2023-07-13T22:29:05.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/3124379753 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5f8003e20 con 0x562eb6f34dc0 2023-07-13T22:29:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/3124379753 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5f8007000 con 0x562eb6f34dc0 2023-07-13T22:29:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc6186a9b80 10 client.8070 fetch_fsmap learned FSMap version 62 2023-07-13T22:29:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc6186a9b80 1 -- 172.21.15.60:0/3124379753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x562eb6cdd1d0 con 0x562eb6f34dc0 2023-07-13T22:29:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/3124379753 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f4015750 con 0x562eb6f34dc0 2023-07-13T22:29:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.401+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/3124379753 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc5f8004250 con 0x562eb6f34dc0 2023-07-13T22:29:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.402+0000 7fc6186a9b80 10 client.8070 fetch_fsmap finished waiting for FSMap version 62 2023-07-13T22:29:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.402+0000 7fc6186a9b80 10 client.8070 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-13T22:29:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.402+0000 7fc6186a9b80 20 client.8070 populate_metadata read hostname 'smithi060' 2023-07-13T22:29:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.402+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] conn(0x562eb6f733f0 0x562eb6f737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.402+0000 7fc6186a9b80 4 client.8070 mds_command: new command op to 7920 tid=0[{"prefix": "status"}] 2023-07-13T22:29:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.402+0000 7fc6186a9b80 1 -- 172.21.15.60:0/3124379753 --> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562eb6e81d20 con 0x562eb6f733f0 2023-07-13T22:29:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.402+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] conn(0x562eb6f733f0 0x562eb6f737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.402+0000 7fc5fe59c700 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] conn(0x562eb6f733f0 0x562eb6f737c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f080b38762d2af server_cookie=66be373b89411b55 in_seq=0 out_seq=0 2023-07-13T22:29:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.402+0000 7fc5dd7fa700 10 client.8070 ms_handle_connect on v2:172.21.15.60:6836/4078186433 2023-07-13T22:29:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc5dd7fa700 1 -- 172.21.15.60:0/3124379753 <== mds.0 v2:172.21.15.60:6836/4078186433 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x562eb6e81d20 con 0x562eb6f733f0 2023-07-13T22:29:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc5dd7fa700 10 client.8070 handle_command_reply: tid=0 2023-07-13T22:29:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 client.8070 shutdown 2023-07-13T22:29:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 -- 172.21.15.60:0/3124379753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 msgr2=0x562eb6f65260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f65260 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8002230 tx=0x7fc5f8004b40 comp rx=0 tx=0).stop 2023-07-13T22:29:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 -- 172.21.15.60:0/3124379753 shutdown_connections 2023-07-13T22:29:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.60:6836/4078186433,v1:172.21.15.60:6837/4078186433] conn(0x562eb6f733f0 0x562eb6f737c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562eb6f34dc0 0x562eb6f65260 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562eb6f32880 0x562eb6fdaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 --2- 172.21.15.60:0/3124379753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x562eb6f32c50 0x562eb6f64d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc5dd7fa700 0 client.8070 ms_handle_reset on v2:172.21.15.60:6836/4078186433 2023-07-13T22:29:05.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 -- 172.21.15.60:0/3124379753 >> 172.21.15.60:0/3124379753 conn(0x562eb6f2d3b0 msgr2=0x562eb6f2e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:05.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 -- 172.21.15.60:0/3124379753 shutdown_connections 2023-07-13T22:29:05.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 1 -- 172.21.15.60:0/3124379753 wait complete. 2023-07-13T22:29:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.403+0000 7fc6186a9b80 20 client.8070 trim_cache size 0 max 16384 2023-07-13T22:29:05.391 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:29:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:29:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:29:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7920, 2023-07-13T22:29:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:29:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:29:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:29:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.6713774340000001, 2023-07-13T22:29:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 62, 2023-07-13T22:29:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 100, 2023-07-13T22:29:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 100, 2023-07-13T22:29:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 13.097709834 2023-07-13T22:29:05.393 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:29:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 -- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5e006ace0 msgr2=0x7fc5e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5e006ace0 0x7fc5e006d190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc60015ee20 tx=0x7fc5f4000ed0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 -- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 msgr2=0x7fc60014a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc60014a000 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000f8b0 tx=0x7fc5f000fbc0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 -- 172.21.15.60:0/2104002981 shutdown_connections 2023-07-13T22:29:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5e006ace0 0x7fc5e006d190 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5e8004d90 0x7fc60014a000 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6000b35d0 0x7fc6000b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 --2- 172.21.15.60:0/2104002981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc60015ea50 0x7fc6000c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 -- 172.21.15.60:0/2104002981 >> 172.21.15.60:0/2104002981 conn(0x7fc60006b390 msgr2=0x7fc6000732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:05.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 -- 172.21.15.60:0/2104002981 shutdown_connections 2023-07-13T22:29:05.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.404+0000 7fc604dff700 1 -- 172.21.15.60:0/2104002981 wait complete. 2023-07-13T22:29:05.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:29:05.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.625+0000 7fbd7e1ff700 1 Processor -- start 2023-07-13T22:29:05.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.625+0000 7fbd7e1ff700 1 -- start start 2023-07-13T22:29:05.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.626+0000 7fbd7e1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd780b8400 0x7fbd780cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.626+0000 7fbd7e1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd780cfca0 0x7fbd780d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.626+0000 7fbd7e1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd780d0ac0 con 0x7fbd780d3750 2023-07-13T22:29:05.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.626+0000 7fbd7e1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd780b87d0 con 0x7fbd780cfca0 2023-07-13T22:29:05.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.626+0000 7fbd7e1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd780b4eb0 con 0x7fbd780b8400 2023-07-13T22:29:05.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd77fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd780cfca0 0x7fbd780d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd77fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd780cfca0 0x7fbd780d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40924/0 (socket says 172.21.15.60:40924) 2023-07-13T22:29:05.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd77fff700 1 -- 172.21.15.60:0/2517451474 learned_addr learned my addr 172.21.15.60:0/2517451474 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:05.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd777fe700 1 --2- 172.21.15.60:0/2517451474 >> v2:172.21.15.105:3300/0 conn(0x7fbd780b8400 0x7fbd780cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd77fff700 1 -- 172.21.15.60:0/2517451474 >> v2:172.21.15.105:3300/0 conn(0x7fbd780b8400 msgr2=0x7fbd780cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:05.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd77fff700 1 --2- 172.21.15.60:0/2517451474 >> v2:172.21.15.105:3300/0 conn(0x7fbd780b8400 0x7fbd780cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd77fff700 1 -- 172.21.15.60:0/2517451474 >> v1:172.21.15.105:6789/0 conn(0x7fbd780d3750 legacy=0x7fbd780cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd77fff700 1 -- 172.21.15.60:0/2517451474 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd780b5190 con 0x7fbd780cfca0 2023-07-13T22:29:05.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd777fe700 1 --2- 172.21.15.60:0/2517451474 >> v2:172.21.15.105:3300/0 conn(0x7fbd780b8400 0x7fbd780cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:05.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.627+0000 7fbd77fff700 1 --2- 172.21.15.60:0/2517451474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780cfca0 0x7fbd780d0f50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbd780cf560 tx=0x7fbd7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa155e313964e4d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd767fc700 1 -- 172.21.15.60:0/2517451474 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000e070 con 0x7fbd780cfca0 2023-07-13T22:29:05.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd767fc700 1 -- 172.21.15.60:0/2517451474 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7000b9c0 con 0x7fbd780cfca0 2023-07-13T22:29:05.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd767fc700 1 -- 172.21.15.60:0/2517451474 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000bd70 con 0x7fbd780cfca0 2023-07-13T22:29:05.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2517451474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780cfca0 msgr2=0x7fbd780d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd7e1ff700 1 --2- 172.21.15.60:0/2517451474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780cfca0 0x7fbd780d0f50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbd780cf560 tx=0x7fbd7000b330 comp rx=0 tx=0).stop 2023-07-13T22:29:05.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2517451474 shutdown_connections 2023-07-13T22:29:05.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd7e1ff700 1 --2- 172.21.15.60:0/2517451474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780cfca0 0x7fbd780d0f50 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd7e1ff700 1 --2- 172.21.15.60:0/2517451474 >> v2:172.21.15.105:3300/0 conn(0x7fbd780b8400 0x7fbd780cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2517451474 >> 172.21.15.60:0/2517451474 conn(0x7fbd7806b390 msgr2=0x7fbd7806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:05.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2517451474 shutdown_connections 2023-07-13T22:29:05.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.628+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2517451474 wait complete. 2023-07-13T22:29:05.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.630+0000 7fbd7e1ff700 1 Processor -- start 2023-07-13T22:29:05.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.630+0000 7fbd7e1ff700 1 -- start start 2023-07-13T22:29:05.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.631+0000 7fbd7e1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd780b8400 0x7fbd780c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.631+0000 7fbd7e1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd780cfca0 0x7fbd7814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.631+0000 7fbd7e1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780d3750 0x7fbd780b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.631+0000 7fbd7e1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7814d4e0 con 0x7fbd780d3750 2023-07-13T22:29:05.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.631+0000 7fbd7e1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780c9820 con 0x7fbd780cfca0 2023-07-13T22:29:05.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.631+0000 7fbd7e1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd781f57f0 con 0x7fbd780b8400 2023-07-13T22:29:05.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.631+0000 7fbd77fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780d3750 0x7fbd780b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.631+0000 7fbd77fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780d3750 0x7fbd780b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40930/0 (socket says 172.21.15.60:40930) 2023-07-13T22:29:05.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.631+0000 7fbd77fff700 1 -- 172.21.15.60:0/2265310602 learned_addr learned my addr 172.21.15.60:0/2265310602 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:05.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd76ffd700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd780cfca0 0x7fbd7814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd77fff700 1 -- 172.21.15.60:0/2265310602 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd780b8400 msgr2=0x7fbd780c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd77fff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd780b8400 0x7fbd780c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd77fff700 1 -- 172.21.15.60:0/2265310602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd780cfca0 msgr2=0x7fbd7814cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd77fff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd780cfca0 0x7fbd7814cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd77fff700 1 -- 172.21.15.60:0/2265310602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70009b30 con 0x7fbd780d3750 2023-07-13T22:29:05.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd76ffd700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd780cfca0 0x7fbd7814cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:05.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd77fff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780d3750 0x7fbd780b6a00 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b370 tx=0x7fbd70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd5ffff700 1 -- 172.21.15.60:0/2265310602 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd700099b0 con 0x7fbd780d3750 2023-07-13T22:29:05.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd5ffff700 1 -- 172.21.15.60:0/2265310602 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd700092e0 con 0x7fbd780d3750 2023-07-13T22:29:05.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd5ffff700 1 -- 172.21.15.60:0/2265310602 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7001c660 con 0x7fbd780d3750 2023-07-13T22:29:05.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.632+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd781f5ad0 con 0x7fbd780d3750 2023-07-13T22:29:05.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.633+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd781f6090 con 0x7fbd780d3750 2023-07-13T22:29:05.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.634+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd781f6250 con 0x7fbd780d3750 2023-07-13T22:29:05.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.635+0000 7fbd5ffff700 1 -- 172.21.15.60:0/2265310602 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbd7001c7c0 con 0x7fbd780d3750 2023-07-13T22:29:05.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.635+0000 7fbd5ffff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd6406ace0 0x7fbd6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:05.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.636+0000 7fbd5ffff700 1 -- 172.21.15.60:0/2265310602 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbd7008ae30 con 0x7fbd780d3750 2023-07-13T22:29:05.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.636+0000 7fbd777fe700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd6406ace0 0x7fbd6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:05.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.636+0000 7fbd777fe700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd6406ace0 0x7fbd6406d190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbd78157b80 tx=0x7fbd68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:05.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.640+0000 7fbd5ffff700 1 -- 172.21.15.60:0/2265310602 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbd70055ed0 con 0x7fbd780d3750 2023-07-13T22:29:05.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.893+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbd780ce740 con 0x7fbd780d3750 2023-07-13T22:29:05.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.894+0000 7fbd5ffff700 1 -- 172.21.15.60:0/2265310602 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbd700594f0 con 0x7fbd780d3750 2023-07-13T22:29:05.872 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:29:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd6406ace0 msgr2=0x7fbd6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd6406ace0 0x7fbd6406d190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbd78157b80 tx=0x7fbd68008040 comp rx=0 tx=0).stop 2023-07-13T22:29:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780d3750 msgr2=0x7fbd780b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780d3750 0x7fbd780b6a00 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b370 tx=0x7fbd70004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 shutdown_connections 2023-07-13T22:29:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbd6406ace0 0x7fbd6406d190 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd780d3750 0x7fbd780b6a00 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd780b8400 0x7fbd780c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 --2- 172.21.15.60:0/2265310602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd780cfca0 0x7fbd7814cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 >> 172.21.15.60:0/2265310602 conn(0x7fbd7806b390 msgr2=0x7fbd780d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:05.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 shutdown_connections 2023-07-13T22:29:05.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:05.896+0000 7fbd7e1ff700 1 -- 172.21.15.60:0/2265310602 wait complete. 2023-07-13T22:29:05.893 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:29:05.893 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:29:05.893 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:29:05.894 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:29:05.894 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:05.894 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:29:05.894 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:29:05.895 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:29:05.895 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:29:05.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:05.896 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:29:05.942 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:29:05.943 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:29:05.943 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:29:05.943 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:05.943 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:29:05.943 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:05.943 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:05.943 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:05.944 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:29:05.944 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:05.944 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:05.944 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:29:05.944 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:29:05.944 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:05.944 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:29:05.945 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:29:05.945 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41227sec preferred_lft 41227sec 2023-07-13T22:29:05.945 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:29:05.945 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:05.945 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:05.945 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:05.945 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:29:05.946 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:29:05.946 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:05.946 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:29:05.946 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:05.946 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:29:05.947 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:05.947 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:29:05.947 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:29:05.947 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:29:05.947 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:05.947 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:29:05.947 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:29:05.947 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:29:05.948 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:05.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:05.949 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:29:06.018 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:29:06.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:06.022 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:29:06.071 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:29:06.072 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:29:06.072 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:29:06.073 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:29:06.073 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:29:06.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:06.074 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:29:06.074 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:29:06.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:29:06.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:29:06.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:29:06.075 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:29:06.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:06.251 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:29:06.298 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:29:06.298 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:29:06.299 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:29:06.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:06.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:29:06.411 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:29:06.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:29:06.412 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:06.459 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:29:06.459 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:29:06.459 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:06.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:29:06.617 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:29:06.618 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:29:06.686 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:29:06.689 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:29:06.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:06.690 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:29:06.736 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:29:06.737 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:29:06.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:29:06.815 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:29:06.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:29:06.894 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:29:06.895 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:29:06.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:06.896 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:29:06.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:29:06.935+0000 7f75ef9605c0 -1 init, newargv = 0x55dbce466050 newargc=15 2023-07-13T22:29:06.914 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[152462]: starting ceph client 2023-07-13T22:29:06.922 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[152462]: starting fuse 2023-07-13T22:29:06.948 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:29:06.949 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:29:06.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:29:06.950 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:29:06.950 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:29:06.950 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:29:06.950 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:29:06.951 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:06.951 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:29:06.951 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:29:06.952 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:29:06.952 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:29:06.952 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:06.953 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:29:06.953 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:29:06.953 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:29:06.953 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:29:06.954 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:06.954 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:29:06.954 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:29:06.955 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:29:06.955 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:29:06.955 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:29:06.955 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:29:06.956 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:29:06.956 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:29:06.956 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:06.957 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:29:06.957 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:29:07.136 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.152462.asok 2023-07-13T22:29:07.313 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.152462.asok 2023-07-13T22:29:07.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:07.314 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.152462.asok status 2023-07-13T22:29:07.546 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:29:07.546 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:29:07.546 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:29:07.546 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:29:07.546 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:29:07.546 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:29:07.547 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:29:07.547 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "152462", 2023-07-13T22:29:07.547 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:29:07.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:07.547 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:29:07.547 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:29:07.547 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8082, 2023-07-13T22:29:07.547 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:29:07.548 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:29:07.548 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:29:07.548 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 8082 2023-07-13T22:29:07.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:07.548 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:29:07.548 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:29:07.549 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:29:07.549 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2416976467 2023-07-13T22:29:07.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:07.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:07.549 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:29:07.549 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:29:07.549 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:29:07.549 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2416976467 2023-07-13T22:29:07.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:07.550 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.8082 192.168.0.1:0/2416976467", 2023-07-13T22:29:07.550 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2416976467", 2023-07-13T22:29:07.550 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:29:07.550 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 62, 2023-07-13T22:29:07.550 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 100, 2023-07-13T22:29:07.550 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:29:07.550 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:29:07.551 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:29:07.551 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:29:07.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:07.561 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:29:07.562 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:07.610 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:29:07.610 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:07.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:07.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:07.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:29:07.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.947+0000 7f82bc9ff700 1 Processor -- start 2023-07-13T22:29:07.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.947+0000 7f82bc9ff700 1 -- start start 2023-07-13T22:29:07.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82bc9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82b8070270 0x7f82b80706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:07.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82bc9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c9c50 0x7f82b80ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:07.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82bc9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b815d970 con 0x7f82b8159170 2023-07-13T22:29:07.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82bc9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b8070c30 con 0x7f82b8070270 2023-07-13T22:29:07.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82bc9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b815cd60 con 0x7f82b80c9c50 2023-07-13T22:29:07.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82affff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82b8070270 0x7f82b80706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:07.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82b659c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f82b8159170 0x7f82b80c4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49176/0 (socket says 172.21.15.60:49176) 2023-07-13T22:29:07.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82b659c700 1 -- 172.21.15.60:0/2447752075 learned_addr learned my addr 172.21.15.60:0/2447752075 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:07.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82b6d9d700 1 --2- 172.21.15.60:0/2447752075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c9c50 0x7f82b80ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:07.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82affff700 1 -- 172.21.15.60:0/2447752075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c9c50 msgr2=0x7f82b80ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:07.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82affff700 1 --2- 172.21.15.60:0/2447752075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c9c50 0x7f82b80ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:07.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82affff700 1 -- 172.21.15.60:0/2447752075 >> v1:172.21.15.60:6789/0 conn(0x7f82b8159170 legacy=0x7f82b80c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:07.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82affff700 1 -- 172.21.15.60:0/2447752075 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b815d040 con 0x7f82b8070270 2023-07-13T22:29:07.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82b5d9b700 1 -- 172.21.15.60:0/2447752075 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092491238 0 0) 0x7f82b815d970 con 0x7f82b8159170 2023-07-13T22:29:07.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.949+0000 7f82b6d9d700 1 --2- 172.21.15.60:0/2447752075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c9c50 0x7f82b80ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:07.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.950+0000 7f82affff700 1 --2- 172.21.15.60:0/2447752075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 0x7f82b80706f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009840 tx=0x7f82a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4cca6b25ca939 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:07.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.950+0000 7f82b5d9b700 1 -- 172.21.15.60:0/2447752075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0011830 con 0x7f82b8070270 2023-07-13T22:29:07.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.950+0000 7f82b5d9b700 1 -- 172.21.15.60:0/2447752075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a0011990 con 0x7f82b8070270 2023-07-13T22:29:07.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.950+0000 7f82b5d9b700 1 -- 172.21.15.60:0/2447752075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0011d20 con 0x7f82b8070270 2023-07-13T22:29:07.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.950+0000 7f82bc9ff700 1 -- 172.21.15.60:0/2447752075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 msgr2=0x7f82b80706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:07.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.950+0000 7f82bc9ff700 1 --2- 172.21.15.60:0/2447752075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 0x7f82b80706f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009840 tx=0x7f82a0009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:07.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.951+0000 7f82bc9ff700 1 -- 172.21.15.60:0/2447752075 shutdown_connections 2023-07-13T22:29:07.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.951+0000 7f82bc9ff700 1 --2- 172.21.15.60:0/2447752075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b80c9c50 0x7f82b80ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:07.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.951+0000 7f82bc9ff700 1 --2- 172.21.15.60:0/2447752075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 0x7f82b80706f0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:07.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.951+0000 7f82bc9ff700 1 -- 172.21.15.60:0/2447752075 >> 172.21.15.60:0/2447752075 conn(0x7f82b80cf410 msgr2=0x7f82b80cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:07.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.951+0000 7f82bc9ff700 1 -- 172.21.15.60:0/2447752075 shutdown_connections 2023-07-13T22:29:07.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.951+0000 7f82bc9ff700 1 -- 172.21.15.60:0/2447752075 wait complete. 2023-07-13T22:29:07.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.952+0000 7f82bc9ff700 1 Processor -- start 2023-07-13T22:29:07.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.953+0000 7f82bc9ff700 1 -- start start 2023-07-13T22:29:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.953+0000 7f82bc9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 0x7f82b814a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.953+0000 7f82bc9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b80c9c50 0x7f82b80ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.953+0000 7f82bc9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b8159170 0x7f82b80caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.953+0000 7f82bc9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80c57c0 con 0x7f82b8070270 2023-07-13T22:29:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.953+0000 7f82bc9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80c5a10 con 0x7f82b80c9c50 2023-07-13T22:29:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.953+0000 7f82bc9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80c5cf0 con 0x7f82b8159170 2023-07-13T22:29:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 0x7f82b814a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:07.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 0x7f82b814a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40954/0 (socket says 172.21.15.60:40954) 2023-07-13T22:29:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b659c700 1 -- 172.21.15.60:0/4028993469 learned_addr learned my addr 172.21.15.60:0/4028993469 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82affff700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b80c9c50 0x7f82b80ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b6d9d700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b8159170 0x7f82b80caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b659c700 1 -- 172.21.15.60:0/4028993469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b8159170 msgr2=0x7f82b80caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b659c700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b8159170 0x7f82b80caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:07.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b659c700 1 -- 172.21.15.60:0/4028993469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b80c9c50 msgr2=0x7f82b80ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b659c700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b80c9c50 0x7f82b80ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b659c700 1 -- 172.21.15.60:0/4028993469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a00094a0 con 0x7f82b8070270 2023-07-13T22:29:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.954+0000 7f82b659c700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 0x7f82b814a1d0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f82a80011e0 tx=0x7f82a800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.955+0000 7f82aeffd700 1 -- 172.21.15.60:0/4028993469 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8004c80 con 0x7f82b8070270 2023-07-13T22:29:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.955+0000 7f82aeffd700 1 -- 172.21.15.60:0/4028993469 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a8004e00 con 0x7f82b8070270 2023-07-13T22:29:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.955+0000 7f82aeffd700 1 -- 172.21.15.60:0/4028993469 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8007310 con 0x7f82b8070270 2023-07-13T22:29:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.955+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b80c5fd0 con 0x7f82b8070270 2023-07-13T22:29:07.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.955+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b80c6590 con 0x7f82b8070270 2023-07-13T22:29:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.956+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b80673f0 con 0x7f82b8070270 2023-07-13T22:29:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.957+0000 7f82aeffd700 1 -- 172.21.15.60:0/4028993469 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82a8007470 con 0x7f82b8070270 2023-07-13T22:29:07.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.958+0000 7f82aeffd700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829806ace0 0x7f829806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:07.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.958+0000 7f82aeffd700 1 -- 172.21.15.60:0/4028993469 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82a8089150 con 0x7f82b8070270 2023-07-13T22:29:07.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.958+0000 7f82affff700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829806ace0 0x7f829806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:07.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.959+0000 7f82affff700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829806ace0 0x7f829806d190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f82a00090b0 tx=0x7f82a001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:07.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:07.962+0000 7f82aeffd700 1 -- 172.21.15.60:0/4028993469 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f82b80673f0 con 0x7f82b8070270 2023-07-13T22:29:08.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.214+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f82b80673f0 con 0x7f82b8070270 2023-07-13T22:29:08.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.215+0000 7f82aeffd700 1 -- 172.21.15.60:0/4028993469 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7f82b80673f0 con 0x7f82b8070270 2023-07-13T22:29:08.194 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:29:08.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829806ace0 msgr2=0x7f829806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:08.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829806ace0 0x7f829806d190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f82a00090b0 tx=0x7f82a001c040 comp rx=0 tx=0).stop 2023-07-13T22:29:08.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 msgr2=0x7f82b814a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:08.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 0x7f82b814a1d0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f82a80011e0 tx=0x7f82a800ca30 comp rx=0 tx=0).stop 2023-07-13T22:29:08.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 shutdown_connections 2023-07-13T22:29:08.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f829806ace0 0x7f829806d190 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8070270 0x7f82b814a1d0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82b8159170 0x7f82b80caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 --2- 172.21.15.60:0/4028993469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b80c9c50 0x7f82b80ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 >> 172.21.15.60:0/4028993469 conn(0x7f82b80cf410 msgr2=0x7f82b80cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 shutdown_connections 2023-07-13T22:29:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:08.217+0000 7f82bc9ff700 1 -- 172.21.15.60:0/4028993469 wait complete. 2023-07-13T22:29:08.213 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:29:08.213 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:29:08.214 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:29:08.214 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:29:08.214 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:08.215 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:29:08.215 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:29:08.215 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:29:08.215 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:29:08.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:08.216 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:29:08.268 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:29:08.268 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:29:08.269 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:29:08.269 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:08.269 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:29:08.269 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:08.269 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:08.269 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:08.269 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:29:08.269 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:08.270 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:08.270 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:29:08.270 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:08.270 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:08.270 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:29:08.270 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:08.270 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:29:08.270 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41222sec preferred_lft 41222sec 2023-07-13T22:29:08.271 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:29:08.271 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:08.271 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:29:08.271 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:08.271 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:29:08.271 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:08.271 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:29:08.272 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:08.272 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:29:08.272 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:29:08.272 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:29:08.272 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:08.272 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:29:08.272 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:29:08.272 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::14e3:bfff:fea8:bf57/64 scope link 2023-07-13T22:29:08.273 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:08.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:08.274 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:29:08.343 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:29:08.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:08.347 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:29:08.396 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:29:08.396 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:29:08.396 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:29:08.397 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:29:08.397 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:29:08.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:08.398 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:29:08.399 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:29:08.399 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:29:08.399 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:29:08.400 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:29:08.400 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:29:08.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:08.573 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:29:08.619 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:29:08.619 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:29:08.620 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:29:08.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:08.621 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:29:08.691 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:29:08.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:29:08.692 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:08.738 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:29:08.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:29:08.739 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:08.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:29:08.933 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:29:08.933 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:29:09.001 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:29:09.005 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:29:09.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:09.006 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:29:09.053 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:29:09.053 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:29:09.097 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:29:09.168 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:29:09.169 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:29:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:29:09.243 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:29:09.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:09.245 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:29:09.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:29:09.287+0000 7fbcd301a5c0 -1 init, newargv = 0x561841acfe10 newargc=15 2023-07-13T22:29:09.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[125439]: starting ceph client 2023-07-13T22:29:09.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[125439]: starting fuse 2023-07-13T22:29:09.296 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:29:09.296 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:29:09.297 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:29:09.297 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:29:09.297 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:29:09.298 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:29:09.298 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:29:09.298 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:29:09.299 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:29:09.299 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:29:09.300 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:29:09.300 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:29:09.300 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:29:09.300 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:29:09.301 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:29:09.301 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:29:09.301 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:29:09.302 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:29:09.302 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:29:09.302 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:29:09.303 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:29:09.303 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:29:09.303 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:29:09.303 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:29:09.304 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:29:09.304 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:29:09.305 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:29:09.305 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:29:09.305 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:29:09.480 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.125439.asok 2023-07-13T22:29:09.658 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.125439.asok 2023-07-13T22:29:09.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:09.659 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.125439.asok status 2023-07-13T22:29:09.882 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:29:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:29:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:29:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:29:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:29:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:29:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:29:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "125439", 2023-07-13T22:29:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:29:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:29:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:29:09.884 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:29:09.884 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 6698, 2023-07-13T22:29:09.884 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:29:09.884 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:29:09.884 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:29:09.885 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 6698 2023-07-13T22:29:09.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:29:09.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:29:09.885 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:29:09.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:29:09.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1045006933 2023-07-13T22:29:09.885 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:29:09.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:29:09.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:29:09.886 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:29:09.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:29:09.886 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1045006933 2023-07-13T22:29:09.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:29:09.886 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.6698 192.168.0.1:0/1045006933", 2023-07-13T22:29:09.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/1045006933", 2023-07-13T22:29:09.887 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:29:09.887 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 62, 2023-07-13T22:29:09.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 100, 2023-07-13T22:29:09.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:29:09.887 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:29:09.887 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:29:09.887 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:29:09.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:09.896 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:29:09.896 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:09.943 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:29:09.943 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:09.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:09.944 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:10.013 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-07-13T22:29:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.252+0000 7f23735ff700 1 Processor -- start 2023-07-13T22:29:10.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.252+0000 7f23735ff700 1 -- start start 2023-07-13T22:29:10.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.253+0000 7f23735ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f236c0cfd50 0x7f236c0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.253+0000 7f23735ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f236c0b1ac0 0x7f236c0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.253+0000 7f23735ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c0d1870 con 0x7f236c0b8200 2023-07-13T22:29:10.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.253+0000 7f23735ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0cf370 con 0x7f236c0cfd50 2023-07-13T22:29:10.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.253+0000 7f23735ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0d12b0 con 0x7f236c0b1ac0 2023-07-13T22:29:10.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2370b9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f236c0cfd50 0x7f236c0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2370b9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f236c0cfd50 0x7f236c0d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40956/0 (socket says 172.21.15.60:40956) 2023-07-13T22:29:10.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f237139b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f236c0b8200 0x7f236c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49178/0 (socket says 172.21.15.60:49178) 2023-07-13T22:29:10.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2370b9a700 1 -- 172.21.15.60:0/1063832177 learned_addr learned my addr 172.21.15.60:0/1063832177 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:10.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2371b9c700 1 --2- 172.21.15.60:0/1063832177 >> v2:172.21.15.105:3300/0 conn(0x7f236c0b1ac0 0x7f236c0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2370b9a700 1 -- 172.21.15.60:0/1063832177 >> v2:172.21.15.105:3300/0 conn(0x7f236c0b1ac0 msgr2=0x7f236c0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:10.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2370b9a700 1 --2- 172.21.15.60:0/1063832177 >> v2:172.21.15.105:3300/0 conn(0x7f236c0b1ac0 0x7f236c0d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2370b9a700 1 -- 172.21.15.60:0/1063832177 >> v1:172.21.15.60:6789/0 conn(0x7f236c0b8200 legacy=0x7f236c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2370b9a700 1 -- 172.21.15.60:0/1063832177 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0d1590 con 0x7f236c0cfd50 2023-07-13T22:29:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2363fff700 1 -- 172.21.15.60:0/1063832177 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329459510 0 0) 0x7f236c0d1870 con 0x7f236c0b8200 2023-07-13T22:29:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.254+0000 7f2370b9a700 1 --2- 172.21.15.60:0/1063832177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0cfd50 0x7f236c0d23c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f236c0b8d70 tx=0x7f235c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4991f65ef56fb83 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f2363fff700 1 -- 172.21.15.60:0/1063832177 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c011830 con 0x7f236c0cfd50 2023-07-13T22:29:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f2363fff700 1 -- 172.21.15.60:0/1063832177 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c011990 con 0x7f236c0cfd50 2023-07-13T22:29:10.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f2363fff700 1 -- 172.21.15.60:0/1063832177 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c011d20 con 0x7f236c0cfd50 2023-07-13T22:29:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f23735ff700 1 -- 172.21.15.60:0/1063832177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0cfd50 msgr2=0x7f236c0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f23735ff700 1 --2- 172.21.15.60:0/1063832177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0cfd50 0x7f236c0d23c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f236c0b8d70 tx=0x7f235c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f23735ff700 1 -- 172.21.15.60:0/1063832177 shutdown_connections 2023-07-13T22:29:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f23735ff700 1 --2- 172.21.15.60:0/1063832177 >> v2:172.21.15.105:3300/0 conn(0x7f236c0b1ac0 0x7f236c0d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f23735ff700 1 --2- 172.21.15.60:0/1063832177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0cfd50 0x7f236c0d23c0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f23735ff700 1 -- 172.21.15.60:0/1063832177 >> 172.21.15.60:0/1063832177 conn(0x7f236c06b2f0 msgr2=0x7f236c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:10.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f23735ff700 1 -- 172.21.15.60:0/1063832177 shutdown_connections 2023-07-13T22:29:10.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.255+0000 7f23735ff700 1 -- 172.21.15.60:0/1063832177 wait complete. 2023-07-13T22:29:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.257+0000 7f23735ff700 1 Processor -- start 2023-07-13T22:29:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.257+0000 7f23735ff700 1 -- start start 2023-07-13T22:29:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.258+0000 7f23735ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0b1ac0 0x7f236c155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.258+0000 7f23735ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8200 0x7f236c15d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.258+0000 7f23735ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfd50 0x7f236c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.258+0000 7f23735ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c156b40 con 0x7f236c0b8200 2023-07-13T22:29:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.258+0000 7f23735ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15dda0 con 0x7f236c0cfd50 2023-07-13T22:29:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.258+0000 7f23735ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15e080 con 0x7f236c0b1ac0 2023-07-13T22:29:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.258+0000 7f2370b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8200 0x7f236c15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.258+0000 7f2370b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8200 0x7f236c15d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40958/0 (socket says 172.21.15.60:40958) 2023-07-13T22:29:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.258+0000 7f2370b9a700 1 -- 172.21.15.60:0/1185131965 learned_addr learned my addr 172.21.15.60:0/1185131965 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f237139b700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0b1ac0 0x7f236c155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2371b9c700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfd50 0x7f236c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2370b9a700 1 -- 172.21.15.60:0/1185131965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0b1ac0 msgr2=0x7f236c155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2370b9a700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0b1ac0 0x7f236c155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2370b9a700 1 -- 172.21.15.60:0/1185131965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfd50 msgr2=0x7f236c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2370b9a700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfd50 0x7f236c15ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2370b9a700 1 -- 172.21.15.60:0/1185131965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c009550 con 0x7f236c0b8200 2023-07-13T22:29:10.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2370b9a700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8200 0x7f236c15d470 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f235c009c10 tx=0x7f235c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:10.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2361ffb700 1 -- 172.21.15.60:0/1185131965 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c004510 con 0x7f236c0b8200 2023-07-13T22:29:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2361ffb700 1 -- 172.21.15.60:0/1185131965 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c0036a0 con 0x7f236c0b8200 2023-07-13T22:29:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.259+0000 7f2361ffb700 1 -- 172.21.15.60:0/1185131965 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c003a10 con 0x7f236c0b8200 2023-07-13T22:29:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.260+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c15e360 con 0x7f236c0b8200 2023-07-13T22:29:10.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.260+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c156280 con 0x7f236c0b8200 2023-07-13T22:29:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.261+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c0b8940 con 0x7f236c0b8200 2023-07-13T22:29:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.262+0000 7f2361ffb700 1 -- 172.21.15.60:0/1185131965 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f235c003b70 con 0x7f236c0b8200 2023-07-13T22:29:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.263+0000 7f2361ffb700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235006ace0 0x7f235006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.263+0000 7f2361ffb700 1 -- 172.21.15.60:0/1185131965 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f235c021030 con 0x7f236c0b8200 2023-07-13T22:29:10.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.263+0000 7f237139b700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235006ace0 0x7f235006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.264+0000 7f237139b700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235006ace0 0x7f235006d190 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2368007fd0 tx=0x7f2368007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:10.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.268+0000 7f2361ffb700 1 -- 172.21.15.60:0/1185131965 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f235c055b40 con 0x7f236c0b8200 2023-07-13T22:29:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.473+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f236c04e210 con 0x7f236c0b8200 2023-07-13T22:29:10.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.478+0000 7f2361ffb700 1 -- 172.21.15.60:0/1185131965 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v35)=0 v35) v1 ==== 121+0+0 (secure 0 0 0) 0x7f235c059160 con 0x7f236c0b8200 2023-07-13T22:29:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235006ace0 msgr2=0x7f235006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235006ace0 0x7f235006d190 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2368007fd0 tx=0x7f2368007de0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8200 msgr2=0x7f236c15d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8200 0x7f236c15d470 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f235c009c10 tx=0x7f235c004750 comp rx=0 tx=0).stop 2023-07-13T22:29:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 shutdown_connections 2023-07-13T22:29:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f235006ace0 0x7f235006d190 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f236c0cfd50 0x7f236c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f236c0b1ac0 0x7f236c155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 --2- 172.21.15.60:0/1185131965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b8200 0x7f236c15d470 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.480+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 >> 172.21.15.60:0/1185131965 conn(0x7f236c06b2f0 msgr2=0x7f236c15c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.481+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 shutdown_connections 2023-07-13T22:29:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.481+0000 7f23735ff700 1 -- 172.21.15.60:0/1185131965 wait complete. 2023-07-13T22:29:10.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 400 2023-07-13T22:29:10.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.705+0000 7f4fd79ff700 1 Processor -- start 2023-07-13T22:29:10.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.705+0000 7f4fd79ff700 1 -- start start 2023-07-13T22:29:10.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.707+0000 7f4fd79ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4fd00b8200 0x7f4fd00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.707+0000 7f4fd79ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 0x7f4fd00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.707+0000 7f4fd79ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fd00cede0 0x7f4fd015ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.707+0000 7f4fd79ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd00d1870 con 0x7f4fd00cede0 2023-07-13T22:29:10.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.707+0000 7f4fd79ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd00cf370 con 0x7f4fd00b8200 2023-07-13T22:29:10.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.707+0000 7f4fd79ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd00d12b0 con 0x7f4fd00b1ac0 2023-07-13T22:29:10.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.707+0000 7f4fd5f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fd00cede0 0x7f4fd015ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.707+0000 7f4fd5f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fd00cede0 0x7f4fd015ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40972/0 (socket says 172.21.15.60:40972) 2023-07-13T22:29:10.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fd5f9c700 1 -- 172.21.15.60:0/1006902299 learned_addr learned my addr 172.21.15.60:0/1006902299 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:10.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fd579b700 1 --2- 172.21.15.60:0/1006902299 >> v2:172.21.15.105:3300/0 conn(0x7f4fd00b8200 0x7f4fd00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fd4f9a700 1 --2- 172.21.15.60:0/1006902299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 0x7f4fd00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fd5f9c700 1 -- 172.21.15.60:0/1006902299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 msgr2=0x7f4fd00cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fd5f9c700 1 --2- 172.21.15.60:0/1006902299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 0x7f4fd00cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fd5f9c700 1 -- 172.21.15.60:0/1006902299 >> v2:172.21.15.105:3300/0 conn(0x7f4fd00b8200 msgr2=0x7f4fd00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fd5f9c700 1 --2- 172.21.15.60:0/1006902299 >> v2:172.21.15.105:3300/0 conn(0x7f4fd00b8200 0x7f4fd00cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fd5f9c700 1 -- 172.21.15.60:0/1006902299 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd00d1590 con 0x7f4fd00cede0 2023-07-13T22:29:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fd5f9c700 1 --2- 172.21.15.60:0/1006902299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 0x7f4fd015ff60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00b2660 tx=0x7f4fc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84c83be4d8b08be server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.708+0000 7f4fc7fff700 1 -- 172.21.15.60:0/1006902299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800e070 con 0x7f4fd00cede0 2023-07-13T22:29:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fc7fff700 1 -- 172.21.15.60:0/1006902299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc800b910 con 0x7f4fd00cede0 2023-07-13T22:29:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fc7fff700 1 -- 172.21.15.60:0/1006902299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800bcc0 con 0x7f4fd00cede0 2023-07-13T22:29:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fd79ff700 1 -- 172.21.15.60:0/1006902299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 msgr2=0x7f4fd015ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/1006902299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 0x7f4fd015ff60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00b2660 tx=0x7f4fc800b330 comp rx=0 tx=0).stop 2023-07-13T22:29:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fd79ff700 1 -- 172.21.15.60:0/1006902299 shutdown_connections 2023-07-13T22:29:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/1006902299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 0x7f4fd015ff60 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/1006902299 >> v2:172.21.15.105:3300/0 conn(0x7f4fd00b8200 0x7f4fd00cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/1006902299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 0x7f4fd00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fd79ff700 1 -- 172.21.15.60:0/1006902299 >> 172.21.15.60:0/1006902299 conn(0x7f4fd006b2f0 msgr2=0x7f4fd006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fd79ff700 1 -- 172.21.15.60:0/1006902299 shutdown_connections 2023-07-13T22:29:10.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.709+0000 7f4fd79ff700 1 -- 172.21.15.60:0/1006902299 wait complete. 2023-07-13T22:29:10.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.712+0000 7f4fd79ff700 1 Processor -- start 2023-07-13T22:29:10.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.712+0000 7f4fd79ff700 1 -- start start 2023-07-13T22:29:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.713+0000 7f4fd79ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 0x7f4fd00c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.713+0000 7f4fd79ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd00b8200 0x7f4fd0158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.713+0000 7f4fd79ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 0x7f4fd0157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.713+0000 7f4fd79ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd01571c0 con 0x7f4fd00cede0 2023-07-13T22:29:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.713+0000 7f4fd79ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0157780 con 0x7f4fd00b8200 2023-07-13T22:29:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.713+0000 7f4fd79ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd00ca240 con 0x7f4fd00b1ac0 2023-07-13T22:29:10.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.713+0000 7f4fd5f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 0x7f4fd0157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.713+0000 7f4fd5f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 0x7f4fd0157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40986/0 (socket says 172.21.15.60:40986) 2023-07-13T22:29:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.713+0000 7f4fd5f9c700 1 -- 172.21.15.60:0/594397863 learned_addr learned my addr 172.21.15.60:0/594397863 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fd579b700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 0x7f4fd00c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fd4f9a700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd00b8200 0x7f4fd0158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fd5f9c700 1 -- 172.21.15.60:0/594397863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 msgr2=0x7f4fd00c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fd5f9c700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 0x7f4fd00c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fd5f9c700 1 -- 172.21.15.60:0/594397863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd00b8200 msgr2=0x7f4fd0158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fd5f9c700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd00b8200 0x7f4fd0158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fd5f9c700 1 -- 172.21.15.60:0/594397863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8009b30 con 0x7f4fd00cede0 2023-07-13T22:29:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fd5f9c700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 0x7f4fd0157c10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8003f50 tx=0x7f4fc8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fc5ffb700 1 -- 172.21.15.60:0/594397863 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8004450 con 0x7f4fd00cede0 2023-07-13T22:29:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fc5ffb700 1 -- 172.21.15.60:0/594397863 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc80045b0 con 0x7f4fd00cede0 2023-07-13T22:29:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fc5ffb700 1 -- 172.21.15.60:0/594397863 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8004960 con 0x7f4fd00cede0 2023-07-13T22:29:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.714+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd00ca8f0 con 0x7f4fd00cede0 2023-07-13T22:29:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.715+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd0157410 con 0x7f4fd00cede0 2023-07-13T22:29:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.716+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd004e210 con 0x7f4fd00cede0 2023-07-13T22:29:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.716+0000 7f4fc5ffb700 1 -- 172.21.15.60:0/594397863 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4fc801b430 con 0x7f4fd00cede0 2023-07-13T22:29:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.717+0000 7f4fc5ffb700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4fb406ace0 0x7f4fb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.717+0000 7f4fc5ffb700 1 -- 172.21.15.60:0/594397863 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fc808a770 con 0x7f4fd00cede0 2023-07-13T22:29:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.718+0000 7f4fd579b700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4fb406ace0 0x7f4fb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.718+0000 7f4fd579b700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4fb406ace0 0x7f4fb406d190 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009b60 tx=0x7f4fcc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:10.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.722+0000 7f4fc5ffb700 1 -- 172.21.15.60:0/594397863 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4fc80592a0 con 0x7f4fd00cede0 2023-07-13T22:29:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.928+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7f4fd01575a0 con 0x7f4fd00cede0 2023-07-13T22:29:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.932+0000 7f4fc5ffb700 1 -- 172.21.15.60:0/594397863 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v36)=0 v36) v1 ==== 119+0+0 (secure 0 0 0) 0x7f4fc8058e30 con 0x7f4fd00cede0 2023-07-13T22:29:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4fb406ace0 msgr2=0x7f4fb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4fb406ace0 0x7f4fb406d190 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009b60 tx=0x7f4fcc009210 comp rx=0 tx=0).stop 2023-07-13T22:29:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 msgr2=0x7f4fd0157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 0x7f4fd0157c10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8003f50 tx=0x7f4fc8004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 shutdown_connections 2023-07-13T22:29:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4fb406ace0 0x7f4fb406d190 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd00cede0 0x7f4fd0157c10 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fd00b1ac0 0x7f4fd00c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 --2- 172.21.15.60:0/594397863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fd00b8200 0x7f4fd0158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:10.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 >> 172.21.15.60:0/594397863 conn(0x7f4fd006b2f0 msgr2=0x7f4fd00b6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:10.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 shutdown_connections 2023-07-13T22:29:10.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:10.935+0000 7f4fd79ff700 1 -- 172.21.15.60:0/594397863 wait complete. 2023-07-13T22:29:10.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 800 2023-07-13T22:29:11.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.161+0000 7f2d28fff700 1 Processor -- start 2023-07-13T22:29:11.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.161+0000 7f2d28fff700 1 -- start start 2023-07-13T22:29:11.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.162+0000 7f2d28fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d240cfd50 0x7f2d240d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.162+0000 7f2d28fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 0x7f2d240d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.162+0000 7f2d28fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240d1870 con 0x7f2d240b8200 2023-07-13T22:29:11.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.162+0000 7f2d28fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d240cf370 con 0x7f2d240cfd50 2023-07-13T22:29:11.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.162+0000 7f2d28fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240d12b0 con 0x7f2d240b1ac0 2023-07-13T22:29:11.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.162+0000 7f2d2259c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2d240b8200 0x7f2d240cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49180/0 (socket says 172.21.15.60:49180) 2023-07-13T22:29:11.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.162+0000 7f2d2259c700 1 -- 172.21.15.60:0/1658801396 learned_addr learned my addr 172.21.15.60:0/1658801396 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:11.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.162+0000 7f2d1bfff700 1 --2- 172.21.15.60:0/1658801396 >> v2:172.21.15.60:3300/0 conn(0x7f2d240cfd50 0x7f2d240d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.162+0000 7f2d22d9d700 1 --2- 172.21.15.60:0/1658801396 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 0x7f2d240d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d1bfff700 1 -- 172.21.15.60:0/1658801396 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 msgr2=0x7f2d240d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:11.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d1bfff700 1 --2- 172.21.15.60:0/1658801396 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 0x7f2d240d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d21d9b700 1 -- 172.21.15.60:0/1658801396 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240953871 0 0) 0x7f2d240d1870 con 0x7f2d240b8200 2023-07-13T22:29:11.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d1bfff700 1 -- 172.21.15.60:0/1658801396 >> v1:172.21.15.60:6789/0 conn(0x7f2d240b8200 legacy=0x7f2d240cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d1bfff700 1 -- 172.21.15.60:0/1658801396 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d240d1590 con 0x7f2d240cfd50 2023-07-13T22:29:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d22d9d700 1 --2- 172.21.15.60:0/1658801396 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 0x7f2d240d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d1bfff700 1 --2- 172.21.15.60:0/1658801396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 0x7f2d240d23c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009840 tx=0x7f2d0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b607cf549d61d33 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d21d9b700 1 -- 172.21.15.60:0/1658801396 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c011830 con 0x7f2d240cfd50 2023-07-13T22:29:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d21d9b700 1 -- 172.21.15.60:0/1658801396 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c011990 con 0x7f2d240cfd50 2023-07-13T22:29:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.163+0000 7f2d21d9b700 1 -- 172.21.15.60:0/1658801396 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c011d20 con 0x7f2d240cfd50 2023-07-13T22:29:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.164+0000 7f2d28fff700 1 -- 172.21.15.60:0/1658801396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 msgr2=0x7f2d240d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.164+0000 7f2d28fff700 1 --2- 172.21.15.60:0/1658801396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 0x7f2d240d23c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009840 tx=0x7f2d0c009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:11.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.164+0000 7f2d28fff700 1 -- 172.21.15.60:0/1658801396 shutdown_connections 2023-07-13T22:29:11.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.164+0000 7f2d28fff700 1 --2- 172.21.15.60:0/1658801396 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 0x7f2d240d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.164+0000 7f2d28fff700 1 --2- 172.21.15.60:0/1658801396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 0x7f2d240d23c0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.164+0000 7f2d28fff700 1 -- 172.21.15.60:0/1658801396 >> 172.21.15.60:0/1658801396 conn(0x7f2d2406b2f0 msgr2=0x7f2d2406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:11.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.164+0000 7f2d28fff700 1 -- 172.21.15.60:0/1658801396 shutdown_connections 2023-07-13T22:29:11.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.164+0000 7f2d28fff700 1 -- 172.21.15.60:0/1658801396 wait complete. 2023-07-13T22:29:11.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.166+0000 7f2d28fff700 1 Processor -- start 2023-07-13T22:29:11.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.166+0000 7f2d28fff700 1 -- start start 2023-07-13T22:29:11.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d28fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 0x7f2d2414d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d28fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d240b8200 0x7f2d240b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d28fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 0x7f2d240b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d28fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241f6560 con 0x7f2d240cfd50 2023-07-13T22:29:11.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d28fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241f6980 con 0x7f2d240b8200 2023-07-13T22:29:11.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d28fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241f5bc0 con 0x7f2d240b1ac0 2023-07-13T22:29:11.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 0x7f2d240b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 0x7f2d240b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40994/0 (socket says 172.21.15.60:40994) 2023-07-13T22:29:11.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d22d9d700 1 -- 172.21.15.60:0/1511030512 learned_addr learned my addr 172.21.15.60:0/1511030512 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:11.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d2259c700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 0x7f2d2414d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d22d9d700 1 -- 172.21.15.60:0/1511030512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 msgr2=0x7f2d2414d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d1bfff700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d240b8200 0x7f2d240b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d22d9d700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 0x7f2d2414d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d22d9d700 1 -- 172.21.15.60:0/1511030512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d240b8200 msgr2=0x7f2d240b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d22d9d700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d240b8200 0x7f2d240b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d22d9d700 1 -- 172.21.15.60:0/1511030512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0094a0 con 0x7f2d240cfd50 2023-07-13T22:29:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.168+0000 7f2d1bfff700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d240b8200 0x7f2d240b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.169+0000 7f2d22d9d700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 0x7f2d240b6a40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c007ef0 tx=0x7f2d1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.169+0000 7f2d1affd700 1 -- 172.21.15.60:0/1511030512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c003a20 con 0x7f2d240cfd50 2023-07-13T22:29:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.169+0000 7f2d1affd700 1 -- 172.21.15.60:0/1511030512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c00f040 con 0x7f2d240cfd50 2023-07-13T22:29:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.169+0000 7f2d1affd700 1 -- 172.21.15.60:0/1511030512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c003dc0 con 0x7f2d240cfd50 2023-07-13T22:29:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.169+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d241f5ea0 con 0x7f2d240cfd50 2023-07-13T22:29:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.169+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2414d740 con 0x7f2d240cfd50 2023-07-13T22:29:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.170+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d241f66a0 con 0x7f2d240cfd50 2023-07-13T22:29:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.171+0000 7f2d1affd700 1 -- 172.21.15.60:0/1511030512 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2d1c004030 con 0x7f2d240cfd50 2023-07-13T22:29:11.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.172+0000 7f2d1affd700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d0406ace0 0x7f2d0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.172+0000 7f2d1affd700 1 -- 172.21.15.60:0/1511030512 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2d1c089700 con 0x7f2d240cfd50 2023-07-13T22:29:11.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.172+0000 7f2d2259c700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d0406ace0 0x7f2d0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.173+0000 7f2d2259c700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d0406ace0 0x7f2d0406d190 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000f60 tx=0x7f2d14007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:11.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.177+0000 7f2d1affd700 1 -- 172.21.15.60:0/1511030512 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2d241f66a0 con 0x7f2d240cfd50 2023-07-13T22:29:11.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.386+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7f2d241f66a0 con 0x7f2d240cfd50 2023-07-13T22:29:11.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.390+0000 7f2d1affd700 1 -- 172.21.15.60:0/1511030512 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v37)=0 v37) v1 ==== 128+0+0 (secure 0 0 0) 0x7f2d241f66a0 con 0x7f2d240cfd50 2023-07-13T22:29:11.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d0406ace0 msgr2=0x7f2d0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d0406ace0 0x7f2d0406d190 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000f60 tx=0x7f2d14007c00 comp rx=0 tx=0).stop 2023-07-13T22:29:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 msgr2=0x7f2d240b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 0x7f2d240b6a40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c007ef0 tx=0x7f2d1c00d3b0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 shutdown_connections 2023-07-13T22:29:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d0406ace0 0x7f2d0406d190 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d240cfd50 0x7f2d240b6a40 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d240b1ac0 0x7f2d2414d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 --2- 172.21.15.60:0/1511030512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d240b8200 0x7f2d240b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.392+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 >> 172.21.15.60:0/1511030512 conn(0x7f2d2406b2f0 msgr2=0x7f2d240d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.393+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 shutdown_connections 2023-07-13T22:29:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.393+0000 7f2d28fff700 1 -- 172.21.15.60:0/1511030512 wait complete. 2023-07-13T22:29:11.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-13T22:29:11.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.615+0000 7f02211ff700 1 Processor -- start 2023-07-13T22:29:11.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.615+0000 7f02211ff700 1 -- start start 2023-07-13T22:29:11.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.616+0000 7f02211ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0b8200 0x7f021c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.616+0000 7f02211ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f021c0b1ac0 0x7f021c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.617+0000 7f02211ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021c0d1870 con 0x7f021c0cede0 2023-07-13T22:29:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.617+0000 7f02211ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021c0cf370 con 0x7f021c0b1ac0 2023-07-13T22:29:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.617+0000 7f02211ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021c0d12b0 con 0x7f021c0b8200 2023-07-13T22:29:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.617+0000 7f021b59e700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f021c0cede0 0x7f021c15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49192/0 (socket says 172.21.15.60:49192) 2023-07-13T22:29:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.617+0000 7f021b59e700 1 -- 172.21.15.60:0/3382552884 learned_addr learned my addr 172.21.15.60:0/3382552884 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.617+0000 7f021a59c700 1 --2- 172.21.15.60:0/3382552884 >> v2:172.21.15.105:3300/0 conn(0x7f021c0b1ac0 0x7f021c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.617+0000 7f021ad9d700 1 --2- 172.21.15.60:0/3382552884 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0b8200 0x7f021c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.617+0000 7f0219d9b700 1 -- 172.21.15.60:0/3382552884 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333709473 0 0) 0x7f021c0d1870 con 0x7f021c0cede0 2023-07-13T22:29:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.617+0000 7f0219d9b700 1 -- 172.21.15.60:0/3382552884 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02040035b0 con 0x7f021c0cede0 2023-07-13T22:29:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.618+0000 7f0219d9b700 1 -- 172.21.15.60:0/3382552884 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3552749379 0 0) 0x7f02040035b0 con 0x7f021c0cede0 2023-07-13T22:29:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.618+0000 7f021ad9d700 1 -- 172.21.15.60:0/3382552884 >> v2:172.21.15.105:3300/0 conn(0x7f021c0b1ac0 msgr2=0x7f021c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.618+0000 7f021ad9d700 1 --2- 172.21.15.60:0/3382552884 >> v2:172.21.15.105:3300/0 conn(0x7f021c0b1ac0 0x7f021c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.618+0000 7f021ad9d700 1 -- 172.21.15.60:0/3382552884 >> v1:172.21.15.60:6789/0 conn(0x7f021c0cede0 legacy=0x7f021c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.618+0000 7f021ad9d700 1 -- 172.21.15.60:0/3382552884 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c0d1590 con 0x7f021c0b8200 2023-07-13T22:29:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.618+0000 7f021ad9d700 1 --2- 172.21.15.60:0/3382552884 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0b8200 0x7f021c0cf640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f020c009840 tx=0x7f020c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e4961d58e5ea17 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.618+0000 7f0219d9b700 1 -- 172.21.15.60:0/3382552884 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140019f0 con 0x7f021c0b8200 2023-07-13T22:29:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.618+0000 7f0219d9b700 1 -- 172.21.15.60:0/3382552884 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c004030 con 0x7f021c0b8200 2023-07-13T22:29:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.618+0000 7f0219d9b700 1 -- 172.21.15.60:0/3382552884 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c0043c0 con 0x7f021c0b8200 2023-07-13T22:29:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.619+0000 7f02211ff700 1 -- 172.21.15.60:0/3382552884 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0b8200 msgr2=0x7f021c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.619+0000 7f02211ff700 1 --2- 172.21.15.60:0/3382552884 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0b8200 0x7f021c0cf640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f020c009840 tx=0x7f020c009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.619+0000 7f02211ff700 1 -- 172.21.15.60:0/3382552884 shutdown_connections 2023-07-13T22:29:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.619+0000 7f02211ff700 1 --2- 172.21.15.60:0/3382552884 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0b8200 0x7f021c0cf640 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.619+0000 7f02211ff700 1 --2- 172.21.15.60:0/3382552884 >> v2:172.21.15.105:3300/0 conn(0x7f021c0b1ac0 0x7f021c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.619+0000 7f02211ff700 1 -- 172.21.15.60:0/3382552884 >> 172.21.15.60:0/3382552884 conn(0x7f021c06b2f0 msgr2=0x7f021c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.619+0000 7f02211ff700 1 -- 172.21.15.60:0/3382552884 shutdown_connections 2023-07-13T22:29:11.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.619+0000 7f02211ff700 1 -- 172.21.15.60:0/3382552884 wait complete. 2023-07-13T22:29:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.621+0000 7f02211ff700 1 Processor -- start 2023-07-13T22:29:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.621+0000 7f02211ff700 1 -- start start 2023-07-13T22:29:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.622+0000 7f02211ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f021c0b1ac0 0x7f021c1f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.622+0000 7f02211ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f021c0b8200 0x7f021c0c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.622+0000 7f02211ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0cede0 0x7f021c1e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.622+0000 7f02211ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c1f1b70 con 0x7f021c0b8200 2023-07-13T22:29:11.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.622+0000 7f02211ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c148c20 con 0x7f021c0b1ac0 2023-07-13T22:29:11.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.622+0000 7f02211ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021c148e70 con 0x7f021c0cede0 2023-07-13T22:29:11.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.622+0000 7f021a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f021c0b8200 0x7f021c0c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.622+0000 7f021a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f021c0b8200 0x7f021c0c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41000/0 (socket says 172.21.15.60:41000) 2023-07-13T22:29:11.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.622+0000 7f021a59c700 1 -- 172.21.15.60:0/3054750446 learned_addr learned my addr 172.21.15.60:0/3054750446 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:11.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f021a59c700 1 -- 172.21.15.60:0/3054750446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0cede0 msgr2=0x7f021c1e63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f021a59c700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0cede0 0x7f021c1e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f021a59c700 1 -- 172.21.15.60:0/3054750446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f021c0b1ac0 msgr2=0x7f021c1f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f021a59c700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f021c0b1ac0 0x7f021c1f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f021a59c700 1 -- 172.21.15.60:0/3054750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0094a0 con 0x7f021c0b8200 2023-07-13T22:29:11.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f021a59c700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f021c0b8200 0x7f021c0c7a80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f021000e9b0 tx=0x7f021000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:11.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f02037fe700 1 -- 172.21.15.60:0/3054750446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210009710 con 0x7f021c0b8200 2023-07-13T22:29:11.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f02037fe700 1 -- 172.21.15.60:0/3054750446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021000c850 con 0x7f021c0b8200 2023-07-13T22:29:11.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f02037fe700 1 -- 172.21.15.60:0/3054750446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021000cbc0 con 0x7f021c0b8200 2023-07-13T22:29:11.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c149150 con 0x7f021c0b8200 2023-07-13T22:29:11.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.623+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c149710 con 0x7f021c0b8200 2023-07-13T22:29:11.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.624+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021c04e210 con 0x7f021c0b8200 2023-07-13T22:29:11.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.625+0000 7f02037fe700 1 -- 172.21.15.60:0/3054750446 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f021000cd20 con 0x7f021c0b8200 2023-07-13T22:29:11.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.626+0000 7f02037fe700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f020806ace0 0x7f020806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:11.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.626+0000 7f02037fe700 1 -- 172.21.15.60:0/3054750446 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0210089720 con 0x7f021c0b8200 2023-07-13T22:29:11.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.626+0000 7f021ad9d700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f020806ace0 0x7f020806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:11.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.627+0000 7f021ad9d700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f020806ace0 0x7f020806d190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f020c000c00 tx=0x7f020c010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:11.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.630+0000 7f02037fe700 1 -- 172.21.15.60:0/3054750446 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0210058130 con 0x7f021c0b8200 2023-07-13T22:29:11.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.837+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7f021c0b4f70 con 0x7f021c0b8200 2023-07-13T22:29:11.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.838+0000 7f02037fe700 1 -- 172.21.15.60:0/3054750446 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v37) v1 ==== 106+0+4 (secure 0 0 0) 0x7f0210057cc0 con 0x7f021c0b8200 2023-07-13T22:29:11.818 INFO:teuthology.orchestra.run.smithi060.stdout:100 2023-07-13T22:29:11.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f020806ace0 msgr2=0x7f020806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f020806ace0 0x7f020806d190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f020c000c00 tx=0x7f020c010000 comp rx=0 tx=0).stop 2023-07-13T22:29:11.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f021c0b8200 msgr2=0x7f021c0c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:11.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f021c0b8200 0x7f021c0c7a80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f021000e9b0 tx=0x7f021000ed70 comp rx=0 tx=0).stop 2023-07-13T22:29:11.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 shutdown_connections 2023-07-13T22:29:11.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f020806ace0 0x7f020806d190 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f021c0cede0 0x7f021c1e63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f021c0b1ac0 0x7f021c1f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 --2- 172.21.15.60:0/3054750446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f021c0b8200 0x7f021c0c7a80 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:11.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.840+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 >> 172.21.15.60:0/3054750446 conn(0x7f021c06b2f0 msgr2=0x7f021c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:11.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.841+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 shutdown_connections 2023-07-13T22:29:11.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:11.841+0000 7f02211ff700 1 -- 172.21.15.60:0/3054750446 wait complete. 2023-07-13T22:29:11.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2023-07-13T22:29:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.065+0000 7fda7ffff700 1 Processor -- start 2023-07-13T22:29:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.066+0000 7fda7ffff700 1 -- start start 2023-07-13T22:29:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda78158930 0x7fda780c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda7806a5e0 0x7fda7815c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7ffff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7806d030 con 0x7fda7815c4a0 2023-07-13T22:29:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7ffff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda780c9620 con 0x7fda78158930 2023-07-13T22:29:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7806ca70 con 0x7fda7806a5e0 2023-07-13T22:29:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7dd9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda78158930 0x7fda780c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7dd9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda78158930 0x7fda780c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41004/0 (socket says 172.21.15.60:41004) 2023-07-13T22:29:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7dd9b700 1 -- 172.21.15.60:0/2407010241 learned_addr learned my addr 172.21.15.60:0/2407010241 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7e59c700 1 --2- 172.21.15.60:0/2407010241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda7806a5e0 0x7fda7815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7dd9b700 1 -- 172.21.15.60:0/2407010241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda7806a5e0 msgr2=0x7fda7815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7dd9b700 1 --2- 172.21.15.60:0/2407010241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda7806a5e0 0x7fda7815c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.067+0000 7fda7dd9b700 1 -- 172.21.15.60:0/2407010241 >> v1:172.21.15.105:6789/0 conn(0x7fda7815c4a0 legacy=0x7fda78069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.068+0000 7fda7dd9b700 1 -- 172.21.15.60:0/2407010241 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780adc20 con 0x7fda78158930 2023-07-13T22:29:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.068+0000 7fda7e59c700 1 --2- 172.21.15.60:0/2407010241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda7806a5e0 0x7fda7815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.068+0000 7fda7dd9b700 1 --2- 172.21.15.60:0/2407010241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda78158930 0x7fda780c90e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fda7400aa90 tx=0x7fda7400ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927355b05a327849 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.068+0000 7fda7cd99700 1 -- 172.21.15.60:0/2407010241 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7400cb70 con 0x7fda78158930 2023-07-13T22:29:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.068+0000 7fda7cd99700 1 -- 172.21.15.60:0/2407010241 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7400ccd0 con 0x7fda78158930 2023-07-13T22:29:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.068+0000 7fda7cd99700 1 -- 172.21.15.60:0/2407010241 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74018620 con 0x7fda78158930 2023-07-13T22:29:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.069+0000 7fda7ffff700 1 -- 172.21.15.60:0/2407010241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda78158930 msgr2=0x7fda780c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.069+0000 7fda7ffff700 1 --2- 172.21.15.60:0/2407010241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda78158930 0x7fda780c90e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fda7400aa90 tx=0x7fda7400ada0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.069+0000 7fda7ffff700 1 -- 172.21.15.60:0/2407010241 shutdown_connections 2023-07-13T22:29:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.069+0000 7fda7ffff700 1 --2- 172.21.15.60:0/2407010241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda7806a5e0 0x7fda7815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.069+0000 7fda7ffff700 1 --2- 172.21.15.60:0/2407010241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda78158930 0x7fda780c90e0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.069+0000 7fda7ffff700 1 -- 172.21.15.60:0/2407010241 >> 172.21.15.60:0/2407010241 conn(0x7fda780cbf50 msgr2=0x7fda780ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.069+0000 7fda7ffff700 1 -- 172.21.15.60:0/2407010241 shutdown_connections 2023-07-13T22:29:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.069+0000 7fda7ffff700 1 -- 172.21.15.60:0/2407010241 wait complete. 2023-07-13T22:29:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.071+0000 7fda7ffff700 1 Processor -- start 2023-07-13T22:29:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.071+0000 7fda7ffff700 1 -- start start 2023-07-13T22:29:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda7806a5e0 0x7fda780ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda78158930 0x7fda78151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda7815c4a0 0x7fda7814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda78152b00 con 0x7fda7815c4a0 2023-07-13T22:29:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7ffff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda780aedf0 con 0x7fda7806a5e0 2023-07-13T22:29:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda781522a0 con 0x7fda78158930 2023-07-13T22:29:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda7815c4a0 0x7fda7814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda7815c4a0 0x7fda7814afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41020/0 (socket says 172.21.15.60:41020) 2023-07-13T22:29:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7e59c700 1 -- 172.21.15.60:0/2933228669 learned_addr learned my addr 172.21.15.60:0/2933228669 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7d59a700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda78158930 0x7fda78151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7dd9b700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda7806a5e0 0x7fda780ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7e59c700 1 -- 172.21.15.60:0/2933228669 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda78158930 msgr2=0x7fda78151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7e59c700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda78158930 0x7fda78151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7e59c700 1 -- 172.21.15.60:0/2933228669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda7806a5e0 msgr2=0x7fda780ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7e59c700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda7806a5e0 0x7fda780ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.073+0000 7fda7e59c700 1 -- 172.21.15.60:0/2933228669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7400a6f0 con 0x7fda7815c4a0 2023-07-13T22:29:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.074+0000 7fda7e59c700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda7815c4a0 0x7fda7814afc0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fda70007e10 tx=0x7fda7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.074+0000 7fda6e7fc700 1 -- 172.21.15.60:0/2933228669 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7000dd20 con 0x7fda7815c4a0 2023-07-13T22:29:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.074+0000 7fda6e7fc700 1 -- 172.21.15.60:0/2933228669 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7000f040 con 0x7fda7815c4a0 2023-07-13T22:29:12.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.074+0000 7fda6e7fc700 1 -- 172.21.15.60:0/2933228669 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70003c60 con 0x7fda7815c4a0 2023-07-13T22:29:12.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.074+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda78152540 con 0x7fda7815c4a0 2023-07-13T22:29:12.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.074+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda78150d20 con 0x7fda7815c4a0 2023-07-13T22:29:12.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.075+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda78150b80 con 0x7fda7815c4a0 2023-07-13T22:29:12.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.076+0000 7fda6e7fc700 1 -- 172.21.15.60:0/2933228669 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda70004030 con 0x7fda7815c4a0 2023-07-13T22:29:12.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.077+0000 7fda6e7fc700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda5c06ace0 0x7fda5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.077+0000 7fda6e7fc700 1 -- 172.21.15.60:0/2933228669 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda70089470 con 0x7fda7815c4a0 2023-07-13T22:29:12.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.078+0000 7fda7dd9b700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda5c06ace0 0x7fda5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.078+0000 7fda7dd9b700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda5c06ace0 0x7fda5c06d190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fda74000c00 tx=0x7fda74012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:12.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.082+0000 7fda6e7fc700 1 -- 172.21.15.60:0/2933228669 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fda78150b80 con 0x7fda7815c4a0 2023-07-13T22:29:12.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.293+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7fda78150b80 con 0x7fda7815c4a0 2023-07-13T22:29:12.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.297+0000 7fda6e7fc700 1 -- 172.21.15.60:0/2933228669 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v38)=0 v38) v1 ==== 124+0+0 (secure 0 0 0) 0x7fda78150b80 con 0x7fda7815c4a0 2023-07-13T22:29:12.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda5c06ace0 msgr2=0x7fda5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda5c06ace0 0x7fda5c06d190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fda74000c00 tx=0x7fda74012000 comp rx=0 tx=0).stop 2023-07-13T22:29:12.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda7815c4a0 msgr2=0x7fda7814afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda7815c4a0 0x7fda7814afc0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fda70007e10 tx=0x7fda7000d3b0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 shutdown_connections 2023-07-13T22:29:12.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda5c06ace0 0x7fda5c06d190 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda7815c4a0 0x7fda7814afc0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda7806a5e0 0x7fda780ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 --2- 172.21.15.60:0/2933228669 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda78158930 0x7fda78151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 >> 172.21.15.60:0/2933228669 conn(0x7fda780cbf50 msgr2=0x7fda780b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:12.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 shutdown_connections 2023-07-13T22:29:12.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.300+0000 7fda7ffff700 1 -- 172.21.15.60:0/2933228669 wait complete. 2023-07-13T22:29:12.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2023-07-13T22:29:12.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.524+0000 7f868a1ff700 1 Processor -- start 2023-07-13T22:29:12.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.524+0000 7f868a1ff700 1 -- start start 2023-07-13T22:29:12.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.525+0000 7f868a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86840b8200 0x7f86840cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.525+0000 7f868a1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f86840cede0 0x7f86840cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.525+0000 7f868a1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840d1870 con 0x7f86840b1ac0 2023-07-13T22:29:12.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.525+0000 7f868a1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840cf370 con 0x7f86840cede0 2023-07-13T22:29:12.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.525+0000 7f868a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840d12b0 con 0x7f86840b8200 2023-07-13T22:29:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f8682ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f86840b1ac0 0x7f868415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49204/0 (socket says 172.21.15.60:49204) 2023-07-13T22:29:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f8682ffd700 1 -- 172.21.15.60:0/278593501 learned_addr learned my addr 172.21.15.60:0/278593501 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86837fe700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86840b8200 0x7f86840cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277357864 0 0) 0x7f86840d1870 con 0x7f86840b1ac0 2023-07-13T22:29:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f866c0035b0 con 0x7f86840b1ac0 2023-07-13T22:29:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f8683fff700 1 --2- 172.21.15.60:0/278593501 >> v2:172.21.15.105:3300/0 conn(0x7f86840cede0 0x7f86840cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 203233009 0 0) 0x7f866c0035b0 con 0x7f86840b1ac0 2023-07-13T22:29:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86840b8200 msgr2=0x7f86840cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86827fc700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86840b8200 0x7f86840cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 >> v2:172.21.15.105:3300/0 conn(0x7f86840cede0 msgr2=0x7f86840cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86827fc700 1 --2- 172.21.15.60:0/278593501 >> v2:172.21.15.105:3300/0 conn(0x7f86840cede0 0x7f86840cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840d1590 con 0x7f86840b1ac0 2023-07-13T22:29:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f8678001670 con 0x7f86840b1ac0 2023-07-13T22:29:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.526+0000 7f8683fff700 1 --2- 172.21.15.60:0/278593501 >> v2:172.21.15.105:3300/0 conn(0x7f86840cede0 0x7f86840cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 >> v1:172.21.15.60:6789/0 conn(0x7f86840b1ac0 legacy=0x7f868415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86827fc700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f866c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86827fc700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86840b8200 0x7f866c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86827fc700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f866c004d90 0x7f866c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840cf370 con 0x7f866c003a00 2023-07-13T22:29:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840d12b0 con 0x7f866c004d90 2023-07-13T22:29:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c00ba40 con 0x7f86840b8200 2023-07-13T22:29:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86837fe700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f866c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f8682ffd700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86840b8200 0x7f866c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f8683fff700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f866c004d90 0x7f866c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86837fe700 1 -- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86840b8200 msgr2=0x7f866c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86837fe700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86840b8200 0x7f866c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86837fe700 1 -- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f866c004d90 msgr2=0x7f866c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86837fe700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f866c004d90 0x7f866c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86837fe700 1 -- 172.21.15.60:0/278593501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0035b0 con 0x7f866c003a00 2023-07-13T22:29:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f8683fff700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f866c004d90 0x7f866c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.527+0000 7f86837fe700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f866c003e10 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8674009b60 tx=0x7f866c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b611b445b44be7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400e660 con 0x7f866c003a00 2023-07-13T22:29:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867400e7c0 con 0x7f866c003a00 2023-07-13T22:29:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f86827fc700 1 -- 172.21.15.60:0/278593501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400eb30 con 0x7f866c003a00 2023-07-13T22:29:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f868a1ff700 1 -- 172.21.15.60:0/278593501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 msgr2=0x7f866c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f868a1ff700 1 --2- 172.21.15.60:0/278593501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f866c003e10 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8674009b60 tx=0x7f866c005880 comp rx=0 tx=0).stop 2023-07-13T22:29:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f8683fff700 1 -- 172.21.15.60:0/278593501 reap_dead start 2023-07-13T22:29:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f868a1ff700 1 -- 172.21.15.60:0/278593501 shutdown_connections 2023-07-13T22:29:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f868a1ff700 1 -- 172.21.15.60:0/278593501 >> 172.21.15.60:0/278593501 conn(0x7f868406b2f0 msgr2=0x7f868406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f868a1ff700 1 -- 172.21.15.60:0/278593501 shutdown_connections 2023-07-13T22:29:12.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.528+0000 7f868a1ff700 1 -- 172.21.15.60:0/278593501 wait complete. 2023-07-13T22:29:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.530+0000 7f868a1ff700 1 Processor -- start 2023-07-13T22:29:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.530+0000 7f868a1ff700 1 -- start start 2023-07-13T22:29:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f868a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f86841557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f868a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86841f5310 0x7f86841f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f868a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86840b65e0 0x7f86840b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f868a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684155d10 con 0x7f866c003a00 2023-07-13T22:29:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f868a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840cb020 con 0x7f86840b65e0 2023-07-13T22:29:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f868a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840b6f90 con 0x7f86841f5310 2023-07-13T22:29:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f86837fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f86841557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f86837fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f86841557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41034/0 (socket says 172.21.15.60:41034) 2023-07-13T22:29:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f86837fe700 1 -- 172.21.15.60:0/1611616649 learned_addr learned my addr 172.21.15.60:0/1611616649 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.531+0000 7f8682ffd700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86841f5310 0x7f86841f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f8683fff700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86840b65e0 0x7f86840b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f86837fe700 1 -- 172.21.15.60:0/1611616649 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86841f5310 msgr2=0x7f86841f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f86837fe700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86841f5310 0x7f86841f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f86837fe700 1 -- 172.21.15.60:0/1611616649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86840b65e0 msgr2=0x7f86840b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f86837fe700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86840b65e0 0x7f86840b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f86837fe700 1 -- 172.21.15.60:0/1611616649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86841f5d90 con 0x7f866c003a00 2023-07-13T22:29:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f86837fe700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f86841557d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f867400e4b0 tx=0x7f86740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f866bfff700 1 -- 172.21.15.60:0/1611616649 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400e660 con 0x7f866c003a00 2023-07-13T22:29:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f866bfff700 1 -- 172.21.15.60:0/1611616649 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86740038a0 con 0x7f866c003a00 2023-07-13T22:29:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f866bfff700 1 -- 172.21.15.60:0/1611616649 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674003c50 con 0x7f866c003a00 2023-07-13T22:29:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.532+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86841f6070 con 0x7f866c003a00 2023-07-13T22:29:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.533+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86840cb510 con 0x7f866c003a00 2023-07-13T22:29:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.534+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86840cb880 con 0x7f866c003a00 2023-07-13T22:29:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.535+0000 7f866bfff700 1 -- 172.21.15.60:0/1611616649 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8674013470 con 0x7f866c003a00 2023-07-13T22:29:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.535+0000 7f866bfff700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f867006ace0 0x7f867006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.536+0000 7f866bfff700 1 -- 172.21.15.60:0/1611616649 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f867408ab80 con 0x7f866c003a00 2023-07-13T22:29:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.536+0000 7f8682ffd700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f867006ace0 0x7f867006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.536+0000 7f8682ffd700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f867006ace0 0x7f867006d190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f86840cafb0 tx=0x7f8678007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.540+0000 7f866bfff700 1 -- 172.21.15.60:0/1611616649 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8674055c20 con 0x7f866c003a00 2023-07-13T22:29:12.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.749+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) v1 -- 0x7f86841f6200 con 0x7f866c003a00 2023-07-13T22:29:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.749+0000 7f866bfff700 1 -- 172.21.15.60:0/1611616649 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v38) v1 ==== 106+0+8 (secure 0 0 0) 0x7f8674059240 con 0x7f866c003a00 2023-07-13T22:29:12.729 INFO:teuthology.orchestra.run.smithi060.stdout:1048576 2023-07-13T22:29:12.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.751+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f867006ace0 msgr2=0x7f867006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.751+0000 7f868a1ff700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f867006ace0 0x7f867006d190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f86840cafb0 tx=0x7f8678007040 comp rx=0 tx=0).stop 2023-07-13T22:29:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.751+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 msgr2=0x7f86841557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.751+0000 7f868a1ff700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f86841557d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f867400e4b0 tx=0x7f86740047c0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.752+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 shutdown_connections 2023-07-13T22:29:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.752+0000 7f868a1ff700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f867006ace0 0x7f867006d190 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.752+0000 7f868a1ff700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f866c003a00 0x7f86841557d0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.752+0000 7f868a1ff700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86840b65e0 0x7f86840b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.752+0000 7f868a1ff700 1 --2- 172.21.15.60:0/1611616649 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86841f5310 0x7f86841f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.752+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 >> 172.21.15.60:0/1611616649 conn(0x7f868406b2f0 msgr2=0x7f86840b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.752+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 shutdown_connections 2023-07-13T22:29:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.752+0000 7f868a1ff700 1 -- 172.21.15.60:0/1611616649 wait complete. 2023-07-13T22:29:12.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-07-13T22:29:12.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.975+0000 7f42c7fff700 1 Processor -- start 2023-07-13T22:29:12.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.975+0000 7f42c7fff700 1 -- start start 2023-07-13T22:29:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.976+0000 7f42c7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42c00add00 0x7f42c006ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.976+0000 7f42c7fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c00c8e90 con 0x7f42c006a5e0 2023-07-13T22:29:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.976+0000 7f42c7fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c00ae0d0 con 0x7f42c00ca9f0 2023-07-13T22:29:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.976+0000 7f42c7fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c006d3b0 con 0x7f42c00add00 2023-07-13T22:29:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.976+0000 7f42c5d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42c00add00 0x7f42c006ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.977+0000 7f42c5d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42c00add00 0x7f42c006ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41046/0 (socket says 172.21.15.60:41046) 2023-07-13T22:29:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.977+0000 7f42c5d9b700 1 -- 172.21.15.60:0/1179442683 learned_addr learned my addr 172.21.15.60:0/1179442683 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.977+0000 7f42c5d9b700 1 -- 172.21.15.60:0/1179442683 >> v1:172.21.15.105:6789/0 conn(0x7f42c00ca9f0 legacy=0x7f42c0069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.977+0000 7f42c5d9b700 1 -- 172.21.15.60:0/1179442683 >> v1:172.21.15.60:6789/0 conn(0x7f42c006a5e0 legacy=0x7f42c00b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.977+0000 7f42c5d9b700 1 -- 172.21.15.60:0/1179442683 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c00c90e0 con 0x7f42c00add00 2023-07-13T22:29:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.977+0000 7f42c5d9b700 1 --2- 172.21.15.60:0/1179442683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00add00 0x7f42c006ce70 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0098f0 tx=0x7f42bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e45cf7a395a6c16 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.977+0000 7f42c4d99700 1 -- 172.21.15.60:0/1179442683 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc004230 con 0x7f42c00add00 2023-07-13T22:29:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.978+0000 7f42c4d99700 1 -- 172.21.15.60:0/1179442683 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc004390 con 0x7f42c00add00 2023-07-13T22:29:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.978+0000 7f42c4d99700 1 -- 172.21.15.60:0/1179442683 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0036a0 con 0x7f42c00add00 2023-07-13T22:29:12.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.978+0000 7f42c7fff700 1 -- 172.21.15.60:0/1179442683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00add00 msgr2=0x7f42c006ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.978+0000 7f42c7fff700 1 --2- 172.21.15.60:0/1179442683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00add00 0x7f42c006ce70 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0098f0 tx=0x7f42bc009c00 comp rx=0 tx=0).stop 2023-07-13T22:29:12.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.978+0000 7f42c7fff700 1 -- 172.21.15.60:0/1179442683 shutdown_connections 2023-07-13T22:29:12.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.978+0000 7f42c7fff700 1 --2- 172.21.15.60:0/1179442683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00add00 0x7f42c006ce70 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.978+0000 7f42c7fff700 1 -- 172.21.15.60:0/1179442683 >> 172.21.15.60:0/1179442683 conn(0x7f42c00cbf50 msgr2=0x7f42c00ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:12.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.978+0000 7f42c7fff700 1 -- 172.21.15.60:0/1179442683 shutdown_connections 2023-07-13T22:29:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.978+0000 7f42c7fff700 1 -- 172.21.15.60:0/1179442683 wait complete. 2023-07-13T22:29:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.980+0000 7f42c7fff700 1 Processor -- start 2023-07-13T22:29:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.980+0000 7f42c7fff700 1 -- start start 2023-07-13T22:29:12.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.981+0000 7f42c7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42c006a5e0 0x7f42c00c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.981+0000 7f42c7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42c00add00 0x7f42c0158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.981+0000 7f42c7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00ca9f0 0x7f42c00cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.981+0000 7f42c7fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c00c4de0 con 0x7f42c00ca9f0 2023-07-13T22:29:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.981+0000 7f42c7fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c00cc5b0 con 0x7f42c00add00 2023-07-13T22:29:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.981+0000 7f42c7fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c00cc890 con 0x7f42c006a5e0 2023-07-13T22:29:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.981+0000 7f42c659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00ca9f0 0x7f42c00cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.981+0000 7f42c659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00ca9f0 0x7f42c00cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41058/0 (socket says 172.21.15.60:41058) 2023-07-13T22:29:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.981+0000 7f42c659c700 1 -- 172.21.15.60:0/4220143622 learned_addr learned my addr 172.21.15.60:0/4220143622 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c559a700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42c00add00 0x7f42c0158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c5d9b700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42c006a5e0 0x7f42c00c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c659c700 1 -- 172.21.15.60:0/4220143622 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42c006a5e0 msgr2=0x7f42c00c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c659c700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42c006a5e0 0x7f42c00c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c659c700 1 -- 172.21.15.60:0/4220143622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42c00add00 msgr2=0x7f42c0158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c659c700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42c00add00 0x7f42c0158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c659c700 1 -- 172.21.15.60:0/4220143622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc009550 con 0x7f42c00ca9f0 2023-07-13T22:29:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c559a700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42c00add00 0x7f42c0158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c659c700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00ca9f0 0x7f42c00cd7a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f42b800cbc0 tx=0x7f42b800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42b67fc700 1 -- 172.21.15.60:0/4220143622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b8003c00 con 0x7f42c00ca9f0 2023-07-13T22:29:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42b67fc700 1 -- 172.21.15.60:0/4220143622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b8003d60 con 0x7f42c00ca9f0 2023-07-13T22:29:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42b67fc700 1 -- 172.21.15.60:0/4220143622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b800ab40 con 0x7f42c00ca9f0 2023-07-13T22:29:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.982+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c00ccb70 con 0x7f42c00ca9f0 2023-07-13T22:29:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.983+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c00cd130 con 0x7f42c00ca9f0 2023-07-13T22:29:12.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.984+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c004e210 con 0x7f42c00ca9f0 2023-07-13T22:29:12.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.985+0000 7f42b67fc700 1 -- 172.21.15.60:0/4220143622 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42b800aca0 con 0x7f42c00ca9f0 2023-07-13T22:29:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.986+0000 7f42b67fc700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42a406ace0 0x7f42a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.986+0000 7f42b67fc700 1 -- 172.21.15.60:0/4220143622 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f42b80894e0 con 0x7f42c00ca9f0 2023-07-13T22:29:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.986+0000 7f42c5d9b700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42a406ace0 0x7f42a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.986+0000 7f42c5d9b700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42a406ace0 0x7f42a406d190 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009130 tx=0x7f42bc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:12.990+0000 7f42b67fc700 1 -- 172.21.15.60:0/4220143622 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f42b8054590 con 0x7f42c00ca9f0 2023-07-13T22:29:13.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.195+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7f42c015c610 con 0x7f42c00ca9f0 2023-07-13T22:29:13.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.196+0000 7f42b67fc700 1 -- 172.21.15.60:0/4220143622 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v38) v1 ==== 112+0+10 (secure 0 0 0) 0x7f42b8057bb0 con 0x7f42c00ca9f0 2023-07-13T22:29:13.176 INFO:teuthology.orchestra.run.smithi060.stdout:60.000000 2023-07-13T22:29:13.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42a406ace0 msgr2=0x7f42a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:13.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42a406ace0 0x7f42a406d190 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009130 tx=0x7f42bc008030 comp rx=0 tx=0).stop 2023-07-13T22:29:13.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00ca9f0 msgr2=0x7f42c00cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:13.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00ca9f0 0x7f42c00cd7a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f42b800cbc0 tx=0x7f42b800ced0 comp rx=0 tx=0).stop 2023-07-13T22:29:13.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 shutdown_connections 2023-07-13T22:29:13.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f42a406ace0 0x7f42a406d190 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:13.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42c00ca9f0 0x7f42c00cd7a0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:13.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f42c006a5e0 0x7f42c00c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:13.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 --2- 172.21.15.60:0/4220143622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42c00add00 0x7f42c0158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:13.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 >> 172.21.15.60:0/4220143622 conn(0x7f42c00cbf50 msgr2=0x7f42c00b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:13.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 shutdown_connections 2023-07-13T22:29:13.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:13.198+0000 7f42c7fff700 1 -- 172.21.15.60:0/4220143622 wait complete. 2023-07-13T22:29:13.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:29:13.196 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:29:13.196 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:29:13.197 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:29:13.197 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:29:13.197 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:13.197 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:29:13.198 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:29:13.198 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:29:13.198 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:29:13.199 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:13.199 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:29:13.199 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:29:13.199 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:29:13.200 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:29:13.200 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:13.200 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:29:13.201 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:29:13.201 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:29:13.201 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:29:13.201 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:29:13.202 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:29:13.202 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:29:13.202 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:29:13.203 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:13.203 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:29:13.203 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:29:13.351 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.152462.asok 2023-07-13T22:29:13.528 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.152462.asok 2023-07-13T22:29:13.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:13.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.152462.asok mds_sessions 2023-07-13T22:29:13.761 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:29:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8082, 2023-07-13T22:29:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:29:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:29:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:29:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 8082 2023-07-13T22:29:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:29:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:29:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:29:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2416976467 2023-07-13T22:29:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.8082 v1:192.168.0.1:0/2416976467", 2023-07-13T22:29:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "v1:192.168.0.1:0/2416976467", 2023-07-13T22:29:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions": [ 2023-07-13T22:29:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": 0, 2023-07-13T22:29:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "addrs": { 2023-07-13T22:29:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2023-07-13T22:29:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2023-07-13T22:29:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6836", 2023-07-13T22:29:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4078186433 2023-07-13T22:29:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:29:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6837", 2023-07-13T22:29:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4078186433 2023-07-13T22:29:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:29:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "seq": 0, 2023-07-13T22:29:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_gen": 0, 2023-07-13T22:29:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_ttl": "2023-07-13T22:30:07.938964+0000", 2023-07-13T22:29:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "last_cap_renew_request": "2023-07-13T22:29:07.938964+0000", 2023-07-13T22:29:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_renew_seq": 2, 2023-07-13T22:29:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "num_caps": 1, 2023-07-13T22:29:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "open" 2023-07-13T22:29:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2023-07-13T22:29:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 62 2023-07-13T22:29:13.772 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:29:13.776 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:29:13.777 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:29:13.823 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:29:13.824 DEBUG:teuthology.orchestra.run.smithi060:> import sys 2023-07-13T22:29:13.824 DEBUG:teuthology.orchestra.run.smithi060:> import time 2023-07-13T22:29:13.824 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:29:13.824 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:13.825 DEBUG:teuthology.orchestra.run.smithi060:> n = 800 2023-07-13T22:29:13.825 DEBUG:teuthology.orchestra.run.smithi060:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-07-13T22:29:13.825 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:13.826 DEBUG:teuthology.orchestra.run.smithi060:> if not os.path.exists(abs_path): 2023-07-13T22:29:13.826 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(abs_path) 2023-07-13T22:29:13.826 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:13.826 DEBUG:teuthology.orchestra.run.smithi060:> handles = [] 2023-07-13T22:29:13.827 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(0, n): 2023-07-13T22:29:13.827 DEBUG:teuthology.orchestra.run.smithi060:> fname = "file_"+str(i) 2023-07-13T22:29:13.827 DEBUG:teuthology.orchestra.run.smithi060:> path = os.path.join(abs_path, fname) 2023-07-13T22:29:13.827 DEBUG:teuthology.orchestra.run.smithi060:> handles.append(open(path, '"'"'w'"'"')) 2023-07-13T22:29:13.828 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:13.828 DEBUG:teuthology.orchestra.run.smithi060:> while True: 2023-07-13T22:29:13.828 DEBUG:teuthology.orchestra.run.smithi060:> time.sleep(1) 2023-07-13T22:29:13.829 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:29:13.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.103+0000 7fb902bff700 1 Processor -- start 2023-07-13T22:29:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.103+0000 7fb902bff700 1 -- start start 2023-07-13T22:29:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.105+0000 7fb902bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8fc159170 0x7fb8fc0c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.105+0000 7fb902bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8fc070980 0x7fb8fc0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.105+0000 7fb902bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8fc15d970 con 0x7fb8fc0c5040 2023-07-13T22:29:14.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.105+0000 7fb902bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc15cd60 con 0x7fb8fc070980 2023-07-13T22:29:14.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.105+0000 7fb902bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc15d040 con 0x7fb8fc159170 2023-07-13T22:29:14.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.105+0000 7fb90119c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8fc070980 0x7fb8fc0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:14.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.105+0000 7fb90119c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8fc070980 0x7fb8fc0c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41064/0 (socket says 172.21.15.60:41064) 2023-07-13T22:29:14.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.105+0000 7fb90119c700 1 -- 172.21.15.60:0/2698445592 learned_addr learned my addr 172.21.15.60:0/2698445592 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:14.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.105+0000 7fb90099b700 1 --2- 172.21.15.60:0/2698445592 >> v2:172.21.15.105:3300/0 conn(0x7fb8fc159170 0x7fb8fc0c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:14.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.106+0000 7fb90119c700 1 -- 172.21.15.60:0/2698445592 >> v2:172.21.15.105:3300/0 conn(0x7fb8fc159170 msgr2=0x7fb8fc0c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.106+0000 7fb90119c700 1 --2- 172.21.15.60:0/2698445592 >> v2:172.21.15.105:3300/0 conn(0x7fb8fc159170 0x7fb8fc0c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.106+0000 7fb90119c700 1 -- 172.21.15.60:0/2698445592 >> v1:172.21.15.105:6789/0 conn(0x7fb8fc0c5040 legacy=0x7fb8fc070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.106+0000 7fb90119c700 1 -- 172.21.15.60:0/2698445592 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc15d320 con 0x7fb8fc070980 2023-07-13T22:29:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.106+0000 7fb90119c700 1 --2- 172.21.15.60:0/2698445592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc070980 0x7fb8fc0c9c50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc067080 tx=0x7fb8f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd71b2e67175b50 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.106+0000 7fb8fb7fe700 1 -- 172.21.15.60:0/2698445592 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400e070 con 0x7fb8fc070980 2023-07-13T22:29:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.106+0000 7fb8fb7fe700 1 -- 172.21.15.60:0/2698445592 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f400b9c0 con 0x7fb8fc070980 2023-07-13T22:29:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.106+0000 7fb8fb7fe700 1 -- 172.21.15.60:0/2698445592 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400bd70 con 0x7fb8fc070980 2023-07-13T22:29:14.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.107+0000 7fb902bff700 1 -- 172.21.15.60:0/2698445592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc070980 msgr2=0x7fb8fc0c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:14.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.107+0000 7fb902bff700 1 --2- 172.21.15.60:0/2698445592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc070980 0x7fb8fc0c9c50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc067080 tx=0x7fb8f400b330 comp rx=0 tx=0).stop 2023-07-13T22:29:14.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.107+0000 7fb902bff700 1 -- 172.21.15.60:0/2698445592 shutdown_connections 2023-07-13T22:29:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.107+0000 7fb902bff700 1 --2- 172.21.15.60:0/2698445592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc070980 0x7fb8fc0c9c50 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.107+0000 7fb902bff700 1 --2- 172.21.15.60:0/2698445592 >> v2:172.21.15.105:3300/0 conn(0x7fb8fc159170 0x7fb8fc0c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.107+0000 7fb902bff700 1 -- 172.21.15.60:0/2698445592 >> 172.21.15.60:0/2698445592 conn(0x7fb8fc0cf410 msgr2=0x7fb8fc0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.107+0000 7fb902bff700 1 -- 172.21.15.60:0/2698445592 shutdown_connections 2023-07-13T22:29:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.107+0000 7fb902bff700 1 -- 172.21.15.60:0/2698445592 wait complete. 2023-07-13T22:29:14.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.109+0000 7fb902bff700 1 Processor -- start 2023-07-13T22:29:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.109+0000 7fb902bff700 1 -- start start 2023-07-13T22:29:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.110+0000 7fb902bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc070980 0x7fb8fc165a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.110+0000 7fb902bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc0c5040 0x7fb8fc149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.110+0000 7fb902bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8fc159170 0x7fb8fc14a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb902bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc0aced0 con 0x7fb8fc0c5040 2023-07-13T22:29:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb902bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc0ad1b0 con 0x7fb8fc070980 2023-07-13T22:29:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb902bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc14ccf0 con 0x7fb8fc159170 2023-07-13T22:29:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb8fbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc0c5040 0x7fb8fc149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb8fbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc0c5040 0x7fb8fc149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41070/0 (socket says 172.21.15.60:41070) 2023-07-13T22:29:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb8fbfff700 1 -- 172.21.15.60:0/1363324294 learned_addr learned my addr 172.21.15.60:0/1363324294 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb90099b700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc070980 0x7fb8fc165a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb90119c700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8fc159170 0x7fb8fc14a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb8fbfff700 1 -- 172.21.15.60:0/1363324294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8fc159170 msgr2=0x7fb8fc14a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb8fbfff700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8fc159170 0x7fb8fc14a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb8fbfff700 1 -- 172.21.15.60:0/1363324294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc070980 msgr2=0x7fb8fc165a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb8fbfff700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc070980 0x7fb8fc165a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.111+0000 7fb8fbfff700 1 -- 172.21.15.60:0/1363324294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f4009b30 con 0x7fb8fc0c5040 2023-07-13T22:29:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.112+0000 7fb8fbfff700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc0c5040 0x7fb8fc149e10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800c4b0 tx=0x7fb8e800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.112+0000 7fb8f97fa700 1 -- 172.21.15.60:0/1363324294 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80052d0 con 0x7fb8fc0c5040 2023-07-13T22:29:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.112+0000 7fb8f97fa700 1 -- 172.21.15.60:0/1363324294 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e8005430 con 0x7fb8fc0c5040 2023-07-13T22:29:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.112+0000 7fb8f97fa700 1 -- 172.21.15.60:0/1363324294 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80038c0 con 0x7fb8fc0c5040 2023-07-13T22:29:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.112+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc14cfd0 con 0x7fb8fc0c5040 2023-07-13T22:29:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.112+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc165fd0 con 0x7fb8fc0c5040 2023-07-13T22:29:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.113+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8fc14d4f0 con 0x7fb8fc0c5040 2023-07-13T22:29:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.115+0000 7fb8f97fa700 1 -- 172.21.15.60:0/1363324294 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8e8003a20 con 0x7fb8fc0c5040 2023-07-13T22:29:14.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.116+0000 7fb8f97fa700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb8e006ace0 0x7fb8e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:14.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.116+0000 7fb8f97fa700 1 -- 172.21.15.60:0/1363324294 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb8e8088820 con 0x7fb8fc0c5040 2023-07-13T22:29:14.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.116+0000 7fb90099b700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb8e006ace0 0x7fb8e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:14.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.117+0000 7fb90099b700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb8e006ace0 0x7fb8e006d190 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc165680 tx=0x7fb8f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:14.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.121+0000 7fb8f97fa700 1 -- 172.21.15.60:0/1363324294 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb8fc14d4f0 con 0x7fb8fc0c5040 2023-07-13T22:29:14.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.420+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8fc14d4f0 con 0x7fb8fc0c5040 2023-07-13T22:29:14.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.421+0000 7fb8f97fa700 1 -- 172.21.15.60:0/1363324294 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4714 (secure 0 0 0) 0x7fb8fc14d4f0 con 0x7fb8fc0c5040 2023-07-13T22:29:14.401 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:14.401 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:58.254590+0000","modified":"2023-07-13T22:29:01.746898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4078186433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":4078186433},{"type":"v1","addr":"172.21.15.60:6837","nonce":4078186433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-13T22:29:14.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb8e006ace0 msgr2=0x7fb8e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:14.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb8e006ace0 0x7fb8e006d190 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc165680 tx=0x7fb8f0008040 comp rx=0 tx=0).stop 2023-07-13T22:29:14.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc0c5040 msgr2=0x7fb8fc149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:14.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc0c5040 0x7fb8fc149e10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800c4b0 tx=0x7fb8e800c7c0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 shutdown_connections 2023-07-13T22:29:14.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb8e006ace0 0x7fb8e006d190 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8fc159170 0x7fb8fc14a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8fc070980 0x7fb8fc165a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 --2- 172.21.15.60:0/1363324294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8fc0c5040 0x7fb8fc149e10 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:14.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 >> 172.21.15.60:0/1363324294 conn(0x7fb8fc0cf410 msgr2=0x7fb8fc070270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:14.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 shutdown_connections 2023-07-13T22:29:14.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:14.424+0000 7fb902bff700 1 -- 172.21.15.60:0/1363324294 wait complete. 2023-07-13T22:29:14.405 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2023-07-13T22:29:14.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-13T22:29:14.676 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":6698,"entity":{"name":{"type":"client","num":6698},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1045006933}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":5.3879453249999996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9845115941008107,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6698 192.168.0.1:0/1045006933","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"125439","root":"/"}},{"id":8082,"entity":{"name":{"type":"client","num":8082},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2416976467}},"state":"open","num_leases":1,"num_caps":360,"request_load_avg":360,"uptime":7.7419214380000003,"requests_in_flight":358,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":721.41231651416319,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8082 192.168.0.1:0/2416976467","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"152462","root":"/"}}] 2023-07-13T22:29:14.677 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "125439", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1045006933, "type": "any" }, "name": { "num": 6698, "type": "client" } }, "id": 6698, "inst": "client.6698 192.168.0.1:0/1045006933", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9845115941008107 }, "state": "open", "uptime": 5.387945325 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "152462", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2416976467, "type": "any" }, "name": { "num": 8082, "type": "client" } }, "id": 8082, "inst": "client.8082 192.168.0.1:0/2416976467", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 360, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 360, "requests_in_flight": 358, "session_cache_liveness": { "halflife": 300, "value": 721.4123165141632 }, "state": "open", "uptime": 7.741921438 } ] 2023-07-13T22:29:14.678 DEBUG:tasks.ceph_test_case:wait_until_equal: 360 != 802, waiting (timeout=600)... 2023-07-13T22:29:19.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.914+0000 7fc128dff700 1 Processor -- start 2023-07-13T22:29:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.915+0000 7fc128dff700 1 -- start start 2023-07-13T22:29:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.916+0000 7fc128dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1240d35c0 0x7fc1240d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.916+0000 7fc128dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1240d09d0 con 0x7fc1240d13f0 2023-07-13T22:29:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.916+0000 7fc128dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1240cda10 con 0x7fc1240b8570 2023-07-13T22:29:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.916+0000 7fc128dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1240d3b30 con 0x7fc1240d35c0 2023-07-13T22:29:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.916+0000 7fc11bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1240d35c0 0x7fc1240d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.916+0000 7fc11bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1240d35c0 0x7fc1240d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41800/0 (socket says 172.21.15.60:41800) 2023-07-13T22:29:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.916+0000 7fc11bfff700 1 -- 172.21.15.60:0/1826238810 learned_addr learned my addr 172.21.15.60:0/1826238810 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.916+0000 7fc122d9d700 1 --1- 172.21.15.60:0/1826238810 >> v1:172.21.15.60:6789/0 conn(0x7fc1240d13f0 0x7fc12415c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56088/0 (socket says 172.21.15.60:56088) 2023-07-13T22:29:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.917+0000 7fc11bfff700 1 -- 172.21.15.60:0/1826238810 >> v1:172.21.15.105:6789/0 conn(0x7fc1240b8570 legacy=0x7fc1240ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.917+0000 7fc11bfff700 1 -- 172.21.15.60:0/1826238810 >> v1:172.21.15.60:6789/0 conn(0x7fc1240d13f0 legacy=0x7fc12415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.917+0000 7fc11bfff700 1 -- 172.21.15.60:0/1826238810 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1240cd450 con 0x7fc1240d35c0 2023-07-13T22:29:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.917+0000 7fc11bfff700 1 --2- 172.21.15.60:0/1826238810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 0x7fc1240d0eb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0098f0 tx=0x7fc10c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e22cb7ba4422d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.917+0000 7fc121d9b700 1 -- 172.21.15.60:0/1826238810 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c004230 con 0x7fc1240d35c0 2023-07-13T22:29:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.917+0000 7fc121d9b700 1 -- 172.21.15.60:0/1826238810 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c004390 con 0x7fc1240d35c0 2023-07-13T22:29:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.917+0000 7fc121d9b700 1 -- 172.21.15.60:0/1826238810 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0036a0 con 0x7fc1240d35c0 2023-07-13T22:29:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.918+0000 7fc128dff700 1 -- 172.21.15.60:0/1826238810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 msgr2=0x7fc1240d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.918+0000 7fc128dff700 1 --2- 172.21.15.60:0/1826238810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 0x7fc1240d0eb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0098f0 tx=0x7fc10c009c00 comp rx=0 tx=0).stop 2023-07-13T22:29:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.918+0000 7fc128dff700 1 -- 172.21.15.60:0/1826238810 shutdown_connections 2023-07-13T22:29:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.918+0000 7fc128dff700 1 --2- 172.21.15.60:0/1826238810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 0x7fc1240d0eb0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.918+0000 7fc128dff700 1 -- 172.21.15.60:0/1826238810 >> 172.21.15.60:0/1826238810 conn(0x7fc124071600 msgr2=0x7fc1240719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.918+0000 7fc128dff700 1 -- 172.21.15.60:0/1826238810 shutdown_connections 2023-07-13T22:29:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.918+0000 7fc128dff700 1 -- 172.21.15.60:0/1826238810 wait complete. 2023-07-13T22:29:19.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.920+0000 7fc128dff700 1 Processor -- start 2023-07-13T22:29:19.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.920+0000 7fc128dff700 1 -- start start 2023-07-13T22:29:19.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc128dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1240b8570 0x7fc1240c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:19.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc128dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1240d13f0 0x7fc12415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:19.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc128dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 0x7fc124153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:19.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc128dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1240cb320 con 0x7fc1240d35c0 2023-07-13T22:29:19.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc128dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1240ca710 con 0x7fc1240b8570 2023-07-13T22:29:19.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc128dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1240ca9f0 con 0x7fc1240d13f0 2023-07-13T22:29:19.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc122d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 0x7fc124153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:19.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc122d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 0x7fc124153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41814/0 (socket says 172.21.15.60:41814) 2023-07-13T22:29:19.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc122d9d700 1 -- 172.21.15.60:0/2508922552 learned_addr learned my addr 172.21.15.60:0/2508922552 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:19.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc12259c700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1240b8570 0x7fc1240c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:19.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc11bfff700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1240d13f0 0x7fc12415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:19.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc122d9d700 1 -- 172.21.15.60:0/2508922552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1240d13f0 msgr2=0x7fc12415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:19.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc122d9d700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1240d13f0 0x7fc12415dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:19.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc122d9d700 1 -- 172.21.15.60:0/2508922552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1240b8570 msgr2=0x7fc1240c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:19.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.921+0000 7fc122d9d700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1240b8570 0x7fc1240c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:19.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.922+0000 7fc122d9d700 1 -- 172.21.15.60:0/2508922552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009550 con 0x7fc1240d35c0 2023-07-13T22:29:19.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.922+0000 7fc12259c700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1240b8570 0x7fc1240c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:19.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.922+0000 7fc122d9d700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 0x7fc124153c50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00cbd0 tx=0x7fc11c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:19.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.922+0000 7fc11affd700 1 -- 172.21.15.60:0/2508922552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c00abb0 con 0x7fc1240d35c0 2023-07-13T22:29:19.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.922+0000 7fc11affd700 1 -- 172.21.15.60:0/2508922552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c00ad10 con 0x7fc1240d35c0 2023-07-13T22:29:19.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.922+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1240cacd0 con 0x7fc1240d35c0 2023-07-13T22:29:19.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.922+0000 7fc11affd700 1 -- 172.21.15.60:0/2508922552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c007270 con 0x7fc1240d35c0 2023-07-13T22:29:19.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.922+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc124159900 con 0x7fc1240d35c0 2023-07-13T22:29:19.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.923+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1240b3db0 con 0x7fc1240d35c0 2023-07-13T22:29:19.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.924+0000 7fc11affd700 1 -- 172.21.15.60:0/2508922552 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc11c0073d0 con 0x7fc1240d35c0 2023-07-13T22:29:19.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.925+0000 7fc11affd700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc10406ace0 0x7fc10406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:19.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.925+0000 7fc11affd700 1 -- 172.21.15.60:0/2508922552 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc11c0892f0 con 0x7fc1240d35c0 2023-07-13T22:29:19.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.925+0000 7fc12259c700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc10406ace0 0x7fc10406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:19.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.926+0000 7fc12259c700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc10406ace0 0x7fc10406d190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc114006fd0 tx=0x7fc114008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:19.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:19.930+0000 7fc11affd700 1 -- 172.21.15.60:0/2508922552 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc11c0543b0 con 0x7fc1240d35c0 2023-07-13T22:29:20.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.180+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1240d10b0 con 0x7fc1240d35c0 2023-07-13T22:29:20.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.181+0000 7fc11affd700 1 -- 172.21.15.60:0/2508922552 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7fc11c017090 con 0x7fc1240d35c0 2023-07-13T22:29:20.161 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:20.161 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":64,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:58.254590+0000","modified":"2023-07-13T22:29:19.596439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6837/4078186433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":4078186433},{"type":"v1","addr":"172.21.15.60:6837","nonce":4078186433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-13T22:29:20.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc10406ace0 msgr2=0x7fc10406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:20.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc10406ace0 0x7fc10406d190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc114006fd0 tx=0x7fc114008040 comp rx=0 tx=0).stop 2023-07-13T22:29:20.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 msgr2=0x7fc124153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:20.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 0x7fc124153c50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00cbd0 tx=0x7fc11c00cf90 comp rx=0 tx=0).stop 2023-07-13T22:29:20.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 shutdown_connections 2023-07-13T22:29:20.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc10406ace0 0x7fc10406d190 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1240d35c0 0x7fc124153c50 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1240b8570 0x7fc1240c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 --2- 172.21.15.60:0/2508922552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1240d13f0 0x7fc12415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 >> 172.21.15.60:0/2508922552 conn(0x7fc124071600 msgr2=0x7fc12415ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:20.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 shutdown_connections 2023-07-13T22:29:20.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.184+0000 7fc128dff700 1 -- 172.21.15.60:0/2508922552 wait complete. 2023-07-13T22:29:20.165 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 64 2023-07-13T22:29:20.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-13T22:29:20.408 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":6698,"entity":{"name":{"type":"client","num":6698},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1045006933}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":11.120887146999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9452396976682396,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6698 192.168.0.1:0/1045006933","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"125439","root":"/"}},{"id":8082,"entity":{"name":{"type":"client","num":8082},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2416976467}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":754,"uptime":13.474863259999999,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2217.7738221876807,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":737.35265608805094,"halflife":1.5},"recall_caps_throttle2o":{"value":211.95703986918841,"halflife":0.5},"session_cache_liveness":{"value":1585.2358005497138,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":507,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8082 192.168.0.1:0/2416976467","completed_requests":[{"tid":507,"created_ino":"0x100000003e9"},{"tid":508,"created_ino":"0x100000003ea"},{"tid":509,"created_ino":"0x100000003eb"},{"tid":510,"created_ino":"0x100000003ec"},{"tid":511,"created_ino":"0x100000003ed"},{"tid":512,"created_ino":"0x100000003ee"},{"tid":513,"created_ino":"0x100000003ef"},{"tid":514,"created_ino":"0x100000003f0"},{"tid":515,"created_ino":"0x100000003f1"},{"tid":516,"created_ino":"0x100000003f2"},{"tid":517,"created_ino":"0x100000003f3"},{"tid":518,"created_ino":"0x100000003f4"},{"tid":519,"created_ino":"0x100000003f5"},{"tid":520,"created_ino":"0x100000003f6"},{"tid":521,"created_ino":"0x100000003f7"},{"tid":522,"created_ino":"0x100000003f8"},{"tid":523,"created_ino":"0x100000003f9"},{"tid":524,"created_ino":"0x100000003fa"},{"tid":525,"created_ino":"0x100000003fb"},{"tid":526,"created_ino":"0x100000003fc"},{"tid":527,"created_ino":"0x100000003fd"},{"tid":528,"created_ino":"0x100000003fe"},{"tid":529,"created_ino":"0x100000003ff"},{"tid":530,"created_ino":"0x10000000400"},{"tid":531,"created_ino":"0x10000000401"},{"tid":532,"created_ino":"0x10000000402"},{"tid":533,"created_ino":"0x10000000403"},{"tid":534,"created_ino":"0x10000000404"},{"tid":535,"created_ino":"0x10000000405"},{"tid":536,"created_ino":"0x10000000406"},{"tid":537,"created_ino":"0x10000000407"},{"tid":538,"created_ino":"0x10000000408"},{"tid":539,"created_ino":"0x10000000409"},{"tid":540,"created_ino":"0x1000000040a"},{"tid":541,"created_ino":"0x1000000040b"},{"tid":542,"created_ino":"0x1000000040c"},{"tid":543,"created_ino":"0x1000000040d"},{"tid":544,"created_ino":"0x1000000040e"},{"tid":545,"created_ino":"0x1000000040f"},{"tid":546,"created_ino":"0x10000000410"},{"tid":547,"created_ino":"0x10000000411"},{"tid":548,"created_ino":"0x10000000412"},{"tid":549,"created_ino":"0x10000000413"},{"tid":550,"created_ino":"0x10000000414"},{"tid":551,"created_ino":"0x10000000415"},{"tid":552,"created_ino":"0x10000000416"},{"tid":553,"created_ino":"0x10000000417"},{"tid":554,"created_ino":"0x10000000418"},{"tid":555,"created_ino":"0x10000000419"},{"tid":556,"created_ino":"0x1000000041a"},{"tid":557,"created_ino":"0x1000000041b"},{"tid":558,"created_ino":"0x1000000041c"},{"tid":559,"created_ino":"0x1000000041d"},{"tid":560,"created_ino":"0x1000000041e"},{"tid":561,"created_ino":"0x1000000041f"},{"tid":562,"created_ino":"0x10000000420"},{"tid":563,"created_ino":"0x10000000421"},{"tid":564,"created_ino":"0x10000000422"},{"tid":565,"created_ino":"0x10000000423"},{"tid":566,"created_ino":"0x10000000424"},{"tid":567,"created_ino":"0x10000000425"},{"tid":568,"created_ino":"0x10000000426"},{"tid":569,"created_ino":"0x10000000427"},{"tid":570,"created_ino":"0x10000000428"},{"tid":571,"created_ino":"0x10000000429"},{"tid":572,"created_ino":"0x1000000042a"},{"tid":573,"created_ino":"0x1000000042b"},{"tid":574,"created_ino":"0x1000000042c"},{"tid":575,"created_ino":"0x1000000042d"},{"tid":576,"created_ino":"0x1000000042e"},{"tid":577,"created_ino":"0x1000000042f"},{"tid":578,"created_ino":"0x10000000430"},{"tid":579,"created_ino":"0x10000000431"},{"tid":580,"created_ino":"0x10000000432"},{"tid":581,"created_ino":"0x10000000433"},{"tid":582,"created_ino":"0x10000000434"},{"tid":583,"created_ino":"0x10000000435"},{"tid":584,"created_ino":"0x10000000436"},{"tid":585,"created_ino":"0x10000000437"},{"tid":586,"created_ino":"0x10000000438"},{"tid":587,"created_ino":"0x10000000439"},{"tid":588,"created_ino":"0x1000000043a"},{"tid":589,"created_ino":"0x1000000043b"},{"tid":590,"created_ino":"0x1000000043c"},{"tid":591,"created_ino":"0x1000000043d"},{"tid":592,"created_ino":"0x1000000043e"},{"tid":593,"created_ino":"0x1000000043f"},{"tid":594,"created_ino":"0x10000000440"},{"tid":595,"created_ino":"0x10000000441"},{"tid":596,"created_ino":"0x10000000442"},{"tid":597,"created_ino":"0x10000000443"},{"tid":598,"created_ino":"0x10000000444"},{"tid":599,"created_ino":"0x10000000445"},{"tid":600,"created_ino":"0x10000000446"},{"tid":601,"created_ino":"0x10000000447"},{"tid":602,"created_ino":"0x10000000448"},{"tid":603,"created_ino":"0x10000000449"},{"tid":604,"created_ino":"0x1000000044a"},{"tid":605,"created_ino":"0x1000000044b"},{"tid":606,"created_ino":"0x1000000044c"},{"tid":607,"created_ino":"0x1000000044d"},{"tid":608,"created_ino":"0x1000000044e"},{"tid":609,"created_ino":"0x1000000044f"},{"tid":610,"created_ino":"0x10000000450"},{"tid":611,"created_ino":"0x10000000451"},{"tid":612,"created_ino":"0x10000000452"},{"tid":613,"created_ino":"0x10000000453"},{"tid":614,"created_ino":"0x10000000454"},{"tid":615,"created_ino":"0x10000000455"},{"tid":616,"created_ino":"0x10000000456"},{"tid":617,"created_ino":"0x10000000457"},{"tid":618,"created_ino":"0x10000000458"},{"tid":619,"created_ino":"0x10000000459"},{"tid":620,"created_ino":"0x1000000045a"},{"tid":621,"created_ino":"0x1000000045b"},{"tid":622,"created_ino":"0x1000000045c"},{"tid":623,"created_ino":"0x1000000045d"},{"tid":624,"created_ino":"0x1000000045e"},{"tid":625,"created_ino":"0x1000000045f"},{"tid":626,"created_ino":"0x10000000460"},{"tid":627,"created_ino":"0x10000000461"},{"tid":628,"created_ino":"0x10000000462"},{"tid":629,"created_ino":"0x10000000463"},{"tid":630,"created_ino":"0x10000000464"},{"tid":631,"created_ino":"0x10000000465"},{"tid":632,"created_ino":"0x10000000466"},{"tid":633,"created_ino":"0x10000000467"},{"tid":634,"created_ino":"0x10000000468"},{"tid":635,"created_ino":"0x10000000469"},{"tid":636,"created_ino":"0x1000000046a"},{"tid":637,"created_ino":"0x1000000046b"},{"tid":638,"created_ino":"0x1000000046c"},{"tid":639,"created_ino":"0x1000000046d"},{"tid":640,"created_ino":"0x1000000046e"},{"tid":641,"created_ino":"0x1000000046f"},{"tid":642,"created_ino":"0x10000000470"},{"tid":643,"created_ino":"0x10000000471"},{"tid":644,"created_ino":"0x10000000472"},{"tid":645,"created_ino":"0x10000000473"},{"tid":646,"created_ino":"0x10000000474"},{"tid":647,"created_ino":"0x10000000475"},{"tid":648,"created_ino":"0x10000000476"},{"tid":649,"created_ino":"0x10000000477"},{"tid":650,"created_ino":"0x10000000478"},{"tid":651,"created_ino":"0x10000000479"},{"tid":652,"created_ino":"0x1000000047a"},{"tid":653,"created_ino":"0x1000000047b"},{"tid":654,"created_ino":"0x1000000047c"},{"tid":655,"created_ino":"0x1000000047d"},{"tid":656,"created_ino":"0x1000000047e"},{"tid":657,"created_ino":"0x1000000047f"},{"tid":658,"created_ino":"0x10000000480"},{"tid":659,"created_ino":"0x10000000481"},{"tid":660,"created_ino":"0x10000000482"},{"tid":661,"created_ino":"0x10000000483"},{"tid":662,"created_ino":"0x10000000484"},{"tid":663,"created_ino":"0x10000000485"},{"tid":664,"created_ino":"0x10000000486"},{"tid":665,"created_ino":"0x10000000487"},{"tid":666,"created_ino":"0x10000000488"},{"tid":667,"created_ino":"0x10000000489"},{"tid":668,"created_ino":"0x1000000048a"},{"tid":669,"created_ino":"0x1000000048b"},{"tid":670,"created_ino":"0x1000000048c"},{"tid":671,"created_ino":"0x1000000048d"},{"tid":672,"created_ino":"0x1000000048e"},{"tid":673,"created_ino":"0x1000000048f"},{"tid":674,"created_ino":"0x10000000490"},{"tid":675,"created_ino":"0x10000000491"},{"tid":676,"created_ino":"0x10000000492"},{"tid":677,"created_ino":"0x10000000493"},{"tid":678,"created_ino":"0x10000000494"},{"tid":679,"created_ino":"0x10000000495"},{"tid":680,"created_ino":"0x10000000496"},{"tid":681,"created_ino":"0x10000000497"},{"tid":682,"created_ino":"0x10000000498"},{"tid":683,"created_ino":"0x10000000499"},{"tid":684,"created_ino":"0x1000000049a"},{"tid":685,"created_ino":"0x1000000049b"},{"tid":686,"created_ino":"0x1000000049c"},{"tid":687,"created_ino":"0x1000000049d"},{"tid":688,"created_ino":"0x1000000049e"},{"tid":689,"created_ino":"0x1000000049f"},{"tid":690,"created_ino":"0x100000004a0"},{"tid":691,"created_ino":"0x100000004a1"},{"tid":692,"created_ino":"0x100000004a2"},{"tid":693,"created_ino":"0x100000004a3"},{"tid":694,"created_ino":"0x100000004a4"},{"tid":695,"created_ino":"0x100000004a5"},{"tid":696,"created_ino":"0x100000004a6"},{"tid":697,"created_ino":"0x100000004a7"},{"tid":698,"created_ino":"0x100000004a8"},{"tid":699,"created_ino":"0x100000004a9"},{"tid":700,"created_ino":"0x100000004aa"},{"tid":701,"created_ino":"0x100000004ab"},{"tid":702,"created_ino":"0x100000004ac"},{"tid":703,"created_ino":"0x100000004ad"},{"tid":704,"created_ino":"0x100000004ae"},{"tid":705,"created_ino":"0x100000004af"},{"tid":706,"created_ino":"0x100000004b0"},{"tid":707,"created_ino":"0x100000004b1"},{"tid":708,"created_ino":"0x100000004b2"},{"tid":709,"created_ino":"0x100000004b3"},{"tid":710,"created_ino":"0x100000004b4"},{"tid":711,"created_ino":"0x100000004b5"},{"tid":712,"created_ino":"0x100000004b6"},{"tid":713,"created_ino":"0x100000004b7"},{"tid":714,"created_ino":"0x100000004b8"},{"tid":715,"created_ino":"0x100000004b9"},{"tid":716,"created_ino":"0x100000004ba"},{"tid":717,"created_ino":"0x100000004bb"},{"tid":718,"created_ino":"0x100000004bc"},{"tid":719,"created_ino":"0x100000004bd"},{"tid":720,"created_ino":"0x100000004be"},{"tid":721,"created_ino":"0x100000004bf"},{"tid":722,"created_ino":"0x100000004c0"},{"tid":723,"created_ino":"0x100000004c1"},{"tid":724,"created_ino":"0x100000004c2"},{"tid":725,"created_ino":"0x100000004c3"},{"tid":726,"created_ino":"0x100000004c4"},{"tid":727,"created_ino":"0x100000004c5"},{"tid":728,"created_ino":"0x100000004c6"},{"tid":729,"created_ino":"0x100000004c7"},{"tid":730,"created_ino":"0x100000004c8"},{"tid":731,"created_ino":"0x100000004c9"},{"tid":732,"created_ino":"0x100000004ca"},{"tid":733,"created_ino":"0x100000004cb"},{"tid":734,"created_ino":"0x100000004cc"},{"tid":735,"created_ino":"0x100000004cd"},{"tid":736,"created_ino":"0x100000004ce"},{"tid":737,"created_ino":"0x100000004cf"},{"tid":738,"created_ino":"0x100000004d0"},{"tid":739,"created_ino":"0x100000004d1"},{"tid":740,"created_ino":"0x100000004d2"},{"tid":741,"created_ino":"0x100000004d3"},{"tid":742,"created_ino":"0x100000004d4"},{"tid":743,"created_ino":"0x100000004d5"},{"tid":744,"created_ino":"0x100000004d6"},{"tid":745,"created_ino":"0x100000004d7"},{"tid":746,"created_ino":"0x100000004d8"},{"tid":747,"created_ino":"0x100000004d9"},{"tid":748,"created_ino":"0x100000004da"},{"tid":749,"created_ino":"0x100000004db"},{"tid":750,"created_ino":"0x100000004dc"},{"tid":751,"created_ino":"0x100000004dd"},{"tid":752,"created_ino":"0x100000004de"},{"tid":753,"created_ino":"0x100000004df"},{"tid":754,"created_ino":"0x100000004e0"},{"tid":755,"created_ino":"0x100000004e1"},{"tid":756,"created_ino":"0x100000004e2"},{"tid":757,"created_ino":"0x100000004e3"},{"tid":758,"created_ino":"0x100000004e4"},{"tid":759,"created_ino":"0x100000004e5"},{"tid":760,"created_ino":"0x100000004e6"},{"tid":761,"created_ino":"0x100000004e7"},{"tid":762,"created_ino":"0x100000004e8"},{"tid":763,"created_ino":"0x100000004e9"},{"tid":764,"created_ino":"0x100000004ea"},{"tid":765,"created_ino":"0x100000004eb"},{"tid":766,"created_ino":"0x100000004ec"},{"tid":767,"created_ino":"0x100000004ed"},{"tid":768,"created_ino":"0x100000004ee"},{"tid":769,"created_ino":"0x100000004ef"},{"tid":770,"created_ino":"0x100000004f0"},{"tid":771,"created_ino":"0x100000004f1"},{"tid":772,"created_ino":"0x100000004f2"},{"tid":773,"created_ino":"0x100000004f3"},{"tid":774,"created_ino":"0x100000004f4"},{"tid":775,"created_ino":"0x100000004f5"},{"tid":776,"created_ino":"0x100000004f6"},{"tid":777,"created_ino":"0x100000004f7"},{"tid":778,"created_ino":"0x100000004f8"},{"tid":779,"created_ino":"0x100000004f9"},{"tid":780,"created_ino":"0x100000004fa"},{"tid":781,"created_ino":"0x100000004fb"},{"tid":782,"created_ino":"0x100000004fc"},{"tid":783,"created_ino":"0x100000004fd"},{"tid":784,"created_ino":"0x100000004fe"},{"tid":785,"created_ino":"0x100000004ff"},{"tid":786,"created_ino":"0x10000000500"},{"tid":787,"created_ino":"0x10000000501"},{"tid":788,"created_ino":"0x10000000502"},{"tid":789,"created_ino":"0x10000000503"},{"tid":790,"created_ino":"0x10000000504"},{"tid":791,"created_ino":"0x10000000505"},{"tid":792,"created_ino":"0x10000000506"},{"tid":793,"created_ino":"0x10000000507"},{"tid":794,"created_ino":"0x10000000508"},{"tid":795,"created_ino":"0x10000000509"},{"tid":796,"created_ino":"0x1000000050a"},{"tid":797,"created_ino":"0x1000000050b"},{"tid":798,"created_ino":"0x1000000050c"},{"tid":799,"created_ino":"0x1000000050d"},{"tid":800,"created_ino":"0x1000000050e"},{"tid":801,"created_ino":"0x1000000050f"},{"tid":802,"created_ino":"0x10000000510"},{"tid":803,"created_ino":"0x10000000511"},{"tid":804,"created_ino":"0x10000000512"},{"tid":805,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"152462","root":"/"}}] 2023-07-13T22:29:20.410 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "125439", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1045006933, "type": "any" }, "name": { "num": 6698, "type": "client" } }, "id": 6698, "inst": "client.6698 192.168.0.1:0/1045006933", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9452396976682396 }, "state": "open", "uptime": 11.120887147 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "152462", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 507 }, { "created_ino": "0x100000003ea", "tid": 508 }, { "created_ino": "0x100000003eb", "tid": 509 }, { "created_ino": "0x100000003ec", "tid": 510 }, { "created_ino": "0x100000003ed", "tid": 511 }, { "created_ino": "0x100000003ee", "tid": 512 }, { "created_ino": "0x100000003ef", "tid": 513 }, { "created_ino": "0x100000003f0", "tid": 514 }, { "created_ino": "0x100000003f1", "tid": 515 }, { "created_ino": "0x100000003f2", "tid": 516 }, { "created_ino": "0x100000003f3", "tid": 517 }, { "created_ino": "0x100000003f4", "tid": 518 }, { "created_ino": "0x100000003f5", "tid": 519 }, { "created_ino": "0x100000003f6", "tid": 520 }, { "created_ino": "0x100000003f7", "tid": 521 }, { "created_ino": "0x100000003f8", "tid": 522 }, { "created_ino": "0x100000003f9", "tid": 523 }, { "created_ino": "0x100000003fa", "tid": 524 }, { "created_ino": "0x100000003fb", "tid": 525 }, { "created_ino": "0x100000003fc", "tid": 526 }, { "created_ino": "0x100000003fd", "tid": 527 }, { "created_ino": "0x100000003fe", "tid": 528 }, { "created_ino": "0x100000003ff", "tid": 529 }, { "created_ino": "0x10000000400", "tid": 530 }, { "created_ino": "0x10000000401", "tid": 531 }, { "created_ino": "0x10000000402", "tid": 532 }, { "created_ino": "0x10000000403", "tid": 533 }, { "created_ino": "0x10000000404", "tid": 534 }, { "created_ino": "0x10000000405", "tid": 535 }, { "created_ino": "0x10000000406", "tid": 536 }, { "created_ino": "0x10000000407", "tid": 537 }, { "created_ino": "0x10000000408", "tid": 538 }, { "created_ino": "0x10000000409", "tid": 539 }, { "created_ino": "0x1000000040a", "tid": 540 }, { "created_ino": "0x1000000040b", "tid": 541 }, { "created_ino": "0x1000000040c", "tid": 542 }, { "created_ino": "0x1000000040d", "tid": 543 }, { "created_ino": "0x1000000040e", "tid": 544 }, { "created_ino": "0x1000000040f", "tid": 545 }, { "created_ino": "0x10000000410", "tid": 546 }, { "created_ino": "0x10000000411", "tid": 547 }, { "created_ino": "0x10000000412", "tid": 548 }, { "created_ino": "0x10000000413", "tid": 549 }, { "created_ino": "0x10000000414", "tid": 550 }, { "created_ino": "0x10000000415", "tid": 551 }, { "created_ino": "0x10000000416", "tid": 552 }, { "created_ino": "0x10000000417", "tid": 553 }, { "created_ino": "0x10000000418", "tid": 554 }, { "created_ino": "0x10000000419", "tid": 555 }, { "created_ino": "0x1000000041a", "tid": 556 }, { "created_ino": "0x1000000041b", "tid": 557 }, { "created_ino": "0x1000000041c", "tid": 558 }, { "created_ino": "0x1000000041d", "tid": 559 }, { "created_ino": "0x1000000041e", "tid": 560 }, { "created_ino": "0x1000000041f", "tid": 561 }, { "created_ino": "0x10000000420", "tid": 562 }, { "created_ino": "0x10000000421", "tid": 563 }, { "created_ino": "0x10000000422", "tid": 564 }, { "created_ino": "0x10000000423", "tid": 565 }, { "created_ino": "0x10000000424", "tid": 566 }, { "created_ino": "0x10000000425", "tid": 567 }, { "created_ino": "0x10000000426", "tid": 568 }, { "created_ino": "0x10000000427", "tid": 569 }, { "created_ino": "0x10000000428", "tid": 570 }, { "created_ino": "0x10000000429", "tid": 571 }, { "created_ino": "0x1000000042a", "tid": 572 }, { "created_ino": "0x1000000042b", "tid": 573 }, { "created_ino": "0x1000000042c", "tid": 574 }, { "created_ino": "0x1000000042d", "tid": 575 }, { "created_ino": "0x1000000042e", "tid": 576 }, { "created_ino": "0x1000000042f", "tid": 577 }, { "created_ino": "0x10000000430", "tid": 578 }, { "created_ino": "0x10000000431", "tid": 579 }, { "created_ino": "0x10000000432", "tid": 580 }, { "created_ino": "0x10000000433", "tid": 581 }, { "created_ino": "0x10000000434", "tid": 582 }, { "created_ino": "0x10000000435", "tid": 583 }, { "created_ino": "0x10000000436", "tid": 584 }, { "created_ino": "0x10000000437", "tid": 585 }, { "created_ino": "0x10000000438", "tid": 586 }, { "created_ino": "0x10000000439", "tid": 587 }, { "created_ino": "0x1000000043a", "tid": 588 }, { "created_ino": "0x1000000043b", "tid": 589 }, { "created_ino": "0x1000000043c", "tid": 590 }, { "created_ino": "0x1000000043d", "tid": 591 }, { "created_ino": "0x1000000043e", "tid": 592 }, { "created_ino": "0x1000000043f", "tid": 593 }, { "created_ino": "0x10000000440", "tid": 594 }, { "created_ino": "0x10000000441", "tid": 595 }, { "created_ino": "0x10000000442", "tid": 596 }, { "created_ino": "0x10000000443", "tid": 597 }, { "created_ino": "0x10000000444", "tid": 598 }, { "created_ino": "0x10000000445", "tid": 599 }, { "created_ino": "0x10000000446", "tid": 600 }, { "created_ino": "0x10000000447", "tid": 601 }, { "created_ino": "0x10000000448", "tid": 602 }, { "created_ino": "0x10000000449", "tid": 603 }, { "created_ino": "0x1000000044a", "tid": 604 }, { "created_ino": "0x1000000044b", "tid": 605 }, { "created_ino": "0x1000000044c", "tid": 606 }, { "created_ino": "0x1000000044d", "tid": 607 }, { "created_ino": "0x1000000044e", "tid": 608 }, { "created_ino": "0x1000000044f", "tid": 609 }, { "created_ino": "0x10000000450", "tid": 610 }, { "created_ino": "0x10000000451", "tid": 611 }, { "created_ino": "0x10000000452", "tid": 612 }, { "created_ino": "0x10000000453", "tid": 613 }, { "created_ino": "0x10000000454", "tid": 614 }, { "created_ino": "0x10000000455", "tid": 615 }, { "created_ino": "0x10000000456", "tid": 616 }, { "created_ino": "0x10000000457", "tid": 617 }, { "created_ino": "0x10000000458", "tid": 618 }, { "created_ino": "0x10000000459", "tid": 619 }, { "created_ino": "0x1000000045a", "tid": 620 }, { "created_ino": "0x1000000045b", "tid": 621 }, { "created_ino": "0x1000000045c", "tid": 622 }, { "created_ino": "0x1000000045d", "tid": 623 }, { "created_ino": "0x1000000045e", "tid": 624 }, { "created_ino": "0x1000000045f", "tid": 625 }, { "created_ino": "0x10000000460", "tid": 626 }, { "created_ino": "0x10000000461", "tid": 627 }, { "created_ino": "0x10000000462", "tid": 628 }, { "created_ino": "0x10000000463", "tid": 629 }, { "created_ino": "0x10000000464", "tid": 630 }, { "created_ino": "0x10000000465", "tid": 631 }, { "created_ino": "0x10000000466", "tid": 632 }, { "created_ino": "0x10000000467", "tid": 633 }, { "created_ino": "0x10000000468", "tid": 634 }, { "created_ino": "0x10000000469", "tid": 635 }, { "created_ino": "0x1000000046a", "tid": 636 }, { "created_ino": "0x1000000046b", "tid": 637 }, { "created_ino": "0x1000000046c", "tid": 638 }, { "created_ino": "0x1000000046d", "tid": 639 }, { "created_ino": "0x1000000046e", "tid": 640 }, { "created_ino": "0x1000000046f", "tid": 641 }, { "created_ino": "0x10000000470", "tid": 642 }, { "created_ino": "0x10000000471", "tid": 643 }, { "created_ino": "0x10000000472", "tid": 644 }, { "created_ino": "0x10000000473", "tid": 645 }, { "created_ino": "0x10000000474", "tid": 646 }, { "created_ino": "0x10000000475", "tid": 647 }, { "created_ino": "0x10000000476", "tid": 648 }, { "created_ino": "0x10000000477", "tid": 649 }, { "created_ino": "0x10000000478", "tid": 650 }, { "created_ino": "0x10000000479", "tid": 651 }, { "created_ino": "0x1000000047a", "tid": 652 }, { "created_ino": "0x1000000047b", "tid": 653 }, { "created_ino": "0x1000000047c", "tid": 654 }, { "created_ino": "0x1000000047d", "tid": 655 }, { "created_ino": "0x1000000047e", "tid": 656 }, { "created_ino": "0x1000000047f", "tid": 657 }, { "created_ino": "0x10000000480", "tid": 658 }, { "created_ino": "0x10000000481", "tid": 659 }, { "created_ino": "0x10000000482", "tid": 660 }, { "created_ino": "0x10000000483", "tid": 661 }, { "created_ino": "0x10000000484", "tid": 662 }, { "created_ino": "0x10000000485", "tid": 663 }, { "created_ino": "0x10000000486", "tid": 664 }, { "created_ino": "0x10000000487", "tid": 665 }, { "created_ino": "0x10000000488", "tid": 666 }, { "created_ino": "0x10000000489", "tid": 667 }, { "created_ino": "0x1000000048a", "tid": 668 }, { "created_ino": "0x1000000048b", "tid": 669 }, { "created_ino": "0x1000000048c", "tid": 670 }, { "created_ino": "0x1000000048d", "tid": 671 }, { "created_ino": "0x1000000048e", "tid": 672 }, { "created_ino": "0x1000000048f", "tid": 673 }, { "created_ino": "0x10000000490", "tid": 674 }, { "created_ino": "0x10000000491", "tid": 675 }, { "created_ino": "0x10000000492", "tid": 676 }, { "created_ino": "0x10000000493", "tid": 677 }, { "created_ino": "0x10000000494", "tid": 678 }, { "created_ino": "0x10000000495", "tid": 679 }, { "created_ino": "0x10000000496", "tid": 680 }, { "created_ino": "0x10000000497", "tid": 681 }, { "created_ino": "0x10000000498", "tid": 682 }, { "created_ino": "0x10000000499", "tid": 683 }, { "created_ino": "0x1000000049a", "tid": 684 }, { "created_ino": "0x1000000049b", "tid": 685 }, { "created_ino": "0x1000000049c", "tid": 686 }, { "created_ino": "0x1000000049d", "tid": 687 }, { "created_ino": "0x1000000049e", "tid": 688 }, { "created_ino": "0x1000000049f", "tid": 689 }, { "created_ino": "0x100000004a0", "tid": 690 }, { "created_ino": "0x100000004a1", "tid": 691 }, { "created_ino": "0x100000004a2", "tid": 692 }, { "created_ino": "0x100000004a3", "tid": 693 }, { "created_ino": "0x100000004a4", "tid": 694 }, { "created_ino": "0x100000004a5", "tid": 695 }, { "created_ino": "0x100000004a6", "tid": 696 }, { "created_ino": "0x100000004a7", "tid": 697 }, { "created_ino": "0x100000004a8", "tid": 698 }, { "created_ino": "0x100000004a9", "tid": 699 }, { "created_ino": "0x100000004aa", "tid": 700 }, { "created_ino": "0x100000004ab", "tid": 701 }, { "created_ino": "0x100000004ac", "tid": 702 }, { "created_ino": "0x100000004ad", "tid": 703 }, { "created_ino": "0x100000004ae", "tid": 704 }, { "created_ino": "0x100000004af", "tid": 705 }, { "created_ino": "0x100000004b0", "tid": 706 }, { "created_ino": "0x100000004b1", "tid": 707 }, { "created_ino": "0x100000004b2", "tid": 708 }, { "created_ino": "0x100000004b3", "tid": 709 }, { "created_ino": "0x100000004b4", "tid": 710 }, { "created_ino": "0x100000004b5", "tid": 711 }, { "created_ino": "0x100000004b6", "tid": 712 }, { "created_ino": "0x100000004b7", "tid": 713 }, { "created_ino": "0x100000004b8", "tid": 714 }, { "created_ino": "0x100000004b9", "tid": 715 }, { "created_ino": "0x100000004ba", "tid": 716 }, { "created_ino": "0x100000004bb", "tid": 717 }, { "created_ino": "0x100000004bc", "tid": 718 }, { "created_ino": "0x100000004bd", "tid": 719 }, { "created_ino": "0x100000004be", "tid": 720 }, { "created_ino": "0x100000004bf", "tid": 721 }, { "created_ino": "0x100000004c0", "tid": 722 }, { "created_ino": "0x100000004c1", "tid": 723 }, { "created_ino": "0x100000004c2", "tid": 724 }, { "created_ino": "0x100000004c3", "tid": 725 }, { "created_ino": "0x100000004c4", "tid": 726 }, { "created_ino": "0x100000004c5", "tid": 727 }, { "created_ino": "0x100000004c6", "tid": 728 }, { "created_ino": "0x100000004c7", "tid": 729 }, { "created_ino": "0x100000004c8", "tid": 730 }, { "created_ino": "0x100000004c9", "tid": 731 }, { "created_ino": "0x100000004ca", "tid": 732 }, { "created_ino": "0x100000004cb", "tid": 733 }, { "created_ino": "0x100000004cc", "tid": 734 }, { "created_ino": "0x100000004cd", "tid": 735 }, { "created_ino": "0x100000004ce", "tid": 736 }, { "created_ino": "0x100000004cf", "tid": 737 }, { "created_ino": "0x100000004d0", "tid": 738 }, { "created_ino": "0x100000004d1", "tid": 739 }, { "created_ino": "0x100000004d2", "tid": 740 }, { "created_ino": "0x100000004d3", "tid": 741 }, { "created_ino": "0x100000004d4", "tid": 742 }, { "created_ino": "0x100000004d5", "tid": 743 }, { "created_ino": "0x100000004d6", "tid": 744 }, { "created_ino": "0x100000004d7", "tid": 745 }, { "created_ino": "0x100000004d8", "tid": 746 }, { "created_ino": "0x100000004d9", "tid": 747 }, { "created_ino": "0x100000004da", "tid": 748 }, { "created_ino": "0x100000004db", "tid": 749 }, { "created_ino": "0x100000004dc", "tid": 750 }, { "created_ino": "0x100000004dd", "tid": 751 }, { "created_ino": "0x100000004de", "tid": 752 }, { "created_ino": "0x100000004df", "tid": 753 }, { "created_ino": "0x100000004e0", "tid": 754 }, { "created_ino": "0x100000004e1", "tid": 755 }, { "created_ino": "0x100000004e2", "tid": 756 }, { "created_ino": "0x100000004e3", "tid": 757 }, { "created_ino": "0x100000004e4", "tid": 758 }, { "created_ino": "0x100000004e5", "tid": 759 }, { "created_ino": "0x100000004e6", "tid": 760 }, { "created_ino": "0x100000004e7", "tid": 761 }, { "created_ino": "0x100000004e8", "tid": 762 }, { "created_ino": "0x100000004e9", "tid": 763 }, { "created_ino": "0x100000004ea", "tid": 764 }, { "created_ino": "0x100000004eb", "tid": 765 }, { "created_ino": "0x100000004ec", "tid": 766 }, { "created_ino": "0x100000004ed", "tid": 767 }, { "created_ino": "0x100000004ee", "tid": 768 }, { "created_ino": "0x100000004ef", "tid": 769 }, { "created_ino": "0x100000004f0", "tid": 770 }, { "created_ino": "0x100000004f1", "tid": 771 }, { "created_ino": "0x100000004f2", "tid": 772 }, { "created_ino": "0x100000004f3", "tid": 773 }, { "created_ino": "0x100000004f4", "tid": 774 }, { "created_ino": "0x100000004f5", "tid": 775 }, { "created_ino": "0x100000004f6", "tid": 776 }, { "created_ino": "0x100000004f7", "tid": 777 }, { "created_ino": "0x100000004f8", "tid": 778 }, { "created_ino": "0x100000004f9", "tid": 779 }, { "created_ino": "0x100000004fa", "tid": 780 }, { "created_ino": "0x100000004fb", "tid": 781 }, { "created_ino": "0x100000004fc", "tid": 782 }, { "created_ino": "0x100000004fd", "tid": 783 }, { "created_ino": "0x100000004fe", "tid": 784 }, { "created_ino": "0x100000004ff", "tid": 785 }, { "created_ino": "0x10000000500", "tid": 786 }, { "created_ino": "0x10000000501", "tid": 787 }, { "created_ino": "0x10000000502", "tid": 788 }, { "created_ino": "0x10000000503", "tid": 789 }, { "created_ino": "0x10000000504", "tid": 790 }, { "created_ino": "0x10000000505", "tid": 791 }, { "created_ino": "0x10000000506", "tid": 792 }, { "created_ino": "0x10000000507", "tid": 793 }, { "created_ino": "0x10000000508", "tid": 794 }, { "created_ino": "0x10000000509", "tid": 795 }, { "created_ino": "0x1000000050a", "tid": 796 }, { "created_ino": "0x1000000050b", "tid": 797 }, { "created_ino": "0x1000000050c", "tid": 798 }, { "created_ino": "0x1000000050d", "tid": 799 }, { "created_ino": "0x1000000050e", "tid": 800 }, { "created_ino": "0x1000000050f", "tid": 801 }, { "created_ino": "0x10000000510", "tid": 802 }, { "created_ino": "0x10000000511", "tid": 803 }, { "created_ino": "0x10000000512", "tid": 804 }, { "created_ino": "0x10000000513", "tid": 805 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2416976467, "type": "any" }, "name": { "num": 8082, "type": "client" } }, "id": 8082, "inst": "client.8082 192.168.0.1:0/2416976467", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 507, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2217.7738221876807 }, "recall_caps_throttle": { "halflife": 1.5, "value": 737.3526560880509 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 211.9570398691884 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 754, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1585.2358005497138 }, "state": "open", "uptime": 13.47486326 } ] 2023-07-13T22:29:20.411 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-07-13T22:29:20.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:29:20.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.634+0000 7f5830fff700 1 Processor -- start 2023-07-13T22:29:20.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.634+0000 7f5830fff700 1 -- start start 2023-07-13T22:29:20.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.635+0000 7f5830fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f582c0ca520 0x7f582c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.635+0000 7f5830fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f582c0cf1f0 0x7f582c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f5830fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582c0cf670 con 0x7f582c15a390 2023-07-13T22:29:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f5830fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c0cab10 con 0x7f582c0cf1f0 2023-07-13T22:29:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f5830fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c0cadf0 con 0x7f582c0ca520 2023-07-13T22:29:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f582ad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f582c0cf1f0 0x7f582c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f582ad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f582c0cf1f0 0x7f582c06c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41826/0 (socket says 172.21.15.60:41826) 2023-07-13T22:29:20.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f582ad9d700 1 -- 172.21.15.60:0/3817839248 learned_addr learned my addr 172.21.15.60:0/3817839248 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:20.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f5823fff700 1 --2- 172.21.15.60:0/3817839248 >> v2:172.21.15.105:3300/0 conn(0x7f582c0ca520 0x7f582c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:20.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f582ad9d700 1 -- 172.21.15.60:0/3817839248 >> v2:172.21.15.105:3300/0 conn(0x7f582c0ca520 msgr2=0x7f582c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:20.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f582ad9d700 1 --2- 172.21.15.60:0/3817839248 >> v2:172.21.15.105:3300/0 conn(0x7f582c0ca520 0x7f582c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f582ad9d700 1 -- 172.21.15.60:0/3817839248 >> v1:172.21.15.105:6789/0 conn(0x7f582c15a390 legacy=0x7f582c070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:20.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.636+0000 7f582ad9d700 1 -- 172.21.15.60:0/3817839248 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c06e030 con 0x7f582c0cf1f0 2023-07-13T22:29:20.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.637+0000 7f582ad9d700 1 --2- 172.21.15.60:0/3817839248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c0cf1f0 0x7f582c06c3e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f582c06b7d0 tx=0x7f582400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851e0bb4daaf13c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:20.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.637+0000 7f5829d9b700 1 -- 172.21.15.60:0/3817839248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400e070 con 0x7f582c0cf1f0 2023-07-13T22:29:20.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.637+0000 7f5829d9b700 1 -- 172.21.15.60:0/3817839248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582400b910 con 0x7f582c0cf1f0 2023-07-13T22:29:20.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.637+0000 7f5829d9b700 1 -- 172.21.15.60:0/3817839248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400bcc0 con 0x7f582c0cf1f0 2023-07-13T22:29:20.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.637+0000 7f5830fff700 1 -- 172.21.15.60:0/3817839248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c0cf1f0 msgr2=0x7f582c06c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:20.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.637+0000 7f5830fff700 1 --2- 172.21.15.60:0/3817839248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c0cf1f0 0x7f582c06c3e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f582c06b7d0 tx=0x7f582400b330 comp rx=0 tx=0).stop 2023-07-13T22:29:20.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.638+0000 7f5830fff700 1 -- 172.21.15.60:0/3817839248 shutdown_connections 2023-07-13T22:29:20.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.638+0000 7f5830fff700 1 --2- 172.21.15.60:0/3817839248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c0cf1f0 0x7f582c06c3e0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.638+0000 7f5830fff700 1 --2- 172.21.15.60:0/3817839248 >> v2:172.21.15.105:3300/0 conn(0x7f582c0ca520 0x7f582c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.638+0000 7f5830fff700 1 -- 172.21.15.60:0/3817839248 >> 172.21.15.60:0/3817839248 conn(0x7f582c0cbe70 msgr2=0x7f582c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:20.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.638+0000 7f5830fff700 1 -- 172.21.15.60:0/3817839248 shutdown_connections 2023-07-13T22:29:20.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.638+0000 7f5830fff700 1 -- 172.21.15.60:0/3817839248 wait complete. 2023-07-13T22:29:20.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.639+0000 7f5830fff700 1 Processor -- start 2023-07-13T22:29:20.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.640+0000 7f5830fff700 1 -- start start 2023-07-13T22:29:20.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.640+0000 7f5830fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f582c0ca520 0x7f582c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:20.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.640+0000 7f5830fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f582c0cf1f0 0x7f582c15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:20.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f5830fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c15a390 0x7f582c1eead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:20.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f5830fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0b1150 con 0x7f582c15a390 2023-07-13T22:29:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f5830fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0c3810 con 0x7f582c0ca520 2023-07-13T22:29:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f5830fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0c3af0 con 0x7f582c0cf1f0 2023-07-13T22:29:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f582ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c15a390 0x7f582c1eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f582ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c15a390 0x7f582c1eead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41840/0 (socket says 172.21.15.60:41840) 2023-07-13T22:29:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f582ad9d700 1 -- 172.21.15.60:0/3784348988 learned_addr learned my addr 172.21.15.60:0/3784348988 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:20.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f582a59c700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f582c0ca520 0x7f582c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:20.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f5823fff700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f582c0cf1f0 0x7f582c15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:20.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f582ad9d700 1 -- 172.21.15.60:0/3784348988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f582c0cf1f0 msgr2=0x7f582c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:20.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f582ad9d700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f582c0cf1f0 0x7f582c15aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f582ad9d700 1 -- 172.21.15.60:0/3784348988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f582c0ca520 msgr2=0x7f582c15a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f582ad9d700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f582c0ca520 0x7f582c15a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.641+0000 7f582ad9d700 1 -- 172.21.15.60:0/3784348988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5824009b30 con 0x7f582c15a390 2023-07-13T22:29:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.642+0000 7f582ad9d700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c15a390 0x7f582c1eead0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f582c070050 tx=0x7f5824004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:20.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.642+0000 7f5822ffd700 1 -- 172.21.15.60:0/3784348988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824004880 con 0x7f582c15a390 2023-07-13T22:29:20.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.642+0000 7f5822ffd700 1 -- 172.21.15.60:0/3784348988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58240099b0 con 0x7f582c15a390 2023-07-13T22:29:20.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.642+0000 7f5822ffd700 1 -- 172.21.15.60:0/3784348988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582401c680 con 0x7f582c15a390 2023-07-13T22:29:20.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.642+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c0c3dd0 con 0x7f582c15a390 2023-07-13T22:29:20.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.642+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c0c4390 con 0x7f582c15a390 2023-07-13T22:29:20.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.643+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c06ab90 con 0x7f582c15a390 2023-07-13T22:29:20.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.644+0000 7f5822ffd700 1 -- 172.21.15.60:0/3784348988 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f582401c7e0 con 0x7f582c15a390 2023-07-13T22:29:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.645+0000 7f5822ffd700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f580c06ace0 0x7f580c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.645+0000 7f5822ffd700 1 -- 172.21.15.60:0/3784348988 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f582408b090 con 0x7f582c15a390 2023-07-13T22:29:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.645+0000 7f582a59c700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f580c06ace0 0x7f580c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.646+0000 7f582a59c700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f580c06ace0 0x7f580c06d190 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f581c000fb0 tx=0x7f581c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:20.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.649+0000 7f5822ffd700 1 -- 172.21.15.60:0/3784348988 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5824056130 con 0x7f582c15a390 2023-07-13T22:29:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.908+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f582c04e280 con 0x7f582c15a390 2023-07-13T22:29:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.908+0000 7f5822ffd700 1 -- 172.21.15.60:0/3784348988 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f5824059750 con 0x7f582c15a390 2023-07-13T22:29:20.889 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:20.889 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:29:20.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f580c06ace0 msgr2=0x7f580c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f580c06ace0 0x7f580c06d190 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f581c000fb0 tx=0x7f581c009040 comp rx=0 tx=0).stop 2023-07-13T22:29:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c15a390 msgr2=0x7f582c1eead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c15a390 0x7f582c1eead0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f582c070050 tx=0x7f5824004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 shutdown_connections 2023-07-13T22:29:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f580c06ace0 0x7f580c06d190 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f582c15a390 0x7f582c1eead0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f582c0ca520 0x7f582c15a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 --2- 172.21.15.60:0/3784348988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f582c0cf1f0 0x7f582c15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 >> 172.21.15.60:0/3784348988 conn(0x7f582c0cbe70 msgr2=0x7f582c155410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 shutdown_connections 2023-07-13T22:29:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:20.911+0000 7f5830fff700 1 -- 172.21.15.60:0/3784348988 wait complete. 2023-07-13T22:29:20.908 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:29:20.908 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-07-13T22:29:20.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:29:21.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.131+0000 7f3620bff700 1 Processor -- start 2023-07-13T22:29:21.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.132+0000 7f3620bff700 1 -- start start 2023-07-13T22:29:21.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.133+0000 7f3620bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f361c0cd260 0x7f361c0d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.133+0000 7f3620bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.133+0000 7f3620bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c0d2410 con 0x7f361c0d0a50 2023-07-13T22:29:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.133+0000 7f3620bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c15c900 con 0x7f361c0cd260 2023-07-13T22:29:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.133+0000 7f3620bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0d2660 con 0x7f361c0d0e20 2023-07-13T22:29:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.133+0000 7f361ad9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.133+0000 7f3613fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f361c0cd260 0x7f361c0d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.133+0000 7f361ad9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:37450/0 (socket says 172.21.15.60:37450) 2023-07-13T22:29:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.133+0000 7f361ad9d700 1 -- 172.21.15.60:0/579962430 learned_addr learned my addr 172.21.15.60:0/579962430 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f3613fff700 1 -- 172.21.15.60:0/579962430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 msgr2=0x7f361c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f3613fff700 1 --2- 172.21.15.60:0/579962430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f3613fff700 1 -- 172.21.15.60:0/579962430 >> v1:172.21.15.105:6789/0 conn(0x7f361c0d0a50 legacy=0x7f361c0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f3613fff700 1 -- 172.21.15.60:0/579962430 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c0d2940 con 0x7f361c0cd260 2023-07-13T22:29:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f361ad9d700 1 --2- 172.21.15.60:0/579962430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f3619d9b700 1 -- 172.21.15.60:0/579962430 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937689064 0 0) 0x7f361c0d2410 con 0x7f361c0d0a50 2023-07-13T22:29:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f3613fff700 1 --2- 172.21.15.60:0/579962430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f361c0cd260 0x7f361c0d3420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3604009840 tx=0x7f3604009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda370f4f40fdc20 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f3619d9b700 1 -- 172.21.15.60:0/579962430 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604011830 con 0x7f361c0cd260 2023-07-13T22:29:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f3619d9b700 1 -- 172.21.15.60:0/579962430 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3604011990 con 0x7f361c0cd260 2023-07-13T22:29:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.134+0000 7f3619d9b700 1 -- 172.21.15.60:0/579962430 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604011d20 con 0x7f361c0cd260 2023-07-13T22:29:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.135+0000 7f3620bff700 1 -- 172.21.15.60:0/579962430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f361c0cd260 msgr2=0x7f361c0d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.135+0000 7f3620bff700 1 --2- 172.21.15.60:0/579962430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f361c0cd260 0x7f361c0d3420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3604009840 tx=0x7f3604009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.135+0000 7f3620bff700 1 -- 172.21.15.60:0/579962430 shutdown_connections 2023-07-13T22:29:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.135+0000 7f3620bff700 1 --2- 172.21.15.60:0/579962430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.135+0000 7f3620bff700 1 --2- 172.21.15.60:0/579962430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f361c0cd260 0x7f361c0d3420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.135+0000 7f3620bff700 1 -- 172.21.15.60:0/579962430 >> 172.21.15.60:0/579962430 conn(0x7f361c071600 msgr2=0x7f361c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:21.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.135+0000 7f3620bff700 1 -- 172.21.15.60:0/579962430 shutdown_connections 2023-07-13T22:29:21.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.135+0000 7f3620bff700 1 -- 172.21.15.60:0/579962430 wait complete. 2023-07-13T22:29:21.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.137+0000 7f3620bff700 1 Processor -- start 2023-07-13T22:29:21.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.137+0000 7f3620bff700 1 -- start start 2023-07-13T22:29:21.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f3620bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f361c0cd260 0x7f361c1551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f3620bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f361c0d0a50 0x7f361c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f3620bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f3620bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1f4350 con 0x7f361c0cd260 2023-07-13T22:29:21.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f3620bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c155730 con 0x7f361c0d0a50 2023-07-13T22:29:21.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f3620bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1f3740 con 0x7f361c0d0e20 2023-07-13T22:29:21.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f361a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f361c0cd260 0x7f361c1551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f361a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f361c0cd260 0x7f361c1551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41850/0 (socket says 172.21.15.60:41850) 2023-07-13T22:29:21.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f361a59c700 1 -- 172.21.15.60:0/2083300498 learned_addr learned my addr 172.21.15.60:0/2083300498 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:21.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f3613fff700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f361c0d0a50 0x7f361c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.138+0000 7f361ad9d700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f361a59c700 1 -- 172.21.15.60:0/2083300498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 msgr2=0x7f361c0c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f361a59c700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f361a59c700 1 -- 172.21.15.60:0/2083300498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f361c0d0a50 msgr2=0x7f361c0c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f361a59c700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f361c0d0a50 0x7f361c0c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f361a59c700 1 -- 172.21.15.60:0/2083300498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614009b30 con 0x7f361c0cd260 2023-07-13T22:29:21.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f361ad9d700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:21.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f361a59c700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f361c0cd260 0x7f361c1551f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f360c0011e0 tx=0x7f360c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:21.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f3612ffd700 1 -- 172.21.15.60:0/2083300498 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0054e0 con 0x7f361c0cd260 2023-07-13T22:29:21.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f3612ffd700 1 -- 172.21.15.60:0/2083300498 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c003680 con 0x7f361c0cd260 2023-07-13T22:29:21.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f3612ffd700 1 -- 172.21.15.60:0/2083300498 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c003a30 con 0x7f361c0cd260 2023-07-13T22:29:21.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36040094a0 con 0x7f361c0cd260 2023-07-13T22:29:21.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.139+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c1f3d00 con 0x7f361c0cd260 2023-07-13T22:29:21.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.140+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c04e280 con 0x7f361c0cd260 2023-07-13T22:29:21.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.141+0000 7f3612ffd700 1 -- 172.21.15.60:0/2083300498 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f360c003b90 con 0x7f361c0cd260 2023-07-13T22:29:21.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.143+0000 7f3612ffd700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.143+0000 7f3612ffd700 1 -- 172.21.15.60:0/2083300498 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f360c088980 con 0x7f361c0cd260 2023-07-13T22:29:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.143+0000 7f3613fff700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.144+0000 7f3613fff700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f36040090b0 tx=0x7f3604008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.147+0000 7f3612ffd700 1 -- 172.21.15.60:0/2083300498 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f360c053a20 con 0x7f361c0cd260 2023-07-13T22:29:21.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.402+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f361c0ce340 con 0x7f361c0cd260 2023-07-13T22:29:21.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.402+0000 7f3612ffd700 1 -- 172.21.15.60:0/2083300498 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f360c017150 con 0x7f361c0cd260 2023-07-13T22:29:21.382 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:21.382 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:29:21.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 msgr2=0x7f35fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f36040090b0 tx=0x7f3604008030 comp rx=0 tx=0).stop 2023-07-13T22:29:21.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f361c0cd260 msgr2=0x7f361c1551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f361c0cd260 0x7f361c1551f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f360c0011e0 tx=0x7f360c00ca30 comp rx=0 tx=0).stop 2023-07-13T22:29:21.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 shutdown_connections 2023-07-13T22:29:21.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f35fc06ace0 0x7f35fc06d190 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f361c0cd260 0x7f361c1551f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f361c0d0e20 0x7f361c0c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 --2- 172.21.15.60:0/2083300498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f361c0d0a50 0x7f361c0c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 >> 172.21.15.60:0/2083300498 conn(0x7f361c071600 msgr2=0x7f361c073380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 shutdown_connections 2023-07-13T22:29:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.405+0000 7f3620bff700 1 -- 172.21.15.60:0/2083300498 wait complete. 2023-07-13T22:29:21.402 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:29:21.403 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2023-07-13T22:29:21.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:21.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.683+0000 7f5d561ff700 1 Processor -- start 2023-07-13T22:29:21.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.683+0000 7f5d561ff700 1 -- start start 2023-07-13T22:29:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.684+0000 7f5d561ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d500cfca0 0x7f5d50167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.684+0000 7f5d561ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500cd950 con 0x7f5d500b8400 2023-07-13T22:29:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.684+0000 7f5d561ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500d0ac0 con 0x7f5d500d3750 2023-07-13T22:29:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.684+0000 7f5d561ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500b87d0 con 0x7f5d500cfca0 2023-07-13T22:29:21.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4f7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5d500b8400 0x7f5d500cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56092/0 (socket says 172.21.15.60:56092) 2023-07-13T22:29:21.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4f7fe700 1 -- 172.21.15.60:0/2701782076 learned_addr learned my addr 172.21.15.60:0/2701782076 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:21.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4ffff700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d500cfca0 0x7f5d50167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368827556 0 0) 0x7f5d40001110 con 0x7f5d500b8400 2023-07-13T22:29:21.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d380035b0 con 0x7f5d500b8400 2023-07-13T22:29:21.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3800847203 0 0) 0x7f5d380035b0 con 0x7f5d500b8400 2023-07-13T22:29:21.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d500cfca0 msgr2=0x7f5d50167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:21.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d500cfca0 0x7f5d50167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 >> v1:172.21.15.105:6789/0 conn(0x7f5d500d3750 legacy=0x7f5d500cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500b4eb0 con 0x7f5d500b8400 2023-07-13T22:29:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f5d40001800 con 0x7f5d500b8400 2023-07-13T22:29:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 >> v1:172.21.15.60:6789/0 conn(0x7f5d500b8400 legacy=0x7f5d500cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38003a30 0x7f5d38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d500cfca0 0x7f5d38004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d38004d90 0x7f5d38005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500b87d0 con 0x7f5d38003a30 2023-07-13T22:29:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.685+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3800bac0 con 0x7f5d38004d90 2023-07-13T22:29:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3800bc80 con 0x7f5d500cfca0 2023-07-13T22:29:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4f7fe700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38003a30 0x7f5d38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4ffff700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d38004d90 0x7f5d38005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4effd700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d500cfca0 0x7f5d38004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4f7fe700 1 -- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d500cfca0 msgr2=0x7f5d38004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4f7fe700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d500cfca0 0x7f5d38004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4f7fe700 1 -- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d38004d90 msgr2=0x7f5d38005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:21.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4f7fe700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d38004d90 0x7f5d38005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4f7fe700 1 -- 172.21.15.60:0/2701782076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d380035b0 con 0x7f5d38003a30 2023-07-13T22:29:21.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4effd700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d500cfca0 0x7f5d38004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:21.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.686+0000 7f5d4f7fe700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38003a30 0x7f5d38003e00 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c1c0 tx=0x7f5d38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a675b7885af7711 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:21.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000cd60 con 0x7f5d38003a30 2023-07-13T22:29:21.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40005960 con 0x7f5d38003a30 2023-07-13T22:29:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d4e7fc700 1 -- 172.21.15.60:0/2701782076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40005d10 con 0x7f5d38003a30 2023-07-13T22:29:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d561ff700 1 -- 172.21.15.60:0/2701782076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38003a30 msgr2=0x7f5d38003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d561ff700 1 --2- 172.21.15.60:0/2701782076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d38003a30 0x7f5d38003e00 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c1c0 tx=0x7f5d38005880 comp rx=0 tx=0).stop 2023-07-13T22:29:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d4ffff700 1 -- 172.21.15.60:0/2701782076 reap_dead start 2023-07-13T22:29:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d561ff700 1 -- 172.21.15.60:0/2701782076 shutdown_connections 2023-07-13T22:29:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d561ff700 1 -- 172.21.15.60:0/2701782076 >> 172.21.15.60:0/2701782076 conn(0x7f5d5006b390 msgr2=0x7f5d5006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d561ff700 1 -- 172.21.15.60:0/2701782076 shutdown_connections 2023-07-13T22:29:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.687+0000 7f5d561ff700 1 -- 172.21.15.60:0/2701782076 wait complete. 2023-07-13T22:29:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.689+0000 7f5d561ff700 1 Processor -- start 2023-07-13T22:29:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.689+0000 7f5d561ff700 1 -- start start 2023-07-13T22:29:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.690+0000 7f5d561ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d38003a30 0x7f5d501595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.690+0000 7f5d561ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d50159e00 0x7f5d5015bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.690+0000 7f5d561ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5015c1c0 0x7f5d5015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.690+0000 7f5d561ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d501f2330 con 0x7f5d50159e00 2023-07-13T22:29:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.690+0000 7f5d561ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50159af0 con 0x7f5d5015c1c0 2023-07-13T22:29:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.690+0000 7f5d561ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5015d140 con 0x7f5d38003a30 2023-07-13T22:29:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.690+0000 7f5d4effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d50159e00 0x7f5d5015bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.690+0000 7f5d4effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d50159e00 0x7f5d5015bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41854/0 (socket says 172.21.15.60:41854) 2023-07-13T22:29:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.690+0000 7f5d4effd700 1 -- 172.21.15.60:0/261850662 learned_addr learned my addr 172.21.15.60:0/261850662 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d4f7fe700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d38003a30 0x7f5d501595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d4ffff700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5015c1c0 0x7f5d5015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d4effd700 1 -- 172.21.15.60:0/261850662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d38003a30 msgr2=0x7f5d501595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d4effd700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d38003a30 0x7f5d501595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d4effd700 1 -- 172.21.15.60:0/261850662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5015c1c0 msgr2=0x7f5d5015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d4effd700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5015c1c0 0x7f5d5015fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d4effd700 1 -- 172.21.15.60:0/261850662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5015c660 con 0x7f5d50159e00 2023-07-13T22:29:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d4effd700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d50159e00 0x7f5d5015bc80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400e780 tx=0x7f5d4400ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d37fff700 1 -- 172.21.15.60:0/261850662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d440052e0 con 0x7f5d50159e00 2023-07-13T22:29:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d37fff700 1 -- 172.21.15.60:0/261850662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d44005440 con 0x7f5d50159e00 2023-07-13T22:29:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d37fff700 1 -- 172.21.15.60:0/261850662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44003880 con 0x7f5d50159e00 2023-07-13T22:29:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.691+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5015c940 con 0x7f5d50159e00 2023-07-13T22:29:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.692+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5015d670 con 0x7f5d50159e00 2023-07-13T22:29:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.692+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d500b1e80 con 0x7f5d50159e00 2023-07-13T22:29:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.694+0000 7f5d37fff700 1 -- 172.21.15.60:0/261850662 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d440039e0 con 0x7f5d50159e00 2023-07-13T22:29:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.694+0000 7f5d37fff700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d3c06ace0 0x7f5d3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.695+0000 7f5d37fff700 1 -- 172.21.15.60:0/261850662 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d44088830 con 0x7f5d50159e00 2023-07-13T22:29:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.695+0000 7f5d4f7fe700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d3c06ace0 0x7f5d3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.695+0000 7f5d4f7fe700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d3c06ace0 0x7f5d3c06d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5d50160cd0 tx=0x7f5d40007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.699+0000 7f5d37fff700 1 -- 172.21.15.60:0/261850662 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5d440538d0 con 0x7f5d50159e00 2023-07-13T22:29:21.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.962+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d500d0250 con 0x7f5d50159e00 2023-07-13T22:29:21.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.962+0000 7f5d37fff700 1 -- 172.21.15.60:0/261850662 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f5d44056ef0 con 0x7f5d50159e00 2023-07-13T22:29:21.942 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:21.942 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":64,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:58.254590+0000","modified":"2023-07-13T22:29:19.596439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6837/4078186433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":4078186433},{"type":"v1","addr":"172.21.15.60:6837","nonce":4078186433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-13T22:29:21.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d3c06ace0 msgr2=0x7f5d3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d3c06ace0 0x7f5d3c06d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5d50160cd0 tx=0x7f5d40007000 comp rx=0 tx=0).stop 2023-07-13T22:29:21.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d50159e00 msgr2=0x7f5d5015bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:21.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d50159e00 0x7f5d5015bc80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400e780 tx=0x7f5d4400ea90 comp rx=0 tx=0).stop 2023-07-13T22:29:21.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 shutdown_connections 2023-07-13T22:29:21.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5d3c06ace0 0x7f5d3c06d190 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5015c1c0 0x7f5d5015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d38003a30 0x7f5d501595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 --2- 172.21.15.60:0/261850662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d50159e00 0x7f5d5015bc80 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:21.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 >> 172.21.15.60:0/261850662 conn(0x7f5d5006b390 msgr2=0x7f5d5015b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:21.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 shutdown_connections 2023-07-13T22:29:21.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:21.965+0000 7f5d561ff700 1 -- 172.21.15.60:0/261850662 wait complete. 2023-07-13T22:29:21.946 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 64 2023-07-13T22:29:21.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-13T22:29:22.192 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":6698,"entity":{"name":{"type":"client","num":6698},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1045006933}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":12.903869053999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9331315546144405,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6698 192.168.0.1:0/1045006933","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"125439","root":"/"}},{"id":8082,"entity":{"name":{"type":"client","num":8082},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2416976467}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":738,"uptime":15.257845166999999,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2963.8732296614039,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":854.09766565955272,"halflife":1.5},"recall_caps_throttle2o":{"value":287.28313864846399,"halflife":0.5},"session_cache_liveness":{"value":1578.7224023847966,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":507,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8082 192.168.0.1:0/2416976467","completed_requests":[{"tid":507,"created_ino":"0x100000003e9"},{"tid":508,"created_ino":"0x100000003ea"},{"tid":509,"created_ino":"0x100000003eb"},{"tid":510,"created_ino":"0x100000003ec"},{"tid":511,"created_ino":"0x100000003ed"},{"tid":512,"created_ino":"0x100000003ee"},{"tid":513,"created_ino":"0x100000003ef"},{"tid":514,"created_ino":"0x100000003f0"},{"tid":515,"created_ino":"0x100000003f1"},{"tid":516,"created_ino":"0x100000003f2"},{"tid":517,"created_ino":"0x100000003f3"},{"tid":518,"created_ino":"0x100000003f4"},{"tid":519,"created_ino":"0x100000003f5"},{"tid":520,"created_ino":"0x100000003f6"},{"tid":521,"created_ino":"0x100000003f7"},{"tid":522,"created_ino":"0x100000003f8"},{"tid":523,"created_ino":"0x100000003f9"},{"tid":524,"created_ino":"0x100000003fa"},{"tid":525,"created_ino":"0x100000003fb"},{"tid":526,"created_ino":"0x100000003fc"},{"tid":527,"created_ino":"0x100000003fd"},{"tid":528,"created_ino":"0x100000003fe"},{"tid":529,"created_ino":"0x100000003ff"},{"tid":530,"created_ino":"0x10000000400"},{"tid":531,"created_ino":"0x10000000401"},{"tid":532,"created_ino":"0x10000000402"},{"tid":533,"created_ino":"0x10000000403"},{"tid":534,"created_ino":"0x10000000404"},{"tid":535,"created_ino":"0x10000000405"},{"tid":536,"created_ino":"0x10000000406"},{"tid":537,"created_ino":"0x10000000407"},{"tid":538,"created_ino":"0x10000000408"},{"tid":539,"created_ino":"0x10000000409"},{"tid":540,"created_ino":"0x1000000040a"},{"tid":541,"created_ino":"0x1000000040b"},{"tid":542,"created_ino":"0x1000000040c"},{"tid":543,"created_ino":"0x1000000040d"},{"tid":544,"created_ino":"0x1000000040e"},{"tid":545,"created_ino":"0x1000000040f"},{"tid":546,"created_ino":"0x10000000410"},{"tid":547,"created_ino":"0x10000000411"},{"tid":548,"created_ino":"0x10000000412"},{"tid":549,"created_ino":"0x10000000413"},{"tid":550,"created_ino":"0x10000000414"},{"tid":551,"created_ino":"0x10000000415"},{"tid":552,"created_ino":"0x10000000416"},{"tid":553,"created_ino":"0x10000000417"},{"tid":554,"created_ino":"0x10000000418"},{"tid":555,"created_ino":"0x10000000419"},{"tid":556,"created_ino":"0x1000000041a"},{"tid":557,"created_ino":"0x1000000041b"},{"tid":558,"created_ino":"0x1000000041c"},{"tid":559,"created_ino":"0x1000000041d"},{"tid":560,"created_ino":"0x1000000041e"},{"tid":561,"created_ino":"0x1000000041f"},{"tid":562,"created_ino":"0x10000000420"},{"tid":563,"created_ino":"0x10000000421"},{"tid":564,"created_ino":"0x10000000422"},{"tid":565,"created_ino":"0x10000000423"},{"tid":566,"created_ino":"0x10000000424"},{"tid":567,"created_ino":"0x10000000425"},{"tid":568,"created_ino":"0x10000000426"},{"tid":569,"created_ino":"0x10000000427"},{"tid":570,"created_ino":"0x10000000428"},{"tid":571,"created_ino":"0x10000000429"},{"tid":572,"created_ino":"0x1000000042a"},{"tid":573,"created_ino":"0x1000000042b"},{"tid":574,"created_ino":"0x1000000042c"},{"tid":575,"created_ino":"0x1000000042d"},{"tid":576,"created_ino":"0x1000000042e"},{"tid":577,"created_ino":"0x1000000042f"},{"tid":578,"created_ino":"0x10000000430"},{"tid":579,"created_ino":"0x10000000431"},{"tid":580,"created_ino":"0x10000000432"},{"tid":581,"created_ino":"0x10000000433"},{"tid":582,"created_ino":"0x10000000434"},{"tid":583,"created_ino":"0x10000000435"},{"tid":584,"created_ino":"0x10000000436"},{"tid":585,"created_ino":"0x10000000437"},{"tid":586,"created_ino":"0x10000000438"},{"tid":587,"created_ino":"0x10000000439"},{"tid":588,"created_ino":"0x1000000043a"},{"tid":589,"created_ino":"0x1000000043b"},{"tid":590,"created_ino":"0x1000000043c"},{"tid":591,"created_ino":"0x1000000043d"},{"tid":592,"created_ino":"0x1000000043e"},{"tid":593,"created_ino":"0x1000000043f"},{"tid":594,"created_ino":"0x10000000440"},{"tid":595,"created_ino":"0x10000000441"},{"tid":596,"created_ino":"0x10000000442"},{"tid":597,"created_ino":"0x10000000443"},{"tid":598,"created_ino":"0x10000000444"},{"tid":599,"created_ino":"0x10000000445"},{"tid":600,"created_ino":"0x10000000446"},{"tid":601,"created_ino":"0x10000000447"},{"tid":602,"created_ino":"0x10000000448"},{"tid":603,"created_ino":"0x10000000449"},{"tid":604,"created_ino":"0x1000000044a"},{"tid":605,"created_ino":"0x1000000044b"},{"tid":606,"created_ino":"0x1000000044c"},{"tid":607,"created_ino":"0x1000000044d"},{"tid":608,"created_ino":"0x1000000044e"},{"tid":609,"created_ino":"0x1000000044f"},{"tid":610,"created_ino":"0x10000000450"},{"tid":611,"created_ino":"0x10000000451"},{"tid":612,"created_ino":"0x10000000452"},{"tid":613,"created_ino":"0x10000000453"},{"tid":614,"created_ino":"0x10000000454"},{"tid":615,"created_ino":"0x10000000455"},{"tid":616,"created_ino":"0x10000000456"},{"tid":617,"created_ino":"0x10000000457"},{"tid":618,"created_ino":"0x10000000458"},{"tid":619,"created_ino":"0x10000000459"},{"tid":620,"created_ino":"0x1000000045a"},{"tid":621,"created_ino":"0x1000000045b"},{"tid":622,"created_ino":"0x1000000045c"},{"tid":623,"created_ino":"0x1000000045d"},{"tid":624,"created_ino":"0x1000000045e"},{"tid":625,"created_ino":"0x1000000045f"},{"tid":626,"created_ino":"0x10000000460"},{"tid":627,"created_ino":"0x10000000461"},{"tid":628,"created_ino":"0x10000000462"},{"tid":629,"created_ino":"0x10000000463"},{"tid":630,"created_ino":"0x10000000464"},{"tid":631,"created_ino":"0x10000000465"},{"tid":632,"created_ino":"0x10000000466"},{"tid":633,"created_ino":"0x10000000467"},{"tid":634,"created_ino":"0x10000000468"},{"tid":635,"created_ino":"0x10000000469"},{"tid":636,"created_ino":"0x1000000046a"},{"tid":637,"created_ino":"0x1000000046b"},{"tid":638,"created_ino":"0x1000000046c"},{"tid":639,"created_ino":"0x1000000046d"},{"tid":640,"created_ino":"0x1000000046e"},{"tid":641,"created_ino":"0x1000000046f"},{"tid":642,"created_ino":"0x10000000470"},{"tid":643,"created_ino":"0x10000000471"},{"tid":644,"created_ino":"0x10000000472"},{"tid":645,"created_ino":"0x10000000473"},{"tid":646,"created_ino":"0x10000000474"},{"tid":647,"created_ino":"0x10000000475"},{"tid":648,"created_ino":"0x10000000476"},{"tid":649,"created_ino":"0x10000000477"},{"tid":650,"created_ino":"0x10000000478"},{"tid":651,"created_ino":"0x10000000479"},{"tid":652,"created_ino":"0x1000000047a"},{"tid":653,"created_ino":"0x1000000047b"},{"tid":654,"created_ino":"0x1000000047c"},{"tid":655,"created_ino":"0x1000000047d"},{"tid":656,"created_ino":"0x1000000047e"},{"tid":657,"created_ino":"0x1000000047f"},{"tid":658,"created_ino":"0x10000000480"},{"tid":659,"created_ino":"0x10000000481"},{"tid":660,"created_ino":"0x10000000482"},{"tid":661,"created_ino":"0x10000000483"},{"tid":662,"created_ino":"0x10000000484"},{"tid":663,"created_ino":"0x10000000485"},{"tid":664,"created_ino":"0x10000000486"},{"tid":665,"created_ino":"0x10000000487"},{"tid":666,"created_ino":"0x10000000488"},{"tid":667,"created_ino":"0x10000000489"},{"tid":668,"created_ino":"0x1000000048a"},{"tid":669,"created_ino":"0x1000000048b"},{"tid":670,"created_ino":"0x1000000048c"},{"tid":671,"created_ino":"0x1000000048d"},{"tid":672,"created_ino":"0x1000000048e"},{"tid":673,"created_ino":"0x1000000048f"},{"tid":674,"created_ino":"0x10000000490"},{"tid":675,"created_ino":"0x10000000491"},{"tid":676,"created_ino":"0x10000000492"},{"tid":677,"created_ino":"0x10000000493"},{"tid":678,"created_ino":"0x10000000494"},{"tid":679,"created_ino":"0x10000000495"},{"tid":680,"created_ino":"0x10000000496"},{"tid":681,"created_ino":"0x10000000497"},{"tid":682,"created_ino":"0x10000000498"},{"tid":683,"created_ino":"0x10000000499"},{"tid":684,"created_ino":"0x1000000049a"},{"tid":685,"created_ino":"0x1000000049b"},{"tid":686,"created_ino":"0x1000000049c"},{"tid":687,"created_ino":"0x1000000049d"},{"tid":688,"created_ino":"0x1000000049e"},{"tid":689,"created_ino":"0x1000000049f"},{"tid":690,"created_ino":"0x100000004a0"},{"tid":691,"created_ino":"0x100000004a1"},{"tid":692,"created_ino":"0x100000004a2"},{"tid":693,"created_ino":"0x100000004a3"},{"tid":694,"created_ino":"0x100000004a4"},{"tid":695,"created_ino":"0x100000004a5"},{"tid":696,"created_ino":"0x100000004a6"},{"tid":697,"created_ino":"0x100000004a7"},{"tid":698,"created_ino":"0x100000004a8"},{"tid":699,"created_ino":"0x100000004a9"},{"tid":700,"created_ino":"0x100000004aa"},{"tid":701,"created_ino":"0x100000004ab"},{"tid":702,"created_ino":"0x100000004ac"},{"tid":703,"created_ino":"0x100000004ad"},{"tid":704,"created_ino":"0x100000004ae"},{"tid":705,"created_ino":"0x100000004af"},{"tid":706,"created_ino":"0x100000004b0"},{"tid":707,"created_ino":"0x100000004b1"},{"tid":708,"created_ino":"0x100000004b2"},{"tid":709,"created_ino":"0x100000004b3"},{"tid":710,"created_ino":"0x100000004b4"},{"tid":711,"created_ino":"0x100000004b5"},{"tid":712,"created_ino":"0x100000004b6"},{"tid":713,"created_ino":"0x100000004b7"},{"tid":714,"created_ino":"0x100000004b8"},{"tid":715,"created_ino":"0x100000004b9"},{"tid":716,"created_ino":"0x100000004ba"},{"tid":717,"created_ino":"0x100000004bb"},{"tid":718,"created_ino":"0x100000004bc"},{"tid":719,"created_ino":"0x100000004bd"},{"tid":720,"created_ino":"0x100000004be"},{"tid":721,"created_ino":"0x100000004bf"},{"tid":722,"created_ino":"0x100000004c0"},{"tid":723,"created_ino":"0x100000004c1"},{"tid":724,"created_ino":"0x100000004c2"},{"tid":725,"created_ino":"0x100000004c3"},{"tid":726,"created_ino":"0x100000004c4"},{"tid":727,"created_ino":"0x100000004c5"},{"tid":728,"created_ino":"0x100000004c6"},{"tid":729,"created_ino":"0x100000004c7"},{"tid":730,"created_ino":"0x100000004c8"},{"tid":731,"created_ino":"0x100000004c9"},{"tid":732,"created_ino":"0x100000004ca"},{"tid":733,"created_ino":"0x100000004cb"},{"tid":734,"created_ino":"0x100000004cc"},{"tid":735,"created_ino":"0x100000004cd"},{"tid":736,"created_ino":"0x100000004ce"},{"tid":737,"created_ino":"0x100000004cf"},{"tid":738,"created_ino":"0x100000004d0"},{"tid":739,"created_ino":"0x100000004d1"},{"tid":740,"created_ino":"0x100000004d2"},{"tid":741,"created_ino":"0x100000004d3"},{"tid":742,"created_ino":"0x100000004d4"},{"tid":743,"created_ino":"0x100000004d5"},{"tid":744,"created_ino":"0x100000004d6"},{"tid":745,"created_ino":"0x100000004d7"},{"tid":746,"created_ino":"0x100000004d8"},{"tid":747,"created_ino":"0x100000004d9"},{"tid":748,"created_ino":"0x100000004da"},{"tid":749,"created_ino":"0x100000004db"},{"tid":750,"created_ino":"0x100000004dc"},{"tid":751,"created_ino":"0x100000004dd"},{"tid":752,"created_ino":"0x100000004de"},{"tid":753,"created_ino":"0x100000004df"},{"tid":754,"created_ino":"0x100000004e0"},{"tid":755,"created_ino":"0x100000004e1"},{"tid":756,"created_ino":"0x100000004e2"},{"tid":757,"created_ino":"0x100000004e3"},{"tid":758,"created_ino":"0x100000004e4"},{"tid":759,"created_ino":"0x100000004e5"},{"tid":760,"created_ino":"0x100000004e6"},{"tid":761,"created_ino":"0x100000004e7"},{"tid":762,"created_ino":"0x100000004e8"},{"tid":763,"created_ino":"0x100000004e9"},{"tid":764,"created_ino":"0x100000004ea"},{"tid":765,"created_ino":"0x100000004eb"},{"tid":766,"created_ino":"0x100000004ec"},{"tid":767,"created_ino":"0x100000004ed"},{"tid":768,"created_ino":"0x100000004ee"},{"tid":769,"created_ino":"0x100000004ef"},{"tid":770,"created_ino":"0x100000004f0"},{"tid":771,"created_ino":"0x100000004f1"},{"tid":772,"created_ino":"0x100000004f2"},{"tid":773,"created_ino":"0x100000004f3"},{"tid":774,"created_ino":"0x100000004f4"},{"tid":775,"created_ino":"0x100000004f5"},{"tid":776,"created_ino":"0x100000004f6"},{"tid":777,"created_ino":"0x100000004f7"},{"tid":778,"created_ino":"0x100000004f8"},{"tid":779,"created_ino":"0x100000004f9"},{"tid":780,"created_ino":"0x100000004fa"},{"tid":781,"created_ino":"0x100000004fb"},{"tid":782,"created_ino":"0x100000004fc"},{"tid":783,"created_ino":"0x100000004fd"},{"tid":784,"created_ino":"0x100000004fe"},{"tid":785,"created_ino":"0x100000004ff"},{"tid":786,"created_ino":"0x10000000500"},{"tid":787,"created_ino":"0x10000000501"},{"tid":788,"created_ino":"0x10000000502"},{"tid":789,"created_ino":"0x10000000503"},{"tid":790,"created_ino":"0x10000000504"},{"tid":791,"created_ino":"0x10000000505"},{"tid":792,"created_ino":"0x10000000506"},{"tid":793,"created_ino":"0x10000000507"},{"tid":794,"created_ino":"0x10000000508"},{"tid":795,"created_ino":"0x10000000509"},{"tid":796,"created_ino":"0x1000000050a"},{"tid":797,"created_ino":"0x1000000050b"},{"tid":798,"created_ino":"0x1000000050c"},{"tid":799,"created_ino":"0x1000000050d"},{"tid":800,"created_ino":"0x1000000050e"},{"tid":801,"created_ino":"0x1000000050f"},{"tid":802,"created_ino":"0x10000000510"},{"tid":803,"created_ino":"0x10000000511"},{"tid":804,"created_ino":"0x10000000512"},{"tid":805,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"152462","root":"/"}}] 2023-07-13T22:29:22.194 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "125439", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1045006933, "type": "any" }, "name": { "num": 6698, "type": "client" } }, "id": 6698, "inst": "client.6698 192.168.0.1:0/1045006933", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9331315546144405 }, "state": "open", "uptime": 12.903869054 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "152462", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 507 }, { "created_ino": "0x100000003ea", "tid": 508 }, { "created_ino": "0x100000003eb", "tid": 509 }, { "created_ino": "0x100000003ec", "tid": 510 }, { "created_ino": "0x100000003ed", "tid": 511 }, { "created_ino": "0x100000003ee", "tid": 512 }, { "created_ino": "0x100000003ef", "tid": 513 }, { "created_ino": "0x100000003f0", "tid": 514 }, { "created_ino": "0x100000003f1", "tid": 515 }, { "created_ino": "0x100000003f2", "tid": 516 }, { "created_ino": "0x100000003f3", "tid": 517 }, { "created_ino": "0x100000003f4", "tid": 518 }, { "created_ino": "0x100000003f5", "tid": 519 }, { "created_ino": "0x100000003f6", "tid": 520 }, { "created_ino": "0x100000003f7", "tid": 521 }, { "created_ino": "0x100000003f8", "tid": 522 }, { "created_ino": "0x100000003f9", "tid": 523 }, { "created_ino": "0x100000003fa", "tid": 524 }, { "created_ino": "0x100000003fb", "tid": 525 }, { "created_ino": "0x100000003fc", "tid": 526 }, { "created_ino": "0x100000003fd", "tid": 527 }, { "created_ino": "0x100000003fe", "tid": 528 }, { "created_ino": "0x100000003ff", "tid": 529 }, { "created_ino": "0x10000000400", "tid": 530 }, { "created_ino": "0x10000000401", "tid": 531 }, { "created_ino": "0x10000000402", "tid": 532 }, { "created_ino": "0x10000000403", "tid": 533 }, { "created_ino": "0x10000000404", "tid": 534 }, { "created_ino": "0x10000000405", "tid": 535 }, { "created_ino": "0x10000000406", "tid": 536 }, { "created_ino": "0x10000000407", "tid": 537 }, { "created_ino": "0x10000000408", "tid": 538 }, { "created_ino": "0x10000000409", "tid": 539 }, { "created_ino": "0x1000000040a", "tid": 540 }, { "created_ino": "0x1000000040b", "tid": 541 }, { "created_ino": "0x1000000040c", "tid": 542 }, { "created_ino": "0x1000000040d", "tid": 543 }, { "created_ino": "0x1000000040e", "tid": 544 }, { "created_ino": "0x1000000040f", "tid": 545 }, { "created_ino": "0x10000000410", "tid": 546 }, { "created_ino": "0x10000000411", "tid": 547 }, { "created_ino": "0x10000000412", "tid": 548 }, { "created_ino": "0x10000000413", "tid": 549 }, { "created_ino": "0x10000000414", "tid": 550 }, { "created_ino": "0x10000000415", "tid": 551 }, { "created_ino": "0x10000000416", "tid": 552 }, { "created_ino": "0x10000000417", "tid": 553 }, { "created_ino": "0x10000000418", "tid": 554 }, { "created_ino": "0x10000000419", "tid": 555 }, { "created_ino": "0x1000000041a", "tid": 556 }, { "created_ino": "0x1000000041b", "tid": 557 }, { "created_ino": "0x1000000041c", "tid": 558 }, { "created_ino": "0x1000000041d", "tid": 559 }, { "created_ino": "0x1000000041e", "tid": 560 }, { "created_ino": "0x1000000041f", "tid": 561 }, { "created_ino": "0x10000000420", "tid": 562 }, { "created_ino": "0x10000000421", "tid": 563 }, { "created_ino": "0x10000000422", "tid": 564 }, { "created_ino": "0x10000000423", "tid": 565 }, { "created_ino": "0x10000000424", "tid": 566 }, { "created_ino": "0x10000000425", "tid": 567 }, { "created_ino": "0x10000000426", "tid": 568 }, { "created_ino": "0x10000000427", "tid": 569 }, { "created_ino": "0x10000000428", "tid": 570 }, { "created_ino": "0x10000000429", "tid": 571 }, { "created_ino": "0x1000000042a", "tid": 572 }, { "created_ino": "0x1000000042b", "tid": 573 }, { "created_ino": "0x1000000042c", "tid": 574 }, { "created_ino": "0x1000000042d", "tid": 575 }, { "created_ino": "0x1000000042e", "tid": 576 }, { "created_ino": "0x1000000042f", "tid": 577 }, { "created_ino": "0x10000000430", "tid": 578 }, { "created_ino": "0x10000000431", "tid": 579 }, { "created_ino": "0x10000000432", "tid": 580 }, { "created_ino": "0x10000000433", "tid": 581 }, { "created_ino": "0x10000000434", "tid": 582 }, { "created_ino": "0x10000000435", "tid": 583 }, { "created_ino": "0x10000000436", "tid": 584 }, { "created_ino": "0x10000000437", "tid": 585 }, { "created_ino": "0x10000000438", "tid": 586 }, { "created_ino": "0x10000000439", "tid": 587 }, { "created_ino": "0x1000000043a", "tid": 588 }, { "created_ino": "0x1000000043b", "tid": 589 }, { "created_ino": "0x1000000043c", "tid": 590 }, { "created_ino": "0x1000000043d", "tid": 591 }, { "created_ino": "0x1000000043e", "tid": 592 }, { "created_ino": "0x1000000043f", "tid": 593 }, { "created_ino": "0x10000000440", "tid": 594 }, { "created_ino": "0x10000000441", "tid": 595 }, { "created_ino": "0x10000000442", "tid": 596 }, { "created_ino": "0x10000000443", "tid": 597 }, { "created_ino": "0x10000000444", "tid": 598 }, { "created_ino": "0x10000000445", "tid": 599 }, { "created_ino": "0x10000000446", "tid": 600 }, { "created_ino": "0x10000000447", "tid": 601 }, { "created_ino": "0x10000000448", "tid": 602 }, { "created_ino": "0x10000000449", "tid": 603 }, { "created_ino": "0x1000000044a", "tid": 604 }, { "created_ino": "0x1000000044b", "tid": 605 }, { "created_ino": "0x1000000044c", "tid": 606 }, { "created_ino": "0x1000000044d", "tid": 607 }, { "created_ino": "0x1000000044e", "tid": 608 }, { "created_ino": "0x1000000044f", "tid": 609 }, { "created_ino": "0x10000000450", "tid": 610 }, { "created_ino": "0x10000000451", "tid": 611 }, { "created_ino": "0x10000000452", "tid": 612 }, { "created_ino": "0x10000000453", "tid": 613 }, { "created_ino": "0x10000000454", "tid": 614 }, { "created_ino": "0x10000000455", "tid": 615 }, { "created_ino": "0x10000000456", "tid": 616 }, { "created_ino": "0x10000000457", "tid": 617 }, { "created_ino": "0x10000000458", "tid": 618 }, { "created_ino": "0x10000000459", "tid": 619 }, { "created_ino": "0x1000000045a", "tid": 620 }, { "created_ino": "0x1000000045b", "tid": 621 }, { "created_ino": "0x1000000045c", "tid": 622 }, { "created_ino": "0x1000000045d", "tid": 623 }, { "created_ino": "0x1000000045e", "tid": 624 }, { "created_ino": "0x1000000045f", "tid": 625 }, { "created_ino": "0x10000000460", "tid": 626 }, { "created_ino": "0x10000000461", "tid": 627 }, { "created_ino": "0x10000000462", "tid": 628 }, { "created_ino": "0x10000000463", "tid": 629 }, { "created_ino": "0x10000000464", "tid": 630 }, { "created_ino": "0x10000000465", "tid": 631 }, { "created_ino": "0x10000000466", "tid": 632 }, { "created_ino": "0x10000000467", "tid": 633 }, { "created_ino": "0x10000000468", "tid": 634 }, { "created_ino": "0x10000000469", "tid": 635 }, { "created_ino": "0x1000000046a", "tid": 636 }, { "created_ino": "0x1000000046b", "tid": 637 }, { "created_ino": "0x1000000046c", "tid": 638 }, { "created_ino": "0x1000000046d", "tid": 639 }, { "created_ino": "0x1000000046e", "tid": 640 }, { "created_ino": "0x1000000046f", "tid": 641 }, { "created_ino": "0x10000000470", "tid": 642 }, { "created_ino": "0x10000000471", "tid": 643 }, { "created_ino": "0x10000000472", "tid": 644 }, { "created_ino": "0x10000000473", "tid": 645 }, { "created_ino": "0x10000000474", "tid": 646 }, { "created_ino": "0x10000000475", "tid": 647 }, { "created_ino": "0x10000000476", "tid": 648 }, { "created_ino": "0x10000000477", "tid": 649 }, { "created_ino": "0x10000000478", "tid": 650 }, { "created_ino": "0x10000000479", "tid": 651 }, { "created_ino": "0x1000000047a", "tid": 652 }, { "created_ino": "0x1000000047b", "tid": 653 }, { "created_ino": "0x1000000047c", "tid": 654 }, { "created_ino": "0x1000000047d", "tid": 655 }, { "created_ino": "0x1000000047e", "tid": 656 }, { "created_ino": "0x1000000047f", "tid": 657 }, { "created_ino": "0x10000000480", "tid": 658 }, { "created_ino": "0x10000000481", "tid": 659 }, { "created_ino": "0x10000000482", "tid": 660 }, { "created_ino": "0x10000000483", "tid": 661 }, { "created_ino": "0x10000000484", "tid": 662 }, { "created_ino": "0x10000000485", "tid": 663 }, { "created_ino": "0x10000000486", "tid": 664 }, { "created_ino": "0x10000000487", "tid": 665 }, { "created_ino": "0x10000000488", "tid": 666 }, { "created_ino": "0x10000000489", "tid": 667 }, { "created_ino": "0x1000000048a", "tid": 668 }, { "created_ino": "0x1000000048b", "tid": 669 }, { "created_ino": "0x1000000048c", "tid": 670 }, { "created_ino": "0x1000000048d", "tid": 671 }, { "created_ino": "0x1000000048e", "tid": 672 }, { "created_ino": "0x1000000048f", "tid": 673 }, { "created_ino": "0x10000000490", "tid": 674 }, { "created_ino": "0x10000000491", "tid": 675 }, { "created_ino": "0x10000000492", "tid": 676 }, { "created_ino": "0x10000000493", "tid": 677 }, { "created_ino": "0x10000000494", "tid": 678 }, { "created_ino": "0x10000000495", "tid": 679 }, { "created_ino": "0x10000000496", "tid": 680 }, { "created_ino": "0x10000000497", "tid": 681 }, { "created_ino": "0x10000000498", "tid": 682 }, { "created_ino": "0x10000000499", "tid": 683 }, { "created_ino": "0x1000000049a", "tid": 684 }, { "created_ino": "0x1000000049b", "tid": 685 }, { "created_ino": "0x1000000049c", "tid": 686 }, { "created_ino": "0x1000000049d", "tid": 687 }, { "created_ino": "0x1000000049e", "tid": 688 }, { "created_ino": "0x1000000049f", "tid": 689 }, { "created_ino": "0x100000004a0", "tid": 690 }, { "created_ino": "0x100000004a1", "tid": 691 }, { "created_ino": "0x100000004a2", "tid": 692 }, { "created_ino": "0x100000004a3", "tid": 693 }, { "created_ino": "0x100000004a4", "tid": 694 }, { "created_ino": "0x100000004a5", "tid": 695 }, { "created_ino": "0x100000004a6", "tid": 696 }, { "created_ino": "0x100000004a7", "tid": 697 }, { "created_ino": "0x100000004a8", "tid": 698 }, { "created_ino": "0x100000004a9", "tid": 699 }, { "created_ino": "0x100000004aa", "tid": 700 }, { "created_ino": "0x100000004ab", "tid": 701 }, { "created_ino": "0x100000004ac", "tid": 702 }, { "created_ino": "0x100000004ad", "tid": 703 }, { "created_ino": "0x100000004ae", "tid": 704 }, { "created_ino": "0x100000004af", "tid": 705 }, { "created_ino": "0x100000004b0", "tid": 706 }, { "created_ino": "0x100000004b1", "tid": 707 }, { "created_ino": "0x100000004b2", "tid": 708 }, { "created_ino": "0x100000004b3", "tid": 709 }, { "created_ino": "0x100000004b4", "tid": 710 }, { "created_ino": "0x100000004b5", "tid": 711 }, { "created_ino": "0x100000004b6", "tid": 712 }, { "created_ino": "0x100000004b7", "tid": 713 }, { "created_ino": "0x100000004b8", "tid": 714 }, { "created_ino": "0x100000004b9", "tid": 715 }, { "created_ino": "0x100000004ba", "tid": 716 }, { "created_ino": "0x100000004bb", "tid": 717 }, { "created_ino": "0x100000004bc", "tid": 718 }, { "created_ino": "0x100000004bd", "tid": 719 }, { "created_ino": "0x100000004be", "tid": 720 }, { "created_ino": "0x100000004bf", "tid": 721 }, { "created_ino": "0x100000004c0", "tid": 722 }, { "created_ino": "0x100000004c1", "tid": 723 }, { "created_ino": "0x100000004c2", "tid": 724 }, { "created_ino": "0x100000004c3", "tid": 725 }, { "created_ino": "0x100000004c4", "tid": 726 }, { "created_ino": "0x100000004c5", "tid": 727 }, { "created_ino": "0x100000004c6", "tid": 728 }, { "created_ino": "0x100000004c7", "tid": 729 }, { "created_ino": "0x100000004c8", "tid": 730 }, { "created_ino": "0x100000004c9", "tid": 731 }, { "created_ino": "0x100000004ca", "tid": 732 }, { "created_ino": "0x100000004cb", "tid": 733 }, { "created_ino": "0x100000004cc", "tid": 734 }, { "created_ino": "0x100000004cd", "tid": 735 }, { "created_ino": "0x100000004ce", "tid": 736 }, { "created_ino": "0x100000004cf", "tid": 737 }, { "created_ino": "0x100000004d0", "tid": 738 }, { "created_ino": "0x100000004d1", "tid": 739 }, { "created_ino": "0x100000004d2", "tid": 740 }, { "created_ino": "0x100000004d3", "tid": 741 }, { "created_ino": "0x100000004d4", "tid": 742 }, { "created_ino": "0x100000004d5", "tid": 743 }, { "created_ino": "0x100000004d6", "tid": 744 }, { "created_ino": "0x100000004d7", "tid": 745 }, { "created_ino": "0x100000004d8", "tid": 746 }, { "created_ino": "0x100000004d9", "tid": 747 }, { "created_ino": "0x100000004da", "tid": 748 }, { "created_ino": "0x100000004db", "tid": 749 }, { "created_ino": "0x100000004dc", "tid": 750 }, { "created_ino": "0x100000004dd", "tid": 751 }, { "created_ino": "0x100000004de", "tid": 752 }, { "created_ino": "0x100000004df", "tid": 753 }, { "created_ino": "0x100000004e0", "tid": 754 }, { "created_ino": "0x100000004e1", "tid": 755 }, { "created_ino": "0x100000004e2", "tid": 756 }, { "created_ino": "0x100000004e3", "tid": 757 }, { "created_ino": "0x100000004e4", "tid": 758 }, { "created_ino": "0x100000004e5", "tid": 759 }, { "created_ino": "0x100000004e6", "tid": 760 }, { "created_ino": "0x100000004e7", "tid": 761 }, { "created_ino": "0x100000004e8", "tid": 762 }, { "created_ino": "0x100000004e9", "tid": 763 }, { "created_ino": "0x100000004ea", "tid": 764 }, { "created_ino": "0x100000004eb", "tid": 765 }, { "created_ino": "0x100000004ec", "tid": 766 }, { "created_ino": "0x100000004ed", "tid": 767 }, { "created_ino": "0x100000004ee", "tid": 768 }, { "created_ino": "0x100000004ef", "tid": 769 }, { "created_ino": "0x100000004f0", "tid": 770 }, { "created_ino": "0x100000004f1", "tid": 771 }, { "created_ino": "0x100000004f2", "tid": 772 }, { "created_ino": "0x100000004f3", "tid": 773 }, { "created_ino": "0x100000004f4", "tid": 774 }, { "created_ino": "0x100000004f5", "tid": 775 }, { "created_ino": "0x100000004f6", "tid": 776 }, { "created_ino": "0x100000004f7", "tid": 777 }, { "created_ino": "0x100000004f8", "tid": 778 }, { "created_ino": "0x100000004f9", "tid": 779 }, { "created_ino": "0x100000004fa", "tid": 780 }, { "created_ino": "0x100000004fb", "tid": 781 }, { "created_ino": "0x100000004fc", "tid": 782 }, { "created_ino": "0x100000004fd", "tid": 783 }, { "created_ino": "0x100000004fe", "tid": 784 }, { "created_ino": "0x100000004ff", "tid": 785 }, { "created_ino": "0x10000000500", "tid": 786 }, { "created_ino": "0x10000000501", "tid": 787 }, { "created_ino": "0x10000000502", "tid": 788 }, { "created_ino": "0x10000000503", "tid": 789 }, { "created_ino": "0x10000000504", "tid": 790 }, { "created_ino": "0x10000000505", "tid": 791 }, { "created_ino": "0x10000000506", "tid": 792 }, { "created_ino": "0x10000000507", "tid": 793 }, { "created_ino": "0x10000000508", "tid": 794 }, { "created_ino": "0x10000000509", "tid": 795 }, { "created_ino": "0x1000000050a", "tid": 796 }, { "created_ino": "0x1000000050b", "tid": 797 }, { "created_ino": "0x1000000050c", "tid": 798 }, { "created_ino": "0x1000000050d", "tid": 799 }, { "created_ino": "0x1000000050e", "tid": 800 }, { "created_ino": "0x1000000050f", "tid": 801 }, { "created_ino": "0x10000000510", "tid": 802 }, { "created_ino": "0x10000000511", "tid": 803 }, { "created_ino": "0x10000000512", "tid": 804 }, { "created_ino": "0x10000000513", "tid": 805 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2416976467, "type": "any" }, "name": { "num": 8082, "type": "client" } }, "id": 8082, "inst": "client.8082 192.168.0.1:0/2416976467", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 507, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2963.873229661404 }, "recall_caps_throttle": { "halflife": 1.5, "value": 854.0976656595527 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 287.283138648464 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 738, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1578.7224023847966 }, "state": "open", "uptime": 15.257845167 } ] 2023-07-13T22:29:22.195 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:29:22.195 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:29:22.196 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:29:22.297 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:29:22.410 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:29:22.410 INFO:tasks.cephfs.mount:Terminating background process 2023-07-13T22:29:22.411 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:29:22.411 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:29:22.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:22.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.725+0000 7f6c7e7ff700 1 Processor -- start 2023-07-13T22:29:22.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.725+0000 7f6c7e7ff700 1 -- start start 2023-07-13T22:29:22.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c7e7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6c780b8400 0x7f6c780cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:22.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c7e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c780d3750 0x7f6c780cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:22.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c7e7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c780cfad0 0x7f6c7815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:22.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c7e7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780cff20 con 0x7f6c780cfad0 2023-07-13T22:29:22.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c7e7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780d0ac0 con 0x7f6c780b8400 2023-07-13T22:29:22.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c7e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780b87d0 con 0x7f6c780d3750 2023-07-13T22:29:22.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c7cd9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c780cfad0 0x7f6c7815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:22.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c7cd9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c780cfad0 0x7f6c7815fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41864/0 (socket says 172.21.15.60:41864) 2023-07-13T22:29:22.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c7cd9c700 1 -- 172.21.15.60:0/3451971737 learned_addr learned my addr 172.21.15.60:0/3451971737 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:22.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c77fff700 1 --2- 172.21.15.60:0/3451971737 >> v2:172.21.15.105:3300/0 conn(0x7f6c780b8400 0x7f6c780cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:22.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.726+0000 7f6c777fe700 1 --2- 172.21.15.60:0/3451971737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c780d3750 0x7f6c780cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:22.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.727+0000 7f6c7cd9c700 1 -- 172.21.15.60:0/3451971737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c780d3750 msgr2=0x7f6c780cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:22.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.727+0000 7f6c7cd9c700 1 --2- 172.21.15.60:0/3451971737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c780d3750 0x7f6c780cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.727+0000 7f6c7cd9c700 1 -- 172.21.15.60:0/3451971737 >> v2:172.21.15.105:3300/0 conn(0x7f6c780b8400 msgr2=0x7f6c780cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:22.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.727+0000 7f6c7cd9c700 1 --2- 172.21.15.60:0/3451971737 >> v2:172.21.15.105:3300/0 conn(0x7f6c780b8400 0x7f6c780cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.727+0000 7f6c7cd9c700 1 -- 172.21.15.60:0/3451971737 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780b4eb0 con 0x7f6c780cfad0 2023-07-13T22:29:22.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.727+0000 7f6c7cd9c700 1 --2- 172.21.15.60:0/3451971737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780cfad0 0x7f6c7815fef0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6c780cf560 tx=0x7f6c7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9580419ad8a7acb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:22.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.727+0000 7f6c76ffd700 1 -- 172.21.15.60:0/3451971737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000e070 con 0x7f6c780cfad0 2023-07-13T22:29:22.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.727+0000 7f6c76ffd700 1 -- 172.21.15.60:0/3451971737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7000b910 con 0x7f6c780cfad0 2023-07-13T22:29:22.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.727+0000 7f6c76ffd700 1 -- 172.21.15.60:0/3451971737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000bcc0 con 0x7f6c780cfad0 2023-07-13T22:29:22.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.728+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/3451971737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780cfad0 msgr2=0x7f6c7815fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:22.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.728+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/3451971737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780cfad0 0x7f6c7815fef0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6c780cf560 tx=0x7f6c7000b330 comp rx=0 tx=0).stop 2023-07-13T22:29:22.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.728+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/3451971737 shutdown_connections 2023-07-13T22:29:22.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.728+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/3451971737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780cfad0 0x7f6c7815fef0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.728+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/3451971737 >> v2:172.21.15.105:3300/0 conn(0x7f6c780b8400 0x7f6c780cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.728+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/3451971737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c780d3750 0x7f6c780cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.728+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/3451971737 >> 172.21.15.60:0/3451971737 conn(0x7f6c7806b390 msgr2=0x7f6c7806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:22.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.728+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/3451971737 shutdown_connections 2023-07-13T22:29:22.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.728+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/3451971737 wait complete. 2023-07-13T22:29:22.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.730+0000 7f6c7e7ff700 1 Processor -- start 2023-07-13T22:29:22.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.730+0000 7f6c7e7ff700 1 -- start start 2023-07-13T22:29:22.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.731+0000 7f6c7e7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780b8400 0x7f6c78153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:22.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.731+0000 7f6c7e7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c780cfad0 0x7f6c7815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:22.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.731+0000 7f6c7e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c780d3750 0x7f6c7815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:22.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.731+0000 7f6c7e7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780c7ce0 con 0x7f6c780b8400 2023-07-13T22:29:22.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.731+0000 7f6c7e7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780c7410 con 0x7f6c780cfad0 2023-07-13T22:29:22.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.731+0000 7f6c7e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78159e70 con 0x7f6c780d3750 2023-07-13T22:29:22.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.731+0000 7f6c77fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780b8400 0x7f6c78153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:22.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.731+0000 7f6c77fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780b8400 0x7f6c78153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41866/0 (socket says 172.21.15.60:41866) 2023-07-13T22:29:22.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.731+0000 7f6c77fff700 1 -- 172.21.15.60:0/1072004348 learned_addr learned my addr 172.21.15.60:0/1072004348 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:22.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c77fff700 1 -- 172.21.15.60:0/1072004348 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c780d3750 msgr2=0x7f6c7815c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:22.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c77fff700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c780d3750 0x7f6c7815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c77fff700 1 -- 172.21.15.60:0/1072004348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c780cfad0 msgr2=0x7f6c7815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:22.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c77fff700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c780cfad0 0x7f6c7815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c77fff700 1 -- 172.21.15.60:0/1072004348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c70009b30 con 0x7f6c780b8400 2023-07-13T22:29:22.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c77fff700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780b8400 0x7f6c78153fa0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e870 tx=0x7f6c6800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:22.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c74ff9700 1 -- 172.21.15.60:0/1072004348 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68009590 con 0x7f6c780b8400 2023-07-13T22:29:22.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c74ff9700 1 -- 172.21.15.60:0/1072004348 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6800ee70 con 0x7f6c780b8400 2023-07-13T22:29:22.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c780c7620 con 0x7f6c780b8400 2023-07-13T22:29:22.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c74ff9700 1 -- 172.21.15.60:0/1072004348 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800ca50 con 0x7f6c780b8400 2023-07-13T22:29:22.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.732+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c781f3020 con 0x7f6c780b8400 2023-07-13T22:29:22.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.733+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c780c7e20 con 0x7f6c780b8400 2023-07-13T22:29:22.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.734+0000 7f6c74ff9700 1 -- 172.21.15.60:0/1072004348 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c6800cbb0 con 0x7f6c780b8400 2023-07-13T22:29:22.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.735+0000 7f6c74ff9700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c6406ace0 0x7f6c6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:22.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.735+0000 7f6c74ff9700 1 -- 172.21.15.60:0/1072004348 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c68089430 con 0x7f6c780b8400 2023-07-13T22:29:22.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.735+0000 7f6c777fe700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c6406ace0 0x7f6c6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:22.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.736+0000 7f6c777fe700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c6406ace0 0x7f6c6406d190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6c781f6510 tx=0x7f6c6c009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:22.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.740+0000 7f6c74ff9700 1 -- 172.21.15.60:0/1072004348 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6c780c7e20 con 0x7f6c780b8400 2023-07-13T22:29:22.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.952+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c780c7e20 con 0x7f6c780b8400 2023-07-13T22:29:22.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.953+0000 7f6c74ff9700 1 -- 172.21.15.60:0/1072004348 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+18190 (secure 0 0 0) 0x7f6c780c7e20 con 0x7f6c780b8400 2023-07-13T22:29:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:22.933 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":100,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:59.528113+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:56.533994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:57.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c6406ace0 msgr2=0x7f6c6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c6406ace0 0x7f6c6406d190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6c781f6510 tx=0x7f6c6c009500 comp rx=0 tx=0).stop 2023-07-13T22:29:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780b8400 msgr2=0x7f6c78153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780b8400 0x7f6c78153fa0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e870 tx=0x7f6c6800ec30 comp rx=0 tx=0).stop 2023-07-13T22:29:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 shutdown_connections 2023-07-13T22:29:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c6406ace0 0x7f6c6406d190 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c780b8400 0x7f6c78153fa0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c780d3750 0x7f6c7815c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 --2- 172.21.15.60:0/1072004348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c780cfad0 0x7f6c7815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 >> 172.21.15.60:0/1072004348 conn(0x7f6c7806b390 msgr2=0x7f6c780b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.955+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 shutdown_connections 2023-07-13T22:29:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:22.956+0000 7f6c7e7ff700 1 -- 172.21.15.60:0/1072004348 wait complete. 2023-07-13T22:29:22.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:28:56.533994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:28:57.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:29:22.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:29:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.181+0000 7fecd99ff700 1 Processor -- start 2023-07-13T22:29:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.181+0000 7fecd99ff700 1 -- start start 2023-07-13T22:29:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.182+0000 7fecd99ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fecd415c530 0x7fecd40b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.182+0000 7fecd99ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fecd40d11c0 0x7fecd40cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.182+0000 7fecd99ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40cefd0 con 0x7fecd40cf9f0 2023-07-13T22:29:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.182+0000 7fecd99ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd40d1780 con 0x7fecd415c530 2023-07-13T22:29:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.182+0000 7fecd99ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd40cd2d0 con 0x7fecd40d11c0 2023-07-13T22:29:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.183+0000 7fecd2ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fecd415c530 0x7fecd40b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.183+0000 7fecd2ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fecd415c530 0x7fecd40b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41882/0 (socket says 172.21.15.60:41882) 2023-07-13T22:29:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.183+0000 7fecd2ffd700 1 -- 172.21.15.60:0/1196449860 learned_addr learned my addr 172.21.15.60:0/1196449860 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.183+0000 7fecd27fc700 1 --2- 172.21.15.60:0/1196449860 >> v2:172.21.15.105:3300/0 conn(0x7fecd40d11c0 0x7fecd40cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.183+0000 7fecd2ffd700 1 -- 172.21.15.60:0/1196449860 >> v2:172.21.15.105:3300/0 conn(0x7fecd40d11c0 msgr2=0x7fecd40cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.183+0000 7fecd2ffd700 1 --2- 172.21.15.60:0/1196449860 >> v2:172.21.15.105:3300/0 conn(0x7fecd40d11c0 0x7fecd40cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.183+0000 7fecd2ffd700 1 -- 172.21.15.60:0/1196449860 >> v1:172.21.15.105:6789/0 conn(0x7fecd40cf9f0 legacy=0x7fecd40d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.183+0000 7fecd2ffd700 1 -- 172.21.15.60:0/1196449860 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40cd5b0 con 0x7fecd415c530 2023-07-13T22:29:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd2ffd700 1 --2- 172.21.15.60:0/1196449860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd415c530 0x7fecd40b5a10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fecd406c680 tx=0x7fecc4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07c0861dc280915 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd1ffb700 1 -- 172.21.15.60:0/1196449860 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4011830 con 0x7fecd415c530 2023-07-13T22:29:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd1ffb700 1 -- 172.21.15.60:0/1196449860 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc4011990 con 0x7fecd415c530 2023-07-13T22:29:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd1ffb700 1 -- 172.21.15.60:0/1196449860 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4011d20 con 0x7fecd415c530 2023-07-13T22:29:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd99ff700 1 -- 172.21.15.60:0/1196449860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd415c530 msgr2=0x7fecd40b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd99ff700 1 --2- 172.21.15.60:0/1196449860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd415c530 0x7fecd40b5a10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fecd406c680 tx=0x7fecc4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd99ff700 1 -- 172.21.15.60:0/1196449860 shutdown_connections 2023-07-13T22:29:23.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd99ff700 1 --2- 172.21.15.60:0/1196449860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd415c530 0x7fecd40b5a10 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd99ff700 1 --2- 172.21.15.60:0/1196449860 >> v2:172.21.15.105:3300/0 conn(0x7fecd40d11c0 0x7fecd40cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.184+0000 7fecd99ff700 1 -- 172.21.15.60:0/1196449860 >> 172.21.15.60:0/1196449860 conn(0x7fecd406b400 msgr2=0x7fecd406b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:23.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.185+0000 7fecd99ff700 1 -- 172.21.15.60:0/1196449860 shutdown_connections 2023-07-13T22:29:23.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.185+0000 7fecd99ff700 1 -- 172.21.15.60:0/1196449860 wait complete. 2023-07-13T22:29:23.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.186+0000 7fecd99ff700 1 Processor -- start 2023-07-13T22:29:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.186+0000 7fecd99ff700 1 -- start start 2023-07-13T22:29:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.187+0000 7fecd99ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd40cf9f0 0x7fecd41490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.187+0000 7fecd99ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd40d11c0 0x7fecd415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.187+0000 7fecd99ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd4155f20 0x7fecd4159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.187+0000 7fecd99ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd415b430 con 0x7fecd40cf9f0 2023-07-13T22:29:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.187+0000 7fecd99ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd41e6d20 con 0x7fecd40d11c0 2023-07-13T22:29:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.187+0000 7fecd99ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd415bd90 con 0x7fecd4155f20 2023-07-13T22:29:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd40cf9f0 0x7fecd41490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd40cf9f0 0x7fecd41490e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41896/0 (socket says 172.21.15.60:41896) 2023-07-13T22:29:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd2ffd700 1 -- 172.21.15.60:0/615692706 learned_addr learned my addr 172.21.15.60:0/615692706 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd27fc700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd40d11c0 0x7fecd415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd37fe700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd4155f20 0x7fecd4159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd2ffd700 1 -- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd4155f20 msgr2=0x7fecd4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd2ffd700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd4155f20 0x7fecd4159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd2ffd700 1 -- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd40d11c0 msgr2=0x7fecd415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd2ffd700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd40d11c0 0x7fecd415b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd2ffd700 1 -- 172.21.15.60:0/615692706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4009550 con 0x7fecd40cf9f0 2023-07-13T22:29:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd37fe700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd4155f20 0x7fecd4159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:23.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd27fc700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd40d11c0 0x7fecd415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:23.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.188+0000 7fecd2ffd700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd40cf9f0 0x7fecd41490e0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fecc4014040 tx=0x7fecc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:23.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.189+0000 7fecbbfff700 1 -- 172.21.15.60:0/615692706 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc40036a0 con 0x7fecd40cf9f0 2023-07-13T22:29:23.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.189+0000 7fecbbfff700 1 -- 172.21.15.60:0/615692706 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc401e070 con 0x7fecd40cf9f0 2023-07-13T22:29:23.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.189+0000 7fecbbfff700 1 -- 172.21.15.60:0/615692706 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4003a60 con 0x7fecd40cf9f0 2023-07-13T22:29:23.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.189+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd415a7b0 con 0x7fecd40cf9f0 2023-07-13T22:29:23.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.189+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd415ad70 con 0x7fecd40cf9f0 2023-07-13T22:29:23.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.190+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd415b570 con 0x7fecd40cf9f0 2023-07-13T22:29:23.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.191+0000 7fecbbfff700 1 -- 172.21.15.60:0/615692706 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fecc4003bc0 con 0x7fecd40cf9f0 2023-07-13T22:29:23.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.192+0000 7fecbbfff700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecc006ace0 0x7fecc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.192+0000 7fecbbfff700 1 -- 172.21.15.60:0/615692706 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fecc408a3b0 con 0x7fecd40cf9f0 2023-07-13T22:29:23.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.192+0000 7fecd27fc700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecc006ace0 0x7fecc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.193+0000 7fecd27fc700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecc006ace0 0x7fecc006d190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fecd4159960 tx=0x7fecc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:23.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.196+0000 7fecbbfff700 1 -- 172.21.15.60:0/615692706 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fecd415b570 con 0x7fecd40cf9f0 2023-07-13T22:29:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.402+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fecd415b570 con 0x7fecd40cf9f0 2023-07-13T22:29:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.402+0000 7fecbbfff700 1 -- 172.21.15.60:0/615692706 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v100) v1 ==== 92+0+10 (secure 0 0 0) 0x7fecd415b570 con 0x7fecd40cf9f0 2023-07-13T22:29:23.382 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:29:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecc006ace0 msgr2=0x7fecc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecc006ace0 0x7fecc006d190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fecd4159960 tx=0x7fecc8008040 comp rx=0 tx=0).stop 2023-07-13T22:29:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd40cf9f0 msgr2=0x7fecd41490e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd40cf9f0 0x7fecd41490e0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fecc4014040 tx=0x7fecc4004750 comp rx=0 tx=0).stop 2023-07-13T22:29:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 shutdown_connections 2023-07-13T22:29:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fecc006ace0 0x7fecc006d190 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecd40cf9f0 0x7fecd41490e0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd4155f20 0x7fecd4159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 --2- 172.21.15.60:0/615692706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd40d11c0 0x7fecd415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 >> 172.21.15.60:0/615692706 conn(0x7fecd406b400 msgr2=0x7fecd40b8110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 shutdown_connections 2023-07-13T22:29:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.405+0000 7fecd99ff700 1 -- 172.21.15.60:0/615692706 wait complete. 2023-07-13T22:29:23.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:29:23.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.631+0000 7fa6587ff700 1 Processor -- start 2023-07-13T22:29:23.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.631+0000 7fa6587ff700 1 -- start start 2023-07-13T22:29:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.632+0000 7fa6587ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6500b1ac0 0x7fa6500d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.632+0000 7fa6587ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6500d1870 con 0x7fa6500cfd50 2023-07-13T22:29:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.632+0000 7fa6587ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6500cf370 con 0x7fa6500b8200 2023-07-13T22:29:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.632+0000 7fa6587ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6500d12b0 con 0x7fa6500b1ac0 2023-07-13T22:29:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.633+0000 7fa655d9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa6500cfd50 0x7fa6500d23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56106/0 (socket says 172.21.15.60:56106) 2023-07-13T22:29:23.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.633+0000 7fa655d9a700 1 -- 172.21.15.60:0/4217911007 learned_addr learned my addr 172.21.15.60:0/4217911007 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:23.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.633+0000 7fa656d9c700 1 --2- 172.21.15.60:0/4217911007 >> v2:172.21.15.105:3300/0 conn(0x7fa6500b1ac0 0x7fa6500d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.633+0000 7fa655599700 1 -- 172.21.15.60:0/4217911007 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262802361 0 0) 0x7fa6500d1870 con 0x7fa6500cfd50 2023-07-13T22:29:23.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.633+0000 7fa655599700 1 -- 172.21.15.60:0/4217911007 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa63c0035b0 con 0x7fa6500cfd50 2023-07-13T22:29:23.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.633+0000 7fa656d9c700 1 -- 172.21.15.60:0/4217911007 >> v1:172.21.15.105:6789/0 conn(0x7fa6500b8200 legacy=0x7fa6500cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.633+0000 7fa655599700 1 -- 172.21.15.60:0/4217911007 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 678415867 0 0) 0x7fa63c0035b0 con 0x7fa6500cfd50 2023-07-13T22:29:23.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.633+0000 7fa656d9c700 1 -- 172.21.15.60:0/4217911007 >> v1:172.21.15.60:6789/0 conn(0x7fa6500cfd50 legacy=0x7fa6500d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.633+0000 7fa656d9c700 1 -- 172.21.15.60:0/4217911007 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500d1590 con 0x7fa6500b1ac0 2023-07-13T22:29:23.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.634+0000 7fa656d9c700 1 --2- 172.21.15.60:0/4217911007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6500b1ac0 0x7fa6500d2ad0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa648009d90 tx=0x7fa64800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c9a04bb14f2b649 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:23.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.634+0000 7fa655599700 1 -- 172.21.15.60:0/4217911007 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64800e070 con 0x7fa6500b1ac0 2023-07-13T22:29:23.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.634+0000 7fa655599700 1 -- 172.21.15.60:0/4217911007 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa648003a20 con 0x7fa6500b1ac0 2023-07-13T22:29:23.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.634+0000 7fa655599700 1 -- 172.21.15.60:0/4217911007 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640001670 con 0x7fa6500b1ac0 2023-07-13T22:29:23.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.635+0000 7fa6587ff700 1 -- 172.21.15.60:0/4217911007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6500b1ac0 msgr2=0x7fa6500d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.635+0000 7fa6587ff700 1 --2- 172.21.15.60:0/4217911007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6500b1ac0 0x7fa6500d2ad0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa648009d90 tx=0x7fa64800b330 comp rx=0 tx=0).stop 2023-07-13T22:29:23.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.635+0000 7fa6587ff700 1 -- 172.21.15.60:0/4217911007 shutdown_connections 2023-07-13T22:29:23.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.635+0000 7fa6587ff700 1 --2- 172.21.15.60:0/4217911007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6500b1ac0 0x7fa6500d2ad0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.635+0000 7fa6587ff700 1 -- 172.21.15.60:0/4217911007 >> 172.21.15.60:0/4217911007 conn(0x7fa65006b2f0 msgr2=0x7fa65006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:23.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.635+0000 7fa6587ff700 1 -- 172.21.15.60:0/4217911007 shutdown_connections 2023-07-13T22:29:23.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.635+0000 7fa6587ff700 1 -- 172.21.15.60:0/4217911007 wait complete. 2023-07-13T22:29:23.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.636+0000 7fa6587ff700 1 Processor -- start 2023-07-13T22:29:23.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.637+0000 7fa6587ff700 1 -- start start 2023-07-13T22:29:23.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa6587ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6500b1ac0 0x7fa6500c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa6587ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6500b8200 0x7fa650157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa6587ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6500cfd50 0x7fa6501583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa6587ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500c6760 con 0x7fa6500b8200 2023-07-13T22:29:23.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa6587ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501576c0 con 0x7fa6500b1ac0 2023-07-13T22:29:23.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa6587ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa650158c40 con 0x7fa6500cfd50 2023-07-13T22:29:23.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa655d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6500b8200 0x7fa650157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa655d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6500b8200 0x7fa650157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41902/0 (socket says 172.21.15.60:41902) 2023-07-13T22:29:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa655d9a700 1 -- 172.21.15.60:0/311331010 learned_addr learned my addr 172.21.15.60:0/311331010 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa65659b700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6500b1ac0 0x7fa6500c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa656d9c700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6500cfd50 0x7fa6501583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa655d9a700 1 -- 172.21.15.60:0/311331010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6500cfd50 msgr2=0x7fa6501583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa655d9a700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6500cfd50 0x7fa6501583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa655d9a700 1 -- 172.21.15.60:0/311331010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6500b1ac0 msgr2=0x7fa6500c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa655d9a700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6500b1ac0 0x7fa6500c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.638+0000 7fa655d9a700 1 -- 172.21.15.60:0/311331010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648009a70 con 0x7fa6500b8200 2023-07-13T22:29:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.639+0000 7fa655d9a700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6500b8200 0x7fa650157180 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fa64000c750 tx=0x7fa64000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.639+0000 7fa646ffd700 1 -- 172.21.15.60:0/311331010 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640005d70 con 0x7fa6500b8200 2023-07-13T22:29:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.639+0000 7fa646ffd700 1 -- 172.21.15.60:0/311331010 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640007070 con 0x7fa6500b8200 2023-07-13T22:29:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.639+0000 7fa646ffd700 1 -- 172.21.15.60:0/311331010 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640007400 con 0x7fa6500b8200 2023-07-13T22:29:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.639+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6500c6940 con 0x7fa6500b8200 2023-07-13T22:29:23.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.639+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6500c6f00 con 0x7fa6500b8200 2023-07-13T22:29:23.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.640+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65004e210 con 0x7fa6500b8200 2023-07-13T22:29:23.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.641+0000 7fa646ffd700 1 -- 172.21.15.60:0/311331010 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa640007560 con 0x7fa6500b8200 2023-07-13T22:29:23.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.642+0000 7fa646ffd700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63406ace0 0x7fa63406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:23.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.642+0000 7fa646ffd700 1 -- 172.21.15.60:0/311331010 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa640088fc0 con 0x7fa6500b8200 2023-07-13T22:29:23.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.642+0000 7fa65659b700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63406ace0 0x7fa63406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:23.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.643+0000 7fa65659b700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63406ace0 0x7fa63406d190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa64c001940 tx=0x7fa64c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:23.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.646+0000 7fa646ffd700 1 -- 172.21.15.60:0/311331010 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa640053fd0 con 0x7fa6500b8200 2023-07-13T22:29:23.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.855+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6500b4f70 con 0x7fa6500b8200 2023-07-13T22:29:23.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.856+0000 7fa646ffd700 1 -- 172.21.15.60:0/311331010 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6400575f0 con 0x7fa6500b8200 2023-07-13T22:29:23.835 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:29:23.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63406ace0 msgr2=0x7fa63406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63406ace0 0x7fa63406d190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa64c001940 tx=0x7fa64c009040 comp rx=0 tx=0).stop 2023-07-13T22:29:23.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6500b8200 msgr2=0x7fa650157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:23.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6500b8200 0x7fa650157180 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fa64000c750 tx=0x7fa64000cb10 comp rx=0 tx=0).stop 2023-07-13T22:29:23.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 shutdown_connections 2023-07-13T22:29:23.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63406ace0 0x7fa63406d190 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6500cfd50 0x7fa6501583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6500b1ac0 0x7fa6500c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 --2- 172.21.15.60:0/311331010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6500b8200 0x7fa650157180 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 >> 172.21.15.60:0/311331010 conn(0x7fa65006b2f0 msgr2=0x7fa6500d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 shutdown_connections 2023-07-13T22:29:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:23.858+0000 7fa6587ff700 1 -- 172.21.15.60:0/311331010 wait complete. 2023-07-13T22:29:23.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:29:24.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.083+0000 7fdd411ff700 1 Processor -- start 2023-07-13T22:29:24.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.083+0000 7fdd411ff700 1 -- start start 2023-07-13T22:29:24.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.084+0000 7fdd411ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd3c0cfd50 0x7fdd3c0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.084+0000 7fdd411ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdd3c0b1ac0 0x7fdd3c0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.084+0000 7fdd411ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd3c0d1870 con 0x7fdd3c0b8200 2023-07-13T22:29:24.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.084+0000 7fdd411ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3c0cf370 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.084+0000 7fdd411ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3c0d12b0 con 0x7fdd3c0b1ac0 2023-07-13T22:29:24.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.084+0000 7fdd3ad9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdd3c0b8200 0x7fdd3c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56114/0 (socket says 172.21.15.60:56114) 2023-07-13T22:29:24.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.084+0000 7fdd3ad9d700 1 -- 172.21.15.60:0/3940064543 learned_addr learned my addr 172.21.15.60:0/3940064543 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:24.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.084+0000 7fdd3a59c700 1 --2- 172.21.15.60:0/3940064543 >> v2:172.21.15.60:3300/0 conn(0x7fdd3c0cfd50 0x7fdd3c0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd3b59e700 1 --2- 172.21.15.60:0/3940064543 >> v2:172.21.15.105:3300/0 conn(0x7fdd3c0b1ac0 0x7fdd3c0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd39d9b700 1 -- 172.21.15.60:0/3940064543 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835958706 0 0) 0x7fdd3c0d1870 con 0x7fdd3c0b8200 2023-07-13T22:29:24.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd39d9b700 1 -- 172.21.15.60:0/3940064543 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd240035b0 con 0x7fdd3c0b8200 2023-07-13T22:29:24.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd3a59c700 1 -- 172.21.15.60:0/3940064543 >> v2:172.21.15.105:3300/0 conn(0x7fdd3c0b1ac0 msgr2=0x7fdd3c0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:24.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd3a59c700 1 --2- 172.21.15.60:0/3940064543 >> v2:172.21.15.105:3300/0 conn(0x7fdd3c0b1ac0 0x7fdd3c0d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd3a59c700 1 -- 172.21.15.60:0/3940064543 >> v1:172.21.15.60:6789/0 conn(0x7fdd3c0b8200 legacy=0x7fdd3c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd3a59c700 1 -- 172.21.15.60:0/3940064543 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c0d1590 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd3a59c700 1 --2- 172.21.15.60:0/3940064543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 0x7fdd3c0d23c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0b8d70 tx=0x7fdd30009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3158c0c32e4c3748 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:24.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd39d9b700 1 -- 172.21.15.60:0/3940064543 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30011830 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd39d9b700 1 -- 172.21.15.60:0/3940064543 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd30011990 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.085+0000 7fdd39d9b700 1 -- 172.21.15.60:0/3940064543 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30011d20 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.086+0000 7fdd411ff700 1 -- 172.21.15.60:0/3940064543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 msgr2=0x7fdd3c0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.086+0000 7fdd411ff700 1 --2- 172.21.15.60:0/3940064543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 0x7fdd3c0d23c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0b8d70 tx=0x7fdd30009b20 comp rx=0 tx=0).stop 2023-07-13T22:29:24.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.086+0000 7fdd411ff700 1 -- 172.21.15.60:0/3940064543 shutdown_connections 2023-07-13T22:29:24.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.086+0000 7fdd411ff700 1 --2- 172.21.15.60:0/3940064543 >> v2:172.21.15.105:3300/0 conn(0x7fdd3c0b1ac0 0x7fdd3c0d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.086+0000 7fdd411ff700 1 --2- 172.21.15.60:0/3940064543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 0x7fdd3c0d23c0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.086+0000 7fdd411ff700 1 -- 172.21.15.60:0/3940064543 >> 172.21.15.60:0/3940064543 conn(0x7fdd3c06b2f0 msgr2=0x7fdd3c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:24.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.086+0000 7fdd411ff700 1 -- 172.21.15.60:0/3940064543 shutdown_connections 2023-07-13T22:29:24.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.086+0000 7fdd411ff700 1 -- 172.21.15.60:0/3940064543 wait complete. 2023-07-13T22:29:24.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.088+0000 7fdd411ff700 1 Processor -- start 2023-07-13T22:29:24.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.088+0000 7fdd411ff700 1 -- start start 2023-07-13T22:29:24.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.089+0000 7fdd411ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd3c0b1ac0 0x7fdd3c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.089+0000 7fdd411ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd3c0b8200 0x7fdd3c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.089+0000 7fdd411ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 0x7fdd3c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.089+0000 7fdd411ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c0ca750 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.089+0000 7fdd411ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c0caa30 con 0x7fdd3c0b8200 2023-07-13T22:29:24.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.089+0000 7fdd411ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c0cad10 con 0x7fdd3c0b1ac0 2023-07-13T22:29:24.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd3b59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 0x7fdd3c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd3b59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 0x7fdd3c151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41920/0 (socket says 172.21.15.60:41920) 2023-07-13T22:29:24.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd3b59e700 1 -- 172.21.15.60:0/844081762 learned_addr learned my addr 172.21.15.60:0/844081762 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:24.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd3b59e700 1 -- 172.21.15.60:0/844081762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd3c0b1ac0 msgr2=0x7fdd3c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd3b59e700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd3c0b1ac0 0x7fdd3c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd3b59e700 1 -- 172.21.15.60:0/844081762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd3c0b8200 msgr2=0x7fdd3c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd3b59e700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd3c0b8200 0x7fdd3c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd3b59e700 1 -- 172.21.15.60:0/844081762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd300094a0 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd3b59e700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 0x7fdd3c151a70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdd34007e40 tx=0x7fdd3400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:24.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd237fe700 1 -- 172.21.15.60:0/844081762 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3400eea0 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd237fe700 1 -- 172.21.15.60:0/844081762 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd34014420 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.090+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3c0caff0 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.091+0000 7fdd237fe700 1 -- 172.21.15.60:0/844081762 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340147b0 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.091+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3c0cb7f0 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.092+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3c0cb510 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.093+0000 7fdd237fe700 1 -- 172.21.15.60:0/844081762 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd34014910 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.093+0000 7fdd237fe700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd2806ace0 0x7fdd2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.094+0000 7fdd237fe700 1 -- 172.21.15.60:0/844081762 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd34089cd0 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.094+0000 7fdd3ad9d700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd2806ace0 0x7fdd2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.094+0000 7fdd3ad9d700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd2806ace0 0x7fdd2806d190 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0016b0 tx=0x7fdd2c0079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:24.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.098+0000 7fdd237fe700 1 -- 172.21.15.60:0/844081762 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdd3c0cb510 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.305+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd3c0cb510 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.305+0000 7fdd237fe700 1 -- 172.21.15.60:0/844081762 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd3c0cb510 con 0x7fdd3c0cfd50 2023-07-13T22:29:24.285 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:29:24.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.307+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd2806ace0 msgr2=0x7fdd2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.307+0000 7fdd411ff700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd2806ace0 0x7fdd2806d190 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0016b0 tx=0x7fdd2c0079c0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.307+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 msgr2=0x7fdd3c151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.307+0000 7fdd411ff700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 0x7fdd3c151a70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdd34007e40 tx=0x7fdd3400e3c0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.307+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 shutdown_connections 2023-07-13T22:29:24.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.307+0000 7fdd411ff700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd2806ace0 0x7fdd2806d190 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.307+0000 7fdd411ff700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3c0cfd50 0x7fdd3c151a70 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.307+0000 7fdd411ff700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd3c0b1ac0 0x7fdd3c0cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.308+0000 7fdd411ff700 1 --2- 172.21.15.60:0/844081762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd3c0b8200 0x7fdd3c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.308+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 >> 172.21.15.60:0/844081762 conn(0x7fdd3c06b2f0 msgr2=0x7fdd3c0d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.308+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 shutdown_connections 2023-07-13T22:29:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.308+0000 7fdd411ff700 1 -- 172.21.15.60:0/844081762 wait complete. 2023-07-13T22:29:24.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:29:24.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.532+0000 7f33eb3ff700 1 Processor -- start 2023-07-13T22:29:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.532+0000 7f33eb3ff700 1 -- start start 2023-07-13T22:29:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.533+0000 7f33eb3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f33e415c790 0x7f33e40ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.533+0000 7f33eb3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e40cf6d0 con 0x7f33e40ac490 2023-07-13T22:29:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.533+0000 7f33eb3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e40ac900 con 0x7f33e40ad9a0 2023-07-13T22:29:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.533+0000 7f33eb3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e40acbe0 con 0x7f33e415c790 2023-07-13T22:29:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33e899a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f33e40ac490 0x7f33e40c9e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56126/0 (socket says 172.21.15.60:56126) 2023-07-13T22:29:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33e899a700 1 -- 172.21.15.60:0/415731328 learned_addr learned my addr 172.21.15.60:0/415731328 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33e919b700 1 --2- 172.21.15.60:0/415731328 >> v2:172.21.15.105:3300/0 conn(0x7f33e415c790 0x7f33e40ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773219683 0 0) 0x7f33e40cf6d0 con 0x7f33e40ac490 2023-07-13T22:29:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d00035b0 con 0x7f33e40ac490 2023-07-13T22:29:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3151274966 0 0) 0x7f33d00035b0 con 0x7f33e40ac490 2023-07-13T22:29:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 >> v2:172.21.15.105:3300/0 conn(0x7f33e415c790 msgr2=0x7f33e40ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33dbfff700 1 --2- 172.21.15.60:0/415731328 >> v2:172.21.15.105:3300/0 conn(0x7f33e415c790 0x7f33e40ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 >> v1:172.21.15.105:6789/0 conn(0x7f33e40ad9a0 legacy=0x7f33e40adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40acec0 con 0x7f33e40ac490 2023-07-13T22:29:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.534+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f33d4001670 con 0x7f33e40ac490 2023-07-13T22:29:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 >> v1:172.21.15.60:6789/0 conn(0x7f33e40ac490 legacy=0x7f33e40c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33dbfff700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d0003a50 0x7f33d0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33dbfff700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33d0004850 0x7f33d0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33dbfff700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33d000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40acbe0 con 0x7f33d0005200 2023-07-13T22:29:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d000c0c0 con 0x7f33d0004850 2023-07-13T22:29:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d000c280 con 0x7f33d0003a50 2023-07-13T22:29:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33e999c700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33d000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33e899a700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33d0004850 0x7f33d0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33e919b700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d0003a50 0x7f33d0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33e999c700 1 -- 172.21.15.60:0/415731328 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d0003a50 msgr2=0x7f33d0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33e999c700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d0003a50 0x7f33d0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33e999c700 1 -- 172.21.15.60:0/415731328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33d0004850 msgr2=0x7f33d0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33e999c700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33d0004850 0x7f33d0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.535+0000 7f33e999c700 1 -- 172.21.15.60:0/415731328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d00035b0 con 0x7f33d0005200 2023-07-13T22:29:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33e999c700 1 -- 172.21.15.60:0/415731328 reap_dead start 2023-07-13T22:29:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33e999c700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33d000ba10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00c190 tx=0x7f33d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d5bae1026aeb34 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:24.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc001db0 con 0x7f33d0005200 2023-07-13T22:29:24.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc00ccf0 con 0x7f33d0005200 2023-07-13T22:29:24.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33dbfff700 1 -- 172.21.15.60:0/415731328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc003c40 con 0x7f33d0005200 2023-07-13T22:29:24.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33eb3ff700 1 -- 172.21.15.60:0/415731328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 msgr2=0x7f33d000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33eb3ff700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33d000ba10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00c190 tx=0x7f33d0005880 comp rx=0 tx=0).stop 2023-07-13T22:29:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33eb3ff700 1 -- 172.21.15.60:0/415731328 shutdown_connections 2023-07-13T22:29:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33eb3ff700 1 --2- 172.21.15.60:0/415731328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33d000ba10 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.536+0000 7f33eb3ff700 1 -- 172.21.15.60:0/415731328 >> 172.21.15.60:0/415731328 conn(0x7f33e406b900 msgr2=0x7f33e40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.537+0000 7f33eb3ff700 1 -- 172.21.15.60:0/415731328 shutdown_connections 2023-07-13T22:29:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.537+0000 7f33eb3ff700 1 -- 172.21.15.60:0/415731328 wait complete. 2023-07-13T22:29:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.538+0000 7f33eb3ff700 1 Processor -- start 2023-07-13T22:29:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.538+0000 7f33eb3ff700 1 -- start start 2023-07-13T22:29:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.539+0000 7f33eb3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33e4158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.539+0000 7f33eb3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e4156120 0x7f33e4154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.539+0000 7f33eb3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33e4159af0 0x7f33e4159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.539+0000 7f33eb3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e415a4a0 con 0x7f33d0005200 2023-07-13T22:29:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.539+0000 7f33eb3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e409a6c0 con 0x7f33e4156120 2023-07-13T22:29:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.539+0000 7f33eb3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e41568c0 con 0x7f33e4159af0 2023-07-13T22:29:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33e4158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33e4158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41936/0 (socket says 172.21.15.60:41936) 2023-07-13T22:29:24.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e919b700 1 -- 172.21.15.60:0/1087436558 learned_addr learned my addr 172.21.15.60:0/1087436558 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:24.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e999c700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33e4159af0 0x7f33e4159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e899a700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e4156120 0x7f33e4154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e919b700 1 -- 172.21.15.60:0/1087436558 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33e4159af0 msgr2=0x7f33e4159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e919b700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33e4159af0 0x7f33e4159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e919b700 1 -- 172.21.15.60:0/1087436558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e4156120 msgr2=0x7f33e4154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e919b700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e4156120 0x7f33e4154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e919b700 1 -- 172.21.15.60:0/1087436558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e4156ba0 con 0x7f33d0005200 2023-07-13T22:29:24.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.540+0000 7f33e919b700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33e4158830 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009fd0 tx=0x7f33e0011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:24.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.541+0000 7f33d9ffb700 1 -- 172.21.15.60:0/1087436558 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0005a20 con 0x7f33d0005200 2023-07-13T22:29:24.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.541+0000 7f33d9ffb700 1 -- 172.21.15.60:0/1087436558 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0005b80 con 0x7f33d0005200 2023-07-13T22:29:24.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.541+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e4156e80 con 0x7f33d0005200 2023-07-13T22:29:24.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.541+0000 7f33d9ffb700 1 -- 172.21.15.60:0/1087436558 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e000f730 con 0x7f33d0005200 2023-07-13T22:29:24.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.541+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e4156590 con 0x7f33d0005200 2023-07-13T22:29:24.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.542+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e409a490 con 0x7f33d0005200 2023-07-13T22:29:24.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.543+0000 7f33d9ffb700 1 -- 172.21.15.60:0/1087436558 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f33e000f890 con 0x7f33d0005200 2023-07-13T22:29:24.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.544+0000 7f33d9ffb700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33c806ace0 0x7f33c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.544+0000 7f33d9ffb700 1 -- 172.21.15.60:0/1087436558 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33e0089b70 con 0x7f33d0005200 2023-07-13T22:29:24.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.544+0000 7f33e899a700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33c806ace0 0x7f33c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.545+0000 7f33e899a700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33c806ace0 0x7f33c806d190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:24.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.548+0000 7f33d9ffb700 1 -- 172.21.15.60:0/1087436558 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f33e409a490 con 0x7f33d0005200 2023-07-13T22:29:24.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.755+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f33e409a490 con 0x7f33d0005200 2023-07-13T22:29:24.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.755+0000 7f33d9ffb700 1 -- 172.21.15.60:0/1087436558 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f33e409a490 con 0x7f33d0005200 2023-07-13T22:29:24.735 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:29:24.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33c806ace0 msgr2=0x7f33c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33c806ace0 0x7f33c806d190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4000ed0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 msgr2=0x7f33e4158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33e4158830 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009fd0 tx=0x7f33e0011c80 comp rx=0 tx=0).stop 2023-07-13T22:29:24.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 shutdown_connections 2023-07-13T22:29:24.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33c806ace0 0x7f33c806d190 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33d0005200 0x7f33e4158830 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33e4159af0 0x7f33e4159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 --2- 172.21.15.60:0/1087436558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e4156120 0x7f33e4154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 >> 172.21.15.60:0/1087436558 conn(0x7f33e406b900 msgr2=0x7f33e40cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:24.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 shutdown_connections 2023-07-13T22:29:24.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.758+0000 7f33eb3ff700 1 -- 172.21.15.60:0/1087436558 wait complete. 2023-07-13T22:29:24.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:29:24.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.982+0000 7f6e10dff700 1 Processor -- start 2023-07-13T22:29:24.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.982+0000 7f6e10dff700 1 -- start start 2023-07-13T22:29:24.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.983+0000 7f6e10dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 0x7f6e0c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.983+0000 7f6e10dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c070270 con 0x7f6e0c0c9450 2023-07-13T22:29:24.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.984+0000 7f6e10dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c15b2e0 con 0x7f6e0c15a820 2023-07-13T22:29:24.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.984+0000 7f6e10dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c15b5c0 con 0x7f6e0c154950 2023-07-13T22:29:24.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.984+0000 7f6e0ad9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6e0c0c9450 0x7f6e0c0c3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49770/0 (socket says 172.21.15.60:49770) 2023-07-13T22:29:24.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.984+0000 7f6e0ad9d700 1 -- 172.21.15.60:0/3447778759 learned_addr learned my addr 172.21.15.60:0/3447778759 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:24.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.984+0000 7f6e0a59c700 1 --2- 172.21.15.60:0/3447778759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 0x7f6e0c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.984+0000 7f6e0959a700 1 -- 172.21.15.60:0/3447778759 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944269333 0 0) 0x7f6e0c070270 con 0x7f6e0c0c9450 2023-07-13T22:29:24.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.984+0000 7f6e0959a700 1 -- 172.21.15.60:0/3447778759 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6df40035b0 con 0x7f6e0c0c9450 2023-07-13T22:29:24.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.984+0000 7f6e0959a700 1 -- 172.21.15.60:0/3447778759 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233167698 0 0) 0x7f6e0c15b2e0 con 0x7f6e0c15a820 2023-07-13T22:29:24.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.984+0000 7f6e0959a700 1 -- 172.21.15.60:0/3447778759 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e0c070270 con 0x7f6e0c15a820 2023-07-13T22:29:24.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.985+0000 7f6e0a59c700 1 -- 172.21.15.60:0/3447778759 >> v1:172.21.15.105:6789/0 conn(0x7f6e0c15a820 legacy=0x7f6e0c0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.985+0000 7f6e0a59c700 1 -- 172.21.15.60:0/3447778759 >> v1:172.21.15.60:6789/0 conn(0x7f6e0c0c9450 legacy=0x7f6e0c0c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.985+0000 7f6e0a59c700 1 -- 172.21.15.60:0/3447778759 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0704c0 con 0x7f6e0c154950 2023-07-13T22:29:24.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.985+0000 7f6e0a59c700 1 --2- 172.21.15.60:0/3447778759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 0x7f6e0c070cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009870 tx=0x7f6dfc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3184495381c3a2ba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:24.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.985+0000 7f6e0959a700 1 -- 172.21.15.60:0/3447778759 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc011830 con 0x7f6e0c154950 2023-07-13T22:29:24.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.985+0000 7f6e0959a700 1 -- 172.21.15.60:0/3447778759 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc011990 con 0x7f6e0c154950 2023-07-13T22:29:24.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.985+0000 7f6e0959a700 1 -- 172.21.15.60:0/3447778759 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc011d20 con 0x7f6e0c154950 2023-07-13T22:29:24.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.986+0000 7f6e10dff700 1 -- 172.21.15.60:0/3447778759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 msgr2=0x7f6e0c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.986+0000 7f6e10dff700 1 --2- 172.21.15.60:0/3447778759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 0x7f6e0c070cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009870 tx=0x7f6dfc009b80 comp rx=0 tx=0).stop 2023-07-13T22:29:24.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.986+0000 7f6e10dff700 1 -- 172.21.15.60:0/3447778759 shutdown_connections 2023-07-13T22:29:24.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.986+0000 7f6e10dff700 1 --2- 172.21.15.60:0/3447778759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 0x7f6e0c070cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.986+0000 7f6e10dff700 1 -- 172.21.15.60:0/3447778759 >> 172.21.15.60:0/3447778759 conn(0x7f6e0c0cf410 msgr2=0x7f6e0c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:24.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.986+0000 7f6e10dff700 1 -- 172.21.15.60:0/3447778759 shutdown_connections 2023-07-13T22:29:24.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.986+0000 7f6e10dff700 1 -- 172.21.15.60:0/3447778759 wait complete. 2023-07-13T22:29:24.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.988+0000 7f6e10dff700 1 Processor -- start 2023-07-13T22:29:24.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.988+0000 7f6e10dff700 1 -- start start 2023-07-13T22:29:24.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e10dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e0c0c9450 0x7f6e0c13f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e10dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 0x7f6e0c0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e10dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e0c15a820 0x7f6e0c0b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e10dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c13fc00 con 0x7f6e0c15a820 2023-07-13T22:29:24.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e10dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c140430 con 0x7f6e0c0c9450 2023-07-13T22:29:24.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e10dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c142ad0 con 0x7f6e0c154950 2023-07-13T22:29:24.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e0ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e0c15a820 0x7f6e0c0b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e0ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e0c15a820 0x7f6e0c0b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32824/0 (socket says 172.21.15.60:32824) 2023-07-13T22:29:24.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e0ad9d700 1 -- 172.21.15.60:0/3687888678 learned_addr learned my addr 172.21.15.60:0/3687888678 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:24.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e0a59c700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e0c0c9450 0x7f6e0c13f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e09d9b700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 0x7f6e0c0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e0ad9d700 1 -- 172.21.15.60:0/3687888678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 msgr2=0x7f6e0c0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e0ad9d700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 0x7f6e0c0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e0ad9d700 1 -- 172.21.15.60:0/3687888678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e0c0c9450 msgr2=0x7f6e0c13f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:24.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e0ad9d700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e0c0c9450 0x7f6e0c13f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:24.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.989+0000 7f6e0ad9d700 1 -- 172.21.15.60:0/3687888678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0094a0 con 0x7f6e0c15a820 2023-07-13T22:29:24.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.990+0000 7f6e0ad9d700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e0c15a820 0x7f6e0c0b1e70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400cb10 tx=0x7f6e0400ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:24.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.990+0000 7f6dfaffd700 1 -- 172.21.15.60:0/3687888678 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400a9e0 con 0x7f6e0c15a820 2023-07-13T22:29:24.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.990+0000 7f6dfaffd700 1 -- 172.21.15.60:0/3687888678 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0400ab60 con 0x7f6e0c15a820 2023-07-13T22:29:24.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.990+0000 7f6dfaffd700 1 -- 172.21.15.60:0/3687888678 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04012400 con 0x7f6e0c15a820 2023-07-13T22:29:24.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.990+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c0b0ac0 con 0x7f6e0c15a820 2023-07-13T22:29:24.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.990+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c1400c0 con 0x7f6e0c15a820 2023-07-13T22:29:24.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.991+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0c1405e0 con 0x7f6e0c15a820 2023-07-13T22:29:24.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.992+0000 7f6dfaffd700 1 -- 172.21.15.60:0/3687888678 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6e0401b3b0 con 0x7f6e0c15a820 2023-07-13T22:29:24.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.993+0000 7f6dfaffd700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6dec06ace0 0x7f6dec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:24.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.993+0000 7f6dfaffd700 1 -- 172.21.15.60:0/3687888678 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e04089220 con 0x7f6e0c15a820 2023-07-13T22:29:24.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.994+0000 7f6e0a59c700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6dec06ace0 0x7f6dec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:24.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.994+0000 7f6e0a59c700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6dec06ace0 0x7f6dec06d190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0097c0 tx=0x7f6dfc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:24.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:24.998+0000 7f6dfaffd700 1 -- 172.21.15.60:0/3687888678 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6e0c1405e0 con 0x7f6e0c15a820 2023-07-13T22:29:25.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.218+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6e0c1405a0 con 0x7f6e0c15a820 2023-07-13T22:29:25.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.219+0000 7f6dfaffd700 1 -- 172.21.15.60:0/3687888678 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6e040541a0 con 0x7f6e0c15a820 2023-07-13T22:29:25.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6dec06ace0 msgr2=0x7f6dec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:25.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6dec06ace0 0x7f6dec06d190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0097c0 tx=0x7f6dfc01c040 comp rx=0 tx=0).stop 2023-07-13T22:29:25.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e0c15a820 msgr2=0x7f6e0c0b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:25.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e0c15a820 0x7f6e0c0b1e70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400cb10 tx=0x7f6e0400ce20 comp rx=0 tx=0).stop 2023-07-13T22:29:25.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 shutdown_connections 2023-07-13T22:29:25.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6dec06ace0 0x7f6dec06d190 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e0c15a820 0x7f6e0c0b1e70 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e0c0c9450 0x7f6e0c13f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 --2- 172.21.15.60:0/3687888678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e0c154950 0x7f6e0c0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 >> 172.21.15.60:0/3687888678 conn(0x7f6e0c0cf410 msgr2=0x7f6e0c0c3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:25.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 shutdown_connections 2023-07-13T22:29:25.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.221+0000 7f6e10dff700 1 -- 172.21.15.60:0/3687888678 wait complete. 2023-07-13T22:29:25.203 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:29:25.218 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:29:25.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:25.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:25.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[152462]: fuse finished with error 0 and tester_r 0 2023-07-13T22:29:25.291 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:29:25.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:25.292 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:29:25.469 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:29:25.469 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:29:25.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:25.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.742+0000 7f0b071ff700 1 Processor -- start 2023-07-13T22:29:25.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.742+0000 7f0b071ff700 1 -- start start 2023-07-13T22:29:25.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.743+0000 7f0b071ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b00158870 0x7f0b000c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:25.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.743+0000 7f0b071ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b000c52c0 0x7f0b00069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:25.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.743+0000 7f0b071ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0006a830 con 0x7f0b0006a410 2023-07-13T22:29:25.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.743+0000 7f0b071ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b000c3fc0 con 0x7f0b000c52c0 2023-07-13T22:29:25.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.743+0000 7f0b071ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b000c42a0 con 0x7f0b00158870 2023-07-13T22:29:25.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0b04f9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0b00158870 0x7f0b000c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:25.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0b0579c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0b0006a410 0x7f0b000c9300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49774/0 (socket says 172.21.15.60:49774) 2023-07-13T22:29:25.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0b0579c700 1 -- 172.21.15.60:0/2806861283 learned_addr learned my addr 172.21.15.60:0/2806861283 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:25.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0afffff700 1 --2- 172.21.15.60:0/2806861283 >> v2:172.21.15.60:3300/0 conn(0x7f0b000c52c0 0x7f0b00069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:25.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0afffff700 1 -- 172.21.15.60:0/2806861283 >> v2:172.21.15.105:3300/0 conn(0x7f0b00158870 msgr2=0x7f0b000c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:25.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0aff7fe700 1 -- 172.21.15.60:0/2806861283 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673972571 0 0) 0x7f0b0006a830 con 0x7f0b0006a410 2023-07-13T22:29:25.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0afffff700 1 --2- 172.21.15.60:0/2806861283 >> v2:172.21.15.105:3300/0 conn(0x7f0b00158870 0x7f0b000c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0afffff700 1 -- 172.21.15.60:0/2806861283 >> v1:172.21.15.60:6789/0 conn(0x7f0b0006a410 legacy=0x7f0b000c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:25.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0afffff700 1 -- 172.21.15.60:0/2806861283 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b000c4580 con 0x7f0b000c52c0 2023-07-13T22:29:25.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.744+0000 7f0afffff700 1 --2- 172.21.15.60:0/2806861283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 0x7f0b00069ed0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0b0006fe20 tx=0x7f0aec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdda6d17a72df13 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:25.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0aff7fe700 1 -- 172.21.15.60:0/2806861283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec011830 con 0x7f0b000c52c0 2023-07-13T22:29:25.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0aff7fe700 1 -- 172.21.15.60:0/2806861283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec011990 con 0x7f0b000c52c0 2023-07-13T22:29:25.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0aff7fe700 1 -- 172.21.15.60:0/2806861283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec011d20 con 0x7f0b000c52c0 2023-07-13T22:29:25.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0b071ff700 1 -- 172.21.15.60:0/2806861283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 msgr2=0x7f0b00069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0b071ff700 1 --2- 172.21.15.60:0/2806861283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 0x7f0b00069ed0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0b0006fe20 tx=0x7f0aec009b20 comp rx=0 tx=0).stop 2023-07-13T22:29:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0b071ff700 1 -- 172.21.15.60:0/2806861283 shutdown_connections 2023-07-13T22:29:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0b071ff700 1 --2- 172.21.15.60:0/2806861283 >> v2:172.21.15.105:3300/0 conn(0x7f0b00158870 0x7f0b000c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0b071ff700 1 --2- 172.21.15.60:0/2806861283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 0x7f0b00069ed0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0b071ff700 1 -- 172.21.15.60:0/2806861283 >> 172.21.15.60:0/2806861283 conn(0x7f0b000cf410 msgr2=0x7f0b000cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0b071ff700 1 -- 172.21.15.60:0/2806861283 shutdown_connections 2023-07-13T22:29:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.745+0000 7f0b071ff700 1 -- 172.21.15.60:0/2806861283 wait complete. 2023-07-13T22:29:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.747+0000 7f0b071ff700 1 Processor -- start 2023-07-13T22:29:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.747+0000 7f0b071ff700 1 -- start start 2023-07-13T22:29:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.748+0000 7f0b071ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b0006a410 0x7f0b0013e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.748+0000 7f0b071ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 0x7f0b0013db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.748+0000 7f0b071ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b00158870 0x7f0b000cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:25.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.748+0000 7f0b071ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b001424a0 con 0x7f0b000c52c0 2023-07-13T22:29:25.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.748+0000 7f0b071ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0013e8a0 con 0x7f0b00158870 2023-07-13T22:29:25.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.748+0000 7f0b071ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0013eb80 con 0x7f0b0006a410 2023-07-13T22:29:25.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.748+0000 7f0afffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 0x7f0b0013db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:25.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.748+0000 7f0afffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 0x7f0b0013db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32850/0 (socket says 172.21.15.60:32850) 2023-07-13T22:29:25.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.748+0000 7f0afffff700 1 -- 172.21.15.60:0/3407559832 learned_addr learned my addr 172.21.15.60:0/3407559832 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:25.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0afffff700 1 -- 172.21.15.60:0/3407559832 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b0006a410 msgr2=0x7f0b0013e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0afffff700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b0006a410 0x7f0b0013e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0afffff700 1 -- 172.21.15.60:0/3407559832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b00158870 msgr2=0x7f0b000cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0b0579c700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b00158870 0x7f0b000cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0afffff700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b00158870 0x7f0b000cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0afffff700 1 -- 172.21.15.60:0/3407559832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec0094a0 con 0x7f0b000c52c0 2023-07-13T22:29:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0b0579c700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b00158870 0x7f0b000cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0afffff700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 0x7f0b0013db20 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009b60 tx=0x7f0aec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0afd7fa700 1 -- 172.21.15.60:0/3407559832 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0036a0 con 0x7f0b000c52c0 2023-07-13T22:29:25.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0afd7fa700 1 -- 172.21.15.60:0/3407559832 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec01f070 con 0x7f0b000c52c0 2023-07-13T22:29:25.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0afd7fa700 1 -- 172.21.15.60:0/3407559832 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec003a40 con 0x7f0b000c52c0 2023-07-13T22:29:25.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.749+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0013ee60 con 0x7f0b000c52c0 2023-07-13T22:29:25.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.750+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0013f420 con 0x7f0b000c52c0 2023-07-13T22:29:25.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.751+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b000c39f0 con 0x7f0b000c52c0 2023-07-13T22:29:25.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.752+0000 7f0afd7fa700 1 -- 172.21.15.60:0/3407559832 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0aec003ba0 con 0x7f0b000c52c0 2023-07-13T22:29:25.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.753+0000 7f0afd7fa700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ae406ace0 0x7f0ae406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:25.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.753+0000 7f0afd7fa700 1 -- 172.21.15.60:0/3407559832 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0aec021070 con 0x7f0b000c52c0 2023-07-13T22:29:25.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.753+0000 7f0b04f9b700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ae406ace0 0x7f0ae406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:25.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.754+0000 7f0b04f9b700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ae406ace0 0x7f0ae406d190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006b20 tx=0x7f0af4006ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:25.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.757+0000 7f0afd7fa700 1 -- 172.21.15.60:0/3407559832 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0aec055eb0 con 0x7f0b000c52c0 2023-07-13T22:29:25.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.963+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b0013e060 con 0x7f0b000c52c0 2023-07-13T22:29:25.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.965+0000 7f0afd7fa700 1 -- 172.21.15.60:0/3407559832 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+18190 (secure 0 0 0) 0x7f0aec015090 con 0x7f0b000c52c0 2023-07-13T22:29:25.944 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:25.945 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":100,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:59.528113+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:56.533994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:57.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:25.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ae406ace0 msgr2=0x7f0ae406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:25.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ae406ace0 0x7f0ae406d190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006b20 tx=0x7f0af4006ab0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 msgr2=0x7f0b0013db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 0x7f0b0013db20 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009b60 tx=0x7f0aec004750 comp rx=0 tx=0).stop 2023-07-13T22:29:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 shutdown_connections 2023-07-13T22:29:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ae406ace0 0x7f0ae406d190 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b00158870 0x7f0b000cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b0006a410 0x7f0b0013e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 --2- 172.21.15.60:0/3407559832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b000c52c0 0x7f0b0013db20 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 >> 172.21.15.60:0/3407559832 conn(0x7f0b000cf410 msgr2=0x7f0b000c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 shutdown_connections 2023-07-13T22:29:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:25.967+0000 7f0b071ff700 1 -- 172.21.15.60:0/3407559832 wait complete. 2023-07-13T22:29:25.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:28:56.533994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:28:57.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:29:25.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:29:26.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.192+0000 7f76979ff700 1 Processor -- start 2023-07-13T22:29:26.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.192+0000 7f76979ff700 1 -- start start 2023-07-13T22:29:26.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.193+0000 7f76979ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7690159260 0x7f76900ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.193+0000 7f76979ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76900cc410 0x7f76900cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.193+0000 7f76979ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76900cf640 con 0x7f76900703e0 2023-07-13T22:29:26.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.193+0000 7f76979ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76900ccf80 con 0x7f76900cc410 2023-07-13T22:29:26.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.193+0000 7f76979ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76900c9b40 con 0x7f7690159260 2023-07-13T22:29:26.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.194+0000 7f7695f9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f76900703e0 0x7f76900707e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49786/0 (socket says 172.21.15.60:49786) 2023-07-13T22:29:26.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.194+0000 7f7694f9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76900cc410 0x7f76900cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.194+0000 7f7695f9c700 1 -- 172.21.15.60:0/892641512 learned_addr learned my addr 172.21.15.60:0/892641512 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:26.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.194+0000 7f769579b700 1 --2- 172.21.15.60:0/892641512 >> v2:172.21.15.105:3300/0 conn(0x7f7690159260 0x7f76900ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.194+0000 7f7694f9a700 1 -- 172.21.15.60:0/892641512 >> v2:172.21.15.105:3300/0 conn(0x7f7690159260 msgr2=0x7f76900ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:26.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.194+0000 7f7694f9a700 1 --2- 172.21.15.60:0/892641512 >> v2:172.21.15.105:3300/0 conn(0x7f7690159260 0x7f76900ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.194+0000 7f7694f9a700 1 -- 172.21.15.60:0/892641512 >> v1:172.21.15.60:6789/0 conn(0x7f76900703e0 legacy=0x7f76900707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.194+0000 7f7694f9a700 1 -- 172.21.15.60:0/892641512 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76900c9e20 con 0x7f76900cc410 2023-07-13T22:29:26.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.194+0000 7f7687fff700 1 -- 172.21.15.60:0/892641512 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654395216 0 0) 0x7f76900cf640 con 0x7f76900703e0 2023-07-13T22:29:26.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.195+0000 7f7694f9a700 1 --2- 172.21.15.60:0/892641512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 0x7f76900cc8b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7690071110 tx=0x7f7680009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a341ae44cd2c21f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:26.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.195+0000 7f7687fff700 1 -- 172.21.15.60:0/892641512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680011830 con 0x7f76900cc410 2023-07-13T22:29:26.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.195+0000 7f7687fff700 1 -- 172.21.15.60:0/892641512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7680011990 con 0x7f76900cc410 2023-07-13T22:29:26.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.195+0000 7f7687fff700 1 -- 172.21.15.60:0/892641512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680011d20 con 0x7f76900cc410 2023-07-13T22:29:26.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.195+0000 7f76979ff700 1 -- 172.21.15.60:0/892641512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 msgr2=0x7f76900cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.195+0000 7f76979ff700 1 --2- 172.21.15.60:0/892641512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 0x7f76900cc8b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7690071110 tx=0x7f7680009b20 comp rx=0 tx=0).stop 2023-07-13T22:29:26.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.196+0000 7f76979ff700 1 -- 172.21.15.60:0/892641512 shutdown_connections 2023-07-13T22:29:26.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.196+0000 7f76979ff700 1 --2- 172.21.15.60:0/892641512 >> v2:172.21.15.105:3300/0 conn(0x7f7690159260 0x7f76900ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.196+0000 7f76979ff700 1 --2- 172.21.15.60:0/892641512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 0x7f76900cc8b0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.196+0000 7f76979ff700 1 -- 172.21.15.60:0/892641512 >> 172.21.15.60:0/892641512 conn(0x7f76900cbf50 msgr2=0x7f76900ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.196+0000 7f76979ff700 1 -- 172.21.15.60:0/892641512 shutdown_connections 2023-07-13T22:29:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.196+0000 7f76979ff700 1 -- 172.21.15.60:0/892641512 wait complete. 2023-07-13T22:29:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.197+0000 7f76979ff700 1 Processor -- start 2023-07-13T22:29:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.198+0000 7f76979ff700 1 -- start start 2023-07-13T22:29:26.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.198+0000 7f76979ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76900703e0 0x7f7690153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.198+0000 7f76979ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 0x7f76900c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.198+0000 7f76979ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7690159260 0x7f76900c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f76979ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7690152a40 con 0x7f76900cc410 2023-07-13T22:29:26.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f76979ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76900b1280 con 0x7f7690159260 2023-07-13T22:29:26.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f76979ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76900c6390 con 0x7f76900703e0 2023-07-13T22:29:26.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7694f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 0x7f76900c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f769579b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76900703e0 0x7f7690153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7694f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 0x7f76900c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32868/0 (socket says 172.21.15.60:32868) 2023-07-13T22:29:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7694f9a700 1 -- 172.21.15.60:0/3176071878 learned_addr learned my addr 172.21.15.60:0/3176071878 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7695f9c700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7690159260 0x7f76900c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7694f9a700 1 -- 172.21.15.60:0/3176071878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76900703e0 msgr2=0x7f7690153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7694f9a700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76900703e0 0x7f7690153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7694f9a700 1 -- 172.21.15.60:0/3176071878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7690159260 msgr2=0x7f76900c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7694f9a700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7690159260 0x7f76900c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7694f9a700 1 -- 172.21.15.60:0/3176071878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76800094a0 con 0x7f76900cc410 2023-07-13T22:29:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.199+0000 7f7694f9a700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 0x7f76900c5910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7680000c00 tx=0x7f7680004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.200+0000 7f7685ffb700 1 -- 172.21.15.60:0/3176071878 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800045d0 con 0x7f76900cc410 2023-07-13T22:29:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.200+0000 7f7685ffb700 1 -- 172.21.15.60:0/3176071878 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768001f070 con 0x7f76900cc410 2023-07-13T22:29:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.200+0000 7f7685ffb700 1 -- 172.21.15.60:0/3176071878 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800038e0 con 0x7f76900cc410 2023-07-13T22:29:26.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.200+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7690152fa0 con 0x7f76900cc410 2023-07-13T22:29:26.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.200+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76900b0a50 con 0x7f76900cc410 2023-07-13T22:29:26.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.201+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7690153130 con 0x7f76900cc410 2023-07-13T22:29:26.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.202+0000 7f7685ffb700 1 -- 172.21.15.60:0/3176071878 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7680003a40 con 0x7f76900cc410 2023-07-13T22:29:26.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.203+0000 7f7685ffb700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f767406ace0 0x7f767406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.203+0000 7f7685ffb700 1 -- 172.21.15.60:0/3176071878 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7680021070 con 0x7f76900cc410 2023-07-13T22:29:26.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.204+0000 7f769579b700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f767406ace0 0x7f767406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.204+0000 7f769579b700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f767406ace0 0x7f767406d190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f76901f1620 tx=0x7f768c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:26.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.208+0000 7f7685ffb700 1 -- 172.21.15.60:0/3176071878 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7680055e70 con 0x7f76900cc410 2023-07-13T22:29:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.416+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f769004e210 con 0x7f76900cc410 2023-07-13T22:29:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.416+0000 7f7685ffb700 1 -- 172.21.15.60:0/3176071878 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v100) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7680059490 con 0x7f76900cc410 2023-07-13T22:29:26.396 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:29:26.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f767406ace0 msgr2=0x7f767406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f767406ace0 0x7f767406d190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f76901f1620 tx=0x7f768c008040 comp rx=0 tx=0).stop 2023-07-13T22:29:26.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 msgr2=0x7f76900c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 0x7f76900c5910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7680000c00 tx=0x7f7680004750 comp rx=0 tx=0).stop 2023-07-13T22:29:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 shutdown_connections 2023-07-13T22:29:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f767406ace0 0x7f767406d190 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7690159260 0x7f76900c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76900703e0 0x7f7690153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 --2- 172.21.15.60:0/3176071878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76900cc410 0x7f76900c5910 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 >> 172.21.15.60:0/3176071878 conn(0x7f76900cbf50 msgr2=0x7f76900ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 shutdown_connections 2023-07-13T22:29:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.419+0000 7f76979ff700 1 -- 172.21.15.60:0/3176071878 wait complete. 2023-07-13T22:29:26.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:29:26.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.642+0000 7f3df9dff700 1 Processor -- start 2023-07-13T22:29:26.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.643+0000 7f3df9dff700 1 -- start start 2023-07-13T22:29:26.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.644+0000 7f3df9dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df4158930 0x7f3df40c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.644+0000 7f3df9dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3df406a5e0 0x7f3df415c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.644+0000 7f3df9dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df406d030 con 0x7f3df415c4a0 2023-07-13T22:29:26.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.644+0000 7f3df9dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df40c9620 con 0x7f3df406a5e0 2023-07-13T22:29:26.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.644+0000 7f3df9dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df406ca70 con 0x7f3df4158930 2023-07-13T22:29:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3debfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3df415c4a0 0x7f3df4069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49792/0 (socket says 172.21.15.60:49792) 2023-07-13T22:29:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3debfff700 1 -- 172.21.15.60:0/1916858234 learned_addr learned my addr 172.21.15.60:0/1916858234 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3df37fe700 1 --2- 172.21.15.60:0/1916858234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df4158930 0x7f3df40c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3df3fff700 1 --2- 172.21.15.60:0/1916858234 >> v2:172.21.15.105:3300/0 conn(0x7f3df406a5e0 0x7f3df415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3df2ffd700 1 -- 172.21.15.60:0/1916858234 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237600073 0 0) 0x7f3df406d030 con 0x7f3df415c4a0 2023-07-13T22:29:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3df2ffd700 1 -- 172.21.15.60:0/1916858234 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3de00035b0 con 0x7f3df415c4a0 2023-07-13T22:29:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3df3fff700 1 -- 172.21.15.60:0/1916858234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df4158930 msgr2=0x7f3df40c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3df3fff700 1 --2- 172.21.15.60:0/1916858234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df4158930 0x7f3df40c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3df3fff700 1 -- 172.21.15.60:0/1916858234 >> v1:172.21.15.60:6789/0 conn(0x7f3df415c4a0 legacy=0x7f3df4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3df3fff700 1 -- 172.21.15.60:0/1916858234 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40adc20 con 0x7f3df406a5e0 2023-07-13T22:29:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.645+0000 7f3df37fe700 1 --2- 172.21.15.60:0/1916858234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df4158930 0x7f3df40c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.646+0000 7f3df3fff700 1 --2- 172.21.15.60:0/1916858234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df406a5e0 0x7f3df415c910 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009d90 tx=0x7f3dec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeb3c766ff051fc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.646+0000 7f3df2ffd700 1 -- 172.21.15.60:0/1916858234 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec00e070 con 0x7f3df406a5e0 2023-07-13T22:29:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.646+0000 7f3df2ffd700 1 -- 172.21.15.60:0/1916858234 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec00bb20 con 0x7f3df406a5e0 2023-07-13T22:29:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.646+0000 7f3df2ffd700 1 -- 172.21.15.60:0/1916858234 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec00be90 con 0x7f3df406a5e0 2023-07-13T22:29:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.646+0000 7f3df9dff700 1 -- 172.21.15.60:0/1916858234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df406a5e0 msgr2=0x7f3df415c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.646+0000 7f3df9dff700 1 --2- 172.21.15.60:0/1916858234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df406a5e0 0x7f3df415c910 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009d90 tx=0x7f3dec00b330 comp rx=0 tx=0).stop 2023-07-13T22:29:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.646+0000 7f3df9dff700 1 -- 172.21.15.60:0/1916858234 shutdown_connections 2023-07-13T22:29:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.647+0000 7f3df9dff700 1 --2- 172.21.15.60:0/1916858234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df406a5e0 0x7f3df415c910 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.647+0000 7f3df9dff700 1 --2- 172.21.15.60:0/1916858234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df4158930 0x7f3df40c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.647+0000 7f3df9dff700 1 -- 172.21.15.60:0/1916858234 >> 172.21.15.60:0/1916858234 conn(0x7f3df40cbf50 msgr2=0x7f3df40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.647+0000 7f3df9dff700 1 -- 172.21.15.60:0/1916858234 shutdown_connections 2023-07-13T22:29:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.647+0000 7f3df9dff700 1 -- 172.21.15.60:0/1916858234 wait complete. 2023-07-13T22:29:26.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.648+0000 7f3df9dff700 1 Processor -- start 2023-07-13T22:29:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.648+0000 7f3df9dff700 1 -- start start 2023-07-13T22:29:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.649+0000 7f3df9dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df406a5e0 0x7f3df41f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.649+0000 7f3df9dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4158930 0x7f3df41f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.649+0000 7f3df9dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df415c4a0 0x7f3df41408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.649+0000 7f3df9dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40c8560 con 0x7f3df415c4a0 2023-07-13T22:29:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.649+0000 7f3df9dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40c8840 con 0x7f3df4158930 2023-07-13T22:29:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.649+0000 7f3df9dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40c8b20 con 0x7f3df406a5e0 2023-07-13T22:29:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df415c4a0 0x7f3df41408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df37fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df406a5e0 0x7f3df41f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df415c4a0 0x7f3df41408d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32882/0 (socket says 172.21.15.60:32882) 2023-07-13T22:29:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df3fff700 1 -- 172.21.15.60:0/2035211441 learned_addr learned my addr 172.21.15.60:0/2035211441 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3debfff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4158930 0x7f3df41f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df3fff700 1 -- 172.21.15.60:0/2035211441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df406a5e0 msgr2=0x7f3df41f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df3fff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df406a5e0 0x7f3df41f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df3fff700 1 -- 172.21.15.60:0/2035211441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4158930 msgr2=0x7f3df41f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df3fff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4158930 0x7f3df41f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df3fff700 1 -- 172.21.15.60:0/2035211441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40094a0 con 0x7f3df415c4a0 2023-07-13T22:29:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.650+0000 7f3df3fff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df415c4a0 0x7f3df41408d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f3dec014040 tx=0x7f3dec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.651+0000 7f3df0ff9700 1 -- 172.21.15.60:0/2035211441 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec004840 con 0x7f3df415c4a0 2023-07-13T22:29:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.651+0000 7f3df0ff9700 1 -- 172.21.15.60:0/2035211441 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec020070 con 0x7f3df415c4a0 2023-07-13T22:29:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.651+0000 7f3df0ff9700 1 -- 172.21.15.60:0/2035211441 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0276c0 con 0x7f3df415c4a0 2023-07-13T22:29:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.651+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dec009a70 con 0x7f3df415c4a0 2023-07-13T22:29:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.651+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df40af3a0 con 0x7f3df415c4a0 2023-07-13T22:29:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.652+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df404e210 con 0x7f3df415c4a0 2023-07-13T22:29:26.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.653+0000 7f3df0ff9700 1 -- 172.21.15.60:0/2035211441 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3dec012430 con 0x7f3df415c4a0 2023-07-13T22:29:26.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.654+0000 7f3df0ff9700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3dd406ace0 0x7f3dd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:26.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.654+0000 7f3df0ff9700 1 -- 172.21.15.60:0/2035211441 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3dec022070 con 0x7f3df415c4a0 2023-07-13T22:29:26.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.654+0000 7f3df37fe700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3dd406ace0 0x7f3dd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:26.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.654+0000 7f3df37fe700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3dd406ace0 0x7f3dd406d190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009cd0 tx=0x7f3de4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.658+0000 7f3df0ff9700 1 -- 172.21.15.60:0/2035211441 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3dec056100 con 0x7f3df415c4a0 2023-07-13T22:29:26.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.866+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3df4153c50 con 0x7f3df415c4a0 2023-07-13T22:29:26.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.866+0000 7f3df0ff9700 1 -- 172.21.15.60:0/2035211441 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3dec059720 con 0x7f3df415c4a0 2023-07-13T22:29:26.846 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:29:26.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3dd406ace0 msgr2=0x7f3dd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3dd406ace0 0x7f3dd406d190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009cd0 tx=0x7f3de4009210 comp rx=0 tx=0).stop 2023-07-13T22:29:26.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df415c4a0 msgr2=0x7f3df41408d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:26.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df415c4a0 0x7f3df41408d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f3dec014040 tx=0x7f3dec004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 shutdown_connections 2023-07-13T22:29:26.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3dd406ace0 0x7f3dd406d190 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df415c4a0 0x7f3df41408d0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3df406a5e0 0x7f3df41f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 --2- 172.21.15.60:0/2035211441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4158930 0x7f3df41f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:26.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.868+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 >> 172.21.15.60:0/2035211441 conn(0x7f3df40cbf50 msgr2=0x7f3df4069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:26.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.869+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 shutdown_connections 2023-07-13T22:29:26.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:26.869+0000 7f3df9dff700 1 -- 172.21.15.60:0/2035211441 wait complete. 2023-07-13T22:29:26.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:29:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.092+0000 7f3a483ff700 1 Processor -- start 2023-07-13T22:29:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.092+0000 7f3a483ff700 1 -- start start 2023-07-13T22:29:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.093+0000 7f3a483ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a400b8200 0x7f3a400cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a483ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 0x7f3a400cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a483ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a400d1870 con 0x7f3a400cede0 2023-07-13T22:29:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a483ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a400cf370 con 0x7f3a400b8200 2023-07-13T22:29:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a483ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400d12b0 con 0x7f3a400b1ac0 2023-07-13T22:29:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a4619b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a400b8200 0x7f3a400cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a4699c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3a400cede0 0x7f3a4015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49806/0 (socket says 172.21.15.60:49806) 2023-07-13T22:29:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a4699c700 1 -- 172.21.15.60:0/2336640312 learned_addr learned my addr 172.21.15.60:0/2336640312 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a4599a700 1 --2- 172.21.15.60:0/2336640312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 0x7f3a400cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a4619b700 1 -- 172.21.15.60:0/2336640312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 msgr2=0x7f3a400cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a4619b700 1 --2- 172.21.15.60:0/2336640312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 0x7f3a400cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a4619b700 1 -- 172.21.15.60:0/2336640312 >> v1:172.21.15.60:6789/0 conn(0x7f3a400cede0 legacy=0x7f3a4015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a4619b700 1 -- 172.21.15.60:0/2336640312 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a400d1590 con 0x7f3a400b8200 2023-07-13T22:29:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.094+0000 7f3a45199700 1 -- 172.21.15.60:0/2336640312 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338740169 0 0) 0x7f3a400d1870 con 0x7f3a400cede0 2023-07-13T22:29:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.095+0000 7f3a4619b700 1 --2- 172.21.15.60:0/2336640312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 0x7f3a400cf640 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009840 tx=0x7f3a3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303c807c673d8e44 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.095+0000 7f3a45199700 1 -- 172.21.15.60:0/2336640312 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c011830 con 0x7f3a400b8200 2023-07-13T22:29:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.095+0000 7f3a45199700 1 -- 172.21.15.60:0/2336640312 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c011990 con 0x7f3a400b8200 2023-07-13T22:29:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.095+0000 7f3a45199700 1 -- 172.21.15.60:0/2336640312 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c011d20 con 0x7f3a400b8200 2023-07-13T22:29:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.095+0000 7f3a483ff700 1 -- 172.21.15.60:0/2336640312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 msgr2=0x7f3a400cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.095+0000 7f3a483ff700 1 --2- 172.21.15.60:0/2336640312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 0x7f3a400cf640 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009840 tx=0x7f3a3c009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.096+0000 7f3a483ff700 1 -- 172.21.15.60:0/2336640312 shutdown_connections 2023-07-13T22:29:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.096+0000 7f3a483ff700 1 --2- 172.21.15.60:0/2336640312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 0x7f3a400cf640 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.096+0000 7f3a483ff700 1 --2- 172.21.15.60:0/2336640312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 0x7f3a400cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.096+0000 7f3a483ff700 1 -- 172.21.15.60:0/2336640312 >> 172.21.15.60:0/2336640312 conn(0x7f3a4006b2f0 msgr2=0x7f3a4006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.096+0000 7f3a483ff700 1 -- 172.21.15.60:0/2336640312 shutdown_connections 2023-07-13T22:29:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.096+0000 7f3a483ff700 1 -- 172.21.15.60:0/2336640312 wait complete. 2023-07-13T22:29:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.097+0000 7f3a483ff700 1 Processor -- start 2023-07-13T22:29:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.097+0000 7f3a483ff700 1 -- start start 2023-07-13T22:29:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.098+0000 7f3a483ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 0x7f3a401e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.098+0000 7f3a483ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 0x7f3a400c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.098+0000 7f3a483ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a400cede0 0x7f3a4015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.098+0000 7f3a483ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a401e3b20 con 0x7f3a400b8200 2023-07-13T22:29:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.098+0000 7f3a483ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a401e3e00 con 0x7f3a400cede0 2023-07-13T22:29:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.098+0000 7f3a483ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a401e40e0 con 0x7f3a400b1ac0 2023-07-13T22:29:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 0x7f3a400c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4619b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 0x7f3a401e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4599a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 0x7f3a400c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32902/0 (socket says 172.21.15.60:32902) 2023-07-13T22:29:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4599a700 1 -- 172.21.15.60:0/1684947208 learned_addr learned my addr 172.21.15.60:0/1684947208 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4699c700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a400cede0 0x7f3a4015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4599a700 1 -- 172.21.15.60:0/1684947208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 msgr2=0x7f3a401e6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4599a700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 0x7f3a401e6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4599a700 1 -- 172.21.15.60:0/1684947208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a400cede0 msgr2=0x7f3a4015a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4599a700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a400cede0 0x7f3a4015a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4599a700 1 -- 172.21.15.60:0/1684947208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0094a0 con 0x7f3a400b8200 2023-07-13T22:29:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.099+0000 7f3a4599a700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 0x7f3a400c7c30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000e810 tx=0x7f3a3000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.100+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1684947208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3000c070 con 0x7f3a400b8200 2023-07-13T22:29:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.100+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1684947208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a30013610 con 0x7f3a400b8200 2023-07-13T22:29:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.100+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1684947208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300139c0 con 0x7f3a400b8200 2023-07-13T22:29:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.100+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a401e43c0 con 0x7f3a400b8200 2023-07-13T22:29:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.100+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a401e6950 con 0x7f3a400b8200 2023-07-13T22:29:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.101+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a401e48e0 con 0x7f3a400b8200 2023-07-13T22:29:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.102+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1684947208 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a30013b20 con 0x7f3a400b8200 2023-07-13T22:29:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.103+0000 7f3a36ffd700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.103+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1684947208 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a30089240 con 0x7f3a400b8200 2023-07-13T22:29:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.103+0000 7f3a4619b700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.104+0000 7f3a4619b700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0090b0 tx=0x7f3a3c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.107+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1684947208 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3a401e48e0 con 0x7f3a400b8200 2023-07-13T22:29:27.215 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:29:27.220 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:29:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.316+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a401e48e0 con 0x7f3a400b8200 2023-07-13T22:29:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.317+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1684947208 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a401e48e0 con 0x7f3a400b8200 2023-07-13T22:29:27.296 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:29:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 msgr2=0x7f3a2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0090b0 tx=0x7f3a3c008030 comp rx=0 tx=0).stop 2023-07-13T22:29:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 msgr2=0x7f3a400c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 0x7f3a400c7c30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000e810 tx=0x7f3a3000eb20 comp rx=0 tx=0).stop 2023-07-13T22:29:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 shutdown_connections 2023-07-13T22:29:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a400cede0 0x7f3a4015a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a400b1ac0 0x7f3a401e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 --2- 172.21.15.60:0/1684947208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400b8200 0x7f3a400c7c30 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 >> 172.21.15.60:0/1684947208 conn(0x7f3a4006b2f0 msgr2=0x7f3a400b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 shutdown_connections 2023-07-13T22:29:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.319+0000 7f3a483ff700 1 -- 172.21.15.60:0/1684947208 wait complete. 2023-07-13T22:29:27.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:29:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.543+0000 7fd74d5ff700 1 Processor -- start 2023-07-13T22:29:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.543+0000 7fd74d5ff700 1 -- start start 2023-07-13T22:29:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.544+0000 7fd74d5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd7480c90e0 0x7fd748069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.544+0000 7fd74d5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd74806a410 0x7fd74806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.544+0000 7fd74d5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74806d030 con 0x7fd748158930 2023-07-13T22:29:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.544+0000 7fd74d5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74806a890 con 0x7fd74806a410 2023-07-13T22:29:27.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.544+0000 7fd74d5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74806ca70 con 0x7fd7480c90e0 2023-07-13T22:29:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd7477fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd74806a410 0x7fd74806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd746ffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd748158930 0x7fd74815c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:34866/0 (socket says 172.21.15.60:34866) 2023-07-13T22:29:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd73ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd7480c90e0 0x7fd748069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd746ffd700 1 -- 172.21.15.60:0/904406409 learned_addr learned my addr 172.21.15.60:0/904406409 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd7467fc700 1 -- 172.21.15.60:0/904406409 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118267891 0 0) 0x7fd74806d030 con 0x7fd748158930 2023-07-13T22:29:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd7467fc700 1 -- 172.21.15.60:0/904406409 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7340035b0 con 0x7fd748158930 2023-07-13T22:29:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd7477fe700 1 -- 172.21.15.60:0/904406409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd7480c90e0 msgr2=0x7fd748069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd7477fe700 1 --2- 172.21.15.60:0/904406409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd7480c90e0 0x7fd748069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd7477fe700 1 -- 172.21.15.60:0/904406409 >> v1:172.21.15.105:6789/0 conn(0x7fd748158930 legacy=0x7fd74815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd7477fe700 1 -- 172.21.15.60:0/904406409 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7480adc20 con 0x7fd74806a410 2023-07-13T22:29:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd73ffff700 1 --2- 172.21.15.60:0/904406409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd7480c90e0 0x7fd748069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.545+0000 7fd7477fe700 1 --2- 172.21.15.60:0/904406409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd74806a410 0x7fd74806d4c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd740009d90 tx=0x7fd74000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2986b59eefe18adb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd7467fc700 1 -- 172.21.15.60:0/904406409 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74000e070 con 0x7fd74806a410 2023-07-13T22:29:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd7467fc700 1 -- 172.21.15.60:0/904406409 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74000bb20 con 0x7fd74806a410 2023-07-13T22:29:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd7467fc700 1 -- 172.21.15.60:0/904406409 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740012430 con 0x7fd74806a410 2023-07-13T22:29:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd74d5ff700 1 -- 172.21.15.60:0/904406409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd74806a410 msgr2=0x7fd74806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd74d5ff700 1 --2- 172.21.15.60:0/904406409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd74806a410 0x7fd74806d4c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd740009d90 tx=0x7fd74000b330 comp rx=0 tx=0).stop 2023-07-13T22:29:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd74d5ff700 1 -- 172.21.15.60:0/904406409 shutdown_connections 2023-07-13T22:29:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd74d5ff700 1 --2- 172.21.15.60:0/904406409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd74806a410 0x7fd74806d4c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd74d5ff700 1 --2- 172.21.15.60:0/904406409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd7480c90e0 0x7fd748069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd74d5ff700 1 -- 172.21.15.60:0/904406409 >> 172.21.15.60:0/904406409 conn(0x7fd7480cbf50 msgr2=0x7fd7480ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.546+0000 7fd74d5ff700 1 -- 172.21.15.60:0/904406409 shutdown_connections 2023-07-13T22:29:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.547+0000 7fd74d5ff700 1 -- 172.21.15.60:0/904406409 wait complete. 2023-07-13T22:29:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.548+0000 7fd74d5ff700 1 Processor -- start 2023-07-13T22:29:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.548+0000 7fd74d5ff700 1 -- start start 2023-07-13T22:29:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.549+0000 7fd74d5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd74806a410 0x7fd748156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.549+0000 7fd74d5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7480c90e0 0x7fd7481edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.549+0000 7fd74d5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd748158930 0x7fd7480b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.549+0000 7fd74d5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7481edfe0 con 0x7fd748158930 2023-07-13T22:29:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.549+0000 7fd74d5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74813e370 con 0x7fd7480c90e0 2023-07-13T22:29:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.549+0000 7fd74d5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74813d7f0 con 0x7fd74806a410 2023-07-13T22:29:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd7477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd748158930 0x7fd7480b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd7477fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd748158930 0x7fd7480b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32914/0 (socket says 172.21.15.60:32914) 2023-07-13T22:29:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd7477fe700 1 -- 172.21.15.60:0/1195047503 learned_addr learned my addr 172.21.15.60:0/1195047503 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd746ffd700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd74806a410 0x7fd748156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd73ffff700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7480c90e0 0x7fd7481edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd7477fe700 1 -- 172.21.15.60:0/1195047503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd74806a410 msgr2=0x7fd748156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd7477fe700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd74806a410 0x7fd748156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd7477fe700 1 -- 172.21.15.60:0/1195047503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7480c90e0 msgr2=0x7fd7481edaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd7477fe700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7480c90e0 0x7fd7481edaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd7477fe700 1 -- 172.21.15.60:0/1195047503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73000a640 con 0x7fd748158930 2023-07-13T22:29:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd73ffff700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7480c90e0 0x7fd7481edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.550+0000 7fd7477fe700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd748158930 0x7fd7480b2920 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b370 tx=0x7fd740004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.551+0000 7fd73f7fe700 1 -- 172.21.15.60:0/1195047503 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740012430 con 0x7fd748158930 2023-07-13T22:29:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.551+0000 7fd73f7fe700 1 -- 172.21.15.60:0/1195047503 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd740020070 con 0x7fd748158930 2023-07-13T22:29:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.551+0000 7fd73f7fe700 1 -- 172.21.15.60:0/1195047503 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7400098f0 con 0x7fd748158930 2023-07-13T22:29:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.551+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd740009a70 con 0x7fd748158930 2023-07-13T22:29:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.551+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74813ddb0 con 0x7fd748158930 2023-07-13T22:29:27.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.552+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74813b450 con 0x7fd748158930 2023-07-13T22:29:27.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.553+0000 7fd73f7fe700 1 -- 172.21.15.60:0/1195047503 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd740004040 con 0x7fd748158930 2023-07-13T22:29:27.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.554+0000 7fd73f7fe700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd72806ace0 0x7fd72806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.554+0000 7fd73f7fe700 1 -- 172.21.15.60:0/1195047503 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd740022070 con 0x7fd748158930 2023-07-13T22:29:27.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.554+0000 7fd746ffd700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd72806ace0 0x7fd72806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.554+0000 7fd746ffd700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd72806ace0 0x7fd72806d190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd738007fd0 tx=0x7fd738007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:27.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.558+0000 7fd73f7fe700 1 -- 172.21.15.60:0/1195047503 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd74813b450 con 0x7fd748158930 2023-07-13T22:29:27.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.768+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd74813b450 con 0x7fd748158930 2023-07-13T22:29:27.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.768+0000 7fd73f7fe700 1 -- 172.21.15.60:0/1195047503 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd74002c020 con 0x7fd748158930 2023-07-13T22:29:27.748 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:29:27.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd72806ace0 msgr2=0x7fd72806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd72806ace0 0x7fd72806d190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd738007fd0 tx=0x7fd738007dc0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd748158930 msgr2=0x7fd7480b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd748158930 0x7fd7480b2920 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b370 tx=0x7fd740004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 shutdown_connections 2023-07-13T22:29:27.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd72806ace0 0x7fd72806d190 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd748158930 0x7fd7480b2920 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd74806a410 0x7fd748156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 --2- 172.21.15.60:0/1195047503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7480c90e0 0x7fd7481edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.770+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 >> 172.21.15.60:0/1195047503 conn(0x7fd7480cbf50 msgr2=0x7fd7480b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:27.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.771+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 shutdown_connections 2023-07-13T22:29:27.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.771+0000 7fd74d5ff700 1 -- 172.21.15.60:0/1195047503 wait complete. 2023-07-13T22:29:27.769 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:29:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.993+0000 7efc6d3ff700 1 Processor -- start 2023-07-13T22:29:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.994+0000 7efc6d3ff700 1 -- start start 2023-07-13T22:29:27.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.995+0000 7efc6d3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc680b8400 0x7efc680cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.995+0000 7efc6d3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc680d3750 0x7efc680cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.995+0000 7efc6d3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc680d0ac0 con 0x7efc680cfad0 2023-07-13T22:29:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.995+0000 7efc6d3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc680cff50 con 0x7efc680d3750 2023-07-13T22:29:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.995+0000 7efc6d3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc680b87d0 con 0x7efc680b8400 2023-07-13T22:29:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.995+0000 7efc5ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc680d3750 0x7efc680cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.995+0000 7efc5ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc680d3750 0x7efc680cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32924/0 (socket says 172.21.15.60:32924) 2023-07-13T22:29:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.995+0000 7efc5ffff700 1 -- 172.21.15.60:0/2442910402 learned_addr learned my addr 172.21.15.60:0/2442910402 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.995+0000 7efc66ffd700 1 --2- 172.21.15.60:0/2442910402 >> v2:172.21.15.105:3300/0 conn(0x7efc680b8400 0x7efc680cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.996+0000 7efc5ffff700 1 -- 172.21.15.60:0/2442910402 >> v2:172.21.15.105:3300/0 conn(0x7efc680b8400 msgr2=0x7efc680cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.996+0000 7efc5ffff700 1 --2- 172.21.15.60:0/2442910402 >> v2:172.21.15.105:3300/0 conn(0x7efc680b8400 0x7efc680cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.996+0000 7efc5ffff700 1 -- 172.21.15.60:0/2442910402 >> v1:172.21.15.105:6789/0 conn(0x7efc680cfad0 legacy=0x7efc6815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.996+0000 7efc5ffff700 1 -- 172.21.15.60:0/2442910402 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc680b4eb0 con 0x7efc680d3750 2023-07-13T22:29:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.996+0000 7efc5ffff700 1 --2- 172.21.15.60:0/2442910402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 0x7efc680cf590 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7efc680cf560 tx=0x7efc50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c3796387a05ece server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.996+0000 7efc667fc700 1 -- 172.21.15.60:0/2442910402 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50011830 con 0x7efc680d3750 2023-07-13T22:29:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.996+0000 7efc667fc700 1 -- 172.21.15.60:0/2442910402 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50011990 con 0x7efc680d3750 2023-07-13T22:29:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.996+0000 7efc667fc700 1 -- 172.21.15.60:0/2442910402 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50011d20 con 0x7efc680d3750 2023-07-13T22:29:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.997+0000 7efc6d3ff700 1 -- 172.21.15.60:0/2442910402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 msgr2=0x7efc680cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.997+0000 7efc6d3ff700 1 --2- 172.21.15.60:0/2442910402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 0x7efc680cf590 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7efc680cf560 tx=0x7efc50009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.997+0000 7efc6d3ff700 1 -- 172.21.15.60:0/2442910402 shutdown_connections 2023-07-13T22:29:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.997+0000 7efc6d3ff700 1 --2- 172.21.15.60:0/2442910402 >> v2:172.21.15.105:3300/0 conn(0x7efc680b8400 0x7efc680cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.997+0000 7efc6d3ff700 1 --2- 172.21.15.60:0/2442910402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 0x7efc680cf590 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.997+0000 7efc6d3ff700 1 -- 172.21.15.60:0/2442910402 >> 172.21.15.60:0/2442910402 conn(0x7efc6806b390 msgr2=0x7efc6806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:27.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.997+0000 7efc6d3ff700 1 -- 172.21.15.60:0/2442910402 shutdown_connections 2023-07-13T22:29:27.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.997+0000 7efc6d3ff700 1 -- 172.21.15.60:0/2442910402 wait complete. 2023-07-13T22:29:27.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.999+0000 7efc6d3ff700 1 Processor -- start 2023-07-13T22:29:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:27.999+0000 7efc6d3ff700 1 -- start start 2023-07-13T22:29:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc6d3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc680b8400 0x7efc680c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc6d3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc680cfad0 0x7efc6815f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc6d3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 0x7efc680c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc6d3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc680c6140 con 0x7efc680d3750 2023-07-13T22:29:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc6d3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6815d8a0 con 0x7efc680cfad0 2023-07-13T22:29:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc6d3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6815e2d0 con 0x7efc680b8400 2023-07-13T22:29:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 0x7efc680c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 0x7efc680c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32934/0 (socket says 172.21.15.60:32934) 2023-07-13T22:29:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc677fe700 1 -- 172.21.15.60:0/182857374 learned_addr learned my addr 172.21.15.60:0/182857374 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc66ffd700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc680b8400 0x7efc680c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.000+0000 7efc5ffff700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc680cfad0 0x7efc6815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc677fe700 1 -- 172.21.15.60:0/182857374 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc680b8400 msgr2=0x7efc680c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc677fe700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc680b8400 0x7efc680c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc677fe700 1 -- 172.21.15.60:0/182857374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc680cfad0 msgr2=0x7efc6815f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc677fe700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc680cfad0 0x7efc6815f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc677fe700 1 -- 172.21.15.60:0/182857374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50009550 con 0x7efc680d3750 2023-07-13T22:29:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc677fe700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 0x7efc680c66b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7efc6000cb50 tx=0x7efc6000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc5f7fe700 1 -- 172.21.15.60:0/182857374 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60003c00 con 0x7efc680d3750 2023-07-13T22:29:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc5f7fe700 1 -- 172.21.15.60:0/182857374 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc60003d60 con 0x7efc680d3750 2023-07-13T22:29:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc5f7fe700 1 -- 172.21.15.60:0/182857374 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6000ab70 con 0x7efc680d3750 2023-07-13T22:29:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6815e5b0 con 0x7efc680d3750 2023-07-13T22:29:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.001+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6815eb70 con 0x7efc680d3750 2023-07-13T22:29:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.002+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc680cf620 con 0x7efc680d3750 2023-07-13T22:29:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.003+0000 7efc5f7fe700 1 -- 172.21.15.60:0/182857374 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc6000acd0 con 0x7efc680d3750 2023-07-13T22:29:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.004+0000 7efc5f7fe700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc4806ace0 0x7efc4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.004+0000 7efc5f7fe700 1 -- 172.21.15.60:0/182857374 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc600895c0 con 0x7efc680d3750 2023-07-13T22:29:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.004+0000 7efc66ffd700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc4806ace0 0x7efc4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.005+0000 7efc66ffd700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc4806ace0 0x7efc4806d190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efc58009ad0 tx=0x7efc58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:27.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.008+0000 7efc5f7fe700 1 -- 172.21.15.60:0/182857374 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efc600545d0 con 0x7efc680d3750 2023-07-13T22:29:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.231+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efc680c9a10 con 0x7efc680d3750 2023-07-13T22:29:28.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.231+0000 7efc5f7fe700 1 -- 172.21.15.60:0/182857374 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) v1 ==== 80+0+0 (secure 0 0 0) 0x7efc60057bf0 con 0x7efc680d3750 2023-07-13T22:29:28.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.233+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc4806ace0 msgr2=0x7efc4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:28.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc4806ace0 0x7efc4806d190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efc58009ad0 tx=0x7efc58008040 comp rx=0 tx=0).stop 2023-07-13T22:29:28.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 msgr2=0x7efc680c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:28.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 0x7efc680c66b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7efc6000cb50 tx=0x7efc6000cf10 comp rx=0 tx=0).stop 2023-07-13T22:29:28.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 shutdown_connections 2023-07-13T22:29:28.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc4806ace0 0x7efc4806d190 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:28.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc680d3750 0x7efc680c66b0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:28.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc680b8400 0x7efc680c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:28.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 --2- 172.21.15.60:0/182857374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc680cfad0 0x7efc6815f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:28.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 >> 172.21.15.60:0/182857374 conn(0x7efc6806b390 msgr2=0x7efc680b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:28.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 shutdown_connections 2023-07-13T22:29:28.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.234+0000 7efc6d3ff700 1 -- 172.21.15.60:0/182857374 wait complete. 2023-07-13T22:29:28.217 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:29:28.231 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:29:28.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:28.232 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:28.306 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[125439]: fuse finished with error 0 and tester_r 0 2023-07-13T22:29:28.310 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:29:28.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:28.311 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:29:28.472 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:29:28.473 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:29:28.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:29:28.522 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:29:28.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:28.523 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:29:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-13T22:29:28.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:29:28.609 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:29:28.610 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:29:28.656 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:29:28.656 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:29:28.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:28.657 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:29:28.741 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-13T22:29:28.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:29:28.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:28.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.966+0000 7f93267ff700 1 Processor -- start 2023-07-13T22:29:28.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.966+0000 7f93267ff700 1 -- start start 2023-07-13T22:29:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.967+0000 7f93267ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93200b8570 0x7f93200ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.967+0000 7f93267ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f93200d35c0 0x7f93200d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f93267ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93200d09d0 con 0x7f93200cf3f0 2023-07-13T22:29:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f93267ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93200d3b30 con 0x7f93200b8570 2023-07-13T22:29:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f93267ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93200cda10 con 0x7f93200d35c0 2023-07-13T22:29:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f9324d9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f93200cf3f0 0x7f932015fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49822/0 (socket says 172.21.15.60:49822) 2023-07-13T22:29:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f931ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93200b8570 0x7f93200ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f9324d9c700 1 -- 172.21.15.60:0/3312485567 learned_addr learned my addr 172.21.15.60:0/3312485567 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f931f7fe700 1 --2- 172.21.15.60:0/3312485567 >> v2:172.21.15.105:3300/0 conn(0x7f93200d35c0 0x7f93200d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f931ffff700 1 -- 172.21.15.60:0/3312485567 >> v2:172.21.15.105:3300/0 conn(0x7f93200d35c0 msgr2=0x7f93200d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f931ffff700 1 --2- 172.21.15.60:0/3312485567 >> v2:172.21.15.105:3300/0 conn(0x7f93200d35c0 0x7f93200d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f931ffff700 1 -- 172.21.15.60:0/3312485567 >> v1:172.21.15.60:6789/0 conn(0x7f93200cf3f0 legacy=0x7f932015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.968+0000 7f931ffff700 1 -- 172.21.15.60:0/3312485567 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200cd450 con 0x7f93200b8570 2023-07-13T22:29:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f931ffff700 1 --2- 172.21.15.60:0/3312485567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 0x7f93200ceeb0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f93200d07e0 tx=0x7f9310009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07738ed849abf5a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f931effd700 1 -- 172.21.15.60:0/3312485567 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310005e30 con 0x7f93200b8570 2023-07-13T22:29:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f931effd700 1 -- 172.21.15.60:0/3312485567 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310006030 con 0x7f93200b8570 2023-07-13T22:29:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f931effd700 1 -- 172.21.15.60:0/3312485567 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93100063c0 con 0x7f93200b8570 2023-07-13T22:29:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f93267ff700 1 -- 172.21.15.60:0/3312485567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 msgr2=0x7f93200ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:28.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f93267ff700 1 --2- 172.21.15.60:0/3312485567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 0x7f93200ceeb0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f93200d07e0 tx=0x7f9310009b00 comp rx=0 tx=0).stop 2023-07-13T22:29:28.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f93267ff700 1 -- 172.21.15.60:0/3312485567 shutdown_connections 2023-07-13T22:29:28.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f93267ff700 1 --2- 172.21.15.60:0/3312485567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 0x7f93200ceeb0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:28.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f93267ff700 1 --2- 172.21.15.60:0/3312485567 >> v2:172.21.15.105:3300/0 conn(0x7f93200d35c0 0x7f93200d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:28.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.969+0000 7f93267ff700 1 -- 172.21.15.60:0/3312485567 >> 172.21.15.60:0/3312485567 conn(0x7f9320071600 msgr2=0x7f93200719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:28.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.970+0000 7f93267ff700 1 -- 172.21.15.60:0/3312485567 shutdown_connections 2023-07-13T22:29:28.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.970+0000 7f93267ff700 1 -- 172.21.15.60:0/3312485567 wait complete. 2023-07-13T22:29:28.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.971+0000 7f93267ff700 1 Processor -- start 2023-07-13T22:29:28.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.971+0000 7f93267ff700 1 -- start start 2023-07-13T22:29:28.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.972+0000 7f93267ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 0x7f9320158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:28.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.972+0000 7f93267ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93200cf3f0 0x7f9320158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:28.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.972+0000 7f93267ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93200d35c0 0x7f9320159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:28.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.972+0000 7f93267ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320155440 con 0x7f93200b8570 2023-07-13T22:29:28.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.972+0000 7f93267ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320159960 con 0x7f93200cf3f0 2023-07-13T22:29:28.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.972+0000 7f93267ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93201574e0 con 0x7f93200d35c0 2023-07-13T22:29:28.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 0x7f9320158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:28.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 0x7f9320158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32958/0 (socket says 172.21.15.60:32958) 2023-07-13T22:29:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931ffff700 1 -- 172.21.15.60:0/1088268298 learned_addr learned my addr 172.21.15.60:0/1088268298 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931f7fe700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93200cf3f0 0x7f9320158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f9324d9c700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93200d35c0 0x7f9320159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931ffff700 1 -- 172.21.15.60:0/1088268298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93200d35c0 msgr2=0x7f9320159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931ffff700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93200d35c0 0x7f9320159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931ffff700 1 -- 172.21.15.60:0/1088268298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93200cf3f0 msgr2=0x7f9320158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931ffff700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93200cf3f0 0x7f9320158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931ffff700 1 -- 172.21.15.60:0/1088268298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9310009430 con 0x7f93200b8570 2023-07-13T22:29:28.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.973+0000 7f931f7fe700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93200cf3f0 0x7f9320158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:28.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.974+0000 7f931ffff700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 0x7f9320158580 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9310000c00 tx=0x7f9310004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:28.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.974+0000 7f931cff9700 1 -- 172.21.15.60:0/1088268298 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310006870 con 0x7f93200b8570 2023-07-13T22:29:28.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.974+0000 7f931cff9700 1 -- 172.21.15.60:0/1088268298 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310020070 con 0x7f93200b8570 2023-07-13T22:29:28.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.974+0000 7f931cff9700 1 -- 172.21.15.60:0/1088268298 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310004280 con 0x7f93200b8570 2023-07-13T22:29:28.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.974+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93201577c0 con 0x7f93200b8570 2023-07-13T22:29:28.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.974+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9320157d80 con 0x7f93200b8570 2023-07-13T22:29:28.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.975+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932004e280 con 0x7f93200b8570 2023-07-13T22:29:28.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.976+0000 7f931cff9700 1 -- 172.21.15.60:0/1088268298 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9310022030 con 0x7f93200b8570 2023-07-13T22:29:28.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.977+0000 7f931cff9700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930c06ace0 0x7f930c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:28.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.977+0000 7f931cff9700 1 -- 172.21.15.60:0/1088268298 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f931008a960 con 0x7f93200b8570 2023-07-13T22:29:28.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.977+0000 7f931f7fe700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930c06ace0 0x7f930c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:28.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.978+0000 7f931f7fe700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930c06ace0 0x7f930c06d190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f93200b7c10 tx=0x7f9314009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:28.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:28.982+0000 7f931cff9700 1 -- 172.21.15.60:0/1088268298 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9310055aa0 con 0x7f93200b8570 2023-07-13T22:29:29.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.232+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93200cea90 con 0x7f93200b8570 2023-07-13T22:29:29.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.233+0000 7f931cff9700 1 -- 172.21.15.60:0/1088268298 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f93100590c0 con 0x7f93200b8570 2023-07-13T22:29:29.213 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:29.213 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":64,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:58.254590+0000","modified":"2023-07-13T22:29:19.596439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6837/4078186433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":4078186433},{"type":"v1","addr":"172.21.15.60:6837","nonce":4078186433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-13T22:29:29.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930c06ace0 msgr2=0x7f930c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930c06ace0 0x7f930c06d190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f93200b7c10 tx=0x7f9314009210 comp rx=0 tx=0).stop 2023-07-13T22:29:29.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 msgr2=0x7f9320158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 0x7f9320158580 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9310000c00 tx=0x7f9310004750 comp rx=0 tx=0).stop 2023-07-13T22:29:29.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 shutdown_connections 2023-07-13T22:29:29.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f930c06ace0 0x7f930c06d190 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93200b8570 0x7f9320158580 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f93200d35c0 0x7f9320159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 --2- 172.21.15.60:0/1088268298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93200cf3f0 0x7f9320158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.235+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 >> 172.21.15.60:0/1088268298 conn(0x7f9320071600 msgr2=0x7f93200cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:29.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.236+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 shutdown_connections 2023-07-13T22:29:29.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.236+0000 7f93267ff700 1 -- 172.21.15.60:0/1088268298 wait complete. 2023-07-13T22:29:29.218 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 64 2023-07-13T22:29:29.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:29.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.459+0000 7f0df5bff700 1 Processor -- start 2023-07-13T22:29:29.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.459+0000 7f0df5bff700 1 -- start start 2023-07-13T22:29:29.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.460+0000 7f0df5bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 0x7f0df0070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.460+0000 7f0df5bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0df00c9450 0x7f0df015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.460+0000 7f0df5bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df0070270 con 0x7f0df015a820 2023-07-13T22:29:29.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.460+0000 7f0df5bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df015b2e0 con 0x7f0df00c9450 2023-07-13T22:29:29.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.460+0000 7f0df5bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df015b5c0 con 0x7f0df0154950 2023-07-13T22:29:29.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0deffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0df00c9450 0x7f0df015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0deeffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0df015a820 0x7f0df00c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49834/0 (socket says 172.21.15.60:49834) 2023-07-13T22:29:29.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0deeffd700 1 -- 172.21.15.60:0/2219461545 learned_addr learned my addr 172.21.15.60:0/2219461545 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:29.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0def7fe700 1 --2- 172.21.15.60:0/2219461545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 0x7f0df0070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0dee7fc700 1 -- 172.21.15.60:0/2219461545 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142517496 0 0) 0x7f0df0070270 con 0x7f0df015a820 2023-07-13T22:29:29.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0deffff700 1 -- 172.21.15.60:0/2219461545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 msgr2=0x7f0df0070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:29.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0deffff700 1 --2- 172.21.15.60:0/2219461545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 0x7f0df0070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0deffff700 1 -- 172.21.15.60:0/2219461545 >> v1:172.21.15.60:6789/0 conn(0x7f0df015a820 legacy=0x7f0df00c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0deffff700 1 -- 172.21.15.60:0/2219461545 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df00704c0 con 0x7f0df00c9450 2023-07-13T22:29:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.461+0000 7f0deffff700 1 --2- 172.21.15.60:0/2219461545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df00c9450 0x7f0df015ac90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009d90 tx=0x7f0de800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6826e26e811db285 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0dee7fc700 1 -- 172.21.15.60:0/2219461545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de800e070 con 0x7f0df00c9450 2023-07-13T22:29:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0dee7fc700 1 -- 172.21.15.60:0/2219461545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de800bb20 con 0x7f0df00c9450 2023-07-13T22:29:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0dee7fc700 1 -- 172.21.15.60:0/2219461545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8012430 con 0x7f0df00c9450 2023-07-13T22:29:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0df5bff700 1 -- 172.21.15.60:0/2219461545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df00c9450 msgr2=0x7f0df015ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0df5bff700 1 --2- 172.21.15.60:0/2219461545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df00c9450 0x7f0df015ac90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009d90 tx=0x7f0de800b330 comp rx=0 tx=0).stop 2023-07-13T22:29:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0df5bff700 1 -- 172.21.15.60:0/2219461545 shutdown_connections 2023-07-13T22:29:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0df5bff700 1 --2- 172.21.15.60:0/2219461545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df00c9450 0x7f0df015ac90 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0df5bff700 1 --2- 172.21.15.60:0/2219461545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 0x7f0df0070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0df5bff700 1 -- 172.21.15.60:0/2219461545 >> 172.21.15.60:0/2219461545 conn(0x7f0df00cf410 msgr2=0x7f0df00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0df5bff700 1 -- 172.21.15.60:0/2219461545 shutdown_connections 2023-07-13T22:29:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.462+0000 7f0df5bff700 1 -- 172.21.15.60:0/2219461545 wait complete. 2023-07-13T22:29:29.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.464+0000 7f0df5bff700 1 Processor -- start 2023-07-13T22:29:29.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.464+0000 7f0df5bff700 1 -- start start 2023-07-13T22:29:29.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.465+0000 7f0df5bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df00c9450 0x7f0df00c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.465+0000 7f0df5bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 0x7f0df00c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.465+0000 7f0df5bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df015a820 0x7f0df00c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.465+0000 7f0df5bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df01efef0 con 0x7f0df015a820 2023-07-13T22:29:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.465+0000 7f0df5bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df01f0d40 con 0x7f0df00c9450 2023-07-13T22:29:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.465+0000 7f0df5bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df01f0130 con 0x7f0df0154950 2023-07-13T22:29:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.465+0000 7f0deffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df015a820 0x7f0df00c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0deffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df015a820 0x7f0df00c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32984/0 (socket says 172.21.15.60:32984) 2023-07-13T22:29:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0deffff700 1 -- 172.21.15.60:0/1732290075 learned_addr learned my addr 172.21.15.60:0/1732290075 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0deeffd700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 0x7f0df00c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0def7fe700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df00c9450 0x7f0df00c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0deffff700 1 -- 172.21.15.60:0/1732290075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 msgr2=0x7f0df00c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0deffff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 0x7f0df00c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0deffff700 1 -- 172.21.15.60:0/1732290075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df00c9450 msgr2=0x7f0df00c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0deffff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df00c9450 0x7f0df00c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0deffff700 1 -- 172.21.15.60:0/1732290075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8009a70 con 0x7f0df015a820 2023-07-13T22:29:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0deffff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df015a820 0x7f0df00c7670 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0de8013000 tx=0x7f0de8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:29.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0dd7fff700 1 -- 172.21.15.60:0/1732290075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8012430 con 0x7f0df015a820 2023-07-13T22:29:29.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0dd7fff700 1 -- 172.21.15.60:0/1732290075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de8020070 con 0x7f0df015a820 2023-07-13T22:29:29.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0dd7fff700 1 -- 172.21.15.60:0/1732290075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8004280 con 0x7f0df015a820 2023-07-13T22:29:29.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.466+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df01f0410 con 0x7f0df015a820 2023-07-13T22:29:29.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.467+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df01f09d0 con 0x7f0df015a820 2023-07-13T22:29:29.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.468+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df004e280 con 0x7f0df015a820 2023-07-13T22:29:29.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.468+0000 7f0dd7fff700 1 -- 172.21.15.60:0/1732290075 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0de80043e0 con 0x7f0df015a820 2023-07-13T22:29:29.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.470+0000 7f0dd7fff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ddc06ace0 0x7f0ddc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.470+0000 7f0dd7fff700 1 -- 172.21.15.60:0/1732290075 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0de8022070 con 0x7f0df015a820 2023-07-13T22:29:29.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.470+0000 7f0def7fe700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ddc06ace0 0x7f0ddc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.471+0000 7f0def7fe700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ddc06ace0 0x7f0ddc06d190 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009b70 tx=0x7f0de0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:29.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.474+0000 7f0dd7fff700 1 -- 172.21.15.60:0/1732290075 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0de80bd330 con 0x7f0df015a820 2023-07-13T22:29:29.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.686+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0df015a210 con 0x7f0df015a820 2023-07-13T22:29:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.688+0000 7f0dd7fff700 1 -- 172.21.15.60:0/1732290075 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+18190 (secure 0 0 0) 0x7f0de80562a0 con 0x7f0df015a820 2023-07-13T22:29:29.668 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:29.669 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":100,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:59.528113+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:56.533994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:57.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:29.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ddc06ace0 msgr2=0x7f0ddc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ddc06ace0 0x7f0ddc06d190 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009b70 tx=0x7f0de0008040 comp rx=0 tx=0).stop 2023-07-13T22:29:29.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df015a820 msgr2=0x7f0df00c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df015a820 0x7f0df00c7670 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0de8013000 tx=0x7f0de8004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 shutdown_connections 2023-07-13T22:29:29.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ddc06ace0 0x7f0ddc06d190 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0df015a820 0x7f0df00c7670 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0df00c9450 0x7f0df00c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 --2- 172.21.15.60:0/1732290075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0df0154950 0x7f0df00c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 >> 172.21.15.60:0/1732290075 conn(0x7f0df00cf410 msgr2=0x7f0df006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:29.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 shutdown_connections 2023-07-13T22:29:29.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.691+0000 7f0df5bff700 1 -- 172.21.15.60:0/1732290075 wait complete. 2023-07-13T22:29:29.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:28:56.533994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:28:57.226035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:29:29.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:29:29.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.917+0000 7f72dabff700 1 Processor -- start 2023-07-13T22:29:29.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.917+0000 7f72dabff700 1 -- start start 2023-07-13T22:29:29.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.918+0000 7f72dabff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d406ce70 0x7f72d40ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.918+0000 7f72dabff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d40c8e90 con 0x7f72d406a5e0 2023-07-13T22:29:29.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.918+0000 7f72dabff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d40caf30 con 0x7f72d40add00 2023-07-13T22:29:29.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.919+0000 7f72dabff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40ae0d0 con 0x7f72d406ce70 2023-07-13T22:29:29.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.919+0000 7f72d3fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f72d406a5e0 0x7f72d40b1690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49850/0 (socket says 172.21.15.60:49850) 2023-07-13T22:29:29.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.919+0000 7f72d3fff700 1 -- 172.21.15.60:0/3707221241 learned_addr learned my addr 172.21.15.60:0/3707221241 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:29.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.919+0000 7f72d919c700 1 --2- 172.21.15.60:0/3707221241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d406ce70 0x7f72d40ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.919+0000 7f72d37fe700 1 -- 172.21.15.60:0/3707221241 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265939367 0 0) 0x7f72d40c8e90 con 0x7f72d406a5e0 2023-07-13T22:29:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.919+0000 7f72d37fe700 1 -- 172.21.15.60:0/3707221241 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c40035b0 con 0x7f72d406a5e0 2023-07-13T22:29:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.919+0000 7f72d37fe700 1 -- 172.21.15.60:0/3707221241 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143022805 0 0) 0x7f72d40caf30 con 0x7f72d40add00 2023-07-13T22:29:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.919+0000 7f72d37fe700 1 -- 172.21.15.60:0/3707221241 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d40c8e90 con 0x7f72d40add00 2023-07-13T22:29:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.920+0000 7f72d919c700 1 -- 172.21.15.60:0/3707221241 >> v1:172.21.15.105:6789/0 conn(0x7f72d40add00 legacy=0x7f72d4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.920+0000 7f72d919c700 1 -- 172.21.15.60:0/3707221241 >> v1:172.21.15.60:6789/0 conn(0x7f72d406a5e0 legacy=0x7f72d40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.920+0000 7f72d37fe700 1 -- 172.21.15.60:0/3707221241 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3371188162 0 0) 0x7f72c40035b0 con 0x7f72d406a5e0 2023-07-13T22:29:29.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.920+0000 7f72d919c700 1 -- 172.21.15.60:0/3707221241 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40c90e0 con 0x7f72d406ce70 2023-07-13T22:29:29.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.920+0000 7f72d919c700 1 --2- 172.21.15.60:0/3707221241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d406ce70 0x7f72d40ca9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009dc0 tx=0x7f72cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f14083b7c08844 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:29.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.920+0000 7f72d37fe700 1 -- 172.21.15.60:0/3707221241 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00e070 con 0x7f72d406ce70 2023-07-13T22:29:29.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.920+0000 7f72d37fe700 1 -- 172.21.15.60:0/3707221241 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc003a20 con 0x7f72d406ce70 2023-07-13T22:29:29.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.920+0000 7f72d37fe700 1 -- 172.21.15.60:0/3707221241 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0001670 con 0x7f72d406ce70 2023-07-13T22:29:29.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.921+0000 7f72dabff700 1 -- 172.21.15.60:0/3707221241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d406ce70 msgr2=0x7f72d40ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.921+0000 7f72dabff700 1 --2- 172.21.15.60:0/3707221241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d406ce70 0x7f72d40ca9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009dc0 tx=0x7f72cc00b330 comp rx=0 tx=0).stop 2023-07-13T22:29:29.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.921+0000 7f72dabff700 1 -- 172.21.15.60:0/3707221241 shutdown_connections 2023-07-13T22:29:29.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.921+0000 7f72dabff700 1 --2- 172.21.15.60:0/3707221241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d406ce70 0x7f72d40ca9f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.921+0000 7f72dabff700 1 -- 172.21.15.60:0/3707221241 >> 172.21.15.60:0/3707221241 conn(0x7f72d40cbf50 msgr2=0x7f72d40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:29.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.921+0000 7f72dabff700 1 -- 172.21.15.60:0/3707221241 shutdown_connections 2023-07-13T22:29:29.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.921+0000 7f72dabff700 1 -- 172.21.15.60:0/3707221241 wait complete. 2023-07-13T22:29:29.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.923+0000 7f72dabff700 1 Processor -- start 2023-07-13T22:29:29.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.923+0000 7f72dabff700 1 -- start start 2023-07-13T22:29:29.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72dabff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d406a5e0 0x7f72d4150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72dabff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d406ce70 0x7f72d40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72dabff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d40add00 0x7f72d40cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72dabff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d41514d0 con 0x7f72d406a5e0 2023-07-13T22:29:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72dabff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40ccd10 con 0x7f72d406ce70 2023-07-13T22:29:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72dabff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40ccff0 con 0x7f72d40add00 2023-07-13T22:29:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d406a5e0 0x7f72d4150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d406a5e0 0x7f72d4150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32986/0 (socket says 172.21.15.60:32986) 2023-07-13T22:29:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d899b700 1 -- 172.21.15.60:0/3262044429 learned_addr learned my addr 172.21.15.60:0/3262044429 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d919c700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d40add00 0x7f72d40cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d3fff700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d406ce70 0x7f72d40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d899b700 1 -- 172.21.15.60:0/3262044429 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d40add00 msgr2=0x7f72d40cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d899b700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d40add00 0x7f72d40cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d899b700 1 -- 172.21.15.60:0/3262044429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d406ce70 msgr2=0x7f72d40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d899b700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d406ce70 0x7f72d40cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.924+0000 7f72d899b700 1 -- 172.21.15.60:0/3262044429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc009a70 con 0x7f72d406a5e0 2023-07-13T22:29:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.925+0000 7f72d899b700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d406a5e0 0x7f72d4150f90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f72c800c520 tx=0x7f72c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.925+0000 7f72d17fa700 1 -- 172.21.15.60:0/3262044429 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c80052d0 con 0x7f72d406a5e0 2023-07-13T22:29:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.925+0000 7f72d17fa700 1 -- 172.21.15.60:0/3262044429 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c8005430 con 0x7f72d406a5e0 2023-07-13T22:29:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.925+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d40cd2d0 con 0x7f72d406a5e0 2023-07-13T22:29:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.925+0000 7f72d17fa700 1 -- 172.21.15.60:0/3262044429 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c80038c0 con 0x7f72d406a5e0 2023-07-13T22:29:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.925+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d40cd890 con 0x7f72d406a5e0 2023-07-13T22:29:29.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.926+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d406a450 con 0x7f72d406a5e0 2023-07-13T22:29:29.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.927+0000 7f72d17fa700 1 -- 172.21.15.60:0/3262044429 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72c8003a20 con 0x7f72d406a5e0 2023-07-13T22:29:29.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.928+0000 7f72d17fa700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b806ace0 0x7f72b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:29.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.928+0000 7f72d17fa700 1 -- 172.21.15.60:0/3262044429 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72c80889a0 con 0x7f72d406a5e0 2023-07-13T22:29:29.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.929+0000 7f72d3fff700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b806ace0 0x7f72b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:29.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.929+0000 7f72d3fff700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b806ace0 0x7f72b806d190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f72c0007e40 tx=0x7f72c0007dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:29.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:29.932+0000 7f72d17fa700 1 -- 172.21.15.60:0/3262044429 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f72c80539b0 con 0x7f72d406a5e0 2023-07-13T22:29:30.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.137+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f72d40cdab0 con 0x7f72d406a5e0 2023-07-13T22:29:30.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.138+0000 7f72d17fa700 1 -- 172.21.15.60:0/3262044429 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v100) v1 ==== 92+0+10 (secure 0 0 0) 0x7f72c8017020 con 0x7f72d406a5e0 2023-07-13T22:29:30.117 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:29:30.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b806ace0 msgr2=0x7f72b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b806ace0 0x7f72b806d190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f72c0007e40 tx=0x7f72c0007dd0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d406a5e0 msgr2=0x7f72d4150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d406a5e0 0x7f72d4150f90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f72c800c520 tx=0x7f72c800c830 comp rx=0 tx=0).stop 2023-07-13T22:29:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 shutdown_connections 2023-07-13T22:29:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f72b806ace0 0x7f72b806d190 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72d406a5e0 0x7f72d4150f90 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72d40add00 0x7f72d40cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 --2- 172.21.15.60:0/3262044429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72d406ce70 0x7f72d40cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 >> 172.21.15.60:0/3262044429 conn(0x7f72d40cbf50 msgr2=0x7f72d4071170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 shutdown_connections 2023-07-13T22:29:30.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.140+0000 7f72dabff700 1 -- 172.21.15.60:0/3262044429 wait complete. 2023-07-13T22:29:30.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:29:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.364+0000 7fdc43fff700 1 Processor -- start 2023-07-13T22:29:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.364+0000 7fdc43fff700 1 -- start start 2023-07-13T22:29:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.365+0000 7fdc43fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdc3c06a5e0 0x7fdc3c06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.365+0000 7fdc43fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c0ca9f0 0x7fdc3c0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.365+0000 7fdc43fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3c0c8e90 con 0x7fdc3c0add00 2023-07-13T22:29:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.365+0000 7fdc43fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3c06d3b0 con 0x7fdc3c06a5e0 2023-07-13T22:29:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.365+0000 7fdc43fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0ae0d0 con 0x7fdc3c0ca9f0 2023-07-13T22:29:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.365+0000 7fdc41d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdc3c0add00 0x7fdc3c069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49864/0 (socket says 172.21.15.60:49864) 2023-07-13T22:29:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.365+0000 7fdc41d9b700 1 -- 172.21.15.60:0/2836842827 learned_addr learned my addr 172.21.15.60:0/2836842827 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.365+0000 7fdc4159a700 1 --2- 172.21.15.60:0/2836842827 >> v2:172.21.15.105:3300/0 conn(0x7fdc3c06a5e0 0x7fdc3c06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.365+0000 7fdc4259c700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c0ca9f0 0x7fdc3c0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400461776 0 0) 0x7fdc3c0c8e90 con 0x7fdc3c0add00 2023-07-13T22:29:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc280035b0 con 0x7fdc3c0add00 2023-07-13T22:29:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987412907 0 0) 0x7fdc280035b0 con 0x7fdc3c0add00 2023-07-13T22:29:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c0ca9f0 msgr2=0x7fdc3c0b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c0ca9f0 0x7fdc3c0b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 >> v2:172.21.15.105:3300/0 conn(0x7fdc3c06a5e0 msgr2=0x7fdc3c06ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 --2- 172.21.15.60:0/2836842827 >> v2:172.21.15.105:3300/0 conn(0x7fdc3c06a5e0 0x7fdc3c06ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c0c90e0 con 0x7fdc3c0add00 2023-07-13T22:29:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fdc38001670 con 0x7fdc3c0add00 2023-07-13T22:29:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 >> v1:172.21.15.60:6789/0 conn(0x7fdc3c0add00 legacy=0x7fdc3c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c0ca9f0 0x7fdc28003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28003f80 0x7fdc28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc28004d90 0x7fdc28005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c06d3b0 con 0x7fdc28004d90 2023-07-13T22:29:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0ae0d0 con 0x7fdc28003f80 2023-07-13T22:29:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2800fe20 con 0x7fdc3c0ca9f0 2023-07-13T22:29:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.366+0000 7fdc4259c700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc28004d90 0x7fdc28005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc4159a700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28003f80 0x7fdc28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc41d9b700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c0ca9f0 0x7fdc28003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc4259c700 1 -- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c0ca9f0 msgr2=0x7fdc28003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc4259c700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c0ca9f0 0x7fdc28003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc4259c700 1 -- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28003f80 msgr2=0x7fdc28004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc4259c700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28003f80 0x7fdc28004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc4259c700 1 -- 172.21.15.60:0/2836842827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280035b0 con 0x7fdc28004d90 2023-07-13T22:29:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc41d9b700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c0ca9f0 0x7fdc28003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc4259c700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc28004d90 0x7fdc28005200 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009fc0 tx=0x7fdc28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10db4dd8983a9d06 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400ec30 con 0x7fdc28004d90 2023-07-13T22:29:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3400f040 con 0x7fdc28004d90 2023-07-13T22:29:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.367+0000 7fdc40d99700 1 -- 172.21.15.60:0/2836842827 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34014480 con 0x7fdc28004d90 2023-07-13T22:29:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.368+0000 7fdc43fff700 1 -- 172.21.15.60:0/2836842827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc28004d90 msgr2=0x7fdc28005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.368+0000 7fdc43fff700 1 --2- 172.21.15.60:0/2836842827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc28004d90 0x7fdc28005200 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009fc0 tx=0x7fdc28005880 comp rx=0 tx=0).stop 2023-07-13T22:29:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.368+0000 7fdc4259c700 1 -- 172.21.15.60:0/2836842827 reap_dead start 2023-07-13T22:29:30.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.368+0000 7fdc43fff700 1 -- 172.21.15.60:0/2836842827 shutdown_connections 2023-07-13T22:29:30.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.368+0000 7fdc43fff700 1 -- 172.21.15.60:0/2836842827 >> 172.21.15.60:0/2836842827 conn(0x7fdc3c0cbf50 msgr2=0x7fdc3c0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:30.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.368+0000 7fdc43fff700 1 -- 172.21.15.60:0/2836842827 shutdown_connections 2023-07-13T22:29:30.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.368+0000 7fdc43fff700 1 -- 172.21.15.60:0/2836842827 wait complete. 2023-07-13T22:29:30.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.370+0000 7fdc43fff700 1 Processor -- start 2023-07-13T22:29:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.370+0000 7fdc43fff700 1 -- start start 2023-07-13T22:29:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc43fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28004d90 0x7fdc3c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc43fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3c0c7990 0x7fdc3c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc43fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c155eb0 0x7fdc3c156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc43fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c15c820 con 0x7fdc3c0c7990 2023-07-13T22:29:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc43fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c156860 con 0x7fdc28004d90 2023-07-13T22:29:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc43fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1ef340 con 0x7fdc3c155eb0 2023-07-13T22:29:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc4159a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3c0c7990 0x7fdc3c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc4159a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3c0c7990 0x7fdc3c15a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33004/0 (socket says 172.21.15.60:33004) 2023-07-13T22:29:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc4159a700 1 -- 172.21.15.60:0/4247559605 learned_addr learned my addr 172.21.15.60:0/4247559605 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc4259c700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c155eb0 0x7fdc3c156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.371+0000 7fdc41d9b700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28004d90 0x7fdc3c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc4159a700 1 -- 172.21.15.60:0/4247559605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c155eb0 msgr2=0x7fdc3c156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc4159a700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c155eb0 0x7fdc3c156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc4159a700 1 -- 172.21.15.60:0/4247559605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28004d90 msgr2=0x7fdc3c150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc4159a700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28004d90 0x7fdc3c150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc4159a700 1 -- 172.21.15.60:0/4247559605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c15b930 con 0x7fdc3c0c7990 2023-07-13T22:29:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc41d9b700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28004d90 0x7fdc3c150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc4159a700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3c0c7990 0x7fdc3c15a990 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c011b00 tx=0x7fdc2c011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc327fc700 1 -- 172.21.15.60:0/4247559605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c00fa00 con 0x7fdc3c0c7990 2023-07-13T22:29:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc327fc700 1 -- 172.21.15.60:0/4247559605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c00fb60 con 0x7fdc3c0c7990 2023-07-13T22:29:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc327fc700 1 -- 172.21.15.60:0/4247559605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c014440 con 0x7fdc3c0c7990 2023-07-13T22:29:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c15b090 con 0x7fdc3c0c7990 2023-07-13T22:29:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.372+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c15b650 con 0x7fdc3c0c7990 2023-07-13T22:29:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.373+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c04e210 con 0x7fdc3c0c7990 2023-07-13T22:29:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.374+0000 7fdc327fc700 1 -- 172.21.15.60:0/4247559605 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc2c0145a0 con 0x7fdc3c0c7990 2023-07-13T22:29:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.375+0000 7fdc327fc700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc2006ace0 0x7fdc2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.375+0000 7fdc327fc700 1 -- 172.21.15.60:0/4247559605 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc2c089bf0 con 0x7fdc3c0c7990 2023-07-13T22:29:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.375+0000 7fdc41d9b700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc2006ace0 0x7fdc2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.376+0000 7fdc41d9b700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc2006ace0 0x7fdc2006d190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdc380017d0 tx=0x7fdc38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.380+0000 7fdc327fc700 1 -- 172.21.15.60:0/4247559605 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdc2c054c90 con 0x7fdc3c0c7990 2023-07-13T22:29:30.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.589+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc3c158ad0 con 0x7fdc3c0c7990 2023-07-13T22:29:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.590+0000 7fdc327fc700 1 -- 172.21.15.60:0/4247559605 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc2c0220a0 con 0x7fdc3c0c7990 2023-07-13T22:29:30.569 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:29:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc2006ace0 msgr2=0x7fdc2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc2006ace0 0x7fdc2006d190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdc380017d0 tx=0x7fdc38009040 comp rx=0 tx=0).stop 2023-07-13T22:29:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3c0c7990 msgr2=0x7fdc3c15a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3c0c7990 0x7fdc3c15a990 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c011b00 tx=0x7fdc2c011e10 comp rx=0 tx=0).stop 2023-07-13T22:29:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 shutdown_connections 2023-07-13T22:29:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdc2006ace0 0x7fdc2006d190 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc3c155eb0 0x7fdc3c156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc28004d90 0x7fdc3c150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 --2- 172.21.15.60:0/4247559605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3c0c7990 0x7fdc3c15a990 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 >> 172.21.15.60:0/4247559605 conn(0x7fdc3c0cbf50 msgr2=0x7fdc3c069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 shutdown_connections 2023-07-13T22:29:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.592+0000 7fdc43fff700 1 -- 172.21.15.60:0/4247559605 wait complete. 2023-07-13T22:29:30.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:29:30.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.818+0000 7f5b2b1ff700 1 Processor -- start 2023-07-13T22:29:30.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.818+0000 7f5b2b1ff700 1 -- start start 2023-07-13T22:29:30.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.819+0000 7f5b2b1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b2415c460 0x7f5b24070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.819+0000 7f5b2b1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5b240ce770 0x7f5b240b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.819+0000 7f5b2b1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2406bf40 con 0x7f5b240caf80 2023-07-13T22:29:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.819+0000 7f5b2b1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2406b9f0 con 0x7f5b240ce770 2023-07-13T22:29:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.819+0000 7f5b2b1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2406c750 con 0x7f5b2415c460 2023-07-13T22:29:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.819+0000 7f5b2979c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5b240ce770 0x7f5b240b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.820+0000 7f5b2979c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5b240ce770 0x7f5b240b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33020/0 (socket says 172.21.15.60:33020) 2023-07-13T22:29:30.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.820+0000 7f5b2979c700 1 -- 172.21.15.60:0/3855477293 learned_addr learned my addr 172.21.15.60:0/3855477293 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:30.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.820+0000 7f5b28f9b700 1 --2- 172.21.15.60:0/3855477293 >> v2:172.21.15.105:3300/0 conn(0x7f5b2415c460 0x7f5b24070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.820+0000 7f5b2979c700 1 -- 172.21.15.60:0/3855477293 >> v2:172.21.15.105:3300/0 conn(0x7f5b2415c460 msgr2=0x7f5b24070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.820+0000 7f5b2979c700 1 --2- 172.21.15.60:0/3855477293 >> v2:172.21.15.105:3300/0 conn(0x7f5b2415c460 0x7f5b24070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.820+0000 7f5b2979c700 1 -- 172.21.15.60:0/3855477293 >> v1:172.21.15.105:6789/0 conn(0x7f5b240caf80 legacy=0x7f5b2406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.820+0000 7f5b2979c700 1 -- 172.21.15.60:0/3855477293 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b240cb3f0 con 0x7f5b240ce770 2023-07-13T22:29:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.820+0000 7f5b2979c700 1 --2- 172.21.15.60:0/3855477293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 0x7f5b240b23b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f5b24072360 tx=0x7f5b1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11ff987eff12a21 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.820+0000 7f5b237fe700 1 -- 172.21.15.60:0/3855477293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00e070 con 0x7f5b240ce770 2023-07-13T22:29:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b237fe700 1 -- 172.21.15.60:0/3855477293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c00b910 con 0x7f5b240ce770 2023-07-13T22:29:30.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b237fe700 1 -- 172.21.15.60:0/3855477293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00bcc0 con 0x7f5b240ce770 2023-07-13T22:29:30.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/3855477293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 msgr2=0x7f5b240b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b2b1ff700 1 --2- 172.21.15.60:0/3855477293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 0x7f5b240b23b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f5b24072360 tx=0x7f5b1c00b330 comp rx=0 tx=0).stop 2023-07-13T22:29:30.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/3855477293 shutdown_connections 2023-07-13T22:29:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b2b1ff700 1 --2- 172.21.15.60:0/3855477293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 0x7f5b240b23b0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b2b1ff700 1 --2- 172.21.15.60:0/3855477293 >> v2:172.21.15.105:3300/0 conn(0x7f5b2415c460 0x7f5b24070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/3855477293 >> 172.21.15.60:0/3855477293 conn(0x7f5b2406e580 msgr2=0x7f5b2406e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/3855477293 shutdown_connections 2023-07-13T22:29:30.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.821+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/3855477293 wait complete. 2023-07-13T22:29:30.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.823+0000 7f5b2b1ff700 1 Processor -- start 2023-07-13T22:29:30.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.823+0000 7f5b2b1ff700 1 -- start start 2023-07-13T22:29:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.824+0000 7f5b2b1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b240caf80 0x7f5b240b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.824+0000 7f5b2b1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 0x7f5b240b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.824+0000 7f5b2b1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2415c460 0x7f5b240b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.824+0000 7f5b2b1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2415bb70 con 0x7f5b240ce770 2023-07-13T22:29:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.824+0000 7f5b2b1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b2940 con 0x7f5b2415c460 2023-07-13T22:29:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.824+0000 7f5b2b1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b57c0 con 0x7f5b240caf80 2023-07-13T22:29:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.824+0000 7f5b23fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 0x7f5b240b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.824+0000 7f5b23fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 0x7f5b240b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33030/0 (socket says 172.21.15.60:33030) 2023-07-13T22:29:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.824+0000 7f5b23fff700 1 -- 172.21.15.60:0/221651180 learned_addr learned my addr 172.21.15.60:0/221651180 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b28f9b700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b240caf80 0x7f5b240b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b2979c700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2415c460 0x7f5b240b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b23fff700 1 -- 172.21.15.60:0/221651180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b240caf80 msgr2=0x7f5b240b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b23fff700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b240caf80 0x7f5b240b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b23fff700 1 -- 172.21.15.60:0/221651180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2415c460 msgr2=0x7f5b240b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b23fff700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2415c460 0x7f5b240b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b23fff700 1 -- 172.21.15.60:0/221651180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c009b30 con 0x7f5b240ce770 2023-07-13T22:29:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b23fff700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 0x7f5b240b0410 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c450 tx=0x7f5b1000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b217fa700 1 -- 172.21.15.60:0/221651180 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b100052d0 con 0x7f5b240ce770 2023-07-13T22:29:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b217fa700 1 -- 172.21.15.60:0/221651180 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b10005430 con 0x7f5b240ce770 2023-07-13T22:29:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.825+0000 7f5b217fa700 1 -- 172.21.15.60:0/221651180 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b100038c0 con 0x7f5b240ce770 2023-07-13T22:29:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.826+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b240b5aa0 con 0x7f5b240ce770 2023-07-13T22:29:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.826+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b240b6060 con 0x7f5b240ce770 2023-07-13T22:29:30.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.827+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b240afcf0 con 0x7f5b240ce770 2023-07-13T22:29:30.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.828+0000 7f5b217fa700 1 -- 172.21.15.60:0/221651180 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5b10003a20 con 0x7f5b240ce770 2023-07-13T22:29:30.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.829+0000 7f5b217fa700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5b0806ace0 0x7f5b0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:30.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.830+0000 7f5b217fa700 1 -- 172.21.15.60:0/221651180 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b100887b0 con 0x7f5b240ce770 2023-07-13T22:29:30.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.830+0000 7f5b28f9b700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5b0806ace0 0x7f5b0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:30.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.830+0000 7f5b28f9b700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5b0806ace0 0x7f5b0806d190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5b240c7010 tx=0x7f5b18009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:30.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:30.834+0000 7f5b217fa700 1 -- 172.21.15.60:0/221651180 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5b10053850 con 0x7f5b240ce770 2023-07-13T22:29:31.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.039+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b240d0480 con 0x7f5b240ce770 2023-07-13T22:29:31.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.040+0000 7f5b217fa700 1 -- 172.21.15.60:0/221651180 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b1000ce60 con 0x7f5b240ce770 2023-07-13T22:29:31.019 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:29:31.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5b0806ace0 msgr2=0x7f5b0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5b0806ace0 0x7f5b0806d190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5b240c7010 tx=0x7f5b18009210 comp rx=0 tx=0).stop 2023-07-13T22:29:31.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 msgr2=0x7f5b240b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 0x7f5b240b0410 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c450 tx=0x7f5b1000c760 comp rx=0 tx=0).stop 2023-07-13T22:29:31.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 shutdown_connections 2023-07-13T22:29:31.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5b0806ace0 0x7f5b0806d190 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2415c460 0x7f5b240b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b240caf80 0x7f5b240b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 --2- 172.21.15.60:0/221651180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b240ce770 0x7f5b240b0410 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 >> 172.21.15.60:0/221651180 conn(0x7f5b2406e580 msgr2=0x7f5b2406e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 shutdown_connections 2023-07-13T22:29:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.042+0000 7f5b2b1ff700 1 -- 172.21.15.60:0/221651180 wait complete. 2023-07-13T22:29:31.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:29:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.267+0000 7efeff1ff700 1 Processor -- start 2023-07-13T22:29:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.268+0000 7efeff1ff700 1 -- start start 2023-07-13T22:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efeff1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efef8159260 0x7efef80ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efeff1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80cc410 0x7efef80cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efeff1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80cf640 con 0x7efef80703e0 2023-07-13T22:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efeff1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef80ccf80 con 0x7efef8159260 2023-07-13T22:29:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efeff1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef80c9b40 con 0x7efef80cc410 2023-07-13T22:29:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efef7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80cc410 0x7efef80cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efefcf9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efef8159260 0x7efef80ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efef7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80cc410 0x7efef80cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:60150/0 (socket says 172.21.15.60:60150) 2023-07-13T22:29:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efefd79c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efef80703e0 0x7efef806c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:34906/0 (socket says 172.21.15.60:34906) 2023-07-13T22:29:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.269+0000 7efef7fff700 1 -- 172.21.15.60:0/2783920727 learned_addr learned my addr 172.21.15.60:0/2783920727 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.270+0000 7efef7fff700 1 -- 172.21.15.60:0/2783920727 >> v2:172.21.15.105:3300/0 conn(0x7efef8159260 msgr2=0x7efef80ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.270+0000 7efef7fff700 1 --2- 172.21.15.60:0/2783920727 >> v2:172.21.15.105:3300/0 conn(0x7efef8159260 0x7efef80ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.270+0000 7efef7fff700 1 -- 172.21.15.60:0/2783920727 >> v1:172.21.15.105:6789/0 conn(0x7efef80703e0 legacy=0x7efef806c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.270+0000 7efef7fff700 1 -- 172.21.15.60:0/2783920727 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80c9e20 con 0x7efef80cc410 2023-07-13T22:29:31.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.270+0000 7efef7fff700 1 --2- 172.21.15.60:0/2783920727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80cc410 0x7efef80cc880 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efee40098f0 tx=0x7efee4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3756ed657ac40b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:31.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.270+0000 7efef77fe700 1 -- 172.21.15.60:0/2783920727 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4011830 con 0x7efef80cc410 2023-07-13T22:29:31.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.270+0000 7efef77fe700 1 -- 172.21.15.60:0/2783920727 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee4011990 con 0x7efef80cc410 2023-07-13T22:29:31.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.271+0000 7efef77fe700 1 -- 172.21.15.60:0/2783920727 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4011d20 con 0x7efef80cc410 2023-07-13T22:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.271+0000 7efeff1ff700 1 -- 172.21.15.60:0/2783920727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80cc410 msgr2=0x7efef80cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.271+0000 7efeff1ff700 1 --2- 172.21.15.60:0/2783920727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80cc410 0x7efef80cc880 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efee40098f0 tx=0x7efee4009c00 comp rx=0 tx=0).stop 2023-07-13T22:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.271+0000 7efeff1ff700 1 -- 172.21.15.60:0/2783920727 shutdown_connections 2023-07-13T22:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.271+0000 7efeff1ff700 1 --2- 172.21.15.60:0/2783920727 >> v2:172.21.15.105:3300/0 conn(0x7efef8159260 0x7efef80ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.271+0000 7efeff1ff700 1 --2- 172.21.15.60:0/2783920727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80cc410 0x7efef80cc880 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.271+0000 7efeff1ff700 1 -- 172.21.15.60:0/2783920727 >> 172.21.15.60:0/2783920727 conn(0x7efef80cbf50 msgr2=0x7efef80ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.271+0000 7efeff1ff700 1 -- 172.21.15.60:0/2783920727 shutdown_connections 2023-07-13T22:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.271+0000 7efeff1ff700 1 -- 172.21.15.60:0/2783920727 wait complete. 2023-07-13T22:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.274+0000 7efeff1ff700 1 Processor -- start 2023-07-13T22:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.274+0000 7efeff1ff700 1 -- start start 2023-07-13T22:29:31.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efeff1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80703e0 0x7efef8154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efeff1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef80cc410 0x7efef8154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efeff1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef8159260 0x7efef81f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efeff1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef81e2750 con 0x7efef8159260 2023-07-13T22:29:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efeff1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80c6850 con 0x7efef80cc410 2023-07-13T22:29:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efeff1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef80c6b30 con 0x7efef80703e0 2023-07-13T22:29:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efefd79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef8159260 0x7efef81f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efefd79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef8159260 0x7efef81f28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33042/0 (socket says 172.21.15.60:33042) 2023-07-13T22:29:31.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efefd79c700 1 -- 172.21.15.60:0/765866136 learned_addr learned my addr 172.21.15.60:0/765866136 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:31.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.275+0000 7efefcf9b700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80703e0 0x7efef8154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efef7fff700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef80cc410 0x7efef8154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efefd79c700 1 -- 172.21.15.60:0/765866136 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80703e0 msgr2=0x7efef8154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efefd79c700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80703e0 0x7efef8154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efefd79c700 1 -- 172.21.15.60:0/765866136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef80cc410 msgr2=0x7efef8154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efefd79c700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef80cc410 0x7efef8154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efefd79c700 1 -- 172.21.15.60:0/765866136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4009550 con 0x7efef8159260 2023-07-13T22:29:31.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efefd79c700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef8159260 0x7efef81f28d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7efef000cbd0 tx=0x7efef000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:31.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efef57fa700 1 -- 172.21.15.60:0/765866136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef000aaa0 con 0x7efef8159260 2023-07-13T22:29:31.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efef57fa700 1 -- 172.21.15.60:0/765866136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef000ac00 con 0x7efef8159260 2023-07-13T22:29:31.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef80c6e10 con 0x7efef8159260 2023-07-13T22:29:31.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.276+0000 7efef57fa700 1 -- 172.21.15.60:0/765866136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0007070 con 0x7efef8159260 2023-07-13T22:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.277+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef80b10b0 con 0x7efef8159260 2023-07-13T22:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.278+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef80ae160 con 0x7efef8159260 2023-07-13T22:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.279+0000 7efef57fa700 1 -- 172.21.15.60:0/765866136 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efef00071d0 con 0x7efef8159260 2023-07-13T22:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.280+0000 7efef57fa700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efedc06ace0 0x7efedc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.280+0000 7efef57fa700 1 -- 172.21.15.60:0/765866136 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efef0089120 con 0x7efef8159260 2023-07-13T22:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.280+0000 7efefcf9b700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efedc06ace0 0x7efedc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.281+0000 7efefcf9b700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efedc06ace0 0x7efedc06d190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efeec009b40 tx=0x7efeec009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:31.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.284+0000 7efef57fa700 1 -- 172.21.15.60:0/765866136 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efef0054270 con 0x7efef8159260 2023-07-13T22:29:31.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.489+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efef804e210 con 0x7efef8159260 2023-07-13T22:29:31.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.490+0000 7efef57fa700 1 -- 172.21.15.60:0/765866136 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7efef00170a0 con 0x7efef8159260 2023-07-13T22:29:31.469 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:29:31.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efedc06ace0 msgr2=0x7efedc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efedc06ace0 0x7efedc06d190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efeec009b40 tx=0x7efeec009210 comp rx=0 tx=0).stop 2023-07-13T22:29:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef8159260 msgr2=0x7efef81f28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef8159260 0x7efef81f28d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7efef000cbd0 tx=0x7efef000cee0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 shutdown_connections 2023-07-13T22:29:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efedc06ace0 0x7efedc06d190 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef8159260 0x7efef81f28d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efef80703e0 0x7efef8154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 --2- 172.21.15.60:0/765866136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efef80cc410 0x7efef8154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 >> 172.21.15.60:0/765866136 conn(0x7efef80cbf50 msgr2=0x7efef8158530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 shutdown_connections 2023-07-13T22:29:31.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.492+0000 7efeff1ff700 1 -- 172.21.15.60:0/765866136 wait complete. 2023-07-13T22:29:31.490 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:31.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.714+0000 7efdeb3ff700 1 Processor -- start 2023-07-13T22:29:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.714+0000 7efdeb3ff700 1 -- start start 2023-07-13T22:29:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.715+0000 7efdeb3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efde40b8570 0x7efde40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.715+0000 7efdeb3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40d15c0 0x7efde415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.715+0000 7efdeb3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde40d09d0 con 0x7efde40d35c0 2023-07-13T22:29:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.715+0000 7efdeb3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde40d3b30 con 0x7efde40b8570 2023-07-13T22:29:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.715+0000 7efdeb3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde40cda10 con 0x7efde40d15c0 2023-07-13T22:29:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.716+0000 7efde919b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efde40b8570 0x7efde40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.716+0000 7efde999c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40d15c0 0x7efde415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.716+0000 7efde999c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40d15c0 0x7efde415c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:60160/0 (socket says 172.21.15.60:60160) 2023-07-13T22:29:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.716+0000 7efde999c700 1 -- 172.21.15.60:0/4263612504 learned_addr learned my addr 172.21.15.60:0/4263612504 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.717+0000 7efde999c700 1 -- 172.21.15.60:0/4263612504 >> v2:172.21.15.105:3300/0 conn(0x7efde40b8570 msgr2=0x7efde40ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.717+0000 7efde999c700 1 --2- 172.21.15.60:0/4263612504 >> v2:172.21.15.105:3300/0 conn(0x7efde40b8570 0x7efde40ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.717+0000 7efde999c700 1 -- 172.21.15.60:0/4263612504 >> v1:172.21.15.105:6789/0 conn(0x7efde40d35c0 legacy=0x7efde40d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.717+0000 7efde999c700 1 -- 172.21.15.60:0/4263612504 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde40cd450 con 0x7efde40d15c0 2023-07-13T22:29:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.717+0000 7efde919b700 1 --2- 172.21.15.60:0/4263612504 >> v2:172.21.15.105:3300/0 conn(0x7efde40b8570 0x7efde40ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.717+0000 7efde999c700 1 --2- 172.21.15.60:0/4263612504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40d15c0 0x7efde415c920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efddc009e50 tx=0x7efddc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f7bf331074702c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.717+0000 7efddbfff700 1 -- 172.21.15.60:0/4263612504 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc00e070 con 0x7efde40d15c0 2023-07-13T22:29:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.717+0000 7efddbfff700 1 -- 172.21.15.60:0/4263612504 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc00b910 con 0x7efde40d15c0 2023-07-13T22:29:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.717+0000 7efddbfff700 1 -- 172.21.15.60:0/4263612504 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc00bcc0 con 0x7efde40d15c0 2023-07-13T22:29:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.718+0000 7efdeb3ff700 1 -- 172.21.15.60:0/4263612504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40d15c0 msgr2=0x7efde415c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.718+0000 7efdeb3ff700 1 --2- 172.21.15.60:0/4263612504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40d15c0 0x7efde415c920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efddc009e50 tx=0x7efddc00b330 comp rx=0 tx=0).stop 2023-07-13T22:29:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.718+0000 7efdeb3ff700 1 -- 172.21.15.60:0/4263612504 shutdown_connections 2023-07-13T22:29:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.718+0000 7efdeb3ff700 1 --2- 172.21.15.60:0/4263612504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40d15c0 0x7efde415c920 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.718+0000 7efdeb3ff700 1 --2- 172.21.15.60:0/4263612504 >> v2:172.21.15.105:3300/0 conn(0x7efde40b8570 0x7efde40ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.718+0000 7efdeb3ff700 1 -- 172.21.15.60:0/4263612504 >> 172.21.15.60:0/4263612504 conn(0x7efde4071600 msgr2=0x7efde40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.718+0000 7efdeb3ff700 1 -- 172.21.15.60:0/4263612504 shutdown_connections 2023-07-13T22:29:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.718+0000 7efdeb3ff700 1 -- 172.21.15.60:0/4263612504 wait complete. 2023-07-13T22:29:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.720+0000 7efdeb3ff700 1 Processor -- start 2023-07-13T22:29:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.720+0000 7efdeb3ff700 1 -- start start 2023-07-13T22:29:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efdeb3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40b8570 0x7efde414ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efdeb3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efde40d15c0 0x7efde41f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efdeb3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde40d35c0 0x7efde40cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efdeb3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde41e55a0 con 0x7efde40d35c0 2023-07-13T22:29:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efdeb3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde41f4fb0 con 0x7efde40d15c0 2023-07-13T22:29:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efdeb3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde40cc9d0 con 0x7efde40b8570 2023-07-13T22:29:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efde999c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde40d35c0 0x7efde40cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efde999c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde40d35c0 0x7efde40cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33058/0 (socket says 172.21.15.60:33058) 2023-07-13T22:29:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efde999c700 1 -- 172.21.15.60:0/3720361937 learned_addr learned my addr 172.21.15.60:0/3720361937 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efde919b700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40b8570 0x7efde414ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.721+0000 7efde899a700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efde40d15c0 0x7efde41f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efde999c700 1 -- 172.21.15.60:0/3720361937 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40b8570 msgr2=0x7efde414ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efde999c700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40b8570 0x7efde414ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efde999c700 1 -- 172.21.15.60:0/3720361937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efde40d15c0 msgr2=0x7efde41f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efde999c700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efde40d15c0 0x7efde41f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efde999c700 1 -- 172.21.15.60:0/3720361937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde0009430 con 0x7efde40d35c0 2023-07-13T22:29:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efde999c700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde40d35c0 0x7efde40cc490 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7efddc012000 tx=0x7efddc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efdd9ffb700 1 -- 172.21.15.60:0/3720361937 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc004800 con 0x7efde40d35c0 2023-07-13T22:29:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efdd9ffb700 1 -- 172.21.15.60:0/3720361937 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc004960 con 0x7efde40d35c0 2023-07-13T22:29:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efdd9ffb700 1 -- 172.21.15.60:0/3720361937 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc01c680 con 0x7efde40d35c0 2023-07-13T22:29:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc009b30 con 0x7efde40d35c0 2023-07-13T22:29:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.722+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde414b5a0 con 0x7efde40d35c0 2023-07-13T22:29:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.723+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde404e280 con 0x7efde40d35c0 2023-07-13T22:29:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.724+0000 7efdd9ffb700 1 -- 172.21.15.60:0/3720361937 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efddc01c7e0 con 0x7efde40d35c0 2023-07-13T22:29:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.725+0000 7efdd9ffb700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efdc806ace0 0x7efdc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.725+0000 7efdd9ffb700 1 -- 172.21.15.60:0/3720361937 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efddc08afe0 con 0x7efde40d35c0 2023-07-13T22:29:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.726+0000 7efde919b700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efdc806ace0 0x7efdc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.726+0000 7efde919b700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efdc806ace0 0x7efdc806d190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efde0009d50 tx=0x7efde00091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.730+0000 7efdd9ffb700 1 -- 172.21.15.60:0/3720361937 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efddc0560e0 con 0x7efde40d35c0 2023-07-13T22:29:31.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.980+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efde40c6510 con 0x7efde40d35c0 2023-07-13T22:29:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.981+0000 7efdd9ffb700 1 -- 172.21.15.60:0/3720361937 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4715 (secure 0 0 0) 0x7efddc059700 con 0x7efde40d35c0 2023-07-13T22:29:31.960 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:31.961 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:58.254590+0000","modified":"2023-07-13T22:29:31.689264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6837/4078186433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":4078186433},{"type":"v1","addr":"172.21.15.60:6837","nonce":4078186433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-13T22:29:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efdc806ace0 msgr2=0x7efdc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efdc806ace0 0x7efdc806d190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efde0009d50 tx=0x7efde00091c0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde40d35c0 msgr2=0x7efde40cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde40d35c0 0x7efde40cc490 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7efddc012000 tx=0x7efddc004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 shutdown_connections 2023-07-13T22:29:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efdc806ace0 0x7efdc806d190 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde40d35c0 0x7efde40cc490 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efde40b8570 0x7efde414ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 --2- 172.21.15.60:0/3720361937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efde40d15c0 0x7efde41f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 >> 172.21.15.60:0/3720361937 conn(0x7efde4071600 msgr2=0x7efde40732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:31.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.983+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 shutdown_connections 2023-07-13T22:29:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:31.984+0000 7efdeb3ff700 1 -- 172.21.15.60:0/3720361937 wait complete. 2023-07-13T22:29:31.965 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 65 2023-07-13T22:29:31.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:32.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.204+0000 7f88d23ff700 1 Processor -- start 2023-07-13T22:29:32.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.205+0000 7f88d23ff700 1 -- start start 2023-07-13T22:29:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.206+0000 7f88d23ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f88cc0b8400 0x7f88cc0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.206+0000 7f88d23ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88cc0d0ac0 con 0x7f88cc0cfca0 2023-07-13T22:29:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.206+0000 7f88d23ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88cc0b87d0 con 0x7f88cc0d3750 2023-07-13T22:29:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.206+0000 7f88d23ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88cc0b4eb0 con 0x7f88cc0b8400 2023-07-13T22:29:32.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.206+0000 7f88cbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f88cc0b8400 0x7f88cc0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.206+0000 7f88d099c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f88cc0cfca0 0x7f88cc0d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49872/0 (socket says 172.21.15.60:49872) 2023-07-13T22:29:32.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.206+0000 7f88d099c700 1 -- 172.21.15.60:0/1585723312 learned_addr learned my addr 172.21.15.60:0/1585723312 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:32.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.207+0000 7f88cbfff700 1 -- 172.21.15.60:0/1585723312 >> v1:172.21.15.105:6789/0 conn(0x7f88cc0d3750 legacy=0x7f88cc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.207+0000 7f88cbfff700 1 -- 172.21.15.60:0/1585723312 >> v1:172.21.15.60:6789/0 conn(0x7f88cc0cfca0 legacy=0x7f88cc0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.207+0000 7f88cbfff700 1 -- 172.21.15.60:0/1585723312 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc0b5190 con 0x7f88cc0b8400 2023-07-13T22:29:32.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.207+0000 7f88caffd700 1 -- 172.21.15.60:0/1585723312 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176585890 0 0) 0x7f88cc0d0ac0 con 0x7f88cc0cfca0 2023-07-13T22:29:32.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.207+0000 7f88cbfff700 1 --2- 172.21.15.60:0/1585723312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 0x7f88cc0cd240 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009840 tx=0x7f88bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e635d2a0166c325 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:32.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.207+0000 7f88caffd700 1 -- 172.21.15.60:0/1585723312 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc011830 con 0x7f88cc0b8400 2023-07-13T22:29:32.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.207+0000 7f88caffd700 1 -- 172.21.15.60:0/1585723312 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc011990 con 0x7f88cc0b8400 2023-07-13T22:29:32.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.207+0000 7f88caffd700 1 -- 172.21.15.60:0/1585723312 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc011d20 con 0x7f88cc0b8400 2023-07-13T22:29:32.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.208+0000 7f88d23ff700 1 -- 172.21.15.60:0/1585723312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 msgr2=0x7f88cc0cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.208+0000 7f88d23ff700 1 --2- 172.21.15.60:0/1585723312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 0x7f88cc0cd240 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009840 tx=0x7f88bc009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:32.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.208+0000 7f88d23ff700 1 -- 172.21.15.60:0/1585723312 shutdown_connections 2023-07-13T22:29:32.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.208+0000 7f88d23ff700 1 --2- 172.21.15.60:0/1585723312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 0x7f88cc0cd240 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.208+0000 7f88d23ff700 1 -- 172.21.15.60:0/1585723312 >> 172.21.15.60:0/1585723312 conn(0x7f88cc06b390 msgr2=0x7f88cc06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:32.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.208+0000 7f88d23ff700 1 -- 172.21.15.60:0/1585723312 shutdown_connections 2023-07-13T22:29:32.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.208+0000 7f88d23ff700 1 -- 172.21.15.60:0/1585723312 wait complete. 2023-07-13T22:29:32.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.209+0000 7f88d23ff700 1 Processor -- start 2023-07-13T22:29:32.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.210+0000 7f88d23ff700 1 -- start start 2023-07-13T22:29:32.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.210+0000 7f88d23ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 0x7f88cc1f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.210+0000 7f88d23ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88cc0cfca0 0x7f88cc1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.210+0000 7f88d23ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88cc0d3750 0x7f88cc158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.210+0000 7f88d23ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1f2310 con 0x7f88cc0b8400 2023-07-13T22:29:32.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.210+0000 7f88d23ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1f6380 con 0x7f88cc0cfca0 2023-07-13T22:29:32.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.210+0000 7f88d23ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc0c9df0 con 0x7f88cc0d3750 2023-07-13T22:29:32.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 0x7f88cc1f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 0x7f88cc1f1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33088/0 (socket says 172.21.15.60:33088) 2023-07-13T22:29:32.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cbfff700 1 -- 172.21.15.60:0/3813016062 learned_addr learned my addr 172.21.15.60:0/3813016062 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:32.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cb7fe700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88cc0cfca0 0x7f88cc1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88d099c700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88cc0d3750 0x7f88cc158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cbfff700 1 -- 172.21.15.60:0/3813016062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88cc0d3750 msgr2=0x7f88cc158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cbfff700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88cc0d3750 0x7f88cc158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cbfff700 1 -- 172.21.15.60:0/3813016062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88cc0cfca0 msgr2=0x7f88cc1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cbfff700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88cc0cfca0 0x7f88cc1f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cbfff700 1 -- 172.21.15.60:0/3813016062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0094a0 con 0x7f88cc0b8400 2023-07-13T22:29:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cb7fe700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88cc0cfca0 0x7f88cc1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.211+0000 7f88cbfff700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 0x7f88cc1f1dd0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f88bc014040 tx=0x7f88bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.212+0000 7f88c8ff9700 1 -- 172.21.15.60:0/3813016062 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc01a9c0 con 0x7f88cc0b8400 2023-07-13T22:29:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.212+0000 7f88c8ff9700 1 -- 172.21.15.60:0/3813016062 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc01ab20 con 0x7f88cc0b8400 2023-07-13T22:29:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.212+0000 7f88c8ff9700 1 -- 172.21.15.60:0/3813016062 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc024420 con 0x7f88cc0b8400 2023-07-13T22:29:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.212+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc1f2500 con 0x7f88cc0b8400 2023-07-13T22:29:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.212+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc1f2ac0 con 0x7f88cc0b8400 2023-07-13T22:29:32.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.213+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc0b1f30 con 0x7f88cc0b8400 2023-07-13T22:29:32.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.214+0000 7f88c8ff9700 1 -- 172.21.15.60:0/3813016062 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f88bc022070 con 0x7f88cc0b8400 2023-07-13T22:29:32.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.215+0000 7f88c8ff9700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88b806ace0 0x7f88b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.215+0000 7f88c8ff9700 1 -- 172.21.15.60:0/3813016062 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f88bc08a520 con 0x7f88cc0b8400 2023-07-13T22:29:32.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.215+0000 7f88cb7fe700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88b806ace0 0x7f88b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.216+0000 7f88cb7fe700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88b806ace0 0x7f88b806d190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0c6940 tx=0x7f88c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:32.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.220+0000 7f88c8ff9700 1 -- 172.21.15.60:0/3813016062 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f88bc0555c0 con 0x7f88cc0b8400 2023-07-13T22:29:32.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.430+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88cc1f2cb0 con 0x7f88cc0b8400 2023-07-13T22:29:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.432+0000 7f88c8ff9700 1 -- 172.21.15.60:0/3813016062 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+18190 (secure 0 0 0) 0x7f88bc006cc0 con 0x7f88cc0b8400 2023-07-13T22:29:32.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:32.412 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":100,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:59.528113+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:56.533994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:57.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:32.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88b806ace0 msgr2=0x7f88b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88b806ace0 0x7f88b806d190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0c6940 tx=0x7f88c0009040 comp rx=0 tx=0).stop 2023-07-13T22:29:32.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 msgr2=0x7f88cc1f1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 0x7f88cc1f1dd0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f88bc014040 tx=0x7f88bc004750 comp rx=0 tx=0).stop 2023-07-13T22:29:32.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 shutdown_connections 2023-07-13T22:29:32.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f88b806ace0 0x7f88b806d190 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88cc0b8400 0x7f88cc1f1dd0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88cc0d3750 0x7f88cc158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 --2- 172.21.15.60:0/3813016062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88cc0cfca0 0x7f88cc1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 >> 172.21.15.60:0/3813016062 conn(0x7f88cc06b390 msgr2=0x7f88cc0ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:32.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 shutdown_connections 2023-07-13T22:29:32.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.434+0000 7f88d23ff700 1 -- 172.21.15.60:0/3813016062 wait complete. 2023-07-13T22:29:32.432 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:29:32.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.656+0000 7f584bfff700 1 Processor -- start 2023-07-13T22:29:32.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.656+0000 7f584bfff700 1 -- start start 2023-07-13T22:29:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.657+0000 7f584bfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f58440b12a0 0x7f58440cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.657+0000 7f584bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f584406bad0 0x7f58440b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.657+0000 7f584bfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58441609a0 con 0x7f584406eed0 2023-07-13T22:29:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.657+0000 7f584bfff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584406a5f0 con 0x7f584406bad0 2023-07-13T22:29:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.657+0000 7f584bfff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5844160ed0 con 0x7f58440b12a0 2023-07-13T22:29:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.657+0000 7f584959a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f584406bad0 0x7f58440b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f584959a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f584406bad0 0x7f58440b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33098/0 (socket says 172.21.15.60:33098) 2023-07-13T22:29:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f584959a700 1 -- 172.21.15.60:0/3988505462 learned_addr learned my addr 172.21.15.60:0/3988505462 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:32.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f5849d9b700 1 --2- 172.21.15.60:0/3988505462 >> v2:172.21.15.105:3300/0 conn(0x7f58440b12a0 0x7f58440cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f584959a700 1 -- 172.21.15.60:0/3988505462 >> v2:172.21.15.105:3300/0 conn(0x7f58440b12a0 msgr2=0x7f58440cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:32.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f584959a700 1 --2- 172.21.15.60:0/3988505462 >> v2:172.21.15.105:3300/0 conn(0x7f58440b12a0 0x7f58440cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f584959a700 1 -- 172.21.15.60:0/3988505462 >> v1:172.21.15.105:6789/0 conn(0x7f584406eed0 legacy=0x7f584406f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:32.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f584959a700 1 -- 172.21.15.60:0/3988505462 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584415c860 con 0x7f584406bad0 2023-07-13T22:29:32.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f584959a700 1 --2- 172.21.15.60:0/3988505462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406bad0 0x7f58440b1670 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f584406c110 tx=0x7f5834009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e797e83c2dc39509 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f5848d99700 1 -- 172.21.15.60:0/3988505462 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834004230 con 0x7f584406bad0 2023-07-13T22:29:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f5848d99700 1 -- 172.21.15.60:0/3988505462 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5834004390 con 0x7f584406bad0 2023-07-13T22:29:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.658+0000 7f5848d99700 1 -- 172.21.15.60:0/3988505462 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340036a0 con 0x7f584406bad0 2023-07-13T22:29:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.659+0000 7f584bfff700 1 -- 172.21.15.60:0/3988505462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406bad0 msgr2=0x7f58440b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.659+0000 7f584bfff700 1 --2- 172.21.15.60:0/3988505462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406bad0 0x7f58440b1670 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f584406c110 tx=0x7f5834009c00 comp rx=0 tx=0).stop 2023-07-13T22:29:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.659+0000 7f584bfff700 1 -- 172.21.15.60:0/3988505462 shutdown_connections 2023-07-13T22:29:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.659+0000 7f584bfff700 1 --2- 172.21.15.60:0/3988505462 >> v2:172.21.15.105:3300/0 conn(0x7f58440b12a0 0x7f58440cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.659+0000 7f584bfff700 1 --2- 172.21.15.60:0/3988505462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406bad0 0x7f58440b1670 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.659+0000 7f584bfff700 1 -- 172.21.15.60:0/3988505462 >> 172.21.15.60:0/3988505462 conn(0x7f58440cc2a0 msgr2=0x7f58440cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.659+0000 7f584bfff700 1 -- 172.21.15.60:0/3988505462 shutdown_connections 2023-07-13T22:29:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.659+0000 7f584bfff700 1 -- 172.21.15.60:0/3988505462 wait complete. 2023-07-13T22:29:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.662+0000 7f584bfff700 1 Processor -- start 2023-07-13T22:29:32.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.662+0000 7f584bfff700 1 -- start start 2023-07-13T22:29:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f584406bad0 0x7f58440b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584bfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f584406eed0 0x7f58440c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58440b12a0 0x7f58441f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584bfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58441e7110 con 0x7f58440b12a0 2023-07-13T22:29:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584bfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584415bd50 con 0x7f584406bad0 2023-07-13T22:29:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584bfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58440b30a0 con 0x7f584406eed0 2023-07-13T22:29:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58440b12a0 0x7f58441f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58440b12a0 0x7f58441f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33102/0 (socket says 172.21.15.60:33102) 2023-07-13T22:29:32.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584a59c700 1 -- 172.21.15.60:0/3010605870 learned_addr learned my addr 172.21.15.60:0/3010605870 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f584959a700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f584406eed0 0x7f58440c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.663+0000 7f5849d9b700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f584406bad0 0x7f58440b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f584a59c700 1 -- 172.21.15.60:0/3010605870 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f584406eed0 msgr2=0x7f58440c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f584a59c700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f584406eed0 0x7f58440c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f584a59c700 1 -- 172.21.15.60:0/3010605870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f584406bad0 msgr2=0x7f58440b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f584a59c700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f584406bad0 0x7f58440b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f584a59c700 1 -- 172.21.15.60:0/3010605870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5834009550 con 0x7f58440b12a0 2023-07-13T22:29:32.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f584a59c700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58440b12a0 0x7f58441f57d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f583c00cc10 tx=0x7f583c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f583a7fc700 1 -- 172.21.15.60:0/3010605870 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c00aa60 con 0x7f58440b12a0 2023-07-13T22:29:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f583a7fc700 1 -- 172.21.15.60:0/3010605870 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c00abc0 con 0x7f58440b12a0 2023-07-13T22:29:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f583a7fc700 1 -- 172.21.15.60:0/3010605870 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c012420 con 0x7f58440b12a0 2023-07-13T22:29:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.664+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58440b3380 con 0x7f58440b12a0 2023-07-13T22:29:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.665+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58440c5ef0 con 0x7f58440b12a0 2023-07-13T22:29:32.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.665+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58440b1b90 con 0x7f58440b12a0 2023-07-13T22:29:32.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.667+0000 7f583a7fc700 1 -- 172.21.15.60:0/3010605870 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f583c012580 con 0x7f58440b12a0 2023-07-13T22:29:32.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.667+0000 7f583a7fc700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f582806ace0 0x7f582806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:32.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.668+0000 7f583a7fc700 1 -- 172.21.15.60:0/3010605870 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f583c0897b0 con 0x7f58440b12a0 2023-07-13T22:29:32.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.668+0000 7f5849d9b700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f582806ace0 0x7f582806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:32.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.668+0000 7f5849d9b700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f582806ace0 0x7f582806d190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5840006b20 tx=0x7f5840008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:32.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.672+0000 7f583a7fc700 1 -- 172.21.15.60:0/3010605870 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f58440b1b90 con 0x7f58440b12a0 2023-07-13T22:29:32.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.922+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58440b1b90 con 0x7f58440b12a0 2023-07-13T22:29:32.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.922+0000 7f583a7fc700 1 -- 172.21.15.60:0/3010605870 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f58440b1b90 con 0x7f58440b12a0 2023-07-13T22:29:32.902 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:32.902 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:58.254590+0000","modified":"2023-07-13T22:29:31.689264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6837/4078186433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":4078186433},{"type":"v1","addr":"172.21.15.60:6837","nonce":4078186433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-13T22:29:32.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f582806ace0 msgr2=0x7f582806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f582806ace0 0x7f582806d190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5840006b20 tx=0x7f5840008040 comp rx=0 tx=0).stop 2023-07-13T22:29:32.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58440b12a0 msgr2=0x7f58441f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:32.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58440b12a0 0x7f58441f57d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f583c00cc10 tx=0x7f583c00cf20 comp rx=0 tx=0).stop 2023-07-13T22:29:32.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 shutdown_connections 2023-07-13T22:29:32.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f582806ace0 0x7f582806d190 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58440b12a0 0x7f58441f57d0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f584406bad0 0x7f58440b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 --2- 172.21.15.60:0/3010605870 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f584406eed0 0x7f58440c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 >> 172.21.15.60:0/3010605870 conn(0x7f58440cc2a0 msgr2=0x7f58440b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 shutdown_connections 2023-07-13T22:29:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:32.925+0000 7f584bfff700 1 -- 172.21.15.60:0/3010605870 wait complete. 2023-07-13T22:29:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 65 2023-07-13T22:29:32.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:33.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.146+0000 7fccf57ff700 1 Processor -- start 2023-07-13T22:29:33.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.146+0000 7fccf57ff700 1 -- start start 2023-07-13T22:29:33.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.147+0000 7fccf57ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fccf00b8400 0x7fccf00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.147+0000 7fccf57ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00d3750 0x7fccf00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.148+0000 7fccf57ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf00cff20 con 0x7fccf00cfad0 2023-07-13T22:29:33.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.148+0000 7fccf57ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf00d0ac0 con 0x7fccf00b8400 2023-07-13T22:29:33.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.148+0000 7fccf57ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00b87d0 con 0x7fccf00d3750 2023-07-13T22:29:33.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.148+0000 7fccef7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fccf00cfad0 0x7fccf015fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49884/0 (socket says 172.21.15.60:49884) 2023-07-13T22:29:33.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.148+0000 7fccef7fe700 1 -- 172.21.15.60:0/1931690636 learned_addr learned my addr 172.21.15.60:0/1931690636 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:33.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.148+0000 7fcce7fff700 1 --2- 172.21.15.60:0/1931690636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00d3750 0x7fccf00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.148+0000 7fcceeffd700 1 --2- 172.21.15.60:0/1931690636 >> v2:172.21.15.105:3300/0 conn(0x7fccf00b8400 0x7fccf00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.148+0000 7fccee7fc700 1 -- 172.21.15.60:0/1931690636 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540554438 0 0) 0x7fcce8001490 con 0x7fccf00cfad0 2023-07-13T22:29:33.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.148+0000 7fccee7fc700 1 -- 172.21.15.60:0/1931690636 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccdc0035b0 con 0x7fccf00cfad0 2023-07-13T22:29:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fcce7fff700 1 -- 172.21.15.60:0/1931690636 >> v2:172.21.15.105:3300/0 conn(0x7fccf00b8400 msgr2=0x7fccf00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fccee7fc700 1 -- 172.21.15.60:0/1931690636 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 660616788 0 0) 0x7fccdc0035b0 con 0x7fccf00cfad0 2023-07-13T22:29:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fcce7fff700 1 --2- 172.21.15.60:0/1931690636 >> v2:172.21.15.105:3300/0 conn(0x7fccf00b8400 0x7fccf00cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fcce7fff700 1 -- 172.21.15.60:0/1931690636 >> v1:172.21.15.60:6789/0 conn(0x7fccf00cfad0 legacy=0x7fccf015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fcce7fff700 1 -- 172.21.15.60:0/1931690636 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00b4eb0 con 0x7fccf00d3750 2023-07-13T22:29:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fcceeffd700 1 --2- 172.21.15.60:0/1931690636 >> v2:172.21.15.105:3300/0 conn(0x7fccf00b8400 0x7fccf00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fcce7fff700 1 --2- 172.21.15.60:0/1931690636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00d3750 0x7fccf00cf590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009840 tx=0x7fccd8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=706f61417ef85d02 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fccee7fc700 1 -- 172.21.15.60:0/1931690636 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce8001b80 con 0x7fccf00d3750 2023-07-13T22:29:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fccee7fc700 1 -- 172.21.15.60:0/1931690636 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8004030 con 0x7fccf00d3750 2023-07-13T22:29:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.149+0000 7fccee7fc700 1 -- 172.21.15.60:0/1931690636 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80043a0 con 0x7fccf00d3750 2023-07-13T22:29:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.150+0000 7fccf57ff700 1 -- 172.21.15.60:0/1931690636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00d3750 msgr2=0x7fccf00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.150+0000 7fccf57ff700 1 --2- 172.21.15.60:0/1931690636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00d3750 0x7fccf00cf590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009840 tx=0x7fccd8009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.150+0000 7fccf57ff700 1 -- 172.21.15.60:0/1931690636 shutdown_connections 2023-07-13T22:29:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.150+0000 7fccf57ff700 1 --2- 172.21.15.60:0/1931690636 >> v2:172.21.15.105:3300/0 conn(0x7fccf00b8400 0x7fccf00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.150+0000 7fccf57ff700 1 --2- 172.21.15.60:0/1931690636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00d3750 0x7fccf00cf590 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.150+0000 7fccf57ff700 1 -- 172.21.15.60:0/1931690636 >> 172.21.15.60:0/1931690636 conn(0x7fccf006b390 msgr2=0x7fccf006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.150+0000 7fccf57ff700 1 -- 172.21.15.60:0/1931690636 shutdown_connections 2023-07-13T22:29:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.150+0000 7fccf57ff700 1 -- 172.21.15.60:0/1931690636 wait complete. 2023-07-13T22:29:33.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.152+0000 7fccf57ff700 1 Processor -- start 2023-07-13T22:29:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.152+0000 7fccf57ff700 1 -- start start 2023-07-13T22:29:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccf57ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00b8400 0x7fccf015f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccf57ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccf00cfad0 0x7fccf00c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccf57ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccf00d3750 0x7fccf00cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccf57ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00c5e90 con 0x7fccf00d3750 2023-07-13T22:29:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccf57ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00cb1e0 con 0x7fccf00cfad0 2023-07-13T22:29:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccf57ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00cb4c0 con 0x7fccf00b8400 2023-07-13T22:29:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccef7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccf00d3750 0x7fccf00cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccef7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccf00d3750 0x7fccf00cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33116/0 (socket says 172.21.15.60:33116) 2023-07-13T22:29:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccef7fe700 1 -- 172.21.15.60:0/3178431998 learned_addr learned my addr 172.21.15.60:0/3178431998 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fcceeffd700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00b8400 0x7fccf015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fcce7fff700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccf00cfad0 0x7fccf00c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccef7fe700 1 -- 172.21.15.60:0/3178431998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00b8400 msgr2=0x7fccf015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccef7fe700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00b8400 0x7fccf015f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccef7fe700 1 -- 172.21.15.60:0/3178431998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccf00cfad0 msgr2=0x7fccf00c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccef7fe700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccf00cfad0 0x7fccf00c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fccef7fe700 1 -- 172.21.15.60:0/3178431998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00094a0 con 0x7fccf00d3750 2023-07-13T22:29:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.153+0000 7fcce7fff700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccf00cfad0 0x7fccf00c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.154+0000 7fccef7fe700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccf00d3750 0x7fccf00cc280 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcce800cc10 tx=0x7fcce800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.154+0000 7fcce77fe700 1 -- 172.21.15.60:0/3178431998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce800aaf0 con 0x7fccf00d3750 2023-07-13T22:29:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.154+0000 7fcce77fe700 1 -- 172.21.15.60:0/3178431998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce800ac70 con 0x7fccf00d3750 2023-07-13T22:29:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.154+0000 7fcce77fe700 1 -- 172.21.15.60:0/3178431998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce80070d0 con 0x7fccf00d3750 2023-07-13T22:29:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.154+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd80094a0 con 0x7fccf00d3750 2023-07-13T22:29:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.154+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf00cba80 con 0x7fccf00d3750 2023-07-13T22:29:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.155+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf00b1bd0 con 0x7fccf00d3750 2023-07-13T22:29:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.156+0000 7fcce77fe700 1 -- 172.21.15.60:0/3178431998 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcce8007230 con 0x7fccf00d3750 2023-07-13T22:29:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.157+0000 7fcce77fe700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fccd006ad10 0x7fccd006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.157+0000 7fcce77fe700 1 -- 172.21.15.60:0/3178431998 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcce8088e70 con 0x7fccf00d3750 2023-07-13T22:29:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.157+0000 7fcceeffd700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fccd006ad10 0x7fccd006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.158+0000 7fcceeffd700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fccd006ad10 0x7fccd006d1c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009cd0 tx=0x7fcce0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.161+0000 7fcce77fe700 1 -- 172.21.15.60:0/3178431998 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fcce8053e80 con 0x7fccf00d3750 2023-07-13T22:29:33.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.411+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccf00cff20 con 0x7fccf00d3750 2023-07-13T22:29:33.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.412+0000 7fcce77fe700 1 -- 172.21.15.60:0/3178431998 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fccf00cff20 con 0x7fccf00d3750 2023-07-13T22:29:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:33.392 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:28:58.254590+0000","modified":"2023-07-13T22:29:31.689264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7920":{"gid":7920,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6837/4078186433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":4078186433},{"type":"v1","addr":"172.21.15.60:6837","nonce":4078186433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-13T22:29:33.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fccd006ad10 msgr2=0x7fccd006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fccd006ad10 0x7fccd006d1c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009cd0 tx=0x7fcce0009210 comp rx=0 tx=0).stop 2023-07-13T22:29:33.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccf00d3750 msgr2=0x7fccf00cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccf00d3750 0x7fccf00cc280 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcce800cc10 tx=0x7fcce800cf20 comp rx=0 tx=0).stop 2023-07-13T22:29:33.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 shutdown_connections 2023-07-13T22:29:33.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fccd006ad10 0x7fccd006d1c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccf00d3750 0x7fccf00cc280 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00b8400 0x7fccf015f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 --2- 172.21.15.60:0/3178431998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccf00cfad0 0x7fccf00c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 >> 172.21.15.60:0/3178431998 conn(0x7fccf006b390 msgr2=0x7fccf00b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:33.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 shutdown_connections 2023-07-13T22:29:33.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.415+0000 7fccf57ff700 1 -- 172.21.15.60:0/3178431998 wait complete. 2023-07-13T22:29:33.396 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 65 2023-07-13T22:29:33.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:33.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.638+0000 7f76867ff700 1 Processor -- start 2023-07-13T22:29:33.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.638+0000 7f76867ff700 1 -- start start 2023-07-13T22:29:33.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f76867ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f76800b2010 0x7f76800cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f76867ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76800d26d0 0x7f76800cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f76867ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b5a00 0x7f76800cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f76867ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76800d2570 con 0x7f76800d26d0 2023-07-13T22:29:33.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f76867ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76800d14a0 con 0x7f76800b2010 2023-07-13T22:29:33.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f76867ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76800d1780 con 0x7f76800b5a00 2023-07-13T22:29:33.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f767f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76800d26d0 0x7f76800cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f767f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76800d26d0 0x7f76800cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33120/0 (socket says 172.21.15.60:33120) 2023-07-13T22:29:33.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f767f7fe700 1 -- 172.21.15.60:0/1616492616 learned_addr learned my addr 172.21.15.60:0/1616492616 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:33.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f7684d9c700 1 --2- 172.21.15.60:0/1616492616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b5a00 0x7f76800cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.640+0000 7f767ffff700 1 --2- 172.21.15.60:0/1616492616 >> v2:172.21.15.105:3300/0 conn(0x7f76800b2010 0x7f76800cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f767f7fe700 1 -- 172.21.15.60:0/1616492616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b5a00 msgr2=0x7f76800cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:33.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f767f7fe700 1 --2- 172.21.15.60:0/1616492616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b5a00 0x7f76800cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f767f7fe700 1 -- 172.21.15.60:0/1616492616 >> v2:172.21.15.105:3300/0 conn(0x7f76800b2010 msgr2=0x7f76800cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:33.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f767f7fe700 1 --2- 172.21.15.60:0/1616492616 >> v2:172.21.15.105:3300/0 conn(0x7f76800b2010 0x7f76800cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f767f7fe700 1 -- 172.21.15.60:0/1616492616 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76800cd2a0 con 0x7f76800d26d0 2023-07-13T22:29:33.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f7684d9c700 1 --2- 172.21.15.60:0/1616492616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b5a00 0x7f76800cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f767f7fe700 1 --2- 172.21.15.60:0/1616492616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800d26d0 0x7f76800cf6e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f768008bd20 tx=0x7f7674009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9830352ca958c89 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f767effd700 1 -- 172.21.15.60:0/1616492616 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767400b870 con 0x7f76800d26d0 2023-07-13T22:29:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f767effd700 1 -- 172.21.15.60:0/1616492616 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767400b9d0 con 0x7f76800d26d0 2023-07-13T22:29:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.641+0000 7f767effd700 1 -- 172.21.15.60:0/1616492616 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767400bd60 con 0x7f76800d26d0 2023-07-13T22:29:33.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.642+0000 7f76867ff700 1 -- 172.21.15.60:0/1616492616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800d26d0 msgr2=0x7f76800cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.642+0000 7f76867ff700 1 --2- 172.21.15.60:0/1616492616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800d26d0 0x7f76800cf6e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f768008bd20 tx=0x7f7674009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.642+0000 7f76867ff700 1 -- 172.21.15.60:0/1616492616 shutdown_connections 2023-07-13T22:29:33.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.642+0000 7f76867ff700 1 --2- 172.21.15.60:0/1616492616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b5a00 0x7f76800cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.642+0000 7f76867ff700 1 --2- 172.21.15.60:0/1616492616 >> v2:172.21.15.105:3300/0 conn(0x7f76800b2010 0x7f76800cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.642+0000 7f76867ff700 1 --2- 172.21.15.60:0/1616492616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800d26d0 0x7f76800cf6e0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.642+0000 7f76867ff700 1 -- 172.21.15.60:0/1616492616 >> 172.21.15.60:0/1616492616 conn(0x7f768006b380 msgr2=0x7f768006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:33.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.642+0000 7f76867ff700 1 -- 172.21.15.60:0/1616492616 shutdown_connections 2023-07-13T22:29:33.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.642+0000 7f76867ff700 1 -- 172.21.15.60:0/1616492616 wait complete. 2023-07-13T22:29:33.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.644+0000 7f76867ff700 1 Processor -- start 2023-07-13T22:29:33.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.644+0000 7f76867ff700 1 -- start start 2023-07-13T22:29:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f76867ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b2010 0x7f76801f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f76867ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800b5a00 0x7f768015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f76867ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76800d26d0 0x7f768015ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f76867ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768015da20 con 0x7f76800b5a00 2023-07-13T22:29:33.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f76867ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768015c2b0 con 0x7f76800d26d0 2023-07-13T22:29:33.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f76867ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76801f1b90 con 0x7f76800b2010 2023-07-13T22:29:33.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f767f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800b5a00 0x7f768015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f767f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800b5a00 0x7f768015a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33122/0 (socket says 172.21.15.60:33122) 2023-07-13T22:29:33.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f767f7fe700 1 -- 172.21.15.60:0/272487911 learned_addr learned my addr 172.21.15.60:0/272487911 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f767ffff700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b2010 0x7f76801f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.645+0000 7f7684d9c700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76800d26d0 0x7f768015ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f767f7fe700 1 -- 172.21.15.60:0/272487911 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b2010 msgr2=0x7f76801f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f767f7fe700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b2010 0x7f76801f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f767f7fe700 1 -- 172.21.15.60:0/272487911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76800d26d0 msgr2=0x7f768015ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f767f7fe700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76800d26d0 0x7f768015ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f767f7fe700 1 -- 172.21.15.60:0/272487911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7674009550 con 0x7f76800b5a00 2023-07-13T22:29:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f767f7fe700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800b5a00 0x7f768015a610 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7674013000 tx=0x7f7674004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f767cff9700 1 -- 172.21.15.60:0/272487911 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674004250 con 0x7f76800b5a00 2023-07-13T22:29:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f767cff9700 1 -- 172.21.15.60:0/272487911 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767401e070 con 0x7f76800b5a00 2023-07-13T22:29:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768015ab50 con 0x7f76800b5a00 2023-07-13T22:29:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f767cff9700 1 -- 172.21.15.60:0/272487911 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76740045d0 con 0x7f76800b5a00 2023-07-13T22:29:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.646+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76800a9970 con 0x7f76800b5a00 2023-07-13T22:29:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.647+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768004e280 con 0x7f76800b5a00 2023-07-13T22:29:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.648+0000 7f767cff9700 1 -- 172.21.15.60:0/272487911 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7674020030 con 0x7f76800b5a00 2023-07-13T22:29:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.649+0000 7f767cff9700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f766c06ace0 0x7f766c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.649+0000 7f767cff9700 1 -- 172.21.15.60:0/272487911 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f767408a3f0 con 0x7f76800b5a00 2023-07-13T22:29:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.650+0000 7f767ffff700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f766c06ace0 0x7f766c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.650+0000 7f767ffff700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f766c06ace0 0x7f766c06d190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7670009950 tx=0x7f7670008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:33.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.654+0000 7f767cff9700 1 -- 172.21.15.60:0/272487911 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7674055490 con 0x7f76800b5a00 2023-07-13T22:29:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.863+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76800d2aa0 con 0x7f76800b5a00 2023-07-13T22:29:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.864+0000 7f767cff9700 1 -- 172.21.15.60:0/272487911 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+18190 (secure 0 0 0) 0x7f7674058ab0 con 0x7f76800b5a00 2023-07-13T22:29:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:33.845 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":100,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:28:59.528113+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:28:56.533994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-13T22:28:57.226035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f766c06ace0 msgr2=0x7f766c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f766c06ace0 0x7f766c06d190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7670009950 tx=0x7f7670008040 comp rx=0 tx=0).stop 2023-07-13T22:29:33.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800b5a00 msgr2=0x7f768015a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:33.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800b5a00 0x7f768015a610 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7674013000 tx=0x7f7674004750 comp rx=0 tx=0).stop 2023-07-13T22:29:33.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 shutdown_connections 2023-07-13T22:29:33.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f766c06ace0 0x7f766c06d190 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76800d26d0 0x7f768015ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76800b2010 0x7f76801f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 --2- 172.21.15.60:0/272487911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76800b5a00 0x7f768015a610 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:33.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 >> 172.21.15.60:0/272487911 conn(0x7f768006b380 msgr2=0x7f76800732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:33.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 shutdown_connections 2023-07-13T22:29:33.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:33.867+0000 7f76867ff700 1 -- 172.21.15.60:0/272487911 wait complete. 2023-07-13T22:29:33.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:29:34.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.089+0000 7f0623fff700 1 Processor -- start 2023-07-13T22:29:34.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.090+0000 7f0623fff700 1 -- start start 2023-07-13T22:29:34.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.091+0000 7f0623fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f061c0b8400 0x7f061c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.091+0000 7f0623fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c0d0ac0 con 0x7f061c0cfca0 2023-07-13T22:29:34.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.091+0000 7f0623fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c0b87d0 con 0x7f061c0d3750 2023-07-13T22:29:34.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.091+0000 7f0623fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0b4eb0 con 0x7f061c0b8400 2023-07-13T22:29:34.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.091+0000 7f062259c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f061c0cfca0 0x7f061c0d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49886/0 (socket says 172.21.15.60:49886) 2023-07-13T22:29:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.091+0000 7f062259c700 1 -- 172.21.15.60:0/3126027052 learned_addr learned my addr 172.21.15.60:0/3126027052 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.091+0000 7f0621d9b700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f061c0b8400 0x7f061c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200626604 0 0) 0x7f061c0d0ac0 con 0x7f061c0cfca0 2023-07-13T22:29:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06080035b0 con 0x7f061c0cfca0 2023-07-13T22:29:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466654642 0 0) 0x7f061c0b87d0 con 0x7f061c0d3750 2023-07-13T22:29:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f061c0d0ac0 con 0x7f061c0d3750 2023-07-13T22:29:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1991382264 0 0) 0x7f06080035b0 con 0x7f061c0cfca0 2023-07-13T22:29:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f061c0b8400 msgr2=0x7f061c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:34.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f061c0b8400 0x7f061c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 >> v1:172.21.15.105:6789/0 conn(0x7f061c0d3750 legacy=0x7f061c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c0b5190 con 0x7f061c0cfca0 2023-07-13T22:29:34.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f06140019f0 con 0x7f061c0cfca0 2023-07-13T22:29:34.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 >> v1:172.21.15.60:6789/0 conn(0x7f061c0cfca0 legacy=0x7f061c0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0608003bc0 0x7f0608003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0608004850 0x7f0608004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f060800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0b4eb0 con 0x7f0608005200 2023-07-13T22:29:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060800c060 con 0x7f0608004850 2023-07-13T22:29:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060800c220 con 0x7f0608003bc0 2023-07-13T22:29:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f0621d9b700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f061c0b8400 0x7f061c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f062259c700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f060800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.092+0000 7f062159a700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0608004850 0x7f0608004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f0621d9b700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0608003bc0 0x7f0608003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f062259c700 1 -- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0608003bc0 msgr2=0x7f0608003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:34.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f062259c700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0608003bc0 0x7f0608003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f062259c700 1 -- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0608004850 msgr2=0x7f0608004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:34.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f062259c700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0608004850 0x7f0608004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f062259c700 1 -- 172.21.15.60:0/3126027052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06080035b0 con 0x7f0608005200 2023-07-13T22:29:34.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f062259c700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f060800ba10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f061c0d0090 tx=0x7f0608005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ba859ee24c5f99 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:34.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140072d0 con 0x7f0608005200 2023-07-13T22:29:34.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614007450 con 0x7f0608005200 2023-07-13T22:29:34.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.093+0000 7f0620d99700 1 -- 172.21.15.60:0/3126027052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614007800 con 0x7f0608005200 2023-07-13T22:29:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.094+0000 7f0623fff700 1 -- 172.21.15.60:0/3126027052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 msgr2=0x7f060800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.094+0000 7f0623fff700 1 --2- 172.21.15.60:0/3126027052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f060800ba10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f061c0d0090 tx=0x7f0608005880 comp rx=0 tx=0).stop 2023-07-13T22:29:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.094+0000 7f062259c700 1 -- 172.21.15.60:0/3126027052 reap_dead start 2023-07-13T22:29:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.094+0000 7f0623fff700 1 -- 172.21.15.60:0/3126027052 shutdown_connections 2023-07-13T22:29:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.094+0000 7f0623fff700 1 -- 172.21.15.60:0/3126027052 >> 172.21.15.60:0/3126027052 conn(0x7f061c06b390 msgr2=0x7f061c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.094+0000 7f0623fff700 1 -- 172.21.15.60:0/3126027052 shutdown_connections 2023-07-13T22:29:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.094+0000 7f0623fff700 1 -- 172.21.15.60:0/3126027052 wait complete. 2023-07-13T22:29:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.096+0000 7f0623fff700 1 Processor -- start 2023-07-13T22:29:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.097+0000 7f0623fff700 1 -- start start 2023-07-13T22:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0623fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f061c1f1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0623fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f061c1f1a80 0x7f061c0b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0623fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f061c0b30f0 0x7f061c15c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0623fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0b3560 con 0x7f0608005200 2023-07-13T22:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0623fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c15bfb0 con 0x7f061c1f1a80 2023-07-13T22:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0623fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1f2010 con 0x7f061c0b30f0 2023-07-13T22:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f061c1f1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f061c1f1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33150/0 (socket says 172.21.15.60:33150) 2023-07-13T22:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0621d9b700 1 -- 172.21.15.60:0/3280523159 learned_addr learned my addr 172.21.15.60:0/3280523159 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0621d9b700 1 -- 172.21.15.60:0/3280523159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f061c0b30f0 msgr2=0x7f061c15c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0621d9b700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f061c0b30f0 0x7f061c15c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0621d9b700 1 -- 172.21.15.60:0/3280523159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f061c1f1a80 msgr2=0x7f061c0b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0621d9b700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f061c1f1a80 0x7f061c0b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.098+0000 7f0621d9b700 1 -- 172.21.15.60:0/3280523159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c1f22f0 con 0x7f0608005200 2023-07-13T22:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.099+0000 7f0621d9b700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f061c1f1540 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0618011830 tx=0x7f0618011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:34.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.099+0000 7f06127fc700 1 -- 172.21.15.60:0/3280523159 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0618015870 con 0x7f0608005200 2023-07-13T22:29:34.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.099+0000 7f06127fc700 1 -- 172.21.15.60:0/3280523159 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06180159d0 con 0x7f0608005200 2023-07-13T22:29:34.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.099+0000 7f06127fc700 1 -- 172.21.15.60:0/3280523159 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0618015d80 con 0x7f0608005200 2023-07-13T22:29:34.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.099+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c1f25d0 con 0x7f0608005200 2023-07-13T22:29:34.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.099+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c0c7770 con 0x7f0608005200 2023-07-13T22:29:34.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.100+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c15a850 con 0x7f0608005200 2023-07-13T22:29:34.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.101+0000 7f06127fc700 1 -- 172.21.15.60:0/3280523159 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0618005a20 con 0x7f0608005200 2023-07-13T22:29:34.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.102+0000 7f06127fc700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f060006ace0 0x7f060006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.102+0000 7f06127fc700 1 -- 172.21.15.60:0/3280523159 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0618089990 con 0x7f0608005200 2023-07-13T22:29:34.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.102+0000 7f062159a700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f060006ace0 0x7f060006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.103+0000 7f062159a700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f060006ace0 0x7f060006d190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f060c007a30 tx=0x7f060c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.107+0000 7f06127fc700 1 -- 172.21.15.60:0/3280523159 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f06180549a0 con 0x7f0608005200 2023-07-13T22:29:34.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.360+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f061c0b28d0 con 0x7f0608005200 2023-07-13T22:29:34.339 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:29:34.360+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:29:34.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.364+0000 7f06127fc700 1 -- 172.21.15.60:0/3280523159 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0618057fc0 con 0x7f0608005200 2023-07-13T22:29:34.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f060006ace0 msgr2=0x7f060006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f060006ace0 0x7f060006d190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f060c007a30 tx=0x7f060c009040 comp rx=0 tx=0).stop 2023-07-13T22:29:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 msgr2=0x7f061c1f1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f061c1f1540 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0618011830 tx=0x7f0618011bf0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 shutdown_connections 2023-07-13T22:29:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f060006ace0 0x7f060006d190 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0608005200 0x7f061c1f1540 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f061c0b30f0 0x7f061c15c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 --2- 172.21.15.60:0/3280523159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f061c1f1a80 0x7f061c0b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 >> 172.21.15.60:0/3280523159 conn(0x7f061c06b390 msgr2=0x7f061c0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:34.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.367+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 shutdown_connections 2023-07-13T22:29:34.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.368+0000 7f0623fff700 1 -- 172.21.15.60:0/3280523159 wait complete. 2023-07-13T22:29:34.351 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:29:34.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:29:34.416 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:29:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.594+0000 7f12fe1ff700 1 Processor -- start 2023-07-13T22:29:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.595+0000 7f12fe1ff700 1 -- start start 2023-07-13T22:29:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.596+0000 7f12fe1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f80cc410 0x7f12f806d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.596+0000 7f12fe1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f80cc870 con 0x7f12f806a5e0 2023-07-13T22:29:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.596+0000 7f12fe1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f80ccaa0 con 0x7f12f815c740 2023-07-13T22:29:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.596+0000 7f12fe1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f806dd50 con 0x7f12f80cc410 2023-07-13T22:29:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.596+0000 7f12f6ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f12f806a5e0 0x7f12f806c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49894/0 (socket says 172.21.15.60:49894) 2023-07-13T22:29:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.596+0000 7f12f6ffd700 1 -- 172.21.15.60:0/1708234549 learned_addr learned my addr 172.21.15.60:0/1708234549 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.596+0000 7f12f7fff700 1 --2- 172.21.15.60:0/1708234549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f80cc410 0x7f12f806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f67fc700 1 -- 172.21.15.60:0/1708234549 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614274711 0 0) 0x7f12f80cc870 con 0x7f12f806a5e0 2023-07-13T22:29:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f67fc700 1 -- 172.21.15.60:0/1708234549 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e00035b0 con 0x7f12f806a5e0 2023-07-13T22:29:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f67fc700 1 -- 172.21.15.60:0/1708234549 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043498379 0 0) 0x7f12f80ccaa0 con 0x7f12f815c740 2023-07-13T22:29:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f67fc700 1 -- 172.21.15.60:0/1708234549 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f80cc870 con 0x7f12f815c740 2023-07-13T22:29:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f7fff700 1 -- 172.21.15.60:0/1708234549 >> v1:172.21.15.105:6789/0 conn(0x7f12f815c740 legacy=0x7f12f8069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f7fff700 1 -- 172.21.15.60:0/1708234549 >> v1:172.21.15.60:6789/0 conn(0x7f12f806a5e0 legacy=0x7f12f806c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f67fc700 1 -- 172.21.15.60:0/1708234549 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2115384022 0 0) 0x7f12e00035b0 con 0x7f12f806a5e0 2023-07-13T22:29:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f7fff700 1 -- 172.21.15.60:0/1708234549 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f81600f0 con 0x7f12f80cc410 2023-07-13T22:29:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f7fff700 1 --2- 172.21.15.60:0/1708234549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f80cc410 0x7f12f806d810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009dc0 tx=0x7f12f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9221f5599e92e04 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.597+0000 7f12f67fc700 1 -- 172.21.15.60:0/1708234549 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f000e070 con 0x7f12f80cc410 2023-07-13T22:29:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.598+0000 7f12f67fc700 1 -- 172.21.15.60:0/1708234549 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0003a20 con 0x7f12f80cc410 2023-07-13T22:29:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.598+0000 7f12f67fc700 1 -- 172.21.15.60:0/1708234549 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec001670 con 0x7f12f80cc410 2023-07-13T22:29:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.598+0000 7f12fe1ff700 1 -- 172.21.15.60:0/1708234549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f80cc410 msgr2=0x7f12f806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.598+0000 7f12fe1ff700 1 --2- 172.21.15.60:0/1708234549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f80cc410 0x7f12f806d810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009dc0 tx=0x7f12f000b330 comp rx=0 tx=0).stop 2023-07-13T22:29:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.598+0000 7f12fe1ff700 1 -- 172.21.15.60:0/1708234549 shutdown_connections 2023-07-13T22:29:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.598+0000 7f12fe1ff700 1 --2- 172.21.15.60:0/1708234549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f80cc410 0x7f12f806d810 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.598+0000 7f12fe1ff700 1 -- 172.21.15.60:0/1708234549 >> 172.21.15.60:0/1708234549 conn(0x7f12f80cbf50 msgr2=0x7f12f80ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.598+0000 7f12fe1ff700 1 -- 172.21.15.60:0/1708234549 shutdown_connections 2023-07-13T22:29:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.598+0000 7f12fe1ff700 1 -- 172.21.15.60:0/1708234549 wait complete. 2023-07-13T22:29:34.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.600+0000 7f12fe1ff700 1 Processor -- start 2023-07-13T22:29:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.600+0000 7f12fe1ff700 1 -- start start 2023-07-13T22:29:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12fe1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f806a5e0 0x7f12f80c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12fe1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12f80cc410 0x7f12f81f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12fe1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f815c740 0x7f12f80b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12fe1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f814ea20 con 0x7f12f80cc410 2023-07-13T22:29:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12fe1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f80c41a0 con 0x7f12f806a5e0 2023-07-13T22:29:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12fe1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f81513f0 con 0x7f12f815c740 2023-07-13T22:29:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12f6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12f80cc410 0x7f12f81f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12f6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12f80cc410 0x7f12f81f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33172/0 (socket says 172.21.15.60:33172) 2023-07-13T22:29:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12f6ffd700 1 -- 172.21.15.60:0/3253472749 learned_addr learned my addr 172.21.15.60:0/3253472749 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12f7fff700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f815c740 0x7f12f80b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.601+0000 7f12f77fe700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f806a5e0 0x7f12f80c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12f6ffd700 1 -- 172.21.15.60:0/3253472749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f815c740 msgr2=0x7f12f80b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12f6ffd700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f815c740 0x7f12f80b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12f6ffd700 1 -- 172.21.15.60:0/3253472749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f806a5e0 msgr2=0x7f12f80c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:34.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12f6ffd700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f806a5e0 0x7f12f80c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:34.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12f6ffd700 1 -- 172.21.15.60:0/3253472749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f0009a70 con 0x7f12f80cc410 2023-07-13T22:29:34.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12f6ffd700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12f80cc410 0x7f12f81f21c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00c780 tx=0x7f12ec00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:34.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12dffff700 1 -- 172.21.15.60:0/3253472749 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec004c80 con 0x7f12f80cc410 2023-07-13T22:29:34.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12dffff700 1 -- 172.21.15.60:0/3253472749 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec004e00 con 0x7f12f80cc410 2023-07-13T22:29:34.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f80c3590 con 0x7f12f80cc410 2023-07-13T22:29:34.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12dffff700 1 -- 172.21.15.60:0/3253472749 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec007310 con 0x7f12f80cc410 2023-07-13T22:29:34.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.602+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f80c3b50 con 0x7f12f80cc410 2023-07-13T22:29:34.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.603+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f80713b0 con 0x7f12f80cc410 2023-07-13T22:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.604+0000 7f12dffff700 1 -- 172.21.15.60:0/3253472749 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f12ec007470 con 0x7f12f80cc410 2023-07-13T22:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.605+0000 7f12dffff700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12e406ace0 0x7f12e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.605+0000 7f12dffff700 1 -- 172.21.15.60:0/3253472749 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f12ec0892a0 con 0x7f12f80cc410 2023-07-13T22:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.606+0000 7f12f77fe700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12e406ace0 0x7f12e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.606+0000 7f12f77fe700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12e406ace0 0x7f12e406d190 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f12e80011e0 tx=0x7f12e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:34.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.610+0000 7f12dffff700 1 -- 172.21.15.60:0/3253472749 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f12ec054250 con 0x7f12f80cc410 2023-07-13T22:29:34.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:34.862+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f12f804e210 con 0x7f12f80cc410 2023-07-13T22:29:35.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.366+0000 7f12dffff700 1 -- 172.21.15.60:0/3253472749 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f12ec057870 con 0x7f12f80cc410 2023-07-13T22:29:35.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.368+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12e406ace0 msgr2=0x7f12e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:35.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.368+0000 7f12fe1ff700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12e406ace0 0x7f12e406d190 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f12e80011e0 tx=0x7f12e8009040 comp rx=0 tx=0).stop 2023-07-13T22:29:35.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.368+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12f80cc410 msgr2=0x7f12f81f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:35.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.369+0000 7f12fe1ff700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12f80cc410 0x7f12f81f21c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00c780 tx=0x7f12ec00ca90 comp rx=0 tx=0).stop 2023-07-13T22:29:35.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.369+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 shutdown_connections 2023-07-13T22:29:35.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.369+0000 7f12fe1ff700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f12e406ace0 0x7f12e406d190 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.369+0000 7f12fe1ff700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12f815c740 0x7f12f80b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.369+0000 7f12fe1ff700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12f806a5e0 0x7f12f80c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.369+0000 7f12fe1ff700 1 --2- 172.21.15.60:0/3253472749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12f80cc410 0x7f12f81f21c0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.369+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 >> 172.21.15.60:0/3253472749 conn(0x7f12f80cbf50 msgr2=0x7f12f80732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:35.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.369+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 shutdown_connections 2023-07-13T22:29:35.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.369+0000 7f12fe1ff700 1 -- 172.21.15.60:0/3253472749 wait complete. 2023-07-13T22:29:35.369 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:29:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.598+0000 7f56dedff700 1 Processor -- start 2023-07-13T22:29:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.599+0000 7f56dedff700 1 -- start start 2023-07-13T22:29:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56dedff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f56d80b8270 0x7f56d80cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56dedff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56d80b1f50 0x7f56d80cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56dedff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d80cd890 con 0x7f56d80cfa30 2023-07-13T22:29:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56dedff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d80cf380 con 0x7f56d80b1f50 2023-07-13T22:29:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56dedff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d80cd160 con 0x7f56d80b8270 2023-07-13T22:29:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56d7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56d80b1f50 0x7f56d80cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56d7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56d80b1f50 0x7f56d80cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45268/0 (socket says 172.21.15.60:45268) 2023-07-13T22:29:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56d7fff700 1 -- 172.21.15.60:0/2073389003 learned_addr learned my addr 172.21.15.60:0/2073389003 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56dcb9b700 1 --2- 172.21.15.60:0/2073389003 >> v2:172.21.15.105:3300/0 conn(0x7f56d80b8270 0x7f56d80cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56d7fff700 1 -- 172.21.15.60:0/2073389003 >> v2:172.21.15.105:3300/0 conn(0x7f56d80b8270 msgr2=0x7f56d80cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.600+0000 7f56d7fff700 1 --2- 172.21.15.60:0/2073389003 >> v2:172.21.15.105:3300/0 conn(0x7f56d80b8270 0x7f56d80cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.601+0000 7f56d7fff700 1 -- 172.21.15.60:0/2073389003 >> v1:172.21.15.105:6789/0 conn(0x7f56d80cfa30 legacy=0x7f56d815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:35.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.601+0000 7f56d7fff700 1 -- 172.21.15.60:0/2073389003 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d80b8640 con 0x7f56d80b1f50 2023-07-13T22:29:35.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.601+0000 7f56dcb9b700 1 --2- 172.21.15.60:0/2073389003 >> v2:172.21.15.105:3300/0 conn(0x7f56d80b8270 0x7f56d80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:35.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.601+0000 7f56d7fff700 1 --2- 172.21.15.60:0/2073389003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80b1f50 0x7f56d80cf4f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f56d806cd50 tx=0x7f56c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c3341bac17cf8f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.601+0000 7f56d77fe700 1 -- 172.21.15.60:0/2073389003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4004230 con 0x7f56d80b1f50 2023-07-13T22:29:35.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.601+0000 7f56d77fe700 1 -- 172.21.15.60:0/2073389003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4004390 con 0x7f56d80b1f50 2023-07-13T22:29:35.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.601+0000 7f56d77fe700 1 -- 172.21.15.60:0/2073389003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40036a0 con 0x7f56d80b1f50 2023-07-13T22:29:35.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.602+0000 7f56dedff700 1 -- 172.21.15.60:0/2073389003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80b1f50 msgr2=0x7f56d80cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.602+0000 7f56dedff700 1 --2- 172.21.15.60:0/2073389003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80b1f50 0x7f56d80cf4f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f56d806cd50 tx=0x7f56c4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.602+0000 7f56dedff700 1 -- 172.21.15.60:0/2073389003 shutdown_connections 2023-07-13T22:29:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.602+0000 7f56dedff700 1 --2- 172.21.15.60:0/2073389003 >> v2:172.21.15.105:3300/0 conn(0x7f56d80b8270 0x7f56d80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.602+0000 7f56dedff700 1 --2- 172.21.15.60:0/2073389003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80b1f50 0x7f56d80cf4f0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.602+0000 7f56dedff700 1 -- 172.21.15.60:0/2073389003 >> 172.21.15.60:0/2073389003 conn(0x7f56d806b330 msgr2=0x7f56d806b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.602+0000 7f56dedff700 1 -- 172.21.15.60:0/2073389003 shutdown_connections 2023-07-13T22:29:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.602+0000 7f56dedff700 1 -- 172.21.15.60:0/2073389003 wait complete. 2023-07-13T22:29:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.604+0000 7f56dedff700 1 Processor -- start 2023-07-13T22:29:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.604+0000 7f56dedff700 1 -- start start 2023-07-13T22:29:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dedff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d80b1f50 0x7f56d80cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dedff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d80b8270 0x7f56d80c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dedff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80cfa30 0x7f56d815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dedff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d80c7700 con 0x7f56d80cfa30 2023-07-13T22:29:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dedff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d80c79e0 con 0x7f56d80b8270 2023-07-13T22:29:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dedff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d80c7cc0 con 0x7f56d80b1f50 2023-07-13T22:29:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dd39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80cfa30 0x7f56d815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dd39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80cfa30 0x7f56d815c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45284/0 (socket says 172.21.15.60:45284) 2023-07-13T22:29:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dd39c700 1 -- 172.21.15.60:0/2295615370 learned_addr learned my addr 172.21.15.60:0/2295615370 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dd39c700 1 -- 172.21.15.60:0/2295615370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d80b1f50 msgr2=0x7f56d80cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dd39c700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d80b1f50 0x7f56d80cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dd39c700 1 -- 172.21.15.60:0/2295615370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d80b8270 msgr2=0x7f56d80c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dd39c700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d80b8270 0x7f56d80c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.605+0000 7f56dd39c700 1 -- 172.21.15.60:0/2295615370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c4009550 con 0x7f56d80cfa30 2023-07-13T22:29:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.606+0000 7f56dd39c700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80cfa30 0x7f56d815c8f0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f56d000c7a0 tx=0x7f56d000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.606+0000 7f56d57fa700 1 -- 172.21.15.60:0/2295615370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0003e10 con 0x7f56d80cfa30 2023-07-13T22:29:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.606+0000 7f56d57fa700 1 -- 172.21.15.60:0/2295615370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d0005070 con 0x7f56d80cfa30 2023-07-13T22:29:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.606+0000 7f56d57fa700 1 -- 172.21.15.60:0/2295615370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0005420 con 0x7f56d80cfa30 2023-07-13T22:29:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.606+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d80c7fa0 con 0x7f56d80cfa30 2023-07-13T22:29:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.606+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d80cc2f0 con 0x7f56d80cfa30 2023-07-13T22:29:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.607+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d804e210 con 0x7f56d80cfa30 2023-07-13T22:29:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.608+0000 7f56d57fa700 1 -- 172.21.15.60:0/2295615370 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56d0005580 con 0x7f56d80cfa30 2023-07-13T22:29:35.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.609+0000 7f56d57fa700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56bc06ace0 0x7f56bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.609+0000 7f56d57fa700 1 -- 172.21.15.60:0/2295615370 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f56d0088910 con 0x7f56d80cfa30 2023-07-13T22:29:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.609+0000 7f56dcb9b700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56bc06ace0 0x7f56bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.610+0000 7f56dcb9b700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56bc06ace0 0x7f56bc06d190 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f56cc007920 tx=0x7f56cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.613+0000 7f56d57fa700 1 -- 172.21.15.60:0/2295615370 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f56d0053950 con 0x7f56d80cfa30 2023-07-13T22:29:35.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:35.821+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f56d80cc4e0 con 0x7f56d80cfa30 2023-07-13T22:29:36.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.712+0000 7f56d57fa700 1 -- 172.21.15.60:0/2295615370 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7f56d0056f70 con 0x7f56d80cfa30 2023-07-13T22:29:36.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56bc06ace0 msgr2=0x7f56bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56bc06ace0 0x7f56bc06d190 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f56cc007920 tx=0x7f56cc008040 comp rx=0 tx=0).stop 2023-07-13T22:29:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80cfa30 msgr2=0x7f56d815c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80cfa30 0x7f56d815c8f0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f56d000c7a0 tx=0x7f56d000cb60 comp rx=0 tx=0).stop 2023-07-13T22:29:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 shutdown_connections 2023-07-13T22:29:36.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f56bc06ace0 0x7f56bc06d190 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:36.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56d80cfa30 0x7f56d815c8f0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:36.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f56d80b1f50 0x7f56d80cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:36.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 --2- 172.21.15.60:0/2295615370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d80b8270 0x7f56d80c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:36.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 >> 172.21.15.60:0/2295615370 conn(0x7f56d806b330 msgr2=0x7f56d80b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:36.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 shutdown_connections 2023-07-13T22:29:36.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.716+0000 7f56dedff700 1 -- 172.21.15.60:0/2295615370 wait complete. 2023-07-13T22:29:36.698 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:29:36.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:29:36.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.951+0000 7f3a485ff700 1 Processor -- start 2023-07-13T22:29:36.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.952+0000 7f3a485ff700 1 -- start start 2023-07-13T22:29:36.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.953+0000 7f3a485ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:36.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.954+0000 7f3a485ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3a400d0890 0x7f3a400d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:36.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.954+0000 7f3a485ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4015d200 con 0x7f3a4015c9b0 2023-07-13T22:29:36.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.954+0000 7f3a485ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a400d2e60 con 0x7f3a400d0890 2023-07-13T22:29:36.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.954+0000 7f3a485ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400d2b50 con 0x7f3a40159ea0 2023-07-13T22:29:36.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.954+0000 7f3a46b9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3a400d0890 0x7f3a400d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:36.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.954+0000 7f3a4639b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:36.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.954+0000 7f3a45b9a700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3a4015c9b0 0x7f3a400d3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:58186/0 (socket says 172.21.15.60:58186) 2023-07-13T22:29:36.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.954+0000 7f3a45b9a700 1 -- 172.21.15.60:0/1590634293 learned_addr learned my addr 172.21.15.60:0/1590634293 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.954+0000 7f3a4639b700 1 --2- 172.21.15.60:0/1590634293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:45452/0 (socket says 172.21.15.60:45452) 2023-07-13T22:29:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.955+0000 7f3a4639b700 1 -- 172.21.15.60:0/1590634293 >> v2:172.21.15.105:3300/0 conn(0x7f3a400d0890 msgr2=0x7f3a400d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.955+0000 7f3a4639b700 1 --2- 172.21.15.60:0/1590634293 >> v2:172.21.15.105:3300/0 conn(0x7f3a400d0890 0x7f3a400d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.955+0000 7f3a4639b700 1 -- 172.21.15.60:0/1590634293 >> v1:172.21.15.105:6789/0 conn(0x7f3a4015c9b0 legacy=0x7f3a400d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.955+0000 7f3a4639b700 1 -- 172.21.15.60:0/1590634293 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4015d4e0 con 0x7f3a40159ea0 2023-07-13T22:29:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.955+0000 7f3a46b9c700 1 --2- 172.21.15.60:0/1590634293 >> v2:172.21.15.105:3300/0 conn(0x7f3a400d0890 0x7f3a400d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:36.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.955+0000 7f3a4639b700 1 --2- 172.21.15.60:0/1590634293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400cd6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009840 tx=0x7f3a3c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b314a2d553579ab server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:36.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.955+0000 7f3a45399700 1 -- 172.21.15.60:0/1590634293 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c011830 con 0x7f3a40159ea0 2023-07-13T22:29:36.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.955+0000 7f3a45399700 1 -- 172.21.15.60:0/1590634293 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c011990 con 0x7f3a40159ea0 2023-07-13T22:29:36.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.955+0000 7f3a45399700 1 -- 172.21.15.60:0/1590634293 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c011d20 con 0x7f3a40159ea0 2023-07-13T22:29:36.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.956+0000 7f3a485ff700 1 -- 172.21.15.60:0/1590634293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 msgr2=0x7f3a400cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:36.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.956+0000 7f3a485ff700 1 --2- 172.21.15.60:0/1590634293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400cd6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009840 tx=0x7f3a3c009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:36.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.956+0000 7f3a485ff700 1 -- 172.21.15.60:0/1590634293 shutdown_connections 2023-07-13T22:29:36.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.956+0000 7f3a485ff700 1 --2- 172.21.15.60:0/1590634293 >> v2:172.21.15.105:3300/0 conn(0x7f3a400d0890 0x7f3a400d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:36.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.956+0000 7f3a485ff700 1 --2- 172.21.15.60:0/1590634293 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400cd6d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:36.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.956+0000 7f3a485ff700 1 -- 172.21.15.60:0/1590634293 >> 172.21.15.60:0/1590634293 conn(0x7f3a40071600 msgr2=0x7f3a400719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:36.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.956+0000 7f3a485ff700 1 -- 172.21.15.60:0/1590634293 shutdown_connections 2023-07-13T22:29:36.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.956+0000 7f3a485ff700 1 -- 172.21.15.60:0/1590634293 wait complete. 2023-07-13T22:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.957+0000 7f3a485ff700 1 Processor -- start 2023-07-13T22:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.958+0000 7f3a485ff700 1 -- start start 2023-07-13T22:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a485ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400d0890 0x7f3a400b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a485ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a485ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a4015c9b0 0x7f3a40153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a485ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400b5af0 con 0x7f3a400d0890 2023-07-13T22:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a485ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400b5dd0 con 0x7f3a4015c9b0 2023-07-13T22:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a485ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400b60b0 con 0x7f3a40159ea0 2023-07-13T22:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a4639b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400d0890 0x7f3a400b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a4639b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400d0890 0x7f3a400b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45296/0 (socket says 172.21.15.60:45296) 2023-07-13T22:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a4639b700 1 -- 172.21.15.60:0/1982346358 learned_addr learned my addr 172.21.15.60:0/1982346358 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a45b9a700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a46b9c700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a4015c9b0 0x7f3a40153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a4639b700 1 -- 172.21.15.60:0/1982346358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 msgr2=0x7f3a400c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a4639b700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a4639b700 1 -- 172.21.15.60:0/1982346358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a4015c9b0 msgr2=0x7f3a40153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a4639b700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a4015c9b0 0x7f3a40153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a4639b700 1 -- 172.21.15.60:0/1982346358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a38009a70 con 0x7f3a400d0890 2023-07-13T22:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.959+0000 7f3a45b9a700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.960+0000 7f3a4639b700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400d0890 0x7f3a400b6e70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009fd0 tx=0x7f3a3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.960+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1982346358 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0036a0 con 0x7f3a400d0890 2023-07-13T22:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.960+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1982346358 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c01f070 con 0x7f3a400d0890 2023-07-13T22:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.960+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1982346358 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c003a60 con 0x7f3a400d0890 2023-07-13T22:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.960+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c0094a0 con 0x7f3a400d0890 2023-07-13T22:29:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.960+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a400b6670 con 0x7f3a400d0890 2023-07-13T22:29:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.961+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a400b6cc0 con 0x7f3a400d0890 2023-07-13T22:29:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.962+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1982346358 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a3c003bc0 con 0x7f3a400d0890 2023-07-13T22:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.963+0000 7f3a36ffd700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.963+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1982346358 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f3a3c021070 con 0x7f3a400d0890 2023-07-13T22:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.963+0000 7f3a45b9a700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.964+0000 7f3a45b9a700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3a30007fd0 tx=0x7f3a30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:36.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:36.967+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1982346358 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3a3c055e10 con 0x7f3a400d0890 2023-07-13T22:29:37.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.172+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3a40067a70 con 0x7f3a400d0890 2023-07-13T22:29:37.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.726+0000 7f3a36ffd700 1 -- 172.21.15.60:0/1982346358 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3a3c0150d0 con 0x7f3a400d0890 2023-07-13T22:29:37.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.729+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 msgr2=0x7f3a2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:37.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.729+0000 7f3a485ff700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3a30007fd0 tx=0x7f3a30009040 comp rx=0 tx=0).stop 2023-07-13T22:29:37.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.729+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400d0890 msgr2=0x7f3a400b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:37.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.729+0000 7f3a485ff700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400d0890 0x7f3a400b6e70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009fd0 tx=0x7f3a3c004750 comp rx=0 tx=0).stop 2023-07-13T22:29:37.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.730+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 shutdown_connections 2023-07-13T22:29:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.730+0000 7f3a485ff700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a2406ace0 0x7f3a2406d190 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.730+0000 7f3a485ff700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400d0890 0x7f3a400b6e70 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.730+0000 7f3a485ff700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a4015c9b0 0x7f3a40153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.730+0000 7f3a485ff700 1 --2- 172.21.15.60:0/1982346358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a40159ea0 0x7f3a400c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.730+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 >> 172.21.15.60:0/1982346358 conn(0x7f3a40071600 msgr2=0x7f3a4015f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.730+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 shutdown_connections 2023-07-13T22:29:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.730+0000 7f3a485ff700 1 -- 172.21.15.60:0/1982346358 wait complete. 2023-07-13T22:29:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:29:37.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-07-13T22:29:37.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.963+0000 7ff6c7fff700 1 Processor -- start 2023-07-13T22:29:37.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.963+0000 7ff6c7fff700 1 -- start start 2023-07-13T22:29:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.964+0000 7ff6c7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6c00b8200 0x7ff6c00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.964+0000 7ff6c7fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6c00b1ac0 0x7ff6c00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.964+0000 7ff6c7fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c00d1870 con 0x7ff6c00cede0 2023-07-13T22:29:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.964+0000 7ff6c7fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00cf370 con 0x7ff6c00b8200 2023-07-13T22:29:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.964+0000 7ff6c7fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00d12b0 con 0x7ff6c00b1ac0 2023-07-13T22:29:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c5d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6c00b8200 0x7ff6c00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c5d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6c00b8200 0x7ff6c00cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45308/0 (socket says 172.21.15.60:45308) 2023-07-13T22:29:37.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c5d9b700 1 -- 172.21.15.60:0/2931524631 learned_addr learned my addr 172.21.15.60:0/2931524631 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c5d9b700 1 -- 172.21.15.60:0/2931524631 >> v2:172.21.15.105:3300/0 conn(0x7ff6c00b1ac0 msgr2=0x7ff6c00cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c559a700 1 --2- 172.21.15.60:0/2931524631 >> v2:172.21.15.105:3300/0 conn(0x7ff6c00b1ac0 0x7ff6c00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c5d9b700 1 --2- 172.21.15.60:0/2931524631 >> v2:172.21.15.105:3300/0 conn(0x7ff6c00b1ac0 0x7ff6c00cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c5d9b700 1 -- 172.21.15.60:0/2931524631 >> v1:172.21.15.105:6789/0 conn(0x7ff6c00cede0 legacy=0x7ff6c015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c5d9b700 1 -- 172.21.15.60:0/2931524631 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c00d1590 con 0x7ff6c00b8200 2023-07-13T22:29:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c559a700 1 --2- 172.21.15.60:0/2931524631 >> v2:172.21.15.105:3300/0 conn(0x7ff6c00b1ac0 0x7ff6c00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.965+0000 7ff6c5d9b700 1 --2- 172.21.15.60:0/2931524631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 0x7ff6c00cf640 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d07b0 tx=0x7ff6bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4903f590a1cf3ec3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c4d99700 1 -- 172.21.15.60:0/2931524631 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc004230 con 0x7ff6c00b8200 2023-07-13T22:29:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c4d99700 1 -- 172.21.15.60:0/2931524631 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc004390 con 0x7ff6c00b8200 2023-07-13T22:29:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c4d99700 1 -- 172.21.15.60:0/2931524631 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0036a0 con 0x7ff6c00b8200 2023-07-13T22:29:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2931524631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 msgr2=0x7ff6c00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c7fff700 1 --2- 172.21.15.60:0/2931524631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 0x7ff6c00cf640 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d07b0 tx=0x7ff6bc009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2931524631 shutdown_connections 2023-07-13T22:29:37.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c7fff700 1 --2- 172.21.15.60:0/2931524631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 0x7ff6c00cf640 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c7fff700 1 --2- 172.21.15.60:0/2931524631 >> v2:172.21.15.105:3300/0 conn(0x7ff6c00b1ac0 0x7ff6c00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2931524631 >> 172.21.15.60:0/2931524631 conn(0x7ff6c006b2f0 msgr2=0x7ff6c006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:37.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2931524631 shutdown_connections 2023-07-13T22:29:37.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.966+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2931524631 wait complete. 2023-07-13T22:29:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.968+0000 7ff6c7fff700 1 Processor -- start 2023-07-13T22:29:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.968+0000 7ff6c7fff700 1 -- start start 2023-07-13T22:29:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c00b1ac0 0x7ff6c015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 0x7ff6c00cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6c00cede0 0x7ff6c0143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c7fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00cc0d0 con 0x7ff6c00b8200 2023-07-13T22:29:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c7fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00cb7c0 con 0x7ff6c00b1ac0 2023-07-13T22:29:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c7fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00cab10 con 0x7ff6c00cede0 2023-07-13T22:29:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c559a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 0x7ff6c00cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c559a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 0x7ff6c00cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45316/0 (socket says 172.21.15.60:45316) 2023-07-13T22:29:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c559a700 1 -- 172.21.15.60:0/2185649487 learned_addr learned my addr 172.21.15.60:0/2185649487 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.969+0000 7ff6c5d9b700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c00b1ac0 0x7ff6c015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6c659c700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6c00cede0 0x7ff6c0143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6c559a700 1 -- 172.21.15.60:0/2185649487 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6c00cede0 msgr2=0x7ff6c0143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6c559a700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6c00cede0 0x7ff6c0143080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6c559a700 1 -- 172.21.15.60:0/2185649487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c00b1ac0 msgr2=0x7ff6c015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6c559a700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c00b1ac0 0x7ff6c015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6c559a700 1 -- 172.21.15.60:0/2185649487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc009550 con 0x7ff6c00b8200 2023-07-13T22:29:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6c559a700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 0x7ff6c00cc4f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c820 tx=0x7ff6b000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6b67fc700 1 -- 172.21.15.60:0/2185649487 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0011800 con 0x7ff6c00b8200 2023-07-13T22:29:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6b67fc700 1 -- 172.21.15.60:0/2185649487 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b0011960 con 0x7ff6c00b8200 2023-07-13T22:29:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c00cba10 con 0x7ff6c00b8200 2023-07-13T22:29:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.970+0000 7ff6b67fc700 1 -- 172.21.15.60:0/2185649487 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0011cf0 con 0x7ff6c00b8200 2023-07-13T22:29:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.971+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c015aeb0 con 0x7ff6c00b8200 2023-07-13T22:29:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.971+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c00cc210 con 0x7ff6c00b8200 2023-07-13T22:29:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.973+0000 7ff6b67fc700 1 -- 172.21.15.60:0/2185649487 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff6b00233a0 con 0x7ff6c00b8200 2023-07-13T22:29:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.973+0000 7ff6b67fc700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6a406ace0 0x7ff6a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.974+0000 7ff6b67fc700 1 -- 172.21.15.60:0/2185649487 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff6b0089380 con 0x7ff6c00b8200 2023-07-13T22:29:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.974+0000 7ff6c5d9b700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6a406ace0 0x7ff6a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.974+0000 7ff6c5d9b700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6a406ace0 0x7ff6a406d190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009130 tx=0x7ff6bc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:37.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:37.978+0000 7ff6b67fc700 1 -- 172.21.15.60:0/2185649487 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff6c00cc210 con 0x7ff6c00b8200 2023-07-13T22:29:38.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.182+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7ff6c00cc210 con 0x7ff6c00b8200 2023-07-13T22:29:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.186+0000 7ff6b67fc700 1 -- 172.21.15.60:0/2185649487 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v39) v1 ==== 102+0+0 (secure 0 0 0) 0x7ff6c00cc210 con 0x7ff6c00b8200 2023-07-13T22:29:38.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6a406ace0 msgr2=0x7ff6a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6a406ace0 0x7ff6a406d190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009130 tx=0x7ff6bc008030 comp rx=0 tx=0).stop 2023-07-13T22:29:38.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 msgr2=0x7ff6c00cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 0x7ff6c00cc4f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c820 tx=0x7ff6b000cb30 comp rx=0 tx=0).stop 2023-07-13T22:29:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 shutdown_connections 2023-07-13T22:29:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff6a406ace0 0x7ff6a406d190 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6c00cede0 0x7ff6c0143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c00b1ac0 0x7ff6c015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 --2- 172.21.15.60:0/2185649487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6c00b8200 0x7ff6c00cc4f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 >> 172.21.15.60:0/2185649487 conn(0x7ff6c006b2f0 msgr2=0x7ff6c00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 shutdown_connections 2023-07-13T22:29:38.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.189+0000 7ff6c7fff700 1 -- 172.21.15.60:0/2185649487 wait complete. 2023-07-13T22:29:38.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-13T22:29:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.417+0000 7fd2069ff700 1 Processor -- start 2023-07-13T22:29:38.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.417+0000 7fd2069ff700 1 -- start start 2023-07-13T22:29:38.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.418+0000 7fd2069ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd200070b30 0x7fd2000caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.418+0000 7fd2069ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2000ce770 0x7fd2000b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.418+0000 7fd2069ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20006bf40 con 0x7fd20015c460 2023-07-13T22:29:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.418+0000 7fd2069ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20006b9f0 con 0x7fd200070b30 2023-07-13T22:29:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.418+0000 7fd2069ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20006c750 con 0x7fd2000ce770 2023-07-13T22:29:38.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.418+0000 7fd1ff7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd200070b30 0x7fd2000caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.418+0000 7fd1ff7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd200070b30 0x7fd2000caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45320/0 (socket says 172.21.15.60:45320) 2023-07-13T22:29:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.418+0000 7fd1ff7fe700 1 -- 172.21.15.60:0/1225021479 learned_addr learned my addr 172.21.15.60:0/1225021479 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd204f9c700 1 --2- 172.21.15.60:0/1225021479 >> v2:172.21.15.105:3300/0 conn(0x7fd2000ce770 0x7fd2000b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd1ff7fe700 1 -- 172.21.15.60:0/1225021479 >> v2:172.21.15.105:3300/0 conn(0x7fd2000ce770 msgr2=0x7fd2000b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd1ff7fe700 1 --2- 172.21.15.60:0/1225021479 >> v2:172.21.15.105:3300/0 conn(0x7fd2000ce770 0x7fd2000b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd1ff7fe700 1 -- 172.21.15.60:0/1225021479 >> v1:172.21.15.105:6789/0 conn(0x7fd20015c460 legacy=0x7fd20006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd1ff7fe700 1 -- 172.21.15.60:0/1225021479 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2000cebf0 con 0x7fd200070b30 2023-07-13T22:29:38.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd204f9c700 1 --2- 172.21.15.60:0/1225021479 >> v2:172.21.15.105:3300/0 conn(0x7fd2000ce770 0x7fd2000b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:38.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd1ff7fe700 1 --2- 172.21.15.60:0/1225021479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 0x7fd2000caf80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fd200160e70 tx=0x7fd1f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844b63c9d1dd47fd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:38.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd1feffd700 1 -- 172.21.15.60:0/1225021479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400b870 con 0x7fd200070b30 2023-07-13T22:29:38.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd1feffd700 1 -- 172.21.15.60:0/1225021479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f400b9d0 con 0x7fd200070b30 2023-07-13T22:29:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.419+0000 7fd1feffd700 1 -- 172.21.15.60:0/1225021479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400bd60 con 0x7fd200070b30 2023-07-13T22:29:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.420+0000 7fd2069ff700 1 -- 172.21.15.60:0/1225021479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 msgr2=0x7fd2000caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.420+0000 7fd2069ff700 1 --2- 172.21.15.60:0/1225021479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 0x7fd2000caf80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fd200160e70 tx=0x7fd1f4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.420+0000 7fd2069ff700 1 -- 172.21.15.60:0/1225021479 shutdown_connections 2023-07-13T22:29:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.420+0000 7fd2069ff700 1 --2- 172.21.15.60:0/1225021479 >> v2:172.21.15.105:3300/0 conn(0x7fd2000ce770 0x7fd2000b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.420+0000 7fd2069ff700 1 --2- 172.21.15.60:0/1225021479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 0x7fd2000caf80 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.420+0000 7fd2069ff700 1 -- 172.21.15.60:0/1225021479 >> 172.21.15.60:0/1225021479 conn(0x7fd20006e580 msgr2=0x7fd20006e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:38.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.420+0000 7fd2069ff700 1 -- 172.21.15.60:0/1225021479 shutdown_connections 2023-07-13T22:29:38.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.420+0000 7fd2069ff700 1 -- 172.21.15.60:0/1225021479 wait complete. 2023-07-13T22:29:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.422+0000 7fd2069ff700 1 Processor -- start 2023-07-13T22:29:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.422+0000 7fd2069ff700 1 -- start start 2023-07-13T22:29:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd2069ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 0x7fd200157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd2069ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2000ce770 0x7fd2000ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd2069ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd20015c460 0x7fd200156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd2069ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000b4650 con 0x7fd200070b30 2023-07-13T22:29:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd2069ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20015b3a0 con 0x7fd20015c460 2023-07-13T22:29:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd2069ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000b48a0 con 0x7fd2000ce770 2023-07-13T22:29:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd1fffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 0x7fd200157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd1fffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 0x7fd200157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45336/0 (socket says 172.21.15.60:45336) 2023-07-13T22:29:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd1fffff700 1 -- 172.21.15.60:0/1157704054 learned_addr learned my addr 172.21.15.60:0/1157704054 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd204f9c700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd20015c460 0x7fd200156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.423+0000 7fd1ff7fe700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2000ce770 0x7fd2000ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd1fffff700 1 -- 172.21.15.60:0/1157704054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2000ce770 msgr2=0x7fd2000ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd1fffff700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2000ce770 0x7fd2000ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd1fffff700 1 -- 172.21.15.60:0/1157704054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd20015c460 msgr2=0x7fd200156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd1fffff700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd20015c460 0x7fd200156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd1fffff700 1 -- 172.21.15.60:0/1157704054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f4009550 con 0x7fd200070b30 2023-07-13T22:29:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd1fffff700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 0x7fd200157750 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000c4e0 tx=0x7fd1f000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd1fcff9700 1 -- 172.21.15.60:0/1157704054 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00052d0 con 0x7fd200070b30 2023-07-13T22:29:38.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd1fcff9700 1 -- 172.21.15.60:0/1157704054 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f0005430 con 0x7fd200070b30 2023-07-13T22:29:38.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd1fcff9700 1 -- 172.21.15.60:0/1157704054 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00038c0 con 0x7fd200070b30 2023-07-13T22:29:38.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.424+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2000b4b80 con 0x7fd200070b30 2023-07-13T22:29:38.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.425+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd200157ea0 con 0x7fd200070b30 2023-07-13T22:29:38.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.425+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2000b50a0 con 0x7fd200070b30 2023-07-13T22:29:38.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.427+0000 7fd1fcff9700 1 -- 172.21.15.60:0/1157704054 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd1f0003a20 con 0x7fd200070b30 2023-07-13T22:29:38.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.427+0000 7fd1fcff9700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1ec06ace0 0x7fd1ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.428+0000 7fd1fcff9700 1 -- 172.21.15.60:0/1157704054 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd1f0088510 con 0x7fd200070b30 2023-07-13T22:29:38.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.428+0000 7fd1ff7fe700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1ec06ace0 0x7fd1ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.428+0000 7fd1ff7fe700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1ec06ace0 0x7fd1ec06d190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:38.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.432+0000 7fd1fcff9700 1 -- 172.21.15.60:0/1157704054 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd1f00538d0 con 0x7fd200070b30 2023-07-13T22:29:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.642+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd20004e210 con 0x7fd200070b30 2023-07-13T22:29:38.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.646+0000 7fd1fcff9700 1 -- 172.21.15.60:0/1157704054 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd1f0056ef0 con 0x7fd200070b30 2023-07-13T22:29:38.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1ec06ace0 msgr2=0x7fd1ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1ec06ace0 0x7fd1ec06d190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f4008030 comp rx=0 tx=0).stop 2023-07-13T22:29:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 msgr2=0x7fd200157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 0x7fd200157750 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000c4e0 tx=0x7fd1f000c7f0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 shutdown_connections 2023-07-13T22:29:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd1ec06ace0 0x7fd1ec06d190 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd200070b30 0x7fd200157750 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd20015c460 0x7fd200156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 --2- 172.21.15.60:0/1157704054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2000ce770 0x7fd2000ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.648+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 >> 172.21.15.60:0/1157704054 conn(0x7fd20006e580 msgr2=0x7fd20015dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.649+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 shutdown_connections 2023-07-13T22:29:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.649+0000 7fd2069ff700 1 -- 172.21.15.60:0/1157704054 wait complete. 2023-07-13T22:29:38.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-07-13T22:29:38.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.874+0000 7f34217ff700 1 Processor -- start 2023-07-13T22:29:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.875+0000 7f34217ff700 1 -- start start 2023-07-13T22:29:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.876+0000 7f34217ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cfd50 0x7f341c0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.876+0000 7f34217ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f341c0b1ac0 0x7f341c0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.876+0000 7f34217ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c0d1870 con 0x7f341c0b8200 2023-07-13T22:29:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.876+0000 7f34217ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c0cf370 con 0x7f341c0b1ac0 2023-07-13T22:29:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.876+0000 7f34217ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0d12b0 con 0x7f341c0cfd50 2023-07-13T22:29:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.876+0000 7f341affd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f341c0b8200 0x7f341c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59970/0 (socket says 172.21.15.60:59970) 2023-07-13T22:29:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.876+0000 7f341affd700 1 -- 172.21.15.60:0/1417809111 learned_addr learned my addr 172.21.15.60:0/1417809111 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:38.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.876+0000 7f3413fff700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cfd50 0x7f341c0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.876+0000 7f341b7fe700 1 --2- 172.21.15.60:0/1417809111 >> v2:172.21.15.105:3300/0 conn(0x7f341c0b1ac0 0x7f341c0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906358982 0 0) 0x7f341c0d1870 con 0x7f341c0b8200 2023-07-13T22:29:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34080035b0 con 0x7f341c0b8200 2023-07-13T22:29:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477129294 0 0) 0x7f34080035b0 con 0x7f341c0b8200 2023-07-13T22:29:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cfd50 msgr2=0x7f341c0d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cfd50 0x7f341c0d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 >> v2:172.21.15.105:3300/0 conn(0x7f341c0b1ac0 msgr2=0x7f341c0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 --2- 172.21.15.60:0/1417809111 >> v2:172.21.15.105:3300/0 conn(0x7f341c0b1ac0 0x7f341c0d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c0d1590 con 0x7f341c0b8200 2023-07-13T22:29:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f340c001670 con 0x7f341c0b8200 2023-07-13T22:29:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 >> v1:172.21.15.60:6789/0 conn(0x7f341c0b8200 legacy=0x7f341c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3408003a00 0x7f3408003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3408004430 0x7f34080048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cfd50 0x7f3408004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0cf370 con 0x7f3408003a00 2023-07-13T22:29:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0d12b0 con 0x7f3408004430 2023-07-13T22:29:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340800ba10 con 0x7f341c0cfd50 2023-07-13T22:29:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341b7fe700 1 --2- 172.21.15.60:0/1417809111 >> v2:172.21.15.105:3300/0 conn(0x7f341c0b1ac0 0x7f341c0d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:38.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f341affd700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3408003a00 0x7f3408003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.877+0000 7f3413fff700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3408004430 0x7f34080048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341b7fe700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cfd50 0x7f3408004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341affd700 1 -- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cfd50 msgr2=0x7f3408004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:38.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341affd700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cfd50 0x7f3408004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341affd700 1 -- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3408004430 msgr2=0x7f34080048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:38.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341affd700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3408004430 0x7f34080048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341affd700 1 -- 172.21.15.60:0/1417809111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080035b0 con 0x7f3408003a00 2023-07-13T22:29:38.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341b7fe700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cfd50 0x7f3408004fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:38.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341affd700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3408003a00 0x7f3408003e10 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c2a0 tx=0x7f3408005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357f2b3109dd4956 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:38.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c007c10 con 0x7f3408003a00 2023-07-13T22:29:38.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c00cde0 con 0x7f3408003a00 2023-07-13T22:29:38.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.878+0000 7f341a7fc700 1 -- 172.21.15.60:0/1417809111 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c005ba0 con 0x7f3408003a00 2023-07-13T22:29:38.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.879+0000 7f34217ff700 1 -- 172.21.15.60:0/1417809111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3408003a00 msgr2=0x7f3408003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.879+0000 7f34217ff700 1 --2- 172.21.15.60:0/1417809111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3408003a00 0x7f3408003e10 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c2a0 tx=0x7f3408005880 comp rx=0 tx=0).stop 2023-07-13T22:29:38.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.879+0000 7f341b7fe700 1 -- 172.21.15.60:0/1417809111 reap_dead start 2023-07-13T22:29:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.879+0000 7f34217ff700 1 -- 172.21.15.60:0/1417809111 shutdown_connections 2023-07-13T22:29:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.879+0000 7f34217ff700 1 -- 172.21.15.60:0/1417809111 >> 172.21.15.60:0/1417809111 conn(0x7f341c06b2f0 msgr2=0x7f341c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.879+0000 7f34217ff700 1 -- 172.21.15.60:0/1417809111 shutdown_connections 2023-07-13T22:29:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.879+0000 7f34217ff700 1 -- 172.21.15.60:0/1417809111 wait complete. 2023-07-13T22:29:38.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.881+0000 7f34217ff700 1 Processor -- start 2023-07-13T22:29:38.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.881+0000 7f34217ff700 1 -- start start 2023-07-13T22:29:38.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f34217ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3408003a00 0x7f341c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f34217ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cc510 0x7f341c0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f34217ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f341c0c6590 0x7f341c0c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f34217ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0ccec0 con 0x7f341c0c6590 2023-07-13T22:29:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f34217ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c7bb0 con 0x7f3408003a00 2023-07-13T22:29:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f34217ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c9f30 con 0x7f341c0cc510 2023-07-13T22:29:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f341b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f341c0c6590 0x7f341c0c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f341b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f341c0c6590 0x7f341c0c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45342/0 (socket says 172.21.15.60:45342) 2023-07-13T22:29:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f341b7fe700 1 -- 172.21.15.60:0/3328028678 learned_addr learned my addr 172.21.15.60:0/3328028678 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f341affd700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3408003a00 0x7f341c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f3413fff700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cc510 0x7f341c0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f341b7fe700 1 -- 172.21.15.60:0/3328028678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cc510 msgr2=0x7f341c0cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f341b7fe700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cc510 0x7f341c0cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f341b7fe700 1 -- 172.21.15.60:0/3328028678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3408003a00 msgr2=0x7f341c150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f341b7fe700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3408003a00 0x7f341c150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.882+0000 7f341b7fe700 1 -- 172.21.15.60:0/3328028678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c0c7310 con 0x7f341c0c6590 2023-07-13T22:29:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.883+0000 7f341b7fe700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f341c0c6590 0x7f341c0c6a00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f341c0cd280 tx=0x7f34140116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:38.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.883+0000 7f34137fe700 1 -- 172.21.15.60:0/3328028678 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341400e030 con 0x7f341c0c6590 2023-07-13T22:29:38.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.883+0000 7f34137fe700 1 -- 172.21.15.60:0/3328028678 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414011c40 con 0x7f341c0c6590 2023-07-13T22:29:38.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.883+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c0c75f0 con 0x7f341c0c6590 2023-07-13T22:29:38.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.883+0000 7f34137fe700 1 -- 172.21.15.60:0/3328028678 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414016470 con 0x7f341c0c6590 2023-07-13T22:29:38.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.883+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c150f70 con 0x7f341c0c6590 2023-07-13T22:29:38.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.884+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c0c9d00 con 0x7f341c0c6590 2023-07-13T22:29:38.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.885+0000 7f34137fe700 1 -- 172.21.15.60:0/3328028678 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34140165d0 con 0x7f341c0c6590 2023-07-13T22:29:38.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.887+0000 7f34137fe700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33fc06ace0 0x7f33fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:38.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.887+0000 7f34137fe700 1 -- 172.21.15.60:0/3328028678 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f341400f070 con 0x7f341c0c6590 2023-07-13T22:29:38.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.887+0000 7f341affd700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33fc06ace0 0x7f33fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:38.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.887+0000 7f341affd700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33fc06ace0 0x7f33fc06d190 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f340c015040 tx=0x7f340c007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:38.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:38.891+0000 7f34137fe700 1 -- 172.21.15.60:0/3328028678 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f341c0c9d00 con 0x7f341c0c6590 2023-07-13T22:29:39.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.098+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7f341c0c9d00 con 0x7f341c0c6590 2023-07-13T22:29:39.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.102+0000 7f34137fe700 1 -- 172.21.15.60:0/3328028678 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v41) v1 ==== 105+0+0 (secure 0 0 0) 0x7f341c0c9d00 con 0x7f341c0c6590 2023-07-13T22:29:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33fc06ace0 msgr2=0x7f33fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33fc06ace0 0x7f33fc06d190 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f340c015040 tx=0x7f340c007000 comp rx=0 tx=0).stop 2023-07-13T22:29:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f341c0c6590 msgr2=0x7f341c0c6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f341c0c6590 0x7f341c0c6a00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f341c0cd280 tx=0x7f34140116e0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 shutdown_connections 2023-07-13T22:29:39.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f33fc06ace0 0x7f33fc06d190 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f341c0c6590 0x7f341c0c6a00 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3408003a00 0x7f341c150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 --2- 172.21.15.60:0/3328028678 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f341c0cc510 0x7f341c0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 >> 172.21.15.60:0/3328028678 conn(0x7f341c06b2f0 msgr2=0x7f341c0b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:39.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 shutdown_connections 2023-07-13T22:29:39.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.105+0000 7f34217ff700 1 -- 172.21.15.60:0/3328028678 wait complete. 2023-07-13T22:29:39.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-07-13T22:29:39.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.328+0000 7ff4793ff700 1 Processor -- start 2023-07-13T22:29:39.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.328+0000 7ff4793ff700 1 -- start start 2023-07-13T22:29:39.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.329+0000 7ff4793ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff4740d11c0 0x7ff4740cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.329+0000 7ff4793ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4740cf9f0 0x7ff474165eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.329+0000 7ff4793ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4740cefd0 con 0x7ff47415c530 2023-07-13T22:29:39.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.329+0000 7ff4793ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4740d1780 con 0x7ff4740cf9f0 2023-07-13T22:29:39.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.329+0000 7ff4793ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4740cd2d0 con 0x7ff4740d11c0 2023-07-13T22:29:39.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.329+0000 7ff4737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4740cf9f0 0x7ff474165eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:39.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.329+0000 7ff4737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4740cf9f0 0x7ff474165eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45344/0 (socket says 172.21.15.60:45344) 2023-07-13T22:29:39.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.329+0000 7ff4737fe700 1 -- 172.21.15.60:0/3687457548 learned_addr learned my addr 172.21.15.60:0/3687457548 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:39.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.329+0000 7ff4727fc700 1 --2- 172.21.15.60:0/3687457548 >> v2:172.21.15.105:3300/0 conn(0x7ff4740d11c0 0x7ff4740cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.330+0000 7ff4737fe700 1 -- 172.21.15.60:0/3687457548 >> v2:172.21.15.105:3300/0 conn(0x7ff4740d11c0 msgr2=0x7ff4740cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.330+0000 7ff4737fe700 1 --2- 172.21.15.60:0/3687457548 >> v2:172.21.15.105:3300/0 conn(0x7ff4740d11c0 0x7ff4740cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.330+0000 7ff4737fe700 1 -- 172.21.15.60:0/3687457548 >> v1:172.21.15.105:6789/0 conn(0x7ff47415c530 legacy=0x7ff4740b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.330+0000 7ff4737fe700 1 -- 172.21.15.60:0/3687457548 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740cd5b0 con 0x7ff4740cf9f0 2023-07-13T22:29:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.330+0000 7ff4737fe700 1 --2- 172.21.15.60:0/3687457548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740cf9f0 0x7ff474165eb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7ff47406c680 tx=0x7ff46c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719a3242be4c8d24 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:39.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.330+0000 7ff471ffb700 1 -- 172.21.15.60:0/3687457548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c00e070 con 0x7ff4740cf9f0 2023-07-13T22:29:39.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.330+0000 7ff471ffb700 1 -- 172.21.15.60:0/3687457548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c00b910 con 0x7ff4740cf9f0 2023-07-13T22:29:39.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.330+0000 7ff471ffb700 1 -- 172.21.15.60:0/3687457548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c00bcc0 con 0x7ff4740cf9f0 2023-07-13T22:29:39.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.331+0000 7ff4793ff700 1 -- 172.21.15.60:0/3687457548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740cf9f0 msgr2=0x7ff474165eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.331+0000 7ff4793ff700 1 --2- 172.21.15.60:0/3687457548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740cf9f0 0x7ff474165eb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7ff47406c680 tx=0x7ff46c00b330 comp rx=0 tx=0).stop 2023-07-13T22:29:39.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.331+0000 7ff4793ff700 1 -- 172.21.15.60:0/3687457548 shutdown_connections 2023-07-13T22:29:39.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.331+0000 7ff4793ff700 1 --2- 172.21.15.60:0/3687457548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740cf9f0 0x7ff474165eb0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.331+0000 7ff4793ff700 1 --2- 172.21.15.60:0/3687457548 >> v2:172.21.15.105:3300/0 conn(0x7ff4740d11c0 0x7ff4740cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.331+0000 7ff4793ff700 1 -- 172.21.15.60:0/3687457548 >> 172.21.15.60:0/3687457548 conn(0x7ff47406b400 msgr2=0x7ff47406b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:39.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.331+0000 7ff4793ff700 1 -- 172.21.15.60:0/3687457548 shutdown_connections 2023-07-13T22:29:39.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.331+0000 7ff4793ff700 1 -- 172.21.15.60:0/3687457548 wait complete. 2023-07-13T22:29:39.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.333+0000 7ff4793ff700 1 Processor -- start 2023-07-13T22:29:39.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.333+0000 7ff4793ff700 1 -- start start 2023-07-13T22:29:39.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4793ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cf9f0 0x7ff47414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4793ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740d11c0 0x7ff4740cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4793ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47414dde0 0x7ff4741f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4793ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47415f790 con 0x7ff4740d11c0 2023-07-13T22:29:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4793ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47415ca00 con 0x7ff4740cf9f0 2023-07-13T22:29:39.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4793ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47414e260 con 0x7ff47414dde0 2023-07-13T22:29:39.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4727fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740d11c0 0x7ff4740cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:39.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4727fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740d11c0 0x7ff4740cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45352/0 (socket says 172.21.15.60:45352) 2023-07-13T22:29:39.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4727fc700 1 -- 172.21.15.60:0/746916182 learned_addr learned my addr 172.21.15.60:0/746916182 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:39.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4727fc700 1 -- 172.21.15.60:0/746916182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47414dde0 msgr2=0x7ff4741f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4727fc700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47414dde0 0x7ff4741f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4727fc700 1 -- 172.21.15.60:0/746916182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cf9f0 msgr2=0x7ff47414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4727fc700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cf9f0 0x7ff47414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.334+0000 7ff4727fc700 1 -- 172.21.15.60:0/746916182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c009b30 con 0x7ff4740d11c0 2023-07-13T22:29:39.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.335+0000 7ff4727fc700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740d11c0 0x7ff4740cc240 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9b0 tx=0x7ff46800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:39.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.335+0000 7ff45b7fe700 1 -- 172.21.15.60:0/746916182 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468009710 con 0x7ff4740d11c0 2023-07-13T22:29:39.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.335+0000 7ff45b7fe700 1 -- 172.21.15.60:0/746916182 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46800c850 con 0x7ff4740d11c0 2023-07-13T22:29:39.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.335+0000 7ff45b7fe700 1 -- 172.21.15.60:0/746916182 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800cbc0 con 0x7ff4740d11c0 2023-07-13T22:29:39.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.335+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4740cc930 con 0x7ff4740d11c0 2023-07-13T22:29:39.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.335+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47415cf80 con 0x7ff4740d11c0 2023-07-13T22:29:39.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.336+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47415d1a0 con 0x7ff4740d11c0 2023-07-13T22:29:39.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.337+0000 7ff45b7fe700 1 -- 172.21.15.60:0/746916182 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff46800cd20 con 0x7ff4740d11c0 2023-07-13T22:29:39.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.338+0000 7ff45b7fe700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff46006ace0 0x7ff46006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.338+0000 7ff45b7fe700 1 -- 172.21.15.60:0/746916182 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff468089310 con 0x7ff4740d11c0 2023-07-13T22:29:39.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.338+0000 7ff472ffd700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff46006ace0 0x7ff46006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:39.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.339+0000 7ff472ffd700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff46006ace0 0x7ff46006d190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff464000f60 tx=0x7ff464007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:39.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.342+0000 7ff45b7fe700 1 -- 172.21.15.60:0/746916182 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff4680580d0 con 0x7ff4740d11c0 2023-07-13T22:29:39.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.554+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7ff47404e210 con 0x7ff4740d11c0 2023-07-13T22:29:39.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.558+0000 7ff45b7fe700 1 -- 172.21.15.60:0/746916182 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v42) v1 ==== 111+0+0 (secure 0 0 0) 0x7ff468057c60 con 0x7ff4740d11c0 2023-07-13T22:29:39.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff46006ace0 msgr2=0x7ff46006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff46006ace0 0x7ff46006d190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff464000f60 tx=0x7ff464007a40 comp rx=0 tx=0).stop 2023-07-13T22:29:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740d11c0 msgr2=0x7ff4740cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740d11c0 0x7ff4740cc240 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9b0 tx=0x7ff46800ed70 comp rx=0 tx=0).stop 2023-07-13T22:29:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 shutdown_connections 2023-07-13T22:29:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff46006ace0 0x7ff46006d190 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff47414dde0 0x7ff4741f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4740cf9f0 0x7ff47414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 --2- 172.21.15.60:0/746916182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4740d11c0 0x7ff4740cc240 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 >> 172.21.15.60:0/746916182 conn(0x7ff47406b400 msgr2=0x7ff4740c92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:39.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 shutdown_connections 2023-07-13T22:29:39.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.561+0000 7ff4793ff700 1 -- 172.21.15.60:0/746916182 wait complete. 2023-07-13T22:29:39.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-07-13T22:29:39.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.785+0000 7f39ae1ff700 1 Processor -- start 2023-07-13T22:29:39.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.786+0000 7f39ae1ff700 1 -- start start 2023-07-13T22:29:39.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.787+0000 7f39ae1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 0x7f39a80d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.787+0000 7f39ae1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a80d1870 con 0x7f39a80b8200 2023-07-13T22:29:39.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.787+0000 7f39ae1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a80cf370 con 0x7f39a80b1ac0 2023-07-13T22:29:39.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.787+0000 7f39ae1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80d12b0 con 0x7f39a80cfd50 2023-07-13T22:29:39.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.787+0000 7f39a77fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f39a80b8200 0x7f39a80cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59974/0 (socket says 172.21.15.60:59974) 2023-07-13T22:29:39.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.787+0000 7f39a77fe700 1 -- 172.21.15.60:0/114516600 learned_addr learned my addr 172.21.15.60:0/114516600 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.787+0000 7f399ffff700 1 --2- 172.21.15.60:0/114516600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 0x7f39a80d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.787+0000 7f39a6ffd700 1 -- 172.21.15.60:0/114516600 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060547603 0 0) 0x7f39a80d1870 con 0x7f39a80b8200 2023-07-13T22:29:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f39a6ffd700 1 -- 172.21.15.60:0/114516600 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39940035b0 con 0x7f39a80b8200 2023-07-13T22:29:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f39a6ffd700 1 -- 172.21.15.60:0/114516600 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251218922 0 0) 0x7f39a80cf370 con 0x7f39a80b1ac0 2023-07-13T22:29:39.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f39a6ffd700 1 -- 172.21.15.60:0/114516600 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39a80d1870 con 0x7f39a80b1ac0 2023-07-13T22:29:39.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f399ffff700 1 -- 172.21.15.60:0/114516600 >> v1:172.21.15.105:6789/0 conn(0x7f39a80b1ac0 legacy=0x7f39a80d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f399ffff700 1 -- 172.21.15.60:0/114516600 >> v1:172.21.15.60:6789/0 conn(0x7f39a80b8200 legacy=0x7f39a80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f399ffff700 1 -- 172.21.15.60:0/114516600 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80d1590 con 0x7f39a80cfd50 2023-07-13T22:29:39.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f399ffff700 1 --2- 172.21.15.60:0/114516600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 0x7f39a80d23c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f39900098a0 tx=0x7f3990009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f79ca17abc4e81 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:39.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f39a6ffd700 1 -- 172.21.15.60:0/114516600 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990011830 con 0x7f39a80cfd50 2023-07-13T22:29:39.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f39a6ffd700 1 -- 172.21.15.60:0/114516600 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3990011990 con 0x7f39a80cfd50 2023-07-13T22:29:39.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.788+0000 7f39a6ffd700 1 -- 172.21.15.60:0/114516600 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990011d20 con 0x7f39a80cfd50 2023-07-13T22:29:39.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.789+0000 7f39ae1ff700 1 -- 172.21.15.60:0/114516600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 msgr2=0x7f39a80d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.789+0000 7f39ae1ff700 1 --2- 172.21.15.60:0/114516600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 0x7f39a80d23c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f39900098a0 tx=0x7f3990009bb0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.789+0000 7f39ae1ff700 1 -- 172.21.15.60:0/114516600 shutdown_connections 2023-07-13T22:29:39.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.789+0000 7f39ae1ff700 1 --2- 172.21.15.60:0/114516600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 0x7f39a80d23c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.789+0000 7f39ae1ff700 1 -- 172.21.15.60:0/114516600 >> 172.21.15.60:0/114516600 conn(0x7f39a806b2f0 msgr2=0x7f39a806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:39.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.789+0000 7f39ae1ff700 1 -- 172.21.15.60:0/114516600 shutdown_connections 2023-07-13T22:29:39.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.789+0000 7f39ae1ff700 1 -- 172.21.15.60:0/114516600 wait complete. 2023-07-13T22:29:39.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.791+0000 7f39ae1ff700 1 Processor -- start 2023-07-13T22:29:39.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.791+0000 7f39ae1ff700 1 -- start start 2023-07-13T22:29:39.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39ae1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a80b1ac0 0x7f39a80c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39ae1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39a80b8200 0x7f39a815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39ae1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 0x7f39a80cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39ae1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80c6fb0 con 0x7f39a80b1ac0 2023-07-13T22:29:39.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39ae1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80c7290 con 0x7f39a80b8200 2023-07-13T22:29:39.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39ae1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80c7570 con 0x7f39a80cfd50 2023-07-13T22:29:39.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39a77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a80b1ac0 0x7f39a80c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:39.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39a77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a80b1ac0 0x7f39a80c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45362/0 (socket says 172.21.15.60:45362) 2023-07-13T22:29:39.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39a77fe700 1 -- 172.21.15.60:0/1469038640 learned_addr learned my addr 172.21.15.60:0/1469038640 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:39.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f399ffff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39a80b8200 0x7f39a815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:39.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.792+0000 7f39a7fff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 0x7f39a80cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:39.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39a77fe700 1 -- 172.21.15.60:0/1469038640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 msgr2=0x7f39a80cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39a77fe700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 0x7f39a80cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39a77fe700 1 -- 172.21.15.60:0/1469038640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39a80b8200 msgr2=0x7f39a815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39a77fe700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39a80b8200 0x7f39a815a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39a77fe700 1 -- 172.21.15.60:0/1469038640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39900094a0 con 0x7f39a80b1ac0 2023-07-13T22:29:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39a77fe700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a80b1ac0 0x7f39a80c8050 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f399800c700 tx=0x7f399800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39a4ff9700 1 -- 172.21.15.60:0/1469038640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998005d70 con 0x7f39a80b1ac0 2023-07-13T22:29:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39a4ff9700 1 -- 172.21.15.60:0/1469038640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39980070d0 con 0x7f39a80b1ac0 2023-07-13T22:29:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39a4ff9700 1 -- 172.21.15.60:0/1469038640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998007460 con 0x7f39a80b1ac0 2023-07-13T22:29:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a80c7850 con 0x7f39a80b1ac0 2023-07-13T22:29:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.793+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a815aa80 con 0x7f39a80b1ac0 2023-07-13T22:29:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.794+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a804e210 con 0x7f39a80b1ac0 2023-07-13T22:29:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.795+0000 7f39a4ff9700 1 -- 172.21.15.60:0/1469038640 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39980075c0 con 0x7f39a80b1ac0 2023-07-13T22:29:39.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.796+0000 7f39a4ff9700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f398806ace0 0x7f398806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:39.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.796+0000 7f39a4ff9700 1 -- 172.21.15.60:0/1469038640 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3998088660 con 0x7f39a80b1ac0 2023-07-13T22:29:39.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.796+0000 7f399ffff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f398806ace0 0x7f398806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:39.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.797+0000 7f399ffff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f398806ace0 0x7f398806d190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f39a80d0140 tx=0x7f399001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:39.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:39.801+0000 7f39a4ff9700 1 -- 172.21.15.60:0/1469038640 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3998053990 con 0x7f39a80b1ac0 2023-07-13T22:29:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.004+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7f39a80b4f70 con 0x7f39a80b1ac0 2023-07-13T22:29:39.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.008+0000 7f39a4ff9700 1 -- 172.21.15.60:0/1469038640 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v43) v1 ==== 107+0+0 (secure 0 0 0) 0x7f3998056fb0 con 0x7f39a80b1ac0 2023-07-13T22:29:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f398806ace0 msgr2=0x7f398806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f398806ace0 0x7f398806d190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f39a80d0140 tx=0x7f399001c040 comp rx=0 tx=0).stop 2023-07-13T22:29:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a80b1ac0 msgr2=0x7f39a80c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a80b1ac0 0x7f39a80c8050 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f399800c700 tx=0x7f399800ca10 comp rx=0 tx=0).stop 2023-07-13T22:29:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 shutdown_connections 2023-07-13T22:29:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f398806ace0 0x7f398806d190 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a80b1ac0 0x7f39a80c8050 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39a80cfd50 0x7f39a80cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 --2- 172.21.15.60:0/1469038640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39a80b8200 0x7f39a815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 >> 172.21.15.60:0/1469038640 conn(0x7f39a806b2f0 msgr2=0x7f39a806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 shutdown_connections 2023-07-13T22:29:39.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.011+0000 7f39ae1ff700 1 -- 172.21.15.60:0/1469038640 wait complete. 2023-07-13T22:29:40.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2023-07-13T22:29:40.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.233+0000 7f4d211ff700 1 Processor -- start 2023-07-13T22:29:40.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.233+0000 7f4d211ff700 1 -- start start 2023-07-13T22:29:40.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.234+0000 7f4d211ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d1c0d0e20 0x7f4d1c0d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.234+0000 7f4d211ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0d2410 con 0x7f4d1c0cd260 2023-07-13T22:29:40.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.234+0000 7f4d211ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0d2660 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.234+0000 7f4d211ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c15c900 con 0x7f4d1c0d0e20 2023-07-13T22:29:40.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.235+0000 7f4d1b59e700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d1c0d0e20 0x7f4d1c0d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.235+0000 7f4d1a59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4d1c0cd260 0x7f4d1c0d3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59988/0 (socket says 172.21.15.60:59988) 2023-07-13T22:29:40.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.235+0000 7f4d1a59c700 1 -- 172.21.15.60:0/455740665 learned_addr learned my addr 172.21.15.60:0/455740665 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:40.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.235+0000 7f4d19d9b700 1 -- 172.21.15.60:0/455740665 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221172939 0 0) 0x7f4d1c0d2410 con 0x7f4d1c0cd260 2023-07-13T22:29:40.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.235+0000 7f4d19d9b700 1 -- 172.21.15.60:0/455740665 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d040035b0 con 0x7f4d1c0cd260 2023-07-13T22:29:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.235+0000 7f4d1b59e700 1 -- 172.21.15.60:0/455740665 >> v1:172.21.15.105:6789/0 conn(0x7f4d1c0d0a50 legacy=0x7f4d1c0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.235+0000 7f4d1b59e700 1 -- 172.21.15.60:0/455740665 >> v1:172.21.15.60:6789/0 conn(0x7f4d1c0cd260 legacy=0x7f4d1c0d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.235+0000 7f4d1b59e700 1 -- 172.21.15.60:0/455740665 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0d2940 con 0x7f4d1c0d0e20 2023-07-13T22:29:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.235+0000 7f4d1b59e700 1 --2- 172.21.15.60:0/455740665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0e20 0x7f4d1c0d1240 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d90 tx=0x7f4d1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def0a8e097ff6936 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d19d9b700 1 -- 172.21.15.60:0/455740665 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400e070 con 0x7f4d1c0d0e20 2023-07-13T22:29:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d19d9b700 1 -- 172.21.15.60:0/455740665 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400bb20 con 0x7f4d1c0d0e20 2023-07-13T22:29:40.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d19d9b700 1 -- 172.21.15.60:0/455740665 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14012430 con 0x7f4d1c0d0e20 2023-07-13T22:29:40.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d211ff700 1 -- 172.21.15.60:0/455740665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0e20 msgr2=0x7f4d1c0d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d211ff700 1 --2- 172.21.15.60:0/455740665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0e20 0x7f4d1c0d1240 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d90 tx=0x7f4d1400b330 comp rx=0 tx=0).stop 2023-07-13T22:29:40.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d211ff700 1 -- 172.21.15.60:0/455740665 shutdown_connections 2023-07-13T22:29:40.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d211ff700 1 --2- 172.21.15.60:0/455740665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0e20 0x7f4d1c0d1240 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d211ff700 1 -- 172.21.15.60:0/455740665 >> 172.21.15.60:0/455740665 conn(0x7f4d1c071600 msgr2=0x7f4d1c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:40.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d211ff700 1 -- 172.21.15.60:0/455740665 shutdown_connections 2023-07-13T22:29:40.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.236+0000 7f4d211ff700 1 -- 172.21.15.60:0/455740665 wait complete. 2023-07-13T22:29:40.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.239+0000 7f4d211ff700 1 Processor -- start 2023-07-13T22:29:40.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.239+0000 7f4d211ff700 1 -- start start 2023-07-13T22:29:40.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.240+0000 7f4d211ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0cd260 0x7f4d1c1f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.240+0000 7f4d211ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0a50 0x7f4d1c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.240+0000 7f4d211ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d1c0d0e20 0x7f4d1c06b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.240+0000 7f4d211ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c141950 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.240+0000 7f4d211ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c141c10 con 0x7f4d1c0cd260 2023-07-13T22:29:40.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.240+0000 7f4d211ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c141ef0 con 0x7f4d1c0d0e20 2023-07-13T22:29:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0a50 0x7f4d1c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0a50 0x7f4d1c15f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45376/0 (socket says 172.21.15.60:45376) 2023-07-13T22:29:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1a59c700 1 -- 172.21.15.60:0/3180577408 learned_addr learned my addr 172.21.15.60:0/3180577408 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1b59e700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d1c0d0e20 0x7f4d1c06b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1ad9d700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0cd260 0x7f4d1c1f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1a59c700 1 -- 172.21.15.60:0/3180577408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d1c0d0e20 msgr2=0x7f4d1c06b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1a59c700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d1c0d0e20 0x7f4d1c06b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1a59c700 1 -- 172.21.15.60:0/3180577408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0cd260 msgr2=0x7f4d1c1f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1a59c700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0cd260 0x7f4d1c1f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1a59c700 1 -- 172.21.15.60:0/3180577408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14009a70 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1ad9d700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0cd260 0x7f4d1c1f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.241+0000 7f4d1a59c700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0a50 0x7f4d1c15f0d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c700 tx=0x7f4d1000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.242+0000 7f4d037fe700 1 -- 172.21.15.60:0/3180577408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10005d70 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.242+0000 7f4d037fe700 1 -- 172.21.15.60:0/3180577408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d100070d0 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.242+0000 7f4d037fe700 1 -- 172.21.15.60:0/3180577408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10007460 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.242+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c1421d0 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.242+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c1f19e0 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.243+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c142820 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.244+0000 7f4d037fe700 1 -- 172.21.15.60:0/3180577408 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d100075c0 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.245+0000 7f4d037fe700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d0806ace0 0x7f4d0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.245+0000 7f4d037fe700 1 -- 172.21.15.60:0/3180577408 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4d100887c0 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.245+0000 7f4d1ad9d700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d0806ace0 0x7f4d0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.246+0000 7f4d1ad9d700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d0806ace0 0x7f4d0806d190 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000f60 tx=0x7f4d0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.249+0000 7f4d037fe700 1 -- 172.21.15.60:0/3180577408 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4d10053af0 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.547+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) v1 -- 0x7f4d1c04e280 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.730+0000 7f4d037fe700 1 -- 172.21.15.60:0/3180577408 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v444) v1 ==== 143+0+0 (secure 0 0 0) 0x7f4d10057110 con 0x7f4d1c0d0a50 2023-07-13T22:29:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.732+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d0806ace0 msgr2=0x7f4d0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.732+0000 7f4d211ff700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d0806ace0 0x7f4d0806d190 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000f60 tx=0x7f4d0c009040 comp rx=0 tx=0).stop 2023-07-13T22:29:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.732+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0a50 msgr2=0x7f4d1c15f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.732+0000 7f4d211ff700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0a50 0x7f4d1c15f0d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c700 tx=0x7f4d1000ca10 comp rx=0 tx=0).stop 2023-07-13T22:29:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.733+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 shutdown_connections 2023-07-13T22:29:40.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.733+0000 7f4d211ff700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4d0806ace0 0x7f4d0806d190 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.733+0000 7f4d211ff700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d1c0d0e20 0x7f4d1c06b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.733+0000 7f4d211ff700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0cd260 0x7f4d1c1f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.733+0000 7f4d211ff700 1 --2- 172.21.15.60:0/3180577408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d1c0d0a50 0x7f4d1c15f0d0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.733+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 >> 172.21.15.60:0/3180577408 conn(0x7f4d1c071600 msgr2=0x7f4d1c158600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:40.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.733+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 shutdown_connections 2023-07-13T22:29:40.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.733+0000 7f4d211ff700 1 -- 172.21.15.60:0/3180577408 wait complete. 2023-07-13T22:29:40.731 INFO:tasks.cephfs_test_runner:test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-13T22:29:40.731 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:29:40.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2023-07-13T22:29:40.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.955+0000 7fc3cdfff700 1 Processor -- start 2023-07-13T22:29:40.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.956+0000 7fc3cdfff700 1 -- start start 2023-07-13T22:29:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.957+0000 7fc3cdfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc3c80d0e20 0x7fc3c80d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.957+0000 7fc3cdfff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c80d2410 con 0x7fc3c80cd260 2023-07-13T22:29:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.957+0000 7fc3cdfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c80d2660 con 0x7fc3c80d0a50 2023-07-13T22:29:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.957+0000 7fc3cdfff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c815c900 con 0x7fc3c80d0e20 2023-07-13T22:29:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.957+0000 7fc3c6ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc3c80cd260 0x7fc3c80d3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59994/0 (socket says 172.21.15.60:59994) 2023-07-13T22:29:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.957+0000 7fc3c6ffd700 1 -- 172.21.15.60:0/1196084204 learned_addr learned my addr 172.21.15.60:0/1196084204 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.957+0000 7fc3c7fff700 1 --2- 172.21.15.60:0/1196084204 >> v2:172.21.15.105:3300/0 conn(0x7fc3c80d0e20 0x7fc3c80d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.957+0000 7fc3c67fc700 1 -- 172.21.15.60:0/1196084204 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388335196 0 0) 0x7fc3c80d2410 con 0x7fc3c80cd260 2023-07-13T22:29:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c67fc700 1 -- 172.21.15.60:0/1196084204 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3b00035b0 con 0x7fc3c80cd260 2023-07-13T22:29:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c67fc700 1 -- 172.21.15.60:0/1196084204 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918743440 0 0) 0x7fc3c80d2660 con 0x7fc3c80d0a50 2023-07-13T22:29:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c67fc700 1 -- 172.21.15.60:0/1196084204 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3c80d2410 con 0x7fc3c80d0a50 2023-07-13T22:29:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c67fc700 1 -- 172.21.15.60:0/1196084204 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2999570629 0 0) 0x7fc3b00035b0 con 0x7fc3c80cd260 2023-07-13T22:29:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c7fff700 1 -- 172.21.15.60:0/1196084204 >> v1:172.21.15.105:6789/0 conn(0x7fc3c80d0a50 legacy=0x7fc3c80cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c7fff700 1 -- 172.21.15.60:0/1196084204 >> v1:172.21.15.60:6789/0 conn(0x7fc3c80cd260 legacy=0x7fc3c80d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c7fff700 1 -- 172.21.15.60:0/1196084204 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c80d2940 con 0x7fc3c80d0e20 2023-07-13T22:29:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c7fff700 1 --2- 172.21.15.60:0/1196084204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c80d0e20 0x7fc3c80d1240 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009d90 tx=0x7fc3c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2413509e5013d85 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:40.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c67fc700 1 -- 172.21.15.60:0/1196084204 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000e070 con 0x7fc3c80d0e20 2023-07-13T22:29:40.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c67fc700 1 -- 172.21.15.60:0/1196084204 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c0003a20 con 0x7fc3c80d0e20 2023-07-13T22:29:40.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.958+0000 7fc3c67fc700 1 -- 172.21.15.60:0/1196084204 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc001670 con 0x7fc3c80d0e20 2023-07-13T22:29:40.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.959+0000 7fc3cdfff700 1 -- 172.21.15.60:0/1196084204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c80d0e20 msgr2=0x7fc3c80d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.959+0000 7fc3cdfff700 1 --2- 172.21.15.60:0/1196084204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c80d0e20 0x7fc3c80d1240 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009d90 tx=0x7fc3c000b330 comp rx=0 tx=0).stop 2023-07-13T22:29:40.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.959+0000 7fc3cdfff700 1 -- 172.21.15.60:0/1196084204 shutdown_connections 2023-07-13T22:29:40.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.959+0000 7fc3cdfff700 1 --2- 172.21.15.60:0/1196084204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c80d0e20 0x7fc3c80d1240 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.959+0000 7fc3cdfff700 1 -- 172.21.15.60:0/1196084204 >> 172.21.15.60:0/1196084204 conn(0x7fc3c8071600 msgr2=0x7fc3c80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:40.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.959+0000 7fc3cdfff700 1 -- 172.21.15.60:0/1196084204 shutdown_connections 2023-07-13T22:29:40.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.959+0000 7fc3cdfff700 1 -- 172.21.15.60:0/1196084204 wait complete. 2023-07-13T22:29:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.961+0000 7fc3cdfff700 1 Processor -- start 2023-07-13T22:29:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.961+0000 7fc3cdfff700 1 -- start start 2023-07-13T22:29:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3cdfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c80cd260 0x7fc3c80c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3cdfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c80d0a50 0x7fc3c81f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3cdfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d0e20 0x7fc3c80c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3cdfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c814bba0 con 0x7fc3c80d0e20 2023-07-13T22:29:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3cdfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80cc1d0 con 0x7fc3c80cd260 2023-07-13T22:29:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3cdfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80cc4b0 con 0x7fc3c80d0a50 2023-07-13T22:29:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3c7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d0e20 0x7fc3c80c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3c7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d0e20 0x7fc3c80c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45380/0 (socket says 172.21.15.60:45380) 2023-07-13T22:29:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3c7fff700 1 -- 172.21.15.60:0/51864541 learned_addr learned my addr 172.21.15.60:0/51864541 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3c6ffd700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c80d0a50 0x7fc3c81f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.962+0000 7fc3c77fe700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c80cd260 0x7fc3c80c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3c7fff700 1 -- 172.21.15.60:0/51864541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c80d0a50 msgr2=0x7fc3c81f15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3c7fff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c80d0a50 0x7fc3c81f15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3c7fff700 1 -- 172.21.15.60:0/51864541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c80cd260 msgr2=0x7fc3c80c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3c7fff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c80cd260 0x7fc3c80c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3c7fff700 1 -- 172.21.15.60:0/51864541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c0009a70 con 0x7fc3c80d0e20 2023-07-13T22:29:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3c7fff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d0e20 0x7fc3c80c90f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0015040 tx=0x7fc3c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3affff700 1 -- 172.21.15.60:0/51864541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c00044c0 con 0x7fc3c80d0e20 2023-07-13T22:29:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3affff700 1 -- 172.21.15.60:0/51864541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c0004620 con 0x7fc3c80d0e20 2023-07-13T22:29:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3affff700 1 -- 172.21.15.60:0/51864541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0012430 con 0x7fc3c80d0e20 2023-07-13T22:29:40.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.963+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c80cc790 con 0x7fc3c80d0e20 2023-07-13T22:29:40.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.964+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c80ccd50 con 0x7fc3c80d0e20 2023-07-13T22:29:40.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.965+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c80c9ad0 con 0x7fc3c80d0e20 2023-07-13T22:29:40.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.966+0000 7fc3affff700 1 -- 172.21.15.60:0/51864541 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3c0012590 con 0x7fc3c80d0e20 2023-07-13T22:29:40.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.966+0000 7fc3affff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3b406ace0 0x7fc3b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:40.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.966+0000 7fc3affff700 1 -- 172.21.15.60:0/51864541 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc3c008a570 con 0x7fc3c80d0e20 2023-07-13T22:29:40.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.967+0000 7fc3c77fe700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3b406ace0 0x7fc3b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:40.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.967+0000 7fc3c77fe700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3b406ace0 0x7fc3b406d190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80011e0 tx=0x7fc3b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:40.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:40.971+0000 7fc3affff700 1 -- 172.21.15.60:0/51864541 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc3c00559b0 con 0x7fc3c80d0e20 2023-07-13T22:29:41.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.261+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) v1 -- 0x7fc3c804e280 con 0x7fc3c80d0e20 2023-07-13T22:29:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.734+0000 7fc3affff700 1 -- 172.21.15.60:0/51864541 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v445) v1 ==== 154+0+0 (secure 0 0 0) 0x7fc3c0058fd0 con 0x7fc3c80d0e20 2023-07-13T22:29:41.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3b406ace0 msgr2=0x7fc3b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3b406ace0 0x7fc3b406d190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80011e0 tx=0x7fc3b8009040 comp rx=0 tx=0).stop 2023-07-13T22:29:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d0e20 msgr2=0x7fc3c80c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d0e20 0x7fc3c80c90f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0015040 tx=0x7fc3c0004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 shutdown_connections 2023-07-13T22:29:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3b406ace0 0x7fc3b406d190 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d0e20 0x7fc3c80c90f0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c80cd260 0x7fc3c80c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 --2- 172.21.15.60:0/51864541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c80d0a50 0x7fc3c81f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 >> 172.21.15.60:0/51864541 conn(0x7fc3c8071600 msgr2=0x7fc3c815ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 shutdown_connections 2023-07-13T22:29:41.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.737+0000 7fc3cdfff700 1 -- 172.21.15.60:0/51864541 wait complete. 2023-07-13T22:29:41.737 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-13T22:29:41.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.965+0000 7f38a75ff700 1 Processor -- start 2023-07-13T22:29:41.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.966+0000 7f38a75ff700 1 -- start start 2023-07-13T22:29:41.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.967+0000 7f38a75ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38a00b8200 0x7f38a00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:41.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.967+0000 7f38a75ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f38a00cede0 0x7f38a00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:41.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.967+0000 7f38a75ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a00d1870 con 0x7f38a00b1ac0 2023-07-13T22:29:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.967+0000 7f38a75ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a00cf370 con 0x7f38a00cede0 2023-07-13T22:29:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.967+0000 7f38a75ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00d12b0 con 0x7f38a00b8200 2023-07-13T22:29:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.967+0000 7f38a4b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f38a00b1ac0 0x7f38a015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59996/0 (socket says 172.21.15.60:59996) 2023-07-13T22:29:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.967+0000 7f38a4b9a700 1 -- 172.21.15.60:0/1997777212 learned_addr learned my addr 172.21.15.60:0/1997777212 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.967+0000 7f38a5b9c700 1 --2- 172.21.15.60:0/1997777212 >> v2:172.21.15.105:3300/0 conn(0x7f38a00cede0 0x7f38a00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f38a539b700 1 --2- 172.21.15.60:0/1997777212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38a00b8200 0x7f38a00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f3897fff700 1 -- 172.21.15.60:0/1997777212 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274666752 0 0) 0x7f38a00d1870 con 0x7f38a00b1ac0 2023-07-13T22:29:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f3897fff700 1 -- 172.21.15.60:0/1997777212 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f388c0035b0 con 0x7f38a00b1ac0 2023-07-13T22:29:41.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f38a5b9c700 1 -- 172.21.15.60:0/1997777212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38a00b8200 msgr2=0x7f38a00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:41.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f38a5b9c700 1 --2- 172.21.15.60:0/1997777212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38a00b8200 0x7f38a00cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f38a5b9c700 1 -- 172.21.15.60:0/1997777212 >> v1:172.21.15.60:6789/0 conn(0x7f38a00b1ac0 legacy=0x7f38a015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:41.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f38a5b9c700 1 -- 172.21.15.60:0/1997777212 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00d1590 con 0x7f38a00cede0 2023-07-13T22:29:41.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f38a5b9c700 1 --2- 172.21.15.60:0/1997777212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00cede0 0x7f38a00cfb80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3898009d90 tx=0x7f389800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b79c5dec174ad4c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:41.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f3897fff700 1 -- 172.21.15.60:0/1997777212 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389800e070 con 0x7f38a00cede0 2023-07-13T22:29:41.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f3897fff700 1 -- 172.21.15.60:0/1997777212 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389800bb20 con 0x7f38a00cede0 2023-07-13T22:29:41.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.968+0000 7f3897fff700 1 -- 172.21.15.60:0/1997777212 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898012430 con 0x7f38a00cede0 2023-07-13T22:29:41.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.969+0000 7f38a75ff700 1 -- 172.21.15.60:0/1997777212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00cede0 msgr2=0x7f38a00cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:41.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.969+0000 7f38a75ff700 1 --2- 172.21.15.60:0/1997777212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00cede0 0x7f38a00cfb80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3898009d90 tx=0x7f389800b330 comp rx=0 tx=0).stop 2023-07-13T22:29:41.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.969+0000 7f38a75ff700 1 -- 172.21.15.60:0/1997777212 shutdown_connections 2023-07-13T22:29:41.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.969+0000 7f38a75ff700 1 --2- 172.21.15.60:0/1997777212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00cede0 0x7f38a00cfb80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.969+0000 7f38a75ff700 1 --2- 172.21.15.60:0/1997777212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38a00b8200 0x7f38a00cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.969+0000 7f38a75ff700 1 -- 172.21.15.60:0/1997777212 >> 172.21.15.60:0/1997777212 conn(0x7f38a006b2f0 msgr2=0x7f38a006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.969+0000 7f38a75ff700 1 -- 172.21.15.60:0/1997777212 shutdown_connections 2023-07-13T22:29:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.969+0000 7f38a75ff700 1 -- 172.21.15.60:0/1997777212 wait complete. 2023-07-13T22:29:41.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.971+0000 7f38a75ff700 1 Processor -- start 2023-07-13T22:29:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.971+0000 7f38a75ff700 1 -- start start 2023-07-13T22:29:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.972+0000 7f38a75ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38a00b1ac0 0x7f38a00c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.972+0000 7f38a75ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00b8200 0x7f38a00c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.972+0000 7f38a75ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38a00cede0 0x7f38a00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.972+0000 7f38a75ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00c7f50 con 0x7f38a00b1ac0 2023-07-13T22:29:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.972+0000 7f38a75ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00c8ee0 con 0x7f38a00b8200 2023-07-13T22:29:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.972+0000 7f38a75ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00c72d0 con 0x7f38a00cede0 2023-07-13T22:29:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.972+0000 7f38a539b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38a00b1ac0 0x7f38a00c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.972+0000 7f38a539b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38a00b1ac0 0x7f38a00c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45388/0 (socket says 172.21.15.60:45388) 2023-07-13T22:29:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.972+0000 7f38a539b700 1 -- 172.21.15.60:0/3035518106 learned_addr learned my addr 172.21.15.60:0/3035518106 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a539b700 1 -- 172.21.15.60:0/3035518106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38a00cede0 msgr2=0x7f38a00c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a539b700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38a00cede0 0x7f38a00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a4b9a700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00b8200 0x7f38a00c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a539b700 1 -- 172.21.15.60:0/3035518106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00b8200 msgr2=0x7f38a00c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a539b700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00b8200 0x7f38a00c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a539b700 1 -- 172.21.15.60:0/3035518106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3898009a70 con 0x7f38a00b1ac0 2023-07-13T22:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a4b9a700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00b8200 0x7f38a00c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a539b700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38a00b1ac0 0x7f38a00c89a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f389c00e9b0 tx=0x7f389c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f3895ffb700 1 -- 172.21.15.60:0/3035518106 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c009710 con 0x7f38a00b1ac0 2023-07-13T22:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f3895ffb700 1 -- 172.21.15.60:0/3035518106 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c00c850 con 0x7f38a00b1ac0 2023-07-13T22:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f3895ffb700 1 -- 172.21.15.60:0/3035518106 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00cbc0 con 0x7f38a00b1ac0 2023-07-13T22:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a00c75b0 con 0x7f38a00b1ac0 2023-07-13T22:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.973+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a00c7b70 con 0x7f38a00b1ac0 2023-07-13T22:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.974+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a0066700 con 0x7f38a00b1ac0 2023-07-13T22:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.975+0000 7f3895ffb700 1 -- 172.21.15.60:0/3035518106 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f389c00cd20 con 0x7f38a00b1ac0 2023-07-13T22:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.976+0000 7f3895ffb700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388406ace0 0x7f388406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.976+0000 7f3895ffb700 1 -- 172.21.15.60:0/3035518106 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f389c089370 con 0x7f38a00b1ac0 2023-07-13T22:29:41.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.976+0000 7f38a4b9a700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388406ace0 0x7f388406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:41.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.977+0000 7f38a4b9a700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388406ace0 0x7f388406d190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f38a0149940 tx=0x7f3890009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:41.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:41.980+0000 7f3895ffb700 1 -- 172.21.15.60:0/3035518106 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f389c058130 con 0x7f38a00b1ac0 2023-07-13T22:29:42.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.186+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f38a004e210 con 0x7f38a00b1ac0 2023-07-13T22:29:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.190+0000 7f3895ffb700 1 -- 172.21.15.60:0/3035518106 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f389c057cc0 con 0x7f38a00b1ac0 2023-07-13T22:29:42.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388406ace0 msgr2=0x7f388406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388406ace0 0x7f388406d190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f38a0149940 tx=0x7f3890009040 comp rx=0 tx=0).stop 2023-07-13T22:29:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38a00b1ac0 msgr2=0x7f38a00c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38a00b1ac0 0x7f38a00c89a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f389c00e9b0 tx=0x7f389c00ed70 comp rx=0 tx=0).stop 2023-07-13T22:29:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 shutdown_connections 2023-07-13T22:29:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f388406ace0 0x7f388406d190 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38a00b1ac0 0x7f38a00c89a0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38a00cede0 0x7f38a00c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 --2- 172.21.15.60:0/3035518106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38a00b8200 0x7f38a00c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.192+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 >> 172.21.15.60:0/3035518106 conn(0x7f38a006b2f0 msgr2=0x7f38a00b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.193+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 shutdown_connections 2023-07-13T22:29:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.193+0000 7f38a75ff700 1 -- 172.21.15.60:0/3035518106 wait complete. 2023-07-13T22:29:42.190 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:29:42.191 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:29:42.294 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:29:42.408 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:29:42.408 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:29:42.409 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:29:42.456 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:29:42.456 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:29:42.457 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:29:42.503 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:29:42.504 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:29:42.504 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:29:42.550 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:29:42.551 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:29:42.551 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:29:42.598 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:29:42.598 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-13T22:29:42.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.822+0000 7fd1915ff700 1 Processor -- start 2023-07-13T22:29:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.822+0000 7fd1915ff700 1 -- start start 2023-07-13T22:29:42.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.823+0000 7fd1915ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18c0b8400 0x7fd18c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:42.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.823+0000 7fd1915ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd18c0cfca0 0x7fd18c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:42.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.823+0000 7fd1915ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0d0ac0 con 0x7fd18c0d3750 2023-07-13T22:29:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.823+0000 7fd1915ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c0b87d0 con 0x7fd18c0b8400 2023-07-13T22:29:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.823+0000 7fd1915ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c0b4eb0 con 0x7fd18c0cfca0 2023-07-13T22:29:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd183fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd18c0d3750 0x7fd18c0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60012/0 (socket says 172.21.15.60:60012) 2023-07-13T22:29:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd183fff700 1 -- 172.21.15.60:0/3721111190 learned_addr learned my addr 172.21.15.60:0/3721111190 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd18affd700 1 --2- 172.21.15.60:0/3721111190 >> v2:172.21.15.60:3300/0 conn(0x7fd18c0b8400 0x7fd18c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd18b7fe700 1 --2- 172.21.15.60:0/3721111190 >> v2:172.21.15.105:3300/0 conn(0x7fd18c0cfca0 0x7fd18c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd18b7fe700 1 -- 172.21.15.60:0/3721111190 >> v2:172.21.15.60:3300/0 conn(0x7fd18c0b8400 msgr2=0x7fd18c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd18b7fe700 1 --2- 172.21.15.60:0/3721111190 >> v2:172.21.15.60:3300/0 conn(0x7fd18c0b8400 0x7fd18c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd18b7fe700 1 -- 172.21.15.60:0/3721111190 >> v1:172.21.15.60:6789/0 conn(0x7fd18c0d3750 legacy=0x7fd18c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd18a7fc700 1 -- 172.21.15.60:0/3721111190 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002168403 0 0) 0x7fd18c0d0ac0 con 0x7fd18c0d3750 2023-07-13T22:29:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd18b7fe700 1 -- 172.21.15.60:0/3721111190 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0b5190 con 0x7fd18c0cfca0 2023-07-13T22:29:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.824+0000 7fd18b7fe700 1 --2- 172.21.15.60:0/3721111190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0cfca0 0x7fd18c0d0f50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0d0070 tx=0x7fd18400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f93bcf19a79498 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.825+0000 7fd18a7fc700 1 -- 172.21.15.60:0/3721111190 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18400e070 con 0x7fd18c0cfca0 2023-07-13T22:29:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.825+0000 7fd18a7fc700 1 -- 172.21.15.60:0/3721111190 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18400bb20 con 0x7fd18c0cfca0 2023-07-13T22:29:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.825+0000 7fd18a7fc700 1 -- 172.21.15.60:0/3721111190 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184012430 con 0x7fd18c0cfca0 2023-07-13T22:29:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.825+0000 7fd1915ff700 1 -- 172.21.15.60:0/3721111190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0cfca0 msgr2=0x7fd18c0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.825+0000 7fd1915ff700 1 --2- 172.21.15.60:0/3721111190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0cfca0 0x7fd18c0d0f50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0d0070 tx=0x7fd18400b330 comp rx=0 tx=0).stop 2023-07-13T22:29:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.825+0000 7fd1915ff700 1 -- 172.21.15.60:0/3721111190 shutdown_connections 2023-07-13T22:29:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.825+0000 7fd1915ff700 1 --2- 172.21.15.60:0/3721111190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0cfca0 0x7fd18c0d0f50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.825+0000 7fd1915ff700 1 --2- 172.21.15.60:0/3721111190 >> v2:172.21.15.60:3300/0 conn(0x7fd18c0b8400 0x7fd18c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.825+0000 7fd1915ff700 1 -- 172.21.15.60:0/3721111190 >> 172.21.15.60:0/3721111190 conn(0x7fd18c06b390 msgr2=0x7fd18c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.826+0000 7fd1915ff700 1 -- 172.21.15.60:0/3721111190 shutdown_connections 2023-07-13T22:29:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.826+0000 7fd1915ff700 1 -- 172.21.15.60:0/3721111190 wait complete. 2023-07-13T22:29:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.828+0000 7fd1915ff700 1 Processor -- start 2023-07-13T22:29:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.828+0000 7fd1915ff700 1 -- start start 2023-07-13T22:29:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.829+0000 7fd1915ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0b8400 0x7fd18c1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.829+0000 7fd1915ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18c0cfca0 0x7fd18c0d2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.829+0000 7fd1915ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd18c0d3750 0x7fd18c0b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd1915ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c1f5c50 con 0x7fd18c0d3750 2023-07-13T22:29:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd1915ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0cbd60 con 0x7fd18c0b8400 2023-07-13T22:29:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd1915ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c143730 con 0x7fd18c0cfca0 2023-07-13T22:29:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0b8400 0x7fd18c1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd18c0d3750 0x7fd18c0b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0b8400 0x7fd18c1431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:46638/0 (socket says 172.21.15.60:46638) 2023-07-13T22:29:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18affd700 1 -- 172.21.15.60:0/3515004028 learned_addr learned my addr 172.21.15.60:0/3515004028 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd183fff700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18c0cfca0 0x7fd18c0d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18affd700 1 -- 172.21.15.60:0/3515004028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18c0cfca0 msgr2=0x7fd18c0d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18affd700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18c0cfca0 0x7fd18c0d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18affd700 1 -- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd18c0d3750 msgr2=0x7fd18c0b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18affd700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd18c0d3750 0x7fd18c0b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18affd700 1 -- 172.21.15.60:0/3515004028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184009a70 con 0x7fd18c0b8400 2023-07-13T22:29:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.830+0000 7fd18b7fe700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd18c0d3750 0x7fd18c0b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.831+0000 7fd18affd700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0b8400 0x7fd18c1431f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e8a0 tx=0x7fd17c00ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.831+0000 7fd1837fe700 1 -- 172.21.15.60:0/3515004028 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c009590 con 0x7fd18c0b8400 2023-07-13T22:29:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.831+0000 7fd1837fe700 1 -- 172.21.15.60:0/3515004028 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c013680 con 0x7fd18c0b8400 2023-07-13T22:29:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.831+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c149540 con 0x7fd18c0b8400 2023-07-13T22:29:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.831+0000 7fd1837fe700 1 -- 172.21.15.60:0/3515004028 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0139f0 con 0x7fd18c0b8400 2023-07-13T22:29:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.831+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c1f5320 con 0x7fd18c0b8400 2023-07-13T22:29:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.832+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c0cf8a0 con 0x7fd18c0b8400 2023-07-13T22:29:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.833+0000 7fd1837fe700 1 -- 172.21.15.60:0/3515004028 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd17c013b50 con 0x7fd18c0b8400 2023-07-13T22:29:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.834+0000 7fd1837fe700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16c06ace0 0x7fd16c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.834+0000 7fd1837fe700 1 -- 172.21.15.60:0/3515004028 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd17c088fe0 con 0x7fd18c0b8400 2023-07-13T22:29:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.834+0000 7fd183fff700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16c06ace0 0x7fd16c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.834+0000 7fd183fff700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16c06ace0 0x7fd16c06d190 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd18c149900 tx=0x7fd174007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:42.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:42.839+0000 7fd1837fe700 1 -- 172.21.15.60:0/3515004028 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd17c057de0 con 0x7fd18c0b8400 2023-07-13T22:29:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.085+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd18c0008d0 con 0x7fd18c0b8400 2023-07-13T22:29:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.086+0000 7fd1837fe700 1 -- 172.21.15.60:0/3515004028 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fd17c021de0 con 0x7fd18c0b8400 2023-07-13T22:29:43.066 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:43.066 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2778215979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2778215979},{"type":"v1","addr":"172.21.15.105:6834","nonce":2778215979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/699581140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":699581140},{"type":"v1","addr":"172.21.15.105:6835","nonce":699581140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3935084739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3935084739},{"type":"v1","addr":"172.21.15.60:6835","nonce":3935084739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/2886632916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":2886632916},{"type":"v1","addr":"172.21.15.60:6837","nonce":2886632916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[]} 2023-07-13T22:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16c06ace0 msgr2=0x7fd16c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16c06ace0 0x7fd16c06d190 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd18c149900 tx=0x7fd174007de0 comp rx=0 tx=0).stop 2023-07-13T22:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0b8400 msgr2=0x7fd18c1431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0b8400 0x7fd18c1431f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e8a0 tx=0x7fd17c00ebb0 comp rx=0 tx=0).stop 2023-07-13T22:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 shutdown_connections 2023-07-13T22:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd16c06ace0 0x7fd16c06d190 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd18c0d3750 0x7fd18c0b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd18c0b8400 0x7fd18c1431f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 --2- 172.21.15.60:0/3515004028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd18c0cfca0 0x7fd18c0d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 >> 172.21.15.60:0/3515004028 conn(0x7fd18c06b390 msgr2=0x7fd18c0b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 shutdown_connections 2023-07-13T22:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.089+0000 7fd1915ff700 1 -- 172.21.15.60:0/3515004028 wait complete. 2023-07-13T22:29:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 67 2023-07-13T22:29:43.086 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:29:43.087 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-13T22:29:43.087 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:29:43.087 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:29:43.087 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:29:43.088 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-13T22:29:43.088 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:29:43.088 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:29:43.088 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:29:43.088 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-13T22:29:43.089 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:29:43.089 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:29:43.089 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:29:43.089 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-13T22:29:43.090 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:29:43.090 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:29:43.101 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:29:43.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:29:43.102 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:29:43.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:29:43.103 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:29:43.104 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:29:43.105 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:29:43.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:29:43.146 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:29:43.147 DEBUG:teuthology.parallel:result is None 2023-07-13T22:29:43.148 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:29:43.149 DEBUG:teuthology.parallel:result is None 2023-07-13T22:29:43.149 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:29:43.150 DEBUG:teuthology.parallel:result is None 2023-07-13T22:29:43.152 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:29:43.153 DEBUG:teuthology.parallel:result is None 2023-07-13T22:29:43.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-13T22:29:43.295 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:29:43.296 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:29:43.298 INFO:tasks.ceph.mds.b.smithi105.stderr:2023-07-13T22:29:43.320+0000 7fe1c69fd700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6834/0: (98) Address already in use 2023-07-13T22:29:43.303 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:29:43.303 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:29:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.389+0000 7fc00a5ff700 1 Processor -- start 2023-07-13T22:29:43.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.390+0000 7fc00a5ff700 1 -- start start 2023-07-13T22:29:43.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.391+0000 7fc00a5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc0040b12a0 0x7fc0040cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:43.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.391+0000 7fc00a5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0041609a0 con 0x7fc00406bad0 2023-07-13T22:29:43.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.391+0000 7fc00a5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00415c860 con 0x7fc0040cd760 2023-07-13T22:29:43.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.391+0000 7fc00a5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00406a5f0 con 0x7fc0040b12a0 2023-07-13T22:29:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.391+0000 7fc0037fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc00406bad0 0x7fc0040b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60028/0 (socket says 172.21.15.60:60028) 2023-07-13T22:29:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.391+0000 7fc003fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc0040b12a0 0x7fc0040cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.391+0000 7fc0037fe700 1 -- 172.21.15.60:0/1056082004 learned_addr learned my addr 172.21.15.60:0/1056082004 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.392+0000 7fc003fff700 1 -- 172.21.15.60:0/1056082004 >> v1:172.21.15.105:6789/0 conn(0x7fc0040cd760 legacy=0x7fc0040cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.392+0000 7fc003fff700 1 -- 172.21.15.60:0/1056082004 >> v1:172.21.15.60:6789/0 conn(0x7fc00406bad0 legacy=0x7fc0040b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.392+0000 7fc002ffd700 1 -- 172.21.15.60:0/1056082004 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450018360 0 0) 0x7fc00415c860 con 0x7fc0040cd760 2023-07-13T22:29:43.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.392+0000 7fc003fff700 1 -- 172.21.15.60:0/1056082004 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00406bf20 con 0x7fc0040b12a0 2023-07-13T22:29:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.392+0000 7fc003fff700 1 --2- 172.21.15.60:0/1056082004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0040b12a0 0x7fc0040cefa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009840 tx=0x7fbff4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d3a0952e5945cf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.392+0000 7fc002ffd700 1 -- 172.21.15.60:0/1056082004 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4004230 con 0x7fc0040b12a0 2023-07-13T22:29:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.392+0000 7fc002ffd700 1 -- 172.21.15.60:0/1056082004 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4004390 con 0x7fc0040b12a0 2023-07-13T22:29:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.392+0000 7fc002ffd700 1 -- 172.21.15.60:0/1056082004 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff40036a0 con 0x7fc0040b12a0 2023-07-13T22:29:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.393+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1056082004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0040b12a0 msgr2=0x7fc0040cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.393+0000 7fc00a5ff700 1 --2- 172.21.15.60:0/1056082004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0040b12a0 0x7fc0040cefa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009840 tx=0x7fbff4009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.393+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1056082004 shutdown_connections 2023-07-13T22:29:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.393+0000 7fc00a5ff700 1 --2- 172.21.15.60:0/1056082004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0040b12a0 0x7fc0040cefa0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.393+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1056082004 >> 172.21.15.60:0/1056082004 conn(0x7fc0040cc2a0 msgr2=0x7fc0040cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.393+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1056082004 shutdown_connections 2023-07-13T22:29:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.393+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1056082004 wait complete. 2023-07-13T22:29:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.395+0000 7fc00a5ff700 1 Processor -- start 2023-07-13T22:29:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.395+0000 7fc00a5ff700 1 -- start start 2023-07-13T22:29:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc00a5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00406bad0 0x7fc00415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc00a5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0040b12a0 0x7fc0040ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc00a5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0040cd760 0x7fc0040ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc00a5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040abe30 con 0x7fc00406bad0 2023-07-13T22:29:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc00a5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415e300 con 0x7fc0040b12a0 2023-07-13T22:29:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc00a5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040aeb10 con 0x7fc0040cd760 2023-07-13T22:29:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc003fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00406bad0 0x7fc00415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:43.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc003fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00406bad0 0x7fc00415ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45458/0 (socket says 172.21.15.60:45458) 2023-07-13T22:29:43.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc003fff700 1 -- 172.21.15.60:0/1256573926 learned_addr learned my addr 172.21.15.60:0/1256573926 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:43.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc003fff700 1 -- 172.21.15.60:0/1256573926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0040cd760 msgr2=0x7fc0040ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:43.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc003fff700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0040cd760 0x7fc0040ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:43.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc003fff700 1 -- 172.21.15.60:0/1256573926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0040b12a0 msgr2=0x7fc0040ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:43.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc003fff700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0040b12a0 0x7fc0040ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:43.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.396+0000 7fc003fff700 1 -- 172.21.15.60:0/1256573926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40094a0 con 0x7fc00406bad0 2023-07-13T22:29:43.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.397+0000 7fc003fff700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00406bad0 0x7fc00415ca10 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fc00406e9e0 tx=0x7fbff4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:43.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.397+0000 7fc000ff9700 1 -- 172.21.15.60:0/1256573926 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4011830 con 0x7fc00406bad0 2023-07-13T22:29:43.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.397+0000 7fc000ff9700 1 -- 172.21.15.60:0/1256573926 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff401f070 con 0x7fc00406bad0 2023-07-13T22:29:43.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.397+0000 7fc000ff9700 1 -- 172.21.15.60:0/1256573926 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4011bd0 con 0x7fc00406bad0 2023-07-13T22:29:43.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.397+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0040ae830 con 0x7fc00406bad0 2023-07-13T22:29:43.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.397+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00415e7f0 con 0x7fc00406bad0 2023-07-13T22:29:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.398+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00406ab90 con 0x7fc00406bad0 2023-07-13T22:29:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.399+0000 7fc000ff9700 1 -- 172.21.15.60:0/1256573926 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbff4011d30 con 0x7fc00406bad0 2023-07-13T22:29:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.400+0000 7fc000ff9700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbff006ace0 0x7fbff006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.400+0000 7fc000ff9700 1 -- 172.21.15.60:0/1256573926 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbff4021070 con 0x7fc00406bad0 2023-07-13T22:29:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.400+0000 7fc0037fe700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbff006ace0 0x7fbff006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.401+0000 7fc0037fe700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbff006ace0 0x7fbff006d190 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc0040b55f0 tx=0x7fbff8007a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.404+0000 7fc000ff9700 1 -- 172.21.15.60:0/1256573926 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbff4056110 con 0x7fc00406bad0 2023-07-13T22:29:43.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:43.634+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc004071c90 con 0x7fc00406bad0 2023-07-13T22:29:44.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.555+0000 7fc000ff9700 1 -- 172.21.15.60:0/1256573926 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbff402b020 con 0x7fc00406bad0 2023-07-13T22:29:44.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbff006ace0 msgr2=0x7fbff006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:44.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbff006ace0 0x7fbff006d190 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc0040b55f0 tx=0x7fbff8007a30 comp rx=0 tx=0).stop 2023-07-13T22:29:44.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00406bad0 msgr2=0x7fc00415ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:44.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00406bad0 0x7fc00415ca10 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fc00406e9e0 tx=0x7fbff4004750 comp rx=0 tx=0).stop 2023-07-13T22:29:44.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 shutdown_connections 2023-07-13T22:29:44.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbff006ace0 0x7fbff006d190 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00406bad0 0x7fc00415ca10 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0040cd760 0x7fc0040ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 --2- 172.21.15.60:0/1256573926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0040b12a0 0x7fc0040ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.558+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 >> 172.21.15.60:0/1256573926 conn(0x7fc0040cc2a0 msgr2=0x7fc0040b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:44.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.559+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 shutdown_connections 2023-07-13T22:29:44.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.559+0000 7fc00a5ff700 1 -- 172.21.15.60:0/1256573926 wait complete. 2023-07-13T22:29:44.542 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-07-13T22:29:44.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-13T22:29:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.788+0000 7efcfcb9d700 1 Processor -- start 2023-07-13T22:29:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.789+0000 7efcfcb9d700 1 -- start start 2023-07-13T22:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.790+0000 7efcfcb9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf815c290 0x7efcf80d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.790+0000 7efcfcb9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efcf80d0ad0 0x7efcf80cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.790+0000 7efcfcb9d700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf80afe40 con 0x7efcf80cf8c0 2023-07-13T22:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.790+0000 7efcfcb9d700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf80d0f40 con 0x7efcf80d0ad0 2023-07-13T22:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.790+0000 7efcfcb9d700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80d0090 con 0x7efcf815c290 2023-07-13T22:29:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.790+0000 7efcf7fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efcf80cf8c0 0x7efcf815fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60036/0 (socket says 172.21.15.60:60036) 2023-07-13T22:29:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.790+0000 7efcf7fff700 1 -- 172.21.15.60:0/2319298579 learned_addr learned my addr 172.21.15.60:0/2319298579 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.790+0000 7efcf77fe700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf815c290 0x7efcf80d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.790+0000 7efcf6ffd700 1 --2- 172.21.15.60:0/2319298579 >> v2:172.21.15.105:3300/0 conn(0x7efcf80d0ad0 0x7efcf80cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892985277 0 0) 0x7efcf80afe40 con 0x7efcf80cf8c0 2023-07-13T22:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce00035b0 con 0x7efcf80cf8c0 2023-07-13T22:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3343574588 0 0) 0x7efce00035b0 con 0x7efcf80cf8c0 2023-07-13T22:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf815c290 msgr2=0x7efcf80d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf815c290 0x7efcf80d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 >> v2:172.21.15.105:3300/0 conn(0x7efcf80d0ad0 msgr2=0x7efcf80cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 --2- 172.21.15.60:0/2319298579 >> v2:172.21.15.105:3300/0 conn(0x7efcf80d0ad0 0x7efcf80cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80af7f0 con 0x7efcf80cf8c0 2023-07-13T22:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7efcf00019f0 con 0x7efcf80cf8c0 2023-07-13T22:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf6ffd700 1 --2- 172.21.15.60:0/2319298579 >> v2:172.21.15.105:3300/0 conn(0x7efcf80d0ad0 0x7efcf80cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 >> v1:172.21.15.60:6789/0 conn(0x7efcf80cf8c0 legacy=0x7efcf815fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efce0003a00 0x7efce0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efce0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf815c290 0x7efce0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80d0f40 con 0x7efce0004850 2023-07-13T22:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80d0090 con 0x7efce0003a00 2023-07-13T22:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce000ba10 con 0x7efcf815c290 2023-07-13T22:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf6ffd700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efce0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.791+0000 7efcf77fe700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efce0003a00 0x7efce0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf7fff700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf815c290 0x7efce0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf6ffd700 1 -- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf815c290 msgr2=0x7efce0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf6ffd700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf815c290 0x7efce0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf6ffd700 1 -- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efce0003a00 msgr2=0x7efce0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf6ffd700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efce0003a00 0x7efce0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf6ffd700 1 -- 172.21.15.60:0/2319298579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00035b0 con 0x7efce0004850 2023-07-13T22:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf6ffd700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efce0004cc0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efcec009d60 tx=0x7efce0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddc63b3c8bc4b17 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec00e9a0 con 0x7efce0004850 2023-07-13T22:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec00eb00 con 0x7efce0004850 2023-07-13T22:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.792+0000 7efcf67fc700 1 -- 172.21.15.60:0/2319298579 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec00eeb0 con 0x7efce0004850 2023-07-13T22:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.793+0000 7efcfcb9d700 1 -- 172.21.15.60:0/2319298579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 msgr2=0x7efce0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.793+0000 7efcfcb9d700 1 --2- 172.21.15.60:0/2319298579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efce0004cc0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efcec009d60 tx=0x7efce0005880 comp rx=0 tx=0).stop 2023-07-13T22:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.793+0000 7efcf7fff700 1 -- 172.21.15.60:0/2319298579 reap_dead start 2023-07-13T22:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.793+0000 7efcfcb9d700 1 -- 172.21.15.60:0/2319298579 shutdown_connections 2023-07-13T22:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.793+0000 7efcfcb9d700 1 -- 172.21.15.60:0/2319298579 >> 172.21.15.60:0/2319298579 conn(0x7efcf8071f20 msgr2=0x7efcf809bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.793+0000 7efcfcb9d700 1 -- 172.21.15.60:0/2319298579 shutdown_connections 2023-07-13T22:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.793+0000 7efcfcb9d700 1 -- 172.21.15.60:0/2319298579 wait complete. 2023-07-13T22:29:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.795+0000 7efcfcb9d700 1 Processor -- start 2023-07-13T22:29:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.795+0000 7efcfcb9d700 1 -- start start 2023-07-13T22:29:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcfcb9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efcf809fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcfcb9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcf80a0350 0x7efcf815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcfcb9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf809e7b0 0x7efcf815d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcfcb9d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80c9090 con 0x7efce0004850 2023-07-13T22:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcfcb9d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf809f980 con 0x7efcf80a0350 2023-07-13T22:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcfcb9d700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf809ed70 con 0x7efcf809e7b0 2023-07-13T22:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcf77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efcf809fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcf77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efcf809fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45478/0 (socket says 172.21.15.60:45478) 2023-07-13T22:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcf77fe700 1 -- 172.21.15.60:0/1160858974 learned_addr learned my addr 172.21.15.60:0/1160858974 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcf77fe700 1 -- 172.21.15.60:0/1160858974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf809e7b0 msgr2=0x7efcf815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcf77fe700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf809e7b0 0x7efcf815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcf77fe700 1 -- 172.21.15.60:0/1160858974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcf80a0350 msgr2=0x7efcf815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcf77fe700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcf80a0350 0x7efcf815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.796+0000 7efcf77fe700 1 -- 172.21.15.60:0/1160858974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf809f050 con 0x7efce0004850 2023-07-13T22:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.797+0000 7efcf77fe700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efcf809fe10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7efce8011aa0 tx=0x7efce8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.797+0000 7efcdffff700 1 -- 172.21.15.60:0/1160858974 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce800f920 con 0x7efce0004850 2023-07-13T22:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.797+0000 7efcdffff700 1 -- 172.21.15.60:0/1160858974 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce800fa80 con 0x7efce0004850 2023-07-13T22:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.797+0000 7efcdffff700 1 -- 172.21.15.60:0/1160858974 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce800fe30 con 0x7efce0004850 2023-07-13T22:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.797+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf809f330 con 0x7efce0004850 2023-07-13T22:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.797+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf809c780 con 0x7efce0004850 2023-07-13T22:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.798+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf804e210 con 0x7efce0004850 2023-07-13T22:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.799+0000 7efcdffff700 1 -- 172.21.15.60:0/1160858974 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efce8014440 con 0x7efce0004850 2023-07-13T22:29:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.800+0000 7efcdffff700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efce406ace0 0x7efce406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.800+0000 7efcdffff700 1 -- 172.21.15.60:0/1160858974 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efce80897b0 con 0x7efce0004850 2023-07-13T22:29:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.800+0000 7efcf6ffd700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efce406ace0 0x7efce406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.801+0000 7efcf6ffd700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efce406ace0 0x7efce406d190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efcf80a0940 tx=0x7efcec013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:44.804+0000 7efcdffff700 1 -- 172.21.15.60:0/1160858974 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efce8054db0 con 0x7efce0004850 2023-07-13T22:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.091+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efcf80b04a0 con 0x7efce0004850 2023-07-13T22:29:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.092+0000 7efcdffff700 1 -- 172.21.15.60:0/1160858974 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7efce80583d0 con 0x7efce0004850 2023-07-13T22:29:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:45.071 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:29:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-07-13T22:29:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-07-13T22:29:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw==", 2023-07-13T22:29:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:29:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:29:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:29:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-07-13T22:29:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g==", 2023-07-13T22:29:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:29:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:29:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:29:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-07-13T22:29:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA==", 2023-07-13T22:29:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:29:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:29:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:29:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-07-13T22:29:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ==", 2023-07-13T22:29:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:29:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:29:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:29:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-07-13T22:29:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ==", 2023-07-13T22:29:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:29:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:29:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-07-13T22:29:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw==", 2023-07-13T22:29:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:29:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:29:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-07-13T22:29:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q==", 2023-07-13T22:29:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:29:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:29:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-07-13T22:29:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ==", 2023-07-13T22:29:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:29:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:29:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-07-13T22:29:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg==", 2023-07-13T22:29:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:29:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:29:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.089 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-07-13T22:29:45.089 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA==", 2023-07-13T22:29:45.089 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.090 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:29:45.090 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:29:45.090 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.090 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.091 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-07-13T22:29:45.091 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw==", 2023-07-13T22:29:45.091 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.092 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:29:45.092 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:29:45.092 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.093 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-07-13T22:29:45.094 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ==", 2023-07-13T22:29:45.094 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.094 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:29:45.094 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:29:45.095 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.095 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.096 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-07-13T22:29:45.096 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w==", 2023-07-13T22:29:45.096 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.096 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:29:45.097 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2023-07-13T22:29:45.097 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-13T22:29:45.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.098 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-07-13T22:29:45.098 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw==", 2023-07-13T22:29:45.098 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.099 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:29:45.099 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:29:45.099 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:29:45.100 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:29:45.100 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.101 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-07-13T22:29:45.101 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg==", 2023-07-13T22:29:45.101 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.101 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:29:45.102 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:29:45.102 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:29:45.102 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:29:45.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.103 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-07-13T22:29:45.103 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q==", 2023-07-13T22:29:45.104 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.104 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:29:45.104 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:29:45.105 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:29:45.105 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:29:45.105 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.106 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-07-13T22:29:45.107 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBk7CgGBhAAaK88V4CEyrYfZx7wv+XG3g==", 2023-07-13T22:29:45.107 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.107 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:29:45.107 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:29:45.108 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:29:45.108 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.108 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.109 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-07-13T22:29:45.109 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ==", 2023-07-13T22:29:45.110 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.110 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:29:45.110 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:29:45.110 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.111 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:45.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:45.111 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-07-13T22:29:45.112 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ==", 2023-07-13T22:29:45.112 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:29:45.112 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:29:45.112 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:29:45.113 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:29:45.113 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.113 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:45.113 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:29:45.114 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:29:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efce406ace0 msgr2=0x7efce406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efce406ace0 0x7efce406d190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efcf80a0940 tx=0x7efcec013000 comp rx=0 tx=0).stop 2023-07-13T22:29:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 msgr2=0x7efcf809fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efcf809fe10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7efce8011aa0 tx=0x7efce8011e60 comp rx=0 tx=0).stop 2023-07-13T22:29:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 shutdown_connections 2023-07-13T22:29:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efce406ace0 0x7efce406d190 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efce0004850 0x7efcf809fe10 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf809e7b0 0x7efcf815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 --2- 172.21.15.60:0/1160858974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcf80a0350 0x7efcf815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 >> 172.21.15.60:0/1160858974 conn(0x7efcf8071f20 msgr2=0x7efcf80b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.094+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 shutdown_connections 2023-07-13T22:29:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.095+0000 7efcfcb9d700 1 -- 172.21.15.60:0/1160858974 wait complete. 2023-07-13T22:29:45.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:45.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.344+0000 7f7d333ff700 1 Processor -- start 2023-07-13T22:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.344+0000 7f7d333ff700 1 -- start start 2023-07-13T22:29:45.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.345+0000 7f7d333ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0b8570 0x7f7d2c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.345+0000 7f7d333ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7d2c0d35c0 0x7f7d2c0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.345+0000 7f7d333ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d2c0cf3f0 0x7f7d2c0d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.345+0000 7f7d333ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c0d3b30 con 0x7f7d2c0cf3f0 2023-07-13T22:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.345+0000 7f7d333ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c0d1930 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.345+0000 7f7d333ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0d09d0 con 0x7f7d2c0b8570 2023-07-13T22:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3199c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d2c0cf3f0 0x7f7d2c0d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3199c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d2c0cf3f0 0x7f7d2c0d13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46226/0 (socket says 172.21.15.60:46226) 2023-07-13T22:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3199c700 1 -- 172.21.15.60:0/1579891903 learned_addr learned my addr 172.21.15.60:0/1579891903 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3099a700 1 --2- 172.21.15.60:0/1579891903 >> v2:172.21.15.105:3300/0 conn(0x7f7d2c0d35c0 0x7f7d2c0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3119b700 1 --2- 172.21.15.60:0/1579891903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0b8570 0x7f7d2c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3199c700 1 -- 172.21.15.60:0/1579891903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0b8570 msgr2=0x7f7d2c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3199c700 1 --2- 172.21.15.60:0/1579891903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0b8570 0x7f7d2c0ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3199c700 1 -- 172.21.15.60:0/1579891903 >> v2:172.21.15.105:3300/0 conn(0x7f7d2c0d35c0 msgr2=0x7f7d2c0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3199c700 1 --2- 172.21.15.60:0/1579891903 >> v2:172.21.15.105:3300/0 conn(0x7f7d2c0d35c0 0x7f7d2c0d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3199c700 1 -- 172.21.15.60:0/1579891903 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c0cda10 con 0x7f7d2c0cf3f0 2023-07-13T22:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.346+0000 7f7d3199c700 1 --2- 172.21.15.60:0/1579891903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0cf3f0 0x7f7d2c0d13f0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0d3d30 tx=0x7f7d2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb88229b7f38a71 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:45.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d23fff700 1 -- 172.21.15.60:0/1579891903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400e070 con 0x7f7d2c0cf3f0 2023-07-13T22:29:45.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d23fff700 1 -- 172.21.15.60:0/1579891903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2400b9c0 con 0x7f7d2c0cf3f0 2023-07-13T22:29:45.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d23fff700 1 -- 172.21.15.60:0/1579891903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400bd70 con 0x7f7d2c0cf3f0 2023-07-13T22:29:45.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d333ff700 1 -- 172.21.15.60:0/1579891903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0cf3f0 msgr2=0x7f7d2c0d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1579891903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0cf3f0 0x7f7d2c0d13f0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0d3d30 tx=0x7f7d2400b330 comp rx=0 tx=0).stop 2023-07-13T22:29:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d333ff700 1 -- 172.21.15.60:0/1579891903 shutdown_connections 2023-07-13T22:29:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1579891903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0cf3f0 0x7f7d2c0d13f0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1579891903 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0b8570 0x7f7d2c0ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1579891903 >> v2:172.21.15.105:3300/0 conn(0x7f7d2c0d35c0 0x7f7d2c0d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d333ff700 1 -- 172.21.15.60:0/1579891903 >> 172.21.15.60:0/1579891903 conn(0x7f7d2c071600 msgr2=0x7f7d2c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d333ff700 1 -- 172.21.15.60:0/1579891903 shutdown_connections 2023-07-13T22:29:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.347+0000 7f7d333ff700 1 -- 172.21.15.60:0/1579891903 wait complete. 2023-07-13T22:29:45.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.349+0000 7f7d333ff700 1 Processor -- start 2023-07-13T22:29:45.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.349+0000 7f7d333ff700 1 -- start start 2023-07-13T22:29:45.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.350+0000 7f7d333ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d2c0b8570 0x7f7d2c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.350+0000 7f7d333ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0cf3f0 0x7f7d2c14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.350+0000 7f7d333ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0d35c0 0x7f7d2c14a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.350+0000 7f7d333ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c15b070 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.350+0000 7f7d333ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0c7160 con 0x7f7d2c0b8570 2023-07-13T22:29:45.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.350+0000 7f7d333ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0c6830 con 0x7f7d2c0cf3f0 2023-07-13T22:29:45.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.350+0000 7f7d3199c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0d35c0 0x7f7d2c14a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.350+0000 7f7d3199c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0d35c0 0x7f7d2c14a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46228/0 (socket says 172.21.15.60:46228) 2023-07-13T22:29:45.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.350+0000 7f7d3199c700 1 -- 172.21.15.60:0/1859736411 learned_addr learned my addr 172.21.15.60:0/1859736411 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:45.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d3099a700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0cf3f0 0x7f7d2c14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d3119b700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d2c0b8570 0x7f7d2c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d3199c700 1 -- 172.21.15.60:0/1859736411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0cf3f0 msgr2=0x7f7d2c14abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d3199c700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0cf3f0 0x7f7d2c14abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d3199c700 1 -- 172.21.15.60:0/1859736411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d2c0b8570 msgr2=0x7f7d2c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d3199c700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d2c0b8570 0x7f7d2c0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d3199c700 1 -- 172.21.15.60:0/1859736411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d24009b30 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d3199c700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0d35c0 0x7f7d2c14a070 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400b370 tx=0x7f7d24004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:45.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d21ffb700 1 -- 172.21.15.60:0/1859736411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24004310 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d21ffb700 1 -- 172.21.15.60:0/1859736411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d24004470 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d21ffb700 1 -- 172.21.15.60:0/1859736411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24004820 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c0c6b10 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.351+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c1e5f60 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.353+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c04e280 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.353+0000 7f7d21ffb700 1 -- 172.21.15.60:0/1859736411 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d2401b430 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.354+0000 7f7d21ffb700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d1006ace0 0x7f7d1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.354+0000 7f7d21ffb700 1 -- 172.21.15.60:0/1859736411 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7d2408a3c0 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.355+0000 7f7d3119b700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d1006ace0 0x7f7d1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.355+0000 7f7d3119b700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d1006ace0 0x7f7d1006d190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800a970 tx=0x7f7d28008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:45.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.359+0000 7f7d21ffb700 1 -- 172.21.15.60:0/1859736411 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7d240557b0 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.573+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d2c15c370 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.573+0000 7f7d21ffb700 1 -- 172.21.15.60:0/1859736411 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+14547 (secure 0 0 0) 0x7f7d24088e60 con 0x7f7d2c0d35c0 2023-07-13T22:29:45.553 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:45.553 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":108,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:29:44.551857+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:45.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d1006ace0 msgr2=0x7f7d1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d1006ace0 0x7f7d1006d190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800a970 tx=0x7f7d28008040 comp rx=0 tx=0).stop 2023-07-13T22:29:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0d35c0 msgr2=0x7f7d2c14a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0d35c0 0x7f7d2c14a070 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400b370 tx=0x7f7d24004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 shutdown_connections 2023-07-13T22:29:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d1006ace0 0x7f7d1006d190 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d2c0d35c0 0x7f7d2c14a070 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d2c0b8570 0x7f7d2c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 --2- 172.21.15.60:0/1859736411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d2c0cf3f0 0x7f7d2c14abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 >> 172.21.15.60:0/1859736411 conn(0x7f7d2c071600 msgr2=0x7f7d2c15c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 shutdown_connections 2023-07-13T22:29:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.576+0000 7f7d333ff700 1 -- 172.21.15.60:0/1859736411 wait complete. 2023-07-13T22:29:45.574 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:29:45.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:29:45.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.802+0000 7f63eadff700 1 Processor -- start 2023-07-13T22:29:45.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.802+0000 7f63eadff700 1 -- start start 2023-07-13T22:29:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.803+0000 7f63eadff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63e40b8200 0x7f63e40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.803+0000 7f63eadff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f63e40cede0 0x7f63e40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.803+0000 7f63eadff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e40d1870 con 0x7f63e40b1ac0 2023-07-13T22:29:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.803+0000 7f63eadff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e40cf370 con 0x7f63e40b8200 2023-07-13T22:29:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.803+0000 7f63eadff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e40d12b0 con 0x7f63e40cede0 2023-07-13T22:29:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.803+0000 7f63e8b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63e40b8200 0x7f63e40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.803+0000 7f63e8b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63e40b8200 0x7f63e40cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46244/0 (socket says 172.21.15.60:46244) 2023-07-13T22:29:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.803+0000 7f63e8b9b700 1 -- 172.21.15.60:0/2072312976 learned_addr learned my addr 172.21.15.60:0/2072312976 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:45.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.803+0000 7f63e939c700 1 --2- 172.21.15.60:0/2072312976 >> v2:172.21.15.105:3300/0 conn(0x7f63e40cede0 0x7f63e40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.804+0000 7f63e8b9b700 1 -- 172.21.15.60:0/2072312976 >> v2:172.21.15.105:3300/0 conn(0x7f63e40cede0 msgr2=0x7f63e40cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:45.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.804+0000 7f63e8b9b700 1 --2- 172.21.15.60:0/2072312976 >> v2:172.21.15.105:3300/0 conn(0x7f63e40cede0 0x7f63e40cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.804+0000 7f63e8b9b700 1 -- 172.21.15.60:0/2072312976 >> v1:172.21.15.105:6789/0 conn(0x7f63e40b1ac0 legacy=0x7f63e415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.804+0000 7f63e8b9b700 1 -- 172.21.15.60:0/2072312976 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e40d1590 con 0x7f63e40b8200 2023-07-13T22:29:45.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.804+0000 7f63e8b9b700 1 --2- 172.21.15.60:0/2072312976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40b8200 0x7f63e40cf640 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f63e40c9c90 tx=0x7f63d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d19fbd0d5607592 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:45.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.804+0000 7f63e37fe700 1 -- 172.21.15.60:0/2072312976 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8011830 con 0x7f63e40b8200 2023-07-13T22:29:45.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.804+0000 7f63e37fe700 1 -- 172.21.15.60:0/2072312976 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d8011990 con 0x7f63e40b8200 2023-07-13T22:29:45.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.804+0000 7f63e37fe700 1 -- 172.21.15.60:0/2072312976 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8011d20 con 0x7f63e40b8200 2023-07-13T22:29:45.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.805+0000 7f63eadff700 1 -- 172.21.15.60:0/2072312976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40b8200 msgr2=0x7f63e40cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.805+0000 7f63eadff700 1 --2- 172.21.15.60:0/2072312976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40b8200 0x7f63e40cf640 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f63e40c9c90 tx=0x7f63d8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.805+0000 7f63eadff700 1 -- 172.21.15.60:0/2072312976 shutdown_connections 2023-07-13T22:29:45.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.805+0000 7f63eadff700 1 --2- 172.21.15.60:0/2072312976 >> v2:172.21.15.105:3300/0 conn(0x7f63e40cede0 0x7f63e40cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.805+0000 7f63eadff700 1 --2- 172.21.15.60:0/2072312976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40b8200 0x7f63e40cf640 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.805+0000 7f63eadff700 1 -- 172.21.15.60:0/2072312976 >> 172.21.15.60:0/2072312976 conn(0x7f63e406b2f0 msgr2=0x7f63e406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:45.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.805+0000 7f63eadff700 1 -- 172.21.15.60:0/2072312976 shutdown_connections 2023-07-13T22:29:45.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.805+0000 7f63eadff700 1 -- 172.21.15.60:0/2072312976 wait complete. 2023-07-13T22:29:45.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.807+0000 7f63eadff700 1 Processor -- start 2023-07-13T22:29:45.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.807+0000 7f63eadff700 1 -- start start 2023-07-13T22:29:45.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63eadff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63e40b1ac0 0x7f63e4142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63eadff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e40b8200 0x7f63e4144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63eadff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40cede0 0x7f63e4144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63eadff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4142ae0 con 0x7f63e40cede0 2023-07-13T22:29:45.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63eadff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4142dc0 con 0x7f63e40b8200 2023-07-13T22:29:45.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63eadff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e41430a0 con 0x7f63e40b1ac0 2023-07-13T22:29:45.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63e939c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40cede0 0x7f63e4144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63e939c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40cede0 0x7f63e4144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46258/0 (socket says 172.21.15.60:46258) 2023-07-13T22:29:45.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63e939c700 1 -- 172.21.15.60:0/828491092 learned_addr learned my addr 172.21.15.60:0/828491092 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:45.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63e8b9b700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63e40b1ac0 0x7f63e4142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.808+0000 7f63e3fff700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e40b8200 0x7f63e4144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e939c700 1 -- 172.21.15.60:0/828491092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63e40b1ac0 msgr2=0x7f63e4142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e939c700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63e40b1ac0 0x7f63e4142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e939c700 1 -- 172.21.15.60:0/828491092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e40b8200 msgr2=0x7f63e4144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:45.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e939c700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e40b8200 0x7f63e4144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:45.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e939c700 1 -- 172.21.15.60:0/828491092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8009550 con 0x7f63e40cede0 2023-07-13T22:29:45.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e8b9b700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63e40b1ac0 0x7f63e4142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:29:45.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e939c700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40cede0 0x7f63e4144660 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00efc0 tx=0x7f63dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:45.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e17fa700 1 -- 172.21.15.60:0/828491092 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc009ea0 con 0x7f63e40cede0 2023-07-13T22:29:45.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e17fa700 1 -- 172.21.15.60:0/828491092 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc00f040 con 0x7f63e40cede0 2023-07-13T22:29:45.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e4143380 con 0x7f63e40cede0 2023-07-13T22:29:45.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63e17fa700 1 -- 172.21.15.60:0/828491092 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00ced0 con 0x7f63e40cede0 2023-07-13T22:29:45.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.809+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e4144e00 con 0x7f63e40cede0 2023-07-13T22:29:45.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.810+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e41438a0 con 0x7f63e40cede0 2023-07-13T22:29:45.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.811+0000 7f63e17fa700 1 -- 172.21.15.60:0/828491092 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63dc014480 con 0x7f63e40cede0 2023-07-13T22:29:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.813+0000 7f63e17fa700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f63c806ace0 0x7f63c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.813+0000 7f63e17fa700 1 -- 172.21.15.60:0/828491092 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f63dc0899d0 con 0x7f63e40cede0 2023-07-13T22:29:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.813+0000 7f63e8b9b700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f63c806ace0 0x7f63c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.814+0000 7f63e8b9b700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f63c806ace0 0x7f63c806d190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009c10 tx=0x7f63d8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:45.817+0000 7f63e17fa700 1 -- 172.21.15.60:0/828491092 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f63e41438a0 con 0x7f63e40cede0 2023-07-13T22:29:45.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.020+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f63e41438a0 con 0x7f63e40cede0 2023-07-13T22:29:45.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.020+0000 7f63e17fa700 1 -- 172.21.15.60:0/828491092 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f63dc018020 con 0x7f63e40cede0 2023-07-13T22:29:46.000 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:29:46.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.022+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f63c806ace0 msgr2=0x7f63c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.022+0000 7f63eadff700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f63c806ace0 0x7f63c806d190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009c10 tx=0x7f63d8008030 comp rx=0 tx=0).stop 2023-07-13T22:29:46.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40cede0 msgr2=0x7f63e4144660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40cede0 0x7f63e4144660 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00efc0 tx=0x7f63dc00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 shutdown_connections 2023-07-13T22:29:46.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f63c806ace0 0x7f63c806d190 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e40cede0 0x7f63e4144660 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63e40b1ac0 0x7f63e4142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 --2- 172.21.15.60:0/828491092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e40b8200 0x7f63e4144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 >> 172.21.15.60:0/828491092 conn(0x7f63e406b2f0 msgr2=0x7f63e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:46.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 shutdown_connections 2023-07-13T22:29:46.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.023+0000 7f63eadff700 1 -- 172.21.15.60:0/828491092 wait complete. 2023-07-13T22:29:46.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:29:46.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.247+0000 7fe748bff700 1 Processor -- start 2023-07-13T22:29:46.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.247+0000 7fe748bff700 1 -- start start 2023-07-13T22:29:46.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe748bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe74415cfa0 0x7fe7440cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe748bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe74415d370 0x7fe744070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe748bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7440714e0 0x7fe7440ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe748bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7440cd170 con 0x7fe74415cfa0 2023-07-13T22:29:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe748bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe744070830 con 0x7fe74415d370 2023-07-13T22:29:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe748bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7440cc940 con 0x7fe7440714e0 2023-07-13T22:29:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe74259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe74415cfa0 0x7fe7440cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe74259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe74415cfa0 0x7fe7440cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46260/0 (socket says 172.21.15.60:46260) 2023-07-13T22:29:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe74259c700 1 -- 172.21.15.60:0/3888623620 learned_addr learned my addr 172.21.15.60:0/3888623620 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe74259c700 1 -- 172.21.15.60:0/3888623620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7440714e0 msgr2=0x7fe7440ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.249+0000 7fe742d9d700 1 --2- 172.21.15.60:0/3888623620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7440714e0 0x7fe7440ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe74259c700 1 --2- 172.21.15.60:0/3888623620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7440714e0 0x7fe7440ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe73bfff700 1 --2- 172.21.15.60:0/3888623620 >> v2:172.21.15.105:3300/0 conn(0x7fe74415d370 0x7fe744070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe74259c700 1 -- 172.21.15.60:0/3888623620 >> v2:172.21.15.105:3300/0 conn(0x7fe74415d370 msgr2=0x7fe744070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:46.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe74259c700 1 --2- 172.21.15.60:0/3888623620 >> v2:172.21.15.105:3300/0 conn(0x7fe74415d370 0x7fe744070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe74259c700 1 -- 172.21.15.60:0/3888623620 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440ccbb0 con 0x7fe74415cfa0 2023-07-13T22:29:46.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe742d9d700 1 --2- 172.21.15.60:0/3888623620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7440714e0 0x7fe7440ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:46.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe73bfff700 1 --2- 172.21.15.60:0/3888623620 >> v2:172.21.15.105:3300/0 conn(0x7fe74415d370 0x7fe744070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:46.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe74259c700 1 --2- 172.21.15.60:0/3888623620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe74415cfa0 0x7fe7440cc260 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe7440cb670 tx=0x7fe734009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aff3dfe8a82dd10 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:46.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe741d9b700 1 -- 172.21.15.60:0/3888623620 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73400b870 con 0x7fe74415cfa0 2023-07-13T22:29:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe741d9b700 1 -- 172.21.15.60:0/3888623620 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73400b9d0 con 0x7fe74415cfa0 2023-07-13T22:29:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.250+0000 7fe741d9b700 1 -- 172.21.15.60:0/3888623620 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73400bd60 con 0x7fe74415cfa0 2023-07-13T22:29:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.251+0000 7fe748bff700 1 -- 172.21.15.60:0/3888623620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe74415cfa0 msgr2=0x7fe7440cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.251+0000 7fe748bff700 1 --2- 172.21.15.60:0/3888623620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe74415cfa0 0x7fe7440cc260 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe7440cb670 tx=0x7fe734009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.251+0000 7fe748bff700 1 -- 172.21.15.60:0/3888623620 shutdown_connections 2023-07-13T22:29:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.251+0000 7fe748bff700 1 --2- 172.21.15.60:0/3888623620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7440714e0 0x7fe7440ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.251+0000 7fe748bff700 1 --2- 172.21.15.60:0/3888623620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe74415cfa0 0x7fe7440cc260 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.251+0000 7fe748bff700 1 --2- 172.21.15.60:0/3888623620 >> v2:172.21.15.105:3300/0 conn(0x7fe74415d370 0x7fe744070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.251+0000 7fe748bff700 1 -- 172.21.15.60:0/3888623620 >> 172.21.15.60:0/3888623620 conn(0x7fe7440cb750 msgr2=0x7fe7440cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:46.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.251+0000 7fe748bff700 1 -- 172.21.15.60:0/3888623620 shutdown_connections 2023-07-13T22:29:46.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.251+0000 7fe748bff700 1 -- 172.21.15.60:0/3888623620 wait complete. 2023-07-13T22:29:46.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.253+0000 7fe748bff700 1 Processor -- start 2023-07-13T22:29:46.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.253+0000 7fe748bff700 1 -- start start 2023-07-13T22:29:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe748bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7440714e0 0x7fe7440c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe748bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe74415cfa0 0x7fe7441f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe748bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74415d370 0x7fe7440c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe748bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7440b4180 con 0x7fe7440714e0 2023-07-13T22:29:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe748bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7440b4a00 con 0x7fe74415d370 2023-07-13T22:29:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe748bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7440b29a0 con 0x7fe74415cfa0 2023-07-13T22:29:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe74259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7440714e0 0x7fe7440c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe74259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7440714e0 0x7fe7440c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46268/0 (socket says 172.21.15.60:46268) 2023-07-13T22:29:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe74259c700 1 -- 172.21.15.60:0/420115025 learned_addr learned my addr 172.21.15.60:0/420115025 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe73bfff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe74415cfa0 0x7fe7441f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.254+0000 7fe742d9d700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74415d370 0x7fe7440c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe74259c700 1 -- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe74415cfa0 msgr2=0x7fe7441f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe74259c700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe74415cfa0 0x7fe7441f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe74259c700 1 -- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74415d370 msgr2=0x7fe7440c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe74259c700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74415d370 0x7fe7440c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe74259c700 1 -- 172.21.15.60:0/420115025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe734009550 con 0x7fe7440714e0 2023-07-13T22:29:46.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe73bfff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe74415cfa0 0x7fe7441f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:46.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe742d9d700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74415d370 0x7fe7440c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:46.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe74259c700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7440714e0 0x7fe7440c6cd0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe734014040 tx=0x7fe734004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:46.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe73affd700 1 -- 172.21.15.60:0/420115025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734004520 con 0x7fe7440714e0 2023-07-13T22:29:46.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe73affd700 1 -- 172.21.15.60:0/420115025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73401e070 con 0x7fe7440714e0 2023-07-13T22:29:46.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe73affd700 1 -- 172.21.15.60:0/420115025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7340038e0 con 0x7fe7440714e0 2023-07-13T22:29:46.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7440b43b0 con 0x7fe7440714e0 2023-07-13T22:29:46.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.255+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7440b2c80 con 0x7fe7440714e0 2023-07-13T22:29:46.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.256+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74404e210 con 0x7fe7440714e0 2023-07-13T22:29:46.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.257+0000 7fe73affd700 1 -- 172.21.15.60:0/420115025 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe734003a40 con 0x7fe7440714e0 2023-07-13T22:29:46.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.258+0000 7fe73affd700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe72406ace0 0x7fe72406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.258+0000 7fe73affd700 1 -- 172.21.15.60:0/420115025 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe734020070 con 0x7fe7440714e0 2023-07-13T22:29:46.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.259+0000 7fe73bfff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe72406ace0 0x7fe72406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.259+0000 7fe73bfff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe72406ace0 0x7fe72406d190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe74406d4c0 tx=0x7fe72c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:46.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.263+0000 7fe73affd700 1 -- 172.21.15.60:0/420115025 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe734055c10 con 0x7fe7440714e0 2023-07-13T22:29:46.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.467+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7440c7230 con 0x7fe7440714e0 2023-07-13T22:29:46.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.468+0000 7fe73affd700 1 -- 172.21.15.60:0/420115025 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe734059230 con 0x7fe7440714e0 2023-07-13T22:29:46.448 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:29:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.470+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe72406ace0 msgr2=0x7fe72406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.470+0000 7fe748bff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe72406ace0 0x7fe72406d190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe74406d4c0 tx=0x7fe72c008040 comp rx=0 tx=0).stop 2023-07-13T22:29:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.470+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7440714e0 msgr2=0x7fe7440c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.470+0000 7fe748bff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7440714e0 0x7fe7440c6cd0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe734014040 tx=0x7fe734004750 comp rx=0 tx=0).stop 2023-07-13T22:29:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.471+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 shutdown_connections 2023-07-13T22:29:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.471+0000 7fe748bff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe72406ace0 0x7fe72406d190 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.471+0000 7fe748bff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7440714e0 0x7fe7440c6cd0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.471+0000 7fe748bff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74415d370 0x7fe7440c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.471+0000 7fe748bff700 1 --2- 172.21.15.60:0/420115025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe74415cfa0 0x7fe7441f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.471+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 >> 172.21.15.60:0/420115025 conn(0x7fe7440cb750 msgr2=0x7fe7440af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.471+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 shutdown_connections 2023-07-13T22:29:46.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.471+0000 7fe748bff700 1 -- 172.21.15.60:0/420115025 wait complete. 2023-07-13T22:29:46.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:29:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.691+0000 7f95bcdff700 1 Processor -- start 2023-07-13T22:29:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.692+0000 7f95bcdff700 1 -- start start 2023-07-13T22:29:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.693+0000 7f95bcdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95b80b2010 0x7f95b80cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.693+0000 7f95bcdff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b80d14a0 con 0x7f95b80d26d0 2023-07-13T22:29:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.693+0000 7f95bcdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b80d1780 con 0x7f95b80b5a00 2023-07-13T22:29:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.693+0000 7f95bcdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b80d2570 con 0x7f95b80b2010 2023-07-13T22:29:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.693+0000 7f95affff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f95b80d26d0 0x7f95b80cf6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51620/0 (socket says 172.21.15.60:51620) 2023-07-13T22:29:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.693+0000 7f95affff700 1 -- 172.21.15.60:0/3930033599 learned_addr learned my addr 172.21.15.60:0/3930033599 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.693+0000 7f95b659c700 1 --2- 172.21.15.60:0/3930033599 >> v2:172.21.15.105:3300/0 conn(0x7f95b80b2010 0x7f95b80cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.694+0000 7f95b5d9b700 1 -- 172.21.15.60:0/3930033599 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529369329 0 0) 0x7f95b80d14a0 con 0x7f95b80d26d0 2023-07-13T22:29:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.694+0000 7f95b5d9b700 1 -- 172.21.15.60:0/3930033599 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95a40035b0 con 0x7f95b80d26d0 2023-07-13T22:29:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.694+0000 7f95b659c700 1 -- 172.21.15.60:0/3930033599 >> v1:172.21.15.105:6789/0 conn(0x7f95b80b5a00 legacy=0x7f95b815fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.694+0000 7f95b659c700 1 -- 172.21.15.60:0/3930033599 >> v1:172.21.15.60:6789/0 conn(0x7f95b80d26d0 legacy=0x7f95b80cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.694+0000 7f95b659c700 1 -- 172.21.15.60:0/3930033599 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b80cd2a0 con 0x7f95b80b2010 2023-07-13T22:29:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.694+0000 7f95b659c700 1 --2- 172.21.15.60:0/3930033599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95b80b2010 0x7f95b80cf1a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009840 tx=0x7f95a8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a42b5bdfa4a8893 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.694+0000 7f95b5d9b700 1 -- 172.21.15.60:0/3930033599 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8011830 con 0x7f95b80b2010 2023-07-13T22:29:46.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.694+0000 7f95b5d9b700 1 -- 172.21.15.60:0/3930033599 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a8011990 con 0x7f95b80b2010 2023-07-13T22:29:46.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.694+0000 7f95b5d9b700 1 -- 172.21.15.60:0/3930033599 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8011d20 con 0x7f95b80b2010 2023-07-13T22:29:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.695+0000 7f95bcdff700 1 -- 172.21.15.60:0/3930033599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95b80b2010 msgr2=0x7f95b80cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.695+0000 7f95bcdff700 1 --2- 172.21.15.60:0/3930033599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95b80b2010 0x7f95b80cf1a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009840 tx=0x7f95a8009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.695+0000 7f95bcdff700 1 -- 172.21.15.60:0/3930033599 shutdown_connections 2023-07-13T22:29:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.695+0000 7f95bcdff700 1 --2- 172.21.15.60:0/3930033599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95b80b2010 0x7f95b80cf1a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.695+0000 7f95bcdff700 1 -- 172.21.15.60:0/3930033599 >> 172.21.15.60:0/3930033599 conn(0x7f95b806b380 msgr2=0x7f95b806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.695+0000 7f95bcdff700 1 -- 172.21.15.60:0/3930033599 shutdown_connections 2023-07-13T22:29:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.695+0000 7f95bcdff700 1 -- 172.21.15.60:0/3930033599 wait complete. 2023-07-13T22:29:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.697+0000 7f95bcdff700 1 Processor -- start 2023-07-13T22:29:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.697+0000 7f95bcdff700 1 -- start start 2023-07-13T22:29:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95bcdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80b2010 0x7f95b814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95bcdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95b80b5a00 0x7f95b815e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95bcdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95b80d26d0 0x7f95b815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95bcdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80aa100 con 0x7f95b80b2010 2023-07-13T22:29:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95bcdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80a9510 con 0x7f95b80b5a00 2023-07-13T22:29:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95bcdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80a9760 con 0x7f95b80d26d0 2023-07-13T22:29:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95b659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80b2010 0x7f95b814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95b659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80b2010 0x7f95b814df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46270/0 (socket says 172.21.15.60:46270) 2023-07-13T22:29:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95b659c700 1 -- 172.21.15.60:0/3790004638 learned_addr learned my addr 172.21.15.60:0/3790004638 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95affff700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95b80b5a00 0x7f95b815e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.698+0000 7f95b6d9d700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95b80d26d0 0x7f95b815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95b659c700 1 -- 172.21.15.60:0/3790004638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95b80d26d0 msgr2=0x7f95b815ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95b659c700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95b80d26d0 0x7f95b815ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95b659c700 1 -- 172.21.15.60:0/3790004638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95b80b5a00 msgr2=0x7f95b815e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95b659c700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95b80b5a00 0x7f95b815e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95b659c700 1 -- 172.21.15.60:0/3790004638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80094a0 con 0x7f95b80b2010 2023-07-13T22:29:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95b6d9d700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95b80d26d0 0x7f95b815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95b659c700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80b2010 0x7f95b814df70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009fd0 tx=0x7f95a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95aeffd700 1 -- 172.21.15.60:0/3790004638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a80036a0 con 0x7f95b80b2010 2023-07-13T22:29:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95aeffd700 1 -- 172.21.15.60:0/3790004638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a801f070 con 0x7f95b80b2010 2023-07-13T22:29:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b80a9a40 con 0x7f95b80b2010 2023-07-13T22:29:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.699+0000 7f95aeffd700 1 -- 172.21.15.60:0/3790004638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8003a20 con 0x7f95b80b2010 2023-07-13T22:29:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.700+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b814e4b0 con 0x7f95b80b2010 2023-07-13T22:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.701+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b80b8760 con 0x7f95b80b2010 2023-07-13T22:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.701+0000 7f95aeffd700 1 -- 172.21.15.60:0/3790004638 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f95a8003b80 con 0x7f95b80b2010 2023-07-13T22:29:46.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.702+0000 7f95aeffd700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f959806ace0 0x7f959806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:46.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.702+0000 7f95aeffd700 1 -- 172.21.15.60:0/3790004638 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f95a8021070 con 0x7f95b80b2010 2023-07-13T22:29:46.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.703+0000 7f95affff700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f959806ace0 0x7f959806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:46.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.703+0000 7f95affff700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f959806ace0 0x7f959806d190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f95b80b3bf0 tx=0x7f95a0007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.707+0000 7f95aeffd700 1 -- 172.21.15.60:0/3790004638 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f95a8055da0 con 0x7f95b80b2010 2023-07-13T22:29:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.918+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f95b80b8480 con 0x7f95b80b2010 2023-07-13T22:29:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.918+0000 7f95aeffd700 1 -- 172.21.15.60:0/3790004638 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) v1 ==== 84+0+126 (secure 0 0 0) 0x7f95a80593c0 con 0x7f95b80b2010 2023-07-13T22:29:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:46.898 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:29:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:29:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:29:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:29:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:29:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:29:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:46.900 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:29:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.920+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f959806ace0 msgr2=0x7f959806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.920+0000 7f95bcdff700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f959806ace0 0x7f959806d190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f95b80b3bf0 tx=0x7f95a0007dc0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.920+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80b2010 msgr2=0x7f95b814df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.920+0000 7f95bcdff700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80b2010 0x7f95b814df70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009fd0 tx=0x7f95a8004750 comp rx=0 tx=0).stop 2023-07-13T22:29:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.920+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 shutdown_connections 2023-07-13T22:29:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.920+0000 7f95bcdff700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f959806ace0 0x7f959806d190 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.920+0000 7f95bcdff700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80b2010 0x7f95b814df70 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.920+0000 7f95bcdff700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95b80d26d0 0x7f95b815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.920+0000 7f95bcdff700 1 --2- 172.21.15.60:0/3790004638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95b80b5a00 0x7f95b815e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:46.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.921+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 >> 172.21.15.60:0/3790004638 conn(0x7f95b806b380 msgr2=0x7f95b80b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:46.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.921+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 shutdown_connections 2023-07-13T22:29:46.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:46.921+0000 7f95bcdff700 1 -- 172.21.15.60:0/3790004638 wait complete. 2023-07-13T22:29:46.918 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:29:46.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:29:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.145+0000 7f207c9ff700 1 Processor -- start 2023-07-13T22:29:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.145+0000 7f207c9ff700 1 -- start start 2023-07-13T22:29:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.146+0000 7f207c9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2078159ea0 0x7f20780cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.146+0000 7f207c9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207815d200 con 0x7f20780d0890 2023-07-13T22:29:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.146+0000 7f207c9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207815d4e0 con 0x7f207815c9b0 2023-07-13T22:29:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.146+0000 7f207c9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20780d2e60 con 0x7f2078159ea0 2023-07-13T22:29:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f2076d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f20780d0890 0x7f20780cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51636/0 (socket says 172.21.15.60:51636) 2023-07-13T22:29:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f2076d9d700 1 -- 172.21.15.60:0/3519138931 learned_addr learned my addr 172.21.15.60:0/3519138931 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f207659c700 1 --2- 172.21.15.60:0/3519138931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2078159ea0 0x7f20780cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f207559a700 1 -- 172.21.15.60:0/3519138931 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232665678 0 0) 0x7f207815d200 con 0x7f20780d0890 2023-07-13T22:29:47.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f207559a700 1 -- 172.21.15.60:0/3519138931 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20600035b0 con 0x7f20780d0890 2023-07-13T22:29:47.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f207559a700 1 -- 172.21.15.60:0/3519138931 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565138657 0 0) 0x7f207815d4e0 con 0x7f207815c9b0 2023-07-13T22:29:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f207559a700 1 -- 172.21.15.60:0/3519138931 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f207815d200 con 0x7f207815c9b0 2023-07-13T22:29:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f207659c700 1 -- 172.21.15.60:0/3519138931 >> v1:172.21.15.105:6789/0 conn(0x7f207815c9b0 legacy=0x7f20780d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f207659c700 1 -- 172.21.15.60:0/3519138931 >> v1:172.21.15.60:6789/0 conn(0x7f20780d0890 legacy=0x7f20780cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f207559a700 1 -- 172.21.15.60:0/3519138931 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1112608627 0 0) 0x7f20600035b0 con 0x7f20780d0890 2023-07-13T22:29:47.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.147+0000 7f207659c700 1 -- 172.21.15.60:0/3519138931 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207815d7c0 con 0x7f2078159ea0 2023-07-13T22:29:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.148+0000 7f207659c700 1 --2- 172.21.15.60:0/3519138931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2078159ea0 0x7f20780cd6d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2068009870 tx=0x7f2068009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd15b4e343fad9c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.148+0000 7f207559a700 1 -- 172.21.15.60:0/3519138931 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068011830 con 0x7f2078159ea0 2023-07-13T22:29:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.148+0000 7f207559a700 1 -- 172.21.15.60:0/3519138931 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068011990 con 0x7f2078159ea0 2023-07-13T22:29:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.148+0000 7f207559a700 1 -- 172.21.15.60:0/3519138931 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068011d20 con 0x7f2078159ea0 2023-07-13T22:29:47.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.149+0000 7f207c9ff700 1 -- 172.21.15.60:0/3519138931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2078159ea0 msgr2=0x7f20780cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.149+0000 7f207c9ff700 1 --2- 172.21.15.60:0/3519138931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2078159ea0 0x7f20780cd6d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2068009870 tx=0x7f2068009b80 comp rx=0 tx=0).stop 2023-07-13T22:29:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.149+0000 7f207c9ff700 1 -- 172.21.15.60:0/3519138931 shutdown_connections 2023-07-13T22:29:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.149+0000 7f207c9ff700 1 --2- 172.21.15.60:0/3519138931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2078159ea0 0x7f20780cd6d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.149+0000 7f207c9ff700 1 -- 172.21.15.60:0/3519138931 >> 172.21.15.60:0/3519138931 conn(0x7f2078071600 msgr2=0x7f20780719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.149+0000 7f207c9ff700 1 -- 172.21.15.60:0/3519138931 shutdown_connections 2023-07-13T22:29:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.149+0000 7f207c9ff700 1 -- 172.21.15.60:0/3519138931 wait complete. 2023-07-13T22:29:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.150+0000 7f207c9ff700 1 Processor -- start 2023-07-13T22:29:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.151+0000 7f207c9ff700 1 -- start start 2023-07-13T22:29:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f207c9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20780d0890 0x7f20780b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f207c9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2078159ea0 0x7f20780b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f207c9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207815c9b0 0x7f20780cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f207c9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20780b61f0 con 0x7f2078159ea0 2023-07-13T22:29:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f207c9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20780b64d0 con 0x7f207815c9b0 2023-07-13T22:29:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f207c9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20780b67b0 con 0x7f20780d0890 2023-07-13T22:29:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f2075d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2078159ea0 0x7f20780b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f2075d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2078159ea0 0x7f20780b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46272/0 (socket says 172.21.15.60:46272) 2023-07-13T22:29:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f2075d9b700 1 -- 172.21.15.60:0/3833083339 learned_addr learned my addr 172.21.15.60:0/3833083339 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f207659c700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20780d0890 0x7f20780b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f2076d9d700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207815c9b0 0x7f20780cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f2075d9b700 1 -- 172.21.15.60:0/3833083339 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20780d0890 msgr2=0x7f20780b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f2075d9b700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20780d0890 0x7f20780b7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f2075d9b700 1 -- 172.21.15.60:0/3833083339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207815c9b0 msgr2=0x7f20780cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f2075d9b700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207815c9b0 0x7f20780cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.152+0000 7f2075d9b700 1 -- 172.21.15.60:0/3833083339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20680094a0 con 0x7f2078159ea0 2023-07-13T22:29:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.153+0000 7f2075d9b700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2078159ea0 0x7f20780b5af0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f206c00c500 tx=0x7f206c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.153+0000 7f2066ffd700 1 -- 172.21.15.60:0/3833083339 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c0052d0 con 0x7f2078159ea0 2023-07-13T22:29:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.153+0000 7f2066ffd700 1 -- 172.21.15.60:0/3833083339 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c005450 con 0x7f2078159ea0 2023-07-13T22:29:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.153+0000 7f2066ffd700 1 -- 172.21.15.60:0/3833083339 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c0038c0 con 0x7f2078159ea0 2023-07-13T22:29:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.153+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20780b6a90 con 0x7f2078159ea0 2023-07-13T22:29:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.153+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207814b070 con 0x7f2078159ea0 2023-07-13T22:29:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.154+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20780b70e0 con 0x7f2078159ea0 2023-07-13T22:29:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.155+0000 7f2066ffd700 1 -- 172.21.15.60:0/3833083339 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f206c003a20 con 0x7f2078159ea0 2023-07-13T22:29:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.156+0000 7f2066ffd700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f205806ace0 0x7f205806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.156+0000 7f2066ffd700 1 -- 172.21.15.60:0/3833083339 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f206c0885a0 con 0x7f2078159ea0 2023-07-13T22:29:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.156+0000 7f207659c700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f205806ace0 0x7f205806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.157+0000 7f207659c700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f205806ace0 0x7f205806d190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f20680097c0 tx=0x7f206801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.160+0000 7f2066ffd700 1 -- 172.21.15.60:0/3833083339 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f206c053900 con 0x7f2078159ea0 2023-07-13T22:29:47.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.368+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2078067a70 con 0x7f2078159ea0 2023-07-13T22:29:47.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.755+0000 7f2066ffd700 1 -- 172.21.15.60:0/3833083339 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) v1 ==== 138+0+0 (secure 0 0 0) 0x7f206c056f20 con 0x7f2078159ea0 2023-07-13T22:29:47.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.759+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f205806ace0 msgr2=0x7f205806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.759+0000 7f207c9ff700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f205806ace0 0x7f205806d190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f20680097c0 tx=0x7f206801c040 comp rx=0 tx=0).stop 2023-07-13T22:29:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.759+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2078159ea0 msgr2=0x7f20780b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.759+0000 7f207c9ff700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2078159ea0 0x7f20780b5af0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f206c00c500 tx=0x7f206c00c810 comp rx=0 tx=0).stop 2023-07-13T22:29:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.760+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 shutdown_connections 2023-07-13T22:29:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.760+0000 7f207c9ff700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f205806ace0 0x7f205806d190 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.760+0000 7f207c9ff700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f207815c9b0 0x7f20780cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.760+0000 7f207c9ff700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20780d0890 0x7f20780b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.760+0000 7f207c9ff700 1 --2- 172.21.15.60:0/3833083339 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2078159ea0 0x7f20780b5af0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.760+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 >> 172.21.15.60:0/3833083339 conn(0x7f2078071600 msgr2=0x7f20780cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.760+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 shutdown_connections 2023-07-13T22:29:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.760+0000 7f207c9ff700 1 -- 172.21.15.60:0/3833083339 wait complete. 2023-07-13T22:29:47.743 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:29:47.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:29:47.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.995+0000 7f308a1ff700 1 Processor -- start 2023-07-13T22:29:47.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.996+0000 7f308a1ff700 1 -- start start 2023-07-13T22:29:47.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.997+0000 7f308a1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f308415c5f0 0x7f308406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.997+0000 7f308a1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308406e5f0 con 0x7f308406b6f0 2023-07-13T22:29:47.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.997+0000 7f308a1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308406f430 con 0x7f308406f8c0 2023-07-13T22:29:47.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.997+0000 7f308a1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308406e820 con 0x7f308415c5f0 2023-07-13T22:29:47.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.997+0000 7f3083fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f308406b6f0 0x7f30840ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51640/0 (socket says 172.21.15.60:51640) 2023-07-13T22:29:47.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.997+0000 7f3083fff700 1 -- 172.21.15.60:0/2748398473 learned_addr learned my addr 172.21.15.60:0/2748398473 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:47.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.997+0000 7f30837fe700 1 --2- 172.21.15.60:0/2748398473 >> v2:172.21.15.105:3300/0 conn(0x7f308415c5f0 0x7f308406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.998+0000 7f3082ffd700 1 -- 172.21.15.60:0/2748398473 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274961019 0 0) 0x7f308406e5f0 con 0x7f308406b6f0 2023-07-13T22:29:47.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.998+0000 7f3082ffd700 1 -- 172.21.15.60:0/2748398473 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30700035b0 con 0x7f308406b6f0 2023-07-13T22:29:47.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.998+0000 7f30837fe700 1 -- 172.21.15.60:0/2748398473 >> v1:172.21.15.105:6789/0 conn(0x7f308406f8c0 legacy=0x7f308406afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.998+0000 7f30837fe700 1 -- 172.21.15.60:0/2748398473 >> v1:172.21.15.60:6789/0 conn(0x7f308406b6f0 legacy=0x7f30840ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.998+0000 7f30837fe700 1 -- 172.21.15.60:0/2748398473 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308406eb00 con 0x7f308415c5f0 2023-07-13T22:29:47.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.998+0000 7f30837fe700 1 --2- 172.21.15.60:0/2748398473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f308415c5f0 0x7f308406e0b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3074009820 tx=0x7f3074009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddcc828fc3f0e1a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:47.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.998+0000 7f3082ffd700 1 -- 172.21.15.60:0/2748398473 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074005e30 con 0x7f308415c5f0 2023-07-13T22:29:47.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.998+0000 7f3082ffd700 1 -- 172.21.15.60:0/2748398473 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3074006030 con 0x7f308415c5f0 2023-07-13T22:29:47.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.999+0000 7f3082ffd700 1 -- 172.21.15.60:0/2748398473 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740063c0 con 0x7f308415c5f0 2023-07-13T22:29:47.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.999+0000 7f308a1ff700 1 -- 172.21.15.60:0/2748398473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f308415c5f0 msgr2=0x7f308406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.999+0000 7f308a1ff700 1 --2- 172.21.15.60:0/2748398473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f308415c5f0 0x7f308406e0b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3074009820 tx=0x7f3074009b30 comp rx=0 tx=0).stop 2023-07-13T22:29:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.999+0000 7f308a1ff700 1 -- 172.21.15.60:0/2748398473 shutdown_connections 2023-07-13T22:29:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.999+0000 7f308a1ff700 1 --2- 172.21.15.60:0/2748398473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f308415c5f0 0x7f308406e0b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.999+0000 7f308a1ff700 1 -- 172.21.15.60:0/2748398473 >> 172.21.15.60:0/2748398473 conn(0x7f3084099c30 msgr2=0x7f30840d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.999+0000 7f308a1ff700 1 -- 172.21.15.60:0/2748398473 shutdown_connections 2023-07-13T22:29:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:47.999+0000 7f308a1ff700 1 -- 172.21.15.60:0/2748398473 wait complete. 2023-07-13T22:29:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.001+0000 7f308a1ff700 1 Processor -- start 2023-07-13T22:29:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.001+0000 7f308a1ff700 1 -- start start 2023-07-13T22:29:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f308a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f308406b6f0 0x7f30840b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f308a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f308406f8c0 0x7f30840c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f308a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f308415c5f0 0x7f30841569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f308a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30840c6fc0 con 0x7f308415c5f0 2023-07-13T22:29:47.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f308a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30840c6d40 con 0x7f308406f8c0 2023-07-13T22:29:47.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f308a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30840cb8e0 con 0x7f308406b6f0 2023-07-13T22:29:47.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f3083fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f308415c5f0 0x7f30841569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f3083fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f308415c5f0 0x7f30841569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46278/0 (socket says 172.21.15.60:46278) 2023-07-13T22:29:47.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f3083fff700 1 -- 172.21.15.60:0/2481308067 learned_addr learned my addr 172.21.15.60:0/2481308067 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:47.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.002+0000 7f30837fe700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f308406b6f0 0x7f30840b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f307bfff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f308406f8c0 0x7f30840c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f3083fff700 1 -- 172.21.15.60:0/2481308067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f308406b6f0 msgr2=0x7f30840b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f3083fff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f308406b6f0 0x7f30840b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f3083fff700 1 -- 172.21.15.60:0/2481308067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f308406f8c0 msgr2=0x7f30840c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f3083fff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f308406f8c0 0x7f30840c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f3083fff700 1 -- 172.21.15.60:0/2481308067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009430 con 0x7f308415c5f0 2023-07-13T22:29:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f3083fff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f308415c5f0 0x7f30841569e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f307c00cae0 tx=0x7f307c00cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f3080ff9700 1 -- 172.21.15.60:0/2481308067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c00aa60 con 0x7f308415c5f0 2023-07-13T22:29:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f3080ff9700 1 -- 172.21.15.60:0/2481308067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c00abc0 con 0x7f308415c5f0 2023-07-13T22:29:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.003+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30840b5ef0 con 0x7f308415c5f0 2023-07-13T22:29:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.004+0000 7f3080ff9700 1 -- 172.21.15.60:0/2481308067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c012400 con 0x7f308415c5f0 2023-07-13T22:29:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.004+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30840b64b0 con 0x7f308415c5f0 2023-07-13T22:29:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.005+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30840cbab0 con 0x7f308415c5f0 2023-07-13T22:29:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.006+0000 7f3080ff9700 1 -- 172.21.15.60:0/2481308067 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f307c01b3b0 con 0x7f308415c5f0 2023-07-13T22:29:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.007+0000 7f3080ff9700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f306406ad10 0x7f306406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.007+0000 7f3080ff9700 1 -- 172.21.15.60:0/2481308067 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f307c088ec0 con 0x7f308415c5f0 2023-07-13T22:29:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.007+0000 7f30837fe700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f306406ad10 0x7f306406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:47.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.008+0000 7f30837fe700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f306406ad10 0x7f306406d1c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3074009fd0 tx=0x7f3074008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.011+0000 7f3080ff9700 1 -- 172.21.15.60:0/2481308067 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f307c054160 con 0x7f308415c5f0 2023-07-13T22:29:48.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.222+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f308404e210 con 0x7f308415c5f0 2023-07-13T22:29:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.758+0000 7f3080ff9700 1 -- 172.21.15.60:0/2481308067 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) v1 ==== 170+0+0 (secure 0 0 0) 0x7f307c057780 con 0x7f308415c5f0 2023-07-13T22:29:48.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.767+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f306406ad10 msgr2=0x7f306406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:48.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.767+0000 7f308a1ff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f306406ad10 0x7f306406d1c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3074009fd0 tx=0x7f3074008070 comp rx=0 tx=0).stop 2023-07-13T22:29:48.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.767+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f308415c5f0 msgr2=0x7f30841569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:48.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.767+0000 7f308a1ff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f308415c5f0 0x7f30841569e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f307c00cae0 tx=0x7f307c00cdf0 comp rx=0 tx=0).stop 2023-07-13T22:29:48.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.770+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 shutdown_connections 2023-07-13T22:29:48.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.770+0000 7f308a1ff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f306406ad10 0x7f306406d1c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.770+0000 7f308a1ff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f308415c5f0 0x7f30841569e0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.770+0000 7f308a1ff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f308406b6f0 0x7f30840b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.770+0000 7f308a1ff700 1 --2- 172.21.15.60:0/2481308067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f308406f8c0 0x7f30840c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.770+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 >> 172.21.15.60:0/2481308067 conn(0x7f3084099c30 msgr2=0x7f308415a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.776+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 shutdown_connections 2023-07-13T22:29:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:48.776+0000 7f308a1ff700 1 -- 172.21.15.60:0/2481308067 wait complete. 2023-07-13T22:29:48.760 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:29:48.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:29:48.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.011+0000 7f030a1ff700 1 Processor -- start 2023-07-13T22:29:48.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.011+0000 7f030a1ff700 1 -- start start 2023-07-13T22:29:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.012+0000 7f030a1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f030406ce70 0x7f03040ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.012+0000 7f030a1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03040c8e90 con 0x7f03040add00 2023-07-13T22:29:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.012+0000 7f030a1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03040caf30 con 0x7f030406a5e0 2023-07-13T22:29:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.012+0000 7f030a1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03040ae0d0 con 0x7f030406ce70 2023-07-13T22:29:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.012+0000 7f03037fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f03040add00 0x7f0304069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51642/0 (socket says 172.21.15.60:51642) 2023-07-13T22:29:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.012+0000 7f03037fe700 1 -- 172.21.15.60:0/1897991886 learned_addr learned my addr 172.21.15.60:0/1897991886 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.012+0000 7f0303fff700 1 --2- 172.21.15.60:0/1897991886 >> v2:172.21.15.60:3300/0 conn(0x7f030406ce70 0x7f03040ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.013+0000 7f03027fc700 1 -- 172.21.15.60:0/1897991886 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585173534 0 0) 0x7f03040c8e90 con 0x7f03040add00 2023-07-13T22:29:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.013+0000 7f0303fff700 1 -- 172.21.15.60:0/1897991886 >> v1:172.21.15.105:6789/0 conn(0x7f030406a5e0 legacy=0x7f03040b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.013+0000 7f0303fff700 1 -- 172.21.15.60:0/1897991886 >> v1:172.21.15.60:6789/0 conn(0x7f03040add00 legacy=0x7f0304069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.013+0000 7f0303fff700 1 -- 172.21.15.60:0/1897991886 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03040c90e0 con 0x7f030406ce70 2023-07-13T22:29:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.013+0000 7f0303fff700 1 --2- 172.21.15.60:0/1897991886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030406ce70 0x7f03040ca9f0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009d90 tx=0x7f02fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b296778c9ec9a8e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.013+0000 7f03027fc700 1 -- 172.21.15.60:0/1897991886 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00e070 con 0x7f030406ce70 2023-07-13T22:29:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.013+0000 7f03027fc700 1 -- 172.21.15.60:0/1897991886 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc00bbd0 con 0x7f030406ce70 2023-07-13T22:29:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.013+0000 7f03027fc700 1 -- 172.21.15.60:0/1897991886 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc012430 con 0x7f030406ce70 2023-07-13T22:29:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.014+0000 7f030a1ff700 1 -- 172.21.15.60:0/1897991886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030406ce70 msgr2=0x7f03040ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.014+0000 7f030a1ff700 1 --2- 172.21.15.60:0/1897991886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030406ce70 0x7f03040ca9f0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009d90 tx=0x7f02fc00b330 comp rx=0 tx=0).stop 2023-07-13T22:29:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.014+0000 7f030a1ff700 1 -- 172.21.15.60:0/1897991886 shutdown_connections 2023-07-13T22:29:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.014+0000 7f030a1ff700 1 --2- 172.21.15.60:0/1897991886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030406ce70 0x7f03040ca9f0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.014+0000 7f030a1ff700 1 -- 172.21.15.60:0/1897991886 >> 172.21.15.60:0/1897991886 conn(0x7f03040cbf50 msgr2=0x7f03040ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.014+0000 7f030a1ff700 1 -- 172.21.15.60:0/1897991886 shutdown_connections 2023-07-13T22:29:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.014+0000 7f030a1ff700 1 -- 172.21.15.60:0/1897991886 wait complete. 2023-07-13T22:29:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.016+0000 7f030a1ff700 1 Processor -- start 2023-07-13T22:29:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.016+0000 7f030a1ff700 1 -- start start 2023-07-13T22:29:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f030a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f030406a5e0 0x7f03040c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f030a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f030406ce70 0x7f03040c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f030a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03040add00 0x7f0304152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f030a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030415bac0 con 0x7f03040add00 2023-07-13T22:29:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f030a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03040c6860 con 0x7f030406a5e0 2023-07-13T22:29:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f030a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03040c6b00 con 0x7f030406ce70 2023-07-13T22:29:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f0303fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03040add00 0x7f0304152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f0303fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03040add00 0x7f0304152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46298/0 (socket says 172.21.15.60:46298) 2023-07-13T22:29:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f0303fff700 1 -- 172.21.15.60:0/4222198234 learned_addr learned my addr 172.21.15.60:0/4222198234 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f03037fe700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f030406a5e0 0x7f03040c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f0302ffd700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f030406ce70 0x7f03040c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f0303fff700 1 -- 172.21.15.60:0/4222198234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f030406ce70 msgr2=0x7f03040c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f0303fff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f030406ce70 0x7f03040c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f0303fff700 1 -- 172.21.15.60:0/4222198234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f030406a5e0 msgr2=0x7f03040c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:48.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f0303fff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f030406a5e0 0x7f03040c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:48.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.017+0000 7f0303fff700 1 -- 172.21.15.60:0/4222198234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc009a70 con 0x7f03040add00 2023-07-13T22:29:48.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.018+0000 7f0303fff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03040add00 0x7f0304152db0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f02fc014040 tx=0x7f02fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:49.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.018+0000 7f02ebfff700 1 -- 172.21.15.60:0/4222198234 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc012430 con 0x7f03040add00 2023-07-13T22:29:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.018+0000 7f02ebfff700 1 -- 172.21.15.60:0/4222198234 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc020070 con 0x7f03040add00 2023-07-13T22:29:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.018+0000 7f02ebfff700 1 -- 172.21.15.60:0/4222198234 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc004280 con 0x7f03040add00 2023-07-13T22:29:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.018+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03040c6de0 con 0x7f03040add00 2023-07-13T22:29:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.018+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03040c73a0 con 0x7f03040add00 2023-07-13T22:29:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.019+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030404e210 con 0x7f03040add00 2023-07-13T22:29:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.020+0000 7f02ebfff700 1 -- 172.21.15.60:0/4222198234 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f02fc0043e0 con 0x7f03040add00 2023-07-13T22:29:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.022+0000 7f02ebfff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f02f006ace0 0x7f02f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:49.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.022+0000 7f02ebfff700 1 -- 172.21.15.60:0/4222198234 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f02fc022070 con 0x7f03040add00 2023-07-13T22:29:49.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.022+0000 7f03037fe700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f02f006ace0 0x7f02f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:49.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.023+0000 7f03037fe700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f02f006ace0 0x7f02f006d190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f02f4000f60 tx=0x7f02f4007db0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:49.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.027+0000 7f02ebfff700 1 -- 172.21.15.60:0/4222198234 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f02fc056450 con 0x7f03040add00 2023-07-13T22:29:49.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.288+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f030406cc10 con 0x7f03040add00 2023-07-13T22:29:49.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.782+0000 7f02ebfff700 1 -- 172.21.15.60:0/4222198234 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v69) v1 ==== 174+0+0 (secure 0 0 0) 0x7f02fc059a70 con 0x7f03040add00 2023-07-13T22:29:49.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f02f006ace0 msgr2=0x7f02f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f02f006ace0 0x7f02f006d190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f02f4000f60 tx=0x7f02f4007db0 comp rx=0 tx=0).stop 2023-07-13T22:29:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03040add00 msgr2=0x7f0304152db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03040add00 0x7f0304152db0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f02fc014040 tx=0x7f02fc004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 shutdown_connections 2023-07-13T22:29:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f02f006ace0 0x7f02f006d190 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03040add00 0x7f0304152db0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f030406a5e0 0x7f03040c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 --2- 172.21.15.60:0/4222198234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f030406ce70 0x7f03040c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.793+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 >> 172.21.15.60:0/4222198234 conn(0x7f03040cbf50 msgr2=0x7f03040cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.796+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 shutdown_connections 2023-07-13T22:29:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:49.798+0000 7f030a1ff700 1 -- 172.21.15.60:0/4222198234 wait complete. 2023-07-13T22:29:49.784 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 17 and data pool 18 2023-07-13T22:29:49.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:50.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.032+0000 7f98b59ff700 1 Processor -- start 2023-07-13T22:29:50.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.032+0000 7f98b59ff700 1 -- start start 2023-07-13T22:29:50.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.033+0000 7f98b59ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98b00d35c0 0x7f98b00d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.033+0000 7f98b59ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 0x7f98b015c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.033+0000 7f98b59ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b00d09d0 con 0x7f98b00b8570 2023-07-13T22:29:50.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.033+0000 7f98b59ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b00d3b30 con 0x7f98b00d35c0 2023-07-13T22:29:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.033+0000 7f98b59ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00d1870 con 0x7f98b00d13f0 2023-07-13T22:29:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98ae7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98b00d35c0 0x7f98b00d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98ae7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98b00d35c0 0x7f98b00d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46304/0 (socket says 172.21.15.60:46304) 2023-07-13T22:29:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98ae7fc700 1 -- 172.21.15.60:0/3918519210 learned_addr learned my addr 172.21.15.60:0/3918519210 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:50.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98af7fe700 1 --2- 172.21.15.60:0/3918519210 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 0x7f98b015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98ae7fc700 1 -- 172.21.15.60:0/3918519210 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 msgr2=0x7f98b015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98ae7fc700 1 --2- 172.21.15.60:0/3918519210 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 0x7f98b015c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98ae7fc700 1 -- 172.21.15.60:0/3918519210 >> v1:172.21.15.60:6789/0 conn(0x7f98b00b8570 legacy=0x7f98b00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98ae7fc700 1 -- 172.21.15.60:0/3918519210 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00cda10 con 0x7f98b00d35c0 2023-07-13T22:29:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98adffb700 1 -- 172.21.15.60:0/3918519210 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653909828 0 0) 0x7f98b00d09d0 con 0x7f98b00b8570 2023-07-13T22:29:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98af7fe700 1 --2- 172.21.15.60:0/3918519210 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 0x7f98b015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:50.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.034+0000 7f98ae7fc700 1 --2- 172.21.15.60:0/3918519210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00d35c0 0x7f98b00d0eb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f98a40098f0 tx=0x7f98a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dcb33f6b89f3a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:50.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98adffb700 1 -- 172.21.15.60:0/3918519210 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a400b8a0 con 0x7f98b00d35c0 2023-07-13T22:29:50.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98adffb700 1 -- 172.21.15.60:0/3918519210 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a400ba00 con 0x7f98b00d35c0 2023-07-13T22:29:50.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98adffb700 1 -- 172.21.15.60:0/3918519210 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a400bd90 con 0x7f98b00d35c0 2023-07-13T22:29:50.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98b59ff700 1 -- 172.21.15.60:0/3918519210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00d35c0 msgr2=0x7f98b00d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98b59ff700 1 --2- 172.21.15.60:0/3918519210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00d35c0 0x7f98b00d0eb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f98a40098f0 tx=0x7f98a4009c00 comp rx=0 tx=0).stop 2023-07-13T22:29:50.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98b59ff700 1 -- 172.21.15.60:0/3918519210 shutdown_connections 2023-07-13T22:29:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98b59ff700 1 --2- 172.21.15.60:0/3918519210 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 0x7f98b015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98b59ff700 1 --2- 172.21.15.60:0/3918519210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00d35c0 0x7f98b00d0eb0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98b59ff700 1 -- 172.21.15.60:0/3918519210 >> 172.21.15.60:0/3918519210 conn(0x7f98b0071600 msgr2=0x7f98b00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98b59ff700 1 -- 172.21.15.60:0/3918519210 shutdown_connections 2023-07-13T22:29:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.035+0000 7f98b59ff700 1 -- 172.21.15.60:0/3918519210 wait complete. 2023-07-13T22:29:50.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.037+0000 7f98b59ff700 1 Processor -- start 2023-07-13T22:29:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.037+0000 7f98b59ff700 1 -- start start 2023-07-13T22:29:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.038+0000 7f98b59ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00b8570 0x7f98b014fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.038+0000 7f98b59ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 0x7f98b00cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.038+0000 7f98b59ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98b00d35c0 0x7f98b00cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.038+0000 7f98b59ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00c8bc0 con 0x7f98b00b8570 2023-07-13T22:29:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.038+0000 7f98b59ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00c7fb0 con 0x7f98b00d35c0 2023-07-13T22:29:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.038+0000 7f98b59ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00c8290 con 0x7f98b00d13f0 2023-07-13T22:29:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98aeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00b8570 0x7f98b014fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98aeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00b8570 0x7f98b014fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46306/0 (socket says 172.21.15.60:46306) 2023-07-13T22:29:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98aeffd700 1 -- 172.21.15.60:0/1906575770 learned_addr learned my addr 172.21.15.60:0/1906575770 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98ae7fc700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 0x7f98b00cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98af7fe700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98b00d35c0 0x7f98b00cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98aeffd700 1 -- 172.21.15.60:0/1906575770 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 msgr2=0x7f98b00cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98aeffd700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 0x7f98b00cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98aeffd700 1 -- 172.21.15.60:0/1906575770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98b00d35c0 msgr2=0x7f98b00cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98aeffd700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98b00d35c0 0x7f98b00cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98aeffd700 1 -- 172.21.15.60:0/1906575770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a4009550 con 0x7f98b00b8570 2023-07-13T22:29:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98af7fe700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98b00d35c0 0x7f98b00cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.039+0000 7f98aeffd700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00b8570 0x7f98b014fc80 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f98a000c4d0 tx=0x7f98a000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.040+0000 7f9897fff700 1 -- 172.21.15.60:0/1906575770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a00054e0 con 0x7f98b00b8570 2023-07-13T22:29:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.040+0000 7f9897fff700 1 -- 172.21.15.60:0/1906575770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0003680 con 0x7f98b00b8570 2023-07-13T22:29:50.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.040+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b00c8570 con 0x7f98b00b8570 2023-07-13T22:29:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.040+0000 7f9897fff700 1 -- 172.21.15.60:0/1906575770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0003a30 con 0x7f98b00b8570 2023-07-13T22:29:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.040+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b00c7230 con 0x7f98b00b8570 2023-07-13T22:29:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.041+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b004e280 con 0x7f98b00b8570 2023-07-13T22:29:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.042+0000 7f9897fff700 1 -- 172.21.15.60:0/1906575770 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98a0003b90 con 0x7f98b00b8570 2023-07-13T22:29:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.043+0000 7f9897fff700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f989c06ace0 0x7f989c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.043+0000 7f9897fff700 1 -- 172.21.15.60:0/1906575770 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f98a00886a0 con 0x7f98b00b8570 2023-07-13T22:29:50.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.043+0000 7f98ae7fc700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f989c06ace0 0x7f989c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.044+0000 7f98ae7fc700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f989c06ace0 0x7f989c06d190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f98a4000c00 tx=0x7f98a4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:50.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.047+0000 7f9897fff700 1 -- 172.21.15.60:0/1906575770 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f98a0053620 con 0x7f98b00b8570 2023-07-13T22:29:50.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.255+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98b0065ee0 con 0x7f98b00b8570 2023-07-13T22:29:50.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.256+0000 7f9897fff700 1 -- 172.21.15.60:0/1906575770 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f98a0056c40 con 0x7f98b00b8570 2023-07-13T22:29:50.236 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:50.236 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":111,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:29:49.290570+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:29:47.410626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-13T22:29:48.264611+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:50.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f989c06ace0 msgr2=0x7f989c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f989c06ace0 0x7f989c06d190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f98a4000c00 tx=0x7f98a4008030 comp rx=0 tx=0).stop 2023-07-13T22:29:50.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00b8570 msgr2=0x7f98b014fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00b8570 0x7f98b014fc80 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f98a000c4d0 tx=0x7f98a000c890 comp rx=0 tx=0).stop 2023-07-13T22:29:50.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 shutdown_connections 2023-07-13T22:29:50.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f989c06ace0 0x7f989c06d190 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98b00b8570 0x7f98b014fc80 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98b00d35c0 0x7f98b00cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 --2- 172.21.15.60:0/1906575770 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98b00d13f0 0x7f98b00cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 >> 172.21.15.60:0/1906575770 conn(0x7f98b0071600 msgr2=0x7f98b015dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:50.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 shutdown_connections 2023-07-13T22:29:50.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.259+0000 7f98b59ff700 1 -- 172.21.15.60:0/1906575770 wait complete. 2023-07-13T22:29:50.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:50.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.480+0000 7f5c58fff700 1 Processor -- start 2023-07-13T22:29:50.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.480+0000 7f5c58fff700 1 -- start start 2023-07-13T22:29:50.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.481+0000 7f5c58fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c540d3750 0x7f5c540cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c58fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c540cfad0 0x7f5c540d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c58fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c540cd950 con 0x7f5c540b8400 2023-07-13T22:29:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c58fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c540cff50 con 0x7f5c540d3750 2023-07-13T22:29:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c58fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c540d0ac0 con 0x7f5c540cfad0 2023-07-13T22:29:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c4bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c540d3750 0x7f5c540cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c5259c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5c540b8400 0x7f5c540cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51660/0 (socket says 172.21.15.60:51660) 2023-07-13T22:29:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c5259c700 1 -- 172.21.15.60:0/3739516023 learned_addr learned my addr 172.21.15.60:0/3739516023 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c4bfff700 1 -- 172.21.15.60:0/3739516023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c540cfad0 msgr2=0x7f5c540d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c4bfff700 1 --2- 172.21.15.60:0/3739516023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c540cfad0 0x7f5c540d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c4bfff700 1 -- 172.21.15.60:0/3739516023 >> v1:172.21.15.60:6789/0 conn(0x7f5c540b8400 legacy=0x7f5c540cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c4bfff700 1 -- 172.21.15.60:0/3739516023 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c540b87d0 con 0x7f5c540d3750 2023-07-13T22:29:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.482+0000 7f5c51d9b700 1 -- 172.21.15.60:0/3739516023 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126331829 0 0) 0x7f5c44001110 con 0x7f5c540b8400 2023-07-13T22:29:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c4bfff700 1 --2- 172.21.15.60:0/3739516023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540d3750 0x7f5c540cf590 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009840 tx=0x7f5c3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5a488cc33c955a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:50.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c51d9b700 1 -- 172.21.15.60:0/3739516023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c011830 con 0x7f5c540d3750 2023-07-13T22:29:50.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c51d9b700 1 -- 172.21.15.60:0/3739516023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c011990 con 0x7f5c540d3750 2023-07-13T22:29:50.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c51d9b700 1 -- 172.21.15.60:0/3739516023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c011d20 con 0x7f5c540d3750 2023-07-13T22:29:50.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c58fff700 1 -- 172.21.15.60:0/3739516023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540d3750 msgr2=0x7f5c540cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c58fff700 1 --2- 172.21.15.60:0/3739516023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540d3750 0x7f5c540cf590 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009840 tx=0x7f5c3c009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:50.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c58fff700 1 -- 172.21.15.60:0/3739516023 shutdown_connections 2023-07-13T22:29:50.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c58fff700 1 --2- 172.21.15.60:0/3739516023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c540cfad0 0x7f5c540d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c58fff700 1 --2- 172.21.15.60:0/3739516023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540d3750 0x7f5c540cf590 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c58fff700 1 -- 172.21.15.60:0/3739516023 >> 172.21.15.60:0/3739516023 conn(0x7f5c5406b390 msgr2=0x7f5c5406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.483+0000 7f5c58fff700 1 -- 172.21.15.60:0/3739516023 shutdown_connections 2023-07-13T22:29:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.484+0000 7f5c58fff700 1 -- 172.21.15.60:0/3739516023 wait complete. 2023-07-13T22:29:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.485+0000 7f5c58fff700 1 Processor -- start 2023-07-13T22:29:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.485+0000 7f5c58fff700 1 -- start start 2023-07-13T22:29:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.486+0000 7f5c58fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540b8400 0x7f5c54150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.486+0000 7f5c58fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c540cfad0 0x7f5c540c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.486+0000 7f5c58fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c540d3750 0x7f5c5415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.486+0000 7f5c58fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c540c7ca0 con 0x7f5c540b8400 2023-07-13T22:29:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.486+0000 7f5c58fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c540c7f80 con 0x7f5c540d3750 2023-07-13T22:29:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.486+0000 7f5c58fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c540c8260 con 0x7f5c540cfad0 2023-07-13T22:29:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c5259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540b8400 0x7f5c54150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c5259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540b8400 0x7f5c54150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46324/0 (socket says 172.21.15.60:46324) 2023-07-13T22:29:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c5259c700 1 -- 172.21.15.60:0/1790850561 learned_addr learned my addr 172.21.15.60:0/1790850561 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:50.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c4bfff700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c540cfad0 0x7f5c540c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c52d9d700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c540d3750 0x7f5c5415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c5259c700 1 -- 172.21.15.60:0/1790850561 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c540cfad0 msgr2=0x7f5c540c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c5259c700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c540cfad0 0x7f5c540c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c5259c700 1 -- 172.21.15.60:0/1790850561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c540d3750 msgr2=0x7f5c5415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c5259c700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c540d3750 0x7f5c5415c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c5259c700 1 -- 172.21.15.60:0/1790850561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c0094a0 con 0x7f5c540b8400 2023-07-13T22:29:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.487+0000 7f5c5259c700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540b8400 0x7f5c54150520 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c550 tx=0x7f5c4400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.488+0000 7f5c4affd700 1 -- 172.21.15.60:0/1790850561 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44004c80 con 0x7f5c540b8400 2023-07-13T22:29:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.488+0000 7f5c4affd700 1 -- 172.21.15.60:0/1790850561 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c44004e00 con 0x7f5c540b8400 2023-07-13T22:29:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.488+0000 7f5c4affd700 1 -- 172.21.15.60:0/1790850561 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44007310 con 0x7f5c540b8400 2023-07-13T22:29:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.488+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c540c8540 con 0x7f5c540b8400 2023-07-13T22:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.488+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c54151280 con 0x7f5c540b8400 2023-07-13T22:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.489+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c540c8a60 con 0x7f5c540b8400 2023-07-13T22:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.490+0000 7f5c4affd700 1 -- 172.21.15.60:0/1790850561 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c44007470 con 0x7f5c540b8400 2023-07-13T22:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.491+0000 7f5c4affd700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c3406ace0 0x7f5c3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.491+0000 7f5c4affd700 1 -- 172.21.15.60:0/1790850561 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c44088fb0 con 0x7f5c540b8400 2023-07-13T22:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.491+0000 7f5c4bfff700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c3406ace0 0x7f5c3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.491+0000 7f5c4bfff700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c3406ace0 0x7f5c3406d190 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5c540d3b60 tx=0x7f5c3c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.495+0000 7f5c4affd700 1 -- 172.21.15.60:0/1790850561 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5c540c8a60 con 0x7f5c540b8400 2023-07-13T22:29:50.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.702+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c540cd950 con 0x7f5c540b8400 2023-07-13T22:29:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.703+0000 7f5c4affd700 1 -- 172.21.15.60:0/1790850561 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+18280 (secure 0 0 0) 0x7f5c540cd950 con 0x7f5c540b8400 2023-07-13T22:29:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:50.683 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":111,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:29:49.290570+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:29:47.410626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-13T22:29:48.264611+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:50.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c3406ace0 msgr2=0x7f5c3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c3406ace0 0x7f5c3406d190 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5c540d3b60 tx=0x7f5c3c008030 comp rx=0 tx=0).stop 2023-07-13T22:29:50.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540b8400 msgr2=0x7f5c54150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540b8400 0x7f5c54150520 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c550 tx=0x7f5c4400c860 comp rx=0 tx=0).stop 2023-07-13T22:29:50.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 shutdown_connections 2023-07-13T22:29:50.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5c3406ace0 0x7f5c3406d190 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c540b8400 0x7f5c54150520 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c540d3750 0x7f5c5415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 --2- 172.21.15.60:0/1790850561 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c540cfad0 0x7f5c540c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 >> 172.21.15.60:0/1790850561 conn(0x7f5c5406b390 msgr2=0x7f5c540d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:50.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.705+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 shutdown_connections 2023-07-13T22:29:50.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.706+0000 7f5c58fff700 1 -- 172.21.15.60:0/1790850561 wait complete. 2023-07-13T22:29:50.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:29:50.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.932+0000 7f50d4fff700 1 Processor -- start 2023-07-13T22:29:50.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.932+0000 7f50d4fff700 1 -- start start 2023-07-13T22:29:50.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.933+0000 7f50d4fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50d00b8200 0x7f50d00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.933+0000 7f50d4fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d00d1870 con 0x7f50d00b1ac0 2023-07-13T22:29:50.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.933+0000 7f50d4fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d0160d80 con 0x7f50d00cede0 2023-07-13T22:29:50.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.933+0000 7f50d4fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d00cf370 con 0x7f50d00b8200 2023-07-13T22:29:50.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.933+0000 7f50cdd9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50d00b1ac0 0x7f50d015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51664/0 (socket says 172.21.15.60:51664) 2023-07-13T22:29:50.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.933+0000 7f50cdd9b700 1 -- 172.21.15.60:0/2946193372 learned_addr learned my addr 172.21.15.60:0/2946193372 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:50.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.934+0000 7f50ce59c700 1 --2- 172.21.15.60:0/2946193372 >> v2:172.21.15.105:3300/0 conn(0x7f50d00b8200 0x7f50d00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.934+0000 7f50cd59a700 1 -- 172.21.15.60:0/2946193372 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246811852 0 0) 0x7f50d00d1870 con 0x7f50d00b1ac0 2023-07-13T22:29:50.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.934+0000 7f50cd59a700 1 -- 172.21.15.60:0/2946193372 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b80035b0 con 0x7f50d00b1ac0 2023-07-13T22:29:50.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.934+0000 7f50ce59c700 1 -- 172.21.15.60:0/2946193372 >> v1:172.21.15.105:6789/0 conn(0x7f50d00cede0 legacy=0x7f50d0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.934+0000 7f50ce59c700 1 -- 172.21.15.60:0/2946193372 >> v1:172.21.15.60:6789/0 conn(0x7f50d00b1ac0 legacy=0x7f50d015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.934+0000 7f50ce59c700 1 -- 172.21.15.60:0/2946193372 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00d12b0 con 0x7f50d00b8200 2023-07-13T22:29:50.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.935+0000 7f50ce59c700 1 --2- 172.21.15.60:0/2946193372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00b8200 0x7f50d00cf640 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009840 tx=0x7f50c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f60a3aecdb201e05 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:50.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.935+0000 7f50cd59a700 1 -- 172.21.15.60:0/2946193372 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0011830 con 0x7f50d00b8200 2023-07-13T22:29:50.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.935+0000 7f50cd59a700 1 -- 172.21.15.60:0/2946193372 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0011990 con 0x7f50d00b8200 2023-07-13T22:29:50.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.935+0000 7f50cd59a700 1 -- 172.21.15.60:0/2946193372 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0011d00 con 0x7f50d00b8200 2023-07-13T22:29:50.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.935+0000 7f50d4fff700 1 -- 172.21.15.60:0/2946193372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00b8200 msgr2=0x7f50d00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.935+0000 7f50d4fff700 1 --2- 172.21.15.60:0/2946193372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00b8200 0x7f50d00cf640 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009840 tx=0x7f50c0009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:50.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.935+0000 7f50d4fff700 1 -- 172.21.15.60:0/2946193372 shutdown_connections 2023-07-13T22:29:50.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.935+0000 7f50d4fff700 1 --2- 172.21.15.60:0/2946193372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00b8200 0x7f50d00cf640 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.935+0000 7f50d4fff700 1 -- 172.21.15.60:0/2946193372 >> 172.21.15.60:0/2946193372 conn(0x7f50d006b2f0 msgr2=0x7f50d006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:50.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.936+0000 7f50d4fff700 1 -- 172.21.15.60:0/2946193372 shutdown_connections 2023-07-13T22:29:50.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.936+0000 7f50d4fff700 1 -- 172.21.15.60:0/2946193372 wait complete. 2023-07-13T22:29:50.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.937+0000 7f50d4fff700 1 Processor -- start 2023-07-13T22:29:50.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.937+0000 7f50d4fff700 1 -- start start 2023-07-13T22:29:50.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.938+0000 7f50d4fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d00b1ac0 0x7f50d00c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.938+0000 7f50d4fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d00b8200 0x7f50d00c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.938+0000 7f50d4fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00cede0 0x7f50d01f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.938+0000 7f50d4fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00cb3d0 con 0x7f50d00b1ac0 2023-07-13T22:29:50.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.938+0000 7f50d4fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00ca7c0 con 0x7f50d00cede0 2023-07-13T22:29:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.938+0000 7f50d4fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00caaa0 con 0x7f50d00b8200 2023-07-13T22:29:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.938+0000 7f50ce59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d00b1ac0 0x7f50d00c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ce59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d00b1ac0 0x7f50d00c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46340/0 (socket says 172.21.15.60:46340) 2023-07-13T22:29:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ce59c700 1 -- 172.21.15.60:0/3409604330 learned_addr learned my addr 172.21.15.60:0/3409604330 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50cdd9b700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d00b8200 0x7f50d00c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ce59c700 1 -- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d00b8200 msgr2=0x7f50d00c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ced9d700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00cede0 0x7f50d01f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ce59c700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d00b8200 0x7f50d00c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ce59c700 1 -- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00cede0 msgr2=0x7f50d01f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:50.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ce59c700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00cede0 0x7f50d01f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:50.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ce59c700 1 -- 172.21.15.60:0/3409604330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00094a0 con 0x7f50d00b1ac0 2023-07-13T22:29:50.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ced9d700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00cede0 0x7f50d01f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:50.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50cdd9b700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d00b8200 0x7f50d00c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:50.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50ce59c700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d00b1ac0 0x7f50d00c8e30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009fd0 tx=0x7f50c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:50.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50beffd700 1 -- 172.21.15.60:0/3409604330 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c00036a0 con 0x7f50d00b1ac0 2023-07-13T22:29:50.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50beffd700 1 -- 172.21.15.60:0/3409604330 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c001f070 con 0x7f50d00b1ac0 2023-07-13T22:29:50.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.939+0000 7f50beffd700 1 -- 172.21.15.60:0/3409604330 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0003a20 con 0x7f50d00b1ac0 2023-07-13T22:29:50.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.940+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d00cad80 con 0x7f50d00b1ac0 2023-07-13T22:29:50.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.940+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d014d610 con 0x7f50d00b1ac0 2023-07-13T22:29:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.941+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d004e210 con 0x7f50d00b1ac0 2023-07-13T22:29:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.942+0000 7f50beffd700 1 -- 172.21.15.60:0/3409604330 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50c0003b80 con 0x7f50d00b1ac0 2023-07-13T22:29:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.942+0000 7f50beffd700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b006ace0 0x7f50b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.943+0000 7f50beffd700 1 -- 172.21.15.60:0/3409604330 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f50c0021070 con 0x7f50d00b1ac0 2023-07-13T22:29:50.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.943+0000 7f50cdd9b700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b006ace0 0x7f50b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:50.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.943+0000 7f50cdd9b700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b006ace0 0x7f50b006d190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f50d014da80 tx=0x7f50c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:50.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:50.947+0000 7f50beffd700 1 -- 172.21.15.60:0/3409604330 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f50c0055ef0 con 0x7f50d00b1ac0 2023-07-13T22:29:51.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.201+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f50d00b3a30 con 0x7f50d00b1ac0 2023-07-13T22:29:51.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.791+0000 7f50beffd700 1 -- 172.21.15.60:0/3409604330 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) v1 ==== 118+0+0 (secure 0 0 0) 0x7f50c00150b0 con 0x7f50d00b1ac0 2023-07-13T22:29:51.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b006ace0 msgr2=0x7f50b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:51.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b006ace0 0x7f50b006d190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f50d014da80 tx=0x7f50c4009040 comp rx=0 tx=0).stop 2023-07-13T22:29:51.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d00b1ac0 msgr2=0x7f50d00c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:51.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d00b1ac0 0x7f50d00c8e30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009fd0 tx=0x7f50c0004750 comp rx=0 tx=0).stop 2023-07-13T22:29:51.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 shutdown_connections 2023-07-13T22:29:51.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f50b006ace0 0x7f50b006d190 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:51.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d00b1ac0 0x7f50d00c8e30 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:51.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50d00cede0 0x7f50d01f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:51.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 --2- 172.21.15.60:0/3409604330 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50d00b8200 0x7f50d00c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:51.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 >> 172.21.15.60:0/3409604330 conn(0x7f50d006b2f0 msgr2=0x7f50d00b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:51.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 shutdown_connections 2023-07-13T22:29:51.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:51.795+0000 7f50d4fff700 1 -- 172.21.15.60:0/3409604330 wait complete. 2023-07-13T22:29:51.795 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:29:51.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:29:52.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.027+0000 7f607cdff700 1 Processor -- start 2023-07-13T22:29:52.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.027+0000 7f607cdff700 1 -- start start 2023-07-13T22:29:52.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.028+0000 7f607cdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f60780b8200 0x7f60780cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:52.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.028+0000 7f607cdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60780cede0 0x7f60780cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:52.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.028+0000 7f607cdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60780d1870 con 0x7f60780b1ac0 2023-07-13T22:29:52.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.028+0000 7f607cdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60780cf370 con 0x7f60780cede0 2023-07-13T22:29:52.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.028+0000 7f607cdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60780d12b0 con 0x7f60780b8200 2023-07-13T22:29:52.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.029+0000 7f6076d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60780cede0 0x7f60780cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:52.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.029+0000 7f6076d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60780cede0 0x7f60780cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46346/0 (socket says 172.21.15.60:46346) 2023-07-13T22:29:52.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.029+0000 7f6076d9d700 1 -- 172.21.15.60:0/12679837 learned_addr learned my addr 172.21.15.60:0/12679837 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:52.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.029+0000 7f607659c700 1 --2- 172.21.15.60:0/12679837 >> v2:172.21.15.105:3300/0 conn(0x7f60780b8200 0x7f60780cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:52.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.029+0000 7f6076d9d700 1 -- 172.21.15.60:0/12679837 >> v2:172.21.15.105:3300/0 conn(0x7f60780b8200 msgr2=0x7f60780cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:52.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.029+0000 7f6076d9d700 1 --2- 172.21.15.60:0/12679837 >> v2:172.21.15.105:3300/0 conn(0x7f60780b8200 0x7f60780cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.029+0000 7f6076d9d700 1 -- 172.21.15.60:0/12679837 >> v1:172.21.15.105:6789/0 conn(0x7f60780b1ac0 legacy=0x7f607815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:52.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.029+0000 7f6076d9d700 1 -- 172.21.15.60:0/12679837 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60780d1590 con 0x7f60780cede0 2023-07-13T22:29:52.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.029+0000 7f6076d9d700 1 --2- 172.21.15.60:0/12679837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 0x7f60780cfb80 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f60780b2660 tx=0x7f607000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9685b9c5f0f524b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:52.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607559a700 1 -- 172.21.15.60:0/12679837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607000e070 con 0x7f60780cede0 2023-07-13T22:29:52.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607559a700 1 -- 172.21.15.60:0/12679837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607000b910 con 0x7f60780cede0 2023-07-13T22:29:52.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607559a700 1 -- 172.21.15.60:0/12679837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607000bcc0 con 0x7f60780cede0 2023-07-13T22:29:52.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607cdff700 1 -- 172.21.15.60:0/12679837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 msgr2=0x7f60780cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:52.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607cdff700 1 --2- 172.21.15.60:0/12679837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 0x7f60780cfb80 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f60780b2660 tx=0x7f607000b330 comp rx=0 tx=0).stop 2023-07-13T22:29:52.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607cdff700 1 -- 172.21.15.60:0/12679837 shutdown_connections 2023-07-13T22:29:52.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607cdff700 1 --2- 172.21.15.60:0/12679837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 0x7f60780cfb80 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607cdff700 1 --2- 172.21.15.60:0/12679837 >> v2:172.21.15.105:3300/0 conn(0x7f60780b8200 0x7f60780cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607cdff700 1 -- 172.21.15.60:0/12679837 >> 172.21.15.60:0/12679837 conn(0x7f607806b2f0 msgr2=0x7f607806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:52.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607cdff700 1 -- 172.21.15.60:0/12679837 shutdown_connections 2023-07-13T22:29:52.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.030+0000 7f607cdff700 1 -- 172.21.15.60:0/12679837 wait complete. 2023-07-13T22:29:52.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.032+0000 7f607cdff700 1 Processor -- start 2023-07-13T22:29:52.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.032+0000 7f607cdff700 1 -- start start 2023-07-13T22:29:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.033+0000 7f607cdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60780b1ac0 0x7f6078155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.033+0000 7f607cdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60780b8200 0x7f6078159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.033+0000 7f607cdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 0x7f60780cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.033+0000 7f607cdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60781567e0 con 0x7f60780cede0 2023-07-13T22:29:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.033+0000 7f607cdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780cc3b0 con 0x7f60780b8200 2023-07-13T22:29:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.033+0000 7f607cdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60780cb7a0 con 0x7f60780b1ac0 2023-07-13T22:29:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.033+0000 7f6076d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 0x7f60780cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.033+0000 7f6076d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 0x7f60780cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46360/0 (socket says 172.21.15.60:46360) 2023-07-13T22:29:52.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6076d9d700 1 -- 172.21.15.60:0/3315154034 learned_addr learned my addr 172.21.15.60:0/3315154034 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:52.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f607659c700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60780b1ac0 0x7f6078155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:52.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6075d9b700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60780b8200 0x7f6078159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:52.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6076d9d700 1 -- 172.21.15.60:0/3315154034 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60780b1ac0 msgr2=0x7f6078155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:52.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6076d9d700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60780b1ac0 0x7f6078155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6076d9d700 1 -- 172.21.15.60:0/3315154034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60780b8200 msgr2=0x7f6078159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:52.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6076d9d700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60780b8200 0x7f6078159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6076d9d700 1 -- 172.21.15.60:0/3315154034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070009b30 con 0x7f60780cede0 2023-07-13T22:29:52.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6076d9d700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 0x7f60780cc840 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f607000b680 tx=0x7f6070004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:52.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6066ffd700 1 -- 172.21.15.60:0/3315154034 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070004870 con 0x7f60780cede0 2023-07-13T22:29:52.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6066ffd700 1 -- 172.21.15.60:0/3315154034 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60700099b0 con 0x7f60780cede0 2023-07-13T22:29:52.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60780cba80 con 0x7f60780cede0 2023-07-13T22:29:52.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.034+0000 7f6066ffd700 1 -- 172.21.15.60:0/3315154034 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607001c650 con 0x7f60780cede0 2023-07-13T22:29:52.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.035+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60780cc040 con 0x7f60780cede0 2023-07-13T22:29:52.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.036+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607804e210 con 0x7f60780cede0 2023-07-13T22:29:52.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.036+0000 7f6066ffd700 1 -- 172.21.15.60:0/3315154034 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f607001c7b0 con 0x7f60780cede0 2023-07-13T22:29:52.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.037+0000 7f6066ffd700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ace0 0x7f605806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:52.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.038+0000 7f6066ffd700 1 -- 172.21.15.60:0/3315154034 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f607008aff0 con 0x7f60780cede0 2023-07-13T22:29:52.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.038+0000 7f607659c700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ace0 0x7f605806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:52.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.038+0000 7f607659c700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ace0 0x7f605806d190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6068009be0 tx=0x7f6068009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:52.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.042+0000 7f6066ffd700 1 -- 172.21.15.60:0/3315154034 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f60700560f0 con 0x7f60780cede0 2023-07-13T22:29:52.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.294+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f60780cc4f0 con 0x7f60780cede0 2023-07-13T22:29:52.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.797+0000 7f6066ffd700 1 -- 172.21.15.60:0/3315154034 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6070059710 con 0x7f60780cede0 2023-07-13T22:29:52.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ace0 msgr2=0x7f605806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:52.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ace0 0x7f605806d190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6068009be0 tx=0x7f6068009210 comp rx=0 tx=0).stop 2023-07-13T22:29:52.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 msgr2=0x7f60780cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:52.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 0x7f60780cc840 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f607000b680 tx=0x7f6070004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 shutdown_connections 2023-07-13T22:29:52.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f605806ace0 0x7f605806d190 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60780cede0 0x7f60780cc840 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60780b1ac0 0x7f6078155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 --2- 172.21.15.60:0/3315154034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60780b8200 0x7f6078159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:52.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 >> 172.21.15.60:0/3315154034 conn(0x7f607806b2f0 msgr2=0x7f60780cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:52.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 shutdown_connections 2023-07-13T22:29:52.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:52.800+0000 7f607cdff700 1 -- 172.21.15.60:0/3315154034 wait complete. 2023-07-13T22:29:52.800 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:29:52.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:53.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.028+0000 7fd3d8bff700 1 Processor -- start 2023-07-13T22:29:53.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.028+0000 7fd3d8bff700 1 -- start start 2023-07-13T22:29:53.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.030+0000 7fd3d8bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3d40b8400 0x7fd3d40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.030+0000 7fd3d8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40cfca0 0x7fd3d40d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.030+0000 7fd3d8bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d40d0ac0 con 0x7fd3d40d3750 2023-07-13T22:29:53.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.030+0000 7fd3d8bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d40b87d0 con 0x7fd3d40b8400 2023-07-13T22:29:53.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.030+0000 7fd3d8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40b4eb0 con 0x7fd3d40cfca0 2023-07-13T22:29:53.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.030+0000 7fd3d259c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3d40b8400 0x7fd3d40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.030+0000 7fd3d2d9d700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40cfca0 0x7fd3d40d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.030+0000 7fd3d259c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd3d40b8400 0x7fd3d40cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:43498/0 (socket says 172.21.15.60:43498) 2023-07-13T22:29:53.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.030+0000 7fd3d259c700 1 -- 172.21.15.60:0/1572916497 learned_addr learned my addr 172.21.15.60:0/1572916497 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:53.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.031+0000 7fd3d2d9d700 1 -- 172.21.15.60:0/1572916497 >> v2:172.21.15.105:3300/0 conn(0x7fd3d40b8400 msgr2=0x7fd3d40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:53.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.031+0000 7fd3d2d9d700 1 --2- 172.21.15.60:0/1572916497 >> v2:172.21.15.105:3300/0 conn(0x7fd3d40b8400 0x7fd3d40cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.031+0000 7fd3d2d9d700 1 -- 172.21.15.60:0/1572916497 >> v1:172.21.15.105:6789/0 conn(0x7fd3d40d3750 legacy=0x7fd3d40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.031+0000 7fd3d2d9d700 1 -- 172.21.15.60:0/1572916497 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d40b5190 con 0x7fd3d40cfca0 2023-07-13T22:29:53.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.031+0000 7fd3d1d9b700 1 -- 172.21.15.60:0/1572916497 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956209699 0 0) 0x7fd3d40d0ac0 con 0x7fd3d40d3750 2023-07-13T22:29:53.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.031+0000 7fd3d2d9d700 1 --2- 172.21.15.60:0/1572916497 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40cfca0 0x7fd3d40d0f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009d90 tx=0x7fd3cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd649be7fae03ca5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:53.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.031+0000 7fd3d1d9b700 1 -- 172.21.15.60:0/1572916497 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc00e070 con 0x7fd3d40cfca0 2023-07-13T22:29:53.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.031+0000 7fd3d1d9b700 1 -- 172.21.15.60:0/1572916497 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc00bb20 con 0x7fd3d40cfca0 2023-07-13T22:29:53.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.031+0000 7fd3d1d9b700 1 -- 172.21.15.60:0/1572916497 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc012430 con 0x7fd3d40cfca0 2023-07-13T22:29:53.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.032+0000 7fd3d8bff700 1 -- 172.21.15.60:0/1572916497 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40cfca0 msgr2=0x7fd3d40d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.032+0000 7fd3d8bff700 1 --2- 172.21.15.60:0/1572916497 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40cfca0 0x7fd3d40d0f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009d90 tx=0x7fd3cc00b330 comp rx=0 tx=0).stop 2023-07-13T22:29:53.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.032+0000 7fd3d8bff700 1 -- 172.21.15.60:0/1572916497 shutdown_connections 2023-07-13T22:29:53.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.032+0000 7fd3d8bff700 1 --2- 172.21.15.60:0/1572916497 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40cfca0 0x7fd3d40d0f50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.032+0000 7fd3d8bff700 1 --2- 172.21.15.60:0/1572916497 >> v2:172.21.15.105:3300/0 conn(0x7fd3d40b8400 0x7fd3d40cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.032+0000 7fd3d8bff700 1 -- 172.21.15.60:0/1572916497 >> 172.21.15.60:0/1572916497 conn(0x7fd3d406b390 msgr2=0x7fd3d406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:53.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.032+0000 7fd3d8bff700 1 -- 172.21.15.60:0/1572916497 shutdown_connections 2023-07-13T22:29:53.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.032+0000 7fd3d8bff700 1 -- 172.21.15.60:0/1572916497 wait complete. 2023-07-13T22:29:53.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.034+0000 7fd3d8bff700 1 Processor -- start 2023-07-13T22:29:53.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.034+0000 7fd3d8bff700 1 -- start start 2023-07-13T22:29:53.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3d8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40b8400 0x7fd3d41f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3d8bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40cfca0 0x7fd3d4156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3d8bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3d40d3750 0x7fd3d40c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3d8bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d41409d0 con 0x7fd3d40cfca0 2023-07-13T22:29:53.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3d8bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d41f2a00 con 0x7fd3d40d3750 2023-07-13T22:29:53.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3d8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40b2510 con 0x7fd3d40b8400 2023-07-13T22:29:53.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3cbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40cfca0 0x7fd3d4156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3cbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40cfca0 0x7fd3d4156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46364/0 (socket says 172.21.15.60:46364) 2023-07-13T22:29:53.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3cbfff700 1 -- 172.21.15.60:0/4177961206 learned_addr learned my addr 172.21.15.60:0/4177961206 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:53.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3d259c700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40b8400 0x7fd3d41f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.035+0000 7fd3d2d9d700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3d40d3750 0x7fd3d40c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3cbfff700 1 -- 172.21.15.60:0/4177961206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40b8400 msgr2=0x7fd3d41f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3cbfff700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40b8400 0x7fd3d41f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3cbfff700 1 -- 172.21.15.60:0/4177961206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3d40d3750 msgr2=0x7fd3d40c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3cbfff700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3d40d3750 0x7fd3d40c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3cbfff700 1 -- 172.21.15.60:0/4177961206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc009a70 con 0x7fd3d40cfca0 2023-07-13T22:29:53.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3cbfff700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40cfca0 0x7fd3d4156420 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c520 tx=0x7fd3bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:53.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3caffd700 1 -- 172.21.15.60:0/4177961206 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0054e0 con 0x7fd3d40cfca0 2023-07-13T22:29:53.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3caffd700 1 -- 172.21.15.60:0/4177961206 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc003680 con 0x7fd3d40cfca0 2023-07-13T22:29:53.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3caffd700 1 -- 172.21.15.60:0/4177961206 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc003a30 con 0x7fd3d40cfca0 2023-07-13T22:29:53.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d40b27f0 con 0x7fd3d40cfca0 2023-07-13T22:29:53.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.036+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d40b2db0 con 0x7fd3d40cfca0 2023-07-13T22:29:53.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.037+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d40b8aa0 con 0x7fd3d40cfca0 2023-07-13T22:29:53.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.038+0000 7fd3caffd700 1 -- 172.21.15.60:0/4177961206 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd3bc003b90 con 0x7fd3d40cfca0 2023-07-13T22:29:53.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.040+0000 7fd3caffd700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3b406ace0 0x7fd3b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.040+0000 7fd3caffd700 1 -- 172.21.15.60:0/4177961206 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd3bc088ab0 con 0x7fd3d40cfca0 2023-07-13T22:29:53.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.040+0000 7fd3d259c700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3b406ace0 0x7fd3b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.040+0000 7fd3d259c700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3b406ace0 0x7fd3b406d190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009ac0 tx=0x7fd3c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:53.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.044+0000 7fd3caffd700 1 -- 172.21.15.60:0/4177961206 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd3bc053ac0 con 0x7fd3d40cfca0 2023-07-13T22:29:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.297+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3d415afb0 con 0x7fd3d40cfca0 2023-07-13T22:29:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.298+0000 7fd3caffd700 1 -- 172.21.15.60:0/4177961206 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fd3bc017090 con 0x7fd3d40cfca0 2023-07-13T22:29:53.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:53.277 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1446832449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1446832449},{"type":"v1","addr":"172.21.15.105:6835","nonce":1446832449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/462201037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":462201037},{"type":"v1","addr":"172.21.15.105:6834","nonce":462201037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/154242070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":154242070},{"type":"v1","addr":"172.21.15.60:6837","nonce":154242070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:29:49.290898+0000","modified":"2023-07-13T22:29:52.790955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7831":{"gid":7831,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6836/3527792396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3527792396},{"type":"v1","addr":"172.21.15.60:6836","nonce":3527792396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-13T22:29:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3b406ace0 msgr2=0x7fd3b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3b406ace0 0x7fd3b406d190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009ac0 tx=0x7fd3c4008040 comp rx=0 tx=0).stop 2023-07-13T22:29:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40cfca0 msgr2=0x7fd3d4156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40cfca0 0x7fd3d4156420 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c520 tx=0x7fd3bc00c8e0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 shutdown_connections 2023-07-13T22:29:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3b406ace0 0x7fd3b406d190 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3d40d3750 0x7fd3d40c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3d40b8400 0x7fd3d41f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 --2- 172.21.15.60:0/4177961206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40cfca0 0x7fd3d4156420 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 >> 172.21.15.60:0/4177961206 conn(0x7fd3d406b390 msgr2=0x7fd3d415a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:53.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 shutdown_connections 2023-07-13T22:29:53.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.300+0000 7fd3d8bff700 1 -- 172.21.15.60:0/4177961206 wait complete. 2023-07-13T22:29:53.281 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 73 2023-07-13T22:29:53.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:29:53.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.521+0000 7fedc49ff700 1 Processor -- start 2023-07-13T22:29:53.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.522+0000 7fedc49ff700 1 -- start start 2023-07-13T22:29:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.523+0000 7fedc49ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fedc00d35c0 0x7fedc00d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.523+0000 7fedc49ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc00d09d0 con 0x7fedc00d13f0 2023-07-13T22:29:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.523+0000 7fedc49ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc00cda10 con 0x7fedc00b8570 2023-07-13T22:29:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.523+0000 7fedc49ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00d3b30 con 0x7fedc00d35c0 2023-07-13T22:29:53.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.523+0000 7fedbed9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fedc00d13f0 0x7fedc015c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51676/0 (socket says 172.21.15.60:51676) 2023-07-13T22:29:53.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.523+0000 7fedbed9d700 1 -- 172.21.15.60:0/3148977002 learned_addr learned my addr 172.21.15.60:0/3148977002 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:53.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.523+0000 7fedb7fff700 1 --2- 172.21.15.60:0/3148977002 >> v2:172.21.15.105:3300/0 conn(0x7fedc00d35c0 0x7fedc00d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedbdd9b700 1 -- 172.21.15.60:0/3148977002 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875256099 0 0) 0x7fedc00d09d0 con 0x7fedc00d13f0 2023-07-13T22:29:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedbdd9b700 1 -- 172.21.15.60:0/3148977002 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac0035b0 con 0x7fedc00d13f0 2023-07-13T22:29:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedbdd9b700 1 -- 172.21.15.60:0/3148977002 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218051436 0 0) 0x7fedc00cda10 con 0x7fedc00b8570 2023-07-13T22:29:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedbdd9b700 1 -- 172.21.15.60:0/3148977002 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedc00d09d0 con 0x7fedc00b8570 2023-07-13T22:29:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedb7fff700 1 -- 172.21.15.60:0/3148977002 >> v1:172.21.15.105:6789/0 conn(0x7fedc00b8570 legacy=0x7fedc00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedb7fff700 1 -- 172.21.15.60:0/3148977002 >> v1:172.21.15.60:6789/0 conn(0x7fedc00d13f0 legacy=0x7fedc015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedb7fff700 1 -- 172.21.15.60:0/3148977002 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00cd450 con 0x7fedc00d35c0 2023-07-13T22:29:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedb7fff700 1 --2- 172.21.15.60:0/3148977002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc00d35c0 0x7fedc00d0eb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7feda8009840 tx=0x7feda8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edcdd5ac91dacf6d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedbdd9b700 1 -- 172.21.15.60:0/3148977002 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8011830 con 0x7fedc00d35c0 2023-07-13T22:29:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedbdd9b700 1 -- 172.21.15.60:0/3148977002 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda8011990 con 0x7fedc00d35c0 2023-07-13T22:29:53.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.524+0000 7fedbdd9b700 1 -- 172.21.15.60:0/3148977002 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8011d20 con 0x7fedc00d35c0 2023-07-13T22:29:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.525+0000 7fedc49ff700 1 -- 172.21.15.60:0/3148977002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc00d35c0 msgr2=0x7fedc00d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.525+0000 7fedc49ff700 1 --2- 172.21.15.60:0/3148977002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc00d35c0 0x7fedc00d0eb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7feda8009840 tx=0x7feda8009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.525+0000 7fedc49ff700 1 -- 172.21.15.60:0/3148977002 shutdown_connections 2023-07-13T22:29:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.525+0000 7fedc49ff700 1 --2- 172.21.15.60:0/3148977002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc00d35c0 0x7fedc00d0eb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.525+0000 7fedc49ff700 1 -- 172.21.15.60:0/3148977002 >> 172.21.15.60:0/3148977002 conn(0x7fedc0071600 msgr2=0x7fedc00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.525+0000 7fedc49ff700 1 -- 172.21.15.60:0/3148977002 shutdown_connections 2023-07-13T22:29:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.525+0000 7fedc49ff700 1 -- 172.21.15.60:0/3148977002 wait complete. 2023-07-13T22:29:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.527+0000 7fedc49ff700 1 Processor -- start 2023-07-13T22:29:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.527+0000 7fedc49ff700 1 -- start start 2023-07-13T22:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedc49ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc00b8570 0x7fedc014c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedc49ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc00d13f0 0x7fedc014ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedc49ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc00d35c0 0x7fedc014ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedc49ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cc830 con 0x7fedc00d35c0 2023-07-13T22:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedc49ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc014cc30 con 0x7fedc00b8570 2023-07-13T22:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedc49ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc014b650 con 0x7fedc00d13f0 2023-07-13T22:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedbed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc00d35c0 0x7fedc014ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedbed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc00d35c0 0x7fedc014ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46372/0 (socket says 172.21.15.60:46372) 2023-07-13T22:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedbed9d700 1 -- 172.21.15.60:0/4062443212 learned_addr learned my addr 172.21.15.60:0/4062443212 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedb7fff700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc00d13f0 0x7fedc014ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.528+0000 7fedbe59c700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc00b8570 0x7fedc014c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedbed9d700 1 -- 172.21.15.60:0/4062443212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc00d13f0 msgr2=0x7fedc014ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedbed9d700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc00d13f0 0x7fedc014ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedbed9d700 1 -- 172.21.15.60:0/4062443212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc00b8570 msgr2=0x7fedc014c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedbed9d700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc00b8570 0x7fedc014c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedbed9d700 1 -- 172.21.15.60:0/4062443212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda80094a0 con 0x7fedc00d35c0 2023-07-13T22:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedbed9d700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc00d35c0 0x7fedc014ce90 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fedb800cba0 tx=0x7fedb800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedb6ffd700 1 -- 172.21.15.60:0/4062443212 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb800aa40 con 0x7fedc00d35c0 2023-07-13T22:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedb6ffd700 1 -- 172.21.15.60:0/4062443212 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800aba0 con 0x7fedc00d35c0 2023-07-13T22:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedb6ffd700 1 -- 172.21.15.60:0/4062443212 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8012400 con 0x7fedc00d35c0 2023-07-13T22:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc014b930 con 0x7fedc00d35c0 2023-07-13T22:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.529+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc014bef0 con 0x7fedc00d35c0 2023-07-13T22:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.531+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc0066700 con 0x7fedc00d35c0 2023-07-13T22:29:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.532+0000 7fedb6ffd700 1 -- 172.21.15.60:0/4062443212 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fedb801b3b0 con 0x7fedc00d35c0 2023-07-13T22:29:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.533+0000 7fedb6ffd700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feda006ace0 0x7feda006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.533+0000 7fedb6ffd700 1 -- 172.21.15.60:0/4062443212 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fedb8089160 con 0x7fedc00d35c0 2023-07-13T22:29:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.533+0000 7fedbe59c700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feda006ace0 0x7feda006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.534+0000 7fedbe59c700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feda006ace0 0x7feda006d190 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fedb0007fd0 tx=0x7fedb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:53.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.537+0000 7fedb6ffd700 1 -- 172.21.15.60:0/4062443212 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fedb8054210 con 0x7fedc00d35c0 2023-07-13T22:29:53.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.745+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedc015cf80 con 0x7fedc00d35c0 2023-07-13T22:29:53.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.746+0000 7fedb6ffd700 1 -- 172.21.15.60:0/4062443212 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18267 (secure 0 0 0) 0x7fedb8057830 con 0x7fedc00d35c0 2023-07-13T22:29:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:53.726 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:29:50.779149+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:29:47.410626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-13T22:29:48.264611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:29:53.726 INFO:tasks.ceph.mds.a.smithi060.stderr:2023-07-13T22:29:53.747+0000 7f94df1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:29:53.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feda006ace0 msgr2=0x7feda006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feda006ace0 0x7feda006d190 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fedb0007fd0 tx=0x7fedb0009040 comp rx=0 tx=0).stop 2023-07-13T22:29:53.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc00d35c0 msgr2=0x7fedc014ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc00d35c0 0x7fedc014ce90 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fedb800cba0 tx=0x7fedb800ceb0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 shutdown_connections 2023-07-13T22:29:53.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feda006ace0 0x7feda006d190 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedc00d35c0 0x7fedc014ce90 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedc00b8570 0x7fedc014c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 --2- 172.21.15.60:0/4062443212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedc00d13f0 0x7fedc014ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 >> 172.21.15.60:0/4062443212 conn(0x7fedc0071600 msgr2=0x7fedc00ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 shutdown_connections 2023-07-13T22:29:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.749+0000 7fedc49ff700 1 -- 172.21.15.60:0/4062443212 wait complete. 2023-07-13T22:29:53.747 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:29:53.747 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:29:53.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.973+0000 7fc5f69ff700 1 Processor -- start 2023-07-13T22:29:53.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.973+0000 7fc5f69ff700 1 -- start start 2023-07-13T22:29:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.974+0000 7fc5f69ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cefa0 0x7fc5f006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.974+0000 7fc5f69ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00cf3f0 con 0x7fc5f00b12a0 2023-07-13T22:29:53.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.974+0000 7fc5f69ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f01609a0 con 0x7fc5f00cd760 2023-07-13T22:29:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.974+0000 7fc5f69ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f006a5f0 con 0x7fc5f00cefa0 2023-07-13T22:29:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5effff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc5f00b12a0 0x7fc5f00b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51686/0 (socket says 172.21.15.60:51686) 2023-07-13T22:29:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5effff700 1 -- 172.21.15.60:0/1657610220 learned_addr learned my addr 172.21.15.60:0/1657610220 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5ef7fe700 1 --2- 172.21.15.60:0/1657610220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cefa0 0x7fc5f006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5eeffd700 1 -- 172.21.15.60:0/1657610220 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005715364 0 0) 0x7fc5f00cf3f0 con 0x7fc5f00b12a0 2023-07-13T22:29:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5eeffd700 1 -- 172.21.15.60:0/1657610220 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5d80035b0 con 0x7fc5f00b12a0 2023-07-13T22:29:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5eeffd700 1 -- 172.21.15.60:0/1657610220 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580066710 0 0) 0x7fc5f01609a0 con 0x7fc5f00cd760 2023-07-13T22:29:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5eeffd700 1 -- 172.21.15.60:0/1657610220 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f00cf3f0 con 0x7fc5f00cd760 2023-07-13T22:29:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5ef7fe700 1 -- 172.21.15.60:0/1657610220 >> v1:172.21.15.105:6789/0 conn(0x7fc5f00cd760 legacy=0x7fc5f00cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5ef7fe700 1 -- 172.21.15.60:0/1657610220 >> v1:172.21.15.60:6789/0 conn(0x7fc5f00b12a0 legacy=0x7fc5f00b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.975+0000 7fc5ef7fe700 1 -- 172.21.15.60:0/1657610220 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f015c860 con 0x7fc5f00cefa0 2023-07-13T22:29:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.976+0000 7fc5ef7fe700 1 --2- 172.21.15.60:0/1657610220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cefa0 0x7fc5f006bad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009870 tx=0x7fc5e4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f15d62bc91ae1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.976+0000 7fc5eeffd700 1 -- 172.21.15.60:0/1657610220 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4011830 con 0x7fc5f00cefa0 2023-07-13T22:29:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.976+0000 7fc5eeffd700 1 -- 172.21.15.60:0/1657610220 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e4011990 con 0x7fc5f00cefa0 2023-07-13T22:29:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.976+0000 7fc5eeffd700 1 -- 172.21.15.60:0/1657610220 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4011d20 con 0x7fc5f00cefa0 2023-07-13T22:29:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.976+0000 7fc5f69ff700 1 -- 172.21.15.60:0/1657610220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cefa0 msgr2=0x7fc5f006bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.976+0000 7fc5f69ff700 1 --2- 172.21.15.60:0/1657610220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cefa0 0x7fc5f006bad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009870 tx=0x7fc5e4009b80 comp rx=0 tx=0).stop 2023-07-13T22:29:53.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.976+0000 7fc5f69ff700 1 -- 172.21.15.60:0/1657610220 shutdown_connections 2023-07-13T22:29:53.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.976+0000 7fc5f69ff700 1 --2- 172.21.15.60:0/1657610220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cefa0 0x7fc5f006bad0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.976+0000 7fc5f69ff700 1 -- 172.21.15.60:0/1657610220 >> 172.21.15.60:0/1657610220 conn(0x7fc5f00cc2a0 msgr2=0x7fc5f00cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:53.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.977+0000 7fc5f69ff700 1 -- 172.21.15.60:0/1657610220 shutdown_connections 2023-07-13T22:29:53.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.977+0000 7fc5f69ff700 1 -- 172.21.15.60:0/1657610220 wait complete. 2023-07-13T22:29:53.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.978+0000 7fc5f69ff700 1 Processor -- start 2023-07-13T22:29:53.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.978+0000 7fc5f69ff700 1 -- start start 2023-07-13T22:29:53.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.979+0000 7fc5f69ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f00b12a0 0x7fc5f00c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.979+0000 7fc5f69ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cd760 0x7fc5f015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.979+0000 7fc5f69ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f00cefa0 0x7fc5f00c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.979+0000 7fc5f69ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f015bb60 con 0x7fc5f00b12a0 2023-07-13T22:29:53.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.979+0000 7fc5f69ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00c68f0 con 0x7fc5f00cefa0 2023-07-13T22:29:53.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.979+0000 7fc5f69ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00c6bd0 con 0x7fc5f00cd760 2023-07-13T22:29:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5effff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f00b12a0 0x7fc5f00c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5effff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f00b12a0 0x7fc5f00c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46374/0 (socket says 172.21.15.60:46374) 2023-07-13T22:29:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5effff700 1 -- 172.21.15.60:0/2340894508 learned_addr learned my addr 172.21.15.60:0/2340894508 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5f4f9c700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f00cefa0 0x7fc5f00c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5ef7fe700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cd760 0x7fc5f015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5effff700 1 -- 172.21.15.60:0/2340894508 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cd760 msgr2=0x7fc5f015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5effff700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cd760 0x7fc5f015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5effff700 1 -- 172.21.15.60:0/2340894508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f00cefa0 msgr2=0x7fc5f00c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:53.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5effff700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f00cefa0 0x7fc5f00c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:53.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5effff700 1 -- 172.21.15.60:0/2340894508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40094a0 con 0x7fc5f00b12a0 2023-07-13T22:29:53.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.980+0000 7fc5effff700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f00b12a0 0x7fc5f00c8ec0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000c700 tx=0x7fc5e000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:53.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.981+0000 7fc5ecff9700 1 -- 172.21.15.60:0/2340894508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0005d70 con 0x7fc5f00b12a0 2023-07-13T22:29:53.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.981+0000 7fc5ecff9700 1 -- 172.21.15.60:0/2340894508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e00070d0 con 0x7fc5f00b12a0 2023-07-13T22:29:53.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.981+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f00c6eb0 con 0x7fc5f00b12a0 2023-07-13T22:29:53.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.981+0000 7fc5ecff9700 1 -- 172.21.15.60:0/2340894508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0007460 con 0x7fc5f00b12a0 2023-07-13T22:29:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.981+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f00c7470 con 0x7fc5f00b12a0 2023-07-13T22:29:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.982+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f00c7ac0 con 0x7fc5f00b12a0 2023-07-13T22:29:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.983+0000 7fc5ecff9700 1 -- 172.21.15.60:0/2340894508 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc5e00075c0 con 0x7fc5f00b12a0 2023-07-13T22:29:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.984+0000 7fc5ecff9700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5dc06ace0 0x7fc5dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.984+0000 7fc5ecff9700 1 -- 172.21.15.60:0/2340894508 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5e0088af0 con 0x7fc5f00b12a0 2023-07-13T22:29:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.984+0000 7fc5ef7fe700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5dc06ace0 0x7fc5dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:53.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.985+0000 7fc5ef7fe700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5dc06ace0 0x7fc5dc06d190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40097c0 tx=0x7fc5e401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:53.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:53.989+0000 7fc5ecff9700 1 -- 172.21.15.60:0/2340894508 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc5e0053b10 con 0x7fc5f00b12a0 2023-07-13T22:29:54.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.190+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5f006a070 con 0x7fc5dc06ace0 2023-07-13T22:29:54.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.200+0000 7fc5ecff9700 1 -- 172.21.15.60:0/2340894508 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311757 (secure 0 0 0) 0x7fc5e4011b00 con 0x7fc5dc06ace0 2023-07-13T22:29:54.182 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:54.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5dc06ace0 msgr2=0x7fc5dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5dc06ace0 0x7fc5dc06d190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40097c0 tx=0x7fc5e401c040 comp rx=0 tx=0).stop 2023-07-13T22:29:54.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f00b12a0 msgr2=0x7fc5f00c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f00b12a0 0x7fc5f00c8ec0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000c700 tx=0x7fc5e000cac0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 shutdown_connections 2023-07-13T22:29:54.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc5dc06ace0 0x7fc5dc06d190 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f00b12a0 0x7fc5f00c8ec0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f00cefa0 0x7fc5f00c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 --2- 172.21.15.60:0/2340894508 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5f00cd760 0x7fc5f015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 >> 172.21.15.60:0/2340894508 conn(0x7fc5f00cc2a0 msgr2=0x7fc5f00b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:54.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 shutdown_connections 2023-07-13T22:29:54.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.204+0000 7fc5f69ff700 1 -- 172.21.15.60:0/2340894508 wait complete. 2023-07-13T22:29:54.186 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:29:54.201 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2023-07-13T22:29:53.243864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284984,"kb_used_data":12024,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754689736,"statfs":{"total":773094113280,"available":772802289664,"internally_reserved":0,"allocated":12312576,"data_stored":6238522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.488902"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789386+0000","last_change":"2023-07-13T22:29:49.789346+0000","last_active":"2023-07-13T22:29:50.789386+0000","last_peered":"2023-07-13T22:29:50.789386+0000","last_clean":"2023-07-13T22:29:50.789386+0000","last_became_active":"2023-07-13T22:29:49.788901+0000","last_became_peered":"2023-07-13T22:29:49.788901+0000","last_unstale":"2023-07-13T22:29:50.789386+0000","last_undegraded":"2023-07-13T22:29:50.789386+0000","last_fullsized":"2023-07-13T22:29:50.789386+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:39:01.446319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786738+0000","last_change":"2023-07-13T22:29:49.793387+0000","last_active":"2023-07-13T22:29:50.786738+0000","last_peered":"2023-07-13T22:29:50.786738+0000","last_clean":"2023-07-13T22:29:50.786738+0000","last_became_active":"2023-07-13T22:29:49.792792+0000","last_became_peered":"2023-07-13T22:29:49.792792+0000","last_unstale":"2023-07-13T22:29:50.786738+0000","last_undegraded":"2023-07-13T22:29:50.786738+0000","last_fullsized":"2023-07-13T22:29:50.786738+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:42:51.567608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786458+0000","last_change":"2023-07-13T22:29:49.799582+0000","last_active":"2023-07-13T22:29:50.786458+0000","last_peered":"2023-07-13T22:29:50.786458+0000","last_clean":"2023-07-13T22:29:50.786458+0000","last_became_active":"2023-07-13T22:29:49.799082+0000","last_became_peered":"2023-07-13T22:29:49.799082+0000","last_unstale":"2023-07-13T22:29:50.786458+0000","last_undegraded":"2023-07-13T22:29:50.786458+0000","last_fullsized":"2023-07-13T22:29:50.786458+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:34:16.133061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.798712+0000","last_change":"2023-07-13T22:29:49.794344+0000","last_active":"2023-07-13T22:29:49.798712+0000","last_peered":"2023-07-13T22:29:49.798712+0000","last_clean":"2023-07-13T22:29:49.798712+0000","last_became_active":"2023-07-13T22:29:49.794092+0000","last_became_peered":"2023-07-13T22:29:49.794092+0000","last_unstale":"2023-07-13T22:29:49.798712+0000","last_undegraded":"2023-07-13T22:29:49.798712+0000","last_fullsized":"2023-07-13T22:29:49.798712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:15:09.575950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788781+0000","last_change":"2023-07-13T22:29:49.796389+0000","last_active":"2023-07-13T22:29:50.788781+0000","last_peered":"2023-07-13T22:29:50.788781+0000","last_clean":"2023-07-13T22:29:50.788781+0000","last_became_active":"2023-07-13T22:29:49.796086+0000","last_became_peered":"2023-07-13T22:29:49.796086+0000","last_unstale":"2023-07-13T22:29:50.788781+0000","last_undegraded":"2023-07-13T22:29:50.788781+0000","last_fullsized":"2023-07-13T22:29:50.788781+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:20:44.967439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.810106+0000","last_change":"2023-07-13T22:29:49.799550+0000","last_active":"2023-07-13T22:29:49.810106+0000","last_peered":"2023-07-13T22:29:49.810106+0000","last_clean":"2023-07-13T22:29:49.810106+0000","last_became_active":"2023-07-13T22:29:49.799218+0000","last_became_peered":"2023-07-13T22:29:49.799218+0000","last_unstale":"2023-07-13T22:29:49.810106+0000","last_undegraded":"2023-07-13T22:29:49.810106+0000","last_fullsized":"2023-07-13T22:29:49.810106+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:00:50.825775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.809593+0000","last_change":"2023-07-13T22:29:49.798054+0000","last_active":"2023-07-13T22:29:49.809593+0000","last_peered":"2023-07-13T22:29:49.809593+0000","last_clean":"2023-07-13T22:29:49.809593+0000","last_became_active":"2023-07-13T22:29:49.797759+0000","last_became_peered":"2023-07-13T22:29:49.797759+0000","last_unstale":"2023-07-13T22:29:49.809593+0000","last_undegraded":"2023-07-13T22:29:49.809593+0000","last_fullsized":"2023-07-13T22:29:49.809593+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:42:28.121026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786048+0000","last_change":"2023-07-13T22:29:49.798300+0000","last_active":"2023-07-13T22:29:50.786048+0000","last_peered":"2023-07-13T22:29:50.786048+0000","last_clean":"2023-07-13T22:29:50.786048+0000","last_became_active":"2023-07-13T22:29:49.797986+0000","last_became_peered":"2023-07-13T22:29:49.797986+0000","last_unstale":"2023-07-13T22:29:50.786048+0000","last_undegraded":"2023-07-13T22:29:50.786048+0000","last_fullsized":"2023-07-13T22:29:50.786048+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:05:53.318240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.800005+0000","last_change":"2023-07-13T22:29:49.797454+0000","last_active":"2023-07-13T22:29:49.800005+0000","last_peered":"2023-07-13T22:29:49.800005+0000","last_clean":"2023-07-13T22:29:49.800005+0000","last_became_active":"2023-07-13T22:29:49.797179+0000","last_became_peered":"2023-07-13T22:29:49.797179+0000","last_unstale":"2023-07-13T22:29:49.800005+0000","last_undegraded":"2023-07-13T22:29:49.800005+0000","last_fullsized":"2023-07-13T22:29:49.800005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:47:40.827548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.795924+0000","last_change":"2023-07-13T22:29:49.791970+0000","last_active":"2023-07-13T22:29:49.795924+0000","last_peered":"2023-07-13T22:29:49.795924+0000","last_clean":"2023-07-13T22:29:49.795924+0000","last_became_active":"2023-07-13T22:29:49.791593+0000","last_became_peered":"2023-07-13T22:29:49.791593+0000","last_unstale":"2023-07-13T22:29:49.795924+0000","last_undegraded":"2023-07-13T22:29:49.795924+0000","last_fullsized":"2023-07-13T22:29:49.795924+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:05:57.218132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789054+0000","last_change":"2023-07-13T22:29:49.791595+0000","last_active":"2023-07-13T22:29:50.789054+0000","last_peered":"2023-07-13T22:29:50.789054+0000","last_clean":"2023-07-13T22:29:50.789054+0000","last_became_active":"2023-07-13T22:29:49.791326+0000","last_became_peered":"2023-07-13T22:29:49.791326+0000","last_unstale":"2023-07-13T22:29:50.789054+0000","last_undegraded":"2023-07-13T22:29:50.789054+0000","last_fullsized":"2023-07-13T22:29:50.789054+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:18:14.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807528+0000","last_change":"2023-07-13T22:29:49.795077+0000","last_active":"2023-07-13T22:29:49.807528+0000","last_peered":"2023-07-13T22:29:49.807528+0000","last_clean":"2023-07-13T22:29:49.807528+0000","last_became_active":"2023-07-13T22:29:49.794731+0000","last_became_peered":"2023-07-13T22:29:49.794731+0000","last_unstale":"2023-07-13T22:29:49.807528+0000","last_undegraded":"2023-07-13T22:29:49.807528+0000","last_fullsized":"2023-07-13T22:29:49.807528+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:40:14.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.791850+0000","last_change":"2023-07-13T22:29:49.790654+0000","last_active":"2023-07-13T22:29:49.791850+0000","last_peered":"2023-07-13T22:29:49.791850+0000","last_clean":"2023-07-13T22:29:49.791850+0000","last_became_active":"2023-07-13T22:29:49.790356+0000","last_became_peered":"2023-07-13T22:29:49.790356+0000","last_unstale":"2023-07-13T22:29:49.791850+0000","last_undegraded":"2023-07-13T22:29:49.791850+0000","last_fullsized":"2023-07-13T22:29:49.791850+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:24:50.307758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786670+0000","last_change":"2023-07-13T22:29:49.797809+0000","last_active":"2023-07-13T22:29:50.786670+0000","last_peered":"2023-07-13T22:29:50.786670+0000","last_clean":"2023-07-13T22:29:50.786670+0000","last_became_active":"2023-07-13T22:29:49.797531+0000","last_became_peered":"2023-07-13T22:29:49.797531+0000","last_unstale":"2023-07-13T22:29:50.786670+0000","last_undegraded":"2023-07-13T22:29:50.786670+0000","last_fullsized":"2023-07-13T22:29:50.786670+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:00.845441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788750+0000","last_change":"2023-07-13T22:29:49.793635+0000","last_active":"2023-07-13T22:29:50.788750+0000","last_peered":"2023-07-13T22:29:50.788750+0000","last_clean":"2023-07-13T22:29:50.788750+0000","last_became_active":"2023-07-13T22:29:49.791983+0000","last_became_peered":"2023-07-13T22:29:49.791983+0000","last_unstale":"2023-07-13T22:29:50.788750+0000","last_undegraded":"2023-07-13T22:29:50.788750+0000","last_fullsized":"2023-07-13T22:29:50.788750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:39:12.348184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807680+0000","last_change":"2023-07-13T22:29:49.793986+0000","last_active":"2023-07-13T22:29:49.807680+0000","last_peered":"2023-07-13T22:29:49.807680+0000","last_clean":"2023-07-13T22:29:49.807680+0000","last_became_active":"2023-07-13T22:29:49.793625+0000","last_became_peered":"2023-07-13T22:29:49.793625+0000","last_unstale":"2023-07-13T22:29:49.807680+0000","last_undegraded":"2023-07-13T22:29:49.807680+0000","last_fullsized":"2023-07-13T22:29:49.807680+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:27:13.192607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.792887+0000","last_change":"2023-07-13T22:29:49.783979+0000","last_active":"2023-07-13T22:29:49.792887+0000","last_peered":"2023-07-13T22:29:49.792887+0000","last_clean":"2023-07-13T22:29:49.792887+0000","last_became_active":"2023-07-13T22:29:49.783653+0000","last_became_peered":"2023-07-13T22:29:49.783653+0000","last_unstale":"2023-07-13T22:29:49.792887+0000","last_undegraded":"2023-07-13T22:29:49.792887+0000","last_fullsized":"2023-07-13T22:29:49.792887+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:24:22.771277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786493+0000","last_change":"2023-07-13T22:29:49.792724+0000","last_active":"2023-07-13T22:29:50.786493+0000","last_peered":"2023-07-13T22:29:50.786493+0000","last_clean":"2023-07-13T22:29:50.786493+0000","last_became_active":"2023-07-13T22:29:49.792440+0000","last_became_peered":"2023-07-13T22:29:49.792440+0000","last_unstale":"2023-07-13T22:29:50.786493+0000","last_undegraded":"2023-07-13T22:29:50.786493+0000","last_fullsized":"2023-07-13T22:29:50.786493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:26:34.668070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.802665+0000","last_change":"2023-07-13T22:29:49.801142+0000","last_active":"2023-07-13T22:29:49.802665+0000","last_peered":"2023-07-13T22:29:49.802665+0000","last_clean":"2023-07-13T22:29:49.802665+0000","last_became_active":"2023-07-13T22:29:49.800623+0000","last_became_peered":"2023-07-13T22:29:49.800623+0000","last_unstale":"2023-07-13T22:29:49.802665+0000","last_undegraded":"2023-07-13T22:29:49.802665+0000","last_fullsized":"2023-07-13T22:29:49.802665+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:01:04.284945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063111+0000","last_change":"2023-07-13T22:29:49.789645+0000","last_active":"2023-07-13T22:29:51.063111+0000","last_peered":"2023-07-13T22:29:51.063111+0000","last_clean":"2023-07-13T22:29:51.063111+0000","last_became_active":"2023-07-13T22:29:49.789356+0000","last_became_peered":"2023-07-13T22:29:49.789356+0000","last_unstale":"2023-07-13T22:29:51.063111+0000","last_undegraded":"2023-07-13T22:29:51.063111+0000","last_fullsized":"2023-07-13T22:29:51.063111+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:59:41.514637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.803880+0000","last_change":"2023-07-13T22:29:49.800037+0000","last_active":"2023-07-13T22:29:49.803880+0000","last_peered":"2023-07-13T22:29:49.803880+0000","last_clean":"2023-07-13T22:29:49.803880+0000","last_became_active":"2023-07-13T22:29:49.799669+0000","last_became_peered":"2023-07-13T22:29:49.799669+0000","last_unstale":"2023-07-13T22:29:49.803880+0000","last_undegraded":"2023-07-13T22:29:49.803880+0000","last_fullsized":"2023-07-13T22:29:49.803880+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:36:20.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789042+0000","last_change":"2023-07-13T22:29:49.798221+0000","last_active":"2023-07-13T22:29:50.789042+0000","last_peered":"2023-07-13T22:29:50.789042+0000","last_clean":"2023-07-13T22:29:50.789042+0000","last_became_active":"2023-07-13T22:29:49.797861+0000","last_became_peered":"2023-07-13T22:29:49.797861+0000","last_unstale":"2023-07-13T22:29:50.789042+0000","last_undegraded":"2023-07-13T22:29:50.789042+0000","last_fullsized":"2023-07-13T22:29:50.789042+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:51.867950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.799868+0000","last_change":"2023-07-13T22:29:49.797379+0000","last_active":"2023-07-13T22:29:49.799868+0000","last_peered":"2023-07-13T22:29:49.799868+0000","last_clean":"2023-07-13T22:29:49.799868+0000","last_became_active":"2023-07-13T22:29:49.797086+0000","last_became_peered":"2023-07-13T22:29:49.797086+0000","last_unstale":"2023-07-13T22:29:49.799868+0000","last_undegraded":"2023-07-13T22:29:49.799868+0000","last_fullsized":"2023-07-13T22:29:49.799868+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:31.428060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787578+0000","last_change":"2023-07-13T22:29:49.793936+0000","last_active":"2023-07-13T22:29:50.787578+0000","last_peered":"2023-07-13T22:29:50.787578+0000","last_clean":"2023-07-13T22:29:50.787578+0000","last_became_active":"2023-07-13T22:29:49.793475+0000","last_became_peered":"2023-07-13T22:29:49.793475+0000","last_unstale":"2023-07-13T22:29:50.787578+0000","last_undegraded":"2023-07-13T22:29:50.787578+0000","last_fullsized":"2023-07-13T22:29:50.787578+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:02:58.850794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788184+0000","last_change":"2023-07-13T22:29:49.790916+0000","last_active":"2023-07-13T22:29:50.788184+0000","last_peered":"2023-07-13T22:29:50.788184+0000","last_clean":"2023-07-13T22:29:50.788184+0000","last_became_active":"2023-07-13T22:29:49.790631+0000","last_became_peered":"2023-07-13T22:29:49.790631+0000","last_unstale":"2023-07-13T22:29:50.788184+0000","last_undegraded":"2023-07-13T22:29:50.788184+0000","last_fullsized":"2023-07-13T22:29:50.788184+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:01:12.208248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063320+0000","last_change":"2023-07-13T22:29:49.784578+0000","last_active":"2023-07-13T22:29:51.063320+0000","last_peered":"2023-07-13T22:29:51.063320+0000","last_clean":"2023-07-13T22:29:51.063320+0000","last_became_active":"2023-07-13T22:29:49.784266+0000","last_became_peered":"2023-07-13T22:29:49.784266+0000","last_unstale":"2023-07-13T22:29:51.063320+0000","last_undegraded":"2023-07-13T22:29:51.063320+0000","last_fullsized":"2023-07-13T22:29:51.063320+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:55:11.376144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.802647+0000","last_change":"2023-07-13T22:29:49.800947+0000","last_active":"2023-07-13T22:29:49.802647+0000","last_peered":"2023-07-13T22:29:49.802647+0000","last_clean":"2023-07-13T22:29:49.802647+0000","last_became_active":"2023-07-13T22:29:49.800283+0000","last_became_peered":"2023-07-13T22:29:49.800283+0000","last_unstale":"2023-07-13T22:29:49.802647+0000","last_undegraded":"2023-07-13T22:29:49.802647+0000","last_fullsized":"2023-07-13T22:29:49.802647+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:08:25.577957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807830+0000","last_change":"2023-07-13T22:29:49.804148+0000","last_active":"2023-07-13T22:29:49.807830+0000","last_peered":"2023-07-13T22:29:49.807830+0000","last_clean":"2023-07-13T22:29:49.807830+0000","last_became_active":"2023-07-13T22:29:49.803790+0000","last_became_peered":"2023-07-13T22:29:49.803790+0000","last_unstale":"2023-07-13T22:29:49.807830+0000","last_undegraded":"2023-07-13T22:29:49.807830+0000","last_fullsized":"2023-07-13T22:29:49.807830+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:33.678966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063215+0000","last_change":"2023-07-13T22:29:49.779930+0000","last_active":"2023-07-13T22:29:51.063215+0000","last_peered":"2023-07-13T22:29:51.063215+0000","last_clean":"2023-07-13T22:29:51.063215+0000","last_became_active":"2023-07-13T22:29:49.779646+0000","last_became_peered":"2023-07-13T22:29:49.779646+0000","last_unstale":"2023-07-13T22:29:51.063215+0000","last_undegraded":"2023-07-13T22:29:51.063215+0000","last_fullsized":"2023-07-13T22:29:51.063215+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:28:52.629288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786543+0000","last_change":"2023-07-13T22:29:49.794111+0000","last_active":"2023-07-13T22:29:50.786543+0000","last_peered":"2023-07-13T22:29:50.786543+0000","last_clean":"2023-07-13T22:29:50.786543+0000","last_became_active":"2023-07-13T22:29:49.793713+0000","last_became_peered":"2023-07-13T22:29:49.793713+0000","last_unstale":"2023-07-13T22:29:50.786543+0000","last_undegraded":"2023-07-13T22:29:50.786543+0000","last_fullsized":"2023-07-13T22:29:50.786543+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:04:28.374231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787655+0000","last_change":"2023-07-13T22:29:49.782946+0000","last_active":"2023-07-13T22:29:50.787655+0000","last_peered":"2023-07-13T22:29:50.787655+0000","last_clean":"2023-07-13T22:29:50.787655+0000","last_became_active":"2023-07-13T22:29:49.782054+0000","last_became_peered":"2023-07-13T22:29:49.782054+0000","last_unstale":"2023-07-13T22:29:50.787655+0000","last_undegraded":"2023-07-13T22:29:50.787655+0000","last_fullsized":"2023-07-13T22:29:50.787655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:54:45.309189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788013+0000","last_change":"2023-07-13T22:29:49.793767+0000","last_active":"2023-07-13T22:29:50.788013+0000","last_peered":"2023-07-13T22:29:50.788013+0000","last_clean":"2023-07-13T22:29:50.788013+0000","last_became_active":"2023-07-13T22:29:49.793084+0000","last_became_peered":"2023-07-13T22:29:49.793084+0000","last_unstale":"2023-07-13T22:29:50.788013+0000","last_undegraded":"2023-07-13T22:29:50.788013+0000","last_fullsized":"2023-07-13T22:29:50.788013+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:17.182760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.803308+0000","last_change":"2023-07-13T22:29:49.800716+0000","last_active":"2023-07-13T22:29:49.803308+0000","last_peered":"2023-07-13T22:29:49.803308+0000","last_clean":"2023-07-13T22:29:49.803308+0000","last_became_active":"2023-07-13T22:29:49.800017+0000","last_became_peered":"2023-07-13T22:29:49.800017+0000","last_unstale":"2023-07-13T22:29:49.803308+0000","last_undegraded":"2023-07-13T22:29:49.803308+0000","last_fullsized":"2023-07-13T22:29:49.803308+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:10:13.698513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"111'4","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.809373+0000","last_change":"2023-07-13T22:29:48.779936+0000","last_active":"2023-07-13T22:29:49.809373+0000","last_peered":"2023-07-13T22:29:49.809373+0000","last_clean":"2023-07-13T22:29:49.809373+0000","last_became_active":"2023-07-13T22:29:48.779621+0000","last_became_peered":"2023-07-13T22:29:48.779621+0000","last_unstale":"2023-07-13T22:29:49.809373+0000","last_undegraded":"2023-07-13T22:29:49.809373+0000","last_fullsized":"2023-07-13T22:29:49.809373+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:14:44.131397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.808583+0000","last_change":"2023-07-13T22:29:49.797115+0000","last_active":"2023-07-13T22:29:49.808583+0000","last_peered":"2023-07-13T22:29:49.808583+0000","last_clean":"2023-07-13T22:29:49.808583+0000","last_became_active":"2023-07-13T22:29:49.796795+0000","last_became_peered":"2023-07-13T22:29:49.796795+0000","last_unstale":"2023-07-13T22:29:49.808583+0000","last_undegraded":"2023-07-13T22:29:49.808583+0000","last_fullsized":"2023-07-13T22:29:49.808583+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:56:23.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.806882+0000","last_change":"2023-07-13T22:29:48.773927+0000","last_active":"2023-07-13T22:29:49.806882+0000","last_peered":"2023-07-13T22:29:49.806882+0000","last_clean":"2023-07-13T22:29:49.806882+0000","last_became_active":"2023-07-13T22:29:48.773628+0000","last_became_peered":"2023-07-13T22:29:48.773628+0000","last_unstale":"2023-07-13T22:29:49.806882+0000","last_undegraded":"2023-07-13T22:29:49.806882+0000","last_fullsized":"2023-07-13T22:29:49.806882+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:03:50.291548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787799+0000","last_change":"2023-07-13T22:29:49.789313+0000","last_active":"2023-07-13T22:29:50.787799+0000","last_peered":"2023-07-13T22:29:50.787799+0000","last_clean":"2023-07-13T22:29:50.787799+0000","last_became_active":"2023-07-13T22:29:49.789003+0000","last_became_peered":"2023-07-13T22:29:49.789003+0000","last_unstale":"2023-07-13T22:29:50.787799+0000","last_undegraded":"2023-07-13T22:29:50.787799+0000","last_fullsized":"2023-07-13T22:29:50.787799+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:27:19.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.804339+0000","last_change":"2023-07-13T22:29:48.773431+0000","last_active":"2023-07-13T22:29:49.804339+0000","last_peered":"2023-07-13T22:29:49.804339+0000","last_clean":"2023-07-13T22:29:49.804339+0000","last_became_active":"2023-07-13T22:29:48.773088+0000","last_became_peered":"2023-07-13T22:29:48.773088+0000","last_unstale":"2023-07-13T22:29:49.804339+0000","last_undegraded":"2023-07-13T22:29:49.804339+0000","last_fullsized":"2023-07-13T22:29:49.804339+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:38:41.991340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.803850+0000","last_change":"2023-07-13T22:29:49.799902+0000","last_active":"2023-07-13T22:29:49.803850+0000","last_peered":"2023-07-13T22:29:49.803850+0000","last_clean":"2023-07-13T22:29:49.803850+0000","last_became_active":"2023-07-13T22:29:49.799607+0000","last_became_peered":"2023-07-13T22:29:49.799607+0000","last_unstale":"2023-07-13T22:29:49.803850+0000","last_undegraded":"2023-07-13T22:29:49.803850+0000","last_fullsized":"2023-07-13T22:29:49.803850+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:48:37.285989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807115+0000","last_change":"2023-07-13T22:29:48.781172+0000","last_active":"2023-07-13T22:29:49.807115+0000","last_peered":"2023-07-13T22:29:49.807115+0000","last_clean":"2023-07-13T22:29:49.807115+0000","last_became_active":"2023-07-13T22:29:48.780720+0000","last_became_peered":"2023-07-13T22:29:48.780720+0000","last_unstale":"2023-07-13T22:29:49.807115+0000","last_undegraded":"2023-07-13T22:29:49.807115+0000","last_fullsized":"2023-07-13T22:29:49.807115+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:46:33.540025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.798036+0000","last_change":"2023-07-13T22:29:49.796710+0000","last_active":"2023-07-13T22:29:49.798036+0000","last_peered":"2023-07-13T22:29:49.798036+0000","last_clean":"2023-07-13T22:29:49.798036+0000","last_became_active":"2023-07-13T22:29:49.796392+0000","last_became_peered":"2023-07-13T22:29:49.796392+0000","last_unstale":"2023-07-13T22:29:49.798036+0000","last_undegraded":"2023-07-13T22:29:49.798036+0000","last_fullsized":"2023-07-13T22:29:49.798036+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:19:25.312547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788951+0000","last_change":"2023-07-13T22:29:48.771882+0000","last_active":"2023-07-13T22:29:50.788951+0000","last_peered":"2023-07-13T22:29:50.788951+0000","last_clean":"2023-07-13T22:29:50.788951+0000","last_became_active":"2023-07-13T22:29:48.771576+0000","last_became_peered":"2023-07-13T22:29:48.771576+0000","last_unstale":"2023-07-13T22:29:50.788951+0000","last_undegraded":"2023-07-13T22:29:50.788951+0000","last_fullsized":"2023-07-13T22:29:50.788951+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:20:42.770389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788671+0000","last_change":"2023-07-13T22:29:49.796609+0000","last_active":"2023-07-13T22:29:50.788671+0000","last_peered":"2023-07-13T22:29:50.788671+0000","last_clean":"2023-07-13T22:29:50.788671+0000","last_became_active":"2023-07-13T22:29:49.796305+0000","last_became_peered":"2023-07-13T22:29:49.796305+0000","last_unstale":"2023-07-13T22:29:50.788671+0000","last_undegraded":"2023-07-13T22:29:50.788671+0000","last_fullsized":"2023-07-13T22:29:50.788671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:33.678966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.810161+0000","last_change":"2023-07-13T22:29:48.782595+0000","last_active":"2023-07-13T22:29:49.810161+0000","last_peered":"2023-07-13T22:29:49.810161+0000","last_clean":"2023-07-13T22:29:49.810161+0000","last_became_active":"2023-07-13T22:29:48.782183+0000","last_became_peered":"2023-07-13T22:29:48.782183+0000","last_unstale":"2023-07-13T22:29:49.810161+0000","last_undegraded":"2023-07-13T22:29:49.810161+0000","last_fullsized":"2023-07-13T22:29:49.810161+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:41:14.764896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787011+0000","last_change":"2023-07-13T22:29:49.786046+0000","last_active":"2023-07-13T22:29:50.787011+0000","last_peered":"2023-07-13T22:29:50.787011+0000","last_clean":"2023-07-13T22:29:50.787011+0000","last_became_active":"2023-07-13T22:29:49.785718+0000","last_became_peered":"2023-07-13T22:29:49.785718+0000","last_unstale":"2023-07-13T22:29:50.787011+0000","last_undegraded":"2023-07-13T22:29:50.787011+0000","last_fullsized":"2023-07-13T22:29:50.787011+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:47.408965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787822+0000","last_change":"2023-07-13T22:29:48.778809+0000","last_active":"2023-07-13T22:29:50.787822+0000","last_peered":"2023-07-13T22:29:50.787822+0000","last_clean":"2023-07-13T22:29:50.787822+0000","last_became_active":"2023-07-13T22:29:48.778515+0000","last_became_peered":"2023-07-13T22:29:48.778515+0000","last_unstale":"2023-07-13T22:29:50.787822+0000","last_undegraded":"2023-07-13T22:29:50.787822+0000","last_fullsized":"2023-07-13T22:29:50.787822+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:55:46.332348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.790764+0000","last_change":"2023-07-13T22:29:49.784589+0000","last_active":"2023-07-13T22:29:49.790764+0000","last_peered":"2023-07-13T22:29:49.790764+0000","last_clean":"2023-07-13T22:29:49.790764+0000","last_became_active":"2023-07-13T22:29:49.784262+0000","last_became_peered":"2023-07-13T22:29:49.784262+0000","last_unstale":"2023-07-13T22:29:49.790764+0000","last_undegraded":"2023-07-13T22:29:49.790764+0000","last_fullsized":"2023-07-13T22:29:49.790764+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:33:43.779422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.781447+0000","last_change":"2023-07-13T22:29:48.783299+0000","last_active":"2023-07-13T22:29:49.781447+0000","last_peered":"2023-07-13T22:29:49.781447+0000","last_clean":"2023-07-13T22:29:49.781447+0000","last_became_active":"2023-07-13T22:29:48.782979+0000","last_became_peered":"2023-07-13T22:29:48.782979+0000","last_unstale":"2023-07-13T22:29:49.781447+0000","last_undegraded":"2023-07-13T22:29:49.781447+0000","last_fullsized":"2023-07-13T22:29:49.781447+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:38:16.683065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787729+0000","last_change":"2023-07-13T22:29:49.795458+0000","last_active":"2023-07-13T22:29:50.787729+0000","last_peered":"2023-07-13T22:29:50.787729+0000","last_clean":"2023-07-13T22:29:50.787729+0000","last_became_active":"2023-07-13T22:29:49.795163+0000","last_became_peered":"2023-07-13T22:29:49.795163+0000","last_unstale":"2023-07-13T22:29:50.787729+0000","last_undegraded":"2023-07-13T22:29:50.787729+0000","last_fullsized":"2023-07-13T22:29:50.787729+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:17:55.654447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.811851+0000","last_change":"2023-07-13T22:29:48.775741+0000","last_active":"2023-07-13T22:29:49.811851+0000","last_peered":"2023-07-13T22:29:49.811851+0000","last_clean":"2023-07-13T22:29:49.811851+0000","last_became_active":"2023-07-13T22:29:48.775439+0000","last_became_peered":"2023-07-13T22:29:48.775439+0000","last_unstale":"2023-07-13T22:29:49.811851+0000","last_undegraded":"2023-07-13T22:29:49.811851+0000","last_fullsized":"2023-07-13T22:29:49.811851+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:12:58.805471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789363+0000","last_change":"2023-07-13T22:29:49.797753+0000","last_active":"2023-07-13T22:29:50.789363+0000","last_peered":"2023-07-13T22:29:50.789363+0000","last_clean":"2023-07-13T22:29:50.789363+0000","last_became_active":"2023-07-13T22:29:49.797466+0000","last_became_peered":"2023-07-13T22:29:49.797466+0000","last_unstale":"2023-07-13T22:29:50.789363+0000","last_undegraded":"2023-07-13T22:29:50.789363+0000","last_fullsized":"2023-07-13T22:29:50.789363+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:07:01.435082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063347+0000","last_change":"2023-07-13T22:29:48.777146+0000","last_active":"2023-07-13T22:29:51.063347+0000","last_peered":"2023-07-13T22:29:51.063347+0000","last_clean":"2023-07-13T22:29:51.063347+0000","last_became_active":"2023-07-13T22:29:48.776885+0000","last_became_peered":"2023-07-13T22:29:48.776885+0000","last_unstale":"2023-07-13T22:29:51.063347+0000","last_undegraded":"2023-07-13T22:29:51.063347+0000","last_fullsized":"2023-07-13T22:29:51.063347+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:17:02.197580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807750+0000","last_change":"2023-07-13T22:29:49.798859+0000","last_active":"2023-07-13T22:29:49.807750+0000","last_peered":"2023-07-13T22:29:49.807750+0000","last_clean":"2023-07-13T22:29:49.807750+0000","last_became_active":"2023-07-13T22:29:49.798343+0000","last_became_peered":"2023-07-13T22:29:49.798343+0000","last_unstale":"2023-07-13T22:29:49.807750+0000","last_undegraded":"2023-07-13T22:29:49.807750+0000","last_fullsized":"2023-07-13T22:29:49.807750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:56:05.838768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.804755+0000","last_change":"2023-07-13T22:29:48.779363+0000","last_active":"2023-07-13T22:29:49.804755+0000","last_peered":"2023-07-13T22:29:49.804755+0000","last_clean":"2023-07-13T22:29:49.804755+0000","last_became_active":"2023-07-13T22:29:48.779075+0000","last_became_peered":"2023-07-13T22:29:48.779075+0000","last_unstale":"2023-07-13T22:29:49.804755+0000","last_undegraded":"2023-07-13T22:29:49.804755+0000","last_fullsized":"2023-07-13T22:29:49.804755+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:38:25.492994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788348+0000","last_change":"2023-07-13T22:29:49.792033+0000","last_active":"2023-07-13T22:29:50.788348+0000","last_peered":"2023-07-13T22:29:50.788348+0000","last_clean":"2023-07-13T22:29:50.788348+0000","last_became_active":"2023-07-13T22:29:49.791767+0000","last_became_peered":"2023-07-13T22:29:49.791767+0000","last_unstale":"2023-07-13T22:29:50.788348+0000","last_undegraded":"2023-07-13T22:29:50.788348+0000","last_fullsized":"2023-07-13T22:29:50.788348+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:17:39.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.786697+0000","last_change":"2023-07-13T22:29:48.783608+0000","last_active":"2023-07-13T22:29:49.786697+0000","last_peered":"2023-07-13T22:29:49.786697+0000","last_clean":"2023-07-13T22:29:49.786697+0000","last_became_active":"2023-07-13T22:29:48.783263+0000","last_became_peered":"2023-07-13T22:29:48.783263+0000","last_unstale":"2023-07-13T22:29:49.786697+0000","last_undegraded":"2023-07-13T22:29:49.786697+0000","last_fullsized":"2023-07-13T22:29:49.786697+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:03:45.506243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807704+0000","last_change":"2023-07-13T22:29:49.796436+0000","last_active":"2023-07-13T22:29:49.807704+0000","last_peered":"2023-07-13T22:29:49.807704+0000","last_clean":"2023-07-13T22:29:49.807704+0000","last_became_active":"2023-07-13T22:29:49.796073+0000","last_became_peered":"2023-07-13T22:29:49.796073+0000","last_unstale":"2023-07-13T22:29:49.807704+0000","last_undegraded":"2023-07-13T22:29:49.807704+0000","last_fullsized":"2023-07-13T22:29:49.807704+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:59:13.912613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.808192+0000","last_change":"2023-07-13T22:29:48.774426+0000","last_active":"2023-07-13T22:29:49.808192+0000","last_peered":"2023-07-13T22:29:49.808192+0000","last_clean":"2023-07-13T22:29:49.808192+0000","last_became_active":"2023-07-13T22:29:48.774019+0000","last_became_peered":"2023-07-13T22:29:48.774019+0000","last_unstale":"2023-07-13T22:29:49.808192+0000","last_undegraded":"2023-07-13T22:29:49.808192+0000","last_fullsized":"2023-07-13T22:29:49.808192+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:08:24.572500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786029+0000","last_change":"2023-07-13T22:29:49.786513+0000","last_active":"2023-07-13T22:29:50.786029+0000","last_peered":"2023-07-13T22:29:50.786029+0000","last_clean":"2023-07-13T22:29:50.786029+0000","last_became_active":"2023-07-13T22:29:49.786209+0000","last_became_peered":"2023-07-13T22:29:49.786209+0000","last_unstale":"2023-07-13T22:29:50.786029+0000","last_undegraded":"2023-07-13T22:29:50.786029+0000","last_fullsized":"2023-07-13T22:29:50.786029+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:03:56.029937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.785058+0000","last_change":"2023-07-13T22:29:48.772542+0000","last_active":"2023-07-13T22:29:49.785058+0000","last_peered":"2023-07-13T22:29:49.785058+0000","last_clean":"2023-07-13T22:29:49.785058+0000","last_became_active":"2023-07-13T22:29:48.772171+0000","last_became_peered":"2023-07-13T22:29:48.772171+0000","last_unstale":"2023-07-13T22:29:49.785058+0000","last_undegraded":"2023-07-13T22:29:49.785058+0000","last_fullsized":"2023-07-13T22:29:49.785058+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:47:00.470196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786820+0000","last_change":"2023-07-13T22:29:49.797636+0000","last_active":"2023-07-13T22:29:50.786820+0000","last_peered":"2023-07-13T22:29:50.786820+0000","last_clean":"2023-07-13T22:29:50.786820+0000","last_became_active":"2023-07-13T22:29:49.797119+0000","last_became_peered":"2023-07-13T22:29:49.797119+0000","last_unstale":"2023-07-13T22:29:50.786820+0000","last_undegraded":"2023-07-13T22:29:50.786820+0000","last_fullsized":"2023-07-13T22:29:50.786820+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:02.830181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786277+0000","last_change":"2023-07-13T22:29:48.781731+0000","last_active":"2023-07-13T22:29:50.786277+0000","last_peered":"2023-07-13T22:29:50.786277+0000","last_clean":"2023-07-13T22:29:50.786277+0000","last_became_active":"2023-07-13T22:29:48.781328+0000","last_became_peered":"2023-07-13T22:29:48.781328+0000","last_unstale":"2023-07-13T22:29:50.786277+0000","last_undegraded":"2023-07-13T22:29:50.786277+0000","last_fullsized":"2023-07-13T22:29:50.786277+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:34:42.755132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787855+0000","last_change":"2023-07-13T22:29:49.788964+0000","last_active":"2023-07-13T22:29:50.787855+0000","last_peered":"2023-07-13T22:29:50.787855+0000","last_clean":"2023-07-13T22:29:50.787855+0000","last_became_active":"2023-07-13T22:29:49.788667+0000","last_became_peered":"2023-07-13T22:29:49.788667+0000","last_unstale":"2023-07-13T22:29:50.787855+0000","last_undegraded":"2023-07-13T22:29:50.787855+0000","last_fullsized":"2023-07-13T22:29:50.787855+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:27:58.321047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.786023+0000","last_change":"2023-07-13T22:29:48.773107+0000","last_active":"2023-07-13T22:29:49.786023+0000","last_peered":"2023-07-13T22:29:49.786023+0000","last_clean":"2023-07-13T22:29:49.786023+0000","last_became_active":"2023-07-13T22:29:48.772765+0000","last_became_peered":"2023-07-13T22:29:48.772765+0000","last_unstale":"2023-07-13T22:29:49.786023+0000","last_undegraded":"2023-07-13T22:29:49.786023+0000","last_fullsized":"2023-07-13T22:29:49.786023+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:50:40.128049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807502+0000","last_change":"2023-07-13T22:29:49.794647+0000","last_active":"2023-07-13T22:29:49.807502+0000","last_peered":"2023-07-13T22:29:49.807502+0000","last_clean":"2023-07-13T22:29:49.807502+0000","last_became_active":"2023-07-13T22:29:49.794335+0000","last_became_peered":"2023-07-13T22:29:49.794335+0000","last_unstale":"2023-07-13T22:29:49.807502+0000","last_undegraded":"2023-07-13T22:29:49.807502+0000","last_fullsized":"2023-07-13T22:29:49.807502+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:20:33.332719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.810553+0000","last_change":"2023-07-13T22:29:48.784127+0000","last_active":"2023-07-13T22:29:49.810553+0000","last_peered":"2023-07-13T22:29:49.810553+0000","last_clean":"2023-07-13T22:29:49.810553+0000","last_became_active":"2023-07-13T22:29:48.783872+0000","last_became_peered":"2023-07-13T22:29:48.783872+0000","last_unstale":"2023-07-13T22:29:49.810553+0000","last_undegraded":"2023-07-13T22:29:49.810553+0000","last_fullsized":"2023-07-13T22:29:49.810553+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:46:04.647997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.798067+0000","last_change":"2023-07-13T22:29:49.796734+0000","last_active":"2023-07-13T22:29:49.798067+0000","last_peered":"2023-07-13T22:29:49.798067+0000","last_clean":"2023-07-13T22:29:49.798067+0000","last_became_active":"2023-07-13T22:29:49.796451+0000","last_became_peered":"2023-07-13T22:29:49.796451+0000","last_unstale":"2023-07-13T22:29:49.798067+0000","last_undegraded":"2023-07-13T22:29:49.798067+0000","last_fullsized":"2023-07-13T22:29:49.798067+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:53:58.236870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.783308+0000","last_change":"2023-07-13T22:29:48.783720+0000","last_active":"2023-07-13T22:29:49.783308+0000","last_peered":"2023-07-13T22:29:49.783308+0000","last_clean":"2023-07-13T22:29:49.783308+0000","last_became_active":"2023-07-13T22:29:48.783466+0000","last_became_peered":"2023-07-13T22:29:48.783466+0000","last_unstale":"2023-07-13T22:29:49.783308+0000","last_undegraded":"2023-07-13T22:29:49.783308+0000","last_fullsized":"2023-07-13T22:29:49.783308+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:09:18.148759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.798661+0000","last_change":"2023-07-13T22:29:49.794288+0000","last_active":"2023-07-13T22:29:49.798661+0000","last_peered":"2023-07-13T22:29:49.798661+0000","last_clean":"2023-07-13T22:29:49.798661+0000","last_became_active":"2023-07-13T22:29:49.793968+0000","last_became_peered":"2023-07-13T22:29:49.793968+0000","last_unstale":"2023-07-13T22:29:49.798661+0000","last_undegraded":"2023-07-13T22:29:49.798661+0000","last_fullsized":"2023-07-13T22:29:49.798661+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:56:30.198141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.786163+0000","last_change":"2023-07-13T22:29:48.786075+0000","last_active":"2023-07-13T22:29:49.786163+0000","last_peered":"2023-07-13T22:29:49.786163+0000","last_clean":"2023-07-13T22:29:49.786163+0000","last_became_active":"2023-07-13T22:29:48.785699+0000","last_became_peered":"2023-07-13T22:29:48.785699+0000","last_unstale":"2023-07-13T22:29:49.786163+0000","last_undegraded":"2023-07-13T22:29:49.786163+0000","last_fullsized":"2023-07-13T22:29:49.786163+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:42:27.115570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063305+0000","last_change":"2023-07-13T22:29:49.783316+0000","last_active":"2023-07-13T22:29:51.063305+0000","last_peered":"2023-07-13T22:29:51.063305+0000","last_clean":"2023-07-13T22:29:51.063305+0000","last_became_active":"2023-07-13T22:29:49.782848+0000","last_became_peered":"2023-07-13T22:29:49.782848+0000","last_unstale":"2023-07-13T22:29:51.063305+0000","last_undegraded":"2023-07-13T22:29:51.063305+0000","last_fullsized":"2023-07-13T22:29:51.063305+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:46:39.038714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.788155+0000","last_change":"2023-07-13T22:29:48.783243+0000","last_active":"2023-07-13T22:29:49.788155+0000","last_peered":"2023-07-13T22:29:49.788155+0000","last_clean":"2023-07-13T22:29:49.788155+0000","last_became_active":"2023-07-13T22:29:48.782864+0000","last_became_peered":"2023-07-13T22:29:48.782864+0000","last_unstale":"2023-07-13T22:29:49.788155+0000","last_undegraded":"2023-07-13T22:29:49.788155+0000","last_fullsized":"2023-07-13T22:29:49.788155+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:39:11.342728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.804883+0000","last_change":"2023-07-13T22:29:49.792102+0000","last_active":"2023-07-13T22:29:49.804883+0000","last_peered":"2023-07-13T22:29:49.804883+0000","last_clean":"2023-07-13T22:29:49.804883+0000","last_became_active":"2023-07-13T22:29:49.791804+0000","last_became_peered":"2023-07-13T22:29:49.791804+0000","last_unstale":"2023-07-13T22:29:49.804883+0000","last_undegraded":"2023-07-13T22:29:49.804883+0000","last_fullsized":"2023-07-13T22:29:49.804883+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:01:04.284945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789031+0000","last_change":"2023-07-13T22:29:48.772522+0000","last_active":"2023-07-13T22:29:50.789031+0000","last_peered":"2023-07-13T22:29:50.789031+0000","last_clean":"2023-07-13T22:29:50.789031+0000","last_became_active":"2023-07-13T22:29:48.772196+0000","last_became_peered":"2023-07-13T22:29:48.772196+0000","last_unstale":"2023-07-13T22:29:50.789031+0000","last_undegraded":"2023-07-13T22:29:50.789031+0000","last_fullsized":"2023-07-13T22:29:50.789031+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:04:07.897982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786037+0000","last_change":"2023-07-13T22:29:49.797544+0000","last_active":"2023-07-13T22:29:50.786037+0000","last_peered":"2023-07-13T22:29:50.786037+0000","last_clean":"2023-07-13T22:29:50.786037+0000","last_became_active":"2023-07-13T22:29:49.797249+0000","last_became_peered":"2023-07-13T22:29:49.797249+0000","last_unstale":"2023-07-13T22:29:50.786037+0000","last_undegraded":"2023-07-13T22:29:50.786037+0000","last_fullsized":"2023-07-13T22:29:50.786037+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:03:28.229578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789211+0000","last_change":"2023-07-13T22:29:48.773717+0000","last_active":"2023-07-13T22:29:50.789211+0000","last_peered":"2023-07-13T22:29:50.789211+0000","last_clean":"2023-07-13T22:29:50.789211+0000","last_became_active":"2023-07-13T22:29:48.773429+0000","last_became_peered":"2023-07-13T22:29:48.773429+0000","last_unstale":"2023-07-13T22:29:50.789211+0000","last_undegraded":"2023-07-13T22:29:50.789211+0000","last_fullsized":"2023-07-13T22:29:50.789211+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:30:06.481851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.806479+0000","last_change":"2023-07-13T22:29:49.806018+0000","last_active":"2023-07-13T22:29:49.806479+0000","last_peered":"2023-07-13T22:29:49.806479+0000","last_clean":"2023-07-13T22:29:49.806479+0000","last_became_active":"2023-07-13T22:29:49.805708+0000","last_became_peered":"2023-07-13T22:29:49.805708+0000","last_unstale":"2023-07-13T22:29:49.806479+0000","last_undegraded":"2023-07-13T22:29:49.806479+0000","last_fullsized":"2023-07-13T22:29:49.806479+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:49:40.598006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.780868+0000","last_change":"2023-07-13T22:29:48.782641+0000","last_active":"2023-07-13T22:29:49.780868+0000","last_peered":"2023-07-13T22:29:49.780868+0000","last_clean":"2023-07-13T22:29:49.780868+0000","last_became_active":"2023-07-13T22:29:48.782163+0000","last_became_peered":"2023-07-13T22:29:48.782163+0000","last_unstale":"2023-07-13T22:29:49.780868+0000","last_undegraded":"2023-07-13T22:29:49.780868+0000","last_fullsized":"2023-07-13T22:29:49.780868+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:33:15.277248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.794750+0000","last_change":"2023-07-13T22:29:49.792741+0000","last_active":"2023-07-13T22:29:49.794750+0000","last_peered":"2023-07-13T22:29:49.794750+0000","last_clean":"2023-07-13T22:29:49.794750+0000","last_became_active":"2023-07-13T22:29:49.792469+0000","last_became_peered":"2023-07-13T22:29:49.792469+0000","last_unstale":"2023-07-13T22:29:49.794750+0000","last_undegraded":"2023-07-13T22:29:49.794750+0000","last_fullsized":"2023-07-13T22:29:49.794750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:37:02.123624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.787219+0000","last_change":"2023-07-13T22:29:48.779674+0000","last_active":"2023-07-13T22:29:49.787219+0000","last_peered":"2023-07-13T22:29:49.787219+0000","last_clean":"2023-07-13T22:29:49.787219+0000","last_became_active":"2023-07-13T22:29:48.779401+0000","last_became_peered":"2023-07-13T22:29:48.779401+0000","last_unstale":"2023-07-13T22:29:49.787219+0000","last_undegraded":"2023-07-13T22:29:49.787219+0000","last_fullsized":"2023-07-13T22:29:49.787219+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:23:03.083682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786644+0000","last_change":"2023-07-13T22:29:49.798098+0000","last_active":"2023-07-13T22:29:50.786644+0000","last_peered":"2023-07-13T22:29:50.786644+0000","last_clean":"2023-07-13T22:29:50.786644+0000","last_became_active":"2023-07-13T22:29:49.797821+0000","last_became_peered":"2023-07-13T22:29:49.797821+0000","last_unstale":"2023-07-13T22:29:50.786644+0000","last_undegraded":"2023-07-13T22:29:50.786644+0000","last_fullsized":"2023-07-13T22:29:50.786644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:31:40.571752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"111'3","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788118+0000","last_change":"2023-07-13T22:29:48.777158+0000","last_active":"2023-07-13T22:29:50.788118+0000","last_peered":"2023-07-13T22:29:50.788118+0000","last_clean":"2023-07-13T22:29:50.788118+0000","last_became_active":"2023-07-13T22:29:48.776838+0000","last_became_peered":"2023-07-13T22:29:48.776838+0000","last_unstale":"2023-07-13T22:29:50.788118+0000","last_undegraded":"2023-07-13T22:29:50.788118+0000","last_fullsized":"2023-07-13T22:29:50.788118+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:41:30.299089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":183,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.790101+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:29:50.790101+0000","last_peered":"2023-07-13T22:29:50.790101+0000","last_clean":"2023-07-13T22:29:50.790101+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:29:50.790101+0000","last_undegraded":"2023-07-13T22:29:50.790101+0000","last_fullsized":"2023-07-13T22:29:50.790101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:43:11.431505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788334+0000","last_change":"2023-07-13T22:29:49.793695+0000","last_active":"2023-07-13T22:29:50.788334+0000","last_peered":"2023-07-13T22:29:50.788334+0000","last_clean":"2023-07-13T22:29:50.788334+0000","last_became_active":"2023-07-13T22:29:49.793368+0000","last_became_peered":"2023-07-13T22:29:49.793368+0000","last_unstale":"2023-07-13T22:29:50.788334+0000","last_undegraded":"2023-07-13T22:29:50.788334+0000","last_fullsized":"2023-07-13T22:29:50.788334+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:09:19.154215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"111'6","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786322+0000","last_change":"2023-07-13T22:29:48.780737+0000","last_active":"2023-07-13T22:29:50.786322+0000","last_peered":"2023-07-13T22:29:50.786322+0000","last_clean":"2023-07-13T22:29:50.786322+0000","last_became_active":"2023-07-13T22:29:48.779924+0000","last_became_peered":"2023-07-13T22:29:48.779924+0000","last_unstale":"2023-07-13T22:29:50.786322+0000","last_undegraded":"2023-07-13T22:29:50.786322+0000","last_fullsized":"2023-07-13T22:29:50.786322+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:22:48.111167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":175,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786360+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:29:50.786360+0000","last_peered":"2023-07-13T22:29:50.786360+0000","last_clean":"2023-07-13T22:29:50.786360+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:29:50.786360+0000","last_undegraded":"2023-07-13T22:29:50.786360+0000","last_fullsized":"2023-07-13T22:29:50.786360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:48:07.461571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.802814+0000","last_change":"2023-07-13T22:29:49.798814+0000","last_active":"2023-07-13T22:29:49.802814+0000","last_peered":"2023-07-13T22:29:49.802814+0000","last_clean":"2023-07-13T22:29:49.802814+0000","last_became_active":"2023-07-13T22:29:49.797694+0000","last_became_peered":"2023-07-13T22:29:49.797694+0000","last_unstale":"2023-07-13T22:29:49.802814+0000","last_undegraded":"2023-07-13T22:29:49.802814+0000","last_fullsized":"2023-07-13T22:29:49.802814+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:00:50.825775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786845+0000","last_change":"2023-07-13T22:29:48.775117+0000","last_active":"2023-07-13T22:29:50.786845+0000","last_peered":"2023-07-13T22:29:50.786845+0000","last_clean":"2023-07-13T22:29:50.786845+0000","last_became_active":"2023-07-13T22:29:48.774709+0000","last_became_peered":"2023-07-13T22:29:48.774709+0000","last_unstale":"2023-07-13T22:29:50.786845+0000","last_undegraded":"2023-07-13T22:29:50.786845+0000","last_fullsized":"2023-07-13T22:29:50.786845+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:40:08.500902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":187,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.780304+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:29:49.780304+0000","last_peered":"2023-07-13T22:29:49.780304+0000","last_clean":"2023-07-13T22:29:49.780304+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:29:49.780304+0000","last_undegraded":"2023-07-13T22:29:49.780304+0000","last_fullsized":"2023-07-13T22:29:49.780304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:57:57.150009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":321,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.786847+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:29:49.786847+0000","last_peered":"2023-07-13T22:29:49.786847+0000","last_clean":"2023-07-13T22:29:49.786847+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:29:49.786847+0000","last_undegraded":"2023-07-13T22:29:49.786847+0000","last_fullsized":"2023-07-13T22:29:49.786847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:28:10.176414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.802896+0000","last_change":"2023-07-13T22:29:49.800414+0000","last_active":"2023-07-13T22:29:49.802896+0000","last_peered":"2023-07-13T22:29:49.802896+0000","last_clean":"2023-07-13T22:29:49.802896+0000","last_became_active":"2023-07-13T22:29:49.800094+0000","last_became_peered":"2023-07-13T22:29:49.800094+0000","last_unstale":"2023-07-13T22:29:49.802896+0000","last_undegraded":"2023-07-13T22:29:49.802896+0000","last_fullsized":"2023-07-13T22:29:49.802896+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:21:33.884645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786943+0000","last_change":"2023-07-13T22:29:48.774258+0000","last_active":"2023-07-13T22:29:50.786943+0000","last_peered":"2023-07-13T22:29:50.786943+0000","last_clean":"2023-07-13T22:29:50.786943+0000","last_became_active":"2023-07-13T22:29:48.773869+0000","last_became_peered":"2023-07-13T22:29:48.773869+0000","last_unstale":"2023-07-13T22:29:50.786943+0000","last_undegraded":"2023-07-13T22:29:50.786943+0000","last_fullsized":"2023-07-13T22:29:50.786943+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:53:16.478797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":173,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.785163+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:29:49.785163+0000","last_peered":"2023-07-13T22:29:49.785163+0000","last_clean":"2023-07-13T22:29:49.785163+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:29:49.785163+0000","last_undegraded":"2023-07-13T22:29:49.785163+0000","last_fullsized":"2023-07-13T22:29:49.785163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:13:05.286337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.803897+0000","last_change":"2023-07-13T22:29:49.800051+0000","last_active":"2023-07-13T22:29:49.803897+0000","last_peered":"2023-07-13T22:29:49.803897+0000","last_clean":"2023-07-13T22:29:49.803897+0000","last_became_active":"2023-07-13T22:29:49.799721+0000","last_became_peered":"2023-07-13T22:29:49.799721+0000","last_unstale":"2023-07-13T22:29:49.803897+0000","last_undegraded":"2023-07-13T22:29:49.803897+0000","last_fullsized":"2023-07-13T22:29:49.803897+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:45:15.768466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"111'8","reported_seq":21,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786073+0000","last_change":"2023-07-13T22:29:48.777653+0000","last_active":"2023-07-13T22:29:50.786073+0000","last_peered":"2023-07-13T22:29:50.786073+0000","last_clean":"2023-07-13T22:29:50.786073+0000","last_became_active":"2023-07-13T22:29:48.774359+0000","last_became_peered":"2023-07-13T22:29:48.774359+0000","last_unstale":"2023-07-13T22:29:50.786073+0000","last_undegraded":"2023-07-13T22:29:50.786073+0000","last_fullsized":"2023-07-13T22:29:50.786073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:51:46.039327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":173,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.787106+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:29:49.787106+0000","last_peered":"2023-07-13T22:29:49.787106+0000","last_clean":"2023-07-13T22:29:49.787106+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:29:49.787106+0000","last_undegraded":"2023-07-13T22:29:49.787106+0000","last_fullsized":"2023-07-13T22:29:49.787106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:32:35.857901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063028+0000","last_change":"2023-07-13T22:29:49.790468+0000","last_active":"2023-07-13T22:29:51.063028+0000","last_peered":"2023-07-13T22:29:51.063028+0000","last_clean":"2023-07-13T22:29:51.063028+0000","last_became_active":"2023-07-13T22:29:49.790195+0000","last_became_peered":"2023-07-13T22:29:49.790195+0000","last_unstale":"2023-07-13T22:29:51.063028+0000","last_undegraded":"2023-07-13T22:29:51.063028+0000","last_fullsized":"2023-07-13T22:29:51.063028+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:42:40.974854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786228+0000","last_change":"2023-07-13T22:29:48.772838+0000","last_active":"2023-07-13T22:29:50.786228+0000","last_peered":"2023-07-13T22:29:50.786228+0000","last_clean":"2023-07-13T22:29:50.786228+0000","last_became_active":"2023-07-13T22:29:48.772522+0000","last_became_peered":"2023-07-13T22:29:48.772522+0000","last_unstale":"2023-07-13T22:29:50.786228+0000","last_undegraded":"2023-07-13T22:29:50.786228+0000","last_fullsized":"2023-07-13T22:29:50.786228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:44:29.136805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":197,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788011+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:29:50.788011+0000","last_peered":"2023-07-13T22:29:50.788011+0000","last_clean":"2023-07-13T22:29:50.788011+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:29:50.788011+0000","last_undegraded":"2023-07-13T22:29:50.788011+0000","last_fullsized":"2023-07-13T22:29:50.788011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:05:43.089032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787947+0000","last_change":"2023-07-13T22:29:49.781518+0000","last_active":"2023-07-13T22:29:50.787947+0000","last_peered":"2023-07-13T22:29:50.787947+0000","last_clean":"2023-07-13T22:29:50.787947+0000","last_became_active":"2023-07-13T22:29:49.781168+0000","last_became_peered":"2023-07-13T22:29:49.781168+0000","last_unstale":"2023-07-13T22:29:50.787947+0000","last_undegraded":"2023-07-13T22:29:50.787947+0000","last_fullsized":"2023-07-13T22:29:50.787947+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:51:54.745934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787945+0000","last_change":"2023-07-13T22:29:48.774310+0000","last_active":"2023-07-13T22:29:50.787945+0000","last_peered":"2023-07-13T22:29:50.787945+0000","last_clean":"2023-07-13T22:29:50.787945+0000","last_became_active":"2023-07-13T22:29:48.773891+0000","last_became_peered":"2023-07-13T22:29:48.773891+0000","last_unstale":"2023-07-13T22:29:50.787945+0000","last_undegraded":"2023-07-13T22:29:50.787945+0000","last_fullsized":"2023-07-13T22:29:50.787945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:50:34.648462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":169,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.783408+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:29:49.783408+0000","last_peered":"2023-07-13T22:29:49.783408+0000","last_clean":"2023-07-13T22:29:49.783408+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:29:49.783408+0000","last_undegraded":"2023-07-13T22:29:49.783408+0000","last_fullsized":"2023-07-13T22:29:49.783408+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:08:37.720940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.801247+0000","last_change":"2023-07-13T22:29:49.797225+0000","last_active":"2023-07-13T22:29:49.801247+0000","last_peered":"2023-07-13T22:29:49.801247+0000","last_clean":"2023-07-13T22:29:49.801247+0000","last_became_active":"2023-07-13T22:29:49.796947+0000","last_became_peered":"2023-07-13T22:29:49.796947+0000","last_unstale":"2023-07-13T22:29:49.801247+0000","last_undegraded":"2023-07-13T22:29:49.801247+0000","last_fullsized":"2023-07-13T22:29:49.801247+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:17.182760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.782827+0000","last_change":"2023-07-13T22:29:48.785888+0000","last_active":"2023-07-13T22:29:49.782827+0000","last_peered":"2023-07-13T22:29:49.782827+0000","last_clean":"2023-07-13T22:29:49.782827+0000","last_became_active":"2023-07-13T22:29:48.785584+0000","last_became_peered":"2023-07-13T22:29:48.785584+0000","last_unstale":"2023-07-13T22:29:49.782827+0000","last_undegraded":"2023-07-13T22:29:49.782827+0000","last_fullsized":"2023-07-13T22:29:49.782827+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:41:30.299089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":169,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.782912+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:29:49.782912+0000","last_peered":"2023-07-13T22:29:49.782912+0000","last_clean":"2023-07-13T22:29:49.782912+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:29:49.782912+0000","last_undegraded":"2023-07-13T22:29:49.782912+0000","last_fullsized":"2023-07-13T22:29:49.782912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:52:10.160426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574961,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37088,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334752,"statfs":{"total":96636764160,"available":96598786048,"internally_reserved":0,"allocated":1990656,"data_stored":1221360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67200000000000004,"15min":0.63900000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.874,"5min":1.0780000000000001,"15min":1.115},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.67300000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.94199999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.42899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.86299999999999999,"5min":0.94799999999999995,"15min":2.2610000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34300000000000003},"max":{"1min":1.0149999999999999,"5min":1.0309999999999999,"15min":1.0609999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70699999999999996,"15min":0.77400000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69999999999999996,"15min":0.76600000000000001},"min":{"1min":0.442,"5min":0.42999999999999999,"15min":0.36499999999999999},"max":{"1min":1.006,"5min":1.107,"15min":3.4710000000000001},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.34599999999999997},"max":{"1min":0.90500000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.496},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.747,"15min":0.75800000000000001},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92200000000000004,"5min":1.024,"15min":1.1399999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.82399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.73899999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.1850000000000001},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.626,"15min":0.65600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0149999999999999,"5min":1.022,"15min":1.2110000000000001},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.67700000000000005,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.93500000000000005,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.623},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69099999999999995,"15min":0.71799999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94399999999999995,"5min":0.98799999999999999,"15min":1.042},"last":0.80700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1352,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1384448,"data_stored":630798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66600000000000004,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.28799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69399999999999995,"15min":0.63300000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.434,"5min":0.39400000000000002,"15min":0.33900000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69399999999999995,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.495},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69199999999999995,"15min":0.66900000000000004},"min":{"1min":0.46100000000000002,"5min":0.45600000000000002,"15min":0.35599999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.84099999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69299999999999995,"15min":0.66300000000000003},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.53400000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60199999999999998,"15min":0.64600000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.36899999999999999},"max":{"1min":0.80600000000000005,"5min":0.90600000000000003,"15min":0.94499999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.65200000000000002,"15min":0.68200000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":0.91500000000000004,"5min":0.997,"15min":0.998},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":0.94199999999999995,"15min":0.95999999999999996},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73499999999999999,"15min":0.755},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.05,"15min":1.05},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.74299999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.375},"max":{"1min":0.93400000000000005,"5min":0.96999999999999997,"15min":1.1890000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69799999999999995,"15min":0.73599999999999999},"min":{"1min":0.45300000000000001,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.111},"last":0.94299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1932,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1978368,"data_stored":1220606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.44500000000000001,"15min":0.48299999999999998},"min":{"1min":0.29199999999999998,"5min":0.27800000000000002,"15min":0.249},"max":{"1min":0.58799999999999997,"5min":0.71599999999999997,"15min":1.952},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.254,"15min":0.254},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.433}]},{"osd":2,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46200000000000002,"15min":0.52300000000000002},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29199999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":2.1200000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.54200000000000004,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.627,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.57599999999999996,"15min":0.61299999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39200000000000002},"max":{"1min":0.70199999999999996,"5min":0.83899999999999997,"15min":2.008},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.52600000000000002,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93700000000000006,"5min":1.054,"15min":1.054},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74399999999999999,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":0.98199999999999998,"15min":2.028},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.76400000000000001,"15min":0.79100000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.46400000000000002},"max":{"1min":0.96999999999999997,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.89100000000000001,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.83499999999999996},"min":{"1min":0.64700000000000002,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.88900000000000001,"5min":1.095,"15min":2.0899999999999999},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.58299999999999996,"5min":0.55100000000000005,"15min":0.52600000000000002},"max":{"1min":0.97099999999999997,"5min":1.073,"15min":1.2230000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76900000000000002,"15min":0.84699999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93999999999999995,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.875}]},{"osd":7,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.79100000000000004,"15min":0.81899999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.50700000000000001},"max":{"1min":0.93100000000000005,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79000000000000004,"15min":0.81399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.93600000000000005,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.78800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36492,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1380352,"data_stored":629252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.45200000000000001,"15min":0.51900000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.68799999999999994,"5min":0.84999999999999998,"15min":11.791},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.29299999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":4.657},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.441,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":1.056,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.51200000000000001,"15min":0.91800000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.55800000000000005,"5min":1.276,"15min":13.398999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":1.8089999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":1.095,"5min":1.095,"15min":4.4530000000000003},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74299999999999999,"15min":1.167},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0149999999999999,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.751,"15min":0.80000000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":1.1220000000000001,"15min":1.776},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77900000000000003,"15min":1.1879999999999999},"min":{"1min":0.499,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0509999999999999,"5min":1.113,"15min":13.349},"last":0.504},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.872},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1739999999999999,"5min":1.302,"15min":4.7649999999999997},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79200000000000004,"15min":0.83299999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77500000000000002,"15min":1.1599999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.046,"5min":1.2330000000000001,"15min":11.679},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.78200000000000003,"15min":1.1950000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":1.1759999999999999,"15min":13.298},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79100000000000004,"15min":0.88400000000000001},"min":{"1min":0.53700000000000003,"5min":0.51800000000000002,"15min":0.50900000000000001},"max":{"1min":1.034,"5min":1.151,"15min":11.456},"last":0.71299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1388544,"data_stored":635883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.434,"15min":0.51800000000000002},"min":{"1min":0.27600000000000002,"5min":0.252,"15min":0.249},"max":{"1min":0.67700000000000005,"5min":1.1779999999999999,"15min":2.282},"last":0.501},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.497,"15min":0.47799999999999998},"min":{"1min":0.32700000000000001,"5min":0.30399999999999999,"15min":0.29099999999999998},"max":{"1min":0.58999999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.26200000000000001},"max":{"1min":0.54200000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.371},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.58299999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73899999999999999,"5min":0.86899999999999999,"15min":2.319},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.33900000000000002,"15min":0.311},"max":{"1min":0.64600000000000002,"5min":1.01,"15min":1.01},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57799999999999996,"15min":0.63300000000000001},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.38700000000000001},"max":{"1min":0.754,"5min":0.95599999999999996,"15min":2.351},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77000000000000002,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95599999999999996,"5min":1.165,"15min":2.238},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78200000000000003,"15min":0.78100000000000003},"min":{"1min":0.53700000000000003,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":1.0269999999999999,"5min":1.274,"15min":1.518},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.61299999999999999,"5min":0.55800000000000005,"15min":0.505},"max":{"1min":1.0109999999999999,"5min":1.099,"15min":1.325},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80400000000000005,"15min":0.84899999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":0.98299999999999998,"5min":1.2490000000000001,"15min":2.1909999999999998},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.56200000000000006,"5min":0.52900000000000003,"15min":0.51700000000000002},"max":{"1min":0.95599999999999996,"5min":1.1040000000000001,"15min":1.488},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.79200000000000004},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.48199999999999998},"max":{"1min":1.0409999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81499999999999995,"15min":0.79500000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":1.0549999999999999,"5min":1.29,"15min":1.29},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81000000000000005,"15min":0.82599999999999996},"min":{"1min":0.57599999999999996,"5min":0.54800000000000004,"15min":0.54400000000000004},"max":{"1min":1.0329999999999999,"5min":1.254,"15min":1.254},"last":0.71899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574960,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36508,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":1396736,"data_stored":631814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.30399999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.63500000000000001,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48199999999999998,"15min":0.45400000000000001},"min":{"1min":0.34899999999999998,"5min":0.29699999999999999,"15min":0.27200000000000002},"max":{"1min":0.61399999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.32400000000000001,"5min":0.313,"15min":0.28299999999999997},"max":{"1min":0.71699999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.314},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.746,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.52300000000000002},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80600000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.38300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.82599999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.496}]},{"osd":4,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":1.1699999999999999,"15min":1.2250000000000001},"last":0.623},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.56200000000000006,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.749}]},{"osd":5,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.53900000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.02,"5min":1.1919999999999999,"15min":1.411},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.78100000000000003},"min":{"1min":0.55300000000000005,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.93700000000000006,"5min":1.1779999999999999,"15min":1.5469999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"max":{"1min":0.92100000000000004,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.77200000000000002},"min":{"1min":0.58799999999999997,"5min":0.53400000000000003,"15min":0.52100000000000002},"max":{"1min":0.97899999999999998,"5min":1.046,"15min":1.1240000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98799999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.63,"5min":0.55600000000000005,"15min":0.47199999999999998},"max":{"1min":0.95199999999999996,"5min":1.153,"15min":1.1950000000000001},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36516,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335324,"statfs":{"total":96636764160,"available":96599371776,"internally_reserved":0,"allocated":1404928,"data_stored":638433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":0.997,"15min":0.997},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.35099999999999998},"max":{"1min":0.92200000000000004,"5min":1.0489999999999999,"15min":1.298},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72099999999999997,"15min":0.69599999999999995},"min":{"1min":0.499,"5min":0.47999999999999998,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.0049999999999999,"15min":1.3109999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.95099999999999996,"5min":1.0629999999999999,"15min":1.1200000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.71799999999999997},"min":{"1min":0.46300000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.089,"15min":1.1839999999999999},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.92500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":1.234,"5min":1.234,"15min":10.657999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72799999999999998,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.0600000000000001,"15min":8.75},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":0.97899999999999998,"5min":1.042,"15min":1.042},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.53900000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66700000000000004,"15min":0.68799999999999994},"min":{"1min":0.442,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99299999999999999,"5min":1.0189999999999999,"15min":1.173},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70699999999999996,"15min":0.72199999999999998},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.099},"last":0.746}]},{"osd":7,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.76900000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0600000000000001,"5min":1.1819999999999999,"15min":1.212},"last":0.79600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36500,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335340,"statfs":{"total":96636764160,"available":96599388160,"internally_reserved":0,"allocated":1388544,"data_stored":630376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.65500000000000003,"15min":0.623},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.752,"5min":1.042,"15min":1.042},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.67600000000000005,"15min":0.63600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81499999999999995,"5min":0.998,"15min":0.998},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69999999999999996,"15min":0.66300000000000003},"min":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.66000000000000003,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84699999999999998,"5min":0.94499999999999995,"15min":8.9930000000000003},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.67700000000000005,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.35899999999999999},"max":{"1min":0.74199999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70099999999999996,"15min":0.67700000000000005},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90900000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71799999999999997,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":0.96599999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72199999999999998,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.66900000000000004,"15min":0.63800000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.746},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92900000000000005,"5min":0.93500000000000005,"15min":0.995},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.495,"15min":0.42299999999999999},"max":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.44700000000000001,"15min":0.41199999999999998},"max":{"1min":0.99299999999999999,"5min":0.997,"15min":1.02},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77800000000000002,"15min":0.76500000000000001},"min":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0289999999999999,"15min":1.0669999999999999},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:29:54.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:29:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.468+0000 7f25ed7ff700 1 Processor -- start 2023-07-13T22:29:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.468+0000 7f25ed7ff700 1 -- start start 2023-07-13T22:29:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25ed7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 0x7f25e80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25ed7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25e80cfad0 0x7f25e80d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25ed7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e80cd950 con 0x7f25e80b8400 2023-07-13T22:29:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25ed7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e80cff20 con 0x7f25e80cfad0 2023-07-13T22:29:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25ed7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80d0ac0 con 0x7f25e80d3750 2023-07-13T22:29:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25e77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25e80cfad0 0x7f25e80d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25e77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25e80cfad0 0x7f25e80d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46378/0 (socket says 172.21.15.60:46378) 2023-07-13T22:29:54.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25e77fe700 1 -- 172.21.15.60:0/510736477 learned_addr learned my addr 172.21.15.60:0/510736477 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:54.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25dffff700 1 --2- 172.21.15.60:0/510736477 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 0x7f25e80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25e77fe700 1 -- 172.21.15.60:0/510736477 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 msgr2=0x7f25e80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25e77fe700 1 --2- 172.21.15.60:0/510736477 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 0x7f25e80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25e77fe700 1 -- 172.21.15.60:0/510736477 >> v1:172.21.15.105:6789/0 conn(0x7f25e80b8400 legacy=0x7f25e80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.470+0000 7f25e77fe700 1 -- 172.21.15.60:0/510736477 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e80b87d0 con 0x7f25e80cfad0 2023-07-13T22:29:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.471+0000 7f25e77fe700 1 --2- 172.21.15.60:0/510736477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 0x7f25e80d0f50 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009e50 tx=0x7f25e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7401bd766c79be2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.471+0000 7f25e67fc700 1 -- 172.21.15.60:0/510736477 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000e070 con 0x7f25e80cfad0 2023-07-13T22:29:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.471+0000 7f25e67fc700 1 -- 172.21.15.60:0/510736477 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e000b910 con 0x7f25e80cfad0 2023-07-13T22:29:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.471+0000 7f25e67fc700 1 -- 172.21.15.60:0/510736477 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000bcc0 con 0x7f25e80cfad0 2023-07-13T22:29:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.471+0000 7f25ed7ff700 1 -- 172.21.15.60:0/510736477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 msgr2=0x7f25e80d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.471+0000 7f25ed7ff700 1 --2- 172.21.15.60:0/510736477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 0x7f25e80d0f50 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009e50 tx=0x7f25e000b330 comp rx=0 tx=0).stop 2023-07-13T22:29:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.472+0000 7f25ed7ff700 1 -- 172.21.15.60:0/510736477 shutdown_connections 2023-07-13T22:29:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.472+0000 7f25ed7ff700 1 --2- 172.21.15.60:0/510736477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 0x7f25e80d0f50 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.472+0000 7f25ed7ff700 1 --2- 172.21.15.60:0/510736477 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 0x7f25e80cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.472+0000 7f25ed7ff700 1 -- 172.21.15.60:0/510736477 >> 172.21.15.60:0/510736477 conn(0x7f25e806b390 msgr2=0x7f25e806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.472+0000 7f25ed7ff700 1 -- 172.21.15.60:0/510736477 shutdown_connections 2023-07-13T22:29:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.472+0000 7f25ed7ff700 1 -- 172.21.15.60:0/510736477 wait complete. 2023-07-13T22:29:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.473+0000 7f25ed7ff700 1 Processor -- start 2023-07-13T22:29:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.474+0000 7f25ed7ff700 1 -- start start 2023-07-13T22:29:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.474+0000 7f25ed7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e80b8400 0x7f25e80c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25ed7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 0x7f25e8150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25ed7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 0x7f25e815ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25ed7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80c8410 con 0x7f25e80cfad0 2023-07-13T22:29:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25ed7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815efd0 con 0x7f25e80b8400 2023-07-13T22:29:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25ed7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80c7e50 con 0x7f25e80d3750 2023-07-13T22:29:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25dffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 0x7f25e8150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25dffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 0x7f25e8150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46394/0 (socket says 172.21.15.60:46394) 2023-07-13T22:29:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25dffff700 1 -- 172.21.15.60:0/3086947600 learned_addr learned my addr 172.21.15.60:0/3086947600 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25e6ffd700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e80b8400 0x7f25e80c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25e77fe700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 0x7f25e815ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25dffff700 1 -- 172.21.15.60:0/3086947600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 msgr2=0x7f25e815ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25dffff700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 0x7f25e815ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25dffff700 1 -- 172.21.15.60:0/3086947600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e80b8400 msgr2=0x7f25e80c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25dffff700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e80b8400 0x7f25e80c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25dffff700 1 -- 172.21.15.60:0/3086947600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0009b30 con 0x7f25e80cfad0 2023-07-13T22:29:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25e6ffd700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e80b8400 0x7f25e80c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.475+0000 7f25dffff700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 0x7f25e8150ed0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f25d000e840 tx=0x7f25d000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.476+0000 7f25df7fe700 1 -- 172.21.15.60:0/3086947600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0009590 con 0x7f25e80cfad0 2023-07-13T22:29:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.476+0000 7f25df7fe700 1 -- 172.21.15.60:0/3086947600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d0013680 con 0x7f25e80cfad0 2023-07-13T22:29:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.476+0000 7f25df7fe700 1 -- 172.21.15.60:0/3086947600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0013a30 con 0x7f25e80cfad0 2023-07-13T22:29:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.476+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e80c8130 con 0x7f25e80cfad0 2023-07-13T22:29:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.476+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e815cdf0 con 0x7f25e80cfad0 2023-07-13T22:29:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.477+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e815c970 con 0x7f25e80cfad0 2023-07-13T22:29:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.478+0000 7f25df7fe700 1 -- 172.21.15.60:0/3086947600 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f25d0013b90 con 0x7f25e80cfad0 2023-07-13T22:29:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.479+0000 7f25df7fe700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c806ace0 0x7f25c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.479+0000 7f25df7fe700 1 -- 172.21.15.60:0/3086947600 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25d0089470 con 0x7f25e80cfad0 2023-07-13T22:29:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.479+0000 7f25e6ffd700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c806ace0 0x7f25c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.480+0000 7f25e6ffd700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c806ace0 0x7f25c806d190 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000f60 tx=0x7f25d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.483+0000 7f25df7fe700 1 -- 172.21.15.60:0/3086947600 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f25e815c970 con 0x7f25e80cfad0 2023-07-13T22:29:54.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.685+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25e80b4eb0 con 0x7f25c806ace0 2023-07-13T22:29:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.695+0000 7f25df7fe700 1 -- 172.21.15.60:0/3086947600 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311757 (secure 0 0 0) 0x7f25d8001450 con 0x7f25c806ace0 2023-07-13T22:29:54.678 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:54.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c806ace0 msgr2=0x7f25c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c806ace0 0x7f25c806d190 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000f60 tx=0x7f25d8009040 comp rx=0 tx=0).stop 2023-07-13T22:29:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 msgr2=0x7f25e8150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 0x7f25e8150ed0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f25d000e840 tx=0x7f25d000ec00 comp rx=0 tx=0).stop 2023-07-13T22:29:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 shutdown_connections 2023-07-13T22:29:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f25c806ace0 0x7f25c806d190 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25e80d3750 0x7f25e815ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25e80b8400 0x7f25e80c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 --2- 172.21.15.60:0/3086947600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25e80cfad0 0x7f25e8150ed0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 >> 172.21.15.60:0/3086947600 conn(0x7f25e806b390 msgr2=0x7f25e80d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:54.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 shutdown_connections 2023-07-13T22:29:54.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.699+0000 7f25ed7ff700 1 -- 172.21.15.60:0/3086947600 wait complete. 2023-07-13T22:29:54.682 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:29:54.696 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2023-07-13T22:29:53.243864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284984,"kb_used_data":12024,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754689736,"statfs":{"total":773094113280,"available":772802289664,"internally_reserved":0,"allocated":12312576,"data_stored":6238522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.488902"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789386+0000","last_change":"2023-07-13T22:29:49.789346+0000","last_active":"2023-07-13T22:29:50.789386+0000","last_peered":"2023-07-13T22:29:50.789386+0000","last_clean":"2023-07-13T22:29:50.789386+0000","last_became_active":"2023-07-13T22:29:49.788901+0000","last_became_peered":"2023-07-13T22:29:49.788901+0000","last_unstale":"2023-07-13T22:29:50.789386+0000","last_undegraded":"2023-07-13T22:29:50.789386+0000","last_fullsized":"2023-07-13T22:29:50.789386+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:39:01.446319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786738+0000","last_change":"2023-07-13T22:29:49.793387+0000","last_active":"2023-07-13T22:29:50.786738+0000","last_peered":"2023-07-13T22:29:50.786738+0000","last_clean":"2023-07-13T22:29:50.786738+0000","last_became_active":"2023-07-13T22:29:49.792792+0000","last_became_peered":"2023-07-13T22:29:49.792792+0000","last_unstale":"2023-07-13T22:29:50.786738+0000","last_undegraded":"2023-07-13T22:29:50.786738+0000","last_fullsized":"2023-07-13T22:29:50.786738+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:42:51.567608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786458+0000","last_change":"2023-07-13T22:29:49.799582+0000","last_active":"2023-07-13T22:29:50.786458+0000","last_peered":"2023-07-13T22:29:50.786458+0000","last_clean":"2023-07-13T22:29:50.786458+0000","last_became_active":"2023-07-13T22:29:49.799082+0000","last_became_peered":"2023-07-13T22:29:49.799082+0000","last_unstale":"2023-07-13T22:29:50.786458+0000","last_undegraded":"2023-07-13T22:29:50.786458+0000","last_fullsized":"2023-07-13T22:29:50.786458+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:34:16.133061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.798712+0000","last_change":"2023-07-13T22:29:49.794344+0000","last_active":"2023-07-13T22:29:49.798712+0000","last_peered":"2023-07-13T22:29:49.798712+0000","last_clean":"2023-07-13T22:29:49.798712+0000","last_became_active":"2023-07-13T22:29:49.794092+0000","last_became_peered":"2023-07-13T22:29:49.794092+0000","last_unstale":"2023-07-13T22:29:49.798712+0000","last_undegraded":"2023-07-13T22:29:49.798712+0000","last_fullsized":"2023-07-13T22:29:49.798712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:15:09.575950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788781+0000","last_change":"2023-07-13T22:29:49.796389+0000","last_active":"2023-07-13T22:29:50.788781+0000","last_peered":"2023-07-13T22:29:50.788781+0000","last_clean":"2023-07-13T22:29:50.788781+0000","last_became_active":"2023-07-13T22:29:49.796086+0000","last_became_peered":"2023-07-13T22:29:49.796086+0000","last_unstale":"2023-07-13T22:29:50.788781+0000","last_undegraded":"2023-07-13T22:29:50.788781+0000","last_fullsized":"2023-07-13T22:29:50.788781+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:20:44.967439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.810106+0000","last_change":"2023-07-13T22:29:49.799550+0000","last_active":"2023-07-13T22:29:49.810106+0000","last_peered":"2023-07-13T22:29:49.810106+0000","last_clean":"2023-07-13T22:29:49.810106+0000","last_became_active":"2023-07-13T22:29:49.799218+0000","last_became_peered":"2023-07-13T22:29:49.799218+0000","last_unstale":"2023-07-13T22:29:49.810106+0000","last_undegraded":"2023-07-13T22:29:49.810106+0000","last_fullsized":"2023-07-13T22:29:49.810106+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:00:50.825775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.809593+0000","last_change":"2023-07-13T22:29:49.798054+0000","last_active":"2023-07-13T22:29:49.809593+0000","last_peered":"2023-07-13T22:29:49.809593+0000","last_clean":"2023-07-13T22:29:49.809593+0000","last_became_active":"2023-07-13T22:29:49.797759+0000","last_became_peered":"2023-07-13T22:29:49.797759+0000","last_unstale":"2023-07-13T22:29:49.809593+0000","last_undegraded":"2023-07-13T22:29:49.809593+0000","last_fullsized":"2023-07-13T22:29:49.809593+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:42:28.121026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786048+0000","last_change":"2023-07-13T22:29:49.798300+0000","last_active":"2023-07-13T22:29:50.786048+0000","last_peered":"2023-07-13T22:29:50.786048+0000","last_clean":"2023-07-13T22:29:50.786048+0000","last_became_active":"2023-07-13T22:29:49.797986+0000","last_became_peered":"2023-07-13T22:29:49.797986+0000","last_unstale":"2023-07-13T22:29:50.786048+0000","last_undegraded":"2023-07-13T22:29:50.786048+0000","last_fullsized":"2023-07-13T22:29:50.786048+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:05:53.318240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.800005+0000","last_change":"2023-07-13T22:29:49.797454+0000","last_active":"2023-07-13T22:29:49.800005+0000","last_peered":"2023-07-13T22:29:49.800005+0000","last_clean":"2023-07-13T22:29:49.800005+0000","last_became_active":"2023-07-13T22:29:49.797179+0000","last_became_peered":"2023-07-13T22:29:49.797179+0000","last_unstale":"2023-07-13T22:29:49.800005+0000","last_undegraded":"2023-07-13T22:29:49.800005+0000","last_fullsized":"2023-07-13T22:29:49.800005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:47:40.827548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.795924+0000","last_change":"2023-07-13T22:29:49.791970+0000","last_active":"2023-07-13T22:29:49.795924+0000","last_peered":"2023-07-13T22:29:49.795924+0000","last_clean":"2023-07-13T22:29:49.795924+0000","last_became_active":"2023-07-13T22:29:49.791593+0000","last_became_peered":"2023-07-13T22:29:49.791593+0000","last_unstale":"2023-07-13T22:29:49.795924+0000","last_undegraded":"2023-07-13T22:29:49.795924+0000","last_fullsized":"2023-07-13T22:29:49.795924+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:05:57.218132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789054+0000","last_change":"2023-07-13T22:29:49.791595+0000","last_active":"2023-07-13T22:29:50.789054+0000","last_peered":"2023-07-13T22:29:50.789054+0000","last_clean":"2023-07-13T22:29:50.789054+0000","last_became_active":"2023-07-13T22:29:49.791326+0000","last_became_peered":"2023-07-13T22:29:49.791326+0000","last_unstale":"2023-07-13T22:29:50.789054+0000","last_undegraded":"2023-07-13T22:29:50.789054+0000","last_fullsized":"2023-07-13T22:29:50.789054+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:18:14.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807528+0000","last_change":"2023-07-13T22:29:49.795077+0000","last_active":"2023-07-13T22:29:49.807528+0000","last_peered":"2023-07-13T22:29:49.807528+0000","last_clean":"2023-07-13T22:29:49.807528+0000","last_became_active":"2023-07-13T22:29:49.794731+0000","last_became_peered":"2023-07-13T22:29:49.794731+0000","last_unstale":"2023-07-13T22:29:49.807528+0000","last_undegraded":"2023-07-13T22:29:49.807528+0000","last_fullsized":"2023-07-13T22:29:49.807528+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:40:14.670560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.791850+0000","last_change":"2023-07-13T22:29:49.790654+0000","last_active":"2023-07-13T22:29:49.791850+0000","last_peered":"2023-07-13T22:29:49.791850+0000","last_clean":"2023-07-13T22:29:49.791850+0000","last_became_active":"2023-07-13T22:29:49.790356+0000","last_became_peered":"2023-07-13T22:29:49.790356+0000","last_unstale":"2023-07-13T22:29:49.791850+0000","last_undegraded":"2023-07-13T22:29:49.791850+0000","last_fullsized":"2023-07-13T22:29:49.791850+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:24:50.307758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786670+0000","last_change":"2023-07-13T22:29:49.797809+0000","last_active":"2023-07-13T22:29:50.786670+0000","last_peered":"2023-07-13T22:29:50.786670+0000","last_clean":"2023-07-13T22:29:50.786670+0000","last_became_active":"2023-07-13T22:29:49.797531+0000","last_became_peered":"2023-07-13T22:29:49.797531+0000","last_unstale":"2023-07-13T22:29:50.786670+0000","last_undegraded":"2023-07-13T22:29:50.786670+0000","last_fullsized":"2023-07-13T22:29:50.786670+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:00.845441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788750+0000","last_change":"2023-07-13T22:29:49.793635+0000","last_active":"2023-07-13T22:29:50.788750+0000","last_peered":"2023-07-13T22:29:50.788750+0000","last_clean":"2023-07-13T22:29:50.788750+0000","last_became_active":"2023-07-13T22:29:49.791983+0000","last_became_peered":"2023-07-13T22:29:49.791983+0000","last_unstale":"2023-07-13T22:29:50.788750+0000","last_undegraded":"2023-07-13T22:29:50.788750+0000","last_fullsized":"2023-07-13T22:29:50.788750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:39:12.348184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807680+0000","last_change":"2023-07-13T22:29:49.793986+0000","last_active":"2023-07-13T22:29:49.807680+0000","last_peered":"2023-07-13T22:29:49.807680+0000","last_clean":"2023-07-13T22:29:49.807680+0000","last_became_active":"2023-07-13T22:29:49.793625+0000","last_became_peered":"2023-07-13T22:29:49.793625+0000","last_unstale":"2023-07-13T22:29:49.807680+0000","last_undegraded":"2023-07-13T22:29:49.807680+0000","last_fullsized":"2023-07-13T22:29:49.807680+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:27:13.192607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.792887+0000","last_change":"2023-07-13T22:29:49.783979+0000","last_active":"2023-07-13T22:29:49.792887+0000","last_peered":"2023-07-13T22:29:49.792887+0000","last_clean":"2023-07-13T22:29:49.792887+0000","last_became_active":"2023-07-13T22:29:49.783653+0000","last_became_peered":"2023-07-13T22:29:49.783653+0000","last_unstale":"2023-07-13T22:29:49.792887+0000","last_undegraded":"2023-07-13T22:29:49.792887+0000","last_fullsized":"2023-07-13T22:29:49.792887+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:24:22.771277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786493+0000","last_change":"2023-07-13T22:29:49.792724+0000","last_active":"2023-07-13T22:29:50.786493+0000","last_peered":"2023-07-13T22:29:50.786493+0000","last_clean":"2023-07-13T22:29:50.786493+0000","last_became_active":"2023-07-13T22:29:49.792440+0000","last_became_peered":"2023-07-13T22:29:49.792440+0000","last_unstale":"2023-07-13T22:29:50.786493+0000","last_undegraded":"2023-07-13T22:29:50.786493+0000","last_fullsized":"2023-07-13T22:29:50.786493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:26:34.668070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.802665+0000","last_change":"2023-07-13T22:29:49.801142+0000","last_active":"2023-07-13T22:29:49.802665+0000","last_peered":"2023-07-13T22:29:49.802665+0000","last_clean":"2023-07-13T22:29:49.802665+0000","last_became_active":"2023-07-13T22:29:49.800623+0000","last_became_peered":"2023-07-13T22:29:49.800623+0000","last_unstale":"2023-07-13T22:29:49.802665+0000","last_undegraded":"2023-07-13T22:29:49.802665+0000","last_fullsized":"2023-07-13T22:29:49.802665+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:01:04.284945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063111+0000","last_change":"2023-07-13T22:29:49.789645+0000","last_active":"2023-07-13T22:29:51.063111+0000","last_peered":"2023-07-13T22:29:51.063111+0000","last_clean":"2023-07-13T22:29:51.063111+0000","last_became_active":"2023-07-13T22:29:49.789356+0000","last_became_peered":"2023-07-13T22:29:49.789356+0000","last_unstale":"2023-07-13T22:29:51.063111+0000","last_undegraded":"2023-07-13T22:29:51.063111+0000","last_fullsized":"2023-07-13T22:29:51.063111+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:59:41.514637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.803880+0000","last_change":"2023-07-13T22:29:49.800037+0000","last_active":"2023-07-13T22:29:49.803880+0000","last_peered":"2023-07-13T22:29:49.803880+0000","last_clean":"2023-07-13T22:29:49.803880+0000","last_became_active":"2023-07-13T22:29:49.799669+0000","last_became_peered":"2023-07-13T22:29:49.799669+0000","last_unstale":"2023-07-13T22:29:49.803880+0000","last_undegraded":"2023-07-13T22:29:49.803880+0000","last_fullsized":"2023-07-13T22:29:49.803880+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:36:20.387766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789042+0000","last_change":"2023-07-13T22:29:49.798221+0000","last_active":"2023-07-13T22:29:50.789042+0000","last_peered":"2023-07-13T22:29:50.789042+0000","last_clean":"2023-07-13T22:29:50.789042+0000","last_became_active":"2023-07-13T22:29:49.797861+0000","last_became_peered":"2023-07-13T22:29:49.797861+0000","last_unstale":"2023-07-13T22:29:50.789042+0000","last_undegraded":"2023-07-13T22:29:50.789042+0000","last_fullsized":"2023-07-13T22:29:50.789042+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:51.867950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.799868+0000","last_change":"2023-07-13T22:29:49.797379+0000","last_active":"2023-07-13T22:29:49.799868+0000","last_peered":"2023-07-13T22:29:49.799868+0000","last_clean":"2023-07-13T22:29:49.799868+0000","last_became_active":"2023-07-13T22:29:49.797086+0000","last_became_peered":"2023-07-13T22:29:49.797086+0000","last_unstale":"2023-07-13T22:29:49.799868+0000","last_undegraded":"2023-07-13T22:29:49.799868+0000","last_fullsized":"2023-07-13T22:29:49.799868+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:31.428060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787578+0000","last_change":"2023-07-13T22:29:49.793936+0000","last_active":"2023-07-13T22:29:50.787578+0000","last_peered":"2023-07-13T22:29:50.787578+0000","last_clean":"2023-07-13T22:29:50.787578+0000","last_became_active":"2023-07-13T22:29:49.793475+0000","last_became_peered":"2023-07-13T22:29:49.793475+0000","last_unstale":"2023-07-13T22:29:50.787578+0000","last_undegraded":"2023-07-13T22:29:50.787578+0000","last_fullsized":"2023-07-13T22:29:50.787578+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:02:58.850794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788184+0000","last_change":"2023-07-13T22:29:49.790916+0000","last_active":"2023-07-13T22:29:50.788184+0000","last_peered":"2023-07-13T22:29:50.788184+0000","last_clean":"2023-07-13T22:29:50.788184+0000","last_became_active":"2023-07-13T22:29:49.790631+0000","last_became_peered":"2023-07-13T22:29:49.790631+0000","last_unstale":"2023-07-13T22:29:50.788184+0000","last_undegraded":"2023-07-13T22:29:50.788184+0000","last_fullsized":"2023-07-13T22:29:50.788184+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:01:12.208248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063320+0000","last_change":"2023-07-13T22:29:49.784578+0000","last_active":"2023-07-13T22:29:51.063320+0000","last_peered":"2023-07-13T22:29:51.063320+0000","last_clean":"2023-07-13T22:29:51.063320+0000","last_became_active":"2023-07-13T22:29:49.784266+0000","last_became_peered":"2023-07-13T22:29:49.784266+0000","last_unstale":"2023-07-13T22:29:51.063320+0000","last_undegraded":"2023-07-13T22:29:51.063320+0000","last_fullsized":"2023-07-13T22:29:51.063320+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:55:11.376144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.802647+0000","last_change":"2023-07-13T22:29:49.800947+0000","last_active":"2023-07-13T22:29:49.802647+0000","last_peered":"2023-07-13T22:29:49.802647+0000","last_clean":"2023-07-13T22:29:49.802647+0000","last_became_active":"2023-07-13T22:29:49.800283+0000","last_became_peered":"2023-07-13T22:29:49.800283+0000","last_unstale":"2023-07-13T22:29:49.802647+0000","last_undegraded":"2023-07-13T22:29:49.802647+0000","last_fullsized":"2023-07-13T22:29:49.802647+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:08:25.577957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807830+0000","last_change":"2023-07-13T22:29:49.804148+0000","last_active":"2023-07-13T22:29:49.807830+0000","last_peered":"2023-07-13T22:29:49.807830+0000","last_clean":"2023-07-13T22:29:49.807830+0000","last_became_active":"2023-07-13T22:29:49.803790+0000","last_became_peered":"2023-07-13T22:29:49.803790+0000","last_unstale":"2023-07-13T22:29:49.807830+0000","last_undegraded":"2023-07-13T22:29:49.807830+0000","last_fullsized":"2023-07-13T22:29:49.807830+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:33.678966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063215+0000","last_change":"2023-07-13T22:29:49.779930+0000","last_active":"2023-07-13T22:29:51.063215+0000","last_peered":"2023-07-13T22:29:51.063215+0000","last_clean":"2023-07-13T22:29:51.063215+0000","last_became_active":"2023-07-13T22:29:49.779646+0000","last_became_peered":"2023-07-13T22:29:49.779646+0000","last_unstale":"2023-07-13T22:29:51.063215+0000","last_undegraded":"2023-07-13T22:29:51.063215+0000","last_fullsized":"2023-07-13T22:29:51.063215+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:28:52.629288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786543+0000","last_change":"2023-07-13T22:29:49.794111+0000","last_active":"2023-07-13T22:29:50.786543+0000","last_peered":"2023-07-13T22:29:50.786543+0000","last_clean":"2023-07-13T22:29:50.786543+0000","last_became_active":"2023-07-13T22:29:49.793713+0000","last_became_peered":"2023-07-13T22:29:49.793713+0000","last_unstale":"2023-07-13T22:29:50.786543+0000","last_undegraded":"2023-07-13T22:29:50.786543+0000","last_fullsized":"2023-07-13T22:29:50.786543+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:04:28.374231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787655+0000","last_change":"2023-07-13T22:29:49.782946+0000","last_active":"2023-07-13T22:29:50.787655+0000","last_peered":"2023-07-13T22:29:50.787655+0000","last_clean":"2023-07-13T22:29:50.787655+0000","last_became_active":"2023-07-13T22:29:49.782054+0000","last_became_peered":"2023-07-13T22:29:49.782054+0000","last_unstale":"2023-07-13T22:29:50.787655+0000","last_undegraded":"2023-07-13T22:29:50.787655+0000","last_fullsized":"2023-07-13T22:29:50.787655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:54:45.309189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788013+0000","last_change":"2023-07-13T22:29:49.793767+0000","last_active":"2023-07-13T22:29:50.788013+0000","last_peered":"2023-07-13T22:29:50.788013+0000","last_clean":"2023-07-13T22:29:50.788013+0000","last_became_active":"2023-07-13T22:29:49.793084+0000","last_became_peered":"2023-07-13T22:29:49.793084+0000","last_unstale":"2023-07-13T22:29:50.788013+0000","last_undegraded":"2023-07-13T22:29:50.788013+0000","last_fullsized":"2023-07-13T22:29:50.788013+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:17.182760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.803308+0000","last_change":"2023-07-13T22:29:49.800716+0000","last_active":"2023-07-13T22:29:49.803308+0000","last_peered":"2023-07-13T22:29:49.803308+0000","last_clean":"2023-07-13T22:29:49.803308+0000","last_became_active":"2023-07-13T22:29:49.800017+0000","last_became_peered":"2023-07-13T22:29:49.800017+0000","last_unstale":"2023-07-13T22:29:49.803308+0000","last_undegraded":"2023-07-13T22:29:49.803308+0000","last_fullsized":"2023-07-13T22:29:49.803308+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:10:13.698513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"111'4","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.809373+0000","last_change":"2023-07-13T22:29:48.779936+0000","last_active":"2023-07-13T22:29:49.809373+0000","last_peered":"2023-07-13T22:29:49.809373+0000","last_clean":"2023-07-13T22:29:49.809373+0000","last_became_active":"2023-07-13T22:29:48.779621+0000","last_became_peered":"2023-07-13T22:29:48.779621+0000","last_unstale":"2023-07-13T22:29:49.809373+0000","last_undegraded":"2023-07-13T22:29:49.809373+0000","last_fullsized":"2023-07-13T22:29:49.809373+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:14:44.131397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.808583+0000","last_change":"2023-07-13T22:29:49.797115+0000","last_active":"2023-07-13T22:29:49.808583+0000","last_peered":"2023-07-13T22:29:49.808583+0000","last_clean":"2023-07-13T22:29:49.808583+0000","last_became_active":"2023-07-13T22:29:49.796795+0000","last_became_peered":"2023-07-13T22:29:49.796795+0000","last_unstale":"2023-07-13T22:29:49.808583+0000","last_undegraded":"2023-07-13T22:29:49.808583+0000","last_fullsized":"2023-07-13T22:29:49.808583+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:56:23.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.806882+0000","last_change":"2023-07-13T22:29:48.773927+0000","last_active":"2023-07-13T22:29:49.806882+0000","last_peered":"2023-07-13T22:29:49.806882+0000","last_clean":"2023-07-13T22:29:49.806882+0000","last_became_active":"2023-07-13T22:29:48.773628+0000","last_became_peered":"2023-07-13T22:29:48.773628+0000","last_unstale":"2023-07-13T22:29:49.806882+0000","last_undegraded":"2023-07-13T22:29:49.806882+0000","last_fullsized":"2023-07-13T22:29:49.806882+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:03:50.291548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787799+0000","last_change":"2023-07-13T22:29:49.789313+0000","last_active":"2023-07-13T22:29:50.787799+0000","last_peered":"2023-07-13T22:29:50.787799+0000","last_clean":"2023-07-13T22:29:50.787799+0000","last_became_active":"2023-07-13T22:29:49.789003+0000","last_became_peered":"2023-07-13T22:29:49.789003+0000","last_unstale":"2023-07-13T22:29:50.787799+0000","last_undegraded":"2023-07-13T22:29:50.787799+0000","last_fullsized":"2023-07-13T22:29:50.787799+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:27:19.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.804339+0000","last_change":"2023-07-13T22:29:48.773431+0000","last_active":"2023-07-13T22:29:49.804339+0000","last_peered":"2023-07-13T22:29:49.804339+0000","last_clean":"2023-07-13T22:29:49.804339+0000","last_became_active":"2023-07-13T22:29:48.773088+0000","last_became_peered":"2023-07-13T22:29:48.773088+0000","last_unstale":"2023-07-13T22:29:49.804339+0000","last_undegraded":"2023-07-13T22:29:49.804339+0000","last_fullsized":"2023-07-13T22:29:49.804339+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:38:41.991340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.803850+0000","last_change":"2023-07-13T22:29:49.799902+0000","last_active":"2023-07-13T22:29:49.803850+0000","last_peered":"2023-07-13T22:29:49.803850+0000","last_clean":"2023-07-13T22:29:49.803850+0000","last_became_active":"2023-07-13T22:29:49.799607+0000","last_became_peered":"2023-07-13T22:29:49.799607+0000","last_unstale":"2023-07-13T22:29:49.803850+0000","last_undegraded":"2023-07-13T22:29:49.803850+0000","last_fullsized":"2023-07-13T22:29:49.803850+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:48:37.285989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807115+0000","last_change":"2023-07-13T22:29:48.781172+0000","last_active":"2023-07-13T22:29:49.807115+0000","last_peered":"2023-07-13T22:29:49.807115+0000","last_clean":"2023-07-13T22:29:49.807115+0000","last_became_active":"2023-07-13T22:29:48.780720+0000","last_became_peered":"2023-07-13T22:29:48.780720+0000","last_unstale":"2023-07-13T22:29:49.807115+0000","last_undegraded":"2023-07-13T22:29:49.807115+0000","last_fullsized":"2023-07-13T22:29:49.807115+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:46:33.540025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.798036+0000","last_change":"2023-07-13T22:29:49.796710+0000","last_active":"2023-07-13T22:29:49.798036+0000","last_peered":"2023-07-13T22:29:49.798036+0000","last_clean":"2023-07-13T22:29:49.798036+0000","last_became_active":"2023-07-13T22:29:49.796392+0000","last_became_peered":"2023-07-13T22:29:49.796392+0000","last_unstale":"2023-07-13T22:29:49.798036+0000","last_undegraded":"2023-07-13T22:29:49.798036+0000","last_fullsized":"2023-07-13T22:29:49.798036+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:19:25.312547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"111'1","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788951+0000","last_change":"2023-07-13T22:29:48.771882+0000","last_active":"2023-07-13T22:29:50.788951+0000","last_peered":"2023-07-13T22:29:50.788951+0000","last_clean":"2023-07-13T22:29:50.788951+0000","last_became_active":"2023-07-13T22:29:48.771576+0000","last_became_peered":"2023-07-13T22:29:48.771576+0000","last_unstale":"2023-07-13T22:29:50.788951+0000","last_undegraded":"2023-07-13T22:29:50.788951+0000","last_fullsized":"2023-07-13T22:29:50.788951+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:20:42.770389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788671+0000","last_change":"2023-07-13T22:29:49.796609+0000","last_active":"2023-07-13T22:29:50.788671+0000","last_peered":"2023-07-13T22:29:50.788671+0000","last_clean":"2023-07-13T22:29:50.788671+0000","last_became_active":"2023-07-13T22:29:49.796305+0000","last_became_peered":"2023-07-13T22:29:49.796305+0000","last_unstale":"2023-07-13T22:29:50.788671+0000","last_undegraded":"2023-07-13T22:29:50.788671+0000","last_fullsized":"2023-07-13T22:29:50.788671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:33.678966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.810161+0000","last_change":"2023-07-13T22:29:48.782595+0000","last_active":"2023-07-13T22:29:49.810161+0000","last_peered":"2023-07-13T22:29:49.810161+0000","last_clean":"2023-07-13T22:29:49.810161+0000","last_became_active":"2023-07-13T22:29:48.782183+0000","last_became_peered":"2023-07-13T22:29:48.782183+0000","last_unstale":"2023-07-13T22:29:49.810161+0000","last_undegraded":"2023-07-13T22:29:49.810161+0000","last_fullsized":"2023-07-13T22:29:49.810161+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:41:14.764896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787011+0000","last_change":"2023-07-13T22:29:49.786046+0000","last_active":"2023-07-13T22:29:50.787011+0000","last_peered":"2023-07-13T22:29:50.787011+0000","last_clean":"2023-07-13T22:29:50.787011+0000","last_became_active":"2023-07-13T22:29:49.785718+0000","last_became_peered":"2023-07-13T22:29:49.785718+0000","last_unstale":"2023-07-13T22:29:50.787011+0000","last_undegraded":"2023-07-13T22:29:50.787011+0000","last_fullsized":"2023-07-13T22:29:50.787011+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:47.408965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787822+0000","last_change":"2023-07-13T22:29:48.778809+0000","last_active":"2023-07-13T22:29:50.787822+0000","last_peered":"2023-07-13T22:29:50.787822+0000","last_clean":"2023-07-13T22:29:50.787822+0000","last_became_active":"2023-07-13T22:29:48.778515+0000","last_became_peered":"2023-07-13T22:29:48.778515+0000","last_unstale":"2023-07-13T22:29:50.787822+0000","last_undegraded":"2023-07-13T22:29:50.787822+0000","last_fullsized":"2023-07-13T22:29:50.787822+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:55:46.332348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.790764+0000","last_change":"2023-07-13T22:29:49.784589+0000","last_active":"2023-07-13T22:29:49.790764+0000","last_peered":"2023-07-13T22:29:49.790764+0000","last_clean":"2023-07-13T22:29:49.790764+0000","last_became_active":"2023-07-13T22:29:49.784262+0000","last_became_peered":"2023-07-13T22:29:49.784262+0000","last_unstale":"2023-07-13T22:29:49.790764+0000","last_undegraded":"2023-07-13T22:29:49.790764+0000","last_fullsized":"2023-07-13T22:29:49.790764+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:33:43.779422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.781447+0000","last_change":"2023-07-13T22:29:48.783299+0000","last_active":"2023-07-13T22:29:49.781447+0000","last_peered":"2023-07-13T22:29:49.781447+0000","last_clean":"2023-07-13T22:29:49.781447+0000","last_became_active":"2023-07-13T22:29:48.782979+0000","last_became_peered":"2023-07-13T22:29:48.782979+0000","last_unstale":"2023-07-13T22:29:49.781447+0000","last_undegraded":"2023-07-13T22:29:49.781447+0000","last_fullsized":"2023-07-13T22:29:49.781447+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:38:16.683065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787729+0000","last_change":"2023-07-13T22:29:49.795458+0000","last_active":"2023-07-13T22:29:50.787729+0000","last_peered":"2023-07-13T22:29:50.787729+0000","last_clean":"2023-07-13T22:29:50.787729+0000","last_became_active":"2023-07-13T22:29:49.795163+0000","last_became_peered":"2023-07-13T22:29:49.795163+0000","last_unstale":"2023-07-13T22:29:50.787729+0000","last_undegraded":"2023-07-13T22:29:50.787729+0000","last_fullsized":"2023-07-13T22:29:50.787729+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:17:55.654447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.811851+0000","last_change":"2023-07-13T22:29:48.775741+0000","last_active":"2023-07-13T22:29:49.811851+0000","last_peered":"2023-07-13T22:29:49.811851+0000","last_clean":"2023-07-13T22:29:49.811851+0000","last_became_active":"2023-07-13T22:29:48.775439+0000","last_became_peered":"2023-07-13T22:29:48.775439+0000","last_unstale":"2023-07-13T22:29:49.811851+0000","last_undegraded":"2023-07-13T22:29:49.811851+0000","last_fullsized":"2023-07-13T22:29:49.811851+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:12:58.805471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789363+0000","last_change":"2023-07-13T22:29:49.797753+0000","last_active":"2023-07-13T22:29:50.789363+0000","last_peered":"2023-07-13T22:29:50.789363+0000","last_clean":"2023-07-13T22:29:50.789363+0000","last_became_active":"2023-07-13T22:29:49.797466+0000","last_became_peered":"2023-07-13T22:29:49.797466+0000","last_unstale":"2023-07-13T22:29:50.789363+0000","last_undegraded":"2023-07-13T22:29:50.789363+0000","last_fullsized":"2023-07-13T22:29:50.789363+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:07:01.435082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063347+0000","last_change":"2023-07-13T22:29:48.777146+0000","last_active":"2023-07-13T22:29:51.063347+0000","last_peered":"2023-07-13T22:29:51.063347+0000","last_clean":"2023-07-13T22:29:51.063347+0000","last_became_active":"2023-07-13T22:29:48.776885+0000","last_became_peered":"2023-07-13T22:29:48.776885+0000","last_unstale":"2023-07-13T22:29:51.063347+0000","last_undegraded":"2023-07-13T22:29:51.063347+0000","last_fullsized":"2023-07-13T22:29:51.063347+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:17:02.197580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807750+0000","last_change":"2023-07-13T22:29:49.798859+0000","last_active":"2023-07-13T22:29:49.807750+0000","last_peered":"2023-07-13T22:29:49.807750+0000","last_clean":"2023-07-13T22:29:49.807750+0000","last_became_active":"2023-07-13T22:29:49.798343+0000","last_became_peered":"2023-07-13T22:29:49.798343+0000","last_unstale":"2023-07-13T22:29:49.807750+0000","last_undegraded":"2023-07-13T22:29:49.807750+0000","last_fullsized":"2023-07-13T22:29:49.807750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:56:05.838768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.804755+0000","last_change":"2023-07-13T22:29:48.779363+0000","last_active":"2023-07-13T22:29:49.804755+0000","last_peered":"2023-07-13T22:29:49.804755+0000","last_clean":"2023-07-13T22:29:49.804755+0000","last_became_active":"2023-07-13T22:29:48.779075+0000","last_became_peered":"2023-07-13T22:29:48.779075+0000","last_unstale":"2023-07-13T22:29:49.804755+0000","last_undegraded":"2023-07-13T22:29:49.804755+0000","last_fullsized":"2023-07-13T22:29:49.804755+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:38:25.492994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788348+0000","last_change":"2023-07-13T22:29:49.792033+0000","last_active":"2023-07-13T22:29:50.788348+0000","last_peered":"2023-07-13T22:29:50.788348+0000","last_clean":"2023-07-13T22:29:50.788348+0000","last_became_active":"2023-07-13T22:29:49.791767+0000","last_became_peered":"2023-07-13T22:29:49.791767+0000","last_unstale":"2023-07-13T22:29:50.788348+0000","last_undegraded":"2023-07-13T22:29:50.788348+0000","last_fullsized":"2023-07-13T22:29:50.788348+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:17:39.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.786697+0000","last_change":"2023-07-13T22:29:48.783608+0000","last_active":"2023-07-13T22:29:49.786697+0000","last_peered":"2023-07-13T22:29:49.786697+0000","last_clean":"2023-07-13T22:29:49.786697+0000","last_became_active":"2023-07-13T22:29:48.783263+0000","last_became_peered":"2023-07-13T22:29:48.783263+0000","last_unstale":"2023-07-13T22:29:49.786697+0000","last_undegraded":"2023-07-13T22:29:49.786697+0000","last_fullsized":"2023-07-13T22:29:49.786697+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:03:45.506243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807704+0000","last_change":"2023-07-13T22:29:49.796436+0000","last_active":"2023-07-13T22:29:49.807704+0000","last_peered":"2023-07-13T22:29:49.807704+0000","last_clean":"2023-07-13T22:29:49.807704+0000","last_became_active":"2023-07-13T22:29:49.796073+0000","last_became_peered":"2023-07-13T22:29:49.796073+0000","last_unstale":"2023-07-13T22:29:49.807704+0000","last_undegraded":"2023-07-13T22:29:49.807704+0000","last_fullsized":"2023-07-13T22:29:49.807704+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:59:13.912613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.808192+0000","last_change":"2023-07-13T22:29:48.774426+0000","last_active":"2023-07-13T22:29:49.808192+0000","last_peered":"2023-07-13T22:29:49.808192+0000","last_clean":"2023-07-13T22:29:49.808192+0000","last_became_active":"2023-07-13T22:29:48.774019+0000","last_became_peered":"2023-07-13T22:29:48.774019+0000","last_unstale":"2023-07-13T22:29:49.808192+0000","last_undegraded":"2023-07-13T22:29:49.808192+0000","last_fullsized":"2023-07-13T22:29:49.808192+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:08:24.572500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786029+0000","last_change":"2023-07-13T22:29:49.786513+0000","last_active":"2023-07-13T22:29:50.786029+0000","last_peered":"2023-07-13T22:29:50.786029+0000","last_clean":"2023-07-13T22:29:50.786029+0000","last_became_active":"2023-07-13T22:29:49.786209+0000","last_became_peered":"2023-07-13T22:29:49.786209+0000","last_unstale":"2023-07-13T22:29:50.786029+0000","last_undegraded":"2023-07-13T22:29:50.786029+0000","last_fullsized":"2023-07-13T22:29:50.786029+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:03:56.029937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.785058+0000","last_change":"2023-07-13T22:29:48.772542+0000","last_active":"2023-07-13T22:29:49.785058+0000","last_peered":"2023-07-13T22:29:49.785058+0000","last_clean":"2023-07-13T22:29:49.785058+0000","last_became_active":"2023-07-13T22:29:48.772171+0000","last_became_peered":"2023-07-13T22:29:48.772171+0000","last_unstale":"2023-07-13T22:29:49.785058+0000","last_undegraded":"2023-07-13T22:29:49.785058+0000","last_fullsized":"2023-07-13T22:29:49.785058+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:47:00.470196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786820+0000","last_change":"2023-07-13T22:29:49.797636+0000","last_active":"2023-07-13T22:29:50.786820+0000","last_peered":"2023-07-13T22:29:50.786820+0000","last_clean":"2023-07-13T22:29:50.786820+0000","last_became_active":"2023-07-13T22:29:49.797119+0000","last_became_peered":"2023-07-13T22:29:49.797119+0000","last_unstale":"2023-07-13T22:29:50.786820+0000","last_undegraded":"2023-07-13T22:29:50.786820+0000","last_fullsized":"2023-07-13T22:29:50.786820+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:31:02.830181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786277+0000","last_change":"2023-07-13T22:29:48.781731+0000","last_active":"2023-07-13T22:29:50.786277+0000","last_peered":"2023-07-13T22:29:50.786277+0000","last_clean":"2023-07-13T22:29:50.786277+0000","last_became_active":"2023-07-13T22:29:48.781328+0000","last_became_peered":"2023-07-13T22:29:48.781328+0000","last_unstale":"2023-07-13T22:29:50.786277+0000","last_undegraded":"2023-07-13T22:29:50.786277+0000","last_fullsized":"2023-07-13T22:29:50.786277+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:34:42.755132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787855+0000","last_change":"2023-07-13T22:29:49.788964+0000","last_active":"2023-07-13T22:29:50.787855+0000","last_peered":"2023-07-13T22:29:50.787855+0000","last_clean":"2023-07-13T22:29:50.787855+0000","last_became_active":"2023-07-13T22:29:49.788667+0000","last_became_peered":"2023-07-13T22:29:49.788667+0000","last_unstale":"2023-07-13T22:29:50.787855+0000","last_undegraded":"2023-07-13T22:29:50.787855+0000","last_fullsized":"2023-07-13T22:29:50.787855+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:27:58.321047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.786023+0000","last_change":"2023-07-13T22:29:48.773107+0000","last_active":"2023-07-13T22:29:49.786023+0000","last_peered":"2023-07-13T22:29:49.786023+0000","last_clean":"2023-07-13T22:29:49.786023+0000","last_became_active":"2023-07-13T22:29:48.772765+0000","last_became_peered":"2023-07-13T22:29:48.772765+0000","last_unstale":"2023-07-13T22:29:49.786023+0000","last_undegraded":"2023-07-13T22:29:49.786023+0000","last_fullsized":"2023-07-13T22:29:49.786023+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:50:40.128049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.807502+0000","last_change":"2023-07-13T22:29:49.794647+0000","last_active":"2023-07-13T22:29:49.807502+0000","last_peered":"2023-07-13T22:29:49.807502+0000","last_clean":"2023-07-13T22:29:49.807502+0000","last_became_active":"2023-07-13T22:29:49.794335+0000","last_became_peered":"2023-07-13T22:29:49.794335+0000","last_unstale":"2023-07-13T22:29:49.807502+0000","last_undegraded":"2023-07-13T22:29:49.807502+0000","last_fullsized":"2023-07-13T22:29:49.807502+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:20:33.332719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.810553+0000","last_change":"2023-07-13T22:29:48.784127+0000","last_active":"2023-07-13T22:29:49.810553+0000","last_peered":"2023-07-13T22:29:49.810553+0000","last_clean":"2023-07-13T22:29:49.810553+0000","last_became_active":"2023-07-13T22:29:48.783872+0000","last_became_peered":"2023-07-13T22:29:48.783872+0000","last_unstale":"2023-07-13T22:29:49.810553+0000","last_undegraded":"2023-07-13T22:29:49.810553+0000","last_fullsized":"2023-07-13T22:29:49.810553+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:46:04.647997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.798067+0000","last_change":"2023-07-13T22:29:49.796734+0000","last_active":"2023-07-13T22:29:49.798067+0000","last_peered":"2023-07-13T22:29:49.798067+0000","last_clean":"2023-07-13T22:29:49.798067+0000","last_became_active":"2023-07-13T22:29:49.796451+0000","last_became_peered":"2023-07-13T22:29:49.796451+0000","last_unstale":"2023-07-13T22:29:49.798067+0000","last_undegraded":"2023-07-13T22:29:49.798067+0000","last_fullsized":"2023-07-13T22:29:49.798067+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:53:58.236870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.783308+0000","last_change":"2023-07-13T22:29:48.783720+0000","last_active":"2023-07-13T22:29:49.783308+0000","last_peered":"2023-07-13T22:29:49.783308+0000","last_clean":"2023-07-13T22:29:49.783308+0000","last_became_active":"2023-07-13T22:29:48.783466+0000","last_became_peered":"2023-07-13T22:29:48.783466+0000","last_unstale":"2023-07-13T22:29:49.783308+0000","last_undegraded":"2023-07-13T22:29:49.783308+0000","last_fullsized":"2023-07-13T22:29:49.783308+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:09:18.148759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.798661+0000","last_change":"2023-07-13T22:29:49.794288+0000","last_active":"2023-07-13T22:29:49.798661+0000","last_peered":"2023-07-13T22:29:49.798661+0000","last_clean":"2023-07-13T22:29:49.798661+0000","last_became_active":"2023-07-13T22:29:49.793968+0000","last_became_peered":"2023-07-13T22:29:49.793968+0000","last_unstale":"2023-07-13T22:29:49.798661+0000","last_undegraded":"2023-07-13T22:29:49.798661+0000","last_fullsized":"2023-07-13T22:29:49.798661+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:56:30.198141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.786163+0000","last_change":"2023-07-13T22:29:48.786075+0000","last_active":"2023-07-13T22:29:49.786163+0000","last_peered":"2023-07-13T22:29:49.786163+0000","last_clean":"2023-07-13T22:29:49.786163+0000","last_became_active":"2023-07-13T22:29:48.785699+0000","last_became_peered":"2023-07-13T22:29:48.785699+0000","last_unstale":"2023-07-13T22:29:49.786163+0000","last_undegraded":"2023-07-13T22:29:49.786163+0000","last_fullsized":"2023-07-13T22:29:49.786163+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:42:27.115570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063305+0000","last_change":"2023-07-13T22:29:49.783316+0000","last_active":"2023-07-13T22:29:51.063305+0000","last_peered":"2023-07-13T22:29:51.063305+0000","last_clean":"2023-07-13T22:29:51.063305+0000","last_became_active":"2023-07-13T22:29:49.782848+0000","last_became_peered":"2023-07-13T22:29:49.782848+0000","last_unstale":"2023-07-13T22:29:51.063305+0000","last_undegraded":"2023-07-13T22:29:51.063305+0000","last_fullsized":"2023-07-13T22:29:51.063305+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:46:39.038714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.788155+0000","last_change":"2023-07-13T22:29:48.783243+0000","last_active":"2023-07-13T22:29:49.788155+0000","last_peered":"2023-07-13T22:29:49.788155+0000","last_clean":"2023-07-13T22:29:49.788155+0000","last_became_active":"2023-07-13T22:29:48.782864+0000","last_became_peered":"2023-07-13T22:29:48.782864+0000","last_unstale":"2023-07-13T22:29:49.788155+0000","last_undegraded":"2023-07-13T22:29:49.788155+0000","last_fullsized":"2023-07-13T22:29:49.788155+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:39:11.342728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.804883+0000","last_change":"2023-07-13T22:29:49.792102+0000","last_active":"2023-07-13T22:29:49.804883+0000","last_peered":"2023-07-13T22:29:49.804883+0000","last_clean":"2023-07-13T22:29:49.804883+0000","last_became_active":"2023-07-13T22:29:49.791804+0000","last_became_peered":"2023-07-13T22:29:49.791804+0000","last_unstale":"2023-07-13T22:29:49.804883+0000","last_undegraded":"2023-07-13T22:29:49.804883+0000","last_fullsized":"2023-07-13T22:29:49.804883+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:01:04.284945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789031+0000","last_change":"2023-07-13T22:29:48.772522+0000","last_active":"2023-07-13T22:29:50.789031+0000","last_peered":"2023-07-13T22:29:50.789031+0000","last_clean":"2023-07-13T22:29:50.789031+0000","last_became_active":"2023-07-13T22:29:48.772196+0000","last_became_peered":"2023-07-13T22:29:48.772196+0000","last_unstale":"2023-07-13T22:29:50.789031+0000","last_undegraded":"2023-07-13T22:29:50.789031+0000","last_fullsized":"2023-07-13T22:29:50.789031+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:04:07.897982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786037+0000","last_change":"2023-07-13T22:29:49.797544+0000","last_active":"2023-07-13T22:29:50.786037+0000","last_peered":"2023-07-13T22:29:50.786037+0000","last_clean":"2023-07-13T22:29:50.786037+0000","last_became_active":"2023-07-13T22:29:49.797249+0000","last_became_peered":"2023-07-13T22:29:49.797249+0000","last_unstale":"2023-07-13T22:29:50.786037+0000","last_undegraded":"2023-07-13T22:29:50.786037+0000","last_fullsized":"2023-07-13T22:29:50.786037+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:03:28.229578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.789211+0000","last_change":"2023-07-13T22:29:48.773717+0000","last_active":"2023-07-13T22:29:50.789211+0000","last_peered":"2023-07-13T22:29:50.789211+0000","last_clean":"2023-07-13T22:29:50.789211+0000","last_became_active":"2023-07-13T22:29:48.773429+0000","last_became_peered":"2023-07-13T22:29:48.773429+0000","last_unstale":"2023-07-13T22:29:50.789211+0000","last_undegraded":"2023-07-13T22:29:50.789211+0000","last_fullsized":"2023-07-13T22:29:50.789211+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:30:06.481851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.806479+0000","last_change":"2023-07-13T22:29:49.806018+0000","last_active":"2023-07-13T22:29:49.806479+0000","last_peered":"2023-07-13T22:29:49.806479+0000","last_clean":"2023-07-13T22:29:49.806479+0000","last_became_active":"2023-07-13T22:29:49.805708+0000","last_became_peered":"2023-07-13T22:29:49.805708+0000","last_unstale":"2023-07-13T22:29:49.806479+0000","last_undegraded":"2023-07-13T22:29:49.806479+0000","last_fullsized":"2023-07-13T22:29:49.806479+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:49:40.598006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.780868+0000","last_change":"2023-07-13T22:29:48.782641+0000","last_active":"2023-07-13T22:29:49.780868+0000","last_peered":"2023-07-13T22:29:49.780868+0000","last_clean":"2023-07-13T22:29:49.780868+0000","last_became_active":"2023-07-13T22:29:48.782163+0000","last_became_peered":"2023-07-13T22:29:48.782163+0000","last_unstale":"2023-07-13T22:29:49.780868+0000","last_undegraded":"2023-07-13T22:29:49.780868+0000","last_fullsized":"2023-07-13T22:29:49.780868+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:33:15.277248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.794750+0000","last_change":"2023-07-13T22:29:49.792741+0000","last_active":"2023-07-13T22:29:49.794750+0000","last_peered":"2023-07-13T22:29:49.794750+0000","last_clean":"2023-07-13T22:29:49.794750+0000","last_became_active":"2023-07-13T22:29:49.792469+0000","last_became_peered":"2023-07-13T22:29:49.792469+0000","last_unstale":"2023-07-13T22:29:49.794750+0000","last_undegraded":"2023-07-13T22:29:49.794750+0000","last_fullsized":"2023-07-13T22:29:49.794750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:37:02.123624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.787219+0000","last_change":"2023-07-13T22:29:48.779674+0000","last_active":"2023-07-13T22:29:49.787219+0000","last_peered":"2023-07-13T22:29:49.787219+0000","last_clean":"2023-07-13T22:29:49.787219+0000","last_became_active":"2023-07-13T22:29:48.779401+0000","last_became_peered":"2023-07-13T22:29:48.779401+0000","last_unstale":"2023-07-13T22:29:49.787219+0000","last_undegraded":"2023-07-13T22:29:49.787219+0000","last_fullsized":"2023-07-13T22:29:49.787219+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:23:03.083682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786644+0000","last_change":"2023-07-13T22:29:49.798098+0000","last_active":"2023-07-13T22:29:50.786644+0000","last_peered":"2023-07-13T22:29:50.786644+0000","last_clean":"2023-07-13T22:29:50.786644+0000","last_became_active":"2023-07-13T22:29:49.797821+0000","last_became_peered":"2023-07-13T22:29:49.797821+0000","last_unstale":"2023-07-13T22:29:50.786644+0000","last_undegraded":"2023-07-13T22:29:50.786644+0000","last_fullsized":"2023-07-13T22:29:50.786644+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:31:40.571752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"111'3","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788118+0000","last_change":"2023-07-13T22:29:48.777158+0000","last_active":"2023-07-13T22:29:50.788118+0000","last_peered":"2023-07-13T22:29:50.788118+0000","last_clean":"2023-07-13T22:29:50.788118+0000","last_became_active":"2023-07-13T22:29:48.776838+0000","last_became_peered":"2023-07-13T22:29:48.776838+0000","last_unstale":"2023-07-13T22:29:50.788118+0000","last_undegraded":"2023-07-13T22:29:50.788118+0000","last_fullsized":"2023-07-13T22:29:50.788118+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:41:30.299089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":183,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.790101+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:29:50.790101+0000","last_peered":"2023-07-13T22:29:50.790101+0000","last_clean":"2023-07-13T22:29:50.790101+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:29:50.790101+0000","last_undegraded":"2023-07-13T22:29:50.790101+0000","last_fullsized":"2023-07-13T22:29:50.790101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:43:11.431505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788334+0000","last_change":"2023-07-13T22:29:49.793695+0000","last_active":"2023-07-13T22:29:50.788334+0000","last_peered":"2023-07-13T22:29:50.788334+0000","last_clean":"2023-07-13T22:29:50.788334+0000","last_became_active":"2023-07-13T22:29:49.793368+0000","last_became_peered":"2023-07-13T22:29:49.793368+0000","last_unstale":"2023-07-13T22:29:50.788334+0000","last_undegraded":"2023-07-13T22:29:50.788334+0000","last_fullsized":"2023-07-13T22:29:50.788334+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:09:19.154215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"111'6","reported_seq":19,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786322+0000","last_change":"2023-07-13T22:29:48.780737+0000","last_active":"2023-07-13T22:29:50.786322+0000","last_peered":"2023-07-13T22:29:50.786322+0000","last_clean":"2023-07-13T22:29:50.786322+0000","last_became_active":"2023-07-13T22:29:48.779924+0000","last_became_peered":"2023-07-13T22:29:48.779924+0000","last_unstale":"2023-07-13T22:29:50.786322+0000","last_undegraded":"2023-07-13T22:29:50.786322+0000","last_fullsized":"2023-07-13T22:29:50.786322+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:22:48.111167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":175,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786360+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:29:50.786360+0000","last_peered":"2023-07-13T22:29:50.786360+0000","last_clean":"2023-07-13T22:29:50.786360+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:29:50.786360+0000","last_undegraded":"2023-07-13T22:29:50.786360+0000","last_fullsized":"2023-07-13T22:29:50.786360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:48:07.461571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.802814+0000","last_change":"2023-07-13T22:29:49.798814+0000","last_active":"2023-07-13T22:29:49.802814+0000","last_peered":"2023-07-13T22:29:49.802814+0000","last_clean":"2023-07-13T22:29:49.802814+0000","last_became_active":"2023-07-13T22:29:49.797694+0000","last_became_peered":"2023-07-13T22:29:49.797694+0000","last_unstale":"2023-07-13T22:29:49.802814+0000","last_undegraded":"2023-07-13T22:29:49.802814+0000","last_fullsized":"2023-07-13T22:29:49.802814+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:00:50.825775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"111'2","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786845+0000","last_change":"2023-07-13T22:29:48.775117+0000","last_active":"2023-07-13T22:29:50.786845+0000","last_peered":"2023-07-13T22:29:50.786845+0000","last_clean":"2023-07-13T22:29:50.786845+0000","last_became_active":"2023-07-13T22:29:48.774709+0000","last_became_peered":"2023-07-13T22:29:48.774709+0000","last_unstale":"2023-07-13T22:29:50.786845+0000","last_undegraded":"2023-07-13T22:29:50.786845+0000","last_fullsized":"2023-07-13T22:29:50.786845+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:40:08.500902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":187,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.780304+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:29:49.780304+0000","last_peered":"2023-07-13T22:29:49.780304+0000","last_clean":"2023-07-13T22:29:49.780304+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:29:49.780304+0000","last_undegraded":"2023-07-13T22:29:49.780304+0000","last_fullsized":"2023-07-13T22:29:49.780304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:57:57.150009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":321,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.786847+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:29:49.786847+0000","last_peered":"2023-07-13T22:29:49.786847+0000","last_clean":"2023-07-13T22:29:49.786847+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:29:49.786847+0000","last_undegraded":"2023-07-13T22:29:49.786847+0000","last_fullsized":"2023-07-13T22:29:49.786847+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:28:10.176414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.802896+0000","last_change":"2023-07-13T22:29:49.800414+0000","last_active":"2023-07-13T22:29:49.802896+0000","last_peered":"2023-07-13T22:29:49.802896+0000","last_clean":"2023-07-13T22:29:49.802896+0000","last_became_active":"2023-07-13T22:29:49.800094+0000","last_became_peered":"2023-07-13T22:29:49.800094+0000","last_unstale":"2023-07-13T22:29:49.802896+0000","last_undegraded":"2023-07-13T22:29:49.802896+0000","last_fullsized":"2023-07-13T22:29:49.802896+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:21:33.884645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786943+0000","last_change":"2023-07-13T22:29:48.774258+0000","last_active":"2023-07-13T22:29:50.786943+0000","last_peered":"2023-07-13T22:29:50.786943+0000","last_clean":"2023-07-13T22:29:50.786943+0000","last_became_active":"2023-07-13T22:29:48.773869+0000","last_became_peered":"2023-07-13T22:29:48.773869+0000","last_unstale":"2023-07-13T22:29:50.786943+0000","last_undegraded":"2023-07-13T22:29:50.786943+0000","last_fullsized":"2023-07-13T22:29:50.786943+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:53:16.478797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":173,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.785163+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:29:49.785163+0000","last_peered":"2023-07-13T22:29:49.785163+0000","last_clean":"2023-07-13T22:29:49.785163+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:29:49.785163+0000","last_undegraded":"2023-07-13T22:29:49.785163+0000","last_fullsized":"2023-07-13T22:29:49.785163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:13:05.286337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.803897+0000","last_change":"2023-07-13T22:29:49.800051+0000","last_active":"2023-07-13T22:29:49.803897+0000","last_peered":"2023-07-13T22:29:49.803897+0000","last_clean":"2023-07-13T22:29:49.803897+0000","last_became_active":"2023-07-13T22:29:49.799721+0000","last_became_peered":"2023-07-13T22:29:49.799721+0000","last_unstale":"2023-07-13T22:29:49.803897+0000","last_undegraded":"2023-07-13T22:29:49.803897+0000","last_fullsized":"2023-07-13T22:29:49.803897+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:45:15.768466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"111'8","reported_seq":21,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786073+0000","last_change":"2023-07-13T22:29:48.777653+0000","last_active":"2023-07-13T22:29:50.786073+0000","last_peered":"2023-07-13T22:29:50.786073+0000","last_clean":"2023-07-13T22:29:50.786073+0000","last_became_active":"2023-07-13T22:29:48.774359+0000","last_became_peered":"2023-07-13T22:29:48.774359+0000","last_unstale":"2023-07-13T22:29:50.786073+0000","last_undegraded":"2023-07-13T22:29:50.786073+0000","last_fullsized":"2023-07-13T22:29:50.786073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:51:46.039327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":173,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.787106+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:29:49.787106+0000","last_peered":"2023-07-13T22:29:49.787106+0000","last_clean":"2023-07-13T22:29:49.787106+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:29:49.787106+0000","last_undegraded":"2023-07-13T22:29:49.787106+0000","last_fullsized":"2023-07-13T22:29:49.787106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:32:35.857901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:51.063028+0000","last_change":"2023-07-13T22:29:49.790468+0000","last_active":"2023-07-13T22:29:51.063028+0000","last_peered":"2023-07-13T22:29:51.063028+0000","last_clean":"2023-07-13T22:29:51.063028+0000","last_became_active":"2023-07-13T22:29:49.790195+0000","last_became_peered":"2023-07-13T22:29:49.790195+0000","last_unstale":"2023-07-13T22:29:51.063028+0000","last_undegraded":"2023-07-13T22:29:51.063028+0000","last_fullsized":"2023-07-13T22:29:51.063028+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:42:40.974854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.786228+0000","last_change":"2023-07-13T22:29:48.772838+0000","last_active":"2023-07-13T22:29:50.786228+0000","last_peered":"2023-07-13T22:29:50.786228+0000","last_clean":"2023-07-13T22:29:50.786228+0000","last_became_active":"2023-07-13T22:29:48.772522+0000","last_became_peered":"2023-07-13T22:29:48.772522+0000","last_unstale":"2023-07-13T22:29:50.786228+0000","last_undegraded":"2023-07-13T22:29:50.786228+0000","last_fullsized":"2023-07-13T22:29:50.786228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:44:29.136805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":197,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.788011+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:29:50.788011+0000","last_peered":"2023-07-13T22:29:50.788011+0000","last_clean":"2023-07-13T22:29:50.788011+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:29:50.788011+0000","last_undegraded":"2023-07-13T22:29:50.788011+0000","last_fullsized":"2023-07-13T22:29:50.788011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:05:43.089032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787947+0000","last_change":"2023-07-13T22:29:49.781518+0000","last_active":"2023-07-13T22:29:50.787947+0000","last_peered":"2023-07-13T22:29:50.787947+0000","last_clean":"2023-07-13T22:29:50.787947+0000","last_became_active":"2023-07-13T22:29:49.781168+0000","last_became_peered":"2023-07-13T22:29:49.781168+0000","last_unstale":"2023-07-13T22:29:50.787947+0000","last_undegraded":"2023-07-13T22:29:50.787947+0000","last_fullsized":"2023-07-13T22:29:50.787947+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:51:54.745934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-07-13T22:29:50.787945+0000","last_change":"2023-07-13T22:29:48.774310+0000","last_active":"2023-07-13T22:29:50.787945+0000","last_peered":"2023-07-13T22:29:50.787945+0000","last_clean":"2023-07-13T22:29:50.787945+0000","last_became_active":"2023-07-13T22:29:48.773891+0000","last_became_peered":"2023-07-13T22:29:48.773891+0000","last_unstale":"2023-07-13T22:29:50.787945+0000","last_undegraded":"2023-07-13T22:29:50.787945+0000","last_fullsized":"2023-07-13T22:29:50.787945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:50:34.648462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":169,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.783408+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:29:49.783408+0000","last_peered":"2023-07-13T22:29:49.783408+0000","last_clean":"2023-07-13T22:29:49.783408+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:29:49.783408+0000","last_undegraded":"2023-07-13T22:29:49.783408+0000","last_fullsized":"2023-07-13T22:29:49.783408+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:08:37.720940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.801247+0000","last_change":"2023-07-13T22:29:49.797225+0000","last_active":"2023-07-13T22:29:49.801247+0000","last_peered":"2023-07-13T22:29:49.801247+0000","last_clean":"2023-07-13T22:29:49.801247+0000","last_became_active":"2023-07-13T22:29:49.796947+0000","last_became_peered":"2023-07-13T22:29:49.796947+0000","last_unstale":"2023-07-13T22:29:49.801247+0000","last_undegraded":"2023-07-13T22:29:49.801247+0000","last_fullsized":"2023-07-13T22:29:49.801247+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:48.753915+0000","last_clean_scrub_stamp":"2023-07-13T22:29:48.753915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:17.182760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.782827+0000","last_change":"2023-07-13T22:29:48.785888+0000","last_active":"2023-07-13T22:29:49.782827+0000","last_peered":"2023-07-13T22:29:49.782827+0000","last_clean":"2023-07-13T22:29:49.782827+0000","last_became_active":"2023-07-13T22:29:48.785584+0000","last_became_peered":"2023-07-13T22:29:48.785584+0000","last_unstale":"2023-07-13T22:29:49.782827+0000","last_undegraded":"2023-07-13T22:29:49.782827+0000","last_fullsized":"2023-07-13T22:29:49.782827+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:29:47.748459+0000","last_clean_scrub_stamp":"2023-07-13T22:29:47.748459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:41:30.299089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":169,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-13T22:29:49.782912+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:29:49.782912+0000","last_peered":"2023-07-13T22:29:49.782912+0000","last_clean":"2023-07-13T22:29:49.782912+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:29:49.782912+0000","last_undegraded":"2023-07-13T22:29:49.782912+0000","last_fullsized":"2023-07-13T22:29:49.782912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:52:10.160426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574961,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37088,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334752,"statfs":{"total":96636764160,"available":96598786048,"internally_reserved":0,"allocated":1990656,"data_stored":1221360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67200000000000004,"15min":0.63900000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.874,"5min":1.0780000000000001,"15min":1.115},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.67300000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.94199999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64300000000000002,"15min":0.68200000000000005},"min":{"1min":0.42899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.86299999999999999,"5min":0.94799999999999995,"15min":2.2610000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66100000000000003,"15min":0.65100000000000002},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.34300000000000003},"max":{"1min":1.0149999999999999,"5min":1.0309999999999999,"15min":1.0609999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70699999999999996,"15min":0.77400000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69999999999999996,"15min":0.76600000000000001},"min":{"1min":0.442,"5min":0.42999999999999999,"15min":0.36499999999999999},"max":{"1min":1.006,"5min":1.107,"15min":3.4710000000000001},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.34599999999999997},"max":{"1min":0.90500000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.496},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.747,"15min":0.75800000000000001},"min":{"1min":0.54300000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92200000000000004,"5min":1.024,"15min":1.1399999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.82399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70899999999999996,"15min":0.73899999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.1850000000000001},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.626,"15min":0.65600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82399999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0149999999999999,"5min":1.022,"15min":1.2110000000000001},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.67700000000000005,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.93500000000000005,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.623},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69099999999999995,"15min":0.71799999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94399999999999995,"5min":0.98799999999999999,"15min":1.042},"last":0.80700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1352,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1384448,"data_stored":630798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66600000000000004,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.28799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69399999999999995,"15min":0.63300000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.434,"5min":0.39400000000000002,"15min":0.33900000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69399999999999995,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.495},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69199999999999995,"15min":0.66900000000000004},"min":{"1min":0.46100000000000002,"5min":0.45600000000000002,"15min":0.35599999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.84099999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69299999999999995,"15min":0.66300000000000003},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.53400000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60199999999999998,"15min":0.64600000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.36899999999999999},"max":{"1min":0.80600000000000005,"5min":0.90600000000000003,"15min":0.94499999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.65200000000000002,"15min":0.68200000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":0.91500000000000004,"5min":0.997,"15min":0.998},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":0.94199999999999995,"15min":0.95999999999999996},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73499999999999999,"15min":0.755},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.998,"5min":1.05,"15min":1.05},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.74299999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.375},"max":{"1min":0.93400000000000005,"5min":0.96999999999999997,"15min":1.1890000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69799999999999995,"15min":0.73599999999999999},"min":{"1min":0.45300000000000001,"5min":0.433,"15min":0.39000000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.111},"last":0.94299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1932,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1978368,"data_stored":1220606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.44500000000000001,"15min":0.48299999999999998},"min":{"1min":0.29199999999999998,"5min":0.27800000000000002,"15min":0.249},"max":{"1min":0.58799999999999997,"5min":0.71599999999999997,"15min":1.952},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.254,"15min":0.254},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.433}]},{"osd":2,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46200000000000002,"15min":0.52300000000000002},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29199999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":2.1200000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.54200000000000004,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.627,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.57599999999999996,"15min":0.61299999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39200000000000002},"max":{"1min":0.70199999999999996,"5min":0.83899999999999997,"15min":2.008},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.52600000000000002,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93700000000000006,"5min":1.054,"15min":1.054},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.74399999999999999,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":0.98199999999999998,"15min":2.028},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.76400000000000001,"15min":0.79100000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.46400000000000002},"max":{"1min":0.96999999999999997,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.89100000000000001,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.83499999999999996},"min":{"1min":0.64700000000000002,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.88900000000000001,"5min":1.095,"15min":2.0899999999999999},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.58299999999999996,"5min":0.55100000000000005,"15min":0.52600000000000002},"max":{"1min":0.97099999999999997,"5min":1.073,"15min":1.2230000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76900000000000002,"15min":0.84699999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93999999999999995,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.875}]},{"osd":7,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.79100000000000004,"15min":0.81899999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.50700000000000001},"max":{"1min":0.93100000000000005,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79000000000000004,"15min":0.81399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.93600000000000005,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.78800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36492,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1380352,"data_stored":629252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.45200000000000001,"15min":0.51900000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.68799999999999994,"5min":0.84999999999999998,"15min":11.791},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.29299999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":4.657},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.441,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":1.056,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.51200000000000001,"15min":0.91800000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.55800000000000005,"5min":1.276,"15min":13.398999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":1.8089999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":1.095,"5min":1.095,"15min":4.4530000000000003},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74299999999999999,"15min":1.167},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0149999999999999,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.751,"15min":0.80000000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":1.1220000000000001,"15min":1.776},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77900000000000003,"15min":1.1879999999999999},"min":{"1min":0.499,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0509999999999999,"5min":1.113,"15min":13.349},"last":0.504},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.872},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1739999999999999,"5min":1.302,"15min":4.7649999999999997},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79200000000000004,"15min":0.83299999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77500000000000002,"15min":1.1599999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.046,"5min":1.2330000000000001,"15min":11.679},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:29:02 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.78200000000000003,"15min":1.1950000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":1.1759999999999999,"15min":13.298},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79100000000000004,"15min":0.88400000000000001},"min":{"1min":0.53700000000000003,"5min":0.51800000000000002,"15min":0.50900000000000001},"max":{"1min":1.034,"5min":1.151,"15min":11.456},"last":0.71299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1388544,"data_stored":635883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.434,"15min":0.51800000000000002},"min":{"1min":0.27600000000000002,"5min":0.252,"15min":0.249},"max":{"1min":0.67700000000000005,"5min":1.1779999999999999,"15min":2.282},"last":0.501},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.497,"15min":0.47799999999999998},"min":{"1min":0.32700000000000001,"5min":0.30399999999999999,"15min":0.29099999999999998},"max":{"1min":0.58999999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47199999999999998,"15min":0.47099999999999997},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.26200000000000001},"max":{"1min":0.54200000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.371},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.58299999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73899999999999999,"5min":0.86899999999999999,"15min":2.319},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.33900000000000002,"15min":0.311},"max":{"1min":0.64600000000000002,"5min":1.01,"15min":1.01},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57799999999999996,"15min":0.63300000000000001},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.38700000000000001},"max":{"1min":0.754,"5min":0.95599999999999996,"15min":2.351},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77000000000000002,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95599999999999996,"5min":1.165,"15min":2.238},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78200000000000003,"15min":0.78100000000000003},"min":{"1min":0.53700000000000003,"5min":0.47499999999999998,"15min":0.46899999999999997},"max":{"1min":1.0269999999999999,"5min":1.274,"15min":1.518},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77900000000000003,"15min":0.78100000000000003},"min":{"1min":0.61299999999999999,"5min":0.55800000000000005,"15min":0.505},"max":{"1min":1.0109999999999999,"5min":1.099,"15min":1.325},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80400000000000005,"15min":0.84899999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":0.98299999999999998,"5min":1.2490000000000001,"15min":2.1909999999999998},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80700000000000005,"15min":0.81000000000000005},"min":{"1min":0.56200000000000006,"5min":0.52900000000000003,"15min":0.51700000000000002},"max":{"1min":0.95599999999999996,"5min":1.1040000000000001,"15min":1.488},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.79200000000000004},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.48199999999999998},"max":{"1min":1.0409999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81499999999999995,"15min":0.79500000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":1.0549999999999999,"5min":1.29,"15min":1.29},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81000000000000005,"15min":0.82599999999999996},"min":{"1min":0.57599999999999996,"5min":0.54800000000000004,"15min":0.54400000000000004},"max":{"1min":1.0329999999999999,"5min":1.254,"15min":1.254},"last":0.71899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574960,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36508,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":1396736,"data_stored":631814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.30399999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.63500000000000001,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48199999999999998,"15min":0.45400000000000001},"min":{"1min":0.34899999999999998,"5min":0.29699999999999999,"15min":0.27200000000000002},"max":{"1min":0.61399999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.32400000000000001,"5min":0.313,"15min":0.28299999999999997},"max":{"1min":0.71699999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.314},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.746,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.52300000000000002},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80600000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.38300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.82599999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.496}]},{"osd":4,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":0.95999999999999996,"5min":1.1699999999999999,"15min":1.2250000000000001},"last":0.623},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.56200000000000006,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.749}]},{"osd":5,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.755},"min":{"1min":0.53900000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.02,"5min":1.1919999999999999,"15min":1.411},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.78100000000000003},"min":{"1min":0.55300000000000005,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.93700000000000006,"5min":1.1779999999999999,"15min":1.5469999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"max":{"1min":0.92100000000000004,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.77200000000000002},"min":{"1min":0.58799999999999997,"5min":0.53400000000000003,"15min":0.52100000000000002},"max":{"1min":0.97899999999999998,"5min":1.046,"15min":1.1240000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98799999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.63,"5min":0.55600000000000005,"15min":0.47199999999999998},"max":{"1min":0.95199999999999996,"5min":1.153,"15min":1.1950000000000001},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36516,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335324,"statfs":{"total":96636764160,"available":96599371776,"internally_reserved":0,"allocated":1404928,"data_stored":638433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":0.997,"15min":0.997},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.35099999999999998},"max":{"1min":0.92200000000000004,"5min":1.0489999999999999,"15min":1.298},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72099999999999997,"15min":0.69599999999999995},"min":{"1min":0.499,"5min":0.47999999999999998,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.0049999999999999,"15min":1.3109999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.95099999999999996,"5min":1.0629999999999999,"15min":1.1200000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73099999999999998,"15min":0.71799999999999997},"min":{"1min":0.46300000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.089,"15min":1.1839999999999999},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.92500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":1.234,"5min":1.234,"15min":10.657999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72799999999999998,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.0600000000000001,"15min":8.75},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":0.97899999999999998,"5min":1.042,"15min":1.042},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.53900000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66700000000000004,"15min":0.68799999999999994},"min":{"1min":0.442,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99299999999999999,"5min":1.0189999999999999,"15min":1.173},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70699999999999996,"15min":0.72199999999999998},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.099},"last":0.746}]},{"osd":7,"last update":"Thu Jul 13 22:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.76900000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0600000000000001,"5min":1.1819999999999999,"15min":1.212},"last":0.79600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36500,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335340,"statfs":{"total":96636764160,"available":96599388160,"internally_reserved":0,"allocated":1388544,"data_stored":630376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.65500000000000003,"15min":0.623},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.752,"5min":1.042,"15min":1.042},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.67600000000000005,"15min":0.63600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81499999999999995,"5min":0.998,"15min":0.998},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69999999999999996,"15min":0.66300000000000003},"min":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.66000000000000003,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84699999999999998,"5min":0.94499999999999995,"15min":8.9930000000000003},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.67700000000000005,"15min":0.63900000000000001},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.35899999999999999},"max":{"1min":0.74199999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70099999999999996,"15min":0.67700000000000005},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90900000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71799999999999997,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":0.96599999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72199999999999998,"15min":0.68799999999999994},"min":{"1min":0.49299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.66900000000000004,"15min":0.63800000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.746},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92900000000000005,"5min":0.93500000000000005,"15min":0.995},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.495,"15min":0.42299999999999999},"max":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.51200000000000001,"5min":0.39300000000000002,"15min":0.39200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72299999999999998,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.44700000000000001,"15min":0.41199999999999998},"max":{"1min":0.99299999999999999,"5min":0.997,"15min":1.02},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77800000000000002,"15min":0.76500000000000001},"min":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.0289999999999999,"15min":1.0669999999999999},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:29:54.703 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:29:54.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-13T22:29:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.968+0000 7fe7039ff700 1 Processor -- start 2023-07-13T22:29:54.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.969+0000 7fe7039ff700 1 -- start start 2023-07-13T22:29:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.970+0000 7fe7039ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6fc159ea0 0x7fe6fc0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.970+0000 7fe7039ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc15d200 con 0x7fe6fc0d0890 2023-07-13T22:29:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.970+0000 7fe7039ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc15d4e0 con 0x7fe6fc15c9b0 2023-07-13T22:29:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.970+0000 7fe7039ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc0d2e60 con 0x7fe6fc159ea0 2023-07-13T22:29:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.970+0000 7fe70179b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6fc159ea0 0x7fe6fc0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.970+0000 7fe701f9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe6fc0d0890 0x7fe6fc0cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39402/0 (socket says 172.21.15.60:39402) 2023-07-13T22:29:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.970+0000 7fe70179b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6fc159ea0 0x7fe6fc0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60458/0 (socket says 172.21.15.60:60458) 2023-07-13T22:29:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.970+0000 7fe701f9c700 1 -- 172.21.15.60:0/1503752658 learned_addr learned my addr 172.21.15.60:0/1503752658 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.971+0000 7fe70179b700 1 -- 172.21.15.60:0/1503752658 >> v1:172.21.15.105:6789/0 conn(0x7fe6fc15c9b0 legacy=0x7fe6fc0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.971+0000 7fe70179b700 1 -- 172.21.15.60:0/1503752658 >> v1:172.21.15.60:6789/0 conn(0x7fe6fc0d0890 legacy=0x7fe6fc0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.971+0000 7fe70179b700 1 -- 172.21.15.60:0/1503752658 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc15d7c0 con 0x7fe6fc159ea0 2023-07-13T22:29:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.971+0000 7fe6f3fff700 1 -- 172.21.15.60:0/1503752658 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924794321 0 0) 0x7fe6fc15d200 con 0x7fe6fc0d0890 2023-07-13T22:29:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.971+0000 7fe70179b700 1 --2- 172.21.15.60:0/1503752658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc159ea0 0x7fe6fc0cd6d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009840 tx=0x7fe6f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb7d0db1b05d5dc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.971+0000 7fe6f3fff700 1 -- 172.21.15.60:0/1503752658 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8011830 con 0x7fe6fc159ea0 2023-07-13T22:29:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.971+0000 7fe6f3fff700 1 -- 172.21.15.60:0/1503752658 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8011990 con 0x7fe6fc159ea0 2023-07-13T22:29:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.971+0000 7fe6f3fff700 1 -- 172.21.15.60:0/1503752658 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8011d20 con 0x7fe6fc159ea0 2023-07-13T22:29:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.972+0000 7fe7039ff700 1 -- 172.21.15.60:0/1503752658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc159ea0 msgr2=0x7fe6fc0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.972+0000 7fe7039ff700 1 --2- 172.21.15.60:0/1503752658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc159ea0 0x7fe6fc0cd6d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009840 tx=0x7fe6f8009b50 comp rx=0 tx=0).stop 2023-07-13T22:29:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.972+0000 7fe7039ff700 1 -- 172.21.15.60:0/1503752658 shutdown_connections 2023-07-13T22:29:54.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.972+0000 7fe7039ff700 1 --2- 172.21.15.60:0/1503752658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc159ea0 0x7fe6fc0cd6d0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.972+0000 7fe7039ff700 1 -- 172.21.15.60:0/1503752658 >> 172.21.15.60:0/1503752658 conn(0x7fe6fc071600 msgr2=0x7fe6fc0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:54.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.972+0000 7fe7039ff700 1 -- 172.21.15.60:0/1503752658 shutdown_connections 2023-07-13T22:29:54.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.972+0000 7fe7039ff700 1 -- 172.21.15.60:0/1503752658 wait complete. 2023-07-13T22:29:54.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.974+0000 7fe7039ff700 1 Processor -- start 2023-07-13T22:29:54.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.974+0000 7fe7039ff700 1 -- start start 2023-07-13T22:29:54.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe7039ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc0d0890 0x7fe6fc0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe7039ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6fc159ea0 0x7fe6fc0c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe7039ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6fc15c9b0 0x7fe6fc1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe7039ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0c7540 con 0x7fe6fc0d0890 2023-07-13T22:29:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe7039ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc153710 con 0x7fe6fc15c9b0 2023-07-13T22:29:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe7039ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0cb9b0 con 0x7fe6fc159ea0 2023-07-13T22:29:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe70179b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc0d0890 0x7fe6fc0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe70179b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc0d0890 0x7fe6fc0cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60462/0 (socket says 172.21.15.60:60462) 2023-07-13T22:29:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe70179b700 1 -- 172.21.15.60:0/4069750344 learned_addr learned my addr 172.21.15.60:0/4069750344 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe701f9c700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6fc15c9b0 0x7fe6fc1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe700f9a700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6fc159ea0 0x7fe6fc0c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe70179b700 1 -- 172.21.15.60:0/4069750344 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6fc159ea0 msgr2=0x7fe6fc0c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe70179b700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6fc159ea0 0x7fe6fc0c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe70179b700 1 -- 172.21.15.60:0/4069750344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6fc15c9b0 msgr2=0x7fe6fc1531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:54.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe70179b700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6fc15c9b0 0x7fe6fc1531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:54.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe70179b700 1 -- 172.21.15.60:0/4069750344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80094a0 con 0x7fe6fc0d0890 2023-07-13T22:29:54.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.975+0000 7fe701f9c700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6fc15c9b0 0x7fe6fc1531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:54.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.976+0000 7fe70179b700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc0d0890 0x7fe6fc0cb2b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80090b0 tx=0x7fe6f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:54.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.976+0000 7fe6f1ffb700 1 -- 172.21.15.60:0/4069750344 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f801a900 con 0x7fe6fc0d0890 2023-07-13T22:29:54.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.976+0000 7fe6f1ffb700 1 -- 172.21.15.60:0/4069750344 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f801aa60 con 0x7fe6fc0d0890 2023-07-13T22:29:54.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.976+0000 7fe6f1ffb700 1 -- 172.21.15.60:0/4069750344 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f801add0 con 0x7fe6fc0d0890 2023-07-13T22:29:54.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.976+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc0cbc90 con 0x7fe6fc0d0890 2023-07-13T22:29:54.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.976+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc0cc250 con 0x7fe6fc0d0890 2023-07-13T22:29:54.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.977+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc067a70 con 0x7fe6fc0d0890 2023-07-13T22:29:54.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.978+0000 7fe6f1ffb700 1 -- 172.21.15.60:0/4069750344 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe6f8022070 con 0x7fe6fc0d0890 2023-07-13T22:29:54.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.979+0000 7fe6f1ffb700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6e006ace0 0x7fe6e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:54.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.979+0000 7fe6f1ffb700 1 -- 172.21.15.60:0/4069750344 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6f808a6f0 con 0x7fe6fc0d0890 2023-07-13T22:29:54.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.979+0000 7fe700f9a700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6e006ace0 0x7fe6e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:54.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.980+0000 7fe700f9a700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6e006ace0 0x7fe6e006d190 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0d28b0 tx=0x7fe6ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:54.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:54.983+0000 7fe6f1ffb700 1 -- 172.21.15.60:0/4069750344 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe6f8055960 con 0x7fe6fc0d0890 2023-07-13T22:29:55.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.277+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe6fc04e210 con 0x7fe6fc0d0890 2023-07-13T22:29:55.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.281+0000 7fe6f1ffb700 1 -- 172.21.15.60:0/4069750344 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe6f8058f80 con 0x7fe6fc0d0890 2023-07-13T22:29:55.260 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:29:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:29:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-07-13T22:29:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-07-13T22:29:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-13T22:29:55.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6e006ace0 msgr2=0x7fe6e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:55.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6e006ace0 0x7fe6e006d190 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0d28b0 tx=0x7fe6ec009040 comp rx=0 tx=0).stop 2023-07-13T22:29:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc0d0890 msgr2=0x7fe6fc0cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc0d0890 0x7fe6fc0cb2b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80090b0 tx=0x7fe6f8004750 comp rx=0 tx=0).stop 2023-07-13T22:29:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 shutdown_connections 2023-07-13T22:29:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6e006ace0 0x7fe6e006d190 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6fc0d0890 0x7fe6fc0cb2b0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6fc15c9b0 0x7fe6fc1531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 --2- 172.21.15.60:0/4069750344 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6fc159ea0 0x7fe6fc0c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 >> 172.21.15.60:0/4069750344 conn(0x7fe6fc071600 msgr2=0x7fe6fc0cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:55.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.283+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 shutdown_connections 2023-07-13T22:29:55.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.284+0000 7fe7039ff700 1 -- 172.21.15.60:0/4069750344 wait complete. 2023-07-13T22:29:55.265 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-07-13T22:29:55.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:29:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.505+0000 7f6c6dfff700 1 Processor -- start 2023-07-13T22:29:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.505+0000 7f6c6dfff700 1 -- start start 2023-07-13T22:29:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.506+0000 7f6c6dfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c680b8570 0x7f6c680ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.506+0000 7f6c6dfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6c680d15c0 0x7f6c6815c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.506+0000 7f6c6dfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c680d09d0 con 0x7f6c680d35c0 2023-07-13T22:29:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.506+0000 7f6c6dfff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c680d3b30 con 0x7f6c680b8570 2023-07-13T22:29:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.506+0000 7f6c6dfff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c680cda10 con 0x7f6c680d15c0 2023-07-13T22:29:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.506+0000 7f6c677fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c680b8570 0x7f6c680ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.506+0000 7f6c677fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c680b8570 0x7f6c680ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60474/0 (socket says 172.21.15.60:60474) 2023-07-13T22:29:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.506+0000 7f6c677fe700 1 -- 172.21.15.60:0/3988028504 learned_addr learned my addr 172.21.15.60:0/3988028504 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c67fff700 1 --2- 172.21.15.60:0/3988028504 >> v2:172.21.15.105:3300/0 conn(0x7f6c680d15c0 0x7f6c6815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c677fe700 1 -- 172.21.15.60:0/3988028504 >> v2:172.21.15.105:3300/0 conn(0x7f6c680d15c0 msgr2=0x7f6c6815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c677fe700 1 --2- 172.21.15.60:0/3988028504 >> v2:172.21.15.105:3300/0 conn(0x7f6c680d15c0 0x7f6c6815c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c677fe700 1 -- 172.21.15.60:0/3988028504 >> v1:172.21.15.105:6789/0 conn(0x7f6c680d35c0 legacy=0x7f6c680d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c677fe700 1 -- 172.21.15.60:0/3988028504 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c680cd450 con 0x7f6c680b8570 2023-07-13T22:29:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c67fff700 1 --2- 172.21.15.60:0/3988028504 >> v2:172.21.15.105:3300/0 conn(0x7f6c680d15c0 0x7f6c6815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c677fe700 1 --2- 172.21.15.60:0/3988028504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 0x7f6c680ceeb0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f6c680b2670 tx=0x7f6c58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dd2afb5ef03900 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c667fc700 1 -- 172.21.15.60:0/3988028504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58005e30 con 0x7f6c680b8570 2023-07-13T22:29:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c667fc700 1 -- 172.21.15.60:0/3988028504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c58006030 con 0x7f6c680b8570 2023-07-13T22:29:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.507+0000 7f6c667fc700 1 -- 172.21.15.60:0/3988028504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c580063e0 con 0x7f6c680b8570 2023-07-13T22:29:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.508+0000 7f6c6dfff700 1 -- 172.21.15.60:0/3988028504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 msgr2=0x7f6c680ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.508+0000 7f6c6dfff700 1 --2- 172.21.15.60:0/3988028504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 0x7f6c680ceeb0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f6c680b2670 tx=0x7f6c58009bd0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.508+0000 7f6c6dfff700 1 -- 172.21.15.60:0/3988028504 shutdown_connections 2023-07-13T22:29:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.508+0000 7f6c6dfff700 1 --2- 172.21.15.60:0/3988028504 >> v2:172.21.15.105:3300/0 conn(0x7f6c680d15c0 0x7f6c6815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.508+0000 7f6c6dfff700 1 --2- 172.21.15.60:0/3988028504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 0x7f6c680ceeb0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.508+0000 7f6c6dfff700 1 -- 172.21.15.60:0/3988028504 >> 172.21.15.60:0/3988028504 conn(0x7f6c68071600 msgr2=0x7f6c680719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.508+0000 7f6c6dfff700 1 -- 172.21.15.60:0/3988028504 shutdown_connections 2023-07-13T22:29:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.508+0000 7f6c6dfff700 1 -- 172.21.15.60:0/3988028504 wait complete. 2023-07-13T22:29:55.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.511+0000 7f6c6dfff700 1 Processor -- start 2023-07-13T22:29:55.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.511+0000 7f6c6dfff700 1 -- start start 2023-07-13T22:29:55.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.512+0000 7f6c6dfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 0x7f6c681f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:55.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.512+0000 7f6c6dfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c680d15c0 0x7f6c681f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:55.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.512+0000 7f6c6dfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c680d35c0 0x7f6c680ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:55.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.512+0000 7f6c6dfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680c6c90 con 0x7f6c680b8570 2023-07-13T22:29:55.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.512+0000 7f6c6dfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680c6f70 con 0x7f6c680d35c0 2023-07-13T22:29:55.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.512+0000 7f6c6dfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680c7250 con 0x7f6c680d15c0 2023-07-13T22:29:55.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.512+0000 7f6c677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 0x7f6c681f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:55.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.512+0000 7f6c677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 0x7f6c681f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60484/0 (socket says 172.21.15.60:60484) 2023-07-13T22:29:55.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.512+0000 7f6c677fe700 1 -- 172.21.15.60:0/904263267 learned_addr learned my addr 172.21.15.60:0/904263267 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:55.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c677fe700 1 -- 172.21.15.60:0/904263267 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c680d15c0 msgr2=0x7f6c681f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:55.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c677fe700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c680d15c0 0x7f6c681f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c677fe700 1 -- 172.21.15.60:0/904263267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c680d35c0 msgr2=0x7f6c680ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:55.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c677fe700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c680d35c0 0x7f6c680ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c677fe700 1 -- 172.21.15.60:0/904263267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c58009500 con 0x7f6c680b8570 2023-07-13T22:29:55.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c677fe700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 0x7f6c681f1f40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6c58013040 tx=0x7f6c58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:55.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c4ffff700 1 -- 172.21.15.60:0/904263267 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58004250 con 0x7f6c680b8570 2023-07-13T22:29:55.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c4ffff700 1 -- 172.21.15.60:0/904263267 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c580043f0 con 0x7f6c680b8570 2023-07-13T22:29:55.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c4ffff700 1 -- 172.21.15.60:0/904263267 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c580038e0 con 0x7f6c680b8570 2023-07-13T22:29:55.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c680c7530 con 0x7f6c680b8570 2023-07-13T22:29:55.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.513+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c681f2480 con 0x7f6c680b8570 2023-07-13T22:29:55.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.514+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c680c7a50 con 0x7f6c680b8570 2023-07-13T22:29:55.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.515+0000 7f6c4ffff700 1 -- 172.21.15.60:0/904263267 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c58022070 con 0x7f6c680b8570 2023-07-13T22:29:55.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.516+0000 7f6c4ffff700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c5406ace0 0x7f6c5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:55.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.516+0000 7f6c4ffff700 1 -- 172.21.15.60:0/904263267 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c5808ae40 con 0x7f6c680b8570 2023-07-13T22:29:55.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.517+0000 7f6c66ffd700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c5406ace0 0x7f6c5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:55.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.517+0000 7f6c66ffd700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c5406ace0 0x7f6c5406d190 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0096f0 tx=0x7f6c5c006c50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:55.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.521+0000 7f6c4ffff700 1 -- 172.21.15.60:0/904263267 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6c58055ee0 con 0x7f6c680b8570 2023-07-13T22:29:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.775+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c6804e280 con 0x7f6c680b8570 2023-07-13T22:29:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.775+0000 7f6c4ffff700 1 -- 172.21.15.60:0/904263267 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f6c58059500 con 0x7f6c680b8570 2023-07-13T22:29:55.755 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:29:55.755 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1446832449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1446832449},{"type":"v1","addr":"172.21.15.105:6835","nonce":1446832449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/462201037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":462201037},{"type":"v1","addr":"172.21.15.105:6834","nonce":462201037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/154242070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":154242070},{"type":"v1","addr":"172.21.15.60:6837","nonce":154242070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:29:49.290898+0000","modified":"2023-07-13T22:29:52.790955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7831":{"gid":7831,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6836/3527792396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3527792396},{"type":"v1","addr":"172.21.15.60:6836","nonce":3527792396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-13T22:29:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c5406ace0 msgr2=0x7f6c5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c5406ace0 0x7f6c5406d190 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0096f0 tx=0x7f6c5c006c50 comp rx=0 tx=0).stop 2023-07-13T22:29:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 msgr2=0x7f6c681f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 0x7f6c681f1f40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6c58013040 tx=0x7f6c58004750 comp rx=0 tx=0).stop 2023-07-13T22:29:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 shutdown_connections 2023-07-13T22:29:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6c5406ace0 0x7f6c5406d190 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c680b8570 0x7f6c681f1f40 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c680d35c0 0x7f6c680ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 --2- 172.21.15.60:0/904263267 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6c680d15c0 0x7f6c681f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:55.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 >> 172.21.15.60:0/904263267 conn(0x7f6c68071600 msgr2=0x7f6c680b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:55.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 shutdown_connections 2023-07-13T22:29:55.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:55.778+0000 7f6c6dfff700 1 -- 172.21.15.60:0/904263267 wait complete. 2023-07-13T22:29:55.759 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 73 2023-07-13T22:29:56.177 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:29:49.290898+0000', 'modified': '2023-07-13T22:29:52.790955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7831}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7831': {'gid': 7831, 'name': 'a', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6836/3527792396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 3527792396}, {'type': 'v1', 'addr': '172.21.15.60:6836', 'nonce': 3527792396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:29:56.177 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:29:56.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-13T22:29:56.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.408+0000 7f239f5ff700 1 Processor -- start 2023-07-13T22:29:56.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.408+0000 7f239f5ff700 1 -- start start 2023-07-13T22:29:56.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.409+0000 7f239f5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23980b2010 0x7f23980cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.409+0000 7f239f5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23980d14a0 con 0x7f23980b5a00 2023-07-13T22:29:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.409+0000 7f239f5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23980d1780 con 0x7f23980d26d0 2023-07-13T22:29:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.409+0000 7f239f5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980d2570 con 0x7f23980b2010 2023-07-13T22:29:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.409+0000 7f239db9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f23980b5a00 0x7f239815fee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39410/0 (socket says 172.21.15.60:39410) 2023-07-13T22:29:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.409+0000 7f239db9c700 1 -- 172.21.15.60:0/4030866116 learned_addr learned my addr 172.21.15.60:0/4030866116 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:29:56.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f239d39b700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23980b2010 0x7f23980cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140587553 0 0) 0x7f23980d14a0 con 0x7f23980b5a00 2023-07-13T22:29:56.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23840035b0 con 0x7f23980b5a00 2023-07-13T22:29:56.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3947420904 0 0) 0x7f23840035b0 con 0x7f23980b5a00 2023-07-13T22:29:56.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23980b2010 msgr2=0x7f23980cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23980b2010 0x7f23980cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 >> v1:172.21.15.105:6789/0 conn(0x7f23980d26d0 legacy=0x7f23980cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980cd2a0 con 0x7f23980b5a00 2023-07-13T22:29:56.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f23900019f0 con 0x7f23980b5a00 2023-07-13T22:29:56.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 >> v1:172.21.15.60:6789/0 conn(0x7f23980b5a00 legacy=0x7f239815fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23980b2010 0x7f2384003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.410+0000 7f238ffff700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2384003f70 0x7f2384004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f238ffff700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2384004d90 0x7f2384005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980d2570 con 0x7f2384004d90 2023-07-13T22:29:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238400baf0 con 0x7f2384003f70 2023-07-13T22:29:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238400bcb0 con 0x7f23980b2010 2023-07-13T22:29:56.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f239db9c700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2384004d90 0x7f2384005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f239d39b700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23980b2010 0x7f2384003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f239cb9a700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2384003f70 0x7f2384004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f239db9c700 1 -- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23980b2010 msgr2=0x7f2384003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f239db9c700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23980b2010 0x7f2384003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f239db9c700 1 -- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2384003f70 msgr2=0x7f2384004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f239db9c700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2384003f70 0x7f2384004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f239db9c700 1 -- 172.21.15.60:0/4030866116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840035b0 con 0x7f2384004d90 2023-07-13T22:29:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.411+0000 7f239db9c700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2384004d90 0x7f2384005200 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2390002230 tx=0x7f2384005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae81b80cd2b09af3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390012600 con 0x7f2384004d90 2023-07-13T22:29:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2390012780 con 0x7f2384004d90 2023-07-13T22:29:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f238ffff700 1 -- 172.21.15.60:0/4030866116 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390012b30 con 0x7f2384004d90 2023-07-13T22:29:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f239f5ff700 1 -- 172.21.15.60:0/4030866116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2384004d90 msgr2=0x7f2384005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f239f5ff700 1 --2- 172.21.15.60:0/4030866116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2384004d90 0x7f2384005200 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2390002230 tx=0x7f2384005880 comp rx=0 tx=0).stop 2023-07-13T22:29:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f239db9c700 1 -- 172.21.15.60:0/4030866116 reap_dead start 2023-07-13T22:29:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f239f5ff700 1 -- 172.21.15.60:0/4030866116 shutdown_connections 2023-07-13T22:29:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f239f5ff700 1 -- 172.21.15.60:0/4030866116 >> 172.21.15.60:0/4030866116 conn(0x7f239806b380 msgr2=0x7f239806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f239f5ff700 1 -- 172.21.15.60:0/4030866116 shutdown_connections 2023-07-13T22:29:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.412+0000 7f239f5ff700 1 -- 172.21.15.60:0/4030866116 wait complete. 2023-07-13T22:29:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.414+0000 7f239f5ff700 1 Processor -- start 2023-07-13T22:29:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.414+0000 7f239f5ff700 1 -- start start 2023-07-13T22:29:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.415+0000 7f239f5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2384004d90 0x7f23980cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.415+0000 7f239f5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23980cacd0 0x7f2398148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.415+0000 7f239f5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f239815e570 0x7f239815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.415+0000 7f239f5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398148e70 con 0x7f23980cacd0 2023-07-13T22:29:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.415+0000 7f239f5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980cb1b0 con 0x7f239815e570 2023-07-13T22:29:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.415+0000 7f239f5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980cb490 con 0x7f2384004d90 2023-07-13T22:29:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239cb9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23980cacd0 0x7f2398148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239cb9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23980cacd0 0x7f2398148930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60504/0 (socket says 172.21.15.60:60504) 2023-07-13T22:29:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239cb9a700 1 -- 172.21.15.60:0/559160942 learned_addr learned my addr 172.21.15.60:0/559160942 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239d39b700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2384004d90 0x7f23980cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239db9c700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f239815e570 0x7f239815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239cb9a700 1 -- 172.21.15.60:0/559160942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2384004d90 msgr2=0x7f23980cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239cb9a700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2384004d90 0x7f23980cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239cb9a700 1 -- 172.21.15.60:0/559160942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f239815e570 msgr2=0x7f239815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239cb9a700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f239815e570 0x7f239815b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239cb9a700 1 -- 172.21.15.60:0/559160942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980cb770 con 0x7f23980cacd0 2023-07-13T22:29:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239db9c700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f239815e570 0x7f239815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.416+0000 7f239cb9a700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23980cacd0 0x7f2398148930 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f238800f860 tx=0x7f238800fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.417+0000 7f238dffb700 1 -- 172.21.15.60:0/559160942 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23880050d0 con 0x7f23980cacd0 2023-07-13T22:29:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.417+0000 7f238dffb700 1 -- 172.21.15.60:0/559160942 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2388005250 con 0x7f23980cacd0 2023-07-13T22:29:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.417+0000 7f238dffb700 1 -- 172.21.15.60:0/559160942 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388003680 con 0x7f23980cacd0 2023-07-13T22:29:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.417+0000 7f239f5ff700 1 -- 172.21.15.60:0/559160942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23980cba50 con 0x7f23980cacd0 2023-07-13T22:29:56.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.417+0000 7f239f5ff700 1 -- 172.21.15.60:0/559160942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239815e9f0 con 0x7f23980cacd0 2023-07-13T22:29:56.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.419+0000 7f238dffb700 1 -- 172.21.15.60:0/559160942 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f23880037e0 con 0x7f23980cacd0 2023-07-13T22:29:56.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.420+0000 7f238dffb700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f237c06ace0 0x7f237c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.420+0000 7f238dffb700 1 -- 172.21.15.60:0/559160942 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23880894b0 con 0x7f23980cacd0 2023-07-13T22:29:56.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.420+0000 7f239d39b700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f237c06ace0 0x7f237c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.421+0000 7f239d39b700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f237c06ace0 0x7f237c06d190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2394009c60 tx=0x7f2394009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.422+0000 7f23b2e97b80 1 Processor -- start 2023-07-13T22:29:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.422+0000 7f23b2e97b80 1 -- start start 2023-07-13T22:29:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec5065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec506b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec4f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cec450ea0 con 0x559cec4d6880 2023-07-13T22:29:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cec3c4600 con 0x559cec4d8dc0 2023-07-13T22:29:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cec433910 con 0x559cec4d6c50 2023-07-13T22:29:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f239d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec5065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f239d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec5065e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60516/0 (socket says 172.21.15.60:60516) 2023-07-13T22:29:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f239d39b700 1 -- 172.21.15.60:0/1273298018 learned_addr learned my addr 172.21.15.60:0/1273298018 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f239d39b700 1 -- 172.21.15.60:0/1273298018 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 msgr2=0x559cec4f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f239d39b700 1 --2- 172.21.15.60:0/1273298018 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec4f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f239d39b700 1 -- 172.21.15.60:0/1273298018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 msgr2=0x559cec506b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f239d39b700 1 --2- 172.21.15.60:0/1273298018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec506b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f239d39b700 1 -- 172.21.15.60:0/1273298018 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cec220790 con 0x559cec4d6880 2023-07-13T22:29:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.423+0000 7f239d39b700 1 --2- 172.21.15.60:0/1273298018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec5065e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2394014bf0 tx=0x7f2394017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dca5c5fca2055 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f237affd700 1 -- 172.21.15.60:0/1273298018 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23940155a0 con 0x559cec4d6880 2023-07-13T22:29:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f237affd700 1 -- 172.21.15.60:0/1273298018 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394025070 con 0x559cec4d6880 2023-07-13T22:29:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f237affd700 1 -- 172.21.15.60:0/1273298018 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394015920 con 0x559cec4d6880 2023-07-13T22:29:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f23b2e97b80 1 -- 172.21.15.60:0/1273298018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 msgr2=0x559cec5065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/1273298018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec5065e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2394014bf0 tx=0x7f2394017000 comp rx=0 tx=0).stop 2023-07-13T22:29:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f23b2e97b80 1 -- 172.21.15.60:0/1273298018 shutdown_connections 2023-07-13T22:29:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/1273298018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec5065e0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/1273298018 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec4f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/1273298018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec506b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f23b2e97b80 1 -- 172.21.15.60:0/1273298018 >> 172.21.15.60:0/1273298018 conn(0x559cec4d13b0 msgr2=0x559cec4dc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.424+0000 7f23b2e97b80 1 -- 172.21.15.60:0/1273298018 shutdown_connections 2023-07-13T22:29:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.425+0000 7f23b2e97b80 1 -- 172.21.15.60:0/1273298018 wait complete. 2023-07-13T22:29:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.425+0000 7f23b2e97b80 1 -- start start 2023-07-13T22:29:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.426+0000 7f23b2e97b80 1 Processor -- start 2023-07-13T22:29:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.426+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec51a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec50a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cec42cbe0 con 0x559cec4d6880 2023-07-13T22:29:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cec3ea5b0 con 0x559cec4d8dc0 2023-07-13T22:29:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cec3d0910 con 0x559cec4d6c50 2023-07-13T22:29:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f23b2e97b80 10 client.0 _ll_register_callbacks cb 0x559cec3e8b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:29:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60530/0 (socket says 172.21.15.60:60530) 2023-07-13T22:29:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239d39b700 1 -- 172.21.15.60:0/3977032262 learned_addr learned my addr 172.21.15.60:0/3977032262 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239cb9a700 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec51a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239db9c700 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec50a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239d39b700 1 -- 172.21.15.60:0/3977032262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 msgr2=0x559cec51a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239d39b700 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec51a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239d39b700 1 -- 172.21.15.60:0/3977032262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 msgr2=0x559cec50a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239d39b700 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec50a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239d39b700 1 -- 172.21.15.60:0/3977032262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23940148a0 con 0x559cec4d6880 2023-07-13T22:29:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.427+0000 7f239db9c700 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec50a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:29:56.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f239d39b700 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4f1f20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2394006b20 tx=0x7f2394004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e81da72c4546c60 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f2373fff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:29:56.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f2373fff700 1 -- 172.21.15.60:0/3977032262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239402a670 con 0x559cec4d6880 2023-07-13T22:29:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f2373fff700 1 -- 172.21.15.60:0/3977032262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f239402a7d0 con 0x559cec4d6880 2023-07-13T22:29:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f23b2e97b80 1 -- 172.21.15.60:0/3977032262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cec4ce9c0 con 0x559cec4d6880 2023-07-13T22:29:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f2373fff700 1 -- 172.21.15.60:0/3977032262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f239402a990 con 0x559cec4d6880 2023-07-13T22:29:56.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f23b2e97b80 10 client.8568 fetch_fsmap learned FSMap version 73 2023-07-13T22:29:56.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f23b2e97b80 1 -- 172.21.15.60:0/3977032262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x559cec30d0a0 con 0x559cec4d6880 2023-07-13T22:29:56.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f2373fff700 1 -- 172.21.15.60:0/3977032262 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559cec4ce9c0 con 0x559cec4d6880 2023-07-13T22:29:56.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f2373fff700 1 -- 172.21.15.60:0/3977032262 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 73) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f239402aaf0 con 0x559cec4d6880 2023-07-13T22:29:56.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f23b2e97b80 10 client.8568 fetch_fsmap finished waiting for FSMap version 73 2023-07-13T22:29:56.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f23b2e97b80 10 client.8568 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:29:56.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f23b2e97b80 20 client.8568 populate_metadata read hostname 'smithi060' 2023-07-13T22:29:56.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.428+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] conn(0x559cec534520 0x559cec5348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.429+0000 7f23b2e97b80 4 client.8568 mds_command: new command op to 7831 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:29:56.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.429+0000 7f23b2e97b80 1 -- 172.21.15.60:0/3977032262 --> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cec4ab170 con 0x559cec534520 2023-07-13T22:29:56.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.429+0000 7f239cb9a700 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] conn(0x559cec534520 0x559cec5348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.429+0000 7f2373fff700 10 client.8568 ms_handle_connect on v2:172.21.15.60:6834/3527792396 2023-07-13T22:29:56.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.429+0000 7f239cb9a700 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] conn(0x559cec534520 0x559cec5348f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f0976e290a811 server_cookie=47e65362a540d252 in_seq=0 out_seq=0 2023-07-13T22:29:56.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f2373fff700 1 -- 172.21.15.60:0/3977032262 <== mds.0 v2:172.21.15.60:6834/3527792396 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559cec4ab170 con 0x559cec534520 2023-07-13T22:29:56.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f2373fff700 10 client.8568 handle_command_reply: tid=0 2023-07-13T22:29:56.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 client.8568 shutdown 2023-07-13T22:29:56.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 -- 172.21.15.60:0/3977032262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 msgr2=0x559cec4f1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4f1f20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2394006b20 tx=0x7f2394004750 comp rx=0 tx=0).stop 2023-07-13T22:29:56.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 -- 172.21.15.60:0/3977032262 shutdown_connections 2023-07-13T22:29:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] conn(0x559cec534520 0x559cec5348f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4f1f20 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec50a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/3977032262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec51a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f2373fff700 0 client.8568 ms_handle_reset on v2:172.21.15.60:6834/3527792396 2023-07-13T22:29:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 -- 172.21.15.60:0/3977032262 >> 172.21.15.60:0/3977032262 conn(0x559cec4d13b0 msgr2=0x559cec506c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:56.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.430+0000 7f23b2e97b80 1 -- 172.21.15.60:0/3977032262 shutdown_connections 2023-07-13T22:29:56.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.431+0000 7f23b2e97b80 1 -- 172.21.15.60:0/3977032262 wait complete. 2023-07-13T22:29:56.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.431+0000 7f23b2e97b80 20 client.8568 trim_cache size 0 max 16384 2023-07-13T22:29:56.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.449+0000 7f23b2e97b80 1 Processor -- start 2023-07-13T22:29:56.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.449+0000 7f23b2e97b80 1 -- start start 2023-07-13T22:29:56.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec4eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec51b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cec221390 con 0x559cec4d6880 2023-07-13T22:29:56.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cec4daa80 con 0x559cec4d8dc0 2023-07-13T22:29:56.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cec522cc0 con 0x559cec4d6c50 2023-07-13T22:29:56.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f239d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f239d39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4ee8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60544/0 (socket says 172.21.15.60:60544) 2023-07-13T22:29:56.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f239d39b700 1 -- 172.21.15.60:0/948620483 learned_addr learned my addr 172.21.15.60:0/948620483 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:56.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f239d39b700 1 -- 172.21.15.60:0/948620483 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 msgr2=0x559cec4eedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f239d39b700 1 --2- 172.21.15.60:0/948620483 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec4eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f239d39b700 1 -- 172.21.15.60:0/948620483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 msgr2=0x559cec51b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f239d39b700 1 --2- 172.21.15.60:0/948620483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec51b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.450+0000 7f239d39b700 1 -- 172.21.15.60:0/948620483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cec4f1990 con 0x559cec4d6880 2023-07-13T22:29:56.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f239d39b700 1 --2- 172.21.15.60:0/948620483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4ee8b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2394009df0 tx=0x7f239402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c204b8867fe7686 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f237b7fe700 1 -- 172.21.15.60:0/948620483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239402a670 con 0x559cec4d6880 2023-07-13T22:29:56.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f237b7fe700 1 -- 172.21.15.60:0/948620483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394019070 con 0x559cec4d6880 2023-07-13T22:29:56.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f237b7fe700 1 -- 172.21.15.60:0/948620483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239402a9d0 con 0x559cec4d6880 2023-07-13T22:29:56.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f23b2e97b80 1 -- 172.21.15.60:0/948620483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 msgr2=0x559cec4ee8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/948620483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4ee8b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2394009df0 tx=0x7f239402c040 comp rx=0 tx=0).stop 2023-07-13T22:29:56.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f23b2e97b80 1 -- 172.21.15.60:0/948620483 shutdown_connections 2023-07-13T22:29:56.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/948620483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d6880 0x559cec4ee8b0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/948620483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d8dc0 0x559cec51b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.451+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/948620483 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec4eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.452+0000 7f23b2e97b80 1 -- 172.21.15.60:0/948620483 >> 172.21.15.60:0/948620483 conn(0x559cec4d13b0 msgr2=0x559cec4f1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.452+0000 7f23b2e97b80 1 -- 172.21.15.60:0/948620483 shutdown_connections 2023-07-13T22:29:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.452+0000 7f23b2e97b80 1 -- 172.21.15.60:0/948620483 wait complete. 2023-07-13T22:29:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.452+0000 7f23b2e97b80 1 -- start start 2023-07-13T22:29:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.453+0000 7f23b2e97b80 1 Processor -- start 2023-07-13T22:29:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.453+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d6880 0x559cec51e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.453+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec587410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f23b2e97b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d8dc0 0x559cec587950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cec4f2330 con 0x559cec4d8dc0 2023-07-13T22:29:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cec4d58e0 con 0x559cec4d6880 2023-07-13T22:29:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f23b2e97b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cec5207e0 con 0x559cec4d6c50 2023-07-13T22:29:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f23b2e97b80 10 client.0 _ll_register_callbacks cb 0x7f23940158d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:29:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239db9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d8dc0 0x559cec587950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239db9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d8dc0 0x559cec587950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60548/0 (socket says 172.21.15.60:60548) 2023-07-13T22:29:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239db9c700 1 -- 172.21.15.60:0/926010145 learned_addr learned my addr 172.21.15.60:0/926010145 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239cb9a700 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec587410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239d39b700 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d6880 0x559cec51e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239db9c700 1 -- 172.21.15.60:0/926010145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 msgr2=0x559cec587410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239db9c700 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec587410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239db9c700 1 -- 172.21.15.60:0/926010145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d6880 msgr2=0x559cec51e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239db9c700 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d6880 0x559cec51e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.454+0000 7f239db9c700 1 -- 172.21.15.60:0/926010145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cec30d0a0 con 0x559cec4d8dc0 2023-07-13T22:29:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f239db9c700 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d8dc0 0x559cec587950 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f2390012000 tx=0x7f23900013f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e4d6498bd05897 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f2373fff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:29:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f2373fff700 1 -- 172.21.15.60:0/926010145 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390024070 con 0x559cec4d8dc0 2023-07-13T22:29:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f2373fff700 1 -- 172.21.15.60:0/926010145 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2390026a80 con 0x559cec4d8dc0 2023-07-13T22:29:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f23b2e97b80 1 -- 172.21.15.60:0/926010145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2394025070 con 0x559cec4d8dc0 2023-07-13T22:29:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f2373fff700 1 -- 172.21.15.60:0/926010145 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2390012610 con 0x559cec4d8dc0 2023-07-13T22:29:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f23b2e97b80 10 client.8574 fetch_fsmap learned FSMap version 73 2023-07-13T22:29:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f23b2e97b80 1 -- 172.21.15.60:0/926010145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x559cec3f2b10 con 0x559cec4d8dc0 2023-07-13T22:29:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f2373fff700 1 -- 172.21.15.60:0/926010145 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394025070 con 0x559cec4d8dc0 2023-07-13T22:29:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f2373fff700 1 -- 172.21.15.60:0/926010145 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 73) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f239000a830 con 0x559cec4d8dc0 2023-07-13T22:29:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f23b2e97b80 10 client.8574 fetch_fsmap finished waiting for FSMap version 73 2023-07-13T22:29:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f23b2e97b80 10 client.8574 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:29:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.455+0000 7f23b2e97b80 20 client.8574 populate_metadata read hostname 'smithi060' 2023-07-13T22:29:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.456+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] conn(0x559cec5028d0 0x559cec502ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.456+0000 7f23b2e97b80 4 client.8574 mds_command: new command op to 7831 tid=0[{"prefix": "status"}] 2023-07-13T22:29:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.456+0000 7f23b2e97b80 1 -- 172.21.15.60:0/926010145 --> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559cec425d20 con 0x559cec5028d0 2023-07-13T22:29:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.456+0000 7f239d39b700 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] conn(0x559cec5028d0 0x559cec502ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.456+0000 7f239d39b700 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] conn(0x559cec5028d0 0x559cec502ca0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff18ca956d1a0b server_cookie=b56fa11991b7652b in_seq=0 out_seq=0 2023-07-13T22:29:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.456+0000 7f2373fff700 10 client.8574 ms_handle_connect on v2:172.21.15.60:6834/3527792396 2023-07-13T22:29:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.456+0000 7f2373fff700 1 -- 172.21.15.60:0/926010145 <== mds.0 v2:172.21.15.60:6834/3527792396 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x559cec425d20 con 0x559cec5028d0 2023-07-13T22:29:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.456+0000 7f2373fff700 10 client.8574 handle_command_reply: tid=0 2023-07-13T22:29:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.456+0000 7f23b2e97b80 1 client.8574 shutdown 2023-07-13T22:29:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 -- 172.21.15.60:0/926010145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d8dc0 msgr2=0x559cec587950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d8dc0 0x559cec587950 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f2390012000 tx=0x7f23900013f0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 -- 172.21.15.60:0/926010145 shutdown_connections 2023-07-13T22:29:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.60:6834/3527792396,v1:172.21.15.60:6836/3527792396] conn(0x559cec5028d0 0x559cec502ca0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559cec4d8dc0 0x559cec587950 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cec4d6880 0x559cec51e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 --2- 172.21.15.60:0/926010145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x559cec4d6c50 0x559cec587410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f2373fff700 0 client.8574 ms_handle_reset on v2:172.21.15.60:6834/3527792396 2023-07-13T22:29:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 -- 172.21.15.60:0/926010145 >> 172.21.15.60:0/926010145 conn(0x559cec4d13b0 msgr2=0x559cec524ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 -- 172.21.15.60:0/926010145 shutdown_connections 2023-07-13T22:29:56.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 1 -- 172.21.15.60:0/926010145 wait complete. 2023-07-13T22:29:56.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f23b2e97b80 20 client.8574 trim_cache size 0 max 16384 2023-07-13T22:29:56.450 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:29:56.450 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:29:56.451 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:29:56.451 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7831, 2023-07-13T22:29:56.452 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:29:56.452 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:29:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:29:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.6736435509999996, 2023-07-13T22:29:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 73, 2023-07-13T22:29:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 112, 2023-07-13T22:29:56.454 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 112, 2023-07-13T22:29:56.454 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 13.126870408 2023-07-13T22:29:56.454 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:29:56.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.457+0000 7f239f5ff700 1 -- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f237c06ace0 msgr2=0x7f237c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f237c06ace0 0x7f237c06d190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2394009c60 tx=0x7f2394009210 comp rx=0 tx=0).stop 2023-07-13T22:29:56.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 -- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23980cacd0 msgr2=0x7f2398148930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23980cacd0 0x7f2398148930 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f238800f860 tx=0x7f238800fc20 comp rx=0 tx=0).stop 2023-07-13T22:29:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 -- 172.21.15.60:0/559160942 shutdown_connections 2023-07-13T22:29:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f237c06ace0 0x7f237c06d190 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f239815e570 0x7f239815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2384004d90 0x7f23980cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 --2- 172.21.15.60:0/559160942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23980cacd0 0x7f2398148930 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 -- 172.21.15.60:0/559160942 >> 172.21.15.60:0/559160942 conn(0x7f239806b380 msgr2=0x7f239815f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 -- 172.21.15.60:0/559160942 shutdown_connections 2023-07-13T22:29:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.458+0000 7f239f5ff700 1 -- 172.21.15.60:0/559160942 wait complete. 2023-07-13T22:29:56.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:29:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.682+0000 7f437fbff700 1 Processor -- start 2023-07-13T22:29:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.682+0000 7f437fbff700 1 -- start start 2023-07-13T22:29:56.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.683+0000 7f437fbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4378159170 0x7f43780c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.683+0000 7f437fbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 0x7f4378070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.683+0000 7f437fbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43780707b0 0x7f43780cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.683+0000 7f437fbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f437815d970 con 0x7f43780707b0 2023-07-13T22:29:56.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.683+0000 7f437fbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4378070c10 con 0x7f4378159170 2023-07-13T22:29:56.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.683+0000 7f437fbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437815cd60 con 0x7f43780c5040 2023-07-13T22:29:56.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.683+0000 7f437e19c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43780707b0 0x7f43780cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.683+0000 7f437e19c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43780707b0 0x7f43780cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60556/0 (socket says 172.21.15.60:60556) 2023-07-13T22:29:56.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.683+0000 7f437e19c700 1 -- 172.21.15.60:0/4198467781 learned_addr learned my addr 172.21.15.60:0/4198467781 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:56.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437d99b700 1 --2- 172.21.15.60:0/4198467781 >> v2:172.21.15.105:3300/0 conn(0x7f4378159170 0x7f43780c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437d19a700 1 --2- 172.21.15.60:0/4198467781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 0x7f4378070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437e19c700 1 -- 172.21.15.60:0/4198467781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 msgr2=0x7f4378070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437e19c700 1 --2- 172.21.15.60:0/4198467781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 0x7f4378070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437e19c700 1 -- 172.21.15.60:0/4198467781 >> v2:172.21.15.105:3300/0 conn(0x7f4378159170 msgr2=0x7f43780c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:56.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437e19c700 1 --2- 172.21.15.60:0/4198467781 >> v2:172.21.15.105:3300/0 conn(0x7f4378159170 0x7f43780c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437e19c700 1 -- 172.21.15.60:0/4198467781 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437815d040 con 0x7f43780707b0 2023-07-13T22:29:56.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437e19c700 1 --2- 172.21.15.60:0/4198467781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43780707b0 0x7f43780cc2c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4378067080 tx=0x7f437000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e291ec16aa017c8b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437c999700 1 -- 172.21.15.60:0/4198467781 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437000bbc0 con 0x7f43780707b0 2023-07-13T22:29:56.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437c999700 1 -- 172.21.15.60:0/4198467781 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437000bd20 con 0x7f43780707b0 2023-07-13T22:29:56.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.684+0000 7f437c999700 1 -- 172.21.15.60:0/4198467781 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43700116a0 con 0x7f43780707b0 2023-07-13T22:29:56.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.685+0000 7f437fbff700 1 -- 172.21.15.60:0/4198467781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43780707b0 msgr2=0x7f43780cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.685+0000 7f437fbff700 1 --2- 172.21.15.60:0/4198467781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43780707b0 0x7f43780cc2c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4378067080 tx=0x7f437000b330 comp rx=0 tx=0).stop 2023-07-13T22:29:56.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.685+0000 7f437fbff700 1 -- 172.21.15.60:0/4198467781 shutdown_connections 2023-07-13T22:29:56.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.685+0000 7f437fbff700 1 --2- 172.21.15.60:0/4198467781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43780707b0 0x7f43780cc2c0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.685+0000 7f437fbff700 1 --2- 172.21.15.60:0/4198467781 >> v2:172.21.15.105:3300/0 conn(0x7f4378159170 0x7f43780c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.685+0000 7f437fbff700 1 --2- 172.21.15.60:0/4198467781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 0x7f4378070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.685+0000 7f437fbff700 1 -- 172.21.15.60:0/4198467781 >> 172.21.15.60:0/4198467781 conn(0x7f43780cf410 msgr2=0x7f43780cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:56.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.685+0000 7f437fbff700 1 -- 172.21.15.60:0/4198467781 shutdown_connections 2023-07-13T22:29:56.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.685+0000 7f437fbff700 1 -- 172.21.15.60:0/4198467781 wait complete. 2023-07-13T22:29:56.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.687+0000 7f437fbff700 1 Processor -- start 2023-07-13T22:29:56.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.687+0000 7f437fbff700 1 -- start start 2023-07-13T22:29:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437fbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43780707b0 0x7f437814cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437fbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 0x7f43780af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437fbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4378159170 0x7f43780afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437fbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43780c65f0 con 0x7f4378159170 2023-07-13T22:29:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437fbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43780c78a0 con 0x7f43780707b0 2023-07-13T22:29:56.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437fbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43780c7b80 con 0x7f43780c5040 2023-07-13T22:29:56.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437e19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4378159170 0x7f43780afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437e19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4378159170 0x7f43780afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60564/0 (socket says 172.21.15.60:60564) 2023-07-13T22:29:56.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437e19c700 1 -- 172.21.15.60:0/212029539 learned_addr learned my addr 172.21.15.60:0/212029539 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:56.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437d99b700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43780707b0 0x7f437814cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.688+0000 7f437d19a700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 0x7f43780af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f437e19c700 1 -- 172.21.15.60:0/212029539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 msgr2=0x7f43780af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f437e19c700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 0x7f43780af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f437e19c700 1 -- 172.21.15.60:0/212029539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43780707b0 msgr2=0x7f437814cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f437e19c700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43780707b0 0x7f437814cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f437e19c700 1 -- 172.21.15.60:0/212029539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370009b30 con 0x7f4378159170 2023-07-13T22:29:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f437e19c700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4378159170 0x7f43780afa50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f437806b790 tx=0x7f4370004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f436e7fc700 1 -- 172.21.15.60:0/212029539 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437000cea0 con 0x7f4378159170 2023-07-13T22:29:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f436e7fc700 1 -- 172.21.15.60:0/212029539 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370004900 con 0x7f4378159170 2023-07-13T22:29:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43780c7e60 con 0x7f4378159170 2023-07-13T22:29:56.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f436e7fc700 1 -- 172.21.15.60:0/212029539 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43700116a0 con 0x7f4378159170 2023-07-13T22:29:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.689+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43780c8660 con 0x7f4378159170 2023-07-13T22:29:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.690+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43780c8380 con 0x7f4378159170 2023-07-13T22:29:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.691+0000 7f436e7fc700 1 -- 172.21.15.60:0/212029539 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4370011800 con 0x7f4378159170 2023-07-13T22:29:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.692+0000 7f436e7fc700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f435c06ace0 0x7f435c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.692+0000 7f436e7fc700 1 -- 172.21.15.60:0/212029539 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f437008ab50 con 0x7f4378159170 2023-07-13T22:29:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.693+0000 7f437d99b700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f435c06ace0 0x7f435c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.693+0000 7f437d99b700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f435c06ace0 0x7f435c06d190 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f43740098b0 tx=0x7f4374008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:56.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.697+0000 7f436e7fc700 1 -- 172.21.15.60:0/212029539 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f43780c8380 con 0x7f4378159170 2023-07-13T22:29:56.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.953+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43780c8380 con 0x7f4378159170 2023-07-13T22:29:56.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.954+0000 7f436e7fc700 1 -- 172.21.15.60:0/212029539 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4370015060 con 0x7f4378159170 2023-07-13T22:29:56.933 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:29:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f435c06ace0 msgr2=0x7f435c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f435c06ace0 0x7f435c06d190 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f43740098b0 tx=0x7f4374008040 comp rx=0 tx=0).stop 2023-07-13T22:29:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4378159170 msgr2=0x7f43780afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4378159170 0x7f43780afa50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f437806b790 tx=0x7f4370004ad0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 shutdown_connections 2023-07-13T22:29:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f435c06ace0 0x7f435c06d190 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4378159170 0x7f43780afa50 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43780707b0 0x7f437814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 --2- 172.21.15.60:0/212029539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43780c5040 0x7f43780af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 >> 172.21.15.60:0/212029539 conn(0x7f43780cf410 msgr2=0x7f4378069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 shutdown_connections 2023-07-13T22:29:56.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:56.956+0000 7f437fbff700 1 -- 172.21.15.60:0/212029539 wait complete. 2023-07-13T22:29:56.953 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:29:56.954 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:29:56.954 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:29:56.954 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:29:56.954 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:56.955 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:29:56.955 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:29:56.955 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:29:56.955 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:29:56.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:56.956 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:29:57.003 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:29:57.004 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:29:57.004 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:29:57.004 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:57.004 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:29:57.004 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:57.004 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:57.005 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:57.005 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:29:57.005 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:57.005 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:57.005 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:29:57.005 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:29:57.006 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:57.006 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:29:57.006 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:29:57.006 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41176sec preferred_lft 41176sec 2023-07-13T22:29:57.006 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:29:57.006 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:57.006 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:57.007 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:57.007 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:29:57.007 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:29:57.007 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:57.007 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:29:57.007 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:57.007 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:29:57.008 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:57.008 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:29:57.008 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:29:57.008 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:29:57.008 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:57.008 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:29:57.009 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:29:57.009 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:29:57.009 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:57.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:57.010 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:29:57.079 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:29:57.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:57.083 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:29:57.131 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:29:57.131 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:29:57.131 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:29:57.132 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:29:57.132 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:29:57.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:57.133 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:29:57.133 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:29:57.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:29:57.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:29:57.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:29:57.135 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:29:57.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:57.309 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:29:57.356 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:29:57.357 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:29:57.357 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:29:57.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:57.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:29:57.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:29:57.410 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:29:57.430 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:29:57.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:29:57.431 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:57.517 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:29:57.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:29:57.518 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:57.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:29:57.715 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:29:57.716 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:29:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:29:57.790 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:29:57.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:57.792 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:29:57.839 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:29:57.839 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:29:57.883 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:29:57.955 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:29:57.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:29:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:29:58.030 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:29:58.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:58.031 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:29:58.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:29:58.076+0000 7f59b7f2c5c0 -1 init, newargv = 0x562f90f2b250 newargc=15 2023-07-13T22:29:58.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[157131]: starting ceph client 2023-07-13T22:29:58.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[157131]: starting fuse 2023-07-13T22:29:58.082 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:29:58.082 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:29:58.083 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:29:58.083 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:29:58.083 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:29:58.084 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:29:58.084 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:29:58.084 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:58.084 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:29:58.085 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:29:58.085 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:29:58.085 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:29:58.086 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:58.086 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:29:58.086 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:29:58.086 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:29:58.087 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:29:58.087 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:58.087 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:29:58.088 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:29:58.088 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:29:58.089 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:29:58.089 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:29:58.090 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:29:58.091 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:29:58.091 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:29:58.092 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:29:58.092 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:29:58.093 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:29:58.268 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.157131.asok 2023-07-13T22:29:58.445 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.157131.asok 2023-07-13T22:29:58.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:58.446 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.157131.asok status 2023-07-13T22:29:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:29:58.678 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:29:58.678 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:29:58.678 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:29:58.679 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:29:58.679 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:29:58.679 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:29:58.679 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "157131", 2023-07-13T22:29:58.680 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:29:58.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:58.680 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:29:58.680 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:29:58.681 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8586, 2023-07-13T22:29:58.681 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:29:58.681 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:29:58.681 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:29:58.682 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 8586 2023-07-13T22:29:58.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:58.682 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:29:58.682 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:29:58.683 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:29:58.683 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1730080076 2023-07-13T22:29:58.683 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:29:58.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:58.684 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:29:58.684 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:29:58.684 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:29:58.684 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1730080076 2023-07-13T22:29:58.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:29:58.685 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.8586 192.168.0.1:0/1730080076", 2023-07-13T22:29:58.685 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/1730080076", 2023-07-13T22:29:58.685 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:29:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 73, 2023-07-13T22:29:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 112, 2023-07-13T22:29:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:29:58.686 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:29:58.687 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:29:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:29:58.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:58.692 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:29:58.693 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:29:58.742 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:58.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:58.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:29:58.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:29:59.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.038+0000 7f2426dff700 1 Processor -- start 2023-07-13T22:29:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.039+0000 7f2426dff700 1 -- start start 2023-07-13T22:29:59.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.040+0000 7f2426dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2420159170 0x7f24200c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:59.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.040+0000 7f2426dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24200c5040 0x7f2420070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:59.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.040+0000 7f2426dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242015d970 con 0x7f24200707b0 2023-07-13T22:29:59.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.040+0000 7f2426dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242015cd60 con 0x7f2420159170 2023-07-13T22:29:59.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.040+0000 7f2426dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242015d040 con 0x7f24200c5040 2023-07-13T22:29:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.040+0000 7f2424b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2420159170 0x7f24200c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.040+0000 7f2424b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2420159170 0x7f24200c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60580/0 (socket says 172.21.15.60:60580) 2023-07-13T22:29:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.040+0000 7f2424b9b700 1 -- 172.21.15.60:0/2131720265 learned_addr learned my addr 172.21.15.60:0/2131720265 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f241ffff700 1 --2- 172.21.15.60:0/2131720265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24200c5040 0x7f2420070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f2424b9b700 1 -- 172.21.15.60:0/2131720265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24200c5040 msgr2=0x7f2420070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f2424b9b700 1 --2- 172.21.15.60:0/2131720265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24200c5040 0x7f2420070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f2424b9b700 1 -- 172.21.15.60:0/2131720265 >> v1:172.21.15.105:6789/0 conn(0x7f24200707b0 legacy=0x7f24200cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:29:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f2424b9b700 1 -- 172.21.15.60:0/2131720265 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2420070c10 con 0x7f2420159170 2023-07-13T22:29:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f241ffff700 1 --2- 172.21.15.60:0/2131720265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24200c5040 0x7f2420070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f2424b9b700 1 --2- 172.21.15.60:0/2131720265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2420159170 0x7f24200c4b00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f24140098f0 tx=0x7f2414009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce031ac723af6d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f241f7fe700 1 -- 172.21.15.60:0/2131720265 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400b8a0 con 0x7f2420159170 2023-07-13T22:29:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f241f7fe700 1 -- 172.21.15.60:0/2131720265 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400ba00 con 0x7f2420159170 2023-07-13T22:29:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.041+0000 7f241f7fe700 1 -- 172.21.15.60:0/2131720265 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400bd90 con 0x7f2420159170 2023-07-13T22:29:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.042+0000 7f2426dff700 1 -- 172.21.15.60:0/2131720265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2420159170 msgr2=0x7f24200c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:59.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.042+0000 7f2426dff700 1 --2- 172.21.15.60:0/2131720265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2420159170 0x7f24200c4b00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f24140098f0 tx=0x7f2414009c00 comp rx=0 tx=0).stop 2023-07-13T22:29:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.042+0000 7f2426dff700 1 -- 172.21.15.60:0/2131720265 shutdown_connections 2023-07-13T22:29:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.042+0000 7f2426dff700 1 --2- 172.21.15.60:0/2131720265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2420159170 0x7f24200c4b00 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.042+0000 7f2426dff700 1 --2- 172.21.15.60:0/2131720265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24200c5040 0x7f2420070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.042+0000 7f2426dff700 1 -- 172.21.15.60:0/2131720265 >> 172.21.15.60:0/2131720265 conn(0x7f24200cf410 msgr2=0x7f24200cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.042+0000 7f2426dff700 1 -- 172.21.15.60:0/2131720265 shutdown_connections 2023-07-13T22:29:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.042+0000 7f2426dff700 1 -- 172.21.15.60:0/2131720265 wait complete. 2023-07-13T22:29:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.044+0000 7f2426dff700 1 Processor -- start 2023-07-13T22:29:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.044+0000 7f2426dff700 1 -- start start 2023-07-13T22:29:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2426dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24200707b0 0x7f242014a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2426dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24200c5040 0x7f242015be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2426dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2420159170 0x7f24201597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2426dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242015a370 con 0x7f24200707b0 2023-07-13T22:29:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2426dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242015a650 con 0x7f24200c5040 2023-07-13T22:29:59.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2426dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24200c65f0 con 0x7f2420159170 2023-07-13T22:29:59.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2424b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24200707b0 0x7f242014a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:59.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2424b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24200707b0 0x7f242014a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60594/0 (socket says 172.21.15.60:60594) 2023-07-13T22:29:59.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2424b9b700 1 -- 172.21.15.60:0/2278149365 learned_addr learned my addr 172.21.15.60:0/2278149365 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:29:59.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2424b9b700 1 -- 172.21.15.60:0/2278149365 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2420159170 msgr2=0x7f24201597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:59.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2424b9b700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2420159170 0x7f24201597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:59.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f241ffff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24200c5040 0x7f242015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:59.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2424b9b700 1 -- 172.21.15.60:0/2278149365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24200c5040 msgr2=0x7f242015be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:59.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.045+0000 7f2424b9b700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24200c5040 0x7f242015be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:59.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.046+0000 7f2424b9b700 1 -- 172.21.15.60:0/2278149365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414009550 con 0x7f24200707b0 2023-07-13T22:29:59.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.046+0000 7f241ffff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24200c5040 0x7f242015be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:29:59.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.046+0000 7f2424b9b700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24200707b0 0x7f242014a980 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f2414000c00 tx=0x7f2414004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:59.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.046+0000 7f241d7fa700 1 -- 172.21.15.60:0/2278149365 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414004570 con 0x7f24200707b0 2023-07-13T22:29:59.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.046+0000 7f241d7fa700 1 -- 172.21.15.60:0/2278149365 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241401f070 con 0x7f24200707b0 2023-07-13T22:29:59.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.046+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24200c68d0 con 0x7f24200707b0 2023-07-13T22:29:59.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.046+0000 7f241d7fa700 1 -- 172.21.15.60:0/2278149365 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140038e0 con 0x7f24200707b0 2023-07-13T22:29:59.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.046+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242015a930 con 0x7f24200707b0 2023-07-13T22:29:59.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.047+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24200c6b10 con 0x7f24200707b0 2023-07-13T22:29:59.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.048+0000 7f241d7fa700 1 -- 172.21.15.60:0/2278149365 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2414021030 con 0x7f24200707b0 2023-07-13T22:29:59.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.049+0000 7f241d7fa700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f240406ace0 0x7f240406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:29:59.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.050+0000 7f241d7fa700 1 -- 172.21.15.60:0/2278149365 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f241408aa10 con 0x7f24200707b0 2023-07-13T22:29:59.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.050+0000 7f241ffff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f240406ace0 0x7f240406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:29:59.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.050+0000 7f241ffff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f240406ace0 0x7f240406d190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f240c008910 tx=0x7f240c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:29:59.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.054+0000 7f241d7fa700 1 -- 172.21.15.60:0/2278149365 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f24200c6b10 con 0x7f24200707b0 2023-07-13T22:29:59.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.304+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f24200c6ac0 con 0x7f24200707b0 2023-07-13T22:29:59.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.305+0000 7f241d7fa700 1 -- 172.21.15.60:0/2278149365 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2414015060 con 0x7f24200707b0 2023-07-13T22:29:59.284 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:29:59.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.307+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f240406ace0 msgr2=0x7f240406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.307+0000 7f2426dff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f240406ace0 0x7f240406d190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f240c008910 tx=0x7f240c008040 comp rx=0 tx=0).stop 2023-07-13T22:29:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.307+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24200707b0 msgr2=0x7f242014a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:29:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.307+0000 7f2426dff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24200707b0 0x7f242014a980 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f2414000c00 tx=0x7f2414004750 comp rx=0 tx=0).stop 2023-07-13T22:29:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.307+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 shutdown_connections 2023-07-13T22:29:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.307+0000 7f2426dff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f240406ace0 0x7f240406d190 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.307+0000 7f2426dff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24200707b0 0x7f242014a980 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.307+0000 7f2426dff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2420159170 0x7f24201597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:59.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.308+0000 7f2426dff700 1 --2- 172.21.15.60:0/2278149365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24200c5040 0x7f242015be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:29:59.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.308+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 >> 172.21.15.60:0/2278149365 conn(0x7f24200cf410 msgr2=0x7f242006dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:29:59.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.308+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 shutdown_connections 2023-07-13T22:29:59.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:29:59.308+0000 7f2426dff700 1 -- 172.21.15.60:0/2278149365 wait complete. 2023-07-13T22:29:59.305 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:29:59.305 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:29:59.305 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:29:59.305 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:29:59.306 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:59.306 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:29:59.306 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:29:59.307 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:29:59.307 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:29:59.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:59.307 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:29:59.357 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:29:59.357 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:29:59.357 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:29:59.358 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:59.358 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:29:59.358 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:59.358 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:59.358 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:59.358 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:29:59.358 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:59.359 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:59.359 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:29:59.359 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:29:59.359 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:59.359 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:29:59.359 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:59.359 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:29:59.360 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41171sec preferred_lft 41171sec 2023-07-13T22:29:59.360 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:29:59.360 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:59.360 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:29:59.360 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:29:59.360 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:29:59.361 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:59.361 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:29:59.361 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:59.361 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:29:59.361 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:29:59.361 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:29:59.361 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:59.362 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:29:59.362 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:29:59.362 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::14e3:bfff:fea8:bf57/64 scope link 2023-07-13T22:29:59.362 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:29:59.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:59.363 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:29:59.433 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:29:59.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:59.436 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:29:59.485 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:29:59.485 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:29:59.485 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:29:59.486 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:29:59.486 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:29:59.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:59.487 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:29:59.487 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:29:59.487 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:29:59.487 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:29:59.488 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:29:59.488 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:29:59.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:59.658 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:29:59.705 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:29:59.705 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:29:59.705 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:29:59.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:29:59.706 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:29:59.816 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:29:59.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:29:59.817 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:59.862 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:29:59.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:29:59.863 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:29:59.947 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:30:00.067 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:30:00.067 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:30:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:30:00.140 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:30:00.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:00.141 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:30:00.186 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:30:00.187 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:30:00.231 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:30:00.303 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:30:00.304 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:30:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:30:00.376 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:30:00.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:00.379 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:30:00.403 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:30:00.425+0000 7f7a66fd45c0 -1 init, newargv = 0x56075fdc3f10 newargc=15 2023-07-13T22:30:00.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[126311]: starting ceph client 2023-07-13T22:30:00.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[126311]: starting fuse 2023-07-13T22:30:00.428 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:30:00.429 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:30:00.429 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:30:00.429 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:30:00.430 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:30:00.430 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:30:00.430 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:30:00.431 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:30:00.431 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:30:00.431 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:30:00.431 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:30:00.432 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:30:00.432 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:30:00.432 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:30:00.433 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:30:00.433 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:30:00.434 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:30:00.434 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:30:00.434 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:30:00.434 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:30:00.435 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:30:00.435 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:30:00.435 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:30:00.436 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:30:00.436 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:30:00.436 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:30:00.436 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:30:00.437 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:30:00.437 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:30:00.613 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.126311.asok 2023-07-13T22:30:00.792 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.126311.asok 2023-07-13T22:30:00.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:00.793 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.126311.asok status 2023-07-13T22:30:01.017 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:30:01.017 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:30:01.017 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:30:01.018 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:30:01.018 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:30:01.018 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:30:01.018 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:30:01.018 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "126311", 2023-07-13T22:30:01.018 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:30:01.018 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:30:01.019 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:30:01.019 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:30:01.019 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 7972, 2023-07-13T22:30:01.019 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:30:01.019 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:30:01.019 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:30:01.019 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 7972 2023-07-13T22:30:01.019 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:30:01.020 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:30:01.020 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:30:01.020 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:30:01.020 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 759874285 2023-07-13T22:30:01.020 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:30:01.020 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:30:01.020 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:30:01.021 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:30:01.021 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:30:01.021 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 759874285 2023-07-13T22:30:01.021 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:30:01.021 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.7972 192.168.0.1:0/759874285", 2023-07-13T22:30:01.021 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/759874285", 2023-07-13T22:30:01.021 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:30:01.022 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 73, 2023-07-13T22:30:01.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 112, 2023-07-13T22:30:01.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:30:01.022 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:30:01.022 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:30:01.022 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:30:01.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:01.032 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:30:01.032 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:01.078 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:30:01.079 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:01.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:01.080 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:01.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-07-13T22:30:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.397+0000 7f5eec7ff700 1 Processor -- start 2023-07-13T22:30:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.397+0000 7f5eec7ff700 1 -- start start 2023-07-13T22:30:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.398+0000 7f5eec7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5ee40b8200 0x7f5ee40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.398+0000 7f5eec7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ee40b1ac0 0x7f5ee40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.398+0000 7f5eec7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee40d1870 con 0x7f5ee40cede0 2023-07-13T22:30:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.398+0000 7f5eec7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee40cf370 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.398+0000 7f5eec7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee40d12b0 con 0x7f5ee40b8200 2023-07-13T22:30:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.399+0000 7f5ee9d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ee40b1ac0 0x7f5ee40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.399+0000 7f5ee9d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ee40b1ac0 0x7f5ee40cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60602/0 (socket says 172.21.15.60:60602) 2023-07-13T22:30:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.399+0000 7f5ee9d9a700 1 -- 172.21.15.60:0/3240334676 learned_addr learned my addr 172.21.15.60:0/3240334676 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.399+0000 7f5eea59b700 1 --2- 172.21.15.60:0/3240334676 >> v2:172.21.15.105:3300/0 conn(0x7f5ee40b8200 0x7f5ee40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.399+0000 7f5ee9d9a700 1 -- 172.21.15.60:0/3240334676 >> v2:172.21.15.105:3300/0 conn(0x7f5ee40b8200 msgr2=0x7f5ee40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.399+0000 7f5ee9d9a700 1 --2- 172.21.15.60:0/3240334676 >> v2:172.21.15.105:3300/0 conn(0x7f5ee40b8200 0x7f5ee40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.399+0000 7f5ee9d9a700 1 -- 172.21.15.60:0/3240334676 >> v1:172.21.15.105:6789/0 conn(0x7f5ee40cede0 legacy=0x7f5ee415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.399+0000 7f5ee9d9a700 1 -- 172.21.15.60:0/3240334676 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee40d1590 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.399+0000 7f5ee9d9a700 1 --2- 172.21.15.60:0/3240334676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 0x7f5ee40cfb80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40b2660 tx=0x7f5ed4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de95b759741d412d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5ee9599700 1 -- 172.21.15.60:0/3240334676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4011830 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5ee9599700 1 -- 172.21.15.60:0/3240334676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed4011990 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5ee9599700 1 -- 172.21.15.60:0/3240334676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4011d20 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3240334676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 msgr2=0x7f5ee40cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5eec7ff700 1 --2- 172.21.15.60:0/3240334676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 0x7f5ee40cfb80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40b2660 tx=0x7f5ed4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3240334676 shutdown_connections 2023-07-13T22:30:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5eec7ff700 1 --2- 172.21.15.60:0/3240334676 >> v2:172.21.15.105:3300/0 conn(0x7f5ee40b8200 0x7f5ee40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5eec7ff700 1 --2- 172.21.15.60:0/3240334676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 0x7f5ee40cfb80 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3240334676 >> 172.21.15.60:0/3240334676 conn(0x7f5ee406b2f0 msgr2=0x7f5ee406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3240334676 shutdown_connections 2023-07-13T22:30:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.400+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3240334676 wait complete. 2023-07-13T22:30:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.402+0000 7f5eec7ff700 1 Processor -- start 2023-07-13T22:30:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.402+0000 7f5eec7ff700 1 -- start start 2023-07-13T22:30:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.403+0000 7f5eec7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 0x7f5ee40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.403+0000 7f5eec7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ee40b8200 0x7f5ee40d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.403+0000 7f5eec7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ee40cede0 0x7f5ee40c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.403+0000 7f5eec7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40ca4f0 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.403+0000 7f5eec7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40caab0 con 0x7f5ee40cede0 2023-07-13T22:30:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.403+0000 7f5eec7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee40c7220 con 0x7f5ee40b8200 2023-07-13T22:30:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.403+0000 7f5eea59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 0x7f5ee40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5eea59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 0x7f5ee40caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60612/0 (socket says 172.21.15.60:60612) 2023-07-13T22:30:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5eea59b700 1 -- 172.21.15.60:0/3461534562 learned_addr learned my addr 172.21.15.60:0/3461534562 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5eea59b700 1 -- 172.21.15.60:0/3461534562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ee40b8200 msgr2=0x7f5ee40d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5eea59b700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ee40b8200 0x7f5ee40d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5eea59b700 1 -- 172.21.15.60:0/3461534562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ee40cede0 msgr2=0x7f5ee40c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5eea59b700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ee40cede0 0x7f5ee40c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5eea59b700 1 -- 172.21.15.60:0/3461534562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed4009550 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5eea59b700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 0x7f5ee40caf40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000e9e0 tx=0x7f5ee000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5edaffd700 1 -- 172.21.15.60:0/3461534562 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0009710 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5edaffd700 1 -- 172.21.15.60:0/3461534562 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee000c850 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee40c7500 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.404+0000 7f5edaffd700 1 -- 172.21.15.60:0/3461534562 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000cbc0 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.405+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee40ccc80 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.405+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee40cb480 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.407+0000 7f5edaffd700 1 -- 172.21.15.60:0/3461534562 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5ee000cd20 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.407+0000 7f5edaffd700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5ec806ace0 0x7f5ec806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.408+0000 7f5edaffd700 1 -- 172.21.15.60:0/3461534562 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ee0089630 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.408+0000 7f5ee9d9a700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5ec806ace0 0x7f5ec806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.408+0000 7f5ee9d9a700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5ec806ace0 0x7f5ec806d190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40b7cc0 tx=0x7f5ed4010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.412+0000 7f5edaffd700 1 -- 172.21.15.60:0/3461534562 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5ee00580d0 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.620+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f5ee40d33a0 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.624+0000 7f5edaffd700 1 -- 172.21.15.60:0/3461534562 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v45)=0 v45) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5ee0057c60 con 0x7f5ee40b1ac0 2023-07-13T22:30:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5ec806ace0 msgr2=0x7f5ec806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5ec806ace0 0x7f5ec806d190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40b7cc0 tx=0x7f5ed4010000 comp rx=0 tx=0).stop 2023-07-13T22:30:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 msgr2=0x7f5ee40caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 0x7f5ee40caf40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000e9e0 tx=0x7f5ee000eda0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 shutdown_connections 2023-07-13T22:30:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5ec806ace0 0x7f5ec806d190 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ee40b1ac0 0x7f5ee40caf40 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ee40cede0 0x7f5ee40c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 --2- 172.21.15.60:0/3461534562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ee40b8200 0x7f5ee40d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 >> 172.21.15.60:0/3461534562 conn(0x7f5ee406b2f0 msgr2=0x7f5ee40b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 shutdown_connections 2023-07-13T22:30:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.627+0000 7f5eec7ff700 1 -- 172.21.15.60:0/3461534562 wait complete. 2023-07-13T22:30:01.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 100 2023-07-13T22:30:01.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.852+0000 7f2f14bff700 1 Processor -- start 2023-07-13T22:30:01.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.853+0000 7f2f14bff700 1 -- start start 2023-07-13T22:30:01.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f14bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f100b8200 0x7f2f100cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f14bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 0x7f2f100cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f14bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f100d1870 con 0x7f2f100b1ac0 2023-07-13T22:30:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f14bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f100cf370 con 0x7f2f100b8200 2023-07-13T22:30:01.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f14bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100d12b0 con 0x7f2f100cede0 2023-07-13T22:30:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f0e59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f100b8200 0x7f2f100cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f0e59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f100b8200 0x7f2f100cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60622/0 (socket says 172.21.15.60:60622) 2023-07-13T22:30:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f0e59c700 1 -- 172.21.15.60:0/419272303 learned_addr learned my addr 172.21.15.60:0/419272303 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f07fff700 1 --1- 172.21.15.60:0/419272303 >> v1:172.21.15.60:6789/0 conn(0x7f2f100b1ac0 0x7f2f1015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39416/0 (socket says 172.21.15.60:39416) 2023-07-13T22:30:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.854+0000 7f2f0ed9d700 1 --2- 172.21.15.60:0/419272303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 0x7f2f100cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.855+0000 7f2f0e59c700 1 -- 172.21.15.60:0/419272303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 msgr2=0x7f2f100cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.855+0000 7f2f0e59c700 1 --2- 172.21.15.60:0/419272303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 0x7f2f100cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.855+0000 7f2f0e59c700 1 -- 172.21.15.60:0/419272303 >> v1:172.21.15.60:6789/0 conn(0x7f2f100b1ac0 legacy=0x7f2f1015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.855+0000 7f2f0e59c700 1 -- 172.21.15.60:0/419272303 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100d1590 con 0x7f2f100b8200 2023-07-13T22:30:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.855+0000 7f2f0dd9b700 1 -- 172.21.15.60:0/419272303 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917738868 0 0) 0x7f2f100d1870 con 0x7f2f100b1ac0 2023-07-13T22:30:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.855+0000 7f2f0e59c700 1 --2- 172.21.15.60:0/419272303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b8200 0x7f2f100cf640 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f2f000098f0 tx=0x7f2f00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967ba81946c609c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.855+0000 7f2f0dd9b700 1 -- 172.21.15.60:0/419272303 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00011830 con 0x7f2f100b8200 2023-07-13T22:30:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.855+0000 7f2f0dd9b700 1 -- 172.21.15.60:0/419272303 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00011990 con 0x7f2f100b8200 2023-07-13T22:30:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.855+0000 7f2f0dd9b700 1 -- 172.21.15.60:0/419272303 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00011d20 con 0x7f2f100b8200 2023-07-13T22:30:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.856+0000 7f2f14bff700 1 -- 172.21.15.60:0/419272303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b8200 msgr2=0x7f2f100cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.856+0000 7f2f14bff700 1 --2- 172.21.15.60:0/419272303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b8200 0x7f2f100cf640 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f2f000098f0 tx=0x7f2f00009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.856+0000 7f2f14bff700 1 -- 172.21.15.60:0/419272303 shutdown_connections 2023-07-13T22:30:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.856+0000 7f2f14bff700 1 --2- 172.21.15.60:0/419272303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 0x7f2f100cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.856+0000 7f2f14bff700 1 --2- 172.21.15.60:0/419272303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b8200 0x7f2f100cf640 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.856+0000 7f2f14bff700 1 -- 172.21.15.60:0/419272303 >> 172.21.15.60:0/419272303 conn(0x7f2f1006b2f0 msgr2=0x7f2f1006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:01.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.856+0000 7f2f14bff700 1 -- 172.21.15.60:0/419272303 shutdown_connections 2023-07-13T22:30:01.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.856+0000 7f2f14bff700 1 -- 172.21.15.60:0/419272303 wait complete. 2023-07-13T22:30:01.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.858+0000 7f2f14bff700 1 Processor -- start 2023-07-13T22:30:01.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.858+0000 7f2f14bff700 1 -- start start 2023-07-13T22:30:01.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f14bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b1ac0 0x7f2f100c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f14bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f100b8200 0x7f2f1015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f14bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 0x7f2f101e1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f14bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100c9730 con 0x7f2f100b1ac0 2023-07-13T22:30:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f14bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f101e5870 con 0x7f2f100b8200 2023-07-13T22:30:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f14bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100c7750 con 0x7f2f100cede0 2023-07-13T22:30:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b1ac0 0x7f2f100c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b1ac0 0x7f2f100c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60628/0 (socket says 172.21.15.60:60628) 2023-07-13T22:30:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0e59c700 1 -- 172.21.15.60:0/1804236899 learned_addr learned my addr 172.21.15.60:0/1804236899 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0ed9d700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 0x7f2f101e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0e59c700 1 -- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 msgr2=0x7f2f101e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f07fff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f100b8200 0x7f2f1015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0e59c700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 0x7f2f101e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0e59c700 1 -- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f100b8200 msgr2=0x7f2f1015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0e59c700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f100b8200 0x7f2f1015ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0e59c700 1 -- 172.21.15.60:0/1804236899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f00009550 con 0x7f2f100b1ac0 2023-07-13T22:30:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f0ed9d700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 0x7f2f101e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.859+0000 7f2f07fff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f100b8200 0x7f2f1015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.860+0000 7f2f0e59c700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b1ac0 0x7f2f100c7170 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009c40 tx=0x7f2f00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.860+0000 7f2f06ffd700 1 -- 172.21.15.60:0/1804236899 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f000036a0 con 0x7f2f100b1ac0 2023-07-13T22:30:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.860+0000 7f2f06ffd700 1 -- 172.21.15.60:0/1804236899 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0001f070 con 0x7f2f100b1ac0 2023-07-13T22:30:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.860+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f100c7a30 con 0x7f2f100b1ac0 2023-07-13T22:30:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.860+0000 7f2f06ffd700 1 -- 172.21.15.60:0/1804236899 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00003a40 con 0x7f2f100b1ac0 2023-07-13T22:30:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.860+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f100c7ff0 con 0x7f2f100b1ac0 2023-07-13T22:30:01.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.861+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f101e5a40 con 0x7f2f100b1ac0 2023-07-13T22:30:01.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.862+0000 7f2f06ffd700 1 -- 172.21.15.60:0/1804236899 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f00003ba0 con 0x7f2f100b1ac0 2023-07-13T22:30:01.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.863+0000 7f2f06ffd700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ef006ace0 0x7f2ef006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:01.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.863+0000 7f2f06ffd700 1 -- 172.21.15.60:0/1804236899 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f00021070 con 0x7f2f100b1ac0 2023-07-13T22:30:01.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.863+0000 7f2f07fff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ef006ace0 0x7f2ef006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:01.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.864+0000 7f2f07fff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ef006ace0 0x7f2ef006d190 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8007fd0 tx=0x7f2ef8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:01.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:01.867+0000 7f2f06ffd700 1 -- 172.21.15.60:0/1804236899 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2f00091080 con 0x7f2f100b1ac0 2023-07-13T22:30:02.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.074+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7f2f1004e210 con 0x7f2f100b1ac0 2023-07-13T22:30:02.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.078+0000 7f2f06ffd700 1 -- 172.21.15.60:0/1804236899 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v46)=0 v46) v1 ==== 119+0+0 (secure 0 0 0) 0x7f2f0002b0b0 con 0x7f2f100b1ac0 2023-07-13T22:30:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ef006ace0 msgr2=0x7f2ef006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ef006ace0 0x7f2ef006d190 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8007fd0 tx=0x7f2ef8009040 comp rx=0 tx=0).stop 2023-07-13T22:30:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b1ac0 msgr2=0x7f2f100c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b1ac0 0x7f2f100c7170 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009c40 tx=0x7f2f00004750 comp rx=0 tx=0).stop 2023-07-13T22:30:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 shutdown_connections 2023-07-13T22:30:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2ef006ace0 0x7f2ef006d190 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f100b1ac0 0x7f2f100c7170 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f100cede0 0x7f2f101e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 --2- 172.21.15.60:0/1804236899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f100b8200 0x7f2f1015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.080+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 >> 172.21.15.60:0/1804236899 conn(0x7f2f1006b2f0 msgr2=0x7f2f100d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.081+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 shutdown_connections 2023-07-13T22:30:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.081+0000 7f2f14bff700 1 -- 172.21.15.60:0/1804236899 wait complete. 2023-07-13T22:30:02.077 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 200 2023-07-13T22:30:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.304+0000 7fb72b1ff700 1 Processor -- start 2023-07-13T22:30:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.304+0000 7fb72b1ff700 1 -- start start 2023-07-13T22:30:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.305+0000 7fb72b1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb7240b8200 0x7fb7240cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.305+0000 7fb72b1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7240d1870 con 0x7fb7240cede0 2023-07-13T22:30:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.305+0000 7fb72b1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb724160d80 con 0x7fb7240b1ac0 2023-07-13T22:30:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.305+0000 7fb72b1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7240cf370 con 0x7fb7240b8200 2023-07-13T22:30:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.306+0000 7fb72979c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb7240cede0 0x7fb724160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39430/0 (socket says 172.21.15.60:39430) 2023-07-13T22:30:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.306+0000 7fb72979c700 1 -- 172.21.15.60:0/3920275144 learned_addr learned my addr 172.21.15.60:0/3920275144 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.306+0000 7fb728f9b700 1 --2- 172.21.15.60:0/3920275144 >> v2:172.21.15.105:3300/0 conn(0x7fb7240b8200 0x7fb7240cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.306+0000 7fb7237fe700 1 -- 172.21.15.60:0/3920275144 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629742576 0 0) 0x7fb7240d1870 con 0x7fb7240cede0 2023-07-13T22:30:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.306+0000 7fb7237fe700 1 -- 172.21.15.60:0/3920275144 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7140035b0 con 0x7fb7240cede0 2023-07-13T22:30:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.306+0000 7fb728f9b700 1 -- 172.21.15.60:0/3920275144 >> v1:172.21.15.105:6789/0 conn(0x7fb7240b1ac0 legacy=0x7fb72415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.306+0000 7fb728f9b700 1 -- 172.21.15.60:0/3920275144 >> v1:172.21.15.60:6789/0 conn(0x7fb7240cede0 legacy=0x7fb724160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.306+0000 7fb728f9b700 1 -- 172.21.15.60:0/3920275144 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240d12b0 con 0x7fb7240b8200 2023-07-13T22:30:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.307+0000 7fb728f9b700 1 --2- 172.21.15.60:0/3920275144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7240b8200 0x7fb7240cf640 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb718009840 tx=0x7fb718009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c211667f0080028 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.307+0000 7fb7237fe700 1 -- 172.21.15.60:0/3920275144 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718011830 con 0x7fb7240b8200 2023-07-13T22:30:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.307+0000 7fb7237fe700 1 -- 172.21.15.60:0/3920275144 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb718011990 con 0x7fb7240b8200 2023-07-13T22:30:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.307+0000 7fb7237fe700 1 -- 172.21.15.60:0/3920275144 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718011d20 con 0x7fb7240b8200 2023-07-13T22:30:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.307+0000 7fb72b1ff700 1 -- 172.21.15.60:0/3920275144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7240b8200 msgr2=0x7fb7240cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.307+0000 7fb72b1ff700 1 --2- 172.21.15.60:0/3920275144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7240b8200 0x7fb7240cf640 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb718009840 tx=0x7fb718009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.307+0000 7fb72b1ff700 1 -- 172.21.15.60:0/3920275144 shutdown_connections 2023-07-13T22:30:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.307+0000 7fb72b1ff700 1 --2- 172.21.15.60:0/3920275144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7240b8200 0x7fb7240cf640 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.307+0000 7fb72b1ff700 1 -- 172.21.15.60:0/3920275144 >> 172.21.15.60:0/3920275144 conn(0x7fb72406b2f0 msgr2=0x7fb72406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.308+0000 7fb72b1ff700 1 -- 172.21.15.60:0/3920275144 shutdown_connections 2023-07-13T22:30:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.308+0000 7fb72b1ff700 1 -- 172.21.15.60:0/3920275144 wait complete. 2023-07-13T22:30:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.310+0000 7fb72b1ff700 1 Processor -- start 2023-07-13T22:30:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.310+0000 7fb72b1ff700 1 -- start start 2023-07-13T22:30:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.311+0000 7fb72b1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7240b1ac0 0x7fb72414da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.311+0000 7fb72b1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7240b8200 0x7fb724159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.311+0000 7fb72b1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7240cede0 0x7fb7240cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.311+0000 7fb72b1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb724158650 con 0x7fb7240b1ac0 2023-07-13T22:30:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.311+0000 7fb72b1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7241594b0 con 0x7fb7240b8200 2023-07-13T22:30:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.311+0000 7fb72b1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7241588a0 con 0x7fb7240cede0 2023-07-13T22:30:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb728f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7240b1ac0 0x7fb72414da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb728f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7240b1ac0 0x7fb72414da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60638/0 (socket says 172.21.15.60:60638) 2023-07-13T22:30:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb728f9b700 1 -- 172.21.15.60:0/2690537151 learned_addr learned my addr 172.21.15.60:0/2690537151 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:02.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb723fff700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7240b8200 0x7fb724159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb72979c700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7240cede0 0x7fb7240cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb728f9b700 1 -- 172.21.15.60:0/2690537151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7240cede0 msgr2=0x7fb7240cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb728f9b700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7240cede0 0x7fb7240cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb728f9b700 1 -- 172.21.15.60:0/2690537151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7240b8200 msgr2=0x7fb724159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb728f9b700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7240b8200 0x7fb724159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb728f9b700 1 -- 172.21.15.60:0/2690537151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7180094a0 con 0x7fb7240b1ac0 2023-07-13T22:30:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.312+0000 7fb728f9b700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7240b1ac0 0x7fb72414da00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb7180090b0 tx=0x7fb718004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.313+0000 7fb7217fa700 1 -- 172.21.15.60:0/2690537151 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7180036a0 con 0x7fb7240b1ac0 2023-07-13T22:30:02.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.313+0000 7fb7217fa700 1 -- 172.21.15.60:0/2690537151 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71801f070 con 0x7fb7240b1ac0 2023-07-13T22:30:02.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.313+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb724158b80 con 0x7fb7240b1ac0 2023-07-13T22:30:02.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.313+0000 7fb7217fa700 1 -- 172.21.15.60:0/2690537151 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718003a60 con 0x7fb7240b1ac0 2023-07-13T22:30:02.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.313+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb724159140 con 0x7fb7240b1ac0 2023-07-13T22:30:02.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.314+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb724159790 con 0x7fb7240b1ac0 2023-07-13T22:30:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.315+0000 7fb7217fa700 1 -- 172.21.15.60:0/2690537151 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb718003bc0 con 0x7fb7240b1ac0 2023-07-13T22:30:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.316+0000 7fb7217fa700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb70806ace0 0x7fb70806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.316+0000 7fb7217fa700 1 -- 172.21.15.60:0/2690537151 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb718021070 con 0x7fb7240b1ac0 2023-07-13T22:30:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.316+0000 7fb723fff700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb70806ace0 0x7fb70806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.317+0000 7fb723fff700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb70806ace0 0x7fb70806d190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb72414df40 tx=0x7fb710009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.320+0000 7fb7217fa700 1 -- 172.21.15.60:0/2690537151 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb718055d60 con 0x7fb7240b1ac0 2023-07-13T22:30:02.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.532+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7fb724159300 con 0x7fb7240b1ac0 2023-07-13T22:30:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.536+0000 7fb7217fa700 1 -- 172.21.15.60:0/2690537151 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v47)=0 v47) v1 ==== 128+0+0 (secure 0 0 0) 0x7fb718059380 con 0x7fb7240b1ac0 2023-07-13T22:30:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb70806ace0 msgr2=0x7fb70806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb70806ace0 0x7fb70806d190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb72414df40 tx=0x7fb710009040 comp rx=0 tx=0).stop 2023-07-13T22:30:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7240b1ac0 msgr2=0x7fb72414da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7240b1ac0 0x7fb72414da00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb7180090b0 tx=0x7fb718004750 comp rx=0 tx=0).stop 2023-07-13T22:30:02.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 shutdown_connections 2023-07-13T22:30:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb70806ace0 0x7fb70806d190 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7240b1ac0 0x7fb72414da00 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7240cede0 0x7fb7240cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 --2- 172.21.15.60:0/2690537151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7240b8200 0x7fb724159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 >> 172.21.15.60:0/2690537151 conn(0x7fb72406b2f0 msgr2=0x7fb7240d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 shutdown_connections 2023-07-13T22:30:02.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.540+0000 7fb72b1ff700 1 -- 172.21.15.60:0/2690537151 wait complete. 2023-07-13T22:30:02.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-13T22:30:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.762+0000 7fe373dff700 1 Processor -- start 2023-07-13T22:30:02.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.763+0000 7fe373dff700 1 -- start start 2023-07-13T22:30:02.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe373dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0b8200 0x7fe36c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe373dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe36c0b1ac0 0x7fe36c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe373dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe36c0cede0 0x7fe36c15ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe373dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c0d1870 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe373dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c0cf370 con 0x7fe36c0cede0 2023-07-13T22:30:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe373dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0d12b0 con 0x7fe36c0b8200 2023-07-13T22:30:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37139a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe36c0b1ac0 0x7fe36c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37139a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe36c0b1ac0 0x7fe36c0cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60640/0 (socket says 172.21.15.60:60640) 2023-07-13T22:30:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37139a700 1 -- 172.21.15.60:0/1899432872 learned_addr learned my addr 172.21.15.60:0/1899432872 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe371b9b700 1 --2- 172.21.15.60:0/1899432872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0b8200 0x7fe36c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37239c700 1 --2- 172.21.15.60:0/1899432872 >> v2:172.21.15.105:3300/0 conn(0x7fe36c0cede0 0x7fe36c15ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37139a700 1 -- 172.21.15.60:0/1899432872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0b8200 msgr2=0x7fe36c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37139a700 1 --2- 172.21.15.60:0/1899432872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0b8200 0x7fe36c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37139a700 1 -- 172.21.15.60:0/1899432872 >> v2:172.21.15.105:3300/0 conn(0x7fe36c0cede0 msgr2=0x7fe36c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37139a700 1 --2- 172.21.15.60:0/1899432872 >> v2:172.21.15.105:3300/0 conn(0x7fe36c0cede0 0x7fe36c15ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37139a700 1 -- 172.21.15.60:0/1899432872 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0d1590 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.764+0000 7fe37239c700 1 --2- 172.21.15.60:0/1899432872 >> v2:172.21.15.105:3300/0 conn(0x7fe36c0cede0 0x7fe36c15ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:02.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.765+0000 7fe37139a700 1 --2- 172.21.15.60:0/1899432872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 0x7fe36c0cfb80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0b8d70 tx=0x7fe35c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb4da75ce134f3f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:02.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.765+0000 7fe370b99700 1 -- 172.21.15.60:0/1899432872 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00b870 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.765+0000 7fe370b99700 1 -- 172.21.15.60:0/1899432872 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c00b9d0 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.765+0000 7fe370b99700 1 -- 172.21.15.60:0/1899432872 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00bd60 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.765+0000 7fe373dff700 1 -- 172.21.15.60:0/1899432872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 msgr2=0x7fe36c0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.765+0000 7fe373dff700 1 --2- 172.21.15.60:0/1899432872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 0x7fe36c0cfb80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0b8d70 tx=0x7fe35c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.766+0000 7fe373dff700 1 -- 172.21.15.60:0/1899432872 shutdown_connections 2023-07-13T22:30:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.766+0000 7fe373dff700 1 --2- 172.21.15.60:0/1899432872 >> v2:172.21.15.105:3300/0 conn(0x7fe36c0cede0 0x7fe36c15ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.766+0000 7fe373dff700 1 --2- 172.21.15.60:0/1899432872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0b8200 0x7fe36c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.766+0000 7fe373dff700 1 --2- 172.21.15.60:0/1899432872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 0x7fe36c0cfb80 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.766+0000 7fe373dff700 1 -- 172.21.15.60:0/1899432872 >> 172.21.15.60:0/1899432872 conn(0x7fe36c06b2f0 msgr2=0x7fe36c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.766+0000 7fe373dff700 1 -- 172.21.15.60:0/1899432872 shutdown_connections 2023-07-13T22:30:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.766+0000 7fe373dff700 1 -- 172.21.15.60:0/1899432872 wait complete. 2023-07-13T22:30:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.768+0000 7fe373dff700 1 Processor -- start 2023-07-13T22:30:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.768+0000 7fe373dff700 1 -- start start 2023-07-13T22:30:02.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.769+0000 7fe373dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 0x7fe36c1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe373dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe36c0b8200 0x7fe36c0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe373dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0cede0 0x7fe36c0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe373dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0c7810 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe373dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0c6ca0 con 0x7fe36c0b8200 2023-07-13T22:30:02.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe373dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0c6f70 con 0x7fe36c0cede0 2023-07-13T22:30:02.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe371b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 0x7fe36c1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe371b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 0x7fe36c1570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60654/0 (socket says 172.21.15.60:60654) 2023-07-13T22:30:02.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe371b9b700 1 -- 172.21.15.60:0/1555179020 learned_addr learned my addr 172.21.15.60:0/1555179020 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:02.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe37239c700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0cede0 0x7fe36c0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe371b9b700 1 -- 172.21.15.60:0/1555179020 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0cede0 msgr2=0x7fe36c0c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe37139a700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe36c0b8200 0x7fe36c0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe371b9b700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0cede0 0x7fe36c0c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe371b9b700 1 -- 172.21.15.60:0/1555179020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe36c0b8200 msgr2=0x7fe36c0c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe371b9b700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe36c0b8200 0x7fe36c0c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe371b9b700 1 -- 172.21.15.60:0/1555179020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c009550 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe37139a700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe36c0b8200 0x7fe36c0c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:02.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.770+0000 7fe371b9b700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 0x7fe36c1570c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e810 tx=0x7fe36800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.771+0000 7fe3627fc700 1 -- 172.21.15.60:0/1555179020 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368013610 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.771+0000 7fe3627fc700 1 -- 172.21.15.60:0/1555179020 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe368013770 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.771+0000 7fe3627fc700 1 -- 172.21.15.60:0/1555179020 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368013b20 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.771+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c0c7250 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.771+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c154630 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.772+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c04e210 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.773+0000 7fe3627fc700 1 -- 172.21.15.60:0/1555179020 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe368013c80 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.774+0000 7fe3627fc700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe35006ace0 0x7fe35006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.774+0000 7fe3627fc700 1 -- 172.21.15.60:0/1555179020 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe368089250 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.774+0000 7fe37139a700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe35006ace0 0x7fe35006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.775+0000 7fe37139a700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe35006ace0 0x7fe35006d190 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009130 tx=0x7fe35c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.778+0000 7fe3627fc700 1 -- 172.21.15.60:0/1555179020 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe368054270 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.982+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7fe36c0b4f70 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.982+0000 7fe3627fc700 1 -- 172.21.15.60:0/1555179020 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v47) v1 ==== 106+0+4 (secure 0 0 0) 0x7fe368057890 con 0x7fe36c0b1ac0 2023-07-13T22:30:02.961 INFO:teuthology.orchestra.run.smithi060.stdout:100 2023-07-13T22:30:02.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe35006ace0 msgr2=0x7fe35006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe35006ace0 0x7fe35006d190 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009130 tx=0x7fe35c008030 comp rx=0 tx=0).stop 2023-07-13T22:30:02.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 msgr2=0x7fe36c1570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:02.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 0x7fe36c1570c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e810 tx=0x7fe36800ebd0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 shutdown_connections 2023-07-13T22:30:02.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe35006ace0 0x7fe35006d190 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36c0b1ac0 0x7fe36c1570c0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe36c0cede0 0x7fe36c0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 --2- 172.21.15.60:0/1555179020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe36c0b8200 0x7fe36c0c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:02.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 >> 172.21.15.60:0/1555179020 conn(0x7fe36c06b2f0 msgr2=0x7fe36c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:02.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.984+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 shutdown_connections 2023-07-13T22:30:02.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:02.985+0000 7fe373dff700 1 -- 172.21.15.60:0/1555179020 wait complete. 2023-07-13T22:30:02.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2023-07-13T22:30:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.208+0000 7f35ac7ff700 1 Processor -- start 2023-07-13T22:30:03.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.208+0000 7f35ac7ff700 1 -- start start 2023-07-13T22:30:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.209+0000 7f35ac7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35a40b1ac0 0x7f35a40d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.209+0000 7f35ac7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40d1870 con 0x7f35a40cfd50 2023-07-13T22:30:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.209+0000 7f35ac7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40cf370 con 0x7f35a40b8200 2023-07-13T22:30:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.209+0000 7f35ac7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40d12b0 con 0x7f35a40b1ac0 2023-07-13T22:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.209+0000 7f35a9d9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f35a40cfd50 0x7f35a40d23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39446/0 (socket says 172.21.15.60:39446) 2023-07-13T22:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.209+0000 7f35a9d9a700 1 -- 172.21.15.60:0/1020383774 learned_addr learned my addr 172.21.15.60:0/1020383774 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35aad9c700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35a40b1ac0 0x7f35a40d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362627462 0 0) 0x7f35a40d1870 con 0x7f35a40cfd50 2023-07-13T22:30:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35900035b0 con 0x7f35a40cfd50 2023-07-13T22:30:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218177510 0 0) 0x7f35a40cf370 con 0x7f35a40b8200 2023-07-13T22:30:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35a40d1870 con 0x7f35a40b8200 2023-07-13T22:30:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3629732960 0 0) 0x7f35900035b0 con 0x7f35a40cfd50 2023-07-13T22:30:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35a40b1ac0 msgr2=0x7f35a40d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35a40b1ac0 0x7f35a40d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 >> v1:172.21.15.105:6789/0 conn(0x7f35a40b8200 legacy=0x7f35a40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40d1590 con 0x7f35a40cfd50 2023-07-13T22:30:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f3594001670 con 0x7f35a40cfd50 2023-07-13T22:30:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 >> v1:172.21.15.60:6789/0 conn(0x7f35a40cfd50 legacy=0x7f35a40d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3590003bc0 0x7f3590003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3590004850 0x7f3590004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35a40b1ac0 0x7f3590005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40d12b0 con 0x7f3590003bc0 2023-07-13T22:30:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359000bac0 con 0x7f3590004850 2023-07-13T22:30:03.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.210+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359000bc80 con 0x7f35a40b1ac0 2023-07-13T22:30:03.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.211+0000 7f35aa59b700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3590003bc0 0x7f3590003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.211+0000 7f35a9d9a700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3590004850 0x7f3590004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.211+0000 7f35aad9c700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35a40b1ac0 0x7f3590005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.211+0000 7f35aa59b700 1 -- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35a40b1ac0 msgr2=0x7f3590005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:03.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.211+0000 7f35aa59b700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35a40b1ac0 0x7f3590005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.211+0000 7f35aa59b700 1 -- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3590004850 msgr2=0x7f3590004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:03.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.211+0000 7f35aa59b700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3590004850 0x7f3590004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.211+0000 7f35aa59b700 1 -- 172.21.15.60:0/1020383774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35900035b0 con 0x7f3590003bc0 2023-07-13T22:30:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.211+0000 7f35aa59b700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3590003bc0 0x7f3590003fb0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f35a0007fd0 tx=0x7f3590005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82455a74a5581458 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0001430 con 0x7f3590003bc0 2023-07-13T22:30:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a00052d0 con 0x7f3590003bc0 2023-07-13T22:30:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35a9599700 1 -- 172.21.15.60:0/1020383774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0001d40 con 0x7f3590003bc0 2023-07-13T22:30:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35ac7ff700 1 -- 172.21.15.60:0/1020383774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3590003bc0 msgr2=0x7f3590003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35ac7ff700 1 --2- 172.21.15.60:0/1020383774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3590003bc0 0x7f3590003fb0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f35a0007fd0 tx=0x7f3590005880 comp rx=0 tx=0).stop 2023-07-13T22:30:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35aad9c700 1 -- 172.21.15.60:0/1020383774 reap_dead start 2023-07-13T22:30:03.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35ac7ff700 1 -- 172.21.15.60:0/1020383774 shutdown_connections 2023-07-13T22:30:03.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35ac7ff700 1 -- 172.21.15.60:0/1020383774 >> 172.21.15.60:0/1020383774 conn(0x7f35a406b2f0 msgr2=0x7f35a406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:03.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35ac7ff700 1 -- 172.21.15.60:0/1020383774 shutdown_connections 2023-07-13T22:30:03.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.212+0000 7f35ac7ff700 1 -- 172.21.15.60:0/1020383774 wait complete. 2023-07-13T22:30:03.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.214+0000 7f35ac7ff700 1 Processor -- start 2023-07-13T22:30:03.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.214+0000 7f35ac7ff700 1 -- start start 2023-07-13T22:30:03.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.215+0000 7f35ac7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3590003bc0 0x7f35a4151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.215+0000 7f35ac7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35a4152090 0x7f35a4152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.215+0000 7f35ac7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35a4155110 0x7f35a41555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.215+0000 7f35ac7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a4153550 con 0x7f35a4152090 2023-07-13T22:30:03.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.215+0000 7f35ac7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a4153830 con 0x7f35a4155110 2023-07-13T22:30:03.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.215+0000 7f35ac7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a4153b10 con 0x7f3590003bc0 2023-07-13T22:30:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.215+0000 7f35a9d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35a4152090 0x7f35a4152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.215+0000 7f35a9d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35a4152090 0x7f35a4152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60666/0 (socket says 172.21.15.60:60666) 2023-07-13T22:30:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.215+0000 7f35a9d9a700 1 -- 172.21.15.60:0/3730993977 learned_addr learned my addr 172.21.15.60:0/3730993977 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35aa59b700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3590003bc0 0x7f35a4151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35aad9c700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35a4155110 0x7f35a41555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35a9d9a700 1 -- 172.21.15.60:0/3730993977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3590003bc0 msgr2=0x7f35a4151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35a9d9a700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3590003bc0 0x7f35a4151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35a9d9a700 1 -- 172.21.15.60:0/3730993977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35a4155110 msgr2=0x7f35a41555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35a9d9a700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35a4155110 0x7f35a41555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35a9d9a700 1 -- 172.21.15.60:0/3730993977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40cb270 con 0x7f35a4152090 2023-07-13T22:30:03.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35a9d9a700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35a4152090 0x7f35a4152a90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3594004f50 tx=0x7f35940073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:03.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f359affd700 1 -- 172.21.15.60:0/3730993977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594011580 con 0x7f35a4152090 2023-07-13T22:30:03.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f359affd700 1 -- 172.21.15.60:0/3730993977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3594011700 con 0x7f35a4152090 2023-07-13T22:30:03.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f359affd700 1 -- 172.21.15.60:0/3730993977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594011ab0 con 0x7f35a4152090 2023-07-13T22:30:03.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a4155af0 con 0x7f35a4152090 2023-07-13T22:30:03.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.216+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a4151cd0 con 0x7f35a4152090 2023-07-13T22:30:03.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.217+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a4153cd0 con 0x7f35a4152090 2023-07-13T22:30:03.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.218+0000 7f359affd700 1 -- 172.21.15.60:0/3730993977 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3594011c10 con 0x7f35a4152090 2023-07-13T22:30:03.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.219+0000 7f359affd700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f358806ace0 0x7f358806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.219+0000 7f359affd700 1 -- 172.21.15.60:0/3730993977 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3594089510 con 0x7f35a4152090 2023-07-13T22:30:03.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.220+0000 7f35aa59b700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f358806ace0 0x7f358806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.220+0000 7f35aa59b700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f358806ace0 0x7f358806d190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005040 tx=0x7f35a000a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:03.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.223+0000 7f359affd700 1 -- 172.21.15.60:0/3730993977 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f35940545b0 con 0x7f35a4152090 2023-07-13T22:30:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.432+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7f35a40cede0 con 0x7f35a4152090 2023-07-13T22:30:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.436+0000 7f359affd700 1 -- 172.21.15.60:0/3730993977 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v48)=0 v48) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3594057bd0 con 0x7f35a4152090 2023-07-13T22:30:03.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f358806ace0 msgr2=0x7f358806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f358806ace0 0x7f358806d190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005040 tx=0x7f35a000a000 comp rx=0 tx=0).stop 2023-07-13T22:30:03.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35a4152090 msgr2=0x7f35a4152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35a4152090 0x7f35a4152a90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3594004f50 tx=0x7f35940073f0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 shutdown_connections 2023-07-13T22:30:03.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f358806ace0 0x7f358806d190 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35a4155110 0x7f35a41555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3590003bc0 0x7f35a4151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 --2- 172.21.15.60:0/3730993977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35a4152090 0x7f35a4152a90 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 >> 172.21.15.60:0/3730993977 conn(0x7f35a406b2f0 msgr2=0x7f35a40734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:03.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 shutdown_connections 2023-07-13T22:30:03.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.438+0000 7f35ac7ff700 1 -- 172.21.15.60:0/3730993977 wait complete. 2023-07-13T22:30:03.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2023-07-13T22:30:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.662+0000 7f71ed3ff700 1 Processor -- start 2023-07-13T22:30:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.662+0000 7f71ed3ff700 1 -- start start 2023-07-13T22:30:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.663+0000 7f71ed3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71e8159260 0x7f71e80ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.663+0000 7f71ed3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80cc410 0x7f71e80cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.663+0000 7f71ed3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71e80703e0 0x7f71e8070840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.663+0000 7f71ed3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e8070d80 con 0x7f71e8159260 2023-07-13T22:30:03.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.663+0000 7f71ed3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e80cf640 con 0x7f71e80703e0 2023-07-13T22:30:03.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.663+0000 7f71ed3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80ccf80 con 0x7f71e80cc410 2023-07-13T22:30:03.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e6ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71e8159260 0x7f71e80ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e6ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71e8159260 0x7f71e80ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60682/0 (socket says 172.21.15.60:60682) 2023-07-13T22:30:03.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e6ffd700 1 -- 172.21.15.60:0/141588951 learned_addr learned my addr 172.21.15.60:0/141588951 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:03.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e77fe700 1 --2- 172.21.15.60:0/141588951 >> v2:172.21.15.105:3300/0 conn(0x7f71e80703e0 0x7f71e8070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e67fc700 1 --2- 172.21.15.60:0/141588951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80cc410 0x7f71e80cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e6ffd700 1 -- 172.21.15.60:0/141588951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80cc410 msgr2=0x7f71e80cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e6ffd700 1 --2- 172.21.15.60:0/141588951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80cc410 0x7f71e80cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e6ffd700 1 -- 172.21.15.60:0/141588951 >> v2:172.21.15.105:3300/0 conn(0x7f71e80703e0 msgr2=0x7f71e8070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e6ffd700 1 --2- 172.21.15.60:0/141588951 >> v2:172.21.15.105:3300/0 conn(0x7f71e80703e0 0x7f71e8070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e6ffd700 1 -- 172.21.15.60:0/141588951 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80c9b40 con 0x7f71e8159260 2023-07-13T22:30:03.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e67fc700 1 --2- 172.21.15.60:0/141588951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80cc410 0x7f71e80cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:03.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.664+0000 7f71e6ffd700 1 --2- 172.21.15.60:0/141588951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e8159260 0x7f71e80ca900 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f71e80cf300 tx=0x7f71d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e90387e6299d38a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:03.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71e5ffb700 1 -- 172.21.15.60:0/141588951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d800f8a0 con 0x7f71e8159260 2023-07-13T22:30:03.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71e5ffb700 1 -- 172.21.15.60:0/141588951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d800fa00 con 0x7f71e8159260 2023-07-13T22:30:03.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71e5ffb700 1 -- 172.21.15.60:0/141588951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d800fd90 con 0x7f71e8159260 2023-07-13T22:30:03.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71ed3ff700 1 -- 172.21.15.60:0/141588951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e8159260 msgr2=0x7f71e80ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/141588951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e8159260 0x7f71e80ca900 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f71e80cf300 tx=0x7f71d8009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71ed3ff700 1 -- 172.21.15.60:0/141588951 shutdown_connections 2023-07-13T22:30:03.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/141588951 >> v2:172.21.15.105:3300/0 conn(0x7f71e80703e0 0x7f71e8070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/141588951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e8159260 0x7f71e80ca900 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/141588951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80cc410 0x7f71e80cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71ed3ff700 1 -- 172.21.15.60:0/141588951 >> 172.21.15.60:0/141588951 conn(0x7f71e80cbf50 msgr2=0x7f71e80ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:03.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71ed3ff700 1 -- 172.21.15.60:0/141588951 shutdown_connections 2023-07-13T22:30:03.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.665+0000 7f71ed3ff700 1 -- 172.21.15.60:0/141588951 wait complete. 2023-07-13T22:30:03.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.668+0000 7f71ed3ff700 1 Processor -- start 2023-07-13T22:30:03.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.668+0000 7f71ed3ff700 1 -- start start 2023-07-13T22:30:03.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.669+0000 7f71ed3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80703e0 0x7f71e80c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.669+0000 7f71ed3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e80cc410 0x7f71e80c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.669+0000 7f71ed3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e8159260 0x7f71e80ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.669+0000 7f71ed3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80c8890 con 0x7f71e80cc410 2023-07-13T22:30:03.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.669+0000 7f71ed3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80c72b0 con 0x7f71e8159260 2023-07-13T22:30:03.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.669+0000 7f71ed3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80c7590 con 0x7f71e80703e0 2023-07-13T22:30:03.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e80cc410 0x7f71e80c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e6ffd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80703e0 0x7f71e80c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e80cc410 0x7f71e80c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60698/0 (socket says 172.21.15.60:60698) 2023-07-13T22:30:03.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e67fc700 1 -- 172.21.15.60:0/823685721 learned_addr learned my addr 172.21.15.60:0/823685721 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:03.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e77fe700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e8159260 0x7f71e80ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e67fc700 1 -- 172.21.15.60:0/823685721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80703e0 msgr2=0x7f71e80c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e67fc700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80703e0 0x7f71e80c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e67fc700 1 -- 172.21.15.60:0/823685721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e8159260 msgr2=0x7f71e80ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e67fc700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e8159260 0x7f71e80ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e67fc700 1 -- 172.21.15.60:0/823685721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d8009550 con 0x7f71e80cc410 2023-07-13T22:30:03.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.670+0000 7f71e67fc700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e80cc410 0x7f71e80c8350 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00e810 tx=0x7f71dc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:03.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.671+0000 7f71cf7fe700 1 -- 172.21.15.60:0/823685721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc013610 con 0x7f71e80cc410 2023-07-13T22:30:03.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.671+0000 7f71cf7fe700 1 -- 172.21.15.60:0/823685721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71dc013770 con 0x7f71e80cc410 2023-07-13T22:30:03.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.671+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e80c7870 con 0x7f71e80cc410 2023-07-13T22:30:03.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.671+0000 7f71cf7fe700 1 -- 172.21.15.60:0/823685721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc013b20 con 0x7f71e80cc410 2023-07-13T22:30:03.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.671+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e80c7e30 con 0x7f71e80cc410 2023-07-13T22:30:03.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.672+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e804e210 con 0x7f71e80cc410 2023-07-13T22:30:03.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.673+0000 7f71cf7fe700 1 -- 172.21.15.60:0/823685721 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f71dc013c80 con 0x7f71e80cc410 2023-07-13T22:30:03.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.674+0000 7f71cf7fe700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f71d406ace0 0x7f71d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:03.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.674+0000 7f71cf7fe700 1 -- 172.21.15.60:0/823685721 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71dc088fb0 con 0x7f71e80cc410 2023-07-13T22:30:03.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.674+0000 7f71e6ffd700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f71d406ace0 0x7f71d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:03.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.675+0000 7f71e6ffd700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f71d406ace0 0x7f71d406d190 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009c10 tx=0x7f71d801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:03.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.678+0000 7f71cf7fe700 1 -- 172.21.15.60:0/823685721 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f71dc057a50 con 0x7f71e80cc410 2023-07-13T22:30:03.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.886+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) v1 -- 0x7f71e80cf050 con 0x7f71e80cc410 2023-07-13T22:30:03.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.887+0000 7f71cf7fe700 1 -- 172.21.15.60:0/823685721 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v48) v1 ==== 106+0+8 (secure 0 0 0) 0x7f71dc0575e0 con 0x7f71e80cc410 2023-07-13T22:30:03.866 INFO:teuthology.orchestra.run.smithi060.stdout:1048576 2023-07-13T22:30:03.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.889+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f71d406ace0 msgr2=0x7f71d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.889+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f71d406ace0 0x7f71d406d190 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009c10 tx=0x7f71d801b040 comp rx=0 tx=0).stop 2023-07-13T22:30:03.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e80cc410 msgr2=0x7f71e80c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:03.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e80cc410 0x7f71e80c8350 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00e810 tx=0x7f71dc00eb20 comp rx=0 tx=0).stop 2023-07-13T22:30:03.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 shutdown_connections 2023-07-13T22:30:03.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f71d406ace0 0x7f71d406d190 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e8159260 0x7f71e80ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e80703e0 0x7f71e80c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 --2- 172.21.15.60:0/823685721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e80cc410 0x7f71e80c8350 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:03.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 >> 172.21.15.60:0/823685721 conn(0x7f71e80cbf50 msgr2=0x7f71e815bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:03.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 shutdown_connections 2023-07-13T22:30:03.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:03.890+0000 7f71ed3ff700 1 -- 172.21.15.60:0/823685721 wait complete. 2023-07-13T22:30:03.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-07-13T22:30:04.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.113+0000 7f547a1ff700 1 Processor -- start 2023-07-13T22:30:04.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.114+0000 7f547a1ff700 1 -- start start 2023-07-13T22:30:04.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.115+0000 7f547a1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5474159260 0x7f54740ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:04.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.115+0000 7f547a1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5474070af0 0x7f54740cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:04.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.115+0000 7f547a1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54740cf640 con 0x7f54740cc410 2023-07-13T22:30:04.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.115+0000 7f547a1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54740ccf80 con 0x7f5474159260 2023-07-13T22:30:04.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.115+0000 7f547a1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54740c9b40 con 0x7f5474070af0 2023-07-13T22:30:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.115+0000 7f54737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5474159260 0x7f54740ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.115+0000 7f54737fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5474159260 0x7f54740ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60704/0 (socket says 172.21.15.60:60704) 2023-07-13T22:30:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.115+0000 7f54737fe700 1 -- 172.21.15.60:0/1759863142 learned_addr learned my addr 172.21.15.60:0/1759863142 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f5473fff700 1 --2- 172.21.15.60:0/1759863142 >> v2:172.21.15.105:3300/0 conn(0x7f5474070af0 0x7f54740cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f54737fe700 1 -- 172.21.15.60:0/1759863142 >> v2:172.21.15.105:3300/0 conn(0x7f5474070af0 msgr2=0x7f54740cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:04.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f54737fe700 1 --2- 172.21.15.60:0/1759863142 >> v2:172.21.15.105:3300/0 conn(0x7f5474070af0 0x7f54740cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f54737fe700 1 -- 172.21.15.60:0/1759863142 >> v1:172.21.15.60:6789/0 conn(0x7f54740cc410 legacy=0x7f54740703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:04.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f54737fe700 1 -- 172.21.15.60:0/1759863142 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54740c9e20 con 0x7f5474159260 2023-07-13T22:30:04.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f5472ffd700 1 -- 172.21.15.60:0/1759863142 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326230637 0 0) 0x7f54740cf640 con 0x7f54740cc410 2023-07-13T22:30:04.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f54737fe700 1 --2- 172.21.15.60:0/1759863142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474159260 0x7f54740ca900 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5474070370 tx=0x7f546400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505607333e9010b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:04.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f5472ffd700 1 -- 172.21.15.60:0/1759863142 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400dd30 con 0x7f5474159260 2023-07-13T22:30:04.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f5472ffd700 1 -- 172.21.15.60:0/1759863142 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546400de90 con 0x7f5474159260 2023-07-13T22:30:04.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.116+0000 7f5472ffd700 1 -- 172.21.15.60:0/1759863142 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464010690 con 0x7f5474159260 2023-07-13T22:30:04.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.117+0000 7f547a1ff700 1 -- 172.21.15.60:0/1759863142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474159260 msgr2=0x7f54740ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:04.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.117+0000 7f547a1ff700 1 --2- 172.21.15.60:0/1759863142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474159260 0x7f54740ca900 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5474070370 tx=0x7f546400be70 comp rx=0 tx=0).stop 2023-07-13T22:30:04.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.117+0000 7f547a1ff700 1 -- 172.21.15.60:0/1759863142 shutdown_connections 2023-07-13T22:30:04.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.117+0000 7f547a1ff700 1 --2- 172.21.15.60:0/1759863142 >> v2:172.21.15.105:3300/0 conn(0x7f5474070af0 0x7f54740cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.117+0000 7f547a1ff700 1 --2- 172.21.15.60:0/1759863142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474159260 0x7f54740ca900 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.117+0000 7f547a1ff700 1 -- 172.21.15.60:0/1759863142 >> 172.21.15.60:0/1759863142 conn(0x7f54740cbf50 msgr2=0x7f54740ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:04.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.117+0000 7f547a1ff700 1 -- 172.21.15.60:0/1759863142 shutdown_connections 2023-07-13T22:30:04.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.117+0000 7f547a1ff700 1 -- 172.21.15.60:0/1759863142 wait complete. 2023-07-13T22:30:04.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.119+0000 7f547a1ff700 1 Processor -- start 2023-07-13T22:30:04.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.119+0000 7f547a1ff700 1 -- start start 2023-07-13T22:30:04.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f547a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5474070af0 0x7f5474149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:04.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f547a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54740cc410 0x7f5474149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:04.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f547a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5474159260 0x7f54741f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:04.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f547a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547415c4a0 con 0x7f54740cc410 2023-07-13T22:30:04.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f547a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5474158650 con 0x7f5474070af0 2023-07-13T22:30:04.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f547a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54741596d0 con 0x7f5474159260 2023-07-13T22:30:04.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f546bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54740cc410 0x7f5474149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:04.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f546bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54740cc410 0x7f5474149da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60706/0 (socket says 172.21.15.60:60706) 2023-07-13T22:30:04.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f546bfff700 1 -- 172.21.15.60:0/2498076562 learned_addr learned my addr 172.21.15.60:0/2498076562 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:04.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f54737fe700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5474070af0 0x7f5474149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:04.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f5473fff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5474159260 0x7f54741f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:04.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f546bfff700 1 -- 172.21.15.60:0/2498076562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5474159260 msgr2=0x7f54741f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:04.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f546bfff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5474159260 0x7f54741f18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f546bfff700 1 -- 172.21.15.60:0/2498076562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5474070af0 msgr2=0x7f5474149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f546bfff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5474070af0 0x7f5474149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.120+0000 7f546bfff700 1 -- 172.21.15.60:0/2498076562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546400b7f0 con 0x7f54740cc410 2023-07-13T22:30:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.121+0000 7f546bfff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54740cc410 0x7f5474149da0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f545c0011e0 tx=0x7f545c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.121+0000 7f5470ff9700 1 -- 172.21.15.60:0/2498076562 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0054e0 con 0x7f54740cc410 2023-07-13T22:30:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.121+0000 7f5470ff9700 1 -- 172.21.15.60:0/2498076562 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c003680 con 0x7f54740cc410 2023-07-13T22:30:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.121+0000 7f5470ff9700 1 -- 172.21.15.60:0/2498076562 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c003a30 con 0x7f54740cc410 2023-07-13T22:30:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.121+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54741599b0 con 0x7f54740cc410 2023-07-13T22:30:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.121+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547415c8c0 con 0x7f54740cc410 2023-07-13T22:30:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.122+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547415c5e0 con 0x7f54740cc410 2023-07-13T22:30:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.123+0000 7f5470ff9700 1 -- 172.21.15.60:0/2498076562 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f545c003b90 con 0x7f54740cc410 2023-07-13T22:30:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.124+0000 7f5470ff9700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f545406ace0 0x7f545406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.124+0000 7f5470ff9700 1 -- 172.21.15.60:0/2498076562 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f545c0889c0 con 0x7f54740cc410 2023-07-13T22:30:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.124+0000 7f54737fe700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f545406ace0 0x7f545406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.125+0000 7f54737fe700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f545406ace0 0x7f545406d190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f546400b6f0 tx=0x7f5464012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.129+0000 7f5470ff9700 1 -- 172.21.15.60:0/2498076562 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f547415c5e0 con 0x7f54740cc410 2023-07-13T22:30:04.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.334+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7f547415c5e0 con 0x7f54740cc410 2023-07-13T22:30:04.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.334+0000 7f5470ff9700 1 -- 172.21.15.60:0/2498076562 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v48) v1 ==== 112+0+10 (secure 0 0 0) 0x7f547415c5e0 con 0x7f54740cc410 2023-07-13T22:30:04.313 INFO:teuthology.orchestra.run.smithi060.stdout:60.000000 2023-07-13T22:30:04.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.336+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f545406ace0 msgr2=0x7f545406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:04.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.336+0000 7f547a1ff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f545406ace0 0x7f545406d190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f546400b6f0 tx=0x7f5464012000 comp rx=0 tx=0).stop 2023-07-13T22:30:04.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.336+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54740cc410 msgr2=0x7f5474149da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:04.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.336+0000 7f547a1ff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54740cc410 0x7f5474149da0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f545c0011e0 tx=0x7f545c00cae0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.336+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 shutdown_connections 2023-07-13T22:30:04.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.337+0000 7f547a1ff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f545406ace0 0x7f545406d190 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.337+0000 7f547a1ff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5474159260 0x7f54741f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.337+0000 7f547a1ff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5474070af0 0x7f5474149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.337+0000 7f547a1ff700 1 --2- 172.21.15.60:0/2498076562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54740cc410 0x7f5474149da0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:04.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.337+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 >> 172.21.15.60:0/2498076562 conn(0x7f54740cbf50 msgr2=0x7f54740ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:04.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.337+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 shutdown_connections 2023-07-13T22:30:04.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:04.337+0000 7f547a1ff700 1 -- 172.21.15.60:0/2498076562 wait complete. 2023-07-13T22:30:04.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:30:04.333 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:30:04.334 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:30:04.334 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:30:04.334 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:30:04.335 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:04.335 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:30:04.335 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:30:04.335 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:30:04.336 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:30:04.336 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:04.336 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:30:04.339 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:30:04.340 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:30:04.340 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:30:04.340 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:04.341 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:30:04.341 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:30:04.341 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:30:04.342 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:30:04.342 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:30:04.342 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:30:04.342 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:30:04.343 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:30:04.343 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:04.343 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:30:04.343 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:30:04.491 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.157131.asok 2023-07-13T22:30:04.667 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.157131.asok 2023-07-13T22:30:04.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:04.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.157131.asok mds_sessions 2023-07-13T22:30:04.901 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:30:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8586, 2023-07-13T22:30:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:30:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:30:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:30:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 8586 2023-07-13T22:30:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:30:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:30:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:30:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1730080076 2023-07-13T22:30:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.8586 v1:192.168.0.1:0/1730080076", 2023-07-13T22:30:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "v1:192.168.0.1:0/1730080076", 2023-07-13T22:30:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions": [ 2023-07-13T22:30:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": 0, 2023-07-13T22:30:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "addrs": { 2023-07-13T22:30:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2023-07-13T22:30:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2023-07-13T22:30:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6834", 2023-07-13T22:30:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3527792396 2023-07-13T22:30:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:30:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6836", 2023-07-13T22:30:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3527792396 2023-07-13T22:30:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:30:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "seq": 0, 2023-07-13T22:30:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_gen": 0, 2023-07-13T22:30:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_ttl": "2023-07-13T22:30:59.080768+0000", 2023-07-13T22:30:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "last_cap_renew_request": "2023-07-13T22:29:59.080768+0000", 2023-07-13T22:30:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_renew_seq": 2, 2023-07-13T22:30:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "num_caps": 1, 2023-07-13T22:30:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "open" 2023-07-13T22:30:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2023-07-13T22:30:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 73 2023-07-13T22:30:04.909 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:30:04.915 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:30:04.915 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:04.963 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:30:04.963 DEBUG:teuthology.orchestra.run.smithi060:> import sys 2023-07-13T22:30:04.963 DEBUG:teuthology.orchestra.run.smithi060:> import time 2023-07-13T22:30:04.964 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:30:04.964 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:04.964 DEBUG:teuthology.orchestra.run.smithi060:> n = 200 2023-07-13T22:30:04.964 DEBUG:teuthology.orchestra.run.smithi060:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-07-13T22:30:04.965 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:04.965 DEBUG:teuthology.orchestra.run.smithi060:> if not os.path.exists(abs_path): 2023-07-13T22:30:04.965 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(abs_path) 2023-07-13T22:30:04.965 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:04.966 DEBUG:teuthology.orchestra.run.smithi060:> handles = [] 2023-07-13T22:30:04.966 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(0, n): 2023-07-13T22:30:04.966 DEBUG:teuthology.orchestra.run.smithi060:> fname = "file_"+str(i) 2023-07-13T22:30:04.966 DEBUG:teuthology.orchestra.run.smithi060:> path = os.path.join(abs_path, fname) 2023-07-13T22:30:04.967 DEBUG:teuthology.orchestra.run.smithi060:> handles.append(open(path, '"'"'w'"'"')) 2023-07-13T22:30:04.967 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:04.967 DEBUG:teuthology.orchestra.run.smithi060:> while True: 2023-07-13T22:30:04.968 DEBUG:teuthology.orchestra.run.smithi060:> time.sleep(1) 2023-07-13T22:30:04.968 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:30:05.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:05.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.251+0000 7f1b5efff700 1 Processor -- start 2023-07-13T22:30:05.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.252+0000 7f1b5efff700 1 -- start start 2023-07-13T22:30:05.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.253+0000 7f1b5efff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b5815c300 0x7f1b580b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.253+0000 7f1b5efff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b580d18a0 con 0x7f1b580b46c0 2023-07-13T22:30:05.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.253+0000 7f1b5efff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b580cf500 con 0x7f1b5815c6d0 2023-07-13T22:30:05.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.253+0000 7f1b5efff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580d15c0 con 0x7f1b5815c300 2023-07-13T22:30:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.253+0000 7f1b5d59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1b580b46c0 0x7f1b5806b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34880/0 (socket says 172.21.15.60:34880) 2023-07-13T22:30:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.253+0000 7f1b5d59c700 1 -- 172.21.15.60:0/2297542135 learned_addr learned my addr 172.21.15.60:0/2297542135 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.253+0000 7f1b5cd9b700 1 --2- 172.21.15.60:0/2297542135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b5815c300 0x7f1b580b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.254+0000 7f1b577fe700 1 -- 172.21.15.60:0/2297542135 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786575990 0 0) 0x7f1b580d18a0 con 0x7f1b580b46c0 2023-07-13T22:30:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.254+0000 7f1b577fe700 1 -- 172.21.15.60:0/2297542135 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b480035b0 con 0x7f1b580b46c0 2023-07-13T22:30:05.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.254+0000 7f1b577fe700 1 -- 172.21.15.60:0/2297542135 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141557392 0 0) 0x7f1b580cf500 con 0x7f1b5815c6d0 2023-07-13T22:30:05.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.254+0000 7f1b577fe700 1 -- 172.21.15.60:0/2297542135 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b580d18a0 con 0x7f1b5815c6d0 2023-07-13T22:30:05.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.254+0000 7f1b5cd9b700 1 -- 172.21.15.60:0/2297542135 >> v1:172.21.15.105:6789/0 conn(0x7f1b5815c6d0 legacy=0x7f1b580cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.254+0000 7f1b5cd9b700 1 -- 172.21.15.60:0/2297542135 >> v1:172.21.15.60:6789/0 conn(0x7f1b580b46c0 legacy=0x7f1b5806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.254+0000 7f1b5cd9b700 1 -- 172.21.15.60:0/2297542135 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580cd3c0 con 0x7f1b5815c300 2023-07-13T22:30:05.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.254+0000 7f1b5cd9b700 1 --2- 172.21.15.60:0/2297542135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b5815c300 0x7f1b580b5a90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009870 tx=0x7f1b4c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1f250830f790a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:05.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.255+0000 7f1b577fe700 1 -- 172.21.15.60:0/2297542135 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c011830 con 0x7f1b5815c300 2023-07-13T22:30:05.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.255+0000 7f1b577fe700 1 -- 172.21.15.60:0/2297542135 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c011990 con 0x7f1b5815c300 2023-07-13T22:30:05.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.255+0000 7f1b577fe700 1 -- 172.21.15.60:0/2297542135 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c011d20 con 0x7f1b5815c300 2023-07-13T22:30:05.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.255+0000 7f1b5efff700 1 -- 172.21.15.60:0/2297542135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b5815c300 msgr2=0x7f1b580b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.255+0000 7f1b5efff700 1 --2- 172.21.15.60:0/2297542135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b5815c300 0x7f1b580b5a90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009870 tx=0x7f1b4c009b80 comp rx=0 tx=0).stop 2023-07-13T22:30:05.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.255+0000 7f1b5efff700 1 -- 172.21.15.60:0/2297542135 shutdown_connections 2023-07-13T22:30:05.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.255+0000 7f1b5efff700 1 --2- 172.21.15.60:0/2297542135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b5815c300 0x7f1b580b5a90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.255+0000 7f1b5efff700 1 -- 172.21.15.60:0/2297542135 >> 172.21.15.60:0/2297542135 conn(0x7f1b580cf6c0 msgr2=0x7f1b580cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:05.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.256+0000 7f1b5efff700 1 -- 172.21.15.60:0/2297542135 shutdown_connections 2023-07-13T22:30:05.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.256+0000 7f1b5efff700 1 -- 172.21.15.60:0/2297542135 wait complete. 2023-07-13T22:30:05.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.257+0000 7f1b5efff700 1 Processor -- start 2023-07-13T22:30:05.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.258+0000 7f1b5efff700 1 -- start start 2023-07-13T22:30:05.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b5efff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b580b46c0 0x7f1b5815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b5efff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b5815c300 0x7f1b5814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b5efff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b5815c6d0 0x7f1b5814e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b5efff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580c90d0 con 0x7f1b5815c300 2023-07-13T22:30:05.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b5efff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580c93b0 con 0x7f1b5815c6d0 2023-07-13T22:30:05.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b5efff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580c9690 con 0x7f1b580b46c0 2023-07-13T22:30:05.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b57fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b5815c300 0x7f1b5814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b57fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b5815c300 0x7f1b5814dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53944/0 (socket says 172.21.15.60:53944) 2023-07-13T22:30:05.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b57fff700 1 -- 172.21.15.60:0/3521245248 learned_addr learned my addr 172.21.15.60:0/3521245248 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:05.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b5cd9b700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b580b46c0 0x7f1b5815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b5d59c700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b5815c6d0 0x7f1b5814e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b57fff700 1 -- 172.21.15.60:0/3521245248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b580b46c0 msgr2=0x7f1b5815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.259+0000 7f1b57fff700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b580b46c0 0x7f1b5815f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b57fff700 1 -- 172.21.15.60:0/3521245248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b5815c6d0 msgr2=0x7f1b5814e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b57fff700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b5815c6d0 0x7f1b5814e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b57fff700 1 -- 172.21.15.60:0/3521245248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c0094a0 con 0x7f1b5815c300 2023-07-13T22:30:05.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b5d59c700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b5815c6d0 0x7f1b5814e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:05.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b57fff700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b5815c300 0x7f1b5814dc80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c4f0 tx=0x7f1b4400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:05.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b557fa700 1 -- 172.21.15.60:0/3521245248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440052d0 con 0x7f1b5815c300 2023-07-13T22:30:05.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b557fa700 1 -- 172.21.15.60:0/3521245248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b44005430 con 0x7f1b5815c300 2023-07-13T22:30:05.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5815cd20 con 0x7f1b5815c300 2023-07-13T22:30:05.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b557fa700 1 -- 172.21.15.60:0/3521245248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44003880 con 0x7f1b5815c300 2023-07-13T22:30:05.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.260+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5815f9f0 con 0x7f1b5815c300 2023-07-13T22:30:05.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.262+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b580c9210 con 0x7f1b5815c300 2023-07-13T22:30:05.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.263+0000 7f1b557fa700 1 -- 172.21.15.60:0/3521245248 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b440039e0 con 0x7f1b5815c300 2023-07-13T22:30:05.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.265+0000 7f1b557fa700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b3c06ace0 0x7f1b3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.265+0000 7f1b557fa700 1 -- 172.21.15.60:0/3521245248 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b440888f0 con 0x7f1b5815c300 2023-07-13T22:30:05.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.265+0000 7f1b5cd9b700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b3c06ace0 0x7f1b3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.266+0000 7f1b5cd9b700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b3c06ace0 0x7f1b3c06d190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0097c0 tx=0x7f1b4c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.270+0000 7f1b557fa700 1 -- 172.21.15.60:0/3521245248 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1b580c9210 con 0x7f1b5815c300 2023-07-13T22:30:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.538+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b5815d240 con 0x7f1b5815c300 2023-07-13T22:30:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.538+0000 7f1b557fa700 1 -- 172.21.15.60:0/3521245248 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f1b5815d240 con 0x7f1b5815c300 2023-07-13T22:30:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:05.518 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1446832449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1446832449},{"type":"v1","addr":"172.21.15.105:6835","nonce":1446832449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/462201037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":462201037},{"type":"v1","addr":"172.21.15.105:6834","nonce":462201037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/154242070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":154242070},{"type":"v1","addr":"172.21.15.60:6837","nonce":154242070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:29:49.290898+0000","modified":"2023-07-13T22:29:52.790955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7831":{"gid":7831,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6836/3527792396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3527792396},{"type":"v1","addr":"172.21.15.60:6836","nonce":3527792396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-13T22:30:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.540+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b3c06ace0 msgr2=0x7f1b3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.540+0000 7f1b5efff700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b3c06ace0 0x7f1b3c06d190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0097c0 tx=0x7f1b4c01c040 comp rx=0 tx=0).stop 2023-07-13T22:30:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.540+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b5815c300 msgr2=0x7f1b5814dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.540+0000 7f1b5efff700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b5815c300 0x7f1b5814dc80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c4f0 tx=0x7f1b4400c8b0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.541+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 shutdown_connections 2023-07-13T22:30:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.541+0000 7f1b5efff700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1b3c06ace0 0x7f1b3c06d190 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.541+0000 7f1b5efff700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b5815c6d0 0x7f1b5814e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.541+0000 7f1b5efff700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b580b46c0 0x7f1b5815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.541+0000 7f1b5efff700 1 --2- 172.21.15.60:0/3521245248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b5815c300 0x7f1b5814dc80 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.541+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 >> 172.21.15.60:0/3521245248 conn(0x7f1b580cf6c0 msgr2=0x7f1b580cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.541+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 shutdown_connections 2023-07-13T22:30:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.541+0000 7f1b5efff700 1 -- 172.21.15.60:0/3521245248 wait complete. 2023-07-13T22:30:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 73 2023-07-13T22:30:05.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-07-13T22:30:05.766 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":7972,"entity":{"name":{"type":"client","num":7972},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":759874285}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":5.3429450210000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9848403722465111,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7972 192.168.0.1:0/759874285","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"126311","root":"/"}},{"id":8586,"entity":{"name":{"type":"client","num":8586},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1730080076}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":202,"uptime":7.6919211839999999,"requests_in_flight":200,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":405.5603761312201,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8586 192.168.0.1:0/1730080076","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"157131","root":"/"}}] 2023-07-13T22:30:05.767 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "126311", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 759874285, "type": "any" }, "name": { "num": 7972, "type": "client" } }, "id": 7972, "inst": "client.7972 192.168.0.1:0/759874285", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.984840372246511 }, "state": "open", "uptime": 5.342945021 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "157131", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1730080076, "type": "any" }, "name": { "num": 8586, "type": "client" } }, "id": 8586, "inst": "client.8586 192.168.0.1:0/1730080076", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 202, "requests_in_flight": 200, "session_cache_liveness": { "halflife": 300, "value": 405.5603761312201 }, "state": "open", "uptime": 7.691921184 } ] 2023-07-13T22:30:05.768 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-07-13T22:30:05.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:30:05.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.994+0000 7fdde3dff700 1 Processor -- start 2023-07-13T22:30:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.994+0000 7fdde3dff700 1 -- start start 2023-07-13T22:30:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.995+0000 7fdde3dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cfcf0 0x7fdddc0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.995+0000 7fdde3dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdddc0cd790 0x7fdddc15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.995+0000 7fdde3dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc0d16b0 con 0x7fdddc15d160 2023-07-13T22:30:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.995+0000 7fdde3dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc0cdc00 con 0x7fdddc0cd790 2023-07-13T22:30:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.995+0000 7fdde3dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0d0d80 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.995+0000 7fdde239c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdddc15d160 0x7fdddc1600c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34896/0 (socket says 172.21.15.60:34896) 2023-07-13T22:30:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.996+0000 7fdde239c700 1 -- 172.21.15.60:0/4249077214 learned_addr learned my addr 172.21.15.60:0/4249077214 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:05.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.996+0000 7fdde139a700 1 --2- 172.21.15.60:0/4249077214 >> v2:172.21.15.105:3300/0 conn(0x7fdddc0cd790 0x7fdddc15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.996+0000 7fdde1b9b700 1 --2- 172.21.15.60:0/4249077214 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cfcf0 0x7fdddc0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.996+0000 7fdde0b99700 1 -- 172.21.15.60:0/4249077214 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407374205 0 0) 0x7fdddc0d16b0 con 0x7fdddc15d160 2023-07-13T22:30:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.996+0000 7fdde0b99700 1 -- 172.21.15.60:0/4249077214 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc80035b0 con 0x7fdddc15d160 2023-07-13T22:30:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.996+0000 7fdde139a700 1 -- 172.21.15.60:0/4249077214 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cfcf0 msgr2=0x7fdddc0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.996+0000 7fdde139a700 1 --2- 172.21.15.60:0/4249077214 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cfcf0 0x7fdddc0cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.996+0000 7fdde139a700 1 -- 172.21.15.60:0/4249077214 >> v1:172.21.15.60:6789/0 conn(0x7fdddc15d160 legacy=0x7fdddc1600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.996+0000 7fdde139a700 1 -- 172.21.15.60:0/4249077214 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0d1060 con 0x7fdddc0cd790 2023-07-13T22:30:05.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde139a700 1 --2- 172.21.15.60:0/4249077214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdddc0cd790 0x7fdddc15cc20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009840 tx=0x7fddcc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e69df0637bb8eb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:05.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde0b99700 1 -- 172.21.15.60:0/4249077214 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc011830 con 0x7fdddc0cd790 2023-07-13T22:30:05.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde0b99700 1 -- 172.21.15.60:0/4249077214 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc011990 con 0x7fdddc0cd790 2023-07-13T22:30:05.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde0b99700 1 -- 172.21.15.60:0/4249077214 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc011d20 con 0x7fdddc0cd790 2023-07-13T22:30:05.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde3dff700 1 -- 172.21.15.60:0/4249077214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdddc0cd790 msgr2=0x7fdddc15cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde3dff700 1 --2- 172.21.15.60:0/4249077214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdddc0cd790 0x7fdddc15cc20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009840 tx=0x7fddcc009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:05.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde3dff700 1 -- 172.21.15.60:0/4249077214 shutdown_connections 2023-07-13T22:30:05.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde3dff700 1 --2- 172.21.15.60:0/4249077214 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cfcf0 0x7fdddc0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde3dff700 1 --2- 172.21.15.60:0/4249077214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdddc0cd790 0x7fdddc15cc20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.997+0000 7fdde3dff700 1 -- 172.21.15.60:0/4249077214 >> 172.21.15.60:0/4249077214 conn(0x7fdddc06b310 msgr2=0x7fdddc06b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.998+0000 7fdde3dff700 1 -- 172.21.15.60:0/4249077214 shutdown_connections 2023-07-13T22:30:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.998+0000 7fdde3dff700 1 -- 172.21.15.60:0/4249077214 wait complete. 2023-07-13T22:30:05.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.999+0000 7fdde3dff700 1 Processor -- start 2023-07-13T22:30:05.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:05.999+0000 7fdde3dff700 1 -- start start 2023-07-13T22:30:05.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.000+0000 7fdde3dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cd790 0x7fdddc0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.000+0000 7fdde3dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdddc0cfcf0 0x7fdddc14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.000+0000 7fdde3dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdddc15d160 0x7fdddc0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.000+0000 7fdde3dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc1e6640 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.000+0000 7fdde3dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc14cf00 con 0x7fdddc15d160 2023-07-13T22:30:05.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde3dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc1e8e00 con 0x7fdddc0cd790 2023-07-13T22:30:05.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde139a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdddc0cfcf0 0x7fdddc14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde139a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdddc0cfcf0 0x7fdddc14c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53948/0 (socket says 172.21.15.60:53948) 2023-07-13T22:30:05.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde139a700 1 -- 172.21.15.60:0/603136164 learned_addr learned my addr 172.21.15.60:0/603136164 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:05.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde1b9b700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cd790 0x7fdddc0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde239c700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdddc15d160 0x7fdddc0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde139a700 1 -- 172.21.15.60:0/603136164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cd790 msgr2=0x7fdddc0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde139a700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cd790 0x7fdddc0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde139a700 1 -- 172.21.15.60:0/603136164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdddc15d160 msgr2=0x7fdddc0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:05.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde139a700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdddc15d160 0x7fdddc0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:05.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde139a700 1 -- 172.21.15.60:0/603136164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc0094a0 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.001+0000 7fdde139a700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdddc0cfcf0 0x7fdddc14c9c0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fddcc000c00 tx=0x7fddcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:05.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.002+0000 7fddd27fc700 1 -- 172.21.15.60:0/603136164 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0036a0 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.002+0000 7fddd27fc700 1 -- 172.21.15.60:0/603136164 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc01f070 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.002+0000 7fddd27fc700 1 -- 172.21.15.60:0/603136164 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc003a60 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.002+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc150360 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.002+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc1e5f80 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.003+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc0b8c10 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.004+0000 7fddd27fc700 1 -- 172.21.15.60:0/603136164 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fddcc003bc0 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.005+0000 7fddd27fc700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fddc006ace0 0x7fddc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.005+0000 7fddd27fc700 1 -- 172.21.15.60:0/603136164 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fddcc021070 con 0x7fdddc0cfcf0 2023-07-13T22:30:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.005+0000 7fdde1b9b700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fddc006ace0 0x7fddc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.006+0000 7fdde1b9b700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fddc006ace0 0x7fddc006d190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdddc1f59d0 tx=0x7fddd8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.009+0000 7fddd27fc700 1 -- 172.21.15.60:0/603136164 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fddcc055d10 con 0x7fdddc0cfcf0 2023-07-13T22:30:06.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.256+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdddc04e280 con 0x7fdddc0cfcf0 2023-07-13T22:30:06.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.256+0000 7fddd27fc700 1 -- 172.21.15.60:0/603136164 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fddcc059330 con 0x7fdddc0cfcf0 2023-07-13T22:30:06.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:06.235 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:30:06.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.258+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fddc006ace0 msgr2=0x7fddc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.258+0000 7fdde3dff700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fddc006ace0 0x7fddc006d190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdddc1f59d0 tx=0x7fddd8009210 comp rx=0 tx=0).stop 2023-07-13T22:30:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.258+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdddc0cfcf0 msgr2=0x7fdddc14c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.258+0000 7fdde3dff700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdddc0cfcf0 0x7fdddc14c9c0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fddcc000c00 tx=0x7fddcc004750 comp rx=0 tx=0).stop 2023-07-13T22:30:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.259+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 shutdown_connections 2023-07-13T22:30:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.259+0000 7fdde3dff700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fddc006ace0 0x7fddc006d190 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.259+0000 7fdde3dff700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdddc15d160 0x7fdddc0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:06.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.259+0000 7fdde3dff700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdddc0cd790 0x7fdddc0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:06.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.259+0000 7fdde3dff700 1 --2- 172.21.15.60:0/603136164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdddc0cfcf0 0x7fdddc14c9c0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:06.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.259+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 >> 172.21.15.60:0/603136164 conn(0x7fdddc06b310 msgr2=0x7fdddc0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:06.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.259+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 shutdown_connections 2023-07-13T22:30:06.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:06.259+0000 7fdde3dff700 1 -- 172.21.15.60:0/603136164 wait complete. 2023-07-13T22:30:06.255 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:30:06.255 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-13T22:30:11.256 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:30:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.493+0000 7f2d461ff700 1 Processor -- start 2023-07-13T22:30:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.493+0000 7f2d461ff700 1 -- start start 2023-07-13T22:30:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.494+0000 7f2d461ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d400b12a0 0x7f2d4015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.494+0000 7f2d461ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400c9e70 0x7f2d40069fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.494+0000 7f2d461ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d400cc0f0 con 0x7f2d4015a4d0 2023-07-13T22:30:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.494+0000 7f2d461ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4006bd80 con 0x7f2d400b12a0 2023-07-13T22:30:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.494+0000 7f2d461ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400cbb30 con 0x7f2d400c9e70 2023-07-13T22:30:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d3f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d400b12a0 0x7f2d4015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d37fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2d4015a4d0 0x7f2d400b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34898/0 (socket says 172.21.15.60:34898) 2023-07-13T22:30:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d37fff700 1 -- 172.21.15.60:0/3312765926 learned_addr learned my addr 172.21.15.60:0/3312765926 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d3ffff700 1 --2- 172.21.15.60:0/3312765926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400c9e70 0x7f2d40069fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d3f7fe700 1 -- 172.21.15.60:0/3312765926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400c9e70 msgr2=0x7f2d40069fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d3f7fe700 1 --2- 172.21.15.60:0/3312765926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400c9e70 0x7f2d40069fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d3f7fe700 1 -- 172.21.15.60:0/3312765926 >> v1:172.21.15.60:6789/0 conn(0x7f2d4015a4d0 legacy=0x7f2d400b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d3effd700 1 -- 172.21.15.60:0/3312765926 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255677519 0 0) 0x7f2d400cc0f0 con 0x7f2d4015a4d0 2023-07-13T22:30:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d3f7fe700 1 -- 172.21.15.60:0/3312765926 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d400cbe10 con 0x7f2d400b12a0 2023-07-13T22:30:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.495+0000 7f2d3f7fe700 1 --2- 172.21.15.60:0/3312765926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d400b12a0 0x7f2d4015e9f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009840 tx=0x7f2d30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195ad7a8509debde server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d3effd700 1 -- 172.21.15.60:0/3312765926 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30011830 con 0x7f2d400b12a0 2023-07-13T22:30:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d3effd700 1 -- 172.21.15.60:0/3312765926 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d30011990 con 0x7f2d400b12a0 2023-07-13T22:30:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d3effd700 1 -- 172.21.15.60:0/3312765926 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30011d20 con 0x7f2d400b12a0 2023-07-13T22:30:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d461ff700 1 -- 172.21.15.60:0/3312765926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d400b12a0 msgr2=0x7f2d4015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d461ff700 1 --2- 172.21.15.60:0/3312765926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d400b12a0 0x7f2d4015e9f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009840 tx=0x7f2d30009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d461ff700 1 -- 172.21.15.60:0/3312765926 shutdown_connections 2023-07-13T22:30:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d461ff700 1 --2- 172.21.15.60:0/3312765926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400c9e70 0x7f2d40069fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d461ff700 1 --2- 172.21.15.60:0/3312765926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d400b12a0 0x7f2d4015e9f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d461ff700 1 -- 172.21.15.60:0/3312765926 >> 172.21.15.60:0/3312765926 conn(0x7f2d400cc2a0 msgr2=0x7f2d400cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:11.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d461ff700 1 -- 172.21.15.60:0/3312765926 shutdown_connections 2023-07-13T22:30:11.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.496+0000 7f2d461ff700 1 -- 172.21.15.60:0/3312765926 wait complete. 2023-07-13T22:30:11.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.498+0000 7f2d461ff700 1 Processor -- start 2023-07-13T22:30:11.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.498+0000 7f2d461ff700 1 -- start start 2023-07-13T22:30:11.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.499+0000 7f2d461ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400b12a0 0x7f2d400ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.499+0000 7f2d461ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d400c9e70 0x7f2d40155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.499+0000 7f2d461ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d4015a4d0 0x7f2d401e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.499+0000 7f2d461ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400abd80 con 0x7f2d4015a4d0 2023-07-13T22:30:11.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.499+0000 7f2d461ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400ac060 con 0x7f2d400c9e70 2023-07-13T22:30:11.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.499+0000 7f2d461ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400ac340 con 0x7f2d400b12a0 2023-07-13T22:30:11.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d4015a4d0 0x7f2d401e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d4015a4d0 0x7f2d401e84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53958/0 (socket says 172.21.15.60:53958) 2023-07-13T22:30:11.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3ffff700 1 -- 172.21.15.60:0/3447015415 learned_addr learned my addr 172.21.15.60:0/3447015415 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:11.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3f7fe700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400b12a0 0x7f2d400ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d37fff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d400c9e70 0x7f2d40155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3ffff700 1 -- 172.21.15.60:0/3447015415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400b12a0 msgr2=0x7f2d400ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3ffff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400b12a0 0x7f2d400ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3ffff700 1 -- 172.21.15.60:0/3447015415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d400c9e70 msgr2=0x7f2d40155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3ffff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d400c9e70 0x7f2d40155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3ffff700 1 -- 172.21.15.60:0/3447015415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d300094a0 con 0x7f2d4015a4d0 2023-07-13T22:30:11.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d37fff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d400c9e70 0x7f2d40155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:11.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.500+0000 7f2d3ffff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d4015a4d0 0x7f2d401e84f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f2d38007e40 tx=0x7f2d3800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:11.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.501+0000 7f2d3cff9700 1 -- 172.21.15.60:0/3447015415 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800ede0 con 0x7f2d4015a4d0 2023-07-13T22:30:11.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.501+0000 7f2d3cff9700 1 -- 172.21.15.60:0/3447015415 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d38014420 con 0x7f2d4015a4d0 2023-07-13T22:30:11.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.501+0000 7f2d3cff9700 1 -- 172.21.15.60:0/3447015415 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d380147d0 con 0x7f2d4015a4d0 2023-07-13T22:30:11.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.501+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d400ac620 con 0x7f2d4015a4d0 2023-07-13T22:30:11.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.501+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d40156260 con 0x7f2d4015a4d0 2023-07-13T22:30:11.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.502+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d400acb40 con 0x7f2d4015a4d0 2023-07-13T22:30:11.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.503+0000 7f2d3cff9700 1 -- 172.21.15.60:0/3447015415 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2d38014930 con 0x7f2d4015a4d0 2023-07-13T22:30:11.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.504+0000 7f2d3cff9700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d2006ace0 0x7f2d2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.504+0000 7f2d3cff9700 1 -- 172.21.15.60:0/3447015415 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2d38089c00 con 0x7f2d4015a4d0 2023-07-13T22:30:11.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.504+0000 7f2d3f7fe700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d2006ace0 0x7f2d2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.505+0000 7f2d3f7fe700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d2006ace0 0x7f2d2006d190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2d300090b0 tx=0x7f2d3001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:11.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.509+0000 7f2d3cff9700 1 -- 172.21.15.60:0/3447015415 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2d400acb40 con 0x7f2d4015a4d0 2023-07-13T22:30:11.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.763+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2d400acb40 con 0x7f2d4015a4d0 2023-07-13T22:30:11.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.763+0000 7f2d3cff9700 1 -- 172.21.15.60:0/3447015415 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f2d400acb40 con 0x7f2d4015a4d0 2023-07-13T22:30:11.743 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:11.743 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:30:11.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.765+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d2006ace0 msgr2=0x7f2d2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.765+0000 7f2d461ff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d2006ace0 0x7f2d2006d190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2d300090b0 tx=0x7f2d3001c040 comp rx=0 tx=0).stop 2023-07-13T22:30:11.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.765+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d4015a4d0 msgr2=0x7f2d401e84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.765+0000 7f2d461ff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d4015a4d0 0x7f2d401e84f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f2d38007e40 tx=0x7f2d3800e3c0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.766+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 shutdown_connections 2023-07-13T22:30:11.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.766+0000 7f2d461ff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2d2006ace0 0x7f2d2006d190 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.766+0000 7f2d461ff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d4015a4d0 0x7f2d401e84f0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.766+0000 7f2d461ff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d400b12a0 0x7f2d400ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.766+0000 7f2d461ff700 1 --2- 172.21.15.60:0/3447015415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d400c9e70 0x7f2d40155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.766+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 >> 172.21.15.60:0/3447015415 conn(0x7f2d400cc2a0 msgr2=0x7f2d400cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.766+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 shutdown_connections 2023-07-13T22:30:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.766+0000 7f2d461ff700 1 -- 172.21.15.60:0/3447015415 wait complete. 2023-07-13T22:30:11.762 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-07-13T22:30:11.762 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-07-13T22:30:11.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:30:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.987+0000 7ffb1b5ff700 1 Processor -- start 2023-07-13T22:30:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.987+0000 7ffb1b5ff700 1 -- start start 2023-07-13T22:30:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.989+0000 7ffb1b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415bef0 0x7ffb140cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.989+0000 7ffb1b5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb140c9d20 0x7ffb1415c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.989+0000 7ffb1b5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1415ce80 con 0x7ffb1415c2c0 2023-07-13T22:30:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.989+0000 7ffb1b5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb1415cc70 con 0x7ffb140c9d20 2023-07-13T22:30:11.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.989+0000 7ffb1b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415d160 con 0x7ffb1415bef0 2023-07-13T22:30:11.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.989+0000 7ffb19b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb140c9d20 0x7ffb1415c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.989+0000 7ffb18b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffb1415c2c0 0x7ffb140c9610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34914/0 (socket says 172.21.15.60:34914) 2023-07-13T22:30:11.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.989+0000 7ffb18b9a700 1 -- 172.21.15.60:0/727677703 learned_addr learned my addr 172.21.15.60:0/727677703 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:11.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb1939b700 1 --2- 172.21.15.60:0/727677703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415bef0 0x7ffb140cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb0bfff700 1 -- 172.21.15.60:0/727677703 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798386336 0 0) 0x7ffb1415ce80 con 0x7ffb1415c2c0 2023-07-13T22:30:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb0bfff700 1 -- 172.21.15.60:0/727677703 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb000035b0 con 0x7ffb1415c2c0 2023-07-13T22:30:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb19b9c700 1 -- 172.21.15.60:0/727677703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415bef0 msgr2=0x7ffb140cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb19b9c700 1 --2- 172.21.15.60:0/727677703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415bef0 0x7ffb140cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb19b9c700 1 -- 172.21.15.60:0/727677703 >> v1:172.21.15.60:6789/0 conn(0x7ffb1415c2c0 legacy=0x7ffb140c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb19b9c700 1 -- 172.21.15.60:0/727677703 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1406c7e0 con 0x7ffb140c9d20 2023-07-13T22:30:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb19b9c700 1 --2- 172.21.15.60:0/727677703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 0x7ffb1415c730 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009d90 tx=0x7ffb0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a195c136d72b7030 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb0bfff700 1 -- 172.21.15.60:0/727677703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00bb20 con 0x7ffb140c9d20 2023-07-13T22:30:11.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb0bfff700 1 -- 172.21.15.60:0/727677703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c00bc80 con 0x7ffb140c9d20 2023-07-13T22:30:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.990+0000 7ffb0bfff700 1 -- 172.21.15.60:0/727677703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c011400 con 0x7ffb140c9d20 2023-07-13T22:30:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.991+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/727677703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 msgr2=0x7ffb1415c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.991+0000 7ffb1b5ff700 1 --2- 172.21.15.60:0/727677703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 0x7ffb1415c730 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009d90 tx=0x7ffb0c00b330 comp rx=0 tx=0).stop 2023-07-13T22:30:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.991+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/727677703 shutdown_connections 2023-07-13T22:30:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.991+0000 7ffb1b5ff700 1 --2- 172.21.15.60:0/727677703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 0x7ffb1415c730 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.991+0000 7ffb1b5ff700 1 --2- 172.21.15.60:0/727677703 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415bef0 0x7ffb140cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.991+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/727677703 >> 172.21.15.60:0/727677703 conn(0x7ffb140cbe70 msgr2=0x7ffb140ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.991+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/727677703 shutdown_connections 2023-07-13T22:30:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.991+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/727677703 wait complete. 2023-07-13T22:30:11.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.993+0000 7ffb1b5ff700 1 Processor -- start 2023-07-13T22:30:11.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.993+0000 7ffb1b5ff700 1 -- start start 2023-07-13T22:30:11.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1b5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 0x7ffb140c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1b5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1415bef0 0x7ffb14154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1b5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415c2c0 0x7ffb140c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1b5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140b1d10 con 0x7ffb140c9d20 2023-07-13T22:30:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1b5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c87d0 con 0x7ffb1415bef0 2023-07-13T22:30:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1b5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c84f0 con 0x7ffb1415c2c0 2023-07-13T22:30:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1939b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 0x7ffb140c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1939b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 0x7ffb140c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53972/0 (socket says 172.21.15.60:53972) 2023-07-13T22:30:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1939b700 1 -- 172.21.15.60:0/3593606613 learned_addr learned my addr 172.21.15.60:0/3593606613 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb19b9c700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415c2c0 0x7ffb140c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb18b9a700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1415bef0 0x7ffb14154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1939b700 1 -- 172.21.15.60:0/3593606613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415c2c0 msgr2=0x7ffb140c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1939b700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415c2c0 0x7ffb140c32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1939b700 1 -- 172.21.15.60:0/3593606613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1415bef0 msgr2=0x7ffb14154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1939b700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1415bef0 0x7ffb14154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb1939b700 1 -- 172.21.15.60:0/3593606613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c009a70 con 0x7ffb140c9d20 2023-07-13T22:30:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.994+0000 7ffb18b9a700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1415bef0 0x7ffb14154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.995+0000 7ffb1939b700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 0x7ffb140c7ca0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000e900 tx=0x7ffb1000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.995+0000 7ffb09ffb700 1 -- 172.21.15.60:0/3593606613 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10009650 con 0x7ffb140c9d20 2023-07-13T22:30:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.995+0000 7ffb09ffb700 1 -- 172.21.15.60:0/3593606613 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1000c850 con 0x7ffb140c9d20 2023-07-13T22:30:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.995+0000 7ffb09ffb700 1 -- 172.21.15.60:0/3593606613 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000cbc0 con 0x7ffb140c9d20 2023-07-13T22:30:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.995+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb140b16c0 con 0x7ffb140c9d20 2023-07-13T22:30:11.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.995+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb140c81e0 con 0x7ffb140c9d20 2023-07-13T22:30:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.996+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb140b1ff0 con 0x7ffb140c9d20 2023-07-13T22:30:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.997+0000 7ffb09ffb700 1 -- 172.21.15.60:0/3593606613 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb1000cd20 con 0x7ffb140c9d20 2023-07-13T22:30:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.999+0000 7ffb09ffb700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffaf806ace0 0x7ffaf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.999+0000 7ffb09ffb700 1 -- 172.21.15.60:0/3593606613 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb100898b0 con 0x7ffb140c9d20 2023-07-13T22:30:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.999+0000 7ffb18b9a700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffaf806ace0 0x7ffaf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:11.999+0000 7ffb18b9a700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffaf806ace0 0x7ffaf806d190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffb1414d120 tx=0x7ffb04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.003+0000 7ffb09ffb700 1 -- 172.21.15.60:0/3593606613 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffb10054950 con 0x7ffb140c9d20 2023-07-13T22:30:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.255+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffb1404e280 con 0x7ffb140c9d20 2023-07-13T22:30:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.255+0000 7ffb09ffb700 1 -- 172.21.15.60:0/3593606613 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7ffb10057f70 con 0x7ffb140c9d20 2023-07-13T22:30:12.234 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:12.234 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:30:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.257+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffaf806ace0 msgr2=0x7ffaf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.257+0000 7ffb1b5ff700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffaf806ace0 0x7ffaf806d190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffb1414d120 tx=0x7ffb04009040 comp rx=0 tx=0).stop 2023-07-13T22:30:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.257+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 msgr2=0x7ffb140c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.257+0000 7ffb1b5ff700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 0x7ffb140c7ca0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000e900 tx=0x7ffb1000ecc0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.258+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 shutdown_connections 2023-07-13T22:30:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.258+0000 7ffb1b5ff700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffaf806ace0 0x7ffaf806d190 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.258+0000 7ffb1b5ff700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb140c9d20 0x7ffb140c7ca0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.258+0000 7ffb1b5ff700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb1415c2c0 0x7ffb140c32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.258+0000 7ffb1b5ff700 1 --2- 172.21.15.60:0/3593606613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb1415bef0 0x7ffb14154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.258+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 >> 172.21.15.60:0/3593606613 conn(0x7ffb140cbe70 msgr2=0x7ffb1415b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.258+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 shutdown_connections 2023-07-13T22:30:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.258+0000 7ffb1b5ff700 1 -- 172.21.15.60:0/3593606613 wait complete. 2023-07-13T22:30:12.254 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:30:12.254 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2023-07-13T22:30:12.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.529+0000 7f03eb7ff700 1 Processor -- start 2023-07-13T22:30:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.530+0000 7f03eb7ff700 1 -- start start 2023-07-13T22:30:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.531+0000 7f03eb7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40b8400 0x7f03e40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.531+0000 7f03eb7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03e40cfca0 0x7f03e40d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.531+0000 7f03eb7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e40d0ac0 con 0x7f03e40d3750 2023-07-13T22:30:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.531+0000 7f03eb7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e40b87d0 con 0x7f03e40cfca0 2023-07-13T22:30:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.531+0000 7f03eb7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40b4eb0 con 0x7f03e40b8400 2023-07-13T22:30:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.531+0000 7f03e9d9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03e40cfca0 0x7f03e40d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.531+0000 7f03e959b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40b8400 0x7f03e40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.532+0000 7f03e9d9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03e40cfca0 0x7f03e40d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:39648/0 (socket says 172.21.15.60:39648) 2023-07-13T22:30:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.532+0000 7f03e9d9c700 1 -- 172.21.15.60:0/2482368987 learned_addr learned my addr 172.21.15.60:0/2482368987 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.532+0000 7f03e9d9c700 1 -- 172.21.15.60:0/2482368987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40b8400 msgr2=0x7f03e40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.532+0000 7f03e9d9c700 1 --2- 172.21.15.60:0/2482368987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40b8400 0x7f03e40cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.532+0000 7f03e9d9c700 1 -- 172.21.15.60:0/2482368987 >> v1:172.21.15.105:6789/0 conn(0x7f03e40d3750 legacy=0x7f03e40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.532+0000 7f03e9d9c700 1 -- 172.21.15.60:0/2482368987 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40b5190 con 0x7f03e40cfca0 2023-07-13T22:30:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.532+0000 7f03e9d9c700 1 --2- 172.21.15.60:0/2482368987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03e40cfca0 0x7f03e40d0f50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009e50 tx=0x7f03dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befc9a57780234e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.532+0000 7f03dbfff700 1 -- 172.21.15.60:0/2482368987 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00e070 con 0x7f03e40cfca0 2023-07-13T22:30:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03dbfff700 1 -- 172.21.15.60:0/2482368987 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc00b910 con 0x7f03e40cfca0 2023-07-13T22:30:12.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03dbfff700 1 -- 172.21.15.60:0/2482368987 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00bcc0 con 0x7f03e40cfca0 2023-07-13T22:30:12.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2482368987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03e40cfca0 msgr2=0x7f03e40d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:12.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03eb7ff700 1 --2- 172.21.15.60:0/2482368987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03e40cfca0 0x7f03e40d0f50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009e50 tx=0x7f03dc00b330 comp rx=0 tx=0).stop 2023-07-13T22:30:12.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2482368987 shutdown_connections 2023-07-13T22:30:12.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03eb7ff700 1 --2- 172.21.15.60:0/2482368987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03e40cfca0 0x7f03e40d0f50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03eb7ff700 1 --2- 172.21.15.60:0/2482368987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40b8400 0x7f03e40cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2482368987 >> 172.21.15.60:0/2482368987 conn(0x7f03e406b390 msgr2=0x7f03e406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2482368987 shutdown_connections 2023-07-13T22:30:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.533+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2482368987 wait complete. 2023-07-13T22:30:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.535+0000 7f03eb7ff700 1 Processor -- start 2023-07-13T22:30:12.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.535+0000 7f03eb7ff700 1 -- start start 2023-07-13T22:30:12.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03eb7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03e40b8400 0x7f03e41485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03eb7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40cfca0 0x7f03e40c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03eb7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03e40d3750 0x7f03e40cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03eb7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cba80 con 0x7f03e40b8400 2023-07-13T22:30:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03eb7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4148b00 con 0x7f03e40d3750 2023-07-13T22:30:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03eb7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cae70 con 0x7f03e40cfca0 2023-07-13T22:30:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03e959b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03e40b8400 0x7f03e41485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:12.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03e959b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03e40b8400 0x7f03e41485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53988/0 (socket says 172.21.15.60:53988) 2023-07-13T22:30:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03e959b700 1 -- 172.21.15.60:0/2066474733 learned_addr learned my addr 172.21.15.60:0/2066474733 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03e9d9c700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03e40d3750 0x7f03e40cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.536+0000 7f03e8d9a700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40cfca0 0x7f03e40c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03e959b700 1 -- 172.21.15.60:0/2066474733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40cfca0 msgr2=0x7f03e40c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03e959b700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40cfca0 0x7f03e40c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03e959b700 1 -- 172.21.15.60:0/2066474733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03e40d3750 msgr2=0x7f03e40cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03e959b700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03e40d3750 0x7f03e40cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03e959b700 1 -- 172.21.15.60:0/2066474733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc009b30 con 0x7f03e40b8400 2023-07-13T22:30:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03e959b700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03e40b8400 0x7f03e41485c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f03e000e870 tx=0x7f03e000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03d9ffb700 1 -- 172.21.15.60:0/2066474733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0009590 con 0x7f03e40b8400 2023-07-13T22:30:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03d9ffb700 1 -- 172.21.15.60:0/2066474733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0013680 con 0x7f03e40b8400 2023-07-13T22:30:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e40cb150 con 0x7f03e40b8400 2023-07-13T22:30:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03d9ffb700 1 -- 172.21.15.60:0/2066474733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00139f0 con 0x7f03e40b8400 2023-07-13T22:30:12.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.537+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e40cb710 con 0x7f03e40b8400 2023-07-13T22:30:12.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.538+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e40cbd60 con 0x7f03e40b8400 2023-07-13T22:30:12.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.539+0000 7f03d9ffb700 1 -- 172.21.15.60:0/2066474733 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f03e0013b50 con 0x7f03e40b8400 2023-07-13T22:30:12.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.540+0000 7f03d9ffb700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03c806ace0 0x7f03c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:12.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.540+0000 7f03d9ffb700 1 -- 172.21.15.60:0/2066474733 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03e00894a0 con 0x7f03e40b8400 2023-07-13T22:30:12.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.541+0000 7f03e8d9a700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03c806ace0 0x7f03c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:12.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.541+0000 7f03e8d9a700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03c806ace0 0x7f03c806d190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f03e414f360 tx=0x7f03d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:12.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.545+0000 7f03d9ffb700 1 -- 172.21.15.60:0/2066474733 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f03e00544b0 con 0x7f03e40b8400 2023-07-13T22:30:12.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.798+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03e415a7a0 con 0x7f03e40b8400 2023-07-13T22:30:12.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.799+0000 7f03d9ffb700 1 -- 172.21.15.60:0/2066474733 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f03e0057ad0 con 0x7f03e40b8400 2023-07-13T22:30:12.778 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:12.778 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":75,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1446832449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1446832449},{"type":"v1","addr":"172.21.15.105:6835","nonce":1446832449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/462201037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":462201037},{"type":"v1","addr":"172.21.15.105:6834","nonce":462201037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/154242070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":154242070},{"type":"v1","addr":"172.21.15.60:6837","nonce":154242070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:29:49.290898+0000","modified":"2023-07-13T22:30:10.653578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7831":{"gid":7831,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6836/3527792396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3527792396},{"type":"v1","addr":"172.21.15.60:6836","nonce":3527792396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-13T22:30:12.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03c806ace0 msgr2=0x7f03c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:12.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03c806ace0 0x7f03c806d190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f03e414f360 tx=0x7f03d4009040 comp rx=0 tx=0).stop 2023-07-13T22:30:12.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03e40b8400 msgr2=0x7f03e41485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:12.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03e40b8400 0x7f03e41485c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f03e000e870 tx=0x7f03e000eb80 comp rx=0 tx=0).stop 2023-07-13T22:30:12.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 shutdown_connections 2023-07-13T22:30:12.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03c806ace0 0x7f03c806d190 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03e40b8400 0x7f03e41485c0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03e40d3750 0x7f03e40cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 --2- 172.21.15.60:0/2066474733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03e40cfca0 0x7f03e40c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:12.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 >> 172.21.15.60:0/2066474733 conn(0x7f03e406b390 msgr2=0x7f03e40b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:12.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 shutdown_connections 2023-07-13T22:30:12.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:12.801+0000 7f03eb7ff700 1 -- 172.21.15.60:0/2066474733 wait complete. 2023-07-13T22:30:12.784 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 75 2023-07-13T22:30:12.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-07-13T22:30:13.028 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":7972,"entity":{"name":{"type":"client","num":7972},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":759874285}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":12.604869624000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9351767127282846,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7972 192.168.0.1:0/759874285","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"126311","root":"/"}},{"id":8586,"entity":{"name":{"type":"client","num":8586},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1730080076}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":187,"uptime":14.953845787000001,"requests_in_flight":0,"num_completed_requests":200,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":766.51342008973654,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":236.95529683602166,"halflife":1.5},"recall_caps_throttle2o":{"value":97.028875350540403,"halflife":0.5},"session_cache_liveness":{"value":398.81240641681097,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8586 192.168.0.1:0/1730080076","completed_requests":[{"tid":5,"created_ino":"0x10000000001"},{"tid":6,"created_ino":"0x100000001f6"},{"tid":7,"created_ino":"0x100000001f7"},{"tid":8,"created_ino":"0x100000001f8"},{"tid":9,"created_ino":"0x100000001f9"},{"tid":10,"created_ino":"0x100000001fa"},{"tid":11,"created_ino":"0x100000001fb"},{"tid":12,"created_ino":"0x100000001fc"},{"tid":13,"created_ino":"0x100000001fd"},{"tid":14,"created_ino":"0x100000001fe"},{"tid":15,"created_ino":"0x100000001ff"},{"tid":16,"created_ino":"0x10000000200"},{"tid":17,"created_ino":"0x10000000201"},{"tid":18,"created_ino":"0x10000000202"},{"tid":19,"created_ino":"0x10000000203"},{"tid":20,"created_ino":"0x10000000204"},{"tid":21,"created_ino":"0x10000000205"},{"tid":22,"created_ino":"0x10000000206"},{"tid":23,"created_ino":"0x10000000207"},{"tid":24,"created_ino":"0x10000000208"},{"tid":25,"created_ino":"0x10000000209"},{"tid":26,"created_ino":"0x1000000020a"},{"tid":27,"created_ino":"0x1000000020b"},{"tid":28,"created_ino":"0x1000000020c"},{"tid":29,"created_ino":"0x1000000020d"},{"tid":30,"created_ino":"0x1000000020e"},{"tid":31,"created_ino":"0x1000000020f"},{"tid":32,"created_ino":"0x10000000210"},{"tid":33,"created_ino":"0x10000000211"},{"tid":34,"created_ino":"0x10000000212"},{"tid":35,"created_ino":"0x10000000213"},{"tid":36,"created_ino":"0x10000000214"},{"tid":37,"created_ino":"0x10000000215"},{"tid":38,"created_ino":"0x10000000216"},{"tid":39,"created_ino":"0x10000000217"},{"tid":40,"created_ino":"0x10000000218"},{"tid":41,"created_ino":"0x10000000219"},{"tid":42,"created_ino":"0x1000000021a"},{"tid":43,"created_ino":"0x1000000021b"},{"tid":44,"created_ino":"0x1000000021c"},{"tid":45,"created_ino":"0x1000000021d"},{"tid":46,"created_ino":"0x1000000021e"},{"tid":47,"created_ino":"0x1000000021f"},{"tid":48,"created_ino":"0x10000000220"},{"tid":49,"created_ino":"0x10000000221"},{"tid":50,"created_ino":"0x10000000222"},{"tid":51,"created_ino":"0x10000000223"},{"tid":52,"created_ino":"0x10000000224"},{"tid":53,"created_ino":"0x10000000225"},{"tid":54,"created_ino":"0x10000000226"},{"tid":55,"created_ino":"0x10000000227"},{"tid":56,"created_ino":"0x10000000228"},{"tid":57,"created_ino":"0x10000000229"},{"tid":58,"created_ino":"0x1000000022a"},{"tid":59,"created_ino":"0x1000000022b"},{"tid":60,"created_ino":"0x1000000022c"},{"tid":61,"created_ino":"0x1000000022d"},{"tid":62,"created_ino":"0x1000000022e"},{"tid":63,"created_ino":"0x1000000022f"},{"tid":64,"created_ino":"0x10000000230"},{"tid":65,"created_ino":"0x10000000231"},{"tid":66,"created_ino":"0x10000000232"},{"tid":67,"created_ino":"0x10000000233"},{"tid":68,"created_ino":"0x10000000234"},{"tid":69,"created_ino":"0x10000000235"},{"tid":70,"created_ino":"0x10000000236"},{"tid":71,"created_ino":"0x10000000237"},{"tid":72,"created_ino":"0x10000000238"},{"tid":73,"created_ino":"0x10000000239"},{"tid":74,"created_ino":"0x1000000023a"},{"tid":75,"created_ino":"0x1000000023b"},{"tid":76,"created_ino":"0x1000000023c"},{"tid":77,"created_ino":"0x1000000023d"},{"tid":78,"created_ino":"0x1000000023e"},{"tid":79,"created_ino":"0x1000000023f"},{"tid":80,"created_ino":"0x10000000240"},{"tid":81,"created_ino":"0x10000000241"},{"tid":82,"created_ino":"0x10000000242"},{"tid":83,"created_ino":"0x10000000243"},{"tid":84,"created_ino":"0x10000000244"},{"tid":85,"created_ino":"0x10000000245"},{"tid":86,"created_ino":"0x10000000246"},{"tid":87,"created_ino":"0x10000000247"},{"tid":88,"created_ino":"0x10000000248"},{"tid":89,"created_ino":"0x10000000249"},{"tid":90,"created_ino":"0x1000000024a"},{"tid":91,"created_ino":"0x1000000024b"},{"tid":92,"created_ino":"0x1000000024c"},{"tid":93,"created_ino":"0x1000000024d"},{"tid":94,"created_ino":"0x1000000024e"},{"tid":95,"created_ino":"0x1000000024f"},{"tid":96,"created_ino":"0x10000000250"},{"tid":97,"created_ino":"0x10000000251"},{"tid":98,"created_ino":"0x10000000252"},{"tid":99,"created_ino":"0x10000000253"},{"tid":100,"created_ino":"0x10000000254"},{"tid":101,"created_ino":"0x10000000255"},{"tid":102,"created_ino":"0x10000000256"},{"tid":103,"created_ino":"0x10000000257"},{"tid":104,"created_ino":"0x10000000258"},{"tid":105,"created_ino":"0x10000000259"},{"tid":106,"created_ino":"0x1000000025a"},{"tid":107,"created_ino":"0x1000000025b"},{"tid":108,"created_ino":"0x1000000025c"},{"tid":109,"created_ino":"0x1000000025d"},{"tid":110,"created_ino":"0x1000000025e"},{"tid":111,"created_ino":"0x1000000025f"},{"tid":112,"created_ino":"0x10000000260"},{"tid":113,"created_ino":"0x10000000261"},{"tid":114,"created_ino":"0x10000000262"},{"tid":115,"created_ino":"0x10000000263"},{"tid":116,"created_ino":"0x10000000264"},{"tid":117,"created_ino":"0x10000000265"},{"tid":118,"created_ino":"0x10000000266"},{"tid":119,"created_ino":"0x10000000267"},{"tid":120,"created_ino":"0x10000000268"},{"tid":121,"created_ino":"0x10000000269"},{"tid":122,"created_ino":"0x1000000026a"},{"tid":123,"created_ino":"0x1000000026b"},{"tid":124,"created_ino":"0x1000000026c"},{"tid":125,"created_ino":"0x1000000026d"},{"tid":126,"created_ino":"0x1000000026e"},{"tid":127,"created_ino":"0x1000000026f"},{"tid":128,"created_ino":"0x10000000270"},{"tid":129,"created_ino":"0x10000000271"},{"tid":130,"created_ino":"0x10000000272"},{"tid":131,"created_ino":"0x10000000273"},{"tid":132,"created_ino":"0x10000000274"},{"tid":133,"created_ino":"0x10000000275"},{"tid":134,"created_ino":"0x10000000276"},{"tid":135,"created_ino":"0x10000000277"},{"tid":136,"created_ino":"0x10000000278"},{"tid":137,"created_ino":"0x10000000279"},{"tid":138,"created_ino":"0x1000000027a"},{"tid":139,"created_ino":"0x1000000027b"},{"tid":140,"created_ino":"0x1000000027c"},{"tid":141,"created_ino":"0x1000000027d"},{"tid":142,"created_ino":"0x1000000027e"},{"tid":143,"created_ino":"0x1000000027f"},{"tid":144,"created_ino":"0x10000000280"},{"tid":145,"created_ino":"0x10000000281"},{"tid":146,"created_ino":"0x10000000282"},{"tid":147,"created_ino":"0x10000000283"},{"tid":148,"created_ino":"0x10000000284"},{"tid":149,"created_ino":"0x10000000285"},{"tid":150,"created_ino":"0x10000000286"},{"tid":151,"created_ino":"0x10000000287"},{"tid":152,"created_ino":"0x10000000288"},{"tid":153,"created_ino":"0x10000000289"},{"tid":154,"created_ino":"0x1000000028a"},{"tid":155,"created_ino":"0x1000000028b"},{"tid":156,"created_ino":"0x1000000028c"},{"tid":157,"created_ino":"0x1000000028d"},{"tid":158,"created_ino":"0x1000000028e"},{"tid":159,"created_ino":"0x1000000028f"},{"tid":160,"created_ino":"0x10000000290"},{"tid":161,"created_ino":"0x10000000291"},{"tid":162,"created_ino":"0x10000000292"},{"tid":163,"created_ino":"0x10000000293"},{"tid":164,"created_ino":"0x10000000294"},{"tid":165,"created_ino":"0x10000000295"},{"tid":166,"created_ino":"0x10000000296"},{"tid":167,"created_ino":"0x10000000297"},{"tid":168,"created_ino":"0x10000000298"},{"tid":169,"created_ino":"0x10000000299"},{"tid":170,"created_ino":"0x1000000029a"},{"tid":171,"created_ino":"0x1000000029b"},{"tid":172,"created_ino":"0x1000000029c"},{"tid":173,"created_ino":"0x1000000029d"},{"tid":174,"created_ino":"0x1000000029e"},{"tid":175,"created_ino":"0x1000000029f"},{"tid":176,"created_ino":"0x100000002a0"},{"tid":177,"created_ino":"0x100000002a1"},{"tid":178,"created_ino":"0x100000002a2"},{"tid":179,"created_ino":"0x100000002a3"},{"tid":180,"created_ino":"0x100000002a4"},{"tid":181,"created_ino":"0x100000002a5"},{"tid":182,"created_ino":"0x100000002a6"},{"tid":183,"created_ino":"0x100000002a7"},{"tid":184,"created_ino":"0x100000002a8"},{"tid":185,"created_ino":"0x100000002a9"},{"tid":186,"created_ino":"0x100000002aa"},{"tid":187,"created_ino":"0x100000002ab"},{"tid":188,"created_ino":"0x100000002ac"},{"tid":189,"created_ino":"0x100000002ad"},{"tid":190,"created_ino":"0x100000002ae"},{"tid":191,"created_ino":"0x100000002af"},{"tid":192,"created_ino":"0x100000002b0"},{"tid":193,"created_ino":"0x100000002b1"},{"tid":194,"created_ino":"0x100000002b2"},{"tid":195,"created_ino":"0x100000002b3"},{"tid":196,"created_ino":"0x100000002b4"},{"tid":197,"created_ino":"0x100000002b5"},{"tid":198,"created_ino":"0x100000002b6"},{"tid":199,"created_ino":"0x100000002b7"},{"tid":200,"created_ino":"0x100000002b8"},{"tid":201,"created_ino":"0x100000002b9"},{"tid":202,"created_ino":"0x100000002ba"},{"tid":203,"created_ino":"0x100000002bb"},{"tid":204,"created_ino":"0x100000002bc"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000002bd","length":802}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"157131","root":"/"}}] 2023-07-13T22:30:13.030 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "126311", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 759874285, "type": "any" }, "name": { "num": 7972, "type": "client" } }, "id": 7972, "inst": "client.7972 192.168.0.1:0/759874285", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9351767127282846 }, "state": "open", "uptime": 12.604869624 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "157131", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 5 }, { "created_ino": "0x100000001f6", "tid": 6 }, { "created_ino": "0x100000001f7", "tid": 7 }, { "created_ino": "0x100000001f8", "tid": 8 }, { "created_ino": "0x100000001f9", "tid": 9 }, { "created_ino": "0x100000001fa", "tid": 10 }, { "created_ino": "0x100000001fb", "tid": 11 }, { "created_ino": "0x100000001fc", "tid": 12 }, { "created_ino": "0x100000001fd", "tid": 13 }, { "created_ino": "0x100000001fe", "tid": 14 }, { "created_ino": "0x100000001ff", "tid": 15 }, { "created_ino": "0x10000000200", "tid": 16 }, { "created_ino": "0x10000000201", "tid": 17 }, { "created_ino": "0x10000000202", "tid": 18 }, { "created_ino": "0x10000000203", "tid": 19 }, { "created_ino": "0x10000000204", "tid": 20 }, { "created_ino": "0x10000000205", "tid": 21 }, { "created_ino": "0x10000000206", "tid": 22 }, { "created_ino": "0x10000000207", "tid": 23 }, { "created_ino": "0x10000000208", "tid": 24 }, { "created_ino": "0x10000000209", "tid": 25 }, { "created_ino": "0x1000000020a", "tid": 26 }, { "created_ino": "0x1000000020b", "tid": 27 }, { "created_ino": "0x1000000020c", "tid": 28 }, { "created_ino": "0x1000000020d", "tid": 29 }, { "created_ino": "0x1000000020e", "tid": 30 }, { "created_ino": "0x1000000020f", "tid": 31 }, { "created_ino": "0x10000000210", "tid": 32 }, { "created_ino": "0x10000000211", "tid": 33 }, { "created_ino": "0x10000000212", "tid": 34 }, { "created_ino": "0x10000000213", "tid": 35 }, { "created_ino": "0x10000000214", "tid": 36 }, { "created_ino": "0x10000000215", "tid": 37 }, { "created_ino": "0x10000000216", "tid": 38 }, { "created_ino": "0x10000000217", "tid": 39 }, { "created_ino": "0x10000000218", "tid": 40 }, { "created_ino": "0x10000000219", "tid": 41 }, { "created_ino": "0x1000000021a", "tid": 42 }, { "created_ino": "0x1000000021b", "tid": 43 }, { "created_ino": "0x1000000021c", "tid": 44 }, { "created_ino": "0x1000000021d", "tid": 45 }, { "created_ino": "0x1000000021e", "tid": 46 }, { "created_ino": "0x1000000021f", "tid": 47 }, { "created_ino": "0x10000000220", "tid": 48 }, { "created_ino": "0x10000000221", "tid": 49 }, { "created_ino": "0x10000000222", "tid": 50 }, { "created_ino": "0x10000000223", "tid": 51 }, { "created_ino": "0x10000000224", "tid": 52 }, { "created_ino": "0x10000000225", "tid": 53 }, { "created_ino": "0x10000000226", "tid": 54 }, { "created_ino": "0x10000000227", "tid": 55 }, { "created_ino": "0x10000000228", "tid": 56 }, { "created_ino": "0x10000000229", "tid": 57 }, { "created_ino": "0x1000000022a", "tid": 58 }, { "created_ino": "0x1000000022b", "tid": 59 }, { "created_ino": "0x1000000022c", "tid": 60 }, { "created_ino": "0x1000000022d", "tid": 61 }, { "created_ino": "0x1000000022e", "tid": 62 }, { "created_ino": "0x1000000022f", "tid": 63 }, { "created_ino": "0x10000000230", "tid": 64 }, { "created_ino": "0x10000000231", "tid": 65 }, { "created_ino": "0x10000000232", "tid": 66 }, { "created_ino": "0x10000000233", "tid": 67 }, { "created_ino": "0x10000000234", "tid": 68 }, { "created_ino": "0x10000000235", "tid": 69 }, { "created_ino": "0x10000000236", "tid": 70 }, { "created_ino": "0x10000000237", "tid": 71 }, { "created_ino": "0x10000000238", "tid": 72 }, { "created_ino": "0x10000000239", "tid": 73 }, { "created_ino": "0x1000000023a", "tid": 74 }, { "created_ino": "0x1000000023b", "tid": 75 }, { "created_ino": "0x1000000023c", "tid": 76 }, { "created_ino": "0x1000000023d", "tid": 77 }, { "created_ino": "0x1000000023e", "tid": 78 }, { "created_ino": "0x1000000023f", "tid": 79 }, { "created_ino": "0x10000000240", "tid": 80 }, { "created_ino": "0x10000000241", "tid": 81 }, { "created_ino": "0x10000000242", "tid": 82 }, { "created_ino": "0x10000000243", "tid": 83 }, { "created_ino": "0x10000000244", "tid": 84 }, { "created_ino": "0x10000000245", "tid": 85 }, { "created_ino": "0x10000000246", "tid": 86 }, { "created_ino": "0x10000000247", "tid": 87 }, { "created_ino": "0x10000000248", "tid": 88 }, { "created_ino": "0x10000000249", "tid": 89 }, { "created_ino": "0x1000000024a", "tid": 90 }, { "created_ino": "0x1000000024b", "tid": 91 }, { "created_ino": "0x1000000024c", "tid": 92 }, { "created_ino": "0x1000000024d", "tid": 93 }, { "created_ino": "0x1000000024e", "tid": 94 }, { "created_ino": "0x1000000024f", "tid": 95 }, { "created_ino": "0x10000000250", "tid": 96 }, { "created_ino": "0x10000000251", "tid": 97 }, { "created_ino": "0x10000000252", "tid": 98 }, { "created_ino": "0x10000000253", "tid": 99 }, { "created_ino": "0x10000000254", "tid": 100 }, { "created_ino": "0x10000000255", "tid": 101 }, { "created_ino": "0x10000000256", "tid": 102 }, { "created_ino": "0x10000000257", "tid": 103 }, { "created_ino": "0x10000000258", "tid": 104 }, { "created_ino": "0x10000000259", "tid": 105 }, { "created_ino": "0x1000000025a", "tid": 106 }, { "created_ino": "0x1000000025b", "tid": 107 }, { "created_ino": "0x1000000025c", "tid": 108 }, { "created_ino": "0x1000000025d", "tid": 109 }, { "created_ino": "0x1000000025e", "tid": 110 }, { "created_ino": "0x1000000025f", "tid": 111 }, { "created_ino": "0x10000000260", "tid": 112 }, { "created_ino": "0x10000000261", "tid": 113 }, { "created_ino": "0x10000000262", "tid": 114 }, { "created_ino": "0x10000000263", "tid": 115 }, { "created_ino": "0x10000000264", "tid": 116 }, { "created_ino": "0x10000000265", "tid": 117 }, { "created_ino": "0x10000000266", "tid": 118 }, { "created_ino": "0x10000000267", "tid": 119 }, { "created_ino": "0x10000000268", "tid": 120 }, { "created_ino": "0x10000000269", "tid": 121 }, { "created_ino": "0x1000000026a", "tid": 122 }, { "created_ino": "0x1000000026b", "tid": 123 }, { "created_ino": "0x1000000026c", "tid": 124 }, { "created_ino": "0x1000000026d", "tid": 125 }, { "created_ino": "0x1000000026e", "tid": 126 }, { "created_ino": "0x1000000026f", "tid": 127 }, { "created_ino": "0x10000000270", "tid": 128 }, { "created_ino": "0x10000000271", "tid": 129 }, { "created_ino": "0x10000000272", "tid": 130 }, { "created_ino": "0x10000000273", "tid": 131 }, { "created_ino": "0x10000000274", "tid": 132 }, { "created_ino": "0x10000000275", "tid": 133 }, { "created_ino": "0x10000000276", "tid": 134 }, { "created_ino": "0x10000000277", "tid": 135 }, { "created_ino": "0x10000000278", "tid": 136 }, { "created_ino": "0x10000000279", "tid": 137 }, { "created_ino": "0x1000000027a", "tid": 138 }, { "created_ino": "0x1000000027b", "tid": 139 }, { "created_ino": "0x1000000027c", "tid": 140 }, { "created_ino": "0x1000000027d", "tid": 141 }, { "created_ino": "0x1000000027e", "tid": 142 }, { "created_ino": "0x1000000027f", "tid": 143 }, { "created_ino": "0x10000000280", "tid": 144 }, { "created_ino": "0x10000000281", "tid": 145 }, { "created_ino": "0x10000000282", "tid": 146 }, { "created_ino": "0x10000000283", "tid": 147 }, { "created_ino": "0x10000000284", "tid": 148 }, { "created_ino": "0x10000000285", "tid": 149 }, { "created_ino": "0x10000000286", "tid": 150 }, { "created_ino": "0x10000000287", "tid": 151 }, { "created_ino": "0x10000000288", "tid": 152 }, { "created_ino": "0x10000000289", "tid": 153 }, { "created_ino": "0x1000000028a", "tid": 154 }, { "created_ino": "0x1000000028b", "tid": 155 }, { "created_ino": "0x1000000028c", "tid": 156 }, { "created_ino": "0x1000000028d", "tid": 157 }, { "created_ino": "0x1000000028e", "tid": 158 }, { "created_ino": "0x1000000028f", "tid": 159 }, { "created_ino": "0x10000000290", "tid": 160 }, { "created_ino": "0x10000000291", "tid": 161 }, { "created_ino": "0x10000000292", "tid": 162 }, { "created_ino": "0x10000000293", "tid": 163 }, { "created_ino": "0x10000000294", "tid": 164 }, { "created_ino": "0x10000000295", "tid": 165 }, { "created_ino": "0x10000000296", "tid": 166 }, { "created_ino": "0x10000000297", "tid": 167 }, { "created_ino": "0x10000000298", "tid": 168 }, { "created_ino": "0x10000000299", "tid": 169 }, { "created_ino": "0x1000000029a", "tid": 170 }, { "created_ino": "0x1000000029b", "tid": 171 }, { "created_ino": "0x1000000029c", "tid": 172 }, { "created_ino": "0x1000000029d", "tid": 173 }, { "created_ino": "0x1000000029e", "tid": 174 }, { "created_ino": "0x1000000029f", "tid": 175 }, { "created_ino": "0x100000002a0", "tid": 176 }, { "created_ino": "0x100000002a1", "tid": 177 }, { "created_ino": "0x100000002a2", "tid": 178 }, { "created_ino": "0x100000002a3", "tid": 179 }, { "created_ino": "0x100000002a4", "tid": 180 }, { "created_ino": "0x100000002a5", "tid": 181 }, { "created_ino": "0x100000002a6", "tid": 182 }, { "created_ino": "0x100000002a7", "tid": 183 }, { "created_ino": "0x100000002a8", "tid": 184 }, { "created_ino": "0x100000002a9", "tid": 185 }, { "created_ino": "0x100000002aa", "tid": 186 }, { "created_ino": "0x100000002ab", "tid": 187 }, { "created_ino": "0x100000002ac", "tid": 188 }, { "created_ino": "0x100000002ad", "tid": 189 }, { "created_ino": "0x100000002ae", "tid": 190 }, { "created_ino": "0x100000002af", "tid": 191 }, { "created_ino": "0x100000002b0", "tid": 192 }, { "created_ino": "0x100000002b1", "tid": 193 }, { "created_ino": "0x100000002b2", "tid": 194 }, { "created_ino": "0x100000002b3", "tid": 195 }, { "created_ino": "0x100000002b4", "tid": 196 }, { "created_ino": "0x100000002b5", "tid": 197 }, { "created_ino": "0x100000002b6", "tid": 198 }, { "created_ino": "0x100000002b7", "tid": 199 }, { "created_ino": "0x100000002b8", "tid": 200 }, { "created_ino": "0x100000002b9", "tid": 201 }, { "created_ino": "0x100000002ba", "tid": 202 }, { "created_ino": "0x100000002bb", "tid": 203 }, { "created_ino": "0x100000002bc", "tid": 204 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1730080076, "type": "any" }, "name": { "num": 8586, "type": "client" } }, "id": 8586, "inst": "client.8586 192.168.0.1:0/1730080076", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 200, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 802, "start": "0x100000002bd" } ], "recall_caps": { "halflife": 60, "value": 766.5134200897365 }, "recall_caps_throttle": { "halflife": 1.5, "value": 236.95529683602166 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 97.0288753505404 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 187, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 398.812406416811 }, "state": "open", "uptime": 14.953845787 } ] 2023-07-13T22:30:13.031 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:30:13.031 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:30:13.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:30:13.137 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:30:13.244 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:30:13.245 INFO:tasks.cephfs.mount:Terminating background process 2023-07-13T22:30:13.245 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:30:13.246 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:13.293 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:30:13.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.557+0000 7efe8e7ff700 1 Processor -- start 2023-07-13T22:30:13.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.557+0000 7efe8e7ff700 1 -- start start 2023-07-13T22:30:13.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.557+0000 7efe8e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe8815c300 0x7efe880b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:13.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.557+0000 7efe8e7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efe8815c6d0 0x7efe880cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:13.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.557+0000 7efe8e7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe880d18a0 con 0x7efe880b46c0 2023-07-13T22:30:13.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.557+0000 7efe8e7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe880d15c0 con 0x7efe8815c6d0 2023-07-13T22:30:13.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.557+0000 7efe8e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe880cd3c0 con 0x7efe8815c300 2023-07-13T22:30:13.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.558+0000 7efe87fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe8815c300 0x7efe880b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.558+0000 7efe877fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efe8815c6d0 0x7efe880cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.558+0000 7efe87fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe8815c300 0x7efe880b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:57066/0 (socket says 172.21.15.60:57066) 2023-07-13T22:30:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.558+0000 7efe87fff700 1 -- 172.21.15.60:0/496422459 learned_addr learned my addr 172.21.15.60:0/496422459 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.558+0000 7efe87fff700 1 -- 172.21.15.60:0/496422459 >> v2:172.21.15.105:3300/0 conn(0x7efe8815c6d0 msgr2=0x7efe880cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.558+0000 7efe87fff700 1 --2- 172.21.15.60:0/496422459 >> v2:172.21.15.105:3300/0 conn(0x7efe8815c6d0 0x7efe880cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.558+0000 7efe87fff700 1 -- 172.21.15.60:0/496422459 >> v1:172.21.15.105:6789/0 conn(0x7efe880b46c0 legacy=0x7efe8806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.558+0000 7efe87fff700 1 -- 172.21.15.60:0/496422459 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880cd6a0 con 0x7efe8815c300 2023-07-13T22:30:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.558+0000 7efe87fff700 1 --2- 172.21.15.60:0/496422459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe8815c300 0x7efe880b5a90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe780098f0 tx=0x7efe78009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289f5270453dd157 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:13.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe86ffd700 1 -- 172.21.15.60:0/496422459 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78011830 con 0x7efe8815c300 2023-07-13T22:30:13.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe86ffd700 1 -- 172.21.15.60:0/496422459 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe78011990 con 0x7efe8815c300 2023-07-13T22:30:13.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe86ffd700 1 -- 172.21.15.60:0/496422459 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78011d20 con 0x7efe8815c300 2023-07-13T22:30:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe8e7ff700 1 -- 172.21.15.60:0/496422459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe8815c300 msgr2=0x7efe880b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe8e7ff700 1 --2- 172.21.15.60:0/496422459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe8815c300 0x7efe880b5a90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe780098f0 tx=0x7efe78009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe8e7ff700 1 -- 172.21.15.60:0/496422459 shutdown_connections 2023-07-13T22:30:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe8e7ff700 1 --2- 172.21.15.60:0/496422459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe8815c300 0x7efe880b5a90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe8e7ff700 1 --2- 172.21.15.60:0/496422459 >> v2:172.21.15.105:3300/0 conn(0x7efe8815c6d0 0x7efe880cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe8e7ff700 1 -- 172.21.15.60:0/496422459 >> 172.21.15.60:0/496422459 conn(0x7efe880cf6c0 msgr2=0x7efe880cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:13.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.559+0000 7efe8e7ff700 1 -- 172.21.15.60:0/496422459 shutdown_connections 2023-07-13T22:30:13.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.560+0000 7efe8e7ff700 1 -- 172.21.15.60:0/496422459 wait complete. 2023-07-13T22:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.561+0000 7efe8e7ff700 1 Processor -- start 2023-07-13T22:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.562+0000 7efe8e7ff700 1 -- start start 2023-07-13T22:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe8e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe880b46c0 0x7efe880cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe8e7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8815c300 0x7efe880c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe8e7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe8815c6d0 0x7efe881576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe8e7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8815d460 con 0x7efe8815c300 2023-07-13T22:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe8e7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe880ccb30 con 0x7efe8815c6d0 2023-07-13T22:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe8e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8815cb30 con 0x7efe880b46c0 2023-07-13T22:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8815c300 0x7efe880c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8815c300 0x7efe880c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53998/0 (socket says 172.21.15.60:53998) 2023-07-13T22:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe877fe700 1 -- 172.21.15.60:0/691361643 learned_addr learned my addr 172.21.15.60:0/691361643 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:13.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe87fff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe880b46c0 0x7efe880cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:13.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe8cd9c700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe8815c6d0 0x7efe881576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:13.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe877fe700 1 -- 172.21.15.60:0/691361643 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe880b46c0 msgr2=0x7efe880cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:13.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe877fe700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe880b46c0 0x7efe880cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe877fe700 1 -- 172.21.15.60:0/691361643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe8815c6d0 msgr2=0x7efe881576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:13.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe877fe700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe8815c6d0 0x7efe881576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe877fe700 1 -- 172.21.15.60:0/691361643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe78009550 con 0x7efe8815c300 2023-07-13T22:30:13.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.563+0000 7efe87fff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe880b46c0 0x7efe880cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:13.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.564+0000 7efe877fe700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8815c300 0x7efe880c9d10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00e840 tx=0x7efe7c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:13.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.564+0000 7efe84ff9700 1 -- 172.21.15.60:0/691361643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c009590 con 0x7efe8815c300 2023-07-13T22:30:13.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.564+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8815ce10 con 0x7efe8815c300 2023-07-13T22:30:13.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.564+0000 7efe84ff9700 1 -- 172.21.15.60:0/691361643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c013680 con 0x7efe8815c300 2023-07-13T22:30:13.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.564+0000 7efe84ff9700 1 -- 172.21.15.60:0/691361643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c013a30 con 0x7efe8815c300 2023-07-13T22:30:13.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.564+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe88158770 con 0x7efe8815c300 2023-07-13T22:30:13.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.565+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8815d610 con 0x7efe8815c300 2023-07-13T22:30:13.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.566+0000 7efe84ff9700 1 -- 172.21.15.60:0/691361643 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe7c013b90 con 0x7efe8815c300 2023-07-13T22:30:13.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.567+0000 7efe84ff9700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efe7406ace0 0x7efe7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:13.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.567+0000 7efe84ff9700 1 -- 172.21.15.60:0/691361643 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe7c089410 con 0x7efe8815c300 2023-07-13T22:30:13.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.567+0000 7efe87fff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efe7406ace0 0x7efe7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:13.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.568+0000 7efe87fff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efe7406ace0 0x7efe7406d190 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efe78009c40 tx=0x7efe78008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:13.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.573+0000 7efe84ff9700 1 -- 172.21.15.60:0/691361643 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efe8815d610 con 0x7efe8815c300 2023-07-13T22:30:13.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.780+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe880cdbd0 con 0x7efe8815c300 2023-07-13T22:30:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.782+0000 7efe84ff9700 1 -- 172.21.15.60:0/691361643 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18267 (secure 0 0 0) 0x7efe7c057f00 con 0x7efe8815c300 2023-07-13T22:30:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:13.761 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:29:50.779149+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:29:47.410626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-13T22:29:48.264611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:30:13.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efe7406ace0 msgr2=0x7efe7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:13.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efe7406ace0 0x7efe7406d190 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efe78009c40 tx=0x7efe78008030 comp rx=0 tx=0).stop 2023-07-13T22:30:13.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8815c300 msgr2=0x7efe880c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8815c300 0x7efe880c9d10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00e840 tx=0x7efe7c00ec00 comp rx=0 tx=0).stop 2023-07-13T22:30:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 shutdown_connections 2023-07-13T22:30:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efe7406ace0 0x7efe7406d190 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe8815c6d0 0x7efe881576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe880b46c0 0x7efe880cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 --2- 172.21.15.60:0/691361643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8815c300 0x7efe880c9d10 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 >> 172.21.15.60:0/691361643 conn(0x7efe880cf6c0 msgr2=0x7efe8815b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 shutdown_connections 2023-07-13T22:30:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:13.784+0000 7efe8e7ff700 1 -- 172.21.15.60:0/691361643 wait complete. 2023-07-13T22:30:13.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:29:47.410626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:29:48.264611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:30:13.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:30:13.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.015+0000 7fae31fff700 1 Processor -- start 2023-07-13T22:30:13.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.015+0000 7fae31fff700 1 -- start start 2023-07-13T22:30:13.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.016+0000 7fae31fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c158930 0x7fae2c0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:13.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.016+0000 7fae31fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae2c06a5e0 0x7fae2c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:13.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.016+0000 7fae31fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c06d030 con 0x7fae2c15c4a0 2023-07-13T22:30:13.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.016+0000 7fae31fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2c0c9620 con 0x7fae2c06a5e0 2023-07-13T22:30:13.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.016+0000 7fae31fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c06ca70 con 0x7fae2c158930 2023-07-13T22:30:13.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae2bfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae2c06a5e0 0x7fae2c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:13.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae2b7fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c158930 0x7fae2c0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:13.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae2bfff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae2c06a5e0 0x7fae2c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:39694/0 (socket says 172.21.15.60:39694) 2023-07-13T22:30:13.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae2bfff700 1 -- 172.21.15.60:0/4184802126 learned_addr learned my addr 172.21.15.60:0/4184802126 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:13.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae23fff700 1 --1- 172.21.15.60:0/4184802126 >> v1:172.21.15.105:6789/0 conn(0x7fae2c15c4a0 0x7fae2c069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:49166/0 (socket says 172.21.15.60:49166) 2023-07-13T22:30:13.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae2b7fe700 1 -- 172.21.15.60:0/4184802126 >> v2:172.21.15.105:3300/0 conn(0x7fae2c06a5e0 msgr2=0x7fae2c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:13.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae2b7fe700 1 --2- 172.21.15.60:0/4184802126 >> v2:172.21.15.105:3300/0 conn(0x7fae2c06a5e0 0x7fae2c15c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae2b7fe700 1 -- 172.21.15.60:0/4184802126 >> v1:172.21.15.105:6789/0 conn(0x7fae2c15c4a0 legacy=0x7fae2c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:13.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae2b7fe700 1 -- 172.21.15.60:0/4184802126 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0adc20 con 0x7fae2c158930 2023-07-13T22:30:13.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.017+0000 7fae2bfff700 1 --2- 172.21.15.60:0/4184802126 >> v2:172.21.15.105:3300/0 conn(0x7fae2c06a5e0 0x7fae2c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:13.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae2b7fe700 1 --2- 172.21.15.60:0/4184802126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c158930 0x7fae2c0c90e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00bae0 tx=0x7fae1c00bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f69bd6a90809eef server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:13.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae2affd700 1 -- 172.21.15.60:0/4184802126 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c00dc70 con 0x7fae2c158930 2023-07-13T22:30:13.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae2affd700 1 -- 172.21.15.60:0/4184802126 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c00ddd0 con 0x7fae2c158930 2023-07-13T22:30:13.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae2affd700 1 -- 172.21.15.60:0/4184802126 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c010690 con 0x7fae2c158930 2023-07-13T22:30:13.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae31fff700 1 -- 172.21.15.60:0/4184802126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c158930 msgr2=0x7fae2c0c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:13.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae31fff700 1 --2- 172.21.15.60:0/4184802126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c158930 0x7fae2c0c90e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00bae0 tx=0x7fae1c00bdf0 comp rx=0 tx=0).stop 2023-07-13T22:30:13.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae31fff700 1 -- 172.21.15.60:0/4184802126 shutdown_connections 2023-07-13T22:30:13.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae31fff700 1 --2- 172.21.15.60:0/4184802126 >> v2:172.21.15.105:3300/0 conn(0x7fae2c06a5e0 0x7fae2c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae31fff700 1 --2- 172.21.15.60:0/4184802126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c158930 0x7fae2c0c90e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae31fff700 1 -- 172.21.15.60:0/4184802126 >> 172.21.15.60:0/4184802126 conn(0x7fae2c0cbf50 msgr2=0x7fae2c0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:14.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae31fff700 1 -- 172.21.15.60:0/4184802126 shutdown_connections 2023-07-13T22:30:14.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.019+0000 7fae31fff700 1 -- 172.21.15.60:0/4184802126 wait complete. 2023-07-13T22:30:14.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.020+0000 7fae31fff700 1 Processor -- start 2023-07-13T22:30:14.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.021+0000 7fae31fff700 1 -- start start 2023-07-13T22:30:14.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.021+0000 7fae31fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae2c06a5e0 0x7fae2c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae31fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae2c158930 0x7fae2c152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae31fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c15c4a0 0x7fae2c0ac210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae31fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0c5f40 con 0x7fae2c06a5e0 2023-07-13T22:30:14.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae31fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0c5910 con 0x7fae2c158930 2023-07-13T22:30:14.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae31fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0affd0 con 0x7fae2c15c4a0 2023-07-13T22:30:14.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae2c06a5e0 0x7fae2c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae2c06a5e0 0x7fae2c0c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54010/0 (socket says 172.21.15.60:54010) 2023-07-13T22:30:14.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2b7fe700 1 -- 172.21.15.60:0/1013341853 learned_addr learned my addr 172.21.15.60:0/1013341853 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:14.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae23fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae2c158930 0x7fae2c152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2bfff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c15c4a0 0x7fae2c0ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2b7fe700 1 -- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c15c4a0 msgr2=0x7fae2c0ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2b7fe700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c15c4a0 0x7fae2c0ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2b7fe700 1 -- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae2c158930 msgr2=0x7fae2c152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2b7fe700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae2c158930 0x7fae2c152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2b7fe700 1 -- 172.21.15.60:0/1013341853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24009b30 con 0x7fae2c06a5e0 2023-07-13T22:30:14.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae23fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae2c158930 0x7fae2c152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:14.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.022+0000 7fae2bfff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c15c4a0 0x7fae2c0ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:14.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.023+0000 7fae2b7fe700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae2c06a5e0 0x7fae2c0c6360 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00b2c0 tx=0x7fae1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:14.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.023+0000 7fae28ff9700 1 -- 172.21.15.60:0/1013341853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c003cf0 con 0x7fae2c06a5e0 2023-07-13T22:30:14.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.023+0000 7fae28ff9700 1 -- 172.21.15.60:0/1013341853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c003e70 con 0x7fae2c06a5e0 2023-07-13T22:30:14.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.023+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c00b740 con 0x7fae2c06a5e0 2023-07-13T22:30:14.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.023+0000 7fae28ff9700 1 -- 172.21.15.60:0/1013341853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c010690 con 0x7fae2c06a5e0 2023-07-13T22:30:14.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.023+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c0c5b60 con 0x7fae2c06a5e0 2023-07-13T22:30:14.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.024+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c04e210 con 0x7fae2c06a5e0 2023-07-13T22:30:14.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.025+0000 7fae28ff9700 1 -- 172.21.15.60:0/1013341853 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fae1c019400 con 0x7fae2c06a5e0 2023-07-13T22:30:14.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.027+0000 7fae28ff9700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fae0c06ace0 0x7fae0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.027+0000 7fae28ff9700 1 -- 172.21.15.60:0/1013341853 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fae1c08aec0 con 0x7fae2c06a5e0 2023-07-13T22:30:14.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.027+0000 7fae23fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fae0c06ace0 0x7fae0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.027+0000 7fae23fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fae0c06ace0 0x7fae0c06d190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae2c154570 tx=0x7fae14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:14.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.031+0000 7fae28ff9700 1 -- 172.21.15.60:0/1013341853 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fae1c055f60 con 0x7fae2c06a5e0 2023-07-13T22:30:14.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.233+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fae2c0c61b0 con 0x7fae2c06a5e0 2023-07-13T22:30:14.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.234+0000 7fae28ff9700 1 -- 172.21.15.60:0/1013341853 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v112) v1 ==== 92+0+10 (secure 0 0 0) 0x7fae1c059580 con 0x7fae2c06a5e0 2023-07-13T22:30:14.213 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:30:14.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fae0c06ace0 msgr2=0x7fae0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fae0c06ace0 0x7fae0c06d190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae2c154570 tx=0x7fae14009040 comp rx=0 tx=0).stop 2023-07-13T22:30:14.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae2c06a5e0 msgr2=0x7fae2c0c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae2c06a5e0 0x7fae2c0c6360 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00b2c0 tx=0x7fae1c004750 comp rx=0 tx=0).stop 2023-07-13T22:30:14.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 shutdown_connections 2023-07-13T22:30:14.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fae0c06ace0 0x7fae0c06d190 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae2c06a5e0 0x7fae2c0c6360 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae2c15c4a0 0x7fae2c0ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 --2- 172.21.15.60:0/1013341853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae2c158930 0x7fae2c152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 >> 172.21.15.60:0/1013341853 conn(0x7fae2c0cbf50 msgr2=0x7fae2c0ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:14.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 shutdown_connections 2023-07-13T22:30:14.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.236+0000 7fae31fff700 1 -- 172.21.15.60:0/1013341853 wait complete. 2023-07-13T22:30:14.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:30:14.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.460+0000 7f15f37ff700 1 Processor -- start 2023-07-13T22:30:14.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.460+0000 7f15f37ff700 1 -- start start 2023-07-13T22:30:14.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.461+0000 7f15f37ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15ec0cfd50 0x7f15ec0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.461+0000 7f15f37ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15ec0b1ac0 0x7f15ec0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.461+0000 7f15f37ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ec0d1870 con 0x7f15ec0b8200 2023-07-13T22:30:14.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.461+0000 7f15f37ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ec0cf370 con 0x7f15ec0b1ac0 2023-07-13T22:30:14.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.461+0000 7f15f37ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ec0d12b0 con 0x7f15ec0cfd50 2023-07-13T22:30:14.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.461+0000 7f15f1d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15ec0b1ac0 0x7f15ec0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.461+0000 7f15f1d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15ec0b1ac0 0x7f15ec0d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54024/0 (socket says 172.21.15.60:54024) 2023-07-13T22:30:14.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.461+0000 7f15f1d9c700 1 -- 172.21.15.60:0/1764093372 learned_addr learned my addr 172.21.15.60:0/1764093372 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:14.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.461+0000 7f15f0d9a700 1 --2- 172.21.15.60:0/1764093372 >> v2:172.21.15.105:3300/0 conn(0x7f15ec0cfd50 0x7f15ec0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.462+0000 7f15f1d9c700 1 -- 172.21.15.60:0/1764093372 >> v2:172.21.15.105:3300/0 conn(0x7f15ec0cfd50 msgr2=0x7f15ec0d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:14.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.462+0000 7f15f1d9c700 1 --2- 172.21.15.60:0/1764093372 >> v2:172.21.15.105:3300/0 conn(0x7f15ec0cfd50 0x7f15ec0d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.462+0000 7f15f1d9c700 1 -- 172.21.15.60:0/1764093372 >> v1:172.21.15.105:6789/0 conn(0x7f15ec0b8200 legacy=0x7f15ec0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.462+0000 7f15f1d9c700 1 -- 172.21.15.60:0/1764093372 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec0d1590 con 0x7f15ec0b1ac0 2023-07-13T22:30:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.462+0000 7f15f1d9c700 1 --2- 172.21.15.60:0/1764093372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0b1ac0 0x7f15ec0d2900 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0d07b0 tx=0x7f15e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6408ea0f7ba79f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.462+0000 7f15e3fff700 1 -- 172.21.15.60:0/1764093372 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e400e070 con 0x7f15ec0b1ac0 2023-07-13T22:30:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.462+0000 7f15e3fff700 1 -- 172.21.15.60:0/1764093372 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e400b910 con 0x7f15ec0b1ac0 2023-07-13T22:30:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.462+0000 7f15e3fff700 1 -- 172.21.15.60:0/1764093372 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e400bcc0 con 0x7f15ec0b1ac0 2023-07-13T22:30:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.463+0000 7f15f37ff700 1 -- 172.21.15.60:0/1764093372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0b1ac0 msgr2=0x7f15ec0d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.463+0000 7f15f37ff700 1 --2- 172.21.15.60:0/1764093372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0b1ac0 0x7f15ec0d2900 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0d07b0 tx=0x7f15e400b330 comp rx=0 tx=0).stop 2023-07-13T22:30:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.463+0000 7f15f37ff700 1 -- 172.21.15.60:0/1764093372 shutdown_connections 2023-07-13T22:30:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.463+0000 7f15f37ff700 1 --2- 172.21.15.60:0/1764093372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0b1ac0 0x7f15ec0d2900 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.463+0000 7f15f37ff700 1 --2- 172.21.15.60:0/1764093372 >> v2:172.21.15.105:3300/0 conn(0x7f15ec0cfd50 0x7f15ec0d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.463+0000 7f15f37ff700 1 -- 172.21.15.60:0/1764093372 >> 172.21.15.60:0/1764093372 conn(0x7f15ec06b2f0 msgr2=0x7f15ec06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.463+0000 7f15f37ff700 1 -- 172.21.15.60:0/1764093372 shutdown_connections 2023-07-13T22:30:14.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.463+0000 7f15f37ff700 1 -- 172.21.15.60:0/1764093372 wait complete. 2023-07-13T22:30:14.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.465+0000 7f15f37ff700 1 Processor -- start 2023-07-13T22:30:14.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.465+0000 7f15f37ff700 1 -- start start 2023-07-13T22:30:14.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f37ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ec0b1ac0 0x7f15ec156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f37ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15ec0b8200 0x7f15ec1f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f37ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0cfd50 0x7f15ec0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f37ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec158670 con 0x7f15ec0cfd50 2023-07-13T22:30:14.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f37ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec159e80 con 0x7f15ec0b1ac0 2023-07-13T22:30:14.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f37ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec1f2230 con 0x7f15ec0b8200 2023-07-13T22:30:14.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f1d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0cfd50 0x7f15ec0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f1d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0cfd50 0x7f15ec0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54040/0 (socket says 172.21.15.60:54040) 2023-07-13T22:30:14.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f1d9c700 1 -- 172.21.15.60:0/854919205 learned_addr learned my addr 172.21.15.60:0/854919205 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:14.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f0d9a700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15ec0b8200 0x7f15ec1f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f159b700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ec0b1ac0 0x7f15ec156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f1d9c700 1 -- 172.21.15.60:0/854919205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15ec0b8200 msgr2=0x7f15ec1f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f1d9c700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15ec0b8200 0x7f15ec1f18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f1d9c700 1 -- 172.21.15.60:0/854919205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ec0b1ac0 msgr2=0x7f15ec156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f1d9c700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ec0b1ac0 0x7f15ec156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f1d9c700 1 -- 172.21.15.60:0/854919205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e4009b30 con 0x7f15ec0cfd50 2023-07-13T22:30:14.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f159b700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ec0b1ac0 0x7f15ec156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:14.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f1d9c700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0cfd50 0x7f15ec0b6390 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f15ec06bf10 tx=0x7f15e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15e1ffb700 1 -- 172.21.15.60:0/854919205 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4004930 con 0x7f15ec0cfd50 2023-07-13T22:30:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15e1ffb700 1 -- 172.21.15.60:0/854919205 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e40099b0 con 0x7f15ec0cfd50 2023-07-13T22:30:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec1f2510 con 0x7f15ec0cfd50 2023-07-13T22:30:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15e1ffb700 1 -- 172.21.15.60:0/854919205 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e401c680 con 0x7f15ec0cfd50 2023-07-13T22:30:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.467+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec1f2d10 con 0x7f15ec0cfd50 2023-07-13T22:30:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.468+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec1f2ed0 con 0x7f15ec0cfd50 2023-07-13T22:30:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.469+0000 7f15e1ffb700 1 -- 172.21.15.60:0/854919205 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15e401c7e0 con 0x7f15ec0cfd50 2023-07-13T22:30:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.470+0000 7f15e1ffb700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f15d006acb0 0x7f15d006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.470+0000 7f15e1ffb700 1 -- 172.21.15.60:0/854919205 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f15e408ae40 con 0x7f15ec0cfd50 2023-07-13T22:30:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.471+0000 7f15f159b700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f15d006acb0 0x7f15d006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.471+0000 7f15f159b700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f15d006acb0 0x7f15d006d160 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f15ec1560c0 tx=0x7f15e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.475+0000 7f15e1ffb700 1 -- 172.21.15.60:0/854919205 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f15e4055ee0 con 0x7f15ec0cfd50 2023-07-13T22:30:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.681+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15ec1f29d0 con 0x7f15ec0cfd50 2023-07-13T22:30:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.681+0000 7f15e1ffb700 1 -- 172.21.15.60:0/854919205 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15e4059500 con 0x7f15ec0cfd50 2023-07-13T22:30:14.661 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:30:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f15d006acb0 msgr2=0x7f15d006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f15d006acb0 0x7f15d006d160 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f15ec1560c0 tx=0x7f15e8009040 comp rx=0 tx=0).stop 2023-07-13T22:30:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0cfd50 msgr2=0x7f15ec0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0cfd50 0x7f15ec0b6390 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f15ec06bf10 tx=0x7f15e4004ad0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 shutdown_connections 2023-07-13T22:30:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f15d006acb0 0x7f15d006d160 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec0cfd50 0x7f15ec0b6390 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ec0b1ac0 0x7f15ec156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 --2- 172.21.15.60:0/854919205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15ec0b8200 0x7f15ec1f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 >> 172.21.15.60:0/854919205 conn(0x7f15ec06b2f0 msgr2=0x7f15ec0c8770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 shutdown_connections 2023-07-13T22:30:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.684+0000 7f15f37ff700 1 -- 172.21.15.60:0/854919205 wait complete. 2023-07-13T22:30:14.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:30:14.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.909+0000 7f98797ff700 1 Processor -- start 2023-07-13T22:30:14.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.909+0000 7f98797ff700 1 -- start start 2023-07-13T22:30:14.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.910+0000 7f98797ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98740add00 0x7f987406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.910+0000 7f98797ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98740ca9f0 0x7f9874069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.910+0000 7f98797ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98740c8e90 con 0x7f987406a410 2023-07-13T22:30:14.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.910+0000 7f98797ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f987406d3b0 con 0x7f98740add00 2023-07-13T22:30:14.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.910+0000 7f98797ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98740ae0d0 con 0x7f98740ca9f0 2023-07-13T22:30:14.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f9872ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98740add00 0x7f987406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98737fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f987406a410 0x7f98740703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49496/0 (socket says 172.21.15.60:49496) 2023-07-13T22:30:14.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98737fe700 1 -- 172.21.15.60:0/2062876578 learned_addr learned my addr 172.21.15.60:0/2062876578 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:14.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98727fc700 1 --2- 172.21.15.60:0/2062876578 >> v2:172.21.15.105:3300/0 conn(0x7f98740ca9f0 0x7f9874069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f9872ffd700 1 -- 172.21.15.60:0/2062876578 >> v2:172.21.15.105:3300/0 conn(0x7f98740ca9f0 msgr2=0x7f9874069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:14.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f9872ffd700 1 --2- 172.21.15.60:0/2062876578 >> v2:172.21.15.105:3300/0 conn(0x7f98740ca9f0 0x7f9874069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f9872ffd700 1 -- 172.21.15.60:0/2062876578 >> v1:172.21.15.60:6789/0 conn(0x7f987406a410 legacy=0x7f98740703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f9872ffd700 1 -- 172.21.15.60:0/2062876578 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987406a820 con 0x7f98740add00 2023-07-13T22:30:14.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f9872ffd700 1 --2- 172.21.15.60:0/2062876578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98740add00 0x7f987406ce70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f9874071690 tx=0x7f9864009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfcfeede1963dae server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:14.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f9871ffb700 1 -- 172.21.15.60:0/2062876578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864011830 con 0x7f98740add00 2023-07-13T22:30:14.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f9871ffb700 1 -- 172.21.15.60:0/2062876578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864011990 con 0x7f98740add00 2023-07-13T22:30:14.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f9871ffb700 1 -- 172.21.15.60:0/2062876578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864011d20 con 0x7f98740add00 2023-07-13T22:30:14.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98797ff700 1 -- 172.21.15.60:0/2062876578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98740add00 msgr2=0x7f987406ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98797ff700 1 --2- 172.21.15.60:0/2062876578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98740add00 0x7f987406ce70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f9874071690 tx=0x7f9864009b20 comp rx=0 tx=0).stop 2023-07-13T22:30:14.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98797ff700 1 -- 172.21.15.60:0/2062876578 shutdown_connections 2023-07-13T22:30:14.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98797ff700 1 --2- 172.21.15.60:0/2062876578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98740add00 0x7f987406ce70 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98797ff700 1 --2- 172.21.15.60:0/2062876578 >> v2:172.21.15.105:3300/0 conn(0x7f98740ca9f0 0x7f9874069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98797ff700 1 -- 172.21.15.60:0/2062876578 >> 172.21.15.60:0/2062876578 conn(0x7f98740cbf50 msgr2=0x7f98740ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:14.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98797ff700 1 -- 172.21.15.60:0/2062876578 shutdown_connections 2023-07-13T22:30:14.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.911+0000 7f98797ff700 1 -- 172.21.15.60:0/2062876578 wait complete. 2023-07-13T22:30:14.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f98797ff700 1 Processor -- start 2023-07-13T22:30:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f98797ff700 1 -- start start 2023-07-13T22:30:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f98797ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f987406a410 0x7f987414bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f98797ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740add00 0x7f98740b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f98797ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ca9f0 0x7f98740b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f98797ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740c7430 con 0x7f987406a410 2023-07-13T22:30:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f98797ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740c6820 con 0x7f98740ca9f0 2023-07-13T22:30:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f98797ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98740c6b00 con 0x7f98740add00 2023-07-13T22:30:14.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f9872ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f987406a410 0x7f987414bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f9872ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f987406a410 0x7f987414bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37282/0 (socket says 172.21.15.60:37282) 2023-07-13T22:30:14.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.914+0000 7f9872ffd700 1 -- 172.21.15.60:0/73439130 learned_addr learned my addr 172.21.15.60:0/73439130 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:14.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f98727fc700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740add00 0x7f98740b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f98737fe700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ca9f0 0x7f98740b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f9872ffd700 1 -- 172.21.15.60:0/73439130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740add00 msgr2=0x7f98740b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f9872ffd700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740add00 0x7f98740b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f9872ffd700 1 -- 172.21.15.60:0/73439130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ca9f0 msgr2=0x7f98740b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:14.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f9872ffd700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ca9f0 0x7f98740b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:14.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f9872ffd700 1 -- 172.21.15.60:0/73439130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98640094a0 con 0x7f987406a410 2023-07-13T22:30:14.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f98737fe700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ca9f0 0x7f98740b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:14.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f9872ffd700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f987406a410 0x7f987414bb80 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f9864000c00 tx=0x7f9864004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:14.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f985b7fe700 1 -- 172.21.15.60:0/73439130 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98640036a0 con 0x7f987406a410 2023-07-13T22:30:14.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f985b7fe700 1 -- 172.21.15.60:0/73439130 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986401f070 con 0x7f987406a410 2023-07-13T22:30:14.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98740c6de0 con 0x7f987406a410 2023-07-13T22:30:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.915+0000 7f985b7fe700 1 -- 172.21.15.60:0/73439130 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864003a60 con 0x7f987406a410 2023-07-13T22:30:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.916+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987414c0c0 con 0x7f987406a410 2023-07-13T22:30:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.918+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987404e210 con 0x7f987406a410 2023-07-13T22:30:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.919+0000 7f985b7fe700 1 -- 172.21.15.60:0/73439130 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9864003bc0 con 0x7f987406a410 2023-07-13T22:30:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.920+0000 7f985b7fe700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f986006ace0 0x7f986006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.920+0000 7f985b7fe700 1 -- 172.21.15.60:0/73439130 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9864021070 con 0x7f987406a410 2023-07-13T22:30:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.920+0000 7f98727fc700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f986006ace0 0x7f986006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.920+0000 7f98727fc700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f986006ace0 0x7f986006d190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f98740b0e90 tx=0x7f9868008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:14.923+0000 7f985b7fe700 1 -- 172.21.15.60:0/73439130 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9864055ec0 con 0x7f987406a410 2023-07-13T22:30:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.128+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98740ac7f0 con 0x7f987406a410 2023-07-13T22:30:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.129+0000 7f985b7fe700 1 -- 172.21.15.60:0/73439130 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9864015020 con 0x7f987406a410 2023-07-13T22:30:15.107 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:30:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f986006ace0 msgr2=0x7f986006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f986006ace0 0x7f986006d190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f98740b0e90 tx=0x7f9868008040 comp rx=0 tx=0).stop 2023-07-13T22:30:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f987406a410 msgr2=0x7f987414bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f987406a410 0x7f987414bb80 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f9864000c00 tx=0x7f9864004750 comp rx=0 tx=0).stop 2023-07-13T22:30:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 shutdown_connections 2023-07-13T22:30:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f986006ace0 0x7f986006d190 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f987406a410 0x7f987414bb80 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ca9f0 0x7f98740b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 --2- 172.21.15.60:0/73439130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740add00 0x7f98740b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.131+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 >> 172.21.15.60:0/73439130 conn(0x7f98740cbf50 msgr2=0x7f98740acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.132+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 shutdown_connections 2023-07-13T22:30:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.132+0000 7f98797ff700 1 -- 172.21.15.60:0/73439130 wait complete. 2023-07-13T22:30:15.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:30:15.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.356+0000 7f0eabbff700 1 Processor -- start 2023-07-13T22:30:15.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.356+0000 7f0eabbff700 1 -- start start 2023-07-13T22:30:15.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.357+0000 7f0eabbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0ea40b8200 0x7f0ea40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.357+0000 7f0eabbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40cede0 0x7f0ea40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.357+0000 7f0eabbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea40d1870 con 0x7f0ea40b1ac0 2023-07-13T22:30:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.357+0000 7f0eabbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea40cf370 con 0x7f0ea40b8200 2023-07-13T22:30:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.357+0000 7f0eabbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea40d12b0 con 0x7f0ea40cede0 2023-07-13T22:30:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.358+0000 7f0ea999b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0ea40b8200 0x7f0ea40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.358+0000 7f0ea999b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0ea40b8200 0x7f0ea40cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37288/0 (socket says 172.21.15.60:37288) 2023-07-13T22:30:15.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.358+0000 7f0ea999b700 1 -- 172.21.15.60:0/3489162484 learned_addr learned my addr 172.21.15.60:0/3489162484 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.358+0000 7f0eaa19c700 1 --2- 172.21.15.60:0/3489162484 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40cede0 0x7f0ea40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.358+0000 7f0ea999b700 1 -- 172.21.15.60:0/3489162484 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40cede0 msgr2=0x7f0ea40cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.358+0000 7f0ea999b700 1 --2- 172.21.15.60:0/3489162484 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40cede0 0x7f0ea40cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.358+0000 7f0ea999b700 1 -- 172.21.15.60:0/3489162484 >> v1:172.21.15.105:6789/0 conn(0x7f0ea40b1ac0 legacy=0x7f0ea415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:15.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.358+0000 7f0ea999b700 1 -- 172.21.15.60:0/3489162484 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea40d1590 con 0x7f0ea40b8200 2023-07-13T22:30:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.358+0000 7f0ea999b700 1 --2- 172.21.15.60:0/3489162484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 0x7f0ea40cf640 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00098f0 tx=0x7f0ea0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dd67da9428a1f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0ea8999700 1 -- 172.21.15.60:0/3489162484 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0004230 con 0x7f0ea40b8200 2023-07-13T22:30:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0ea8999700 1 -- 172.21.15.60:0/3489162484 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea0004390 con 0x7f0ea40b8200 2023-07-13T22:30:15.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0ea8999700 1 -- 172.21.15.60:0/3489162484 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00036a0 con 0x7f0ea40b8200 2023-07-13T22:30:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0eabbff700 1 -- 172.21.15.60:0/3489162484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 msgr2=0x7f0ea40cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0eabbff700 1 --2- 172.21.15.60:0/3489162484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 0x7f0ea40cf640 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00098f0 tx=0x7f0ea0009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0eabbff700 1 -- 172.21.15.60:0/3489162484 shutdown_connections 2023-07-13T22:30:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0eabbff700 1 --2- 172.21.15.60:0/3489162484 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40cede0 0x7f0ea40cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0eabbff700 1 --2- 172.21.15.60:0/3489162484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 0x7f0ea40cf640 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0eabbff700 1 -- 172.21.15.60:0/3489162484 >> 172.21.15.60:0/3489162484 conn(0x7f0ea406b2f0 msgr2=0x7f0ea406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.359+0000 7f0eabbff700 1 -- 172.21.15.60:0/3489162484 shutdown_connections 2023-07-13T22:30:15.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.360+0000 7f0eabbff700 1 -- 172.21.15.60:0/3489162484 wait complete. 2023-07-13T22:30:15.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.361+0000 7f0eabbff700 1 Processor -- start 2023-07-13T22:30:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.361+0000 7f0eabbff700 1 -- start start 2023-07-13T22:30:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.362+0000 7f0eabbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40b1ac0 0x7f0ea41e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.362+0000 7f0eabbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 0x7f0ea41e1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.362+0000 7f0eabbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ea40cede0 0x7f0ea40cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.362+0000 7f0eabbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea41f2240 con 0x7f0ea40b8200 2023-07-13T22:30:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.362+0000 7f0eabbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea41e3f50 con 0x7f0ea40cede0 2023-07-13T22:30:15.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.362+0000 7f0eabbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea41e1cb0 con 0x7f0ea40b1ac0 2023-07-13T22:30:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 0x7f0ea41e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea919a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 0x7f0ea41e1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37302/0 (socket says 172.21.15.60:37302) 2023-07-13T22:30:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea919a700 1 -- 172.21.15.60:0/2963550062 learned_addr learned my addr 172.21.15.60:0/2963550062 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0eaa19c700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ea40cede0 0x7f0ea40cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea999b700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40b1ac0 0x7f0ea41e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea919a700 1 -- 172.21.15.60:0/2963550062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40b1ac0 msgr2=0x7f0ea41e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea919a700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40b1ac0 0x7f0ea41e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea919a700 1 -- 172.21.15.60:0/2963550062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ea40cede0 msgr2=0x7f0ea40cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea919a700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ea40cede0 0x7f0ea40cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea919a700 1 -- 172.21.15.60:0/2963550062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0009550 con 0x7f0ea40b8200 2023-07-13T22:30:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0ea919a700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 0x7f0ea41e1770 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400b4a0 tx=0x7f0e9400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.363+0000 7f0e9a7fc700 1 -- 172.21.15.60:0/2963550062 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e94011830 con 0x7f0ea40b8200 2023-07-13T22:30:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.364+0000 7f0e9a7fc700 1 -- 172.21.15.60:0/2963550062 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e94011990 con 0x7f0ea40b8200 2023-07-13T22:30:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.364+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea40c9590 con 0x7f0ea40b8200 2023-07-13T22:30:15.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.364+0000 7f0e9a7fc700 1 -- 172.21.15.60:0/2963550062 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e94011d40 con 0x7f0ea40b8200 2023-07-13T22:30:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.364+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea41e21c0 con 0x7f0ea40b8200 2023-07-13T22:30:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.365+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea41f2380 con 0x7f0ea40b8200 2023-07-13T22:30:15.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.366+0000 7f0e9a7fc700 1 -- 172.21.15.60:0/2963550062 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e9401a400 con 0x7f0ea40b8200 2023-07-13T22:30:15.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.367+0000 7f0e9a7fc700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0e8806ace0 0x7f0e8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.367+0000 7f0e9a7fc700 1 -- 172.21.15.60:0/2963550062 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e94088ff0 con 0x7f0ea40b8200 2023-07-13T22:30:15.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.367+0000 7f0ea999b700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0e8806ace0 0x7f0e8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.368+0000 7f0ea999b700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0e8806ace0 0x7f0e8806d190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009130 tx=0x7f0ea0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:15.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.371+0000 7f0e9a7fc700 1 -- 172.21.15.60:0/2963550062 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0e94054130 con 0x7f0ea40b8200 2023-07-13T22:30:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.573+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ea404e210 con 0x7f0ea40b8200 2023-07-13T22:30:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.573+0000 7f0e9a7fc700 1 -- 172.21.15.60:0/2963550062 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e94057750 con 0x7f0ea40b8200 2023-07-13T22:30:15.551 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:30:15.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.575+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0e8806ace0 msgr2=0x7f0e8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.575+0000 7f0eabbff700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0e8806ace0 0x7f0e8806d190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009130 tx=0x7f0ea0008030 comp rx=0 tx=0).stop 2023-07-13T22:30:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.575+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 msgr2=0x7f0ea41e1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.575+0000 7f0eabbff700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 0x7f0ea41e1770 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400b4a0 tx=0x7f0e9400b7b0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.576+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 shutdown_connections 2023-07-13T22:30:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.576+0000 7f0eabbff700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0e8806ace0 0x7f0e8806d190 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.576+0000 7f0eabbff700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ea40cede0 0x7f0ea40cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.576+0000 7f0eabbff700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ea40b1ac0 0x7f0ea41e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.576+0000 7f0eabbff700 1 --2- 172.21.15.60:0/2963550062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea40b8200 0x7f0ea41e1770 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.576+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 >> 172.21.15.60:0/2963550062 conn(0x7f0ea406b2f0 msgr2=0x7f0ea40d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.576+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 shutdown_connections 2023-07-13T22:30:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.576+0000 7f0eabbff700 1 -- 172.21.15.60:0/2963550062 wait complete. 2023-07-13T22:30:15.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:30:15.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.798+0000 7fddc8bff700 1 Processor -- start 2023-07-13T22:30:15.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.798+0000 7fddc8bff700 1 -- start start 2023-07-13T22:30:15.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.799+0000 7fddc8bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fddc40d3750 0x7fddc40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.799+0000 7fddc8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40cfad0 0x7fddc40d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.799+0000 7fddc8bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc40cd950 con 0x7fddc40b8400 2023-07-13T22:30:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.799+0000 7fddc8bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc40cff50 con 0x7fddc40d3750 2023-07-13T22:30:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.799+0000 7fddc8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40d0ac0 con 0x7fddc40cfad0 2023-07-13T22:30:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc1d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fddc40d3750 0x7fddc40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc1d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fddc40d3750 0x7fddc40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37316/0 (socket says 172.21.15.60:37316) 2023-07-13T22:30:15.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc1d9b700 1 -- 172.21.15.60:0/1827794355 learned_addr learned my addr 172.21.15.60:0/1827794355 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:15.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc2d9d700 1 --2- 172.21.15.60:0/1827794355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40cfad0 0x7fddc40d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc1d9b700 1 -- 172.21.15.60:0/1827794355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40cfad0 msgr2=0x7fddc40d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc1d9b700 1 --2- 172.21.15.60:0/1827794355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40cfad0 0x7fddc40d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc1d9b700 1 -- 172.21.15.60:0/1827794355 >> v1:172.21.15.105:6789/0 conn(0x7fddc40b8400 legacy=0x7fddc40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc1d9b700 1 -- 172.21.15.60:0/1827794355 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40b87d0 con 0x7fddc40d3750 2023-07-13T22:30:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc2d9d700 1 --2- 172.21.15.60:0/1827794355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40cfad0 0x7fddc40d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.800+0000 7fddc1d9b700 1 --2- 172.21.15.60:0/1827794355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40d3750 0x7fddc40cf590 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fddb80098f0 tx=0x7fddb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf09416b952a506e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc159a700 1 -- 172.21.15.60:0/1827794355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800b8a0 con 0x7fddc40d3750 2023-07-13T22:30:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc159a700 1 -- 172.21.15.60:0/1827794355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb800ba00 con 0x7fddc40d3750 2023-07-13T22:30:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc159a700 1 -- 172.21.15.60:0/1827794355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800bd90 con 0x7fddc40d3750 2023-07-13T22:30:15.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc8bff700 1 -- 172.21.15.60:0/1827794355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40d3750 msgr2=0x7fddc40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc8bff700 1 --2- 172.21.15.60:0/1827794355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40d3750 0x7fddc40cf590 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fddb80098f0 tx=0x7fddb8009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc8bff700 1 -- 172.21.15.60:0/1827794355 shutdown_connections 2023-07-13T22:30:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc8bff700 1 --2- 172.21.15.60:0/1827794355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40cfad0 0x7fddc40d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc8bff700 1 --2- 172.21.15.60:0/1827794355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40d3750 0x7fddc40cf590 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc8bff700 1 -- 172.21.15.60:0/1827794355 >> 172.21.15.60:0/1827794355 conn(0x7fddc406b390 msgr2=0x7fddc406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc8bff700 1 -- 172.21.15.60:0/1827794355 shutdown_connections 2023-07-13T22:30:15.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.801+0000 7fddc8bff700 1 -- 172.21.15.60:0/1827794355 wait complete. 2023-07-13T22:30:15.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.803+0000 7fddc8bff700 1 Processor -- start 2023-07-13T22:30:15.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.803+0000 7fddc8bff700 1 -- start start 2023-07-13T22:30:15.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.804+0000 7fddc8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40b8400 0x7fddc41f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.804+0000 7fddc8bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40cfad0 0x7fddc4151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.804+0000 7fddc8bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddc40d3750 0x7fddc415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.804+0000 7fddc8bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41520b0 con 0x7fddc40cfad0 2023-07-13T22:30:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.804+0000 7fddc8bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41f6520 con 0x7fddc40d3750 2023-07-13T22:30:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.804+0000 7fddc8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40c6c20 con 0x7fddc40b8400 2023-07-13T22:30:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40cfad0 0x7fddc4151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40cfad0 0x7fddc4151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37320/0 (socket says 172.21.15.60:37320) 2023-07-13T22:30:15.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc1d9b700 1 -- 172.21.15.60:0/2538235721 learned_addr learned my addr 172.21.15.60:0/2538235721 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc259c700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40b8400 0x7fddc41f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc2d9d700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddc40d3750 0x7fddc415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc1d9b700 1 -- 172.21.15.60:0/2538235721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40b8400 msgr2=0x7fddc41f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc1d9b700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40b8400 0x7fddc41f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc1d9b700 1 -- 172.21.15.60:0/2538235721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddc40d3750 msgr2=0x7fddc415aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:15.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc1d9b700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddc40d3750 0x7fddc415aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc1d9b700 1 -- 172.21.15.60:0/2538235721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8009550 con 0x7fddc40cfad0 2023-07-13T22:30:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddc1d9b700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40cfad0 0x7fddc4151090 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000c00 tx=0x7fddb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddb2ffd700 1 -- 172.21.15.60:0/2538235721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8004480 con 0x7fddc40cfad0 2023-07-13T22:30:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddb2ffd700 1 -- 172.21.15.60:0/2538235721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb801f070 con 0x7fddc40cfad0 2023-07-13T22:30:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.805+0000 7fddb2ffd700 1 -- 172.21.15.60:0/2538235721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80038c0 con 0x7fddc40cfad0 2023-07-13T22:30:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.806+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc40c6f00 con 0x7fddc40cfad0 2023-07-13T22:30:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.806+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc40c74c0 con 0x7fddc40cfad0 2023-07-13T22:30:15.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.807+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc40d19d0 con 0x7fddc40cfad0 2023-07-13T22:30:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.808+0000 7fddb2ffd700 1 -- 172.21.15.60:0/2538235721 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fddb8021030 con 0x7fddc40cfad0 2023-07-13T22:30:15.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.809+0000 7fddb2ffd700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdda406ace0 0x7fdda406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.809+0000 7fddb2ffd700 1 -- 172.21.15.60:0/2538235721 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fddb808aa20 con 0x7fddc40cfad0 2023-07-13T22:30:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.809+0000 7fddc259c700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdda406ace0 0x7fdda406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.810+0000 7fddc259c700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdda406ace0 0x7fdda406d190 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fddb4000f60 tx=0x7fddb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:15.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:15.813+0000 7fddb2ffd700 1 -- 172.21.15.60:0/2538235721 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fddb8055ac0 con 0x7fddc40cfad0 2023-07-13T22:30:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.037+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fddc40d16f0 con 0x7fddc40cfad0 2023-07-13T22:30:16.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.037+0000 7fddb2ffd700 1 -- 172.21.15.60:0/2538235721 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7fddb808a030 con 0x7fddc40cfad0 2023-07-13T22:30:16.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdda406ace0 msgr2=0x7fdda406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdda406ace0 0x7fdda406d190 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fddb4000f60 tx=0x7fddb4009040 comp rx=0 tx=0).stop 2023-07-13T22:30:16.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40cfad0 msgr2=0x7fddc4151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40cfad0 0x7fddc4151090 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000c00 tx=0x7fddb8004750 comp rx=0 tx=0).stop 2023-07-13T22:30:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 shutdown_connections 2023-07-13T22:30:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdda406ace0 0x7fdda406d190 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddc40d3750 0x7fddc415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fddc40b8400 0x7fddc41f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 --2- 172.21.15.60:0/2538235721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc40cfad0 0x7fddc4151090 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 >> 172.21.15.60:0/2538235721 conn(0x7fddc406b390 msgr2=0x7fddc415fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:16.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 shutdown_connections 2023-07-13T22:30:16.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.040+0000 7fddc8bff700 1 -- 172.21.15.60:0/2538235721 wait complete. 2023-07-13T22:30:16.020 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:30:16.035 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:30:16.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:16.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:30:16.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[157131]: fuse finished with error 0 and tester_r 0 2023-07-13T22:30:16.108 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:30:16.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:16.109 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:30:16.258 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:30:16.259 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:16.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:30:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.533+0000 7f81821ff700 1 Processor -- start 2023-07-13T22:30:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.533+0000 7f81821ff700 1 -- start start 2023-07-13T22:30:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.534+0000 7f81821ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f817c0d26d0 0x7f817c0cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.534+0000 7f81821ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817c0d14a0 con 0x7f817c0b2010 2023-07-13T22:30:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.534+0000 7f81821ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817c0d1780 con 0x7f817c0b5a00 2023-07-13T22:30:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.534+0000 7f81821ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817c0d2570 con 0x7f817c0d26d0 2023-07-13T22:30:16.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.534+0000 7f817b7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f817c0b2010 0x7f817c0cf1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49502/0 (socket says 172.21.15.60:49502) 2023-07-13T22:30:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.534+0000 7f817b7fe700 1 -- 172.21.15.60:0/2353159169 learned_addr learned my addr 172.21.15.60:0/2353159169 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:16.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.534+0000 7f817affd700 1 --2- 172.21.15.60:0/2353159169 >> v2:172.21.15.105:3300/0 conn(0x7f817c0d26d0 0x7f817c0cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.535+0000 7f817a7fc700 1 -- 172.21.15.60:0/2353159169 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845469973 0 0) 0x7f817c0d14a0 con 0x7f817c0b2010 2023-07-13T22:30:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.535+0000 7f817a7fc700 1 -- 172.21.15.60:0/2353159169 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81640035b0 con 0x7f817c0b2010 2023-07-13T22:30:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.535+0000 7f817a7fc700 1 -- 172.21.15.60:0/2353159169 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3168814386 0 0) 0x7f81640035b0 con 0x7f817c0b2010 2023-07-13T22:30:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.535+0000 7f817affd700 1 -- 172.21.15.60:0/2353159169 >> v1:172.21.15.105:6789/0 conn(0x7f817c0b5a00 legacy=0x7f817c15fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.535+0000 7f817affd700 1 -- 172.21.15.60:0/2353159169 >> v1:172.21.15.60:6789/0 conn(0x7f817c0b2010 legacy=0x7f817c0cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.535+0000 7f817affd700 1 -- 172.21.15.60:0/2353159169 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c0cd2a0 con 0x7f817c0d26d0 2023-07-13T22:30:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.535+0000 7f817affd700 1 --2- 172.21.15.60:0/2353159169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0d26d0 0x7f817c0cf8b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8170009840 tx=0x7f8170009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8721b6c650487d63 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:16.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.535+0000 7f817a7fc700 1 -- 172.21.15.60:0/2353159169 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c001670 con 0x7f817c0d26d0 2023-07-13T22:30:16.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.536+0000 7f817a7fc700 1 -- 172.21.15.60:0/2353159169 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8170004030 con 0x7f817c0d26d0 2023-07-13T22:30:16.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.536+0000 7f817a7fc700 1 -- 172.21.15.60:0/2353159169 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81700043c0 con 0x7f817c0d26d0 2023-07-13T22:30:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.536+0000 7f81821ff700 1 -- 172.21.15.60:0/2353159169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0d26d0 msgr2=0x7f817c0cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.536+0000 7f81821ff700 1 --2- 172.21.15.60:0/2353159169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0d26d0 0x7f817c0cf8b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8170009840 tx=0x7f8170009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.536+0000 7f81821ff700 1 -- 172.21.15.60:0/2353159169 shutdown_connections 2023-07-13T22:30:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.536+0000 7f81821ff700 1 --2- 172.21.15.60:0/2353159169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0d26d0 0x7f817c0cf8b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.536+0000 7f81821ff700 1 -- 172.21.15.60:0/2353159169 >> 172.21.15.60:0/2353159169 conn(0x7f817c06b380 msgr2=0x7f817c06b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.536+0000 7f81821ff700 1 -- 172.21.15.60:0/2353159169 shutdown_connections 2023-07-13T22:30:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.536+0000 7f81821ff700 1 -- 172.21.15.60:0/2353159169 wait complete. 2023-07-13T22:30:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.538+0000 7f81821ff700 1 Processor -- start 2023-07-13T22:30:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.538+0000 7f81821ff700 1 -- start start 2023-07-13T22:30:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f81821ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0b2010 0x7f817c151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f81821ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817c0b5a00 0x7f817c1f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f81821ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817c0d26d0 0x7f817c1f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f81821ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c15eb90 con 0x7f817c0b5a00 2023-07-13T22:30:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f81821ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c14bd00 con 0x7f817c0b2010 2023-07-13T22:30:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f81821ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c14bfe0 con 0x7f817c0d26d0 2023-07-13T22:30:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f817affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817c0b5a00 0x7f817c1f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f817affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817c0b5a00 0x7f817c1f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37334/0 (socket says 172.21.15.60:37334) 2023-07-13T22:30:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f817affd700 1 -- 172.21.15.60:0/1721478041 learned_addr learned my addr 172.21.15.60:0/1721478041 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f817b7fe700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0b2010 0x7f817c151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.539+0000 7f817bfff700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817c0d26d0 0x7f817c1f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f817affd700 1 -- 172.21.15.60:0/1721478041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817c0d26d0 msgr2=0x7f817c1f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f817affd700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817c0d26d0 0x7f817c1f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f817affd700 1 -- 172.21.15.60:0/1721478041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0b2010 msgr2=0x7f817c151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f817affd700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0b2010 0x7f817c151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f817affd700 1 -- 172.21.15.60:0/1721478041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81700094a0 con 0x7f817c0b5a00 2023-07-13T22:30:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f817b7fe700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0b2010 0x7f817c151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f817affd700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817c0b5a00 0x7f817c1f5310 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f817000a010 tx=0x7f8170004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f8163fff700 1 -- 172.21.15.60:0/1721478041 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170003e30 con 0x7f817c0b5a00 2023-07-13T22:30:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f8163fff700 1 -- 172.21.15.60:0/1721478041 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817001f070 con 0x7f817c0b5a00 2023-07-13T22:30:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c14c2c0 con 0x7f817c0b5a00 2023-07-13T22:30:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f8163fff700 1 -- 172.21.15.60:0/1721478041 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170011870 con 0x7f817c0b5a00 2023-07-13T22:30:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.540+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c152110 con 0x7f817c0b5a00 2023-07-13T22:30:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.542+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c14c780 con 0x7f817c0b5a00 2023-07-13T22:30:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.542+0000 7f8163fff700 1 -- 172.21.15.60:0/1721478041 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8170021030 con 0x7f817c0b5a00 2023-07-13T22:30:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.543+0000 7f8163fff700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f816806ace0 0x7f816806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.543+0000 7f8163fff700 1 -- 172.21.15.60:0/1721478041 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f817008a850 con 0x7f817c0b5a00 2023-07-13T22:30:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.544+0000 7f817b7fe700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f816806ace0 0x7f816806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.544+0000 7f817b7fe700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f816806ace0 0x7f816806d190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f816c001eb0 tx=0x7f816c001d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:16.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.548+0000 7f8163fff700 1 -- 172.21.15.60:0/1721478041 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8170055990 con 0x7f817c0b5a00 2023-07-13T22:30:16.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.754+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f817c0d2230 con 0x7f817c0b5a00 2023-07-13T22:30:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.756+0000 7f8163fff700 1 -- 172.21.15.60:0/1721478041 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18267 (secure 0 0 0) 0x7f8170058fb0 con 0x7f817c0b5a00 2023-07-13T22:30:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:16.734 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:29:50.779149+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:29:47.410626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-13T22:29:48.264611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:30:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f816806ace0 msgr2=0x7f816806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f816806ace0 0x7f816806d190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f816c001eb0 tx=0x7f816c001d00 comp rx=0 tx=0).stop 2023-07-13T22:30:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817c0b5a00 msgr2=0x7f817c1f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817c0b5a00 0x7f817c1f5310 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f817000a010 tx=0x7f8170004750 comp rx=0 tx=0).stop 2023-07-13T22:30:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 shutdown_connections 2023-07-13T22:30:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f816806ace0 0x7f816806d190 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817c0d26d0 0x7f817c1f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817c0b2010 0x7f817c151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 --2- 172.21.15.60:0/1721478041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817c0b5a00 0x7f817c1f5310 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 >> 172.21.15.60:0/1721478041 conn(0x7f817c06b380 msgr2=0x7f817c0b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:16.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.758+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 shutdown_connections 2023-07-13T22:30:16.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.759+0000 7f81821ff700 1 -- 172.21.15.60:0/1721478041 wait complete. 2023-07-13T22:30:16.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:29:47.410626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:29:48.264611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:30:16.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:30:16.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.983+0000 7fa7d8bff700 1 Processor -- start 2023-07-13T22:30:16.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.983+0000 7fa7d8bff700 1 -- start start 2023-07-13T22:30:16.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.984+0000 7fa7d8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40b8200 0x7fa7d40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.984+0000 7fa7d8bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7d40b1ac0 0x7fa7d40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.984+0000 7fa7d8bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d40d1870 con 0x7fa7d40cede0 2023-07-13T22:30:16.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.984+0000 7fa7d8bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d40cf370 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.984+0000 7fa7d8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40d12b0 con 0x7fa7d40b8200 2023-07-13T22:30:16.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.984+0000 7fa7cbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7d40b1ac0 0x7fa7d40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.984+0000 7fa7cbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7d40b1ac0 0x7fa7d40cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37344/0 (socket says 172.21.15.60:37344) 2023-07-13T22:30:16.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.984+0000 7fa7cbfff700 1 -- 172.21.15.60:0/1113422725 learned_addr learned my addr 172.21.15.60:0/1113422725 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:16.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.984+0000 7fa7d259c700 1 --2- 172.21.15.60:0/1113422725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40b8200 0x7fa7d40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.985+0000 7fa7cbfff700 1 -- 172.21.15.60:0/1113422725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40b8200 msgr2=0x7fa7d40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.985+0000 7fa7cbfff700 1 --2- 172.21.15.60:0/1113422725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40b8200 0x7fa7d40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.985+0000 7fa7cbfff700 1 -- 172.21.15.60:0/1113422725 >> v1:172.21.15.105:6789/0 conn(0x7fa7d40cede0 legacy=0x7fa7d415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.985+0000 7fa7cbfff700 1 -- 172.21.15.60:0/1113422725 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d40d1590 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.985+0000 7fa7cbfff700 1 --2- 172.21.15.60:0/1113422725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 0x7fa7d40cfb80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0098f0 tx=0x7fa7bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651cd30f7a57a0f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.985+0000 7fa7d1d9b700 1 -- 172.21.15.60:0/1113422725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00b8a0 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.985+0000 7fa7d1d9b700 1 -- 172.21.15.60:0/1113422725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc00ba00 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.985+0000 7fa7d1d9b700 1 -- 172.21.15.60:0/1113422725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00bd90 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.986+0000 7fa7d8bff700 1 -- 172.21.15.60:0/1113422725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 msgr2=0x7fa7d40cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.986+0000 7fa7d8bff700 1 --2- 172.21.15.60:0/1113422725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 0x7fa7d40cfb80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0098f0 tx=0x7fa7bc009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.986+0000 7fa7d8bff700 1 -- 172.21.15.60:0/1113422725 shutdown_connections 2023-07-13T22:30:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.986+0000 7fa7d8bff700 1 --2- 172.21.15.60:0/1113422725 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40b8200 0x7fa7d40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.986+0000 7fa7d8bff700 1 --2- 172.21.15.60:0/1113422725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 0x7fa7d40cfb80 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.986+0000 7fa7d8bff700 1 -- 172.21.15.60:0/1113422725 >> 172.21.15.60:0/1113422725 conn(0x7fa7d406b2f0 msgr2=0x7fa7d406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.986+0000 7fa7d8bff700 1 -- 172.21.15.60:0/1113422725 shutdown_connections 2023-07-13T22:30:16.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.986+0000 7fa7d8bff700 1 -- 172.21.15.60:0/1113422725 wait complete. 2023-07-13T22:30:16.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.988+0000 7fa7d8bff700 1 Processor -- start 2023-07-13T22:30:16.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.988+0000 7fa7d8bff700 1 -- start start 2023-07-13T22:30:16.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d8bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 0x7fa7d4156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d8bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d40b8200 0x7fa7d40cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40cede0 0x7fa7d40cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d8bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40c6890 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d8bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40c8420 con 0x7fa7d40b8200 2023-07-13T22:30:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40c8700 con 0x7fa7d40cede0 2023-07-13T22:30:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 0x7fa7d4156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 0x7fa7d4156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37352/0 (socket says 172.21.15.60:37352) 2023-07-13T22:30:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d259c700 1 -- 172.21.15.60:0/610792391 learned_addr learned my addr 172.21.15.60:0/610792391 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d259c700 1 -- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40cede0 msgr2=0x7fa7d40cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d2d9d700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40cede0 0x7fa7d40cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7cbfff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d40b8200 0x7fa7d40cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d259c700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40cede0 0x7fa7d40cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d259c700 1 -- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d40b8200 msgr2=0x7fa7d40cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d259c700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d40b8200 0x7fa7d40cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d259c700 1 -- 172.21.15.60:0/610792391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc009550 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7d2d9d700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40cede0 0x7fa7d40cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:16.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.989+0000 7fa7cbfff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d40b8200 0x7fa7d40cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:16.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.990+0000 7fa7d259c700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 0x7fa7d4156110 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400e810 tx=0x7fa7c400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:16.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.990+0000 7fa7caffd700 1 -- 172.21.15.60:0/610792391 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4013610 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.990+0000 7fa7caffd700 1 -- 172.21.15.60:0/610792391 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c4013770 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.990+0000 7fa7caffd700 1 -- 172.21.15.60:0/610792391 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4013b20 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.990+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d40c89e0 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.990+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d4156880 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.991+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d40c6b70 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.992+0000 7fa7caffd700 1 -- 172.21.15.60:0/610792391 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7c4013c80 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.993+0000 7fa7caffd700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa7b406ace0 0x7fa7b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.993+0000 7fa7caffd700 1 -- 172.21.15.60:0/610792391 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7c40891f0 con 0x7fa7d40b1ac0 2023-07-13T22:30:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.993+0000 7fa7cbfff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa7b406ace0 0x7fa7b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.994+0000 7fa7cbfff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa7b406ace0 0x7fa7b406d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa7d41f5590 tx=0x7fa7bc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:16.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:16.997+0000 7fa7caffd700 1 -- 172.21.15.60:0/610792391 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa7c4054210 con 0x7fa7d40b1ac0 2023-07-13T22:30:17.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.202+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa7d40ce150 con 0x7fa7d40b1ac0 2023-07-13T22:30:17.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.203+0000 7fa7caffd700 1 -- 172.21.15.60:0/610792391 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v112) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa7c4057830 con 0x7fa7d40b1ac0 2023-07-13T22:30:17.181 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:30:17.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa7b406ace0 msgr2=0x7fa7b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa7b406ace0 0x7fa7b406d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa7d41f5590 tx=0x7fa7bc008030 comp rx=0 tx=0).stop 2023-07-13T22:30:17.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 msgr2=0x7fa7d4156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 0x7fa7d4156110 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400e810 tx=0x7fa7c400ebd0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 shutdown_connections 2023-07-13T22:30:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa7b406ace0 0x7fa7b406d190 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d40b1ac0 0x7fa7d4156110 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7d40cede0 0x7fa7d40cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 --2- 172.21.15.60:0/610792391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d40b8200 0x7fa7d40cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 >> 172.21.15.60:0/610792391 conn(0x7fa7d406b2f0 msgr2=0x7fa7d40d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 shutdown_connections 2023-07-13T22:30:17.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.205+0000 7fa7d8bff700 1 -- 172.21.15.60:0/610792391 wait complete. 2023-07-13T22:30:17.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:30:17.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.429+0000 7f87b8bff700 1 Processor -- start 2023-07-13T22:30:17.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.429+0000 7f87b8bff700 1 -- start start 2023-07-13T22:30:17.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.431+0000 7f87b8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40cfd50 0x7f87b40d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.431+0000 7f87b8bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40d1870 con 0x7f87b40b1ac0 2023-07-13T22:30:17.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.431+0000 7f87b8bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40cf370 con 0x7f87b40b8200 2023-07-13T22:30:17.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.431+0000 7f87b8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40d12b0 con 0x7f87b40cfd50 2023-07-13T22:30:17.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.431+0000 7f87b2d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f87b40b1ac0 0x7f87b40d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49516/0 (socket says 172.21.15.60:49516) 2023-07-13T22:30:17.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.431+0000 7f87b2d9d700 1 -- 172.21.15.60:0/1330701995 learned_addr learned my addr 172.21.15.60:0/1330701995 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:17.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.431+0000 7f87abfff700 1 --2- 172.21.15.60:0/1330701995 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40cfd50 0x7f87b40d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.431+0000 7f87b1d9b700 1 -- 172.21.15.60:0/1330701995 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717337910 0 0) 0x7f87b40d1870 con 0x7f87b40b1ac0 2023-07-13T22:30:17.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.431+0000 7f87b1d9b700 1 -- 172.21.15.60:0/1330701995 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a00035b0 con 0x7f87b40b1ac0 2023-07-13T22:30:17.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.432+0000 7f87b1d9b700 1 -- 172.21.15.60:0/1330701995 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387638897 0 0) 0x7f87b40cf370 con 0x7f87b40b8200 2023-07-13T22:30:17.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.432+0000 7f87b1d9b700 1 -- 172.21.15.60:0/1330701995 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b40d1870 con 0x7f87b40b8200 2023-07-13T22:30:17.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.432+0000 7f87abfff700 1 -- 172.21.15.60:0/1330701995 >> v1:172.21.15.105:6789/0 conn(0x7f87b40b8200 legacy=0x7f87b40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.432+0000 7f87abfff700 1 -- 172.21.15.60:0/1330701995 >> v1:172.21.15.60:6789/0 conn(0x7f87b40b1ac0 legacy=0x7f87b40d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.432+0000 7f87abfff700 1 -- 172.21.15.60:0/1330701995 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40d1590 con 0x7f87b40cfd50 2023-07-13T22:30:17.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.432+0000 7f87abfff700 1 --2- 172.21.15.60:0/1330701995 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40cfd50 0x7f87b40d23c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f879c0098a0 tx=0x7f879c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db794f4a84a77ca6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:17.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.432+0000 7f87b1d9b700 1 -- 172.21.15.60:0/1330701995 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c011830 con 0x7f87b40cfd50 2023-07-13T22:30:17.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.432+0000 7f87b1d9b700 1 -- 172.21.15.60:0/1330701995 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c011990 con 0x7f87b40cfd50 2023-07-13T22:30:17.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.432+0000 7f87b1d9b700 1 -- 172.21.15.60:0/1330701995 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c011d20 con 0x7f87b40cfd50 2023-07-13T22:30:17.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.433+0000 7f87b8bff700 1 -- 172.21.15.60:0/1330701995 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40cfd50 msgr2=0x7f87b40d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.433+0000 7f87b8bff700 1 --2- 172.21.15.60:0/1330701995 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40cfd50 0x7f87b40d23c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f879c0098a0 tx=0x7f879c009bb0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.433+0000 7f87b8bff700 1 -- 172.21.15.60:0/1330701995 shutdown_connections 2023-07-13T22:30:17.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.433+0000 7f87b8bff700 1 --2- 172.21.15.60:0/1330701995 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40cfd50 0x7f87b40d23c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.433+0000 7f87b8bff700 1 -- 172.21.15.60:0/1330701995 >> 172.21.15.60:0/1330701995 conn(0x7f87b406b2f0 msgr2=0x7f87b406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:17.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.433+0000 7f87b8bff700 1 -- 172.21.15.60:0/1330701995 shutdown_connections 2023-07-13T22:30:17.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.433+0000 7f87b8bff700 1 -- 172.21.15.60:0/1330701995 wait complete. 2023-07-13T22:30:17.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.435+0000 7f87b8bff700 1 Processor -- start 2023-07-13T22:30:17.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.435+0000 7f87b8bff700 1 -- start start 2023-07-13T22:30:17.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b8bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b40b1ac0 0x7f87b4153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b8bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40b8200 0x7f87b415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b8bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87b40cfd50 0x7f87b4152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b8bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40ca350 con 0x7f87b40cfd50 2023-07-13T22:30:17.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b8bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b415f470 con 0x7f87b40b1ac0 2023-07-13T22:30:17.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b8bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4152f80 con 0x7f87b40b8200 2023-07-13T22:30:17.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b2d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87b40cfd50 0x7f87b4152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b2d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87b40cfd50 0x7f87b4152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37360/0 (socket says 172.21.15.60:37360) 2023-07-13T22:30:17.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b2d9d700 1 -- 172.21.15.60:0/2360243422 learned_addr learned my addr 172.21.15.60:0/2360243422 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87abfff700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40b8200 0x7f87b415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b2d9d700 1 -- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40b8200 msgr2=0x7f87b415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b259c700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b40b1ac0 0x7f87b4153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b2d9d700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40b8200 0x7f87b415ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b2d9d700 1 -- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b40b1ac0 msgr2=0x7f87b4153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b2d9d700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b40b1ac0 0x7f87b4153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b2d9d700 1 -- 172.21.15.60:0/2360243422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0094a0 con 0x7f87b40cfd50 2023-07-13T22:30:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87abfff700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40b8200 0x7f87b415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:17.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.436+0000 7f87b259c700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b40b1ac0 0x7f87b4153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:17.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.437+0000 7f87b2d9d700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87b40cfd50 0x7f87b4152a40 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00cbc0 tx=0x7f87ac00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:17.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.437+0000 7f87aaffd700 1 -- 172.21.15.60:0/2360243422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00aaf0 con 0x7f87b40cfd50 2023-07-13T22:30:17.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.437+0000 7f87aaffd700 1 -- 172.21.15.60:0/2360243422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac00ac50 con 0x7f87b40cfd50 2023-07-13T22:30:17.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.437+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b415f6d0 con 0x7f87b40cfd50 2023-07-13T22:30:17.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.437+0000 7f87aaffd700 1 -- 172.21.15.60:0/2360243422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac012400 con 0x7f87b40cfd50 2023-07-13T22:30:17.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.437+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b40c9d00 con 0x7f87b40cfd50 2023-07-13T22:30:17.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.438+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b404e210 con 0x7f87b40cfd50 2023-07-13T22:30:17.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.439+0000 7f87aaffd700 1 -- 172.21.15.60:0/2360243422 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87ac01b3b0 con 0x7f87b40cfd50 2023-07-13T22:30:17.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.440+0000 7f87aaffd700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f879406ace0 0x7f879406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.440+0000 7f87aaffd700 1 -- 172.21.15.60:0/2360243422 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87ac0891c0 con 0x7f87b40cfd50 2023-07-13T22:30:17.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.440+0000 7f87b259c700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f879406ace0 0x7f879406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.441+0000 7f87b259c700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f879406ace0 0x7f879406d190 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f87a40011e0 tx=0x7f87a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:17.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.444+0000 7f87aaffd700 1 -- 172.21.15.60:0/2360243422 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f87ac054290 con 0x7f87b40cfd50 2023-07-13T22:30:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.649+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87b40ca490 con 0x7f87b40cfd50 2023-07-13T22:30:17.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.650+0000 7f87aaffd700 1 -- 172.21.15.60:0/2360243422 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87ac0578b0 con 0x7f87b40cfd50 2023-07-13T22:30:17.628 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:30:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f879406ace0 msgr2=0x7f879406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f879406ace0 0x7f879406d190 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f87a40011e0 tx=0x7f87a4009040 comp rx=0 tx=0).stop 2023-07-13T22:30:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87b40cfd50 msgr2=0x7f87b4152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87b40cfd50 0x7f87b4152a40 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00cbc0 tx=0x7f87ac00cf80 comp rx=0 tx=0).stop 2023-07-13T22:30:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 shutdown_connections 2023-07-13T22:30:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f879406ace0 0x7f879406d190 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87b40cfd50 0x7f87b4152a40 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b40b1ac0 0x7f87b4153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 --2- 172.21.15.60:0/2360243422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87b40b8200 0x7f87b415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 >> 172.21.15.60:0/2360243422 conn(0x7f87b406b2f0 msgr2=0x7f87b40b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:17.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 shutdown_connections 2023-07-13T22:30:17.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.652+0000 7f87b8bff700 1 -- 172.21.15.60:0/2360243422 wait complete. 2023-07-13T22:30:17.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:30:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.878+0000 7f2167fff700 1 Processor -- start 2023-07-13T22:30:17.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.879+0000 7f2167fff700 1 -- start start 2023-07-13T22:30:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.880+0000 7f2167fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2160159260 0x7f21600ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.880+0000 7f2167fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21600cc410 0x7f21600cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.880+0000 7f2167fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21600cf640 con 0x7f21600703e0 2023-07-13T22:30:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.880+0000 7f2167fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21600ccf80 con 0x7f21600cc410 2023-07-13T22:30:17.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.880+0000 7f2167fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21600c9b40 con 0x7f2160159260 2023-07-13T22:30:17.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.880+0000 7f216559a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21600cc410 0x7f21600cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.880+0000 7f216559a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21600cc410 0x7f21600cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37374/0 (socket says 172.21.15.60:37374) 2023-07-13T22:30:17.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.880+0000 7f216559a700 1 -- 172.21.15.60:0/1580027766 learned_addr learned my addr 172.21.15.60:0/1580027766 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.880+0000 7f2165d9b700 1 --2- 172.21.15.60:0/1580027766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2160159260 0x7f21600ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.881+0000 7f216559a700 1 -- 172.21.15.60:0/1580027766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2160159260 msgr2=0x7f21600ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.881+0000 7f216559a700 1 --2- 172.21.15.60:0/1580027766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2160159260 0x7f21600ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.881+0000 7f216559a700 1 -- 172.21.15.60:0/1580027766 >> v1:172.21.15.105:6789/0 conn(0x7f21600703e0 legacy=0x7f216006c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.881+0000 7f216559a700 1 -- 172.21.15.60:0/1580027766 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600c9e20 con 0x7f21600cc410 2023-07-13T22:30:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.881+0000 7f2165d9b700 1 --2- 172.21.15.60:0/1580027766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2160159260 0x7f21600ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.881+0000 7f216559a700 1 --2- 172.21.15.60:0/1580027766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21600cc410 0x7f21600cc880 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f21500098f0 tx=0x7f2150009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac4ac3b20fcfb48 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.881+0000 7f2164d99700 1 -- 172.21.15.60:0/1580027766 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215000f8a0 con 0x7f21600cc410 2023-07-13T22:30:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.881+0000 7f2164d99700 1 -- 172.21.15.60:0/1580027766 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215000fa00 con 0x7f21600cc410 2023-07-13T22:30:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.881+0000 7f2164d99700 1 -- 172.21.15.60:0/1580027766 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215000fd90 con 0x7f21600cc410 2023-07-13T22:30:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.882+0000 7f2167fff700 1 -- 172.21.15.60:0/1580027766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21600cc410 msgr2=0x7f21600cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.882+0000 7f2167fff700 1 --2- 172.21.15.60:0/1580027766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21600cc410 0x7f21600cc880 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f21500098f0 tx=0x7f2150009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.882+0000 7f2167fff700 1 -- 172.21.15.60:0/1580027766 shutdown_connections 2023-07-13T22:30:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.882+0000 7f2167fff700 1 --2- 172.21.15.60:0/1580027766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2160159260 0x7f21600ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.882+0000 7f2167fff700 1 --2- 172.21.15.60:0/1580027766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21600cc410 0x7f21600cc880 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.882+0000 7f2167fff700 1 -- 172.21.15.60:0/1580027766 >> 172.21.15.60:0/1580027766 conn(0x7f21600cbf50 msgr2=0x7f21600ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.882+0000 7f2167fff700 1 -- 172.21.15.60:0/1580027766 shutdown_connections 2023-07-13T22:30:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.882+0000 7f2167fff700 1 -- 172.21.15.60:0/1580027766 wait complete. 2023-07-13T22:30:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.884+0000 7f2167fff700 1 Processor -- start 2023-07-13T22:30:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.884+0000 7f2167fff700 1 -- start start 2023-07-13T22:30:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f2167fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21600703e0 0x7f21601546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f2167fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21600cc410 0x7f21600c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f2167fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2160159260 0x7f21600c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f2167fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601e3b10 con 0x7f2160159260 2023-07-13T22:30:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f2167fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601f2900 con 0x7f21600cc410 2023-07-13T22:30:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f2167fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21601f2be0 con 0x7f21600703e0 2023-07-13T22:30:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f216659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2160159260 0x7f21600c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f216659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2160159260 0x7f21600c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37382/0 (socket says 172.21.15.60:37382) 2023-07-13T22:30:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f216659c700 1 -- 172.21.15.60:0/3590370686 learned_addr learned my addr 172.21.15.60:0/3590370686 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f2165d9b700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21600703e0 0x7f21601546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.885+0000 7f216559a700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21600cc410 0x7f21600c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f216659c700 1 -- 172.21.15.60:0/3590370686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21600703e0 msgr2=0x7f21601546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f216659c700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21600703e0 0x7f21601546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f216659c700 1 -- 172.21.15.60:0/3590370686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21600cc410 msgr2=0x7f21600c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f216659c700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21600cc410 0x7f21600c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f216659c700 1 -- 172.21.15.60:0/3590370686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2150009550 con 0x7f2160159260 2023-07-13T22:30:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f216559a700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21600cc410 0x7f21600c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f216659c700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2160159260 0x7f21600c8510 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f215800cc50 tx=0x7f215800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f21567fc700 1 -- 172.21.15.60:0/3590370686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158003c00 con 0x7f2160159260 2023-07-13T22:30:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f21567fc700 1 -- 172.21.15.60:0/3590370686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2158003d60 con 0x7f2160159260 2023-07-13T22:30:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f21567fc700 1 -- 172.21.15.60:0/3590370686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215800ab60 con 0x7f2160159260 2023-07-13T22:30:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21601f2ec0 con 0x7f2160159260 2023-07-13T22:30:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.886+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21601f25b0 con 0x7f2160159260 2023-07-13T22:30:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.887+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21601f33e0 con 0x7f2160159260 2023-07-13T22:30:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.888+0000 7f21567fc700 1 -- 172.21.15.60:0/3590370686 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f215800acc0 con 0x7f2160159260 2023-07-13T22:30:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.889+0000 7f21567fc700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.889+0000 7f21567fc700 1 -- 172.21.15.60:0/3590370686 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f21580895e0 con 0x7f2160159260 2023-07-13T22:30:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.890+0000 7f2165d9b700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.890+0000 7f2165d9b700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f21600cdd10 tx=0x7f215c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:17.894+0000 7f21567fc700 1 -- 172.21.15.60:0/3590370686 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f21580545f0 con 0x7f2160159260 2023-07-13T22:30:18.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.101+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f216004e210 con 0x7f2160159260 2023-07-13T22:30:18.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.101+0000 7f21567fc700 1 -- 172.21.15.60:0/3590370686 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2158057c10 con 0x7f2160159260 2023-07-13T22:30:18.080 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:30:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 msgr2=0x7f214406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f21600cdd10 tx=0x7f215c008040 comp rx=0 tx=0).stop 2023-07-13T22:30:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2160159260 msgr2=0x7f21600c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2160159260 0x7f21600c8510 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f215800cc50 tx=0x7f215800cf60 comp rx=0 tx=0).stop 2023-07-13T22:30:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 shutdown_connections 2023-07-13T22:30:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2160159260 0x7f21600c8510 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21600703e0 0x7f21601546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 --2- 172.21.15.60:0/3590370686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21600cc410 0x7f21600c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 >> 172.21.15.60:0/3590370686 conn(0x7f21600cbf50 msgr2=0x7f21600ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 shutdown_connections 2023-07-13T22:30:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.104+0000 7f2167fff700 1 -- 172.21.15.60:0/3590370686 wait complete. 2023-07-13T22:30:18.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:30:18.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.329+0000 7fa8605ff700 1 Processor -- start 2023-07-13T22:30:18.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.330+0000 7fa8605ff700 1 -- start start 2023-07-13T22:30:18.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.331+0000 7fa8605ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa8580b8200 0x7fa8580cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.331+0000 7fa8605ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8580d1870 con 0x7fa8580cede0 2023-07-13T22:30:18.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.331+0000 7fa8605ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa858160d80 con 0x7fa8580b1ac0 2023-07-13T22:30:18.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.331+0000 7fa8605ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8580cf370 con 0x7fa8580b8200 2023-07-13T22:30:18.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.331+0000 7fa85e39b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa8580b8200 0x7fa8580cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.331+0000 7fa85e39b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa8580b8200 0x7fa8580cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37388/0 (socket says 172.21.15.60:37388) 2023-07-13T22:30:18.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.331+0000 7fa85e39b700 1 -- 172.21.15.60:0/4157355489 learned_addr learned my addr 172.21.15.60:0/4157355489 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:18.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.331+0000 7fa85eb9c700 1 --1- 172.21.15.60:0/4157355489 >> v1:172.21.15.60:6789/0 conn(0x7fa8580cede0 0x7fa858160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49520/0 (socket says 172.21.15.60:49520) 2023-07-13T22:30:18.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.332+0000 7fa85e39b700 1 -- 172.21.15.60:0/4157355489 >> v1:172.21.15.105:6789/0 conn(0x7fa8580b1ac0 legacy=0x7fa85815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:18.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.332+0000 7fa85e39b700 1 -- 172.21.15.60:0/4157355489 >> v1:172.21.15.60:6789/0 conn(0x7fa8580cede0 legacy=0x7fa858160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.332+0000 7fa85e39b700 1 -- 172.21.15.60:0/4157355489 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580d12b0 con 0x7fa8580b8200 2023-07-13T22:30:18.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.332+0000 7fa85e39b700 1 --2- 172.21.15.60:0/4157355489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b8200 0x7fa8580cf640 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098f0 tx=0x7fa854009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856f497184c2a3b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:18.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.332+0000 7fa85d399700 1 -- 172.21.15.60:0/4157355489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854004230 con 0x7fa8580b8200 2023-07-13T22:30:18.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.332+0000 7fa85d399700 1 -- 172.21.15.60:0/4157355489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854004390 con 0x7fa8580b8200 2023-07-13T22:30:18.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.332+0000 7fa85d399700 1 -- 172.21.15.60:0/4157355489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540036a0 con 0x7fa8580b8200 2023-07-13T22:30:18.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.333+0000 7fa8605ff700 1 -- 172.21.15.60:0/4157355489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b8200 msgr2=0x7fa8580cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.333+0000 7fa8605ff700 1 --2- 172.21.15.60:0/4157355489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b8200 0x7fa8580cf640 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098f0 tx=0x7fa854009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:18.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.333+0000 7fa8605ff700 1 -- 172.21.15.60:0/4157355489 shutdown_connections 2023-07-13T22:30:18.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.333+0000 7fa8605ff700 1 --2- 172.21.15.60:0/4157355489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b8200 0x7fa8580cf640 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.333+0000 7fa8605ff700 1 -- 172.21.15.60:0/4157355489 >> 172.21.15.60:0/4157355489 conn(0x7fa85806b2f0 msgr2=0x7fa85806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:18.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.333+0000 7fa8605ff700 1 -- 172.21.15.60:0/4157355489 shutdown_connections 2023-07-13T22:30:18.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.333+0000 7fa8605ff700 1 -- 172.21.15.60:0/4157355489 wait complete. 2023-07-13T22:30:18.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.335+0000 7fa8605ff700 1 Processor -- start 2023-07-13T22:30:18.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.335+0000 7fa8605ff700 1 -- start start 2023-07-13T22:30:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa8605ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b1ac0 0x7fa8581492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa8605ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8580b8200 0x7fa85815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa8605ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8580cede0 0x7fa85815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa8605ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580cade0 con 0x7fa8580b1ac0 2023-07-13T22:30:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa8605ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa858149800 con 0x7fa8580b8200 2023-07-13T22:30:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa8605ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581f5a40 con 0x7fa8580cede0 2023-07-13T22:30:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85e39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b1ac0 0x7fa8581492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85e39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b1ac0 0x7fa8581492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37390/0 (socket says 172.21.15.60:37390) 2023-07-13T22:30:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85e39b700 1 -- 172.21.15.60:0/3603200647 learned_addr learned my addr 172.21.15.60:0/3603200647 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:18.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85db9a700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8580b8200 0x7fa85815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85eb9c700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8580cede0 0x7fa85815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85e39b700 1 -- 172.21.15.60:0/3603200647 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8580cede0 msgr2=0x7fa85815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85e39b700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8580cede0 0x7fa85815eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85e39b700 1 -- 172.21.15.60:0/3603200647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8580b8200 msgr2=0x7fa85815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85e39b700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8580b8200 0x7fa85815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85e39b700 1 -- 172.21.15.60:0/3603200647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa854009550 con 0x7fa8580b1ac0 2023-07-13T22:30:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.337+0000 7fa85db9a700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8580b8200 0x7fa85815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.338+0000 7fa85e39b700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b1ac0 0x7fa8581492c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa85400b0e0 tx=0x7fa854004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:18.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.338+0000 7fa84effd700 1 -- 172.21.15.60:0/3603200647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854003d80 con 0x7fa8580b1ac0 2023-07-13T22:30:18.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.338+0000 7fa84effd700 1 -- 172.21.15.60:0/3603200647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85401f070 con 0x7fa8580b1ac0 2023-07-13T22:30:18.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.338+0000 7fa84effd700 1 -- 172.21.15.60:0/3603200647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400b8a0 con 0x7fa8580b1ac0 2023-07-13T22:30:18.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.338+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8581f5d20 con 0x7fa8580b1ac0 2023-07-13T22:30:18.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.338+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8581f62e0 con 0x7fa8580b1ac0 2023-07-13T22:30:18.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.339+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85804e210 con 0x7fa8580b1ac0 2023-07-13T22:30:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.340+0000 7fa84effd700 1 -- 172.21.15.60:0/3603200647 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa854021030 con 0x7fa8580b1ac0 2023-07-13T22:30:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.342+0000 7fa84effd700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa83c06ace0 0x7fa83c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.342+0000 7fa84effd700 1 -- 172.21.15.60:0/3603200647 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa85408ac30 con 0x7fa8580b1ac0 2023-07-13T22:30:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.342+0000 7fa85db9a700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa83c06ace0 0x7fa83c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.342+0000 7fa85db9a700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa83c06ace0 0x7fa83c06d190 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa8581f5420 tx=0x7fa848009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:18.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.346+0000 7fa84effd700 1 -- 172.21.15.60:0/3603200647 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa854055df0 con 0x7fa8580b1ac0 2023-07-13T22:30:18.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.549+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa85804e210 con 0x7fa8580b1ac0 2023-07-13T22:30:18.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.550+0000 7fa84effd700 1 -- 172.21.15.60:0/3603200647 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa854059410 con 0x7fa8580b1ac0 2023-07-13T22:30:18.528 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:30:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa83c06ace0 msgr2=0x7fa83c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa83c06ace0 0x7fa83c06d190 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa8581f5420 tx=0x7fa848009040 comp rx=0 tx=0).stop 2023-07-13T22:30:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b1ac0 msgr2=0x7fa8581492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b1ac0 0x7fa8581492c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa85400b0e0 tx=0x7fa854004750 comp rx=0 tx=0).stop 2023-07-13T22:30:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 shutdown_connections 2023-07-13T22:30:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa83c06ace0 0x7fa83c06d190 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8580b1ac0 0x7fa8581492c0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8580cede0 0x7fa85815eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 --2- 172.21.15.60:0/3603200647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8580b8200 0x7fa85815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 >> 172.21.15.60:0/3603200647 conn(0x7fa85806b2f0 msgr2=0x7fa8580b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.552+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 shutdown_connections 2023-07-13T22:30:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.553+0000 7fa8605ff700 1 -- 172.21.15.60:0/3603200647 wait complete. 2023-07-13T22:30:18.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:30:18.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.777+0000 7f83801ff700 1 Processor -- start 2023-07-13T22:30:18.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.777+0000 7f83801ff700 1 -- start start 2023-07-13T22:30:18.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.778+0000 7f83801ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83780b8570 0x7f83780ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.778+0000 7f83801ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.778+0000 7f83801ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83780d09d0 con 0x7f83780cf3f0 2023-07-13T22:30:18.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.778+0000 7f83801ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83780d3b30 con 0x7f83780b8570 2023-07-13T22:30:18.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.778+0000 7f83801ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83780cda10 con 0x7f83780d35c0 2023-07-13T22:30:18.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.779+0000 7f837d79a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.779+0000 7f837d79a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:58352/0 (socket says 172.21.15.60:58352) 2023-07-13T22:30:18.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.779+0000 7f837d79a700 1 -- 172.21.15.60:0/3547371103 learned_addr learned my addr 172.21.15.60:0/3547371103 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:18.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.779+0000 7f837df9b700 1 --2- 172.21.15.60:0/3547371103 >> v2:172.21.15.105:3300/0 conn(0x7f83780b8570 0x7f83780ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.779+0000 7f837d79a700 1 -- 172.21.15.60:0/3547371103 >> v2:172.21.15.105:3300/0 conn(0x7f83780b8570 msgr2=0x7f83780ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:18.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.779+0000 7f837d79a700 1 --2- 172.21.15.60:0/3547371103 >> v2:172.21.15.105:3300/0 conn(0x7f83780b8570 0x7f83780ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.779+0000 7f837d79a700 1 -- 172.21.15.60:0/3547371103 >> v1:172.21.15.105:6789/0 conn(0x7f83780cf3f0 legacy=0x7f837815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.779+0000 7f837d79a700 1 -- 172.21.15.60:0/3547371103 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780cd450 con 0x7f83780d35c0 2023-07-13T22:30:18.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f837d79a700 1 --2- 172.21.15.60:0/3547371103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780d0eb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83680098f0 tx=0x7f8368009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fc08521b1c9513d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:18.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f837cf99700 1 -- 172.21.15.60:0/3547371103 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368011830 con 0x7f83780d35c0 2023-07-13T22:30:18.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f837cf99700 1 -- 172.21.15.60:0/3547371103 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8368011990 con 0x7f83780d35c0 2023-07-13T22:30:18.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f837cf99700 1 -- 172.21.15.60:0/3547371103 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368011d20 con 0x7f83780d35c0 2023-07-13T22:30:18.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f83801ff700 1 -- 172.21.15.60:0/3547371103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 msgr2=0x7f83780d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f83801ff700 1 --2- 172.21.15.60:0/3547371103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780d0eb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83680098f0 tx=0x7f8368009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:18.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f83801ff700 1 -- 172.21.15.60:0/3547371103 shutdown_connections 2023-07-13T22:30:18.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f83801ff700 1 --2- 172.21.15.60:0/3547371103 >> v2:172.21.15.105:3300/0 conn(0x7f83780b8570 0x7f83780ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f83801ff700 1 --2- 172.21.15.60:0/3547371103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780d0eb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.780+0000 7f83801ff700 1 -- 172.21.15.60:0/3547371103 >> 172.21.15.60:0/3547371103 conn(0x7f8378071600 msgr2=0x7f83780719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:18.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.781+0000 7f83801ff700 1 -- 172.21.15.60:0/3547371103 shutdown_connections 2023-07-13T22:30:18.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.781+0000 7f83801ff700 1 -- 172.21.15.60:0/3547371103 wait complete. 2023-07-13T22:30:18.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.782+0000 7f83801ff700 1 Processor -- start 2023-07-13T22:30:18.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.782+0000 7f83801ff700 1 -- start start 2023-07-13T22:30:18.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.783+0000 7f83801ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83780b8570 0x7f8378146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.783+0000 7f83801ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83780cf3f0 0x7f83780cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.783+0000 7f83801ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.783+0000 7f83801ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83781f1e90 con 0x7f83780b8570 2023-07-13T22:30:18.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.783+0000 7f83801ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83780ca740 con 0x7f83780cf3f0 2023-07-13T22:30:18.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.783+0000 7f83801ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83780cb5a0 con 0x7f83780d35c0 2023-07-13T22:30:18.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837df9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83780b8570 0x7f8378146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837df9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83780b8570 0x7f8378146130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37394/0 (socket says 172.21.15.60:37394) 2023-07-13T22:30:18.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837df9b700 1 -- 172.21.15.60:0/3686058535 learned_addr learned my addr 172.21.15.60:0/3686058535 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:18.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837d79a700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83780cf3f0 0x7f83780cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837e79c700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837df9b700 1 -- 172.21.15.60:0/3686058535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 msgr2=0x7f83780c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837df9b700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837df9b700 1 -- 172.21.15.60:0/3686058535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83780cf3f0 msgr2=0x7f83780cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837df9b700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83780cf3f0 0x7f83780cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837df9b700 1 -- 172.21.15.60:0/3686058535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8368009550 con 0x7f83780b8570 2023-07-13T22:30:18.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837e79c700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:18.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.784+0000 7f837df9b700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83780b8570 0x7f8378146130 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f837400e990 tx=0x7f837400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:18.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.785+0000 7f836e7fc700 1 -- 172.21.15.60:0/3686058535 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740096f0 con 0x7f83780b8570 2023-07-13T22:30:18.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.785+0000 7f836e7fc700 1 -- 172.21.15.60:0/3686058535 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837400c850 con 0x7f83780b8570 2023-07-13T22:30:18.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.785+0000 7f836e7fc700 1 -- 172.21.15.60:0/3686058535 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400eee0 con 0x7f83780b8570 2023-07-13T22:30:18.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.785+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83780cba30 con 0x7f83780b8570 2023-07-13T22:30:18.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.785+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83780cac70 con 0x7f83780b8570 2023-07-13T22:30:18.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.786+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837804e280 con 0x7f83780b8570 2023-07-13T22:30:18.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.787+0000 7f836e7fc700 1 -- 172.21.15.60:0/3686058535 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f837400cbc0 con 0x7f83780b8570 2023-07-13T22:30:18.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.788+0000 7f836e7fc700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f835c06ace0 0x7f835c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:18.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.788+0000 7f836e7fc700 1 -- 172.21.15.60:0/3686058535 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8374089980 con 0x7f83780b8570 2023-07-13T22:30:18.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.788+0000 7f837d79a700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f835c06ace0 0x7f835c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:18.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.789+0000 7f837d79a700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f835c06ace0 0x7f835c06d190 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8368000c00 tx=0x7f8368008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:18.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:18.793+0000 7f836e7fc700 1 -- 172.21.15.60:0/3686058535 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8374054990 con 0x7f83780b8570 2023-07-13T22:30:18.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.015+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83780cb880 con 0x7f83780b8570 2023-07-13T22:30:18.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.015+0000 7f836e7fc700 1 -- 172.21.15.60:0/3686058535 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8374017080 con 0x7f83780b8570 2023-07-13T22:30:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.017+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f835c06ace0 msgr2=0x7f835c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.017+0000 7f83801ff700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f835c06ace0 0x7f835c06d190 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8368000c00 tx=0x7f8368008030 comp rx=0 tx=0).stop 2023-07-13T22:30:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.017+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83780b8570 msgr2=0x7f8378146130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.017+0000 7f83801ff700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83780b8570 0x7f8378146130 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f837400e990 tx=0x7f837400eca0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.018+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 shutdown_connections 2023-07-13T22:30:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.018+0000 7f83801ff700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f835c06ace0 0x7f835c06d190 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.018+0000 7f83801ff700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83780b8570 0x7f8378146130 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.018+0000 7f83801ff700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83780d35c0 0x7f83780c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.018+0000 7f83801ff700 1 --2- 172.21.15.60:0/3686058535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83780cf3f0 0x7f83780cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.018+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 >> 172.21.15.60:0/3686058535 conn(0x7f8378071600 msgr2=0x7f83780cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.018+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 shutdown_connections 2023-07-13T22:30:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.018+0000 7f83801ff700 1 -- 172.21.15.60:0/3686058535 wait complete. 2023-07-13T22:30:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:30:19.014 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:30:19.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:19.014 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:19.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[126311]: fuse finished with error 0 and tester_r 0 2023-07-13T22:30:19.090 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:30:19.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:19.091 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:30:19.213 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:30:19.213 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:19.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:30:19.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:30:19.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:19.262 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:30:19.347 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-13T22:30:19.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:30:19.348 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:30:19.349 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:19.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:30:19.396 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:30:19.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:19.397 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:30:19.480 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-13T22:30:19.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:30:19.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:19.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.707+0000 7f70589ff700 1 Processor -- start 2023-07-13T22:30:19.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.707+0000 7f70589ff700 1 -- start start 2023-07-13T22:30:19.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.708+0000 7f70589ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70540b8400 0x7f70540cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:19.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.708+0000 7f70589ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70540d3750 0x7f70540cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:19.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.708+0000 7f70589ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540cfad0 0x7f705415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:19.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.708+0000 7f70589ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70540cff50 con 0x7f70540d3750 2023-07-13T22:30:19.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.708+0000 7f70589ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70540d0ac0 con 0x7f70540b8400 2023-07-13T22:30:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.708+0000 7f70589ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70540b87d0 con 0x7f70540cfad0 2023-07-13T22:30:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f704bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70540d3750 0x7f70540cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f704bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70540d3750 0x7f70540cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37404/0 (socket says 172.21.15.60:37404) 2023-07-13T22:30:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f704bfff700 1 -- 172.21.15.60:0/2177915536 learned_addr learned my addr 172.21.15.60:0/2177915536 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f705259c700 1 --2- 172.21.15.60:0/2177915536 >> v2:172.21.15.105:3300/0 conn(0x7f70540b8400 0x7f70540cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f7052d9d700 1 --2- 172.21.15.60:0/2177915536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540cfad0 0x7f705415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:19.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f704bfff700 1 -- 172.21.15.60:0/2177915536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540cfad0 msgr2=0x7f705415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:19.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f704bfff700 1 --2- 172.21.15.60:0/2177915536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540cfad0 0x7f705415fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f704bfff700 1 -- 172.21.15.60:0/2177915536 >> v2:172.21.15.105:3300/0 conn(0x7f70540b8400 msgr2=0x7f70540cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:19.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f704bfff700 1 --2- 172.21.15.60:0/2177915536 >> v2:172.21.15.105:3300/0 conn(0x7f70540b8400 0x7f70540cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f704bfff700 1 -- 172.21.15.60:0/2177915536 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540b4eb0 con 0x7f70540d3750 2023-07-13T22:30:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.709+0000 7f704bfff700 1 --2- 172.21.15.60:0/2177915536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540d3750 0x7f70540cf590 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f70540cf560 tx=0x7f703c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ec25676f693914 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f7051d9b700 1 -- 172.21.15.60:0/2177915536 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c011830 con 0x7f70540d3750 2023-07-13T22:30:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f7051d9b700 1 -- 172.21.15.60:0/2177915536 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c011990 con 0x7f70540d3750 2023-07-13T22:30:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f7051d9b700 1 -- 172.21.15.60:0/2177915536 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c011d20 con 0x7f70540d3750 2023-07-13T22:30:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f70589ff700 1 -- 172.21.15.60:0/2177915536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540d3750 msgr2=0x7f70540cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f70589ff700 1 --2- 172.21.15.60:0/2177915536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540d3750 0x7f70540cf590 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f70540cf560 tx=0x7f703c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f70589ff700 1 -- 172.21.15.60:0/2177915536 shutdown_connections 2023-07-13T22:30:19.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f70589ff700 1 --2- 172.21.15.60:0/2177915536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540cfad0 0x7f705415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f70589ff700 1 --2- 172.21.15.60:0/2177915536 >> v2:172.21.15.105:3300/0 conn(0x7f70540b8400 0x7f70540cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f70589ff700 1 --2- 172.21.15.60:0/2177915536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540d3750 0x7f70540cf590 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f70589ff700 1 -- 172.21.15.60:0/2177915536 >> 172.21.15.60:0/2177915536 conn(0x7f705406b390 msgr2=0x7f705406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:19.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.710+0000 7f70589ff700 1 -- 172.21.15.60:0/2177915536 shutdown_connections 2023-07-13T22:30:19.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.711+0000 7f70589ff700 1 -- 172.21.15.60:0/2177915536 wait complete. 2023-07-13T22:30:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.713+0000 7f70589ff700 1 Processor -- start 2023-07-13T22:30:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.713+0000 7f70589ff700 1 -- start start 2023-07-13T22:30:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.714+0000 7f70589ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540b8400 0x7f7054151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.714+0000 7f70589ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540cfad0 0x7f705415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.714+0000 7f70589ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70540d3750 0x7f70540d1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:19.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.714+0000 7f70589ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70541515b0 con 0x7f70540cfad0 2023-07-13T22:30:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.714+0000 7f70589ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705415ca10 con 0x7f70540d3750 2023-07-13T22:30:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.714+0000 7f70589ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705415ccb0 con 0x7f70540b8400 2023-07-13T22:30:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f705259c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540b8400 0x7f7054151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f704bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540cfad0 0x7f705415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f705259c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540b8400 0x7f7054151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:58378/0 (socket says 172.21.15.60:58378) 2023-07-13T22:30:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f705259c700 1 -- 172.21.15.60:0/2058940010 learned_addr learned my addr 172.21.15.60:0/2058940010 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f7052d9d700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70540d3750 0x7f70540d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:19.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f704bfff700 1 -- 172.21.15.60:0/2058940010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540b8400 msgr2=0x7f7054151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:19.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f704bfff700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540b8400 0x7f7054151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f704bfff700 1 -- 172.21.15.60:0/2058940010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70540d3750 msgr2=0x7f70540d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:19.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f704bfff700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70540d3750 0x7f70540d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f704bfff700 1 -- 172.21.15.60:0/2058940010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c009550 con 0x7f70540cfad0 2023-07-13T22:30:19.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.715+0000 7f704bfff700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540cfad0 0x7f705415dc90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f703c016000 tx=0x7f703c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:19.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.716+0000 7f704affd700 1 -- 172.21.15.60:0/2058940010 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c004450 con 0x7f70540cfad0 2023-07-13T22:30:19.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.716+0000 7f704affd700 1 -- 172.21.15.60:0/2058940010 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c01e070 con 0x7f70540cfad0 2023-07-13T22:30:19.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.716+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705415cf90 con 0x7f70540cfad0 2023-07-13T22:30:19.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.716+0000 7f704affd700 1 -- 172.21.15.60:0/2058940010 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c0038e0 con 0x7f70540cfad0 2023-07-13T22:30:19.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.716+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705415e1d0 con 0x7f70540cfad0 2023-07-13T22:30:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.717+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705415d790 con 0x7f70540cfad0 2023-07-13T22:30:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.718+0000 7f704affd700 1 -- 172.21.15.60:0/2058940010 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f703c003a40 con 0x7f70540cfad0 2023-07-13T22:30:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.719+0000 7f704affd700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f703406ace0 0x7f703406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.719+0000 7f704affd700 1 -- 172.21.15.60:0/2058940010 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f703c08a170 con 0x7f70540cfad0 2023-07-13T22:30:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.719+0000 7f705259c700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f703406ace0 0x7f703406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.720+0000 7f705259c700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f703406ace0 0x7f703406d190 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f70540b4c10 tx=0x7f7044009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.723+0000 7f704affd700 1 -- 172.21.15.60:0/2058940010 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f705415d790 con 0x7f70540cfad0 2023-07-13T22:30:19.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.974+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f705415d790 con 0x7f70540cfad0 2023-07-13T22:30:19.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.974+0000 7f704affd700 1 -- 172.21.15.60:0/2058940010 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f705415d790 con 0x7f70540cfad0 2023-07-13T22:30:19.953 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:19.953 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":76,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1446832449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1446832449},{"type":"v1","addr":"172.21.15.105:6835","nonce":1446832449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/462201037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":462201037},{"type":"v1","addr":"172.21.15.105:6834","nonce":462201037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/154242070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":154242070},{"type":"v1","addr":"172.21.15.60:6837","nonce":154242070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:29:49.290898+0000","modified":"2023-07-13T22:30:14.666909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7831":{"gid":7831,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":9,"addr":"172.21.15.60:6836/3527792396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3527792396},{"type":"v1","addr":"172.21.15.60:6836","nonce":3527792396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-13T22:30:19.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f703406ace0 msgr2=0x7f703406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:19.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f703406ace0 0x7f703406d190 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f70540b4c10 tx=0x7f7044009210 comp rx=0 tx=0).stop 2023-07-13T22:30:19.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540cfad0 msgr2=0x7f705415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:19.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540cfad0 0x7f705415dc90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f703c016000 tx=0x7f703c004750 comp rx=0 tx=0).stop 2023-07-13T22:30:19.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 shutdown_connections 2023-07-13T22:30:19.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f703406ace0 0x7f703406d190 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70540d3750 0x7f70540d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70540b8400 0x7f7054151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 --2- 172.21.15.60:0/2058940010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70540cfad0 0x7f705415dc90 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:19.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 >> 172.21.15.60:0/2058940010 conn(0x7f705406b390 msgr2=0x7f70540b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:19.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 shutdown_connections 2023-07-13T22:30:19.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:19.977+0000 7f70589ff700 1 -- 172.21.15.60:0/2058940010 wait complete. 2023-07-13T22:30:19.957 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 76 2023-07-13T22:30:19.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:30:20.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.199+0000 7f8c55bff700 1 Processor -- start 2023-07-13T22:30:20.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.199+0000 7f8c55bff700 1 -- start start 2023-07-13T22:30:20.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.200+0000 7f8c55bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c500b8400 0x7f8c500cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c55bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c500cfca0 0x7f8c500d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c55bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c500d0ac0 con 0x7f8c500d3750 2023-07-13T22:30:20.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c55bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c500b87d0 con 0x7f8c500cfca0 2023-07-13T22:30:20.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c55bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b4eb0 con 0x7f8c500b8400 2023-07-13T22:30:20.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c47fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8c500d3750 0x7f8c500cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49536/0 (socket says 172.21.15.60:49536) 2023-07-13T22:30:20.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c47fff700 1 -- 172.21.15.60:0/3038165814 learned_addr learned my addr 172.21.15.60:0/3038165814 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:20.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c4f7fe700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c500b8400 0x7f8c500cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c4ffff700 1 --2- 172.21.15.60:0/3038165814 >> v2:172.21.15.105:3300/0 conn(0x7f8c500cfca0 0x7f8c500d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821569154 0 0) 0x7f8c500d0ac0 con 0x7f8c500d3750 2023-07-13T22:30:20.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.201+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c3c0035b0 con 0x7f8c500d3750 2023-07-13T22:30:20.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28385978 0 0) 0x7f8c3c0035b0 con 0x7f8c500d3750 2023-07-13T22:30:20.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c500b8400 msgr2=0x7f8c500cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:20.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c500b8400 0x7f8c500cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 >> v2:172.21.15.105:3300/0 conn(0x7f8c500cfca0 msgr2=0x7f8c500d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:20.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 --2- 172.21.15.60:0/3038165814 >> v2:172.21.15.105:3300/0 conn(0x7f8c500cfca0 0x7f8c500d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500b5190 con 0x7f8c500d3750 2023-07-13T22:30:20.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f8c38001670 con 0x7f8c500d3750 2023-07-13T22:30:20.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 >> v1:172.21.15.60:6789/0 conn(0x7f8c500d3750 legacy=0x7f8c500cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c3c003a00 0x7f8c3c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c004430 0x7f8c3c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c500b8400 0x7f8c3c005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b87d0 con 0x7f8c3c004430 2023-07-13T22:30:20.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b4eb0 con 0x7f8c3c003a00 2023-07-13T22:30:20.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c00ba10 con 0x7f8c500b8400 2023-07-13T22:30:20.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c47fff700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c004430 0x7f8c3c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4f7fe700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c3c003a00 0x7f8c3c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4ffff700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c500b8400 0x7f8c3c005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c47fff700 1 -- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c500b8400 msgr2=0x7f8c3c005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:20.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c47fff700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c500b8400 0x7f8c3c005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c47fff700 1 -- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c3c003a00 msgr2=0x7f8c3c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:20.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c47fff700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c3c003a00 0x7f8c3c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c47fff700 1 -- 172.21.15.60:0/3038165814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0035b0 con 0x7f8c3c004430 2023-07-13T22:30:20.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4f7fe700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c3c003a00 0x7f8c3c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:20.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.202+0000 7f8c4ffff700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c500b8400 0x7f8c3c005060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:20.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.203+0000 7f8c47fff700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c004430 0x7f8c3c0048f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800c320 tx=0x7f8c3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f34822e0409290 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:20.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.203+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800ce20 con 0x7f8c3c004430 2023-07-13T22:30:20.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.203+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38005960 con 0x7f8c3c004430 2023-07-13T22:30:20.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.203+0000 7f8c4effd700 1 -- 172.21.15.60:0/3038165814 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38005d10 con 0x7f8c3c004430 2023-07-13T22:30:20.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.203+0000 7f8c55bff700 1 -- 172.21.15.60:0/3038165814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c004430 msgr2=0x7f8c3c0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.203+0000 7f8c55bff700 1 --2- 172.21.15.60:0/3038165814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c004430 0x7f8c3c0048f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800c320 tx=0x7f8c3c005880 comp rx=0 tx=0).stop 2023-07-13T22:30:20.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.203+0000 7f8c4ffff700 1 -- 172.21.15.60:0/3038165814 reap_dead start 2023-07-13T22:30:20.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.203+0000 7f8c55bff700 1 -- 172.21.15.60:0/3038165814 shutdown_connections 2023-07-13T22:30:20.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.203+0000 7f8c55bff700 1 -- 172.21.15.60:0/3038165814 >> 172.21.15.60:0/3038165814 conn(0x7f8c5006b390 msgr2=0x7f8c5006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:20.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.204+0000 7f8c55bff700 1 -- 172.21.15.60:0/3038165814 shutdown_connections 2023-07-13T22:30:20.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.204+0000 7f8c55bff700 1 -- 172.21.15.60:0/3038165814 wait complete. 2023-07-13T22:30:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.206+0000 7f8c55bff700 1 Processor -- start 2023-07-13T22:30:20.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.206+0000 7f8c55bff700 1 -- start start 2023-07-13T22:30:20.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.207+0000 7f8c55bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c3c004430 0x7f8c501f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.207+0000 7f8c55bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c500c9da0 0x7f8c501540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.207+0000 7f8c55bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c500b6a00 0x7f8c500b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c55bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b73b0 con 0x7f8c500b6a00 2023-07-13T22:30:20.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c55bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b7980 con 0x7f8c500c9da0 2023-07-13T22:30:20.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c55bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50154600 con 0x7f8c3c004430 2023-07-13T22:30:20.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c500b6a00 0x7f8c500b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c500b6a00 0x7f8c500b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37430/0 (socket says 172.21.15.60:37430) 2023-07-13T22:30:20.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4f7fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c3c004430 0x7f8c501f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4ffff700 1 -- 172.21.15.60:0/2557504601 learned_addr learned my addr 172.21.15.60:0/2557504601 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:20.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c47fff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c500c9da0 0x7f8c501540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4ffff700 1 -- 172.21.15.60:0/2557504601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c3c004430 msgr2=0x7f8c501f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4ffff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c3c004430 0x7f8c501f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4ffff700 1 -- 172.21.15.60:0/2557504601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c500c9da0 msgr2=0x7f8c501540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4ffff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c500c9da0 0x7f8c501540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4ffff700 1 -- 172.21.15.60:0/2557504601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500ca1f0 con 0x7f8c500b6a00 2023-07-13T22:30:20.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.208+0000 7f8c4ffff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c500b6a00 0x7f8c500b6e70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f8c50160f90 tx=0x7f8c480104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:20.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.209+0000 7f8c4cff9700 1 -- 172.21.15.60:0/2557504601 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4801b070 con 0x7f8c500b6a00 2023-07-13T22:30:20.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.209+0000 7f8c4cff9700 1 -- 172.21.15.60:0/2557504601 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4800f040 con 0x7f8c500b6a00 2023-07-13T22:30:20.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.209+0000 7f8c4cff9700 1 -- 172.21.15.60:0/2557504601 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800e050 con 0x7f8c500b6a00 2023-07-13T22:30:20.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.209+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c501f2380 con 0x7f8c500b6a00 2023-07-13T22:30:20.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.209+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c501f2940 con 0x7f8c500b6a00 2023-07-13T22:30:20.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.210+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5015b110 con 0x7f8c500b6a00 2023-07-13T22:30:20.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.211+0000 7f8c4cff9700 1 -- 172.21.15.60:0/2557504601 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c48003a20 con 0x7f8c500b6a00 2023-07-13T22:30:20.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.212+0000 7f8c4cff9700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8c3006ace0 0x7f8c3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.213+0000 7f8c4cff9700 1 -- 172.21.15.60:0/2557504601 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c480897c0 con 0x7f8c500b6a00 2023-07-13T22:30:20.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.213+0000 7f8c4f7fe700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8c3006ace0 0x7f8c3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.213+0000 7f8c4f7fe700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8c3006ace0 0x7f8c3006d190 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8c500b7d90 tx=0x7f8c40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:20.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.217+0000 7f8c4cff9700 1 -- 172.21.15.60:0/2557504601 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8c48054860 con 0x7f8c500b6a00 2023-07-13T22:30:20.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.425+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c5015ae30 con 0x7f8c500b6a00 2023-07-13T22:30:20.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.426+0000 7f8c4cff9700 1 -- 172.21.15.60:0/2557504601 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18267 (secure 0 0 0) 0x7f8c48057e80 con 0x7f8c500b6a00 2023-07-13T22:30:20.404 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:20.405 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:29:50.779149+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:29:47.410626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-13T22:29:48.264611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:30:20.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8c3006ace0 msgr2=0x7f8c3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8c3006ace0 0x7f8c3006d190 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8c500b7d90 tx=0x7f8c40008040 comp rx=0 tx=0).stop 2023-07-13T22:30:20.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c500b6a00 msgr2=0x7f8c500b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c500b6a00 0x7f8c500b6e70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f8c50160f90 tx=0x7f8c480104a0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 shutdown_connections 2023-07-13T22:30:20.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8c3006ace0 0x7f8c3006d190 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c500b6a00 0x7f8c500b6e70 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c3c004430 0x7f8c501f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 --2- 172.21.15.60:0/2557504601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c500c9da0 0x7f8c501540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 >> 172.21.15.60:0/2557504601 conn(0x7f8c5006b390 msgr2=0x7f8c500b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:20.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 shutdown_connections 2023-07-13T22:30:20.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.428+0000 7f8c55bff700 1 -- 172.21.15.60:0/2557504601 wait complete. 2023-07-13T22:30:20.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:29:47.410626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:29:48.264611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:30:20.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:30:20.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.652+0000 7ffa953ff700 1 Processor -- start 2023-07-13T22:30:20.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.652+0000 7ffa953ff700 1 -- start start 2023-07-13T22:30:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa953ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900b8200 0x7ffa900cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa953ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa900b1ac0 0x7ffa900cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa953ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa900d1870 con 0x7ffa900cede0 2023-07-13T22:30:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa953ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa900cf370 con 0x7ffa900b1ac0 2023-07-13T22:30:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa953ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900d12b0 con 0x7ffa900b8200 2023-07-13T22:30:20.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa8effd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900b8200 0x7ffa900cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa87fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa900b1ac0 0x7ffa900cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa8effd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900b8200 0x7ffa900cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:58412/0 (socket says 172.21.15.60:58412) 2023-07-13T22:30:20.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa8effd700 1 -- 172.21.15.60:0/872040411 learned_addr learned my addr 172.21.15.60:0/872040411 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:20.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.654+0000 7ffa8f7fe700 1 --1- 172.21.15.60:0/872040411 >> v1:172.21.15.105:6789/0 conn(0x7ffa900cede0 0x7ffa9015ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:38270/0 (socket says 172.21.15.60:38270) 2023-07-13T22:30:20.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.655+0000 7ffa8effd700 1 -- 172.21.15.60:0/872040411 >> v2:172.21.15.105:3300/0 conn(0x7ffa900b1ac0 msgr2=0x7ffa900cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:20.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.655+0000 7ffa8e7fc700 1 -- 172.21.15.60:0/872040411 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302345509 0 0) 0x7ffa900d1870 con 0x7ffa900cede0 2023-07-13T22:30:20.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.655+0000 7ffa8effd700 1 --2- 172.21.15.60:0/872040411 >> v2:172.21.15.105:3300/0 conn(0x7ffa900b1ac0 0x7ffa900cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.655+0000 7ffa8effd700 1 -- 172.21.15.60:0/872040411 >> v1:172.21.15.105:6789/0 conn(0x7ffa900cede0 legacy=0x7ffa9015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.655+0000 7ffa8effd700 1 -- 172.21.15.60:0/872040411 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa900d1590 con 0x7ffa900b8200 2023-07-13T22:30:20.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.655+0000 7ffa87fff700 1 --2- 172.21.15.60:0/872040411 >> v2:172.21.15.105:3300/0 conn(0x7ffa900b1ac0 0x7ffa900cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:20.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.655+0000 7ffa8effd700 1 --2- 172.21.15.60:0/872040411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900b8200 0x7ffa900cf640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa800098f0 tx=0x7ffa80009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284be8e8fe213afc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:20.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.655+0000 7ffa8e7fc700 1 -- 172.21.15.60:0/872040411 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa80011830 con 0x7ffa900b8200 2023-07-13T22:30:20.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa8e7fc700 1 -- 172.21.15.60:0/872040411 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa80011990 con 0x7ffa900b8200 2023-07-13T22:30:20.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa8e7fc700 1 -- 172.21.15.60:0/872040411 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa80011d20 con 0x7ffa900b8200 2023-07-13T22:30:20.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa953ff700 1 -- 172.21.15.60:0/872040411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900b8200 msgr2=0x7ffa900cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa953ff700 1 --2- 172.21.15.60:0/872040411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900b8200 0x7ffa900cf640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa800098f0 tx=0x7ffa80009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa953ff700 1 -- 172.21.15.60:0/872040411 shutdown_connections 2023-07-13T22:30:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa953ff700 1 --2- 172.21.15.60:0/872040411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900b8200 0x7ffa900cf640 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa953ff700 1 --2- 172.21.15.60:0/872040411 >> v2:172.21.15.105:3300/0 conn(0x7ffa900b1ac0 0x7ffa900cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa953ff700 1 -- 172.21.15.60:0/872040411 >> 172.21.15.60:0/872040411 conn(0x7ffa9006b2f0 msgr2=0x7ffa9006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:20.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa953ff700 1 -- 172.21.15.60:0/872040411 shutdown_connections 2023-07-13T22:30:20.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.656+0000 7ffa953ff700 1 -- 172.21.15.60:0/872040411 wait complete. 2023-07-13T22:30:20.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.658+0000 7ffa953ff700 1 Processor -- start 2023-07-13T22:30:20.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.658+0000 7ffa953ff700 1 -- start start 2023-07-13T22:30:20.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.659+0000 7ffa953ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa900b1ac0 0x7ffa9015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.659+0000 7ffa953ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa900b8200 0x7ffa90158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.659+0000 7ffa953ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900cede0 0x7ffa901e27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.659+0000 7ffa953ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900c6900 con 0x7ffa900b8200 2023-07-13T22:30:20.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.659+0000 7ffa953ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900c77c0 con 0x7ffa900b1ac0 2023-07-13T22:30:20.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.659+0000 7ffa953ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa90158660 con 0x7ffa900cede0 2023-07-13T22:30:20.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.659+0000 7ffa87fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa900b8200 0x7ffa90158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.659+0000 7ffa87fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa900b8200 0x7ffa90158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37446/0 (socket says 172.21.15.60:37446) 2023-07-13T22:30:20.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.659+0000 7ffa87fff700 1 -- 172.21.15.60:0/2501154835 learned_addr learned my addr 172.21.15.60:0/2501154835 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:20.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa8f7fe700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900cede0 0x7ffa901e27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa8effd700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa900b1ac0 0x7ffa9015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa87fff700 1 -- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900cede0 msgr2=0x7ffa901e27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa87fff700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900cede0 0x7ffa901e27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa87fff700 1 -- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa900b1ac0 msgr2=0x7ffa9015ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa87fff700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa900b1ac0 0x7ffa9015ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa87fff700 1 -- 172.21.15.60:0/2501154835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780094a0 con 0x7ffa900b8200 2023-07-13T22:30:20.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa8effd700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa900b1ac0 0x7ffa9015ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:20.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa8f7fe700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900cede0 0x7ffa901e27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:20.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa87fff700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa900b8200 0x7ffa90158120 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800ebe0 tx=0x7ffa78009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:20.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa877fe700 1 -- 172.21.15.60:0/2501154835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800c910 con 0x7ffa900b8200 2023-07-13T22:30:20.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa877fe700 1 -- 172.21.15.60:0/2501154835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7800ca70 con 0x7ffa900b8200 2023-07-13T22:30:20.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa877fe700 1 -- 172.21.15.60:0/2501154835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800cde0 con 0x7ffa900b8200 2023-07-13T22:30:20.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.660+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa80009550 con 0x7ffa900b8200 2023-07-13T22:30:20.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.661+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa900c6ba0 con 0x7ffa900b8200 2023-07-13T22:30:20.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.661+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa900c6760 con 0x7ffa900b8200 2023-07-13T22:30:20.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.662+0000 7ffa877fe700 1 -- 172.21.15.60:0/2501154835 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa78013470 con 0x7ffa900b8200 2023-07-13T22:30:20.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.663+0000 7ffa877fe700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7006ace0 0x7ffa7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:20.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.663+0000 7ffa877fe700 1 -- 172.21.15.60:0/2501154835 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa780897c0 con 0x7ffa900b8200 2023-07-13T22:30:20.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.664+0000 7ffa8effd700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7006ace0 0x7ffa7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:20.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.664+0000 7ffa8effd700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7006ace0 0x7ffa7006d190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000c00 tx=0x7ffa80008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:20.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.668+0000 7ffa877fe700 1 -- 172.21.15.60:0/2501154835 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffa900c6760 con 0x7ffa900b8200 2023-07-13T22:30:20.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.877+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffa900c6760 con 0x7ffa900b8200 2023-07-13T22:30:20.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.878+0000 7ffa877fe700 1 -- 172.21.15.60:0/2501154835 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v112) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffa900c6760 con 0x7ffa900b8200 2023-07-13T22:30:20.856 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:30:20.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7006ace0 msgr2=0x7ffa7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7006ace0 0x7ffa7006d190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000c00 tx=0x7ffa80008030 comp rx=0 tx=0).stop 2023-07-13T22:30:20.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa900b8200 msgr2=0x7ffa90158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:20.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa900b8200 0x7ffa90158120 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800ebe0 tx=0x7ffa78009ec0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 shutdown_connections 2023-07-13T22:30:20.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7006ace0 0x7ffa7006d190 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa900cede0 0x7ffa901e27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa900b1ac0 0x7ffa9015ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 --2- 172.21.15.60:0/2501154835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa900b8200 0x7ffa90158120 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 >> 172.21.15.60:0/2501154835 conn(0x7ffa9006b2f0 msgr2=0x7ffa900b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 shutdown_connections 2023-07-13T22:30:20.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:20.880+0000 7ffa953ff700 1 -- 172.21.15.60:0/2501154835 wait complete. 2023-07-13T22:30:20.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:30:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.105+0000 7f2c9a7ff700 1 Processor -- start 2023-07-13T22:30:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.106+0000 7f2c9a7ff700 1 -- start start 2023-07-13T22:30:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c9a7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2c940b8200 0x7f2c940cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c9a7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2c940b1ac0 0x7f2c940cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c9a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2c940cede0 0x7f2c9415ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c9a7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940d1870 con 0x7f2c940b8200 2023-07-13T22:30:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c9a7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940cf370 con 0x7f2c940b1ac0 2023-07-13T22:30:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c9a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940d12b0 con 0x7f2c940cede0 2023-07-13T22:30:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c93fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2c940b8200 0x7f2c940cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c93fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2c940b8200 0x7f2c940cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37460/0 (socket says 172.21.15.60:37460) 2023-07-13T22:30:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c93fff700 1 -- 172.21.15.60:0/2735598492 learned_addr learned my addr 172.21.15.60:0/2735598492 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c937fe700 1 --2- 172.21.15.60:0/2735598492 >> v2:172.21.15.105:3300/0 conn(0x7f2c940b1ac0 0x7f2c940cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.107+0000 7f2c98d9c700 1 --2- 172.21.15.60:0/2735598492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2c940cede0 0x7f2c9415ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.108+0000 7f2c93fff700 1 -- 172.21.15.60:0/2735598492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2c940cede0 msgr2=0x7f2c9415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.108+0000 7f2c93fff700 1 --2- 172.21.15.60:0/2735598492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2c940cede0 0x7f2c9415ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.108+0000 7f2c93fff700 1 -- 172.21.15.60:0/2735598492 >> v2:172.21.15.105:3300/0 conn(0x7f2c940b1ac0 msgr2=0x7f2c940cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.108+0000 7f2c93fff700 1 --2- 172.21.15.60:0/2735598492 >> v2:172.21.15.105:3300/0 conn(0x7f2c940b1ac0 0x7f2c940cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.108+0000 7f2c93fff700 1 -- 172.21.15.60:0/2735598492 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940d1590 con 0x7f2c940b8200 2023-07-13T22:30:21.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.108+0000 7f2c93fff700 1 --2- 172.21.15.60:0/2735598492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b8200 0x7f2c940cf640 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2c940d07b0 tx=0x7f2c84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdf4f335c17638 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:21.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.108+0000 7f2c92ffd700 1 -- 172.21.15.60:0/2735598492 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84011830 con 0x7f2c940b8200 2023-07-13T22:30:21.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.108+0000 7f2c92ffd700 1 -- 172.21.15.60:0/2735598492 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c84011990 con 0x7f2c940b8200 2023-07-13T22:30:21.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.108+0000 7f2c92ffd700 1 -- 172.21.15.60:0/2735598492 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84011d20 con 0x7f2c940b8200 2023-07-13T22:30:21.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.109+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/2735598492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b8200 msgr2=0x7f2c940cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.109+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/2735598492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b8200 0x7f2c940cf640 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2c940d07b0 tx=0x7f2c84009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.109+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/2735598492 shutdown_connections 2023-07-13T22:30:21.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.109+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/2735598492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2c940cede0 0x7f2c9415ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.109+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/2735598492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b8200 0x7f2c940cf640 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.109+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/2735598492 >> v2:172.21.15.105:3300/0 conn(0x7f2c940b1ac0 0x7f2c940cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.109+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/2735598492 >> 172.21.15.60:0/2735598492 conn(0x7f2c9406b2f0 msgr2=0x7f2c9406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:21.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.109+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/2735598492 shutdown_connections 2023-07-13T22:30:21.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.109+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/2735598492 wait complete. 2023-07-13T22:30:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.111+0000 7f2c9a7ff700 1 Processor -- start 2023-07-13T22:30:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.111+0000 7f2c9a7ff700 1 -- start start 2023-07-13T22:30:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c9a7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b1ac0 0x7f2c941493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c9a7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2c940b8200 0x7f2c94159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c9a7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c940cede0 0x7f2c94156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c9a7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415a230 con 0x7f2c940b1ac0 2023-07-13T22:30:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c9a7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c941573d0 con 0x7f2c940cede0 2023-07-13T22:30:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c9a7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940c7e80 con 0x7f2c940b8200 2023-07-13T22:30:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c93fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b1ac0 0x7f2c941493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c93fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b1ac0 0x7f2c941493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37470/0 (socket says 172.21.15.60:37470) 2023-07-13T22:30:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c93fff700 1 -- 172.21.15.60:0/1454041988 learned_addr learned my addr 172.21.15.60:0/1454041988 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c93fff700 1 -- 172.21.15.60:0/1454041988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2c940b8200 msgr2=0x7f2c94159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c93fff700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2c940b8200 0x7f2c94159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c93fff700 1 -- 172.21.15.60:0/1454041988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c940cede0 msgr2=0x7f2c94156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c93fff700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c940cede0 0x7f2c94156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.112+0000 7f2c93fff700 1 -- 172.21.15.60:0/1454041988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c84009550 con 0x7f2c940b1ac0 2023-07-13T22:30:21.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.113+0000 7f2c93fff700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b1ac0 0x7f2c941493c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000c00 tx=0x7f2c84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:21.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.113+0000 7f2c90ff9700 1 -- 172.21.15.60:0/1454041988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84004300 con 0x7f2c940b1ac0 2023-07-13T22:30:21.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.113+0000 7f2c90ff9700 1 -- 172.21.15.60:0/1454041988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8401f070 con 0x7f2c940b1ac0 2023-07-13T22:30:21.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.113+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c940c8a00 con 0x7f2c940b1ac0 2023-07-13T22:30:21.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.113+0000 7f2c90ff9700 1 -- 172.21.15.60:0/1454041988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840036a0 con 0x7f2c940b1ac0 2023-07-13T22:30:21.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.113+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c940c83b0 con 0x7f2c940b1ac0 2023-07-13T22:30:21.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.114+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9404e210 con 0x7f2c940b1ac0 2023-07-13T22:30:21.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.115+0000 7f2c90ff9700 1 -- 172.21.15.60:0/1454041988 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c84021030 con 0x7f2c940b1ac0 2023-07-13T22:30:21.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.116+0000 7f2c90ff9700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2c8006ace0 0x7f2c8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.116+0000 7f2c90ff9700 1 -- 172.21.15.60:0/1454041988 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2c8408a6c0 con 0x7f2c940b1ac0 2023-07-13T22:30:21.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.116+0000 7f2c937fe700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2c8006ace0 0x7f2c8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.117+0000 7f2c937fe700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2c8006ace0 0x7f2c8006d190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2c88006b20 tx=0x7f2c88006ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:21.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.120+0000 7f2c90ff9700 1 -- 172.21.15.60:0/1454041988 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2c84055760 con 0x7f2c940b1ac0 2023-07-13T22:30:21.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.327+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c940b4f70 con 0x7f2c940b1ac0 2023-07-13T22:30:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.327+0000 7f2c90ff9700 1 -- 172.21.15.60:0/1454041988 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c84058d80 con 0x7f2c940b1ac0 2023-07-13T22:30:21.306 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:30:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2c8006ace0 msgr2=0x7f2c8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2c8006ace0 0x7f2c8006d190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2c88006b20 tx=0x7f2c88006ab0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b1ac0 msgr2=0x7f2c941493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b1ac0 0x7f2c941493c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000c00 tx=0x7f2c84004750 comp rx=0 tx=0).stop 2023-07-13T22:30:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 shutdown_connections 2023-07-13T22:30:21.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2c8006ace0 0x7f2c8006d190 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c940b1ac0 0x7f2c941493c0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c940cede0 0x7f2c94156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 --2- 172.21.15.60:0/1454041988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2c940b8200 0x7f2c94159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 >> 172.21.15.60:0/1454041988 conn(0x7f2c9406b2f0 msgr2=0x7f2c940d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:21.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 shutdown_connections 2023-07-13T22:30:21.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.330+0000 7f2c9a7ff700 1 -- 172.21.15.60:0/1454041988 wait complete. 2023-07-13T22:30:21.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:30:21.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.553+0000 7ff266bff700 1 Processor -- start 2023-07-13T22:30:21.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.553+0000 7ff266bff700 1 -- start start 2023-07-13T22:30:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff266bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff2600b8200 0x7ff2600cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff266bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2600b1ac0 0x7ff2600cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff266bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2600cede0 0x7ff26015ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff266bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2600d1870 con 0x7ff2600b1ac0 2023-07-13T22:30:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff266bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2600cf370 con 0x7ff2600b8200 2023-07-13T22:30:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff266bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600d12b0 con 0x7ff2600cede0 2023-07-13T22:30:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff25ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2600b1ac0 0x7ff2600cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff25ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2600b1ac0 0x7ff2600cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37482/0 (socket says 172.21.15.60:37482) 2023-07-13T22:30:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff25ffff700 1 -- 172.21.15.60:0/3060510583 learned_addr learned my addr 172.21.15.60:0/3060510583 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff26519c700 1 --2- 172.21.15.60:0/3060510583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2600cede0 0x7ff26015ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.555+0000 7ff26499b700 1 --2- 172.21.15.60:0/3060510583 >> v2:172.21.15.105:3300/0 conn(0x7ff2600b8200 0x7ff2600cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff25ffff700 1 -- 172.21.15.60:0/3060510583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2600cede0 msgr2=0x7ff26015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff25ffff700 1 --2- 172.21.15.60:0/3060510583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2600cede0 0x7ff26015ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff25ffff700 1 -- 172.21.15.60:0/3060510583 >> v2:172.21.15.105:3300/0 conn(0x7ff2600b8200 msgr2=0x7ff2600cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff25ffff700 1 --2- 172.21.15.60:0/3060510583 >> v2:172.21.15.105:3300/0 conn(0x7ff2600b8200 0x7ff2600cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff25ffff700 1 -- 172.21.15.60:0/3060510583 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600d1590 con 0x7ff2600b1ac0 2023-07-13T22:30:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff26499b700 1 --2- 172.21.15.60:0/3060510583 >> v2:172.21.15.105:3300/0 conn(0x7ff2600b8200 0x7ff2600cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff25ffff700 1 --2- 172.21.15.60:0/3060510583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 0x7ff2600cfb80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b2660 tx=0x7ff24c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4430b344a46caf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff25f7fe700 1 -- 172.21.15.60:0/3060510583 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00b870 con 0x7ff2600b1ac0 2023-07-13T22:30:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff25f7fe700 1 -- 172.21.15.60:0/3060510583 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c00b9d0 con 0x7ff2600b1ac0 2023-07-13T22:30:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.556+0000 7ff25f7fe700 1 -- 172.21.15.60:0/3060510583 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00bd60 con 0x7ff2600b1ac0 2023-07-13T22:30:21.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.557+0000 7ff266bff700 1 -- 172.21.15.60:0/3060510583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 msgr2=0x7ff2600cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.557+0000 7ff266bff700 1 --2- 172.21.15.60:0/3060510583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 0x7ff2600cfb80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b2660 tx=0x7ff24c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.557+0000 7ff266bff700 1 -- 172.21.15.60:0/3060510583 shutdown_connections 2023-07-13T22:30:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.557+0000 7ff266bff700 1 --2- 172.21.15.60:0/3060510583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2600cede0 0x7ff26015ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.557+0000 7ff266bff700 1 --2- 172.21.15.60:0/3060510583 >> v2:172.21.15.105:3300/0 conn(0x7ff2600b8200 0x7ff2600cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.557+0000 7ff266bff700 1 --2- 172.21.15.60:0/3060510583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 0x7ff2600cfb80 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.557+0000 7ff266bff700 1 -- 172.21.15.60:0/3060510583 >> 172.21.15.60:0/3060510583 conn(0x7ff26006b2f0 msgr2=0x7ff26006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.557+0000 7ff266bff700 1 -- 172.21.15.60:0/3060510583 shutdown_connections 2023-07-13T22:30:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.557+0000 7ff266bff700 1 -- 172.21.15.60:0/3060510583 wait complete. 2023-07-13T22:30:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.559+0000 7ff266bff700 1 Processor -- start 2023-07-13T22:30:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.559+0000 7ff266bff700 1 -- start start 2023-07-13T22:30:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff266bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 0x7ff2601f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff266bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2600b8200 0x7ff26015c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff266bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2600cede0 0x7ff26015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff266bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff260151af0 con 0x7ff2600b1ac0 2023-07-13T22:30:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff266bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26015d4f0 con 0x7ff2600b8200 2023-07-13T22:30:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff266bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26015d7d0 con 0x7ff2600cede0 2023-07-13T22:30:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff26499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 0x7ff2601f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff26499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 0x7ff2601f1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37488/0 (socket says 172.21.15.60:37488) 2023-07-13T22:30:21.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff26499b700 1 -- 172.21.15.60:0/1669272805 learned_addr learned my addr 172.21.15.60:0/1669272805 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:21.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff26499b700 1 -- 172.21.15.60:0/1669272805 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2600cede0 msgr2=0x7ff26015cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff26499b700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2600cede0 0x7ff26015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff26499b700 1 -- 172.21.15.60:0/1669272805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2600b8200 msgr2=0x7ff26015c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff26499b700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2600b8200 0x7ff26015c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.560+0000 7ff26499b700 1 -- 172.21.15.60:0/1669272805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009550 con 0x7ff2600b1ac0 2023-07-13T22:30:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.561+0000 7ff26499b700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 0x7ff2601f1380 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff25400e810 tx=0x7ff25400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.561+0000 7ff25d7fa700 1 -- 172.21.15.60:0/1669272805 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254013610 con 0x7ff2600b1ac0 2023-07-13T22:30:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.561+0000 7ff25d7fa700 1 -- 172.21.15.60:0/1669272805 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254013770 con 0x7ff2600b1ac0 2023-07-13T22:30:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.561+0000 7ff25d7fa700 1 -- 172.21.15.60:0/1669272805 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254013b20 con 0x7ff2600b1ac0 2023-07-13T22:30:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.561+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26015dab0 con 0x7ff2600b1ac0 2023-07-13T22:30:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.561+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26015e070 con 0x7ff2600b1ac0 2023-07-13T22:30:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.562+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26004e210 con 0x7ff2600b1ac0 2023-07-13T22:30:21.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.563+0000 7ff25d7fa700 1 -- 172.21.15.60:0/1669272805 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff254013c80 con 0x7ff2600b1ac0 2023-07-13T22:30:21.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.564+0000 7ff25d7fa700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff24406ace0 0x7ff24406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.564+0000 7ff25d7fa700 1 -- 172.21.15.60:0/1669272805 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff254089200 con 0x7ff2600b1ac0 2023-07-13T22:30:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.565+0000 7ff25ffff700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff24406ace0 0x7ff24406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.565+0000 7ff25ffff700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff24406ace0 0x7ff24406d190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2600c8a90 tx=0x7ff24c010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.568+0000 7ff25d7fa700 1 -- 172.21.15.60:0/1669272805 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff254057ca0 con 0x7ff2600b1ac0 2023-07-13T22:30:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.777+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2600b4f70 con 0x7ff2600b1ac0 2023-07-13T22:30:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.777+0000 7ff25d7fa700 1 -- 172.21.15.60:0/1669272805 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff254057830 con 0x7ff2600b1ac0 2023-07-13T22:30:21.756 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:30:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.779+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff24406ace0 msgr2=0x7ff24406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.779+0000 7ff266bff700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff24406ace0 0x7ff24406d190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2600c8a90 tx=0x7ff24c010000 comp rx=0 tx=0).stop 2023-07-13T22:30:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.779+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 msgr2=0x7ff2601f1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.779+0000 7ff266bff700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 0x7ff2601f1380 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff25400e810 tx=0x7ff25400ebd0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.780+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 shutdown_connections 2023-07-13T22:30:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.780+0000 7ff266bff700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff24406ace0 0x7ff24406d190 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.780+0000 7ff266bff700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600b1ac0 0x7ff2601f1380 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.780+0000 7ff266bff700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2600cede0 0x7ff26015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.780+0000 7ff266bff700 1 --2- 172.21.15.60:0/1669272805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2600b8200 0x7ff26015c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.780+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 >> 172.21.15.60:0/1669272805 conn(0x7ff26006b2f0 msgr2=0x7ff26006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.780+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 shutdown_connections 2023-07-13T22:30:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:21.780+0000 7ff266bff700 1 -- 172.21.15.60:0/1669272805 wait complete. 2023-07-13T22:30:21.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:30:21.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.005+0000 7f3e1a1ff700 1 Processor -- start 2023-07-13T22:30:21.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.005+0000 7f3e1a1ff700 1 -- start start 2023-07-13T22:30:21.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.006+0000 7f3e1a1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e14159260 0x7f3e140ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.006+0000 7f3e1a1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e140cf640 con 0x7f3e140cc410 2023-07-13T22:30:21.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.006+0000 7f3e1a1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e140ccf80 con 0x7f3e14070af0 2023-07-13T22:30:21.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.006+0000 7f3e1a1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140c9b40 con 0x7f3e14159260 2023-07-13T22:30:21.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.007+0000 7f3e12ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3e140cc410 0x7f3e140703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49548/0 (socket says 172.21.15.60:49548) 2023-07-13T22:30:21.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.007+0000 7f3e12ffd700 1 -- 172.21.15.60:0/570218043 learned_addr learned my addr 172.21.15.60:0/570218043 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:21.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.007+0000 7f3e137fe700 1 --2- 172.21.15.60:0/570218043 >> v2:172.21.15.105:3300/0 conn(0x7f3e14159260 0x7f3e140ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.007+0000 7f3e127fc700 1 -- 172.21.15.60:0/570218043 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548008986 0 0) 0x7f3e140cf640 con 0x7f3e140cc410 2023-07-13T22:30:21.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.007+0000 7f3e127fc700 1 -- 172.21.15.60:0/570218043 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dfc0035b0 con 0x7f3e140cc410 2023-07-13T22:30:21.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.007+0000 7f3e137fe700 1 -- 172.21.15.60:0/570218043 >> v1:172.21.15.105:6789/0 conn(0x7f3e14070af0 legacy=0x7f3e1406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.007+0000 7f3e137fe700 1 -- 172.21.15.60:0/570218043 >> v1:172.21.15.60:6789/0 conn(0x7f3e140cc410 legacy=0x7f3e140703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.007+0000 7f3e137fe700 1 -- 172.21.15.60:0/570218043 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140c9e20 con 0x7f3e14159260 2023-07-13T22:30:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.008+0000 7f3e137fe700 1 --2- 172.21.15.60:0/570218043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e14159260 0x7f3e140ca900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009840 tx=0x7f3e04009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a35e8f4e9ccbd2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.008+0000 7f3e127fc700 1 -- 172.21.15.60:0/570218043 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04011830 con 0x7f3e14159260 2023-07-13T22:30:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.008+0000 7f3e127fc700 1 -- 172.21.15.60:0/570218043 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e04011990 con 0x7f3e14159260 2023-07-13T22:30:21.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.008+0000 7f3e127fc700 1 -- 172.21.15.60:0/570218043 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04011d00 con 0x7f3e14159260 2023-07-13T22:30:21.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.008+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/570218043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e14159260 msgr2=0x7f3e140ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.008+0000 7f3e1a1ff700 1 --2- 172.21.15.60:0/570218043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e14159260 0x7f3e140ca900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009840 tx=0x7f3e04009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:21.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.008+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/570218043 shutdown_connections 2023-07-13T22:30:21.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.008+0000 7f3e1a1ff700 1 --2- 172.21.15.60:0/570218043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e14159260 0x7f3e140ca900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.008+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/570218043 >> 172.21.15.60:0/570218043 conn(0x7f3e140cbf50 msgr2=0x7f3e140ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:21.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.009+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/570218043 shutdown_connections 2023-07-13T22:30:21.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.009+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/570218043 wait complete. 2023-07-13T22:30:21.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.010+0000 7f3e1a1ff700 1 Processor -- start 2023-07-13T22:30:21.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.010+0000 7f3e1a1ff700 1 -- start start 2023-07-13T22:30:21.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.011+0000 7f3e1a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e14070af0 0x7f3e14158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.011+0000 7f3e1a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140cc410 0x7f3e140ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.011+0000 7f3e1a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e14159260 0x7f3e14155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.011+0000 7f3e1a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140ae6a0 con 0x7f3e14159260 2023-07-13T22:30:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.011+0000 7f3e1a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140ae920 con 0x7f3e14070af0 2023-07-13T22:30:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.011+0000 7f3e1a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140aec00 con 0x7f3e140cc410 2023-07-13T22:30:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e13fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e14159260 0x7f3e14155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e13fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e14159260 0x7f3e14155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37500/0 (socket says 172.21.15.60:37500) 2023-07-13T22:30:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e13fff700 1 -- 172.21.15.60:0/744813887 learned_addr learned my addr 172.21.15.60:0/744813887 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e12ffd700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140cc410 0x7f3e140ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e137fe700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e14070af0 0x7f3e14158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:21.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e13fff700 1 -- 172.21.15.60:0/744813887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140cc410 msgr2=0x7f3e140ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e13fff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140cc410 0x7f3e140ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e13fff700 1 -- 172.21.15.60:0/744813887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e14070af0 msgr2=0x7f3e14158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:21.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e13fff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e14070af0 0x7f3e14158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:21.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e13fff700 1 -- 172.21.15.60:0/744813887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040094a0 con 0x7f3e14159260 2023-07-13T22:30:21.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e12ffd700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140cc410 0x7f3e140ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:21.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.012+0000 7f3e13fff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e14159260 0x7f3e14155060 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00cc50 tx=0x7f3e0c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:21.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.013+0000 7f3dfbfff700 1 -- 172.21.15.60:0/744813887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c003c00 con 0x7f3e14159260 2023-07-13T22:30:21.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.013+0000 7f3dfbfff700 1 -- 172.21.15.60:0/744813887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c003d60 con 0x7f3e14159260 2023-07-13T22:30:22.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.013+0000 7f3dfbfff700 1 -- 172.21.15.60:0/744813887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00ab60 con 0x7f3e14159260 2023-07-13T22:30:22.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.013+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e140aeee0 con 0x7f3e14159260 2023-07-13T22:30:22.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.013+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e140af4a0 con 0x7f3e14159260 2023-07-13T22:30:22.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.014+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e140af6c0 con 0x7f3e14159260 2023-07-13T22:30:22.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.015+0000 7f3dfbfff700 1 -- 172.21.15.60:0/744813887 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e0c00acc0 con 0x7f3e14159260 2023-07-13T22:30:22.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.016+0000 7f3dfbfff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.016+0000 7f3dfbfff700 1 -- 172.21.15.60:0/744813887 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e0c0895e0 con 0x7f3e14159260 2023-07-13T22:30:22.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.016+0000 7f3e137fe700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.017+0000 7f3e137fe700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3e04000c00 tx=0x7f3e0401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:22.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.021+0000 7f3dfbfff700 1 -- 172.21.15.60:0/744813887 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3e0c0545f0 con 0x7f3e14159260 2023-07-13T22:30:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.229+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e1404e210 con 0x7f3e14159260 2023-07-13T22:30:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.229+0000 7f3dfbfff700 1 -- 172.21.15.60:0/744813887 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e0c057c10 con 0x7f3e14159260 2023-07-13T22:30:22.208 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:30:22.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.231+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 msgr2=0x7f3e0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.231+0000 7f3e1a1ff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3e04000c00 tx=0x7f3e0401c040 comp rx=0 tx=0).stop 2023-07-13T22:30:22.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e14159260 msgr2=0x7f3e14155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e14159260 0x7f3e14155060 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00cc50 tx=0x7f3e0c00cf60 comp rx=0 tx=0).stop 2023-07-13T22:30:22.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 shutdown_connections 2023-07-13T22:30:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e14159260 0x7f3e14155060 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e14070af0 0x7f3e14158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 --2- 172.21.15.60:0/744813887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140cc410 0x7f3e140ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 >> 172.21.15.60:0/744813887 conn(0x7f3e140cbf50 msgr2=0x7f3e1406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 shutdown_connections 2023-07-13T22:30:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.232+0000 7f3e1a1ff700 1 -- 172.21.15.60:0/744813887 wait complete. 2023-07-13T22:30:22.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:22.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.453+0000 7fef969ff700 1 Processor -- start 2023-07-13T22:30:22.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.453+0000 7fef969ff700 1 -- start start 2023-07-13T22:30:22.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.454+0000 7fef969ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900d3750 0x7fef900cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.454+0000 7fef969ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900cd950 con 0x7fef900cfad0 2023-07-13T22:30:22.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.454+0000 7fef969ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900cff20 con 0x7fef900b8400 2023-07-13T22:30:22.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.454+0000 7fef969ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900d0ac0 con 0x7fef900d3750 2023-07-13T22:30:22.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.454+0000 7fef94f9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fef900cfad0 0x7fef900d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49564/0 (socket says 172.21.15.60:49564) 2023-07-13T22:30:22.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.455+0000 7fef94f9c700 1 -- 172.21.15.60:0/611687258 learned_addr learned my addr 172.21.15.60:0/611687258 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:22.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.455+0000 7fef8f7fe700 1 --2- 172.21.15.60:0/611687258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900d3750 0x7fef900cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.455+0000 7fef8effd700 1 -- 172.21.15.60:0/611687258 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25366483 0 0) 0x7fef88001490 con 0x7fef900cfad0 2023-07-13T22:30:22.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.455+0000 7fef8effd700 1 -- 172.21.15.60:0/611687258 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef780035b0 con 0x7fef900cfad0 2023-07-13T22:30:22.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.455+0000 7fef8effd700 1 -- 172.21.15.60:0/611687258 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534836377 0 0) 0x7fef80001110 con 0x7fef900b8400 2023-07-13T22:30:22.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.455+0000 7fef8f7fe700 1 -- 172.21.15.60:0/611687258 >> v1:172.21.15.105:6789/0 conn(0x7fef900b8400 legacy=0x7fef900cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.455+0000 7fef8f7fe700 1 -- 172.21.15.60:0/611687258 >> v1:172.21.15.60:6789/0 conn(0x7fef900cfad0 legacy=0x7fef900d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.455+0000 7fef8f7fe700 1 -- 172.21.15.60:0/611687258 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900b87d0 con 0x7fef900d3750 2023-07-13T22:30:22.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.455+0000 7fef8f7fe700 1 --2- 172.21.15.60:0/611687258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900d3750 0x7fef900cf590 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fef84009870 tx=0x7fef84009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ddff7561e4fd756 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:22.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef8effd700 1 -- 172.21.15.60:0/611687258 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84011830 con 0x7fef900d3750 2023-07-13T22:30:22.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef8effd700 1 -- 172.21.15.60:0/611687258 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84011990 con 0x7fef900d3750 2023-07-13T22:30:22.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef8effd700 1 -- 172.21.15.60:0/611687258 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84011d20 con 0x7fef900d3750 2023-07-13T22:30:22.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef969ff700 1 -- 172.21.15.60:0/611687258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900d3750 msgr2=0x7fef900cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef969ff700 1 --2- 172.21.15.60:0/611687258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900d3750 0x7fef900cf590 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fef84009870 tx=0x7fef84009b80 comp rx=0 tx=0).stop 2023-07-13T22:30:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef969ff700 1 -- 172.21.15.60:0/611687258 shutdown_connections 2023-07-13T22:30:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef969ff700 1 --2- 172.21.15.60:0/611687258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900d3750 0x7fef900cf590 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef969ff700 1 -- 172.21.15.60:0/611687258 >> 172.21.15.60:0/611687258 conn(0x7fef9006b390 msgr2=0x7fef9006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef969ff700 1 -- 172.21.15.60:0/611687258 shutdown_connections 2023-07-13T22:30:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.456+0000 7fef969ff700 1 -- 172.21.15.60:0/611687258 wait complete. 2023-07-13T22:30:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.458+0000 7fef969ff700 1 Processor -- start 2023-07-13T22:30:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.458+0000 7fef969ff700 1 -- start start 2023-07-13T22:30:22.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.459+0000 7fef969ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900b8400 0x7fef90150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.459+0000 7fef969ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef900cfad0 0x7fef9015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.459+0000 7fef969ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef900d3750 0x7fef900c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.459+0000 7fef969ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900c6a80 con 0x7fef900d3750 2023-07-13T22:30:22.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.459+0000 7fef969ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90150ec0 con 0x7fef900cfad0 2023-07-13T22:30:22.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.459+0000 7fef969ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900c6750 con 0x7fef900b8400 2023-07-13T22:30:22.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.459+0000 7fef94f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef900d3750 0x7fef900c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.459+0000 7fef8ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900b8400 0x7fef90150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.459+0000 7fef94f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef900d3750 0x7fef900c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37510/0 (socket says 172.21.15.60:37510) 2023-07-13T22:30:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef8f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef900cfad0 0x7fef9015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef8ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900b8400 0x7fef90150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:58504/0 (socket says 172.21.15.60:58504) 2023-07-13T22:30:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef94f9c700 1 -- 172.21.15.60:0/231140534 learned_addr learned my addr 172.21.15.60:0/231140534 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef94f9c700 1 -- 172.21.15.60:0/231140534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900b8400 msgr2=0x7fef90150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef94f9c700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900b8400 0x7fef90150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef94f9c700 1 -- 172.21.15.60:0/231140534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef900cfad0 msgr2=0x7fef9015e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef94f9c700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef900cfad0 0x7fef9015e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef94f9c700 1 -- 172.21.15.60:0/231140534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840094a0 con 0x7fef900d3750 2023-07-13T22:30:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef8f7fe700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef900cfad0 0x7fef9015e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.460+0000 7fef94f9c700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef900d3750 0x7fef900c6f10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fef8800cae0 tx=0x7fef8800cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.461+0000 7fef8cff9700 1 -- 172.21.15.60:0/231140534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800a9a0 con 0x7fef900d3750 2023-07-13T22:30:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.461+0000 7fef8cff9700 1 -- 172.21.15.60:0/231140534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800ab00 con 0x7fef900d3750 2023-07-13T22:30:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.461+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef900c7cd0 con 0x7fef900d3750 2023-07-13T22:30:22.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.461+0000 7fef8cff9700 1 -- 172.21.15.60:0/231140534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800ae90 con 0x7fef900d3750 2023-07-13T22:30:22.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.461+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef900c8290 con 0x7fef900d3750 2023-07-13T22:30:22.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.462+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef900c6bc0 con 0x7fef900d3750 2023-07-13T22:30:22.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.463+0000 7fef8cff9700 1 -- 172.21.15.60:0/231140534 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef8801b3b0 con 0x7fef900d3750 2023-07-13T22:30:22.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.463+0000 7fef8cff9700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef7c06ace0 0x7fef7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.464+0000 7fef8cff9700 1 -- 172.21.15.60:0/231140534 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef88088f20 con 0x7fef900d3750 2023-07-13T22:30:22.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.464+0000 7fef8ffff700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef7c06ace0 0x7fef7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.464+0000 7fef8ffff700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef7c06ace0 0x7fef7c06d190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fef80000f60 tx=0x7fef80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:22.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.468+0000 7fef8cff9700 1 -- 172.21.15.60:0/231140534 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fef88057a50 con 0x7fef900d3750 2023-07-13T22:30:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.721+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef900cd950 con 0x7fef900d3750 2023-07-13T22:30:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.721+0000 7fef8cff9700 1 -- 172.21.15.60:0/231140534 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fef900cd950 con 0x7fef900d3750 2023-07-13T22:30:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:22.700 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":77,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1446832449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1446832449},{"type":"v1","addr":"172.21.15.105:6835","nonce":1446832449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/462201037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":462201037},{"type":"v1","addr":"172.21.15.105:6834","nonce":462201037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/154242070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":154242070},{"type":"v1","addr":"172.21.15.60:6837","nonce":154242070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:29:49.290898+0000","modified":"2023-07-13T22:30:22.701620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7831":{"gid":7831,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6836/3527792396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3527792396},{"type":"v1","addr":"172.21.15.60:6836","nonce":3527792396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-13T22:30:22.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.723+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef7c06ace0 msgr2=0x7fef7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.723+0000 7fef969ff700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef7c06ace0 0x7fef7c06d190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fef80000f60 tx=0x7fef80009040 comp rx=0 tx=0).stop 2023-07-13T22:30:22.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.723+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef900d3750 msgr2=0x7fef900c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.723+0000 7fef969ff700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef900d3750 0x7fef900c6f10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fef8800cae0 tx=0x7fef8800cdf0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.723+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 shutdown_connections 2023-07-13T22:30:22.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.723+0000 7fef969ff700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fef7c06ace0 0x7fef7c06d190 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.723+0000 7fef969ff700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef900d3750 0x7fef900c6f10 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.723+0000 7fef969ff700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fef900b8400 0x7fef90150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.723+0000 7fef969ff700 1 --2- 172.21.15.60:0/231140534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef900cfad0 0x7fef9015e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.724+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 >> 172.21.15.60:0/231140534 conn(0x7fef9006b390 msgr2=0x7fef900b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:22.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.724+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 shutdown_connections 2023-07-13T22:30:22.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.724+0000 7fef969ff700 1 -- 172.21.15.60:0/231140534 wait complete. 2023-07-13T22:30:22.704 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 77 2023-07-13T22:30:22.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:30:22.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.945+0000 7f7e7abff700 1 Processor -- start 2023-07-13T22:30:22.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.945+0000 7f7e7abff700 1 -- start start 2023-07-13T22:30:22.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.946+0000 7f7e7abff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7e7415c6d0 0x7f7e740b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.946+0000 7f7e7abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e740b46c0 0x7f7e740b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.946+0000 7f7e7abff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e740d18a0 con 0x7f7e7415c300 2023-07-13T22:30:22.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.946+0000 7f7e7abff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e740cf500 con 0x7f7e7415c6d0 2023-07-13T22:30:22.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.946+0000 7f7e7abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740b5020 con 0x7f7e740b46c0 2023-07-13T22:30:22.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.947+0000 7f7e73fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7e7415c6d0 0x7f7e740b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.947+0000 7f7e73fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7e7415c6d0 0x7f7e740b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37512/0 (socket says 172.21.15.60:37512) 2023-07-13T22:30:22.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.947+0000 7f7e73fff700 1 -- 172.21.15.60:0/677338130 learned_addr learned my addr 172.21.15.60:0/677338130 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:22.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.947+0000 7f7e7919c700 1 --2- 172.21.15.60:0/677338130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e740b46c0 0x7f7e740b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.947+0000 7f7e73fff700 1 -- 172.21.15.60:0/677338130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e740b46c0 msgr2=0x7f7e740b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:22.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.947+0000 7f7e73fff700 1 --2- 172.21.15.60:0/677338130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e740b46c0 0x7f7e740b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.947+0000 7f7e73fff700 1 -- 172.21.15.60:0/677338130 >> v1:172.21.15.105:6789/0 conn(0x7f7e7415c300 legacy=0x7f7e740cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.947+0000 7f7e73fff700 1 -- 172.21.15.60:0/677338130 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740d15c0 con 0x7f7e7415c6d0 2023-07-13T22:30:22.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.947+0000 7f7e73fff700 1 --2- 172.21.15.60:0/677338130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 0x7f7e740b5a90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f7e600098f0 tx=0x7f7e60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d7acd4e1ccb485 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:22.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.948+0000 7f7e737fe700 1 -- 172.21.15.60:0/677338130 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000b8a0 con 0x7f7e7415c6d0 2023-07-13T22:30:22.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.948+0000 7f7e737fe700 1 -- 172.21.15.60:0/677338130 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6000ba00 con 0x7f7e7415c6d0 2023-07-13T22:30:22.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.948+0000 7f7e737fe700 1 -- 172.21.15.60:0/677338130 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000bd90 con 0x7f7e7415c6d0 2023-07-13T22:30:22.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.948+0000 7f7e7abff700 1 -- 172.21.15.60:0/677338130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 msgr2=0x7f7e740b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.948+0000 7f7e7abff700 1 --2- 172.21.15.60:0/677338130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 0x7f7e740b5a90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f7e600098f0 tx=0x7f7e60009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:22.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.948+0000 7f7e7abff700 1 -- 172.21.15.60:0/677338130 shutdown_connections 2023-07-13T22:30:22.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.949+0000 7f7e7abff700 1 --2- 172.21.15.60:0/677338130 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e740b46c0 0x7f7e740b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.949+0000 7f7e7abff700 1 --2- 172.21.15.60:0/677338130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 0x7f7e740b5a90 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.949+0000 7f7e7abff700 1 -- 172.21.15.60:0/677338130 >> 172.21.15.60:0/677338130 conn(0x7f7e740cf6c0 msgr2=0x7f7e740cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:22.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.949+0000 7f7e7abff700 1 -- 172.21.15.60:0/677338130 shutdown_connections 2023-07-13T22:30:22.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.949+0000 7f7e7abff700 1 -- 172.21.15.60:0/677338130 wait complete. 2023-07-13T22:30:22.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.950+0000 7f7e7abff700 1 Processor -- start 2023-07-13T22:30:22.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.950+0000 7f7e7abff700 1 -- start start 2023-07-13T22:30:22.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.951+0000 7f7e7abff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e740b46c0 0x7f7e74156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.951+0000 7f7e7abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e7415c300 0x7f7e741f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.951+0000 7f7e7abff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 0x7f7e740cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.951+0000 7f7e7abff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740b3d90 con 0x7f7e7415c6d0 2023-07-13T22:30:22.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.951+0000 7f7e7abff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e74155800 con 0x7f7e740b46c0 2023-07-13T22:30:22.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.951+0000 7f7e7abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e74155aa0 con 0x7f7e7415c300 2023-07-13T22:30:22.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e7919c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 0x7f7e740cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e73fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e7415c300 0x7f7e741f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e7919c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 0x7f7e740cc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37528/0 (socket says 172.21.15.60:37528) 2023-07-13T22:30:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e7919c700 1 -- 172.21.15.60:0/377108207 learned_addr learned my addr 172.21.15.60:0/377108207 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e7899b700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e740b46c0 0x7f7e74156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e73fff700 1 -- 172.21.15.60:0/377108207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e740b46c0 msgr2=0x7f7e74156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e73fff700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e740b46c0 0x7f7e74156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e73fff700 1 -- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 msgr2=0x7f7e740cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e73fff700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 0x7f7e740cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:22.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e73fff700 1 -- 172.21.15.60:0/377108207 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60009550 con 0x7f7e7415c300 2023-07-13T22:30:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.952+0000 7f7e7919c700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 0x7f7e740cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.953+0000 7f7e73fff700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e7415c300 0x7f7e741f17f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e60016000 tx=0x7f7e60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.953+0000 7f7e717fa700 1 -- 172.21.15.60:0/377108207 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60004300 con 0x7f7e7415c300 2023-07-13T22:30:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.953+0000 7f7e717fa700 1 -- 172.21.15.60:0/377108207 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6001f070 con 0x7f7e7415c300 2023-07-13T22:30:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.953+0000 7f7e717fa700 1 -- 172.21.15.60:0/377108207 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e600036a0 con 0x7f7e7415c300 2023-07-13T22:30:22.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.953+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e74155d80 con 0x7f7e7415c300 2023-07-13T22:30:22.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.953+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e741f1d30 con 0x7f7e7415c300 2023-07-13T22:30:22.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.954+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e741f1ec0 con 0x7f7e7415c300 2023-07-13T22:30:22.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.955+0000 7f7e717fa700 1 -- 172.21.15.60:0/377108207 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e60021030 con 0x7f7e7415c300 2023-07-13T22:30:22.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.956+0000 7f7e717fa700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e5806ace0 0x7f7e5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:22.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.956+0000 7f7e717fa700 1 -- 172.21.15.60:0/377108207 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e6008a6f0 con 0x7f7e7415c300 2023-07-13T22:30:22.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.957+0000 7f7e7899b700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e5806ace0 0x7f7e5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:22.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.957+0000 7f7e7899b700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e5806ace0 0x7f7e5806d190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7e68007fd0 tx=0x7f7e68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:22.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:22.961+0000 7f7e717fa700 1 -- 172.21.15.60:0/377108207 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7e60055710 con 0x7f7e7415c300 2023-07-13T22:30:23.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.167+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e740cd390 con 0x7f7e7415c300 2023-07-13T22:30:23.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.169+0000 7f7e717fa700 1 -- 172.21.15.60:0/377108207 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18267 (secure 0 0 0) 0x7f7e60058d30 con 0x7f7e7415c300 2023-07-13T22:30:23.147 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:23.148 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:29:50.779149+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:29:47.410626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-13T22:29:48.264611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:30:23.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e5806ace0 msgr2=0x7f7e5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e5806ace0 0x7f7e5806d190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7e68007fd0 tx=0x7f7e68009040 comp rx=0 tx=0).stop 2023-07-13T22:30:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e7415c300 msgr2=0x7f7e741f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e7415c300 0x7f7e741f17f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e60016000 tx=0x7f7e60004750 comp rx=0 tx=0).stop 2023-07-13T22:30:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 shutdown_connections 2023-07-13T22:30:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e5806ace0 0x7f7e5806d190 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e7415c6d0 0x7f7e740cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e740b46c0 0x7f7e74156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 --2- 172.21.15.60:0/377108207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e7415c300 0x7f7e741f17f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 >> 172.21.15.60:0/377108207 conn(0x7f7e740cf6c0 msgr2=0x7f7e740732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:23.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 shutdown_connections 2023-07-13T22:30:23.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.171+0000 7f7e7abff700 1 -- 172.21.15.60:0/377108207 wait complete. 2023-07-13T22:30:23.169 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:30:23.170 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.396+0000 7ff768dff700 1 Processor -- start 2023-07-13T22:30:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.397+0000 7ff768dff700 1 -- start start 2023-07-13T22:30:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff768dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7640b8400 0x7ff7640cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff768dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7640d3750 0x7ff7640cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff768dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7640cfad0 0x7ff76415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff768dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7640cff50 con 0x7ff7640b8400 2023-07-13T22:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff768dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7640d0ac0 con 0x7ff7640d3750 2023-07-13T22:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff768dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7640b87d0 con 0x7ff7640cfad0 2023-07-13T22:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff76259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7640b8400 0x7ff7640cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff76259c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7640b8400 0x7ff7640cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37542/0 (socket says 172.21.15.60:37542) 2023-07-13T22:30:23.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff76259c700 1 -- 172.21.15.60:0/444465108 learned_addr learned my addr 172.21.15.60:0/444465108 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff76259c700 1 -- 172.21.15.60:0/444465108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7640cfad0 msgr2=0x7ff76415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff76259c700 1 --2- 172.21.15.60:0/444465108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7640cfad0 0x7ff76415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff76259c700 1 -- 172.21.15.60:0/444465108 >> v2:172.21.15.105:3300/0 conn(0x7ff7640d3750 msgr2=0x7ff7640cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.398+0000 7ff75bfff700 1 --2- 172.21.15.60:0/444465108 >> v2:172.21.15.105:3300/0 conn(0x7ff7640d3750 0x7ff7640cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.399+0000 7ff76259c700 1 --2- 172.21.15.60:0/444465108 >> v2:172.21.15.105:3300/0 conn(0x7ff7640d3750 0x7ff7640cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.399+0000 7ff76259c700 1 -- 172.21.15.60:0/444465108 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7640b4eb0 con 0x7ff7640b8400 2023-07-13T22:30:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.399+0000 7ff75bfff700 1 --2- 172.21.15.60:0/444465108 >> v2:172.21.15.105:3300/0 conn(0x7ff7640d3750 0x7ff7640cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.399+0000 7ff76259c700 1 --2- 172.21.15.60:0/444465108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640b8400 0x7ff7640cd240 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff7640b8cf0 tx=0x7ff754009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c59714bf6fcc24 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.399+0000 7ff761d9b700 1 -- 172.21.15.60:0/444465108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400b870 con 0x7ff7640b8400 2023-07-13T22:30:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.399+0000 7ff761d9b700 1 -- 172.21.15.60:0/444465108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75400b9d0 con 0x7ff7640b8400 2023-07-13T22:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.399+0000 7ff761d9b700 1 -- 172.21.15.60:0/444465108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400bd60 con 0x7ff7640b8400 2023-07-13T22:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.400+0000 7ff768dff700 1 -- 172.21.15.60:0/444465108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640b8400 msgr2=0x7ff7640cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.400+0000 7ff768dff700 1 --2- 172.21.15.60:0/444465108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640b8400 0x7ff7640cd240 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff7640b8cf0 tx=0x7ff754009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.400+0000 7ff768dff700 1 -- 172.21.15.60:0/444465108 shutdown_connections 2023-07-13T22:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.400+0000 7ff768dff700 1 --2- 172.21.15.60:0/444465108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7640cfad0 0x7ff76415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.400+0000 7ff768dff700 1 --2- 172.21.15.60:0/444465108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640b8400 0x7ff7640cd240 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.400+0000 7ff768dff700 1 --2- 172.21.15.60:0/444465108 >> v2:172.21.15.105:3300/0 conn(0x7ff7640d3750 0x7ff7640cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.400+0000 7ff768dff700 1 -- 172.21.15.60:0/444465108 >> 172.21.15.60:0/444465108 conn(0x7ff76406b390 msgr2=0x7ff76406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.400+0000 7ff768dff700 1 -- 172.21.15.60:0/444465108 shutdown_connections 2023-07-13T22:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.400+0000 7ff768dff700 1 -- 172.21.15.60:0/444465108 wait complete. 2023-07-13T22:30:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.402+0000 7ff768dff700 1 Processor -- start 2023-07-13T22:30:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.402+0000 7ff768dff700 1 -- start start 2023-07-13T22:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff768dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7640b8400 0x7ff7640cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff768dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7640cfad0 0x7ff7641539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff768dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640d3750 0x7ff764150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff768dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff764153ee0 con 0x7ff7640d3750 2023-07-13T22:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff768dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76415f000 con 0x7ff7640cfad0 2023-07-13T22:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff768dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7641546b0 con 0x7ff7640b8400 2023-07-13T22:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff762d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640d3750 0x7ff764150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff762d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640d3750 0x7ff764150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37544/0 (socket says 172.21.15.60:37544) 2023-07-13T22:30:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff762d9d700 1 -- 172.21.15.60:0/550872749 learned_addr learned my addr 172.21.15.60:0/550872749 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff76259c700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7640b8400 0x7ff7640cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff75bfff700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7640cfad0 0x7ff7641539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff762d9d700 1 -- 172.21.15.60:0/550872749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7640b8400 msgr2=0x7ff7640cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.403+0000 7ff762d9d700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7640b8400 0x7ff7640cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.404+0000 7ff762d9d700 1 -- 172.21.15.60:0/550872749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7640cfad0 msgr2=0x7ff7641539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.404+0000 7ff762d9d700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7640cfad0 0x7ff7641539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.404+0000 7ff762d9d700 1 -- 172.21.15.60:0/550872749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754009550 con 0x7ff7640d3750 2023-07-13T22:30:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.404+0000 7ff762d9d700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640d3750 0x7ff764150bb0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00b9a0 tx=0x7ff75c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.404+0000 7ff75affd700 1 -- 172.21.15.60:0/550872749 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c00d850 con 0x7ff7640d3750 2023-07-13T22:30:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.404+0000 7ff75affd700 1 -- 172.21.15.60:0/550872749 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c00d9d0 con 0x7ff7640d3750 2023-07-13T22:30:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.404+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7641540f0 con 0x7ff7640d3750 2023-07-13T22:30:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.404+0000 7ff75affd700 1 -- 172.21.15.60:0/550872749 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c00dd80 con 0x7ff7640d3750 2023-07-13T22:30:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.404+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7641f5e10 con 0x7ff7640d3750 2023-07-13T22:30:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.405+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7640b89f0 con 0x7ff7640d3750 2023-07-13T22:30:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.406+0000 7ff75affd700 1 -- 172.21.15.60:0/550872749 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff75c00e420 con 0x7ff7640d3750 2023-07-13T22:30:23.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.407+0000 7ff75affd700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff74406ace0 0x7ff74406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.407+0000 7ff75affd700 1 -- 172.21.15.60:0/550872749 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff75c089620 con 0x7ff7640d3750 2023-07-13T22:30:23.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.407+0000 7ff76259c700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff74406ace0 0x7ff74406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.408+0000 7ff76259c700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff74406ace0 0x7ff74406d190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff754014040 tx=0x7ff754008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.411+0000 7ff75affd700 1 -- 172.21.15.60:0/550872749 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff75c054630 con 0x7ff7640d3750 2023-07-13T22:30:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.661+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7640b3e90 con 0x7ff7640d3750 2023-07-13T22:30:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.662+0000 7ff75affd700 1 -- 172.21.15.60:0/550872749 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4712 (secure 0 0 0) 0x7ff75c057c50 con 0x7ff7640d3750 2023-07-13T22:30:23.640 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:23.641 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":77,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1446832449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1446832449},{"type":"v1","addr":"172.21.15.105:6835","nonce":1446832449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/462201037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":462201037},{"type":"v1","addr":"172.21.15.105:6834","nonce":462201037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/154242070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":154242070},{"type":"v1","addr":"172.21.15.60:6837","nonce":154242070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:29:49.290898+0000","modified":"2023-07-13T22:30:22.701620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7831":{"gid":7831,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6836/3527792396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3527792396},{"type":"v1","addr":"172.21.15.60:6836","nonce":3527792396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-13T22:30:23.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff74406ace0 msgr2=0x7ff74406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff74406ace0 0x7ff74406d190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff754014040 tx=0x7ff754008030 comp rx=0 tx=0).stop 2023-07-13T22:30:23.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640d3750 msgr2=0x7ff764150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640d3750 0x7ff764150bb0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00b9a0 tx=0x7ff75c00bcb0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 shutdown_connections 2023-07-13T22:30:23.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff74406ace0 0x7ff74406d190 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7640d3750 0x7ff764150bb0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7640b8400 0x7ff7640cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 --2- 172.21.15.60:0/550872749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7640cfad0 0x7ff7641539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.664+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 >> 172.21.15.60:0/550872749 conn(0x7ff76406b390 msgr2=0x7ff7640d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:23.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.665+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 shutdown_connections 2023-07-13T22:30:23.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.665+0000 7ff768dff700 1 -- 172.21.15.60:0/550872749 wait complete. 2023-07-13T22:30:23.644 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 77 2023-07-13T22:30:23.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.888+0000 7ffa8d3ff700 1 Processor -- start 2023-07-13T22:30:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.889+0000 7ffa8d3ff700 1 -- start start 2023-07-13T22:30:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.890+0000 7ffa8d3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffa880b2010 0x7ffa880cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.890+0000 7ffa8d3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880d26d0 0x7ffa880cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.890+0000 7ffa8d3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa880d14a0 con 0x7ffa880b5a00 2023-07-13T22:30:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.890+0000 7ffa8d3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa880d2570 con 0x7ffa880b2010 2023-07-13T22:30:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.890+0000 7ffa8d3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880d1780 con 0x7ffa880d26d0 2023-07-13T22:30:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.890+0000 7ffa86ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffa880b2010 0x7ffa880cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.890+0000 7ffa86ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffa880b2010 0x7ffa880cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37550/0 (socket says 172.21.15.60:37550) 2023-07-13T22:30:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.890+0000 7ffa86ffd700 1 -- 172.21.15.60:0/2361468702 learned_addr learned my addr 172.21.15.60:0/2361468702 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.890+0000 7ffa867fc700 1 --2- 172.21.15.60:0/2361468702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880d26d0 0x7ffa880cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.891+0000 7ffa86ffd700 1 -- 172.21.15.60:0/2361468702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880d26d0 msgr2=0x7ffa880cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.891+0000 7ffa86ffd700 1 --2- 172.21.15.60:0/2361468702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880d26d0 0x7ffa880cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.891+0000 7ffa86ffd700 1 -- 172.21.15.60:0/2361468702 >> v1:172.21.15.105:6789/0 conn(0x7ffa880b5a00 legacy=0x7ffa880d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.891+0000 7ffa86ffd700 1 -- 172.21.15.60:0/2361468702 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa880cd2a0 con 0x7ffa880b2010 2023-07-13T22:30:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.891+0000 7ffa867fc700 1 --2- 172.21.15.60:0/2361468702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880d26d0 0x7ffa880cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.891+0000 7ffa86ffd700 1 --2- 172.21.15.60:0/2361468702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880b2010 0x7ffa880cf1a0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ffa780098f0 tx=0x7ffa78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e5a1bdac266904 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.891+0000 7ffa85ffb700 1 -- 172.21.15.60:0/2361468702 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78004230 con 0x7ffa880b2010 2023-07-13T22:30:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.891+0000 7ffa85ffb700 1 -- 172.21.15.60:0/2361468702 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa78004390 con 0x7ffa880b2010 2023-07-13T22:30:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.891+0000 7ffa85ffb700 1 -- 172.21.15.60:0/2361468702 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa780036a0 con 0x7ffa880b2010 2023-07-13T22:30:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.892+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/2361468702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880b2010 msgr2=0x7ffa880cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.892+0000 7ffa8d3ff700 1 --2- 172.21.15.60:0/2361468702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880b2010 0x7ffa880cf1a0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ffa780098f0 tx=0x7ffa78009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.892+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/2361468702 shutdown_connections 2023-07-13T22:30:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.892+0000 7ffa8d3ff700 1 --2- 172.21.15.60:0/2361468702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880b2010 0x7ffa880cf1a0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.892+0000 7ffa8d3ff700 1 --2- 172.21.15.60:0/2361468702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880d26d0 0x7ffa880cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.892+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/2361468702 >> 172.21.15.60:0/2361468702 conn(0x7ffa8806b380 msgr2=0x7ffa8806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.892+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/2361468702 shutdown_connections 2023-07-13T22:30:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.892+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/2361468702 wait complete. 2023-07-13T22:30:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.894+0000 7ffa8d3ff700 1 Processor -- start 2023-07-13T22:30:23.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.894+0000 7ffa8d3ff700 1 -- start start 2023-07-13T22:30:23.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa8d3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa880b2010 0x7ffa8815f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa8d3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880b5a00 0x7ffa881f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa8d3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880d26d0 0x7ffa8815df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa8d3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa881f3080 con 0x7ffa880d26d0 2023-07-13T22:30:23.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa8d3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880a9a80 con 0x7ffa880b2010 2023-07-13T22:30:23.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa8d3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880cc600 con 0x7ffa880b5a00 2023-07-13T22:30:23.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880d26d0 0x7ffa8815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa877fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880d26d0 0x7ffa8815df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37558/0 (socket says 172.21.15.60:37558) 2023-07-13T22:30:23.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa877fe700 1 -- 172.21.15.60:0/1824764950 learned_addr learned my addr 172.21.15.60:0/1824764950 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:23.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa86ffd700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa880b2010 0x7ffa8815f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa867fc700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880b5a00 0x7ffa881f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa877fe700 1 -- 172.21.15.60:0/1824764950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880b5a00 msgr2=0x7ffa881f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa877fe700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880b5a00 0x7ffa881f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa877fe700 1 -- 172.21.15.60:0/1824764950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa880b2010 msgr2=0x7ffa8815f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa877fe700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa880b2010 0x7ffa8815f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa877fe700 1 -- 172.21.15.60:0/1824764950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa78009550 con 0x7ffa880d26d0 2023-07-13T22:30:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.895+0000 7ffa86ffd700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa880b2010 0x7ffa8815f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.896+0000 7ffa877fe700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880d26d0 0x7ffa8815df90 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000cc10 tx=0x7ffa8000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.896+0000 7ffa6f7fe700 1 -- 172.21.15.60:0/1824764950 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000abb0 con 0x7ffa880d26d0 2023-07-13T22:30:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.896+0000 7ffa6f7fe700 1 -- 172.21.15.60:0/1824764950 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8000ad10 con 0x7ffa880d26d0 2023-07-13T22:30:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.896+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa880cb860 con 0x7ffa880d26d0 2023-07-13T22:30:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.896+0000 7ffa6f7fe700 1 -- 172.21.15.60:0/1824764950 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa80012660 con 0x7ffa880d26d0 2023-07-13T22:30:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.896+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa880cbe00 con 0x7ffa880d26d0 2023-07-13T22:30:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.897+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8804e280 con 0x7ffa880d26d0 2023-07-13T22:30:23.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.898+0000 7ffa6f7fe700 1 -- 172.21.15.60:0/1824764950 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa800127c0 con 0x7ffa880d26d0 2023-07-13T22:30:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.899+0000 7ffa6f7fe700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7406ace0 0x7ffa7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.899+0000 7ffa6f7fe700 1 -- 172.21.15.60:0/1824764950 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa80089c10 con 0x7ffa880d26d0 2023-07-13T22:30:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.899+0000 7ffa86ffd700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7406ace0 0x7ffa7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.900+0000 7ffa86ffd700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7406ace0 0x7ffa7406d190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009130 tx=0x7ffa78008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:23.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:23.904+0000 7ffa6f7fe700 1 -- 172.21.15.60:0/1824764950 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffa80054cb0 con 0x7ffa880d26d0 2023-07-13T22:30:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.154+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa880d1160 con 0x7ffa880d26d0 2023-07-13T22:30:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.154+0000 7ffa6f7fe700 1 -- 172.21.15.60:0/1824764950 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4712 (secure 0 0 0) 0x7ffa80016020 con 0x7ffa880d26d0 2023-07-13T22:30:24.133 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:24.133 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":77,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1446832449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1446832449},{"type":"v1","addr":"172.21.15.105:6835","nonce":1446832449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/462201037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":462201037},{"type":"v1","addr":"172.21.15.105:6834","nonce":462201037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/154242070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":154242070},{"type":"v1","addr":"172.21.15.60:6837","nonce":154242070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:29:49.290898+0000","modified":"2023-07-13T22:30:22.701620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7831":{"gid":7831,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6836/3527792396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3527792396},{"type":"v1","addr":"172.21.15.60:6836","nonce":3527792396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-13T22:30:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7406ace0 msgr2=0x7ffa7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7406ace0 0x7ffa7406d190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009130 tx=0x7ffa78008030 comp rx=0 tx=0).stop 2023-07-13T22:30:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880d26d0 msgr2=0x7ffa8815df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880d26d0 0x7ffa8815df90 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000cc10 tx=0x7ffa8000a060 comp rx=0 tx=0).stop 2023-07-13T22:30:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 shutdown_connections 2023-07-13T22:30:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffa7406ace0 0x7ffa7406d190 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa880d26d0 0x7ffa8815df90 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa880b2010 0x7ffa8815f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 --2- 172.21.15.60:0/1824764950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa880b5a00 0x7ffa881f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.156+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 >> 172.21.15.60:0/1824764950 conn(0x7ffa8806b380 msgr2=0x7ffa880c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:24.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.157+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 shutdown_connections 2023-07-13T22:30:24.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.157+0000 7ffa8d3ff700 1 -- 172.21.15.60:0/1824764950 wait complete. 2023-07-13T22:30:24.137 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 77 2023-07-13T22:30:24.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:30:24.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.378+0000 7f9efc1ff700 1 Processor -- start 2023-07-13T22:30:24.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.378+0000 7f9efc1ff700 1 -- start start 2023-07-13T22:30:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.379+0000 7f9efc1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ef40b8570 0x7f9ef40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.379+0000 7f9efc1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 0x7f9ef415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.379+0000 7f9efc1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef40d09d0 con 0x7f9ef40d35c0 2023-07-13T22:30:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9efc1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef40d3b30 con 0x7f9ef40b8570 2023-07-13T22:30:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9efc1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40cda10 con 0x7f9ef40d15c0 2023-07-13T22:30:24.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9ef9f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ef40b8570 0x7f9ef40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9ef9f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ef40b8570 0x7f9ef40ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37574/0 (socket says 172.21.15.60:37574) 2023-07-13T22:30:24.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9ef979a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9ef40d35c0 0x7f9ef40d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49570/0 (socket says 172.21.15.60:49570) 2023-07-13T22:30:24.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9ef9f9b700 1 -- 172.21.15.60:0/3699077246 learned_addr learned my addr 172.21.15.60:0/3699077246 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:24.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9efa79c700 1 --2- 172.21.15.60:0/3699077246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 0x7f9ef415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9ef9f9b700 1 -- 172.21.15.60:0/3699077246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 msgr2=0x7f9ef415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9ef9f9b700 1 --2- 172.21.15.60:0/3699077246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 0x7f9ef415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9ef9f9b700 1 -- 172.21.15.60:0/3699077246 >> v1:172.21.15.60:6789/0 conn(0x7f9ef40d35c0 legacy=0x7f9ef40d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9ef9f9b700 1 -- 172.21.15.60:0/3699077246 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40cd450 con 0x7f9ef40b8570 2023-07-13T22:30:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9efa79c700 1 --2- 172.21.15.60:0/3699077246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 0x7f9ef415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.380+0000 7f9ef9f9b700 1 --2- 172.21.15.60:0/3699077246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 0x7f9ef40ceeb0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0006fd0 tx=0x7f9ef0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddd48e29e43fd60 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9ef8f99700 1 -- 172.21.15.60:0/3699077246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0005e30 con 0x7f9ef40b8570 2023-07-13T22:30:24.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9ef8f99700 1 -- 172.21.15.60:0/3699077246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef0004030 con 0x7f9ef40b8570 2023-07-13T22:30:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9ef8f99700 1 -- 172.21.15.60:0/3699077246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00043e0 con 0x7f9ef40b8570 2023-07-13T22:30:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9efc1ff700 1 -- 172.21.15.60:0/3699077246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 msgr2=0x7f9ef40ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9efc1ff700 1 --2- 172.21.15.60:0/3699077246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 0x7f9ef40ceeb0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0006fd0 tx=0x7f9ef0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9efc1ff700 1 -- 172.21.15.60:0/3699077246 shutdown_connections 2023-07-13T22:30:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9efc1ff700 1 --2- 172.21.15.60:0/3699077246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 0x7f9ef415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9efc1ff700 1 --2- 172.21.15.60:0/3699077246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 0x7f9ef40ceeb0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9efc1ff700 1 -- 172.21.15.60:0/3699077246 >> 172.21.15.60:0/3699077246 conn(0x7f9ef4071600 msgr2=0x7f9ef40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:24.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9efc1ff700 1 -- 172.21.15.60:0/3699077246 shutdown_connections 2023-07-13T22:30:24.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.381+0000 7f9efc1ff700 1 -- 172.21.15.60:0/3699077246 wait complete. 2023-07-13T22:30:24.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.383+0000 7f9efc1ff700 1 Processor -- start 2023-07-13T22:30:24.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.383+0000 7f9efc1ff700 1 -- start start 2023-07-13T22:30:24.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.384+0000 7f9efc1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 0x7f9ef414a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.384+0000 7f9efc1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 0x7f9ef40b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.384+0000 7f9efc1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef40d35c0 0x7f9ef40b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.384+0000 7f9efc1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b26a0 con 0x7f9ef40b8570 2023-07-13T22:30:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.384+0000 7f9efc1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40b3500 con 0x7f9ef40d35c0 2023-07-13T22:30:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.384+0000 7f9efc1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef414a9d0 con 0x7f9ef40d15c0 2023-07-13T22:30:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.384+0000 7f9ef9f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 0x7f9ef414a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.384+0000 7f9ef9f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 0x7f9ef414a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37584/0 (socket says 172.21.15.60:37584) 2023-07-13T22:30:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.384+0000 7f9ef9f9b700 1 -- 172.21.15.60:0/4196325548 learned_addr learned my addr 172.21.15.60:0/4196325548 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9ef979a700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 0x7f9ef40b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9efa79c700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef40d35c0 0x7f9ef40b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9ef9f9b700 1 -- 172.21.15.60:0/4196325548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 msgr2=0x7f9ef40b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9ef9f9b700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 0x7f9ef40b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9ef9f9b700 1 -- 172.21.15.60:0/4196325548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef40d35c0 msgr2=0x7f9ef40b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9ef9f9b700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef40d35c0 0x7f9ef40b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9ef9f9b700 1 -- 172.21.15.60:0/4196325548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0009500 con 0x7f9ef40b8570 2023-07-13T22:30:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9ef9f9b700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 0x7f9ef414a490 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009c10 tx=0x7f9ef0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9eea7fc700 1 -- 172.21.15.60:0/4196325548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0011620 con 0x7f9ef40b8570 2023-07-13T22:30:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9eea7fc700 1 -- 172.21.15.60:0/4196325548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef00117a0 con 0x7f9ef40b8570 2023-07-13T22:30:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef40b28f0 con 0x7f9ef40b8570 2023-07-13T22:30:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.385+0000 7f9eea7fc700 1 -- 172.21.15.60:0/4196325548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0011b50 con 0x7f9ef40b8570 2023-07-13T22:30:24.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.386+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef40b2eb0 con 0x7f9ef40b8570 2023-07-13T22:30:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.387+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef40cf3a0 con 0x7f9ef40b8570 2023-07-13T22:30:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.388+0000 7f9eea7fc700 1 -- 172.21.15.60:0/4196325548 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ef0011cb0 con 0x7f9ef40b8570 2023-07-13T22:30:24.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.389+0000 7f9eea7fc700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ed806ace0 0x7f9ed806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.389+0000 7f9eea7fc700 1 -- 172.21.15.60:0/4196325548 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ef008a5f0 con 0x7f9ef40b8570 2023-07-13T22:30:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.389+0000 7f9ef979a700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ed806ace0 0x7f9ed806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.390+0000 7f9ef979a700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ed806ace0 0x7f9ed806d190 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4151840 tx=0x7f9ee4007d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:24.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.393+0000 7f9eea7fc700 1 -- 172.21.15.60:0/4196325548 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9ef0055690 con 0x7f9ef40b8570 2023-07-13T22:30:24.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.602+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ef404e280 con 0x7f9ef40b8570 2023-07-13T22:30:24.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.603+0000 7f9eea7fc700 1 -- 172.21.15.60:0/4196325548 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18267 (secure 0 0 0) 0x7f9ef0058cb0 con 0x7f9ef40b8570 2023-07-13T22:30:24.582 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:24.582 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:29:50.779149+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:29:47.410626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-13T22:29:48.264611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:30:24.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ed806ace0 msgr2=0x7f9ed806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ed806ace0 0x7f9ed806d190 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4151840 tx=0x7f9ee4007d10 comp rx=0 tx=0).stop 2023-07-13T22:30:24.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 msgr2=0x7f9ef414a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 0x7f9ef414a490 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009c10 tx=0x7f9ef0004750 comp rx=0 tx=0).stop 2023-07-13T22:30:24.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 shutdown_connections 2023-07-13T22:30:24.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ed806ace0 0x7f9ed806d190 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef40b8570 0x7f9ef414a490 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef40d35c0 0x7f9ef40b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 --2- 172.21.15.60:0/4196325548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ef40d15c0 0x7f9ef40b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 >> 172.21.15.60:0/4196325548 conn(0x7f9ef4071600 msgr2=0x7f9ef40cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:24.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 shutdown_connections 2023-07-13T22:30:24.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.606+0000 7f9efc1ff700 1 -- 172.21.15.60:0/4196325548 wait complete. 2023-07-13T22:30:24.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:30:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.831+0000 7f76743ff700 1 Processor -- start 2023-07-13T22:30:24.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.831+0000 7f76743ff700 1 -- start start 2023-07-13T22:30:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.832+0000 7f76743ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f766c15c6e0 0x7f766c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.832+0000 7f76743ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f766c0b3050 0x7f766c06afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.832+0000 7f76743ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c06b8e0 con 0x7f766c0b24b0 2023-07-13T22:30:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.832+0000 7f76743ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c15cab0 con 0x7f766c0b3050 2023-07-13T22:30:24.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.832+0000 7f76743ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c06bbc0 con 0x7f766c15c6e0 2023-07-13T22:30:24.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.833+0000 7f767199a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f766c0b3050 0x7f766c06afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.833+0000 7f767299c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f766c0b24b0 0x7f766c0cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49576/0 (socket says 172.21.15.60:49576) 2023-07-13T22:30:24.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.833+0000 7f767299c700 1 -- 172.21.15.60:0/3993512810 learned_addr learned my addr 172.21.15.60:0/3993512810 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:24.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.833+0000 7f767219b700 1 --2- 172.21.15.60:0/3993512810 >> v2:172.21.15.105:3300/0 conn(0x7f766c15c6e0 0x7f766c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.833+0000 7f767199a700 1 -- 172.21.15.60:0/3993512810 >> v2:172.21.15.105:3300/0 conn(0x7f766c15c6e0 msgr2=0x7f766c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:24.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.833+0000 7f767199a700 1 --2- 172.21.15.60:0/3993512810 >> v2:172.21.15.105:3300/0 conn(0x7f766c15c6e0 0x7f766c06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.833+0000 7f767199a700 1 -- 172.21.15.60:0/3993512810 >> v1:172.21.15.60:6789/0 conn(0x7f766c0b24b0 legacy=0x7f766c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.833+0000 7f767199a700 1 -- 172.21.15.60:0/3993512810 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c06bea0 con 0x7f766c0b3050 2023-07-13T22:30:24.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.833+0000 7f7671199700 1 -- 172.21.15.60:0/3993512810 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253663027 0 0) 0x7f766c06b8e0 con 0x7f766c0b24b0 2023-07-13T22:30:24.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f767199a700 1 --2- 172.21.15.60:0/3993512810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0b3050 0x7f766c06afe0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f766c0ce0f0 tx=0x7f765c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ad5baf84091135 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:24.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f7671199700 1 -- 172.21.15.60:0/3993512810 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c011830 con 0x7f766c0b3050 2023-07-13T22:30:24.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f7671199700 1 -- 172.21.15.60:0/3993512810 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c011990 con 0x7f766c0b3050 2023-07-13T22:30:24.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f7671199700 1 -- 172.21.15.60:0/3993512810 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c011d20 con 0x7f766c0b3050 2023-07-13T22:30:24.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f76743ff700 1 -- 172.21.15.60:0/3993512810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0b3050 msgr2=0x7f766c06afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f76743ff700 1 --2- 172.21.15.60:0/3993512810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0b3050 0x7f766c06afe0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f766c0ce0f0 tx=0x7f765c009b20 comp rx=0 tx=0).stop 2023-07-13T22:30:24.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f76743ff700 1 -- 172.21.15.60:0/3993512810 shutdown_connections 2023-07-13T22:30:24.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f76743ff700 1 --2- 172.21.15.60:0/3993512810 >> v2:172.21.15.105:3300/0 conn(0x7f766c15c6e0 0x7f766c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f76743ff700 1 --2- 172.21.15.60:0/3993512810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0b3050 0x7f766c06afe0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.834+0000 7f76743ff700 1 -- 172.21.15.60:0/3993512810 >> 172.21.15.60:0/3993512810 conn(0x7f766c06e680 msgr2=0x7f766c06ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:24.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.835+0000 7f76743ff700 1 -- 172.21.15.60:0/3993512810 shutdown_connections 2023-07-13T22:30:24.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.835+0000 7f76743ff700 1 -- 172.21.15.60:0/3993512810 wait complete. 2023-07-13T22:30:24.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.836+0000 7f76743ff700 1 Processor -- start 2023-07-13T22:30:24.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.836+0000 7f76743ff700 1 -- start start 2023-07-13T22:30:24.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.837+0000 7f76743ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f766c0b24b0 0x7f766c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.837+0000 7f76743ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f766c0b3050 0x7f766c0b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.837+0000 7f76743ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c15c6e0 0x7f766c15e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.837+0000 7f76743ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c148710 con 0x7f766c15c6e0 2023-07-13T22:30:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.837+0000 7f76743ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0b3da0 con 0x7f766c0b3050 2023-07-13T22:30:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.837+0000 7f76743ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0b6620 con 0x7f766c0b24b0 2023-07-13T22:30:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767299c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c15c6e0 0x7f766c15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767299c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c15c6e0 0x7f766c15e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37598/0 (socket says 172.21.15.60:37598) 2023-07-13T22:30:24.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767299c700 1 -- 172.21.15.60:0/1004208935 learned_addr learned my addr 172.21.15.60:0/1004208935 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:24.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767219b700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f766c0b24b0 0x7f766c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767199a700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f766c0b3050 0x7f766c0b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767299c700 1 -- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f766c0b24b0 msgr2=0x7f766c15b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767299c700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f766c0b24b0 0x7f766c15b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767299c700 1 -- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f766c0b3050 msgr2=0x7f766c0b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:24.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767299c700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f766c0b3050 0x7f766c0b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:24.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767299c700 1 -- 172.21.15.60:0/1004208935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0094a0 con 0x7f766c15c6e0 2023-07-13T22:30:24.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767219b700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f766c0b24b0 0x7f766c15b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:24.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767199a700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f766c0b3050 0x7f766c0b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:24.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.838+0000 7f767299c700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c15c6e0 0x7f766c15e060 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7664002230 tx=0x7f766400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:24.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.839+0000 7f7662ffd700 1 -- 172.21.15.60:0/1004208935 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400ac20 con 0x7f766c15c6e0 2023-07-13T22:30:24.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.839+0000 7f7662ffd700 1 -- 172.21.15.60:0/1004208935 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400ada0 con 0x7f766c15c6e0 2023-07-13T22:30:24.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.839+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c15a1b0 con 0x7f766c15c6e0 2023-07-13T22:30:24.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.839+0000 7f7662ffd700 1 -- 172.21.15.60:0/1004208935 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664007310 con 0x7f766c15c6e0 2023-07-13T22:30:24.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.839+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c15a770 con 0x7f766c15c6e0 2023-07-13T22:30:24.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.840+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c04e280 con 0x7f766c15c6e0 2023-07-13T22:30:24.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.841+0000 7f7662ffd700 1 -- 172.21.15.60:0/1004208935 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7664007470 con 0x7f766c15c6e0 2023-07-13T22:30:24.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.842+0000 7f7662ffd700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f765006ace0 0x7f765006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:24.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.842+0000 7f7662ffd700 1 -- 172.21.15.60:0/1004208935 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76640890c0 con 0x7f766c15c6e0 2023-07-13T22:30:24.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.842+0000 7f767219b700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f765006ace0 0x7f765006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:24.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.842+0000 7f767219b700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f765006ace0 0x7f765006d190 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7668009b90 tx=0x7f7668008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:24.846+0000 7f7662ffd700 1 -- 172.21.15.60:0/1004208935 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f76640540d0 con 0x7f766c15c6e0 2023-07-13T22:30:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.099+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f766c073a50 con 0x7f766c15c6e0 2023-07-13T22:30:25.078 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:30:25.100+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:30:25.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.104+0000 7f7662ffd700 1 -- 172.21.15.60:0/1004208935 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7f76640576f0 con 0x7f766c15c6e0 2023-07-13T22:30:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f765006ace0 msgr2=0x7f765006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f765006ace0 0x7f765006d190 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7668009b90 tx=0x7f7668008040 comp rx=0 tx=0).stop 2023-07-13T22:30:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c15c6e0 msgr2=0x7f766c15e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c15c6e0 0x7f766c15e060 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7664002230 tx=0x7f766400a060 comp rx=0 tx=0).stop 2023-07-13T22:30:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 shutdown_connections 2023-07-13T22:30:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f765006ace0 0x7f765006d190 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c15c6e0 0x7f766c15e060 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f766c0b24b0 0x7f766c15b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 --2- 172.21.15.60:0/1004208935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f766c0b3050 0x7f766c0b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.107+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 >> 172.21.15.60:0/1004208935 conn(0x7f766c06e680 msgr2=0x7f766c0cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.108+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 shutdown_connections 2023-07-13T22:30:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.108+0000 7f76743ff700 1 -- 172.21.15.60:0/1004208935 wait complete. 2023-07-13T22:30:25.088 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:30:25.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:30:25.154 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:30:25.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.336+0000 7f7a02bff700 1 Processor -- start 2023-07-13T22:30:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.336+0000 7f7a02bff700 1 -- start start 2023-07-13T22:30:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.337+0000 7f7a02bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f79fc0cfd50 0x7f79fc0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.337+0000 7f7a02bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79fc0b1ac0 0x7f79fc0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a02bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc0d1870 con 0x7f79fc0b8200 2023-07-13T22:30:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a02bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0cf370 con 0x7f79fc0b1ac0 2023-07-13T22:30:25.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a02bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0d12b0 con 0x7f79fc0cfd50 2023-07-13T22:30:25.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a0099b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f79fc0b8200 0x7f79fc0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56760/0 (socket says 172.21.15.60:56760) 2023-07-13T22:30:25.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a0099b700 1 -- 172.21.15.60:0/2356257204 learned_addr learned my addr 172.21.15.60:0/2356257204 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:25.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a0119c700 1 --2- 172.21.15.60:0/2356257204 >> v2:172.21.15.60:3300/0 conn(0x7f79fc0b1ac0 0x7f79fc0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f79fbfff700 1 --2- 172.21.15.60:0/2356257204 >> v2:172.21.15.105:3300/0 conn(0x7f79fc0cfd50 0x7f79fc0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f79fb7fe700 1 -- 172.21.15.60:0/2356257204 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277805375 0 0) 0x7f79fc0d1870 con 0x7f79fc0b8200 2023-07-13T22:30:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f79fb7fe700 1 -- 172.21.15.60:0/2356257204 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79ec0035b0 con 0x7f79fc0b8200 2023-07-13T22:30:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a0119c700 1 -- 172.21.15.60:0/2356257204 >> v2:172.21.15.105:3300/0 conn(0x7f79fc0cfd50 msgr2=0x7f79fc0d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a0119c700 1 --2- 172.21.15.60:0/2356257204 >> v2:172.21.15.105:3300/0 conn(0x7f79fc0cfd50 0x7f79fc0d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a0119c700 1 -- 172.21.15.60:0/2356257204 >> v1:172.21.15.60:6789/0 conn(0x7f79fc0b8200 legacy=0x7f79fc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.338+0000 7f7a0119c700 1 -- 172.21.15.60:0/2356257204 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc0d1590 con 0x7f79fc0b1ac0 2023-07-13T22:30:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.339+0000 7f7a0119c700 1 --2- 172.21.15.60:0/2356257204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b1ac0 0x7f79fc0d2900 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0d07b0 tx=0x7f79f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fdb66f27b43c70 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.339+0000 7f79fb7fe700 1 -- 172.21.15.60:0/2356257204 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f400e070 con 0x7f79fc0b1ac0 2023-07-13T22:30:25.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.339+0000 7f79fb7fe700 1 -- 172.21.15.60:0/2356257204 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f400bb20 con 0x7f79fc0b1ac0 2023-07-13T22:30:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.339+0000 7f79fb7fe700 1 -- 172.21.15.60:0/2356257204 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4012430 con 0x7f79fc0b1ac0 2023-07-13T22:30:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.339+0000 7f7a02bff700 1 -- 172.21.15.60:0/2356257204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b1ac0 msgr2=0x7f79fc0d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.339+0000 7f7a02bff700 1 --2- 172.21.15.60:0/2356257204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b1ac0 0x7f79fc0d2900 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0d07b0 tx=0x7f79f400b330 comp rx=0 tx=0).stop 2023-07-13T22:30:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.339+0000 7f7a02bff700 1 -- 172.21.15.60:0/2356257204 shutdown_connections 2023-07-13T22:30:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.340+0000 7f7a02bff700 1 --2- 172.21.15.60:0/2356257204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b1ac0 0x7f79fc0d2900 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.340+0000 7f7a02bff700 1 --2- 172.21.15.60:0/2356257204 >> v2:172.21.15.105:3300/0 conn(0x7f79fc0cfd50 0x7f79fc0d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.340+0000 7f7a02bff700 1 -- 172.21.15.60:0/2356257204 >> 172.21.15.60:0/2356257204 conn(0x7f79fc06b2f0 msgr2=0x7f79fc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.340+0000 7f7a02bff700 1 -- 172.21.15.60:0/2356257204 shutdown_connections 2023-07-13T22:30:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.340+0000 7f7a02bff700 1 -- 172.21.15.60:0/2356257204 wait complete. 2023-07-13T22:30:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.341+0000 7f7a02bff700 1 Processor -- start 2023-07-13T22:30:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.341+0000 7f7a02bff700 1 -- start start 2023-07-13T22:30:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.342+0000 7f7a02bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79fc0b1ac0 0x7f79fc1f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.342+0000 7f7a02bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79fc0b8200 0x7f79fc0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.342+0000 7f7a02bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0cfd50 0x7f79fc1e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.342+0000 7f7a02bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc1480b0 con 0x7f79fc0cfd50 2023-07-13T22:30:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.342+0000 7f7a02bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc148390 con 0x7f79fc0b1ac0 2023-07-13T22:30:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.342+0000 7f7a02bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0cbca0 con 0x7f79fc0b8200 2023-07-13T22:30:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0119c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0cfd50 0x7f79fc1e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:25.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0119c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0cfd50 0x7f79fc1e3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38362/0 (socket says 172.21.15.60:38362) 2023-07-13T22:30:25.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0119c700 1 -- 172.21.15.60:0/1308462503 learned_addr learned my addr 172.21.15.60:0/1308462503 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:25.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0099b700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79fc0b1ac0 0x7f79fc1f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:25.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f79fbfff700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79fc0b8200 0x7f79fc0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:25.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0119c700 1 -- 172.21.15.60:0/1308462503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79fc0b8200 msgr2=0x7f79fc0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:25.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0119c700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79fc0b8200 0x7f79fc0cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:25.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0119c700 1 -- 172.21.15.60:0/1308462503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79fc0b1ac0 msgr2=0x7f79fc1f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:25.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0119c700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79fc0b1ac0 0x7f79fc1f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:25.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0119c700 1 -- 172.21.15.60:0/1308462503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f4009a70 con 0x7f79fc0cfd50 2023-07-13T22:30:25.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.343+0000 7f7a0119c700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0cfd50 0x7f79fc1e3de0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f79f400b680 tx=0x7f79f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:25.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.344+0000 7f79f97fa700 1 -- 172.21.15.60:0/1308462503 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4012430 con 0x7f79fc0cfd50 2023-07-13T22:30:25.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.344+0000 7f79f97fa700 1 -- 172.21.15.60:0/1308462503 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f4020070 con 0x7f79fc0cfd50 2023-07-13T22:30:25.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.344+0000 7f79f97fa700 1 -- 172.21.15.60:0/1308462503 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f40098f0 con 0x7f79fc0cfd50 2023-07-13T22:30:25.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.344+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc0cbf20 con 0x7f79fc0cfd50 2023-07-13T22:30:25.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.344+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc0cc4e0 con 0x7f79fc0cfd50 2023-07-13T22:30:25.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.345+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc04e210 con 0x7f79fc0cfd50 2023-07-13T22:30:25.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.346+0000 7f79f97fa700 1 -- 172.21.15.60:0/1308462503 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79f401c400 con 0x7f79fc0cfd50 2023-07-13T22:30:25.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.347+0000 7f79f97fa700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79e006ace0 0x7f79e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:25.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.347+0000 7f79f97fa700 1 -- 172.21.15.60:0/1308462503 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f79f4022070 con 0x7f79fc0cfd50 2023-07-13T22:30:25.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.347+0000 7f7a0099b700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79e006ace0 0x7f79e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:25.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.348+0000 7f7a0099b700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79e006ace0 0x7f79e006d190 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f79f0007fd0 tx=0x7f79f0007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:25.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.351+0000 7f79f97fa700 1 -- 172.21.15.60:0/1308462503 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f79f4059a90 con 0x7f79fc0cfd50 2023-07-13T22:30:25.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:25.603+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f79fc1484d0 con 0x7f79fc0cfd50 2023-07-13T22:30:26.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.109+0000 7f79f97fa700 1 -- 172.21.15.60:0/1308462503 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7f79f4059620 con 0x7f79fc0cfd50 2023-07-13T22:30:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79e006ace0 msgr2=0x7f79e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79e006ace0 0x7f79e006d190 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f79f0007fd0 tx=0x7f79f0007dc0 comp rx=0 tx=0).stop 2023-07-13T22:30:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0cfd50 msgr2=0x7f79fc1e3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0cfd50 0x7f79fc1e3de0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f79f400b680 tx=0x7f79f4004ad0 comp rx=0 tx=0).stop 2023-07-13T22:30:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 shutdown_connections 2023-07-13T22:30:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79e006ace0 0x7f79e006d190 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0cfd50 0x7f79fc1e3de0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79fc0b1ac0 0x7f79fc1f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 --2- 172.21.15.60:0/1308462503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79fc0b8200 0x7f79fc0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 >> 172.21.15.60:0/1308462503 conn(0x7f79fc06b2f0 msgr2=0x7f79fc0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 shutdown_connections 2023-07-13T22:30:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.112+0000 7f7a02bff700 1 -- 172.21.15.60:0/1308462503 wait complete. 2023-07-13T22:30:26.111 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:30:26.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.343+0000 7f9936dff700 1 Processor -- start 2023-07-13T22:30:26.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.343+0000 7f9936dff700 1 -- start start 2023-07-13T22:30:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f9936dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f99300ce2c0 0x7f9930164040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f9936dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993015cab0 con 0x7f99300c8da0 2023-07-13T22:30:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f9936dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99300ca820 con 0x7f993015c6e0 2023-07-13T22:30:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f9936dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99300cafd0 con 0x7f99300ce2c0 2023-07-13T22:30:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f992ffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f99300c8da0 0x7f99300cdbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56770/0 (socket says 172.21.15.60:56770) 2023-07-13T22:30:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f992ffff700 1 -- 172.21.15.60:0/1751445220 learned_addr learned my addr 172.21.15.60:0/1751445220 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f993539c700 1 --2- 172.21.15.60:0/1751445220 >> v2:172.21.15.60:3300/0 conn(0x7f99300ce2c0 0x7f9930164040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f992f7fe700 1 -- 172.21.15.60:0/1751445220 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943796029 0 0) 0x7f993015cab0 con 0x7f99300c8da0 2023-07-13T22:30:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f992f7fe700 1 -- 172.21.15.60:0/1751445220 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99200057c0 con 0x7f99300c8da0 2023-07-13T22:30:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f993539c700 1 -- 172.21.15.60:0/1751445220 >> v1:172.21.15.105:6789/0 conn(0x7f993015c6e0 legacy=0x7f993015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f993539c700 1 -- 172.21.15.60:0/1751445220 >> v1:172.21.15.60:6789/0 conn(0x7f99300c8da0 legacy=0x7f99300cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.345+0000 7f993539c700 1 -- 172.21.15.60:0/1751445220 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300caa50 con 0x7f99300ce2c0 2023-07-13T22:30:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.346+0000 7f993539c700 1 --2- 172.21.15.60:0/1751445220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 0x7f9930164040 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f9928009d90 tx=0x7f992800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8cabc7489338a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.346+0000 7f992f7fe700 1 -- 172.21.15.60:0/1751445220 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800e070 con 0x7f99300ce2c0 2023-07-13T22:30:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.346+0000 7f992f7fe700 1 -- 172.21.15.60:0/1751445220 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992800bb20 con 0x7f99300ce2c0 2023-07-13T22:30:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.346+0000 7f992f7fe700 1 -- 172.21.15.60:0/1751445220 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800be90 con 0x7f99300ce2c0 2023-07-13T22:30:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.346+0000 7f9936dff700 1 -- 172.21.15.60:0/1751445220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 msgr2=0x7f9930164040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.346+0000 7f9936dff700 1 --2- 172.21.15.60:0/1751445220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 0x7f9930164040 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f9928009d90 tx=0x7f992800b330 comp rx=0 tx=0).stop 2023-07-13T22:30:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.347+0000 7f9936dff700 1 -- 172.21.15.60:0/1751445220 shutdown_connections 2023-07-13T22:30:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.347+0000 7f9936dff700 1 --2- 172.21.15.60:0/1751445220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 0x7f9930164040 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.347+0000 7f9936dff700 1 -- 172.21.15.60:0/1751445220 >> 172.21.15.60:0/1751445220 conn(0x7f99300cf3b0 msgr2=0x7f99300cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.347+0000 7f9936dff700 1 -- 172.21.15.60:0/1751445220 shutdown_connections 2023-07-13T22:30:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.347+0000 7f9936dff700 1 -- 172.21.15.60:0/1751445220 wait complete. 2023-07-13T22:30:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.348+0000 7f9936dff700 1 Processor -- start 2023-07-13T22:30:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.349+0000 7f9936dff700 1 -- start start 2023-07-13T22:30:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f9936dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300c8da0 0x7f99301f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f9936dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 0x7f993006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f9936dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015c6e0 0x7f993006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f9936dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300c76b0 con 0x7f99300ce2c0 2023-07-13T22:30:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f9936dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300c7990 con 0x7f99300c8da0 2023-07-13T22:30:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f9936dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99300c7c70 con 0x7f993015c6e0 2023-07-13T22:30:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f992ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 0x7f993006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f992ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 0x7f993006d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38378/0 (socket says 172.21.15.60:38378) 2023-07-13T22:30:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f992ffff700 1 -- 172.21.15.60:0/2023319721 learned_addr learned my addr 172.21.15.60:0/2023319721 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f9934b9b700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300c8da0 0x7f99301f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f993539c700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015c6e0 0x7f993006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f992ffff700 1 -- 172.21.15.60:0/2023319721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015c6e0 msgr2=0x7f993006d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f992ffff700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015c6e0 0x7f993006d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f992ffff700 1 -- 172.21.15.60:0/2023319721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300c8da0 msgr2=0x7f99301f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f992ffff700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300c8da0 0x7f99301f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f992ffff700 1 -- 172.21.15.60:0/2023319721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928009a70 con 0x7f99300ce2c0 2023-07-13T22:30:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.350+0000 7f993539c700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015c6e0 0x7f993006d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.351+0000 7f992ffff700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 0x7f993006d370 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c700 tx=0x7f991c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.351+0000 7f992d7fa700 1 -- 172.21.15.60:0/2023319721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c005d70 con 0x7f99300ce2c0 2023-07-13T22:30:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.351+0000 7f992d7fa700 1 -- 172.21.15.60:0/2023319721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c0070d0 con 0x7f99300ce2c0 2023-07-13T22:30:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.351+0000 7f992d7fa700 1 -- 172.21.15.60:0/2023319721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c007460 con 0x7f99300ce2c0 2023-07-13T22:30:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.351+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99300c7f50 con 0x7f99300ce2c0 2023-07-13T22:30:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.351+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99301f23a0 con 0x7f99300ce2c0 2023-07-13T22:30:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.352+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99300673f0 con 0x7f99300ce2c0 2023-07-13T22:30:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.353+0000 7f992d7fa700 1 -- 172.21.15.60:0/2023319721 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f991c0075c0 con 0x7f99300ce2c0 2023-07-13T22:30:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.354+0000 7f992d7fa700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991406ace0 0x7f991406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.354+0000 7f992d7fa700 1 -- 172.21.15.60:0/2023319721 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f991c088b50 con 0x7f99300ce2c0 2023-07-13T22:30:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.355+0000 7f9934b9b700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991406ace0 0x7f991406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.355+0000 7f9934b9b700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991406ace0 0x7f991406d190 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9924000f60 tx=0x7f9924009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.359+0000 7f992d7fa700 1 -- 172.21.15.60:0/2023319721 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f99300673f0 con 0x7f99300ce2c0 2023-07-13T22:30:26.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:26.565+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99300673f0 con 0x7f99300ce2c0 2023-07-13T22:30:27.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.112+0000 7f992d7fa700 1 -- 172.21.15.60:0/2023319721 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 181+0+0 (secure 0 0 0) 0x7f99300673f0 con 0x7f99300ce2c0 2023-07-13T22:30:27.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.116+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991406ace0 msgr2=0x7f991406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:27.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.116+0000 7f9936dff700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991406ace0 0x7f991406d190 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9924000f60 tx=0x7f9924009040 comp rx=0 tx=0).stop 2023-07-13T22:30:27.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.116+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 msgr2=0x7f993006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:27.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.116+0000 7f9936dff700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 0x7f993006d370 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c700 tx=0x7f991c00cac0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.116+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 shutdown_connections 2023-07-13T22:30:27.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.116+0000 7f9936dff700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f991406ace0 0x7f991406d190 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.116+0000 7f9936dff700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f993015c6e0 0x7f993006d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.116+0000 7f9936dff700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99300c8da0 0x7f99301f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.116+0000 7f9936dff700 1 --2- 172.21.15.60:0/2023319721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99300ce2c0 0x7f993006d370 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.117+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 >> 172.21.15.60:0/2023319721 conn(0x7f99300cf3b0 msgr2=0x7f9930159470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:27.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.117+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 shutdown_connections 2023-07-13T22:30:27.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.117+0000 7f9936dff700 1 -- 172.21.15.60:0/2023319721 wait complete. 2023-07-13T22:30:27.099 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:30:27.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:30:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.350+0000 7feded5ff700 1 Processor -- start 2023-07-13T22:30:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.350+0000 7feded5ff700 1 -- start start 2023-07-13T22:30:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.351+0000 7feded5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80cd6d0 0x7fede80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.351+0000 7feded5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fede80d37c0 0x7fede80d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.351+0000 7feded5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede80cdb20 con 0x7fede8159ea0 2023-07-13T22:30:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.351+0000 7feded5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede80d2e60 con 0x7fede80d37c0 2023-07-13T22:30:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.351+0000 7feded5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede80d2b50 con 0x7fede80cd6d0 2023-07-13T22:30:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.351+0000 7fede6ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fede8159ea0 0x7fede815c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56786/0 (socket says 172.21.15.60:56786) 2023-07-13T22:30:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.351+0000 7fede6ffd700 1 -- 172.21.15.60:0/1203549645 learned_addr learned my addr 172.21.15.60:0/1203549645 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.351+0000 7fede77fe700 1 --2- 172.21.15.60:0/1203549645 >> v2:172.21.15.60:3300/0 conn(0x7fede80d37c0 0x7fede80d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7feddffff700 1 --2- 172.21.15.60:0/1203549645 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80cd6d0 0x7fede80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede67fc700 1 -- 172.21.15.60:0/1203549645 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579756212 0 0) 0x7fede80cdb20 con 0x7fede8159ea0 2023-07-13T22:30:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede67fc700 1 -- 172.21.15.60:0/1203549645 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedd40035b0 con 0x7fede8159ea0 2023-07-13T22:30:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede77fe700 1 -- 172.21.15.60:0/1203549645 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80cd6d0 msgr2=0x7fede80d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede77fe700 1 --2- 172.21.15.60:0/1203549645 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80cd6d0 0x7fede80d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede77fe700 1 -- 172.21.15.60:0/1203549645 >> v1:172.21.15.60:6789/0 conn(0x7fede8159ea0 legacy=0x7fede815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede77fe700 1 -- 172.21.15.60:0/1203549645 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede815d200 con 0x7fede80d37c0 2023-07-13T22:30:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede77fe700 1 --2- 172.21.15.60:0/1203549645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80d37c0 0x7fede80d0890 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fede0009d90 tx=0x7fede000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca826e55484147b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede67fc700 1 -- 172.21.15.60:0/1203549645 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede000e070 con 0x7fede80d37c0 2023-07-13T22:30:27.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede67fc700 1 -- 172.21.15.60:0/1203549645 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede000bb20 con 0x7fede80d37c0 2023-07-13T22:30:27.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.352+0000 7fede67fc700 1 -- 172.21.15.60:0/1203549645 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0012430 con 0x7fede80d37c0 2023-07-13T22:30:27.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.353+0000 7feded5ff700 1 -- 172.21.15.60:0/1203549645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80d37c0 msgr2=0x7fede80d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:27.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.353+0000 7feded5ff700 1 --2- 172.21.15.60:0/1203549645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80d37c0 0x7fede80d0890 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fede0009d90 tx=0x7fede000b330 comp rx=0 tx=0).stop 2023-07-13T22:30:27.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.353+0000 7feded5ff700 1 -- 172.21.15.60:0/1203549645 shutdown_connections 2023-07-13T22:30:27.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.353+0000 7feded5ff700 1 --2- 172.21.15.60:0/1203549645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80d37c0 0x7fede80d0890 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.353+0000 7feded5ff700 1 --2- 172.21.15.60:0/1203549645 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80cd6d0 0x7fede80d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.353+0000 7feded5ff700 1 -- 172.21.15.60:0/1203549645 >> 172.21.15.60:0/1203549645 conn(0x7fede8071600 msgr2=0x7fede80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:27.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.353+0000 7feded5ff700 1 -- 172.21.15.60:0/1203549645 shutdown_connections 2023-07-13T22:30:27.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.353+0000 7feded5ff700 1 -- 172.21.15.60:0/1203549645 wait complete. 2023-07-13T22:30:27.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.355+0000 7feded5ff700 1 Processor -- start 2023-07-13T22:30:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.355+0000 7feded5ff700 1 -- start start 2023-07-13T22:30:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7feded5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80cd6d0 0x7fede80b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7feded5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80d37c0 0x7fede8156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7feded5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fede8159ea0 0x7fede80b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7feded5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede80b3c60 con 0x7fede80cd6d0 2023-07-13T22:30:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7feded5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede81568d0 con 0x7fede8159ea0 2023-07-13T22:30:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7feded5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede80b2570 con 0x7fede80d37c0 2023-07-13T22:30:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7fede6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80cd6d0 0x7fede80b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7fede6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80cd6d0 0x7fede80b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38408/0 (socket says 172.21.15.60:38408) 2023-07-13T22:30:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7fede6ffd700 1 -- 172.21.15.60:0/2192853314 learned_addr learned my addr 172.21.15.60:0/2192853314 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.356+0000 7feddffff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80d37c0 0x7fede8156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7fede6ffd700 1 -- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80d37c0 msgr2=0x7fede8156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7fede6ffd700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80d37c0 0x7fede8156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7fede77fe700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fede8159ea0 0x7fede80b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7fede6ffd700 1 -- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fede8159ea0 msgr2=0x7fede80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7fede6ffd700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fede8159ea0 0x7fede80b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7fede6ffd700 1 -- 172.21.15.60:0/2192853314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0009a70 con 0x7fede80cd6d0 2023-07-13T22:30:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7feddffff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80d37c0 0x7fede8156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7fede77fe700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fede8159ea0 0x7fede80b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7fede6ffd700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80cd6d0 0x7fede80b1f90 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fedd800c700 tx=0x7fedd800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7feddf7fe700 1 -- 172.21.15.60:0/2192853314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8005d70 con 0x7fede80cd6d0 2023-07-13T22:30:27.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7feddf7fe700 1 -- 172.21.15.60:0/2192853314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd80070d0 con 0x7fede80cd6d0 2023-07-13T22:30:27.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7feddf7fe700 1 -- 172.21.15.60:0/2192853314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8007460 con 0x7fede80cd6d0 2023-07-13T22:30:27.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede815afa0 con 0x7fede80cd6d0 2023-07-13T22:30:27.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.357+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede815b560 con 0x7fede80cd6d0 2023-07-13T22:30:27.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.358+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede8067a70 con 0x7fede80cd6d0 2023-07-13T22:30:27.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.359+0000 7feddf7fe700 1 -- 172.21.15.60:0/2192853314 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fedd80075c0 con 0x7fede80cd6d0 2023-07-13T22:30:27.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.360+0000 7feddf7fe700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedc806ace0 0x7fedc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:27.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.360+0000 7feddf7fe700 1 -- 172.21.15.60:0/2192853314 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fedd80889c0 con 0x7fede80cd6d0 2023-07-13T22:30:27.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.360+0000 7feddffff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedc806ace0 0x7fedc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:27.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.361+0000 7feddffff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedc806ace0 0x7fedc806d190 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fede80c6340 tx=0x7fedd0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:27.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.364+0000 7feddf7fe700 1 -- 172.21.15.60:0/2192853314 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fedd8053b40 con 0x7fede80cd6d0 2023-07-13T22:30:27.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:30:27.516 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:30:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:27.572+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fede804e210 con 0x7fede80cd6d0 2023-07-13T22:30:28.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.118+0000 7feddf7fe700 1 -- 172.21.15.60:0/2192853314 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 169+0+0 (secure 0 0 0) 0x7fedd8057160 con 0x7fede80cd6d0 2023-07-13T22:30:28.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.123+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedc806ace0 msgr2=0x7fedc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.123+0000 7feded5ff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedc806ace0 0x7fedc806d190 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fede80c6340 tx=0x7fedd0008040 comp rx=0 tx=0).stop 2023-07-13T22:30:28.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.123+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80cd6d0 msgr2=0x7fede80b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.123+0000 7feded5ff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80cd6d0 0x7fede80b1f90 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fedd800c700 tx=0x7fedd800cac0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.124+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 shutdown_connections 2023-07-13T22:30:28.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.124+0000 7feded5ff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fedc806ace0 0x7fedc806d190 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.124+0000 7feded5ff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fede80cd6d0 0x7fede80b1f90 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.124+0000 7feded5ff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fede8159ea0 0x7fede80b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.124+0000 7feded5ff700 1 --2- 172.21.15.60:0/2192853314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fede80d37c0 0x7fede8156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.124+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 >> 172.21.15.60:0/2192853314 conn(0x7fede8071600 msgr2=0x7fede815f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:28.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.125+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 shutdown_connections 2023-07-13T22:30:28.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.125+0000 7feded5ff700 1 -- 172.21.15.60:0/2192853314 wait complete. 2023-07-13T22:30:28.105 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:30:28.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-07-13T22:30:28.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.353+0000 7f39be5ff700 1 Processor -- start 2023-07-13T22:30:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.353+0000 7f39be5ff700 1 -- start start 2023-07-13T22:30:28.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.354+0000 7f39be5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 0x7f39b8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.354+0000 7f39be5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39b80caf80 0x7f39b80ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.354+0000 7f39be5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f39b806c8f0 0x7f39b815fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.354+0000 7f39be5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b806b9f0 con 0x7f39b80caf80 2023-07-13T22:30:28.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.354+0000 7f39be5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b806c750 con 0x7f39b806c8f0 2023-07-13T22:30:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.354+0000 7f39be5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b806bf40 con 0x7f39b815c460 2023-07-13T22:30:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.354+0000 7f39b77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39b80caf80 0x7f39b80ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39b77fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39b80caf80 0x7f39b80ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38410/0 (socket says 172.21.15.60:38410) 2023-07-13T22:30:28.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39b77fe700 1 -- 172.21.15.60:0/1214715000 learned_addr learned my addr 172.21.15.60:0/1214715000 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:28.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39b7fff700 1 --2- 172.21.15.60:0/1214715000 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 0x7f39b8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39bcb9c700 1 --2- 172.21.15.60:0/1214715000 >> v2:172.21.15.105:3300/0 conn(0x7f39b806c8f0 0x7f39b815fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39b77fe700 1 -- 172.21.15.60:0/1214715000 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 msgr2=0x7f39b8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:28.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39b77fe700 1 --2- 172.21.15.60:0/1214715000 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 0x7f39b8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39b77fe700 1 -- 172.21.15.60:0/1214715000 >> v2:172.21.15.105:3300/0 conn(0x7f39b806c8f0 msgr2=0x7f39b815fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:28.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39b77fe700 1 --2- 172.21.15.60:0/1214715000 >> v2:172.21.15.105:3300/0 conn(0x7f39b806c8f0 0x7f39b815fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39b77fe700 1 -- 172.21.15.60:0/1214715000 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b80cb3f0 con 0x7f39b80caf80 2023-07-13T22:30:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.355+0000 7f39b77fe700 1 --2- 172.21.15.60:0/1214715000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b80caf80 0x7f39b80ce770 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f39b80c3fb0 tx=0x7f39ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff66ba966530cb1e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39b6ffd700 1 -- 172.21.15.60:0/1214715000 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00b870 con 0x7f39b80caf80 2023-07-13T22:30:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39b6ffd700 1 -- 172.21.15.60:0/1214715000 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac00b9d0 con 0x7f39b80caf80 2023-07-13T22:30:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39b6ffd700 1 -- 172.21.15.60:0/1214715000 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00bd60 con 0x7f39b80caf80 2023-07-13T22:30:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39be5ff700 1 -- 172.21.15.60:0/1214715000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b80caf80 msgr2=0x7f39b80ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39be5ff700 1 --2- 172.21.15.60:0/1214715000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b80caf80 0x7f39b80ce770 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f39b80c3fb0 tx=0x7f39ac009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39be5ff700 1 -- 172.21.15.60:0/1214715000 shutdown_connections 2023-07-13T22:30:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39be5ff700 1 --2- 172.21.15.60:0/1214715000 >> v2:172.21.15.105:3300/0 conn(0x7f39b806c8f0 0x7f39b815fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39be5ff700 1 --2- 172.21.15.60:0/1214715000 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 0x7f39b8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39be5ff700 1 --2- 172.21.15.60:0/1214715000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b80caf80 0x7f39b80ce770 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39be5ff700 1 -- 172.21.15.60:0/1214715000 >> 172.21.15.60:0/1214715000 conn(0x7f39b806e580 msgr2=0x7f39b806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39be5ff700 1 -- 172.21.15.60:0/1214715000 shutdown_connections 2023-07-13T22:30:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.356+0000 7f39be5ff700 1 -- 172.21.15.60:0/1214715000 wait complete. 2023-07-13T22:30:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.358+0000 7f39be5ff700 1 Processor -- start 2023-07-13T22:30:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.358+0000 7f39be5ff700 1 -- start start 2023-07-13T22:30:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.359+0000 7f39be5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b806c8f0 0x7f39b80b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.359+0000 7f39be5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b80caf80 0x7f39b8156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.359+0000 7f39be5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 0x7f39b815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.359+0000 7f39be5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b8156010 con 0x7f39b806c8f0 2023-07-13T22:30:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.359+0000 7f39be5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b80b65f0 con 0x7f39b80caf80 2023-07-13T22:30:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.359+0000 7f39be5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b81f5c80 con 0x7f39b815c460 2023-07-13T22:30:28.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.359+0000 7f39b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b806c8f0 0x7f39b80b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.359+0000 7f39b7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b806c8f0 0x7f39b80b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38426/0 (socket says 172.21.15.60:38426) 2023-07-13T22:30:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.359+0000 7f39b7fff700 1 -- 172.21.15.60:0/3090076817 learned_addr learned my addr 172.21.15.60:0/3090076817 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b77fe700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b80caf80 0x7f39b8156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39bcb9c700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 0x7f39b815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b7fff700 1 -- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 msgr2=0x7f39b815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b7fff700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 0x7f39b815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b7fff700 1 -- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b80caf80 msgr2=0x7f39b8156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b7fff700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b80caf80 0x7f39b8156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b7fff700 1 -- 172.21.15.60:0/3090076817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac009550 con 0x7f39b806c8f0 2023-07-13T22:30:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39bcb9c700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 0x7f39b815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b77fe700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b80caf80 0x7f39b8156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b7fff700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b806c8f0 0x7f39b80b6010 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e810 tx=0x7f39a800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b4ff9700 1 -- 172.21.15.60:0/3090076817 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8013610 con 0x7f39b806c8f0 2023-07-13T22:30:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b4ff9700 1 -- 172.21.15.60:0/3090076817 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a8013770 con 0x7f39b806c8f0 2023-07-13T22:30:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39b4ff9700 1 -- 172.21.15.60:0/3090076817 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8013b20 con 0x7f39b806c8f0 2023-07-13T22:30:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.360+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b81f5f60 con 0x7f39b806c8f0 2023-07-13T22:30:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.361+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b81f6760 con 0x7f39b806c8f0 2023-07-13T22:30:28.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.362+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b81f65b0 con 0x7f39b806c8f0 2023-07-13T22:30:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.362+0000 7f39b4ff9700 1 -- 172.21.15.60:0/3090076817 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39a8013c80 con 0x7f39b806c8f0 2023-07-13T22:30:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.363+0000 7f39b4ff9700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f39a406ace0 0x7f39a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.363+0000 7f39b4ff9700 1 -- 172.21.15.60:0/3090076817 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f39a8088e10 con 0x7f39b806c8f0 2023-07-13T22:30:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.364+0000 7f39b77fe700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f39a406ace0 0x7f39a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.364+0000 7f39b77fe700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f39a406ace0 0x7f39a406d190 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f39ac000c00 tx=0x7f39ac008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.367+0000 7f39b4ff9700 1 -- 172.21.15.60:0/3090076817 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f39a808d070 con 0x7f39b806c8f0 2023-07-13T22:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.584+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7f39b804e210 con 0x7f39b806c8f0 2023-07-13T22:30:28.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.588+0000 7f39b4ff9700 1 -- 172.21.15.60:0/3090076817 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v49) v1 ==== 102+0+0 (secure 0 0 0) 0x7f39a8054150 con 0x7f39b806c8f0 2023-07-13T22:30:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f39a406ace0 msgr2=0x7f39a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f39a406ace0 0x7f39a406d190 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f39ac000c00 tx=0x7f39ac008030 comp rx=0 tx=0).stop 2023-07-13T22:30:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b806c8f0 msgr2=0x7f39b80b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b806c8f0 0x7f39b80b6010 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e810 tx=0x7f39a800ebd0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 shutdown_connections 2023-07-13T22:30:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f39a406ace0 0x7f39a406d190 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b806c8f0 0x7f39b80b6010 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39b815c460 0x7f39b815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 --2- 172.21.15.60:0/3090076817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39b80caf80 0x7f39b8156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 >> 172.21.15.60:0/3090076817 conn(0x7f39b806e580 msgr2=0x7f39b80b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 shutdown_connections 2023-07-13T22:30:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.591+0000 7f39be5ff700 1 -- 172.21.15.60:0/3090076817 wait complete. 2023-07-13T22:30:28.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-13T22:30:28.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.816+0000 7fe1f4dff700 1 Processor -- start 2023-07-13T22:30:28.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.816+0000 7fe1f4dff700 1 -- start start 2023-07-13T22:30:28.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.817+0000 7fe1f4dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe1f00b8200 0x7fe1f00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.817+0000 7fe1f4dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f00d1870 con 0x7fe1f00cede0 2023-07-13T22:30:28.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.817+0000 7fe1f4dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f0160d80 con 0x7fe1f00b1ac0 2023-07-13T22:30:28.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.817+0000 7fe1f4dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f00cf370 con 0x7fe1f00b8200 2023-07-13T22:30:28.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.817+0000 7fe1eed9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe1f00cede0 0x7fe1f0160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56802/0 (socket says 172.21.15.60:56802) 2023-07-13T22:30:28.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.817+0000 7fe1eed9d700 1 -- 172.21.15.60:0/2666647797 learned_addr learned my addr 172.21.15.60:0/2666647797 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:28.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.817+0000 7fe1ee59c700 1 --2- 172.21.15.60:0/2666647797 >> v2:172.21.15.105:3300/0 conn(0x7fe1f00b8200 0x7fe1f00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.818+0000 7fe1edd9b700 1 -- 172.21.15.60:0/2666647797 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982585835 0 0) 0x7fe1f00d1870 con 0x7fe1f00cede0 2023-07-13T22:30:28.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.818+0000 7fe1edd9b700 1 -- 172.21.15.60:0/2666647797 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1dc0035b0 con 0x7fe1f00cede0 2023-07-13T22:30:28.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.818+0000 7fe1ee59c700 1 -- 172.21.15.60:0/2666647797 >> v1:172.21.15.105:6789/0 conn(0x7fe1f00b1ac0 legacy=0x7fe1f015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.818+0000 7fe1edd9b700 1 -- 172.21.15.60:0/2666647797 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442018788 0 0) 0x7fe1f0160d80 con 0x7fe1f00b1ac0 2023-07-13T22:30:28.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.818+0000 7fe1ee59c700 1 -- 172.21.15.60:0/2666647797 >> v1:172.21.15.60:6789/0 conn(0x7fe1f00cede0 legacy=0x7fe1f0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.818+0000 7fe1ee59c700 1 -- 172.21.15.60:0/2666647797 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f00d12b0 con 0x7fe1f00b8200 2023-07-13T22:30:28.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.818+0000 7fe1ee59c700 1 --2- 172.21.15.60:0/2666647797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00b8200 0x7fe1f00cf640 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009840 tx=0x7fe1e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4314061fa2ed2f02 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:28.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1edd9b700 1 -- 172.21.15.60:0/2666647797 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0011830 con 0x7fe1f00b8200 2023-07-13T22:30:28.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1edd9b700 1 -- 172.21.15.60:0/2666647797 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0011990 con 0x7fe1f00b8200 2023-07-13T22:30:28.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1edd9b700 1 -- 172.21.15.60:0/2666647797 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0011d20 con 0x7fe1f00b8200 2023-07-13T22:30:28.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1f4dff700 1 -- 172.21.15.60:0/2666647797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00b8200 msgr2=0x7fe1f00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1f4dff700 1 --2- 172.21.15.60:0/2666647797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00b8200 0x7fe1f00cf640 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009840 tx=0x7fe1e0009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:28.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1f4dff700 1 -- 172.21.15.60:0/2666647797 shutdown_connections 2023-07-13T22:30:28.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1f4dff700 1 --2- 172.21.15.60:0/2666647797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00b8200 0x7fe1f00cf640 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1f4dff700 1 -- 172.21.15.60:0/2666647797 >> 172.21.15.60:0/2666647797 conn(0x7fe1f006b2f0 msgr2=0x7fe1f006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:28.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1f4dff700 1 -- 172.21.15.60:0/2666647797 shutdown_connections 2023-07-13T22:30:28.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.819+0000 7fe1f4dff700 1 -- 172.21.15.60:0/2666647797 wait complete. 2023-07-13T22:30:28.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.821+0000 7fe1f4dff700 1 Processor -- start 2023-07-13T22:30:28.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.821+0000 7fe1f4dff700 1 -- start start 2023-07-13T22:30:28.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1f4dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00b1ac0 0x7fe1f015f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1f4dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f00b8200 0x7fe1f00c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1f4dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f00cede0 0x7fe1f00c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1f4dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00c6f50 con 0x7fe1f00cede0 2023-07-13T22:30:28.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1f4dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00c9a00 con 0x7fe1f00b1ac0 2023-07-13T22:30:28.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1f4dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f015e200 con 0x7fe1f00b8200 2023-07-13T22:30:28.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1eed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f00cede0 0x7fe1f00c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1eed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f00cede0 0x7fe1f00c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38432/0 (socket says 172.21.15.60:38432) 2023-07-13T22:30:28.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1eed9d700 1 -- 172.21.15.60:0/1119272815 learned_addr learned my addr 172.21.15.60:0/1119272815 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:28.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1ee59c700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00b1ac0 0x7fe1f015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.822+0000 7fe1e7fff700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f00b8200 0x7fe1f00c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1eed9d700 1 -- 172.21.15.60:0/1119272815 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f00b8200 msgr2=0x7fe1f00c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1eed9d700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f00b8200 0x7fe1f00c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1eed9d700 1 -- 172.21.15.60:0/1119272815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00b1ac0 msgr2=0x7fe1f015f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:28.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1eed9d700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00b1ac0 0x7fe1f015f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:28.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1eed9d700 1 -- 172.21.15.60:0/1119272815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e00094a0 con 0x7fe1f00cede0 2023-07-13T22:30:28.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1e7fff700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f00b8200 0x7fe1f00c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:28.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1eed9d700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f00cede0 0x7fe1f00c94c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800caa0 tx=0x7fe1e800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:28.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1e6ffd700 1 -- 172.21.15.60:0/1119272815 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800a9e0 con 0x7fe1f00cede0 2023-07-13T22:30:28.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1e6ffd700 1 -- 172.21.15.60:0/1119272815 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e800ab40 con 0x7fe1f00cede0 2023-07-13T22:30:28.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1e6ffd700 1 -- 172.21.15.60:0/1119272815 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8012400 con 0x7fe1f00cede0 2023-07-13T22:30:28.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.823+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f015df20 con 0x7fe1f00cede0 2023-07-13T22:30:28.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.824+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f00c6b70 con 0x7fe1f00cede0 2023-07-13T22:30:28.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.825+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f00c7090 con 0x7fe1f00cede0 2023-07-13T22:30:28.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.825+0000 7fe1e6ffd700 1 -- 172.21.15.60:0/1119272815 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe1e801b3b0 con 0x7fe1f00cede0 2023-07-13T22:30:28.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.826+0000 7fe1e6ffd700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1d006ad30 0x7fe1d006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:28.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.827+0000 7fe1e6ffd700 1 -- 172.21.15.60:0/1119272815 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe1e8088f00 con 0x7fe1f00cede0 2023-07-13T22:30:28.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.827+0000 7fe1ee59c700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1d006ad30 0x7fe1d006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:28.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.827+0000 7fe1ee59c700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1d006ad30 0x7fe1d006d1e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009fd0 tx=0x7fe1e001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:28.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:28.831+0000 7fe1e6ffd700 1 -- 172.21.15.60:0/1119272815 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe1e8054340 con 0x7fe1f00cede0 2023-07-13T22:30:29.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.039+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe1f004e210 con 0x7fe1f00cede0 2023-07-13T22:30:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.043+0000 7fe1e6ffd700 1 -- 172.21.15.60:0/1119272815 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1e8057960 con 0x7fe1f00cede0 2023-07-13T22:30:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1d006ad30 msgr2=0x7fe1d006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1d006ad30 0x7fe1d006d1e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009fd0 tx=0x7fe1e001c040 comp rx=0 tx=0).stop 2023-07-13T22:30:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f00cede0 msgr2=0x7fe1f00c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f00cede0 0x7fe1f00c94c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800caa0 tx=0x7fe1e800cdb0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 shutdown_connections 2023-07-13T22:30:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1d006ad30 0x7fe1d006d1e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f00cede0 0x7fe1f00c94c0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00b1ac0 0x7fe1f015f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 --2- 172.21.15.60:0/1119272815 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f00b8200 0x7fe1f00c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 >> 172.21.15.60:0/1119272815 conn(0x7fe1f006b2f0 msgr2=0x7fe1f00b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 shutdown_connections 2023-07-13T22:30:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.046+0000 7fe1f4dff700 1 -- 172.21.15.60:0/1119272815 wait complete. 2023-07-13T22:30:29.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-07-13T22:30:29.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.270+0000 7f4c999ff700 1 Processor -- start 2023-07-13T22:30:29.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.270+0000 7f4c999ff700 1 -- start start 2023-07-13T22:30:29.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c999ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940b8200 0x7f4c940cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c999ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c940b1ac0 0x7f4c940cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c999ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c940d1870 con 0x7f4c940cede0 2023-07-13T22:30:29.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c999ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940cf370 con 0x7f4c940b1ac0 2023-07-13T22:30:29.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c999ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940d12b0 con 0x7f4c940b8200 2023-07-13T22:30:29.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c8bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c940b1ac0 0x7f4c940cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c8bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c940b1ac0 0x7f4c940cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38434/0 (socket says 172.21.15.60:38434) 2023-07-13T22:30:29.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c8bfff700 1 -- 172.21.15.60:0/931295419 learned_addr learned my addr 172.21.15.60:0/931295419 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c937fe700 1 --1- 172.21.15.60:0/931295419 >> v1:172.21.15.60:6789/0 conn(0x7f4c940cede0 0x7f4c9415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56804/0 (socket says 172.21.15.60:56804) 2023-07-13T22:30:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.271+0000 7f4c92ffd700 1 --2- 172.21.15.60:0/931295419 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940b8200 0x7f4c940cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.272+0000 7f4c8bfff700 1 -- 172.21.15.60:0/931295419 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940b8200 msgr2=0x7f4c940cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.272+0000 7f4c8bfff700 1 --2- 172.21.15.60:0/931295419 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940b8200 0x7f4c940cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.272+0000 7f4c8bfff700 1 -- 172.21.15.60:0/931295419 >> v1:172.21.15.60:6789/0 conn(0x7f4c940cede0 legacy=0x7f4c9415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.272+0000 7f4c8bfff700 1 -- 172.21.15.60:0/931295419 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940d1590 con 0x7f4c940b1ac0 2023-07-13T22:30:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.272+0000 7f4c927fc700 1 -- 172.21.15.60:0/931295419 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849158218 0 0) 0x7f4c940d1870 con 0x7f4c940cede0 2023-07-13T22:30:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.272+0000 7f4c8bfff700 1 --2- 172.21.15.60:0/931295419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b1ac0 0x7f4c940cfb80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0098f0 tx=0x7f4c7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2ec514ea6173da server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.272+0000 7f4c927fc700 1 -- 172.21.15.60:0/931295419 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00b8a0 con 0x7f4c940b1ac0 2023-07-13T22:30:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.272+0000 7f4c927fc700 1 -- 172.21.15.60:0/931295419 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7c00ba00 con 0x7f4c940b1ac0 2023-07-13T22:30:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.272+0000 7f4c927fc700 1 -- 172.21.15.60:0/931295419 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00bd90 con 0x7f4c940b1ac0 2023-07-13T22:30:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.273+0000 7f4c999ff700 1 -- 172.21.15.60:0/931295419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b1ac0 msgr2=0x7f4c940cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.273+0000 7f4c999ff700 1 --2- 172.21.15.60:0/931295419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b1ac0 0x7f4c940cfb80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0098f0 tx=0x7f4c7c009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.273+0000 7f4c999ff700 1 -- 172.21.15.60:0/931295419 shutdown_connections 2023-07-13T22:30:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.273+0000 7f4c999ff700 1 --2- 172.21.15.60:0/931295419 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940b8200 0x7f4c940cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.273+0000 7f4c999ff700 1 --2- 172.21.15.60:0/931295419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b1ac0 0x7f4c940cfb80 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.273+0000 7f4c999ff700 1 -- 172.21.15.60:0/931295419 >> 172.21.15.60:0/931295419 conn(0x7f4c9406b2f0 msgr2=0x7f4c9406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.273+0000 7f4c999ff700 1 -- 172.21.15.60:0/931295419 shutdown_connections 2023-07-13T22:30:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.273+0000 7f4c999ff700 1 -- 172.21.15.60:0/931295419 wait complete. 2023-07-13T22:30:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.275+0000 7f4c999ff700 1 Processor -- start 2023-07-13T22:30:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.275+0000 7f4c999ff700 1 -- start start 2023-07-13T22:30:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c999ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c940b1ac0 0x7f4c9415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c999ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b8200 0x7f4c941e5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c999ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940cede0 0x7f4c941e4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c999ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940c6ed0 con 0x7f4c940b8200 2023-07-13T22:30:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c999ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9415bd70 con 0x7f4c940b1ac0 2023-07-13T22:30:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c999ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940c6bf0 con 0x7f4c940cede0 2023-07-13T22:30:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c8bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b8200 0x7f4c941e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c8bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b8200 0x7f4c941e5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38446/0 (socket says 172.21.15.60:38446) 2023-07-13T22:30:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c8bfff700 1 -- 172.21.15.60:0/1714064834 learned_addr learned my addr 172.21.15.60:0/1714064834 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c92ffd700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c940b1ac0 0x7f4c9415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c937fe700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940cede0 0x7f4c941e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c8bfff700 1 -- 172.21.15.60:0/1714064834 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940cede0 msgr2=0x7f4c941e4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c8bfff700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940cede0 0x7f4c941e4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c8bfff700 1 -- 172.21.15.60:0/1714064834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c940b1ac0 msgr2=0x7f4c9415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c8bfff700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c940b1ac0 0x7f4c9415b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c8bfff700 1 -- 172.21.15.60:0/1714064834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c009550 con 0x7f4c940b8200 2023-07-13T22:30:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.276+0000 7f4c92ffd700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c940b1ac0 0x7f4c9415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.277+0000 7f4c8bfff700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b8200 0x7f4c941e5510 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009c40 tx=0x7f4c7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.277+0000 7f4c90ff9700 1 -- 172.21.15.60:0/1714064834 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c004380 con 0x7f4c940b8200 2023-07-13T22:30:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.277+0000 7f4c90ff9700 1 -- 172.21.15.60:0/1714064834 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7c01f070 con 0x7f4c940b8200 2023-07-13T22:30:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.277+0000 7f4c90ff9700 1 -- 172.21.15.60:0/1714064834 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0036a0 con 0x7f4c940b8200 2023-07-13T22:30:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.277+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c941e3d90 con 0x7f4c940b8200 2023-07-13T22:30:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.277+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c941e4350 con 0x7f4c940b8200 2023-07-13T22:30:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.278+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c941e3bf0 con 0x7f4c940b8200 2023-07-13T22:30:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.279+0000 7f4c90ff9700 1 -- 172.21.15.60:0/1714064834 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c7c021030 con 0x7f4c940b8200 2023-07-13T22:30:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.280+0000 7f4c90ff9700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c7406ace0 0x7f4c7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.280+0000 7f4c90ff9700 1 -- 172.21.15.60:0/1714064834 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4c7c08a5f0 con 0x7f4c940b8200 2023-07-13T22:30:29.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.280+0000 7f4c92ffd700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c7406ace0 0x7f4c7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.281+0000 7f4c92ffd700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c7406ace0 0x7f4c7406d190 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009a00 tx=0x7f4c84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:29.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.284+0000 7f4c90ff9700 1 -- 172.21.15.60:0/1714064834 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4c941e3bf0 con 0x7f4c940b8200 2023-07-13T22:30:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.495+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7f4c941e3bf0 con 0x7f4c940b8200 2023-07-13T22:30:29.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.499+0000 7f4c90ff9700 1 -- 172.21.15.60:0/1714064834 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v51) v1 ==== 105+0+0 (secure 0 0 0) 0x7f4c941e3bf0 con 0x7f4c940b8200 2023-07-13T22:30:29.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c7406ace0 msgr2=0x7f4c7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c7406ace0 0x7f4c7406d190 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009a00 tx=0x7f4c84008040 comp rx=0 tx=0).stop 2023-07-13T22:30:29.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b8200 msgr2=0x7f4c941e5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b8200 0x7f4c941e5510 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009c40 tx=0x7f4c7c004750 comp rx=0 tx=0).stop 2023-07-13T22:30:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 shutdown_connections 2023-07-13T22:30:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c7406ace0 0x7f4c7406d190 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c940cede0 0x7f4c941e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c940b1ac0 0x7f4c9415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 --2- 172.21.15.60:0/1714064834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c940b8200 0x7f4c941e5510 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 >> 172.21.15.60:0/1714064834 conn(0x7f4c9406b2f0 msgr2=0x7f4c940b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:29.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 shutdown_connections 2023-07-13T22:30:29.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.501+0000 7f4c999ff700 1 -- 172.21.15.60:0/1714064834 wait complete. 2023-07-13T22:30:29.498 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-07-13T22:30:29.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.724+0000 7fec513ff700 1 Processor -- start 2023-07-13T22:30:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.724+0000 7fec513ff700 1 -- start start 2023-07-13T22:30:29.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec513ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec4c0b8200 0x7fec4c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec513ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec4c0b1ac0 0x7fec4c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec513ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 0x7fec4c15ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec513ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4c0d1870 con 0x7fec4c0b1ac0 2023-07-13T22:30:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec513ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4c0cf370 con 0x7fec4c0b8200 2023-07-13T22:30:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec513ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c0d12b0 con 0x7fec4c0cede0 2023-07-13T22:30:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec4a7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec4c0b1ac0 0x7fec4c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec4a7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec4c0b1ac0 0x7fec4c0cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38462/0 (socket says 172.21.15.60:38462) 2023-07-13T22:30:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec4a7fc700 1 -- 172.21.15.60:0/1138022044 learned_addr learned my addr 172.21.15.60:0/1138022044 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:29.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec4affd700 1 --2- 172.21.15.60:0/1138022044 >> v2:172.21.15.105:3300/0 conn(0x7fec4c0b8200 0x7fec4c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.725+0000 7fec4b7fe700 1 --2- 172.21.15.60:0/1138022044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 0x7fec4c15ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec4a7fc700 1 -- 172.21.15.60:0/1138022044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 msgr2=0x7fec4c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec4a7fc700 1 --2- 172.21.15.60:0/1138022044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 0x7fec4c15ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec4a7fc700 1 -- 172.21.15.60:0/1138022044 >> v2:172.21.15.105:3300/0 conn(0x7fec4c0b8200 msgr2=0x7fec4c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec4a7fc700 1 --2- 172.21.15.60:0/1138022044 >> v2:172.21.15.105:3300/0 conn(0x7fec4c0b8200 0x7fec4c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec4a7fc700 1 -- 172.21.15.60:0/1138022044 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c0d1590 con 0x7fec4c0b1ac0 2023-07-13T22:30:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec4b7fe700 1 --2- 172.21.15.60:0/1138022044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 0x7fec4c15ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec4a7fc700 1 --2- 172.21.15.60:0/1138022044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b1ac0 0x7fec4c0cfb80 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0b2660 tx=0x7fec40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3379cd38356f8c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec49ffb700 1 -- 172.21.15.60:0/1138022044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40011830 con 0x7fec4c0b1ac0 2023-07-13T22:30:29.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec49ffb700 1 -- 172.21.15.60:0/1138022044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec40011990 con 0x7fec4c0b1ac0 2023-07-13T22:30:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.726+0000 7fec49ffb700 1 -- 172.21.15.60:0/1138022044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40011d20 con 0x7fec4c0b1ac0 2023-07-13T22:30:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.727+0000 7fec513ff700 1 -- 172.21.15.60:0/1138022044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b1ac0 msgr2=0x7fec4c0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.727+0000 7fec513ff700 1 --2- 172.21.15.60:0/1138022044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b1ac0 0x7fec4c0cfb80 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0b2660 tx=0x7fec40009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.727+0000 7fec513ff700 1 -- 172.21.15.60:0/1138022044 shutdown_connections 2023-07-13T22:30:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.727+0000 7fec513ff700 1 --2- 172.21.15.60:0/1138022044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 0x7fec4c15ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.727+0000 7fec513ff700 1 --2- 172.21.15.60:0/1138022044 >> v2:172.21.15.105:3300/0 conn(0x7fec4c0b8200 0x7fec4c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.727+0000 7fec513ff700 1 --2- 172.21.15.60:0/1138022044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b1ac0 0x7fec4c0cfb80 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.727+0000 7fec513ff700 1 -- 172.21.15.60:0/1138022044 >> 172.21.15.60:0/1138022044 conn(0x7fec4c06b2f0 msgr2=0x7fec4c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.727+0000 7fec513ff700 1 -- 172.21.15.60:0/1138022044 shutdown_connections 2023-07-13T22:30:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.727+0000 7fec513ff700 1 -- 172.21.15.60:0/1138022044 wait complete. 2023-07-13T22:30:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.729+0000 7fec513ff700 1 Processor -- start 2023-07-13T22:30:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.729+0000 7fec513ff700 1 -- start start 2023-07-13T22:30:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec513ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec4c0b1ac0 0x7fec4c157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec513ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b8200 0x7fec4c1e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec513ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 0x7fec4c0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec513ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c1e6340 con 0x7fec4c0b8200 2023-07-13T22:30:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec513ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c1e6590 con 0x7fec4c0b1ac0 2023-07-13T22:30:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec513ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c1e6870 con 0x7fec4c0cede0 2023-07-13T22:30:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b8200 0x7fec4c1e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b8200 0x7fec4c1e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38470/0 (socket says 172.21.15.60:38470) 2023-07-13T22:30:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4a7fc700 1 -- 172.21.15.60:0/3528736050 learned_addr learned my addr 172.21.15.60:0/3528736050 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4a7fc700 1 -- 172.21.15.60:0/3528736050 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 msgr2=0x7fec4c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4a7fc700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 0x7fec4c0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4a7fc700 1 -- 172.21.15.60:0/3528736050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec4c0b1ac0 msgr2=0x7fec4c157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4affd700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec4c0b1ac0 0x7fec4c157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4a7fc700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec4c0b1ac0 0x7fec4c157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4a7fc700 1 -- 172.21.15.60:0/3528736050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40009550 con 0x7fec4c0b8200 2023-07-13T22:30:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.730+0000 7fec4affd700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec4c0b1ac0 0x7fec4c157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.731+0000 7fec4a7fc700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b8200 0x7fec4c1e7630 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fec40000c00 tx=0x7fec40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.731+0000 7fec337fe700 1 -- 172.21.15.60:0/3528736050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40004380 con 0x7fec4c0b8200 2023-07-13T22:30:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.731+0000 7fec337fe700 1 -- 172.21.15.60:0/3528736050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4001f070 con 0x7fec4c0b8200 2023-07-13T22:30:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.731+0000 7fec337fe700 1 -- 172.21.15.60:0/3528736050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec400036a0 con 0x7fec4c0b8200 2023-07-13T22:30:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.731+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c1e6b50 con 0x7fec4c0b8200 2023-07-13T22:30:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.731+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c1e7110 con 0x7fec4c0b8200 2023-07-13T22:30:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.732+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c065ee0 con 0x7fec4c0b8200 2023-07-13T22:30:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.733+0000 7fec337fe700 1 -- 172.21.15.60:0/3528736050 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec40021030 con 0x7fec4c0b8200 2023-07-13T22:30:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.734+0000 7fec337fe700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec3806ace0 0x7fec3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.734+0000 7fec337fe700 1 -- 172.21.15.60:0/3528736050 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fec4008a550 con 0x7fec4c0b8200 2023-07-13T22:30:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.734+0000 7fec4affd700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec3806ace0 0x7fec3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.735+0000 7fec4affd700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec3806ace0 0x7fec3806d190 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009b60 tx=0x7fec3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.738+0000 7fec337fe700 1 -- 172.21.15.60:0/3528736050 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fec40055880 con 0x7fec4c0b8200 2023-07-13T22:30:29.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.947+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7fec4c04e210 con 0x7fec4c0b8200 2023-07-13T22:30:29.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.951+0000 7fec337fe700 1 -- 172.21.15.60:0/3528736050 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v52) v1 ==== 111+0+0 (secure 0 0 0) 0x7fec40058ea0 con 0x7fec4c0b8200 2023-07-13T22:30:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.953+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec3806ace0 msgr2=0x7fec3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.953+0000 7fec513ff700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec3806ace0 0x7fec3806d190 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009b60 tx=0x7fec3c008040 comp rx=0 tx=0).stop 2023-07-13T22:30:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.953+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b8200 msgr2=0x7fec4c1e7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.953+0000 7fec513ff700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b8200 0x7fec4c1e7630 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fec40000c00 tx=0x7fec40004750 comp rx=0 tx=0).stop 2023-07-13T22:30:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.954+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 shutdown_connections 2023-07-13T22:30:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.954+0000 7fec513ff700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec3806ace0 0x7fec3806d190 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.954+0000 7fec513ff700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec4c0cede0 0x7fec4c0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.954+0000 7fec513ff700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec4c0b1ac0 0x7fec4c157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.954+0000 7fec513ff700 1 --2- 172.21.15.60:0/3528736050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec4c0b8200 0x7fec4c1e7630 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.954+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 >> 172.21.15.60:0/3528736050 conn(0x7fec4c06b2f0 msgr2=0x7fec4c0b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.954+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 shutdown_connections 2023-07-13T22:30:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:29.954+0000 7fec513ff700 1 -- 172.21.15.60:0/3528736050 wait complete. 2023-07-13T22:30:29.951 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-07-13T22:30:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.180+0000 7f41815ff700 1 Processor -- start 2023-07-13T22:30:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.180+0000 7f41815ff700 1 -- start start 2023-07-13T22:30:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.181+0000 7f41815ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 0x7f417c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.181+0000 7f41815ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f417c0b1ac0 0x7f417c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.181+0000 7f41815ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417c0d1870 con 0x7f417c0cede0 2023-07-13T22:30:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.181+0000 7f41815ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c0cf370 con 0x7f417c0b1ac0 2023-07-13T22:30:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.181+0000 7f41815ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0d12b0 con 0x7f417c0b8200 2023-07-13T22:30:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f417b7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f417c0cede0 0x7f417c15ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56820/0 (socket says 172.21.15.60:56820) 2023-07-13T22:30:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f417b7fe700 1 -- 172.21.15.60:0/3736843113 learned_addr learned my addr 172.21.15.60:0/3736843113 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f417a7fc700 1 --2- 172.21.15.60:0/3736843113 >> v2:172.21.15.105:3300/0 conn(0x7f417c0b1ac0 0x7f417c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f417affd700 1 --2- 172.21.15.60:0/3736843113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 0x7f417c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f4179ffb700 1 -- 172.21.15.60:0/3736843113 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180417257 0 0) 0x7f417c0d1870 con 0x7f417c0cede0 2023-07-13T22:30:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f4179ffb700 1 -- 172.21.15.60:0/3736843113 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41640035b0 con 0x7f417c0cede0 2023-07-13T22:30:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f417affd700 1 -- 172.21.15.60:0/3736843113 >> v2:172.21.15.105:3300/0 conn(0x7f417c0b1ac0 msgr2=0x7f417c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f417affd700 1 --2- 172.21.15.60:0/3736843113 >> v2:172.21.15.105:3300/0 conn(0x7f417c0b1ac0 0x7f417c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f417affd700 1 -- 172.21.15.60:0/3736843113 >> v1:172.21.15.60:6789/0 conn(0x7f417c0cede0 legacy=0x7f417c15ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.182+0000 7f417affd700 1 -- 172.21.15.60:0/3736843113 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c0d1590 con 0x7f417c0b8200 2023-07-13T22:30:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f417affd700 1 --2- 172.21.15.60:0/3736843113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 0x7f417c0cf640 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f416c009840 tx=0x7f416c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87119d3d8e90425e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f4179ffb700 1 -- 172.21.15.60:0/3736843113 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c011830 con 0x7f417c0b8200 2023-07-13T22:30:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f4179ffb700 1 -- 172.21.15.60:0/3736843113 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c011990 con 0x7f417c0b8200 2023-07-13T22:30:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f4179ffb700 1 -- 172.21.15.60:0/3736843113 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c011d20 con 0x7f417c0b8200 2023-07-13T22:30:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f41815ff700 1 -- 172.21.15.60:0/3736843113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 msgr2=0x7f417c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f41815ff700 1 --2- 172.21.15.60:0/3736843113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 0x7f417c0cf640 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f416c009840 tx=0x7f416c009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f41815ff700 1 -- 172.21.15.60:0/3736843113 shutdown_connections 2023-07-13T22:30:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f41815ff700 1 --2- 172.21.15.60:0/3736843113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 0x7f417c0cf640 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f41815ff700 1 --2- 172.21.15.60:0/3736843113 >> v2:172.21.15.105:3300/0 conn(0x7f417c0b1ac0 0x7f417c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.183+0000 7f41815ff700 1 -- 172.21.15.60:0/3736843113 >> 172.21.15.60:0/3736843113 conn(0x7f417c06b2f0 msgr2=0x7f417c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.184+0000 7f41815ff700 1 -- 172.21.15.60:0/3736843113 shutdown_connections 2023-07-13T22:30:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.184+0000 7f41815ff700 1 -- 172.21.15.60:0/3736843113 wait complete. 2023-07-13T22:30:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.185+0000 7f41815ff700 1 Processor -- start 2023-07-13T22:30:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.185+0000 7f41815ff700 1 -- start start 2023-07-13T22:30:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.186+0000 7f41815ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f417c0b1ac0 0x7f417c0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.186+0000 7f41815ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 0x7f417c1f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.186+0000 7f41815ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c0cede0 0x7f417c1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.186+0000 7f41815ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0c78a0 con 0x7f417c0b1ac0 2023-07-13T22:30:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.186+0000 7f41815ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0c7b20 con 0x7f417c0cede0 2023-07-13T22:30:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.186+0000 7f41815ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0cc1b0 con 0x7f417c0b8200 2023-07-13T22:30:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f417c0b1ac0 0x7f417c0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f417c0b1ac0 0x7f417c0cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38474/0 (socket says 172.21.15.60:38474) 2023-07-13T22:30:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417affd700 1 -- 172.21.15.60:0/2317129303 learned_addr learned my addr 172.21.15.60:0/2317129303 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417a7fc700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 0x7f417c1f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417b7fe700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c0cede0 0x7f417c1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417affd700 1 -- 172.21.15.60:0/2317129303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 msgr2=0x7f417c1f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417affd700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 0x7f417c1f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417affd700 1 -- 172.21.15.60:0/2317129303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c0cede0 msgr2=0x7f417c1f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417affd700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c0cede0 0x7f417c1f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417affd700 1 -- 172.21.15.60:0/2317129303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c0094a0 con 0x7f417c0b1ac0 2023-07-13T22:30:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f417affd700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f417c0b1ac0 0x7f417c0cb760 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f416c0090b0 tx=0x7f416c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.187+0000 7f41637fe700 1 -- 172.21.15.60:0/2317129303 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c004590 con 0x7f417c0b1ac0 2023-07-13T22:30:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.188+0000 7f41637fe700 1 -- 172.21.15.60:0/2317129303 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c01f070 con 0x7f417c0b1ac0 2023-07-13T22:30:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.188+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c0cc490 con 0x7f417c0b1ac0 2023-07-13T22:30:30.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.188+0000 7f41637fe700 1 -- 172.21.15.60:0/2317129303 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0038e0 con 0x7f417c0b1ac0 2023-07-13T22:30:30.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.188+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c14b710 con 0x7f417c0b1ac0 2023-07-13T22:30:30.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.189+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417c04e210 con 0x7f417c0b1ac0 2023-07-13T22:30:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.190+0000 7f41637fe700 1 -- 172.21.15.60:0/2317129303 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f416c003a40 con 0x7f417c0b1ac0 2023-07-13T22:30:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.191+0000 7f41637fe700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f416806ace0 0x7f416806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.191+0000 7f41637fe700 1 -- 172.21.15.60:0/2317129303 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f416c021070 con 0x7f417c0b1ac0 2023-07-13T22:30:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.191+0000 7f417a7fc700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f416806ace0 0x7f416806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.191+0000 7f417a7fc700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f416806ace0 0x7f416806d190 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4170009be0 tx=0x7f4170009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.195+0000 7f41637fe700 1 -- 172.21.15.60:0/2317129303 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f416c055cc0 con 0x7f417c0b1ac0 2023-07-13T22:30:30.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.402+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7f417c0cc680 con 0x7f417c0b1ac0 2023-07-13T22:30:30.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.406+0000 7f41637fe700 1 -- 172.21.15.60:0/2317129303 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v53) v1 ==== 107+0+0 (secure 0 0 0) 0x7f416c0592e0 con 0x7f417c0b1ac0 2023-07-13T22:30:30.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.408+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f416806ace0 msgr2=0x7f416806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.408+0000 7f41815ff700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f416806ace0 0x7f416806d190 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4170009be0 tx=0x7f4170009210 comp rx=0 tx=0).stop 2023-07-13T22:30:30.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.408+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f417c0b1ac0 msgr2=0x7f417c0cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.408+0000 7f41815ff700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f417c0b1ac0 0x7f417c0cb760 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f416c0090b0 tx=0x7f416c004750 comp rx=0 tx=0).stop 2023-07-13T22:30:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.408+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 shutdown_connections 2023-07-13T22:30:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.408+0000 7f41815ff700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f416806ace0 0x7f416806d190 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.409+0000 7f41815ff700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f417c0b1ac0 0x7f417c0cb760 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.409+0000 7f41815ff700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f417c0cede0 0x7f417c1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.409+0000 7f41815ff700 1 --2- 172.21.15.60:0/2317129303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f417c0b8200 0x7f417c1f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.409+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 >> 172.21.15.60:0/2317129303 conn(0x7f417c06b2f0 msgr2=0x7f417c0b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:30.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.409+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 shutdown_connections 2023-07-13T22:30:30.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.409+0000 7f41815ff700 1 -- 172.21.15.60:0/2317129303 wait complete. 2023-07-13T22:30:30.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2023-07-13T22:30:30.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.629+0000 7f11329ff700 1 Processor -- start 2023-07-13T22:30:30.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.630+0000 7f11329ff700 1 -- start start 2023-07-13T22:30:30.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f11329ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f112c0d09e0 0x7f112c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f11329ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0cd1f0 0x7f112c0d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f11329ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112c0d2ed0 con 0x7f112c0d0db0 2023-07-13T22:30:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f11329ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112c0cd960 con 0x7f112c0d09e0 2023-07-13T22:30:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f11329ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0cd680 con 0x7f112c0cd1f0 2023-07-13T22:30:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f112bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f112c0d09e0 0x7f112c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f1130f9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f112c0d0db0 0x7f112c0b59f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56832/0 (socket says 172.21.15.60:56832) 2023-07-13T22:30:30.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f112bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f112c0d09e0 0x7f112c0cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38490/0 (socket says 172.21.15.60:38490) 2023-07-13T22:30:30.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f1130f9c700 1 -- 172.21.15.60:0/3545709845 learned_addr learned my addr 172.21.15.60:0/3545709845 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:30.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.631+0000 7f112b7fe700 1 --2- 172.21.15.60:0/3545709845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0cd1f0 0x7f112c0d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f112bfff700 1 -- 172.21.15.60:0/3545709845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0cd1f0 msgr2=0x7f112c0d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:30.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f112bfff700 1 --2- 172.21.15.60:0/3545709845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0cd1f0 0x7f112c0d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f112bfff700 1 -- 172.21.15.60:0/3545709845 >> v1:172.21.15.60:6789/0 conn(0x7f112c0d0db0 legacy=0x7f112c0b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f112bfff700 1 -- 172.21.15.60:0/3545709845 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c0d2880 con 0x7f112c0d09e0 2023-07-13T22:30:30.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f112affd700 1 -- 172.21.15.60:0/3545709845 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093330120 0 0) 0x7f112c0d2ed0 con 0x7f112c0d0db0 2023-07-13T22:30:30.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f112bfff700 1 --2- 172.21.15.60:0/3545709845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d09e0 0x7f112c0cf900 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f111c009840 tx=0x7f111c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94db0a2b1dbf7eb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:30.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f112affd700 1 -- 172.21.15.60:0/3545709845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011830 con 0x7f112c0d09e0 2023-07-13T22:30:30.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f112affd700 1 -- 172.21.15.60:0/3545709845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c011990 con 0x7f112c0d09e0 2023-07-13T22:30:30.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f112affd700 1 -- 172.21.15.60:0/3545709845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011d20 con 0x7f112c0d09e0 2023-07-13T22:30:30.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f11329ff700 1 -- 172.21.15.60:0/3545709845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d09e0 msgr2=0x7f112c0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.632+0000 7f11329ff700 1 --2- 172.21.15.60:0/3545709845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d09e0 0x7f112c0cf900 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f111c009840 tx=0x7f111c009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:30.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.633+0000 7f11329ff700 1 -- 172.21.15.60:0/3545709845 shutdown_connections 2023-07-13T22:30:30.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.633+0000 7f11329ff700 1 --2- 172.21.15.60:0/3545709845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d09e0 0x7f112c0cf900 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.633+0000 7f11329ff700 1 --2- 172.21.15.60:0/3545709845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0cd1f0 0x7f112c0d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.633+0000 7f11329ff700 1 -- 172.21.15.60:0/3545709845 >> 172.21.15.60:0/3545709845 conn(0x7f112c071600 msgr2=0x7f112c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:30.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.633+0000 7f11329ff700 1 -- 172.21.15.60:0/3545709845 shutdown_connections 2023-07-13T22:30:30.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.633+0000 7f11329ff700 1 -- 172.21.15.60:0/3545709845 wait complete. 2023-07-13T22:30:30.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.634+0000 7f11329ff700 1 Processor -- start 2023-07-13T22:30:30.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.635+0000 7f11329ff700 1 -- start start 2023-07-13T22:30:30.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.635+0000 7f11329ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f112c0cd1f0 0x7f112c14e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.635+0000 7f11329ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0d09e0 0x7f112c15e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.635+0000 7f11329ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d0db0 0x7f112c15ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f11329ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0c9510 con 0x7f112c0d0db0 2023-07-13T22:30:30.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f11329ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c14e580 con 0x7f112c0cd1f0 2023-07-13T22:30:30.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f11329ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0cab10 con 0x7f112c0d09e0 2023-07-13T22:30:30.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f1130f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d0db0 0x7f112c15ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f1130f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d0db0 0x7f112c15ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38502/0 (socket says 172.21.15.60:38502) 2023-07-13T22:30:30.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f1130f9c700 1 -- 172.21.15.60:0/3333085909 learned_addr learned my addr 172.21.15.60:0/3333085909 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:30.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f112bfff700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f112c0cd1f0 0x7f112c14e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f112b7fe700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0d09e0 0x7f112c15e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f1130f9c700 1 -- 172.21.15.60:0/3333085909 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0d09e0 msgr2=0x7f112c15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f1130f9c700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0d09e0 0x7f112c15e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f1130f9c700 1 -- 172.21.15.60:0/3333085909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f112c0cd1f0 msgr2=0x7f112c14e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:30.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f1130f9c700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f112c0cd1f0 0x7f112c14e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:30.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f1130f9c700 1 -- 172.21.15.60:0/3333085909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0094a0 con 0x7f112c0d0db0 2023-07-13T22:30:30.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.636+0000 7f1130f9c700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d0db0 0x7f112c15ed40 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f112400cba0 tx=0x7f112400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:30.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.637+0000 7f1128ff9700 1 -- 172.21.15.60:0/3333085909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400aaa0 con 0x7f112c0d0db0 2023-07-13T22:30:30.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.637+0000 7f1128ff9700 1 -- 172.21.15.60:0/3333085909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112400ac20 con 0x7f112c0d0db0 2023-07-13T22:30:30.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.637+0000 7f1128ff9700 1 -- 172.21.15.60:0/3333085909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11240070d0 con 0x7f112c0d0db0 2023-07-13T22:30:30.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.637+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c0cffc0 con 0x7f112c0d0db0 2023-07-13T22:30:30.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.637+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c0cb0d0 con 0x7f112c0d0db0 2023-07-13T22:30:30.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.638+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112c04e280 con 0x7f112c0d0db0 2023-07-13T22:30:30.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.639+0000 7f1128ff9700 1 -- 172.21.15.60:0/3333085909 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1124007230 con 0x7f112c0d0db0 2023-07-13T22:30:30.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.640+0000 7f1128ff9700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f111806ace0 0x7f111806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:30.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.641+0000 7f1128ff9700 1 -- 172.21.15.60:0/3333085909 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1124088c70 con 0x7f112c0d0db0 2023-07-13T22:30:30.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.641+0000 7f112bfff700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f111806ace0 0x7f111806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:30.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.641+0000 7f112bfff700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f111806ace0 0x7f111806d190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f111c0090b0 tx=0x7f111c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.645+0000 7f1128ff9700 1 -- 172.21.15.60:0/3333085909 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1124054030 con 0x7f112c0d0db0 2023-07-13T22:30:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:30.936+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) v1 -- 0x7f112c073660 con 0x7f112c0d0db0 2023-07-13T22:30:31.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.126+0000 7f1128ff9700 1 -- 172.21.15.60:0/3333085909 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v491) v1 ==== 151+0+0 (secure 0 0 0) 0x7f1124057650 con 0x7f112c0d0db0 2023-07-13T22:30:31.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f111806ace0 msgr2=0x7f111806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:31.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f111806ace0 0x7f111806d190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f111c0090b0 tx=0x7f111c008030 comp rx=0 tx=0).stop 2023-07-13T22:30:31.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d0db0 msgr2=0x7f112c15ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:31.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d0db0 0x7f112c15ed40 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f112400cba0 tx=0x7f112400ceb0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 shutdown_connections 2023-07-13T22:30:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f111806ace0 0x7f111806d190 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f112c0d0db0 0x7f112c15ed40 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f112c0cd1f0 0x7f112c14e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 --2- 172.21.15.60:0/3333085909 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f112c0d09e0 0x7f112c15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 >> 172.21.15.60:0/3333085909 conn(0x7f112c071600 msgr2=0x7f112c0b7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 shutdown_connections 2023-07-13T22:30:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.129+0000 7f11329ff700 1 -- 172.21.15.60:0/3333085909 wait complete. 2023-07-13T22:30:31.126 INFO:tasks.cephfs_test_runner:test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-13T22:30:31.126 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:30:31.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2023-07-13T22:30:31.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.353+0000 7f1e49dff700 1 Processor -- start 2023-07-13T22:30:31.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.353+0000 7f1e49dff700 1 -- start start 2023-07-13T22:30:31.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.354+0000 7f1e49dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1e4406a5e0 0x7f1e440719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:31.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.354+0000 7f1e49dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e440abc20 0x7f1e440ac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:31.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.354+0000 7f1e49dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440cfa00 con 0x7f1e4415c330 2023-07-13T22:30:31.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e49dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e44071f00 con 0x7f1e4406a5e0 2023-07-13T22:30:31.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e49dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440cf720 con 0x7f1e440abc20 2023-07-13T22:30:31.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e437fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1e4415c330 0x7f1e44069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56834/0 (socket says 172.21.15.60:56834) 2023-07-13T22:30:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e437fe700 1 -- 172.21.15.60:0/1720951515 learned_addr learned my addr 172.21.15.60:0/1720951515 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e42ffd700 1 --2- 172.21.15.60:0/1720951515 >> v2:172.21.15.105:3300/0 conn(0x7f1e4406a5e0 0x7f1e440719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e43fff700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e440abc20 0x7f1e440ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076174982 0 0) 0x7f1e440cfa00 con 0x7f1e4415c330 2023-07-13T22:30:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e2c0035b0 con 0x7f1e4415c330 2023-07-13T22:30:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2192967221 0 0) 0x7f1e2c0035b0 con 0x7f1e4415c330 2023-07-13T22:30:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e440abc20 msgr2=0x7f1e440ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e427fc700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e440abc20 0x7f1e440ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 >> v2:172.21.15.105:3300/0 conn(0x7f1e4406a5e0 msgr2=0x7f1e440719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e427fc700 1 --2- 172.21.15.60:0/1720951515 >> v2:172.21.15.105:3300/0 conn(0x7f1e4406a5e0 0x7f1e440719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440ccd60 con 0x7f1e4415c330 2023-07-13T22:30:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.355+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f1e34001670 con 0x7f1e4415c330 2023-07-13T22:30:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 >> v1:172.21.15.60:6789/0 conn(0x7f1e4415c330 legacy=0x7f1e44069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e427fc700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e440abc20 0x7f1e2c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e427fc700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e2c003fd0 0x7f1e2c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:31.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e427fc700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e2c004d90 0x7f1e2c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:31.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e44071f00 con 0x7f1e2c003fd0 2023-07-13T22:30:31.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440cf720 con 0x7f1e2c004d90 2023-07-13T22:30:31.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c00bb40 con 0x7f1e440abc20 2023-07-13T22:30:31.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e42ffd700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e2c003fd0 0x7f1e2c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:31.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e437fe700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e440abc20 0x7f1e2c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:31.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e43fff700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e2c004d90 0x7f1e2c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:31.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e42ffd700 1 -- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e440abc20 msgr2=0x7f1e2c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:31.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e42ffd700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e440abc20 0x7f1e2c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e42ffd700 1 -- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e2c004d90 msgr2=0x7f1e2c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:31.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e42ffd700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e2c004d90 0x7f1e2c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.356+0000 7f1e42ffd700 1 -- 172.21.15.60:0/1720951515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0035b0 con 0x7f1e2c003fd0 2023-07-13T22:30:31.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.357+0000 7f1e42ffd700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e2c003fd0 0x7f1e2c004850 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1e440ccac0 tx=0x7f1e2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da354983d8093c79 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:31.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.357+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800e690 con 0x7f1e2c003fd0 2023-07-13T22:30:31.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.357+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3800e7f0 con 0x7f1e2c003fd0 2023-07-13T22:30:31.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.357+0000 7f1e427fc700 1 -- 172.21.15.60:0/1720951515 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800eb80 con 0x7f1e2c003fd0 2023-07-13T22:30:31.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.357+0000 7f1e49dff700 1 -- 172.21.15.60:0/1720951515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e2c003fd0 msgr2=0x7f1e2c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:31.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.357+0000 7f1e49dff700 1 --2- 172.21.15.60:0/1720951515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e2c003fd0 0x7f1e2c004850 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1e440ccac0 tx=0x7f1e2c005880 comp rx=0 tx=0).stop 2023-07-13T22:30:31.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.357+0000 7f1e43fff700 1 -- 172.21.15.60:0/1720951515 reap_dead start 2023-07-13T22:30:31.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.357+0000 7f1e49dff700 1 -- 172.21.15.60:0/1720951515 shutdown_connections 2023-07-13T22:30:31.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.357+0000 7f1e49dff700 1 -- 172.21.15.60:0/1720951515 >> 172.21.15.60:0/1720951515 conn(0x7f1e440b0b90 msgr2=0x7f1e440b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:31.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.358+0000 7f1e49dff700 1 -- 172.21.15.60:0/1720951515 shutdown_connections 2023-07-13T22:30:31.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.358+0000 7f1e49dff700 1 -- 172.21.15.60:0/1720951515 wait complete. 2023-07-13T22:30:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.359+0000 7f1e49dff700 1 Processor -- start 2023-07-13T22:30:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.359+0000 7f1e49dff700 1 -- start start 2023-07-13T22:30:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.360+0000 7f1e49dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e2c003fd0 0x7f1e4415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.360+0000 7f1e49dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e440cabb0 0x7f1e440cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.360+0000 7f1e49dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e440b1e90 0x7f1e440b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.360+0000 7f1e49dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440b2840 con 0x7f1e440cabb0 2023-07-13T22:30:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.360+0000 7f1e49dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440b4930 con 0x7f1e440b1e90 2023-07-13T22:30:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.360+0000 7f1e49dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440b4dc0 con 0x7f1e2c003fd0 2023-07-13T22:30:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e42ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e440cabb0 0x7f1e440cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e42ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e440cabb0 0x7f1e440cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38510/0 (socket says 172.21.15.60:38510) 2023-07-13T22:30:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e42ffd700 1 -- 172.21.15.60:0/3665302739 learned_addr learned my addr 172.21.15.60:0/3665302739 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e437fe700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e2c003fd0 0x7f1e4415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e43fff700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e440b1e90 0x7f1e440b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e42ffd700 1 -- 172.21.15.60:0/3665302739 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e2c003fd0 msgr2=0x7f1e4415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e42ffd700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e2c003fd0 0x7f1e4415a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e42ffd700 1 -- 172.21.15.60:0/3665302739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e440b1e90 msgr2=0x7f1e440b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e42ffd700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e440b1e90 0x7f1e440b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:31.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e42ffd700 1 -- 172.21.15.60:0/3665302739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440cb630 con 0x7f1e440cabb0 2023-07-13T22:30:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.361+0000 7f1e42ffd700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e440cabb0 0x7f1e440cb020 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1e380042f0 tx=0x7f1e380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.362+0000 7f1e2bfff700 1 -- 172.21.15.60:0/3665302739 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800e690 con 0x7f1e440cabb0 2023-07-13T22:30:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.362+0000 7f1e2bfff700 1 -- 172.21.15.60:0/3665302739 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e380038a0 con 0x7f1e440cabb0 2023-07-13T22:30:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.362+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e440b42e0 con 0x7f1e440cabb0 2023-07-13T22:30:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.362+0000 7f1e2bfff700 1 -- 172.21.15.60:0/3665302739 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38006e60 con 0x7f1e440cabb0 2023-07-13T22:30:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.362+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e440b3fd0 con 0x7f1e440cabb0 2023-07-13T22:30:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.363+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4404e280 con 0x7f1e440cabb0 2023-07-13T22:30:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.364+0000 7f1e2bfff700 1 -- 172.21.15.60:0/3665302739 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e38003c30 con 0x7f1e440cabb0 2023-07-13T22:30:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.365+0000 7f1e2bfff700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e3006ace0 0x7f1e3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.365+0000 7f1e2bfff700 1 -- 172.21.15.60:0/3665302739 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1e3808abb0 con 0x7f1e440cabb0 2023-07-13T22:30:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.366+0000 7f1e437fe700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e3006ace0 0x7f1e3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.367+0000 7f1e437fe700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e3006ace0 0x7f1e3006d190 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1e34001120 tx=0x7f1e34001dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:31.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.370+0000 7f1e2bfff700 1 -- 172.21.15.60:0/3665302739 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1e38059b20 con 0x7f1e440cabb0 2023-07-13T22:30:31.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:31.666+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) v1 -- 0x7f1e440b4a70 con 0x7f1e440cabb0 2023-07-13T22:30:32.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.132+0000 7f1e2bfff700 1 -- 172.21.15.60:0/3665302739 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v492) v1 ==== 151+0+0 (secure 0 0 0) 0x7f1e380596b0 con 0x7f1e440cabb0 2023-07-13T22:30:32.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e3006ace0 msgr2=0x7f1e3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:32.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e3006ace0 0x7f1e3006d190 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1e34001120 tx=0x7f1e34001dd0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e440cabb0 msgr2=0x7f1e440cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e440cabb0 0x7f1e440cb020 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1e380042f0 tx=0x7f1e380047c0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 shutdown_connections 2023-07-13T22:30:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e3006ace0 0x7f1e3006d190 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e440b1e90 0x7f1e440b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e2c003fd0 0x7f1e4415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 --2- 172.21.15.60:0/3665302739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e440cabb0 0x7f1e440cb020 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.135+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 >> 172.21.15.60:0/3665302739 conn(0x7f1e440b0b90 msgr2=0x7f1e4406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.136+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 shutdown_connections 2023-07-13T22:30:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.136+0000 7f1e49dff700 1 -- 172.21.15.60:0/3665302739 wait complete. 2023-07-13T22:30:32.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-13T22:30:32.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.365+0000 7f7e323ff700 1 Processor -- start 2023-07-13T22:30:32.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.366+0000 7f7e323ff700 1 -- start start 2023-07-13T22:30:32.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.367+0000 7f7e323ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e2c0cfd50 0x7f7e2c0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.367+0000 7f7e323ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0d1870 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.367+0000 7f7e323ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0cf370 con 0x7f7e2c0b8200 2023-07-13T22:30:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.367+0000 7f7e323ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c0d12b0 con 0x7f7e2c0cfd50 2023-07-13T22:30:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.367+0000 7f7e3099c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7e2c0b1ac0 0x7f7e2c0d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56840/0 (socket says 172.21.15.60:56840) 2023-07-13T22:30:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.367+0000 7f7e3099c700 1 -- 172.21.15.60:0/2086408119 learned_addr learned my addr 172.21.15.60:0/2086408119 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.367+0000 7f7e2b7fe700 1 --2- 172.21.15.60:0/2086408119 >> v2:172.21.15.105:3300/0 conn(0x7f7e2c0cfd50 0x7f7e2c0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:32.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.368+0000 7f7e2affd700 1 -- 172.21.15.60:0/2086408119 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580616744 0 0) 0x7f7e2c0d1870 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.368+0000 7f7e2affd700 1 -- 172.21.15.60:0/2086408119 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e140035b0 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.368+0000 7f7e2affd700 1 -- 172.21.15.60:0/2086408119 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464900000 0 0) 0x7f7e2c0cf370 con 0x7f7e2c0b8200 2023-07-13T22:30:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.368+0000 7f7e2affd700 1 -- 172.21.15.60:0/2086408119 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e2c0d1870 con 0x7f7e2c0b8200 2023-07-13T22:30:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.368+0000 7f7e2b7fe700 1 -- 172.21.15.60:0/2086408119 >> v1:172.21.15.105:6789/0 conn(0x7f7e2c0b8200 legacy=0x7f7e2c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.368+0000 7f7e2b7fe700 1 -- 172.21.15.60:0/2086408119 >> v1:172.21.15.60:6789/0 conn(0x7f7e2c0b1ac0 legacy=0x7f7e2c0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.368+0000 7f7e2b7fe700 1 -- 172.21.15.60:0/2086408119 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0d1590 con 0x7f7e2c0cfd50 2023-07-13T22:30:32.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.368+0000 7f7e2b7fe700 1 --2- 172.21.15.60:0/2086408119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e2c0cfd50 0x7f7e2c0d23c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009840 tx=0x7f7e20009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb15b022ad9680b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:32.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e2affd700 1 -- 172.21.15.60:0/2086408119 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20011830 con 0x7f7e2c0cfd50 2023-07-13T22:30:32.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e2affd700 1 -- 172.21.15.60:0/2086408119 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e20011990 con 0x7f7e2c0cfd50 2023-07-13T22:30:32.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e2affd700 1 -- 172.21.15.60:0/2086408119 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20011d20 con 0x7f7e2c0cfd50 2023-07-13T22:30:32.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e323ff700 1 -- 172.21.15.60:0/2086408119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e2c0cfd50 msgr2=0x7f7e2c0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e323ff700 1 --2- 172.21.15.60:0/2086408119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e2c0cfd50 0x7f7e2c0d23c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009840 tx=0x7f7e20009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e323ff700 1 -- 172.21.15.60:0/2086408119 shutdown_connections 2023-07-13T22:30:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e323ff700 1 --2- 172.21.15.60:0/2086408119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e2c0cfd50 0x7f7e2c0d23c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e323ff700 1 -- 172.21.15.60:0/2086408119 >> 172.21.15.60:0/2086408119 conn(0x7f7e2c06b2f0 msgr2=0x7f7e2c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e323ff700 1 -- 172.21.15.60:0/2086408119 shutdown_connections 2023-07-13T22:30:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.369+0000 7f7e323ff700 1 -- 172.21.15.60:0/2086408119 wait complete. 2023-07-13T22:30:32.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.371+0000 7f7e323ff700 1 Processor -- start 2023-07-13T22:30:32.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.371+0000 7f7e323ff700 1 -- start start 2023-07-13T22:30:32.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.372+0000 7f7e323ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e2c0b1ac0 0x7f7e2c1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:32.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.372+0000 7f7e323ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e2c0b8200 0x7f7e2c148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.372+0000 7f7e323ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e2c0cfd50 0x7f7e2c0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.372+0000 7f7e323ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1f5870 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.372+0000 7f7e323ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0c6c60 con 0x7f7e2c0b8200 2023-07-13T22:30:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.372+0000 7f7e323ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0c6f40 con 0x7f7e2c0cfd50 2023-07-13T22:30:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.372+0000 7f7e2bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e2c0b1ac0 0x7f7e2c1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.372+0000 7f7e2bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e2c0b1ac0 0x7f7e2c1561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38526/0 (socket says 172.21.15.60:38526) 2023-07-13T22:30:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.372+0000 7f7e2bfff700 1 -- 172.21.15.60:0/3169938384 learned_addr learned my addr 172.21.15.60:0/3169938384 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e2b7fe700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e2c0b8200 0x7f7e2c148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e3099c700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e2c0cfd50 0x7f7e2c0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e2bfff700 1 -- 172.21.15.60:0/3169938384 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e2c0cfd50 msgr2=0x7f7e2c0c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e2bfff700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e2c0cfd50 0x7f7e2c0c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e2bfff700 1 -- 172.21.15.60:0/3169938384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e2c0b8200 msgr2=0x7f7e2c148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e2bfff700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e2c0b8200 0x7f7e2c148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e2bfff700 1 -- 172.21.15.60:0/3169938384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e200094a0 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e2bfff700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e2c0b1ac0 0x7f7e2c1561a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00c520 tx=0x7f7e1c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e28ff9700 1 -- 172.21.15.60:0/3169938384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0052d0 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e28ff9700 1 -- 172.21.15.60:0/3169938384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c005430 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c0c7220 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e28ff9700 1 -- 172.21.15.60:0/3169938384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0038c0 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.373+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c149500 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.375+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c04e210 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.376+0000 7f7e28ff9700 1 -- 172.21.15.60:0/3169938384 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e1c003a20 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.376+0000 7f7e28ff9700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e1806ace0 0x7f7e1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.377+0000 7f7e28ff9700 1 -- 172.21.15.60:0/3169938384 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7e1c0887a0 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.377+0000 7f7e2b7fe700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e1806ace0 0x7f7e1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.377+0000 7f7e2b7fe700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e1806ace0 0x7f7e1806d190 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0d0140 tx=0x7f7e2001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:32.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.381+0000 7f7e28ff9700 1 -- 172.21.15.60:0/3169938384 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7e1c053ad0 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.588+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7e2c0cee70 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.593+0000 7f7e28ff9700 1 -- 172.21.15.60:0/3169938384 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7e1c0570f0 con 0x7f7e2c0b1ac0 2023-07-13T22:30:32.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.595+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e1806ace0 msgr2=0x7f7e1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.595+0000 7f7e323ff700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e1806ace0 0x7f7e1806d190 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0d0140 tx=0x7f7e2001c040 comp rx=0 tx=0).stop 2023-07-13T22:30:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.595+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e2c0b1ac0 msgr2=0x7f7e2c1561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.595+0000 7f7e323ff700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e2c0b1ac0 0x7f7e2c1561a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00c520 tx=0x7f7e1c00c8e0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.596+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 shutdown_connections 2023-07-13T22:30:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.596+0000 7f7e323ff700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7e1806ace0 0x7f7e1806d190 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.596+0000 7f7e323ff700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e2c0b1ac0 0x7f7e2c1561a0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.596+0000 7f7e323ff700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e2c0cfd50 0x7f7e2c0c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.596+0000 7f7e323ff700 1 --2- 172.21.15.60:0/3169938384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e2c0b8200 0x7f7e2c148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.596+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 >> 172.21.15.60:0/3169938384 conn(0x7f7e2c06b2f0 msgr2=0x7f7e2c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.596+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 shutdown_connections 2023-07-13T22:30:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:32.596+0000 7f7e323ff700 1 -- 172.21.15.60:0/3169938384 wait complete. 2023-07-13T22:30:32.593 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:30:32.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:30:32.697 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:30:32.804 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:30:32.804 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:30:32.804 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:32.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:30:32.851 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:30:32.851 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:32.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:30:32.900 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:30:32.901 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:32.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:30:32.947 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:30:32.948 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:32.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:30:32.995 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-13T22:30:32.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:33.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.220+0000 7f2163bff700 1 Processor -- start 2023-07-13T22:30:33.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.220+0000 7f2163bff700 1 -- start start 2023-07-13T22:30:33.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.221+0000 7f2163bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f215c15c300 0x7f215c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.221+0000 7f2163bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f215c15c6d0 0x7f215c0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.221+0000 7f2163bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215c0d18a0 con 0x7f215c0b46c0 2023-07-13T22:30:33.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.221+0000 7f2163bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215c0d15c0 con 0x7f215c15c6d0 2023-07-13T22:30:33.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.221+0000 7f2163bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215c0cd3c0 con 0x7f215c15c300 2023-07-13T22:30:33.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.221+0000 7f216119a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f215c15c6d0 0x7f215c0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.221+0000 7f216119a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f215c15c6d0 0x7f215c0cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38532/0 (socket says 172.21.15.60:38532) 2023-07-13T22:30:33.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.221+0000 7f216119a700 1 -- 172.21.15.60:0/74493305 learned_addr learned my addr 172.21.15.60:0/74493305 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f216199b700 1 --2- 172.21.15.60:0/74493305 >> v2:172.21.15.105:3300/0 conn(0x7f215c15c300 0x7f215c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f216119a700 1 -- 172.21.15.60:0/74493305 >> v2:172.21.15.105:3300/0 conn(0x7f215c15c300 msgr2=0x7f215c0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f216119a700 1 --2- 172.21.15.60:0/74493305 >> v2:172.21.15.105:3300/0 conn(0x7f215c15c300 0x7f215c0b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f216119a700 1 -- 172.21.15.60:0/74493305 >> v1:172.21.15.105:6789/0 conn(0x7f215c0b46c0 legacy=0x7f215c0b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f216119a700 1 -- 172.21.15.60:0/74493305 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c0cd6a0 con 0x7f215c15c6d0 2023-07-13T22:30:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f216199b700 1 --2- 172.21.15.60:0/74493305 >> v2:172.21.15.105:3300/0 conn(0x7f215c15c300 0x7f215c0b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f216119a700 1 --2- 172.21.15.60:0/74493305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 0x7f215c0cedf0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f215c0d2ba0 tx=0x7f214c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5bbdd4f8acb59d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f2160999700 1 -- 172.21.15.60:0/74493305 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c004230 con 0x7f215c15c6d0 2023-07-13T22:30:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f2160999700 1 -- 172.21.15.60:0/74493305 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c004390 con 0x7f215c15c6d0 2023-07-13T22:30:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.222+0000 7f2160999700 1 -- 172.21.15.60:0/74493305 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0036a0 con 0x7f215c15c6d0 2023-07-13T22:30:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.223+0000 7f2163bff700 1 -- 172.21.15.60:0/74493305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 msgr2=0x7f215c0cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.223+0000 7f2163bff700 1 --2- 172.21.15.60:0/74493305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 0x7f215c0cedf0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f215c0d2ba0 tx=0x7f214c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.223+0000 7f2163bff700 1 -- 172.21.15.60:0/74493305 shutdown_connections 2023-07-13T22:30:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.223+0000 7f2163bff700 1 --2- 172.21.15.60:0/74493305 >> v2:172.21.15.105:3300/0 conn(0x7f215c15c300 0x7f215c0b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.223+0000 7f2163bff700 1 --2- 172.21.15.60:0/74493305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 0x7f215c0cedf0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.223+0000 7f2163bff700 1 -- 172.21.15.60:0/74493305 >> 172.21.15.60:0/74493305 conn(0x7f215c0cf6c0 msgr2=0x7f215c0cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.223+0000 7f2163bff700 1 -- 172.21.15.60:0/74493305 shutdown_connections 2023-07-13T22:30:33.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.223+0000 7f2163bff700 1 -- 172.21.15.60:0/74493305 wait complete. 2023-07-13T22:30:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.225+0000 7f2163bff700 1 Processor -- start 2023-07-13T22:30:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.225+0000 7f2163bff700 1 -- start start 2023-07-13T22:30:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f2163bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f215c0b46c0 0x7f215c156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f2163bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f215c15c300 0x7f215c0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f2163bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 0x7f215c0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f2163bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c15cba0 con 0x7f215c15c6d0 2023-07-13T22:30:33.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f2163bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c0c67e0 con 0x7f215c15c300 2023-07-13T22:30:33.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f2163bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215c0cc140 con 0x7f215c0b46c0 2023-07-13T22:30:33.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216219c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 0x7f215c0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216219c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 0x7f215c0b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38544/0 (socket says 172.21.15.60:38544) 2023-07-13T22:30:33.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216219c700 1 -- 172.21.15.60:0/2512222287 learned_addr learned my addr 172.21.15.60:0/2512222287 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:33.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216119a700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f215c15c300 0x7f215c0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216199b700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f215c0b46c0 0x7f215c156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216219c700 1 -- 172.21.15.60:0/2512222287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f215c0b46c0 msgr2=0x7f215c156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216219c700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f215c0b46c0 0x7f215c156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216219c700 1 -- 172.21.15.60:0/2512222287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f215c15c300 msgr2=0x7f215c0cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216219c700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f215c15c300 0x7f215c0cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.226+0000 7f216219c700 1 -- 172.21.15.60:0/2512222287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c009550 con 0x7f215c15c6d0 2023-07-13T22:30:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.227+0000 7f216219c700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 0x7f215c0b36c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f215400b970 tx=0x7f215400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.227+0000 7f21527fc700 1 -- 172.21.15.60:0/2512222287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215400c9b0 con 0x7f215c15c6d0 2023-07-13T22:30:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.227+0000 7f21527fc700 1 -- 172.21.15.60:0/2512222287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215400cb10 con 0x7f215c15c6d0 2023-07-13T22:30:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.227+0000 7f21527fc700 1 -- 172.21.15.60:0/2512222287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154014440 con 0x7f215c15c6d0 2023-07-13T22:30:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.227+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c0cc420 con 0x7f215c15c6d0 2023-07-13T22:30:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.227+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215c0cc9e0 con 0x7f215c15c6d0 2023-07-13T22:30:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.228+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215c04e280 con 0x7f215c15c6d0 2023-07-13T22:30:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.229+0000 7f21527fc700 1 -- 172.21.15.60:0/2512222287 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f215401d3b0 con 0x7f215c15c6d0 2023-07-13T22:30:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.230+0000 7f21527fc700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214006ace0 0x7f214006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.230+0000 7f21527fc700 1 -- 172.21.15.60:0/2512222287 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2154088f50 con 0x7f215c15c6d0 2023-07-13T22:30:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.230+0000 7f216199b700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214006ace0 0x7f214006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.231+0000 7f216199b700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214006ace0 0x7f214006d190 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2158006fd0 tx=0x7f2158008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:33.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.235+0000 7f21527fc700 1 -- 172.21.15.60:0/2512222287 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f21540541f0 con 0x7f215c15c6d0 2023-07-13T22:30:33.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.484+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f215c0cfd70 con 0x7f215c15c6d0 2023-07-13T22:30:33.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.484+0000 7f21527fc700 1 -- 172.21.15.60:0/2512222287 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f2154019090 con 0x7f215c15c6d0 2023-07-13T22:30:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:33.464 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1446832449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1446832449},{"type":"v1","addr":"172.21.15.105:6835","nonce":1446832449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/462201037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":462201037},{"type":"v1","addr":"172.21.15.105:6834","nonce":462201037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/154242070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":154242070},{"type":"v1","addr":"172.21.15.60:6837","nonce":154242070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":8832,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/1872653914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1872653914},{"type":"v1","addr":"172.21.15.60:6836","nonce":1872653914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[]} 2023-07-13T22:30:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214006ace0 msgr2=0x7f214006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214006ace0 0x7f214006d190 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2158006fd0 tx=0x7f2158008040 comp rx=0 tx=0).stop 2023-07-13T22:30:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 msgr2=0x7f215c0b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 0x7f215c0b36c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f215400b970 tx=0x7f215400bd30 comp rx=0 tx=0).stop 2023-07-13T22:30:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 shutdown_connections 2023-07-13T22:30:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214006ace0 0x7f214006d190 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15c6d0 0x7f215c0b36c0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f215c0b46c0 0x7f215c156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 --2- 172.21.15.60:0/2512222287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f215c15c300 0x7f215c0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 >> 172.21.15.60:0/2512222287 conn(0x7f215c0cf6c0 msgr2=0x7f215c0732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 shutdown_connections 2023-07-13T22:30:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.487+0000 7f2163bff700 1 -- 172.21.15.60:0/2512222287 wait complete. 2023-07-13T22:30:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 79 2023-07-13T22:30:33.484 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:30:33.484 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-13T22:30:33.484 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:30:33.484 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:30:33.485 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:30:33.485 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-13T22:30:33.485 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:30:33.485 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:30:33.486 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:30:33.486 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-13T22:30:33.486 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:30:33.486 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:30:33.486 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:30:33.487 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-13T22:30:33.487 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:30:33.487 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:30:33.498 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:30:33.498 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:30:33.499 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:30:33.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:30:33.500 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:30:33.501 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:30:33.502 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:30:33.502 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:30:33.544 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:30:33.545 DEBUG:teuthology.parallel:result is None 2023-07-13T22:30:33.546 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:30:33.546 DEBUG:teuthology.parallel:result is None 2023-07-13T22:30:33.548 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:30:33.548 DEBUG:teuthology.parallel:result is None 2023-07-13T22:30:33.549 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:30:33.549 DEBUG:teuthology.parallel:result is None 2023-07-13T22:30:33.550 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-13T22:30:33.693 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:30:33.693 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:30:33.701 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:30:33.701 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:30:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.786+0000 7fd5b63ff700 1 Processor -- start 2023-07-13T22:30:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.786+0000 7fd5b63ff700 1 -- start start 2023-07-13T22:30:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.787+0000 7fd5b63ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 0x7fd5b00cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.787+0000 7fd5b63ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd5b00cfca0 0x7fd5b00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.787+0000 7fd5b63ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b00d0ac0 con 0x7fd5b00d3750 2023-07-13T22:30:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.787+0000 7fd5b63ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b00b87d0 con 0x7fd5b00cfca0 2023-07-13T22:30:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.787+0000 7fd5b63ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b00b4eb0 con 0x7fd5b00b8400 2023-07-13T22:30:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.787+0000 7fd5b499c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd5b00cfca0 0x7fd5b00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.787+0000 7fd5b499c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd5b00cfca0 0x7fd5b00d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38594/0 (socket says 172.21.15.60:38594) 2023-07-13T22:30:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.787+0000 7fd5b499c700 1 -- 172.21.15.60:0/3348294713 learned_addr learned my addr 172.21.15.60:0/3348294713 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:33.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.787+0000 7fd5affff700 1 --2- 172.21.15.60:0/3348294713 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 0x7fd5b00cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.788+0000 7fd5b499c700 1 -- 172.21.15.60:0/3348294713 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 msgr2=0x7fd5b00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.788+0000 7fd5b499c700 1 --2- 172.21.15.60:0/3348294713 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 0x7fd5b00cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.788+0000 7fd5b499c700 1 -- 172.21.15.60:0/3348294713 >> v1:172.21.15.105:6789/0 conn(0x7fd5b00d3750 legacy=0x7fd5b00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.788+0000 7fd5b499c700 1 -- 172.21.15.60:0/3348294713 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00b5190 con 0x7fd5b00cfca0 2023-07-13T22:30:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.788+0000 7fd5aeffd700 1 -- 172.21.15.60:0/3348294713 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459542447 0 0) 0x7fd5b00d0ac0 con 0x7fd5b00d3750 2023-07-13T22:30:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.788+0000 7fd5b499c700 1 --2- 172.21.15.60:0/3348294713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00cfca0 0x7fd5b00d0f50 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009e50 tx=0x7fd5a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5622ae8aa5827f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.788+0000 7fd5aeffd700 1 -- 172.21.15.60:0/3348294713 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800e070 con 0x7fd5b00cfca0 2023-07-13T22:30:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.788+0000 7fd5aeffd700 1 -- 172.21.15.60:0/3348294713 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a800b910 con 0x7fd5b00cfca0 2023-07-13T22:30:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.788+0000 7fd5aeffd700 1 -- 172.21.15.60:0/3348294713 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800bcc0 con 0x7fd5b00cfca0 2023-07-13T22:30:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.789+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3348294713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00cfca0 msgr2=0x7fd5b00d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.789+0000 7fd5b63ff700 1 --2- 172.21.15.60:0/3348294713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00cfca0 0x7fd5b00d0f50 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009e50 tx=0x7fd5a800b330 comp rx=0 tx=0).stop 2023-07-13T22:30:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.789+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3348294713 shutdown_connections 2023-07-13T22:30:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.789+0000 7fd5b63ff700 1 --2- 172.21.15.60:0/3348294713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00cfca0 0x7fd5b00d0f50 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.789+0000 7fd5b63ff700 1 --2- 172.21.15.60:0/3348294713 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 0x7fd5b00cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.789+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3348294713 >> 172.21.15.60:0/3348294713 conn(0x7fd5b006b390 msgr2=0x7fd5b006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.789+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3348294713 shutdown_connections 2023-07-13T22:30:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.789+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3348294713 wait complete. 2023-07-13T22:30:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.791+0000 7fd5b63ff700 1 Processor -- start 2023-07-13T22:30:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.791+0000 7fd5b63ff700 1 -- start start 2023-07-13T22:30:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b63ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 0x7fd5b00cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b63ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5b00cfca0 0x7fd5b0158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b63ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00d3750 0x7fd5b00c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b63ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b00cb2f0 con 0x7fd5b00d3750 2023-07-13T22:30:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b63ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b00cb5d0 con 0x7fd5b00cfca0 2023-07-13T22:30:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b63ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b00cb8b0 con 0x7fd5b00b8400 2023-07-13T22:30:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b499c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00d3750 0x7fd5b00c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b499c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00d3750 0x7fd5b00c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38610/0 (socket says 172.21.15.60:38610) 2023-07-13T22:30:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b499c700 1 -- 172.21.15.60:0/3625983824 learned_addr learned my addr 172.21.15.60:0/3625983824 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5affff700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 0x7fd5b00cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5af7fe700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5b00cfca0 0x7fd5b0158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b499c700 1 -- 172.21.15.60:0/3625983824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 msgr2=0x7fd5b00cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b499c700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 0x7fd5b00cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b499c700 1 -- 172.21.15.60:0/3625983824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5b00cfca0 msgr2=0x7fd5b0158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b499c700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5b00cfca0 0x7fd5b0158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5b499c700 1 -- 172.21.15.60:0/3625983824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a8009b30 con 0x7fd5b00d3750 2023-07-13T22:30:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.792+0000 7fd5af7fe700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5b00cfca0 0x7fd5b0158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.793+0000 7fd5b499c700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00d3750 0x7fd5b00c66b0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fd5b006c720 tx=0x7fd5a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.793+0000 7fd5acff9700 1 -- 172.21.15.60:0/3625983824 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8004930 con 0x7fd5b00d3750 2023-07-13T22:30:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.793+0000 7fd5acff9700 1 -- 172.21.15.60:0/3625983824 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a80099b0 con 0x7fd5b00d3750 2023-07-13T22:30:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.793+0000 7fd5acff9700 1 -- 172.21.15.60:0/3625983824 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a801c680 con 0x7fd5b00d3750 2023-07-13T22:30:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.793+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b00cbb90 con 0x7fd5b00d3750 2023-07-13T22:30:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.793+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b01576f0 con 0x7fd5b00d3750 2023-07-13T22:30:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.794+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b00b4a50 con 0x7fd5b00d3750 2023-07-13T22:30:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.795+0000 7fd5acff9700 1 -- 172.21.15.60:0/3625983824 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5a801c7e0 con 0x7fd5b00d3750 2023-07-13T22:30:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.796+0000 7fd5acff9700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd59c06ad10 0x7fd59c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.797+0000 7fd5acff9700 1 -- 172.21.15.60:0/3625983824 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd5a808a9b0 con 0x7fd5b00d3750 2023-07-13T22:30:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.797+0000 7fd5affff700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd59c06ad10 0x7fd59c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.797+0000 7fd5affff700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd59c06ad10 0x7fd59c06d1c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009b40 tx=0x7fd5a0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:33.801+0000 7fd5acff9700 1 -- 172.21.15.60:0/3625983824 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd5a8055d70 con 0x7fd5b00d3750 2023-07-13T22:30:34.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:34.027+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd5b00d1be0 con 0x7fd5b00d3750 2023-07-13T22:30:35.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.151+0000 7fd5acff9700 1 -- 172.21.15.60:0/3625983824 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd5a8059390 con 0x7fd5b00d3750 2023-07-13T22:30:35.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd59c06ad10 msgr2=0x7fd59c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd59c06ad10 0x7fd59c06d1c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009b40 tx=0x7fd5a0009210 comp rx=0 tx=0).stop 2023-07-13T22:30:35.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00d3750 msgr2=0x7fd5b00c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00d3750 0x7fd5b00c66b0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fd5b006c720 tx=0x7fd5a8004ad0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 shutdown_connections 2023-07-13T22:30:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd59c06ad10 0x7fd59c06d1c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5b00d3750 0x7fd5b00c66b0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd5b00b8400 0x7fd5b00cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 --2- 172.21.15.60:0/3625983824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5b00cfca0 0x7fd5b0158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 >> 172.21.15.60:0/3625983824 conn(0x7fd5b006b390 msgr2=0x7fd5b00d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 shutdown_connections 2023-07-13T22:30:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.154+0000 7fd5b63ff700 1 -- 172.21.15.60:0/3625983824 wait complete. 2023-07-13T22:30:35.135 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-07-13T22:30:35.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-13T22:30:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.384+0000 7f60359ff700 1 Processor -- start 2023-07-13T22:30:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.384+0000 7f60359ff700 1 -- start start 2023-07-13T22:30:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f60359ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6030154950 0x7f6030070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f60359ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6030070270 con 0x7f603015a820 2023-07-13T22:30:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f60359ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603015b2e0 con 0x7f60300c9450 2023-07-13T22:30:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f60359ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603015b5c0 con 0x7f6030154950 2023-07-13T22:30:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f602effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6030154950 0x7f6030070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f602e7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f603015a820 0x7f60300c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43570/0 (socket says 172.21.15.60:43570) 2023-07-13T22:30:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f602effd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6030154950 0x7f6030070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44556/0 (socket says 172.21.15.60:44556) 2023-07-13T22:30:35.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f602e7fc700 1 -- 172.21.15.60:0/866300481 learned_addr learned my addr 172.21.15.60:0/866300481 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f602effd700 1 -- 172.21.15.60:0/866300481 >> v1:172.21.15.105:6789/0 conn(0x7f60300c9450 legacy=0x7f60300c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f602effd700 1 -- 172.21.15.60:0/866300481 >> v1:172.21.15.60:6789/0 conn(0x7f603015a820 legacy=0x7f60300c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.387+0000 7f602effd700 1 -- 172.21.15.60:0/866300481 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300704c0 con 0x7f6030154950 2023-07-13T22:30:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.386+0000 7f602dffb700 1 -- 172.21.15.60:0/866300481 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285282654 0 0) 0x7f6030070270 con 0x7f603015a820 2023-07-13T22:30:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.387+0000 7f602effd700 1 --2- 172.21.15.60:0/866300481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6030154950 0x7f6030070cc0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f6020009840 tx=0x7f6020009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f831fdc5d3537df5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.387+0000 7f602dffb700 1 -- 172.21.15.60:0/866300481 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020004230 con 0x7f6030154950 2023-07-13T22:30:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.387+0000 7f602dffb700 1 -- 172.21.15.60:0/866300481 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020004390 con 0x7f6030154950 2023-07-13T22:30:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.387+0000 7f602dffb700 1 -- 172.21.15.60:0/866300481 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60200036a0 con 0x7f6030154950 2023-07-13T22:30:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.387+0000 7f60359ff700 1 -- 172.21.15.60:0/866300481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6030154950 msgr2=0x7f6030070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.387+0000 7f60359ff700 1 --2- 172.21.15.60:0/866300481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6030154950 0x7f6030070cc0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f6020009840 tx=0x7f6020009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.388+0000 7f60359ff700 1 -- 172.21.15.60:0/866300481 shutdown_connections 2023-07-13T22:30:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.388+0000 7f60359ff700 1 --2- 172.21.15.60:0/866300481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6030154950 0x7f6030070cc0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.388+0000 7f60359ff700 1 -- 172.21.15.60:0/866300481 >> 172.21.15.60:0/866300481 conn(0x7f60300cf410 msgr2=0x7f60300cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:35.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.388+0000 7f60359ff700 1 -- 172.21.15.60:0/866300481 shutdown_connections 2023-07-13T22:30:35.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.388+0000 7f60359ff700 1 -- 172.21.15.60:0/866300481 wait complete. 2023-07-13T22:30:35.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.390+0000 7f60359ff700 1 Processor -- start 2023-07-13T22:30:35.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.390+0000 7f60359ff700 1 -- start start 2023-07-13T22:30:35.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.391+0000 7f60359ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60300c9450 0x7f6030153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.391+0000 7f60359ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6030154950 0x7f60300ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.391+0000 7f60359ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603015a820 0x7f60300adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.391+0000 7f60359ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300aefb0 con 0x7f603015a820 2023-07-13T22:30:35.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.391+0000 7f60359ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6030154110 con 0x7f6030154950 2023-07-13T22:30:35.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.391+0000 7f60359ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60300ae3a0 con 0x7f60300c9450 2023-07-13T22:30:35.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603015a820 0x7f60300adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603015a820 0x7f60300adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44560/0 (socket says 172.21.15.60:44560) 2023-07-13T22:30:35.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602f7fe700 1 -- 172.21.15.60:0/1615033798 learned_addr learned my addr 172.21.15.60:0/1615033798 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:35.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602e7fc700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6030154950 0x7f60300ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602effd700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60300c9450 0x7f6030153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602f7fe700 1 -- 172.21.15.60:0/1615033798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60300c9450 msgr2=0x7f6030153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602f7fe700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60300c9450 0x7f6030153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602f7fe700 1 -- 172.21.15.60:0/1615033798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6030154950 msgr2=0x7f60300ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602f7fe700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6030154950 0x7f60300ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602f7fe700 1 -- 172.21.15.60:0/1615033798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60200094a0 con 0x7f603015a820 2023-07-13T22:30:35.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602effd700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60300c9450 0x7f6030153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:35.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.392+0000 7f602f7fe700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603015a820 0x7f60300adc80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f602800cc40 tx=0x7f602800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:35.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.393+0000 7f6017fff700 1 -- 172.21.15.60:0/1615033798 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800aba0 con 0x7f603015a820 2023-07-13T22:30:35.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.393+0000 7f6017fff700 1 -- 172.21.15.60:0/1615033798 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602800ad00 con 0x7f603015a820 2023-07-13T22:30:35.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.393+0000 7f6017fff700 1 -- 172.21.15.60:0/1615033798 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028012660 con 0x7f603015a820 2023-07-13T22:30:35.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.393+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60300ae680 con 0x7f603015a820 2023-07-13T22:30:35.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.393+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60300aec40 con 0x7f603015a820 2023-07-13T22:30:35.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.394+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60300cbc20 con 0x7f603015a820 2023-07-13T22:30:35.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.395+0000 7f6017fff700 1 -- 172.21.15.60:0/1615033798 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f60280127c0 con 0x7f603015a820 2023-07-13T22:30:35.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.396+0000 7f6017fff700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f601c06ace0 0x7f601c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.396+0000 7f6017fff700 1 -- 172.21.15.60:0/1615033798 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f60280899b0 con 0x7f603015a820 2023-07-13T22:30:35.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.396+0000 7f602effd700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f601c06ace0 0x7f601c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.397+0000 7f602effd700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f601c06ace0 0x7f601c06d190 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6020009b90 tx=0x7f6020008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:35.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.400+0000 7f6017fff700 1 -- 172.21.15.60:0/1615033798 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6028054d10 con 0x7f603015a820 2023-07-13T22:30:35.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.687+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f60300c43d0 con 0x7f603015a820 2023-07-13T22:30:35.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.688+0000 7f6017fff700 1 -- 172.21.15.60:0/1615033798 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6028016020 con 0x7f603015a820 2023-07-13T22:30:35.667 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:35.667 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:30:35.667 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-07-13T22:30:35.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.668 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-07-13T22:30:35.668 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw==", 2023-07-13T22:30:35.668 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.668 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:30:35.668 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:30:35.669 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:30:35.669 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.669 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.669 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-07-13T22:30:35.670 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g==", 2023-07-13T22:30:35.670 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.670 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:30:35.670 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:30:35.670 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:30:35.670 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.670 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.671 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-07-13T22:30:35.671 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA==", 2023-07-13T22:30:35.671 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.671 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:30:35.671 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:30:35.672 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:30:35.672 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.672 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.672 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-07-13T22:30:35.672 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ==", 2023-07-13T22:30:35.673 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.673 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:30:35.673 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:30:35.673 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:30:35.673 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.673 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.674 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-07-13T22:30:35.674 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ==", 2023-07-13T22:30:35.674 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.674 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:30:35.674 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:30:35.674 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.675 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.675 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-07-13T22:30:35.675 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw==", 2023-07-13T22:30:35.675 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.676 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:30:35.676 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:30:35.676 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.676 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.676 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-07-13T22:30:35.677 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q==", 2023-07-13T22:30:35.677 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.677 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:30:35.677 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:30:35.677 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.677 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.678 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-07-13T22:30:35.678 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ==", 2023-07-13T22:30:35.678 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.678 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:30:35.678 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:30:35.678 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.678 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.679 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-07-13T22:30:35.679 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg==", 2023-07-13T22:30:35.679 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.679 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:30:35.679 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:30:35.680 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.680 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.680 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-07-13T22:30:35.680 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA==", 2023-07-13T22:30:35.680 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.680 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:30:35.681 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:30:35.681 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.681 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-07-13T22:30:35.682 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw==", 2023-07-13T22:30:35.682 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.682 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:30:35.682 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:30:35.682 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.682 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.683 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-07-13T22:30:35.683 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ==", 2023-07-13T22:30:35.683 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.683 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:30:35.683 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:30:35.683 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.683 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.684 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-07-13T22:30:35.684 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w==", 2023-07-13T22:30:35.684 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.684 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:30:35.684 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2023-07-13T22:30:35.685 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-13T22:30:35.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.685 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-07-13T22:30:35.685 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw==", 2023-07-13T22:30:35.685 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.685 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:30:35.686 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:30:35.686 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:30:35.686 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:30:35.686 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.687 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-07-13T22:30:35.687 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg==", 2023-07-13T22:30:35.687 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.687 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:30:35.687 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:30:35.687 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:30:35.687 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:30:35.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.688 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-07-13T22:30:35.688 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q==", 2023-07-13T22:30:35.688 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:30:35.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:30:35.689 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:30:35.689 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:30:35.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.689 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-07-13T22:30:35.689 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBk7CgGBhAAaK88V4CEyrYfZx7wv+XG3g==", 2023-07-13T22:30:35.689 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.690 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:30:35.690 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:30:35.690 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:30:35.690 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.690 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.690 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-07-13T22:30:35.691 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ==", 2023-07-13T22:30:35.691 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.691 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:30:35.691 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:30:35.691 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:35.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:35.692 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-07-13T22:30:35.692 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ==", 2023-07-13T22:30:35.692 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:30:35.692 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:30:35.692 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:30:35.692 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:30:35.692 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.693 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:35.693 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:30:35.693 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:30:35.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f601c06ace0 msgr2=0x7f601c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f601c06ace0 0x7f601c06d190 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6020009b90 tx=0x7f6020008030 comp rx=0 tx=0).stop 2023-07-13T22:30:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603015a820 msgr2=0x7f60300adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603015a820 0x7f60300adc80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f602800cc40 tx=0x7f602800a060 comp rx=0 tx=0).stop 2023-07-13T22:30:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 shutdown_connections 2023-07-13T22:30:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f601c06ace0 0x7f601c06d190 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603015a820 0x7f60300adc80 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60300c9450 0x7f6030153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 --2- 172.21.15.60:0/1615033798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6030154950 0x7f60300ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.690+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 >> 172.21.15.60:0/1615033798 conn(0x7f60300cf410 msgr2=0x7f60300c3850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.691+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 shutdown_connections 2023-07-13T22:30:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.691+0000 7f60359ff700 1 -- 172.21.15.60:0/1615033798 wait complete. 2023-07-13T22:30:35.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:30:35.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.920+0000 7f41f25ff700 1 Processor -- start 2023-07-13T22:30:35.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.920+0000 7f41f25ff700 1 -- start start 2023-07-13T22:30:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.921+0000 7f41f25ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f41ec069ed0 0x7f41ec06a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.921+0000 7f41f25ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec0c9300 0x7f41ec0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.921+0000 7f41f25ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ec06a860 con 0x7f41ec158870 2023-07-13T22:30:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.921+0000 7f41f25ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ec0c3fc0 con 0x7f41ec069ed0 2023-07-13T22:30:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.921+0000 7f41f25ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0c42a0 con 0x7f41ec0c9300 2023-07-13T22:30:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.922+0000 7f41eb7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f41ec069ed0 0x7f41ec06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.922+0000 7f41eb7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f41ec069ed0 0x7f41ec06a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44574/0 (socket says 172.21.15.60:44574) 2023-07-13T22:30:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.922+0000 7f41eb7fe700 1 -- 172.21.15.60:0/1174761837 learned_addr learned my addr 172.21.15.60:0/1174761837 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:35.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.922+0000 7f41f0b9c700 1 --2- 172.21.15.60:0/1174761837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec0c9300 0x7f41ec0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.922+0000 7f41eb7fe700 1 -- 172.21.15.60:0/1174761837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec0c9300 msgr2=0x7f41ec0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.922+0000 7f41eb7fe700 1 --2- 172.21.15.60:0/1174761837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec0c9300 0x7f41ec0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.922+0000 7f41eb7fe700 1 -- 172.21.15.60:0/1174761837 >> v1:172.21.15.105:6789/0 conn(0x7f41ec158870 legacy=0x7f41ec0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.922+0000 7f41eb7fe700 1 -- 172.21.15.60:0/1174761837 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0c4580 con 0x7f41ec069ed0 2023-07-13T22:30:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.922+0000 7f41eb7fe700 1 --2- 172.21.15.60:0/1174761837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec069ed0 0x7f41ec06a320 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f41e000bb90 tx=0x7f41e000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce4641bb942b72a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41eaffd700 1 -- 172.21.15.60:0/1174761837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e000dd50 con 0x7f41ec069ed0 2023-07-13T22:30:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41eaffd700 1 -- 172.21.15.60:0/1174761837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e0010470 con 0x7f41ec069ed0 2023-07-13T22:30:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41eaffd700 1 -- 172.21.15.60:0/1174761837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0010800 con 0x7f41ec069ed0 2023-07-13T22:30:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41f25ff700 1 -- 172.21.15.60:0/1174761837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec069ed0 msgr2=0x7f41ec06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41f25ff700 1 --2- 172.21.15.60:0/1174761837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec069ed0 0x7f41ec06a320 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f41e000bb90 tx=0x7f41e000bea0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41f25ff700 1 -- 172.21.15.60:0/1174761837 shutdown_connections 2023-07-13T22:30:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41f25ff700 1 --2- 172.21.15.60:0/1174761837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec0c9300 0x7f41ec0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41f25ff700 1 --2- 172.21.15.60:0/1174761837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec069ed0 0x7f41ec06a320 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41f25ff700 1 -- 172.21.15.60:0/1174761837 >> 172.21.15.60:0/1174761837 conn(0x7f41ec0cf410 msgr2=0x7f41ec0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41f25ff700 1 -- 172.21.15.60:0/1174761837 shutdown_connections 2023-07-13T22:30:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.923+0000 7f41f25ff700 1 -- 172.21.15.60:0/1174761837 wait complete. 2023-07-13T22:30:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.925+0000 7f41f25ff700 1 Processor -- start 2023-07-13T22:30:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.925+0000 7f41f25ff700 1 -- start start 2023-07-13T22:30:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.926+0000 7f41f25ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec069ed0 0x7f41ec06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.926+0000 7f41f25ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec0c9300 0x7f41ec146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.926+0000 7f41f25ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41ec158870 0x7f41ec0ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.926+0000 7f41f25ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec06d930 con 0x7f41ec0c9300 2023-07-13T22:30:35.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.926+0000 7f41f25ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0ab8e0 con 0x7f41ec158870 2023-07-13T22:30:35.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.926+0000 7f41f25ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0abbc0 con 0x7f41ec069ed0 2023-07-13T22:30:35.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec0c9300 0x7f41ec146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41eb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec0c9300 0x7f41ec146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44580/0 (socket says 172.21.15.60:44580) 2023-07-13T22:30:35.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41eb7fe700 1 -- 172.21.15.60:0/2275423971 learned_addr learned my addr 172.21.15.60:0/2275423971 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:35.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41ebfff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec069ed0 0x7f41ec06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41eb7fe700 1 -- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec069ed0 msgr2=0x7f41ec06d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41f0b9c700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41ec158870 0x7f41ec0ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41eb7fe700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec069ed0 0x7f41ec06d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41eb7fe700 1 -- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41ec158870 msgr2=0x7f41ec0ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:35.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41eb7fe700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41ec158870 0x7f41ec0ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:35.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41eb7fe700 1 -- 172.21.15.60:0/2275423971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e000b7f0 con 0x7f41ec0c9300 2023-07-13T22:30:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41ebfff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec069ed0 0x7f41ec06d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41f0b9c700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41ec158870 0x7f41ec0ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:30:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41eb7fe700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec0c9300 0x7f41ec146220 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f41e000bee0 tx=0x7f41e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.927+0000 7f41e8ff9700 1 -- 172.21.15.60:0/2275423971 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0003b30 con 0x7f41ec0c9300 2023-07-13T22:30:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.928+0000 7f41e8ff9700 1 -- 172.21.15.60:0/2275423971 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e0003c90 con 0x7f41ec0c9300 2023-07-13T22:30:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.928+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec0abea0 con 0x7f41ec0c9300 2023-07-13T22:30:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.928+0000 7f41e8ff9700 1 -- 172.21.15.60:0/2275423971 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e00105e0 con 0x7f41ec0c9300 2023-07-13T22:30:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.928+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec06d570 con 0x7f41ec0c9300 2023-07-13T22:30:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.929+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ec0ac3c0 con 0x7f41ec0c9300 2023-07-13T22:30:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.930+0000 7f41e8ff9700 1 -- 172.21.15.60:0/2275423971 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41e0019400 con 0x7f41ec0c9300 2023-07-13T22:30:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.931+0000 7f41e8ff9700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f41d806ace0 0x7f41d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.931+0000 7f41e8ff9700 1 -- 172.21.15.60:0/2275423971 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f41e008a5b0 con 0x7f41ec0c9300 2023-07-13T22:30:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.931+0000 7f41ebfff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f41d806ace0 0x7f41d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.932+0000 7f41ebfff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f41d806ace0 0x7f41d806d190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f41dc000f60 tx=0x7f41dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:35.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:35.936+0000 7f41e8ff9700 1 -- 172.21.15.60:0/2275423971 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f41ec0ac3c0 con 0x7f41ec0c9300 2023-07-13T22:30:36.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.144+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41ec0ac3c0 con 0x7f41ec0c9300 2023-07-13T22:30:36.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.145+0000 7f41e8ff9700 1 -- 172.21.15.60:0/2275423971 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+14555 (secure 0 0 0) 0x7f41ec0ac3c0 con 0x7f41ec0c9300 2023-07-13T22:30:36.124 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:36.125 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":120,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:30:35.147545+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:30:36.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f41d806ace0 msgr2=0x7f41d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f41d806ace0 0x7f41d806d190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f41dc000f60 tx=0x7f41dc009040 comp rx=0 tx=0).stop 2023-07-13T22:30:36.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec0c9300 msgr2=0x7f41ec146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec0c9300 0x7f41ec146220 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f41e000bee0 tx=0x7f41e0004750 comp rx=0 tx=0).stop 2023-07-13T22:30:36.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 shutdown_connections 2023-07-13T22:30:36.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f41d806ace0 0x7f41d806d190 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41ec158870 0x7f41ec0ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f41ec069ed0 0x7f41ec06d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 --2- 172.21.15.60:0/2275423971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41ec0c9300 0x7f41ec146220 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 >> 172.21.15.60:0/2275423971 conn(0x7f41ec0cf410 msgr2=0x7f41ec06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:36.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 shutdown_connections 2023-07-13T22:30:36.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.148+0000 7f41f25ff700 1 -- 172.21.15.60:0/2275423971 wait complete. 2023-07-13T22:30:36.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:30:36.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:30:36.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.375+0000 7f8e9f3ff700 1 Processor -- start 2023-07-13T22:30:36.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.375+0000 7f8e9f3ff700 1 -- start start 2023-07-13T22:30:36.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.376+0000 7f8e9f3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e98070b30 0x7f8e980caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.376+0000 7f8e9f3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e980ce770 0x7f8e980b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.376+0000 7f8e9f3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9806bf40 con 0x7f8e9815c460 2023-07-13T22:30:36.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.376+0000 7f8e9f3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9806b9f0 con 0x7f8e980ce770 2023-07-13T22:30:36.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.376+0000 7f8e9f3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9806c750 con 0x7f8e98070b30 2023-07-13T22:30:36.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9d99c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e980ce770 0x7f8e980b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9d19b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8e9815c460 0x7f8e9806c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43576/0 (socket says 172.21.15.60:43576) 2023-07-13T22:30:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9d99c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e980ce770 0x7f8e980b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44596/0 (socket says 172.21.15.60:44596) 2023-07-13T22:30:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9d19b700 1 -- 172.21.15.60:0/2591921251 learned_addr learned my addr 172.21.15.60:0/2591921251 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9c99a700 1 --2- 172.21.15.60:0/2591921251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e98070b30 0x7f8e980caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9d99c700 1 -- 172.21.15.60:0/2591921251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e98070b30 msgr2=0x7f8e980caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9d99c700 1 --2- 172.21.15.60:0/2591921251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e98070b30 0x7f8e980caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9d99c700 1 -- 172.21.15.60:0/2591921251 >> v1:172.21.15.60:6789/0 conn(0x7f8e9815c460 legacy=0x7f8e9806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9d99c700 1 -- 172.21.15.60:0/2591921251 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e980cebf0 con 0x7f8e980ce770 2023-07-13T22:30:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e8ffff700 1 -- 172.21.15.60:0/2591921251 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201769532 0 0) 0x7f8e9806bf40 con 0x7f8e9815c460 2023-07-13T22:30:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.377+0000 7f8e9d99c700 1 --2- 172.21.15.60:0/2591921251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 0x7f8e980b23b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009d90 tx=0x7f8e9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdb05f9a16113ae server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e8ffff700 1 -- 172.21.15.60:0/2591921251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9000e070 con 0x7f8e980ce770 2023-07-13T22:30:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e8ffff700 1 -- 172.21.15.60:0/2591921251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9000bbd0 con 0x7f8e980ce770 2023-07-13T22:30:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e8ffff700 1 -- 172.21.15.60:0/2591921251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90012430 con 0x7f8e980ce770 2023-07-13T22:30:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/2591921251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 msgr2=0x7f8e980b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e9f3ff700 1 --2- 172.21.15.60:0/2591921251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 0x7f8e980b23b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009d90 tx=0x7f8e9000b330 comp rx=0 tx=0).stop 2023-07-13T22:30:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/2591921251 shutdown_connections 2023-07-13T22:30:36.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e9f3ff700 1 --2- 172.21.15.60:0/2591921251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 0x7f8e980b23b0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e9f3ff700 1 --2- 172.21.15.60:0/2591921251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e98070b30 0x7f8e980caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/2591921251 >> 172.21.15.60:0/2591921251 conn(0x7f8e9806e580 msgr2=0x7f8e9806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:36.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/2591921251 shutdown_connections 2023-07-13T22:30:36.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.378+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/2591921251 wait complete. 2023-07-13T22:30:36.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.380+0000 7f8e9f3ff700 1 Processor -- start 2023-07-13T22:30:36.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.380+0000 7f8e9f3ff700 1 -- start start 2023-07-13T22:30:36.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.381+0000 7f8e9f3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e98070b30 0x7f8e980b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.381+0000 7f8e9f3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 0x7f8e980c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.381+0000 7f8e9f3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e9815c460 0x7f8e980b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.381+0000 7f8e9f3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e980c4b70 con 0x7f8e980ce770 2023-07-13T22:30:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.381+0000 7f8e9f3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9815f220 con 0x7f8e98070b30 2023-07-13T22:30:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.381+0000 7f8e9f3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e980b3780 con 0x7f8e9815c460 2023-07-13T22:30:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.381+0000 7f8e9c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 0x7f8e980c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.381+0000 7f8e9c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 0x7f8e980c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44608/0 (socket says 172.21.15.60:44608) 2023-07-13T22:30:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.381+0000 7f8e9c99a700 1 -- 172.21.15.60:0/1501504727 learned_addr learned my addr 172.21.15.60:0/1501504727 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e9d99c700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e9815c460 0x7f8e980b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e9d19b700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e98070b30 0x7f8e980b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e9c99a700 1 -- 172.21.15.60:0/1501504727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e9815c460 msgr2=0x7f8e980b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e9c99a700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e9815c460 0x7f8e980b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e9c99a700 1 -- 172.21.15.60:0/1501504727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e98070b30 msgr2=0x7f8e980b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e9c99a700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e98070b30 0x7f8e980b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e9c99a700 1 -- 172.21.15.60:0/1501504727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e90009a70 con 0x7f8e980ce770 2023-07-13T22:30:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e9c99a700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 0x7f8e980c6f10 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800d800 tx=0x7f8e8800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1501504727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e880125f0 con 0x7f8e980ce770 2023-07-13T22:30:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1501504727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e88012750 con 0x7f8e980ce770 2023-07-13T22:30:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1501504727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88012b00 con 0x7f8e980ce770 2023-07-13T22:30:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.382+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e981f5cf0 con 0x7f8e980ce770 2023-07-13T22:30:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.383+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e980b3cb0 con 0x7f8e980ce770 2023-07-13T22:30:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.384+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9804e210 con 0x7f8e980ce770 2023-07-13T22:30:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.385+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1501504727 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e88012c60 con 0x7f8e980ce770 2023-07-13T22:30:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.385+0000 7f8e8dffb700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.386+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1501504727 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8e88089480 con 0x7f8e980ce770 2023-07-13T22:30:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.386+0000 7f8e9d19b700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.386+0000 7f8e9d19b700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8e94000f60 tx=0x7f8e94007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:36.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.390+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1501504727 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8e88054870 con 0x7f8e980ce770 2023-07-13T22:30:36.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.598+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8e9806f460 con 0x7f8e980ce770 2023-07-13T22:30:36.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.599+0000 7f8e8dffb700 1 -- 172.21.15.60:0/1501504727 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v120) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e88057e90 con 0x7f8e980ce770 2023-07-13T22:30:36.578 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:30:36.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 msgr2=0x7f8e7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8e94000f60 tx=0x7f8e94007be0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 msgr2=0x7f8e980c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 0x7f8e980c6f10 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800d800 tx=0x7f8e8800dbc0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 shutdown_connections 2023-07-13T22:30:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e7c06ace0 0x7f8e7c06d190 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e9815c460 0x7f8e980b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e98070b30 0x7f8e980b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 --2- 172.21.15.60:0/1501504727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e980ce770 0x7f8e980c6f10 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 >> 172.21.15.60:0/1501504727 conn(0x7f8e9806e580 msgr2=0x7f8e980cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 shutdown_connections 2023-07-13T22:30:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.601+0000 7f8e9f3ff700 1 -- 172.21.15.60:0/1501504727 wait complete. 2023-07-13T22:30:36.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:30:36.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.827+0000 7f552efff700 1 Processor -- start 2023-07-13T22:30:36.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.828+0000 7f552efff700 1 -- start start 2023-07-13T22:30:36.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552efff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f55280add00 0x7f552806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552efff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f552806a5e0 0x7f55280b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552efff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55280c8e90 con 0x7f55280ca9f0 2023-07-13T22:30:36.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552efff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552806d3b0 con 0x7f55280add00 2023-07-13T22:30:36.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552efff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55280ae0d0 con 0x7f552806a5e0 2023-07-13T22:30:36.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552cd9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f55280add00 0x7f552806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552cd9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f55280add00 0x7f552806ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44620/0 (socket says 172.21.15.60:44620) 2023-07-13T22:30:36.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552cd9b700 1 -- 172.21.15.60:0/3232980539 learned_addr learned my addr 172.21.15.60:0/3232980539 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:36.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552d59c700 1 --2- 172.21.15.60:0/3232980539 >> v2:172.21.15.105:3300/0 conn(0x7f552806a5e0 0x7f55280b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552cd9b700 1 -- 172.21.15.60:0/3232980539 >> v2:172.21.15.105:3300/0 conn(0x7f552806a5e0 msgr2=0x7f55280b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:36.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.829+0000 7f552cd9b700 1 --2- 172.21.15.60:0/3232980539 >> v2:172.21.15.105:3300/0 conn(0x7f552806a5e0 0x7f55280b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.830+0000 7f552cd9b700 1 -- 172.21.15.60:0/3232980539 >> v1:172.21.15.105:6789/0 conn(0x7f55280ca9f0 legacy=0x7f5528069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.830+0000 7f552cd9b700 1 -- 172.21.15.60:0/3232980539 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280c90e0 con 0x7f55280add00 2023-07-13T22:30:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.830+0000 7f552d59c700 1 --2- 172.21.15.60:0/3232980539 >> v2:172.21.15.105:3300/0 conn(0x7f552806a5e0 0x7f55280b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.830+0000 7f552cd9b700 1 --2- 172.21.15.60:0/3232980539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55280add00 0x7f552806ce70 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f55280701b0 tx=0x7f551c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0c7443471b441b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.830+0000 7f55277fe700 1 -- 172.21.15.60:0/3232980539 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c00b870 con 0x7f55280add00 2023-07-13T22:30:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.830+0000 7f55277fe700 1 -- 172.21.15.60:0/3232980539 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c00b9d0 con 0x7f55280add00 2023-07-13T22:30:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.830+0000 7f55277fe700 1 -- 172.21.15.60:0/3232980539 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c00bd60 con 0x7f55280add00 2023-07-13T22:30:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.831+0000 7f552efff700 1 -- 172.21.15.60:0/3232980539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55280add00 msgr2=0x7f552806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.831+0000 7f552efff700 1 --2- 172.21.15.60:0/3232980539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55280add00 0x7f552806ce70 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f55280701b0 tx=0x7f551c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.831+0000 7f552efff700 1 -- 172.21.15.60:0/3232980539 shutdown_connections 2023-07-13T22:30:36.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.831+0000 7f552efff700 1 --2- 172.21.15.60:0/3232980539 >> v2:172.21.15.105:3300/0 conn(0x7f552806a5e0 0x7f55280b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.831+0000 7f552efff700 1 --2- 172.21.15.60:0/3232980539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55280add00 0x7f552806ce70 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.831+0000 7f552efff700 1 -- 172.21.15.60:0/3232980539 >> 172.21.15.60:0/3232980539 conn(0x7f55280cbf50 msgr2=0x7f55280ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:36.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.831+0000 7f552efff700 1 -- 172.21.15.60:0/3232980539 shutdown_connections 2023-07-13T22:30:36.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.831+0000 7f552efff700 1 -- 172.21.15.60:0/3232980539 wait complete. 2023-07-13T22:30:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.832+0000 7f552efff700 1 Processor -- start 2023-07-13T22:30:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.833+0000 7f552efff700 1 -- start start 2023-07-13T22:30:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552efff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f552806a5e0 0x7f55280c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552efff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55280add00 0x7f55280b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552efff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55280ca9f0 0x7f55280aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552efff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55280b11e0 con 0x7f552806a5e0 2023-07-13T22:30:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552efff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55280ae2a0 con 0x7f55280add00 2023-07-13T22:30:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552efff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55280aeb40 con 0x7f55280ca9f0 2023-07-13T22:30:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552cd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f552806a5e0 0x7f55280c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552cd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f552806a5e0 0x7f55280c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44628/0 (socket says 172.21.15.60:44628) 2023-07-13T22:30:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552cd9b700 1 -- 172.21.15.60:0/1190889689 learned_addr learned my addr 172.21.15.60:0/1190889689 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552d59c700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55280ca9f0 0x7f55280aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f5527fff700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55280add00 0x7f55280b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552cd9b700 1 -- 172.21.15.60:0/1190889689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55280ca9f0 msgr2=0x7f55280aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552cd9b700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55280ca9f0 0x7f55280aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552cd9b700 1 -- 172.21.15.60:0/1190889689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55280add00 msgr2=0x7f55280b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552cd9b700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55280add00 0x7f55280b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.834+0000 7f552cd9b700 1 -- 172.21.15.60:0/1190889689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c009550 con 0x7f552806a5e0 2023-07-13T22:30:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.835+0000 7f552cd9b700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f552806a5e0 0x7f55280c4110 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f5528071d20 tx=0x7f551c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.835+0000 7f55257fa700 1 -- 172.21.15.60:0/1190889689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c004540 con 0x7f552806a5e0 2023-07-13T22:30:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.835+0000 7f55257fa700 1 -- 172.21.15.60:0/1190889689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c0036a0 con 0x7f552806a5e0 2023-07-13T22:30:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.835+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55281505a0 con 0x7f552806a5e0 2023-07-13T22:30:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.835+0000 7f55257fa700 1 -- 172.21.15.60:0/1190889689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c003a30 con 0x7f552806a5e0 2023-07-13T22:30:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.835+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55280ae4f0 con 0x7f552806a5e0 2023-07-13T22:30:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.836+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552804e210 con 0x7f552806a5e0 2023-07-13T22:30:36.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.837+0000 7f55257fa700 1 -- 172.21.15.60:0/1190889689 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f551c021070 con 0x7f552806a5e0 2023-07-13T22:30:36.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.838+0000 7f55257fa700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f550c06ace0 0x7f550c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:36.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.838+0000 7f55257fa700 1 -- 172.21.15.60:0/1190889689 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f551c08a870 con 0x7f552806a5e0 2023-07-13T22:30:36.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.838+0000 7f5527fff700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f550c06ace0 0x7f550c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:36.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.839+0000 7f5527fff700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f550c06ace0 0x7f550c06d190 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5514000f60 tx=0x7f5514009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:36.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:36.842+0000 7f55257fa700 1 -- 172.21.15.60:0/1190889689 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f551c055d00 con 0x7f552806a5e0 2023-07-13T22:30:37.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.044+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f552806a3c0 con 0x7f552806a5e0 2023-07-13T22:30:37.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.045+0000 7f55257fa700 1 -- 172.21.15.60:0/1190889689 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f551c059320 con 0x7f552806a5e0 2023-07-13T22:30:37.024 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:30:37.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f550c06ace0 msgr2=0x7f550c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f550c06ace0 0x7f550c06d190 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5514000f60 tx=0x7f5514009040 comp rx=0 tx=0).stop 2023-07-13T22:30:37.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f552806a5e0 msgr2=0x7f55280c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f552806a5e0 0x7f55280c4110 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f5528071d20 tx=0x7f551c004750 comp rx=0 tx=0).stop 2023-07-13T22:30:37.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 shutdown_connections 2023-07-13T22:30:37.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f550c06ace0 0x7f550c06d190 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f552806a5e0 0x7f55280c4110 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55280ca9f0 0x7f55280aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 --2- 172.21.15.60:0/1190889689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55280add00 0x7f55280b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 >> 172.21.15.60:0/1190889689 conn(0x7f55280cbf50 msgr2=0x7f55280702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:37.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 shutdown_connections 2023-07-13T22:30:37.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.047+0000 7f552efff700 1 -- 172.21.15.60:0/1190889689 wait complete. 2023-07-13T22:30:37.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:30:37.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.269+0000 7fc3c93ff700 1 Processor -- start 2023-07-13T22:30:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.270+0000 7fc3c93ff700 1 -- start start 2023-07-13T22:30:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.271+0000 7fc3c93ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc3c40d35c0 0x7fc3c40d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.271+0000 7fc3c93ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c40d09d0 con 0x7fc3c40d13f0 2023-07-13T22:30:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.271+0000 7fc3c93ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c40cda10 con 0x7fc3c40b8570 2023-07-13T22:30:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.271+0000 7fc3c93ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c40d3b30 con 0x7fc3c40d35c0 2023-07-13T22:30:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.271+0000 7fc3bbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc3c40d35c0 0x7fc3c40d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.271+0000 7fc3c37fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc3c40d13f0 0x7fc3c415c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43590/0 (socket says 172.21.15.60:43590) 2023-07-13T22:30:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.271+0000 7fc3c37fe700 1 -- 172.21.15.60:0/2581696924 learned_addr learned my addr 172.21.15.60:0/2581696924 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.271+0000 7fc3c27fc700 1 -- 172.21.15.60:0/2581696924 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029023679 0 0) 0x7fc3c40d09d0 con 0x7fc3c40d13f0 2023-07-13T22:30:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.272+0000 7fc3bbfff700 1 -- 172.21.15.60:0/2581696924 >> v1:172.21.15.105:6789/0 conn(0x7fc3c40b8570 legacy=0x7fc3c40ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.272+0000 7fc3bbfff700 1 -- 172.21.15.60:0/2581696924 >> v1:172.21.15.60:6789/0 conn(0x7fc3c40d13f0 legacy=0x7fc3c415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.272+0000 7fc3bbfff700 1 -- 172.21.15.60:0/2581696924 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40cd450 con 0x7fc3c40d35c0 2023-07-13T22:30:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.272+0000 7fc3bbfff700 1 --2- 172.21.15.60:0/2581696924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d35c0 0x7fc3c40d0eb0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009840 tx=0x7fc3ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1008f7a13983dba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.272+0000 7fc3c27fc700 1 -- 172.21.15.60:0/2581696924 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac011830 con 0x7fc3c40d35c0 2023-07-13T22:30:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.272+0000 7fc3c27fc700 1 -- 172.21.15.60:0/2581696924 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac011990 con 0x7fc3c40d35c0 2023-07-13T22:30:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.272+0000 7fc3c27fc700 1 -- 172.21.15.60:0/2581696924 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac011d20 con 0x7fc3c40d35c0 2023-07-13T22:30:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.272+0000 7fc3c93ff700 1 -- 172.21.15.60:0/2581696924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d35c0 msgr2=0x7fc3c40d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.273+0000 7fc3c93ff700 1 --2- 172.21.15.60:0/2581696924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d35c0 0x7fc3c40d0eb0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009840 tx=0x7fc3ac009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.273+0000 7fc3c93ff700 1 -- 172.21.15.60:0/2581696924 shutdown_connections 2023-07-13T22:30:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.273+0000 7fc3c93ff700 1 --2- 172.21.15.60:0/2581696924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d35c0 0x7fc3c40d0eb0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.273+0000 7fc3c93ff700 1 -- 172.21.15.60:0/2581696924 >> 172.21.15.60:0/2581696924 conn(0x7fc3c4071600 msgr2=0x7fc3c40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.273+0000 7fc3c93ff700 1 -- 172.21.15.60:0/2581696924 shutdown_connections 2023-07-13T22:30:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.273+0000 7fc3c93ff700 1 -- 172.21.15.60:0/2581696924 wait complete. 2023-07-13T22:30:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.274+0000 7fc3c93ff700 1 Processor -- start 2023-07-13T22:30:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.275+0000 7fc3c93ff700 1 -- start start 2023-07-13T22:30:37.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3c93ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c40b8570 0x7fc3c4159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3c93ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d13f0 0x7fc3c40c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3c93ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c40d35c0 0x7fc3c40b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3c93ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c41f2630 con 0x7fc3c40d13f0 2023-07-13T22:30:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3c93ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c41f1da0 con 0x7fc3c40b8570 2023-07-13T22:30:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3c93ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c415a030 con 0x7fc3c40d35c0 2023-07-13T22:30:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3bbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d13f0 0x7fc3c40c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3bbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d13f0 0x7fc3c40c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44652/0 (socket says 172.21.15.60:44652) 2023-07-13T22:30:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3bbfff700 1 -- 172.21.15.60:0/3200763605 learned_addr learned my addr 172.21.15.60:0/3200763605 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3c2ffd700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c40b8570 0x7fc3c4159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3c37fe700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c40d35c0 0x7fc3c40b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3bbfff700 1 -- 172.21.15.60:0/3200763605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c40d35c0 msgr2=0x7fc3c40b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3bbfff700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c40d35c0 0x7fc3c40b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3bbfff700 1 -- 172.21.15.60:0/3200763605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c40b8570 msgr2=0x7fc3c4159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3bbfff700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c40b8570 0x7fc3c4159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.276+0000 7fc3bbfff700 1 -- 172.21.15.60:0/3200763605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac0094a0 con 0x7fc3c40d13f0 2023-07-13T22:30:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.277+0000 7fc3bbfff700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d13f0 0x7fc3c40c7340 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0090b0 tx=0x7fc3ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.277+0000 7fc3bb7fe700 1 -- 172.21.15.60:0/3200763605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0036a0 con 0x7fc3c40d13f0 2023-07-13T22:30:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.277+0000 7fc3bb7fe700 1 -- 172.21.15.60:0/3200763605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac01f070 con 0x7fc3c40d13f0 2023-07-13T22:30:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.277+0000 7fc3bb7fe700 1 -- 172.21.15.60:0/3200763605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac003a20 con 0x7fc3c40d13f0 2023-07-13T22:30:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.277+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c41f1fb0 con 0x7fc3c40d13f0 2023-07-13T22:30:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.277+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c40c7c40 con 0x7fc3c40d13f0 2023-07-13T22:30:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.278+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c40cea90 con 0x7fc3c40d13f0 2023-07-13T22:30:37.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.279+0000 7fc3bb7fe700 1 -- 172.21.15.60:0/3200763605 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3ac003b80 con 0x7fc3c40d13f0 2023-07-13T22:30:37.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.280+0000 7fc3bb7fe700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3a406ace0 0x7fc3a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.280+0000 7fc3bb7fe700 1 -- 172.21.15.60:0/3200763605 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc3ac021070 con 0x7fc3c40d13f0 2023-07-13T22:30:37.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.281+0000 7fc3c2ffd700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3a406ace0 0x7fc3a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.281+0000 7fc3c2ffd700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3a406ace0 0x7fc3a406d190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc3c415a9f0 tx=0x7fc3b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:37.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.285+0000 7fc3bb7fe700 1 -- 172.21.15.60:0/3200763605 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc3ac055f80 con 0x7fc3c40d13f0 2023-07-13T22:30:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.492+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc3c40d2110 con 0x7fc3c40d13f0 2023-07-13T22:30:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.493+0000 7fc3bb7fe700 1 -- 172.21.15.60:0/3200763605 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v120) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc3ac015090 con 0x7fc3c40d13f0 2023-07-13T22:30:37.472 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:37.472 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:30:37.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:37.473 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:30:37.473 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:30:37.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:37.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:37.473 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:30:37.473 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:30:37.473 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:37.474 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:30:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3a406ace0 msgr2=0x7fc3a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3a406ace0 0x7fc3a406d190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc3c415a9f0 tx=0x7fc3b4009040 comp rx=0 tx=0).stop 2023-07-13T22:30:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d13f0 msgr2=0x7fc3c40c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d13f0 0x7fc3c40c7340 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0090b0 tx=0x7fc3ac004750 comp rx=0 tx=0).stop 2023-07-13T22:30:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 shutdown_connections 2023-07-13T22:30:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3a406ace0 0x7fc3a406d190 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3c40d35c0 0x7fc3c40b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3c40b8570 0x7fc3c4159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 --2- 172.21.15.60:0/3200763605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c40d13f0 0x7fc3c40c7340 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 >> 172.21.15.60:0/3200763605 conn(0x7fc3c4071600 msgr2=0x7fc3c4160720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:37.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 shutdown_connections 2023-07-13T22:30:37.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.495+0000 7fc3c93ff700 1 -- 172.21.15.60:0/3200763605 wait complete. 2023-07-13T22:30:37.492 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:30:37.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:30:37.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.721+0000 7f4c037ff700 1 Processor -- start 2023-07-13T22:30:37.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.721+0000 7f4c037ff700 1 -- start start 2023-07-13T22:30:37.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c037ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4bfc15c6e0 0x7f4bfc15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c037ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc06cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c037ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc15cab0 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c037ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc0cafd0 con 0x7f4bfc15c6e0 2023-07-13T22:30:37.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c037ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0caa50 con 0x7f4bfc0ce2c0 2023-07-13T22:30:37.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c0159b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4bfc15c6e0 0x7f4bfc15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c00d9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4bfc0c8da0 0x7f4bfc0cdbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43592/0 (socket says 172.21.15.60:43592) 2023-07-13T22:30:37.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c0159b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4bfc15c6e0 0x7f4bfc15d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44666/0 (socket says 172.21.15.60:44666) 2023-07-13T22:30:37.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c00d9a700 1 -- 172.21.15.60:0/1774131588 learned_addr learned my addr 172.21.15.60:0/1774131588 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:37.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c01d9c700 1 --2- 172.21.15.60:0/1774131588 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc06cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c0159b700 1 -- 172.21.15.60:0/1774131588 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 msgr2=0x7f4bfc06cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:37.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c0159b700 1 --2- 172.21.15.60:0/1774131588 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc06cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c0159b700 1 -- 172.21.15.60:0/1774131588 >> v1:172.21.15.60:6789/0 conn(0x7f4bfc0c8da0 legacy=0x7f4bfc0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c0159b700 1 -- 172.21.15.60:0/1774131588 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc0ca820 con 0x7f4bfc15c6e0 2023-07-13T22:30:37.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.723+0000 7f4c01d9c700 1 --2- 172.21.15.60:0/1774131588 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc06cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:37.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.724+0000 7f4c0159b700 1 --2- 172.21.15.60:0/1774131588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc15c6e0 0x7f4bfc15d6c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009840 tx=0x7f4bf8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd8c21817fe26fe server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:37.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.724+0000 7f4bf3fff700 1 -- 172.21.15.60:0/1774131588 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8011830 con 0x7f4bfc15c6e0 2023-07-13T22:30:37.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.724+0000 7f4bf3fff700 1 -- 172.21.15.60:0/1774131588 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf8011990 con 0x7f4bfc15c6e0 2023-07-13T22:30:37.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.724+0000 7f4bf3fff700 1 -- 172.21.15.60:0/1774131588 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8011d20 con 0x7f4bfc15c6e0 2023-07-13T22:30:37.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.724+0000 7f4c037ff700 1 -- 172.21.15.60:0/1774131588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc15c6e0 msgr2=0x7f4bfc15d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.724+0000 7f4c037ff700 1 --2- 172.21.15.60:0/1774131588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc15c6e0 0x7f4bfc15d6c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009840 tx=0x7f4bf8009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:37.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.724+0000 7f4c037ff700 1 -- 172.21.15.60:0/1774131588 shutdown_connections 2023-07-13T22:30:37.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.724+0000 7f4c037ff700 1 --2- 172.21.15.60:0/1774131588 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc06cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.724+0000 7f4c037ff700 1 --2- 172.21.15.60:0/1774131588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc15c6e0 0x7f4bfc15d6c0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.725+0000 7f4c037ff700 1 -- 172.21.15.60:0/1774131588 >> 172.21.15.60:0/1774131588 conn(0x7f4bfc0cf3b0 msgr2=0x7f4bfc0cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:37.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.725+0000 7f4c037ff700 1 -- 172.21.15.60:0/1774131588 shutdown_connections 2023-07-13T22:30:37.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.725+0000 7f4c037ff700 1 -- 172.21.15.60:0/1774131588 wait complete. 2023-07-13T22:30:37.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.727+0000 7f4c037ff700 1 Processor -- start 2023-07-13T22:30:37.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.727+0000 7f4c037ff700 1 -- start start 2023-07-13T22:30:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.728+0000 7f4c037ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc0c8da0 0x7f4bfc15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.728+0000 7f4c037ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc14c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.728+0000 7f4c037ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bfc15c6e0 0x7f4bfc06d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.728+0000 7f4c037ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15bbd0 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.728+0000 7f4c037ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15af50 con 0x7f4bfc15c6e0 2023-07-13T22:30:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.728+0000 7f4c037ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15b230 con 0x7f4bfc0ce2c0 2023-07-13T22:30:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c0159b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc0c8da0 0x7f4bfc15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c0159b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc0c8da0 0x7f4bfc15a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44672/0 (socket says 172.21.15.60:44672) 2023-07-13T22:30:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c0159b700 1 -- 172.21.15.60:0/1312268686 learned_addr learned my addr 172.21.15.60:0/1312268686 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:37.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c00d9a700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc14c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c01d9c700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bfc15c6e0 0x7f4bfc06d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c0159b700 1 -- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 msgr2=0x7f4bfc14c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c0159b700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc14c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c0159b700 1 -- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bfc15c6e0 msgr2=0x7f4bfc06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c0159b700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bfc15c6e0 0x7f4bfc06d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c0159b700 1 -- 172.21.15.60:0/1312268686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf80094a0 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c00d9a700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc14c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:37.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c01d9c700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bfc15c6e0 0x7f4bfc06d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:37.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.729+0000 7f4c0159b700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc0c8da0 0x7f4bfc15a720 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009b90 tx=0x7f4bf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:37.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.730+0000 7f4bf1ffb700 1 -- 172.21.15.60:0/1312268686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf80036a0 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.730+0000 7f4bf1ffb700 1 -- 172.21.15.60:0/1312268686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf801f070 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.730+0000 7f4bf1ffb700 1 -- 172.21.15.60:0/1312268686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8003a40 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.730+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc15b510 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.730+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc15ac60 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.731+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc15be40 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.732+0000 7f4bf1ffb700 1 -- 172.21.15.60:0/1312268686 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4bf8003ba0 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.733+0000 7f4bf1ffb700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4be006ace0 0x7f4be006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.733+0000 7f4bf1ffb700 1 -- 172.21.15.60:0/1312268686 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4bf8021070 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.733+0000 7f4c00d9a700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4be006ace0 0x7f4be006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:37.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.734+0000 7f4c00d9a700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4be006ace0 0x7f4be006d190 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0a7980 tx=0x7f4bec007e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:37.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.737+0000 7f4bf1ffb700 1 -- 172.21.15.60:0/1312268686 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4bf8055fa0 con 0x7f4bfc0c8da0 2023-07-13T22:30:37.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:37.942+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4bfc04e210 con 0x7f4bfc0c8da0 2023-07-13T22:30:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.164+0000 7f4bf1ffb700 1 -- 172.21.15.60:0/1312268686 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v121) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4bf80150b0 con 0x7f4bfc0c8da0 2023-07-13T22:30:38.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4be006ace0 msgr2=0x7f4be006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:38.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4be006ace0 0x7f4be006d190 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0a7980 tx=0x7f4bec007e30 comp rx=0 tx=0).stop 2023-07-13T22:30:38.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc0c8da0 msgr2=0x7f4bfc15a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:38.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc0c8da0 0x7f4bfc15a720 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009b90 tx=0x7f4bf8004750 comp rx=0 tx=0).stop 2023-07-13T22:30:38.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 shutdown_connections 2023-07-13T22:30:38.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4be006ace0 0x7f4be006d190 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:38.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bfc0c8da0 0x7f4bfc15a720 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:38.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bfc15c6e0 0x7f4bfc06d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:38.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 --2- 172.21.15.60:0/1312268686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bfc0ce2c0 0x7f4bfc14c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:38.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 >> 172.21.15.60:0/1312268686 conn(0x7f4bfc0cf3b0 msgr2=0x7f4bfc0c42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:38.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 shutdown_connections 2023-07-13T22:30:38.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.168+0000 7f4c037ff700 1 -- 172.21.15.60:0/1312268686 wait complete. 2023-07-13T22:30:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:30:38.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:30:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.399+0000 7f23f53ff700 1 Processor -- start 2023-07-13T22:30:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.399+0000 7f23f53ff700 1 -- start start 2023-07-13T22:30:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.401+0000 7f23f53ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23f0159ea0 0x7f23f00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:38.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.401+0000 7f23f53ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 0x7f23f00d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:38.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.401+0000 7f23f53ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f015d200 con 0x7f23f00d37c0 2023-07-13T22:30:38.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.401+0000 7f23f53ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f00d2e60 con 0x7f23f0159ea0 2023-07-13T22:30:38.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.401+0000 7f23f53ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f00d2b50 con 0x7f23f015c9b0 2023-07-13T22:30:38.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.401+0000 7f23eeffd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23f0159ea0 0x7f23f00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:38.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.401+0000 7f23ee7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 0x7f23f00d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:38.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.401+0000 7f23ef7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f23f00d37c0 0x7f23f00d0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:50938/0 (socket says 172.21.15.60:50938) 2023-07-13T22:30:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.401+0000 7f23ef7fe700 1 -- 172.21.15.60:0/1855168680 learned_addr learned my addr 172.21.15.60:0/1855168680 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23edffb700 1 -- 172.21.15.60:0/1855168680 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868151152 0 0) 0x7f23f015d200 con 0x7f23f00d37c0 2023-07-13T22:30:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23edffb700 1 -- 172.21.15.60:0/1855168680 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23d80035b0 con 0x7f23f00d37c0 2023-07-13T22:30:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23ee7fc700 1 -- 172.21.15.60:0/1855168680 >> v2:172.21.15.105:3300/0 conn(0x7f23f0159ea0 msgr2=0x7f23f00cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23ee7fc700 1 --2- 172.21.15.60:0/1855168680 >> v2:172.21.15.105:3300/0 conn(0x7f23f0159ea0 0x7f23f00cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23ee7fc700 1 -- 172.21.15.60:0/1855168680 >> v1:172.21.15.105:6789/0 conn(0x7f23f00d37c0 legacy=0x7f23f00d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23ee7fc700 1 -- 172.21.15.60:0/1855168680 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f015d4e0 con 0x7f23f015c9b0 2023-07-13T22:30:38.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23eeffd700 1 --2- 172.21.15.60:0/1855168680 >> v2:172.21.15.105:3300/0 conn(0x7f23f0159ea0 0x7f23f00cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:38.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23ee7fc700 1 --2- 172.21.15.60:0/1855168680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 0x7f23f00d3280 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009840 tx=0x7f23e4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540dfc91744ad360 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:38.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23edffb700 1 -- 172.21.15.60:0/1855168680 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4011830 con 0x7f23f015c9b0 2023-07-13T22:30:38.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23edffb700 1 -- 172.21.15.60:0/1855168680 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e4011990 con 0x7f23f015c9b0 2023-07-13T22:30:38.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.402+0000 7f23edffb700 1 -- 172.21.15.60:0/1855168680 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4011d00 con 0x7f23f015c9b0 2023-07-13T22:30:38.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.403+0000 7f23f53ff700 1 -- 172.21.15.60:0/1855168680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 msgr2=0x7f23f00d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:38.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.403+0000 7f23f53ff700 1 --2- 172.21.15.60:0/1855168680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 0x7f23f00d3280 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009840 tx=0x7f23e4009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:38.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.403+0000 7f23f53ff700 1 -- 172.21.15.60:0/1855168680 shutdown_connections 2023-07-13T22:30:38.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.403+0000 7f23f53ff700 1 --2- 172.21.15.60:0/1855168680 >> v2:172.21.15.105:3300/0 conn(0x7f23f0159ea0 0x7f23f00cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.403+0000 7f23f53ff700 1 --2- 172.21.15.60:0/1855168680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 0x7f23f00d3280 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.403+0000 7f23f53ff700 1 -- 172.21.15.60:0/1855168680 >> 172.21.15.60:0/1855168680 conn(0x7f23f0071600 msgr2=0x7f23f00719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.403+0000 7f23f53ff700 1 -- 172.21.15.60:0/1855168680 shutdown_connections 2023-07-13T22:30:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.403+0000 7f23f53ff700 1 -- 172.21.15.60:0/1855168680 wait complete. 2023-07-13T22:30:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.405+0000 7f23f53ff700 1 Processor -- start 2023-07-13T22:30:38.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.405+0000 7f23f53ff700 1 -- start start 2023-07-13T22:30:38.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23f53ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23f00d37c0 0x7f23f0159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:38.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23f53ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23f0159ea0 0x7f23f01f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:38.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23f53ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 0x7f23f00c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:38.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23f53ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f00cad00 con 0x7f23f00d37c0 2023-07-13T22:30:38.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23f53ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f01595d0 con 0x7f23f0159ea0 2023-07-13T22:30:38.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23f53ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f01f2c80 con 0x7f23f015c9b0 2023-07-13T22:30:38.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23eeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23f00d37c0 0x7f23f0159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:38.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23eeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23f00d37c0 0x7f23f0159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44674/0 (socket says 172.21.15.60:44674) 2023-07-13T22:30:38.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23eeffd700 1 -- 172.21.15.60:0/1430919577 learned_addr learned my addr 172.21.15.60:0/1430919577 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:38.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23ef7fe700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 0x7f23f00c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:38.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.406+0000 7f23ee7fc700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23f0159ea0 0x7f23f01f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:38.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23eeffd700 1 -- 172.21.15.60:0/1430919577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 msgr2=0x7f23f00c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:38.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23eeffd700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 0x7f23f00c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:38.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23eeffd700 1 -- 172.21.15.60:0/1430919577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23f0159ea0 msgr2=0x7f23f01f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23eeffd700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23f0159ea0 0x7f23f01f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23eeffd700 1 -- 172.21.15.60:0/1430919577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e00094a0 con 0x7f23f00d37c0 2023-07-13T22:30:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23eeffd700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23f00d37c0 0x7f23f0159090 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f23e000ec10 tx=0x7f23e000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23d77fe700 1 -- 172.21.15.60:0/1430919577 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e000c850 con 0x7f23f00d37c0 2023-07-13T22:30:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23d77fe700 1 -- 172.21.15.60:0/1430919577 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e000c9b0 con 0x7f23f00d37c0 2023-07-13T22:30:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e40094a0 con 0x7f23f00d37c0 2023-07-13T22:30:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23d77fe700 1 -- 172.21.15.60:0/1430919577 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e000cd20 con 0x7f23f00d37c0 2023-07-13T22:30:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.407+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f00b7210 con 0x7f23f00d37c0 2023-07-13T22:30:38.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.408+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f0067a70 con 0x7f23f00d37c0 2023-07-13T22:30:38.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.409+0000 7f23d77fe700 1 -- 172.21.15.60:0/1430919577 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f23e0013470 con 0x7f23f00d37c0 2023-07-13T22:30:38.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.410+0000 7f23d77fe700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f23dc06ace0 0x7f23dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:38.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.410+0000 7f23d77fe700 1 -- 172.21.15.60:0/1430919577 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f23e00893a0 con 0x7f23f00d37c0 2023-07-13T22:30:38.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.411+0000 7f23ee7fc700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f23dc06ace0 0x7f23dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:38.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.411+0000 7f23ee7fc700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f23dc06ace0 0x7f23dc06d190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009fd0 tx=0x7f23e401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:38.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.415+0000 7f23d77fe700 1 -- 172.21.15.60:0/1430919577 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f23e00545a0 con 0x7f23f00d37c0 2023-07-13T22:30:38.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:38.624+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f23f00b6ba0 con 0x7f23f00d37c0 2023-07-13T22:30:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.174+0000 7f23d77fe700 1 -- 172.21.15.60:0/1430919577 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v122) v1 ==== 170+0+0 (secure 0 0 0) 0x7f23e0057bc0 con 0x7f23f00d37c0 2023-07-13T22:30:39.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.180+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f23dc06ace0 msgr2=0x7f23dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:39.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.180+0000 7f23f53ff700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f23dc06ace0 0x7f23dc06d190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009fd0 tx=0x7f23e401c040 comp rx=0 tx=0).stop 2023-07-13T22:30:39.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.181+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23f00d37c0 msgr2=0x7f23f0159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:39.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.181+0000 7f23f53ff700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23f00d37c0 0x7f23f0159090 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f23e000ec10 tx=0x7f23e000ef20 comp rx=0 tx=0).stop 2023-07-13T22:30:39.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.182+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 shutdown_connections 2023-07-13T22:30:39.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.182+0000 7f23f53ff700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f23dc06ace0 0x7f23dc06d190 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:39.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.182+0000 7f23f53ff700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23f00d37c0 0x7f23f0159090 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:39.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.182+0000 7f23f53ff700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23f015c9b0 0x7f23f00c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:39.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.182+0000 7f23f53ff700 1 --2- 172.21.15.60:0/1430919577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23f0159ea0 0x7f23f01f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:39.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.182+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 >> 172.21.15.60:0/1430919577 conn(0x7f23f0071600 msgr2=0x7f23f00cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:39.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.183+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 shutdown_connections 2023-07-13T22:30:39.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.184+0000 7f23f53ff700 1 -- 172.21.15.60:0/1430919577 wait complete. 2023-07-13T22:30:39.170 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:30:39.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:30:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.423+0000 7fed9c9ff700 1 Processor -- start 2023-07-13T22:30:39.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.423+0000 7fed9c9ff700 1 -- start start 2023-07-13T22:30:39.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed9c9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fed980cfd50 0x7fed980d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:39.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed9c9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed980b1ac0 0x7fed980d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:39.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed9c9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed980d1870 con 0x7fed980b8200 2023-07-13T22:30:39.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed9c9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed980cf370 con 0x7fed980b1ac0 2023-07-13T22:30:39.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed9c9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed980d12b0 con 0x7fed980cfd50 2023-07-13T22:30:39.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed96d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed980b1ac0 0x7fed980d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:39.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed96d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed980b1ac0 0x7fed980d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44688/0 (socket says 172.21.15.60:44688) 2023-07-13T22:30:39.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed96d9d700 1 -- 172.21.15.60:0/1534074137 learned_addr learned my addr 172.21.15.60:0/1534074137 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:39.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed8ffff700 1 --2- 172.21.15.60:0/1534074137 >> v2:172.21.15.105:3300/0 conn(0x7fed980cfd50 0x7fed980d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:39.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed96d9d700 1 -- 172.21.15.60:0/1534074137 >> v2:172.21.15.105:3300/0 conn(0x7fed980cfd50 msgr2=0x7fed980d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:39.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed96d9d700 1 --2- 172.21.15.60:0/1534074137 >> v2:172.21.15.105:3300/0 conn(0x7fed980cfd50 0x7fed980d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:39.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed96d9d700 1 -- 172.21.15.60:0/1534074137 >> v1:172.21.15.105:6789/0 conn(0x7fed980b8200 legacy=0x7fed980cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:39.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.425+0000 7fed96d9d700 1 -- 172.21.15.60:0/1534074137 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed980d1590 con 0x7fed980b1ac0 2023-07-13T22:30:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.426+0000 7fed96d9d700 1 --2- 172.21.15.60:0/1534074137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980b1ac0 0x7fed980d2900 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fed980d07b0 tx=0x7fed9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cd9cfae2c639d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.426+0000 7fed95d9b700 1 -- 172.21.15.60:0/1534074137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000e070 con 0x7fed980b1ac0 2023-07-13T22:30:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.426+0000 7fed95d9b700 1 -- 172.21.15.60:0/1534074137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9000b910 con 0x7fed980b1ac0 2023-07-13T22:30:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.426+0000 7fed95d9b700 1 -- 172.21.15.60:0/1534074137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000bcc0 con 0x7fed980b1ac0 2023-07-13T22:30:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.426+0000 7fed9c9ff700 1 -- 172.21.15.60:0/1534074137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980b1ac0 msgr2=0x7fed980d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.426+0000 7fed9c9ff700 1 --2- 172.21.15.60:0/1534074137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980b1ac0 0x7fed980d2900 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fed980d07b0 tx=0x7fed9000b330 comp rx=0 tx=0).stop 2023-07-13T22:30:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.427+0000 7fed9c9ff700 1 -- 172.21.15.60:0/1534074137 shutdown_connections 2023-07-13T22:30:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.427+0000 7fed9c9ff700 1 --2- 172.21.15.60:0/1534074137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980b1ac0 0x7fed980d2900 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.427+0000 7fed9c9ff700 1 --2- 172.21.15.60:0/1534074137 >> v2:172.21.15.105:3300/0 conn(0x7fed980cfd50 0x7fed980d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.427+0000 7fed9c9ff700 1 -- 172.21.15.60:0/1534074137 >> 172.21.15.60:0/1534074137 conn(0x7fed9806b2f0 msgr2=0x7fed9806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.427+0000 7fed9c9ff700 1 -- 172.21.15.60:0/1534074137 shutdown_connections 2023-07-13T22:30:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.427+0000 7fed9c9ff700 1 -- 172.21.15.60:0/1534074137 wait complete. 2023-07-13T22:30:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.428+0000 7fed9c9ff700 1 Processor -- start 2023-07-13T22:30:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.429+0000 7fed9c9ff700 1 -- start start 2023-07-13T22:30:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.429+0000 7fed9c9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed980b1ac0 0x7fed98156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:39.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.429+0000 7fed9c9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed980b8200 0x7fed981f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:39.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed9c9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980cfd50 0x7fed980b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:39.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed9c9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed98158670 con 0x7fed980cfd50 2023-07-13T22:30:39.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed9c9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed98159e80 con 0x7fed980b1ac0 2023-07-13T22:30:39.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed9c9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed981f2230 con 0x7fed980b8200 2023-07-13T22:30:39.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed96d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980cfd50 0x7fed980b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:39.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed96d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980cfd50 0x7fed980b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44694/0 (socket says 172.21.15.60:44694) 2023-07-13T22:30:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed96d9d700 1 -- 172.21.15.60:0/2630394127 learned_addr learned my addr 172.21.15.60:0/2630394127 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed9659c700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed980b1ac0 0x7fed98156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed8ffff700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed980b8200 0x7fed981f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed96d9d700 1 -- 172.21.15.60:0/2630394127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed980b8200 msgr2=0x7fed981f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed96d9d700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed980b8200 0x7fed981f18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed96d9d700 1 -- 172.21.15.60:0/2630394127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed980b1ac0 msgr2=0x7fed98156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed96d9d700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed980b1ac0 0x7fed98156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:39.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.430+0000 7fed96d9d700 1 -- 172.21.15.60:0/2630394127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed90009b30 con 0x7fed980cfd50 2023-07-13T22:30:39.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.431+0000 7fed96d9d700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980cfd50 0x7fed980b6390 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fed9806bf10 tx=0x7fed90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.431+0000 7fed8effd700 1 -- 172.21.15.60:0/2630394127 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90004880 con 0x7fed980cfd50 2023-07-13T22:30:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.431+0000 7fed8effd700 1 -- 172.21.15.60:0/2630394127 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed900099b0 con 0x7fed980cfd50 2023-07-13T22:30:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.431+0000 7fed8effd700 1 -- 172.21.15.60:0/2630394127 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9001c680 con 0x7fed980cfd50 2023-07-13T22:30:39.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.431+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed981f2510 con 0x7fed980cfd50 2023-07-13T22:30:39.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.431+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed981f2d10 con 0x7fed980cfd50 2023-07-13T22:30:39.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.432+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed981f2a30 con 0x7fed980cfd50 2023-07-13T22:30:39.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.433+0000 7fed8effd700 1 -- 172.21.15.60:0/2630394127 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed9001c7e0 con 0x7fed980cfd50 2023-07-13T22:30:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.434+0000 7fed8effd700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed7806acb0 0x7fed7806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.434+0000 7fed8effd700 1 -- 172.21.15.60:0/2630394127 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fed9008af50 con 0x7fed980cfd50 2023-07-13T22:30:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.434+0000 7fed9659c700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed7806acb0 0x7fed7806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:39.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.435+0000 7fed9659c700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed7806acb0 0x7fed7806d160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fed981560c0 tx=0x7fed88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:39.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.438+0000 7fed8effd700 1 -- 172.21.15.60:0/2630394127 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fed900561f0 con 0x7fed980cfd50 2023-07-13T22:30:39.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:39.689+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fed981f2a30 con 0x7fed980cfd50 2023-07-13T22:30:40.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.189+0000 7fed8effd700 1 -- 172.21.15.60:0/2630394127 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v81) v1 ==== 174+0+0 (secure 0 0 0) 0x7fed981f2a30 con 0x7fed980cfd50 2023-07-13T22:30:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.198+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed7806acb0 msgr2=0x7fed7806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.198+0000 7fed9c9ff700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed7806acb0 0x7fed7806d160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fed981560c0 tx=0x7fed88009040 comp rx=0 tx=0).stop 2023-07-13T22:30:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.198+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980cfd50 msgr2=0x7fed980b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.198+0000 7fed9c9ff700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980cfd50 0x7fed980b6390 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fed9806bf10 tx=0x7fed90004ad0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.200+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 shutdown_connections 2023-07-13T22:30:40.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.200+0000 7fed9c9ff700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed7806acb0 0x7fed7806d160 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.200+0000 7fed9c9ff700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed980cfd50 0x7fed980b6390 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.200+0000 7fed9c9ff700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed980b1ac0 0x7fed98156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.200+0000 7fed9c9ff700 1 --2- 172.21.15.60:0/2630394127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed980b8200 0x7fed981f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.200+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 >> 172.21.15.60:0/2630394127 conn(0x7fed9806b2f0 msgr2=0x7fed980c8770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:40.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.203+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 shutdown_connections 2023-07-13T22:30:40.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.204+0000 7fed9c9ff700 1 -- 172.21.15.60:0/2630394127 wait complete. 2023-07-13T22:30:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 19 and data pool 20 2023-07-13T22:30:40.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:30:40.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.432+0000 7ff3287ff700 1 Processor -- start 2023-07-13T22:30:40.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.433+0000 7ff3287ff700 1 -- start start 2023-07-13T22:30:40.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff3287ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff3200cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff3287ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3200cfca0 0x7ff3200d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff3287ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3200d0ac0 con 0x7ff3200d3750 2023-07-13T22:30:40.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff3287ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200b87d0 con 0x7ff3200cfca0 2023-07-13T22:30:40.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff3287ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200b4eb0 con 0x7ff3200b8400 2023-07-13T22:30:40.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff326d9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3200cfca0 0x7ff3200d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff32659b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff3200cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff326d9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3200cfca0 0x7ff3200d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:38028/0 (socket says 172.21.15.60:38028) 2023-07-13T22:30:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff32659b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff3200cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:56972/0 (socket says 172.21.15.60:56972) 2023-07-13T22:30:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff325d9a700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff3200d3750 0x7ff3200cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:50958/0 (socket says 172.21.15.60:50958) 2023-07-13T22:30:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.434+0000 7ff326d9c700 1 -- 172.21.15.60:0/2061075794 learned_addr learned my addr 172.21.15.60:0/2061075794 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.435+0000 7ff325599700 1 -- 172.21.15.60:0/2061075794 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084170477 0 0) 0x7ff3200d0ac0 con 0x7ff3200d3750 2023-07-13T22:30:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.435+0000 7ff32659b700 1 -- 172.21.15.60:0/2061075794 >> v2:172.21.15.105:3300/0 conn(0x7ff3200cfca0 msgr2=0x7ff3200d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.435+0000 7ff32659b700 1 --2- 172.21.15.60:0/2061075794 >> v2:172.21.15.105:3300/0 conn(0x7ff3200cfca0 0x7ff3200d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.435+0000 7ff32659b700 1 -- 172.21.15.60:0/2061075794 >> v1:172.21.15.105:6789/0 conn(0x7ff3200d3750 legacy=0x7ff3200cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.435+0000 7ff32659b700 1 -- 172.21.15.60:0/2061075794 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200b5190 con 0x7ff3200b8400 2023-07-13T22:30:40.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.435+0000 7ff32659b700 1 --2- 172.21.15.60:0/2061075794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff3200cd240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009840 tx=0x7ff31c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ae8b415db6dbacd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:40.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.435+0000 7ff325599700 1 -- 172.21.15.60:0/2061075794 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c011830 con 0x7ff3200b8400 2023-07-13T22:30:40.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff325599700 1 -- 172.21.15.60:0/2061075794 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c011990 con 0x7ff3200b8400 2023-07-13T22:30:40.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff325599700 1 -- 172.21.15.60:0/2061075794 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c011d20 con 0x7ff3200b8400 2023-07-13T22:30:40.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff3287ff700 1 -- 172.21.15.60:0/2061075794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 msgr2=0x7ff3200cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff3287ff700 1 --2- 172.21.15.60:0/2061075794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff3200cd240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009840 tx=0x7ff31c009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:40.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff3287ff700 1 -- 172.21.15.60:0/2061075794 shutdown_connections 2023-07-13T22:30:40.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff3287ff700 1 --2- 172.21.15.60:0/2061075794 >> v2:172.21.15.105:3300/0 conn(0x7ff3200cfca0 0x7ff3200d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff3287ff700 1 --2- 172.21.15.60:0/2061075794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff3200cd240 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff3287ff700 1 -- 172.21.15.60:0/2061075794 >> 172.21.15.60:0/2061075794 conn(0x7ff32006b390 msgr2=0x7ff32006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:40.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff3287ff700 1 -- 172.21.15.60:0/2061075794 shutdown_connections 2023-07-13T22:30:40.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.436+0000 7ff3287ff700 1 -- 172.21.15.60:0/2061075794 wait complete. 2023-07-13T22:30:40.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.438+0000 7ff3287ff700 1 Processor -- start 2023-07-13T22:30:40.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.438+0000 7ff3287ff700 1 -- start start 2023-07-13T22:30:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff3287ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff32014c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff3287ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3200cfca0 0x7ff320153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff3287ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3200d3750 0x7ff3200b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff3287ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320154770 con 0x7ff3200cfca0 2023-07-13T22:30:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff3287ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320153ed0 con 0x7ff3200d3750 2023-07-13T22:30:40.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff3287ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201541b0 con 0x7ff3200b8400 2023-07-13T22:30:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff325d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3200cfca0 0x7ff320153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff325d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3200cfca0 0x7ff320153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44708/0 (socket says 172.21.15.60:44708) 2023-07-13T22:30:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff325d9a700 1 -- 172.21.15.60:0/4042730927 learned_addr learned my addr 172.21.15.60:0/4042730927 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.439+0000 7ff32659b700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff32014c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff326d9c700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3200d3750 0x7ff3200b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff325d9a700 1 -- 172.21.15.60:0/4042730927 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 msgr2=0x7ff32014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff325d9a700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff32014c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff325d9a700 1 -- 172.21.15.60:0/4042730927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3200d3750 msgr2=0x7ff3200b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff325d9a700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3200d3750 0x7ff3200b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff325d9a700 1 -- 172.21.15.60:0/4042730927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0094a0 con 0x7ff3200cfca0 2023-07-13T22:30:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff325d9a700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3200cfca0 0x7ff320153480 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c540 tx=0x7ff31000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff316ffd700 1 -- 172.21.15.60:0/4042730927 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310004c80 con 0x7ff3200cfca0 2023-07-13T22:30:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff316ffd700 1 -- 172.21.15.60:0/4042730927 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff310004de0 con 0x7ff3200cfca0 2023-07-13T22:30:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320154490 con 0x7ff3200cfca0 2023-07-13T22:30:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff316ffd700 1 -- 172.21.15.60:0/4042730927 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310007310 con 0x7ff3200cfca0 2023-07-13T22:30:40.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.440+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3200ccbb0 con 0x7ff3200cfca0 2023-07-13T22:30:40.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.441+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3200c7d40 con 0x7ff3200cfca0 2023-07-13T22:30:40.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.442+0000 7ff316ffd700 1 -- 172.21.15.60:0/4042730927 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff310007470 con 0x7ff3200cfca0 2023-07-13T22:30:40.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.443+0000 7ff316ffd700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff30406ace0 0x7ff30406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.443+0000 7ff316ffd700 1 -- 172.21.15.60:0/4042730927 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff3100891a0 con 0x7ff3200cfca0 2023-07-13T22:30:40.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.444+0000 7ff32659b700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff30406ace0 0x7ff30406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.444+0000 7ff32659b700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff30406ace0 0x7ff30406d190 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009fd0 tx=0x7ff31c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:40.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.448+0000 7ff316ffd700 1 -- 172.21.15.60:0/4042730927 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff3200c7d40 con 0x7ff3200cfca0 2023-07-13T22:30:40.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.655+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3200c7d40 con 0x7ff3200cfca0 2023-07-13T22:30:40.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.656+0000 7ff316ffd700 1 -- 172.21.15.60:0/4042730927 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+18228 (secure 0 0 0) 0x7ff3200c7d40 con 0x7ff3200cfca0 2023-07-13T22:30:40.636 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:40.636 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:30:39.690098+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:30:37.983326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-13T22:30:38.665534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:30:40.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff30406ace0 msgr2=0x7ff30406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff30406ace0 0x7ff30406d190 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009fd0 tx=0x7ff31c01c040 comp rx=0 tx=0).stop 2023-07-13T22:30:40.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3200cfca0 msgr2=0x7ff320153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3200cfca0 0x7ff320153480 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c540 tx=0x7ff31000c850 comp rx=0 tx=0).stop 2023-07-13T22:30:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 shutdown_connections 2023-07-13T22:30:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff30406ace0 0x7ff30406d190 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3200d3750 0x7ff3200b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3200b8400 0x7ff32014c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 --2- 172.21.15.60:0/4042730927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3200cfca0 0x7ff320153480 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 >> 172.21.15.60:0/4042730927 conn(0x7ff32006b390 msgr2=0x7ff3200b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 shutdown_connections 2023-07-13T22:30:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.659+0000 7ff3287ff700 1 -- 172.21.15.60:0/4042730927 wait complete. 2023-07-13T22:30:40.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:30:40.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.881+0000 7fe4d9fff700 1 Processor -- start 2023-07-13T22:30:40.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.881+0000 7fe4d9fff700 1 -- start start 2023-07-13T22:30:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.882+0000 7fe4d9fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40d26d0 0x7fe4d40cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.882+0000 7fe4d9fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4d40b5a00 0x7fe4d415fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.882+0000 7fe4d9fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d40d14a0 con 0x7fe4d40b2010 2023-07-13T22:30:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.882+0000 7fe4d9fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d40d2570 con 0x7fe4d40b5a00 2023-07-13T22:30:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.883+0000 7fe4d9fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40d1780 con 0x7fe4d40d26d0 2023-07-13T22:30:40.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.883+0000 7fe4d3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4d40b5a00 0x7fe4d415fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.883+0000 7fe4d3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4d40b5a00 0x7fe4d415fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44714/0 (socket says 172.21.15.60:44714) 2023-07-13T22:30:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.883+0000 7fe4d3fff700 1 -- 172.21.15.60:0/3012545370 learned_addr learned my addr 172.21.15.60:0/3012545370 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.883+0000 7fe4d2ffd700 1 --2- 172.21.15.60:0/3012545370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40d26d0 0x7fe4d40cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.883+0000 7fe4d3fff700 1 -- 172.21.15.60:0/3012545370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40d26d0 msgr2=0x7fe4d40cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.883+0000 7fe4d3fff700 1 --2- 172.21.15.60:0/3012545370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40d26d0 0x7fe4d40cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.883+0000 7fe4d3fff700 1 -- 172.21.15.60:0/3012545370 >> v1:172.21.15.105:6789/0 conn(0x7fe4d40b2010 legacy=0x7fe4d40cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.883+0000 7fe4d3fff700 1 -- 172.21.15.60:0/3012545370 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d40cd2a0 con 0x7fe4d40b5a00 2023-07-13T22:30:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d3fff700 1 --2- 172.21.15.60:0/3012545370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b5a00 0x7fe4d415fee0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009e50 tx=0x7fe4cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbb8a0aefdcebb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d27fc700 1 -- 172.21.15.60:0/3012545370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00e070 con 0x7fe4d40b5a00 2023-07-13T22:30:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d27fc700 1 -- 172.21.15.60:0/3012545370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc00b910 con 0x7fe4d40b5a00 2023-07-13T22:30:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d27fc700 1 -- 172.21.15.60:0/3012545370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00bcc0 con 0x7fe4d40b5a00 2023-07-13T22:30:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d9fff700 1 -- 172.21.15.60:0/3012545370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b5a00 msgr2=0x7fe4d415fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d9fff700 1 --2- 172.21.15.60:0/3012545370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b5a00 0x7fe4d415fee0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009e50 tx=0x7fe4cc00b330 comp rx=0 tx=0).stop 2023-07-13T22:30:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d9fff700 1 -- 172.21.15.60:0/3012545370 shutdown_connections 2023-07-13T22:30:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d9fff700 1 --2- 172.21.15.60:0/3012545370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b5a00 0x7fe4d415fee0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d9fff700 1 --2- 172.21.15.60:0/3012545370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40d26d0 0x7fe4d40cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.884+0000 7fe4d9fff700 1 -- 172.21.15.60:0/3012545370 >> 172.21.15.60:0/3012545370 conn(0x7fe4d406b380 msgr2=0x7fe4d406b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.885+0000 7fe4d9fff700 1 -- 172.21.15.60:0/3012545370 shutdown_connections 2023-07-13T22:30:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.885+0000 7fe4d9fff700 1 -- 172.21.15.60:0/3012545370 wait complete. 2023-07-13T22:30:40.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.886+0000 7fe4d9fff700 1 Processor -- start 2023-07-13T22:30:40.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.886+0000 7fe4d9fff700 1 -- start start 2023-07-13T22:30:40.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.887+0000 7fe4d9fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b2010 0x7fe4d41f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.887+0000 7fe4d9fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40b5a00 0x7fe4d40ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.887+0000 7fe4d9fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d40d26d0 0x7fe4d415a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.887+0000 7fe4d9fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40cabb0 con 0x7fe4d40b2010 2023-07-13T22:30:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.887+0000 7fe4d9fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40cb640 con 0x7fe4d40d26d0 2023-07-13T22:30:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.887+0000 7fe4d9fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40cada0 con 0x7fe4d40b5a00 2023-07-13T22:30:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b2010 0x7fe4d41f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b2010 0x7fe4d41f28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44720/0 (socket says 172.21.15.60:44720) 2023-07-13T22:30:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d37fe700 1 -- 172.21.15.60:0/4039766809 learned_addr learned my addr 172.21.15.60:0/4039766809 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d2ffd700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40b5a00 0x7fe4d40ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d3fff700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d40d26d0 0x7fe4d415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d37fe700 1 -- 172.21.15.60:0/4039766809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40b5a00 msgr2=0x7fe4d40ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d37fe700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40b5a00 0x7fe4d40ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d37fe700 1 -- 172.21.15.60:0/4039766809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d40d26d0 msgr2=0x7fe4d415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d37fe700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d40d26d0 0x7fe4d415a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d37fe700 1 -- 172.21.15.60:0/4039766809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc009b30 con 0x7fe4d40b2010 2023-07-13T22:30:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d3fff700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d40d26d0 0x7fe4d415a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.888+0000 7fe4d37fe700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b2010 0x7fe4d41f28d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400c450 tx=0x7fe4c400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.889+0000 7fe4bbfff700 1 -- 172.21.15.60:0/4039766809 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c40052d0 con 0x7fe4d40b2010 2023-07-13T22:30:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.889+0000 7fe4bbfff700 1 -- 172.21.15.60:0/4039766809 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c4005430 con 0x7fe4d40b2010 2023-07-13T22:30:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.889+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d40cb080 con 0x7fe4d40b2010 2023-07-13T22:30:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.889+0000 7fe4bbfff700 1 -- 172.21.15.60:0/4039766809 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c40038c0 con 0x7fe4d40b2010 2023-07-13T22:30:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.889+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d41f2fd0 con 0x7fe4d40b2010 2023-07-13T22:30:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.890+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d404e280 con 0x7fe4d40b2010 2023-07-13T22:30:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.891+0000 7fe4bbfff700 1 -- 172.21.15.60:0/4039766809 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4c4003a20 con 0x7fe4d40b2010 2023-07-13T22:30:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.892+0000 7fe4bbfff700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4c006ace0 0x7fe4c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.892+0000 7fe4bbfff700 1 -- 172.21.15.60:0/4039766809 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4c4088860 con 0x7fe4d40b2010 2023-07-13T22:30:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.892+0000 7fe4d2ffd700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4c006ace0 0x7fe4c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.893+0000 7fe4d2ffd700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4c006ace0 0x7fe4c006d190 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40b8830 tx=0x7fe4c8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:40.897+0000 7fe4bbfff700 1 -- 172.21.15.60:0/4039766809 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe4c4053900 con 0x7fe4d40b2010 2023-07-13T22:30:41.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.104+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4d40b74f0 con 0x7fe4d40b2010 2023-07-13T22:30:41.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.105+0000 7fe4bbfff700 1 -- 172.21.15.60:0/4039766809 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+18228 (secure 0 0 0) 0x7fe4c4056f20 con 0x7fe4d40b2010 2023-07-13T22:30:41.085 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:41.085 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:30:39.690098+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:30:37.983326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-13T22:30:38.665534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:30:41.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4c006ace0 msgr2=0x7fe4c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:41.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4c006ace0 0x7fe4c006d190 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40b8830 tx=0x7fe4c8009210 comp rx=0 tx=0).stop 2023-07-13T22:30:41.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b2010 msgr2=0x7fe4d41f28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b2010 0x7fe4d41f28d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400c450 tx=0x7fe4c400c760 comp rx=0 tx=0).stop 2023-07-13T22:30:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 shutdown_connections 2023-07-13T22:30:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe4c006ace0 0x7fe4c006d190 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d40b2010 0x7fe4d41f28d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d40d26d0 0x7fe4d415a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 --2- 172.21.15.60:0/4039766809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4d40b5a00 0x7fe4d40ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 >> 172.21.15.60:0/4039766809 conn(0x7fe4d406b380 msgr2=0x7fe4d40d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:41.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 shutdown_connections 2023-07-13T22:30:41.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.108+0000 7fe4d9fff700 1 -- 172.21.15.60:0/4039766809 wait complete. 2023-07-13T22:30:41.106 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:30:41.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.337+0000 7fd3ba1ff700 1 Processor -- start 2023-07-13T22:30:41.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.337+0000 7fd3ba1ff700 1 -- start start 2023-07-13T22:30:41.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.338+0000 7fd3ba1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b8200 0x7fd3b40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:41.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.338+0000 7fd3ba1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3b40cede0 0x7fd3b40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:41.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.338+0000 7fd3ba1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b40d1870 con 0x7fd3b40b1ac0 2023-07-13T22:30:41.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.338+0000 7fd3ba1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b40cf370 con 0x7fd3b40cede0 2023-07-13T22:30:41.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.338+0000 7fd3ba1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40d12b0 con 0x7fd3b40b8200 2023-07-13T22:30:41.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.339+0000 7fd3b3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3b40cede0 0x7fd3b40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:41.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.339+0000 7fd3abfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd3b40b1ac0 0x7fd3b415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43606/0 (socket says 172.21.15.60:43606) 2023-07-13T22:30:41.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.339+0000 7fd3abfff700 1 -- 172.21.15.60:0/2719777605 learned_addr learned my addr 172.21.15.60:0/2719777605 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:41.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.339+0000 7fd3b37fe700 1 --2- 172.21.15.60:0/2719777605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b8200 0x7fd3b40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:41.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.339+0000 7fd3b3fff700 1 -- 172.21.15.60:0/2719777605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b8200 msgr2=0x7fd3b40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:41.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.339+0000 7fd3b3fff700 1 --2- 172.21.15.60:0/2719777605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b8200 0x7fd3b40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:41.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.339+0000 7fd3b3fff700 1 -- 172.21.15.60:0/2719777605 >> v1:172.21.15.60:6789/0 conn(0x7fd3b40b1ac0 legacy=0x7fd3b415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:41.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.339+0000 7fd3b3fff700 1 -- 172.21.15.60:0/2719777605 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b40d1590 con 0x7fd3b40cede0 2023-07-13T22:30:41.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.339+0000 7fd3b2ffd700 1 -- 172.21.15.60:0/2719777605 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240335028 0 0) 0x7fd3b40d1870 con 0x7fd3b40b1ac0 2023-07-13T22:30:41.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3b3fff700 1 --2- 172.21.15.60:0/2719777605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40cede0 0x7fd3b40cfb80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009d90 tx=0x7fd3ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ba8841c5a5e484 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:41.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3b2ffd700 1 -- 172.21.15.60:0/2719777605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00e070 con 0x7fd3b40cede0 2023-07-13T22:30:41.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3b2ffd700 1 -- 172.21.15.60:0/2719777605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac00bbd0 con 0x7fd3b40cede0 2023-07-13T22:30:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3b2ffd700 1 -- 172.21.15.60:0/2719777605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac012430 con 0x7fd3b40cede0 2023-07-13T22:30:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/2719777605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40cede0 msgr2=0x7fd3b40cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3ba1ff700 1 --2- 172.21.15.60:0/2719777605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40cede0 0x7fd3b40cfb80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009d90 tx=0x7fd3ac00b330 comp rx=0 tx=0).stop 2023-07-13T22:30:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/2719777605 shutdown_connections 2023-07-13T22:30:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3ba1ff700 1 --2- 172.21.15.60:0/2719777605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40cede0 0x7fd3b40cfb80 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3ba1ff700 1 --2- 172.21.15.60:0/2719777605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b8200 0x7fd3b40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/2719777605 >> 172.21.15.60:0/2719777605 conn(0x7fd3b406b2f0 msgr2=0x7fd3b406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.340+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/2719777605 shutdown_connections 2023-07-13T22:30:41.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.341+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/2719777605 wait complete. 2023-07-13T22:30:41.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.342+0000 7fd3ba1ff700 1 Processor -- start 2023-07-13T22:30:41.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.342+0000 7fd3ba1ff700 1 -- start start 2023-07-13T22:30:41.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.343+0000 7fd3ba1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b1ac0 0x7fd3b41f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:41.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.343+0000 7fd3ba1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40b8200 0x7fd3b40cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.343+0000 7fd3ba1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3b40cede0 0x7fd3b40cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.343+0000 7fd3ba1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b41f1cc0 con 0x7fd3b40b8200 2023-07-13T22:30:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.343+0000 7fd3ba1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b41e3e50 con 0x7fd3b40cede0 2023-07-13T22:30:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.343+0000 7fd3ba1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40cb8c0 con 0x7fd3b40b1ac0 2023-07-13T22:30:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3abfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40b8200 0x7fd3b40cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3abfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40b8200 0x7fd3b40cad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44738/0 (socket says 172.21.15.60:44738) 2023-07-13T22:30:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3abfff700 1 -- 172.21.15.60:0/29313529 learned_addr learned my addr 172.21.15.60:0/29313529 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3b37fe700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b1ac0 0x7fd3b41f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3b3fff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3b40cede0 0x7fd3b40cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3abfff700 1 -- 172.21.15.60:0/29313529 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b1ac0 msgr2=0x7fd3b41f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3abfff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b1ac0 0x7fd3b41f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3abfff700 1 -- 172.21.15.60:0/29313529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3b40cede0 msgr2=0x7fd3b40cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3abfff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3b40cede0 0x7fd3b40cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3abfff700 1 -- 172.21.15.60:0/29313529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac009a70 con 0x7fd3b40b8200 2023-07-13T22:30:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.344+0000 7fd3abfff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40b8200 0x7fd3b40cad50 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0011e0 tx=0x7fd39c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.345+0000 7fd3b0ff9700 1 -- 172.21.15.60:0/29313529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c004c80 con 0x7fd3b40b8200 2023-07-13T22:30:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.345+0000 7fd3b0ff9700 1 -- 172.21.15.60:0/29313529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c004e00 con 0x7fd3b40b8200 2023-07-13T22:30:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.345+0000 7fd3b0ff9700 1 -- 172.21.15.60:0/29313529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c007310 con 0x7fd3b40b8200 2023-07-13T22:30:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.345+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b40cbba0 con 0x7fd3b40b8200 2023-07-13T22:30:41.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.345+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b40cc160 con 0x7fd3b40b8200 2023-07-13T22:30:41.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.346+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b40b4d30 con 0x7fd3b40b8200 2023-07-13T22:30:41.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.347+0000 7fd3b0ff9700 1 -- 172.21.15.60:0/29313529 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd39c007470 con 0x7fd3b40b8200 2023-07-13T22:30:41.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.348+0000 7fd3b0ff9700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd39406ace0 0x7fd39406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:41.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.348+0000 7fd3b0ff9700 1 -- 172.21.15.60:0/29313529 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd39c0890c0 con 0x7fd3b40b8200 2023-07-13T22:30:41.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.348+0000 7fd3b37fe700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd39406ace0 0x7fd39406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:41.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.349+0000 7fd3b37fe700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd39406ace0 0x7fd39406d190 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd3b41f5310 tx=0x7fd3a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:41.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.352+0000 7fd3b0ff9700 1 -- 172.21.15.60:0/29313529 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd39c054160 con 0x7fd3b40b8200 2023-07-13T22:30:41.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:41.611+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd3b404e210 con 0x7fd3b40b8200 2023-07-13T22:30:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.200+0000 7fd3b0ff9700 1 -- 172.21.15.60:0/29313529 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd39c057780 con 0x7fd3b40b8200 2023-07-13T22:30:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.203+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd39406ace0 msgr2=0x7fd39406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.203+0000 7fd3ba1ff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd39406ace0 0x7fd39406d190 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd3b41f5310 tx=0x7fd3a4008040 comp rx=0 tx=0).stop 2023-07-13T22:30:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.203+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40b8200 msgr2=0x7fd3b40cad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.203+0000 7fd3ba1ff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40b8200 0x7fd3b40cad50 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0011e0 tx=0x7fd39c00ca30 comp rx=0 tx=0).stop 2023-07-13T22:30:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.203+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 shutdown_connections 2023-07-13T22:30:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.204+0000 7fd3ba1ff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd39406ace0 0x7fd39406d190 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.204+0000 7fd3ba1ff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3b40cede0 0x7fd3b40cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.204+0000 7fd3ba1ff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b40b1ac0 0x7fd3b41f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.204+0000 7fd3ba1ff700 1 --2- 172.21.15.60:0/29313529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3b40b8200 0x7fd3b40cad50 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.204+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 >> 172.21.15.60:0/29313529 conn(0x7fd3b406b2f0 msgr2=0x7fd3b40b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:42.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.204+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 shutdown_connections 2023-07-13T22:30:42.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.204+0000 7fd3ba1ff700 1 -- 172.21.15.60:0/29313529 wait complete. 2023-07-13T22:30:42.205 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:30:42.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:30:42.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.439+0000 7fda08bff700 1 Processor -- start 2023-07-13T22:30:42.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.439+0000 7fda08bff700 1 -- start start 2023-07-13T22:30:42.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.440+0000 7fda08bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040b8200 0x7fda040cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:42.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.440+0000 7fda08bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda040b1ac0 0x7fda040cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:42.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.440+0000 7fda08bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda040d1870 con 0x7fda040cede0 2023-07-13T22:30:42.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.440+0000 7fda08bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040cf370 con 0x7fda040b1ac0 2023-07-13T22:30:42.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.440+0000 7fda08bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040d12b0 con 0x7fda040b8200 2023-07-13T22:30:42.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.440+0000 7fda01d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda040b1ac0 0x7fda040cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:42.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.440+0000 7fda02d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fda040cede0 0x7fda0415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43616/0 (socket says 172.21.15.60:43616) 2023-07-13T22:30:42.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.440+0000 7fda02d9d700 1 -- 172.21.15.60:0/2613582110 learned_addr learned my addr 172.21.15.60:0/2613582110 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:42.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.441+0000 7fda0259c700 1 --2- 172.21.15.60:0/2613582110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040b8200 0x7fda040cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.441+0000 7fda01d9b700 1 -- 172.21.15.60:0/2613582110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040b8200 msgr2=0x7fda040cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.441+0000 7fda01d9b700 1 --2- 172.21.15.60:0/2613582110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040b8200 0x7fda040cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.441+0000 7fda01d9b700 1 -- 172.21.15.60:0/2613582110 >> v1:172.21.15.60:6789/0 conn(0x7fda040cede0 legacy=0x7fda0415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.441+0000 7fda01d9b700 1 -- 172.21.15.60:0/2613582110 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040d1590 con 0x7fda040b1ac0 2023-07-13T22:30:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.441+0000 7fda01d9b700 1 --2- 172.21.15.60:0/2613582110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 0x7fda040cfb80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009840 tx=0x7fd9f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6638d960a3ace87e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.441+0000 7fda0159a700 1 -- 172.21.15.60:0/2613582110 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011830 con 0x7fda040b1ac0 2023-07-13T22:30:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.441+0000 7fda0159a700 1 -- 172.21.15.60:0/2613582110 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8011990 con 0x7fda040b1ac0 2023-07-13T22:30:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.441+0000 7fda0159a700 1 -- 172.21.15.60:0/2613582110 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011d20 con 0x7fda040b1ac0 2023-07-13T22:30:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.442+0000 7fda08bff700 1 -- 172.21.15.60:0/2613582110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 msgr2=0x7fda040cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.442+0000 7fda08bff700 1 --2- 172.21.15.60:0/2613582110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 0x7fda040cfb80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009840 tx=0x7fd9f8009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.442+0000 7fda08bff700 1 -- 172.21.15.60:0/2613582110 shutdown_connections 2023-07-13T22:30:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.442+0000 7fda08bff700 1 --2- 172.21.15.60:0/2613582110 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040b8200 0x7fda040cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.442+0000 7fda08bff700 1 --2- 172.21.15.60:0/2613582110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 0x7fda040cfb80 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.442+0000 7fda08bff700 1 -- 172.21.15.60:0/2613582110 >> 172.21.15.60:0/2613582110 conn(0x7fda0406b2f0 msgr2=0x7fda0406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.442+0000 7fda08bff700 1 -- 172.21.15.60:0/2613582110 shutdown_connections 2023-07-13T22:30:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.442+0000 7fda08bff700 1 -- 172.21.15.60:0/2613582110 wait complete. 2023-07-13T22:30:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.444+0000 7fda08bff700 1 Processor -- start 2023-07-13T22:30:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.444+0000 7fda08bff700 1 -- start start 2023-07-13T22:30:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda08bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 0x7fda04159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda08bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda040b8200 0x7fda041531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda08bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040cede0 0x7fda040c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda08bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda041f5d70 con 0x7fda040b1ac0 2023-07-13T22:30:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda08bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda041f50f0 con 0x7fda040b8200 2023-07-13T22:30:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda08bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda041f53d0 con 0x7fda040cede0 2023-07-13T22:30:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda0259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 0x7fda04159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda0259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 0x7fda04159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44756/0 (socket says 172.21.15.60:44756) 2023-07-13T22:30:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda0259c700 1 -- 172.21.15.60:0/1666646694 learned_addr learned my addr 172.21.15.60:0/1666646694 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda02d9d700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040cede0 0x7fda040c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.445+0000 7fda01d9b700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda040b8200 0x7fda041531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fda0259c700 1 -- 172.21.15.60:0/1666646694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040cede0 msgr2=0x7fda040c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fda0259c700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040cede0 0x7fda040c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fda0259c700 1 -- 172.21.15.60:0/1666646694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda040b8200 msgr2=0x7fda041531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fda0259c700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda040b8200 0x7fda041531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fda0259c700 1 -- 172.21.15.60:0/1666646694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80094a0 con 0x7fda040b1ac0 2023-07-13T22:30:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fda0259c700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 0x7fda04159cc0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400e810 tx=0x7fd9f400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fd9f2ffd700 1 -- 172.21.15.60:0/1666646694 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4013610 con 0x7fda040b1ac0 2023-07-13T22:30:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fd9f2ffd700 1 -- 172.21.15.60:0/1666646694 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4013770 con 0x7fda040b1ac0 2023-07-13T22:30:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fd9f2ffd700 1 -- 172.21.15.60:0/1666646694 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4013b20 con 0x7fda040b1ac0 2023-07-13T22:30:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda041f56b0 con 0x7fda040b1ac0 2023-07-13T22:30:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.446+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0415a200 con 0x7fda040b1ac0 2023-07-13T22:30:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.447+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0404e210 con 0x7fda040b1ac0 2023-07-13T22:30:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.448+0000 7fd9f2ffd700 1 -- 172.21.15.60:0/1666646694 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd9f4013c80 con 0x7fda040b1ac0 2023-07-13T22:30:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.449+0000 7fd9f2ffd700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9e406ace0 0x7fd9e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:42.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.449+0000 7fd9f2ffd700 1 -- 172.21.15.60:0/1666646694 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd9f40891b0 con 0x7fda040b1ac0 2023-07-13T22:30:42.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.449+0000 7fda01d9b700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9e406ace0 0x7fd9e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:42.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.450+0000 7fda01d9b700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9e406ace0 0x7fd9e406d190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80090b0 tx=0x7fd9f8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.453+0000 7fd9f2ffd700 1 -- 172.21.15.60:0/1666646694 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd9f4054270 con 0x7fda040b1ac0 2023-07-13T22:30:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:42.705+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fda040b4f70 con 0x7fda040b1ac0 2023-07-13T22:30:43.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.206+0000 7fd9f2ffd700 1 -- 172.21.15.60:0/1666646694 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd9f4057890 con 0x7fda040b1ac0 2023-07-13T22:30:43.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9e406ace0 msgr2=0x7fd9e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9e406ace0 0x7fd9e406d190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80090b0 tx=0x7fd9f8008030 comp rx=0 tx=0).stop 2023-07-13T22:30:43.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 msgr2=0x7fda04159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 0x7fda04159cc0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400e810 tx=0x7fd9f400ebd0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 shutdown_connections 2023-07-13T22:30:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd9e406ace0 0x7fd9e406d190 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda040b1ac0 0x7fda04159cc0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda040cede0 0x7fda040c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 --2- 172.21.15.60:0/1666646694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda040b8200 0x7fda041531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.209+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 >> 172.21.15.60:0/1666646694 conn(0x7fda0406b2f0 msgr2=0x7fda040d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.210+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 shutdown_connections 2023-07-13T22:30:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.210+0000 7fda08bff700 1 -- 172.21.15.60:0/1666646694 wait complete. 2023-07-13T22:30:43.209 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:30:43.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.436+0000 7f8b4e5ff700 1 Processor -- start 2023-07-13T22:30:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.436+0000 7f8b4e5ff700 1 -- start start 2023-07-13T22:30:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.437+0000 7f8b4e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480cefa0 0x7f8b4806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.437+0000 7f8b4e5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8b480cd760 0x7f8b480cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.437+0000 7f8b4e5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b480cf3f0 con 0x7f8b480b12a0 2023-07-13T22:30:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.437+0000 7f8b4e5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4806a5f0 con 0x7f8b480cd760 2023-07-13T22:30:43.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.437+0000 7f8b4e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48160ed0 con 0x7f8b480cefa0 2023-07-13T22:30:43.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.438+0000 7f8b477fe700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480cefa0 0x7f8b4806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.438+0000 7f8b4cb9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8b480cd760 0x7f8b480cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.438+0000 7f8b4cb9c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8b480cd760 0x7f8b480cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:38076/0 (socket says 172.21.15.60:38076) 2023-07-13T22:30:43.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.438+0000 7f8b4cb9c700 1 -- 172.21.15.60:0/1084287118 learned_addr learned my addr 172.21.15.60:0/1084287118 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:43.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b46ffd700 1 -- 172.21.15.60:0/1084287118 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285516552 0 0) 0x7f8b480cf3f0 con 0x7f8b480b12a0 2023-07-13T22:30:43.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b4cb9c700 1 -- 172.21.15.60:0/1084287118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480cefa0 msgr2=0x7f8b4806bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:43.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b4cb9c700 1 --2- 172.21.15.60:0/1084287118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480cefa0 0x7f8b4806bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b4cb9c700 1 -- 172.21.15.60:0/1084287118 >> v1:172.21.15.105:6789/0 conn(0x7f8b480b12a0 legacy=0x7f8b480b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b4cb9c700 1 -- 172.21.15.60:0/1084287118 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b481609a0 con 0x7f8b480cd760 2023-07-13T22:30:43.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b477fe700 1 --2- 172.21.15.60:0/1084287118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480cefa0 0x7f8b4806bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:43.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b4cb9c700 1 --2- 172.21.15.60:0/1084287118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b480cd760 0x7f8b480cdbe0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009e80 tx=0x7f8b4000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baa206285bff4dd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:43.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b46ffd700 1 -- 172.21.15.60:0/1084287118 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000e070 con 0x7f8b480cd760 2023-07-13T22:30:43.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b46ffd700 1 -- 172.21.15.60:0/1084287118 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4000b8d0 con 0x7f8b480cd760 2023-07-13T22:30:43.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.439+0000 7f8b46ffd700 1 -- 172.21.15.60:0/1084287118 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000bc80 con 0x7f8b480cd760 2023-07-13T22:30:43.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.440+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/1084287118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b480cd760 msgr2=0x7f8b480cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.440+0000 7f8b4e5ff700 1 --2- 172.21.15.60:0/1084287118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b480cd760 0x7f8b480cdbe0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009e80 tx=0x7f8b4000b330 comp rx=0 tx=0).stop 2023-07-13T22:30:43.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.440+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/1084287118 shutdown_connections 2023-07-13T22:30:43.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.440+0000 7f8b4e5ff700 1 --2- 172.21.15.60:0/1084287118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b480cd760 0x7f8b480cdbe0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.440+0000 7f8b4e5ff700 1 --2- 172.21.15.60:0/1084287118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480cefa0 0x7f8b4806bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.440+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/1084287118 >> 172.21.15.60:0/1084287118 conn(0x7f8b480cc2a0 msgr2=0x7f8b480cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:43.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.440+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/1084287118 shutdown_connections 2023-07-13T22:30:43.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.440+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/1084287118 wait complete. 2023-07-13T22:30:43.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.442+0000 7f8b4e5ff700 1 Processor -- start 2023-07-13T22:30:43.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.442+0000 7f8b4e5ff700 1 -- start start 2023-07-13T22:30:43.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4e5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480b12a0 0x7f8b48155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4e5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b480cd760 0x7f8b4815eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4e5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b480cefa0 0x7f8b4815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4e5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48155780 con 0x7f8b480cefa0 2023-07-13T22:30:43.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4e5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4815d710 con 0x7f8b480cd760 2023-07-13T22:30:43.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4e5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4815d9f0 con 0x7f8b480b12a0 2023-07-13T22:30:43.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4cb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b480cefa0 0x7f8b4815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4cb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b480cefa0 0x7f8b4815f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44766/0 (socket says 172.21.15.60:44766) 2023-07-13T22:30:43.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4cb9c700 1 -- 172.21.15.60:0/2659727225 learned_addr learned my addr 172.21.15.60:0/2659727225 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:43.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b477fe700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b480cd760 0x7f8b4815eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b47fff700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480b12a0 0x7f8b48155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4cb9c700 1 -- 172.21.15.60:0/2659727225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480b12a0 msgr2=0x7f8b48155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.443+0000 7f8b4cb9c700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480b12a0 0x7f8b48155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.444+0000 7f8b4cb9c700 1 -- 172.21.15.60:0/2659727225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b480cd760 msgr2=0x7f8b4815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.444+0000 7f8b4cb9c700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b480cd760 0x7f8b4815eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.444+0000 7f8b4cb9c700 1 -- 172.21.15.60:0/2659727225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c0094a0 con 0x7f8b480cefa0 2023-07-13T22:30:43.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.444+0000 7f8b4cb9c700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b480cefa0 0x7f8b4815f3e0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000b680 tx=0x7f8b40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:43.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.444+0000 7f8b44ff9700 1 -- 172.21.15.60:0/2659727225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4001c640 con 0x7f8b480cefa0 2023-07-13T22:30:43.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.444+0000 7f8b44ff9700 1 -- 172.21.15.60:0/2659727225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4001c7c0 con 0x7f8b480cefa0 2023-07-13T22:30:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.444+0000 7f8b44ff9700 1 -- 172.21.15.60:0/2659727225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4001cb70 con 0x7f8b480cefa0 2023-07-13T22:30:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.444+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b40009b30 con 0x7f8b480cefa0 2023-07-13T22:30:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.444+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4815dfb0 con 0x7f8b480cefa0 2023-07-13T22:30:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.445+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b48071c90 con 0x7f8b480cefa0 2023-07-13T22:30:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.446+0000 7f8b44ff9700 1 -- 172.21.15.60:0/2659727225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b4001ccd0 con 0x7f8b480cefa0 2023-07-13T22:30:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.447+0000 7f8b44ff9700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b3406ace0 0x7f8b3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.447+0000 7f8b44ff9700 1 -- 172.21.15.60:0/2659727225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b4008ae30 con 0x7f8b480cefa0 2023-07-13T22:30:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.448+0000 7f8b47fff700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b3406ace0 0x7f8b3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.448+0000 7f8b47fff700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b3406ace0 0x7f8b3406d190 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000f60 tx=0x7f8b38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:43.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.452+0000 7f8b44ff9700 1 -- 172.21.15.60:0/2659727225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8b40055ff0 con 0x7f8b480cefa0 2023-07-13T22:30:43.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.705+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b480734a0 con 0x7f8b480cefa0 2023-07-13T22:30:43.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.706+0000 7f8b44ff9700 1 -- 172.21.15.60:0/2659727225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f8b40059610 con 0x7f8b480cefa0 2023-07-13T22:30:43.685 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:43.686 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:30:39.690428+0000","modified":"2023-07-13T22:30:43.199497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6836/1781435251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1781435251},{"type":"v1","addr":"172.21.15.60:6836","nonce":1781435251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-13T22:30:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b3406ace0 msgr2=0x7f8b3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b3406ace0 0x7f8b3406d190 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8b38000f60 tx=0x7f8b38009040 comp rx=0 tx=0).stop 2023-07-13T22:30:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b480cefa0 msgr2=0x7f8b4815f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b480cefa0 0x7f8b4815f3e0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000b680 tx=0x7f8b40004ad0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 shutdown_connections 2023-07-13T22:30:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b3406ace0 0x7f8b3406d190 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b480cefa0 0x7f8b4815f3e0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b480b12a0 0x7f8b48155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 --2- 172.21.15.60:0/2659727225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b480cd760 0x7f8b4815eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.708+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 >> 172.21.15.60:0/2659727225 conn(0x7f8b480cc2a0 msgr2=0x7f8b480b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.709+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 shutdown_connections 2023-07-13T22:30:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.709+0000 7f8b4e5ff700 1 -- 172.21.15.60:0/2659727225 wait complete. 2023-07-13T22:30:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 85 2023-07-13T22:30:43.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:30:43.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.930+0000 7fdf85fff700 1 Processor -- start 2023-07-13T22:30:43.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.930+0000 7fdf85fff700 1 -- start start 2023-07-13T22:30:43.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.931+0000 7fdf85fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800b8400 0x7fdf800cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.931+0000 7fdf85fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf800cfca0 0x7fdf800d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.931+0000 7fdf85fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf800d0ac0 con 0x7fdf800d3750 2023-07-13T22:30:43.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.931+0000 7fdf85fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf800b87d0 con 0x7fdf800cfca0 2023-07-13T22:30:43.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.931+0000 7fdf85fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800b4eb0 con 0x7fdf800b8400 2023-07-13T22:30:43.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.931+0000 7fdf7effd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdf800d3750 0x7fdf800cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:50980/0 (socket says 172.21.15.60:50980) 2023-07-13T22:30:43.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.931+0000 7fdf7effd700 1 -- 172.21.15.60:0/2661305979 learned_addr learned my addr 172.21.15.60:0/2661305979 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:43.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.932+0000 7fdf7f7fe700 1 --2- 172.21.15.60:0/2661305979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800b8400 0x7fdf800cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.932+0000 7fdf7ffff700 1 --2- 172.21.15.60:0/2661305979 >> v2:172.21.15.105:3300/0 conn(0x7fdf800cfca0 0x7fdf800d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.932+0000 7fdf7ffff700 1 -- 172.21.15.60:0/2661305979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800b8400 msgr2=0x7fdf800cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:43.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.932+0000 7fdf7ffff700 1 --2- 172.21.15.60:0/2661305979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800b8400 0x7fdf800cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.932+0000 7fdf7ffff700 1 -- 172.21.15.60:0/2661305979 >> v1:172.21.15.105:6789/0 conn(0x7fdf800d3750 legacy=0x7fdf800cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.932+0000 7fdf7ffff700 1 -- 172.21.15.60:0/2661305979 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800b5190 con 0x7fdf800cfca0 2023-07-13T22:30:43.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.932+0000 7fdf7f7fe700 1 --2- 172.21.15.60:0/2661305979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800b8400 0x7fdf800cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:43.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.932+0000 7fdf7ffff700 1 --2- 172.21.15.60:0/2661305979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800cfca0 0x7fdf800d0f50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009d90 tx=0x7fdf7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac1bb20fec22665b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:43.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf7e7fc700 1 -- 172.21.15.60:0/2661305979 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800e070 con 0x7fdf800cfca0 2023-07-13T22:30:43.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf7e7fc700 1 -- 172.21.15.60:0/2661305979 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7800bb20 con 0x7fdf800cfca0 2023-07-13T22:30:43.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf7e7fc700 1 -- 172.21.15.60:0/2661305979 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78012430 con 0x7fdf800cfca0 2023-07-13T22:30:43.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf85fff700 1 -- 172.21.15.60:0/2661305979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800cfca0 msgr2=0x7fdf800d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf85fff700 1 --2- 172.21.15.60:0/2661305979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800cfca0 0x7fdf800d0f50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009d90 tx=0x7fdf7800b330 comp rx=0 tx=0).stop 2023-07-13T22:30:43.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf85fff700 1 -- 172.21.15.60:0/2661305979 shutdown_connections 2023-07-13T22:30:43.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf85fff700 1 --2- 172.21.15.60:0/2661305979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800cfca0 0x7fdf800d0f50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf85fff700 1 --2- 172.21.15.60:0/2661305979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800b8400 0x7fdf800cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf85fff700 1 -- 172.21.15.60:0/2661305979 >> 172.21.15.60:0/2661305979 conn(0x7fdf8006b390 msgr2=0x7fdf8006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:43.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf85fff700 1 -- 172.21.15.60:0/2661305979 shutdown_connections 2023-07-13T22:30:43.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.933+0000 7fdf85fff700 1 -- 172.21.15.60:0/2661305979 wait complete. 2023-07-13T22:30:43.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.935+0000 7fdf85fff700 1 Processor -- start 2023-07-13T22:30:43.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.935+0000 7fdf85fff700 1 -- start start 2023-07-13T22:30:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.936+0000 7fdf85fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800b8400 0x7fdf801f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.936+0000 7fdf85fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800cfca0 0x7fdf800c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.936+0000 7fdf85fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf800d3750 0x7fdf80154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.936+0000 7fdf85fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801561d0 con 0x7fdf800d3750 2023-07-13T22:30:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.936+0000 7fdf85fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801f5bc0 con 0x7fdf800b8400 2023-07-13T22:30:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.936+0000 7fdf85fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801f5ea0 con 0x7fdf800cfca0 2023-07-13T22:30:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.936+0000 7fdf7ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf800d3750 0x7fdf80154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.936+0000 7fdf7ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf800d3750 0x7fdf80154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44776/0 (socket says 172.21.15.60:44776) 2023-07-13T22:30:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.936+0000 7fdf7ffff700 1 -- 172.21.15.60:0/832758728 learned_addr learned my addr 172.21.15.60:0/832758728 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7f7fe700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800b8400 0x7fdf801f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7effd700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800cfca0 0x7fdf800c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7ffff700 1 -- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800cfca0 msgr2=0x7fdf800c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7ffff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800cfca0 0x7fdf800c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7ffff700 1 -- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800b8400 msgr2=0x7fdf801f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:43.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7ffff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800b8400 0x7fdf801f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:43.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7ffff700 1 -- 172.21.15.60:0/832758728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700094a0 con 0x7fdf800d3750 2023-07-13T22:30:43.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7f7fe700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800b8400 0x7fdf801f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:43.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7effd700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800cfca0 0x7fdf800c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:43.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf7ffff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf800d3750 0x7fdf80154c80 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800b370 tx=0x7fdf78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf67fff700 1 -- 172.21.15.60:0/832758728 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78012430 con 0x7fdf800d3750 2023-07-13T22:30:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf67fff700 1 -- 172.21.15.60:0/832758728 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf78020070 con 0x7fdf800d3750 2023-07-13T22:30:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.937+0000 7fdf67fff700 1 -- 172.21.15.60:0/832758728 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78004280 con 0x7fdf800d3750 2023-07-13T22:30:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.938+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf78009a70 con 0x7fdf800d3750 2023-07-13T22:30:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.938+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf800c6750 con 0x7fdf800d3750 2023-07-13T22:30:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.939+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf800cd740 con 0x7fdf800d3750 2023-07-13T22:30:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.940+0000 7fdf67fff700 1 -- 172.21.15.60:0/832758728 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf780043e0 con 0x7fdf800d3750 2023-07-13T22:30:43.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.941+0000 7fdf67fff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf6c06ace0 0x7fdf6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.941+0000 7fdf67fff700 1 -- 172.21.15.60:0/832758728 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf78022070 con 0x7fdf800d3750 2023-07-13T22:30:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.941+0000 7fdf7f7fe700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf6c06ace0 0x7fdf6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.941+0000 7fdf7f7fe700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf6c06ace0 0x7fdf6c06d190 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009d20 tx=0x7fdf70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:43.945+0000 7fdf67fff700 1 -- 172.21.15.60:0/832758728 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdf78056340 con 0x7fdf800d3750 2023-07-13T22:30:44.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.154+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf800d2470 con 0x7fdf800d3750 2023-07-13T22:30:44.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.156+0000 7fdf67fff700 1 -- 172.21.15.60:0/832758728 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18215 (secure 0 0 0) 0x7fdf78059960 con 0x7fdf800d3750 2023-07-13T22:30:44.135 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:44.135 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":124,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:30:41.182757+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:30:37.983326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-13T22:30:38.665534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:30:44.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf6c06ace0 msgr2=0x7fdf6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf6c06ace0 0x7fdf6c06d190 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009d20 tx=0x7fdf70008040 comp rx=0 tx=0).stop 2023-07-13T22:30:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf800d3750 msgr2=0x7fdf80154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf800d3750 0x7fdf80154c80 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800b370 tx=0x7fdf78004ad0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 shutdown_connections 2023-07-13T22:30:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdf6c06ace0 0x7fdf6c06d190 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf800d3750 0x7fdf80154c80 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf800b8400 0x7fdf801f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 --2- 172.21.15.60:0/832758728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf800cfca0 0x7fdf800c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 >> 172.21.15.60:0/832758728 conn(0x7fdf8006b390 msgr2=0x7fdf800d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 shutdown_connections 2023-07-13T22:30:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.158+0000 7fdf85fff700 1 -- 172.21.15.60:0/832758728 wait complete. 2023-07-13T22:30:44.156 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:30:44.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:30:44.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.381+0000 7f69909ff700 1 Processor -- start 2023-07-13T22:30:44.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.382+0000 7f69909ff700 1 -- start start 2023-07-13T22:30:44.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.383+0000 7f69909ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f698c0cefa0 0x7f698c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.383+0000 7f69909ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c0cf3f0 con 0x7f698c0cd760 2023-07-13T22:30:44.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.383+0000 7f69909ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c1609a0 con 0x7f698c0b12a0 2023-07-13T22:30:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.383+0000 7f69909ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698c06a5f0 con 0x7f698c0cefa0 2023-07-13T22:30:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.383+0000 7f698ad9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f698c0cd760 0x7f698c0cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43622/0 (socket says 172.21.15.60:43622) 2023-07-13T22:30:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.383+0000 7f698ad9d700 1 -- 172.21.15.60:0/3829838125 learned_addr learned my addr 172.21.15.60:0/3829838125 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:44.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.383+0000 7f6983fff700 1 --2- 172.21.15.60:0/3829838125 >> v2:172.21.15.105:3300/0 conn(0x7f698c0cefa0 0x7f698c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976281369 0 0) 0x7f698c0cf3f0 con 0x7f698c0cd760 2023-07-13T22:30:44.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69780035b0 con 0x7f698c0cd760 2023-07-13T22:30:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3074660455 0 0) 0x7f69780035b0 con 0x7f698c0cd760 2023-07-13T22:30:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 >> v2:172.21.15.105:3300/0 conn(0x7f698c0cefa0 msgr2=0x7f698c06bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 --2- 172.21.15.60:0/3829838125 >> v2:172.21.15.105:3300/0 conn(0x7f698c0cefa0 0x7f698c06bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 >> v1:172.21.15.105:6789/0 conn(0x7f698c0b12a0 legacy=0x7f698c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c15c860 con 0x7f698c0cd760 2023-07-13T22:30:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f69840019f0 con 0x7f698c0cd760 2023-07-13T22:30:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6983fff700 1 --2- 172.21.15.60:0/3829838125 >> v2:172.21.15.105:3300/0 conn(0x7f698c0cefa0 0x7f698c06bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 >> v1:172.21.15.60:6789/0 conn(0x7f698c0cd760 legacy=0x7f698c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978003a00 0x7f6978003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978004850 0x7f6978004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6978005200 0x7f697800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c06a5f0 con 0x7f6978004850 2023-07-13T22:30:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697800c110 con 0x7f6978003a00 2023-07-13T22:30:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697800c2d0 con 0x7f6978005200 2023-07-13T22:30:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f6983fff700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978004850 0x7f6978004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.384+0000 7f698a59c700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978003a00 0x7f6978003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f698ad9d700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6978005200 0x7f697800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f6983fff700 1 -- 172.21.15.60:0/3829838125 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6978005200 msgr2=0x7f697800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f6983fff700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6978005200 0x7f697800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f6983fff700 1 -- 172.21.15.60:0/3829838125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978003a00 msgr2=0x7f6978003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f6983fff700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978003a00 0x7f6978003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f6983fff700 1 -- 172.21.15.60:0/3829838125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69780035b0 con 0x7f6978004850 2023-07-13T22:30:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f698a59c700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978003a00 0x7f6978003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f698ad9d700 1 -- 172.21.15.60:0/3829838125 reap_dead start 2023-07-13T22:30:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f6983fff700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978004850 0x7f6978004cc0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f6974009d30 tx=0x7f6978005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d909fafad86afdc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974009e80 con 0x7f6978004850 2023-07-13T22:30:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697400e9a0 con 0x7f6978004850 2023-07-13T22:30:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.385+0000 7f6989d9b700 1 -- 172.21.15.60:0/3829838125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697400ed10 con 0x7f6978004850 2023-07-13T22:30:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.386+0000 7f69909ff700 1 -- 172.21.15.60:0/3829838125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978004850 msgr2=0x7f6978004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.386+0000 7f69909ff700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978004850 0x7f6978004cc0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f6974009d30 tx=0x7f6978005880 comp rx=0 tx=0).stop 2023-07-13T22:30:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.386+0000 7f69909ff700 1 -- 172.21.15.60:0/3829838125 shutdown_connections 2023-07-13T22:30:44.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.386+0000 7f69909ff700 1 --2- 172.21.15.60:0/3829838125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978004850 0x7f6978004cc0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.386+0000 7f69909ff700 1 -- 172.21.15.60:0/3829838125 >> 172.21.15.60:0/3829838125 conn(0x7f698c0cc2a0 msgr2=0x7f698c0cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:44.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.386+0000 7f69909ff700 1 -- 172.21.15.60:0/3829838125 shutdown_connections 2023-07-13T22:30:44.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.386+0000 7f69909ff700 1 -- 172.21.15.60:0/3829838125 wait complete. 2023-07-13T22:30:44.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.388+0000 7f69909ff700 1 Processor -- start 2023-07-13T22:30:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.388+0000 7f69909ff700 1 -- start start 2023-07-13T22:30:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f69909ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978004850 0x7f698c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f69909ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698c0b3990 0x7f698c159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f69909ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f698c15e7f0 0x7f698c15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f69909ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c15f1b0 con 0x7f698c0b3990 2023-07-13T22:30:44.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f69909ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c0b57a0 con 0x7f6978004850 2023-07-13T22:30:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f69909ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698c0b51e0 con 0x7f698c15e7f0 2023-07-13T22:30:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f6983fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698c0b3990 0x7f698c159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f6983fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698c0b3990 0x7f698c159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44790/0 (socket says 172.21.15.60:44790) 2023-07-13T22:30:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f6983fff700 1 -- 172.21.15.60:0/427016766 learned_addr learned my addr 172.21.15.60:0/427016766 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f698ad9d700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f698c15e7f0 0x7f698c15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f698a59c700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978004850 0x7f698c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f6983fff700 1 -- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f698c15e7f0 msgr2=0x7f698c15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f6983fff700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f698c15e7f0 0x7f698c15ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f6983fff700 1 -- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978004850 msgr2=0x7f698c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f6983fff700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978004850 0x7f698c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f6983fff700 1 -- 172.21.15.60:0/427016766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c0b54c0 con 0x7f698c0b3990 2023-07-13T22:30:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f698ad9d700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f698c15e7f0 0x7f698c15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.389+0000 7f698a59c700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978004850 0x7f698c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.390+0000 7f6983fff700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698c0b3990 0x7f698c159050 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f6974000c00 tx=0x7f6974004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.390+0000 7f6982ffd700 1 -- 172.21.15.60:0/427016766 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974006e60 con 0x7f698c0b3990 2023-07-13T22:30:44.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.390+0000 7f6982ffd700 1 -- 172.21.15.60:0/427016766 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974009e80 con 0x7f698c0b3990 2023-07-13T22:30:44.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.390+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c0b3190 con 0x7f698c0b3990 2023-07-13T22:30:44.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.390+0000 7f6982ffd700 1 -- 172.21.15.60:0/427016766 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974004250 con 0x7f698c0b3990 2023-07-13T22:30:44.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.390+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c159590 con 0x7f698c0b3990 2023-07-13T22:30:44.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.391+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c0b36b0 con 0x7f698c0b3990 2023-07-13T22:30:44.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.392+0000 7f6982ffd700 1 -- 172.21.15.60:0/427016766 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69740043b0 con 0x7f698c0b3990 2023-07-13T22:30:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.393+0000 7f6982ffd700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f696c06ace0 0x7f696c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.393+0000 7f6982ffd700 1 -- 172.21.15.60:0/427016766 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f697408b120 con 0x7f698c0b3990 2023-07-13T22:30:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.393+0000 7f698a59c700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f696c06ace0 0x7f696c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.394+0000 7f698a59c700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f696c06ace0 0x7f696c06d190 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f698c068f70 tx=0x7f697c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:44.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.397+0000 7f6982ffd700 1 -- 172.21.15.60:0/427016766 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f698c0b36b0 con 0x7f698c0b3990 2023-07-13T22:30:44.539 INFO:tasks.ceph.mds.a.smithi060.stderr:2023-07-13T22:30:44.559+0000 7fc2203f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:30:44.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.602+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f698c15f1b0 con 0x7f696c06ace0 2023-07-13T22:30:44.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.612+0000 7f6982ffd700 1 -- 172.21.15.60:0/427016766 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311851 (secure 0 0 0) 0x7f698c15f1b0 con 0x7f696c06ace0 2023-07-13T22:30:44.594 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:44.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f696c06ace0 msgr2=0x7f696c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f696c06ace0 0x7f696c06d190 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f698c068f70 tx=0x7f697c009040 comp rx=0 tx=0).stop 2023-07-13T22:30:44.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698c0b3990 msgr2=0x7f698c159050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698c0b3990 0x7f698c159050 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f6974000c00 tx=0x7f6974004750 comp rx=0 tx=0).stop 2023-07-13T22:30:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 shutdown_connections 2023-07-13T22:30:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f696c06ace0 0x7f696c06d190 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f698c15e7f0 0x7f698c15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6978004850 0x7f698c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 --2- 172.21.15.60:0/427016766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698c0b3990 0x7f698c159050 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 >> 172.21.15.60:0/427016766 conn(0x7f698c0cc2a0 msgr2=0x7f698c0afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 shutdown_connections 2023-07-13T22:30:44.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.616+0000 7f69909ff700 1 -- 172.21.15.60:0/427016766 wait complete. 2023-07-13T22:30:44.598 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:30:44.613 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2023-07-13T22:30:43.253878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286176,"kb_used_data":13216,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754688544,"statfs":{"total":773094113280,"available":772801069056,"internally_reserved":0,"allocated":13533184,"data_stored":6807160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.086433"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201511+0000","last_change":"2023-07-13T22:30:40.190432+0000","last_active":"2023-07-13T22:30:40.201511+0000","last_peered":"2023-07-13T22:30:40.201511+0000","last_clean":"2023-07-13T22:30:40.201511+0000","last_became_active":"2023-07-13T22:30:40.190121+0000","last_became_peered":"2023-07-13T22:30:40.190121+0000","last_unstale":"2023-07-13T22:30:40.201511+0000","last_undegraded":"2023-07-13T22:30:40.201511+0000","last_fullsized":"2023-07-13T22:30:40.201511+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:00:19.770457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.194019+0000","last_change":"2023-07-13T22:30:40.191642+0000","last_active":"2023-07-13T22:30:41.194019+0000","last_peered":"2023-07-13T22:30:41.194019+0000","last_clean":"2023-07-13T22:30:41.194019+0000","last_became_active":"2023-07-13T22:30:40.191360+0000","last_became_peered":"2023-07-13T22:30:40.191360+0000","last_unstale":"2023-07-13T22:30:41.194019+0000","last_undegraded":"2023-07-13T22:30:41.194019+0000","last_fullsized":"2023-07-13T22:30:41.194019+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:08:30.556883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203080+0000","last_change":"2023-07-13T22:30:40.194402+0000","last_active":"2023-07-13T22:30:40.203080+0000","last_peered":"2023-07-13T22:30:40.203080+0000","last_clean":"2023-07-13T22:30:40.203080+0000","last_became_active":"2023-07-13T22:30:40.193536+0000","last_became_peered":"2023-07-13T22:30:40.193536+0000","last_unstale":"2023-07-13T22:30:40.203080+0000","last_undegraded":"2023-07-13T22:30:40.203080+0000","last_fullsized":"2023-07-13T22:30:40.203080+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:49.950378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193742+0000","last_change":"2023-07-13T22:30:40.192117+0000","last_active":"2023-07-13T22:30:41.193742+0000","last_peered":"2023-07-13T22:30:41.193742+0000","last_clean":"2023-07-13T22:30:41.193742+0000","last_became_active":"2023-07-13T22:30:40.191836+0000","last_became_peered":"2023-07-13T22:30:40.191836+0000","last_unstale":"2023-07-13T22:30:41.193742+0000","last_undegraded":"2023-07-13T22:30:41.193742+0000","last_fullsized":"2023-07-13T22:30:41.193742+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:12:50.731596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.200588+0000","last_change":"2023-07-13T22:30:40.193248+0000","last_active":"2023-07-13T22:30:40.200588+0000","last_peered":"2023-07-13T22:30:40.200588+0000","last_clean":"2023-07-13T22:30:40.200588+0000","last_became_active":"2023-07-13T22:30:40.192985+0000","last_became_peered":"2023-07-13T22:30:40.192985+0000","last_unstale":"2023-07-13T22:30:40.200588+0000","last_undegraded":"2023-07-13T22:30:40.200588+0000","last_fullsized":"2023-07-13T22:30:40.200588+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:23.379147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191595+0000","last_change":"2023-07-13T22:30:40.193852+0000","last_active":"2023-07-13T22:30:41.191595+0000","last_peered":"2023-07-13T22:30:41.191595+0000","last_clean":"2023-07-13T22:30:41.191595+0000","last_became_active":"2023-07-13T22:30:40.193526+0000","last_became_peered":"2023-07-13T22:30:40.193526+0000","last_unstale":"2023-07-13T22:30:41.191595+0000","last_undegraded":"2023-07-13T22:30:41.191595+0000","last_fullsized":"2023-07-13T22:30:41.191595+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:23:41.404651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.206744+0000","last_change":"2023-07-13T22:30:40.200841+0000","last_active":"2023-07-13T22:30:40.206744+0000","last_peered":"2023-07-13T22:30:40.206744+0000","last_clean":"2023-07-13T22:30:40.206744+0000","last_became_active":"2023-07-13T22:30:40.200573+0000","last_became_peered":"2023-07-13T22:30:40.200573+0000","last_unstale":"2023-07-13T22:30:40.206744+0000","last_undegraded":"2023-07-13T22:30:40.206744+0000","last_fullsized":"2023-07-13T22:30:40.206744+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:49:15.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192181+0000","last_change":"2023-07-13T22:30:40.196485+0000","last_active":"2023-07-13T22:30:41.192181+0000","last_peered":"2023-07-13T22:30:41.192181+0000","last_clean":"2023-07-13T22:30:41.192181+0000","last_became_active":"2023-07-13T22:30:40.196161+0000","last_became_peered":"2023-07-13T22:30:40.196161+0000","last_unstale":"2023-07-13T22:30:41.192181+0000","last_undegraded":"2023-07-13T22:30:41.192181+0000","last_fullsized":"2023-07-13T22:30:41.192181+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:43.861241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191942+0000","last_change":"2023-07-13T22:30:40.184054+0000","last_active":"2023-07-13T22:30:41.191942+0000","last_peered":"2023-07-13T22:30:41.191942+0000","last_clean":"2023-07-13T22:30:41.191942+0000","last_became_active":"2023-07-13T22:30:40.183775+0000","last_became_peered":"2023-07-13T22:30:40.183775+0000","last_unstale":"2023-07-13T22:30:41.191942+0000","last_undegraded":"2023-07-13T22:30:41.191942+0000","last_fullsized":"2023-07-13T22:30:41.191942+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:09.556827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.204225+0000","last_change":"2023-07-13T22:30:40.203051+0000","last_active":"2023-07-13T22:30:40.204225+0000","last_peered":"2023-07-13T22:30:40.204225+0000","last_clean":"2023-07-13T22:30:40.204225+0000","last_became_active":"2023-07-13T22:30:40.202760+0000","last_became_peered":"2023-07-13T22:30:40.202760+0000","last_unstale":"2023-07-13T22:30:40.204225+0000","last_undegraded":"2023-07-13T22:30:40.204225+0000","last_fullsized":"2023-07-13T22:30:40.204225+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:47.380232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193908+0000","last_change":"2023-07-13T22:30:40.193563+0000","last_active":"2023-07-13T22:30:41.193908+0000","last_peered":"2023-07-13T22:30:41.193908+0000","last_clean":"2023-07-13T22:30:41.193908+0000","last_became_active":"2023-07-13T22:30:40.193074+0000","last_became_peered":"2023-07-13T22:30:40.193074+0000","last_unstale":"2023-07-13T22:30:41.193908+0000","last_undegraded":"2023-07-13T22:30:41.193908+0000","last_fullsized":"2023-07-13T22:30:41.193908+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:42.677782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.197808+0000","last_change":"2023-07-13T22:30:40.187168+0000","last_active":"2023-07-13T22:30:40.197808+0000","last_peered":"2023-07-13T22:30:40.197808+0000","last_clean":"2023-07-13T22:30:40.197808+0000","last_became_active":"2023-07-13T22:30:40.186809+0000","last_became_peered":"2023-07-13T22:30:40.186809+0000","last_unstale":"2023-07-13T22:30:40.197808+0000","last_undegraded":"2023-07-13T22:30:40.197808+0000","last_fullsized":"2023-07-13T22:30:40.197808+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:09:00.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191898+0000","last_change":"2023-07-13T22:30:40.203003+0000","last_active":"2023-07-13T22:30:41.191898+0000","last_peered":"2023-07-13T22:30:41.191898+0000","last_clean":"2023-07-13T22:30:41.191898+0000","last_became_active":"2023-07-13T22:30:40.202705+0000","last_became_peered":"2023-07-13T22:30:40.202705+0000","last_unstale":"2023-07-13T22:30:41.191898+0000","last_undegraded":"2023-07-13T22:30:41.191898+0000","last_fullsized":"2023-07-13T22:30:41.191898+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:08:30.556883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203654+0000","last_change":"2023-07-13T22:30:40.195746+0000","last_active":"2023-07-13T22:30:40.203654+0000","last_peered":"2023-07-13T22:30:40.203654+0000","last_clean":"2023-07-13T22:30:40.203654+0000","last_became_active":"2023-07-13T22:30:40.195485+0000","last_became_peered":"2023-07-13T22:30:40.195485+0000","last_unstale":"2023-07-13T22:30:40.203654+0000","last_undegraded":"2023-07-13T22:30:40.203654+0000","last_fullsized":"2023-07-13T22:30:40.203654+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:07.094968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201975+0000","last_change":"2023-07-13T22:30:40.198353+0000","last_active":"2023-07-13T22:30:40.201975+0000","last_peered":"2023-07-13T22:30:40.201975+0000","last_clean":"2023-07-13T22:30:40.201975+0000","last_became_active":"2023-07-13T22:30:40.198086+0000","last_became_peered":"2023-07-13T22:30:40.198086+0000","last_unstale":"2023-07-13T22:30:40.201975+0000","last_undegraded":"2023-07-13T22:30:40.201975+0000","last_fullsized":"2023-07-13T22:30:40.201975+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:02.408817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191683+0000","last_change":"2023-07-13T22:30:40.203350+0000","last_active":"2023-07-13T22:30:41.191683+0000","last_peered":"2023-07-13T22:30:41.191683+0000","last_clean":"2023-07-13T22:30:41.191683+0000","last_became_active":"2023-07-13T22:30:40.194537+0000","last_became_peered":"2023-07-13T22:30:40.194537+0000","last_unstale":"2023-07-13T22:30:41.191683+0000","last_undegraded":"2023-07-13T22:30:41.191683+0000","last_fullsized":"2023-07-13T22:30:41.191683+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:53:56.085379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191924+0000","last_change":"2023-07-13T22:30:40.194243+0000","last_active":"2023-07-13T22:30:41.191924+0000","last_peered":"2023-07-13T22:30:41.191924+0000","last_clean":"2023-07-13T22:30:41.191924+0000","last_became_active":"2023-07-13T22:30:40.193652+0000","last_became_peered":"2023-07-13T22:30:40.193652+0000","last_unstale":"2023-07-13T22:30:41.191924+0000","last_undegraded":"2023-07-13T22:30:41.191924+0000","last_fullsized":"2023-07-13T22:30:41.191924+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:38:32.075031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201780+0000","last_change":"2023-07-13T22:30:40.196195+0000","last_active":"2023-07-13T22:30:40.201780+0000","last_peered":"2023-07-13T22:30:40.201780+0000","last_clean":"2023-07-13T22:30:40.201780+0000","last_became_active":"2023-07-13T22:30:40.195865+0000","last_became_peered":"2023-07-13T22:30:40.195865+0000","last_unstale":"2023-07-13T22:30:40.201780+0000","last_undegraded":"2023-07-13T22:30:40.201780+0000","last_fullsized":"2023-07-13T22:30:40.201780+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:59:08.599187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.206232+0000","last_change":"2023-07-13T22:30:40.199476+0000","last_active":"2023-07-13T22:30:40.206232+0000","last_peered":"2023-07-13T22:30:40.206232+0000","last_clean":"2023-07-13T22:30:40.206232+0000","last_became_active":"2023-07-13T22:30:40.199151+0000","last_became_peered":"2023-07-13T22:30:40.199151+0000","last_unstale":"2023-07-13T22:30:40.206232+0000","last_undegraded":"2023-07-13T22:30:40.206232+0000","last_fullsized":"2023-07-13T22:30:40.206232+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:53:33.705631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201142+0000","last_change":"2023-07-13T22:30:40.193671+0000","last_active":"2023-07-13T22:30:40.201142+0000","last_peered":"2023-07-13T22:30:40.201142+0000","last_clean":"2023-07-13T22:30:40.201142+0000","last_became_active":"2023-07-13T22:30:40.193404+0000","last_became_peered":"2023-07-13T22:30:40.193404+0000","last_unstale":"2023-07-13T22:30:40.201142+0000","last_undegraded":"2023-07-13T22:30:40.201142+0000","last_fullsized":"2023-07-13T22:30:40.201142+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:14:45.866598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193706+0000","last_change":"2023-07-13T22:30:40.195545+0000","last_active":"2023-07-13T22:30:41.193706+0000","last_peered":"2023-07-13T22:30:41.193706+0000","last_clean":"2023-07-13T22:30:41.193706+0000","last_became_active":"2023-07-13T22:30:40.195226+0000","last_became_peered":"2023-07-13T22:30:40.195226+0000","last_unstale":"2023-07-13T22:30:41.193706+0000","last_undegraded":"2023-07-13T22:30:41.193706+0000","last_fullsized":"2023-07-13T22:30:41.193706+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:34:18.820692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.200678+0000","last_change":"2023-07-13T22:30:40.187917+0000","last_active":"2023-07-13T22:30:40.200678+0000","last_peered":"2023-07-13T22:30:40.200678+0000","last_clean":"2023-07-13T22:30:40.200678+0000","last_became_active":"2023-07-13T22:30:40.187567+0000","last_became_peered":"2023-07-13T22:30:40.187567+0000","last_unstale":"2023-07-13T22:30:40.200678+0000","last_undegraded":"2023-07-13T22:30:40.200678+0000","last_fullsized":"2023-07-13T22:30:40.200678+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:25:18.146934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193745+0000","last_change":"2023-07-13T22:30:40.193869+0000","last_active":"2023-07-13T22:30:41.193745+0000","last_peered":"2023-07-13T22:30:41.193745+0000","last_clean":"2023-07-13T22:30:41.193745+0000","last_became_active":"2023-07-13T22:30:40.193608+0000","last_became_peered":"2023-07-13T22:30:40.193608+0000","last_unstale":"2023-07-13T22:30:41.193745+0000","last_undegraded":"2023-07-13T22:30:41.193745+0000","last_fullsized":"2023-07-13T22:30:41.193745+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:27:38.856902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191224+0000","last_change":"2023-07-13T22:30:40.184811+0000","last_active":"2023-07-13T22:30:41.191224+0000","last_peered":"2023-07-13T22:30:41.191224+0000","last_clean":"2023-07-13T22:30:41.191224+0000","last_became_active":"2023-07-13T22:30:40.184291+0000","last_became_peered":"2023-07-13T22:30:40.184291+0000","last_unstale":"2023-07-13T22:30:41.191224+0000","last_undegraded":"2023-07-13T22:30:41.191224+0000","last_fullsized":"2023-07-13T22:30:41.191224+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:29.992933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207415+0000","last_change":"2023-07-13T22:30:40.199932+0000","last_active":"2023-07-13T22:30:40.207415+0000","last_peered":"2023-07-13T22:30:40.207415+0000","last_clean":"2023-07-13T22:30:40.207415+0000","last_became_active":"2023-07-13T22:30:40.199611+0000","last_became_peered":"2023-07-13T22:30:40.199611+0000","last_unstale":"2023-07-13T22:30:40.207415+0000","last_undegraded":"2023-07-13T22:30:40.207415+0000","last_fullsized":"2023-07-13T22:30:40.207415+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:25:21.644922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203231+0000","last_change":"2023-07-13T22:30:40.195808+0000","last_active":"2023-07-13T22:30:40.203231+0000","last_peered":"2023-07-13T22:30:40.203231+0000","last_clean":"2023-07-13T22:30:40.203231+0000","last_became_active":"2023-07-13T22:30:40.193918+0000","last_became_peered":"2023-07-13T22:30:40.193918+0000","last_unstale":"2023-07-13T22:30:40.203231+0000","last_undegraded":"2023-07-13T22:30:40.203231+0000","last_fullsized":"2023-07-13T22:30:40.203231+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:42.677782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207528+0000","last_change":"2023-07-13T22:30:40.205583+0000","last_active":"2023-07-13T22:30:40.207528+0000","last_peered":"2023-07-13T22:30:40.207528+0000","last_clean":"2023-07-13T22:30:40.207528+0000","last_became_active":"2023-07-13T22:30:40.205287+0000","last_became_peered":"2023-07-13T22:30:40.205287+0000","last_unstale":"2023-07-13T22:30:40.207528+0000","last_undegraded":"2023-07-13T22:30:40.207528+0000","last_fullsized":"2023-07-13T22:30:40.207528+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:48:10.825797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210229+0000","last_change":"2023-07-13T22:30:40.206024+0000","last_active":"2023-07-13T22:30:40.210229+0000","last_peered":"2023-07-13T22:30:40.210229+0000","last_clean":"2023-07-13T22:30:40.210229+0000","last_became_active":"2023-07-13T22:30:40.204733+0000","last_became_peered":"2023-07-13T22:30:40.204733+0000","last_unstale":"2023-07-13T22:30:40.210229+0000","last_undegraded":"2023-07-13T22:30:40.210229+0000","last_fullsized":"2023-07-13T22:30:40.210229+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:10:08.683125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207326+0000","last_change":"2023-07-13T22:30:40.199483+0000","last_active":"2023-07-13T22:30:40.207326+0000","last_peered":"2023-07-13T22:30:40.207326+0000","last_clean":"2023-07-13T22:30:40.207326+0000","last_became_active":"2023-07-13T22:30:40.199170+0000","last_became_peered":"2023-07-13T22:30:40.199170+0000","last_unstale":"2023-07-13T22:30:40.207326+0000","last_undegraded":"2023-07-13T22:30:40.207326+0000","last_fullsized":"2023-07-13T22:30:40.207326+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:27:38.856902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203159+0000","last_change":"2023-07-13T22:30:40.195472+0000","last_active":"2023-07-13T22:30:40.203159+0000","last_peered":"2023-07-13T22:30:40.203159+0000","last_clean":"2023-07-13T22:30:40.203159+0000","last_became_active":"2023-07-13T22:30:40.195148+0000","last_became_peered":"2023-07-13T22:30:40.195148+0000","last_unstale":"2023-07-13T22:30:40.203159+0000","last_undegraded":"2023-07-13T22:30:40.203159+0000","last_fullsized":"2023-07-13T22:30:40.203159+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:09.556827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192196+0000","last_change":"2023-07-13T22:30:40.183187+0000","last_active":"2023-07-13T22:30:41.192196+0000","last_peered":"2023-07-13T22:30:41.192196+0000","last_clean":"2023-07-13T22:30:41.192196+0000","last_became_active":"2023-07-13T22:30:40.182835+0000","last_became_peered":"2023-07-13T22:30:40.182835+0000","last_unstale":"2023-07-13T22:30:41.192196+0000","last_undegraded":"2023-07-13T22:30:41.192196+0000","last_fullsized":"2023-07-13T22:30:41.192196+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:26:17.126717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210555+0000","last_change":"2023-07-13T22:30:40.205915+0000","last_active":"2023-07-13T22:30:40.210555+0000","last_peered":"2023-07-13T22:30:40.210555+0000","last_clean":"2023-07-13T22:30:40.210555+0000","last_became_active":"2023-07-13T22:30:40.205478+0000","last_became_peered":"2023-07-13T22:30:40.205478+0000","last_unstale":"2023-07-13T22:30:40.210555+0000","last_undegraded":"2023-07-13T22:30:40.210555+0000","last_fullsized":"2023-07-13T22:30:40.210555+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:16:42.400114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192299+0000","last_change":"2023-07-13T22:30:40.191693+0000","last_active":"2023-07-13T22:30:41.192299+0000","last_peered":"2023-07-13T22:30:41.192299+0000","last_clean":"2023-07-13T22:30:41.192299+0000","last_became_active":"2023-07-13T22:30:40.191362+0000","last_became_peered":"2023-07-13T22:30:40.191362+0000","last_unstale":"2023-07-13T22:30:41.192299+0000","last_undegraded":"2023-07-13T22:30:41.192299+0000","last_fullsized":"2023-07-13T22:30:41.192299+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:11:48.441467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"123'4","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210760+0000","last_change":"2023-07-13T22:30:39.190158+0000","last_active":"2023-07-13T22:30:40.210760+0000","last_peered":"2023-07-13T22:30:40.210760+0000","last_clean":"2023-07-13T22:30:40.210760+0000","last_became_active":"2023-07-13T22:30:39.189844+0000","last_became_peered":"2023-07-13T22:30:39.189844+0000","last_unstale":"2023-07-13T22:30:40.210760+0000","last_undegraded":"2023-07-13T22:30:40.210760+0000","last_fullsized":"2023-07-13T22:30:40.210760+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:08:13.226695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.200604+0000","last_change":"2023-07-13T22:30:40.187823+0000","last_active":"2023-07-13T22:30:40.200604+0000","last_peered":"2023-07-13T22:30:40.200604+0000","last_clean":"2023-07-13T22:30:40.200604+0000","last_became_active":"2023-07-13T22:30:40.187446+0000","last_became_peered":"2023-07-13T22:30:40.187446+0000","last_unstale":"2023-07-13T22:30:40.200604+0000","last_undegraded":"2023-07-13T22:30:40.200604+0000","last_fullsized":"2023-07-13T22:30:40.200604+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:27:54.560877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.205590+0000","last_change":"2023-07-13T22:30:39.188576+0000","last_active":"2023-07-13T22:30:40.205590+0000","last_peered":"2023-07-13T22:30:40.205590+0000","last_clean":"2023-07-13T22:30:40.205590+0000","last_became_active":"2023-07-13T22:30:39.188297+0000","last_became_peered":"2023-07-13T22:30:39.188297+0000","last_unstale":"2023-07-13T22:30:40.205590+0000","last_undegraded":"2023-07-13T22:30:40.205590+0000","last_fullsized":"2023-07-13T22:30:40.205590+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:45.254653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193827+0000","last_change":"2023-07-13T22:30:40.195496+0000","last_active":"2023-07-13T22:30:41.193827+0000","last_peered":"2023-07-13T22:30:41.193827+0000","last_clean":"2023-07-13T22:30:41.193827+0000","last_became_active":"2023-07-13T22:30:40.195225+0000","last_became_peered":"2023-07-13T22:30:40.195225+0000","last_unstale":"2023-07-13T22:30:41.193827+0000","last_undegraded":"2023-07-13T22:30:41.193827+0000","last_fullsized":"2023-07-13T22:30:41.193827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:23:41.404651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210330+0000","last_change":"2023-07-13T22:30:39.194075+0000","last_active":"2023-07-13T22:30:40.210330+0000","last_peered":"2023-07-13T22:30:40.210330+0000","last_clean":"2023-07-13T22:30:40.210330+0000","last_became_active":"2023-07-13T22:30:39.193740+0000","last_became_peered":"2023-07-13T22:30:39.193740+0000","last_unstale":"2023-07-13T22:30:40.210330+0000","last_undegraded":"2023-07-13T22:30:40.210330+0000","last_fullsized":"2023-07-13T22:30:40.210330+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:54:22.096286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207533+0000","last_change":"2023-07-13T22:30:40.200740+0000","last_active":"2023-07-13T22:30:40.207533+0000","last_peered":"2023-07-13T22:30:40.207533+0000","last_clean":"2023-07-13T22:30:40.207533+0000","last_became_active":"2023-07-13T22:30:40.200466+0000","last_became_peered":"2023-07-13T22:30:40.200466+0000","last_unstale":"2023-07-13T22:30:40.207533+0000","last_undegraded":"2023-07-13T22:30:40.207533+0000","last_fullsized":"2023-07-13T22:30:40.207533+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:58:12.446263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.208655+0000","last_change":"2023-07-13T22:30:39.193897+0000","last_active":"2023-07-13T22:30:40.208655+0000","last_peered":"2023-07-13T22:30:40.208655+0000","last_clean":"2023-07-13T22:30:40.208655+0000","last_became_active":"2023-07-13T22:30:39.193323+0000","last_became_peered":"2023-07-13T22:30:39.193323+0000","last_unstale":"2023-07-13T22:30:40.208655+0000","last_undegraded":"2023-07-13T22:30:40.208655+0000","last_fullsized":"2023-07-13T22:30:40.208655+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:44:43.316354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192611+0000","last_change":"2023-07-13T22:30:40.185192+0000","last_active":"2023-07-13T22:30:41.192611+0000","last_peered":"2023-07-13T22:30:41.192611+0000","last_clean":"2023-07-13T22:30:41.192611+0000","last_became_active":"2023-07-13T22:30:40.184917+0000","last_became_peered":"2023-07-13T22:30:40.184917+0000","last_unstale":"2023-07-13T22:30:41.192611+0000","last_undegraded":"2023-07-13T22:30:41.192611+0000","last_fullsized":"2023-07-13T22:30:41.192611+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:50:31.006950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"123'1","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192766+0000","last_change":"2023-07-13T22:30:39.197496+0000","last_active":"2023-07-13T22:30:41.192766+0000","last_peered":"2023-07-13T22:30:41.192766+0000","last_clean":"2023-07-13T22:30:41.192766+0000","last_became_active":"2023-07-13T22:30:39.197134+0000","last_became_peered":"2023-07-13T22:30:39.197134+0000","last_unstale":"2023-07-13T22:30:41.192766+0000","last_undegraded":"2023-07-13T22:30:41.192766+0000","last_fullsized":"2023-07-13T22:30:41.192766+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:22:56.737031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192524+0000","last_change":"2023-07-13T22:30:40.196191+0000","last_active":"2023-07-13T22:30:41.192524+0000","last_peered":"2023-07-13T22:30:41.192524+0000","last_clean":"2023-07-13T22:30:41.192524+0000","last_became_active":"2023-07-13T22:30:40.195892+0000","last_became_peered":"2023-07-13T22:30:40.195892+0000","last_unstale":"2023-07-13T22:30:41.192524+0000","last_undegraded":"2023-07-13T22:30:41.192524+0000","last_fullsized":"2023-07-13T22:30:41.192524+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:15:34.280385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.205338+0000","last_change":"2023-07-13T22:30:39.193923+0000","last_active":"2023-07-13T22:30:40.205338+0000","last_peered":"2023-07-13T22:30:40.205338+0000","last_clean":"2023-07-13T22:30:40.205338+0000","last_became_active":"2023-07-13T22:30:39.193596+0000","last_became_peered":"2023-07-13T22:30:39.193596+0000","last_unstale":"2023-07-13T22:30:40.205338+0000","last_undegraded":"2023-07-13T22:30:40.205338+0000","last_fullsized":"2023-07-13T22:30:40.205338+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:13.388691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201802+0000","last_change":"2023-07-13T22:30:40.197129+0000","last_active":"2023-07-13T22:30:40.201802+0000","last_peered":"2023-07-13T22:30:40.201802+0000","last_clean":"2023-07-13T22:30:40.201802+0000","last_became_active":"2023-07-13T22:30:40.196864+0000","last_became_peered":"2023-07-13T22:30:40.196864+0000","last_unstale":"2023-07-13T22:30:40.201802+0000","last_undegraded":"2023-07-13T22:30:40.201802+0000","last_fullsized":"2023-07-13T22:30:40.201802+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:22:41.562343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192107+0000","last_change":"2023-07-13T22:30:39.183459+0000","last_active":"2023-07-13T22:30:41.192107+0000","last_peered":"2023-07-13T22:30:41.192107+0000","last_clean":"2023-07-13T22:30:41.192107+0000","last_became_active":"2023-07-13T22:30:39.183160+0000","last_became_peered":"2023-07-13T22:30:39.183160+0000","last_unstale":"2023-07-13T22:30:41.192107+0000","last_undegraded":"2023-07-13T22:30:41.192107+0000","last_fullsized":"2023-07-13T22:30:41.192107+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:08:13.226695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201438+0000","last_change":"2023-07-13T22:30:40.195506+0000","last_active":"2023-07-13T22:30:40.201438+0000","last_peered":"2023-07-13T22:30:40.201438+0000","last_clean":"2023-07-13T22:30:40.201438+0000","last_became_active":"2023-07-13T22:30:40.195215+0000","last_became_peered":"2023-07-13T22:30:40.195215+0000","last_unstale":"2023-07-13T22:30:40.201438+0000","last_undegraded":"2023-07-13T22:30:40.201438+0000","last_fullsized":"2023-07-13T22:30:40.201438+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:13:27.143493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190524+0000","last_change":"2023-07-13T22:30:39.191376+0000","last_active":"2023-07-13T22:30:40.190524+0000","last_peered":"2023-07-13T22:30:40.190524+0000","last_clean":"2023-07-13T22:30:40.190524+0000","last_became_active":"2023-07-13T22:30:39.191053+0000","last_became_peered":"2023-07-13T22:30:39.191053+0000","last_unstale":"2023-07-13T22:30:40.190524+0000","last_undegraded":"2023-07-13T22:30:40.190524+0000","last_fullsized":"2023-07-13T22:30:40.190524+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:49.836095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207398+0000","last_change":"2023-07-13T22:30:40.199514+0000","last_active":"2023-07-13T22:30:40.207398+0000","last_peered":"2023-07-13T22:30:40.207398+0000","last_clean":"2023-07-13T22:30:40.207398+0000","last_became_active":"2023-07-13T22:30:40.199223+0000","last_became_peered":"2023-07-13T22:30:40.199223+0000","last_unstale":"2023-07-13T22:30:40.207398+0000","last_undegraded":"2023-07-13T22:30:40.207398+0000","last_fullsized":"2023-07-13T22:30:40.207398+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:38:32.075031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"123'1","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.194197+0000","last_change":"2023-07-13T22:30:39.197288+0000","last_active":"2023-07-13T22:30:41.194197+0000","last_peered":"2023-07-13T22:30:41.194197+0000","last_clean":"2023-07-13T22:30:41.194197+0000","last_became_active":"2023-07-13T22:30:39.196997+0000","last_became_peered":"2023-07-13T22:30:39.196997+0000","last_unstale":"2023-07-13T22:30:41.194197+0000","last_undegraded":"2023-07-13T22:30:41.194197+0000","last_fullsized":"2023-07-13T22:30:41.194197+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:16:31.024429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201443+0000","last_change":"2023-07-13T22:30:40.197590+0000","last_active":"2023-07-13T22:30:40.201443+0000","last_peered":"2023-07-13T22:30:40.201443+0000","last_clean":"2023-07-13T22:30:40.201443+0000","last_became_active":"2023-07-13T22:30:40.197051+0000","last_became_peered":"2023-07-13T22:30:40.197051+0000","last_unstale":"2023-07-13T22:30:40.201443+0000","last_undegraded":"2023-07-13T22:30:40.201443+0000","last_fullsized":"2023-07-13T22:30:40.201443+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:23:39.021951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210530+0000","last_change":"2023-07-13T22:30:39.190825+0000","last_active":"2023-07-13T22:30:40.210530+0000","last_peered":"2023-07-13T22:30:40.210530+0000","last_clean":"2023-07-13T22:30:40.210530+0000","last_became_active":"2023-07-13T22:30:39.190527+0000","last_became_peered":"2023-07-13T22:30:39.190527+0000","last_unstale":"2023-07-13T22:30:40.210530+0000","last_undegraded":"2023-07-13T22:30:40.210530+0000","last_fullsized":"2023-07-13T22:30:40.210530+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:13.530708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.209146+0000","last_change":"2023-07-13T22:30:40.206605+0000","last_active":"2023-07-13T22:30:40.209146+0000","last_peered":"2023-07-13T22:30:40.209146+0000","last_clean":"2023-07-13T22:30:40.209146+0000","last_became_active":"2023-07-13T22:30:40.206235+0000","last_became_peered":"2023-07-13T22:30:40.206235+0000","last_unstale":"2023-07-13T22:30:40.209146+0000","last_undegraded":"2023-07-13T22:30:40.209146+0000","last_fullsized":"2023-07-13T22:30:40.209146+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:57:26.820778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"123'1","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191874+0000","last_change":"2023-07-13T22:30:39.183359+0000","last_active":"2023-07-13T22:30:41.191874+0000","last_peered":"2023-07-13T22:30:41.191874+0000","last_clean":"2023-07-13T22:30:41.191874+0000","last_became_active":"2023-07-13T22:30:39.183064+0000","last_became_peered":"2023-07-13T22:30:39.183064+0000","last_unstale":"2023-07-13T22:30:41.191874+0000","last_undegraded":"2023-07-13T22:30:41.191874+0000","last_fullsized":"2023-07-13T22:30:41.191874+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:48.944306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210182+0000","last_change":"2023-07-13T22:30:40.205760+0000","last_active":"2023-07-13T22:30:40.210182+0000","last_peered":"2023-07-13T22:30:40.210182+0000","last_clean":"2023-07-13T22:30:40.210182+0000","last_became_active":"2023-07-13T22:30:40.205449+0000","last_became_peered":"2023-07-13T22:30:40.205449+0000","last_unstale":"2023-07-13T22:30:40.210182+0000","last_undegraded":"2023-07-13T22:30:40.210182+0000","last_fullsized":"2023-07-13T22:30:40.210182+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:35:32.375163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.183767+0000","last_change":"2023-07-13T22:30:39.188025+0000","last_active":"2023-07-13T22:30:40.183767+0000","last_peered":"2023-07-13T22:30:40.183767+0000","last_clean":"2023-07-13T22:30:40.183767+0000","last_became_active":"2023-07-13T22:30:39.187653+0000","last_became_peered":"2023-07-13T22:30:39.187653+0000","last_unstale":"2023-07-13T22:30:40.183767+0000","last_undegraded":"2023-07-13T22:30:40.183767+0000","last_fullsized":"2023-07-13T22:30:40.183767+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:04:08.557622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201123+0000","last_change":"2023-07-13T22:30:40.192402+0000","last_active":"2023-07-13T22:30:40.201123+0000","last_peered":"2023-07-13T22:30:40.201123+0000","last_clean":"2023-07-13T22:30:40.201123+0000","last_became_active":"2023-07-13T22:30:40.192031+0000","last_became_peered":"2023-07-13T22:30:40.192031+0000","last_unstale":"2023-07-13T22:30:40.201123+0000","last_undegraded":"2023-07-13T22:30:40.201123+0000","last_fullsized":"2023-07-13T22:30:40.201123+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:54:46.563026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"123'2","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192780+0000","last_change":"2023-07-13T22:30:39.195557+0000","last_active":"2023-07-13T22:30:41.192780+0000","last_peered":"2023-07-13T22:30:41.192780+0000","last_clean":"2023-07-13T22:30:41.192780+0000","last_became_active":"2023-07-13T22:30:39.195305+0000","last_became_peered":"2023-07-13T22:30:39.195305+0000","last_unstale":"2023-07-13T22:30:41.192780+0000","last_undegraded":"2023-07-13T22:30:41.192780+0000","last_fullsized":"2023-07-13T22:30:41.192780+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:57:20.830932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203626+0000","last_change":"2023-07-13T22:30:40.195192+0000","last_active":"2023-07-13T22:30:40.203626+0000","last_peered":"2023-07-13T22:30:40.203626+0000","last_clean":"2023-07-13T22:30:40.203626+0000","last_became_active":"2023-07-13T22:30:40.194847+0000","last_became_peered":"2023-07-13T22:30:40.194847+0000","last_unstale":"2023-07-13T22:30:40.203626+0000","last_undegraded":"2023-07-13T22:30:40.203626+0000","last_fullsized":"2023-07-13T22:30:40.203626+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:44:01.374628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.180805+0000","last_change":"2023-07-13T22:30:39.196206+0000","last_active":"2023-07-13T22:30:40.180805+0000","last_peered":"2023-07-13T22:30:40.180805+0000","last_clean":"2023-07-13T22:30:40.180805+0000","last_became_active":"2023-07-13T22:30:39.195810+0000","last_became_peered":"2023-07-13T22:30:39.195810+0000","last_unstale":"2023-07-13T22:30:40.180805+0000","last_undegraded":"2023-07-13T22:30:40.180805+0000","last_fullsized":"2023-07-13T22:30:40.180805+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:09:56.887611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203502+0000","last_change":"2023-07-13T22:30:40.201911+0000","last_active":"2023-07-13T22:30:40.203502+0000","last_peered":"2023-07-13T22:30:40.203502+0000","last_clean":"2023-07-13T22:30:40.203502+0000","last_became_active":"2023-07-13T22:30:40.201569+0000","last_became_peered":"2023-07-13T22:30:40.201569+0000","last_unstale":"2023-07-13T22:30:40.203502+0000","last_undegraded":"2023-07-13T22:30:40.203502+0000","last_fullsized":"2023-07-13T22:30:40.203502+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:54:15.386884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.186610+0000","last_change":"2023-07-13T22:30:39.197270+0000","last_active":"2023-07-13T22:30:40.186610+0000","last_peered":"2023-07-13T22:30:40.186610+0000","last_clean":"2023-07-13T22:30:40.186610+0000","last_became_active":"2023-07-13T22:30:39.196967+0000","last_became_peered":"2023-07-13T22:30:39.196967+0000","last_unstale":"2023-07-13T22:30:40.186610+0000","last_undegraded":"2023-07-13T22:30:40.186610+0000","last_fullsized":"2023-07-13T22:30:40.186610+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:34:32.892453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.194200+0000","last_change":"2023-07-13T22:30:40.202506+0000","last_active":"2023-07-13T22:30:41.194200+0000","last_peered":"2023-07-13T22:30:41.194200+0000","last_clean":"2023-07-13T22:30:41.194200+0000","last_became_active":"2023-07-13T22:30:40.202174+0000","last_became_peered":"2023-07-13T22:30:40.202174+0000","last_unstale":"2023-07-13T22:30:41.194200+0000","last_undegraded":"2023-07-13T22:30:41.194200+0000","last_fullsized":"2023-07-13T22:30:41.194200+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:38:32.075031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191517+0000","last_change":"2023-07-13T22:30:39.197455+0000","last_active":"2023-07-13T22:30:41.191517+0000","last_peered":"2023-07-13T22:30:41.191517+0000","last_clean":"2023-07-13T22:30:41.191517+0000","last_became_active":"2023-07-13T22:30:39.197107+0000","last_became_peered":"2023-07-13T22:30:39.197107+0000","last_unstale":"2023-07-13T22:30:41.191517+0000","last_undegraded":"2023-07-13T22:30:41.191517+0000","last_fullsized":"2023-07-13T22:30:41.191517+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:34:17.814620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201772+0000","last_change":"2023-07-13T22:30:40.197713+0000","last_active":"2023-07-13T22:30:40.201772+0000","last_peered":"2023-07-13T22:30:40.201772+0000","last_clean":"2023-07-13T22:30:40.201772+0000","last_became_active":"2023-07-13T22:30:40.197442+0000","last_became_peered":"2023-07-13T22:30:40.197442+0000","last_unstale":"2023-07-13T22:30:40.201772+0000","last_undegraded":"2023-07-13T22:30:40.201772+0000","last_fullsized":"2023-07-13T22:30:40.201772+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:25:17.632017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210425+0000","last_change":"2023-07-13T22:30:39.194503+0000","last_active":"2023-07-13T22:30:40.210425+0000","last_peered":"2023-07-13T22:30:40.210425+0000","last_clean":"2023-07-13T22:30:40.210425+0000","last_became_active":"2023-07-13T22:30:39.194115+0000","last_became_peered":"2023-07-13T22:30:39.194115+0000","last_unstale":"2023-07-13T22:30:40.210425+0000","last_undegraded":"2023-07-13T22:30:40.210425+0000","last_fullsized":"2023-07-13T22:30:40.210425+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:03:25.313962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203252+0000","last_change":"2023-07-13T22:30:40.190711+0000","last_active":"2023-07-13T22:30:40.203252+0000","last_peered":"2023-07-13T22:30:40.203252+0000","last_clean":"2023-07-13T22:30:40.203252+0000","last_became_active":"2023-07-13T22:30:40.190199+0000","last_became_peered":"2023-07-13T22:30:40.190199+0000","last_unstale":"2023-07-13T22:30:40.203252+0000","last_undegraded":"2023-07-13T22:30:40.203252+0000","last_fullsized":"2023-07-13T22:30:40.203252+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:53:56.085379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192786+0000","last_change":"2023-07-13T22:30:39.195292+0000","last_active":"2023-07-13T22:30:41.192786+0000","last_peered":"2023-07-13T22:30:41.192786+0000","last_clean":"2023-07-13T22:30:41.192786+0000","last_became_active":"2023-07-13T22:30:39.195017+0000","last_became_peered":"2023-07-13T22:30:39.195017+0000","last_unstale":"2023-07-13T22:30:41.192786+0000","last_undegraded":"2023-07-13T22:30:41.192786+0000","last_fullsized":"2023-07-13T22:30:41.192786+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:51:30.536377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.209164+0000","last_change":"2023-07-13T22:30:40.206943+0000","last_active":"2023-07-13T22:30:40.209164+0000","last_peered":"2023-07-13T22:30:40.209164+0000","last_clean":"2023-07-13T22:30:40.209164+0000","last_became_active":"2023-07-13T22:30:40.206617+0000","last_became_peered":"2023-07-13T22:30:40.206617+0000","last_unstale":"2023-07-13T22:30:40.209164+0000","last_undegraded":"2023-07-13T22:30:40.209164+0000","last_fullsized":"2023-07-13T22:30:40.209164+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:09:35.812927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192580+0000","last_change":"2023-07-13T22:30:39.197773+0000","last_active":"2023-07-13T22:30:41.192580+0000","last_peered":"2023-07-13T22:30:41.192580+0000","last_clean":"2023-07-13T22:30:41.192580+0000","last_became_active":"2023-07-13T22:30:39.197294+0000","last_became_peered":"2023-07-13T22:30:39.197294+0000","last_unstale":"2023-07-13T22:30:41.192580+0000","last_undegraded":"2023-07-13T22:30:41.192580+0000","last_fullsized":"2023-07-13T22:30:41.192580+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:42.684100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210161+0000","last_change":"2023-07-13T22:30:40.205098+0000","last_active":"2023-07-13T22:30:40.210161+0000","last_peered":"2023-07-13T22:30:40.210161+0000","last_clean":"2023-07-13T22:30:40.210161+0000","last_became_active":"2023-07-13T22:30:40.204745+0000","last_became_peered":"2023-07-13T22:30:40.204745+0000","last_unstale":"2023-07-13T22:30:40.210161+0000","last_undegraded":"2023-07-13T22:30:40.210161+0000","last_fullsized":"2023-07-13T22:30:40.210161+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:44:21.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192013+0000","last_change":"2023-07-13T22:30:39.191471+0000","last_active":"2023-07-13T22:30:41.192013+0000","last_peered":"2023-07-13T22:30:41.192013+0000","last_clean":"2023-07-13T22:30:41.192013+0000","last_became_active":"2023-07-13T22:30:39.191158+0000","last_became_peered":"2023-07-13T22:30:39.191158+0000","last_unstale":"2023-07-13T22:30:41.192013+0000","last_undegraded":"2023-07-13T22:30:41.192013+0000","last_fullsized":"2023-07-13T22:30:41.192013+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:20:40.928869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191939+0000","last_change":"2023-07-13T22:30:40.193852+0000","last_active":"2023-07-13T22:30:41.191939+0000","last_peered":"2023-07-13T22:30:41.191939+0000","last_clean":"2023-07-13T22:30:41.191939+0000","last_became_active":"2023-07-13T22:30:40.193521+0000","last_became_peered":"2023-07-13T22:30:40.193521+0000","last_unstale":"2023-07-13T22:30:41.191939+0000","last_undegraded":"2023-07-13T22:30:41.191939+0000","last_fullsized":"2023-07-13T22:30:41.191939+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:16:32.030501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193950+0000","last_change":"2023-07-13T22:30:39.189314+0000","last_active":"2023-07-13T22:30:41.193950+0000","last_peered":"2023-07-13T22:30:41.193950+0000","last_clean":"2023-07-13T22:30:41.193950+0000","last_became_active":"2023-07-13T22:30:39.188963+0000","last_became_peered":"2023-07-13T22:30:39.188963+0000","last_unstale":"2023-07-13T22:30:41.193950+0000","last_undegraded":"2023-07-13T22:30:41.193950+0000","last_fullsized":"2023-07-13T22:30:41.193950+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:08.550755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191728+0000","last_change":"2023-07-13T22:30:40.195058+0000","last_active":"2023-07-13T22:30:41.191728+0000","last_peered":"2023-07-13T22:30:41.191728+0000","last_clean":"2023-07-13T22:30:41.191728+0000","last_became_active":"2023-07-13T22:30:40.194780+0000","last_became_peered":"2023-07-13T22:30:40.194780+0000","last_unstale":"2023-07-13T22:30:41.191728+0000","last_undegraded":"2023-07-13T22:30:41.191728+0000","last_fullsized":"2023-07-13T22:30:41.191728+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:42.677782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190560+0000","last_change":"2023-07-13T22:30:39.196661+0000","last_active":"2023-07-13T22:30:40.190560+0000","last_peered":"2023-07-13T22:30:40.190560+0000","last_clean":"2023-07-13T22:30:40.190560+0000","last_became_active":"2023-07-13T22:30:39.196324+0000","last_became_peered":"2023-07-13T22:30:39.196324+0000","last_unstale":"2023-07-13T22:30:40.190560+0000","last_undegraded":"2023-07-13T22:30:40.190560+0000","last_fullsized":"2023-07-13T22:30:40.190560+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:56.876139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191933+0000","last_change":"2023-07-13T22:30:40.201339+0000","last_active":"2023-07-13T22:30:41.191933+0000","last_peered":"2023-07-13T22:30:41.191933+0000","last_clean":"2023-07-13T22:30:41.191933+0000","last_became_active":"2023-07-13T22:30:40.200985+0000","last_became_peered":"2023-07-13T22:30:40.200985+0000","last_unstale":"2023-07-13T22:30:41.191933+0000","last_undegraded":"2023-07-13T22:30:41.191933+0000","last_fullsized":"2023-07-13T22:30:41.191933+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:27:38.856902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192234+0000","last_change":"2023-07-13T22:30:39.187771+0000","last_active":"2023-07-13T22:30:41.192234+0000","last_peered":"2023-07-13T22:30:41.192234+0000","last_clean":"2023-07-13T22:30:41.192234+0000","last_became_active":"2023-07-13T22:30:39.187450+0000","last_became_peered":"2023-07-13T22:30:39.187450+0000","last_unstale":"2023-07-13T22:30:41.192234+0000","last_undegraded":"2023-07-13T22:30:41.192234+0000","last_fullsized":"2023-07-13T22:30:41.192234+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:46.374160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207564+0000","last_change":"2023-07-13T22:30:40.200705+0000","last_active":"2023-07-13T22:30:40.207564+0000","last_peered":"2023-07-13T22:30:40.207564+0000","last_clean":"2023-07-13T22:30:40.207564+0000","last_became_active":"2023-07-13T22:30:40.200434+0000","last_became_peered":"2023-07-13T22:30:40.200434+0000","last_unstale":"2023-07-13T22:30:40.207564+0000","last_undegraded":"2023-07-13T22:30:40.207564+0000","last_fullsized":"2023-07-13T22:30:40.207564+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:24:23.477257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.182689+0000","last_change":"2023-07-13T22:30:39.195419+0000","last_active":"2023-07-13T22:30:40.182689+0000","last_peered":"2023-07-13T22:30:40.182689+0000","last_clean":"2023-07-13T22:30:40.182689+0000","last_became_active":"2023-07-13T22:30:39.195077+0000","last_became_peered":"2023-07-13T22:30:39.195077+0000","last_unstale":"2023-07-13T22:30:40.182689+0000","last_undegraded":"2023-07-13T22:30:40.182689+0000","last_fullsized":"2023-07-13T22:30:40.182689+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:22:59.279034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203596+0000","last_change":"2023-07-13T22:30:40.197129+0000","last_active":"2023-07-13T22:30:40.203596+0000","last_peered":"2023-07-13T22:30:40.203596+0000","last_clean":"2023-07-13T22:30:40.203596+0000","last_became_active":"2023-07-13T22:30:40.196831+0000","last_became_peered":"2023-07-13T22:30:40.196831+0000","last_unstale":"2023-07-13T22:30:40.203596+0000","last_undegraded":"2023-07-13T22:30:40.203596+0000","last_fullsized":"2023-07-13T22:30:40.203596+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:34:08.171713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193923+0000","last_change":"2023-07-13T22:30:39.187439+0000","last_active":"2023-07-13T22:30:41.193923+0000","last_peered":"2023-07-13T22:30:41.193923+0000","last_clean":"2023-07-13T22:30:41.193923+0000","last_became_active":"2023-07-13T22:30:39.187158+0000","last_became_peered":"2023-07-13T22:30:39.187158+0000","last_unstale":"2023-07-13T22:30:41.193923+0000","last_undegraded":"2023-07-13T22:30:41.193923+0000","last_fullsized":"2023-07-13T22:30:41.193923+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:48.944306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":203,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192616+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:30:41.192616+0000","last_peered":"2023-07-13T22:30:41.192616+0000","last_clean":"2023-07-13T22:30:41.192616+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:30:41.192616+0000","last_undegraded":"2023-07-13T22:30:41.192616+0000","last_fullsized":"2023-07-13T22:30:41.192616+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:37:59.427831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191976+0000","last_change":"2023-07-13T22:30:40.197356+0000","last_active":"2023-07-13T22:30:41.191976+0000","last_peered":"2023-07-13T22:30:41.191976+0000","last_clean":"2023-07-13T22:30:41.191976+0000","last_became_active":"2023-07-13T22:30:40.196964+0000","last_became_peered":"2023-07-13T22:30:40.196964+0000","last_unstale":"2023-07-13T22:30:41.191976+0000","last_undegraded":"2023-07-13T22:30:41.191976+0000","last_fullsized":"2023-07-13T22:30:41.191976+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:50.842167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.180016+0000","last_change":"2023-07-13T22:30:39.198859+0000","last_active":"2023-07-13T22:30:40.180016+0000","last_peered":"2023-07-13T22:30:40.180016+0000","last_clean":"2023-07-13T22:30:40.180016+0000","last_became_active":"2023-07-13T22:30:39.198316+0000","last_became_peered":"2023-07-13T22:30:39.198316+0000","last_unstale":"2023-07-13T22:30:40.180016+0000","last_undegraded":"2023-07-13T22:30:40.180016+0000","last_fullsized":"2023-07-13T22:30:40.180016+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:48:49.331181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":195,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.183859+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:30:40.183859+0000","last_peered":"2023-07-13T22:30:40.183859+0000","last_clean":"2023-07-13T22:30:40.183859+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:30:40.183859+0000","last_undegraded":"2023-07-13T22:30:40.183859+0000","last_fullsized":"2023-07-13T22:30:40.183859+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:48.921485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207628+0000","last_change":"2023-07-13T22:30:40.199452+0000","last_active":"2023-07-13T22:30:40.207628+0000","last_peered":"2023-07-13T22:30:40.207628+0000","last_clean":"2023-07-13T22:30:40.207628+0000","last_became_active":"2023-07-13T22:30:40.199165+0000","last_became_peered":"2023-07-13T22:30:40.199165+0000","last_unstale":"2023-07-13T22:30:40.207628+0000","last_undegraded":"2023-07-13T22:30:40.207628+0000","last_fullsized":"2023-07-13T22:30:40.207628+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:33:18.133451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190533+0000","last_change":"2023-07-13T22:30:39.193116+0000","last_active":"2023-07-13T22:30:40.190533+0000","last_peered":"2023-07-13T22:30:40.190533+0000","last_clean":"2023-07-13T22:30:40.190533+0000","last_became_active":"2023-07-13T22:30:39.192334+0000","last_became_peered":"2023-07-13T22:30:39.192334+0000","last_unstale":"2023-07-13T22:30:40.190533+0000","last_undegraded":"2023-07-13T22:30:40.190533+0000","last_fullsized":"2023-07-13T22:30:40.190533+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:20:40.928869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":211,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191883+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:30:41.191883+0000","last_peered":"2023-07-13T22:30:41.191883+0000","last_clean":"2023-07-13T22:30:41.191883+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:30:41.191883+0000","last_undegraded":"2023-07-13T22:30:41.191883+0000","last_fullsized":"2023-07-13T22:30:41.191883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:15:29.188156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":339,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.187784+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:30:40.187784+0000","last_peered":"2023-07-13T22:30:40.187784+0000","last_clean":"2023-07-13T22:30:40.187784+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:30:40.187784+0000","last_undegraded":"2023-07-13T22:30:40.187784+0000","last_fullsized":"2023-07-13T22:30:40.187784+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:17.361367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.192523+0000","last_change":"2023-07-13T22:30:40.186112+0000","last_active":"2023-07-13T22:30:40.192523+0000","last_peered":"2023-07-13T22:30:40.192523+0000","last_clean":"2023-07-13T22:30:40.192523+0000","last_became_active":"2023-07-13T22:30:40.185826+0000","last_became_peered":"2023-07-13T22:30:40.185826+0000","last_unstale":"2023-07-13T22:30:40.192523+0000","last_undegraded":"2023-07-13T22:30:40.192523+0000","last_fullsized":"2023-07-13T22:30:40.192523+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:08:58.753068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"123'8","reported_seq":21,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191599+0000","last_change":"2023-07-13T22:30:39.188971+0000","last_active":"2023-07-13T22:30:41.191599+0000","last_peered":"2023-07-13T22:30:41.191599+0000","last_clean":"2023-07-13T22:30:41.191599+0000","last_became_active":"2023-07-13T22:30:39.188674+0000","last_became_peered":"2023-07-13T22:30:39.188674+0000","last_unstale":"2023-07-13T22:30:41.191599+0000","last_undegraded":"2023-07-13T22:30:41.191599+0000","last_fullsized":"2023-07-13T22:30:41.191599+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:12:49.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":191,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190375+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:30:40.190375+0000","last_peered":"2023-07-13T22:30:40.190375+0000","last_clean":"2023-07-13T22:30:40.190375+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:30:40.190375+0000","last_undegraded":"2023-07-13T22:30:40.190375+0000","last_fullsized":"2023-07-13T22:30:40.190375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:06:39.722360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203101+0000","last_change":"2023-07-13T22:30:40.194585+0000","last_active":"2023-07-13T22:30:40.203101+0000","last_peered":"2023-07-13T22:30:40.203101+0000","last_clean":"2023-07-13T22:30:40.203101+0000","last_became_active":"2023-07-13T22:30:40.193900+0000","last_became_peered":"2023-07-13T22:30:40.193900+0000","last_unstale":"2023-07-13T22:30:40.203101+0000","last_undegraded":"2023-07-13T22:30:40.203101+0000","last_fullsized":"2023-07-13T22:30:40.203101+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:08:30.556883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190633+0000","last_change":"2023-07-13T22:30:39.196379+0000","last_active":"2023-07-13T22:30:40.190633+0000","last_peered":"2023-07-13T22:30:40.190633+0000","last_clean":"2023-07-13T22:30:40.190633+0000","last_became_active":"2023-07-13T22:30:39.195889+0000","last_became_peered":"2023-07-13T22:30:39.195889+0000","last_unstale":"2023-07-13T22:30:40.190633+0000","last_undegraded":"2023-07-13T22:30:40.190633+0000","last_fullsized":"2023-07-13T22:30:40.190633+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:45:27.186239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":191,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.187918+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:30:40.187918+0000","last_peered":"2023-07-13T22:30:40.187918+0000","last_clean":"2023-07-13T22:30:40.187918+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:30:40.187918+0000","last_undegraded":"2023-07-13T22:30:40.187918+0000","last_fullsized":"2023-07-13T22:30:40.187918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:54:07.240537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210211+0000","last_change":"2023-07-13T22:30:40.205807+0000","last_active":"2023-07-13T22:30:40.210211+0000","last_peered":"2023-07-13T22:30:40.210211+0000","last_clean":"2023-07-13T22:30:40.210211+0000","last_became_active":"2023-07-13T22:30:40.204333+0000","last_became_peered":"2023-07-13T22:30:40.204333+0000","last_unstale":"2023-07-13T22:30:40.210211+0000","last_undegraded":"2023-07-13T22:30:40.210211+0000","last_fullsized":"2023-07-13T22:30:40.210211+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:14.536780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.211480+0000","last_change":"2023-07-13T22:30:39.197229+0000","last_active":"2023-07-13T22:30:40.211480+0000","last_peered":"2023-07-13T22:30:40.211480+0000","last_clean":"2023-07-13T22:30:40.211480+0000","last_became_active":"2023-07-13T22:30:39.196939+0000","last_became_peered":"2023-07-13T22:30:39.196939+0000","last_unstale":"2023-07-13T22:30:40.211480+0000","last_undegraded":"2023-07-13T22:30:40.211480+0000","last_fullsized":"2023-07-13T22:30:40.211480+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:25:26.805497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":221,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193803+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:30:41.193803+0000","last_peered":"2023-07-13T22:30:41.193803+0000","last_clean":"2023-07-13T22:30:41.193803+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:30:41.193803+0000","last_undegraded":"2023-07-13T22:30:41.193803+0000","last_fullsized":"2023-07-13T22:30:41.193803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:44:03.628613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207350+0000","last_change":"2023-07-13T22:30:40.200725+0000","last_active":"2023-07-13T22:30:40.207350+0000","last_peered":"2023-07-13T22:30:40.207350+0000","last_clean":"2023-07-13T22:30:40.207350+0000","last_became_active":"2023-07-13T22:30:40.200471+0000","last_became_peered":"2023-07-13T22:30:40.200471+0000","last_unstale":"2023-07-13T22:30:40.207350+0000","last_undegraded":"2023-07-13T22:30:40.207350+0000","last_fullsized":"2023-07-13T22:30:40.207350+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:16:32.030501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"123'6","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.213322+0000","last_change":"2023-07-13T22:30:39.194137+0000","last_active":"2023-07-13T22:30:40.213322+0000","last_peered":"2023-07-13T22:30:40.213322+0000","last_clean":"2023-07-13T22:30:40.213322+0000","last_became_active":"2023-07-13T22:30:39.193839+0000","last_became_peered":"2023-07-13T22:30:39.193839+0000","last_unstale":"2023-07-13T22:30:40.213322+0000","last_undegraded":"2023-07-13T22:30:40.213322+0000","last_fullsized":"2023-07-13T22:30:40.213322+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:34:40.483086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":189,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.192243+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:30:40.192243+0000","last_peered":"2023-07-13T22:30:40.192243+0000","last_clean":"2023-07-13T22:30:40.192243+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:30:40.192243+0000","last_undegraded":"2023-07-13T22:30:40.192243+0000","last_fullsized":"2023-07-13T22:30:40.192243+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:16.226359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.199724+0000","last_change":"2023-07-13T22:30:40.189517+0000","last_active":"2023-07-13T22:30:40.199724+0000","last_peered":"2023-07-13T22:30:40.199724+0000","last_clean":"2023-07-13T22:30:40.199724+0000","last_became_active":"2023-07-13T22:30:40.189233+0000","last_became_peered":"2023-07-13T22:30:40.189233+0000","last_unstale":"2023-07-13T22:30:40.199724+0000","last_undegraded":"2023-07-13T22:30:40.199724+0000","last_fullsized":"2023-07-13T22:30:40.199724+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:12:50.731596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"123'3","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.209899+0000","last_change":"2023-07-13T22:30:39.192950+0000","last_active":"2023-07-13T22:30:40.209899+0000","last_peered":"2023-07-13T22:30:40.209899+0000","last_clean":"2023-07-13T22:30:40.209899+0000","last_became_active":"2023-07-13T22:30:39.192619+0000","last_became_peered":"2023-07-13T22:30:39.192619+0000","last_unstale":"2023-07-13T22:30:40.209899+0000","last_undegraded":"2023-07-13T22:30:40.209899+0000","last_fullsized":"2023-07-13T22:30:40.209899+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:40:16.319138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":189,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.191173+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:30:40.191173+0000","last_peered":"2023-07-13T22:30:40.191173+0000","last_clean":"2023-07-13T22:30:40.191173+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:30:40.191173+0000","last_undegraded":"2023-07-13T22:30:40.191173+0000","last_fullsized":"2023-07-13T22:30:40.191173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:13:54.923466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":2146304,"data_stored":1294981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.67600000000000005,"15min":0.63800000000000001},"min":{"1min":0.436,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78700000000000003,"5min":1.0780000000000001,"15min":1.115},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.83699999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.75}]},{"osd":1,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.64200000000000002,"15min":0.67200000000000004},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.29399999999999998},"max":{"1min":0.76200000000000001,"5min":0.94299999999999995,"15min":2.2610000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.34300000000000003},"max":{"1min":0.93300000000000005,"5min":1.0309999999999999,"15min":1.0609999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71499999999999997,"15min":0.76500000000000001},"min":{"1min":0.51600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90400000000000003,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.75700000000000001},"min":{"1min":0.54600000000000004,"5min":0.42999999999999999,"15min":0.36499999999999999},"max":{"1min":0.80900000000000005,"5min":1.107,"15min":3.4710000000000001},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.68799999999999994},"min":{"1min":0.496,"5min":0.45500000000000002,"15min":0.34599999999999997},"max":{"1min":0.86799999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.75900000000000001,"15min":0.75600000000000001},"min":{"1min":0.60199999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90300000000000002,"5min":1.024,"15min":1.1399999999999999},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.622,"15min":0.63800000000000001},"min":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.34899999999999998},"max":{"1min":0.86499999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.46700000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.1850000000000001},"last":0.622}]},{"osd":5,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63800000000000001,"15min":0.65600000000000003},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85099999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.0149999999999999,"15min":1.2110000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.69099999999999995,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.90500000000000003,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.46999999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.042},"last":0.57199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1544192,"data_stored":703631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.629},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.28799999999999998},"max":{"1min":0.88100000000000001,"5min":1.089,"15min":1.089},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.68500000000000005,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.33900000000000002},"max":{"1min":0.83599999999999997,"5min":1.075,"15min":1.075},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73199999999999998,"15min":0.70199999999999996},"min":{"1min":0.58099999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82399999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.48899999999999999,"5min":0.38500000000000001,"15min":0.379},"max":{"1min":0.84099999999999997,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.66900000000000004},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.042,"15min":1.042},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69899999999999995,"15min":0.66500000000000004},"min":{"1min":0.48599999999999999,"5min":0.435,"15min":0.37},"max":{"1min":0.88500000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73799999999999999,"15min":0.72599999999999998},"min":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.44},"max":{"1min":0.86899999999999999,"5min":1.107,"15min":1.107},"last":0.83599999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60499999999999998,"15min":0.64500000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.65600000000000003,"15min":0.68200000000000005},"min":{"1min":0.40400000000000003,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":0.90900000000000003,"5min":0.997,"15min":0.998},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.49199999999999999,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":0.90200000000000002,"15min":0.95999999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70899999999999996,"15min":0.73999999999999999},"min":{"1min":0.44,"5min":0.439,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.96099999999999997,"15min":1.1890000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69899999999999995,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.39000000000000001},"max":{"1min":0.94299999999999995,"5min":0.98099999999999998,"15min":1.111},"last":0.75900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":2134016,"data_stored":1293397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.438,"15min":0.47599999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.249},"max":{"1min":0.60199999999999998,"5min":0.70199999999999996,"15min":1.952},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.63300000000000001,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.53500000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.45900000000000002,"15min":0.51200000000000001},"min":{"1min":0.30299999999999999,"5min":0.29899999999999999,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":0.71299999999999997,"15min":2.1200000000000001},"last":0.495},{"interface":"front","average":{"1min":0.497,"5min":0.53700000000000003,"15min":0.57799999999999996},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.67300000000000004,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.60499999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.83899999999999997,"15min":2.008},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82299999999999995,"5min":1.054,"15min":1.054},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.745,"15min":0.78400000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96999999999999997,"5min":0.98199999999999998,"15min":2.028},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.59499999999999997,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.93899999999999995,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.78800000000000003,"15min":0.82599999999999996},"min":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.0720000000000001,"5min":1.095,"15min":2.0899999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.60899999999999999,"5min":0.55100000000000005,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":1.073,"15min":1.2230000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.83599999999999997},"min":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0289999999999999,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.50700000000000001},"max":{"1min":1.04,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.56499999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.046,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.67200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1540096,"data_stored":702029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45900000000000002,"15min":0.51400000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61599999999999999,"5min":0.84999999999999998,"15min":11.791},"last":0.496},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.32200000000000001,"5min":0.313,"15min":0.29299999999999998},"max":{"1min":0.58199999999999996,"5min":0.99199999999999999,"15min":4.657},"last":0.443}]},{"osd":2,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80500000000000005,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50700000000000001,"15min":0.872},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96199999999999997,"5min":1.276,"15min":13.398999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.502,"15min":0.54800000000000004},"min":{"1min":0.376,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.59699999999999998,"5min":0.75800000000000001,"15min":1.8089999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.55800000000000005,"15min":0.56499999999999995},"min":{"1min":0.40400000000000003,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.70699999999999996,"5min":1.095,"15min":4.4530000000000003},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.748,"15min":1.1160000000000001},"min":{"1min":0.54700000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75,"15min":0.79100000000000004},"min":{"1min":0.59499999999999997,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":1.0429999999999999,"5min":1.1220000000000001,"15min":1.776},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79000000000000004,"15min":1.139},"min":{"1min":0.504,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.113,"15min":13.349},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.86199999999999999},"min":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96099999999999997,"5min":1.302,"15min":4.7649999999999997},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.79700000000000004,"15min":0.82499999999999996},"min":{"1min":0.55500000000000005,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.0149999999999999,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77200000000000002,"15min":1.109},"min":{"1min":0.47199999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97799999999999998,"5min":1.2330000000000001,"15min":11.679},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":1.145},"min":{"1min":0.60899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.032,"5min":1.1759999999999999,"15min":13.298},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":0.876},"min":{"1min":0.58099999999999996,"5min":0.51800000000000002,"15min":0.50900000000000001},"max":{"1min":1.091,"5min":1.151,"15min":11.456},"last":0.63200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1540096,"data_stored":702589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44400000000000001,"15min":0.50900000000000001},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.249},"max":{"1min":1.605,"5min":1.605,"15min":2.282},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51200000000000001,"15min":0.47899999999999998},"min":{"1min":0.35699999999999998,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":0.66100000000000003,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47799999999999998,"15min":0.47199999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.26200000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.57699999999999996},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":2.319},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.311},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.627},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38700000000000001},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":2.351},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78200000000000003,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":2.238},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78000000000000003,"15min":0.77700000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.46899999999999997},"max":{"1min":1.0449999999999999,"5min":1.274,"15min":1.518},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.505},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.81399999999999995,"15min":0.84099999999999997},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":2.1909999999999998},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98499999999999999,"5min":1.0760000000000001,"15min":1.488},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.80100000000000005,"15min":0.78800000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.48199999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.747}]},{"osd":7,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.81000000000000005,"15min":0.79200000000000004},"min":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":1.0980000000000001,"5min":1.29,"15min":1.29},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.69899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1552384,"data_stored":704593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.248},"max":{"1min":0.56499999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.45500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.27200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.314,"5min":0.313,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.499},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.68999999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.44}]},{"osd":2,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.629}]},{"osd":4,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.51200000000000001,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":0.90900000000000003,"5min":1.149,"15min":1.2250000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.52900000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91900000000000004,"5min":1.113,"15min":1.1539999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":0.96899999999999997,"5min":1.1919999999999999,"15min":1.411},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53700000000000003,"5min":0.53400000000000003,"15min":0.48599999999999999},"max":{"1min":0.92300000000000004,"5min":1.175,"15min":1.5469999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.51800000000000002},"max":{"1min":0.97299999999999998,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77100000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"max":{"1min":0.99199999999999999,"5min":1.02,"15min":1.1240000000000001},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.58899999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78600000000000003,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.47199999999999998},"max":{"1min":1.0249999999999999,"5min":1.153,"15min":1.1950000000000001},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1536000,"data_stored":702959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65500000000000003,"15min":0.61899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.311},"max":{"1min":0.85399999999999998,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67500000000000004,"15min":0.64400000000000002},"min":{"1min":0.54600000000000004,"5min":0.39400000000000002,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.997,"15min":0.997},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.35099999999999998},"max":{"1min":0.90700000000000003,"5min":1.0489999999999999,"15min":1.298},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73599999999999999,"15min":0.69799999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.438},"max":{"1min":0.89600000000000002,"5min":1.0049999999999999,"15min":1.3109999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.371},"max":{"1min":0.88200000000000001,"5min":1.0629999999999999,"15min":1.1200000000000001},"last":0.628},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73299999999999998,"15min":0.71499999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86699999999999999,"5min":1.089,"15min":1.1839999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69999999999999996,"15min":0.89400000000000002},"min":{"1min":0.44700000000000001,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":0.93700000000000006,"5min":1.234,"15min":10.657999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.88600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85199999999999998,"5min":1.0600000000000001,"15min":8.75},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.65200000000000002,"15min":0.67900000000000005},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":0.89600000000000002,"5min":1.042,"15min":1.042},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.42199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82999999999999996,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":1.0189999999999999,"15min":1.173},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.48499999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93300000000000005,"5min":1.0640000000000001,"15min":1.099},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.46200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.87,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73999999999999999,"15min":0.76300000000000001},"min":{"1min":0.45100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.94599999999999995,"5min":1.1819999999999999,"15min":1.212},"last":0.80400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574970,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1540096,"data_stored":702981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.624},"min":{"1min":0.41499999999999998,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.76500000000000001,"5min":1.042,"15min":1.042},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.67600000000000005,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83799999999999997,"5min":0.998,"15min":0.998},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.66600000000000004},"min":{"1min":0.46899999999999997,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.84799999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.68000000000000005},"min":{"1min":0.505,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82699999999999996,"5min":0.94499999999999995,"15min":8.9930000000000003},"last":0.628}]},{"osd":2,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68300000000000005,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.35899999999999999},"max":{"1min":0.879,"5min":0.88300000000000001,"15min":0.96199999999999997},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71299999999999997,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.69499999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96599999999999997},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72499999999999998,"15min":0.69199999999999995},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67700000000000005,"15min":0.64300000000000002},"min":{"1min":0.52200000000000002,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.51600000000000001,"5min":0.438,"15min":0.40000000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.496,"15min":0.42299999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.88},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69099999999999995,"15min":0.67400000000000004},"min":{"1min":0.44800000000000001,"5min":0.40600000000000003,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.51000000000000001,"5min":0.47799999999999998,"15min":0.41199999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.76800000000000002},"min":{"1min":0.64900000000000002,"5min":0.54100000000000004,"15min":0.503},"max":{"1min":1.034,"5min":1.034,"15min":1.0669999999999999},"last":0.71299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:30:44.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:30:44.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.876+0000 7f7daa9ff700 1 Processor -- start 2023-07-13T22:30:44.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.876+0000 7f7daa9ff700 1 -- start start 2023-07-13T22:30:44.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.877+0000 7f7daa9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7da40b8570 0x7f7da40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.877+0000 7f7daa9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7da40d15c0 0x7f7da415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.877+0000 7f7daa9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da40d09d0 con 0x7f7da40d35c0 2023-07-13T22:30:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.877+0000 7f7daa9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da40d3b30 con 0x7f7da40b8570 2023-07-13T22:30:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.877+0000 7f7daa9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da40cda10 con 0x7f7da40d15c0 2023-07-13T22:30:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.878+0000 7f7da3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7da40b8570 0x7f7da40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.878+0000 7f7da3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7da40b8570 0x7f7da40ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44800/0 (socket says 172.21.15.60:44800) 2023-07-13T22:30:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.878+0000 7f7da3fff700 1 -- 172.21.15.60:0/4079449987 learned_addr learned my addr 172.21.15.60:0/4079449987 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.878+0000 7f7da8f9c700 1 --2- 172.21.15.60:0/4079449987 >> v2:172.21.15.105:3300/0 conn(0x7f7da40d15c0 0x7f7da415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.878+0000 7f7da3fff700 1 -- 172.21.15.60:0/4079449987 >> v2:172.21.15.105:3300/0 conn(0x7f7da40d15c0 msgr2=0x7f7da415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.878+0000 7f7da3fff700 1 --2- 172.21.15.60:0/4079449987 >> v2:172.21.15.105:3300/0 conn(0x7f7da40d15c0 0x7f7da415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.878+0000 7f7da3fff700 1 -- 172.21.15.60:0/4079449987 >> v1:172.21.15.105:6789/0 conn(0x7f7da40d35c0 legacy=0x7f7da40d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.878+0000 7f7da3fff700 1 -- 172.21.15.60:0/4079449987 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da40cd450 con 0x7f7da40b8570 2023-07-13T22:30:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.878+0000 7f7da3fff700 1 --2- 172.21.15.60:0/4079449987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40b8570 0x7f7da40ceeb0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7da40b2670 tx=0x7f7d94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520d47aa8384518c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7da2ffd700 1 -- 172.21.15.60:0/4079449987 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94005e30 con 0x7f7da40b8570 2023-07-13T22:30:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7da2ffd700 1 -- 172.21.15.60:0/4079449987 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d94004030 con 0x7f7da40b8570 2023-07-13T22:30:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7da2ffd700 1 -- 172.21.15.60:0/4079449987 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d940043e0 con 0x7f7da40b8570 2023-07-13T22:30:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7daa9ff700 1 -- 172.21.15.60:0/4079449987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40b8570 msgr2=0x7f7da40ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7daa9ff700 1 --2- 172.21.15.60:0/4079449987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40b8570 0x7f7da40ceeb0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7da40b2670 tx=0x7f7d94009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7daa9ff700 1 -- 172.21.15.60:0/4079449987 shutdown_connections 2023-07-13T22:30:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7daa9ff700 1 --2- 172.21.15.60:0/4079449987 >> v2:172.21.15.105:3300/0 conn(0x7f7da40d15c0 0x7f7da415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7daa9ff700 1 --2- 172.21.15.60:0/4079449987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40b8570 0x7f7da40ceeb0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7daa9ff700 1 -- 172.21.15.60:0/4079449987 >> 172.21.15.60:0/4079449987 conn(0x7f7da4071600 msgr2=0x7f7da40719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7daa9ff700 1 -- 172.21.15.60:0/4079449987 shutdown_connections 2023-07-13T22:30:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.879+0000 7f7daa9ff700 1 -- 172.21.15.60:0/4079449987 wait complete. 2023-07-13T22:30:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.881+0000 7f7daa9ff700 1 Processor -- start 2023-07-13T22:30:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.881+0000 7f7daa9ff700 1 -- start start 2023-07-13T22:30:44.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.882+0000 7f7daa9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da40b8570 0x7f7da4150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.882+0000 7f7daa9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da40d15c0 0x7f7da41f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.882+0000 7f7daa9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40d35c0 0x7f7da41f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.882+0000 7f7daa9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40c6790 con 0x7f7da40d35c0 2023-07-13T22:30:44.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.882+0000 7f7daa9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40c6a70 con 0x7f7da40d15c0 2023-07-13T22:30:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.882+0000 7f7daa9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40c6d50 con 0x7f7da40b8570 2023-07-13T22:30:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.882+0000 7f7da8f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40d35c0 0x7f7da41f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.882+0000 7f7da8f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40d35c0 0x7f7da41f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44806/0 (socket says 172.21.15.60:44806) 2023-07-13T22:30:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.882+0000 7f7da8f9c700 1 -- 172.21.15.60:0/1585691724 learned_addr learned my addr 172.21.15.60:0/1585691724 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da3fff700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da40b8570 0x7f7da4150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da37fe700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da40d15c0 0x7f7da41f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da8f9c700 1 -- 172.21.15.60:0/1585691724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da40b8570 msgr2=0x7f7da4150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da8f9c700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da40b8570 0x7f7da4150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da8f9c700 1 -- 172.21.15.60:0/1585691724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da40d15c0 msgr2=0x7f7da41f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da8f9c700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da40d15c0 0x7f7da41f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da8f9c700 1 -- 172.21.15.60:0/1585691724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d94009500 con 0x7f7da40d35c0 2023-07-13T22:30:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da37fe700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da40d15c0 0x7f7da41f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:44.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da8f9c700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40d35c0 0x7f7da41f28e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00efc0 tx=0x7f7d9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:44.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da0ff9700 1 -- 172.21.15.60:0/1585691724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c009ea0 con 0x7f7da40d35c0 2023-07-13T22:30:44.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da0ff9700 1 -- 172.21.15.60:0/1585691724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c00f040 con 0x7f7da40d35c0 2023-07-13T22:30:44.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da40c7030 con 0x7f7da40d35c0 2023-07-13T22:30:44.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.883+0000 7f7da0ff9700 1 -- 172.21.15.60:0/1585691724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c014480 con 0x7f7da40d35c0 2023-07-13T22:30:44.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.884+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da40c75f0 con 0x7f7da40d35c0 2023-07-13T22:30:44.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.885+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da40c7810 con 0x7f7da40d35c0 2023-07-13T22:30:44.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.885+0000 7f7da0ff9700 1 -- 172.21.15.60:0/1585691724 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d9c0145e0 con 0x7f7da40d35c0 2023-07-13T22:30:44.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.886+0000 7f7da0ff9700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d9006ace0 0x7f7d9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:44.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.886+0000 7f7da0ff9700 1 -- 172.21.15.60:0/1585691724 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d9c089de0 con 0x7f7da40d35c0 2023-07-13T22:30:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.887+0000 7f7da3fff700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d9006ace0 0x7f7d9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.887+0000 7f7da3fff700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d9006ace0 0x7f7d9006d190 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009c10 tx=0x7f7d94008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:44.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:44.891+0000 7f7da0ff9700 1 -- 172.21.15.60:0/1585691724 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7d9c054df0 con 0x7f7da40d35c0 2023-07-13T22:30:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.091+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7da40b5000 con 0x7f7d9006ace0 2023-07-13T22:30:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.100+0000 7f7da0ff9700 1 -- 172.21.15.60:0/1585691724 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311851 (secure 0 0 0) 0x7f7d94021070 con 0x7f7d9006ace0 2023-07-13T22:30:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d9006ace0 msgr2=0x7f7d9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d9006ace0 0x7f7d9006d190 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009c10 tx=0x7f7d94008070 comp rx=0 tx=0).stop 2023-07-13T22:30:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40d35c0 msgr2=0x7f7da41f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40d35c0 0x7f7da41f28e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00efc0 tx=0x7f7d9c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 shutdown_connections 2023-07-13T22:30:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7d9006ace0 0x7f7d9006d190 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7da40d35c0 0x7f7da41f28e0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da40b8570 0x7f7da4150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 --2- 172.21.15.60:0/1585691724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da40d15c0 0x7f7da41f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.104+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 >> 172.21.15.60:0/1585691724 conn(0x7f7da4071600 msgr2=0x7f7da406b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.105+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 shutdown_connections 2023-07-13T22:30:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.105+0000 7f7daa9ff700 1 -- 172.21.15.60:0/1585691724 wait complete. 2023-07-13T22:30:45.086 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:30:45.102 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2023-07-13T22:30:43.253878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286176,"kb_used_data":13216,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754688544,"statfs":{"total":773094113280,"available":772801069056,"internally_reserved":0,"allocated":13533184,"data_stored":6807160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.086433"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201511+0000","last_change":"2023-07-13T22:30:40.190432+0000","last_active":"2023-07-13T22:30:40.201511+0000","last_peered":"2023-07-13T22:30:40.201511+0000","last_clean":"2023-07-13T22:30:40.201511+0000","last_became_active":"2023-07-13T22:30:40.190121+0000","last_became_peered":"2023-07-13T22:30:40.190121+0000","last_unstale":"2023-07-13T22:30:40.201511+0000","last_undegraded":"2023-07-13T22:30:40.201511+0000","last_fullsized":"2023-07-13T22:30:40.201511+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:00:19.770457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.194019+0000","last_change":"2023-07-13T22:30:40.191642+0000","last_active":"2023-07-13T22:30:41.194019+0000","last_peered":"2023-07-13T22:30:41.194019+0000","last_clean":"2023-07-13T22:30:41.194019+0000","last_became_active":"2023-07-13T22:30:40.191360+0000","last_became_peered":"2023-07-13T22:30:40.191360+0000","last_unstale":"2023-07-13T22:30:41.194019+0000","last_undegraded":"2023-07-13T22:30:41.194019+0000","last_fullsized":"2023-07-13T22:30:41.194019+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:08:30.556883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203080+0000","last_change":"2023-07-13T22:30:40.194402+0000","last_active":"2023-07-13T22:30:40.203080+0000","last_peered":"2023-07-13T22:30:40.203080+0000","last_clean":"2023-07-13T22:30:40.203080+0000","last_became_active":"2023-07-13T22:30:40.193536+0000","last_became_peered":"2023-07-13T22:30:40.193536+0000","last_unstale":"2023-07-13T22:30:40.203080+0000","last_undegraded":"2023-07-13T22:30:40.203080+0000","last_fullsized":"2023-07-13T22:30:40.203080+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:49.950378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193742+0000","last_change":"2023-07-13T22:30:40.192117+0000","last_active":"2023-07-13T22:30:41.193742+0000","last_peered":"2023-07-13T22:30:41.193742+0000","last_clean":"2023-07-13T22:30:41.193742+0000","last_became_active":"2023-07-13T22:30:40.191836+0000","last_became_peered":"2023-07-13T22:30:40.191836+0000","last_unstale":"2023-07-13T22:30:41.193742+0000","last_undegraded":"2023-07-13T22:30:41.193742+0000","last_fullsized":"2023-07-13T22:30:41.193742+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:12:50.731596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.200588+0000","last_change":"2023-07-13T22:30:40.193248+0000","last_active":"2023-07-13T22:30:40.200588+0000","last_peered":"2023-07-13T22:30:40.200588+0000","last_clean":"2023-07-13T22:30:40.200588+0000","last_became_active":"2023-07-13T22:30:40.192985+0000","last_became_peered":"2023-07-13T22:30:40.192985+0000","last_unstale":"2023-07-13T22:30:40.200588+0000","last_undegraded":"2023-07-13T22:30:40.200588+0000","last_fullsized":"2023-07-13T22:30:40.200588+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:23.379147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191595+0000","last_change":"2023-07-13T22:30:40.193852+0000","last_active":"2023-07-13T22:30:41.191595+0000","last_peered":"2023-07-13T22:30:41.191595+0000","last_clean":"2023-07-13T22:30:41.191595+0000","last_became_active":"2023-07-13T22:30:40.193526+0000","last_became_peered":"2023-07-13T22:30:40.193526+0000","last_unstale":"2023-07-13T22:30:41.191595+0000","last_undegraded":"2023-07-13T22:30:41.191595+0000","last_fullsized":"2023-07-13T22:30:41.191595+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:23:41.404651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.206744+0000","last_change":"2023-07-13T22:30:40.200841+0000","last_active":"2023-07-13T22:30:40.206744+0000","last_peered":"2023-07-13T22:30:40.206744+0000","last_clean":"2023-07-13T22:30:40.206744+0000","last_became_active":"2023-07-13T22:30:40.200573+0000","last_became_peered":"2023-07-13T22:30:40.200573+0000","last_unstale":"2023-07-13T22:30:40.206744+0000","last_undegraded":"2023-07-13T22:30:40.206744+0000","last_fullsized":"2023-07-13T22:30:40.206744+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:49:15.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192181+0000","last_change":"2023-07-13T22:30:40.196485+0000","last_active":"2023-07-13T22:30:41.192181+0000","last_peered":"2023-07-13T22:30:41.192181+0000","last_clean":"2023-07-13T22:30:41.192181+0000","last_became_active":"2023-07-13T22:30:40.196161+0000","last_became_peered":"2023-07-13T22:30:40.196161+0000","last_unstale":"2023-07-13T22:30:41.192181+0000","last_undegraded":"2023-07-13T22:30:41.192181+0000","last_fullsized":"2023-07-13T22:30:41.192181+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:43.861241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191942+0000","last_change":"2023-07-13T22:30:40.184054+0000","last_active":"2023-07-13T22:30:41.191942+0000","last_peered":"2023-07-13T22:30:41.191942+0000","last_clean":"2023-07-13T22:30:41.191942+0000","last_became_active":"2023-07-13T22:30:40.183775+0000","last_became_peered":"2023-07-13T22:30:40.183775+0000","last_unstale":"2023-07-13T22:30:41.191942+0000","last_undegraded":"2023-07-13T22:30:41.191942+0000","last_fullsized":"2023-07-13T22:30:41.191942+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:09.556827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.204225+0000","last_change":"2023-07-13T22:30:40.203051+0000","last_active":"2023-07-13T22:30:40.204225+0000","last_peered":"2023-07-13T22:30:40.204225+0000","last_clean":"2023-07-13T22:30:40.204225+0000","last_became_active":"2023-07-13T22:30:40.202760+0000","last_became_peered":"2023-07-13T22:30:40.202760+0000","last_unstale":"2023-07-13T22:30:40.204225+0000","last_undegraded":"2023-07-13T22:30:40.204225+0000","last_fullsized":"2023-07-13T22:30:40.204225+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:47.380232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193908+0000","last_change":"2023-07-13T22:30:40.193563+0000","last_active":"2023-07-13T22:30:41.193908+0000","last_peered":"2023-07-13T22:30:41.193908+0000","last_clean":"2023-07-13T22:30:41.193908+0000","last_became_active":"2023-07-13T22:30:40.193074+0000","last_became_peered":"2023-07-13T22:30:40.193074+0000","last_unstale":"2023-07-13T22:30:41.193908+0000","last_undegraded":"2023-07-13T22:30:41.193908+0000","last_fullsized":"2023-07-13T22:30:41.193908+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:42.677782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.197808+0000","last_change":"2023-07-13T22:30:40.187168+0000","last_active":"2023-07-13T22:30:40.197808+0000","last_peered":"2023-07-13T22:30:40.197808+0000","last_clean":"2023-07-13T22:30:40.197808+0000","last_became_active":"2023-07-13T22:30:40.186809+0000","last_became_peered":"2023-07-13T22:30:40.186809+0000","last_unstale":"2023-07-13T22:30:40.197808+0000","last_undegraded":"2023-07-13T22:30:40.197808+0000","last_fullsized":"2023-07-13T22:30:40.197808+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:09:00.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191898+0000","last_change":"2023-07-13T22:30:40.203003+0000","last_active":"2023-07-13T22:30:41.191898+0000","last_peered":"2023-07-13T22:30:41.191898+0000","last_clean":"2023-07-13T22:30:41.191898+0000","last_became_active":"2023-07-13T22:30:40.202705+0000","last_became_peered":"2023-07-13T22:30:40.202705+0000","last_unstale":"2023-07-13T22:30:41.191898+0000","last_undegraded":"2023-07-13T22:30:41.191898+0000","last_fullsized":"2023-07-13T22:30:41.191898+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:08:30.556883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203654+0000","last_change":"2023-07-13T22:30:40.195746+0000","last_active":"2023-07-13T22:30:40.203654+0000","last_peered":"2023-07-13T22:30:40.203654+0000","last_clean":"2023-07-13T22:30:40.203654+0000","last_became_active":"2023-07-13T22:30:40.195485+0000","last_became_peered":"2023-07-13T22:30:40.195485+0000","last_unstale":"2023-07-13T22:30:40.203654+0000","last_undegraded":"2023-07-13T22:30:40.203654+0000","last_fullsized":"2023-07-13T22:30:40.203654+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:07.094968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201975+0000","last_change":"2023-07-13T22:30:40.198353+0000","last_active":"2023-07-13T22:30:40.201975+0000","last_peered":"2023-07-13T22:30:40.201975+0000","last_clean":"2023-07-13T22:30:40.201975+0000","last_became_active":"2023-07-13T22:30:40.198086+0000","last_became_peered":"2023-07-13T22:30:40.198086+0000","last_unstale":"2023-07-13T22:30:40.201975+0000","last_undegraded":"2023-07-13T22:30:40.201975+0000","last_fullsized":"2023-07-13T22:30:40.201975+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:02.408817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191683+0000","last_change":"2023-07-13T22:30:40.203350+0000","last_active":"2023-07-13T22:30:41.191683+0000","last_peered":"2023-07-13T22:30:41.191683+0000","last_clean":"2023-07-13T22:30:41.191683+0000","last_became_active":"2023-07-13T22:30:40.194537+0000","last_became_peered":"2023-07-13T22:30:40.194537+0000","last_unstale":"2023-07-13T22:30:41.191683+0000","last_undegraded":"2023-07-13T22:30:41.191683+0000","last_fullsized":"2023-07-13T22:30:41.191683+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:53:56.085379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191924+0000","last_change":"2023-07-13T22:30:40.194243+0000","last_active":"2023-07-13T22:30:41.191924+0000","last_peered":"2023-07-13T22:30:41.191924+0000","last_clean":"2023-07-13T22:30:41.191924+0000","last_became_active":"2023-07-13T22:30:40.193652+0000","last_became_peered":"2023-07-13T22:30:40.193652+0000","last_unstale":"2023-07-13T22:30:41.191924+0000","last_undegraded":"2023-07-13T22:30:41.191924+0000","last_fullsized":"2023-07-13T22:30:41.191924+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:38:32.075031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201780+0000","last_change":"2023-07-13T22:30:40.196195+0000","last_active":"2023-07-13T22:30:40.201780+0000","last_peered":"2023-07-13T22:30:40.201780+0000","last_clean":"2023-07-13T22:30:40.201780+0000","last_became_active":"2023-07-13T22:30:40.195865+0000","last_became_peered":"2023-07-13T22:30:40.195865+0000","last_unstale":"2023-07-13T22:30:40.201780+0000","last_undegraded":"2023-07-13T22:30:40.201780+0000","last_fullsized":"2023-07-13T22:30:40.201780+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:59:08.599187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.206232+0000","last_change":"2023-07-13T22:30:40.199476+0000","last_active":"2023-07-13T22:30:40.206232+0000","last_peered":"2023-07-13T22:30:40.206232+0000","last_clean":"2023-07-13T22:30:40.206232+0000","last_became_active":"2023-07-13T22:30:40.199151+0000","last_became_peered":"2023-07-13T22:30:40.199151+0000","last_unstale":"2023-07-13T22:30:40.206232+0000","last_undegraded":"2023-07-13T22:30:40.206232+0000","last_fullsized":"2023-07-13T22:30:40.206232+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:53:33.705631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201142+0000","last_change":"2023-07-13T22:30:40.193671+0000","last_active":"2023-07-13T22:30:40.201142+0000","last_peered":"2023-07-13T22:30:40.201142+0000","last_clean":"2023-07-13T22:30:40.201142+0000","last_became_active":"2023-07-13T22:30:40.193404+0000","last_became_peered":"2023-07-13T22:30:40.193404+0000","last_unstale":"2023-07-13T22:30:40.201142+0000","last_undegraded":"2023-07-13T22:30:40.201142+0000","last_fullsized":"2023-07-13T22:30:40.201142+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:14:45.866598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193706+0000","last_change":"2023-07-13T22:30:40.195545+0000","last_active":"2023-07-13T22:30:41.193706+0000","last_peered":"2023-07-13T22:30:41.193706+0000","last_clean":"2023-07-13T22:30:41.193706+0000","last_became_active":"2023-07-13T22:30:40.195226+0000","last_became_peered":"2023-07-13T22:30:40.195226+0000","last_unstale":"2023-07-13T22:30:41.193706+0000","last_undegraded":"2023-07-13T22:30:41.193706+0000","last_fullsized":"2023-07-13T22:30:41.193706+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:34:18.820692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.200678+0000","last_change":"2023-07-13T22:30:40.187917+0000","last_active":"2023-07-13T22:30:40.200678+0000","last_peered":"2023-07-13T22:30:40.200678+0000","last_clean":"2023-07-13T22:30:40.200678+0000","last_became_active":"2023-07-13T22:30:40.187567+0000","last_became_peered":"2023-07-13T22:30:40.187567+0000","last_unstale":"2023-07-13T22:30:40.200678+0000","last_undegraded":"2023-07-13T22:30:40.200678+0000","last_fullsized":"2023-07-13T22:30:40.200678+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:25:18.146934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193745+0000","last_change":"2023-07-13T22:30:40.193869+0000","last_active":"2023-07-13T22:30:41.193745+0000","last_peered":"2023-07-13T22:30:41.193745+0000","last_clean":"2023-07-13T22:30:41.193745+0000","last_became_active":"2023-07-13T22:30:40.193608+0000","last_became_peered":"2023-07-13T22:30:40.193608+0000","last_unstale":"2023-07-13T22:30:41.193745+0000","last_undegraded":"2023-07-13T22:30:41.193745+0000","last_fullsized":"2023-07-13T22:30:41.193745+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:27:38.856902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191224+0000","last_change":"2023-07-13T22:30:40.184811+0000","last_active":"2023-07-13T22:30:41.191224+0000","last_peered":"2023-07-13T22:30:41.191224+0000","last_clean":"2023-07-13T22:30:41.191224+0000","last_became_active":"2023-07-13T22:30:40.184291+0000","last_became_peered":"2023-07-13T22:30:40.184291+0000","last_unstale":"2023-07-13T22:30:41.191224+0000","last_undegraded":"2023-07-13T22:30:41.191224+0000","last_fullsized":"2023-07-13T22:30:41.191224+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:29.992933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207415+0000","last_change":"2023-07-13T22:30:40.199932+0000","last_active":"2023-07-13T22:30:40.207415+0000","last_peered":"2023-07-13T22:30:40.207415+0000","last_clean":"2023-07-13T22:30:40.207415+0000","last_became_active":"2023-07-13T22:30:40.199611+0000","last_became_peered":"2023-07-13T22:30:40.199611+0000","last_unstale":"2023-07-13T22:30:40.207415+0000","last_undegraded":"2023-07-13T22:30:40.207415+0000","last_fullsized":"2023-07-13T22:30:40.207415+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:25:21.644922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203231+0000","last_change":"2023-07-13T22:30:40.195808+0000","last_active":"2023-07-13T22:30:40.203231+0000","last_peered":"2023-07-13T22:30:40.203231+0000","last_clean":"2023-07-13T22:30:40.203231+0000","last_became_active":"2023-07-13T22:30:40.193918+0000","last_became_peered":"2023-07-13T22:30:40.193918+0000","last_unstale":"2023-07-13T22:30:40.203231+0000","last_undegraded":"2023-07-13T22:30:40.203231+0000","last_fullsized":"2023-07-13T22:30:40.203231+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:42.677782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207528+0000","last_change":"2023-07-13T22:30:40.205583+0000","last_active":"2023-07-13T22:30:40.207528+0000","last_peered":"2023-07-13T22:30:40.207528+0000","last_clean":"2023-07-13T22:30:40.207528+0000","last_became_active":"2023-07-13T22:30:40.205287+0000","last_became_peered":"2023-07-13T22:30:40.205287+0000","last_unstale":"2023-07-13T22:30:40.207528+0000","last_undegraded":"2023-07-13T22:30:40.207528+0000","last_fullsized":"2023-07-13T22:30:40.207528+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:48:10.825797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210229+0000","last_change":"2023-07-13T22:30:40.206024+0000","last_active":"2023-07-13T22:30:40.210229+0000","last_peered":"2023-07-13T22:30:40.210229+0000","last_clean":"2023-07-13T22:30:40.210229+0000","last_became_active":"2023-07-13T22:30:40.204733+0000","last_became_peered":"2023-07-13T22:30:40.204733+0000","last_unstale":"2023-07-13T22:30:40.210229+0000","last_undegraded":"2023-07-13T22:30:40.210229+0000","last_fullsized":"2023-07-13T22:30:40.210229+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:10:08.683125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207326+0000","last_change":"2023-07-13T22:30:40.199483+0000","last_active":"2023-07-13T22:30:40.207326+0000","last_peered":"2023-07-13T22:30:40.207326+0000","last_clean":"2023-07-13T22:30:40.207326+0000","last_became_active":"2023-07-13T22:30:40.199170+0000","last_became_peered":"2023-07-13T22:30:40.199170+0000","last_unstale":"2023-07-13T22:30:40.207326+0000","last_undegraded":"2023-07-13T22:30:40.207326+0000","last_fullsized":"2023-07-13T22:30:40.207326+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:27:38.856902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203159+0000","last_change":"2023-07-13T22:30:40.195472+0000","last_active":"2023-07-13T22:30:40.203159+0000","last_peered":"2023-07-13T22:30:40.203159+0000","last_clean":"2023-07-13T22:30:40.203159+0000","last_became_active":"2023-07-13T22:30:40.195148+0000","last_became_peered":"2023-07-13T22:30:40.195148+0000","last_unstale":"2023-07-13T22:30:40.203159+0000","last_undegraded":"2023-07-13T22:30:40.203159+0000","last_fullsized":"2023-07-13T22:30:40.203159+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:09.556827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192196+0000","last_change":"2023-07-13T22:30:40.183187+0000","last_active":"2023-07-13T22:30:41.192196+0000","last_peered":"2023-07-13T22:30:41.192196+0000","last_clean":"2023-07-13T22:30:41.192196+0000","last_became_active":"2023-07-13T22:30:40.182835+0000","last_became_peered":"2023-07-13T22:30:40.182835+0000","last_unstale":"2023-07-13T22:30:41.192196+0000","last_undegraded":"2023-07-13T22:30:41.192196+0000","last_fullsized":"2023-07-13T22:30:41.192196+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:26:17.126717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210555+0000","last_change":"2023-07-13T22:30:40.205915+0000","last_active":"2023-07-13T22:30:40.210555+0000","last_peered":"2023-07-13T22:30:40.210555+0000","last_clean":"2023-07-13T22:30:40.210555+0000","last_became_active":"2023-07-13T22:30:40.205478+0000","last_became_peered":"2023-07-13T22:30:40.205478+0000","last_unstale":"2023-07-13T22:30:40.210555+0000","last_undegraded":"2023-07-13T22:30:40.210555+0000","last_fullsized":"2023-07-13T22:30:40.210555+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:16:42.400114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192299+0000","last_change":"2023-07-13T22:30:40.191693+0000","last_active":"2023-07-13T22:30:41.192299+0000","last_peered":"2023-07-13T22:30:41.192299+0000","last_clean":"2023-07-13T22:30:41.192299+0000","last_became_active":"2023-07-13T22:30:40.191362+0000","last_became_peered":"2023-07-13T22:30:40.191362+0000","last_unstale":"2023-07-13T22:30:41.192299+0000","last_undegraded":"2023-07-13T22:30:41.192299+0000","last_fullsized":"2023-07-13T22:30:41.192299+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:11:48.441467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"123'4","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210760+0000","last_change":"2023-07-13T22:30:39.190158+0000","last_active":"2023-07-13T22:30:40.210760+0000","last_peered":"2023-07-13T22:30:40.210760+0000","last_clean":"2023-07-13T22:30:40.210760+0000","last_became_active":"2023-07-13T22:30:39.189844+0000","last_became_peered":"2023-07-13T22:30:39.189844+0000","last_unstale":"2023-07-13T22:30:40.210760+0000","last_undegraded":"2023-07-13T22:30:40.210760+0000","last_fullsized":"2023-07-13T22:30:40.210760+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:08:13.226695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.200604+0000","last_change":"2023-07-13T22:30:40.187823+0000","last_active":"2023-07-13T22:30:40.200604+0000","last_peered":"2023-07-13T22:30:40.200604+0000","last_clean":"2023-07-13T22:30:40.200604+0000","last_became_active":"2023-07-13T22:30:40.187446+0000","last_became_peered":"2023-07-13T22:30:40.187446+0000","last_unstale":"2023-07-13T22:30:40.200604+0000","last_undegraded":"2023-07-13T22:30:40.200604+0000","last_fullsized":"2023-07-13T22:30:40.200604+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:27:54.560877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.205590+0000","last_change":"2023-07-13T22:30:39.188576+0000","last_active":"2023-07-13T22:30:40.205590+0000","last_peered":"2023-07-13T22:30:40.205590+0000","last_clean":"2023-07-13T22:30:40.205590+0000","last_became_active":"2023-07-13T22:30:39.188297+0000","last_became_peered":"2023-07-13T22:30:39.188297+0000","last_unstale":"2023-07-13T22:30:40.205590+0000","last_undegraded":"2023-07-13T22:30:40.205590+0000","last_fullsized":"2023-07-13T22:30:40.205590+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:45.254653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193827+0000","last_change":"2023-07-13T22:30:40.195496+0000","last_active":"2023-07-13T22:30:41.193827+0000","last_peered":"2023-07-13T22:30:41.193827+0000","last_clean":"2023-07-13T22:30:41.193827+0000","last_became_active":"2023-07-13T22:30:40.195225+0000","last_became_peered":"2023-07-13T22:30:40.195225+0000","last_unstale":"2023-07-13T22:30:41.193827+0000","last_undegraded":"2023-07-13T22:30:41.193827+0000","last_fullsized":"2023-07-13T22:30:41.193827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:23:41.404651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210330+0000","last_change":"2023-07-13T22:30:39.194075+0000","last_active":"2023-07-13T22:30:40.210330+0000","last_peered":"2023-07-13T22:30:40.210330+0000","last_clean":"2023-07-13T22:30:40.210330+0000","last_became_active":"2023-07-13T22:30:39.193740+0000","last_became_peered":"2023-07-13T22:30:39.193740+0000","last_unstale":"2023-07-13T22:30:40.210330+0000","last_undegraded":"2023-07-13T22:30:40.210330+0000","last_fullsized":"2023-07-13T22:30:40.210330+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:54:22.096286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207533+0000","last_change":"2023-07-13T22:30:40.200740+0000","last_active":"2023-07-13T22:30:40.207533+0000","last_peered":"2023-07-13T22:30:40.207533+0000","last_clean":"2023-07-13T22:30:40.207533+0000","last_became_active":"2023-07-13T22:30:40.200466+0000","last_became_peered":"2023-07-13T22:30:40.200466+0000","last_unstale":"2023-07-13T22:30:40.207533+0000","last_undegraded":"2023-07-13T22:30:40.207533+0000","last_fullsized":"2023-07-13T22:30:40.207533+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:58:12.446263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.208655+0000","last_change":"2023-07-13T22:30:39.193897+0000","last_active":"2023-07-13T22:30:40.208655+0000","last_peered":"2023-07-13T22:30:40.208655+0000","last_clean":"2023-07-13T22:30:40.208655+0000","last_became_active":"2023-07-13T22:30:39.193323+0000","last_became_peered":"2023-07-13T22:30:39.193323+0000","last_unstale":"2023-07-13T22:30:40.208655+0000","last_undegraded":"2023-07-13T22:30:40.208655+0000","last_fullsized":"2023-07-13T22:30:40.208655+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:44:43.316354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192611+0000","last_change":"2023-07-13T22:30:40.185192+0000","last_active":"2023-07-13T22:30:41.192611+0000","last_peered":"2023-07-13T22:30:41.192611+0000","last_clean":"2023-07-13T22:30:41.192611+0000","last_became_active":"2023-07-13T22:30:40.184917+0000","last_became_peered":"2023-07-13T22:30:40.184917+0000","last_unstale":"2023-07-13T22:30:41.192611+0000","last_undegraded":"2023-07-13T22:30:41.192611+0000","last_fullsized":"2023-07-13T22:30:41.192611+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:50:31.006950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"123'1","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192766+0000","last_change":"2023-07-13T22:30:39.197496+0000","last_active":"2023-07-13T22:30:41.192766+0000","last_peered":"2023-07-13T22:30:41.192766+0000","last_clean":"2023-07-13T22:30:41.192766+0000","last_became_active":"2023-07-13T22:30:39.197134+0000","last_became_peered":"2023-07-13T22:30:39.197134+0000","last_unstale":"2023-07-13T22:30:41.192766+0000","last_undegraded":"2023-07-13T22:30:41.192766+0000","last_fullsized":"2023-07-13T22:30:41.192766+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:22:56.737031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192524+0000","last_change":"2023-07-13T22:30:40.196191+0000","last_active":"2023-07-13T22:30:41.192524+0000","last_peered":"2023-07-13T22:30:41.192524+0000","last_clean":"2023-07-13T22:30:41.192524+0000","last_became_active":"2023-07-13T22:30:40.195892+0000","last_became_peered":"2023-07-13T22:30:40.195892+0000","last_unstale":"2023-07-13T22:30:41.192524+0000","last_undegraded":"2023-07-13T22:30:41.192524+0000","last_fullsized":"2023-07-13T22:30:41.192524+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:15:34.280385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.205338+0000","last_change":"2023-07-13T22:30:39.193923+0000","last_active":"2023-07-13T22:30:40.205338+0000","last_peered":"2023-07-13T22:30:40.205338+0000","last_clean":"2023-07-13T22:30:40.205338+0000","last_became_active":"2023-07-13T22:30:39.193596+0000","last_became_peered":"2023-07-13T22:30:39.193596+0000","last_unstale":"2023-07-13T22:30:40.205338+0000","last_undegraded":"2023-07-13T22:30:40.205338+0000","last_fullsized":"2023-07-13T22:30:40.205338+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:13.388691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201802+0000","last_change":"2023-07-13T22:30:40.197129+0000","last_active":"2023-07-13T22:30:40.201802+0000","last_peered":"2023-07-13T22:30:40.201802+0000","last_clean":"2023-07-13T22:30:40.201802+0000","last_became_active":"2023-07-13T22:30:40.196864+0000","last_became_peered":"2023-07-13T22:30:40.196864+0000","last_unstale":"2023-07-13T22:30:40.201802+0000","last_undegraded":"2023-07-13T22:30:40.201802+0000","last_fullsized":"2023-07-13T22:30:40.201802+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:22:41.562343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192107+0000","last_change":"2023-07-13T22:30:39.183459+0000","last_active":"2023-07-13T22:30:41.192107+0000","last_peered":"2023-07-13T22:30:41.192107+0000","last_clean":"2023-07-13T22:30:41.192107+0000","last_became_active":"2023-07-13T22:30:39.183160+0000","last_became_peered":"2023-07-13T22:30:39.183160+0000","last_unstale":"2023-07-13T22:30:41.192107+0000","last_undegraded":"2023-07-13T22:30:41.192107+0000","last_fullsized":"2023-07-13T22:30:41.192107+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:08:13.226695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201438+0000","last_change":"2023-07-13T22:30:40.195506+0000","last_active":"2023-07-13T22:30:40.201438+0000","last_peered":"2023-07-13T22:30:40.201438+0000","last_clean":"2023-07-13T22:30:40.201438+0000","last_became_active":"2023-07-13T22:30:40.195215+0000","last_became_peered":"2023-07-13T22:30:40.195215+0000","last_unstale":"2023-07-13T22:30:40.201438+0000","last_undegraded":"2023-07-13T22:30:40.201438+0000","last_fullsized":"2023-07-13T22:30:40.201438+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:13:27.143493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190524+0000","last_change":"2023-07-13T22:30:39.191376+0000","last_active":"2023-07-13T22:30:40.190524+0000","last_peered":"2023-07-13T22:30:40.190524+0000","last_clean":"2023-07-13T22:30:40.190524+0000","last_became_active":"2023-07-13T22:30:39.191053+0000","last_became_peered":"2023-07-13T22:30:39.191053+0000","last_unstale":"2023-07-13T22:30:40.190524+0000","last_undegraded":"2023-07-13T22:30:40.190524+0000","last_fullsized":"2023-07-13T22:30:40.190524+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:49.836095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207398+0000","last_change":"2023-07-13T22:30:40.199514+0000","last_active":"2023-07-13T22:30:40.207398+0000","last_peered":"2023-07-13T22:30:40.207398+0000","last_clean":"2023-07-13T22:30:40.207398+0000","last_became_active":"2023-07-13T22:30:40.199223+0000","last_became_peered":"2023-07-13T22:30:40.199223+0000","last_unstale":"2023-07-13T22:30:40.207398+0000","last_undegraded":"2023-07-13T22:30:40.207398+0000","last_fullsized":"2023-07-13T22:30:40.207398+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:38:32.075031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"123'1","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.194197+0000","last_change":"2023-07-13T22:30:39.197288+0000","last_active":"2023-07-13T22:30:41.194197+0000","last_peered":"2023-07-13T22:30:41.194197+0000","last_clean":"2023-07-13T22:30:41.194197+0000","last_became_active":"2023-07-13T22:30:39.196997+0000","last_became_peered":"2023-07-13T22:30:39.196997+0000","last_unstale":"2023-07-13T22:30:41.194197+0000","last_undegraded":"2023-07-13T22:30:41.194197+0000","last_fullsized":"2023-07-13T22:30:41.194197+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:16:31.024429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201443+0000","last_change":"2023-07-13T22:30:40.197590+0000","last_active":"2023-07-13T22:30:40.201443+0000","last_peered":"2023-07-13T22:30:40.201443+0000","last_clean":"2023-07-13T22:30:40.201443+0000","last_became_active":"2023-07-13T22:30:40.197051+0000","last_became_peered":"2023-07-13T22:30:40.197051+0000","last_unstale":"2023-07-13T22:30:40.201443+0000","last_undegraded":"2023-07-13T22:30:40.201443+0000","last_fullsized":"2023-07-13T22:30:40.201443+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:23:39.021951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210530+0000","last_change":"2023-07-13T22:30:39.190825+0000","last_active":"2023-07-13T22:30:40.210530+0000","last_peered":"2023-07-13T22:30:40.210530+0000","last_clean":"2023-07-13T22:30:40.210530+0000","last_became_active":"2023-07-13T22:30:39.190527+0000","last_became_peered":"2023-07-13T22:30:39.190527+0000","last_unstale":"2023-07-13T22:30:40.210530+0000","last_undegraded":"2023-07-13T22:30:40.210530+0000","last_fullsized":"2023-07-13T22:30:40.210530+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:13.530708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.209146+0000","last_change":"2023-07-13T22:30:40.206605+0000","last_active":"2023-07-13T22:30:40.209146+0000","last_peered":"2023-07-13T22:30:40.209146+0000","last_clean":"2023-07-13T22:30:40.209146+0000","last_became_active":"2023-07-13T22:30:40.206235+0000","last_became_peered":"2023-07-13T22:30:40.206235+0000","last_unstale":"2023-07-13T22:30:40.209146+0000","last_undegraded":"2023-07-13T22:30:40.209146+0000","last_fullsized":"2023-07-13T22:30:40.209146+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:57:26.820778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"123'1","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191874+0000","last_change":"2023-07-13T22:30:39.183359+0000","last_active":"2023-07-13T22:30:41.191874+0000","last_peered":"2023-07-13T22:30:41.191874+0000","last_clean":"2023-07-13T22:30:41.191874+0000","last_became_active":"2023-07-13T22:30:39.183064+0000","last_became_peered":"2023-07-13T22:30:39.183064+0000","last_unstale":"2023-07-13T22:30:41.191874+0000","last_undegraded":"2023-07-13T22:30:41.191874+0000","last_fullsized":"2023-07-13T22:30:41.191874+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:48.944306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210182+0000","last_change":"2023-07-13T22:30:40.205760+0000","last_active":"2023-07-13T22:30:40.210182+0000","last_peered":"2023-07-13T22:30:40.210182+0000","last_clean":"2023-07-13T22:30:40.210182+0000","last_became_active":"2023-07-13T22:30:40.205449+0000","last_became_peered":"2023-07-13T22:30:40.205449+0000","last_unstale":"2023-07-13T22:30:40.210182+0000","last_undegraded":"2023-07-13T22:30:40.210182+0000","last_fullsized":"2023-07-13T22:30:40.210182+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:35:32.375163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.183767+0000","last_change":"2023-07-13T22:30:39.188025+0000","last_active":"2023-07-13T22:30:40.183767+0000","last_peered":"2023-07-13T22:30:40.183767+0000","last_clean":"2023-07-13T22:30:40.183767+0000","last_became_active":"2023-07-13T22:30:39.187653+0000","last_became_peered":"2023-07-13T22:30:39.187653+0000","last_unstale":"2023-07-13T22:30:40.183767+0000","last_undegraded":"2023-07-13T22:30:40.183767+0000","last_fullsized":"2023-07-13T22:30:40.183767+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:04:08.557622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201123+0000","last_change":"2023-07-13T22:30:40.192402+0000","last_active":"2023-07-13T22:30:40.201123+0000","last_peered":"2023-07-13T22:30:40.201123+0000","last_clean":"2023-07-13T22:30:40.201123+0000","last_became_active":"2023-07-13T22:30:40.192031+0000","last_became_peered":"2023-07-13T22:30:40.192031+0000","last_unstale":"2023-07-13T22:30:40.201123+0000","last_undegraded":"2023-07-13T22:30:40.201123+0000","last_fullsized":"2023-07-13T22:30:40.201123+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:54:46.563026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"123'2","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192780+0000","last_change":"2023-07-13T22:30:39.195557+0000","last_active":"2023-07-13T22:30:41.192780+0000","last_peered":"2023-07-13T22:30:41.192780+0000","last_clean":"2023-07-13T22:30:41.192780+0000","last_became_active":"2023-07-13T22:30:39.195305+0000","last_became_peered":"2023-07-13T22:30:39.195305+0000","last_unstale":"2023-07-13T22:30:41.192780+0000","last_undegraded":"2023-07-13T22:30:41.192780+0000","last_fullsized":"2023-07-13T22:30:41.192780+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:57:20.830932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203626+0000","last_change":"2023-07-13T22:30:40.195192+0000","last_active":"2023-07-13T22:30:40.203626+0000","last_peered":"2023-07-13T22:30:40.203626+0000","last_clean":"2023-07-13T22:30:40.203626+0000","last_became_active":"2023-07-13T22:30:40.194847+0000","last_became_peered":"2023-07-13T22:30:40.194847+0000","last_unstale":"2023-07-13T22:30:40.203626+0000","last_undegraded":"2023-07-13T22:30:40.203626+0000","last_fullsized":"2023-07-13T22:30:40.203626+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:44:01.374628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.180805+0000","last_change":"2023-07-13T22:30:39.196206+0000","last_active":"2023-07-13T22:30:40.180805+0000","last_peered":"2023-07-13T22:30:40.180805+0000","last_clean":"2023-07-13T22:30:40.180805+0000","last_became_active":"2023-07-13T22:30:39.195810+0000","last_became_peered":"2023-07-13T22:30:39.195810+0000","last_unstale":"2023-07-13T22:30:40.180805+0000","last_undegraded":"2023-07-13T22:30:40.180805+0000","last_fullsized":"2023-07-13T22:30:40.180805+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:09:56.887611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203502+0000","last_change":"2023-07-13T22:30:40.201911+0000","last_active":"2023-07-13T22:30:40.203502+0000","last_peered":"2023-07-13T22:30:40.203502+0000","last_clean":"2023-07-13T22:30:40.203502+0000","last_became_active":"2023-07-13T22:30:40.201569+0000","last_became_peered":"2023-07-13T22:30:40.201569+0000","last_unstale":"2023-07-13T22:30:40.203502+0000","last_undegraded":"2023-07-13T22:30:40.203502+0000","last_fullsized":"2023-07-13T22:30:40.203502+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:54:15.386884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.186610+0000","last_change":"2023-07-13T22:30:39.197270+0000","last_active":"2023-07-13T22:30:40.186610+0000","last_peered":"2023-07-13T22:30:40.186610+0000","last_clean":"2023-07-13T22:30:40.186610+0000","last_became_active":"2023-07-13T22:30:39.196967+0000","last_became_peered":"2023-07-13T22:30:39.196967+0000","last_unstale":"2023-07-13T22:30:40.186610+0000","last_undegraded":"2023-07-13T22:30:40.186610+0000","last_fullsized":"2023-07-13T22:30:40.186610+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:34:32.892453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.194200+0000","last_change":"2023-07-13T22:30:40.202506+0000","last_active":"2023-07-13T22:30:41.194200+0000","last_peered":"2023-07-13T22:30:41.194200+0000","last_clean":"2023-07-13T22:30:41.194200+0000","last_became_active":"2023-07-13T22:30:40.202174+0000","last_became_peered":"2023-07-13T22:30:40.202174+0000","last_unstale":"2023-07-13T22:30:41.194200+0000","last_undegraded":"2023-07-13T22:30:41.194200+0000","last_fullsized":"2023-07-13T22:30:41.194200+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:38:32.075031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191517+0000","last_change":"2023-07-13T22:30:39.197455+0000","last_active":"2023-07-13T22:30:41.191517+0000","last_peered":"2023-07-13T22:30:41.191517+0000","last_clean":"2023-07-13T22:30:41.191517+0000","last_became_active":"2023-07-13T22:30:39.197107+0000","last_became_peered":"2023-07-13T22:30:39.197107+0000","last_unstale":"2023-07-13T22:30:41.191517+0000","last_undegraded":"2023-07-13T22:30:41.191517+0000","last_fullsized":"2023-07-13T22:30:41.191517+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:34:17.814620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.201772+0000","last_change":"2023-07-13T22:30:40.197713+0000","last_active":"2023-07-13T22:30:40.201772+0000","last_peered":"2023-07-13T22:30:40.201772+0000","last_clean":"2023-07-13T22:30:40.201772+0000","last_became_active":"2023-07-13T22:30:40.197442+0000","last_became_peered":"2023-07-13T22:30:40.197442+0000","last_unstale":"2023-07-13T22:30:40.201772+0000","last_undegraded":"2023-07-13T22:30:40.201772+0000","last_fullsized":"2023-07-13T22:30:40.201772+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:25:17.632017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210425+0000","last_change":"2023-07-13T22:30:39.194503+0000","last_active":"2023-07-13T22:30:40.210425+0000","last_peered":"2023-07-13T22:30:40.210425+0000","last_clean":"2023-07-13T22:30:40.210425+0000","last_became_active":"2023-07-13T22:30:39.194115+0000","last_became_peered":"2023-07-13T22:30:39.194115+0000","last_unstale":"2023-07-13T22:30:40.210425+0000","last_undegraded":"2023-07-13T22:30:40.210425+0000","last_fullsized":"2023-07-13T22:30:40.210425+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:03:25.313962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203252+0000","last_change":"2023-07-13T22:30:40.190711+0000","last_active":"2023-07-13T22:30:40.203252+0000","last_peered":"2023-07-13T22:30:40.203252+0000","last_clean":"2023-07-13T22:30:40.203252+0000","last_became_active":"2023-07-13T22:30:40.190199+0000","last_became_peered":"2023-07-13T22:30:40.190199+0000","last_unstale":"2023-07-13T22:30:40.203252+0000","last_undegraded":"2023-07-13T22:30:40.203252+0000","last_fullsized":"2023-07-13T22:30:40.203252+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:53:56.085379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192786+0000","last_change":"2023-07-13T22:30:39.195292+0000","last_active":"2023-07-13T22:30:41.192786+0000","last_peered":"2023-07-13T22:30:41.192786+0000","last_clean":"2023-07-13T22:30:41.192786+0000","last_became_active":"2023-07-13T22:30:39.195017+0000","last_became_peered":"2023-07-13T22:30:39.195017+0000","last_unstale":"2023-07-13T22:30:41.192786+0000","last_undegraded":"2023-07-13T22:30:41.192786+0000","last_fullsized":"2023-07-13T22:30:41.192786+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:51:30.536377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.209164+0000","last_change":"2023-07-13T22:30:40.206943+0000","last_active":"2023-07-13T22:30:40.209164+0000","last_peered":"2023-07-13T22:30:40.209164+0000","last_clean":"2023-07-13T22:30:40.209164+0000","last_became_active":"2023-07-13T22:30:40.206617+0000","last_became_peered":"2023-07-13T22:30:40.206617+0000","last_unstale":"2023-07-13T22:30:40.209164+0000","last_undegraded":"2023-07-13T22:30:40.209164+0000","last_fullsized":"2023-07-13T22:30:40.209164+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:09:35.812927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192580+0000","last_change":"2023-07-13T22:30:39.197773+0000","last_active":"2023-07-13T22:30:41.192580+0000","last_peered":"2023-07-13T22:30:41.192580+0000","last_clean":"2023-07-13T22:30:41.192580+0000","last_became_active":"2023-07-13T22:30:39.197294+0000","last_became_peered":"2023-07-13T22:30:39.197294+0000","last_unstale":"2023-07-13T22:30:41.192580+0000","last_undegraded":"2023-07-13T22:30:41.192580+0000","last_fullsized":"2023-07-13T22:30:41.192580+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:42.684100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210161+0000","last_change":"2023-07-13T22:30:40.205098+0000","last_active":"2023-07-13T22:30:40.210161+0000","last_peered":"2023-07-13T22:30:40.210161+0000","last_clean":"2023-07-13T22:30:40.210161+0000","last_became_active":"2023-07-13T22:30:40.204745+0000","last_became_peered":"2023-07-13T22:30:40.204745+0000","last_unstale":"2023-07-13T22:30:40.210161+0000","last_undegraded":"2023-07-13T22:30:40.210161+0000","last_fullsized":"2023-07-13T22:30:40.210161+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:44:21.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192013+0000","last_change":"2023-07-13T22:30:39.191471+0000","last_active":"2023-07-13T22:30:41.192013+0000","last_peered":"2023-07-13T22:30:41.192013+0000","last_clean":"2023-07-13T22:30:41.192013+0000","last_became_active":"2023-07-13T22:30:39.191158+0000","last_became_peered":"2023-07-13T22:30:39.191158+0000","last_unstale":"2023-07-13T22:30:41.192013+0000","last_undegraded":"2023-07-13T22:30:41.192013+0000","last_fullsized":"2023-07-13T22:30:41.192013+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:20:40.928869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191939+0000","last_change":"2023-07-13T22:30:40.193852+0000","last_active":"2023-07-13T22:30:41.191939+0000","last_peered":"2023-07-13T22:30:41.191939+0000","last_clean":"2023-07-13T22:30:41.191939+0000","last_became_active":"2023-07-13T22:30:40.193521+0000","last_became_peered":"2023-07-13T22:30:40.193521+0000","last_unstale":"2023-07-13T22:30:41.191939+0000","last_undegraded":"2023-07-13T22:30:41.191939+0000","last_fullsized":"2023-07-13T22:30:41.191939+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:16:32.030501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193950+0000","last_change":"2023-07-13T22:30:39.189314+0000","last_active":"2023-07-13T22:30:41.193950+0000","last_peered":"2023-07-13T22:30:41.193950+0000","last_clean":"2023-07-13T22:30:41.193950+0000","last_became_active":"2023-07-13T22:30:39.188963+0000","last_became_peered":"2023-07-13T22:30:39.188963+0000","last_unstale":"2023-07-13T22:30:41.193950+0000","last_undegraded":"2023-07-13T22:30:41.193950+0000","last_fullsized":"2023-07-13T22:30:41.193950+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:20:08.550755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191728+0000","last_change":"2023-07-13T22:30:40.195058+0000","last_active":"2023-07-13T22:30:41.191728+0000","last_peered":"2023-07-13T22:30:41.191728+0000","last_clean":"2023-07-13T22:30:41.191728+0000","last_became_active":"2023-07-13T22:30:40.194780+0000","last_became_peered":"2023-07-13T22:30:40.194780+0000","last_unstale":"2023-07-13T22:30:41.191728+0000","last_undegraded":"2023-07-13T22:30:41.191728+0000","last_fullsized":"2023-07-13T22:30:41.191728+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:55:42.677782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190560+0000","last_change":"2023-07-13T22:30:39.196661+0000","last_active":"2023-07-13T22:30:40.190560+0000","last_peered":"2023-07-13T22:30:40.190560+0000","last_clean":"2023-07-13T22:30:40.190560+0000","last_became_active":"2023-07-13T22:30:39.196324+0000","last_became_peered":"2023-07-13T22:30:39.196324+0000","last_unstale":"2023-07-13T22:30:40.190560+0000","last_undegraded":"2023-07-13T22:30:40.190560+0000","last_fullsized":"2023-07-13T22:30:40.190560+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:56.876139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191933+0000","last_change":"2023-07-13T22:30:40.201339+0000","last_active":"2023-07-13T22:30:41.191933+0000","last_peered":"2023-07-13T22:30:41.191933+0000","last_clean":"2023-07-13T22:30:41.191933+0000","last_became_active":"2023-07-13T22:30:40.200985+0000","last_became_peered":"2023-07-13T22:30:40.200985+0000","last_unstale":"2023-07-13T22:30:41.191933+0000","last_undegraded":"2023-07-13T22:30:41.191933+0000","last_fullsized":"2023-07-13T22:30:41.191933+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:27:38.856902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192234+0000","last_change":"2023-07-13T22:30:39.187771+0000","last_active":"2023-07-13T22:30:41.192234+0000","last_peered":"2023-07-13T22:30:41.192234+0000","last_clean":"2023-07-13T22:30:41.192234+0000","last_became_active":"2023-07-13T22:30:39.187450+0000","last_became_peered":"2023-07-13T22:30:39.187450+0000","last_unstale":"2023-07-13T22:30:41.192234+0000","last_undegraded":"2023-07-13T22:30:41.192234+0000","last_fullsized":"2023-07-13T22:30:41.192234+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:23:46.374160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207564+0000","last_change":"2023-07-13T22:30:40.200705+0000","last_active":"2023-07-13T22:30:40.207564+0000","last_peered":"2023-07-13T22:30:40.207564+0000","last_clean":"2023-07-13T22:30:40.207564+0000","last_became_active":"2023-07-13T22:30:40.200434+0000","last_became_peered":"2023-07-13T22:30:40.200434+0000","last_unstale":"2023-07-13T22:30:40.207564+0000","last_undegraded":"2023-07-13T22:30:40.207564+0000","last_fullsized":"2023-07-13T22:30:40.207564+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:24:23.477257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.182689+0000","last_change":"2023-07-13T22:30:39.195419+0000","last_active":"2023-07-13T22:30:40.182689+0000","last_peered":"2023-07-13T22:30:40.182689+0000","last_clean":"2023-07-13T22:30:40.182689+0000","last_became_active":"2023-07-13T22:30:39.195077+0000","last_became_peered":"2023-07-13T22:30:39.195077+0000","last_unstale":"2023-07-13T22:30:40.182689+0000","last_undegraded":"2023-07-13T22:30:40.182689+0000","last_fullsized":"2023-07-13T22:30:40.182689+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:22:59.279034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203596+0000","last_change":"2023-07-13T22:30:40.197129+0000","last_active":"2023-07-13T22:30:40.203596+0000","last_peered":"2023-07-13T22:30:40.203596+0000","last_clean":"2023-07-13T22:30:40.203596+0000","last_became_active":"2023-07-13T22:30:40.196831+0000","last_became_peered":"2023-07-13T22:30:40.196831+0000","last_unstale":"2023-07-13T22:30:40.203596+0000","last_undegraded":"2023-07-13T22:30:40.203596+0000","last_fullsized":"2023-07-13T22:30:40.203596+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:34:08.171713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193923+0000","last_change":"2023-07-13T22:30:39.187439+0000","last_active":"2023-07-13T22:30:41.193923+0000","last_peered":"2023-07-13T22:30:41.193923+0000","last_clean":"2023-07-13T22:30:41.193923+0000","last_became_active":"2023-07-13T22:30:39.187158+0000","last_became_peered":"2023-07-13T22:30:39.187158+0000","last_unstale":"2023-07-13T22:30:41.193923+0000","last_undegraded":"2023-07-13T22:30:41.193923+0000","last_fullsized":"2023-07-13T22:30:41.193923+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:48.944306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":203,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.192616+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:30:41.192616+0000","last_peered":"2023-07-13T22:30:41.192616+0000","last_clean":"2023-07-13T22:30:41.192616+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:30:41.192616+0000","last_undegraded":"2023-07-13T22:30:41.192616+0000","last_fullsized":"2023-07-13T22:30:41.192616+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:37:59.427831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191976+0000","last_change":"2023-07-13T22:30:40.197356+0000","last_active":"2023-07-13T22:30:41.191976+0000","last_peered":"2023-07-13T22:30:41.191976+0000","last_clean":"2023-07-13T22:30:41.191976+0000","last_became_active":"2023-07-13T22:30:40.196964+0000","last_became_peered":"2023-07-13T22:30:40.196964+0000","last_unstale":"2023-07-13T22:30:41.191976+0000","last_undegraded":"2023-07-13T22:30:41.191976+0000","last_fullsized":"2023-07-13T22:30:41.191976+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:52:50.842167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.180016+0000","last_change":"2023-07-13T22:30:39.198859+0000","last_active":"2023-07-13T22:30:40.180016+0000","last_peered":"2023-07-13T22:30:40.180016+0000","last_clean":"2023-07-13T22:30:40.180016+0000","last_became_active":"2023-07-13T22:30:39.198316+0000","last_became_peered":"2023-07-13T22:30:39.198316+0000","last_unstale":"2023-07-13T22:30:40.180016+0000","last_undegraded":"2023-07-13T22:30:40.180016+0000","last_fullsized":"2023-07-13T22:30:40.180016+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:48:49.331181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":195,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.183859+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:30:40.183859+0000","last_peered":"2023-07-13T22:30:40.183859+0000","last_clean":"2023-07-13T22:30:40.183859+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:30:40.183859+0000","last_undegraded":"2023-07-13T22:30:40.183859+0000","last_fullsized":"2023-07-13T22:30:40.183859+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:58:48.921485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207628+0000","last_change":"2023-07-13T22:30:40.199452+0000","last_active":"2023-07-13T22:30:40.207628+0000","last_peered":"2023-07-13T22:30:40.207628+0000","last_clean":"2023-07-13T22:30:40.207628+0000","last_became_active":"2023-07-13T22:30:40.199165+0000","last_became_peered":"2023-07-13T22:30:40.199165+0000","last_unstale":"2023-07-13T22:30:40.207628+0000","last_undegraded":"2023-07-13T22:30:40.207628+0000","last_fullsized":"2023-07-13T22:30:40.207628+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:33:18.133451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190533+0000","last_change":"2023-07-13T22:30:39.193116+0000","last_active":"2023-07-13T22:30:40.190533+0000","last_peered":"2023-07-13T22:30:40.190533+0000","last_clean":"2023-07-13T22:30:40.190533+0000","last_became_active":"2023-07-13T22:30:39.192334+0000","last_became_peered":"2023-07-13T22:30:39.192334+0000","last_unstale":"2023-07-13T22:30:40.190533+0000","last_undegraded":"2023-07-13T22:30:40.190533+0000","last_fullsized":"2023-07-13T22:30:40.190533+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:20:40.928869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":211,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191883+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:30:41.191883+0000","last_peered":"2023-07-13T22:30:41.191883+0000","last_clean":"2023-07-13T22:30:41.191883+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:30:41.191883+0000","last_undegraded":"2023-07-13T22:30:41.191883+0000","last_fullsized":"2023-07-13T22:30:41.191883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:15:29.188156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":339,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.187784+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:30:40.187784+0000","last_peered":"2023-07-13T22:30:40.187784+0000","last_clean":"2023-07-13T22:30:40.187784+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:30:40.187784+0000","last_undegraded":"2023-07-13T22:30:40.187784+0000","last_fullsized":"2023-07-13T22:30:40.187784+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:17.361367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.192523+0000","last_change":"2023-07-13T22:30:40.186112+0000","last_active":"2023-07-13T22:30:40.192523+0000","last_peered":"2023-07-13T22:30:40.192523+0000","last_clean":"2023-07-13T22:30:40.192523+0000","last_became_active":"2023-07-13T22:30:40.185826+0000","last_became_peered":"2023-07-13T22:30:40.185826+0000","last_unstale":"2023-07-13T22:30:40.192523+0000","last_undegraded":"2023-07-13T22:30:40.192523+0000","last_fullsized":"2023-07-13T22:30:40.192523+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:08:58.753068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"123'8","reported_seq":21,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.191599+0000","last_change":"2023-07-13T22:30:39.188971+0000","last_active":"2023-07-13T22:30:41.191599+0000","last_peered":"2023-07-13T22:30:41.191599+0000","last_clean":"2023-07-13T22:30:41.191599+0000","last_became_active":"2023-07-13T22:30:39.188674+0000","last_became_peered":"2023-07-13T22:30:39.188674+0000","last_unstale":"2023-07-13T22:30:41.191599+0000","last_undegraded":"2023-07-13T22:30:41.191599+0000","last_fullsized":"2023-07-13T22:30:41.191599+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:12:49.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":191,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190375+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:30:40.190375+0000","last_peered":"2023-07-13T22:30:40.190375+0000","last_clean":"2023-07-13T22:30:40.190375+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:30:40.190375+0000","last_undegraded":"2023-07-13T22:30:40.190375+0000","last_fullsized":"2023-07-13T22:30:40.190375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:06:39.722360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.203101+0000","last_change":"2023-07-13T22:30:40.194585+0000","last_active":"2023-07-13T22:30:40.203101+0000","last_peered":"2023-07-13T22:30:40.203101+0000","last_clean":"2023-07-13T22:30:40.203101+0000","last_became_active":"2023-07-13T22:30:40.193900+0000","last_became_peered":"2023-07-13T22:30:40.193900+0000","last_unstale":"2023-07-13T22:30:40.203101+0000","last_undegraded":"2023-07-13T22:30:40.203101+0000","last_fullsized":"2023-07-13T22:30:40.203101+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:08:30.556883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.190633+0000","last_change":"2023-07-13T22:30:39.196379+0000","last_active":"2023-07-13T22:30:40.190633+0000","last_peered":"2023-07-13T22:30:40.190633+0000","last_clean":"2023-07-13T22:30:40.190633+0000","last_became_active":"2023-07-13T22:30:39.195889+0000","last_became_peered":"2023-07-13T22:30:39.195889+0000","last_unstale":"2023-07-13T22:30:40.190633+0000","last_undegraded":"2023-07-13T22:30:40.190633+0000","last_fullsized":"2023-07-13T22:30:40.190633+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:45:27.186239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":191,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.187918+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:30:40.187918+0000","last_peered":"2023-07-13T22:30:40.187918+0000","last_clean":"2023-07-13T22:30:40.187918+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:30:40.187918+0000","last_undegraded":"2023-07-13T22:30:40.187918+0000","last_fullsized":"2023-07-13T22:30:40.187918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:54:07.240537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.210211+0000","last_change":"2023-07-13T22:30:40.205807+0000","last_active":"2023-07-13T22:30:40.210211+0000","last_peered":"2023-07-13T22:30:40.210211+0000","last_clean":"2023-07-13T22:30:40.210211+0000","last_became_active":"2023-07-13T22:30:40.204333+0000","last_became_peered":"2023-07-13T22:30:40.204333+0000","last_unstale":"2023-07-13T22:30:40.210211+0000","last_undegraded":"2023-07-13T22:30:40.210211+0000","last_fullsized":"2023-07-13T22:30:40.210211+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:14.536780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.211480+0000","last_change":"2023-07-13T22:30:39.197229+0000","last_active":"2023-07-13T22:30:40.211480+0000","last_peered":"2023-07-13T22:30:40.211480+0000","last_clean":"2023-07-13T22:30:40.211480+0000","last_became_active":"2023-07-13T22:30:39.196939+0000","last_became_peered":"2023-07-13T22:30:39.196939+0000","last_unstale":"2023-07-13T22:30:40.211480+0000","last_undegraded":"2023-07-13T22:30:40.211480+0000","last_fullsized":"2023-07-13T22:30:40.211480+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:25:26.805497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":221,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-07-13T22:30:41.193803+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:30:41.193803+0000","last_peered":"2023-07-13T22:30:41.193803+0000","last_clean":"2023-07-13T22:30:41.193803+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:30:41.193803+0000","last_undegraded":"2023-07-13T22:30:41.193803+0000","last_fullsized":"2023-07-13T22:30:41.193803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:44:03.628613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.207350+0000","last_change":"2023-07-13T22:30:40.200725+0000","last_active":"2023-07-13T22:30:40.207350+0000","last_peered":"2023-07-13T22:30:40.207350+0000","last_clean":"2023-07-13T22:30:40.207350+0000","last_became_active":"2023-07-13T22:30:40.200471+0000","last_became_peered":"2023-07-13T22:30:40.200471+0000","last_unstale":"2023-07-13T22:30:40.207350+0000","last_undegraded":"2023-07-13T22:30:40.207350+0000","last_fullsized":"2023-07-13T22:30:40.207350+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:16:32.030501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"123'6","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.213322+0000","last_change":"2023-07-13T22:30:39.194137+0000","last_active":"2023-07-13T22:30:40.213322+0000","last_peered":"2023-07-13T22:30:40.213322+0000","last_clean":"2023-07-13T22:30:40.213322+0000","last_became_active":"2023-07-13T22:30:39.193839+0000","last_became_peered":"2023-07-13T22:30:39.193839+0000","last_unstale":"2023-07-13T22:30:40.213322+0000","last_undegraded":"2023-07-13T22:30:40.213322+0000","last_fullsized":"2023-07-13T22:30:40.213322+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:34:40.483086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":189,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.192243+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:30:40.192243+0000","last_peered":"2023-07-13T22:30:40.192243+0000","last_clean":"2023-07-13T22:30:40.192243+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:30:40.192243+0000","last_undegraded":"2023-07-13T22:30:40.192243+0000","last_fullsized":"2023-07-13T22:30:40.192243+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:16.226359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.199724+0000","last_change":"2023-07-13T22:30:40.189517+0000","last_active":"2023-07-13T22:30:40.199724+0000","last_peered":"2023-07-13T22:30:40.199724+0000","last_clean":"2023-07-13T22:30:40.199724+0000","last_became_active":"2023-07-13T22:30:40.189233+0000","last_became_peered":"2023-07-13T22:30:40.189233+0000","last_unstale":"2023-07-13T22:30:40.199724+0000","last_undegraded":"2023-07-13T22:30:40.199724+0000","last_fullsized":"2023-07-13T22:30:40.199724+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:39.162859+0000","last_clean_scrub_stamp":"2023-07-13T22:30:39.162859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:12:50.731596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"123'3","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.209899+0000","last_change":"2023-07-13T22:30:39.192950+0000","last_active":"2023-07-13T22:30:40.209899+0000","last_peered":"2023-07-13T22:30:40.209899+0000","last_clean":"2023-07-13T22:30:40.209899+0000","last_became_active":"2023-07-13T22:30:39.192619+0000","last_became_peered":"2023-07-13T22:30:39.192619+0000","last_unstale":"2023-07-13T22:30:40.209899+0000","last_undegraded":"2023-07-13T22:30:40.209899+0000","last_fullsized":"2023-07-13T22:30:40.209899+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:30:38.156787+0000","last_clean_scrub_stamp":"2023-07-13T22:30:38.156787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:40:16.319138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":189,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-13T22:30:40.191173+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:30:40.191173+0000","last_peered":"2023-07-13T22:30:40.191173+0000","last_clean":"2023-07-13T22:30:40.191173+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:30:40.191173+0000","last_undegraded":"2023-07-13T22:30:40.191173+0000","last_fullsized":"2023-07-13T22:30:40.191173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:13:54.923466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":2146304,"data_stored":1294981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.67600000000000005,"15min":0.63800000000000001},"min":{"1min":0.436,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78700000000000003,"5min":1.0780000000000001,"15min":1.115},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.83699999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.75}]},{"osd":1,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.64200000000000002,"15min":0.67200000000000004},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.29399999999999998},"max":{"1min":0.76200000000000001,"5min":0.94299999999999995,"15min":2.2610000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.34300000000000003},"max":{"1min":0.93300000000000005,"5min":1.0309999999999999,"15min":1.0609999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71499999999999997,"15min":0.76500000000000001},"min":{"1min":0.51600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90400000000000003,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.75700000000000001},"min":{"1min":0.54600000000000004,"5min":0.42999999999999999,"15min":0.36499999999999999},"max":{"1min":0.80900000000000005,"5min":1.107,"15min":3.4710000000000001},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.68799999999999994},"min":{"1min":0.496,"5min":0.45500000000000002,"15min":0.34599999999999997},"max":{"1min":0.86799999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.75900000000000001,"15min":0.75600000000000001},"min":{"1min":0.60199999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90300000000000002,"5min":1.024,"15min":1.1399999999999999},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.622,"15min":0.63800000000000001},"min":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.34899999999999998},"max":{"1min":0.86499999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.46700000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":1.1850000000000001},"last":0.622}]},{"osd":5,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63800000000000001,"15min":0.65600000000000003},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85099999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0089999999999999,"5min":1.0149999999999999,"15min":1.2110000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.69099999999999995,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.90500000000000003,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.46999999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.042},"last":0.57199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1544192,"data_stored":703631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.629},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.28799999999999998},"max":{"1min":0.88100000000000001,"5min":1.089,"15min":1.089},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.68500000000000005,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.33900000000000002},"max":{"1min":0.83599999999999997,"5min":1.075,"15min":1.075},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73199999999999998,"15min":0.70199999999999996},"min":{"1min":0.58099999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82399999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.48899999999999999,"5min":0.38500000000000001,"15min":0.379},"max":{"1min":0.84099999999999997,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.66900000000000004},"min":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":1.042,"15min":1.042},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69899999999999995,"15min":0.66500000000000004},"min":{"1min":0.48599999999999999,"5min":0.435,"15min":0.37},"max":{"1min":0.88500000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73799999999999999,"15min":0.72599999999999998},"min":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.44},"max":{"1min":0.86899999999999999,"5min":1.107,"15min":1.107},"last":0.83599999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60499999999999998,"15min":0.64500000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.65600000000000003,"15min":0.68200000000000005},"min":{"1min":0.40400000000000003,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":0.90900000000000003,"5min":0.997,"15min":0.998},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.49199999999999999,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.879,"5min":0.90200000000000002,"15min":0.95999999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73699999999999999,"15min":0.75700000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70899999999999996,"15min":0.73999999999999999},"min":{"1min":0.44,"5min":0.439,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":0.96099999999999997,"15min":1.1890000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.69899999999999995,"15min":0.73499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.39000000000000001},"max":{"1min":0.94299999999999995,"5min":0.98099999999999998,"15min":1.111},"last":0.75900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":2134016,"data_stored":1293397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.438,"15min":0.47599999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.249},"max":{"1min":0.60199999999999998,"5min":0.70199999999999996,"15min":1.952},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.63300000000000001,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.53500000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.45900000000000002,"15min":0.51200000000000001},"min":{"1min":0.30299999999999999,"5min":0.29899999999999999,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":0.71299999999999997,"15min":2.1200000000000001},"last":0.495},{"interface":"front","average":{"1min":0.497,"5min":0.53700000000000003,"15min":0.57799999999999996},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.67300000000000004,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.60499999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.83899999999999997,"15min":2.008},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82299999999999995,"5min":1.054,"15min":1.054},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.745,"15min":0.78400000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96999999999999997,"5min":0.98199999999999998,"15min":2.028},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.59499999999999997,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.93899999999999995,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.78800000000000003,"15min":0.82599999999999996},"min":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":1.0720000000000001,"5min":1.095,"15min":2.0899999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.60899999999999999,"5min":0.55100000000000005,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":1.073,"15min":1.2230000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.83599999999999997},"min":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0289999999999999,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:30:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.50700000000000001},"max":{"1min":1.04,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.56499999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.046,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.67200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1540096,"data_stored":702029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45900000000000002,"15min":0.51400000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61599999999999999,"5min":0.84999999999999998,"15min":11.791},"last":0.496},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.32200000000000001,"5min":0.313,"15min":0.29299999999999998},"max":{"1min":0.58199999999999996,"5min":0.99199999999999999,"15min":4.657},"last":0.443}]},{"osd":2,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80500000000000005,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50700000000000001,"15min":0.872},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96199999999999997,"5min":1.276,"15min":13.398999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.502,"15min":0.54800000000000004},"min":{"1min":0.376,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.59699999999999998,"5min":0.75800000000000001,"15min":1.8089999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.55800000000000005,"15min":0.56499999999999995},"min":{"1min":0.40400000000000003,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.70699999999999996,"5min":1.095,"15min":4.4530000000000003},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.748,"15min":1.1160000000000001},"min":{"1min":0.54700000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75,"15min":0.79100000000000004},"min":{"1min":0.59499999999999997,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":1.0429999999999999,"5min":1.1220000000000001,"15min":1.776},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79000000000000004,"15min":1.139},"min":{"1min":0.504,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.113,"15min":13.349},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.82699999999999996,"15min":0.86199999999999999},"min":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96099999999999997,"5min":1.302,"15min":4.7649999999999997},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.79700000000000004,"15min":0.82499999999999996},"min":{"1min":0.55500000000000005,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.0149999999999999,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77200000000000002,"15min":1.109},"min":{"1min":0.47199999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97799999999999998,"5min":1.2330000000000001,"15min":11.679},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jul 13 22:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":1.145},"min":{"1min":0.60899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.032,"5min":1.1759999999999999,"15min":13.298},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":0.876},"min":{"1min":0.58099999999999996,"5min":0.51800000000000002,"15min":0.50900000000000001},"max":{"1min":1.091,"5min":1.151,"15min":11.456},"last":0.63200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1540096,"data_stored":702589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44400000000000001,"15min":0.50900000000000001},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.249},"max":{"1min":1.605,"5min":1.605,"15min":2.282},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51200000000000001,"15min":0.47899999999999998},"min":{"1min":0.35699999999999998,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":0.66100000000000003,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47799999999999998,"15min":0.47199999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.26200000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.57699999999999996},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":2.319},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.311},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.627},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38700000000000001},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":2.351},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78200000000000003,"15min":0.82099999999999995},"min":{"1min":0.57099999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":2.238},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78000000000000003,"15min":0.77700000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.46899999999999997},"max":{"1min":1.0449999999999999,"5min":1.274,"15min":1.518},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.505},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.81399999999999995,"15min":0.84099999999999997},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":2.1909999999999998},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98499999999999999,"5min":1.0760000000000001,"15min":1.488},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.80100000000000005,"15min":0.78800000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.48199999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.747}]},{"osd":7,"last update":"Thu Jul 13 22:29:59 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.81000000000000005,"15min":0.79200000000000004},"min":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":1.0980000000000001,"5min":1.29,"15min":1.29},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.69899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1552384,"data_stored":704593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.29199999999999998,"5min":0.26400000000000001,"15min":0.248},"max":{"1min":0.56499999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.45500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.27200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73899999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.314,"5min":0.313,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.499},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.68999999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.44}]},{"osd":2,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.629}]},{"osd":4,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.51200000000000001,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":0.90900000000000003,"5min":1.149,"15min":1.2250000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.52900000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91900000000000004,"5min":1.113,"15min":1.1539999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":0.96899999999999997,"5min":1.1919999999999999,"15min":1.411},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53700000000000003,"5min":0.53400000000000003,"15min":0.48599999999999999},"max":{"1min":0.92300000000000004,"5min":1.175,"15min":1.5469999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.51800000000000002},"max":{"1min":0.97299999999999998,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.77100000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"max":{"1min":0.99199999999999999,"5min":1.02,"15min":1.1240000000000001},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.58899999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78600000000000003,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.47199999999999998},"max":{"1min":1.0249999999999999,"5min":1.153,"15min":1.1950000000000001},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1536000,"data_stored":702959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65500000000000003,"15min":0.61899999999999999},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.311},"max":{"1min":0.85399999999999998,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67500000000000004,"15min":0.64400000000000002},"min":{"1min":0.54600000000000004,"5min":0.39400000000000002,"15min":0.36199999999999999},"max":{"1min":0.77900000000000003,"5min":0.997,"15min":0.997},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.35099999999999998},"max":{"1min":0.90700000000000003,"5min":1.0489999999999999,"15min":1.298},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73599999999999999,"15min":0.69799999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.438},"max":{"1min":0.89600000000000002,"5min":1.0049999999999999,"15min":1.3109999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.371},"max":{"1min":0.88200000000000001,"5min":1.0629999999999999,"15min":1.1200000000000001},"last":0.628},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73299999999999998,"15min":0.71499999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86699999999999999,"5min":1.089,"15min":1.1839999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69999999999999996,"15min":0.89400000000000002},"min":{"1min":0.44700000000000001,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":0.93700000000000006,"5min":1.234,"15min":10.657999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.88600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85199999999999998,"5min":1.0600000000000001,"15min":8.75},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.65200000000000002,"15min":0.67900000000000005},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":0.89600000000000002,"5min":1.042,"15min":1.042},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.42199999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82999999999999996,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":1.0189999999999999,"15min":1.173},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.48499999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93300000000000005,"5min":1.0640000000000001,"15min":1.099},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.46200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.87,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73999999999999999,"15min":0.76300000000000001},"min":{"1min":0.45100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.94599999999999995,"5min":1.1819999999999999,"15min":1.212},"last":0.80400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574970,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1540096,"data_stored":702981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.66400000000000003,"15min":0.624},"min":{"1min":0.41499999999999998,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.76500000000000001,"5min":1.042,"15min":1.042},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.67600000000000005,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83799999999999997,"5min":0.998,"15min":0.998},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.66600000000000004},"min":{"1min":0.46899999999999997,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.84799999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.68000000000000005},"min":{"1min":0.505,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82699999999999996,"5min":0.94499999999999995,"15min":8.9930000000000003},"last":0.628}]},{"osd":2,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68300000000000005,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.35899999999999999},"max":{"1min":0.879,"5min":0.88300000000000001,"15min":0.96199999999999997},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71299999999999997,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.69499999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96599999999999997},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72499999999999998,"15min":0.69199999999999995},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67700000000000005,"15min":0.64300000000000002},"min":{"1min":0.52200000000000002,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.51600000000000001,"5min":0.438,"15min":0.40000000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.496,"15min":0.42299999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.88},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69099999999999995,"15min":0.67400000000000004},"min":{"1min":0.44800000000000001,"5min":0.40600000000000003,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.51000000000000001,"5min":0.47799999999999998,"15min":0.41199999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.76800000000000002},"min":{"1min":0.64900000000000002,"5min":0.54100000000000004,"15min":0.503},"max":{"1min":1.034,"5min":1.034,"15min":1.0669999999999999},"last":0.71299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:30:45.110 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:30:45.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-13T22:30:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.374+0000 7f518ffff700 1 Processor -- start 2023-07-13T22:30:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.375+0000 7f518ffff700 1 -- start start 2023-07-13T22:30:45.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.376+0000 7f518ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5188070350 0x7f518806cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.376+0000 7f518ffff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5188070720 0x7f518815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.376+0000 7f518ffff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518815d2c0 con 0x7f518815c980 2023-07-13T22:30:45.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.376+0000 7f518ffff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518815cde0 con 0x7f5188070720 2023-07-13T22:30:45.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.376+0000 7f518ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518815cfe0 con 0x7f5188070350 2023-07-13T22:30:45.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.376+0000 7f518dd9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5188070350 0x7f518806cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.376+0000 7f518d59a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5188070720 0x7f518815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518dd9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5188070350 0x7f518806cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:37060/0 (socket says 172.21.15.60:37060) 2023-07-13T22:30:45.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518d59a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5188070720 0x7f518815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:57538/0 (socket says 172.21.15.60:57538) 2023-07-13T22:30:45.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518dd9b700 1 -- 172.21.15.60:0/3881541036 learned_addr learned my addr 172.21.15.60:0/3881541036 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:45.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518e59c700 1 --1- 172.21.15.60:0/3881541036 >> v1:172.21.15.105:6789/0 conn(0x7f518815c980 0x7f51880cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:46182/0 (socket says 172.21.15.60:46182) 2023-07-13T22:30:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518dd9b700 1 -- 172.21.15.60:0/3881541036 >> v2:172.21.15.105:3300/0 conn(0x7f5188070720 msgr2=0x7f518815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518dd9b700 1 --2- 172.21.15.60:0/3881541036 >> v2:172.21.15.105:3300/0 conn(0x7f5188070720 0x7f518815c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518dd9b700 1 -- 172.21.15.60:0/3881541036 >> v1:172.21.15.105:6789/0 conn(0x7f518815c980 legacy=0x7f51880cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518dd9b700 1 -- 172.21.15.60:0/3881541036 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518806c4c0 con 0x7f5188070350 2023-07-13T22:30:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518dd9b700 1 --2- 172.21.15.60:0/3881541036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5188070350 0x7f518806cfa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f51840098f0 tx=0x7f5184009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef60ce803baaf8ad server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.377+0000 7f518cd99700 1 -- 172.21.15.60:0/3881541036 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184011830 con 0x7f5188070350 2023-07-13T22:30:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518cd99700 1 -- 172.21.15.60:0/3881541036 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5184011990 con 0x7f5188070350 2023-07-13T22:30:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518cd99700 1 -- 172.21.15.60:0/3881541036 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184011d20 con 0x7f5188070350 2023-07-13T22:30:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518ffff700 1 -- 172.21.15.60:0/3881541036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5188070350 msgr2=0x7f518806cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518ffff700 1 --2- 172.21.15.60:0/3881541036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5188070350 0x7f518806cfa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f51840098f0 tx=0x7f5184009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518ffff700 1 -- 172.21.15.60:0/3881541036 shutdown_connections 2023-07-13T22:30:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518ffff700 1 --2- 172.21.15.60:0/3881541036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5188070350 0x7f518806cfa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518ffff700 1 --2- 172.21.15.60:0/3881541036 >> v2:172.21.15.105:3300/0 conn(0x7f5188070720 0x7f518815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518ffff700 1 -- 172.21.15.60:0/3881541036 >> 172.21.15.60:0/3881541036 conn(0x7f51880cf3b0 msgr2=0x7f51880cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518ffff700 1 -- 172.21.15.60:0/3881541036 shutdown_connections 2023-07-13T22:30:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.378+0000 7f518ffff700 1 -- 172.21.15.60:0/3881541036 wait complete. 2023-07-13T22:30:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.380+0000 7f518ffff700 1 Processor -- start 2023-07-13T22:30:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.380+0000 7f518ffff700 1 -- start start 2023-07-13T22:30:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5188070350 0x7f518813e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5188070720 0x7f518813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f518815c980 0x7f518815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518813ed00 con 0x7f5188070350 2023-07-13T22:30:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518ffff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518815af30 con 0x7f5188070720 2023-07-13T22:30:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518815a320 con 0x7f518815c980 2023-07-13T22:30:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5188070350 0x7f518813e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5188070350 0x7f518813e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50450/0 (socket says 172.21.15.60:50450) 2023-07-13T22:30:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518dd9b700 1 -- 172.21.15.60:0/1784098959 learned_addr learned my addr 172.21.15.60:0/1784098959 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518e59c700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f518815c980 0x7f518815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.381+0000 7f518d59a700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5188070720 0x7f518813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f518dd9b700 1 -- 172.21.15.60:0/1784098959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f518815c980 msgr2=0x7f518815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f518dd9b700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f518815c980 0x7f518815b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f518dd9b700 1 -- 172.21.15.60:0/1784098959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5188070720 msgr2=0x7f518813ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f518dd9b700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5188070720 0x7f518813ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f518dd9b700 1 -- 172.21.15.60:0/1784098959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184009550 con 0x7f5188070350 2023-07-13T22:30:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f518dd9b700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5188070350 0x7f518813e7c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f5184000c00 tx=0x7f5184004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f517e7fc700 1 -- 172.21.15.60:0/1784098959 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184004570 con 0x7f5188070350 2023-07-13T22:30:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f517e7fc700 1 -- 172.21.15.60:0/1784098959 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518401f070 con 0x7f5188070350 2023-07-13T22:30:45.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f517e7fc700 1 -- 172.21.15.60:0/1784098959 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840038c0 con 0x7f5188070350 2023-07-13T22:30:45.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518815a600 con 0x7f5188070350 2023-07-13T22:30:45.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.382+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518815abc0 con 0x7f5188070350 2023-07-13T22:30:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.383+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518804e210 con 0x7f5188070350 2023-07-13T22:30:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.384+0000 7f517e7fc700 1 -- 172.21.15.60:0/1784098959 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5184021030 con 0x7f5188070350 2023-07-13T22:30:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.385+0000 7f517e7fc700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f516c06ace0 0x7f516c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.385+0000 7f517e7fc700 1 -- 172.21.15.60:0/1784098959 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f518408a9a0 con 0x7f5188070350 2023-07-13T22:30:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.385+0000 7f518d59a700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f516c06ace0 0x7f516c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.386+0000 7f518d59a700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f516c06ace0 0x7f516c06d190 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5188164210 tx=0x7f5178008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:45.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.389+0000 7f517e7fc700 1 -- 172.21.15.60:0/1784098959 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5184055b60 con 0x7f5188070350 2023-07-13T22:30:45.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.676+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f51880cb8b0 con 0x7f5188070350 2023-07-13T22:30:45.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.680+0000 7f517e7fc700 1 -- 172.21.15.60:0/1784098959 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f51840150e0 con 0x7f5188070350 2023-07-13T22:30:45.661 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:30:45.661 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:30:45.661 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-07-13T22:30:45.661 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-07-13T22:30:45.662 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-13T22:30:45.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f516c06ace0 msgr2=0x7f516c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f516c06ace0 0x7f516c06d190 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5188164210 tx=0x7f5178008040 comp rx=0 tx=0).stop 2023-07-13T22:30:45.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5188070350 msgr2=0x7f518813e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5188070350 0x7f518813e7c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f5184000c00 tx=0x7f5184004750 comp rx=0 tx=0).stop 2023-07-13T22:30:45.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 shutdown_connections 2023-07-13T22:30:45.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f516c06ace0 0x7f516c06d190 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5188070350 0x7f518813e7c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f518815c980 0x7f518815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 --2- 172.21.15.60:0/1784098959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5188070720 0x7f518813ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 >> 172.21.15.60:0/1784098959 conn(0x7f51880cf3b0 msgr2=0x7f51880cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:45.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 shutdown_connections 2023-07-13T22:30:45.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.683+0000 7f518ffff700 1 -- 172.21.15.60:0/1784098959 wait complete. 2023-07-13T22:30:45.665 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-07-13T22:30:45.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:45.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.905+0000 7ff5455ff700 1 Processor -- start 2023-07-13T22:30:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.905+0000 7ff5455ff700 1 -- start start 2023-07-13T22:30:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.906+0000 7ff5455ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 0x7ff5400cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.906+0000 7ff5455ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff5400d26d0 0x7ff5400cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.906+0000 7ff5455ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5400d14a0 con 0x7ff5400b5a00 2023-07-13T22:30:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.906+0000 7ff5455ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5400d2570 con 0x7ff5400d26d0 2023-07-13T22:30:45.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.906+0000 7ff5455ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400d1780 con 0x7ff5400b2010 2023-07-13T22:30:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.907+0000 7ff53e7fc700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff5400d26d0 0x7ff5400cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.907+0000 7ff53effd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 0x7ff5400cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.907+0000 7ff53f7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff5400b5a00 0x7ff5400d2b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:46184/0 (socket says 172.21.15.60:46184) 2023-07-13T22:30:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.907+0000 7ff53f7fe700 1 -- 172.21.15.60:0/89252651 learned_addr learned my addr 172.21.15.60:0/89252651 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:45.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.907+0000 7ff53e7fc700 1 -- 172.21.15.60:0/89252651 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 msgr2=0x7ff5400cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.907+0000 7ff53e7fc700 1 --2- 172.21.15.60:0/89252651 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 0x7ff5400cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.907+0000 7ff53e7fc700 1 -- 172.21.15.60:0/89252651 >> v1:172.21.15.105:6789/0 conn(0x7ff5400b5a00 legacy=0x7ff5400d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.907+0000 7ff53e7fc700 1 -- 172.21.15.60:0/89252651 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400cd2a0 con 0x7ff5400d26d0 2023-07-13T22:30:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.907+0000 7ff53effd700 1 --2- 172.21.15.60:0/89252651 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 0x7ff5400cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.908+0000 7ff53e7fc700 1 --2- 172.21.15.60:0/89252651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 0x7ff5400cf6e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff534009840 tx=0x7ff534009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a073c0061cbb4582 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.908+0000 7ff53dffb700 1 -- 172.21.15.60:0/89252651 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534011830 con 0x7ff5400d26d0 2023-07-13T22:30:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.908+0000 7ff53dffb700 1 -- 172.21.15.60:0/89252651 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff534011990 con 0x7ff5400d26d0 2023-07-13T22:30:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.908+0000 7ff53dffb700 1 -- 172.21.15.60:0/89252651 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534011d20 con 0x7ff5400d26d0 2023-07-13T22:30:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.908+0000 7ff5455ff700 1 -- 172.21.15.60:0/89252651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 msgr2=0x7ff5400cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.908+0000 7ff5455ff700 1 --2- 172.21.15.60:0/89252651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 0x7ff5400cf6e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff534009840 tx=0x7ff534009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.909+0000 7ff5455ff700 1 -- 172.21.15.60:0/89252651 shutdown_connections 2023-07-13T22:30:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.909+0000 7ff5455ff700 1 --2- 172.21.15.60:0/89252651 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 0x7ff5400cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.909+0000 7ff5455ff700 1 --2- 172.21.15.60:0/89252651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 0x7ff5400cf6e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.909+0000 7ff5455ff700 1 -- 172.21.15.60:0/89252651 >> 172.21.15.60:0/89252651 conn(0x7ff54006b380 msgr2=0x7ff54006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:45.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.909+0000 7ff5455ff700 1 -- 172.21.15.60:0/89252651 shutdown_connections 2023-07-13T22:30:45.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.909+0000 7ff5455ff700 1 -- 172.21.15.60:0/89252651 wait complete. 2023-07-13T22:30:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.910+0000 7ff5455ff700 1 Processor -- start 2023-07-13T22:30:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.910+0000 7ff5455ff700 1 -- start start 2023-07-13T22:30:45.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.911+0000 7ff5455ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 0x7ff54015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.911+0000 7ff5455ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5400b5a00 0x7ff5401f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.911+0000 7ff5455ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 0x7ff54015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.911+0000 7ff5455ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400ca8e0 con 0x7ff5400b5a00 2023-07-13T22:30:45.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.911+0000 7ff5455ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54015c360 con 0x7ff5400d26d0 2023-07-13T22:30:45.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.911+0000 7ff5455ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400c9c80 con 0x7ff5400b2010 2023-07-13T22:30:45.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5400b5a00 0x7ff5401f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5400b5a00 0x7ff5401f1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50460/0 (socket says 172.21.15.60:50460) 2023-07-13T22:30:45.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53e7fc700 1 -- 172.21.15.60:0/724967390 learned_addr learned my addr 172.21.15.60:0/724967390 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53f7fe700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 0x7ff54015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53effd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 0x7ff54015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53e7fc700 1 -- 172.21.15.60:0/724967390 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 msgr2=0x7ff54015e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53e7fc700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 0x7ff54015e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53e7fc700 1 -- 172.21.15.60:0/724967390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 msgr2=0x7ff54015cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53e7fc700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 0x7ff54015cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53e7fc700 1 -- 172.21.15.60:0/724967390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5300094a0 con 0x7ff5400b5a00 2023-07-13T22:30:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53f7fe700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 0x7ff54015cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.912+0000 7ff53e7fc700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5400b5a00 0x7ff5401f1a10 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff5340090b0 tx=0x7ff534004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.913+0000 7ff5277fe700 1 -- 172.21.15.60:0/724967390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5340036a0 con 0x7ff5400b5a00 2023-07-13T22:30:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.913+0000 7ff5277fe700 1 -- 172.21.15.60:0/724967390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53401f070 con 0x7ff5400b5a00 2023-07-13T22:30:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.913+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5340094a0 con 0x7ff5400b5a00 2023-07-13T22:30:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.913+0000 7ff5277fe700 1 -- 172.21.15.60:0/724967390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534003a20 con 0x7ff5400b5a00 2023-07-13T22:30:45.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.913+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5400ca220 con 0x7ff5400b5a00 2023-07-13T22:30:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.914+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5400caa20 con 0x7ff5400b5a00 2023-07-13T22:30:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.915+0000 7ff5277fe700 1 -- 172.21.15.60:0/724967390 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff534003b80 con 0x7ff5400b5a00 2023-07-13T22:30:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.916+0000 7ff5277fe700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff52c06ace0 0x7ff52c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.917+0000 7ff5277fe700 1 -- 172.21.15.60:0/724967390 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff534021070 con 0x7ff5400b5a00 2023-07-13T22:30:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.917+0000 7ff53effd700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff52c06ace0 0x7ff52c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:45.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.917+0000 7ff53effd700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff52c06ace0 0x7ff52c06d190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff530009d70 tx=0x7ff530009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:45.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:45.921+0000 7ff5277fe700 1 -- 172.21.15.60:0/724967390 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff5400caa20 con 0x7ff5400b5a00 2023-07-13T22:30:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.171+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5400caa20 con 0x7ff5400b5a00 2023-07-13T22:30:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.172+0000 7ff5277fe700 1 -- 172.21.15.60:0/724967390 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4714 (secure 0 0 0) 0x7ff5400caa20 con 0x7ff5400b5a00 2023-07-13T22:30:46.152 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:46.152 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:30:39.690428+0000","modified":"2023-07-13T22:30:43.199497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6836/1781435251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1781435251},{"type":"v1","addr":"172.21.15.60:6836","nonce":1781435251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-13T22:30:46.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff52c06ace0 msgr2=0x7ff52c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff52c06ace0 0x7ff52c06d190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff530009d70 tx=0x7ff530009210 comp rx=0 tx=0).stop 2023-07-13T22:30:46.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5400b5a00 msgr2=0x7ff5401f1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5400b5a00 0x7ff5401f1a10 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff5340090b0 tx=0x7ff534004750 comp rx=0 tx=0).stop 2023-07-13T22:30:46.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 shutdown_connections 2023-07-13T22:30:46.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff52c06ace0 0x7ff52c06d190 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5400d26d0 0x7ff54015cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5400b2010 0x7ff54015e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 --2- 172.21.15.60:0/724967390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5400b5a00 0x7ff5401f1a10 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.174+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 >> 172.21.15.60:0/724967390 conn(0x7ff54006b380 msgr2=0x7ff5400c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:46.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.175+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 shutdown_connections 2023-07-13T22:30:46.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.175+0000 7ff5455ff700 1 -- 172.21.15.60:0/724967390 wait complete. 2023-07-13T22:30:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 85 2023-07-13T22:30:46.573 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:30:39.690428+0000', 'modified': '2023-07-13T22:30:43.199497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8937}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8937': {'gid': 8937, 'name': 'a', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6836/1781435251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 1781435251}, {'type': 'v1', 'addr': '172.21.15.60:6836', 'nonce': 1781435251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:30:46.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:30:46.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-13T22:30:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.804+0000 7fad32fff700 1 Processor -- start 2023-07-13T22:30:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.804+0000 7fad32fff700 1 -- start start 2023-07-13T22:30:46.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.805+0000 7fad32fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fad2c0b8570 0x7fad2c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.805+0000 7fad32fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0d35c0 0x7fad2c0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.805+0000 7fad32fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2c0d09d0 con 0x7fad2c0cf3f0 2023-07-13T22:30:46.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.805+0000 7fad32fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c0d3b30 con 0x7fad2c0b8570 2023-07-13T22:30:46.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.805+0000 7fad32fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0cda10 con 0x7fad2c0d35c0 2023-07-13T22:30:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.806+0000 7fad30d9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fad2c0b8570 0x7fad2c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.806+0000 7fad2bfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0d35c0 0x7fad2c0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.806+0000 7fad30d9b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fad2c0b8570 0x7fad2c0ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:57590/0 (socket says 172.21.15.60:57590) 2023-07-13T22:30:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.806+0000 7fad30d9b700 1 -- 172.21.15.60:0/1814360375 learned_addr learned my addr 172.21.15.60:0/1814360375 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.806+0000 7fad3159c700 1 --1- 172.21.15.60:0/1814360375 >> v1:172.21.15.105:6789/0 conn(0x7fad2c0cf3f0 0x7fad2c15fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:46200/0 (socket says 172.21.15.60:46200) 2023-07-13T22:30:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.806+0000 7fad30d9b700 1 -- 172.21.15.60:0/1814360375 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0d35c0 msgr2=0x7fad2c0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.806+0000 7fad30d9b700 1 --2- 172.21.15.60:0/1814360375 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0d35c0 0x7fad2c0d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.806+0000 7fad30d9b700 1 -- 172.21.15.60:0/1814360375 >> v1:172.21.15.105:6789/0 conn(0x7fad2c0cf3f0 legacy=0x7fad2c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.806+0000 7fad30d9b700 1 -- 172.21.15.60:0/1814360375 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c0cd450 con 0x7fad2c0b8570 2023-07-13T22:30:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad30d9b700 1 --2- 172.21.15.60:0/1814360375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad2c0b8570 0x7fad2c0ceeb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fad20006fd0 tx=0x7fad20009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aa9e502c54d2948 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad2b7fe700 1 -- 172.21.15.60:0/1814360375 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20005e30 con 0x7fad2c0b8570 2023-07-13T22:30:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad2b7fe700 1 -- 172.21.15.60:0/1814360375 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20004030 con 0x7fad2c0b8570 2023-07-13T22:30:46.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad2b7fe700 1 -- 172.21.15.60:0/1814360375 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad200043e0 con 0x7fad2c0b8570 2023-07-13T22:30:46.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad32fff700 1 -- 172.21.15.60:0/1814360375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad2c0b8570 msgr2=0x7fad2c0ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad32fff700 1 --2- 172.21.15.60:0/1814360375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad2c0b8570 0x7fad2c0ceeb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fad20006fd0 tx=0x7fad20009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad32fff700 1 -- 172.21.15.60:0/1814360375 shutdown_connections 2023-07-13T22:30:46.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad32fff700 1 --2- 172.21.15.60:0/1814360375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad2c0b8570 0x7fad2c0ceeb0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad32fff700 1 --2- 172.21.15.60:0/1814360375 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0d35c0 0x7fad2c0d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.807+0000 7fad32fff700 1 -- 172.21.15.60:0/1814360375 >> 172.21.15.60:0/1814360375 conn(0x7fad2c071600 msgr2=0x7fad2c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:46.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.808+0000 7fad32fff700 1 -- 172.21.15.60:0/1814360375 shutdown_connections 2023-07-13T22:30:46.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.808+0000 7fad32fff700 1 -- 172.21.15.60:0/1814360375 wait complete. 2023-07-13T22:30:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.809+0000 7fad32fff700 1 Processor -- start 2023-07-13T22:30:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.809+0000 7fad32fff700 1 -- start start 2023-07-13T22:30:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.810+0000 7fad32fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad2c0b8570 0x7fad2c1f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.810+0000 7fad32fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0cf3f0 0x7fad2c1ef310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.810+0000 7fad32fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad2c0d35c0 0x7fad2c0c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.810+0000 7fad32fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0c8600 con 0x7fad2c0d35c0 2023-07-13T22:30:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.810+0000 7fad32fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0c8860 con 0x7fad2c0b8570 2023-07-13T22:30:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.810+0000 7fad32fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0c8b40 con 0x7fad2c0cf3f0 2023-07-13T22:30:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad3159c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad2c0d35c0 0x7fad2c0c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad3159c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad2c0d35c0 0x7fad2c0c6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50466/0 (socket says 172.21.15.60:50466) 2023-07-13T22:30:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad30d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad2c0b8570 0x7fad2c1f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad3159c700 1 -- 172.21.15.60:0/2616849841 learned_addr learned my addr 172.21.15.60:0/2616849841 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad2bfff700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0cf3f0 0x7fad2c1ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad3159c700 1 -- 172.21.15.60:0/2616849841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0cf3f0 msgr2=0x7fad2c1ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad3159c700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0cf3f0 0x7fad2c1ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad3159c700 1 -- 172.21.15.60:0/2616849841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad2c0b8570 msgr2=0x7fad2c1f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad3159c700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad2c0b8570 0x7fad2c1f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad3159c700 1 -- 172.21.15.60:0/2616849841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20009500 con 0x7fad2c0d35c0 2023-07-13T22:30:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.811+0000 7fad3159c700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad2c0d35c0 0x7fad2c0c6790 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fad2400cbd0 tx=0x7fad2400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.812+0000 7fad297fa700 1 -- 172.21.15.60:0/2616849841 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400aaa0 con 0x7fad2c0d35c0 2023-07-13T22:30:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.812+0000 7fad32fff700 1 -- 172.21.15.60:0/2616849841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c0c8e20 con 0x7fad2c0d35c0 2023-07-13T22:30:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.812+0000 7fad297fa700 1 -- 172.21.15.60:0/2616849841 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2400ac00 con 0x7fad2c0d35c0 2023-07-13T22:30:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.812+0000 7fad297fa700 1 -- 172.21.15.60:0/2616849841 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24007070 con 0x7fad2c0d35c0 2023-07-13T22:30:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.812+0000 7fad32fff700 1 -- 172.21.15.60:0/2616849841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c0c93e0 con 0x7fad2c0d35c0 2023-07-13T22:30:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.814+0000 7fad297fa700 1 -- 172.21.15.60:0/2616849841 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad240071d0 con 0x7fad2c0d35c0 2023-07-13T22:30:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.815+0000 7fad297fa700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad1006ace0 0x7fad1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.815+0000 7fad297fa700 1 -- 172.21.15.60:0/2616849841 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad240890c0 con 0x7fad2c0d35c0 2023-07-13T22:30:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.815+0000 7fad30d9b700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad1006ace0 0x7fad1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.816+0000 7fad30d9b700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad1006ace0 0x7fad1006d190 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fad20006fd0 tx=0x7fad20008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:46.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.817+0000 7fad46a0ab80 1 Processor -- start 2023-07-13T22:30:46.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.817+0000 7fad46a0ab80 1 -- start start 2023-07-13T22:30:46.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.818+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f87f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.818+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f851dc0 0x555e2f87fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.818+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 0x555e2f86a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f7c9ea0 con 0x555e2f84f880 2023-07-13T22:30:46.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f73d600 con 0x555e2f851dc0 2023-07-13T22:30:46.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e2f7ac910 con 0x555e2f84fc50 2023-07-13T22:30:46.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad30d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f87f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad30d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f87f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50478/0 (socket says 172.21.15.60:50478) 2023-07-13T22:30:46.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad30d9b700 1 -- 172.21.15.60:0/294231574 learned_addr learned my addr 172.21.15.60:0/294231574 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:46.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad2bfff700 1 --2- 172.21.15.60:0/294231574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f851dc0 0x555e2f87fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad3159c700 1 --2- 172.21.15.60:0/294231574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 0x555e2f86a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad30d9b700 1 -- 172.21.15.60:0/294231574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 msgr2=0x555e2f86a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad30d9b700 1 --2- 172.21.15.60:0/294231574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 0x555e2f86a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad30d9b700 1 -- 172.21.15.60:0/294231574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f851dc0 msgr2=0x555e2f87fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad30d9b700 1 --2- 172.21.15.60:0/294231574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f851dc0 0x555e2f87fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.819+0000 7fad30d9b700 1 -- 172.21.15.60:0/294231574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e2f599790 con 0x555e2f84f880 2023-07-13T22:30:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.820+0000 7fad30d9b700 1 --2- 172.21.15.60:0/294231574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f87f5e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fad20004710 tx=0x7fad20010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbcb4299cc0b612 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.820+0000 7fad167fc700 1 -- 172.21.15.60:0/294231574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20003b10 con 0x555e2f84f880 2023-07-13T22:30:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.820+0000 7fad167fc700 1 -- 172.21.15.60:0/294231574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20022070 con 0x555e2f84f880 2023-07-13T22:30:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.820+0000 7fad167fc700 1 -- 172.21.15.60:0/294231574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20003e90 con 0x555e2f84f880 2023-07-13T22:30:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.820+0000 7fad46a0ab80 1 -- 172.21.15.60:0/294231574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 msgr2=0x555e2f87f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.820+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/294231574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f87f5e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fad20004710 tx=0x7fad20010000 comp rx=0 tx=0).stop 2023-07-13T22:30:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.821+0000 7fad46a0ab80 1 -- 172.21.15.60:0/294231574 shutdown_connections 2023-07-13T22:30:46.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.821+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/294231574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f87f5e0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.821+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/294231574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 0x555e2f86a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.821+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/294231574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f851dc0 0x555e2f87fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.821+0000 7fad46a0ab80 1 -- 172.21.15.60:0/294231574 >> 172.21.15.60:0/294231574 conn(0x555e2f84a3b0 msgr2=0x555e2f855550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.821+0000 7fad46a0ab80 1 -- 172.21.15.60:0/294231574 shutdown_connections 2023-07-13T22:30:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.821+0000 7fad46a0ab80 1 -- 172.21.15.60:0/294231574 wait complete. 2023-07-13T22:30:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.821+0000 7fad46a0ab80 1 -- start start 2023-07-13T22:30:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.822+0000 7fad46a0ab80 1 Processor -- start 2023-07-13T22:30:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f86ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 0x555e2f89c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f851dc0 0x555e2f880780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f7a5be0 con 0x555e2f84f880 2023-07-13T22:30:46.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f7635b0 con 0x555e2f84fc50 2023-07-13T22:30:46.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e2f749910 con 0x555e2f851dc0 2023-07-13T22:30:46.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad46a0ab80 10 client.0 _ll_register_callbacks cb 0x555e2f761b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:30:46.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad30d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f86ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad30d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f86ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50486/0 (socket says 172.21.15.60:50486) 2023-07-13T22:30:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad30d9b700 1 -- 172.21.15.60:0/1013126655 learned_addr learned my addr 172.21.15.60:0/1013126655 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad3159c700 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f851dc0 0x555e2f880780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad2bfff700 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 0x555e2f89c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad30d9b700 1 -- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f851dc0 msgr2=0x555e2f880780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad30d9b700 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f851dc0 0x555e2f880780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad30d9b700 1 -- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 msgr2=0x555e2f89c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:46.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad30d9b700 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 0x555e2f89c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad30d9b700 1 -- 172.21.15.60:0/1013126655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c0cd450 con 0x555e2f84f880 2023-07-13T22:30:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad3159c700 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f851dc0 0x555e2f880780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.823+0000 7fad2bfff700 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 0x555e2f89c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:46.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.824+0000 7fad30d9b700 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f86ae60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fad20000c00 tx=0x7fad200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634fa70ef784a573 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.824+0000 7fad0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:30:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.824+0000 7fad0b7fe700 1 -- 172.21.15.60:0/1013126655 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20011830 con 0x555e2f84f880 2023-07-13T22:30:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.824+0000 7fad46a0ab80 1 -- 172.21.15.60:0/1013126655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e2f8479c0 con 0x555e2f84f880 2023-07-13T22:30:46.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.824+0000 7fad0b7fe700 1 -- 172.21.15.60:0/1013126655 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad20024070 con 0x555e2f84f880 2023-07-13T22:30:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.824+0000 7fad0b7fe700 1 -- 172.21.15.60:0/1013126655 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad20011990 con 0x555e2f84f880 2023-07-13T22:30:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.824+0000 7fad46a0ab80 10 client.9063 fetch_fsmap learned FSMap version 85 2023-07-13T22:30:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.824+0000 7fad46a0ab80 1 -- 172.21.15.60:0/1013126655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x555e2f6860a0 con 0x555e2f84f880 2023-07-13T22:30:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.824+0000 7fad0b7fe700 1 -- 172.21.15.60:0/1013126655 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555e2f8479c0 con 0x555e2f84f880 2023-07-13T22:30:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad0b7fe700 1 -- 172.21.15.60:0/1013126655 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fad20039070 con 0x555e2f84f880 2023-07-13T22:30:46.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad46a0ab80 10 client.9063 fetch_fsmap finished waiting for FSMap version 85 2023-07-13T22:30:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad46a0ab80 10 client.9063 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:30:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad46a0ab80 20 client.9063 populate_metadata read hostname 'smithi060' 2023-07-13T22:30:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] conn(0x555e2f8501b0 0x555e2f850580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad46a0ab80 4 client.9063 mds_command: new command op to 8937 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:30:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad46a0ab80 1 -- 172.21.15.60:0/1013126655 --> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e2f824170 con 0x555e2f8501b0 2023-07-13T22:30:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad2bfff700 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] conn(0x555e2f8501b0 0x555e2f850580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad2bfff700 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] conn(0x555e2f8501b0 0x555e2f850580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a26e03e33fd8c84 server_cookie=c714d524351a651a in_seq=0 out_seq=0 2023-07-13T22:30:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.825+0000 7fad0b7fe700 10 client.9063 ms_handle_connect on v2:172.21.15.60:6834/1781435251 2023-07-13T22:30:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.826+0000 7fad0b7fe700 1 -- 172.21.15.60:0/1013126655 <== mds.0 v2:172.21.15.60:6834/1781435251 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555e2f824170 con 0x555e2f8501b0 2023-07-13T22:30:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.826+0000 7fad0b7fe700 10 client.9063 handle_command_reply: tid=0 2023-07-13T22:30:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.826+0000 7fad46a0ab80 1 client.9063 shutdown 2023-07-13T22:30:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.826+0000 7fad46a0ab80 1 -- 172.21.15.60:0/1013126655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 msgr2=0x555e2f86ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.826+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f86ae60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fad20000c00 tx=0x7fad200047c0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad46a0ab80 1 -- 172.21.15.60:0/1013126655 shutdown_connections 2023-07-13T22:30:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] conn(0x555e2f8501b0 0x555e2f850580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f84f880 0x555e2f86ae60 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f851dc0 0x555e2f880780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/1013126655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 0x555e2f89c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad46a0ab80 1 -- 172.21.15.60:0/1013126655 >> 172.21.15.60:0/1013126655 conn(0x555e2f84a3b0 msgr2=0x555e2f89b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad0b7fe700 0 client.9063 ms_handle_reset on v2:172.21.15.60:6834/1781435251 2023-07-13T22:30:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad46a0ab80 1 -- 172.21.15.60:0/1013126655 shutdown_connections 2023-07-13T22:30:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad46a0ab80 1 -- 172.21.15.60:0/1013126655 wait complete. 2023-07-13T22:30:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.827+0000 7fad46a0ab80 20 client.9063 trim_cache size 0 max 16384 2023-07-13T22:30:46.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.845+0000 7fad46a0ab80 1 Processor -- start 2023-07-13T22:30:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.845+0000 7fad46a0ab80 1 -- start start 2023-07-13T22:30:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.846+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84f880 0x555e2f894c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.846+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 0x555e2f84e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.846+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f84eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.846+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f59a390 con 0x555e2f851dc0 2023-07-13T22:30:46.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.846+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f8951c0 con 0x555e2f84fc50 2023-07-13T22:30:46.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.846+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e2f831c80 con 0x555e2f84f880 2023-07-13T22:30:46.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad3159c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f84eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad3159c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f84eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50494/0 (socket says 172.21.15.60:50494) 2023-07-13T22:30:46.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad3159c700 1 -- 172.21.15.60:0/138385198 learned_addr learned my addr 172.21.15.60:0/138385198 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:46.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad2bfff700 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 0x555e2f84e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad30d9b700 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84f880 0x555e2f894c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad3159c700 1 -- 172.21.15.60:0/138385198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84f880 msgr2=0x555e2f894c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:46.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad3159c700 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84f880 0x555e2f894c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad3159c700 1 -- 172.21.15.60:0/138385198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 msgr2=0x555e2f84e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:46.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad3159c700 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 0x555e2f84e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad3159c700 1 -- 172.21.15.60:0/138385198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e2f76bb10 con 0x555e2f851dc0 2023-07-13T22:30:46.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad30d9b700 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84f880 0x555e2f894c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:46.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad3159c700 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f84eb80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fad2400cba0 tx=0x7fad2405d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9b1625812cf9b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:46.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad16ffd700 1 -- 172.21.15.60:0/138385198 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24056270 con 0x555e2f851dc0 2023-07-13T22:30:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad16ffd700 1 -- 172.21.15.60:0/138385198 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2405fce0 con 0x555e2f851dc0 2023-07-13T22:30:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.847+0000 7fad16ffd700 1 -- 172.21.15.60:0/138385198 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24053cc0 con 0x555e2f851dc0 2023-07-13T22:30:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.848+0000 7fad46a0ab80 1 -- 172.21.15.60:0/138385198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 msgr2=0x555e2f84eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.848+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f84eb80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fad2400cba0 tx=0x7fad2405d040 comp rx=0 tx=0).stop 2023-07-13T22:30:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.848+0000 7fad46a0ab80 1 -- 172.21.15.60:0/138385198 shutdown_connections 2023-07-13T22:30:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.848+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f84eb80 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.848+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84f880 0x555e2f894c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.848+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/138385198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84fc50 0x555e2f84e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.848+0000 7fad46a0ab80 1 -- 172.21.15.60:0/138385198 >> 172.21.15.60:0/138385198 conn(0x555e2f84a3b0 msgr2=0x555e2f89c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.848+0000 7fad46a0ab80 1 -- 172.21.15.60:0/138385198 shutdown_connections 2023-07-13T22:30:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.848+0000 7fad46a0ab80 1 -- 172.21.15.60:0/138385198 wait complete. 2023-07-13T22:30:46.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.849+0000 7fad46a0ab80 1 -- start start 2023-07-13T22:30:46.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.849+0000 7fad46a0ab80 1 Processor -- start 2023-07-13T22:30:46.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84f880 0x555e2f89b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 0x555e2f901440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad46a0ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f890ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f890ad0 con 0x555e2f851dc0 2023-07-13T22:30:46.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f880a40 con 0x555e2f84f880 2023-07-13T22:30:46.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad46a0ab80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e2f89a270 con 0x555e2f84fc50 2023-07-13T22:30:46.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad46a0ab80 10 client.0 _ll_register_callbacks cb 0x7fad200045e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:30:46.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad3159c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f890ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad3159c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f890ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50506/0 (socket says 172.21.15.60:50506) 2023-07-13T22:30:46.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad3159c700 1 -- 172.21.15.60:0/946165453 learned_addr learned my addr 172.21.15.60:0/946165453 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:46.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad30d9b700 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84f880 0x555e2f89b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.850+0000 7fad2bfff700 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 0x555e2f901440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad3159c700 1 -- 172.21.15.60:0/946165453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 msgr2=0x555e2f901440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:46.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad3159c700 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 0x555e2f901440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad3159c700 1 -- 172.21.15.60:0/946165453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84f880 msgr2=0x555e2f89b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:46.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad3159c700 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84f880 0x555e2f89b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad3159c700 1 -- 172.21.15.60:0/946165453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c0c93e0 con 0x555e2f851dc0 2023-07-13T22:30:46.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad30d9b700 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84f880 0x555e2f89b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad3159c700 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f890ef0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fad2405fc40 tx=0x7fad2400af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b924cfa55b0db08 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:30:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad0b7fe700 1 -- 172.21.15.60:0/946165453 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24054030 con 0x555e2f851dc0 2023-07-13T22:30:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad46a0ab80 1 -- 172.21.15.60:0/946165453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad20022070 con 0x555e2f851dc0 2023-07-13T22:30:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad0b7fe700 1 -- 172.21.15.60:0/946165453 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad24076b40 con 0x555e2f851dc0 2023-07-13T22:30:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad0b7fe700 1 -- 172.21.15.60:0/946165453 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad24076d20 con 0x555e2f851dc0 2023-07-13T22:30:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad46a0ab80 10 client.9069 fetch_fsmap learned FSMap version 85 2023-07-13T22:30:46.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.851+0000 7fad46a0ab80 1 -- 172.21.15.60:0/946165453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x555e2f5fa1d0 con 0x555e2f851dc0 2023-07-13T22:30:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad0b7fe700 1 -- 172.21.15.60:0/946165453 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20022070 con 0x555e2f851dc0 2023-07-13T22:30:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad0b7fe700 1 -- 172.21.15.60:0/946165453 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fad24056e30 con 0x555e2f851dc0 2023-07-13T22:30:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad46a0ab80 10 client.9069 fetch_fsmap finished waiting for FSMap version 85 2023-07-13T22:30:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad46a0ab80 10 client.9069 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:30:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad46a0ab80 20 client.9069 populate_metadata read hostname 'smithi060' 2023-07-13T22:30:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] conn(0x555e2f84e640 0x555e2f84ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad46a0ab80 4 client.9069 mds_command: new command op to 8937 tid=0[{"prefix": "status"}] 2023-07-13T22:30:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad46a0ab80 1 -- 172.21.15.60:0/946165453 --> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555e2f79ed20 con 0x555e2f84e640 2023-07-13T22:30:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad30d9b700 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] conn(0x555e2f84e640 0x555e2f84ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad0b7fe700 10 client.9069 ms_handle_connect on v2:172.21.15.60:6834/1781435251 2023-07-13T22:30:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.852+0000 7fad30d9b700 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] conn(0x555e2f84e640 0x555e2f84ea10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecfa19577b46df02 server_cookie=f0fb378a360d7536 in_seq=0 out_seq=0 2023-07-13T22:30:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad0b7fe700 1 -- 172.21.15.60:0/946165453 <== mds.0 v2:172.21.15.60:6834/1781435251 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x555e2f79ed20 con 0x555e2f84e640 2023-07-13T22:30:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad0b7fe700 10 client.9069 handle_command_reply: tid=0 2023-07-13T22:30:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 client.9069 shutdown 2023-07-13T22:30:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 -- 172.21.15.60:0/946165453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 msgr2=0x555e2f890ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f890ef0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fad2405fc40 tx=0x7fad2400af70 comp rx=0 tx=0).stop 2023-07-13T22:30:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 -- 172.21.15.60:0/946165453 shutdown_connections 2023-07-13T22:30:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.60:6834/1781435251,v1:172.21.15.60:6836/1781435251] conn(0x555e2f84e640 0x555e2f84ea10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e2f851dc0 0x555e2f890ef0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e2f84f880 0x555e2f89b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad0b7fe700 0 client.9069 ms_handle_reset on v2:172.21.15.60:6834/1781435251 2023-07-13T22:30:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 --2- 172.21.15.60:0/946165453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x555e2f84fc50 0x555e2f901440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 -- 172.21.15.60:0/946165453 >> 172.21.15.60:0/946165453 conn(0x555e2f84a3b0 msgr2=0x555e2f869320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 -- 172.21.15.60:0/946165453 shutdown_connections 2023-07-13T22:30:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 1 -- 172.21.15.60:0/946165453 wait complete. 2023-07-13T22:30:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.853+0000 7fad46a0ab80 20 client.9069 trim_cache size 0 max 16384 2023-07-13T22:30:46.841 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:30:46.841 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:30:46.841 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:30:46.842 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8937, 2023-07-13T22:30:46.842 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:30:46.842 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:30:46.842 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:30:46.842 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.6686490650000003, 2023-07-13T22:30:46.842 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 85, 2023-07-13T22:30:46.842 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 124, 2023-07-13T22:30:46.843 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 124, 2023-07-13T22:30:46.843 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 13.126803237000001 2023-07-13T22:30:46.843 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:30:46.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 -- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad1006ace0 msgr2=0x7fad1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad1006ace0 0x7fad1006d190 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fad20006fd0 tx=0x7fad20008070 comp rx=0 tx=0).stop 2023-07-13T22:30:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 -- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad2c0d35c0 msgr2=0x7fad2c0c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad2c0d35c0 0x7fad2c0c6790 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fad2400cbd0 tx=0x7fad2400cee0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 -- 172.21.15.60:0/2616849841 shutdown_connections 2023-07-13T22:30:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fad1006ace0 0x7fad1006d190 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad2c0d35c0 0x7fad2c0c6790 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad2c0b8570 0x7fad2c1f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 --2- 172.21.15.60:0/2616849841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad2c0cf3f0 0x7fad2c1ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 -- 172.21.15.60:0/2616849841 >> 172.21.15.60:0/2616849841 conn(0x7fad2c071600 msgr2=0x7fad2c0d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 -- 172.21.15.60:0/2616849841 shutdown_connections 2023-07-13T22:30:46.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:46.854+0000 7fad32fff700 1 -- 172.21.15.60:0/2616849841 wait complete. 2023-07-13T22:30:46.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:30:47.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.074+0000 7f46243ff700 1 Processor -- start 2023-07-13T22:30:47.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.074+0000 7f46243ff700 1 -- start start 2023-07-13T22:30:47.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.075+0000 7f46243ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f461c0d15c0 0x7f461c167030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:47.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.075+0000 7f46243ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0d09d0 con 0x7f461c0b8570 2023-07-13T22:30:47.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.075+0000 7f46243ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0cda10 con 0x7f461c0d35c0 2023-07-13T22:30:47.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.075+0000 7f46243ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c0d3b30 con 0x7f461c0d15c0 2023-07-13T22:30:47.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.076+0000 7f462299c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f461c0d15c0 0x7f461c167030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:47.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.076+0000 7f462219b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f461c0b8570 0x7f461c0ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58196/0 (socket says 172.21.15.60:58196) 2023-07-13T22:30:47.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.076+0000 7f462219b700 1 -- 172.21.15.60:0/1105744414 learned_addr learned my addr 172.21.15.60:0/1105744414 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:47.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.076+0000 7f4621199700 1 -- 172.21.15.60:0/1105744414 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267290397 0 0) 0x7f461c0d09d0 con 0x7f461c0b8570 2023-07-13T22:30:47.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.076+0000 7f4621199700 1 -- 172.21.15.60:0/1105744414 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46080035b0 con 0x7f461c0b8570 2023-07-13T22:30:47.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.076+0000 7f462299c700 1 -- 172.21.15.60:0/1105744414 >> v1:172.21.15.105:6789/0 conn(0x7f461c0d35c0 legacy=0x7f461c0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:47.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.076+0000 7f462299c700 1 -- 172.21.15.60:0/1105744414 >> v1:172.21.15.60:6789/0 conn(0x7f461c0b8570 legacy=0x7f461c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:47.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.076+0000 7f462299c700 1 -- 172.21.15.60:0/1105744414 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0cd450 con 0x7f461c0d15c0 2023-07-13T22:30:47.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.076+0000 7f462299c700 1 --2- 172.21.15.60:0/1105744414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0d15c0 0x7f461c167030 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f4614009d90 tx=0x7f461400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f831b3352d2f204 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:47.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f4621199700 1 -- 172.21.15.60:0/1105744414 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400e070 con 0x7f461c0d15c0 2023-07-13T22:30:47.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f4621199700 1 -- 172.21.15.60:0/1105744414 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461400bb20 con 0x7f461c0d15c0 2023-07-13T22:30:47.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f4621199700 1 -- 172.21.15.60:0/1105744414 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614012430 con 0x7f461c0d15c0 2023-07-13T22:30:47.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f46243ff700 1 -- 172.21.15.60:0/1105744414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0d15c0 msgr2=0x7f461c167030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:47.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f46243ff700 1 --2- 172.21.15.60:0/1105744414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0d15c0 0x7f461c167030 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f4614009d90 tx=0x7f461400b330 comp rx=0 tx=0).stop 2023-07-13T22:30:47.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f46243ff700 1 -- 172.21.15.60:0/1105744414 shutdown_connections 2023-07-13T22:30:47.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f46243ff700 1 --2- 172.21.15.60:0/1105744414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0d15c0 0x7f461c167030 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:47.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f46243ff700 1 -- 172.21.15.60:0/1105744414 >> 172.21.15.60:0/1105744414 conn(0x7f461c071600 msgr2=0x7f461c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:47.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f46243ff700 1 -- 172.21.15.60:0/1105744414 shutdown_connections 2023-07-13T22:30:47.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.077+0000 7f46243ff700 1 -- 172.21.15.60:0/1105744414 wait complete. 2023-07-13T22:30:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.079+0000 7f46243ff700 1 Processor -- start 2023-07-13T22:30:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.079+0000 7f46243ff700 1 -- start start 2023-07-13T22:30:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.080+0000 7f46243ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0b8570 0x7f461c0b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.080+0000 7f46243ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461c0d15c0 0x7f461c0c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.080+0000 7f46243ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f461c0d35c0 0x7f461c1f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.080+0000 7f46243ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0c7d80 con 0x7f461c0b8570 2023-07-13T22:30:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.080+0000 7f46243ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0c8060 con 0x7f461c0d15c0 2023-07-13T22:30:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.080+0000 7f46243ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0c8340 con 0x7f461c0d35c0 2023-07-13T22:30:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.080+0000 7f462219b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0b8570 0x7f461c0b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.080+0000 7f462219b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0b8570 0x7f461c0b6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50536/0 (socket says 172.21.15.60:50536) 2023-07-13T22:30:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.080+0000 7f462219b700 1 -- 172.21.15.60:0/3018031241 learned_addr learned my addr 172.21.15.60:0/3018031241 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f462199a700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461c0d15c0 0x7f461c0c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f462299c700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f461c0d35c0 0x7f461c1f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f462219b700 1 -- 172.21.15.60:0/3018031241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f461c0d35c0 msgr2=0x7f461c1f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f462219b700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f461c0d35c0 0x7f461c1f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f462219b700 1 -- 172.21.15.60:0/3018031241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461c0d15c0 msgr2=0x7f461c0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f462219b700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461c0d15c0 0x7f461c0c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f462219b700 1 -- 172.21.15.60:0/3018031241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614009a70 con 0x7f461c0b8570 2023-07-13T22:30:47.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f462299c700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f461c0d35c0 0x7f461c1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f462219b700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0b8570 0x7f461c0b6e80 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f4618001260 tx=0x7f461800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f4612ffd700 1 -- 172.21.15.60:0/3018031241 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180074e0 con 0x7f461c0b8570 2023-07-13T22:30:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f4612ffd700 1 -- 172.21.15.60:0/3018031241 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4618007640 con 0x7f461c0b8570 2023-07-13T22:30:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c0c8620 con 0x7f461c0b8570 2023-07-13T22:30:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.081+0000 7f4612ffd700 1 -- 172.21.15.60:0/3018031241 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618005a50 con 0x7f461c0b8570 2023-07-13T22:30:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.082+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c158510 con 0x7f461c0b8570 2023-07-13T22:30:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.082+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c15c340 con 0x7f461c0b8570 2023-07-13T22:30:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.084+0000 7f4612ffd700 1 -- 172.21.15.60:0/3018031241 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4618005bb0 con 0x7f461c0b8570 2023-07-13T22:30:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.085+0000 7f4612ffd700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f460006ace0 0x7f460006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.085+0000 7f4612ffd700 1 -- 172.21.15.60:0/3018031241 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46180888f0 con 0x7f461c0b8570 2023-07-13T22:30:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.085+0000 7f462199a700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f460006ace0 0x7f460006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:47.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.085+0000 7f462199a700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f460006ace0 0x7f460006d190 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f461c0cb2b0 tx=0x7f460c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:47.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.089+0000 7f4612ffd700 1 -- 172.21.15.60:0/3018031241 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f46180574c0 con 0x7f461c0b8570 2023-07-13T22:30:47.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.346+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f461c15c060 con 0x7f461c0b8570 2023-07-13T22:30:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.346+0000 7f4612ffd700 1 -- 172.21.15.60:0/3018031241 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4618017030 con 0x7f461c0b8570 2023-07-13T22:30:47.326 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:30:47.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f460006ace0 msgr2=0x7f460006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f460006ace0 0x7f460006d190 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f461c0cb2b0 tx=0x7f460c009040 comp rx=0 tx=0).stop 2023-07-13T22:30:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0b8570 msgr2=0x7f461c0b6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0b8570 0x7f461c0b6e80 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f4618001260 tx=0x7f461800c9e0 comp rx=0 tx=0).stop 2023-07-13T22:30:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 shutdown_connections 2023-07-13T22:30:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f460006ace0 0x7f460006d190 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f461c0b8570 0x7f461c0b6e80 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f461c0d35c0 0x7f461c1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 --2- 172.21.15.60:0/3018031241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461c0d15c0 0x7f461c0c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 >> 172.21.15.60:0/3018031241 conn(0x7f461c071600 msgr2=0x7f461c0734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 shutdown_connections 2023-07-13T22:30:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:47.349+0000 7f46243ff700 1 -- 172.21.15.60:0/3018031241 wait complete. 2023-07-13T22:30:47.346 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:30:47.347 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:30:47.347 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:30:47.348 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:30:47.349 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:30:47.349 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:30:47.350 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:30:47.350 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:30:47.351 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:30:47.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:47.352 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:30:47.399 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:30:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:30:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:30:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:30:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:47.400 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:30:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:30:47.401 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:30:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:30:47.401 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:30:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:30:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:30:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41126sec preferred_lft 41126sec 2023-07-13T22:30:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:30:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:47.402 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:30:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:30:47.403 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:30:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:30:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:30:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:47.403 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:30:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:30:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:30:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:47.404 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:30:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:30:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:30:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:47.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:47.406 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:30:47.514 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:30:47.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:47.518 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:30:47.567 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:30:47.567 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:30:47.567 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:30:47.568 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:30:47.568 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:30:47.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:47.569 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:30:47.569 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:30:47.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:30:47.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:30:47.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:30:47.571 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:30:47.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:47.744 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:30:47.791 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:30:47.791 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:30:47.792 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:30:47.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:47.793 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:30:47.904 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:30:47.904 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:30:47.905 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:30:47.951 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:30:47.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:30:47.953 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:30:47.999 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:30:48.111 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:30:48.112 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:30:48.184 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:30:48.187 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:30:48.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:48.188 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:30:48.234 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:30:48.235 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:30:48.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:30:48.314 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:30:48.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:30:48.386 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:30:48.388 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:30:48.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:48.390 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:30:48.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:30:48.434+0000 7f24da2995c0 -1 init, newargv = 0x565203cb6090 newargc=15 2023-07-13T22:30:48.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[161669]: starting ceph client 2023-07-13T22:30:48.421 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[161669]: starting fuse 2023-07-13T22:30:48.443 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:30:48.443 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:30:48.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:30:48.444 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:30:48.444 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:30:48.444 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:30:48.445 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:30:48.445 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:48.445 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:30:48.445 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:30:48.446 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:30:48.446 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:30:48.446 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:48.447 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:30:48.447 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:30:48.447 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:30:48.447 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:30:48.448 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:48.448 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:30:48.448 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:30:48.449 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:30:48.449 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:30:48.449 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:30:48.449 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:30:48.450 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:30:48.450 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:30:48.450 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:48.451 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:30:48.451 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:30:48.631 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.161669.asok 2023-07-13T22:30:48.808 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.161669.asok 2023-07-13T22:30:48.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:48.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.161669.asok status 2023-07-13T22:30:49.040 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:30:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:30:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:30:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:30:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:30:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:30:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:30:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "161669", 2023-07-13T22:30:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:30:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:30:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:30:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9084, 2023-07-13T22:30:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:30:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:30:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:30:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 9084 2023-07-13T22:30:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:30:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:30:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:30:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3195742970 2023-07-13T22:30:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:30:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:30:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:30:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3195742970 2023-07-13T22:30:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.9084 192.168.0.1:0/3195742970", 2023-07-13T22:30:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3195742970", 2023-07-13T22:30:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:30:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 85, 2023-07-13T22:30:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 124, 2023-07-13T22:30:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:30:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:30:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:30:49.046 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:30:49.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:49.055 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:30:49.055 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:30:49.102 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:30:49.102 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:30:49.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:49.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:30:49.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:30:49.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.397+0000 7fc7131ff700 1 Processor -- start 2023-07-13T22:30:49.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.397+0000 7fc7131ff700 1 -- start start 2023-07-13T22:30:49.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.398+0000 7fc7131ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc70c15c6e0 0x7fc70c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:49.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.398+0000 7fc7131ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc70c0b3050 0x7fc70c06afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:49.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.398+0000 7fc7131ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70c06b8e0 con 0x7fc70c0b24b0 2023-07-13T22:30:49.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.398+0000 7fc7131ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70c15cab0 con 0x7fc70c0b3050 2023-07-13T22:30:49.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.398+0000 7fc7131ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70c06bbc0 con 0x7fc70c15c6e0 2023-07-13T22:30:49.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.399+0000 7fc70bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc70c0b3050 0x7fc70c06afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:49.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.399+0000 7fc71179c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc70c0b24b0 0x7fc70c0cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58206/0 (socket says 172.21.15.60:58206) 2023-07-13T22:30:49.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.399+0000 7fc71179c700 1 -- 172.21.15.60:0/3871630475 learned_addr learned my addr 172.21.15.60:0/3871630475 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:49.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.399+0000 7fc710f9b700 1 --2- 172.21.15.60:0/3871630475 >> v2:172.21.15.105:3300/0 conn(0x7fc70c15c6e0 0x7fc70c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:49.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.399+0000 7fc70bfff700 1 -- 172.21.15.60:0/3871630475 >> v2:172.21.15.105:3300/0 conn(0x7fc70c15c6e0 msgr2=0x7fc70c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:49.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.399+0000 7fc70bfff700 1 --2- 172.21.15.60:0/3871630475 >> v2:172.21.15.105:3300/0 conn(0x7fc70c15c6e0 0x7fc70c06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.399+0000 7fc70bfff700 1 -- 172.21.15.60:0/3871630475 >> v1:172.21.15.60:6789/0 conn(0x7fc70c0b24b0 legacy=0x7fc70c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.399+0000 7fc70bfff700 1 -- 172.21.15.60:0/3871630475 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c06bea0 con 0x7fc70c0b3050 2023-07-13T22:30:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.399+0000 7fc70bfff700 1 --2- 172.21.15.60:0/3871630475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 0x7fc70c06afe0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0ce0f0 tx=0x7fc6f8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51622718c062724e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc70b7fe700 1 -- 172.21.15.60:0/3871630475 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8011830 con 0x7fc70c0b3050 2023-07-13T22:30:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc70b7fe700 1 -- 172.21.15.60:0/3871630475 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f8011990 con 0x7fc70c0b3050 2023-07-13T22:30:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc70b7fe700 1 -- 172.21.15.60:0/3871630475 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8011d20 con 0x7fc70c0b3050 2023-07-13T22:30:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc7131ff700 1 -- 172.21.15.60:0/3871630475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 msgr2=0x7fc70c06afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:49.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc7131ff700 1 --2- 172.21.15.60:0/3871630475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 0x7fc70c06afe0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0ce0f0 tx=0x7fc6f8009b20 comp rx=0 tx=0).stop 2023-07-13T22:30:49.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc7131ff700 1 -- 172.21.15.60:0/3871630475 shutdown_connections 2023-07-13T22:30:49.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc7131ff700 1 --2- 172.21.15.60:0/3871630475 >> v2:172.21.15.105:3300/0 conn(0x7fc70c15c6e0 0x7fc70c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:49.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc7131ff700 1 --2- 172.21.15.60:0/3871630475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 0x7fc70c06afe0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:49.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc7131ff700 1 -- 172.21.15.60:0/3871630475 >> 172.21.15.60:0/3871630475 conn(0x7fc70c06e680 msgr2=0x7fc70c06ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:49.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc7131ff700 1 -- 172.21.15.60:0/3871630475 shutdown_connections 2023-07-13T22:30:49.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.400+0000 7fc7131ff700 1 -- 172.21.15.60:0/3871630475 wait complete. 2023-07-13T22:30:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.402+0000 7fc7131ff700 1 Processor -- start 2023-07-13T22:30:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.402+0000 7fc7131ff700 1 -- start start 2023-07-13T22:30:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.403+0000 7fc7131ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc70c0b24b0 0x7fc70c1ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.403+0000 7fc7131ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 0x7fc70c1ef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:49.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.403+0000 7fc7131ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc70c15c6e0 0x7fc70c15ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:49.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.403+0000 7fc7131ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1f20e0 con 0x7fc70c0b3050 2023-07-13T22:30:49.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.403+0000 7fc7131ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1f23c0 con 0x7fc70c15c6e0 2023-07-13T22:30:49.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.403+0000 7fc7131ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1f26a0 con 0x7fc70c0b24b0 2023-07-13T22:30:49.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.403+0000 7fc70bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 0x7fc70c1ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:49.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc70bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 0x7fc70c1ef600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50564/0 (socket says 172.21.15.60:50564) 2023-07-13T22:30:49.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc70bfff700 1 -- 172.21.15.60:0/987978303 learned_addr learned my addr 172.21.15.60:0/987978303 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc710f9b700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc70c0b24b0 0x7fc70c1ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc71179c700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc70c15c6e0 0x7fc70c15ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc70bfff700 1 -- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc70c0b24b0 msgr2=0x7fc70c1ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc70bfff700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc70c0b24b0 0x7fc70c1ef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc70bfff700 1 -- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc70c15c6e0 msgr2=0x7fc70c15ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc70bfff700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc70c15c6e0 0x7fc70c15ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc70bfff700 1 -- 172.21.15.60:0/987978303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f80094a0 con 0x7fc70c0b3050 2023-07-13T22:30:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc710f9b700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc70c0b24b0 0x7fc70c1ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc71179c700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc70c15c6e0 0x7fc70c15ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc70bfff700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 0x7fc70c1ef600 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009b60 tx=0x7fc6f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.404+0000 7fc7097fa700 1 -- 172.21.15.60:0/987978303 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80036a0 con 0x7fc70c0b3050 2023-07-13T22:30:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.405+0000 7fc7097fa700 1 -- 172.21.15.60:0/987978303 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f801f070 con 0x7fc70c0b3050 2023-07-13T22:30:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.405+0000 7fc7097fa700 1 -- 172.21.15.60:0/987978303 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8003a40 con 0x7fc70c0b3050 2023-07-13T22:30:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.405+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c1f2980 con 0x7fc70c0b3050 2023-07-13T22:30:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.405+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c15f4a0 con 0x7fc70c0b3050 2023-07-13T22:30:49.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.406+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70c1f2ea0 con 0x7fc70c0b3050 2023-07-13T22:30:49.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.407+0000 7fc7097fa700 1 -- 172.21.15.60:0/987978303 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6f8003ba0 con 0x7fc70c0b3050 2023-07-13T22:30:49.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.408+0000 7fc7097fa700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6f006ace0 0x7fc6f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:49.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.408+0000 7fc7097fa700 1 -- 172.21.15.60:0/987978303 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6f8021070 con 0x7fc70c0b3050 2023-07-13T22:30:49.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.408+0000 7fc710f9b700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6f006ace0 0x7fc6f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:49.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.409+0000 7fc710f9b700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6f006ace0 0x7fc6f006d190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc700009b90 tx=0x7fc700008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.413+0000 7fc7097fa700 1 -- 172.21.15.60:0/987978303 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc70c1f2ea0 con 0x7fc70c0b3050 2023-07-13T22:30:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.667+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc70c1f2ea0 con 0x7fc70c0b3050 2023-07-13T22:30:49.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.668+0000 7fc7097fa700 1 -- 172.21.15.60:0/987978303 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc70c1f2ea0 con 0x7fc70c0b3050 2023-07-13T22:30:49.648 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:30:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.670+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6f006ace0 msgr2=0x7fc6f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.670+0000 7fc7131ff700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6f006ace0 0x7fc6f006d190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc700009b90 tx=0x7fc700008040 comp rx=0 tx=0).stop 2023-07-13T22:30:49.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.670+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 msgr2=0x7fc70c1ef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.670+0000 7fc7131ff700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 0x7fc70c1ef600 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009b60 tx=0x7fc6f8004750 comp rx=0 tx=0).stop 2023-07-13T22:30:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.670+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 shutdown_connections 2023-07-13T22:30:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.670+0000 7fc7131ff700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6f006ace0 0x7fc6f006d190 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.670+0000 7fc7131ff700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc70c15c6e0 0x7fc70c15ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.671+0000 7fc7131ff700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc70c0b24b0 0x7fc70c1ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.671+0000 7fc7131ff700 1 --2- 172.21.15.60:0/987978303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70c0b3050 0x7fc70c1ef600 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.671+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 >> 172.21.15.60:0/987978303 conn(0x7fc70c06e680 msgr2=0x7fc70c0b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.671+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 shutdown_connections 2023-07-13T22:30:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:49.671+0000 7fc7131ff700 1 -- 172.21.15.60:0/987978303 wait complete. 2023-07-13T22:30:49.668 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:30:49.669 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:30:49.669 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:30:49.669 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:30:49.669 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:49.670 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:30:49.670 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:30:49.670 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:30:49.670 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:30:49.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:49.671 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:30:49.723 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:30:49.724 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:30:49.724 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:30:49.724 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:49.724 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:30:49.725 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:49.725 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:30:49.725 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:49.725 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:30:49.725 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:30:49.725 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:49.725 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:30:49.725 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:30:49.726 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:49.726 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:30:49.726 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:49.726 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:30:49.726 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41121sec preferred_lft 41121sec 2023-07-13T22:30:49.726 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:30:49.726 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:49.727 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:30:49.727 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:30:49.727 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:30:49.727 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:49.727 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:30:49.727 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:49.727 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:30:49.728 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:30:49.728 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:30:49.728 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:49.728 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:30:49.728 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:30:49.728 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::14e3:bfff:fea8:bf57/64 scope link 2023-07-13T22:30:49.729 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:30:49.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:49.730 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:30:49.801 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:30:49.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:49.804 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:30:49.853 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:30:49.853 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:30:49.853 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:30:49.853 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:30:49.854 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:30:49.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:49.855 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:30:49.855 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:30:49.856 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:30:49.856 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:30:49.856 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:30:49.856 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:30:50.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:50.025 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:30:50.071 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:30:50.071 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:30:50.072 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:30:50.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:50.073 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:30:50.182 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:30:50.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:30:50.183 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:50.229 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:30:50.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:30:50.230 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:50.314 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:30:50.424 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:30:50.424 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:30:50.492 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:30:50.494 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:30:50.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:50.495 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:30:50.542 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:30:50.542 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:30:50.587 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:30:50.657 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:30:50.657 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:30:50.727 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:30:50.728 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:30:50.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:50.730 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:30:50.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:30:50.771+0000 7f92586d65c0 -1 init, newargv = 0x5629e7b15930 newargc=15 2023-07-13T22:30:50.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[127147]: starting ceph client 2023-07-13T22:30:50.759 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[127147]: starting fuse 2023-07-13T22:30:50.780 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:30:50.781 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:30:50.781 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:30:50.781 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:30:50.782 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:30:50.782 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:30:50.782 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:30:50.783 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:30:50.783 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:30:50.783 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:30:50.783 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:30:50.784 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:30:50.784 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:30:50.784 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:30:50.785 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:30:50.785 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:30:50.785 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:30:50.786 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:30:50.786 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:30:50.786 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:30:50.786 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:30:50.787 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:30:50.787 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:30:50.787 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:30:50.788 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:30:50.788 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:30:50.788 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:30:50.789 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:30:50.789 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:30:50.966 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.127147.asok 2023-07-13T22:30:51.144 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.127147.asok 2023-07-13T22:30:51.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:51.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.127147.asok status 2023-07-13T22:30:51.368 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:30:51.369 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:30:51.369 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:30:51.369 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:30:51.369 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:30:51.369 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:30:51.369 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:30:51.369 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "127147", 2023-07-13T22:30:51.370 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:30:51.370 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:30:51.370 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:30:51.370 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:30:51.370 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8422, 2023-07-13T22:30:51.370 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:30:51.370 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:30:51.370 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:30:51.371 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 8422 2023-07-13T22:30:51.371 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:30:51.371 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:30:51.371 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:30:51.371 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:30:51.371 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 814386305 2023-07-13T22:30:51.371 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:30:51.372 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:30:51.372 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:30:51.372 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:30:51.372 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:30:51.372 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 814386305 2023-07-13T22:30:51.372 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:30:51.372 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.8422 192.168.0.1:0/814386305", 2023-07-13T22:30:51.372 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/814386305", 2023-07-13T22:30:51.373 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:30:51.373 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 85, 2023-07-13T22:30:51.373 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 124, 2023-07-13T22:30:51.373 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:30:51.373 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:30:51.373 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:30:51.373 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:30:51.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:51.383 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:30:51.383 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:51.429 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:30:51.430 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:51.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:51.430 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:30:51.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-07-13T22:30:51.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.737+0000 7f3a2f3ff700 1 Processor -- start 2023-07-13T22:30:51.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.737+0000 7f3a2f3ff700 1 -- start start 2023-07-13T22:30:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.738+0000 7f3a2f3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280add00 0x7f3a2806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2f3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a2806a5e0 0x7f3a280b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2f3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a280c8e90 con 0x7f3a280ca9f0 2023-07-13T22:30:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2f3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2806d3b0 con 0x7f3a2806a5e0 2023-07-13T22:30:51.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2f3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280ae0d0 con 0x7f3a280add00 2023-07-13T22:30:51.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2d99c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a2806a5e0 0x7f3a280b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:51.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2d99c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a2806a5e0 0x7f3a280b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50568/0 (socket says 172.21.15.60:50568) 2023-07-13T22:30:51.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2d99c700 1 -- 172.21.15.60:0/1463289320 learned_addr learned my addr 172.21.15.60:0/1463289320 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2d19b700 1 --2- 172.21.15.60:0/1463289320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280add00 0x7f3a2806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2d99c700 1 -- 172.21.15.60:0/1463289320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280add00 msgr2=0x7f3a2806ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2d99c700 1 --2- 172.21.15.60:0/1463289320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280add00 0x7f3a2806ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2d99c700 1 -- 172.21.15.60:0/1463289320 >> v1:172.21.15.105:6789/0 conn(0x7f3a280ca9f0 legacy=0x7f3a28069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.739+0000 7f3a2d99c700 1 -- 172.21.15.60:0/1463289320 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a280c90e0 con 0x7f3a2806a5e0 2023-07-13T22:30:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.740+0000 7f3a2d99c700 1 --2- 172.21.15.60:0/1463289320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a2806a5e0 0x7f3a280b1690 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009e50 tx=0x7f3a2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ae94fb26b42974 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.740+0000 7f3a1ffff700 1 -- 172.21.15.60:0/1463289320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000e070 con 0x7f3a2806a5e0 2023-07-13T22:30:51.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.740+0000 7f3a1ffff700 1 -- 172.21.15.60:0/1463289320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2000b910 con 0x7f3a2806a5e0 2023-07-13T22:30:51.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.740+0000 7f3a1ffff700 1 -- 172.21.15.60:0/1463289320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000bcc0 con 0x7f3a2806a5e0 2023-07-13T22:30:51.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.740+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1463289320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a2806a5e0 msgr2=0x7f3a280b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:51.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.740+0000 7f3a2f3ff700 1 --2- 172.21.15.60:0/1463289320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a2806a5e0 0x7f3a280b1690 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009e50 tx=0x7f3a2000b330 comp rx=0 tx=0).stop 2023-07-13T22:30:51.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.741+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1463289320 shutdown_connections 2023-07-13T22:30:51.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.741+0000 7f3a2f3ff700 1 --2- 172.21.15.60:0/1463289320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a2806a5e0 0x7f3a280b1690 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.741+0000 7f3a2f3ff700 1 --2- 172.21.15.60:0/1463289320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280add00 0x7f3a2806ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.741+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1463289320 >> 172.21.15.60:0/1463289320 conn(0x7f3a280cbf50 msgr2=0x7f3a280ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:51.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.741+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1463289320 shutdown_connections 2023-07-13T22:30:51.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.741+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1463289320 wait complete. 2023-07-13T22:30:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.742+0000 7f3a2f3ff700 1 Processor -- start 2023-07-13T22:30:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.743+0000 7f3a2f3ff700 1 -- start start 2023-07-13T22:30:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.743+0000 7f3a2f3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2806a5e0 0x7f3a280cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.743+0000 7f3a2f3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a280add00 0x7f3a280af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2f3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280ca9f0 0x7f3a28158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2f3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280cd6c0 con 0x7f3a280add00 2023-07-13T22:30:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2f3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280cd9a0 con 0x7f3a2806a5e0 2023-07-13T22:30:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2f3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280cdc80 con 0x7f3a280ca9f0 2023-07-13T22:30:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a280add00 0x7f3a280af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2c99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a280add00 0x7f3a280af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50582/0 (socket says 172.21.15.60:50582) 2023-07-13T22:30:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2c99a700 1 -- 172.21.15.60:0/1651567929 learned_addr learned my addr 172.21.15.60:0/1651567929 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2d99c700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280ca9f0 0x7f3a28158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2d19b700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2806a5e0 0x7f3a280cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:51.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2c99a700 1 -- 172.21.15.60:0/1651567929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280ca9f0 msgr2=0x7f3a28158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:51.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2c99a700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280ca9f0 0x7f3a28158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2c99a700 1 -- 172.21.15.60:0/1651567929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2806a5e0 msgr2=0x7f3a280cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:51.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2c99a700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2806a5e0 0x7f3a280cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2c99a700 1 -- 172.21.15.60:0/1651567929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20009b30 con 0x7f3a280add00 2023-07-13T22:30:51.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.744+0000 7f3a2c99a700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a280add00 0x7f3a280af710 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c4e0 tx=0x7f3a1800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:51.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.745+0000 7f3a1dffb700 1 -- 172.21.15.60:0/1651567929 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180052d0 con 0x7f3a280add00 2023-07-13T22:30:51.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.745+0000 7f3a1dffb700 1 -- 172.21.15.60:0/1651567929 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18005430 con 0x7f3a280add00 2023-07-13T22:30:51.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.745+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a280cdf60 con 0x7f3a280add00 2023-07-13T22:30:51.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.745+0000 7f3a1dffb700 1 -- 172.21.15.60:0/1651567929 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180038c0 con 0x7f3a280add00 2023-07-13T22:30:51.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.745+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a280cd3b0 con 0x7f3a280add00 2023-07-13T22:30:51.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.746+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a28073a90 con 0x7f3a280add00 2023-07-13T22:30:51.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.747+0000 7f3a1dffb700 1 -- 172.21.15.60:0/1651567929 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a18003a20 con 0x7f3a280add00 2023-07-13T22:30:51.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.748+0000 7f3a1dffb700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0c06ace0 0x7f3a0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:51.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.748+0000 7f3a1dffb700 1 -- 172.21.15.60:0/1651567929 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a180888c0 con 0x7f3a280add00 2023-07-13T22:30:51.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.748+0000 7f3a2d19b700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0c06ace0 0x7f3a0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:51.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.749+0000 7f3a2d19b700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0c06ace0 0x7f3a0c06d190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009be0 tx=0x7f3a24009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.752+0000 7f3a1dffb700 1 -- 172.21.15.60:0/1651567929 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3a180538d0 con 0x7f3a280add00 2023-07-13T22:30:51.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.959+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f3a280ab8f0 con 0x7f3a280add00 2023-07-13T22:30:51.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.963+0000 7f3a1dffb700 1 -- 172.21.15.60:0/1651567929 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v55)=0 v55) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3a18056ef0 con 0x7f3a280add00 2023-07-13T22:30:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0c06ace0 msgr2=0x7f3a0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0c06ace0 0x7f3a0c06d190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009be0 tx=0x7f3a24009210 comp rx=0 tx=0).stop 2023-07-13T22:30:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a280add00 msgr2=0x7f3a280af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a280add00 0x7f3a280af710 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c4e0 tx=0x7f3a1800c7f0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 shutdown_connections 2023-07-13T22:30:51.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3a0c06ace0 0x7f3a0c06d190 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3a280ca9f0 0x7f3a28158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a2806a5e0 0x7f3a280cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 --2- 172.21.15.60:0/1651567929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a280add00 0x7f3a280af710 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 >> 172.21.15.60:0/1651567929 conn(0x7f3a280cbf50 msgr2=0x7f3a280732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 shutdown_connections 2023-07-13T22:30:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:51.966+0000 7f3a2f3ff700 1 -- 172.21.15.60:0/1651567929 wait complete. 2023-07-13T22:30:51.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2023-07-13T22:30:52.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.190+0000 7ff74abff700 1 Processor -- start 2023-07-13T22:30:52.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.190+0000 7ff74abff700 1 -- start start 2023-07-13T22:30:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.191+0000 7ff74abff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7440cfd50 0x7ff7440d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.191+0000 7ff74abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 0x7ff7440d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.191+0000 7ff74abff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7440d1870 con 0x7ff7440b8200 2023-07-13T22:30:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.191+0000 7ff74abff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440cf370 con 0x7ff7440cfd50 2023-07-13T22:30:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.191+0000 7ff74abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440d12b0 con 0x7ff7440b1ac0 2023-07-13T22:30:52.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff74919c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 0x7ff7440d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff74899b700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff7440b8200 0x7ff7440cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:46230/0 (socket says 172.21.15.60:46230) 2023-07-13T22:30:52.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff74899b700 1 -- 172.21.15.60:0/2161404044 learned_addr learned my addr 172.21.15.60:0/2161404044 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:52.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff743fff700 1 --2- 172.21.15.60:0/2161404044 >> v2:172.21.15.105:3300/0 conn(0x7ff7440cfd50 0x7ff7440d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff7437fe700 1 -- 172.21.15.60:0/2161404044 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909208352 0 0) 0x7ff7440d1870 con 0x7ff7440b8200 2023-07-13T22:30:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff7437fe700 1 -- 172.21.15.60:0/2161404044 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7340035b0 con 0x7ff7440b8200 2023-07-13T22:30:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff743fff700 1 -- 172.21.15.60:0/2161404044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 msgr2=0x7ff7440d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff743fff700 1 --2- 172.21.15.60:0/2161404044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 0x7ff7440d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff743fff700 1 -- 172.21.15.60:0/2161404044 >> v1:172.21.15.105:6789/0 conn(0x7ff7440b8200 legacy=0x7ff7440cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff743fff700 1 -- 172.21.15.60:0/2161404044 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440d1590 con 0x7ff7440cfd50 2023-07-13T22:30:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.192+0000 7ff74919c700 1 --2- 172.21.15.60:0/2161404044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 0x7ff7440d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff743fff700 1 --2- 172.21.15.60:0/2161404044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 0x7ff7440d23c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff730009840 tx=0x7ff730009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d2bde4541734df server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff7437fe700 1 -- 172.21.15.60:0/2161404044 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730011830 con 0x7ff7440cfd50 2023-07-13T22:30:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff7437fe700 1 -- 172.21.15.60:0/2161404044 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff730011990 con 0x7ff7440cfd50 2023-07-13T22:30:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff7437fe700 1 -- 172.21.15.60:0/2161404044 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730011d20 con 0x7ff7440cfd50 2023-07-13T22:30:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff74abff700 1 -- 172.21.15.60:0/2161404044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 msgr2=0x7ff7440d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff74abff700 1 --2- 172.21.15.60:0/2161404044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 0x7ff7440d23c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff730009840 tx=0x7ff730009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff74abff700 1 -- 172.21.15.60:0/2161404044 shutdown_connections 2023-07-13T22:30:52.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff74abff700 1 --2- 172.21.15.60:0/2161404044 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 0x7ff7440d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff74abff700 1 --2- 172.21.15.60:0/2161404044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 0x7ff7440d23c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff74abff700 1 -- 172.21.15.60:0/2161404044 >> 172.21.15.60:0/2161404044 conn(0x7ff74406b2f0 msgr2=0x7ff74406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:52.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff74abff700 1 -- 172.21.15.60:0/2161404044 shutdown_connections 2023-07-13T22:30:52.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.193+0000 7ff74abff700 1 -- 172.21.15.60:0/2161404044 wait complete. 2023-07-13T22:30:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.195+0000 7ff74abff700 1 Processor -- start 2023-07-13T22:30:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.195+0000 7ff74abff700 1 -- start start 2023-07-13T22:30:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.196+0000 7ff74abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 0x7ff74414d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.196+0000 7ff74abff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7440b8200 0x7ff74414d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.196+0000 7ff74abff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 0x7ff7440c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.196+0000 7ff74abff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74414d640 con 0x7ff7440b8200 2023-07-13T22:30:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.196+0000 7ff74abff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7441598b0 con 0x7ff7440cfd50 2023-07-13T22:30:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.196+0000 7ff74abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440c9d00 con 0x7ff7440b1ac0 2023-07-13T22:30:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff743fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7440b8200 0x7ff74414d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff74899b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 0x7ff74414d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff743fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7440b8200 0x7ff74414d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50596/0 (socket says 172.21.15.60:50596) 2023-07-13T22:30:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff743fff700 1 -- 172.21.15.60:0/3851408440 learned_addr learned my addr 172.21.15.60:0/3851408440 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff74919c700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 0x7ff7440c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff743fff700 1 -- 172.21.15.60:0/3851408440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 msgr2=0x7ff74414d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff743fff700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 0x7ff74414d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff743fff700 1 -- 172.21.15.60:0/3851408440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 msgr2=0x7ff7440c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff743fff700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 0x7ff7440c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff743fff700 1 -- 172.21.15.60:0/3851408440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c009a70 con 0x7ff7440b8200 2023-07-13T22:30:52.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff74919c700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 0x7ff7440c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:52.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.197+0000 7ff743fff700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7440b8200 0x7ff74414d9a0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7ff7300090b0 tx=0x7ff730004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:52.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.198+0000 7ff7417fa700 1 -- 172.21.15.60:0/3851408440 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300036a0 con 0x7ff7440b8200 2023-07-13T22:30:52.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.198+0000 7ff7417fa700 1 -- 172.21.15.60:0/3851408440 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73001f070 con 0x7ff7440b8200 2023-07-13T22:30:52.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.198+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7300094a0 con 0x7ff7440b8200 2023-07-13T22:30:52.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.198+0000 7ff7417fa700 1 -- 172.21.15.60:0/3851408440 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730003a20 con 0x7ff7440b8200 2023-07-13T22:30:52.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.198+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7440ca2c0 con 0x7ff7440b8200 2023-07-13T22:30:52.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.199+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74404e210 con 0x7ff7440b8200 2023-07-13T22:30:52.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.200+0000 7ff7417fa700 1 -- 172.21.15.60:0/3851408440 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff730003b80 con 0x7ff7440b8200 2023-07-13T22:30:52.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.201+0000 7ff7417fa700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72806ace0 0x7ff72806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.201+0000 7ff7417fa700 1 -- 172.21.15.60:0/3851408440 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff730021070 con 0x7ff7440b8200 2023-07-13T22:30:52.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.201+0000 7ff74899b700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72806ace0 0x7ff72806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.202+0000 7ff74899b700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72806ace0 0x7ff72806d190 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff738007fd0 tx=0x7ff738007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:52.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.205+0000 7ff7417fa700 1 -- 172.21.15.60:0/3851408440 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff730055cf0 con 0x7ff7440b8200 2023-07-13T22:30:52.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.410+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7ff7440b4f70 con 0x7ff7440b8200 2023-07-13T22:30:52.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.414+0000 7ff7417fa700 1 -- 172.21.15.60:0/3851408440 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v56)=0 v56) v1 ==== 119+0+0 (secure 0 0 0) 0x7ff7300150b0 con 0x7ff7440b8200 2023-07-13T22:30:52.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72806ace0 msgr2=0x7ff72806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72806ace0 0x7ff72806d190 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff738007fd0 tx=0x7ff738007dc0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7440b8200 msgr2=0x7ff74414d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7440b8200 0x7ff74414d9a0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7ff7300090b0 tx=0x7ff730004750 comp rx=0 tx=0).stop 2023-07-13T22:30:52.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 shutdown_connections 2023-07-13T22:30:52.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff72806ace0 0x7ff72806d190 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7440cfd50 0x7ff7440c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7440b1ac0 0x7ff74414d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 --2- 172.21.15.60:0/3851408440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7440b8200 0x7ff74414d9a0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 >> 172.21.15.60:0/3851408440 conn(0x7ff74406b2f0 msgr2=0x7ff74415ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 shutdown_connections 2023-07-13T22:30:52.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.417+0000 7ff74abff700 1 -- 172.21.15.60:0/3851408440 wait complete. 2023-07-13T22:30:52.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2023-07-13T22:30:52.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.644+0000 7f8b509ff700 1 Processor -- start 2023-07-13T22:30:52.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.644+0000 7f8b509ff700 1 -- start start 2023-07-13T22:30:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.645+0000 7f8b509ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0b1ac0 0x7f8b4c0d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.645+0000 7f8b509ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c0d1870 con 0x7f8b4c0b8200 2023-07-13T22:30:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.645+0000 7f8b509ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c0cf370 con 0x7f8b4c0cfd50 2023-07-13T22:30:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b509ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0d12b0 con 0x7f8b4c0b1ac0 2023-07-13T22:30:52.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b4a59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8b4c0b8200 0x7f8b4c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58212/0 (socket says 172.21.15.60:58212) 2023-07-13T22:30:52.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b4a59c700 1 -- 172.21.15.60:0/1381250583 learned_addr learned my addr 172.21.15.60:0/1381250583 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:52.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b4ad9d700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0b1ac0 0x7f8b4c0d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686209967 0 0) 0x7f8b4c0d1870 con 0x7f8b4c0b8200 2023-07-13T22:30:52.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b380035b0 con 0x7f8b4c0b8200 2023-07-13T22:30:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588738915 0 0) 0x7f8b4c0cf370 con 0x7f8b4c0cfd50 2023-07-13T22:30:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b4c0d1870 con 0x7f8b4c0cfd50 2023-07-13T22:30:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103863473 0 0) 0x7f8b380035b0 con 0x7f8b4c0b8200 2023-07-13T22:30:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0b1ac0 msgr2=0x7f8b4c0d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0b1ac0 0x7f8b4c0d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 >> v1:172.21.15.105:6789/0 conn(0x7f8b4c0cfd50 legacy=0x7f8b4c0d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c0d1590 con 0x7f8b4c0b8200 2023-07-13T22:30:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.646+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f8b3c001670 con 0x7f8b4c0b8200 2023-07-13T22:30:52.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 >> v1:172.21.15.60:6789/0 conn(0x7f8b4c0b8200 legacy=0x7f8b4c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b49d9b700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0b1ac0 0x7f8b38003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b49d9b700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b38004120 0x7f8b38004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b49d9b700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b38004d90 0x7f8b38005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0d12b0 con 0x7f8b38004d90 2023-07-13T22:30:52.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3800bac0 con 0x7f8b38004120 2023-07-13T22:30:52.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3800bc80 con 0x7f8b4c0b1ac0 2023-07-13T22:30:52.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b4ad9d700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b38004d90 0x7f8b38005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b4a59c700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0b1ac0 0x7f8b38003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b43fff700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b38004120 0x7f8b38004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b4a59c700 1 -- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b38004120 msgr2=0x7f8b38004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:52.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b4a59c700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b38004120 0x7f8b38004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b4a59c700 1 -- 172.21.15.60:0/1381250583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b38004d90 msgr2=0x7f8b38005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:52.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b4a59c700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b38004d90 0x7f8b38005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b4a59c700 1 -- 172.21.15.60:0/1381250583 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380035b0 con 0x7f8b4c0b1ac0 2023-07-13T22:30:52.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.647+0000 7f8b4ad9d700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b38004d90 0x7f8b38005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:52.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.648+0000 7f8b4a59c700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0b1ac0 0x7f8b38003bc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c001eb0 tx=0x7f8b38005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61348af27c4bed8d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:52.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.648+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00cd20 con 0x7f8b4c0b1ac0 2023-07-13T22:30:52.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.648+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c005960 con 0x7f8b4c0b1ac0 2023-07-13T22:30:52.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.648+0000 7f8b49d9b700 1 -- 172.21.15.60:0/1381250583 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00cea0 con 0x7f8b4c0b1ac0 2023-07-13T22:30:52.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.648+0000 7f8b509ff700 1 -- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0b1ac0 msgr2=0x7f8b38003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.648+0000 7f8b509ff700 1 --2- 172.21.15.60:0/1381250583 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0b1ac0 0x7f8b38003bc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c001eb0 tx=0x7f8b38005880 comp rx=0 tx=0).stop 2023-07-13T22:30:52.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.648+0000 7f8b4ad9d700 1 -- 172.21.15.60:0/1381250583 reap_dead start 2023-07-13T22:30:52.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.648+0000 7f8b509ff700 1 -- 172.21.15.60:0/1381250583 shutdown_connections 2023-07-13T22:30:52.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.648+0000 7f8b509ff700 1 -- 172.21.15.60:0/1381250583 >> 172.21.15.60:0/1381250583 conn(0x7f8b4c06b2f0 msgr2=0x7f8b4c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:52.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.649+0000 7f8b509ff700 1 -- 172.21.15.60:0/1381250583 shutdown_connections 2023-07-13T22:30:52.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.649+0000 7f8b509ff700 1 -- 172.21.15.60:0/1381250583 wait complete. 2023-07-13T22:30:52.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.650+0000 7f8b509ff700 1 Processor -- start 2023-07-13T22:30:52.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.650+0000 7f8b509ff700 1 -- start start 2023-07-13T22:30:52.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.651+0000 7f8b509ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c0b1ac0 0x7f8b4c156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.651+0000 7f8b509ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c15cf70 0x7f8b4c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.651+0000 7f8b509ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c15f690 0x7f8b4c0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.651+0000 7f8b509ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c1f2bd0 con 0x7f8b4c15f690 2023-07-13T22:30:52.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.651+0000 7f8b509ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c1f1fe0 con 0x7f8b4c0b1ac0 2023-07-13T22:30:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.651+0000 7f8b509ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c1f22a0 con 0x7f8b4c15cf70 2023-07-13T22:30:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c15f690 0x7f8b4c0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c15f690 0x7f8b4c0d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50614/0 (socket says 172.21.15.60:50614) 2023-07-13T22:30:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4ad9d700 1 -- 172.21.15.60:0/3147593958 learned_addr learned my addr 172.21.15.60:0/3147593958 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4a59c700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c0b1ac0 0x7f8b4c156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b43fff700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c15cf70 0x7f8b4c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4ad9d700 1 -- 172.21.15.60:0/3147593958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c15cf70 msgr2=0x7f8b4c15f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4ad9d700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c15cf70 0x7f8b4c15f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4ad9d700 1 -- 172.21.15.60:0/3147593958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c0b1ac0 msgr2=0x7f8b4c156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4ad9d700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c0b1ac0 0x7f8b4c156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4ad9d700 1 -- 172.21.15.60:0/3147593958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c1f2580 con 0x7f8b4c15f690 2023-07-13T22:30:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.652+0000 7f8b4ad9d700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c15f690 0x7f8b4c0d2a50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0c9cb0 tx=0x7f8b4400f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.653+0000 7f8b42ffd700 1 -- 172.21.15.60:0/3147593958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400e040 con 0x7f8b4c15f690 2023-07-13T22:30:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.653+0000 7f8b42ffd700 1 -- 172.21.15.60:0/3147593958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4400f040 con 0x7f8b4c15f690 2023-07-13T22:30:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.653+0000 7f8b42ffd700 1 -- 172.21.15.60:0/3147593958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44003a20 con 0x7f8b4c15f690 2023-07-13T22:30:52.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.653+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c1f2860 con 0x7f8b4c15f690 2023-07-13T22:30:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.653+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c1f5390 con 0x7f8b4c15f690 2023-07-13T22:30:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.654+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c1f2d40 con 0x7f8b4c15f690 2023-07-13T22:30:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.655+0000 7f8b42ffd700 1 -- 172.21.15.60:0/3147593958 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b44015430 con 0x7f8b4c15f690 2023-07-13T22:30:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.656+0000 7f8b42ffd700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b2c06ace0 0x7f8b2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:52.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.657+0000 7f8b42ffd700 1 -- 172.21.15.60:0/3147593958 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b44089990 con 0x7f8b4c15f690 2023-07-13T22:30:52.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.657+0000 7f8b4a59c700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b2c06ace0 0x7f8b2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:52.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.657+0000 7f8b4a59c700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b2c06ace0 0x7f8b2c06d190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c15fca0 tx=0x7f8b3c007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:52.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.661+0000 7f8b42ffd700 1 -- 172.21.15.60:0/3147593958 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8b44054a30 con 0x7f8b4c15f690 2023-07-13T22:30:52.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.869+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7f8b4c04e210 con 0x7f8b4c15f690 2023-07-13T22:30:52.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.873+0000 7f8b42ffd700 1 -- 172.21.15.60:0/3147593958 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v57)=0 v57) v1 ==== 128+0+0 (secure 0 0 0) 0x7f8b44058050 con 0x7f8b4c15f690 2023-07-13T22:30:52.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b2c06ace0 msgr2=0x7f8b2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b2c06ace0 0x7f8b2c06d190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c15fca0 tx=0x7f8b3c007f90 comp rx=0 tx=0).stop 2023-07-13T22:30:52.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c15f690 msgr2=0x7f8b4c0d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:52.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c15f690 0x7f8b4c0d2a50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0c9cb0 tx=0x7f8b4400f6c0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 shutdown_connections 2023-07-13T22:30:52.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b2c06ace0 0x7f8b2c06d190 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c15f690 0x7f8b4c0d2a50 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c0b1ac0 0x7f8b4c156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 --2- 172.21.15.60:0/3147593958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c15cf70 0x7f8b4c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:52.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 >> 172.21.15.60:0/3147593958 conn(0x7f8b4c06b2f0 msgr2=0x7f8b4c0b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:52.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 shutdown_connections 2023-07-13T22:30:52.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:52.876+0000 7f8b509ff700 1 -- 172.21.15.60:0/3147593958 wait complete. 2023-07-13T22:30:52.875 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-13T22:30:53.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.100+0000 7f210f1ff700 1 Processor -- start 2023-07-13T22:30:53.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.100+0000 7f210f1ff700 1 -- start start 2023-07-13T22:30:53.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210f1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21080b8200 0x7f21080cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 0x7f21080cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210f1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21080d1870 con 0x7f21080b1ac0 2023-07-13T22:30:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210f1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21080cf370 con 0x7f21080b8200 2023-07-13T22:30:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21080d12b0 con 0x7f21080cede0 2023-07-13T22:30:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210cf9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21080b8200 0x7f21080cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210cf9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21080b8200 0x7f21080cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50624/0 (socket says 172.21.15.60:50624) 2023-07-13T22:30:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210cf9b700 1 -- 172.21.15.60:0/3398882694 learned_addr learned my addr 172.21.15.60:0/3398882694 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210d79c700 1 --2- 172.21.15.60:0/3398882694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 0x7f21080cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210cf9b700 1 -- 172.21.15.60:0/3398882694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 msgr2=0x7f21080cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:53.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210cf9b700 1 --2- 172.21.15.60:0/3398882694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 0x7f21080cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210cf9b700 1 -- 172.21.15.60:0/3398882694 >> v1:172.21.15.105:6789/0 conn(0x7f21080b1ac0 legacy=0x7f210815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.102+0000 7f210cf9b700 1 -- 172.21.15.60:0/3398882694 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21080d1590 con 0x7f21080b8200 2023-07-13T22:30:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.103+0000 7f210cf9b700 1 --2- 172.21.15.60:0/3398882694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b8200 0x7f21080cf640 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0098f0 tx=0x7f20fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f1ad845ba85074 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.103+0000 7f21077fe700 1 -- 172.21.15.60:0/3398882694 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00b8a0 con 0x7f21080b8200 2023-07-13T22:30:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.103+0000 7f21077fe700 1 -- 172.21.15.60:0/3398882694 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc00ba00 con 0x7f21080b8200 2023-07-13T22:30:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.103+0000 7f21077fe700 1 -- 172.21.15.60:0/3398882694 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00bd90 con 0x7f21080b8200 2023-07-13T22:30:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.103+0000 7f210f1ff700 1 -- 172.21.15.60:0/3398882694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b8200 msgr2=0x7f21080cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.103+0000 7f210f1ff700 1 --2- 172.21.15.60:0/3398882694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b8200 0x7f21080cf640 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0098f0 tx=0x7f20fc009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.104+0000 7f210f1ff700 1 -- 172.21.15.60:0/3398882694 shutdown_connections 2023-07-13T22:30:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.104+0000 7f210f1ff700 1 --2- 172.21.15.60:0/3398882694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 0x7f21080cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.104+0000 7f210f1ff700 1 --2- 172.21.15.60:0/3398882694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b8200 0x7f21080cf640 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.104+0000 7f210f1ff700 1 -- 172.21.15.60:0/3398882694 >> 172.21.15.60:0/3398882694 conn(0x7f210806b2f0 msgr2=0x7f210806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.104+0000 7f210f1ff700 1 -- 172.21.15.60:0/3398882694 shutdown_connections 2023-07-13T22:30:53.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.104+0000 7f210f1ff700 1 -- 172.21.15.60:0/3398882694 wait complete. 2023-07-13T22:30:53.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.105+0000 7f210f1ff700 1 Processor -- start 2023-07-13T22:30:53.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.106+0000 7f210f1ff700 1 -- start start 2023-07-13T22:30:53.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210f1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b1ac0 0x7f21081e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210f1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b8200 0x7f21080a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210f1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 0x7f21081e2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210f1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21081e2d90 con 0x7f21080b1ac0 2023-07-13T22:30:53.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210f1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21081e47d0 con 0x7f21080b8200 2023-07-13T22:30:53.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210f1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21080c9ea0 con 0x7f21080cede0 2023-07-13T22:30:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210cf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b1ac0 0x7f21081e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210cf9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b1ac0 0x7f21081e3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50638/0 (socket says 172.21.15.60:50638) 2023-07-13T22:30:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210cf9b700 1 -- 172.21.15.60:0/1222692582 learned_addr learned my addr 172.21.15.60:0/1222692582 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:53.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f2107fff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b8200 0x7f21080a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210d79c700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 0x7f21081e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210cf9b700 1 -- 172.21.15.60:0/1222692582 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 msgr2=0x7f21081e2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210cf9b700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 0x7f21081e2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210cf9b700 1 -- 172.21.15.60:0/1222692582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b8200 msgr2=0x7f21080a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210cf9b700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b8200 0x7f21080a9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f210cf9b700 1 -- 172.21.15.60:0/1222692582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc009550 con 0x7f21080b1ac0 2023-07-13T22:30:53.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.107+0000 7f2107fff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b8200 0x7f21080a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:53.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.108+0000 7f210cf9b700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b1ac0 0x7f21081e3b60 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000c00 tx=0x7f20fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:53.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.108+0000 7f21057fa700 1 -- 172.21.15.60:0/1222692582 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc004380 con 0x7f21080b1ac0 2023-07-13T22:30:53.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.108+0000 7f21057fa700 1 -- 172.21.15.60:0/1222692582 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc01f070 con 0x7f21080b1ac0 2023-07-13T22:30:53.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.108+0000 7f21057fa700 1 -- 172.21.15.60:0/1222692582 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0036a0 con 0x7f21080b1ac0 2023-07-13T22:30:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.108+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21080ca180 con 0x7f21080b1ac0 2023-07-13T22:30:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.108+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21080ca740 con 0x7f21080b1ac0 2023-07-13T22:30:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.109+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210804e210 con 0x7f21080b1ac0 2023-07-13T22:30:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.110+0000 7f21057fa700 1 -- 172.21.15.60:0/1222692582 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20fc021030 con 0x7f21080b1ac0 2023-07-13T22:30:53.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.111+0000 7f21057fa700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20ec06ace0 0x7f20ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.111+0000 7f21057fa700 1 -- 172.21.15.60:0/1222692582 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f20fc08a7f0 con 0x7f21080b1ac0 2023-07-13T22:30:53.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.112+0000 7f2107fff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20ec06ace0 0x7f20ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.112+0000 7f2107fff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20ec06ace0 0x7f20ec06d190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f21081e3690 tx=0x7f20f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:53.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.116+0000 7f21057fa700 1 -- 172.21.15.60:0/1222692582 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f20fc055890 con 0x7f21080b1ac0 2023-07-13T22:30:53.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.324+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7f21080b4f70 con 0x7f21080b1ac0 2023-07-13T22:30:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.324+0000 7f21057fa700 1 -- 172.21.15.60:0/1222692582 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v57) v1 ==== 106+0+4 (secure 0 0 0) 0x7f20fc058eb0 con 0x7f21080b1ac0 2023-07-13T22:30:53.305 INFO:teuthology.orchestra.run.smithi060.stdout:100 2023-07-13T22:30:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20ec06ace0 msgr2=0x7f20ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20ec06ace0 0x7f20ec06d190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f21081e3690 tx=0x7f20f4009040 comp rx=0 tx=0).stop 2023-07-13T22:30:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b1ac0 msgr2=0x7f21081e3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b1ac0 0x7f21081e3b60 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f20fc000c00 tx=0x7f20fc004750 comp rx=0 tx=0).stop 2023-07-13T22:30:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 shutdown_connections 2023-07-13T22:30:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f20ec06ace0 0x7f20ec06d190 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21080b1ac0 0x7f21081e3b60 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080cede0 0x7f21081e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 --2- 172.21.15.60:0/1222692582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b8200 0x7f21080a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.326+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 >> 172.21.15.60:0/1222692582 conn(0x7f210806b2f0 msgr2=0x7f21080d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.327+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 shutdown_connections 2023-07-13T22:30:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.327+0000 7f210f1ff700 1 -- 172.21.15.60:0/1222692582 wait complete. 2023-07-13T22:30:53.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2023-07-13T22:30:53.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.551+0000 7fc6b1fff700 1 Processor -- start 2023-07-13T22:30:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.551+0000 7fc6b1fff700 1 -- start start 2023-07-13T22:30:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.552+0000 7fc6b1fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc6ac070b30 0x7fc6ac0caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.552+0000 7fc6b1fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac0ce770 0x7fc6ac0b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.552+0000 7fc6b1fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ac06bf40 con 0x7fc6ac15c460 2023-07-13T22:30:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.552+0000 7fc6b1fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ac06b9f0 con 0x7fc6ac070b30 2023-07-13T22:30:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.552+0000 7fc6b1fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac06c750 con 0x7fc6ac0ce770 2023-07-13T22:30:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.552+0000 7fc6aaffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc6ac070b30 0x7fc6ac0caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.553+0000 7fc6aaffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc6ac070b30 0x7fc6ac0caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50648/0 (socket says 172.21.15.60:50648) 2023-07-13T22:30:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.553+0000 7fc6aaffd700 1 -- 172.21.15.60:0/1630652326 learned_addr learned my addr 172.21.15.60:0/1630652326 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.553+0000 7fc6abfff700 1 --2- 172.21.15.60:0/1630652326 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac0ce770 0x7fc6ac0b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.553+0000 7fc6aaffd700 1 -- 172.21.15.60:0/1630652326 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac0ce770 msgr2=0x7fc6ac0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.553+0000 7fc6aaffd700 1 --2- 172.21.15.60:0/1630652326 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac0ce770 0x7fc6ac0b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.553+0000 7fc6aaffd700 1 -- 172.21.15.60:0/1630652326 >> v1:172.21.15.105:6789/0 conn(0x7fc6ac15c460 legacy=0x7fc6ac06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.553+0000 7fc6aaffd700 1 -- 172.21.15.60:0/1630652326 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac0cebf0 con 0x7fc6ac070b30 2023-07-13T22:30:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.553+0000 7fc6aaffd700 1 --2- 172.21.15.60:0/1630652326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac070b30 0x7fc6ac0caf80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00098f0 tx=0x7fc6a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d943a4f26c4cfe server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.555+0000 7fc6aa7fc700 1 -- 172.21.15.60:0/1630652326 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011830 con 0x7fc6ac070b30 2023-07-13T22:30:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.555+0000 7fc6aa7fc700 1 -- 172.21.15.60:0/1630652326 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a0011990 con 0x7fc6ac070b30 2023-07-13T22:30:53.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.555+0000 7fc6aa7fc700 1 -- 172.21.15.60:0/1630652326 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011d20 con 0x7fc6ac070b30 2023-07-13T22:30:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.556+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1630652326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac070b30 msgr2=0x7fc6ac0caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.556+0000 7fc6b1fff700 1 --2- 172.21.15.60:0/1630652326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac070b30 0x7fc6ac0caf80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00098f0 tx=0x7fc6a0009c00 comp rx=0 tx=0).stop 2023-07-13T22:30:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.556+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1630652326 shutdown_connections 2023-07-13T22:30:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.556+0000 7fc6b1fff700 1 --2- 172.21.15.60:0/1630652326 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac0ce770 0x7fc6ac0b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.556+0000 7fc6b1fff700 1 --2- 172.21.15.60:0/1630652326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac070b30 0x7fc6ac0caf80 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.556+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1630652326 >> 172.21.15.60:0/1630652326 conn(0x7fc6ac06e580 msgr2=0x7fc6ac06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.556+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1630652326 shutdown_connections 2023-07-13T22:30:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.556+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1630652326 wait complete. 2023-07-13T22:30:53.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.558+0000 7fc6b1fff700 1 Processor -- start 2023-07-13T22:30:53.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.558+0000 7fc6b1fff700 1 -- start start 2023-07-13T22:30:53.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.559+0000 7fc6b1fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac070b30 0x7fc6ac0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.559+0000 7fc6b1fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac0ce770 0x7fc6ac151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.559+0000 7fc6b1fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6ac15c460 0x7fc6ac0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.559+0000 7fc6b1fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac15a6f0 con 0x7fc6ac0ce770 2023-07-13T22:30:53.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.559+0000 7fc6b1fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac0c7e90 con 0x7fc6ac15c460 2023-07-13T22:30:53.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.559+0000 7fc6b1fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac15a990 con 0x7fc6ac070b30 2023-07-13T22:30:53.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.559+0000 7fc6aaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac0ce770 0x7fc6ac151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.559+0000 7fc6aaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac0ce770 0x7fc6ac151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50658/0 (socket says 172.21.15.60:50658) 2023-07-13T22:30:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.559+0000 7fc6aaffd700 1 -- 172.21.15.60:0/1950976862 learned_addr learned my addr 172.21.15.60:0/1950976862 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6ab7fe700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac070b30 0x7fc6ac0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6abfff700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6ac15c460 0x7fc6ac0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6aaffd700 1 -- 172.21.15.60:0/1950976862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac070b30 msgr2=0x7fc6ac0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6aaffd700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac070b30 0x7fc6ac0b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6aaffd700 1 -- 172.21.15.60:0/1950976862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6ac15c460 msgr2=0x7fc6ac0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6aaffd700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6ac15c460 0x7fc6ac0ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6aaffd700 1 -- 172.21.15.60:0/1950976862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a0009550 con 0x7fc6ac0ce770 2023-07-13T22:30:53.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6abfff700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6ac15c460 0x7fc6ac0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:30:53.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6aaffd700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac0ce770 0x7fc6ac151760 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009c40 tx=0x7fc6a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:53.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc693fff700 1 -- 172.21.15.60:0/1950976862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a00036a0 con 0x7fc6ac0ce770 2023-07-13T22:30:53.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc693fff700 1 -- 172.21.15.60:0/1950976862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a001e070 con 0x7fc6ac0ce770 2023-07-13T22:30:53.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc693fff700 1 -- 172.21.15.60:0/1950976862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0003a60 con 0x7fc6ac0ce770 2023-07-13T22:30:53.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.560+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac0c8170 con 0x7fc6ac0ce770 2023-07-13T22:30:53.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.561+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac0c7a00 con 0x7fc6ac0ce770 2023-07-13T22:30:53.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.562+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac15aad0 con 0x7fc6ac0ce770 2023-07-13T22:30:53.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.562+0000 7fc693fff700 1 -- 172.21.15.60:0/1950976862 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6a0003bc0 con 0x7fc6ac0ce770 2023-07-13T22:30:53.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.563+0000 7fc693fff700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc69806ace0 0x7fc69806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.563+0000 7fc693fff700 1 -- 172.21.15.60:0/1950976862 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6a008a460 con 0x7fc6ac0ce770 2023-07-13T22:30:53.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.564+0000 7fc6ab7fe700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc69806ace0 0x7fc69806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.564+0000 7fc6ab7fe700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc69806ace0 0x7fc69806d190 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000f60 tx=0x7fc69c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.568+0000 7fc693fff700 1 -- 172.21.15.60:0/1950976862 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc6ac15aad0 con 0x7fc6ac0ce770 2023-07-13T22:30:53.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.774+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7fc6ac15aad0 con 0x7fc6ac0ce770 2023-07-13T22:30:53.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.778+0000 7fc693fff700 1 -- 172.21.15.60:0/1950976862 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v58)=0 v58) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc6ac15aad0 con 0x7fc6ac0ce770 2023-07-13T22:30:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.780+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc69806ace0 msgr2=0x7fc69806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.780+0000 7fc6b1fff700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc69806ace0 0x7fc69806d190 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc69c000f60 tx=0x7fc69c009040 comp rx=0 tx=0).stop 2023-07-13T22:30:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.780+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac0ce770 msgr2=0x7fc6ac151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.780+0000 7fc6b1fff700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac0ce770 0x7fc6ac151760 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009c40 tx=0x7fc6a0004750 comp rx=0 tx=0).stop 2023-07-13T22:30:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.781+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 shutdown_connections 2023-07-13T22:30:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.781+0000 7fc6b1fff700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc69806ace0 0x7fc69806d190 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.781+0000 7fc6b1fff700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6ac15c460 0x7fc6ac0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.781+0000 7fc6b1fff700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6ac070b30 0x7fc6ac0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.781+0000 7fc6b1fff700 1 --2- 172.21.15.60:0/1950976862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6ac0ce770 0x7fc6ac151760 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.781+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 >> 172.21.15.60:0/1950976862 conn(0x7fc6ac06e580 msgr2=0x7fc6ac0b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.781+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 shutdown_connections 2023-07-13T22:30:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:53.781+0000 7fc6b1fff700 1 -- 172.21.15.60:0/1950976862 wait complete. 2023-07-13T22:30:53.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2023-07-13T22:30:53.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.006+0000 7fc19d5ff700 1 Processor -- start 2023-07-13T22:30:53.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.006+0000 7fc19d5ff700 1 -- start start 2023-07-13T22:30:53.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.007+0000 7fc19d5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1980b8200 0x7fc1980cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.007+0000 7fc19d5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1980cede0 0x7fc1980cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.007+0000 7fc19d5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1980d1870 con 0x7fc1980b1ac0 2023-07-13T22:30:53.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.007+0000 7fc19d5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1980cf370 con 0x7fc1980cede0 2023-07-13T22:30:53.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.007+0000 7fc19d5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1980d12b0 con 0x7fc1980b8200 2023-07-13T22:30:53.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.007+0000 7fc1977fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1980cede0 0x7fc1980cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.007+0000 7fc1977fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1980cede0 0x7fc1980cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50670/0 (socket says 172.21.15.60:50670) 2023-07-13T22:30:53.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.007+0000 7fc1977fe700 1 -- 172.21.15.60:0/1162915722 learned_addr learned my addr 172.21.15.60:0/1162915722 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:53.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.008+0000 7fc196ffd700 1 --2- 172.21.15.60:0/1162915722 >> v2:172.21.15.105:3300/0 conn(0x7fc1980b8200 0x7fc1980cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:53.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.008+0000 7fc1977fe700 1 -- 172.21.15.60:0/1162915722 >> v2:172.21.15.105:3300/0 conn(0x7fc1980b8200 msgr2=0x7fc1980cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:53.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.008+0000 7fc1977fe700 1 --2- 172.21.15.60:0/1162915722 >> v2:172.21.15.105:3300/0 conn(0x7fc1980b8200 0x7fc1980cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.008+0000 7fc1977fe700 1 -- 172.21.15.60:0/1162915722 >> v1:172.21.15.105:6789/0 conn(0x7fc1980b1ac0 legacy=0x7fc19815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:53.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.008+0000 7fc1977fe700 1 -- 172.21.15.60:0/1162915722 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980d1590 con 0x7fc1980cede0 2023-07-13T22:30:53.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.008+0000 7fc1977fe700 1 --2- 172.21.15.60:0/1162915722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 0x7fc1980cfb80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fc1980b2660 tx=0x7fc19000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2366e509e34cbc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:53.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.008+0000 7fc195ffb700 1 -- 172.21.15.60:0/1162915722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000e070 con 0x7fc1980cede0 2023-07-13T22:30:53.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.008+0000 7fc195ffb700 1 -- 172.21.15.60:0/1162915722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19000b9c0 con 0x7fc1980cede0 2023-07-13T22:30:53.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.009+0000 7fc195ffb700 1 -- 172.21.15.60:0/1162915722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000bd70 con 0x7fc1980cede0 2023-07-13T22:30:53.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.009+0000 7fc19d5ff700 1 -- 172.21.15.60:0/1162915722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 msgr2=0x7fc1980cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:53.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.009+0000 7fc19d5ff700 1 --2- 172.21.15.60:0/1162915722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 0x7fc1980cfb80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fc1980b2660 tx=0x7fc19000b330 comp rx=0 tx=0).stop 2023-07-13T22:30:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.009+0000 7fc19d5ff700 1 -- 172.21.15.60:0/1162915722 shutdown_connections 2023-07-13T22:30:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.009+0000 7fc19d5ff700 1 --2- 172.21.15.60:0/1162915722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 0x7fc1980cfb80 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.009+0000 7fc19d5ff700 1 --2- 172.21.15.60:0/1162915722 >> v2:172.21.15.105:3300/0 conn(0x7fc1980b8200 0x7fc1980cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.009+0000 7fc19d5ff700 1 -- 172.21.15.60:0/1162915722 >> 172.21.15.60:0/1162915722 conn(0x7fc19806b2f0 msgr2=0x7fc19806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.009+0000 7fc19d5ff700 1 -- 172.21.15.60:0/1162915722 shutdown_connections 2023-07-13T22:30:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.009+0000 7fc19d5ff700 1 -- 172.21.15.60:0/1162915722 wait complete. 2023-07-13T22:30:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.011+0000 7fc19d5ff700 1 Processor -- start 2023-07-13T22:30:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.011+0000 7fc19d5ff700 1 -- start start 2023-07-13T22:30:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.012+0000 7fc19d5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1980b1ac0 0x7fc198155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.012+0000 7fc19d5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1980b8200 0x7fc198159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.012+0000 7fc19d5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 0x7fc1980cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.012+0000 7fc19d5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981567e0 con 0x7fc1980cede0 2023-07-13T22:30:54.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.012+0000 7fc19d5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980cc3b0 con 0x7fc1980b1ac0 2023-07-13T22:30:54.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.012+0000 7fc19d5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980cb7a0 con 0x7fc1980b8200 2023-07-13T22:30:54.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.012+0000 7fc1977fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 0x7fc1980cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.012+0000 7fc1977fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 0x7fc1980cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50674/0 (socket says 172.21.15.60:50674) 2023-07-13T22:30:54.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.012+0000 7fc1977fe700 1 -- 172.21.15.60:0/111399359 learned_addr learned my addr 172.21.15.60:0/111399359 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:54.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc196ffd700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1980b1ac0 0x7fc198155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc1967fc700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1980b8200 0x7fc198159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc1977fe700 1 -- 172.21.15.60:0/111399359 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1980b8200 msgr2=0x7fc198159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc1977fe700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1980b8200 0x7fc198159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc1977fe700 1 -- 172.21.15.60:0/111399359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1980b1ac0 msgr2=0x7fc198155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc1977fe700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1980b1ac0 0x7fc198155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc1977fe700 1 -- 172.21.15.60:0/111399359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc190009b30 con 0x7fc1980cede0 2023-07-13T22:30:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc1977fe700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 0x7fc1980cc840 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fc19000b730 tx=0x7fc190004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc17f7fe700 1 -- 172.21.15.60:0/111399359 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900099b0 con 0x7fc1980cede0 2023-07-13T22:30:54.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc17f7fe700 1 -- 172.21.15.60:0/111399359 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1900092e0 con 0x7fc1980cede0 2023-07-13T22:30:54.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc17f7fe700 1 -- 172.21.15.60:0/111399359 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19001c610 con 0x7fc1980cede0 2023-07-13T22:30:54.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.013+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1980cba80 con 0x7fc1980cede0 2023-07-13T22:30:54.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.014+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1980cc040 con 0x7fc1980cede0 2023-07-13T22:30:54.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.015+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19804e210 con 0x7fc1980cede0 2023-07-13T22:30:54.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.016+0000 7fc17f7fe700 1 -- 172.21.15.60:0/111399359 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc19001c770 con 0x7fc1980cede0 2023-07-13T22:30:54.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.016+0000 7fc17f7fe700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc18406ace0 0x7fc18406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.016+0000 7fc17f7fe700 1 -- 172.21.15.60:0/111399359 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc19008ad30 con 0x7fc1980cede0 2023-07-13T22:30:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.017+0000 7fc196ffd700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc18406ace0 0x7fc18406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.017+0000 7fc196ffd700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc18406ace0 0x7fc18406d190 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc188009930 tx=0x7fc188008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.021+0000 7fc17f7fe700 1 -- 172.21.15.60:0/111399359 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc190055de0 con 0x7fc1980cede0 2023-07-13T22:30:54.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.230+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) v1 -- 0x7fc1980cc4f0 con 0x7fc1980cede0 2023-07-13T22:30:54.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.230+0000 7fc17f7fe700 1 -- 172.21.15.60:0/111399359 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v58) v1 ==== 106+0+8 (secure 0 0 0) 0x7fc190059400 con 0x7fc1980cede0 2023-07-13T22:30:54.211 INFO:teuthology.orchestra.run.smithi060.stdout:1048576 2023-07-13T22:30:54.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc18406ace0 msgr2=0x7fc18406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc18406ace0 0x7fc18406d190 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc188009930 tx=0x7fc188008040 comp rx=0 tx=0).stop 2023-07-13T22:30:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 msgr2=0x7fc1980cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 0x7fc1980cc840 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fc19000b730 tx=0x7fc190004ad0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 shutdown_connections 2023-07-13T22:30:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc18406ace0 0x7fc18406d190 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1980cede0 0x7fc1980cc840 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1980b1ac0 0x7fc198155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 --2- 172.21.15.60:0/111399359 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1980b8200 0x7fc198159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 >> 172.21.15.60:0/111399359 conn(0x7fc19806b2f0 msgr2=0x7fc1980cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 shutdown_connections 2023-07-13T22:30:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.233+0000 7fc19d5ff700 1 -- 172.21.15.60:0/111399359 wait complete. 2023-07-13T22:30:54.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-07-13T22:30:54.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.457+0000 7f304a1ff700 1 Processor -- start 2023-07-13T22:30:54.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.458+0000 7f304a1ff700 1 -- start start 2023-07-13T22:30:54.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f304a1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f30440add00 0x7f304406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f304a1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30440ca9f0 0x7f3044069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f304a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f304406a410 0x7f30440703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f304a1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304406d3b0 con 0x7f30440ca9f0 2023-07-13T22:30:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f304a1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304406a870 con 0x7f30440add00 2023-07-13T22:30:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f304a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30440c8e90 con 0x7f304406a410 2023-07-13T22:30:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f3042ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30440ca9f0 0x7f3044069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f3042ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30440ca9f0 0x7f3044069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50678/0 (socket says 172.21.15.60:50678) 2023-07-13T22:30:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f3042ffd700 1 -- 172.21.15.60:0/293832343 learned_addr learned my addr 172.21.15.60:0/293832343 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f3043fff700 1 --2- 172.21.15.60:0/293832343 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f304406a410 0x7f30440703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.459+0000 7f30437fe700 1 --2- 172.21.15.60:0/293832343 >> v2:172.21.15.105:3300/0 conn(0x7f30440add00 0x7f304406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.460+0000 7f3042ffd700 1 -- 172.21.15.60:0/293832343 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f304406a410 msgr2=0x7f30440703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.460+0000 7f3042ffd700 1 --2- 172.21.15.60:0/293832343 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f304406a410 0x7f30440703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.460+0000 7f3042ffd700 1 -- 172.21.15.60:0/293832343 >> v2:172.21.15.105:3300/0 conn(0x7f30440add00 msgr2=0x7f304406ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.460+0000 7f3042ffd700 1 --2- 172.21.15.60:0/293832343 >> v2:172.21.15.105:3300/0 conn(0x7f30440add00 0x7f304406ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.460+0000 7f3042ffd700 1 -- 172.21.15.60:0/293832343 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30440ae0d0 con 0x7f30440ca9f0 2023-07-13T22:30:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.460+0000 7f3042ffd700 1 --2- 172.21.15.60:0/293832343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30440ca9f0 0x7f3044069ed0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f30440cb220 tx=0x7f3038009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21abd2222832c9aa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.460+0000 7f30427fc700 1 -- 172.21.15.60:0/293832343 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303800b870 con 0x7f30440ca9f0 2023-07-13T22:30:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.460+0000 7f30427fc700 1 -- 172.21.15.60:0/293832343 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303800b9d0 con 0x7f30440ca9f0 2023-07-13T22:30:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.460+0000 7f30427fc700 1 -- 172.21.15.60:0/293832343 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303800bd60 con 0x7f30440ca9f0 2023-07-13T22:30:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.461+0000 7f304a1ff700 1 -- 172.21.15.60:0/293832343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30440ca9f0 msgr2=0x7f3044069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.461+0000 7f304a1ff700 1 --2- 172.21.15.60:0/293832343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30440ca9f0 0x7f3044069ed0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f30440cb220 tx=0x7f3038009bd0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.461+0000 7f304a1ff700 1 -- 172.21.15.60:0/293832343 shutdown_connections 2023-07-13T22:30:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.461+0000 7f304a1ff700 1 --2- 172.21.15.60:0/293832343 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f304406a410 0x7f30440703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.461+0000 7f304a1ff700 1 --2- 172.21.15.60:0/293832343 >> v2:172.21.15.105:3300/0 conn(0x7f30440add00 0x7f304406ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.461+0000 7f304a1ff700 1 --2- 172.21.15.60:0/293832343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30440ca9f0 0x7f3044069ed0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.461+0000 7f304a1ff700 1 -- 172.21.15.60:0/293832343 >> 172.21.15.60:0/293832343 conn(0x7f30440cbf50 msgr2=0x7f30440ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.461+0000 7f304a1ff700 1 -- 172.21.15.60:0/293832343 shutdown_connections 2023-07-13T22:30:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.461+0000 7f304a1ff700 1 -- 172.21.15.60:0/293832343 wait complete. 2023-07-13T22:30:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.463+0000 7f304a1ff700 1 Processor -- start 2023-07-13T22:30:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.463+0000 7f304a1ff700 1 -- start start 2023-07-13T22:30:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f304a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f304406a410 0x7f3044158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f304a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30440add00 0x7f30440c2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f304a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30440ca9f0 0x7f30440c2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f304a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30441599e0 con 0x7f304406a410 2023-07-13T22:30:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f304a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30440aeef0 con 0x7f30440ca9f0 2023-07-13T22:30:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f304a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30440af1d0 con 0x7f30440add00 2023-07-13T22:30:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f30437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f304406a410 0x7f3044158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f30437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f304406a410 0x7f3044158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50690/0 (socket says 172.21.15.60:50690) 2023-07-13T22:30:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f30437fe700 1 -- 172.21.15.60:0/3104942498 learned_addr learned my addr 172.21.15.60:0/3104942498 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f30437fe700 1 -- 172.21.15.60:0/3104942498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30440add00 msgr2=0x7f30440c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f3042ffd700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30440add00 0x7f30440c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f30437fe700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30440add00 0x7f30440c2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f30437fe700 1 -- 172.21.15.60:0/3104942498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30440ca9f0 msgr2=0x7f30440c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f30437fe700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30440ca9f0 0x7f30440c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.464+0000 7f30437fe700 1 -- 172.21.15.60:0/3104942498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3038009550 con 0x7f304406a410 2023-07-13T22:30:54.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.465+0000 7f30437fe700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f304406a410 0x7f3044158fa0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f303400e810 tx=0x7f303400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.465+0000 7f302bfff700 1 -- 172.21.15.60:0/3104942498 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034013610 con 0x7f304406a410 2023-07-13T22:30:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.465+0000 7f302bfff700 1 -- 172.21.15.60:0/3104942498 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3034013770 con 0x7f304406a410 2023-07-13T22:30:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.465+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30440af4b0 con 0x7f304406a410 2023-07-13T22:30:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.465+0000 7f302bfff700 1 -- 172.21.15.60:0/3104942498 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034013b20 con 0x7f304406a410 2023-07-13T22:30:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.465+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30440ae2a0 con 0x7f304406a410 2023-07-13T22:30:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.466+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30440afde0 con 0x7f304406a410 2023-07-13T22:30:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.467+0000 7f302bfff700 1 -- 172.21.15.60:0/3104942498 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3034013c80 con 0x7f304406a410 2023-07-13T22:30:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.468+0000 7f302bfff700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f303006ace0 0x7f303006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.468+0000 7f302bfff700 1 -- 172.21.15.60:0/3104942498 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3034089190 con 0x7f304406a410 2023-07-13T22:30:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.468+0000 7f3042ffd700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f303006ace0 0x7f303006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.469+0000 7f3042ffd700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f303006ace0 0x7f303006d190 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f30440c3310 tx=0x7f3038008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:54.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.472+0000 7f302bfff700 1 -- 172.21.15.60:0/3104942498 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f30340541b0 con 0x7f304406a410 2023-07-13T22:30:54.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.679+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7f30440afb00 con 0x7f304406a410 2023-07-13T22:30:54.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.679+0000 7f302bfff700 1 -- 172.21.15.60:0/3104942498 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v58) v1 ==== 112+0+10 (secure 0 0 0) 0x7f30340577d0 con 0x7f304406a410 2023-07-13T22:30:54.661 INFO:teuthology.orchestra.run.smithi060.stdout:60.000000 2023-07-13T22:30:54.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f303006ace0 msgr2=0x7f303006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:54.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f303006ace0 0x7f303006d190 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f30440c3310 tx=0x7f3038008030 comp rx=0 tx=0).stop 2023-07-13T22:30:54.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f304406a410 msgr2=0x7f3044158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:54.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f304406a410 0x7f3044158fa0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f303400e810 tx=0x7f303400ebd0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 shutdown_connections 2023-07-13T22:30:54.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f303006ace0 0x7f303006d190 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f304406a410 0x7f3044158fa0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30440ca9f0 0x7f30440c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 --2- 172.21.15.60:0/3104942498 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30440add00 0x7f30440c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:54.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 >> 172.21.15.60:0/3104942498 conn(0x7f30440cbf50 msgr2=0x7f30440cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:54.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 shutdown_connections 2023-07-13T22:30:54.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:54.681+0000 7f304a1ff700 1 -- 172.21.15.60:0/3104942498 wait complete. 2023-07-13T22:30:54.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:30:54.680 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:30:54.681 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:30:54.681 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:30:54.681 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:30:54.682 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:54.682 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:30:54.682 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:30:54.682 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:30:54.683 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:30:54.683 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:54.683 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:30:54.684 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:30:54.684 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:30:54.684 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:30:54.684 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:54.685 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:30:54.685 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:30:54.685 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:30:54.686 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:30:54.686 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:30:54.686 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:30:54.686 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:30:54.687 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:30:54.687 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:54.687 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:30:54.688 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:30:54.834 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.161669.asok 2023-07-13T22:30:55.012 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.161669.asok 2023-07-13T22:30:55.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:30:55.013 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.161669.asok mds_sessions 2023-07-13T22:30:55.244 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:30:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9084, 2023-07-13T22:30:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:30:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:30:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:30:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 9084 2023-07-13T22:30:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:30:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:30:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:30:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3195742970 2023-07-13T22:30:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.9084 v1:192.168.0.1:0/3195742970", 2023-07-13T22:30:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "v1:192.168.0.1:0/3195742970", 2023-07-13T22:30:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions": [ 2023-07-13T22:30:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": 0, 2023-07-13T22:30:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "addrs": { 2023-07-13T22:30:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2023-07-13T22:30:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2023-07-13T22:30:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6834", 2023-07-13T22:30:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1781435251 2023-07-13T22:30:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:30:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:30:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6836", 2023-07-13T22:30:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1781435251 2023-07-13T22:30:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:30:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:30:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "seq": 0, 2023-07-13T22:30:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_gen": 0, 2023-07-13T22:30:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_ttl": "2023-07-13T22:31:49.437826+0000", 2023-07-13T22:30:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "last_cap_renew_request": "2023-07-13T22:30:49.437826+0000", 2023-07-13T22:30:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_renew_seq": 2, 2023-07-13T22:30:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "num_caps": 1, 2023-07-13T22:30:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "open" 2023-07-13T22:30:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:30:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2023-07-13T22:30:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 85 2023-07-13T22:30:55.250 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:30:55.258 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:30:55.259 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:30:55.306 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:30:55.306 DEBUG:teuthology.orchestra.run.smithi060:> import sys 2023-07-13T22:30:55.306 DEBUG:teuthology.orchestra.run.smithi060:> import time 2023-07-13T22:30:55.307 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:30:55.307 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:55.307 DEBUG:teuthology.orchestra.run.smithi060:> n = 400 2023-07-13T22:30:55.308 DEBUG:teuthology.orchestra.run.smithi060:> abs_path = "/home/ubuntu/cephtest/mnt.0/." 2023-07-13T22:30:55.308 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:55.308 DEBUG:teuthology.orchestra.run.smithi060:> if not os.path.exists(abs_path): 2023-07-13T22:30:55.308 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(abs_path) 2023-07-13T22:30:55.309 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:55.309 DEBUG:teuthology.orchestra.run.smithi060:> handles = [] 2023-07-13T22:30:55.310 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(0, n): 2023-07-13T22:30:55.310 DEBUG:teuthology.orchestra.run.smithi060:> fname = "file_"+str(i) 2023-07-13T22:30:55.311 DEBUG:teuthology.orchestra.run.smithi060:> path = os.path.join(abs_path, fname) 2023-07-13T22:30:55.311 DEBUG:teuthology.orchestra.run.smithi060:> handles.append(open(path, '"'"'w'"'"')) 2023-07-13T22:30:55.311 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:30:55.311 DEBUG:teuthology.orchestra.run.smithi060:> while True: 2023-07-13T22:30:55.312 DEBUG:teuthology.orchestra.run.smithi060:> time.sleep(1) 2023-07-13T22:30:55.313 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:30:55.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:30:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.606+0000 7f62ab59e700 1 Processor -- start 2023-07-13T22:30:55.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.606+0000 7f62ab59e700 1 -- start start 2023-07-13T22:30:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.607+0000 7f62ab59e700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 0x7f62ac0d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.607+0000 7f62ab59e700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f62ac0cfa90 0x7f62ac0b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.607+0000 7f62ab59e700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac0afe40 con 0x7f62ac0d0ad0 2023-07-13T22:30:55.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.607+0000 7f62ab59e700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ac0d0f40 con 0x7f62ac0cfa90 2023-07-13T22:30:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.607+0000 7f62ab59e700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0d0090 con 0x7f62ac15c290 2023-07-13T22:30:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62a9d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f62ac0d0ad0 0x7f62ac0cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60434/0 (socket says 172.21.15.60:60434) 2023-07-13T22:30:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62a9d9b700 1 -- 172.21.15.60:0/4022910707 learned_addr learned my addr 172.21.15.60:0/4022910707 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:30:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62aa59c700 1 --2- 172.21.15.60:0/4022910707 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 0x7f62ac0d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62aad9d700 1 --2- 172.21.15.60:0/4022910707 >> v2:172.21.15.105:3300/0 conn(0x7f62ac0cfa90 0x7f62ac0b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62a959a700 1 -- 172.21.15.60:0/4022910707 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826716964 0 0) 0x7f62ac0afe40 con 0x7f62ac0d0ad0 2023-07-13T22:30:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62a959a700 1 -- 172.21.15.60:0/4022910707 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62940035b0 con 0x7f62ac0d0ad0 2023-07-13T22:30:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62aa59c700 1 -- 172.21.15.60:0/4022910707 >> v2:172.21.15.105:3300/0 conn(0x7f62ac0cfa90 msgr2=0x7f62ac0b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:30:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62aa59c700 1 --2- 172.21.15.60:0/4022910707 >> v2:172.21.15.105:3300/0 conn(0x7f62ac0cfa90 0x7f62ac0b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62aa59c700 1 -- 172.21.15.60:0/4022910707 >> v1:172.21.15.60:6789/0 conn(0x7f62ac0d0ad0 legacy=0x7f62ac0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62aa59c700 1 -- 172.21.15.60:0/4022910707 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac0af7f0 con 0x7f62ac15c290 2023-07-13T22:30:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.608+0000 7f62aad9d700 1 --2- 172.21.15.60:0/4022910707 >> v2:172.21.15.105:3300/0 conn(0x7f62ac0cfa90 0x7f62ac0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:30:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.609+0000 7f62aa59c700 1 --2- 172.21.15.60:0/4022910707 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 0x7f62ac0d0590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f629c009840 tx=0x7f629c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d66491b3702b8a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.609+0000 7f62a959a700 1 -- 172.21.15.60:0/4022910707 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c011830 con 0x7f62ac15c290 2023-07-13T22:30:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.609+0000 7f62a959a700 1 -- 172.21.15.60:0/4022910707 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c011990 con 0x7f62ac15c290 2023-07-13T22:30:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.609+0000 7f62a959a700 1 -- 172.21.15.60:0/4022910707 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c011d00 con 0x7f62ac15c290 2023-07-13T22:30:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.610+0000 7f62ab59e700 1 -- 172.21.15.60:0/4022910707 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 msgr2=0x7f62ac0d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.610+0000 7f62ab59e700 1 --2- 172.21.15.60:0/4022910707 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 0x7f62ac0d0590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f629c009840 tx=0x7f629c009b50 comp rx=0 tx=0).stop 2023-07-13T22:30:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.610+0000 7f62ab59e700 1 -- 172.21.15.60:0/4022910707 shutdown_connections 2023-07-13T22:30:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.610+0000 7f62ab59e700 1 --2- 172.21.15.60:0/4022910707 >> v2:172.21.15.105:3300/0 conn(0x7f62ac0cfa90 0x7f62ac0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.610+0000 7f62ab59e700 1 --2- 172.21.15.60:0/4022910707 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 0x7f62ac0d0590 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.610+0000 7f62ab59e700 1 -- 172.21.15.60:0/4022910707 >> 172.21.15.60:0/4022910707 conn(0x7f62ac071f20 msgr2=0x7f62ac09bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.610+0000 7f62ab59e700 1 -- 172.21.15.60:0/4022910707 shutdown_connections 2023-07-13T22:30:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.610+0000 7f62ab59e700 1 -- 172.21.15.60:0/4022910707 wait complete. 2023-07-13T22:30:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.612+0000 7f62ab59e700 1 Processor -- start 2023-07-13T22:30:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.612+0000 7f62ab59e700 1 -- start start 2023-07-13T22:30:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.613+0000 7f62ab59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ac0cfa90 0x7f62ac153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62ab59e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62ac0d0ad0 0x7f62ac151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62ab59e700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 0x7f62ac1f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62ab59e700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac1f6080 con 0x7f62ac0cfa90 2023-07-13T22:30:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62ab59e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac151740 con 0x7f62ac0d0ad0 2023-07-13T22:30:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62ab59e700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac1f5ac0 con 0x7f62ac15c290 2023-07-13T22:30:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62aa59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ac0cfa90 0x7f62ac153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62aa59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ac0cfa90 0x7f62ac153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34594/0 (socket says 172.21.15.60:34594) 2023-07-13T22:30:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62aa59c700 1 -- 172.21.15.60:0/3511880205 learned_addr learned my addr 172.21.15.60:0/3511880205 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:30:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62aad9d700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 0x7f62ac1f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62a9d9b700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62ac0d0ad0 0x7f62ac151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62aa59c700 1 -- 172.21.15.60:0/3511880205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 msgr2=0x7f62ac1f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62aa59c700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 0x7f62ac1f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62aa59c700 1 -- 172.21.15.60:0/3511880205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62ac0d0ad0 msgr2=0x7f62ac151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62aa59c700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62ac0d0ad0 0x7f62ac151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.614+0000 7f62aa59c700 1 -- 172.21.15.60:0/3511880205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4009a70 con 0x7f62ac0cfa90 2023-07-13T22:30:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.615+0000 7f62aa59c700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ac0cfa90 0x7f62ac153180 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f629c0090b0 tx=0x7f629c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.615+0000 7f629affd700 1 -- 172.21.15.60:0/3511880205 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c004550 con 0x7f62ac0cfa90 2023-07-13T22:30:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.615+0000 7f629affd700 1 -- 172.21.15.60:0/3511880205 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c01f070 con 0x7f62ac0cfa90 2023-07-13T22:30:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.615+0000 7f629affd700 1 -- 172.21.15.60:0/3511880205 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0038a0 con 0x7f62ac0cfa90 2023-07-13T22:30:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.615+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c0094a0 con 0x7f62ac0cfa90 2023-07-13T22:30:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.615+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac0b1da0 con 0x7f62ac0cfa90 2023-07-13T22:30:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.616+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac04e210 con 0x7f62ac0cfa90 2023-07-13T22:30:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.617+0000 7f629affd700 1 -- 172.21.15.60:0/3511880205 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f629c003a00 con 0x7f62ac0cfa90 2023-07-13T22:30:55.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.618+0000 7f629affd700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f628c06ace0 0x7f628c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:30:55.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.618+0000 7f629affd700 1 -- 172.21.15.60:0/3511880205 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f629c021070 con 0x7f62ac0cfa90 2023-07-13T22:30:55.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.619+0000 7f62a9d9b700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f628c06ace0 0x7f628c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:30:55.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.619+0000 7f62a9d9b700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f628c06ace0 0x7f628c06d190 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0b5d80 tx=0x7f62a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:30:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.625+0000 7f629affd700 1 -- 172.21.15.60:0/3511880205 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f629c055c60 con 0x7f62ac0cfa90 2023-07-13T22:30:55.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.944+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62ac0b5ad0 con 0x7f62ac0cfa90 2023-07-13T22:30:55.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.944+0000 7f629affd700 1 -- 172.21.15.60:0/3511880205 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f629c02b090 con 0x7f62ac0cfa90 2023-07-13T22:30:55.926 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:30:55.926 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:30:39.690428+0000","modified":"2023-07-13T22:30:43.199497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6836/1781435251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1781435251},{"type":"v1","addr":"172.21.15.60:6836","nonce":1781435251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-13T22:30:55.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f628c06ace0 msgr2=0x7f628c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:55.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f628c06ace0 0x7f628c06d190 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0b5d80 tx=0x7f62a0009040 comp rx=0 tx=0).stop 2023-07-13T22:30:55.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ac0cfa90 msgr2=0x7f62ac153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:30:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ac0cfa90 0x7f62ac153180 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f629c0090b0 tx=0x7f629c004750 comp rx=0 tx=0).stop 2023-07-13T22:30:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 shutdown_connections 2023-07-13T22:30:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f628c06ace0 0x7f628c06d190 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ac0cfa90 0x7f62ac153180 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62ac15c290 0x7f62ac1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 --2- 172.21.15.60:0/3511880205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62ac0d0ad0 0x7f62ac151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:30:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.947+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 >> 172.21.15.60:0/3511880205 conn(0x7f62ac071f20 msgr2=0x7f62ac15f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:30:55.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.948+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 shutdown_connections 2023-07-13T22:30:55.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:30:55.948+0000 7f62ab59e700 1 -- 172.21.15.60:0/3511880205 wait complete. 2023-07-13T22:30:55.930 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 85 2023-07-13T22:30:55.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-07-13T22:30:56.224 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":8422,"entity":{"name":{"type":"client","num":8422},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":814386305}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":5.4499434139999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9840817687822718,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.8422 192.168.0.1:0/814386305","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"127147","root":"/"}},{"id":9084,"entity":{"name":{"type":"client","num":9084},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3195742970}},"state":"open","num_leases":274,"num_caps":275,"request_load_avg":820,"uptime":7.7869191500000001,"requests_in_flight":264,"num_completed_requests":11,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":172.96402977705026,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":171.56700932251309,"halflife":1.5},"recall_caps_throttle2o":{"value":168.73653882095661,"halflife":0.5},"session_cache_liveness":{"value":822.28870711535637,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":7,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.9084 192.168.0.1:0/3195742970","completed_requests":[{"tid":7,"created_ino":"0x10000000001"},{"tid":10,"created_ino":"0x100000001f6"},{"tid":13,"created_ino":"0x100000001f7"},{"tid":16,"created_ino":"0x100000001f8"},{"tid":19,"created_ino":"0x100000001f9"},{"tid":22,"created_ino":"0x100000001fa"},{"tid":25,"created_ino":"0x100000001fb"},{"tid":28,"created_ino":"0x100000001fc"},{"tid":31,"created_ino":"0x100000001fd"},{"tid":34,"created_ino":"0x100000001fe"},{"tid":37,"created_ino":"0x100000001ff"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000200","length":991}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"161669","root":"/"}}] 2023-07-13T22:30:56.225 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "127147", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 814386305, "type": "any" }, "name": { "num": 8422, "type": "client" } }, "id": 8422, "inst": "client.8422 192.168.0.1:0/814386305", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.984081768782272 }, "state": "open", "uptime": 5.449943414 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "161669", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 7 }, { "created_ino": "0x100000001f6", "tid": 10 }, { "created_ino": "0x100000001f7", "tid": 13 }, { "created_ino": "0x100000001f8", "tid": 16 }, { "created_ino": "0x100000001f9", "tid": 19 }, { "created_ino": "0x100000001fa", "tid": 22 }, { "created_ino": "0x100000001fb", "tid": 25 }, { "created_ino": "0x100000001fc", "tid": 28 }, { "created_ino": "0x100000001fd", "tid": 31 }, { "created_ino": "0x100000001fe", "tid": 34 }, { "created_ino": "0x100000001ff", "tid": 37 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3195742970, "type": "any" }, "name": { "num": 9084, "type": "client" } }, "id": 9084, "inst": "client.9084 192.168.0.1:0/3195742970", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 7, "num_caps": 275, "num_completed_flushes": 0, "num_completed_requests": 11, "num_leases": 274, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 991, "start": "0x10000000200" } ], "recall_caps": { "halflife": 60, "value": 172.96402977705026 }, "recall_caps_throttle": { "halflife": 1.5, "value": 171.5670093225131 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 168.7365388209566 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 820, "requests_in_flight": 264, "session_cache_liveness": { "halflife": 300, "value": 822.2887071153564 }, "state": "open", "uptime": 7.78691915 } ] 2023-07-13T22:30:56.225 DEBUG:tasks.ceph_test_case:wait_until_equal: 275 != 401, waiting (timeout=600)... 2023-07-13T22:30:57.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:30:57.986 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:31:01.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:01.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.462+0000 7f20a13ff700 1 Processor -- start 2023-07-13T22:31:01.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.462+0000 7f20a13ff700 1 -- start start 2023-07-13T22:31:01.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.463+0000 7f20a13ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f209c15c6e0 0x7f209c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:01.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.463+0000 7f20a13ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f209c0b3050 0x7f209c06afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:01.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.463+0000 7f20a13ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209c06b8e0 con 0x7f209c0b24b0 2023-07-13T22:31:01.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.463+0000 7f20a13ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209c15cab0 con 0x7f209c0b3050 2023-07-13T22:31:01.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.463+0000 7f20a13ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209c06bbc0 con 0x7f209c15c6e0 2023-07-13T22:31:01.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.463+0000 7f209a7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f209c0b3050 0x7f209c06afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:01.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.464+0000 7f209a7fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f209c0b3050 0x7f209c06afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34602/0 (socket says 172.21.15.60:34602) 2023-07-13T22:31:01.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.464+0000 7f209a7fc700 1 -- 172.21.15.60:0/951209676 learned_addr learned my addr 172.21.15.60:0/951209676 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:01.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.464+0000 7f209affd700 1 --2- 172.21.15.60:0/951209676 >> v2:172.21.15.105:3300/0 conn(0x7f209c15c6e0 0x7f209c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:01.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.464+0000 7f209a7fc700 1 -- 172.21.15.60:0/951209676 >> v2:172.21.15.105:3300/0 conn(0x7f209c15c6e0 msgr2=0x7f209c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:01.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.464+0000 7f209a7fc700 1 --2- 172.21.15.60:0/951209676 >> v2:172.21.15.105:3300/0 conn(0x7f209c15c6e0 0x7f209c06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.464+0000 7f209a7fc700 1 -- 172.21.15.60:0/951209676 >> v1:172.21.15.105:6789/0 conn(0x7f209c0b24b0 legacy=0x7f209c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:01.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.464+0000 7f209a7fc700 1 -- 172.21.15.60:0/951209676 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c06bea0 con 0x7f209c0b3050 2023-07-13T22:31:01.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.464+0000 7f209a7fc700 1 --2- 172.21.15.60:0/951209676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 0x7f209c06afe0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f209c0ce0f0 tx=0x7f2090009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0db8a04d723ebe server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:01.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f2099ffb700 1 -- 172.21.15.60:0/951209676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209000b870 con 0x7f209c0b3050 2023-07-13T22:31:01.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f2099ffb700 1 -- 172.21.15.60:0/951209676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209000b9d0 con 0x7f209c0b3050 2023-07-13T22:31:01.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f2099ffb700 1 -- 172.21.15.60:0/951209676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209000bd60 con 0x7f209c0b3050 2023-07-13T22:31:01.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f20a13ff700 1 -- 172.21.15.60:0/951209676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 msgr2=0x7f209c06afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:01.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f20a13ff700 1 --2- 172.21.15.60:0/951209676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 0x7f209c06afe0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f209c0ce0f0 tx=0x7f2090009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f20a13ff700 1 -- 172.21.15.60:0/951209676 shutdown_connections 2023-07-13T22:31:01.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f20a13ff700 1 --2- 172.21.15.60:0/951209676 >> v2:172.21.15.105:3300/0 conn(0x7f209c15c6e0 0x7f209c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f20a13ff700 1 --2- 172.21.15.60:0/951209676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 0x7f209c06afe0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f20a13ff700 1 -- 172.21.15.60:0/951209676 >> 172.21.15.60:0/951209676 conn(0x7f209c06e680 msgr2=0x7f209c06ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:01.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f20a13ff700 1 -- 172.21.15.60:0/951209676 shutdown_connections 2023-07-13T22:31:01.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.465+0000 7f20a13ff700 1 -- 172.21.15.60:0/951209676 wait complete. 2023-07-13T22:31:01.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.467+0000 7f20a13ff700 1 Processor -- start 2023-07-13T22:31:01.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.467+0000 7f20a13ff700 1 -- start start 2023-07-13T22:31:01.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.468+0000 7f20a13ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f209c0b24b0 0x7f209c1ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:01.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.468+0000 7f20a13ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 0x7f209c1ef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:01.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.468+0000 7f20a13ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f209c15c6e0 0x7f209c15ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.468+0000 7f20a13ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1f20e0 con 0x7f209c0b3050 2023-07-13T22:31:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.468+0000 7f20a13ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1f23c0 con 0x7f209c15c6e0 2023-07-13T22:31:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.468+0000 7f20a13ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1f26a0 con 0x7f209c0b24b0 2023-07-13T22:31:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.468+0000 7f209a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 0x7f209c1ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 0x7f209c1ef600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34606/0 (socket says 172.21.15.60:34606) 2023-07-13T22:31:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209a7fc700 1 -- 172.21.15.60:0/12437900 learned_addr learned my addr 172.21.15.60:0/12437900 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209affd700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f209c0b24b0 0x7f209c1ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209b7fe700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f209c15c6e0 0x7f209c15ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209a7fc700 1 -- 172.21.15.60:0/12437900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f209c0b24b0 msgr2=0x7f209c1ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209a7fc700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f209c0b24b0 0x7f209c1ef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209a7fc700 1 -- 172.21.15.60:0/12437900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f209c15c6e0 msgr2=0x7f209c15ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209a7fc700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f209c15c6e0 0x7f209c15ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209a7fc700 1 -- 172.21.15.60:0/12437900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090009550 con 0x7f209c0b3050 2023-07-13T22:31:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209b7fe700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f209c15c6e0 0x7f209c15ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:01.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f209a7fc700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 0x7f209c1ef600 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f2090004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:01.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f20837fe700 1 -- 172.21.15.60:0/12437900 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090004540 con 0x7f209c0b3050 2023-07-13T22:31:01.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.469+0000 7f20837fe700 1 -- 172.21.15.60:0/12437900 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20900036a0 con 0x7f209c0b3050 2023-07-13T22:31:01.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.470+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c1f2980 con 0x7f209c0b3050 2023-07-13T22:31:01.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.470+0000 7f20837fe700 1 -- 172.21.15.60:0/12437900 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090003a10 con 0x7f209c0b3050 2023-07-13T22:31:01.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.470+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c15f4a0 con 0x7f209c0b3050 2023-07-13T22:31:01.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.471+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c1f2ea0 con 0x7f209c0b3050 2023-07-13T22:31:01.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.472+0000 7f20837fe700 1 -- 172.21.15.60:0/12437900 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2090021070 con 0x7f209c0b3050 2023-07-13T22:31:01.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.472+0000 7f20837fe700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f208806ace0 0x7f208806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:01.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.473+0000 7f20837fe700 1 -- 172.21.15.60:0/12437900 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f209008ab70 con 0x7f209c0b3050 2023-07-13T22:31:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.473+0000 7f209affd700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f208806ace0 0x7f208806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.473+0000 7f209affd700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f208806ace0 0x7f208806d190 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f208c006b20 tx=0x7f208c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:01.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.477+0000 7f20837fe700 1 -- 172.21.15.60:0/12437900 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f209c1f2ea0 con 0x7f209c0b3050 2023-07-13T22:31:01.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.726+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f209c1f2ea0 con 0x7f209c0b3050 2023-07-13T22:31:01.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.727+0000 7f20837fe700 1 -- 172.21.15.60:0/12437900 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f209c1f2ea0 con 0x7f209c0b3050 2023-07-13T22:31:01.709 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:01.709 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:30:39.690428+0000","modified":"2023-07-13T22:31:00.559611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6836/1781435251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1781435251},{"type":"v1","addr":"172.21.15.60:6836","nonce":1781435251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-13T22:31:01.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f208806ace0 msgr2=0x7f208806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:01.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f208806ace0 0x7f208806d190 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f208c006b20 tx=0x7f208c008040 comp rx=0 tx=0).stop 2023-07-13T22:31:01.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 msgr2=0x7f209c1ef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:01.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 0x7f209c1ef600 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f2090004750 comp rx=0 tx=0).stop 2023-07-13T22:31:01.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 shutdown_connections 2023-07-13T22:31:01.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f208806ace0 0x7f208806d190 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f209c15c6e0 0x7f209c15ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f209c0b24b0 0x7f209c1ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 --2- 172.21.15.60:0/12437900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c0b3050 0x7f209c1ef600 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:01.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 >> 172.21.15.60:0/12437900 conn(0x7f209c06e680 msgr2=0x7f209c0b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:01.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.729+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 shutdown_connections 2023-07-13T22:31:01.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:01.730+0000 7f20a13ff700 1 -- 172.21.15.60:0/12437900 wait complete. 2023-07-13T22:31:01.712 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 87 2023-07-13T22:31:01.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-07-13T22:31:01.955 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":8422,"entity":{"name":{"type":"client","num":8422},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":814386305}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":11.184883871,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9448019296166592,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.8422 192.168.0.1:0/814386305","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"127147","root":"/"}},{"id":9084,"entity":{"name":{"type":"client","num":9084},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3195742970}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":1124,"uptime":13.521859607,"requests_in_flight":0,"num_completed_requests":125,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1130.7248354788831,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":356.36134183689916,"halflife":1.5},"recall_caps_throttle2o":{"value":94.337548929614414,"halflife":0.5},"session_cache_liveness":{"value":1184.6588323636333,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":831,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.9084 192.168.0.1:0/3195742970","completed_requests":[{"tid":831,"created_ino":"0x10000000307"},{"tid":834,"created_ino":"0x10000000308"},{"tid":837,"created_ino":"0x10000000309"},{"tid":840,"created_ino":"0x1000000030a"},{"tid":843,"created_ino":"0x1000000030b"},{"tid":846,"created_ino":"0x1000000030c"},{"tid":849,"created_ino":"0x1000000030d"},{"tid":852,"created_ino":"0x1000000030e"},{"tid":855,"created_ino":"0x1000000030f"},{"tid":858,"created_ino":"0x10000000310"},{"tid":861,"created_ino":"0x10000000311"},{"tid":864,"created_ino":"0x10000000312"},{"tid":867,"created_ino":"0x10000000313"},{"tid":870,"created_ino":"0x10000000314"},{"tid":873,"created_ino":"0x10000000315"},{"tid":876,"created_ino":"0x10000000316"},{"tid":879,"created_ino":"0x10000000317"},{"tid":882,"created_ino":"0x10000000318"},{"tid":885,"created_ino":"0x10000000319"},{"tid":888,"created_ino":"0x1000000031a"},{"tid":891,"created_ino":"0x1000000031b"},{"tid":894,"created_ino":"0x1000000031c"},{"tid":897,"created_ino":"0x1000000031d"},{"tid":900,"created_ino":"0x1000000031e"},{"tid":903,"created_ino":"0x1000000031f"},{"tid":906,"created_ino":"0x10000000320"},{"tid":909,"created_ino":"0x10000000321"},{"tid":912,"created_ino":"0x10000000322"},{"tid":915,"created_ino":"0x10000000323"},{"tid":918,"created_ino":"0x10000000324"},{"tid":921,"created_ino":"0x10000000325"},{"tid":924,"created_ino":"0x10000000326"},{"tid":927,"created_ino":"0x10000000327"},{"tid":930,"created_ino":"0x10000000328"},{"tid":933,"created_ino":"0x10000000329"},{"tid":936,"created_ino":"0x1000000032a"},{"tid":939,"created_ino":"0x1000000032b"},{"tid":942,"created_ino":"0x1000000032c"},{"tid":945,"created_ino":"0x1000000032d"},{"tid":948,"created_ino":"0x1000000032e"},{"tid":951,"created_ino":"0x1000000032f"},{"tid":954,"created_ino":"0x10000000330"},{"tid":957,"created_ino":"0x10000000331"},{"tid":960,"created_ino":"0x10000000332"},{"tid":963,"created_ino":"0x10000000333"},{"tid":966,"created_ino":"0x10000000334"},{"tid":969,"created_ino":"0x10000000335"},{"tid":972,"created_ino":"0x10000000336"},{"tid":975,"created_ino":"0x10000000337"},{"tid":978,"created_ino":"0x10000000338"},{"tid":981,"created_ino":"0x10000000339"},{"tid":984,"created_ino":"0x1000000033a"},{"tid":987,"created_ino":"0x1000000033b"},{"tid":990,"created_ino":"0x1000000033c"},{"tid":993,"created_ino":"0x1000000033d"},{"tid":996,"created_ino":"0x1000000033e"},{"tid":999,"created_ino":"0x1000000033f"},{"tid":1002,"created_ino":"0x10000000340"},{"tid":1005,"created_ino":"0x10000000341"},{"tid":1008,"created_ino":"0x10000000342"},{"tid":1011,"created_ino":"0x10000000343"},{"tid":1014,"created_ino":"0x10000000344"},{"tid":1017,"created_ino":"0x10000000345"},{"tid":1020,"created_ino":"0x10000000346"},{"tid":1023,"created_ino":"0x10000000347"},{"tid":1026,"created_ino":"0x10000000348"},{"tid":1029,"created_ino":"0x10000000349"},{"tid":1032,"created_ino":"0x1000000034a"},{"tid":1035,"created_ino":"0x1000000034b"},{"tid":1038,"created_ino":"0x1000000034c"},{"tid":1041,"created_ino":"0x1000000034d"},{"tid":1044,"created_ino":"0x1000000034e"},{"tid":1047,"created_ino":"0x1000000034f"},{"tid":1050,"created_ino":"0x10000000350"},{"tid":1053,"created_ino":"0x10000000351"},{"tid":1056,"created_ino":"0x10000000352"},{"tid":1059,"created_ino":"0x10000000353"},{"tid":1062,"created_ino":"0x10000000354"},{"tid":1065,"created_ino":"0x10000000355"},{"tid":1068,"created_ino":"0x10000000356"},{"tid":1071,"created_ino":"0x10000000357"},{"tid":1074,"created_ino":"0x10000000358"},{"tid":1077,"created_ino":"0x10000000359"},{"tid":1080,"created_ino":"0x1000000035a"},{"tid":1083,"created_ino":"0x1000000035b"},{"tid":1086,"created_ino":"0x1000000035c"},{"tid":1089,"created_ino":"0x1000000035d"},{"tid":1092,"created_ino":"0x1000000035e"},{"tid":1095,"created_ino":"0x1000000035f"},{"tid":1098,"created_ino":"0x10000000360"},{"tid":1101,"created_ino":"0x10000000361"},{"tid":1104,"created_ino":"0x10000000362"},{"tid":1107,"created_ino":"0x10000000363"},{"tid":1110,"created_ino":"0x10000000364"},{"tid":1113,"created_ino":"0x10000000365"},{"tid":1116,"created_ino":"0x10000000366"},{"tid":1119,"created_ino":"0x10000000367"},{"tid":1122,"created_ino":"0x10000000368"},{"tid":1125,"created_ino":"0x10000000369"},{"tid":1128,"created_ino":"0x1000000036a"},{"tid":1131,"created_ino":"0x1000000036b"},{"tid":1134,"created_ino":"0x1000000036c"},{"tid":1137,"created_ino":"0x1000000036d"},{"tid":1140,"created_ino":"0x1000000036e"},{"tid":1143,"created_ino":"0x1000000036f"},{"tid":1146,"created_ino":"0x10000000370"},{"tid":1149,"created_ino":"0x10000000371"},{"tid":1152,"created_ino":"0x10000000372"},{"tid":1155,"created_ino":"0x10000000373"},{"tid":1158,"created_ino":"0x10000000374"},{"tid":1161,"created_ino":"0x10000000375"},{"tid":1164,"created_ino":"0x10000000376"},{"tid":1167,"created_ino":"0x10000000377"},{"tid":1170,"created_ino":"0x10000000378"},{"tid":1173,"created_ino":"0x10000000379"},{"tid":1176,"created_ino":"0x1000000037a"},{"tid":1179,"created_ino":"0x1000000037b"},{"tid":1182,"created_ino":"0x1000000037c"},{"tid":1185,"created_ino":"0x1000000037d"},{"tid":1188,"created_ino":"0x1000000037e"},{"tid":1191,"created_ino":"0x1000000037f"},{"tid":1194,"created_ino":"0x10000000380"},{"tid":1197,"created_ino":"0x10000000381"},{"tid":1200,"created_ino":"0x10000000382"},{"tid":1203,"created_ino":"0x10000000383"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"161669","root":"/"}}] 2023-07-13T22:31:01.957 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "127147", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 814386305, "type": "any" }, "name": { "num": 8422, "type": "client" } }, "id": 8422, "inst": "client.8422 192.168.0.1:0/814386305", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.944801929616659 }, "state": "open", "uptime": 11.184883871 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "161669", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000307", "tid": 831 }, { "created_ino": "0x10000000308", "tid": 834 }, { "created_ino": "0x10000000309", "tid": 837 }, { "created_ino": "0x1000000030a", "tid": 840 }, { "created_ino": "0x1000000030b", "tid": 843 }, { "created_ino": "0x1000000030c", "tid": 846 }, { "created_ino": "0x1000000030d", "tid": 849 }, { "created_ino": "0x1000000030e", "tid": 852 }, { "created_ino": "0x1000000030f", "tid": 855 }, { "created_ino": "0x10000000310", "tid": 858 }, { "created_ino": "0x10000000311", "tid": 861 }, { "created_ino": "0x10000000312", "tid": 864 }, { "created_ino": "0x10000000313", "tid": 867 }, { "created_ino": "0x10000000314", "tid": 870 }, { "created_ino": "0x10000000315", "tid": 873 }, { "created_ino": "0x10000000316", "tid": 876 }, { "created_ino": "0x10000000317", "tid": 879 }, { "created_ino": "0x10000000318", "tid": 882 }, { "created_ino": "0x10000000319", "tid": 885 }, { "created_ino": "0x1000000031a", "tid": 888 }, { "created_ino": "0x1000000031b", "tid": 891 }, { "created_ino": "0x1000000031c", "tid": 894 }, { "created_ino": "0x1000000031d", "tid": 897 }, { "created_ino": "0x1000000031e", "tid": 900 }, { "created_ino": "0x1000000031f", "tid": 903 }, { "created_ino": "0x10000000320", "tid": 906 }, { "created_ino": "0x10000000321", "tid": 909 }, { "created_ino": "0x10000000322", "tid": 912 }, { "created_ino": "0x10000000323", "tid": 915 }, { "created_ino": "0x10000000324", "tid": 918 }, { "created_ino": "0x10000000325", "tid": 921 }, { "created_ino": "0x10000000326", "tid": 924 }, { "created_ino": "0x10000000327", "tid": 927 }, { "created_ino": "0x10000000328", "tid": 930 }, { "created_ino": "0x10000000329", "tid": 933 }, { "created_ino": "0x1000000032a", "tid": 936 }, { "created_ino": "0x1000000032b", "tid": 939 }, { "created_ino": "0x1000000032c", "tid": 942 }, { "created_ino": "0x1000000032d", "tid": 945 }, { "created_ino": "0x1000000032e", "tid": 948 }, { "created_ino": "0x1000000032f", "tid": 951 }, { "created_ino": "0x10000000330", "tid": 954 }, { "created_ino": "0x10000000331", "tid": 957 }, { "created_ino": "0x10000000332", "tid": 960 }, { "created_ino": "0x10000000333", "tid": 963 }, { "created_ino": "0x10000000334", "tid": 966 }, { "created_ino": "0x10000000335", "tid": 969 }, { "created_ino": "0x10000000336", "tid": 972 }, { "created_ino": "0x10000000337", "tid": 975 }, { "created_ino": "0x10000000338", "tid": 978 }, { "created_ino": "0x10000000339", "tid": 981 }, { "created_ino": "0x1000000033a", "tid": 984 }, { "created_ino": "0x1000000033b", "tid": 987 }, { "created_ino": "0x1000000033c", "tid": 990 }, { "created_ino": "0x1000000033d", "tid": 993 }, { "created_ino": "0x1000000033e", "tid": 996 }, { "created_ino": "0x1000000033f", "tid": 999 }, { "created_ino": "0x10000000340", "tid": 1002 }, { "created_ino": "0x10000000341", "tid": 1005 }, { "created_ino": "0x10000000342", "tid": 1008 }, { "created_ino": "0x10000000343", "tid": 1011 }, { "created_ino": "0x10000000344", "tid": 1014 }, { "created_ino": "0x10000000345", "tid": 1017 }, { "created_ino": "0x10000000346", "tid": 1020 }, { "created_ino": "0x10000000347", "tid": 1023 }, { "created_ino": "0x10000000348", "tid": 1026 }, { "created_ino": "0x10000000349", "tid": 1029 }, { "created_ino": "0x1000000034a", "tid": 1032 }, { "created_ino": "0x1000000034b", "tid": 1035 }, { "created_ino": "0x1000000034c", "tid": 1038 }, { "created_ino": "0x1000000034d", "tid": 1041 }, { "created_ino": "0x1000000034e", "tid": 1044 }, { "created_ino": "0x1000000034f", "tid": 1047 }, { "created_ino": "0x10000000350", "tid": 1050 }, { "created_ino": "0x10000000351", "tid": 1053 }, { "created_ino": "0x10000000352", "tid": 1056 }, { "created_ino": "0x10000000353", "tid": 1059 }, { "created_ino": "0x10000000354", "tid": 1062 }, { "created_ino": "0x10000000355", "tid": 1065 }, { "created_ino": "0x10000000356", "tid": 1068 }, { "created_ino": "0x10000000357", "tid": 1071 }, { "created_ino": "0x10000000358", "tid": 1074 }, { "created_ino": "0x10000000359", "tid": 1077 }, { "created_ino": "0x1000000035a", "tid": 1080 }, { "created_ino": "0x1000000035b", "tid": 1083 }, { "created_ino": "0x1000000035c", "tid": 1086 }, { "created_ino": "0x1000000035d", "tid": 1089 }, { "created_ino": "0x1000000035e", "tid": 1092 }, { "created_ino": "0x1000000035f", "tid": 1095 }, { "created_ino": "0x10000000360", "tid": 1098 }, { "created_ino": "0x10000000361", "tid": 1101 }, { "created_ino": "0x10000000362", "tid": 1104 }, { "created_ino": "0x10000000363", "tid": 1107 }, { "created_ino": "0x10000000364", "tid": 1110 }, { "created_ino": "0x10000000365", "tid": 1113 }, { "created_ino": "0x10000000366", "tid": 1116 }, { "created_ino": "0x10000000367", "tid": 1119 }, { "created_ino": "0x10000000368", "tid": 1122 }, { "created_ino": "0x10000000369", "tid": 1125 }, { "created_ino": "0x1000000036a", "tid": 1128 }, { "created_ino": "0x1000000036b", "tid": 1131 }, { "created_ino": "0x1000000036c", "tid": 1134 }, { "created_ino": "0x1000000036d", "tid": 1137 }, { "created_ino": "0x1000000036e", "tid": 1140 }, { "created_ino": "0x1000000036f", "tid": 1143 }, { "created_ino": "0x10000000370", "tid": 1146 }, { "created_ino": "0x10000000371", "tid": 1149 }, { "created_ino": "0x10000000372", "tid": 1152 }, { "created_ino": "0x10000000373", "tid": 1155 }, { "created_ino": "0x10000000374", "tid": 1158 }, { "created_ino": "0x10000000375", "tid": 1161 }, { "created_ino": "0x10000000376", "tid": 1164 }, { "created_ino": "0x10000000377", "tid": 1167 }, { "created_ino": "0x10000000378", "tid": 1170 }, { "created_ino": "0x10000000379", "tid": 1173 }, { "created_ino": "0x1000000037a", "tid": 1176 }, { "created_ino": "0x1000000037b", "tid": 1179 }, { "created_ino": "0x1000000037c", "tid": 1182 }, { "created_ino": "0x1000000037d", "tid": 1185 }, { "created_ino": "0x1000000037e", "tid": 1188 }, { "created_ino": "0x1000000037f", "tid": 1191 }, { "created_ino": "0x10000000380", "tid": 1194 }, { "created_ino": "0x10000000381", "tid": 1197 }, { "created_ino": "0x10000000382", "tid": 1200 }, { "created_ino": "0x10000000383", "tid": 1203 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3195742970, "type": "any" }, "name": { "num": 9084, "type": "client" } }, "id": 9084, "inst": "client.9084 192.168.0.1:0/3195742970", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 831, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 125, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1130.724835478883 }, "recall_caps_throttle": { "halflife": 1.5, "value": 356.36134183689916 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 94.33754892961441 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1124, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1184.6588323636333 }, "state": "open", "uptime": 13.521859607 } ] 2023-07-13T22:31:01.958 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-07-13T22:31:01.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:31:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.180+0000 7fb0109ff700 1 Processor -- start 2023-07-13T22:31:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.181+0000 7fb0109ff700 1 -- start start 2023-07-13T22:31:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.182+0000 7fb0109ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c0cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.182+0000 7fb0109ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb00c0d0e20 0x7fb00c0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.182+0000 7fb0109ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00c0d2410 con 0x7fb00c0cd260 2023-07-13T22:31:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.182+0000 7fb0109ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c15c900 con 0x7fb00c0d0e20 2023-07-13T22:31:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.182+0000 7fb0109ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0d2660 con 0x7fb00c0d0a50 2023-07-13T22:31:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.182+0000 7fb00a59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c0cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.182+0000 7fb00a59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c0cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:48212/0 (socket says 172.21.15.60:48212) 2023-07-13T22:31:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.182+0000 7fb00a59c700 1 -- 172.21.15.60:0/873727118 learned_addr learned my addr 172.21.15.60:0/873727118 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.182+0000 7fb00ad9d700 1 --2- 172.21.15.60:0/873727118 >> v2:172.21.15.105:3300/0 conn(0x7fb00c0d0e20 0x7fb00c0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.183+0000 7fb00a59c700 1 -- 172.21.15.60:0/873727118 >> v2:172.21.15.105:3300/0 conn(0x7fb00c0d0e20 msgr2=0x7fb00c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.183+0000 7fb00a59c700 1 --2- 172.21.15.60:0/873727118 >> v2:172.21.15.105:3300/0 conn(0x7fb00c0d0e20 0x7fb00c0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.183+0000 7fb00a59c700 1 -- 172.21.15.60:0/873727118 >> v1:172.21.15.105:6789/0 conn(0x7fb00c0cd260 legacy=0x7fb00c0d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.183+0000 7fb00a59c700 1 -- 172.21.15.60:0/873727118 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c0d2940 con 0x7fb00c0d0a50 2023-07-13T22:31:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.183+0000 7fb00a59c700 1 --2- 172.21.15.60:0/873727118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c0cf8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faffc0098f0 tx=0x7faffc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbe7b48b0260b2d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:02.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.183+0000 7fb009d9b700 1 -- 172.21.15.60:0/873727118 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc011830 con 0x7fb00c0d0a50 2023-07-13T22:31:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.183+0000 7fb009d9b700 1 -- 172.21.15.60:0/873727118 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc011990 con 0x7fb00c0d0a50 2023-07-13T22:31:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.183+0000 7fb009d9b700 1 -- 172.21.15.60:0/873727118 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc011d20 con 0x7fb00c0d0a50 2023-07-13T22:31:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.184+0000 7fb0109ff700 1 -- 172.21.15.60:0/873727118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 msgr2=0x7fb00c0cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.184+0000 7fb0109ff700 1 --2- 172.21.15.60:0/873727118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c0cf8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faffc0098f0 tx=0x7faffc009c00 comp rx=0 tx=0).stop 2023-07-13T22:31:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.184+0000 7fb0109ff700 1 -- 172.21.15.60:0/873727118 shutdown_connections 2023-07-13T22:31:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.184+0000 7fb0109ff700 1 --2- 172.21.15.60:0/873727118 >> v2:172.21.15.105:3300/0 conn(0x7fb00c0d0e20 0x7fb00c0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.184+0000 7fb0109ff700 1 --2- 172.21.15.60:0/873727118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c0cf8a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.184+0000 7fb0109ff700 1 -- 172.21.15.60:0/873727118 >> 172.21.15.60:0/873727118 conn(0x7fb00c071600 msgr2=0x7fb00c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.184+0000 7fb0109ff700 1 -- 172.21.15.60:0/873727118 shutdown_connections 2023-07-13T22:31:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.184+0000 7fb0109ff700 1 -- 172.21.15.60:0/873727118 wait complete. 2023-07-13T22:31:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.186+0000 7fb0109ff700 1 Processor -- start 2023-07-13T22:31:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.186+0000 7fb0109ff700 1 -- start start 2023-07-13T22:31:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb0109ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cd260 0x7fb00c1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb0109ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb0109ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0d0e20 0x7fb00c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb0109ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c1598e0 con 0x7fb00c0cd260 2023-07-13T22:31:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb0109ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c159bc0 con 0x7fb00c0d0e20 2023-07-13T22:31:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb0109ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c159ea0 con 0x7fb00c0d0a50 2023-07-13T22:31:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb00a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cd260 0x7fb00c1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb00a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cd260 0x7fb00c1577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34618/0 (socket says 172.21.15.60:34618) 2023-07-13T22:31:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb00a59c700 1 -- 172.21.15.60:0/1565303431 learned_addr learned my addr 172.21.15.60:0/1565303431 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.187+0000 7fb003fff700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb00ad9d700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0d0e20 0x7fb00c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb00a59c700 1 -- 172.21.15.60:0/1565303431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 msgr2=0x7fb00c158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb00a59c700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb00a59c700 1 -- 172.21.15.60:0/1565303431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0d0e20 msgr2=0x7fb00c0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb00a59c700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0d0e20 0x7fb00c0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb00a59c700 1 -- 172.21.15.60:0/1565303431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc009550 con 0x7fb00c0cd260 2023-07-13T22:31:02.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb00a59c700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cd260 0x7fb00c1577d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7faffc000c00 tx=0x7faffc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:02.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb002ffd700 1 -- 172.21.15.60:0/1565303431 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc004570 con 0x7fb00c0cd260 2023-07-13T22:31:02.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb002ffd700 1 -- 172.21.15.60:0/1565303431 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc01f070 con 0x7fb00c0cd260 2023-07-13T22:31:02.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb002ffd700 1 -- 172.21.15.60:0/1565303431 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0038c0 con 0x7fb00c0cd260 2023-07-13T22:31:02.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c15a180 con 0x7fb00c0cd260 2023-07-13T22:31:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.188+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c158280 con 0x7fb00c0cd260 2023-07-13T22:31:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.189+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c04e280 con 0x7fb00c0cd260 2023-07-13T22:31:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.190+0000 7fb002ffd700 1 -- 172.21.15.60:0/1565303431 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faffc021030 con 0x7fb00c0cd260 2023-07-13T22:31:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.191+0000 7fb002ffd700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafec06ace0 0x7fafec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.191+0000 7fb002ffd700 1 -- 172.21.15.60:0/1565303431 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faffc08a9e0 con 0x7fb00c0cd260 2023-07-13T22:31:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.192+0000 7fb003fff700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafec06ace0 0x7fafec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.192+0000 7fb003fff700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafec06ace0 0x7fafec06d190 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7faff4000f60 tx=0x7faff4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:02.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.196+0000 7fb002ffd700 1 -- 172.21.15.60:0/1565303431 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7faffc055a80 con 0x7fb00c0cd260 2023-07-13T22:31:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.447+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb00c073860 con 0x7fb00c0cd260 2023-07-13T22:31:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.448+0000 7fb002ffd700 1 -- 172.21.15.60:0/1565303431 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7faffc015150 con 0x7fb00c0cd260 2023-07-13T22:31:02.429 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:02.429 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:31:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafec06ace0 msgr2=0x7fafec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafec06ace0 0x7fafec06d190 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7faff4000f60 tx=0x7faff4009040 comp rx=0 tx=0).stop 2023-07-13T22:31:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cd260 msgr2=0x7fb00c1577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cd260 0x7fb00c1577d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7faffc000c00 tx=0x7faffc004750 comp rx=0 tx=0).stop 2023-07-13T22:31:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 shutdown_connections 2023-07-13T22:31:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fafec06ace0 0x7fafec06d190 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb00c0cd260 0x7fb00c1577d0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00c0d0e20 0x7fb00c0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 --2- 172.21.15.60:0/1565303431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb00c0d0a50 0x7fb00c158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 >> 172.21.15.60:0/1565303431 conn(0x7fb00c071600 msgr2=0x7fb00c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:02.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 shutdown_connections 2023-07-13T22:31:02.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.450+0000 7fb0109ff700 1 -- 172.21.15.60:0/1565303431 wait complete. 2023-07-13T22:31:02.449 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:31:02.449 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-07-13T22:31:02.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:31:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.671+0000 7f0d1e1ff700 1 Processor -- start 2023-07-13T22:31:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.671+0000 7f0d1e1ff700 1 -- start start 2023-07-13T22:31:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.672+0000 7f0d1e1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 0x7f0d180d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.672+0000 7f0d1e1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1815fe20 con 0x7f0d180b2bc0 2023-07-13T22:31:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.672+0000 7f0d1e1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1815cb50 con 0x7f0d180cdb20 2023-07-13T22:31:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.672+0000 7f0d1e1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d180cdef0 con 0x7f0d180ce600 2023-07-13T22:31:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d16ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0d180b2bc0 0x7f0d181643b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60450/0 (socket says 172.21.15.60:60450) 2023-07-13T22:31:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d16ffd700 1 -- 172.21.15.60:0/759144534 learned_addr learned my addr 172.21.15.60:0/759144534 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d17fff700 1 --2- 172.21.15.60:0/759144534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 0x7f0d180d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d167fc700 1 -- 172.21.15.60:0/759144534 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435825733 0 0) 0x7f0d0c001110 con 0x7f0d180b2bc0 2023-07-13T22:31:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d167fc700 1 -- 172.21.15.60:0/759144534 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d000035b0 con 0x7f0d180b2bc0 2023-07-13T22:31:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d167fc700 1 -- 172.21.15.60:0/759144534 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064396294 0 0) 0x7f0d1815cb50 con 0x7f0d180cdb20 2023-07-13T22:31:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d167fc700 1 -- 172.21.15.60:0/759144534 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d0c001110 con 0x7f0d180cdb20 2023-07-13T22:31:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d17fff700 1 -- 172.21.15.60:0/759144534 >> v1:172.21.15.105:6789/0 conn(0x7f0d180cdb20 legacy=0x7f0d180b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d17fff700 1 -- 172.21.15.60:0/759144534 >> v1:172.21.15.60:6789/0 conn(0x7f0d180b2bc0 legacy=0x7f0d181643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.673+0000 7f0d17fff700 1 -- 172.21.15.60:0/759144534 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1815cdc0 con 0x7f0d180ce600 2023-07-13T22:31:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.674+0000 7f0d17fff700 1 --2- 172.21.15.60:0/759144534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 0x7f0d180d0320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009dc0 tx=0x7f0d1000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9602e49ad86639 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.674+0000 7f0d167fc700 1 -- 172.21.15.60:0/759144534 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1000e070 con 0x7f0d180ce600 2023-07-13T22:31:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.674+0000 7f0d167fc700 1 -- 172.21.15.60:0/759144534 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1000bb20 con 0x7f0d180ce600 2023-07-13T22:31:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.674+0000 7f0d167fc700 1 -- 172.21.15.60:0/759144534 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10012430 con 0x7f0d180ce600 2023-07-13T22:31:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.674+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/759144534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 msgr2=0x7f0d180d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.674+0000 7f0d1e1ff700 1 --2- 172.21.15.60:0/759144534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 0x7f0d180d0320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009dc0 tx=0x7f0d1000b330 comp rx=0 tx=0).stop 2023-07-13T22:31:02.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.674+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/759144534 shutdown_connections 2023-07-13T22:31:02.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.674+0000 7f0d1e1ff700 1 --2- 172.21.15.60:0/759144534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 0x7f0d180d0320 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.674+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/759144534 >> 172.21.15.60:0/759144534 conn(0x7f0d1806e660 msgr2=0x7f0d1806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:02.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.675+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/759144534 shutdown_connections 2023-07-13T22:31:02.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.675+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/759144534 wait complete. 2023-07-13T22:31:02.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.676+0000 7f0d1e1ff700 1 Processor -- start 2023-07-13T22:31:02.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.676+0000 7f0d1e1ff700 1 -- start start 2023-07-13T22:31:02.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.677+0000 7f0d1e1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b2bc0 0x7f0d1806b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.677+0000 7f0d1e1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d180cdb20 0x7f0d1814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.677+0000 7f0d1e1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 0x7f0d180c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.677+0000 7f0d1e1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18157a10 con 0x7f0d180cdb20 2023-07-13T22:31:02.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.677+0000 7f0d1e1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1806bfd0 con 0x7f0d180b2bc0 2023-07-13T22:31:02.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.677+0000 7f0d1e1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1806c2b0 con 0x7f0d180ce600 2023-07-13T22:31:02.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d16ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d180cdb20 0x7f0d1814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d16ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d180cdb20 0x7f0d1814fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34626/0 (socket says 172.21.15.60:34626) 2023-07-13T22:31:02.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d16ffd700 1 -- 172.21.15.60:0/3638820238 learned_addr learned my addr 172.21.15.60:0/3638820238 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:02.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d17fff700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 0x7f0d180c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d177fe700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b2bc0 0x7f0d1806b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d16ffd700 1 -- 172.21.15.60:0/3638820238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 msgr2=0x7f0d180c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d16ffd700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 0x7f0d180c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d16ffd700 1 -- 172.21.15.60:0/3638820238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b2bc0 msgr2=0x7f0d1806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d16ffd700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b2bc0 0x7f0d1806b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d16ffd700 1 -- 172.21.15.60:0/3638820238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10009a70 con 0x7f0d180cdb20 2023-07-13T22:31:02.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d177fe700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b2bc0 0x7f0d1806b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.678+0000 7f0d16ffd700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d180cdb20 0x7f0d1814fd70 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005910 tx=0x7f0d0c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.679+0000 7f0cfffff700 1 -- 172.21.15.60:0/3638820238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c005d70 con 0x7f0d180cdb20 2023-07-13T22:31:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.679+0000 7f0cfffff700 1 -- 172.21.15.60:0/3638820238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c007070 con 0x7f0d180cdb20 2023-07-13T22:31:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.679+0000 7f0cfffff700 1 -- 172.21.15.60:0/3638820238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c007420 con 0x7f0d180cdb20 2023-07-13T22:31:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.679+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1806c590 con 0x7f0d180cdb20 2023-07-13T22:31:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.679+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d180c68a0 con 0x7f0d180cdb20 2023-07-13T22:31:02.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.680+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18157870 con 0x7f0d180cdb20 2023-07-13T22:31:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.681+0000 7f0cfffff700 1 -- 172.21.15.60:0/3638820238 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d0c007580 con 0x7f0d180cdb20 2023-07-13T22:31:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.682+0000 7f0cfffff700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d0406ace0 0x7f0d0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.682+0000 7f0cfffff700 1 -- 172.21.15.60:0/3638820238 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d0c0889a0 con 0x7f0d180cdb20 2023-07-13T22:31:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.682+0000 7f0d177fe700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d0406ace0 0x7f0d0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:02.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.683+0000 7f0d177fe700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d0406ace0 0x7f0d0406d190 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0d080011e0 tx=0x7f0d08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.686+0000 7f0cfffff700 1 -- 172.21.15.60:0/3638820238 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0d18157870 con 0x7f0d180cdb20 2023-07-13T22:31:02.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.940+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0d18157c10 con 0x7f0d180cdb20 2023-07-13T22:31:02.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.940+0000 7f0cfffff700 1 -- 172.21.15.60:0/3638820238 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f0d18157c10 con 0x7f0d180cdb20 2023-07-13T22:31:02.921 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:02.922 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-13T22:31:02.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d0406ace0 msgr2=0x7f0d0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d0406ace0 0x7f0d0406d190 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0d080011e0 tx=0x7f0d08009040 comp rx=0 tx=0).stop 2023-07-13T22:31:02.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d180cdb20 msgr2=0x7f0d1814fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:02.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d180cdb20 0x7f0d1814fd70 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005910 tx=0x7f0d0c00c9e0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 shutdown_connections 2023-07-13T22:31:02.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0d0406ace0 0x7f0d0406d190 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0d180ce600 0x7f0d180c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d180b2bc0 0x7f0d1806b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 --2- 172.21.15.60:0/3638820238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d180cdb20 0x7f0d1814fd70 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:02.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 >> 172.21.15.60:0/3638820238 conn(0x7f0d1806e660 msgr2=0x7f0d1815f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:02.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 shutdown_connections 2023-07-13T22:31:02.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:02.942+0000 7f0d1e1ff700 1 -- 172.21.15.60:0/3638820238 wait complete. 2023-07-13T22:31:02.941 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:31:02.942 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2023-07-13T22:31:02.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:03.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.220+0000 7fd773fff700 1 Processor -- start 2023-07-13T22:31:03.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.221+0000 7fd773fff700 1 -- start start 2023-07-13T22:31:03.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.222+0000 7fd773fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 0x7fd76c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:03.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.222+0000 7fd773fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd76c0cfad0 0x7fd76c0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:03.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.222+0000 7fd773fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c0cd950 con 0x7fd76c0b8400 2023-07-13T22:31:03.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.222+0000 7fd773fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76c0cff20 con 0x7fd76c0cfad0 2023-07-13T22:31:03.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.222+0000 7fd773fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c0d0ac0 con 0x7fd76c0d3750 2023-07-13T22:31:03.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.222+0000 7fd771d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd76c0b8400 0x7fd76c0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60466/0 (socket says 172.21.15.60:60466) 2023-07-13T22:31:03.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.222+0000 7fd771d9b700 1 -- 172.21.15.60:0/2491170355 learned_addr learned my addr 172.21.15.60:0/2491170355 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.222+0000 7fd77259c700 1 --2- 172.21.15.60:0/2491170355 >> v2:172.21.15.60:3300/0 conn(0x7fd76c0cfad0 0x7fd76c0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.222+0000 7fd77159a700 1 --2- 172.21.15.60:0/2491170355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 0x7fd76c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.223+0000 7fd77159a700 1 -- 172.21.15.60:0/2491170355 >> v2:172.21.15.60:3300/0 conn(0x7fd76c0cfad0 msgr2=0x7fd76c0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.223+0000 7fd77159a700 1 --2- 172.21.15.60:0/2491170355 >> v2:172.21.15.60:3300/0 conn(0x7fd76c0cfad0 0x7fd76c0d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.223+0000 7fd77159a700 1 -- 172.21.15.60:0/2491170355 >> v1:172.21.15.60:6789/0 conn(0x7fd76c0b8400 legacy=0x7fd76c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.223+0000 7fd770d99700 1 -- 172.21.15.60:0/2491170355 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929597040 0 0) 0x7fd768001110 con 0x7fd76c0b8400 2023-07-13T22:31:03.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.223+0000 7fd77159a700 1 -- 172.21.15.60:0/2491170355 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c0b87d0 con 0x7fd76c0d3750 2023-07-13T22:31:03.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.223+0000 7fd77159a700 1 --2- 172.21.15.60:0/2491170355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 0x7fd76c0cf590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009840 tx=0x7fd75c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e313b63aba12098 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:03.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.224+0000 7fd770d99700 1 -- 172.21.15.60:0/2491170355 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c011830 con 0x7fd76c0d3750 2023-07-13T22:31:03.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.224+0000 7fd770d99700 1 -- 172.21.15.60:0/2491170355 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c011990 con 0x7fd76c0d3750 2023-07-13T22:31:03.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.224+0000 7fd770d99700 1 -- 172.21.15.60:0/2491170355 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c011d20 con 0x7fd76c0d3750 2023-07-13T22:31:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.225+0000 7fd773fff700 1 -- 172.21.15.60:0/2491170355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 msgr2=0x7fd76c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.225+0000 7fd773fff700 1 --2- 172.21.15.60:0/2491170355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 0x7fd76c0cf590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009840 tx=0x7fd75c009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.225+0000 7fd773fff700 1 -- 172.21.15.60:0/2491170355 shutdown_connections 2023-07-13T22:31:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.225+0000 7fd773fff700 1 --2- 172.21.15.60:0/2491170355 >> v2:172.21.15.60:3300/0 conn(0x7fd76c0cfad0 0x7fd76c0d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.225+0000 7fd773fff700 1 --2- 172.21.15.60:0/2491170355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 0x7fd76c0cf590 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.225+0000 7fd773fff700 1 -- 172.21.15.60:0/2491170355 >> 172.21.15.60:0/2491170355 conn(0x7fd76c06b390 msgr2=0x7fd76c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:03.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.225+0000 7fd773fff700 1 -- 172.21.15.60:0/2491170355 shutdown_connections 2023-07-13T22:31:03.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.225+0000 7fd773fff700 1 -- 172.21.15.60:0/2491170355 wait complete. 2023-07-13T22:31:03.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.228+0000 7fd773fff700 1 Processor -- start 2023-07-13T22:31:03.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.228+0000 7fd773fff700 1 -- start start 2023-07-13T22:31:03.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd773fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd76c0b8400 0x7fd76c150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:03.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd773fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd76c0cfad0 0x7fd76c0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:03.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd773fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 0x7fd76c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:03.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd773fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c150c50 con 0x7fd76c0b8400 2023-07-13T22:31:03.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd773fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c15e490 con 0x7fd76c0cfad0 2023-07-13T22:31:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd773fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c0c7700 con 0x7fd76c0d3750 2023-07-13T22:31:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd771d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd76c0b8400 0x7fd76c150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd771d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd76c0b8400 0x7fd76c150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34650/0 (socket says 172.21.15.60:34650) 2023-07-13T22:31:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd771d9b700 1 -- 172.21.15.60:0/903729336 learned_addr learned my addr 172.21.15.60:0/903729336 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:03.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd77259c700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 0x7fd76c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:03.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd77159a700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd76c0cfad0 0x7fd76c0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:03.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd771d9b700 1 -- 172.21.15.60:0/903729336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 msgr2=0x7fd76c0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:03.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd771d9b700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 0x7fd76c0b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:03.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd771d9b700 1 -- 172.21.15.60:0/903729336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd76c0cfad0 msgr2=0x7fd76c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:03.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd771d9b700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd76c0cfad0 0x7fd76c0b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:03.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.229+0000 7fd771d9b700 1 -- 172.21.15.60:0/903729336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c0094a0 con 0x7fd76c0b8400 2023-07-13T22:31:03.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.230+0000 7fd771d9b700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd76c0b8400 0x7fd76c150710 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd76800c550 tx=0x7fd76800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:03.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.230+0000 7fd7627fc700 1 -- 172.21.15.60:0/903729336 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768004c80 con 0x7fd76c0b8400 2023-07-13T22:31:03.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.230+0000 7fd7627fc700 1 -- 172.21.15.60:0/903729336 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd768004e00 con 0x7fd76c0b8400 2023-07-13T22:31:03.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.230+0000 7fd7627fc700 1 -- 172.21.15.60:0/903729336 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768007310 con 0x7fd76c0b8400 2023-07-13T22:31:03.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.230+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c0c79e0 con 0x7fd76c0b8400 2023-07-13T22:31:03.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.230+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76c0c7fa0 con 0x7fd76c0b8400 2023-07-13T22:31:03.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.231+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76c0d1510 con 0x7fd76c0b8400 2023-07-13T22:31:03.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.232+0000 7fd7627fc700 1 -- 172.21.15.60:0/903729336 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd768007470 con 0x7fd76c0b8400 2023-07-13T22:31:03.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.234+0000 7fd7627fc700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd75006ace0 0x7fd75006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:03.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.234+0000 7fd7627fc700 1 -- 172.21.15.60:0/903729336 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd768088f80 con 0x7fd76c0b8400 2023-07-13T22:31:03.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.234+0000 7fd77159a700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd75006ace0 0x7fd75006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:03.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.235+0000 7fd77159a700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd75006ace0 0x7fd75006d190 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0d3b60 tx=0x7fd75c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:03.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.239+0000 7fd7627fc700 1 -- 172.21.15.60:0/903729336 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd7680bb330 con 0x7fd76c0b8400 2023-07-13T22:31:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.491+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd76c0cd950 con 0x7fd76c0b8400 2023-07-13T22:31:03.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.491+0000 7fd7627fc700 1 -- 172.21.15.60:0/903729336 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4714 (secure 0 0 0) 0x7fd76805b280 con 0x7fd76c0b8400 2023-07-13T22:31:03.473 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:03.473 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:30:39.690428+0000","modified":"2023-07-13T22:31:00.559611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6836/1781435251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1781435251},{"type":"v1","addr":"172.21.15.60:6836","nonce":1781435251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-13T22:31:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd75006ace0 msgr2=0x7fd75006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd75006ace0 0x7fd75006d190 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0d3b60 tx=0x7fd75c01c040 comp rx=0 tx=0).stop 2023-07-13T22:31:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd76c0b8400 msgr2=0x7fd76c150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd76c0b8400 0x7fd76c150710 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd76800c550 tx=0x7fd76800c910 comp rx=0 tx=0).stop 2023-07-13T22:31:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 shutdown_connections 2023-07-13T22:31:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd75006ace0 0x7fd75006d190 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd76c0b8400 0x7fd76c150710 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd76c0d3750 0x7fd76c0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 --2- 172.21.15.60:0/903729336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd76c0cfad0 0x7fd76c0b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 >> 172.21.15.60:0/903729336 conn(0x7fd76c06b390 msgr2=0x7fd76c15a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 shutdown_connections 2023-07-13T22:31:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:03.494+0000 7fd773fff700 1 -- 172.21.15.60:0/903729336 wait complete. 2023-07-13T22:31:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 87 2023-07-13T22:31:03.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-07-13T22:31:03.719 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":8422,"entity":{"name":{"type":"client","num":8422},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":814386305}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":12.948865555999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9328243339428188,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.8422 192.168.0.1:0/814386305","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"1","hostname":"smithi105","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"127147","root":"/"}},{"id":9084,"entity":{"name":{"type":"client","num":9084},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3195742970}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":1101,"uptime":15.285841292000001,"requests_in_flight":0,"num_completed_requests":125,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1503.2709324916582,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":414.98310820779074,"halflife":1.5},"recall_caps_throttle2o":{"value":131.08323331072387,"halflife":0.5},"session_cache_liveness":{"value":1179.8403879165592,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":831,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.9084 192.168.0.1:0/3195742970","completed_requests":[{"tid":831,"created_ino":"0x10000000307"},{"tid":834,"created_ino":"0x10000000308"},{"tid":837,"created_ino":"0x10000000309"},{"tid":840,"created_ino":"0x1000000030a"},{"tid":843,"created_ino":"0x1000000030b"},{"tid":846,"created_ino":"0x1000000030c"},{"tid":849,"created_ino":"0x1000000030d"},{"tid":852,"created_ino":"0x1000000030e"},{"tid":855,"created_ino":"0x1000000030f"},{"tid":858,"created_ino":"0x10000000310"},{"tid":861,"created_ino":"0x10000000311"},{"tid":864,"created_ino":"0x10000000312"},{"tid":867,"created_ino":"0x10000000313"},{"tid":870,"created_ino":"0x10000000314"},{"tid":873,"created_ino":"0x10000000315"},{"tid":876,"created_ino":"0x10000000316"},{"tid":879,"created_ino":"0x10000000317"},{"tid":882,"created_ino":"0x10000000318"},{"tid":885,"created_ino":"0x10000000319"},{"tid":888,"created_ino":"0x1000000031a"},{"tid":891,"created_ino":"0x1000000031b"},{"tid":894,"created_ino":"0x1000000031c"},{"tid":897,"created_ino":"0x1000000031d"},{"tid":900,"created_ino":"0x1000000031e"},{"tid":903,"created_ino":"0x1000000031f"},{"tid":906,"created_ino":"0x10000000320"},{"tid":909,"created_ino":"0x10000000321"},{"tid":912,"created_ino":"0x10000000322"},{"tid":915,"created_ino":"0x10000000323"},{"tid":918,"created_ino":"0x10000000324"},{"tid":921,"created_ino":"0x10000000325"},{"tid":924,"created_ino":"0x10000000326"},{"tid":927,"created_ino":"0x10000000327"},{"tid":930,"created_ino":"0x10000000328"},{"tid":933,"created_ino":"0x10000000329"},{"tid":936,"created_ino":"0x1000000032a"},{"tid":939,"created_ino":"0x1000000032b"},{"tid":942,"created_ino":"0x1000000032c"},{"tid":945,"created_ino":"0x1000000032d"},{"tid":948,"created_ino":"0x1000000032e"},{"tid":951,"created_ino":"0x1000000032f"},{"tid":954,"created_ino":"0x10000000330"},{"tid":957,"created_ino":"0x10000000331"},{"tid":960,"created_ino":"0x10000000332"},{"tid":963,"created_ino":"0x10000000333"},{"tid":966,"created_ino":"0x10000000334"},{"tid":969,"created_ino":"0x10000000335"},{"tid":972,"created_ino":"0x10000000336"},{"tid":975,"created_ino":"0x10000000337"},{"tid":978,"created_ino":"0x10000000338"},{"tid":981,"created_ino":"0x10000000339"},{"tid":984,"created_ino":"0x1000000033a"},{"tid":987,"created_ino":"0x1000000033b"},{"tid":990,"created_ino":"0x1000000033c"},{"tid":993,"created_ino":"0x1000000033d"},{"tid":996,"created_ino":"0x1000000033e"},{"tid":999,"created_ino":"0x1000000033f"},{"tid":1002,"created_ino":"0x10000000340"},{"tid":1005,"created_ino":"0x10000000341"},{"tid":1008,"created_ino":"0x10000000342"},{"tid":1011,"created_ino":"0x10000000343"},{"tid":1014,"created_ino":"0x10000000344"},{"tid":1017,"created_ino":"0x10000000345"},{"tid":1020,"created_ino":"0x10000000346"},{"tid":1023,"created_ino":"0x10000000347"},{"tid":1026,"created_ino":"0x10000000348"},{"tid":1029,"created_ino":"0x10000000349"},{"tid":1032,"created_ino":"0x1000000034a"},{"tid":1035,"created_ino":"0x1000000034b"},{"tid":1038,"created_ino":"0x1000000034c"},{"tid":1041,"created_ino":"0x1000000034d"},{"tid":1044,"created_ino":"0x1000000034e"},{"tid":1047,"created_ino":"0x1000000034f"},{"tid":1050,"created_ino":"0x10000000350"},{"tid":1053,"created_ino":"0x10000000351"},{"tid":1056,"created_ino":"0x10000000352"},{"tid":1059,"created_ino":"0x10000000353"},{"tid":1062,"created_ino":"0x10000000354"},{"tid":1065,"created_ino":"0x10000000355"},{"tid":1068,"created_ino":"0x10000000356"},{"tid":1071,"created_ino":"0x10000000357"},{"tid":1074,"created_ino":"0x10000000358"},{"tid":1077,"created_ino":"0x10000000359"},{"tid":1080,"created_ino":"0x1000000035a"},{"tid":1083,"created_ino":"0x1000000035b"},{"tid":1086,"created_ino":"0x1000000035c"},{"tid":1089,"created_ino":"0x1000000035d"},{"tid":1092,"created_ino":"0x1000000035e"},{"tid":1095,"created_ino":"0x1000000035f"},{"tid":1098,"created_ino":"0x10000000360"},{"tid":1101,"created_ino":"0x10000000361"},{"tid":1104,"created_ino":"0x10000000362"},{"tid":1107,"created_ino":"0x10000000363"},{"tid":1110,"created_ino":"0x10000000364"},{"tid":1113,"created_ino":"0x10000000365"},{"tid":1116,"created_ino":"0x10000000366"},{"tid":1119,"created_ino":"0x10000000367"},{"tid":1122,"created_ino":"0x10000000368"},{"tid":1125,"created_ino":"0x10000000369"},{"tid":1128,"created_ino":"0x1000000036a"},{"tid":1131,"created_ino":"0x1000000036b"},{"tid":1134,"created_ino":"0x1000000036c"},{"tid":1137,"created_ino":"0x1000000036d"},{"tid":1140,"created_ino":"0x1000000036e"},{"tid":1143,"created_ino":"0x1000000036f"},{"tid":1146,"created_ino":"0x10000000370"},{"tid":1149,"created_ino":"0x10000000371"},{"tid":1152,"created_ino":"0x10000000372"},{"tid":1155,"created_ino":"0x10000000373"},{"tid":1158,"created_ino":"0x10000000374"},{"tid":1161,"created_ino":"0x10000000375"},{"tid":1164,"created_ino":"0x10000000376"},{"tid":1167,"created_ino":"0x10000000377"},{"tid":1170,"created_ino":"0x10000000378"},{"tid":1173,"created_ino":"0x10000000379"},{"tid":1176,"created_ino":"0x1000000037a"},{"tid":1179,"created_ino":"0x1000000037b"},{"tid":1182,"created_ino":"0x1000000037c"},{"tid":1185,"created_ino":"0x1000000037d"},{"tid":1188,"created_ino":"0x1000000037e"},{"tid":1191,"created_ino":"0x1000000037f"},{"tid":1194,"created_ino":"0x10000000380"},{"tid":1197,"created_ino":"0x10000000381"},{"tid":1200,"created_ino":"0x10000000382"},{"tid":1203,"created_ino":"0x10000000383"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"3fc0089cc1082313a98853d48b892acdc1385b8b","ceph_version":"ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)","entity_id":"0","hostname":"smithi060","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"161669","root":"/"}}] 2023-07-13T22:31:03.721 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi105", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "127147", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 814386305, "type": "any" }, "name": { "num": 8422, "type": "client" } }, "id": 8422, "inst": "client.8422 192.168.0.1:0/814386305", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.932824333942819 }, "state": "open", "uptime": 12.948865556 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "161669", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000307", "tid": 831 }, { "created_ino": "0x10000000308", "tid": 834 }, { "created_ino": "0x10000000309", "tid": 837 }, { "created_ino": "0x1000000030a", "tid": 840 }, { "created_ino": "0x1000000030b", "tid": 843 }, { "created_ino": "0x1000000030c", "tid": 846 }, { "created_ino": "0x1000000030d", "tid": 849 }, { "created_ino": "0x1000000030e", "tid": 852 }, { "created_ino": "0x1000000030f", "tid": 855 }, { "created_ino": "0x10000000310", "tid": 858 }, { "created_ino": "0x10000000311", "tid": 861 }, { "created_ino": "0x10000000312", "tid": 864 }, { "created_ino": "0x10000000313", "tid": 867 }, { "created_ino": "0x10000000314", "tid": 870 }, { "created_ino": "0x10000000315", "tid": 873 }, { "created_ino": "0x10000000316", "tid": 876 }, { "created_ino": "0x10000000317", "tid": 879 }, { "created_ino": "0x10000000318", "tid": 882 }, { "created_ino": "0x10000000319", "tid": 885 }, { "created_ino": "0x1000000031a", "tid": 888 }, { "created_ino": "0x1000000031b", "tid": 891 }, { "created_ino": "0x1000000031c", "tid": 894 }, { "created_ino": "0x1000000031d", "tid": 897 }, { "created_ino": "0x1000000031e", "tid": 900 }, { "created_ino": "0x1000000031f", "tid": 903 }, { "created_ino": "0x10000000320", "tid": 906 }, { "created_ino": "0x10000000321", "tid": 909 }, { "created_ino": "0x10000000322", "tid": 912 }, { "created_ino": "0x10000000323", "tid": 915 }, { "created_ino": "0x10000000324", "tid": 918 }, { "created_ino": "0x10000000325", "tid": 921 }, { "created_ino": "0x10000000326", "tid": 924 }, { "created_ino": "0x10000000327", "tid": 927 }, { "created_ino": "0x10000000328", "tid": 930 }, { "created_ino": "0x10000000329", "tid": 933 }, { "created_ino": "0x1000000032a", "tid": 936 }, { "created_ino": "0x1000000032b", "tid": 939 }, { "created_ino": "0x1000000032c", "tid": 942 }, { "created_ino": "0x1000000032d", "tid": 945 }, { "created_ino": "0x1000000032e", "tid": 948 }, { "created_ino": "0x1000000032f", "tid": 951 }, { "created_ino": "0x10000000330", "tid": 954 }, { "created_ino": "0x10000000331", "tid": 957 }, { "created_ino": "0x10000000332", "tid": 960 }, { "created_ino": "0x10000000333", "tid": 963 }, { "created_ino": "0x10000000334", "tid": 966 }, { "created_ino": "0x10000000335", "tid": 969 }, { "created_ino": "0x10000000336", "tid": 972 }, { "created_ino": "0x10000000337", "tid": 975 }, { "created_ino": "0x10000000338", "tid": 978 }, { "created_ino": "0x10000000339", "tid": 981 }, { "created_ino": "0x1000000033a", "tid": 984 }, { "created_ino": "0x1000000033b", "tid": 987 }, { "created_ino": "0x1000000033c", "tid": 990 }, { "created_ino": "0x1000000033d", "tid": 993 }, { "created_ino": "0x1000000033e", "tid": 996 }, { "created_ino": "0x1000000033f", "tid": 999 }, { "created_ino": "0x10000000340", "tid": 1002 }, { "created_ino": "0x10000000341", "tid": 1005 }, { "created_ino": "0x10000000342", "tid": 1008 }, { "created_ino": "0x10000000343", "tid": 1011 }, { "created_ino": "0x10000000344", "tid": 1014 }, { "created_ino": "0x10000000345", "tid": 1017 }, { "created_ino": "0x10000000346", "tid": 1020 }, { "created_ino": "0x10000000347", "tid": 1023 }, { "created_ino": "0x10000000348", "tid": 1026 }, { "created_ino": "0x10000000349", "tid": 1029 }, { "created_ino": "0x1000000034a", "tid": 1032 }, { "created_ino": "0x1000000034b", "tid": 1035 }, { "created_ino": "0x1000000034c", "tid": 1038 }, { "created_ino": "0x1000000034d", "tid": 1041 }, { "created_ino": "0x1000000034e", "tid": 1044 }, { "created_ino": "0x1000000034f", "tid": 1047 }, { "created_ino": "0x10000000350", "tid": 1050 }, { "created_ino": "0x10000000351", "tid": 1053 }, { "created_ino": "0x10000000352", "tid": 1056 }, { "created_ino": "0x10000000353", "tid": 1059 }, { "created_ino": "0x10000000354", "tid": 1062 }, { "created_ino": "0x10000000355", "tid": 1065 }, { "created_ino": "0x10000000356", "tid": 1068 }, { "created_ino": "0x10000000357", "tid": 1071 }, { "created_ino": "0x10000000358", "tid": 1074 }, { "created_ino": "0x10000000359", "tid": 1077 }, { "created_ino": "0x1000000035a", "tid": 1080 }, { "created_ino": "0x1000000035b", "tid": 1083 }, { "created_ino": "0x1000000035c", "tid": 1086 }, { "created_ino": "0x1000000035d", "tid": 1089 }, { "created_ino": "0x1000000035e", "tid": 1092 }, { "created_ino": "0x1000000035f", "tid": 1095 }, { "created_ino": "0x10000000360", "tid": 1098 }, { "created_ino": "0x10000000361", "tid": 1101 }, { "created_ino": "0x10000000362", "tid": 1104 }, { "created_ino": "0x10000000363", "tid": 1107 }, { "created_ino": "0x10000000364", "tid": 1110 }, { "created_ino": "0x10000000365", "tid": 1113 }, { "created_ino": "0x10000000366", "tid": 1116 }, { "created_ino": "0x10000000367", "tid": 1119 }, { "created_ino": "0x10000000368", "tid": 1122 }, { "created_ino": "0x10000000369", "tid": 1125 }, { "created_ino": "0x1000000036a", "tid": 1128 }, { "created_ino": "0x1000000036b", "tid": 1131 }, { "created_ino": "0x1000000036c", "tid": 1134 }, { "created_ino": "0x1000000036d", "tid": 1137 }, { "created_ino": "0x1000000036e", "tid": 1140 }, { "created_ino": "0x1000000036f", "tid": 1143 }, { "created_ino": "0x10000000370", "tid": 1146 }, { "created_ino": "0x10000000371", "tid": 1149 }, { "created_ino": "0x10000000372", "tid": 1152 }, { "created_ino": "0x10000000373", "tid": 1155 }, { "created_ino": "0x10000000374", "tid": 1158 }, { "created_ino": "0x10000000375", "tid": 1161 }, { "created_ino": "0x10000000376", "tid": 1164 }, { "created_ino": "0x10000000377", "tid": 1167 }, { "created_ino": "0x10000000378", "tid": 1170 }, { "created_ino": "0x10000000379", "tid": 1173 }, { "created_ino": "0x1000000037a", "tid": 1176 }, { "created_ino": "0x1000000037b", "tid": 1179 }, { "created_ino": "0x1000000037c", "tid": 1182 }, { "created_ino": "0x1000000037d", "tid": 1185 }, { "created_ino": "0x1000000037e", "tid": 1188 }, { "created_ino": "0x1000000037f", "tid": 1191 }, { "created_ino": "0x10000000380", "tid": 1194 }, { "created_ino": "0x10000000381", "tid": 1197 }, { "created_ino": "0x10000000382", "tid": 1200 }, { "created_ino": "0x10000000383", "tid": 1203 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3195742970, "type": "any" }, "name": { "num": 9084, "type": "client" } }, "id": 9084, "inst": "client.9084 192.168.0.1:0/3195742970", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 831, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 125, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1503.2709324916582 }, "recall_caps_throttle": { "halflife": 1.5, "value": 414.98310820779074 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 131.08323331072387 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1101, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1179.8403879165592 }, "state": "open", "uptime": 15.285841292 } ] 2023-07-13T22:31:03.721 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-13T22:31:03.721 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:31:03.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:31:03.820 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:31:03.926 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:31:03.927 INFO:tasks.cephfs.mount:Terminating background process 2023-07-13T22:31:03.927 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:31:03.928 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:03.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:04.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.241+0000 7f9920bff700 1 Processor -- start 2023-07-13T22:31:04.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.241+0000 7f9920bff700 1 -- start start 2023-07-13T22:31:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.243+0000 7f9920bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f991c15a820 0x7f991c15ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.243+0000 7f9920bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f991c0c8d40 0x7f991c0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.243+0000 7f9920bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c070270 con 0x7f991c154950 2023-07-13T22:31:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.243+0000 7f9920bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c0713d0 con 0x7f991c15a820 2023-07-13T22:31:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.243+0000 7f9920bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c0c96a0 con 0x7f991c0c8d40 2023-07-13T22:31:04.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.243+0000 7f9919d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f991c15a820 0x7f991c15ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.243+0000 7f9919d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f991c15a820 0x7f991c15ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34662/0 (socket says 172.21.15.60:34662) 2023-07-13T22:31:04.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.243+0000 7f9919d9b700 1 -- 172.21.15.60:0/3963994446 learned_addr learned my addr 172.21.15.60:0/3963994446 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:04.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.243+0000 7f991ad9d700 1 --2- 172.21.15.60:0/3963994446 >> v2:172.21.15.105:3300/0 conn(0x7f991c0c8d40 0x7f991c0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.244+0000 7f9919d9b700 1 -- 172.21.15.60:0/3963994446 >> v2:172.21.15.105:3300/0 conn(0x7f991c0c8d40 msgr2=0x7f991c0c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:04.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.244+0000 7f9919d9b700 1 --2- 172.21.15.60:0/3963994446 >> v2:172.21.15.105:3300/0 conn(0x7f991c0c8d40 0x7f991c0c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.244+0000 7f9919d9b700 1 -- 172.21.15.60:0/3963994446 >> v1:172.21.15.105:6789/0 conn(0x7f991c154950 legacy=0x7f991c070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.244+0000 7f9919d9b700 1 -- 172.21.15.60:0/3963994446 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c15b2e0 con 0x7f991c15a820 2023-07-13T22:31:04.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.244+0000 7f9919d9b700 1 --2- 172.21.15.60:0/3963994446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c15a820 0x7f991c15ac70 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f991c06b060 tx=0x7f9910009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eec71faceadbb1c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:04.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.244+0000 7f991959a700 1 -- 172.21.15.60:0/3963994446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910011830 con 0x7f991c15a820 2023-07-13T22:31:04.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.244+0000 7f991959a700 1 -- 172.21.15.60:0/3963994446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9910011990 con 0x7f991c15a820 2023-07-13T22:31:04.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.244+0000 7f991959a700 1 -- 172.21.15.60:0/3963994446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910011d20 con 0x7f991c15a820 2023-07-13T22:31:04.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.245+0000 7f9920bff700 1 -- 172.21.15.60:0/3963994446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c15a820 msgr2=0x7f991c15ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.245+0000 7f9920bff700 1 --2- 172.21.15.60:0/3963994446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c15a820 0x7f991c15ac70 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f991c06b060 tx=0x7f9910009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.245+0000 7f9920bff700 1 -- 172.21.15.60:0/3963994446 shutdown_connections 2023-07-13T22:31:04.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.245+0000 7f9920bff700 1 --2- 172.21.15.60:0/3963994446 >> v2:172.21.15.105:3300/0 conn(0x7f991c0c8d40 0x7f991c0c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.245+0000 7f9920bff700 1 --2- 172.21.15.60:0/3963994446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c15a820 0x7f991c15ac70 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.245+0000 7f9920bff700 1 -- 172.21.15.60:0/3963994446 >> 172.21.15.60:0/3963994446 conn(0x7f991c0cf410 msgr2=0x7f991c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:04.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.245+0000 7f9920bff700 1 -- 172.21.15.60:0/3963994446 shutdown_connections 2023-07-13T22:31:04.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.245+0000 7f9920bff700 1 -- 172.21.15.60:0/3963994446 wait complete. 2023-07-13T22:31:04.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.247+0000 7f9920bff700 1 Processor -- start 2023-07-13T22:31:04.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.247+0000 7f9920bff700 1 -- start start 2023-07-13T22:31:04.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.248+0000 7f9920bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f991c0c8d40 0x7f991c1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.248+0000 7f9920bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c154950 0x7f991c0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.248+0000 7f9920bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c15a820 0x7f991c0c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.248+0000 7f9920bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c150ad0 con 0x7f991c154950 2023-07-13T22:31:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.248+0000 7f9920bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0b0320 con 0x7f991c15a820 2023-07-13T22:31:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.248+0000 7f9920bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0b0600 con 0x7f991c0c8d40 2023-07-13T22:31:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.248+0000 7f9919d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c154950 0x7f991c0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f9919d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c154950 0x7f991c0ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34674/0 (socket says 172.21.15.60:34674) 2023-07-13T22:31:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f9919d9b700 1 -- 172.21.15.60:0/2609348138 learned_addr learned my addr 172.21.15.60:0/2609348138 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f991a59c700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f991c0c8d40 0x7f991c1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f991ad9d700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c15a820 0x7f991c0c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f9919d9b700 1 -- 172.21.15.60:0/2609348138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f991c0c8d40 msgr2=0x7f991c1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f9919d9b700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f991c0c8d40 0x7f991c1593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f9919d9b700 1 -- 172.21.15.60:0/2609348138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c15a820 msgr2=0x7f991c0c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f9919d9b700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c15a820 0x7f991c0c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f9919d9b700 1 -- 172.21.15.60:0/2609348138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9910009550 con 0x7f991c154950 2023-07-13T22:31:04.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.249+0000 7f9919d9b700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c154950 0x7f991c0ae340 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f9910009c10 tx=0x7f9910004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:04.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.250+0000 7f990affd700 1 -- 172.21.15.60:0/2609348138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100036a0 con 0x7f991c154950 2023-07-13T22:31:04.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.250+0000 7f990affd700 1 -- 172.21.15.60:0/2609348138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991001e070 con 0x7f991c154950 2023-07-13T22:31:04.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.250+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c0b08e0 con 0x7f991c154950 2023-07-13T22:31:04.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.250+0000 7f990affd700 1 -- 172.21.15.60:0/2609348138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910003a60 con 0x7f991c154950 2023-07-13T22:31:04.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.250+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c0b0ea0 con 0x7f991c154950 2023-07-13T22:31:04.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.251+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c066700 con 0x7f991c154950 2023-07-13T22:31:04.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.253+0000 7f990affd700 1 -- 172.21.15.60:0/2609348138 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9910003bc0 con 0x7f991c154950 2023-07-13T22:31:04.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.254+0000 7f990affd700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f98fc06ace0 0x7f98fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.255+0000 7f990affd700 1 -- 172.21.15.60:0/2609348138 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f991008a430 con 0x7f991c154950 2023-07-13T22:31:04.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.255+0000 7f991a59c700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f98fc06ace0 0x7f98fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.256+0000 7f991a59c700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f98fc06ace0 0x7f98fc06d190 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f990c0016c0 tx=0x7f990c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:04.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.259+0000 7f990affd700 1 -- 172.21.15.60:0/2609348138 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f99100554d0 con 0x7f991c154950 2023-07-13T22:31:04.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.468+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f991c151ca0 con 0x7f991c154950 2023-07-13T22:31:04.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.469+0000 7f990affd700 1 -- 172.21.15.60:0/2609348138 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18215 (secure 0 0 0) 0x7f9910058af0 con 0x7f991c154950 2023-07-13T22:31:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:04.452 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":124,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:30:41.182757+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:30:37.983326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-13T22:30:38.665534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:04.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f98fc06ace0 msgr2=0x7f98fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f98fc06ace0 0x7f98fc06d190 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f990c0016c0 tx=0x7f990c009040 comp rx=0 tx=0).stop 2023-07-13T22:31:04.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c154950 msgr2=0x7f991c0ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c154950 0x7f991c0ae340 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f9910009c10 tx=0x7f9910004750 comp rx=0 tx=0).stop 2023-07-13T22:31:04.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 shutdown_connections 2023-07-13T22:31:04.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f98fc06ace0 0x7f98fc06d190 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f991c15a820 0x7f991c0c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f991c0c8d40 0x7f991c1593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 --2- 172.21.15.60:0/2609348138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f991c154950 0x7f991c0ae340 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 >> 172.21.15.60:0/2609348138 conn(0x7f991c0cf410 msgr2=0x7f991c0adad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:04.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 shutdown_connections 2023-07-13T22:31:04.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.472+0000 7f9920bff700 1 -- 172.21.15.60:0/2609348138 wait complete. 2023-07-13T22:31:04.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:30:37.983326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:30:38.665534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:31:04.472 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:31:04.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.698+0000 7fc3e25ff700 1 Processor -- start 2023-07-13T22:31:04.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.698+0000 7fc3e25ff700 1 -- start start 2023-07-13T22:31:04.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.699+0000 7fc3e25ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc3dc0cfd50 0x7fc3dc0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.699+0000 7fc3e25ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3dc0b1ac0 0x7fc3dc0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.699+0000 7fc3e25ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3dc0d1870 con 0x7fc3dc0b8200 2023-07-13T22:31:04.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.699+0000 7fc3e25ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3dc0cf370 con 0x7fc3dc0cfd50 2023-07-13T22:31:04.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.699+0000 7fc3e25ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc0d12b0 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.699+0000 7fc3db7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc3dc0cfd50 0x7fc3dc0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.699+0000 7fc3db7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc3dc0cfd50 0x7fc3dc0d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34680/0 (socket says 172.21.15.60:34680) 2023-07-13T22:31:04.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.699+0000 7fc3db7fe700 1 -- 172.21.15.60:0/3429870262 learned_addr learned my addr 172.21.15.60:0/3429870262 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:04.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.699+0000 7fc3e0b9c700 1 --2- 172.21.15.60:0/3429870262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3dc0b1ac0 0x7fc3dc0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.700+0000 7fc3db7fe700 1 -- 172.21.15.60:0/3429870262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3dc0b1ac0 msgr2=0x7fc3dc0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:04.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.700+0000 7fc3db7fe700 1 --2- 172.21.15.60:0/3429870262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3dc0b1ac0 0x7fc3dc0d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.700+0000 7fc3db7fe700 1 -- 172.21.15.60:0/3429870262 >> v1:172.21.15.105:6789/0 conn(0x7fc3dc0b8200 legacy=0x7fc3dc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.700+0000 7fc3db7fe700 1 -- 172.21.15.60:0/3429870262 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0d1590 con 0x7fc3dc0cfd50 2023-07-13T22:31:04.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.700+0000 7fc3db7fe700 1 --2- 172.21.15.60:0/3429870262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0cfd50 0x7fc3dc0d23c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00098f0 tx=0x7fc3d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e3313b17e0c473 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:04.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.700+0000 7fc3daffd700 1 -- 172.21.15.60:0/3429870262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000b8a0 con 0x7fc3dc0cfd50 2023-07-13T22:31:04.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.700+0000 7fc3daffd700 1 -- 172.21.15.60:0/3429870262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d000ba00 con 0x7fc3dc0cfd50 2023-07-13T22:31:04.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.700+0000 7fc3daffd700 1 -- 172.21.15.60:0/3429870262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000bd90 con 0x7fc3dc0cfd50 2023-07-13T22:31:04.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.701+0000 7fc3e25ff700 1 -- 172.21.15.60:0/3429870262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0cfd50 msgr2=0x7fc3dc0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.701+0000 7fc3e25ff700 1 --2- 172.21.15.60:0/3429870262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0cfd50 0x7fc3dc0d23c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00098f0 tx=0x7fc3d0009c00 comp rx=0 tx=0).stop 2023-07-13T22:31:04.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.701+0000 7fc3e25ff700 1 -- 172.21.15.60:0/3429870262 shutdown_connections 2023-07-13T22:31:04.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.701+0000 7fc3e25ff700 1 --2- 172.21.15.60:0/3429870262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3dc0b1ac0 0x7fc3dc0d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.701+0000 7fc3e25ff700 1 --2- 172.21.15.60:0/3429870262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0cfd50 0x7fc3dc0d23c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.701+0000 7fc3e25ff700 1 -- 172.21.15.60:0/3429870262 >> 172.21.15.60:0/3429870262 conn(0x7fc3dc06b2f0 msgr2=0x7fc3dc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:04.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.701+0000 7fc3e25ff700 1 -- 172.21.15.60:0/3429870262 shutdown_connections 2023-07-13T22:31:04.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.701+0000 7fc3e25ff700 1 -- 172.21.15.60:0/3429870262 wait complete. 2023-07-13T22:31:04.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.703+0000 7fc3e25ff700 1 Processor -- start 2023-07-13T22:31:04.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.703+0000 7fc3e25ff700 1 -- start start 2023-07-13T22:31:04.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.704+0000 7fc3e25ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0b1ac0 0x7fc3dc14afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.704+0000 7fc3e25ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3dc0b8200 0x7fc3dc1e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.704+0000 7fc3e25ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3dc0cfd50 0x7fc3dc1e8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.704+0000 7fc3e25ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc1f6930 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.704+0000 7fc3e25ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc1f5b70 con 0x7fc3dc0b8200 2023-07-13T22:31:04.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.704+0000 7fc3e25ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc1f5e50 con 0x7fc3dc0cfd50 2023-07-13T22:31:04.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.704+0000 7fc3dbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0b1ac0 0x7fc3dc14afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.704+0000 7fc3dbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0b1ac0 0x7fc3dc14afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34688/0 (socket says 172.21.15.60:34688) 2023-07-13T22:31:04.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.704+0000 7fc3dbfff700 1 -- 172.21.15.60:0/2349378225 learned_addr learned my addr 172.21.15.60:0/2349378225 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:04.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3dbfff700 1 -- 172.21.15.60:0/2349378225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3dc0cfd50 msgr2=0x7fc3dc1e8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3dbfff700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3dc0cfd50 0x7fc3dc1e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3dbfff700 1 -- 172.21.15.60:0/2349378225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3dc0b8200 msgr2=0x7fc3dc1e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3db7fe700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3dc0b8200 0x7fc3dc1e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3dbfff700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3dc0b8200 0x7fc3dc1e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3dbfff700 1 -- 172.21.15.60:0/2349378225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0009550 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3db7fe700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3dc0b8200 0x7fc3dc1e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:04.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3dbfff700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0b1ac0 0x7fc3dc14afc0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00c4e0 tx=0x7fc3cc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:04.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3d8ff9700 1 -- 172.21.15.60:0/2349378225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0052d0 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3d8ff9700 1 -- 172.21.15.60:0/2349378225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3cc005430 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3dc1f6130 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3d8ff9700 1 -- 172.21.15.60:0/2349378225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0038c0 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.705+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3dc14b500 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.706+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3dc1f65f0 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.707+0000 7fc3d8ff9700 1 -- 172.21.15.60:0/2349378225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3cc003a20 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.708+0000 7fc3d8ff9700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3c806ace0 0x7fc3c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:04.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.708+0000 7fc3d8ff9700 1 -- 172.21.15.60:0/2349378225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3cc088a40 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.709+0000 7fc3db7fe700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3c806ace0 0x7fc3c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:04.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.709+0000 7fc3db7fe700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3c806ace0 0x7fc3c806d190 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0d0140 tx=0x7fc3d0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:04.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.713+0000 7fc3d8ff9700 1 -- 172.21.15.60:0/2349378225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc3cc053a50 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.918+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc3dc04e210 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.918+0000 7fc3d8ff9700 1 -- 172.21.15.60:0/2349378225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v124) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc3cc017020 con 0x7fc3dc0b1ac0 2023-07-13T22:31:04.900 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:31:04.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3c806ace0 msgr2=0x7fc3c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3c806ace0 0x7fc3c806d190 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0d0140 tx=0x7fc3d0008030 comp rx=0 tx=0).stop 2023-07-13T22:31:04.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0b1ac0 msgr2=0x7fc3dc14afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:04.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0b1ac0 0x7fc3dc14afc0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00c4e0 tx=0x7fc3cc00c8a0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 shutdown_connections 2023-07-13T22:31:04.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc3c806ace0 0x7fc3c806d190 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3dc0b1ac0 0x7fc3dc14afc0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3dc0cfd50 0x7fc3dc1e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 --2- 172.21.15.60:0/2349378225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3dc0b8200 0x7fc3dc1e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:04.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 >> 172.21.15.60:0/2349378225 conn(0x7fc3dc06b2f0 msgr2=0x7fc3dc0d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:04.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 shutdown_connections 2023-07-13T22:31:04.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:04.921+0000 7fc3e25ff700 1 -- 172.21.15.60:0/2349378225 wait complete. 2023-07-13T22:31:04.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:31:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.145+0000 7fb82e3ff700 1 Processor -- start 2023-07-13T22:31:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.145+0000 7fb82e3ff700 1 -- start start 2023-07-13T22:31:05.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.146+0000 7fb82e3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8280b8200 0x7fb8280cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.146+0000 7fb82e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280cede0 0x7fb8280cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.146+0000 7fb82e3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8280d1870 con 0x7fb8280b1ac0 2023-07-13T22:31:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.146+0000 7fb82e3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8280cf370 con 0x7fb8280b8200 2023-07-13T22:31:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.146+0000 7fb82e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280d12b0 con 0x7fb8280cede0 2023-07-13T22:31:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.146+0000 7fb8277fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb8280b1ac0 0x7fb82815ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56664/0 (socket says 172.21.15.60:56664) 2023-07-13T22:31:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb8277fe700 1 -- 172.21.15.60:0/3906706741 learned_addr learned my addr 172.21.15.60:0/3906706741 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb827fff700 1 --2- 172.21.15.60:0/3906706741 >> v2:172.21.15.105:3300/0 conn(0x7fb8280b8200 0x7fb8280cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb82c99c700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280cede0 0x7fb8280cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967331293 0 0) 0x7fb8280d1870 con 0x7fb8280b1ac0 2023-07-13T22:31:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8100035b0 con 0x7fb8280b1ac0 2023-07-13T22:31:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1883942586 0 0) 0x7fb8100035b0 con 0x7fb8280b1ac0 2023-07-13T22:31:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280cede0 msgr2=0x7fb8280cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280cede0 0x7fb8280cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 >> v2:172.21.15.105:3300/0 conn(0x7fb8280b8200 msgr2=0x7fb8280cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 --2- 172.21.15.60:0/3906706741 >> v2:172.21.15.105:3300/0 conn(0x7fb8280b8200 0x7fb8280cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8280d1590 con 0x7fb8280b1ac0 2023-07-13T22:31:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fb81c001670 con 0x7fb8280b1ac0 2023-07-13T22:31:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 >> v1:172.21.15.60:6789/0 conn(0x7fb8280b1ac0 legacy=0x7fb82815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.147+0000 7fb826ffd700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb810003a00 0x7fb810003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb826ffd700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280cede0 0x7fb810004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb826ffd700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810004d90 0x7fb810005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280cf370 con 0x7fb810003a00 2023-07-13T22:31:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280d12b0 con 0x7fb810004d90 2023-07-13T22:31:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81000ba10 con 0x7fb8280cede0 2023-07-13T22:31:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb827fff700 1 --2- 172.21.15.60:0/3906706741 >> v2:172.21.15.105:3300/0 conn(0x7fb8280b8200 0x7fb8280cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb8277fe700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280cede0 0x7fb810004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb827fff700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb810003a00 0x7fb810003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb82c99c700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810004d90 0x7fb810005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb827fff700 1 -- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280cede0 msgr2=0x7fb810004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb827fff700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280cede0 0x7fb810004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb827fff700 1 -- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810004d90 msgr2=0x7fb810005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb827fff700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810004d90 0x7fb810005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.148+0000 7fb827fff700 1 -- 172.21.15.60:0/3906706741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8100035b0 con 0x7fb810003a00 2023-07-13T22:31:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.149+0000 7fb827fff700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb810003a00 0x7fb810003e10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fb818009d10 tx=0x7fb810005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f1e7a1cd77a099 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.149+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818009e60 con 0x7fb810003a00 2023-07-13T22:31:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.149+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818004030 con 0x7fb810003a00 2023-07-13T22:31:05.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.149+0000 7fb826ffd700 1 -- 172.21.15.60:0/3906706741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180043a0 con 0x7fb810003a00 2023-07-13T22:31:05.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.149+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3906706741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb810003a00 msgr2=0x7fb810003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.149+0000 7fb82e3ff700 1 --2- 172.21.15.60:0/3906706741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb810003a00 0x7fb810003e10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fb818009d10 tx=0x7fb810005880 comp rx=0 tx=0).stop 2023-07-13T22:31:05.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.149+0000 7fb82c99c700 1 -- 172.21.15.60:0/3906706741 reap_dead start 2023-07-13T22:31:05.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.150+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3906706741 shutdown_connections 2023-07-13T22:31:05.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.150+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3906706741 >> 172.21.15.60:0/3906706741 conn(0x7fb82806b2f0 msgr2=0x7fb82806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:05.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.150+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3906706741 shutdown_connections 2023-07-13T22:31:05.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.150+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3906706741 wait complete. 2023-07-13T22:31:05.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.151+0000 7fb82e3ff700 1 Processor -- start 2023-07-13T22:31:05.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.152+0000 7fb82e3ff700 1 -- start start 2023-07-13T22:31:05.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.152+0000 7fb82e3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810003a00 0x7fb8280aa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82e3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280ca780 0x7fb828159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82e3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8280d1a90 0x7fb8280d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82e3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280ac2e0 con 0x7fb8280d1a90 2023-07-13T22:31:05.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82e3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280d2440 con 0x7fb810003a00 2023-07-13T22:31:05.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82e3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280cb810 con 0x7fb8280ca780 2023-07-13T22:31:05.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82c99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8280d1a90 0x7fb8280d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82c99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8280d1a90 0x7fb8280d1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58108/0 (socket says 172.21.15.60:58108) 2023-07-13T22:31:05.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82c99c700 1 -- 172.21.15.60:0/3623984662 learned_addr learned my addr 172.21.15.60:0/3623984662 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:05.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb827fff700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810003a00 0x7fb8280aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb8277fe700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280ca780 0x7fb828159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82c99c700 1 -- 172.21.15.60:0/3623984662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280ca780 msgr2=0x7fb828159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82c99c700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280ca780 0x7fb828159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82c99c700 1 -- 172.21.15.60:0/3623984662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810003a00 msgr2=0x7fb8280aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82c99c700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810003a00 0x7fb8280aa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb82c99c700 1 -- 172.21.15.60:0/3623984662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8280aad70 con 0x7fb8280d1a90 2023-07-13T22:31:05.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.153+0000 7fb8277fe700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280ca780 0x7fb828159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:05.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.154+0000 7fb82c99c700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8280d1a90 0x7fb8280d1f00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fb820011bd0 tx=0x7fb820011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:05.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.154+0000 7fb824ff9700 1 -- 172.21.15.60:0/3623984662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000e040 con 0x7fb8280d1a90 2023-07-13T22:31:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.154+0000 7fb824ff9700 1 -- 172.21.15.60:0/3623984662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82000f040 con 0x7fb8280d1a90 2023-07-13T22:31:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.154+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8280aa7b0 con 0x7fb8280d1a90 2023-07-13T22:31:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.154+0000 7fb824ff9700 1 -- 172.21.15.60:0/3623984662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000fd90 con 0x7fb8280d1a90 2023-07-13T22:31:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.154+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8280cb4a0 con 0x7fb8280d1a90 2023-07-13T22:31:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.155+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8280cb300 con 0x7fb8280d1a90 2023-07-13T22:31:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.156+0000 7fb824ff9700 1 -- 172.21.15.60:0/3623984662 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb820015430 con 0x7fb8280d1a90 2023-07-13T22:31:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.157+0000 7fb824ff9700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb81406ace0 0x7fb81406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.157+0000 7fb824ff9700 1 -- 172.21.15.60:0/3623984662 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb820089d00 con 0x7fb8280d1a90 2023-07-13T22:31:05.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.157+0000 7fb827fff700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb81406ace0 0x7fb81406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.158+0000 7fb827fff700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb81406ace0 0x7fb81406d190 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb8180090b0 tx=0x7fb818008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:05.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.161+0000 7fb824ff9700 1 -- 172.21.15.60:0/3623984662 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb8280cb300 con 0x7fb8280d1a90 2023-07-13T22:31:05.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.371+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8280cb300 con 0x7fb8280d1a90 2023-07-13T22:31:05.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.371+0000 7fb824ff9700 1 -- 172.21.15.60:0/3623984662 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8280cb300 con 0x7fb8280d1a90 2023-07-13T22:31:05.353 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:31:05.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.373+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb81406ace0 msgr2=0x7fb81406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.373+0000 7fb82e3ff700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb81406ace0 0x7fb81406d190 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb8180090b0 tx=0x7fb818008030 comp rx=0 tx=0).stop 2023-07-13T22:31:05.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.373+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8280d1a90 msgr2=0x7fb8280d1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.373+0000 7fb82e3ff700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8280d1a90 0x7fb8280d1f00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fb820011bd0 tx=0x7fb820011ee0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.374+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 shutdown_connections 2023-07-13T22:31:05.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.374+0000 7fb82e3ff700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb81406ace0 0x7fb81406d190 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.374+0000 7fb82e3ff700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8280d1a90 0x7fb8280d1f00 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.374+0000 7fb82e3ff700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810003a00 0x7fb8280aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.374+0000 7fb82e3ff700 1 --2- 172.21.15.60:0/3623984662 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8280ca780 0x7fb828159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.374+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 >> 172.21.15.60:0/3623984662 conn(0x7fb82806b2f0 msgr2=0x7fb828071600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:05.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.374+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 shutdown_connections 2023-07-13T22:31:05.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.374+0000 7fb82e3ff700 1 -- 172.21.15.60:0/3623984662 wait complete. 2023-07-13T22:31:05.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:31:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.598+0000 7f9ee85ff700 1 Processor -- start 2023-07-13T22:31:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.598+0000 7f9ee85ff700 1 -- start start 2023-07-13T22:31:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.599+0000 7f9ee85ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00b1ac0 0x7f9ee00d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.599+0000 7f9ee85ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee00d1870 con 0x7f9ee00b8200 2023-07-13T22:31:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.599+0000 7f9ee85ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee00cf370 con 0x7f9ee00cfd50 2023-07-13T22:31:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.599+0000 7f9ee85ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00d12b0 con 0x7f9ee00b1ac0 2023-07-13T22:31:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee639b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9ee00b8200 0x7f9ee00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56674/0 (socket says 172.21.15.60:56674) 2023-07-13T22:31:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee639b700 1 -- 172.21.15.60:0/4261101142 learned_addr learned my addr 172.21.15.60:0/4261101142 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee6b9c700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00b1ac0 0x7f9ee00d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378682768 0 0) 0x7f9ee00d1870 con 0x7f9ee00b8200 2023-07-13T22:31:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ecc0035b0 con 0x7f9ee00b8200 2023-07-13T22:31:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722630935 0 0) 0x7f9ee00cf370 con 0x7f9ee00cfd50 2023-07-13T22:31:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee00d1870 con 0x7f9ee00cfd50 2023-07-13T22:31:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3029818577 0 0) 0x7f9ecc0035b0 con 0x7f9ee00b8200 2023-07-13T22:31:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00b1ac0 msgr2=0x7f9ee00d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00b1ac0 0x7f9ee00d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 >> v1:172.21.15.105:6789/0 conn(0x7f9ee00cfd50 legacy=0x7f9ee00d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee00d1590 con 0x7f9ee00b8200 2023-07-13T22:31:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f9edc001670 con 0x7f9ee00b8200 2023-07-13T22:31:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.600+0000 7f9ee6b9c700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00b1ac0 0x7f9ee00d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 >> v1:172.21.15.60:6789/0 conn(0x7f9ee00b8200 legacy=0x7f9ee00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee5399700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ecc003bc0 0x7f9ecc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee5399700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00b1ac0 0x7f9ecc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee5399700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc004d90 0x7f9ecc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00d12b0 con 0x7f9ecc004d90 2023-07-13T22:31:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc00bac0 con 0x7f9ecc003bc0 2023-07-13T22:31:05.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc00bc30 con 0x7f9ee00b1ac0 2023-07-13T22:31:05.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee6b9c700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc004d90 0x7f9ecc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee639b700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ecc003bc0 0x7f9ecc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee5b9a700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00b1ac0 0x7f9ecc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee6b9c700 1 -- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00b1ac0 msgr2=0x7f9ecc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:05.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee6b9c700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00b1ac0 0x7f9ecc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee6b9c700 1 -- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ecc003bc0 msgr2=0x7f9ecc003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:05.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee6b9c700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ecc003bc0 0x7f9ecc003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.601+0000 7f9ee6b9c700 1 -- 172.21.15.60:0/4261101142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed8009a70 con 0x7f9ecc004d90 2023-07-13T22:31:05.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.602+0000 7f9ee6b9c700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc004d90 0x7f9ecc005200 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00b1ee0 tx=0x7f9ecc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8932f41ebbdddb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:05.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.602+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800ed30 con 0x7f9ecc004d90 2023-07-13T22:31:05.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.602+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed800f040 con 0x7f9ecc004d90 2023-07-13T22:31:05.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.602+0000 7f9ee5399700 1 -- 172.21.15.60:0/4261101142 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80146a0 con 0x7f9ecc004d90 2023-07-13T22:31:05.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.602+0000 7f9ee85ff700 1 -- 172.21.15.60:0/4261101142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc004d90 msgr2=0x7f9ecc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.602+0000 7f9ee85ff700 1 --2- 172.21.15.60:0/4261101142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc004d90 0x7f9ecc005200 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00b1ee0 tx=0x7f9ecc005880 comp rx=0 tx=0).stop 2023-07-13T22:31:05.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.602+0000 7f9ee6b9c700 1 -- 172.21.15.60:0/4261101142 reap_dead start 2023-07-13T22:31:05.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.602+0000 7f9ee85ff700 1 -- 172.21.15.60:0/4261101142 shutdown_connections 2023-07-13T22:31:05.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.602+0000 7f9ee85ff700 1 -- 172.21.15.60:0/4261101142 >> 172.21.15.60:0/4261101142 conn(0x7f9ee006b2f0 msgr2=0x7f9ee006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:05.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.603+0000 7f9ee85ff700 1 -- 172.21.15.60:0/4261101142 shutdown_connections 2023-07-13T22:31:05.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.603+0000 7f9ee85ff700 1 -- 172.21.15.60:0/4261101142 wait complete. 2023-07-13T22:31:05.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.604+0000 7f9ee85ff700 1 Processor -- start 2023-07-13T22:31:05.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.604+0000 7f9ee85ff700 1 -- start start 2023-07-13T22:31:05.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.605+0000 7f9ee85ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ecc004d90 0x7f9ee014a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.605+0000 7f9ee85ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee0147da0 0x7f9ee00b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.605+0000 7f9ee85ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee01e5a40 0x7f9ee01e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.605+0000 7f9ee85ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee015b740 con 0x7f9ee0147da0 2023-07-13T22:31:05.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.605+0000 7f9ee85ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee014ad50 con 0x7f9ecc004d90 2023-07-13T22:31:05.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.605+0000 7f9ee85ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0148220 con 0x7f9ee01e5a40 2023-07-13T22:31:05.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee5b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee0147da0 0x7f9ee00b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee5b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee0147da0 0x7f9ee00b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58126/0 (socket says 172.21.15.60:58126) 2023-07-13T22:31:05.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee5b9a700 1 -- 172.21.15.60:0/265825609 learned_addr learned my addr 172.21.15.60:0/265825609 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:05.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee639b700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ecc004d90 0x7f9ee014a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee6b9c700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee01e5a40 0x7f9ee01e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee5b9a700 1 -- 172.21.15.60:0/265825609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee01e5a40 msgr2=0x7f9ee01e5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee5b9a700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee01e5a40 0x7f9ee01e5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee5b9a700 1 -- 172.21.15.60:0/265825609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ecc004d90 msgr2=0x7f9ee014a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee5b9a700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ecc004d90 0x7f9ee014a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee5b9a700 1 -- 172.21.15.60:0/265825609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0148500 con 0x7f9ee0147da0 2023-07-13T22:31:05.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee6b9c700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee01e5a40 0x7f9ee01e5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:05.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ee5b9a700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee0147da0 0x7f9ee00b6d20 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000f800 tx=0x7f9ed000fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:05.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.606+0000 7f9ed6ffd700 1 -- 172.21.15.60:0/265825609 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0005320 con 0x7f9ee0147da0 2023-07-13T22:31:05.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.607+0000 7f9ed6ffd700 1 -- 172.21.15.60:0/265825609 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0005480 con 0x7f9ee0147da0 2023-07-13T22:31:05.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.607+0000 7f9ed6ffd700 1 -- 172.21.15.60:0/265825609 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0006a80 con 0x7f9ee0147da0 2023-07-13T22:31:05.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.607+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee01487e0 con 0x7f9ee0147da0 2023-07-13T22:31:05.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.607+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee0148da0 con 0x7f9ee0147da0 2023-07-13T22:31:05.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.608+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0148fc0 con 0x7f9ee0147da0 2023-07-13T22:31:05.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.609+0000 7f9ed6ffd700 1 -- 172.21.15.60:0/265825609 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ed0006be0 con 0x7f9ee0147da0 2023-07-13T22:31:05.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.610+0000 7f9ed6ffd700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ec406ace0 0x7f9ec406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:05.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.610+0000 7f9ed6ffd700 1 -- 172.21.15.60:0/265825609 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ed0088aa0 con 0x7f9ee0147da0 2023-07-13T22:31:05.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.610+0000 7f9ee639b700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ec406ace0 0x7f9ec406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:05.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.611+0000 7f9ee639b700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ec406ace0 0x7f9ec406d190 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9edc001050 tx=0x7f9edc000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:05.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.615+0000 7f9ed6ffd700 1 -- 172.21.15.60:0/265825609 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9ed0053ab0 con 0x7f9ee0147da0 2023-07-13T22:31:05.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.817+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ee004e210 con 0x7f9ee0147da0 2023-07-13T22:31:05.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.818+0000 7f9ed6ffd700 1 -- 172.21.15.60:0/265825609 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ed00570d0 con 0x7f9ee0147da0 2023-07-13T22:31:05.799 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:31:05.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ec406ace0 msgr2=0x7f9ec406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ec406ace0 0x7f9ec406d190 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9edc001050 tx=0x7f9edc000f80 comp rx=0 tx=0).stop 2023-07-13T22:31:05.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee0147da0 msgr2=0x7f9ee00b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:05.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee0147da0 0x7f9ee00b6d20 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000f800 tx=0x7f9ed000fb10 comp rx=0 tx=0).stop 2023-07-13T22:31:05.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 shutdown_connections 2023-07-13T22:31:05.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f9ec406ace0 0x7f9ec406d190 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee01e5a40 0x7f9ee01e5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ecc004d90 0x7f9ee014a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 --2- 172.21.15.60:0/265825609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee0147da0 0x7f9ee00b6d20 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:05.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 >> 172.21.15.60:0/265825609 conn(0x7f9ee006b2f0 msgr2=0x7f9ee00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:05.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 shutdown_connections 2023-07-13T22:31:05.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:05.820+0000 7f9ee85ff700 1 -- 172.21.15.60:0/265825609 wait complete. 2023-07-13T22:31:05.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:31:06.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.045+0000 7fb889bff700 1 Processor -- start 2023-07-13T22:31:06.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.045+0000 7fb889bff700 1 -- start start 2023-07-13T22:31:06.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.047+0000 7fb889bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8840cfd50 0x7fb8840d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.047+0000 7fb889bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8840b1ac0 0x7fb8840d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.047+0000 7fb889bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8840d1870 con 0x7fb8840b8200 2023-07-13T22:31:06.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.047+0000 7fb889bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8840cf370 con 0x7fb8840cfd50 2023-07-13T22:31:06.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.047+0000 7fb889bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8840d12b0 con 0x7fb8840b1ac0 2023-07-13T22:31:06.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.047+0000 7fb882ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8840cfd50 0x7fb8840d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.047+0000 7fb8837fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb8840b8200 0x7fb8840cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56686/0 (socket says 172.21.15.60:56686) 2023-07-13T22:31:06.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.047+0000 7fb8837fe700 1 -- 172.21.15.60:0/3855994830 learned_addr learned my addr 172.21.15.60:0/3855994830 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:06.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.048+0000 7fb883fff700 1 --2- 172.21.15.60:0/3855994830 >> v2:172.21.15.105:3300/0 conn(0x7fb8840b1ac0 0x7fb8840d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.048+0000 7fb8827fc700 1 -- 172.21.15.60:0/3855994830 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761032163 0 0) 0x7fb8840d1870 con 0x7fb8840b8200 2023-07-13T22:31:06.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.048+0000 7fb8827fc700 1 -- 172.21.15.60:0/3855994830 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb86c0035b0 con 0x7fb8840b8200 2023-07-13T22:31:06.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.048+0000 7fb882ffd700 1 -- 172.21.15.60:0/3855994830 >> v2:172.21.15.105:3300/0 conn(0x7fb8840b1ac0 msgr2=0x7fb8840d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:06.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.048+0000 7fb882ffd700 1 --2- 172.21.15.60:0/3855994830 >> v2:172.21.15.105:3300/0 conn(0x7fb8840b1ac0 0x7fb8840d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.048+0000 7fb882ffd700 1 -- 172.21.15.60:0/3855994830 >> v1:172.21.15.60:6789/0 conn(0x7fb8840b8200 legacy=0x7fb8840cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.048+0000 7fb882ffd700 1 -- 172.21.15.60:0/3855994830 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8840d1590 con 0x7fb8840cfd50 2023-07-13T22:31:06.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.048+0000 7fb882ffd700 1 --2- 172.21.15.60:0/3855994830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 0x7fb8840d23c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fb8840b8d70 tx=0x7fb878009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebf1b992e5eeea8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:06.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.048+0000 7fb8827fc700 1 -- 172.21.15.60:0/3855994830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878011830 con 0x7fb8840cfd50 2023-07-13T22:31:06.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb8827fc700 1 -- 172.21.15.60:0/3855994830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb878011990 con 0x7fb8840cfd50 2023-07-13T22:31:06.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb8827fc700 1 -- 172.21.15.60:0/3855994830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878011d20 con 0x7fb8840cfd50 2023-07-13T22:31:06.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb889bff700 1 -- 172.21.15.60:0/3855994830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 msgr2=0x7fb8840d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb889bff700 1 --2- 172.21.15.60:0/3855994830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 0x7fb8840d23c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fb8840b8d70 tx=0x7fb878009b20 comp rx=0 tx=0).stop 2023-07-13T22:31:06.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb889bff700 1 -- 172.21.15.60:0/3855994830 shutdown_connections 2023-07-13T22:31:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb889bff700 1 --2- 172.21.15.60:0/3855994830 >> v2:172.21.15.105:3300/0 conn(0x7fb8840b1ac0 0x7fb8840d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb889bff700 1 --2- 172.21.15.60:0/3855994830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 0x7fb8840d23c0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb889bff700 1 -- 172.21.15.60:0/3855994830 >> 172.21.15.60:0/3855994830 conn(0x7fb88406b2f0 msgr2=0x7fb88406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb889bff700 1 -- 172.21.15.60:0/3855994830 shutdown_connections 2023-07-13T22:31:06.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.049+0000 7fb889bff700 1 -- 172.21.15.60:0/3855994830 wait complete. 2023-07-13T22:31:06.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.051+0000 7fb889bff700 1 Processor -- start 2023-07-13T22:31:06.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.051+0000 7fb889bff700 1 -- start start 2023-07-13T22:31:06.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.052+0000 7fb889bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8840b1ac0 0x7fb8840cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.052+0000 7fb889bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8840b8200 0x7fb8840c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.052+0000 7fb889bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 0x7fb884151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.052+0000 7fb889bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840ca750 con 0x7fb8840cfd50 2023-07-13T22:31:06.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.052+0000 7fb889bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840caa30 con 0x7fb8840b8200 2023-07-13T22:31:06.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.052+0000 7fb889bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840cad10 con 0x7fb8840b1ac0 2023-07-13T22:31:06.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.052+0000 7fb883fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 0x7fb884151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.052+0000 7fb883fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 0x7fb884151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58154/0 (socket says 172.21.15.60:58154) 2023-07-13T22:31:06.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.052+0000 7fb883fff700 1 -- 172.21.15.60:0/3516969213 learned_addr learned my addr 172.21.15.60:0/3516969213 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:06.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb883fff700 1 -- 172.21.15.60:0/3516969213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8840b1ac0 msgr2=0x7fb8840cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb883fff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8840b1ac0 0x7fb8840cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb883fff700 1 -- 172.21.15.60:0/3516969213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8840b8200 msgr2=0x7fb8840c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb882ffd700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8840b8200 0x7fb8840c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb883fff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8840b8200 0x7fb8840c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb883fff700 1 -- 172.21.15.60:0/3516969213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8780094a0 con 0x7fb8840cfd50 2023-07-13T22:31:06.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb882ffd700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8840b8200 0x7fb8840c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:06.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb883fff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 0x7fb884151a70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb87c007e40 tx=0x7fb87c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:06.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb86bfff700 1 -- 172.21.15.60:0/3516969213 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c00eea0 con 0x7fb8840cfd50 2023-07-13T22:31:06.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb86bfff700 1 -- 172.21.15.60:0/3516969213 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87c014420 con 0x7fb8840cfd50 2023-07-13T22:31:06.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb86bfff700 1 -- 172.21.15.60:0/3516969213 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0147b0 con 0x7fb8840cfd50 2023-07-13T22:31:06.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8840caff0 con 0x7fb8840cfd50 2023-07-13T22:31:06.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.053+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8840cb7f0 con 0x7fb8840cfd50 2023-07-13T22:31:06.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.054+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8840cb510 con 0x7fb8840cfd50 2023-07-13T22:31:06.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.055+0000 7fb86bfff700 1 -- 172.21.15.60:0/3516969213 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb87c014910 con 0x7fb8840cfd50 2023-07-13T22:31:06.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.056+0000 7fb86bfff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87006ace0 0x7fb87006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.056+0000 7fb86bfff700 1 -- 172.21.15.60:0/3516969213 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb87c089cd0 con 0x7fb8840cfd50 2023-07-13T22:31:06.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.057+0000 7fb8837fe700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87006ace0 0x7fb87006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.057+0000 7fb8837fe700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87006ace0 0x7fb87006d190 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8740016b0 tx=0x7fb8740079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:06.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.061+0000 7fb86bfff700 1 -- 172.21.15.60:0/3516969213 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb8840cb510 con 0x7fb8840cfd50 2023-07-13T22:31:06.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.281+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8840cb510 con 0x7fb8840cfd50 2023-07-13T22:31:06.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.281+0000 7fb86bfff700 1 -- 172.21.15.60:0/3516969213 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8840cb510 con 0x7fb8840cfd50 2023-07-13T22:31:06.263 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:31:06.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.283+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87006ace0 msgr2=0x7fb87006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.283+0000 7fb889bff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87006ace0 0x7fb87006d190 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8740016b0 tx=0x7fb8740079c0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.283+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 msgr2=0x7fb884151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.283+0000 7fb889bff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 0x7fb884151a70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb87c007e40 tx=0x7fb87c00e3c0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.284+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 shutdown_connections 2023-07-13T22:31:06.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.284+0000 7fb889bff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb87006ace0 0x7fb87006d190 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.284+0000 7fb889bff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8840cfd50 0x7fb884151a70 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.284+0000 7fb889bff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8840b1ac0 0x7fb8840cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.284+0000 7fb889bff700 1 --2- 172.21.15.60:0/3516969213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8840b8200 0x7fb8840c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.284+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 >> 172.21.15.60:0/3516969213 conn(0x7fb88406b2f0 msgr2=0x7fb8840d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:06.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.284+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 shutdown_connections 2023-07-13T22:31:06.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.284+0000 7fb889bff700 1 -- 172.21.15.60:0/3516969213 wait complete. 2023-07-13T22:31:06.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:31:06.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.505+0000 7ff142bff700 1 Processor -- start 2023-07-13T22:31:06.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.505+0000 7ff142bff700 1 -- start start 2023-07-13T22:31:06.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff142bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d35c0 0x7ff13c0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff142bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c0d09d0 con 0x7ff13c0b8570 2023-07-13T22:31:06.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff142bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c0d1870 con 0x7ff13c0d13f0 2023-07-13T22:31:06.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff142bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0d3b30 con 0x7ff13c0d35c0 2023-07-13T22:31:06.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff14099b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff13c0b8570 0x7ff13c0ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56692/0 (socket says 172.21.15.60:56692) 2023-07-13T22:31:06.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff14099b700 1 -- 172.21.15.60:0/1586159482 learned_addr learned my addr 172.21.15.60:0/1586159482 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:06.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff13bfff700 1 --2- 172.21.15.60:0/1586159482 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d35c0 0x7ff13c0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff13b7fe700 1 -- 172.21.15.60:0/1586159482 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516557455 0 0) 0x7ff13c0d09d0 con 0x7ff13c0b8570 2023-07-13T22:31:06.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff13b7fe700 1 -- 172.21.15.60:0/1586159482 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff12c0035b0 con 0x7ff13c0b8570 2023-07-13T22:31:06.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff13b7fe700 1 -- 172.21.15.60:0/1586159482 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399650215 0 0) 0x7ff13c0d1870 con 0x7ff13c0d13f0 2023-07-13T22:31:06.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.507+0000 7ff13b7fe700 1 -- 172.21.15.60:0/1586159482 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff13c0d09d0 con 0x7ff13c0d13f0 2023-07-13T22:31:06.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.508+0000 7ff13bfff700 1 -- 172.21.15.60:0/1586159482 >> v1:172.21.15.105:6789/0 conn(0x7ff13c0d13f0 legacy=0x7ff13c15c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.508+0000 7ff13bfff700 1 -- 172.21.15.60:0/1586159482 >> v1:172.21.15.60:6789/0 conn(0x7ff13c0b8570 legacy=0x7ff13c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.508+0000 7ff13bfff700 1 -- 172.21.15.60:0/1586159482 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c0cda10 con 0x7ff13c0d35c0 2023-07-13T22:31:06.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.508+0000 7ff13bfff700 1 --2- 172.21.15.60:0/1586159482 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d35c0 0x7ff13c0d0eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff128009870 tx=0x7ff128009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3dbe55343ed6e71 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:06.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.508+0000 7ff13b7fe700 1 -- 172.21.15.60:0/1586159482 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128011830 con 0x7ff13c0d35c0 2023-07-13T22:31:06.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.508+0000 7ff13b7fe700 1 -- 172.21.15.60:0/1586159482 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff128011990 con 0x7ff13c0d35c0 2023-07-13T22:31:06.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.508+0000 7ff13b7fe700 1 -- 172.21.15.60:0/1586159482 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128011d20 con 0x7ff13c0d35c0 2023-07-13T22:31:06.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.509+0000 7ff142bff700 1 -- 172.21.15.60:0/1586159482 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d35c0 msgr2=0x7ff13c0d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.509+0000 7ff142bff700 1 --2- 172.21.15.60:0/1586159482 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d35c0 0x7ff13c0d0eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff128009870 tx=0x7ff128009b80 comp rx=0 tx=0).stop 2023-07-13T22:31:06.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.509+0000 7ff142bff700 1 -- 172.21.15.60:0/1586159482 shutdown_connections 2023-07-13T22:31:06.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.509+0000 7ff142bff700 1 --2- 172.21.15.60:0/1586159482 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d35c0 0x7ff13c0d0eb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.509+0000 7ff142bff700 1 -- 172.21.15.60:0/1586159482 >> 172.21.15.60:0/1586159482 conn(0x7ff13c071600 msgr2=0x7ff13c0719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:06.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.509+0000 7ff142bff700 1 -- 172.21.15.60:0/1586159482 shutdown_connections 2023-07-13T22:31:06.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.509+0000 7ff142bff700 1 -- 172.21.15.60:0/1586159482 wait complete. 2023-07-13T22:31:06.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.511+0000 7ff142bff700 1 Processor -- start 2023-07-13T22:31:06.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.511+0000 7ff142bff700 1 -- start start 2023-07-13T22:31:06.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff142bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff13c0b8570 0x7ff13c15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff142bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d13f0 0x7ff13c15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff142bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff13c0d35c0 0x7ff13c0c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff142bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0c9d30 con 0x7ff13c0d35c0 2023-07-13T22:31:06.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff142bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c1f30c0 con 0x7ff13c0b8570 2023-07-13T22:31:06.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff142bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c1f33a0 con 0x7ff13c0d13f0 2023-07-13T22:31:06.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14119c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff13c0d35c0 0x7ff13c0c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14119c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff13c0d35c0 0x7ff13c0c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58168/0 (socket says 172.21.15.60:58168) 2023-07-13T22:31:06.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14119c700 1 -- 172.21.15.60:0/265544787 learned_addr learned my addr 172.21.15.60:0/265544787 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:06.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff13bfff700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d13f0 0x7ff13c15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14099b700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff13c0b8570 0x7ff13c15dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14119c700 1 -- 172.21.15.60:0/265544787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d13f0 msgr2=0x7ff13c15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14119c700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d13f0 0x7ff13c15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14119c700 1 -- 172.21.15.60:0/265544787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff13c0b8570 msgr2=0x7ff13c15dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14119c700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff13c0b8570 0x7ff13c15dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14119c700 1 -- 172.21.15.60:0/265544787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280094a0 con 0x7ff13c0d35c0 2023-07-13T22:31:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.512+0000 7ff14099b700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff13c0b8570 0x7ff13c15dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.513+0000 7ff14119c700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff13c0d35c0 0x7ff13c0c8980 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7ff13400cc20 tx=0x7ff13400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:06.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.513+0000 7ff1397fa700 1 -- 172.21.15.60:0/265544787 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13400ac20 con 0x7ff13c0d35c0 2023-07-13T22:31:06.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.513+0000 7ff1397fa700 1 -- 172.21.15.60:0/265544787 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13400ad80 con 0x7ff13c0d35c0 2023-07-13T22:31:06.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.513+0000 7ff1397fa700 1 -- 172.21.15.60:0/265544787 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134003c20 con 0x7ff13c0d35c0 2023-07-13T22:31:06.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.513+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c1f3680 con 0x7ff13c0d35c0 2023-07-13T22:31:06.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.513+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c1f3c40 con 0x7ff13c0d35c0 2023-07-13T22:31:06.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.514+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c0cea90 con 0x7ff13c0d35c0 2023-07-13T22:31:06.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.515+0000 7ff1397fa700 1 -- 172.21.15.60:0/265544787 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1340050d0 con 0x7ff13c0d35c0 2023-07-13T22:31:06.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.516+0000 7ff1397fa700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff12006ace0 0x7ff12006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.516+0000 7ff1397fa700 1 -- 172.21.15.60:0/265544787 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1340895c0 con 0x7ff13c0d35c0 2023-07-13T22:31:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.517+0000 7ff14099b700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff12006ace0 0x7ff12006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.517+0000 7ff14099b700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff12006ace0 0x7ff12006d190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff1300010e0 tx=0x7ff130009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.520+0000 7ff1397fa700 1 -- 172.21.15.60:0/265544787 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff1340546e0 con 0x7ff13c0d35c0 2023-07-13T22:31:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.744+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff13c15c910 con 0x7ff13c0d35c0 2023-07-13T22:31:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.745+0000 7ff1397fa700 1 -- 172.21.15.60:0/265544787 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v124) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff134057d00 con 0x7ff13c0d35c0 2023-07-13T22:31:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.747+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff12006ace0 msgr2=0x7ff12006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.747+0000 7ff142bff700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff12006ace0 0x7ff12006d190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff1300010e0 tx=0x7ff130009040 comp rx=0 tx=0).stop 2023-07-13T22:31:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.747+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff13c0d35c0 msgr2=0x7ff13c0c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.747+0000 7ff142bff700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff13c0d35c0 0x7ff13c0c8980 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7ff13400cc20 tx=0x7ff13400a060 comp rx=0 tx=0).stop 2023-07-13T22:31:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.747+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 shutdown_connections 2023-07-13T22:31:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.747+0000 7ff142bff700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff12006ace0 0x7ff12006d190 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.747+0000 7ff142bff700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff13c0d35c0 0x7ff13c0c8980 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.747+0000 7ff142bff700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff13c0b8570 0x7ff13c15dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.747+0000 7ff142bff700 1 --2- 172.21.15.60:0/265544787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff13c0d13f0 0x7ff13c15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.748+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 >> 172.21.15.60:0/265544787 conn(0x7ff13c071600 msgr2=0x7ff13c15f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.748+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 shutdown_connections 2023-07-13T22:31:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:06.748+0000 7ff142bff700 1 -- 172.21.15.60:0/265544787 wait complete. 2023-07-13T22:31:06.731 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:31:06.747 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:31:06.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:06.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:06.817 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[161669]: fuse finished with error 0 and tester_r 0 2023-07-13T22:31:06.820 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:31:06.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:06.822 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:31:06.991 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:31:06.992 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:07.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:07.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.262+0000 7f0f11bff700 1 Processor -- start 2023-07-13T22:31:07.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.262+0000 7f0f11bff700 1 -- start start 2023-07-13T22:31:07.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f11bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 0x7f0f0c0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f11bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0f0c0c52c0 0x7f0f0c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f11bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c0c3fc0 con 0x7f0f0c06a410 2023-07-13T22:31:07.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f11bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c06a890 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f11bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c42a0 con 0x7f0f0c158870 2023-07-13T22:31:07.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f0bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0f0c06a410 0x7f0f0c0c9300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56700/0 (socket says 172.21.15.60:56700) 2023-07-13T22:31:07.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f0bfff700 1 -- 172.21.15.60:0/4168088527 learned_addr learned my addr 172.21.15.60:0/4168088527 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:07.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f0affd700 1 --2- 172.21.15.60:0/4168088527 >> v2:172.21.15.105:3300/0 conn(0x7f0f0c0c52c0 0x7f0f0c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f0b7fe700 1 --2- 172.21.15.60:0/4168088527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 0x7f0f0c0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.264+0000 7f0f0a7fc700 1 -- 172.21.15.60:0/4168088527 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574088959 0 0) 0x7f0f0c0c3fc0 con 0x7f0f0c06a410 2023-07-13T22:31:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0a7fc700 1 -- 172.21.15.60:0/4168088527 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef40035b0 con 0x7f0f0c06a410 2023-07-13T22:31:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0affd700 1 -- 172.21.15.60:0/4168088527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 msgr2=0x7f0f0c0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0affd700 1 --2- 172.21.15.60:0/4168088527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 0x7f0f0c0c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0affd700 1 -- 172.21.15.60:0/4168088527 >> v1:172.21.15.60:6789/0 conn(0x7f0f0c06a410 legacy=0x7f0f0c0c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0affd700 1 -- 172.21.15.60:0/4168088527 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0c4580 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0b7fe700 1 --2- 172.21.15.60:0/4168088527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 0x7f0f0c0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0affd700 1 --2- 172.21.15.60:0/4168088527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c0c52c0 0x7f0f0c069ed0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009840 tx=0x7f0f00009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be6fadc832419fb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0a7fc700 1 -- 172.21.15.60:0/4168088527 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00011830 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0a7fc700 1 -- 172.21.15.60:0/4168088527 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f00011990 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.265+0000 7f0f0a7fc700 1 -- 172.21.15.60:0/4168088527 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00011d20 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.266+0000 7f0f11bff700 1 -- 172.21.15.60:0/4168088527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c0c52c0 msgr2=0x7f0f0c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.266+0000 7f0f11bff700 1 --2- 172.21.15.60:0/4168088527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c0c52c0 0x7f0f0c069ed0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009840 tx=0x7f0f00009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.266+0000 7f0f11bff700 1 -- 172.21.15.60:0/4168088527 shutdown_connections 2023-07-13T22:31:07.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.266+0000 7f0f11bff700 1 --2- 172.21.15.60:0/4168088527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 0x7f0f0c0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.266+0000 7f0f11bff700 1 --2- 172.21.15.60:0/4168088527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c0c52c0 0x7f0f0c069ed0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.266+0000 7f0f11bff700 1 -- 172.21.15.60:0/4168088527 >> 172.21.15.60:0/4168088527 conn(0x7f0f0c0cf410 msgr2=0x7f0f0c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:07.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.266+0000 7f0f11bff700 1 -- 172.21.15.60:0/4168088527 shutdown_connections 2023-07-13T22:31:07.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.266+0000 7f0f11bff700 1 -- 172.21.15.60:0/4168088527 wait complete. 2023-07-13T22:31:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.268+0000 7f0f11bff700 1 Processor -- start 2023-07-13T22:31:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.268+0000 7f0f11bff700 1 -- start start 2023-07-13T22:31:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f11bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c06a410 0x7f0f0c0ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f11bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f0c0c52c0 0x7f0f0c1494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f11bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 0x7f0f0c0c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f11bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c79d0 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f11bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c149a20 con 0x7f0f0c06a410 2023-07-13T22:31:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f11bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c76f0 con 0x7f0f0c158870 2023-07-13T22:31:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f0affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f0c0c52c0 0x7f0f0c1494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f0affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f0c0c52c0 0x7f0f0c1494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58174/0 (socket says 172.21.15.60:58174) 2023-07-13T22:31:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f0affd700 1 -- 172.21.15.60:0/2248794668 learned_addr learned my addr 172.21.15.60:0/2248794668 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:07.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f0b7fe700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c06a410 0x7f0f0c0ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.269+0000 7f0f0bfff700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 0x7f0f0c0c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0f0affd700 1 -- 172.21.15.60:0/2248794668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 msgr2=0x7f0f0c0c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0f0affd700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 0x7f0f0c0c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0f0affd700 1 -- 172.21.15.60:0/2248794668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c06a410 msgr2=0x7f0f0c0ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0f0affd700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c06a410 0x7f0f0c0ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0f0affd700 1 -- 172.21.15.60:0/2248794668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc0094a0 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0f0b7fe700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c06a410 0x7f0f0c0ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0f0affd700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f0c0c52c0 0x7f0f0c1494e0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f0f000090b0 tx=0x7f0f00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0ef3fff700 1 -- 172.21.15.60:0/2248794668 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f000036a0 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0ef3fff700 1 -- 172.21.15.60:0/2248794668 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0001f070 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f000094a0 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0ef3fff700 1 -- 172.21.15.60:0/2248794668 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00003a40 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.270+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c0c5bd0 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.271+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c15c110 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.272+0000 7f0ef3fff700 1 -- 172.21.15.60:0/2248794668 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f00003ba0 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.273+0000 7f0ef3fff700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ef806ace0 0x7f0ef806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.273+0000 7f0ef3fff700 1 -- 172.21.15.60:0/2248794668 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f00021070 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.273+0000 7f0f0b7fe700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ef806ace0 0x7f0ef806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.274+0000 7f0f0b7fe700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ef806ace0 0x7f0ef806d190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0ad9f0 tx=0x7f0efc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:07.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.277+0000 7f0ef3fff700 1 -- 172.21.15.60:0/2248794668 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0f00055de0 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.485+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f0c155e60 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.487+0000 7f0ef3fff700 1 -- 172.21.15.60:0/2248794668 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18215 (secure 0 0 0) 0x7f0f00059400 con 0x7f0f0c0c52c0 2023-07-13T22:31:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:07.469 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":124,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:30:41.182757+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:30:37.983326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-13T22:30:38.665534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:07.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ef806ace0 msgr2=0x7f0ef806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ef806ace0 0x7f0ef806d190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0ad9f0 tx=0x7f0efc008040 comp rx=0 tx=0).stop 2023-07-13T22:31:07.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f0c0c52c0 msgr2=0x7f0f0c1494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f0c0c52c0 0x7f0f0c1494e0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f0f000090b0 tx=0x7f0f00004750 comp rx=0 tx=0).stop 2023-07-13T22:31:07.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 shutdown_connections 2023-07-13T22:31:07.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f0ef806ace0 0x7f0ef806d190 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0f0c158870 0x7f0f0c0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0c06a410 0x7f0f0c0ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 --2- 172.21.15.60:0/2248794668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f0c0c52c0 0x7f0f0c1494e0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 >> 172.21.15.60:0/2248794668 conn(0x7f0f0c0cf410 msgr2=0x7f0f0c157460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:07.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.489+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 shutdown_connections 2023-07-13T22:31:07.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.490+0000 7f0f11bff700 1 -- 172.21.15.60:0/2248794668 wait complete. 2023-07-13T22:31:07.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:30:37.983326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:30:38.665534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:31:07.490 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:31:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.714+0000 7fb141fff700 1 Processor -- start 2023-07-13T22:31:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.714+0000 7fb141fff700 1 -- start start 2023-07-13T22:31:07.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.715+0000 7fb141fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb13c15a550 0x7fb13c0ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.715+0000 7fb141fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb13c0cbb20 0x7fb13c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.715+0000 7fb141fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c0cacc0 con 0x7fb13c0b10d0 2023-07-13T22:31:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb141fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13c0d0780 con 0x7fb13c0cbb20 2023-07-13T22:31:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb141fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13c0b21a0 con 0x7fb13c15a550 2023-07-13T22:31:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb13bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb13c0cbb20 0x7fb13c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb13bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb13c0cbb20 0x7fb13c069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58188/0 (socket says 172.21.15.60:58188) 2023-07-13T22:31:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb13bfff700 1 -- 172.21.15.60:0/1460955662 learned_addr learned my addr 172.21.15.60:0/1460955662 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb13b7fe700 1 --2- 172.21.15.60:0/1460955662 >> v2:172.21.15.105:3300/0 conn(0x7fb13c15a550 0x7fb13c0ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb13bfff700 1 -- 172.21.15.60:0/1460955662 >> v2:172.21.15.105:3300/0 conn(0x7fb13c15a550 msgr2=0x7fb13c0ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb13bfff700 1 --2- 172.21.15.60:0/1460955662 >> v2:172.21.15.105:3300/0 conn(0x7fb13c15a550 0x7fb13c0ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb13bfff700 1 -- 172.21.15.60:0/1460955662 >> v1:172.21.15.105:6789/0 conn(0x7fb13c0b10d0 legacy=0x7fb13c0cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.716+0000 7fb13bfff700 1 -- 172.21.15.60:0/1460955662 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c0b1900 con 0x7fb13c0cbb20 2023-07-13T22:31:07.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb13bfff700 1 --2- 172.21.15.60:0/1460955662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 0x7fb13c069ed0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0712f0 tx=0x7fb13400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6299c2472ede2ae8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:07.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb13affd700 1 -- 172.21.15.60:0/1460955662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400e070 con 0x7fb13c0cbb20 2023-07-13T22:31:07.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb13affd700 1 -- 172.21.15.60:0/1460955662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13400b9c0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb13affd700 1 -- 172.21.15.60:0/1460955662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400bd70 con 0x7fb13c0cbb20 2023-07-13T22:31:07.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb141fff700 1 -- 172.21.15.60:0/1460955662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 msgr2=0x7fb13c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb141fff700 1 --2- 172.21.15.60:0/1460955662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 0x7fb13c069ed0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0712f0 tx=0x7fb13400b330 comp rx=0 tx=0).stop 2023-07-13T22:31:07.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb141fff700 1 -- 172.21.15.60:0/1460955662 shutdown_connections 2023-07-13T22:31:07.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb141fff700 1 --2- 172.21.15.60:0/1460955662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 0x7fb13c069ed0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb141fff700 1 --2- 172.21.15.60:0/1460955662 >> v2:172.21.15.105:3300/0 conn(0x7fb13c15a550 0x7fb13c0ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.717+0000 7fb141fff700 1 -- 172.21.15.60:0/1460955662 >> 172.21.15.60:0/1460955662 conn(0x7fb13c06b900 msgr2=0x7fb13c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:07.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.718+0000 7fb141fff700 1 -- 172.21.15.60:0/1460955662 shutdown_connections 2023-07-13T22:31:07.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.718+0000 7fb141fff700 1 -- 172.21.15.60:0/1460955662 wait complete. 2023-07-13T22:31:07.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.719+0000 7fb141fff700 1 Processor -- start 2023-07-13T22:31:07.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.720+0000 7fb141fff700 1 -- start start 2023-07-13T22:31:07.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.720+0000 7fb141fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb13c0b10d0 0x7fb13c14eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.720+0000 7fb141fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 0x7fb13c0c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb141fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c15a550 0x7fb13c15c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb141fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0c49c0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb141fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0c4ca0 con 0x7fb13c15a550 2023-07-13T22:31:07.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb141fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0c4f80 con 0x7fb13c0b10d0 2023-07-13T22:31:07.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb133fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 0x7fb13c0c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb133fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 0x7fb13c0c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58204/0 (socket says 172.21.15.60:58204) 2023-07-13T22:31:07.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb133fff700 1 -- 172.21.15.60:0/1498373393 learned_addr learned my addr 172.21.15.60:0/1498373393 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:07.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb13b7fe700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb13c0b10d0 0x7fb13c14eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb13bfff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c15a550 0x7fb13c15c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb133fff700 1 -- 172.21.15.60:0/1498373393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb13c0b10d0 msgr2=0x7fb13c14eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb133fff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb13c0b10d0 0x7fb13c14eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb133fff700 1 -- 172.21.15.60:0/1498373393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c15a550 msgr2=0x7fb13c15c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb133fff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c15a550 0x7fb13c15c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb133fff700 1 -- 172.21.15.60:0/1498373393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134009b30 con 0x7fb13c0cbb20 2023-07-13T22:31:07.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.721+0000 7fb133fff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 0x7fb13c0c3f50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb12400c450 tx=0x7fb12400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:07.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.722+0000 7fb138ff9700 1 -- 172.21.15.60:0/1498373393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240052d0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.722+0000 7fb138ff9700 1 -- 172.21.15.60:0/1498373393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb124005430 con 0x7fb13c0cbb20 2023-07-13T22:31:07.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.722+0000 7fb138ff9700 1 -- 172.21.15.60:0/1498373393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240038c0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.722+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c0c5260 con 0x7fb13c0cbb20 2023-07-13T22:31:07.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.722+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c14f1e0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.723+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c0c6bc0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.724+0000 7fb138ff9700 1 -- 172.21.15.60:0/1498373393 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb124003a20 con 0x7fb13c0cbb20 2023-07-13T22:31:07.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.725+0000 7fb138ff9700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb11c06ace0 0x7fb11c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:07.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.725+0000 7fb138ff9700 1 -- 172.21.15.60:0/1498373393 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb1240887b0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.725+0000 7fb13b7fe700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb11c06ace0 0x7fb11c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:07.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.726+0000 7fb13b7fe700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb11c06ace0 0x7fb11c06d190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb12c006fd0 tx=0x7fb12c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:07.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.729+0000 7fb138ff9700 1 -- 172.21.15.60:0/1498373393 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb13c0c6bc0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.935+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb13c0c6bc0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.936+0000 7fb138ff9700 1 -- 172.21.15.60:0/1498373393 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v124) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb13c0c6bc0 con 0x7fb13c0cbb20 2023-07-13T22:31:07.918 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:31:07.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.938+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb11c06ace0 msgr2=0x7fb11c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.938+0000 7fb141fff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb11c06ace0 0x7fb11c06d190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb12c006fd0 tx=0x7fb12c008040 comp rx=0 tx=0).stop 2023-07-13T22:31:07.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.938+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 msgr2=0x7fb13c0c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:07.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.938+0000 7fb141fff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 0x7fb13c0c3f50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb12400c450 tx=0x7fb12400c760 comp rx=0 tx=0).stop 2023-07-13T22:31:07.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.939+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 shutdown_connections 2023-07-13T22:31:07.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.939+0000 7fb141fff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb11c06ace0 0x7fb11c06d190 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.939+0000 7fb141fff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c15a550 0x7fb13c15c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.939+0000 7fb141fff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb13c0b10d0 0x7fb13c14eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.939+0000 7fb141fff700 1 --2- 172.21.15.60:0/1498373393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb13c0cbb20 0x7fb13c0c3f50 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:07.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.939+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 >> 172.21.15.60:0/1498373393 conn(0x7fb13c06b900 msgr2=0x7fb13c06e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:07.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.939+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 shutdown_connections 2023-07-13T22:31:07.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:07.939+0000 7fb141fff700 1 -- 172.21.15.60:0/1498373393 wait complete. 2023-07-13T22:31:07.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:31:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.163+0000 7f87af7ff700 1 Processor -- start 2023-07-13T22:31:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.163+0000 7f87af7ff700 1 -- start start 2023-07-13T22:31:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.165+0000 7f87af7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87a80cfd50 0x7f87a80d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.165+0000 7f87af7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a80d1870 con 0x7f87a80b1ac0 2023-07-13T22:31:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.165+0000 7f87af7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a80cf370 con 0x7f87a80b8200 2023-07-13T22:31:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.165+0000 7f87af7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a80d12b0 con 0x7f87a80cfd50 2023-07-13T22:31:08.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.165+0000 7f87add9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f87a80b1ac0 0x7f87a80d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56712/0 (socket says 172.21.15.60:56712) 2023-07-13T22:31:08.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.165+0000 7f87add9c700 1 -- 172.21.15.60:0/1681204045 learned_addr learned my addr 172.21.15.60:0/1681204045 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:08.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.165+0000 7f87acd9a700 1 --2- 172.21.15.60:0/1681204045 >> v2:172.21.15.105:3300/0 conn(0x7f87a80cfd50 0x7f87a80d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.165+0000 7f879ffff700 1 -- 172.21.15.60:0/1681204045 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572737005 0 0) 0x7f87a80d1870 con 0x7f87a80b1ac0 2023-07-13T22:31:08.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.165+0000 7f879ffff700 1 -- 172.21.15.60:0/1681204045 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87940035b0 con 0x7f87a80b1ac0 2023-07-13T22:31:08.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.166+0000 7f879ffff700 1 -- 172.21.15.60:0/1681204045 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458321084 0 0) 0x7f87a80cf370 con 0x7f87a80b8200 2023-07-13T22:31:08.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.166+0000 7f879ffff700 1 -- 172.21.15.60:0/1681204045 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a80d1870 con 0x7f87a80b8200 2023-07-13T22:31:08.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.166+0000 7f87acd9a700 1 -- 172.21.15.60:0/1681204045 >> v1:172.21.15.105:6789/0 conn(0x7f87a80b8200 legacy=0x7f87a80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.166+0000 7f87acd9a700 1 -- 172.21.15.60:0/1681204045 >> v1:172.21.15.60:6789/0 conn(0x7f87a80b1ac0 legacy=0x7f87a80d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.166+0000 7f87acd9a700 1 -- 172.21.15.60:0/1681204045 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80d1590 con 0x7f87a80cfd50 2023-07-13T22:31:08.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.166+0000 7f87acd9a700 1 --2- 172.21.15.60:0/1681204045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87a80cfd50 0x7f87a80d23c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8798009840 tx=0x7f8798009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=433c042ba57cc2a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:08.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.166+0000 7f879ffff700 1 -- 172.21.15.60:0/1681204045 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798011830 con 0x7f87a80cfd50 2023-07-13T22:31:08.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.166+0000 7f879ffff700 1 -- 172.21.15.60:0/1681204045 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798011990 con 0x7f87a80cfd50 2023-07-13T22:31:08.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.166+0000 7f879ffff700 1 -- 172.21.15.60:0/1681204045 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798011d20 con 0x7f87a80cfd50 2023-07-13T22:31:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.167+0000 7f87af7ff700 1 -- 172.21.15.60:0/1681204045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87a80cfd50 msgr2=0x7f87a80d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.167+0000 7f87af7ff700 1 --2- 172.21.15.60:0/1681204045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87a80cfd50 0x7f87a80d23c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8798009840 tx=0x7f8798009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.167+0000 7f87af7ff700 1 -- 172.21.15.60:0/1681204045 shutdown_connections 2023-07-13T22:31:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.167+0000 7f87af7ff700 1 --2- 172.21.15.60:0/1681204045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87a80cfd50 0x7f87a80d23c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.167+0000 7f87af7ff700 1 -- 172.21.15.60:0/1681204045 >> 172.21.15.60:0/1681204045 conn(0x7f87a806b2f0 msgr2=0x7f87a806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.167+0000 7f87af7ff700 1 -- 172.21.15.60:0/1681204045 shutdown_connections 2023-07-13T22:31:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.167+0000 7f87af7ff700 1 -- 172.21.15.60:0/1681204045 wait complete. 2023-07-13T22:31:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.169+0000 7f87af7ff700 1 Processor -- start 2023-07-13T22:31:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.169+0000 7f87af7ff700 1 -- start start 2023-07-13T22:31:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87af7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87a80b1ac0 0x7f87a81561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87af7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87a80b8200 0x7f87a8148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87af7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87a80cfd50 0x7f87a80c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87af7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a81f5870 con 0x7f87a80b1ac0 2023-07-13T22:31:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87af7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80c6c60 con 0x7f87a80cfd50 2023-07-13T22:31:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87af7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80c6f40 con 0x7f87a80b8200 2023-07-13T22:31:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87ad59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87a80b1ac0 0x7f87a81561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87ad59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87a80b1ac0 0x7f87a81561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58212/0 (socket says 172.21.15.60:58212) 2023-07-13T22:31:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87ad59b700 1 -- 172.21.15.60:0/3883897879 learned_addr learned my addr 172.21.15.60:0/3883897879 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87acd9a700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87a80b8200 0x7f87a8148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87add9c700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87a80cfd50 0x7f87a80c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87ad59b700 1 -- 172.21.15.60:0/3883897879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87a80b8200 msgr2=0x7f87a8148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87ad59b700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87a80b8200 0x7f87a8148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87ad59b700 1 -- 172.21.15.60:0/3883897879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87a80cfd50 msgr2=0x7f87a80c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87ad59b700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87a80cfd50 0x7f87a80c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.170+0000 7f87ad59b700 1 -- 172.21.15.60:0/3883897879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87980094a0 con 0x7f87a80b1ac0 2023-07-13T22:31:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.171+0000 7f87ad59b700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87a80b1ac0 0x7f87a81561a0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c520 tx=0x7f87a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.171+0000 7f879dffb700 1 -- 172.21.15.60:0/3883897879 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40052d0 con 0x7f87a80b1ac0 2023-07-13T22:31:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.171+0000 7f879dffb700 1 -- 172.21.15.60:0/3883897879 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a4005430 con 0x7f87a80b1ac0 2023-07-13T22:31:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.171+0000 7f879dffb700 1 -- 172.21.15.60:0/3883897879 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40038c0 con 0x7f87a80b1ac0 2023-07-13T22:31:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.171+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a80c7220 con 0x7f87a80b1ac0 2023-07-13T22:31:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.171+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a8149500 con 0x7f87a80b1ac0 2023-07-13T22:31:08.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.172+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a804e210 con 0x7f87a80b1ac0 2023-07-13T22:31:08.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.173+0000 7f879dffb700 1 -- 172.21.15.60:0/3883897879 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87a4003a20 con 0x7f87a80b1ac0 2023-07-13T22:31:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.174+0000 7f879dffb700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f878c06ace0 0x7f878c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.174+0000 7f879dffb700 1 -- 172.21.15.60:0/3883897879 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87a4088970 con 0x7f87a80b1ac0 2023-07-13T22:31:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.174+0000 7f87acd9a700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f878c06ace0 0x7f878c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.175+0000 7f87acd9a700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f878c06ace0 0x7f878c06d190 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f87a80d0140 tx=0x7f879801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:08.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.178+0000 7f879dffb700 1 -- 172.21.15.60:0/3883897879 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f87a4053980 con 0x7f87a80b1ac0 2023-07-13T22:31:08.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.385+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87a80cee70 con 0x7f87a80b1ac0 2023-07-13T22:31:08.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.386+0000 7f879dffb700 1 -- 172.21.15.60:0/3883897879 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87a4017020 con 0x7f87a80b1ac0 2023-07-13T22:31:08.367 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:31:08.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f878c06ace0 msgr2=0x7f878c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f878c06ace0 0x7f878c06d190 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f87a80d0140 tx=0x7f879801c040 comp rx=0 tx=0).stop 2023-07-13T22:31:08.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87a80b1ac0 msgr2=0x7f87a81561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87a80b1ac0 0x7f87a81561a0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c520 tx=0x7f87a400c830 comp rx=0 tx=0).stop 2023-07-13T22:31:08.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 shutdown_connections 2023-07-13T22:31:08.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f878c06ace0 0x7f878c06d190 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87a80b1ac0 0x7f87a81561a0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87a80cfd50 0x7f87a80c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 --2- 172.21.15.60:0/3883897879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87a80b8200 0x7f87a8148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 >> 172.21.15.60:0/3883897879 conn(0x7f87a806b2f0 msgr2=0x7f87a80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:08.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 shutdown_connections 2023-07-13T22:31:08.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.388+0000 7f87af7ff700 1 -- 172.21.15.60:0/3883897879 wait complete. 2023-07-13T22:31:08.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:31:08.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.613+0000 7f28f59ff700 1 Processor -- start 2023-07-13T22:31:08.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.614+0000 7f28f59ff700 1 -- start start 2023-07-13T22:31:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.615+0000 7f28f59ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28f0159260 0x7f28f00ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.615+0000 7f28f59ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f0070af0 0x7f28f00cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.615+0000 7f28f59ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f00cf640 con 0x7f28f00cc410 2023-07-13T22:31:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.615+0000 7f28f59ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f00ccf80 con 0x7f28f0159260 2023-07-13T22:31:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.615+0000 7f28f59ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00c9b40 con 0x7f28f0070af0 2023-07-13T22:31:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.615+0000 7f28eeffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28f0159260 0x7f28f00ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.615+0000 7f28eeffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28f0159260 0x7f28f00ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58216/0 (socket says 172.21.15.60:58216) 2023-07-13T22:31:08.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.615+0000 7f28eeffd700 1 -- 172.21.15.60:0/3088926704 learned_addr learned my addr 172.21.15.60:0/3088926704 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.615+0000 7f28ef7fe700 1 --2- 172.21.15.60:0/3088926704 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f0070af0 0x7f28f00cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.616+0000 7f28eeffd700 1 -- 172.21.15.60:0/3088926704 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f0070af0 msgr2=0x7f28f00cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.616+0000 7f28eeffd700 1 --2- 172.21.15.60:0/3088926704 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f0070af0 0x7f28f00cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.616+0000 7f28eeffd700 1 -- 172.21.15.60:0/3088926704 >> v1:172.21.15.105:6789/0 conn(0x7f28f00cc410 legacy=0x7f28f00703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.616+0000 7f28eeffd700 1 -- 172.21.15.60:0/3088926704 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f00c9e20 con 0x7f28f0159260 2023-07-13T22:31:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.616+0000 7f28eeffd700 1 --2- 172.21.15.60:0/3088926704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 0x7f28f00ca900 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f28e00098f0 tx=0x7f28e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43935277ebca48f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:08.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.616+0000 7f28ee7fc700 1 -- 172.21.15.60:0/3088926704 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e000b8a0 con 0x7f28f0159260 2023-07-13T22:31:08.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.616+0000 7f28ee7fc700 1 -- 172.21.15.60:0/3088926704 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e000ba00 con 0x7f28f0159260 2023-07-13T22:31:08.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.616+0000 7f28ee7fc700 1 -- 172.21.15.60:0/3088926704 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e000bd90 con 0x7f28f0159260 2023-07-13T22:31:08.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.617+0000 7f28f59ff700 1 -- 172.21.15.60:0/3088926704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 msgr2=0x7f28f00ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.617+0000 7f28f59ff700 1 --2- 172.21.15.60:0/3088926704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 0x7f28f00ca900 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f28e00098f0 tx=0x7f28e0009c00 comp rx=0 tx=0).stop 2023-07-13T22:31:08.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.617+0000 7f28f59ff700 1 -- 172.21.15.60:0/3088926704 shutdown_connections 2023-07-13T22:31:08.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.617+0000 7f28f59ff700 1 --2- 172.21.15.60:0/3088926704 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f0070af0 0x7f28f00cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.617+0000 7f28f59ff700 1 --2- 172.21.15.60:0/3088926704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 0x7f28f00ca900 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.617+0000 7f28f59ff700 1 -- 172.21.15.60:0/3088926704 >> 172.21.15.60:0/3088926704 conn(0x7f28f00cbf50 msgr2=0x7f28f00ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:08.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.617+0000 7f28f59ff700 1 -- 172.21.15.60:0/3088926704 shutdown_connections 2023-07-13T22:31:08.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.617+0000 7f28f59ff700 1 -- 172.21.15.60:0/3088926704 wait complete. 2023-07-13T22:31:08.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.619+0000 7f28f59ff700 1 Processor -- start 2023-07-13T22:31:08.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.619+0000 7f28f59ff700 1 -- start start 2023-07-13T22:31:08.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28f59ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f0070af0 0x7f28f01548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28f59ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f00cc410 0x7f28f00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28f59ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 0x7f28f00ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28f59ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00c75f0 con 0x7f28f0159260 2023-07-13T22:31:08.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28f59ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f01e3a30 con 0x7f28f0070af0 2023-07-13T22:31:08.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28f59ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0154e30 con 0x7f28f00cc410 2023-07-13T22:31:08.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28ef7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 0x7f28f00ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28ef7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 0x7f28f00ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58218/0 (socket says 172.21.15.60:58218) 2023-07-13T22:31:08.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28ef7fe700 1 -- 172.21.15.60:0/2841285971 learned_addr learned my addr 172.21.15.60:0/2841285971 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:08.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28eeffd700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f0070af0 0x7f28f01548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28e7fff700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f00cc410 0x7f28f00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28ef7fe700 1 -- 172.21.15.60:0/2841285971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f00cc410 msgr2=0x7f28f00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28ef7fe700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f00cc410 0x7f28f00c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28ef7fe700 1 -- 172.21.15.60:0/2841285971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f0070af0 msgr2=0x7f28f01548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28ef7fe700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f0070af0 0x7f28f01548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.620+0000 7f28ef7fe700 1 -- 172.21.15.60:0/2841285971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0009550 con 0x7f28f0159260 2023-07-13T22:31:08.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.621+0000 7f28ef7fe700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 0x7f28f00ae210 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f28e8007e30 tx=0x7f28e800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:08.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.621+0000 7f28ecff9700 1 -- 172.21.15.60:0/2841285971 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e800eea0 con 0x7f28f0159260 2023-07-13T22:31:08.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.621+0000 7f28ecff9700 1 -- 172.21.15.60:0/2841285971 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e8014420 con 0x7f28f0159260 2023-07-13T22:31:08.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.621+0000 7f28ecff9700 1 -- 172.21.15.60:0/2841285971 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80147d0 con 0x7f28f0159260 2023-07-13T22:31:08.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.621+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f01e2e20 con 0x7f28f0159260 2023-07-13T22:31:08.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.621+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f01e33e0 con 0x7f28f0159260 2023-07-13T22:31:08.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.622+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f006c3c0 con 0x7f28f0159260 2023-07-13T22:31:08.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.623+0000 7f28ecff9700 1 -- 172.21.15.60:0/2841285971 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28e8014930 con 0x7f28f0159260 2023-07-13T22:31:08.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.624+0000 7f28ecff9700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28d006ad10 0x7f28d006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:08.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.624+0000 7f28ecff9700 1 -- 172.21.15.60:0/2841285971 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f28e8089c60 con 0x7f28f0159260 2023-07-13T22:31:08.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.624+0000 7f28eeffd700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28d006ad10 0x7f28d006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:08.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.625+0000 7f28eeffd700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28d006ad10 0x7f28d006d1c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f28e0014040 tx=0x7f28e0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:08.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.628+0000 7f28ecff9700 1 -- 172.21.15.60:0/2841285971 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f28e8054c70 con 0x7f28f0159260 2023-07-13T22:31:08.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.836+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28f004e210 con 0x7f28f0159260 2023-07-13T22:31:08.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.837+0000 7f28ecff9700 1 -- 172.21.15.60:0/2841285971 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28e8058290 con 0x7f28f0159260 2023-07-13T22:31:08.818 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:31:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28d006ad10 msgr2=0x7f28d006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28d006ad10 0x7f28d006d1c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f28e0014040 tx=0x7f28e0008030 comp rx=0 tx=0).stop 2023-07-13T22:31:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 msgr2=0x7f28f00ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 0x7f28f00ae210 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f28e8007e30 tx=0x7f28e800e3c0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 shutdown_connections 2023-07-13T22:31:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f28d006ad10 0x7f28d006d1c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28f0159260 0x7f28f00ae210 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f0070af0 0x7f28f01548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 --2- 172.21.15.60:0/2841285971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f28f00cc410 0x7f28f00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 >> 172.21.15.60:0/2841285971 conn(0x7f28f00cbf50 msgr2=0x7f28f00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 shutdown_connections 2023-07-13T22:31:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:08.839+0000 7f28f59ff700 1 -- 172.21.15.60:0/2841285971 wait complete. 2023-07-13T22:31:08.839 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:31:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.064+0000 7fb1c4dff700 1 Processor -- start 2023-07-13T22:31:09.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.064+0000 7fb1c4dff700 1 -- start start 2023-07-13T22:31:09.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.065+0000 7fb1c4dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c006a5e0 0x7fb1c006ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.065+0000 7fb1c4dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c00c8e90 con 0x7fb1c00add00 2023-07-13T22:31:09.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.065+0000 7fb1c4dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c00ae0d0 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.065+0000 7fb1c4dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c006d3b0 con 0x7fb1c006a5e0 2023-07-13T22:31:09.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.066+0000 7fb1be59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb1c00add00 0x7fb1c0069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56724/0 (socket says 172.21.15.60:56724) 2023-07-13T22:31:09.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.066+0000 7fb1be59c700 1 -- 172.21.15.60:0/1036445054 learned_addr learned my addr 172.21.15.60:0/1036445054 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:09.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.066+0000 7fb1bdd9b700 1 --2- 172.21.15.60:0/1036445054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c006a5e0 0x7fb1c006ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.066+0000 7fb1bd59a700 1 -- 172.21.15.60:0/1036445054 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638914116 0 0) 0x7fb1c00c8e90 con 0x7fb1c00add00 2023-07-13T22:31:09.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.066+0000 7fb1bd59a700 1 -- 172.21.15.60:0/1036445054 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a80035b0 con 0x7fb1c00add00 2023-07-13T22:31:09.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.066+0000 7fb1bd59a700 1 -- 172.21.15.60:0/1036445054 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265398679 0 0) 0x7fb1c00ae0d0 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.066+0000 7fb1bd59a700 1 -- 172.21.15.60:0/1036445054 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c00c8e90 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.066+0000 7fb1bd59a700 1 -- 172.21.15.60:0/1036445054 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967094294 0 0) 0x7fb1a80035b0 con 0x7fb1c00add00 2023-07-13T22:31:09.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.067+0000 7fb1bdd9b700 1 -- 172.21.15.60:0/1036445054 >> v1:172.21.15.105:6789/0 conn(0x7fb1c00ca9f0 legacy=0x7fb1c00b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.067+0000 7fb1bdd9b700 1 -- 172.21.15.60:0/1036445054 >> v1:172.21.15.60:6789/0 conn(0x7fb1c00add00 legacy=0x7fb1c0069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.067+0000 7fb1bdd9b700 1 -- 172.21.15.60:0/1036445054 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00c90e0 con 0x7fb1c006a5e0 2023-07-13T22:31:09.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.067+0000 7fb1bdd9b700 1 --2- 172.21.15.60:0/1036445054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c006a5e0 0x7fb1c006ce70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009870 tx=0x7fb1b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dda7cf14f17d0eec server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:09.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.067+0000 7fb1bd59a700 1 -- 172.21.15.60:0/1036445054 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b00015f0 con 0x7fb1c006a5e0 2023-07-13T22:31:09.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.067+0000 7fb1bd59a700 1 -- 172.21.15.60:0/1036445054 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4004030 con 0x7fb1c006a5e0 2023-07-13T22:31:09.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.067+0000 7fb1bd59a700 1 -- 172.21.15.60:0/1036445054 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40043c0 con 0x7fb1c006a5e0 2023-07-13T22:31:09.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.068+0000 7fb1c4dff700 1 -- 172.21.15.60:0/1036445054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c006a5e0 msgr2=0x7fb1c006ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.068+0000 7fb1c4dff700 1 --2- 172.21.15.60:0/1036445054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c006a5e0 0x7fb1c006ce70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009870 tx=0x7fb1b4009b80 comp rx=0 tx=0).stop 2023-07-13T22:31:09.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.068+0000 7fb1c4dff700 1 -- 172.21.15.60:0/1036445054 shutdown_connections 2023-07-13T22:31:09.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.068+0000 7fb1c4dff700 1 --2- 172.21.15.60:0/1036445054 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c006a5e0 0x7fb1c006ce70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.068+0000 7fb1c4dff700 1 -- 172.21.15.60:0/1036445054 >> 172.21.15.60:0/1036445054 conn(0x7fb1c00cbf50 msgr2=0x7fb1c00ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:09.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.068+0000 7fb1c4dff700 1 -- 172.21.15.60:0/1036445054 shutdown_connections 2023-07-13T22:31:09.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.068+0000 7fb1c4dff700 1 -- 172.21.15.60:0/1036445054 wait complete. 2023-07-13T22:31:09.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.070+0000 7fb1c4dff700 1 Processor -- start 2023-07-13T22:31:09.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.071+0000 7fb1c4dff700 1 -- start start 2023-07-13T22:31:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1c4dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c006a5e0 0x7fb1c015d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1c4dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c00add00 0x7fb1c0152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1c4dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1c00ca9f0 0x7fb1c00b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1c4dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c015d010 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1c4dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00c46b0 con 0x7fb1c006a5e0 2023-07-13T22:31:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1c4dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00c4990 con 0x7fb1c00add00 2023-07-13T22:31:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1bed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1c00ca9f0 0x7fb1c00b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1bed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1c00ca9f0 0x7fb1c00b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58228/0 (socket says 172.21.15.60:58228) 2023-07-13T22:31:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1bed9d700 1 -- 172.21.15.60:0/3550778459 learned_addr learned my addr 172.21.15.60:0/3550778459 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1bdd9b700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c00add00 0x7fb1c0152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.072+0000 7fb1be59c700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c006a5e0 0x7fb1c015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1bed9d700 1 -- 172.21.15.60:0/3550778459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c00add00 msgr2=0x7fb1c0152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1bed9d700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c00add00 0x7fb1c0152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1bed9d700 1 -- 172.21.15.60:0/3550778459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c006a5e0 msgr2=0x7fb1c015d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1bed9d700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c006a5e0 0x7fb1c015d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1bed9d700 1 -- 172.21.15.60:0/3550778459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40094a0 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1bed9d700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1c00ca9f0 0x7fb1c00b26c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800cc50 tx=0x7fb1b800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1aeffd700 1 -- 172.21.15.60:0/3550778459 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8003c00 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1aeffd700 1 -- 172.21.15.60:0/3550778459 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8003d60 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1aeffd700 1 -- 172.21.15.60:0/3550778459 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800ab10 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c00c4c70 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.073+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c00c2ed0 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.074+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c004e210 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.075+0000 7fb1aeffd700 1 -- 172.21.15.60:0/3550778459 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb1b800ac70 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.077+0000 7fb1aeffd700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb1a006ace0 0x7fb1a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.077+0000 7fb1aeffd700 1 -- 172.21.15.60:0/3550778459 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb1b8089570 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.077+0000 7fb1be59c700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb1a006ace0 0x7fb1a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.078+0000 7fb1be59c700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb1a006ace0 0x7fb1a006d190 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00011e0 tx=0x7fb1b0001170 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.081+0000 7fb1aeffd700 1 -- 172.21.15.60:0/3550778459 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb1b8054580 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.285+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1c006a250 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.285+0000 7fb1aeffd700 1 -- 172.21.15.60:0/3550778459 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1b8057ba0 con 0x7fb1c00ca9f0 2023-07-13T22:31:09.267 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:31:09.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb1a006ace0 msgr2=0x7fb1a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb1a006ace0 0x7fb1a006d190 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00011e0 tx=0x7fb1b0001170 comp rx=0 tx=0).stop 2023-07-13T22:31:09.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1c00ca9f0 msgr2=0x7fb1c00b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1c00ca9f0 0x7fb1c00b26c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800cc50 tx=0x7fb1b800cf60 comp rx=0 tx=0).stop 2023-07-13T22:31:09.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 shutdown_connections 2023-07-13T22:31:09.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb1a006ace0 0x7fb1a006d190 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1c00ca9f0 0x7fb1c00b26c0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1c006a5e0 0x7fb1c015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 --2- 172.21.15.60:0/3550778459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1c00add00 0x7fb1c0152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 >> 172.21.15.60:0/3550778459 conn(0x7fb1c00cbf50 msgr2=0x7fb1c006cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:09.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 shutdown_connections 2023-07-13T22:31:09.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.288+0000 7fb1c4dff700 1 -- 172.21.15.60:0/3550778459 wait complete. 2023-07-13T22:31:09.288 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:31:09.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.512+0000 7faa1b1ff700 1 Processor -- start 2023-07-13T22:31:09.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.512+0000 7faa1b1ff700 1 -- start start 2023-07-13T22:31:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.513+0000 7faa1b1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa140d3750 0x7faa140cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa1b1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa140cd950 con 0x7faa140b8400 2023-07-13T22:31:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa1b1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa140cff20 con 0x7faa140cfad0 2023-07-13T22:31:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa1b1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa140d0ac0 con 0x7faa140d3750 2023-07-13T22:31:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa18f9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faa140b8400 0x7faa140cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56738/0 (socket says 172.21.15.60:56738) 2023-07-13T22:31:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa18f9b700 1 -- 172.21.15.60:0/3996120258 learned_addr learned my addr 172.21.15.60:0/3996120258 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:09.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa13fff700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa140d3750 0x7faa140cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388659858 0 0) 0x7faa08001110 con 0x7faa140b8400 2023-07-13T22:31:09.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa040035b0 con 0x7faa140b8400 2023-07-13T22:31:09.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118075436 0 0) 0x7faa0c001490 con 0x7faa140cfad0 2023-07-13T22:31:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa08001110 con 0x7faa140cfad0 2023-07-13T22:31:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3582363805 0 0) 0x7faa040035b0 con 0x7faa140b8400 2023-07-13T22:31:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa140d3750 msgr2=0x7faa140cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa137fe700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa140d3750 0x7faa140cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 >> v1:172.21.15.105:6789/0 conn(0x7faa140cfad0 legacy=0x7faa140d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.514+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa140b87d0 con 0x7faa140b8400 2023-07-13T22:31:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7faa08001800 con 0x7faa140b8400 2023-07-13T22:31:09.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 >> v1:172.21.15.60:6789/0 conn(0x7faa140b8400 legacy=0x7faa140cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa137fe700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa04003bc0 0x7faa04003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa137fe700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa04004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa137fe700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa04005200 0x7faa0400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa140d0ac0 con 0x7faa04004850 2023-07-13T22:31:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0400c060 con 0x7faa04003bc0 2023-07-13T22:31:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0400c220 con 0x7faa04005200 2023-07-13T22:31:09.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa13fff700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa140d3750 0x7faa140cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa18f9b700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa04003bc0 0x7faa04003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa13fff700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa04004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa1979c700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa04005200 0x7faa0400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa13fff700 1 -- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa04005200 msgr2=0x7faa0400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa13fff700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa04005200 0x7faa0400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa13fff700 1 -- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa04003bc0 msgr2=0x7faa04003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa13fff700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa04003bc0 0x7faa04003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.515+0000 7faa13fff700 1 -- 172.21.15.60:0/3996120258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa040035b0 con 0x7faa04004850 2023-07-13T22:31:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.516+0000 7faa13fff700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa04004cc0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7faa140d27d0 tx=0x7faa04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71255974d1d13dd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.516+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00004030 con 0x7faa04004850 2023-07-13T22:31:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.516+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa00004190 con 0x7faa04004850 2023-07-13T22:31:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.516+0000 7faa137fe700 1 -- 172.21.15.60:0/3996120258 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00004540 con 0x7faa04004850 2023-07-13T22:31:09.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.516+0000 7faa1b1ff700 1 -- 172.21.15.60:0/3996120258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 msgr2=0x7faa04004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.516+0000 7faa1b1ff700 1 --2- 172.21.15.60:0/3996120258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa04004cc0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7faa140d27d0 tx=0x7faa04005880 comp rx=0 tx=0).stop 2023-07-13T22:31:09.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.516+0000 7faa1979c700 1 -- 172.21.15.60:0/3996120258 reap_dead start 2023-07-13T22:31:09.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.517+0000 7faa1b1ff700 1 -- 172.21.15.60:0/3996120258 shutdown_connections 2023-07-13T22:31:09.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.517+0000 7faa1b1ff700 1 -- 172.21.15.60:0/3996120258 >> 172.21.15.60:0/3996120258 conn(0x7faa1406b390 msgr2=0x7faa1406b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:09.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.517+0000 7faa1b1ff700 1 -- 172.21.15.60:0/3996120258 shutdown_connections 2023-07-13T22:31:09.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.517+0000 7faa1b1ff700 1 -- 172.21.15.60:0/3996120258 wait complete. 2023-07-13T22:31:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.519+0000 7faa1b1ff700 1 Processor -- start 2023-07-13T22:31:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.519+0000 7faa1b1ff700 1 -- start start 2023-07-13T22:31:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.520+0000 7faa1b1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa14154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa1b1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa141f2b60 0x7faa14160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa1b1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa14160580 0x7faa141f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa1b1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa141f2740 con 0x7faa04004850 2023-07-13T22:31:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa1b1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1415d530 con 0x7faa141f2b60 2023-07-13T22:31:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa1b1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa141609f0 con 0x7faa14160580 2023-07-13T22:31:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa18f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa14154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa18f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa14154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58252/0 (socket says 172.21.15.60:58252) 2023-07-13T22:31:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa18f9b700 1 -- 172.21.15.60:0/1116983737 learned_addr learned my addr 172.21.15.60:0/1116983737 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa1979c700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa14160580 0x7faa141f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa13fff700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa141f2b60 0x7faa14160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa18f9b700 1 -- 172.21.15.60:0/1116983737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa14160580 msgr2=0x7faa141f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa18f9b700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa14160580 0x7faa141f54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa18f9b700 1 -- 172.21.15.60:0/1116983737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa141f2b60 msgr2=0x7faa14160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa18f9b700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa141f2b60 0x7faa14160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.521+0000 7faa18f9b700 1 -- 172.21.15.60:0/1116983737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa141f63c0 con 0x7faa04004850 2023-07-13T22:31:09.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.522+0000 7faa18f9b700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa14154140 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7faa08007310 tx=0x7faa080076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.522+0000 7faa117fa700 1 -- 172.21.15.60:0/1116983737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa080077f0 con 0x7faa04004850 2023-07-13T22:31:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.522+0000 7faa117fa700 1 -- 172.21.15.60:0/1116983737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0800f870 con 0x7faa04004850 2023-07-13T22:31:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.522+0000 7faa117fa700 1 -- 172.21.15.60:0/1116983737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800fc40 con 0x7faa04004850 2023-07-13T22:31:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.522+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa141f60e0 con 0x7faa04004850 2023-07-13T22:31:09.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.522+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa141f2360 con 0x7faa04004850 2023-07-13T22:31:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.523+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa141f2880 con 0x7faa04004850 2023-07-13T22:31:09.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.524+0000 7faa117fa700 1 -- 172.21.15.60:0/1116983737 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa0801c410 con 0x7faa04004850 2023-07-13T22:31:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.525+0000 7faa117fa700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9f806ad10 0x7fa9f806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.525+0000 7faa117fa700 1 -- 172.21.15.60:0/1116983737 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa08088ed0 con 0x7faa04004850 2023-07-13T22:31:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.525+0000 7faa13fff700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9f806ad10 0x7fa9f806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.526+0000 7faa13fff700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9f806ad10 0x7fa9f806d1c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7faa00016040 tx=0x7faa00008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:09.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.529+0000 7faa117fa700 1 -- 172.21.15.60:0/1116983737 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7faa141f2880 con 0x7faa04004850 2023-07-13T22:31:09.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.755+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa140cd950 con 0x7faa04004850 2023-07-13T22:31:09.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.756+0000 7faa117fa700 1 -- 172.21.15.60:0/1116983737 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v124) v1 ==== 80+0+0 (secure 0 0 0) 0x7faa140cd950 con 0x7faa04004850 2023-07-13T22:31:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9f806ad10 msgr2=0x7fa9f806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9f806ad10 0x7fa9f806d1c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7faa00016040 tx=0x7faa00008070 comp rx=0 tx=0).stop 2023-07-13T22:31:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 msgr2=0x7faa14154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:09.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa14154140 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7faa08007310 tx=0x7faa080076d0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 shutdown_connections 2023-07-13T22:31:09.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa9f806ad10 0x7fa9f806d1c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa04004850 0x7faa14154140 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa14160580 0x7faa141f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 --2- 172.21.15.60:0/1116983737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa141f2b60 0x7faa14160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:09.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 >> 172.21.15.60:0/1116983737 conn(0x7faa1406b390 msgr2=0x7faa1415af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:09.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 shutdown_connections 2023-07-13T22:31:09.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:09.758+0000 7faa1b1ff700 1 -- 172.21.15.60:0/1116983737 wait complete. 2023-07-13T22:31:09.742 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:31:09.758 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:31:09.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:09.760 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:31:09.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[127147]: fuse finished with error 0 and tester_r 0 2023-07-13T22:31:09.837 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:31:09.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:09.839 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:31:09.963 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:31:09.964 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:10.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:31:10.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:31:10.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:10.012 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:31:10.097 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-13T22:31:10.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:31:10.098 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:31:10.099 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:10.146 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:31:10.146 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:31:10.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:10.147 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:31:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-13T22:31:10.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:31:10.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.454+0000 7fefce3ff700 1 Processor -- start 2023-07-13T22:31:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.454+0000 7fefce3ff700 1 -- start start 2023-07-13T22:31:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefce3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc8154950 0x7fefc8070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefce3ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fefc80c9450 0x7fefc815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefce3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc8070270 con 0x7fefc815a820 2023-07-13T22:31:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefce3ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc815b2e0 con 0x7fefc80c9450 2023-07-13T22:31:10.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefce3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc815b5c0 con 0x7fefc8154950 2023-07-13T22:31:10.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefcc99c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fefc80c9450 0x7fefc815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefc77fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fefc815a820 0x7fefc80c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56754/0 (socket says 172.21.15.60:56754) 2023-07-13T22:31:10.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefcc99c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fefc80c9450 0x7fefc815ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58254/0 (socket says 172.21.15.60:58254) 2023-07-13T22:31:10.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefc77fe700 1 -- 172.21.15.60:0/2285961889 learned_addr learned my addr 172.21.15.60:0/2285961889 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:10.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefc7fff700 1 --2- 172.21.15.60:0/2285961889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc8154950 0x7fefc8070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefcc99c700 1 -- 172.21.15.60:0/2285961889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc8154950 msgr2=0x7fefc8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:10.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefcc99c700 1 --2- 172.21.15.60:0/2285961889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc8154950 0x7fefc8070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefcc99c700 1 -- 172.21.15.60:0/2285961889 >> v1:172.21.15.60:6789/0 conn(0x7fefc815a820 legacy=0x7fefc80c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.457+0000 7fefcc99c700 1 -- 172.21.15.60:0/2285961889 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc80704c0 con 0x7fefc80c9450 2023-07-13T22:31:10.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.456+0000 7fefc6ffd700 1 -- 172.21.15.60:0/2285961889 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143416385 0 0) 0x7fefc8070270 con 0x7fefc815a820 2023-07-13T22:31:10.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.457+0000 7fefcc99c700 1 --2- 172.21.15.60:0/2285961889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc80c9450 0x7fefc815ac90 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009d90 tx=0x7fefc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1efc585e1661291 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:10.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.457+0000 7fefc6ffd700 1 -- 172.21.15.60:0/2285961889 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc000e070 con 0x7fefc80c9450 2023-07-13T22:31:10.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.457+0000 7fefc6ffd700 1 -- 172.21.15.60:0/2285961889 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc000bb20 con 0x7fefc80c9450 2023-07-13T22:31:10.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.457+0000 7fefc6ffd700 1 -- 172.21.15.60:0/2285961889 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0012430 con 0x7fefc80c9450 2023-07-13T22:31:10.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.457+0000 7fefce3ff700 1 -- 172.21.15.60:0/2285961889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc80c9450 msgr2=0x7fefc815ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.457+0000 7fefce3ff700 1 --2- 172.21.15.60:0/2285961889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc80c9450 0x7fefc815ac90 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009d90 tx=0x7fefc000b330 comp rx=0 tx=0).stop 2023-07-13T22:31:10.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.458+0000 7fefce3ff700 1 -- 172.21.15.60:0/2285961889 shutdown_connections 2023-07-13T22:31:10.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.458+0000 7fefce3ff700 1 --2- 172.21.15.60:0/2285961889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc80c9450 0x7fefc815ac90 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.458+0000 7fefce3ff700 1 --2- 172.21.15.60:0/2285961889 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc8154950 0x7fefc8070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.458+0000 7fefce3ff700 1 -- 172.21.15.60:0/2285961889 >> 172.21.15.60:0/2285961889 conn(0x7fefc80cf410 msgr2=0x7fefc80cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:10.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.458+0000 7fefce3ff700 1 -- 172.21.15.60:0/2285961889 shutdown_connections 2023-07-13T22:31:10.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.458+0000 7fefce3ff700 1 -- 172.21.15.60:0/2285961889 wait complete. 2023-07-13T22:31:10.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.460+0000 7fefce3ff700 1 Processor -- start 2023-07-13T22:31:10.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.460+0000 7fefce3ff700 1 -- start start 2023-07-13T22:31:10.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefce3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefc80c9450 0x7fefc814c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefce3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc8154950 0x7fefc80b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefce3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc815a820 0x7fefc80b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefce3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80b0440 con 0x7fefc8154950 2023-07-13T22:31:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefce3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80af3a0 con 0x7fefc80c9450 2023-07-13T22:31:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefce3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80af680 con 0x7fefc815a820 2023-07-13T22:31:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc8154950 0x7fefc80b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc8154950 0x7fefc80b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58256/0 (socket says 172.21.15.60:58256) 2023-07-13T22:31:10.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc77fe700 1 -- 172.21.15.60:0/2854718637 learned_addr learned my addr 172.21.15.60:0/2854718637 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:10.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefcc99c700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc815a820 0x7fefc80b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc7fff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefc80c9450 0x7fefc814c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc77fe700 1 -- 172.21.15.60:0/2854718637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc815a820 msgr2=0x7fefc80b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc77fe700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc815a820 0x7fefc80b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc77fe700 1 -- 172.21.15.60:0/2854718637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefc80c9450 msgr2=0x7fefc814c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc77fe700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefc80c9450 0x7fefc814c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc77fe700 1 -- 172.21.15.60:0/2854718637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc0009a70 con 0x7fefc8154950 2023-07-13T22:31:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.462+0000 7fefc7fff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefc80c9450 0x7fefc814c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.463+0000 7fefc77fe700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc8154950 0x7fefc80b0ae0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c550 tx=0x7fefbc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.463+0000 7fefc4ff9700 1 -- 172.21.15.60:0/2854718637 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc004c80 con 0x7fefc8154950 2023-07-13T22:31:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.463+0000 7fefc4ff9700 1 -- 172.21.15.60:0/2854718637 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc004e00 con 0x7fefc8154950 2023-07-13T22:31:10.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.463+0000 7fefc4ff9700 1 -- 172.21.15.60:0/2854718637 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc007310 con 0x7fefc8154950 2023-07-13T22:31:10.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.463+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc80af960 con 0x7fefc8154950 2023-07-13T22:31:10.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.463+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc80aff20 con 0x7fefc8154950 2023-07-13T22:31:10.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.464+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc80b0140 con 0x7fefc8154950 2023-07-13T22:31:10.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.465+0000 7fefc4ff9700 1 -- 172.21.15.60:0/2854718637 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefbc007470 con 0x7fefc8154950 2023-07-13T22:31:10.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.466+0000 7fefc4ff9700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fefb406ace0 0x7fefb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.466+0000 7fefc4ff9700 1 -- 172.21.15.60:0/2854718637 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fefbc089290 con 0x7fefc8154950 2023-07-13T22:31:10.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.467+0000 7fefc7fff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fefb406ace0 0x7fefb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.467+0000 7fefc7fff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fefb406ace0 0x7fefb406d190 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fefc815a360 tx=0x7fefb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:10.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.471+0000 7fefc4ff9700 1 -- 172.21.15.60:0/2854718637 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fefbc0542a0 con 0x7fefc8154950 2023-07-13T22:31:10.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.725+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefc804e280 con 0x7fefc8154950 2023-07-13T22:31:10.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.726+0000 7fefc4ff9700 1 -- 172.21.15.60:0/2854718637 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4714 (secure 0 0 0) 0x7fefbc0578c0 con 0x7fefc8154950 2023-07-13T22:31:10.708 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:10.708 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:30:39.690428+0000","modified":"2023-07-13T22:31:00.559611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6836/1781435251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1781435251},{"type":"v1","addr":"172.21.15.60:6836","nonce":1781435251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-13T22:31:10.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.728+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fefb406ace0 msgr2=0x7fefb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.728+0000 7fefce3ff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fefb406ace0 0x7fefb406d190 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fefc815a360 tx=0x7fefb8008040 comp rx=0 tx=0).stop 2023-07-13T22:31:10.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.728+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc8154950 msgr2=0x7fefc80b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.728+0000 7fefce3ff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc8154950 0x7fefc80b0ae0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c550 tx=0x7fefbc00c910 comp rx=0 tx=0).stop 2023-07-13T22:31:10.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.729+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 shutdown_connections 2023-07-13T22:31:10.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.729+0000 7fefce3ff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fefb406ace0 0x7fefb406d190 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.729+0000 7fefce3ff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefc815a820 0x7fefc80b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.729+0000 7fefce3ff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefc80c9450 0x7fefc814c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.729+0000 7fefce3ff700 1 --2- 172.21.15.60:0/2854718637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefc8154950 0x7fefc80b0ae0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.729+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 >> 172.21.15.60:0/2854718637 conn(0x7fefc80cf410 msgr2=0x7fefc8153be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.729+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 shutdown_connections 2023-07-13T22:31:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.729+0000 7fefce3ff700 1 -- 172.21.15.60:0/2854718637 wait complete. 2023-07-13T22:31:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 87 2023-07-13T22:31:10.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:10.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.949+0000 7f61f6fff700 1 Processor -- start 2023-07-13T22:31:10.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.949+0000 7f61f6fff700 1 -- start start 2023-07-13T22:31:10.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.950+0000 7f61f6fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f0070270 0x7f61f00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.950+0000 7f61f6fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61f00c9c50 0x7f61f00ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.950+0000 7f61f6fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f015d970 con 0x7f61f0159170 2023-07-13T22:31:10.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61f6fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f0070c00 con 0x7f61f00c9c50 2023-07-13T22:31:10.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61f6fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f015cd60 con 0x7f61f0070270 2023-07-13T22:31:10.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61f559c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61f00c9c50 0x7f61f00ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61f4d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f61f0159170 0x7f61f00c4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56772/0 (socket says 172.21.15.60:56772) 2023-07-13T22:31:10.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61f4d9b700 1 -- 172.21.15.60:0/734662230 learned_addr learned my addr 172.21.15.60:0/734662230 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:10.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61effff700 1 --2- 172.21.15.60:0/734662230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f0070270 0x7f61f00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61f559c700 1 -- 172.21.15.60:0/734662230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f0070270 msgr2=0x7f61f00706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:10.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61f559c700 1 --2- 172.21.15.60:0/734662230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f0070270 0x7f61f00706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61f559c700 1 -- 172.21.15.60:0/734662230 >> v1:172.21.15.60:6789/0 conn(0x7f61f0159170 legacy=0x7f61f00c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61f559c700 1 -- 172.21.15.60:0/734662230 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f015d040 con 0x7f61f00c9c50 2023-07-13T22:31:10.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.951+0000 7f61ef7fe700 1 -- 172.21.15.60:0/734662230 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208008274 0 0) 0x7f61f015d970 con 0x7f61f0159170 2023-07-13T22:31:10.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61f559c700 1 --2- 172.21.15.60:0/734662230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f00c9c50 0x7f61f00ca070 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009d90 tx=0x7f61e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb850e1daab2f0bf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:10.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61ef7fe700 1 -- 172.21.15.60:0/734662230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e800e070 con 0x7f61f00c9c50 2023-07-13T22:31:10.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61ef7fe700 1 -- 172.21.15.60:0/734662230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e800bb20 con 0x7f61f00c9c50 2023-07-13T22:31:10.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61ef7fe700 1 -- 172.21.15.60:0/734662230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8012430 con 0x7f61f00c9c50 2023-07-13T22:31:10.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61f6fff700 1 -- 172.21.15.60:0/734662230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f00c9c50 msgr2=0x7f61f00ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61f6fff700 1 --2- 172.21.15.60:0/734662230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f00c9c50 0x7f61f00ca070 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009d90 tx=0x7f61e800b330 comp rx=0 tx=0).stop 2023-07-13T22:31:10.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61f6fff700 1 -- 172.21.15.60:0/734662230 shutdown_connections 2023-07-13T22:31:10.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61f6fff700 1 --2- 172.21.15.60:0/734662230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f00c9c50 0x7f61f00ca070 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61f6fff700 1 --2- 172.21.15.60:0/734662230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f0070270 0x7f61f00706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.952+0000 7f61f6fff700 1 -- 172.21.15.60:0/734662230 >> 172.21.15.60:0/734662230 conn(0x7f61f00cf410 msgr2=0x7f61f00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:10.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.953+0000 7f61f6fff700 1 -- 172.21.15.60:0/734662230 shutdown_connections 2023-07-13T22:31:10.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.953+0000 7f61f6fff700 1 -- 172.21.15.60:0/734662230 wait complete. 2023-07-13T22:31:10.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.954+0000 7f61f6fff700 1 Processor -- start 2023-07-13T22:31:10.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.954+0000 7f61f6fff700 1 -- start start 2023-07-13T22:31:10.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.955+0000 7f61f6fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f0070270 0x7f61f014c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.955+0000 7f61f6fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f00c9c50 0x7f61f0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.955+0000 7f61f6fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61f0159170 0x7f61f006a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.955+0000 7f61f6fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f014c9e0 con 0x7f61f0070270 2023-07-13T22:31:10.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.955+0000 7f61f6fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00b31b0 con 0x7f61f0159170 2023-07-13T22:31:10.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.955+0000 7f61f6fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00b2910 con 0x7f61f00c9c50 2023-07-13T22:31:10.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f4d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f0070270 0x7f61f014c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f4d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f0070270 0x7f61f014c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58272/0 (socket says 172.21.15.60:58272) 2023-07-13T22:31:10.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f4d9b700 1 -- 172.21.15.60:0/1437937385 learned_addr learned my addr 172.21.15.60:0/1437937385 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:10.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f559c700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61f0159170 0x7f61f006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61effff700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f00c9c50 0x7f61f0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f4d9b700 1 -- 172.21.15.60:0/1437937385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f00c9c50 msgr2=0x7f61f0069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f4d9b700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f00c9c50 0x7f61f0069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f4d9b700 1 -- 172.21.15.60:0/1437937385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61f0159170 msgr2=0x7f61f006a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:10.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f4d9b700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61f0159170 0x7f61f006a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:10.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f4d9b700 1 -- 172.21.15.60:0/1437937385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e8009a70 con 0x7f61f0070270 2023-07-13T22:31:10.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61f4d9b700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f0070270 0x7f61f014c4a0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f61e40011e0 tx=0x7f61e400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:10.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.956+0000 7f61ed7fa700 1 -- 172.21.15.60:0/1437937385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4004c80 con 0x7f61f0070270 2023-07-13T22:31:10.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.957+0000 7f61ed7fa700 1 -- 172.21.15.60:0/1437937385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e4004e00 con 0x7f61f0070270 2023-07-13T22:31:10.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.957+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f00b2bf0 con 0x7f61f0070270 2023-07-13T22:31:10.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.957+0000 7f61ed7fa700 1 -- 172.21.15.60:0/1437937385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4007310 con 0x7f61f0070270 2023-07-13T22:31:10.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.957+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f00b1ba0 con 0x7f61f0070270 2023-07-13T22:31:10.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.958+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f00c9830 con 0x7f61f0070270 2023-07-13T22:31:10.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.959+0000 7f61ed7fa700 1 -- 172.21.15.60:0/1437937385 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61e4007470 con 0x7f61f0070270 2023-07-13T22:31:10.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.960+0000 7f61ed7fa700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61d406ace0 0x7f61d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:10.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.960+0000 7f61ed7fa700 1 -- 172.21.15.60:0/1437937385 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61e40891c0 con 0x7f61f0070270 2023-07-13T22:31:10.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.960+0000 7f61effff700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61d406ace0 0x7f61d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:10.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.961+0000 7f61effff700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61d406ace0 0x7f61d406d190 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f61f00b4440 tx=0x7f61dc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:10.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:10.964+0000 7f61ed7fa700 1 -- 172.21.15.60:0/1437937385 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f61e40541d0 con 0x7f61f0070270 2023-07-13T22:31:11.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.175+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61f00af810 con 0x7f61f0070270 2023-07-13T22:31:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.176+0000 7f61ed7fa700 1 -- 172.21.15.60:0/1437937385 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18215 (secure 0 0 0) 0x7f61e40577f0 con 0x7f61f0070270 2023-07-13T22:31:11.158 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:11.158 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":124,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:30:41.182757+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:30:37.983326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-13T22:30:38.665534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:11.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.178+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61d406ace0 msgr2=0x7f61d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.178+0000 7f61f6fff700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61d406ace0 0x7f61d406d190 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f61f00b4440 tx=0x7f61dc009210 comp rx=0 tx=0).stop 2023-07-13T22:31:11.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.178+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f0070270 msgr2=0x7f61f014c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.178+0000 7f61f6fff700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f0070270 0x7f61f014c4a0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f61e40011e0 tx=0x7f61e400ca30 comp rx=0 tx=0).stop 2023-07-13T22:31:11.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.178+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 shutdown_connections 2023-07-13T22:31:11.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.179+0000 7f61f6fff700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f61d406ace0 0x7f61d406d190 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.179+0000 7f61f6fff700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61f0070270 0x7f61f014c4a0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.179+0000 7f61f6fff700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61f0159170 0x7f61f006a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.179+0000 7f61f6fff700 1 --2- 172.21.15.60:0/1437937385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61f00c9c50 0x7f61f0069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.179+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 >> 172.21.15.60:0/1437937385 conn(0x7f61f00cf410 msgr2=0x7f61f006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:11.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.179+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 shutdown_connections 2023-07-13T22:31:11.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.179+0000 7f61f6fff700 1 -- 172.21.15.60:0/1437937385 wait complete. 2023-07-13T22:31:11.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:30:37.983326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:30:38.665534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:31:11.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:31:11.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.406+0000 7fa71ffff700 1 Processor -- start 2023-07-13T22:31:11.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.406+0000 7fa71ffff700 1 -- start start 2023-07-13T22:31:11.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.407+0000 7fa71ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b8200 0x7fa7180cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.407+0000 7fa71ffff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7180d1870 con 0x7fa7180cede0 2023-07-13T22:31:11.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.407+0000 7fa71ffff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa718160d80 con 0x7fa7180b1ac0 2023-07-13T22:31:11.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.407+0000 7fa71ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180cf370 con 0x7fa7180b8200 2023-07-13T22:31:11.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.407+0000 7fa71e59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa7180cede0 0x7fa718160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56774/0 (socket says 172.21.15.60:56774) 2023-07-13T22:31:11.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.407+0000 7fa71e59c700 1 -- 172.21.15.60:0/2901431997 learned_addr learned my addr 172.21.15.60:0/2901431997 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:11.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.408+0000 7fa71dd9b700 1 --2- 172.21.15.60:0/2901431997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b8200 0x7fa7180cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.408+0000 7fa71cd99700 1 -- 172.21.15.60:0/2901431997 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815236997 0 0) 0x7fa7180d1870 con 0x7fa7180cede0 2023-07-13T22:31:11.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.408+0000 7fa71cd99700 1 -- 172.21.15.60:0/2901431997 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7040035b0 con 0x7fa7180cede0 2023-07-13T22:31:11.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.408+0000 7fa71cd99700 1 -- 172.21.15.60:0/2901431997 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550500791 0 0) 0x7fa718160d80 con 0x7fa7180b1ac0 2023-07-13T22:31:11.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.408+0000 7fa71cd99700 1 -- 172.21.15.60:0/2901431997 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7180d1870 con 0x7fa7180b1ac0 2023-07-13T22:31:11.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.408+0000 7fa71dd9b700 1 -- 172.21.15.60:0/2901431997 >> v1:172.21.15.105:6789/0 conn(0x7fa7180b1ac0 legacy=0x7fa71815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.408+0000 7fa71dd9b700 1 -- 172.21.15.60:0/2901431997 >> v1:172.21.15.60:6789/0 conn(0x7fa7180cede0 legacy=0x7fa718160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.408+0000 7fa71dd9b700 1 -- 172.21.15.60:0/2901431997 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180d12b0 con 0x7fa7180b8200 2023-07-13T22:31:11.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.408+0000 7fa71dd9b700 1 --2- 172.21.15.60:0/2901431997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b8200 0x7fa7180cf640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa714009870 tx=0x7fa714009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc240fd27688ac3c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:11.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71cd99700 1 -- 172.21.15.60:0/2901431997 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714011830 con 0x7fa7180b8200 2023-07-13T22:31:11.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71cd99700 1 -- 172.21.15.60:0/2901431997 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa714011990 con 0x7fa7180b8200 2023-07-13T22:31:11.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71cd99700 1 -- 172.21.15.60:0/2901431997 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714011d20 con 0x7fa7180b8200 2023-07-13T22:31:11.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71ffff700 1 -- 172.21.15.60:0/2901431997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b8200 msgr2=0x7fa7180cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71ffff700 1 --2- 172.21.15.60:0/2901431997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b8200 0x7fa7180cf640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa714009870 tx=0x7fa714009b80 comp rx=0 tx=0).stop 2023-07-13T22:31:11.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71ffff700 1 -- 172.21.15.60:0/2901431997 shutdown_connections 2023-07-13T22:31:11.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71ffff700 1 --2- 172.21.15.60:0/2901431997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b8200 0x7fa7180cf640 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71ffff700 1 -- 172.21.15.60:0/2901431997 >> 172.21.15.60:0/2901431997 conn(0x7fa71806b2f0 msgr2=0x7fa71806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:11.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71ffff700 1 -- 172.21.15.60:0/2901431997 shutdown_connections 2023-07-13T22:31:11.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.409+0000 7fa71ffff700 1 -- 172.21.15.60:0/2901431997 wait complete. 2023-07-13T22:31:11.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.411+0000 7fa71ffff700 1 Processor -- start 2023-07-13T22:31:11.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.411+0000 7fa71ffff700 1 -- start start 2023-07-13T22:31:11.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.412+0000 7fa71ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b1ac0 0x7fa718155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.412+0000 7fa71ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7180b8200 0x7fa7180c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.412+0000 7fa71ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7180cede0 0x7fa718146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.412+0000 7fa71ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180cc0f0 con 0x7fa7180b8200 2023-07-13T22:31:11.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.412+0000 7fa71ffff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180cb800 con 0x7fa7180cede0 2023-07-13T22:31:11.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.412+0000 7fa71ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180cbaa0 con 0x7fa7180b1ac0 2023-07-13T22:31:11.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71d59a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7180b8200 0x7fa7180c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71d59a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7180b8200 0x7fa7180c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58286/0 (socket says 172.21.15.60:58286) 2023-07-13T22:31:11.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71d59a700 1 -- 172.21.15.60:0/878219958 learned_addr learned my addr 172.21.15.60:0/878219958 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:11.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71dd9b700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b1ac0 0x7fa718155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71e59c700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7180cede0 0x7fa718146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71d59a700 1 -- 172.21.15.60:0/878219958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b1ac0 msgr2=0x7fa718155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71d59a700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b1ac0 0x7fa718155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71d59a700 1 -- 172.21.15.60:0/878219958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7180cede0 msgr2=0x7fa718146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71d59a700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7180cede0 0x7fa718146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71d59a700 1 -- 172.21.15.60:0/878219958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7140094a0 con 0x7fa7180b8200 2023-07-13T22:31:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71d59a700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7180b8200 0x7fa7180c9230 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa70800c520 tx=0x7fa70800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa70e7fc700 1 -- 172.21.15.60:0/878219958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080052d0 con 0x7fa7180b8200 2023-07-13T22:31:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa70e7fc700 1 -- 172.21.15.60:0/878219958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708005430 con 0x7fa7180b8200 2023-07-13T22:31:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa70e7fc700 1 -- 172.21.15.60:0/878219958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080038c0 con 0x7fa7180b8200 2023-07-13T22:31:11.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.413+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7180cbd80 con 0x7fa7180b8200 2023-07-13T22:31:11.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.414+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa718158d30 con 0x7fa7180b8200 2023-07-13T22:31:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.415+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7180cc260 con 0x7fa7180b8200 2023-07-13T22:31:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.416+0000 7fa70e7fc700 1 -- 172.21.15.60:0/878219958 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa708003a20 con 0x7fa7180b8200 2023-07-13T22:31:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.416+0000 7fa70e7fc700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6fc06ace0 0x7fa6fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.417+0000 7fa70e7fc700 1 -- 172.21.15.60:0/878219958 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa708088b90 con 0x7fa7180b8200 2023-07-13T22:31:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.417+0000 7fa71dd9b700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6fc06ace0 0x7fa6fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.417+0000 7fa71dd9b700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6fc06ace0 0x7fa6fc06d190 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa7140097c0 tx=0x7fa71401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:11.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.421+0000 7fa70e7fc700 1 -- 172.21.15.60:0/878219958 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa708053b10 con 0x7fa7180b8200 2023-07-13T22:31:11.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.626+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa71804e210 con 0x7fa7180b8200 2023-07-13T22:31:11.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.627+0000 7fa70e7fc700 1 -- 172.21.15.60:0/878219958 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v124) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa708017020 con 0x7fa7180b8200 2023-07-13T22:31:11.609 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:31:11.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6fc06ace0 msgr2=0x7fa6fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6fc06ace0 0x7fa6fc06d190 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa7140097c0 tx=0x7fa71401c040 comp rx=0 tx=0).stop 2023-07-13T22:31:11.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7180b8200 msgr2=0x7fa7180c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7180b8200 0x7fa7180c9230 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa70800c520 tx=0x7fa70800c8e0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 shutdown_connections 2023-07-13T22:31:11.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa6fc06ace0 0x7fa6fc06d190 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7180cede0 0x7fa718146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7180b1ac0 0x7fa718155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 --2- 172.21.15.60:0/878219958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7180b8200 0x7fa7180c9230 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 >> 172.21.15.60:0/878219958 conn(0x7fa71806b2f0 msgr2=0x7fa7180b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:11.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 shutdown_connections 2023-07-13T22:31:11.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.629+0000 7fa71ffff700 1 -- 172.21.15.60:0/878219958 wait complete. 2023-07-13T22:31:11.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:31:11.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.854+0000 7f22723ff700 1 Processor -- start 2023-07-13T22:31:11.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.854+0000 7f22723ff700 1 -- start start 2023-07-13T22:31:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.856+0000 7f22723ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c15c530 0x7f226c0b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.856+0000 7f22723ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f226c0cfbc0 0x7f226c165eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.856+0000 7f22723ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c0cefd0 con 0x7f226c0d11c0 2023-07-13T22:31:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.856+0000 7f22723ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226c0d0020 con 0x7f226c0cfbc0 2023-07-13T22:31:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.856+0000 7f22723ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0d1780 con 0x7f226c15c530 2023-07-13T22:31:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f227099c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f226c0cfbc0 0x7f226c165eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f226bfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c15c530 0x7f226c0b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f227099c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f226c0cfbc0 0x7f226c165eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:56452/0 (socket says 172.21.15.60:56452) 2023-07-13T22:31:11.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f227099c700 1 -- 172.21.15.60:0/1734099514 learned_addr learned my addr 172.21.15.60:0/1734099514 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f226bfff700 1 -- 172.21.15.60:0/1734099514 >> v2:172.21.15.105:3300/0 conn(0x7f226c0cfbc0 msgr2=0x7f226c165eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:11.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f226bfff700 1 --2- 172.21.15.60:0/1734099514 >> v2:172.21.15.105:3300/0 conn(0x7f226c0cfbc0 0x7f226c165eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f226bfff700 1 -- 172.21.15.60:0/1734099514 >> v1:172.21.15.105:6789/0 conn(0x7f226c0d11c0 legacy=0x7f226c0cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f226bfff700 1 -- 172.21.15.60:0/1734099514 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0cd2d0 con 0x7f226c15c530 2023-07-13T22:31:11.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f226b7fe700 1 -- 172.21.15.60:0/1734099514 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290172281 0 0) 0x7f226c0cefd0 con 0x7f226c0d11c0 2023-07-13T22:31:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.857+0000 7f227099c700 1 --2- 172.21.15.60:0/1734099514 >> v2:172.21.15.105:3300/0 conn(0x7f226c0cfbc0 0x7f226c165eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f226bfff700 1 --2- 172.21.15.60:0/1734099514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c15c530 0x7f226c0b5a10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f225c009840 tx=0x7f225c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d8d44fb954645e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f226b7fe700 1 -- 172.21.15.60:0/1734099514 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c011830 con 0x7f226c15c530 2023-07-13T22:31:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f226b7fe700 1 -- 172.21.15.60:0/1734099514 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c011990 con 0x7f226c15c530 2023-07-13T22:31:11.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f226b7fe700 1 -- 172.21.15.60:0/1734099514 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c011d20 con 0x7f226c15c530 2023-07-13T22:31:11.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f22723ff700 1 -- 172.21.15.60:0/1734099514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c15c530 msgr2=0x7f226c0b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f22723ff700 1 --2- 172.21.15.60:0/1734099514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c15c530 0x7f226c0b5a10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f225c009840 tx=0x7f225c009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f22723ff700 1 -- 172.21.15.60:0/1734099514 shutdown_connections 2023-07-13T22:31:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f22723ff700 1 --2- 172.21.15.60:0/1734099514 >> v2:172.21.15.105:3300/0 conn(0x7f226c0cfbc0 0x7f226c165eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f22723ff700 1 --2- 172.21.15.60:0/1734099514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c15c530 0x7f226c0b5a10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.858+0000 7f22723ff700 1 -- 172.21.15.60:0/1734099514 >> 172.21.15.60:0/1734099514 conn(0x7f226c06b400 msgr2=0x7f226c06b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.859+0000 7f22723ff700 1 -- 172.21.15.60:0/1734099514 shutdown_connections 2023-07-13T22:31:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.859+0000 7f22723ff700 1 -- 172.21.15.60:0/1734099514 wait complete. 2023-07-13T22:31:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.860+0000 7f22723ff700 1 Processor -- start 2023-07-13T22:31:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.860+0000 7f22723ff700 1 -- start start 2023-07-13T22:31:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.861+0000 7f22723ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c0cfbc0 0x7f226c159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.861+0000 7f22723ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f226c0d11c0 0x7f226c0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.861+0000 7f22723ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c15a590 0x7f226c0c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.861+0000 7f22723ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c1516b0 con 0x7f226c15a590 2023-07-13T22:31:11.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.861+0000 7f22723ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c152c70 con 0x7f226c0d11c0 2023-07-13T22:31:11.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.861+0000 7f22723ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c15adc0 con 0x7f226c0cfbc0 2023-07-13T22:31:11.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.861+0000 7f227099c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c15a590 0x7f226c0c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f227099c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c15a590 0x7f226c0c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58292/0 (socket says 172.21.15.60:58292) 2023-07-13T22:31:11.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f227099c700 1 -- 172.21.15.60:0/3661423717 learned_addr learned my addr 172.21.15.60:0/3661423717 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:11.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f2263fff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f226c0d11c0 0x7f226c0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f226bfff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c0cfbc0 0x7f226c159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f227099c700 1 -- 172.21.15.60:0/3661423717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c0cfbc0 msgr2=0x7f226c159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f227099c700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c0cfbc0 0x7f226c159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f227099c700 1 -- 172.21.15.60:0/3661423717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f226c0d11c0 msgr2=0x7f226c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:11.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f227099c700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f226c0d11c0 0x7f226c0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:11.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f227099c700 1 -- 172.21.15.60:0/3661423717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009b30 con 0x7f226c15a590 2023-07-13T22:31:11.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f227099c700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c15a590 0x7f226c0c9ea0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f226400efc0 tx=0x7f226400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:11.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.862+0000 7f22697fa700 1 -- 172.21.15.60:0/3661423717 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264018070 con 0x7f226c15a590 2023-07-13T22:31:11.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.863+0000 7f22697fa700 1 -- 172.21.15.60:0/3661423717 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400f040 con 0x7f226c15a590 2023-07-13T22:31:11.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.863+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c0094a0 con 0x7f226c15a590 2023-07-13T22:31:11.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.863+0000 7f22697fa700 1 -- 172.21.15.60:0/3661423717 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264003c20 con 0x7f226c15a590 2023-07-13T22:31:11.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.863+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c0c9a70 con 0x7f226c15a590 2023-07-13T22:31:11.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.864+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c152de0 con 0x7f226c15a590 2023-07-13T22:31:11.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.865+0000 7f22697fa700 1 -- 172.21.15.60:0/3661423717 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f226400c880 con 0x7f226c15a590 2023-07-13T22:31:11.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.866+0000 7f22697fa700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f224c06ace0 0x7f224c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:11.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.866+0000 7f22697fa700 1 -- 172.21.15.60:0/3661423717 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2264089670 con 0x7f226c15a590 2023-07-13T22:31:11.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.866+0000 7f226bfff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f224c06ace0 0x7f224c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:11.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.867+0000 7f226bfff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f224c06ace0 0x7f224c06d190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f225c000c00 tx=0x7f225c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:11.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:11.871+0000 7f22697fa700 1 -- 172.21.15.60:0/3661423717 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2264054710 con 0x7f226c15a590 2023-07-13T22:31:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.073+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f226c0d0840 con 0x7f226c15a590 2023-07-13T22:31:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.074+0000 7f22697fa700 1 -- 172.21.15.60:0/3661423717 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2264057d30 con 0x7f226c15a590 2023-07-13T22:31:12.056 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:31:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f224c06ace0 msgr2=0x7f224c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f224c06ace0 0x7f224c06d190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f225c000c00 tx=0x7f225c008030 comp rx=0 tx=0).stop 2023-07-13T22:31:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c15a590 msgr2=0x7f226c0c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c15a590 0x7f226c0c9ea0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f226400efc0 tx=0x7f226400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 shutdown_connections 2023-07-13T22:31:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f224c06ace0 0x7f224c06d190 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c15a590 0x7f226c0c9ea0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f226c0cfbc0 0x7f226c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 --2- 172.21.15.60:0/3661423717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f226c0d11c0 0x7f226c0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 >> 172.21.15.60:0/3661423717 conn(0x7f226c06b400 msgr2=0x7f226c06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 shutdown_connections 2023-07-13T22:31:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.076+0000 7f22723ff700 1 -- 172.21.15.60:0/3661423717 wait complete. 2023-07-13T22:31:12.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:31:12.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.302+0000 7f8cd63ff700 1 Processor -- start 2023-07-13T22:31:12.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.303+0000 7f8cd63ff700 1 -- start start 2023-07-13T22:31:12.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd63ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8cd00cfd50 0x7f8cd00d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd63ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8cd00b1ac0 0x7f8cd00d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd63ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd00d1870 con 0x7f8cd00b8200 2023-07-13T22:31:12.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd63ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd00cf370 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd63ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd00d12b0 con 0x7f8cd00cfd50 2023-07-13T22:31:12.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd499c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8cd00b1ac0 0x7f8cd00d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd499c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8cd00b1ac0 0x7f8cd00d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58306/0 (socket says 172.21.15.60:58306) 2023-07-13T22:31:12.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd499c700 1 -- 172.21.15.60:0/1221606780 learned_addr learned my addr 172.21.15.60:0/1221606780 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:12.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8ccf7fe700 1 --2- 172.21.15.60:0/1221606780 >> v2:172.21.15.105:3300/0 conn(0x7f8cd00cfd50 0x7f8cd00d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd499c700 1 -- 172.21.15.60:0/1221606780 >> v2:172.21.15.105:3300/0 conn(0x7f8cd00cfd50 msgr2=0x7f8cd00d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:12.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd499c700 1 --2- 172.21.15.60:0/1221606780 >> v2:172.21.15.105:3300/0 conn(0x7f8cd00cfd50 0x7f8cd00d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd499c700 1 -- 172.21.15.60:0/1221606780 >> v1:172.21.15.105:6789/0 conn(0x7f8cd00b8200 legacy=0x7f8cd00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:12.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.304+0000 7f8cd499c700 1 -- 172.21.15.60:0/1221606780 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd00d1590 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.305+0000 7f8ccf7fe700 1 --2- 172.21.15.60:0/1221606780 >> v2:172.21.15.105:3300/0 conn(0x7f8cd00cfd50 0x7f8cd00d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:12.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.305+0000 7f8cd499c700 1 --2- 172.21.15.60:0/1221606780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 0x7f8cd00d2900 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00d07b0 tx=0x7f8cc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a627045bfeeeb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.305+0000 7f8cceffd700 1 -- 172.21.15.60:0/1221606780 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800e070 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.305+0000 7f8cceffd700 1 -- 172.21.15.60:0/1221606780 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc8003a20 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.305+0000 7f8cceffd700 1 -- 172.21.15.60:0/1221606780 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8003dd0 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.306+0000 7f8cd63ff700 1 -- 172.21.15.60:0/1221606780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 msgr2=0x7f8cd00d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.306+0000 7f8cd63ff700 1 --2- 172.21.15.60:0/1221606780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 0x7f8cd00d2900 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00d07b0 tx=0x7f8cc800b330 comp rx=0 tx=0).stop 2023-07-13T22:31:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.306+0000 7f8cd63ff700 1 -- 172.21.15.60:0/1221606780 shutdown_connections 2023-07-13T22:31:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.306+0000 7f8cd63ff700 1 --2- 172.21.15.60:0/1221606780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 0x7f8cd00d2900 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.306+0000 7f8cd63ff700 1 --2- 172.21.15.60:0/1221606780 >> v2:172.21.15.105:3300/0 conn(0x7f8cd00cfd50 0x7f8cd00d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.306+0000 7f8cd63ff700 1 -- 172.21.15.60:0/1221606780 >> 172.21.15.60:0/1221606780 conn(0x7f8cd006b2f0 msgr2=0x7f8cd006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:12.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.306+0000 7f8cd63ff700 1 -- 172.21.15.60:0/1221606780 shutdown_connections 2023-07-13T22:31:12.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.306+0000 7f8cd63ff700 1 -- 172.21.15.60:0/1221606780 wait complete. 2023-07-13T22:31:12.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.307+0000 7f8cd63ff700 1 Processor -- start 2023-07-13T22:31:12.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.308+0000 7f8cd63ff700 1 -- start start 2023-07-13T22:31:12.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.308+0000 7f8cd63ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 0x7f8cd00cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8cd63ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8cd00b8200 0x7f8cd01e3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8cd63ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8cd00cfd50 0x7f8cd015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8cd63ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd00cad40 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8cd63ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd00cb6a0 con 0x7f8cd00cfd50 2023-07-13T22:31:12.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8cd63ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd00ca6c0 con 0x7f8cd00b8200 2023-07-13T22:31:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 0x7f8cd00cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 0x7f8cd00cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58308/0 (socket says 172.21.15.60:58308) 2023-07-13T22:31:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccffff700 1 -- 172.21.15.60:0/40936549 learned_addr learned my addr 172.21.15.60:0/40936549 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccf7fe700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8cd00b8200 0x7f8cd01e3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8cd499c700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8cd00cfd50 0x7f8cd015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccffff700 1 -- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8cd00b8200 msgr2=0x7f8cd01e3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccffff700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8cd00b8200 0x7f8cd01e3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccffff700 1 -- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8cd00cfd50 msgr2=0x7f8cd015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccffff700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8cd00cfd50 0x7f8cd015a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccffff700 1 -- 172.21.15.60:0/40936549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8009b30 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8cd499c700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8cd00cfd50 0x7f8cd015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.309+0000 7f8ccf7fe700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8cd00b8200 0x7f8cd01e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.310+0000 7f8ccffff700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 0x7f8cd00cb160 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c4d0 tx=0x7f8cc000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.310+0000 7f8cccff9700 1 -- 172.21.15.60:0/40936549 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0011790 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.310+0000 7f8cccff9700 1 -- 172.21.15.60:0/40936549 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc00118f0 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.310+0000 7f8cccff9700 1 -- 172.21.15.60:0/40936549 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0011ca0 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.310+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd00cc810 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.310+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd00ca960 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.311+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd004e210 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.312+0000 7f8cccff9700 1 -- 172.21.15.60:0/40936549 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8cc001c460 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.314+0000 7f8cccff9700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8cbc06acb0 0x7f8cbc06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.314+0000 7f8cccff9700 1 -- 172.21.15.60:0/40936549 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8cc0089060 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.314+0000 7f8ccf7fe700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8cbc06acb0 0x7f8cbc06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.315+0000 7f8ccf7fe700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8cbc06acb0 0x7f8cbc06d160 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8cd01e35a0 tx=0x7f8cc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:12.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.318+0000 7f8cccff9700 1 -- 172.21.15.60:0/40936549 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8cc00bb330 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.524+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8cd00b4f70 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.525+0000 7f8cccff9700 1 -- 172.21.15.60:0/40936549 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8cc0054100 con 0x7f8cd00b1ac0 2023-07-13T22:31:12.507 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:31:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8cbc06acb0 msgr2=0x7f8cbc06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8cbc06acb0 0x7f8cbc06d160 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8cd01e35a0 tx=0x7f8cc4008040 comp rx=0 tx=0).stop 2023-07-13T22:31:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 msgr2=0x7f8cd00cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 0x7f8cd00cb160 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c4d0 tx=0x7f8cc000c890 comp rx=0 tx=0).stop 2023-07-13T22:31:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 shutdown_connections 2023-07-13T22:31:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8cbc06acb0 0x7f8cbc06d160 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cd00b1ac0 0x7f8cd00cb160 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8cd00cfd50 0x7f8cd015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 --2- 172.21.15.60:0/40936549 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8cd00b8200 0x7f8cd01e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 >> 172.21.15.60:0/40936549 conn(0x7f8cd006b2f0 msgr2=0x7f8cd00d2310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 shutdown_connections 2023-07-13T22:31:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.527+0000 7f8cd63ff700 1 -- 172.21.15.60:0/40936549 wait complete. 2023-07-13T22:31:12.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:31:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.752+0000 7f4811dff700 1 Processor -- start 2023-07-13T22:31:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.752+0000 7f4811dff700 1 -- start start 2023-07-13T22:31:12.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.753+0000 7f4811dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f480c159260 0x7f480c0ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.753+0000 7f4811dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 0x7f480c0cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.754+0000 7f4811dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f480c0cf640 con 0x7f480c0703e0 2023-07-13T22:31:12.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.754+0000 7f4811dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480c0ccf80 con 0x7f480c159260 2023-07-13T22:31:12.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.754+0000 7f4811dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c0c9b40 con 0x7f480c0cc410 2023-07-13T22:31:12.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.754+0000 7f480b7fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f480c159260 0x7f480c0ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.754+0000 7f480affd700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 0x7f480c0cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.754+0000 7f480bfff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f480c0703e0 0x7f480c06c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:33260/0 (socket says 172.21.15.60:33260) 2023-07-13T22:31:12.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.754+0000 7f480bfff700 1 -- 172.21.15.60:0/13687720 learned_addr learned my addr 172.21.15.60:0/13687720 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:12.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.754+0000 7f480b7fe700 1 -- 172.21.15.60:0/13687720 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 msgr2=0x7f480c0cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:12.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.754+0000 7f480b7fe700 1 --2- 172.21.15.60:0/13687720 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 0x7f480c0cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.755+0000 7f480b7fe700 1 -- 172.21.15.60:0/13687720 >> v1:172.21.15.105:6789/0 conn(0x7f480c0703e0 legacy=0x7f480c06c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.755+0000 7f480b7fe700 1 -- 172.21.15.60:0/13687720 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c0c9e20 con 0x7f480c159260 2023-07-13T22:31:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.755+0000 7f480affd700 1 --2- 172.21.15.60:0/13687720 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 0x7f480c0cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.755+0000 7f480b7fe700 1 --2- 172.21.15.60:0/13687720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c159260 0x7f480c0ca900 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009840 tx=0x7f47fc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8409ec253eb784ad server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.755+0000 7f480a7fc700 1 -- 172.21.15.60:0/13687720 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc011830 con 0x7f480c159260 2023-07-13T22:31:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.755+0000 7f480a7fc700 1 -- 172.21.15.60:0/13687720 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc011990 con 0x7f480c159260 2023-07-13T22:31:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.755+0000 7f480a7fc700 1 -- 172.21.15.60:0/13687720 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc011d20 con 0x7f480c159260 2023-07-13T22:31:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.756+0000 7f4811dff700 1 -- 172.21.15.60:0/13687720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c159260 msgr2=0x7f480c0ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.756+0000 7f4811dff700 1 --2- 172.21.15.60:0/13687720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c159260 0x7f480c0ca900 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009840 tx=0x7f47fc009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.756+0000 7f4811dff700 1 -- 172.21.15.60:0/13687720 shutdown_connections 2023-07-13T22:31:12.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.756+0000 7f4811dff700 1 --2- 172.21.15.60:0/13687720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c159260 0x7f480c0ca900 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.756+0000 7f4811dff700 1 --2- 172.21.15.60:0/13687720 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 0x7f480c0cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.756+0000 7f4811dff700 1 -- 172.21.15.60:0/13687720 >> 172.21.15.60:0/13687720 conn(0x7f480c0cbf50 msgr2=0x7f480c0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:12.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.756+0000 7f4811dff700 1 -- 172.21.15.60:0/13687720 shutdown_connections 2023-07-13T22:31:12.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.756+0000 7f4811dff700 1 -- 172.21.15.60:0/13687720 wait complete. 2023-07-13T22:31:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.757+0000 7f4811dff700 1 Processor -- start 2023-07-13T22:31:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.758+0000 7f4811dff700 1 -- start start 2023-07-13T22:31:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f4811dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c0703e0 0x7f480c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f4811dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 0x7f480c145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f4811dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c159260 0x7f480c1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f4811dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c15b870 con 0x7f480c159260 2023-07-13T22:31:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f4811dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c158ad0 con 0x7f480c0703e0 2023-07-13T22:31:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f4811dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c15af40 con 0x7f480c0cc410 2023-07-13T22:31:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f480bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c159260 0x7f480c1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f480bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c159260 0x7f480c1581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58322/0 (socket says 172.21.15.60:58322) 2023-07-13T22:31:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f480bfff700 1 -- 172.21.15.60:0/2797502883 learned_addr learned my addr 172.21.15.60:0/2797502883 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f480b7fe700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c0703e0 0x7f480c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.759+0000 7f480affd700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 0x7f480c145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f480bfff700 1 -- 172.21.15.60:0/2797502883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 msgr2=0x7f480c145d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f480bfff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 0x7f480c145d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f480bfff700 1 -- 172.21.15.60:0/2797502883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c0703e0 msgr2=0x7f480c155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f480bfff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c0703e0 0x7f480c155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f480bfff700 1 -- 172.21.15.60:0/2797502883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48000094a0 con 0x7f480c159260 2023-07-13T22:31:12.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f480bfff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c159260 0x7f480c1581f0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f480400cbf0 tx=0x7f480400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:12.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f47f3fff700 1 -- 172.21.15.60:0/2797502883 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480400aa60 con 0x7f480c159260 2023-07-13T22:31:12.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f47f3fff700 1 -- 172.21.15.60:0/2797502883 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480400abc0 con 0x7f480c159260 2023-07-13T22:31:12.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f47f3fff700 1 -- 172.21.15.60:0/2797502883 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804012400 con 0x7f480c159260 2023-07-13T22:31:12.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc0094a0 con 0x7f480c159260 2023-07-13T22:31:12.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.760+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480c15b2d0 con 0x7f480c159260 2023-07-13T22:31:12.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.761+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480c15b9e0 con 0x7f480c159260 2023-07-13T22:31:12.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.762+0000 7f47f3fff700 1 -- 172.21.15.60:0/2797502883 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f480401b3b0 con 0x7f480c159260 2023-07-13T22:31:12.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.764+0000 7f47f3fff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47f806ace0 0x7f47f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:12.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.764+0000 7f47f3fff700 1 -- 172.21.15.60:0/2797502883 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4804088fe0 con 0x7f480c159260 2023-07-13T22:31:12.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.764+0000 7f480b7fe700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47f806ace0 0x7f47f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:12.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.765+0000 7f480b7fe700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47f806ace0 0x7f47f806d190 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f47fc000c00 tx=0x7f47fc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:12.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.768+0000 7f47f3fff700 1 -- 172.21.15.60:0/2797502883 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4804054070 con 0x7f480c159260 2023-07-13T22:31:12.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.972+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f480c04e210 con 0x7f480c159260 2023-07-13T22:31:12.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.973+0000 7f47f3fff700 1 -- 172.21.15.60:0/2797502883 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48040170a0 con 0x7f480c159260 2023-07-13T22:31:12.955 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:31:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47f806ace0 msgr2=0x7f47f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47f806ace0 0x7f47f806d190 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f47fc000c00 tx=0x7f47fc01c040 comp rx=0 tx=0).stop 2023-07-13T22:31:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c159260 msgr2=0x7f480c1581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c159260 0x7f480c1581f0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f480400cbf0 tx=0x7f480400cf00 comp rx=0 tx=0).stop 2023-07-13T22:31:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 shutdown_connections 2023-07-13T22:31:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f47f806ace0 0x7f47f806d190 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c159260 0x7f480c1581f0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c0703e0 0x7f480c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 --2- 172.21.15.60:0/2797502883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f480c0cc410 0x7f480c145d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 >> 172.21.15.60:0/2797502883 conn(0x7f480c0cbf50 msgr2=0x7f480c0b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 shutdown_connections 2023-07-13T22:31:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:12.975+0000 7f4811dff700 1 -- 172.21.15.60:0/2797502883 wait complete. 2023-07-13T22:31:12.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:13.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.197+0000 7f82871ff700 1 Processor -- start 2023-07-13T22:31:13.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.197+0000 7f82871ff700 1 -- start start 2023-07-13T22:31:13.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.198+0000 7f82871ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82800b8570 0x7f82800ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.198+0000 7f82871ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f82800d35c0 0x7f82800d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.198+0000 7f82871ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82800d09d0 con 0x7f82800cf3f0 2023-07-13T22:31:13.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.198+0000 7f82871ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82800d3b30 con 0x7f82800b8570 2023-07-13T22:31:13.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.198+0000 7f82871ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82800cda10 con 0x7f82800d35c0 2023-07-13T22:31:13.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.199+0000 7f8284f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82800b8570 0x7f82800ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:13.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.199+0000 7f8284f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82800b8570 0x7f82800ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58324/0 (socket says 172.21.15.60:58324) 2023-07-13T22:31:13.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.199+0000 7f8284f9b700 1 -- 172.21.15.60:0/3962532455 learned_addr learned my addr 172.21.15.60:0/3962532455 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:13.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.199+0000 7f827ffff700 1 --2- 172.21.15.60:0/3962532455 >> v2:172.21.15.105:3300/0 conn(0x7f82800d35c0 0x7f82800d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:13.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.199+0000 7f8284f9b700 1 -- 172.21.15.60:0/3962532455 >> v2:172.21.15.105:3300/0 conn(0x7f82800d35c0 msgr2=0x7f82800d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:13.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.199+0000 7f8284f9b700 1 --2- 172.21.15.60:0/3962532455 >> v2:172.21.15.105:3300/0 conn(0x7f82800d35c0 0x7f82800d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.199+0000 7f8284f9b700 1 -- 172.21.15.60:0/3962532455 >> v1:172.21.15.105:6789/0 conn(0x7f82800cf3f0 legacy=0x7f828015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:13.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.199+0000 7f8284f9b700 1 -- 172.21.15.60:0/3962532455 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82800cd450 con 0x7f82800b8570 2023-07-13T22:31:13.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.199+0000 7f8284f9b700 1 --2- 172.21.15.60:0/3962532455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800b8570 0x7f82800ceeb0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f82800d07e0 tx=0x7f8274009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110733e39ba2bd61 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:13.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f827f7fe700 1 -- 172.21.15.60:0/3962532455 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274005e30 con 0x7f82800b8570 2023-07-13T22:31:13.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f827f7fe700 1 -- 172.21.15.60:0/3962532455 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8274006030 con 0x7f82800b8570 2023-07-13T22:31:13.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f827f7fe700 1 -- 172.21.15.60:0/3962532455 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740063e0 con 0x7f82800b8570 2023-07-13T22:31:13.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f82871ff700 1 -- 172.21.15.60:0/3962532455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800b8570 msgr2=0x7f82800ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f82871ff700 1 --2- 172.21.15.60:0/3962532455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800b8570 0x7f82800ceeb0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f82800d07e0 tx=0x7f8274009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f82871ff700 1 -- 172.21.15.60:0/3962532455 shutdown_connections 2023-07-13T22:31:13.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f82871ff700 1 --2- 172.21.15.60:0/3962532455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800b8570 0x7f82800ceeb0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f82871ff700 1 --2- 172.21.15.60:0/3962532455 >> v2:172.21.15.105:3300/0 conn(0x7f82800d35c0 0x7f82800d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f82871ff700 1 -- 172.21.15.60:0/3962532455 >> 172.21.15.60:0/3962532455 conn(0x7f8280071600 msgr2=0x7f82800719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:13.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f82871ff700 1 -- 172.21.15.60:0/3962532455 shutdown_connections 2023-07-13T22:31:13.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.200+0000 7f82871ff700 1 -- 172.21.15.60:0/3962532455 wait complete. 2023-07-13T22:31:13.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.202+0000 7f82871ff700 1 Processor -- start 2023-07-13T22:31:13.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.202+0000 7f82871ff700 1 -- start start 2023-07-13T22:31:13.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.203+0000 7f82871ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82800b8570 0x7f828015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.203+0000 7f82871ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800cf3f0 0x7f828015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.203+0000 7f82871ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82800d35c0 0x7f82800b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.203+0000 7f82871ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828015d1c0 con 0x7f82800cf3f0 2023-07-13T22:31:13.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.203+0000 7f82871ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828015c330 con 0x7f82800d35c0 2023-07-13T22:31:13.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.203+0000 7f82871ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828015d5e0 con 0x7f82800b8570 2023-07-13T22:31:13.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.203+0000 7f827ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800cf3f0 0x7f828015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:13.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.203+0000 7f827ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800cf3f0 0x7f828015bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58326/0 (socket says 172.21.15.60:58326) 2023-07-13T22:31:13.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.203+0000 7f827ffff700 1 -- 172.21.15.60:0/2842642207 learned_addr learned my addr 172.21.15.60:0/2842642207 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:13.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f827ffff700 1 -- 172.21.15.60:0/2842642207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82800b8570 msgr2=0x7f828015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f827ffff700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82800b8570 0x7f828015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f827ffff700 1 -- 172.21.15.60:0/2842642207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82800d35c0 msgr2=0x7f82800b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f827ffff700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82800d35c0 0x7f82800b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f827ffff700 1 -- 172.21.15.60:0/2842642207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274009500 con 0x7f82800cf3f0 2023-07-13T22:31:13.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f827ffff700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800cf3f0 0x7f828015bdf0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f826c00d800 tx=0x7f826c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:13.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f827d7fa700 1 -- 172.21.15.60:0/2842642207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c0125f0 con 0x7f82800cf3f0 2023-07-13T22:31:13.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f827d7fa700 1 -- 172.21.15.60:0/2842642207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c012750 con 0x7f82800cf3f0 2023-07-13T22:31:13.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f827d7fa700 1 -- 172.21.15.60:0/2842642207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c012b00 con 0x7f82800cf3f0 2023-07-13T22:31:13.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.204+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828015c540 con 0x7f82800cf3f0 2023-07-13T22:31:13.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.205+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828015cb00 con 0x7f82800cf3f0 2023-07-13T22:31:13.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.205+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82800cef40 con 0x7f82800cf3f0 2023-07-13T22:31:13.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.206+0000 7f827d7fa700 1 -- 172.21.15.60:0/2842642207 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f826c012c60 con 0x7f82800cf3f0 2023-07-13T22:31:13.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.208+0000 7f827d7fa700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f826406ace0 0x7f826406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.208+0000 7f827d7fa700 1 -- 172.21.15.60:0/2842642207 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f826c089880 con 0x7f82800cf3f0 2023-07-13T22:31:13.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.208+0000 7f8284f9b700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f826406ace0 0x7f826406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:13.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.208+0000 7f8284f9b700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f826406ace0 0x7f826406d190 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f82800d07e0 tx=0x7f8274010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:13.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.212+0000 7f827d7fa700 1 -- 172.21.15.60:0/2842642207 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f826c054920 con 0x7f82800cf3f0 2023-07-13T22:31:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.460+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f828015d300 con 0x7f82800cf3f0 2023-07-13T22:31:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.461+0000 7f827d7fa700 1 -- 172.21.15.60:0/2842642207 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f828015d300 con 0x7f82800cf3f0 2023-07-13T22:31:13.442 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:13.443 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":88,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:30:39.690428+0000","modified":"2023-07-13T22:31:12.601793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6836/1781435251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1781435251},{"type":"v1","addr":"172.21.15.60:6836","nonce":1781435251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-13T22:31:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f826406ace0 msgr2=0x7f826406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f826406ace0 0x7f826406d190 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f82800d07e0 tx=0x7f8274010000 comp rx=0 tx=0).stop 2023-07-13T22:31:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800cf3f0 msgr2=0x7f828015bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800cf3f0 0x7f828015bdf0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f826c00d800 tx=0x7f826c00dbc0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 shutdown_connections 2023-07-13T22:31:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f826406ace0 0x7f826406d190 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82800d35c0 0x7f82800b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82800b8570 0x7f828015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 --2- 172.21.15.60:0/2842642207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82800cf3f0 0x7f828015bdf0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 >> 172.21.15.60:0/2842642207 conn(0x7f8280071600 msgr2=0x7f82800cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 shutdown_connections 2023-07-13T22:31:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.463+0000 7f82871ff700 1 -- 172.21.15.60:0/2842642207 wait complete. 2023-07-13T22:31:13.447 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 88 2023-07-13T22:31:13.463 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:13.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.685+0000 7fa37bbff700 1 Processor -- start 2023-07-13T22:31:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.686+0000 7fa37bbff700 1 -- start start 2023-07-13T22:31:13.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.687+0000 7fa37bbff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3740c3640 0x7fa374069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.687+0000 7fa37bbff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa37406a410 0x7fa374160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.687+0000 7fa37bbff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37406a830 con 0x7fa374160870 2023-07-13T22:31:13.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.687+0000 7fa37bbff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3740cd070 con 0x7fa3740c3640 2023-07-13T22:31:13.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.687+0000 7fa37bbff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3741643b0 con 0x7fa37406a410 2023-07-13T22:31:13.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.687+0000 7fa37919a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3740c3640 0x7fa374069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:13.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.687+0000 7fa37919a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3740c3640 0x7fa374069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58328/0 (socket says 172.21.15.60:58328) 2023-07-13T22:31:13.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.687+0000 7fa37919a700 1 -- 172.21.15.60:0/3207195922 learned_addr learned my addr 172.21.15.60:0/3207195922 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:13.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.687+0000 7fa37a19c700 1 --2- 172.21.15.60:0/3207195922 >> v2:172.21.15.105:3300/0 conn(0x7fa37406a410 0x7fa374160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:13.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.688+0000 7fa37919a700 1 -- 172.21.15.60:0/3207195922 >> v2:172.21.15.105:3300/0 conn(0x7fa37406a410 msgr2=0x7fa374160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:13.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.688+0000 7fa37919a700 1 --2- 172.21.15.60:0/3207195922 >> v2:172.21.15.105:3300/0 conn(0x7fa37406a410 0x7fa374160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.688+0000 7fa37919a700 1 -- 172.21.15.60:0/3207195922 >> v1:172.21.15.105:6789/0 conn(0x7fa374160870 legacy=0x7fa3740adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.688+0000 7fa37919a700 1 -- 172.21.15.60:0/3207195922 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374164690 con 0x7fa3740c3640 2023-07-13T22:31:13.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.688+0000 7fa37919a700 1 --2- 172.21.15.60:0/3207195922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3740c3640 0x7fa374069ed0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fa374070170 tx=0x7fa364009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddfccf1e892dd50 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:13.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.688+0000 7fa378999700 1 -- 172.21.15.60:0/3207195922 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36400b870 con 0x7fa3740c3640 2023-07-13T22:31:13.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.688+0000 7fa378999700 1 -- 172.21.15.60:0/3207195922 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36400b9d0 con 0x7fa3740c3640 2023-07-13T22:31:13.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.688+0000 7fa378999700 1 -- 172.21.15.60:0/3207195922 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36400bd60 con 0x7fa3740c3640 2023-07-13T22:31:13.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.689+0000 7fa37bbff700 1 -- 172.21.15.60:0/3207195922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3740c3640 msgr2=0x7fa374069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.689+0000 7fa37bbff700 1 --2- 172.21.15.60:0/3207195922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3740c3640 0x7fa374069ed0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fa374070170 tx=0x7fa364009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.689+0000 7fa37bbff700 1 -- 172.21.15.60:0/3207195922 shutdown_connections 2023-07-13T22:31:13.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.689+0000 7fa37bbff700 1 --2- 172.21.15.60:0/3207195922 >> v2:172.21.15.105:3300/0 conn(0x7fa37406a410 0x7fa374160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.689+0000 7fa37bbff700 1 --2- 172.21.15.60:0/3207195922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3740c3640 0x7fa374069ed0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.689+0000 7fa37bbff700 1 -- 172.21.15.60:0/3207195922 >> 172.21.15.60:0/3207195922 conn(0x7fa3740cf410 msgr2=0x7fa3740cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:13.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.689+0000 7fa37bbff700 1 -- 172.21.15.60:0/3207195922 shutdown_connections 2023-07-13T22:31:13.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.689+0000 7fa37bbff700 1 -- 172.21.15.60:0/3207195922 wait complete. 2023-07-13T22:31:13.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.691+0000 7fa37bbff700 1 Processor -- start 2023-07-13T22:31:13.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.691+0000 7fa37bbff700 1 -- start start 2023-07-13T22:31:13.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37bbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa37406a410 0x7fa3740b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37bbff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740c3640 0x7fa374070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37bbff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa374160870 0x7fa3740707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37bbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa374154ec0 con 0x7fa37406a410 2023-07-13T22:31:13.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37bbff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa374143380 con 0x7fa3740c3640 2023-07-13T22:31:13.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37bbff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa374142ae0 con 0x7fa374160870 2023-07-13T22:31:13.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37999b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa37406a410 0x7fa3740b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:13.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37999b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa37406a410 0x7fa3740b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58334/0 (socket says 172.21.15.60:58334) 2023-07-13T22:31:13.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37999b700 1 -- 172.21.15.60:0/3690645410 learned_addr learned my addr 172.21.15.60:0/3690645410 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:13.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37999b700 1 -- 172.21.15.60:0/3690645410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa374160870 msgr2=0x7fa3740707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37999b700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa374160870 0x7fa3740707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37999b700 1 -- 172.21.15.60:0/3690645410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740c3640 msgr2=0x7fa374070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37919a700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740c3640 0x7fa374070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.692+0000 7fa37999b700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740c3640 0x7fa374070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.693+0000 7fa37999b700 1 -- 172.21.15.60:0/3690645410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa364009550 con 0x7fa37406a410 2023-07-13T22:31:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.693+0000 7fa37919a700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740c3640 0x7fa374070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.693+0000 7fa37999b700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa37406a410 0x7fa3740b3f20 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c450 tx=0x7fa37000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.693+0000 7fa36a7fc700 1 -- 172.21.15.60:0/3690645410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700052d0 con 0x7fa37406a410 2023-07-13T22:31:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.693+0000 7fa36a7fc700 1 -- 172.21.15.60:0/3690645410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa370005430 con 0x7fa37406a410 2023-07-13T22:31:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.693+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa374142dc0 con 0x7fa37406a410 2023-07-13T22:31:13.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.693+0000 7fa36a7fc700 1 -- 172.21.15.60:0/3690645410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700038c0 con 0x7fa37406a410 2023-07-13T22:31:13.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.693+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3741ea350 con 0x7fa37406a410 2023-07-13T22:31:13.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.694+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37404e280 con 0x7fa37406a410 2023-07-13T22:31:13.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.695+0000 7fa36a7fc700 1 -- 172.21.15.60:0/3690645410 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa370003a20 con 0x7fa37406a410 2023-07-13T22:31:13.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.696+0000 7fa36a7fc700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa35806ace0 0x7fa35806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:13.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.696+0000 7fa36a7fc700 1 -- 172.21.15.60:0/3690645410 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3700889c0 con 0x7fa37406a410 2023-07-13T22:31:13.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.696+0000 7fa37919a700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa35806ace0 0x7fa35806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:13.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.697+0000 7fa37919a700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa35806ace0 0x7fa35806d190 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa3740c3a60 tx=0x7fa364008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:13.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.701+0000 7fa36a7fc700 1 -- 172.21.15.60:0/3690645410 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa3700539d0 con 0x7fa37406a410 2023-07-13T22:31:13.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.907+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3740cbe60 con 0x7fa37406a410 2023-07-13T22:31:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.908+0000 7fa36a7fc700 1 -- 172.21.15.60:0/3690645410 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18215 (secure 0 0 0) 0x7fa370056ff0 con 0x7fa37406a410 2023-07-13T22:31:13.890 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:13.890 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":124,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:30:41.182757+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:30:37.983326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-13T22:30:38.665534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:13.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.910+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa35806ace0 msgr2=0x7fa35806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.910+0000 7fa37bbff700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa35806ace0 0x7fa35806d190 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa3740c3a60 tx=0x7fa364008030 comp rx=0 tx=0).stop 2023-07-13T22:31:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa37406a410 msgr2=0x7fa3740b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa37406a410 0x7fa3740b3f20 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c450 tx=0x7fa37000c810 comp rx=0 tx=0).stop 2023-07-13T22:31:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 shutdown_connections 2023-07-13T22:31:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa35806ace0 0x7fa35806d190 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa37406a410 0x7fa3740b3f20 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa374160870 0x7fa3740707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 --2- 172.21.15.60:0/3690645410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3740c3640 0x7fa374070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:13.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 >> 172.21.15.60:0/3690645410 conn(0x7fa3740cf410 msgr2=0x7fa374160c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:13.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 shutdown_connections 2023-07-13T22:31:13.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:13.911+0000 7fa37bbff700 1 -- 172.21.15.60:0/3690645410 wait complete. 2023-07-13T22:31:13.911 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:31:13.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:14.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.132+0000 7f21c0bff700 1 Processor -- start 2023-07-13T22:31:14.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.133+0000 7f21c0bff700 1 -- start start 2023-07-13T22:31:14.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21c0bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 0x7f21bc06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21c0bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f21bc0b3050 0x7f21bc06afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21c0bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21bc0b24b0 0x7f21bc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21c0bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21bc15cab0 con 0x7f21bc0b24b0 2023-07-13T22:31:14.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21c0bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21bc0b2e70 con 0x7f21bc0b3050 2023-07-13T22:31:14.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21c0bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc06b8e0 con 0x7f21bc15c6e0 2023-07-13T22:31:14.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21bad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21bc0b24b0 0x7f21bc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21bad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21bc0b24b0 0x7f21bc0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58350/0 (socket says 172.21.15.60:58350) 2023-07-13T22:31:14.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21bad9d700 1 -- 172.21.15.60:0/16003563 learned_addr learned my addr 172.21.15.60:0/16003563 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:14.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21ba59c700 1 --2- 172.21.15.60:0/16003563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 0x7f21bc06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.134+0000 7f21b3fff700 1 --2- 172.21.15.60:0/16003563 >> v2:172.21.15.105:3300/0 conn(0x7f21bc0b3050 0x7f21bc06afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21bad9d700 1 -- 172.21.15.60:0/16003563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 msgr2=0x7f21bc06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:14.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21bad9d700 1 --2- 172.21.15.60:0/16003563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 0x7f21bc06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21bad9d700 1 -- 172.21.15.60:0/16003563 >> v2:172.21.15.105:3300/0 conn(0x7f21bc0b3050 msgr2=0x7f21bc06afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:14.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21bad9d700 1 --2- 172.21.15.60:0/16003563 >> v2:172.21.15.105:3300/0 conn(0x7f21bc0b3050 0x7f21bc06afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21bad9d700 1 -- 172.21.15.60:0/16003563 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc06bbc0 con 0x7f21bc0b24b0 2023-07-13T22:31:14.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21ba59c700 1 --2- 172.21.15.60:0/16003563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 0x7f21bc06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:14.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21b3fff700 1 --2- 172.21.15.60:0/16003563 >> v2:172.21.15.105:3300/0 conn(0x7f21bc0b3050 0x7f21bc06afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21bad9d700 1 --2- 172.21.15.60:0/16003563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 0x7f21bc0b2930 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0b35f0 tx=0x7f21b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a861f1a0cb11170 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21b9d9b700 1 -- 172.21.15.60:0/16003563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400e070 con 0x7f21bc0b24b0 2023-07-13T22:31:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21b9d9b700 1 -- 172.21.15.60:0/16003563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b400b9c0 con 0x7f21bc0b24b0 2023-07-13T22:31:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.135+0000 7f21b9d9b700 1 -- 172.21.15.60:0/16003563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400bd70 con 0x7f21bc0b24b0 2023-07-13T22:31:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.136+0000 7f21c0bff700 1 -- 172.21.15.60:0/16003563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 msgr2=0x7f21bc0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.136+0000 7f21c0bff700 1 --2- 172.21.15.60:0/16003563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 0x7f21bc0b2930 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0b35f0 tx=0x7f21b400b330 comp rx=0 tx=0).stop 2023-07-13T22:31:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.136+0000 7f21c0bff700 1 -- 172.21.15.60:0/16003563 shutdown_connections 2023-07-13T22:31:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.136+0000 7f21c0bff700 1 --2- 172.21.15.60:0/16003563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 0x7f21bc0b2930 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.136+0000 7f21c0bff700 1 --2- 172.21.15.60:0/16003563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 0x7f21bc06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.136+0000 7f21c0bff700 1 --2- 172.21.15.60:0/16003563 >> v2:172.21.15.105:3300/0 conn(0x7f21bc0b3050 0x7f21bc06afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.136+0000 7f21c0bff700 1 -- 172.21.15.60:0/16003563 >> 172.21.15.60:0/16003563 conn(0x7f21bc06e680 msgr2=0x7f21bc06ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.136+0000 7f21c0bff700 1 -- 172.21.15.60:0/16003563 shutdown_connections 2023-07-13T22:31:14.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.136+0000 7f21c0bff700 1 -- 172.21.15.60:0/16003563 wait complete. 2023-07-13T22:31:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.138+0000 7f21c0bff700 1 Processor -- start 2023-07-13T22:31:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.138+0000 7f21c0bff700 1 -- start start 2023-07-13T22:31:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21c0bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 0x7f21bc0b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21c0bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21bc0b3050 0x7f21bc156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21c0bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 0x7f21bc0c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21c0bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0c7c80 con 0x7f21bc0b24b0 2023-07-13T22:31:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21c0bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0c7f40 con 0x7f21bc0b3050 2023-07-13T22:31:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21c0bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0b3c10 con 0x7f21bc15c6e0 2023-07-13T22:31:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21ba59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 0x7f21bc0b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21ba59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 0x7f21bc0b6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58356/0 (socket says 172.21.15.60:58356) 2023-07-13T22:31:14.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21ba59c700 1 -- 172.21.15.60:0/1714570290 learned_addr learned my addr 172.21.15.60:0/1714570290 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:14.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21ba59c700 1 -- 172.21.15.60:0/1714570290 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 msgr2=0x7f21bc0c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21ba59c700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 0x7f21bc0c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21ba59c700 1 -- 172.21.15.60:0/1714570290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21bc0b3050 msgr2=0x7f21bc156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21ba59c700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21bc0b3050 0x7f21bc156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21ba59c700 1 -- 172.21.15.60:0/1714570290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b4009b30 con 0x7f21bc0b24b0 2023-07-13T22:31:14.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.139+0000 7f21ba59c700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 0x7f21bc0b6290 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c7f0 tx=0x7f21ac00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:14.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.140+0000 7f21b2ffd700 1 -- 172.21.15.60:0/1714570290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac011800 con 0x7f21bc0b24b0 2023-07-13T22:31:14.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.140+0000 7f21b2ffd700 1 -- 172.21.15.60:0/1714570290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ac011960 con 0x7f21bc0b24b0 2023-07-13T22:31:14.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.140+0000 7f21b2ffd700 1 -- 172.21.15.60:0/1714570290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac011d10 con 0x7f21bc0b24b0 2023-07-13T22:31:14.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.140+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc0c8220 con 0x7f21bc0b24b0 2023-07-13T22:31:14.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.140+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc0c5af0 con 0x7f21bc0b24b0 2023-07-13T22:31:14.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.141+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc0b3e30 con 0x7f21bc0b24b0 2023-07-13T22:31:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.142+0000 7f21b2ffd700 1 -- 172.21.15.60:0/1714570290 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21ac018360 con 0x7f21bc0b24b0 2023-07-13T22:31:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.143+0000 7f21b2ffd700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f219c06ace0 0x7f219c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.143+0000 7f21b2ffd700 1 -- 172.21.15.60:0/1714570290 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f21ac089470 con 0x7f21bc0b24b0 2023-07-13T22:31:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.143+0000 7f21b3fff700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f219c06ace0 0x7f219c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.143+0000 7f21b3fff700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f219c06ace0 0x7f219c06d190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006fd0 tx=0x7f21a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.147+0000 7f21b2ffd700 1 -- 172.21.15.60:0/1714570290 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f21ac057fa0 con 0x7f21bc0b24b0 2023-07-13T22:31:14.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.396+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21bc04e280 con 0x7f21bc0b24b0 2023-07-13T22:31:14.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.397+0000 7f21b2ffd700 1 -- 172.21.15.60:0/1714570290 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f21ac057b30 con 0x7f21bc0b24b0 2023-07-13T22:31:14.379 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:14.380 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":88,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:30:39.690428+0000","modified":"2023-07-13T22:31:12.601793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6836/1781435251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1781435251},{"type":"v1","addr":"172.21.15.60:6836","nonce":1781435251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-13T22:31:14.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f219c06ace0 msgr2=0x7f219c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f219c06ace0 0x7f219c06d190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006fd0 tx=0x7f21a4008040 comp rx=0 tx=0).stop 2023-07-13T22:31:14.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 msgr2=0x7f21bc0b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 0x7f21bc0b6290 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c7f0 tx=0x7f21ac00cbb0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 shutdown_connections 2023-07-13T22:31:14.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f219c06ace0 0x7f219c06d190 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21bc0b24b0 0x7f21bc0b6290 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21bc15c6e0 0x7f21bc0c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 --2- 172.21.15.60:0/1714570290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21bc0b3050 0x7f21bc156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 >> 172.21.15.60:0/1714570290 conn(0x7f21bc06e680 msgr2=0x7f21bc06ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:14.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 shutdown_connections 2023-07-13T22:31:14.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.399+0000 7f21c0bff700 1 -- 172.21.15.60:0/1714570290 wait complete. 2023-07-13T22:31:14.386 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 88 2023-07-13T22:31:14.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:14.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.622+0000 7f198bdff700 1 Processor -- start 2023-07-13T22:31:14.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.622+0000 7f198bdff700 1 -- start start 2023-07-13T22:31:14.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.623+0000 7f198bdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19840b8570 0x7f19840ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.623+0000 7f198bdff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19840d09d0 con 0x7f19840d15c0 2023-07-13T22:31:14.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.623+0000 7f198bdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19840cda10 con 0x7f19840d35c0 2023-07-13T22:31:14.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.623+0000 7f198bdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19840d3b30 con 0x7f19840b8570 2023-07-13T22:31:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.623+0000 7f198a39c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f19840d15c0 0x7f198415c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56788/0 (socket says 172.21.15.60:56788) 2023-07-13T22:31:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.623+0000 7f198a39c700 1 -- 172.21.15.60:0/2643068700 learned_addr learned my addr 172.21.15.60:0/2643068700 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.624+0000 7f1989b9b700 1 --2- 172.21.15.60:0/2643068700 >> v2:172.21.15.105:3300/0 conn(0x7f19840b8570 0x7f19840ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.624+0000 7f1988b99700 1 -- 172.21.15.60:0/2643068700 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39382613 0 0) 0x7f19840d09d0 con 0x7f19840d15c0 2023-07-13T22:31:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.624+0000 7f1988b99700 1 -- 172.21.15.60:0/2643068700 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19700035b0 con 0x7f19840d15c0 2023-07-13T22:31:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.624+0000 7f1989b9b700 1 -- 172.21.15.60:0/2643068700 >> v1:172.21.15.105:6789/0 conn(0x7f19840d35c0 legacy=0x7f19840d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.624+0000 7f1989b9b700 1 -- 172.21.15.60:0/2643068700 >> v1:172.21.15.60:6789/0 conn(0x7f19840d15c0 legacy=0x7f198415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.624+0000 7f1989b9b700 1 -- 172.21.15.60:0/2643068700 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19840cd450 con 0x7f19840b8570 2023-07-13T22:31:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.624+0000 7f1989b9b700 1 --2- 172.21.15.60:0/2643068700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840b8570 0x7f19840ceeb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1980009820 tx=0x7f1980009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c90ff6224993ea8e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f1988b99700 1 -- 172.21.15.60:0/2643068700 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980005e30 con 0x7f19840b8570 2023-07-13T22:31:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f1988b99700 1 -- 172.21.15.60:0/2643068700 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980006030 con 0x7f19840b8570 2023-07-13T22:31:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f1988b99700 1 -- 172.21.15.60:0/2643068700 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800063c0 con 0x7f19840b8570 2023-07-13T22:31:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f198bdff700 1 -- 172.21.15.60:0/2643068700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840b8570 msgr2=0x7f19840ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f198bdff700 1 --2- 172.21.15.60:0/2643068700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840b8570 0x7f19840ceeb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1980009820 tx=0x7f1980009b30 comp rx=0 tx=0).stop 2023-07-13T22:31:14.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f198bdff700 1 -- 172.21.15.60:0/2643068700 shutdown_connections 2023-07-13T22:31:14.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f198bdff700 1 --2- 172.21.15.60:0/2643068700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840b8570 0x7f19840ceeb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f198bdff700 1 -- 172.21.15.60:0/2643068700 >> 172.21.15.60:0/2643068700 conn(0x7f1984071600 msgr2=0x7f19840719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:14.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f198bdff700 1 -- 172.21.15.60:0/2643068700 shutdown_connections 2023-07-13T22:31:14.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.625+0000 7f198bdff700 1 -- 172.21.15.60:0/2643068700 wait complete. 2023-07-13T22:31:14.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.627+0000 7f198bdff700 1 Processor -- start 2023-07-13T22:31:14.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.627+0000 7f198bdff700 1 -- start start 2023-07-13T22:31:14.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.628+0000 7f198bdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19840b8570 0x7f198415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.628+0000 7f198bdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840d15c0 0x7f19840c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.628+0000 7f198bdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840d35c0 0x7f198406b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.628+0000 7f198bdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198415a6b0 con 0x7f19840b8570 2023-07-13T22:31:14.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.628+0000 7f198bdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19840c90b0 con 0x7f19840d35c0 2023-07-13T22:31:14.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.628+0000 7f198bdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19840c9d30 con 0x7f19840d15c0 2023-07-13T22:31:14.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.628+0000 7f1989b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19840b8570 0x7f198415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.628+0000 7f1989b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19840b8570 0x7f198415a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58364/0 (socket says 172.21.15.60:58364) 2023-07-13T22:31:14.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.628+0000 7f1989b9b700 1 -- 172.21.15.60:0/816670117 learned_addr learned my addr 172.21.15.60:0/816670117 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:14.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f198939a700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840d15c0 0x7f19840c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f198a39c700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840d35c0 0x7f198406b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f1989b9b700 1 -- 172.21.15.60:0/816670117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840d15c0 msgr2=0x7f19840c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f1989b9b700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840d15c0 0x7f19840c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f1989b9b700 1 -- 172.21.15.60:0/816670117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840d35c0 msgr2=0x7f198406b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f1989b9b700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840d35c0 0x7f198406b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f1989b9b700 1 -- 172.21.15.60:0/816670117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980009430 con 0x7f19840b8570 2023-07-13T22:31:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f198a39c700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840d35c0 0x7f198406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f1989b9b700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19840b8570 0x7f198415a170 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f197a7fc700 1 -- 172.21.15.60:0/816670117 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980004240 con 0x7f19840b8570 2023-07-13T22:31:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f197a7fc700 1 -- 172.21.15.60:0/816670117 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19800043a0 con 0x7f19840b8570 2023-07-13T22:31:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f197a7fc700 1 -- 172.21.15.60:0/816670117 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800036a0 con 0x7f19840b8570 2023-07-13T22:31:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.629+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19840ca010 con 0x7f19840b8570 2023-07-13T22:31:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.630+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19840ca5d0 con 0x7f19840b8570 2023-07-13T22:31:14.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.630+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198415d120 con 0x7f19840b8570 2023-07-13T22:31:14.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.631+0000 7f197a7fc700 1 -- 172.21.15.60:0/816670117 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1980022070 con 0x7f19840b8570 2023-07-13T22:31:14.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.632+0000 7f197a7fc700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f196806ad10 0x7f196806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:14.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.632+0000 7f197a7fc700 1 -- 172.21.15.60:0/816670117 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f198008ad60 con 0x7f19840b8570 2023-07-13T22:31:14.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.633+0000 7f198939a700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f196806ad10 0x7f196806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:14.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.633+0000 7f198939a700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f196806ad10 0x7f196806d1c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f19840b1ed0 tx=0x7f1974009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.637+0000 7f197a7fc700 1 -- 172.21.15.60:0/816670117 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1980055e00 con 0x7f19840b8570 2023-07-13T22:31:14.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.888+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19840cea90 con 0x7f19840b8570 2023-07-13T22:31:14.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.888+0000 7f197a7fc700 1 -- 172.21.15.60:0/816670117 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f198002a020 con 0x7f19840b8570 2023-07-13T22:31:14.870 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:14.870 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":88,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:30:39.690428+0000","modified":"2023-07-13T22:31:12.601793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8937":{"gid":8937,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6836/1781435251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1781435251},{"type":"v1","addr":"172.21.15.60:6836","nonce":1781435251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-13T22:31:14.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f196806ad10 msgr2=0x7f196806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f196806ad10 0x7f196806d1c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f19840b1ed0 tx=0x7f1974009040 comp rx=0 tx=0).stop 2023-07-13T22:31:14.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19840b8570 msgr2=0x7f198415a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:14.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19840b8570 0x7f198415a170 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980004750 comp rx=0 tx=0).stop 2023-07-13T22:31:14.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 shutdown_connections 2023-07-13T22:31:14.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f196806ad10 0x7f196806d1c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19840b8570 0x7f198415a170 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840d35c0 0x7f198406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 --2- 172.21.15.60:0/816670117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840d15c0 0x7f19840c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:14.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 >> 172.21.15.60:0/816670117 conn(0x7f1984071600 msgr2=0x7f1984160890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:14.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 shutdown_connections 2023-07-13T22:31:14.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:14.891+0000 7f198bdff700 1 -- 172.21.15.60:0/816670117 wait complete. 2023-07-13T22:31:14.875 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 88 2023-07-13T22:31:14.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:15.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.120+0000 7f465bfff700 1 Processor -- start 2023-07-13T22:31:15.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.120+0000 7f465bfff700 1 -- start start 2023-07-13T22:31:15.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.121+0000 7f465bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46540c9a10 0x7f465406a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.121+0000 7f465bfff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465406a860 con 0x7f4654158870 2023-07-13T22:31:15.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.121+0000 7f465bfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46540c3fc0 con 0x7f4654069ed0 2023-07-13T22:31:15.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.121+0000 7f465bfff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46540c42a0 con 0x7f46540c9a10 2023-07-13T22:31:15.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.122+0000 7f4659d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4654158870 0x7f46540c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35384/0 (socket says 172.21.15.60:35384) 2023-07-13T22:31:15.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.122+0000 7f4659d9b700 1 -- 172.21.15.60:0/635136716 learned_addr learned my addr 172.21.15.60:0/635136716 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:15.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.122+0000 7f465a59c700 1 --2- 172.21.15.60:0/635136716 >> v2:172.21.15.60:3300/0 conn(0x7f46540c9a10 0x7f465406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.122+0000 7f465a59c700 1 -- 172.21.15.60:0/635136716 >> v1:172.21.15.105:6789/0 conn(0x7f4654069ed0 legacy=0x7f46540c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.122+0000 7f4658d99700 1 -- 172.21.15.60:0/635136716 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402296966 0 0) 0x7f4650001110 con 0x7f4654158870 2023-07-13T22:31:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.122+0000 7f465a59c700 1 -- 172.21.15.60:0/635136716 >> v1:172.21.15.60:6789/0 conn(0x7f4654158870 legacy=0x7f46540c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.122+0000 7f465a59c700 1 -- 172.21.15.60:0/635136716 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46540c4580 con 0x7f46540c9a10 2023-07-13T22:31:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.122+0000 7f465a59c700 1 --2- 172.21.15.60:0/635136716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46540c9a10 0x7f465406a320 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f464c009d90 tx=0x7f464c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8103e03bc6df9394 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f4658d99700 1 -- 172.21.15.60:0/635136716 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c00e070 con 0x7f46540c9a10 2023-07-13T22:31:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f4658d99700 1 -- 172.21.15.60:0/635136716 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c00bb90 con 0x7f46540c9a10 2023-07-13T22:31:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f4658d99700 1 -- 172.21.15.60:0/635136716 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c003a20 con 0x7f46540c9a10 2023-07-13T22:31:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f465bfff700 1 -- 172.21.15.60:0/635136716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46540c9a10 msgr2=0x7f465406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f465bfff700 1 --2- 172.21.15.60:0/635136716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46540c9a10 0x7f465406a320 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f464c009d90 tx=0x7f464c00b330 comp rx=0 tx=0).stop 2023-07-13T22:31:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f465bfff700 1 -- 172.21.15.60:0/635136716 shutdown_connections 2023-07-13T22:31:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f465bfff700 1 --2- 172.21.15.60:0/635136716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46540c9a10 0x7f465406a320 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f465bfff700 1 -- 172.21.15.60:0/635136716 >> 172.21.15.60:0/635136716 conn(0x7f46540cf410 msgr2=0x7f46540cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f465bfff700 1 -- 172.21.15.60:0/635136716 shutdown_connections 2023-07-13T22:31:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.123+0000 7f465bfff700 1 -- 172.21.15.60:0/635136716 wait complete. 2023-07-13T22:31:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.125+0000 7f465bfff700 1 Processor -- start 2023-07-13T22:31:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.125+0000 7f465bfff700 1 -- start start 2023-07-13T22:31:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465bfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4654069ed0 0x7f46540b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46540c9a10 0x7f46540c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4654158870 0x7f46540c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465bfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46540b12b0 con 0x7f4654158870 2023-07-13T22:31:15.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465bfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46541420f0 con 0x7f46540c9a10 2023-07-13T22:31:15.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465bfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46540b0cf0 con 0x7f4654069ed0 2023-07-13T22:31:15.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4654158870 0x7f46540c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4654158870 0x7f46540c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40100/0 (socket says 172.21.15.60:40100) 2023-07-13T22:31:15.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465a59c700 1 -- 172.21.15.60:0/2651466113 learned_addr learned my addr 172.21.15.60:0/2651466113 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:15.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f4659d9b700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4654069ed0 0x7f46540b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.126+0000 7f465959a700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46540c9a10 0x7f46540c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f465a59c700 1 -- 172.21.15.60:0/2651466113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4654069ed0 msgr2=0x7f46540b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f465a59c700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4654069ed0 0x7f46540b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f465a59c700 1 -- 172.21.15.60:0/2651466113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46540c9a10 msgr2=0x7f46540c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f465a59c700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46540c9a10 0x7f46540c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f465a59c700 1 -- 172.21.15.60:0/2651466113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c009a70 con 0x7f4654158870 2023-07-13T22:31:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f465a59c700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4654158870 0x7f46540c7d20 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f465406b790 tx=0x7f464c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f464a7fc700 1 -- 172.21.15.60:0/2651466113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c012630 con 0x7f4654158870 2023-07-13T22:31:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f464a7fc700 1 -- 172.21.15.60:0/2651466113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c012790 con 0x7f4654158870 2023-07-13T22:31:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f464a7fc700 1 -- 172.21.15.60:0/2651466113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c012b40 con 0x7f4654158870 2023-07-13T22:31:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.127+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46540b0fd0 con 0x7f4654158870 2023-07-13T22:31:15.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.128+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46540abd60 con 0x7f4654158870 2023-07-13T22:31:15.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.129+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46540ab8e0 con 0x7f4654158870 2023-07-13T22:31:15.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.131+0000 7f464a7fc700 1 -- 172.21.15.60:0/2651466113 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f464c012ca0 con 0x7f4654158870 2023-07-13T22:31:15.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.132+0000 7f464a7fc700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f463806ace0 0x7f463806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.132+0000 7f464a7fc700 1 -- 172.21.15.60:0/2651466113 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f464c08b680 con 0x7f4654158870 2023-07-13T22:31:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.133+0000 7f4659d9b700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f463806ace0 0x7f463806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.133+0000 7f4659d9b700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f463806ace0 0x7f463806d190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4650000f60 tx=0x7f4650007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.137+0000 7f464a7fc700 1 -- 172.21.15.60:0/2651466113 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f46540ab8e0 con 0x7f4654158870 2023-07-13T22:31:15.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.343+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46540ab8e0 con 0x7f4654158870 2023-07-13T22:31:15.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.344+0000 7f464a7fc700 1 -- 172.21.15.60:0/2651466113 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18215 (secure 0 0 0) 0x7f46540ab8e0 con 0x7f4654158870 2023-07-13T22:31:15.326 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:15.327 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":124,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:30:41.182757+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:30:37.983326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-13T22:30:38.665534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.346+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f463806ace0 msgr2=0x7f463806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.346+0000 7f465bfff700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f463806ace0 0x7f463806d190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4650000f60 tx=0x7f4650007e70 comp rx=0 tx=0).stop 2023-07-13T22:31:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.346+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4654158870 msgr2=0x7f46540c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.346+0000 7f465bfff700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4654158870 0x7f46540c7d20 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f465406b790 tx=0x7f464c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.347+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 shutdown_connections 2023-07-13T22:31:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.347+0000 7f465bfff700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f463806ace0 0x7f463806d190 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.347+0000 7f465bfff700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4654158870 0x7f46540c7d20 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.347+0000 7f465bfff700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4654069ed0 0x7f46540b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.347+0000 7f465bfff700 1 --2- 172.21.15.60:0/2651466113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46540c9a10 0x7f46540c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.347+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 >> 172.21.15.60:0/2651466113 conn(0x7f46540cf410 msgr2=0x7f4654070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.347+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 shutdown_connections 2023-07-13T22:31:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.347+0000 7f465bfff700 1 -- 172.21.15.60:0/2651466113 wait complete. 2023-07-13T22:31:15.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:31:15.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.568+0000 7fa5d13ff700 1 Processor -- start 2023-07-13T22:31:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.568+0000 7fa5d13ff700 1 -- start start 2023-07-13T22:31:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.569+0000 7fa5d13ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa5cc158870 0x7fa5cc0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.569+0000 7fa5d13ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc0c52c0 0x7fa5cc069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.569+0000 7fa5d13ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc0c3fc0 con 0x7fa5cc06a410 2023-07-13T22:31:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.569+0000 7fa5d13ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc06a890 con 0x7fa5cc158870 2023-07-13T22:31:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.569+0000 7fa5d13ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0c42a0 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.569+0000 7fa5cb7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa5cc06a410 0x7fa5cc0c9300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35400/0 (socket says 172.21.15.60:35400) 2023-07-13T22:31:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5cb7fe700 1 -- 172.21.15.60:0/1188506509 learned_addr learned my addr 172.21.15.60:0/1188506509 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:15.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5caffd700 1 --2- 172.21.15.60:0/1188506509 >> v2:172.21.15.105:3300/0 conn(0x7fa5cc158870 0x7fa5cc0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5ca7fc700 1 --2- 172.21.15.60:0/1188506509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc0c52c0 0x7fa5cc069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5c9ffb700 1 -- 172.21.15.60:0/1188506509 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205732056 0 0) 0x7fa5cc0c3fc0 con 0x7fa5cc06a410 2023-07-13T22:31:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5c9ffb700 1 -- 172.21.15.60:0/1188506509 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b40035b0 con 0x7fa5cc06a410 2023-07-13T22:31:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5ca7fc700 1 -- 172.21.15.60:0/1188506509 >> v2:172.21.15.105:3300/0 conn(0x7fa5cc158870 msgr2=0x7fa5cc0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5ca7fc700 1 --2- 172.21.15.60:0/1188506509 >> v2:172.21.15.105:3300/0 conn(0x7fa5cc158870 0x7fa5cc0c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5ca7fc700 1 -- 172.21.15.60:0/1188506509 >> v1:172.21.15.60:6789/0 conn(0x7fa5cc06a410 legacy=0x7fa5cc0c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5ca7fc700 1 -- 172.21.15.60:0/1188506509 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0c4580 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.570+0000 7fa5caffd700 1 --2- 172.21.15.60:0/1188506509 >> v2:172.21.15.105:3300/0 conn(0x7fa5cc158870 0x7fa5cc0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.571+0000 7fa5ca7fc700 1 --2- 172.21.15.60:0/1188506509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc0c52c0 0x7fa5cc069ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009840 tx=0x7fa5c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e497278becf18d6f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.571+0000 7fa5c9ffb700 1 -- 172.21.15.60:0/1188506509 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0011830 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.571+0000 7fa5c9ffb700 1 -- 172.21.15.60:0/1188506509 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0011990 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.571+0000 7fa5c9ffb700 1 -- 172.21.15.60:0/1188506509 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0011d20 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.571+0000 7fa5d13ff700 1 -- 172.21.15.60:0/1188506509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc0c52c0 msgr2=0x7fa5cc069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.571+0000 7fa5d13ff700 1 --2- 172.21.15.60:0/1188506509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc0c52c0 0x7fa5cc069ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009840 tx=0x7fa5c0009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.572+0000 7fa5d13ff700 1 -- 172.21.15.60:0/1188506509 shutdown_connections 2023-07-13T22:31:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.572+0000 7fa5d13ff700 1 --2- 172.21.15.60:0/1188506509 >> v2:172.21.15.105:3300/0 conn(0x7fa5cc158870 0x7fa5cc0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.572+0000 7fa5d13ff700 1 --2- 172.21.15.60:0/1188506509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc0c52c0 0x7fa5cc069ed0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.572+0000 7fa5d13ff700 1 -- 172.21.15.60:0/1188506509 >> 172.21.15.60:0/1188506509 conn(0x7fa5cc0cf410 msgr2=0x7fa5cc0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.572+0000 7fa5d13ff700 1 -- 172.21.15.60:0/1188506509 shutdown_connections 2023-07-13T22:31:15.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.572+0000 7fa5d13ff700 1 -- 172.21.15.60:0/1188506509 wait complete. 2023-07-13T22:31:15.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.573+0000 7fa5d13ff700 1 Processor -- start 2023-07-13T22:31:15.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.573+0000 7fa5d13ff700 1 -- start start 2023-07-13T22:31:15.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.574+0000 7fa5d13ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc06a410 0x7fa5cc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.574+0000 7fa5d13ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5cc0c52c0 0x7fa5cc0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.574+0000 7fa5d13ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc158870 0x7fa5cc164780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.574+0000 7fa5d13ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0b4690 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.574+0000 7fa5d13ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0c63f0 con 0x7fa5cc158870 2023-07-13T22:31:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.574+0000 7fa5d13ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0b1c10 con 0x7fa5cc06a410 2023-07-13T22:31:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5ca7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5cc0c52c0 0x7fa5cc0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5ca7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5cc0c52c0 0x7fa5cc0b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40110/0 (socket says 172.21.15.60:40110) 2023-07-13T22:31:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5ca7fc700 1 -- 172.21.15.60:0/4049751609 learned_addr learned my addr 172.21.15.60:0/4049751609 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5caffd700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc06a410 0x7fa5cc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5cb7fe700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc158870 0x7fa5cc164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5ca7fc700 1 -- 172.21.15.60:0/4049751609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc06a410 msgr2=0x7fa5cc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5ca7fc700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc06a410 0x7fa5cc0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5ca7fc700 1 -- 172.21.15.60:0/4049751609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc158870 msgr2=0x7fa5cc164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5ca7fc700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc158870 0x7fa5cc164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5ca7fc700 1 -- 172.21.15.60:0/4049751609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc0094a0 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5cb7fe700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc158870 0x7fa5cc164780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5ca7fc700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5cc0c52c0 0x7fa5cc0b16d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00090b0 tx=0x7fa5c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.575+0000 7fa5b37fe700 1 -- 172.21.15.60:0/4049751609 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00036a0 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.576+0000 7fa5b37fe700 1 -- 172.21.15.60:0/4049751609 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c001f070 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.576+0000 7fa5b37fe700 1 -- 172.21.15.60:0/4049751609 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0003a60 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.576+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c00094a0 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.576+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc0b2190 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.577+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc164560 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.578+0000 7fa5b37fe700 1 -- 172.21.15.60:0/4049751609 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5c0003bc0 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.579+0000 7fa5b37fe700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5b806ace0 0x7fa5b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:15.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.579+0000 7fa5b37fe700 1 -- 172.21.15.60:0/4049751609 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5c0021070 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.579+0000 7fa5caffd700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5b806ace0 0x7fa5b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:15.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.580+0000 7fa5caffd700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5b806ace0 0x7fa5b806d190 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc15d290 tx=0x7fa5bc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:15.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.584+0000 7fa5b37fe700 1 -- 172.21.15.60:0/4049751609 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa5c0055d80 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.835+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa5cc0c39f0 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.817 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:31:15.835+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:31:15.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.841+0000 7fa5b37fe700 1 -- 172.21.15.60:0/4049751609 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa5c0015090 con 0x7fa5cc0c52c0 2023-07-13T22:31:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5b806ace0 msgr2=0x7fa5b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5b806ace0 0x7fa5b806d190 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc15d290 tx=0x7fa5bc009210 comp rx=0 tx=0).stop 2023-07-13T22:31:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5cc0c52c0 msgr2=0x7fa5cc0b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:15.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5cc0c52c0 0x7fa5cc0b16d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00090b0 tx=0x7fa5c0004750 comp rx=0 tx=0).stop 2023-07-13T22:31:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 shutdown_connections 2023-07-13T22:31:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa5b806ace0 0x7fa5b806d190 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc158870 0x7fa5cc164780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5cc06a410 0x7fa5cc0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 --2- 172.21.15.60:0/4049751609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5cc0c52c0 0x7fa5cc0b16d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 >> 172.21.15.60:0/4049751609 conn(0x7fa5cc0cf410 msgr2=0x7fa5cc0cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:15.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 shutdown_connections 2023-07-13T22:31:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:15.843+0000 7fa5d13ff700 1 -- 172.21.15.60:0/4049751609 wait complete. 2023-07-13T22:31:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:31:15.844 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:31:15.891 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:31:16.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.071+0000 7f6aa3fff700 1 Processor -- start 2023-07-13T22:31:16.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.071+0000 7f6aa3fff700 1 -- start start 2023-07-13T22:31:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa3fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a9c0cfd50 0x7f6a9c0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa3fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6a9c0b1ac0 0x7f6a9c0d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa3fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9c0d1870 con 0x7f6a9c0b8200 2023-07-13T22:31:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa3fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9c0cf370 con 0x7f6a9c0cfd50 2023-07-13T22:31:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa3fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9c0d12b0 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa159a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a9c0cfd50 0x7f6a9c0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:16.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa159a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a9c0cfd50 0x7f6a9c0d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40142/0 (socket says 172.21.15.60:40142) 2023-07-13T22:31:16.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa159a700 1 -- 172.21.15.60:0/383411475 learned_addr learned my addr 172.21.15.60:0/383411475 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:16.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa259c700 1 --2- 172.21.15.60:0/383411475 >> v2:172.21.15.105:3300/0 conn(0x7f6a9c0b1ac0 0x7f6a9c0d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:16.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa159a700 1 -- 172.21.15.60:0/383411475 >> v2:172.21.15.105:3300/0 conn(0x7f6a9c0b1ac0 msgr2=0x7f6a9c0d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa159a700 1 --2- 172.21.15.60:0/383411475 >> v2:172.21.15.105:3300/0 conn(0x7f6a9c0b1ac0 0x7f6a9c0d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa159a700 1 -- 172.21.15.60:0/383411475 >> v1:172.21.15.105:6789/0 conn(0x7f6a9c0b8200 legacy=0x7f6a9c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.073+0000 7f6aa159a700 1 -- 172.21.15.60:0/383411475 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0d1590 con 0x7f6a9c0cfd50 2023-07-13T22:31:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.074+0000 7f6aa159a700 1 --2- 172.21.15.60:0/383411475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0cfd50 0x7f6a9c0d23c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0b8d70 tx=0x7f6a8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b787495047a22c37 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.074+0000 7f6aa0d99700 1 -- 172.21.15.60:0/383411475 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c004230 con 0x7f6a9c0cfd50 2023-07-13T22:31:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.074+0000 7f6aa0d99700 1 -- 172.21.15.60:0/383411475 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c004390 con 0x7f6a9c0cfd50 2023-07-13T22:31:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.074+0000 7f6aa0d99700 1 -- 172.21.15.60:0/383411475 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0036a0 con 0x7f6a9c0cfd50 2023-07-13T22:31:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.074+0000 7f6aa3fff700 1 -- 172.21.15.60:0/383411475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0cfd50 msgr2=0x7f6a9c0d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.074+0000 7f6aa3fff700 1 --2- 172.21.15.60:0/383411475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0cfd50 0x7f6a9c0d23c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0b8d70 tx=0x7f6a8c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.075+0000 7f6aa3fff700 1 -- 172.21.15.60:0/383411475 shutdown_connections 2023-07-13T22:31:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.075+0000 7f6aa3fff700 1 --2- 172.21.15.60:0/383411475 >> v2:172.21.15.105:3300/0 conn(0x7f6a9c0b1ac0 0x7f6a9c0d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.075+0000 7f6aa3fff700 1 --2- 172.21.15.60:0/383411475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0cfd50 0x7f6a9c0d23c0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.075+0000 7f6aa3fff700 1 -- 172.21.15.60:0/383411475 >> 172.21.15.60:0/383411475 conn(0x7f6a9c06b2f0 msgr2=0x7f6a9c06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.075+0000 7f6aa3fff700 1 -- 172.21.15.60:0/383411475 shutdown_connections 2023-07-13T22:31:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.075+0000 7f6aa3fff700 1 -- 172.21.15.60:0/383411475 wait complete. 2023-07-13T22:31:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.076+0000 7f6aa3fff700 1 Processor -- start 2023-07-13T22:31:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.077+0000 7f6aa3fff700 1 -- start start 2023-07-13T22:31:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.077+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0b1ac0 0x7f6a9c0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.077+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a9c0b8200 0x7f6a9c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.077+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a9c0cfd50 0x7f6a9c0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa3fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c15ab20 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa3fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0c8090 con 0x7f6a9c0b8200 2023-07-13T22:31:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa3fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0c8370 con 0x7f6a9c0cfd50 2023-07-13T22:31:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0b1ac0 0x7f6a9c0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0b1ac0 0x7f6a9c0c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40146/0 (socket says 172.21.15.60:40146) 2023-07-13T22:31:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa1d9b700 1 -- 172.21.15.60:0/1217657585 learned_addr learned my addr 172.21.15.60:0/1217657585 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa1d9b700 1 -- 172.21.15.60:0/1217657585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a9c0cfd50 msgr2=0x7f6a9c0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa1d9b700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a9c0cfd50 0x7f6a9c0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa1d9b700 1 -- 172.21.15.60:0/1217657585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a9c0b8200 msgr2=0x7f6a9c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa1d9b700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a9c0b8200 0x7f6a9c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa1d9b700 1 -- 172.21.15.60:0/1217657585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c009550 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.078+0000 7f6aa1d9b700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0b1ac0 0x7f6a9c0c9130 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c4d0 tx=0x7f6a9800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:16.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.079+0000 7f6a927fc700 1 -- 172.21.15.60:0/1217657585 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98011790 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.079+0000 7f6a927fc700 1 -- 172.21.15.60:0/1217657585 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a980118f0 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.079+0000 7f6a927fc700 1 -- 172.21.15.60:0/1217657585 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98011ca0 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.079+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c0c8650 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.079+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c0c8c10 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.080+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9c04e210 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.081+0000 7f6a927fc700 1 -- 172.21.15.60:0/1217657585 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a9801c460 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.082+0000 7f6a927fc700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a8006acb0 0x7f6a8006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:16.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.082+0000 7f6a927fc700 1 -- 172.21.15.60:0/1217657585 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f6a98089360 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.082+0000 7f6aa159a700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a8006acb0 0x7f6a8006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:16.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.083+0000 7f6aa159a700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a8006acb0 0x7f6a8006d160 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0d0140 tx=0x7f6a8c010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:16.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.087+0000 7f6a927fc700 1 -- 172.21.15.60:0/1217657585 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6a980543b0 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.339+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6a9c0b4f70 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.842+0000 7f6a927fc700 1 -- 172.21.15.60:0/1217657585 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v90) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6a980579d0 con 0x7f6a9c0b1ac0 2023-07-13T22:31:16.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a8006acb0 msgr2=0x7f6a8006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:16.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a8006acb0 0x7f6a8006d160 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0d0140 tx=0x7f6a8c010000 comp rx=0 tx=0).stop 2023-07-13T22:31:16.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0b1ac0 msgr2=0x7f6a9c0c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:16.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0b1ac0 0x7f6a9c0c9130 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c4d0 tx=0x7f6a9800c890 comp rx=0 tx=0).stop 2023-07-13T22:31:16.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 shutdown_connections 2023-07-13T22:31:16.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6a8006acb0 0x7f6a8006d160 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a9c0b1ac0 0x7f6a9c0c9130 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a9c0cfd50 0x7f6a9c0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 --2- 172.21.15.60:0/1217657585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a9c0b8200 0x7f6a9c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:16.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 >> 172.21.15.60:0/1217657585 conn(0x7f6a9c06b2f0 msgr2=0x7f6a9c15ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:16.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 shutdown_connections 2023-07-13T22:31:16.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:16.845+0000 7f6aa3fff700 1 -- 172.21.15.60:0/1217657585 wait complete. 2023-07-13T22:31:16.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:31:17.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.078+0000 7f827b1ff700 1 Processor -- start 2023-07-13T22:31:17.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.078+0000 7f827b1ff700 1 -- start start 2023-07-13T22:31:17.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.079+0000 7f827b1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8274159ea0 0x7f82740cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:17.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.079+0000 7f827b1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f827415c9b0 0x7f82740d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:17.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.079+0000 7f827b1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827415d200 con 0x7f82740d37c0 2023-07-13T22:31:17.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.079+0000 7f827b1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82740d2e60 con 0x7f8274159ea0 2023-07-13T22:31:17.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f827b1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82740d2b50 con 0x7f827415c9b0 2023-07-13T22:31:17.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f8278f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8274159ea0 0x7f82740cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:17.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f827979c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f82740d37c0 0x7f82740d0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35424/0 (socket says 172.21.15.60:35424) 2023-07-13T22:31:17.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f8278f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8274159ea0 0x7f82740cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40160/0 (socket says 172.21.15.60:40160) 2023-07-13T22:31:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f827979c700 1 -- 172.21.15.60:0/967095024 learned_addr learned my addr 172.21.15.60:0/967095024 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f8273fff700 1 --2- 172.21.15.60:0/967095024 >> v2:172.21.15.105:3300/0 conn(0x7f827415c9b0 0x7f82740d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f8278f9b700 1 -- 172.21.15.60:0/967095024 >> v2:172.21.15.105:3300/0 conn(0x7f827415c9b0 msgr2=0x7f82740d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f8278f9b700 1 --2- 172.21.15.60:0/967095024 >> v2:172.21.15.105:3300/0 conn(0x7f827415c9b0 0x7f82740d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f8278f9b700 1 -- 172.21.15.60:0/967095024 >> v1:172.21.15.60:6789/0 conn(0x7f82740d37c0 legacy=0x7f82740d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f8278f9b700 1 -- 172.21.15.60:0/967095024 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827415d4e0 con 0x7f8274159ea0 2023-07-13T22:31:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.080+0000 7f82737fe700 1 -- 172.21.15.60:0/967095024 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178401320 0 0) 0x7f827415d200 con 0x7f82740d37c0 2023-07-13T22:31:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f8278f9b700 1 --2- 172.21.15.60:0/967095024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8274159ea0 0x7f82740cd6d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f82740d2910 tx=0x7f8268009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28761ddba36ca0c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f82737fe700 1 -- 172.21.15.60:0/967095024 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268011830 con 0x7f8274159ea0 2023-07-13T22:31:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f82737fe700 1 -- 172.21.15.60:0/967095024 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8268011990 con 0x7f8274159ea0 2023-07-13T22:31:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f82737fe700 1 -- 172.21.15.60:0/967095024 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268011d20 con 0x7f8274159ea0 2023-07-13T22:31:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f827b1ff700 1 -- 172.21.15.60:0/967095024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8274159ea0 msgr2=0x7f82740cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f827b1ff700 1 --2- 172.21.15.60:0/967095024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8274159ea0 0x7f82740cd6d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f82740d2910 tx=0x7f8268009b20 comp rx=0 tx=0).stop 2023-07-13T22:31:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f827b1ff700 1 -- 172.21.15.60:0/967095024 shutdown_connections 2023-07-13T22:31:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f827b1ff700 1 --2- 172.21.15.60:0/967095024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8274159ea0 0x7f82740cd6d0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:17.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f827b1ff700 1 --2- 172.21.15.60:0/967095024 >> v2:172.21.15.105:3300/0 conn(0x7f827415c9b0 0x7f82740d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:17.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f827b1ff700 1 -- 172.21.15.60:0/967095024 >> 172.21.15.60:0/967095024 conn(0x7f8274071600 msgr2=0x7f82740719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:17.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.081+0000 7f827b1ff700 1 -- 172.21.15.60:0/967095024 shutdown_connections 2023-07-13T22:31:17.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.082+0000 7f827b1ff700 1 -- 172.21.15.60:0/967095024 wait complete. 2023-07-13T22:31:17.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.083+0000 7f827b1ff700 1 Processor -- start 2023-07-13T22:31:17.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.083+0000 7f827b1ff700 1 -- start start 2023-07-13T22:31:17.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.084+0000 7f827b1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82740d37c0 0x7f82740c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:17.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.084+0000 7f827b1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8274159ea0 0x7f82741533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:17.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.084+0000 7f827b1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f827415c9b0 0x7f82740cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:17.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.084+0000 7f827b1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8274153930 con 0x7f82740d37c0 2023-07-13T22:31:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.084+0000 7f827b1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82740c8160 con 0x7f827415c9b0 2023-07-13T22:31:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.084+0000 7f827b1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82740c8440 con 0x7f8274159ea0 2023-07-13T22:31:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8278f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82740d37c0 0x7f82740c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8278f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82740d37c0 0x7f82740c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40162/0 (socket says 172.21.15.60:40162) 2023-07-13T22:31:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8278f9b700 1 -- 172.21.15.60:0/3059534517 learned_addr learned my addr 172.21.15.60:0/3059534517 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f827979c700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f827415c9b0 0x7f82740cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8273fff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8274159ea0 0x7f82741533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8278f9b700 1 -- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8274159ea0 msgr2=0x7f82741533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8278f9b700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8274159ea0 0x7f82741533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8278f9b700 1 -- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f827415c9b0 msgr2=0x7f82740cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8278f9b700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f827415c9b0 0x7f82740cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8278f9b700 1 -- 172.21.15.60:0/3059534517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82680094a0 con 0x7f82740d37c0 2023-07-13T22:31:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8273fff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8274159ea0 0x7f82741533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f827979c700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f827415c9b0 0x7f82740cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.085+0000 7f8278f9b700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82740d37c0 0x7f82740c9200 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f82680090b0 tx=0x7f8268004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.086+0000 7f82717fa700 1 -- 172.21.15.60:0/3059534517 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82680038a0 con 0x7f82740d37c0 2023-07-13T22:31:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.086+0000 7f82717fa700 1 -- 172.21.15.60:0/3059534517 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826801f070 con 0x7f82740d37c0 2023-07-13T22:31:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.086+0000 7f82717fa700 1 -- 172.21.15.60:0/3059534517 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268003c20 con 0x7f82740d37c0 2023-07-13T22:31:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.086+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82740c8720 con 0x7f82740d37c0 2023-07-13T22:31:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.086+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82740c8ce0 con 0x7f82740d37c0 2023-07-13T22:31:17.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.087+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8274067a70 con 0x7f82740d37c0 2023-07-13T22:31:17.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.088+0000 7f82717fa700 1 -- 172.21.15.60:0/3059534517 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f826801a430 con 0x7f82740d37c0 2023-07-13T22:31:17.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.089+0000 7f82717fa700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f825806ace0 0x7f825806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:17.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.089+0000 7f82717fa700 1 -- 172.21.15.60:0/3059534517 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f8268021070 con 0x7f82740d37c0 2023-07-13T22:31:17.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.089+0000 7f8273fff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f825806ace0 0x7f825806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:17.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.090+0000 7f8273fff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f825806ace0 0x7f825806d190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f82740b3640 tx=0x7f8260008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:17.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.093+0000 7f82717fa700 1 -- 172.21.15.60:0/3059534517 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8268055fa0 con 0x7f82740d37c0 2023-07-13T22:31:17.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.300+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f827404e210 con 0x7f82740d37c0 2023-07-13T22:31:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.846+0000 7f82717fa700 1 -- 172.21.15.60:0/3059534517 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 181+0+0 (secure 0 0 0) 0x7f826802b0d0 con 0x7f82740d37c0 2023-07-13T22:31:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.850+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f825806ace0 msgr2=0x7f825806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.850+0000 7f827b1ff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f825806ace0 0x7f825806d190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f82740b3640 tx=0x7f8260008040 comp rx=0 tx=0).stop 2023-07-13T22:31:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.850+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82740d37c0 msgr2=0x7f82740c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.850+0000 7f827b1ff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82740d37c0 0x7f82740c9200 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f82680090b0 tx=0x7f8268004750 comp rx=0 tx=0).stop 2023-07-13T22:31:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.851+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 shutdown_connections 2023-07-13T22:31:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.851+0000 7f827b1ff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f825806ace0 0x7f825806d190 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.851+0000 7f827b1ff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82740d37c0 0x7f82740c9200 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.851+0000 7f827b1ff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f827415c9b0 0x7f82740cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.851+0000 7f827b1ff700 1 --2- 172.21.15.60:0/3059534517 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8274159ea0 0x7f82741533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.851+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 >> 172.21.15.60:0/3059534517 conn(0x7f8274071600 msgr2=0x7f82740d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.851+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 shutdown_connections 2023-07-13T22:31:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:17.851+0000 7f827b1ff700 1 -- 172.21.15.60:0/3059534517 wait complete. 2023-07-13T22:31:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:31:17.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:31:18.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.083+0000 7f8b997ff700 1 Processor -- start 2023-07-13T22:31:18.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.083+0000 7f8b997ff700 1 -- start start 2023-07-13T22:31:18.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.084+0000 7f8b997ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8b94070a60 0x7f8b94164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:18.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.085+0000 7f8b997ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b940c94a0 con 0x7f8b940c8da0 2023-07-13T22:31:18.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.085+0000 7f8b997ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b940c91d0 con 0x7f8b9415a530 2023-07-13T22:31:18.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.085+0000 7f8b997ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9406de70 con 0x7f8b94070a60 2023-07-13T22:31:18.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.085+0000 7f8b927fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8b940c8da0 0x7f8b94070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35426/0 (socket says 172.21.15.60:35426) 2023-07-13T22:31:18.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.085+0000 7f8b927fc700 1 -- 172.21.15.60:0/2160067712 learned_addr learned my addr 172.21.15.60:0/2160067712 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:18.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.085+0000 7f8b937fe700 1 --2- 172.21.15.60:0/2160067712 >> v2:172.21.15.105:3300/0 conn(0x7f8b94070a60 0x7f8b94164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:18.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.085+0000 7f8b91ffb700 1 -- 172.21.15.60:0/2160067712 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033579619 0 0) 0x7f8b940c94a0 con 0x7f8b940c8da0 2023-07-13T22:31:18.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.085+0000 7f8b91ffb700 1 -- 172.21.15.60:0/2160067712 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b7c0035b0 con 0x7f8b940c8da0 2023-07-13T22:31:18.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.086+0000 7f8b937fe700 1 -- 172.21.15.60:0/2160067712 >> v1:172.21.15.105:6789/0 conn(0x7f8b9415a530 legacy=0x7f8b940addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:18.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.086+0000 7f8b937fe700 1 -- 172.21.15.60:0/2160067712 >> v1:172.21.15.60:6789/0 conn(0x7f8b940c8da0 legacy=0x7f8b94070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:18.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.086+0000 7f8b937fe700 1 -- 172.21.15.60:0/2160067712 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9406d4d0 con 0x7f8b94070a60 2023-07-13T22:31:18.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.086+0000 7f8b937fe700 1 --2- 172.21.15.60:0/2160067712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b94070a60 0x7f8b94164ed0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009d90 tx=0x7f8b8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a61d91c4a62667e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:18.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.086+0000 7f8b91ffb700 1 -- 172.21.15.60:0/2160067712 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00e070 con 0x7f8b94070a60 2023-07-13T22:31:18.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.086+0000 7f8b91ffb700 1 -- 172.21.15.60:0/2160067712 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c00bb20 con 0x7f8b94070a60 2023-07-13T22:31:18.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.086+0000 7f8b91ffb700 1 -- 172.21.15.60:0/2160067712 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c012430 con 0x7f8b94070a60 2023-07-13T22:31:18.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.087+0000 7f8b997ff700 1 -- 172.21.15.60:0/2160067712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b94070a60 msgr2=0x7f8b94164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:18.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.087+0000 7f8b997ff700 1 --2- 172.21.15.60:0/2160067712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b94070a60 0x7f8b94164ed0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009d90 tx=0x7f8b8c00b330 comp rx=0 tx=0).stop 2023-07-13T22:31:18.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.087+0000 7f8b997ff700 1 -- 172.21.15.60:0/2160067712 shutdown_connections 2023-07-13T22:31:18.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.087+0000 7f8b997ff700 1 --2- 172.21.15.60:0/2160067712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b94070a60 0x7f8b94164ed0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:18.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.087+0000 7f8b997ff700 1 -- 172.21.15.60:0/2160067712 >> 172.21.15.60:0/2160067712 conn(0x7f8b940cf3b0 msgr2=0x7f8b940cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:18.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.087+0000 7f8b997ff700 1 -- 172.21.15.60:0/2160067712 shutdown_connections 2023-07-13T22:31:18.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.087+0000 7f8b997ff700 1 -- 172.21.15.60:0/2160067712 wait complete. 2023-07-13T22:31:18.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.089+0000 7f8b997ff700 1 Processor -- start 2023-07-13T22:31:18.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.089+0000 7f8b997ff700 1 -- start start 2023-07-13T22:31:18.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b997ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b94070a60 0x7f8b94151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:18.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b997ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b940c8da0 0x7f8b9415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:18.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b997ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b9415a530 0x7f8b9415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:18.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b997ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b941518b0 con 0x7f8b9415a530 2023-07-13T22:31:18.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b997ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b941f3e20 con 0x7f8b940c8da0 2023-07-13T22:31:18.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b997ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b941505b0 con 0x7f8b94070a60 2023-07-13T22:31:18.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b9415a530 0x7f8b9415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:18.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b937fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b9415a530 0x7f8b9415cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40176/0 (socket says 172.21.15.60:40176) 2023-07-13T22:31:18.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b937fe700 1 -- 172.21.15.60:0/2099460765 learned_addr learned my addr 172.21.15.60:0/2099460765 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:18.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b927fc700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b940c8da0 0x7f8b9415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:18.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.090+0000 7f8b92ffd700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b94070a60 0x7f8b94151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:18.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b937fe700 1 -- 172.21.15.60:0/2099460765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b94070a60 msgr2=0x7f8b94151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:18.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b937fe700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b94070a60 0x7f8b94151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:18.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b937fe700 1 -- 172.21.15.60:0/2099460765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b940c8da0 msgr2=0x7f8b9415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:18.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b937fe700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b940c8da0 0x7f8b9415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:18.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b937fe700 1 -- 172.21.15.60:0/2099460765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009a70 con 0x7f8b9415a530 2023-07-13T22:31:18.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b937fe700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b9415a530 0x7f8b9415cee0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c013000 tx=0x7f8b8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:18.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b7b7fe700 1 -- 172.21.15.60:0/2099460765 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c012430 con 0x7f8b9415a530 2023-07-13T22:31:18.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b7b7fe700 1 -- 172.21.15.60:0/2099460765 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c020070 con 0x7f8b9415a530 2023-07-13T22:31:18.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b94150890 con 0x7f8b9415a530 2023-07-13T22:31:18.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b7b7fe700 1 -- 172.21.15.60:0/2099460765 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c004480 con 0x7f8b9415a530 2023-07-13T22:31:18.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.091+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b94150e50 con 0x7f8b9415a530 2023-07-13T22:31:18.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.092+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b94150fe0 con 0x7f8b9415a530 2023-07-13T22:31:18.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.093+0000 7f8b7b7fe700 1 -- 172.21.15.60:0/2099460765 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b8c0045e0 con 0x7f8b9415a530 2023-07-13T22:31:18.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.094+0000 7f8b7b7fe700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b8006ace0 0x7f8b8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:18.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.094+0000 7f8b7b7fe700 1 -- 172.21.15.60:0/2099460765 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f8b8c022070 con 0x7f8b9415a530 2023-07-13T22:31:18.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.095+0000 7f8b92ffd700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b8006ace0 0x7f8b8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:18.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.095+0000 7f8b92ffd700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b8006ace0 0x7f8b8006d190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8b84000f60 tx=0x7f8b84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:18.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.099+0000 7f8b7b7fe700 1 -- 172.21.15.60:0/2099460765 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8b8c056500 con 0x7f8b9415a530 2023-07-13T22:31:18.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.313+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b9404e210 con 0x7f8b9415a530 2023-07-13T22:31:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.850+0000 7f8b7b7fe700 1 -- 172.21.15.60:0/2099460765 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b8c059b20 con 0x7f8b9415a530 2023-07-13T22:31:18.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.853+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b8006ace0 msgr2=0x7f8b8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:18.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b8006ace0 0x7f8b8006d190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8b84000f60 tx=0x7f8b84009040 comp rx=0 tx=0).stop 2023-07-13T22:31:18.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b9415a530 msgr2=0x7f8b9415cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:18.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b9415a530 0x7f8b9415cee0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c013000 tx=0x7f8b8c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:31:18.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 shutdown_connections 2023-07-13T22:31:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8b8006ace0 0x7f8b8006d190 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b9415a530 0x7f8b9415cee0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b94070a60 0x7f8b94151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 --2- 172.21.15.60:0/2099460765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b940c8da0 0x7f8b9415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 >> 172.21.15.60:0/2099460765 conn(0x7f8b940cf3b0 msgr2=0x7f8b940c3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 shutdown_connections 2023-07-13T22:31:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:18.854+0000 7f8b997ff700 1 -- 172.21.15.60:0/2099460765 wait complete. 2023-07-13T22:31:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:31:18.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-07-13T22:31:19.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.081+0000 7f53827ff700 1 Processor -- start 2023-07-13T22:31:19.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.082+0000 7f53827ff700 1 -- start start 2023-07-13T22:31:19.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.083+0000 7f53827ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f537c0c90e0 0x7f537c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.083+0000 7f53827ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f537c06a410 0x7f537c06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.083+0000 7f53827ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c06d030 con 0x7f537c158930 2023-07-13T22:31:19.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.083+0000 7f53827ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c06ca70 con 0x7f537c0c90e0 2023-07-13T22:31:19.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.083+0000 7f53827ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c0adc20 con 0x7f537c06a410 2023-07-13T22:31:19.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.083+0000 7f537b7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f537c0c90e0 0x7f537c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.083+0000 7f537bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f537c158930 0x7f537c15c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35436/0 (socket says 172.21.15.60:35436) 2023-07-13T22:31:19.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.083+0000 7f537bfff700 1 -- 172.21.15.60:0/2694781827 learned_addr learned my addr 172.21.15.60:0/2694781827 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:19.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.084+0000 7f5380d9c700 1 --2- 172.21.15.60:0/2694781827 >> v2:172.21.15.105:3300/0 conn(0x7f537c06a410 0x7f537c06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.084+0000 7f537b7fe700 1 -- 172.21.15.60:0/2694781827 >> v2:172.21.15.105:3300/0 conn(0x7f537c06a410 msgr2=0x7f537c06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:19.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.084+0000 7f537b7fe700 1 --2- 172.21.15.60:0/2694781827 >> v2:172.21.15.105:3300/0 conn(0x7f537c06a410 0x7f537c06d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.084+0000 7f537b7fe700 1 -- 172.21.15.60:0/2694781827 >> v1:172.21.15.60:6789/0 conn(0x7f537c158930 legacy=0x7f537c15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.084+0000 7f537b7fe700 1 -- 172.21.15.60:0/2694781827 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c06a870 con 0x7f537c0c90e0 2023-07-13T22:31:19.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.084+0000 7f537b7fe700 1 --2- 172.21.15.60:0/2694781827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c0c90e0 0x7f537c069ed0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f537c071690 tx=0x7f537000acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c00cd2cc13b1d73 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:19.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.084+0000 7f537affd700 1 -- 172.21.15.60:0/2694781827 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370004230 con 0x7f537c0c90e0 2023-07-13T22:31:19.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.084+0000 7f537affd700 1 -- 172.21.15.60:0/2694781827 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5370004390 con 0x7f537c0c90e0 2023-07-13T22:31:19.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.084+0000 7f537affd700 1 -- 172.21.15.60:0/2694781827 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537000c870 con 0x7f537c0c90e0 2023-07-13T22:31:19.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.085+0000 7f53827ff700 1 -- 172.21.15.60:0/2694781827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c0c90e0 msgr2=0x7f537c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.085+0000 7f53827ff700 1 --2- 172.21.15.60:0/2694781827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c0c90e0 0x7f537c069ed0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f537c071690 tx=0x7f537000acc0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.085+0000 7f53827ff700 1 -- 172.21.15.60:0/2694781827 shutdown_connections 2023-07-13T22:31:19.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.085+0000 7f53827ff700 1 --2- 172.21.15.60:0/2694781827 >> v2:172.21.15.105:3300/0 conn(0x7f537c06a410 0x7f537c06d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.085+0000 7f53827ff700 1 --2- 172.21.15.60:0/2694781827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c0c90e0 0x7f537c069ed0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.085+0000 7f53827ff700 1 -- 172.21.15.60:0/2694781827 >> 172.21.15.60:0/2694781827 conn(0x7f537c0cbf50 msgr2=0x7f537c0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:19.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.085+0000 7f53827ff700 1 -- 172.21.15.60:0/2694781827 shutdown_connections 2023-07-13T22:31:19.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.085+0000 7f53827ff700 1 -- 172.21.15.60:0/2694781827 wait complete. 2023-07-13T22:31:19.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.087+0000 7f53827ff700 1 Processor -- start 2023-07-13T22:31:19.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.087+0000 7f53827ff700 1 -- start start 2023-07-13T22:31:19.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f53827ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f537c06a410 0x7f537c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f53827ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f537c0c90e0 0x7f537c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f53827ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c158930 0x7f537c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f53827ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1f2740 con 0x7f537c158930 2023-07-13T22:31:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f53827ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1f2a20 con 0x7f537c06a410 2023-07-13T22:31:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f53827ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1f2d00 con 0x7f537c0c90e0 2023-07-13T22:31:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f5380d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c158930 0x7f537c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f5380d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c158930 0x7f537c1558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40196/0 (socket says 172.21.15.60:40196) 2023-07-13T22:31:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f5380d9c700 1 -- 172.21.15.60:0/695777724 learned_addr learned my addr 172.21.15.60:0/695777724 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f537bfff700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f537c06a410 0x7f537c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.088+0000 7f537b7fe700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f537c0c90e0 0x7f537c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f5380d9c700 1 -- 172.21.15.60:0/695777724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f537c0c90e0 msgr2=0x7f537c1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f5380d9c700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f537c0c90e0 0x7f537c1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f5380d9c700 1 -- 172.21.15.60:0/695777724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f537c06a410 msgr2=0x7f537c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f5380d9c700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f537c06a410 0x7f537c0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f5380d9c700 1 -- 172.21.15.60:0/695777724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537000a640 con 0x7f537c158930 2023-07-13T22:31:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f5380d9c700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c158930 0x7f537c1558d0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5374007e60 tx=0x7f537400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:19.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f5378ff9700 1 -- 172.21.15.60:0/695777724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400ed30 con 0x7f537c158930 2023-07-13T22:31:19.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f5378ff9700 1 -- 172.21.15.60:0/695777724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5374014420 con 0x7f537c158930 2023-07-13T22:31:19.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f5378ff9700 1 -- 172.21.15.60:0/695777724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53740147d0 con 0x7f537c158930 2023-07-13T22:31:19.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c1f2fe0 con 0x7f537c158930 2023-07-13T22:31:19.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.089+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c0b34b0 con 0x7f537c158930 2023-07-13T22:31:19.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.090+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c0ab8f0 con 0x7f537c158930 2023-07-13T22:31:19.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.091+0000 7f5378ff9700 1 -- 172.21.15.60:0/695777724 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5374014930 con 0x7f537c158930 2023-07-13T22:31:19.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.092+0000 7f5378ff9700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f536806ace0 0x7f536806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.092+0000 7f5378ff9700 1 -- 172.21.15.60:0/695777724 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5374089910 con 0x7f537c158930 2023-07-13T22:31:19.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.092+0000 7f537bfff700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f536806ace0 0x7f536806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.093+0000 7f537bfff700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f536806ace0 0x7f536806d190 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f536c000f60 tx=0x7f536c004a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:19.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.097+0000 7f5378ff9700 1 -- 172.21.15.60:0/695777724 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f537c0ab8f0 con 0x7f537c158930 2023-07-13T22:31:19.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.306+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7f537c0ab8f0 con 0x7f537c158930 2023-07-13T22:31:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.310+0000 7f5378ff9700 1 -- 172.21.15.60:0/695777724 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v59) v1 ==== 102+0+0 (secure 0 0 0) 0x7f537c0ab8f0 con 0x7f537c158930 2023-07-13T22:31:19.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f536806ace0 msgr2=0x7f536806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f536806ace0 0x7f536806d190 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f536c000f60 tx=0x7f536c004a50 comp rx=0 tx=0).stop 2023-07-13T22:31:19.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c158930 msgr2=0x7f537c1558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c158930 0x7f537c1558d0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5374007e60 tx=0x7f537400e3c0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 shutdown_connections 2023-07-13T22:31:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f536806ace0 0x7f536806d190 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537c158930 0x7f537c1558d0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f537c06a410 0x7f537c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 --2- 172.21.15.60:0/695777724 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f537c0c90e0 0x7f537c1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 >> 172.21.15.60:0/695777724 conn(0x7f537c0cbf50 msgr2=0x7f537c0c9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 shutdown_connections 2023-07-13T22:31:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.313+0000 7f53827ff700 1 -- 172.21.15.60:0/695777724 wait complete. 2023-07-13T22:31:19.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-13T22:31:19.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.543+0000 7effbebff700 1 Processor -- start 2023-07-13T22:31:19.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.543+0000 7effbebff700 1 -- start start 2023-07-13T22:31:19.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.544+0000 7effbebff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80cfd50 0x7effb80d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.544+0000 7effbebff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb80d1870 con 0x7effb80b1ac0 2023-07-13T22:31:19.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.544+0000 7effbebff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb80cf370 con 0x7effb80b8200 2023-07-13T22:31:19.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.544+0000 7effbebff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb80d12b0 con 0x7effb80cfd50 2023-07-13T22:31:19.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.545+0000 7effbd19c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7effb80b1ac0 0x7effb80d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35442/0 (socket says 172.21.15.60:35442) 2023-07-13T22:31:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.545+0000 7effbd19c700 1 -- 172.21.15.60:0/3027485581 learned_addr learned my addr 172.21.15.60:0/3027485581 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.545+0000 7effb7fff700 1 --2- 172.21.15.60:0/3027485581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80cfd50 0x7effb80d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.545+0000 7effb77fe700 1 -- 172.21.15.60:0/3027485581 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335597230 0 0) 0x7effb80d1870 con 0x7effb80b1ac0 2023-07-13T22:31:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.545+0000 7effb77fe700 1 -- 172.21.15.60:0/3027485581 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa80035b0 con 0x7effb80b1ac0 2023-07-13T22:31:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.545+0000 7effb77fe700 1 -- 172.21.15.60:0/3027485581 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605058159 0 0) 0x7effb80cf370 con 0x7effb80b8200 2023-07-13T22:31:19.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.545+0000 7effb77fe700 1 -- 172.21.15.60:0/3027485581 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effb80d1870 con 0x7effb80b8200 2023-07-13T22:31:19.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.546+0000 7effb77fe700 1 -- 172.21.15.60:0/3027485581 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1129560389 0 0) 0x7effa80035b0 con 0x7effb80b1ac0 2023-07-13T22:31:19.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.546+0000 7effb7fff700 1 -- 172.21.15.60:0/3027485581 >> v1:172.21.15.105:6789/0 conn(0x7effb80b8200 legacy=0x7effb80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.546+0000 7effb7fff700 1 -- 172.21.15.60:0/3027485581 >> v1:172.21.15.60:6789/0 conn(0x7effb80b1ac0 legacy=0x7effb80d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.546+0000 7effb7fff700 1 -- 172.21.15.60:0/3027485581 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb80d1590 con 0x7effb80cfd50 2023-07-13T22:31:19.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.546+0000 7effb7fff700 1 --2- 172.21.15.60:0/3027485581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80cfd50 0x7effb80d23c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7effa40098a0 tx=0x7effa4009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3800ca7917d72eb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:19.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.546+0000 7effb77fe700 1 -- 172.21.15.60:0/3027485581 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4011830 con 0x7effb80cfd50 2023-07-13T22:31:19.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.546+0000 7effb77fe700 1 -- 172.21.15.60:0/3027485581 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa4011990 con 0x7effb80cfd50 2023-07-13T22:31:19.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.546+0000 7effb77fe700 1 -- 172.21.15.60:0/3027485581 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4011d20 con 0x7effb80cfd50 2023-07-13T22:31:19.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.547+0000 7effbebff700 1 -- 172.21.15.60:0/3027485581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80cfd50 msgr2=0x7effb80d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.547+0000 7effbebff700 1 --2- 172.21.15.60:0/3027485581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80cfd50 0x7effb80d23c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7effa40098a0 tx=0x7effa4009bb0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.547+0000 7effbebff700 1 -- 172.21.15.60:0/3027485581 shutdown_connections 2023-07-13T22:31:19.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.547+0000 7effbebff700 1 --2- 172.21.15.60:0/3027485581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80cfd50 0x7effb80d23c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.547+0000 7effbebff700 1 -- 172.21.15.60:0/3027485581 >> 172.21.15.60:0/3027485581 conn(0x7effb806b2f0 msgr2=0x7effb806b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:19.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.547+0000 7effbebff700 1 -- 172.21.15.60:0/3027485581 shutdown_connections 2023-07-13T22:31:19.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.547+0000 7effbebff700 1 -- 172.21.15.60:0/3027485581 wait complete. 2023-07-13T22:31:19.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.549+0000 7effbebff700 1 Processor -- start 2023-07-13T22:31:19.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.549+0000 7effbebff700 1 -- start start 2023-07-13T22:31:19.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbebff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80b1ac0 0x7effb815af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbebff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb80b8200 0x7effb80c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbebff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb80cfd50 0x7effb815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbebff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb815ad40 con 0x7effb80cfd50 2023-07-13T22:31:19.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbebff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb815b7b0 con 0x7effb80b8200 2023-07-13T22:31:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbebff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb80ca400 con 0x7effb80b1ac0 2023-07-13T22:31:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbd19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb80cfd50 0x7effb815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbd19c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb80cfd50 0x7effb815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40204/0 (socket says 172.21.15.60:40204) 2023-07-13T22:31:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbd19c700 1 -- 172.21.15.60:0/1204016400 learned_addr learned my addr 172.21.15.60:0/1204016400 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbc99b700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80b1ac0 0x7effb815af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effb7fff700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb80b8200 0x7effb80c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbd19c700 1 -- 172.21.15.60:0/1204016400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80b1ac0 msgr2=0x7effb815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbd19c700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80b1ac0 0x7effb815af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbd19c700 1 -- 172.21.15.60:0/1204016400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb80b8200 msgr2=0x7effb80c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbd19c700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb80b8200 0x7effb80c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.550+0000 7effbd19c700 1 -- 172.21.15.60:0/1204016400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa40094a0 con 0x7effb80cfd50 2023-07-13T22:31:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.551+0000 7effbd19c700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb80cfd50 0x7effb815a800 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7effb000cb10 tx=0x7effb000ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.551+0000 7effb57fa700 1 -- 172.21.15.60:0/1204016400 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb000a9e0 con 0x7effb80cfd50 2023-07-13T22:31:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.551+0000 7effb57fa700 1 -- 172.21.15.60:0/1204016400 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb000ab60 con 0x7effb80cfd50 2023-07-13T22:31:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.551+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb80ca6e0 con 0x7effb80cfd50 2023-07-13T22:31:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.551+0000 7effb57fa700 1 -- 172.21.15.60:0/1204016400 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0012400 con 0x7effb80cfd50 2023-07-13T22:31:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.551+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb80caca0 con 0x7effb80cfd50 2023-07-13T22:31:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.552+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb804e210 con 0x7effb80cfd50 2023-07-13T22:31:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.553+0000 7effb57fa700 1 -- 172.21.15.60:0/1204016400 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effb0012560 con 0x7effb80cfd50 2023-07-13T22:31:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.554+0000 7effb57fa700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9c06acb0 0x7eff9c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.554+0000 7effb57fa700 1 -- 172.21.15.60:0/1204016400 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7effb00893e0 con 0x7effb80cfd50 2023-07-13T22:31:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.555+0000 7effbc99b700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9c06acb0 0x7eff9c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.555+0000 7effbc99b700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9c06acb0 0x7eff9c06d160 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7effac0011e0 tx=0x7effac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.559+0000 7effb57fa700 1 -- 172.21.15.60:0/1204016400 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7effb00547a0 con 0x7effb80cfd50 2023-07-13T22:31:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.768+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7effb80b2840 con 0x7effb80cfd50 2023-07-13T22:31:19.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.773+0000 7effb57fa700 1 -- 172.21.15.60:0/1204016400 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7effb0057dc0 con 0x7effb80cfd50 2023-07-13T22:31:19.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9c06acb0 msgr2=0x7eff9c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9c06acb0 0x7eff9c06d160 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7effac0011e0 tx=0x7effac009040 comp rx=0 tx=0).stop 2023-07-13T22:31:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb80cfd50 msgr2=0x7effb815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb80cfd50 0x7effb815a800 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7effb000cb10 tx=0x7effb000ce20 comp rx=0 tx=0).stop 2023-07-13T22:31:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 shutdown_connections 2023-07-13T22:31:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9c06acb0 0x7eff9c06d160 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb80cfd50 0x7effb815a800 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effb80b1ac0 0x7effb815af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 --2- 172.21.15.60:0/1204016400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effb80b8200 0x7effb80c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.775+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 >> 172.21.15.60:0/1204016400 conn(0x7effb806b2f0 msgr2=0x7effb80b7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:19.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.776+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 shutdown_connections 2023-07-13T22:31:19.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:19.776+0000 7effbebff700 1 -- 172.21.15.60:0/1204016400 wait complete. 2023-07-13T22:31:19.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-07-13T22:31:19.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.001+0000 7f66f8dff700 1 Processor -- start 2023-07-13T22:31:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.002+0000 7f66f8dff700 1 -- start start 2023-07-13T22:31:19.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.003+0000 7f66f8dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f40add00 0x7f66f406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.003+0000 7f66f8dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f66f406a5e0 0x7f66f40b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.003+0000 7f66f8dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f40c8e90 con 0x7f66f40ca9f0 2023-07-13T22:31:19.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.003+0000 7f66f8dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f406d3b0 con 0x7f66f406a5e0 2023-07-13T22:31:19.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.003+0000 7f66f8dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40ae0d0 con 0x7f66f40add00 2023-07-13T22:31:19.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.003+0000 7f66f2d9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f66f406a5e0 0x7f66f40b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.003+0000 7f66f259c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f40add00 0x7f66f406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f2d9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f66f406a5e0 0x7f66f40b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:49876/0 (socket says 172.21.15.60:49876) 2023-07-13T22:31:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.003+0000 7f66ebfff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f66f40ca9f0 0x7f66f4069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:54854/0 (socket says 172.21.15.60:54854) 2023-07-13T22:31:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f2d9d700 1 -- 172.21.15.60:0/1529100341 learned_addr learned my addr 172.21.15.60:0/1529100341 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f259c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f40add00 0x7f66f406ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:33384/0 (socket says 172.21.15.60:33384) 2023-07-13T22:31:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f259c700 1 -- 172.21.15.60:0/1529100341 >> v2:172.21.15.105:3300/0 conn(0x7f66f406a5e0 msgr2=0x7f66f40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f259c700 1 --2- 172.21.15.60:0/1529100341 >> v2:172.21.15.105:3300/0 conn(0x7f66f406a5e0 0x7f66f40b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f259c700 1 -- 172.21.15.60:0/1529100341 >> v1:172.21.15.105:6789/0 conn(0x7f66f40ca9f0 legacy=0x7f66f4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f259c700 1 -- 172.21.15.60:0/1529100341 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f40c90e0 con 0x7f66f40add00 2023-07-13T22:31:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f1d9b700 1 -- 172.21.15.60:0/1529100341 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210777602 0 0) 0x7f66f40c8e90 con 0x7f66f40ca9f0 2023-07-13T22:31:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f2d9d700 1 --2- 172.21.15.60:0/1529100341 >> v2:172.21.15.105:3300/0 conn(0x7f66f406a5e0 0x7f66f40b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.004+0000 7f66f259c700 1 --2- 172.21.15.60:0/1529100341 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f40add00 0x7f66f406ce70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009840 tx=0x7f66e4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7546d95827b49bfa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:19.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f1d9b700 1 -- 172.21.15.60:0/1529100341 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011830 con 0x7f66f40add00 2023-07-13T22:31:19.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f1d9b700 1 -- 172.21.15.60:0/1529100341 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4011990 con 0x7f66f40add00 2023-07-13T22:31:19.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f1d9b700 1 -- 172.21.15.60:0/1529100341 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011d20 con 0x7f66f40add00 2023-07-13T22:31:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f8dff700 1 -- 172.21.15.60:0/1529100341 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f40add00 msgr2=0x7f66f406ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f8dff700 1 --2- 172.21.15.60:0/1529100341 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f40add00 0x7f66f406ce70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009840 tx=0x7f66e4009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:19.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f8dff700 1 -- 172.21.15.60:0/1529100341 shutdown_connections 2023-07-13T22:31:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f8dff700 1 --2- 172.21.15.60:0/1529100341 >> v2:172.21.15.105:3300/0 conn(0x7f66f406a5e0 0x7f66f40b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f8dff700 1 --2- 172.21.15.60:0/1529100341 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f40add00 0x7f66f406ce70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f8dff700 1 -- 172.21.15.60:0/1529100341 >> 172.21.15.60:0/1529100341 conn(0x7f66f40cbf50 msgr2=0x7f66f40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f8dff700 1 -- 172.21.15.60:0/1529100341 shutdown_connections 2023-07-13T22:31:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.005+0000 7f66f8dff700 1 -- 172.21.15.60:0/1529100341 wait complete. 2023-07-13T22:31:19.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.007+0000 7f66f8dff700 1 Processor -- start 2023-07-13T22:31:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.007+0000 7f66f8dff700 1 -- start start 2023-07-13T22:31:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.008+0000 7f66f8dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f406a5e0 0x7f66f41f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.008+0000 7f66f8dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66f40add00 0x7f66f40b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.008+0000 7f66f8dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66f40ca9f0 0x7f66f40aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.008+0000 7f66f8dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40b1060 con 0x7f66f40ca9f0 2023-07-13T22:31:19.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.008+0000 7f66f8dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40c5bb0 con 0x7f66f40add00 2023-07-13T22:31:19.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.008+0000 7f66f8dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40cc540 con 0x7f66f406a5e0 2023-07-13T22:31:19.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f259c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f406a5e0 0x7f66f41f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66ebfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66f40add00 0x7f66f40b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f259c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f406a5e0 0x7f66f41f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:33390/0 (socket says 172.21.15.60:33390) 2023-07-13T22:31:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f259c700 1 -- 172.21.15.60:0/1112786591 learned_addr learned my addr 172.21.15.60:0/1112786591 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f2d9d700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66f40ca9f0 0x7f66f40aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f259c700 1 -- 172.21.15.60:0/1112786591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66f40add00 msgr2=0x7f66f40b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f259c700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66f40add00 0x7f66f40b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f259c700 1 -- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66f40ca9f0 msgr2=0x7f66f40aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f259c700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66f40ca9f0 0x7f66f40aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f259c700 1 -- 172.21.15.60:0/1112786591 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec009b30 con 0x7f66f406a5e0 2023-07-13T22:31:19.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.009+0000 7f66f259c700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f406a5e0 0x7f66f41f2c90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009fd0 tx=0x7f66e4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:19.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.010+0000 7f66eaffd700 1 -- 172.21.15.60:0/1112786591 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e40036a0 con 0x7f66f406a5e0 2023-07-13T22:31:19.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.010+0000 7f66eaffd700 1 -- 172.21.15.60:0/1112786591 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e401f070 con 0x7f66f406a5e0 2023-07-13T22:31:19.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.010+0000 7f66eaffd700 1 -- 172.21.15.60:0/1112786591 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4003a60 con 0x7f66f406a5e0 2023-07-13T22:31:19.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.010+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e40094a0 con 0x7f66f406a5e0 2023-07-13T22:31:19.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.010+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f40ccb00 con 0x7f66f406a5e0 2023-07-13T22:31:19.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.011+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f415ce20 con 0x7f66f406a5e0 2023-07-13T22:31:19.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.012+0000 7f66eaffd700 1 -- 172.21.15.60:0/1112786591 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66e4003bc0 con 0x7f66f406a5e0 2023-07-13T22:31:19.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.013+0000 7f66eaffd700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66d406ace0 0x7f66d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:19.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.013+0000 7f66eaffd700 1 -- 172.21.15.60:0/1112786591 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f66e4021070 con 0x7f66f406a5e0 2023-07-13T22:31:19.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.013+0000 7f66ebfff700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66d406ace0 0x7f66d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:19.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.014+0000 7f66ebfff700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66d406ace0 0x7f66d406d190 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f66dc007fd0 tx=0x7f66dc007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:19.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.018+0000 7f66eaffd700 1 -- 172.21.15.60:0/1112786591 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f66e4055d20 con 0x7f66f406a5e0 2023-07-13T22:31:20.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.224+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7f66f404e210 con 0x7f66f406a5e0 2023-07-13T22:31:20.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.230+0000 7f66eaffd700 1 -- 172.21.15.60:0/1112786591 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v61) v1 ==== 105+0+0 (secure 0 0 0) 0x7f66e4059340 con 0x7f66f406a5e0 2023-07-13T22:31:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66d406ace0 msgr2=0x7f66d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66d406ace0 0x7f66d406d190 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f66dc007fd0 tx=0x7f66dc007de0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f406a5e0 msgr2=0x7f66f41f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f406a5e0 0x7f66f41f2c90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009fd0 tx=0x7f66e4004750 comp rx=0 tx=0).stop 2023-07-13T22:31:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 shutdown_connections 2023-07-13T22:31:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f66d406ace0 0x7f66d406d190 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66f40ca9f0 0x7f66f40aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66f406a5e0 0x7f66f41f2c90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 --2- 172.21.15.60:0/1112786591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66f40add00 0x7f66f40b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 >> 172.21.15.60:0/1112786591 conn(0x7f66f40cbf50 msgr2=0x7f66f415b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 shutdown_connections 2023-07-13T22:31:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.233+0000 7f66f8dff700 1 -- 172.21.15.60:0/1112786591 wait complete. 2023-07-13T22:31:20.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-07-13T22:31:20.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.458+0000 7fa64a5ff700 1 Processor -- start 2023-07-13T22:31:20.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.458+0000 7fa64a5ff700 1 -- start start 2023-07-13T22:31:20.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.459+0000 7fa64a5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6440cfd50 0x7fa6440d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.459+0000 7fa64a5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b1ac0 0x7fa6440d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.459+0000 7fa64a5ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6440d1870 con 0x7fa6440b8200 2023-07-13T22:31:20.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.459+0000 7fa64a5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6440cf370 con 0x7fa6440cfd50 2023-07-13T22:31:20.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.459+0000 7fa64a5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440d12b0 con 0x7fa6440b1ac0 2023-07-13T22:31:20.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa643fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa6440b8200 0x7fa6440cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35454/0 (socket says 172.21.15.60:35454) 2023-07-13T22:31:20.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa643fff700 1 -- 172.21.15.60:0/1215687027 learned_addr learned my addr 172.21.15.60:0/1215687027 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:20.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa6437fe700 1 --2- 172.21.15.60:0/1215687027 >> v2:172.21.15.105:3300/0 conn(0x7fa6440cfd50 0x7fa6440d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa648b9c700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b1ac0 0x7fa6440d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193278044 0 0) 0x7fa6440d1870 con 0x7fa6440b8200 2023-07-13T22:31:20.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa62c0035b0 con 0x7fa6440b8200 2023-07-13T22:31:20.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 383960970 0 0) 0x7fa62c0035b0 con 0x7fa6440b8200 2023-07-13T22:31:20.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b1ac0 msgr2=0x7fa6440d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:20.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa642ffd700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b1ac0 0x7fa6440d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 >> v2:172.21.15.105:3300/0 conn(0x7fa6440cfd50 msgr2=0x7fa6440d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:20.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa642ffd700 1 --2- 172.21.15.60:0/1215687027 >> v2:172.21.15.105:3300/0 conn(0x7fa6440cfd50 0x7fa6440d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440d1590 con 0x7fa6440b8200 2023-07-13T22:31:20.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.460+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fa634001670 con 0x7fa6440b8200 2023-07-13T22:31:20.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 >> v1:172.21.15.60:6789/0 conn(0x7fa6440b8200 legacy=0x7fa6440cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa642ffd700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c003a00 0x7fa62c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa642ffd700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b1ac0 0x7fa62c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa642ffd700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa62c004d90 0x7fa62c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440cf370 con 0x7fa62c004d90 2023-07-13T22:31:20.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440d12b0 con 0x7fa62c003a00 2023-07-13T22:31:20.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c00ba10 con 0x7fa6440b1ac0 2023-07-13T22:31:20.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa648b9c700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa62c004d90 0x7fa62c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa6437fe700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b1ac0 0x7fa62c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa643fff700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c003a00 0x7fa62c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa648b9c700 1 -- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b1ac0 msgr2=0x7fa62c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:20.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa648b9c700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b1ac0 0x7fa62c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa648b9c700 1 -- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c003a00 msgr2=0x7fa62c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:20.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa648b9c700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c003a00 0x7fa62c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.461+0000 7fa648b9c700 1 -- 172.21.15.60:0/1215687027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c0035b0 con 0x7fa62c004d90 2023-07-13T22:31:20.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa648b9c700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa62c004d90 0x7fa62c005200 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fa6440b1ee0 tx=0x7fa62c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8f65512c4cc081 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:20.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00eb70 con 0x7fa62c004d90 2023-07-13T22:31:20.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c00f040 con 0x7fa62c004d90 2023-07-13T22:31:20.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa642ffd700 1 -- 172.21.15.60:0/1215687027 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c014480 con 0x7fa62c004d90 2023-07-13T22:31:20.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa64a5ff700 1 -- 172.21.15.60:0/1215687027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa62c004d90 msgr2=0x7fa62c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa64a5ff700 1 --2- 172.21.15.60:0/1215687027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa62c004d90 0x7fa62c005200 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fa6440b1ee0 tx=0x7fa62c005880 comp rx=0 tx=0).stop 2023-07-13T22:31:20.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa648b9c700 1 -- 172.21.15.60:0/1215687027 reap_dead start 2023-07-13T22:31:20.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa64a5ff700 1 -- 172.21.15.60:0/1215687027 shutdown_connections 2023-07-13T22:31:20.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa64a5ff700 1 -- 172.21.15.60:0/1215687027 >> 172.21.15.60:0/1215687027 conn(0x7fa64406b2f0 msgr2=0x7fa64406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:20.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.462+0000 7fa64a5ff700 1 -- 172.21.15.60:0/1215687027 shutdown_connections 2023-07-13T22:31:20.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.463+0000 7fa64a5ff700 1 -- 172.21.15.60:0/1215687027 wait complete. 2023-07-13T22:31:20.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.464+0000 7fa64a5ff700 1 Processor -- start 2023-07-13T22:31:20.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.465+0000 7fa64a5ff700 1 -- start start 2023-07-13T22:31:20.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.465+0000 7fa64a5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c004d90 0x7fa644157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.465+0000 7fa64a5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644155d20 0x7fa6441553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.465+0000 7fa64a5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b6ab0 0x7fa6440b6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.465+0000 7fa64a5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440c81e0 con 0x7fa644155d20 2023-07-13T22:31:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa64a5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b7490 con 0x7fa62c004d90 2023-07-13T22:31:20.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa64a5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa644157c30 con 0x7fa6440b6ab0 2023-07-13T22:31:20.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa6437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644155d20 0x7fa6441553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa6437fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644155d20 0x7fa6441553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40238/0 (socket says 172.21.15.60:40238) 2023-07-13T22:31:20.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa6437fe700 1 -- 172.21.15.60:0/2032873407 learned_addr learned my addr 172.21.15.60:0/2032873407 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:20.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa648b9c700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b6ab0 0x7fa6440b6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa643fff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c004d90 0x7fa644157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa6437fe700 1 -- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b6ab0 msgr2=0x7fa6440b6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa6437fe700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b6ab0 0x7fa6440b6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa6437fe700 1 -- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c004d90 msgr2=0x7fa644157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa6437fe700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c004d90 0x7fa644157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa6437fe700 1 -- 172.21.15.60:0/2032873407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440c7b20 con 0x7fa644155d20 2023-07-13T22:31:20.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa648b9c700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b6ab0 0x7fa6440b6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:20.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa643fff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c004d90 0x7fa644157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:20.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.466+0000 7fa6437fe700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644155d20 0x7fa6441553f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fa638011a30 tx=0x7fa638011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:20.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.467+0000 7fa640ff9700 1 -- 172.21.15.60:0/2032873407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638004240 con 0x7fa644155d20 2023-07-13T22:31:20.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.467+0000 7fa640ff9700 1 -- 172.21.15.60:0/2032873407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6380043a0 con 0x7fa644155d20 2023-07-13T22:31:20.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.467+0000 7fa640ff9700 1 -- 172.21.15.60:0/2032873407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63800f7f0 con 0x7fa644155d20 2023-07-13T22:31:20.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.467+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6440c7e00 con 0x7fa644155d20 2023-07-13T22:31:20.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.467+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa644157870 con 0x7fa644155d20 2023-07-13T22:31:20.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.468+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6440c8320 con 0x7fa644155d20 2023-07-13T22:31:20.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.469+0000 7fa640ff9700 1 -- 172.21.15.60:0/2032873407 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa63800f950 con 0x7fa644155d20 2023-07-13T22:31:20.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.470+0000 7fa640ff9700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63006ace0 0x7fa63006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.471+0000 7fa640ff9700 1 -- 172.21.15.60:0/2032873407 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa638089790 con 0x7fa644155d20 2023-07-13T22:31:20.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.471+0000 7fa643fff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63006ace0 0x7fa63006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.471+0000 7fa643fff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63006ace0 0x7fa63006d190 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa634000c00 tx=0x7fa634009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:20.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.475+0000 7fa640ff9700 1 -- 172.21.15.60:0/2032873407 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa6440c8320 con 0x7fa644155d20 2023-07-13T22:31:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.681+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7fa6440c8320 con 0x7fa644155d20 2023-07-13T22:31:20.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.685+0000 7fa640ff9700 1 -- 172.21.15.60:0/2032873407 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v62) v1 ==== 111+0+0 (secure 0 0 0) 0x7fa6440c8320 con 0x7fa644155d20 2023-07-13T22:31:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63006ace0 msgr2=0x7fa63006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63006ace0 0x7fa63006d190 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa634000c00 tx=0x7fa634009040 comp rx=0 tx=0).stop 2023-07-13T22:31:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644155d20 msgr2=0x7fa6441553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644155d20 0x7fa6441553f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fa638011a30 tx=0x7fa638011df0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 shutdown_connections 2023-07-13T22:31:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa63006ace0 0x7fa63006d190 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6440b6ab0 0x7fa6440b6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa62c004d90 0x7fa644157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 --2- 172.21.15.60:0/2032873407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa644155d20 0x7fa6441553f0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.687+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 >> 172.21.15.60:0/2032873407 conn(0x7fa64406b2f0 msgr2=0x7fa6440b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.688+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 shutdown_connections 2023-07-13T22:31:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.688+0000 7fa64a5ff700 1 -- 172.21.15.60:0/2032873407 wait complete. 2023-07-13T22:31:20.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-07-13T22:31:20.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.912+0000 7f5fb03ff700 1 Processor -- start 2023-07-13T22:31:20.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.912+0000 7f5fb03ff700 1 -- start start 2023-07-13T22:31:20.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.913+0000 7f5fb03ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5fa80add00 0x7f5fa806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.913+0000 7f5fb03ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80ca9f0 0x7f5fa8069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.913+0000 7f5fb03ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa80c8e90 con 0x7f5fa806a410 2023-07-13T22:31:20.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.913+0000 7f5fb03ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa806d3b0 con 0x7f5fa80add00 2023-07-13T22:31:20.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.913+0000 7f5fb03ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa806a890 con 0x7f5fa80ca9f0 2023-07-13T22:31:20.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.914+0000 7f5fad99a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80ca9f0 0x7f5fa8069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.914+0000 7f5fae19b700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5fa80add00 0x7f5fa806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.914+0000 7f5fae99c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5fa806a410 0x7f5fa80703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:54860/0 (socket says 172.21.15.60:54860) 2023-07-13T22:31:20.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.914+0000 7f5fae99c700 1 -- 172.21.15.60:0/1130230655 learned_addr learned my addr 172.21.15.60:0/1130230655 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.914+0000 7f5fae19b700 1 -- 172.21.15.60:0/1130230655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80ca9f0 msgr2=0x7f5fa8069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.914+0000 7f5fae19b700 1 --2- 172.21.15.60:0/1130230655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80ca9f0 0x7f5fa8069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.914+0000 7f5fad199700 1 -- 172.21.15.60:0/1130230655 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624616153 0 0) 0x7f5fa80c8e90 con 0x7f5fa806a410 2023-07-13T22:31:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.914+0000 7f5fae19b700 1 -- 172.21.15.60:0/1130230655 >> v1:172.21.15.105:6789/0 conn(0x7f5fa806a410 legacy=0x7f5fa80703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.914+0000 7f5fae19b700 1 -- 172.21.15.60:0/1130230655 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80ae0d0 con 0x7f5fa80add00 2023-07-13T22:31:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.915+0000 7f5fad99a700 1 --2- 172.21.15.60:0/1130230655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80ca9f0 0x7f5fa8069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.915+0000 7f5fae19b700 1 --2- 172.21.15.60:0/1130230655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80add00 0x7f5fa806ce70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009840 tx=0x7f5fa4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2943c2939a608fd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:20.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.915+0000 7f5fad199700 1 -- 172.21.15.60:0/1130230655 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4011830 con 0x7f5fa80add00 2023-07-13T22:31:20.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.915+0000 7f5fad199700 1 -- 172.21.15.60:0/1130230655 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa4011990 con 0x7f5fa80add00 2023-07-13T22:31:20.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.915+0000 7f5fad199700 1 -- 172.21.15.60:0/1130230655 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4011d20 con 0x7f5fa80add00 2023-07-13T22:31:20.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.916+0000 7f5fb03ff700 1 -- 172.21.15.60:0/1130230655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80add00 msgr2=0x7f5fa806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.916+0000 7f5fb03ff700 1 --2- 172.21.15.60:0/1130230655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80add00 0x7f5fa806ce70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009840 tx=0x7f5fa4009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:20.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.916+0000 7f5fb03ff700 1 -- 172.21.15.60:0/1130230655 shutdown_connections 2023-07-13T22:31:20.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.916+0000 7f5fb03ff700 1 --2- 172.21.15.60:0/1130230655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80add00 0x7f5fa806ce70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.916+0000 7f5fb03ff700 1 --2- 172.21.15.60:0/1130230655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80ca9f0 0x7f5fa8069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.916+0000 7f5fb03ff700 1 -- 172.21.15.60:0/1130230655 >> 172.21.15.60:0/1130230655 conn(0x7f5fa80cbf50 msgr2=0x7f5fa80ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:20.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.916+0000 7f5fb03ff700 1 -- 172.21.15.60:0/1130230655 shutdown_connections 2023-07-13T22:31:20.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.916+0000 7f5fb03ff700 1 -- 172.21.15.60:0/1130230655 wait complete. 2023-07-13T22:31:20.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.918+0000 7f5fb03ff700 1 Processor -- start 2023-07-13T22:31:20.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.919+0000 7f5fb03ff700 1 -- start start 2023-07-13T22:31:20.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fb03ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa806a410 0x7f5fa80ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fb03ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa80add00 0x7f5fa814aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fb03ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80ca9f0 0x7f5fa80c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fb03ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80c6280 con 0x7f5fa80add00 2023-07-13T22:31:20.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fb03ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81573f0 con 0x7f5fa80ca9f0 2023-07-13T22:31:20.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fb03ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80c6490 con 0x7f5fa806a410 2023-07-13T22:31:20.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fad99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa80add00 0x7f5fa814aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fad99a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa80add00 0x7f5fa814aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40254/0 (socket says 172.21.15.60:40254) 2023-07-13T22:31:20.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fad99a700 1 -- 172.21.15.60:0/2011636383 learned_addr learned my addr 172.21.15.60:0/2011636383 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:20.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fae19b700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa806a410 0x7f5fa80ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.920+0000 7f5fae99c700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80ca9f0 0x7f5fa80c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5fad99a700 1 -- 172.21.15.60:0/2011636383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa806a410 msgr2=0x7f5fa80ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5fad99a700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa806a410 0x7f5fa80ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5fad99a700 1 -- 172.21.15.60:0/2011636383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80ca9f0 msgr2=0x7f5fa80c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:20.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5fad99a700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80ca9f0 0x7f5fa80c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:20.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5fad99a700 1 -- 172.21.15.60:0/2011636383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f980094a0 con 0x7f5fa80add00 2023-07-13T22:31:20.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5fad99a700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa80add00 0x7f5fa814aae0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800ebe0 tx=0x7f5f9800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:20.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5f9effd700 1 -- 172.21.15.60:0/2011636383 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98009760 con 0x7f5fa80add00 2023-07-13T22:31:20.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5f9effd700 1 -- 172.21.15.60:0/2011636383 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9800c850 con 0x7f5fa80add00 2023-07-13T22:31:20.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa40094a0 con 0x7f5fa80add00 2023-07-13T22:31:20.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5f9effd700 1 -- 172.21.15.60:0/2011636383 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9800cbe0 con 0x7f5fa80add00 2023-07-13T22:31:20.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.921+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa80ccf50 con 0x7f5fa80add00 2023-07-13T22:31:20.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.922+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa806aab0 con 0x7f5fa80add00 2023-07-13T22:31:20.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.923+0000 7f5f9effd700 1 -- 172.21.15.60:0/2011636383 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f9800cd40 con 0x7f5fa80add00 2023-07-13T22:31:20.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.924+0000 7f5f9effd700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f8c06ace0 0x7f5f8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:20.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.924+0000 7f5f9effd700 1 -- 172.21.15.60:0/2011636383 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5f980891b0 con 0x7f5fa80add00 2023-07-13T22:31:20.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.924+0000 7f5fae19b700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f8c06ace0 0x7f5f8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:20.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.925+0000 7f5fae19b700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f8c06ace0 0x7f5f8c06d190 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40090b0 tx=0x7f5fa4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:20.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:20.928+0000 7f5f9effd700 1 -- 172.21.15.60:0/2011636383 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5f98054570 con 0x7f5fa80add00 2023-07-13T22:31:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.138+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7f5fa804e210 con 0x7f5fa80add00 2023-07-13T22:31:21.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.142+0000 7f5f9effd700 1 -- 172.21.15.60:0/2011636383 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v63) v1 ==== 107+0+0 (secure 0 0 0) 0x7f5f98057b90 con 0x7f5fa80add00 2023-07-13T22:31:21.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f8c06ace0 msgr2=0x7f5f8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:21.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f8c06ace0 0x7f5f8c06d190 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40090b0 tx=0x7f5fa4008030 comp rx=0 tx=0).stop 2023-07-13T22:31:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa80add00 msgr2=0x7f5fa814aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa80add00 0x7f5fa814aae0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800ebe0 tx=0x7f5f9800eef0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 shutdown_connections 2023-07-13T22:31:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5f8c06ace0 0x7f5f8c06d190 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80ca9f0 0x7f5fa80c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa806a410 0x7f5fa80ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 --2- 172.21.15.60:0/2011636383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fa80add00 0x7f5fa814aae0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.144+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 >> 172.21.15.60:0/2011636383 conn(0x7f5fa80cbf50 msgr2=0x7f5fa80b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:21.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.145+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 shutdown_connections 2023-07-13T22:31:21.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.145+0000 7f5fb03ff700 1 -- 172.21.15.60:0/2011636383 wait complete. 2023-07-13T22:31:21.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2023-07-13T22:31:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.369+0000 7f8739bff700 1 Processor -- start 2023-07-13T22:31:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.369+0000 7f8739bff700 1 -- start start 2023-07-13T22:31:21.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f8739bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87340cd260 0x7f87340d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:21.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f8739bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340d0e20 0x7f87340d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:21.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f8739bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87340d2410 con 0x7f87340d0a50 2023-07-13T22:31:21.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f8739bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873415c900 con 0x7f87340cd260 2023-07-13T22:31:21.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f8739bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87340d2660 con 0x7f87340d0e20 2023-07-13T22:31:21.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f8733fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340d0e20 0x7f87340d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:21.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f8732ffd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87340cd260 0x7f87340d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:21.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f8733fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340d0e20 0x7f87340d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:33448/0 (socket says 172.21.15.60:33448) 2023-07-13T22:31:21.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f8733fff700 1 -- 172.21.15.60:0/3370003029 learned_addr learned my addr 172.21.15.60:0/3370003029 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:21.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.371+0000 7f87337fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f87340d0a50 0x7f87340cf8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:54870/0 (socket says 172.21.15.60:54870) 2023-07-13T22:31:21.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.372+0000 7f8733fff700 1 -- 172.21.15.60:0/3370003029 >> v2:172.21.15.105:3300/0 conn(0x7f87340cd260 msgr2=0x7f87340d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:21.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.372+0000 7f8733fff700 1 --2- 172.21.15.60:0/3370003029 >> v2:172.21.15.105:3300/0 conn(0x7f87340cd260 0x7f87340d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.372+0000 7f8733fff700 1 -- 172.21.15.60:0/3370003029 >> v1:172.21.15.105:6789/0 conn(0x7f87340d0a50 legacy=0x7f87340cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:21.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.372+0000 7f8733fff700 1 -- 172.21.15.60:0/3370003029 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87340d2940 con 0x7f87340d0e20 2023-07-13T22:31:21.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.372+0000 7f8732ffd700 1 --2- 172.21.15.60:0/3370003029 >> v2:172.21.15.105:3300/0 conn(0x7f87340cd260 0x7f87340d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:21.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.372+0000 7f8733fff700 1 --2- 172.21.15.60:0/3370003029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340d0e20 0x7f87340d1280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f872c009e50 tx=0x7f872c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc4e6283cb255ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:21.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.372+0000 7f87327fc700 1 -- 172.21.15.60:0/3370003029 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00e070 con 0x7f87340d0e20 2023-07-13T22:31:21.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.372+0000 7f87327fc700 1 -- 172.21.15.60:0/3370003029 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00b910 con 0x7f87340d0e20 2023-07-13T22:31:21.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.372+0000 7f87327fc700 1 -- 172.21.15.60:0/3370003029 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00bcc0 con 0x7f87340d0e20 2023-07-13T22:31:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.373+0000 7f8739bff700 1 -- 172.21.15.60:0/3370003029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340d0e20 msgr2=0x7f87340d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.373+0000 7f8739bff700 1 --2- 172.21.15.60:0/3370003029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340d0e20 0x7f87340d1280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f872c009e50 tx=0x7f872c00b330 comp rx=0 tx=0).stop 2023-07-13T22:31:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.373+0000 7f8739bff700 1 -- 172.21.15.60:0/3370003029 shutdown_connections 2023-07-13T22:31:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.373+0000 7f8739bff700 1 --2- 172.21.15.60:0/3370003029 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340d0e20 0x7f87340d1280 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.373+0000 7f8739bff700 1 --2- 172.21.15.60:0/3370003029 >> v2:172.21.15.105:3300/0 conn(0x7f87340cd260 0x7f87340d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.373+0000 7f8739bff700 1 -- 172.21.15.60:0/3370003029 >> 172.21.15.60:0/3370003029 conn(0x7f8734071600 msgr2=0x7f87340719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.373+0000 7f8739bff700 1 -- 172.21.15.60:0/3370003029 shutdown_connections 2023-07-13T22:31:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.373+0000 7f8739bff700 1 -- 172.21.15.60:0/3370003029 wait complete. 2023-07-13T22:31:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.375+0000 7f8739bff700 1 Processor -- start 2023-07-13T22:31:21.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.376+0000 7f8739bff700 1 -- start start 2023-07-13T22:31:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8739bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340cd260 0x7f8734153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8739bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87340d0a50 0x7f87340b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:21.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8739bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87340d0e20 0x7f87340b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8739bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734153d40 con 0x7f87340d0e20 2023-07-13T22:31:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8739bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734154020 con 0x7f87340d0a50 2023-07-13T22:31:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8739bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8734154300 con 0x7f87340cd260 2023-07-13T22:31:21.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8733fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87340d0e20 0x7f87340b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:21.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8733fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87340d0e20 0x7f87340b7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40256/0 (socket says 172.21.15.60:40256) 2023-07-13T22:31:21.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8733fff700 1 -- 172.21.15.60:0/4294595589 learned_addr learned my addr 172.21.15.60:0/4294595589 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:21.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f8732ffd700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87340d0a50 0x7f87340b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:21.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.377+0000 7f87337fe700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340cd260 0x7f8734153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:21.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f8733fff700 1 -- 172.21.15.60:0/4294595589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340cd260 msgr2=0x7f8734153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:21.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f8733fff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340cd260 0x7f8734153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f8733fff700 1 -- 172.21.15.60:0/4294595589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87340d0a50 msgr2=0x7f87340b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:21.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f8733fff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87340d0a50 0x7f87340b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f8733fff700 1 -- 172.21.15.60:0/4294595589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280094a0 con 0x7f87340d0e20 2023-07-13T22:31:21.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f8733fff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87340d0e20 0x7f87340b7060 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f872c012000 tx=0x7f872c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:21.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f871bfff700 1 -- 172.21.15.60:0/4294595589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c004800 con 0x7f87340d0e20 2023-07-13T22:31:21.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f871bfff700 1 -- 172.21.15.60:0/4294595589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c004960 con 0x7f87340d0e20 2023-07-13T22:31:21.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c009b30 con 0x7f87340d0e20 2023-07-13T22:31:21.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f871bfff700 1 -- 172.21.15.60:0/4294595589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c01c680 con 0x7f87340d0e20 2023-07-13T22:31:21.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.378+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87341548c0 con 0x7f87340d0e20 2023-07-13T22:31:21.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.380+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8734154a50 con 0x7f87340d0e20 2023-07-13T22:31:21.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.380+0000 7f871bfff700 1 -- 172.21.15.60:0/4294595589 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f872c01c7e0 con 0x7f87340d0e20 2023-07-13T22:31:21.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.381+0000 7f871bfff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f872006ace0 0x7f872006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.381+0000 7f871bfff700 1 -- 172.21.15.60:0/4294595589 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f872c08a9b0 con 0x7f87340d0e20 2023-07-13T22:31:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.382+0000 7f87337fe700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f872006ace0 0x7f872006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.382+0000 7f87337fe700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f872006ace0 0x7f872006d190 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8724000f60 tx=0x7f8724007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.386+0000 7f871bfff700 1 -- 172.21.15.60:0/4294595589 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f872c055d70 con 0x7f87340d0e20 2023-07-13T22:31:21.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.672+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) v1 -- 0x7f873404e280 con 0x7f87340d0e20 2023-07-13T22:31:21.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.859+0000 7f871bfff700 1 -- 172.21.15.60:0/4294595589 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v536) v1 ==== 148+0+0 (secure 0 0 0) 0x7f872c059390 con 0x7f87340d0e20 2023-07-13T22:31:21.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f872006ace0 msgr2=0x7f872006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:21.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f872006ace0 0x7f872006d190 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8724000f60 tx=0x7f8724007ed0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87340d0e20 msgr2=0x7f87340b7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:21.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87340d0e20 0x7f87340b7060 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f872c012000 tx=0x7f872c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 shutdown_connections 2023-07-13T22:31:21.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f872006ace0 0x7f872006d190 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87340d0e20 0x7f87340b7060 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87340cd260 0x7f8734153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 --2- 172.21.15.60:0/4294595589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87340d0a50 0x7f87340b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:21.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 >> 172.21.15.60:0/4294595589 conn(0x7f8734071600 msgr2=0x7f873415cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:21.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 shutdown_connections 2023-07-13T22:31:21.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:21.862+0000 7f8739bff700 1 -- 172.21.15.60:0/4294595589 wait complete. 2023-07-13T22:31:21.862 INFO:tasks.cephfs_test_runner:test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-13T22:31:21.863 INFO:tasks.cephfs_test_runner:Starting test: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:31:21.863 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2023-07-13T22:31:22.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.085+0000 7fdb2d7ff700 1 Processor -- start 2023-07-13T22:31:22.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.085+0000 7fdb2d7ff700 1 -- start start 2023-07-13T22:31:22.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.086+0000 7fdb2d7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb280cd260 0x7fdb280d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:22.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.086+0000 7fdb2d7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdb280d0e20 0x7fdb280d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:22.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.086+0000 7fdb2d7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb280d2410 con 0x7fdb280d0a50 2023-07-13T22:31:22.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.086+0000 7fdb2d7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2815c900 con 0x7fdb280cd260 2023-07-13T22:31:22.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.086+0000 7fdb2d7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb280d2660 con 0x7fdb280d0e20 2023-07-13T22:31:22.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.086+0000 7fdb1ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb280cd260 0x7fdb280d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:22.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.086+0000 7fdb1ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb280cd260 0x7fdb280d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40272/0 (socket says 172.21.15.60:40272) 2023-07-13T22:31:22.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.086+0000 7fdb1ffff700 1 -- 172.21.15.60:0/2153237138 learned_addr learned my addr 172.21.15.60:0/2153237138 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:22.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb277fe700 1 --2- 172.21.15.60:0/2153237138 >> v2:172.21.15.105:3300/0 conn(0x7fdb280d0e20 0x7fdb280d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:22.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb1ffff700 1 -- 172.21.15.60:0/2153237138 >> v2:172.21.15.105:3300/0 conn(0x7fdb280d0e20 msgr2=0x7fdb280d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:22.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb1ffff700 1 --2- 172.21.15.60:0/2153237138 >> v2:172.21.15.105:3300/0 conn(0x7fdb280d0e20 0x7fdb280d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb1ffff700 1 -- 172.21.15.60:0/2153237138 >> v1:172.21.15.105:6789/0 conn(0x7fdb280d0a50 legacy=0x7fdb280cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:22.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb1ffff700 1 -- 172.21.15.60:0/2153237138 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280d2940 con 0x7fdb280cd260 2023-07-13T22:31:22.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb267fc700 1 -- 172.21.15.60:0/2153237138 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651585376 0 0) 0x7fdb280d2410 con 0x7fdb280d0a50 2023-07-13T22:31:22.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb1ffff700 1 --2- 172.21.15.60:0/2153237138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 0x7fdb280d3420 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fdb280b8ce0 tx=0x7fdb10009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3ec16e672339e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:22.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb267fc700 1 -- 172.21.15.60:0/2153237138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10011830 con 0x7fdb280cd260 2023-07-13T22:31:22.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb267fc700 1 -- 172.21.15.60:0/2153237138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb10011990 con 0x7fdb280cd260 2023-07-13T22:31:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.087+0000 7fdb267fc700 1 -- 172.21.15.60:0/2153237138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10011d20 con 0x7fdb280cd260 2023-07-13T22:31:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.088+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2153237138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 msgr2=0x7fdb280d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.088+0000 7fdb2d7ff700 1 --2- 172.21.15.60:0/2153237138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 0x7fdb280d3420 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fdb280b8ce0 tx=0x7fdb10009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.088+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2153237138 shutdown_connections 2023-07-13T22:31:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.088+0000 7fdb2d7ff700 1 --2- 172.21.15.60:0/2153237138 >> v2:172.21.15.105:3300/0 conn(0x7fdb280d0e20 0x7fdb280d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.088+0000 7fdb2d7ff700 1 --2- 172.21.15.60:0/2153237138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 0x7fdb280d3420 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.088+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2153237138 >> 172.21.15.60:0/2153237138 conn(0x7fdb28071600 msgr2=0x7fdb280719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.088+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2153237138 shutdown_connections 2023-07-13T22:31:22.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.088+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2153237138 wait complete. 2023-07-13T22:31:22.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.090+0000 7fdb2d7ff700 1 Processor -- start 2023-07-13T22:31:22.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.090+0000 7fdb2d7ff700 1 -- start start 2023-07-13T22:31:22.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.091+0000 7fdb2d7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 0x7fdb28147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:22.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.091+0000 7fdb2d7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb280d0a50 0x7fdb280b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:22.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.091+0000 7fdb2d7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280d0e20 0x7fdb2815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:22.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.091+0000 7fdb2d7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb281483d0 con 0x7fdb280cd260 2023-07-13T22:31:22.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.091+0000 7fdb2d7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28147ed0 con 0x7fdb280d0e20 2023-07-13T22:31:22.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.091+0000 7fdb2d7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2815cde0 con 0x7fdb280d0a50 2023-07-13T22:31:22.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.091+0000 7fdb26ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 0x7fdb28147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:22.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.091+0000 7fdb26ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 0x7fdb28147990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40276/0 (socket says 172.21.15.60:40276) 2023-07-13T22:31:22.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.091+0000 7fdb26ffd700 1 -- 172.21.15.60:0/2478164479 learned_addr learned my addr 172.21.15.60:0/2478164479 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb26ffd700 1 -- 172.21.15.60:0/2478164479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb280d0a50 msgr2=0x7fdb280b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb26ffd700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb280d0a50 0x7fdb280b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb26ffd700 1 -- 172.21.15.60:0/2478164479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280d0e20 msgr2=0x7fdb2815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb277fe700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280d0e20 0x7fdb2815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb26ffd700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280d0e20 0x7fdb2815e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb26ffd700 1 -- 172.21.15.60:0/2478164479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb10009550 con 0x7fdb280cd260 2023-07-13T22:31:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb277fe700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280d0e20 0x7fdb2815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb26ffd700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 0x7fdb28147990 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fdb180011e0 tx=0x7fdb1800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb1f7fe700 1 -- 172.21.15.60:0/2478164479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb180054e0 con 0x7fdb280cd260 2023-07-13T22:31:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb1f7fe700 1 -- 172.21.15.60:0/2478164479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb18003680 con 0x7fdb280cd260 2023-07-13T22:31:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb1f7fe700 1 -- 172.21.15.60:0/2478164479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb18003a30 con 0x7fdb280cd260 2023-07-13T22:31:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2815d0c0 con 0x7fdb280cd260 2023-07-13T22:31:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.092+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2815d680 con 0x7fdb280cd260 2023-07-13T22:31:22.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.093+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2804e280 con 0x7fdb280cd260 2023-07-13T22:31:22.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.094+0000 7fdb1f7fe700 1 -- 172.21.15.60:0/2478164479 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb18003b90 con 0x7fdb280cd260 2023-07-13T22:31:22.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.095+0000 7fdb1f7fe700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0806ace0 0x7fdb0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:22.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.095+0000 7fdb1f7fe700 1 -- 172.21.15.60:0/2478164479 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdb18088770 con 0x7fdb280cd260 2023-07-13T22:31:22.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.095+0000 7fdb1ffff700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0806ace0 0x7fdb0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:22.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.096+0000 7fdb1ffff700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0806ace0 0x7fdb0806d190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fdb28159f70 tx=0x7fdb10010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:22.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.099+0000 7fdb1f7fe700 1 -- 172.21.15.60:0/2478164479 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdb18053aa0 con 0x7fdb280cd260 2023-07-13T22:31:22.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.386+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) v1 -- 0x7fdb28159600 con 0x7fdb280cd260 2023-07-13T22:31:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.862+0000 7fdb1f7fe700 1 -- 172.21.15.60:0/2478164479 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v537) v1 ==== 154+0+0 (secure 0 0 0) 0x7fdb180170e0 con 0x7fdb280cd260 2023-07-13T22:31:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0806ace0 msgr2=0x7fdb0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0806ace0 0x7fdb0806d190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fdb28159f70 tx=0x7fdb10010000 comp rx=0 tx=0).stop 2023-07-13T22:31:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 msgr2=0x7fdb28147990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 0x7fdb28147990 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fdb180011e0 tx=0x7fdb1800cae0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 shutdown_connections 2023-07-13T22:31:22.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdb0806ace0 0x7fdb0806d190 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb280cd260 0x7fdb28147990 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280d0e20 0x7fdb2815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 --2- 172.21.15.60:0/2478164479 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb280d0a50 0x7fdb280b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:22.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 >> 172.21.15.60:0/2478164479 conn(0x7fdb28071600 msgr2=0x7fdb2806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:22.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 shutdown_connections 2023-07-13T22:31:22.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:22.865+0000 7fdb2d7ff700 1 -- 172.21.15.60:0/2478164479 wait complete. 2023-07-13T22:31:22.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-13T22:31:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.097+0000 7ffaf3bff700 1 Processor -- start 2023-07-13T22:31:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.097+0000 7ffaf3bff700 1 -- start start 2023-07-13T22:31:23.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.098+0000 7ffaf3bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffaec15c790 0x7ffaec0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.098+0000 7ffaf3bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaec0cf6d0 con 0x7ffaec0ac490 2023-07-13T22:31:23.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.098+0000 7ffaf3bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaec0ac900 con 0x7ffaec0ad9a0 2023-07-13T22:31:23.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.098+0000 7ffaf3bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaec0acbe0 con 0x7ffaec15c790 2023-07-13T22:31:23.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf119a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffaec0ac490 0x7ffaec0c9e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35464/0 (socket says 172.21.15.60:35464) 2023-07-13T22:31:23.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf119a700 1 -- 172.21.15.60:0/1719651404 learned_addr learned my addr 172.21.15.60:0/1719651404 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:23.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf199b700 1 --2- 172.21.15.60:0/1719651404 >> v2:172.21.15.105:3300/0 conn(0x7ffaec15c790 0x7ffaec0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf0999700 1 -- 172.21.15.60:0/1719651404 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170425305 0 0) 0x7ffaec0cf6d0 con 0x7ffaec0ac490 2023-07-13T22:31:23.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf0999700 1 -- 172.21.15.60:0/1719651404 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad80035b0 con 0x7ffaec0ac490 2023-07-13T22:31:23.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf0999700 1 -- 172.21.15.60:0/1719651404 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663286113 0 0) 0x7ffaec0ac900 con 0x7ffaec0ad9a0 2023-07-13T22:31:23.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf0999700 1 -- 172.21.15.60:0/1719651404 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaec0cf6d0 con 0x7ffaec0ad9a0 2023-07-13T22:31:23.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf199b700 1 -- 172.21.15.60:0/1719651404 >> v1:172.21.15.105:6789/0 conn(0x7ffaec0ad9a0 legacy=0x7ffaec0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf199b700 1 -- 172.21.15.60:0/1719651404 >> v1:172.21.15.60:6789/0 conn(0x7ffaec0ac490 legacy=0x7ffaec0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf0999700 1 -- 172.21.15.60:0/1719651404 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036893888 0 0) 0x7ffad80035b0 con 0x7ffaec0ac490 2023-07-13T22:31:23.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.099+0000 7ffaf199b700 1 -- 172.21.15.60:0/1719651404 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec0acec0 con 0x7ffaec15c790 2023-07-13T22:31:23.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.100+0000 7ffaf199b700 1 --2- 172.21.15.60:0/1719651404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 0x7ffaec0ce4b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009820 tx=0x7ffae8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36cac6299a08123 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:23.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.100+0000 7ffaf0999700 1 -- 172.21.15.60:0/1719651404 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8005e30 con 0x7ffaec15c790 2023-07-13T22:31:23.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.100+0000 7ffaf0999700 1 -- 172.21.15.60:0/1719651404 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae8006030 con 0x7ffaec15c790 2023-07-13T22:31:23.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.100+0000 7ffaf0999700 1 -- 172.21.15.60:0/1719651404 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae80063c0 con 0x7ffaec15c790 2023-07-13T22:31:23.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.100+0000 7ffaf3bff700 1 -- 172.21.15.60:0/1719651404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 msgr2=0x7ffaec0ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.100+0000 7ffaf3bff700 1 --2- 172.21.15.60:0/1719651404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 0x7ffaec0ce4b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009820 tx=0x7ffae8009b30 comp rx=0 tx=0).stop 2023-07-13T22:31:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.101+0000 7ffaf3bff700 1 -- 172.21.15.60:0/1719651404 shutdown_connections 2023-07-13T22:31:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.101+0000 7ffaf3bff700 1 --2- 172.21.15.60:0/1719651404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 0x7ffaec0ce4b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.101+0000 7ffaf3bff700 1 -- 172.21.15.60:0/1719651404 >> 172.21.15.60:0/1719651404 conn(0x7ffaec06b900 msgr2=0x7ffaec0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.101+0000 7ffaf3bff700 1 -- 172.21.15.60:0/1719651404 shutdown_connections 2023-07-13T22:31:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.101+0000 7ffaf3bff700 1 -- 172.21.15.60:0/1719651404 wait complete. 2023-07-13T22:31:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.102+0000 7ffaf3bff700 1 Processor -- start 2023-07-13T22:31:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.103+0000 7ffaf3bff700 1 -- start start 2023-07-13T22:31:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.103+0000 7ffaf3bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec0ac490 0x7ffaec15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.103+0000 7ffaf3bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0ad9a0 0x7ffaec0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.103+0000 7ffaf3bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 0x7ffaec0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.103+0000 7ffaf3bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec15bbf0 con 0x7ffaec0ac490 2023-07-13T22:31:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.103+0000 7ffaf3bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0c9620 con 0x7ffaec15c790 2023-07-13T22:31:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf3bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec09a750 con 0x7ffaec0ad9a0 2023-07-13T22:31:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec0ac490 0x7ffaec15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf199b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec0ac490 0x7ffaec15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40284/0 (socket says 172.21.15.60:40284) 2023-07-13T22:31:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf199b700 1 -- 172.21.15.60:0/3969583620 learned_addr learned my addr 172.21.15.60:0/3969583620 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:23.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf119a700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0ad9a0 0x7ffaec0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf219c700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 0x7ffaec0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf199b700 1 -- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0ad9a0 msgr2=0x7ffaec0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf199b700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0ad9a0 0x7ffaec0c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf199b700 1 -- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 msgr2=0x7ffaec0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf199b700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 0x7ffaec0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf199b700 1 -- 172.21.15.60:0/3969583620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae8009430 con 0x7ffaec0ac490 2023-07-13T22:31:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf119a700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0ad9a0 0x7ffaec0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf219c700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 0x7ffaec0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.104+0000 7ffaf199b700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec0ac490 0x7ffaec15b380 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ffae8013040 tx=0x7ffae8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.105+0000 7ffae27fc700 1 -- 172.21.15.60:0/3969583620 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8025430 con 0x7ffaec0ac490 2023-07-13T22:31:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.105+0000 7ffae27fc700 1 -- 172.21.15.60:0/3969583620 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae8020070 con 0x7ffaec0ac490 2023-07-13T22:31:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.105+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaec09aa30 con 0x7ffaec0ac490 2023-07-13T22:31:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.105+0000 7ffae27fc700 1 -- 172.21.15.60:0/3969583620 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae80257f0 con 0x7ffaec0ac490 2023-07-13T22:31:23.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.105+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaec0c9ba0 con 0x7ffaec0ac490 2023-07-13T22:31:23.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.106+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaec04e210 con 0x7ffaec0ac490 2023-07-13T22:31:23.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.107+0000 7ffae27fc700 1 -- 172.21.15.60:0/3969583620 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffae8022030 con 0x7ffaec0ac490 2023-07-13T22:31:23.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.108+0000 7ffae27fc700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffad006ace0 0x7ffad006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.108+0000 7ffae27fc700 1 -- 172.21.15.60:0/3969583620 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ffae808a700 con 0x7ffaec0ac490 2023-07-13T22:31:23.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.108+0000 7ffaf119a700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffad006ace0 0x7ffad006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.109+0000 7ffaf119a700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffad006ace0 0x7ffad006d190 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ffadc007fd0 tx=0x7ffadc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:23.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.112+0000 7ffae27fc700 1 -- 172.21.15.60:0/3969583620 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ffae8055ac0 con 0x7ffaec0ac490 2023-07-13T22:31:23.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.320+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ffaec0c30f0 con 0x7ffaec0ac490 2023-07-13T22:31:23.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.324+0000 7ffae27fc700 1 -- 172.21.15.60:0/3969583620 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffae808a030 con 0x7ffaec0ac490 2023-07-13T22:31:23.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffad006ace0 msgr2=0x7ffad006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffad006ace0 0x7ffad006d190 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ffadc007fd0 tx=0x7ffadc009040 comp rx=0 tx=0).stop 2023-07-13T22:31:23.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec0ac490 msgr2=0x7ffaec15b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec0ac490 0x7ffaec15b380 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ffae8013040 tx=0x7ffae8004750 comp rx=0 tx=0).stop 2023-07-13T22:31:23.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 shutdown_connections 2023-07-13T22:31:23.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ffad006ace0 0x7ffad006d190 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaec0ac490 0x7ffaec15b380 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec15c790 0x7ffaec0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 --2- 172.21.15.60:0/3969583620 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0ad9a0 0x7ffaec0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 >> 172.21.15.60:0/3969583620 conn(0x7ffaec06b900 msgr2=0x7ffaec0ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:23.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 shutdown_connections 2023-07-13T22:31:23.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.326+0000 7ffaf3bff700 1 -- 172.21.15.60:0/3969583620 wait complete. 2023-07-13T22:31:23.326 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:31:23.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:31:23.429 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:31:23.537 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:31:23.537 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:31:23.538 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:23.584 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:31:23.584 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:31:23.585 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:23.632 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:31:23.632 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:31:23.633 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:23.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:31:23.679 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:31:23.680 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:23.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:31:23.726 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-13T22:31:23.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:23.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.948+0000 7f43a63ff700 1 Processor -- start 2023-07-13T22:31:23.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.948+0000 7f43a63ff700 1 -- start start 2023-07-13T22:31:23.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.949+0000 7f43a63ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43a00d3750 0x7f43a00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.949+0000 7f43a63ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f43a00cfad0 0x7f43a00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.949+0000 7f43a63ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a00cd950 con 0x7f43a00b8400 2023-07-13T22:31:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.949+0000 7f43a63ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a00cff50 con 0x7f43a00d3750 2023-07-13T22:31:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.949+0000 7f43a63ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a00d0ac0 con 0x7f43a00cfad0 2023-07-13T22:31:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f439f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43a00d3750 0x7f43a00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f439f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43a00d3750 0x7f43a00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40290/0 (socket says 172.21.15.60:40290) 2023-07-13T22:31:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f439f7fe700 1 -- 172.21.15.60:0/455227784 learned_addr learned my addr 172.21.15.60:0/455227784 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f43a499c700 1 --2- 172.21.15.60:0/455227784 >> v2:172.21.15.105:3300/0 conn(0x7f43a00cfad0 0x7f43a00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f439f7fe700 1 -- 172.21.15.60:0/455227784 >> v2:172.21.15.105:3300/0 conn(0x7f43a00cfad0 msgr2=0x7f43a00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f439f7fe700 1 --2- 172.21.15.60:0/455227784 >> v2:172.21.15.105:3300/0 conn(0x7f43a00cfad0 0x7f43a00d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f439f7fe700 1 -- 172.21.15.60:0/455227784 >> v1:172.21.15.60:6789/0 conn(0x7f43a00b8400 legacy=0x7f43a00cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f439f7fe700 1 -- 172.21.15.60:0/455227784 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a00b87d0 con 0x7f43a00d3750 2023-07-13T22:31:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f439effd700 1 -- 172.21.15.60:0/455227784 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568173396 0 0) 0x7f4390001110 con 0x7f43a00b8400 2023-07-13T22:31:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.950+0000 7f439f7fe700 1 --2- 172.21.15.60:0/455227784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00d3750 0x7f43a00cf590 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f43a00b1b40 tx=0x7f4394009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea00ed861decdcd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:23.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f439effd700 1 -- 172.21.15.60:0/455227784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394011830 con 0x7f43a00d3750 2023-07-13T22:31:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f439effd700 1 -- 172.21.15.60:0/455227784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4394011990 con 0x7f43a00d3750 2023-07-13T22:31:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f439effd700 1 -- 172.21.15.60:0/455227784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394011d20 con 0x7f43a00d3750 2023-07-13T22:31:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f43a63ff700 1 -- 172.21.15.60:0/455227784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00d3750 msgr2=0x7f43a00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f43a63ff700 1 --2- 172.21.15.60:0/455227784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00d3750 0x7f43a00cf590 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f43a00b1b40 tx=0x7f4394009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f43a63ff700 1 -- 172.21.15.60:0/455227784 shutdown_connections 2023-07-13T22:31:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f43a63ff700 1 --2- 172.21.15.60:0/455227784 >> v2:172.21.15.105:3300/0 conn(0x7f43a00cfad0 0x7f43a00d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f43a63ff700 1 --2- 172.21.15.60:0/455227784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00d3750 0x7f43a00cf590 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f43a63ff700 1 -- 172.21.15.60:0/455227784 >> 172.21.15.60:0/455227784 conn(0x7f43a006b390 msgr2=0x7f43a006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:23.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f43a63ff700 1 -- 172.21.15.60:0/455227784 shutdown_connections 2023-07-13T22:31:23.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.951+0000 7f43a63ff700 1 -- 172.21.15.60:0/455227784 wait complete. 2023-07-13T22:31:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.953+0000 7f43a63ff700 1 Processor -- start 2023-07-13T22:31:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.953+0000 7f43a63ff700 1 -- start start 2023-07-13T22:31:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.954+0000 7f43a63ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00b8400 0x7f43a01539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.954+0000 7f43a63ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43a00cfad0 0x7f43a00b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.954+0000 7f43a63ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00d3750 0x7f43a00b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.954+0000 7f43a63ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a01f33b0 con 0x7f43a00b8400 2023-07-13T22:31:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.954+0000 7f43a63ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a015e650 con 0x7f43a00d3750 2023-07-13T22:31:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.954+0000 7f43a63ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a01f30d0 con 0x7f43a00cfad0 2023-07-13T22:31:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.954+0000 7f439ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00b8400 0x7f43a01539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.954+0000 7f439ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00b8400 0x7f43a01539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40296/0 (socket says 172.21.15.60:40296) 2023-07-13T22:31:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.954+0000 7f439ffff700 1 -- 172.21.15.60:0/3486277318 learned_addr learned my addr 172.21.15.60:0/3486277318 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f43a499c700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00d3750 0x7f43a00b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439f7fe700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43a00cfad0 0x7f43a00b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439ffff700 1 -- 172.21.15.60:0/3486277318 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43a00cfad0 msgr2=0x7f43a00b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439ffff700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43a00cfad0 0x7f43a00b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439ffff700 1 -- 172.21.15.60:0/3486277318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00d3750 msgr2=0x7f43a00b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439ffff700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00d3750 0x7f43a00b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439ffff700 1 -- 172.21.15.60:0/3486277318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4394009550 con 0x7f43a00b8400 2023-07-13T22:31:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439ffff700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00b8400 0x7f43a01539b0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f439000c550 tx=0x7f439000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439cff9700 1 -- 172.21.15.60:0/3486277318 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43900054e0 con 0x7f43a00b8400 2023-07-13T22:31:23.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439cff9700 1 -- 172.21.15.60:0/3486277318 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4390003680 con 0x7f43a00b8400 2023-07-13T22:31:23.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f439cff9700 1 -- 172.21.15.60:0/3486277318 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390003a30 con 0x7f43a00b8400 2023-07-13T22:31:23.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.955+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a00cb810 con 0x7f43a00b8400 2023-07-13T22:31:23.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.956+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a00cbdd0 con 0x7f43a00b8400 2023-07-13T22:31:23.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.957+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a00cb670 con 0x7f43a00b8400 2023-07-13T22:31:23.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.957+0000 7f439cff9700 1 -- 172.21.15.60:0/3486277318 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4390003b90 con 0x7f43a00b8400 2023-07-13T22:31:23.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.958+0000 7f439cff9700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f438c06ace0 0x7f438c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:23.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.958+0000 7f439cff9700 1 -- 172.21.15.60:0/3486277318 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4390088450 con 0x7f43a00b8400 2023-07-13T22:31:23.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.959+0000 7f439f7fe700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f438c06ace0 0x7f438c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.959+0000 7f439f7fe700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f438c06ace0 0x7f438c06d190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f43a00d3b60 tx=0x7f4394008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:23.963+0000 7f439cff9700 1 -- 172.21.15.60:0/3486277318 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f43a00cb670 con 0x7f43a00b8400 2023-07-13T22:31:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.216+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43a00cd950 con 0x7f43a00b8400 2023-07-13T22:31:24.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.216+0000 7f439cff9700 1 -- 172.21.15.60:0/3486277318 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f4390017de0 con 0x7f43a00b8400 2023-07-13T22:31:24.200 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:24.200 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":90,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8230,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1454911448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1454911448},{"type":"v1","addr":"172.21.15.105:6834","nonce":1454911448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1356220490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1356220490},{"type":"v1","addr":"172.21.15.105:6835","nonce":1356220490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/156713026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":156713026},{"type":"v1","addr":"172.21.15.60:6837","nonce":156713026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":9327,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2936989941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2936989941},{"type":"v1","addr":"172.21.15.60:6836","nonce":2936989941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[]} 2023-07-13T22:31:24.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f438c06ace0 msgr2=0x7f438c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:24.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f438c06ace0 0x7f438c06d190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f43a00d3b60 tx=0x7f4394008030 comp rx=0 tx=0).stop 2023-07-13T22:31:24.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00b8400 msgr2=0x7f43a01539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:24.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00b8400 0x7f43a01539b0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f439000c550 tx=0x7f439000c910 comp rx=0 tx=0).stop 2023-07-13T22:31:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 shutdown_connections 2023-07-13T22:31:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f438c06ace0 0x7f438c06d190 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a00b8400 0x7f43a01539b0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a00d3750 0x7f43a00b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 --2- 172.21.15.60:0/3486277318 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43a00cfad0 0x7f43a00b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 >> 172.21.15.60:0/3486277318 conn(0x7f43a006b390 msgr2=0x7f43a00d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:24.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 shutdown_connections 2023-07-13T22:31:24.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.219+0000 7f43a63ff700 1 -- 172.21.15.60:0/3486277318 wait complete. 2023-07-13T22:31:24.203 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 90 2023-07-13T22:31:24.219 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-13T22:31:24.219 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-13T22:31:24.220 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:31:24.220 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:31:24.221 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-13T22:31:24.221 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-13T22:31:24.221 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:31:24.221 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:31:24.222 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-13T22:31:24.222 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-13T22:31:24.222 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:31:24.222 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:31:24.223 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-13T22:31:24.223 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-13T22:31:24.223 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:31:24.223 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:31:24.233 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:31:24.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-13T22:31:24.235 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:31:24.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-13T22:31:24.236 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:31:24.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-13T22:31:24.238 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:31:24.238 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-13T22:31:24.279 INFO:tasks.ceph.mds.a:Started 2023-07-13T22:31:24.279 DEBUG:teuthology.parallel:result is None 2023-07-13T22:31:24.281 INFO:tasks.ceph.mds.b:Started 2023-07-13T22:31:24.282 DEBUG:teuthology.parallel:result is None 2023-07-13T22:31:24.284 INFO:tasks.ceph.mds.c:Started 2023-07-13T22:31:24.284 DEBUG:teuthology.parallel:result is None 2023-07-13T22:31:24.285 INFO:tasks.ceph.mds.d:Started 2023-07-13T22:31:24.285 DEBUG:teuthology.parallel:result is None 2023-07-13T22:31:24.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-13T22:31:24.430 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2023-07-13T22:31:24.430 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2023-07-13T22:31:24.433 INFO:tasks.ceph.mds.d.smithi105.stderr:2023-07-13T22:31:24.451+0000 7f0d2b9fd700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6834/0: (98) Address already in use 2023-07-13T22:31:24.435 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-07-13T22:31:24.438 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:31:24.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.520+0000 7f4e68dff700 1 Processor -- start 2023-07-13T22:31:24.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.520+0000 7f4e68dff700 1 -- start start 2023-07-13T22:31:24.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.521+0000 7f4e68dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4e640b8570 0x7f4e640ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:24.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.521+0000 7f4e68dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e640d35c0 0x7f4e640d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:24.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.521+0000 7f4e68dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e640d09d0 con 0x7f4e640cf3f0 2023-07-13T22:31:24.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.521+0000 7f4e68dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e640d3b30 con 0x7f4e640d35c0 2023-07-13T22:31:24.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.521+0000 7f4e68dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e640cda10 con 0x7f4e640b8570 2023-07-13T22:31:24.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.521+0000 7f4e5bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e640d35c0 0x7f4e640d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:24.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.521+0000 7f4e5bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e640d35c0 0x7f4e640d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40354/0 (socket says 172.21.15.60:40354) 2023-07-13T22:31:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.521+0000 7f4e5bfff700 1 -- 172.21.15.60:0/1004803684 learned_addr learned my addr 172.21.15.60:0/1004803684 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.521+0000 7f4e6259c700 1 --2- 172.21.15.60:0/1004803684 >> v2:172.21.15.105:3300/0 conn(0x7f4e640b8570 0x7f4e640ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.522+0000 7f4e5bfff700 1 -- 172.21.15.60:0/1004803684 >> v2:172.21.15.105:3300/0 conn(0x7f4e640b8570 msgr2=0x7f4e640ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.522+0000 7f4e5bfff700 1 --2- 172.21.15.60:0/1004803684 >> v2:172.21.15.105:3300/0 conn(0x7f4e640b8570 0x7f4e640ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.522+0000 7f4e5bfff700 1 -- 172.21.15.60:0/1004803684 >> v1:172.21.15.105:6789/0 conn(0x7f4e640cf3f0 legacy=0x7f4e6415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.522+0000 7f4e5bfff700 1 -- 172.21.15.60:0/1004803684 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e640cd450 con 0x7f4e640d35c0 2023-07-13T22:31:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.522+0000 7f4e5bfff700 1 --2- 172.21.15.60:0/1004803684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 0x7f4e640d0eb0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f4e6406c6c0 tx=0x7f4e4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53d4966204c340 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:24.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.522+0000 7f4e61d9b700 1 -- 172.21.15.60:0/1004803684 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c011830 con 0x7f4e640d35c0 2023-07-13T22:31:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.522+0000 7f4e61d9b700 1 -- 172.21.15.60:0/1004803684 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4c011990 con 0x7f4e640d35c0 2023-07-13T22:31:24.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.522+0000 7f4e61d9b700 1 -- 172.21.15.60:0/1004803684 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c011d20 con 0x7f4e640d35c0 2023-07-13T22:31:24.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.523+0000 7f4e68dff700 1 -- 172.21.15.60:0/1004803684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 msgr2=0x7f4e640d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:24.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.523+0000 7f4e68dff700 1 --2- 172.21.15.60:0/1004803684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 0x7f4e640d0eb0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f4e6406c6c0 tx=0x7f4e4c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.523+0000 7f4e68dff700 1 -- 172.21.15.60:0/1004803684 shutdown_connections 2023-07-13T22:31:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.523+0000 7f4e68dff700 1 --2- 172.21.15.60:0/1004803684 >> v2:172.21.15.105:3300/0 conn(0x7f4e640b8570 0x7f4e640ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.523+0000 7f4e68dff700 1 --2- 172.21.15.60:0/1004803684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 0x7f4e640d0eb0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.523+0000 7f4e68dff700 1 -- 172.21.15.60:0/1004803684 >> 172.21.15.60:0/1004803684 conn(0x7f4e64071600 msgr2=0x7f4e640719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:24.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.523+0000 7f4e68dff700 1 -- 172.21.15.60:0/1004803684 shutdown_connections 2023-07-13T22:31:24.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.523+0000 7f4e68dff700 1 -- 172.21.15.60:0/1004803684 wait complete. 2023-07-13T22:31:24.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.525+0000 7f4e68dff700 1 Processor -- start 2023-07-13T22:31:24.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.526+0000 7f4e68dff700 1 -- start start 2023-07-13T22:31:24.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.527+0000 7f4e68dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e640b8570 0x7f4e64156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:24.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.527+0000 7f4e68dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e640cf3f0 0x7f4e64157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:24.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.527+0000 7f4e68dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 0x7f4e6415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.527+0000 7f4e68dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e641597a0 con 0x7f4e640d35c0 2023-07-13T22:31:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.527+0000 7f4e68dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64157230 con 0x7f4e640cf3f0 2023-07-13T22:31:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.527+0000 7f4e68dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64158700 con 0x7f4e640b8570 2023-07-13T22:31:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.527+0000 7f4e62d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 0x7f4e6415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.527+0000 7f4e62d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 0x7f4e6415c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40364/0 (socket says 172.21.15.60:40364) 2023-07-13T22:31:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.527+0000 7f4e62d9d700 1 -- 172.21.15.60:0/55957345 learned_addr learned my addr 172.21.15.60:0/55957345 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e6259c700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e640b8570 0x7f4e64156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e5bfff700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e640cf3f0 0x7f4e64157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e62d9d700 1 -- 172.21.15.60:0/55957345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e640b8570 msgr2=0x7f4e64156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e62d9d700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e640b8570 0x7f4e64156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e62d9d700 1 -- 172.21.15.60:0/55957345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e640cf3f0 msgr2=0x7f4e64157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e62d9d700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e640cf3f0 0x7f4e64157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e62d9d700 1 -- 172.21.15.60:0/55957345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c009550 con 0x7f4e640d35c0 2023-07-13T22:31:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e62d9d700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 0x7f4e6415c130 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00cc50 tx=0x7f4e5c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e5affd700 1 -- 172.21.15.60:0/55957345 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00ad80 con 0x7f4e640d35c0 2023-07-13T22:31:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e5affd700 1 -- 172.21.15.60:0/55957345 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c00ce50 con 0x7f4e640d35c0 2023-07-13T22:31:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e5affd700 1 -- 172.21.15.60:0/55957345 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c003c40 con 0x7f4e640d35c0 2023-07-13T22:31:24.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e641589e0 con 0x7f4e640d35c0 2023-07-13T22:31:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.528+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e64158fa0 con 0x7f4e640d35c0 2023-07-13T22:31:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.529+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6404e280 con 0x7f4e640d35c0 2023-07-13T22:31:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.530+0000 7f4e5affd700 1 -- 172.21.15.60:0/55957345 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e5c0050d0 con 0x7f4e640d35c0 2023-07-13T22:31:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.531+0000 7f4e5affd700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ad10 0x7f4e4406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.531+0000 7f4e5affd700 1 -- 172.21.15.60:0/55957345 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4e5c089350 con 0x7f4e640d35c0 2023-07-13T22:31:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.532+0000 7f4e6259c700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ad10 0x7f4e4406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:24.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.532+0000 7f4e6259c700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ad10 0x7f4e4406d1c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009b50 tx=0x7f4e54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:24.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.536+0000 7f4e5affd700 1 -- 172.21.15.60:0/55957345 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4e5c054680 con 0x7f4e640d35c0 2023-07-13T22:31:24.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.757+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4e6415ee10 con 0x7f4e640d35c0 2023-07-13T22:31:24.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.875+0000 7f4e5affd700 1 -- 172.21.15.60:0/55957345 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v129) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4e5c057ca0 con 0x7f4e640d35c0 2023-07-13T22:31:24.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ad10 msgr2=0x7f4e4406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:24.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ad10 0x7f4e4406d1c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009b50 tx=0x7f4e54008040 comp rx=0 tx=0).stop 2023-07-13T22:31:24.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 msgr2=0x7f4e6415c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 0x7f4e6415c130 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00cc50 tx=0x7f4e5c00a340 comp rx=0 tx=0).stop 2023-07-13T22:31:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 shutdown_connections 2023-07-13T22:31:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4e4406ad10 0x7f4e4406d1c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e640d35c0 0x7f4e6415c130 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e640b8570 0x7f4e64156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 --2- 172.21.15.60:0/55957345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e640cf3f0 0x7f4e64157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:24.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 >> 172.21.15.60:0/55957345 conn(0x7f4e64071600 msgr2=0x7f4e6415f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:24.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 shutdown_connections 2023-07-13T22:31:24.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:24.878+0000 7f4e68dff700 1 -- 172.21.15.60:0/55957345 wait complete. 2023-07-13T22:31:24.862 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-07-13T22:31:24.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-13T22:31:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.103+0000 7f17717ff700 1 Processor -- start 2023-07-13T22:31:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.103+0000 7f17717ff700 1 -- start start 2023-07-13T22:31:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.105+0000 7f17717ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f176c158870 0x7f176c0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.105+0000 7f17717ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c0c52c0 0x7f176c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.105+0000 7f17717ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0c3fc0 con 0x7f176c06a410 2023-07-13T22:31:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.105+0000 7f17717ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176c06a890 con 0x7f176c158870 2023-07-13T22:31:25.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.105+0000 7f17717ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0c42a0 con 0x7f176c0c52c0 2023-07-13T22:31:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.105+0000 7f1763fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c0c52c0 0x7f176c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.105+0000 7f176affd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f176c158870 0x7f176c0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.105+0000 7f1763fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c0c52c0 0x7f176c069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:44420/0 (socket says 172.21.15.60:44420) 2023-07-13T22:31:25.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.105+0000 7f1763fff700 1 -- 172.21.15.60:0/2392324579 learned_addr learned my addr 172.21.15.60:0/2392324579 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:25.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.106+0000 7f176affd700 1 -- 172.21.15.60:0/2392324579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c0c52c0 msgr2=0x7f176c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:25.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.106+0000 7f176affd700 1 --2- 172.21.15.60:0/2392324579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c0c52c0 0x7f176c069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.106+0000 7f176affd700 1 -- 172.21.15.60:0/2392324579 >> v1:172.21.15.105:6789/0 conn(0x7f176c06a410 legacy=0x7f176c0c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.106+0000 7f176affd700 1 -- 172.21.15.60:0/2392324579 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0c4580 con 0x7f176c158870 2023-07-13T22:31:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.106+0000 7f176affd700 1 --2- 172.21.15.60:0/2392324579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 0x7f176c0c4d80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f175c009840 tx=0x7f175c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96dd95257f61b3a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.106+0000 7f176a7fc700 1 -- 172.21.15.60:0/2392324579 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c011830 con 0x7f176c158870 2023-07-13T22:31:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.106+0000 7f176a7fc700 1 -- 172.21.15.60:0/2392324579 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c011990 con 0x7f176c158870 2023-07-13T22:31:25.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.106+0000 7f176a7fc700 1 -- 172.21.15.60:0/2392324579 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c011d20 con 0x7f176c158870 2023-07-13T22:31:25.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.107+0000 7f17717ff700 1 -- 172.21.15.60:0/2392324579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 msgr2=0x7f176c0c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.107+0000 7f17717ff700 1 --2- 172.21.15.60:0/2392324579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 0x7f176c0c4d80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f175c009840 tx=0x7f175c009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:25.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.107+0000 7f17717ff700 1 -- 172.21.15.60:0/2392324579 shutdown_connections 2023-07-13T22:31:25.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.107+0000 7f17717ff700 1 --2- 172.21.15.60:0/2392324579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 0x7f176c0c4d80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.107+0000 7f17717ff700 1 --2- 172.21.15.60:0/2392324579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c0c52c0 0x7f176c069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.107+0000 7f17717ff700 1 -- 172.21.15.60:0/2392324579 >> 172.21.15.60:0/2392324579 conn(0x7f176c0cf410 msgr2=0x7f176c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.107+0000 7f17717ff700 1 -- 172.21.15.60:0/2392324579 shutdown_connections 2023-07-13T22:31:25.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.107+0000 7f17717ff700 1 -- 172.21.15.60:0/2392324579 wait complete. 2023-07-13T22:31:25.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.108+0000 7f17717ff700 1 Processor -- start 2023-07-13T22:31:25.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.109+0000 7f17717ff700 1 -- start start 2023-07-13T22:31:25.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.109+0000 7f17717ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c06a410 0x7f176c1481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.109+0000 7f17717ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f176c0c52c0 0x7f176c14c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f17717ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 0x7f176c14c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f17717ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0ae7a0 con 0x7f176c0c52c0 2023-07-13T22:31:25.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f17717ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0c7f90 con 0x7f176c158870 2023-07-13T22:31:25.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f17717ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0c8270 con 0x7f176c06a410 2023-07-13T22:31:25.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f1763fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f176c0c52c0 0x7f176c14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f1763fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f176c0c52c0 0x7f176c14c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57850/0 (socket says 172.21.15.60:57850) 2023-07-13T22:31:25.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f1763fff700 1 -- 172.21.15.60:0/4199516415 learned_addr learned my addr 172.21.15.60:0/4199516415 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:25.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f176affd700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c06a410 0x7f176c1481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f176b7fe700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 0x7f176c14c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f1763fff700 1 -- 172.21.15.60:0/4199516415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c06a410 msgr2=0x7f176c1481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f1763fff700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c06a410 0x7f176c1481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f1763fff700 1 -- 172.21.15.60:0/4199516415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 msgr2=0x7f176c14c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f1763fff700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 0x7f176c14c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f1763fff700 1 -- 172.21.15.60:0/4199516415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0094a0 con 0x7f176c0c52c0 2023-07-13T22:31:25.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.110+0000 7f176b7fe700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 0x7f176c14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:25.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.111+0000 7f1763fff700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f176c0c52c0 0x7f176c14c280 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f175400e910 tx=0x7f175400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:25.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.111+0000 7f17637fe700 1 -- 172.21.15.60:0/4199516415 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540096a0 con 0x7f176c0c52c0 2023-07-13T22:31:25.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.111+0000 7f17637fe700 1 -- 172.21.15.60:0/4199516415 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175400ee60 con 0x7f176c0c52c0 2023-07-13T22:31:25.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.111+0000 7f17637fe700 1 -- 172.21.15.60:0/4199516415 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175400ca90 con 0x7f176c0c52c0 2023-07-13T22:31:25.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.111+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c0c8550 con 0x7f176c0c52c0 2023-07-13T22:31:25.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.111+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c0c72d0 con 0x7f176c0c52c0 2023-07-13T22:31:25.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.112+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c0ae8e0 con 0x7f176c0c52c0 2023-07-13T22:31:25.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.113+0000 7f17637fe700 1 -- 172.21.15.60:0/4199516415 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f175400cbf0 con 0x7f176c0c52c0 2023-07-13T22:31:25.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.114+0000 7f17637fe700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f174c06ad10 0x7f174c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.114+0000 7f17637fe700 1 -- 172.21.15.60:0/4199516415 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1754089100 con 0x7f176c0c52c0 2023-07-13T22:31:25.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.114+0000 7f176affd700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f174c06ad10 0x7f174c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.115+0000 7f176affd700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f174c06ad10 0x7f174c06d1c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f175c000c00 tx=0x7f175c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:25.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.118+0000 7f17637fe700 1 -- 172.21.15.60:0/4199516415 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f176c0ae8e0 con 0x7f176c0c52c0 2023-07-13T22:31:25.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.406+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f176c0ae8e0 con 0x7f176c0c52c0 2023-07-13T22:31:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.407+0000 7f17637fe700 1 -- 172.21.15.60:0/4199516415 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f176c0ae8e0 con 0x7f176c0c52c0 2023-07-13T22:31:25.389 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:25.390 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:31:25.390 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-07-13T22:31:25.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.390 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-07-13T22:31:25.390 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk21S0AxAAkigFxr5VIo2UuIdld8bOlw==", 2023-07-13T22:31:25.390 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:31:25.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:31:25.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:31:25.391 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.392 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-07-13T22:31:25.392 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkYOy2GxAALOGAll2rjtdA6fSMYlLi8g==", 2023-07-13T22:31:25.392 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:31:25.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:31:25.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:31:25.392 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.393 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.393 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-07-13T22:31:25.393 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBk1rDoDxAA86LRLZUyJxcN3zvMJx9fvA==", 2023-07-13T22:31:25.393 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:31:25.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:31:25.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:31:25.394 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.394 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.394 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-07-13T22:31:25.394 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkKxJ/JxAAgqEBzD/MiqFoaxHRNu+wJQ==", 2023-07-13T22:31:25.395 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:31:25.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:31:25.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:31:25.395 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.395 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.396 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-07-13T22:31:25.396 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQeLBkTjvbLxAAYpqVJ4BAgEzB5yPolCuGvQ==", 2023-07-13T22:31:25.396 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:31:25.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:31:25.396 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.396 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.397 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-07-13T22:31:25.397 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCTeLBkHkGENBAAnhAmipj6AS6Vh40qwAb6gw==", 2023-07-13T22:31:25.397 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:31:25.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:31:25.397 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.397 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.398 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-07-13T22:31:25.398 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWeLBk1UvzOBAASpd4uWaRZ76C2CKulNoT0Q==", 2023-07-13T22:31:25.398 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:31:25.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:31:25.398 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.399 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.399 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-07-13T22:31:25.399 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCaeLBkNP1wARAAzXWg4P8IYopwC1sN+OoKiQ==", 2023-07-13T22:31:25.399 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:31:25.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:31:25.400 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.400 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.400 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-07-13T22:31:25.400 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCgeLBk3ll6GBAADZSYlLSV0D1gk8XNAc3hIg==", 2023-07-13T22:31:25.400 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:31:25.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:31:25.401 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.401 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.401 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-07-13T22:31:25.401 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjeLBkm9gtHBAAnkVRYR21oQAxmmKe1gGNvA==", 2023-07-13T22:31:25.402 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:31:25.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:31:25.402 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.402 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.402 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-07-13T22:31:25.403 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCmeLBk4gY9IBAAWKeUlsSrTEgHqipYKmNtqw==", 2023-07-13T22:31:25.403 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:31:25.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:31:25.403 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.403 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.404 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-07-13T22:31:25.404 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCpeLBk0j/sIxAAt+6nsifmHWt4HzQBynpieQ==", 2023-07-13T22:31:25.404 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-07-13T22:31:25.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-07-13T22:31:25.404 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.404 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.405 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-07-13T22:31:25.405 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w==", 2023-07-13T22:31:25.405 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:31:25.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2023-07-13T22:31:25.405 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-13T22:31:25.406 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.406 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-07-13T22:31:25.406 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkVbeDBRAA3N0grhFu2Ib8uRyX6Xj/gw==", 2023-07-13T22:31:25.406 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:31:25.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:31:25.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:31:25.407 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:31:25.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.407 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-07-13T22:31:25.407 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCOeLBkMNSKORAAr/fQE88kxuDtCqG68v0GMg==", 2023-07-13T22:31:25.407 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:31:25.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:31:25.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:31:25.408 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:31:25.408 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.409 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-07-13T22:31:25.409 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCPeLBkn7QEDRAAVa7AImGsNIkXrhYW8+I+7Q==", 2023-07-13T22:31:25.409 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-07-13T22:31:25.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-07-13T22:31:25.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-07-13T22:31:25.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-07-13T22:31:25.409 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.410 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-07-13T22:31:25.410 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBk7CgGBhAAaK88V4CEyrYfZx7wv+XG3g==", 2023-07-13T22:31:25.410 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:31:25.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-07-13T22:31:25.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-07-13T22:31:25.411 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.411 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.411 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-07-13T22:31:25.411 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkbkQxNxAAfO8dFPzCBsBjYHmg89oXpQ==", 2023-07-13T22:31:25.411 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:31:25.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:31:25.412 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.412 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:25.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:25.412 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-07-13T22:31:25.412 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNeLBkx9LRLhAA+XfmtesldgH4hGk9AGGSkQ==", 2023-07-13T22:31:25.412 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-07-13T22:31:25.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-07-13T22:31:25.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-07-13T22:31:25.413 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-07-13T22:31:25.413 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.413 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:25.413 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:31:25.413 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:31:25.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f174c06ad10 msgr2=0x7f174c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f174c06ad10 0x7f174c06d1c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f175c000c00 tx=0x7f175c01c040 comp rx=0 tx=0).stop 2023-07-13T22:31:25.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f176c0c52c0 msgr2=0x7f176c14c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f176c0c52c0 0x7f176c14c280 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f175400e910 tx=0x7f175400ec20 comp rx=0 tx=0).stop 2023-07-13T22:31:25.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 shutdown_connections 2023-07-13T22:31:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f174c06ad10 0x7f174c06d1c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f176c158870 0x7f176c14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f176c06a410 0x7f176c1481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 --2- 172.21.15.60:0/4199516415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f176c0c52c0 0x7f176c14c280 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 >> 172.21.15.60:0/4199516415 conn(0x7f176c0cf410 msgr2=0x7f176c149610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 shutdown_connections 2023-07-13T22:31:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.409+0000 7f17717ff700 1 -- 172.21.15.60:0/4199516415 wait complete. 2023-07-13T22:31:25.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:25.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.639+0000 7f8bfb9ff700 1 Processor -- start 2023-07-13T22:31:25.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.640+0000 7f8bfb9ff700 1 -- start start 2023-07-13T22:31:25.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.641+0000 7f8bfb9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8bf4069ed0 0x7f8bf406a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.641+0000 7f8bfb9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8bf40c9300 0x7f8bf40c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.641+0000 7f8bfb9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf406a860 con 0x7f8bf4158870 2023-07-13T22:31:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.641+0000 7f8bfb9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf40c3fc0 con 0x7f8bf4069ed0 2023-07-13T22:31:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.641+0000 7f8bfb9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf40c42a0 con 0x7f8bf40c9300 2023-07-13T22:31:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.641+0000 7f8bf8f9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8bf4069ed0 0x7f8bf406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.641+0000 7f8bf979b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8bf4158870 0x7f8bf40c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39116/0 (socket says 172.21.15.60:39116) 2023-07-13T22:31:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.641+0000 7f8bf979b700 1 -- 172.21.15.60:0/3303092939 learned_addr learned my addr 172.21.15.60:0/3303092939 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:25.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.642+0000 7f8bf8f9a700 1 -- 172.21.15.60:0/3303092939 >> v2:172.21.15.105:3300/0 conn(0x7f8bf40c9300 msgr2=0x7f8bf40c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.642+0000 7f8bf8f9a700 1 --2- 172.21.15.60:0/3303092939 >> v2:172.21.15.105:3300/0 conn(0x7f8bf40c9300 0x7f8bf40c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.642+0000 7f8bf8f9a700 1 -- 172.21.15.60:0/3303092939 >> v1:172.21.15.60:6789/0 conn(0x7f8bf4158870 legacy=0x7f8bf40c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.642+0000 7f8bf8f9a700 1 -- 172.21.15.60:0/3303092939 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf40c4580 con 0x7f8bf4069ed0 2023-07-13T22:31:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.642+0000 7f8bebfff700 1 -- 172.21.15.60:0/3303092939 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887631752 0 0) 0x7f8bf0001110 con 0x7f8bf4158870 2023-07-13T22:31:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.642+0000 7f8bf8f9a700 1 --2- 172.21.15.60:0/3303092939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf4069ed0 0x7f8bf406a320 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40c8cd0 tx=0x7f8be4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d8af5c5e5fbeb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.642+0000 7f8bebfff700 1 -- 172.21.15.60:0/3303092939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4011830 con 0x7f8bf4069ed0 2023-07-13T22:31:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.642+0000 7f8bebfff700 1 -- 172.21.15.60:0/3303092939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4011990 con 0x7f8bf4069ed0 2023-07-13T22:31:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.642+0000 7f8bebfff700 1 -- 172.21.15.60:0/3303092939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4011d20 con 0x7f8bf4069ed0 2023-07-13T22:31:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.643+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/3303092939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf4069ed0 msgr2=0x7f8bf406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.643+0000 7f8bfb9ff700 1 --2- 172.21.15.60:0/3303092939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf4069ed0 0x7f8bf406a320 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40c8cd0 tx=0x7f8be4009b20 comp rx=0 tx=0).stop 2023-07-13T22:31:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.643+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/3303092939 shutdown_connections 2023-07-13T22:31:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.643+0000 7f8bfb9ff700 1 --2- 172.21.15.60:0/3303092939 >> v2:172.21.15.105:3300/0 conn(0x7f8bf40c9300 0x7f8bf40c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.643+0000 7f8bfb9ff700 1 --2- 172.21.15.60:0/3303092939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf4069ed0 0x7f8bf406a320 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.643+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/3303092939 >> 172.21.15.60:0/3303092939 conn(0x7f8bf40cf410 msgr2=0x7f8bf40cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:25.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.643+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/3303092939 shutdown_connections 2023-07-13T22:31:25.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.643+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/3303092939 wait complete. 2023-07-13T22:31:25.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.644+0000 7f8bfb9ff700 1 Processor -- start 2023-07-13T22:31:25.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.645+0000 7f8bfb9ff700 1 -- start start 2023-07-13T22:31:25.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.645+0000 7f8bfb9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf4069ed0 0x7f8bf4145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.645+0000 7f8bfb9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9300 0x7f8bf4146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bfb9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4158870 0x7f8bf4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bfb9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4147e50 con 0x7f8bf40c9300 2023-07-13T22:31:25.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bfb9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf41489d0 con 0x7f8bf4069ed0 2023-07-13T22:31:25.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bfb9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf41480a0 con 0x7f8bf4158870 2023-07-13T22:31:25.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf8f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9300 0x7f8bf4146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf8f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9300 0x7f8bf4146d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57870/0 (socket says 172.21.15.60:57870) 2023-07-13T22:31:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf8f9a700 1 -- 172.21.15.60:0/1782560381 learned_addr learned my addr 172.21.15.60:0/1782560381 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf979b700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf4069ed0 0x7f8bf4145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf9f9c700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4158870 0x7f8bf4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf8f9a700 1 -- 172.21.15.60:0/1782560381 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4158870 msgr2=0x7f8bf4157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf8f9a700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4158870 0x7f8bf4157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf8f9a700 1 -- 172.21.15.60:0/1782560381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf4069ed0 msgr2=0x7f8bf4145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf8f9a700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf4069ed0 0x7f8bf4145210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.646+0000 7f8bf8f9a700 1 -- 172.21.15.60:0/1782560381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be40094a0 con 0x7f8bf40c9300 2023-07-13T22:31:25.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.647+0000 7f8bf8f9a700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9300 0x7f8bf4146d90 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f8be4014040 tx=0x7f8be4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:25.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.647+0000 7f8be9ffb700 1 -- 172.21.15.60:0/1782560381 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be40036a0 con 0x7f8bf40c9300 2023-07-13T22:31:25.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.647+0000 7f8be9ffb700 1 -- 172.21.15.60:0/1782560381 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be401e070 con 0x7f8bf40c9300 2023-07-13T22:31:25.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.647+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf4148380 con 0x7f8bf40c9300 2023-07-13T22:31:25.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.647+0000 7f8be9ffb700 1 -- 172.21.15.60:0/1782560381 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4003a00 con 0x7f8bf40c9300 2023-07-13T22:31:25.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.647+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf41472d0 con 0x7f8bf40c9300 2023-07-13T22:31:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.648+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf40c39f0 con 0x7f8bf40c9300 2023-07-13T22:31:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.649+0000 7f8be9ffb700 1 -- 172.21.15.60:0/1782560381 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8be4003b60 con 0x7f8bf40c9300 2023-07-13T22:31:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.650+0000 7f8be9ffb700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.650+0000 7f8be9ffb700 1 -- 172.21.15.60:0/1782560381 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8be408a230 con 0x7f8bf40c9300 2023-07-13T22:31:25.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.650+0000 7f8bf979b700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:25.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.651+0000 7f8bf979b700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000f60 tx=0x7f8bf0007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:25.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.654+0000 7f8be9ffb700 1 -- 172.21.15.60:0/1782560381 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8be4055620 con 0x7f8bf40c9300 2023-07-13T22:31:25.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.862+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bf415c650 con 0x7f8bf40c9300 2023-07-13T22:31:25.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.863+0000 7f8be9ffb700 1 -- 172.21.15.60:0/1782560381 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+14555 (secure 0 0 0) 0x7f8be4058c40 con 0x7f8bf40c9300 2023-07-13T22:31:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:25.846 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":131,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:24.882175+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:25.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.865+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 msgr2=0x7f8bd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000f60 tx=0x7f8bf0007c20 comp rx=0 tx=0).stop 2023-07-13T22:31:25.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9300 msgr2=0x7f8bf4146d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:25.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9300 0x7f8bf4146d90 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f8be4014040 tx=0x7f8be4004750 comp rx=0 tx=0).stop 2023-07-13T22:31:25.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 shutdown_connections 2023-07-13T22:31:25.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8bd806ace0 0x7f8bd806d190 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4158870 0x7f8bf4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf4069ed0 0x7f8bf4145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 --2- 172.21.15.60:0/1782560381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf40c9300 0x7f8bf4146d90 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:25.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 >> 172.21.15.60:0/1782560381 conn(0x7f8bf40cf410 msgr2=0x7f8bf40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:25.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 shutdown_connections 2023-07-13T22:31:25.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:25.866+0000 7f8bfb9ff700 1 -- 172.21.15.60:0/1782560381 wait complete. 2023-07-13T22:31:25.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:31:25.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:31:26.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.095+0000 7f3fd89ff700 1 Processor -- start 2023-07-13T22:31:26.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.095+0000 7f3fd89ff700 1 -- start start 2023-07-13T22:31:26.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.096+0000 7f3fd89ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.096+0000 7f3fd89ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fd40cede0 0x7f3fd40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.096+0000 7f3fd89ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd40d1870 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.096+0000 7f3fd89ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd40cf370 con 0x7f3fd40cede0 2023-07-13T22:31:26.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.096+0000 7f3fd89ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40d12b0 con 0x7f3fd40b8200 2023-07-13T22:31:26.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fd2d9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fd40cede0 0x7f3fd40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fd259c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fd2d9d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fd40cede0 0x7f3fd40cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:46990/0 (socket says 172.21.15.60:46990) 2023-07-13T22:31:26.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fd2d9d700 1 -- 172.21.15.60:0/2142653354 learned_addr learned my addr 172.21.15.60:0/2142653354 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:26.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fcbfff700 1 --1- 172.21.15.60:0/2142653354 >> v1:172.21.15.105:6789/0 conn(0x7f3fd40b1ac0 0x7f3fd415ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:43474/0 (socket says 172.21.15.60:43474) 2023-07-13T22:31:26.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fd2d9d700 1 -- 172.21.15.60:0/2142653354 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 msgr2=0x7f3fd40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:26.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fd2d9d700 1 --2- 172.21.15.60:0/2142653354 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fd2d9d700 1 -- 172.21.15.60:0/2142653354 >> v1:172.21.15.105:6789/0 conn(0x7f3fd40b1ac0 legacy=0x7f3fd415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fd2d9d700 1 -- 172.21.15.60:0/2142653354 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd40d1590 con 0x7f3fd40cede0 2023-07-13T22:31:26.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.097+0000 7f3fd259c700 1 --2- 172.21.15.60:0/2142653354 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:26.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd2d9d700 1 --2- 172.21.15.60:0/2142653354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 0x7f3fd40cfb80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009e50 tx=0x7f3fcc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a23f8966e9afdebe server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:26.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd1d9b700 1 -- 172.21.15.60:0/2142653354 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc00e070 con 0x7f3fd40cede0 2023-07-13T22:31:26.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd1d9b700 1 -- 172.21.15.60:0/2142653354 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fcc00b910 con 0x7f3fd40cede0 2023-07-13T22:31:26.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd1d9b700 1 -- 172.21.15.60:0/2142653354 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc00bcc0 con 0x7f3fd40cede0 2023-07-13T22:31:26.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2142653354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 msgr2=0x7f3fd40cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd89ff700 1 --2- 172.21.15.60:0/2142653354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 0x7f3fd40cfb80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009e50 tx=0x7f3fcc00b330 comp rx=0 tx=0).stop 2023-07-13T22:31:26.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2142653354 shutdown_connections 2023-07-13T22:31:26.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd89ff700 1 --2- 172.21.15.60:0/2142653354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 0x7f3fd40cfb80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd89ff700 1 --2- 172.21.15.60:0/2142653354 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.098+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2142653354 >> 172.21.15.60:0/2142653354 conn(0x7f3fd406b2f0 msgr2=0x7f3fd406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.099+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2142653354 shutdown_connections 2023-07-13T22:31:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.099+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2142653354 wait complete. 2023-07-13T22:31:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.100+0000 7f3fd89ff700 1 Processor -- start 2023-07-13T22:31:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.100+0000 7f3fd89ff700 1 -- start start 2023-07-13T22:31:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.101+0000 7f3fd89ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd40b1ac0 0x7f3fd40c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.101+0000 7f3fd89ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.101+0000 7f3fd89ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 0x7f3fd40b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.101+0000 7f3fd89ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4156780 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.101+0000 7f3fd89ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40cc1d0 con 0x7f3fd40cede0 2023-07-13T22:31:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.101+0000 7f3fd89ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40cc4b0 con 0x7f3fd40b8200 2023-07-13T22:31:26.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd40b1ac0 0x7f3fd40c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd40b1ac0 0x7f3fd40c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57886/0 (socket says 172.21.15.60:57886) 2023-07-13T22:31:26.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd259c700 1 -- 172.21.15.60:0/2985491785 learned_addr learned my addr 172.21.15.60:0/2985491785 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:26.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fcbfff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd2d9d700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 0x7f3fd40b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd259c700 1 -- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 msgr2=0x7f3fd40b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd259c700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40b6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd259c700 1 -- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 msgr2=0x7f3fd40b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd259c700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 0x7f3fd40b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd259c700 1 -- 172.21.15.60:0/2985491785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40094a0 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fcbfff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:26.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd2d9d700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 0x7f3fd40b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:26.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.102+0000 7f3fd259c700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd40b1ac0 0x7f3fd40c5900 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400ebe0 tx=0x7f3fc4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:26.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.103+0000 7f3fcaffd700 1 -- 172.21.15.60:0/2985491785 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400c910 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.103+0000 7f3fcaffd700 1 -- 172.21.15.60:0/2985491785 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc400ca70 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.103+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc009b30 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.103+0000 7f3fcaffd700 1 -- 172.21.15.60:0/2985491785 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400cde0 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.103+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd4146ca0 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.104+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd4146820 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.105+0000 7f3fcaffd700 1 -- 172.21.15.60:0/2985491785 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3fc4013470 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.106+0000 7f3fcaffd700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fb406acb0 0x7f3fb406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.106+0000 7f3fcaffd700 1 -- 172.21.15.60:0/2985491785 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3fc4089420 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.106+0000 7f3fcbfff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fb406acb0 0x7f3fb406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.107+0000 7f3fcbfff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fb406acb0 0x7f3fb406d160 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3fd41499d0 tx=0x7f3fbc007dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:26.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.110+0000 7f3fcaffd700 1 -- 172.21.15.60:0/2985491785 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3fd4146820 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.316+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3fd4146820 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.317+0000 7f3fcaffd700 1 -- 172.21.15.60:0/2985491785 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v131) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3fd4146820 con 0x7f3fd40b1ac0 2023-07-13T22:31:26.299 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:31:26.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fb406acb0 msgr2=0x7f3fb406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fb406acb0 0x7f3fb406d160 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3fd41499d0 tx=0x7f3fbc007dd0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd40b1ac0 msgr2=0x7f3fd40c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd40b1ac0 0x7f3fd40c5900 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400ebe0 tx=0x7f3fc4009ec0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 shutdown_connections 2023-07-13T22:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3fb406acb0 0x7f3fb406d160 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd40b1ac0 0x7f3fd40c5900 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fd40cede0 0x7f3fd40b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 --2- 172.21.15.60:0/2985491785 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3fd40b8200 0x7f3fd40b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 >> 172.21.15.60:0/2985491785 conn(0x7f3fd406b2f0 msgr2=0x7f3fd40b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 shutdown_connections 2023-07-13T22:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.319+0000 7f3fd89ff700 1 -- 172.21.15.60:0/2985491785 wait complete. 2023-07-13T22:31:26.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:31:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.541+0000 7fb5a5dff700 1 Processor -- start 2023-07-13T22:31:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.541+0000 7fb5a5dff700 1 -- start start 2023-07-13T22:31:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb5a5dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5a00cfd50 0x7fb5a00d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb5a5dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5a00b1ac0 0x7fb5a00d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb5a5dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a00d1870 con 0x7fb5a00b8200 2023-07-13T22:31:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb5a5dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a00cf370 con 0x7fb5a00cfd50 2023-07-13T22:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb5a5dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a00d12b0 con 0x7fb5a00b1ac0 2023-07-13T22:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb597fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5a00cfd50 0x7fb5a00d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb597fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5a00cfd50 0x7fb5a00d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57902/0 (socket says 172.21.15.60:57902) 2023-07-13T22:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb597fff700 1 -- 172.21.15.60:0/3339946060 learned_addr learned my addr 172.21.15.60:0/3339946060 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:26.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb59ffff700 1 --2- 172.21.15.60:0/3339946060 >> v2:172.21.15.105:3300/0 conn(0x7fb5a00b1ac0 0x7fb5a00d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb597fff700 1 -- 172.21.15.60:0/3339946060 >> v2:172.21.15.105:3300/0 conn(0x7fb5a00b1ac0 msgr2=0x7fb5a00d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb597fff700 1 --2- 172.21.15.60:0/3339946060 >> v2:172.21.15.105:3300/0 conn(0x7fb5a00b1ac0 0x7fb5a00d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb597fff700 1 -- 172.21.15.60:0/3339946060 >> v1:172.21.15.60:6789/0 conn(0x7fb5a00b8200 legacy=0x7fb5a00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb597fff700 1 -- 172.21.15.60:0/3339946060 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a00d1590 con 0x7fb5a00cfd50 2023-07-13T22:31:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.543+0000 7fb59effd700 1 -- 172.21.15.60:0/3339946060 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997904815 0 0) 0x7fb5a00d1870 con 0x7fb5a00b8200 2023-07-13T22:31:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.544+0000 7fb597fff700 1 --2- 172.21.15.60:0/3339946060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 0x7fb5a00d23c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00b8d70 tx=0x7fb588009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1183f93f98e9429 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.544+0000 7fb59effd700 1 -- 172.21.15.60:0/3339946060 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800b870 con 0x7fb5a00cfd50 2023-07-13T22:31:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.544+0000 7fb59effd700 1 -- 172.21.15.60:0/3339946060 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58800b9d0 con 0x7fb5a00cfd50 2023-07-13T22:31:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.544+0000 7fb59effd700 1 -- 172.21.15.60:0/3339946060 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800bd60 con 0x7fb5a00cfd50 2023-07-13T22:31:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.544+0000 7fb5a5dff700 1 -- 172.21.15.60:0/3339946060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 msgr2=0x7fb5a00d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.544+0000 7fb5a5dff700 1 --2- 172.21.15.60:0/3339946060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 0x7fb5a00d23c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00b8d70 tx=0x7fb588009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.545+0000 7fb5a5dff700 1 -- 172.21.15.60:0/3339946060 shutdown_connections 2023-07-13T22:31:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.545+0000 7fb5a5dff700 1 --2- 172.21.15.60:0/3339946060 >> v2:172.21.15.105:3300/0 conn(0x7fb5a00b1ac0 0x7fb5a00d2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.545+0000 7fb5a5dff700 1 --2- 172.21.15.60:0/3339946060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 0x7fb5a00d23c0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.545+0000 7fb5a5dff700 1 -- 172.21.15.60:0/3339946060 >> 172.21.15.60:0/3339946060 conn(0x7fb5a006b2f0 msgr2=0x7fb5a006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.545+0000 7fb5a5dff700 1 -- 172.21.15.60:0/3339946060 shutdown_connections 2023-07-13T22:31:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.545+0000 7fb5a5dff700 1 -- 172.21.15.60:0/3339946060 wait complete. 2023-07-13T22:31:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.546+0000 7fb5a5dff700 1 Processor -- start 2023-07-13T22:31:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.547+0000 7fb5a5dff700 1 -- start start 2023-07-13T22:31:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.547+0000 7fb5a5dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a00b1ac0 0x7fb5a0155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.547+0000 7fb5a5dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5a00b8200 0x7fb5a015d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb5a5dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 0x7fb5a015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb5a5dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a0156b40 con 0x7fb5a00cfd50 2023-07-13T22:31:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb5a5dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015dda0 con 0x7fb5a00b1ac0 2023-07-13T22:31:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb5a5dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015e080 con 0x7fb5a00b8200 2023-07-13T22:31:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 0x7fb5a015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 0x7fb5a015ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57910/0 (socket says 172.21.15.60:57910) 2023-07-13T22:31:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59ffff700 1 -- 172.21.15.60:0/1531301687 learned_addr learned my addr 172.21.15.60:0/1531301687 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb597fff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5a00b8200 0x7fb5a015d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59f7fe700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a00b1ac0 0x7fb5a0155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59ffff700 1 -- 172.21.15.60:0/1531301687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5a00b8200 msgr2=0x7fb5a015d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59ffff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5a00b8200 0x7fb5a015d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59ffff700 1 -- 172.21.15.60:0/1531301687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a00b1ac0 msgr2=0x7fb5a0155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59ffff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a00b1ac0 0x7fb5a0155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59ffff700 1 -- 172.21.15.60:0/1531301687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb588009550 con 0x7fb5a00cfd50 2023-07-13T22:31:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.548+0000 7fb59f7fe700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a00b1ac0 0x7fb5a0155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.549+0000 7fb59ffff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 0x7fb5a015ee40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb598007e40 tx=0x7fb59800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.549+0000 7fb59cff9700 1 -- 172.21.15.60:0/1531301687 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59800eea0 con 0x7fb5a00cfd50 2023-07-13T22:31:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.549+0000 7fb59cff9700 1 -- 172.21.15.60:0/1531301687 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb598014420 con 0x7fb5a00cfd50 2023-07-13T22:31:26.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.549+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a015e360 con 0x7fb5a00cfd50 2023-07-13T22:31:26.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.549+0000 7fb59cff9700 1 -- 172.21.15.60:0/1531301687 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980147b0 con 0x7fb5a00cfd50 2023-07-13T22:31:26.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.549+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a0156280 con 0x7fb5a00cfd50 2023-07-13T22:31:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.550+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a00b8940 con 0x7fb5a00cfd50 2023-07-13T22:31:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.551+0000 7fb59cff9700 1 -- 172.21.15.60:0/1531301687 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb598014910 con 0x7fb5a00cfd50 2023-07-13T22:31:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.552+0000 7fb59cff9700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb58006ace0 0x7fb58006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.552+0000 7fb59cff9700 1 -- 172.21.15.60:0/1531301687 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb5980898e0 con 0x7fb5a00cfd50 2023-07-13T22:31:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.552+0000 7fb59f7fe700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb58006ace0 0x7fb58006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.553+0000 7fb59f7fe700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb58006ace0 0x7fb58006d190 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb590000f60 tx=0x7fb590009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:26.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.556+0000 7fb59cff9700 1 -- 172.21.15.60:0/1531301687 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb598054d70 con 0x7fb5a00cfd50 2023-07-13T22:31:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.760+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5a004e210 con 0x7fb5a00cfd50 2023-07-13T22:31:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.760+0000 7fb59cff9700 1 -- 172.21.15.60:0/1531301687 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb598058390 con 0x7fb5a00cfd50 2023-07-13T22:31:26.743 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:31:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.762+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb58006ace0 msgr2=0x7fb58006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb58006ace0 0x7fb58006d190 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb590000f60 tx=0x7fb590009040 comp rx=0 tx=0).stop 2023-07-13T22:31:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 msgr2=0x7fb5a015ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 0x7fb5a015ee40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb598007e40 tx=0x7fb59800e3c0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 shutdown_connections 2023-07-13T22:31:26.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb58006ace0 0x7fb58006d190 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a00cfd50 0x7fb5a015ee40 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a00b1ac0 0x7fb5a0155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 --2- 172.21.15.60:0/1531301687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5a00b8200 0x7fb5a015d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 >> 172.21.15.60:0/1531301687 conn(0x7fb5a006b2f0 msgr2=0x7fb5a015c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 shutdown_connections 2023-07-13T22:31:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.763+0000 7fb5a5dff700 1 -- 172.21.15.60:0/1531301687 wait complete. 2023-07-13T22:31:26.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:31:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.985+0000 7fc9dcdff700 1 Processor -- start 2023-07-13T22:31:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.985+0000 7fc9dcdff700 1 -- start start 2023-07-13T22:31:26.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.987+0000 7fc9dcdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80b8570 0x7fc9d80ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.987+0000 7fc9dcdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc9d80d35c0 0x7fc9d80d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.987+0000 7fc9dcdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d80d09d0 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.987+0000 7fc9dcdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d80d3b30 con 0x7fc9d80d35c0 2023-07-13T22:31:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.987+0000 7fc9dcdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80cda10 con 0x7fc9d80b8570 2023-07-13T22:31:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.987+0000 7fc9cffff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc9d80d35c0 0x7fc9d80d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.987+0000 7fc9d659c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80b8570 0x7fc9d80ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.987+0000 7fc9cffff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc9d80d35c0 0x7fc9d80d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:47026/0 (socket says 172.21.15.60:47026) 2023-07-13T22:31:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.987+0000 7fc9cffff700 1 -- 172.21.15.60:0/1138421167 learned_addr learned my addr 172.21.15.60:0/1138421167 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9d5d9b700 1 -- 172.21.15.60:0/1138421167 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210311441 0 0) 0x7fc9d80d09d0 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9d5d9b700 1 -- 172.21.15.60:0/1138421167 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9c40035b0 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9d659c700 1 -- 172.21.15.60:0/1138421167 >> v2:172.21.15.105:3300/0 conn(0x7fc9d80d35c0 msgr2=0x7fc9d80d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9d659c700 1 --2- 172.21.15.60:0/1138421167 >> v2:172.21.15.105:3300/0 conn(0x7fc9d80d35c0 0x7fc9d80d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9d659c700 1 -- 172.21.15.60:0/1138421167 >> v1:172.21.15.105:6789/0 conn(0x7fc9d80cf3f0 legacy=0x7fc9d815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9d659c700 1 -- 172.21.15.60:0/1138421167 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d80cd450 con 0x7fc9d80b8570 2023-07-13T22:31:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9cffff700 1 --2- 172.21.15.60:0/1138421167 >> v2:172.21.15.105:3300/0 conn(0x7fc9d80d35c0 0x7fc9d80d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9d659c700 1 --2- 172.21.15.60:0/1138421167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80b8570 0x7fc9d80ceeb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009820 tx=0x7fc9c8009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f2da607df2877f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:26.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9d5d9b700 1 -- 172.21.15.60:0/1138421167 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8005e30 con 0x7fc9d80b8570 2023-07-13T22:31:26.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.988+0000 7fc9d5d9b700 1 -- 172.21.15.60:0/1138421167 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8006030 con 0x7fc9d80b8570 2023-07-13T22:31:26.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.989+0000 7fc9d5d9b700 1 -- 172.21.15.60:0/1138421167 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80063c0 con 0x7fc9d80b8570 2023-07-13T22:31:26.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.989+0000 7fc9dcdff700 1 -- 172.21.15.60:0/1138421167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80b8570 msgr2=0x7fc9d80ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.989+0000 7fc9dcdff700 1 --2- 172.21.15.60:0/1138421167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80b8570 0x7fc9d80ceeb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009820 tx=0x7fc9c8009b30 comp rx=0 tx=0).stop 2023-07-13T22:31:26.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.989+0000 7fc9dcdff700 1 -- 172.21.15.60:0/1138421167 shutdown_connections 2023-07-13T22:31:26.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.989+0000 7fc9dcdff700 1 --2- 172.21.15.60:0/1138421167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80b8570 0x7fc9d80ceeb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.989+0000 7fc9dcdff700 1 --2- 172.21.15.60:0/1138421167 >> v2:172.21.15.105:3300/0 conn(0x7fc9d80d35c0 0x7fc9d80d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.989+0000 7fc9dcdff700 1 -- 172.21.15.60:0/1138421167 >> 172.21.15.60:0/1138421167 conn(0x7fc9d8071600 msgr2=0x7fc9d80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.989+0000 7fc9dcdff700 1 -- 172.21.15.60:0/1138421167 shutdown_connections 2023-07-13T22:31:26.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.989+0000 7fc9dcdff700 1 -- 172.21.15.60:0/1138421167 wait complete. 2023-07-13T22:31:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.991+0000 7fc9dcdff700 1 Processor -- start 2023-07-13T22:31:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.992+0000 7fc9dcdff700 1 -- start start 2023-07-13T22:31:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9dcdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d80b8570 0x7fc9d8152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9dcdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d80cf3f0 0x7fc9d80cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9dcdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80d35c0 0x7fc9d80a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9dcdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80c7a40 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9dcdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80cc480 con 0x7fc9d80b8570 2023-07-13T22:31:26.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9dcdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80cc760 con 0x7fc9d80d35c0 2023-07-13T22:31:26.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9cffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d80cf3f0 0x7fc9d80cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9cffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d80cf3f0 0x7fc9d80cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57926/0 (socket says 172.21.15.60:57926) 2023-07-13T22:31:26.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9cffff700 1 -- 172.21.15.60:0/276225469 learned_addr learned my addr 172.21.15.60:0/276225469 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:26.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9d659c700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d80b8570 0x7fc9d8152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.993+0000 7fc9d6d9d700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80d35c0 0x7fc9d80a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9cffff700 1 -- 172.21.15.60:0/276225469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80d35c0 msgr2=0x7fc9d80a9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9cffff700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80d35c0 0x7fc9d80a9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9cffff700 1 -- 172.21.15.60:0/276225469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d80b8570 msgr2=0x7fc9d8152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9cffff700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d80b8570 0x7fc9d8152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9cffff700 1 -- 172.21.15.60:0/276225469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0009550 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9cffff700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d80cf3f0 0x7fc9d80cad60 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e9b0 tx=0x7fc9c000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9ceffd700 1 -- 172.21.15.60:0/276225469 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000ee50 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9ceffd700 1 -- 172.21.15.60:0/276225469 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c000c850 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c8009430 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.994+0000 7fc9ceffd700 1 -- 172.21.15.60:0/276225469 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000cbc0 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.995+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d80c7ce0 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.996+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d804e280 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.996+0000 7fc9ceffd700 1 -- 172.21.15.60:0/276225469 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc9c000cd20 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.997+0000 7fc9ceffd700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9b806ace0 0x7fc9b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.997+0000 7fc9ceffd700 1 -- 172.21.15.60:0/276225469 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc9c0089100 con 0x7fc9d80cf3f0 2023-07-13T22:31:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.998+0000 7fc9d659c700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9b806ace0 0x7fc9b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:26.998+0000 7fc9d659c700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9b806ace0 0x7fc9b806d190 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009b70 tx=0x7fc9c8008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.002+0000 7fc9ceffd700 1 -- 172.21.15.60:0/276225469 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc9c00544f0 con 0x7fc9d80cf3f0 2023-07-13T22:31:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.209+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc9d80cea90 con 0x7fc9d80cf3f0 2023-07-13T22:31:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.210+0000 7fc9ceffd700 1 -- 172.21.15.60:0/276225469 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v131) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc9c0057b10 con 0x7fc9d80cf3f0 2023-07-13T22:31:27.192 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:27.193 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-07-13T22:31:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-07-13T22:31:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": "rbd" 2023-07-13T22:31:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 2, 2023-07-13T22:31:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-07-13T22:31:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:27.194 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-07-13T22:31:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9b806ace0 msgr2=0x7fc9b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9b806ace0 0x7fc9b806d190 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009b70 tx=0x7fc9c8008070 comp rx=0 tx=0).stop 2023-07-13T22:31:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d80cf3f0 msgr2=0x7fc9d80cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d80cf3f0 0x7fc9d80cad60 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e9b0 tx=0x7fc9c000ecc0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 shutdown_connections 2023-07-13T22:31:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc9b806ace0 0x7fc9b806d190 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d80d35c0 0x7fc9d80a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d80b8570 0x7fc9d8152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 --2- 172.21.15.60:0/276225469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d80cf3f0 0x7fc9d80cad60 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.212+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 >> 172.21.15.60:0/276225469 conn(0x7fc9d8071600 msgr2=0x7fc9d80cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.213+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 shutdown_connections 2023-07-13T22:31:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.213+0000 7fc9dcdff700 1 -- 172.21.15.60:0/276225469 wait complete. 2023-07-13T22:31:27.213 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:31:27.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:31:27.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.438+0000 7f87e93ff700 1 Processor -- start 2023-07-13T22:31:27.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.438+0000 7f87e93ff700 1 -- start start 2023-07-13T22:31:27.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.439+0000 7f87e93ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e415c6e0 0x7f87e415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:27.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.439+0000 7f87e93ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87e40c8da0 0x7f87e40cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:27.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.439+0000 7f87e93ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e415cab0 con 0x7f87e40ce0f0 2023-07-13T22:31:27.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.439+0000 7f87e93ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e40cafd0 con 0x7f87e40c8da0 2023-07-13T22:31:27.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.439+0000 7f87e93ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40ce570 con 0x7f87e415c6e0 2023-07-13T22:31:27.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.439+0000 7f87e27fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87e40c8da0 0x7f87e40cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:27.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e27fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87e40c8da0 0x7f87e40cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57928/0 (socket says 172.21.15.60:57928) 2023-07-13T22:31:27.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e27fc700 1 -- 172.21.15.60:0/2333379797 learned_addr learned my addr 172.21.15.60:0/2333379797 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:27.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e37fe700 1 --1- 172.21.15.60:0/2333379797 >> v1:172.21.15.60:6789/0 conn(0x7f87e40ce0f0 0x7f87e40cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39138/0 (socket says 172.21.15.60:39138) 2023-07-13T22:31:27.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e2ffd700 1 --2- 172.21.15.60:0/2333379797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e415c6e0 0x7f87e415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:27.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e27fc700 1 -- 172.21.15.60:0/2333379797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e415c6e0 msgr2=0x7f87e415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:27.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e27fc700 1 --2- 172.21.15.60:0/2333379797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e415c6e0 0x7f87e415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e27fc700 1 -- 172.21.15.60:0/2333379797 >> v1:172.21.15.60:6789/0 conn(0x7f87e40ce0f0 legacy=0x7f87e40cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:27.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e27fc700 1 -- 172.21.15.60:0/2333379797 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40ca820 con 0x7f87e40c8da0 2023-07-13T22:31:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e27fc700 1 --2- 172.21.15.60:0/2333379797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e40c8da0 0x7f87e40cdbb0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f87d80098f0 tx=0x7f87d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9357fc149da7223 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.440+0000 7f87e1ffb700 1 -- 172.21.15.60:0/2333379797 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800b8a0 con 0x7f87e40c8da0 2023-07-13T22:31:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e1ffb700 1 -- 172.21.15.60:0/2333379797 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d800ba00 con 0x7f87e40c8da0 2023-07-13T22:31:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e1ffb700 1 -- 172.21.15.60:0/2333379797 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800bd90 con 0x7f87e40c8da0 2023-07-13T22:31:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e93ff700 1 -- 172.21.15.60:0/2333379797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e40c8da0 msgr2=0x7f87e40cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e93ff700 1 --2- 172.21.15.60:0/2333379797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e40c8da0 0x7f87e40cdbb0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f87d80098f0 tx=0x7f87d8009c00 comp rx=0 tx=0).stop 2023-07-13T22:31:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e93ff700 1 -- 172.21.15.60:0/2333379797 shutdown_connections 2023-07-13T22:31:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e93ff700 1 --2- 172.21.15.60:0/2333379797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e415c6e0 0x7f87e415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e93ff700 1 --2- 172.21.15.60:0/2333379797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e40c8da0 0x7f87e40cdbb0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e93ff700 1 -- 172.21.15.60:0/2333379797 >> 172.21.15.60:0/2333379797 conn(0x7f87e40cf3b0 msgr2=0x7f87e40cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e93ff700 1 -- 172.21.15.60:0/2333379797 shutdown_connections 2023-07-13T22:31:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.441+0000 7f87e93ff700 1 -- 172.21.15.60:0/2333379797 wait complete. 2023-07-13T22:31:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.443+0000 7f87e93ff700 1 Processor -- start 2023-07-13T22:31:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.443+0000 7f87e93ff700 1 -- start start 2023-07-13T22:31:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.444+0000 7f87e93ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87e40c8da0 0x7f87e4156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.444+0000 7f87e93ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e40ce0f0 0x7f87e40b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.444+0000 7f87e93ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e415c6e0 0x7f87e40b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.444+0000 7f87e93ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4154400 con 0x7f87e415c6e0 2023-07-13T22:31:27.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.444+0000 7f87e93ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4153380 con 0x7f87e40c8da0 2023-07-13T22:31:27.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.444+0000 7f87e93ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4153660 con 0x7f87e40ce0f0 2023-07-13T22:31:27.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.444+0000 7f87e37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e415c6e0 0x7f87e40b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:27.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.444+0000 7f87e37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e415c6e0 0x7f87e40b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57944/0 (socket says 172.21.15.60:57944) 2023-07-13T22:31:27.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.444+0000 7f87e37fe700 1 -- 172.21.15.60:0/2155875332 learned_addr learned my addr 172.21.15.60:0/2155875332 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:27.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e27fc700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e40ce0f0 0x7f87e40b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:27.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e2ffd700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87e40c8da0 0x7f87e4156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:27.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e37fe700 1 -- 172.21.15.60:0/2155875332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e40ce0f0 msgr2=0x7f87e40b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:27.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e37fe700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e40ce0f0 0x7f87e40b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e37fe700 1 -- 172.21.15.60:0/2155875332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87e40c8da0 msgr2=0x7f87e4156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:27.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e37fe700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87e40c8da0 0x7f87e4156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e37fe700 1 -- 172.21.15.60:0/2155875332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8009550 con 0x7f87e415c6e0 2023-07-13T22:31:27.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e2ffd700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87e40c8da0 0x7f87e4156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:27.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e37fe700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e415c6e0 0x7f87e40b3280 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00cbd0 tx=0x7f87dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:27.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87cb7fe700 1 -- 172.21.15.60:0/2155875332 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00abb0 con 0x7f87e415c6e0 2023-07-13T22:31:27.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87cb7fe700 1 -- 172.21.15.60:0/2155875332 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc00ad10 con 0x7f87e415c6e0 2023-07-13T22:31:27.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87cb7fe700 1 -- 172.21.15.60:0/2155875332 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc007270 con 0x7f87e415c6e0 2023-07-13T22:31:27.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e4153940 con 0x7f87e415c6e0 2023-07-13T22:31:27.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.445+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e4153f00 con 0x7f87e415c6e0 2023-07-13T22:31:27.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.446+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e404e210 con 0x7f87e415c6e0 2023-07-13T22:31:27.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.447+0000 7f87cb7fe700 1 -- 172.21.15.60:0/2155875332 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87dc0073d0 con 0x7f87e415c6e0 2023-07-13T22:31:27.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.448+0000 7f87cb7fe700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87d006ace0 0x7f87d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:27.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.448+0000 7f87cb7fe700 1 -- 172.21.15.60:0/2155875332 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f87dc088f40 con 0x7f87e415c6e0 2023-07-13T22:31:27.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.449+0000 7f87e2ffd700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87d006ace0 0x7f87d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:27.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.449+0000 7f87e2ffd700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87d006ace0 0x7f87d006d190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009820 tx=0x7f87d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:27.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.452+0000 7f87cb7fe700 1 -- 172.21.15.60:0/2155875332 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f87dc054350 con 0x7f87e415c6e0 2023-07-13T22:31:27.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.660+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f87e40c55e0 con 0x7f87e415c6e0 2023-07-13T22:31:27.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.883+0000 7f87cb7fe700 1 -- 172.21.15.60:0/2155875332 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v132) v1 ==== 138+0+0 (secure 0 0 0) 0x7f87dc017070 con 0x7f87e415c6e0 2023-07-13T22:31:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87d006ace0 msgr2=0x7f87d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87d006ace0 0x7f87d006d190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009820 tx=0x7f87d4008040 comp rx=0 tx=0).stop 2023-07-13T22:31:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e415c6e0 msgr2=0x7f87e40b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:27.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e415c6e0 0x7f87e40b3280 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00cbd0 tx=0x7f87dc00cf90 comp rx=0 tx=0).stop 2023-07-13T22:31:27.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 shutdown_connections 2023-07-13T22:31:27.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87d006ace0 0x7f87d006d190 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87e415c6e0 0x7f87e40b3280 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87e40c8da0 0x7f87e4156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 --2- 172.21.15.60:0/2155875332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e40ce0f0 0x7f87e40b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:27.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 >> 172.21.15.60:0/2155875332 conn(0x7f87e40cf3b0 msgr2=0x7f87e40b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:27.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.887+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 shutdown_connections 2023-07-13T22:31:27.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:27.888+0000 7f87e93ff700 1 -- 172.21.15.60:0/2155875332 wait complete. 2023-07-13T22:31:27.872 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-07-13T22:31:27.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:31:28.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:31:28.102 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:31:28.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.120+0000 7ff9903ff700 1 Processor -- start 2023-07-13T22:31:28.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.120+0000 7ff9903ff700 1 -- start start 2023-07-13T22:31:28.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.121+0000 7ff9903ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9880cd6d0 0x7ff9880d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:28.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.121+0000 7ff9903ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9880cdb20 con 0x7ff988159ea0 2023-07-13T22:31:28.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.121+0000 7ff9903ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98815d200 con 0x7ff9880d37c0 2023-07-13T22:31:28.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.121+0000 7ff9903ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9880d2e60 con 0x7ff9880cd6d0 2023-07-13T22:31:28.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.121+0000 7ff98d99a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9880cd6d0 0x7ff9880d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:28.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.121+0000 7ff98d99a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9880cd6d0 0x7ff9880d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57952/0 (socket says 172.21.15.60:57952) 2023-07-13T22:31:28.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.121+0000 7ff98d99a700 1 -- 172.21.15.60:0/1730929650 learned_addr learned my addr 172.21.15.60:0/1730929650 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:28.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.122+0000 7ff98d99a700 1 -- 172.21.15.60:0/1730929650 >> v1:172.21.15.105:6789/0 conn(0x7ff9880d37c0 legacy=0x7ff9880d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:28.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.122+0000 7ff98d99a700 1 -- 172.21.15.60:0/1730929650 >> v1:172.21.15.60:6789/0 conn(0x7ff988159ea0 legacy=0x7ff98815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:28.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.122+0000 7ff98d99a700 1 -- 172.21.15.60:0/1730929650 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98815d4e0 con 0x7ff9880cd6d0 2023-07-13T22:31:28.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.122+0000 7ff98d99a700 1 --2- 172.21.15.60:0/1730929650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 0x7ff9880d3280 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ff9780098f0 tx=0x7ff978009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c2af70b40de2aa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:28.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.122+0000 7ff98d199700 1 -- 172.21.15.60:0/1730929650 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800b8a0 con 0x7ff9880cd6d0 2023-07-13T22:31:28.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.122+0000 7ff98d199700 1 -- 172.21.15.60:0/1730929650 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97800ba00 con 0x7ff9880cd6d0 2023-07-13T22:31:28.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.122+0000 7ff98d199700 1 -- 172.21.15.60:0/1730929650 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800bd90 con 0x7ff9880cd6d0 2023-07-13T22:31:28.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.123+0000 7ff9903ff700 1 -- 172.21.15.60:0/1730929650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 msgr2=0x7ff9880d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:28.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.123+0000 7ff9903ff700 1 --2- 172.21.15.60:0/1730929650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 0x7ff9880d3280 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ff9780098f0 tx=0x7ff978009c00 comp rx=0 tx=0).stop 2023-07-13T22:31:28.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.123+0000 7ff9903ff700 1 -- 172.21.15.60:0/1730929650 shutdown_connections 2023-07-13T22:31:28.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.123+0000 7ff9903ff700 1 --2- 172.21.15.60:0/1730929650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 0x7ff9880d3280 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:28.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.123+0000 7ff9903ff700 1 -- 172.21.15.60:0/1730929650 >> 172.21.15.60:0/1730929650 conn(0x7ff988071600 msgr2=0x7ff9880719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:28.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.123+0000 7ff9903ff700 1 -- 172.21.15.60:0/1730929650 shutdown_connections 2023-07-13T22:31:28.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.123+0000 7ff9903ff700 1 -- 172.21.15.60:0/1730929650 wait complete. 2023-07-13T22:31:28.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.125+0000 7ff9903ff700 1 Processor -- start 2023-07-13T22:31:28.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.125+0000 7ff9903ff700 1 -- start start 2023-07-13T22:31:28.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.126+0000 7ff9903ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 0x7ff98814c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:28.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.126+0000 7ff9903ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9880d37c0 0x7ff98814cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:28.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.126+0000 7ff9903ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff988159ea0 0x7ff9880b8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:28.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.126+0000 7ff9903ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c7e50 con 0x7ff9880cd6d0 2023-07-13T22:31:28.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.126+0000 7ff9903ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c8130 con 0x7ff9880d37c0 2023-07-13T22:31:28.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.126+0000 7ff9903ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c8410 con 0x7ff988159ea0 2023-07-13T22:31:28.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 0x7ff98814c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:28.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e19b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 0x7ff98814c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57960/0 (socket says 172.21.15.60:57960) 2023-07-13T22:31:28.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e19b700 1 -- 172.21.15.60:0/2126884638 learned_addr learned my addr 172.21.15.60:0/2126884638 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:28.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e99c700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff988159ea0 0x7ff9880b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:28.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98d99a700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9880d37c0 0x7ff98814cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:28.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e19b700 1 -- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff988159ea0 msgr2=0x7ff9880b8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:28.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e19b700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff988159ea0 0x7ff9880b8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:28.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e19b700 1 -- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9880d37c0 msgr2=0x7ff98814cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:28.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e19b700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9880d37c0 0x7ff98814cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:28.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e19b700 1 -- 172.21.15.60:0/2126884638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff978009550 con 0x7ff9880cd6d0 2023-07-13T22:31:28.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98d99a700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9880d37c0 0x7ff98814cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:28.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e99c700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff988159ea0 0x7ff9880b8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:28.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.127+0000 7ff98e19b700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 0x7ff98814c310 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7ff98400c450 tx=0x7ff98400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:28.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.128+0000 7ff97effd700 1 -- 172.21.15.60:0/2126884638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9840052d0 con 0x7ff9880cd6d0 2023-07-13T22:31:28.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.128+0000 7ff97effd700 1 -- 172.21.15.60:0/2126884638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff984005430 con 0x7ff9880cd6d0 2023-07-13T22:31:28.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.128+0000 7ff97effd700 1 -- 172.21.15.60:0/2126884638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9840038c0 con 0x7ff9880cd6d0 2023-07-13T22:31:28.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.128+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9880c86f0 con 0x7ff9880cd6d0 2023-07-13T22:31:28.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.128+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98814c850 con 0x7ff9880cd6d0 2023-07-13T22:31:28.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.129+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9880c8d40 con 0x7ff9880cd6d0 2023-07-13T22:31:28.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.131+0000 7ff97effd700 1 -- 172.21.15.60:0/2126884638 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff984003a20 con 0x7ff9880cd6d0 2023-07-13T22:31:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.132+0000 7ff97effd700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff96c06ace0 0x7ff96c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.132+0000 7ff97effd700 1 -- 172.21.15.60:0/2126884638 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5580+0+0 (secure 0 0 0) 0x7ff9840886e0 con 0x7ff9880cd6d0 2023-07-13T22:31:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.132+0000 7ff98d99a700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff96c06ace0 0x7ff96c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:28.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.133+0000 7ff98d99a700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff96c06ace0 0x7ff96c06d190 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff978009c40 tx=0x7ff978008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:28.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.137+0000 7ff97effd700 1 -- 172.21.15.60:0/2126884638 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff9840538d0 con 0x7ff9880cd6d0 2023-07-13T22:31:28.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.344+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff988067a70 con 0x7ff9880cd6d0 2023-07-13T22:31:28.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.893+0000 7ff97effd700 1 -- 172.21.15.60:0/2126884638 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v133) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff984056ef0 con 0x7ff9880cd6d0 2023-07-13T22:31:28.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.900+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff96c06ace0 msgr2=0x7ff96c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:28.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.900+0000 7ff9903ff700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff96c06ace0 0x7ff96c06d190 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff978009c40 tx=0x7ff978008030 comp rx=0 tx=0).stop 2023-07-13T22:31:28.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.900+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 msgr2=0x7ff98814c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:28.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.900+0000 7ff9903ff700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 0x7ff98814c310 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7ff98400c450 tx=0x7ff98400c810 comp rx=0 tx=0).stop 2023-07-13T22:31:28.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.902+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 shutdown_connections 2023-07-13T22:31:28.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.902+0000 7ff9903ff700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff96c06ace0 0x7ff96c06d190 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:28.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.902+0000 7ff9903ff700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9880cd6d0 0x7ff98814c310 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:28.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.902+0000 7ff9903ff700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff988159ea0 0x7ff9880b8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:28.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.902+0000 7ff9903ff700 1 --2- 172.21.15.60:0/2126884638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9880d37c0 0x7ff98814cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:28.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.902+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 >> 172.21.15.60:0/2126884638 conn(0x7ff988071600 msgr2=0x7ff9880d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:28.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.902+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 shutdown_connections 2023-07-13T22:31:28.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:28.903+0000 7ff9903ff700 1 -- 172.21.15.60:0/2126884638 wait complete. 2023-07-13T22:31:28.893 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-07-13T22:31:28.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:31:29.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.137+0000 7fc6fb5ff700 1 Processor -- start 2023-07-13T22:31:29.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.137+0000 7fc6fb5ff700 1 -- start start 2023-07-13T22:31:29.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.138+0000 7fc6fb5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 0x7fc6f40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:29.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6fb5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc6f40b1ac0 0x7fc6f40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:29.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6fb5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6f40cede0 0x7fc6f415ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:29.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6fb5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f40d1870 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6fb5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f40cf370 con 0x7fc6f40cede0 2023-07-13T22:31:29.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6fb5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40d12b0 con 0x7fc6f40b8200 2023-07-13T22:31:29.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f8b9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc6f40b1ac0 0x7fc6f40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f8b9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc6f40b1ac0 0x7fc6f40cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57976/0 (socket says 172.21.15.60:57976) 2023-07-13T22:31:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f8b9a700 1 -- 172.21.15.60:0/858137722 learned_addr learned my addr 172.21.15.60:0/858137722 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f9b9c700 1 --2- 172.21.15.60:0/858137722 >> v2:172.21.15.105:3300/0 conn(0x7fc6f40cede0 0x7fc6f415ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f939b700 1 --2- 172.21.15.60:0/858137722 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 0x7fc6f40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:29.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f8b9a700 1 -- 172.21.15.60:0/858137722 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 msgr2=0x7fc6f40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:29.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f8b9a700 1 --2- 172.21.15.60:0/858137722 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 0x7fc6f40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f8b9a700 1 -- 172.21.15.60:0/858137722 >> v2:172.21.15.105:3300/0 conn(0x7fc6f40cede0 msgr2=0x7fc6f415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:29.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f8b9a700 1 --2- 172.21.15.60:0/858137722 >> v2:172.21.15.105:3300/0 conn(0x7fc6f40cede0 0x7fc6f415ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.139+0000 7fc6f8b9a700 1 -- 172.21.15.60:0/858137722 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40d1590 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.140+0000 7fc6f8b9a700 1 --2- 172.21.15.60:0/858137722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f40cfb80 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40b8d70 tx=0x7fc6e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63507557e14f5fe8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.140+0000 7fc6ebfff700 1 -- 172.21.15.60:0/858137722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e400b870 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.140+0000 7fc6ebfff700 1 -- 172.21.15.60:0/858137722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e400b9d0 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.140+0000 7fc6ebfff700 1 -- 172.21.15.60:0/858137722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e400bd60 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.140+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/858137722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 msgr2=0x7fc6f40cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.140+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/858137722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f40cfb80 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40b8d70 tx=0x7fc6e4009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.141+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/858137722 shutdown_connections 2023-07-13T22:31:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.141+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/858137722 >> v2:172.21.15.105:3300/0 conn(0x7fc6f40cede0 0x7fc6f415ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.141+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/858137722 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40b8200 0x7fc6f40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.141+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/858137722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f40cfb80 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.141+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/858137722 >> 172.21.15.60:0/858137722 conn(0x7fc6f406b2f0 msgr2=0x7fc6f406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.141+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/858137722 shutdown_connections 2023-07-13T22:31:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.141+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/858137722 wait complete. 2023-07-13T22:31:29.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.143+0000 7fc6fb5ff700 1 Processor -- start 2023-07-13T22:31:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.143+0000 7fc6fb5ff700 1 -- start start 2023-07-13T22:31:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.144+0000 7fc6fb5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f41f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6fb5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b8200 0x7fc6f40b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6fb5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 0x7fc6f40b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:29.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6fb5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f41f3160 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6fb5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f41f6680 con 0x7fc6f40b8200 2023-07-13T22:31:29.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6fb5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f41f2550 con 0x7fc6f40cede0 2023-07-13T22:31:29.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f939b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f41f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:29.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f939b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f41f6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57990/0 (socket says 172.21.15.60:57990) 2023-07-13T22:31:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f939b700 1 -- 172.21.15.60:0/1729316797 learned_addr learned my addr 172.21.15.60:0/1729316797 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f8b9a700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b8200 0x7fc6f40b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f9b9c700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 0x7fc6f40b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f939b700 1 -- 172.21.15.60:0/1729316797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 msgr2=0x7fc6f40b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:29.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f939b700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 0x7fc6f40b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f939b700 1 -- 172.21.15.60:0/1729316797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b8200 msgr2=0x7fc6f40b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f939b700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b8200 0x7fc6f40b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f939b700 1 -- 172.21.15.60:0/1729316797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4009550 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.145+0000 7fc6f9b9c700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 0x7fc6f40b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:29.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.146+0000 7fc6f939b700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f41f6140 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000e840 tx=0x7fc6f000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:29.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.146+0000 7fc6e9ffb700 1 -- 172.21.15.60:0/1729316797 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0013610 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.146+0000 7fc6e9ffb700 1 -- 172.21.15.60:0/1729316797 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f0013770 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.146+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f41f2830 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.146+0000 7fc6e9ffb700 1 -- 172.21.15.60:0/1729316797 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0013b20 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.146+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f41f2df0 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.147+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f41e1900 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.148+0000 7fc6e9ffb700 1 -- 172.21.15.60:0/1729316797 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6f0013c80 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.149+0000 7fc6e9ffb700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6d806ace0 0x7fc6d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.149+0000 7fc6e9ffb700 1 -- 172.21.15.60:0/1729316797 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc6f0089540 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.150+0000 7fc6f8b9a700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6d806ace0 0x7fc6d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.150+0000 7fc6f8b9a700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6d806ace0 0x7fc6d806d190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4000c00 tx=0x7fc6e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:29.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.154+0000 7fc6e9ffb700 1 -- 172.21.15.60:0/1729316797 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc6f00546e0 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.406+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc6f404e210 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.916+0000 7fc6e9ffb700 1 -- 172.21.15.60:0/1729316797 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v93) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc6f0057d00 con 0x7fc6f40b1ac0 2023-07-13T22:31:29.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.925+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6d806ace0 msgr2=0x7fc6d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:29.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.925+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6d806ace0 0x7fc6d806d190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4000c00 tx=0x7fc6e4008030 comp rx=0 tx=0).stop 2023-07-13T22:31:29.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.925+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 msgr2=0x7fc6f41f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:29.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.925+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f41f6140 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000e840 tx=0x7fc6f000eb50 comp rx=0 tx=0).stop 2023-07-13T22:31:29.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.926+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 shutdown_connections 2023-07-13T22:31:29.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.926+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc6d806ace0 0x7fc6d806d190 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.926+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6f40b1ac0 0x7fc6f41f6140 secure :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000e840 tx=0x7fc6f000eb50 comp rx=0 tx=0).stop 2023-07-13T22:31:29.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.926+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc6f40cede0 0x7fc6f40b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.926+0000 7fc6fb5ff700 1 --2- 172.21.15.60:0/1729316797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6f40b8200 0x7fc6f40b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.926+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 >> 172.21.15.60:0/1729316797 conn(0x7fc6f406b2f0 msgr2=0x7fc6f406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.928+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 shutdown_connections 2023-07-13T22:31:29.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:29.928+0000 7fc6fb5ff700 1 -- 172.21.15.60:0/1729316797 wait complete. 2023-07-13T22:31:29.913 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 21 and data pool 22 2023-07-13T22:31:29.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:30.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.154+0000 7f58d53ff700 1 Processor -- start 2023-07-13T22:31:30.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.154+0000 7f58d53ff700 1 -- start start 2023-07-13T22:31:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.155+0000 7f58d53ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58d00706f0 0x7f58d00ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.155+0000 7f58d53ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d015d970 con 0x7f58d0159170 2023-07-13T22:31:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.155+0000 7f58d53ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d015cd60 con 0x7f58d0070270 2023-07-13T22:31:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.155+0000 7f58d53ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d015d040 con 0x7f58d00706f0 2023-07-13T22:31:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.156+0000 7f58cf7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58d00706f0 0x7f58d00ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.156+0000 7f58ceffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f58d0159170 0x7f58d00c4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39156/0 (socket says 172.21.15.60:39156) 2023-07-13T22:31:30.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.156+0000 7f58ceffd700 1 -- 172.21.15.60:0/1642828359 learned_addr learned my addr 172.21.15.60:0/1642828359 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.156+0000 7f58cf7fe700 1 -- 172.21.15.60:0/1642828359 >> v1:172.21.15.105:6789/0 conn(0x7f58d0070270 legacy=0x7f58d00c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.156+0000 7f58cf7fe700 1 -- 172.21.15.60:0/1642828359 >> v1:172.21.15.60:6789/0 conn(0x7f58d0159170 legacy=0x7f58d00c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.156+0000 7f58cf7fe700 1 -- 172.21.15.60:0/1642828359 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d015d320 con 0x7f58d00706f0 2023-07-13T22:31:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.156+0000 7f58cdffb700 1 -- 172.21.15.60:0/1642828359 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043994222 0 0) 0x7f58d015d970 con 0x7f58d0159170 2023-07-13T22:31:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.156+0000 7f58cf7fe700 1 --2- 172.21.15.60:0/1642828359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d00706f0 0x7f58d00ca360 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009d90 tx=0x7f58c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecfce393e9a70f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58cdffb700 1 -- 172.21.15.60:0/1642828359 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c800e070 con 0x7f58d00706f0 2023-07-13T22:31:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58cdffb700 1 -- 172.21.15.60:0/1642828359 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c800bb90 con 0x7f58d00706f0 2023-07-13T22:31:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58cdffb700 1 -- 172.21.15.60:0/1642828359 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8003a20 con 0x7f58d00706f0 2023-07-13T22:31:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58d53ff700 1 -- 172.21.15.60:0/1642828359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d00706f0 msgr2=0x7f58d00ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58d53ff700 1 --2- 172.21.15.60:0/1642828359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d00706f0 0x7f58d00ca360 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009d90 tx=0x7f58c800b330 comp rx=0 tx=0).stop 2023-07-13T22:31:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58d53ff700 1 -- 172.21.15.60:0/1642828359 shutdown_connections 2023-07-13T22:31:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58d53ff700 1 --2- 172.21.15.60:0/1642828359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d00706f0 0x7f58d00ca360 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58d53ff700 1 -- 172.21.15.60:0/1642828359 >> 172.21.15.60:0/1642828359 conn(0x7f58d00cf410 msgr2=0x7f58d00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58d53ff700 1 -- 172.21.15.60:0/1642828359 shutdown_connections 2023-07-13T22:31:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.157+0000 7f58d53ff700 1 -- 172.21.15.60:0/1642828359 wait complete. 2023-07-13T22:31:30.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.159+0000 7f58d53ff700 1 Processor -- start 2023-07-13T22:31:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.159+0000 7f58d53ff700 1 -- start start 2023-07-13T22:31:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.160+0000 7f58d53ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58d0070270 0x7f58d00af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.160+0000 7f58d53ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58d00706f0 0x7f58d014a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.160+0000 7f58d53ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d0159170 0x7f58d014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.160+0000 7f58d53ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d00c75e0 con 0x7f58d0159170 2023-07-13T22:31:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.160+0000 7f58d53ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d00c7830 con 0x7f58d00706f0 2023-07-13T22:31:30.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.160+0000 7f58d53ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d00c7b10 con 0x7f58d0070270 2023-07-13T22:31:30.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.160+0000 7f58cf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d0159170 0x7f58d014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.160+0000 7f58cf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d0159170 0x7f58d014ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58002/0 (socket says 172.21.15.60:58002) 2023-07-13T22:31:30.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.160+0000 7f58cf7fe700 1 -- 172.21.15.60:0/2512972367 learned_addr learned my addr 172.21.15.60:0/2512972367 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:30.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58ce7fc700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58d00706f0 0x7f58d014a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58ceffd700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58d0070270 0x7f58d00af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58cf7fe700 1 -- 172.21.15.60:0/2512972367 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58d0070270 msgr2=0x7f58d00af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58cf7fe700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58d0070270 0x7f58d00af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58cf7fe700 1 -- 172.21.15.60:0/2512972367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58d00706f0 msgr2=0x7f58d014a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58cf7fe700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58d00706f0 0x7f58d014a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58cf7fe700 1 -- 172.21.15.60:0/2512972367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8009a70 con 0x7f58d0159170 2023-07-13T22:31:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58cf7fe700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d0159170 0x7f58d014ab80 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f58c800b730 tx=0x7f58c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58b77fe700 1 -- 172.21.15.60:0/2512972367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8012840 con 0x7f58d0159170 2023-07-13T22:31:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58b77fe700 1 -- 172.21.15.60:0/2512972367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c80129a0 con 0x7f58d0159170 2023-07-13T22:31:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58b77fe700 1 -- 172.21.15.60:0/2512972367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8012d50 con 0x7f58d0159170 2023-07-13T22:31:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.161+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d00c7df0 con 0x7f58d0159170 2023-07-13T22:31:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.162+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d00c83b0 con 0x7f58d0159170 2023-07-13T22:31:30.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.163+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d00ad860 con 0x7f58d0159170 2023-07-13T22:31:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.163+0000 7f58b77fe700 1 -- 172.21.15.60:0/2512972367 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f58c801d420 con 0x7f58d0159170 2023-07-13T22:31:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.164+0000 7f58b77fe700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f58bc06ace0 0x7f58bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.164+0000 7f58b77fe700 1 -- 172.21.15.60:0/2512972367 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f58c808b7b0 con 0x7f58d0159170 2023-07-13T22:31:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.165+0000 7f58ceffd700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f58bc06ace0 0x7f58bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.165+0000 7f58ceffd700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f58bc06ace0 0x7f58bc06d190 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000f60 tx=0x7f58c0007d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:30.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.169+0000 7f58b77fe700 1 -- 172.21.15.60:0/2512972367 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f58c8056850 con 0x7f58d0159170 2023-07-13T22:31:30.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.379+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58d0071440 con 0x7f58d0159170 2023-07-13T22:31:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.380+0000 7f58b77fe700 1 -- 172.21.15.60:0/2512972367 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+18220 (secure 0 0 0) 0x7f58c8059e70 con 0x7f58d0159170 2023-07-13T22:31:30.363 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:30.363 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":134,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:29.408119+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:31:27.702135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-13T22:31:28.386570+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:30.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f58bc06ace0 msgr2=0x7f58bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f58bc06ace0 0x7f58bc06d190 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000f60 tx=0x7f58c0007d90 comp rx=0 tx=0).stop 2023-07-13T22:31:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d0159170 msgr2=0x7f58d014ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d0159170 0x7f58d014ab80 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f58c800b730 tx=0x7f58c8004ad0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 shutdown_connections 2023-07-13T22:31:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f58bc06ace0 0x7f58bc06d190 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d0159170 0x7f58d014ab80 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58d0070270 0x7f58d00af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 --2- 172.21.15.60:0/2512972367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58d00706f0 0x7f58d014a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 >> 172.21.15.60:0/2512972367 conn(0x7f58d00cf410 msgr2=0x7f58d0069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:30.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 shutdown_connections 2023-07-13T22:31:30.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.383+0000 7f58d53ff700 1 -- 172.21.15.60:0/2512972367 wait complete. 2023-07-13T22:31:30.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.604+0000 7f820a1ff700 1 Processor -- start 2023-07-13T22:31:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.604+0000 7f820a1ff700 1 -- start start 2023-07-13T22:31:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.605+0000 7f820a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8204158870 0x7f82040c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.605+0000 7f820a1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f820406a5e0 0x7f82040c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.605+0000 7f820a1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040c3fc0 con 0x7f82040c52c0 2023-07-13T22:31:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.605+0000 7f820a1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82040c42a0 con 0x7f820406a5e0 2023-07-13T22:31:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.605+0000 7f820a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82040c4580 con 0x7f8204158870 2023-07-13T22:31:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.605+0000 7f8203fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f820406a5e0 0x7f82040c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.606+0000 7f8203fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f820406a5e0 0x7f82040c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58006/0 (socket says 172.21.15.60:58006) 2023-07-13T22:31:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.606+0000 7f8203fff700 1 -- 172.21.15.60:0/1666004632 learned_addr learned my addr 172.21.15.60:0/1666004632 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.606+0000 7f82037fe700 1 --2- 172.21.15.60:0/1666004632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8204158870 0x7f82040c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.606+0000 7f8203fff700 1 -- 172.21.15.60:0/1666004632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8204158870 msgr2=0x7f82040c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.606+0000 7f8203fff700 1 --2- 172.21.15.60:0/1666004632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8204158870 0x7f82040c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.606+0000 7f8203fff700 1 -- 172.21.15.60:0/1666004632 >> v1:172.21.15.105:6789/0 conn(0x7f82040c52c0 legacy=0x7f8204069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.606+0000 7f8203fff700 1 -- 172.21.15.60:0/1666004632 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040c4860 con 0x7f820406a5e0 2023-07-13T22:31:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.606+0000 7f8203fff700 1 --2- 172.21.15.60:0/1666004632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820406a5e0 0x7f82040c9300 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009e50 tx=0x7f81fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24636cc4e566a6ec server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f82027fc700 1 -- 172.21.15.60:0/1666004632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00e070 con 0x7f820406a5e0 2023-07-13T22:31:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f82027fc700 1 -- 172.21.15.60:0/1666004632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc00b910 con 0x7f820406a5e0 2023-07-13T22:31:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f82027fc700 1 -- 172.21.15.60:0/1666004632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00bcc0 con 0x7f820406a5e0 2023-07-13T22:31:30.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f820a1ff700 1 -- 172.21.15.60:0/1666004632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820406a5e0 msgr2=0x7f82040c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f820a1ff700 1 --2- 172.21.15.60:0/1666004632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820406a5e0 0x7f82040c9300 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009e50 tx=0x7f81fc00b330 comp rx=0 tx=0).stop 2023-07-13T22:31:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f820a1ff700 1 -- 172.21.15.60:0/1666004632 shutdown_connections 2023-07-13T22:31:30.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f820a1ff700 1 --2- 172.21.15.60:0/1666004632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820406a5e0 0x7f82040c9300 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f820a1ff700 1 --2- 172.21.15.60:0/1666004632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8204158870 0x7f82040c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f820a1ff700 1 -- 172.21.15.60:0/1666004632 >> 172.21.15.60:0/1666004632 conn(0x7f82040cf410 msgr2=0x7f82040cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:30.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f820a1ff700 1 -- 172.21.15.60:0/1666004632 shutdown_connections 2023-07-13T22:31:30.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.607+0000 7f820a1ff700 1 -- 172.21.15.60:0/1666004632 wait complete. 2023-07-13T22:31:30.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.609+0000 7f820a1ff700 1 Processor -- start 2023-07-13T22:31:30.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.609+0000 7f820a1ff700 1 -- start start 2023-07-13T22:31:30.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.610+0000 7f820a1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820406a5e0 0x7f82041495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.610+0000 7f820a1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82040c52c0 0x7f82040c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.610+0000 7f820a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8204158870 0x7f82041e2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.610+0000 7f820a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8204149b00 con 0x7f8204158870 2023-07-13T22:31:30.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.610+0000 7f820a1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820415a330 con 0x7f820406a5e0 2023-07-13T22:31:30.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.610+0000 7f820a1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820415a610 con 0x7f82040c52c0 2023-07-13T22:31:30.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8203fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8204158870 0x7f82041e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8203fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8204158870 0x7f82041e2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58012/0 (socket says 172.21.15.60:58012) 2023-07-13T22:31:30.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8203fff700 1 -- 172.21.15.60:0/873377163 learned_addr learned my addr 172.21.15.60:0/873377163 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:30.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8203fff700 1 -- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82040c52c0 msgr2=0x7f82040c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8202ffd700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82040c52c0 0x7f82040c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f82037fe700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820406a5e0 0x7f82041495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8203fff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82040c52c0 0x7f82040c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8203fff700 1 -- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820406a5e0 msgr2=0x7f82041495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8203fff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820406a5e0 0x7f82041495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8203fff700 1 -- 172.21.15.60:0/873377163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc009b30 con 0x7f8204158870 2023-07-13T22:31:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8202ffd700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82040c52c0 0x7f82040c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f82037fe700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820406a5e0 0x7f82041495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f8203fff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8204158870 0x7f82041e2410 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f81fc012000 tx=0x7f81fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f81ebfff700 1 -- 172.21.15.60:0/873377163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc003ea0 con 0x7f8204158870 2023-07-13T22:31:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f81ebfff700 1 -- 172.21.15.60:0/873377163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc0099b0 con 0x7f8204158870 2023-07-13T22:31:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820415a8f0 con 0x7f8204158870 2023-07-13T22:31:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.611+0000 7f81ebfff700 1 -- 172.21.15.60:0/873377163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc01c680 con 0x7f8204158870 2023-07-13T22:31:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.612+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820415aeb0 con 0x7f8204158870 2023-07-13T22:31:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.613+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82040c39f0 con 0x7f8204158870 2023-07-13T22:31:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.614+0000 7f81ebfff700 1 -- 172.21.15.60:0/873377163 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f81fc01c7e0 con 0x7f8204158870 2023-07-13T22:31:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.615+0000 7f81ebfff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81f006ace0 0x7f81f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:30.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.615+0000 7f81ebfff700 1 -- 172.21.15.60:0/873377163 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f81fc08aed0 con 0x7f8204158870 2023-07-13T22:31:30.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.615+0000 7f82037fe700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81f006ace0 0x7f81f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:30.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.616+0000 7f82037fe700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81f006ace0 0x7f81f006d190 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009ac0 tx=0x7f81f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:30.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.619+0000 7f81ebfff700 1 -- 172.21.15.60:0/873377163 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f81fc055f70 con 0x7f8204158870 2023-07-13T22:31:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.829+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82040cbe30 con 0x7f8204158870 2023-07-13T22:31:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.830+0000 7f81ebfff700 1 -- 172.21.15.60:0/873377163 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+18220 (secure 0 0 0) 0x7f81fc059590 con 0x7f8204158870 2023-07-13T22:31:30.813 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:30.813 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":134,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:29.408119+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:31:27.702135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-13T22:31:28.386570+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:30.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81f006ace0 msgr2=0x7f81f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81f006ace0 0x7f81f006d190 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009ac0 tx=0x7f81f4008040 comp rx=0 tx=0).stop 2023-07-13T22:31:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8204158870 msgr2=0x7f82041e2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8204158870 0x7f82041e2410 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f81fc012000 tx=0x7f81fc004ad0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 shutdown_connections 2023-07-13T22:31:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f81f006ace0 0x7f81f006d190 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8204158870 0x7f82041e2410 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f820406a5e0 0x7f82041495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 --2- 172.21.15.60:0/873377163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f82040c52c0 0x7f82040c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:30.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 >> 172.21.15.60:0/873377163 conn(0x7f82040cf410 msgr2=0x7f820406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:30.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 shutdown_connections 2023-07-13T22:31:30.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:30.833+0000 7f820a1ff700 1 -- 172.21.15.60:0/873377163 wait complete. 2023-07-13T22:31:30.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:31:31.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.060+0000 7fbcc3fff700 1 Processor -- start 2023-07-13T22:31:31.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.060+0000 7fbcc3fff700 1 -- start start 2023-07-13T22:31:31.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.061+0000 7fbcc3fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbcbc06a5e0 0x7fbcbc06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:31.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.061+0000 7fbcc3fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc0c8e90 con 0x7fbcbc0add00 2023-07-13T22:31:31.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.061+0000 7fbcc3fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc0ae0d0 con 0x7fbcbc0ca9f0 2023-07-13T22:31:31.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.061+0000 7fbcc3fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc06d3b0 con 0x7fbcbc06a5e0 2023-07-13T22:31:31.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.062+0000 7fbcc1d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbcbc0add00 0x7fbcbc069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39170/0 (socket says 172.21.15.60:39170) 2023-07-13T22:31:31.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.062+0000 7fbcc1d9b700 1 -- 172.21.15.60:0/1803558093 learned_addr learned my addr 172.21.15.60:0/1803558093 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:31.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.062+0000 7fbcc159a700 1 --2- 172.21.15.60:0/1803558093 >> v2:172.21.15.105:3300/0 conn(0x7fbcbc06a5e0 0x7fbcbc06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:31.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.062+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869861901 0 0) 0x7fbcbc0c8e90 con 0x7fbcbc0add00 2023-07-13T22:31:31.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.062+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca80035b0 con 0x7fbcbc0add00 2023-07-13T22:31:31.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.062+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2534627683 0 0) 0x7fbca80035b0 con 0x7fbcbc0add00 2023-07-13T22:31:31.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.062+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 >> v2:172.21.15.105:3300/0 conn(0x7fbcbc06a5e0 msgr2=0x7fbcbc06ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:31.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.062+0000 7fbcc0d99700 1 --2- 172.21.15.60:0/1803558093 >> v2:172.21.15.105:3300/0 conn(0x7fbcbc06a5e0 0x7fbcbc06ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.062+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 >> v1:172.21.15.105:6789/0 conn(0x7fbcbc0ca9f0 legacy=0x7fbcbc0b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:31.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0c90e0 con 0x7fbcbc0add00 2023-07-13T22:31:31.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fbcb8001670 con 0x7fbcbc0add00 2023-07-13T22:31:31.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 >> v1:172.21.15.60:6789/0 conn(0x7fbcbc0add00 legacy=0x7fbcbc069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:31.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc0d99700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbca8003a50 0x7fbca8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:31.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc0d99700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca8004850 0x7fbca8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:31.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc0d99700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbca8005200 0x7fbca800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:31.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc06d3b0 con 0x7fbca8004850 2023-07-13T22:31:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca800c0e0 con 0x7fbca8003a50 2023-07-13T22:31:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca800c2a0 con 0x7fbca8005200 2023-07-13T22:31:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc159a700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca8004850 0x7fbca8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc1d9b700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbca8003a50 0x7fbca8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc259c700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbca8005200 0x7fbca800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc159a700 1 -- 172.21.15.60:0/1803558093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbca8005200 msgr2=0x7fbca800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc159a700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbca8005200 0x7fbca800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc159a700 1 -- 172.21.15.60:0/1803558093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbca8003a50 msgr2=0x7fbca8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:31.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc159a700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbca8003a50 0x7fbca8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc159a700 1 -- 172.21.15.60:0/1803558093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80035b0 con 0x7fbca8004850 2023-07-13T22:31:31.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.063+0000 7fbcc259c700 1 -- 172.21.15.60:0/1803558093 reap_dead start 2023-07-13T22:31:31.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.064+0000 7fbcc159a700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca8004850 0x7fbca8004cc0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009b30 tx=0x7fbca8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd2c484597fff6a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:31.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.064+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac00e6c0 con 0x7fbca8004850 2023-07-13T22:31:31.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.064+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac00e820 con 0x7fbca8004850 2023-07-13T22:31:31.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.064+0000 7fbcc0d99700 1 -- 172.21.15.60:0/1803558093 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac00eb90 con 0x7fbca8004850 2023-07-13T22:31:31.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.064+0000 7fbcc3fff700 1 -- 172.21.15.60:0/1803558093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca8004850 msgr2=0x7fbca8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:31.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.064+0000 7fbcc3fff700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca8004850 0x7fbca8004cc0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009b30 tx=0x7fbca8005880 comp rx=0 tx=0).stop 2023-07-13T22:31:31.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.064+0000 7fbcc3fff700 1 -- 172.21.15.60:0/1803558093 shutdown_connections 2023-07-13T22:31:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.064+0000 7fbcc3fff700 1 --2- 172.21.15.60:0/1803558093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbca8004850 0x7fbca8004cc0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.064+0000 7fbcc3fff700 1 -- 172.21.15.60:0/1803558093 >> 172.21.15.60:0/1803558093 conn(0x7fbcbc0cbf50 msgr2=0x7fbcbc0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.065+0000 7fbcc3fff700 1 -- 172.21.15.60:0/1803558093 shutdown_connections 2023-07-13T22:31:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.065+0000 7fbcc3fff700 1 -- 172.21.15.60:0/1803558093 wait complete. 2023-07-13T22:31:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.066+0000 7fbcc3fff700 1 Processor -- start 2023-07-13T22:31:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.066+0000 7fbcc3fff700 1 -- start start 2023-07-13T22:31:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.067+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbca8004850 0x7fbcbc0aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.067+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcbc15cdd0 0x7fbcbc14b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.067+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcbc15d240 0x7fbcbc14be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.067+0000 7fbcc3fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc1592e0 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.067+0000 7fbcc3fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc0aeeb0 con 0x7fbcbc15d240 2023-07-13T22:31:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.067+0000 7fbcc3fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc0af190 con 0x7fbca8004850 2023-07-13T22:31:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc159a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcbc15cdd0 0x7fbcbc14b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc159a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcbc15cdd0 0x7fbcbc14b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58032/0 (socket says 172.21.15.60:58032) 2023-07-13T22:31:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc159a700 1 -- 172.21.15.60:0/912879773 learned_addr learned my addr 172.21.15.60:0/912879773 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc1d9b700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbca8004850 0x7fbcbc0aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc259c700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcbc15d240 0x7fbcbc14be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc159a700 1 -- 172.21.15.60:0/912879773 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbca8004850 msgr2=0x7fbcbc0aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc159a700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbca8004850 0x7fbcbc0aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc159a700 1 -- 172.21.15.60:0/912879773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcbc15d240 msgr2=0x7fbcbc14be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc159a700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcbc15d240 0x7fbcbc14be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc159a700 1 -- 172.21.15.60:0/912879773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0af470 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.068+0000 7fbcc159a700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcbc15cdd0 0x7fbcbc14b910 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0090b0 tx=0x7fbcac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.069+0000 7fbcb27fc700 1 -- 172.21.15.60:0/912879773 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac00e6c0 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.069+0000 7fbcb27fc700 1 -- 172.21.15.60:0/912879773 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac003ad0 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.069+0000 7fbcb27fc700 1 -- 172.21.15.60:0/912879773 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac003e80 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.069+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc0af750 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.069+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc0b0490 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.070+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc0afda0 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.071+0000 7fbcb27fc700 1 -- 172.21.15.60:0/912879773 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbcac013470 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.072+0000 7fbcb27fc700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbca006ace0 0x7fbca006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.072+0000 7fbcb27fc700 1 -- 172.21.15.60:0/912879773 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcac08ac70 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.072+0000 7fbcc1d9b700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbca006ace0 0x7fbca006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.073+0000 7fbcc1d9b700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbca006ace0 0x7fbca006d190 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8001050 tx=0x7fbcb8000fc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.076+0000 7fbcb27fc700 1 -- 172.21.15.60:0/912879773 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbcac055d10 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.330+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbcbc04e210 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.920+0000 7fbcb27fc700 1 -- 172.21.15.60:0/912879773 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v96) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbcac059330 con 0x7fbcbc15cdd0 2023-07-13T22:31:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbca006ace0 msgr2=0x7fbca006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbca006ace0 0x7fbca006d190 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8001050 tx=0x7fbcb8000fc0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcbc15cdd0 msgr2=0x7fbcbc14b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcbc15cdd0 0x7fbcbc14b910 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0090b0 tx=0x7fbcac0047c0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 shutdown_connections 2023-07-13T22:31:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbca006ace0 0x7fbca006d190 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcbc15d240 0x7fbcbc14be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbca8004850 0x7fbcbc0aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 --2- 172.21.15.60:0/912879773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcbc15cdd0 0x7fbcbc14b910 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 >> 172.21.15.60:0/912879773 conn(0x7fbcbc0cbf50 msgr2=0x7fbcbc0ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 shutdown_connections 2023-07-13T22:31:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:31.923+0000 7fbcc3fff700 1 -- 172.21.15.60:0/912879773 wait complete. 2023-07-13T22:31:31.926 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-13T22:31:31.927 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-13T22:31:32.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.154+0000 7f7178dff700 1 Processor -- start 2023-07-13T22:31:32.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.154+0000 7f7178dff700 1 -- start start 2023-07-13T22:31:32.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.155+0000 7f7178dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f717415c530 0x7f71740b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:32.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.155+0000 7f7178dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71740d11c0 0x7f71740cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:32.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.155+0000 7f7178dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71740cf9f0 0x7f71740d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:32.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.155+0000 7f7178dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71740cefd0 con 0x7f71740d11c0 2023-07-13T22:31:32.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.155+0000 7f7178dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71740d1780 con 0x7f71740cf9f0 2023-07-13T22:31:32.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.155+0000 7f7178dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71740cd2d0 con 0x7f717415c530 2023-07-13T22:31:32.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.155+0000 7f7171d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71740d11c0 0x7f71740cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:32.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.155+0000 7f7171d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71740d11c0 0x7f71740cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58044/0 (socket says 172.21.15.60:58044) 2023-07-13T22:31:32.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.155+0000 7f7171d9b700 1 -- 172.21.15.60:0/1907788781 learned_addr learned my addr 172.21.15.60:0/1907788781 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:32.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f7171d9b700 1 -- 172.21.15.60:0/1907788781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f717415c530 msgr2=0x7f71740b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:32.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f7171d9b700 1 --2- 172.21.15.60:0/1907788781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f717415c530 0x7f71740b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f7172d9d700 1 --2- 172.21.15.60:0/1907788781 >> v2:172.21.15.105:3300/0 conn(0x7f71740cf9f0 0x7f71740d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:32.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f7171d9b700 1 -- 172.21.15.60:0/1907788781 >> v2:172.21.15.105:3300/0 conn(0x7f71740cf9f0 msgr2=0x7f71740d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:32.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f7171d9b700 1 --2- 172.21.15.60:0/1907788781 >> v2:172.21.15.105:3300/0 conn(0x7f71740cf9f0 0x7f71740d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f7171d9b700 1 -- 172.21.15.60:0/1907788781 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71740cd5b0 con 0x7f71740d11c0 2023-07-13T22:31:32.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f7172d9d700 1 --2- 172.21.15.60:0/1907788781 >> v2:172.21.15.105:3300/0 conn(0x7f71740cf9f0 0x7f71740d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:32.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f7171d9b700 1 --2- 172.21.15.60:0/1907788781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740d11c0 0x7f71740cf4b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f71740b4660 tx=0x7f7168009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717808f9394bbf9c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:32.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f717159a700 1 -- 172.21.15.60:0/1907788781 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800b870 con 0x7f71740d11c0 2023-07-13T22:31:32.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f717159a700 1 -- 172.21.15.60:0/1907788781 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716800b9d0 con 0x7f71740d11c0 2023-07-13T22:31:32.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.156+0000 7f717159a700 1 -- 172.21.15.60:0/1907788781 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800bd60 con 0x7f71740d11c0 2023-07-13T22:31:32.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.157+0000 7f7178dff700 1 -- 172.21.15.60:0/1907788781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740d11c0 msgr2=0x7f71740cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:32.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.157+0000 7f7178dff700 1 --2- 172.21.15.60:0/1907788781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740d11c0 0x7f71740cf4b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f71740b4660 tx=0x7f7168009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.157+0000 7f7178dff700 1 -- 172.21.15.60:0/1907788781 shutdown_connections 2023-07-13T22:31:32.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.157+0000 7f7178dff700 1 --2- 172.21.15.60:0/1907788781 >> v2:172.21.15.105:3300/0 conn(0x7f71740cf9f0 0x7f71740d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.157+0000 7f7178dff700 1 --2- 172.21.15.60:0/1907788781 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f717415c530 0x7f71740b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.157+0000 7f7178dff700 1 --2- 172.21.15.60:0/1907788781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740d11c0 0x7f71740cf4b0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.157+0000 7f7178dff700 1 -- 172.21.15.60:0/1907788781 >> 172.21.15.60:0/1907788781 conn(0x7f717406b400 msgr2=0x7f717406b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:32.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.157+0000 7f7178dff700 1 -- 172.21.15.60:0/1907788781 shutdown_connections 2023-07-13T22:31:32.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.157+0000 7f7178dff700 1 -- 172.21.15.60:0/1907788781 wait complete. 2023-07-13T22:31:32.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.158+0000 7f7178dff700 1 Processor -- start 2023-07-13T22:31:32.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.159+0000 7f7178dff700 1 -- start start 2023-07-13T22:31:32.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.159+0000 7f7178dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740cf9f0 0x7f71741e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:32.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.159+0000 7f7178dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71740d11c0 0x7f71741f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:32.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f7178dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71741f1410 0x7f71740b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:32.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f7178dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71741e55b0 con 0x7f71740cf9f0 2023-07-13T22:31:32.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f7178dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71741e6970 con 0x7f71741f1410 2023-07-13T22:31:32.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f7178dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71740b88c0 con 0x7f71740d11c0 2023-07-13T22:31:32.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f717259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740cf9f0 0x7f71741e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:32.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f717259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740cf9f0 0x7f71741e4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58056/0 (socket says 172.21.15.60:58056) 2023-07-13T22:31:32.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f717259c700 1 -- 172.21.15.60:0/2225574836 learned_addr learned my addr 172.21.15.60:0/2225574836 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:32.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f7171d9b700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71740d11c0 0x7f71741f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:32.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f7172d9d700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71741f1410 0x7f71740b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f717259c700 1 -- 172.21.15.60:0/2225574836 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71740d11c0 msgr2=0x7f71741f0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f717259c700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71740d11c0 0x7f71741f0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f717259c700 1 -- 172.21.15.60:0/2225574836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71741f1410 msgr2=0x7f71740b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f717259c700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71741f1410 0x7f71740b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f717259c700 1 -- 172.21.15.60:0/2225574836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7168009550 con 0x7f71740cf9f0 2023-07-13T22:31:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f7171d9b700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71740d11c0 0x7f71741f0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.160+0000 7f717259c700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740cf9f0 0x7f71741e4b40 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f716400b4d0 tx=0x7f716400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:32.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.161+0000 7f7162ffd700 1 -- 172.21.15.60:0/2225574836 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164011830 con 0x7f71740cf9f0 2023-07-13T22:31:32.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.161+0000 7f7162ffd700 1 -- 172.21.15.60:0/2225574836 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7164011990 con 0x7f71740cf9f0 2023-07-13T22:31:32.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.161+0000 7f7162ffd700 1 -- 172.21.15.60:0/2225574836 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164011d40 con 0x7f71740cf9f0 2023-07-13T22:31:32.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.161+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71741f18b0 con 0x7f71740cf9f0 2023-07-13T22:31:32.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.161+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71741e6b80 con 0x7f71740cf9f0 2023-07-13T22:31:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.162+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717404e210 con 0x7f71740cf9f0 2023-07-13T22:31:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.163+0000 7f7162ffd700 1 -- 172.21.15.60:0/2225574836 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f716401a400 con 0x7f71740cf9f0 2023-07-13T22:31:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.164+0000 7f7162ffd700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f715406ace0 0x7f715406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:32.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.164+0000 7f7162ffd700 1 -- 172.21.15.60:0/2225574836 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7164089530 con 0x7f71740cf9f0 2023-07-13T22:31:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.164+0000 7f7171d9b700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f715406ace0 0x7f715406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:32.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.165+0000 7f7171d9b700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f715406ace0 0x7f715406d190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7168000c00 tx=0x7f7168008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:32.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.168+0000 7f7162ffd700 1 -- 172.21.15.60:0/2225574836 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7164054550 con 0x7f71740cf9f0 2023-07-13T22:31:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.420+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f71740d2500 con 0x7f71740cf9f0 2023-07-13T22:31:32.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.921+0000 7f7162ffd700 1 -- 172.21.15.60:0/2225574836 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v97) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7164057b70 con 0x7f71740cf9f0 2023-07-13T22:31:32.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f715406ace0 msgr2=0x7f715406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:32.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f715406ace0 0x7f715406d190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7168000c00 tx=0x7f7168008030 comp rx=0 tx=0).stop 2023-07-13T22:31:32.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740cf9f0 msgr2=0x7f71741e4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:32.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740cf9f0 0x7f71741e4b40 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f716400b4d0 tx=0x7f716400b890 comp rx=0 tx=0).stop 2023-07-13T22:31:32.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 shutdown_connections 2023-07-13T22:31:32.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f715406ace0 0x7f715406d190 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71740cf9f0 0x7f71741e4b40 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71741f1410 0x7f71740b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 --2- 172.21.15.60:0/2225574836 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71740d11c0 0x7f71741f0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:32.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 >> 172.21.15.60:0/2225574836 conn(0x7f717406b400 msgr2=0x7f71740b64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:32.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 shutdown_connections 2023-07-13T22:31:32.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:32.924+0000 7f7178dff700 1 -- 172.21.15.60:0/2225574836 wait complete. 2023-07-13T22:31:32.927 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-13T22:31:32.927 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:33.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.150+0000 7fbb27dff700 1 Processor -- start 2023-07-13T22:31:33.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.150+0000 7fbb27dff700 1 -- start start 2023-07-13T22:31:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.151+0000 7fbb27dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb200b8570 0x7fbb200ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb27dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200d35c0 0x7fbb200d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb27dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb200d09d0 con 0x7fbb200cf3f0 2023-07-13T22:31:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb27dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb200d3b30 con 0x7fbb200b8570 2023-07-13T22:31:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb27dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200cda10 con 0x7fbb200d35c0 2023-07-13T22:31:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb25b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb200b8570 0x7fbb200ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb25b9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb200b8570 0x7fbb200ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58072/0 (socket says 172.21.15.60:58072) 2023-07-13T22:31:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb25b9b700 1 -- 172.21.15.60:0/268194235 learned_addr learned my addr 172.21.15.60:0/268194235 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb2539a700 1 --2- 172.21.15.60:0/268194235 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200d35c0 0x7fbb200d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb25b9b700 1 -- 172.21.15.60:0/268194235 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200d35c0 msgr2=0x7fbb200d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb25b9b700 1 --2- 172.21.15.60:0/268194235 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200d35c0 0x7fbb200d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb25b9b700 1 -- 172.21.15.60:0/268194235 >> v1:172.21.15.105:6789/0 conn(0x7fbb200cf3f0 legacy=0x7fbb2015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb25b9b700 1 -- 172.21.15.60:0/268194235 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb200cd450 con 0x7fbb200b8570 2023-07-13T22:31:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.152+0000 7fbb2539a700 1 --2- 172.21.15.60:0/268194235 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200d35c0 0x7fbb200d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb25b9b700 1 --2- 172.21.15.60:0/268194235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200b8570 0x7fbb200ceeb0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c006fd0 tx=0x7fbb1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fb96e980f6ee8d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb24b99700 1 -- 172.21.15.60:0/268194235 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c005e30 con 0x7fbb200b8570 2023-07-13T22:31:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb24b99700 1 -- 172.21.15.60:0/268194235 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c006030 con 0x7fbb200b8570 2023-07-13T22:31:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb24b99700 1 -- 172.21.15.60:0/268194235 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0063c0 con 0x7fbb200b8570 2023-07-13T22:31:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb27dff700 1 -- 172.21.15.60:0/268194235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200b8570 msgr2=0x7fbb200ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb27dff700 1 --2- 172.21.15.60:0/268194235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200b8570 0x7fbb200ceeb0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c006fd0 tx=0x7fbb1c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb27dff700 1 -- 172.21.15.60:0/268194235 shutdown_connections 2023-07-13T22:31:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb27dff700 1 --2- 172.21.15.60:0/268194235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200b8570 0x7fbb200ceeb0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb27dff700 1 --2- 172.21.15.60:0/268194235 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200d35c0 0x7fbb200d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.153+0000 7fbb27dff700 1 -- 172.21.15.60:0/268194235 >> 172.21.15.60:0/268194235 conn(0x7fbb20071600 msgr2=0x7fbb200719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.154+0000 7fbb27dff700 1 -- 172.21.15.60:0/268194235 shutdown_connections 2023-07-13T22:31:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.154+0000 7fbb27dff700 1 -- 172.21.15.60:0/268194235 wait complete. 2023-07-13T22:31:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.155+0000 7fbb27dff700 1 Processor -- start 2023-07-13T22:31:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.155+0000 7fbb27dff700 1 -- start start 2023-07-13T22:31:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.156+0000 7fbb27dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb200b8570 0x7fbb2014d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.156+0000 7fbb27dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200cf3f0 0x7fbb200b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.156+0000 7fbb27dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200d35c0 0x7fbb200b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.156+0000 7fbb27dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2014d870 con 0x7fbb200d35c0 2023-07-13T22:31:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.156+0000 7fbb27dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200caa20 con 0x7fbb200b8570 2023-07-13T22:31:33.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.156+0000 7fbb27dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200cad00 con 0x7fbb200cf3f0 2023-07-13T22:31:33.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2639c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200d35c0 0x7fbb200b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb25b9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb200b8570 0x7fbb2014d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2539a700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200cf3f0 0x7fbb200b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2639c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200d35c0 0x7fbb200b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58074/0 (socket says 172.21.15.60:58074) 2023-07-13T22:31:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2639c700 1 -- 172.21.15.60:0/2602919373 learned_addr learned my addr 172.21.15.60:0/2602919373 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2639c700 1 -- 172.21.15.60:0/2602919373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200cf3f0 msgr2=0x7fbb200b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2639c700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200cf3f0 0x7fbb200b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2639c700 1 -- 172.21.15.60:0/2602919373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb200b8570 msgr2=0x7fbb2014d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2639c700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb200b8570 0x7fbb2014d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2639c700 1 -- 172.21.15.60:0/2602919373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c009500 con 0x7fbb200d35c0 2023-07-13T22:31:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.157+0000 7fbb2639c700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200d35c0 0x7fbb200b3450 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800cc10 tx=0x7fbb1800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.158+0000 7fbb167fc700 1 -- 172.21.15.60:0/2602919373 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800aa60 con 0x7fbb200d35c0 2023-07-13T22:31:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.158+0000 7fbb167fc700 1 -- 172.21.15.60:0/2602919373 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1800abc0 con 0x7fbb200d35c0 2023-07-13T22:31:33.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.158+0000 7fbb167fc700 1 -- 172.21.15.60:0/2602919373 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18012420 con 0x7fbb200d35c0 2023-07-13T22:31:33.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.158+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb200cafe0 con 0x7fbb200d35c0 2023-07-13T22:31:33.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.158+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb200cb5a0 con 0x7fbb200d35c0 2023-07-13T22:31:33.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.159+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb200cb7c0 con 0x7fbb200d35c0 2023-07-13T22:31:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.160+0000 7fbb167fc700 1 -- 172.21.15.60:0/2602919373 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb18012580 con 0x7fbb200d35c0 2023-07-13T22:31:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.161+0000 7fbb167fc700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb0406ace0 0x7fbb0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.161+0000 7fbb167fc700 1 -- 172.21.15.60:0/2602919373 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb180897e0 con 0x7fbb200d35c0 2023-07-13T22:31:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.161+0000 7fbb25b9b700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb0406ace0 0x7fbb0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.162+0000 7fbb25b9b700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb0406ace0 0x7fbb0406d190 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c013040 tx=0x7fbb1c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:33.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.165+0000 7fbb167fc700 1 -- 172.21.15.60:0/2602919373 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fbb180547f0 con 0x7fbb200d35c0 2023-07-13T22:31:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.417+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb2015ce00 con 0x7fbb200d35c0 2023-07-13T22:31:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.417+0000 7fbb167fc700 1 -- 172.21.15.60:0/2602919373 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4719 (secure 0 0 0) 0x7fbb18057e10 con 0x7fbb200d35c0 2023-07-13T22:31:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:33.400 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3652157896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3652157896},{"type":"v1","addr":"172.21.15.105:6834","nonce":3652157896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1878256861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1878256861},{"type":"v1","addr":"172.21.15.105:6835","nonce":1878256861}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92},{"gid":8728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2695331242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2695331242},{"type":"v1","addr":"172.21.15.60:6836","nonce":2695331242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:31:29.408608+0000","modified":"2023-07-13T22:31:32.919029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9423":{"gid":9423,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4061691799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4061691799},{"type":"v1","addr":"172.21.15.60:6837","nonce":4061691799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-13T22:31:33.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb0406ace0 msgr2=0x7fbb0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb0406ace0 0x7fbb0406d190 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c013040 tx=0x7fbb1c008070 comp rx=0 tx=0).stop 2023-07-13T22:31:33.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200d35c0 msgr2=0x7fbb200b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200d35c0 0x7fbb200b3450 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800cc10 tx=0x7fbb1800cf20 comp rx=0 tx=0).stop 2023-07-13T22:31:33.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 shutdown_connections 2023-07-13T22:31:33.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fbb0406ace0 0x7fbb0406d190 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb200d35c0 0x7fbb200b3450 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb200b8570 0x7fbb2014d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 --2- 172.21.15.60:0/2602919373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb200cf3f0 0x7fbb200b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 >> 172.21.15.60:0/2602919373 conn(0x7fbb20071600 msgr2=0x7fbb2015b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 shutdown_connections 2023-07-13T22:31:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.420+0000 7fbb27dff700 1 -- 172.21.15.60:0/2602919373 wait complete. 2023-07-13T22:31:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 97 2023-07-13T22:31:33.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:33.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.641+0000 7f3e1b9ff700 1 Processor -- start 2023-07-13T22:31:33.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.641+0000 7f3e1b9ff700 1 -- start start 2023-07-13T22:31:33.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e1b9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140d35c0 0x7f3e140d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e1b9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3e140d13f0 0x7f3e1415c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e1b9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e140d09d0 con 0x7f3e140b8570 2023-07-13T22:31:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e1b9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140d3b30 con 0x7f3e140d13f0 2023-07-13T22:31:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e1b9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140d1870 con 0x7f3e140d35c0 2023-07-13T22:31:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e1979b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3e140b8570 0x7f3e140ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39180/0 (socket says 172.21.15.60:39180) 2023-07-13T22:31:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e1979b700 1 -- 172.21.15.60:0/4019819783 learned_addr learned my addr 172.21.15.60:0/4019819783 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e19f9c700 1 --2- 172.21.15.60:0/4019819783 >> v2:172.21.15.60:3300/0 conn(0x7f3e140d13f0 0x7f3e1415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e0bfff700 1 -- 172.21.15.60:0/4019819783 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719324281 0 0) 0x7f3e140d09d0 con 0x7f3e140b8570 2023-07-13T22:31:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e0bfff700 1 -- 172.21.15.60:0/4019819783 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e000035b0 con 0x7f3e140b8570 2023-07-13T22:31:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e18f9a700 1 --2- 172.21.15.60:0/4019819783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140d35c0 0x7f3e140d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e19f9c700 1 -- 172.21.15.60:0/4019819783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140d35c0 msgr2=0x7f3e140d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.643+0000 7f3e19f9c700 1 --2- 172.21.15.60:0/4019819783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140d35c0 0x7f3e140d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.644+0000 7f3e19f9c700 1 -- 172.21.15.60:0/4019819783 >> v1:172.21.15.60:6789/0 conn(0x7f3e140b8570 legacy=0x7f3e140ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.644+0000 7f3e19f9c700 1 -- 172.21.15.60:0/4019819783 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140cda10 con 0x7f3e140d13f0 2023-07-13T22:31:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.644+0000 7f3e18f9a700 1 --2- 172.21.15.60:0/4019819783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140d35c0 0x7f3e140d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:31:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.644+0000 7f3e19f9c700 1 --2- 172.21.15.60:0/4019819783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d13f0 0x7f3e1415c920 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009d90 tx=0x7f3e0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd66b0d8ec68f10 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.644+0000 7f3e0bfff700 1 -- 172.21.15.60:0/4019819783 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00e070 con 0x7f3e140d13f0 2023-07-13T22:31:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.644+0000 7f3e0bfff700 1 -- 172.21.15.60:0/4019819783 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c00bb20 con 0x7f3e140d13f0 2023-07-13T22:31:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.644+0000 7f3e0bfff700 1 -- 172.21.15.60:0/4019819783 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c012430 con 0x7f3e140d13f0 2023-07-13T22:31:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.644+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/4019819783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d13f0 msgr2=0x7f3e1415c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.644+0000 7f3e1b9ff700 1 --2- 172.21.15.60:0/4019819783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d13f0 0x7f3e1415c920 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009d90 tx=0x7f3e0c00b330 comp rx=0 tx=0).stop 2023-07-13T22:31:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.645+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/4019819783 shutdown_connections 2023-07-13T22:31:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.645+0000 7f3e1b9ff700 1 --2- 172.21.15.60:0/4019819783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d13f0 0x7f3e1415c920 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.645+0000 7f3e1b9ff700 1 --2- 172.21.15.60:0/4019819783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140d35c0 0x7f3e140d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.645+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/4019819783 >> 172.21.15.60:0/4019819783 conn(0x7f3e14071600 msgr2=0x7f3e140719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.645+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/4019819783 shutdown_connections 2023-07-13T22:31:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.645+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/4019819783 wait complete. 2023-07-13T22:31:33.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.646+0000 7f3e1b9ff700 1 Processor -- start 2023-07-13T22:31:33.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.647+0000 7f3e1b9ff700 1 -- start start 2023-07-13T22:31:33.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e1b9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140b8570 0x7f3e140b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e1b9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e140d13f0 0x7f3e141f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e1b9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d35c0 0x7f3e1415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e1b9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1414de60 con 0x7f3e140d35c0 2023-07-13T22:31:33.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e1b9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140b69b0 con 0x7f3e140d13f0 2023-07-13T22:31:33.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e1b9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140ca7b0 con 0x7f3e140b8570 2023-07-13T22:31:33.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e19f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d35c0 0x7f3e1415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e19f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d35c0 0x7f3e1415a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58102/0 (socket says 172.21.15.60:58102) 2023-07-13T22:31:33.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e19f9c700 1 -- 172.21.15.60:0/450968127 learned_addr learned my addr 172.21.15.60:0/450968127 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:33.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e18f9a700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e140d13f0 0x7f3e141f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e1979b700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140b8570 0x7f3e140b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e19f9c700 1 -- 172.21.15.60:0/450968127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140b8570 msgr2=0x7f3e140b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.635 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e19f9c700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140b8570 0x7f3e140b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e19f9c700 1 -- 172.21.15.60:0/450968127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e140d13f0 msgr2=0x7f3e141f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e19f9c700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e140d13f0 0x7f3e141f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.648+0000 7f3e19f9c700 1 -- 172.21.15.60:0/450968127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c009a70 con 0x7f3e140d35c0 2023-07-13T22:31:33.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.649+0000 7f3e19f9c700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d35c0 0x7f3e1415a7e0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00b370 tx=0x7f3e0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:33.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.649+0000 7f3e09ffb700 1 -- 172.21.15.60:0/450968127 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c012430 con 0x7f3e140d35c0 2023-07-13T22:31:33.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.649+0000 7f3e09ffb700 1 -- 172.21.15.60:0/450968127 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c020070 con 0x7f3e140d35c0 2023-07-13T22:31:33.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.649+0000 7f3e09ffb700 1 -- 172.21.15.60:0/450968127 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0098f0 con 0x7f3e140d35c0 2023-07-13T22:31:33.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.649+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e140c8540 con 0x7f3e140d35c0 2023-07-13T22:31:33.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.649+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e140c8d80 con 0x7f3e140d35c0 2023-07-13T22:31:33.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.650+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e14073ad0 con 0x7f3e140d35c0 2023-07-13T22:31:33.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.651+0000 7f3e09ffb700 1 -- 172.21.15.60:0/450968127 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e0c004040 con 0x7f3e140d35c0 2023-07-13T22:31:33.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.652+0000 7f3e09ffb700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3df806ace0 0x7f3df806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:33.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.652+0000 7f3e09ffb700 1 -- 172.21.15.60:0/450968127 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e0c022070 con 0x7f3e140d35c0 2023-07-13T22:31:33.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.652+0000 7f3e1979b700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3df806ace0 0x7f3df806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:33.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.653+0000 7f3e1979b700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3df806ace0 0x7f3df806d190 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3e100010e0 tx=0x7f3e10007c70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:33.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.657+0000 7f3e09ffb700 1 -- 172.21.15.60:0/450968127 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3e0c0561b0 con 0x7f3e140d35c0 2023-07-13T22:31:33.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.866+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e140ca970 con 0x7f3e140d35c0 2023-07-13T22:31:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.867+0000 7f3e09ffb700 1 -- 172.21.15.60:0/450968127 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f3e0c0597d0 con 0x7f3e140d35c0 2023-07-13T22:31:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:33.850 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":135,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:30.894942+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:31:27.702135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-13T22:31:28.386570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:33.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3df806ace0 msgr2=0x7f3df806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3df806ace0 0x7f3df806d190 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3e100010e0 tx=0x7f3e10007c70 comp rx=0 tx=0).stop 2023-07-13T22:31:33.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d35c0 msgr2=0x7f3e1415a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:33.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d35c0 0x7f3e1415a7e0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00b370 tx=0x7f3e0c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 shutdown_connections 2023-07-13T22:31:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3df806ace0 0x7f3df806d190 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e140d35c0 0x7f3e1415a7e0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e140b8570 0x7f3e140b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 --2- 172.21.15.60:0/450968127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e140d13f0 0x7f3e141f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 >> 172.21.15.60:0/450968127 conn(0x7f3e14071600 msgr2=0x7f3e140ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 shutdown_connections 2023-07-13T22:31:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:33.870+0000 7f3e1b9ff700 1 -- 172.21.15.60:0/450968127 wait complete. 2023-07-13T22:31:33.871 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:31:33.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:31:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.092+0000 7ff611bff700 1 Processor -- start 2023-07-13T22:31:34.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.092+0000 7ff611bff700 1 -- start start 2023-07-13T22:31:34.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.094+0000 7ff611bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff60c159170 0x7ff60c0c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.094+0000 7ff611bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c15d970 con 0x7ff60c070980 2023-07-13T22:31:34.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.094+0000 7ff611bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c15cd60 con 0x7ff60c0c5040 2023-07-13T22:31:34.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.094+0000 7ff611bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c15d040 con 0x7ff60c159170 2023-07-13T22:31:34.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.094+0000 7ff60bfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff60c070980 0x7ff60c0c9c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39196/0 (socket says 172.21.15.60:39196) 2023-07-13T22:31:34.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.094+0000 7ff60bfff700 1 -- 172.21.15.60:0/451006272 learned_addr learned my addr 172.21.15.60:0/451006272 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.094+0000 7ff60b7fe700 1 --2- 172.21.15.60:0/451006272 >> v2:172.21.15.105:3300/0 conn(0x7ff60c159170 0x7ff60c0c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633566084 0 0) 0x7ff60c15d970 con 0x7ff60c070980 2023-07-13T22:31:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f40035b0 con 0x7ff60c070980 2023-07-13T22:31:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1770551453 0 0) 0x7ff5f40035b0 con 0x7ff60c070980 2023-07-13T22:31:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 >> v2:172.21.15.105:3300/0 conn(0x7ff60c159170 msgr2=0x7ff60c0c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 --2- 172.21.15.60:0/451006272 >> v2:172.21.15.105:3300/0 conn(0x7ff60c159170 0x7ff60c0c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 >> v1:172.21.15.105:6789/0 conn(0x7ff60c0c5040 legacy=0x7ff60c070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c15d320 con 0x7ff60c070980 2023-07-13T22:31:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7ff6040019f0 con 0x7ff60c070980 2023-07-13T22:31:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60b7fe700 1 --2- 172.21.15.60:0/451006272 >> v2:172.21.15.105:3300/0 conn(0x7ff60c159170 0x7ff60c0c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 >> v1:172.21.15.60:6789/0 conn(0x7ff60c070980 legacy=0x7ff60c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f4003a00 0x7ff5f4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f4004850 0x7ff5f4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5f4005200 0x7ff5f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c15d040 con 0x7ff5f4004850 2023-07-13T22:31:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f400c080 con 0x7ff5f4005200 2023-07-13T22:31:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f400e450 con 0x7ff5f4003a00 2023-07-13T22:31:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.095+0000 7ff60affd700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f4004850 0x7ff5f4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60b7fe700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f4003a00 0x7ff5f4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60bfff700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5f4005200 0x7ff5f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60affd700 1 -- 172.21.15.60:0/451006272 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f4003a00 msgr2=0x7ff5f4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60affd700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f4003a00 0x7ff5f4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60affd700 1 -- 172.21.15.60:0/451006272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5f4005200 msgr2=0x7ff5f400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60affd700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5f4005200 0x7ff5f400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60affd700 1 -- 172.21.15.60:0/451006272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f40035b0 con 0x7ff5f4004850 2023-07-13T22:31:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60bfff700 1 -- 172.21.15.60:0/451006272 reap_dead start 2023-07-13T22:31:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60affd700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f4004850 0x7ff5f4004cc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0c5480 tx=0x7ff5f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93f1b7d3752aa85 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:34.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6000052d0 con 0x7ff5f4004850 2023-07-13T22:31:34.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff600005450 con 0x7ff5f4004850 2023-07-13T22:31:34.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.096+0000 7ff60a7fc700 1 -- 172.21.15.60:0/451006272 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6000038c0 con 0x7ff5f4004850 2023-07-13T22:31:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.097+0000 7ff611bff700 1 -- 172.21.15.60:0/451006272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f4004850 msgr2=0x7ff5f4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.097+0000 7ff611bff700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f4004850 0x7ff5f4004cc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0c5480 tx=0x7ff5f4005880 comp rx=0 tx=0).stop 2023-07-13T22:31:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.097+0000 7ff611bff700 1 -- 172.21.15.60:0/451006272 shutdown_connections 2023-07-13T22:31:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.097+0000 7ff611bff700 1 --2- 172.21.15.60:0/451006272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5f4004850 0x7ff5f4004cc0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.097+0000 7ff611bff700 1 -- 172.21.15.60:0/451006272 >> 172.21.15.60:0/451006272 conn(0x7ff60c0cf410 msgr2=0x7ff60c0cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.097+0000 7ff611bff700 1 -- 172.21.15.60:0/451006272 shutdown_connections 2023-07-13T22:31:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.097+0000 7ff611bff700 1 -- 172.21.15.60:0/451006272 wait complete. 2023-07-13T22:31:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.099+0000 7ff611bff700 1 Processor -- start 2023-07-13T22:31:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.099+0000 7ff611bff700 1 -- start start 2023-07-13T22:31:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff611bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f4004850 0x7ff60c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff611bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff60c156800 0x7ff60c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff611bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b4370 0x7ff60c157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff611bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0b33b0 con 0x7ff60c156800 2023-07-13T22:31:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff611bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c15c080 con 0x7ff60c0b4370 2023-07-13T22:31:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff611bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c15c360 con 0x7ff5f4004850 2023-07-13T22:31:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff60affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff60c156800 0x7ff60c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff60affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff60c156800 0x7ff60c0b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58124/0 (socket says 172.21.15.60:58124) 2023-07-13T22:31:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff60affd700 1 -- 172.21.15.60:0/1124136599 learned_addr learned my addr 172.21.15.60:0/1124136599 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff60b7fe700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f4004850 0x7ff60c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.100+0000 7ff60bfff700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b4370 0x7ff60c157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff60affd700 1 -- 172.21.15.60:0/1124136599 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f4004850 msgr2=0x7ff60c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff60affd700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f4004850 0x7ff60c153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff60affd700 1 -- 172.21.15.60:0/1124136599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b4370 msgr2=0x7ff60c157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff60affd700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b4370 0x7ff60c157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff60affd700 1 -- 172.21.15.60:0/1124136599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c156000 con 0x7ff60c156800 2023-07-13T22:31:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff60bfff700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b4370 0x7ff60c157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff60affd700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff60c156800 0x7ff60c0b3e30 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff600005040 tx=0x7ff600001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff5f3fff700 1 -- 172.21.15.60:0/1124136599 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600001290 con 0x7ff60c156800 2023-07-13T22:31:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff5f3fff700 1 -- 172.21.15.60:0/1124136599 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff600007040 con 0x7ff60c156800 2023-07-13T22:31:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff5f3fff700 1 -- 172.21.15.60:0/1124136599 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6000073f0 con 0x7ff60c156800 2023-07-13T22:31:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c1562e0 con 0x7ff60c156800 2023-07-13T22:31:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.101+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c1547e0 con 0x7ff60c156800 2023-07-13T22:31:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.102+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c0ad860 con 0x7ff60c156800 2023-07-13T22:31:34.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.103+0000 7ff5f3fff700 1 -- 172.21.15.60:0/1124136599 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff600007550 con 0x7ff60c156800 2023-07-13T22:31:34.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.104+0000 7ff5f3fff700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5f806ace0 0x7ff5f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.104+0000 7ff5f3fff700 1 -- 172.21.15.60:0/1124136599 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff600089db0 con 0x7ff60c156800 2023-07-13T22:31:34.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.105+0000 7ff60b7fe700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5f806ace0 0x7ff5f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.105+0000 7ff60b7fe700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5f806ace0 0x7ff5f806d190 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0afb30 tx=0x7ff5fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:34.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.109+0000 7ff5f3fff700 1 -- 172.21.15.60:0/1124136599 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff600054e50 con 0x7ff60c156800 2023-07-13T22:31:34.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.311+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff60c15d600 con 0x7ff5f806ace0 2023-07-13T22:31:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.320+0000 7ff5f3fff700 1 -- 172.21.15.60:0/1124136599 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311887 (secure 0 0 0) 0x7ff5fc009a20 con 0x7ff5f806ace0 2023-07-13T22:31:34.306 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:34.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5f806ace0 msgr2=0x7ff5f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5f806ace0 0x7ff5f806d190 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0afb30 tx=0x7ff5fc008040 comp rx=0 tx=0).stop 2023-07-13T22:31:34.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff60c156800 msgr2=0x7ff60c0b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff60c156800 0x7ff60c0b3e30 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff600005040 tx=0x7ff600001120 comp rx=0 tx=0).stop 2023-07-13T22:31:34.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 shutdown_connections 2023-07-13T22:31:34.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff5f806ace0 0x7ff5f806d190 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b4370 0x7ff60c157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5f4004850 0x7ff60c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 --2- 172.21.15.60:0/1124136599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff60c156800 0x7ff60c0b3e30 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.324+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 >> 172.21.15.60:0/1124136599 conn(0x7ff60c0cf410 msgr2=0x7ff60c070270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:34.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.325+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 shutdown_connections 2023-07-13T22:31:34.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.325+0000 7ff611bff700 1 -- 172.21.15.60:0/1124136599 wait complete. 2023-07-13T22:31:34.310 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:31:34.325 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2023-07-13T22:31:33.264045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295512,"kb_used_data":14360,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754679208,"statfs":{"total":773094113280,"available":772791508992,"internally_reserved":0,"allocated":14704640,"data_stored":7357130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.379832"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900526+0000","last_change":"2023-07-13T22:31:29.917401+0000","last_active":"2023-07-13T22:31:30.900526+0000","last_peered":"2023-07-13T22:31:30.900526+0000","last_clean":"2023-07-13T22:31:30.900526+0000","last_became_active":"2023-07-13T22:31:29.916972+0000","last_became_peered":"2023-07-13T22:31:29.916972+0000","last_unstale":"2023-07-13T22:31:30.900526+0000","last_undegraded":"2023-07-13T22:31:30.900526+0000","last_fullsized":"2023-07-13T22:31:30.900526+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:18:33.645643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899288+0000","last_change":"2023-07-13T22:31:29.922523+0000","last_active":"2023-07-13T22:31:30.899288+0000","last_peered":"2023-07-13T22:31:30.899288+0000","last_clean":"2023-07-13T22:31:30.899288+0000","last_became_active":"2023-07-13T22:31:29.922205+0000","last_became_peered":"2023-07-13T22:31:29.922205+0000","last_unstale":"2023-07-13T22:31:30.899288+0000","last_undegraded":"2023-07-13T22:31:30.899288+0000","last_fullsized":"2023-07-13T22:31:30.899288+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:12:01.285269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899457+0000","last_change":"2023-07-13T22:31:29.914638+0000","last_active":"2023-07-13T22:31:30.899457+0000","last_peered":"2023-07-13T22:31:30.899457+0000","last_clean":"2023-07-13T22:31:30.899457+0000","last_became_active":"2023-07-13T22:31:29.914317+0000","last_became_peered":"2023-07-13T22:31:29.914317+0000","last_unstale":"2023-07-13T22:31:30.899457+0000","last_undegraded":"2023-07-13T22:31:30.899457+0000","last_fullsized":"2023-07-13T22:31:30.899457+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:02:46.062610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143569+0000","last_change":"2023-07-13T22:31:29.919617+0000","last_active":"2023-07-13T22:31:31.143569+0000","last_peered":"2023-07-13T22:31:31.143569+0000","last_clean":"2023-07-13T22:31:31.143569+0000","last_became_active":"2023-07-13T22:31:29.919355+0000","last_became_peered":"2023-07-13T22:31:29.919355+0000","last_unstale":"2023-07-13T22:31:31.143569+0000","last_undegraded":"2023-07-13T22:31:31.143569+0000","last_fullsized":"2023-07-13T22:31:31.143569+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:43:38.942537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144032+0000","last_change":"2023-07-13T22:31:29.915088+0000","last_active":"2023-07-13T22:31:31.144032+0000","last_peered":"2023-07-13T22:31:31.144032+0000","last_clean":"2023-07-13T22:31:31.144032+0000","last_became_active":"2023-07-13T22:31:29.914804+0000","last_became_peered":"2023-07-13T22:31:29.914804+0000","last_unstale":"2023-07-13T22:31:31.144032+0000","last_undegraded":"2023-07-13T22:31:31.144032+0000","last_fullsized":"2023-07-13T22:31:31.144032+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:47.603029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143867+0000","last_change":"2023-07-13T22:31:29.911681+0000","last_active":"2023-07-13T22:31:31.143867+0000","last_peered":"2023-07-13T22:31:31.143867+0000","last_clean":"2023-07-13T22:31:31.143867+0000","last_became_active":"2023-07-13T22:31:29.911314+0000","last_became_peered":"2023-07-13T22:31:29.911314+0000","last_unstale":"2023-07-13T22:31:31.143867+0000","last_undegraded":"2023-07-13T22:31:31.143867+0000","last_fullsized":"2023-07-13T22:31:31.143867+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:58.536230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899811+0000","last_change":"2023-07-13T22:31:29.914194+0000","last_active":"2023-07-13T22:31:30.899811+0000","last_peered":"2023-07-13T22:31:30.899811+0000","last_clean":"2023-07-13T22:31:30.899811+0000","last_became_active":"2023-07-13T22:31:29.913929+0000","last_became_peered":"2023-07-13T22:31:29.913929+0000","last_unstale":"2023-07-13T22:31:30.899811+0000","last_undegraded":"2023-07-13T22:31:30.899811+0000","last_fullsized":"2023-07-13T22:31:30.899811+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:09:58.455819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900976+0000","last_change":"2023-07-13T22:31:29.908754+0000","last_active":"2023-07-13T22:31:30.900976+0000","last_peered":"2023-07-13T22:31:30.900976+0000","last_clean":"2023-07-13T22:31:30.900976+0000","last_became_active":"2023-07-13T22:31:29.908252+0000","last_became_peered":"2023-07-13T22:31:29.908252+0000","last_unstale":"2023-07-13T22:31:30.900976+0000","last_undegraded":"2023-07-13T22:31:30.900976+0000","last_fullsized":"2023-07-13T22:31:30.900976+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:58.420845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144396+0000","last_change":"2023-07-13T22:31:29.907645+0000","last_active":"2023-07-13T22:31:31.144396+0000","last_peered":"2023-07-13T22:31:31.144396+0000","last_clean":"2023-07-13T22:31:31.144396+0000","last_became_active":"2023-07-13T22:31:29.907352+0000","last_became_peered":"2023-07-13T22:31:29.907352+0000","last_unstale":"2023-07-13T22:31:31.144396+0000","last_undegraded":"2023-07-13T22:31:31.144396+0000","last_fullsized":"2023-07-13T22:31:31.144396+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:58:16.541596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144220+0000","last_change":"2023-07-13T22:31:29.908582+0000","last_active":"2023-07-13T22:31:31.144220+0000","last_peered":"2023-07-13T22:31:31.144220+0000","last_clean":"2023-07-13T22:31:31.144220+0000","last_became_active":"2023-07-13T22:31:29.908250+0000","last_became_peered":"2023-07-13T22:31:29.908250+0000","last_unstale":"2023-07-13T22:31:31.144220+0000","last_undegraded":"2023-07-13T22:31:31.144220+0000","last_fullsized":"2023-07-13T22:31:31.144220+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:10:25.533745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144215+0000","last_change":"2023-07-13T22:31:29.915160+0000","last_active":"2023-07-13T22:31:31.144215+0000","last_peered":"2023-07-13T22:31:31.144215+0000","last_clean":"2023-07-13T22:31:31.144215+0000","last_became_active":"2023-07-13T22:31:29.914679+0000","last_became_peered":"2023-07-13T22:31:29.914679+0000","last_unstale":"2023-07-13T22:31:31.144215+0000","last_undegraded":"2023-07-13T22:31:31.144215+0000","last_fullsized":"2023-07-13T22:31:31.144215+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:44:08.250788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143721+0000","last_change":"2023-07-13T22:31:29.923814+0000","last_active":"2023-07-13T22:31:31.143721+0000","last_peered":"2023-07-13T22:31:31.143721+0000","last_clean":"2023-07-13T22:31:31.143721+0000","last_became_active":"2023-07-13T22:31:29.923556+0000","last_became_peered":"2023-07-13T22:31:29.923556+0000","last_unstale":"2023-07-13T22:31:31.143721+0000","last_undegraded":"2023-07-13T22:31:31.143721+0000","last_fullsized":"2023-07-13T22:31:31.143721+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:59:16.137668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143994+0000","last_change":"2023-07-13T22:31:29.910377+0000","last_active":"2023-07-13T22:31:31.143994+0000","last_peered":"2023-07-13T22:31:31.143994+0000","last_clean":"2023-07-13T22:31:31.143994+0000","last_became_active":"2023-07-13T22:31:29.909831+0000","last_became_peered":"2023-07-13T22:31:29.909831+0000","last_unstale":"2023-07-13T22:31:31.143994+0000","last_undegraded":"2023-07-13T22:31:31.143994+0000","last_fullsized":"2023-07-13T22:31:31.143994+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:34:07.854269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143149+0000","last_change":"2023-07-13T22:31:29.924577+0000","last_active":"2023-07-13T22:31:31.143149+0000","last_peered":"2023-07-13T22:31:31.143149+0000","last_clean":"2023-07-13T22:31:31.143149+0000","last_became_active":"2023-07-13T22:31:29.924267+0000","last_became_peered":"2023-07-13T22:31:29.924267+0000","last_unstale":"2023-07-13T22:31:31.143149+0000","last_undegraded":"2023-07-13T22:31:31.143149+0000","last_fullsized":"2023-07-13T22:31:31.143149+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:44:45.264078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898975+0000","last_change":"2023-07-13T22:31:29.917523+0000","last_active":"2023-07-13T22:31:30.898975+0000","last_peered":"2023-07-13T22:31:30.898975+0000","last_clean":"2023-07-13T22:31:30.898975+0000","last_became_active":"2023-07-13T22:31:29.917242+0000","last_became_peered":"2023-07-13T22:31:29.917242+0000","last_unstale":"2023-07-13T22:31:30.898975+0000","last_undegraded":"2023-07-13T22:31:30.898975+0000","last_fullsized":"2023-07-13T22:31:30.898975+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:09:45.314198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142846+0000","last_change":"2023-07-13T22:31:29.907251+0000","last_active":"2023-07-13T22:31:31.142846+0000","last_peered":"2023-07-13T22:31:31.142846+0000","last_clean":"2023-07-13T22:31:31.142846+0000","last_became_active":"2023-07-13T22:31:29.906984+0000","last_became_peered":"2023-07-13T22:31:29.906984+0000","last_unstale":"2023-07-13T22:31:31.142846+0000","last_undegraded":"2023-07-13T22:31:31.142846+0000","last_fullsized":"2023-07-13T22:31:31.142846+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:35:23.619343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143462+0000","last_change":"2023-07-13T22:31:29.917980+0000","last_active":"2023-07-13T22:31:31.143462+0000","last_peered":"2023-07-13T22:31:31.143462+0000","last_clean":"2023-07-13T22:31:31.143462+0000","last_became_active":"2023-07-13T22:31:29.917682+0000","last_became_peered":"2023-07-13T22:31:29.917682+0000","last_unstale":"2023-07-13T22:31:31.143462+0000","last_undegraded":"2023-07-13T22:31:31.143462+0000","last_fullsized":"2023-07-13T22:31:31.143462+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:39:53.060278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143474+0000","last_change":"2023-07-13T22:31:29.919725+0000","last_active":"2023-07-13T22:31:31.143474+0000","last_peered":"2023-07-13T22:31:31.143474+0000","last_clean":"2023-07-13T22:31:31.143474+0000","last_became_active":"2023-07-13T22:31:29.919271+0000","last_became_peered":"2023-07-13T22:31:29.919271+0000","last_unstale":"2023-07-13T22:31:31.143474+0000","last_undegraded":"2023-07-13T22:31:31.143474+0000","last_fullsized":"2023-07-13T22:31:31.143474+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:02:44.414707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143994+0000","last_change":"2023-07-13T22:31:29.923034+0000","last_active":"2023-07-13T22:31:31.143994+0000","last_peered":"2023-07-13T22:31:31.143994+0000","last_clean":"2023-07-13T22:31:31.143994+0000","last_became_active":"2023-07-13T22:31:29.922397+0000","last_became_peered":"2023-07-13T22:31:29.922397+0000","last_unstale":"2023-07-13T22:31:31.143994+0000","last_undegraded":"2023-07-13T22:31:31.143994+0000","last_fullsized":"2023-07-13T22:31:31.143994+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:19:35.150074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143360+0000","last_change":"2023-07-13T22:31:29.915309+0000","last_active":"2023-07-13T22:31:31.143360+0000","last_peered":"2023-07-13T22:31:31.143360+0000","last_clean":"2023-07-13T22:31:31.143360+0000","last_became_active":"2023-07-13T22:31:29.915019+0000","last_became_peered":"2023-07-13T22:31:29.915019+0000","last_unstale":"2023-07-13T22:31:31.143360+0000","last_undegraded":"2023-07-13T22:31:31.143360+0000","last_fullsized":"2023-07-13T22:31:31.143360+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:55:54.167692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143899+0000","last_change":"2023-07-13T22:31:29.911309+0000","last_active":"2023-07-13T22:31:31.143899+0000","last_peered":"2023-07-13T22:31:31.143899+0000","last_clean":"2023-07-13T22:31:31.143899+0000","last_became_active":"2023-07-13T22:31:29.910941+0000","last_became_peered":"2023-07-13T22:31:29.910941+0000","last_unstale":"2023-07-13T22:31:31.143899+0000","last_undegraded":"2023-07-13T22:31:31.143899+0000","last_fullsized":"2023-07-13T22:31:31.143899+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:04:49.418121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900607+0000","last_change":"2023-07-13T22:31:29.915096+0000","last_active":"2023-07-13T22:31:30.900607+0000","last_peered":"2023-07-13T22:31:30.900607+0000","last_clean":"2023-07-13T22:31:30.900607+0000","last_became_active":"2023-07-13T22:31:29.914826+0000","last_became_peered":"2023-07-13T22:31:29.914826+0000","last_unstale":"2023-07-13T22:31:30.900607+0000","last_undegraded":"2023-07-13T22:31:30.900607+0000","last_fullsized":"2023-07-13T22:31:30.900607+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:47:06.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900221+0000","last_change":"2023-07-13T22:31:29.912611+0000","last_active":"2023-07-13T22:31:30.900221+0000","last_peered":"2023-07-13T22:31:30.900221+0000","last_clean":"2023-07-13T22:31:30.900221+0000","last_became_active":"2023-07-13T22:31:29.912347+0000","last_became_peered":"2023-07-13T22:31:29.912347+0000","last_unstale":"2023-07-13T22:31:30.900221+0000","last_undegraded":"2023-07-13T22:31:30.900221+0000","last_fullsized":"2023-07-13T22:31:30.900221+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:06:29.624478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142772+0000","last_change":"2023-07-13T22:31:29.910905+0000","last_active":"2023-07-13T22:31:31.142772+0000","last_peered":"2023-07-13T22:31:31.142772+0000","last_clean":"2023-07-13T22:31:31.142772+0000","last_became_active":"2023-07-13T22:31:29.910627+0000","last_became_peered":"2023-07-13T22:31:29.910627+0000","last_unstale":"2023-07-13T22:31:31.142772+0000","last_undegraded":"2023-07-13T22:31:31.142772+0000","last_fullsized":"2023-07-13T22:31:31.142772+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:43:11.434307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900755+0000","last_change":"2023-07-13T22:31:29.911816+0000","last_active":"2023-07-13T22:31:30.900755+0000","last_peered":"2023-07-13T22:31:30.900755+0000","last_clean":"2023-07-13T22:31:30.900755+0000","last_became_active":"2023-07-13T22:31:29.911411+0000","last_became_peered":"2023-07-13T22:31:29.911411+0000","last_unstale":"2023-07-13T22:31:30.900755+0000","last_undegraded":"2023-07-13T22:31:30.900755+0000","last_fullsized":"2023-07-13T22:31:30.900755+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:39:39.508445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900973+0000","last_change":"2023-07-13T22:31:29.919876+0000","last_active":"2023-07-13T22:31:30.900973+0000","last_peered":"2023-07-13T22:31:30.900973+0000","last_clean":"2023-07-13T22:31:30.900973+0000","last_became_active":"2023-07-13T22:31:29.919598+0000","last_became_peered":"2023-07-13T22:31:29.919598+0000","last_unstale":"2023-07-13T22:31:30.900973+0000","last_undegraded":"2023-07-13T22:31:30.900973+0000","last_fullsized":"2023-07-13T22:31:30.900973+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:58.420845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143988+0000","last_change":"2023-07-13T22:31:29.910507+0000","last_active":"2023-07-13T22:31:31.143988+0000","last_peered":"2023-07-13T22:31:31.143988+0000","last_clean":"2023-07-13T22:31:31.143988+0000","last_became_active":"2023-07-13T22:31:29.910212+0000","last_became_peered":"2023-07-13T22:31:29.910212+0000","last_unstale":"2023-07-13T22:31:31.143988+0000","last_undegraded":"2023-07-13T22:31:31.143988+0000","last_fullsized":"2023-07-13T22:31:31.143988+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:54:23.426448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142983+0000","last_change":"2023-07-13T22:31:29.920722+0000","last_active":"2023-07-13T22:31:31.142983+0000","last_peered":"2023-07-13T22:31:31.142983+0000","last_clean":"2023-07-13T22:31:31.142983+0000","last_became_active":"2023-07-13T22:31:29.920434+0000","last_became_peered":"2023-07-13T22:31:29.920434+0000","last_unstale":"2023-07-13T22:31:31.142983+0000","last_undegraded":"2023-07-13T22:31:31.142983+0000","last_fullsized":"2023-07-13T22:31:31.142983+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:52.338010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901219+0000","last_change":"2023-07-13T22:31:29.908239+0000","last_active":"2023-07-13T22:31:30.901219+0000","last_peered":"2023-07-13T22:31:30.901219+0000","last_clean":"2023-07-13T22:31:30.901219+0000","last_became_active":"2023-07-13T22:31:29.907198+0000","last_became_peered":"2023-07-13T22:31:29.907198+0000","last_unstale":"2023-07-13T22:31:30.901219+0000","last_undegraded":"2023-07-13T22:31:30.901219+0000","last_fullsized":"2023-07-13T22:31:30.901219+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:48:08.438876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143254+0000","last_change":"2023-07-13T22:31:29.916965+0000","last_active":"2023-07-13T22:31:31.143254+0000","last_peered":"2023-07-13T22:31:31.143254+0000","last_clean":"2023-07-13T22:31:31.143254+0000","last_became_active":"2023-07-13T22:31:29.916691+0000","last_became_peered":"2023-07-13T22:31:29.916691+0000","last_unstale":"2023-07-13T22:31:31.143254+0000","last_undegraded":"2023-07-13T22:31:31.143254+0000","last_fullsized":"2023-07-13T22:31:31.143254+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:19:19.898524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143267+0000","last_change":"2023-07-13T22:31:29.915891+0000","last_active":"2023-07-13T22:31:31.143267+0000","last_peered":"2023-07-13T22:31:31.143267+0000","last_clean":"2023-07-13T22:31:31.143267+0000","last_became_active":"2023-07-13T22:31:29.915620+0000","last_became_peered":"2023-07-13T22:31:29.915620+0000","last_unstale":"2023-07-13T22:31:31.143267+0000","last_undegraded":"2023-07-13T22:31:31.143267+0000","last_fullsized":"2023-07-13T22:31:31.143267+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:45:34.043244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900785+0000","last_change":"2023-07-13T22:31:29.908592+0000","last_active":"2023-07-13T22:31:30.900785+0000","last_peered":"2023-07-13T22:31:30.900785+0000","last_clean":"2023-07-13T22:31:30.900785+0000","last_became_active":"2023-07-13T22:31:29.907889+0000","last_became_peered":"2023-07-13T22:31:29.907889+0000","last_unstale":"2023-07-13T22:31:30.900785+0000","last_undegraded":"2023-07-13T22:31:30.900785+0000","last_fullsized":"2023-07-13T22:31:30.900785+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:39:39.508445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899128+0000","last_change":"2023-07-13T22:31:29.908706+0000","last_active":"2023-07-13T22:31:30.899128+0000","last_peered":"2023-07-13T22:31:30.899128+0000","last_clean":"2023-07-13T22:31:30.899128+0000","last_became_active":"2023-07-13T22:31:29.908444+0000","last_became_peered":"2023-07-13T22:31:29.908444+0000","last_unstale":"2023-07-13T22:31:30.899128+0000","last_undegraded":"2023-07-13T22:31:30.899128+0000","last_fullsized":"2023-07-13T22:31:30.899128+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:34:49.040606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"134'4","reported_seq":17,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144539+0000","last_change":"2023-07-13T22:31:28.909458+0000","last_active":"2023-07-13T22:31:31.144539+0000","last_peered":"2023-07-13T22:31:31.144539+0000","last_clean":"2023-07-13T22:31:31.144539+0000","last_became_active":"2023-07-13T22:31:28.909045+0000","last_became_peered":"2023-07-13T22:31:28.909045+0000","last_unstale":"2023-07-13T22:31:31.144539+0000","last_undegraded":"2023-07-13T22:31:31.144539+0000","last_fullsized":"2023-07-13T22:31:31.144539+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:14:04.160858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899127+0000","last_change":"2023-07-13T22:31:29.917255+0000","last_active":"2023-07-13T22:31:30.899127+0000","last_peered":"2023-07-13T22:31:30.899127+0000","last_clean":"2023-07-13T22:31:30.899127+0000","last_became_active":"2023-07-13T22:31:29.916959+0000","last_became_peered":"2023-07-13T22:31:29.916959+0000","last_unstale":"2023-07-13T22:31:30.899127+0000","last_undegraded":"2023-07-13T22:31:30.899127+0000","last_fullsized":"2023-07-13T22:31:30.899127+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:46.014828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143872+0000","last_change":"2023-07-13T22:31:28.916116+0000","last_active":"2023-07-13T22:31:31.143872+0000","last_peered":"2023-07-13T22:31:31.143872+0000","last_clean":"2023-07-13T22:31:31.143872+0000","last_became_active":"2023-07-13T22:31:28.915689+0000","last_became_peered":"2023-07-13T22:31:28.915689+0000","last_unstale":"2023-07-13T22:31:31.143872+0000","last_undegraded":"2023-07-13T22:31:31.143872+0000","last_fullsized":"2023-07-13T22:31:31.143872+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:58:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901082+0000","last_change":"2023-07-13T22:31:29.921351+0000","last_active":"2023-07-13T22:31:30.901082+0000","last_peered":"2023-07-13T22:31:30.901082+0000","last_clean":"2023-07-13T22:31:30.901082+0000","last_became_active":"2023-07-13T22:31:29.921039+0000","last_became_peered":"2023-07-13T22:31:29.921039+0000","last_unstale":"2023-07-13T22:31:30.901082+0000","last_undegraded":"2023-07-13T22:31:30.901082+0000","last_fullsized":"2023-07-13T22:31:30.901082+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:24:56.967947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143940+0000","last_change":"2023-07-13T22:31:28.915708+0000","last_active":"2023-07-13T22:31:31.143940+0000","last_peered":"2023-07-13T22:31:31.143940+0000","last_clean":"2023-07-13T22:31:31.143940+0000","last_became_active":"2023-07-13T22:31:28.915418+0000","last_became_peered":"2023-07-13T22:31:28.915418+0000","last_unstale":"2023-07-13T22:31:31.143940+0000","last_undegraded":"2023-07-13T22:31:31.143940+0000","last_fullsized":"2023-07-13T22:31:31.143940+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:57:49.013838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143256+0000","last_change":"2023-07-13T22:31:29.913971+0000","last_active":"2023-07-13T22:31:31.143256+0000","last_peered":"2023-07-13T22:31:31.143256+0000","last_clean":"2023-07-13T22:31:31.143256+0000","last_became_active":"2023-07-13T22:31:29.913670+0000","last_became_peered":"2023-07-13T22:31:29.913670+0000","last_unstale":"2023-07-13T22:31:31.143256+0000","last_undegraded":"2023-07-13T22:31:31.143256+0000","last_fullsized":"2023-07-13T22:31:31.143256+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:02:43.926077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899321+0000","last_change":"2023-07-13T22:31:28.914081+0000","last_active":"2023-07-13T22:31:30.899321+0000","last_peered":"2023-07-13T22:31:30.899321+0000","last_clean":"2023-07-13T22:31:30.899321+0000","last_became_active":"2023-07-13T22:31:28.913809+0000","last_became_peered":"2023-07-13T22:31:28.913809+0000","last_unstale":"2023-07-13T22:31:30.899321+0000","last_undegraded":"2023-07-13T22:31:30.899321+0000","last_fullsized":"2023-07-13T22:31:30.899321+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:16:46.467097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142806+0000","last_change":"2023-07-13T22:31:29.913646+0000","last_active":"2023-07-13T22:31:31.142806+0000","last_peered":"2023-07-13T22:31:31.142806+0000","last_clean":"2023-07-13T22:31:31.142806+0000","last_became_active":"2023-07-13T22:31:29.913331+0000","last_became_peered":"2023-07-13T22:31:29.913331+0000","last_unstale":"2023-07-13T22:31:31.142806+0000","last_undegraded":"2023-07-13T22:31:31.142806+0000","last_fullsized":"2023-07-13T22:31:31.142806+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:36:44.735862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901330+0000","last_change":"2023-07-13T22:31:28.911510+0000","last_active":"2023-07-13T22:31:30.901330+0000","last_peered":"2023-07-13T22:31:30.901330+0000","last_clean":"2023-07-13T22:31:30.901330+0000","last_became_active":"2023-07-13T22:31:28.911218+0000","last_became_peered":"2023-07-13T22:31:28.911218+0000","last_unstale":"2023-07-13T22:31:30.901330+0000","last_undegraded":"2023-07-13T22:31:30.901330+0000","last_fullsized":"2023-07-13T22:31:30.901330+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:00:38.998682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142847+0000","last_change":"2023-07-13T22:31:29.920346+0000","last_active":"2023-07-13T22:31:31.142847+0000","last_peered":"2023-07-13T22:31:31.142847+0000","last_clean":"2023-07-13T22:31:31.142847+0000","last_became_active":"2023-07-13T22:31:29.919973+0000","last_became_peered":"2023-07-13T22:31:29.919973+0000","last_unstale":"2023-07-13T22:31:31.142847+0000","last_undegraded":"2023-07-13T22:31:31.142847+0000","last_fullsized":"2023-07-13T22:31:31.142847+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:23:35.383110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144150+0000","last_change":"2023-07-13T22:31:28.910447+0000","last_active":"2023-07-13T22:31:31.144150+0000","last_peered":"2023-07-13T22:31:31.144150+0000","last_clean":"2023-07-13T22:31:31.144150+0000","last_became_active":"2023-07-13T22:31:28.910112+0000","last_became_peered":"2023-07-13T22:31:28.910112+0000","last_unstale":"2023-07-13T22:31:31.144150+0000","last_undegraded":"2023-07-13T22:31:31.144150+0000","last_fullsized":"2023-07-13T22:31:31.144150+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:46:16.906867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898894+0000","last_change":"2023-07-13T22:31:29.907620+0000","last_active":"2023-07-13T22:31:30.898894+0000","last_peered":"2023-07-13T22:31:30.898894+0000","last_clean":"2023-07-13T22:31:30.898894+0000","last_became_active":"2023-07-13T22:31:29.907343+0000","last_became_peered":"2023-07-13T22:31:29.907343+0000","last_unstale":"2023-07-13T22:31:30.898894+0000","last_undegraded":"2023-07-13T22:31:30.898894+0000","last_fullsized":"2023-07-13T22:31:30.898894+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:58.420845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899956+0000","last_change":"2023-07-13T22:31:28.910754+0000","last_active":"2023-07-13T22:31:30.899956+0000","last_peered":"2023-07-13T22:31:30.899956+0000","last_clean":"2023-07-13T22:31:30.899956+0000","last_became_active":"2023-07-13T22:31:28.910392+0000","last_became_peered":"2023-07-13T22:31:28.910392+0000","last_unstale":"2023-07-13T22:31:30.899956+0000","last_undegraded":"2023-07-13T22:31:30.899956+0000","last_fullsized":"2023-07-13T22:31:30.899956+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:42:37.191767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900094+0000","last_change":"2023-07-13T22:31:29.917805+0000","last_active":"2023-07-13T22:31:30.900094+0000","last_peered":"2023-07-13T22:31:30.900094+0000","last_clean":"2023-07-13T22:31:30.900094+0000","last_became_active":"2023-07-13T22:31:29.917525+0000","last_became_peered":"2023-07-13T22:31:29.917525+0000","last_unstale":"2023-07-13T22:31:30.900094+0000","last_undegraded":"2023-07-13T22:31:30.900094+0000","last_fullsized":"2023-07-13T22:31:30.900094+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:31:26.698669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143049+0000","last_change":"2023-07-13T22:31:28.910406+0000","last_active":"2023-07-13T22:31:31.143049+0000","last_peered":"2023-07-13T22:31:31.143049+0000","last_clean":"2023-07-13T22:31:31.143049+0000","last_became_active":"2023-07-13T22:31:28.910107+0000","last_became_peered":"2023-07-13T22:31:28.910107+0000","last_unstale":"2023-07-13T22:31:31.143049+0000","last_undegraded":"2023-07-13T22:31:31.143049+0000","last_fullsized":"2023-07-13T22:31:31.143049+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:25:12.191812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143756+0000","last_change":"2023-07-13T22:31:29.907588+0000","last_active":"2023-07-13T22:31:31.143756+0000","last_peered":"2023-07-13T22:31:31.143756+0000","last_clean":"2023-07-13T22:31:31.143756+0000","last_became_active":"2023-07-13T22:31:29.907041+0000","last_became_peered":"2023-07-13T22:31:29.907041+0000","last_unstale":"2023-07-13T22:31:31.143756+0000","last_undegraded":"2023-07-13T22:31:31.143756+0000","last_fullsized":"2023-07-13T22:31:31.143756+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:11:00.603980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143136+0000","last_change":"2023-07-13T22:31:28.917103+0000","last_active":"2023-07-13T22:31:31.143136+0000","last_peered":"2023-07-13T22:31:31.143136+0000","last_clean":"2023-07-13T22:31:31.143136+0000","last_became_active":"2023-07-13T22:31:28.916803+0000","last_became_peered":"2023-07-13T22:31:28.916803+0000","last_unstale":"2023-07-13T22:31:31.143136+0000","last_undegraded":"2023-07-13T22:31:31.143136+0000","last_fullsized":"2023-07-13T22:31:31.143136+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:58.277715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143864+0000","last_change":"2023-07-13T22:31:29.920781+0000","last_active":"2023-07-13T22:31:31.143864+0000","last_peered":"2023-07-13T22:31:31.143864+0000","last_clean":"2023-07-13T22:31:31.143864+0000","last_became_active":"2023-07-13T22:31:29.915420+0000","last_became_peered":"2023-07-13T22:31:29.915420+0000","last_unstale":"2023-07-13T22:31:31.143864+0000","last_undegraded":"2023-07-13T22:31:31.143864+0000","last_fullsized":"2023-07-13T22:31:31.143864+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:33:12.868567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143990+0000","last_change":"2023-07-13T22:31:28.915944+0000","last_active":"2023-07-13T22:31:31.143990+0000","last_peered":"2023-07-13T22:31:31.143990+0000","last_clean":"2023-07-13T22:31:31.143990+0000","last_became_active":"2023-07-13T22:31:28.915347+0000","last_became_peered":"2023-07-13T22:31:28.915347+0000","last_unstale":"2023-07-13T22:31:31.143990+0000","last_undegraded":"2023-07-13T22:31:31.143990+0000","last_fullsized":"2023-07-13T22:31:31.143990+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:32:39.516956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900523+0000","last_change":"2023-07-13T22:31:29.923758+0000","last_active":"2023-07-13T22:31:30.900523+0000","last_peered":"2023-07-13T22:31:30.900523+0000","last_clean":"2023-07-13T22:31:30.900523+0000","last_became_active":"2023-07-13T22:31:29.923494+0000","last_became_peered":"2023-07-13T22:31:29.923494+0000","last_unstale":"2023-07-13T22:31:30.900523+0000","last_undegraded":"2023-07-13T22:31:30.900523+0000","last_fullsized":"2023-07-13T22:31:30.900523+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:12:00.035656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899290+0000","last_change":"2023-07-13T22:31:28.912402+0000","last_active":"2023-07-13T22:31:30.899290+0000","last_peered":"2023-07-13T22:31:30.899290+0000","last_clean":"2023-07-13T22:31:30.899290+0000","last_became_active":"2023-07-13T22:31:28.911836+0000","last_became_peered":"2023-07-13T22:31:28.911836+0000","last_unstale":"2023-07-13T22:31:30.899290+0000","last_undegraded":"2023-07-13T22:31:30.899290+0000","last_fullsized":"2023-07-13T22:31:30.899290+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:55:15.371951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900805+0000","last_change":"2023-07-13T22:31:29.920339+0000","last_active":"2023-07-13T22:31:30.900805+0000","last_peered":"2023-07-13T22:31:30.900805+0000","last_clean":"2023-07-13T22:31:30.900805+0000","last_became_active":"2023-07-13T22:31:29.920019+0000","last_became_peered":"2023-07-13T22:31:29.920019+0000","last_unstale":"2023-07-13T22:31:30.900805+0000","last_undegraded":"2023-07-13T22:31:30.900805+0000","last_fullsized":"2023-07-13T22:31:30.900805+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:13.878250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143839+0000","last_change":"2023-07-13T22:31:28.915889+0000","last_active":"2023-07-13T22:31:31.143839+0000","last_peered":"2023-07-13T22:31:31.143839+0000","last_clean":"2023-07-13T22:31:31.143839+0000","last_became_active":"2023-07-13T22:31:28.915591+0000","last_became_peered":"2023-07-13T22:31:28.915591+0000","last_unstale":"2023-07-13T22:31:31.143839+0000","last_undegraded":"2023-07-13T22:31:31.143839+0000","last_fullsized":"2023-07-13T22:31:31.143839+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:40:41.810555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142851+0000","last_change":"2023-07-13T22:31:29.922108+0000","last_active":"2023-07-13T22:31:31.142851+0000","last_peered":"2023-07-13T22:31:31.142851+0000","last_clean":"2023-07-13T22:31:31.142851+0000","last_became_active":"2023-07-13T22:31:29.921702+0000","last_became_peered":"2023-07-13T22:31:29.921702+0000","last_unstale":"2023-07-13T22:31:31.142851+0000","last_undegraded":"2023-07-13T22:31:31.142851+0000","last_fullsized":"2023-07-13T22:31:31.142851+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:27:01.163976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143113+0000","last_change":"2023-07-13T22:31:28.913125+0000","last_active":"2023-07-13T22:31:31.143113+0000","last_peered":"2023-07-13T22:31:31.143113+0000","last_clean":"2023-07-13T22:31:31.143113+0000","last_became_active":"2023-07-13T22:31:28.912855+0000","last_became_peered":"2023-07-13T22:31:28.912855+0000","last_unstale":"2023-07-13T22:31:31.143113+0000","last_undegraded":"2023-07-13T22:31:31.143113+0000","last_fullsized":"2023-07-13T22:31:31.143113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:14:28.578064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899038+0000","last_change":"2023-07-13T22:31:29.912181+0000","last_active":"2023-07-13T22:31:30.899038+0000","last_peered":"2023-07-13T22:31:30.899038+0000","last_clean":"2023-07-13T22:31:30.899038+0000","last_became_active":"2023-07-13T22:31:29.911900+0000","last_became_peered":"2023-07-13T22:31:29.911900+0000","last_unstale":"2023-07-13T22:31:30.899038+0000","last_undegraded":"2023-07-13T22:31:30.899038+0000","last_fullsized":"2023-07-13T22:31:30.899038+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:00:40.004945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900676+0000","last_change":"2023-07-13T22:31:28.912030+0000","last_active":"2023-07-13T22:31:30.900676+0000","last_peered":"2023-07-13T22:31:30.900676+0000","last_clean":"2023-07-13T22:31:30.900676+0000","last_became_active":"2023-07-13T22:31:28.911751+0000","last_became_peered":"2023-07-13T22:31:28.911751+0000","last_unstale":"2023-07-13T22:31:30.900676+0000","last_undegraded":"2023-07-13T22:31:30.900676+0000","last_fullsized":"2023-07-13T22:31:30.900676+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:47:05.072566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143448+0000","last_change":"2023-07-13T22:31:29.922669+0000","last_active":"2023-07-13T22:31:31.143448+0000","last_peered":"2023-07-13T22:31:31.143448+0000","last_clean":"2023-07-13T22:31:31.143448+0000","last_became_active":"2023-07-13T22:31:29.922360+0000","last_became_peered":"2023-07-13T22:31:29.922360+0000","last_unstale":"2023-07-13T22:31:31.143448+0000","last_undegraded":"2023-07-13T22:31:31.143448+0000","last_fullsized":"2023-07-13T22:31:31.143448+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:00:18.957574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144421+0000","last_change":"2023-07-13T22:31:28.914198+0000","last_active":"2023-07-13T22:31:31.144421+0000","last_peered":"2023-07-13T22:31:31.144421+0000","last_clean":"2023-07-13T22:31:31.144421+0000","last_became_active":"2023-07-13T22:31:28.913936+0000","last_became_peered":"2023-07-13T22:31:28.913936+0000","last_unstale":"2023-07-13T22:31:31.144421+0000","last_undegraded":"2023-07-13T22:31:31.144421+0000","last_fullsized":"2023-07-13T22:31:31.144421+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:50:43.147855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899058+0000","last_change":"2023-07-13T22:31:29.922907+0000","last_active":"2023-07-13T22:31:30.899058+0000","last_peered":"2023-07-13T22:31:30.899058+0000","last_clean":"2023-07-13T22:31:30.899058+0000","last_became_active":"2023-07-13T22:31:29.922521+0000","last_became_peered":"2023-07-13T22:31:29.922521+0000","last_unstale":"2023-07-13T22:31:30.899058+0000","last_undegraded":"2023-07-13T22:31:30.899058+0000","last_fullsized":"2023-07-13T22:31:30.899058+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:25:35.242014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900975+0000","last_change":"2023-07-13T22:31:28.910620+0000","last_active":"2023-07-13T22:31:30.900975+0000","last_peered":"2023-07-13T22:31:30.900975+0000","last_clean":"2023-07-13T22:31:30.900975+0000","last_became_active":"2023-07-13T22:31:28.910178+0000","last_became_peered":"2023-07-13T22:31:28.910178+0000","last_unstale":"2023-07-13T22:31:30.900975+0000","last_undegraded":"2023-07-13T22:31:30.900975+0000","last_fullsized":"2023-07-13T22:31:30.900975+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:05.936031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143503+0000","last_change":"2023-07-13T22:31:29.914535+0000","last_active":"2023-07-13T22:31:31.143503+0000","last_peered":"2023-07-13T22:31:31.143503+0000","last_clean":"2023-07-13T22:31:31.143503+0000","last_became_active":"2023-07-13T22:31:29.914266+0000","last_became_peered":"2023-07-13T22:31:29.914266+0000","last_unstale":"2023-07-13T22:31:31.143503+0000","last_undegraded":"2023-07-13T22:31:31.143503+0000","last_fullsized":"2023-07-13T22:31:31.143503+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:49:00.546615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143621+0000","last_change":"2023-07-13T22:31:28.918409+0000","last_active":"2023-07-13T22:31:31.143621+0000","last_peered":"2023-07-13T22:31:31.143621+0000","last_clean":"2023-07-13T22:31:31.143621+0000","last_became_active":"2023-07-13T22:31:28.918147+0000","last_became_peered":"2023-07-13T22:31:28.918147+0000","last_unstale":"2023-07-13T22:31:31.143621+0000","last_undegraded":"2023-07-13T22:31:31.143621+0000","last_fullsized":"2023-07-13T22:31:31.143621+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:26:56.815961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143243+0000","last_change":"2023-07-13T22:31:29.923384+0000","last_active":"2023-07-13T22:31:31.143243+0000","last_peered":"2023-07-13T22:31:31.143243+0000","last_clean":"2023-07-13T22:31:31.143243+0000","last_became_active":"2023-07-13T22:31:29.923097+0000","last_became_peered":"2023-07-13T22:31:29.923097+0000","last_unstale":"2023-07-13T22:31:31.143243+0000","last_undegraded":"2023-07-13T22:31:31.143243+0000","last_fullsized":"2023-07-13T22:31:31.143243+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:08:38.201736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900227+0000","last_change":"2023-07-13T22:31:28.914425+0000","last_active":"2023-07-13T22:31:30.900227+0000","last_peered":"2023-07-13T22:31:30.900227+0000","last_clean":"2023-07-13T22:31:30.900227+0000","last_became_active":"2023-07-13T22:31:28.914080+0000","last_became_peered":"2023-07-13T22:31:28.914080+0000","last_unstale":"2023-07-13T22:31:30.900227+0000","last_undegraded":"2023-07-13T22:31:30.900227+0000","last_fullsized":"2023-07-13T22:31:30.900227+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:47:09.869239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900844+0000","last_change":"2023-07-13T22:31:29.918684+0000","last_active":"2023-07-13T22:31:30.900844+0000","last_peered":"2023-07-13T22:31:30.900844+0000","last_clean":"2023-07-13T22:31:30.900844+0000","last_became_active":"2023-07-13T22:31:29.918269+0000","last_became_peered":"2023-07-13T22:31:29.918269+0000","last_unstale":"2023-07-13T22:31:30.900844+0000","last_undegraded":"2023-07-13T22:31:30.900844+0000","last_fullsized":"2023-07-13T22:31:30.900844+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:06.942293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899423+0000","last_change":"2023-07-13T22:31:28.915645+0000","last_active":"2023-07-13T22:31:30.899423+0000","last_peered":"2023-07-13T22:31:30.899423+0000","last_clean":"2023-07-13T22:31:30.899423+0000","last_became_active":"2023-07-13T22:31:28.915280+0000","last_became_peered":"2023-07-13T22:31:28.915280+0000","last_unstale":"2023-07-13T22:31:30.899423+0000","last_undegraded":"2023-07-13T22:31:30.899423+0000","last_fullsized":"2023-07-13T22:31:30.899423+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:32:12.332721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144600+0000","last_change":"2023-07-13T22:31:29.917608+0000","last_active":"2023-07-13T22:31:31.144600+0000","last_peered":"2023-07-13T22:31:31.144600+0000","last_clean":"2023-07-13T22:31:31.144600+0000","last_became_active":"2023-07-13T22:31:29.917297+0000","last_became_peered":"2023-07-13T22:31:29.917297+0000","last_unstale":"2023-07-13T22:31:31.144600+0000","last_undegraded":"2023-07-13T22:31:31.144600+0000","last_fullsized":"2023-07-13T22:31:31.144600+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:11:39.792314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143702+0000","last_change":"2023-07-13T22:31:28.913135+0000","last_active":"2023-07-13T22:31:31.143702+0000","last_peered":"2023-07-13T22:31:31.143702+0000","last_clean":"2023-07-13T22:31:31.143702+0000","last_became_active":"2023-07-13T22:31:28.912846+0000","last_became_peered":"2023-07-13T22:31:28.912846+0000","last_unstale":"2023-07-13T22:31:31.143702+0000","last_undegraded":"2023-07-13T22:31:31.143702+0000","last_fullsized":"2023-07-13T22:31:31.143702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:35:30.203714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900510+0000","last_change":"2023-07-13T22:31:29.914039+0000","last_active":"2023-07-13T22:31:30.900510+0000","last_peered":"2023-07-13T22:31:30.900510+0000","last_clean":"2023-07-13T22:31:30.900510+0000","last_became_active":"2023-07-13T22:31:29.913424+0000","last_became_peered":"2023-07-13T22:31:29.913424+0000","last_unstale":"2023-07-13T22:31:30.900510+0000","last_undegraded":"2023-07-13T22:31:30.900510+0000","last_fullsized":"2023-07-13T22:31:30.900510+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:12:00.035656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900244+0000","last_change":"2023-07-13T22:31:28.918966+0000","last_active":"2023-07-13T22:31:30.900244+0000","last_peered":"2023-07-13T22:31:30.900244+0000","last_clean":"2023-07-13T22:31:30.900244+0000","last_became_active":"2023-07-13T22:31:28.917496+0000","last_became_peered":"2023-07-13T22:31:28.917496+0000","last_unstale":"2023-07-13T22:31:30.900244+0000","last_undegraded":"2023-07-13T22:31:30.900244+0000","last_fullsized":"2023-07-13T22:31:30.900244+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:34:23.764312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142920+0000","last_change":"2023-07-13T22:31:29.923434+0000","last_active":"2023-07-13T22:31:31.142920+0000","last_peered":"2023-07-13T22:31:31.142920+0000","last_clean":"2023-07-13T22:31:31.142920+0000","last_became_active":"2023-07-13T22:31:29.923112+0000","last_became_peered":"2023-07-13T22:31:29.923112+0000","last_unstale":"2023-07-13T22:31:31.142920+0000","last_undegraded":"2023-07-13T22:31:31.142920+0000","last_fullsized":"2023-07-13T22:31:31.142920+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:52:56.209931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143345+0000","last_change":"2023-07-13T22:31:28.914573+0000","last_active":"2023-07-13T22:31:31.143345+0000","last_peered":"2023-07-13T22:31:31.143345+0000","last_clean":"2023-07-13T22:31:31.143345+0000","last_became_active":"2023-07-13T22:31:28.914279+0000","last_became_peered":"2023-07-13T22:31:28.914279+0000","last_unstale":"2023-07-13T22:31:31.143345+0000","last_undegraded":"2023-07-13T22:31:31.143345+0000","last_fullsized":"2023-07-13T22:31:31.143345+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:10:04.701456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898941+0000","last_change":"2023-07-13T22:31:29.915172+0000","last_active":"2023-07-13T22:31:30.898941+0000","last_peered":"2023-07-13T22:31:30.898941+0000","last_clean":"2023-07-13T22:31:30.898941+0000","last_became_active":"2023-07-13T22:31:29.914905+0000","last_became_peered":"2023-07-13T22:31:29.914905+0000","last_unstale":"2023-07-13T22:31:30.898941+0000","last_undegraded":"2023-07-13T22:31:30.898941+0000","last_fullsized":"2023-07-13T22:31:30.898941+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:48:08.438876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900118+0000","last_change":"2023-07-13T22:31:28.910694+0000","last_active":"2023-07-13T22:31:30.900118+0000","last_peered":"2023-07-13T22:31:30.900118+0000","last_clean":"2023-07-13T22:31:30.900118+0000","last_became_active":"2023-07-13T22:31:28.910051+0000","last_became_peered":"2023-07-13T22:31:28.910051+0000","last_unstale":"2023-07-13T22:31:30.900118+0000","last_undegraded":"2023-07-13T22:31:30.900118+0000","last_fullsized":"2023-07-13T22:31:30.900118+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:52:37.758402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901244+0000","last_change":"2023-07-13T22:31:29.908049+0000","last_active":"2023-07-13T22:31:30.901244+0000","last_peered":"2023-07-13T22:31:30.901244+0000","last_clean":"2023-07-13T22:31:30.901244+0000","last_became_active":"2023-07-13T22:31:29.906811+0000","last_became_peered":"2023-07-13T22:31:29.906811+0000","last_unstale":"2023-07-13T22:31:30.901244+0000","last_undegraded":"2023-07-13T22:31:30.901244+0000","last_fullsized":"2023-07-13T22:31:30.901244+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:09:45.314198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143584+0000","last_change":"2023-07-13T22:31:28.915033+0000","last_active":"2023-07-13T22:31:31.143584+0000","last_peered":"2023-07-13T22:31:31.143584+0000","last_clean":"2023-07-13T22:31:31.143584+0000","last_became_active":"2023-07-13T22:31:28.914707+0000","last_became_peered":"2023-07-13T22:31:28.914707+0000","last_unstale":"2023-07-13T22:31:31.143584+0000","last_undegraded":"2023-07-13T22:31:31.143584+0000","last_fullsized":"2023-07-13T22:31:31.143584+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:35:35.714280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144011+0000","last_change":"2023-07-13T22:31:29.908580+0000","last_active":"2023-07-13T22:31:31.144011+0000","last_peered":"2023-07-13T22:31:31.144011+0000","last_clean":"2023-07-13T22:31:31.144011+0000","last_became_active":"2023-07-13T22:31:29.908272+0000","last_became_peered":"2023-07-13T22:31:29.908272+0000","last_unstale":"2023-07-13T22:31:31.144011+0000","last_undegraded":"2023-07-13T22:31:31.144011+0000","last_fullsized":"2023-07-13T22:31:31.144011+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:17:21.899943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143097+0000","last_change":"2023-07-13T22:31:28.909169+0000","last_active":"2023-07-13T22:31:31.143097+0000","last_peered":"2023-07-13T22:31:31.143097+0000","last_clean":"2023-07-13T22:31:31.143097+0000","last_became_active":"2023-07-13T22:31:28.908871+0000","last_became_peered":"2023-07-13T22:31:28.908871+0000","last_unstale":"2023-07-13T22:31:31.143097+0000","last_undegraded":"2023-07-13T22:31:31.143097+0000","last_fullsized":"2023-07-13T22:31:31.143097+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:41:04.472359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":220,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901177+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:31:30.901177+0000","last_peered":"2023-07-13T22:31:30.901177+0000","last_clean":"2023-07-13T22:31:30.901177+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:31:30.901177+0000","last_undegraded":"2023-07-13T22:31:30.901177+0000","last_fullsized":"2023-07-13T22:31:30.901177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:14:53.064430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898900+0000","last_change":"2023-07-13T22:31:29.913843+0000","last_active":"2023-07-13T22:31:30.898900+0000","last_peered":"2023-07-13T22:31:30.898900+0000","last_clean":"2023-07-13T22:31:30.898900+0000","last_became_active":"2023-07-13T22:31:29.913566+0000","last_became_peered":"2023-07-13T22:31:29.913566+0000","last_unstale":"2023-07-13T22:31:30.898900+0000","last_undegraded":"2023-07-13T22:31:30.898900+0000","last_fullsized":"2023-07-13T22:31:30.898900+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:24:56.967947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"134'8","reported_seq":21,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899799+0000","last_change":"2023-07-13T22:31:28.915026+0000","last_active":"2023-07-13T22:31:30.899799+0000","last_peered":"2023-07-13T22:31:30.899799+0000","last_clean":"2023-07-13T22:31:30.899799+0000","last_became_active":"2023-07-13T22:31:28.914732+0000","last_became_peered":"2023-07-13T22:31:28.914732+0000","last_unstale":"2023-07-13T22:31:30.899799+0000","last_undegraded":"2023-07-13T22:31:30.899799+0000","last_fullsized":"2023-07-13T22:31:30.899799+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:09:49.478415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":214,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143787+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:31:31.143787+0000","last_peered":"2023-07-13T22:31:31.143787+0000","last_clean":"2023-07-13T22:31:31.143787+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:31:31.143787+0000","last_undegraded":"2023-07-13T22:31:31.143787+0000","last_fullsized":"2023-07-13T22:31:31.143787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:30:24.868444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900615+0000","last_change":"2023-07-13T22:31:29.909853+0000","last_active":"2023-07-13T22:31:30.900615+0000","last_peered":"2023-07-13T22:31:30.900615+0000","last_clean":"2023-07-13T22:31:30.900615+0000","last_became_active":"2023-07-13T22:31:29.909559+0000","last_became_peered":"2023-07-13T22:31:29.909559+0000","last_unstale":"2023-07-13T22:31:30.900615+0000","last_undegraded":"2023-07-13T22:31:30.900615+0000","last_fullsized":"2023-07-13T22:31:30.900615+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:18:33.645643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143349+0000","last_change":"2023-07-13T22:31:28.916817+0000","last_active":"2023-07-13T22:31:31.143349+0000","last_peered":"2023-07-13T22:31:31.143349+0000","last_clean":"2023-07-13T22:31:31.143349+0000","last_became_active":"2023-07-13T22:31:28.916552+0000","last_became_peered":"2023-07-13T22:31:28.916552+0000","last_unstale":"2023-07-13T22:31:31.143349+0000","last_undegraded":"2023-07-13T22:31:31.143349+0000","last_fullsized":"2023-07-13T22:31:31.143349+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:25:22.434285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":234,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899018+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:31:30.899018+0000","last_peered":"2023-07-13T22:31:30.899018+0000","last_clean":"2023-07-13T22:31:30.899018+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:31:30.899018+0000","last_undegraded":"2023-07-13T22:31:30.899018+0000","last_fullsized":"2023-07-13T22:31:30.899018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:20:08.916595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":358,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142983+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:31:31.142983+0000","last_peered":"2023-07-13T22:31:31.142983+0000","last_clean":"2023-07-13T22:31:31.142983+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:31:31.142983+0000","last_undegraded":"2023-07-13T22:31:31.142983+0000","last_fullsized":"2023-07-13T22:31:31.142983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:51:49.655836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144251+0000","last_change":"2023-07-13T22:31:29.914955+0000","last_active":"2023-07-13T22:31:31.144251+0000","last_peered":"2023-07-13T22:31:31.144251+0000","last_clean":"2023-07-13T22:31:31.144251+0000","last_became_active":"2023-07-13T22:31:29.914328+0000","last_became_peered":"2023-07-13T22:31:29.914328+0000","last_unstale":"2023-07-13T22:31:31.144251+0000","last_undegraded":"2023-07-13T22:31:31.144251+0000","last_fullsized":"2023-07-13T22:31:31.144251+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:45.968530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900363+0000","last_change":"2023-07-13T22:31:28.908754+0000","last_active":"2023-07-13T22:31:30.900363+0000","last_peered":"2023-07-13T22:31:30.900363+0000","last_clean":"2023-07-13T22:31:30.900363+0000","last_became_active":"2023-07-13T22:31:28.908428+0000","last_became_peered":"2023-07-13T22:31:28.908428+0000","last_unstale":"2023-07-13T22:31:30.900363+0000","last_undegraded":"2023-07-13T22:31:30.900363+0000","last_fullsized":"2023-07-13T22:31:30.900363+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:06:28.618215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":210,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143494+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:31:31.143494+0000","last_peered":"2023-07-13T22:31:31.143494+0000","last_clean":"2023-07-13T22:31:31.143494+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:31:31.143494+0000","last_undegraded":"2023-07-13T22:31:31.143494+0000","last_fullsized":"2023-07-13T22:31:31.143494+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:38:31.680772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901384+0000","last_change":"2023-07-13T22:31:29.912772+0000","last_active":"2023-07-13T22:31:30.901384+0000","last_peered":"2023-07-13T22:31:30.901384+0000","last_clean":"2023-07-13T22:31:30.901384+0000","last_became_active":"2023-07-13T22:31:29.912495+0000","last_became_peered":"2023-07-13T22:31:29.912495+0000","last_unstale":"2023-07-13T22:31:30.901384+0000","last_undegraded":"2023-07-13T22:31:30.901384+0000","last_fullsized":"2023-07-13T22:31:30.901384+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:30:51.094178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"134'6","reported_seq":19,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143358+0000","last_change":"2023-07-13T22:31:28.916799+0000","last_active":"2023-07-13T22:31:31.143358+0000","last_peered":"2023-07-13T22:31:31.143358+0000","last_clean":"2023-07-13T22:31:31.143358+0000","last_became_active":"2023-07-13T22:31:28.916481+0000","last_became_peered":"2023-07-13T22:31:28.916481+0000","last_unstale":"2023-07-13T22:31:31.143358+0000","last_undegraded":"2023-07-13T22:31:31.143358+0000","last_fullsized":"2023-07-13T22:31:31.143358+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:05:36.913096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":210,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143129+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:31:31.143129+0000","last_peered":"2023-07-13T22:31:31.143129+0000","last_clean":"2023-07-13T22:31:31.143129+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:31:31.143129+0000","last_undegraded":"2023-07-13T22:31:31.143129+0000","last_fullsized":"2023-07-13T22:31:31.143129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:28:36.436718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143578+0000","last_change":"2023-07-13T22:31:29.919529+0000","last_active":"2023-07-13T22:31:31.143578+0000","last_peered":"2023-07-13T22:31:31.143578+0000","last_clean":"2023-07-13T22:31:31.143578+0000","last_became_active":"2023-07-13T22:31:29.918931+0000","last_became_peered":"2023-07-13T22:31:29.918931+0000","last_unstale":"2023-07-13T22:31:31.143578+0000","last_undegraded":"2023-07-13T22:31:31.143578+0000","last_fullsized":"2023-07-13T22:31:31.143578+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:02:30.955537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"134'3","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144289+0000","last_change":"2023-07-13T22:31:28.909995+0000","last_active":"2023-07-13T22:31:31.144289+0000","last_peered":"2023-07-13T22:31:31.144289+0000","last_clean":"2023-07-13T22:31:31.144289+0000","last_became_active":"2023-07-13T22:31:28.909660+0000","last_became_peered":"2023-07-13T22:31:28.909660+0000","last_unstale":"2023-07-13T22:31:31.144289+0000","last_undegraded":"2023-07-13T22:31:31.144289+0000","last_fullsized":"2023-07-13T22:31:31.144289+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:42:54.893852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":244,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900283+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:31:30.900283+0000","last_peered":"2023-07-13T22:31:30.900283+0000","last_clean":"2023-07-13T22:31:30.900283+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:31:30.900283+0000","last_undegraded":"2023-07-13T22:31:30.900283+0000","last_fullsized":"2023-07-13T22:31:30.900283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:15:54.502293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898895+0000","last_change":"2023-07-13T22:31:29.923179+0000","last_active":"2023-07-13T22:31:30.898895+0000","last_peered":"2023-07-13T22:31:30.898895+0000","last_clean":"2023-07-13T22:31:30.898895+0000","last_became_active":"2023-07-13T22:31:29.922918+0000","last_became_peered":"2023-07-13T22:31:29.922918+0000","last_unstale":"2023-07-13T22:31:30.898895+0000","last_undegraded":"2023-07-13T22:31:30.898895+0000","last_fullsized":"2023-07-13T22:31:30.898895+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:06.942293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143357+0000","last_change":"2023-07-13T22:31:28.917690+0000","last_active":"2023-07-13T22:31:31.143357+0000","last_peered":"2023-07-13T22:31:31.143357+0000","last_clean":"2023-07-13T22:31:31.143357+0000","last_became_active":"2023-07-13T22:31:28.917278+0000","last_became_peered":"2023-07-13T22:31:28.917278+0000","last_unstale":"2023-07-13T22:31:31.143357+0000","last_undegraded":"2023-07-13T22:31:31.143357+0000","last_fullsized":"2023-07-13T22:31:31.143357+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:44:00.359237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":211,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143770+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:31:31.143770+0000","last_peered":"2023-07-13T22:31:31.143770+0000","last_clean":"2023-07-13T22:31:31.143770+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:31:31.143770+0000","last_undegraded":"2023-07-13T22:31:31.143770+0000","last_fullsized":"2023-07-13T22:31:31.143770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:22:00.964662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900805+0000","last_change":"2023-07-13T22:31:29.913823+0000","last_active":"2023-07-13T22:31:30.900805+0000","last_peered":"2023-07-13T22:31:30.900805+0000","last_clean":"2023-07-13T22:31:30.900805+0000","last_became_active":"2023-07-13T22:31:29.913101+0000","last_became_peered":"2023-07-13T22:31:29.913101+0000","last_unstale":"2023-07-13T22:31:30.900805+0000","last_undegraded":"2023-07-13T22:31:30.900805+0000","last_fullsized":"2023-07-13T22:31:30.900805+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:13.878250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143759+0000","last_change":"2023-07-13T22:31:28.909333+0000","last_active":"2023-07-13T22:31:31.143759+0000","last_peered":"2023-07-13T22:31:31.143759+0000","last_clean":"2023-07-13T22:31:31.143759+0000","last_became_active":"2023-07-13T22:31:28.909014+0000","last_became_peered":"2023-07-13T22:31:28.909014+0000","last_unstale":"2023-07-13T22:31:31.143759+0000","last_undegraded":"2023-07-13T22:31:31.143759+0000","last_fullsized":"2023-07-13T22:31:31.143759+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:55:11.816914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":211,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143571+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:31:31.143571+0000","last_peered":"2023-07-13T22:31:31.143571+0000","last_clean":"2023-07-13T22:31:31.143571+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:31:31.143571+0000","last_undegraded":"2023-07-13T22:31:31.143571+0000","last_fullsized":"2023-07-13T22:31:31.143571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:13:50.339914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37368,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2277376,"data_stored":1360432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66300000000000003,"15min":0.63700000000000001},"min":{"1min":0.371,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.89100000000000001,"5min":1.0780000000000001,"15min":1.115},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.53100000000000003,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.878,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.66500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.94299999999999995,"15min":2.2610000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.34300000000000003},"max":{"1min":0.88100000000000001,"5min":1.0309999999999999,"15min":1.0609999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71099999999999997,"15min":0.755},"min":{"1min":0.5,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95599999999999996,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70799999999999996,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.442,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":1.107,"15min":3.4710000000000001},"last":0.871}]},{"osd":3,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.34599999999999997},"max":{"1min":0.85999999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.755,"15min":0.755},"min":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.48499999999999999},"max":{"1min":0.93600000000000005,"5min":1.024,"15min":1.1399999999999999},"last":0.96399999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.76900000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.73199999999999998,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.1850000000000001},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.65300000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89500000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.627},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.53700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97099999999999997,"5min":1.0149999999999999,"15min":1.2110000000000001},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.92800000000000005,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.751},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.99099999999999999,"15min":1.042},"last":0.88300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1683456,"data_stored":770112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.65900000000000003,"15min":0.63},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.28799999999999998},"max":{"1min":0.97699999999999998,"5min":1.089,"15min":1.089},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68300000000000005,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.33900000000000002},"max":{"1min":0.89700000000000002,"5min":1.075,"15min":1.075},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.499,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.67300000000000004},"min":{"1min":0.46999999999999997,"5min":0.38500000000000001,"15min":0.379},"max":{"1min":0.871,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.67100000000000004},"min":{"1min":0.50800000000000001,"5min":0.46100000000000002,"15min":0.35599999999999998},"max":{"1min":0.90500000000000003,"5min":1.042,"15min":1.042},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69499999999999995,"15min":0.67000000000000004},"min":{"1min":0.54400000000000004,"5min":0.435,"15min":0.37},"max":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73799999999999999,"15min":0.72899999999999998},"min":{"1min":0.60399999999999998,"5min":0.52800000000000002,"15min":0.44},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.627,"15min":0.64600000000000002},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.45000000000000001,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.88600000000000001,"5min":0.90200000000000002,"15min":0.95999999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.68400000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97499999999999998,"5min":1.121,"15min":1.121},"last":0.82199999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.53500000000000003,"5min":0.439,"15min":0.375},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1890000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.72099999999999997,"15min":0.73699999999999999},"min":{"1min":0.58099999999999996,"5min":0.45300000000000001,"15min":0.39000000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.111},"last":0.84499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37368,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2277376,"data_stored":1360480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.441,"15min":0.47099999999999997},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.249},"max":{"1min":0.67900000000000005,"5min":0.70199999999999996,"15min":1.952},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.502,"15min":0.505},"min":{"1min":0.35299999999999998,"5min":0.29199999999999998,"15min":0.254},"max":{"1min":0.68000000000000005,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.503}]},{"osd":2,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46300000000000002,"15min":0.505},"min":{"1min":0.39200000000000002,"5min":0.30299999999999999,"15min":0.29199999999999998},"max":{"1min":0.58199999999999996,"5min":0.71299999999999997,"15min":2.1200000000000001},"last":0.432},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53200000000000003,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.71499999999999997,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68000000000000005,"5min":0.77200000000000002,"15min":2.008},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71599999999999997,"5min":0.93700000000000006,"15min":1.054},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.753,"15min":0.77600000000000002},"min":{"1min":0.57599999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":2.028},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76200000000000001,"15min":0.77700000000000002},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.46400000000000002},"max":{"1min":0.86699999999999999,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.93000000000000005,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88,"5min":1.0720000000000001,"15min":2.0899999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79000000000000004,"15min":0.78900000000000003},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.52600000000000002},"max":{"1min":0.97799999999999998,"5min":1.073,"15min":1.2230000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78400000000000003,"15min":0.82599999999999996},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95099999999999996,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79000000000000004,"15min":0.80700000000000005},"min":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.50700000000000001},"max":{"1min":0.95099999999999996,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.55900000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.012,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1699840,"data_stored":771326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.51300000000000001},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76800000000000002,"5min":0.84999999999999998,"15min":11.791},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29299999999999998},"max":{"1min":0.56699999999999995,"5min":0.99199999999999999,"15min":4.657},"last":0.54700000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.60799999999999998},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0349999999999999,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.51200000000000001,"15min":0.82799999999999996},"min":{"1min":0.36499999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1,"5min":1.276,"15min":13.398999999999999},"last":0.442}]},{"osd":3,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.5,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.65800000000000003,"5min":0.75800000000000001,"15min":1.8089999999999999},"last":0.501},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54300000000000004,"15min":0.56299999999999994},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.59699999999999998,"5min":1.095,"15min":4.4530000000000003},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":1.0669999999999999},"min":{"1min":0.497,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97899999999999998,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72999999999999998,"15min":0.78000000000000003},"min":{"1min":0.54800000000000004,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":0.85099999999999998,"5min":1.1220000000000001,"15min":1.776},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77300000000000002,"15min":1.091},"min":{"1min":0.56999999999999995,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":0.95999999999999996,"5min":1.113,"15min":13.349},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80600000000000005,"15min":0.85299999999999998},"min":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.88400000000000001,"5min":1.302,"15min":4.7649999999999997},"last":0.83899999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.81599999999999995},"min":{"1min":0.54300000000000004,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":0.90400000000000003,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.75},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76200000000000001,"15min":1.0589999999999999},"min":{"1min":0.59199999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89600000000000002,"5min":1.2330000000000001,"15min":11.679},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77400000000000002,"15min":1.095},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91300000000000003,"5min":1.1759999999999999,"15min":13.298},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78500000000000003,"15min":0.86699999999999999},"min":{"1min":0.63200000000000001,"5min":0.53700000000000003,"15min":0.50900000000000001},"max":{"1min":1.073,"5min":1.151,"15min":11.456},"last":0.85799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1683456,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44700000000000001,"15min":0.497},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.249},"max":{"1min":0.72899999999999998,"5min":1.605,"15min":2.282},"last":0.434},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.47799999999999998},"min":{"1min":0.32600000000000001,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":1.034,"5min":1.399,"15min":1.4990000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.47099999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.26200000000000001},"max":{"1min":0.93600000000000005,"5min":1.228,"15min":1.228},"last":0.497},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81999999999999995,"5min":1.6879999999999999,"15min":2.319},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55800000000000005},"min":{"1min":0.40699999999999997,"5min":0.371,"15min":0.311},"max":{"1min":0.84999999999999998,"5min":1.119,"15min":1.119},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58399999999999996,"15min":0.61899999999999999},"min":{"1min":0.44900000000000001,"5min":0.39300000000000002,"15min":0.38700000000000001},"max":{"1min":0.77700000000000002,"5min":1.5149999999999999,"15min":2.351},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77500000000000002,"15min":0.81000000000000005},"min":{"1min":0.59899999999999998,"5min":0.57099999999999995,"15min":0.46000000000000002},"max":{"1min":1.0069999999999999,"5min":1.6499999999999999,"15min":2.238},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76800000000000002,"15min":0.77200000000000002},"min":{"1min":0.53700000000000003,"5min":0.51200000000000001,"15min":0.46899999999999997},"max":{"1min":1.083,"5min":1.274,"15min":1.518},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61099999999999999,"5min":0.53300000000000003,"15min":0.505},"max":{"1min":1.0660000000000001,"5min":1.341,"15min":1.341},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.80600000000000005,"15min":0.82999999999999996},"min":{"1min":0.60399999999999998,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":1.052,"5min":1.9099999999999999,"15min":2.1909999999999998},"last":0.747}]},{"osd":6,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57699999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.488},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.63500000000000001,"5min":0.54900000000000004,"15min":0.48199999999999998},"max":{"1min":0.98899999999999999,"5min":1.448,"15min":1.448},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80300000000000005,"15min":0.79000000000000004},"min":{"1min":0.60699999999999998,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":1.1259999999999999,"5min":1.222,"15min":1.29},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.54400000000000004},"max":{"1min":0.91300000000000003,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.76100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40888,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330952,"statfs":{"total":96636764160,"available":96594894848,"internally_reserved":0,"allocated":1687552,"data_stored":770658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.248},"max":{"1min":0.59199999999999997,"5min":0.73199999999999998,"15min":0.73999999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48199999999999998,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.29899999999999999,"15min":0.27200000000000002},"max":{"1min":0.67500000000000004,"5min":0.73199999999999998,"15min":0.73899999999999999},"last":0.435}]},{"osd":1,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46500000000000002,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28299999999999997},"max":{"1min":0.60399999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.52600000000000002},"min":{"1min":0.39500000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.94299999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90000000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56100000000000005,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":1.0149999999999999,"5min":1.149,"15min":1.2250000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.59999999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93400000000000005,"5min":1.113,"15min":1.1539999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76400000000000001,"15min":0.753},"min":{"1min":0.57099999999999995,"5min":0.51000000000000001,"15min":0.496},"max":{"1min":1.0509999999999999,"5min":1.1919999999999999,"15min":1.411},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78600000000000003,"15min":0.77800000000000002},"min":{"1min":0.58499999999999996,"5min":0.53700000000000003,"15min":0.48599999999999999},"max":{"1min":1.05,"5min":1.175,"15min":1.5469999999999999},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.51800000000000002},"max":{"1min":1.0149999999999999,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78200000000000003,"15min":0.77100000000000002},"min":{"1min":0.622,"5min":0.52500000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":1.02,"15min":1.1240000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78600000000000003,"15min":0.79100000000000004},"min":{"1min":0.61499999999999999,"5min":0.497,"15min":0.497},"max":{"1min":0.97899999999999998,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.47199999999999998},"max":{"1min":1.0800000000000001,"5min":1.153,"15min":1.1950000000000001},"last":0.879}]}]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36808,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335032,"statfs":{"total":96636764160,"available":96599072768,"internally_reserved":0,"allocated":1703936,"data_stored":778259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66600000000000004,"15min":0.623},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.311},"max":{"1min":0.91200000000000003,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.502},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67300000000000004,"15min":0.64500000000000002},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.36199999999999999},"max":{"1min":0.96799999999999997,"5min":0.997,"15min":0.997},"last":0.71099999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.68700000000000006},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.35099999999999998},"max":{"1min":0.98899999999999999,"5min":1.0489999999999999,"15min":1.298},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72999999999999998,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.438},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.3109999999999999},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.42799999999999999,"15min":0.371},"max":{"1min":0.94499999999999995,"5min":1.0169999999999999,"15min":1.1200000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73799999999999999,"15min":0.71899999999999997},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1839999999999999},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69699999999999995,"15min":0.86499999999999999},"min":{"1min":0.53400000000000003,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":1.02,"5min":1.234,"15min":10.657999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.86299999999999999},"min":{"1min":0.51000000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":8.75},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.65900000000000003,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98199999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95799999999999996,"5min":1.0189999999999999,"15min":1.173},"last":1.012},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.45800000000000002,"15min":0.432},"max":{"1min":1.006,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.64400000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.0289999999999999,"5min":1.1819999999999999,"15min":1.212},"last":1.0389999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36796,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335044,"statfs":{"total":96636764160,"available":96599085056,"internally_reserved":0,"allocated":1691648,"data_stored":776695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65200000000000002,"15min":0.626},"min":{"1min":0.48399999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.81100000000000005,"5min":1.042,"15min":1.042},"last":0.747},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67000000000000004,"15min":0.64300000000000002},"min":{"1min":0.48199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91700000000000004,"5min":0.94099999999999995,"15min":0.998},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.88700000000000001,"5min":0.89600000000000002,"15min":0.97999999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.52900000000000003,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":8.9930000000000003},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67500000000000004,"15min":0.64600000000000002},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.96199999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.68400000000000005},"min":{"1min":0.55300000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71999999999999997,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96599999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94699999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72199999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.46200000000000002,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.93500000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68999999999999995,"15min":0.68100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94099999999999995,"5min":1.012,"15min":1.012},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68999999999999995,"15min":0.67300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":0.86799999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73999999999999999,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.41199999999999998},"max":{"1min":0.94999999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.93200000000000005,"5min":1.034,"15min":1.0669999999999999},"last":0.86299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:31:34.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:31:34.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.586+0000 7fca957ff700 1 Processor -- start 2023-07-13T22:31:34.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.586+0000 7fca957ff700 1 -- start start 2023-07-13T22:31:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.587+0000 7fca957ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fca900706f0 0x7fca900ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.587+0000 7fca957ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9015d970 con 0x7fca90070270 2023-07-13T22:31:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.587+0000 7fca957ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9015cd60 con 0x7fca90159170 2023-07-13T22:31:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.587+0000 7fca957ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9015d040 con 0x7fca900706f0 2023-07-13T22:31:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.587+0000 7fca8e7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fca90070270 0x7fca900c9c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39202/0 (socket says 172.21.15.60:39202) 2023-07-13T22:31:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.587+0000 7fca8e7fc700 1 -- 172.21.15.60:0/3947343195 learned_addr learned my addr 172.21.15.60:0/3947343195 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.587+0000 7fca8f7fe700 1 --2- 172.21.15.60:0/3947343195 >> v2:172.21.15.60:3300/0 conn(0x7fca900706f0 0x7fca900ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.588+0000 7fca8dffb700 1 -- 172.21.15.60:0/3947343195 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031223797 0 0) 0x7fca9015d970 con 0x7fca90070270 2023-07-13T22:31:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.588+0000 7fca8dffb700 1 -- 172.21.15.60:0/3947343195 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca780035b0 con 0x7fca90070270 2023-07-13T22:31:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.588+0000 7fca8f7fe700 1 -- 172.21.15.60:0/3947343195 >> v1:172.21.15.105:6789/0 conn(0x7fca90159170 legacy=0x7fca900c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.588+0000 7fca8f7fe700 1 -- 172.21.15.60:0/3947343195 >> v1:172.21.15.60:6789/0 conn(0x7fca90070270 legacy=0x7fca900c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.588+0000 7fca8f7fe700 1 -- 172.21.15.60:0/3947343195 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9015d320 con 0x7fca900706f0 2023-07-13T22:31:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.588+0000 7fca8f7fe700 1 --2- 172.21.15.60:0/3947343195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 0x7fca900ca360 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fca88009d90 tx=0x7fca8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175528245228a1a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.588+0000 7fca8dffb700 1 -- 172.21.15.60:0/3947343195 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8800e070 con 0x7fca900706f0 2023-07-13T22:31:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.589+0000 7fca8dffb700 1 -- 172.21.15.60:0/3947343195 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8800bb20 con 0x7fca900706f0 2023-07-13T22:31:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.589+0000 7fca8dffb700 1 -- 172.21.15.60:0/3947343195 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88012430 con 0x7fca900706f0 2023-07-13T22:31:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.589+0000 7fca957ff700 1 -- 172.21.15.60:0/3947343195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 msgr2=0x7fca900ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.589+0000 7fca957ff700 1 --2- 172.21.15.60:0/3947343195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 0x7fca900ca360 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fca88009d90 tx=0x7fca8800b330 comp rx=0 tx=0).stop 2023-07-13T22:31:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.589+0000 7fca957ff700 1 -- 172.21.15.60:0/3947343195 shutdown_connections 2023-07-13T22:31:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.589+0000 7fca957ff700 1 --2- 172.21.15.60:0/3947343195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 0x7fca900ca360 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.589+0000 7fca957ff700 1 -- 172.21.15.60:0/3947343195 >> 172.21.15.60:0/3947343195 conn(0x7fca900cf410 msgr2=0x7fca900cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.589+0000 7fca957ff700 1 -- 172.21.15.60:0/3947343195 shutdown_connections 2023-07-13T22:31:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.589+0000 7fca957ff700 1 -- 172.21.15.60:0/3947343195 wait complete. 2023-07-13T22:31:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.591+0000 7fca957ff700 1 Processor -- start 2023-07-13T22:31:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.591+0000 7fca957ff700 1 -- start start 2023-07-13T22:31:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca957ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca90070270 0x7fca900b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca957ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 0x7fca900b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca957ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca90159170 0x7fca900ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca957ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9013fa80 con 0x7fca900706f0 2023-07-13T22:31:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca957ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90145a10 con 0x7fca90070270 2023-07-13T22:31:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca957ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca901491e0 con 0x7fca90159170 2023-07-13T22:31:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca8e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 0x7fca900b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca8e7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 0x7fca900b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58132/0 (socket says 172.21.15.60:58132) 2023-07-13T22:31:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca8e7fc700 1 -- 172.21.15.60:0/2970205299 learned_addr learned my addr 172.21.15.60:0/2970205299 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca8f7fe700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca90159170 0x7fca900ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.592+0000 7fca8effd700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca90070270 0x7fca900b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca8e7fc700 1 -- 172.21.15.60:0/2970205299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca90159170 msgr2=0x7fca900ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca8e7fc700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca90159170 0x7fca900ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca8e7fc700 1 -- 172.21.15.60:0/2970205299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca90070270 msgr2=0x7fca900b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca8e7fc700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca90070270 0x7fca900b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca8e7fc700 1 -- 172.21.15.60:0/2970205299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca88009a70 con 0x7fca900706f0 2023-07-13T22:31:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca8e7fc700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 0x7fca900b08d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fca8400c700 tx=0x7fca8400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca777fe700 1 -- 172.21.15.60:0/2970205299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84005d70 con 0x7fca900706f0 2023-07-13T22:31:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca777fe700 1 -- 172.21.15.60:0/2970205299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca840070d0 con 0x7fca900706f0 2023-07-13T22:31:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca777fe700 1 -- 172.21.15.60:0/2970205299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84007460 con 0x7fca900706f0 2023-07-13T22:31:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca901494c0 con 0x7fca900706f0 2023-07-13T22:31:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.593+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90149a80 con 0x7fca900706f0 2023-07-13T22:31:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.594+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca900ad860 con 0x7fca900706f0 2023-07-13T22:31:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.595+0000 7fca777fe700 1 -- 172.21.15.60:0/2970205299 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca840075c0 con 0x7fca900706f0 2023-07-13T22:31:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.596+0000 7fca777fe700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca7c06ace0 0x7fca7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.596+0000 7fca777fe700 1 -- 172.21.15.60:0/2970205299 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fca84088af0 con 0x7fca900706f0 2023-07-13T22:31:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.596+0000 7fca8effd700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca7c06ace0 0x7fca7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:34.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.597+0000 7fca8effd700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca7c06ace0 0x7fca7c06d190 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fca80000f60 tx=0x7fca80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.601+0000 7fca777fe700 1 -- 172.21.15.60:0/2970205299 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fca84053b10 con 0x7fca900706f0 2023-07-13T22:31:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.803+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca90159540 con 0x7fca7c06ace0 2023-07-13T22:31:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.812+0000 7fca777fe700 1 -- 172.21.15.60:0/2970205299 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311887 (secure 0 0 0) 0x7fca800013f0 con 0x7fca7c06ace0 2023-07-13T22:31:34.798 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca7c06ace0 msgr2=0x7fca7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca7c06ace0 0x7fca7c06d190 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fca80000f60 tx=0x7fca80009040 comp rx=0 tx=0).stop 2023-07-13T22:31:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 msgr2=0x7fca900b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 0x7fca900b08d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fca8400c700 tx=0x7fca8400cac0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 shutdown_connections 2023-07-13T22:31:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca7c06ace0 0x7fca7c06d190 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca90159170 0x7fca900ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca90070270 0x7fca900b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 --2- 172.21.15.60:0/2970205299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca900706f0 0x7fca900b08d0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 >> 172.21.15.60:0/2970205299 conn(0x7fca900cf410 msgr2=0x7fca9014ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 shutdown_connections 2023-07-13T22:31:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:34.817+0000 7fca957ff700 1 -- 172.21.15.60:0/2970205299 wait complete. 2023-07-13T22:31:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:31:34.818 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2023-07-13T22:31:33.264045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295512,"kb_used_data":14360,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754679208,"statfs":{"total":773094113280,"available":772791508992,"internally_reserved":0,"allocated":14704640,"data_stored":7357130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.379832"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900526+0000","last_change":"2023-07-13T22:31:29.917401+0000","last_active":"2023-07-13T22:31:30.900526+0000","last_peered":"2023-07-13T22:31:30.900526+0000","last_clean":"2023-07-13T22:31:30.900526+0000","last_became_active":"2023-07-13T22:31:29.916972+0000","last_became_peered":"2023-07-13T22:31:29.916972+0000","last_unstale":"2023-07-13T22:31:30.900526+0000","last_undegraded":"2023-07-13T22:31:30.900526+0000","last_fullsized":"2023-07-13T22:31:30.900526+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:18:33.645643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899288+0000","last_change":"2023-07-13T22:31:29.922523+0000","last_active":"2023-07-13T22:31:30.899288+0000","last_peered":"2023-07-13T22:31:30.899288+0000","last_clean":"2023-07-13T22:31:30.899288+0000","last_became_active":"2023-07-13T22:31:29.922205+0000","last_became_peered":"2023-07-13T22:31:29.922205+0000","last_unstale":"2023-07-13T22:31:30.899288+0000","last_undegraded":"2023-07-13T22:31:30.899288+0000","last_fullsized":"2023-07-13T22:31:30.899288+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:12:01.285269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899457+0000","last_change":"2023-07-13T22:31:29.914638+0000","last_active":"2023-07-13T22:31:30.899457+0000","last_peered":"2023-07-13T22:31:30.899457+0000","last_clean":"2023-07-13T22:31:30.899457+0000","last_became_active":"2023-07-13T22:31:29.914317+0000","last_became_peered":"2023-07-13T22:31:29.914317+0000","last_unstale":"2023-07-13T22:31:30.899457+0000","last_undegraded":"2023-07-13T22:31:30.899457+0000","last_fullsized":"2023-07-13T22:31:30.899457+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:02:46.062610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143569+0000","last_change":"2023-07-13T22:31:29.919617+0000","last_active":"2023-07-13T22:31:31.143569+0000","last_peered":"2023-07-13T22:31:31.143569+0000","last_clean":"2023-07-13T22:31:31.143569+0000","last_became_active":"2023-07-13T22:31:29.919355+0000","last_became_peered":"2023-07-13T22:31:29.919355+0000","last_unstale":"2023-07-13T22:31:31.143569+0000","last_undegraded":"2023-07-13T22:31:31.143569+0000","last_fullsized":"2023-07-13T22:31:31.143569+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:43:38.942537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144032+0000","last_change":"2023-07-13T22:31:29.915088+0000","last_active":"2023-07-13T22:31:31.144032+0000","last_peered":"2023-07-13T22:31:31.144032+0000","last_clean":"2023-07-13T22:31:31.144032+0000","last_became_active":"2023-07-13T22:31:29.914804+0000","last_became_peered":"2023-07-13T22:31:29.914804+0000","last_unstale":"2023-07-13T22:31:31.144032+0000","last_undegraded":"2023-07-13T22:31:31.144032+0000","last_fullsized":"2023-07-13T22:31:31.144032+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:47.603029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143867+0000","last_change":"2023-07-13T22:31:29.911681+0000","last_active":"2023-07-13T22:31:31.143867+0000","last_peered":"2023-07-13T22:31:31.143867+0000","last_clean":"2023-07-13T22:31:31.143867+0000","last_became_active":"2023-07-13T22:31:29.911314+0000","last_became_peered":"2023-07-13T22:31:29.911314+0000","last_unstale":"2023-07-13T22:31:31.143867+0000","last_undegraded":"2023-07-13T22:31:31.143867+0000","last_fullsized":"2023-07-13T22:31:31.143867+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:56:58.536230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899811+0000","last_change":"2023-07-13T22:31:29.914194+0000","last_active":"2023-07-13T22:31:30.899811+0000","last_peered":"2023-07-13T22:31:30.899811+0000","last_clean":"2023-07-13T22:31:30.899811+0000","last_became_active":"2023-07-13T22:31:29.913929+0000","last_became_peered":"2023-07-13T22:31:29.913929+0000","last_unstale":"2023-07-13T22:31:30.899811+0000","last_undegraded":"2023-07-13T22:31:30.899811+0000","last_fullsized":"2023-07-13T22:31:30.899811+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:09:58.455819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900976+0000","last_change":"2023-07-13T22:31:29.908754+0000","last_active":"2023-07-13T22:31:30.900976+0000","last_peered":"2023-07-13T22:31:30.900976+0000","last_clean":"2023-07-13T22:31:30.900976+0000","last_became_active":"2023-07-13T22:31:29.908252+0000","last_became_peered":"2023-07-13T22:31:29.908252+0000","last_unstale":"2023-07-13T22:31:30.900976+0000","last_undegraded":"2023-07-13T22:31:30.900976+0000","last_fullsized":"2023-07-13T22:31:30.900976+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:58.420845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144396+0000","last_change":"2023-07-13T22:31:29.907645+0000","last_active":"2023-07-13T22:31:31.144396+0000","last_peered":"2023-07-13T22:31:31.144396+0000","last_clean":"2023-07-13T22:31:31.144396+0000","last_became_active":"2023-07-13T22:31:29.907352+0000","last_became_peered":"2023-07-13T22:31:29.907352+0000","last_unstale":"2023-07-13T22:31:31.144396+0000","last_undegraded":"2023-07-13T22:31:31.144396+0000","last_fullsized":"2023-07-13T22:31:31.144396+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:58:16.541596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144220+0000","last_change":"2023-07-13T22:31:29.908582+0000","last_active":"2023-07-13T22:31:31.144220+0000","last_peered":"2023-07-13T22:31:31.144220+0000","last_clean":"2023-07-13T22:31:31.144220+0000","last_became_active":"2023-07-13T22:31:29.908250+0000","last_became_peered":"2023-07-13T22:31:29.908250+0000","last_unstale":"2023-07-13T22:31:31.144220+0000","last_undegraded":"2023-07-13T22:31:31.144220+0000","last_fullsized":"2023-07-13T22:31:31.144220+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:10:25.533745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144215+0000","last_change":"2023-07-13T22:31:29.915160+0000","last_active":"2023-07-13T22:31:31.144215+0000","last_peered":"2023-07-13T22:31:31.144215+0000","last_clean":"2023-07-13T22:31:31.144215+0000","last_became_active":"2023-07-13T22:31:29.914679+0000","last_became_peered":"2023-07-13T22:31:29.914679+0000","last_unstale":"2023-07-13T22:31:31.144215+0000","last_undegraded":"2023-07-13T22:31:31.144215+0000","last_fullsized":"2023-07-13T22:31:31.144215+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:44:08.250788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143721+0000","last_change":"2023-07-13T22:31:29.923814+0000","last_active":"2023-07-13T22:31:31.143721+0000","last_peered":"2023-07-13T22:31:31.143721+0000","last_clean":"2023-07-13T22:31:31.143721+0000","last_became_active":"2023-07-13T22:31:29.923556+0000","last_became_peered":"2023-07-13T22:31:29.923556+0000","last_unstale":"2023-07-13T22:31:31.143721+0000","last_undegraded":"2023-07-13T22:31:31.143721+0000","last_fullsized":"2023-07-13T22:31:31.143721+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:59:16.137668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143994+0000","last_change":"2023-07-13T22:31:29.910377+0000","last_active":"2023-07-13T22:31:31.143994+0000","last_peered":"2023-07-13T22:31:31.143994+0000","last_clean":"2023-07-13T22:31:31.143994+0000","last_became_active":"2023-07-13T22:31:29.909831+0000","last_became_peered":"2023-07-13T22:31:29.909831+0000","last_unstale":"2023-07-13T22:31:31.143994+0000","last_undegraded":"2023-07-13T22:31:31.143994+0000","last_fullsized":"2023-07-13T22:31:31.143994+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:34:07.854269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143149+0000","last_change":"2023-07-13T22:31:29.924577+0000","last_active":"2023-07-13T22:31:31.143149+0000","last_peered":"2023-07-13T22:31:31.143149+0000","last_clean":"2023-07-13T22:31:31.143149+0000","last_became_active":"2023-07-13T22:31:29.924267+0000","last_became_peered":"2023-07-13T22:31:29.924267+0000","last_unstale":"2023-07-13T22:31:31.143149+0000","last_undegraded":"2023-07-13T22:31:31.143149+0000","last_fullsized":"2023-07-13T22:31:31.143149+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:44:45.264078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898975+0000","last_change":"2023-07-13T22:31:29.917523+0000","last_active":"2023-07-13T22:31:30.898975+0000","last_peered":"2023-07-13T22:31:30.898975+0000","last_clean":"2023-07-13T22:31:30.898975+0000","last_became_active":"2023-07-13T22:31:29.917242+0000","last_became_peered":"2023-07-13T22:31:29.917242+0000","last_unstale":"2023-07-13T22:31:30.898975+0000","last_undegraded":"2023-07-13T22:31:30.898975+0000","last_fullsized":"2023-07-13T22:31:30.898975+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:09:45.314198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142846+0000","last_change":"2023-07-13T22:31:29.907251+0000","last_active":"2023-07-13T22:31:31.142846+0000","last_peered":"2023-07-13T22:31:31.142846+0000","last_clean":"2023-07-13T22:31:31.142846+0000","last_became_active":"2023-07-13T22:31:29.906984+0000","last_became_peered":"2023-07-13T22:31:29.906984+0000","last_unstale":"2023-07-13T22:31:31.142846+0000","last_undegraded":"2023-07-13T22:31:31.142846+0000","last_fullsized":"2023-07-13T22:31:31.142846+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:35:23.619343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143462+0000","last_change":"2023-07-13T22:31:29.917980+0000","last_active":"2023-07-13T22:31:31.143462+0000","last_peered":"2023-07-13T22:31:31.143462+0000","last_clean":"2023-07-13T22:31:31.143462+0000","last_became_active":"2023-07-13T22:31:29.917682+0000","last_became_peered":"2023-07-13T22:31:29.917682+0000","last_unstale":"2023-07-13T22:31:31.143462+0000","last_undegraded":"2023-07-13T22:31:31.143462+0000","last_fullsized":"2023-07-13T22:31:31.143462+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:39:53.060278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143474+0000","last_change":"2023-07-13T22:31:29.919725+0000","last_active":"2023-07-13T22:31:31.143474+0000","last_peered":"2023-07-13T22:31:31.143474+0000","last_clean":"2023-07-13T22:31:31.143474+0000","last_became_active":"2023-07-13T22:31:29.919271+0000","last_became_peered":"2023-07-13T22:31:29.919271+0000","last_unstale":"2023-07-13T22:31:31.143474+0000","last_undegraded":"2023-07-13T22:31:31.143474+0000","last_fullsized":"2023-07-13T22:31:31.143474+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:02:44.414707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143994+0000","last_change":"2023-07-13T22:31:29.923034+0000","last_active":"2023-07-13T22:31:31.143994+0000","last_peered":"2023-07-13T22:31:31.143994+0000","last_clean":"2023-07-13T22:31:31.143994+0000","last_became_active":"2023-07-13T22:31:29.922397+0000","last_became_peered":"2023-07-13T22:31:29.922397+0000","last_unstale":"2023-07-13T22:31:31.143994+0000","last_undegraded":"2023-07-13T22:31:31.143994+0000","last_fullsized":"2023-07-13T22:31:31.143994+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:19:35.150074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143360+0000","last_change":"2023-07-13T22:31:29.915309+0000","last_active":"2023-07-13T22:31:31.143360+0000","last_peered":"2023-07-13T22:31:31.143360+0000","last_clean":"2023-07-13T22:31:31.143360+0000","last_became_active":"2023-07-13T22:31:29.915019+0000","last_became_peered":"2023-07-13T22:31:29.915019+0000","last_unstale":"2023-07-13T22:31:31.143360+0000","last_undegraded":"2023-07-13T22:31:31.143360+0000","last_fullsized":"2023-07-13T22:31:31.143360+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:55:54.167692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143899+0000","last_change":"2023-07-13T22:31:29.911309+0000","last_active":"2023-07-13T22:31:31.143899+0000","last_peered":"2023-07-13T22:31:31.143899+0000","last_clean":"2023-07-13T22:31:31.143899+0000","last_became_active":"2023-07-13T22:31:29.910941+0000","last_became_peered":"2023-07-13T22:31:29.910941+0000","last_unstale":"2023-07-13T22:31:31.143899+0000","last_undegraded":"2023-07-13T22:31:31.143899+0000","last_fullsized":"2023-07-13T22:31:31.143899+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:04:49.418121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900607+0000","last_change":"2023-07-13T22:31:29.915096+0000","last_active":"2023-07-13T22:31:30.900607+0000","last_peered":"2023-07-13T22:31:30.900607+0000","last_clean":"2023-07-13T22:31:30.900607+0000","last_became_active":"2023-07-13T22:31:29.914826+0000","last_became_peered":"2023-07-13T22:31:29.914826+0000","last_unstale":"2023-07-13T22:31:30.900607+0000","last_undegraded":"2023-07-13T22:31:30.900607+0000","last_fullsized":"2023-07-13T22:31:30.900607+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:47:06.078829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900221+0000","last_change":"2023-07-13T22:31:29.912611+0000","last_active":"2023-07-13T22:31:30.900221+0000","last_peered":"2023-07-13T22:31:30.900221+0000","last_clean":"2023-07-13T22:31:30.900221+0000","last_became_active":"2023-07-13T22:31:29.912347+0000","last_became_peered":"2023-07-13T22:31:29.912347+0000","last_unstale":"2023-07-13T22:31:30.900221+0000","last_undegraded":"2023-07-13T22:31:30.900221+0000","last_fullsized":"2023-07-13T22:31:30.900221+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:06:29.624478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142772+0000","last_change":"2023-07-13T22:31:29.910905+0000","last_active":"2023-07-13T22:31:31.142772+0000","last_peered":"2023-07-13T22:31:31.142772+0000","last_clean":"2023-07-13T22:31:31.142772+0000","last_became_active":"2023-07-13T22:31:29.910627+0000","last_became_peered":"2023-07-13T22:31:29.910627+0000","last_unstale":"2023-07-13T22:31:31.142772+0000","last_undegraded":"2023-07-13T22:31:31.142772+0000","last_fullsized":"2023-07-13T22:31:31.142772+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:43:11.434307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900755+0000","last_change":"2023-07-13T22:31:29.911816+0000","last_active":"2023-07-13T22:31:30.900755+0000","last_peered":"2023-07-13T22:31:30.900755+0000","last_clean":"2023-07-13T22:31:30.900755+0000","last_became_active":"2023-07-13T22:31:29.911411+0000","last_became_peered":"2023-07-13T22:31:29.911411+0000","last_unstale":"2023-07-13T22:31:30.900755+0000","last_undegraded":"2023-07-13T22:31:30.900755+0000","last_fullsized":"2023-07-13T22:31:30.900755+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:39:39.508445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900973+0000","last_change":"2023-07-13T22:31:29.919876+0000","last_active":"2023-07-13T22:31:30.900973+0000","last_peered":"2023-07-13T22:31:30.900973+0000","last_clean":"2023-07-13T22:31:30.900973+0000","last_became_active":"2023-07-13T22:31:29.919598+0000","last_became_peered":"2023-07-13T22:31:29.919598+0000","last_unstale":"2023-07-13T22:31:30.900973+0000","last_undegraded":"2023-07-13T22:31:30.900973+0000","last_fullsized":"2023-07-13T22:31:30.900973+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:58.420845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143988+0000","last_change":"2023-07-13T22:31:29.910507+0000","last_active":"2023-07-13T22:31:31.143988+0000","last_peered":"2023-07-13T22:31:31.143988+0000","last_clean":"2023-07-13T22:31:31.143988+0000","last_became_active":"2023-07-13T22:31:29.910212+0000","last_became_peered":"2023-07-13T22:31:29.910212+0000","last_unstale":"2023-07-13T22:31:31.143988+0000","last_undegraded":"2023-07-13T22:31:31.143988+0000","last_fullsized":"2023-07-13T22:31:31.143988+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:54:23.426448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142983+0000","last_change":"2023-07-13T22:31:29.920722+0000","last_active":"2023-07-13T22:31:31.142983+0000","last_peered":"2023-07-13T22:31:31.142983+0000","last_clean":"2023-07-13T22:31:31.142983+0000","last_became_active":"2023-07-13T22:31:29.920434+0000","last_became_peered":"2023-07-13T22:31:29.920434+0000","last_unstale":"2023-07-13T22:31:31.142983+0000","last_undegraded":"2023-07-13T22:31:31.142983+0000","last_fullsized":"2023-07-13T22:31:31.142983+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:02:52.338010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901219+0000","last_change":"2023-07-13T22:31:29.908239+0000","last_active":"2023-07-13T22:31:30.901219+0000","last_peered":"2023-07-13T22:31:30.901219+0000","last_clean":"2023-07-13T22:31:30.901219+0000","last_became_active":"2023-07-13T22:31:29.907198+0000","last_became_peered":"2023-07-13T22:31:29.907198+0000","last_unstale":"2023-07-13T22:31:30.901219+0000","last_undegraded":"2023-07-13T22:31:30.901219+0000","last_fullsized":"2023-07-13T22:31:30.901219+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:48:08.438876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143254+0000","last_change":"2023-07-13T22:31:29.916965+0000","last_active":"2023-07-13T22:31:31.143254+0000","last_peered":"2023-07-13T22:31:31.143254+0000","last_clean":"2023-07-13T22:31:31.143254+0000","last_became_active":"2023-07-13T22:31:29.916691+0000","last_became_peered":"2023-07-13T22:31:29.916691+0000","last_unstale":"2023-07-13T22:31:31.143254+0000","last_undegraded":"2023-07-13T22:31:31.143254+0000","last_fullsized":"2023-07-13T22:31:31.143254+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:19:19.898524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143267+0000","last_change":"2023-07-13T22:31:29.915891+0000","last_active":"2023-07-13T22:31:31.143267+0000","last_peered":"2023-07-13T22:31:31.143267+0000","last_clean":"2023-07-13T22:31:31.143267+0000","last_became_active":"2023-07-13T22:31:29.915620+0000","last_became_peered":"2023-07-13T22:31:29.915620+0000","last_unstale":"2023-07-13T22:31:31.143267+0000","last_undegraded":"2023-07-13T22:31:31.143267+0000","last_fullsized":"2023-07-13T22:31:31.143267+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:45:34.043244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900785+0000","last_change":"2023-07-13T22:31:29.908592+0000","last_active":"2023-07-13T22:31:30.900785+0000","last_peered":"2023-07-13T22:31:30.900785+0000","last_clean":"2023-07-13T22:31:30.900785+0000","last_became_active":"2023-07-13T22:31:29.907889+0000","last_became_peered":"2023-07-13T22:31:29.907889+0000","last_unstale":"2023-07-13T22:31:30.900785+0000","last_undegraded":"2023-07-13T22:31:30.900785+0000","last_fullsized":"2023-07-13T22:31:30.900785+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:39:39.508445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899128+0000","last_change":"2023-07-13T22:31:29.908706+0000","last_active":"2023-07-13T22:31:30.899128+0000","last_peered":"2023-07-13T22:31:30.899128+0000","last_clean":"2023-07-13T22:31:30.899128+0000","last_became_active":"2023-07-13T22:31:29.908444+0000","last_became_peered":"2023-07-13T22:31:29.908444+0000","last_unstale":"2023-07-13T22:31:30.899128+0000","last_undegraded":"2023-07-13T22:31:30.899128+0000","last_fullsized":"2023-07-13T22:31:30.899128+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:34:49.040606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"134'4","reported_seq":17,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144539+0000","last_change":"2023-07-13T22:31:28.909458+0000","last_active":"2023-07-13T22:31:31.144539+0000","last_peered":"2023-07-13T22:31:31.144539+0000","last_clean":"2023-07-13T22:31:31.144539+0000","last_became_active":"2023-07-13T22:31:28.909045+0000","last_became_peered":"2023-07-13T22:31:28.909045+0000","last_unstale":"2023-07-13T22:31:31.144539+0000","last_undegraded":"2023-07-13T22:31:31.144539+0000","last_fullsized":"2023-07-13T22:31:31.144539+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:14:04.160858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899127+0000","last_change":"2023-07-13T22:31:29.917255+0000","last_active":"2023-07-13T22:31:30.899127+0000","last_peered":"2023-07-13T22:31:30.899127+0000","last_clean":"2023-07-13T22:31:30.899127+0000","last_became_active":"2023-07-13T22:31:29.916959+0000","last_became_peered":"2023-07-13T22:31:29.916959+0000","last_unstale":"2023-07-13T22:31:30.899127+0000","last_undegraded":"2023-07-13T22:31:30.899127+0000","last_fullsized":"2023-07-13T22:31:30.899127+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:52:46.014828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143872+0000","last_change":"2023-07-13T22:31:28.916116+0000","last_active":"2023-07-13T22:31:31.143872+0000","last_peered":"2023-07-13T22:31:31.143872+0000","last_clean":"2023-07-13T22:31:31.143872+0000","last_became_active":"2023-07-13T22:31:28.915689+0000","last_became_peered":"2023-07-13T22:31:28.915689+0000","last_unstale":"2023-07-13T22:31:31.143872+0000","last_undegraded":"2023-07-13T22:31:31.143872+0000","last_fullsized":"2023-07-13T22:31:31.143872+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:58:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901082+0000","last_change":"2023-07-13T22:31:29.921351+0000","last_active":"2023-07-13T22:31:30.901082+0000","last_peered":"2023-07-13T22:31:30.901082+0000","last_clean":"2023-07-13T22:31:30.901082+0000","last_became_active":"2023-07-13T22:31:29.921039+0000","last_became_peered":"2023-07-13T22:31:29.921039+0000","last_unstale":"2023-07-13T22:31:30.901082+0000","last_undegraded":"2023-07-13T22:31:30.901082+0000","last_fullsized":"2023-07-13T22:31:30.901082+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:24:56.967947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143940+0000","last_change":"2023-07-13T22:31:28.915708+0000","last_active":"2023-07-13T22:31:31.143940+0000","last_peered":"2023-07-13T22:31:31.143940+0000","last_clean":"2023-07-13T22:31:31.143940+0000","last_became_active":"2023-07-13T22:31:28.915418+0000","last_became_peered":"2023-07-13T22:31:28.915418+0000","last_unstale":"2023-07-13T22:31:31.143940+0000","last_undegraded":"2023-07-13T22:31:31.143940+0000","last_fullsized":"2023-07-13T22:31:31.143940+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:57:49.013838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143256+0000","last_change":"2023-07-13T22:31:29.913971+0000","last_active":"2023-07-13T22:31:31.143256+0000","last_peered":"2023-07-13T22:31:31.143256+0000","last_clean":"2023-07-13T22:31:31.143256+0000","last_became_active":"2023-07-13T22:31:29.913670+0000","last_became_peered":"2023-07-13T22:31:29.913670+0000","last_unstale":"2023-07-13T22:31:31.143256+0000","last_undegraded":"2023-07-13T22:31:31.143256+0000","last_fullsized":"2023-07-13T22:31:31.143256+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:02:43.926077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899321+0000","last_change":"2023-07-13T22:31:28.914081+0000","last_active":"2023-07-13T22:31:30.899321+0000","last_peered":"2023-07-13T22:31:30.899321+0000","last_clean":"2023-07-13T22:31:30.899321+0000","last_became_active":"2023-07-13T22:31:28.913809+0000","last_became_peered":"2023-07-13T22:31:28.913809+0000","last_unstale":"2023-07-13T22:31:30.899321+0000","last_undegraded":"2023-07-13T22:31:30.899321+0000","last_fullsized":"2023-07-13T22:31:30.899321+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:16:46.467097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142806+0000","last_change":"2023-07-13T22:31:29.913646+0000","last_active":"2023-07-13T22:31:31.142806+0000","last_peered":"2023-07-13T22:31:31.142806+0000","last_clean":"2023-07-13T22:31:31.142806+0000","last_became_active":"2023-07-13T22:31:29.913331+0000","last_became_peered":"2023-07-13T22:31:29.913331+0000","last_unstale":"2023-07-13T22:31:31.142806+0000","last_undegraded":"2023-07-13T22:31:31.142806+0000","last_fullsized":"2023-07-13T22:31:31.142806+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:36:44.735862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901330+0000","last_change":"2023-07-13T22:31:28.911510+0000","last_active":"2023-07-13T22:31:30.901330+0000","last_peered":"2023-07-13T22:31:30.901330+0000","last_clean":"2023-07-13T22:31:30.901330+0000","last_became_active":"2023-07-13T22:31:28.911218+0000","last_became_peered":"2023-07-13T22:31:28.911218+0000","last_unstale":"2023-07-13T22:31:30.901330+0000","last_undegraded":"2023-07-13T22:31:30.901330+0000","last_fullsized":"2023-07-13T22:31:30.901330+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:00:38.998682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142847+0000","last_change":"2023-07-13T22:31:29.920346+0000","last_active":"2023-07-13T22:31:31.142847+0000","last_peered":"2023-07-13T22:31:31.142847+0000","last_clean":"2023-07-13T22:31:31.142847+0000","last_became_active":"2023-07-13T22:31:29.919973+0000","last_became_peered":"2023-07-13T22:31:29.919973+0000","last_unstale":"2023-07-13T22:31:31.142847+0000","last_undegraded":"2023-07-13T22:31:31.142847+0000","last_fullsized":"2023-07-13T22:31:31.142847+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:23:35.383110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144150+0000","last_change":"2023-07-13T22:31:28.910447+0000","last_active":"2023-07-13T22:31:31.144150+0000","last_peered":"2023-07-13T22:31:31.144150+0000","last_clean":"2023-07-13T22:31:31.144150+0000","last_became_active":"2023-07-13T22:31:28.910112+0000","last_became_peered":"2023-07-13T22:31:28.910112+0000","last_unstale":"2023-07-13T22:31:31.144150+0000","last_undegraded":"2023-07-13T22:31:31.144150+0000","last_fullsized":"2023-07-13T22:31:31.144150+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:46:16.906867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898894+0000","last_change":"2023-07-13T22:31:29.907620+0000","last_active":"2023-07-13T22:31:30.898894+0000","last_peered":"2023-07-13T22:31:30.898894+0000","last_clean":"2023-07-13T22:31:30.898894+0000","last_became_active":"2023-07-13T22:31:29.907343+0000","last_became_peered":"2023-07-13T22:31:29.907343+0000","last_unstale":"2023-07-13T22:31:30.898894+0000","last_undegraded":"2023-07-13T22:31:30.898894+0000","last_fullsized":"2023-07-13T22:31:30.898894+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:58.420845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899956+0000","last_change":"2023-07-13T22:31:28.910754+0000","last_active":"2023-07-13T22:31:30.899956+0000","last_peered":"2023-07-13T22:31:30.899956+0000","last_clean":"2023-07-13T22:31:30.899956+0000","last_became_active":"2023-07-13T22:31:28.910392+0000","last_became_peered":"2023-07-13T22:31:28.910392+0000","last_unstale":"2023-07-13T22:31:30.899956+0000","last_undegraded":"2023-07-13T22:31:30.899956+0000","last_fullsized":"2023-07-13T22:31:30.899956+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:42:37.191767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900094+0000","last_change":"2023-07-13T22:31:29.917805+0000","last_active":"2023-07-13T22:31:30.900094+0000","last_peered":"2023-07-13T22:31:30.900094+0000","last_clean":"2023-07-13T22:31:30.900094+0000","last_became_active":"2023-07-13T22:31:29.917525+0000","last_became_peered":"2023-07-13T22:31:29.917525+0000","last_unstale":"2023-07-13T22:31:30.900094+0000","last_undegraded":"2023-07-13T22:31:30.900094+0000","last_fullsized":"2023-07-13T22:31:30.900094+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:31:26.698669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143049+0000","last_change":"2023-07-13T22:31:28.910406+0000","last_active":"2023-07-13T22:31:31.143049+0000","last_peered":"2023-07-13T22:31:31.143049+0000","last_clean":"2023-07-13T22:31:31.143049+0000","last_became_active":"2023-07-13T22:31:28.910107+0000","last_became_peered":"2023-07-13T22:31:28.910107+0000","last_unstale":"2023-07-13T22:31:31.143049+0000","last_undegraded":"2023-07-13T22:31:31.143049+0000","last_fullsized":"2023-07-13T22:31:31.143049+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:25:12.191812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143756+0000","last_change":"2023-07-13T22:31:29.907588+0000","last_active":"2023-07-13T22:31:31.143756+0000","last_peered":"2023-07-13T22:31:31.143756+0000","last_clean":"2023-07-13T22:31:31.143756+0000","last_became_active":"2023-07-13T22:31:29.907041+0000","last_became_peered":"2023-07-13T22:31:29.907041+0000","last_unstale":"2023-07-13T22:31:31.143756+0000","last_undegraded":"2023-07-13T22:31:31.143756+0000","last_fullsized":"2023-07-13T22:31:31.143756+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:11:00.603980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143136+0000","last_change":"2023-07-13T22:31:28.917103+0000","last_active":"2023-07-13T22:31:31.143136+0000","last_peered":"2023-07-13T22:31:31.143136+0000","last_clean":"2023-07-13T22:31:31.143136+0000","last_became_active":"2023-07-13T22:31:28.916803+0000","last_became_peered":"2023-07-13T22:31:28.916803+0000","last_unstale":"2023-07-13T22:31:31.143136+0000","last_undegraded":"2023-07-13T22:31:31.143136+0000","last_fullsized":"2023-07-13T22:31:31.143136+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:58.277715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143864+0000","last_change":"2023-07-13T22:31:29.920781+0000","last_active":"2023-07-13T22:31:31.143864+0000","last_peered":"2023-07-13T22:31:31.143864+0000","last_clean":"2023-07-13T22:31:31.143864+0000","last_became_active":"2023-07-13T22:31:29.915420+0000","last_became_peered":"2023-07-13T22:31:29.915420+0000","last_unstale":"2023-07-13T22:31:31.143864+0000","last_undegraded":"2023-07-13T22:31:31.143864+0000","last_fullsized":"2023-07-13T22:31:31.143864+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:33:12.868567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143990+0000","last_change":"2023-07-13T22:31:28.915944+0000","last_active":"2023-07-13T22:31:31.143990+0000","last_peered":"2023-07-13T22:31:31.143990+0000","last_clean":"2023-07-13T22:31:31.143990+0000","last_became_active":"2023-07-13T22:31:28.915347+0000","last_became_peered":"2023-07-13T22:31:28.915347+0000","last_unstale":"2023-07-13T22:31:31.143990+0000","last_undegraded":"2023-07-13T22:31:31.143990+0000","last_fullsized":"2023-07-13T22:31:31.143990+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:32:39.516956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900523+0000","last_change":"2023-07-13T22:31:29.923758+0000","last_active":"2023-07-13T22:31:30.900523+0000","last_peered":"2023-07-13T22:31:30.900523+0000","last_clean":"2023-07-13T22:31:30.900523+0000","last_became_active":"2023-07-13T22:31:29.923494+0000","last_became_peered":"2023-07-13T22:31:29.923494+0000","last_unstale":"2023-07-13T22:31:30.900523+0000","last_undegraded":"2023-07-13T22:31:30.900523+0000","last_fullsized":"2023-07-13T22:31:30.900523+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:12:00.035656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"134'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899290+0000","last_change":"2023-07-13T22:31:28.912402+0000","last_active":"2023-07-13T22:31:30.899290+0000","last_peered":"2023-07-13T22:31:30.899290+0000","last_clean":"2023-07-13T22:31:30.899290+0000","last_became_active":"2023-07-13T22:31:28.911836+0000","last_became_peered":"2023-07-13T22:31:28.911836+0000","last_unstale":"2023-07-13T22:31:30.899290+0000","last_undegraded":"2023-07-13T22:31:30.899290+0000","last_fullsized":"2023-07-13T22:31:30.899290+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:55:15.371951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900805+0000","last_change":"2023-07-13T22:31:29.920339+0000","last_active":"2023-07-13T22:31:30.900805+0000","last_peered":"2023-07-13T22:31:30.900805+0000","last_clean":"2023-07-13T22:31:30.900805+0000","last_became_active":"2023-07-13T22:31:29.920019+0000","last_became_peered":"2023-07-13T22:31:29.920019+0000","last_unstale":"2023-07-13T22:31:30.900805+0000","last_undegraded":"2023-07-13T22:31:30.900805+0000","last_fullsized":"2023-07-13T22:31:30.900805+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:13.878250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143839+0000","last_change":"2023-07-13T22:31:28.915889+0000","last_active":"2023-07-13T22:31:31.143839+0000","last_peered":"2023-07-13T22:31:31.143839+0000","last_clean":"2023-07-13T22:31:31.143839+0000","last_became_active":"2023-07-13T22:31:28.915591+0000","last_became_peered":"2023-07-13T22:31:28.915591+0000","last_unstale":"2023-07-13T22:31:31.143839+0000","last_undegraded":"2023-07-13T22:31:31.143839+0000","last_fullsized":"2023-07-13T22:31:31.143839+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:40:41.810555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142851+0000","last_change":"2023-07-13T22:31:29.922108+0000","last_active":"2023-07-13T22:31:31.142851+0000","last_peered":"2023-07-13T22:31:31.142851+0000","last_clean":"2023-07-13T22:31:31.142851+0000","last_became_active":"2023-07-13T22:31:29.921702+0000","last_became_peered":"2023-07-13T22:31:29.921702+0000","last_unstale":"2023-07-13T22:31:31.142851+0000","last_undegraded":"2023-07-13T22:31:31.142851+0000","last_fullsized":"2023-07-13T22:31:31.142851+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:27:01.163976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143113+0000","last_change":"2023-07-13T22:31:28.913125+0000","last_active":"2023-07-13T22:31:31.143113+0000","last_peered":"2023-07-13T22:31:31.143113+0000","last_clean":"2023-07-13T22:31:31.143113+0000","last_became_active":"2023-07-13T22:31:28.912855+0000","last_became_peered":"2023-07-13T22:31:28.912855+0000","last_unstale":"2023-07-13T22:31:31.143113+0000","last_undegraded":"2023-07-13T22:31:31.143113+0000","last_fullsized":"2023-07-13T22:31:31.143113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:14:28.578064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899038+0000","last_change":"2023-07-13T22:31:29.912181+0000","last_active":"2023-07-13T22:31:30.899038+0000","last_peered":"2023-07-13T22:31:30.899038+0000","last_clean":"2023-07-13T22:31:30.899038+0000","last_became_active":"2023-07-13T22:31:29.911900+0000","last_became_peered":"2023-07-13T22:31:29.911900+0000","last_unstale":"2023-07-13T22:31:30.899038+0000","last_undegraded":"2023-07-13T22:31:30.899038+0000","last_fullsized":"2023-07-13T22:31:30.899038+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:00:40.004945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900676+0000","last_change":"2023-07-13T22:31:28.912030+0000","last_active":"2023-07-13T22:31:30.900676+0000","last_peered":"2023-07-13T22:31:30.900676+0000","last_clean":"2023-07-13T22:31:30.900676+0000","last_became_active":"2023-07-13T22:31:28.911751+0000","last_became_peered":"2023-07-13T22:31:28.911751+0000","last_unstale":"2023-07-13T22:31:30.900676+0000","last_undegraded":"2023-07-13T22:31:30.900676+0000","last_fullsized":"2023-07-13T22:31:30.900676+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:47:05.072566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143448+0000","last_change":"2023-07-13T22:31:29.922669+0000","last_active":"2023-07-13T22:31:31.143448+0000","last_peered":"2023-07-13T22:31:31.143448+0000","last_clean":"2023-07-13T22:31:31.143448+0000","last_became_active":"2023-07-13T22:31:29.922360+0000","last_became_peered":"2023-07-13T22:31:29.922360+0000","last_unstale":"2023-07-13T22:31:31.143448+0000","last_undegraded":"2023-07-13T22:31:31.143448+0000","last_fullsized":"2023-07-13T22:31:31.143448+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:00:18.957574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144421+0000","last_change":"2023-07-13T22:31:28.914198+0000","last_active":"2023-07-13T22:31:31.144421+0000","last_peered":"2023-07-13T22:31:31.144421+0000","last_clean":"2023-07-13T22:31:31.144421+0000","last_became_active":"2023-07-13T22:31:28.913936+0000","last_became_peered":"2023-07-13T22:31:28.913936+0000","last_unstale":"2023-07-13T22:31:31.144421+0000","last_undegraded":"2023-07-13T22:31:31.144421+0000","last_fullsized":"2023-07-13T22:31:31.144421+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:50:43.147855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899058+0000","last_change":"2023-07-13T22:31:29.922907+0000","last_active":"2023-07-13T22:31:30.899058+0000","last_peered":"2023-07-13T22:31:30.899058+0000","last_clean":"2023-07-13T22:31:30.899058+0000","last_became_active":"2023-07-13T22:31:29.922521+0000","last_became_peered":"2023-07-13T22:31:29.922521+0000","last_unstale":"2023-07-13T22:31:30.899058+0000","last_undegraded":"2023-07-13T22:31:30.899058+0000","last_fullsized":"2023-07-13T22:31:30.899058+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:25:35.242014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900975+0000","last_change":"2023-07-13T22:31:28.910620+0000","last_active":"2023-07-13T22:31:30.900975+0000","last_peered":"2023-07-13T22:31:30.900975+0000","last_clean":"2023-07-13T22:31:30.900975+0000","last_became_active":"2023-07-13T22:31:28.910178+0000","last_became_peered":"2023-07-13T22:31:28.910178+0000","last_unstale":"2023-07-13T22:31:30.900975+0000","last_undegraded":"2023-07-13T22:31:30.900975+0000","last_fullsized":"2023-07-13T22:31:30.900975+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:05.936031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143503+0000","last_change":"2023-07-13T22:31:29.914535+0000","last_active":"2023-07-13T22:31:31.143503+0000","last_peered":"2023-07-13T22:31:31.143503+0000","last_clean":"2023-07-13T22:31:31.143503+0000","last_became_active":"2023-07-13T22:31:29.914266+0000","last_became_peered":"2023-07-13T22:31:29.914266+0000","last_unstale":"2023-07-13T22:31:31.143503+0000","last_undegraded":"2023-07-13T22:31:31.143503+0000","last_fullsized":"2023-07-13T22:31:31.143503+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:49:00.546615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143621+0000","last_change":"2023-07-13T22:31:28.918409+0000","last_active":"2023-07-13T22:31:31.143621+0000","last_peered":"2023-07-13T22:31:31.143621+0000","last_clean":"2023-07-13T22:31:31.143621+0000","last_became_active":"2023-07-13T22:31:28.918147+0000","last_became_peered":"2023-07-13T22:31:28.918147+0000","last_unstale":"2023-07-13T22:31:31.143621+0000","last_undegraded":"2023-07-13T22:31:31.143621+0000","last_fullsized":"2023-07-13T22:31:31.143621+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:26:56.815961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143243+0000","last_change":"2023-07-13T22:31:29.923384+0000","last_active":"2023-07-13T22:31:31.143243+0000","last_peered":"2023-07-13T22:31:31.143243+0000","last_clean":"2023-07-13T22:31:31.143243+0000","last_became_active":"2023-07-13T22:31:29.923097+0000","last_became_peered":"2023-07-13T22:31:29.923097+0000","last_unstale":"2023-07-13T22:31:31.143243+0000","last_undegraded":"2023-07-13T22:31:31.143243+0000","last_fullsized":"2023-07-13T22:31:31.143243+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:08:38.201736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900227+0000","last_change":"2023-07-13T22:31:28.914425+0000","last_active":"2023-07-13T22:31:30.900227+0000","last_peered":"2023-07-13T22:31:30.900227+0000","last_clean":"2023-07-13T22:31:30.900227+0000","last_became_active":"2023-07-13T22:31:28.914080+0000","last_became_peered":"2023-07-13T22:31:28.914080+0000","last_unstale":"2023-07-13T22:31:30.900227+0000","last_undegraded":"2023-07-13T22:31:30.900227+0000","last_fullsized":"2023-07-13T22:31:30.900227+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:47:09.869239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900844+0000","last_change":"2023-07-13T22:31:29.918684+0000","last_active":"2023-07-13T22:31:30.900844+0000","last_peered":"2023-07-13T22:31:30.900844+0000","last_clean":"2023-07-13T22:31:30.900844+0000","last_became_active":"2023-07-13T22:31:29.918269+0000","last_became_peered":"2023-07-13T22:31:29.918269+0000","last_unstale":"2023-07-13T22:31:30.900844+0000","last_undegraded":"2023-07-13T22:31:30.900844+0000","last_fullsized":"2023-07-13T22:31:30.900844+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:06.942293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899423+0000","last_change":"2023-07-13T22:31:28.915645+0000","last_active":"2023-07-13T22:31:30.899423+0000","last_peered":"2023-07-13T22:31:30.899423+0000","last_clean":"2023-07-13T22:31:30.899423+0000","last_became_active":"2023-07-13T22:31:28.915280+0000","last_became_peered":"2023-07-13T22:31:28.915280+0000","last_unstale":"2023-07-13T22:31:30.899423+0000","last_undegraded":"2023-07-13T22:31:30.899423+0000","last_fullsized":"2023-07-13T22:31:30.899423+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:32:12.332721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144600+0000","last_change":"2023-07-13T22:31:29.917608+0000","last_active":"2023-07-13T22:31:31.144600+0000","last_peered":"2023-07-13T22:31:31.144600+0000","last_clean":"2023-07-13T22:31:31.144600+0000","last_became_active":"2023-07-13T22:31:29.917297+0000","last_became_peered":"2023-07-13T22:31:29.917297+0000","last_unstale":"2023-07-13T22:31:31.144600+0000","last_undegraded":"2023-07-13T22:31:31.144600+0000","last_fullsized":"2023-07-13T22:31:31.144600+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:11:39.792314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143702+0000","last_change":"2023-07-13T22:31:28.913135+0000","last_active":"2023-07-13T22:31:31.143702+0000","last_peered":"2023-07-13T22:31:31.143702+0000","last_clean":"2023-07-13T22:31:31.143702+0000","last_became_active":"2023-07-13T22:31:28.912846+0000","last_became_peered":"2023-07-13T22:31:28.912846+0000","last_unstale":"2023-07-13T22:31:31.143702+0000","last_undegraded":"2023-07-13T22:31:31.143702+0000","last_fullsized":"2023-07-13T22:31:31.143702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:35:30.203714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900510+0000","last_change":"2023-07-13T22:31:29.914039+0000","last_active":"2023-07-13T22:31:30.900510+0000","last_peered":"2023-07-13T22:31:30.900510+0000","last_clean":"2023-07-13T22:31:30.900510+0000","last_became_active":"2023-07-13T22:31:29.913424+0000","last_became_peered":"2023-07-13T22:31:29.913424+0000","last_unstale":"2023-07-13T22:31:30.900510+0000","last_undegraded":"2023-07-13T22:31:30.900510+0000","last_fullsized":"2023-07-13T22:31:30.900510+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:12:00.035656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900244+0000","last_change":"2023-07-13T22:31:28.918966+0000","last_active":"2023-07-13T22:31:30.900244+0000","last_peered":"2023-07-13T22:31:30.900244+0000","last_clean":"2023-07-13T22:31:30.900244+0000","last_became_active":"2023-07-13T22:31:28.917496+0000","last_became_peered":"2023-07-13T22:31:28.917496+0000","last_unstale":"2023-07-13T22:31:30.900244+0000","last_undegraded":"2023-07-13T22:31:30.900244+0000","last_fullsized":"2023-07-13T22:31:30.900244+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:34:23.764312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142920+0000","last_change":"2023-07-13T22:31:29.923434+0000","last_active":"2023-07-13T22:31:31.142920+0000","last_peered":"2023-07-13T22:31:31.142920+0000","last_clean":"2023-07-13T22:31:31.142920+0000","last_became_active":"2023-07-13T22:31:29.923112+0000","last_became_peered":"2023-07-13T22:31:29.923112+0000","last_unstale":"2023-07-13T22:31:31.142920+0000","last_undegraded":"2023-07-13T22:31:31.142920+0000","last_fullsized":"2023-07-13T22:31:31.142920+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:52:56.209931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143345+0000","last_change":"2023-07-13T22:31:28.914573+0000","last_active":"2023-07-13T22:31:31.143345+0000","last_peered":"2023-07-13T22:31:31.143345+0000","last_clean":"2023-07-13T22:31:31.143345+0000","last_became_active":"2023-07-13T22:31:28.914279+0000","last_became_peered":"2023-07-13T22:31:28.914279+0000","last_unstale":"2023-07-13T22:31:31.143345+0000","last_undegraded":"2023-07-13T22:31:31.143345+0000","last_fullsized":"2023-07-13T22:31:31.143345+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:10:04.701456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898941+0000","last_change":"2023-07-13T22:31:29.915172+0000","last_active":"2023-07-13T22:31:30.898941+0000","last_peered":"2023-07-13T22:31:30.898941+0000","last_clean":"2023-07-13T22:31:30.898941+0000","last_became_active":"2023-07-13T22:31:29.914905+0000","last_became_peered":"2023-07-13T22:31:29.914905+0000","last_unstale":"2023-07-13T22:31:30.898941+0000","last_undegraded":"2023-07-13T22:31:30.898941+0000","last_fullsized":"2023-07-13T22:31:30.898941+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:48:08.438876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900118+0000","last_change":"2023-07-13T22:31:28.910694+0000","last_active":"2023-07-13T22:31:30.900118+0000","last_peered":"2023-07-13T22:31:30.900118+0000","last_clean":"2023-07-13T22:31:30.900118+0000","last_became_active":"2023-07-13T22:31:28.910051+0000","last_became_peered":"2023-07-13T22:31:28.910051+0000","last_unstale":"2023-07-13T22:31:30.900118+0000","last_undegraded":"2023-07-13T22:31:30.900118+0000","last_fullsized":"2023-07-13T22:31:30.900118+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:52:37.758402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901244+0000","last_change":"2023-07-13T22:31:29.908049+0000","last_active":"2023-07-13T22:31:30.901244+0000","last_peered":"2023-07-13T22:31:30.901244+0000","last_clean":"2023-07-13T22:31:30.901244+0000","last_became_active":"2023-07-13T22:31:29.906811+0000","last_became_peered":"2023-07-13T22:31:29.906811+0000","last_unstale":"2023-07-13T22:31:30.901244+0000","last_undegraded":"2023-07-13T22:31:30.901244+0000","last_fullsized":"2023-07-13T22:31:30.901244+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:09:45.314198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143584+0000","last_change":"2023-07-13T22:31:28.915033+0000","last_active":"2023-07-13T22:31:31.143584+0000","last_peered":"2023-07-13T22:31:31.143584+0000","last_clean":"2023-07-13T22:31:31.143584+0000","last_became_active":"2023-07-13T22:31:28.914707+0000","last_became_peered":"2023-07-13T22:31:28.914707+0000","last_unstale":"2023-07-13T22:31:31.143584+0000","last_undegraded":"2023-07-13T22:31:31.143584+0000","last_fullsized":"2023-07-13T22:31:31.143584+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:35:35.714280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144011+0000","last_change":"2023-07-13T22:31:29.908580+0000","last_active":"2023-07-13T22:31:31.144011+0000","last_peered":"2023-07-13T22:31:31.144011+0000","last_clean":"2023-07-13T22:31:31.144011+0000","last_became_active":"2023-07-13T22:31:29.908272+0000","last_became_peered":"2023-07-13T22:31:29.908272+0000","last_unstale":"2023-07-13T22:31:31.144011+0000","last_undegraded":"2023-07-13T22:31:31.144011+0000","last_fullsized":"2023-07-13T22:31:31.144011+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:17:21.899943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143097+0000","last_change":"2023-07-13T22:31:28.909169+0000","last_active":"2023-07-13T22:31:31.143097+0000","last_peered":"2023-07-13T22:31:31.143097+0000","last_clean":"2023-07-13T22:31:31.143097+0000","last_became_active":"2023-07-13T22:31:28.908871+0000","last_became_peered":"2023-07-13T22:31:28.908871+0000","last_unstale":"2023-07-13T22:31:31.143097+0000","last_undegraded":"2023-07-13T22:31:31.143097+0000","last_fullsized":"2023-07-13T22:31:31.143097+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:41:04.472359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":220,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901177+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:31:30.901177+0000","last_peered":"2023-07-13T22:31:30.901177+0000","last_clean":"2023-07-13T22:31:30.901177+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:31:30.901177+0000","last_undegraded":"2023-07-13T22:31:30.901177+0000","last_fullsized":"2023-07-13T22:31:30.901177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:14:53.064430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898900+0000","last_change":"2023-07-13T22:31:29.913843+0000","last_active":"2023-07-13T22:31:30.898900+0000","last_peered":"2023-07-13T22:31:30.898900+0000","last_clean":"2023-07-13T22:31:30.898900+0000","last_became_active":"2023-07-13T22:31:29.913566+0000","last_became_peered":"2023-07-13T22:31:29.913566+0000","last_unstale":"2023-07-13T22:31:30.898900+0000","last_undegraded":"2023-07-13T22:31:30.898900+0000","last_fullsized":"2023-07-13T22:31:30.898900+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:24:56.967947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"134'8","reported_seq":21,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899799+0000","last_change":"2023-07-13T22:31:28.915026+0000","last_active":"2023-07-13T22:31:30.899799+0000","last_peered":"2023-07-13T22:31:30.899799+0000","last_clean":"2023-07-13T22:31:30.899799+0000","last_became_active":"2023-07-13T22:31:28.914732+0000","last_became_peered":"2023-07-13T22:31:28.914732+0000","last_unstale":"2023-07-13T22:31:30.899799+0000","last_undegraded":"2023-07-13T22:31:30.899799+0000","last_fullsized":"2023-07-13T22:31:30.899799+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:09:49.478415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":214,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143787+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:31:31.143787+0000","last_peered":"2023-07-13T22:31:31.143787+0000","last_clean":"2023-07-13T22:31:31.143787+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:31:31.143787+0000","last_undegraded":"2023-07-13T22:31:31.143787+0000","last_fullsized":"2023-07-13T22:31:31.143787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:30:24.868444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900615+0000","last_change":"2023-07-13T22:31:29.909853+0000","last_active":"2023-07-13T22:31:30.900615+0000","last_peered":"2023-07-13T22:31:30.900615+0000","last_clean":"2023-07-13T22:31:30.900615+0000","last_became_active":"2023-07-13T22:31:29.909559+0000","last_became_peered":"2023-07-13T22:31:29.909559+0000","last_unstale":"2023-07-13T22:31:30.900615+0000","last_undegraded":"2023-07-13T22:31:30.900615+0000","last_fullsized":"2023-07-13T22:31:30.900615+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:18:33.645643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143349+0000","last_change":"2023-07-13T22:31:28.916817+0000","last_active":"2023-07-13T22:31:31.143349+0000","last_peered":"2023-07-13T22:31:31.143349+0000","last_clean":"2023-07-13T22:31:31.143349+0000","last_became_active":"2023-07-13T22:31:28.916552+0000","last_became_peered":"2023-07-13T22:31:28.916552+0000","last_unstale":"2023-07-13T22:31:31.143349+0000","last_undegraded":"2023-07-13T22:31:31.143349+0000","last_fullsized":"2023-07-13T22:31:31.143349+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:25:22.434285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":234,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.899018+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:31:30.899018+0000","last_peered":"2023-07-13T22:31:30.899018+0000","last_clean":"2023-07-13T22:31:30.899018+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:31:30.899018+0000","last_undegraded":"2023-07-13T22:31:30.899018+0000","last_fullsized":"2023-07-13T22:31:30.899018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:20:08.916595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":358,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.142983+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:31:31.142983+0000","last_peered":"2023-07-13T22:31:31.142983+0000","last_clean":"2023-07-13T22:31:31.142983+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:31:31.142983+0000","last_undegraded":"2023-07-13T22:31:31.142983+0000","last_fullsized":"2023-07-13T22:31:31.142983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:51:49.655836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144251+0000","last_change":"2023-07-13T22:31:29.914955+0000","last_active":"2023-07-13T22:31:31.144251+0000","last_peered":"2023-07-13T22:31:31.144251+0000","last_clean":"2023-07-13T22:31:31.144251+0000","last_became_active":"2023-07-13T22:31:29.914328+0000","last_became_peered":"2023-07-13T22:31:29.914328+0000","last_unstale":"2023-07-13T22:31:31.144251+0000","last_undegraded":"2023-07-13T22:31:31.144251+0000","last_fullsized":"2023-07-13T22:31:31.144251+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:57:45.968530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900363+0000","last_change":"2023-07-13T22:31:28.908754+0000","last_active":"2023-07-13T22:31:30.900363+0000","last_peered":"2023-07-13T22:31:30.900363+0000","last_clean":"2023-07-13T22:31:30.900363+0000","last_became_active":"2023-07-13T22:31:28.908428+0000","last_became_peered":"2023-07-13T22:31:28.908428+0000","last_unstale":"2023-07-13T22:31:30.900363+0000","last_undegraded":"2023-07-13T22:31:30.900363+0000","last_fullsized":"2023-07-13T22:31:30.900363+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:06:28.618215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":210,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143494+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:31:31.143494+0000","last_peered":"2023-07-13T22:31:31.143494+0000","last_clean":"2023-07-13T22:31:31.143494+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:31:31.143494+0000","last_undegraded":"2023-07-13T22:31:31.143494+0000","last_fullsized":"2023-07-13T22:31:31.143494+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:38:31.680772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.901384+0000","last_change":"2023-07-13T22:31:29.912772+0000","last_active":"2023-07-13T22:31:30.901384+0000","last_peered":"2023-07-13T22:31:30.901384+0000","last_clean":"2023-07-13T22:31:30.901384+0000","last_became_active":"2023-07-13T22:31:29.912495+0000","last_became_peered":"2023-07-13T22:31:29.912495+0000","last_unstale":"2023-07-13T22:31:30.901384+0000","last_undegraded":"2023-07-13T22:31:30.901384+0000","last_fullsized":"2023-07-13T22:31:30.901384+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:30:51.094178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"134'6","reported_seq":19,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143358+0000","last_change":"2023-07-13T22:31:28.916799+0000","last_active":"2023-07-13T22:31:31.143358+0000","last_peered":"2023-07-13T22:31:31.143358+0000","last_clean":"2023-07-13T22:31:31.143358+0000","last_became_active":"2023-07-13T22:31:28.916481+0000","last_became_peered":"2023-07-13T22:31:28.916481+0000","last_unstale":"2023-07-13T22:31:31.143358+0000","last_undegraded":"2023-07-13T22:31:31.143358+0000","last_fullsized":"2023-07-13T22:31:31.143358+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:05:36.913096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":210,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143129+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:31:31.143129+0000","last_peered":"2023-07-13T22:31:31.143129+0000","last_clean":"2023-07-13T22:31:31.143129+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:31:31.143129+0000","last_undegraded":"2023-07-13T22:31:31.143129+0000","last_fullsized":"2023-07-13T22:31:31.143129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:28:36.436718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143578+0000","last_change":"2023-07-13T22:31:29.919529+0000","last_active":"2023-07-13T22:31:31.143578+0000","last_peered":"2023-07-13T22:31:31.143578+0000","last_clean":"2023-07-13T22:31:31.143578+0000","last_became_active":"2023-07-13T22:31:29.918931+0000","last_became_peered":"2023-07-13T22:31:29.918931+0000","last_unstale":"2023-07-13T22:31:31.143578+0000","last_undegraded":"2023-07-13T22:31:31.143578+0000","last_fullsized":"2023-07-13T22:31:31.143578+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:02:30.955537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"134'3","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.144289+0000","last_change":"2023-07-13T22:31:28.909995+0000","last_active":"2023-07-13T22:31:31.144289+0000","last_peered":"2023-07-13T22:31:31.144289+0000","last_clean":"2023-07-13T22:31:31.144289+0000","last_became_active":"2023-07-13T22:31:28.909660+0000","last_became_peered":"2023-07-13T22:31:28.909660+0000","last_unstale":"2023-07-13T22:31:31.144289+0000","last_undegraded":"2023-07-13T22:31:31.144289+0000","last_fullsized":"2023-07-13T22:31:31.144289+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:42:54.893852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":244,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900283+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:31:30.900283+0000","last_peered":"2023-07-13T22:31:30.900283+0000","last_clean":"2023-07-13T22:31:30.900283+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:31:30.900283+0000","last_undegraded":"2023-07-13T22:31:30.900283+0000","last_fullsized":"2023-07-13T22:31:30.900283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:15:54.502293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.898895+0000","last_change":"2023-07-13T22:31:29.923179+0000","last_active":"2023-07-13T22:31:30.898895+0000","last_peered":"2023-07-13T22:31:30.898895+0000","last_clean":"2023-07-13T22:31:30.898895+0000","last_became_active":"2023-07-13T22:31:29.922918+0000","last_became_peered":"2023-07-13T22:31:29.922918+0000","last_unstale":"2023-07-13T22:31:30.898895+0000","last_undegraded":"2023-07-13T22:31:30.898895+0000","last_fullsized":"2023-07-13T22:31:30.898895+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:06.942293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143357+0000","last_change":"2023-07-13T22:31:28.917690+0000","last_active":"2023-07-13T22:31:31.143357+0000","last_peered":"2023-07-13T22:31:31.143357+0000","last_clean":"2023-07-13T22:31:31.143357+0000","last_became_active":"2023-07-13T22:31:28.917278+0000","last_became_peered":"2023-07-13T22:31:28.917278+0000","last_unstale":"2023-07-13T22:31:31.143357+0000","last_undegraded":"2023-07-13T22:31:31.143357+0000","last_fullsized":"2023-07-13T22:31:31.143357+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:44:00.359237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":211,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143770+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:31:31.143770+0000","last_peered":"2023-07-13T22:31:31.143770+0000","last_clean":"2023-07-13T22:31:31.143770+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:31:31.143770+0000","last_undegraded":"2023-07-13T22:31:31.143770+0000","last_fullsized":"2023-07-13T22:31:31.143770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:22:00.964662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:30.900805+0000","last_change":"2023-07-13T22:31:29.913823+0000","last_active":"2023-07-13T22:31:30.900805+0000","last_peered":"2023-07-13T22:31:30.900805+0000","last_clean":"2023-07-13T22:31:30.900805+0000","last_became_active":"2023-07-13T22:31:29.913101+0000","last_became_peered":"2023-07-13T22:31:29.913101+0000","last_unstale":"2023-07-13T22:31:30.900805+0000","last_undegraded":"2023-07-13T22:31:30.900805+0000","last_fullsized":"2023-07-13T22:31:30.900805+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:28.883677+0000","last_clean_scrub_stamp":"2023-07-13T22:31:28.883677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:13.878250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"134'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143759+0000","last_change":"2023-07-13T22:31:28.909333+0000","last_active":"2023-07-13T22:31:31.143759+0000","last_peered":"2023-07-13T22:31:31.143759+0000","last_clean":"2023-07-13T22:31:31.143759+0000","last_became_active":"2023-07-13T22:31:28.909014+0000","last_became_peered":"2023-07-13T22:31:28.909014+0000","last_unstale":"2023-07-13T22:31:31.143759+0000","last_undegraded":"2023-07-13T22:31:31.143759+0000","last_fullsized":"2023-07-13T22:31:31.143759+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:31:27.877414+0000","last_clean_scrub_stamp":"2023-07-13T22:31:27.877414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:55:11.816914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":211,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-13T22:31:31.143571+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:31:31.143571+0000","last_peered":"2023-07-13T22:31:31.143571+0000","last_clean":"2023-07-13T22:31:31.143571+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:31:31.143571+0000","last_undegraded":"2023-07-13T22:31:31.143571+0000","last_fullsized":"2023-07-13T22:31:31.143571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:13:50.339914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37368,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2277376,"data_stored":1360432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66300000000000003,"15min":0.63700000000000001},"min":{"1min":0.371,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.89100000000000001,"5min":1.0780000000000001,"15min":1.115},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.53100000000000003,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.878,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63500000000000001,"15min":0.66500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.94299999999999995,"15min":2.2610000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.34300000000000003},"max":{"1min":0.88100000000000001,"5min":1.0309999999999999,"15min":1.0609999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71099999999999997,"15min":0.755},"min":{"1min":0.5,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95599999999999996,"5min":0.97499999999999998,"15min":3.5190000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70799999999999996,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.442,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":1.107,"15min":3.4710000000000001},"last":0.871}]},{"osd":3,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.34599999999999997},"max":{"1min":0.85999999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.755,"15min":0.755},"min":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.48499999999999999},"max":{"1min":0.93600000000000005,"5min":1.024,"15min":1.1399999999999999},"last":0.96399999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.76900000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.73199999999999998,"15min":0.73699999999999999},"min":{"1min":0.54700000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.1850000000000001},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.65300000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89500000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.627},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.53700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97099999999999997,"5min":1.0149999999999999,"15min":1.2110000000000001},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:31:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.92800000000000005,"5min":0.97999999999999998,"15min":1.0549999999999999},"last":0.751},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.99099999999999999,"15min":1.042},"last":0.88300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1683456,"data_stored":770112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.65900000000000003,"15min":0.63},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.28799999999999998},"max":{"1min":0.97699999999999998,"5min":1.089,"15min":1.089},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68300000000000005,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.33900000000000002},"max":{"1min":0.89700000000000002,"5min":1.075,"15min":1.075},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.499,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.67300000000000004},"min":{"1min":0.46999999999999997,"5min":0.38500000000000001,"15min":0.379},"max":{"1min":0.871,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.67100000000000004},"min":{"1min":0.50800000000000001,"5min":0.46100000000000002,"15min":0.35599999999999998},"max":{"1min":0.90500000000000003,"5min":1.042,"15min":1.042},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69499999999999995,"15min":0.67000000000000004},"min":{"1min":0.54400000000000004,"5min":0.435,"15min":0.37},"max":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73799999999999999,"15min":0.72899999999999998},"min":{"1min":0.60399999999999998,"5min":0.52800000000000002,"15min":0.44},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.627,"15min":0.64600000000000002},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.45000000000000001,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.88600000000000001,"5min":0.90200000000000002,"15min":0.95999999999999996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.68400000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97499999999999998,"5min":1.121,"15min":1.121},"last":0.82199999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.53500000000000003,"5min":0.439,"15min":0.375},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1890000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.72099999999999997,"15min":0.73699999999999999},"min":{"1min":0.58099999999999996,"5min":0.45300000000000001,"15min":0.39000000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.111},"last":0.84499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37368,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2277376,"data_stored":1360480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.441,"15min":0.47099999999999997},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.249},"max":{"1min":0.67900000000000005,"5min":0.70199999999999996,"15min":1.952},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.502,"15min":0.505},"min":{"1min":0.35299999999999998,"5min":0.29199999999999998,"15min":0.254},"max":{"1min":0.68000000000000005,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.503}]},{"osd":2,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46300000000000002,"15min":0.505},"min":{"1min":0.39200000000000002,"5min":0.30299999999999999,"15min":0.29199999999999998},"max":{"1min":0.58199999999999996,"5min":0.71299999999999997,"15min":2.1200000000000001},"last":0.432},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53200000000000003,"15min":0.56999999999999995},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.71499999999999997,"5min":0.89600000000000002,"15min":1.9810000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.59799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68000000000000005,"5min":0.77200000000000002,"15min":2.008},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71599999999999997,"5min":0.93700000000000006,"15min":1.054},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.753,"15min":0.77600000000000002},"min":{"1min":0.57599999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":2.028},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76200000000000001,"15min":0.77700000000000002},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.46400000000000002},"max":{"1min":0.86699999999999999,"5min":1.1120000000000001,"15min":1.9219999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.44},"max":{"1min":0.93000000000000005,"5min":1.0960000000000001,"15min":1.1759999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88,"5min":1.0720000000000001,"15min":2.0899999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.79000000000000004,"15min":0.78900000000000003},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.52600000000000002},"max":{"1min":0.97799999999999998,"5min":1.073,"15min":1.2230000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78400000000000003,"15min":0.82599999999999996},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95099999999999996,"5min":1.0860000000000001,"15min":2.0609999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79000000000000004,"15min":0.80700000000000005},"min":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.50700000000000001},"max":{"1min":0.95099999999999996,"5min":1.0900000000000001,"15min":1.8700000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.55900000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.012,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1699840,"data_stored":771326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.51300000000000001},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76800000000000002,"5min":0.84999999999999998,"15min":11.791},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29299999999999998},"max":{"1min":0.56699999999999995,"5min":0.99199999999999999,"15min":4.657},"last":0.54700000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.60799999999999998},"min":{"1min":0.41799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0349999999999999,"5min":1.2230000000000001,"15min":4.5389999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.51200000000000001,"15min":0.82799999999999996},"min":{"1min":0.36499999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1,"5min":1.276,"15min":13.398999999999999},"last":0.442}]},{"osd":3,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.5,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.65800000000000003,"5min":0.75800000000000001,"15min":1.8089999999999999},"last":0.501},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54300000000000004,"15min":0.56299999999999994},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.59699999999999998,"5min":1.095,"15min":4.4530000000000003},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":1.0669999999999999},"min":{"1min":0.497,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97899999999999998,"5min":1.1359999999999999,"15min":13.667999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72999999999999998,"15min":0.78000000000000003},"min":{"1min":0.54800000000000004,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":0.85099999999999998,"5min":1.1220000000000001,"15min":1.776},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77300000000000002,"15min":1.091},"min":{"1min":0.56999999999999995,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":0.95999999999999996,"5min":1.113,"15min":13.349},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80600000000000005,"15min":0.85299999999999998},"min":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.88400000000000001,"5min":1.302,"15min":4.7649999999999997},"last":0.83899999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.81599999999999995},"min":{"1min":0.54300000000000004,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":0.90400000000000003,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.75},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76200000000000001,"15min":1.0589999999999999},"min":{"1min":0.59199999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89600000000000002,"5min":1.2330000000000001,"15min":11.679},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77400000000000002,"15min":1.095},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91300000000000003,"5min":1.1759999999999999,"15min":13.298},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78500000000000003,"15min":0.86699999999999999},"min":{"1min":0.63200000000000001,"5min":0.53700000000000003,"15min":0.50900000000000001},"max":{"1min":1.073,"5min":1.151,"15min":11.456},"last":0.85799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1683456,"data_stored":769168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44700000000000001,"15min":0.497},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.249},"max":{"1min":0.72899999999999998,"5min":1.605,"15min":2.282},"last":0.434},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.47799999999999998},"min":{"1min":0.32600000000000001,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":1.034,"5min":1.399,"15min":1.4990000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.47099999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.26200000000000001},"max":{"1min":0.93600000000000005,"5min":1.228,"15min":1.228},"last":0.497},{"interface":"front","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81999999999999995,"5min":1.6879999999999999,"15min":2.319},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55800000000000005},"min":{"1min":0.40699999999999997,"5min":0.371,"15min":0.311},"max":{"1min":0.84999999999999998,"5min":1.119,"15min":1.119},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58399999999999996,"15min":0.61899999999999999},"min":{"1min":0.44900000000000001,"5min":0.39300000000000002,"15min":0.38700000000000001},"max":{"1min":0.77700000000000002,"5min":1.5149999999999999,"15min":2.351},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77500000000000002,"15min":0.81000000000000005},"min":{"1min":0.59899999999999998,"5min":0.57099999999999995,"15min":0.46000000000000002},"max":{"1min":1.0069999999999999,"5min":1.6499999999999999,"15min":2.238},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76800000000000002,"15min":0.77200000000000002},"min":{"1min":0.53700000000000003,"5min":0.51200000000000001,"15min":0.46899999999999997},"max":{"1min":1.083,"5min":1.274,"15min":1.518},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61099999999999999,"5min":0.53300000000000003,"15min":0.505},"max":{"1min":1.0660000000000001,"5min":1.341,"15min":1.341},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.80600000000000005,"15min":0.82999999999999996},"min":{"1min":0.60399999999999998,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":1.052,"5min":1.9099999999999999,"15min":2.1909999999999998},"last":0.747}]},{"osd":6,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57699999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.488},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.63500000000000001,"5min":0.54900000000000004,"15min":0.48199999999999998},"max":{"1min":0.98899999999999999,"5min":1.448,"15min":1.448},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:31:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80300000000000005,"15min":0.79000000000000004},"min":{"1min":0.60699999999999998,"5min":0.57599999999999996,"15min":0.51400000000000001},"max":{"1min":1.1259999999999999,"5min":1.222,"15min":1.29},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.58999999999999997,"5min":0.57599999999999996,"15min":0.54400000000000004},"max":{"1min":0.91300000000000003,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.76100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40888,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330952,"statfs":{"total":96636764160,"available":96594894848,"internally_reserved":0,"allocated":1687552,"data_stored":770658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.248},"max":{"1min":0.59199999999999997,"5min":0.73199999999999998,"15min":0.73999999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48199999999999998,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.29899999999999999,"15min":0.27200000000000002},"max":{"1min":0.67500000000000004,"5min":0.73199999999999998,"15min":0.73899999999999999},"last":0.435}]},{"osd":1,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46500000000000002,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28299999999999997},"max":{"1min":0.60399999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.501},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.52600000000000002},"min":{"1min":0.39500000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.94299999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90000000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.56100000000000005,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":1.0149999999999999,"5min":1.149,"15min":1.2250000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.59999999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93400000000000005,"5min":1.113,"15min":1.1539999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76400000000000001,"15min":0.753},"min":{"1min":0.57099999999999995,"5min":0.51000000000000001,"15min":0.496},"max":{"1min":1.0509999999999999,"5min":1.1919999999999999,"15min":1.411},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78600000000000003,"15min":0.77800000000000002},"min":{"1min":0.58499999999999996,"5min":0.53700000000000003,"15min":0.48599999999999999},"max":{"1min":1.05,"5min":1.175,"15min":1.5469999999999999},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.51800000000000002},"max":{"1min":1.0149999999999999,"5min":1.1259999999999999,"15min":1.1559999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78200000000000003,"15min":0.77100000000000002},"min":{"1min":0.622,"5min":0.52500000000000002,"15min":0.52100000000000002},"max":{"1min":0.996,"5min":1.02,"15min":1.1240000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78600000000000003,"15min":0.79100000000000004},"min":{"1min":0.61499999999999999,"5min":0.497,"15min":0.497},"max":{"1min":0.97899999999999998,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.47199999999999998},"max":{"1min":1.0800000000000001,"5min":1.153,"15min":1.1950000000000001},"last":0.879}]}]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36808,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335032,"statfs":{"total":96636764160,"available":96599072768,"internally_reserved":0,"allocated":1703936,"data_stored":778259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66600000000000004,"15min":0.623},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.311},"max":{"1min":0.91200000000000003,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.502},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67300000000000004,"15min":0.64500000000000002},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.36199999999999999},"max":{"1min":0.96799999999999997,"5min":0.997,"15min":0.997},"last":0.71099999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.68700000000000006},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.35099999999999998},"max":{"1min":0.98899999999999999,"5min":1.0489999999999999,"15min":1.298},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72999999999999998,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.438},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.3109999999999999},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.42799999999999999,"15min":0.371},"max":{"1min":0.94499999999999995,"5min":1.0169999999999999,"15min":1.1200000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73799999999999999,"15min":0.71899999999999997},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1839999999999999},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69699999999999995,"15min":0.86499999999999999},"min":{"1min":0.53400000000000003,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":1.02,"5min":1.234,"15min":10.657999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.86299999999999999},"min":{"1min":0.51000000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":8.75},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.65900000000000003,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.32200000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51200000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98199999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95799999999999996,"5min":1.0189999999999999,"15min":1.173},"last":1.012},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.71299999999999997,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.45800000000000002,"15min":0.432},"max":{"1min":1.006,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.754,"15min":0.76600000000000001},"min":{"1min":0.64400000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.0289999999999999,"5min":1.1819999999999999,"15min":1.212},"last":1.0389999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36796,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335044,"statfs":{"total":96636764160,"available":96599085056,"internally_reserved":0,"allocated":1691648,"data_stored":776695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65200000000000002,"15min":0.626},"min":{"1min":0.48399999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.81100000000000005,"5min":1.042,"15min":1.042},"last":0.747},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67000000000000004,"15min":0.64300000000000002},"min":{"1min":0.48199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91700000000000004,"5min":0.94099999999999995,"15min":0.998},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.88700000000000001,"5min":0.89600000000000002,"15min":0.97999999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.52900000000000003,"5min":0.40100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":8.9930000000000003},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67500000000000004,"15min":0.64600000000000002},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.96199999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.68400000000000005},"min":{"1min":0.55300000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71999999999999997,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96599999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.59899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94699999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72199999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.46200000000000002,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.93500000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68999999999999995,"15min":0.68100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94099999999999995,"5min":1.012,"15min":1.012},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68999999999999995,"15min":0.67300000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":0.86799999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:31:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73999999999999999,"15min":0.72799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.41199999999999998},"max":{"1min":0.94999999999999996,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.93200000000000005,"5min":1.034,"15min":1.0669999999999999},"last":0.86299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:31:34.825 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:31:34.826 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-13T22:31:35.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.086+0000 7f295d7ff700 1 Processor -- start 2023-07-13T22:31:35.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.086+0000 7f295d7ff700 1 -- start start 2023-07-13T22:31:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.087+0000 7f295d7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f295815c320 0x7f29581609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.087+0000 7f295d7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 0x7f29580b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.087+0000 7f295d7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2958069ed0 con 0x7f29580b1280 2023-07-13T22:31:35.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.087+0000 7f295d7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29580b1c40 con 0x7f295815c320 2023-07-13T22:31:35.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.087+0000 7f295d7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29580cd7b0 con 0x7f295806b9a0 2023-07-13T22:31:35.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.087+0000 7f2956ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f295815c320 0x7f29581609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.087+0000 7f2956ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f295815c320 0x7f29581609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45826/0 (socket says 172.21.15.60:45826) 2023-07-13T22:31:35.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.087+0000 7f2956ffd700 1 -- 172.21.15.60:0/2473562965 learned_addr learned my addr 172.21.15.60:0/2473562965 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:35.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f29577fe700 1 --2- 172.21.15.60:0/2473562965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 0x7f29580b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f2956ffd700 1 -- 172.21.15.60:0/2473562965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 msgr2=0x7f29580b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:35.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f2956ffd700 1 --2- 172.21.15.60:0/2473562965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 0x7f29580b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f2956ffd700 1 -- 172.21.15.60:0/2473562965 >> v1:172.21.15.60:6789/0 conn(0x7f29580b1280 legacy=0x7f295806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f2956ffd700 1 -- 172.21.15.60:0/2473562965 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29580cda90 con 0x7f295815c320 2023-07-13T22:31:35.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f29567fc700 1 -- 172.21.15.60:0/2473562965 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931384873 0 0) 0x7f2958069ed0 con 0x7f29580b1280 2023-07-13T22:31:35.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f2956ffd700 1 --2- 172.21.15.60:0/2473562965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 0x7f29581609b0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f29480098f0 tx=0x7f2948009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c00c05d9aa823f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f29567fc700 1 -- 172.21.15.60:0/2473562965 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800b8a0 con 0x7f295815c320 2023-07-13T22:31:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f29567fc700 1 -- 172.21.15.60:0/2473562965 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294800ba00 con 0x7f295815c320 2023-07-13T22:31:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.088+0000 7f29567fc700 1 -- 172.21.15.60:0/2473562965 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800bd90 con 0x7f295815c320 2023-07-13T22:31:35.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.089+0000 7f295d7ff700 1 -- 172.21.15.60:0/2473562965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 msgr2=0x7f29581609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.089+0000 7f295d7ff700 1 --2- 172.21.15.60:0/2473562965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 0x7f29581609b0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f29480098f0 tx=0x7f2948009c00 comp rx=0 tx=0).stop 2023-07-13T22:31:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.089+0000 7f295d7ff700 1 -- 172.21.15.60:0/2473562965 shutdown_connections 2023-07-13T22:31:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.089+0000 7f295d7ff700 1 --2- 172.21.15.60:0/2473562965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 0x7f29580b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.089+0000 7f295d7ff700 1 --2- 172.21.15.60:0/2473562965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 0x7f29581609b0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.089+0000 7f295d7ff700 1 -- 172.21.15.60:0/2473562965 >> 172.21.15.60:0/2473562965 conn(0x7f29580cc280 msgr2=0x7f29580cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.089+0000 7f295d7ff700 1 -- 172.21.15.60:0/2473562965 shutdown_connections 2023-07-13T22:31:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.089+0000 7f295d7ff700 1 -- 172.21.15.60:0/2473562965 wait complete. 2023-07-13T22:31:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.091+0000 7f295d7ff700 1 Processor -- start 2023-07-13T22:31:35.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.091+0000 7f295d7ff700 1 -- start start 2023-07-13T22:31:35.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f295d7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 0x7f29580c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f295d7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29580b1280 0x7f2958156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f295d7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 0x7f29580ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f295d7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29580c8460 con 0x7f295815c320 2023-07-13T22:31:35.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f295d7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29580c7850 con 0x7f29580b1280 2023-07-13T22:31:35.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f295d7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29580c7b30 con 0x7f295806b9a0 2023-07-13T22:31:35.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f29577fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 0x7f29580ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f29577fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 0x7f29580ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45834/0 (socket says 172.21.15.60:45834) 2023-07-13T22:31:35.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f29577fe700 1 -- 172.21.15.60:0/3350186571 learned_addr learned my addr 172.21.15.60:0/3350186571 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f2956ffd700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 0x7f29580c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.092+0000 7f294ffff700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29580b1280 0x7f2958156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f29577fe700 1 -- 172.21.15.60:0/3350186571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 msgr2=0x7f29580c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f29577fe700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 0x7f29580c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f29577fe700 1 -- 172.21.15.60:0/3350186571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29580b1280 msgr2=0x7f2958156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f29577fe700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29580b1280 0x7f2958156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f29577fe700 1 -- 172.21.15.60:0/3350186571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948009550 con 0x7f295815c320 2023-07-13T22:31:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f294ffff700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29580b1280 0x7f2958156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f29577fe700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 0x7f29580ca160 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f2950007e40 tx=0x7f295000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f294f7fe700 1 -- 172.21.15.60:0/3350186571 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000ede0 con 0x7f295815c320 2023-07-13T22:31:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f294f7fe700 1 -- 172.21.15.60:0/3350186571 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2950014420 con 0x7f295815c320 2023-07-13T22:31:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f294f7fe700 1 -- 172.21.15.60:0/3350186571 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29500147d0 con 0x7f295815c320 2023-07-13T22:31:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29580c7e10 con 0x7f295815c320 2023-07-13T22:31:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.093+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29580c6080 con 0x7f295815c320 2023-07-13T22:31:35.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.094+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29580c85d0 con 0x7f295815c320 2023-07-13T22:31:35.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.095+0000 7f294f7fe700 1 -- 172.21.15.60:0/3350186571 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2950014930 con 0x7f295815c320 2023-07-13T22:31:35.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.097+0000 7f294f7fe700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.097+0000 7f294f7fe700 1 -- 172.21.15.60:0/3350186571 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2950089d30 con 0x7f295815c320 2023-07-13T22:31:35.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.097+0000 7f2956ffd700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.098+0000 7f2956ffd700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2948014040 tx=0x7f2948008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.101+0000 7f294f7fe700 1 -- 172.21.15.60:0/3350186571 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2950058740 con 0x7f295815c320 2023-07-13T22:31:35.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.387+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f295804e210 con 0x7f295815c320 2023-07-13T22:31:35.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.392+0000 7f294f7fe700 1 -- 172.21.15.60:0/3350186571 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f29500582d0 con 0x7f295815c320 2023-07-13T22:31:35.375 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:31:35.375 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCOeLBkQX7fMRAAQ2dtZgw5ZR7DGakPOpxu9w== 2023-07-13T22:31:35.375 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-07-13T22:31:35.375 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-07-13T22:31:35.375 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-13T22:31:35.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.394+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 msgr2=0x7f293806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.394+0000 7f295d7ff700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2948014040 tx=0x7f2948008030 comp rx=0 tx=0).stop 2023-07-13T22:31:35.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.394+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 msgr2=0x7f29580ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.394+0000 7f295d7ff700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 0x7f29580ca160 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f2950007e40 tx=0x7f295000e3c0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.395+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 shutdown_connections 2023-07-13T22:31:35.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.395+0000 7f295d7ff700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f293806ace0 0x7f293806d190 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.395+0000 7f295d7ff700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f295815c320 0x7f29580ca160 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.395+0000 7f295d7ff700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f295806b9a0 0x7f29580c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.395+0000 7f295d7ff700 1 --2- 172.21.15.60:0/3350186571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29580b1280 0x7f2958156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.395+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 >> 172.21.15.60:0/3350186571 conn(0x7f29580cc280 msgr2=0x7f295806a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:35.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.395+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 shutdown_connections 2023-07-13T22:31:35.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.395+0000 7f295d7ff700 1 -- 172.21.15.60:0/3350186571 wait complete. 2023-07-13T22:31:35.379 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-07-13T22:31:35.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:35.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.617+0000 7effb01ff700 1 Processor -- start 2023-07-13T22:31:35.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.617+0000 7effb01ff700 1 -- start start 2023-07-13T22:31:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.618+0000 7effb01ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effa80b8570 0x7effa80ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.618+0000 7effb01ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7effa80d35c0 0x7effa80d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.618+0000 7effb01ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa80d09d0 con 0x7effa80cf3f0 2023-07-13T22:31:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.618+0000 7effb01ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa80d3b30 con 0x7effa80b8570 2023-07-13T22:31:35.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.618+0000 7effb01ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa80cda10 con 0x7effa80d35c0 2023-07-13T22:31:35.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effadf9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effa80b8570 0x7effa80ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effae79c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7effa80cf3f0 0x7effa815fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54084/0 (socket says 172.21.15.60:54084) 2023-07-13T22:31:35.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effae79c700 1 -- 172.21.15.60:0/1871208125 learned_addr learned my addr 172.21.15.60:0/1871208125 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effad79a700 1 --2- 172.21.15.60:0/1871208125 >> v2:172.21.15.105:3300/0 conn(0x7effa80d35c0 0x7effa80d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effadf9b700 1 -- 172.21.15.60:0/1871208125 >> v2:172.21.15.105:3300/0 conn(0x7effa80d35c0 msgr2=0x7effa80d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:35.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effadf9b700 1 --2- 172.21.15.60:0/1871208125 >> v2:172.21.15.105:3300/0 conn(0x7effa80d35c0 0x7effa80d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effadf9b700 1 -- 172.21.15.60:0/1871208125 >> v1:172.21.15.60:6789/0 conn(0x7effa80cf3f0 legacy=0x7effa815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effadf9b700 1 -- 172.21.15.60:0/1871208125 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80cd450 con 0x7effa80b8570 2023-07-13T22:31:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effacf99700 1 -- 172.21.15.60:0/1871208125 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712003959 0 0) 0x7effa80d09d0 con 0x7effa80cf3f0 2023-07-13T22:31:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.619+0000 7effadf9b700 1 --2- 172.21.15.60:0/1871208125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80b8570 0x7effa80ceeb0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7effa80d07e0 tx=0x7effa4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48579844bc3e621a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effacf99700 1 -- 172.21.15.60:0/1871208125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4005e30 con 0x7effa80b8570 2023-07-13T22:31:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effacf99700 1 -- 172.21.15.60:0/1871208125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa4006030 con 0x7effa80b8570 2023-07-13T22:31:35.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effacf99700 1 -- 172.21.15.60:0/1871208125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa40063c0 con 0x7effa80b8570 2023-07-13T22:31:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effb01ff700 1 -- 172.21.15.60:0/1871208125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80b8570 msgr2=0x7effa80ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effb01ff700 1 --2- 172.21.15.60:0/1871208125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80b8570 0x7effa80ceeb0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7effa80d07e0 tx=0x7effa4009b00 comp rx=0 tx=0).stop 2023-07-13T22:31:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effb01ff700 1 -- 172.21.15.60:0/1871208125 shutdown_connections 2023-07-13T22:31:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effb01ff700 1 --2- 172.21.15.60:0/1871208125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80b8570 0x7effa80ceeb0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effb01ff700 1 --2- 172.21.15.60:0/1871208125 >> v2:172.21.15.105:3300/0 conn(0x7effa80d35c0 0x7effa80d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effb01ff700 1 -- 172.21.15.60:0/1871208125 >> 172.21.15.60:0/1871208125 conn(0x7effa8071600 msgr2=0x7effa80719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effb01ff700 1 -- 172.21.15.60:0/1871208125 shutdown_connections 2023-07-13T22:31:35.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.620+0000 7effb01ff700 1 -- 172.21.15.60:0/1871208125 wait complete. 2023-07-13T22:31:35.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.623+0000 7effb01ff700 1 Processor -- start 2023-07-13T22:31:35.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.623+0000 7effb01ff700 1 -- start start 2023-07-13T22:31:35.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.624+0000 7effb01ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa80b8570 0x7effa8159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.624+0000 7effb01ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa80cf3f0 0x7effa8159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.624+0000 7effb01ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80d35c0 0x7effa80c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.624+0000 7effb01ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa81f1c20 con 0x7effa80d35c0 2023-07-13T22:31:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.624+0000 7effb01ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa80ca2f0 con 0x7effa80b8570 2023-07-13T22:31:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.624+0000 7effb01ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa80ca5d0 con 0x7effa80cf3f0 2023-07-13T22:31:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.624+0000 7effae79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80d35c0 0x7effa80c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effae79c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80d35c0 0x7effa80c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45854/0 (socket says 172.21.15.60:45854) 2023-07-13T22:31:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effae79c700 1 -- 172.21.15.60:0/1301620988 learned_addr learned my addr 172.21.15.60:0/1301620988 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effadf9b700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa80b8570 0x7effa8159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effad79a700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa80cf3f0 0x7effa8159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effae79c700 1 -- 172.21.15.60:0/1301620988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa80cf3f0 msgr2=0x7effa8159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effae79c700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa80cf3f0 0x7effa8159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effae79c700 1 -- 172.21.15.60:0/1301620988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa80b8570 msgr2=0x7effa8159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effae79c700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa80b8570 0x7effa8159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effae79c700 1 -- 172.21.15.60:0/1301620988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa4009430 con 0x7effa80d35c0 2023-07-13T22:31:35.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effae79c700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80d35c0 0x7effa80c7f30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7effa000cca0 tx=0x7effa000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:35.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7eff9e7fc700 1 -- 172.21.15.60:0/1301620988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000ab70 con 0x7effa80d35c0 2023-07-13T22:31:35.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7eff9e7fc700 1 -- 172.21.15.60:0/1301620988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa000acd0 con 0x7effa80d35c0 2023-07-13T22:31:35.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7eff9e7fc700 1 -- 172.21.15.60:0/1301620988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0007310 con 0x7effa80d35c0 2023-07-13T22:31:35.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.625+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa80ca8b0 con 0x7effa80d35c0 2023-07-13T22:31:35.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.626+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa80cae70 con 0x7effa80d35c0 2023-07-13T22:31:35.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.627+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa80cb090 con 0x7effa80d35c0 2023-07-13T22:31:35.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.628+0000 7eff9e7fc700 1 -- 172.21.15.60:0/1301620988 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effa0007470 con 0x7effa80d35c0 2023-07-13T22:31:35.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.629+0000 7eff9e7fc700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:35.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.629+0000 7eff9e7fc700 1 -- 172.21.15.60:0/1301620988 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effa0088fe0 con 0x7effa80d35c0 2023-07-13T22:31:35.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.629+0000 7effadf9b700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:35.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.629+0000 7effadf9b700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7effa4000c00 tx=0x7effa4008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:35.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.633+0000 7eff9e7fc700 1 -- 172.21.15.60:0/1301620988 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7effa00540c0 con 0x7effa80d35c0 2023-07-13T22:31:35.863 INFO:tasks.ceph.mds.c.smithi060.stderr:2023-07-13T22:31:35.880+0000 7f2bfc1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:31:35.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.884+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effa804e280 con 0x7effa80d35c0 2023-07-13T22:31:35.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.885+0000 7eff9e7fc700 1 -- 172.21.15.60:0/1301620988 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4719 (secure 0 0 0) 0x7effa005b280 con 0x7effa80d35c0 2023-07-13T22:31:35.868 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:35.868 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3652157896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3652157896},{"type":"v1","addr":"172.21.15.105:6834","nonce":3652157896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1878256861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1878256861},{"type":"v1","addr":"172.21.15.105:6835","nonce":1878256861}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92},{"gid":8728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2695331242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2695331242},{"type":"v1","addr":"172.21.15.60:6836","nonce":2695331242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:31:29.408608+0000","modified":"2023-07-13T22:31:32.919029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9423":{"gid":9423,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4061691799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4061691799},{"type":"v1","addr":"172.21.15.60:6837","nonce":4061691799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-13T22:31:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.887+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 msgr2=0x7eff8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.887+0000 7effb01ff700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7effa4000c00 tx=0x7effa4008070 comp rx=0 tx=0).stop 2023-07-13T22:31:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.887+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80d35c0 msgr2=0x7effa80c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:35.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.887+0000 7effb01ff700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80d35c0 0x7effa80c7f30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7effa000cca0 tx=0x7effa000a060 comp rx=0 tx=0).stop 2023-07-13T22:31:35.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.888+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 shutdown_connections 2023-07-13T22:31:35.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.888+0000 7effb01ff700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff8c06ace0 0x7eff8c06d190 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.888+0000 7effb01ff700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa80d35c0 0x7effa80c7f30 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.888+0000 7effb01ff700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa80b8570 0x7effa8159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.888+0000 7effb01ff700 1 --2- 172.21.15.60:0/1301620988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa80cf3f0 0x7effa8159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:35.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.888+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 >> 172.21.15.60:0/1301620988 conn(0x7effa8071600 msgr2=0x7effa815fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:35.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.888+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 shutdown_connections 2023-07-13T22:31:35.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:35.888+0000 7effb01ff700 1 -- 172.21.15.60:0/1301620988 wait complete. 2023-07-13T22:31:35.872 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 97 2023-07-13T22:31:36.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-13T22:31:29.408608+0000', 'modified': '2023-07-13T22:31:32.919029+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9423}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9423': {'gid': 9423, 'name': 'c', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6837/4061691799', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6835', 'nonce': 4061691799}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 4061691799}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:31:36.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-13T22:31:36.293 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-13T22:31:36.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.520+0000 7f26c07ff700 1 Processor -- start 2023-07-13T22:31:36.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.520+0000 7f26c07ff700 1 -- start start 2023-07-13T22:31:36.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26c07ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26b8154950 0x7f26b8070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26c07ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b815a820 0x7f26b815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26c07ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26b80c8d40 0x7f26b80c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26c07ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b8070270 con 0x7f26b80c8d40 2023-07-13T22:31:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26c07ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b80c96c0 con 0x7f26b8154950 2023-07-13T22:31:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26c07ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b815b2e0 con 0x7f26b815a820 2023-07-13T22:31:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26bed9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26b80c8d40 0x7f26b80c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26bed9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26b80c8d40 0x7f26b80c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45862/0 (socket says 172.21.15.60:45862) 2023-07-13T22:31:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26bed9c700 1 -- 172.21.15.60:0/890896584 learned_addr learned my addr 172.21.15.60:0/890896584 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26be59b700 1 --2- 172.21.15.60:0/890896584 >> v2:172.21.15.105:3300/0 conn(0x7f26b8154950 0x7f26b8070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.522+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/890896584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b815a820 0x7f26b815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bed9c700 1 -- 172.21.15.60:0/890896584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b815a820 msgr2=0x7f26b815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bed9c700 1 --2- 172.21.15.60:0/890896584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b815a820 0x7f26b815ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bed9c700 1 -- 172.21.15.60:0/890896584 >> v2:172.21.15.105:3300/0 conn(0x7f26b8154950 msgr2=0x7f26b8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bed9c700 1 --2- 172.21.15.60:0/890896584 >> v2:172.21.15.105:3300/0 conn(0x7f26b8154950 0x7f26b8070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bed9c700 1 -- 172.21.15.60:0/890896584 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b815b5c0 con 0x7f26b80c8d40 2023-07-13T22:31:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/890896584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b815a820 0x7f26b815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bed9c700 1 --2- 172.21.15.60:0/890896584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 0x7f26b80c9180 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f26b80ce6e0 tx=0x7f26b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d150fe82f45b77 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bd599700 1 -- 172.21.15.60:0/890896584 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b000e070 con 0x7f26b80c8d40 2023-07-13T22:31:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bd599700 1 -- 172.21.15.60:0/890896584 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b000b910 con 0x7f26b80c8d40 2023-07-13T22:31:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.523+0000 7f26bd599700 1 -- 172.21.15.60:0/890896584 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b000bcc0 con 0x7f26b80c8d40 2023-07-13T22:31:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.524+0000 7f26c07ff700 1 -- 172.21.15.60:0/890896584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 msgr2=0x7f26b80c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.524+0000 7f26c07ff700 1 --2- 172.21.15.60:0/890896584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 0x7f26b80c9180 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f26b80ce6e0 tx=0x7f26b000b330 comp rx=0 tx=0).stop 2023-07-13T22:31:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.524+0000 7f26c07ff700 1 -- 172.21.15.60:0/890896584 shutdown_connections 2023-07-13T22:31:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.524+0000 7f26c07ff700 1 --2- 172.21.15.60:0/890896584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 0x7f26b80c9180 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.524+0000 7f26c07ff700 1 --2- 172.21.15.60:0/890896584 >> v2:172.21.15.105:3300/0 conn(0x7f26b8154950 0x7f26b8070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.524+0000 7f26c07ff700 1 --2- 172.21.15.60:0/890896584 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b815a820 0x7f26b815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.524+0000 7f26c07ff700 1 -- 172.21.15.60:0/890896584 >> 172.21.15.60:0/890896584 conn(0x7f26b80cf410 msgr2=0x7f26b80cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.524+0000 7f26c07ff700 1 -- 172.21.15.60:0/890896584 shutdown_connections 2023-07-13T22:31:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.524+0000 7f26c07ff700 1 -- 172.21.15.60:0/890896584 wait complete. 2023-07-13T22:31:36.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.526+0000 7f26c07ff700 1 Processor -- start 2023-07-13T22:31:36.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.526+0000 7f26c07ff700 1 -- start start 2023-07-13T22:31:36.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26c07ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 0x7f26b80c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26c07ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b8154950 0x7f26b81440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26c07ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b815a820 0x7f26b8159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26c07ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8144610 con 0x7f26b80c8d40 2023-07-13T22:31:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26c07ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80c73b0 con 0x7f26b815a820 2023-07-13T22:31:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26c07ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80c7690 con 0x7f26b8154950 2023-07-13T22:31:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26be59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 0x7f26b80c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26be59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 0x7f26b80c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45866/0 (socket says 172.21.15.60:45866) 2023-07-13T22:31:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26be59b700 1 -- 172.21.15.60:0/3950771329 learned_addr learned my addr 172.21.15.60:0/3950771329 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b8154950 0x7f26b81440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26bed9c700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b815a820 0x7f26b8159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26be59b700 1 -- 172.21.15.60:0/3950771329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b8154950 msgr2=0x7f26b81440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26be59b700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b8154950 0x7f26b81440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26be59b700 1 -- 172.21.15.60:0/3950771329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b815a820 msgr2=0x7f26b8159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26be59b700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b815a820 0x7f26b8159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.527+0000 7f26be59b700 1 -- 172.21.15.60:0/3950771329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b0009b30 con 0x7f26b80c8d40 2023-07-13T22:31:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.528+0000 7f26be59b700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 0x7f26b80c6da0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f26b400e930 tx=0x7f26b400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.528+0000 7f26aeffd700 1 -- 172.21.15.60:0/3950771329 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4009690 con 0x7f26b80c8d40 2023-07-13T22:31:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.528+0000 7f26aeffd700 1 -- 172.21.15.60:0/3950771329 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b400c850 con 0x7f26b80c8d40 2023-07-13T22:31:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.528+0000 7f26aeffd700 1 -- 172.21.15.60:0/3950771329 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b400cbe0 con 0x7f26b80c8d40 2023-07-13T22:31:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.528+0000 7f26c07ff700 1 -- 172.21.15.60:0/3950771329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b80c7970 con 0x7f26b80c8d40 2023-07-13T22:31:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.528+0000 7f26c07ff700 1 -- 172.21.15.60:0/3950771329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b80c7f30 con 0x7f26b80c8d40 2023-07-13T22:31:36.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.530+0000 7f26aeffd700 1 -- 172.21.15.60:0/3950771329 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26b400cd40 con 0x7f26b80c8d40 2023-07-13T22:31:36.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.531+0000 7f26aeffd700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f269c06ace0 0x7f269c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.531+0000 7f26aeffd700 1 -- 172.21.15.60:0/3950771329 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f26b4089640 con 0x7f26b80c8d40 2023-07-13T22:31:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.531+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f269c06ace0 0x7f269c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.532+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f269c06ace0 0x7f269c06d190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f26b814c560 tx=0x7f26a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.533+0000 7f26d4116b80 1 Processor -- start 2023-07-13T22:31:36.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.533+0000 7f26d4116b80 1 -- start start 2023-07-13T22:31:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.534+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539ec75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.534+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ec7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.534+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539eb2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.534+0000 7f26d4116b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557539e11ea0 con 0x557539e97c50 2023-07-13T22:31:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.534+0000 7f26d4116b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x557539d85600 con 0x557539e97880 2023-07-13T22:31:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.534+0000 7f26d4116b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x557539df4910 con 0x557539e99dc0 2023-07-13T22:31:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.534+0000 7f26bed9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539eb2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.534+0000 7f26bed9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539eb2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45882/0 (socket says 172.21.15.60:45882) 2023-07-13T22:31:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.534+0000 7f26bed9c700 1 -- 172.21.15.60:0/3378594062 learned_addr learned my addr 172.21.15.60:0/3378594062 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f26be59b700 1 --2- 172.21.15.60:0/3378594062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539ec75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/3378594062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ec7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f26bed9c700 1 -- 172.21.15.60:0/3378594062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 msgr2=0x557539ec7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f26bed9c700 1 --2- 172.21.15.60:0/3378594062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ec7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f26bed9c700 1 -- 172.21.15.60:0/3378594062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 msgr2=0x557539ec75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f26bed9c700 1 --2- 172.21.15.60:0/3378594062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539ec75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f26bed9c700 1 -- 172.21.15.60:0/3378594062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557539be1790 con 0x557539e97c50 2023-07-13T22:31:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f26bed9c700 1 --2- 172.21.15.60:0/3378594062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539eb2120 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f26b000b680 tx=0x7f26b0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd664f43ba25d730 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f269bfff700 1 -- 172.21.15.60:0/3378594062 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b000bd80 con 0x557539e97c50 2023-07-13T22:31:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f269bfff700 1 -- 172.21.15.60:0/3378594062 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b00099b0 con 0x557539e97c50 2023-07-13T22:31:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.535+0000 7f269bfff700 1 -- 172.21.15.60:0/3378594062 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0003c60 con 0x557539e97c50 2023-07-13T22:31:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.536+0000 7f26d4116b80 1 -- 172.21.15.60:0/3378594062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 msgr2=0x557539eb2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.536+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3378594062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539eb2120 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f26b000b680 tx=0x7f26b0010000 comp rx=0 tx=0).stop 2023-07-13T22:31:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.536+0000 7f26d4116b80 1 -- 172.21.15.60:0/3378594062 shutdown_connections 2023-07-13T22:31:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.536+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3378594062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539eb2120 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.536+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3378594062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539ec75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.536+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3378594062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ec7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.536+0000 7f26d4116b80 1 -- 172.21.15.60:0/3378594062 >> 172.21.15.60:0/3378594062 conn(0x557539e923b0 msgr2=0x557539e9d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.536+0000 7f26d4116b80 1 -- 172.21.15.60:0/3378594062 shutdown_connections 2023-07-13T22:31:36.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.536+0000 7f26d4116b80 1 -- 172.21.15.60:0/3378594062 wait complete. 2023-07-13T22:31:36.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.537+0000 7f26d4116b80 1 -- start start 2023-07-13T22:31:36.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.537+0000 7f26d4116b80 1 Processor -- start 2023-07-13T22:31:36.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539ec4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ef16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ef1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26d4116b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557539dedbe0 con 0x557539e97c50 2023-07-13T22:31:36.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26d4116b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x557539dab5b0 con 0x557539e97880 2023-07-13T22:31:36.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26d4116b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x557539d91910 con 0x557539e99dc0 2023-07-13T22:31:36.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26d4116b80 10 client.0 _ll_register_callbacks cb 0x557539da9b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:31:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26bdd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ef16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26bdd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ef16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45892/0 (socket says 172.21.15.60:45892) 2023-07-13T22:31:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.539+0000 7f26bdd9a700 1 -- 172.21.15.60:0/3209411761 learned_addr learned my addr 172.21.15.60:0/3209411761 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26be59b700 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539ec4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26bed9c700 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ef1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26bdd9a700 1 -- 172.21.15.60:0/3209411761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 msgr2=0x557539ef1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ef1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26bdd9a700 1 -- 172.21.15.60:0/3209411761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 msgr2=0x557539ec4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539ec4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26bdd9a700 1 -- 172.21.15.60:0/3209411761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b815b5c0 con 0x557539e97c50 2023-07-13T22:31:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ef16d0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009ef0 tx=0x7f26a80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6248d15bac686eb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26997fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:31:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26997fa700 1 -- 172.21.15.60:0/3209411761 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8015e60 con 0x557539e97c50 2023-07-13T22:31:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26997fa700 1 -- 172.21.15.60:0/3209411761 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f26a8022470 con 0x557539e97c50 2023-07-13T22:31:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.540+0000 7f26d4116b80 1 -- 172.21.15.60:0/3209411761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557539e6c170 con 0x557539e97c50 2023-07-13T22:31:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26997fa700 1 -- 172.21.15.60:0/3209411761 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26a8022630 con 0x557539e97c50 2023-07-13T22:31:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26d4116b80 10 client.9570 fetch_fsmap learned FSMap version 97 2023-07-13T22:31:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26d4116b80 1 -- 172.21.15.60:0/3209411761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x557539cce0a0 con 0x557539e97c50 2023-07-13T22:31:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26997fa700 1 -- 172.21.15.60:0/3209411761 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557539e6c170 con 0x557539e97c50 2023-07-13T22:31:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26997fa700 1 -- 172.21.15.60:0/3209411761 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 97) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f26a8022790 con 0x557539e97c50 2023-07-13T22:31:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26d4116b80 10 client.9570 fetch_fsmap finished waiting for FSMap version 97 2023-07-13T22:31:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26d4116b80 10 client.9570 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-13T22:31:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26d4116b80 20 client.9570 populate_metadata read hostname 'smithi060' 2023-07-13T22:31:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] conn(0x557539ef2af0 0x557539ef2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26d4116b80 4 client.9570 mds_command: new command op to 9423 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:31:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26d4116b80 1 -- 172.21.15.60:0/3209411761 --> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557539de6d20 con 0x557539ef2af0 2023-07-13T22:31:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.541+0000 7f26be59b700 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] conn(0x557539ef2af0 0x557539ef2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.542+0000 7f26be59b700 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] conn(0x557539ef2af0 0x557539ef2ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7767e5d0aeddd71 server_cookie=8195e0552720a809 in_seq=0 out_seq=0 2023-07-13T22:31:36.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.542+0000 7f26997fa700 10 client.9570 ms_handle_connect on v2:172.21.15.60:6835/4061691799 2023-07-13T22:31:36.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26997fa700 1 -- 172.21.15.60:0/3209411761 <== mds.0 v2:172.21.15.60:6835/4061691799 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557539de6d20 con 0x557539ef2af0 2023-07-13T22:31:36.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26997fa700 10 client.9570 handle_command_reply: tid=0 2023-07-13T22:31:36.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 client.9570 shutdown 2023-07-13T22:31:36.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 -- 172.21.15.60:0/3209411761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 msgr2=0x557539ef16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ef16d0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009ef0 tx=0x7f26a80047c0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 -- 172.21.15.60:0/3209411761 shutdown_connections 2023-07-13T22:31:36.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] conn(0x557539ef2af0 0x557539ef2ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ef1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539ec4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 --2- 172.21.15.60:0/3209411761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ef16d0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 -- 172.21.15.60:0/3209411761 >> 172.21.15.60:0/3209411761 conn(0x557539e923b0 msgr2=0x557539ee4910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:36.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26997fa700 0 client.9570 ms_handle_reset on v2:172.21.15.60:6835/4061691799 2023-07-13T22:31:36.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 -- 172.21.15.60:0/3209411761 shutdown_connections 2023-07-13T22:31:36.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.543+0000 7f26d4116b80 1 -- 172.21.15.60:0/3209411761 wait complete. 2023-07-13T22:31:36.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.544+0000 7f26d4116b80 20 client.9570 trim_cache size 0 max 16384 2023-07-13T22:31:36.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.562+0000 7f26d4116b80 1 Processor -- start 2023-07-13T22:31:36.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.562+0000 7f26d4116b80 1 -- start start 2023-07-13T22:31:36.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e97880 0x557539e9a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ec4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e99dc0 0x557539f42980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26d4116b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557539be2390 con 0x557539e97c50 2023-07-13T22:31:36.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26d4116b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x557539ec8080 con 0x557539e99dc0 2023-07-13T22:31:36.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26d4116b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x557539eda750 con 0x557539e97880 2023-07-13T22:31:36.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26bdd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ec4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26bdd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ec4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45900/0 (socket says 172.21.15.60:45900) 2023-07-13T22:31:36.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26bdd9a700 1 -- 172.21.15.60:0/2610745986 learned_addr learned my addr 172.21.15.60:0/2610745986 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26be59b700 1 --2- 172.21.15.60:0/2610745986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e97880 0x557539e9a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26bed9c700 1 --2- 172.21.15.60:0/2610745986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e99dc0 0x557539f42980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26bdd9a700 1 -- 172.21.15.60:0/2610745986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e97880 msgr2=0x557539e9a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/2610745986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e97880 0x557539e9a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26bdd9a700 1 -- 172.21.15.60:0/2610745986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e99dc0 msgr2=0x557539f42980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/2610745986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e99dc0 0x557539f42980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.564+0000 7f26bdd9a700 1 -- 172.21.15.60:0/2610745986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557539db3b10 con 0x557539e97c50 2023-07-13T22:31:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/2610745986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ec4480 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f26a80093d0 tx=0x7f26a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889d51f07e6a18b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26acff9700 1 -- 172.21.15.60:0/2610745986 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8015ce0 con 0x557539e97c50 2023-07-13T22:31:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26acff9700 1 -- 172.21.15.60:0/2610745986 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a8015e60 con 0x557539e97c50 2023-07-13T22:31:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26acff9700 1 -- 172.21.15.60:0/2610745986 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a80226b0 con 0x557539e97c50 2023-07-13T22:31:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26d4116b80 1 -- 172.21.15.60:0/2610745986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 msgr2=0x557539ec4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2610745986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ec4480 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f26a80093d0 tx=0x7f26a801a000 comp rx=0 tx=0).stop 2023-07-13T22:31:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26d4116b80 1 -- 172.21.15.60:0/2610745986 shutdown_connections 2023-07-13T22:31:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2610745986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e99dc0 0x557539f42980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2610745986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e97880 0x557539e9a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2610745986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539ec4480 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26d4116b80 1 -- 172.21.15.60:0/2610745986 >> 172.21.15.60:0/2610745986 conn(0x557539e923b0 msgr2=0x557539e9d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:36.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.565+0000 7f26d4116b80 1 -- 172.21.15.60:0/2610745986 shutdown_connections 2023-07-13T22:31:36.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.566+0000 7f26d4116b80 1 -- 172.21.15.60:0/2610745986 wait complete. 2023-07-13T22:31:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.566+0000 7f26d4116b80 1 -- start start 2023-07-13T22:31:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.567+0000 7f26d4116b80 1 Processor -- start 2023-07-13T22:31:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.568+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539f41220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.568+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539efe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.568+0000 7f26d4116b80 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ed9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.568+0000 7f26d4116b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557539f00540 con 0x557539e97c50 2023-07-13T22:31:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.568+0000 7f26d4116b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x557539e9e610 con 0x557539e97880 2023-07-13T22:31:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.568+0000 7f26d4116b80 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x557539ed99d0 con 0x557539e99dc0 2023-07-13T22:31:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.568+0000 7f26d4116b80 10 client.0 _ll_register_callbacks cb 0x7f26a80095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:31:36.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bdd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539efe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bdd9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539efe180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45908/0 (socket says 172.21.15.60:45908) 2023-07-13T22:31:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bdd9a700 1 -- 172.21.15.60:0/2882879117 learned_addr learned my addr 172.21.15.60:0/2882879117 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bed9c700 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ed9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26be59b700 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539f41220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bdd9a700 1 -- 172.21.15.60:0/2882879117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 msgr2=0x557539ed9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ed9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bdd9a700 1 -- 172.21.15.60:0/2882879117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 msgr2=0x557539f41220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539f41220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bdd9a700 1 -- 172.21.15.60:0/2882879117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557539cce0a0 con 0x557539e97c50 2023-07-13T22:31:36.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26be59b700 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539f41220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:36.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26bdd9a700 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539efe180 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009fd0 tx=0x7f26a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5389698f7972f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26997fa700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-07-13T22:31:36.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26997fa700 1 -- 172.21.15.60:0/2882879117 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8016420 con 0x557539e97c50 2023-07-13T22:31:36.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.569+0000 7f26997fa700 1 -- 172.21.15.60:0/2882879117 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f26a8043020 con 0x557539e97c50 2023-07-13T22:31:36.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26d4116b80 1 -- 172.21.15.60:0/2882879117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557539e8f9c0 con 0x557539e97c50 2023-07-13T22:31:36.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26997fa700 1 -- 172.21.15.60:0/2882879117 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26a8005a20 con 0x557539e97c50 2023-07-13T22:31:36.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26d4116b80 10 client.9576 fetch_fsmap learned FSMap version 97 2023-07-13T22:31:36.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26d4116b80 1 -- 172.21.15.60:0/2882879117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x557539c421d0 con 0x557539e97c50 2023-07-13T22:31:36.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26997fa700 1 -- 172.21.15.60:0/2882879117 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557539e8f9c0 con 0x557539e97c50 2023-07-13T22:31:36.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26997fa700 1 -- 172.21.15.60:0/2882879117 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 97) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f26a8005b80 con 0x557539e97c50 2023-07-13T22:31:36.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26d4116b80 10 client.9576 fetch_fsmap finished waiting for FSMap version 97 2023-07-13T22:31:36.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26d4116b80 10 client.9576 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-13T22:31:36.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26d4116b80 20 client.9576 populate_metadata read hostname 'smithi060' 2023-07-13T22:31:36.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] conn(0x557539ed7f70 0x557539ed8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26d4116b80 4 client.9576 mds_command: new command op to 9423 tid=0[{"prefix": "status"}] 2023-07-13T22:31:36.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26d4116b80 1 -- 172.21.15.60:0/2882879117 --> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557539c3cef0 con 0x557539ed7f70 2023-07-13T22:31:36.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.570+0000 7f26be59b700 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] conn(0x557539ed7f70 0x557539ed8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26be59b700 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] conn(0x557539ed7f70 0x557539ed8340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7a9d23b9dae94fe server_cookie=7bf47c87f0675eb4 in_seq=0 out_seq=0 2023-07-13T22:31:36.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26997fa700 10 client.9576 ms_handle_connect on v2:172.21.15.60:6835/4061691799 2023-07-13T22:31:36.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26997fa700 1 -- 172.21.15.60:0/2882879117 <== mds.0 v2:172.21.15.60:6835/4061691799 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x557539c3cef0 con 0x557539ed7f70 2023-07-13T22:31:36.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26997fa700 10 client.9576 handle_command_reply: tid=0 2023-07-13T22:31:36.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26d4116b80 1 client.9576 shutdown 2023-07-13T22:31:36.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26d4116b80 1 -- 172.21.15.60:0/2882879117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 msgr2=0x557539efe180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539efe180 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009fd0 tx=0x7f26a8015000 comp rx=0 tx=0).stop 2023-07-13T22:31:36.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26d4116b80 1 -- 172.21.15.60:0/2882879117 shutdown_connections 2023-07-13T22:31:36.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.60:6835/4061691799,v1:172.21.15.60:6837/4061691799] conn(0x557539ed7f70 0x557539ed8340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x557539e99dc0 0x557539ed9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557539e97880 0x557539f41220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26997fa700 0 client.9576 ms_handle_reset on v2:172.21.15.60:6835/4061691799 2023-07-13T22:31:36.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26d4116b80 1 --2- 172.21.15.60:0/2882879117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557539e97c50 0x557539efe180 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.571+0000 7f26d4116b80 1 -- 172.21.15.60:0/2882879117 >> 172.21.15.60:0/2882879117 conn(0x557539e923b0 msgr2=0x557539ec7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:36.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26d4116b80 1 -- 172.21.15.60:0/2882879117 shutdown_connections 2023-07-13T22:31:36.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26d4116b80 1 -- 172.21.15.60:0/2882879117 wait complete. 2023-07-13T22:31:36.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26d4116b80 20 client.9576 trim_cache size 0 max 16384 2023-07-13T22:31:36.562 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:31:36.562 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce", 2023-07-13T22:31:36.562 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-07-13T22:31:36.562 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9423, 2023-07-13T22:31:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-07-13T22:31:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-07-13T22:31:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-07-13T22:31:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.6627773389999998, 2023-07-13T22:31:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 97, 2023-07-13T22:31:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 135, 2023-07-13T22:31:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 135, 2023-07-13T22:31:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 12.111752451999999 2023-07-13T22:31:36.564 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:31:36.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 -- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f269c06ace0 msgr2=0x7f269c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f269c06ace0 0x7f269c06d190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f26b814c560 tx=0x7f26a8008040 comp rx=0 tx=0).stop 2023-07-13T22:31:36.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 -- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 msgr2=0x7f26b80c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 0x7f26b80c6da0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f26b400e930 tx=0x7f26b400ecf0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 -- 172.21.15.60:0/3950771329 shutdown_connections 2023-07-13T22:31:36.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f269c06ace0 0x7f269c06d190 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26b80c8d40 0x7f26b80c6da0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b815a820 0x7f26b8159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 --2- 172.21.15.60:0/3950771329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26b8154950 0x7f26b81440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 -- 172.21.15.60:0/3950771329 >> 172.21.15.60:0/3950771329 conn(0x7f26b80cf410 msgr2=0x7f26b815cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:36.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.572+0000 7f26c07ff700 1 -- 172.21.15.60:0/3950771329 shutdown_connections 2023-07-13T22:31:36.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.573+0000 7f26c07ff700 1 -- 172.21.15.60:0/3950771329 wait complete. 2023-07-13T22:31:36.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:31:36.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.792+0000 7fe2f81ff700 1 Processor -- start 2023-07-13T22:31:36.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.792+0000 7fe2f81ff700 1 -- start start 2023-07-13T22:31:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f81ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f015a820 0x7fe2f015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f81ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2f00c8d40 0x7fe2f00c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f81ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f0070270 con 0x7fe2f0154950 2023-07-13T22:31:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f81ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f00713d0 con 0x7fe2f00c8d40 2023-07-13T22:31:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f81ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00c96c0 con 0x7fe2f015a820 2023-07-13T22:31:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f5f9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe2f0154950 0x7fe2f0070cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54098/0 (socket says 172.21.15.60:54098) 2023-07-13T22:31:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f5f9b700 1 -- 172.21.15.60:0/2956968470 learned_addr learned my addr 172.21.15.60:0/2956968470 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f679c700 1 --2- 172.21.15.60:0/2956968470 >> v2:172.21.15.60:3300/0 conn(0x7fe2f00c8d40 0x7fe2f00c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f579a700 1 --2- 172.21.15.60:0/2956968470 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f015a820 0x7fe2f015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f4f99700 1 -- 172.21.15.60:0/2956968470 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952747833 0 0) 0x7fe2f0070270 con 0x7fe2f0154950 2023-07-13T22:31:36.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f4f99700 1 -- 172.21.15.60:0/2956968470 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2dc0035b0 con 0x7fe2f0154950 2023-07-13T22:31:36.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f679c700 1 -- 172.21.15.60:0/2956968470 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f015a820 msgr2=0x7fe2f015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:36.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f679c700 1 --2- 172.21.15.60:0/2956968470 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f015a820 0x7fe2f015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f679c700 1 -- 172.21.15.60:0/2956968470 >> v1:172.21.15.60:6789/0 conn(0x7fe2f0154950 legacy=0x7fe2f0070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.794+0000 7fe2f679c700 1 -- 172.21.15.60:0/2956968470 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f015b2e0 con 0x7fe2f00c8d40 2023-07-13T22:31:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.795+0000 7fe2f679c700 1 --2- 172.21.15.60:0/2956968470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f00c8d40 0x7fe2f00c9180 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009d90 tx=0x7fe2e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df53b80e65a4ea server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.795+0000 7fe2f4f99700 1 -- 172.21.15.60:0/2956968470 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e800e070 con 0x7fe2f00c8d40 2023-07-13T22:31:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.795+0000 7fe2f4f99700 1 -- 172.21.15.60:0/2956968470 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e800bb20 con 0x7fe2f00c8d40 2023-07-13T22:31:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.795+0000 7fe2f4f99700 1 -- 172.21.15.60:0/2956968470 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8012430 con 0x7fe2f00c8d40 2023-07-13T22:31:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.795+0000 7fe2f81ff700 1 -- 172.21.15.60:0/2956968470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f00c8d40 msgr2=0x7fe2f00c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.795+0000 7fe2f81ff700 1 --2- 172.21.15.60:0/2956968470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f00c8d40 0x7fe2f00c9180 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009d90 tx=0x7fe2e800b330 comp rx=0 tx=0).stop 2023-07-13T22:31:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.796+0000 7fe2f81ff700 1 -- 172.21.15.60:0/2956968470 shutdown_connections 2023-07-13T22:31:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.796+0000 7fe2f81ff700 1 --2- 172.21.15.60:0/2956968470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f00c8d40 0x7fe2f00c9180 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.796+0000 7fe2f81ff700 1 --2- 172.21.15.60:0/2956968470 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f015a820 0x7fe2f015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.796+0000 7fe2f81ff700 1 -- 172.21.15.60:0/2956968470 >> 172.21.15.60:0/2956968470 conn(0x7fe2f00cf410 msgr2=0x7fe2f00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.796+0000 7fe2f81ff700 1 -- 172.21.15.60:0/2956968470 shutdown_connections 2023-07-13T22:31:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.796+0000 7fe2f81ff700 1 -- 172.21.15.60:0/2956968470 wait complete. 2023-07-13T22:31:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.797+0000 7fe2f81ff700 1 Processor -- start 2023-07-13T22:31:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.798+0000 7fe2f81ff700 1 -- start start 2023-07-13T22:31:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.798+0000 7fe2f81ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f00c8d40 0x7fe2f00b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.798+0000 7fe2f81ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f0154950 0x7fe2f00b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f81ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f015a820 0x7fe2f014c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f81ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00af5a0 con 0x7fe2f0154950 2023-07-13T22:31:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f81ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00af880 con 0x7fe2f015a820 2023-07-13T22:31:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f81ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00afb60 con 0x7fe2f00c8d40 2023-07-13T22:31:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f579a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f0154950 0x7fe2f00b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f579a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f0154950 0x7fe2f00b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45926/0 (socket says 172.21.15.60:45926) 2023-07-13T22:31:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f579a700 1 -- 172.21.15.60:0/441666932 learned_addr learned my addr 172.21.15.60:0/441666932 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f5f9b700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f00c8d40 0x7fe2f00b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f679c700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f015a820 0x7fe2f014c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f579a700 1 -- 172.21.15.60:0/441666932 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f00c8d40 msgr2=0x7fe2f00b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f579a700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f00c8d40 0x7fe2f00b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f579a700 1 -- 172.21.15.60:0/441666932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f015a820 msgr2=0x7fe2f014c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f579a700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f015a820 0x7fe2f014c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f579a700 1 -- 172.21.15.60:0/441666932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8009a70 con 0x7fe2f0154950 2023-07-13T22:31:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f5f9b700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f00c8d40 0x7fe2f00b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.799+0000 7fe2f579a700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f0154950 0x7fe2f00b3130 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000e870 tx=0x7fe2e000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.800+0000 7fe2e67fc700 1 -- 172.21.15.60:0/441666932 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0009590 con 0x7fe2f0154950 2023-07-13T22:31:36.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.800+0000 7fe2e67fc700 1 -- 172.21.15.60:0/441666932 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e000ee70 con 0x7fe2f0154950 2023-07-13T22:31:36.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.800+0000 7fe2e67fc700 1 -- 172.21.15.60:0/441666932 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e000ca50 con 0x7fe2f0154950 2023-07-13T22:31:36.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.800+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f00afe40 con 0x7fe2f0154950 2023-07-13T22:31:36.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.800+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f00af250 con 0x7fe2f0154950 2023-07-13T22:31:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.801+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f004e280 con 0x7fe2f0154950 2023-07-13T22:31:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.802+0000 7fe2e67fc700 1 -- 172.21.15.60:0/441666932 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe2e000cbb0 con 0x7fe2f0154950 2023-07-13T22:31:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.803+0000 7fe2e67fc700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2d406ace0 0x7fe2d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.803+0000 7fe2e67fc700 1 -- 172.21.15.60:0/441666932 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe2e00894f0 con 0x7fe2f0154950 2023-07-13T22:31:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.803+0000 7fe2f5f9b700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2d406ace0 0x7fe2d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.804+0000 7fe2f5f9b700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2d406ace0 0x7fe2d406d190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0011e0 tx=0x7fe2ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:36.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:36.807+0000 7fe2e67fc700 1 -- 172.21.15.60:0/441666932 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe2e0057f90 con 0x7fe2f0154950 2023-07-13T22:31:37.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.059+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe2f00af6e0 con 0x7fe2f0154950 2023-07-13T22:31:37.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.060+0000 7fe2e67fc700 1 -- 172.21.15.60:0/441666932 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe2f00af6e0 con 0x7fe2f0154950 2023-07-13T22:31:37.043 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:31:37.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2d406ace0 msgr2=0x7fe2d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:37.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2d406ace0 0x7fe2d406d190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0011e0 tx=0x7fe2ec009040 comp rx=0 tx=0).stop 2023-07-13T22:31:37.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f0154950 msgr2=0x7fe2f00b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:37.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f0154950 0x7fe2f00b3130 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000e870 tx=0x7fe2e000ec30 comp rx=0 tx=0).stop 2023-07-13T22:31:37.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 shutdown_connections 2023-07-13T22:31:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe2d406ace0 0x7fe2d406d190 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f015a820 0x7fe2f014c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2f00c8d40 0x7fe2f00b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 --2- 172.21.15.60:0/441666932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f0154950 0x7fe2f00b3130 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 >> 172.21.15.60:0/441666932 conn(0x7fe2f00cf410 msgr2=0x7fe2f0159130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 shutdown_connections 2023-07-13T22:31:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:37.062+0000 7fe2f81ff700 1 -- 172.21.15.60:0/441666932 wait complete. 2023-07-13T22:31:37.063 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:31:37.064 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:31:37.064 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:31:37.064 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:31:37.065 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:37.065 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:31:37.065 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:31:37.065 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:31:37.066 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:31:37.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:37.066 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:31:37.117 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:31:37.117 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:31:37.117 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:31:37.117 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:37.117 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:31:37.118 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:37.118 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:31:37.118 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:37.118 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:31:37.118 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:31:37.118 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:37.118 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:31:37.118 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:31:37.119 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:37.119 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:31:37.119 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:31:37.119 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41076sec preferred_lft 41076sec 2023-07-13T22:31:37.119 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:31:37.119 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:37.119 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:31:37.120 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:37.120 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:31:37.120 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:31:37.120 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:37.120 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:31:37.120 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:37.121 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:31:37.121 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:37.121 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:31:37.121 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:31:37.121 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:31:37.121 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:37.121 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:31:37.122 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:31:37.122 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:31:37.122 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:37.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:37.123 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:31:37.192 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:31:37.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:37.196 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:31:37.245 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:31:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:31:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:31:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:31:37.246 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:31:37.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:37.247 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:31:37.247 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:31:37.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:31:37.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:31:37.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:31:37.248 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:31:37.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:37.422 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:31:37.468 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:31:37.469 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:31:37.470 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:31:37.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:37.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:31:37.543 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:31:37.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:31:37.545 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:37.590 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:31:37.591 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:31:37.591 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:37.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:31:37.790 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:31:37.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:31:37.860 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:31:37.863 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:31:37.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:37.864 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:31:37.913 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:31:37.914 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:31:37.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:31:37.995 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:31:37.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:31:38.067 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:31:38.069 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:31:38.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:38.071 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:31:38.096 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:31:38.113+0000 7f76328f65c0 -1 init, newargv = 0x563715b6ece0 newargc=15 2023-07-13T22:31:38.096 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[166287]: starting ceph client 2023-07-13T22:31:38.106 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[166287]: starting fuse 2023-07-13T22:31:38.127 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:31:38.127 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:31:38.128 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:31:38.128 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:31:38.129 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:31:38.129 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:31:38.129 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:31:38.129 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:38.130 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:31:38.130 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:31:38.130 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:31:38.131 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:31:38.131 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:38.131 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:31:38.131 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:31:38.132 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:31:38.133 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:31:38.133 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:38.133 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:31:38.133 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:31:38.134 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:31:38.134 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:31:38.134 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:31:38.135 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:31:38.135 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:31:38.135 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:31:38.135 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:38.136 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:31:38.136 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:31:38.314 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.166287.asok 2023-07-13T22:31:38.492 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.166287.asok 2023-07-13T22:31:38.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:38.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.166287.asok status 2023-07-13T22:31:38.725 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:31:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:31:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:31:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:31:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:31:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:31:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:31:38.726 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "166287", 2023-07-13T22:31:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:31:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:31:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:31:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9591, 2023-07-13T22:31:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:31:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:31:38.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:31:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 9591 2023-07-13T22:31:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:31:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:31:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:31:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3281679451 2023-07-13T22:31:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:38.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:31:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:31:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:31:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3281679451 2023-07-13T22:31:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.9591 192.168.0.1:0/3281679451", 2023-07-13T22:31:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3281679451", 2023-07-13T22:31:38.729 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:31:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 97, 2023-07-13T22:31:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 135, 2023-07-13T22:31:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:31:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:31:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:31:38.730 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:31:38.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:38.740 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:31:38.741 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:38.787 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:31:38.788 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:38.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:38.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:38.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:31:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.119+0000 7f22e27ff700 1 Processor -- start 2023-07-13T22:31:39.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.119+0000 7f22e27ff700 1 -- start start 2023-07-13T22:31:39.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.121+0000 7f22e27ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f22dc0b0dd0 0x7f22dc160950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:39.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.121+0000 7f22e27ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc15c790 con 0x7f22dc15c3c0 2023-07-13T22:31:39.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.121+0000 7f22e27ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc0ccae0 con 0x7f22dc15d340 2023-07-13T22:31:39.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.121+0000 7f22e27ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc0ccdc0 con 0x7f22dc0b0dd0 2023-07-13T22:31:39.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.121+0000 7f22dbfff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f22dc15c3c0 0x7f22dc0b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54106/0 (socket says 172.21.15.60:54106) 2023-07-13T22:31:39.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.121+0000 7f22dbfff700 1 -- 172.21.15.60:0/595079400 learned_addr learned my addr 172.21.15.60:0/595079400 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:39.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.121+0000 7f22db7fe700 1 --2- 172.21.15.60:0/595079400 >> v2:172.21.15.105:3300/0 conn(0x7f22dc0b0dd0 0x7f22dc160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:39.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.121+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214920514 0 0) 0x7f22cc001110 con 0x7f22dc15c3c0 2023-07-13T22:31:39.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.121+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c40035b0 con 0x7f22dc15c3c0 2023-07-13T22:31:39.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043113119 0 0) 0x7f22dc0ccae0 con 0x7f22dc15d340 2023-07-13T22:31:39.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22cc001110 con 0x7f22dc15d340 2023-07-13T22:31:39.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 370256881 0 0) 0x7f22c40035b0 con 0x7f22dc15c3c0 2023-07-13T22:31:39.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 >> v2:172.21.15.105:3300/0 conn(0x7f22dc0b0dd0 msgr2=0x7f22dc160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:39.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 --2- 172.21.15.60:0/595079400 >> v2:172.21.15.105:3300/0 conn(0x7f22dc0b0dd0 0x7f22dc160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 >> v1:172.21.15.105:6789/0 conn(0x7f22dc15d340 legacy=0x7f22dc0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:39.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc15fe70 con 0x7f22dc15c3c0 2023-07-13T22:31:39.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f22cc001800 con 0x7f22dc15c3c0 2023-07-13T22:31:39.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 >> v1:172.21.15.60:6789/0 conn(0x7f22dc15c3c0 legacy=0x7f22dc0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:39.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4003b90 0x7f22c4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22c4004850 0x7f22c4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22c4005200 0x7f22c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0ccdc0 con 0x7f22c4005200 2023-07-13T22:31:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c400c110 con 0x7f22c4003b90 2023-07-13T22:31:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c400c2d0 con 0x7f22c4004850 2023-07-13T22:31:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22e0d9c700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22c4005200 0x7f22c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22dbfff700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4003b90 0x7f22c4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.122+0000 7f22db7fe700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22c4004850 0x7f22c4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:39.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22e0d9c700 1 -- 172.21.15.60:0/595079400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22c4004850 msgr2=0x7f22c4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:39.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22e0d9c700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22c4004850 0x7f22c4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22e0d9c700 1 -- 172.21.15.60:0/595079400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4003b90 msgr2=0x7f22c4003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:39.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22e0d9c700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4003b90 0x7f22c4003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22e0d9c700 1 -- 172.21.15.60:0/595079400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c40035b0 con 0x7f22c4005200 2023-07-13T22:31:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22e0d9c700 1 -- 172.21.15.60:0/595079400 reap_dead start 2023-07-13T22:31:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22e0d9c700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22c4005200 0x7f22c400ba10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f22d4002230 tx=0x7f22c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb0b62d0c51c90a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4001f30 con 0x7f22c4005200 2023-07-13T22:31:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d400ce60 con 0x7f22c4005200 2023-07-13T22:31:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.123+0000 7f22daffd700 1 -- 172.21.15.60:0/595079400 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4003c40 con 0x7f22c4005200 2023-07-13T22:31:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.124+0000 7f22e27ff700 1 -- 172.21.15.60:0/595079400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22c4005200 msgr2=0x7f22c400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.124+0000 7f22e27ff700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22c4005200 0x7f22c400ba10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f22d4002230 tx=0x7f22c4005880 comp rx=0 tx=0).stop 2023-07-13T22:31:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.124+0000 7f22e27ff700 1 -- 172.21.15.60:0/595079400 shutdown_connections 2023-07-13T22:31:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.124+0000 7f22e27ff700 1 --2- 172.21.15.60:0/595079400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22c4005200 0x7f22c400ba10 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.124+0000 7f22e27ff700 1 -- 172.21.15.60:0/595079400 >> 172.21.15.60:0/595079400 conn(0x7f22dc0cebc0 msgr2=0x7f22dc0cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.124+0000 7f22e27ff700 1 -- 172.21.15.60:0/595079400 shutdown_connections 2023-07-13T22:31:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.124+0000 7f22e27ff700 1 -- 172.21.15.60:0/595079400 wait complete. 2023-07-13T22:31:39.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.126+0000 7f22e27ff700 1 Processor -- start 2023-07-13T22:31:39.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.127+0000 7f22e27ff700 1 -- start start 2023-07-13T22:31:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22e27ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4005200 0x7f22dc15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22e27ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc15d740 0x7f22dc0c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22e27ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22dc0c30f0 0x7f22dc15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22e27ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc1f6570 con 0x7f22dc15d740 2023-07-13T22:31:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22e27ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc15fc70 con 0x7f22c4005200 2023-07-13T22:31:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22e27ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc146880 con 0x7f22dc0c30f0 2023-07-13T22:31:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22db7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc15d740 0x7f22dc0c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22db7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc15d740 0x7f22dc0c2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45944/0 (socket says 172.21.15.60:45944) 2023-07-13T22:31:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22db7fe700 1 -- 172.21.15.60:0/504648197 learned_addr learned my addr 172.21.15.60:0/504648197 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22dbfff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4005200 0x7f22dc15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22e0d9c700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22dc0c30f0 0x7f22dc15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22db7fe700 1 -- 172.21.15.60:0/504648197 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22dc0c30f0 msgr2=0x7f22dc15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22db7fe700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22dc0c30f0 0x7f22dc15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22db7fe700 1 -- 172.21.15.60:0/504648197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4005200 msgr2=0x7f22dc15f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:39.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22db7fe700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4005200 0x7f22dc15f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.128+0000 7f22db7fe700 1 -- 172.21.15.60:0/504648197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc1f59f0 con 0x7f22dc15d740 2023-07-13T22:31:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.129+0000 7f22dbfff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4005200 0x7f22dc15f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.129+0000 7f22db7fe700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc15d740 0x7f22dc0c2bb0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f22d00119a0 tx=0x7f22d0011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.129+0000 7f22d8ff9700 1 -- 172.21.15.60:0/504648197 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000f940 con 0x7f22dc15d740 2023-07-13T22:31:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.129+0000 7f22d8ff9700 1 -- 172.21.15.60:0/504648197 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d000faa0 con 0x7f22dc15d740 2023-07-13T22:31:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.129+0000 7f22d8ff9700 1 -- 172.21.15.60:0/504648197 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000fe30 con 0x7f22dc15d740 2023-07-13T22:31:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.129+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc1f5cd0 con 0x7f22dc15d740 2023-07-13T22:31:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.129+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc1f6290 con 0x7f22dc15d740 2023-07-13T22:31:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.130+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc146a10 con 0x7f22dc15d740 2023-07-13T22:31:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.131+0000 7f22d8ff9700 1 -- 172.21.15.60:0/504648197 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f22d0014440 con 0x7f22dc15d740 2023-07-13T22:31:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.133+0000 7f22d8ff9700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f22c806ace0 0x7f22c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.133+0000 7f22d8ff9700 1 -- 172.21.15.60:0/504648197 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f22d0089c40 con 0x7f22dc15d740 2023-07-13T22:31:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.133+0000 7f22dbfff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f22c806ace0 0x7f22c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.133+0000 7f22dbfff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f22c806ace0 0x7f22c806d190 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f22cc007fd0 tx=0x7f22cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.138+0000 7f22d8ff9700 1 -- 172.21.15.60:0/504648197 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f22d0058790 con 0x7f22dc15d740 2023-07-13T22:31:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.391+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22dc0cf940 con 0x7f22dc15d740 2023-07-13T22:31:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.391+0000 7f22d8ff9700 1 -- 172.21.15.60:0/504648197 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) v1 ==== 53+0+73 (secure 0 0 0) 0x7f22d0022070 con 0x7f22dc15d740 2023-07-13T22:31:39.374 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:31:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f22c806ace0 msgr2=0x7f22c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f22c806ace0 0x7f22c806d190 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f22cc007fd0 tx=0x7f22cc009040 comp rx=0 tx=0).stop 2023-07-13T22:31:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc15d740 msgr2=0x7f22dc0c2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc15d740 0x7f22dc0c2bb0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f22d00119a0 tx=0x7f22d0011d60 comp rx=0 tx=0).stop 2023-07-13T22:31:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 shutdown_connections 2023-07-13T22:31:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f22c806ace0 0x7f22c806d190 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22dc0c30f0 0x7f22dc15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22c4005200 0x7f22dc15f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 --2- 172.21.15.60:0/504648197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22dc15d740 0x7f22dc0c2bb0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 >> 172.21.15.60:0/504648197 conn(0x7f22dc0cebc0 msgr2=0x7f22dc156840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 shutdown_connections 2023-07-13T22:31:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:39.394+0000 7f22e27ff700 1 -- 172.21.15.60:0/504648197 wait complete. 2023-07-13T22:31:39.394 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:31:39.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:31:39.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2023-07-13T22:31:39.395 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-13T22:31:39.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-13T22:31:39.396 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:31:39.396 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:31:39.396 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:31:39.397 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:31:39.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:39.397 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2023-07-13T22:31:39.450 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:31:39.450 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:31:39.450 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:31:39.450 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:39.450 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2023-07-13T22:31:39.451 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:39.451 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:31:39.451 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:39.451 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2023-07-13T22:31:39.451 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:31:39.451 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:39.451 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2023-07-13T22:31:39.451 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:31:39.452 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:39.452 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:31:39.452 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:39.452 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-13T22:31:39.452 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 41071sec preferred_lft 41071sec 2023-07-13T22:31:39.452 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2023-07-13T22:31:39.452 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:39.453 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:31:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:31:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::509d:3eff:fe19:ae59/64 scope link 2023-07-13T22:31:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:39.453 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:31:39.453 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ae:42:15:58:1e:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:31:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ac42:15ff:fe58:1e47/64 scope link 2023-07-13T22:31:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:39.454 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:31:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 16:e3:bf:a8:bf:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:31:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::14e3:bfff:fea8:bf57/64 scope link 2023-07-13T22:31:39.454 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:39.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:39.456 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:31:39.526 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-07-13T22:31:39.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:39.530 DEBUG:teuthology.orchestra.run.smithi105:> route 2023-07-13T22:31:39.580 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2023-07-13T22:31:39.581 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:31:39.581 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-13T22:31:39.581 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-13T22:31:39.581 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:31:39.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:39.582 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2023-07-13T22:31:39.583 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2023-07-13T22:31:39.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-13T22:31:39.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-13T22:31:39.584 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-13T22:31:39.584 DEBUG:teuthology.orchestra.run.smithi105:> ') 2023-07-13T22:31:39.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:39.755 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:31:39.801 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:31:39.802 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:31:39.802 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:31:39.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:39.803 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2023-07-13T22:31:39.911 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-13T22:31:39.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:31:39.912 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-13T22:31:39.957 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-13T22:31:39.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:31:39.958 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:31:40.043 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:31:40.152 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:31:40.153 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:31:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:31:40.223 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:31:40.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:40.225 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:31:40.270 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:31:40.271 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-13T22:31:40.275 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2023-07-13T22:31:40.346 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:31:40.346 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:31:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:31:40.417 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:31:40.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:40.418 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2023-07-13T22:31:40.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:2023-07-13T22:31:40.460+0000 7faed0c0c5c0 -1 init, newargv = 0x560ec6967060 newargc=15 2023-07-13T22:31:40.443 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[128014]: starting ceph client 2023-07-13T22:31:40.451 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[128014]: starting fuse 2023-07-13T22:31:40.469 INFO:teuthology.orchestra.run.smithi105.stdout:43 2023-07-13T22:31:40.469 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:31:40.470 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:31:40.470 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2023-07-13T22:31:40.470 DEBUG:teuthology.orchestra.run.smithi105:> import re 2023-07-13T22:31:40.470 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:31:40.471 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2023-07-13T22:31:40.471 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:31:40.471 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2023-07-13T22:31:40.472 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-13T22:31:40.472 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2023-07-13T22:31:40.472 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-13T22:31:40.472 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:31:40.473 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2023-07-13T22:31:40.473 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2023-07-13T22:31:40.474 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2023-07-13T22:31:40.475 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2023-07-13T22:31:40.475 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:31:40.476 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2023-07-13T22:31:40.477 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:31:40.477 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:31:40.478 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:31:40.478 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2023-07-13T22:31:40.478 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2023-07-13T22:31:40.479 DEBUG:teuthology.orchestra.run.smithi105:> return f 2023-07-13T22:31:40.479 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:31:40.479 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:31:40.479 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.1")) 2023-07-13T22:31:40.480 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:31:40.653 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.1.128014.asok 2023-07-13T22:31:40.831 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.128014.asok 2023-07-13T22:31:40.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:40.832 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.128014.asok status 2023-07-13T22:31:41.055 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2023-07-13T22:31:41.055 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2023-07-13T22:31:41.056 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:31:41.056 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:31:41.056 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "1", 2023-07-13T22:31:41.056 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2023-07-13T22:31:41.056 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-13T22:31:41.056 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "128014", 2023-07-13T22:31:41.056 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2023-07-13T22:31:41.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:31:41.057 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2023-07-13T22:31:41.057 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2023-07-13T22:31:41.057 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 8872, 2023-07-13T22:31:41.057 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2023-07-13T22:31:41.057 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2023-07-13T22:31:41.057 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2023-07-13T22:31:41.057 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 8872 2023-07-13T22:31:41.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:31:41.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:31:41.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:31:41.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:31:41.058 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3352034258 2023-07-13T22:31:41.058 INFO:teuthology.orchestra.run.smithi105.stdout: } 2023-07-13T22:31:41.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:31:41.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2023-07-13T22:31:41.059 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2023-07-13T22:31:41.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:31:41.059 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3352034258 2023-07-13T22:31:41.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2023-07-13T22:31:41.059 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.8872 192.168.0.1:0/3352034258", 2023-07-13T22:31:41.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/3352034258", 2023-07-13T22:31:41.059 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2023-07-13T22:31:41.060 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 97, 2023-07-13T22:31:41.060 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 135, 2023-07-13T22:31:41.060 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:31:41.060 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2023-07-13T22:31:41.060 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2023-07-13T22:31:41.060 INFO:teuthology.orchestra.run.smithi105.stdout:} 2023-07-13T22:31:41.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:41.069 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2023-07-13T22:31:41.070 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-13T22:31:41.117 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2023-07-13T22:31:41.117 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-13T22:31:41.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:41.118 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-13T22:31:41.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:31:41.233 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:31:41.351 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-07-13T22:31:41.352 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-07-13T22:31:41.353 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:31:41.353 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-07-13T22:31:41.353 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.354 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-07-13T22:31:41.354 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.354 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-07-13T22:31:41.354 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.355 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-07-13T22:31:41.355 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-07-13T22:31:41.355 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.355 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-07-13T22:31:41.356 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-07-13T22:31:41.356 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.356 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:31:41.356 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-07-13T22:31:41.357 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.357 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:31:41.357 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:31:41.358 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.358 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-07-13T22:31:41.358 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.358 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-07-13T22:31:41.358 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-07-13T22:31:41.358 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:31:41.358 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:31:41.359 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-07-13T22:31:41.359 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:31:41.359 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-07-13T22:31:41.359 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:31:41.359 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:31:41.359 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-07-13T22:31:41.359 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.360 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:31:41.360 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.360 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-07-13T22:31:41.360 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.360 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-07-13T22:31:41.360 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.361 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-07-13T22:31:41.361 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-07-13T22:31:41.361 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-07-13T22:31:41.361 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:31:41.361 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:31:41.361 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.361 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-07-13T22:31:41.361 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-07-13T22:31:41.362 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.362 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-07-13T22:31:41.362 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2023-07-13T22:31:41.362 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:31:41.363 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.363 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-07-13T22:31:41.363 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-07-13T22:31:41.363 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.363 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-07-13T22:31:41.363 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-07-13T22:31:41.364 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-07-13T22:31:41.364 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.364 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-07-13T22:31:41.364 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-07-13T22:31:41.364 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.364 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:31:41.365 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.365 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-07-13T22:31:41.365 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.365 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-07-13T22:31:41.365 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.365 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:31:41.365 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-07-13T22:31:41.366 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.366 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-07-13T22:31:41.366 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-07-13T22:31:41.366 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2023-07-13T22:31:41.366 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2023-07-13T22:31:41.366 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2023-07-13T22:31:41.366 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2023-07-13T22:31:41.367 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2023-07-13T22:31:41.367 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2023-07-13T22:31:41.367 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2023-07-13T22:31:41.367 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:31:41.367 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-07-13T22:31:41.367 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2023-07-13T22:31:41.367 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:31:41.368 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2023-07-13T22:31:41.368 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:31:41.368 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:31:41.368 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2023-07-13T22:31:41.368 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:31:41.368 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.368 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-07-13T22:31:41.369 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:31:41.369 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-07-13T22:31:41.369 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:31:41.369 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:31:41.369 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:31:41.369 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:31:41.369 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-07-13T22:31:41.370 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.370 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-07-13T22:31:41.370 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:31:41.370 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:31:41.370 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-07-13T22:31:41.370 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:31:41.370 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-07-13T22:31:41.370 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-07-13T22:31:41.371 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:31:41.371 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-07-13T22:31:41.371 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-07-13T22:31:41.371 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:31:41.371 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.371 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:31:41.371 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:31:41.372 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:31:41.372 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.372 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:31:41.372 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:31:41.372 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:31:41.372 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.372 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-07-13T22:31:41.373 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:31:41.373 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.373 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:31:41.373 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2023-07-13T22:31:41.373 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:31:41.373 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-07-13T22:31:41.374 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-07-13T22:31:41.374 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-07-13T22:31:41.374 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-07-13T22:31:41.375 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:31:41.375 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:31:41.375 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2023-07-13T22:31:41.375 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2023-07-13T22:31:41.376 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:31:41.376 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2023-07-13T22:31:41.376 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:31:41.377 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:31:41.377 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-07-13T22:31:41.377 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2023-07-13T22:31:41.377 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2023-07-13T22:31:41.378 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-07-13T22:31:41.378 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-07-13T22:31:41.378 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-07-13T22:31:41.379 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-07-13T22:31:41.379 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2023-07-13T22:31:41.379 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2023-07-13T22:31:41.379 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2023-07-13T22:31:41.380 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:31:41.380 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2023-07-13T22:31:41.380 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2023-07-13T22:31:41.380 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2023-07-13T22:31:41.381 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2023-07-13T22:31:41.381 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:31:41.381 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:31:41.381 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:31:41.381 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-07-13T22:31:41.381 INFO:teuthology.orchestra.run.smithi105.stdout: client inject release failure = true 2023-07-13T22:31:41.382 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2023-07-13T22:31:41.382 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2023-07-13T22:31:41.382 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:31:41.383 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2023-07-13T22:31:41.383 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.383 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2023-07-13T22:31:41.383 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.383 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2023-07-13T22:31:41.383 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.383 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2023-07-13T22:31:41.384 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2023-07-13T22:31:41.384 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.384 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2023-07-13T22:31:41.384 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2023-07-13T22:31:41.384 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.384 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:31:41.384 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2023-07-13T22:31:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:31:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:31:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2023-07-13T22:31:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2023-07-13T22:31:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2023-07-13T22:31:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:31:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:31:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2023-07-13T22:31:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:31:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2023-07-13T22:31:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:31:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:31:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2023-07-13T22:31:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:31:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2023-07-13T22:31:41.388 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.388 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2023-07-13T22:31:41.388 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.389 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2023-07-13T22:31:41.389 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2023-07-13T22:31:41.389 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2023-07-13T22:31:41.389 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:31:41.390 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:31:41.390 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.391 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2023-07-13T22:31:41.391 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2023-07-13T22:31:41.391 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.392 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2023-07-13T22:31:41.392 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2023-07-13T22:31:41.392 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:31:41.393 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.393 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2023-07-13T22:31:41.393 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2023-07-13T22:31:41.394 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.394 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2023-07-13T22:31:41.394 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2023-07-13T22:31:41.394 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2023-07-13T22:31:41.395 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.395 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2023-07-13T22:31:41.395 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2023-07-13T22:31:41.396 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.396 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:31:41.396 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.396 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2023-07-13T22:31:41.397 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.397 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2023-07-13T22:31:41.397 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.397 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:31:41.397 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2023-07-13T22:31:41.398 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.398 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2023-07-13T22:31:41.398 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2023-07-13T22:31:41.399 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2023-07-13T22:31:41.399 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2023-07-13T22:31:41.399 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2023-07-13T22:31:41.399 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2023-07-13T22:31:41.400 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2023-07-13T22:31:41.400 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2023-07-13T22:31:41.400 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2023-07-13T22:31:41.400 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:31:41.400 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2023-07-13T22:31:41.400 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2023-07-13T22:31:41.401 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:31:41.401 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2023-07-13T22:31:41.401 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:31:41.401 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:31:41.401 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2023-07-13T22:31:41.401 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:31:41.401 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.402 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2023-07-13T22:31:41.402 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:31:41.402 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2023-07-13T22:31:41.402 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:31:41.402 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:31:41.403 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:31:41.403 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:31:41.403 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2023-07-13T22:31:41.403 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.403 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2023-07-13T22:31:41.404 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:31:41.404 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:31:41.404 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2023-07-13T22:31:41.404 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:31:41.404 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2023-07-13T22:31:41.404 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2023-07-13T22:31:41.405 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:31:41.405 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2023-07-13T22:31:41.405 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2023-07-13T22:31:41.405 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:31:41.405 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.405 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:31:41.405 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:31:41.405 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:31:41.406 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.406 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:31:41.406 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:31:41.406 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:31:41.406 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.406 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2023-07-13T22:31:41.406 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:31:41.407 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.407 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:31:41.407 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2023-07-13T22:31:41.407 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.407 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2023-07-13T22:31:41.407 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2023-07-13T22:31:41.407 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2023-07-13T22:31:41.408 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2023-07-13T22:31:41.408 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:31:41.408 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:31:41.408 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2023-07-13T22:31:41.408 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2023-07-13T22:31:41.408 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:31:41.408 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2023-07-13T22:31:41.409 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:31:41.409 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:31:41.409 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2023-07-13T22:31:41.409 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2023-07-13T22:31:41.409 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2023-07-13T22:31:41.409 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-07-13T22:31:41.409 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-07-13T22:31:41.410 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-07-13T22:31:41.410 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-07-13T22:31:41.410 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2023-07-13T22:31:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2023-07-13T22:31:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2023-07-13T22:31:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:31:41.410 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2023-07-13T22:31:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2023-07-13T22:31:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2023-07-13T22:31:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2023-07-13T22:31:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:31:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:31:41.411 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:31:41.411 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:31:41.412 INFO:teuthology.orchestra.run.smithi060.stdout: client inject release failure = true 2023-07-13T22:31:41.413 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:31:41.414 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:41.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:31:41.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.721+0000 7f39677ff700 1 Processor -- start 2023-07-13T22:31:41.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.721+0000 7f39677ff700 1 -- start start 2023-07-13T22:31:41.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f39677ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 0x7f3960160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:41.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f39677ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39600cf010 0x7f39600d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:41.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f39677ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39600722e0 con 0x7f396015c6f0 2023-07-13T22:31:41.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f39677ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39600d0df0 con 0x7f39600cf010 2023-07-13T22:31:41.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f39677ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39600cdbd0 con 0x7f39600cfdd0 2023-07-13T22:31:41.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f3965d9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39600cf010 0x7f39600d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:41.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f396559b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f396015c6f0 0x7f39600cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54110/0 (socket says 172.21.15.60:54110) 2023-07-13T22:31:41.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f396559b700 1 -- 172.21.15.60:0/2934796567 learned_addr learned my addr 172.21.15.60:0/2934796567 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:41.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f3964d9a700 1 --2- 172.21.15.60:0/2934796567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 0x7f3960160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:41.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f3965d9c700 1 -- 172.21.15.60:0/2934796567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 msgr2=0x7f3960160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:41.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f3957fff700 1 -- 172.21.15.60:0/2934796567 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209277405 0 0) 0x7f395c001110 con 0x7f396015c6f0 2023-07-13T22:31:41.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f3965d9c700 1 --2- 172.21.15.60:0/2934796567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 0x7f3960160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:41.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f3965d9c700 1 -- 172.21.15.60:0/2934796567 >> v1:172.21.15.60:6789/0 conn(0x7f396015c6f0 legacy=0x7f39600cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:41.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f3965d9c700 1 -- 172.21.15.60:0/2934796567 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396015cac0 con 0x7f39600cf010 2023-07-13T22:31:41.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.723+0000 7f3964d9a700 1 --2- 172.21.15.60:0/2934796567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 0x7f3960160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:41.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.724+0000 7f3965d9c700 1 --2- 172.21.15.60:0/2934796567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 0x7f39600d1280 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f3958009d90 tx=0x7f395800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616fc2cfdb002aab server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.724+0000 7f3957fff700 1 -- 172.21.15.60:0/2934796567 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395800e070 con 0x7f39600cf010 2023-07-13T22:31:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.724+0000 7f3957fff700 1 -- 172.21.15.60:0/2934796567 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395800bbd0 con 0x7f39600cf010 2023-07-13T22:31:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.724+0000 7f3957fff700 1 -- 172.21.15.60:0/2934796567 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3958012430 con 0x7f39600cf010 2023-07-13T22:31:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.724+0000 7f39677ff700 1 -- 172.21.15.60:0/2934796567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 msgr2=0x7f39600d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:41.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.724+0000 7f39677ff700 1 --2- 172.21.15.60:0/2934796567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 0x7f39600d1280 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f3958009d90 tx=0x7f395800b330 comp rx=0 tx=0).stop 2023-07-13T22:31:41.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.725+0000 7f39677ff700 1 -- 172.21.15.60:0/2934796567 shutdown_connections 2023-07-13T22:31:41.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.725+0000 7f39677ff700 1 --2- 172.21.15.60:0/2934796567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 0x7f39600d1280 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:41.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.725+0000 7f39677ff700 1 --2- 172.21.15.60:0/2934796567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 0x7f3960160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:41.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.725+0000 7f39677ff700 1 -- 172.21.15.60:0/2934796567 >> 172.21.15.60:0/2934796567 conn(0x7f396006a4b0 msgr2=0x7f396006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:41.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.725+0000 7f39677ff700 1 -- 172.21.15.60:0/2934796567 shutdown_connections 2023-07-13T22:31:41.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.725+0000 7f39677ff700 1 -- 172.21.15.60:0/2934796567 wait complete. 2023-07-13T22:31:41.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.726+0000 7f39677ff700 1 Processor -- start 2023-07-13T22:31:41.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.726+0000 7f39677ff700 1 -- start start 2023-07-13T22:31:41.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.727+0000 7f39677ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 0x7f396014d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.727+0000 7f39677ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 0x7f39600b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.727+0000 7f39677ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f396015c6f0 0x7f39600cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.727+0000 7f39677ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b6860 con 0x7f39600cf010 2023-07-13T22:31:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.727+0000 7f39677ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b4fc0 con 0x7f396015c6f0 2023-07-13T22:31:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.727+0000 7f39677ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396014d760 con 0x7f39600cfdd0 2023-07-13T22:31:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f396559b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 0x7f396014d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f396559b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 0x7f396014d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45966/0 (socket says 172.21.15.60:45966) 2023-07-13T22:31:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f396559b700 1 -- 172.21.15.60:0/586808239 learned_addr learned my addr 172.21.15.60:0/586808239 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f3964d9a700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 0x7f39600b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f3965d9c700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f396015c6f0 0x7f39600cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f396559b700 1 -- 172.21.15.60:0/586808239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 msgr2=0x7f39600b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f396559b700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 0x7f39600b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f396559b700 1 -- 172.21.15.60:0/586808239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f396015c6f0 msgr2=0x7f39600cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f396559b700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f396015c6f0 0x7f39600cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:41.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f396559b700 1 -- 172.21.15.60:0/586808239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3958009a70 con 0x7f39600cf010 2023-07-13T22:31:41.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.728+0000 7f396559b700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 0x7f396014d220 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c580 tx=0x7f395c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:41.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.729+0000 7f3955ffb700 1 -- 172.21.15.60:0/586808239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c004c80 con 0x7f39600cf010 2023-07-13T22:31:41.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.729+0000 7f3955ffb700 1 -- 172.21.15.60:0/586808239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c004e00 con 0x7f39600cf010 2023-07-13T22:31:41.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.729+0000 7f3955ffb700 1 -- 172.21.15.60:0/586808239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c007310 con 0x7f39600cf010 2023-07-13T22:31:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.729+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3960158130 con 0x7f39600cf010 2023-07-13T22:31:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.729+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39600b4ce0 con 0x7f39600cf010 2023-07-13T22:31:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.730+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3960074220 con 0x7f39600cf010 2023-07-13T22:31:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.731+0000 7f3955ffb700 1 -- 172.21.15.60:0/586808239 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f395c007470 con 0x7f39600cf010 2023-07-13T22:31:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.732+0000 7f3955ffb700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f394406ace0 0x7f394406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.732+0000 7f3955ffb700 1 -- 172.21.15.60:0/586808239 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f395c088eb0 con 0x7f39600cf010 2023-07-13T22:31:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.732+0000 7f3964d9a700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f394406ace0 0x7f394406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.733+0000 7f3964d9a700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f394406ace0 0x7f394406d190 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f39601f96b0 tx=0x7f3950008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.736+0000 7f3955ffb700 1 -- 172.21.15.60:0/586808239 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f395c053f50 con 0x7f39600cf010 2023-07-13T22:31:41.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.945+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39600722e0 con 0x7f39600cf010 2023-07-13T22:31:41.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.947+0000 7f3955ffb700 1 -- 172.21.15.60:0/586808239 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f39600722e0 con 0x7f39600cf010 2023-07-13T22:31:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:41.931 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":135,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:30.894942+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:31:27.702135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-13T22:31:28.386570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:31:41.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f394406ace0 msgr2=0x7f394406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:41.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f394406ace0 0x7f394406d190 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f39601f96b0 tx=0x7f3950008040 comp rx=0 tx=0).stop 2023-07-13T22:31:41.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 msgr2=0x7f396014d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:41.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 0x7f396014d220 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c580 tx=0x7f395c00c890 comp rx=0 tx=0).stop 2023-07-13T22:31:41.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 shutdown_connections 2023-07-13T22:31:41.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f394406ace0 0x7f394406d190 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:41.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39600cf010 0x7f396014d220 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:41.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f396015c6f0 0x7f39600cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:41.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 --2- 172.21.15.60:0/586808239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39600cfdd0 0x7f39600b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:41.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.949+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 >> 172.21.15.60:0/586808239 conn(0x7f396006a4b0 msgr2=0x7f396006a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:41.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.950+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 shutdown_connections 2023-07-13T22:31:41.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:41.950+0000 7f39677ff700 1 -- 172.21.15.60:0/586808239 wait complete. 2023-07-13T22:31:41.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:31:27.702135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:31:28.386570+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:31:41.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:31:42.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.177+0000 7fe40a9ff700 1 Processor -- start 2023-07-13T22:31:42.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.177+0000 7fe40a9ff700 1 -- start start 2023-07-13T22:31:42.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.178+0000 7fe40a9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4041595f0 0x7fe4040af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.178+0000 7fe40a9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe4041599c0 0x7fe4040ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.178+0000 7fe40a9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4040c8fe0 con 0x7fe4040ad810 2023-07-13T22:31:42.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.178+0000 7fe40a9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4040af6e0 con 0x7fe4041595f0 2023-07-13T22:31:42.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.178+0000 7fe40a9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4040c92c0 con 0x7fe4041599c0 2023-07-13T22:31:42.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.178+0000 7fe408f9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe4040ad810 0x7fe4040ca5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54126/0 (socket says 172.21.15.60:54126) 2023-07-13T22:31:42.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.178+0000 7fe403fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4041595f0 0x7fe4040af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.178+0000 7fe408f9c700 1 -- 172.21.15.60:0/2621340140 learned_addr learned my addr 172.21.15.60:0/2621340140 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:42.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.179+0000 7fe4037fe700 1 --2- 172.21.15.60:0/2621340140 >> v2:172.21.15.105:3300/0 conn(0x7fe4041599c0 0x7fe4040ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.179+0000 7fe403fff700 1 -- 172.21.15.60:0/2621340140 >> v2:172.21.15.105:3300/0 conn(0x7fe4041599c0 msgr2=0x7fe4040ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:42.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.179+0000 7fe403fff700 1 --2- 172.21.15.60:0/2621340140 >> v2:172.21.15.105:3300/0 conn(0x7fe4041599c0 0x7fe4040ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.179+0000 7fe403fff700 1 -- 172.21.15.60:0/2621340140 >> v1:172.21.15.60:6789/0 conn(0x7fe4040ad810 legacy=0x7fe4040ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.179+0000 7fe403fff700 1 -- 172.21.15.60:0/2621340140 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4040c95a0 con 0x7fe4041595f0 2023-07-13T22:31:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.179+0000 7fe402ffd700 1 -- 172.21.15.60:0/2621340140 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109120661 0 0) 0x7fe4040c8fe0 con 0x7fe4040ad810 2023-07-13T22:31:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.179+0000 7fe403fff700 1 --2- 172.21.15.60:0/2621340140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 0x7fe4040af1a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fe40406f880 tx=0x7fe3f4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fc7e5e05a96802 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.179+0000 7fe402ffd700 1 -- 172.21.15.60:0/2621340140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4011830 con 0x7fe4041595f0 2023-07-13T22:31:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.179+0000 7fe402ffd700 1 -- 172.21.15.60:0/2621340140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f4011990 con 0x7fe4041595f0 2023-07-13T22:31:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.180+0000 7fe402ffd700 1 -- 172.21.15.60:0/2621340140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4011d20 con 0x7fe4041595f0 2023-07-13T22:31:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.180+0000 7fe40a9ff700 1 -- 172.21.15.60:0/2621340140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 msgr2=0x7fe4040af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.180+0000 7fe40a9ff700 1 --2- 172.21.15.60:0/2621340140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 0x7fe4040af1a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fe40406f880 tx=0x7fe3f4009b20 comp rx=0 tx=0).stop 2023-07-13T22:31:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.180+0000 7fe40a9ff700 1 -- 172.21.15.60:0/2621340140 shutdown_connections 2023-07-13T22:31:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.180+0000 7fe40a9ff700 1 --2- 172.21.15.60:0/2621340140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 0x7fe4040af1a0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.180+0000 7fe40a9ff700 1 --2- 172.21.15.60:0/2621340140 >> v2:172.21.15.105:3300/0 conn(0x7fe4041599c0 0x7fe4040ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.180+0000 7fe40a9ff700 1 -- 172.21.15.60:0/2621340140 >> 172.21.15.60:0/2621340140 conn(0x7fe4040cf5e0 msgr2=0x7fe4040cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.180+0000 7fe40a9ff700 1 -- 172.21.15.60:0/2621340140 shutdown_connections 2023-07-13T22:31:42.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.180+0000 7fe40a9ff700 1 -- 172.21.15.60:0/2621340140 wait complete. 2023-07-13T22:31:42.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.182+0000 7fe40a9ff700 1 Processor -- start 2023-07-13T22:31:42.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.182+0000 7fe40a9ff700 1 -- start start 2023-07-13T22:31:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe40a9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4040ad810 0x7fe4040b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe40a9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 0x7fe4041f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe40a9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4041599c0 0x7fe4040cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe40a9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4041f6090 con 0x7fe4041595f0 2023-07-13T22:31:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe40a9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040ce1d0 con 0x7fe4040ad810 2023-07-13T22:31:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe40a9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040cbf50 con 0x7fe4041599c0 2023-07-13T22:31:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe4037fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 0x7fe4041f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe4037fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 0x7fe4041f5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45990/0 (socket says 172.21.15.60:45990) 2023-07-13T22:31:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe4037fe700 1 -- 172.21.15.60:0/96748916 learned_addr learned my addr 172.21.15.60:0/96748916 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe403fff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4040ad810 0x7fe4040b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.183+0000 7fe408f9c700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4041599c0 0x7fe4040cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe4037fe700 1 -- 172.21.15.60:0/96748916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4041599c0 msgr2=0x7fe4040cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe4037fe700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4041599c0 0x7fe4040cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe4037fe700 1 -- 172.21.15.60:0/96748916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4040ad810 msgr2=0x7fe4040b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe4037fe700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4040ad810 0x7fe4040b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe4037fe700 1 -- 172.21.15.60:0/96748916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f40094a0 con 0x7fe4041595f0 2023-07-13T22:31:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe403fff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4040ad810 0x7fe4040b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe4037fe700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 0x7fe4041f5b50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800d800 tx=0x7fe3f800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe400ff9700 1 -- 172.21.15.60:0/96748916 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80125f0 con 0x7fe4041595f0 2023-07-13T22:31:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe400ff9700 1 -- 172.21.15.60:0/96748916 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8012750 con 0x7fe4041595f0 2023-07-13T22:31:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe400ff9700 1 -- 172.21.15.60:0/96748916 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8012b00 con 0x7fe4041595f0 2023-07-13T22:31:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4041f6330 con 0x7fe4041595f0 2023-07-13T22:31:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.184+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4041f68f0 con 0x7fe4041595f0 2023-07-13T22:31:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.185+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe404074220 con 0x7fe4041595f0 2023-07-13T22:31:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.186+0000 7fe400ff9700 1 -- 172.21.15.60:0/96748916 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3f8012c60 con 0x7fe4041595f0 2023-07-13T22:31:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.187+0000 7fe400ff9700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3f006ace0 0x7fe3f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.187+0000 7fe400ff9700 1 -- 172.21.15.60:0/96748916 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3f8089910 con 0x7fe4041595f0 2023-07-13T22:31:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.187+0000 7fe403fff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3f006ace0 0x7fe3f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.188+0000 7fe403fff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3f006ace0 0x7fe3f006d190 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40090b0 tx=0x7fe3f4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.191+0000 7fe400ff9700 1 -- 172.21.15.60:0/96748916 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe3f8054920 con 0x7fe4041595f0 2023-07-13T22:31:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.398+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe40406dcc0 con 0x7fe4041595f0 2023-07-13T22:31:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.398+0000 7fe400ff9700 1 -- 172.21.15.60:0/96748916 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v135) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe3f8057f40 con 0x7fe4041595f0 2023-07-13T22:31:42.382 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:31:42.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3f006ace0 msgr2=0x7fe3f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3f006ace0 0x7fe3f006d190 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40090b0 tx=0x7fe3f4008030 comp rx=0 tx=0).stop 2023-07-13T22:31:42.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 msgr2=0x7fe4041f5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 0x7fe4041f5b50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800d800 tx=0x7fe3f800dbc0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 shutdown_connections 2023-07-13T22:31:42.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe3f006ace0 0x7fe3f006d190 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4041599c0 0x7fe4040cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4040ad810 0x7fe4040b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 --2- 172.21.15.60:0/96748916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4041595f0 0x7fe4041f5b50 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 >> 172.21.15.60:0/96748916 conn(0x7fe4040cf5e0 msgr2=0x7fe40406bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:42.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 shutdown_connections 2023-07-13T22:31:42.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.401+0000 7fe40a9ff700 1 -- 172.21.15.60:0/96748916 wait complete. 2023-07-13T22:31:42.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:31:42.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.626+0000 7f71581ff700 1 Processor -- start 2023-07-13T22:31:42.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.626+0000 7f71581ff700 1 -- start start 2023-07-13T22:31:42.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.627+0000 7f71581ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f715015ca10 0x7f715006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.627+0000 7f71581ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 0x7f715006a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.627+0000 7f71581ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71500ce720 0x7f71500ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.627+0000 7f71581ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715006a860 con 0x7f715015ca10 2023-07-13T22:31:42.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.627+0000 7f71581ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715006cc80 con 0x7f71500ce720 2023-07-13T22:31:42.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.627+0000 7f71581ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715015cde0 con 0x7f7150069eb0 2023-07-13T22:31:42.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f7155f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f715015ca10 0x7f715006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f7155f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f715015ca10 0x7f715006d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46004/0 (socket says 172.21.15.60:46004) 2023-07-13T22:31:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f7155f9b700 1 -- 172.21.15.60:0/2860097100 learned_addr learned my addr 172.21.15.60:0/2860097100 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f715679c700 1 --2- 172.21.15.60:0/2860097100 >> v2:172.21.15.105:3300/0 conn(0x7f71500ce720 0x7f71500ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f715579a700 1 --2- 172.21.15.60:0/2860097100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 0x7f715006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f7155f9b700 1 -- 172.21.15.60:0/2860097100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 msgr2=0x7f715006a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:42.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f7155f9b700 1 --2- 172.21.15.60:0/2860097100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 0x7f715006a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f7155f9b700 1 -- 172.21.15.60:0/2860097100 >> v2:172.21.15.105:3300/0 conn(0x7f71500ce720 msgr2=0x7f71500ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f7155f9b700 1 --2- 172.21.15.60:0/2860097100 >> v2:172.21.15.105:3300/0 conn(0x7f71500ce720 0x7f71500ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f7155f9b700 1 -- 172.21.15.60:0/2860097100 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500cf0c0 con 0x7f715015ca10 2023-07-13T22:31:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.628+0000 7f7155f9b700 1 --2- 172.21.15.60:0/2860097100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 0x7f715006d6d0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f71500cd550 tx=0x7f714c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38555511733348a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f7154f99700 1 -- 172.21.15.60:0/2860097100 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c011830 con 0x7f715015ca10 2023-07-13T22:31:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f7154f99700 1 -- 172.21.15.60:0/2860097100 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c011990 con 0x7f715015ca10 2023-07-13T22:31:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f7154f99700 1 -- 172.21.15.60:0/2860097100 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c011d20 con 0x7f715015ca10 2023-07-13T22:31:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f71581ff700 1 -- 172.21.15.60:0/2860097100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 msgr2=0x7f715006d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f71581ff700 1 --2- 172.21.15.60:0/2860097100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 0x7f715006d6d0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f71500cd550 tx=0x7f714c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f71581ff700 1 -- 172.21.15.60:0/2860097100 shutdown_connections 2023-07-13T22:31:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f71581ff700 1 --2- 172.21.15.60:0/2860097100 >> v2:172.21.15.105:3300/0 conn(0x7f71500ce720 0x7f71500ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f71581ff700 1 --2- 172.21.15.60:0/2860097100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 0x7f715006d6d0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f71581ff700 1 --2- 172.21.15.60:0/2860097100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 0x7f715006a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f71581ff700 1 -- 172.21.15.60:0/2860097100 >> 172.21.15.60:0/2860097100 conn(0x7f71500b15a0 msgr2=0x7f71500b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.629+0000 7f71581ff700 1 -- 172.21.15.60:0/2860097100 shutdown_connections 2023-07-13T22:31:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.630+0000 7f71581ff700 1 -- 172.21.15.60:0/2860097100 wait complete. 2023-07-13T22:31:42.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.632+0000 7f71581ff700 1 Processor -- start 2023-07-13T22:31:42.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.632+0000 7f71581ff700 1 -- start start 2023-07-13T22:31:42.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.633+0000 7f71581ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 0x7f71500ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.633+0000 7f71581ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71500ce720 0x7f71500af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.633+0000 7f71581ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 0x7f71501f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.633+0000 7f71581ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71500c7b30 con 0x7f715015ca10 2023-07-13T22:31:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.633+0000 7f71581ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71500c7df0 con 0x7f71500ce720 2023-07-13T22:31:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.633+0000 7f71581ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71500c80d0 con 0x7f7150069eb0 2023-07-13T22:31:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715679c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 0x7f71501f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715679c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 0x7f71501f2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46018/0 (socket says 172.21.15.60:46018) 2023-07-13T22:31:42.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715679c700 1 -- 172.21.15.60:0/1516226823 learned_addr learned my addr 172.21.15.60:0/1516226823 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f7155f9b700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 0x7f71500ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715579a700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71500ce720 0x7f71500af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715679c700 1 -- 172.21.15.60:0/1516226823 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 msgr2=0x7f71500ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715679c700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 0x7f71500ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715679c700 1 -- 172.21.15.60:0/1516226823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71500ce720 msgr2=0x7f71500af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715679c700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71500ce720 0x7f71500af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715679c700 1 -- 172.21.15.60:0/1516226823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c009550 con 0x7f715015ca10 2023-07-13T22:31:42.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.634+0000 7f715679c700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 0x7f71501f2000 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f714800efc0 tx=0x7f714800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:42.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.635+0000 7f71467fc700 1 -- 172.21.15.60:0/1516226823 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714800cd40 con 0x7f715015ca10 2023-07-13T22:31:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.635+0000 7f71467fc700 1 -- 172.21.15.60:0/1516226823 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714800f040 con 0x7f715015ca10 2023-07-13T22:31:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.635+0000 7f71467fc700 1 -- 172.21.15.60:0/1516226823 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71480146c0 con 0x7f715015ca10 2023-07-13T22:31:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.635+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71500c83b0 con 0x7f715015ca10 2023-07-13T22:31:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.635+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71500ae580 con 0x7f715015ca10 2023-07-13T22:31:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.636+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71500c88d0 con 0x7f715015ca10 2023-07-13T22:31:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.637+0000 7f71467fc700 1 -- 172.21.15.60:0/1516226823 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7148014820 con 0x7f715015ca10 2023-07-13T22:31:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.638+0000 7f71467fc700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f713406ace0 0x7f713406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.638+0000 7f71467fc700 1 -- 172.21.15.60:0/1516226823 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7148089da0 con 0x7f715015ca10 2023-07-13T22:31:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.638+0000 7f7155f9b700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f713406ace0 0x7f713406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:42.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.638+0000 7f7155f9b700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f713406ace0 0x7f713406d190 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f714c009c10 tx=0x7f714c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:42.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.642+0000 7f71467fc700 1 -- 172.21.15.60:0/1516226823 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f71480588d0 con 0x7f715015ca10 2023-07-13T22:31:42.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.845+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7150074220 con 0x7f715015ca10 2023-07-13T22:31:42.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.846+0000 7f71467fc700 1 -- 172.21.15.60:0/1516226823 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7148018020 con 0x7f715015ca10 2023-07-13T22:31:42.829 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:31:42.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f713406ace0 msgr2=0x7f713406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f713406ace0 0x7f713406d190 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f714c009c10 tx=0x7f714c008030 comp rx=0 tx=0).stop 2023-07-13T22:31:42.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 msgr2=0x7f71501f2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:42.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 0x7f71501f2000 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f714800efc0 tx=0x7f714800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 shutdown_connections 2023-07-13T22:31:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f713406ace0 0x7f713406d190 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f715015ca10 0x7f71501f2000 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7150069eb0 0x7f71500ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 --2- 172.21.15.60:0/1516226823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71500ce720 0x7f71500af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 >> 172.21.15.60:0/1516226823 conn(0x7f71500b15a0 msgr2=0x7f71500b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 shutdown_connections 2023-07-13T22:31:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:42.848+0000 7f71581ff700 1 -- 172.21.15.60:0/1516226823 wait complete. 2023-07-13T22:31:42.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:31:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.073+0000 7faccb1ff700 1 Processor -- start 2023-07-13T22:31:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.073+0000 7faccb1ff700 1 -- start start 2023-07-13T22:31:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.074+0000 7faccb1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40cefc0 0x7facc40b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.074+0000 7faccb1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7facc40b4ce0 0x7facc40d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.074+0000 7faccb1ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc40cf410 con 0x7facc40b8730 2023-07-13T22:31:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.074+0000 7faccb1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc40b5160 con 0x7facc40b4ce0 2023-07-13T22:31:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7faccb1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc40d06a0 con 0x7facc40cefc0 2023-07-13T22:31:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc8f9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7facc40b8730 0x7facc4160a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54138/0 (socket says 172.21.15.60:54138) 2023-07-13T22:31:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc8f9b700 1 -- 172.21.15.60:0/567294333 learned_addr learned my addr 172.21.15.60:0/567294333 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc979c700 1 --2- 172.21.15.60:0/567294333 >> v2:172.21.15.105:3300/0 conn(0x7facc40b4ce0 0x7facc40d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc3fff700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40cefc0 0x7facc40b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273745350 0 0) 0x7facc40cf410 con 0x7facc40b8730 2023-07-13T22:31:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb40035b0 con 0x7facc40b8730 2023-07-13T22:31:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1414741964 0 0) 0x7facb40035b0 con 0x7facc40b8730 2023-07-13T22:31:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40cefc0 msgr2=0x7facc40b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc37fe700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40cefc0 0x7facc40b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 >> v2:172.21.15.105:3300/0 conn(0x7facc40b4ce0 msgr2=0x7facc40d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc37fe700 1 --2- 172.21.15.60:0/567294333 >> v2:172.21.15.105:3300/0 conn(0x7facc40b4ce0 0x7facc40d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.075+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc40cd810 con 0x7facc40b8730 2023-07-13T22:31:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7facb8001670 con 0x7facc40b8730 2023-07-13T22:31:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 >> v1:172.21.15.60:6789/0 conn(0x7facc40b8730 legacy=0x7facc4160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc37fe700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facb4003a00 0x7facb4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc37fe700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4004430 0x7facb40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc37fe700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40cefc0 0x7facb4005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40b5160 con 0x7facb4004430 2023-07-13T22:31:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40d06a0 con 0x7facb4003a00 2023-07-13T22:31:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb400ba10 con 0x7facc40cefc0 2023-07-13T22:31:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc3fff700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4004430 0x7facb40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc8f9b700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facb4003a00 0x7facb4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc979c700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40cefc0 0x7facb4005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc3fff700 1 -- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40cefc0 msgr2=0x7facb4005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc3fff700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facc40cefc0 0x7facb4005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc3fff700 1 -- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facb4003a00 msgr2=0x7facb4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc3fff700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facb4003a00 0x7facb4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.076+0000 7facc3fff700 1 -- 172.21.15.60:0/567294333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb40035b0 con 0x7facb4004430 2023-07-13T22:31:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.077+0000 7facc3fff700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4004430 0x7facb40048f0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7facb0009b90 tx=0x7facb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd15ed5c20f5949 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.077+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb000e6c0 con 0x7facb4004430 2023-07-13T22:31:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.077+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb000e820 con 0x7facb4004430 2023-07-13T22:31:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.077+0000 7facc37fe700 1 -- 172.21.15.60:0/567294333 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb000eb90 con 0x7facb4004430 2023-07-13T22:31:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.077+0000 7faccb1ff700 1 -- 172.21.15.60:0/567294333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4004430 msgr2=0x7facb40048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.077+0000 7faccb1ff700 1 --2- 172.21.15.60:0/567294333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facb4004430 0x7facb40048f0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7facb0009b90 tx=0x7facb4005880 comp rx=0 tx=0).stop 2023-07-13T22:31:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.077+0000 7facc979c700 1 -- 172.21.15.60:0/567294333 reap_dead start 2023-07-13T22:31:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.077+0000 7faccb1ff700 1 -- 172.21.15.60:0/567294333 shutdown_connections 2023-07-13T22:31:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.077+0000 7faccb1ff700 1 -- 172.21.15.60:0/567294333 >> 172.21.15.60:0/567294333 conn(0x7facc406b1a0 msgr2=0x7facc40cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.078+0000 7faccb1ff700 1 -- 172.21.15.60:0/567294333 shutdown_connections 2023-07-13T22:31:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.078+0000 7faccb1ff700 1 -- 172.21.15.60:0/567294333 wait complete. 2023-07-13T22:31:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.079+0000 7faccb1ff700 1 Processor -- start 2023-07-13T22:31:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.080+0000 7faccb1ff700 1 -- start start 2023-07-13T22:31:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.080+0000 7faccb1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facb4004430 0x7facc40c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.080+0000 7faccb1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc41e2960 0x7facc415b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.080+0000 7faccb1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc41e3760 0x7facc41e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7faccb1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc41e4110 con 0x7facc41e2960 2023-07-13T22:31:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7faccb1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc41e5de0 con 0x7facc41e3760 2023-07-13T22:31:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7faccb1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c8200 con 0x7facb4004430 2023-07-13T22:31:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc41e2960 0x7facc415b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc3fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc41e2960 0x7facc415b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46040/0 (socket says 172.21.15.60:46040) 2023-07-13T22:31:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc3fff700 1 -- 172.21.15.60:0/3768126909 learned_addr learned my addr 172.21.15.60:0/3768126909 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc8f9b700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facb4004430 0x7facc40c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc979c700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc41e3760 0x7facc41e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc3fff700 1 -- 172.21.15.60:0/3768126909 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facb4004430 msgr2=0x7facc40c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc3fff700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facb4004430 0x7facc40c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc3fff700 1 -- 172.21.15.60:0/3768126909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc41e3760 msgr2=0x7facc41e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc3fff700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc41e3760 0x7facc41e3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc3fff700 1 -- 172.21.15.60:0/3768126909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc41e2dd0 con 0x7facc41e2960 2023-07-13T22:31:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.081+0000 7facc3fff700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc41e2960 0x7facc415b320 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7facb000e4b0 tx=0x7facb00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.082+0000 7facc17fa700 1 -- 172.21.15.60:0/3768126909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0021070 con 0x7facc41e2960 2023-07-13T22:31:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.082+0000 7facc17fa700 1 -- 172.21.15.60:0/3768126909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb00043e0 con 0x7facc41e2960 2023-07-13T22:31:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.082+0000 7facc17fa700 1 -- 172.21.15.60:0/3768126909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb000e6c0 con 0x7facc41e2960 2023-07-13T22:31:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.082+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc40c7960 con 0x7facc41e2960 2023-07-13T22:31:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.082+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc40c7f20 con 0x7facc41e2960 2023-07-13T22:31:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.083+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc41e5f90 con 0x7facc41e2960 2023-07-13T22:31:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.084+0000 7facc17fa700 1 -- 172.21.15.60:0/3768126909 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7facb0025070 con 0x7facc41e2960 2023-07-13T22:31:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.085+0000 7facc17fa700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.085+0000 7facc17fa700 1 -- 172.21.15.60:0/3768126909 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7facb008a530 con 0x7facc41e2960 2023-07-13T22:31:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.085+0000 7facc8f9b700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.086+0000 7facc8f9b700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7facb8001550 tx=0x7facb8007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.089+0000 7facc17fa700 1 -- 172.21.15.60:0/3768126909 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7facc41e5f90 con 0x7facc41e2960 2023-07-13T22:31:43.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.297+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7facc41e5f90 con 0x7facc41e2960 2023-07-13T22:31:43.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.298+0000 7facc17fa700 1 -- 172.21.15.60:0/3768126909 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v135) v1 ==== 104+0+11 (secure 0 0 0) 0x7facc41e5f90 con 0x7facc41e2960 2023-07-13T22:31:43.281 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:31:43.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 msgr2=0x7faca806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7facb8001550 tx=0x7facb8007980 comp rx=0 tx=0).stop 2023-07-13T22:31:43.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc41e2960 msgr2=0x7facc415b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc41e2960 0x7facc415b320 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7facb000e4b0 tx=0x7facb00047c0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 shutdown_connections 2023-07-13T22:31:43.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faca806ace0 0x7faca806d190 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facc41e3760 0x7facc41e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7facb4004430 0x7facc40c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 --2- 172.21.15.60:0/3768126909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facc41e2960 0x7facc415b320 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 >> 172.21.15.60:0/3768126909 conn(0x7facc406b1a0 msgr2=0x7facc40b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:43.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 shutdown_connections 2023-07-13T22:31:43.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.300+0000 7faccb1ff700 1 -- 172.21.15.60:0/3768126909 wait complete. 2023-07-13T22:31:43.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:31:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.526+0000 7fe7137ff700 1 Processor -- start 2023-07-13T22:31:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.527+0000 7fe7137ff700 1 -- start start 2023-07-13T22:31:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe7137ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe70c1595f0 0x7fe70c0af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe7137ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c0ad810 0x7fe70c0ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe7137ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c0c8fe0 con 0x7fe70c1599c0 2023-07-13T22:31:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe7137ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c0af6e0 con 0x7fe70c1595f0 2023-07-13T22:31:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe7137ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0c92c0 con 0x7fe70c0ad810 2023-07-13T22:31:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe710d9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe70c1599c0 0x7fe70c0ca080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54146/0 (socket says 172.21.15.60:54146) 2023-07-13T22:31:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe710d9a700 1 -- 172.21.15.60:0/925284905 learned_addr learned my addr 172.21.15.60:0/925284905 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe71159b700 1 --2- 172.21.15.60:0/925284905 >> v2:172.21.15.105:3300/0 conn(0x7fe70c1595f0 0x7fe70c0af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe711d9c700 1 --2- 172.21.15.60:0/925284905 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c0ad810 0x7fe70c0ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.528+0000 7fe703fff700 1 -- 172.21.15.60:0/925284905 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965609340 0 0) 0x7fe70c0c8fe0 con 0x7fe70c1599c0 2023-07-13T22:31:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe703fff700 1 -- 172.21.15.60:0/925284905 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6f80035b0 con 0x7fe70c1599c0 2023-07-13T22:31:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe71159b700 1 -- 172.21.15.60:0/925284905 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c0ad810 msgr2=0x7fe70c0ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe71159b700 1 --2- 172.21.15.60:0/925284905 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c0ad810 0x7fe70c0ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe71159b700 1 -- 172.21.15.60:0/925284905 >> v1:172.21.15.60:6789/0 conn(0x7fe70c1599c0 legacy=0x7fe70c0ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe71159b700 1 -- 172.21.15.60:0/925284905 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c0c95a0 con 0x7fe70c1595f0 2023-07-13T22:31:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe711d9c700 1 --2- 172.21.15.60:0/925284905 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c0ad810 0x7fe70c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:31:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe71159b700 1 --2- 172.21.15.60:0/925284905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70c1595f0 0x7fe70c0af1a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe708009840 tx=0x7fe708009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d46de7b1a8f42c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe703fff700 1 -- 172.21.15.60:0/925284905 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70800f8a0 con 0x7fe70c1595f0 2023-07-13T22:31:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe703fff700 1 -- 172.21.15.60:0/925284905 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70800fa00 con 0x7fe70c1595f0 2023-07-13T22:31:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.529+0000 7fe703fff700 1 -- 172.21.15.60:0/925284905 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70800fd90 con 0x7fe70c1595f0 2023-07-13T22:31:43.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.530+0000 7fe7137ff700 1 -- 172.21.15.60:0/925284905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70c1595f0 msgr2=0x7fe70c0af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.530+0000 7fe7137ff700 1 --2- 172.21.15.60:0/925284905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70c1595f0 0x7fe70c0af1a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe708009840 tx=0x7fe708009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:43.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.530+0000 7fe7137ff700 1 -- 172.21.15.60:0/925284905 shutdown_connections 2023-07-13T22:31:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.530+0000 7fe7137ff700 1 --2- 172.21.15.60:0/925284905 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c0ad810 0x7fe70c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.530+0000 7fe7137ff700 1 --2- 172.21.15.60:0/925284905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70c1595f0 0x7fe70c0af1a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.530+0000 7fe7137ff700 1 -- 172.21.15.60:0/925284905 >> 172.21.15.60:0/925284905 conn(0x7fe70c0cf5e0 msgr2=0x7fe70c0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.530+0000 7fe7137ff700 1 -- 172.21.15.60:0/925284905 shutdown_connections 2023-07-13T22:31:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.530+0000 7fe7137ff700 1 -- 172.21.15.60:0/925284905 wait complete. 2023-07-13T22:31:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.532+0000 7fe7137ff700 1 Processor -- start 2023-07-13T22:31:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.532+0000 7fe7137ff700 1 -- start start 2023-07-13T22:31:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe7137ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70c0ad810 0x7fe70c0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe7137ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c1595f0 0x7fe70c1e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe7137ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe70c1599c0 0x7fe70c1e6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe7137ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0b2360 con 0x7fe70c1599c0 2023-07-13T22:31:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe7137ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c1e6a60 con 0x7fe70c0ad810 2023-07-13T22:31:43.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe7137ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0b19c0 con 0x7fe70c1595f0 2023-07-13T22:31:43.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe711d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe70c1599c0 0x7fe70c1e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe711d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe70c1599c0 0x7fe70c1e6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46050/0 (socket says 172.21.15.60:46050) 2023-07-13T22:31:43.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe711d9c700 1 -- 172.21.15.60:0/4170105142 learned_addr learned my addr 172.21.15.60:0/4170105142 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:43.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe710d9a700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c1595f0 0x7fe70c1e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe71159b700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70c0ad810 0x7fe70c0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe711d9c700 1 -- 172.21.15.60:0/4170105142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c1595f0 msgr2=0x7fe70c1e5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.533+0000 7fe711d9c700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c1595f0 0x7fe70c1e5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.534+0000 7fe711d9c700 1 -- 172.21.15.60:0/4170105142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70c0ad810 msgr2=0x7fe70c0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.534+0000 7fe711d9c700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70c0ad810 0x7fe70c0b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.534+0000 7fe711d9c700 1 -- 172.21.15.60:0/4170105142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704009a70 con 0x7fe70c1599c0 2023-07-13T22:31:43.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.534+0000 7fe711d9c700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe70c1599c0 0x7fe70c1e6450 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe70400efc0 tx=0x7fe70400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:43.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.534+0000 7fe701ffb700 1 -- 172.21.15.60:0/4170105142 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70400ca90 con 0x7fe70c1599c0 2023-07-13T22:31:43.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.534+0000 7fe701ffb700 1 -- 172.21.15.60:0/4170105142 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70400f040 con 0x7fe70c1599c0 2023-07-13T22:31:43.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.534+0000 7fe701ffb700 1 -- 172.21.15.60:0/4170105142 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70400ce10 con 0x7fe70c1599c0 2023-07-13T22:31:43.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.534+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7080094a0 con 0x7fe70c1599c0 2023-07-13T22:31:43.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.534+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c0b1f80 con 0x7fe70c1599c0 2023-07-13T22:31:43.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.535+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70c0b24a0 con 0x7fe70c1599c0 2023-07-13T22:31:43.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.536+0000 7fe701ffb700 1 -- 172.21.15.60:0/4170105142 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe704014480 con 0x7fe70c1599c0 2023-07-13T22:31:43.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.537+0000 7fe701ffb700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6f006ace0 0x7fe6f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.537+0000 7fe701ffb700 1 -- 172.21.15.60:0/4170105142 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe704089b00 con 0x7fe70c1599c0 2023-07-13T22:31:43.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.537+0000 7fe71159b700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6f006ace0 0x7fe6f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.538+0000 7fe71159b700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6f006ace0 0x7fe6f006d190 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe708000c00 tx=0x7fe70801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:43.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.541+0000 7fe701ffb700 1 -- 172.21.15.60:0/4170105142 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe70c0b24a0 con 0x7fe70c1599c0 2023-07-13T22:31:43.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.752+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe70c0b24a0 con 0x7fe70c1599c0 2023-07-13T22:31:43.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.752+0000 7fe701ffb700 1 -- 172.21.15.60:0/4170105142 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v135) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe70c0b24a0 con 0x7fe70c1599c0 2023-07-13T22:31:43.736 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:31:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6f006ace0 msgr2=0x7fe6f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6f006ace0 0x7fe6f006d190 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe708000c00 tx=0x7fe70801b040 comp rx=0 tx=0).stop 2023-07-13T22:31:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe70c1599c0 msgr2=0x7fe70c1e6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe70c1599c0 0x7fe70c1e6450 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe70400efc0 tx=0x7fe70400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 shutdown_connections 2023-07-13T22:31:43.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe6f006ace0 0x7fe6f006d190 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe70c1599c0 0x7fe70c1e6450 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70c0ad810 0x7fe70c0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 --2- 172.21.15.60:0/4170105142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe70c1595f0 0x7fe70c1e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 >> 172.21.15.60:0/4170105142 conn(0x7fe70c0cf5e0 msgr2=0x7fe70c06fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:43.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 shutdown_connections 2023-07-13T22:31:43.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.755+0000 7fe7137ff700 1 -- 172.21.15.60:0/4170105142 wait complete. 2023-07-13T22:31:43.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:31:43.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.977+0000 7f84f6bff700 1 Processor -- start 2023-07-13T22:31:43.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.977+0000 7f84f6bff700 1 -- start start 2023-07-13T22:31:43.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.978+0000 7f84f6bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84f015c6f0 0x7f84f0160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.978+0000 7f84f6bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00722e0 con 0x7f84f00cf010 2023-07-13T22:31:43.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84f6bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00d0df0 con 0x7f84f00cfdd0 2023-07-13T22:31:43.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84f6bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f00cdbd0 con 0x7f84f015c6f0 2023-07-13T22:31:43.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84effff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f84f00cf010 0x7f84f00cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54154/0 (socket says 172.21.15.60:54154) 2023-07-13T22:31:43.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84effff700 1 -- 172.21.15.60:0/1003933253 learned_addr learned my addr 172.21.15.60:0/1003933253 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:43.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84f499b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84f015c6f0 0x7f84f0160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84f499b700 1 -- 172.21.15.60:0/1003933253 >> v1:172.21.15.105:6789/0 conn(0x7f84f00cfdd0 legacy=0x7f84f00d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:43.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84f499b700 1 -- 172.21.15.60:0/1003933253 >> v1:172.21.15.60:6789/0 conn(0x7f84f00cf010 legacy=0x7f84f00cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84f499b700 1 -- 172.21.15.60:0/1003933253 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f015cac0 con 0x7f84f015c6f0 2023-07-13T22:31:43.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84ef7fe700 1 -- 172.21.15.60:0/1003933253 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870620893 0 0) 0x7f84dc001110 con 0x7f84f00cf010 2023-07-13T22:31:43.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.979+0000 7f84f499b700 1 --2- 172.21.15.60:0/1003933253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f015c6f0 0x7f84f0160b70 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009840 tx=0x7f84e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150c8d5c81f7897b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:43.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84ef7fe700 1 -- 172.21.15.60:0/1003933253 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4011830 con 0x7f84f015c6f0 2023-07-13T22:31:43.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84ef7fe700 1 -- 172.21.15.60:0/1003933253 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e4011990 con 0x7f84f015c6f0 2023-07-13T22:31:43.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84ef7fe700 1 -- 172.21.15.60:0/1003933253 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4011d20 con 0x7f84f015c6f0 2023-07-13T22:31:43.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84f6bff700 1 -- 172.21.15.60:0/1003933253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f015c6f0 msgr2=0x7f84f0160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84f6bff700 1 --2- 172.21.15.60:0/1003933253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f015c6f0 0x7f84f0160b70 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009840 tx=0x7f84e4009b50 comp rx=0 tx=0).stop 2023-07-13T22:31:43.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84f6bff700 1 -- 172.21.15.60:0/1003933253 shutdown_connections 2023-07-13T22:31:43.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84f6bff700 1 --2- 172.21.15.60:0/1003933253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f015c6f0 0x7f84f0160b70 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84f6bff700 1 -- 172.21.15.60:0/1003933253 >> 172.21.15.60:0/1003933253 conn(0x7f84f006a4b0 msgr2=0x7f84f006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:43.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84f6bff700 1 -- 172.21.15.60:0/1003933253 shutdown_connections 2023-07-13T22:31:43.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.980+0000 7f84f6bff700 1 -- 172.21.15.60:0/1003933253 wait complete. 2023-07-13T22:31:43.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.982+0000 7f84f6bff700 1 Processor -- start 2023-07-13T22:31:43.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.982+0000 7f84f6bff700 1 -- start start 2023-07-13T22:31:43.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.983+0000 7f84f6bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f00cf010 0x7f84f00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.983+0000 7f84f6bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f00cfdd0 0x7f84f01e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.983+0000 7f84f6bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f84f015c6f0 0x7f84f01f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.983+0000 7f84f6bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01e7f30 con 0x7f84f00cfdd0 2023-07-13T22:31:43.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.983+0000 7f84f6bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01e8a00 con 0x7f84f00cf010 2023-07-13T22:31:43.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.983+0000 7f84f6bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01e8160 con 0x7f84f015c6f0 2023-07-13T22:31:43.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.983+0000 7f84effff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f00cfdd0 0x7f84f01e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84effff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f00cfdd0 0x7f84f01e79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46072/0 (socket says 172.21.15.60:46072) 2023-07-13T22:31:43.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84effff700 1 -- 172.21.15.60:0/2308233160 learned_addr learned my addr 172.21.15.60:0/2308233160 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:43.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84f499b700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f00cf010 0x7f84f00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84f519c700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f84f015c6f0 0x7f84f01f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84effff700 1 -- 172.21.15.60:0/2308233160 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f84f015c6f0 msgr2=0x7f84f01f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84effff700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f84f015c6f0 0x7f84f01f5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84effff700 1 -- 172.21.15.60:0/2308233160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f00cf010 msgr2=0x7f84f00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:43.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84effff700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f00cf010 0x7f84f00cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:43.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84effff700 1 -- 172.21.15.60:0/2308233160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40094a0 con 0x7f84f00cfdd0 2023-07-13T22:31:43.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84effff700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f00cfdd0 0x7f84f01e79f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00c520 tx=0x7f84dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:43.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84ed7fa700 1 -- 172.21.15.60:0/2308233160 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004c80 con 0x7f84f00cfdd0 2023-07-13T22:31:43.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84ed7fa700 1 -- 172.21.15.60:0/2308233160 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc004e00 con 0x7f84f00cfdd0 2023-07-13T22:31:43.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.984+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f01e8440 con 0x7f84f00cfdd0 2023-07-13T22:31:43.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.985+0000 7f84ed7fa700 1 -- 172.21.15.60:0/2308233160 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc007310 con 0x7f84f00cfdd0 2023-07-13T22:31:43.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.985+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f00b2870 con 0x7f84f00cfdd0 2023-07-13T22:31:43.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.986+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f00b2a90 con 0x7f84f00cfdd0 2023-07-13T22:31:43.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.987+0000 7f84ed7fa700 1 -- 172.21.15.60:0/2308233160 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f84dc007470 con 0x7f84f00cfdd0 2023-07-13T22:31:43.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.988+0000 7f84ed7fa700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f84d406ace0 0x7f84d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:43.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.988+0000 7f84ed7fa700 1 -- 172.21.15.60:0/2308233160 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f84dc088fe0 con 0x7f84f00cfdd0 2023-07-13T22:31:43.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.988+0000 7f84f499b700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f84d406ace0 0x7f84d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:43.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.989+0000 7f84f499b700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f84d406ace0 0x7f84d406d190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000c00 tx=0x7f84e401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:43.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:43.992+0000 7f84ed7fa700 1 -- 172.21.15.60:0/2308233160 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f84dc0540a0 con 0x7f84f00cfdd0 2023-07-13T22:31:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.212+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f84f00722e0 con 0x7f84f00cfdd0 2023-07-13T22:31:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.213+0000 7f84ed7fa700 1 -- 172.21.15.60:0/2308233160 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v135) v1 ==== 80+0+0 (secure 0 0 0) 0x7f84f00722e0 con 0x7f84f00cfdd0 2023-07-13T22:31:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f84d406ace0 msgr2=0x7f84d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f84d406ace0 0x7f84d406d190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000c00 tx=0x7f84e401c040 comp rx=0 tx=0).stop 2023-07-13T22:31:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f00cfdd0 msgr2=0x7f84f01e79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f00cfdd0 0x7f84f01e79f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00c520 tx=0x7f84dc00c830 comp rx=0 tx=0).stop 2023-07-13T22:31:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 shutdown_connections 2023-07-13T22:31:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f84d406ace0 0x7f84d406d190 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f84f015c6f0 0x7f84f01f5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f00cf010 0x7f84f00cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 --2- 172.21.15.60:0/2308233160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f00cfdd0 0x7f84f01e79f0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 >> 172.21.15.60:0/2308233160 conn(0x7f84f006a4b0 msgr2=0x7f84f0156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 shutdown_connections 2023-07-13T22:31:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.215+0000 7f84f6bff700 1 -- 172.21.15.60:0/2308233160 wait complete. 2023-07-13T22:31:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:31:44.215 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:31:44.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:44.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:44.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[166287]: fuse finished with error 0 and tester_r 0 2023-07-13T22:31:44.288 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:31:44.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:44.288 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:31:44.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-13T22:31:44.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.710+0000 7f7655fff700 1 Processor -- start 2023-07-13T22:31:44.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.710+0000 7f7655fff700 1 -- start start 2023-07-13T22:31:44.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.711+0000 7f7655fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76500adc90 0x7f76500b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:44.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.711+0000 7f7655fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765015e020 con 0x7f76500b08a0 2023-07-13T22:31:44.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.711+0000 7f7655fff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765015e270 con 0x7f76500ca8b0 2023-07-13T22:31:44.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.711+0000 7f7655fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76500b0d10 con 0x7f76500adc90 2023-07-13T22:31:44.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.711+0000 7f764f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76500adc90 0x7f76500b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:44.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.711+0000 7f764f7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76500adc90 0x7f76500b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46082/0 (socket says 172.21.15.60:46082) 2023-07-13T22:31:44.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.711+0000 7f764f7fe700 1 -- 172.21.15.60:0/1141363434 learned_addr learned my addr 172.21.15.60:0/1141363434 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:44.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.711+0000 7f764effd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f76500b08a0 0x7f76500ca1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54156/0 (socket says 172.21.15.60:54156) 2023-07-13T22:31:44.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.712+0000 7f764f7fe700 1 -- 172.21.15.60:0/1141363434 >> v1:172.21.15.105:6789/0 conn(0x7f76500ca8b0 legacy=0x7f765006b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:44.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.712+0000 7f764f7fe700 1 -- 172.21.15.60:0/1141363434 >> v1:172.21.15.60:6789/0 conn(0x7f76500b08a0 legacy=0x7f76500ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:44.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.712+0000 7f764f7fe700 1 -- 172.21.15.60:0/1141363434 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765015e550 con 0x7f76500adc90 2023-07-13T22:31:44.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.712+0000 7f764f7fe700 1 --2- 172.21.15.60:0/1141363434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500adc90 0x7f76500b0360 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f76400098f0 tx=0x7f7640009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5909efc1d290f602 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:44.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.712+0000 7f764e7fc700 1 -- 172.21.15.60:0/1141363434 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000b8a0 con 0x7f76500adc90 2023-07-13T22:31:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.712+0000 7f764e7fc700 1 -- 172.21.15.60:0/1141363434 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764000ba00 con 0x7f76500adc90 2023-07-13T22:31:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.712+0000 7f764e7fc700 1 -- 172.21.15.60:0/1141363434 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000bd90 con 0x7f76500adc90 2023-07-13T22:31:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.713+0000 7f7655fff700 1 -- 172.21.15.60:0/1141363434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500adc90 msgr2=0x7f76500b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.713+0000 7f7655fff700 1 --2- 172.21.15.60:0/1141363434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500adc90 0x7f76500b0360 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f76400098f0 tx=0x7f7640009c00 comp rx=0 tx=0).stop 2023-07-13T22:31:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.713+0000 7f7655fff700 1 -- 172.21.15.60:0/1141363434 shutdown_connections 2023-07-13T22:31:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.713+0000 7f7655fff700 1 --2- 172.21.15.60:0/1141363434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500adc90 0x7f76500b0360 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.713+0000 7f7655fff700 1 -- 172.21.15.60:0/1141363434 >> 172.21.15.60:0/1141363434 conn(0x7f76500cb360 msgr2=0x7f76500cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.713+0000 7f7655fff700 1 -- 172.21.15.60:0/1141363434 shutdown_connections 2023-07-13T22:31:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.713+0000 7f7655fff700 1 -- 172.21.15.60:0/1141363434 wait complete. 2023-07-13T22:31:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.715+0000 7f7655fff700 1 Processor -- start 2023-07-13T22:31:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.715+0000 7f7655fff700 1 -- start start 2023-07-13T22:31:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f7655fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76500adc90 0x7f76500c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f7655fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500b08a0 0x7f76500c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f7655fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76500ca8b0 0x7f76500b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f7655fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500cde60 con 0x7f76500b08a0 2023-07-13T22:31:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f7655fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500cd270 con 0x7f76500adc90 2023-07-13T22:31:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f7655fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500cd4c0 con 0x7f76500ca8b0 2023-07-13T22:31:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f764effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500b08a0 0x7f76500c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f764effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500b08a0 0x7f76500c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46092/0 (socket says 172.21.15.60:46092) 2023-07-13T22:31:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f764effd700 1 -- 172.21.15.60:0/2099253314 learned_addr learned my addr 172.21.15.60:0/2099253314 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.716+0000 7f764f7fe700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76500adc90 0x7f76500c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f764ffff700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76500ca8b0 0x7f76500b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f764effd700 1 -- 172.21.15.60:0/2099253314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76500ca8b0 msgr2=0x7f76500b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f764effd700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76500ca8b0 0x7f76500b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f764effd700 1 -- 172.21.15.60:0/2099253314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76500adc90 msgr2=0x7f76500c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f764effd700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76500adc90 0x7f76500c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f764effd700 1 -- 172.21.15.60:0/2099253314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7640009550 con 0x7f76500b08a0 2023-07-13T22:31:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f764f7fe700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76500adc90 0x7f76500c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f764effd700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500b08a0 0x7f76500c7f00 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f764400c520 tx=0x7f764400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f7637fff700 1 -- 172.21.15.60:0/2099253314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644005cb0 con 0x7f76500b08a0 2023-07-13T22:31:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f7637fff700 1 -- 172.21.15.60:0/2099253314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644005e10 con 0x7f76500b08a0 2023-07-13T22:31:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f7637fff700 1 -- 172.21.15.60:0/2099253314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76440072d0 con 0x7f76500b08a0 2023-07-13T22:31:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76500cd7a0 con 0x7f76500b08a0 2023-07-13T22:31:44.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.717+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76500ce280 con 0x7f76500b08a0 2023-07-13T22:31:44.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.718+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7650074220 con 0x7f76500b08a0 2023-07-13T22:31:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.719+0000 7f7637fff700 1 -- 172.21.15.60:0/2099253314 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7644007430 con 0x7f76500b08a0 2023-07-13T22:31:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.720+0000 7f7637fff700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763c06ace0 0x7f763c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.720+0000 7f7637fff700 1 -- 172.21.15.60:0/2099253314 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76440888e0 con 0x7f76500b08a0 2023-07-13T22:31:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.721+0000 7f764f7fe700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763c06ace0 0x7f763c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.721+0000 7f764f7fe700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763c06ace0 0x7f763c06d190 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7640009c40 tx=0x7f7640008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.724+0000 7f7637fff700 1 -- 172.21.15.60:0/2099253314 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7644053a30 con 0x7f76500b08a0 2023-07-13T22:31:44.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.979+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f76500cbf40 con 0x7f76500b08a0 2023-07-13T22:31:44.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.979+0000 7f7637fff700 1 -- 172.21.15.60:0/2099253314 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7644017070 con 0x7f76500b08a0 2023-07-13T22:31:44.962 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:31:44.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.981+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763c06ace0 msgr2=0x7f763c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:44.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.981+0000 7f7655fff700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763c06ace0 0x7f763c06d190 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7640009c40 tx=0x7f7640008030 comp rx=0 tx=0).stop 2023-07-13T22:31:44.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.981+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500b08a0 msgr2=0x7f76500c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:44.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.981+0000 7f7655fff700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500b08a0 0x7f76500c7f00 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f764400c520 tx=0x7f764400c8e0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.982+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 shutdown_connections 2023-07-13T22:31:44.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.982+0000 7f7655fff700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f763c06ace0 0x7f763c06d190 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.982+0000 7f7655fff700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76500ca8b0 0x7f76500b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.982+0000 7f7655fff700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76500adc90 0x7f76500c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.982+0000 7f7655fff700 1 --2- 172.21.15.60:0/2099253314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76500b08a0 0x7f76500c7f00 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:44.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.982+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 >> 172.21.15.60:0/2099253314 conn(0x7f76500cb360 msgr2=0x7f765015c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:44.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.982+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 shutdown_connections 2023-07-13T22:31:44.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:44.982+0000 7f7655fff700 1 -- 172.21.15.60:0/2099253314 wait complete. 2023-07-13T22:31:44.983 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:31:44.983 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:31:44.983 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-07-13T22:31:44.984 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:31:44.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:44.984 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:31:44.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:31:44.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:31:44.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:31:44.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:44.986 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-07-13T22:31:45.033 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:31:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:31:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:31:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-07-13T22:31:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:45.034 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:31:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-07-13T22:31:45.034 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:31:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-07-13T22:31:45.035 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:31:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-07-13T22:31:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:31:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41068sec preferred_lft 41068sec 2023-07-13T22:31:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2023-07-13T22:31:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:45.036 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:31:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-07-13T22:31:45.036 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-13T22:31:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:31:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-13T22:31:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5882:2dff:febc:f42e/64 scope link 2023-07-13T22:31:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:45.037 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:31:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 4e:2f:5b:5c:0d:90 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:31:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4c2f:5bff:fe5c:d90/64 scope link 2023-07-13T22:31:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:45.037 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-13T22:31:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 3e:33:ff:7d:fb:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:31:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::3c33:ffff:fe7d:fbc6/64 scope link 2023-07-13T22:31:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:31:45.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:45.039 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:31:45.108 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-07-13T22:31:45.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:45.111 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-07-13T22:31:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-07-13T22:31:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:31:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:31:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:31:45.162 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:31:45.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:45.162 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-07-13T22:31:45.163 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-07-13T22:31:45.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:31:45.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:31:45.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:31:45.164 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-07-13T22:31:45.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:45.337 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:31:45.384 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:31:45.384 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:31:45.385 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:31:45.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:45.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-07-13T22:31:45.457 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-13T22:31:45.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:31:45.458 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:45.505 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:31:45.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:31:45.506 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:45.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:31:45.665 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:31:45.666 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:31:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:31:45.738 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:31:45.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:45.739 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:31:45.785 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-13T22:31:45.786 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-13T22:31:45.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-07-13T22:31:45.906 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-13T22:31:45.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-13T22:31:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-13T22:31:45.980 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-13T22:31:45.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:45.981 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-07-13T22:31:46.008 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-07-13T22:31:46.025+0000 7f4e9dbcf5c0 -1 init, newargv = 0x557195876010 newargc=15 2023-07-13T22:31:46.008 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[167291]: starting ceph client 2023-07-13T22:31:46.015 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[167291]: starting fuse 2023-07-13T22:31:46.035 INFO:teuthology.orchestra.run.smithi060.stdout:43 2023-07-13T22:31:46.035 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-13T22:31:46.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:31:46.036 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:31:46.037 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:31:46.037 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:31:46.037 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:31:46.038 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:46.038 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:31:46.039 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:31:46.039 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:31:46.039 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:31:46.040 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:46.041 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:31:46.041 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:31:46.041 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:31:46.042 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:31:46.042 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:46.042 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:31:46.043 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:31:46.043 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:31:46.043 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:31:46.044 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:31:46.044 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:31:46.045 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:31:46.045 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:31:46.046 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:46.046 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:31:46.046 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:31:46.223 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.167291.asok 2023-07-13T22:31:46.400 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.167291.asok 2023-07-13T22:31:46.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:46.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.167291.asok status 2023-07-13T22:31:46.634 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:31:46.635 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-07-13T22:31:46.635 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-13T22:31:46.635 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:31:46.635 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-07-13T22:31:46.635 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-07-13T22:31:46.635 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-13T22:31:46.636 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "167291", 2023-07-13T22:31:46.636 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-07-13T22:31:46.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:46.636 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-07-13T22:31:46.636 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-07-13T22:31:46.636 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9663, 2023-07-13T22:31:46.636 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:31:46.636 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:31:46.637 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:31:46.637 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 9663 2023-07-13T22:31:46.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:46.637 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:31:46.637 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:31:46.637 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:31:46.637 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 761222286 2023-07-13T22:31:46.638 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:46.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:46.638 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:31:46.638 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:31:46.638 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:31:46.638 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 761222286 2023-07-13T22:31:46.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:46.638 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.9663 192.168.0.1:0/761222286", 2023-07-13T22:31:46.639 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/761222286", 2023-07-13T22:31:46.639 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-07-13T22:31:46.639 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 97, 2023-07-13T22:31:46.639 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 136, 2023-07-13T22:31:46.639 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-07-13T22:31:46.639 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-07-13T22:31:46.639 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-07-13T22:31:46.639 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:31:46.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:46.649 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-07-13T22:31:46.649 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:46.696 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-07-13T22:31:46.696 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:46.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:46.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:31:46.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:31:46.807 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-07-13T22:31:46.807 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-07-13T22:31:46.807 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-07-13T22:31:46.808 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-07-13T22:31:46.808 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:46.809 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-07-13T22:31:46.809 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-13T22:31:46.809 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-07-13T22:31:46.810 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-13T22:31:46.810 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:46.810 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-07-13T22:31:46.810 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-07-13T22:31:46.811 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-07-13T22:31:46.811 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-07-13T22:31:46.811 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:46.812 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-07-13T22:31:46.812 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-13T22:31:46.812 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-13T22:31:46.813 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-13T22:31:46.813 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-07-13T22:31:46.813 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-07-13T22:31:46.813 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-07-13T22:31:46.814 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-13T22:31:46.814 DEBUG:teuthology.orchestra.run.smithi060:> 2023-07-13T22:31:46.814 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-07-13T22:31:46.815 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-07-13T22:31:46.961 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.167291.asok 2023-07-13T22:31:47.139 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.167291.asok 2023-07-13T22:31:47.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:47.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.167291.asok mds_sessions 2023-07-13T22:31:47.372 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:31:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9663, 2023-07-13T22:31:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-07-13T22:31:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-07-13T22:31:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-07-13T22:31:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 9663 2023-07-13T22:31:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-07-13T22:31:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:31:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-07-13T22:31:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 761222286 2023-07-13T22:31:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.9663 v1:192.168.0.1:0/761222286", 2023-07-13T22:31:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "v1:192.168.0.1:0/761222286", 2023-07-13T22:31:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions": [ 2023-07-13T22:31:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": 0, 2023-07-13T22:31:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "addrs": { 2023-07-13T22:31:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2023-07-13T22:31:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2023-07-13T22:31:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6835", 2023-07-13T22:31:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4061691799 2023-07-13T22:31:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-07-13T22:31:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-07-13T22:31:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6837", 2023-07-13T22:31:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4061691799 2023-07-13T22:31:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-07-13T22:31:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-07-13T22:31:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "seq": 0, 2023-07-13T22:31:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_gen": 0, 2023-07-13T22:31:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_ttl": "2023-07-13T22:32:47.029473+0000", 2023-07-13T22:31:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "last_cap_renew_request": "2023-07-13T22:31:47.029473+0000", 2023-07-13T22:31:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_renew_seq": 2, 2023-07-13T22:31:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "num_caps": 1, 2023-07-13T22:31:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "open" 2023-07-13T22:31:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-07-13T22:31:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2023-07-13T22:31:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 97 2023-07-13T22:31:47.378 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:31:47.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:31:47.386 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch file1) 2023-07-13T22:31:47.438 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:31:47.439 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:31:47.490 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-13T22:31:47.491 DEBUG:teuthology.orchestra.run.smithi105:> import os 2023-07-13T22:31:47.491 DEBUG:teuthology.orchestra.run.smithi105:> import time 2023-07-13T22:31:47.491 DEBUG:teuthology.orchestra.run.smithi105:> 2023-07-13T22:31:47.491 DEBUG:teuthology.orchestra.run.smithi105:> fd = os.open("/home/ubuntu/cephtest/mnt.1/file1", os.O_RDWR | os.O_CREAT, 0o644) 2023-07-13T22:31:47.492 DEBUG:teuthology.orchestra.run.smithi105:> try: 2023-07-13T22:31:47.492 DEBUG:teuthology.orchestra.run.smithi105:> while True: 2023-07-13T22:31:47.492 DEBUG:teuthology.orchestra.run.smithi105:> os.write(fd, b'"'"'content'"'"') 2023-07-13T22:31:47.493 DEBUG:teuthology.orchestra.run.smithi105:> time.sleep(1) 2023-07-13T22:31:47.493 DEBUG:teuthology.orchestra.run.smithi105:> if not False: 2023-07-13T22:31:47.494 DEBUG:teuthology.orchestra.run.smithi105:> break 2023-07-13T22:31:47.494 DEBUG:teuthology.orchestra.run.smithi105:> except IOError as e: 2023-07-13T22:31:47.495 DEBUG:teuthology.orchestra.run.smithi105:> pass 2023-07-13T22:31:47.496 DEBUG:teuthology.orchestra.run.smithi105:> os.close(fd) 2023-07-13T22:31:47.496 DEBUG:teuthology.orchestra.run.smithi105:> ' 2023-07-13T22:31:47.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:31:47.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.724+0000 7fc7447ff700 1 Processor -- start 2023-07-13T22:31:47.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.725+0000 7fc7447ff700 1 -- start start 2023-07-13T22:31:47.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.726+0000 7fc7447ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0ca8b0 0x7fc73c1653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:47.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.726+0000 7fc7447ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73c15e020 con 0x7fc73c0adc90 2023-07-13T22:31:47.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.726+0000 7fc7447ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73c06c760 con 0x7fc73c0b0a70 2023-07-13T22:31:47.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.726+0000 7fc7447ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c15e270 con 0x7fc73c0ca8b0 2023-07-13T22:31:47.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.726+0000 7fc74259b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc73c0adc90 0x7fc73c0b0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35346/0 (socket says 172.21.15.60:35346) 2023-07-13T22:31:47.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.726+0000 7fc74259b700 1 -- 172.21.15.60:0/2704497783 learned_addr learned my addr 172.21.15.60:0/2704497783 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:47.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.726+0000 7fc742d9c700 1 --2- 172.21.15.60:0/2704497783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0ca8b0 0x7fc73c1653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:47.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.726+0000 7fc741599700 1 -- 172.21.15.60:0/2704497783 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151337436 0 0) 0x7fc73c15e020 con 0x7fc73c0adc90 2023-07-13T22:31:47.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc741599700 1 -- 172.21.15.60:0/2704497783 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7280035b0 con 0x7fc73c0adc90 2023-07-13T22:31:47.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc741599700 1 -- 172.21.15.60:0/2704497783 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062902888 0 0) 0x7fc73c06c760 con 0x7fc73c0b0a70 2023-07-13T22:31:47.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc741599700 1 -- 172.21.15.60:0/2704497783 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc73c15e020 con 0x7fc73c0b0a70 2023-07-13T22:31:47.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc742d9c700 1 -- 172.21.15.60:0/2704497783 >> v1:172.21.15.105:6789/0 conn(0x7fc73c0b0a70 legacy=0x7fc73c0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:47.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc742d9c700 1 -- 172.21.15.60:0/2704497783 >> v1:172.21.15.60:6789/0 conn(0x7fc73c0adc90 legacy=0x7fc73c0b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:47.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc742d9c700 1 -- 172.21.15.60:0/2704497783 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c15e550 con 0x7fc73c0ca8b0 2023-07-13T22:31:47.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc742d9c700 1 --2- 172.21.15.60:0/2704497783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0ca8b0 0x7fc73c1653b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc734009dc0 tx=0x7fc73400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb10c9c6ea48a81 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:47.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc741599700 1 -- 172.21.15.60:0/2704497783 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73400e070 con 0x7fc73c0ca8b0 2023-07-13T22:31:47.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc741599700 1 -- 172.21.15.60:0/2704497783 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73400bb20 con 0x7fc73c0ca8b0 2023-07-13T22:31:47.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.727+0000 7fc741599700 1 -- 172.21.15.60:0/2704497783 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734012430 con 0x7fc73c0ca8b0 2023-07-13T22:31:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.728+0000 7fc7447ff700 1 -- 172.21.15.60:0/2704497783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0ca8b0 msgr2=0x7fc73c1653b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.728+0000 7fc7447ff700 1 --2- 172.21.15.60:0/2704497783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0ca8b0 0x7fc73c1653b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc734009dc0 tx=0x7fc73400b330 comp rx=0 tx=0).stop 2023-07-13T22:31:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.728+0000 7fc7447ff700 1 -- 172.21.15.60:0/2704497783 shutdown_connections 2023-07-13T22:31:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.728+0000 7fc7447ff700 1 --2- 172.21.15.60:0/2704497783 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0ca8b0 0x7fc73c1653b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.728+0000 7fc7447ff700 1 -- 172.21.15.60:0/2704497783 >> 172.21.15.60:0/2704497783 conn(0x7fc73c0cb360 msgr2=0x7fc73c0cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.728+0000 7fc7447ff700 1 -- 172.21.15.60:0/2704497783 shutdown_connections 2023-07-13T22:31:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.728+0000 7fc7447ff700 1 -- 172.21.15.60:0/2704497783 wait complete. 2023-07-13T22:31:47.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.730+0000 7fc7447ff700 1 Processor -- start 2023-07-13T22:31:47.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.731+0000 7fc7447ff700 1 -- start start 2023-07-13T22:31:47.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc7447ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c0adc90 0x7fc73c14fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:47.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc7447ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0b0a70 0x7fc73c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:47.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc7447ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0ca8b0 0x7fc73c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:47.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc7447ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0cdf60 con 0x7fc73c0adc90 2023-07-13T22:31:47.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc7447ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0c4720 con 0x7fc73c0ca8b0 2023-07-13T22:31:47.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc7447ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c1500a0 con 0x7fc73c0b0a70 2023-07-13T22:31:47.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc74259b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c0adc90 0x7fc73c14fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:47.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc74259b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c0adc90 0x7fc73c14fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50680/0 (socket says 172.21.15.60:50680) 2023-07-13T22:31:47.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc74259b700 1 -- 172.21.15.60:0/1453946873 learned_addr learned my addr 172.21.15.60:0/1453946873 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:47.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc741d9a700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0b0a70 0x7fc73c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:47.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.732+0000 7fc742d9c700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0ca8b0 0x7fc73c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:47.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc74259b700 1 -- 172.21.15.60:0/1453946873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0b0a70 msgr2=0x7fc73c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:47.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc74259b700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0b0a70 0x7fc73c0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:47.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc74259b700 1 -- 172.21.15.60:0/1453946873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0ca8b0 msgr2=0x7fc73c0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:47.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc74259b700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0ca8b0 0x7fc73c0acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:47.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc74259b700 1 -- 172.21.15.60:0/1453946873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc734009a70 con 0x7fc73c0adc90 2023-07-13T22:31:47.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc74259b700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c0adc90 0x7fc73c14fb60 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fc738001eb0 tx=0x7fc73800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:47.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc732ffd700 1 -- 172.21.15.60:0/1453946873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738005d70 con 0x7fc73c0adc90 2023-07-13T22:31:47.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc732ffd700 1 -- 172.21.15.60:0/1453946873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73800ce50 con 0x7fc73c0adc90 2023-07-13T22:31:47.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc732ffd700 1 -- 172.21.15.60:0/1453946873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7380072f0 con 0x7fc73c0adc90 2023-07-13T22:31:47.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73c0cd2e0 con 0x7fc73c0adc90 2023-07-13T22:31:47.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.733+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73c0cd8a0 con 0x7fc73c0adc90 2023-07-13T22:31:47.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.734+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73c0ce0a0 con 0x7fc73c0adc90 2023-07-13T22:31:47.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.735+0000 7fc732ffd700 1 -- 172.21.15.60:0/1453946873 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc738007450 con 0x7fc73c0adc90 2023-07-13T22:31:47.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.737+0000 7fc732ffd700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:47.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.737+0000 7fc732ffd700 1 -- 172.21.15.60:0/1453946873 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc738088840 con 0x7fc73c0adc90 2023-07-13T22:31:47.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.737+0000 7fc741d9a700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:47.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.737+0000 7fc741d9a700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0c8da0 tx=0x7fc72c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:47.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.741+0000 7fc732ffd700 1 -- 172.21.15.60:0/1453946873 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc73c0ce0a0 con 0x7fc73c0adc90 2023-07-13T22:31:47.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.993+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc73c0ce0a0 con 0x7fc73c0adc90 2023-07-13T22:31:47.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.994+0000 7fc732ffd700 1 -- 172.21.15.60:0/1453946873 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4719 (secure 0 0 0) 0x7fc73c0ce0a0 con 0x7fc73c0adc90 2023-07-13T22:31:47.977 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:47.977 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3652157896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3652157896},{"type":"v1","addr":"172.21.15.105:6834","nonce":3652157896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1878256861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1878256861},{"type":"v1","addr":"172.21.15.105:6835","nonce":1878256861}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92},{"gid":8728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2695331242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2695331242},{"type":"v1","addr":"172.21.15.60:6836","nonce":2695331242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:31:29.408608+0000","modified":"2023-07-13T22:31:32.919029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9423":{"gid":9423,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4061691799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4061691799},{"type":"v1","addr":"172.21.15.60:6837","nonce":4061691799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-13T22:31:47.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 msgr2=0x7fc72006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0c8da0 tx=0x7fc72c009040 comp rx=0 tx=0).stop 2023-07-13T22:31:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c0adc90 msgr2=0x7fc73c14fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c0adc90 0x7fc73c14fb60 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fc738001eb0 tx=0x7fc73800c7b0 comp rx=0 tx=0).stop 2023-07-13T22:31:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 shutdown_connections 2023-07-13T22:31:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc72006ace0 0x7fc72006d190 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc73c0adc90 0x7fc73c14fb60 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc73c0ca8b0 0x7fc73c0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 --2- 172.21.15.60:0/1453946873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc73c0b0a70 0x7fc73c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.996+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 >> 172.21.15.60:0/1453946873 conn(0x7fc73c0cb360 msgr2=0x7fc73c0cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.997+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 shutdown_connections 2023-07-13T22:31:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:47.997+0000 7fc7447ff700 1 -- 172.21.15.60:0/1453946873 wait complete. 2023-07-13T22:31:47.981 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 97 2023-07-13T22:31:47.997 INFO:tasks.ceph_test_case:waiting 70s for health warning matching MDS_CLIENT_LATE_RELEASE 2023-07-13T22:31:47.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:31:48.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.216+0000 7f2ce25ff700 1 Processor -- start 2023-07-13T22:31:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.216+0000 7f2ce25ff700 1 -- start start 2023-07-13T22:31:48.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2ce25ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2cdc15a320 0x7f2cdc0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:48.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2ce25ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2cdc0ca700 0x7f2cdc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:48.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2ce25ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc0707c0 con 0x7f2cdc0b0840 2023-07-13T22:31:48.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2ce25ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc0b0ce0 con 0x7f2cdc0ca700 2023-07-13T22:31:48.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2ce25ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc15b700 con 0x7f2cdc15a320 2023-07-13T22:31:48.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2ce0b9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2cdc0ca700 0x7f2cdc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:48.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2cdb7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2cdc0b0840 0x7f2cdc0c9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35356/0 (socket says 172.21.15.60:35356) 2023-07-13T22:31:48.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2cdb7fe700 1 -- 172.21.15.60:0/1312040647 learned_addr learned my addr 172.21.15.60:0/1312040647 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:48.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2cdbfff700 1 --2- 172.21.15.60:0/1312040647 >> v2:172.21.15.105:3300/0 conn(0x7f2cdc15a320 0x7f2cdc0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:48.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2cdaffd700 1 -- 172.21.15.60:0/1312040647 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150921656 0 0) 0x7f2cdc0707c0 con 0x7f2cdc0b0840 2023-07-13T22:31:48.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.218+0000 7f2cdaffd700 1 -- 172.21.15.60:0/1312040647 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc40035b0 con 0x7f2cdc0b0840 2023-07-13T22:31:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.219+0000 7f2ce0b9c700 1 -- 172.21.15.60:0/1312040647 >> v2:172.21.15.105:3300/0 conn(0x7f2cdc15a320 msgr2=0x7f2cdc0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.219+0000 7f2ce0b9c700 1 --2- 172.21.15.60:0/1312040647 >> v2:172.21.15.105:3300/0 conn(0x7f2cdc15a320 0x7f2cdc0b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.219+0000 7f2ce0b9c700 1 -- 172.21.15.60:0/1312040647 >> v1:172.21.15.60:6789/0 conn(0x7f2cdc0b0840 legacy=0x7f2cdc0c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.219+0000 7f2ce0b9c700 1 -- 172.21.15.60:0/1312040647 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc15df00 con 0x7f2cdc0ca700 2023-07-13T22:31:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.219+0000 7f2ce0b9c700 1 --2- 172.21.15.60:0/1312040647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc0ca700 0x7f2cdc06b5c0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0b1470 tx=0x7f2cd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39144373de609961 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.219+0000 7f2cdaffd700 1 -- 172.21.15.60:0/1312040647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400e070 con 0x7f2cdc0ca700 2023-07-13T22:31:48.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.219+0000 7f2cdaffd700 1 -- 172.21.15.60:0/1312040647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd400bb20 con 0x7f2cdc0ca700 2023-07-13T22:31:48.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.219+0000 7f2cdaffd700 1 -- 172.21.15.60:0/1312040647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4012430 con 0x7f2cdc0ca700 2023-07-13T22:31:48.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.220+0000 7f2ce25ff700 1 -- 172.21.15.60:0/1312040647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc0ca700 msgr2=0x7f2cdc06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:48.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.220+0000 7f2ce25ff700 1 --2- 172.21.15.60:0/1312040647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc0ca700 0x7f2cdc06b5c0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0b1470 tx=0x7f2cd400b330 comp rx=0 tx=0).stop 2023-07-13T22:31:48.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.220+0000 7f2ce25ff700 1 -- 172.21.15.60:0/1312040647 shutdown_connections 2023-07-13T22:31:48.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.220+0000 7f2ce25ff700 1 --2- 172.21.15.60:0/1312040647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc0ca700 0x7f2cdc06b5c0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.220+0000 7f2ce25ff700 1 --2- 172.21.15.60:0/1312040647 >> v2:172.21.15.105:3300/0 conn(0x7f2cdc15a320 0x7f2cdc0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.220+0000 7f2ce25ff700 1 -- 172.21.15.60:0/1312040647 >> 172.21.15.60:0/1312040647 conn(0x7f2cdc0cb300 msgr2=0x7f2cdc0cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:48.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.220+0000 7f2ce25ff700 1 -- 172.21.15.60:0/1312040647 shutdown_connections 2023-07-13T22:31:48.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.220+0000 7f2ce25ff700 1 -- 172.21.15.60:0/1312040647 wait complete. 2023-07-13T22:31:48.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.221+0000 7f2ce25ff700 1 Processor -- start 2023-07-13T22:31:48.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.222+0000 7f2ce25ff700 1 -- start start 2023-07-13T22:31:48.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.222+0000 7f2ce25ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc0b0840 0x7f2cdc141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:48.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce25ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cdc0ca700 0x7f2cdc0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:48.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce25ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc15a320 0x7f2cdc0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:48.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce25ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0cc5f0 con 0x7f2cdc15a320 2023-07-13T22:31:48.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce25ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0cc8d0 con 0x7f2cdc0ca700 2023-07-13T22:31:48.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce25ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0ccbb0 con 0x7f2cdc0b0840 2023-07-13T22:31:48.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce0b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc15a320 0x7f2cdc0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:48.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce0b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc15a320 0x7f2cdc0cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50692/0 (socket says 172.21.15.60:50692) 2023-07-13T22:31:48.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce0b9c700 1 -- 172.21.15.60:0/2739722967 learned_addr learned my addr 172.21.15.60:0/2739722967 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:48.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2cdb7fe700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cdc0ca700 0x7f2cdc0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:48.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2cdbfff700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc0b0840 0x7f2cdc141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:48.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce0b9c700 1 -- 172.21.15.60:0/2739722967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc0b0840 msgr2=0x7f2cdc141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:48.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce0b9c700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc0b0840 0x7f2cdc141690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce0b9c700 1 -- 172.21.15.60:0/2739722967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cdc0ca700 msgr2=0x7f2cdc0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:48.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce0b9c700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cdc0ca700 0x7f2cdc0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.223+0000 7f2ce0b9c700 1 -- 172.21.15.60:0/2739722967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4009a70 con 0x7f2cdc15a320 2023-07-13T22:31:48.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.224+0000 7f2ce0b9c700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc15a320 0x7f2cdc0cdbd0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4014040 tx=0x7f2cd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:48.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.224+0000 7f2cd8ff9700 1 -- 172.21.15.60:0/2739722967 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4012430 con 0x7f2cdc15a320 2023-07-13T22:31:48.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.224+0000 7f2cd8ff9700 1 -- 172.21.15.60:0/2739722967 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd4020070 con 0x7f2cdc15a320 2023-07-13T22:31:48.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.224+0000 7f2cd8ff9700 1 -- 172.21.15.60:0/2739722967 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40098f0 con 0x7f2cdc15a320 2023-07-13T22:31:48.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.224+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc0cce90 con 0x7f2cdc15a320 2023-07-13T22:31:48.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.224+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc141bd0 con 0x7f2cdc15a320 2023-07-13T22:31:48.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.225+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc0cd3b0 con 0x7f2cdc15a320 2023-07-13T22:31:48.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.226+0000 7f2cd8ff9700 1 -- 172.21.15.60:0/2739722967 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2cd4004040 con 0x7f2cdc15a320 2023-07-13T22:31:48.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.227+0000 7f2cd8ff9700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cc806ace0 0x7f2cc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:48.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.227+0000 7f2cd8ff9700 1 -- 172.21.15.60:0/2739722967 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2cd4022070 con 0x7f2cdc15a320 2023-07-13T22:31:48.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.227+0000 7f2cdbfff700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cc806ace0 0x7f2cc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:48.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.228+0000 7f2cdbfff700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cc806ace0 0x7f2cc806d190 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009be0 tx=0x7f2ccc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:48.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.231+0000 7f2cd8ff9700 1 -- 172.21.15.60:0/2739722967 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2cdc0cd3b0 con 0x7f2cdc15a320 2023-07-13T22:31:48.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.482+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2cdc0cd3b0 con 0x7f2cdc15a320 2023-07-13T22:31:48.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.482+0000 7f2cd8ff9700 1 -- 172.21.15.60:0/2739722967 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2cd402c040 con 0x7f2cdc15a320 2023-07-13T22:31:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:48.466 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:31:48.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.484+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cc806ace0 msgr2=0x7f2cc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:48.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.484+0000 7f2ce25ff700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cc806ace0 0x7f2cc806d190 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009be0 tx=0x7f2ccc009210 comp rx=0 tx=0).stop 2023-07-13T22:31:48.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.484+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc15a320 msgr2=0x7f2cdc0cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:48.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.484+0000 7f2ce25ff700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc15a320 0x7f2cdc0cdbd0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4014040 tx=0x7f2cd4004ad0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.484+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 shutdown_connections 2023-07-13T22:31:48.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.484+0000 7f2ce25ff700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f2cc806ace0 0x7f2cc806d190 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.485+0000 7f2ce25ff700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cdc15a320 0x7f2cdc0cdbd0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.485+0000 7f2ce25ff700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cdc0b0840 0x7f2cdc141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.485+0000 7f2ce25ff700 1 --2- 172.21.15.60:0/2739722967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cdc0ca700 0x7f2cdc0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.485+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 >> 172.21.15.60:0/2739722967 conn(0x7f2cdc0cb300 msgr2=0x7f2cdc0c9510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.485+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 shutdown_connections 2023-07-13T22:31:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:48.485+0000 7f2ce25ff700 1 -- 172.21.15.60:0/2739722967 wait complete. 2023-07-13T22:31:48.485 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:31:48.485 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:31:53.487 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:31:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.719+0000 7fd420dff700 1 Processor -- start 2023-07-13T22:31:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.720+0000 7fd420dff700 1 -- start start 2023-07-13T22:31:53.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.721+0000 7fd420dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 0x7fd41c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:53.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.721+0000 7fd420dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd41c0ca700 0x7fd41c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.721+0000 7fd420dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41c0707c0 con 0x7fd41c0b0840 2023-07-13T22:31:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.721+0000 7fd420dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c0b0cb0 con 0x7fd41c0ca700 2023-07-13T22:31:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.721+0000 7fd420dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0cab60 con 0x7fd41c15a320 2023-07-13T22:31:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.721+0000 7fd41ad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd41c0ca700 0x7fd41c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.721+0000 7fd413fff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd41c0b0840 0x7fd41c0c9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35372/0 (socket says 172.21.15.60:35372) 2023-07-13T22:31:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.721+0000 7fd413fff700 1 -- 172.21.15.60:0/228469803 learned_addr learned my addr 172.21.15.60:0/228469803 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.721+0000 7fd41a59c700 1 --2- 172.21.15.60:0/228469803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 0x7fd41c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.722+0000 7fd41ad9d700 1 -- 172.21.15.60:0/228469803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 msgr2=0x7fd41c0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.722+0000 7fd41ad9d700 1 --2- 172.21.15.60:0/228469803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 0x7fd41c0b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.722+0000 7fd41ad9d700 1 -- 172.21.15.60:0/228469803 >> v1:172.21.15.60:6789/0 conn(0x7fd41c0b0840 legacy=0x7fd41c0c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.722+0000 7fd41ad9d700 1 -- 172.21.15.60:0/228469803 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c15b700 con 0x7fd41c0ca700 2023-07-13T22:31:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.722+0000 7fd41a59c700 1 --2- 172.21.15.60:0/228469803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 0x7fd41c0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:31:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.722+0000 7fd41ad9d700 1 --2- 172.21.15.60:0/228469803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 0x7fd41c06b5c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fd414009d90 tx=0x7fd41400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd38e2f36e9d7dd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.722+0000 7fd419d9b700 1 -- 172.21.15.60:0/228469803 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400e070 con 0x7fd41c0ca700 2023-07-13T22:31:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.722+0000 7fd419d9b700 1 -- 172.21.15.60:0/228469803 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41400bbd0 con 0x7fd41c0ca700 2023-07-13T22:31:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.722+0000 7fd419d9b700 1 -- 172.21.15.60:0/228469803 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414012430 con 0x7fd41c0ca700 2023-07-13T22:31:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.723+0000 7fd420dff700 1 -- 172.21.15.60:0/228469803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 msgr2=0x7fd41c06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.723+0000 7fd420dff700 1 --2- 172.21.15.60:0/228469803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 0x7fd41c06b5c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fd414009d90 tx=0x7fd41400b330 comp rx=0 tx=0).stop 2023-07-13T22:31:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.723+0000 7fd420dff700 1 -- 172.21.15.60:0/228469803 shutdown_connections 2023-07-13T22:31:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.723+0000 7fd420dff700 1 --2- 172.21.15.60:0/228469803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 0x7fd41c06b5c0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.723+0000 7fd420dff700 1 --2- 172.21.15.60:0/228469803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 0x7fd41c0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.723+0000 7fd420dff700 1 -- 172.21.15.60:0/228469803 >> 172.21.15.60:0/228469803 conn(0x7fd41c0cb300 msgr2=0x7fd41c0cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.723+0000 7fd420dff700 1 -- 172.21.15.60:0/228469803 shutdown_connections 2023-07-13T22:31:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.723+0000 7fd420dff700 1 -- 172.21.15.60:0/228469803 wait complete. 2023-07-13T22:31:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.725+0000 7fd420dff700 1 Processor -- start 2023-07-13T22:31:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.725+0000 7fd420dff700 1 -- start start 2023-07-13T22:31:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd420dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c0b0840 0x7fd41c1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd420dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 0x7fd41c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd420dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 0x7fd41c144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd420dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0b1740 con 0x7fd41c0ca700 2023-07-13T22:31:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd420dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0c6cb0 con 0x7fd41c0b0840 2023-07-13T22:31:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd420dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0c6f90 con 0x7fd41c15a320 2023-07-13T22:31:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd413fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 0x7fd41c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd413fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 0x7fd41c0c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50710/0 (socket says 172.21.15.60:50710) 2023-07-13T22:31:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd413fff700 1 -- 172.21.15.60:0/3380179714 learned_addr learned my addr 172.21.15.60:0/3380179714 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd41a59c700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c0b0840 0x7fd41c1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.726+0000 7fd41ad9d700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 0x7fd41c144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd413fff700 1 -- 172.21.15.60:0/3380179714 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 msgr2=0x7fd41c144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd413fff700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 0x7fd41c144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd413fff700 1 -- 172.21.15.60:0/3380179714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c0b0840 msgr2=0x7fd41c1518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd413fff700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c0b0840 0x7fd41c1518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd413fff700 1 -- 172.21.15.60:0/3380179714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd414009a70 con 0x7fd41c0ca700 2023-07-13T22:31:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd413fff700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 0x7fd41c0c6360 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c430 tx=0x7fd40400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd412ffd700 1 -- 172.21.15.60:0/3380179714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404005d70 con 0x7fd41c0ca700 2023-07-13T22:31:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd412ffd700 1 -- 172.21.15.60:0/3380179714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4040070d0 con 0x7fd41c0ca700 2023-07-13T22:31:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c0c7270 con 0x7fd41c0ca700 2023-07-13T22:31:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd412ffd700 1 -- 172.21.15.60:0/3380179714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404007460 con 0x7fd41c0ca700 2023-07-13T22:31:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.727+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c151e10 con 0x7fd41c0ca700 2023-07-13T22:31:53.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.728+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41c074220 con 0x7fd41c0ca700 2023-07-13T22:31:53.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.729+0000 7fd412ffd700 1 -- 172.21.15.60:0/3380179714 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd4040075c0 con 0x7fd41c0ca700 2023-07-13T22:31:53.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.730+0000 7fd412ffd700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3fc06ace0 0x7fd3fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:53.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.730+0000 7fd412ffd700 1 -- 172.21.15.60:0/3380179714 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd4040889c0 con 0x7fd41c0ca700 2023-07-13T22:31:53.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.730+0000 7fd41a59c700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3fc06ace0 0x7fd3fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:53.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.731+0000 7fd41a59c700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3fc06ace0 0x7fd3fc06d190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c900 tx=0x7fd40c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:53.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.735+0000 7fd412ffd700 1 -- 172.21.15.60:0/3380179714 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd404053990 con 0x7fd41c0ca700 2023-07-13T22:31:53.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.993+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd41c0b1880 con 0x7fd41c0ca700 2023-07-13T22:31:53.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.993+0000 7fd412ffd700 1 -- 172.21.15.60:0/3380179714 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd404017040 con 0x7fd41c0ca700 2023-07-13T22:31:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:53.976 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:31:53.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3fc06ace0 msgr2=0x7fd3fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:53.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3fc06ace0 0x7fd3fc06d190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c900 tx=0x7fd40c008040 comp rx=0 tx=0).stop 2023-07-13T22:31:53.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 msgr2=0x7fd41c0c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 0x7fd41c0c6360 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c430 tx=0x7fd40400c7f0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 shutdown_connections 2023-07-13T22:31:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd3fc06ace0 0x7fd3fc06d190 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd41c15a320 0x7fd41c144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd41c0b0840 0x7fd41c1518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 --2- 172.21.15.60:0/3380179714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd41c0ca700 0x7fd41c0c6360 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:53.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 >> 172.21.15.60:0/3380179714 conn(0x7fd41c0cb300 msgr2=0x7fd41c06ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:53.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 shutdown_connections 2023-07-13T22:31:53.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:53.996+0000 7fd420dff700 1 -- 172.21.15.60:0/3380179714 wait complete. 2023-07-13T22:31:53.996 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:31:53.997 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:31:58.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:31:58.227 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:31:58.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:31:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.229+0000 7f4c41dff700 1 Processor -- start 2023-07-13T22:31:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.229+0000 7f4c41dff700 1 -- start start 2023-07-13T22:31:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.230+0000 7f4c41dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0cd2e0 0x7f4c3c0d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.230+0000 7f4c41dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c0cd7a0 con 0x7f4c3c15c570 2023-07-13T22:31:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.230+0000 7f4c41dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c0d1bb0 con 0x7f4c3c0b4e20 2023-07-13T22:31:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.230+0000 7f4c41dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0d1e90 con 0x7f4c3c0cd2e0 2023-07-13T22:31:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.230+0000 7f4c3b7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4c3c15c570 0x7f4c3c0b4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41698/0 (socket says 172.21.15.60:41698) 2023-07-13T22:31:59.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.230+0000 7f4c3b7fe700 1 -- 172.21.15.60:0/2040616289 learned_addr learned my addr 172.21.15.60:0/2040616289 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:31:59.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3bfff700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0cd2e0 0x7f4c3c0d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:59.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858755530 0 0) 0x7f4c3c0cd7a0 con 0x7f4c3c15c570 2023-07-13T22:31:59.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c240035b0 con 0x7f4c3c15c570 2023-07-13T22:31:59.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244413784 0 0) 0x7f4c3c0d1bb0 con 0x7f4c3c0b4e20 2023-07-13T22:31:59.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c3c0cd7a0 con 0x7f4c3c0b4e20 2023-07-13T22:31:59.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1597371416 0 0) 0x7f4c240035b0 con 0x7f4c3c15c570 2023-07-13T22:31:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0cd2e0 msgr2=0x7f4c3c0d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0cd2e0 0x7f4c3c0d2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 >> v1:172.21.15.105:6789/0 conn(0x7f4c3c0b4e20 legacy=0x7f4c3c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0d2170 con 0x7f4c3c15c570 2023-07-13T22:31:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f4c2c001670 con 0x7f4c3c15c570 2023-07-13T22:31:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 >> v1:172.21.15.60:6789/0 conn(0x7f4c3c15c570 legacy=0x7f4c3c0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0cd2e0 0x7f4c24003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:59.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24004170 0x7f4c24004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:59.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c24004d90 0x7f4c24005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:59.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0d1e90 con 0x7f4c24004170 2023-07-13T22:31:59.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2400bba0 con 0x7f4c24004d90 2023-07-13T22:31:59.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.231+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2400bd60 con 0x7f4c3c0cd2e0 2023-07-13T22:31:59.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.232+0000 7f4c3affd700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24004170 0x7f4c24004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:59.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.232+0000 7f4c3b7fe700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0cd2e0 0x7f4c24003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:59.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.232+0000 7f4c3bfff700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c24004d90 0x7f4c24005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:59.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.232+0000 7f4c3affd700 1 -- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0cd2e0 msgr2=0x7f4c24003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:59.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.232+0000 7f4c3affd700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0cd2e0 0x7f4c24003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.232+0000 7f4c3affd700 1 -- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c24004d90 msgr2=0x7f4c24005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:31:59.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.232+0000 7f4c3affd700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c24004d90 0x7f4c24005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.232+0000 7f4c3affd700 1 -- 172.21.15.60:0/2040616289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240035b0 con 0x7f4c24004170 2023-07-13T22:31:59.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.232+0000 7f4c3affd700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24004170 0x7f4c24004850 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c08b220 tx=0x7f4c24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f3657c9b4d7669 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:59.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30001430 con 0x7f4c24004170 2023-07-13T22:31:59.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c300052d0 con 0x7f4c24004170 2023-07-13T22:31:59.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c3a7fc700 1 -- 172.21.15.60:0/2040616289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30001d40 con 0x7f4c24004170 2023-07-13T22:31:59.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c41dff700 1 -- 172.21.15.60:0/2040616289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24004170 msgr2=0x7f4c24004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:59.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c41dff700 1 --2- 172.21.15.60:0/2040616289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c24004170 0x7f4c24004850 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c08b220 tx=0x7f4c24005880 comp rx=0 tx=0).stop 2023-07-13T22:31:59.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c3bfff700 1 -- 172.21.15.60:0/2040616289 reap_dead start 2023-07-13T22:31:59.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c41dff700 1 -- 172.21.15.60:0/2040616289 shutdown_connections 2023-07-13T22:31:59.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c41dff700 1 -- 172.21.15.60:0/2040616289 >> 172.21.15.60:0/2040616289 conn(0x7f4c3c06b0f0 msgr2=0x7f4c3c0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:59.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c41dff700 1 -- 172.21.15.60:0/2040616289 shutdown_connections 2023-07-13T22:31:59.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.233+0000 7f4c41dff700 1 -- 172.21.15.60:0/2040616289 wait complete. 2023-07-13T22:31:59.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.235+0000 7f4c41dff700 1 Processor -- start 2023-07-13T22:31:59.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.235+0000 7f4c41dff700 1 -- start start 2023-07-13T22:31:59.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.236+0000 7f4c41dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c24004170 0x7f4c3c153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:59.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.236+0000 7f4c41dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c3c0c9d60 0x7f4c3c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:59.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.236+0000 7f4c41dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0ca1d0 0x7f4c3c14efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:59.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.236+0000 7f4c41dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c15eb30 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.236+0000 7f4c41dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c15deb0 con 0x7f4c24004170 2023-07-13T22:31:59.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.236+0000 7f4c41dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c15e190 con 0x7f4c3c0ca1d0 2023-07-13T22:31:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.236+0000 7f4c3affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c3c0c9d60 0x7f4c3c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.236+0000 7f4c3affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c3c0c9d60 0x7f4c3c0cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34544/0 (socket says 172.21.15.60:34544) 2023-07-13T22:31:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.236+0000 7f4c3affd700 1 -- 172.21.15.60:0/2814279484 learned_addr learned my addr 172.21.15.60:0/2814279484 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:31:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c3b7fe700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c24004170 0x7f4c3c153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c3bfff700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0ca1d0 0x7f4c3c14efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c3affd700 1 -- 172.21.15.60:0/2814279484 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0ca1d0 msgr2=0x7f4c3c14efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c3affd700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0ca1d0 0x7f4c3c14efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c3affd700 1 -- 172.21.15.60:0/2814279484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c24004170 msgr2=0x7f4c3c153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:59.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c3affd700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c24004170 0x7f4c3c153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c3affd700 1 -- 172.21.15.60:0/2814279484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c15e470 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c3affd700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c3c0c9d60 0x7f4c3c0cc640 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000c3c0 tx=0x7f4c300012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:59.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c23fff700 1 -- 172.21.15.60:0/2814279484 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30001d40 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c23fff700 1 -- 172.21.15.60:0/2814279484 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c300013d0 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.237+0000 7f4c23fff700 1 -- 172.21.15.60:0/2814279484 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30007280 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.238+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c15e750 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.238+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c0ccd60 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.239+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c074220 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.240+0000 7f4c23fff700 1 -- 172.21.15.60:0/2814279484 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c300073e0 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.241+0000 7f4c23fff700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c2806ace0 0x7f4c2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:31:59.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.241+0000 7f4c23fff700 1 -- 172.21.15.60:0/2814279484 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4c3008a390 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.241+0000 7f4c3b7fe700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c2806ace0 0x7f4c2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:31:59.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.242+0000 7f4c3b7fe700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c2806ace0 0x7f4c2806d190 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c000c00 tx=0x7f4c2c000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:31:59.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.245+0000 7f4c23fff700 1 -- 172.21.15.60:0/2814279484 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4c300553f0 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.496+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4c3c0ccd60 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.497+0000 7f4c23fff700 1 -- 172.21.15.60:0/2814279484 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4c3c0ccd60 con 0x7f4c3c0c9d60 2023-07-13T22:31:59.480 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:31:59.480 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:31:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c2806ace0 msgr2=0x7f4c2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c2806ace0 0x7f4c2806d190 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c000c00 tx=0x7f4c2c000f80 comp rx=0 tx=0).stop 2023-07-13T22:31:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c3c0c9d60 msgr2=0x7f4c3c0cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:31:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c3c0c9d60 0x7f4c3c0cc640 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000c3c0 tx=0x7f4c300012d0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 shutdown_connections 2023-07-13T22:31:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4c2806ace0 0x7f4c2806d190 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4c3c0ca1d0 0x7f4c3c14efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c24004170 0x7f4c3c153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 --2- 172.21.15.60:0/2814279484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c3c0c9d60 0x7f4c3c0cc640 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:31:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 >> 172.21.15.60:0/2814279484 conn(0x7f4c3c06b0f0 msgr2=0x7f4c3c0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:31:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 shutdown_connections 2023-07-13T22:31:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:31:59.499+0000 7f4c41dff700 1 -- 172.21.15.60:0/2814279484 wait complete. 2023-07-13T22:31:59.499 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:31:59.500 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:04.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:04.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.732+0000 7f9478dff700 1 Processor -- start 2023-07-13T22:32:04.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.732+0000 7f9478dff700 1 -- start start 2023-07-13T22:32:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.733+0000 7f9478dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f947415c940 0x7f947406c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.733+0000 7f9478dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947415dd70 con 0x7f94740b31c0 2023-07-13T22:32:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.733+0000 7f9478dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947415cd10 con 0x7f947406c7d0 2023-07-13T22:32:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.733+0000 7f9478dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947406ce20 con 0x7f947415c940 2023-07-13T22:32:04.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.734+0000 7f9472d9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f94740b31c0 0x7f9474160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41702/0 (socket says 172.21.15.60:41702) 2023-07-13T22:32:04.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.734+0000 7f9472d9d700 1 -- 172.21.15.60:0/226762637 learned_addr learned my addr 172.21.15.60:0/226762637 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:32:04.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.734+0000 7f947259c700 1 --2- 172.21.15.60:0/226762637 >> v2:172.21.15.105:3300/0 conn(0x7f947415c940 0x7f947406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:04.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.734+0000 7f947159a700 1 -- 172.21.15.60:0/226762637 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934063298 0 0) 0x7f947415dd70 con 0x7f94740b31c0 2023-07-13T22:32:04.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.734+0000 7f947159a700 1 -- 172.21.15.60:0/226762637 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f945c0035b0 con 0x7f94740b31c0 2023-07-13T22:32:04.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.734+0000 7f947259c700 1 -- 172.21.15.60:0/226762637 >> v1:172.21.15.105:6789/0 conn(0x7f947406c7d0 legacy=0x7f94740b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:04.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.734+0000 7f947259c700 1 -- 172.21.15.60:0/226762637 >> v1:172.21.15.60:6789/0 conn(0x7f94740b31c0 legacy=0x7f9474160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:04.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.734+0000 7f947259c700 1 -- 172.21.15.60:0/226762637 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947406be20 con 0x7f947415c940 2023-07-13T22:32:04.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f947259c700 1 --2- 172.21.15.60:0/226762637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947415c940 0x7f947406c290 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9464009840 tx=0x7f9464009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4187051c08a617e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:04.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f947159a700 1 -- 172.21.15.60:0/226762637 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464011830 con 0x7f947415c940 2023-07-13T22:32:04.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f947159a700 1 -- 172.21.15.60:0/226762637 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9464011990 con 0x7f947415c940 2023-07-13T22:32:04.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f947159a700 1 -- 172.21.15.60:0/226762637 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464011d20 con 0x7f947415c940 2023-07-13T22:32:04.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f9478dff700 1 -- 172.21.15.60:0/226762637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947415c940 msgr2=0x7f947406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:04.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f9478dff700 1 --2- 172.21.15.60:0/226762637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947415c940 0x7f947406c290 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9464009840 tx=0x7f9464009b50 comp rx=0 tx=0).stop 2023-07-13T22:32:04.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f9478dff700 1 -- 172.21.15.60:0/226762637 shutdown_connections 2023-07-13T22:32:04.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f9478dff700 1 --2- 172.21.15.60:0/226762637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947415c940 0x7f947406c290 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:04.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f9478dff700 1 -- 172.21.15.60:0/226762637 >> 172.21.15.60:0/226762637 conn(0x7f947406dd10 msgr2=0x7f947406e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:04.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.735+0000 7f9478dff700 1 -- 172.21.15.60:0/226762637 shutdown_connections 2023-07-13T22:32:04.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.736+0000 7f9478dff700 1 -- 172.21.15.60:0/226762637 wait complete. 2023-07-13T22:32:04.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.737+0000 7f9478dff700 1 Processor -- start 2023-07-13T22:32:04.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.737+0000 7f9478dff700 1 -- start start 2023-07-13T22:32:04.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.738+0000 7f9478dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947406c7d0 0x7f94741e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:04.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.738+0000 7f9478dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94740b31c0 0x7f94740b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:04.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.738+0000 7f9478dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f947415c940 0x7f94740cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:04.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.738+0000 7f9478dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947415d950 con 0x7f947415c940 2023-07-13T22:32:04.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.738+0000 7f9478dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740b5ef0 con 0x7f947406c7d0 2023-07-13T22:32:04.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.738+0000 7f9478dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94740b52e0 con 0x7f94740b31c0 2023-07-13T22:32:04.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9472d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f947415c940 0x7f94740cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:04.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9472d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f947415c940 0x7f94740cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34556/0 (socket says 172.21.15.60:34556) 2023-07-13T22:32:04.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9472d9d700 1 -- 172.21.15.60:0/1282593371 learned_addr learned my addr 172.21.15.60:0/1282593371 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:04.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9471d9b700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94740b31c0 0x7f94740b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:04.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f947259c700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947406c7d0 0x7f94741e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:04.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9472d9d700 1 -- 172.21.15.60:0/1282593371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94740b31c0 msgr2=0x7f94740b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:04.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9472d9d700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94740b31c0 0x7f94740b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:04.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9472d9d700 1 -- 172.21.15.60:0/1282593371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947406c7d0 msgr2=0x7f94741e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:04.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9472d9d700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947406c7d0 0x7f94741e7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:04.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9472d9d700 1 -- 172.21.15.60:0/1282593371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640094a0 con 0x7f947415c940 2023-07-13T22:32:04.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.739+0000 7f9472d9d700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f947415c940 0x7f94740cc9b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f946c00caa0 tx=0x7f946c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:04.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.740+0000 7f9462ffd700 1 -- 172.21.15.60:0/1282593371 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c00a980 con 0x7f947415c940 2023-07-13T22:32:04.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.740+0000 7f9462ffd700 1 -- 172.21.15.60:0/1282593371 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c00aae0 con 0x7f947415c940 2023-07-13T22:32:04.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.740+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94740b55c0 con 0x7f947415c940 2023-07-13T22:32:04.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.740+0000 7f9462ffd700 1 -- 172.21.15.60:0/1282593371 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c00ae90 con 0x7f947415c940 2023-07-13T22:32:04.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.740+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94740b5b80 con 0x7f947415c940 2023-07-13T22:32:04.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.741+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947406a9e0 con 0x7f947415c940 2023-07-13T22:32:04.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.742+0000 7f9462ffd700 1 -- 172.21.15.60:0/1282593371 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f946c01b3b0 con 0x7f947415c940 2023-07-13T22:32:04.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.744+0000 7f9462ffd700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f945406ace0 0x7f945406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:04.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.744+0000 7f9462ffd700 1 -- 172.21.15.60:0/1282593371 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f946c089160 con 0x7f947415c940 2023-07-13T22:32:04.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.744+0000 7f947259c700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f945406ace0 0x7f945406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:04.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.745+0000 7f947259c700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f945406ace0 0x7f945406d190 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f94640090b0 tx=0x7f9464008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:04.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.748+0000 7f9462ffd700 1 -- 172.21.15.60:0/1282593371 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f946c0541f0 con 0x7f947415c940 2023-07-13T22:32:04.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.999+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9474074220 con 0x7f947415c940 2023-07-13T22:32:04.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:04.999+0000 7f9462ffd700 1 -- 172.21.15.60:0/1282593371 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f946c057810 con 0x7f947415c940 2023-07-13T22:32:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:04.982 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.001+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f945406ace0 msgr2=0x7f945406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.001+0000 7f9478dff700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f945406ace0 0x7f945406d190 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f94640090b0 tx=0x7f9464008030 comp rx=0 tx=0).stop 2023-07-13T22:32:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.001+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f947415c940 msgr2=0x7f94740cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.001+0000 7f9478dff700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f947415c940 0x7f94740cc9b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f946c00caa0 tx=0x7f946c00cdb0 comp rx=0 tx=0).stop 2023-07-13T22:32:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.002+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 shutdown_connections 2023-07-13T22:32:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.002+0000 7f9478dff700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f945406ace0 0x7f945406d190 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.002+0000 7f9478dff700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f947415c940 0x7f94740cc9b0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.002+0000 7f9478dff700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947406c7d0 0x7f94741e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.002+0000 7f9478dff700 1 --2- 172.21.15.60:0/1282593371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94740b31c0 0x7f94740b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.002+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 >> 172.21.15.60:0/1282593371 conn(0x7f947406dd10 msgr2=0x7f94740b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.002+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 shutdown_connections 2023-07-13T22:32:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:05.002+0000 7f9478dff700 1 -- 172.21.15.60:0/1282593371 wait complete. 2023-07-13T22:32:05.003 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:05.003 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:10.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.235+0000 7fdd5d9ff700 1 Processor -- start 2023-07-13T22:32:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.235+0000 7fdd5d9ff700 1 -- start start 2023-07-13T22:32:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.236+0000 7fdd5d9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd58071ad0 0x7fdd580c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.236+0000 7fdd5d9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806cd50 0x7fdd580cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.236+0000 7fdd5d9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd580cd320 con 0x7fdd5806fa60 2023-07-13T22:32:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.236+0000 7fdd5d9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd580cc9f0 con 0x7fdd58071ad0 2023-07-13T22:32:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.236+0000 7fdd5d9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580cccd0 con 0x7fdd5806cd50 2023-07-13T22:32:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.237+0000 7fdd56ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd58071ad0 0x7fdd580c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:10.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.237+0000 7fdd56ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd58071ad0 0x7fdd580c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57900/0 (socket says 172.21.15.60:57900) 2023-07-13T22:32:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.237+0000 7fdd56ffd700 1 -- 172.21.15.60:0/1867796796 learned_addr learned my addr 172.21.15.60:0/1867796796 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.237+0000 7fdd577fe700 1 --2- 172.21.15.60:0/1867796796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806cd50 0x7fdd580cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.237+0000 7fdd56ffd700 1 -- 172.21.15.60:0/1867796796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806cd50 msgr2=0x7fdd580cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.237+0000 7fdd56ffd700 1 --2- 172.21.15.60:0/1867796796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806cd50 0x7fdd580cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.237+0000 7fdd56ffd700 1 -- 172.21.15.60:0/1867796796 >> v1:172.21.15.105:6789/0 conn(0x7fdd5806fa60 legacy=0x7fdd5806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.237+0000 7fdd56ffd700 1 -- 172.21.15.60:0/1867796796 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580ccfb0 con 0x7fdd58071ad0 2023-07-13T22:32:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd56ffd700 1 --2- 172.21.15.60:0/1867796796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 0x7fdd580c8ed0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098f0 tx=0x7fdd48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4239e6c9c5d71d46 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd55ffb700 1 -- 172.21.15.60:0/1867796796 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800b8a0 con 0x7fdd58071ad0 2023-07-13T22:32:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd55ffb700 1 -- 172.21.15.60:0/1867796796 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800ba00 con 0x7fdd58071ad0 2023-07-13T22:32:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd55ffb700 1 -- 172.21.15.60:0/1867796796 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800bd90 con 0x7fdd58071ad0 2023-07-13T22:32:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1867796796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 msgr2=0x7fdd580c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd5d9ff700 1 --2- 172.21.15.60:0/1867796796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 0x7fdd580c8ed0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098f0 tx=0x7fdd48009c00 comp rx=0 tx=0).stop 2023-07-13T22:32:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1867796796 shutdown_connections 2023-07-13T22:32:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd5d9ff700 1 --2- 172.21.15.60:0/1867796796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806cd50 0x7fdd580cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd5d9ff700 1 --2- 172.21.15.60:0/1867796796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 0x7fdd580c8ed0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.238+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1867796796 >> 172.21.15.60:0/1867796796 conn(0x7fdd580cb300 msgr2=0x7fdd580cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.239+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1867796796 shutdown_connections 2023-07-13T22:32:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.239+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1867796796 wait complete. 2023-07-13T22:32:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.240+0000 7fdd5d9ff700 1 Processor -- start 2023-07-13T22:32:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.240+0000 7fdd5d9ff700 1 -- start start 2023-07-13T22:32:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.241+0000 7fdd5d9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd5806cd50 0x7fdd58157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.241+0000 7fdd5d9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806fa60 0x7fdd5815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.241+0000 7fdd5d9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 0x7fdd5815d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.241+0000 7fdd5d9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5815bc20 con 0x7fdd58071ad0 2023-07-13T22:32:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.241+0000 7fdd5d9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5815bf00 con 0x7fdd5806cd50 2023-07-13T22:32:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.241+0000 7fdd5d9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5815c1e0 con 0x7fdd5806fa60 2023-07-13T22:32:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd577fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 0x7fdd5815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd56ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd5806cd50 0x7fdd58157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd577fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 0x7fdd5815d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57904/0 (socket says 172.21.15.60:57904) 2023-07-13T22:32:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd577fe700 1 -- 172.21.15.60:0/1175089093 learned_addr learned my addr 172.21.15.60:0/1175089093 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd567fc700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806fa60 0x7fdd5815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd577fe700 1 -- 172.21.15.60:0/1175089093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806fa60 msgr2=0x7fdd5815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd577fe700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806fa60 0x7fdd5815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd577fe700 1 -- 172.21.15.60:0/1175089093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd5806cd50 msgr2=0x7fdd58157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd577fe700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd5806cd50 0x7fdd58157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd577fe700 1 -- 172.21.15.60:0/1175089093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009550 con 0x7fdd58071ad0 2023-07-13T22:32:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.242+0000 7fdd577fe700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 0x7fdd5815d300 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fdd50007e40 tx=0x7fdd5000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.243+0000 7fdd3ffff700 1 -- 172.21.15.60:0/1175089093 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000ed30 con 0x7fdd58071ad0 2023-07-13T22:32:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.243+0000 7fdd3ffff700 1 -- 172.21.15.60:0/1175089093 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50014420 con 0x7fdd58071ad0 2023-07-13T22:32:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.243+0000 7fdd3ffff700 1 -- 172.21.15.60:0/1175089093 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd500147d0 con 0x7fdd58071ad0 2023-07-13T22:32:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.243+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5815c4c0 con 0x7fdd58071ad0 2023-07-13T22:32:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.243+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5815aec0 con 0x7fdd58071ad0 2023-07-13T22:32:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.244+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd58074220 con 0x7fdd58071ad0 2023-07-13T22:32:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.245+0000 7fdd3ffff700 1 -- 172.21.15.60:0/1175089093 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd50014930 con 0x7fdd58071ad0 2023-07-13T22:32:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.246+0000 7fdd3ffff700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd4406ace0 0x7fdd4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.246+0000 7fdd3ffff700 1 -- 172.21.15.60:0/1175089093 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdd50089b70 con 0x7fdd58071ad0 2023-07-13T22:32:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.246+0000 7fdd56ffd700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd4406ace0 0x7fdd4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:10.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.247+0000 7fdd56ffd700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd4406ace0 0x7fdd4406d190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdd48014040 tx=0x7fdd48008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:10.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.251+0000 7fdd3ffff700 1 -- 172.21.15.60:0/1175089093 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdd50054bd0 con 0x7fdd58071ad0 2023-07-13T22:32:10.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.500+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdd580cc010 con 0x7fdd58071ad0 2023-07-13T22:32:10.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.501+0000 7fdd3ffff700 1 -- 172.21.15.60:0/1175089093 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdd500581f0 con 0x7fdd58071ad0 2023-07-13T22:32:10.484 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:10.484 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:10.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd4406ace0 msgr2=0x7fdd4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:10.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd4406ace0 0x7fdd4406d190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdd48014040 tx=0x7fdd48008030 comp rx=0 tx=0).stop 2023-07-13T22:32:10.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 msgr2=0x7fdd5815d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:10.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 0x7fdd5815d300 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fdd50007e40 tx=0x7fdd5000e3c0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 shutdown_connections 2023-07-13T22:32:10.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdd4406ace0 0x7fdd4406d190 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd58071ad0 0x7fdd5815d300 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd5806cd50 0x7fdd58157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 --2- 172.21.15.60:0/1175089093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd5806fa60 0x7fdd5815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:10.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 >> 172.21.15.60:0/1175089093 conn(0x7fdd580cb300 msgr2=0x7fdd580cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:10.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 shutdown_connections 2023-07-13T22:32:10.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:10.503+0000 7fdd5d9ff700 1 -- 172.21.15.60:0/1175089093 wait complete. 2023-07-13T22:32:10.504 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:10.505 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:15.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:15.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.737+0000 7fa5989ff700 1 Processor -- start 2023-07-13T22:32:15.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.737+0000 7fa5989ff700 1 -- start start 2023-07-13T22:32:15.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.738+0000 7fa5989ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa5940cea40 0x7fa594069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:15.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.738+0000 7fa5989ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa59406cc00 0x7fa59406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:15.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.738+0000 7fa5989ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa594070ba0 con 0x7fa59415c4b0 2023-07-13T22:32:15.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.738+0000 7fa5989ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59415c880 con 0x7fa59406cc00 2023-07-13T22:32:15.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.738+0000 7fa5989ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940ceeb0 con 0x7fa5940cea40 2023-07-13T22:32:15.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.738+0000 7fa59259c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa59415c4b0 0x7fa5940ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35276/0 (socket says 172.21.15.60:35276) 2023-07-13T22:32:15.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.738+0000 7fa59259c700 1 -- 172.21.15.60:0/177084291 learned_addr learned my addr 172.21.15.60:0/177084291 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:32:15.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.738+0000 7fa592d9d700 1 --2- 172.21.15.60:0/177084291 >> v2:172.21.15.60:3300/0 conn(0x7fa59406cc00 0x7fa59406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:15.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa58bfff700 1 --2- 172.21.15.60:0/177084291 >> v2:172.21.15.105:3300/0 conn(0x7fa5940cea40 0x7fa594069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:15.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa591d9b700 1 -- 172.21.15.60:0/177084291 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887802841 0 0) 0x7fa594070ba0 con 0x7fa59415c4b0 2023-07-13T22:32:15.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa591d9b700 1 -- 172.21.15.60:0/177084291 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5800035b0 con 0x7fa59415c4b0 2023-07-13T22:32:15.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa592d9d700 1 -- 172.21.15.60:0/177084291 >> v2:172.21.15.105:3300/0 conn(0x7fa5940cea40 msgr2=0x7fa594069eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:15.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa592d9d700 1 --2- 172.21.15.60:0/177084291 >> v2:172.21.15.105:3300/0 conn(0x7fa5940cea40 0x7fa594069eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:15.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa592d9d700 1 -- 172.21.15.60:0/177084291 >> v1:172.21.15.60:6789/0 conn(0x7fa59415c4b0 legacy=0x7fa5940ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:15.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa592d9d700 1 -- 172.21.15.60:0/177084291 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5940cf190 con 0x7fa59406cc00 2023-07-13T22:32:15.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa592d9d700 1 --2- 172.21.15.60:0/177084291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa59406cc00 0x7fa59406a3f0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fa59415fea0 tx=0x7fa58c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe1ab3311d7ee47 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:15.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa591d9b700 1 -- 172.21.15.60:0/177084291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00e070 con 0x7fa59406cc00 2023-07-13T22:32:15.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa591d9b700 1 -- 172.21.15.60:0/177084291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00bb20 con 0x7fa59406cc00 2023-07-13T22:32:15.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.739+0000 7fa591d9b700 1 -- 172.21.15.60:0/177084291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c012430 con 0x7fa59406cc00 2023-07-13T22:32:15.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.740+0000 7fa5989ff700 1 -- 172.21.15.60:0/177084291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa59406cc00 msgr2=0x7fa59406a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:15.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.740+0000 7fa5989ff700 1 --2- 172.21.15.60:0/177084291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa59406cc00 0x7fa59406a3f0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fa59415fea0 tx=0x7fa58c00b330 comp rx=0 tx=0).stop 2023-07-13T22:32:15.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.740+0000 7fa5989ff700 1 -- 172.21.15.60:0/177084291 shutdown_connections 2023-07-13T22:32:15.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.740+0000 7fa5989ff700 1 --2- 172.21.15.60:0/177084291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa59406cc00 0x7fa59406a3f0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:15.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.740+0000 7fa5989ff700 1 --2- 172.21.15.60:0/177084291 >> v2:172.21.15.105:3300/0 conn(0x7fa5940cea40 0x7fa594069eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:15.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.740+0000 7fa5989ff700 1 -- 172.21.15.60:0/177084291 >> 172.21.15.60:0/177084291 conn(0x7fa5940d05b0 msgr2=0x7fa5940cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:15.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.740+0000 7fa5989ff700 1 -- 172.21.15.60:0/177084291 shutdown_connections 2023-07-13T22:32:15.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.740+0000 7fa5989ff700 1 -- 172.21.15.60:0/177084291 wait complete. 2023-07-13T22:32:15.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.742+0000 7fa5989ff700 1 Processor -- start 2023-07-13T22:32:15.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.742+0000 7fa5989ff700 1 -- start start 2023-07-13T22:32:15.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa5989ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa59406cc00 0x7fa5941585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:15.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa5989ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5940cea40 0x7fa594149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:15.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa5989ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa59415c4b0 0x7fa5940c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:15.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa5989ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59415d910 con 0x7fa5940cea40 2023-07-13T22:32:15.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa5989ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c3c40 con 0x7fa59415c4b0 2023-07-13T22:32:15.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa5989ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c3f20 con 0x7fa59406cc00 2023-07-13T22:32:15.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa58bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5940cea40 0x7fa594149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:15.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa58bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5940cea40 0x7fa594149ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57660/0 (socket says 172.21.15.60:57660) 2023-07-13T22:32:15.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa58bfff700 1 -- 172.21.15.60:0/4235686921 learned_addr learned my addr 172.21.15.60:0/4235686921 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:15.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa59259c700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa59406cc00 0x7fa5941585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:15.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa592d9d700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa59415c4b0 0x7fa5940c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:15.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa58bfff700 1 -- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa59406cc00 msgr2=0x7fa5941585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:15.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa58bfff700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa59406cc00 0x7fa5941585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:15.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa58bfff700 1 -- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa59415c4b0 msgr2=0x7fa5940c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:15.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa58bfff700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa59415c4b0 0x7fa5940c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:15.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.743+0000 7fa58bfff700 1 -- 172.21.15.60:0/4235686921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c009a70 con 0x7fa5940cea40 2023-07-13T22:32:15.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.744+0000 7fa59259c700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa59406cc00 0x7fa5941585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:32:15.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.744+0000 7fa592d9d700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa59415c4b0 0x7fa5940c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:32:15.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.744+0000 7fa58bfff700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5940cea40 0x7fa594149ed0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00d7d0 tx=0x7fa57c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:15.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.744+0000 7fa58affd700 1 -- 172.21.15.60:0/4235686921 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0125f0 con 0x7fa5940cea40 2023-07-13T22:32:15.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.744+0000 7fa58affd700 1 -- 172.21.15.60:0/4235686921 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c012750 con 0x7fa5940cea40 2023-07-13T22:32:15.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.744+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59415d5a0 con 0x7fa5940cea40 2023-07-13T22:32:15.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.744+0000 7fa58affd700 1 -- 172.21.15.60:0/4235686921 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c012ae0 con 0x7fa5940cea40 2023-07-13T22:32:15.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.744+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59414a410 con 0x7fa5940cea40 2023-07-13T22:32:15.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.745+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59406ae30 con 0x7fa5940cea40 2023-07-13T22:32:15.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.746+0000 7fa58affd700 1 -- 172.21.15.60:0/4235686921 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa57c012c40 con 0x7fa5940cea40 2023-07-13T22:32:15.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.747+0000 7fa58affd700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa57406ace0 0x7fa57406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:15.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.747+0000 7fa58affd700 1 -- 172.21.15.60:0/4235686921 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa57c089620 con 0x7fa5940cea40 2023-07-13T22:32:15.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.747+0000 7fa59259c700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa57406ace0 0x7fa57406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:15.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.748+0000 7fa59259c700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa57406ace0 0x7fa57406d190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa5840011e0 tx=0x7fa584009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:15.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:15.752+0000 7fa58affd700 1 -- 172.21.15.60:0/4235686921 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa57c054680 con 0x7fa5940cea40 2023-07-13T22:32:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.005+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa594074220 con 0x7fa5940cea40 2023-07-13T22:32:15.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.006+0000 7fa58affd700 1 -- 172.21.15.60:0/4235686921 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa57c057ca0 con 0x7fa5940cea40 2023-07-13T22:32:15.989 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:15.989 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:15.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa57406ace0 msgr2=0x7fa57406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa57406ace0 0x7fa57406d190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa5840011e0 tx=0x7fa584009040 comp rx=0 tx=0).stop 2023-07-13T22:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5940cea40 msgr2=0x7fa594149ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5940cea40 0x7fa594149ed0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00d7d0 tx=0x7fa57c00db90 comp rx=0 tx=0).stop 2023-07-13T22:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 shutdown_connections 2023-07-13T22:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa57406ace0 0x7fa57406d190 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa59415c4b0 0x7fa5940c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:15.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa59406cc00 0x7fa5941585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:15.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 --2- 172.21.15.60:0/4235686921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5940cea40 0x7fa594149ed0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:15.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 >> 172.21.15.60:0/4235686921 conn(0x7fa5940d05b0 msgr2=0x7fa5940ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:15.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 shutdown_connections 2023-07-13T22:32:15.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:16.008+0000 7fa5989ff700 1 -- 172.21.15.60:0/4235686921 wait complete. 2023-07-13T22:32:16.009 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:16.009 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:21.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:21.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.240+0000 7faa47bff700 1 Processor -- start 2023-07-13T22:32:21.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.240+0000 7faa47bff700 1 -- start start 2023-07-13T22:32:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.241+0000 7faa47bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.241+0000 7faa47bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa4006fa60 0x7faa4006c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:21.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.241+0000 7faa47bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa400cd320 con 0x7faa400c9410 2023-07-13T22:32:21.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.241+0000 7faa47bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa400cc9f0 con 0x7faa4006fa60 2023-07-13T22:32:21.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.241+0000 7faa47bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa400cccd0 con 0x7faa40071ad0 2023-07-13T22:32:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.242+0000 7faa4599b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.242+0000 7faa4519a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa4006fa60 0x7faa4006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.242+0000 7faa4599b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:50470/0 (socket says 172.21.15.60:50470) 2023-07-13T22:32:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.242+0000 7faa4599b700 1 -- 172.21.15.60:0/2180988399 learned_addr learned my addr 172.21.15.60:0/2180988399 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.242+0000 7faa4599b700 1 -- 172.21.15.60:0/2180988399 >> v2:172.21.15.105:3300/0 conn(0x7faa4006fa60 msgr2=0x7faa4006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.242+0000 7faa4599b700 1 --2- 172.21.15.60:0/2180988399 >> v2:172.21.15.105:3300/0 conn(0x7faa4006fa60 0x7faa4006c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.242+0000 7faa4599b700 1 -- 172.21.15.60:0/2180988399 >> v1:172.21.15.105:6789/0 conn(0x7faa400c9410 legacy=0x7faa400b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.242+0000 7faa4599b700 1 -- 172.21.15.60:0/2180988399 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400ccfb0 con 0x7faa40071ad0 2023-07-13T22:32:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.242+0000 7faa4599b700 1 --2- 172.21.15.60:0/2180988399 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c8ed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009920 tx=0x7faa3c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22cff3e9f9ede6a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:21.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa44999700 1 -- 172.21.15.60:0/2180988399 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c00f8a0 con 0x7faa40071ad0 2023-07-13T22:32:21.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa44999700 1 -- 172.21.15.60:0/2180988399 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c00fa00 con 0x7faa40071ad0 2023-07-13T22:32:21.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa44999700 1 -- 172.21.15.60:0/2180988399 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c00fd90 con 0x7faa40071ad0 2023-07-13T22:32:21.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa47bff700 1 -- 172.21.15.60:0/2180988399 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 msgr2=0x7faa400c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:21.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa47bff700 1 --2- 172.21.15.60:0/2180988399 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c8ed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009920 tx=0x7faa3c009c30 comp rx=0 tx=0).stop 2023-07-13T22:32:21.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa47bff700 1 -- 172.21.15.60:0/2180988399 shutdown_connections 2023-07-13T22:32:21.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa47bff700 1 --2- 172.21.15.60:0/2180988399 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c8ed0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa47bff700 1 --2- 172.21.15.60:0/2180988399 >> v2:172.21.15.105:3300/0 conn(0x7faa4006fa60 0x7faa4006c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa47bff700 1 -- 172.21.15.60:0/2180988399 >> 172.21.15.60:0/2180988399 conn(0x7faa400cb300 msgr2=0x7faa400cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.243+0000 7faa47bff700 1 -- 172.21.15.60:0/2180988399 shutdown_connections 2023-07-13T22:32:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.244+0000 7faa47bff700 1 -- 172.21.15.60:0/2180988399 wait complete. 2023-07-13T22:32:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.245+0000 7faa47bff700 1 Processor -- start 2023-07-13T22:32:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.246+0000 7faa47bff700 1 -- start start 2023-07-13T22:32:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.246+0000 7faa47bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4006fa60 0x7faa40143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.246+0000 7faa47bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.247+0000 7faa47bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa400c9410 0x7faa400ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.247+0000 7faa47bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa400b1180 con 0x7faa4006fa60 2023-07-13T22:32:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.247+0000 7faa47bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa400b0ea0 con 0x7faa400c9410 2023-07-13T22:32:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.247+0000 7faa47bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa400b2960 con 0x7faa40071ad0 2023-07-13T22:32:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.247+0000 7faa4599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4006fa60 0x7faa40143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.247+0000 7faa4599b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4006fa60 0x7faa40143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57674/0 (socket says 172.21.15.60:57674) 2023-07-13T22:32:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.247+0000 7faa4599b700 1 -- 172.21.15.60:0/1424229878 learned_addr learned my addr 172.21.15.60:0/1424229878 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.247+0000 7faa4619c700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa400c9410 0x7faa400ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.247+0000 7faa4519a700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa4599b700 1 -- 172.21.15.60:0/1424229878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 msgr2=0x7faa400c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa4599b700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa4599b700 1 -- 172.21.15.60:0/1424229878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa400c9410 msgr2=0x7faa400ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa4599b700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa400c9410 0x7faa400ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa4599b700 1 -- 172.21.15.60:0/1424229878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c009550 con 0x7faa4006fa60 2023-07-13T22:32:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa4599b700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4006fa60 0x7faa40143c90 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7faa3c000c00 tx=0x7faa3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa367fc700 1 -- 172.21.15.60:0/1424229878 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0038a0 con 0x7faa4006fa60 2023-07-13T22:32:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa367fc700 1 -- 172.21.15.60:0/1424229878 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c003a20 con 0x7faa4006fa60 2023-07-13T22:32:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa400b2c40 con 0x7faa4006fa60 2023-07-13T22:32:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa367fc700 1 -- 172.21.15.60:0/1424229878 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c003dd0 con 0x7faa4006fa60 2023-07-13T22:32:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.248+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa400c6780 con 0x7faa4006fa60 2023-07-13T22:32:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.249+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa400c6910 con 0x7faa4006fa60 2023-07-13T22:32:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.250+0000 7faa367fc700 1 -- 172.21.15.60:0/1424229878 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa3c02a430 con 0x7faa4006fa60 2023-07-13T22:32:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.251+0000 7faa367fc700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2406ace0 0x7faa2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.251+0000 7faa367fc700 1 -- 172.21.15.60:0/1424229878 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7faa3c08a200 con 0x7faa4006fa60 2023-07-13T22:32:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.252+0000 7faa4519a700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2406ace0 0x7faa2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.252+0000 7faa4519a700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2406ace0 0x7faa2406d190 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7faa400c6140 tx=0x7faa30009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.256+0000 7faa367fc700 1 -- 172.21.15.60:0/1424229878 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7faa3c055260 con 0x7faa4006fa60 2023-07-13T22:32:21.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.505+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faa40074220 con 0x7faa4006fa60 2023-07-13T22:32:21.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.505+0000 7faa367fc700 1 -- 172.21.15.60:0/1424229878 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7faa3c058880 con 0x7faa4006fa60 2023-07-13T22:32:21.489 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:21.489 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2406ace0 msgr2=0x7faa2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2406ace0 0x7faa2406d190 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7faa400c6140 tx=0x7faa30009210 comp rx=0 tx=0).stop 2023-07-13T22:32:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4006fa60 msgr2=0x7faa40143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4006fa60 0x7faa40143c90 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7faa3c000c00 tx=0x7faa3c004750 comp rx=0 tx=0).stop 2023-07-13T22:32:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 shutdown_connections 2023-07-13T22:32:21.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7faa2406ace0 0x7faa2406d190 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:21.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa4006fa60 0x7faa40143c90 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:21.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa400c9410 0x7faa400ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:21.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 --2- 172.21.15.60:0/1424229878 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa40071ad0 0x7faa400c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:21.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.507+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 >> 172.21.15.60:0/1424229878 conn(0x7faa400cb300 msgr2=0x7faa4015b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:21.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.508+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 shutdown_connections 2023-07-13T22:32:21.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:21.508+0000 7faa47bff700 1 -- 172.21.15.60:0/1424229878 wait complete. 2023-07-13T22:32:21.509 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:21.510 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:26.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:26.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.739+0000 7fec389ff700 1 Processor -- start 2023-07-13T22:32:26.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.740+0000 7fec389ff700 1 -- start start 2023-07-13T22:32:26.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.741+0000 7fec389ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec3406fa60 0x7fec3406c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:26.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.741+0000 7fec389ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec340cd320 con 0x7fec34071ad0 2023-07-13T22:32:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.741+0000 7fec389ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec340cc9f0 con 0x7fec3406cb80 2023-07-13T22:32:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.741+0000 7fec389ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3406d000 con 0x7fec3406fa60 2023-07-13T22:32:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.741+0000 7fec3259c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fec34071ad0 0x7fec340c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48932/0 (socket says 172.21.15.60:48932) 2023-07-13T22:32:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.741+0000 7fec3259c700 1 -- 172.21.15.60:0/619795244 learned_addr learned my addr 172.21.15.60:0/619795244 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:32:26.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.741+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897361481 0 0) 0x7fec340cd320 con 0x7fec34071ad0 2023-07-13T22:32:26.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.741+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec200035b0 con 0x7fec34071ad0 2023-07-13T22:32:26.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.741+0000 7fec2bfff700 1 --2- 172.21.15.60:0/619795244 >> v2:172.21.15.105:3300/0 conn(0x7fec3406fa60 0x7fec3406c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:26.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3894605268 0 0) 0x7fec200035b0 con 0x7fec34071ad0 2023-07-13T22:32:26.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 >> v2:172.21.15.105:3300/0 conn(0x7fec3406fa60 msgr2=0x7fec3406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:26.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 --2- 172.21.15.60:0/619795244 >> v2:172.21.15.105:3300/0 conn(0x7fec3406fa60 0x7fec3406c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 >> v1:172.21.15.105:6789/0 conn(0x7fec3406cb80 legacy=0x7fec340cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:26.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec340cccd0 con 0x7fec34071ad0 2023-07-13T22:32:26.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fec24001670 con 0x7fec34071ad0 2023-07-13T22:32:26.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 >> v1:172.21.15.60:6789/0 conn(0x7fec34071ad0 legacy=0x7fec340c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:26.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec20003a50 0x7fec20003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:26.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20004850 0x7fec20004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:26.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec20005250 0x7fec2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:26.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3406d000 con 0x7fec20003a50 2023-07-13T22:32:26.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2000c160 con 0x7fec20005250 2023-07-13T22:32:26.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2000c320 con 0x7fec20004850 2023-07-13T22:32:26.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec3259c700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec20003a50 0x7fec20003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:26.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec2bfff700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20004850 0x7fec20004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:26.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.742+0000 7fec32d9d700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec20005250 0x7fec2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec3259c700 1 -- 172.21.15.60:0/619795244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20004850 msgr2=0x7fec20004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec3259c700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20004850 0x7fec20004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec3259c700 1 -- 172.21.15.60:0/619795244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec20005250 msgr2=0x7fec2000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec3259c700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec20005250 0x7fec2000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec3259c700 1 -- 172.21.15.60:0/619795244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200035b0 con 0x7fec20003a50 2023-07-13T22:32:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec32d9d700 1 -- 172.21.15.60:0/619795244 reap_dead start 2023-07-13T22:32:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec3259c700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec20003a50 0x7fec20003e60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fec24007c90 tx=0x7fec20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd41f8bb77737ac server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2400ccd0 con 0x7fec20003a50 2023-07-13T22:32:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2400ce50 con 0x7fec20003a50 2023-07-13T22:32:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.743+0000 7fec31d9b700 1 -- 172.21.15.60:0/619795244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24005ba0 con 0x7fec20003a50 2023-07-13T22:32:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.744+0000 7fec389ff700 1 -- 172.21.15.60:0/619795244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec20003a50 msgr2=0x7fec20003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.744+0000 7fec389ff700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec20003a50 0x7fec20003e60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fec24007c90 tx=0x7fec20005880 comp rx=0 tx=0).stop 2023-07-13T22:32:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.744+0000 7fec389ff700 1 -- 172.21.15.60:0/619795244 shutdown_connections 2023-07-13T22:32:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.744+0000 7fec389ff700 1 --2- 172.21.15.60:0/619795244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec20003a50 0x7fec20003e60 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.744+0000 7fec389ff700 1 -- 172.21.15.60:0/619795244 >> 172.21.15.60:0/619795244 conn(0x7fec340cb300 msgr2=0x7fec340cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.744+0000 7fec389ff700 1 -- 172.21.15.60:0/619795244 shutdown_connections 2023-07-13T22:32:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.744+0000 7fec389ff700 1 -- 172.21.15.60:0/619795244 wait complete. 2023-07-13T22:32:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.746+0000 7fec389ff700 1 Processor -- start 2023-07-13T22:32:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.746+0000 7fec389ff700 1 -- start start 2023-07-13T22:32:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.747+0000 7fec389ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20003a50 0x7fec340b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.747+0000 7fec389ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec340afa10 0x7fec34154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:26.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.747+0000 7fec389ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec34155c80 0x7fec340cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:26.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.747+0000 7fec389ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34155a80 con 0x7fec34155c80 2023-07-13T22:32:26.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.747+0000 7fec389ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec340b2110 con 0x7fec340afa10 2023-07-13T22:32:26.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.747+0000 7fec389ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec340b17e0 con 0x7fec20003a50 2023-07-13T22:32:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.747+0000 7fec32d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec34155c80 0x7fec340cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.747+0000 7fec32d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec34155c80 0x7fec340cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42636/0 (socket says 172.21.15.60:42636) 2023-07-13T22:32:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.747+0000 7fec32d9d700 1 -- 172.21.15.60:0/461064843 learned_addr learned my addr 172.21.15.60:0/461064843 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec3259c700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20003a50 0x7fec340b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec32d9d700 1 -- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20003a50 msgr2=0x7fec340b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec2bfff700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec340afa10 0x7fec34154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec32d9d700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20003a50 0x7fec340b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec32d9d700 1 -- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec340afa10 msgr2=0x7fec34154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec32d9d700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec340afa10 0x7fec34154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec32d9d700 1 -- 172.21.15.60:0/461064843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec340b1ac0 con 0x7fec34155c80 2023-07-13T22:32:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec3259c700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20003a50 0x7fec340b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:32:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec2bfff700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec340afa10 0x7fec34154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:32:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec32d9d700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec34155c80 0x7fec340cd6b0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fec3406f540 tx=0x7fec2c00d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec2affd700 1 -- 172.21.15.60:0/461064843 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c003cd0 con 0x7fec34155c80 2023-07-13T22:32:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec2affd700 1 -- 172.21.15.60:0/461064843 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c003e70 con 0x7fec34155c80 2023-07-13T22:32:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec2affd700 1 -- 172.21.15.60:0/461064843 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00d9f0 con 0x7fec34155c80 2023-07-13T22:32:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec340b1da0 con 0x7fec34155c80 2023-07-13T22:32:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.748+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec34155730 con 0x7fec34155c80 2023-07-13T22:32:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.749+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec34074220 con 0x7fec34155c80 2023-07-13T22:32:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.750+0000 7fec2affd700 1 -- 172.21.15.60:0/461064843 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec2c00db50 con 0x7fec34155c80 2023-07-13T22:32:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.751+0000 7fec2affd700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec1406ace0 0x7fec1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.751+0000 7fec2affd700 1 -- 172.21.15.60:0/461064843 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fec2c089730 con 0x7fec34155c80 2023-07-13T22:32:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.751+0000 7fec3259c700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec1406ace0 0x7fec1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.752+0000 7fec3259c700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec1406ace0 0x7fec1406d190 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fec24001030 tx=0x7fec24007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:26.756+0000 7fec2affd700 1 -- 172.21.15.60:0/461064843 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fec2c054790 con 0x7fec34155c80 2023-07-13T22:32:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.007+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fec340cc010 con 0x7fec34155c80 2023-07-13T22:32:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.007+0000 7fec2affd700 1 -- 172.21.15.60:0/461064843 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fec2c057db0 con 0x7fec34155c80 2023-07-13T22:32:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:26.992 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec1406ace0 msgr2=0x7fec1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec1406ace0 0x7fec1406d190 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fec24001030 tx=0x7fec24007000 comp rx=0 tx=0).stop 2023-07-13T22:32:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec34155c80 msgr2=0x7fec340cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec34155c80 0x7fec340cd6b0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fec3406f540 tx=0x7fec2c00d150 comp rx=0 tx=0).stop 2023-07-13T22:32:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 shutdown_connections 2023-07-13T22:32:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fec1406ace0 0x7fec1406d190 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec34155c80 0x7fec340cd6b0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec20003a50 0x7fec340b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 --2- 172.21.15.60:0/461064843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec340afa10 0x7fec34154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:26.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 >> 172.21.15.60:0/461064843 conn(0x7fec340cb300 msgr2=0x7fec3406bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:26.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 shutdown_connections 2023-07-13T22:32:26.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:27.010+0000 7fec389ff700 1 -- 172.21.15.60:0/461064843 wait complete. 2023-07-13T22:32:27.012 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:27.013 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:28.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:32:28.345 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:32:32.013 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:32.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.242+0000 7fd2539ff700 1 Processor -- start 2023-07-13T22:32:32.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.242+0000 7fd2539ff700 1 -- start start 2023-07-13T22:32:32.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.243+0000 7fd2539ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd24c15c570 0x7fd24c0d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:32.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.243+0000 7fd2539ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd24c0b4e20 0x7fd24c071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:32.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.243+0000 7fd2539ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c0cd7a0 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.243+0000 7fd2539ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0d1bb0 con 0x7fd24c15c570 2023-07-13T22:32:32.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.243+0000 7fd2539ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0d1e90 con 0x7fd24c0b4e20 2023-07-13T22:32:32.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd25179b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd24c15c570 0x7fd24c0d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:32.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd25179b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd24c15c570 0x7fd24c0d2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42646/0 (socket says 172.21.15.60:42646) 2023-07-13T22:32:32.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd25179b700 1 -- 172.21.15.60:0/2469603084 learned_addr learned my addr 172.21.15.60:0/2469603084 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:32.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd251f9c700 1 --2- 172.21.15.60:0/2469603084 >> v2:172.21.15.105:3300/0 conn(0x7fd24c0b4e20 0x7fd24c071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:32.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd25179b700 1 -- 172.21.15.60:0/2469603084 >> v2:172.21.15.105:3300/0 conn(0x7fd24c0b4e20 msgr2=0x7fd24c071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:32.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd25179b700 1 --2- 172.21.15.60:0/2469603084 >> v2:172.21.15.105:3300/0 conn(0x7fd24c0b4e20 0x7fd24c071ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd25179b700 1 -- 172.21.15.60:0/2469603084 >> v1:172.21.15.105:6789/0 conn(0x7fd24c0cd2e0 legacy=0x7fd24c0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:32.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd25179b700 1 -- 172.21.15.60:0/2469603084 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0d2170 con 0x7fd24c15c570 2023-07-13T22:32:32.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd251f9c700 1 --2- 172.21.15.60:0/2469603084 >> v2:172.21.15.105:3300/0 conn(0x7fd24c0b4e20 0x7fd24c071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:32:32.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.244+0000 7fd25179b700 1 --2- 172.21.15.60:0/2469603084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c15c570 0x7fd24c0d2690 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fd24c06b990 tx=0x7fd248009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e562a8944acc3435 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:32.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd243fff700 1 -- 172.21.15.60:0/2469603084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24800b870 con 0x7fd24c15c570 2023-07-13T22:32:32.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd243fff700 1 -- 172.21.15.60:0/2469603084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24800b9d0 con 0x7fd24c15c570 2023-07-13T22:32:32.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd243fff700 1 -- 172.21.15.60:0/2469603084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24800bd60 con 0x7fd24c15c570 2023-07-13T22:32:32.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd2539ff700 1 -- 172.21.15.60:0/2469603084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c15c570 msgr2=0x7fd24c0d2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:32.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd2539ff700 1 --2- 172.21.15.60:0/2469603084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c15c570 0x7fd24c0d2690 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fd24c06b990 tx=0x7fd248009bd0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd2539ff700 1 -- 172.21.15.60:0/2469603084 shutdown_connections 2023-07-13T22:32:32.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd2539ff700 1 --2- 172.21.15.60:0/2469603084 >> v2:172.21.15.105:3300/0 conn(0x7fd24c0b4e20 0x7fd24c071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd2539ff700 1 --2- 172.21.15.60:0/2469603084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c15c570 0x7fd24c0d2690 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd2539ff700 1 -- 172.21.15.60:0/2469603084 >> 172.21.15.60:0/2469603084 conn(0x7fd24c06b0f0 msgr2=0x7fd24c0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:32.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd2539ff700 1 -- 172.21.15.60:0/2469603084 shutdown_connections 2023-07-13T22:32:32.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.245+0000 7fd2539ff700 1 -- 172.21.15.60:0/2469603084 wait complete. 2023-07-13T22:32:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.247+0000 7fd2539ff700 1 Processor -- start 2023-07-13T22:32:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.247+0000 7fd2539ff700 1 -- start start 2023-07-13T22:32:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.248+0000 7fd2539ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c0b4e20 0x7fd24c147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.248+0000 7fd2539ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c0cd2e0 0x7fd24c0c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.248+0000 7fd2539ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd24c15c570 0x7fd24c147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.248+0000 7fd2539ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0c8510 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.248+0000 7fd2539ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0c87f0 con 0x7fd24c0b4e20 2023-07-13T22:32:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.248+0000 7fd2539ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0c8ad0 con 0x7fd24c15c570 2023-07-13T22:32:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd250f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c0cd2e0 0x7fd24c0c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd250f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c0cd2e0 0x7fd24c0c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42660/0 (socket says 172.21.15.60:42660) 2023-07-13T22:32:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd250f9a700 1 -- 172.21.15.60:0/957854546 learned_addr learned my addr 172.21.15.60:0/957854546 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd25179b700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c0b4e20 0x7fd24c147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd251f9c700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd24c15c570 0x7fd24c147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd250f9a700 1 -- 172.21.15.60:0/957854546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd24c15c570 msgr2=0x7fd24c147060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd250f9a700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd24c15c570 0x7fd24c147060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd250f9a700 1 -- 172.21.15.60:0/957854546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c0b4e20 msgr2=0x7fd24c147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd250f9a700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c0b4e20 0x7fd24c147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd250f9a700 1 -- 172.21.15.60:0/957854546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd248009550 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd250f9a700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c0cd2e0 0x7fd24c0c7c50 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00c4b0 tx=0x7fd23c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.249+0000 7fd241ffb700 1 -- 172.21.15.60:0/957854546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0052d0 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.250+0000 7fd241ffb700 1 -- 172.21.15.60:0/957854546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c005430 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.250+0000 7fd241ffb700 1 -- 172.21.15.60:0/957854546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0038c0 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.250+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c0c8db0 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.250+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c0c8190 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.251+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c0c92d0 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.252+0000 7fd241ffb700 1 -- 172.21.15.60:0/957854546 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd23c003a20 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.252+0000 7fd241ffb700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd23006ace0 0x7fd23006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.253+0000 7fd241ffb700 1 -- 172.21.15.60:0/957854546 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd23c088a40 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.253+0000 7fd25179b700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd23006ace0 0x7fd23006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.253+0000 7fd25179b700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd23006ace0 0x7fd23006d190 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd248014040 tx=0x7fd248008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.257+0000 7fd241ffb700 1 -- 172.21.15.60:0/957854546 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fd24c0c92d0 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.508+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd24c0c92d0 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.509+0000 7fd241ffb700 1 -- 172.21.15.60:0/957854546 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd24c0c92d0 con 0x7fd24c0cd2e0 2023-07-13T22:32:32.494 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:32.494 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:32.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd23006ace0 msgr2=0x7fd23006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:32.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd23006ace0 0x7fd23006d190 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd248014040 tx=0x7fd248008030 comp rx=0 tx=0).stop 2023-07-13T22:32:32.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c0cd2e0 msgr2=0x7fd24c0c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:32.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c0cd2e0 0x7fd24c0c7c50 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00c4b0 tx=0x7fd23c00c870 comp rx=0 tx=0).stop 2023-07-13T22:32:32.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 shutdown_connections 2023-07-13T22:32:32.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fd23006ace0 0x7fd23006d190 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd24c15c570 0x7fd24c147060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c0b4e20 0x7fd24c147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 --2- 172.21.15.60:0/957854546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c0cd2e0 0x7fd24c0c7c50 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:32.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 >> 172.21.15.60:0/957854546 conn(0x7fd24c06b0f0 msgr2=0x7fd24c0b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:32.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 shutdown_connections 2023-07-13T22:32:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:32.511+0000 7fd2539ff700 1 -- 172.21.15.60:0/957854546 wait complete. 2023-07-13T22:32:32.514 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:32.514 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:37.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.745+0000 7f2501bff700 1 Processor -- start 2023-07-13T22:32:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.745+0000 7f2501bff700 1 -- start start 2023-07-13T22:32:37.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.746+0000 7f2501bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24fc160af0 0x7f24fc0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:37.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.746+0000 7f2501bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0ade10 0x7f24fc160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:37.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.746+0000 7f2501bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24fc0b0dc0 con 0x7f24fc0b0840 2023-07-13T22:32:37.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.746+0000 7f2501bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24fc164600 con 0x7f24fc160af0 2023-07-13T22:32:37.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.746+0000 7f2501bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1648e0 con 0x7f24fc0ade10 2023-07-13T22:32:37.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.747+0000 7f24fb7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24fc160af0 0x7f24fc0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:37.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.747+0000 7f24fb7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24fc160af0 0x7f24fc0b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34168/0 (socket says 172.21.15.60:34168) 2023-07-13T22:32:37.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.747+0000 7f24fb7fe700 1 -- 172.21.15.60:0/3926448281 learned_addr learned my addr 172.21.15.60:0/3926448281 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:37.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.747+0000 7f24fbfff700 1 --2- 172.21.15.60:0/3926448281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0ade10 0x7f24fc160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.747+0000 7f24fb7fe700 1 -- 172.21.15.60:0/3926448281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0ade10 msgr2=0x7f24fc160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.747+0000 7f24fb7fe700 1 --2- 172.21.15.60:0/3926448281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0ade10 0x7f24fc160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.747+0000 7f24fb7fe700 1 -- 172.21.15.60:0/3926448281 >> v1:172.21.15.105:6789/0 conn(0x7f24fc0b0840 legacy=0x7f24fc0ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.747+0000 7f24fb7fe700 1 -- 172.21.15.60:0/3926448281 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc164bc0 con 0x7f24fc160af0 2023-07-13T22:32:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.747+0000 7f24fb7fe700 1 --2- 172.21.15.60:0/3926448281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc160af0 0x7f24fc0b0300 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0098f0 tx=0x7f24ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7f1ae77b860431 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f24fa7fc700 1 -- 172.21.15.60:0/3926448281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00b8a0 con 0x7f24fc160af0 2023-07-13T22:32:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f24fa7fc700 1 -- 172.21.15.60:0/3926448281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec00ba00 con 0x7f24fc160af0 2023-07-13T22:32:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f24fa7fc700 1 -- 172.21.15.60:0/3926448281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00bd90 con 0x7f24fc160af0 2023-07-13T22:32:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f2501bff700 1 -- 172.21.15.60:0/3926448281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc160af0 msgr2=0x7f24fc0b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f2501bff700 1 --2- 172.21.15.60:0/3926448281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc160af0 0x7f24fc0b0300 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0098f0 tx=0x7f24ec009c00 comp rx=0 tx=0).stop 2023-07-13T22:32:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f2501bff700 1 -- 172.21.15.60:0/3926448281 shutdown_connections 2023-07-13T22:32:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f2501bff700 1 --2- 172.21.15.60:0/3926448281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0ade10 0x7f24fc160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f2501bff700 1 --2- 172.21.15.60:0/3926448281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc160af0 0x7f24fc0b0300 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f2501bff700 1 -- 172.21.15.60:0/3926448281 >> 172.21.15.60:0/3926448281 conn(0x7f24fc0cb300 msgr2=0x7f24fc0cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:37.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f2501bff700 1 -- 172.21.15.60:0/3926448281 shutdown_connections 2023-07-13T22:32:37.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.748+0000 7f2501bff700 1 -- 172.21.15.60:0/3926448281 wait complete. 2023-07-13T22:32:37.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.750+0000 7f2501bff700 1 Processor -- start 2023-07-13T22:32:37.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.750+0000 7f2501bff700 1 -- start start 2023-07-13T22:32:37.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.751+0000 7f2501bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc0ade10 0x7f24fc1f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:37.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.751+0000 7f2501bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0b0840 0x7f24fc0b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:37.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.751+0000 7f2501bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc160af0 0x7f24fc1ec430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:37.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.751+0000 7f2501bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1eb090 con 0x7f24fc0ade10 2023-07-13T22:32:37.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.751+0000 7f2501bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1eb370 con 0x7f24fc160af0 2023-07-13T22:32:37.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.751+0000 7f2501bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1eb650 con 0x7f24fc0b0840 2023-07-13T22:32:37.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.751+0000 7f24fb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc0ade10 0x7f24fc1f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:37.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.751+0000 7f24fb7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc0ade10 0x7f24fc1f9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34180/0 (socket says 172.21.15.60:34180) 2023-07-13T22:32:37.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.751+0000 7f24fb7fe700 1 -- 172.21.15.60:0/578657987 learned_addr learned my addr 172.21.15.60:0/578657987 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:37.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24faffd700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0b0840 0x7f24fc0b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:37.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24fbfff700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc160af0 0x7f24fc1ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:37.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24fb7fe700 1 -- 172.21.15.60:0/578657987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0b0840 msgr2=0x7f24fc0b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:37.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24fb7fe700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0b0840 0x7f24fc0b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:37.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24fb7fe700 1 -- 172.21.15.60:0/578657987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc160af0 msgr2=0x7f24fc1ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:37.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24fb7fe700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc160af0 0x7f24fc1ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:37.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24fb7fe700 1 -- 172.21.15.60:0/578657987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec009550 con 0x7f24fc0ade10 2023-07-13T22:32:37.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24fb7fe700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc0ade10 0x7f24fc1f9c10 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f24ec000c00 tx=0x7f24ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:37.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24e3fff700 1 -- 172.21.15.60:0/578657987 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec004480 con 0x7f24fc0ade10 2023-07-13T22:32:37.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24e3fff700 1 -- 172.21.15.60:0/578657987 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec01f070 con 0x7f24fc0ade10 2023-07-13T22:32:37.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f24e3fff700 1 -- 172.21.15.60:0/578657987 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec0038c0 con 0x7f24fc0ade10 2023-07-13T22:32:37.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc1eb930 con 0x7f24fc0ade10 2023-07-13T22:32:37.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.752+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc0b1bf0 con 0x7f24fc0ade10 2023-07-13T22:32:37.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.753+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc1ebac0 con 0x7f24fc0ade10 2023-07-13T22:32:37.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.755+0000 7f24e3fff700 1 -- 172.21.15.60:0/578657987 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f24ec021030 con 0x7f24fc0ade10 2023-07-13T22:32:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.755+0000 7f24e3fff700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24e806ace0 0x7f24e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.756+0000 7f24e3fff700 1 -- 172.21.15.60:0/578657987 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f24ec08aaa0 con 0x7f24fc0ade10 2023-07-13T22:32:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.756+0000 7f24faffd700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24e806ace0 0x7f24e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.756+0000 7f24faffd700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24e806ace0 0x7f24e806d190 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0ce010 tx=0x7f24f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:37.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:37.760+0000 7f24e3fff700 1 -- 172.21.15.60:0/578657987 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f24ec055c20 con 0x7f24fc0ade10 2023-07-13T22:32:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.008+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f24fc0c9b50 con 0x7f24fc0ade10 2023-07-13T22:32:37.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.009+0000 7f24e3fff700 1 -- 172.21.15.60:0/578657987 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f24ec059240 con 0x7f24fc0ade10 2023-07-13T22:32:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:37.994 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24e806ace0 msgr2=0x7f24e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24e806ace0 0x7f24e806d190 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0ce010 tx=0x7f24f0009040 comp rx=0 tx=0).stop 2023-07-13T22:32:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc0ade10 msgr2=0x7f24fc1f9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc0ade10 0x7f24fc1f9c10 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f24ec000c00 tx=0x7f24ec004750 comp rx=0 tx=0).stop 2023-07-13T22:32:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 shutdown_connections 2023-07-13T22:32:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f24e806ace0 0x7f24e806d190 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc0ade10 0x7f24fc1f9c10 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc160af0 0x7f24fc1ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 --2- 172.21.15.60:0/578657987 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24fc0b0840 0x7f24fc0b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 >> 172.21.15.60:0/578657987 conn(0x7f24fc0cb300 msgr2=0x7f24fc0ca8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 shutdown_connections 2023-07-13T22:32:37.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:38.011+0000 7f2501bff700 1 -- 172.21.15.60:0/578657987 wait complete. 2023-07-13T22:32:38.014 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:38.014 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:43.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:43.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.243+0000 7fdea2bff700 1 Processor -- start 2023-07-13T22:32:43.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.243+0000 7fdea2bff700 1 -- start start 2023-07-13T22:32:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fdea2bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fde9c06c9a0 0x7fde9c0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fdea2bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde9c0b2ff0 0x7fde9c160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fdea2bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c15dd70 con 0x7fde9c15c940 2023-07-13T22:32:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fdea2bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c06ce20 con 0x7fde9c06c9a0 2023-07-13T22:32:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fdea2bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c15d950 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fde9bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fde9c06c9a0 0x7fde9c0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fde9bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fde9c06c9a0 0x7fde9c0b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34184/0 (socket says 172.21.15.60:34184) 2023-07-13T22:32:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fde9bfff700 1 -- 172.21.15.60:0/1922547326 learned_addr learned my addr 172.21.15.60:0/1922547326 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fdea119c700 1 --2- 172.21.15.60:0/1922547326 >> v2:172.21.15.105:3300/0 conn(0x7fde9c0b2ff0 0x7fde9c160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:43.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fde9bfff700 1 -- 172.21.15.60:0/1922547326 >> v2:172.21.15.105:3300/0 conn(0x7fde9c0b2ff0 msgr2=0x7fde9c160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.245+0000 7fde9bfff700 1 --2- 172.21.15.60:0/1922547326 >> v2:172.21.15.105:3300/0 conn(0x7fde9c0b2ff0 0x7fde9c160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.246+0000 7fde9bfff700 1 -- 172.21.15.60:0/1922547326 >> v1:172.21.15.105:6789/0 conn(0x7fde9c15c940 legacy=0x7fde9c06c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.246+0000 7fde9bfff700 1 -- 172.21.15.60:0/1922547326 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0b3410 con 0x7fde9c06c9a0 2023-07-13T22:32:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.246+0000 7fde9bfff700 1 --2- 172.21.15.60:0/1922547326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c06c9a0 0x7fde9c0b2ab0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0afae0 tx=0x7fde88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c532e9978e43670f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.246+0000 7fde9b7fe700 1 -- 172.21.15.60:0/1922547326 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88011830 con 0x7fde9c06c9a0 2023-07-13T22:32:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.246+0000 7fde9b7fe700 1 -- 172.21.15.60:0/1922547326 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde88011990 con 0x7fde9c06c9a0 2023-07-13T22:32:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.246+0000 7fde9b7fe700 1 -- 172.21.15.60:0/1922547326 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88011d20 con 0x7fde9c06c9a0 2023-07-13T22:32:43.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.247+0000 7fdea2bff700 1 -- 172.21.15.60:0/1922547326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c06c9a0 msgr2=0x7fde9c0b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:43.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.247+0000 7fdea2bff700 1 --2- 172.21.15.60:0/1922547326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c06c9a0 0x7fde9c0b2ab0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0afae0 tx=0x7fde88009bd0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.247+0000 7fdea2bff700 1 -- 172.21.15.60:0/1922547326 shutdown_connections 2023-07-13T22:32:43.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.247+0000 7fdea2bff700 1 --2- 172.21.15.60:0/1922547326 >> v2:172.21.15.105:3300/0 conn(0x7fde9c0b2ff0 0x7fde9c160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.247+0000 7fdea2bff700 1 --2- 172.21.15.60:0/1922547326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c06c9a0 0x7fde9c0b2ab0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.247+0000 7fdea2bff700 1 -- 172.21.15.60:0/1922547326 >> 172.21.15.60:0/1922547326 conn(0x7fde9c06dd10 msgr2=0x7fde9c06e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:43.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.247+0000 7fdea2bff700 1 -- 172.21.15.60:0/1922547326 shutdown_connections 2023-07-13T22:32:43.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.247+0000 7fdea2bff700 1 -- 172.21.15.60:0/1922547326 wait complete. 2023-07-13T22:32:43.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.249+0000 7fdea2bff700 1 Processor -- start 2023-07-13T22:32:43.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.249+0000 7fdea2bff700 1 -- start start 2023-07-13T22:32:43.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea2bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c0b2ff0 0x7fde9c14df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:43.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea2bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde9c15c940 0x7fde9c0c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:43.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea2bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde9c15ae50 0x7fde9c0c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:43.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea2bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0c5b40 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea2bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0c5f60 con 0x7fde9c15ae50 2023-07-13T22:32:43.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea2bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0c4ec0 con 0x7fde9c15c940 2023-07-13T22:32:43.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea099b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c0b2ff0 0x7fde9c14df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea099b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c0b2ff0 0x7fde9c14df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34196/0 (socket says 172.21.15.60:34196) 2023-07-13T22:32:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea099b700 1 -- 172.21.15.60:0/351752780 learned_addr learned my addr 172.21.15.60:0/351752780 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea119c700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde9c15ae50 0x7fde9c0c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fde9bfff700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde9c15c940 0x7fde9c0c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea099b700 1 -- 172.21.15.60:0/351752780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde9c15c940 msgr2=0x7fde9c0c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea099b700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde9c15c940 0x7fde9c0c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea099b700 1 -- 172.21.15.60:0/351752780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde9c15ae50 msgr2=0x7fde9c0c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:43.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea099b700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde9c15ae50 0x7fde9c0c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.250+0000 7fdea099b700 1 -- 172.21.15.60:0/351752780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde88009550 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.251+0000 7fdea099b700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c0b2ff0 0x7fde9c14df40 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fde9000c4e0 tx=0x7fde9000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:43.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.251+0000 7fde997fa700 1 -- 172.21.15.60:0/351752780 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde900052d0 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.251+0000 7fde997fa700 1 -- 172.21.15.60:0/351752780 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde90005430 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.251+0000 7fde997fa700 1 -- 172.21.15.60:0/351752780 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde900038c0 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.251+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c0c51a0 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.251+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c0c5760 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.252+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c074220 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.253+0000 7fde997fa700 1 -- 172.21.15.60:0/351752780 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde90003a20 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.254+0000 7fde997fa700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8006ace0 0x7fde8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:43.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.254+0000 7fde997fa700 1 -- 172.21.15.60:0/351752780 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fde900889f0 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.254+0000 7fde9bfff700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8006ace0 0x7fde8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:43.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.255+0000 7fde9bfff700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8006ace0 0x7fde8006d190 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fde88014040 tx=0x7fde88008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:43.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.258+0000 7fde997fa700 1 -- 172.21.15.60:0/351752780 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fde90053a50 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.513+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fde9c0704e0 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.513+0000 7fde997fa700 1 -- 172.21.15.60:0/351752780 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fde90057070 con 0x7fde9c0b2ff0 2023-07-13T22:32:43.499 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:43.499 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8006ace0 msgr2=0x7fde8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8006ace0 0x7fde8006d190 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fde88014040 tx=0x7fde88008030 comp rx=0 tx=0).stop 2023-07-13T22:32:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c0b2ff0 msgr2=0x7fde9c14df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c0b2ff0 0x7fde9c14df40 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fde9000c4e0 tx=0x7fde9000c8a0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 shutdown_connections 2023-07-13T22:32:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fde8006ace0 0x7fde8006d190 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde9c0b2ff0 0x7fde9c14df40 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde9c15ae50 0x7fde9c0c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 --2- 172.21.15.60:0/351752780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde9c15c940 0x7fde9c0c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 >> 172.21.15.60:0/351752780 conn(0x7fde9c06dd10 msgr2=0x7fde9c15f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 shutdown_connections 2023-07-13T22:32:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:43.516+0000 7fdea2bff700 1 -- 172.21.15.60:0/351752780 wait complete. 2023-07-13T22:32:43.520 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:43.520 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:48.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.752+0000 7fecbe7ff700 1 Processor -- start 2023-07-13T22:32:48.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.753+0000 7fecbe7ff700 1 -- start start 2023-07-13T22:32:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecbe7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80cf850 0x7fecb80cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecbe7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fecb815b710 0x7fecb80b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecbe7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fecb80b2ab0 0x7fecb80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:48.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecbe7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb80b34f0 con 0x7fecb80b2ab0 2023-07-13T22:32:48.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecbe7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb815aa40 con 0x7fecb815b710 2023-07-13T22:32:48.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecbe7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb815b280 con 0x7fecb80cf850 2023-07-13T22:32:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecbcd9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fecb80b2ab0 0x7fecb80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecbcd9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fecb80b2ab0 0x7fecb80ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56760/0 (socket says 172.21.15.60:56760) 2023-07-13T22:32:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecbcd9c700 1 -- 172.21.15.60:0/2620460312 learned_addr learned my addr 172.21.15.60:0/2620460312 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecb7fff700 1 --2- 172.21.15.60:0/2620460312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80cf850 0x7fecb80cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.754+0000 7fecb77fe700 1 --2- 172.21.15.60:0/2620460312 >> v2:172.21.15.105:3300/0 conn(0x7fecb815b710 0x7fecb80b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.755+0000 7fecbcd9c700 1 -- 172.21.15.60:0/2620460312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80cf850 msgr2=0x7fecb80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:48.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.755+0000 7fecbcd9c700 1 --2- 172.21.15.60:0/2620460312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80cf850 0x7fecb80cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:48.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.755+0000 7fecbcd9c700 1 -- 172.21.15.60:0/2620460312 >> v2:172.21.15.105:3300/0 conn(0x7fecb815b710 msgr2=0x7fecb80b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:48.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.755+0000 7fecbcd9c700 1 --2- 172.21.15.60:0/2620460312 >> v2:172.21.15.105:3300/0 conn(0x7fecb815b710 0x7fecb80b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:48.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.755+0000 7fecbcd9c700 1 -- 172.21.15.60:0/2620460312 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb80b3210 con 0x7fecb80b2ab0 2023-07-13T22:32:48.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.755+0000 7fecbcd9c700 1 --2- 172.21.15.60:0/2620460312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb80b2ab0 0x7fecb80ce630 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fecb806f1e0 tx=0x7fecb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6962d3be88ed64e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:48.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.755+0000 7fecb6ffd700 1 -- 172.21.15.60:0/2620460312 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb000e070 con 0x7fecb80b2ab0 2023-07-13T22:32:48.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.755+0000 7fecb6ffd700 1 -- 172.21.15.60:0/2620460312 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb000b9c0 con 0x7fecb80b2ab0 2023-07-13T22:32:48.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.755+0000 7fecb6ffd700 1 -- 172.21.15.60:0/2620460312 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb000bd70 con 0x7fecb80b2ab0 2023-07-13T22:32:48.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.756+0000 7fecbe7ff700 1 -- 172.21.15.60:0/2620460312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb80b2ab0 msgr2=0x7fecb80ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:48.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.756+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/2620460312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb80b2ab0 0x7fecb80ce630 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fecb806f1e0 tx=0x7fecb000b330 comp rx=0 tx=0).stop 2023-07-13T22:32:48.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.756+0000 7fecbe7ff700 1 -- 172.21.15.60:0/2620460312 shutdown_connections 2023-07-13T22:32:48.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.756+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/2620460312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb80b2ab0 0x7fecb80ce630 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:48.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.756+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/2620460312 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80cf850 0x7fecb80cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:48.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.756+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/2620460312 >> v2:172.21.15.105:3300/0 conn(0x7fecb815b710 0x7fecb80b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:48.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.756+0000 7fecbe7ff700 1 -- 172.21.15.60:0/2620460312 >> 172.21.15.60:0/2620460312 conn(0x7fecb806dd10 msgr2=0x7fecb806e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:48.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.756+0000 7fecbe7ff700 1 -- 172.21.15.60:0/2620460312 shutdown_connections 2023-07-13T22:32:48.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.756+0000 7fecbe7ff700 1 -- 172.21.15.60:0/2620460312 wait complete. 2023-07-13T22:32:48.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.758+0000 7fecbe7ff700 1 Processor -- start 2023-07-13T22:32:48.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.758+0000 7fecbe7ff700 1 -- start start 2023-07-13T22:32:48.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbe7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80b2ab0 0x7fecb806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:48.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbe7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb80cf850 0x7fecb815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:48.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbe7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb815b710 0x7fecb806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:48.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbe7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb815c290 con 0x7fecb815b710 2023-07-13T22:32:48.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbe7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb806c8f0 con 0x7fecb80cf850 2023-07-13T22:32:48.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbe7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb81f3230 con 0x7fecb80b2ab0 2023-07-13T22:32:48.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbcd9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb815b710 0x7fecb806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:48.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbcd9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb815b710 0x7fecb806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56774/0 (socket says 172.21.15.60:56774) 2023-07-13T22:32:48.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbcd9c700 1 -- 172.21.15.60:0/126315087 learned_addr learned my addr 172.21.15.60:0/126315087 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:48.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecb7fff700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80b2ab0 0x7fecb806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:48.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecb77fe700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb80cf850 0x7fecb815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:48.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbcd9c700 1 -- 172.21.15.60:0/126315087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80b2ab0 msgr2=0x7fecb806c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:48.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbcd9c700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80b2ab0 0x7fecb806c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:48.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbcd9c700 1 -- 172.21.15.60:0/126315087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb80cf850 msgr2=0x7fecb815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:48.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbcd9c700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb80cf850 0x7fecb815c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.760+0000 7fecbcd9c700 1 -- 172.21.15.60:0/126315087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb0009b30 con 0x7fecb815b710 2023-07-13T22:32:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.761+0000 7fecbcd9c700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb815b710 0x7fecb806afc0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b730 tx=0x7fecb0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.761+0000 7fecb4ff9700 1 -- 172.21.15.60:0/126315087 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00099b0 con 0x7fecb815b710 2023-07-13T22:32:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.761+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb81582f0 con 0x7fecb815b710 2023-07-13T22:32:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.761+0000 7fecb4ff9700 1 -- 172.21.15.60:0/126315087 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb0004250 con 0x7fecb815b710 2023-07-13T22:32:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.761+0000 7fecb4ff9700 1 -- 172.21.15.60:0/126315087 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00045e0 con 0x7fecb815b710 2023-07-13T22:32:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.761+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb815bc40 con 0x7fecb815b710 2023-07-13T22:32:48.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.762+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb815c400 con 0x7fecb815b710 2023-07-13T22:32:48.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.763+0000 7fecb4ff9700 1 -- 172.21.15.60:0/126315087 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fecb0004740 con 0x7fecb815b710 2023-07-13T22:32:48.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.764+0000 7fecb4ff9700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feca406ace0 0x7feca406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:48.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.764+0000 7fecb4ff9700 1 -- 172.21.15.60:0/126315087 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fecb008a5e0 con 0x7fecb815b710 2023-07-13T22:32:48.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.764+0000 7fecb7fff700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feca406ace0 0x7feca406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:48.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.765+0000 7fecb7fff700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feca406ace0 0x7feca406d190 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7feca80045f0 tx=0x7feca8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:48.769+0000 7fecb4ff9700 1 -- 172.21.15.60:0/126315087 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fecb00555c0 con 0x7fecb815b710 2023-07-13T22:32:49.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.025+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fecb8074220 con 0x7fecb815b710 2023-07-13T22:32:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.025+0000 7fecb4ff9700 1 -- 172.21.15.60:0/126315087 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fecb0058be0 con 0x7fecb815b710 2023-07-13T22:32:49.011 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:49.011 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:49.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feca406ace0 msgr2=0x7feca406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:49.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feca406ace0 0x7feca406d190 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7feca80045f0 tx=0x7feca8008040 comp rx=0 tx=0).stop 2023-07-13T22:32:49.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb815b710 msgr2=0x7fecb806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb815b710 0x7fecb806afc0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b730 tx=0x7fecb0004ad0 comp rx=0 tx=0).stop 2023-07-13T22:32:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 shutdown_connections 2023-07-13T22:32:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feca406ace0 0x7feca406d190 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecb815b710 0x7fecb806afc0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecb80b2ab0 0x7fecb806c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 --2- 172.21.15.60:0/126315087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb80cf850 0x7fecb815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 >> 172.21.15.60:0/126315087 conn(0x7fecb806dd10 msgr2=0x7fecb80aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 shutdown_connections 2023-07-13T22:32:49.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:49.028+0000 7fecbe7ff700 1 -- 172.21.15.60:0/126315087 wait complete. 2023-07-13T22:32:49.032 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:49.032 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:54.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:54.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.260+0000 7effa99ff700 1 Processor -- start 2023-07-13T22:32:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.260+0000 7effa99ff700 1 -- start start 2023-07-13T22:32:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.261+0000 7effa99ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effa40b0a10 0x7effa40c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:54.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.261+0000 7effa99ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa40707c0 con 0x7effa40ca530 2023-07-13T22:32:54.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.261+0000 7effa99ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa415b700 con 0x7effa415a320 2023-07-13T22:32:54.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.261+0000 7effa99ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa415df00 con 0x7effa40b0a10 2023-07-13T22:32:54.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.261+0000 7effa37fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7effa40ca530 0x7effa406b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35888/0 (socket says 172.21.15.60:35888) 2023-07-13T22:32:54.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.261+0000 7effa27fc700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effa40b0a10 0x7effa40c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:54.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.261+0000 7effa37fe700 1 -- 172.21.15.60:0/3840616774 learned_addr learned my addr 172.21.15.60:0/3840616774 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:32:54.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.262+0000 7effa1ffb700 1 -- 172.21.15.60:0/3840616774 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769404801 0 0) 0x7effa40707c0 con 0x7effa40ca530 2023-07-13T22:32:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.262+0000 7effa1ffb700 1 -- 172.21.15.60:0/3840616774 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff8c0035b0 con 0x7effa40ca530 2023-07-13T22:32:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.262+0000 7effa27fc700 1 -- 172.21.15.60:0/3840616774 >> v1:172.21.15.105:6789/0 conn(0x7effa415a320 legacy=0x7effa40b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:32:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.262+0000 7effa27fc700 1 -- 172.21.15.60:0/3840616774 >> v1:172.21.15.60:6789/0 conn(0x7effa40ca530 legacy=0x7effa406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.262+0000 7effa27fc700 1 -- 172.21.15.60:0/3840616774 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa415e1e0 con 0x7effa40b0a10 2023-07-13T22:32:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.262+0000 7effa27fc700 1 --2- 172.21.15.60:0/3840616774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa40b0a10 0x7effa40c9ff0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7eff98009840 tx=0x7eff98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efbd2b3119e8ce3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:54.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.262+0000 7effa1ffb700 1 -- 172.21.15.60:0/3840616774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98004230 con 0x7effa40b0a10 2023-07-13T22:32:54.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.262+0000 7effa1ffb700 1 -- 172.21.15.60:0/3840616774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98004390 con 0x7effa40b0a10 2023-07-13T22:32:54.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.262+0000 7effa1ffb700 1 -- 172.21.15.60:0/3840616774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980036c0 con 0x7effa40b0a10 2023-07-13T22:32:54.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.263+0000 7effa99ff700 1 -- 172.21.15.60:0/3840616774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa40b0a10 msgr2=0x7effa40c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:54.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.263+0000 7effa99ff700 1 --2- 172.21.15.60:0/3840616774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa40b0a10 0x7effa40c9ff0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7eff98009840 tx=0x7eff98009b50 comp rx=0 tx=0).stop 2023-07-13T22:32:54.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.263+0000 7effa99ff700 1 -- 172.21.15.60:0/3840616774 shutdown_connections 2023-07-13T22:32:54.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.263+0000 7effa99ff700 1 --2- 172.21.15.60:0/3840616774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa40b0a10 0x7effa40c9ff0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:54.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.263+0000 7effa99ff700 1 -- 172.21.15.60:0/3840616774 >> 172.21.15.60:0/3840616774 conn(0x7effa40cb300 msgr2=0x7effa40cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:54.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.263+0000 7effa99ff700 1 -- 172.21.15.60:0/3840616774 shutdown_connections 2023-07-13T22:32:54.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.263+0000 7effa99ff700 1 -- 172.21.15.60:0/3840616774 wait complete. 2023-07-13T22:32:54.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.265+0000 7effa99ff700 1 Processor -- start 2023-07-13T22:32:54.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.265+0000 7effa99ff700 1 -- start start 2023-07-13T22:32:54.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa99ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa40b0a10 0x7effa40cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:54.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa99ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa40ca530 0x7effa41548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:54.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa99ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa415a320 0x7effa40acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:54.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa99ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa4153820 con 0x7effa415a320 2023-07-13T22:32:54.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa99ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa4153b00 con 0x7effa40b0a10 2023-07-13T22:32:54.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa99ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa4153de0 con 0x7effa40ca530 2023-07-13T22:32:54.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa415a320 0x7effa40acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:54.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa415a320 0x7effa40acb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56796/0 (socket says 172.21.15.60:56796) 2023-07-13T22:32:54.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa37fe700 1 -- 172.21.15.60:0/3105721008 learned_addr learned my addr 172.21.15.60:0/3105721008 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:54.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa2ffd700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa40b0a10 0x7effa40cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:54.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa27fc700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa40ca530 0x7effa41548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:54.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa37fe700 1 -- 172.21.15.60:0/3105721008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa40ca530 msgr2=0x7effa41548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:54.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa37fe700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa40ca530 0x7effa41548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:54.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa37fe700 1 -- 172.21.15.60:0/3105721008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa40b0a10 msgr2=0x7effa40cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:54.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa37fe700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa40b0a10 0x7effa40cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:54.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa37fe700 1 -- 172.21.15.60:0/3105721008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980094a0 con 0x7effa415a320 2023-07-13T22:32:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.266+0000 7effa2ffd700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa40b0a10 0x7effa40cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:32:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.267+0000 7effa37fe700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa415a320 0x7effa40acb20 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00cb40 tx=0x7eff9c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.267+0000 7eff8bfff700 1 -- 172.21.15.60:0/3105721008 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00ab50 con 0x7effa415a320 2023-07-13T22:32:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.267+0000 7eff8bfff700 1 -- 172.21.15.60:0/3105721008 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00acb0 con 0x7effa415a320 2023-07-13T22:32:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.267+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa41540c0 con 0x7effa415a320 2023-07-13T22:32:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.267+0000 7eff8bfff700 1 -- 172.21.15.60:0/3105721008 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c012620 con 0x7effa415a320 2023-07-13T22:32:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.267+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa40ce350 con 0x7effa415a320 2023-07-13T22:32:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.268+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa41545e0 con 0x7effa415a320 2023-07-13T22:32:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.269+0000 7eff8bfff700 1 -- 172.21.15.60:0/3105721008 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff9c01b3b0 con 0x7effa415a320 2023-07-13T22:32:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.270+0000 7eff8bfff700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9006ace0 0x7eff9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.270+0000 7eff8bfff700 1 -- 172.21.15.60:0/3105721008 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7eff9c089280 con 0x7effa415a320 2023-07-13T22:32:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.270+0000 7effa2ffd700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9006ace0 0x7eff9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.271+0000 7effa2ffd700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9006ace0 0x7eff9006d190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7eff94006fd0 tx=0x7eff94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.275+0000 7eff8bfff700 1 -- 172.21.15.60:0/3105721008 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7effa41545e0 con 0x7effa415a320 2023-07-13T22:32:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.526+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7effa41545e0 con 0x7effa415a320 2023-07-13T22:32:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.527+0000 7eff8bfff700 1 -- 172.21.15.60:0/3105721008 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7effa41545e0 con 0x7effa415a320 2023-07-13T22:32:54.513 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:32:54.513 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:32:54.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9006ace0 msgr2=0x7eff9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:54.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9006ace0 0x7eff9006d190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7eff94006fd0 tx=0x7eff94008040 comp rx=0 tx=0).stop 2023-07-13T22:32:54.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa415a320 msgr2=0x7effa40acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:54.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa415a320 0x7effa40acb20 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00cb40 tx=0x7eff9c00cf00 comp rx=0 tx=0).stop 2023-07-13T22:32:54.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 shutdown_connections 2023-07-13T22:32:54.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7eff9006ace0 0x7eff9006d190 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:54.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa415a320 0x7effa40acb20 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:54.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effa40b0a10 0x7effa40cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:54.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 --2- 172.21.15.60:0/3105721008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7effa40ca530 0x7effa41548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:54.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 >> 172.21.15.60:0/3105721008 conn(0x7effa40cb300 msgr2=0x7effa4071370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:54.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 shutdown_connections 2023-07-13T22:32:54.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:54.529+0000 7effa99ff700 1 -- 172.21.15.60:0/3105721008 wait complete. 2023-07-13T22:32:54.532 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:32:54.533 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:32:58.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:32:58.536 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:32:59.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:32:59.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.764+0000 7fa6697ff700 1 Processor -- start 2023-07-13T22:32:59.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.765+0000 7fa6697ff700 1 -- start start 2023-07-13T22:32:59.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.766+0000 7fa6697ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6640cea40 0x7fa664069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:59.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.766+0000 7fa6697ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa664070ba0 con 0x7fa66406cc00 2023-07-13T22:32:59.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.766+0000 7fa6697ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6640ceeb0 con 0x7fa66415c4b0 2023-07-13T22:32:59.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.766+0000 7fa6697ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66415c880 con 0x7fa6640cea40 2023-07-13T22:32:59.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.766+0000 7fa6637fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa66406cc00 0x7fa6640b1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58320/0 (socket says 172.21.15.60:58320) 2023-07-13T22:32:59.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.766+0000 7fa6637fe700 1 -- 172.21.15.60:0/3303491260 learned_addr learned my addr 172.21.15.60:0/3303491260 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:32:59.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.766+0000 7fa6627fc700 1 --2- 172.21.15.60:0/3303491260 >> v2:172.21.15.105:3300/0 conn(0x7fa6640cea40 0x7fa664069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa661ffb700 1 -- 172.21.15.60:0/3303491260 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149141445 0 0) 0x7fa664070ba0 con 0x7fa66406cc00 2023-07-13T22:32:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa661ffb700 1 -- 172.21.15.60:0/3303491260 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa64c0035b0 con 0x7fa66406cc00 2023-07-13T22:32:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa6627fc700 1 -- 172.21.15.60:0/3303491260 >> v1:172.21.15.105:6789/0 conn(0x7fa66415c4b0 legacy=0x7fa6640ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa661ffb700 1 -- 172.21.15.60:0/3303491260 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684632270 0 0) 0x7fa6640ceeb0 con 0x7fa66415c4b0 2023-07-13T22:32:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa6627fc700 1 -- 172.21.15.60:0/3303491260 >> v1:172.21.15.60:6789/0 conn(0x7fa66406cc00 legacy=0x7fa6640b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa6627fc700 1 -- 172.21.15.60:0/3303491260 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6640cf190 con 0x7fa6640cea40 2023-07-13T22:32:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa6627fc700 1 --2- 172.21.15.60:0/3303491260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6640cea40 0x7fa664069eb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa658009840 tx=0x7fa658009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7436eb01b256bc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa661ffb700 1 -- 172.21.15.60:0/3303491260 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658011830 con 0x7fa6640cea40 2023-07-13T22:32:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa661ffb700 1 -- 172.21.15.60:0/3303491260 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa658011990 con 0x7fa6640cea40 2023-07-13T22:32:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.767+0000 7fa661ffb700 1 -- 172.21.15.60:0/3303491260 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658011d20 con 0x7fa6640cea40 2023-07-13T22:32:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.768+0000 7fa6697ff700 1 -- 172.21.15.60:0/3303491260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6640cea40 msgr2=0x7fa664069eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.768+0000 7fa6697ff700 1 --2- 172.21.15.60:0/3303491260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6640cea40 0x7fa664069eb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa658009840 tx=0x7fa658009b50 comp rx=0 tx=0).stop 2023-07-13T22:32:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.768+0000 7fa6697ff700 1 -- 172.21.15.60:0/3303491260 shutdown_connections 2023-07-13T22:32:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.768+0000 7fa6697ff700 1 --2- 172.21.15.60:0/3303491260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6640cea40 0x7fa664069eb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.768+0000 7fa6697ff700 1 -- 172.21.15.60:0/3303491260 >> 172.21.15.60:0/3303491260 conn(0x7fa6640d05b0 msgr2=0x7fa6640cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:32:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.768+0000 7fa6697ff700 1 -- 172.21.15.60:0/3303491260 shutdown_connections 2023-07-13T22:32:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.768+0000 7fa6697ff700 1 -- 172.21.15.60:0/3303491260 wait complete. 2023-07-13T22:32:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.770+0000 7fa6697ff700 1 Processor -- start 2023-07-13T22:32:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.770+0000 7fa6697ff700 1 -- start start 2023-07-13T22:32:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6697ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa66406cc00 0x7fa664154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6697ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6640cea40 0x7fa664147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:59.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6697ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa66415c4b0 0x7fa6640c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6697ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa664155660 con 0x7fa6640cea40 2023-07-13T22:32:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6697ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640b4ed0 con 0x7fa66415c4b0 2023-07-13T22:32:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6697ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6641482d0 con 0x7fa66406cc00 2023-07-13T22:32:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6627fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6640cea40 0x7fa664147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6627fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6640cea40 0x7fa664147d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52820/0 (socket says 172.21.15.60:52820) 2023-07-13T22:32:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6627fc700 1 -- 172.21.15.60:0/2877438180 learned_addr learned my addr 172.21.15.60:0/2877438180 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:32:59.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6627fc700 1 -- 172.21.15.60:0/2877438180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa66406cc00 msgr2=0x7fa664154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:59.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6627fc700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa66406cc00 0x7fa664154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:59.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6627fc700 1 -- 172.21.15.60:0/2877438180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa66415c4b0 msgr2=0x7fa6640c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:32:59.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6627fc700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa66415c4b0 0x7fa6640c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:32:59.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.771+0000 7fa6627fc700 1 -- 172.21.15.60:0/2877438180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6580094a0 con 0x7fa6640cea40 2023-07-13T22:32:59.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.772+0000 7fa6627fc700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6640cea40 0x7fa664147d90 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fa6580090b0 tx=0x7fa658004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.772+0000 7fa64b7fe700 1 -- 172.21.15.60:0/2877438180 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65801a910 con 0x7fa6640cea40 2023-07-13T22:32:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.772+0000 7fa64b7fe700 1 -- 172.21.15.60:0/2877438180 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65801aa70 con 0x7fa6640cea40 2023-07-13T22:32:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.772+0000 7fa64b7fe700 1 -- 172.21.15.60:0/2877438180 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65801ade0 con 0x7fa6640cea40 2023-07-13T22:32:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.772+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6640c6160 con 0x7fa6640cea40 2023-07-13T22:32:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.772+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6640b45a0 con 0x7fa6640cea40 2023-07-13T22:32:59.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.773+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa664074220 con 0x7fa6640cea40 2023-07-13T22:32:59.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.774+0000 7fa64b7fe700 1 -- 172.21.15.60:0/2877438180 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa658022070 con 0x7fa6640cea40 2023-07-13T22:32:59.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.775+0000 7fa64b7fe700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa65006ace0 0x7fa65006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:32:59.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.775+0000 7fa64b7fe700 1 -- 172.21.15.60:0/2877438180 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa65808a230 con 0x7fa6640cea40 2023-07-13T22:32:59.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.775+0000 7fa662ffd700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa65006ace0 0x7fa65006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:32:59.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.776+0000 7fa662ffd700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa65006ace0 0x7fa65006d190 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa654007fd0 tx=0x7fa6540079a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:32:59.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:32:59.780+0000 7fa64b7fe700 1 -- 172.21.15.60:0/2877438180 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa6580553e0 con 0x7fa6640cea40 2023-07-13T22:33:00.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.028+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa6640b5010 con 0x7fa6640cea40 2023-07-13T22:33:00.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.028+0000 7fa64b7fe700 1 -- 172.21.15.60:0/2877438180 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa658015040 con 0x7fa6640cea40 2023-07-13T22:33:00.014 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:00.015 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:33:00.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa65006ace0 msgr2=0x7fa65006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:00.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa65006ace0 0x7fa65006d190 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa654007fd0 tx=0x7fa6540079a0 comp rx=0 tx=0).stop 2023-07-13T22:33:00.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6640cea40 msgr2=0x7fa664147d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:00.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6640cea40 0x7fa664147d90 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fa6580090b0 tx=0x7fa658004750 comp rx=0 tx=0).stop 2023-07-13T22:33:00.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 shutdown_connections 2023-07-13T22:33:00.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa65006ace0 0x7fa65006d190 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:00.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa66415c4b0 0x7fa6640c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:00.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa66406cc00 0x7fa664154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:00.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 --2- 172.21.15.60:0/2877438180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6640cea40 0x7fa664147d90 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:00.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 >> 172.21.15.60:0/2877438180 conn(0x7fa6640d05b0 msgr2=0x7fa6640ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:00.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 shutdown_connections 2023-07-13T22:33:00.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:00.031+0000 7fa6697ff700 1 -- 172.21.15.60:0/2877438180 wait complete. 2023-07-13T22:33:00.034 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:33:00.035 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-13T22:33:05.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-13T22:33:05.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.264+0000 7f5a3abff700 1 Processor -- start 2023-07-13T22:33:05.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.264+0000 7f5a3abff700 1 -- start start 2023-07-13T22:33:05.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.265+0000 7f5a3abff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a340cd060 0x7f5a340cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:05.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.265+0000 7f5a3abff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a340cdb80 con 0x7f5a34158970 2023-07-13T22:33:05.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.265+0000 7f5a3abff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a340716c0 con 0x7f5a340ca720 2023-07-13T22:33:05.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.265+0000 7f5a3abff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a340cb1c0 con 0x7f5a340cd060 2023-07-13T22:33:05.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.265+0000 7f5a3899b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5a34158970 0x7f5a340ce010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46208/0 (socket says 172.21.15.60:46208) 2023-07-13T22:33:05.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.265+0000 7f5a3899b700 1 -- 172.21.15.60:0/3963224693 learned_addr learned my addr 172.21.15.60:0/3963224693 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.266+0000 7f5a33fff700 1 --2- 172.21.15.60:0/3963224693 >> v2:172.21.15.60:3300/0 conn(0x7f5a340cd060 0x7f5a340cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.266+0000 7f5a337fe700 1 -- 172.21.15.60:0/3963224693 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2193065643 0 0) 0x7f5a340cdb80 con 0x7f5a34158970 2023-07-13T22:33:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.266+0000 7f5a337fe700 1 -- 172.21.15.60:0/3963224693 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a240057c0 con 0x7f5a34158970 2023-07-13T22:33:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.266+0000 7f5a33fff700 1 -- 172.21.15.60:0/3963224693 >> v1:172.21.15.105:6789/0 conn(0x7f5a340ca720 legacy=0x7f5a3406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.266+0000 7f5a33fff700 1 -- 172.21.15.60:0/3963224693 >> v1:172.21.15.60:6789/0 conn(0x7f5a34158970 legacy=0x7f5a340ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.266+0000 7f5a33fff700 1 -- 172.21.15.60:0/3963224693 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a340cac00 con 0x7f5a340cd060 2023-07-13T22:33:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.266+0000 7f5a33fff700 1 --2- 172.21.15.60:0/3963224693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 0x7f5a340cc450 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009840 tx=0x7f5a20009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0e257eaf48481f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a337fe700 1 -- 172.21.15.60:0/3963224693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20011830 con 0x7f5a340cd060 2023-07-13T22:33:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a337fe700 1 -- 172.21.15.60:0/3963224693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a20011990 con 0x7f5a340cd060 2023-07-13T22:33:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a337fe700 1 -- 172.21.15.60:0/3963224693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20011d20 con 0x7f5a340cd060 2023-07-13T22:33:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a3abff700 1 -- 172.21.15.60:0/3963224693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 msgr2=0x7f5a340cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a3abff700 1 --2- 172.21.15.60:0/3963224693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 0x7f5a340cc450 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009840 tx=0x7f5a20009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a3abff700 1 -- 172.21.15.60:0/3963224693 shutdown_connections 2023-07-13T22:33:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a3abff700 1 --2- 172.21.15.60:0/3963224693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 0x7f5a340cc450 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a3abff700 1 -- 172.21.15.60:0/3963224693 >> 172.21.15.60:0/3963224693 conn(0x7f5a340cb300 msgr2=0x7f5a340cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a3abff700 1 -- 172.21.15.60:0/3963224693 shutdown_connections 2023-07-13T22:33:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.267+0000 7f5a3abff700 1 -- 172.21.15.60:0/3963224693 wait complete. 2023-07-13T22:33:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.269+0000 7f5a3abff700 1 Processor -- start 2023-07-13T22:33:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.269+0000 7f5a3abff700 1 -- start start 2023-07-13T22:33:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.270+0000 7f5a3abff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a340ca720 0x7f5a340af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.270+0000 7f5a3abff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 0x7f5a340afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.270+0000 7f5a3abff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a34158970 0x7f5a340c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.270+0000 7f5a3abff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a341f61b0 con 0x7f5a340cd060 2023-07-13T22:33:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.270+0000 7f5a3abff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a341f6490 con 0x7f5a340ca720 2023-07-13T22:33:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.270+0000 7f5a3abff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a341f6770 con 0x7f5a34158970 2023-07-13T22:33:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.270+0000 7f5a33fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 0x7f5a340afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a33fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 0x7f5a340afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55590/0 (socket says 172.21.15.60:55590) 2023-07-13T22:33:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a33fff700 1 -- 172.21.15.60:0/3761310779 learned_addr learned my addr 172.21.15.60:0/3761310779 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:05.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a3899b700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a340ca720 0x7f5a340af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:05.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a3919c700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a34158970 0x7f5a340c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:05.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a33fff700 1 -- 172.21.15.60:0/3761310779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a34158970 msgr2=0x7f5a340c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:05.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a33fff700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a34158970 0x7f5a340c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:05.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a33fff700 1 -- 172.21.15.60:0/3761310779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a340ca720 msgr2=0x7f5a340af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:05.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a33fff700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a340ca720 0x7f5a340af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:05.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a33fff700 1 -- 172.21.15.60:0/3761310779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a200094a0 con 0x7f5a340cd060 2023-07-13T22:33:05.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.271+0000 7f5a33fff700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 0x7f5a340afd00 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009fd0 tx=0x7f5a20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.272+0000 7f5a317fa700 1 -- 172.21.15.60:0/3761310779 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a200036a0 con 0x7f5a340cd060 2023-07-13T22:33:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.272+0000 7f5a317fa700 1 -- 172.21.15.60:0/3761310779 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2001f070 con 0x7f5a340cd060 2023-07-13T22:33:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.272+0000 7f5a317fa700 1 -- 172.21.15.60:0/3761310779 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20003a60 con 0x7f5a340cd060 2023-07-13T22:33:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.272+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a341591e0 con 0x7f5a340cd060 2023-07-13T22:33:05.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.272+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a34153f90 con 0x7f5a340cd060 2023-07-13T22:33:05.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.273+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a34159400 con 0x7f5a340cd060 2023-07-13T22:33:05.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.274+0000 7f5a317fa700 1 -- 172.21.15.60:0/3761310779 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a20003bc0 con 0x7f5a340cd060 2023-07-13T22:33:05.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.275+0000 7f5a317fa700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a1806ace0 0x7f5a1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:05.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.276+0000 7f5a317fa700 1 -- 172.21.15.60:0/3761310779 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5a20021070 con 0x7f5a340cd060 2023-07-13T22:33:05.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.276+0000 7f5a3899b700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a1806ace0 0x7f5a1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:05.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.276+0000 7f5a3899b700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a1806ace0 0x7f5a1806d190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5a28007fd0 tx=0x7f5a28007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:05.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.280+0000 7f5a317fa700 1 -- 172.21.15.60:0/3761310779 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f5a20055d60 con 0x7f5a340cd060 2023-07-13T22:33:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.530+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5a34074220 con 0x7f5a340cd060 2023-07-13T22:33:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.531+0000 7f5a317fa700 1 -- 172.21.15.60:0/3761310779 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5a20059380 con 0x7f5a340cd060 2023-07-13T22:33:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:05.517 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:33:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a1806ace0 msgr2=0x7f5a1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a1806ace0 0x7f5a1806d190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5a28007fd0 tx=0x7f5a28007dc0 comp rx=0 tx=0).stop 2023-07-13T22:33:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 msgr2=0x7f5a340afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 0x7f5a340afd00 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009fd0 tx=0x7f5a20004750 comp rx=0 tx=0).stop 2023-07-13T22:33:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 shutdown_connections 2023-07-13T22:33:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f5a1806ace0 0x7f5a1806d190 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a34158970 0x7f5a340c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a340ca720 0x7f5a340af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 --2- 172.21.15.60:0/3761310779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a340cd060 0x7f5a340afd00 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.533+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 >> 172.21.15.60:0/3761310779 conn(0x7f5a340cb300 msgr2=0x7f5a34157ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.534+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 shutdown_connections 2023-07-13T22:33:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:05.534+0000 7f5a3abff700 1 -- 172.21.15.60:0/3761310779 wait complete. 2023-07-13T22:33:05.537 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-13T22:33:05.538 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-13T22:33:05.539 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:33:05.634 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-13T22:33:05.751 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-13T22:33:05.752 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:33:05.752 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:33:05.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:33:06.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.058+0000 7fc1f31ff700 1 Processor -- start 2023-07-13T22:33:06.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.058+0000 7fc1f31ff700 1 -- start start 2023-07-13T22:33:06.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.059+0000 7fc1f31ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec15c6f0 0x7fc1ec160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.059+0000 7fc1f31ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1ec0cf010 0x7fc1ec0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.059+0000 7fc1f31ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1ec0722e0 con 0x7fc1ec0cfc00 2023-07-13T22:33:06.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.059+0000 7fc1f31ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ec0d0df0 con 0x7fc1ec0cf010 2023-07-13T22:33:06.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.059+0000 7fc1f31ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec0cdbd0 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1f179c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc1ec0cfc00 0x7fc1ec0b4760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46216/0 (socket says 172.21.15.60:46216) 2023-07-13T22:33:06.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1f179c700 1 -- 172.21.15.60:0/219546304 learned_addr learned my addr 172.21.15.60:0/219546304 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:06.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1ebfff700 1 --2- 172.21.15.60:0/219546304 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec0cf010 0x7fc1ec0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1f0f9b700 1 --2- 172.21.15.60:0/219546304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec15c6f0 0x7fc1ec160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1eb7fe700 1 -- 172.21.15.60:0/219546304 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590461657 0 0) 0x7fc1e4001490 con 0x7fc1ec0cfc00 2023-07-13T22:33:06.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1eb7fe700 1 -- 172.21.15.60:0/219546304 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1dc0035b0 con 0x7fc1ec0cfc00 2023-07-13T22:33:06.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1ebfff700 1 -- 172.21.15.60:0/219546304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec15c6f0 msgr2=0x7fc1ec160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:06.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1eb7fe700 1 -- 172.21.15.60:0/219546304 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 994747796 0 0) 0x7fc1dc0035b0 con 0x7fc1ec0cfc00 2023-07-13T22:33:06.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1ebfff700 1 --2- 172.21.15.60:0/219546304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec15c6f0 0x7fc1ec160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1ebfff700 1 -- 172.21.15.60:0/219546304 >> v1:172.21.15.60:6789/0 conn(0x7fc1ec0cfc00 legacy=0x7fc1ec0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1ebfff700 1 -- 172.21.15.60:0/219546304 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec15cac0 con 0x7fc1ec0cf010 2023-07-13T22:33:06.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.060+0000 7fc1f0f9b700 1 --2- 172.21.15.60:0/219546304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec15c6f0 0x7fc1ec160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:06.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.061+0000 7fc1ebfff700 1 --2- 172.21.15.60:0/219546304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0cf010 0x7fc1ec0cf6c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009840 tx=0x7fc1d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a73b7e83c913a9f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:06.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.061+0000 7fc1eb7fe700 1 -- 172.21.15.60:0/219546304 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4001b80 con 0x7fc1ec0cf010 2023-07-13T22:33:06.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.061+0000 7fc1eb7fe700 1 -- 172.21.15.60:0/219546304 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d8004030 con 0x7fc1ec0cf010 2023-07-13T22:33:06.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.061+0000 7fc1eb7fe700 1 -- 172.21.15.60:0/219546304 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80043a0 con 0x7fc1ec0cf010 2023-07-13T22:33:06.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.062+0000 7fc1f31ff700 1 -- 172.21.15.60:0/219546304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0cf010 msgr2=0x7fc1ec0cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.062+0000 7fc1f31ff700 1 --2- 172.21.15.60:0/219546304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0cf010 0x7fc1ec0cf6c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009840 tx=0x7fc1d8009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:06.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.062+0000 7fc1f31ff700 1 -- 172.21.15.60:0/219546304 shutdown_connections 2023-07-13T22:33:06.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.062+0000 7fc1f31ff700 1 --2- 172.21.15.60:0/219546304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec15c6f0 0x7fc1ec160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.062+0000 7fc1f31ff700 1 --2- 172.21.15.60:0/219546304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0cf010 0x7fc1ec0cf6c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.062+0000 7fc1f31ff700 1 -- 172.21.15.60:0/219546304 >> 172.21.15.60:0/219546304 conn(0x7fc1ec06a4b0 msgr2=0x7fc1ec06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:06.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.062+0000 7fc1f31ff700 1 -- 172.21.15.60:0/219546304 shutdown_connections 2023-07-13T22:33:06.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.062+0000 7fc1f31ff700 1 -- 172.21.15.60:0/219546304 wait complete. 2023-07-13T22:33:06.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.064+0000 7fc1f31ff700 1 Processor -- start 2023-07-13T22:33:06.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.064+0000 7fc1f31ff700 1 -- start start 2023-07-13T22:33:06.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f31ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0cf010 0x7fc1ec0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f31ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0cfc00 0x7fc1ec1f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f31ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1ec15c6f0 0x7fc1ec0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f31ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec150a00 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f31ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec0b8580 con 0x7fc1ec0cf010 2023-07-13T22:33:06.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f31ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec1f2050 con 0x7fc1ec0cfc00 2023-07-13T22:33:06.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1ec15c6f0 0x7fc1ec0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1ec15c6f0 0x7fc1ec0b75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55604/0 (socket says 172.21.15.60:55604) 2023-07-13T22:33:06.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f179c700 1 -- 172.21.15.60:0/2560174612 learned_addr learned my addr 172.21.15.60:0/2560174612 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:06.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1f0f9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0cf010 0x7fc1ec0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.065+0000 7fc1ebfff700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0cfc00 0x7fc1ec1f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1f179c700 1 -- 172.21.15.60:0/2560174612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0cfc00 msgr2=0x7fc1ec1f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1f179c700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0cfc00 0x7fc1ec1f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1f179c700 1 -- 172.21.15.60:0/2560174612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0cf010 msgr2=0x7fc1ec0cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1f179c700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0cf010 0x7fc1ec0cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1f179c700 1 -- 172.21.15.60:0/2560174612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e00094a0 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1f179c700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1ec15c6f0 0x7fc1ec0b75a0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400cc40 tx=0x7fc1e400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:06.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1e97fa700 1 -- 172.21.15.60:0/2560174612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e400aac0 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1e97fa700 1 -- 172.21.15.60:0/2560174612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e400ac20 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1e97fa700 1 -- 172.21.15.60:0/2560174612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40070d0 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d80094a0 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.066+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ec0cc390 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.067+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ec1f21c0 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.068+0000 7fc1e97fa700 1 -- 172.21.15.60:0/2560174612 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc1e4007230 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.070+0000 7fc1e97fa700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1d006ace0 0x7fc1d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.070+0000 7fc1e97fa700 1 -- 172.21.15.60:0/2560174612 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc1e4088f90 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.070+0000 7fc1f0f9b700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1d006ace0 0x7fc1d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.070+0000 7fc1f0f9b700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1d006ace0 0x7fc1d006d190 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009cd0 tx=0x7fc1e0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.074+0000 7fc1e97fa700 1 -- 172.21.15.60:0/2560174612 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc1e4053f60 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.282+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1ec0722e0 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.284+0000 7fc1e97fa700 1 -- 172.21.15.60:0/2560174612 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+18363 (secure 0 0 0) 0x7fc1ec0722e0 con 0x7fc1ec15c6f0 2023-07-13T22:33:06.270 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:06.270 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":137,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:46.593649+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:31:27.702135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-13T22:31:28.386570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.7","mappings":[{"from":3,"to":0}]},{"pgid":"22.c","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:33:06.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1d006ace0 msgr2=0x7fc1d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1d006ace0 0x7fc1d006d190 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009cd0 tx=0x7fc1e0009210 comp rx=0 tx=0).stop 2023-07-13T22:33:06.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1ec15c6f0 msgr2=0x7fc1ec0b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1ec15c6f0 0x7fc1ec0b75a0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400cc40 tx=0x7fc1e400cf50 comp rx=0 tx=0).stop 2023-07-13T22:33:06.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 shutdown_connections 2023-07-13T22:33:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc1d006ace0 0x7fc1d006d190 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1ec15c6f0 0x7fc1ec0b75a0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0cf010 0x7fc1ec0cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 --2- 172.21.15.60:0/2560174612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0cfc00 0x7fc1ec1f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 >> 172.21.15.60:0/2560174612 conn(0x7fc1ec06a4b0 msgr2=0x7fc1ec15e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 shutdown_connections 2023-07-13T22:33:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.286+0000 7fc1f31ff700 1 -- 172.21.15.60:0/2560174612 wait complete. 2023-07-13T22:33:06.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:31:27.702135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:31:28.386570+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:33:06.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:33:06.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.511+0000 7f258f7ff700 1 Processor -- start 2023-07-13T22:33:06.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.511+0000 7f258f7ff700 1 -- start start 2023-07-13T22:33:06.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.512+0000 7f258f7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f258815ca10 0x7f258806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.512+0000 7f258f7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2588069eb0 0x7f258806a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.512+0000 7f258f7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258806cc80 con 0x7f25880ce720 2023-07-13T22:33:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.512+0000 7f258f7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258806a890 con 0x7f258815ca10 2023-07-13T22:33:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.512+0000 7f258f7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258815cde0 con 0x7f2588069eb0 2023-07-13T22:33:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.512+0000 7f258d59b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f258815ca10 0x7f258806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f258d59b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f258815ca10 0x7f258806d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55612/0 (socket says 172.21.15.60:55612) 2023-07-13T22:33:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f258d59b700 1 -- 172.21.15.60:0/1734005665 learned_addr learned my addr 172.21.15.60:0/1734005665 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:06.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f258cd9a700 1 --2- 172.21.15.60:0/1734005665 >> v2:172.21.15.105:3300/0 conn(0x7f2588069eb0 0x7f258806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f258d59b700 1 -- 172.21.15.60:0/1734005665 >> v2:172.21.15.105:3300/0 conn(0x7f2588069eb0 msgr2=0x7f258806a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:06.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f258d59b700 1 --2- 172.21.15.60:0/1734005665 >> v2:172.21.15.105:3300/0 conn(0x7f2588069eb0 0x7f258806a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f258d59b700 1 -- 172.21.15.60:0/1734005665 >> v1:172.21.15.105:6789/0 conn(0x7f25880ce720 legacy=0x7f25880ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f258d59b700 1 -- 172.21.15.60:0/1734005665 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25880cf390 con 0x7f258815ca10 2023-07-13T22:33:06.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f258d59b700 1 --2- 172.21.15.60:0/1734005665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f258815ca10 0x7f258806d6d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f25880cd4a0 tx=0x7f2584009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad80fcad28bdc0e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:06.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f257ffff700 1 -- 172.21.15.60:0/1734005665 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584011830 con 0x7f258815ca10 2023-07-13T22:33:06.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f257ffff700 1 -- 172.21.15.60:0/1734005665 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584011990 con 0x7f258815ca10 2023-07-13T22:33:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.513+0000 7f257ffff700 1 -- 172.21.15.60:0/1734005665 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584011d20 con 0x7f258815ca10 2023-07-13T22:33:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.514+0000 7f258f7ff700 1 -- 172.21.15.60:0/1734005665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f258815ca10 msgr2=0x7f258806d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.514+0000 7f258f7ff700 1 --2- 172.21.15.60:0/1734005665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f258815ca10 0x7f258806d6d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f25880cd4a0 tx=0x7f2584009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.514+0000 7f258f7ff700 1 -- 172.21.15.60:0/1734005665 shutdown_connections 2023-07-13T22:33:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.514+0000 7f258f7ff700 1 --2- 172.21.15.60:0/1734005665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f258815ca10 0x7f258806d6d0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.514+0000 7f258f7ff700 1 --2- 172.21.15.60:0/1734005665 >> v2:172.21.15.105:3300/0 conn(0x7f2588069eb0 0x7f258806a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.514+0000 7f258f7ff700 1 -- 172.21.15.60:0/1734005665 >> 172.21.15.60:0/1734005665 conn(0x7f25880b15a0 msgr2=0x7f25880b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:06.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.514+0000 7f258f7ff700 1 -- 172.21.15.60:0/1734005665 shutdown_connections 2023-07-13T22:33:06.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.514+0000 7f258f7ff700 1 -- 172.21.15.60:0/1734005665 wait complete. 2023-07-13T22:33:06.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.516+0000 7f258f7ff700 1 Processor -- start 2023-07-13T22:33:06.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.516+0000 7f258f7ff700 1 -- start start 2023-07-13T22:33:06.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.517+0000 7f258f7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2588069eb0 0x7f25880b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.517+0000 7f258f7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25880ce720 0x7f2588144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.517+0000 7f258f7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f258815ca10 0x7f2588143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.517+0000 7f258f7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588146290 con 0x7f2588069eb0 2023-07-13T22:33:06.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.517+0000 7f258f7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588145680 con 0x7f258815ca10 2023-07-13T22:33:06.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.517+0000 7f258f7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2588145960 con 0x7f25880ce720 2023-07-13T22:33:06.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.517+0000 7f258d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2588069eb0 0x7f25880b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.517+0000 7f258d59b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2588069eb0 0x7f25880b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55620/0 (socket says 172.21.15.60:55620) 2023-07-13T22:33:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.517+0000 7f258d59b700 1 -- 172.21.15.60:0/1410265225 learned_addr learned my addr 172.21.15.60:0/1410265225 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258d59b700 1 -- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25880ce720 msgr2=0x7f2588144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258cd9a700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25880ce720 0x7f2588144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258dd9c700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f258815ca10 0x7f2588143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258d59b700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25880ce720 0x7f2588144b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258d59b700 1 -- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f258815ca10 msgr2=0x7f2588143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258d59b700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f258815ca10 0x7f2588143f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258d59b700 1 -- 172.21.15.60:0/1410265225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2584009550 con 0x7f2588069eb0 2023-07-13T22:33:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258cd9a700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25880ce720 0x7f2588144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258dd9c700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f258815ca10 0x7f2588143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258d59b700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2588069eb0 0x7f25880b48c0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2584000c00 tx=0x7f2584004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f257dffb700 1 -- 172.21.15.60:0/1410265225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584004480 con 0x7f2588069eb0 2023-07-13T22:33:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f257dffb700 1 -- 172.21.15.60:0/1410265225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258401f070 con 0x7f2588069eb0 2023-07-13T22:33:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2588145c40 con 0x7f2588069eb0 2023-07-13T22:33:06.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f257dffb700 1 -- 172.21.15.60:0/1410265225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840038e0 con 0x7f2588069eb0 2023-07-13T22:33:06.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.518+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2588158460 con 0x7f2588069eb0 2023-07-13T22:33:06.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.519+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2588074220 con 0x7f2588069eb0 2023-07-13T22:33:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.520+0000 7f257dffb700 1 -- 172.21.15.60:0/1410265225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2584021030 con 0x7f2588069eb0 2023-07-13T22:33:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.521+0000 7f257dffb700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f256c06ace0 0x7f256c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.521+0000 7f257dffb700 1 -- 172.21.15.60:0/1410265225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f258408a9b0 con 0x7f2588069eb0 2023-07-13T22:33:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.521+0000 7f258cd9a700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f256c06ace0 0x7f256c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.522+0000 7f258cd9a700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f256c06ace0 0x7f256c06d190 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f25880678d0 tx=0x7f2578008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:06.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.525+0000 7f257dffb700 1 -- 172.21.15.60:0/1410265225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2584055a10 con 0x7f2588069eb0 2023-07-13T22:33:06.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.732+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f25880d07c0 con 0x7f2588069eb0 2023-07-13T22:33:06.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.732+0000 7f257dffb700 1 -- 172.21.15.60:0/1410265225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2584059030 con 0x7f2588069eb0 2023-07-13T22:33:06.719 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:33:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f256c06ace0 msgr2=0x7f256c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f256c06ace0 0x7f256c06d190 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f25880678d0 tx=0x7f2578008040 comp rx=0 tx=0).stop 2023-07-13T22:33:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2588069eb0 msgr2=0x7f25880b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2588069eb0 0x7f25880b48c0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2584000c00 tx=0x7f2584004750 comp rx=0 tx=0).stop 2023-07-13T22:33:06.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 shutdown_connections 2023-07-13T22:33:06.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f256c06ace0 0x7f256c06d190 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2588069eb0 0x7f25880b48c0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f258815ca10 0x7f2588143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 --2- 172.21.15.60:0/1410265225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25880ce720 0x7f2588144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 >> 172.21.15.60:0/1410265225 conn(0x7f25880b15a0 msgr2=0x7f25880cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:06.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 shutdown_connections 2023-07-13T22:33:06.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.735+0000 7f258f7ff700 1 -- 172.21.15.60:0/1410265225 wait complete. 2023-07-13T22:33:06.739 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:33:06.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.959+0000 7f60f7dff700 1 Processor -- start 2023-07-13T22:33:06.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.959+0000 7f60f7dff700 1 -- start start 2023-07-13T22:33:06.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.961+0000 7f60f7dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f015ca10 0x7f60f006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.961+0000 7f60f7dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f60f00cee30 0x7f60f006a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.961+0000 7f60f7dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f006cc80 con 0x7f60f0069eb0 2023-07-13T22:33:06.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.961+0000 7f60f7dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f006a860 con 0x7f60f00cee30 2023-07-13T22:33:06.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.961+0000 7f60f7dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f015cde0 con 0x7f60f015ca10 2023-07-13T22:33:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.961+0000 7f60f5b9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f015ca10 0x7f60f006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.961+0000 7f60f639c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f60f00cee30 0x7f60f006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.961+0000 7f60f5b9b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f015ca10 0x7f60f006d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:44094/0 (socket says 172.21.15.60:44094) 2023-07-13T22:33:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.961+0000 7f60f5b9b700 1 -- 172.21.15.60:0/3902125958 learned_addr learned my addr 172.21.15.60:0/3902125958 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f639c700 1 -- 172.21.15.60:0/3902125958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f015ca10 msgr2=0x7f60f006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f639c700 1 --2- 172.21.15.60:0/3902125958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f015ca10 0x7f60f006d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f639c700 1 -- 172.21.15.60:0/3902125958 >> v1:172.21.15.105:6789/0 conn(0x7f60f0069eb0 legacy=0x7f60f00ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f639c700 1 -- 172.21.15.60:0/3902125958 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f00cf390 con 0x7f60f00cee30 2023-07-13T22:33:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f5b9b700 1 --2- 172.21.15.60:0/3902125958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f015ca10 0x7f60f006d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:06.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f4b99700 1 -- 172.21.15.60:0/3902125958 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993705299 0 0) 0x7f60f006cc80 con 0x7f60f0069eb0 2023-07-13T22:33:06.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f639c700 1 --2- 172.21.15.60:0/3902125958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f00cee30 0x7f60f006a320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009d90 tx=0x7f60e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed5e036fb8fa83d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:06.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f4b99700 1 -- 172.21.15.60:0/3902125958 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e800e070 con 0x7f60f00cee30 2023-07-13T22:33:06.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f4b99700 1 -- 172.21.15.60:0/3902125958 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e800bb20 con 0x7f60f00cee30 2023-07-13T22:33:06.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.962+0000 7f60f4b99700 1 -- 172.21.15.60:0/3902125958 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8012430 con 0x7f60f00cee30 2023-07-13T22:33:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.963+0000 7f60f7dff700 1 -- 172.21.15.60:0/3902125958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f00cee30 msgr2=0x7f60f006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.963+0000 7f60f7dff700 1 --2- 172.21.15.60:0/3902125958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f00cee30 0x7f60f006a320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009d90 tx=0x7f60e800b330 comp rx=0 tx=0).stop 2023-07-13T22:33:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.963+0000 7f60f7dff700 1 -- 172.21.15.60:0/3902125958 shutdown_connections 2023-07-13T22:33:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.963+0000 7f60f7dff700 1 --2- 172.21.15.60:0/3902125958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f00cee30 0x7f60f006a320 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.963+0000 7f60f7dff700 1 --2- 172.21.15.60:0/3902125958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f015ca10 0x7f60f006d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.963+0000 7f60f7dff700 1 -- 172.21.15.60:0/3902125958 >> 172.21.15.60:0/3902125958 conn(0x7f60f00b15a0 msgr2=0x7f60f00b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:06.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.963+0000 7f60f7dff700 1 -- 172.21.15.60:0/3902125958 shutdown_connections 2023-07-13T22:33:06.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.963+0000 7f60f7dff700 1 -- 172.21.15.60:0/3902125958 wait complete. 2023-07-13T22:33:06.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.966+0000 7f60f7dff700 1 Processor -- start 2023-07-13T22:33:06.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.966+0000 7f60f7dff700 1 -- start start 2023-07-13T22:33:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.967+0000 7f60f7dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f0069eb0 0x7f60f01f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.967+0000 7f60f7dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f00cee30 0x7f60f00b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.967+0000 7f60f7dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f015ca10 0x7f60f014bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.967+0000 7f60f7dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0149bd0 con 0x7f60f015ca10 2023-07-13T22:33:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.967+0000 7f60f7dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00b4fb0 con 0x7f60f0069eb0 2023-07-13T22:33:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.967+0000 7f60f7dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00b5290 con 0x7f60f00cee30 2023-07-13T22:33:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.967+0000 7f60f639c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f015ca10 0x7f60f014bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.967+0000 7f60f639c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f015ca10 0x7f60f014bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55626/0 (socket says 172.21.15.60:55626) 2023-07-13T22:33:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.967+0000 7f60f639c700 1 -- 172.21.15.60:0/968419441 learned_addr learned my addr 172.21.15.60:0/968419441 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60f5b9b700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f0069eb0 0x7f60f01f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60f539a700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f00cee30 0x7f60f00b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60f639c700 1 -- 172.21.15.60:0/968419441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f00cee30 msgr2=0x7f60f00b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60f639c700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f00cee30 0x7f60f00b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60f639c700 1 -- 172.21.15.60:0/968419441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f0069eb0 msgr2=0x7f60f01f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60f639c700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f0069eb0 0x7f60f01f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60f639c700 1 -- 172.21.15.60:0/968419441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec009550 con 0x7f60f015ca10 2023-07-13T22:33:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60f639c700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f015ca10 0x7f60f014bc60 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f60e8014040 tx=0x7f60e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60e67fc700 1 -- 172.21.15.60:0/968419441 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8012430 con 0x7f60f015ca10 2023-07-13T22:33:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.968+0000 7f60e67fc700 1 -- 172.21.15.60:0/968419441 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e8020070 con 0x7f60f015ca10 2023-07-13T22:33:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.969+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e8009a70 con 0x7f60f015ca10 2023-07-13T22:33:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.969+0000 7f60e67fc700 1 -- 172.21.15.60:0/968419441 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8004280 con 0x7f60f015ca10 2023-07-13T22:33:06.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.969+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f014a0c0 con 0x7f60f015ca10 2023-07-13T22:33:06.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.970+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f0074220 con 0x7f60f015ca10 2023-07-13T22:33:06.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.971+0000 7f60e67fc700 1 -- 172.21.15.60:0/968419441 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f60e80043e0 con 0x7f60f015ca10 2023-07-13T22:33:06.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.971+0000 7f60e67fc700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f60d406ace0 0x7f60d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:06.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.972+0000 7f60e67fc700 1 -- 172.21.15.60:0/968419441 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f60e8022070 con 0x7f60f015ca10 2023-07-13T22:33:06.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.972+0000 7f60f5b9b700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f60d406ace0 0x7f60d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:06.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.972+0000 7f60f5b9b700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f60d406ace0 0x7f60d406d190 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f60ec006d30 tx=0x7f60ec0092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:06.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:06.976+0000 7f60e67fc700 1 -- 172.21.15.60:0/968419441 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f60e8056420 con 0x7f60f015ca10 2023-07-13T22:33:07.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.180+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60f006e8c0 con 0x7f60f015ca10 2023-07-13T22:33:07.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.181+0000 7f60e67fc700 1 -- 172.21.15.60:0/968419441 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60e8059a40 con 0x7f60f015ca10 2023-07-13T22:33:07.167 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:33:07.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f60d406ace0 msgr2=0x7f60d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f60d406ace0 0x7f60d406d190 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f60ec006d30 tx=0x7f60ec0092c0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f015ca10 msgr2=0x7f60f014bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f015ca10 0x7f60f014bc60 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f60e8014040 tx=0x7f60e8004ad0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 shutdown_connections 2023-07-13T22:33:07.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f60d406ace0 0x7f60d406d190 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f015ca10 0x7f60f014bc60 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f0069eb0 0x7f60f01f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 --2- 172.21.15.60:0/968419441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60f00cee30 0x7f60f00b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 >> 172.21.15.60:0/968419441 conn(0x7f60f00b15a0 msgr2=0x7f60f00c3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:07.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 shutdown_connections 2023-07-13T22:33:07.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.183+0000 7f60f7dff700 1 -- 172.21.15.60:0/968419441 wait complete. 2023-07-13T22:33:07.188 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:33:07.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.408+0000 7f62915ff700 1 Processor -- start 2023-07-13T22:33:07.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.408+0000 7f62915ff700 1 -- start start 2023-07-13T22:33:07.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.409+0000 7f62915ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b8730 0x7f628c0cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.409+0000 7f62915ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f628c0b4eb0 0x7f628c0d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.409+0000 7f62915ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0d06a0 con 0x7f628c160a30 2023-07-13T22:33:07.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.409+0000 7f62915ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c0cd810 con 0x7f628c0b4eb0 2023-07-13T22:33:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.409+0000 7f62915ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0d00e0 con 0x7f628c0b8730 2023-07-13T22:33:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.409+0000 7f628a7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f628c160a30 0x7f628c0b47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46222/0 (socket says 172.21.15.60:46222) 2023-07-13T22:33:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.409+0000 7f628a7fc700 1 -- 172.21.15.60:0/3067254335 learned_addr learned my addr 172.21.15.60:0/3067254335 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.410+0000 7f628b7fe700 1 --2- 172.21.15.60:0/3067254335 >> v2:172.21.15.60:3300/0 conn(0x7f628c0b4eb0 0x7f628c0d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.410+0000 7f628affd700 1 --2- 172.21.15.60:0/3067254335 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b8730 0x7f628c0cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.410+0000 7f6289ffb700 1 -- 172.21.15.60:0/3067254335 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842273619 0 0) 0x7f628c0d06a0 con 0x7f628c160a30 2023-07-13T22:33:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.410+0000 7f6289ffb700 1 -- 172.21.15.60:0/3067254335 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62740035b0 con 0x7f628c160a30 2023-07-13T22:33:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.410+0000 7f628b7fe700 1 -- 172.21.15.60:0/3067254335 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b8730 msgr2=0x7f628c0cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.410+0000 7f628b7fe700 1 --2- 172.21.15.60:0/3067254335 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b8730 0x7f628c0cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.410+0000 7f628b7fe700 1 -- 172.21.15.60:0/3067254335 >> v1:172.21.15.60:6789/0 conn(0x7f628c160a30 legacy=0x7f628c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.410+0000 7f628b7fe700 1 -- 172.21.15.60:0/3067254335 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0d03c0 con 0x7f628c0b4eb0 2023-07-13T22:33:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.410+0000 7f628b7fe700 1 --2- 172.21.15.60:0/3067254335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b4eb0 0x7f628c0d08f0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f6284009d90 tx=0x7f628400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a335dc29bd4a25e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f6289ffb700 1 -- 172.21.15.60:0/3067254335 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400e070 con 0x7f628c0b4eb0 2023-07-13T22:33:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f6289ffb700 1 -- 172.21.15.60:0/3067254335 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400bb20 con 0x7f628c0b4eb0 2023-07-13T22:33:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f6289ffb700 1 -- 172.21.15.60:0/3067254335 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284012430 con 0x7f628c0b4eb0 2023-07-13T22:33:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f62915ff700 1 -- 172.21.15.60:0/3067254335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b4eb0 msgr2=0x7f628c0d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f62915ff700 1 --2- 172.21.15.60:0/3067254335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b4eb0 0x7f628c0d08f0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f6284009d90 tx=0x7f628400b330 comp rx=0 tx=0).stop 2023-07-13T22:33:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f62915ff700 1 -- 172.21.15.60:0/3067254335 shutdown_connections 2023-07-13T22:33:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f62915ff700 1 --2- 172.21.15.60:0/3067254335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b4eb0 0x7f628c0d08f0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f62915ff700 1 --2- 172.21.15.60:0/3067254335 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b8730 0x7f628c0cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f62915ff700 1 -- 172.21.15.60:0/3067254335 >> 172.21.15.60:0/3067254335 conn(0x7f628c06b1a0 msgr2=0x7f628c0cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f62915ff700 1 -- 172.21.15.60:0/3067254335 shutdown_connections 2023-07-13T22:33:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.411+0000 7f62915ff700 1 -- 172.21.15.60:0/3067254335 wait complete. 2023-07-13T22:33:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.413+0000 7f62915ff700 1 Processor -- start 2023-07-13T22:33:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.413+0000 7f62915ff700 1 -- start start 2023-07-13T22:33:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.414+0000 7f62915ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b4eb0 0x7f628c1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.414+0000 7f62915ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b8730 0x7f628c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.414+0000 7f62915ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c160a30 0x7f628c1f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.414+0000 7f62915ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0c6cd0 con 0x7f628c0b8730 2023-07-13T22:33:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.414+0000 7f62915ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c1f2da0 con 0x7f628c160a30 2023-07-13T22:33:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.414+0000 7f62915ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c1f27e0 con 0x7f628c0b4eb0 2023-07-13T22:33:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.414+0000 7f628a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b8730 0x7f628c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.414+0000 7f628a7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b8730 0x7f628c0cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55652/0 (socket says 172.21.15.60:55652) 2023-07-13T22:33:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.414+0000 7f628a7fc700 1 -- 172.21.15.60:0/3833196928 learned_addr learned my addr 172.21.15.60:0/3833196928 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f628affd700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b4eb0 0x7f628c1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f628b7fe700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c160a30 0x7f628c1f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f628a7fc700 1 -- 172.21.15.60:0/3833196928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b4eb0 msgr2=0x7f628c1544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f628a7fc700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b4eb0 0x7f628c1544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f628a7fc700 1 -- 172.21.15.60:0/3833196928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c160a30 msgr2=0x7f628c1f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f628a7fc700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c160a30 0x7f628c1f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f628a7fc700 1 -- 172.21.15.60:0/3833196928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284009a70 con 0x7f628c0b8730 2023-07-13T22:33:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f628b7fe700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c160a30 0x7f628c1f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f628a7fc700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b8730 0x7f628c0cc600 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f6280001eb0 tx=0x7f628000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f62737fe700 1 -- 172.21.15.60:0/3833196928 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280005d70 con 0x7f628c0b8730 2023-07-13T22:33:07.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f62737fe700 1 -- 172.21.15.60:0/3833196928 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62800070d0 con 0x7f628c0b8730 2023-07-13T22:33:07.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f62737fe700 1 -- 172.21.15.60:0/3833196928 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280007460 con 0x7f628c0b8730 2023-07-13T22:33:07.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.415+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c1f2ac0 con 0x7f628c0b8730 2023-07-13T22:33:07.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.416+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c1f23a0 con 0x7f628c0b8730 2023-07-13T22:33:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.417+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c074220 con 0x7f628c0b8730 2023-07-13T22:33:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.418+0000 7f62737fe700 1 -- 172.21.15.60:0/3833196928 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f62800075c0 con 0x7f628c0b8730 2023-07-13T22:33:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.419+0000 7f62737fe700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627806ace0 0x7f627806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.419+0000 7f62737fe700 1 -- 172.21.15.60:0/3833196928 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f62800889a0 con 0x7f628c0b8730 2023-07-13T22:33:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.419+0000 7f628affd700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627806ace0 0x7f627806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.420+0000 7f628affd700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627806ace0 0x7f627806d190 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f627c009be0 tx=0x7f627c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:07.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.423+0000 7f62737fe700 1 -- 172.21.15.60:0/3833196928 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6280053a00 con 0x7f628c0b8730 2023-07-13T22:33:07.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.631+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f628c1f2530 con 0x7f628c0b8730 2023-07-13T22:33:07.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.632+0000 7f62737fe700 1 -- 172.21.15.60:0/3833196928 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6280057020 con 0x7f628c0b8730 2023-07-13T22:33:07.618 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:33:07.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627806ace0 msgr2=0x7f627806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627806ace0 0x7f627806d190 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f627c009be0 tx=0x7f627c009210 comp rx=0 tx=0).stop 2023-07-13T22:33:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b8730 msgr2=0x7f628c0cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b8730 0x7f628c0cc600 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f6280001eb0 tx=0x7f628000c7b0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 shutdown_connections 2023-07-13T22:33:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f627806ace0 0x7f627806d190 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c160a30 0x7f628c1f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f628c0b4eb0 0x7f628c1544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 --2- 172.21.15.60:0/3833196928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f628c0b8730 0x7f628c0cc600 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.634+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 >> 172.21.15.60:0/3833196928 conn(0x7f628c06b1a0 msgr2=0x7f628c0b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:07.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.635+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 shutdown_connections 2023-07-13T22:33:07.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.635+0000 7f62915ff700 1 -- 172.21.15.60:0/3833196928 wait complete. 2023-07-13T22:33:07.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:33:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.860+0000 7f1e741ff700 1 Processor -- start 2023-07-13T22:33:07.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.860+0000 7f1e741ff700 1 -- start start 2023-07-13T22:33:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.861+0000 7f1e741ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e6c0b8730 0x7f1e6c0cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.861+0000 7f1e741ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1e6c0b4eb0 0x7f1e6c0d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.861+0000 7f1e741ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6c0d06a0 con 0x7f1e6c160a30 2023-07-13T22:33:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.861+0000 7f1e741ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6c0cd810 con 0x7f1e6c0b8730 2023-07-13T22:33:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.861+0000 7f1e741ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6c0d00e0 con 0x7f1e6c0b4eb0 2023-07-13T22:33:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.861+0000 7f1e71f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e6c0b8730 0x7f1e6c0cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.862+0000 7f1e71f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e6c0b8730 0x7f1e6c0cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55664/0 (socket says 172.21.15.60:55664) 2023-07-13T22:33:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.862+0000 7f1e71f9b700 1 -- 172.21.15.60:0/2357849431 learned_addr learned my addr 172.21.15.60:0/2357849431 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.862+0000 7f1e7279c700 1 --2- 172.21.15.60:0/2357849431 >> v2:172.21.15.105:3300/0 conn(0x7f1e6c0b4eb0 0x7f1e6c0d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.862+0000 7f1e71f9b700 1 -- 172.21.15.60:0/2357849431 >> v2:172.21.15.105:3300/0 conn(0x7f1e6c0b4eb0 msgr2=0x7f1e6c0d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.862+0000 7f1e71f9b700 1 --2- 172.21.15.60:0/2357849431 >> v2:172.21.15.105:3300/0 conn(0x7f1e6c0b4eb0 0x7f1e6c0d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.862+0000 7f1e71f9b700 1 -- 172.21.15.60:0/2357849431 >> v1:172.21.15.105:6789/0 conn(0x7f1e6c160a30 legacy=0x7f1e6c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.862+0000 7f1e71f9b700 1 -- 172.21.15.60:0/2357849431 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c0d03c0 con 0x7f1e6c0b8730 2023-07-13T22:33:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.862+0000 7f1e7279c700 1 --2- 172.21.15.60:0/2357849431 >> v2:172.21.15.105:3300/0 conn(0x7f1e6c0b4eb0 0x7f1e6c0d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.862+0000 7f1e71f9b700 1 --2- 172.21.15.60:0/2357849431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c0b8730 0x7f1e6c0cefc0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0b5280 tx=0x7f1e68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12bc78f7dc8d07e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e70f99700 1 -- 172.21.15.60:0/2357849431 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800b870 con 0x7f1e6c0b8730 2023-07-13T22:33:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e70f99700 1 -- 172.21.15.60:0/2357849431 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6800b9d0 con 0x7f1e6c0b8730 2023-07-13T22:33:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e70f99700 1 -- 172.21.15.60:0/2357849431 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800bd60 con 0x7f1e6c0b8730 2023-07-13T22:33:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e741ff700 1 -- 172.21.15.60:0/2357849431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c0b8730 msgr2=0x7f1e6c0cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e741ff700 1 --2- 172.21.15.60:0/2357849431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c0b8730 0x7f1e6c0cefc0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0b5280 tx=0x7f1e68009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e741ff700 1 -- 172.21.15.60:0/2357849431 shutdown_connections 2023-07-13T22:33:07.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e741ff700 1 --2- 172.21.15.60:0/2357849431 >> v2:172.21.15.105:3300/0 conn(0x7f1e6c0b4eb0 0x7f1e6c0d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e741ff700 1 --2- 172.21.15.60:0/2357849431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c0b8730 0x7f1e6c0cefc0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e741ff700 1 -- 172.21.15.60:0/2357849431 >> 172.21.15.60:0/2357849431 conn(0x7f1e6c06b1a0 msgr2=0x7f1e6c0cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e741ff700 1 -- 172.21.15.60:0/2357849431 shutdown_connections 2023-07-13T22:33:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.863+0000 7f1e741ff700 1 -- 172.21.15.60:0/2357849431 wait complete. 2023-07-13T22:33:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.866+0000 7f1e741ff700 1 Processor -- start 2023-07-13T22:33:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.866+0000 7f1e741ff700 1 -- start start 2023-07-13T22:33:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.867+0000 7f1e741ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e6c0b4eb0 0x7f1e6c156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.867+0000 7f1e741ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e6c0b8730 0x7f1e6c0c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.867+0000 7f1e741ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c160a30 0x7f1e6c0c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.867+0000 7f1e741ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c0c8150 con 0x7f1e6c160a30 2023-07-13T22:33:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.867+0000 7f1e741ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c0c7540 con 0x7f1e6c0b4eb0 2023-07-13T22:33:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.867+0000 7f1e741ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c0c7820 con 0x7f1e6c0b8730 2023-07-13T22:33:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.867+0000 7f1e7279c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c160a30 0x7f1e6c0c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.867+0000 7f1e7279c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c160a30 0x7f1e6c0c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55668/0 (socket says 172.21.15.60:55668) 2023-07-13T22:33:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.867+0000 7f1e7279c700 1 -- 172.21.15.60:0/3717970230 learned_addr learned my addr 172.21.15.60:0/3717970230 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e71f9b700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e6c0b4eb0 0x7f1e6c156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e7179a700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e6c0b8730 0x7f1e6c0c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e7279c700 1 -- 172.21.15.60:0/3717970230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e6c0b8730 msgr2=0x7f1e6c0c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e7279c700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e6c0b8730 0x7f1e6c0c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e7279c700 1 -- 172.21.15.60:0/3717970230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e6c0b4eb0 msgr2=0x7f1e6c156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e7279c700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e6c0b4eb0 0x7f1e6c156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e7279c700 1 -- 172.21.15.60:0/3717970230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68009550 con 0x7f1e6c160a30 2023-07-13T22:33:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e7179a700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e6c0b8730 0x7f1e6c0c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e7279c700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c160a30 0x7f1e6c0c8d60 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f1e64007e30 tx=0x7f1e6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e627fc700 1 -- 172.21.15.60:0/3717970230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6400ddd0 con 0x7f1e6c160a30 2023-07-13T22:33:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e627fc700 1 -- 172.21.15.60:0/3717970230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6400f040 con 0x7f1e6c160a30 2023-07-13T22:33:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e627fc700 1 -- 172.21.15.60:0/3717970230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64003c60 con 0x7f1e6c160a30 2023-07-13T22:33:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.868+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6c0c7b00 con 0x7f1e6c160a30 2023-07-13T22:33:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.869+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6c1572d0 con 0x7f1e6c160a30 2023-07-13T22:33:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.870+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6c1e5d40 con 0x7f1e6c160a30 2023-07-13T22:33:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.871+0000 7f1e627fc700 1 -- 172.21.15.60:0/3717970230 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e64004030 con 0x7f1e6c160a30 2023-07-13T22:33:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.872+0000 7f1e627fc700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e5006ace0 0x7f1e5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.872+0000 7f1e627fc700 1 -- 172.21.15.60:0/3717970230 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1e640896a0 con 0x7f1e6c160a30 2023-07-13T22:33:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.872+0000 7f1e71f9b700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e5006ace0 0x7f1e5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.873+0000 7f1e71f9b700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e5006ace0 0x7f1e5006d190 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1e68000c00 tx=0x7f1e68008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:07.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:07.876+0000 7f1e627fc700 1 -- 172.21.15.60:0/3717970230 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f1e6c1e5d40 con 0x7f1e6c160a30 2023-07-13T22:33:08.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.082+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e6c1e5d40 con 0x7f1e6c160a30 2023-07-13T22:33:08.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.082+0000 7f1e627fc700 1 -- 172.21.15.60:0/3717970230 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e6401a020 con 0x7f1e6c160a30 2023-07-13T22:33:08.069 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:33:08.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.084+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e5006ace0 msgr2=0x7f1e5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.084+0000 7f1e741ff700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e5006ace0 0x7f1e5006d190 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1e68000c00 tx=0x7f1e68008030 comp rx=0 tx=0).stop 2023-07-13T22:33:08.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.084+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c160a30 msgr2=0x7f1e6c0c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.084+0000 7f1e741ff700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c160a30 0x7f1e6c0c8d60 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f1e64007e30 tx=0x7f1e6400d3b0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.085+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 shutdown_connections 2023-07-13T22:33:08.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.085+0000 7f1e741ff700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f1e5006ace0 0x7f1e5006d190 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.085+0000 7f1e741ff700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e6c160a30 0x7f1e6c0c8d60 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.085+0000 7f1e741ff700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e6c0b4eb0 0x7f1e6c156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.085+0000 7f1e741ff700 1 --2- 172.21.15.60:0/3717970230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e6c0b8730 0x7f1e6c0c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.085+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 >> 172.21.15.60:0/3717970230 conn(0x7f1e6c06b1a0 msgr2=0x7f1e6c0b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:08.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.085+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 shutdown_connections 2023-07-13T22:33:08.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.085+0000 7f1e741ff700 1 -- 172.21.15.60:0/3717970230 wait complete. 2023-07-13T22:33:08.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:33:08.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.305+0000 7fdfc33ff700 1 Processor -- start 2023-07-13T22:33:08.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.306+0000 7fdfc33ff700 1 -- start start 2023-07-13T22:33:08.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.307+0000 7fdfc33ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdfbc15c6f0 0x7fdfbc160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.307+0000 7fdfc33ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdfbc0cfdd0 0x7fdfbc0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.307+0000 7fdfc33ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfbc0722e0 con 0x7fdfbc0cf010 2023-07-13T22:33:08.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.307+0000 7fdfc33ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc0d0df0 con 0x7fdfbc15c6f0 2023-07-13T22:33:08.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.307+0000 7fdfc33ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc0cdbd0 con 0x7fdfbc0cfdd0 2023-07-13T22:33:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.307+0000 7fdfc099a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdfbc0cf010 0x7fdfbc0cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46234/0 (socket says 172.21.15.60:46234) 2023-07-13T22:33:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.307+0000 7fdfc119b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdfbc15c6f0 0x7fdfbc160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.307+0000 7fdfc099a700 1 -- 172.21.15.60:0/2792915519 learned_addr learned my addr 172.21.15.60:0/2792915519 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfc199c700 1 --2- 172.21.15.60:0/2792915519 >> v2:172.21.15.105:3300/0 conn(0x7fdfbc0cfdd0 0x7fdfbc0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfc119b700 1 -- 172.21.15.60:0/2792915519 >> v2:172.21.15.105:3300/0 conn(0x7fdfbc0cfdd0 msgr2=0x7fdfbc0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfc119b700 1 --2- 172.21.15.60:0/2792915519 >> v2:172.21.15.105:3300/0 conn(0x7fdfbc0cfdd0 0x7fdfbc0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfc119b700 1 -- 172.21.15.60:0/2792915519 >> v1:172.21.15.60:6789/0 conn(0x7fdfbc0cf010 legacy=0x7fdfbc0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfc119b700 1 -- 172.21.15.60:0/2792915519 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc15cac0 con 0x7fdfbc15c6f0 2023-07-13T22:33:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfb3fff700 1 -- 172.21.15.60:0/2792915519 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554404094 0 0) 0x7fdfac001110 con 0x7fdfbc0cf010 2023-07-13T22:33:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfc119b700 1 --2- 172.21.15.60:0/2792915519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc15c6f0 0x7fdfbc160b70 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0c9d40 tx=0x7fdfb8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8cb6199fdc5388 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfb3fff700 1 -- 172.21.15.60:0/2792915519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8011830 con 0x7fdfbc15c6f0 2023-07-13T22:33:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfb3fff700 1 -- 172.21.15.60:0/2792915519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb8011990 con 0x7fdfbc15c6f0 2023-07-13T22:33:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.308+0000 7fdfb3fff700 1 -- 172.21.15.60:0/2792915519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8011d20 con 0x7fdfbc15c6f0 2023-07-13T22:33:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.309+0000 7fdfc33ff700 1 -- 172.21.15.60:0/2792915519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc15c6f0 msgr2=0x7fdfbc160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.309+0000 7fdfc33ff700 1 --2- 172.21.15.60:0/2792915519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc15c6f0 0x7fdfbc160b70 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0c9d40 tx=0x7fdfb8009b20 comp rx=0 tx=0).stop 2023-07-13T22:33:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.309+0000 7fdfc33ff700 1 -- 172.21.15.60:0/2792915519 shutdown_connections 2023-07-13T22:33:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.309+0000 7fdfc33ff700 1 --2- 172.21.15.60:0/2792915519 >> v2:172.21.15.105:3300/0 conn(0x7fdfbc0cfdd0 0x7fdfbc0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.309+0000 7fdfc33ff700 1 --2- 172.21.15.60:0/2792915519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc15c6f0 0x7fdfbc160b70 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.309+0000 7fdfc33ff700 1 -- 172.21.15.60:0/2792915519 >> 172.21.15.60:0/2792915519 conn(0x7fdfbc06a4b0 msgr2=0x7fdfbc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.309+0000 7fdfc33ff700 1 -- 172.21.15.60:0/2792915519 shutdown_connections 2023-07-13T22:33:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.309+0000 7fdfc33ff700 1 -- 172.21.15.60:0/2792915519 wait complete. 2023-07-13T22:33:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.311+0000 7fdfc33ff700 1 Processor -- start 2023-07-13T22:33:08.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.311+0000 7fdfc33ff700 1 -- start start 2023-07-13T22:33:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc33ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc0cf010 0x7fdfbc0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc33ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfbc0cfdd0 0x7fdfbc0b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc33ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfbc15c6f0 0x7fdfbc1457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc33ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0ca600 con 0x7fdfbc0cf010 2023-07-13T22:33:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc33ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0b3040 con 0x7fdfbc15c6f0 2023-07-13T22:33:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc33ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0ca8a0 con 0x7fdfbc0cfdd0 2023-07-13T22:33:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc0cf010 0x7fdfbc0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc0cf010 0x7fdfbc0b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55684/0 (socket says 172.21.15.60:55684) 2023-07-13T22:33:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc119b700 1 -- 172.21.15.60:0/841161306 learned_addr learned my addr 172.21.15.60:0/841161306 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc099a700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfbc0cfdd0 0x7fdfbc0b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.312+0000 7fdfc199c700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfbc15c6f0 0x7fdfbc1457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfc119b700 1 -- 172.21.15.60:0/841161306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfbc0cfdd0 msgr2=0x7fdfbc0b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfc119b700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfbc0cfdd0 0x7fdfbc0b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfc119b700 1 -- 172.21.15.60:0/841161306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfbc15c6f0 msgr2=0x7fdfbc1457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfc119b700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfbc15c6f0 0x7fdfbc1457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfc119b700 1 -- 172.21.15.60:0/841161306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb80094a0 con 0x7fdfbc0cf010 2023-07-13T22:33:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfc199c700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfbc15c6f0 0x7fdfbc1457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfc119b700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc0cf010 0x7fdfbc0b2740 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009b60 tx=0x7fdfb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfb1ffb700 1 -- 172.21.15.60:0/841161306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80038a0 con 0x7fdfbc0cf010 2023-07-13T22:33:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfb1ffb700 1 -- 172.21.15.60:0/841161306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb801e070 con 0x7fdfbc0cf010 2023-07-13T22:33:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc0cab80 con 0x7fdfbc0cf010 2023-07-13T22:33:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfb1ffb700 1 -- 172.21.15.60:0/841161306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8003c00 con 0x7fdfbc0cf010 2023-07-13T22:33:08.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.313+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc0b2c80 con 0x7fdfbc0cf010 2023-07-13T22:33:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.314+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc074220 con 0x7fdfbc0cf010 2023-07-13T22:33:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.315+0000 7fdfb1ffb700 1 -- 172.21.15.60:0/841161306 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdfb802d450 con 0x7fdfbc0cf010 2023-07-13T22:33:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.317+0000 7fdfb1ffb700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdfa006ace0 0x7fdfa006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.317+0000 7fdfb1ffb700 1 -- 172.21.15.60:0/841161306 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdfb808a650 con 0x7fdfbc0cf010 2023-07-13T22:33:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.317+0000 7fdfc099a700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdfa006ace0 0x7fdfa006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.318+0000 7fdfc099a700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdfa006ace0 0x7fdfa006d190 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc1565d0 tx=0x7fdfac007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:08.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.321+0000 7fdfb1ffb700 1 -- 172.21.15.60:0/841161306 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fdfb8055620 con 0x7fdfbc0cf010 2023-07-13T22:33:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.543+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdfbc071d10 con 0x7fdfbc0cf010 2023-07-13T22:33:08.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.543+0000 7fdfb1ffb700 1 -- 172.21.15.60:0/841161306 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v137) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdfb8058c40 con 0x7fdfbc0cf010 2023-07-13T22:33:08.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.545+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdfa006ace0 msgr2=0x7fdfa006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.545+0000 7fdfc33ff700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdfa006ace0 0x7fdfa006d190 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc1565d0 tx=0x7fdfac007a40 comp rx=0 tx=0).stop 2023-07-13T22:33:08.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.545+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc0cf010 msgr2=0x7fdfbc0b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.545+0000 7fdfc33ff700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc0cf010 0x7fdfbc0b2740 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009b60 tx=0x7fdfb8004750 comp rx=0 tx=0).stop 2023-07-13T22:33:08.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.546+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 shutdown_connections 2023-07-13T22:33:08.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.546+0000 7fdfc33ff700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fdfa006ace0 0x7fdfa006d190 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.546+0000 7fdfc33ff700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfbc0cf010 0x7fdfbc0b2740 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.546+0000 7fdfc33ff700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfbc15c6f0 0x7fdfbc1457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.546+0000 7fdfc33ff700 1 --2- 172.21.15.60:0/841161306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdfbc0cfdd0 0x7fdfbc0b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.546+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 >> 172.21.15.60:0/841161306 conn(0x7fdfbc06a4b0 msgr2=0x7fdfbc15f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:08.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.546+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 shutdown_connections 2023-07-13T22:33:08.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.546+0000 7fdfc33ff700 1 -- 172.21.15.60:0/841161306 wait complete. 2023-07-13T22:33:08.536 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:33:08.550 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-07-13T22:33:08.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:08.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-13T22:33:08.620 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[167291]: fuse finished with error 0 and tester_r 0 2023-07-13T22:33:08.624 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:33:08.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:08.625 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:33:08.713 INFO:tasks.cephfs.mount:Terminating background process 2023-07-13T22:33:08.713 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:33:08.714 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:33:08.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:33:08.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.984+0000 7fed313ff700 1 Processor -- start 2023-07-13T22:33:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.984+0000 7fed313ff700 1 -- start start 2023-07-13T22:33:08.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.985+0000 7fed313ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fed2c15c5f0 0x7fed2c06c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.985+0000 7fed313ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce3a0 0x7fed2c069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.985+0000 7fed313ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2c160240 con 0x7fed2c0ce810 2023-07-13T22:33:08.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.985+0000 7fed313ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2c0713a0 con 0x7fed2c15c5f0 2023-07-13T22:33:08.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.985+0000 7fed313ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c071680 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.986+0000 7fed2a7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce3a0 0x7fed2c069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.986+0000 7fed2affd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fed2c15c5f0 0x7fed2c06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.986+0000 7fed2a7fc700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce3a0 0x7fed2c069eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.60:44132/0 (socket says 172.21.15.60:44132) 2023-07-13T22:33:08.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.986+0000 7fed2a7fc700 1 -- 172.21.15.60:0/3706203010 learned_addr learned my addr 172.21.15.60:0/3706203010 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:08.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.986+0000 7fed29ffb700 1 -- 172.21.15.60:0/3706203010 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098691379 0 0) 0x7fed2c160240 con 0x7fed2c0ce810 2023-07-13T22:33:08.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed29ffb700 1 -- 172.21.15.60:0/3706203010 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed140035b0 con 0x7fed2c0ce810 2023-07-13T22:33:08.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed2a7fc700 1 -- 172.21.15.60:0/3706203010 >> v2:172.21.15.105:3300/0 conn(0x7fed2c15c5f0 msgr2=0x7fed2c06c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:08.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed2a7fc700 1 --2- 172.21.15.60:0/3706203010 >> v2:172.21.15.105:3300/0 conn(0x7fed2c15c5f0 0x7fed2c06c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed2a7fc700 1 -- 172.21.15.60:0/3706203010 >> v1:172.21.15.105:6789/0 conn(0x7fed2c0ce810 legacy=0x7fed2c1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed2a7fc700 1 -- 172.21.15.60:0/3706203010 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c071960 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed2affd700 1 --2- 172.21.15.60:0/3706203010 >> v2:172.21.15.105:3300/0 conn(0x7fed2c15c5f0 0x7fed2c06c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:08.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed2a7fc700 1 --2- 172.21.15.60:0/3706203010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce3a0 0x7fed2c069eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fed200098f0 tx=0x7fed20009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=727f9ce1662c7b60 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:08.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed29ffb700 1 -- 172.21.15.60:0/3706203010 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20011830 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed29ffb700 1 -- 172.21.15.60:0/3706203010 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed20011990 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.987+0000 7fed29ffb700 1 -- 172.21.15.60:0/3706203010 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20011d20 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.988+0000 7fed313ff700 1 -- 172.21.15.60:0/3706203010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce3a0 msgr2=0x7fed2c069eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.988+0000 7fed313ff700 1 --2- 172.21.15.60:0/3706203010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce3a0 0x7fed2c069eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fed200098f0 tx=0x7fed20009c00 comp rx=0 tx=0).stop 2023-07-13T22:33:08.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.988+0000 7fed313ff700 1 -- 172.21.15.60:0/3706203010 shutdown_connections 2023-07-13T22:33:08.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.988+0000 7fed313ff700 1 --2- 172.21.15.60:0/3706203010 >> v2:172.21.15.105:3300/0 conn(0x7fed2c15c5f0 0x7fed2c06c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.988+0000 7fed313ff700 1 --2- 172.21.15.60:0/3706203010 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce3a0 0x7fed2c069eb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.988+0000 7fed313ff700 1 -- 172.21.15.60:0/3706203010 >> 172.21.15.60:0/3706203010 conn(0x7fed2c06bc50 msgr2=0x7fed2c0cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:08.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.988+0000 7fed313ff700 1 -- 172.21.15.60:0/3706203010 shutdown_connections 2023-07-13T22:33:08.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.988+0000 7fed313ff700 1 -- 172.21.15.60:0/3706203010 wait complete. 2023-07-13T22:33:08.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.990+0000 7fed313ff700 1 Processor -- start 2023-07-13T22:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.990+0000 7fed313ff700 1 -- start start 2023-07-13T22:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed313ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed2c0ce3a0 0x7fed2c13f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed313ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce810 0x7fed2c142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed313ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c15c5f0 0x7fed2c15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed313ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c158490 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed313ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c158cf0 con 0x7fed2c15c5f0 2023-07-13T22:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed313ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1f1fb0 con 0x7fed2c0ce810 2023-07-13T22:33:08.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed2affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed2c0ce3a0 0x7fed2c13f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed2affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed2c0ce3a0 0x7fed2c13f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55700/0 (socket says 172.21.15.60:55700) 2023-07-13T22:33:08.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed2affd700 1 -- 172.21.15.60:0/3154289835 learned_addr learned my addr 172.21.15.60:0/3154289835 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:08.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed2b7fe700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c15c5f0 0x7fed2c15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed2a7fc700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce810 0x7fed2c142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed2affd700 1 -- 172.21.15.60:0/3154289835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce810 msgr2=0x7fed2c142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.991+0000 7fed2affd700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce810 0x7fed2c142190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.992+0000 7fed2affd700 1 -- 172.21.15.60:0/3154289835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c15c5f0 msgr2=0x7fed2c15c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.992+0000 7fed2affd700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c15c5f0 0x7fed2c15c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.992+0000 7fed2affd700 1 -- 172.21.15.60:0/3154289835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c0094a0 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.992+0000 7fed2affd700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed2c0ce3a0 0x7fed2c13f750 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00ebe0 tx=0x7fed1c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.992+0000 7fed137fe700 1 -- 172.21.15.60:0/3154289835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00c850 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.992+0000 7fed137fe700 1 -- 172.21.15.60:0/3154289835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c00c9b0 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.992+0000 7fed137fe700 1 -- 172.21.15.60:0/3154289835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00cd40 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.992+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed20009550 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.992+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c1586a0 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.993+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2c074220 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.994+0000 7fed137fe700 1 -- 172.21.15.60:0/3154289835 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed1c013470 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.995+0000 7fed137fe700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed1806ace0 0x7fed1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:08.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.995+0000 7fed137fe700 1 -- 172.21.15.60:0/3154289835 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fed1c089690 con 0x7fed2c0ce3a0 2023-07-13T22:33:08.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.996+0000 7fed2a7fc700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed1806ace0 0x7fed1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:08.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:08.996+0000 7fed2a7fc700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed1806ace0 0x7fed1806d190 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fed20014040 tx=0x7fed20008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.000+0000 7fed137fe700 1 -- 172.21.15.60:0/3154289835 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fed1c0545e0 con 0x7fed2c0ce3a0 2023-07-13T22:33:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.208+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed2c0cd200 con 0x7fed2c0ce3a0 2023-07-13T22:33:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.210+0000 7fed137fe700 1 -- 172.21.15.60:0/3154289835 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+18363 (secure 0 0 0) 0x7fed1c00cea0 con 0x7fed2c0ce3a0 2023-07-13T22:33:09.197 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:09.197 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":137,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:46.593649+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:31:27.702135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-13T22:31:28.386570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.7","mappings":[{"from":3,"to":0}]},{"pgid":"22.c","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:33:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed1806ace0 msgr2=0x7fed1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed1806ace0 0x7fed1806d190 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fed20014040 tx=0x7fed20008030 comp rx=0 tx=0).stop 2023-07-13T22:33:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed2c0ce3a0 msgr2=0x7fed2c13f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed2c0ce3a0 0x7fed2c13f750 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00ebe0 tx=0x7fed1c00eef0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 shutdown_connections 2023-07-13T22:33:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fed1806ace0 0x7fed1806d190 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed2c0ce3a0 0x7fed2c13f750 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c15c5f0 0x7fed2c15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 --2- 172.21.15.60:0/3154289835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fed2c0ce810 0x7fed2c142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.212+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 >> 172.21.15.60:0/3154289835 conn(0x7fed2c06bc50 msgr2=0x7fed2c0cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.213+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 shutdown_connections 2023-07-13T22:33:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.213+0000 7fed313ff700 1 -- 172.21.15.60:0/3154289835 wait complete. 2023-07-13T22:33:09.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:31:27.702135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:31:28.386570+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:33:09.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:33:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.439+0000 7f67771ff700 1 Processor -- start 2023-07-13T22:33:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.439+0000 7f67771ff700 1 -- start start 2023-07-13T22:33:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.440+0000 7f67771ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67700d11e0 0x7f677006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.440+0000 7f67771ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f67700b1b70 0x7f67700ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.440+0000 7f67771ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67700b2150 con 0x7f677006b500 2023-07-13T22:33:09.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.440+0000 7f67771ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677006b9a0 con 0x7f67700d11e0 2023-07-13T22:33:09.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.440+0000 7f67771ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67700d15b0 con 0x7f67700b1b70 2023-07-13T22:33:09.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.440+0000 7f6774f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67700d11e0 0x7f677006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.440+0000 7f6774f9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67700d11e0 0x7f677006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55708/0 (socket says 172.21.15.60:55708) 2023-07-13T22:33:09.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.440+0000 7f6774f9b700 1 -- 172.21.15.60:0/709728095 learned_addr learned my addr 172.21.15.60:0/709728095 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.441+0000 7f677579c700 1 --2- 172.21.15.60:0/709728095 >> v2:172.21.15.105:3300/0 conn(0x7f67700b1b70 0x7f67700ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.441+0000 7f6774f9b700 1 -- 172.21.15.60:0/709728095 >> v2:172.21.15.105:3300/0 conn(0x7f67700b1b70 msgr2=0x7f67700ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.441+0000 7f6774f9b700 1 --2- 172.21.15.60:0/709728095 >> v2:172.21.15.105:3300/0 conn(0x7f67700b1b70 0x7f67700ae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.441+0000 7f6774f9b700 1 -- 172.21.15.60:0/709728095 >> v1:172.21.15.105:6789/0 conn(0x7f677006b500 legacy=0x7f67700b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.441+0000 7f6774f9b700 1 -- 172.21.15.60:0/709728095 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67700ad850 con 0x7f67700d11e0 2023-07-13T22:33:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.441+0000 7f6774f9b700 1 --2- 172.21.15.60:0/709728095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 0x7f677006afc0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f677006d8a0 tx=0x7f6764009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ee48e73c325033 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.441+0000 7f676f7fe700 1 -- 172.21.15.60:0/709728095 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764005e30 con 0x7f67700d11e0 2023-07-13T22:33:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.441+0000 7f676f7fe700 1 -- 172.21.15.60:0/709728095 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6764004030 con 0x7f67700d11e0 2023-07-13T22:33:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.441+0000 7f676f7fe700 1 -- 172.21.15.60:0/709728095 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67640043e0 con 0x7f67700d11e0 2023-07-13T22:33:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.442+0000 7f67771ff700 1 -- 172.21.15.60:0/709728095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 msgr2=0x7f677006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.442+0000 7f67771ff700 1 --2- 172.21.15.60:0/709728095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 0x7f677006afc0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f677006d8a0 tx=0x7f6764009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.442+0000 7f67771ff700 1 -- 172.21.15.60:0/709728095 shutdown_connections 2023-07-13T22:33:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.442+0000 7f67771ff700 1 --2- 172.21.15.60:0/709728095 >> v2:172.21.15.105:3300/0 conn(0x7f67700b1b70 0x7f67700ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.442+0000 7f67771ff700 1 --2- 172.21.15.60:0/709728095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 0x7f677006afc0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.442+0000 7f67771ff700 1 -- 172.21.15.60:0/709728095 >> 172.21.15.60:0/709728095 conn(0x7f67700664f0 msgr2=0x7f6770099ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.442+0000 7f67771ff700 1 -- 172.21.15.60:0/709728095 shutdown_connections 2023-07-13T22:33:09.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.442+0000 7f67771ff700 1 -- 172.21.15.60:0/709728095 wait complete. 2023-07-13T22:33:09.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.444+0000 7f67771ff700 1 Processor -- start 2023-07-13T22:33:09.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.445+0000 7f67771ff700 1 -- start start 2023-07-13T22:33:09.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f67771ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677006b500 0x7f677015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f67771ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67700b1b70 0x7f67701f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f67771ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 0x7f67700c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f67771ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677009ac60 con 0x7f67700d11e0 2023-07-13T22:33:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f67771ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677015a650 con 0x7f677006b500 2023-07-13T22:33:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f67771ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700c5590 con 0x7f67700b1b70 2023-07-13T22:33:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f677579c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 0x7f67700c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f677579c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 0x7f67700c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55710/0 (socket says 172.21.15.60:55710) 2023-07-13T22:33:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f677579c700 1 -- 172.21.15.60:0/199488887 learned_addr learned my addr 172.21.15.60:0/199488887 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.446+0000 7f6774f9b700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677006b500 0x7f677015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f676ffff700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67700b1b70 0x7f67701f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f677579c700 1 -- 172.21.15.60:0/199488887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67700b1b70 msgr2=0x7f67701f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f677579c700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67700b1b70 0x7f67701f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f677579c700 1 -- 172.21.15.60:0/199488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677006b500 msgr2=0x7f677015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f677579c700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677006b500 0x7f677015a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f677579c700 1 -- 172.21.15.60:0/199488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764009500 con 0x7f67700d11e0 2023-07-13T22:33:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f6774f9b700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677006b500 0x7f677015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f677579c700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 0x7f67700c5a20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f676800efc0 tx=0x7f676800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f676d7fa700 1 -- 172.21.15.60:0/199488887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676800cd40 con 0x7f67700d11e0 2023-07-13T22:33:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f676d7fa700 1 -- 172.21.15.60:0/199488887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676800f040 con 0x7f67700d11e0 2023-07-13T22:33:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67700c49c0 con 0x7f67700d11e0 2023-07-13T22:33:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f676d7fa700 1 -- 172.21.15.60:0/199488887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67680146c0 con 0x7f67700d11e0 2023-07-13T22:33:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.447+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67700c4f40 con 0x7f67700d11e0 2023-07-13T22:33:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.448+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67700c5740 con 0x7f67700d11e0 2023-07-13T22:33:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.449+0000 7f676d7fa700 1 -- 172.21.15.60:0/199488887 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6768014820 con 0x7f67700d11e0 2023-07-13T22:33:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.450+0000 7f676d7fa700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f675406ace0 0x7f675406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.451+0000 7f676d7fa700 1 -- 172.21.15.60:0/199488887 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f676808a1b0 con 0x7f67700d11e0 2023-07-13T22:33:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.451+0000 7f6774f9b700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f675406ace0 0x7f675406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.451+0000 7f6774f9b700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f675406ace0 0x7f675406d190 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6764009c10 tx=0x7f6764008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:09.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.455+0000 7f676d7fa700 1 -- 172.21.15.60:0/199488887 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f67700c5740 con 0x7f67700d11e0 2023-07-13T22:33:09.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.660+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f67700c5700 con 0x7f67700d11e0 2023-07-13T22:33:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.660+0000 7f676d7fa700 1 -- 172.21.15.60:0/199488887 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6768058c10 con 0x7f67700d11e0 2023-07-13T22:33:09.647 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:33:09.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.662+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f675406ace0 msgr2=0x7f675406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.662+0000 7f67771ff700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f675406ace0 0x7f675406d190 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6764009c10 tx=0x7f6764008070 comp rx=0 tx=0).stop 2023-07-13T22:33:09.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.662+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 msgr2=0x7f67700c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.662+0000 7f67771ff700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 0x7f67700c5a20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f676800efc0 tx=0x7f676800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.663+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 shutdown_connections 2023-07-13T22:33:09.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.663+0000 7f67771ff700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f675406ace0 0x7f675406d190 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.663+0000 7f67771ff700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67700d11e0 0x7f67700c5a20 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.663+0000 7f67771ff700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f677006b500 0x7f677015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.663+0000 7f67771ff700 1 --2- 172.21.15.60:0/199488887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67700b1b70 0x7f67701f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.663+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 >> 172.21.15.60:0/199488887 conn(0x7f67700664f0 msgr2=0x7f67700d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:09.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.663+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 shutdown_connections 2023-07-13T22:33:09.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.663+0000 7f67771ff700 1 -- 172.21.15.60:0/199488887 wait complete. 2023-07-13T22:33:09.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:33:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.888+0000 7f215d1ff700 1 Processor -- start 2023-07-13T22:33:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.888+0000 7f215d1ff700 1 -- start start 2023-07-13T22:33:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.889+0000 7f215d1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f21580b8730 0x7f21580cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.889+0000 7f215d1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21580b4eb0 0x7f21580d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.889+0000 7f215d1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21580d06a0 con 0x7f2158160a30 2023-07-13T22:33:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.889+0000 7f215d1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21580cd810 con 0x7f21580b4eb0 2023-07-13T22:33:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.889+0000 7f215d1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21580d00e0 con 0x7f21580b8730 2023-07-13T22:33:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.889+0000 7f215759e700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21580b4eb0 0x7f21580d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.889+0000 7f215759e700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21580b4eb0 0x7f21580d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55726/0 (socket says 172.21.15.60:55726) 2023-07-13T22:33:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.889+0000 7f215759e700 1 -- 172.21.15.60:0/2329865225 learned_addr learned my addr 172.21.15.60:0/2329865225 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f2156d9d700 1 --2- 172.21.15.60:0/2329865225 >> v2:172.21.15.105:3300/0 conn(0x7f21580b8730 0x7f21580cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f215759e700 1 -- 172.21.15.60:0/2329865225 >> v2:172.21.15.105:3300/0 conn(0x7f21580b8730 msgr2=0x7f21580cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f215759e700 1 --2- 172.21.15.60:0/2329865225 >> v2:172.21.15.105:3300/0 conn(0x7f21580b8730 0x7f21580cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f215759e700 1 -- 172.21.15.60:0/2329865225 >> v1:172.21.15.105:6789/0 conn(0x7f2158160a30 legacy=0x7f21580b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f215759e700 1 -- 172.21.15.60:0/2329865225 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21580d03c0 con 0x7f21580b4eb0 2023-07-13T22:33:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f2156d9d700 1 --2- 172.21.15.60:0/2329865225 >> v2:172.21.15.105:3300/0 conn(0x7f21580b8730 0x7f21580cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f215759e700 1 --2- 172.21.15.60:0/2329865225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21580b4eb0 0x7f21580d08f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f21580d3e20 tx=0x7f215000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c862d76c81e44c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f2155d9b700 1 -- 172.21.15.60:0/2329865225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215000e070 con 0x7f21580b4eb0 2023-07-13T22:33:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f2155d9b700 1 -- 172.21.15.60:0/2329865225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2150003a20 con 0x7f21580b4eb0 2023-07-13T22:33:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.890+0000 7f2155d9b700 1 -- 172.21.15.60:0/2329865225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150003dd0 con 0x7f21580b4eb0 2023-07-13T22:33:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.891+0000 7f215d1ff700 1 -- 172.21.15.60:0/2329865225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21580b4eb0 msgr2=0x7f21580d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.891+0000 7f215d1ff700 1 --2- 172.21.15.60:0/2329865225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21580b4eb0 0x7f21580d08f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f21580d3e20 tx=0x7f215000b330 comp rx=0 tx=0).stop 2023-07-13T22:33:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.891+0000 7f215d1ff700 1 -- 172.21.15.60:0/2329865225 shutdown_connections 2023-07-13T22:33:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.891+0000 7f215d1ff700 1 --2- 172.21.15.60:0/2329865225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21580b4eb0 0x7f21580d08f0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.891+0000 7f215d1ff700 1 --2- 172.21.15.60:0/2329865225 >> v2:172.21.15.105:3300/0 conn(0x7f21580b8730 0x7f21580cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.891+0000 7f215d1ff700 1 -- 172.21.15.60:0/2329865225 >> 172.21.15.60:0/2329865225 conn(0x7f215806b1a0 msgr2=0x7f21580cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.891+0000 7f215d1ff700 1 -- 172.21.15.60:0/2329865225 shutdown_connections 2023-07-13T22:33:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.891+0000 7f215d1ff700 1 -- 172.21.15.60:0/2329865225 wait complete. 2023-07-13T22:33:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.893+0000 7f215d1ff700 1 Processor -- start 2023-07-13T22:33:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.893+0000 7f215d1ff700 1 -- start start 2023-07-13T22:33:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.894+0000 7f215d1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21580b4eb0 0x7f21580c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.894+0000 7f215d1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21580b8730 0x7f2158157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.894+0000 7f215d1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2158160a30 0x7f2158153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.894+0000 7f215d1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2158157740 con 0x7f2158160a30 2023-07-13T22:33:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.894+0000 7f215d1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21580c6ee0 con 0x7f21580b8730 2023-07-13T22:33:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.894+0000 7f215d1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21580c71c0 con 0x7f21580b4eb0 2023-07-13T22:33:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.894+0000 7f215759e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2158160a30 0x7f2158153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.894+0000 7f215759e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2158160a30 0x7f2158153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55738/0 (socket says 172.21.15.60:55738) 2023-07-13T22:33:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.894+0000 7f215759e700 1 -- 172.21.15.60:0/3212459319 learned_addr learned my addr 172.21.15.60:0/3212459319 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f2156d9d700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21580b4eb0 0x7f21580c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f215659c700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21580b8730 0x7f2158157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f215759e700 1 -- 172.21.15.60:0/3212459319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21580b4eb0 msgr2=0x7f21580c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f215759e700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21580b4eb0 0x7f21580c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f215759e700 1 -- 172.21.15.60:0/3212459319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21580b8730 msgr2=0x7f2158157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f215759e700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21580b8730 0x7f2158157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f215759e700 1 -- 172.21.15.60:0/3212459319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2150009b30 con 0x7f2158160a30 2023-07-13T22:33:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f215659c700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21580b8730 0x7f2158157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f215759e700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2158160a30 0x7f2158153c30 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f2150015040 tx=0x7f2150004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f213f7fe700 1 -- 172.21.15.60:0/3212459319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21500092e0 con 0x7f2158160a30 2023-07-13T22:33:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f213f7fe700 1 -- 172.21.15.60:0/3212459319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2150004240 con 0x7f2158160a30 2023-07-13T22:33:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f213f7fe700 1 -- 172.21.15.60:0/3212459319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150004630 con 0x7f2158160a30 2023-07-13T22:33:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.895+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21580c74a0 con 0x7f2158160a30 2023-07-13T22:33:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.896+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21580c7a60 con 0x7f2158160a30 2023-07-13T22:33:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.896+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2158160e90 con 0x7f2158160a30 2023-07-13T22:33:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.897+0000 7f213f7fe700 1 -- 172.21.15.60:0/3212459319 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2150019070 con 0x7f2158160a30 2023-07-13T22:33:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.899+0000 7f213f7fe700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:09.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.899+0000 7f213f7fe700 1 -- 172.21.15.60:0/3212459319 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f215008a510 con 0x7f2158160a30 2023-07-13T22:33:09.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.899+0000 7f2156d9d700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:09.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.900+0000 7f2156d9d700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2148006fd0 tx=0x7f2148008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:09.903+0000 7f213f7fe700 1 -- 172.21.15.60:0/3212459319 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f2158160e90 con 0x7f2158160a30 2023-07-13T22:33:10.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.110+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2158160e90 con 0x7f2158160a30 2023-07-13T22:33:10.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.110+0000 7f213f7fe700 1 -- 172.21.15.60:0/3212459319 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2158160e90 con 0x7f2158160a30 2023-07-13T22:33:10.097 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 msgr2=0x7f214406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2148006fd0 tx=0x7f2148008040 comp rx=0 tx=0).stop 2023-07-13T22:33:10.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2158160a30 msgr2=0x7f2158153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2158160a30 0x7f2158153c30 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f2150015040 tx=0x7f2150004ad0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 shutdown_connections 2023-07-13T22:33:10.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f214406ace0 0x7f214406d190 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2158160a30 0x7f2158153c30 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21580b4eb0 0x7f21580c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 --2- 172.21.15.60:0/3212459319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21580b8730 0x7f2158157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 >> 172.21.15.60:0/3212459319 conn(0x7f215806b1a0 msgr2=0x7f21580cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:10.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 shutdown_connections 2023-07-13T22:33:10.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.113+0000 7f215d1ff700 1 -- 172.21.15.60:0/3212459319 wait complete. 2023-07-13T22:33:10.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:33:10.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.336+0000 7fc14e7ff700 1 Processor -- start 2023-07-13T22:33:10.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.336+0000 7fc14e7ff700 1 -- start start 2023-07-13T22:33:10.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.338+0000 7fc14e7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1480cefc0 0x7fc1480b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.338+0000 7fc14e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 0x7fc1480d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.338+0000 7fc14e7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1480d06a0 con 0x7fc1480b8730 2023-07-13T22:33:10.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.338+0000 7fc14e7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1480cf440 con 0x7fc1480cefc0 2023-07-13T22:33:10.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.338+0000 7fc14e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480b5160 con 0x7fc1480b4ce0 2023-07-13T22:33:10.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.338+0000 7fc1477fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1480cefc0 0x7fc1480b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.338+0000 7fc1477fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1480cefc0 0x7fc1480b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:34544/0 (socket says 172.21.15.60:34544) 2023-07-13T22:33:10.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.338+0000 7fc147fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc1480b8730 0x7fc148160a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:49096/0 (socket says 172.21.15.60:49096) 2023-07-13T22:33:10.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.338+0000 7fc1477fe700 1 -- 172.21.15.60:0/3342632639 learned_addr learned my addr 172.21.15.60:0/3342632639 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:10.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.339+0000 7fc14cd9c700 1 --2- 172.21.15.60:0/3342632639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 0x7fc1480d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.339+0000 7fc146ffd700 1 -- 172.21.15.60:0/3342632639 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728431029 0 0) 0x7fc1480d06a0 con 0x7fc1480b8730 2023-07-13T22:33:10.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.339+0000 7fc146ffd700 1 -- 172.21.15.60:0/3342632639 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1300035b0 con 0x7fc1480b8730 2023-07-13T22:33:10.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.339+0000 7fc1477fe700 1 -- 172.21.15.60:0/3342632639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 msgr2=0x7fc1480d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:10.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.339+0000 7fc1477fe700 1 --2- 172.21.15.60:0/3342632639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 0x7fc1480d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.339+0000 7fc1477fe700 1 -- 172.21.15.60:0/3342632639 >> v1:172.21.15.105:6789/0 conn(0x7fc1480b8730 legacy=0x7fc148160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.339+0000 7fc1477fe700 1 -- 172.21.15.60:0/3342632639 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1480cd810 con 0x7fc1480cefc0 2023-07-13T22:33:10.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.339+0000 7fc14cd9c700 1 --2- 172.21.15.60:0/3342632639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 0x7fc1480d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:10.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.339+0000 7fc1477fe700 1 --2- 172.21.15.60:0/3342632639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1480cefc0 0x7fc1480b47a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0098f0 tx=0x7fc13c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=136bc0d8c0d51f4d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:10.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc146ffd700 1 -- 172.21.15.60:0/3342632639 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c011830 con 0x7fc1480cefc0 2023-07-13T22:33:10.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc146ffd700 1 -- 172.21.15.60:0/3342632639 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c011990 con 0x7fc1480cefc0 2023-07-13T22:33:10.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc146ffd700 1 -- 172.21.15.60:0/3342632639 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c011d20 con 0x7fc1480cefc0 2023-07-13T22:33:10.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc14e7ff700 1 -- 172.21.15.60:0/3342632639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1480cefc0 msgr2=0x7fc1480b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc14e7ff700 1 --2- 172.21.15.60:0/3342632639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1480cefc0 0x7fc1480b47a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0098f0 tx=0x7fc13c009c00 comp rx=0 tx=0).stop 2023-07-13T22:33:10.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc14e7ff700 1 -- 172.21.15.60:0/3342632639 shutdown_connections 2023-07-13T22:33:10.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc14e7ff700 1 --2- 172.21.15.60:0/3342632639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 0x7fc1480d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc14e7ff700 1 --2- 172.21.15.60:0/3342632639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1480cefc0 0x7fc1480b47a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc14e7ff700 1 -- 172.21.15.60:0/3342632639 >> 172.21.15.60:0/3342632639 conn(0x7fc14806b1a0 msgr2=0x7fc1480cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:10.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc14e7ff700 1 -- 172.21.15.60:0/3342632639 shutdown_connections 2023-07-13T22:33:10.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.340+0000 7fc14e7ff700 1 -- 172.21.15.60:0/3342632639 wait complete. 2023-07-13T22:33:10.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.343+0000 7fc14e7ff700 1 Processor -- start 2023-07-13T22:33:10.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.343+0000 7fc14e7ff700 1 -- start start 2023-07-13T22:33:10.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.344+0000 7fc14e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 0x7fc1480b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.344+0000 7fc14e7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1480b8730 0x7fc148156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.344+0000 7fc14e7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1480cefc0 0x7fc148156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.344+0000 7fc14e7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480ccd40 con 0x7fc1480cefc0 2023-07-13T22:33:10.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.344+0000 7fc14e7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480cace0 con 0x7fc1480b8730 2023-07-13T22:33:10.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.344+0000 7fc14e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480c9df0 con 0x7fc1480b4ce0 2023-07-13T22:33:10.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc14cd9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1480cefc0 0x7fc148156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc14cd9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1480cefc0 0x7fc148156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55754/0 (socket says 172.21.15.60:55754) 2023-07-13T22:33:10.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc14cd9c700 1 -- 172.21.15.60:0/2322598140 learned_addr learned my addr 172.21.15.60:0/2322598140 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:10.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc147fff700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 0x7fc1480b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc1477fe700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1480b8730 0x7fc148156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc14cd9c700 1 -- 172.21.15.60:0/2322598140 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 msgr2=0x7fc1480b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc14cd9c700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 0x7fc1480b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc14cd9c700 1 -- 172.21.15.60:0/2322598140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1480b8730 msgr2=0x7fc148156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc14cd9c700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1480b8730 0x7fc148156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc14cd9c700 1 -- 172.21.15.60:0/2322598140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc140009a70 con 0x7fc1480cefc0 2023-07-13T22:33:10.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc14cd9c700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1480cefc0 0x7fc148156550 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fc14000efc0 tx=0x7fc14000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:10.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.345+0000 7fc144ff9700 1 -- 172.21.15.60:0/2322598140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14000ca90 con 0x7fc1480cefc0 2023-07-13T22:33:10.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.346+0000 7fc144ff9700 1 -- 172.21.15.60:0/2322598140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14000f040 con 0x7fc1480cefc0 2023-07-13T22:33:10.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.346+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c009550 con 0x7fc1480cefc0 2023-07-13T22:33:10.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.346+0000 7fc144ff9700 1 -- 172.21.15.60:0/2322598140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14000ce30 con 0x7fc1480cefc0 2023-07-13T22:33:10.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.346+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1480ca3b0 con 0x7fc1480cefc0 2023-07-13T22:33:10.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.347+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1480cae90 con 0x7fc1480cefc0 2023-07-13T22:33:10.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.348+0000 7fc144ff9700 1 -- 172.21.15.60:0/2322598140 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc140014480 con 0x7fc1480cefc0 2023-07-13T22:33:10.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.348+0000 7fc144ff9700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc13406ace0 0x7fc13406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.349+0000 7fc144ff9700 1 -- 172.21.15.60:0/2322598140 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc140089c90 con 0x7fc1480cefc0 2023-07-13T22:33:10.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.349+0000 7fc147fff700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc13406ace0 0x7fc13406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.349+0000 7fc147fff700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc13406ace0 0x7fc13406d190 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc138007fd0 tx=0x7fc138007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:10.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.353+0000 7fc144ff9700 1 -- 172.21.15.60:0/2322598140 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc140054be0 con 0x7fc1480cefc0 2023-07-13T22:33:10.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.559+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc148074220 con 0x7fc1480cefc0 2023-07-13T22:33:10.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.559+0000 7fc144ff9700 1 -- 172.21.15.60:0/2322598140 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc140009cd0 con 0x7fc1480cefc0 2023-07-13T22:33:10.546 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:33:10.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc13406ace0 msgr2=0x7fc13406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc13406ace0 0x7fc13406d190 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc138007fd0 tx=0x7fc138007dc0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1480cefc0 msgr2=0x7fc148156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1480cefc0 0x7fc148156550 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fc14000efc0 tx=0x7fc14000c3a0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 shutdown_connections 2023-07-13T22:33:10.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc13406ace0 0x7fc13406d190 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1480cefc0 0x7fc148156550 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1480b4ce0 0x7fc1480b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 --2- 172.21.15.60:0/2322598140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1480b8730 0x7fc148156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 >> 172.21.15.60:0/2322598140 conn(0x7fc14806b1a0 msgr2=0x7fc1480b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:10.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 shutdown_connections 2023-07-13T22:33:10.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.562+0000 7fc14e7ff700 1 -- 172.21.15.60:0/2322598140 wait complete. 2023-07-13T22:33:10.566 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:33:10.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.787+0000 7f97a8dff700 1 Processor -- start 2023-07-13T22:33:10.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.787+0000 7f97a8dff700 1 -- start start 2023-07-13T22:33:10.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.788+0000 7f97a8dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97a40ce380 0x7f97a406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.788+0000 7f97a8dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a4070010 0x7f97a40b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.788+0000 7f97a8dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a40b4920 con 0x7f97a415a870 2023-07-13T22:33:10.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.788+0000 7f97a8dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a40aef80 con 0x7f97a40ce380 2023-07-13T22:33:10.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.788+0000 7f97a8dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40ce7f0 con 0x7f97a4070010 2023-07-13T22:33:10.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.788+0000 7f97a1d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97a40ce380 0x7f97a406fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a1d9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97a40ce380 0x7f97a406fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55762/0 (socket says 172.21.15.60:55762) 2023-07-13T22:33:10.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a1d9b700 1 -- 172.21.15.60:0/4003931874 learned_addr learned my addr 172.21.15.60:0/4003931874 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:10.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a2d9d700 1 --2- 172.21.15.60:0/4003931874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a4070010 0x7f97a40b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a1d9b700 1 -- 172.21.15.60:0/4003931874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a4070010 msgr2=0x7f97a40b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:10.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a1d9b700 1 --2- 172.21.15.60:0/4003931874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a4070010 0x7f97a40b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a1d9b700 1 -- 172.21.15.60:0/4003931874 >> v1:172.21.15.105:6789/0 conn(0x7f97a415a870 legacy=0x7f97a40adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a1d9b700 1 -- 172.21.15.60:0/4003931874 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a415ac40 con 0x7f97a40ce380 2023-07-13T22:33:10.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a1d9b700 1 --2- 172.21.15.60:0/4003931874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 0x7f97a406fad0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f97980098f0 tx=0x7f9798009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c885514c2f31876 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:10.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a159a700 1 -- 172.21.15.60:0/4003931874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798011830 con 0x7f97a40ce380 2023-07-13T22:33:10.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a159a700 1 -- 172.21.15.60:0/4003931874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9798011990 con 0x7f97a40ce380 2023-07-13T22:33:10.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.789+0000 7f97a159a700 1 -- 172.21.15.60:0/4003931874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798011d20 con 0x7f97a40ce380 2023-07-13T22:33:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.790+0000 7f97a8dff700 1 -- 172.21.15.60:0/4003931874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 msgr2=0x7f97a406fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.790+0000 7f97a8dff700 1 --2- 172.21.15.60:0/4003931874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 0x7f97a406fad0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f97980098f0 tx=0x7f9798009c00 comp rx=0 tx=0).stop 2023-07-13T22:33:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.790+0000 7f97a8dff700 1 -- 172.21.15.60:0/4003931874 shutdown_connections 2023-07-13T22:33:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.790+0000 7f97a8dff700 1 --2- 172.21.15.60:0/4003931874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a4070010 0x7f97a40b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.790+0000 7f97a8dff700 1 --2- 172.21.15.60:0/4003931874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 0x7f97a406fad0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.790+0000 7f97a8dff700 1 -- 172.21.15.60:0/4003931874 >> 172.21.15.60:0/4003931874 conn(0x7f97a40cf5e0 msgr2=0x7f97a40cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.790+0000 7f97a8dff700 1 -- 172.21.15.60:0/4003931874 shutdown_connections 2023-07-13T22:33:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.790+0000 7f97a8dff700 1 -- 172.21.15.60:0/4003931874 wait complete. 2023-07-13T22:33:10.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.793+0000 7f97a8dff700 1 Processor -- start 2023-07-13T22:33:10.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.793+0000 7f97a8dff700 1 -- start start 2023-07-13T22:33:10.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.794+0000 7f97a8dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a4070010 0x7f97a4155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.794+0000 7f97a8dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 0x7f97a41e0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.794+0000 7f97a8dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a415a870 0x7f97a41e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.794+0000 7f97a8dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a41566a0 con 0x7f97a40ce380 2023-07-13T22:33:10.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.794+0000 7f97a8dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a41e23f0 con 0x7f97a4070010 2023-07-13T22:33:10.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.794+0000 7f97a8dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a41e26d0 con 0x7f97a415a870 2023-07-13T22:33:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.794+0000 7f97a1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 0x7f97a41e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.794+0000 7f97a1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 0x7f97a41e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55766/0 (socket says 172.21.15.60:55766) 2023-07-13T22:33:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.794+0000 7f97a1d9b700 1 -- 172.21.15.60:0/3604147349 learned_addr learned my addr 172.21.15.60:0/3604147349 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a259c700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a4070010 0x7f97a4155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a2d9d700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a415a870 0x7f97a41e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a1d9b700 1 -- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a415a870 msgr2=0x7f97a41e31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a1d9b700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a415a870 0x7f97a41e31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a1d9b700 1 -- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a4070010 msgr2=0x7f97a4155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a1d9b700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a4070010 0x7f97a4155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a1d9b700 1 -- 172.21.15.60:0/3604147349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9798009550 con 0x7f97a40ce380 2023-07-13T22:33:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a259c700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a4070010 0x7f97a4155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a2d9d700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a415a870 0x7f97a41e31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a1d9b700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 0x7f97a41e0510 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f9798000c00 tx=0x7f9798004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f9792ffd700 1 -- 172.21.15.60:0/3604147349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798004570 con 0x7f97a40ce380 2023-07-13T22:33:10.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f9792ffd700 1 -- 172.21.15.60:0/3604147349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979801f070 con 0x7f97a40ce380 2023-07-13T22:33:10.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a41e29b0 con 0x7f97a40ce380 2023-07-13T22:33:10.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.795+0000 7f9792ffd700 1 -- 172.21.15.60:0/3604147349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980038c0 con 0x7f97a40ce380 2023-07-13T22:33:10.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.796+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a4156390 con 0x7f97a40ce380 2023-07-13T22:33:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.797+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a41e2ed0 con 0x7f97a40ce380 2023-07-13T22:33:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.798+0000 7f9792ffd700 1 -- 172.21.15.60:0/3604147349 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9798021030 con 0x7f97a40ce380 2023-07-13T22:33:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.798+0000 7f9792ffd700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f978406ace0 0x7f978406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.799+0000 7f9792ffd700 1 -- 172.21.15.60:0/3604147349 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f979808a9b0 con 0x7f97a40ce380 2023-07-13T22:33:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.799+0000 7f97a259c700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f978406ace0 0x7f978406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.799+0000 7f97a259c700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f978406ace0 0x7f978406d190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f97a40b0580 tx=0x7f9794009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:10.803+0000 7f9792ffd700 1 -- 172.21.15.60:0/3604147349 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f97a41e2ed0 con 0x7f97a40ce380 2023-07-13T22:33:10.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.012+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97a41e2ed0 con 0x7f97a40ce380 2023-07-13T22:33:10.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.012+0000 7f9792ffd700 1 -- 172.21.15.60:0/3604147349 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97a41e2ed0 con 0x7f97a40ce380 2023-07-13T22:33:10.999 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:33:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f978406ace0 msgr2=0x7f978406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f978406ace0 0x7f978406d190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f97a40b0580 tx=0x7f9794009040 comp rx=0 tx=0).stop 2023-07-13T22:33:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 msgr2=0x7f97a41e0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 0x7f97a41e0510 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f9798000c00 tx=0x7f9798004750 comp rx=0 tx=0).stop 2023-07-13T22:33:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 shutdown_connections 2023-07-13T22:33:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f978406ace0 0x7f978406d190 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97a415a870 0x7f97a41e31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a4070010 0x7f97a4155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 --2- 172.21.15.60:0/3604147349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97a40ce380 0x7f97a41e0510 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 >> 172.21.15.60:0/3604147349 conn(0x7f97a40cf5e0 msgr2=0x7f97a40cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:11.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 shutdown_connections 2023-07-13T22:33:11.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.015+0000 7f97a8dff700 1 -- 172.21.15.60:0/3604147349 wait complete. 2023-07-13T22:33:11.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:33:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.237+0000 7fa2b53ff700 1 Processor -- start 2023-07-13T22:33:11.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.238+0000 7fa2b53ff700 1 -- start start 2023-07-13T22:33:11.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.239+0000 7fa2b53ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2b015c6f0 0x7fa2b0160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:11.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.239+0000 7fa2b53ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2b00cf010 0x7fa2b00cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.239+0000 7fa2b53ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b00722e0 con 0x7fa2b00cfc00 2023-07-13T22:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.239+0000 7fa2b53ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b00d0df0 con 0x7fa2b00cf010 2023-07-13T22:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.239+0000 7fa2b53ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00cdbd0 con 0x7fa2b015c6f0 2023-07-13T22:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.239+0000 7fa2a7fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2b00cf010 0x7fa2b00cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.239+0000 7fa2af7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa2b00cfc00 0x7fa2b00b4760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46250/0 (socket says 172.21.15.60:46250) 2023-07-13T22:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.239+0000 7fa2af7fe700 1 -- 172.21.15.60:0/728981088 learned_addr learned my addr 172.21.15.60:0/728981088 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.239+0000 7fa2aeffd700 1 --2- 172.21.15.60:0/728981088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2b015c6f0 0x7fa2b0160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.240+0000 7fa2a7fff700 1 -- 172.21.15.60:0/728981088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2b015c6f0 msgr2=0x7fa2b0160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.240+0000 7fa2a7fff700 1 --2- 172.21.15.60:0/728981088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2b015c6f0 0x7fa2b0160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.240+0000 7fa2a7fff700 1 -- 172.21.15.60:0/728981088 >> v1:172.21.15.60:6789/0 conn(0x7fa2b00cfc00 legacy=0x7fa2b00b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.240+0000 7fa2ae7fc700 1 -- 172.21.15.60:0/728981088 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060936718 0 0) 0x7fa2a8001490 con 0x7fa2b00cfc00 2023-07-13T22:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.240+0000 7fa2a7fff700 1 -- 172.21.15.60:0/728981088 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b015cac0 con 0x7fa2b00cf010 2023-07-13T22:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.240+0000 7fa2a7fff700 1 --2- 172.21.15.60:0/728981088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 0x7fa2b00cf6c0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fa298009840 tx=0x7fa298009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b6ea6fc8d7d2e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.240+0000 7fa2ae7fc700 1 -- 172.21.15.60:0/728981088 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298011830 con 0x7fa2b00cf010 2023-07-13T22:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.240+0000 7fa2ae7fc700 1 -- 172.21.15.60:0/728981088 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa298011990 con 0x7fa2b00cf010 2023-07-13T22:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.240+0000 7fa2ae7fc700 1 -- 172.21.15.60:0/728981088 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298011d20 con 0x7fa2b00cf010 2023-07-13T22:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.241+0000 7fa2b53ff700 1 -- 172.21.15.60:0/728981088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 msgr2=0x7fa2b00cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.241+0000 7fa2b53ff700 1 --2- 172.21.15.60:0/728981088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 0x7fa2b00cf6c0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fa298009840 tx=0x7fa298009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.241+0000 7fa2b53ff700 1 -- 172.21.15.60:0/728981088 shutdown_connections 2023-07-13T22:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.241+0000 7fa2b53ff700 1 --2- 172.21.15.60:0/728981088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2b015c6f0 0x7fa2b0160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.241+0000 7fa2b53ff700 1 --2- 172.21.15.60:0/728981088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 0x7fa2b00cf6c0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.241+0000 7fa2b53ff700 1 -- 172.21.15.60:0/728981088 >> 172.21.15.60:0/728981088 conn(0x7fa2b006a4b0 msgr2=0x7fa2b006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.241+0000 7fa2b53ff700 1 -- 172.21.15.60:0/728981088 shutdown_connections 2023-07-13T22:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.241+0000 7fa2b53ff700 1 -- 172.21.15.60:0/728981088 wait complete. 2023-07-13T22:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.243+0000 7fa2b53ff700 1 Processor -- start 2023-07-13T22:33:11.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.243+0000 7fa2b53ff700 1 -- start start 2023-07-13T22:33:11.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2b53ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 0x7fa2b00c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:11.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2b53ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b00cfc00 0x7fa2b00c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:11.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2b53ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2b015c6f0 0x7fa2b00c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2b53ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00cc000 con 0x7fa2b00cf010 2023-07-13T22:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2b53ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00c7a50 con 0x7fa2b00cfc00 2023-07-13T22:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2b53ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00cb660 con 0x7fa2b015c6f0 2023-07-13T22:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2aeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 0x7fa2b00c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2aeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 0x7fa2b00c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55782/0 (socket says 172.21.15.60:55782) 2023-07-13T22:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2aeffd700 1 -- 172.21.15.60:0/154356776 learned_addr learned my addr 172.21.15.60:0/154356776 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2aeffd700 1 -- 172.21.15.60:0/154356776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2b015c6f0 msgr2=0x7fa2b00c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2aeffd700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2b015c6f0 0x7fa2b00c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2aeffd700 1 -- 172.21.15.60:0/154356776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b00cfc00 msgr2=0x7fa2b00c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2a7fff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b00cfc00 0x7fa2b00c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2aeffd700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b00cfc00 0x7fa2b00c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.245+0000 7fa2aeffd700 1 -- 172.21.15.60:0/154356776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2980094a0 con 0x7fa2b00cf010 2023-07-13T22:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.246+0000 7fa2a7fff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b00cfc00 0x7fa2b00c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.246+0000 7fa2aeffd700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 0x7fa2b00c8ec0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000e870 tx=0x7fa2a000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.246+0000 7fa2a77fe700 1 -- 172.21.15.60:0/154356776 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0009590 con 0x7fa2b00cf010 2023-07-13T22:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.246+0000 7fa2a77fe700 1 -- 172.21.15.60:0/154356776 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a000ee70 con 0x7fa2b00cf010 2023-07-13T22:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.246+0000 7fa2a77fe700 1 -- 172.21.15.60:0/154356776 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000ca50 con 0x7fa2b00cf010 2023-07-13T22:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.246+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b00cb940 con 0x7fa2b00cf010 2023-07-13T22:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.246+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b00c7740 con 0x7fa2b00cf010 2023-07-13T22:33:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.247+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b00cc270 con 0x7fa2b00cf010 2023-07-13T22:33:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.248+0000 7fa2a77fe700 1 -- 172.21.15.60:0/154356776 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa2a000cbb0 con 0x7fa2b00cf010 2023-07-13T22:33:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.249+0000 7fa2a77fe700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa29006ace0 0x7fa29006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.249+0000 7fa2a77fe700 1 -- 172.21.15.60:0/154356776 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa2a00894a0 con 0x7fa2b00cf010 2023-07-13T22:33:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.250+0000 7fa2a7fff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa29006ace0 0x7fa29006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.250+0000 7fa2a7fff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa29006ace0 0x7fa29006d190 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00cc870 tx=0x7fa298008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:11.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.253+0000 7fa2a77fe700 1 -- 172.21.15.60:0/154356776 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa2a0057f90 con 0x7fa2b00cf010 2023-07-13T22:33:11.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.475+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa2b0074220 con 0x7fa2b00cf010 2023-07-13T22:33:11.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.476+0000 7fa2a77fe700 1 -- 172.21.15.60:0/154356776 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v137) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa2a000ce60 con 0x7fa2b00cf010 2023-07-13T22:33:11.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa29006ace0 msgr2=0x7fa29006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:11.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa29006ace0 0x7fa29006d190 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00cc870 tx=0x7fa298008030 comp rx=0 tx=0).stop 2023-07-13T22:33:11.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 msgr2=0x7fa2b00c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:11.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 0x7fa2b00c8ec0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000e870 tx=0x7fa2a000ec30 comp rx=0 tx=0).stop 2023-07-13T22:33:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 shutdown_connections 2023-07-13T22:33:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa29006ace0 0x7fa29006d190 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2b00cf010 0x7fa2b00c8ec0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2b015c6f0 0x7fa2b00c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 --2- 172.21.15.60:0/154356776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b00cfc00 0x7fa2b00c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 >> 172.21.15.60:0/154356776 conn(0x7fa2b006a4b0 msgr2=0x7fa2b0156c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 shutdown_connections 2023-07-13T22:33:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:11.478+0000 7fa2b53ff700 1 -- 172.21.15.60:0/154356776 wait complete. 2023-07-13T22:33:11.468 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-07-13T22:33:11.483 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2023-07-13T22:33:11.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:11.484 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-13T22:33:11.557 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi105.stderr:ceph-fuse[128014]: fuse finished with error 0 and tester_r 0 2023-07-13T22:33:11.560 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:33:11.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:11.561 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-13T22:33:11.724 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:33:11.724 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:33:11.774 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:33:11.774 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:33:11.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:11.775 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-13T22:33:11.860 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-13T22:33:11.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:33:11.862 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:33:11.862 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:33:11.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:33:11.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:33:11.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:11.910 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-13T22:33:11.994 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-13T22:33:11.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:33:11.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:33:12.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.212+0000 7f4b44dff700 1 Processor -- start 2023-07-13T22:33:12.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.213+0000 7f4b44dff700 1 -- start start 2023-07-13T22:33:12.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.214+0000 7f4b44dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b4015c6f0 0x7f4b40160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.214+0000 7f4b44dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b400cf010 0x7f4b400cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.214+0000 7f4b44dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b400722e0 con 0x7f4b400cfc00 2023-07-13T22:33:12.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.214+0000 7f4b44dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b400d0df0 con 0x7f4b4015c6f0 2023-07-13T22:33:12.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.214+0000 7f4b44dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400cdbd0 con 0x7f4b400cf010 2023-07-13T22:33:12.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.214+0000 7f4b3ed9d700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4b400cfc00 0x7f4b400b4760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46264/0 (socket says 172.21.15.60:46264) 2023-07-13T22:33:12.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.214+0000 7f4b3ed9d700 1 -- 172.21.15.60:0/434360944 learned_addr learned my addr 172.21.15.60:0/434360944 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3e59c700 1 --2- 172.21.15.60:0/434360944 >> v2:172.21.15.105:3300/0 conn(0x7f4b4015c6f0 0x7f4b40160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b37fff700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b400cf010 0x7f4b400cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991636242 0 0) 0x7f4b38001490 con 0x7f4b400cfc00 2023-07-13T22:33:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b2c0035b0 con 0x7f4b400cfc00 2023-07-13T22:33:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2280767541 0 0) 0x7f4b2c0035b0 con 0x7f4b400cfc00 2023-07-13T22:33:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b400cf010 msgr2=0x7f4b400cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b400cf010 0x7f4b400cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 >> v2:172.21.15.105:3300/0 conn(0x7f4b4015c6f0 msgr2=0x7f4b40160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 --2- 172.21.15.60:0/434360944 >> v2:172.21.15.105:3300/0 conn(0x7f4b4015c6f0 0x7f4b40160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4015cac0 con 0x7f4b400cfc00 2023-07-13T22:33:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7f4b38001b80 con 0x7f4b400cfc00 2023-07-13T22:33:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3e59c700 1 --2- 172.21.15.60:0/434360944 >> v2:172.21.15.105:3300/0 conn(0x7f4b4015c6f0 0x7f4b40160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 >> v1:172.21.15.60:6789/0 conn(0x7f4b400cfc00 legacy=0x7f4b400b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b400cf010 0x7f4b2c003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c003ff0 0x7f4b2c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c004d90 0x7f4b2c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400d0df0 con 0x7f4b2c004d90 2023-07-13T22:33:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400cdbd0 con 0x7f4b2c003ff0 2023-07-13T22:33:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.215+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c00ba90 con 0x7f4b400cf010 2023-07-13T22:33:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b3e59c700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b400cf010 0x7f4b2c003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b3ed9d700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c004d90 0x7f4b2c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b37fff700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c003ff0 0x7f4b2c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b37fff700 1 -- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b400cf010 msgr2=0x7f4b2c003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:12.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b37fff700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b400cf010 0x7f4b2c003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b37fff700 1 -- 172.21.15.60:0/434360944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c004d90 msgr2=0x7f4b2c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:12.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b37fff700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c004d90 0x7f4b2c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b37fff700 1 -- 172.21.15.60:0/434360944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0035b0 con 0x7f4b2c003ff0 2023-07-13T22:33:12.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b3ed9d700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c004d90 0x7f4b2c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:12.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b37fff700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c003ff0 0x7f4b2c004850 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009b60 tx=0x7f4b2c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdd7938c30d45c09 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:12.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800e6c0 con 0x7f4b2c003ff0 2023-07-13T22:33:12.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2800e820 con 0x7f4b2c003ff0 2023-07-13T22:33:12.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.216+0000 7f4b3dd9b700 1 -- 172.21.15.60:0/434360944 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800eb90 con 0x7f4b2c003ff0 2023-07-13T22:33:12.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.217+0000 7f4b44dff700 1 -- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c003ff0 msgr2=0x7f4b2c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.217+0000 7f4b44dff700 1 --2- 172.21.15.60:0/434360944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c003ff0 0x7f4b2c004850 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009b60 tx=0x7f4b2c005880 comp rx=0 tx=0).stop 2023-07-13T22:33:12.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.217+0000 7f4b3ed9d700 1 -- 172.21.15.60:0/434360944 reap_dead start 2023-07-13T22:33:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.217+0000 7f4b44dff700 1 -- 172.21.15.60:0/434360944 shutdown_connections 2023-07-13T22:33:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.217+0000 7f4b44dff700 1 -- 172.21.15.60:0/434360944 >> 172.21.15.60:0/434360944 conn(0x7f4b4006a4b0 msgr2=0x7f4b4006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.217+0000 7f4b44dff700 1 -- 172.21.15.60:0/434360944 shutdown_connections 2023-07-13T22:33:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.217+0000 7f4b44dff700 1 -- 172.21.15.60:0/434360944 wait complete. 2023-07-13T22:33:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.220+0000 7f4b44dff700 1 Processor -- start 2023-07-13T22:33:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.220+0000 7f4b44dff700 1 -- start start 2023-07-13T22:33:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b44dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c003ff0 0x7f4b40153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b44dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b400b5200 0x7f4b400b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b44dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b40154010 0x7f4b40152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b44dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400ca3f0 con 0x7f4b2c003ff0 2023-07-13T22:33:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b44dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400ca6d0 con 0x7f4b400b5200 2023-07-13T22:33:12.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b44dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400ca9b0 con 0x7f4b40154010 2023-07-13T22:33:12.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b3e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c003ff0 0x7f4b40153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b3e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c003ff0 0x7f4b40153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55802/0 (socket says 172.21.15.60:55802) 2023-07-13T22:33:12.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b3e59c700 1 -- 172.21.15.60:0/1039394046 learned_addr learned my addr 172.21.15.60:0/1039394046 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:12.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.221+0000 7f4b37fff700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b400b5200 0x7f4b400b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b3ed9d700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b40154010 0x7f4b40152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b3e59c700 1 -- 172.21.15.60:0/1039394046 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b40154010 msgr2=0x7f4b40152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b3e59c700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b40154010 0x7f4b40152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b3e59c700 1 -- 172.21.15.60:0/1039394046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b400b5200 msgr2=0x7f4b400b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b3e59c700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b400b5200 0x7f4b400b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b3e59c700 1 -- 172.21.15.60:0/1039394046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b400cac90 con 0x7f4b2c003ff0 2023-07-13T22:33:12.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b3e59c700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c003ff0 0x7f4b40153a30 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f4b30011c20 tx=0x7f4b30011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:12.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b36ffd700 1 -- 172.21.15.60:0/1039394046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000f9f0 con 0x7f4b2c003ff0 2023-07-13T22:33:12.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b36ffd700 1 -- 172.21.15.60:0/1039394046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3000fb50 con 0x7f4b2c003ff0 2023-07-13T22:33:12.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b36ffd700 1 -- 172.21.15.60:0/1039394046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000fee0 con 0x7f4b2c003ff0 2023-07-13T22:33:12.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.222+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b400b7310 con 0x7f4b2c003ff0 2023-07-13T22:33:12.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.223+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b40153500 con 0x7f4b2c003ff0 2023-07-13T22:33:12.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.223+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b400b74a0 con 0x7f4b2c003ff0 2023-07-13T22:33:12.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.225+0000 7f4b36ffd700 1 -- 172.21.15.60:0/1039394046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b30004020 con 0x7f4b2c003ff0 2023-07-13T22:33:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.225+0000 7f4b36ffd700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2006ace0 0x7f4b2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.226+0000 7f4b36ffd700 1 -- 172.21.15.60:0/1039394046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4b30089ac0 con 0x7f4b2c003ff0 2023-07-13T22:33:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.226+0000 7f4b37fff700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2006ace0 0x7f4b2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.226+0000 7f4b37fff700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2006ace0 0x7f4b2006d190 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4b280090b0 tx=0x7f4b2801f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:12.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.230+0000 7f4b36ffd700 1 -- 172.21.15.60:0/1039394046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4b30054a90 con 0x7f4b2c003ff0 2023-07-13T22:33:12.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.486+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b400ca530 con 0x7f4b2c003ff0 2023-07-13T22:33:12.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.487+0000 7f4b36ffd700 1 -- 172.21.15.60:0/1039394046 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f4b400ca530 con 0x7f4b2c003ff0 2023-07-13T22:33:12.473 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:12.474 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3652157896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3652157896},{"type":"v1","addr":"172.21.15.105:6834","nonce":3652157896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1878256861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1878256861},{"type":"v1","addr":"172.21.15.105:6835","nonce":1878256861}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92},{"gid":8728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2695331242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2695331242},{"type":"v1","addr":"172.21.15.60:6836","nonce":2695331242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:31:29.408608+0000","modified":"2023-07-13T22:31:32.919029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9423":{"gid":9423,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4061691799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4061691799},{"type":"v1","addr":"172.21.15.60:6837","nonce":4061691799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-13T22:33:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2006ace0 msgr2=0x7f4b2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2006ace0 0x7f4b2006d190 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4b280090b0 tx=0x7f4b2801f040 comp rx=0 tx=0).stop 2023-07-13T22:33:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c003ff0 msgr2=0x7f4b40153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c003ff0 0x7f4b40153a30 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f4b30011c20 tx=0x7f4b30011f30 comp rx=0 tx=0).stop 2023-07-13T22:33:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 shutdown_connections 2023-07-13T22:33:12.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4b2006ace0 0x7f4b2006d190 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b2c003ff0 0x7f4b40153a30 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b40154010 0x7f4b40152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 --2- 172.21.15.60:0/1039394046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b400b5200 0x7f4b400b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 >> 172.21.15.60:0/1039394046 conn(0x7f4b4006a4b0 msgr2=0x7f4b4015cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:12.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 shutdown_connections 2023-07-13T22:33:12.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.489+0000 7f4b44dff700 1 -- 172.21.15.60:0/1039394046 wait complete. 2023-07-13T22:33:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 97 2023-07-13T22:33:12.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:33:12.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.711+0000 7fe46cdff700 1 Processor -- start 2023-07-13T22:33:12.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.712+0000 7fe46cdff700 1 -- start start 2023-07-13T22:33:12.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe46cdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe46815c6f0 0x7fe468160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe46cdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cf010 0x7fe4680cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe46cdff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe4680cfc00 0x7fe4680b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe46cdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4680722e0 con 0x7fe46815c6f0 2023-07-13T22:33:12.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe46cdff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4680d0df0 con 0x7fe4680cfc00 2023-07-13T22:33:12.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe46cdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680cdbd0 con 0x7fe4680cf010 2023-07-13T22:33:12.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe46659c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe46815c6f0 0x7fe468160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe46659c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe46815c6f0 0x7fe468160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55812/0 (socket says 172.21.15.60:55812) 2023-07-13T22:33:12.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe46659c700 1 -- 172.21.15.60:0/937639113 learned_addr learned my addr 172.21.15.60:0/937639113 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:12.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe466d9d700 1 --2- 172.21.15.60:0/937639113 >> v2:172.21.15.105:3300/0 conn(0x7fe4680cfc00 0x7fe4680b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.713+0000 7fe45ffff700 1 --2- 172.21.15.60:0/937639113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cf010 0x7fe4680cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe46659c700 1 -- 172.21.15.60:0/937639113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cf010 msgr2=0x7fe4680cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:12.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe46659c700 1 --2- 172.21.15.60:0/937639113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cf010 0x7fe4680cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe46659c700 1 -- 172.21.15.60:0/937639113 >> v2:172.21.15.105:3300/0 conn(0x7fe4680cfc00 msgr2=0x7fe4680b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:12.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe46659c700 1 --2- 172.21.15.60:0/937639113 >> v2:172.21.15.105:3300/0 conn(0x7fe4680cfc00 0x7fe4680b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe46659c700 1 -- 172.21.15.60:0/937639113 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46815cac0 con 0x7fe46815c6f0 2023-07-13T22:33:12.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe45ffff700 1 --2- 172.21.15.60:0/937639113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cf010 0x7fe4680cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:12.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe46659c700 1 --2- 172.21.15.60:0/937639113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe46815c6f0 0x7fe468160b70 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe4680b58d0 tx=0x7fe458009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf55d3d2da741e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:12.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe465d9b700 1 -- 172.21.15.60:0/937639113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458011830 con 0x7fe46815c6f0 2023-07-13T22:33:12.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe465d9b700 1 -- 172.21.15.60:0/937639113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe458011990 con 0x7fe46815c6f0 2023-07-13T22:33:12.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.714+0000 7fe465d9b700 1 -- 172.21.15.60:0/937639113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458011d20 con 0x7fe46815c6f0 2023-07-13T22:33:12.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.715+0000 7fe46cdff700 1 -- 172.21.15.60:0/937639113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe46815c6f0 msgr2=0x7fe468160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.715+0000 7fe46cdff700 1 --2- 172.21.15.60:0/937639113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe46815c6f0 0x7fe468160b70 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe4680b58d0 tx=0x7fe458009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.715+0000 7fe46cdff700 1 -- 172.21.15.60:0/937639113 shutdown_connections 2023-07-13T22:33:12.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.715+0000 7fe46cdff700 1 --2- 172.21.15.60:0/937639113 >> v2:172.21.15.105:3300/0 conn(0x7fe4680cfc00 0x7fe4680b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.715+0000 7fe46cdff700 1 --2- 172.21.15.60:0/937639113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe46815c6f0 0x7fe468160b70 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.715+0000 7fe46cdff700 1 --2- 172.21.15.60:0/937639113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cf010 0x7fe4680cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.715+0000 7fe46cdff700 1 -- 172.21.15.60:0/937639113 >> 172.21.15.60:0/937639113 conn(0x7fe46806a4b0 msgr2=0x7fe46806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:12.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.715+0000 7fe46cdff700 1 -- 172.21.15.60:0/937639113 shutdown_connections 2023-07-13T22:33:12.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.715+0000 7fe46cdff700 1 -- 172.21.15.60:0/937639113 wait complete. 2023-07-13T22:33:12.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.717+0000 7fe46cdff700 1 Processor -- start 2023-07-13T22:33:12.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.717+0000 7fe46cdff700 1 -- start start 2023-07-13T22:33:12.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46cdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4680cf010 0x7fe4680caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46cdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cfc00 0x7fe4680cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46cdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46815c6f0 0x7fe4680b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46cdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680cb290 con 0x7fe4680cf010 2023-07-13T22:33:12.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46cdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680cb570 con 0x7fe46815c6f0 2023-07-13T22:33:12.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46cdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680cb850 con 0x7fe4680cfc00 2023-07-13T22:33:12.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4680cf010 0x7fe4680caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4680cf010 0x7fe4680caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55828/0 (socket says 172.21.15.60:55828) 2023-07-13T22:33:12.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46659c700 1 -- 172.21.15.60:0/1604063644 learned_addr learned my addr 172.21.15.60:0/1604063644 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:12.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe45ffff700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cfc00 0x7fe4680cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe466d9d700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46815c6f0 0x7fe4680b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46659c700 1 -- 172.21.15.60:0/1604063644 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cfc00 msgr2=0x7fe4680cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.718+0000 7fe46659c700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cfc00 0x7fe4680cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.719+0000 7fe46659c700 1 -- 172.21.15.60:0/1604063644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46815c6f0 msgr2=0x7fe4680b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.719+0000 7fe46659c700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46815c6f0 0x7fe4680b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.719+0000 7fe46659c700 1 -- 172.21.15.60:0/1604063644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe458009550 con 0x7fe4680cf010 2023-07-13T22:33:12.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.719+0000 7fe46659c700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4680cf010 0x7fe4680caa40 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe458009c10 tx=0x7fe458004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:12.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.719+0000 7fe45effd700 1 -- 172.21.15.60:0/1604063644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458004450 con 0x7fe4680cf010 2023-07-13T22:33:12.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.719+0000 7fe45effd700 1 -- 172.21.15.60:0/1604063644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45801e070 con 0x7fe4680cf010 2023-07-13T22:33:12.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.719+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4680cbb30 con 0x7fe4680cf010 2023-07-13T22:33:12.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.719+0000 7fe45effd700 1 -- 172.21.15.60:0/1604063644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4580038e0 con 0x7fe4680cf010 2023-07-13T22:33:12.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.719+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4680caf80 con 0x7fe4680cf010 2023-07-13T22:33:12.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.720+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4680cc050 con 0x7fe4680cf010 2023-07-13T22:33:12.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.721+0000 7fe45effd700 1 -- 172.21.15.60:0/1604063644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe458003a40 con 0x7fe4680cf010 2023-07-13T22:33:12.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.722+0000 7fe45effd700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe44806ace0 0x7fe44806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:12.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.722+0000 7fe45effd700 1 -- 172.21.15.60:0/1604063644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe45808a330 con 0x7fe4680cf010 2023-07-13T22:33:12.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.722+0000 7fe45ffff700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe44806ace0 0x7fe44806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:12.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.723+0000 7fe45ffff700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe44806ace0 0x7fe44806d190 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe46815e050 tx=0x7fe450008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:12.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.726+0000 7fe45effd700 1 -- 172.21.15.60:0/1604063644 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe4680cc050 con 0x7fe4680cf010 2023-07-13T22:33:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.939+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4680cc050 con 0x7fe4680cf010 2023-07-13T22:33:12.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.940+0000 7fe45effd700 1 -- 172.21.15.60:0/1604063644 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+18363 (secure 0 0 0) 0x7fe4680cc050 con 0x7fe4680cf010 2023-07-13T22:33:12.927 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:12.927 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":137,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:46.593649+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:31:27.702135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-13T22:31:28.386570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.7","mappings":[{"from":3,"to":0}]},{"pgid":"22.c","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:33:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe44806ace0 msgr2=0x7fe44806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe44806ace0 0x7fe44806d190 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe46815e050 tx=0x7fe450008040 comp rx=0 tx=0).stop 2023-07-13T22:33:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4680cf010 msgr2=0x7fe4680caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4680cf010 0x7fe4680caa40 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe458009c10 tx=0x7fe458004750 comp rx=0 tx=0).stop 2023-07-13T22:33:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 shutdown_connections 2023-07-13T22:33:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe44806ace0 0x7fe44806d190 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4680cf010 0x7fe4680caa40 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46815c6f0 0x7fe4680b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 --2- 172.21.15.60:0/1604063644 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe4680cfc00 0x7fe4680cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 >> 172.21.15.60:0/1604063644 conn(0x7fe46806a4b0 msgr2=0x7fe4680b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 shutdown_connections 2023-07-13T22:33:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:12.943+0000 7fe46cdff700 1 -- 172.21.15.60:0/1604063644 wait complete. 2023-07-13T22:33:12.948 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-13T22:20:46.388037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:20:47.173552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:31:27.702135+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:31:28.386570+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:33:12.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-13T22:33:13.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.169+0000 7fb52edff700 1 Processor -- start 2023-07-13T22:33:13.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.169+0000 7fb52edff700 1 -- start start 2023-07-13T22:33:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52edff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5280b8730 0x7fb5280cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52edff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb528160a30 0x7fb5280b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52edff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b4ce0 0x7fb5280715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52edff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5280b5160 con 0x7fb528160a30 2023-07-13T22:33:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52edff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5280d06a0 con 0x7fb5280b8730 2023-07-13T22:33:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52edff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280cd810 con 0x7fb5280b4ce0 2023-07-13T22:33:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb527fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb528160a30 0x7fb5280b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb527fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb528160a30 0x7fb5280b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55840/0 (socket says 172.21.15.60:55840) 2023-07-13T22:33:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb527fff700 1 -- 172.21.15.60:0/2483629768 learned_addr learned my addr 172.21.15.60:0/2483629768 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb527fff700 1 -- 172.21.15.60:0/2483629768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b4ce0 msgr2=0x7fb5280715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52cb9b700 1 --2- 172.21.15.60:0/2483629768 >> v2:172.21.15.105:3300/0 conn(0x7fb5280b8730 0x7fb5280cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52d39c700 1 --2- 172.21.15.60:0/2483629768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b4ce0 0x7fb5280715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb527fff700 1 --2- 172.21.15.60:0/2483629768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b4ce0 0x7fb5280715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb527fff700 1 -- 172.21.15.60:0/2483629768 >> v2:172.21.15.105:3300/0 conn(0x7fb5280b8730 msgr2=0x7fb5280cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:13.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb527fff700 1 --2- 172.21.15.60:0/2483629768 >> v2:172.21.15.105:3300/0 conn(0x7fb5280b8730 0x7fb5280cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb527fff700 1 -- 172.21.15.60:0/2483629768 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5280d00e0 con 0x7fb528160a30 2023-07-13T22:33:13.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52cb9b700 1 --2- 172.21.15.60:0/2483629768 >> v2:172.21.15.105:3300/0 conn(0x7fb5280b8730 0x7fb5280cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:13.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.171+0000 7fb52d39c700 1 --2- 172.21.15.60:0/2483629768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b4ce0 0x7fb5280715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:13.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.172+0000 7fb527fff700 1 --2- 172.21.15.60:0/2483629768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb528160a30 0x7fb5280b47a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb5280d3e20 tx=0x7fb514009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fa424fa047373e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:13.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.172+0000 7fb5277fe700 1 -- 172.21.15.60:0/2483629768 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400b870 con 0x7fb528160a30 2023-07-13T22:33:13.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.172+0000 7fb5277fe700 1 -- 172.21.15.60:0/2483629768 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51400b9d0 con 0x7fb528160a30 2023-07-13T22:33:13.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.172+0000 7fb5277fe700 1 -- 172.21.15.60:0/2483629768 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400bd60 con 0x7fb528160a30 2023-07-13T22:33:13.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.172+0000 7fb52edff700 1 -- 172.21.15.60:0/2483629768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb528160a30 msgr2=0x7fb5280b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.172+0000 7fb52edff700 1 --2- 172.21.15.60:0/2483629768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb528160a30 0x7fb5280b47a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb5280d3e20 tx=0x7fb514009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.173+0000 7fb52edff700 1 -- 172.21.15.60:0/2483629768 shutdown_connections 2023-07-13T22:33:13.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.173+0000 7fb52edff700 1 --2- 172.21.15.60:0/2483629768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b4ce0 0x7fb5280715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.173+0000 7fb52edff700 1 --2- 172.21.15.60:0/2483629768 >> v2:172.21.15.105:3300/0 conn(0x7fb5280b8730 0x7fb5280cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.173+0000 7fb52edff700 1 --2- 172.21.15.60:0/2483629768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb528160a30 0x7fb5280b47a0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.173+0000 7fb52edff700 1 -- 172.21.15.60:0/2483629768 >> 172.21.15.60:0/2483629768 conn(0x7fb52806b1a0 msgr2=0x7fb5280cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:13.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.173+0000 7fb52edff700 1 -- 172.21.15.60:0/2483629768 shutdown_connections 2023-07-13T22:33:13.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.173+0000 7fb52edff700 1 -- 172.21.15.60:0/2483629768 wait complete. 2023-07-13T22:33:13.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.174+0000 7fb52edff700 1 Processor -- start 2023-07-13T22:33:13.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.175+0000 7fb52edff700 1 -- start start 2023-07-13T22:33:13.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52edff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5280b4ce0 0x7fb5281f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52edff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b8730 0x7fb52815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52edff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb528160a30 0x7fb52815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52edff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815f140 con 0x7fb5280b4ce0 2023-07-13T22:33:13.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52edff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815d770 con 0x7fb528160a30 2023-07-13T22:33:13.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52edff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815cb60 con 0x7fb5280b8730 2023-07-13T22:33:13.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52cb9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5280b4ce0 0x7fb5281f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52cb9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5280b4ce0 0x7fb5281f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55850/0 (socket says 172.21.15.60:55850) 2023-07-13T22:33:13.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52cb9b700 1 -- 172.21.15.60:0/3187598280 learned_addr learned my addr 172.21.15.60:0/3187598280 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:13.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb527fff700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b8730 0x7fb52815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52d39c700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb528160a30 0x7fb52815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52cb9b700 1 -- 172.21.15.60:0/3187598280 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b8730 msgr2=0x7fb52815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52cb9b700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b8730 0x7fb52815dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52cb9b700 1 -- 172.21.15.60:0/3187598280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb528160a30 msgr2=0x7fb52815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52cb9b700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb528160a30 0x7fb52815e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.176+0000 7fb52cb9b700 1 -- 172.21.15.60:0/3187598280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514009550 con 0x7fb5280b4ce0 2023-07-13T22:33:13.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.177+0000 7fb52cb9b700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5280b4ce0 0x7fb5281f3020 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c7f0 tx=0x7fb51c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:13.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.177+0000 7fb5257fa700 1 -- 172.21.15.60:0/3187598280 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c011800 con 0x7fb5280b4ce0 2023-07-13T22:33:13.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.177+0000 7fb5257fa700 1 -- 172.21.15.60:0/3187598280 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c011960 con 0x7fb5280b4ce0 2023-07-13T22:33:13.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.177+0000 7fb5257fa700 1 -- 172.21.15.60:0/3187598280 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c011cf0 con 0x7fb5280b4ce0 2023-07-13T22:33:13.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.177+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52815ce40 con 0x7fb5280b4ce0 2023-07-13T22:33:13.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.177+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52815d400 con 0x7fb5280b4ce0 2023-07-13T22:33:13.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.178+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb528074220 con 0x7fb5280b4ce0 2023-07-13T22:33:13.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.179+0000 7fb5257fa700 1 -- 172.21.15.60:0/3187598280 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb51c018360 con 0x7fb5280b4ce0 2023-07-13T22:33:13.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.180+0000 7fb5257fa700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb50c06ace0 0x7fb50c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.180+0000 7fb5257fa700 1 -- 172.21.15.60:0/3187598280 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb51c089420 con 0x7fb5280b4ce0 2023-07-13T22:33:13.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.181+0000 7fb527fff700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb50c06ace0 0x7fb50c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.181+0000 7fb527fff700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb50c06ace0 0x7fb50c06d190 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb514014040 tx=0x7fb514008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:13.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.184+0000 7fb5257fa700 1 -- 172.21.15.60:0/3187598280 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb51c0543f0 con 0x7fb5280b4ce0 2023-07-13T22:33:13.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.391+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb52815c7c0 con 0x7fb5280b4ce0 2023-07-13T22:33:13.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.392+0000 7fb5257fa700 1 -- 172.21.15.60:0/3187598280 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb51c057a10 con 0x7fb5280b4ce0 2023-07-13T22:33:13.378 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 8 2023-07-13T22:33:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb50c06ace0 msgr2=0x7fb50c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb50c06ace0 0x7fb50c06d190 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb514014040 tx=0x7fb514008030 comp rx=0 tx=0).stop 2023-07-13T22:33:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5280b4ce0 msgr2=0x7fb5281f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5280b4ce0 0x7fb5281f3020 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c7f0 tx=0x7fb51c00cb00 comp rx=0 tx=0).stop 2023-07-13T22:33:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 shutdown_connections 2023-07-13T22:33:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb50c06ace0 0x7fb50c06d190 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5280b4ce0 0x7fb5281f3020 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb528160a30 0x7fb52815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 --2- 172.21.15.60:0/3187598280 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5280b8730 0x7fb52815dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 >> 172.21.15.60:0/3187598280 conn(0x7fb52806b1a0 msgr2=0x7fb5280d1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 shutdown_connections 2023-07-13T22:33:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.394+0000 7fb52edff700 1 -- 172.21.15.60:0/3187598280 wait complete. 2023-07-13T22:33:13.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:33:13.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.621+0000 7fa892bff700 1 Processor -- start 2023-07-13T22:33:13.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.621+0000 7fa892bff700 1 -- start start 2023-07-13T22:33:13.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.622+0000 7fa892bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa88c0cefc0 0x7fa88c0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa892bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0b4ce0 0x7fa88c0d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa892bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c0d06a0 con 0x7fa88c0b8730 2023-07-13T22:33:13.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa892bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c0cf440 con 0x7fa88c0cefc0 2023-07-13T22:33:13.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa892bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0b5160 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa89099b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa88c0b8730 0x7fa88c160a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46268/0 (socket says 172.21.15.60:46268) 2023-07-13T22:33:13.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa89099b700 1 -- 172.21.15.60:0/2456790113 learned_addr learned my addr 172.21.15.60:0/2456790113 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:13.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa89119c700 1 --2- 172.21.15.60:0/2456790113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0b4ce0 0x7fa88c0d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa88bfff700 1 --2- 172.21.15.60:0/2456790113 >> v2:172.21.15.105:3300/0 conn(0x7fa88c0cefc0 0x7fa88c0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa88b7fe700 1 -- 172.21.15.60:0/2456790113 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690875479 0 0) 0x7fa88c0d06a0 con 0x7fa88c0b8730 2023-07-13T22:33:13.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.623+0000 7fa88b7fe700 1 -- 172.21.15.60:0/2456790113 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c0035b0 con 0x7fa88c0b8730 2023-07-13T22:33:13.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.624+0000 7fa88b7fe700 1 -- 172.21.15.60:0/2456790113 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2966775518 0 0) 0x7fa87c0035b0 con 0x7fa88c0b8730 2023-07-13T22:33:13.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.624+0000 7fa89119c700 1 -- 172.21.15.60:0/2456790113 >> v2:172.21.15.105:3300/0 conn(0x7fa88c0cefc0 msgr2=0x7fa88c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:13.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.624+0000 7fa89119c700 1 --2- 172.21.15.60:0/2456790113 >> v2:172.21.15.105:3300/0 conn(0x7fa88c0cefc0 0x7fa88c0b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.624+0000 7fa89119c700 1 -- 172.21.15.60:0/2456790113 >> v1:172.21.15.60:6789/0 conn(0x7fa88c0b8730 legacy=0x7fa88c160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.624+0000 7fa89119c700 1 -- 172.21.15.60:0/2456790113 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c0cd810 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.624+0000 7fa89119c700 1 --2- 172.21.15.60:0/2456790113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0b4ce0 0x7fa88c0d08f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa884009d90 tx=0x7fa88400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c04da932540ad432 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:13.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.624+0000 7fa88b7fe700 1 -- 172.21.15.60:0/2456790113 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400e070 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.624+0000 7fa88b7fe700 1 -- 172.21.15.60:0/2456790113 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa884003a20 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.624+0000 7fa88b7fe700 1 -- 172.21.15.60:0/2456790113 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880001670 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.625+0000 7fa892bff700 1 -- 172.21.15.60:0/2456790113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0b4ce0 msgr2=0x7fa88c0d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.625+0000 7fa892bff700 1 --2- 172.21.15.60:0/2456790113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0b4ce0 0x7fa88c0d08f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa884009d90 tx=0x7fa88400b330 comp rx=0 tx=0).stop 2023-07-13T22:33:13.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.625+0000 7fa892bff700 1 -- 172.21.15.60:0/2456790113 shutdown_connections 2023-07-13T22:33:13.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.625+0000 7fa892bff700 1 --2- 172.21.15.60:0/2456790113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0b4ce0 0x7fa88c0d08f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.625+0000 7fa892bff700 1 --2- 172.21.15.60:0/2456790113 >> v2:172.21.15.105:3300/0 conn(0x7fa88c0cefc0 0x7fa88c0b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.625+0000 7fa892bff700 1 -- 172.21.15.60:0/2456790113 >> 172.21.15.60:0/2456790113 conn(0x7fa88c06b1a0 msgr2=0x7fa88c0cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:13.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.625+0000 7fa892bff700 1 -- 172.21.15.60:0/2456790113 shutdown_connections 2023-07-13T22:33:13.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.625+0000 7fa892bff700 1 -- 172.21.15.60:0/2456790113 wait complete. 2023-07-13T22:33:13.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.627+0000 7fa892bff700 1 Processor -- start 2023-07-13T22:33:13.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.627+0000 7fa892bff700 1 -- start start 2023-07-13T22:33:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa892bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c0b4ce0 0x7fa88c15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa892bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b8730 0x7fa88c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa892bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0cefc0 0x7fa88c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa892bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0c77e0 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa892bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0c9720 con 0x7fa88c0b8730 2023-07-13T22:33:13.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa892bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c1e2950 con 0x7fa88c0cefc0 2023-07-13T22:33:13.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa89099b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c0b4ce0 0x7fa88c15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa89099b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c0b4ce0 0x7fa88c15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55864/0 (socket says 172.21.15.60:55864) 2023-07-13T22:33:13.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa89099b700 1 -- 172.21.15.60:0/688188164 learned_addr learned my addr 172.21.15.60:0/688188164 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:13.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa88bfff700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b8730 0x7fa88c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.628+0000 7fa89119c700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0cefc0 0x7fa88c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa89099b700 1 -- 172.21.15.60:0/688188164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0cefc0 msgr2=0x7fa88c0c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa89099b700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0cefc0 0x7fa88c0c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa89099b700 1 -- 172.21.15.60:0/688188164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b8730 msgr2=0x7fa88c0c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa89099b700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b8730 0x7fa88c0c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa89099b700 1 -- 172.21.15.60:0/688188164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa884009a70 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa89119c700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0cefc0 0x7fa88c0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa89099b700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c0b4ce0 0x7fa88c15ace0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fa88000c7b0 tx=0x7fa88000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:13.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa8897fa700 1 -- 172.21.15.60:0/688188164 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880004c80 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa8897fa700 1 -- 172.21.15.60:0/688188164 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880004de0 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa8897fa700 1 -- 172.21.15.60:0/688188164 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800072f0 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c1e2c30 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.629+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c0c8140 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.630+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c0c7260 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.631+0000 7fa8897fa700 1 -- 172.21.15.60:0/688188164 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa880007450 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.632+0000 7fa8897fa700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa87006ace0 0x7fa87006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:13.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.633+0000 7fa8897fa700 1 -- 172.21.15.60:0/688188164 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa880089180 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.633+0000 7fa88bfff700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa87006ace0 0x7fa87006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:13.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.633+0000 7fa88bfff700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa87006ace0 0x7fa87006d190 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa878009be0 tx=0x7fa878009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:13.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.637+0000 7fa8897fa700 1 -- 172.21.15.60:0/688188164 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa88c0c7260 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.843+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa88c0c7260 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.843+0000 7fa8897fa700 1 -- 172.21.15.60:0/688188164 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa88c0c7260 con 0x7fa88c0b4ce0 2023-07-13T22:33:13.830 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-07-13T22:33:13.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa87006ace0 msgr2=0x7fa87006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa87006ace0 0x7fa87006d190 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa878009be0 tx=0x7fa878009210 comp rx=0 tx=0).stop 2023-07-13T22:33:13.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c0b4ce0 msgr2=0x7fa88c15ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:13.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c0b4ce0 0x7fa88c15ace0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fa88000c7b0 tx=0x7fa88000cac0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 shutdown_connections 2023-07-13T22:33:13.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa87006ace0 0x7fa87006d190 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa88c0b4ce0 0x7fa88c15ace0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa88c0cefc0 0x7fa88c0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 --2- 172.21.15.60:0/688188164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88c0b8730 0x7fa88c0c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:13.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 >> 172.21.15.60:0/688188164 conn(0x7fa88c06b1a0 msgr2=0x7fa88c0d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:13.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 shutdown_connections 2023-07-13T22:33:13.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:13.845+0000 7fa892bff700 1 -- 172.21.15.60:0/688188164 wait complete. 2023-07-13T22:33:13.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:33:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.072+0000 7ff1cadff700 1 Processor -- start 2023-07-13T22:33:14.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.072+0000 7ff1cadff700 1 -- start start 2023-07-13T22:33:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.073+0000 7ff1cadff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c40ce380 0x7ff1c406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.073+0000 7ff1cadff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff1c4070010 0x7ff1c40b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.073+0000 7ff1cadff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c40b4920 con 0x7ff1c415a870 2023-07-13T22:33:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.073+0000 7ff1cadff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c40aef80 con 0x7ff1c4070010 2023-07-13T22:33:14.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.073+0000 7ff1cadff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40ce7f0 con 0x7ff1c40ce380 2023-07-13T22:33:14.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.073+0000 7ff1c8b9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff1c415a870 0x7ff1c40adf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46274/0 (socket says 172.21.15.60:46274) 2023-07-13T22:33:14.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c8b9b700 1 -- 172.21.15.60:0/1832629050 learned_addr learned my addr 172.21.15.60:0/1832629050 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:14.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c939c700 1 --2- 172.21.15.60:0/1832629050 >> v2:172.21.15.60:3300/0 conn(0x7ff1c4070010 0x7ff1c40b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c3fff700 1 --2- 172.21.15.60:0/1832629050 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c40ce380 0x7ff1c406fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c939c700 1 -- 172.21.15.60:0/1832629050 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c40ce380 msgr2=0x7ff1c406fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c939c700 1 --2- 172.21.15.60:0/1832629050 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c40ce380 0x7ff1c406fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c939c700 1 -- 172.21.15.60:0/1832629050 >> v1:172.21.15.60:6789/0 conn(0x7ff1c415a870 legacy=0x7ff1c40adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c37fe700 1 -- 172.21.15.60:0/1832629050 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520779882 0 0) 0x7ff1c40b4920 con 0x7ff1c415a870 2023-07-13T22:33:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c939c700 1 -- 172.21.15.60:0/1832629050 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c415ac40 con 0x7ff1c4070010 2023-07-13T22:33:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c3fff700 1 --2- 172.21.15.60:0/1832629050 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c40ce380 0x7ff1c406fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:14.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c939c700 1 --2- 172.21.15.60:0/1832629050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c4070010 0x7ff1c40b0b80 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009d90 tx=0x7ff1bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d26ab2c36c7ad1b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:14.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.074+0000 7ff1c37fe700 1 -- 172.21.15.60:0/1832629050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc00e070 con 0x7ff1c4070010 2023-07-13T22:33:14.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1c37fe700 1 -- 172.21.15.60:0/1832629050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc00bbd0 con 0x7ff1c4070010 2023-07-13T22:33:14.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1c37fe700 1 -- 172.21.15.60:0/1832629050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc012430 con 0x7ff1c4070010 2023-07-13T22:33:14.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1cadff700 1 -- 172.21.15.60:0/1832629050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c4070010 msgr2=0x7ff1c40b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1cadff700 1 --2- 172.21.15.60:0/1832629050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c4070010 0x7ff1c40b0b80 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009d90 tx=0x7ff1bc00b330 comp rx=0 tx=0).stop 2023-07-13T22:33:14.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1cadff700 1 -- 172.21.15.60:0/1832629050 shutdown_connections 2023-07-13T22:33:14.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1cadff700 1 --2- 172.21.15.60:0/1832629050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c4070010 0x7ff1c40b0b80 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1cadff700 1 --2- 172.21.15.60:0/1832629050 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c40ce380 0x7ff1c406fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1cadff700 1 -- 172.21.15.60:0/1832629050 >> 172.21.15.60:0/1832629050 conn(0x7ff1c40cf5e0 msgr2=0x7ff1c40cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:14.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1cadff700 1 -- 172.21.15.60:0/1832629050 shutdown_connections 2023-07-13T22:33:14.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.075+0000 7ff1cadff700 1 -- 172.21.15.60:0/1832629050 wait complete. 2023-07-13T22:33:14.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.077+0000 7ff1cadff700 1 Processor -- start 2023-07-13T22:33:14.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.077+0000 7ff1cadff700 1 -- start start 2023-07-13T22:33:14.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.078+0000 7ff1cadff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c4070010 0x7ff1c4157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.078+0000 7ff1cadff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c40ce380 0x7ff1c406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.078+0000 7ff1cadff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c415a870 0x7ff1c406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.078+0000 7ff1cadff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c415c5c0 con 0x7ff1c415a870 2023-07-13T22:33:14.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.078+0000 7ff1cadff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40ad470 con 0x7ff1c40ce380 2023-07-13T22:33:14.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.078+0000 7ff1cadff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c41582b0 con 0x7ff1c4070010 2023-07-13T22:33:14.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.078+0000 7ff1c939c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c415a870 0x7ff1c406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.078+0000 7ff1c939c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c415a870 0x7ff1c406ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55874/0 (socket says 172.21.15.60:55874) 2023-07-13T22:33:14.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.078+0000 7ff1c939c700 1 -- 172.21.15.60:0/452588940 learned_addr learned my addr 172.21.15.60:0/452588940 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:14.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c8b9b700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c4070010 0x7ff1c4157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c3fff700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c40ce380 0x7ff1c406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c939c700 1 -- 172.21.15.60:0/452588940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c4070010 msgr2=0x7ff1c4157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c939c700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c4070010 0x7ff1c4157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c939c700 1 -- 172.21.15.60:0/452588940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c40ce380 msgr2=0x7ff1c406c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c939c700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c40ce380 0x7ff1c406c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c939c700 1 -- 172.21.15.60:0/452588940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc009a70 con 0x7ff1c415a870 2023-07-13T22:33:14.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c3fff700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c40ce380 0x7ff1c406c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:14.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c939c700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c415a870 0x7ff1c406ccf0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc013000 tx=0x7ff1bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:14.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c17fa700 1 -- 172.21.15.60:0/452588940 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc003ea0 con 0x7ff1c415a870 2023-07-13T22:33:14.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c17fa700 1 -- 172.21.15.60:0/452588940 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc020070 con 0x7ff1c415a870 2023-07-13T22:33:14.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c415bfb0 con 0x7ff1c415a870 2023-07-13T22:33:14.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1c17fa700 1 -- 172.21.15.60:0/452588940 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc004280 con 0x7ff1c415a870 2023-07-13T22:33:14.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.079+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c415c250 con 0x7ff1c415a870 2023-07-13T22:33:14.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.080+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c415c730 con 0x7ff1c415a870 2023-07-13T22:33:14.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.081+0000 7ff1c17fa700 1 -- 172.21.15.60:0/452588940 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1bc0043e0 con 0x7ff1c415a870 2023-07-13T22:33:14.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.083+0000 7ff1c17fa700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1a806ace0 0x7ff1a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.083+0000 7ff1c17fa700 1 -- 172.21.15.60:0/452588940 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff1bc022070 con 0x7ff1c415a870 2023-07-13T22:33:14.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.083+0000 7ff1c8b9b700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1a806ace0 0x7ff1a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.083+0000 7ff1c8b9b700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1a806ace0 0x7ff1a806d190 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8000f60 tx=0x7ff1b8007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:14.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.087+0000 7ff1c17fa700 1 -- 172.21.15.60:0/452588940 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7ff1bc056620 con 0x7ff1c415a870 2023-07-13T22:33:14.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.290+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1c4074220 con 0x7ff1c415a870 2023-07-13T22:33:14.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.291+0000 7ff1c17fa700 1 -- 172.21.15.60:0/452588940 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1bc059c40 con 0x7ff1c415a870 2023-07-13T22:33:14.277 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-07-13T22:33:14.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1a806ace0 msgr2=0x7ff1a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1a806ace0 0x7ff1a806d190 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8000f60 tx=0x7ff1b8007c90 comp rx=0 tx=0).stop 2023-07-13T22:33:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c415a870 msgr2=0x7ff1c406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c415a870 0x7ff1c406ccf0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc013000 tx=0x7ff1bc004ad0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 shutdown_connections 2023-07-13T22:33:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7ff1a806ace0 0x7ff1a806d190 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1c415a870 0x7ff1c406ccf0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1c4070010 0x7ff1c4157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 --2- 172.21.15.60:0/452588940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1c40ce380 0x7ff1c406c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 >> 172.21.15.60:0/452588940 conn(0x7ff1c40cf5e0 msgr2=0x7ff1c40b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 shutdown_connections 2023-07-13T22:33:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.293+0000 7ff1cadff700 1 -- 172.21.15.60:0/452588940 wait complete. 2023-07-13T22:33:14.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:33:14.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.517+0000 7f7a26dff700 1 Processor -- start 2023-07-13T22:33:14.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.517+0000 7f7a26dff700 1 -- start start 2023-07-13T22:33:14.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a26dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a2015ca10 0x7f7a2006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a26dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a200cee30 0x7f7a2006a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a26dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2006cc80 con 0x7f7a20069eb0 2023-07-13T22:33:14.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a26dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2006a890 con 0x7f7a200cee30 2023-07-13T22:33:14.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a26dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2015cde0 con 0x7f7a2015ca10 2023-07-13T22:33:14.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a2539c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a200cee30 0x7f7a2006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a1ffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7a20069eb0 0x7f7a200ce720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46276/0 (socket says 172.21.15.60:46276) 2023-07-13T22:33:14.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a1ffff700 1 -- 172.21.15.60:0/1281564245 learned_addr learned my addr 172.21.15.60:0/1281564245 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:14.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a24b9b700 1 --2- 172.21.15.60:0/1281564245 >> v2:172.21.15.105:3300/0 conn(0x7f7a2015ca10 0x7f7a2006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a2539c700 1 -- 172.21.15.60:0/1281564245 >> v2:172.21.15.105:3300/0 conn(0x7f7a2015ca10 msgr2=0x7f7a2006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:14.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a2539c700 1 --2- 172.21.15.60:0/1281564245 >> v2:172.21.15.105:3300/0 conn(0x7f7a2015ca10 0x7f7a2006d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a2539c700 1 -- 172.21.15.60:0/1281564245 >> v1:172.21.15.60:6789/0 conn(0x7f7a20069eb0 legacy=0x7f7a200ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a2539c700 1 -- 172.21.15.60:0/1281564245 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a200cf390 con 0x7f7a200cee30 2023-07-13T22:33:14.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.519+0000 7f7a1f7fe700 1 -- 172.21.15.60:0/1281564245 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894419352 0 0) 0x7f7a2006cc80 con 0x7f7a20069eb0 2023-07-13T22:33:14.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.520+0000 7f7a2539c700 1 --2- 172.21.15.60:0/1281564245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a200cee30 0x7f7a2006a350 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f7a2006ad80 tx=0x7f7a1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c4acffa764179a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:14.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.520+0000 7f7a1f7fe700 1 -- 172.21.15.60:0/1281564245 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800e070 con 0x7f7a200cee30 2023-07-13T22:33:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.520+0000 7f7a1f7fe700 1 -- 172.21.15.60:0/1281564245 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1800bb20 con 0x7f7a200cee30 2023-07-13T22:33:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.520+0000 7f7a1f7fe700 1 -- 172.21.15.60:0/1281564245 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18012430 con 0x7f7a200cee30 2023-07-13T22:33:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.520+0000 7f7a26dff700 1 -- 172.21.15.60:0/1281564245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a200cee30 msgr2=0x7f7a2006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.520+0000 7f7a26dff700 1 --2- 172.21.15.60:0/1281564245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a200cee30 0x7f7a2006a350 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f7a2006ad80 tx=0x7f7a1800b330 comp rx=0 tx=0).stop 2023-07-13T22:33:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.521+0000 7f7a26dff700 1 -- 172.21.15.60:0/1281564245 shutdown_connections 2023-07-13T22:33:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.521+0000 7f7a26dff700 1 --2- 172.21.15.60:0/1281564245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a200cee30 0x7f7a2006a350 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.521+0000 7f7a26dff700 1 --2- 172.21.15.60:0/1281564245 >> v2:172.21.15.105:3300/0 conn(0x7f7a2015ca10 0x7f7a2006d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.521+0000 7f7a26dff700 1 -- 172.21.15.60:0/1281564245 >> 172.21.15.60:0/1281564245 conn(0x7f7a200b15a0 msgr2=0x7f7a200b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.521+0000 7f7a26dff700 1 -- 172.21.15.60:0/1281564245 shutdown_connections 2023-07-13T22:33:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.521+0000 7f7a26dff700 1 -- 172.21.15.60:0/1281564245 wait complete. 2023-07-13T22:33:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.522+0000 7f7a26dff700 1 Processor -- start 2023-07-13T22:33:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.523+0000 7f7a26dff700 1 -- start start 2023-07-13T22:33:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.523+0000 7f7a26dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a20069eb0 0x7f7a20155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a26dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a200cee30 0x7f7a200c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a26dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a2015ca10 0x7f7a200b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a26dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a20155d70 con 0x7f7a2015ca10 2023-07-13T22:33:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a26dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200cba90 con 0x7f7a20069eb0 2023-07-13T22:33:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a26dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200cbd70 con 0x7f7a200cee30 2023-07-13T22:33:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a2539c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a2015ca10 0x7f7a200b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a2539c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a2015ca10 0x7f7a200b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55898/0 (socket says 172.21.15.60:55898) 2023-07-13T22:33:14.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a2539c700 1 -- 172.21.15.60:0/3223918806 learned_addr learned my addr 172.21.15.60:0/3223918806 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:14.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a24b9b700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a20069eb0 0x7f7a20155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a1ffff700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a200cee30 0x7f7a200c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a2539c700 1 -- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a200cee30 msgr2=0x7f7a200c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a2539c700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a200cee30 0x7f7a200c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a2539c700 1 -- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a20069eb0 msgr2=0x7f7a20155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a2539c700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a20069eb0 0x7f7a20155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a2539c700 1 -- 172.21.15.60:0/3223918806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a18009a70 con 0x7f7a2015ca10 2023-07-13T22:33:14.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a24b9b700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a20069eb0 0x7f7a20155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:14.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.524+0000 7f7a1ffff700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a200cee30 0x7f7a200c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:14.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.525+0000 7f7a2539c700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a2015ca10 0x7f7a200b48a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7a18014040 tx=0x7f7a18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:14.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.525+0000 7f7a1d7fa700 1 -- 172.21.15.60:0/3223918806 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18012430 con 0x7f7a2015ca10 2023-07-13T22:33:14.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.525+0000 7f7a1d7fa700 1 -- 172.21.15.60:0/3223918806 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18020070 con 0x7f7a2015ca10 2023-07-13T22:33:14.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.525+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a200cc050 con 0x7f7a2015ca10 2023-07-13T22:33:14.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.525+0000 7f7a1d7fa700 1 -- 172.21.15.60:0/3223918806 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18004280 con 0x7f7a2015ca10 2023-07-13T22:33:14.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.525+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a200c35e0 con 0x7f7a2015ca10 2023-07-13T22:33:14.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.526+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a20074220 con 0x7f7a2015ca10 2023-07-13T22:33:14.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.527+0000 7f7a1d7fa700 1 -- 172.21.15.60:0/3223918806 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a180043e0 con 0x7f7a2015ca10 2023-07-13T22:33:14.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.528+0000 7f7a1d7fa700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a0406ace0 0x7f7a0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.529+0000 7f7a1d7fa700 1 -- 172.21.15.60:0/3223918806 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7a18022070 con 0x7f7a2015ca10 2023-07-13T22:33:14.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.529+0000 7f7a24b9b700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a0406ace0 0x7f7a0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.529+0000 7f7a24b9b700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a0406ace0 0x7f7a0406d190 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009b60 tx=0x7f7a14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:14.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.532+0000 7f7a1d7fa700 1 -- 172.21.15.60:0/3223918806 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7a180562c0 con 0x7f7a2015ca10 2023-07-13T22:33:14.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.738+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a2006e8c0 con 0x7f7a2015ca10 2023-07-13T22:33:14.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.739+0000 7f7a1d7fa700 1 -- 172.21.15.60:0/3223918806 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a180598e0 con 0x7f7a2015ca10 2023-07-13T22:33:14.725 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-07-13T22:33:14.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a0406ace0 msgr2=0x7f7a0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a0406ace0 0x7f7a0406d190 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009b60 tx=0x7f7a14008040 comp rx=0 tx=0).stop 2023-07-13T22:33:14.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a2015ca10 msgr2=0x7f7a200b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a2015ca10 0x7f7a200b48a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7a18014040 tx=0x7f7a18004ad0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 shutdown_connections 2023-07-13T22:33:14.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a0406ace0 0x7f7a0406d190 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a2015ca10 0x7f7a200b48a0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a20069eb0 0x7f7a20155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 --2- 172.21.15.60:0/3223918806 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a200cee30 0x7f7a200c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 >> 172.21.15.60:0/3223918806 conn(0x7f7a200b15a0 msgr2=0x7f7a2006d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:14.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 shutdown_connections 2023-07-13T22:33:14.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.741+0000 7f7a26dff700 1 -- 172.21.15.60:0/3223918806 wait complete. 2023-07-13T22:33:14.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:33:14.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.962+0000 7fa1b4bff700 1 Processor -- start 2023-07-13T22:33:14.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.963+0000 7fa1b4bff700 1 -- start start 2023-07-13T22:33:14.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1b4bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1b00adc90 0x7fa1b00b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1b4bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 0x7fa1b00ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1b4bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1b00ca6e0 0x7fa1b00c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1b4bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b00b0d10 con 0x7fa1b00adc90 2023-07-13T22:33:14.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1b4bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b015e020 con 0x7fa1b00ca6e0 2023-07-13T22:33:14.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1b4bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b015e270 con 0x7fa1b00b08a0 2023-07-13T22:33:14.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1ae59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1b00adc90 0x7fa1b00b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1ae59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1b00adc90 0x7fa1b00b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52954/0 (socket says 172.21.15.60:52954) 2023-07-13T22:33:14.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1ae59c700 1 -- 172.21.15.60:0/3956482613 learned_addr learned my addr 172.21.15.60:0/3956482613 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:14.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1aed9d700 1 --2- 172.21.15.60:0/3956482613 >> v2:172.21.15.105:3300/0 conn(0x7fa1b00ca6e0 0x7fa1b00c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.964+0000 7fa1a7fff700 1 --2- 172.21.15.60:0/3956482613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 0x7fa1b00ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.965+0000 7fa1ae59c700 1 -- 172.21.15.60:0/3956482613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 msgr2=0x7fa1b00ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:14.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.965+0000 7fa1ae59c700 1 --2- 172.21.15.60:0/3956482613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 0x7fa1b00ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.965+0000 7fa1ae59c700 1 -- 172.21.15.60:0/3956482613 >> v2:172.21.15.105:3300/0 conn(0x7fa1b00ca6e0 msgr2=0x7fa1b00c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:14.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.965+0000 7fa1ae59c700 1 --2- 172.21.15.60:0/3956482613 >> v2:172.21.15.105:3300/0 conn(0x7fa1b00ca6e0 0x7fa1b00c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.965+0000 7fa1ae59c700 1 -- 172.21.15.60:0/3956482613 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b015e550 con 0x7fa1b00adc90 2023-07-13T22:33:14.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.965+0000 7fa1ae59c700 1 --2- 172.21.15.60:0/3956482613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 0x7fa1b00b0360 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00cbf10 tx=0x7fa1a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f8e86ea2e466b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:14.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.965+0000 7fa1add9b700 1 -- 172.21.15.60:0/3956482613 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000b870 con 0x7fa1b00adc90 2023-07-13T22:33:14.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.965+0000 7fa1add9b700 1 -- 172.21.15.60:0/3956482613 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a000b9d0 con 0x7fa1b00adc90 2023-07-13T22:33:14.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.965+0000 7fa1add9b700 1 -- 172.21.15.60:0/3956482613 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000bd60 con 0x7fa1b00adc90 2023-07-13T22:33:14.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.966+0000 7fa1b4bff700 1 -- 172.21.15.60:0/3956482613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 msgr2=0x7fa1b00b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.966+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/3956482613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 0x7fa1b00b0360 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00cbf10 tx=0x7fa1a0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.966+0000 7fa1b4bff700 1 -- 172.21.15.60:0/3956482613 shutdown_connections 2023-07-13T22:33:14.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.966+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/3956482613 >> v2:172.21.15.105:3300/0 conn(0x7fa1b00ca6e0 0x7fa1b00c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.966+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/3956482613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 0x7fa1b00b0360 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.966+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/3956482613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 0x7fa1b00ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.966+0000 7fa1b4bff700 1 -- 172.21.15.60:0/3956482613 >> 172.21.15.60:0/3956482613 conn(0x7fa1b00cb360 msgr2=0x7fa1b00cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:14.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.966+0000 7fa1b4bff700 1 -- 172.21.15.60:0/3956482613 shutdown_connections 2023-07-13T22:33:14.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.966+0000 7fa1b4bff700 1 -- 172.21.15.60:0/3956482613 wait complete. 2023-07-13T22:33:14.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.968+0000 7fa1b4bff700 1 Processor -- start 2023-07-13T22:33:14.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.969+0000 7fa1b4bff700 1 -- start start 2023-07-13T22:33:14.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1b4bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 0x7fa1b01f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1b4bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 0x7fa1b0150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1b4bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1b00ca6e0 0x7fa1b00c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1b4bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b01f0190 con 0x7fa1b00adc90 2023-07-13T22:33:14.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1b4bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b015d5f0 con 0x7fa1b00ca6e0 2023-07-13T22:33:14.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1b4bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b015d8b0 con 0x7fa1b00b08a0 2023-07-13T22:33:14.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1ae59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 0x7fa1b01f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1ae59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 0x7fa1b01f05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52960/0 (socket says 172.21.15.60:52960) 2023-07-13T22:33:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1ae59c700 1 -- 172.21.15.60:0/2304281827 learned_addr learned my addr 172.21.15.60:0/2304281827 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.970+0000 7fa1a7fff700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 0x7fa1b0150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1aed9d700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1b00ca6e0 0x7fa1b00c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1ae59c700 1 -- 172.21.15.60:0/2304281827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 msgr2=0x7fa1b0150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1ae59c700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 0x7fa1b0150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1ae59c700 1 -- 172.21.15.60:0/2304281827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1b00ca6e0 msgr2=0x7fa1b00c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1ae59c700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1b00ca6e0 0x7fa1b00c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1ae59c700 1 -- 172.21.15.60:0/2304281827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a0009550 con 0x7fa1b00adc90 2023-07-13T22:33:14.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1aed9d700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1b00ca6e0 0x7fa1b00c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:14.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1ae59c700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 0x7fa1b01f05b0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0000c00 tx=0x7fa1a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:14.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1a6ffd700 1 -- 172.21.15.60:0/2304281827 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004540 con 0x7fa1b00adc90 2023-07-13T22:33:14.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1a6ffd700 1 -- 172.21.15.60:0/2304281827 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a00036a0 con 0x7fa1b00adc90 2023-07-13T22:33:14.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1a6ffd700 1 -- 172.21.15.60:0/2304281827 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0003a50 con 0x7fa1b00adc90 2023-07-13T22:33:14.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b015db90 con 0x7fa1b00adc90 2023-07-13T22:33:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.971+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b00c5cb0 con 0x7fa1b00adc90 2023-07-13T22:33:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.972+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b0074220 con 0x7fa1b00adc90 2023-07-13T22:33:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.973+0000 7fa1a6ffd700 1 -- 172.21.15.60:0/2304281827 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa1a0021070 con 0x7fa1b00adc90 2023-07-13T22:33:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.974+0000 7fa1a6ffd700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa19006ace0 0x7fa19006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.974+0000 7fa1a6ffd700 1 -- 172.21.15.60:0/2304281827 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa1a008ac00 con 0x7fa1b00adc90 2023-07-13T22:33:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.974+0000 7fa1a7fff700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa19006ace0 0x7fa19006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.975+0000 7fa1a7fff700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa19006ace0 0x7fa19006d190 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c900 tx=0x7fa198008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:14.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:14.979+0000 7fa1a6ffd700 1 -- 172.21.15.60:0/2304281827 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fa1a0055c60 con 0x7fa1b00adc90 2023-07-13T22:33:15.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.231+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1b00cbf40 con 0x7fa1b00adc90 2023-07-13T22:33:15.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.232+0000 7fa1a6ffd700 1 -- 172.21.15.60:0/2304281827 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4719 (secure 0 0 0) 0x7fa1a0015090 con 0x7fa1b00adc90 2023-07-13T22:33:15.218 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:15.218 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3652157896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3652157896},{"type":"v1","addr":"172.21.15.105:6834","nonce":3652157896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1878256861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1878256861},{"type":"v1","addr":"172.21.15.105:6835","nonce":1878256861}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92},{"gid":8728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2695331242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2695331242},{"type":"v1","addr":"172.21.15.60:6836","nonce":2695331242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:31:29.408608+0000","modified":"2023-07-13T22:31:32.919029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9423":{"gid":9423,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4061691799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4061691799},{"type":"v1","addr":"172.21.15.60:6837","nonce":4061691799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-13T22:33:15.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa19006ace0 msgr2=0x7fa19006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa19006ace0 0x7fa19006d190 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c900 tx=0x7fa198008040 comp rx=0 tx=0).stop 2023-07-13T22:33:15.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 msgr2=0x7fa1b01f05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 0x7fa1b01f05b0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0000c00 tx=0x7fa1a0004750 comp rx=0 tx=0).stop 2023-07-13T22:33:15.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 shutdown_connections 2023-07-13T22:33:15.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fa19006ace0 0x7fa19006d190 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b00adc90 0x7fa1b01f05b0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1b00ca6e0 0x7fa1b00c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 --2- 172.21.15.60:0/2304281827 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1b00b08a0 0x7fa1b0150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 >> 172.21.15.60:0/2304281827 conn(0x7fa1b00cb360 msgr2=0x7fa1b00aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:15.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 shutdown_connections 2023-07-13T22:33:15.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.234+0000 7fa1b4bff700 1 -- 172.21.15.60:0/2304281827 wait complete. 2023-07-13T22:33:15.222 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 97 2023-07-13T22:33:15.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:33:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.456+0000 7fe205bff700 1 Processor -- start 2023-07-13T22:33:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.456+0000 7fe205bff700 1 -- start start 2023-07-13T22:33:15.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.457+0000 7fe205bff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe20015c6f0 0x7fe200160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.457+0000 7fe205bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000722e0 con 0x7fe2000cf010 2023-07-13T22:33:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.457+0000 7fe205bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000d0df0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.457+0000 7fe205bff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000cdbd0 con 0x7fe20015c6f0 2023-07-13T22:33:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.457+0000 7fe1ff7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe20015c6f0 0x7fe200160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.458+0000 7fe1ff7fe700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe20015c6f0 0x7fe200160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52968/0 (socket says 172.21.15.60:52968) 2023-07-13T22:33:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.458+0000 7fe1ff7fe700 1 -- 172.21.15.60:0/3979185610 learned_addr learned my addr 172.21.15.60:0/3979185610 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.458+0000 7fe1ff7fe700 1 -- 172.21.15.60:0/3979185610 >> v1:172.21.15.105:6789/0 conn(0x7fe2000cfdd0 legacy=0x7fe2000d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.458+0000 7fe1ff7fe700 1 -- 172.21.15.60:0/3979185610 >> v1:172.21.15.60:6789/0 conn(0x7fe2000cf010 legacy=0x7fe2000cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.458+0000 7fe1ff7fe700 1 -- 172.21.15.60:0/3979185610 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20015cac0 con 0x7fe20015c6f0 2023-07-13T22:33:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.458+0000 7fe1ff7fe700 1 --2- 172.21.15.60:0/3979185610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20015c6f0 0x7fe200160b70 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00098f0 tx=0x7fe1f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672b96b8a0bbace7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.458+0000 7fe1fe7fc700 1 -- 172.21.15.60:0/3979185610 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0004230 con 0x7fe20015c6f0 2023-07-13T22:33:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.459+0000 7fe1fe7fc700 1 -- 172.21.15.60:0/3979185610 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f0004390 con 0x7fe20015c6f0 2023-07-13T22:33:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.459+0000 7fe1fe7fc700 1 -- 172.21.15.60:0/3979185610 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00036a0 con 0x7fe20015c6f0 2023-07-13T22:33:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.459+0000 7fe205bff700 1 -- 172.21.15.60:0/3979185610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20015c6f0 msgr2=0x7fe200160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.459+0000 7fe205bff700 1 --2- 172.21.15.60:0/3979185610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20015c6f0 0x7fe200160b70 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00098f0 tx=0x7fe1f0009c00 comp rx=0 tx=0).stop 2023-07-13T22:33:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.459+0000 7fe205bff700 1 -- 172.21.15.60:0/3979185610 shutdown_connections 2023-07-13T22:33:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.459+0000 7fe205bff700 1 --2- 172.21.15.60:0/3979185610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe20015c6f0 0x7fe200160b70 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.459+0000 7fe205bff700 1 -- 172.21.15.60:0/3979185610 >> 172.21.15.60:0/3979185610 conn(0x7fe20006a4b0 msgr2=0x7fe20006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.459+0000 7fe205bff700 1 -- 172.21.15.60:0/3979185610 shutdown_connections 2023-07-13T22:33:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.459+0000 7fe205bff700 1 -- 172.21.15.60:0/3979185610 wait complete. 2023-07-13T22:33:15.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.462+0000 7fe205bff700 1 Processor -- start 2023-07-13T22:33:15.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.462+0000 7fe205bff700 1 -- start start 2023-07-13T22:33:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe205bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2000cf010 0x7fe2000b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe205bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2000cfdd0 0x7fe2001f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe205bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe20015c6f0 0x7fe2000b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe205bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000b3db0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe205bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2001f6750 con 0x7fe2000cf010 2023-07-13T22:33:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe205bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000b4bb0 con 0x7fe20015c6f0 2023-07-13T22:33:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1feffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2000cfdd0 0x7fe2001f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1feffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2000cfdd0 0x7fe2001f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52974/0 (socket says 172.21.15.60:52974) 2023-07-13T22:33:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1feffd700 1 -- 172.21.15.60:0/3926039657 learned_addr learned my addr 172.21.15.60:0/3926039657 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1fffff700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe20015c6f0 0x7fe2000b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1ff7fe700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2000cf010 0x7fe2000b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1feffd700 1 -- 172.21.15.60:0/3926039657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe20015c6f0 msgr2=0x7fe2000b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1feffd700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe20015c6f0 0x7fe2000b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1feffd700 1 -- 172.21.15.60:0/3926039657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2000cf010 msgr2=0x7fe2000b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1feffd700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2000cf010 0x7fe2000b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1feffd700 1 -- 172.21.15.60:0/3926039657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0009550 con 0x7fe2000cfdd0 2023-07-13T22:33:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.463+0000 7fe1ff7fe700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2000cf010 0x7fe2000b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.464+0000 7fe1feffd700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2000cfdd0 0x7fe2001f6210 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c430 tx=0x7fe1f400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.464+0000 7fe1e7fff700 1 -- 172.21.15.60:0/3926039657 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40052d0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.464+0000 7fe1e7fff700 1 -- 172.21.15.60:0/3926039657 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4005450 con 0x7fe2000cfdd0 2023-07-13T22:33:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.464+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2000b3fa0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.464+0000 7fe1e7fff700 1 -- 172.21.15.60:0/3926039657 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4006aa0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.464+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2000b4560 con 0x7fe2000cfdd0 2023-07-13T22:33:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.465+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2000c67f0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.466+0000 7fe1e7fff700 1 -- 172.21.15.60:0/3926039657 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe1f4003680 con 0x7fe2000cfdd0 2023-07-13T22:33:15.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.467+0000 7fe1e7fff700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1ec06ace0 0x7fe1ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.467+0000 7fe1e7fff700 1 -- 172.21.15.60:0/3926039657 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe1f40884e0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.467+0000 7fe1ff7fe700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1ec06ace0 0x7fe1ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.468+0000 7fe1ff7fe700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1ec06ace0 0x7fe1ec06d190 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009130 tx=0x7fe1f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:15.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.472+0000 7fe1e7fff700 1 -- 172.21.15.60:0/3926039657 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe2000c67f0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.681+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2000722e0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.683+0000 7fe1e7fff700 1 -- 172.21.15.60:0/3926039657 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+18363 (secure 0 0 0) 0x7fe2000722e0 con 0x7fe2000cfdd0 2023-07-13T22:33:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:15.669 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":137,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:46.593649+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:31:27.702135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-13T22:31:28.386570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.7","mappings":[{"from":3,"to":0}]},{"pgid":"22.c","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:33:15.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1ec06ace0 msgr2=0x7fe1ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1ec06ace0 0x7fe1ec06d190 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009130 tx=0x7fe1f0008030 comp rx=0 tx=0).stop 2023-07-13T22:33:15.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2000cfdd0 msgr2=0x7fe2001f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2000cfdd0 0x7fe2001f6210 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c430 tx=0x7fe1f400c740 comp rx=0 tx=0).stop 2023-07-13T22:33:15.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 shutdown_connections 2023-07-13T22:33:15.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe1ec06ace0 0x7fe1ec06d190 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe20015c6f0 0x7fe2000b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2000cf010 0x7fe2000b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 --2- 172.21.15.60:0/3926039657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2000cfdd0 0x7fe2001f6210 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 >> 172.21.15.60:0/3926039657 conn(0x7fe20006a4b0 msgr2=0x7fe20015fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:15.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 shutdown_connections 2023-07-13T22:33:15.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.685+0000 7fe205bff700 1 -- 172.21.15.60:0/3926039657 wait complete. 2023-07-13T22:33:15.689 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:33:15.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:33:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.908+0000 7f3e239ff700 1 Processor -- start 2023-07-13T22:33:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.908+0000 7f3e239ff700 1 -- start start 2023-07-13T22:33:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.909+0000 7f3e239ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e1c0c8f30 0x7f3e1c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.910+0000 7f3e239ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1c0ccde0 con 0x7f3e1c06c620 2023-07-13T22:33:15.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.910+0000 7f3e239ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1c0c9980 con 0x7f3e1c0c93b0 2023-07-13T22:33:15.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.910+0000 7f3e239ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e1c0cc4b0 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.910+0000 7f3e2179b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3e1c06c620 0x7f3e1c06c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52868/0 (socket says 172.21.15.60:52868) 2023-07-13T22:33:15.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.910+0000 7f3e2179b700 1 -- 172.21.15.60:0/3902884560 learned_addr learned my addr 172.21.15.60:0/3902884560 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:15.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.910+0000 7f3e20f9a700 1 --2- 172.21.15.60:0/3902884560 >> v2:172.21.15.105:3300/0 conn(0x7f3e1c0c8f30 0x7f3e1c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.910+0000 7f3e13fff700 1 -- 172.21.15.60:0/3902884560 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489751891 0 0) 0x7f3e1c0ccde0 con 0x7f3e1c06c620 2023-07-13T22:33:15.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.910+0000 7f3e13fff700 1 -- 172.21.15.60:0/3902884560 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e080035b0 con 0x7f3e1c06c620 2023-07-13T22:33:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.911+0000 7f3e13fff700 1 -- 172.21.15.60:0/3902884560 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505683660 0 0) 0x7f3e080035b0 con 0x7f3e1c06c620 2023-07-13T22:33:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.911+0000 7f3e20f9a700 1 -- 172.21.15.60:0/3902884560 >> v1:172.21.15.105:6789/0 conn(0x7f3e1c0c93b0 legacy=0x7f3e1c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.911+0000 7f3e20f9a700 1 -- 172.21.15.60:0/3902884560 >> v1:172.21.15.60:6789/0 conn(0x7f3e1c06c620 legacy=0x7f3e1c06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.911+0000 7f3e20f9a700 1 -- 172.21.15.60:0/3902884560 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c0cc790 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.911+0000 7f3e20f9a700 1 --2- 172.21.15.60:0/3902884560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c8f30 0x7f3e1c0cd270 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009840 tx=0x7f3e0c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96e4b7aea858422 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.911+0000 7f3e13fff700 1 -- 172.21.15.60:0/3902884560 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18001670 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.911+0000 7f3e13fff700 1 -- 172.21.15.60:0/3902884560 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c004030 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.911+0000 7f3e13fff700 1 -- 172.21.15.60:0/3902884560 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0043c0 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.912+0000 7f3e239ff700 1 -- 172.21.15.60:0/3902884560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c8f30 msgr2=0x7f3e1c0cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.912+0000 7f3e239ff700 1 --2- 172.21.15.60:0/3902884560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c8f30 0x7f3e1c0cd270 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009840 tx=0x7f3e0c009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.912+0000 7f3e239ff700 1 -- 172.21.15.60:0/3902884560 shutdown_connections 2023-07-13T22:33:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.912+0000 7f3e239ff700 1 --2- 172.21.15.60:0/3902884560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c8f30 0x7f3e1c0cd270 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.912+0000 7f3e239ff700 1 -- 172.21.15.60:0/3902884560 >> 172.21.15.60:0/3902884560 conn(0x7f3e1c0cb360 msgr2=0x7f3e1c0cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.912+0000 7f3e239ff700 1 -- 172.21.15.60:0/3902884560 shutdown_connections 2023-07-13T22:33:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.912+0000 7f3e239ff700 1 -- 172.21.15.60:0/3902884560 wait complete. 2023-07-13T22:33:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.914+0000 7f3e239ff700 1 Processor -- start 2023-07-13T22:33:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.914+0000 7f3e239ff700 1 -- start start 2023-07-13T22:33:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e239ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e1c06c620 0x7f3e1c1f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e239ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e1c0c8f30 0x7f3e1c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e239ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c93b0 0x7f3e1c1e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e239ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c1f6550 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e239ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c0b4710 con 0x7f3e1c0c93b0 2023-07-13T22:33:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e239ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c14de60 con 0x7f3e1c06c620 2023-07-13T22:33:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e20f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e1c0c8f30 0x7f3e1c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e20f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e1c0c8f30 0x7f3e1c14d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52976/0 (socket says 172.21.15.60:52976) 2023-07-13T22:33:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e20f9a700 1 -- 172.21.15.60:0/3104260222 learned_addr learned my addr 172.21.15.60:0/3104260222 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e2179b700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e1c06c620 0x7f3e1c1f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e21f9c700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c93b0 0x7f3e1c1e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e20f9a700 1 -- 172.21.15.60:0/3104260222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e1c06c620 msgr2=0x7f3e1c1f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e20f9a700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e1c06c620 0x7f3e1c1f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e20f9a700 1 -- 172.21.15.60:0/3104260222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c93b0 msgr2=0x7f3e1c1e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e20f9a700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c93b0 0x7f3e1c1e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e20f9a700 1 -- 172.21.15.60:0/3104260222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c0094a0 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.915+0000 7f3e21f9c700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c93b0 0x7f3e1c1e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.916+0000 7f3e20f9a700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e1c0c8f30 0x7f3e1c14d920 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009790 tx=0x7f3e0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.916+0000 7f3e11ffb700 1 -- 172.21.15.60:0/3104260222 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c011630 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.916+0000 7f3e11ffb700 1 -- 172.21.15.60:0/3104260222 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c01f070 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.916+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1c0b2dc0 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.916+0000 7f3e11ffb700 1 -- 172.21.15.60:0/3104260222 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0119f0 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.916+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1c0b3380 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.917+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1c074220 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.918+0000 7f3e11ffb700 1 -- 172.21.15.60:0/3104260222 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e0c011b50 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.919+0000 7f3e11ffb700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:15.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.919+0000 7f3e11ffb700 1 -- 172.21.15.60:0/3104260222 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3e0c021070 con 0x7f3e1c0c8f30 2023-07-13T22:33:15.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.919+0000 7f3e2179b700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:15.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.920+0000 7f3e2179b700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3e18001eb0 tx=0x7f3e18001e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:15.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:15.924+0000 7f3e11ffb700 1 -- 172.21.15.60:0/3104260222 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3e0c055ea0 con 0x7f3e1c0c8f30 2023-07-13T22:33:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.174+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e1c0b4850 con 0x7f3e1c0c8f30 2023-07-13T22:33:16.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.174+0000 7f3e11ffb700 1 -- 172.21.15.60:0/3104260222 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f3e0c0594c0 con 0x7f3e1c0c8f30 2023-07-13T22:33:16.161 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:16.161 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3652157896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3652157896},{"type":"v1","addr":"172.21.15.105:6834","nonce":3652157896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1878256861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1878256861},{"type":"v1","addr":"172.21.15.105:6835","nonce":1878256861}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92},{"gid":8728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2695331242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2695331242},{"type":"v1","addr":"172.21.15.60:6836","nonce":2695331242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:31:29.408608+0000","modified":"2023-07-13T22:31:32.919029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9423":{"gid":9423,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4061691799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4061691799},{"type":"v1","addr":"172.21.15.60:6837","nonce":4061691799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-13T22:33:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 msgr2=0x7f3e0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3e18001eb0 tx=0x7f3e18001e40 comp rx=0 tx=0).stop 2023-07-13T22:33:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e1c0c8f30 msgr2=0x7f3e1c14d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e1c0c8f30 0x7f3e1c14d920 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009790 tx=0x7f3e0c004750 comp rx=0 tx=0).stop 2023-07-13T22:33:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 shutdown_connections 2023-07-13T22:33:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f3e0006ace0 0x7f3e0006d190 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e1c0c93b0 0x7f3e1c1e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e1c06c620 0x7f3e1c1f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 --2- 172.21.15.60:0/3104260222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e1c0c8f30 0x7f3e1c14d920 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 >> 172.21.15.60:0/3104260222 conn(0x7f3e1c0cb360 msgr2=0x7f3e1c0b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 shutdown_connections 2023-07-13T22:33:16.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.177+0000 7f3e239ff700 1 -- 172.21.15.60:0/3104260222 wait complete. 2023-07-13T22:33:16.165 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 97 2023-07-13T22:33:16.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:33:16.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.398+0000 7f9a1edff700 1 Processor -- start 2023-07-13T22:33:16.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.398+0000 7f9a1edff700 1 -- start start 2023-07-13T22:33:16.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.399+0000 7f9a1edff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a1815c5f0 0x7f9a1806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.399+0000 7f9a1edff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a180ce3a0 0x7f9a18069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.399+0000 7f9a1edff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a180ce810 0x7f9a1806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.399+0000 7f9a1edff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a18160240 con 0x7f9a1815c5f0 2023-07-13T22:33:16.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.399+0000 7f9a1edff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180713a0 con 0x7f9a180ce810 2023-07-13T22:33:16.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.399+0000 7f9a1edff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18071680 con 0x7f9a180ce3a0 2023-07-13T22:33:16.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1cb9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a1815c5f0 0x7f9a1806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1cb9b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a1815c5f0 0x7f9a1806c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52992/0 (socket says 172.21.15.60:52992) 2023-07-13T22:33:16.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1cb9b700 1 -- 172.21.15.60:0/3325361837 learned_addr learned my addr 172.21.15.60:0/3325361837 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a17fff700 1 --2- 172.21.15.60:0/3325361837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a180ce3a0 0x7f9a18069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1d39c700 1 --2- 172.21.15.60:0/3325361837 >> v2:172.21.15.105:3300/0 conn(0x7f9a180ce810 0x7f9a1806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1cb9b700 1 -- 172.21.15.60:0/3325361837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a180ce3a0 msgr2=0x7f9a18069eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:16.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1cb9b700 1 --2- 172.21.15.60:0/3325361837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a180ce3a0 0x7f9a18069eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1cb9b700 1 -- 172.21.15.60:0/3325361837 >> v2:172.21.15.105:3300/0 conn(0x7f9a180ce810 msgr2=0x7f9a1806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:16.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1cb9b700 1 --2- 172.21.15.60:0/3325361837 >> v2:172.21.15.105:3300/0 conn(0x7f9a180ce810 0x7f9a1806a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1cb9b700 1 -- 172.21.15.60:0/3325361837 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a18071960 con 0x7f9a1815c5f0 2023-07-13T22:33:16.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.400+0000 7f9a1cb9b700 1 --2- 172.21.15.60:0/3325361837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1815c5f0 0x7f9a1806c620 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f9a180cd5d0 tx=0x7f9a0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68448b473742a15 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:16.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a177fe700 1 -- 172.21.15.60:0/3325361837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00b870 con 0x7f9a1815c5f0 2023-07-13T22:33:16.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a177fe700 1 -- 172.21.15.60:0/3325361837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00b9d0 con 0x7f9a1815c5f0 2023-07-13T22:33:16.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a177fe700 1 -- 172.21.15.60:0/3325361837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00bd60 con 0x7f9a1815c5f0 2023-07-13T22:33:16.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a1edff700 1 -- 172.21.15.60:0/3325361837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1815c5f0 msgr2=0x7f9a1806c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a1edff700 1 --2- 172.21.15.60:0/3325361837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1815c5f0 0x7f9a1806c620 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f9a180cd5d0 tx=0x7f9a0c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a1edff700 1 -- 172.21.15.60:0/3325361837 shutdown_connections 2023-07-13T22:33:16.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a1edff700 1 --2- 172.21.15.60:0/3325361837 >> v2:172.21.15.105:3300/0 conn(0x7f9a180ce810 0x7f9a1806a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a1edff700 1 --2- 172.21.15.60:0/3325361837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1815c5f0 0x7f9a1806c620 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a1edff700 1 --2- 172.21.15.60:0/3325361837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a180ce3a0 0x7f9a18069eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a1edff700 1 -- 172.21.15.60:0/3325361837 >> 172.21.15.60:0/3325361837 conn(0x7f9a1806bc50 msgr2=0x7f9a180cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:16.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a1edff700 1 -- 172.21.15.60:0/3325361837 shutdown_connections 2023-07-13T22:33:16.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.401+0000 7f9a1edff700 1 -- 172.21.15.60:0/3325361837 wait complete. 2023-07-13T22:33:16.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.403+0000 7f9a1edff700 1 Processor -- start 2023-07-13T22:33:16.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.403+0000 7f9a1edff700 1 -- start start 2023-07-13T22:33:16.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.404+0000 7f9a1edff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a180ce3a0 0x7f9a1815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.404+0000 7f9a1edff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a180ce810 0x7f9a181f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.404+0000 7f9a1edff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a1815c5f0 0x7f9a1815f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.404+0000 7f9a1edff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180b0370 con 0x7f9a180ce810 2023-07-13T22:33:16.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.404+0000 7f9a1edff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1815dbc0 con 0x7f9a180ce3a0 2023-07-13T22:33:16.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.404+0000 7f9a1edff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180b0060 con 0x7f9a1815c5f0 2023-07-13T22:33:16.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.404+0000 7f9a17fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a180ce810 0x7f9a181f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.404+0000 7f9a17fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a180ce810 0x7f9a181f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53004/0 (socket says 172.21.15.60:53004) 2023-07-13T22:33:16.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.404+0000 7f9a17fff700 1 -- 172.21.15.60:0/1562982915 learned_addr learned my addr 172.21.15.60:0/1562982915 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:16.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a1d39c700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a1815c5f0 0x7f9a1815f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a1cb9b700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a180ce3a0 0x7f9a1815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a17fff700 1 -- 172.21.15.60:0/1562982915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a1815c5f0 msgr2=0x7f9a1815f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a17fff700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a1815c5f0 0x7f9a1815f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a17fff700 1 -- 172.21.15.60:0/1562982915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a180ce3a0 msgr2=0x7f9a1815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a17fff700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a180ce3a0 0x7f9a1815d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a17fff700 1 -- 172.21.15.60:0/1562982915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c009550 con 0x7f9a180ce810 2023-07-13T22:33:16.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a17fff700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a180ce810 0x7f9a181f2390 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400d800 tx=0x7f9a0400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a157fa700 1 -- 172.21.15.60:0/1562982915 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040125f0 con 0x7f9a180ce810 2023-07-13T22:33:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a157fa700 1 -- 172.21.15.60:0/1562982915 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04012750 con 0x7f9a180ce810 2023-07-13T22:33:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a180b54b0 con 0x7f9a180ce810 2023-07-13T22:33:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.405+0000 7f9a157fa700 1 -- 172.21.15.60:0/1562982915 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04012ae0 con 0x7f9a180ce810 2023-07-13T22:33:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.406+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a181f2f10 con 0x7f9a180ce810 2023-07-13T22:33:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.407+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a18074220 con 0x7f9a180ce810 2023-07-13T22:33:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.407+0000 7f9a157fa700 1 -- 172.21.15.60:0/1562982915 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a04012c40 con 0x7f9a180ce810 2023-07-13T22:33:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.409+0000 7f9a157fa700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f99fc06ace0 0x7f99fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.409+0000 7f9a157fa700 1 -- 172.21.15.60:0/1562982915 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9a04089470 con 0x7f9a180ce810 2023-07-13T22:33:16.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.409+0000 7f9a1cb9b700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f99fc06ace0 0x7f99fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.410+0000 7f9a1cb9b700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f99fc06ace0 0x7f99fc06d190 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c014040 tx=0x7f9a0c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:16.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.413+0000 7f9a157fa700 1 -- 172.21.15.60:0/1562982915 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f9a04054580 con 0x7f9a180ce810 2023-07-13T22:33:16.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.662+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a180cd200 con 0x7f9a180ce810 2023-07-13T22:33:16.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.662+0000 7f9a157fa700 1 -- 172.21.15.60:0/1562982915 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f9a04057ba0 con 0x7f9a180ce810 2023-07-13T22:33:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:16.649 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3652157896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3652157896},{"type":"v1","addr":"172.21.15.105:6834","nonce":3652157896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1878256861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1878256861},{"type":"v1","addr":"172.21.15.105:6835","nonce":1878256861}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92},{"gid":8728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2695331242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2695331242},{"type":"v1","addr":"172.21.15.60:6836","nonce":2695331242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-13T22:31:29.408608+0000","modified":"2023-07-13T22:31:32.919029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9423":{"gid":9423,"name":"c","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6837/4061691799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":4061691799},{"type":"v1","addr":"172.21.15.60:6837","nonce":4061691799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-13T22:33:16.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f99fc06ace0 msgr2=0x7f99fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f99fc06ace0 0x7f99fc06d190 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c014040 tx=0x7f9a0c008030 comp rx=0 tx=0).stop 2023-07-13T22:33:16.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a180ce810 msgr2=0x7f9a181f2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a180ce810 0x7f9a181f2390 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400d800 tx=0x7f9a0400db10 comp rx=0 tx=0).stop 2023-07-13T22:33:16.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 shutdown_connections 2023-07-13T22:33:16.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f99fc06ace0 0x7f99fc06d190 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a1815c5f0 0x7f9a1815f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a180ce3a0 0x7f9a1815d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 --2- 172.21.15.60:0/1562982915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a180ce810 0x7f9a181f2390 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 >> 172.21.15.60:0/1562982915 conn(0x7f9a1806bc50 msgr2=0x7f9a18069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:16.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 shutdown_connections 2023-07-13T22:33:16.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.665+0000 7f9a1edff700 1 -- 172.21.15.60:0/1562982915 wait complete. 2023-07-13T22:33:16.653 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 97 2023-07-13T22:33:16.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:33:16.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.886+0000 7f83c21ff700 1 Processor -- start 2023-07-13T22:33:16.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.887+0000 7f83c21ff700 1 -- start start 2023-07-13T22:33:16.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83c21ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83bc15cb50 0x7f83bc15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83c21ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 0x7f83bc0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83c21ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83bc0cf310 con 0x7f83bc0715e0 2023-07-13T22:33:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83c21ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83bc15d930 con 0x7f83bc15cb50 2023-07-13T22:33:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83c21ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0d0a60 con 0x7f83bc0d1540 2023-07-13T22:33:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83baffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83bc15cb50 0x7f83bc15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83baffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83bc15cb50 0x7f83bc15cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53014/0 (socket says 172.21.15.60:53014) 2023-07-13T22:33:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83baffd700 1 -- 172.21.15.60:0/751319084 learned_addr learned my addr 172.21.15.60:0/751319084 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:16.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83bbfff700 1 --2- 172.21.15.60:0/751319084 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 0x7f83bc0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83baffd700 1 -- 172.21.15.60:0/751319084 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 msgr2=0x7f83bc0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:16.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.888+0000 7f83baffd700 1 --2- 172.21.15.60:0/751319084 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 0x7f83bc0d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.889+0000 7f83baffd700 1 -- 172.21.15.60:0/751319084 >> v1:172.21.15.105:6789/0 conn(0x7f83bc0715e0 legacy=0x7f83bc0719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.889+0000 7f83baffd700 1 -- 172.21.15.60:0/751319084 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc0d0d40 con 0x7f83bc15cb50 2023-07-13T22:33:16.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.889+0000 7f83baffd700 1 --2- 172.21.15.60:0/751319084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 0x7f83bc15cff0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f83b00098f0 tx=0x7f83b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d11ab10d1a94dc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:16.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.889+0000 7f83ba7fc700 1 -- 172.21.15.60:0/751319084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b000b8a0 con 0x7f83bc15cb50 2023-07-13T22:33:16.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.889+0000 7f83ba7fc700 1 -- 172.21.15.60:0/751319084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b000ba00 con 0x7f83bc15cb50 2023-07-13T22:33:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.889+0000 7f83ba7fc700 1 -- 172.21.15.60:0/751319084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b000bd90 con 0x7f83bc15cb50 2023-07-13T22:33:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.890+0000 7f83c21ff700 1 -- 172.21.15.60:0/751319084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 msgr2=0x7f83bc15cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.890+0000 7f83c21ff700 1 --2- 172.21.15.60:0/751319084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 0x7f83bc15cff0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f83b00098f0 tx=0x7f83b0009c00 comp rx=0 tx=0).stop 2023-07-13T22:33:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.890+0000 7f83c21ff700 1 -- 172.21.15.60:0/751319084 shutdown_connections 2023-07-13T22:33:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.890+0000 7f83c21ff700 1 --2- 172.21.15.60:0/751319084 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 0x7f83bc0d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.890+0000 7f83c21ff700 1 --2- 172.21.15.60:0/751319084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 0x7f83bc15cff0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.890+0000 7f83c21ff700 1 -- 172.21.15.60:0/751319084 >> 172.21.15.60:0/751319084 conn(0x7f83bc0d2bc0 msgr2=0x7f83bc0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.890+0000 7f83c21ff700 1 -- 172.21.15.60:0/751319084 shutdown_connections 2023-07-13T22:33:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.890+0000 7f83c21ff700 1 -- 172.21.15.60:0/751319084 wait complete. 2023-07-13T22:33:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.891+0000 7f83c21ff700 1 Processor -- start 2023-07-13T22:33:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.892+0000 7f83c21ff700 1 -- start start 2023-07-13T22:33:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.892+0000 7f83c21ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83bc0715e0 0x7f83bc0b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.892+0000 7f83c21ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 0x7f83bc157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.892+0000 7f83c21ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 0x7f83bc158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.892+0000 7f83c21ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1e4f30 con 0x7f83bc15cb50 2023-07-13T22:33:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.892+0000 7f83c21ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1e5210 con 0x7f83bc0715e0 2023-07-13T22:33:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.892+0000 7f83c21ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1e54f0 con 0x7f83bc0d1540 2023-07-13T22:33:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 0x7f83bc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 0x7f83bc158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53030/0 (socket says 172.21.15.60:53030) 2023-07-13T22:33:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bbfff700 1 -- 172.21.15.60:0/2154925329 learned_addr learned my addr 172.21.15.60:0/2154925329 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bb7fe700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83bc0715e0 0x7f83bc0b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83baffd700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 0x7f83bc157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bbfff700 1 -- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 msgr2=0x7f83bc157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bbfff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 0x7f83bc157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bbfff700 1 -- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83bc0715e0 msgr2=0x7f83bc0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:16.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bbfff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83bc0715e0 0x7f83bc0b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:16.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bbfff700 1 -- 172.21.15.60:0/2154925329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b0009550 con 0x7f83bc15cb50 2023-07-13T22:33:16.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83bb7fe700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83bc0715e0 0x7f83bc0b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:16.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.893+0000 7f83baffd700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 0x7f83bc157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:16.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.894+0000 7f83bbfff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 0x7f83bc158110 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f83b4007e40 tx=0x7f83b400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:16.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.894+0000 7f83a3fff700 1 -- 172.21.15.60:0/2154925329 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400eea0 con 0x7f83bc15cb50 2023-07-13T22:33:16.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.894+0000 7f83a3fff700 1 -- 172.21.15.60:0/2154925329 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b4014420 con 0x7f83bc15cb50 2023-07-13T22:33:16.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.894+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc1e57d0 con 0x7f83bc15cb50 2023-07-13T22:33:16.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.894+0000 7f83a3fff700 1 -- 172.21.15.60:0/2154925329 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b40147b0 con 0x7f83bc15cb50 2023-07-13T22:33:16.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.894+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc1e5d90 con 0x7f83bc15cb50 2023-07-13T22:33:16.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.895+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc04ea60 con 0x7f83bc15cb50 2023-07-13T22:33:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.896+0000 7f83a3fff700 1 -- 172.21.15.60:0/2154925329 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83b4014910 con 0x7f83bc15cb50 2023-07-13T22:33:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.897+0000 7f83a3fff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83a806ace0 0x7f83a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.897+0000 7f83a3fff700 1 -- 172.21.15.60:0/2154925329 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f83b4089ce0 con 0x7f83bc15cb50 2023-07-13T22:33:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.897+0000 7f83bb7fe700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83a806ace0 0x7f83a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.898+0000 7f83bb7fe700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83a806ace0 0x7f83a806d190 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f83bc158a20 tx=0x7f83ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:16.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:16.901+0000 7f83a3fff700 1 -- 172.21.15.60:0/2154925329 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f83b4054d40 con 0x7f83bc15cb50 2023-07-13T22:33:17.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.110+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83bc06a2c0 con 0x7f83bc15cb50 2023-07-13T22:33:17.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.111+0000 7f83a3fff700 1 -- 172.21.15.60:0/2154925329 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+18363 (secure 0 0 0) 0x7f83b4058360 con 0x7f83bc15cb50 2023-07-13T22:33:17.098 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:17.098 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":137,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:31:46.593649+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:31:27.702135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-13T22:31:28.386570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.7","mappings":[{"from":3,"to":0}]},{"pgid":"22.c","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:33:17.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.113+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83a806ace0 msgr2=0x7f83a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.113+0000 7f83c21ff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83a806ace0 0x7f83a806d190 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f83bc158a20 tx=0x7f83ac009040 comp rx=0 tx=0).stop 2023-07-13T22:33:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.113+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 msgr2=0x7f83bc158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.113+0000 7f83c21ff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 0x7f83bc158110 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f83b4007e40 tx=0x7f83b400e3c0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.113+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 shutdown_connections 2023-07-13T22:33:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.114+0000 7f83c21ff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f83a806ace0 0x7f83a806d190 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.114+0000 7f83c21ff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83bc15cb50 0x7f83bc158110 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.114+0000 7f83c21ff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83bc0715e0 0x7f83bc0b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.114+0000 7f83c21ff700 1 --2- 172.21.15.60:0/2154925329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83bc0d1540 0x7f83bc157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.114+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 >> 172.21.15.60:0/2154925329 conn(0x7f83bc0d2bc0 msgr2=0x7f83bc0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:17.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.114+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 shutdown_connections 2023-07-13T22:33:17.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.114+0000 7f83c21ff700 1 -- 172.21.15.60:0/2154925329 wait complete. 2023-07-13T22:33:17.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:33:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.336+0000 7f3865fff700 1 Processor -- start 2023-07-13T22:33:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.336+0000 7f3865fff700 1 -- start start 2023-07-13T22:33:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f3865fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f386015c5f0 0x7f386006c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f3865fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f3860069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f3865fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38600ce810 0x7f386006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f3865fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3860160240 con 0x7f38600ce810 2023-07-13T22:33:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f3865fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38600713a0 con 0x7f386015c5f0 2023-07-13T22:33:17.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f3865fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3860071680 con 0x7f38600ce3a0 2023-07-13T22:33:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38600ce810 0x7f386006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385ffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38600ce810 0x7f386006a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53034/0 (socket says 172.21.15.60:53034) 2023-07-13T22:33:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385ffff700 1 -- 172.21.15.60:0/212814609 learned_addr learned my addr 172.21.15.60:0/212814609 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385effd700 1 --2- 172.21.15.60:0/212814609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f3860069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385f7fe700 1 --2- 172.21.15.60:0/212814609 >> v2:172.21.15.105:3300/0 conn(0x7f386015c5f0 0x7f386006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385ffff700 1 -- 172.21.15.60:0/212814609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 msgr2=0x7f3860069eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385ffff700 1 --2- 172.21.15.60:0/212814609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f3860069eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385ffff700 1 -- 172.21.15.60:0/212814609 >> v2:172.21.15.105:3300/0 conn(0x7f386015c5f0 msgr2=0x7f386006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385ffff700 1 --2- 172.21.15.60:0/212814609 >> v2:172.21.15.105:3300/0 conn(0x7f386015c5f0 0x7f386006c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385ffff700 1 -- 172.21.15.60:0/212814609 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860071960 con 0x7f38600ce810 2023-07-13T22:33:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385effd700 1 --2- 172.21.15.60:0/212814609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f3860069eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.338+0000 7f385f7fe700 1 --2- 172.21.15.60:0/212814609 >> v2:172.21.15.105:3300/0 conn(0x7f386015c5f0 0x7f386006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.339+0000 7f385ffff700 1 --2- 172.21.15.60:0/212814609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 0x7f386006a3f0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f38600ccc20 tx=0x7f385800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6626321092aa9b4b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.339+0000 7f385e7fc700 1 -- 172.21.15.60:0/212814609 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800e070 con 0x7f38600ce810 2023-07-13T22:33:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.339+0000 7f385e7fc700 1 -- 172.21.15.60:0/212814609 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385800b9c0 con 0x7f38600ce810 2023-07-13T22:33:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.339+0000 7f385e7fc700 1 -- 172.21.15.60:0/212814609 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800bd70 con 0x7f38600ce810 2023-07-13T22:33:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.340+0000 7f3865fff700 1 -- 172.21.15.60:0/212814609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 msgr2=0x7f386006a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.340+0000 7f3865fff700 1 --2- 172.21.15.60:0/212814609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 0x7f386006a3f0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f38600ccc20 tx=0x7f385800b330 comp rx=0 tx=0).stop 2023-07-13T22:33:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.340+0000 7f3865fff700 1 -- 172.21.15.60:0/212814609 shutdown_connections 2023-07-13T22:33:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.340+0000 7f3865fff700 1 --2- 172.21.15.60:0/212814609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 0x7f386006a3f0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.340+0000 7f3865fff700 1 --2- 172.21.15.60:0/212814609 >> v2:172.21.15.105:3300/0 conn(0x7f386015c5f0 0x7f386006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.340+0000 7f3865fff700 1 --2- 172.21.15.60:0/212814609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f3860069eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.340+0000 7f3865fff700 1 -- 172.21.15.60:0/212814609 >> 172.21.15.60:0/212814609 conn(0x7f386006bc50 msgr2=0x7f38600cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.340+0000 7f3865fff700 1 -- 172.21.15.60:0/212814609 shutdown_connections 2023-07-13T22:33:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.340+0000 7f3865fff700 1 -- 172.21.15.60:0/212814609 wait complete. 2023-07-13T22:33:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.342+0000 7f3865fff700 1 Processor -- start 2023-07-13T22:33:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.342+0000 7f3865fff700 1 -- start start 2023-07-13T22:33:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f3865fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f38601f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f3865fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 0x7f3860141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f3865fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386015c5f0 0x7f386015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f3865fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38601f2a40 con 0x7f38600ce810 2023-07-13T22:33:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f3865fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38601f2470 con 0x7f386015c5f0 2023-07-13T22:33:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f3865fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38601424e0 con 0x7f38600ce3a0 2023-07-13T22:33:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f385effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 0x7f3860141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f385effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 0x7f3860141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53038/0 (socket says 172.21.15.60:53038) 2023-07-13T22:33:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f385effd700 1 -- 172.21.15.60:0/3552034119 learned_addr learned my addr 172.21.15.60:0/3552034119 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f385f7fe700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f38601f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.343+0000 7f385effd700 1 -- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 msgr2=0x7f38601f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f385ffff700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386015c5f0 0x7f386015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f385effd700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f38601f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f385effd700 1 -- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386015c5f0 msgr2=0x7f386015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f385effd700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386015c5f0 0x7f386015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f385effd700 1 -- 172.21.15.60:0/3552034119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3858009b30 con 0x7f38600ce810 2023-07-13T22:33:17.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f385f7fe700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f38601f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:17.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f385ffff700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386015c5f0 0x7f386015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:17.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f385effd700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 0x7f3860141a90 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f385400d800 tx=0x7f385400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:17.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f3847fff700 1 -- 172.21.15.60:0/3552034119 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854004030 con 0x7f38600ce810 2023-07-13T22:33:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f3847fff700 1 -- 172.21.15.60:0/3552034119 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854004190 con 0x7f38600ce810 2023-07-13T22:33:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f3847fff700 1 -- 172.21.15.60:0/3552034119 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854004520 con 0x7f38600ce810 2023-07-13T22:33:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38601427c0 con 0x7f38600ce810 2023-07-13T22:33:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.344+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38601f33a0 con 0x7f38600ce810 2023-07-13T22:33:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.345+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860074220 con 0x7f38600ce810 2023-07-13T22:33:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.346+0000 7f3847fff700 1 -- 172.21.15.60:0/3552034119 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38540036a0 con 0x7f38600ce810 2023-07-13T22:33:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.347+0000 7f3847fff700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f384c06ace0 0x7f384c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.347+0000 7f3847fff700 1 -- 172.21.15.60:0/3552034119 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3854088d70 con 0x7f38600ce810 2023-07-13T22:33:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.347+0000 7f385f7fe700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f384c06ace0 0x7f384c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.348+0000 7f385f7fe700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f384c06ace0 0x7f384c06d190 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3850009960 tx=0x7f3850008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:17.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.351+0000 7f3847fff700 1 -- 172.21.15.60:0/3552034119 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f3854057910 con 0x7f38600ce810 2023-07-13T22:33:17.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.607+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f38601429b0 con 0x7f38600ce810 2023-07-13T22:33:17.594 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-07-13T22:33:17.608+0000 7f5c7f3f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:33:17.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.612+0000 7f3847fff700 1 -- 172.21.15.60:0/3552034119 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3854019090 con 0x7f38600ce810 2023-07-13T22:33:17.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f384c06ace0 msgr2=0x7f384c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f384c06ace0 0x7f384c06d190 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3850009960 tx=0x7f3850008040 comp rx=0 tx=0).stop 2023-07-13T22:33:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 msgr2=0x7f3860141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 0x7f3860141a90 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f385400d800 tx=0x7f385400dbc0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 shutdown_connections 2023-07-13T22:33:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f384c06ace0 0x7f384c06d190 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386015c5f0 0x7f386015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38600ce3a0 0x7f38601f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 --2- 172.21.15.60:0/3552034119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600ce810 0x7f3860141a90 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 >> 172.21.15.60:0/3552034119 conn(0x7f386006bc50 msgr2=0x7f38600ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 shutdown_connections 2023-07-13T22:33:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.615+0000 7f3865fff700 1 -- 172.21.15.60:0/3552034119 wait complete. 2023-07-13T22:33:17.603 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:33:17.610 INFO:tasks.ceph.mds.c.smithi060.stderr: -6532> 2023-07-13T22:31:35.880+0000 7f2bfc1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:33:17.621 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:33:17.664 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-07-13T22:33:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.842+0000 7f06fd7ff700 1 Processor -- start 2023-07-13T22:33:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.842+0000 7f06fd7ff700 1 -- start start 2023-07-13T22:33:17.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.843+0000 7f06fd7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06f80b4e50 0x7f06f80ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.843+0000 7f06fd7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f8161140 con 0x7f06f815c400 2023-07-13T22:33:17.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.843+0000 7f06fd7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f80d0c60 con 0x7f06f815c7d0 2023-07-13T22:33:17.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.844+0000 7f06fd7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f80cd3b0 con 0x7f06f80b4e50 2023-07-13T22:33:17.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.844+0000 7f06f6ffd700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f06f815c400 0x7f06f80b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52884/0 (socket says 172.21.15.60:52884) 2023-07-13T22:33:17.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.844+0000 7f06f6ffd700 1 -- 172.21.15.60:0/1400055262 learned_addr learned my addr 172.21.15.60:0/1400055262 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.844+0000 7f06f67fc700 1 --2- 172.21.15.60:0/1400055262 >> v2:172.21.15.105:3300/0 conn(0x7f06f80b4e50 0x7f06f80ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.844+0000 7f06f5ffb700 1 -- 172.21.15.60:0/1400055262 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258490027 0 0) 0x7f06f8161140 con 0x7f06f815c400 2023-07-13T22:33:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.844+0000 7f06f5ffb700 1 -- 172.21.15.60:0/1400055262 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06e00035b0 con 0x7f06f815c400 2023-07-13T22:33:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.844+0000 7f06f67fc700 1 -- 172.21.15.60:0/1400055262 >> v1:172.21.15.105:6789/0 conn(0x7f06f815c7d0 legacy=0x7f06f80d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.844+0000 7f06f67fc700 1 -- 172.21.15.60:0/1400055262 >> v1:172.21.15.60:6789/0 conn(0x7f06f815c400 legacy=0x7f06f80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.844+0000 7f06f67fc700 1 -- 172.21.15.60:0/1400055262 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f80cd690 con 0x7f06f80b4e50 2023-07-13T22:33:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.845+0000 7f06f67fc700 1 --2- 172.21.15.60:0/1400055262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 0x7f06f80ceef0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009840 tx=0x7f06ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=865b59ed956edc83 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.845+0000 7f06f5ffb700 1 -- 172.21.15.60:0/1400055262 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec011830 con 0x7f06f80b4e50 2023-07-13T22:33:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.845+0000 7f06f5ffb700 1 -- 172.21.15.60:0/1400055262 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ec011990 con 0x7f06f80b4e50 2023-07-13T22:33:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.845+0000 7f06f5ffb700 1 -- 172.21.15.60:0/1400055262 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec011d20 con 0x7f06f80b4e50 2023-07-13T22:33:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.845+0000 7f06fd7ff700 1 -- 172.21.15.60:0/1400055262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 msgr2=0x7f06f80ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.845+0000 7f06fd7ff700 1 --2- 172.21.15.60:0/1400055262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 0x7f06f80ceef0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009840 tx=0x7f06ec009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.846+0000 7f06fd7ff700 1 -- 172.21.15.60:0/1400055262 shutdown_connections 2023-07-13T22:33:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.846+0000 7f06fd7ff700 1 --2- 172.21.15.60:0/1400055262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 0x7f06f80ceef0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.846+0000 7f06fd7ff700 1 -- 172.21.15.60:0/1400055262 >> 172.21.15.60:0/1400055262 conn(0x7f06f806b140 msgr2=0x7f06f80d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.846+0000 7f06fd7ff700 1 -- 172.21.15.60:0/1400055262 shutdown_connections 2023-07-13T22:33:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.846+0000 7f06fd7ff700 1 -- 172.21.15.60:0/1400055262 wait complete. 2023-07-13T22:33:17.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.847+0000 7f06fd7ff700 1 Processor -- start 2023-07-13T22:33:17.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.847+0000 7f06fd7ff700 1 -- start start 2023-07-13T22:33:17.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.848+0000 7f06fd7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 0x7f06f81f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.848+0000 7f06fd7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06f815c400 0x7f06f80715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.848+0000 7f06fd7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06f815c7d0 0x7f06f8071b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.848+0000 7f06fd7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f814c6e0 con 0x7f06f815c400 2023-07-13T22:33:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.848+0000 7f06fd7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f814a660 con 0x7f06f80b4e50 2023-07-13T22:33:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.848+0000 7f06fd7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f80ca950 con 0x7f06f815c7d0 2023-07-13T22:33:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06f815c400 0x7f06f80715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f67fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06f815c400 0x7f06f80715e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53078/0 (socket says 172.21.15.60:53078) 2023-07-13T22:33:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f67fc700 1 -- 172.21.15.60:0/2148088872 learned_addr learned my addr 172.21.15.60:0/2148088872 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f67fc700 1 -- 172.21.15.60:0/2148088872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06f815c7d0 msgr2=0x7f06f8071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f67fc700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06f815c7d0 0x7f06f8071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f6ffd700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 0x7f06f81f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f67fc700 1 -- 172.21.15.60:0/2148088872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 msgr2=0x7f06f81f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f67fc700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 0x7f06f81f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f67fc700 1 -- 172.21.15.60:0/2148088872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec0094a0 con 0x7f06f815c400 2023-07-13T22:33:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f6ffd700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 0x7f06f81f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.849+0000 7f06f67fc700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06f815c400 0x7f06f80715e0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009fd0 tx=0x7f06ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.850+0000 7f06df7fe700 1 -- 172.21.15.60:0/2148088872 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0038a0 con 0x7f06f815c400 2023-07-13T22:33:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.850+0000 7f06df7fe700 1 -- 172.21.15.60:0/2148088872 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ec01f070 con 0x7f06f815c400 2023-07-13T22:33:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.850+0000 7f06df7fe700 1 -- 172.21.15.60:0/2148088872 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec003c40 con 0x7f06f815c400 2023-07-13T22:33:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.850+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f8149d30 con 0x7f06f815c400 2023-07-13T22:33:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.850+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f814a2f0 con 0x7f06f815c400 2023-07-13T22:33:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.851+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f814a480 con 0x7f06f815c400 2023-07-13T22:33:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.852+0000 7f06df7fe700 1 -- 172.21.15.60:0/2148088872 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f06ec026480 con 0x7f06f815c400 2023-07-13T22:33:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.853+0000 7f06df7fe700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e406ace0 0x7f06e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.853+0000 7f06df7fe700 1 -- 172.21.15.60:0/2148088872 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f06ec021070 con 0x7f06f815c400 2023-07-13T22:33:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.853+0000 7f06f6ffd700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e406ace0 0x7f06e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.853+0000 7f06f6ffd700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e406ace0 0x7f06e406d190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f06e80011e0 tx=0x7f06e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:17.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:17.857+0000 7f06df7fe700 1 -- 172.21.15.60:0/2148088872 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f06ec055f00 con 0x7f06f815c400 2023-07-13T22:33:18.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.111+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f06f8074220 con 0x7f06f815c400 2023-07-13T22:33:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.613+0000 7f06df7fe700 1 -- 172.21.15.60:0/2148088872 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v99) v1 ==== 104+0+0 (secure 0 0 0) 0x7f06ec059520 con 0x7f06f815c400 2023-07-13T22:33:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.615+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e406ace0 msgr2=0x7f06e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:18.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.615+0000 7f06fd7ff700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e406ace0 0x7f06e406d190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f06e80011e0 tx=0x7f06e8009040 comp rx=0 tx=0).stop 2023-07-13T22:33:18.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.615+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06f815c400 msgr2=0x7f06f80715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:18.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.615+0000 7f06fd7ff700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06f815c400 0x7f06f80715e0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009fd0 tx=0x7f06ec004750 comp rx=0 tx=0).stop 2023-07-13T22:33:18.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.616+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 shutdown_connections 2023-07-13T22:33:18.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.616+0000 7f06fd7ff700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f06e406ace0 0x7f06e406d190 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:18.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.616+0000 7f06fd7ff700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06f815c7d0 0x7f06f8071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.616+0000 7f06fd7ff700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06f80b4e50 0x7f06f81f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.616+0000 7f06fd7ff700 1 --2- 172.21.15.60:0/2148088872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06f815c400 0x7f06f80715e0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.616+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 >> 172.21.15.60:0/2148088872 conn(0x7f06f806b140 msgr2=0x7f06f80d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.616+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 shutdown_connections 2023-07-13T22:33:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.616+0000 7f06fd7ff700 1 -- 172.21.15.60:0/2148088872 wait complete. 2023-07-13T22:33:18.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:33:18.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.889+0000 7f8dcabff700 1 Processor -- start 2023-07-13T22:33:18.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.889+0000 7f8dcabff700 1 -- start start 2023-07-13T22:33:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.890+0000 7f8dcabff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 0x7f8dc40cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.890+0000 7f8dcabff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8dc4072140 0x7f8dc40d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.890+0000 7f8dcabff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc40725e0 con 0x7f8dc415fb90 2023-07-13T22:33:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.890+0000 7f8dcabff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc40d0130 con 0x7f8dc4072140 2023-07-13T22:33:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.890+0000 7f8dcabff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40cd420 con 0x7f8dc40b4e70 2023-07-13T22:33:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.890+0000 7f8dc899b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8dc415fb90 0x7f8dc40b4760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52890/0 (socket says 172.21.15.60:52890) 2023-07-13T22:33:18.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.890+0000 7f8dc899b700 1 -- 172.21.15.60:0/1381512621 learned_addr learned my addr 172.21.15.60:0/1381512621 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:18.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.891+0000 7f8dc919c700 1 --2- 172.21.15.60:0/1381512621 >> v2:172.21.15.60:3300/0 conn(0x7f8dc4072140 0x7f8dc40d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:18.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.891+0000 7f8dc3fff700 1 --2- 172.21.15.60:0/1381512621 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 0x7f8dc40cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:18.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.891+0000 7f8dc919c700 1 -- 172.21.15.60:0/1381512621 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 msgr2=0x7f8dc40cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:18.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.891+0000 7f8dc919c700 1 --2- 172.21.15.60:0/1381512621 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 0x7f8dc40cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:18.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.891+0000 7f8dc37fe700 1 -- 172.21.15.60:0/1381512621 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088408435 0 0) 0x7f8dc40725e0 con 0x7f8dc415fb90 2023-07-13T22:33:18.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.891+0000 7f8dc919c700 1 -- 172.21.15.60:0/1381512621 >> v1:172.21.15.60:6789/0 conn(0x7f8dc415fb90 legacy=0x7f8dc40b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:18.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.891+0000 7f8dc919c700 1 -- 172.21.15.60:0/1381512621 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc40cd700 con 0x7f8dc4072140 2023-07-13T22:33:18.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.891+0000 7f8dc919c700 1 --2- 172.21.15.60:0/1381512621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 0x7f8dc40d11a0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009d90 tx=0x7f8dbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a1c90c38a6ecd5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:18.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.891+0000 7f8dc37fe700 1 -- 172.21.15.60:0/1381512621 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00e070 con 0x7f8dc4072140 2023-07-13T22:33:18.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dc37fe700 1 -- 172.21.15.60:0/1381512621 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc00bbd0 con 0x7f8dc4072140 2023-07-13T22:33:18.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dc37fe700 1 -- 172.21.15.60:0/1381512621 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc012430 con 0x7f8dc4072140 2023-07-13T22:33:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dcabff700 1 -- 172.21.15.60:0/1381512621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 msgr2=0x7f8dc40d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dcabff700 1 --2- 172.21.15.60:0/1381512621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 0x7f8dc40d11a0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009d90 tx=0x7f8dbc00b330 comp rx=0 tx=0).stop 2023-07-13T22:33:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dcabff700 1 -- 172.21.15.60:0/1381512621 shutdown_connections 2023-07-13T22:33:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dcabff700 1 --2- 172.21.15.60:0/1381512621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 0x7f8dc40d11a0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dcabff700 1 --2- 172.21.15.60:0/1381512621 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 0x7f8dc40cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dcabff700 1 -- 172.21.15.60:0/1381512621 >> 172.21.15.60:0/1381512621 conn(0x7f8dc406a4b0 msgr2=0x7f8dc406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:18.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dcabff700 1 -- 172.21.15.60:0/1381512621 shutdown_connections 2023-07-13T22:33:18.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.892+0000 7f8dcabff700 1 -- 172.21.15.60:0/1381512621 wait complete. 2023-07-13T22:33:18.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.894+0000 7f8dcabff700 1 Processor -- start 2023-07-13T22:33:18.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.894+0000 7f8dcabff700 1 -- start start 2023-07-13T22:33:18.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.895+0000 7f8dcabff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 0x7f8dc41f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:18.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.895+0000 7f8dcabff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 0x7f8dc40b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:18.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.895+0000 7f8dcabff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc415fb90 0x7f8dc40b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:18.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.895+0000 7f8dcabff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41f2ba0 con 0x7f8dc4072140 2023-07-13T22:33:18.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.895+0000 7f8dcabff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41f2300 con 0x7f8dc415fb90 2023-07-13T22:33:18.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.895+0000 7f8dcabff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41f25e0 con 0x7f8dc40b4e70 2023-07-13T22:33:18.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 0x7f8dc41f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:18.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc899b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 0x7f8dc41f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53094/0 (socket says 172.21.15.60:53094) 2023-07-13T22:33:18.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc899b700 1 -- 172.21.15.60:0/1178338456 learned_addr learned my addr 172.21.15.60:0/1178338456 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:18.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc919c700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc415fb90 0x7f8dc40b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:18.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc3fff700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 0x7f8dc40b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:18.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc899b700 1 -- 172.21.15.60:0/1178338456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 msgr2=0x7f8dc40b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc899b700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 0x7f8dc40b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc899b700 1 -- 172.21.15.60:0/1178338456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc415fb90 msgr2=0x7f8dc40b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc899b700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc415fb90 0x7f8dc40b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc899b700 1 -- 172.21.15.60:0/1178338456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc009a70 con 0x7f8dc4072140 2023-07-13T22:33:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.896+0000 7f8dc899b700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 0x7f8dc41f1c00 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8db80011e0 tx=0x7f8db800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.897+0000 7f8dc17fa700 1 -- 172.21.15.60:0/1178338456 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8004c80 con 0x7f8dc4072140 2023-07-13T22:33:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.897+0000 7f8dc17fa700 1 -- 172.21.15.60:0/1178338456 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db8004e00 con 0x7f8dc4072140 2023-07-13T22:33:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.897+0000 7f8dc17fa700 1 -- 172.21.15.60:0/1178338456 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8007310 con 0x7f8dc4072140 2023-07-13T22:33:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.897+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc41f28c0 con 0x7f8dc4072140 2023-07-13T22:33:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.897+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc414bf70 con 0x7f8dc4072140 2023-07-13T22:33:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.898+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc41f3260 con 0x7f8dc4072140 2023-07-13T22:33:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.899+0000 7f8dc17fa700 1 -- 172.21.15.60:0/1178338456 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8db8007470 con 0x7f8dc4072140 2023-07-13T22:33:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.901+0000 7f8dc17fa700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8da806ace0 0x7f8da806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.901+0000 7f8dc17fa700 1 -- 172.21.15.60:0/1178338456 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f8db8089110 con 0x7f8dc4072140 2023-07-13T22:33:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.901+0000 7f8dc3fff700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8da806ace0 0x7f8da806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.902+0000 7f8dc3fff700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8da806ace0 0x7f8da806d190 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40ca050 tx=0x7f8db0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:18.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:18.905+0000 7f8dc17fa700 1 -- 172.21.15.60:0/1178338456 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8db80541c0 con 0x7f8dc4072140 2023-07-13T22:33:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.114+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8dc4074220 con 0x7f8dc4072140 2023-07-13T22:33:19.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.825+0000 7f8dc17fa700 1 -- 172.21.15.60:0/1178338456 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v139) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8db80577e0 con 0x7f8dc4072140 2023-07-13T22:33:19.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8da806ace0 msgr2=0x7f8da806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:19.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8da806ace0 0x7f8da806d190 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40ca050 tx=0x7f8db0008040 comp rx=0 tx=0).stop 2023-07-13T22:33:19.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 msgr2=0x7f8dc41f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:19.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 0x7f8dc41f1c00 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8db80011e0 tx=0x7f8db800ca30 comp rx=0 tx=0).stop 2023-07-13T22:33:19.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 shutdown_connections 2023-07-13T22:33:19.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8da806ace0 0x7f8da806d190 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:19.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8dc4072140 0x7f8dc41f1c00 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:19.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc415fb90 0x7f8dc40b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:19.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 --2- 172.21.15.60:0/1178338456 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dc40b4e70 0x7f8dc40b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:19.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 >> 172.21.15.60:0/1178338456 conn(0x7f8dc406a4b0 msgr2=0x7f8dc40b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:19.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.829+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 shutdown_connections 2023-07-13T22:33:19.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:19.830+0000 7f8dcabff700 1 -- 172.21.15.60:0/1178338456 wait complete. 2023-07-13T22:33:19.818 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:33:19.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:33:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.066+0000 7f930cfff700 1 Processor -- start 2023-07-13T22:33:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.066+0000 7f930cfff700 1 -- start start 2023-07-13T22:33:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.067+0000 7f930cfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f930806c8b0 0x7f93080714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.067+0000 7f930cfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9308071a10 0x7f93080b0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.067+0000 7f930cfff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9308070770 con 0x7f930815a360 2023-07-13T22:33:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f930cfff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f930806cd20 con 0x7f9308071a10 2023-07-13T22:33:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f930cfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930815e950 con 0x7f930806c8b0 2023-07-13T22:33:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f9306d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9308071a10 0x7f93080b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f9306d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9308071a10 0x7f93080b0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53104/0 (socket says 172.21.15.60:53104) 2023-07-13T22:33:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f9306d9d700 1 -- 172.21.15.60:0/3171132420 learned_addr learned my addr 172.21.15.60:0/3171132420 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f92fffff700 1 --2- 172.21.15.60:0/3171132420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f930806c8b0 0x7f93080714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f9306d9d700 1 -- 172.21.15.60:0/3171132420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f930806c8b0 msgr2=0x7f93080714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f9306d9d700 1 --2- 172.21.15.60:0/3171132420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f930806c8b0 0x7f93080714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f9306d9d700 1 -- 172.21.15.60:0/3171132420 >> v1:172.21.15.105:6789/0 conn(0x7f930815a360 legacy=0x7f93080c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f9306d9d700 1 -- 172.21.15.60:0/3171132420 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930815ec30 con 0x7f9308071a10 2023-07-13T22:33:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.068+0000 7f92fffff700 1 --2- 172.21.15.60:0/3171132420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f930806c8b0 0x7f93080714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f9306d9d700 1 --2- 172.21.15.60:0/3171132420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9308071a10 0x7f93080b0340 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f9300009e50 tx=0x7f930000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c056eea2155d6bf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f9305d9b700 1 -- 172.21.15.60:0/3171132420 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930000e070 con 0x7f9308071a10 2023-07-13T22:33:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f9305d9b700 1 -- 172.21.15.60:0/3171132420 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9300003a20 con 0x7f9308071a10 2023-07-13T22:33:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f9305d9b700 1 -- 172.21.15.60:0/3171132420 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300003dd0 con 0x7f9308071a10 2023-07-13T22:33:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f930cfff700 1 -- 172.21.15.60:0/3171132420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9308071a10 msgr2=0x7f93080b0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f930cfff700 1 --2- 172.21.15.60:0/3171132420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9308071a10 0x7f93080b0340 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f9300009e50 tx=0x7f930000b330 comp rx=0 tx=0).stop 2023-07-13T22:33:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f930cfff700 1 -- 172.21.15.60:0/3171132420 shutdown_connections 2023-07-13T22:33:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f930cfff700 1 --2- 172.21.15.60:0/3171132420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9308071a10 0x7f93080b0340 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f930cfff700 1 --2- 172.21.15.60:0/3171132420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f930806c8b0 0x7f93080714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.069+0000 7f930cfff700 1 -- 172.21.15.60:0/3171132420 >> 172.21.15.60:0/3171132420 conn(0x7f93080cb340 msgr2=0x7f93080cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.070+0000 7f930cfff700 1 -- 172.21.15.60:0/3171132420 shutdown_connections 2023-07-13T22:33:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.070+0000 7f930cfff700 1 -- 172.21.15.60:0/3171132420 wait complete. 2023-07-13T22:33:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.072+0000 7f930cfff700 1 Processor -- start 2023-07-13T22:33:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.072+0000 7f930cfff700 1 -- start start 2023-07-13T22:33:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.073+0000 7f930cfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930806c8b0 0x7f930815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.073+0000 7f930cfff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9308071a10 0x7f93080a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.073+0000 7f930cfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f930815a360 0x7f930814cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.073+0000 7f930cfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93080a9500 con 0x7f930806c8b0 2023-07-13T22:33:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.073+0000 7f930cfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930815c8d0 con 0x7f930815a360 2023-07-13T22:33:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.073+0000 7f930cfff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930815cbb0 con 0x7f9308071a10 2023-07-13T22:33:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f930659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930806c8b0 0x7f930815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f930659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930806c8b0 0x7f930815c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53112/0 (socket says 172.21.15.60:53112) 2023-07-13T22:33:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f930659c700 1 -- 172.21.15.60:0/586832145 learned_addr learned my addr 172.21.15.60:0/586832145 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f92fffff700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9308071a10 0x7f93080a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f9306d9d700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f930815a360 0x7f930814cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:20.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f930659c700 1 -- 172.21.15.60:0/586832145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9308071a10 msgr2=0x7f93080a8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:20.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f930659c700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9308071a10 0x7f93080a8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f930659c700 1 -- 172.21.15.60:0/586832145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f930815a360 msgr2=0x7f930814cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f930659c700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f930815a360 0x7f930814cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f930659c700 1 -- 172.21.15.60:0/586832145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9300009b30 con 0x7f930806c8b0 2023-07-13T22:33:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f930659c700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930806c8b0 0x7f930815c260 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f92f800b4a0 tx=0x7f92f800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f92feffd700 1 -- 172.21.15.60:0/586832145 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8011830 con 0x7f930806c8b0 2023-07-13T22:33:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f92feffd700 1 -- 172.21.15.60:0/586832145 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f8011990 con 0x7f930806c8b0 2023-07-13T22:33:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.074+0000 7f92feffd700 1 -- 172.21.15.60:0/586832145 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8011d40 con 0x7f930806c8b0 2023-07-13T22:33:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.075+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930815ce90 con 0x7f930806c8b0 2023-07-13T22:33:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.075+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930815d450 con 0x7f930806c8b0 2023-07-13T22:33:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.076+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930815e650 con 0x7f930806c8b0 2023-07-13T22:33:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.077+0000 7f92feffd700 1 -- 172.21.15.60:0/586832145 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f92f801a400 con 0x7f930806c8b0 2023-07-13T22:33:20.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.077+0000 7f92feffd700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f92e806ad10 0x7f92e806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:20.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.078+0000 7f92feffd700 1 -- 172.21.15.60:0/586832145 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f92f8089180 con 0x7f930806c8b0 2023-07-13T22:33:20.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.078+0000 7f92fffff700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f92e806ad10 0x7f92e806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:20.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.078+0000 7f92fffff700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f92e806ad10 0x7f92e806d1c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f93081f3860 tx=0x7f92f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:20.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.082+0000 7f92feffd700 1 -- 172.21.15.60:0/586832145 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f92f8054360 con 0x7f930806c8b0 2023-07-13T22:33:20.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.294+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9308074220 con 0x7f930806c8b0 2023-07-13T22:33:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.828+0000 7f92feffd700 1 -- 172.21.15.60:0/586832145 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v140) v1 ==== 169+0+0 (secure 0 0 0) 0x7f92f8057980 con 0x7f930806c8b0 2023-07-13T22:33:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.832+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f92e806ad10 msgr2=0x7f92e806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.832+0000 7f930cfff700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f92e806ad10 0x7f92e806d1c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f93081f3860 tx=0x7f92f0008040 comp rx=0 tx=0).stop 2023-07-13T22:33:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930806c8b0 msgr2=0x7f930815c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930806c8b0 0x7f930815c260 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f92f800b4a0 tx=0x7f92f800b860 comp rx=0 tx=0).stop 2023-07-13T22:33:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 shutdown_connections 2023-07-13T22:33:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f92e806ad10 0x7f92e806d1c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930806c8b0 0x7f930815c260 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f930815a360 0x7f930814cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 --2- 172.21.15.60:0/586832145 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9308071a10 0x7f93080a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 >> 172.21.15.60:0/586832145 conn(0x7f93080cb340 msgr2=0x7f93080aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 shutdown_connections 2023-07-13T22:33:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:20.833+0000 7f930cfff700 1 -- 172.21.15.60:0/586832145 wait complete. 2023-07-13T22:33:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-07-13T22:33:20.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:33:20.846 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-13T22:33:20.968 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2023-07-13T22:33:20.968 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2023-07-13T22:33:20.968 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:33:20.968 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2023-07-13T22:33:20.968 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.969 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2023-07-13T22:33:20.969 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.969 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2023-07-13T22:33:20.969 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.969 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2023-07-13T22:33:20.969 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2023-07-13T22:33:20.969 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.969 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2023-07-13T22:33:20.970 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2023-07-13T22:33:20.970 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.970 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:33:20.970 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2023-07-13T22:33:20.970 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.971 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:33:20.971 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:33:20.971 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.971 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2023-07-13T22:33:20.971 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.971 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2023-07-13T22:33:20.971 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2023-07-13T22:33:20.971 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:33:20.972 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:33:20.972 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2023-07-13T22:33:20.972 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:33:20.972 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2023-07-13T22:33:20.972 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:33:20.973 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:33:20.973 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2023-07-13T22:33:20.973 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.973 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:33:20.974 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.974 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2023-07-13T22:33:20.974 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.974 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2023-07-13T22:33:20.974 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.974 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2023-07-13T22:33:20.974 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2023-07-13T22:33:20.975 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2023-07-13T22:33:20.975 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:33:20.975 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:33:20.975 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.975 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2023-07-13T22:33:20.975 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2023-07-13T22:33:20.975 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.975 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2023-07-13T22:33:20.976 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2023-07-13T22:33:20.976 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:33:20.976 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.976 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2023-07-13T22:33:20.976 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2023-07-13T22:33:20.976 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.977 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2023-07-13T22:33:20.977 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2023-07-13T22:33:20.977 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2023-07-13T22:33:20.977 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.977 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2023-07-13T22:33:20.978 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2023-07-13T22:33:20.978 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.978 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:33:20.978 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.978 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2023-07-13T22:33:20.978 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.978 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2023-07-13T22:33:20.978 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.979 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:33:20.979 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2023-07-13T22:33:20.979 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.979 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2023-07-13T22:33:20.979 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2023-07-13T22:33:20.979 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2023-07-13T22:33:20.979 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2023-07-13T22:33:20.979 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2023-07-13T22:33:20.980 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2023-07-13T22:33:20.980 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2023-07-13T22:33:20.980 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2023-07-13T22:33:20.980 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2023-07-13T22:33:20.980 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:33:20.980 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2023-07-13T22:33:20.980 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2023-07-13T22:33:20.980 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:33:20.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2023-07-13T22:33:20.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:33:20.981 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:33:20.981 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2023-07-13T22:33:20.981 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:33:20.981 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.982 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2023-07-13T22:33:20.982 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:33:20.982 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2023-07-13T22:33:20.982 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:33:20.982 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:33:20.982 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:33:20.982 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:33:20.982 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2023-07-13T22:33:20.983 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.983 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2023-07-13T22:33:20.984 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:33:20.984 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-07-13T22:33:20.984 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2023-07-13T22:33:20.984 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-07-13T22:33:20.984 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2023-07-13T22:33:20.984 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2023-07-13T22:33:20.985 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:33:20.985 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2023-07-13T22:33:20.985 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2023-07-13T22:33:20.985 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:33:20.985 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.986 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:33:20.986 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:33:20.986 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:33:20.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.986 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:33:20.986 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:33:20.986 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:33:20.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.987 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2023-07-13T22:33:20.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:33:20.987 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:33:20.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2023-07-13T22:33:20.987 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:20.987 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2023-07-13T22:33:20.988 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2023-07-13T22:33:20.988 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2023-07-13T22:33:20.988 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2023-07-13T22:33:20.988 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:33:20.988 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:33:20.988 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2023-07-13T22:33:20.988 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2023-07-13T22:33:20.988 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:33:20.989 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2023-07-13T22:33:20.989 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:33:20.989 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:33:20.989 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2023-07-13T22:33:20.989 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2023-07-13T22:33:20.989 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2023-07-13T22:33:20.989 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-07-13T22:33:20.990 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-07-13T22:33:20.990 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-07-13T22:33:20.990 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-07-13T22:33:20.990 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2023-07-13T22:33:20.990 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2023-07-13T22:33:20.990 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2023-07-13T22:33:20.990 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-07-13T22:33:20.990 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2023-07-13T22:33:20.991 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2023-07-13T22:33:20.991 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2023-07-13T22:33:20.991 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2023-07-13T22:33:20.991 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-07-13T22:33:20.991 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-07-13T22:33:20.991 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-07-13T22:33:20.991 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-07-13T22:33:20.992 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-07-13T22:33:20.992 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-07-13T22:33:20.992 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-13T22:33:20.992 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-07-13T22:33:20.993 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.993 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-07-13T22:33:20.993 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.993 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-07-13T22:33:20.993 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.993 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-07-13T22:33:20.993 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-07-13T22:33:20.994 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.994 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-07-13T22:33:20.994 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-07-13T22:33:20.994 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.994 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-13T22:33:20.994 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-07-13T22:33:20.994 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.994 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-07-13T22:33:20.995 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-13T22:33:20.995 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.995 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-07-13T22:33:20.995 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.995 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-07-13T22:33:20.995 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-07-13T22:33:20.995 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-07-13T22:33:20.995 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-07-13T22:33:20.996 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-07-13T22:33:20.996 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-07-13T22:33:20.996 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-07-13T22:33:20.996 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-13T22:33:20.996 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-13T22:33:20.996 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-07-13T22:33:20.996 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.997 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-13T22:33:20.997 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.997 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-07-13T22:33:20.997 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.997 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-07-13T22:33:20.997 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.997 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-07-13T22:33:20.997 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-07-13T22:33:20.998 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-07-13T22:33:20.998 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.60,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2023-07-13T22:33:20.998 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce 2023-07-13T22:33:20.998 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.998 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-07-13T22:33:20.998 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-07-13T22:33:20.998 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.999 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-07-13T22:33:20.999 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2023-07-13T22:33:20.999 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2023-07-13T22:33:20.999 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.999 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-07-13T22:33:20.999 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-07-13T22:33:20.999 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:20.999 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-07-13T22:33:21.000 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-07-13T22:33:21.000 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-07-13T22:33:21.000 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.000 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-07-13T22:33:21.000 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-07-13T22:33:21.000 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.000 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-07-13T22:33:21.001 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.001 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-07-13T22:33:21.001 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.001 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-07-13T22:33:21.001 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.001 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-07-13T22:33:21.001 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-07-13T22:33:21.001 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-07-13T22:33:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-07-13T22:33:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2023-07-13T22:33:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2023-07-13T22:33:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2023-07-13T22:33:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2023-07-13T22:33:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2023-07-13T22:33:21.002 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2023-07-13T22:33:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2023-07-13T22:33:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:33:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-07-13T22:33:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2023-07-13T22:33:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2023-07-13T22:33:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2023-07-13T22:33:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2023-07-13T22:33:21.003 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2023-07-13T22:33:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2023-07-13T22:33:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:33:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.004 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-07-13T22:33:21.004 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:33:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-07-13T22:33:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:33:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:33:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:33:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:33:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-07-13T22:33:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.005 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-07-13T22:33:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:33:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-07-13T22:33:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-07-13T22:33:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-07-13T22:33:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-07-13T22:33:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-07-13T22:33:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-07-13T22:33:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-07-13T22:33:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-07-13T22:33:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-07-13T22:33:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-13T22:33:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-07-13T22:33:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-07-13T22:33:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-07-13T22:33:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-13T22:33:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-13T22:33:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-07-13T22:33:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-07-13T22:33:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-07-13T22:33:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2023-07-13T22:33:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-07-13T22:33:21.009 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-07-13T22:33:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-07-13T22:33:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-07-13T22:33:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-07-13T22:33:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-13T22:33:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-13T22:33:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2023-07-13T22:33:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2023-07-13T22:33:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:33:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2023-07-13T22:33:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:33:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:33:21.010 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-07-13T22:33:21.010 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2023-07-13T22:33:21.011 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2023-07-13T22:33:21.011 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2023-07-13T22:33:21.011 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2023-07-13T22:33:21.011 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2023-07-13T22:33:21.011 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2023-07-13T22:33:21.011 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2023-07-13T22:33:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2023-07-13T22:33:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2023-07-13T22:33:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-07-13T22:33:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2023-07-13T22:33:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2023-07-13T22:33:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2023-07-13T22:33:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2023-07-13T22:33:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2023-07-13T22:33:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2023-07-13T22:33:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2023-07-13T22:33:21.013 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-07-13T22:33:21.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-13T22:33:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.240+0000 7f88965ff700 1 Processor -- start 2023-07-13T22:33:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.240+0000 7f88965ff700 1 -- start start 2023-07-13T22:33:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.241+0000 7f88965ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f889006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.241+0000 7f88965ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889006bf10 con 0x7f88900ca660 2023-07-13T22:33:21.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.241+0000 7f88965ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889006b950 con 0x7f88900cb9c0 2023-07-13T22:33:21.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.241+0000 7f88965ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88900cabd0 con 0x7f8890158990 2023-07-13T22:33:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.241+0000 7f888f7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f88900ca660 0x7f88900cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52900/0 (socket says 172.21.15.60:52900) 2023-07-13T22:33:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.241+0000 7f888f7fe700 1 -- 172.21.15.60:0/1947511776 learned_addr learned my addr 172.21.15.60:0/1947511776 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.242+0000 7f888ffff700 1 --2- 172.21.15.60:0/1947511776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f889006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.242+0000 7f888effd700 1 -- 172.21.15.60:0/1947511776 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912533416 0 0) 0x7f889006bf10 con 0x7f88900ca660 2023-07-13T22:33:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.242+0000 7f888effd700 1 -- 172.21.15.60:0/1947511776 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88780035b0 con 0x7f88900ca660 2023-07-13T22:33:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.242+0000 7f888effd700 1 -- 172.21.15.60:0/1947511776 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840335452 0 0) 0x7f889006b950 con 0x7f88900cb9c0 2023-07-13T22:33:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.242+0000 7f888effd700 1 -- 172.21.15.60:0/1947511776 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f889006bf10 con 0x7f88900cb9c0 2023-07-13T22:33:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.242+0000 7f888ffff700 1 -- 172.21.15.60:0/1947511776 >> v1:172.21.15.105:6789/0 conn(0x7f88900cb9c0 legacy=0x7f889015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.242+0000 7f888ffff700 1 -- 172.21.15.60:0/1947511776 >> v1:172.21.15.60:6789/0 conn(0x7f88900ca660 legacy=0x7f88900cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.242+0000 7f888ffff700 1 -- 172.21.15.60:0/1947511776 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88900caeb0 con 0x7f8890158990 2023-07-13T22:33:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.242+0000 7f888ffff700 1 --2- 172.21.15.60:0/1947511776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f889006c3a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8880009870 tx=0x7f8880009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd890aa9ea526e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f888effd700 1 -- 172.21.15.60:0/1947511776 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880011830 con 0x7f8890158990 2023-07-13T22:33:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f888effd700 1 -- 172.21.15.60:0/1947511776 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8880011990 con 0x7f8890158990 2023-07-13T22:33:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f888effd700 1 -- 172.21.15.60:0/1947511776 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880011d20 con 0x7f8890158990 2023-07-13T22:33:21.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f88965ff700 1 -- 172.21.15.60:0/1947511776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 msgr2=0x7f889006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f88965ff700 1 --2- 172.21.15.60:0/1947511776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f889006c3a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8880009870 tx=0x7f8880009b80 comp rx=0 tx=0).stop 2023-07-13T22:33:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f88965ff700 1 -- 172.21.15.60:0/1947511776 shutdown_connections 2023-07-13T22:33:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f88965ff700 1 --2- 172.21.15.60:0/1947511776 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f889006c3a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f88965ff700 1 -- 172.21.15.60:0/1947511776 >> 172.21.15.60:0/1947511776 conn(0x7f889006ab20 msgr2=0x7f88900b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f88965ff700 1 -- 172.21.15.60:0/1947511776 shutdown_connections 2023-07-13T22:33:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.243+0000 7f88965ff700 1 -- 172.21.15.60:0/1947511776 wait complete. 2023-07-13T22:33:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.246+0000 7f88965ff700 1 Processor -- start 2023-07-13T22:33:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.246+0000 7f88965ff700 1 -- start start 2023-07-13T22:33:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.247+0000 7f88965ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88900ca660 0x7f8890070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.247+0000 7f88965ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88900cb9c0 0x7f8890151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.247+0000 7f88965ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f8890152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.247+0000 7f88965ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88901526a0 con 0x7f88900ca660 2023-07-13T22:33:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.247+0000 7f88965ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890153eb0 con 0x7f88900cb9c0 2023-07-13T22:33:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.247+0000 7f88965ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8890154190 con 0x7f8890158990 2023-07-13T22:33:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88900ca660 0x7f8890070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88900ca660 0x7f8890070ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53114/0 (socket says 172.21.15.60:53114) 2023-07-13T22:33:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888ffff700 1 -- 172.21.15.60:0/3685694868 learned_addr learned my addr 172.21.15.60:0/3685694868 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f8894b9c700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f8890152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888f7fe700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88900cb9c0 0x7f8890151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888ffff700 1 -- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 msgr2=0x7f8890152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888ffff700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f8890152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888ffff700 1 -- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88900cb9c0 msgr2=0x7f8890151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888ffff700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88900cb9c0 0x7f8890151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888ffff700 1 -- 172.21.15.60:0/3685694868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88800094a0 con 0x7f88900ca660 2023-07-13T22:33:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f8894b9c700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f8890152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888f7fe700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88900cb9c0 0x7f8890151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.248+0000 7f888ffff700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88900ca660 0x7f8890070ab0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8880009bc0 tx=0x7f8880004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.249+0000 7f888cff9700 1 -- 172.21.15.60:0/3685694868 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888001a6e0 con 0x7f88900ca660 2023-07-13T22:33:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.249+0000 7f888cff9700 1 -- 172.21.15.60:0/3685694868 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888001a860 con 0x7f88900ca660 2023-07-13T22:33:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.249+0000 7f888cff9700 1 -- 172.21.15.60:0/3685694868 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888001ac10 con 0x7f88900ca660 2023-07-13T22:33:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.249+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8890154470 con 0x7f88900ca660 2023-07-13T22:33:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.249+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8890153ad0 con 0x7f88900ca660 2023-07-13T22:33:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.250+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8890153c60 con 0x7f88900ca660 2023-07-13T22:33:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.251+0000 7f888cff9700 1 -- 172.21.15.60:0/3685694868 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8880022070 con 0x7f88900ca660 2023-07-13T22:33:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.252+0000 7f888cff9700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f887c06ace0 0x7f887c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.252+0000 7f888cff9700 1 -- 172.21.15.60:0/3685694868 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8880089dc0 con 0x7f88900ca660 2023-07-13T22:33:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.252+0000 7f888f7fe700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f887c06ace0 0x7f887c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.252+0000 7f888f7fe700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f887c06ace0 0x7f887c06d190 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8884007fd0 tx=0x7f8884009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:21.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.256+0000 7f888cff9700 1 -- 172.21.15.60:0/3685694868 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f8880055180 con 0x7f88900ca660 2023-07-13T22:33:21.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.464+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f88900741c0 con 0x7f88900ca660 2023-07-13T22:33:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.468+0000 7f888cff9700 1 -- 172.21.15.60:0/3685694868 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7f88800587a0 con 0x7f88900ca660 2023-07-13T22:33:21.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f887c06ace0 msgr2=0x7f887c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f887c06ace0 0x7f887c06d190 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8884007fd0 tx=0x7f8884009040 comp rx=0 tx=0).stop 2023-07-13T22:33:21.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88900ca660 msgr2=0x7f8890070ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88900ca660 0x7f8890070ab0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8880009bc0 tx=0x7f8880004750 comp rx=0 tx=0).stop 2023-07-13T22:33:21.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 shutdown_connections 2023-07-13T22:33:21.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f887c06ace0 0x7f887c06d190 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88900ca660 0x7f8890070ab0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8890158990 0x7f8890152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 --2- 172.21.15.60:0/3685694868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88900cb9c0 0x7f8890151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 >> 172.21.15.60:0/3685694868 conn(0x7f889006ab20 msgr2=0x7f88901ddbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:21.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 shutdown_connections 2023-07-13T22:33:21.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.470+0000 7f88965ff700 1 -- 172.21.15.60:0/3685694868 wait complete. 2023-07-13T22:33:21.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2023-07-13T22:33:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.692+0000 7f05535ff700 1 Processor -- start 2023-07-13T22:33:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.692+0000 7f05535ff700 1 -- start start 2023-07-13T22:33:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.693+0000 7f05535ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f054c0d07d0 0x7f054c0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.693+0000 7f05535ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054c08d030 con 0x7f054c0b2230 2023-07-13T22:33:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.693+0000 7f05535ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054c06b1c0 con 0x7f054c0b1580 2023-07-13T22:33:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.693+0000 7f05535ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c0af7f0 con 0x7f054c0d07d0 2023-07-13T22:33:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.693+0000 7f055139b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f054c0d07d0 0x7f054c0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.693+0000 7f0550b9a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f054c0b2230 0x7f054c06a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52914/0 (socket says 172.21.15.60:52914) 2023-07-13T22:33:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.693+0000 7f0550b9a700 1 -- 172.21.15.60:0/3275115165 learned_addr learned my addr 172.21.15.60:0/3275115165 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.693+0000 7f055139b700 1 --2- 172.21.15.60:0/3275115165 >> v2:172.21.15.60:3300/0 conn(0x7f054c0d07d0 0x7f054c0d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53126/0 (socket says 172.21.15.60:53126) 2023-07-13T22:33:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.694+0000 7f055139b700 1 -- 172.21.15.60:0/3275115165 >> v1:172.21.15.105:6789/0 conn(0x7f054c0b1580 legacy=0x7f054c06aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.694+0000 7f055139b700 1 -- 172.21.15.60:0/3275115165 >> v1:172.21.15.60:6789/0 conn(0x7f054c0b2230 legacy=0x7f054c06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.694+0000 7f055139b700 1 -- 172.21.15.60:0/3275115165 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c0d10e0 con 0x7f054c0d07d0 2023-07-13T22:33:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.694+0000 7f055139b700 1 --2- 172.21.15.60:0/3275115165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 0x7f054c0d0ba0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f0548009840 tx=0x7f0548009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41000cf4d0a62572 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.694+0000 7f0543fff700 1 -- 172.21.15.60:0/3275115165 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548011830 con 0x7f054c0d07d0 2023-07-13T22:33:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.694+0000 7f0543fff700 1 -- 172.21.15.60:0/3275115165 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0548011990 con 0x7f054c0d07d0 2023-07-13T22:33:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.694+0000 7f0543fff700 1 -- 172.21.15.60:0/3275115165 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548011d20 con 0x7f054c0d07d0 2023-07-13T22:33:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.695+0000 7f05535ff700 1 -- 172.21.15.60:0/3275115165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 msgr2=0x7f054c0d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.695+0000 7f05535ff700 1 --2- 172.21.15.60:0/3275115165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 0x7f054c0d0ba0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f0548009840 tx=0x7f0548009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.695+0000 7f05535ff700 1 -- 172.21.15.60:0/3275115165 shutdown_connections 2023-07-13T22:33:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.695+0000 7f05535ff700 1 --2- 172.21.15.60:0/3275115165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 0x7f054c0d0ba0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.695+0000 7f05535ff700 1 -- 172.21.15.60:0/3275115165 >> 172.21.15.60:0/3275115165 conn(0x7f054c09bf90 msgr2=0x7f054c0d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.695+0000 7f05535ff700 1 -- 172.21.15.60:0/3275115165 shutdown_connections 2023-07-13T22:33:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.695+0000 7f05535ff700 1 -- 172.21.15.60:0/3275115165 wait complete. 2023-07-13T22:33:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.697+0000 7f05535ff700 1 Processor -- start 2023-07-13T22:33:21.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.698+0000 7f05535ff700 1 -- start start 2023-07-13T22:33:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f05535ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f054c0b1580 0x7f054c0cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f05535ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c0b2230 0x7f054c15eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f05535ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 0x7f054c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f05535ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1595a0 con 0x7f054c0d07d0 2023-07-13T22:33:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f05535ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c15e280 con 0x7f054c0b2230 2023-07-13T22:33:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f05535ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0cf0d0 con 0x7f054c0b1580 2023-07-13T22:33:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f0551b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 0x7f054c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f0551b9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 0x7f054c1580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53140/0 (socket says 172.21.15.60:53140) 2023-07-13T22:33:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f0551b9c700 1 -- 172.21.15.60:0/3150674944 learned_addr learned my addr 172.21.15.60:0/3150674944 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:21.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.699+0000 7f0550b9a700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c0b2230 0x7f054c15eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f055139b700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f054c0b1580 0x7f054c0cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0551b9c700 1 -- 172.21.15.60:0/3150674944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f054c0b1580 msgr2=0x7f054c0cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0551b9c700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f054c0b1580 0x7f054c0cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0551b9c700 1 -- 172.21.15.60:0/3150674944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c0b2230 msgr2=0x7f054c15eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:21.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0551b9c700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c0b2230 0x7f054c15eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:21.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0551b9c700 1 -- 172.21.15.60:0/3150674944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480094a0 con 0x7f054c0d07d0 2023-07-13T22:33:21.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0550b9a700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c0b2230 0x7f054c15eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0551b9c700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 0x7f054c1580a0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f054400cc50 tx=0x7f054400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0541ffb700 1 -- 172.21.15.60:0/3150674944 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054400ade0 con 0x7f054c0d07d0 2023-07-13T22:33:21.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0541ffb700 1 -- 172.21.15.60:0/3150674944 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054400ce50 con 0x7f054c0d07d0 2023-07-13T22:33:21.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f0541ffb700 1 -- 172.21.15.60:0/3150674944 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544003c20 con 0x7f054c0d07d0 2023-07-13T22:33:21.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c0d03f0 con 0x7f054c0d07d0 2023-07-13T22:33:21.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.700+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c0cea80 con 0x7f054c0d07d0 2023-07-13T22:33:21.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.701+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c0cf240 con 0x7f054c0d07d0 2023-07-13T22:33:21.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.702+0000 7f0541ffb700 1 -- 172.21.15.60:0/3150674944 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f05440050d0 con 0x7f054c0d07d0 2023-07-13T22:33:21.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.704+0000 7f0541ffb700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f053006ace0 0x7f053006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:21.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.704+0000 7f0541ffb700 1 -- 172.21.15.60:0/3150674944 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f05440892d0 con 0x7f054c0d07d0 2023-07-13T22:33:21.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.704+0000 7f055139b700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f053006ace0 0x7f053006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:21.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.705+0000 7f055139b700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f053006ace0 0x7f053006d190 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f05480090b0 tx=0x7f054801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:21.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.708+0000 7f0541ffb700 1 -- 172.21.15.60:0/3150674944 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0544054690 con 0x7f054c0d07d0 2023-07-13T22:33:21.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:21.994+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) v1 -- 0x7f054c0741c0 con 0x7f054c0d07d0 2023-07-13T22:33:22.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.843+0000 7f0541ffb700 1 -- 172.21.15.60:0/3150674944 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v623) v1 ==== 151+0+0 (secure 0 0 0) 0x7f0544057cb0 con 0x7f054c0d07d0 2023-07-13T22:33:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f053006ace0 msgr2=0x7f053006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f053006ace0 0x7f053006d190 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f05480090b0 tx=0x7f054801c040 comp rx=0 tx=0).stop 2023-07-13T22:33:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 msgr2=0x7f054c1580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 0x7f054c1580a0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f054400cc50 tx=0x7f054400a340 comp rx=0 tx=0).stop 2023-07-13T22:33:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 shutdown_connections 2023-07-13T22:33:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f053006ace0 0x7f053006d190 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f054c0d07d0 0x7f054c1580a0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f054c0b1580 0x7f054c0cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 --2- 172.21.15.60:0/3150674944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c0b2230 0x7f054c15eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 >> 172.21.15.60:0/3150674944 conn(0x7f054c09bf90 msgr2=0x7f054c0b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 shutdown_connections 2023-07-13T22:33:22.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:22.846+0000 7f05535ff700 1 -- 172.21.15.60:0/3150674944 wait complete. 2023-07-13T22:33:22.855 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) ... ERROR 2023-07-13T22:33:22.855 INFO:tasks.cephfs_test_runner: 2023-07-13T22:33:22.856 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-13T22:33:22.856 INFO:tasks.cephfs_test_runner:ERROR: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:33:22.857 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-13T22:33:22.857 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-13T22:33:22.858 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs/test_client_limits.py", line 208, in test_client_release_bug 2023-07-13T22:33:22.858 INFO:tasks.cephfs_test_runner: self.wait_for_health("MDS_CLIENT_LATE_RELEASE", session_timeout + 10) 2023-07-13T22:33:22.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/ceph_test_case.py", line 184, in wait_for_health 2023-07-13T22:33:22.859 INFO:tasks.cephfs_test_runner: self.wait_until_true(seen_health_warning, timeout) 2023-07-13T22:33:22.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/ceph_test_case.py", line 231, in wait_until_true 2023-07-13T22:33:22.860 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-07-13T22:33:22.860 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 70s and 0 retries 2023-07-13T22:33:22.861 INFO:tasks.cephfs_test_runner: 2023-07-13T22:33:22.861 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-13T22:33:22.861 INFO:tasks.cephfs_test_runner:Ran 9 tests in 706.926s 2023-07-13T22:33:22.862 INFO:tasks.cephfs_test_runner: 2023-07-13T22:33:22.862 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-07-13T22:33:22.862 INFO:tasks.cephfs_test_runner: 2023-07-13T22:33:22.863 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-13T22:33:22.863 INFO:tasks.cephfs_test_runner:ERROR: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:33:22.865 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-13T22:33:22.866 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-13T22:33:22.866 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs/test_client_limits.py", line 208, in test_client_release_bug 2023-07-13T22:33:22.866 INFO:tasks.cephfs_test_runner: self.wait_for_health("MDS_CLIENT_LATE_RELEASE", session_timeout + 10) 2023-07-13T22:33:22.866 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/ceph_test_case.py", line 184, in wait_for_health 2023-07-13T22:33:22.866 INFO:tasks.cephfs_test_runner: self.wait_until_true(seen_health_warning, timeout) 2023-07-13T22:33:22.867 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/ceph_test_case.py", line 231, in wait_until_true 2023-07-13T22:33:22.867 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-07-13T22:33:22.867 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 70s and 0 retries 2023-07-13T22:33:22.867 INFO:tasks.cephfs_test_runner: 2023-07-13T22:33:22.868 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:33:22.990 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=33cde7b3508c41d5911278b2b8da0289 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-13T22:33:22.992 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-07-13T22:33:23.001 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-07-13T22:33:23.022 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-07-13T22:33:23.023 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:33:23.023 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:33:23.186 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:33:23.186 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-07-13T22:33:23.187 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:33:23.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348668k,nr_inodes=4087167,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-13T22:33:23.277 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:33:23.277 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:33:23.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:33:23.329 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-07-13T22:33:23.329 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:33:23.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345568k,nr_inodes=4086392,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-13T22:33:23.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:23.380 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2023-07-13T22:33:23.466 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-13T22:33:23.466 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-13T22:33:23.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:23.467 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-13T22:33:23.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:23.539 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-13T22:33:23.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:23.613 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2023-07-13T22:33:23.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:23.721 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-07-13T22:33:23.774 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-13T22:33:23.775 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:33:23.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:23.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-13T22:33:23.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:23.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:33:23.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:33:23.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2023-07-13T22:33:24.047 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-07-13T22:33:24.060 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-13T22:33:24.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:33:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.279+0000 7fc3661ff700 1 Processor -- start 2023-07-13T22:33:24.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.279+0000 7fc3661ff700 1 -- start start 2023-07-13T22:33:24.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.280+0000 7fc3661ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 0x7fc3600cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.280+0000 7fc3661ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3600cb620 con 0x7fc3600ca010 2023-07-13T22:33:24.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.280+0000 7fc3661ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3600cb900 con 0x7fc36015e640 2023-07-13T22:33:24.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.280+0000 7fc3661ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36015b760 con 0x7fc360154950 2023-07-13T22:33:24.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.280+0000 7fc35ffff700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc3600ca010 0x7fc36006b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52916/0 (socket says 172.21.15.60:52916) 2023-07-13T22:33:24.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.280+0000 7fc35ffff700 1 -- 172.21.15.60:0/4201459892 learned_addr learned my addr 172.21.15.60:0/4201459892 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.280+0000 7fc35f7fe700 1 --2- 172.21.15.60:0/4201459892 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 0x7fc3600cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.281+0000 7fc35e7fc700 1 -- 172.21.15.60:0/4201459892 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919993535 0 0) 0x7fc3600cb620 con 0x7fc3600ca010 2023-07-13T22:33:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.281+0000 7fc35e7fc700 1 -- 172.21.15.60:0/4201459892 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3480035b0 con 0x7fc3600ca010 2023-07-13T22:33:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.281+0000 7fc35e7fc700 1 -- 172.21.15.60:0/4201459892 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462348081 0 0) 0x7fc3600cb900 con 0x7fc36015e640 2023-07-13T22:33:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.281+0000 7fc35e7fc700 1 -- 172.21.15.60:0/4201459892 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3600cb620 con 0x7fc36015e640 2023-07-13T22:33:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.281+0000 7fc35f7fe700 1 -- 172.21.15.60:0/4201459892 >> v1:172.21.15.105:6789/0 conn(0x7fc36015e640 legacy=0x7fc36015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.281+0000 7fc35f7fe700 1 -- 172.21.15.60:0/4201459892 >> v1:172.21.15.60:6789/0 conn(0x7fc3600ca010 legacy=0x7fc36006b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.281+0000 7fc35f7fe700 1 -- 172.21.15.60:0/4201459892 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3600cbbe0 con 0x7fc360154950 2023-07-13T22:33:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.281+0000 7fc35f7fe700 1 --2- 172.21.15.60:0/4201459892 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 0x7fc3600cafc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc350009870 tx=0x7fc350009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d544c817d84b142 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc35e7fc700 1 -- 172.21.15.60:0/4201459892 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc350011830 con 0x7fc360154950 2023-07-13T22:33:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc35e7fc700 1 -- 172.21.15.60:0/4201459892 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc350011990 con 0x7fc360154950 2023-07-13T22:33:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc35e7fc700 1 -- 172.21.15.60:0/4201459892 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc350011d00 con 0x7fc360154950 2023-07-13T22:33:24.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc3661ff700 1 -- 172.21.15.60:0/4201459892 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 msgr2=0x7fc3600cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc3661ff700 1 --2- 172.21.15.60:0/4201459892 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 0x7fc3600cafc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc350009870 tx=0x7fc350009b80 comp rx=0 tx=0).stop 2023-07-13T22:33:24.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc3661ff700 1 -- 172.21.15.60:0/4201459892 shutdown_connections 2023-07-13T22:33:24.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc3661ff700 1 --2- 172.21.15.60:0/4201459892 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 0x7fc3600cafc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc3661ff700 1 -- 172.21.15.60:0/4201459892 >> 172.21.15.60:0/4201459892 conn(0x7fc36006ab20 msgr2=0x7fc3600b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:24.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc3661ff700 1 -- 172.21.15.60:0/4201459892 shutdown_connections 2023-07-13T22:33:24.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.282+0000 7fc3661ff700 1 -- 172.21.15.60:0/4201459892 wait complete. 2023-07-13T22:33:24.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.285+0000 7fc3661ff700 1 Processor -- start 2023-07-13T22:33:24.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.285+0000 7fc3661ff700 1 -- start start 2023-07-13T22:33:24.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc3661ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3600ca010 0x7fc36015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc3661ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 0x7fc3600c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc3661ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc36015e640 0x7fc3600c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc3661ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc360159d30 con 0x7fc36015e640 2023-07-13T22:33:24.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc3661ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3601e6430 con 0x7fc3600ca010 2023-07-13T22:33:24.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc3661ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc360159770 con 0x7fc360154950 2023-07-13T22:33:24.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc36015e640 0x7fc3600c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc36015e640 0x7fc3600c2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53144/0 (socket says 172.21.15.60:53144) 2023-07-13T22:33:24.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35ffff700 1 -- 172.21.15.60:0/2779883988 learned_addr learned my addr 172.21.15.60:0/2779883988 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:24.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35f7fe700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3600ca010 0x7fc36015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35effd700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 0x7fc3600c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35ffff700 1 -- 172.21.15.60:0/2779883988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 msgr2=0x7fc3600c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35ffff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 0x7fc3600c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35ffff700 1 -- 172.21.15.60:0/2779883988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3600ca010 msgr2=0x7fc36015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35ffff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3600ca010 0x7fc36015bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35ffff700 1 -- 172.21.15.60:0/2779883988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500094a0 con 0x7fc36015e640 2023-07-13T22:33:24.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35f7fe700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3600ca010 0x7fc36015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:24.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.287+0000 7fc35ffff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc36015e640 0x7fc3600c2bb0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fc35800caf0 tx=0x7fc35800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:24.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.288+0000 7fc347fff700 1 -- 172.21.15.60:0/2779883988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35800aad0 con 0x7fc36015e640 2023-07-13T22:33:24.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.288+0000 7fc347fff700 1 -- 172.21.15.60:0/2779883988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35800ac30 con 0x7fc36015e640 2023-07-13T22:33:24.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.288+0000 7fc347fff700 1 -- 172.21.15.60:0/2779883988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358012400 con 0x7fc36015e640 2023-07-13T22:33:24.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.288+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc360159a50 con 0x7fc36015e640 2023-07-13T22:33:24.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.288+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3601e6150 con 0x7fc36015e640 2023-07-13T22:33:24.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.289+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3601e5cd0 con 0x7fc36015e640 2023-07-13T22:33:24.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.290+0000 7fc347fff700 1 -- 172.21.15.60:0/2779883988 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc35801b3b0 con 0x7fc36015e640 2023-07-13T22:33:24.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.291+0000 7fc347fff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34c06ace0 0x7fc34c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.291+0000 7fc347fff700 1 -- 172.21.15.60:0/2779883988 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc358088fb0 con 0x7fc36015e640 2023-07-13T22:33:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.291+0000 7fc35f7fe700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34c06ace0 0x7fc34c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.292+0000 7fc35f7fe700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34c06ace0 0x7fc34c06d190 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc3500090b0 tx=0x7fc350008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.295+0000 7fc347fff700 1 -- 172.21.15.60:0/2779883988 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc3601e5cd0 con 0x7fc36015e640 2023-07-13T22:33:24.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.500+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc36015ab30 con 0x7fc34c06ace0 2023-07-13T22:33:24.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.503+0000 7fc347fff700 1 -- 172.21.15.60:0/2779883988 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73042 (secure 0 0 0) 0x7fc350011630 con 0x7fc34c06ace0 2023-07-13T22:33:24.491 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:24.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34c06ace0 msgr2=0x7fc34c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34c06ace0 0x7fc34c06d190 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc3500090b0 tx=0x7fc350008030 comp rx=0 tx=0).stop 2023-07-13T22:33:24.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc36015e640 msgr2=0x7fc3600c2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc36015e640 0x7fc3600c2bb0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fc35800caf0 tx=0x7fc35800ceb0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 shutdown_connections 2023-07-13T22:33:24.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc34c06ace0 0x7fc34c06d190 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc36015e640 0x7fc3600c2bb0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3600ca010 0x7fc36015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 --2- 172.21.15.60:0/2779883988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc360154950 0x7fc3600c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 >> 172.21.15.60:0/2779883988 conn(0x7fc36006ab20 msgr2=0x7fc36015d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:24.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.505+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 shutdown_connections 2023-07-13T22:33:24.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.506+0000 7fc3661ff700 1 -- 172.21.15.60:0/2779883988 wait complete. 2023-07-13T22:33:24.495 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:33:24.510 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2023-07-13T22:33:23.291505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295976,"kb_used_data":14824,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678744,"statfs":{"total":773094113280,"available":772791033856,"internally_reserved":0,"allocated":15179776,"data_stored":7610594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14367,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001347"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":231,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:20.831979+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:33:20.831979+0000","last_peered":"2023-07-13T22:33:20.831979+0000","last_clean":"2023-07-13T22:33:20.831979+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:33:20.831979+0000","last_undegraded":"2023-07-13T22:33:20.831979+0000","last_fullsized":"2023-07-13T22:33:20.831979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:34:55.009871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":225,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.063401+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:33:21.063401+0000","last_peered":"2023-07-13T22:33:21.063401+0000","last_clean":"2023-07-13T22:33:21.063401+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:33:21.063401+0000","last_undegraded":"2023-07-13T22:33:21.063401+0000","last_fullsized":"2023-07-13T22:33:21.063401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:15:23.249437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":245,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:20.829395+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:33:20.829395+0000","last_peered":"2023-07-13T22:33:20.829395+0000","last_clean":"2023-07-13T22:33:20.829395+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:33:20.829395+0000","last_undegraded":"2023-07-13T22:33:20.829395+0000","last_fullsized":"2023-07-13T22:33:20.829395+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:08:52.972490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":369,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059670+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:33:21.059670+0000","last_peered":"2023-07-13T22:33:21.059670+0000","last_clean":"2023-07-13T22:33:21.059670+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:33:21.059670+0000","last_undegraded":"2023-07-13T22:33:21.059670+0000","last_fullsized":"2023-07-13T22:33:21.059670+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:59:58.301151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":221,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059338+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:33:21.059338+0000","last_peered":"2023-07-13T22:33:21.059338+0000","last_clean":"2023-07-13T22:33:21.059338+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:33:21.059338+0000","last_undegraded":"2023-07-13T22:33:21.059338+0000","last_fullsized":"2023-07-13T22:33:21.059338+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:04:24.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":221,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059698+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:33:21.059698+0000","last_peered":"2023-07-13T22:33:21.059698+0000","last_clean":"2023-07-13T22:33:21.059698+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:33:21.059698+0000","last_undegraded":"2023-07-13T22:33:21.059698+0000","last_fullsized":"2023-07-13T22:33:21.059698+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:02:44.690690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":255,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:20.831375+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:33:20.831375+0000","last_peered":"2023-07-13T22:33:20.831375+0000","last_clean":"2023-07-13T22:33:20.831375+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:33:20.831375+0000","last_undegraded":"2023-07-13T22:33:20.831375+0000","last_fullsized":"2023-07-13T22:33:20.831375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:32:19.338329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":222,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059401+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:33:21.059401+0000","last_peered":"2023-07-13T22:33:21.059401+0000","last_clean":"2023-07-13T22:33:21.059401+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:33:21.059401+0000","last_undegraded":"2023-07-13T22:33:21.059401+0000","last_fullsized":"2023-07-13T22:33:21.059401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:41:20.284345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":222,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059650+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:33:21.059650+0000","last_peered":"2023-07-13T22:33:21.059650+0000","last_clean":"2023-07-13T22:33:21.059650+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:33:21.059650+0000","last_undegraded":"2023-07-13T22:33:21.059650+0000","last_fullsized":"2023-07-13T22:33:21.059650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:03:32.536211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2338816,"data_stored":1392857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.39500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":1.115},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68799999999999994,"15min":0.68100000000000005},"min":{"1min":0.58399999999999996,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.91300000000000003,"5min":0.995,"15min":0.995},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.85599999999999998,"5min":0.88200000000000001,"15min":2.2610000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.34300000000000003},"max":{"1min":0.93700000000000006,"5min":1.0149999999999999,"15min":1.0609999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.73699999999999999},"min":{"1min":0.51000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88500000000000001,"5min":0.95599999999999996,"15min":3.5190000000000001},"last":0.63},{"interface":"front","average":{"1min":0.749,"5min":0.70699999999999996,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.36499999999999999},"max":{"1min":0.88200000000000001,"5min":1.028,"15min":3.4710000000000001},"last":0.435}]},{"osd":3,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.50700000000000001,"5min":0.41499999999999998,"15min":0.34599999999999997},"max":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":1.0489999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.61499999999999999,"5min":0.51200000000000001,"15min":0.48499999999999999},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1399999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.627,"15min":0.628},"min":{"1min":0.434,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.94799999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.1850000000000001},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.64500000000000002},"min":{"1min":0.439,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":1.0329999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.57799999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":1.0149999999999999,"15min":1.2110000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.0549999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.496,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":0.99099999999999999,"15min":1.042},"last":0.745}]}]},{"osd":6,"up_from":13,"seq":55834575001,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1744896,"data_stored":802489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.86299999999999999,"5min":1.089,"15min":1.089},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.65400000000000003},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":0.88500000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.40400000000000003,"15min":0.33900000000000002},"max":{"1min":0.90500000000000003,"5min":1.075,"15min":1.075},"last":0.872},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.94199999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.379},"max":{"1min":0.92800000000000005,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.47699999999999998,"5min":0.46100000000000002,"15min":0.35599999999999998},"max":{"1min":0.86699999999999999,"5min":1.042,"15min":1.042},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.54000000000000004,"5min":0.435,"15min":0.37},"max":{"1min":0.85299999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":1.123,"15min":1.123},"last":0.91400000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.49399999999999999,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.66400000000000003},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.84799999999999998,"5min":0.90200000000000002,"15min":0.95999999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.5,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94299999999999995,"5min":1.121,"15min":1.121},"last":1.052}]},{"osd":7,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.375},"max":{"1min":0.96999999999999997,"5min":1.1639999999999999,"15min":1.1890000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72899999999999998,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.39000000000000001},"max":{"1min":0.95399999999999996,"5min":1.071,"15min":1.111},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37444,"kb_used_data":2300,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334396,"statfs":{"total":96636764160,"available":96598421504,"internally_reserved":0,"allocated":2355200,"data_stored":1398809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.45900000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.249},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.952},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.499},"min":{"1min":0.308,"5min":0.29199999999999998,"15min":0.254},"max":{"1min":0.73199999999999998,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":0.752,"15min":2.1200000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":1.9810000000000001},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":2.008},"last":0.62},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.53600000000000003},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80400000000000005,"5min":0.93700000000000006,"15min":1.054},"last":0.371}]},{"osd":4,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95999999999999996,"5min":1.1870000000000001,"15min":2.028},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.76700000000000002},"min":{"1min":0.622,"5min":0.55500000000000005,"15min":0.46400000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.9219999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.751,"15min":0.75800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.44},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76100000000000001,"15min":0.79900000000000004},"min":{"1min":0.52800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.03,"5min":1.0720000000000001,"15min":2.0899999999999999},"last":1.01}]},{"osd":6,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.52600000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.2230000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77000000000000002,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":2.0609999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77100000000000002,"15min":0.79800000000000004},"min":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.50700000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.8700000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2609999999999999},"last":0.83699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575003,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1740800,"data_stored":801503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":11.791},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.47799999999999998,"15min":0.51700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":4.657},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57099999999999995,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84999999999999998,"5min":1.056,"15min":4.5389999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.504,"15min":0.74099999999999999},"min":{"1min":0.34999999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83899999999999997,"5min":1,"15min":13.398999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":1.8089999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54100000000000004,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.85299999999999998,"5min":1.095,"15min":4.4530000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":13.667999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.77000000000000002},"min":{"1min":0.63,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.776},"last":0.78900000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76600000000000001,"15min":1.0029999999999999},"min":{"1min":0.63700000000000001,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":13.349},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.79600000000000004,"15min":0.84499999999999997},"min":{"1min":0.73199999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":4.7649999999999997},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.67500000000000004,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.202,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.747,"15min":0.96899999999999997},"min":{"1min":0.60399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0209999999999999,"5min":1.046,"15min":11.679},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.76700000000000002,"15min":1.0069999999999999},"min":{"1min":0.69599999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96499999999999997,"5min":1.032,"15min":13.298},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.79200000000000004,"15min":0.85899999999999999},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.50900000000000001},"max":{"1min":1.119,"5min":1.119,"15min":11.456},"last":0.873}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1740800,"data_stored":801503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.434,"15min":0.47899999999999998},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.65100000000000002,"5min":1.605,"15min":2.282},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.29099999999999998},"max":{"1min":0.68300000000000005,"5min":1.034,"15min":1.4990000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.59899999999999998,"5min":1.228,"15min":1.228},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.6879999999999999,"15min":2.319},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.439,"5min":0.371,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":1.119,"15min":1.119},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.60499999999999998},"min":{"1min":0.47699999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"max":{"1min":0.71999999999999997,"5min":1.5149999999999999,"15min":2.351},"last":0.71799999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76000000000000001,"15min":0.79500000000000004},"min":{"1min":0.63500000000000001,"5min":0.56699999999999995,"15min":0.46000000000000002},"max":{"1min":1.0289999999999999,"5min":1.6499999999999999,"15min":2.238},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.51200000000000001,"15min":0.46899999999999997},"max":{"1min":1.0920000000000001,"5min":1.1140000000000001,"15min":1.518},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.77000000000000002},"min":{"1min":0.621,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":1.341,"15min":1.341},"last":0.876},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.78200000000000003,"15min":0.81799999999999995},"min":{"1min":0.64900000000000002,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.9099999999999999,"15min":2.1909999999999998},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.66300000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":1.107,"15min":1.488},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.63600000000000001,"5min":0.52300000000000002,"15min":0.48199999999999998},"max":{"1min":0.98599999999999999,"5min":1.448,"15min":1.448},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.77500000000000002,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.29},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.73199999999999998,"5min":0.57599999999999996,"15min":0.54400000000000004},"max":{"1min":1.0840000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834575002,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40948,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330892,"statfs":{"total":96636764160,"available":96594833408,"internally_reserved":0,"allocated":1748992,"data_stored":802496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.248},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.73999999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.32700000000000001,"5min":0.32400000000000001,"15min":0.27200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.373}]},{"osd":1,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.503,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.873,"15min":0.873},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35099999999999998},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.94299999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.46400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82099999999999995,"5min":0.90000000000000002,"15min":1.0409999999999999},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.57499999999999996,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.2250000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.1539999999999999},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.745,"15min":0.749},"min":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.496},"max":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.411},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.48599999999999999},"max":{"1min":0.996,"5min":1.05,"15min":1.5469999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.621,"5min":0.53400000000000003,"15min":0.51800000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.1559999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.621,"5min":0.52500000000000002,"15min":0.52100000000000002},"max":{"1min":0.97499999999999998,"5min":0.996,"15min":1.1240000000000001},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.78200000000000003},"min":{"1min":0.60699999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.054,"5min":1.054,"15min":1.1779999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.47199999999999998},"max":{"1min":1.0389999999999999,"5min":1.0800000000000001,"15min":1.1950000000000001},"last":0.85799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1748992,"data_stored":802496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.64800000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.96799999999999997,"15min":0.997},"last":0.432}]},{"osd":1,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.58799999999999997,"5min":0.44600000000000001,"15min":0.35099999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.298},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.45900000000000002,"15min":0.438},"max":{"1min":0.96899999999999997,"5min":1.0529999999999999,"15min":1.3109999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.371},"max":{"1min":0.94799999999999995,"5min":0.95099999999999996,"15min":1.1200000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.51400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90800000000000003,"5min":1.1279999999999999,"15min":1.1839999999999999},"last":0.75}]},{"osd":3,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":0.80200000000000005},"min":{"1min":0.495,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":1.234,"15min":10.657999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72399999999999998,"15min":0.81799999999999995},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":8.75},"last":0.503}]},{"osd":5,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.629},{"interface":"front","average":{"1min":0.621,"5min":0.69399999999999995,"15min":0.71399999999999997},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81899999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.81699999999999995,"5min":1.012,"15min":1.173},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.70099999999999996,"15min":0.70999999999999996},"min":{"1min":0.48699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84399999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.432},"max":{"1min":0.86299999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.56000000000000005,"5min":0.432,"15min":0.432},"max":{"1min":0.84899999999999998,"5min":1.0600000000000001,"15min":1.212},"last":0.73899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575002,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36864,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334976,"statfs":{"total":96636764160,"available":96599015424,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":1.042},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88500000000000001,"5min":0.92000000000000004,"15min":0.998},"last":0.82299999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.68100000000000005,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.68500000000000005},"min":{"1min":0.52600000000000002,"5min":0.34999999999999998,"15min":0.33100000000000002},"max":{"1min":0.90300000000000002,"5min":1.0129999999999999,"15min":8.9930000000000003},"last":0.621}]},{"osd":2,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96199999999999997},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.55100000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.86799999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.59199999999999997,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.91300000000000003,"5min":0.96499999999999997,"15min":0.96599999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.70699999999999996},"min":{"1min":0.621,"5min":0.48399999999999999,"15min":0.47899999999999998},"max":{"1min":0.93899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70699999999999996,"15min":0.68000000000000005},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88300000000000001,"5min":1.012,"15min":1.012},"last":0.86199999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72699999999999998,"15min":0.70999999999999996},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.42299999999999999},"max":{"1min":0.86499999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.749}]},{"osd":7,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.74399999999999999,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.41199999999999998},"max":{"1min":0.93799999999999994,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.84799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:33:24.513 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:33:24.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.731+0000 7f03717ff700 1 Processor -- start 2023-07-13T22:33:24.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.731+0000 7f03717ff700 1 -- start start 2023-07-13T22:33:24.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f03717ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f036c0ca010 0x7f036c0ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f03717ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c0ca970 con 0x7f036c0cafc0 2023-07-13T22:33:24.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f03717ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c0cb620 con 0x7f036c154950 2023-07-13T22:33:24.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f03717ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c0cb440 con 0x7f036c0ca010 2023-07-13T22:33:24.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f036a7fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f036c0cafc0 0x7f036c15b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52928/0 (socket says 172.21.15.60:52928) 2023-07-13T22:33:24.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f036a7fc700 1 -- 172.21.15.60:0/797166664 learned_addr learned my addr 172.21.15.60:0/797166664 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:24.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f036b7fe700 1 --2- 172.21.15.60:0/797166664 >> v2:172.21.15.105:3300/0 conn(0x7f036c0ca010 0x7f036c0ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f0369ffb700 1 -- 172.21.15.60:0/797166664 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869015174 0 0) 0x7f036c0cb620 con 0x7f036c154950 2023-07-13T22:33:24.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f036b7fe700 1 -- 172.21.15.60:0/797166664 >> v1:172.21.15.105:6789/0 conn(0x7f036c154950 legacy=0x7f036c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f036b7fe700 1 -- 172.21.15.60:0/797166664 >> v1:172.21.15.60:6789/0 conn(0x7f036c0cafc0 legacy=0x7f036c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f036b7fe700 1 -- 172.21.15.60:0/797166664 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c0cb900 con 0x7f036c0ca010 2023-07-13T22:33:24.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.732+0000 7f036b7fe700 1 --2- 172.21.15.60:0/797166664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c0ca010 0x7f036c0ca430 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0364009d90 tx=0x7f036400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f488b72625412ff3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:24.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f0369ffb700 1 -- 172.21.15.60:0/797166664 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400e070 con 0x7f036c0ca010 2023-07-13T22:33:24.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f0369ffb700 1 -- 172.21.15.60:0/797166664 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036400bad0 con 0x7f036c0ca010 2023-07-13T22:33:24.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f0369ffb700 1 -- 172.21.15.60:0/797166664 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400be80 con 0x7f036c0ca010 2023-07-13T22:33:24.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f03717ff700 1 -- 172.21.15.60:0/797166664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c0ca010 msgr2=0x7f036c0ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f03717ff700 1 --2- 172.21.15.60:0/797166664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c0ca010 0x7f036c0ca430 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0364009d90 tx=0x7f036400b330 comp rx=0 tx=0).stop 2023-07-13T22:33:24.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f03717ff700 1 -- 172.21.15.60:0/797166664 shutdown_connections 2023-07-13T22:33:24.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f03717ff700 1 --2- 172.21.15.60:0/797166664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c0ca010 0x7f036c0ca430 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f03717ff700 1 -- 172.21.15.60:0/797166664 >> 172.21.15.60:0/797166664 conn(0x7f036c06ab20 msgr2=0x7f036c0b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:24.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f03717ff700 1 -- 172.21.15.60:0/797166664 shutdown_connections 2023-07-13T22:33:24.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.733+0000 7f03717ff700 1 -- 172.21.15.60:0/797166664 wait complete. 2023-07-13T22:33:24.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.735+0000 7f03717ff700 1 Processor -- start 2023-07-13T22:33:24.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.735+0000 7f03717ff700 1 -- start start 2023-07-13T22:33:24.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.736+0000 7f03717ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036c0ca010 0x7f036c1f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.736+0000 7f03717ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f036c0cafc0 0x7f036c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.736+0000 7f03717ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c154950 0x7f036c14a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.736+0000 7f03717ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0c4210 con 0x7f036c0ca010 2023-07-13T22:33:24.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.736+0000 7f03717ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0c44f0 con 0x7f036c154950 2023-07-13T22:33:24.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.736+0000 7f03717ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0c47d0 con 0x7f036c0cafc0 2023-07-13T22:33:24.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.736+0000 7f036affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036c0ca010 0x7f036c1f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.736+0000 7f036affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036c0ca010 0x7f036c1f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53154/0 (socket says 172.21.15.60:53154) 2023-07-13T22:33:24.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.736+0000 7f036affd700 1 -- 172.21.15.60:0/1038628648 learned_addr learned my addr 172.21.15.60:0/1038628648 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:24.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f036a7fc700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f036c0cafc0 0x7f036c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f036b7fe700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c154950 0x7f036c14a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f036affd700 1 -- 172.21.15.60:0/1038628648 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f036c0cafc0 msgr2=0x7f036c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f036affd700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f036c0cafc0 0x7f036c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f036affd700 1 -- 172.21.15.60:0/1038628648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c154950 msgr2=0x7f036c14a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f036affd700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c154950 0x7f036c14a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f036affd700 1 -- 172.21.15.60:0/1038628648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364009a70 con 0x7f036c0ca010 2023-07-13T22:33:24.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f036b7fe700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c154950 0x7f036c14a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:24.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f036affd700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036c0ca010 0x7f036c1f28e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f035c00c550 tx=0x7f035c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:24.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f03537fe700 1 -- 172.21.15.60:0/1038628648 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0054e0 con 0x7f036c0ca010 2023-07-13T22:33:24.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f03537fe700 1 -- 172.21.15.60:0/1038628648 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c003680 con 0x7f036c0ca010 2023-07-13T22:33:24.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c0c4ab0 con 0x7f036c0ca010 2023-07-13T22:33:24.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.737+0000 7f03537fe700 1 -- 172.21.15.60:0/1038628648 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c003a30 con 0x7f036c0ca010 2023-07-13T22:33:24.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.738+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c14b820 con 0x7f036c0ca010 2023-07-13T22:33:24.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.739+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036c0741c0 con 0x7f036c0ca010 2023-07-13T22:33:24.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.740+0000 7f03537fe700 1 -- 172.21.15.60:0/1038628648 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f035c003b90 con 0x7f036c0ca010 2023-07-13T22:33:24.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.740+0000 7f03537fe700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f035806ace0 0x7f035806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:24.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.741+0000 7f03537fe700 1 -- 172.21.15.60:0/1038628648 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f035c0885e0 con 0x7f036c0ca010 2023-07-13T22:33:24.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.741+0000 7f036a7fc700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f035806ace0 0x7f035806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:24.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.741+0000 7f036a7fc700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f035806ace0 0x7f035806d190 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f036c153cf0 tx=0x7f0360004a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:24.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.745+0000 7f03537fe700 1 -- 172.21.15.60:0/1038628648 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f035c053910 con 0x7f036c0ca010 2023-07-13T22:33:24.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.949+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f036c0c4940 con 0x7f035806ace0 2023-07-13T22:33:24.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.954+0000 7f03537fe700 1 -- 172.21.15.60:0/1038628648 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73042 (secure 0 0 0) 0x7f036c0c4940 con 0x7f035806ace0 2023-07-13T22:33:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:24.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f035806ace0 msgr2=0x7f035806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f035806ace0 0x7f035806d190 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f036c153cf0 tx=0x7f0360004a50 comp rx=0 tx=0).stop 2023-07-13T22:33:24.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036c0ca010 msgr2=0x7f036c1f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:24.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036c0ca010 0x7f036c1f28e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f035c00c550 tx=0x7f035c00c860 comp rx=0 tx=0).stop 2023-07-13T22:33:24.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 shutdown_connections 2023-07-13T22:33:24.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f035806ace0 0x7f035806d190 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f036c0ca010 0x7f036c1f28e0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f036c154950 0x7f036c14a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 --2- 172.21.15.60:0/1038628648 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f036c0cafc0 0x7f036c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:24.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.955+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 >> 172.21.15.60:0/1038628648 conn(0x7f036c06ab20 msgr2=0x7f036c070610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:24.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.957+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 shutdown_connections 2023-07-13T22:33:24.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:24.957+0000 7f03717ff700 1 -- 172.21.15.60:0/1038628648 wait complete. 2023-07-13T22:33:24.947 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:33:24.960 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2023-07-13T22:33:23.291505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295976,"kb_used_data":14824,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678744,"statfs":{"total":773094113280,"available":772791033856,"internally_reserved":0,"allocated":15179776,"data_stored":7610594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14367,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001347"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":231,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:20.831979+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:33:20.831979+0000","last_peered":"2023-07-13T22:33:20.831979+0000","last_clean":"2023-07-13T22:33:20.831979+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:33:20.831979+0000","last_undegraded":"2023-07-13T22:33:20.831979+0000","last_fullsized":"2023-07-13T22:33:20.831979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:34:55.009871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":225,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.063401+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:33:21.063401+0000","last_peered":"2023-07-13T22:33:21.063401+0000","last_clean":"2023-07-13T22:33:21.063401+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:33:21.063401+0000","last_undegraded":"2023-07-13T22:33:21.063401+0000","last_fullsized":"2023-07-13T22:33:21.063401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:15:23.249437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":245,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:20.829395+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:33:20.829395+0000","last_peered":"2023-07-13T22:33:20.829395+0000","last_clean":"2023-07-13T22:33:20.829395+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:33:20.829395+0000","last_undegraded":"2023-07-13T22:33:20.829395+0000","last_fullsized":"2023-07-13T22:33:20.829395+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:08:52.972490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":369,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059670+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:33:21.059670+0000","last_peered":"2023-07-13T22:33:21.059670+0000","last_clean":"2023-07-13T22:33:21.059670+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:33:21.059670+0000","last_undegraded":"2023-07-13T22:33:21.059670+0000","last_fullsized":"2023-07-13T22:33:21.059670+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:59:58.301151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":221,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059338+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:33:21.059338+0000","last_peered":"2023-07-13T22:33:21.059338+0000","last_clean":"2023-07-13T22:33:21.059338+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:33:21.059338+0000","last_undegraded":"2023-07-13T22:33:21.059338+0000","last_fullsized":"2023-07-13T22:33:21.059338+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:04:24.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":221,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059698+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:33:21.059698+0000","last_peered":"2023-07-13T22:33:21.059698+0000","last_clean":"2023-07-13T22:33:21.059698+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:33:21.059698+0000","last_undegraded":"2023-07-13T22:33:21.059698+0000","last_fullsized":"2023-07-13T22:33:21.059698+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:02:44.690690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":255,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:20.831375+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:33:20.831375+0000","last_peered":"2023-07-13T22:33:20.831375+0000","last_clean":"2023-07-13T22:33:20.831375+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:33:20.831375+0000","last_undegraded":"2023-07-13T22:33:20.831375+0000","last_fullsized":"2023-07-13T22:33:20.831375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:32:19.338329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":222,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059401+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:33:21.059401+0000","last_peered":"2023-07-13T22:33:21.059401+0000","last_clean":"2023-07-13T22:33:21.059401+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:33:21.059401+0000","last_undegraded":"2023-07-13T22:33:21.059401+0000","last_fullsized":"2023-07-13T22:33:21.059401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:41:20.284345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":222,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059650+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:33:21.059650+0000","last_peered":"2023-07-13T22:33:21.059650+0000","last_clean":"2023-07-13T22:33:21.059650+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:33:21.059650+0000","last_undegraded":"2023-07-13T22:33:21.059650+0000","last_fullsized":"2023-07-13T22:33:21.059650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:03:32.536211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2338816,"data_stored":1392857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.39500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":1.115},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68799999999999994,"15min":0.68100000000000005},"min":{"1min":0.58399999999999996,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.91300000000000003,"5min":0.995,"15min":0.995},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.85599999999999998,"5min":0.88200000000000001,"15min":2.2610000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.34300000000000003},"max":{"1min":0.93700000000000006,"5min":1.0149999999999999,"15min":1.0609999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.73699999999999999},"min":{"1min":0.51000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88500000000000001,"5min":0.95599999999999996,"15min":3.5190000000000001},"last":0.63},{"interface":"front","average":{"1min":0.749,"5min":0.70699999999999996,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.36499999999999999},"max":{"1min":0.88200000000000001,"5min":1.028,"15min":3.4710000000000001},"last":0.435}]},{"osd":3,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.50700000000000001,"5min":0.41499999999999998,"15min":0.34599999999999997},"max":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":1.0489999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.61499999999999999,"5min":0.51200000000000001,"15min":0.48499999999999999},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1399999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.627,"15min":0.628},"min":{"1min":0.434,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.94799999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.1850000000000001},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.64500000000000002},"min":{"1min":0.439,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":1.0329999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.57799999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":1.0149999999999999,"15min":1.2110000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.0549999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.496,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":0.99099999999999999,"15min":1.042},"last":0.745}]}]},{"osd":6,"up_from":13,"seq":55834575001,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1744896,"data_stored":802489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.86299999999999999,"5min":1.089,"15min":1.089},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.65400000000000003},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":0.88500000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.40400000000000003,"15min":0.33900000000000002},"max":{"1min":0.90500000000000003,"5min":1.075,"15min":1.075},"last":0.872},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.94199999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.379},"max":{"1min":0.92800000000000005,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.47699999999999998,"5min":0.46100000000000002,"15min":0.35599999999999998},"max":{"1min":0.86699999999999999,"5min":1.042,"15min":1.042},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.54000000000000004,"5min":0.435,"15min":0.37},"max":{"1min":0.85299999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":1.123,"15min":1.123},"last":0.91400000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.49399999999999999,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.66400000000000003},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.84799999999999998,"5min":0.90200000000000002,"15min":0.95999999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.5,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94299999999999995,"5min":1.121,"15min":1.121},"last":1.052}]},{"osd":7,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.375},"max":{"1min":0.96999999999999997,"5min":1.1639999999999999,"15min":1.1890000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72899999999999998,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.39000000000000001},"max":{"1min":0.95399999999999996,"5min":1.071,"15min":1.111},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37444,"kb_used_data":2300,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334396,"statfs":{"total":96636764160,"available":96598421504,"internally_reserved":0,"allocated":2355200,"data_stored":1398809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.45900000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.249},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.952},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.499},"min":{"1min":0.308,"5min":0.29199999999999998,"15min":0.254},"max":{"1min":0.73199999999999998,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":0.752,"15min":2.1200000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":1.9810000000000001},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":2.008},"last":0.62},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.53600000000000003},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80400000000000005,"5min":0.93700000000000006,"15min":1.054},"last":0.371}]},{"osd":4,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95999999999999996,"5min":1.1870000000000001,"15min":2.028},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.76700000000000002},"min":{"1min":0.622,"5min":0.55500000000000005,"15min":0.46400000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.9219999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.751,"15min":0.75800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.44},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76100000000000001,"15min":0.79900000000000004},"min":{"1min":0.52800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.03,"5min":1.0720000000000001,"15min":2.0899999999999999},"last":1.01}]},{"osd":6,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.52600000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.2230000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77000000000000002,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":2.0609999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77100000000000002,"15min":0.79800000000000004},"min":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.50700000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.8700000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2609999999999999},"last":0.83699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575003,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1740800,"data_stored":801503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":11.791},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.47799999999999998,"15min":0.51700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":4.657},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57099999999999995,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84999999999999998,"5min":1.056,"15min":4.5389999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.504,"15min":0.74099999999999999},"min":{"1min":0.34999999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83899999999999997,"5min":1,"15min":13.398999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":1.8089999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54100000000000004,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.85299999999999998,"5min":1.095,"15min":4.4530000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":13.667999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.77000000000000002},"min":{"1min":0.63,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.776},"last":0.78900000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76600000000000001,"15min":1.0029999999999999},"min":{"1min":0.63700000000000001,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":13.349},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.79600000000000004,"15min":0.84499999999999997},"min":{"1min":0.73199999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":4.7649999999999997},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.67500000000000004,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.202,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.747,"15min":0.96899999999999997},"min":{"1min":0.60399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0209999999999999,"5min":1.046,"15min":11.679},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.76700000000000002,"15min":1.0069999999999999},"min":{"1min":0.69599999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96499999999999997,"5min":1.032,"15min":13.298},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.79200000000000004,"15min":0.85899999999999999},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.50900000000000001},"max":{"1min":1.119,"5min":1.119,"15min":11.456},"last":0.873}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1740800,"data_stored":801503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.434,"15min":0.47899999999999998},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.65100000000000002,"5min":1.605,"15min":2.282},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.29099999999999998},"max":{"1min":0.68300000000000005,"5min":1.034,"15min":1.4990000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.59899999999999998,"5min":1.228,"15min":1.228},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.6879999999999999,"15min":2.319},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.439,"5min":0.371,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":1.119,"15min":1.119},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.60499999999999998},"min":{"1min":0.47699999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"max":{"1min":0.71999999999999997,"5min":1.5149999999999999,"15min":2.351},"last":0.71799999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76000000000000001,"15min":0.79500000000000004},"min":{"1min":0.63500000000000001,"5min":0.56699999999999995,"15min":0.46000000000000002},"max":{"1min":1.0289999999999999,"5min":1.6499999999999999,"15min":2.238},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.51200000000000001,"15min":0.46899999999999997},"max":{"1min":1.0920000000000001,"5min":1.1140000000000001,"15min":1.518},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.77000000000000002},"min":{"1min":0.621,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":1.341,"15min":1.341},"last":0.876},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.78200000000000003,"15min":0.81799999999999995},"min":{"1min":0.64900000000000002,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.9099999999999999,"15min":2.1909999999999998},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.66300000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":1.107,"15min":1.488},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.63600000000000001,"5min":0.52300000000000002,"15min":0.48199999999999998},"max":{"1min":0.98599999999999999,"5min":1.448,"15min":1.448},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.77500000000000002,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.29},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.73199999999999998,"5min":0.57599999999999996,"15min":0.54400000000000004},"max":{"1min":1.0840000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834575002,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40948,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330892,"statfs":{"total":96636764160,"available":96594833408,"internally_reserved":0,"allocated":1748992,"data_stored":802496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.248},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.73999999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.32700000000000001,"5min":0.32400000000000001,"15min":0.27200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.373}]},{"osd":1,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.503,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.873,"15min":0.873},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35099999999999998},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.94299999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.46400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82099999999999995,"5min":0.90000000000000002,"15min":1.0409999999999999},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.57499999999999996,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.2250000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.1539999999999999},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.745,"15min":0.749},"min":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.496},"max":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.411},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.48599999999999999},"max":{"1min":0.996,"5min":1.05,"15min":1.5469999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.621,"5min":0.53400000000000003,"15min":0.51800000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.1559999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.621,"5min":0.52500000000000002,"15min":0.52100000000000002},"max":{"1min":0.97499999999999998,"5min":0.996,"15min":1.1240000000000001},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.78200000000000003},"min":{"1min":0.60699999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.054,"5min":1.054,"15min":1.1779999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.47199999999999998},"max":{"1min":1.0389999999999999,"5min":1.0800000000000001,"15min":1.1950000000000001},"last":0.85799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1748992,"data_stored":802496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.64800000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.96799999999999997,"15min":0.997},"last":0.432}]},{"osd":1,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.58799999999999997,"5min":0.44600000000000001,"15min":0.35099999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.298},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.45900000000000002,"15min":0.438},"max":{"1min":0.96899999999999997,"5min":1.0529999999999999,"15min":1.3109999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.371},"max":{"1min":0.94799999999999995,"5min":0.95099999999999996,"15min":1.1200000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.51400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90800000000000003,"5min":1.1279999999999999,"15min":1.1839999999999999},"last":0.75}]},{"osd":3,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":0.80200000000000005},"min":{"1min":0.495,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":1.234,"15min":10.657999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72399999999999998,"15min":0.81799999999999995},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":8.75},"last":0.503}]},{"osd":5,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.629},{"interface":"front","average":{"1min":0.621,"5min":0.69399999999999995,"15min":0.71399999999999997},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81899999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.81699999999999995,"5min":1.012,"15min":1.173},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.70099999999999996,"15min":0.70999999999999996},"min":{"1min":0.48699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84399999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.432},"max":{"1min":0.86299999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.56000000000000005,"5min":0.432,"15min":0.432},"max":{"1min":0.84899999999999998,"5min":1.0600000000000001,"15min":1.212},"last":0.73899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575002,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36864,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334976,"statfs":{"total":96636764160,"available":96599015424,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":1.042},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88500000000000001,"5min":0.92000000000000004,"15min":0.998},"last":0.82299999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.68100000000000005,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.68500000000000005},"min":{"1min":0.52600000000000002,"5min":0.34999999999999998,"15min":0.33100000000000002},"max":{"1min":0.90300000000000002,"5min":1.0129999999999999,"15min":8.9930000000000003},"last":0.621}]},{"osd":2,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96199999999999997},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.55100000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.86799999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.59199999999999997,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.91300000000000003,"5min":0.96499999999999997,"15min":0.96599999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.70699999999999996},"min":{"1min":0.621,"5min":0.48399999999999999,"15min":0.47899999999999998},"max":{"1min":0.93899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70699999999999996,"15min":0.68000000000000005},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88300000000000001,"5min":1.012,"15min":1.012},"last":0.86199999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72699999999999998,"15min":0.70999999999999996},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.42299999999999999},"max":{"1min":0.86499999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.749}]},{"osd":7,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.74399999999999999,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.41199999999999998},"max":{"1min":0.93799999999999994,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.84799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:33:24.962 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-13T22:33:24.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:33:25.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.180+0000 7fb55c5ff700 1 Processor -- start 2023-07-13T22:33:25.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.181+0000 7fb55c5ff700 1 -- start start 2023-07-13T22:33:25.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb55c5ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5540b20d0 0x7fb5541608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb55c5ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5540ceec0 0x7fb5540b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb55c5ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540cd1d0 con 0x7fb5540b59d0 2023-07-13T22:33:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb55c5ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540b24a0 con 0x7fb5540ceec0 2023-07-13T22:33:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb55c5ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540cd790 con 0x7fb5540b20d0 2023-07-13T22:33:25.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb559b9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5540ceec0 0x7fb5540b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb559b9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5540ceec0 0x7fb5540b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50426/0 (socket says 172.21.15.60:50426) 2023-07-13T22:33:25.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb559b9a700 1 -- 172.21.15.60:0/3321934534 learned_addr learned my addr 172.21.15.60:0/3321934534 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:25.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb55a39b700 1 --2- 172.21.15.60:0/3321934534 >> v2:172.21.15.105:3300/0 conn(0x7fb5540b20d0 0x7fb5541608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb559b9a700 1 -- 172.21.15.60:0/3321934534 >> v2:172.21.15.105:3300/0 conn(0x7fb5540b20d0 msgr2=0x7fb5541608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:25.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb559b9a700 1 --2- 172.21.15.60:0/3321934534 >> v2:172.21.15.105:3300/0 conn(0x7fb5540b20d0 0x7fb5541608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.182+0000 7fb559b9a700 1 -- 172.21.15.60:0/3321934534 >> v1:172.21.15.105:6789/0 conn(0x7fb5540b59d0 legacy=0x7fb5540d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:25.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.183+0000 7fb559b9a700 1 -- 172.21.15.60:0/3321934534 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554160e20 con 0x7fb5540ceec0 2023-07-13T22:33:25.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.183+0000 7fb55a39b700 1 --2- 172.21.15.60:0/3321934534 >> v2:172.21.15.105:3300/0 conn(0x7fb5540b20d0 0x7fb5541608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:25.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.183+0000 7fb559b9a700 1 --2- 172.21.15.60:0/3321934534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540ceec0 0x7fb5540b4600 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fb5540d0700 tx=0x7fb544009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875b03d0b3452227 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:25.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.183+0000 7fb559399700 1 -- 172.21.15.60:0/3321934534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544004230 con 0x7fb5540ceec0 2023-07-13T22:33:25.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.183+0000 7fb559399700 1 -- 172.21.15.60:0/3321934534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544004390 con 0x7fb5540ceec0 2023-07-13T22:33:25.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.183+0000 7fb559399700 1 -- 172.21.15.60:0/3321934534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5440036a0 con 0x7fb5540ceec0 2023-07-13T22:33:25.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.184+0000 7fb55c5ff700 1 -- 172.21.15.60:0/3321934534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540ceec0 msgr2=0x7fb5540b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.184+0000 7fb55c5ff700 1 --2- 172.21.15.60:0/3321934534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540ceec0 0x7fb5540b4600 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fb5540d0700 tx=0x7fb544009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.184+0000 7fb55c5ff700 1 -- 172.21.15.60:0/3321934534 shutdown_connections 2023-07-13T22:33:25.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.184+0000 7fb55c5ff700 1 --2- 172.21.15.60:0/3321934534 >> v2:172.21.15.105:3300/0 conn(0x7fb5540b20d0 0x7fb5541608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.184+0000 7fb55c5ff700 1 --2- 172.21.15.60:0/3321934534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540ceec0 0x7fb5540b4600 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.184+0000 7fb55c5ff700 1 -- 172.21.15.60:0/3321934534 >> 172.21.15.60:0/3321934534 conn(0x7fb5540cf600 msgr2=0x7fb5540cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:25.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.184+0000 7fb55c5ff700 1 -- 172.21.15.60:0/3321934534 shutdown_connections 2023-07-13T22:33:25.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.184+0000 7fb55c5ff700 1 -- 172.21.15.60:0/3321934534 wait complete. 2023-07-13T22:33:25.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.185+0000 7fb55c5ff700 1 Processor -- start 2023-07-13T22:33:25.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.186+0000 7fb55c5ff700 1 -- start start 2023-07-13T22:33:25.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.186+0000 7fb55c5ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5540b20d0 0x7fb554154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.186+0000 7fb55c5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540b59d0 0x7fb5540cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb55c5ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5540ceec0 0x7fb554150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb55c5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540cc390 con 0x7fb5540b59d0 2023-07-13T22:33:25.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb55c5ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540cc820 con 0x7fb5540ceec0 2023-07-13T22:33:25.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb55c5ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540cb4e0 con 0x7fb5540b20d0 2023-07-13T22:33:25.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb559b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540b59d0 0x7fb5540cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb559b9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540b59d0 0x7fb5540cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50432/0 (socket says 172.21.15.60:50432) 2023-07-13T22:33:25.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb559b9a700 1 -- 172.21.15.60:0/2781856873 learned_addr learned my addr 172.21.15.60:0/2781856873 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:25.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb55ab9c700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5540ceec0 0x7fb554150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb55a39b700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5540b20d0 0x7fb554154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb559b9a700 1 -- 172.21.15.60:0/2781856873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5540b20d0 msgr2=0x7fb554154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb559b9a700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5540b20d0 0x7fb554154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb559b9a700 1 -- 172.21.15.60:0/2781856873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5540ceec0 msgr2=0x7fb554150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb559b9a700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5540ceec0 0x7fb554150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb559b9a700 1 -- 172.21.15.60:0/2781856873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544009550 con 0x7fb5540b59d0 2023-07-13T22:33:25.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.187+0000 7fb559b9a700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540b59d0 0x7fb5540cafa0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fb544004000 tx=0x7fb544004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:25.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.188+0000 7fb54affd700 1 -- 172.21.15.60:0/2781856873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544003dc0 con 0x7fb5540b59d0 2023-07-13T22:33:25.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.188+0000 7fb54affd700 1 -- 172.21.15.60:0/2781856873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54401f070 con 0x7fb5540b59d0 2023-07-13T22:33:25.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.188+0000 7fb54affd700 1 -- 172.21.15.60:0/2781856873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544011870 con 0x7fb5540b59d0 2023-07-13T22:33:25.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.188+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5540cb780 con 0x7fb5540b59d0 2023-07-13T22:33:25.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.188+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5540cbd40 con 0x7fb5540b59d0 2023-07-13T22:33:25.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.189+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5540cc540 con 0x7fb5540b59d0 2023-07-13T22:33:25.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.190+0000 7fb54affd700 1 -- 172.21.15.60:0/2781856873 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb544021030 con 0x7fb5540b59d0 2023-07-13T22:33:25.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.191+0000 7fb54affd700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb53806ace0 0x7fb53806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.191+0000 7fb54affd700 1 -- 172.21.15.60:0/2781856873 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb54408a7d0 con 0x7fb5540b59d0 2023-07-13T22:33:25.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.191+0000 7fb55a39b700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb53806ace0 0x7fb53806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.192+0000 7fb55a39b700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb53806ace0 0x7fb53806d190 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb550006fd0 tx=0x7fb550008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:25.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.195+0000 7fb54affd700 1 -- 172.21.15.60:0/2781856873 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fb5540cc540 con 0x7fb5540b59d0 2023-07-13T22:33:25.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.395+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5540d2800 con 0x7fb53806ace0 2023-07-13T22:33:25.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.398+0000 7fb54affd700 1 -- 172.21.15.60:0/2781856873 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73042 (secure 0 0 0) 0x7fb55000a720 con 0x7fb53806ace0 2023-07-13T22:33:25.385 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:25.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb53806ace0 msgr2=0x7fb53806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb53806ace0 0x7fb53806d190 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb550006fd0 tx=0x7fb550008040 comp rx=0 tx=0).stop 2023-07-13T22:33:25.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540b59d0 msgr2=0x7fb5540cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540b59d0 0x7fb5540cafa0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fb544004000 tx=0x7fb544004750 comp rx=0 tx=0).stop 2023-07-13T22:33:25.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 shutdown_connections 2023-07-13T22:33:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fb53806ace0 0x7fb53806d190 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5540ceec0 0x7fb554150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5540b20d0 0x7fb554154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 --2- 172.21.15.60:0/2781856873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5540b59d0 0x7fb5540cafa0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 >> 172.21.15.60:0/2781856873 conn(0x7fb5540cf600 msgr2=0x7fb5540b80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:25.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 shutdown_connections 2023-07-13T22:33:25.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.401+0000 7fb55c5ff700 1 -- 172.21.15.60:0/2781856873 wait complete. 2023-07-13T22:33:25.390 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:33:25.405 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2023-07-13T22:33:25.291846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295976,"kb_used_data":14824,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678744,"statfs":{"total":773094113280,"available":772791033856,"internally_reserved":0,"allocated":15179776,"data_stored":7610594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14367,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.679162"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":231,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:20.831979+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:33:20.831979+0000","last_peered":"2023-07-13T22:33:20.831979+0000","last_clean":"2023-07-13T22:33:20.831979+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:33:20.831979+0000","last_undegraded":"2023-07-13T22:33:20.831979+0000","last_fullsized":"2023-07-13T22:33:20.831979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:34:55.009871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":225,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.063401+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:33:21.063401+0000","last_peered":"2023-07-13T22:33:21.063401+0000","last_clean":"2023-07-13T22:33:21.063401+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:33:21.063401+0000","last_undegraded":"2023-07-13T22:33:21.063401+0000","last_fullsized":"2023-07-13T22:33:21.063401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:15:23.249437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":245,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:20.829395+0000","last_change":"2023-07-13T22:20:47.725995+0000","last_active":"2023-07-13T22:33:20.829395+0000","last_peered":"2023-07-13T22:33:20.829395+0000","last_clean":"2023-07-13T22:33:20.829395+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:33:20.829395+0000","last_undegraded":"2023-07-13T22:33:20.829395+0000","last_fullsized":"2023-07-13T22:33:20.829395+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:08:52.972490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":369,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059670+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:33:21.059670+0000","last_peered":"2023-07-13T22:33:21.059670+0000","last_clean":"2023-07-13T22:33:21.059670+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:33:21.059670+0000","last_undegraded":"2023-07-13T22:33:21.059670+0000","last_fullsized":"2023-07-13T22:33:21.059670+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:59:58.301151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":221,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059338+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:33:21.059338+0000","last_peered":"2023-07-13T22:33:21.059338+0000","last_clean":"2023-07-13T22:33:21.059338+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:33:21.059338+0000","last_undegraded":"2023-07-13T22:33:21.059338+0000","last_fullsized":"2023-07-13T22:33:21.059338+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:04:24.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":221,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059698+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:33:21.059698+0000","last_peered":"2023-07-13T22:33:21.059698+0000","last_clean":"2023-07-13T22:33:21.059698+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:33:21.059698+0000","last_undegraded":"2023-07-13T22:33:21.059698+0000","last_fullsized":"2023-07-13T22:33:21.059698+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:02:44.690690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":255,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:20.831375+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:33:20.831375+0000","last_peered":"2023-07-13T22:33:20.831375+0000","last_clean":"2023-07-13T22:33:20.831375+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:33:20.831375+0000","last_undegraded":"2023-07-13T22:33:20.831375+0000","last_fullsized":"2023-07-13T22:33:20.831375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:32:19.338329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":222,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059401+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:33:21.059401+0000","last_peered":"2023-07-13T22:33:21.059401+0000","last_clean":"2023-07-13T22:33:21.059401+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:33:21.059401+0000","last_undegraded":"2023-07-13T22:33:21.059401+0000","last_fullsized":"2023-07-13T22:33:21.059401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:41:20.284345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":222,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059650+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:33:21.059650+0000","last_peered":"2023-07-13T22:33:21.059650+0000","last_clean":"2023-07-13T22:33:21.059650+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:33:21.059650+0000","last_undegraded":"2023-07-13T22:33:21.059650+0000","last_fullsized":"2023-07-13T22:33:21.059650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:03:32.536211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2338816,"data_stored":1392857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.39500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":1.115},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68799999999999994,"15min":0.68100000000000005},"min":{"1min":0.58399999999999996,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.91300000000000003,"5min":0.995,"15min":0.995},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.85599999999999998,"5min":0.88200000000000001,"15min":2.2610000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.34300000000000003},"max":{"1min":0.93700000000000006,"5min":1.0149999999999999,"15min":1.0609999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.73699999999999999},"min":{"1min":0.51000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88500000000000001,"5min":0.95599999999999996,"15min":3.5190000000000001},"last":0.63},{"interface":"front","average":{"1min":0.749,"5min":0.70699999999999996,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.36499999999999999},"max":{"1min":0.88200000000000001,"5min":1.028,"15min":3.4710000000000001},"last":0.435}]},{"osd":3,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.50700000000000001,"5min":0.41499999999999998,"15min":0.34599999999999997},"max":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":1.0489999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.61499999999999999,"5min":0.51200000000000001,"15min":0.48499999999999999},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1399999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.627,"15min":0.628},"min":{"1min":0.434,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.94799999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.1850000000000001},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.64500000000000002},"min":{"1min":0.439,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":1.0329999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.57799999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":1.0149999999999999,"15min":1.2110000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.0549999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.496,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":0.99099999999999999,"15min":1.042},"last":0.745}]}]},{"osd":6,"up_from":13,"seq":55834575001,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1744896,"data_stored":802489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.86299999999999999,"5min":1.089,"15min":1.089},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.65400000000000003},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":0.88500000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.40400000000000003,"15min":0.33900000000000002},"max":{"1min":0.90500000000000003,"5min":1.075,"15min":1.075},"last":0.872},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.94199999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.379},"max":{"1min":0.92800000000000005,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.47699999999999998,"5min":0.46100000000000002,"15min":0.35599999999999998},"max":{"1min":0.86699999999999999,"5min":1.042,"15min":1.042},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.54000000000000004,"5min":0.435,"15min":0.37},"max":{"1min":0.85299999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":1.123,"15min":1.123},"last":0.91400000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.49399999999999999,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.66400000000000003},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.84799999999999998,"5min":0.90200000000000002,"15min":0.95999999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.5,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94299999999999995,"5min":1.121,"15min":1.121},"last":1.052}]},{"osd":7,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.375},"max":{"1min":0.96999999999999997,"5min":1.1639999999999999,"15min":1.1890000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72899999999999998,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.39000000000000001},"max":{"1min":0.95399999999999996,"5min":1.071,"15min":1.111},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37444,"kb_used_data":2300,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334396,"statfs":{"total":96636764160,"available":96598421504,"internally_reserved":0,"allocated":2355200,"data_stored":1398809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.45900000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.249},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.952},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.499},"min":{"1min":0.308,"5min":0.29199999999999998,"15min":0.254},"max":{"1min":0.73199999999999998,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":0.752,"15min":2.1200000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":1.9810000000000001},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":2.008},"last":0.62},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.53600000000000003},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80400000000000005,"5min":0.93700000000000006,"15min":1.054},"last":0.371}]},{"osd":4,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95999999999999996,"5min":1.1870000000000001,"15min":2.028},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.76700000000000002},"min":{"1min":0.622,"5min":0.55500000000000005,"15min":0.46400000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.9219999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.751,"15min":0.75800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.44},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76100000000000001,"15min":0.79900000000000004},"min":{"1min":0.52800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.03,"5min":1.0720000000000001,"15min":2.0899999999999999},"last":1.01}]},{"osd":6,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.52600000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.2230000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77000000000000002,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":2.0609999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77100000000000002,"15min":0.79800000000000004},"min":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.50700000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.8700000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2609999999999999},"last":0.83699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575003,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1740800,"data_stored":801503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":11.791},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.47799999999999998,"15min":0.51700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":4.657},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57099999999999995,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84999999999999998,"5min":1.056,"15min":4.5389999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.504,"15min":0.74099999999999999},"min":{"1min":0.34999999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83899999999999997,"5min":1,"15min":13.398999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":1.8089999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54100000000000004,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.85299999999999998,"5min":1.095,"15min":4.4530000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":13.667999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.77000000000000002},"min":{"1min":0.63,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.776},"last":0.78900000000000003}]},{"osd":5,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76600000000000001,"15min":1.0029999999999999},"min":{"1min":0.63700000000000001,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":13.349},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.79600000000000004,"15min":0.84499999999999997},"min":{"1min":0.73199999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":4.7649999999999997},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.67500000000000004,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.202,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.747,"15min":0.96899999999999997},"min":{"1min":0.60399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0209999999999999,"5min":1.046,"15min":11.679},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.76700000000000002,"15min":1.0069999999999999},"min":{"1min":0.69599999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96499999999999997,"5min":1.032,"15min":13.298},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.79200000000000004,"15min":0.85899999999999999},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.50900000000000001},"max":{"1min":1.119,"5min":1.119,"15min":11.456},"last":0.873}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1740800,"data_stored":801503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.434,"15min":0.47899999999999998},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.65100000000000002,"5min":1.605,"15min":2.282},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.29099999999999998},"max":{"1min":0.68300000000000005,"5min":1.034,"15min":1.4990000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.59899999999999998,"5min":1.228,"15min":1.228},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.6879999999999999,"15min":2.319},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.439,"5min":0.371,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":1.119,"15min":1.119},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.60499999999999998},"min":{"1min":0.47699999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"max":{"1min":0.71999999999999997,"5min":1.5149999999999999,"15min":2.351},"last":0.71799999999999997}]},{"osd":4,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76000000000000001,"15min":0.79500000000000004},"min":{"1min":0.63500000000000001,"5min":0.56699999999999995,"15min":0.46000000000000002},"max":{"1min":1.0289999999999999,"5min":1.6499999999999999,"15min":2.238},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.51200000000000001,"15min":0.46899999999999997},"max":{"1min":1.0920000000000001,"5min":1.1140000000000001,"15min":1.518},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.77000000000000002},"min":{"1min":0.621,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":1.341,"15min":1.341},"last":0.876},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.78200000000000003,"15min":0.81799999999999995},"min":{"1min":0.64900000000000002,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.9099999999999999,"15min":2.1909999999999998},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.66300000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":1.107,"15min":1.488},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.63600000000000001,"5min":0.52300000000000002,"15min":0.48199999999999998},"max":{"1min":0.98599999999999999,"5min":1.448,"15min":1.448},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.77500000000000002,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.29},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.73199999999999998,"5min":0.57599999999999996,"15min":0.54400000000000004},"max":{"1min":1.0840000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834575002,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40948,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330892,"statfs":{"total":96636764160,"available":96594833408,"internally_reserved":0,"allocated":1748992,"data_stored":802496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.248},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.73999999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.32700000000000001,"5min":0.32400000000000001,"15min":0.27200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.373}]},{"osd":1,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.503,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.873,"15min":0.873},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35099999999999998},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.94299999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.46400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82099999999999995,"5min":0.90000000000000002,"15min":1.0409999999999999},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.57499999999999996,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.2250000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.1539999999999999},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.745,"15min":0.749},"min":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.496},"max":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.411},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.48599999999999999},"max":{"1min":0.996,"5min":1.05,"15min":1.5469999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.621,"5min":0.53400000000000003,"15min":0.51800000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.1559999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.621,"5min":0.52500000000000002,"15min":0.52100000000000002},"max":{"1min":0.97499999999999998,"5min":0.996,"15min":1.1240000000000001},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.78200000000000003},"min":{"1min":0.60699999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.054,"5min":1.054,"15min":1.1779999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.47199999999999998},"max":{"1min":1.0389999999999999,"5min":1.0800000000000001,"15min":1.1950000000000001},"last":0.85799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1748992,"data_stored":802496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.64800000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.96799999999999997,"15min":0.997},"last":0.432}]},{"osd":1,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.58799999999999997,"5min":0.44600000000000001,"15min":0.35099999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.298},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.45900000000000002,"15min":0.438},"max":{"1min":0.96899999999999997,"5min":1.0529999999999999,"15min":1.3109999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.371},"max":{"1min":0.94799999999999995,"5min":0.95099999999999996,"15min":1.1200000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.51400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90800000000000003,"5min":1.1279999999999999,"15min":1.1839999999999999},"last":0.75}]},{"osd":3,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":0.80200000000000005},"min":{"1min":0.495,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":1.234,"15min":10.657999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72399999999999998,"15min":0.81799999999999995},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":8.75},"last":0.503}]},{"osd":5,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.629},{"interface":"front","average":{"1min":0.621,"5min":0.69399999999999995,"15min":0.71399999999999997},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81899999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.81699999999999995,"5min":1.012,"15min":1.173},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.70099999999999996,"15min":0.70999999999999996},"min":{"1min":0.48699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84399999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.432},"max":{"1min":0.86299999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.56000000000000005,"5min":0.432,"15min":0.432},"max":{"1min":0.84899999999999998,"5min":1.0600000000000001,"15min":1.212},"last":0.73899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575002,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36864,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334976,"statfs":{"total":96636764160,"available":96599015424,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":1.042},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88500000000000001,"5min":0.92000000000000004,"15min":0.998},"last":0.82299999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.68100000000000005,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.68500000000000005},"min":{"1min":0.52600000000000002,"5min":0.34999999999999998,"15min":0.33100000000000002},"max":{"1min":0.90300000000000002,"5min":1.0129999999999999,"15min":8.9930000000000003},"last":0.621}]},{"osd":2,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96199999999999997},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.55100000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.86799999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.59199999999999997,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.91300000000000003,"5min":0.96499999999999997,"15min":0.96599999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.70699999999999996},"min":{"1min":0.621,"5min":0.48399999999999999,"15min":0.47899999999999998},"max":{"1min":0.93899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70699999999999996,"15min":0.68000000000000005},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88300000000000001,"5min":1.012,"15min":1.012},"last":0.86199999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72699999999999998,"15min":0.70999999999999996},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.42299999999999999},"max":{"1min":0.86499999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.749}]},{"osd":7,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.74399999999999999,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.41199999999999998},"max":{"1min":0.93799999999999994,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.84799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:33:25.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:33:25.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.623+0000 7efc3ffff700 1 Processor -- start 2023-07-13T22:33:25.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.624+0000 7efc3ffff700 1 -- start start 2023-07-13T22:33:25.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.625+0000 7efc3ffff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc380b2230 0x7efc380b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.625+0000 7efc3ffff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3808d030 con 0x7efc380d07d0 2023-07-13T22:33:25.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.625+0000 7efc3ffff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc380af7f0 con 0x7efc380b1ce0 2023-07-13T22:33:25.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.625+0000 7efc3ffff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3815ca30 con 0x7efc380b2230 2023-07-13T22:33:25.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.625+0000 7efc3dd9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efc380d07d0 0x7efc380d0ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35654/0 (socket says 172.21.15.60:35654) 2023-07-13T22:33:25.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.625+0000 7efc3dd9b700 1 -- 172.21.15.60:0/2074148920 learned_addr learned my addr 172.21.15.60:0/2074148920 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:25.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.625+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478816342 0 0) 0x7efc3808d030 con 0x7efc380d07d0 2023-07-13T22:33:25.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc240035b0 con 0x7efc380d07d0 2023-07-13T22:33:25.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3d59a700 1 --2- 172.21.15.60:0/2074148920 >> v2:172.21.15.105:3300/0 conn(0x7efc380b2230 0x7efc380b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3343651263 0 0) 0x7efc240035b0 con 0x7efc380d07d0 2023-07-13T22:33:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 >> v2:172.21.15.105:3300/0 conn(0x7efc380b2230 msgr2=0x7efc380b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 --2- 172.21.15.60:0/2074148920 >> v2:172.21.15.105:3300/0 conn(0x7efc380b2230 0x7efc380b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 >> v1:172.21.15.105:6789/0 conn(0x7efc380b1ce0 legacy=0x7efc380b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3806bdc0 con 0x7efc380d07d0 2023-07-13T22:33:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7efc34001670 con 0x7efc380d07d0 2023-07-13T22:33:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 >> v1:172.21.15.60:6789/0 conn(0x7efc380d07d0 legacy=0x7efc380d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc24003a50 0x7efc24003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 0x7efc24004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc24005200 0x7efc2400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3815ca30 con 0x7efc24005200 2023-07-13T22:33:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2400c0e0 con 0x7efc24004850 2023-07-13T22:33:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2400c2a0 con 0x7efc24003a50 2023-07-13T22:33:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3dd9b700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc24003a50 0x7efc24003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3d59a700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 0x7efc24004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.626+0000 7efc3e59c700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc24005200 0x7efc2400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.627+0000 7efc3d59a700 1 -- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc24003a50 msgr2=0x7efc24003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.627+0000 7efc3d59a700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc24003a50 0x7efc24003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.627+0000 7efc3d59a700 1 -- 172.21.15.60:0/2074148920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc24005200 msgr2=0x7efc2400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.627+0000 7efc3d59a700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc24005200 0x7efc2400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.627+0000 7efc3d59a700 1 -- 172.21.15.60:0/2074148920 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc240035b0 con 0x7efc24004850 2023-07-13T22:33:25.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.627+0000 7efc3e59c700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc24005200 0x7efc2400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:25.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.627+0000 7efc3e59c700 1 -- 172.21.15.60:0/2074148920 reap_dead start 2023-07-13T22:33:25.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.627+0000 7efc3d59a700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 0x7efc24004cc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efc28009b60 tx=0x7efc24005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b6d8438b87c73c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:25.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.627+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800e6c0 con 0x7efc24004850 2023-07-13T22:33:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.628+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2800e820 con 0x7efc24004850 2023-07-13T22:33:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.628+0000 7efc3cd99700 1 -- 172.21.15.60:0/2074148920 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800eb90 con 0x7efc24004850 2023-07-13T22:33:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.628+0000 7efc3ffff700 1 -- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 msgr2=0x7efc24004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.628+0000 7efc3ffff700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 0x7efc24004cc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efc28009b60 tx=0x7efc24005880 comp rx=0 tx=0).stop 2023-07-13T22:33:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.628+0000 7efc3ffff700 1 -- 172.21.15.60:0/2074148920 shutdown_connections 2023-07-13T22:33:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.628+0000 7efc3ffff700 1 --2- 172.21.15.60:0/2074148920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 0x7efc24004cc0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.628+0000 7efc3ffff700 1 -- 172.21.15.60:0/2074148920 >> 172.21.15.60:0/2074148920 conn(0x7efc3809bf90 msgr2=0x7efc380d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.628+0000 7efc3ffff700 1 -- 172.21.15.60:0/2074148920 shutdown_connections 2023-07-13T22:33:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.628+0000 7efc3ffff700 1 -- 172.21.15.60:0/2074148920 wait complete. 2023-07-13T22:33:25.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.630+0000 7efc3ffff700 1 Processor -- start 2023-07-13T22:33:25.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.630+0000 7efc3ffff700 1 -- start start 2023-07-13T22:33:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3ffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 0x7efc3814e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc380b7cf0 0x7efc380b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3ffff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc380b48a0 0x7efc380b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc381f6bc0 con 0x7efc380b7cf0 2023-07-13T22:33:25.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3ffff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc381e8320 con 0x7efc24004850 2023-07-13T22:33:25.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3ffff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc380b5250 con 0x7efc380b48a0 2023-07-13T22:33:25.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3d59a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc380b7cf0 0x7efc380b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3d59a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc380b7cf0 0x7efc380b8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50442/0 (socket says 172.21.15.60:50442) 2023-07-13T22:33:25.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3d59a700 1 -- 172.21.15.60:0/3528316426 learned_addr learned my addr 172.21.15.60:0/3528316426 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:25.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.631+0000 7efc3dd9b700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 0x7efc3814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc3e59c700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc380b48a0 0x7efc380b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc3d59a700 1 -- 172.21.15.60:0/3528316426 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc380b48a0 msgr2=0x7efc380b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc3d59a700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc380b48a0 0x7efc380b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc3d59a700 1 -- 172.21.15.60:0/3528316426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 msgr2=0x7efc3814e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc3d59a700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 0x7efc3814e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc3d59a700 1 -- 172.21.15.60:0/3528316426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc380b87b0 con 0x7efc380b7cf0 2023-07-13T22:33:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc3d59a700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc380b7cf0 0x7efc380b8160 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c270 tx=0x7efc280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc2e7fc700 1 -- 172.21.15.60:0/3528316426 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800e6c0 con 0x7efc380b7cf0 2023-07-13T22:33:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc2e7fc700 1 -- 172.21.15.60:0/3528316426 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc28003ad0 con 0x7efc380b7cf0 2023-07-13T22:33:25.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc381f6220 con 0x7efc380b7cf0 2023-07-13T22:33:25.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc2e7fc700 1 -- 172.21.15.60:0/3528316426 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28003e80 con 0x7efc380b7cf0 2023-07-13T22:33:25.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.632+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc381f67e0 con 0x7efc380b7cf0 2023-07-13T22:33:25.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.633+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc380741c0 con 0x7efc380b7cf0 2023-07-13T22:33:25.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.634+0000 7efc2e7fc700 1 -- 172.21.15.60:0/3528316426 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc28013470 con 0x7efc380b7cf0 2023-07-13T22:33:25.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.635+0000 7efc2e7fc700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc1c06ace0 0x7efc1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:25.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.635+0000 7efc2e7fc700 1 -- 172.21.15.60:0/3528316426 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7efc2808a9a0 con 0x7efc380b7cf0 2023-07-13T22:33:25.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.636+0000 7efc3dd9b700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc1c06ace0 0x7efc1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:25.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.636+0000 7efc3dd9b700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc1c06ace0 0x7efc1c06d190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7efc34001050 tx=0x7efc34000fc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:25.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.639+0000 7efc2e7fc700 1 -- 172.21.15.60:0/3528316426 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7efc28055d60 con 0x7efc380b7cf0 2023-07-13T22:33:25.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.847+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc380b2940 con 0x7efc380b7cf0 2023-07-13T22:33:25.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.848+0000 7efc2e7fc700 1 -- 172.21.15.60:0/3528316426 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+14684 (secure 0 0 0) 0x7efc28059380 con 0x7efc380b7cf0 2023-07-13T22:33:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:25.834 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":140,"fsid":"fbfa21f9-fc93-4234-a1a2-d7b5ad2dc0ce","created":"2023-07-13T22:20:33.608933+0000","modified":"2023-07-13T22:33:20.824296+0000","last_up_change":"2023-07-13T22:20:45.703751+0000","last_in_change":"2023-07-13T22:20:37.148188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-13T22:20:46.388037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-13T22:20:47.173552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b74ca5bc-389c-43b7-8f76-69cfa5793530","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6802","nonce":1806898910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6806","nonce":1806898910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6814","nonce":1806898910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1806898910},{"type":"v1","addr":"172.21.15.60:6810","nonce":1806898910}]},"public_addr":"172.21.15.60:6802/1806898910","cluster_addr":"172.21.15.60:6806/1806898910","heartbeat_back_addr":"172.21.15.60:6814/1806898910","heartbeat_front_addr":"172.21.15.60:6810/1806898910","state":["exists","up"]},{"osd":1,"uuid":"e3f29053-af1f-4c50-92f9-fd2eb879967b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6817","nonce":4106681594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6821","nonce":4106681594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6829","nonce":4106681594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":4106681594},{"type":"v1","addr":"172.21.15.60:6825","nonce":4106681594}]},"public_addr":"172.21.15.60:6817/4106681594","cluster_addr":"172.21.15.60:6821/4106681594","heartbeat_back_addr":"172.21.15.60:6829/4106681594","heartbeat_front_addr":"172.21.15.60:6825/4106681594","state":["exists","up"]},{"osd":2,"uuid":"d04e4f57-6185-4898-8417-14b13b6707be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6803","nonce":4022134141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6807","nonce":4022134141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6813","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6815","nonce":4022134141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":4022134141},{"type":"v1","addr":"172.21.15.60:6811","nonce":4022134141}]},"public_addr":"172.21.15.60:6803/4022134141","cluster_addr":"172.21.15.60:6807/4022134141","heartbeat_back_addr":"172.21.15.60:6815/4022134141","heartbeat_front_addr":"172.21.15.60:6811/4022134141","state":["exists","up"]},{"osd":3,"uuid":"d56053c2-fa6e-465f-944f-7b7c268b6d96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6820","nonce":3719659161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6824","nonce":3719659161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6831","nonce":3719659161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3719659161},{"type":"v1","addr":"172.21.15.60:6828","nonce":3719659161}]},"public_addr":"172.21.15.60:6820/3719659161","cluster_addr":"172.21.15.60:6824/3719659161","heartbeat_back_addr":"172.21.15.60:6831/3719659161","heartbeat_front_addr":"172.21.15.60:6828/3719659161","state":["exists","up"]},{"osd":4,"uuid":"63d94abf-4298-48ad-987c-77af756648eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6809","nonce":922467711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6811","nonce":922467711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6815","nonce":922467711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":922467711},{"type":"v1","addr":"172.21.15.105:6813","nonce":922467711}]},"public_addr":"172.21.15.105:6809/922467711","cluster_addr":"172.21.15.105:6811/922467711","heartbeat_back_addr":"172.21.15.105:6815/922467711","heartbeat_front_addr":"172.21.15.105:6813/922467711","state":["exists","up"]},{"osd":5,"uuid":"e3017c6e-22fe-4dc0-83b5-1bfa67440281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6817","nonce":584130633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6819","nonce":584130633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6823","nonce":584130633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":584130633},{"type":"v1","addr":"172.21.15.105:6821","nonce":584130633}]},"public_addr":"172.21.15.105:6817/584130633","cluster_addr":"172.21.15.105:6819/584130633","heartbeat_back_addr":"172.21.15.105:6823/584130633","heartbeat_front_addr":"172.21.15.105:6821/584130633","state":["exists","up"]},{"osd":6,"uuid":"3650aa71-6ed6-4187-aa47-1b8ab38c7158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6801","nonce":58712571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6803","nonce":58712571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6807","nonce":58712571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":58712571},{"type":"v1","addr":"172.21.15.105:6805","nonce":58712571}]},"public_addr":"172.21.15.105:6801/58712571","cluster_addr":"172.21.15.105:6803/58712571","heartbeat_back_addr":"172.21.15.105:6807/58712571","heartbeat_front_addr":"172.21.15.105:6805/58712571","state":["exists","up"]},{"osd":7,"uuid":"37f5f3b5-f0f5-41f0-88b8-6d9495170cfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6825","nonce":2508216201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6827","nonce":2508216201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6831","nonce":2508216201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2508216201},{"type":"v1","addr":"172.21.15.105:6829","nonce":2508216201}]},"public_addr":"172.21.15.105:6825/2508216201","cluster_addr":"172.21.15.105:6827/2508216201","heartbeat_back_addr":"172.21.15.105:6831/2508216201","heartbeat_front_addr":"172.21.15.105:6829/2508216201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.425954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.550681+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.462951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.738903+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.773232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.480513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.369677+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:20:42.455887+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6837/4061691799":"2023-07-14T22:33:17.609277+0000","172.21.15.60:6835/4061691799":"2023-07-14T22:33:17.609277+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:33:25.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.850+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc1c06ace0 msgr2=0x7efc1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.850+0000 7efc3ffff700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc1c06ace0 0x7efc1c06d190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7efc34001050 tx=0x7efc34000fc0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.850+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc380b7cf0 msgr2=0x7efc380b8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.850+0000 7efc3ffff700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc380b7cf0 0x7efc380b8160 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c270 tx=0x7efc280047c0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.851+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 shutdown_connections 2023-07-13T22:33:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.851+0000 7efc3ffff700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7efc1c06ace0 0x7efc1c06d190 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.851+0000 7efc3ffff700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc380b48a0 0x7efc380b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.851+0000 7efc3ffff700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc24004850 0x7efc3814e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.851+0000 7efc3ffff700 1 --2- 172.21.15.60:0/3528316426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc380b7cf0 0x7efc380b8160 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.851+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 >> 172.21.15.60:0/3528316426 conn(0x7efc3809bf90 msgr2=0x7efc380b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.851+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 shutdown_connections 2023-07-13T22:33:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:25.851+0000 7efc3ffff700 1 -- 172.21.15.60:0/3528316426 wait complete. 2023-07-13T22:33:26.856 INFO:tasks.ceph:Scrubbing osd.0 2023-07-13T22:33:26.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-07-13T22:33:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.089+0000 7f31d23ff700 1 Processor -- start 2023-07-13T22:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.089+0000 7f31d23ff700 1 -- start start 2023-07-13T22:33:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.090+0000 7f31d23ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f31cc15c710 0x7f31cc0af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.090+0000 7f31d23ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f31cc06b420 0x7f31cc0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.090+0000 7f31d23ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31cc15cae0 con 0x7f31cc0d0ca0 2023-07-13T22:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.090+0000 7f31d23ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31cc0d10a0 con 0x7f31cc15c710 2023-07-13T22:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.090+0000 7f31d23ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31cc0affd0 con 0x7f31cc06b420 2023-07-13T22:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.090+0000 7f31cbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f31cc15c710 0x7f31cc0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.091+0000 7f31cbfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f31cc15c710 0x7f31cc0af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50458/0 (socket says 172.21.15.60:50458) 2023-07-13T22:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.091+0000 7f31cbfff700 1 -- 172.21.15.60:0/996025252 learned_addr learned my addr 172.21.15.60:0/996025252 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.091+0000 7f31cb7fe700 1 --2- 172.21.15.60:0/996025252 >> v2:172.21.15.105:3300/0 conn(0x7f31cc06b420 0x7f31cc0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.091+0000 7f31cbfff700 1 -- 172.21.15.60:0/996025252 >> v2:172.21.15.105:3300/0 conn(0x7f31cc06b420 msgr2=0x7f31cc0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.091+0000 7f31cbfff700 1 --2- 172.21.15.60:0/996025252 >> v2:172.21.15.105:3300/0 conn(0x7f31cc06b420 0x7f31cc0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.091+0000 7f31cbfff700 1 -- 172.21.15.60:0/996025252 >> v1:172.21.15.105:6789/0 conn(0x7f31cc0d0ca0 legacy=0x7f31cc06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.091+0000 7f31cbfff700 1 -- 172.21.15.60:0/996025252 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc0b02b0 con 0x7f31cc15c710 2023-07-13T22:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.091+0000 7f31cbfff700 1 --2- 172.21.15.60:0/996025252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc15c710 0x7f31cc0af780 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0c6610 tx=0x7f31bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973f7c9d352104b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31caffd700 1 -- 172.21.15.60:0/996025252 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc011830 con 0x7f31cc15c710 2023-07-13T22:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31caffd700 1 -- 172.21.15.60:0/996025252 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31bc011990 con 0x7f31cc15c710 2023-07-13T22:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31caffd700 1 -- 172.21.15.60:0/996025252 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc011d20 con 0x7f31cc15c710 2023-07-13T22:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31d23ff700 1 -- 172.21.15.60:0/996025252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc15c710 msgr2=0x7f31cc0af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31d23ff700 1 --2- 172.21.15.60:0/996025252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc15c710 0x7f31cc0af780 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0c6610 tx=0x7f31bc009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31d23ff700 1 -- 172.21.15.60:0/996025252 shutdown_connections 2023-07-13T22:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31d23ff700 1 --2- 172.21.15.60:0/996025252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc15c710 0x7f31cc0af780 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31d23ff700 1 --2- 172.21.15.60:0/996025252 >> v2:172.21.15.105:3300/0 conn(0x7f31cc06b420 0x7f31cc0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31d23ff700 1 -- 172.21.15.60:0/996025252 >> 172.21.15.60:0/996025252 conn(0x7f31cc09bf20 msgr2=0x7f31cc0d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31d23ff700 1 -- 172.21.15.60:0/996025252 shutdown_connections 2023-07-13T22:33:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.092+0000 7f31d23ff700 1 -- 172.21.15.60:0/996025252 wait complete. 2023-07-13T22:33:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.094+0000 7f31d23ff700 1 Processor -- start 2023-07-13T22:33:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.094+0000 7f31d23ff700 1 -- start start 2023-07-13T22:33:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.095+0000 7f31d23ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc06b420 0x7f31cc1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.095+0000 7f31d23ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31cc0d0ca0 0x7f31cc152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.095+0000 7f31d23ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31cc15c710 0x7f31cc0d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.095+0000 7f31d23ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1562c0 con 0x7f31cc06b420 2023-07-13T22:33:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.095+0000 7f31d23ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1525d0 con 0x7f31cc15c710 2023-07-13T22:33:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.095+0000 7f31d23ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc155d00 con 0x7f31cc0d0ca0 2023-07-13T22:33:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.095+0000 7f31cbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc06b420 0x7f31cc1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.095+0000 7f31cbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc06b420 0x7f31cc1578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50466/0 (socket says 172.21.15.60:50466) 2023-07-13T22:33:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.095+0000 7f31cbfff700 1 -- 172.21.15.60:0/1398873563 learned_addr learned my addr 172.21.15.60:0/1398873563 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.096+0000 7f31d099c700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31cc15c710 0x7f31cc0d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.096+0000 7f31cb7fe700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31cc0d0ca0 0x7f31cc152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.096+0000 7f31cbfff700 1 -- 172.21.15.60:0/1398873563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31cc0d0ca0 msgr2=0x7f31cc152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.096+0000 7f31cbfff700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31cc0d0ca0 0x7f31cc152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.096+0000 7f31cbfff700 1 -- 172.21.15.60:0/1398873563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31cc15c710 msgr2=0x7f31cc0d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.096+0000 7f31cbfff700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31cc15c710 0x7f31cc0d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.096+0000 7f31cbfff700 1 -- 172.21.15.60:0/1398873563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc009550 con 0x7f31cc06b420 2023-07-13T22:33:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.096+0000 7f31cbfff700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc06b420 0x7f31cc1578e0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009c10 tx=0x7f31bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.096+0000 7f31c8ff9700 1 -- 172.21.15.60:0/1398873563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc004510 con 0x7f31cc06b420 2023-07-13T22:33:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.097+0000 7f31c8ff9700 1 -- 172.21.15.60:0/1398873563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31bc01e070 con 0x7f31cc06b420 2023-07-13T22:33:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.097+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31cc155fe0 con 0x7f31cc06b420 2023-07-13T22:33:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.097+0000 7f31c8ff9700 1 -- 172.21.15.60:0/1398873563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc0038e0 con 0x7f31cc06b420 2023-07-13T22:33:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.097+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc1533c0 con 0x7f31cc06b420 2023-07-13T22:33:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.097+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31cc0b0870 con 0x7f31cc06b420 2023-07-13T22:33:27.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.099+0000 7f31c8ff9700 1 -- 172.21.15.60:0/1398873563 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f31bc003a40 con 0x7f31cc06b420 2023-07-13T22:33:27.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.100+0000 7f31c8ff9700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31b806ace0 0x7f31b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.100+0000 7f31c8ff9700 1 -- 172.21.15.60:0/1398873563 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f31bc08a5b0 con 0x7f31cc06b420 2023-07-13T22:33:27.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.100+0000 7f31cb7fe700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31b806ace0 0x7f31b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.100+0000 7f31c8ff9700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7f31b8071150 0x7f31b8073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.100+0000 7f31c8ff9700 1 -- 172.21.15.60:0/1398873563 --> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31b8073c30 con 0x7f31b8071150 2023-07-13T22:33:27.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.100+0000 7f31c8ff9700 1 -- 172.21.15.60:0/1398873563 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31bc08a920 con 0x7f31cc06b420 2023-07-13T22:33:27.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.100+0000 7f31d099c700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7f31b8071150 0x7f31b8073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.100+0000 7f31cb7fe700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31b806ace0 0x7f31b806d190 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0d20f0 tx=0x7f31c0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.101+0000 7f31d099c700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7f31b8071150 0x7f31b8073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.101+0000 7f31c8ff9700 1 -- 172.21.15.60:0/1398873563 <== osd.0 v2:172.21.15.60:6800/1806898910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f31b8073c30 con 0x7f31b8071150 2023-07-13T22:33:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.116+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 --> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f31cc0b0590 con 0x7f31b8071150 2023-07-13T22:33:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31c8ff9700 1 -- 172.21.15.60:0/1398873563 <== osd.0 v2:172.21.15.60:6800/1806898910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f31c4001fc0 con 0x7f31b8071150 2023-07-13T22:33:27.103 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:33:27.103 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-13T22:33:27.103 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:33:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7f31b8071150 msgr2=0x7f31b8073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6800/1806898910,v1:172.21.15.60:6802/1806898910] conn(0x7f31b8071150 0x7f31b8073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31b806ace0 msgr2=0x7f31b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f31b806ace0 0x7f31b806d190 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0d20f0 tx=0x7f31c0009210 comp rx=0 tx=0).stop 2023-07-13T22:33:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc06b420 msgr2=0x7f31cc1578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 --2- 172.21.15.60:0/1398873563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31cc06b420 0x7f31cc1578e0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009c10 tx=0x7f31bc004750 comp rx=0 tx=0).stop 2023-07-13T22:33:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d099c700 1 -- 172.21.15.60:0/1398873563 reap_dead start 2023-07-13T22:33:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 shutdown_connections 2023-07-13T22:33:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 >> 172.21.15.60:0/1398873563 conn(0x7f31cc09bf20 msgr2=0x7f31cc1e7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 shutdown_connections 2023-07-13T22:33:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.117+0000 7f31d23ff700 1 -- 172.21.15.60:0/1398873563 wait complete. 2023-07-13T22:33:27.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-07-13T22:33:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.338+0000 7f87fd3ff700 1 Processor -- start 2023-07-13T22:33:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.338+0000 7f87fd3ff700 1 -- start start 2023-07-13T22:33:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.339+0000 7f87fd3ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87f80d07d0 0x7f87f80d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.339+0000 7f87fd3ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f808d030 con 0x7f87f80b2230 2023-07-13T22:33:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.339+0000 7f87fd3ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f806b1c0 con 0x7f87f80b1580 2023-07-13T22:33:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.339+0000 7f87fd3ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f80af7f0 con 0x7f87f80d07d0 2023-07-13T22:33:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f67fc700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f87f80b2230 0x7f87f806a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35662/0 (socket says 172.21.15.60:35662) 2023-07-13T22:33:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f67fc700 1 -- 172.21.15.60:0/249502654 learned_addr learned my addr 172.21.15.60:0/249502654 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f6ffd700 1 --2- 172.21.15.60:0/249502654 >> v2:172.21.15.105:3300/0 conn(0x7f87f80d07d0 0x7f87f80d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f5ffb700 1 -- 172.21.15.60:0/249502654 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417711319 0 0) 0x7f87f808d030 con 0x7f87f80b2230 2023-07-13T22:33:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f5ffb700 1 -- 172.21.15.60:0/249502654 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e00035b0 con 0x7f87f80b2230 2023-07-13T22:33:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f5ffb700 1 -- 172.21.15.60:0/249502654 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1372409455 0 0) 0x7f87f806b1c0 con 0x7f87f80b1580 2023-07-13T22:33:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f5ffb700 1 -- 172.21.15.60:0/249502654 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f808d030 con 0x7f87f80b1580 2023-07-13T22:33:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f6ffd700 1 -- 172.21.15.60:0/249502654 >> v1:172.21.15.105:6789/0 conn(0x7f87f80b1580 legacy=0x7f87f806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f6ffd700 1 -- 172.21.15.60:0/249502654 >> v1:172.21.15.60:6789/0 conn(0x7f87f80b2230 legacy=0x7f87f806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.340+0000 7f87f6ffd700 1 -- 172.21.15.60:0/249502654 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f80d10e0 con 0x7f87f80d07d0 2023-07-13T22:33:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.341+0000 7f87f6ffd700 1 --2- 172.21.15.60:0/249502654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80d07d0 0x7f87f80d0ba0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009840 tx=0x7f87e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c5dd580b796f34 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.341+0000 7f87f5ffb700 1 -- 172.21.15.60:0/249502654 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8011830 con 0x7f87f80d07d0 2023-07-13T22:33:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.341+0000 7f87f5ffb700 1 -- 172.21.15.60:0/249502654 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e8011990 con 0x7f87f80d07d0 2023-07-13T22:33:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.341+0000 7f87f5ffb700 1 -- 172.21.15.60:0/249502654 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8011d20 con 0x7f87f80d07d0 2023-07-13T22:33:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.341+0000 7f87fd3ff700 1 -- 172.21.15.60:0/249502654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80d07d0 msgr2=0x7f87f80d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.341+0000 7f87fd3ff700 1 --2- 172.21.15.60:0/249502654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80d07d0 0x7f87f80d0ba0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009840 tx=0x7f87e8009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.342+0000 7f87fd3ff700 1 -- 172.21.15.60:0/249502654 shutdown_connections 2023-07-13T22:33:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.342+0000 7f87fd3ff700 1 --2- 172.21.15.60:0/249502654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80d07d0 0x7f87f80d0ba0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.342+0000 7f87fd3ff700 1 -- 172.21.15.60:0/249502654 >> 172.21.15.60:0/249502654 conn(0x7f87f809bf90 msgr2=0x7f87f80d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.342+0000 7f87fd3ff700 1 -- 172.21.15.60:0/249502654 shutdown_connections 2023-07-13T22:33:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.342+0000 7f87fd3ff700 1 -- 172.21.15.60:0/249502654 wait complete. 2023-07-13T22:33:27.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.343+0000 7f87fd3ff700 1 Processor -- start 2023-07-13T22:33:27.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.344+0000 7f87fd3ff700 1 -- start start 2023-07-13T22:33:27.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.344+0000 7f87fd3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87f80b1580 0x7f87f815dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.344+0000 7f87fd3ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80b2230 0x7f87f80cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87fd3ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87f80d07d0 0x7f87f80cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87fd3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80ce490 con 0x7f87f80b1580 2023-07-13T22:33:27.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87fd3ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80ceb30 con 0x7f87f80b2230 2023-07-13T22:33:27.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87fd3ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80cded0 con 0x7f87f80d07d0 2023-07-13T22:33:27.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87f80b1580 0x7f87f815dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f6ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87f80b1580 0x7f87f815dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50478/0 (socket says 172.21.15.60:50478) 2023-07-13T22:33:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f6ffd700 1 -- 172.21.15.60:0/3095204073 learned_addr learned my addr 172.21.15.60:0/3095204073 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f67fc700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80b2230 0x7f87f80cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f77fe700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87f80d07d0 0x7f87f80cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f6ffd700 1 -- 172.21.15.60:0/3095204073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87f80d07d0 msgr2=0x7f87f80cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f6ffd700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87f80d07d0 0x7f87f80cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f6ffd700 1 -- 172.21.15.60:0/3095204073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80b2230 msgr2=0x7f87f80cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f6ffd700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80b2230 0x7f87f80cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.345+0000 7f87f6ffd700 1 -- 172.21.15.60:0/3095204073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e80094a0 con 0x7f87f80b1580 2023-07-13T22:33:27.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.346+0000 7f87f6ffd700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87f80b1580 0x7f87f815dea0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f87e8014040 tx=0x7f87e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.346+0000 7f87df7fe700 1 -- 172.21.15.60:0/3095204073 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e801a760 con 0x7f87f80b1580 2023-07-13T22:33:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.346+0000 7f87df7fe700 1 -- 172.21.15.60:0/3095204073 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e801a8c0 con 0x7f87f80b1580 2023-07-13T22:33:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.346+0000 7f87df7fe700 1 -- 172.21.15.60:0/3095204073 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e801ac30 con 0x7f87f80b1580 2023-07-13T22:33:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.346+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f80ce1b0 con 0x7f87f80b1580 2023-07-13T22:33:27.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.346+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f80b5a30 con 0x7f87f80b1580 2023-07-13T22:33:27.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.347+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f80ce620 con 0x7f87f80b1580 2023-07-13T22:33:27.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.348+0000 7f87df7fe700 1 -- 172.21.15.60:0/3095204073 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87e8022070 con 0x7f87f80b1580 2023-07-13T22:33:27.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.349+0000 7f87df7fe700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87e406ace0 0x7f87e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.349+0000 7f87df7fe700 1 -- 172.21.15.60:0/3095204073 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f87e8089d60 con 0x7f87f80b1580 2023-07-13T22:33:27.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.349+0000 7f87f67fc700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87e406ace0 0x7f87e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.350+0000 7f87f67fc700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87e406ace0 0x7f87e406d190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f87f80c9c40 tx=0x7f87ec007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.353+0000 7f87df7fe700 1 -- 172.21.15.60:0/3095204073 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f87f80ce620 con 0x7f87f80b1580 2023-07-13T22:33:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.559+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f87f806bdc0 con 0x7f87e406ace0 2023-07-13T22:33:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.559+0000 7f87df7fe700 1 -- 172.21.15.60:0/3095204073 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f87ec0058e0 con 0x7f87e406ace0 2023-07-13T22:33:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87e406ace0 msgr2=0x7f87e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87e406ace0 0x7f87e406d190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f87f80c9c40 tx=0x7f87ec007e00 comp rx=0 tx=0).stop 2023-07-13T22:33:27.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87f80b1580 msgr2=0x7f87f815dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87f80b1580 0x7f87f815dea0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f87e8014040 tx=0x7f87e8004750 comp rx=0 tx=0).stop 2023-07-13T22:33:27.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 shutdown_connections 2023-07-13T22:33:27.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f87e406ace0 0x7f87e406d190 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87f80b1580 0x7f87f815dea0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87f80d07d0 0x7f87f80cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 --2- 172.21.15.60:0/3095204073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80b2230 0x7f87f80cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 >> 172.21.15.60:0/3095204073 conn(0x7f87f809bf90 msgr2=0x7f87f80b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 shutdown_connections 2023-07-13T22:33:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.562+0000 7f87fd3ff700 1 -- 172.21.15.60:0/3095204073 wait complete. 2023-07-13T22:33:27.553 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 0 to deep-scrub 2023-07-13T22:33:27.565 INFO:tasks.ceph:Scrubbing osd.1 2023-07-13T22:33:27.566 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-07-13T22:33:27.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.787+0000 7f721e1ff700 1 Processor -- start 2023-07-13T22:33:27.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.787+0000 7f721e1ff700 1 -- start start 2023-07-13T22:33:27.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.788+0000 7f721e1ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f721815c710 0x7f72180af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.788+0000 7f721e1ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f721806b420 0x7f72180d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.788+0000 7f721e1ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721815cae0 con 0x7f72180d0ca0 2023-07-13T22:33:27.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.788+0000 7f721e1ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72180d10a0 con 0x7f721806b420 2023-07-13T22:33:27.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.788+0000 7f721e1ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72180affd0 con 0x7f721815c710 2023-07-13T22:33:27.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.788+0000 7f7216ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f721806b420 0x7f72180d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.788+0000 7f7216ffd700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f721806b420 0x7f72180d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50484/0 (socket says 172.21.15.60:50484) 2023-07-13T22:33:27.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.788+0000 7f7216ffd700 1 -- 172.21.15.60:0/2286672703 learned_addr learned my addr 172.21.15.60:0/2286672703 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:27.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f72177fe700 1 --2- 172.21.15.60:0/2286672703 >> v2:172.21.15.105:3300/0 conn(0x7f721815c710 0x7f72180af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f7216ffd700 1 -- 172.21.15.60:0/2286672703 >> v2:172.21.15.105:3300/0 conn(0x7f721815c710 msgr2=0x7f72180af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:27.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f7216ffd700 1 --2- 172.21.15.60:0/2286672703 >> v2:172.21.15.105:3300/0 conn(0x7f721815c710 0x7f72180af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f7216ffd700 1 -- 172.21.15.60:0/2286672703 >> v1:172.21.15.105:6789/0 conn(0x7f72180d0ca0 legacy=0x7f721806a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:27.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f7216ffd700 1 -- 172.21.15.60:0/2286672703 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180b02b0 con 0x7f721806b420 2023-07-13T22:33:27.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f72177fe700 1 --2- 172.21.15.60:0/2286672703 >> v2:172.21.15.105:3300/0 conn(0x7f721815c710 0x7f72180af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:27.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f7216ffd700 1 --2- 172.21.15.60:0/2286672703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 0x7f72180d0760 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f72180737a0 tx=0x7f720c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81400886a0e19a80 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f72167fc700 1 -- 172.21.15.60:0/2286672703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c004230 con 0x7f721806b420 2023-07-13T22:33:27.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f72167fc700 1 -- 172.21.15.60:0/2286672703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c004390 con 0x7f721806b420 2023-07-13T22:33:27.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.789+0000 7f72167fc700 1 -- 172.21.15.60:0/2286672703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c0036a0 con 0x7f721806b420 2023-07-13T22:33:27.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.790+0000 7f721e1ff700 1 -- 172.21.15.60:0/2286672703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 msgr2=0x7f72180d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.790+0000 7f721e1ff700 1 --2- 172.21.15.60:0/2286672703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 0x7f72180d0760 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f72180737a0 tx=0x7f720c009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.790+0000 7f721e1ff700 1 -- 172.21.15.60:0/2286672703 shutdown_connections 2023-07-13T22:33:27.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.790+0000 7f721e1ff700 1 --2- 172.21.15.60:0/2286672703 >> v2:172.21.15.105:3300/0 conn(0x7f721815c710 0x7f72180af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.790+0000 7f721e1ff700 1 --2- 172.21.15.60:0/2286672703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 0x7f72180d0760 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.790+0000 7f721e1ff700 1 -- 172.21.15.60:0/2286672703 >> 172.21.15.60:0/2286672703 conn(0x7f721809bf20 msgr2=0x7f72180d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.790+0000 7f721e1ff700 1 -- 172.21.15.60:0/2286672703 shutdown_connections 2023-07-13T22:33:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.790+0000 7f721e1ff700 1 -- 172.21.15.60:0/2286672703 wait complete. 2023-07-13T22:33:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.792+0000 7f721e1ff700 1 Processor -- start 2023-07-13T22:33:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.792+0000 7f721e1ff700 1 -- start start 2023-07-13T22:33:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f721e1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 0x7f72181f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f721e1ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72180d0ca0 0x7f7218156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f721e1ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f721815c710 0x7f7218156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f721e1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72180c7e00 con 0x7f721806b420 2023-07-13T22:33:27.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f721e1ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7218157480 con 0x7f721815c710 2023-07-13T22:33:27.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f721e1ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7218157760 con 0x7f72180d0ca0 2023-07-13T22:33:27.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f72177fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 0x7f72181f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f72177fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 0x7f72181f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50492/0 (socket says 172.21.15.60:50492) 2023-07-13T22:33:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f72177fe700 1 -- 172.21.15.60:0/2658770643 learned_addr learned my addr 172.21.15.60:0/2658770643 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f72177fe700 1 -- 172.21.15.60:0/2658770643 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72180d0ca0 msgr2=0x7f7218156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f72177fe700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72180d0ca0 0x7f7218156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f72177fe700 1 -- 172.21.15.60:0/2658770643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f721815c710 msgr2=0x7f7218156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f7217fff700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f721815c710 0x7f7218156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f72177fe700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f721815c710 0x7f7218156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f72177fe700 1 -- 172.21.15.60:0/2658770643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c009550 con 0x7f721806b420 2023-07-13T22:33:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.793+0000 7f7217fff700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f721815c710 0x7f7218156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.794+0000 7f72177fe700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 0x7f72181f4f80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f720800c7f0 tx=0x7f720800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.794+0000 7f71fffff700 1 -- 172.21.15.60:0/2658770643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7208011800 con 0x7f721806b420 2023-07-13T22:33:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.794+0000 7f71fffff700 1 -- 172.21.15.60:0/2658770643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7208011960 con 0x7f721806b420 2023-07-13T22:33:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.794+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7218157a40 con 0x7f721806b420 2023-07-13T22:33:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.794+0000 7f71fffff700 1 -- 172.21.15.60:0/2658770643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7208011d10 con 0x7f721806b420 2023-07-13T22:33:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.794+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7218158000 con 0x7f721806b420 2023-07-13T22:33:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.795+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72180b0870 con 0x7f721806b420 2023-07-13T22:33:27.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.796+0000 7f71fffff700 1 -- 172.21.15.60:0/2658770643 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7208018360 con 0x7f721806b420 2023-07-13T22:33:27.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.797+0000 7f71fffff700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f720406ace0 0x7f720406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.797+0000 7f71fffff700 1 -- 172.21.15.60:0/2658770643 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f72080891a0 con 0x7f721806b420 2023-07-13T22:33:27.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.797+0000 7f7216ffd700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f720406ace0 0x7f720406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.797+0000 7f71fffff700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f7204071150 0x7f7204073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:27.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.797+0000 7f71fffff700 1 -- 172.21.15.60:0/2658770643 --> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7204073c30 con 0x7f7204071150 2023-07-13T22:33:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.797+0000 7f71fffff700 1 -- 172.21.15.60:0/2658770643 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f720808b780 con 0x7f721806b420 2023-07-13T22:33:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.798+0000 7f7217fff700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f7204071150 0x7f7204073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.798+0000 7f7216ffd700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f720406ace0 0x7f720406d190 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f72180d04c0 tx=0x7f720c010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.798+0000 7f7217fff700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f7204071150 0x7f7204073560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.799+0000 7f71fffff700 1 -- 172.21.15.60:0/2658770643 <== osd.1 v2:172.21.15.60:6816/4106681594 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7204073c30 con 0x7f7204071150 2023-07-13T22:33:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.814+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 --> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f72181578d0 con 0x7f7204071150 2023-07-13T22:33:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.814+0000 7f71fffff700 1 -- 172.21.15.60:0/2658770643 <== osd.1 v2:172.21.15.60:6816/4106681594 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f72181578d0 con 0x7f7204071150 2023-07-13T22:33:27.800 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:33:27.801 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-13T22:33:27.801 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:33:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.814+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f7204071150 msgr2=0x7f7204073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.814+0000 7f721e1ff700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6816/4106681594,v1:172.21.15.60:6817/4106681594] conn(0x7f7204071150 0x7f7204073560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.815+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f720406ace0 msgr2=0x7f720406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.815+0000 7f721e1ff700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f720406ace0 0x7f720406d190 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f72180d04c0 tx=0x7f720c010000 comp rx=0 tx=0).stop 2023-07-13T22:33:27.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.815+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 msgr2=0x7f72181f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:27.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.815+0000 7f721e1ff700 1 --2- 172.21.15.60:0/2658770643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f721806b420 0x7f72181f4f80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f720800c7f0 tx=0x7f720800cbb0 comp rx=0 tx=0).stop 2023-07-13T22:33:27.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.815+0000 7f7217fff700 1 -- 172.21.15.60:0/2658770643 reap_dead start 2023-07-13T22:33:27.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.815+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 shutdown_connections 2023-07-13T22:33:27.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.815+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 >> 172.21.15.60:0/2658770643 conn(0x7f721809bf20 msgr2=0x7f721815a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:27.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.815+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 shutdown_connections 2023-07-13T22:33:27.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:27.815+0000 7f721e1ff700 1 -- 172.21.15.60:0/2658770643 wait complete. 2023-07-13T22:33:27.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-07-13T22:33:28.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.034+0000 7f040b9ff700 1 Processor -- start 2023-07-13T22:33:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.035+0000 7f040b9ff700 1 -- start start 2023-07-13T22:33:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.036+0000 7f040b9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f04040b2230 0x7f04040b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.036+0000 7f040b9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f04040b1ce0 0x7f04040b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.036+0000 7f040b9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040408d030 con 0x7f04040d07d0 2023-07-13T22:33:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.036+0000 7f040b9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040af7f0 con 0x7f04040b1ce0 2023-07-13T22:33:28.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.036+0000 7f040b9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040415ca30 con 0x7f04040b2230 2023-07-13T22:33:28.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.036+0000 7f0409f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f04040b1ce0 0x7f04040b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.036+0000 7f0409f9c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f04040b1ce0 0x7f04040b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50502/0 (socket says 172.21.15.60:50502) 2023-07-13T22:33:28.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.036+0000 7f0409f9c700 1 -- 172.21.15.60:0/2075349446 learned_addr learned my addr 172.21.15.60:0/2075349446 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:28.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.036+0000 7f0408f9a700 1 --2- 172.21.15.60:0/2075349446 >> v2:172.21.15.105:3300/0 conn(0x7f04040b2230 0x7f04040b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.037+0000 7f0409f9c700 1 -- 172.21.15.60:0/2075349446 >> v2:172.21.15.105:3300/0 conn(0x7f04040b2230 msgr2=0x7f04040b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:28.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.037+0000 7f0409f9c700 1 --2- 172.21.15.60:0/2075349446 >> v2:172.21.15.105:3300/0 conn(0x7f04040b2230 0x7f04040b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.037+0000 7f0409f9c700 1 -- 172.21.15.60:0/2075349446 >> v1:172.21.15.105:6789/0 conn(0x7f04040d07d0 legacy=0x7f04040d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.037+0000 7f0409f9c700 1 -- 172.21.15.60:0/2075349446 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040406bdc0 con 0x7f04040b1ce0 2023-07-13T22:33:28.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.037+0000 7f0409f9c700 1 --2- 172.21.15.60:0/2075349446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 0x7f04040b0ab0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f04040d2b30 tx=0x7f03fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ac564a47fc9fa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.037+0000 7f03fbfff700 1 -- 172.21.15.60:0/2075349446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00e070 con 0x7f04040b1ce0 2023-07-13T22:33:28.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.037+0000 7f03fbfff700 1 -- 172.21.15.60:0/2075349446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc00b910 con 0x7f04040b1ce0 2023-07-13T22:33:28.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.037+0000 7f03fbfff700 1 -- 172.21.15.60:0/2075349446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00bcc0 con 0x7f04040b1ce0 2023-07-13T22:33:28.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.038+0000 7f040b9ff700 1 -- 172.21.15.60:0/2075349446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 msgr2=0x7f04040b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.038+0000 7f040b9ff700 1 --2- 172.21.15.60:0/2075349446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 0x7f04040b0ab0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f04040d2b30 tx=0x7f03fc00b330 comp rx=0 tx=0).stop 2023-07-13T22:33:28.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.038+0000 7f040b9ff700 1 -- 172.21.15.60:0/2075349446 shutdown_connections 2023-07-13T22:33:28.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.038+0000 7f040b9ff700 1 --2- 172.21.15.60:0/2075349446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 0x7f04040b0ab0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.038+0000 7f040b9ff700 1 --2- 172.21.15.60:0/2075349446 >> v2:172.21.15.105:3300/0 conn(0x7f04040b2230 0x7f04040b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.038+0000 7f040b9ff700 1 -- 172.21.15.60:0/2075349446 >> 172.21.15.60:0/2075349446 conn(0x7f040409bf90 msgr2=0x7f04040d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:28.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.038+0000 7f040b9ff700 1 -- 172.21.15.60:0/2075349446 shutdown_connections 2023-07-13T22:33:28.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.038+0000 7f040b9ff700 1 -- 172.21.15.60:0/2075349446 wait complete. 2023-07-13T22:33:28.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.040+0000 7f040b9ff700 1 Processor -- start 2023-07-13T22:33:28.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.040+0000 7f040b9ff700 1 -- start start 2023-07-13T22:33:28.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f040b9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 0x7f04040c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f040b9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04040b2230 0x7f04041f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f040b9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04040d07d0 0x7f04041f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f040b9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04040b7410 con 0x7f04040b1ce0 2023-07-13T22:33:28.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f040b9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04041f28e0 con 0x7f04040b2230 2023-07-13T22:33:28.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f040b9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04040ce3e0 con 0x7f04040d07d0 2023-07-13T22:33:28.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f040979b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 0x7f04040c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f040979b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 0x7f04040c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50516/0 (socket says 172.21.15.60:50516) 2023-07-13T22:33:28.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f040979b700 1 -- 172.21.15.60:0/501736373 learned_addr learned my addr 172.21.15.60:0/501736373 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:28.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f0409f9c700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04040d07d0 0x7f04041f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.041+0000 7f0408f9a700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04040b2230 0x7f04041f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f040979b700 1 -- 172.21.15.60:0/501736373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04040d07d0 msgr2=0x7f04041f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f040979b700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04040d07d0 0x7f04041f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f040979b700 1 -- 172.21.15.60:0/501736373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04040b2230 msgr2=0x7f04041f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f040979b700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04040b2230 0x7f04041f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f040979b700 1 -- 172.21.15.60:0/501736373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc009b30 con 0x7f04040b1ce0 2023-07-13T22:33:28.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f040979b700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 0x7f04040c8d00 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f040000c450 tx=0x7f040000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f03f9ffb700 1 -- 172.21.15.60:0/501736373 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04000052d0 con 0x7f04040b1ce0 2023-07-13T22:33:28.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f03f9ffb700 1 -- 172.21.15.60:0/501736373 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0400005430 con 0x7f04040b1ce0 2023-07-13T22:33:28.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04040c9240 con 0x7f04040b1ce0 2023-07-13T22:33:28.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f03f9ffb700 1 -- 172.21.15.60:0/501736373 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04000038c0 con 0x7f04040b1ce0 2023-07-13T22:33:28.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.042+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04040ce640 con 0x7f04040b1ce0 2023-07-13T22:33:28.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.043+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04040741c0 con 0x7f04040b1ce0 2023-07-13T22:33:28.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.044+0000 7f03f9ffb700 1 -- 172.21.15.60:0/501736373 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0400003a20 con 0x7f04040b1ce0 2023-07-13T22:33:28.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.045+0000 7f03f9ffb700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03e806ace0 0x7f03e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.045+0000 7f03f9ffb700 1 -- 172.21.15.60:0/501736373 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0400088470 con 0x7f04040b1ce0 2023-07-13T22:33:28.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.045+0000 7f0408f9a700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03e806ace0 0x7f03e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.046+0000 7f0408f9a700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03e806ace0 0x7f03e806d190 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009b70 tx=0x7f03f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.050+0000 7f03f9ffb700 1 -- 172.21.15.60:0/501736373 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f0400053830 con 0x7f04040b1ce0 2023-07-13T22:33:28.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.254+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f04040afa90 con 0x7f03e806ace0 2023-07-13T22:33:28.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.254+0000 7f03f9ffb700 1 -- 172.21.15.60:0/501736373 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f03f40117b0 con 0x7f03e806ace0 2023-07-13T22:33:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03e806ace0 msgr2=0x7f03e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03e806ace0 0x7f03e806d190 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009b70 tx=0x7f03f4008040 comp rx=0 tx=0).stop 2023-07-13T22:33:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 msgr2=0x7f04040c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 0x7f04040c8d00 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f040000c450 tx=0x7f040000c810 comp rx=0 tx=0).stop 2023-07-13T22:33:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 shutdown_connections 2023-07-13T22:33:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f03e806ace0 0x7f03e806d190 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04040b1ce0 0x7f04040c8d00 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04040d07d0 0x7f04041f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 --2- 172.21.15.60:0/501736373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04040b2230 0x7f04041f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 >> 172.21.15.60:0/501736373 conn(0x7f040409bf90 msgr2=0x7f04040b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 shutdown_connections 2023-07-13T22:33:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.257+0000 7f040b9ff700 1 -- 172.21.15.60:0/501736373 wait complete. 2023-07-13T22:33:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 1 to deep-scrub 2023-07-13T22:33:28.261 INFO:tasks.ceph:Scrubbing osd.2 2023-07-13T22:33:28.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-07-13T22:33:28.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.483+0000 7f34911ff700 1 Processor -- start 2023-07-13T22:33:28.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.483+0000 7f34911ff700 1 -- start start 2023-07-13T22:33:28.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.484+0000 7f34911ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f348c15c710 0x7f348c0af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.484+0000 7f34911ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f348c06b420 0x7f348c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.484+0000 7f34911ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f348c0d0ca0 0x7f348c06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.484+0000 7f34911ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c15cae0 con 0x7f348c06b420 2023-07-13T22:33:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.484+0000 7f34911ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0d10c0 con 0x7f348c0d0ca0 2023-07-13T22:33:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.484+0000 7f34911ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0affd0 con 0x7f348c15c710 2023-07-13T22:33:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.484+0000 7f348a59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f348c06b420 0x7f348c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.484+0000 7f348a59c700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f348c06b420 0x7f348c0d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50524/0 (socket says 172.21.15.60:50524) 2023-07-13T22:33:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.484+0000 7f348a59c700 1 -- 172.21.15.60:0/151328575 learned_addr learned my addr 172.21.15.60:0/151328575 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:28.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f348b59e700 1 --2- 172.21.15.60:0/151328575 >> v2:172.21.15.105:3300/0 conn(0x7f348c0d0ca0 0x7f348c06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f348ad9d700 1 --2- 172.21.15.60:0/151328575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f348c15c710 0x7f348c0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f348a59c700 1 -- 172.21.15.60:0/151328575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f348c15c710 msgr2=0x7f348c0af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:28.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f348a59c700 1 --2- 172.21.15.60:0/151328575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f348c15c710 0x7f348c0af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f348a59c700 1 -- 172.21.15.60:0/151328575 >> v2:172.21.15.105:3300/0 conn(0x7f348c0d0ca0 msgr2=0x7f348c06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f348a59c700 1 --2- 172.21.15.60:0/151328575 >> v2:172.21.15.105:3300/0 conn(0x7f348c0d0ca0 0x7f348c06a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f348a59c700 1 -- 172.21.15.60:0/151328575 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0b02b0 con 0x7f348c06b420 2023-07-13T22:33:28.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f348b59e700 1 --2- 172.21.15.60:0/151328575 >> v2:172.21.15.105:3300/0 conn(0x7f348c0d0ca0 0x7f348c06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:28.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f348a59c700 1 --2- 172.21.15.60:0/151328575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 0x7f348c0d0760 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f348c0c6610 tx=0x7f3480009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd76f659e476b7ac server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f3489d9b700 1 -- 172.21.15.60:0/151328575 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348000b870 con 0x7f348c06b420 2023-07-13T22:33:28.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f3489d9b700 1 -- 172.21.15.60:0/151328575 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348000b9d0 con 0x7f348c06b420 2023-07-13T22:33:28.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.485+0000 7f3489d9b700 1 -- 172.21.15.60:0/151328575 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348000bd60 con 0x7f348c06b420 2023-07-13T22:33:28.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.486+0000 7f34911ff700 1 -- 172.21.15.60:0/151328575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 msgr2=0x7f348c0d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.486+0000 7f34911ff700 1 --2- 172.21.15.60:0/151328575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 0x7f348c0d0760 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f348c0c6610 tx=0x7f3480009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.486+0000 7f34911ff700 1 -- 172.21.15.60:0/151328575 shutdown_connections 2023-07-13T22:33:28.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.486+0000 7f34911ff700 1 --2- 172.21.15.60:0/151328575 >> v2:172.21.15.105:3300/0 conn(0x7f348c0d0ca0 0x7f348c06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.486+0000 7f34911ff700 1 --2- 172.21.15.60:0/151328575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f348c15c710 0x7f348c0af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.486+0000 7f34911ff700 1 --2- 172.21.15.60:0/151328575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 0x7f348c0d0760 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.486+0000 7f34911ff700 1 -- 172.21.15.60:0/151328575 >> 172.21.15.60:0/151328575 conn(0x7f348c09bf20 msgr2=0x7f348c0d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:28.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.486+0000 7f34911ff700 1 -- 172.21.15.60:0/151328575 shutdown_connections 2023-07-13T22:33:28.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.486+0000 7f34911ff700 1 -- 172.21.15.60:0/151328575 wait complete. 2023-07-13T22:33:28.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.488+0000 7f34911ff700 1 Processor -- start 2023-07-13T22:33:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.488+0000 7f34911ff700 1 -- start start 2023-07-13T22:33:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.489+0000 7f34911ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 0x7f348c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.489+0000 7f34911ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f348c0d0ca0 0x7f348c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.489+0000 7f34911ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f348c15c710 0x7f348c0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.489+0000 7f34911ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c1f24c0 con 0x7f348c06b420 2023-07-13T22:33:28.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.489+0000 7f34911ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c1f27a0 con 0x7f348c15c710 2023-07-13T22:33:28.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.489+0000 7f34911ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c1f2a80 con 0x7f348c0d0ca0 2023-07-13T22:33:28.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.489+0000 7f348ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 0x7f348c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.489+0000 7f348ad9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 0x7f348c1533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50536/0 (socket says 172.21.15.60:50536) 2023-07-13T22:33:28.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.489+0000 7f348ad9d700 1 -- 172.21.15.60:0/2392591055 learned_addr learned my addr 172.21.15.60:0/2392591055 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:28.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f348b59e700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f348c15c710 0x7f348c0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f348a59c700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f348c0d0ca0 0x7f348c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f348ad9d700 1 -- 172.21.15.60:0/2392591055 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f348c0d0ca0 msgr2=0x7f348c153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f348ad9d700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f348c0d0ca0 0x7f348c153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f348ad9d700 1 -- 172.21.15.60:0/2392591055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f348c15c710 msgr2=0x7f348c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f348ad9d700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f348c15c710 0x7f348c0c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f348ad9d700 1 -- 172.21.15.60:0/2392591055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480009550 con 0x7f348c06b420 2023-07-13T22:33:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f348ad9d700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 0x7f348c1533c0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f347c00d800 tx=0x7f347c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f34737fe700 1 -- 172.21.15.60:0/2392591055 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0125f0 con 0x7f348c06b420 2023-07-13T22:33:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f34737fe700 1 -- 172.21.15.60:0/2392591055 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c012750 con 0x7f348c06b420 2023-07-13T22:33:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c1f2d60 con 0x7f348c06b420 2023-07-13T22:33:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f34737fe700 1 -- 172.21.15.60:0/2392591055 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c012b00 con 0x7f348c06b420 2023-07-13T22:33:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.490+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c0c93a0 con 0x7f348c06b420 2023-07-13T22:33:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.491+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f348c1f2910 con 0x7f348c06b420 2023-07-13T22:33:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.492+0000 7f34737fe700 1 -- 172.21.15.60:0/2392591055 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f347c012c60 con 0x7f348c06b420 2023-07-13T22:33:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.493+0000 7f34737fe700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f347806ace0 0x7f347806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.493+0000 7f34737fe700 1 -- 172.21.15.60:0/2392591055 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f347c089560 con 0x7f348c06b420 2023-07-13T22:33:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.493+0000 7f348a59c700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f347806ace0 0x7f347806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.493+0000 7f34737fe700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7f3478071150 0x7f3478073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.493+0000 7f34737fe700 1 -- 172.21.15.60:0/2392591055 --> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3478073c30 con 0x7f3478071150 2023-07-13T22:33:28.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.493+0000 7f34737fe700 1 -- 172.21.15.60:0/2392591055 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f347c08bb40 con 0x7f348c06b420 2023-07-13T22:33:28.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.494+0000 7f348b59e700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7f3478071150 0x7f3478073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.494+0000 7f348a59c700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f347806ace0 0x7f347806d190 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3480000c00 tx=0x7f3480008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.494+0000 7f348b59e700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7f3478071150 0x7f3478073560 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.495+0000 7f34737fe700 1 -- 172.21.15.60:0/2392591055 <== osd.2 v2:172.21.15.60:6801/4022134141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3478073c30 con 0x7f3478071150 2023-07-13T22:33:28.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.510+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 --> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f348c0b0870 con 0x7f3478071150 2023-07-13T22:33:28.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.510+0000 7f34737fe700 1 -- 172.21.15.60:0/2392591055 <== osd.2 v2:172.21.15.60:6801/4022134141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3484009ac0 con 0x7f3478071150 2023-07-13T22:33:28.497 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:33:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-13T22:33:28.497 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:33:28.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7f3478071150 msgr2=0x7f3478073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6801/4022134141,v1:172.21.15.60:6803/4022134141] conn(0x7f3478071150 0x7f3478073560 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f347806ace0 msgr2=0x7f347806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f347806ace0 0x7f347806d190 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3480000c00 tx=0x7f3480008030 comp rx=0 tx=0).stop 2023-07-13T22:33:28.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 msgr2=0x7f348c1533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 --2- 172.21.15.60:0/2392591055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f348c06b420 0x7f348c1533c0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f347c00d800 tx=0x7f347c00dbc0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f348b59e700 1 -- 172.21.15.60:0/2392591055 reap_dead start 2023-07-13T22:33:28.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 shutdown_connections 2023-07-13T22:33:28.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 >> 172.21.15.60:0/2392591055 conn(0x7f348c09bf20 msgr2=0x7f348c0b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:28.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 shutdown_connections 2023-07-13T22:33:28.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.511+0000 7f34911ff700 1 -- 172.21.15.60:0/2392591055 wait complete. 2023-07-13T22:33:28.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-07-13T22:33:28.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:33:28.659 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:33:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.736+0000 7fba23dff700 1 Processor -- start 2023-07-13T22:33:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.736+0000 7fba23dff700 1 -- start start 2023-07-13T22:33:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.737+0000 7fba23dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b1ce0 0x7fba1c0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.737+0000 7fba23dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c08d030 con 0x7fba1c0b2230 2023-07-13T22:33:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.737+0000 7fba23dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c0af7f0 con 0x7fba1c0d07d0 2023-07-13T22:33:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.737+0000 7fba23dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c15ca30 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.737+0000 7fba2139a700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fba1c0b2230 0x7fba1c0b0570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35666/0 (socket says 172.21.15.60:35666) 2023-07-13T22:33:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.737+0000 7fba2139a700 1 -- 172.21.15.60:0/459305019 learned_addr learned my addr 172.21.15.60:0/459305019 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.738+0000 7fba2239c700 1 --2- 172.21.15.60:0/459305019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b1ce0 0x7fba1c0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.738+0000 7fba20b99700 1 -- 172.21.15.60:0/459305019 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448616030 0 0) 0x7fba1c08d030 con 0x7fba1c0b2230 2023-07-13T22:33:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.738+0000 7fba20b99700 1 -- 172.21.15.60:0/459305019 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba080035b0 con 0x7fba1c0b2230 2023-07-13T22:33:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.738+0000 7fba20b99700 1 -- 172.21.15.60:0/459305019 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488942383 0 0) 0x7fba1c0af7f0 con 0x7fba1c0d07d0 2023-07-13T22:33:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.738+0000 7fba20b99700 1 -- 172.21.15.60:0/459305019 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba1c08d030 con 0x7fba1c0d07d0 2023-07-13T22:33:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.738+0000 7fba2239c700 1 -- 172.21.15.60:0/459305019 >> v1:172.21.15.105:6789/0 conn(0x7fba1c0d07d0 legacy=0x7fba1c0d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.738+0000 7fba2239c700 1 -- 172.21.15.60:0/459305019 >> v1:172.21.15.60:6789/0 conn(0x7fba1c0b2230 legacy=0x7fba1c0b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.738+0000 7fba2239c700 1 -- 172.21.15.60:0/459305019 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c06bdc0 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.738+0000 7fba2239c700 1 --2- 172.21.15.60:0/459305019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b1ce0 0x7fba1c0b0c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba14009dc0 tx=0x7fba1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1cd243d5edbbc71 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.739+0000 7fba20b99700 1 -- 172.21.15.60:0/459305019 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400e070 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.739+0000 7fba20b99700 1 -- 172.21.15.60:0/459305019 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1400bb20 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.739+0000 7fba20b99700 1 -- 172.21.15.60:0/459305019 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14012430 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.739+0000 7fba23dff700 1 -- 172.21.15.60:0/459305019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b1ce0 msgr2=0x7fba1c0b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.739+0000 7fba23dff700 1 --2- 172.21.15.60:0/459305019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b1ce0 0x7fba1c0b0c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba14009dc0 tx=0x7fba1400b330 comp rx=0 tx=0).stop 2023-07-13T22:33:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.740+0000 7fba23dff700 1 -- 172.21.15.60:0/459305019 shutdown_connections 2023-07-13T22:33:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.740+0000 7fba23dff700 1 --2- 172.21.15.60:0/459305019 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b1ce0 0x7fba1c0b0c80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.740+0000 7fba23dff700 1 -- 172.21.15.60:0/459305019 >> 172.21.15.60:0/459305019 conn(0x7fba1c09bf90 msgr2=0x7fba1c0d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.740+0000 7fba23dff700 1 -- 172.21.15.60:0/459305019 shutdown_connections 2023-07-13T22:33:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.740+0000 7fba23dff700 1 -- 172.21.15.60:0/459305019 wait complete. 2023-07-13T22:33:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.742+0000 7fba23dff700 1 Processor -- start 2023-07-13T22:33:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.742+0000 7fba23dff700 1 -- start start 2023-07-13T22:33:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba23dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba1c0b1ce0 0x7fba1c0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba23dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b2230 0x7fba1c15f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba23dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba1c0d07d0 0x7fba1c15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba23dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0ccd90 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba23dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0c9e60 con 0x7fba1c0d07d0 2023-07-13T22:33:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba23dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c15cea0 con 0x7fba1c0b2230 2023-07-13T22:33:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba21b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba1c0b1ce0 0x7fba1c0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba21b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba1c0b1ce0 0x7fba1c0cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50546/0 (socket says 172.21.15.60:50546) 2023-07-13T22:33:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba21b9b700 1 -- 172.21.15.60:0/519120422 learned_addr learned my addr 172.21.15.60:0/519120422 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba2139a700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b2230 0x7fba1c15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba2239c700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba1c0d07d0 0x7fba1c15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba21b9b700 1 -- 172.21.15.60:0/519120422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b2230 msgr2=0x7fba1c15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba21b9b700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b2230 0x7fba1c15f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba21b9b700 1 -- 172.21.15.60:0/519120422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba1c0d07d0 msgr2=0x7fba1c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba21b9b700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba1c0d07d0 0x7fba1c15e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.743+0000 7fba21b9b700 1 -- 172.21.15.60:0/519120422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba14009a70 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.744+0000 7fba21b9b700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba1c0b1ce0 0x7fba1c0cc850 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fba1800c500 tx=0x7fba1800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.744+0000 7fba127fc700 1 -- 172.21.15.60:0/519120422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba180052d0 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.744+0000 7fba127fc700 1 -- 172.21.15.60:0/519120422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba18005450 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.744+0000 7fba127fc700 1 -- 172.21.15.60:0/519120422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba180038c0 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.744+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c0ca0b0 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.744+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c15fb40 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.745+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c0741c0 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.746+0000 7fba127fc700 1 -- 172.21.15.60:0/519120422 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba18003a20 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.747+0000 7fba127fc700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba0006ace0 0x7fba0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:28.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.747+0000 7fba127fc700 1 -- 172.21.15.60:0/519120422 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fba180885d0 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.747+0000 7fba2139a700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba0006ace0 0x7fba0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:28.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.748+0000 7fba2139a700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba0006ace0 0x7fba0006d190 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0a0b40 tx=0x7fba0c007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:28.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.752+0000 7fba127fc700 1 -- 172.21.15.60:0/519120422 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fba18053900 con 0x7fba1c0b1ce0 2023-07-13T22:33:28.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.958+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fba1c0afa90 con 0x7fba0006ace0 2023-07-13T22:33:28.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.959+0000 7fba127fc700 1 -- 172.21.15.60:0/519120422 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fba0c0058e0 con 0x7fba0006ace0 2023-07-13T22:33:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba0006ace0 msgr2=0x7fba0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba0006ace0 0x7fba0006d190 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0a0b40 tx=0x7fba0c007dc0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba1c0b1ce0 msgr2=0x7fba1c0cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba1c0b1ce0 0x7fba1c0cc850 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fba1800c500 tx=0x7fba1800c810 comp rx=0 tx=0).stop 2023-07-13T22:33:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 shutdown_connections 2023-07-13T22:33:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fba0006ace0 0x7fba0006d190 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba1c0b1ce0 0x7fba1c0cc850 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba1c0d07d0 0x7fba1c15e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 --2- 172.21.15.60:0/519120422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba1c0b2230 0x7fba1c15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 >> 172.21.15.60:0/519120422 conn(0x7fba1c09bf90 msgr2=0x7fba1c157ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 shutdown_connections 2023-07-13T22:33:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:28.961+0000 7fba23dff700 1 -- 172.21.15.60:0/519120422 wait complete. 2023-07-13T22:33:28.949 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 2 to deep-scrub 2023-07-13T22:33:28.965 INFO:tasks.ceph:Scrubbing osd.3 2023-07-13T22:33:28.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-07-13T22:33:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.186+0000 7f8ea63ff700 1 Processor -- start 2023-07-13T22:33:29.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.186+0000 7f8ea63ff700 1 -- start start 2023-07-13T22:33:29.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8ea63ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8ea013f290 0x7f8ea00b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8ea63ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8ea006c2c0 0x7f8ea00b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8ea63ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea00cb350 con 0x7f8ea00cb7e0 2023-07-13T22:33:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8ea63ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea006c8e0 con 0x7f8ea006c2c0 2023-07-13T22:33:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8ea63ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea00caa20 con 0x7f8ea013f290 2023-07-13T22:33:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8e9f7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8ea00cb7e0 0x7f8ea006bbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35668/0 (socket says 172.21.15.60:35668) 2023-07-13T22:33:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8e9f7fe700 1 -- 172.21.15.60:0/698303940 learned_addr learned my addr 172.21.15.60:0/698303940 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8ea499c700 1 --2- 172.21.15.60:0/698303940 >> v2:172.21.15.60:3300/0 conn(0x7f8ea006c2c0 0x7f8ea00b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8e9effd700 1 -- 172.21.15.60:0/698303940 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299955993 0 0) 0x7f8ea00cb350 con 0x7f8ea00cb7e0 2023-07-13T22:33:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8e9effd700 1 -- 172.21.15.60:0/698303940 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e880035b0 con 0x7f8ea00cb7e0 2023-07-13T22:33:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8e9ffff700 1 --2- 172.21.15.60:0/698303940 >> v2:172.21.15.105:3300/0 conn(0x7f8ea013f290 0x7f8ea00b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.188+0000 7f8ea499c700 1 -- 172.21.15.60:0/698303940 >> v2:172.21.15.105:3300/0 conn(0x7f8ea013f290 msgr2=0x7f8ea00b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.189+0000 7f8ea499c700 1 --2- 172.21.15.60:0/698303940 >> v2:172.21.15.105:3300/0 conn(0x7f8ea013f290 0x7f8ea00b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.189+0000 7f8ea499c700 1 -- 172.21.15.60:0/698303940 >> v1:172.21.15.60:6789/0 conn(0x7f8ea00cb7e0 legacy=0x7f8ea006bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.189+0000 7f8ea499c700 1 -- 172.21.15.60:0/698303940 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea00cad00 con 0x7f8ea006c2c0 2023-07-13T22:33:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.189+0000 7f8e9ffff700 1 --2- 172.21.15.60:0/698303940 >> v2:172.21.15.105:3300/0 conn(0x7f8ea013f290 0x7f8ea00b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.189+0000 7f8ea499c700 1 --2- 172.21.15.60:0/698303940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea006c2c0 0x7f8ea00b08a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f8ea006af50 tx=0x7f8e9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7d0659c267e86 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.189+0000 7f8e9effd700 1 -- 172.21.15.60:0/698303940 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800e070 con 0x7f8ea006c2c0 2023-07-13T22:33:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.189+0000 7f8e9effd700 1 -- 172.21.15.60:0/698303940 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800bb20 con 0x7f8ea006c2c0 2023-07-13T22:33:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.189+0000 7f8e9effd700 1 -- 172.21.15.60:0/698303940 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98012430 con 0x7f8ea006c2c0 2023-07-13T22:33:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.190+0000 7f8ea63ff700 1 -- 172.21.15.60:0/698303940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea006c2c0 msgr2=0x7f8ea00b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.190+0000 7f8ea63ff700 1 --2- 172.21.15.60:0/698303940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea006c2c0 0x7f8ea00b08a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f8ea006af50 tx=0x7f8e9800b330 comp rx=0 tx=0).stop 2023-07-13T22:33:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.190+0000 7f8ea63ff700 1 -- 172.21.15.60:0/698303940 shutdown_connections 2023-07-13T22:33:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.190+0000 7f8ea63ff700 1 --2- 172.21.15.60:0/698303940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea006c2c0 0x7f8ea00b08a0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.190+0000 7f8ea63ff700 1 --2- 172.21.15.60:0/698303940 >> v2:172.21.15.105:3300/0 conn(0x7f8ea013f290 0x7f8ea00b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.190+0000 7f8ea63ff700 1 -- 172.21.15.60:0/698303940 >> 172.21.15.60:0/698303940 conn(0x7f8ea0065bd0 msgr2=0x7f8ea0065fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.190+0000 7f8ea63ff700 1 -- 172.21.15.60:0/698303940 shutdown_connections 2023-07-13T22:33:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.190+0000 7f8ea63ff700 1 -- 172.21.15.60:0/698303940 wait complete. 2023-07-13T22:33:29.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.191+0000 7f8ea63ff700 1 Processor -- start 2023-07-13T22:33:29.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.192+0000 7f8ea63ff700 1 -- start start 2023-07-13T22:33:29.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea63ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ea006c2c0 0x7f8ea013a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea63ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea00cb7e0 0x7f8ea0154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea63ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea013f290 0x7f8ea0156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea63ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0155740 con 0x7f8ea013f290 2023-07-13T22:33:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea63ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0154b30 con 0x7f8ea00cb7e0 2023-07-13T22:33:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea63ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0154e10 con 0x7f8ea006c2c0 2023-07-13T22:33:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea499c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea013f290 0x7f8ea0156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea499c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea013f290 0x7f8ea0156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50566/0 (socket says 172.21.15.60:50566) 2023-07-13T22:33:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea499c700 1 -- 172.21.15.60:0/4284845446 learned_addr learned my addr 172.21.15.60:0/4284845446 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8e9f7fe700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea00cb7e0 0x7f8ea0154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8e9ffff700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ea006c2c0 0x7f8ea013a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea499c700 1 -- 172.21.15.60:0/4284845446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ea006c2c0 msgr2=0x7f8ea013a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea499c700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ea006c2c0 0x7f8ea013a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea499c700 1 -- 172.21.15.60:0/4284845446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea00cb7e0 msgr2=0x7f8ea0154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea499c700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea00cb7e0 0x7f8ea0154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8ea499c700 1 -- 172.21.15.60:0/4284845446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e98009a70 con 0x7f8ea013f290 2023-07-13T22:33:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.193+0000 7f8e9f7fe700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ea00cb7e0 0x7f8ea0154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:29.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.194+0000 7f8ea499c700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea013f290 0x7f8ea0156610 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f8e98014040 tx=0x7f8e98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.194+0000 7f8e9cff9700 1 -- 172.21.15.60:0/4284845446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98012430 con 0x7f8ea013f290 2023-07-13T22:33:29.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.194+0000 7f8e9cff9700 1 -- 172.21.15.60:0/4284845446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e98020070 con 0x7f8ea013f290 2023-07-13T22:33:29.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.194+0000 7f8e9cff9700 1 -- 172.21.15.60:0/4284845446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98004280 con 0x7f8ea013f290 2023-07-13T22:33:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.194+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea01550f0 con 0x7f8ea013f290 2023-07-13T22:33:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.194+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea013aef0 con 0x7f8ea013f290 2023-07-13T22:33:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.195+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ea013f660 con 0x7f8ea013f290 2023-07-13T22:33:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.196+0000 7f8e9cff9700 1 -- 172.21.15.60:0/4284845446 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e980043e0 con 0x7f8ea013f290 2023-07-13T22:33:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.197+0000 7f8e9cff9700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e8c06ace0 0x7f8e8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.197+0000 7f8e9cff9700 1 -- 172.21.15.60:0/4284845446 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8e98022070 con 0x7f8ea013f290 2023-07-13T22:33:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.197+0000 7f8e9ffff700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e8c06ace0 0x7f8e8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.197+0000 7f8e9cff9700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f8e8c071150 0x7f8e8c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.197+0000 7f8e9cff9700 1 -- 172.21.15.60:0/4284845446 --> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e8c073c30 con 0x7f8e8c071150 2023-07-13T22:33:29.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.197+0000 7f8e9cff9700 1 -- 172.21.15.60:0/4284845446 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e980902c0 con 0x7f8ea013f290 2023-07-13T22:33:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.197+0000 7f8e9f7fe700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f8e8c071150 0x7f8e8c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.198+0000 7f8e9ffff700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e8c06ace0 0x7f8e8c06d190 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8e90006fd0 tx=0x7f8e90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.198+0000 7f8e9f7fe700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f8e8c071150 0x7f8e8c073560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.199+0000 7f8e9cff9700 1 -- 172.21.15.60:0/4284845446 <== osd.3 v2:172.21.15.60:6818/3719659161 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8e8c073c30 con 0x7f8e8c071150 2023-07-13T22:33:29.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 --> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8ea00cafe0 con 0x7f8e8c071150 2023-07-13T22:33:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8e9cff9700 1 -- 172.21.15.60:0/4284845446 <== osd.3 v2:172.21.15.60:6818/3719659161 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8e94001950 con 0x7f8e8c071150 2023-07-13T22:33:29.200 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:33:29.201 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-13T22:33:29.201 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:33:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f8e8c071150 msgr2=0x7f8e8c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea63ff700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6818/3719659161,v1:172.21.15.60:6820/3719659161] conn(0x7f8e8c071150 0x7f8e8c073560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e8c06ace0 msgr2=0x7f8e8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea63ff700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f8e8c06ace0 0x7f8e8c06d190 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f8e90006fd0 tx=0x7f8e90008040 comp rx=0 tx=0).stop 2023-07-13T22:33:29.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea013f290 msgr2=0x7f8ea0156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea63ff700 1 --2- 172.21.15.60:0/4284845446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ea013f290 0x7f8ea0156610 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f8e98014040 tx=0x7f8e98004ad0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea499c700 1 -- 172.21.15.60:0/4284845446 reap_dead start 2023-07-13T22:33:29.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 shutdown_connections 2023-07-13T22:33:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.214+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 >> 172.21.15.60:0/4284845446 conn(0x7f8ea0065bd0 msgr2=0x7f8ea0159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.215+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 shutdown_connections 2023-07-13T22:33:29.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.215+0000 7f8ea63ff700 1 -- 172.21.15.60:0/4284845446 wait complete. 2023-07-13T22:33:29.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-07-13T22:33:29.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.436+0000 7fc3127ff700 1 Processor -- start 2023-07-13T22:33:29.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.436+0000 7fc3127ff700 1 -- start start 2023-07-13T22:33:29.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.437+0000 7fc3127ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc30c158990 0x7fc30c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc3127ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c0cb9c0 0x7fc30c06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc3127ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30c06bf10 con 0x7fc30c0ca660 2023-07-13T22:33:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc3127ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30c06b950 con 0x7fc30c158990 2023-07-13T22:33:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc3127ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c0cabd0 con 0x7fc30c0cb9c0 2023-07-13T22:33:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc30b7fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc30c0ca660 0x7fc30c0cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35676/0 (socket says 172.21.15.60:35676) 2023-07-13T22:33:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc30b7fe700 1 -- 172.21.15.60:0/2859392761 learned_addr learned my addr 172.21.15.60:0/2859392761 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:29.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc310d9c700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c0cb9c0 0x7fc30c06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc30bfff700 1 --2- 172.21.15.60:0/2859392761 >> v2:172.21.15.105:3300/0 conn(0x7fc30c158990 0x7fc30c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199879938 0 0) 0x7fc30c06bf10 con 0x7fc30c0ca660 2023-07-13T22:33:29.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.438+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f40035b0 con 0x7fc30c0ca660 2023-07-13T22:33:29.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2708560217 0 0) 0x7fc2f40035b0 con 0x7fc30c0ca660 2023-07-13T22:33:29.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c0cb9c0 msgr2=0x7fc30c06d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:29.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c0cb9c0 0x7fc30c06d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 >> v2:172.21.15.105:3300/0 conn(0x7fc30c158990 msgr2=0x7fc30c06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:29.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 --2- 172.21.15.60:0/2859392761 >> v2:172.21.15.105:3300/0 conn(0x7fc30c158990 0x7fc30c06c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c0caeb0 con 0x7fc30c0ca660 2023-07-13T22:33:29.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fc300001670 con 0x7fc30c0ca660 2023-07-13T22:33:29.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 >> v1:172.21.15.60:6789/0 conn(0x7fc30c0ca660 legacy=0x7fc30c0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c0cb9c0 0x7fc2f4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2f4003f80 0x7fc2f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2f4004d90 0x7fc2f4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c06b950 con 0x7fc2f4003f80 2023-07-13T22:33:29.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c0cabd0 con 0x7fc2f4004d90 2023-07-13T22:33:29.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f400ba40 con 0x7fc30c0cb9c0 2023-07-13T22:33:29.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30b7fe700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2f4003f80 0x7fc2f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30bfff700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c0cb9c0 0x7fc2f4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc310d9c700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2f4004d90 0x7fc2f4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30b7fe700 1 -- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c0cb9c0 msgr2=0x7fc2f4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:29.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30b7fe700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c0cb9c0 0x7fc2f4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30b7fe700 1 -- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2f4004d90 msgr2=0x7fc2f4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:29.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30b7fe700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2f4004d90 0x7fc2f4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.439+0000 7fc30b7fe700 1 -- 172.21.15.60:0/2859392761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f40035b0 con 0x7fc2f4003f80 2023-07-13T22:33:29.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.440+0000 7fc30b7fe700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2f4003f80 0x7fc2f4004850 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c240 tx=0x7fc2f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d9f25c8e07c877 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.440+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300007c10 con 0x7fc2f4003f80 2023-07-13T22:33:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.440+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30000ce00 con 0x7fc2f4003f80 2023-07-13T22:33:29.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.440+0000 7fc30affd700 1 -- 172.21.15.60:0/2859392761 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300005ba0 con 0x7fc2f4003f80 2023-07-13T22:33:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.440+0000 7fc3127ff700 1 -- 172.21.15.60:0/2859392761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2f4003f80 msgr2=0x7fc2f4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.440+0000 7fc3127ff700 1 --2- 172.21.15.60:0/2859392761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2f4003f80 0x7fc2f4004850 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c240 tx=0x7fc2f4005880 comp rx=0 tx=0).stop 2023-07-13T22:33:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.440+0000 7fc310d9c700 1 -- 172.21.15.60:0/2859392761 reap_dead start 2023-07-13T22:33:29.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.440+0000 7fc3127ff700 1 -- 172.21.15.60:0/2859392761 shutdown_connections 2023-07-13T22:33:29.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.440+0000 7fc3127ff700 1 -- 172.21.15.60:0/2859392761 >> 172.21.15.60:0/2859392761 conn(0x7fc30c06ab20 msgr2=0x7fc30c0b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:29.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.441+0000 7fc3127ff700 1 -- 172.21.15.60:0/2859392761 shutdown_connections 2023-07-13T22:33:29.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.441+0000 7fc3127ff700 1 -- 172.21.15.60:0/2859392761 wait complete. 2023-07-13T22:33:29.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.442+0000 7fc3127ff700 1 Processor -- start 2023-07-13T22:33:29.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.442+0000 7fc3127ff700 1 -- start start 2023-07-13T22:33:29.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.443+0000 7fc3127ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2f4003f80 0x7fc30c0c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.443+0000 7fc3127ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc30c154000 0x7fc30c0c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.443+0000 7fc3127ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c1527f0 0x7fc30c0c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.443+0000 7fc3127ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c0c5b40 con 0x7fc30c154000 2023-07-13T22:33:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.443+0000 7fc3127ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c153b70 con 0x7fc2f4003f80 2023-07-13T22:33:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.443+0000 7fc3127ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c152f60 con 0x7fc30c1527f0 2023-07-13T22:33:29.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc30c154000 0x7fc30c0c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc30c154000 0x7fc30c0c47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50582/0 (socket says 172.21.15.60:50582) 2023-07-13T22:33:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30b7fe700 1 -- 172.21.15.60:0/361450243 learned_addr learned my addr 172.21.15.60:0/361450243 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30bfff700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2f4003f80 0x7fc30c0c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc310d9c700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c1527f0 0x7fc30c0c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30b7fe700 1 -- 172.21.15.60:0/361450243 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c1527f0 msgr2=0x7fc30c0c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30b7fe700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c1527f0 0x7fc30c0c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30b7fe700 1 -- 172.21.15.60:0/361450243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2f4003f80 msgr2=0x7fc30c0c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30b7fe700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2f4003f80 0x7fc30c0c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30b7fe700 1 -- 172.21.15.60:0/361450243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c153240 con 0x7fc30c154000 2023-07-13T22:33:29.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.444+0000 7fc30b7fe700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc30c154000 0x7fc30c0c47c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4004760 tx=0x7fc300001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.445+0000 7fc308ff9700 1 -- 172.21.15.60:0/361450243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300001630 con 0x7fc30c154000 2023-07-13T22:33:29.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.445+0000 7fc308ff9700 1 -- 172.21.15.60:0/361450243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3000070d0 con 0x7fc30c154000 2023-07-13T22:33:29.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.445+0000 7fc308ff9700 1 -- 172.21.15.60:0/361450243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3000074a0 con 0x7fc30c154000 2023-07-13T22:33:29.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.445+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30c153520 con 0x7fc30c154000 2023-07-13T22:33:29.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.445+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30c152c60 con 0x7fc30c154000 2023-07-13T22:33:29.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.446+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30c0741c0 con 0x7fc30c154000 2023-07-13T22:33:29.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.447+0000 7fc308ff9700 1 -- 172.21.15.60:0/361450243 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc300019030 con 0x7fc30c154000 2023-07-13T22:33:29.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.448+0000 7fc308ff9700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc2f806ace0 0x7fc2f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.448+0000 7fc308ff9700 1 -- 172.21.15.60:0/361450243 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc300089f40 con 0x7fc30c154000 2023-07-13T22:33:29.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.448+0000 7fc30bfff700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc2f806ace0 0x7fc2f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.448+0000 7fc30bfff700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc2f806ace0 0x7fc2f806d190 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009c60 tx=0x7fc2fc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.452+0000 7fc308ff9700 1 -- 172.21.15.60:0/361450243 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fc300055300 con 0x7fc30c154000 2023-07-13T22:33:29.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.657+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc30c158d60 con 0x7fc2f806ace0 2023-07-13T22:33:29.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.658+0000 7fc308ff9700 1 -- 172.21.15.60:0/361450243 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc2fc019070 con 0x7fc2f806ace0 2023-07-13T22:33:29.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc2f806ace0 msgr2=0x7fc2f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc2f806ace0 0x7fc2f806d190 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009c60 tx=0x7fc2fc009210 comp rx=0 tx=0).stop 2023-07-13T22:33:29.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc30c154000 msgr2=0x7fc30c0c47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc30c154000 0x7fc30c0c47c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4004760 tx=0x7fc300001a00 comp rx=0 tx=0).stop 2023-07-13T22:33:29.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 shutdown_connections 2023-07-13T22:33:29.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fc2f806ace0 0x7fc2f806d190 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc30c1527f0 0x7fc30c0c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2f4003f80 0x7fc30c0c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 --2- 172.21.15.60:0/361450243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc30c154000 0x7fc30c0c47c0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 >> 172.21.15.60:0/361450243 conn(0x7fc30c06ab20 msgr2=0x7fc30c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:29.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 shutdown_connections 2023-07-13T22:33:29.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.660+0000 7fc3127ff700 1 -- 172.21.15.60:0/361450243 wait complete. 2023-07-13T22:33:29.649 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 3 to deep-scrub 2023-07-13T22:33:29.663 INFO:tasks.ceph:Scrubbing osd.4 2023-07-13T22:33:29.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-07-13T22:33:29.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.884+0000 7fce287ff700 1 Processor -- start 2023-07-13T22:33:29.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.885+0000 7fce287ff700 1 -- start start 2023-07-13T22:33:29.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.886+0000 7fce287ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce2015c710 0x7fce200af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.886+0000 7fce287ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce2006b420 0x7fce200d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.886+0000 7fce287ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2015cae0 con 0x7fce200d0ca0 2023-07-13T22:33:29.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.886+0000 7fce287ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce200d10c0 con 0x7fce2006b420 2023-07-13T22:33:29.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.886+0000 7fce287ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce200affd0 con 0x7fce2015c710 2023-07-13T22:33:29.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce25d9a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce2006b420 0x7fce200d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce2659b700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce2015c710 0x7fce200af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce25d9a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fce2006b420 0x7fce200d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.60:60712/0 (socket says 172.21.15.60:60712) 2023-07-13T22:33:29.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce26d9c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fce200d0ca0 0x7fce2006a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:32838/0 (socket says 172.21.15.60:32838) 2023-07-13T22:33:29.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce25d9a700 1 -- 172.21.15.60:0/951923030 learned_addr learned my addr 172.21.15.60:0/951923030 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:29.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce2659b700 1 -- 172.21.15.60:0/951923030 >> v2:172.21.15.105:3300/0 conn(0x7fce2006b420 msgr2=0x7fce200d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:29.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce2659b700 1 --2- 172.21.15.60:0/951923030 >> v2:172.21.15.105:3300/0 conn(0x7fce2006b420 0x7fce200d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce2659b700 1 -- 172.21.15.60:0/951923030 >> v1:172.21.15.105:6789/0 conn(0x7fce200d0ca0 legacy=0x7fce2006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce2659b700 1 -- 172.21.15.60:0/951923030 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce200b02b0 con 0x7fce2015c710 2023-07-13T22:33:29.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce25599700 1 -- 172.21.15.60:0/951923030 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773524735 0 0) 0x7fce18001490 con 0x7fce200d0ca0 2023-07-13T22:33:29.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce25d9a700 1 --2- 172.21.15.60:0/951923030 >> v2:172.21.15.105:3300/0 conn(0x7fce2006b420 0x7fce200d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:29.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.887+0000 7fce2659b700 1 --2- 172.21.15.60:0/951923030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce2015c710 0x7fce200af780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009840 tx=0x7fce1c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde3793d8acf3a04 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce25599700 1 -- 172.21.15.60:0/951923030 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c011830 con 0x7fce2015c710 2023-07-13T22:33:29.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce25599700 1 -- 172.21.15.60:0/951923030 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c011990 con 0x7fce2015c710 2023-07-13T22:33:29.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce25599700 1 -- 172.21.15.60:0/951923030 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c011d20 con 0x7fce2015c710 2023-07-13T22:33:29.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce287ff700 1 -- 172.21.15.60:0/951923030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce2015c710 msgr2=0x7fce200af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce287ff700 1 --2- 172.21.15.60:0/951923030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce2015c710 0x7fce200af780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009840 tx=0x7fce1c009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:29.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce287ff700 1 -- 172.21.15.60:0/951923030 shutdown_connections 2023-07-13T22:33:29.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce287ff700 1 --2- 172.21.15.60:0/951923030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce2015c710 0x7fce200af780 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce287ff700 1 --2- 172.21.15.60:0/951923030 >> v2:172.21.15.105:3300/0 conn(0x7fce2006b420 0x7fce200d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce287ff700 1 -- 172.21.15.60:0/951923030 >> 172.21.15.60:0/951923030 conn(0x7fce2009bf20 msgr2=0x7fce200d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:29.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.888+0000 7fce287ff700 1 -- 172.21.15.60:0/951923030 shutdown_connections 2023-07-13T22:33:29.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.889+0000 7fce287ff700 1 -- 172.21.15.60:0/951923030 wait complete. 2023-07-13T22:33:29.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.891+0000 7fce287ff700 1 Processor -- start 2023-07-13T22:33:29.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.891+0000 7fce287ff700 1 -- start start 2023-07-13T22:33:29.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.892+0000 7fce287ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce2006b420 0x7fce201f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.892+0000 7fce287ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce200d0ca0 0x7fce2015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.892+0000 7fce287ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2015c710 0x7fce20157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.892+0000 7fce287ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce20153d30 con 0x7fce2015c710 2023-07-13T22:33:29.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.892+0000 7fce287ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce20153f80 con 0x7fce2006b420 2023-07-13T22:33:29.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.892+0000 7fce287ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce20154260 con 0x7fce200d0ca0 2023-07-13T22:33:29.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce26d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2015c710 0x7fce20157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce26d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2015c710 0x7fce20157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50586/0 (socket says 172.21.15.60:50586) 2023-07-13T22:33:29.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce26d9c700 1 -- 172.21.15.60:0/1119282038 learned_addr learned my addr 172.21.15.60:0/1119282038 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:29.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce2659b700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce2006b420 0x7fce201f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce25d9a700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce200d0ca0 0x7fce2015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce26d9c700 1 -- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce200d0ca0 msgr2=0x7fce2015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce26d9c700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce200d0ca0 0x7fce2015d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce26d9c700 1 -- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce2006b420 msgr2=0x7fce201f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce26d9c700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce2006b420 0x7fce201f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce26d9c700 1 -- 172.21.15.60:0/1119282038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce10009550 con 0x7fce2015c710 2023-07-13T22:33:29.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce2659b700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce2006b420 0x7fce201f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:29.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.893+0000 7fce26d9c700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2015c710 0x7fce20157620 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fce1800cca0 tx=0x7fce1800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.894+0000 7fce16ffd700 1 -- 172.21.15.60:0/1119282038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1800cea0 con 0x7fce2015c710 2023-07-13T22:33:29.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.894+0000 7fce16ffd700 1 -- 172.21.15.60:0/1119282038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1800ada0 con 0x7fce2015c710 2023-07-13T22:33:29.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.894+0000 7fce16ffd700 1 -- 172.21.15.60:0/1119282038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18007310 con 0x7fce2015c710 2023-07-13T22:33:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.894+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c0094a0 con 0x7fce2015c710 2023-07-13T22:33:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.894+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce20154820 con 0x7fce2015c710 2023-07-13T22:33:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.895+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce200b0870 con 0x7fce2015c710 2023-07-13T22:33:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.896+0000 7fce16ffd700 1 -- 172.21.15.60:0/1119282038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fce18007470 con 0x7fce2015c710 2023-07-13T22:33:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.897+0000 7fce16ffd700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce0406ace0 0x7fce0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.897+0000 7fce16ffd700 1 -- 172.21.15.60:0/1119282038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fce18088be0 con 0x7fce2015c710 2023-07-13T22:33:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.897+0000 7fce16ffd700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7fce04071150 0x7fce04073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.897+0000 7fce16ffd700 1 -- 172.21.15.60:0/1119282038 --> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce04073c30 con 0x7fce04071150 2023-07-13T22:33:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.897+0000 7fce16ffd700 1 -- 172.21.15.60:0/1119282038 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce1808d2a0 con 0x7fce2015c710 2023-07-13T22:33:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.897+0000 7fce2659b700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce0406ace0 0x7fce0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.898+0000 7fce25d9a700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7fce04071150 0x7fce04073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.898+0000 7fce2659b700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce0406ace0 0x7fce0406d190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0090b0 tx=0x7fce1c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.898+0000 7fce25d9a700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7fce04071150 0x7fce04073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:29.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.899+0000 7fce16ffd700 1 -- 172.21.15.60:0/1119282038 <== osd.4 v2:172.21.15.105:6808/922467711 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fce04073c30 con 0x7fce04071150 2023-07-13T22:33:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.914+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 --> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fce200b0590 con 0x7fce04071150 2023-07-13T22:33:29.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce16ffd700 1 -- 172.21.15.60:0/1119282038 <== osd.4 v2:172.21.15.105:6808/922467711 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fce10009e40 con 0x7fce04071150 2023-07-13T22:33:29.902 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:33:29.903 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-13T22:33:29.903 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:33:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7fce04071150 msgr2=0x7fce04073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce287ff700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.105:6808/922467711,v1:172.21.15.105:6809/922467711] conn(0x7fce04071150 0x7fce04073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce0406ace0 msgr2=0x7fce0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce287ff700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fce0406ace0 0x7fce0406d190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0090b0 tx=0x7fce1c01c040 comp rx=0 tx=0).stop 2023-07-13T22:33:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2015c710 msgr2=0x7fce20157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce287ff700 1 --2- 172.21.15.60:0/1119282038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2015c710 0x7fce20157620 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fce1800cca0 tx=0x7fce1800a340 comp rx=0 tx=0).stop 2023-07-13T22:33:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce26d9c700 1 -- 172.21.15.60:0/1119282038 reap_dead start 2023-07-13T22:33:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 shutdown_connections 2023-07-13T22:33:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.915+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 >> 172.21.15.60:0/1119282038 conn(0x7fce2009bf20 msgr2=0x7fce200d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:29.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.916+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 shutdown_connections 2023-07-13T22:33:29.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:29.916+0000 7fce287ff700 1 -- 172.21.15.60:0/1119282038 wait complete. 2023-07-13T22:33:29.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-07-13T22:33:30.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.136+0000 7f4280fff700 1 Processor -- start 2023-07-13T22:33:30.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.137+0000 7f4280fff700 1 -- start start 2023-07-13T22:33:30.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.138+0000 7f4280fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f427c0d07d0 0x7f427c0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.138+0000 7f4280fff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f427c0b1580 0x7f427c06aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.138+0000 7f4280fff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427c08d030 con 0x7f427c0b2230 2023-07-13T22:33:30.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.138+0000 7f4280fff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427c0af7f0 con 0x7f427c0b1580 2023-07-13T22:33:30.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.138+0000 7f4280fff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427c0d10e0 con 0x7f427c0d07d0 2023-07-13T22:33:30.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.138+0000 7f427ad9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f427c0b1580 0x7f427c06aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.138+0000 7f4279d9b700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f427c0b2230 0x7f427c06a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35678/0 (socket says 172.21.15.60:35678) 2023-07-13T22:33:30.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.138+0000 7f4279d9b700 1 -- 172.21.15.60:0/587712208 learned_addr learned my addr 172.21.15.60:0/587712208 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:30.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.138+0000 7f427a59c700 1 --2- 172.21.15.60:0/587712208 >> v2:172.21.15.105:3300/0 conn(0x7f427c0d07d0 0x7f427c0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.139+0000 7f427959a700 1 -- 172.21.15.60:0/587712208 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637822727 0 0) 0x7f427c08d030 con 0x7f427c0b2230 2023-07-13T22:33:30.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.139+0000 7f427959a700 1 -- 172.21.15.60:0/587712208 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42640035b0 con 0x7f427c0b2230 2023-07-13T22:33:30.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.139+0000 7f427a59c700 1 -- 172.21.15.60:0/587712208 >> v2:172.21.15.60:3300/0 conn(0x7f427c0b1580 msgr2=0x7f427c06aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:30.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.139+0000 7f427a59c700 1 --2- 172.21.15.60:0/587712208 >> v2:172.21.15.60:3300/0 conn(0x7f427c0b1580 0x7f427c06aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.139+0000 7f427a59c700 1 -- 172.21.15.60:0/587712208 >> v1:172.21.15.60:6789/0 conn(0x7f427c0b2230 legacy=0x7f427c06a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.139+0000 7f427a59c700 1 -- 172.21.15.60:0/587712208 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c15ca30 con 0x7f427c0d07d0 2023-07-13T22:33:30.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.139+0000 7f427ad9d700 1 --2- 172.21.15.60:0/587712208 >> v2:172.21.15.60:3300/0 conn(0x7f427c0b1580 0x7f427c06aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:30.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.139+0000 7f427a59c700 1 --2- 172.21.15.60:0/587712208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427c0d07d0 0x7f427c0d0ba0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f427c06cd30 tx=0x7f426c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf919bc482cff9b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.140+0000 7f427959a700 1 -- 172.21.15.60:0/587712208 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c011830 con 0x7f427c0d07d0 2023-07-13T22:33:30.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.140+0000 7f427959a700 1 -- 172.21.15.60:0/587712208 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c011990 con 0x7f427c0d07d0 2023-07-13T22:33:30.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.140+0000 7f427959a700 1 -- 172.21.15.60:0/587712208 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c011d20 con 0x7f427c0d07d0 2023-07-13T22:33:30.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.140+0000 7f4280fff700 1 -- 172.21.15.60:0/587712208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427c0d07d0 msgr2=0x7f427c0d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.140+0000 7f4280fff700 1 --2- 172.21.15.60:0/587712208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427c0d07d0 0x7f427c0d0ba0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f427c06cd30 tx=0x7f426c009b20 comp rx=0 tx=0).stop 2023-07-13T22:33:30.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.140+0000 7f4280fff700 1 -- 172.21.15.60:0/587712208 shutdown_connections 2023-07-13T22:33:30.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.140+0000 7f4280fff700 1 --2- 172.21.15.60:0/587712208 >> v2:172.21.15.60:3300/0 conn(0x7f427c0b1580 0x7f427c06aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.140+0000 7f4280fff700 1 --2- 172.21.15.60:0/587712208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427c0d07d0 0x7f427c0d0ba0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.140+0000 7f4280fff700 1 -- 172.21.15.60:0/587712208 >> 172.21.15.60:0/587712208 conn(0x7f427c09bf90 msgr2=0x7f427c0d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:30.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.141+0000 7f4280fff700 1 -- 172.21.15.60:0/587712208 shutdown_connections 2023-07-13T22:33:30.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.141+0000 7f4280fff700 1 -- 172.21.15.60:0/587712208 wait complete. 2023-07-13T22:33:30.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.142+0000 7f4280fff700 1 Processor -- start 2023-07-13T22:33:30.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.142+0000 7f4280fff700 1 -- start start 2023-07-13T22:33:30.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.143+0000 7f4280fff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f427c0b1580 0x7f427c1423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.143+0000 7f4280fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f427c0b2230 0x7f427c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.143+0000 7f4280fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427c0d07d0 0x7f427c153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.143+0000 7f4280fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c142930 con 0x7f427c0b2230 2023-07-13T22:33:30.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.143+0000 7f4280fff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0ca4e0 con 0x7f427c0d07d0 2023-07-13T22:33:30.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.143+0000 7f4280fff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0cb940 con 0x7f427c0b1580 2023-07-13T22:33:30.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f4279d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f427c0b2230 0x7f427c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f427a59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f427c0b1580 0x7f427c1423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f4279d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f427c0b2230 0x7f427c0c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50616/0 (socket says 172.21.15.60:50616) 2023-07-13T22:33:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f4279d9b700 1 -- 172.21.15.60:0/983671254 learned_addr learned my addr 172.21.15.60:0/983671254 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f427ad9d700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427c0d07d0 0x7f427c153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f4279d9b700 1 -- 172.21.15.60:0/983671254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f427c0b1580 msgr2=0x7f427c1423f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f4279d9b700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f427c0b1580 0x7f427c1423f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f4279d9b700 1 -- 172.21.15.60:0/983671254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427c0d07d0 msgr2=0x7f427c153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f4279d9b700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427c0d07d0 0x7f427c153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f4279d9b700 1 -- 172.21.15.60:0/983671254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4274009a70 con 0x7f427c0b2230 2023-07-13T22:33:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.144+0000 7f4279d9b700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f427c0b2230 0x7f427c0c9cd0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f427000c700 tx=0x7f427000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.145+0000 7f426affd700 1 -- 172.21.15.60:0/983671254 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270005d70 con 0x7f427c0b2230 2023-07-13T22:33:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.145+0000 7f426affd700 1 -- 172.21.15.60:0/983671254 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42700070d0 con 0x7f427c0b2230 2023-07-13T22:33:30.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.145+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c0094a0 con 0x7f427c0b2230 2023-07-13T22:33:30.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.145+0000 7f426affd700 1 -- 172.21.15.60:0/983671254 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270007460 con 0x7f427c0b2230 2023-07-13T22:33:30.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.145+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c0cbec0 con 0x7f427c0b2230 2023-07-13T22:33:30.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.146+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427c0741c0 con 0x7f427c0b2230 2023-07-13T22:33:30.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.147+0000 7f426affd700 1 -- 172.21.15.60:0/983671254 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42700075c0 con 0x7f427c0b2230 2023-07-13T22:33:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.148+0000 7f426affd700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.148+0000 7f426affd700 1 -- 172.21.15.60:0/983671254 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f42700886f0 con 0x7f427c0b2230 2023-07-13T22:33:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.148+0000 7f427a59c700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.149+0000 7f427a59c700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f426c0090b0 tx=0x7f426c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.153+0000 7f426affd700 1 -- 172.21.15.60:0/983671254 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f4270053a30 con 0x7f427c0b2230 2023-07-13T22:33:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.359+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f427c06c0a0 con 0x7f425c06ace0 2023-07-13T22:33:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.359+0000 7f426affd700 1 -- 172.21.15.60:0/983671254 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f426c011b80 con 0x7f425c06ace0 2023-07-13T22:33:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 msgr2=0x7f425c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f426c0090b0 tx=0x7f426c01c040 comp rx=0 tx=0).stop 2023-07-13T22:33:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f427c0b2230 msgr2=0x7f427c0c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f427c0b2230 0x7f427c0c9cd0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f427000c700 tx=0x7f427000ca10 comp rx=0 tx=0).stop 2023-07-13T22:33:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 shutdown_connections 2023-07-13T22:33:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f425c06ace0 0x7f425c06d190 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f427c0d07d0 0x7f427c153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f427c0b1580 0x7f427c1423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 --2- 172.21.15.60:0/983671254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f427c0b2230 0x7f427c0c9cd0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 >> 172.21.15.60:0/983671254 conn(0x7f427c09bf90 msgr2=0x7f427c0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 shutdown_connections 2023-07-13T22:33:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.362+0000 7f4280fff700 1 -- 172.21.15.60:0/983671254 wait complete. 2023-07-13T22:33:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 4 to deep-scrub 2023-07-13T22:33:30.365 INFO:tasks.ceph:Scrubbing osd.5 2023-07-13T22:33:30.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-07-13T22:33:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.586+0000 7fca2e7ff700 1 Processor -- start 2023-07-13T22:33:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.586+0000 7fca2e7ff700 1 -- start start 2023-07-13T22:33:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.587+0000 7fca2e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2815c5a0 0x7fca280cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca2e7ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fca2806b6b0 0x7fca280cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca2e7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2815c970 con 0x7fca280ccd60 2023-07-13T22:33:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca2e7ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca280cd640 con 0x7fca2806b6b0 2023-07-13T22:33:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca2e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca280cbcc0 con 0x7fca2815c5a0 2023-07-13T22:33:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca277fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fca280ccd60 0x7fca2806afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35680/0 (socket says 172.21.15.60:35680) 2023-07-13T22:33:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca277fe700 1 -- 172.21.15.60:0/3639794590 learned_addr learned my addr 172.21.15.60:0/3639794590 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca27fff700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2815c5a0 0x7fca280cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca2cd9c700 1 --2- 172.21.15.60:0/3639794590 >> v2:172.21.15.105:3300/0 conn(0x7fca2806b6b0 0x7fca280cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121756485 0 0) 0x7fca2815c970 con 0x7fca280ccd60 2023-07-13T22:33:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.588+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca100035b0 con 0x7fca280ccd60 2023-07-13T22:33:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3338293560 0 0) 0x7fca100035b0 con 0x7fca280ccd60 2023-07-13T22:33:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2815c5a0 msgr2=0x7fca280cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2815c5a0 0x7fca280cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 >> v2:172.21.15.105:3300/0 conn(0x7fca2806b6b0 msgr2=0x7fca280cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 --2- 172.21.15.60:0/3639794590 >> v2:172.21.15.105:3300/0 conn(0x7fca2806b6b0 0x7fca280cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca280cbfa0 con 0x7fca280ccd60 2023-07-13T22:33:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1072894657 0 0) 0x7fca1c001670 con 0x7fca280ccd60 2023-07-13T22:33:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 >> v1:172.21.15.60:6789/0 conn(0x7fca280ccd60 legacy=0x7fca2806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca10003a00 0x7fca10003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2815c5a0 0x7fca10004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca10004d90 0x7fca10005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280cd640 con 0x7fca10004d90 2023-07-13T22:33:30.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280cbcc0 con 0x7fca10003a00 2023-07-13T22:33:30.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1000ba10 con 0x7fca2815c5a0 2023-07-13T22:33:30.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca2cd9c700 1 --2- 172.21.15.60:0/3639794590 >> v2:172.21.15.105:3300/0 conn(0x7fca2806b6b0 0x7fca280cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:30.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca2cd9c700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca10004d90 0x7fca10005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca27fff700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca10003a00 0x7fca10003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca277fe700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2815c5a0 0x7fca10004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca2cd9c700 1 -- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2815c5a0 msgr2=0x7fca10004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca2cd9c700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2815c5a0 0x7fca10004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca2cd9c700 1 -- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca10003a00 msgr2=0x7fca10003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca2cd9c700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca10003a00 0x7fca10003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca2cd9c700 1 -- 172.21.15.60:0/3639794590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca100035b0 con 0x7fca10004d90 2023-07-13T22:33:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.589+0000 7fca27fff700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca10003a00 0x7fca10003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.590+0000 7fca2cd9c700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca10004d90 0x7fca10005200 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fca2000e350 tx=0x7fca10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba43e0b2139cd5b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.590+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000edb0 con 0x7fca10004d90 2023-07-13T22:33:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.590+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000f040 con 0x7fca10004d90 2023-07-13T22:33:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.590+0000 7fca26ffd700 1 -- 172.21.15.60:0/3639794590 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20003c40 con 0x7fca10004d90 2023-07-13T22:33:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.590+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3639794590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca10004d90 msgr2=0x7fca10005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.590+0000 7fca2e7ff700 1 --2- 172.21.15.60:0/3639794590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca10004d90 0x7fca10005200 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fca2000e350 tx=0x7fca10005880 comp rx=0 tx=0).stop 2023-07-13T22:33:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.590+0000 7fca2cd9c700 1 -- 172.21.15.60:0/3639794590 reap_dead start 2023-07-13T22:33:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.591+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3639794590 shutdown_connections 2023-07-13T22:33:30.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.591+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3639794590 >> 172.21.15.60:0/3639794590 conn(0x7fca28099a30 msgr2=0x7fca28099e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:30.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.591+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3639794590 shutdown_connections 2023-07-13T22:33:30.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.591+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3639794590 wait complete. 2023-07-13T22:33:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.592+0000 7fca2e7ff700 1 Processor -- start 2023-07-13T22:33:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.593+0000 7fca2e7ff700 1 -- start start 2023-07-13T22:33:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.593+0000 7fca2e7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca10004d90 0x7fca2815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.593+0000 7fca2e7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca2814b660 0x7fca2814bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.593+0000 7fca2e7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2814c010 0x7fca28158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.593+0000 7fca2e7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2814c460 con 0x7fca2814b660 2023-07-13T22:33:30.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.593+0000 7fca2e7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280b0e20 con 0x7fca10004d90 2023-07-13T22:33:30.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.593+0000 7fca2e7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca280b1100 con 0x7fca2814c010 2023-07-13T22:33:30.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca277fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca2814b660 0x7fca2814bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca277fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca2814b660 0x7fca2814bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50622/0 (socket says 172.21.15.60:50622) 2023-07-13T22:33:30.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca277fe700 1 -- 172.21.15.60:0/3364267285 learned_addr learned my addr 172.21.15.60:0/3364267285 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:30.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca27fff700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca10004d90 0x7fca2815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca2cd9c700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2814c010 0x7fca28158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca277fe700 1 -- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2814c010 msgr2=0x7fca28158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca277fe700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fca2814c010 0x7fca28158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca277fe700 1 -- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca10004d90 msgr2=0x7fca2815ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca277fe700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca10004d90 0x7fca2815ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca277fe700 1 -- 172.21.15.60:0/3364267285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca280b13e0 con 0x7fca2814b660 2023-07-13T22:33:30.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.594+0000 7fca277fe700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca2814b660 0x7fca2814bad0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00eaf0 tx=0x7fca1c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.595+0000 7fca24ff9700 1 -- 172.21.15.60:0/3364267285 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c005e00 con 0x7fca2814b660 2023-07-13T22:33:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.595+0000 7fca24ff9700 1 -- 172.21.15.60:0/3364267285 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c00c7e0 con 0x7fca2814b660 2023-07-13T22:33:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.595+0000 7fca24ff9700 1 -- 172.21.15.60:0/3364267285 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c00cb90 con 0x7fca2814b660 2023-07-13T22:33:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.595+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca280b16c0 con 0x7fca2814b660 2023-07-13T22:33:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.595+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2815b240 con 0x7fca2814b660 2023-07-13T22:33:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.596+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca280cc560 con 0x7fca2814b660 2023-07-13T22:33:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.597+0000 7fca24ff9700 1 -- 172.21.15.60:0/3364267285 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca1c00ccf0 con 0x7fca2814b660 2023-07-13T22:33:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.598+0000 7fca24ff9700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca1406ace0 0x7fca1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.598+0000 7fca24ff9700 1 -- 172.21.15.60:0/3364267285 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fca1c088dc0 con 0x7fca2814b660 2023-07-13T22:33:30.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.598+0000 7fca27fff700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca1406ace0 0x7fca1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.598+0000 7fca24ff9700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7fca14071150 0x7fca14073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.598+0000 7fca24ff9700 1 -- 172.21.15.60:0/3364267285 --> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca14073c30 con 0x7fca14071150 2023-07-13T22:33:30.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.598+0000 7fca24ff9700 1 -- 172.21.15.60:0/3364267285 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca1c08b3a0 con 0x7fca2814b660 2023-07-13T22:33:30.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.599+0000 7fca2cd9c700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7fca14071150 0x7fca14073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.599+0000 7fca27fff700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca1406ace0 0x7fca1406d190 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fca18009b40 tx=0x7fca18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.600+0000 7fca2cd9c700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7fca14071150 0x7fca14073560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.601+0000 7fca24ff9700 1 -- 172.21.15.60:0/3364267285 <== osd.5 v2:172.21.15.105:6816/584130633 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fca14073c30 con 0x7fca14071150 2023-07-13T22:33:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.616+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 --> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fca280cc280 con 0x7fca14071150 2023-07-13T22:33:30.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca24ff9700 1 -- 172.21.15.60:0/3364267285 <== osd.5 v2:172.21.15.105:6816/584130633 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fca2000ebb0 con 0x7fca14071150 2023-07-13T22:33:30.603 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:33:30.603 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-13T22:33:30.603 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:33:30.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7fca14071150 msgr2=0x7fca14073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.105:6816/584130633,v1:172.21.15.105:6817/584130633] conn(0x7fca14071150 0x7fca14073560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca1406ace0 msgr2=0x7fca1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fca1406ace0 0x7fca1406d190 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fca18009b40 tx=0x7fca18008040 comp rx=0 tx=0).stop 2023-07-13T22:33:30.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca2814b660 msgr2=0x7fca2814bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 --2- 172.21.15.60:0/3364267285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca2814b660 0x7fca2814bad0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00eaf0 tx=0x7fca1c00ee00 comp rx=0 tx=0).stop 2023-07-13T22:33:30.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2cd9c700 1 -- 172.21.15.60:0/3364267285 reap_dead start 2023-07-13T22:33:30.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 shutdown_connections 2023-07-13T22:33:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 >> 172.21.15.60:0/3364267285 conn(0x7fca28099a30 msgr2=0x7fca280cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 shutdown_connections 2023-07-13T22:33:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.617+0000 7fca2e7ff700 1 -- 172.21.15.60:0/3364267285 wait complete. 2023-07-13T22:33:30.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-07-13T22:33:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.840+0000 7f53e4dff700 1 Processor -- start 2023-07-13T22:33:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.840+0000 7f53e4dff700 1 -- start start 2023-07-13T22:33:30.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53e4dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53e00b1ce0 0x7f53e00b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53e4dff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e008d030 con 0x7f53e00d07d0 2023-07-13T22:33:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53e4dff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e00af7f0 con 0x7f53e00b2230 2023-07-13T22:33:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53e4dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e015ca30 con 0x7f53e00b1ce0 2023-07-13T22:33:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53de59c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f53e00d07d0 0x7f53e00d0ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35694/0 (socket says 172.21.15.60:35694) 2023-07-13T22:33:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53de59c700 1 -- 172.21.15.60:0/718173752 learned_addr learned my addr 172.21.15.60:0/718173752 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53ded9d700 1 --2- 172.21.15.60:0/718173752 >> v2:172.21.15.60:3300/0 conn(0x7f53e00b1ce0 0x7f53e00b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53ddd9b700 1 -- 172.21.15.60:0/718173752 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657579066 0 0) 0x7f53e008d030 con 0x7f53e00d07d0 2023-07-13T22:33:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53ddd9b700 1 -- 172.21.15.60:0/718173752 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53cc0035b0 con 0x7f53e00d07d0 2023-07-13T22:33:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53ded9d700 1 -- 172.21.15.60:0/718173752 >> v1:172.21.15.105:6789/0 conn(0x7f53e00b2230 legacy=0x7f53e00b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53ded9d700 1 -- 172.21.15.60:0/718173752 >> v1:172.21.15.60:6789/0 conn(0x7f53e00d07d0 legacy=0x7f53e00d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.842+0000 7f53ded9d700 1 -- 172.21.15.60:0/718173752 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e006bdc0 con 0x7f53e00b1ce0 2023-07-13T22:33:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.843+0000 7f53ded9d700 1 --2- 172.21.15.60:0/718173752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b1ce0 0x7f53e00b0c80 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009d90 tx=0x7f53d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a360f6a9daed21 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.843+0000 7f53ddd9b700 1 -- 172.21.15.60:0/718173752 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d800e070 con 0x7f53e00b1ce0 2023-07-13T22:33:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.843+0000 7f53ddd9b700 1 -- 172.21.15.60:0/718173752 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d800bb20 con 0x7f53e00b1ce0 2023-07-13T22:33:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.843+0000 7f53ddd9b700 1 -- 172.21.15.60:0/718173752 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8012430 con 0x7f53e00b1ce0 2023-07-13T22:33:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.843+0000 7f53e4dff700 1 -- 172.21.15.60:0/718173752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b1ce0 msgr2=0x7f53e00b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.843+0000 7f53e4dff700 1 --2- 172.21.15.60:0/718173752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b1ce0 0x7f53e00b0c80 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009d90 tx=0x7f53d800b330 comp rx=0 tx=0).stop 2023-07-13T22:33:30.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.843+0000 7f53e4dff700 1 -- 172.21.15.60:0/718173752 shutdown_connections 2023-07-13T22:33:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.843+0000 7f53e4dff700 1 --2- 172.21.15.60:0/718173752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b1ce0 0x7f53e00b0c80 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.843+0000 7f53e4dff700 1 -- 172.21.15.60:0/718173752 >> 172.21.15.60:0/718173752 conn(0x7f53e009bf90 msgr2=0x7f53e00d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.844+0000 7f53e4dff700 1 -- 172.21.15.60:0/718173752 shutdown_connections 2023-07-13T22:33:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.844+0000 7f53e4dff700 1 -- 172.21.15.60:0/718173752 wait complete. 2023-07-13T22:33:30.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.846+0000 7f53e4dff700 1 Processor -- start 2023-07-13T22:33:30.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.846+0000 7f53e4dff700 1 -- start start 2023-07-13T22:33:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.847+0000 7f53e4dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e00b1ce0 0x7f53e00cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.847+0000 7f53e4dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b2230 0x7f53e0156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.847+0000 7f53e4dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e00d07d0 0x7f53e00b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.847+0000 7f53e4dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00cf850 con 0x7f53e00b2230 2023-07-13T22:33:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.847+0000 7f53e4dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01f1c60 con 0x7f53e00b1ce0 2023-07-13T22:33:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.847+0000 7f53e4dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00cdc20 con 0x7f53e00d07d0 2023-07-13T22:33:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b2230 0x7f53e0156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53d7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b2230 0x7f53e0156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50638/0 (socket says 172.21.15.60:50638) 2023-07-13T22:33:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53d7fff700 1 -- 172.21.15.60:0/3256380996 learned_addr learned my addr 172.21.15.60:0/3256380996 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53de59c700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e00b1ce0 0x7f53e00cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53ded9d700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e00d07d0 0x7f53e00b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53d7fff700 1 -- 172.21.15.60:0/3256380996 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e00d07d0 msgr2=0x7f53e00b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53d7fff700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e00d07d0 0x7f53e00b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53d7fff700 1 -- 172.21.15.60:0/3256380996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e00b1ce0 msgr2=0x7f53e00cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53d7fff700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e00b1ce0 0x7f53e00cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53d7fff700 1 -- 172.21.15.60:0/3256380996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d8009a70 con 0x7f53e00b2230 2023-07-13T22:33:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53ded9d700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e00d07d0 0x7f53e00b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.848+0000 7f53d7fff700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b2230 0x7f53e0156c90 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f53c800c700 tx=0x7f53c800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.849+0000 7f53d6ffd700 1 -- 172.21.15.60:0/3256380996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80052d0 con 0x7f53e00b2230 2023-07-13T22:33:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.849+0000 7f53d6ffd700 1 -- 172.21.15.60:0/3256380996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8005430 con 0x7f53e00b2230 2023-07-13T22:33:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.849+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e00cdf00 con 0x7f53e00b2230 2023-07-13T22:33:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.849+0000 7f53d6ffd700 1 -- 172.21.15.60:0/3256380996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8003880 con 0x7f53e00b2230 2023-07-13T22:33:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.849+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e00ce4c0 con 0x7f53e00b2230 2023-07-13T22:33:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.850+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e00afa70 con 0x7f53e00b2230 2023-07-13T22:33:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.851+0000 7f53d6ffd700 1 -- 172.21.15.60:0/3256380996 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53c80039e0 con 0x7f53e00b2230 2023-07-13T22:33:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.852+0000 7f53d6ffd700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f53c006ace0 0x7f53c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.852+0000 7f53d6ffd700 1 -- 172.21.15.60:0/3256380996 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f53c8088490 con 0x7f53e00b2230 2023-07-13T22:33:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.852+0000 7f53de59c700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f53c006ace0 0x7f53c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.853+0000 7f53de59c700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f53c006ace0 0x7f53c006d190 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f53d0007fd0 tx=0x7f53d0007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:30.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:30.856+0000 7f53d6ffd700 1 -- 172.21.15.60:0/3256380996 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f53e00afa70 con 0x7f53e00b2230 2023-07-13T22:33:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.063+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f53e006c0a0 con 0x7f53c006ace0 2023-07-13T22:33:31.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.064+0000 7f53d6ffd700 1 -- 172.21.15.60:0/3256380996 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f53d00058e0 con 0x7f53c006ace0 2023-07-13T22:33:31.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f53c006ace0 msgr2=0x7f53c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f53c006ace0 0x7f53c006d190 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f53d0007fd0 tx=0x7f53d0007dc0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b2230 msgr2=0x7f53e0156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b2230 0x7f53e0156c90 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f53c800c700 tx=0x7f53c800ca10 comp rx=0 tx=0).stop 2023-07-13T22:33:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 shutdown_connections 2023-07-13T22:33:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f53c006ace0 0x7f53c006d190 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e00d07d0 0x7f53e00b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e00b1ce0 0x7f53e00cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 --2- 172.21.15.60:0/3256380996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e00b2230 0x7f53e0156c90 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 >> 172.21.15.60:0/3256380996 conn(0x7f53e009bf90 msgr2=0x7f53e00b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 shutdown_connections 2023-07-13T22:33:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.066+0000 7f53e4dff700 1 -- 172.21.15.60:0/3256380996 wait complete. 2023-07-13T22:33:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 5 to deep-scrub 2023-07-13T22:33:31.070 INFO:tasks.ceph:Scrubbing osd.6 2023-07-13T22:33:31.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-07-13T22:33:31.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.291+0000 7feaba9ff700 1 Processor -- start 2023-07-13T22:33:31.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.292+0000 7feaba9ff700 1 -- start start 2023-07-13T22:33:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.293+0000 7feaba9ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feab4156300 0x7feab40cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.293+0000 7feaba9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab40b3a80 0x7feab40b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.293+0000 7feaba9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab4159060 con 0x7feab40b3600 2023-07-13T22:33:31.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.293+0000 7feaba9ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab40ccc40 con 0x7feab4156300 2023-07-13T22:33:31.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.293+0000 7feaba9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab4159340 con 0x7feab40b3a80 2023-07-13T22:33:31.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.293+0000 7feab37fe700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feab40b3600 0x7feab40ce6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35704/0 (socket says 172.21.15.60:35704) 2023-07-13T22:33:31.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.293+0000 7feab37fe700 1 -- 172.21.15.60:0/2187458986 learned_addr learned my addr 172.21.15.60:0/2187458986 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:31.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.293+0000 7feab3fff700 1 --2- 172.21.15.60:0/2187458986 >> v2:172.21.15.105:3300/0 conn(0x7feab4156300 0x7feab40cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.293+0000 7feab8f9c700 1 --2- 172.21.15.60:0/2187458986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab40b3a80 0x7feab40b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.294+0000 7feab2ffd700 1 -- 172.21.15.60:0/2187458986 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639953603 0 0) 0x7feab4159060 con 0x7feab40b3600 2023-07-13T22:33:31.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.294+0000 7feab2ffd700 1 -- 172.21.15.60:0/2187458986 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea9c0035b0 con 0x7feab40b3600 2023-07-13T22:33:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.294+0000 7feab8f9c700 1 -- 172.21.15.60:0/2187458986 >> v2:172.21.15.105:3300/0 conn(0x7feab4156300 msgr2=0x7feab40cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.294+0000 7feab8f9c700 1 --2- 172.21.15.60:0/2187458986 >> v2:172.21.15.105:3300/0 conn(0x7feab4156300 0x7feab40cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.294+0000 7feab8f9c700 1 -- 172.21.15.60:0/2187458986 >> v1:172.21.15.60:6789/0 conn(0x7feab40b3600 legacy=0x7feab40ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.294+0000 7feab8f9c700 1 -- 172.21.15.60:0/2187458986 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4159620 con 0x7feab40b3a80 2023-07-13T22:33:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.294+0000 7feab3fff700 1 --2- 172.21.15.60:0/2187458986 >> v2:172.21.15.105:3300/0 conn(0x7feab4156300 0x7feab40cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.294+0000 7feab8f9c700 1 --2- 172.21.15.60:0/2187458986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab40b3a80 0x7feab40b1a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feaac00aea0 tx=0x7feaac00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e076ca5dac0b737 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.294+0000 7feab2ffd700 1 -- 172.21.15.60:0/2187458986 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac00f070 con 0x7feab40b3a80 2023-07-13T22:33:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feab2ffd700 1 -- 172.21.15.60:0/2187458986 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac00a2e0 con 0x7feab40b3a80 2023-07-13T22:33:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feab2ffd700 1 -- 172.21.15.60:0/2187458986 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac00cb50 con 0x7feab40b3a80 2023-07-13T22:33:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feaba9ff700 1 -- 172.21.15.60:0/2187458986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab40b3a80 msgr2=0x7feab40b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feaba9ff700 1 --2- 172.21.15.60:0/2187458986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab40b3a80 0x7feab40b1a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feaac00aea0 tx=0x7feaac00c330 comp rx=0 tx=0).stop 2023-07-13T22:33:31.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feaba9ff700 1 -- 172.21.15.60:0/2187458986 shutdown_connections 2023-07-13T22:33:31.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feaba9ff700 1 --2- 172.21.15.60:0/2187458986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab40b3a80 0x7feab40b1a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feaba9ff700 1 --2- 172.21.15.60:0/2187458986 >> v2:172.21.15.105:3300/0 conn(0x7feab4156300 0x7feab40cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feaba9ff700 1 -- 172.21.15.60:0/2187458986 >> 172.21.15.60:0/2187458986 conn(0x7feab4065bd0 msgr2=0x7feab4065fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:31.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feaba9ff700 1 -- 172.21.15.60:0/2187458986 shutdown_connections 2023-07-13T22:33:31.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.295+0000 7feaba9ff700 1 -- 172.21.15.60:0/2187458986 wait complete. 2023-07-13T22:33:31.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.297+0000 7feaba9ff700 1 Processor -- start 2023-07-13T22:33:31.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.297+0000 7feaba9ff700 1 -- start start 2023-07-13T22:33:31.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.298+0000 7feaba9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40b3600 0x7feab414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.298+0000 7feaba9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feab40b3a80 0x7feab41f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.298+0000 7feaba9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab4156300 0x7feab41f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.298+0000 7feaba9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab41efea0 con 0x7feab40b3a80 2023-07-13T22:33:31.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.298+0000 7feaba9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab414c760 con 0x7feab40b3600 2023-07-13T22:33:31.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.298+0000 7feaba9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab414d900 con 0x7feab4156300 2023-07-13T22:33:31.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.298+0000 7feab37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feab40b3a80 0x7feab41f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.298+0000 7feab37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feab40b3a80 0x7feab41f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50644/0 (socket says 172.21.15.60:50644) 2023-07-13T22:33:31.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.298+0000 7feab37fe700 1 -- 172.21.15.60:0/3421568809 learned_addr learned my addr 172.21.15.60:0/3421568809 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:31.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab3fff700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40b3600 0x7feab414bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab8f9c700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab4156300 0x7feab41f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab37fe700 1 -- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab4156300 msgr2=0x7feab41f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab37fe700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab4156300 0x7feab41f3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab37fe700 1 -- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40b3600 msgr2=0x7feab414bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab37fe700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40b3600 0x7feab414bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab37fe700 1 -- 172.21.15.60:0/3421568809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa40094a0 con 0x7feab40b3a80 2023-07-13T22:33:31.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab8f9c700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feab4156300 0x7feab41f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:31.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab37fe700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feab40b3a80 0x7feab41f3600 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7feaa800c700 tx=0x7feaa800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab0ff9700 1 -- 172.21.15.60:0/3421568809 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8005d70 con 0x7feab40b3a80 2023-07-13T22:33:31.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab0ff9700 1 -- 172.21.15.60:0/3421568809 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa80070d0 con 0x7feab40b3a80 2023-07-13T22:33:31.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaac00ab80 con 0x7feab40b3a80 2023-07-13T22:33:31.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.299+0000 7feab0ff9700 1 -- 172.21.15.60:0/3421568809 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8007460 con 0x7feab40b3a80 2023-07-13T22:33:31.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.300+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab41f4220 con 0x7feab40b3a80 2023-07-13T22:33:31.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.300+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feab4159be0 con 0x7feab40b3a80 2023-07-13T22:33:31.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.301+0000 7feab0ff9700 1 -- 172.21.15.60:0/3421568809 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feaa80075c0 con 0x7feab40b3a80 2023-07-13T22:33:31.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.302+0000 7feab0ff9700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaa006ace0 0x7feaa006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.302+0000 7feab0ff9700 1 -- 172.21.15.60:0/3421568809 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7feaa8088730 con 0x7feab40b3a80 2023-07-13T22:33:31.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.303+0000 7feab0ff9700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7feaa0071150 0x7feaa0073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.303+0000 7feab0ff9700 1 -- 172.21.15.60:0/3421568809 --> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feaa0073c30 con 0x7feaa0071150 2023-07-13T22:33:31.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.303+0000 7feab3fff700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaa006ace0 0x7feaa006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.303+0000 7feab0ff9700 1 -- 172.21.15.60:0/3421568809 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaa808ad10 con 0x7feab40b3a80 2023-07-13T22:33:31.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.303+0000 7feab8f9c700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7feaa0071150 0x7feaa0073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.303+0000 7feab3fff700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaa006ace0 0x7feaa006d190 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009cd0 tx=0x7feaa4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.304+0000 7feab8f9c700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7feaa0071150 0x7feaa0073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.305+0000 7feab0ff9700 1 -- 172.21.15.60:0/3421568809 <== osd.6 v2:172.21.15.105:6800/58712571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7feaa0073c30 con 0x7feaa0071150 2023-07-13T22:33:31.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.320+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 --> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feab4159900 con 0x7feaa0071150 2023-07-13T22:33:31.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feab0ff9700 1 -- 172.21.15.60:0/3421568809 <== osd.6 v2:172.21.15.105:6800/58712571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7feaac003b80 con 0x7feaa0071150 2023-07-13T22:33:31.307 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:33:31.307 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-13T22:33:31.307 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:33:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7feaa0071150 msgr2=0x7feaa0073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.105:6800/58712571,v1:172.21.15.105:6801/58712571] conn(0x7feaa0071150 0x7feaa0073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaa006ace0 msgr2=0x7feaa006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7feaa006ace0 0x7feaa006d190 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009cd0 tx=0x7feaa4009210 comp rx=0 tx=0).stop 2023-07-13T22:33:31.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feab40b3a80 msgr2=0x7feab41f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 --2- 172.21.15.60:0/3421568809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feab40b3a80 0x7feab41f3600 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7feaa800c700 tx=0x7feaa800ca10 comp rx=0 tx=0).stop 2023-07-13T22:33:31.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feab8f9c700 1 -- 172.21.15.60:0/3421568809 reap_dead start 2023-07-13T22:33:31.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 shutdown_connections 2023-07-13T22:33:31.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 >> 172.21.15.60:0/3421568809 conn(0x7feab4065bd0 msgr2=0x7feab406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:31.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 shutdown_connections 2023-07-13T22:33:31.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.321+0000 7feaba9ff700 1 -- 172.21.15.60:0/3421568809 wait complete. 2023-07-13T22:33:31.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-07-13T22:33:31.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.541+0000 7f79d7bff700 1 Processor -- start 2023-07-13T22:33:31.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.542+0000 7f79d7bff700 1 -- start start 2023-07-13T22:33:31.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.543+0000 7f79d7bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f79d015c460 0x7f79d006c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.543+0000 7f79d7bff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d00afeb0 con 0x7f79d00b2c00 2023-07-13T22:33:31.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.543+0000 7f79d7bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d006bf70 con 0x7f79d006b460 2023-07-13T22:33:31.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.543+0000 7f79d7bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d006b990 con 0x7f79d015c460 2023-07-13T22:33:31.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.543+0000 7f79d619c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f79d00b2c00 0x7f79d00cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35718/0 (socket says 172.21.15.60:35718) 2023-07-13T22:33:31.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.543+0000 7f79d619c700 1 -- 172.21.15.60:0/3290963551 learned_addr learned my addr 172.21.15.60:0/3290963551 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:31.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.543+0000 7f79d599b700 1 --2- 172.21.15.60:0/3290963551 >> v2:172.21.15.105:3300/0 conn(0x7f79d015c460 0x7f79d006c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d4999700 1 -- 172.21.15.60:0/3290963551 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123922861 0 0) 0x7f79d00afeb0 con 0x7f79d00b2c00 2023-07-13T22:33:31.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d4999700 1 -- 172.21.15.60:0/3290963551 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79bc0035b0 con 0x7f79d00b2c00 2023-07-13T22:33:31.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d4999700 1 -- 172.21.15.60:0/3290963551 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006352779 0 0) 0x7f79d006bf70 con 0x7f79d006b460 2023-07-13T22:33:31.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d4999700 1 -- 172.21.15.60:0/3290963551 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d00afeb0 con 0x7f79d006b460 2023-07-13T22:33:31.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d599b700 1 -- 172.21.15.60:0/3290963551 >> v1:172.21.15.105:6789/0 conn(0x7f79d006b460 legacy=0x7f79d00b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d599b700 1 -- 172.21.15.60:0/3290963551 >> v1:172.21.15.60:6789/0 conn(0x7f79d00b2c00 legacy=0x7f79d00cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d599b700 1 -- 172.21.15.60:0/3290963551 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d00b0100 con 0x7f79d015c460 2023-07-13T22:33:31.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d599b700 1 --2- 172.21.15.60:0/3290963551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d015c460 0x7f79d006c400 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009840 tx=0x7f79cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d780293ee2b1d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d4999700 1 -- 172.21.15.60:0/3290963551 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc011830 con 0x7f79d015c460 2023-07-13T22:33:31.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d4999700 1 -- 172.21.15.60:0/3290963551 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc011990 con 0x7f79d015c460 2023-07-13T22:33:31.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.544+0000 7f79d4999700 1 -- 172.21.15.60:0/3290963551 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc011d20 con 0x7f79d015c460 2023-07-13T22:33:31.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.545+0000 7f79d7bff700 1 -- 172.21.15.60:0/3290963551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d015c460 msgr2=0x7f79d006c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.545+0000 7f79d7bff700 1 --2- 172.21.15.60:0/3290963551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d015c460 0x7f79d006c400 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009840 tx=0x7f79cc009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:31.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.545+0000 7f79d7bff700 1 -- 172.21.15.60:0/3290963551 shutdown_connections 2023-07-13T22:33:31.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.545+0000 7f79d7bff700 1 --2- 172.21.15.60:0/3290963551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d015c460 0x7f79d006c400 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.545+0000 7f79d7bff700 1 -- 172.21.15.60:0/3290963551 >> 172.21.15.60:0/3290963551 conn(0x7f79d006f5e0 msgr2=0x7f79d006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:31.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.545+0000 7f79d7bff700 1 -- 172.21.15.60:0/3290963551 shutdown_connections 2023-07-13T22:33:31.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.545+0000 7f79d7bff700 1 -- 172.21.15.60:0/3290963551 wait complete. 2023-07-13T22:33:31.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.547+0000 7f79d7bff700 1 Processor -- start 2023-07-13T22:33:31.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.547+0000 7f79d7bff700 1 -- start start 2023-07-13T22:33:31.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d7bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d006b460 0x7f79d01571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d7bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79d00b2c00 0x7f79d00c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d7bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d015c460 0x7f79d00b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d7bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d00c5bb0 con 0x7f79d015c460 2023-07-13T22:33:31.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d7bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d00c5e90 con 0x7f79d006b460 2023-07-13T22:33:31.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d7bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d00c6170 con 0x7f79d00b2c00 2023-07-13T22:33:31.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d015c460 0x7f79d00b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d619c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d015c460 0x7f79d00b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50646/0 (socket says 172.21.15.60:50646) 2023-07-13T22:33:31.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d619c700 1 -- 172.21.15.60:0/630950127 learned_addr learned my addr 172.21.15.60:0/630950127 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:31.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d599b700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d006b460 0x7f79d01571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.548+0000 7f79d519a700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79d00b2c00 0x7f79d00c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79d619c700 1 -- 172.21.15.60:0/630950127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79d00b2c00 msgr2=0x7f79d00c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79d619c700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79d00b2c00 0x7f79d00c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79d619c700 1 -- 172.21.15.60:0/630950127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d006b460 msgr2=0x7f79d01571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79d619c700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d006b460 0x7f79d01571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79d619c700 1 -- 172.21.15.60:0/630950127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc0094a0 con 0x7f79d015c460 2023-07-13T22:33:31.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79d599b700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d006b460 0x7f79d01571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:31.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79d619c700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d015c460 0x7f79d00b5a50 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f79c800cb70 tx=0x7f79c800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79c67fc700 1 -- 172.21.15.60:0/630950127 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c800aaf0 con 0x7f79d015c460 2023-07-13T22:33:31.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79c67fc700 1 -- 172.21.15.60:0/630950127 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c800ac50 con 0x7f79d015c460 2023-07-13T22:33:31.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d00c6450 con 0x7f79d015c460 2023-07-13T22:33:31.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79c67fc700 1 -- 172.21.15.60:0/630950127 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8012400 con 0x7f79d015c460 2023-07-13T22:33:31.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.549+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d0157700 con 0x7f79d015c460 2023-07-13T22:33:31.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.550+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d00c6970 con 0x7f79d015c460 2023-07-13T22:33:31.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.551+0000 7f79c67fc700 1 -- 172.21.15.60:0/630950127 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79c801b3b0 con 0x7f79d015c460 2023-07-13T22:33:31.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.552+0000 7f79c67fc700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79b406ace0 0x7f79b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.553+0000 7f79c67fc700 1 -- 172.21.15.60:0/630950127 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f79c8088fe0 con 0x7f79d015c460 2023-07-13T22:33:31.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.553+0000 7f79d599b700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79b406ace0 0x7f79b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.553+0000 7f79d599b700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79b406ace0 0x7f79b406d190 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009fd0 tx=0x7f79cc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.557+0000 7f79c67fc700 1 -- 172.21.15.60:0/630950127 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f79d00c6970 con 0x7f79d015c460 2023-07-13T22:33:31.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.765+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f79d00c6000 con 0x7f79b406ace0 2023-07-13T22:33:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.766+0000 7f79c67fc700 1 -- 172.21.15.60:0/630950127 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f79d00c6000 con 0x7f79b406ace0 2023-07-13T22:33:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79b406ace0 msgr2=0x7f79b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79b406ace0 0x7f79b406d190 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009fd0 tx=0x7f79cc01c040 comp rx=0 tx=0).stop 2023-07-13T22:33:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d015c460 msgr2=0x7f79d00b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d015c460 0x7f79d00b5a50 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f79c800cb70 tx=0x7f79c800cf30 comp rx=0 tx=0).stop 2023-07-13T22:33:31.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 shutdown_connections 2023-07-13T22:33:31.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f79b406ace0 0x7f79b406d190 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d015c460 0x7f79d00b5a50 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79d006b460 0x7f79d01571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 --2- 172.21.15.60:0/630950127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f79d00b2c00 0x7f79d00c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 >> 172.21.15.60:0/630950127 conn(0x7f79d006f5e0 msgr2=0x7f79d00b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 shutdown_connections 2023-07-13T22:33:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.768+0000 7f79d7bff700 1 -- 172.21.15.60:0/630950127 wait complete. 2023-07-13T22:33:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 6 to deep-scrub 2023-07-13T22:33:31.772 INFO:tasks.ceph:Scrubbing osd.7 2023-07-13T22:33:31.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-07-13T22:33:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.994+0000 7f4bc4bff700 1 Processor -- start 2023-07-13T22:33:31.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.994+0000 7f4bc4bff700 1 -- start start 2023-07-13T22:33:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.995+0000 7f4bc4bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc015c710 0x7f4bc00af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.995+0000 7f4bc4bff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4bc006b420 0x7f4bc00d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.995+0000 7f4bc4bff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc015cae0 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.995+0000 7f4bc4bff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc00d10c0 con 0x7f4bc006b420 2023-07-13T22:33:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.995+0000 7f4bc4bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00affd0 con 0x7f4bc015c710 2023-07-13T22:33:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.995+0000 7f4bb7fff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4bc006b420 0x7f4bc00d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.995+0000 7f4bbe59c700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc015c710 0x7f4bc00af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.995+0000 7f4bbed9d700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4bc00d0ca0 0x7f4bc006a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.60:32864/0 (socket says 172.21.15.60:32864) 2023-07-13T22:33:31.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.996+0000 7f4bbed9d700 1 -- 172.21.15.60:0/900035512 learned_addr learned my addr 172.21.15.60:0/900035512 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.996+0000 7f4bbdd9b700 1 -- 172.21.15.60:0/900035512 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824647582 0 0) 0x7f4bb8001490 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.996+0000 7f4bbdd9b700 1 -- 172.21.15.60:0/900035512 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bac0035b0 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.996+0000 7f4bb7fff700 1 -- 172.21.15.60:0/900035512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc015c710 msgr2=0x7f4bc00af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:31.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.996+0000 7f4bb7fff700 1 --2- 172.21.15.60:0/900035512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc015c710 0x7f4bc00af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.996+0000 7f4bb7fff700 1 -- 172.21.15.60:0/900035512 >> v1:172.21.15.105:6789/0 conn(0x7f4bc00d0ca0 legacy=0x7f4bc006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.996+0000 7f4bb7fff700 1 -- 172.21.15.60:0/900035512 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00b02b0 con 0x7f4bc006b420 2023-07-13T22:33:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.996+0000 7f4bbe59c700 1 --2- 172.21.15.60:0/900035512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc015c710 0x7f4bc00af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:33:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.996+0000 7f4bb7fff700 1 --2- 172.21.15.60:0/900035512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc006b420 0x7f4bc00d0760 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009840 tx=0x7f4ba8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97312e6fdf27730 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bbdd9b700 1 -- 172.21.15.60:0/900035512 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8011830 con 0x7f4bc006b420 2023-07-13T22:33:31.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bbdd9b700 1 -- 172.21.15.60:0/900035512 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba8011990 con 0x7f4bc006b420 2023-07-13T22:33:31.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bbdd9b700 1 -- 172.21.15.60:0/900035512 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8011d20 con 0x7f4bc006b420 2023-07-13T22:33:31.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bc4bff700 1 -- 172.21.15.60:0/900035512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc006b420 msgr2=0x7f4bc00d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bc4bff700 1 --2- 172.21.15.60:0/900035512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc006b420 0x7f4bc00d0760 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009840 tx=0x7f4ba8009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:31.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bc4bff700 1 -- 172.21.15.60:0/900035512 shutdown_connections 2023-07-13T22:33:31.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bc4bff700 1 --2- 172.21.15.60:0/900035512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc015c710 0x7f4bc00af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bc4bff700 1 --2- 172.21.15.60:0/900035512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc006b420 0x7f4bc00d0760 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bc4bff700 1 -- 172.21.15.60:0/900035512 >> 172.21.15.60:0/900035512 conn(0x7f4bc009bf20 msgr2=0x7f4bc00d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:31.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bc4bff700 1 -- 172.21.15.60:0/900035512 shutdown_connections 2023-07-13T22:33:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.997+0000 7f4bc4bff700 1 -- 172.21.15.60:0/900035512 wait complete. 2023-07-13T22:33:31.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.999+0000 7f4bc4bff700 1 Processor -- start 2023-07-13T22:33:31.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:31.999+0000 7f4bc4bff700 1 -- start start 2023-07-13T22:33:31.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.000+0000 7f4bc4bff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc006b420 0x7f4bc01f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.000+0000 7f4bc4bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00d0ca0 0x7f4bc015dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.000+0000 7f4bc4bff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc015c710 0x7f4bc015e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.000+0000 7f4bc4bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00a0710 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.000+0000 7f4bc4bff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc009fb00 con 0x7f4bc015c710 2023-07-13T22:33:31.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.000+0000 7f4bc4bff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc009fde0 con 0x7f4bc006b420 2023-07-13T22:33:31.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.000+0000 7f4bb7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00d0ca0 0x7f4bc015dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00d0ca0 0x7f4bc015dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50662/0 (socket says 172.21.15.60:50662) 2023-07-13T22:33:31.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb7fff700 1 -- 172.21.15.60:0/224377380 learned_addr learned my addr 172.21.15.60:0/224377380 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:31.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bbe59c700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc006b420 0x7f4bc01f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bbed9d700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc015c710 0x7f4bc015e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb7fff700 1 -- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc006b420 msgr2=0x7f4bc01f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb7fff700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4bc006b420 0x7f4bc01f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb7fff700 1 -- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc015c710 msgr2=0x7f4bc015e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:31.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb7fff700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc015c710 0x7f4bc015e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:31.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb7fff700 1 -- 172.21.15.60:0/224377380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00094a0 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb7fff700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00d0ca0 0x7f4bc015dcd0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80090b0 tx=0x7f4ba8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb6ffd700 1 -- 172.21.15.60:0/224377380 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8004590 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb6ffd700 1 -- 172.21.15.60:0/224377380 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba801f070 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.001+0000 7f4bb6ffd700 1 -- 172.21.15.60:0/224377380 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80038e0 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.002+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba80094a0 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.002+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc00a03a0 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.003+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4bc00b0870 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.003+0000 7f4bb6ffd700 1 -- 172.21.15.60:0/224377380 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ba8003a40 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.004+0000 7f4bb6ffd700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4ba006ace0 0x7f4ba006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.005+0000 7f4bb6ffd700 1 -- 172.21.15.60:0/224377380 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4ba8021070 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.005+0000 7f4bbe59c700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4ba006ace0 0x7f4ba006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.005+0000 7f4bb6ffd700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f4ba0071150 0x7f4ba0073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:31.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.005+0000 7f4bb6ffd700 1 -- 172.21.15.60:0/224377380 --> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ba0073c30 con 0x7f4ba0071150 2023-07-13T22:33:31.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.005+0000 7f4bb6ffd700 1 -- 172.21.15.60:0/224377380 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ba808ce00 con 0x7f4bc00d0ca0 2023-07-13T22:33:31.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.005+0000 7f4bbe59c700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4ba006ace0 0x7f4ba006d190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009cd0 tx=0x7f4bb0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.005+0000 7f4bbed9d700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f4ba0071150 0x7f4ba0073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:31.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.006+0000 7f4bbed9d700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f4ba0071150 0x7f4ba0073560 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:31.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.007+0000 7f4bb6ffd700 1 -- 172.21.15.60:0/224377380 <== osd.7 v2:172.21.15.105:6824/2508216201 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4ba0073c30 con 0x7f4ba0071150 2023-07-13T22:33:32.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.022+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 --> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4bc00a0580 con 0x7f4ba0071150 2023-07-13T22:33:32.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.022+0000 7f4bb6ffd700 1 -- 172.21.15.60:0/224377380 <== osd.7 v2:172.21.15.105:6824/2508216201 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4bc00a0580 con 0x7f4ba0071150 2023-07-13T22:33:32.008 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-07-13T22:33:32.009 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-13T22:33:32.009 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-07-13T22:33:32.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f4ba0071150 msgr2=0x7f4ba0073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.105:6824/2508216201,v1:172.21.15.105:6825/2508216201] conn(0x7f4ba0071150 0x7f4ba0073560 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4ba006ace0 msgr2=0x7f4ba006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f4ba006ace0 0x7f4ba006d190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009cd0 tx=0x7f4bb0009210 comp rx=0 tx=0).stop 2023-07-13T22:33:32.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00d0ca0 msgr2=0x7f4bc015dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 --2- 172.21.15.60:0/224377380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00d0ca0 0x7f4bc015dcd0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80090b0 tx=0x7f4ba8004750 comp rx=0 tx=0).stop 2023-07-13T22:33:32.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bbed9d700 1 -- 172.21.15.60:0/224377380 reap_dead start 2023-07-13T22:33:32.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 shutdown_connections 2023-07-13T22:33:32.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 >> 172.21.15.60:0/224377380 conn(0x7f4bc009bf20 msgr2=0x7f4bc00d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:32.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 shutdown_connections 2023-07-13T22:33:32.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.023+0000 7f4bc4bff700 1 -- 172.21.15.60:0/224377380 wait complete. 2023-07-13T22:33:32.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-07-13T22:33:32.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.245+0000 7f6fd65ff700 1 Processor -- start 2023-07-13T22:33:32.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.245+0000 7f6fd65ff700 1 -- start start 2023-07-13T22:33:32.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.246+0000 7f6fd65ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6fd00d07d0 0x7f6fd00d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.246+0000 7f6fd65ff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6fd00b2230 0x7f6fd006a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.246+0000 7f6fd65ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd008d030 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.246+0000 7f6fd65ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00af7f0 con 0x7f6fd00d07d0 2023-07-13T22:33:32.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.246+0000 7f6fd65ff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00d10e0 con 0x7f6fd00b2230 2023-07-13T22:33:32.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.246+0000 7f6fcffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6fd00d07d0 0x7f6fd00d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.246+0000 7f6fcffff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6fd00d07d0 0x7f6fd00d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50672/0 (socket says 172.21.15.60:50672) 2023-07-13T22:33:32.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.246+0000 7f6fcffff700 1 -- 172.21.15.60:0/727539196 learned_addr learned my addr 172.21.15.60:0/727539196 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:32.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.246+0000 7f6fd4b9c700 1 --1- 172.21.15.60:0/727539196 >> v1:172.21.15.60:6789/0 conn(0x7f6fd00b1ce0 0x7f6fd006a8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35730/0 (socket says 172.21.15.60:35730) 2023-07-13T22:33:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.247+0000 7f6fcf7fe700 1 --2- 172.21.15.60:0/727539196 >> v2:172.21.15.105:3300/0 conn(0x7f6fd00b2230 0x7f6fd006a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.247+0000 7f6fcffff700 1 -- 172.21.15.60:0/727539196 >> v2:172.21.15.105:3300/0 conn(0x7f6fd00b2230 msgr2=0x7f6fd006a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.247+0000 7f6fcffff700 1 --2- 172.21.15.60:0/727539196 >> v2:172.21.15.105:3300/0 conn(0x7f6fd00b2230 0x7f6fd006a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.247+0000 7f6fcffff700 1 -- 172.21.15.60:0/727539196 >> v1:172.21.15.60:6789/0 conn(0x7f6fd00b1ce0 legacy=0x7f6fd006a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.247+0000 7f6fcffff700 1 -- 172.21.15.60:0/727539196 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd015ca30 con 0x7f6fd00d07d0 2023-07-13T22:33:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.247+0000 7f6fcffff700 1 --2- 172.21.15.60:0/727539196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00d07d0 0x7f6fd00d0ba0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00d2b30 tx=0x7f6fc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502deb292c9d4db1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:32.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.247+0000 7f6fceffd700 1 -- 172.21.15.60:0/727539196 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc000b870 con 0x7f6fd00d07d0 2023-07-13T22:33:32.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.247+0000 7f6fceffd700 1 -- 172.21.15.60:0/727539196 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc000b9d0 con 0x7f6fd00d07d0 2023-07-13T22:33:32.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.247+0000 7f6fceffd700 1 -- 172.21.15.60:0/727539196 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc000bd60 con 0x7f6fd00d07d0 2023-07-13T22:33:32.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.248+0000 7f6fd65ff700 1 -- 172.21.15.60:0/727539196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00d07d0 msgr2=0x7f6fd00d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.248+0000 7f6fd65ff700 1 --2- 172.21.15.60:0/727539196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00d07d0 0x7f6fd00d0ba0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00d2b30 tx=0x7f6fc0009bd0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.248+0000 7f6fd65ff700 1 -- 172.21.15.60:0/727539196 shutdown_connections 2023-07-13T22:33:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.248+0000 7f6fd65ff700 1 --2- 172.21.15.60:0/727539196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00d07d0 0x7f6fd00d0ba0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.248+0000 7f6fd65ff700 1 --2- 172.21.15.60:0/727539196 >> v2:172.21.15.105:3300/0 conn(0x7f6fd00b2230 0x7f6fd006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.248+0000 7f6fd65ff700 1 -- 172.21.15.60:0/727539196 >> 172.21.15.60:0/727539196 conn(0x7f6fd009bf90 msgr2=0x7f6fd00d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.248+0000 7f6fd65ff700 1 -- 172.21.15.60:0/727539196 shutdown_connections 2023-07-13T22:33:32.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.248+0000 7f6fd65ff700 1 -- 172.21.15.60:0/727539196 wait complete. 2023-07-13T22:33:32.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.250+0000 7f6fd65ff700 1 Processor -- start 2023-07-13T22:33:32.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.250+0000 7f6fd65ff700 1 -- start start 2023-07-13T22:33:32.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fd65ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00b1ce0 0x7f6fd01f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fd65ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd00b2230 0x7f6fd00cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fd65ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd00d07d0 0x7f6fd0147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fd65ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00cde40 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fd65ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00cf8c0 con 0x7f6fd00b2230 2023-07-13T22:33:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fd65ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd01f58c0 con 0x7f6fd00d07d0 2023-07-13T22:33:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00b1ce0 0x7f6fd01f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00b1ce0 0x7f6fd01f5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50684/0 (socket says 172.21.15.60:50684) 2023-07-13T22:33:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fcffff700 1 -- 172.21.15.60:0/221726390 learned_addr learned my addr 172.21.15.60:0/221726390 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fcf7fe700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd00b2230 0x7f6fd00cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.251+0000 7f6fd4b9c700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd00d07d0 0x7f6fd0147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fcffff700 1 -- 172.21.15.60:0/221726390 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd00d07d0 msgr2=0x7f6fd0147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fcffff700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd00d07d0 0x7f6fd0147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fcffff700 1 -- 172.21.15.60:0/221726390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd00b2230 msgr2=0x7f6fd00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fcffff700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd00b2230 0x7f6fd00cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fcffff700 1 -- 172.21.15.60:0/221726390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc0009550 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fcf7fe700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd00b2230 0x7f6fd00cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fcffff700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00b1ce0 0x7f6fd01f5380 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0000c00 tx=0x7f6fc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fccff9700 1 -- 172.21.15.60:0/221726390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00129c0 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fccff9700 1 -- 172.21.15.60:0/221726390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc001e070 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd0148ad0 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fccff9700 1 -- 172.21.15.60:0/221726390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0012d40 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.252+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd00cf550 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.253+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd00cfba0 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.254+0000 7f6fccff9700 1 -- 172.21.15.60:0/221726390 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6fc001a420 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.255+0000 7f6fccff9700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fbc06ace0 0x7f6fbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.255+0000 7f6fccff9700 1 -- 172.21.15.60:0/221726390 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6fc008a2d0 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.256+0000 7f6fcf7fe700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fbc06ace0 0x7f6fbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.256+0000 7f6fcf7fe700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fbc06ace0 0x7f6fbc06d190 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0160050 tx=0x7f6fc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:32.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.260+0000 7f6fccff9700 1 -- 172.21.15.60:0/221726390 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f6fc0055690 con 0x7f6fd00b1ce0 2023-07-13T22:33:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.466+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fd006c0a0 con 0x7f6fbc06ace0 2023-07-13T22:33:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.466+0000 7f6fccff9700 1 -- 172.21.15.60:0/221726390 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6fc4009680 con 0x7f6fbc06ace0 2023-07-13T22:33:32.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fbc06ace0 msgr2=0x7f6fbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fbc06ace0 0x7f6fbc06d190 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0160050 tx=0x7f6fc4008040 comp rx=0 tx=0).stop 2023-07-13T22:33:32.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00b1ce0 msgr2=0x7f6fd01f5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00b1ce0 0x7f6fd01f5380 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0000c00 tx=0x7f6fc0004750 comp rx=0 tx=0).stop 2023-07-13T22:33:32.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 shutdown_connections 2023-07-13T22:33:32.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f6fbc06ace0 0x7f6fbc06d190 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00b1ce0 0x7f6fd01f5380 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fd00d07d0 0x7f6fd0147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 --2- 172.21.15.60:0/221726390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd00b2230 0x7f6fd00cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 >> 172.21.15.60:0/221726390 conn(0x7f6fd009bf90 msgr2=0x7f6fd00b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:32.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 shutdown_connections 2023-07-13T22:33:32.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.468+0000 7f6fd65ff700 1 -- 172.21.15.60:0/221726390 wait complete. 2023-07-13T22:33:32.460 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 7 to deep-scrub 2023-07-13T22:33:32.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:33:32.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.691+0000 7f7a9cdff700 1 Processor -- start 2023-07-13T22:33:32.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.691+0000 7f7a9cdff700 1 -- start start 2023-07-13T22:33:32.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.692+0000 7f7a9cdff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a980ca010 0x7f7a980ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.692+0000 7f7a9cdff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980ca970 con 0x7f7a98154950 2023-07-13T22:33:32.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.692+0000 7f7a9cdff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980cb620 con 0x7f7a980cafc0 2023-07-13T22:33:32.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.692+0000 7f7a9cdff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a980cb440 con 0x7f7a980ca010 2023-07-13T22:33:32.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.692+0000 7f7a96d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a980ca010 0x7f7a980ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.692+0000 7f7a96d9d700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a980ca010 0x7f7a980ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50686/0 (socket says 172.21.15.60:50686) 2023-07-13T22:33:32.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.692+0000 7f7a96d9d700 1 -- 172.21.15.60:0/2526241351 learned_addr learned my addr 172.21.15.60:0/2526241351 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:32.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.692+0000 7f7a9659c700 1 --1- 172.21.15.60:0/2526241351 >> v1:172.21.15.60:6789/0 conn(0x7f7a98154950 0x7f7a9815e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35738/0 (socket says 172.21.15.60:35738) 2023-07-13T22:33:32.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.693+0000 7f7a96d9d700 1 -- 172.21.15.60:0/2526241351 >> v1:172.21.15.105:6789/0 conn(0x7f7a980cafc0 legacy=0x7f7a9815b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.693+0000 7f7a96d9d700 1 -- 172.21.15.60:0/2526241351 >> v1:172.21.15.60:6789/0 conn(0x7f7a98154950 legacy=0x7f7a9815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.693+0000 7f7a96d9d700 1 -- 172.21.15.60:0/2526241351 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980cb900 con 0x7f7a980ca010 2023-07-13T22:33:32.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.693+0000 7f7a96d9d700 1 --2- 172.21.15.60:0/2526241351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a980ca010 0x7f7a980ca430 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009e50 tx=0x7f7a9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828c64bd14242716 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.693+0000 7f7a95d9b700 1 -- 172.21.15.60:0/2526241351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000e070 con 0x7f7a980ca010 2023-07-13T22:33:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.693+0000 7f7a95d9b700 1 -- 172.21.15.60:0/2526241351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9000b9c0 con 0x7f7a980ca010 2023-07-13T22:33:32.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.693+0000 7f7a95d9b700 1 -- 172.21.15.60:0/2526241351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000bd70 con 0x7f7a980ca010 2023-07-13T22:33:32.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.694+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2526241351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a980ca010 msgr2=0x7f7a980ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.694+0000 7f7a9cdff700 1 --2- 172.21.15.60:0/2526241351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a980ca010 0x7f7a980ca430 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009e50 tx=0x7f7a9000b330 comp rx=0 tx=0).stop 2023-07-13T22:33:32.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.694+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2526241351 shutdown_connections 2023-07-13T22:33:32.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.694+0000 7f7a9cdff700 1 --2- 172.21.15.60:0/2526241351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a980ca010 0x7f7a980ca430 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.694+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2526241351 >> 172.21.15.60:0/2526241351 conn(0x7f7a9806ab20 msgr2=0x7f7a980b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:32.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.694+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2526241351 shutdown_connections 2023-07-13T22:33:32.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.694+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2526241351 wait complete. 2023-07-13T22:33:32.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.696+0000 7f7a9cdff700 1 Processor -- start 2023-07-13T22:33:32.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.697+0000 7f7a9cdff700 1 -- start start 2023-07-13T22:33:32.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.698+0000 7f7a9cdff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a980ca010 0x7f7a980c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.698+0000 7f7a9cdff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a980cafc0 0x7f7a980b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.698+0000 7f7a9cdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a98154950 0x7f7a981519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.698+0000 7f7a9cdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980b3390 con 0x7f7a98154950 2023-07-13T22:33:32.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.698+0000 7f7a9cdff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a98151ef0 con 0x7f7a980cafc0 2023-07-13T22:33:32.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.698+0000 7f7a9cdff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980b35c0 con 0x7f7a980ca010 2023-07-13T22:33:32.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.698+0000 7f7a96d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a98154950 0x7f7a981519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.698+0000 7f7a96d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a98154950 0x7f7a981519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50692/0 (socket says 172.21.15.60:50692) 2023-07-13T22:33:32.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.698+0000 7f7a96d9d700 1 -- 172.21.15.60:0/2977049099 learned_addr learned my addr 172.21.15.60:0/2977049099 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:32.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a8ffff700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a980cafc0 0x7f7a980b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a9659c700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a980ca010 0x7f7a980c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a96d9d700 1 -- 172.21.15.60:0/2977049099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a980ca010 msgr2=0x7f7a980c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a96d9d700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a980ca010 0x7f7a980c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a96d9d700 1 -- 172.21.15.60:0/2977049099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a980cafc0 msgr2=0x7f7a980b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a96d9d700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a980cafc0 0x7f7a980b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a96d9d700 1 -- 172.21.15.60:0/2977049099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90009b30 con 0x7f7a98154950 2023-07-13T22:33:32.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a9659c700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a980ca010 0x7f7a980c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:33:32.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a96d9d700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a98154950 0x7f7a981519b0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f7a980721f0 tx=0x7f7a90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:32.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a8effd700 1 -- 172.21.15.60:0/2977049099 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90004960 con 0x7f7a98154950 2023-07-13T22:33:32.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a8effd700 1 -- 172.21.15.60:0/2977049099 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a900099b0 con 0x7f7a98154950 2023-07-13T22:33:32.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a8effd700 1 -- 172.21.15.60:0/2977049099 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9001c680 con 0x7f7a98154950 2023-07-13T22:33:32.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a980b1770 con 0x7f7a98154950 2023-07-13T22:33:32.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.699+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a980b1d30 con 0x7f7a98154950 2023-07-13T22:33:32.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.700+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a980741c0 con 0x7f7a98154950 2023-07-13T22:33:32.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.701+0000 7f7a8effd700 1 -- 172.21.15.60:0/2977049099 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a9001c7e0 con 0x7f7a98154950 2023-07-13T22:33:32.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.702+0000 7f7a8effd700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7806ace0 0x7f7a7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:32.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.702+0000 7f7a8effd700 1 -- 172.21.15.60:0/2977049099 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7a9008af80 con 0x7f7a98154950 2023-07-13T22:33:32.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.702+0000 7f7a9659c700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7806ace0 0x7f7a7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:32.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.703+0000 7f7a9659c700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7806ace0 0x7f7a7806d190 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7a88000f60 tx=0x7f7a88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:32.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.706+0000 7f7a8effd700 1 -- 172.21.15.60:0/2977049099 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7f7a90059e50 con 0x7f7a98154950 2023-07-13T22:33:32.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.910+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a980cbbe0 con 0x7f7a7806ace0 2023-07-13T22:33:32.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.913+0000 7f7a8effd700 1 -- 172.21.15.60:0/2977049099 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72947 (secure 0 0 0) 0x7f7a880058a0 con 0x7f7a7806ace0 2023-07-13T22:33:32.901 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:32.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7806ace0 msgr2=0x7f7a7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7806ace0 0x7f7a7806d190 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7a88000f60 tx=0x7f7a88009040 comp rx=0 tx=0).stop 2023-07-13T22:33:32.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a98154950 msgr2=0x7f7a981519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:32.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a98154950 0x7f7a981519b0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f7a980721f0 tx=0x7f7a90004ad0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 shutdown_connections 2023-07-13T22:33:32.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7f7a7806ace0 0x7f7a7806d190 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a98154950 0x7f7a981519b0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a980ca010 0x7f7a980c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 --2- 172.21.15.60:0/2977049099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a980cafc0 0x7f7a980b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:32.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 >> 172.21.15.60:0/2977049099 conn(0x7f7a9806ab20 msgr2=0x7f7a980ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:32.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 shutdown_connections 2023-07-13T22:33:32.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:32.915+0000 7f7a9cdff700 1 -- 172.21.15.60:0/2977049099 wait complete. 2023-07-13T22:33:32.908 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:33:32.919 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2023-07-13T22:33:31.292778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296064,"kb_used_data":14912,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678656,"statfs":{"total":773094113280,"available":772790943744,"internally_reserved":0,"allocated":15269888,"data_stored":7646292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.465506"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":232,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.281491+0000","last_change":"2023-07-13T22:20:48.495729+0000","last_active":"2023-07-13T22:33:21.281491+0000","last_peered":"2023-07-13T22:33:21.281491+0000","last_clean":"2023-07-13T22:33:21.281491+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:33:21.281491+0000","last_undegraded":"2023-07-13T22:33:21.281491+0000","last_fullsized":"2023-07-13T22:33:21.281491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":226,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.266974+0000","last_change":"2023-07-13T22:20:47.726654+0000","last_active":"2023-07-13T22:33:21.266974+0000","last_peered":"2023-07-13T22:33:21.266974+0000","last_clean":"2023-07-13T22:33:21.266974+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:33:21.266974+0000","last_undegraded":"2023-07-13T22:33:21.266974+0000","last_fullsized":"2023-07-13T22:33:21.266974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":254,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:30.141235+0000","last_change":"2023-07-13T22:33:30.141279+0000","last_active":"2023-07-13T22:33:30.141235+0000","last_peered":"2023-07-13T22:33:30.141235+0000","last_clean":"2023-07-13T22:33:30.141235+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:33:30.141235+0000","last_undegraded":"2023-07-13T22:33:30.141235+0000","last_fullsized":"2023-07-13T22:33:30.141235+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:33:30.141154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:33:30.141154+0000","last_clean_scrub_stamp":"2023-07-13T22:33:30.141154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:40.045273+0000","scrub_duration":0.002375659,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":369,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059670+0000","last_change":"2023-07-13T22:20:48.727245+0000","last_active":"2023-07-13T22:33:21.059670+0000","last_peered":"2023-07-13T22:33:21.059670+0000","last_clean":"2023-07-13T22:33:21.059670+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:33:21.059670+0000","last_undegraded":"2023-07-13T22:33:21.059670+0000","last_fullsized":"2023-07-13T22:33:21.059670+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:47.708813+0000","last_clean_scrub_stamp":"2023-07-13T22:20:47.708813+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:59:58.301151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":221,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059338+0000","last_change":"2023-07-13T22:20:48.445088+0000","last_active":"2023-07-13T22:33:21.059338+0000","last_peered":"2023-07-13T22:33:21.059338+0000","last_clean":"2023-07-13T22:33:21.059338+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:33:21.059338+0000","last_undegraded":"2023-07-13T22:33:21.059338+0000","last_fullsized":"2023-07-13T22:33:21.059338+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:04:24.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":221,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059698+0000","last_change":"2023-07-13T22:20:48.445513+0000","last_active":"2023-07-13T22:33:21.059698+0000","last_peered":"2023-07-13T22:33:21.059698+0000","last_clean":"2023-07-13T22:33:21.059698+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:33:21.059698+0000","last_undegraded":"2023-07-13T22:33:21.059698+0000","last_fullsized":"2023-07-13T22:33:21.059698+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:02:44.690690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":256,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.281428+0000","last_change":"2023-07-13T22:20:48.444932+0000","last_active":"2023-07-13T22:33:21.281428+0000","last_peered":"2023-07-13T22:33:21.281428+0000","last_clean":"2023-07-13T22:33:21.281428+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:33:21.281428+0000","last_undegraded":"2023-07-13T22:33:21.281428+0000","last_fullsized":"2023-07-13T22:33:21.281428+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":222,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059401+0000","last_change":"2023-07-13T22:20:48.445041+0000","last_active":"2023-07-13T22:33:21.059401+0000","last_peered":"2023-07-13T22:33:21.059401+0000","last_clean":"2023-07-13T22:33:21.059401+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:33:21.059401+0000","last_undegraded":"2023-07-13T22:33:21.059401+0000","last_fullsized":"2023-07-13T22:33:21.059401+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:41:20.284345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":222,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:21.059650+0000","last_change":"2023-07-13T22:20:47.725306+0000","last_active":"2023-07-13T22:33:21.059650+0000","last_peered":"2023-07-13T22:33:21.059650+0000","last_clean":"2023-07-13T22:33:21.059650+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:33:21.059650+0000","last_undegraded":"2023-07-13T22:33:21.059650+0000","last_fullsized":"2023-07-13T22:33:21.059650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:20:46.706093+0000","last_clean_scrub_stamp":"2023-07-13T22:20:46.706093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:03:32.536211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37444,"kb_used_data":2300,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334396,"statfs":{"total":96636764160,"available":96598421504,"internally_reserved":0,"allocated":2355200,"data_stored":1398809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.39500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":1.115},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68799999999999994,"15min":0.68100000000000005},"min":{"1min":0.58399999999999996,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.91300000000000003,"5min":0.995,"15min":0.995},"last":0.74399999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.85599999999999998,"5min":0.88200000000000001,"15min":2.2610000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.34300000000000003},"max":{"1min":0.93700000000000006,"5min":1.0149999999999999,"15min":1.0609999999999999},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.73699999999999999},"min":{"1min":0.51000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88500000000000001,"5min":0.95599999999999996,"15min":3.5190000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.70699999999999996,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.36499999999999999},"max":{"1min":0.88200000000000001,"5min":1.028,"15min":3.4710000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.50700000000000001,"5min":0.41499999999999998,"15min":0.34599999999999997},"max":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":1.0489999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.61499999999999999,"5min":0.51200000000000001,"15min":0.48499999999999999},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1399999999999999},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.627,"15min":0.628},"min":{"1min":0.434,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.94799999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.1850000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.64500000000000002},"min":{"1min":0.439,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":1.0329999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.57799999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":1.0149999999999999,"15min":1.2110000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.0549999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.496,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":0.99099999999999999,"15min":1.042},"last":0.60999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.86299999999999999,"5min":1.089,"15min":1.089},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.65400000000000003},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":0.88500000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.40400000000000003,"15min":0.33900000000000002},"max":{"1min":0.90500000000000003,"5min":1.075,"15min":1.075},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.379},"max":{"1min":0.92800000000000005,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.47699999999999998,"5min":0.46100000000000002,"15min":0.35599999999999998},"max":{"1min":0.86699999999999999,"5min":1.042,"15min":1.042},"last":0.79700000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.54000000000000004,"5min":0.435,"15min":0.37},"max":{"1min":0.85299999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":1.123,"15min":1.123},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.49399999999999999,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.66400000000000003},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.84799999999999998,"5min":0.90200000000000002,"15min":0.95999999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.5,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94299999999999995,"5min":1.121,"15min":1.121},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.375},"max":{"1min":0.96999999999999997,"5min":1.1639999999999999,"15min":1.1890000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72899999999999998,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.39000000000000001},"max":{"1min":0.95399999999999996,"5min":1.071,"15min":1.111},"last":0.82999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575004,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37444,"kb_used_data":2300,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334396,"statfs":{"total":96636764160,"available":96598421504,"internally_reserved":0,"allocated":2355200,"data_stored":1398809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.45900000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.249},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.952},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.499},"min":{"1min":0.308,"5min":0.29199999999999998,"15min":0.254},"max":{"1min":0.73199999999999998,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.46600000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":0.752,"15min":2.1200000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":1.9810000000000001},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":2.008},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.53600000000000003},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80400000000000005,"5min":0.93700000000000006,"15min":1.054},"last":0.58399999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95999999999999996,"5min":1.1870000000000001,"15min":2.028},"last":0.749},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.76700000000000002},"min":{"1min":0.622,"5min":0.55500000000000005,"15min":0.46400000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.9219999999999999},"last":0.88800000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.751,"15min":0.75800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.44},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76100000000000001,"15min":0.79900000000000004},"min":{"1min":0.52800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.03,"5min":1.0720000000000001,"15min":2.0899999999999999},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.52600000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.2230000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77000000000000002,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":2.0609999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77100000000000002,"15min":0.79800000000000004},"min":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.50700000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.8700000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2609999999999999},"last":0.79900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575005,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":807455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":11.791},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.47799999999999998,"15min":0.51700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":4.657},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57099999999999995,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84999999999999998,"5min":1.056,"15min":4.5389999999999997},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.504,"15min":0.74099999999999999},"min":{"1min":0.34999999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83899999999999997,"5min":1,"15min":13.398999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":1.8089999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54100000000000004,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.85299999999999998,"5min":1.095,"15min":4.4530000000000003},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":13.667999999999999},"last":0.63},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.77000000000000002},"min":{"1min":0.63,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.776},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76600000000000001,"15min":1.0029999999999999},"min":{"1min":0.63700000000000001,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":13.349},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.79600000000000004,"15min":0.84499999999999997},"min":{"1min":0.73199999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":4.7649999999999997},"last":0.72599999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.67500000000000004,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.202,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.747,"15min":0.96899999999999997},"min":{"1min":0.60399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0209999999999999,"5min":1.046,"15min":11.679},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.76700000000000002,"15min":1.0069999999999999},"min":{"1min":0.69599999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96499999999999997,"5min":1.032,"15min":13.298},"last":0.753},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.79200000000000004,"15min":0.85899999999999999},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.50900000000000001},"max":{"1min":1.119,"5min":1.119,"15min":11.456},"last":0.66200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":807455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.434,"15min":0.47899999999999998},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.65100000000000002,"5min":1.605,"15min":2.282},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.29099999999999998},"max":{"1min":0.68300000000000005,"5min":1.034,"15min":1.4990000000000001},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.59899999999999998,"5min":1.228,"15min":1.228},"last":0.498},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.6879999999999999,"15min":2.319},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.439,"5min":0.371,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":1.119,"15min":1.119},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.60499999999999998},"min":{"1min":0.47699999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"max":{"1min":0.71999999999999997,"5min":1.5149999999999999,"15min":2.351},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76000000000000001,"15min":0.79500000000000004},"min":{"1min":0.63500000000000001,"5min":0.56699999999999995,"15min":0.46000000000000002},"max":{"1min":1.0289999999999999,"5min":1.6499999999999999,"15min":2.238},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.51200000000000001,"15min":0.46899999999999997},"max":{"1min":1.0920000000000001,"5min":1.1140000000000001,"15min":1.518},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.77000000000000002},"min":{"1min":0.621,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":1.341,"15min":1.341},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.78200000000000003,"15min":0.81799999999999995},"min":{"1min":0.64900000000000002,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.9099999999999999,"15min":2.1909999999999998},"last":0.752}]},{"osd":6,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.66300000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":1.107,"15min":1.488},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.63600000000000001,"5min":0.52300000000000002,"15min":0.48199999999999998},"max":{"1min":0.98599999999999999,"5min":1.448,"15min":1.448},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.77500000000000002,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.29},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.73199999999999998,"5min":0.57599999999999996,"15min":0.54400000000000004},"max":{"1min":1.0840000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.82699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40960,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330880,"statfs":{"total":96636764160,"available":96594821120,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.248},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.73999999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.32700000000000001,"5min":0.32400000000000001,"15min":0.27200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.503,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.873,"15min":0.873},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35099999999999998},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.94299999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.46400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82099999999999995,"5min":0.90000000000000002,"15min":1.0409999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.57499999999999996,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.2250000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.1539999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.745,"15min":0.749},"min":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.496},"max":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.411},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.48599999999999999},"max":{"1min":0.996,"5min":1.05,"15min":1.5469999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.621,"5min":0.53400000000000003,"15min":0.51800000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.1559999999999999},"last":0.623},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.621,"5min":0.52500000000000002,"15min":0.52100000000000002},"max":{"1min":0.97499999999999998,"5min":0.996,"15min":1.1240000000000001},"last":0.68500000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.78200000000000003},"min":{"1min":0.60699999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.054,"5min":1.054,"15min":1.1779999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.47199999999999998},"max":{"1min":1.0389999999999999,"5min":1.0800000000000001,"15min":1.1950000000000001},"last":0.66800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36864,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334976,"statfs":{"total":96636764160,"available":96599015424,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.64800000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.96799999999999997,"15min":0.997},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.58799999999999997,"5min":0.44600000000000001,"15min":0.35099999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.298},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.45900000000000002,"15min":0.438},"max":{"1min":0.96899999999999997,"5min":1.0529999999999999,"15min":1.3109999999999999},"last":0.79500000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.371},"max":{"1min":0.94799999999999995,"5min":0.95099999999999996,"15min":1.1200000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.51400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90800000000000003,"5min":1.1279999999999999,"15min":1.1839999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":0.80200000000000005},"min":{"1min":0.495,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":1.234,"15min":10.657999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72399999999999998,"15min":0.81799999999999995},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":8.75},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.69399999999999995,"15min":0.71399999999999997},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81899999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.877}]},{"osd":6,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.81699999999999995,"5min":1.012,"15min":1.173},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.70099999999999996,"15min":0.70999999999999996},"min":{"1min":0.48699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84399999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.432},"max":{"1min":0.86299999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.56000000000000005,"5min":0.432,"15min":0.432},"max":{"1min":0.84899999999999998,"5min":1.0600000000000001,"15min":1.212},"last":1.0149999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36864,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334976,"statfs":{"total":96636764160,"available":96599015424,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":1.042},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88500000000000001,"5min":0.92000000000000004,"15min":0.998},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.68100000000000005,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.68500000000000005},"min":{"1min":0.52600000000000002,"5min":0.34999999999999998,"15min":0.33100000000000002},"max":{"1min":0.90300000000000002,"5min":1.0129999999999999,"15min":8.9930000000000003},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96199999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.55100000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.86799999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.59199999999999997,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.91300000000000003,"5min":0.96499999999999997,"15min":0.96599999999999997},"last":0.624},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.70699999999999996},"min":{"1min":0.621,"5min":0.48399999999999999,"15min":0.47899999999999998},"max":{"1min":0.93899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.95699999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70699999999999996,"15min":0.68000000000000005},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88300000000000001,"5min":1.012,"15min":1.012},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72699999999999998,"15min":0.70999999999999996},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.42299999999999999},"max":{"1min":0.86499999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.81000000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.74399999999999999,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.41199999999999998},"max":{"1min":0.93799999999999994,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:33:32.924 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-07-13T22:20:46.706093+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=20, tm_sec=46, tm_wday=3, tm_yday=194, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=33, tm_sec=25, tm_wday=3, tm_yday=194, tm_isdst=0) 2023-07-13T22:33:32.925 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-07-13T22:20:46.706093+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=20, tm_sec=46, tm_wday=3, tm_yday=194, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=33, tm_sec=25, tm_wday=3, tm_yday=194, tm_isdst=0) 2023-07-13T22:33:32.926 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-07-13T22:20:47.708813+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=20, tm_sec=47, tm_wday=3, tm_yday=194, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=33, tm_sec=25, tm_wday=3, tm_yday=194, tm_isdst=0) 2023-07-13T22:33:32.926 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-07-13T22:20:46.706093+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=20, tm_sec=46, tm_wday=3, tm_yday=194, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=33, tm_sec=25, tm_wday=3, tm_yday=194, tm_isdst=0) 2023-07-13T22:33:32.926 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-07-13T22:20:46.706093+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=20, tm_sec=46, tm_wday=3, tm_yday=194, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=33, tm_sec=25, tm_wday=3, tm_yday=194, tm_isdst=0) 2023-07-13T22:33:32.927 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-07-13T22:20:46.706093+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=20, tm_sec=46, tm_wday=3, tm_yday=194, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=33, tm_sec=25, tm_wday=3, tm_yday=194, tm_isdst=0) 2023-07-13T22:33:32.927 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-07-13T22:20:46.706093+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=20, tm_sec=46, tm_wday=3, tm_yday=194, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=33, tm_sec=25, tm_wday=3, tm_yday=194, tm_isdst=0) 2023-07-13T22:33:32.927 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-07-13T22:20:46.706093+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=20, tm_sec=46, tm_wday=3, tm_yday=194, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=13, tm_hour=22, tm_min=33, tm_sec=25, tm_wday=3, tm_yday=194, tm_isdst=0) 2023-07-13T22:33:32.927 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-13T22:33:52.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:33:53.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.156+0000 7fee31dff700 1 Processor -- start 2023-07-13T22:33:53.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.157+0000 7fee31dff700 1 -- start start 2023-07-13T22:33:53.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee31dff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fee2c154950 0x7fee2c0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee31dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 0x7fee2c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee31dff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fee2c15ea90 0x7fee2c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee31dff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c0cb620 con 0x7fee2c15ea90 2023-07-13T22:33:53.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee31dff700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c0cb900 con 0x7fee2c154950 2023-07-13T22:33:53.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee31dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0cbbe0 con 0x7fee2c15e640 2023-07-13T22:33:53.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee2bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fee2c15ea90 0x7fee2c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee2bfff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fee2c15ea90 0x7fee2c0ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60888/0 (socket says 172.21.15.60:60888) 2023-07-13T22:33:53.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee2bfff700 1 -- 172.21.15.60:0/2637903952 learned_addr learned my addr 172.21.15.60:0/2637903952 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee2affd700 1 --2- 172.21.15.60:0/2637903952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 0x7fee2c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.158+0000 7fee2b7fe700 1 --2- 172.21.15.60:0/2637903952 >> v2:172.21.15.105:3300/0 conn(0x7fee2c154950 0x7fee2c0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2bfff700 1 -- 172.21.15.60:0/2637903952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 msgr2=0x7fee2c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2bfff700 1 --2- 172.21.15.60:0/2637903952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 0x7fee2c15b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2bfff700 1 -- 172.21.15.60:0/2637903952 >> v2:172.21.15.105:3300/0 conn(0x7fee2c154950 msgr2=0x7fee2c0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:33:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2bfff700 1 --2- 172.21.15.60:0/2637903952 >> v2:172.21.15.105:3300/0 conn(0x7fee2c154950 0x7fee2c0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2bfff700 1 -- 172.21.15.60:0/2637903952 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c15a850 con 0x7fee2c15ea90 2023-07-13T22:33:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2b7fe700 1 --2- 172.21.15.60:0/2637903952 >> v2:172.21.15.105:3300/0 conn(0x7fee2c154950 0x7fee2c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2bfff700 1 --2- 172.21.15.60:0/2637903952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 0x7fee2c0ca010 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0721c0 tx=0x7fee2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf67c2678c7c8da7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2a7fc700 1 -- 172.21.15.60:0/2637903952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400e070 con 0x7fee2c15ea90 2023-07-13T22:33:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2a7fc700 1 -- 172.21.15.60:0/2637903952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2400b9c0 con 0x7fee2c15ea90 2023-07-13T22:33:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.159+0000 7fee2a7fc700 1 -- 172.21.15.60:0/2637903952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400bd70 con 0x7fee2c15ea90 2023-07-13T22:33:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.160+0000 7fee31dff700 1 -- 172.21.15.60:0/2637903952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 msgr2=0x7fee2c0ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.160+0000 7fee31dff700 1 --2- 172.21.15.60:0/2637903952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 0x7fee2c0ca010 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0721c0 tx=0x7fee2400b330 comp rx=0 tx=0).stop 2023-07-13T22:33:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.160+0000 7fee31dff700 1 -- 172.21.15.60:0/2637903952 shutdown_connections 2023-07-13T22:33:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.160+0000 7fee31dff700 1 --2- 172.21.15.60:0/2637903952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 0x7fee2c0ca010 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.160+0000 7fee31dff700 1 --2- 172.21.15.60:0/2637903952 >> v2:172.21.15.105:3300/0 conn(0x7fee2c154950 0x7fee2c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.160+0000 7fee31dff700 1 --2- 172.21.15.60:0/2637903952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 0x7fee2c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.160+0000 7fee31dff700 1 -- 172.21.15.60:0/2637903952 >> 172.21.15.60:0/2637903952 conn(0x7fee2c06ab20 msgr2=0x7fee2c0b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.160+0000 7fee31dff700 1 -- 172.21.15.60:0/2637903952 shutdown_connections 2023-07-13T22:33:53.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.160+0000 7fee31dff700 1 -- 172.21.15.60:0/2637903952 wait complete. 2023-07-13T22:33:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.162+0000 7fee31dff700 1 Processor -- start 2023-07-13T22:33:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.162+0000 7fee31dff700 1 -- start start 2023-07-13T22:33:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee31dff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee2c154950 0x7fee2c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee31dff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 0x7fee2c1e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee31dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 0x7fee2c150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee31dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0c6710 con 0x7fee2c15ea90 2023-07-13T22:33:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee31dff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0c69f0 con 0x7fee2c154950 2023-07-13T22:33:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee31dff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0b2830 con 0x7fee2c15e640 2023-07-13T22:33:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee2bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 0x7fee2c150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee2bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 0x7fee2c150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60904/0 (socket says 172.21.15.60:60904) 2023-07-13T22:33:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee2bfff700 1 -- 172.21.15.60:0/2751515422 learned_addr learned my addr 172.21.15.60:0/2751515422 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee2affd700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 0x7fee2c1e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.163+0000 7fee2b7fe700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee2c154950 0x7fee2c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee2bfff700 1 -- 172.21.15.60:0/2751515422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 msgr2=0x7fee2c1e58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee2bfff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 0x7fee2c1e58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee2bfff700 1 -- 172.21.15.60:0/2751515422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee2c154950 msgr2=0x7fee2c0c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee2bfff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee2c154950 0x7fee2c0c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee2bfff700 1 -- 172.21.15.60:0/2751515422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24009b30 con 0x7fee2c15ea90 2023-07-13T22:33:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee2affd700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 0x7fee2c1e58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:33:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee2bfff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 0x7fee2c150470 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fee24012000 tx=0x7fee24004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee13fff700 1 -- 172.21.15.60:0/2751515422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee240099b0 con 0x7fee2c15ea90 2023-07-13T22:33:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee13fff700 1 -- 172.21.15.60:0/2751515422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee24004250 con 0x7fee2c15ea90 2023-07-13T22:33:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee13fff700 1 -- 172.21.15.60:0/2751515422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24004600 con 0x7fee2c15ea90 2023-07-13T22:33:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c0b2b10 con 0x7fee2c15ea90 2023-07-13T22:33:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.164+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c0c62e0 con 0x7fee2c15ea90 2023-07-13T22:33:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.165+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c0c64d0 con 0x7fee2c15ea90 2023-07-13T22:33:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.166+0000 7fee13fff700 1 -- 172.21.15.60:0/2751515422 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee24004760 con 0x7fee2c15ea90 2023-07-13T22:33:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.167+0000 7fee13fff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fee1806ace0 0x7fee1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.167+0000 7fee13fff700 1 -- 172.21.15.60:0/2751515422 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fee2408a2f0 con 0x7fee2c15ea90 2023-07-13T22:33:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.167+0000 7fee2b7fe700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fee1806ace0 0x7fee1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.168+0000 7fee2b7fe700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fee1806ace0 0x7fee1806d190 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0045f0 tx=0x7fee1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:53.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.172+0000 7fee13fff700 1 -- 172.21.15.60:0/2751515422 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fee2c0c64d0 con 0x7fee2c15ea90 2023-07-13T22:33:53.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.373+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 --> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee2c0c6b60 con 0x7fee1806ace0 2023-07-13T22:33:53.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.376+0000 7fee13fff700 1 -- 172.21.15.60:0/2751515422 <== mgr.4100 v2:172.21.15.60:6832/3777735812 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73148 (secure 0 0 0) 0x7fee2c0c6b60 con 0x7fee1806ace0 2023-07-13T22:33:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:53.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fee1806ace0 msgr2=0x7fee1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fee1806ace0 0x7fee1806d190 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0045f0 tx=0x7fee1c008040 comp rx=0 tx=0).stop 2023-07-13T22:33:53.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 msgr2=0x7fee2c150470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 0x7fee2c150470 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fee24012000 tx=0x7fee24004ad0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 shutdown_connections 2023-07-13T22:33:53.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fee1806ace0 0x7fee1806d190 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee2c15ea90 0x7fee2c150470 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee2c154950 0x7fee2c0c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 --2- 172.21.15.60:0/2751515422 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee2c15e640 0x7fee2c1e58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 >> 172.21.15.60:0/2751515422 conn(0x7fee2c06ab20 msgr2=0x7fee2c0b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:53.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 shutdown_connections 2023-07-13T22:33:53.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.378+0000 7fee31dff700 1 -- 172.21.15.60:0/2751515422 wait complete. 2023-07-13T22:33:53.368 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-07-13T22:33:53.383 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2023-07-13T22:33:53.297817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296064,"kb_used_data":14912,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678656,"statfs":{"total":773094113280,"available":772790943744,"internally_reserved":0,"allocated":15269888,"data_stored":7646292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002682"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":240,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:31.602188+0000","last_change":"2023-07-13T22:33:31.602247+0000","last_active":"2023-07-13T22:33:31.602188+0000","last_peered":"2023-07-13T22:33:31.602188+0000","last_clean":"2023-07-13T22:33:31.602188+0000","last_became_active":"2023-07-13T22:20:48.495452+0000","last_became_peered":"2023-07-13T22:20:48.495452+0000","last_unstale":"2023-07-13T22:33:31.602188+0000","last_undegraded":"2023-07-13T22:33:31.602188+0000","last_fullsized":"2023-07-13T22:33:31.602188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:33:31.602095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:33:31.602095+0000","last_clean_scrub_stamp":"2023-07-13T22:33:31.602095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:50:58.362040+0000","scrub_duration":0.0028910839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":234,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:34.716953+0000","last_change":"2023-07-13T22:33:34.717007+0000","last_active":"2023-07-13T22:33:34.716953+0000","last_peered":"2023-07-13T22:33:34.716953+0000","last_clean":"2023-07-13T22:33:34.716953+0000","last_became_active":"2023-07-13T22:20:47.726275+0000","last_became_peered":"2023-07-13T22:20:47.726275+0000","last_unstale":"2023-07-13T22:33:34.716953+0000","last_undegraded":"2023-07-13T22:33:34.716953+0000","last_fullsized":"2023-07-13T22:33:34.716953+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:33:34.716848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:33:34.716848+0000","last_clean_scrub_stamp":"2023-07-13T22:33:34.716848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:09:52.035435+0000","scrub_duration":0.0030020110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":254,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:30.141235+0000","last_change":"2023-07-13T22:33:30.141279+0000","last_active":"2023-07-13T22:33:30.141235+0000","last_peered":"2023-07-13T22:33:30.141235+0000","last_clean":"2023-07-13T22:33:30.141235+0000","last_became_active":"2023-07-13T22:20:47.725658+0000","last_became_peered":"2023-07-13T22:20:47.725658+0000","last_unstale":"2023-07-13T22:33:30.141235+0000","last_undegraded":"2023-07-13T22:33:30.141235+0000","last_fullsized":"2023-07-13T22:33:30.141235+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:33:30.141154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:33:30.141154+0000","last_clean_scrub_stamp":"2023-07-13T22:33:30.141154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:40.045273+0000","scrub_duration":0.002375659,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"140'119","reported_seq":380,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:41.478691+0000","last_change":"2023-07-13T22:33:41.478754+0000","last_active":"2023-07-13T22:33:41.478691+0000","last_peered":"2023-07-13T22:33:41.478691+0000","last_clean":"2023-07-13T22:33:41.478691+0000","last_became_active":"2023-07-13T22:20:48.726207+0000","last_became_peered":"2023-07-13T22:20:48.726207+0000","last_unstale":"2023-07-13T22:33:41.478691+0000","last_undegraded":"2023-07-13T22:33:41.478691+0000","last_fullsized":"2023-07-13T22:33:41.478691+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'119","last_scrub_stamp":"2023-07-13T22:33:41.478599+0000","last_deep_scrub":"140'119","last_deep_scrub_stamp":"2023-07-13T22:33:41.478599+0000","last_clean_scrub_stamp":"2023-07-13T22:33:41.478599+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:06:38.216441+0000","scrub_duration":0.0060911409999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":230,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:39.535044+0000","last_change":"2023-07-13T22:33:39.535108+0000","last_active":"2023-07-13T22:33:39.535044+0000","last_peered":"2023-07-13T22:33:39.535044+0000","last_clean":"2023-07-13T22:33:39.535044+0000","last_became_active":"2023-07-13T22:20:48.444608+0000","last_became_peered":"2023-07-13T22:20:48.444608+0000","last_unstale":"2023-07-13T22:33:39.535044+0000","last_undegraded":"2023-07-13T22:33:39.535044+0000","last_fullsized":"2023-07-13T22:33:39.535044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:33:39.534928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:33:39.534928+0000","last_clean_scrub_stamp":"2023-07-13T22:33:39.534928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:11:56.186143+0000","scrub_duration":0.0029007289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":230,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:40.519934+0000","last_change":"2023-07-13T22:33:40.519984+0000","last_active":"2023-07-13T22:33:40.519934+0000","last_peered":"2023-07-13T22:33:40.519934+0000","last_clean":"2023-07-13T22:33:40.519934+0000","last_became_active":"2023-07-13T22:20:48.444763+0000","last_became_peered":"2023-07-13T22:20:48.444763+0000","last_unstale":"2023-07-13T22:33:40.519934+0000","last_undegraded":"2023-07-13T22:33:40.519934+0000","last_fullsized":"2023-07-13T22:33:40.519934+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:33:40.519843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:33:40.519843+0000","last_clean_scrub_stamp":"2023-07-13T22:33:40.519843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:51:07.763395+0000","scrub_duration":0.0022149439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":264,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:32.127129+0000","last_change":"2023-07-13T22:33:32.127181+0000","last_active":"2023-07-13T22:33:32.127129+0000","last_peered":"2023-07-13T22:33:32.127129+0000","last_clean":"2023-07-13T22:33:32.127129+0000","last_became_active":"2023-07-13T22:20:48.444594+0000","last_became_peered":"2023-07-13T22:20:48.444594+0000","last_unstale":"2023-07-13T22:33:32.127129+0000","last_undegraded":"2023-07-13T22:33:32.127129+0000","last_fullsized":"2023-07-13T22:33:32.127129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:33:32.127007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:33:32.127007+0000","last_clean_scrub_stamp":"2023-07-13T22:33:32.127007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:34:43.766548+0000","scrub_duration":0.0026668909999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":232,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:40.990819+0000","last_change":"2023-07-13T22:33:40.990819+0000","last_active":"2023-07-13T22:33:40.990819+0000","last_peered":"2023-07-13T22:33:40.990819+0000","last_clean":"2023-07-13T22:33:40.990819+0000","last_became_active":"2023-07-13T22:20:48.444618+0000","last_became_peered":"2023-07-13T22:20:48.444618+0000","last_unstale":"2023-07-13T22:33:40.990819+0000","last_undegraded":"2023-07-13T22:33:40.990819+0000","last_fullsized":"2023-07-13T22:33:40.990819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:33:40.990650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:33:40.990650+0000","last_clean_scrub_stamp":"2023-07-13T22:33:40.990650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:45:20.952687+0000","scrub_duration":0.002722637,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":232,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-13T22:33:38.990715+0000","last_change":"2023-07-13T22:33:38.990715+0000","last_active":"2023-07-13T22:33:38.990715+0000","last_peered":"2023-07-13T22:33:38.990715+0000","last_clean":"2023-07-13T22:33:38.990715+0000","last_became_active":"2023-07-13T22:20:47.724950+0000","last_became_peered":"2023-07-13T22:20:47.724950+0000","last_unstale":"2023-07-13T22:33:38.990715+0000","last_undegraded":"2023-07-13T22:33:38.990715+0000","last_fullsized":"2023-07-13T22:33:38.990715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:33:38.990559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:33:38.990559+0000","last_clean_scrub_stamp":"2023-07-13T22:33:38.990559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:29:12.015639+0000","scrub_duration":0.002714424,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37444,"kb_used_data":2300,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334396,"statfs":{"total":96636764160,"available":96598421504,"internally_reserved":0,"allocated":2355200,"data_stored":1398809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.39500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":1.115},"last":0.87},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68799999999999994,"15min":0.68100000000000005},"min":{"1min":0.58399999999999996,"5min":0.40699999999999997,"15min":0.30099999999999999},"max":{"1min":0.91300000000000003,"5min":0.995,"15min":0.995},"last":0.748}]},{"osd":1,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.85599999999999998,"5min":0.88200000000000001,"15min":2.2610000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.34300000000000003},"max":{"1min":0.93700000000000006,"5min":1.0149999999999999,"15min":1.0609999999999999},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.73699999999999999},"min":{"1min":0.51000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88500000000000001,"5min":0.95599999999999996,"15min":3.5190000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.70699999999999996,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.36499999999999999},"max":{"1min":0.88200000000000001,"5min":1.028,"15min":3.4710000000000001},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.50700000000000001,"5min":0.41499999999999998,"15min":0.34599999999999997},"max":{"1min":0.91400000000000003,"5min":0.92000000000000004,"15min":1.0489999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.61499999999999999,"5min":0.51200000000000001,"15min":0.48499999999999999},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1399999999999999},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.627,"15min":0.628},"min":{"1min":0.434,"5min":0.36899999999999999,"15min":0.34899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.94799999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.1850000000000001},"last":0.72699999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.626,"15min":0.64500000000000002},"min":{"1min":0.439,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":1.0329999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.57799999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":1.0149999999999999,"15min":1.2110000000000001},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:33:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.0549999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.496,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":0.99099999999999999,"15min":1.042},"last":0.56200000000000006}]}]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28799999999999998},"max":{"1min":0.86299999999999999,"5min":1.089,"15min":1.089},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.65400000000000003},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.33500000000000002},"max":{"1min":0.88500000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.80500000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.40400000000000003,"15min":0.33900000000000002},"max":{"1min":0.90500000000000003,"5min":1.075,"15min":1.075},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.78400000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.379},"max":{"1min":0.92800000000000005,"5min":1.0589999999999999,"15min":1.0700000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.47699999999999998,"5min":0.46100000000000002,"15min":0.35599999999999998},"max":{"1min":0.86699999999999999,"5min":1.042,"15min":1.042},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.54000000000000004,"5min":0.435,"15min":0.37},"max":{"1min":0.85299999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":1.123,"15min":1.123},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.48199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.49399999999999999,"5min":0.376,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.66400000000000003},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.84799999999999998,"5min":0.90200000000000002,"15min":0.95999999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76100000000000001,"15min":0.754},"min":{"1min":0.5,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94299999999999995,"5min":1.121,"15min":1.121},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.45300000000000001,"5min":0.439,"15min":0.375},"max":{"1min":0.96999999999999997,"5min":1.1639999999999999,"15min":1.1890000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72899999999999998,"15min":0.72799999999999998},"min":{"1min":0.49199999999999999,"5min":0.45300000000000001,"15min":0.39000000000000001},"max":{"1min":0.95399999999999996,"5min":1.071,"15min":1.111},"last":0.63100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37444,"kb_used_data":2300,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334396,"statfs":{"total":96636764160,"available":96598421504,"internally_reserved":0,"allocated":2355200,"data_stored":1398809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.45900000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.249},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.952},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.499},"min":{"1min":0.308,"5min":0.29199999999999998,"15min":0.254},"max":{"1min":0.73199999999999998,"5min":0.79900000000000004,"15min":1.1060000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29199999999999998},"max":{"1min":0.752,"5min":0.752,"15min":2.1200000000000001},"last":0.628},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35099999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":1.9810000000000001},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":2.008},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.53600000000000003},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80400000000000005,"5min":0.93700000000000006,"15min":1.054},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95999999999999996,"5min":1.1870000000000001,"15min":2.028},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.76700000000000002},"min":{"1min":0.622,"5min":0.55500000000000005,"15min":0.46400000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.9219999999999999},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.751,"15min":0.75800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.44},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.006},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76100000000000001,"15min":0.79900000000000004},"min":{"1min":0.52800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.03,"5min":1.0720000000000001,"15min":2.0899999999999999},"last":0.98799999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.52600000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.2230000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77000000000000002,"15min":0.80700000000000005},"min":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":2.0609999999999999},"last":1.0369999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77100000000000002,"15min":0.79800000000000004},"min":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.50700000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.8700000000000001},"last":1.048},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2609999999999999},"last":0.91900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":807455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":11.791},"last":0.436},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.47799999999999998,"15min":0.51700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":4.657},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57099999999999995,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84999999999999998,"5min":1.056,"15min":4.5389999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.504,"15min":0.74099999999999999},"min":{"1min":0.34999999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83899999999999997,"5min":1,"15min":13.398999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":1.8089999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54100000000000004,"15min":0.56599999999999995},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.85299999999999998,"5min":1.095,"15min":4.4530000000000003},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":13.667999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.77000000000000002},"min":{"1min":0.63,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0429999999999999,"15min":1.776},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76600000000000001,"15min":1.0029999999999999},"min":{"1min":0.63700000000000001,"5min":0.499,"15min":0.45800000000000002},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":13.349},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.79600000000000004,"15min":0.84499999999999997},"min":{"1min":0.73199999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1539999999999999,"5min":1.1739999999999999,"15min":4.7649999999999997},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.67500000000000004,"5min":0.505,"15min":0.49099999999999999},"max":{"1min":1.202,"5min":1.2130000000000001,"15min":4.5039999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.747,"15min":0.96899999999999997},"min":{"1min":0.60399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0209999999999999,"5min":1.046,"15min":11.679},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.76700000000000002,"15min":1.0069999999999999},"min":{"1min":0.69599999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96499999999999997,"5min":1.032,"15min":13.298},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.79200000000000004,"15min":0.85899999999999999},"min":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.50900000000000001},"max":{"1min":1.119,"5min":1.119,"15min":11.456},"last":0.59999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":807455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.434,"15min":0.47899999999999998},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.65100000000000002,"5min":1.605,"15min":2.282},"last":0.432},{"interface":"front","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.48099999999999998},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.29099999999999998},"max":{"1min":0.68300000000000005,"5min":1.034,"15min":1.4990000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.35299999999999998,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.59899999999999998,"5min":1.228,"15min":1.228},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.55200000000000005},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.754,"5min":1.6879999999999999,"15min":2.319},"last":0.621}]},{"osd":3,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.55700000000000005},"min":{"1min":0.439,"5min":0.371,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":1.119,"15min":1.119},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57199999999999995,"15min":0.60499999999999998},"min":{"1min":0.47699999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"max":{"1min":0.71999999999999997,"5min":1.5149999999999999,"15min":2.351},"last":0.67700000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76000000000000001,"15min":0.79500000000000004},"min":{"1min":0.63500000000000001,"5min":0.56699999999999995,"15min":0.46000000000000002},"max":{"1min":1.0289999999999999,"5min":1.6499999999999999,"15min":2.238},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.51200000000000001,"15min":0.46899999999999997},"max":{"1min":1.0920000000000001,"5min":1.1140000000000001,"15min":1.518},"last":0.83599999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.77000000000000002},"min":{"1min":0.621,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":1.341,"15min":1.341},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.78200000000000003,"15min":0.81799999999999995},"min":{"1min":0.64900000000000002,"5min":0.51400000000000001,"15min":0.495},"max":{"1min":1.0609999999999999,"5min":1.9099999999999999,"15min":2.1909999999999998},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77000000000000002,"15min":0.79600000000000004},"min":{"1min":0.66300000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.01,"5min":1.107,"15min":1.488},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.63600000000000001,"5min":0.52300000000000002,"15min":0.48199999999999998},"max":{"1min":0.98599999999999999,"5min":1.448,"15min":1.448},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.77500000000000002,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.29},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.80100000000000005,"15min":0.81399999999999995},"min":{"1min":0.73199999999999998,"5min":0.57599999999999996,"15min":0.54400000000000004},"max":{"1min":1.0840000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.96399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40960,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330880,"statfs":{"total":96636764160,"available":96594821120,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.248},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.73999999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.32700000000000001,"5min":0.32400000000000001,"15min":0.27200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.375}]},{"osd":1,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.503,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.873,"15min":0.873},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35099999999999998},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.94299999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.46400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82099999999999995,"5min":0.90000000000000002,"15min":1.0409999999999999},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.57499999999999996,"5min":0.45200000000000001,"15min":0.44700000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.2250000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.1539999999999999},"last":1.0429999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.745,"15min":0.749},"min":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.496},"max":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.411},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.57699999999999996,"5min":0.53700000000000003,"15min":0.48599999999999999},"max":{"1min":0.996,"5min":1.05,"15min":1.5469999999999999},"last":0.86599999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.621,"5min":0.53400000000000003,"15min":0.51800000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.1559999999999999},"last":1.018},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.76900000000000002,"15min":0.76700000000000002},"min":{"1min":0.621,"5min":0.52500000000000002,"15min":0.52100000000000002},"max":{"1min":0.97499999999999998,"5min":0.996,"15min":1.1240000000000001},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.78200000000000003},"min":{"1min":0.60699999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.054,"5min":1.054,"15min":1.1779999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.47199999999999998},"max":{"1min":1.0389999999999999,"5min":1.0800000000000001,"15min":1.1950000000000001},"last":1.0629999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575008,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36864,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334976,"statfs":{"total":96636764160,"available":96599015424,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66300000000000003,"15min":0.629},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.97999999999999998,"15min":1.0349999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65900000000000003,"15min":0.64800000000000002},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":0.79700000000000004,"5min":0.96799999999999997,"15min":0.997},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.58799999999999997,"5min":0.44600000000000001,"15min":0.35099999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.298},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.57299999999999995,"5min":0.45900000000000002,"15min":0.438},"max":{"1min":0.96899999999999997,"5min":1.0529999999999999,"15min":1.3109999999999999},"last":0.93500000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.371},"max":{"1min":0.94799999999999995,"5min":0.95099999999999996,"15min":1.1200000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.51400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90800000000000003,"5min":1.1279999999999999,"15min":1.1839999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":0.80200000000000005},"min":{"1min":0.495,"5min":0.434,"15min":0.35899999999999999},"max":{"1min":0.92500000000000004,"5min":1.234,"15min":10.657999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72399999999999998,"15min":0.81799999999999995},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":8.75},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.69399999999999995,"15min":0.71399999999999997},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81899999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.81699999999999995,"5min":1.012,"15min":1.173},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.70099999999999996,"15min":0.70999999999999996},"min":{"1min":0.48699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84399999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.97399999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:33:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.432},"max":{"1min":0.86299999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.56000000000000005,"5min":0.432,"15min":0.432},"max":{"1min":0.84899999999999998,"5min":1.0600000000000001,"15min":1.212},"last":0.69099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36864,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334976,"statfs":{"total":96636764160,"available":96599015424,"internally_reserved":0,"allocated":1761280,"data_stored":808441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":1.042},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88500000000000001,"5min":0.92000000000000004,"15min":0.998},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.68100000000000005,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.96799999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66000000000000003,"15min":0.68500000000000005},"min":{"1min":0.52600000000000002,"5min":0.34999999999999998,"15min":0.33100000000000002},"max":{"1min":0.90300000000000002,"5min":1.0129999999999999,"15min":8.9930000000000003},"last":0.621}]},{"osd":2,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96199999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.55100000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.86799999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.59199999999999997,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.91300000000000003,"5min":0.96499999999999997,"15min":0.96599999999999997},"last":0.747},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.70699999999999996},"min":{"1min":0.621,"5min":0.48399999999999999,"15min":0.47899999999999998},"max":{"1min":0.93899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70699999999999996,"15min":0.68000000000000005},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88300000000000001,"5min":1.012,"15min":1.012},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72699999999999998,"15min":0.70999999999999996},"min":{"1min":0.50700000000000001,"5min":0.44400000000000001,"15min":0.42299999999999999},"max":{"1min":0.86499999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.74399999999999999,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.41199999999999998},"max":{"1min":0.93799999999999994,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.67300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:33:53.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-07-13T22:33:53.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.597+0000 7fdaab9ff700 1 Processor -- start 2023-07-13T22:33:53.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.597+0000 7fdaab9ff700 1 -- start start 2023-07-13T22:33:53.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.598+0000 7fdaab9ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdaa40d07d0 0x7fdaa40d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.599+0000 7fdaab9ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa408d030 con 0x7fdaa40b2230 2023-07-13T22:33:53.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.599+0000 7fdaab9ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa406b1c0 con 0x7fdaa40b1580 2023-07-13T22:33:53.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.599+0000 7fdaab9ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa40af7f0 con 0x7fdaa40d07d0 2023-07-13T22:33:53.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.599+0000 7fdaa979b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdaa40d07d0 0x7fdaa40d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.599+0000 7fdaa979b700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdaa40d07d0 0x7fdaa40d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60910/0 (socket says 172.21.15.60:60910) 2023-07-13T22:33:53.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.599+0000 7fdaa979b700 1 -- 172.21.15.60:0/1692893220 learned_addr learned my addr 172.21.15.60:0/1692893220 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:53.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.599+0000 7fdaa979b700 1 -- 172.21.15.60:0/1692893220 >> v1:172.21.15.105:6789/0 conn(0x7fdaa40b1580 legacy=0x7fdaa406aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.599+0000 7fdaa979b700 1 -- 172.21.15.60:0/1692893220 >> v1:172.21.15.60:6789/0 conn(0x7fdaa40b2230 legacy=0x7fdaa406a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.599+0000 7fdaa979b700 1 -- 172.21.15.60:0/1692893220 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa40d10e0 con 0x7fdaa40d07d0 2023-07-13T22:33:53.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.600+0000 7fdaa979b700 1 --2- 172.21.15.60:0/1692893220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40d07d0 0x7fdaa40d0ba0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00098f0 tx=0x7fdaa0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0eeb3a30aefbceb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:53.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.600+0000 7fda9bfff700 1 -- 172.21.15.60:0/1692893220 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000b8a0 con 0x7fdaa40d07d0 2023-07-13T22:33:53.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.600+0000 7fda9bfff700 1 -- 172.21.15.60:0/1692893220 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa000ba00 con 0x7fdaa40d07d0 2023-07-13T22:33:53.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.600+0000 7fda9bfff700 1 -- 172.21.15.60:0/1692893220 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000bd90 con 0x7fdaa40d07d0 2023-07-13T22:33:53.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.601+0000 7fdaab9ff700 1 -- 172.21.15.60:0/1692893220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40d07d0 msgr2=0x7fdaa40d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.601+0000 7fdaab9ff700 1 --2- 172.21.15.60:0/1692893220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40d07d0 0x7fdaa40d0ba0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00098f0 tx=0x7fdaa0009c00 comp rx=0 tx=0).stop 2023-07-13T22:33:53.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.601+0000 7fdaab9ff700 1 -- 172.21.15.60:0/1692893220 shutdown_connections 2023-07-13T22:33:53.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.601+0000 7fdaab9ff700 1 --2- 172.21.15.60:0/1692893220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40d07d0 0x7fdaa40d0ba0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.601+0000 7fdaab9ff700 1 -- 172.21.15.60:0/1692893220 >> 172.21.15.60:0/1692893220 conn(0x7fdaa409bf90 msgr2=0x7fdaa40d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:53.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.601+0000 7fdaab9ff700 1 -- 172.21.15.60:0/1692893220 shutdown_connections 2023-07-13T22:33:53.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.601+0000 7fdaab9ff700 1 -- 172.21.15.60:0/1692893220 wait complete. 2023-07-13T22:33:53.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.603+0000 7fdaab9ff700 1 Processor -- start 2023-07-13T22:33:53.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.603+0000 7fdaab9ff700 1 -- start start 2023-07-13T22:33:53.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaab9ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdaa40b1580 0x7fdaa4158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaab9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40b2230 0x7fdaa41f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaab9ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaa40d07d0 0x7fdaa41f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaab9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa41f66a0 con 0x7fdaa40b2230 2023-07-13T22:33:53.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaab9ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa41ec720 con 0x7fdaa40d07d0 2023-07-13T22:33:53.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaab9ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa415e960 con 0x7fdaa40b1580 2023-07-13T22:33:53.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa8f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40b2230 0x7fdaa41f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa8f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40b2230 0x7fdaa41f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60922/0 (socket says 172.21.15.60:60922) 2023-07-13T22:33:53.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa8f9a700 1 -- 172.21.15.60:0/776127992 learned_addr learned my addr 172.21.15.60:0/776127992 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:53.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa979b700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdaa40b1580 0x7fdaa4158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa9f9c700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaa40d07d0 0x7fdaa41f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa8f9a700 1 -- 172.21.15.60:0/776127992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdaa40b1580 msgr2=0x7fdaa4158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa8f9a700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdaa40b1580 0x7fdaa4158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa8f9a700 1 -- 172.21.15.60:0/776127992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaa40d07d0 msgr2=0x7fdaa41f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa8f9a700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaa40d07d0 0x7fdaa41f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.604+0000 7fdaa8f9a700 1 -- 172.21.15.60:0/776127992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0009550 con 0x7fdaa40b2230 2023-07-13T22:33:53.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.605+0000 7fdaa8f9a700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40b2230 0x7fdaa41f5820 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c510 tx=0x7fda9400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:53.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.605+0000 7fda99ffb700 1 -- 172.21.15.60:0/776127992 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda940052d0 con 0x7fdaa40b2230 2023-07-13T22:33:53.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.605+0000 7fda99ffb700 1 -- 172.21.15.60:0/776127992 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda94005430 con 0x7fdaa40b2230 2023-07-13T22:33:53.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.605+0000 7fda99ffb700 1 -- 172.21.15.60:0/776127992 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda940038c0 con 0x7fdaa40b2230 2023-07-13T22:33:53.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.605+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa415ec40 con 0x7fdaa40b2230 2023-07-13T22:33:53.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.605+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa415f200 con 0x7fdaa40b2230 2023-07-13T22:33:53.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.606+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa40741c0 con 0x7fdaa40b2230 2023-07-13T22:33:53.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.607+0000 7fda99ffb700 1 -- 172.21.15.60:0/776127992 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda94003a20 con 0x7fdaa40b2230 2023-07-13T22:33:53.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.608+0000 7fda99ffb700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda8806ace0 0x7fda8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:53.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.608+0000 7fda99ffb700 1 -- 172.21.15.60:0/776127992 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fda94088720 con 0x7fdaa40b2230 2023-07-13T22:33:53.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.608+0000 7fdaa979b700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda8806ace0 0x7fda8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:53.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.609+0000 7fdaa979b700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda8806ace0 0x7fda8806d190 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0014040 tx=0x7fdaa0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:53.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.613+0000 7fda99ffb700 1 -- 172.21.15.60:0/776127992 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fda94053a50 con 0x7fdaa40b2230 2023-07-13T22:33:53.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.820+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fdaa40d3640 con 0x7fdaa40b2230 2023-07-13T22:33:53.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.825+0000 7fda99ffb700 1 -- 172.21.15.60:0/776127992 <== mon.0 v2:172.21.15.60:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fda94001240 con 0x7fdaa40b2230 2023-07-13T22:33:53.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.825+0000 7fda99ffb700 1 -- 172.21.15.60:0/776127992 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v66)=0 v66) v1 ==== 123+0+0 (secure 0 0 0) 0x7fda94057070 con 0x7fdaa40b2230 2023-07-13T22:33:53.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda8806ace0 msgr2=0x7fda8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda8806ace0 0x7fda8806d190 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0014040 tx=0x7fdaa0008030 comp rx=0 tx=0).stop 2023-07-13T22:33:53.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40b2230 msgr2=0x7fdaa41f5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:53.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40b2230 0x7fdaa41f5820 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c510 tx=0x7fda9400c8d0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 shutdown_connections 2023-07-13T22:33:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fda8806ace0 0x7fda8806d190 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaa40d07d0 0x7fdaa41f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdaa40b1580 0x7fdaa4158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 --2- 172.21.15.60:0/776127992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdaa40b2230 0x7fdaa41f5820 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:53.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 >> 172.21.15.60:0/776127992 conn(0x7fdaa409bf90 msgr2=0x7fdaa41591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:53.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 shutdown_connections 2023-07-13T22:33:53.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:53.828+0000 7fdaab9ff700 1 -- 172.21.15.60:0/776127992 wait complete. 2023-07-13T22:33:53.833 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:33:53.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:33:54.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.048+0000 7fe96c7ff700 1 Processor -- start 2023-07-13T22:33:54.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.049+0000 7fe96c7ff700 1 -- start start 2023-07-13T22:33:54.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.050+0000 7fe96c7ff700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9640b2230 0x7fe9640b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:54.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.050+0000 7fe96c7ff700 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96408d030 con 0x7fe9640b1ce0 2023-07-13T22:33:54.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.050+0000 7fe96c7ff700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9640af7f0 con 0x7fe9640d07d0 2023-07-13T22:33:54.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.050+0000 7fe96c7ff700 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96415ca30 con 0x7fe9640b2230 2023-07-13T22:33:54.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.050+0000 7fe969d9a700 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9640b2230 0x7fe9640b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.050+0000 7fe96ad9c700 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe9640b1ce0 0x7fe9640b0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50498/0 (socket says 172.21.15.60:50498) 2023-07-13T22:33:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.050+0000 7fe96ad9c700 1 -- 172.21.15.60:0/1608910271 learned_addr learned my addr 172.21.15.60:0/1608910271 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-07-13T22:33:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.051+0000 7fe969d9a700 1 -- 172.21.15.60:0/1608910271 >> v1:172.21.15.105:6789/0 conn(0x7fe9640d07d0 legacy=0x7fe9640d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.051+0000 7fe969d9a700 1 -- 172.21.15.60:0/1608910271 >> v1:172.21.15.60:6789/0 conn(0x7fe9640b1ce0 legacy=0x7fe9640b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.051+0000 7fe969d9a700 1 -- 172.21.15.60:0/1608910271 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96406bdc0 con 0x7fe9640b2230 2023-07-13T22:33:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.051+0000 7fe969599700 1 -- 172.21.15.60:0/1608910271 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589215326 0 0) 0x7fe96408d030 con 0x7fe9640b1ce0 2023-07-13T22:33:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.051+0000 7fe969d9a700 1 --2- 172.21.15.60:0/1608910271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640b2230 0x7fe9640b0570 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fe954009840 tx=0x7fe954009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd502d0ec03f894f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.051+0000 7fe969599700 1 -- 172.21.15.60:0/1608910271 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954011830 con 0x7fe9640b2230 2023-07-13T22:33:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.051+0000 7fe969599700 1 -- 172.21.15.60:0/1608910271 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe954011990 con 0x7fe9640b2230 2023-07-13T22:33:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.051+0000 7fe969599700 1 -- 172.21.15.60:0/1608910271 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954011d40 con 0x7fe9640b2230 2023-07-13T22:33:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.052+0000 7fe96c7ff700 1 -- 172.21.15.60:0/1608910271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640b2230 msgr2=0x7fe9640b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.052+0000 7fe96c7ff700 1 --2- 172.21.15.60:0/1608910271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640b2230 0x7fe9640b0570 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fe954009840 tx=0x7fe954009b50 comp rx=0 tx=0).stop 2023-07-13T22:33:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.052+0000 7fe96c7ff700 1 -- 172.21.15.60:0/1608910271 shutdown_connections 2023-07-13T22:33:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.052+0000 7fe96c7ff700 1 --2- 172.21.15.60:0/1608910271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640b2230 0x7fe9640b0570 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.052+0000 7fe96c7ff700 1 -- 172.21.15.60:0/1608910271 >> 172.21.15.60:0/1608910271 conn(0x7fe96409bf90 msgr2=0x7fe9640d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.052+0000 7fe96c7ff700 1 -- 172.21.15.60:0/1608910271 shutdown_connections 2023-07-13T22:33:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.052+0000 7fe96c7ff700 1 -- 172.21.15.60:0/1608910271 wait complete. 2023-07-13T22:33:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.054+0000 7fe96c7ff700 1 Processor -- start 2023-07-13T22:33:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.054+0000 7fe96c7ff700 1 -- start start 2023-07-13T22:33:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96c7ff700 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9640b1ce0 0x7fe9640b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96c7ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9640b2230 0x7fe96415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96c7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640d07d0 0x7fe964156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96c7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640b3e40 con 0x7fe9640d07d0 2023-07-13T22:33:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96c7ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640b40e0 con 0x7fe9640b2230 2023-07-13T22:33:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96c7ff700 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640b43c0 con 0x7fe9640b1ce0 2023-07-13T22:33:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96ad9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640d07d0 0x7fe964156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96ad9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640d07d0 0x7fe964156d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60930/0 (socket says 172.21.15.60:60930) 2023-07-13T22:33:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96ad9c700 1 -- 172.21.15.60:0/3081719046 learned_addr learned my addr 172.21.15.60:0/3081719046 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-07-13T22:33:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe969d9a700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9640b2230 0x7fe96415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96a59b700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9640b1ce0 0x7fe9640b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96ad9c700 1 -- 172.21.15.60:0/3081719046 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9640b1ce0 msgr2=0x7fe9640b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96ad9c700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9640b1ce0 0x7fe9640b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96ad9c700 1 -- 172.21.15.60:0/3081719046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9640b2230 msgr2=0x7fe96415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96ad9c700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9640b2230 0x7fe96415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.055+0000 7fe96ad9c700 1 -- 172.21.15.60:0/3081719046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9540094a0 con 0x7fe9640d07d0 2023-07-13T22:33:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.056+0000 7fe96ad9c700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640d07d0 0x7fe964156d00 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00cca0 tx=0x7fe95c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.056+0000 7fe95affd700 1 -- 172.21.15.60:0/3081719046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00ac20 con 0x7fe9640d07d0 2023-07-13T22:33:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.056+0000 7fe95affd700 1 -- 172.21.15.60:0/3081719046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe95c00ad80 con 0x7fe9640d07d0 2023-07-13T22:33:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.056+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9640b46a0 con 0x7fe9640d07d0 2023-07-13T22:33:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.056+0000 7fe95affd700 1 -- 172.21.15.60:0/3081719046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c007330 con 0x7fe9640d07d0 2023-07-13T22:33:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.056+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9640b53e0 con 0x7fe9640d07d0 2023-07-13T22:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.057+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9640741c0 con 0x7fe9640d07d0 2023-07-13T22:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.058+0000 7fe95affd700 1 -- 172.21.15.60:0/3081719046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe95c007490 con 0x7fe9640d07d0 2023-07-13T22:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.059+0000 7fe95affd700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94806ace0 0x7fe94806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.060+0000 7fe95affd700 1 -- 172.21.15.60:0/3081719046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe95c089090 con 0x7fe9640d07d0 2023-07-13T22:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.060+0000 7fe96a59b700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94806ace0 0x7fe94806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.060+0000 7fe96a59b700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94806ace0 0x7fe94806d190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe960000f60 tx=0x7fe960009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.064+0000 7fe95affd700 1 -- 172.21.15.60:0/3081719046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176914 (secure 0 0 0) 0x7fe95c0543c0 con 0x7fe9640d07d0 2023-07-13T22:33:54.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.320+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9640d3640 con 0x7fe9640d07d0 2023-07-13T22:33:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.320+0000 7fe95affd700 1 -- 172.21.15.60:0/3081719046 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe95c018090 con 0x7fe9640d07d0 2023-07-13T22:33:54.307 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-07-13T22:33:54.307 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/3652157896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3652157896},{"type":"v1","addr":"172.21.15.105:6834","nonce":3652157896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":7565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1878256861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1878256861},{"type":"v1","addr":"172.21.15.105:6835","nonce":1878256861}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92},{"gid":8728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6836/2695331242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2695331242},{"type":"v1","addr":"172.21.15.60:6836","nonce":2695331242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92},{"gid":9933,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/627817355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6835","nonce":627817355},{"type":"v1","addr":"172.21.15.60:6837","nonce":627817355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[]} 2023-07-13T22:33:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94806ace0 msgr2=0x7fe94806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94806ace0 0x7fe94806d190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe960000f60 tx=0x7fe960009040 comp rx=0 tx=0).stop 2023-07-13T22:33:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640d07d0 msgr2=0x7fe964156d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:33:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640d07d0 0x7fe964156d00 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00cca0 tx=0x7fe95c00a060 comp rx=0 tx=0).stop 2023-07-13T22:33:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 shutdown_connections 2023-07-13T22:33:54.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:6832/3777735812,v1:172.21.15.60:6833/3777735812] conn(0x7fe94806ace0 0x7fe94806d190 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:54.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9640d07d0 0x7fe964156d00 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:54.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9640b1ce0 0x7fe9640b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:54.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 --2- 172.21.15.60:0/3081719046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9640b2230 0x7fe96415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:33:54.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 >> 172.21.15.60:0/3081719046 conn(0x7fe96409bf90 msgr2=0x7fe964157c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:33:54.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 shutdown_connections 2023-07-13T22:33:54.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-07-13T22:33:54.323+0000 7fe96c7ff700 1 -- 172.21.15.60:0/3081719046 wait complete. 2023-07-13T22:33:54.312 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 99 2023-07-13T22:33:54.327 DEBUG:tasks.cephfs.filesystem:already dead... 2023-07-13T22:33:54.328 INFO:teuthology.misc:Shutting down mds daemons... 2023-07-13T22:33:54.328 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-13T22:33:54.328 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.341 INFO:tasks.ceph.mds.a:Stopped 2023-07-13T22:33:54.342 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-13T22:33:54.342 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.394 INFO:tasks.ceph.mds.c:Stopped 2023-07-13T22:33:54.395 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-13T22:33:54.395 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.411 INFO:tasks.ceph.mds.b:Stopped 2023-07-13T22:33:54.412 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-13T22:33:54.412 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.467 INFO:tasks.ceph.mds.d:Stopped 2023-07-13T22:33:54.467 INFO:teuthology.misc:Shutting down osd daemons... 2023-07-13T22:33:54.467 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-07-13T22:33:54.468 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.512 INFO:tasks.ceph.osd.0:Stopped 2023-07-13T22:33:54.513 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-07-13T22:33:54.513 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.557 INFO:tasks.ceph.osd.1:Stopped 2023-07-13T22:33:54.558 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-07-13T22:33:54.558 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.607 INFO:tasks.ceph.osd.2:Stopped 2023-07-13T22:33:54.608 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-07-13T22:33:54.608 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.666 INFO:tasks.ceph.osd.3:Stopped 2023-07-13T22:33:54.667 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-07-13T22:33:54.667 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.711 INFO:tasks.ceph.osd.4:Stopped 2023-07-13T22:33:54.712 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-07-13T22:33:54.712 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.766 INFO:tasks.ceph.osd.5:Stopped 2023-07-13T22:33:54.766 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-07-13T22:33:54.766 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.811 INFO:tasks.ceph.osd.6:Stopped 2023-07-13T22:33:54.811 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-07-13T22:33:54.812 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.866 INFO:tasks.ceph.osd.7:Stopped 2023-07-13T22:33:54.866 INFO:teuthology.misc:Shutting down mgr daemons... 2023-07-13T22:33:54.867 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-07-13T22:33:54.867 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.931 INFO:tasks.ceph.mgr.y:Stopped 2023-07-13T22:33:54.931 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-07-13T22:33:54.932 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:54.990 INFO:tasks.ceph.mgr.x:Stopped 2023-07-13T22:33:54.990 INFO:teuthology.misc:Shutting down mon daemons... 2023-07-13T22:33:54.991 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-07-13T22:33:54.991 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:55.012 INFO:tasks.ceph.mon.a:Stopped 2023-07-13T22:33:55.013 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-07-13T22:33:55.013 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:55.030 INFO:tasks.ceph.mon.b:Stopped 2023-07-13T22:33:55.031 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-07-13T22:33:55.031 INFO:teuthology.orchestra.run:waiting for 300 2023-07-13T22:33:55.089 INFO:tasks.ceph.mon.c:Stopped 2023-07-13T22:33:55.090 INFO:tasks.ceph:Checking cluster log for badness... 2023-07-13T22:33:55.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2023-07-13T22:33:55.184 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:33:55.186 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-07-13T22:33:55.501 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:33:55.502 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-07-13T22:33:55.712 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:33:55.713 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-07-13T22:33:55.931 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:33:55.932 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-07-13T22:33:56.175 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:33:56.176 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-07-13T22:33:56.495 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:33:56.496 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-07-13T22:33:56.692 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:33:56.693 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-07-13T22:33:56.941 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:33:56.941 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-07-13T22:33:57.144 INFO:tasks.ceph:Archiving mon data... 2023-07-13T22:33:57.146 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/data/mon.a.tgz 2023-07-13T22:33:57.146 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2023-07-13T22:33:57.199 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.DFyPfCK6r6 2023-07-13T22:33:57.200 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.DFyPfCK6r6 2023-07-13T22:33:57.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.DFyPfCK6r6 2023-07-13T22:33:57.503 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.DFyPfCK6r6 is 454KB 2023-07-13T22:33:57.602 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.DFyPfCK6r6 2023-07-13T22:33:57.648 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/data/mon.b.tgz 2023-07-13T22:33:57.648 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2023-07-13T22:33:57.697 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.rVsCABx4JX 2023-07-13T22:33:57.698 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.rVsCABx4JX 2023-07-13T22:33:57.876 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.rVsCABx4JX 2023-07-13T22:33:57.993 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.rVsCABx4JX is 466KB 2023-07-13T22:33:58.091 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.rVsCABx4JX 2023-07-13T22:33:58.136 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/data/mon.c.tgz 2023-07-13T22:33:58.137 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2023-07-13T22:33:58.221 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.WPCCV8ZUL8 2023-07-13T22:33:58.222 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WPCCV8ZUL8 2023-07-13T22:33:58.422 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.WPCCV8ZUL8 2023-07-13T22:33:58.540 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.WPCCV8ZUL8 is 466KB 2023-07-13T22:33:58.640 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.WPCCV8ZUL8 2023-07-13T22:33:58.684 INFO:tasks.ceph:Cleaning ceph cluster... 2023-07-13T22:33:58.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-13T22:33:58.690 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-13T22:33:58.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:33:58.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:33:58.795 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-07-13T22:33:58.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-13T22:33:58.829 DEBUG:teuthology.orchestra.run.smithi105:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-13T22:33:58.905 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-13T22:33:58.913 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-13T22:33:58.929 INFO:tasks.ceph:Archiving crash dumps... 2023-07-13T22:33:58.934 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/remote/smithi060/crash 2023-07-13T22:33:58.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-13T22:33:59.030 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/remote/smithi105/crash 2023-07-13T22:33:59.032 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-13T22:33:59.100 INFO:tasks.ceph:Shutting down logrotate 2023-07-13T22:33:59.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:33:59.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-13T22:33:59.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-13T22:33:59.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-13T22:33:59.339 INFO:tasks.ceph:Compressing logs... 2023-07-13T22:33:59.340 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-13T22:33:59.345 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-13T22:33:59.410 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156212.log 2023-07-13T22:33:59.411 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154211.log 2023-07-13T22:33:59.412 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156212.log: gzip -5 88.1%--verbose -- replaced with /var/log/ceph/ceph-client.admin.156212.log.gz-- 2023-07-13T22:33:59.412 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164913.log 2023-07-13T22:33:59.413 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.123518.log 2023-07-13T22:33:59.414 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124102.log 2023-07-13T22:33:59.414 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154211.log.gz 2023-07-13T22:33:59.415 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112362.log 2023-07-13T22:33:59.415 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164913.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.164913.log.gz 2023-07-13T22:33:59.416 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155857.log 2023-07-13T22:33:59.416 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.123518.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-07-13T22:33:59.417 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124102.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124102.log.gz 2023-07-13T22:33:59.417 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153346.log 2023-07-13T22:33:59.417 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155857.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155857.log.gz 2023-07-13T22:33:59.418 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132979.log 2023-07-13T22:33:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166815.log 2023-07-13T22:33:59.419 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153346.log.gz 2023-07-13T22:33:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.132979.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132979.log.gz 2023-07-13T22:33:59.420 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154893.log 2023-07-13T22:33:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.112362.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112362.log.gz 2023-07-13T22:33:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112410.log 2023-07-13T22:33:59.420 INFO:teuthology.orchestra.run.smithi105.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.1.123518.log.gz 2023-07-13T22:33:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112312.log 2023-07-13T22:33:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.166815.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.166815.log.gz 2023-07-13T22:33:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130993.log 2023-07-13T22:33:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154893.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154893.log.gz 2023-07-13T22:33:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138953.log 2023-07-13T22:33:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130993.log: gzip -5 --verbose -- 88.1%/var/log/ceph/ceph-client.admin.155955.log 2023-07-13T22:33:59.423 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130993.log.gz 2023-07-13T22:33:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.112410.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112410.log.gz 2023-07-13T22:33:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112263.log 2023-07-13T22:33:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.112312.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112312.log.gz 2023-07-13T22:33:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.122675.log 2023-07-13T22:33:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.112263.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.112263.log.gz 2023-07-13T22:33:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-13T22:33:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.122675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112213.log 2023-07-13T22:33:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.audit.log: 89.5% -- replaced with /var/log/ceph/ceph-client.1.122675.log.gz 2023-07-13T22:33:59.426 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112114.log 2023-07-13T22:33:59.427 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.112213.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112213.log.gz 2023-07-13T22:33:59.427 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.128014.log 2023-07-13T22:33:59.428 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138953.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138953.log.gz 2023-07-13T22:33:59.428 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145805.log 2023-07-13T22:33:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155955.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155955.log.gz 2023-07-13T22:33:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135406.log 2023-07-13T22:33:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145805.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145805.log.gz 2023-07-13T22:33:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143106.log 2023-07-13T22:33:59.429 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135406.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135406.log.gz 2023-07-13T22:33:59.430 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168139.log 2023-07-13T22:33:59.430 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143106.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143106.log.gz 2023-07-13T22:33:59.430 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126641.log 2023-07-13T22:33:59.430 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168139.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.168139.log.gz 2023-07-13T22:33:59.430 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161119.log 2023-07-13T22:33:59.431 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.112114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.120845.log 2023-07-13T22:33:59.431 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112114.log.gz 2023-07-13T22:33:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.128014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.119170.log 2023-07-13T22:33:59.434 INFO:teuthology.orchestra.run.smithi105.stderr: 93.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-13T22:33:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.120845.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-07-13T22:33:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.119170.log: 94.0% -- replaced with /var/log/ceph/ceph-client.1.128014.log.gz 2023-07-13T22:33:59.435 INFO:teuthology.orchestra.run.smithi105.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.1.120845.log.gz 2023-07-13T22:33:59.435 INFO:teuthology.orchestra.run.smithi105.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.1.119170.log.gz 2023-07-13T22:33:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112065.log 2023-07-13T22:33:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-07-13T22:33:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.112065.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112065.log.gz 2023-07-13T22:33:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126641.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126641.log.gz 2023-07-13T22:33:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154354.log 2023-07-13T22:33:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161119.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161119.log.gz 2023-07-13T22:33:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130578.log 2023-07-13T22:33:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154354.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154354.log.gz 2023-07-13T22:33:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144270.log 2023-07-13T22:33:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130578.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130578.log.gz 2023-07-13T22:33:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161945.log 2023-07-13T22:33:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144270.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.144270.log.gz 2023-07-13T22:33:59.438 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163510.log 2023-07-13T22:33:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118222.log 2023-07-13T22:33:59.439 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161945.log.gz 2023-07-13T22:33:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163510.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163510.log.gz 2023-07-13T22:33:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120026.log 2023-07-13T22:33:59.439 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141355.log 2023-07-13T22:33:59.440 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118222.log.gz 2023-07-13T22:33:59.441 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.126311.log 2023-07-13T22:33:59.441 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-07-13T22:33:59.442 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.126311.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-07-13T22:33:59.442 INFO:teuthology.orchestra.run.smithi105.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.1.126311.log.gz 2023-07-13T22:33:59.442 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-07-13T22:33:59.443 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.120026.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120026.log.gz 2023-07-13T22:33:59.444 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126592.log 2023-07-13T22:33:59.444 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.141355.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141355.log.gz 2023-07-13T22:33:59.444 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147043.log 2023-07-13T22:33:59.444 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126592.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126592.log.gz 2023-07-13T22:33:59.445 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129168.log 2023-07-13T22:33:59.445 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.147043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136787.log 2023-07-13T22:33:59.445 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.147043.log.gz 2023-07-13T22:33:59.445 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129168.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129168.log.gz 2023-07-13T22:33:59.446 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157991.log 2023-07-13T22:33:59.446 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136787.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136787.log.gz 2023-07-13T22:33:59.446 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123759.log 2023-07-13T22:33:59.446 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157991.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157991.log.gz 2023-07-13T22:33:59.447 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163657.log 2023-07-13T22:33:59.448 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.124502.log 2023-07-13T22:33:59.448 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.1.124502.log: 91.0% -- replaced with /var/log/ceph/ceph-client.1.124502.log.gz 2023-07-13T22:33:59.448 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-07-13T22:33:59.448 INFO:teuthology.orchestra.run.smithi105.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-07-13T22:33:59.449 INFO:teuthology.orchestra.run.smithi105.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-07-13T22:33:59.449 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-13T22:33:59.450 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123759.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123759.log.gz 2023-07-13T22:33:59.450 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160825.log 2023-07-13T22:33:59.451 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163657.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163657.log.gz 2023-07-13T22:33:59.451 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139635.log 2023-07-13T22:33:59.451 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160825.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160825.log.gz 2023-07-13T22:33:59.451 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133491.log 2023-07-13T22:33:59.451 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139635.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139635.log.gz 2023-07-13T22:33:59.452 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117616.log 2023-07-13T22:33:59.452 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133491.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.133491.log.gz 2023-07-13T22:33:59.452 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131437.log 2023-07-13T22:33:59.452 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117616.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117616.log.gz 2023-07-13T22:33:59.452 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168906.log 2023-07-13T22:33:59.453 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131437.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131437.log.gz 2023-07-13T22:33:59.453 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146633.log 2023-07-13T22:33:59.454 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-07-13T22:33:59.454 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.127147.log 2023-07-13T22:33:59.455 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168906.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168906.log.gz 2023-07-13T22:33:59.455 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154599.log 2023-07-13T22:33:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146633.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.146633.log.gz 2023-07-13T22:33:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159866.log 2023-07-13T22:33:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154599.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154599.log.gz 2023-07-13T22:33:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132606.log 2023-07-13T22:33:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159866.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159866.log.gz 2023-07-13T22:33:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140070.log 2023-07-13T22:33:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.132606.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132606.log.gz 2023-07-13T22:33:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156410.log 2023-07-13T22:33:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.1.127147.log: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-13T22:33:59.458 INFO:teuthology.orchestra.run.smithi105.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.1.127147.log.gz 2023-07-13T22:33:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-07-13T22:33:59.459 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.125439.log 2023-07-13T22:33:59.460 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140070.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.140070.log.gz 2023-07-13T22:33:59.460 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123466.log 2023-07-13T22:33:59.460 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156410.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156410.log.gz 2023-07-13T22:33:59.460 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166570.log 2023-07-13T22:33:59.461 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123466.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123466.log.gz 2023-07-13T22:33:59.461 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148023.log 2023-07-13T22:33:59.461 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.166570.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166570.log.gz 2023-07-13T22:33:59.461 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141176.log 2023-07-13T22:33:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.1.125439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112163.log 2023-07-13T22:33:59.462 INFO:teuthology.orchestra.run.smithi105.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.1.125439.log.gz 2023-07-13T22:33:59.463 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.148023.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148023.log.gz 2023-07-13T22:33:59.463 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122246.log 2023-07-13T22:33:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.141176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163017.log 2023-07-13T22:33:59.464 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141176.log.gz 2023-07-13T22:33:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122246.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122246.log.gz 2023-07-13T22:33:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124249.log 2023-07-13T22:33:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163017.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.163017.log.gz 2023-07-13T22:33:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136489.log 2023-07-13T22:33:59.465 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.112163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.121721.log 2023-07-13T22:33:59.466 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112163.log.gz 2023-07-13T22:33:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124249.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124249.log.gz 2023-07-13T22:33:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164215.log 2023-07-13T22:33:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136489.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136489.log.gz 2023-07-13T22:33:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158308.log 2023-07-13T22:33:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164215.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.164215.log.gz 2023-07-13T22:33:59.468 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119528.log 2023-07-13T22:33:59.468 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158308.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158308.log.gz 2023-07-13T22:33:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169803.log 2023-07-13T22:33:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119528.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119528.log.gz 2023-07-13T22:33:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135798.log 2023-07-13T22:33:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169803.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169803.log.gz 2023-07-13T22:33:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.121556.log 2023-07-13T22:33:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135798.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135798.log.gz 2023-07-13T22:33:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123857.log 2023-07-13T22:33:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.121721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.119869.log 2023-07-13T22:33:59.471 INFO:teuthology.orchestra.run.smithi105.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.1.121721.log.gz 2023-07-13T22:33:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.2.121556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116641.log 2023-07-13T22:33:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123857.log: 92.0% -- replaced with /var/log/ceph/ceph-client.2.121556.log.gz 2023-07-13T22:33:59.473 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123857.log.gz 2023-07-13T22:33:59.473 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155571.log 2023-07-13T22:33:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.116641.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116641.log.gz 2023-07-13T22:33:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130774.log 2023-07-13T22:33:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.3.119869.log: 91.9% -- replaced with /var/log/ceph/ceph-client.3.119869.log.gz 2023-07-13T22:33:59.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155571.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155571.log.gz 2023-07-13T22:33:59.476 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125661.log 2023-07-13T22:33:59.476 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130774.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130774.log.gz 2023-07-13T22:33:59.476 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119879.log 2023-07-13T22:33:59.476 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139997.log 2023-07-13T22:33:59.477 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.125661.log.gz 2023-07-13T22:33:59.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119879.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119879.log.gz 2023-07-13T22:33:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.152462.log 2023-07-13T22:33:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139997.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139997.log.gz 2023-07-13T22:33:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131286.log 2023-07-13T22:33:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.152462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151912.log 2023-07-13T22:33:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131286.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131286.log.gz 2023-07-13T22:33:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140882.log 2023-07-13T22:33:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164166.log 2023-07-13T22:33:59.482 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151912.log.gz 2023-07-13T22:33:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140882.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140882.log.gz 2023-07-13T22:33:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157607.log 2023-07-13T22:33:59.484 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164166.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164166.log.gz 2023-07-13T22:33:59.484 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159498.log 2023-07-13T22:33:59.485 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157607.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.157607.log.gz 2023-07-13T22:33:59.485 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144855.log 2023-07-13T22:33:59.486 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159498.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159498.log.gz 2023-07-13T22:33:59.486 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148824.log 2023-07-13T22:33:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144855.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144855.log.gz 2023-07-13T22:33:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138090.log 2023-07-13T22:33:59.488 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.148824.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148824.log.gz 2023-07-13T22:33:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139850.log 2023-07-13T22:33:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138090.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138090.log.gz 2023-07-13T22:33:59.490 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156459.log 2023-07-13T22:33:59.491 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139850.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.139850.log.gz 2023-07-13T22:33:59.491 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134554.log 2023-07-13T22:33:59.492 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156459.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.156459.log.gz 2023-07-13T22:33:59.492 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146382.log 2023-07-13T22:33:59.493 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.134554.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134554.log.gz 2023-07-13T22:33:59.494 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119023.log 2023-07-13T22:33:59.494 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165286.log 2023-07-13T22:33:59.495 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146382.log.gz 2023-07-13T22:33:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119023.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119023.log.gz 2023-07-13T22:33:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165392.log 2023-07-13T22:33:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165286.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165286.log.gz 2023-07-13T22:33:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150453.log 2023-07-13T22:33:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165392.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.165392.log.gz 2023-07-13T22:33:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124053.log 2023-07-13T22:33:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150453.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.150453.log.gz 2023-07-13T22:33:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140376.log 2023-07-13T22:33:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124053.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124053.log.gz 2023-07-13T22:33:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136402.log 2023-07-13T22:33:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140376.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140376.log.gz 2023-07-13T22:33:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167820.log 2023-07-13T22:33:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136402.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136402.log.gz 2023-07-13T22:33:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159817.log 2023-07-13T22:33:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.167820.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.167820.log.gz 2023-07-13T22:33:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166465.log 2023-07-13T22:33:59.508 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159817.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.159817.log.gz 2023-07-13T22:33:59.508 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140831.log 2023-07-13T22:33:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.166465.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.166465.log.gz 2023-07-13T22:33:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139586.log 2023-07-13T22:33:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140831.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140831.log.gz 2023-07-13T22:33:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164264.log 2023-07-13T22:33:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139586.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139586.log.gz 2023-07-13T22:33:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149508.log 2023-07-13T22:33:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164264.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164264.log.gz 2023-07-13T22:33:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116710.log 2023-07-13T22:33:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149508.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149508.log.gz 2023-07-13T22:33:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156728.log 2023-07-13T22:33:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.116710.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116710.log.gz 2023-07-13T22:33:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125710.log 2023-07-13T22:33:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156728.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.156728.log.gz 2023-07-13T22:33:59.515 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139341.log 2023-07-13T22:33:59.515 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125710.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125710.log.gz 2023-07-13T22:33:59.516 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135553.log 2023-07-13T22:33:59.516 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139341.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139341.log.gz 2023-07-13T22:33:59.516 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159235.log 2023-07-13T22:33:59.517 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135553.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135553.log.gz 2023-07-13T22:33:59.517 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168359.log 2023-07-13T22:33:59.518 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159235.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159235.log.gz 2023-07-13T22:33:59.518 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118923.log 2023-07-13T22:33:59.519 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168359.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.168359.log.gz 2023-07-13T22:33:59.519 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153697.log 2023-07-13T22:33:59.520 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118923.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118923.log.gz 2023-07-13T22:33:59.520 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129266.log 2023-07-13T22:33:59.521 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158456.log 2023-07-13T22:33:59.521 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153697.log.gz 2023-07-13T22:33:59.522 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129266.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129266.log.gz 2023-07-13T22:33:59.522 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142559.log 2023-07-13T22:33:59.523 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165160.log 2023-07-13T22:33:59.523 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158456.log.gz 2023-07-13T22:33:59.524 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142559.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142559.log.gz 2023-07-13T22:33:59.524 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143402.log 2023-07-13T22:33:59.525 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164607.log 2023-07-13T22:33:59.525 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.165160.log.gz 2023-07-13T22:33:59.526 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143402.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143402.log.gz 2023-07-13T22:33:59.526 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142934.log 2023-07-13T22:33:59.527 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164607.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164607.log.gz 2023-07-13T22:33:59.527 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119233.log 2023-07-13T22:33:59.528 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142934.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142934.log.gz 2023-07-13T22:33:59.528 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160668.log 2023-07-13T22:33:59.529 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168857.log 2023-07-13T22:33:59.529 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119233.log.gz 2023-07-13T22:33:59.530 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160668.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160668.log.gz 2023-07-13T22:33:59.530 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153522.log 2023-07-13T22:33:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168857.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.168857.log.gz 2023-07-13T22:33:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120196.log 2023-07-13T22:33:59.532 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153522.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153522.log.gz 2023-07-13T22:33:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124004.log 2023-07-13T22:33:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.120196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127390.log 2023-07-13T22:33:59.535 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120196.log.gz 2023-07-13T22:33:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124004.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124004.log.gz 2023-07-13T22:33:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152795.log 2023-07-13T22:33:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127390.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127390.log.gz 2023-07-13T22:33:59.537 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112297.log 2023-07-13T22:33:59.537 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152795.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152795.log.gz 2023-07-13T22:33:59.537 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168955.log 2023-07-13T22:33:59.538 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.112297.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112297.log.gz 2023-07-13T22:33:59.538 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130265.log 2023-07-13T22:33:59.539 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168955.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168955.log.gz 2023-07-13T22:33:59.539 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139390.log 2023-07-13T22:33:59.539 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140425.log 2023-07-13T22:33:59.540 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.130265.log.gz 2023-07-13T22:33:59.540 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139390.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.139390.log.gz 2023-07-13T22:33:59.541 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154501.log 2023-07-13T22:33:59.542 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140425.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.140425.log.gz 2023-07-13T22:33:59.542 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158357.log 2023-07-13T22:33:59.543 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154501.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.154501.log.gz 2023-07-13T22:33:59.543 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158259.log 2023-07-13T22:33:59.544 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158357.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158357.log.gz 2023-07-13T22:33:59.544 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151139.log 2023-07-13T22:33:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158259.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158259.log.gz 2023-07-13T22:33:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146431.log 2023-07-13T22:33:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151139.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.151139.log.gz-5 --verbose 2023-07-13T22:33:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:-- /var/log/ceph/ceph-client.admin.138041.log 2023-07-13T22:33:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146431.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146431.log.gz 2023-07-13T22:33:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118615.log 2023-07-13T22:33:59.548 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138041.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138041.log.gz 2023-07-13T22:33:59.548 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119977.log 2023-07-13T22:33:59.549 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118615.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118615.log.gz 2023-07-13T22:33:59.549 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160133.log 2023-07-13T22:33:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119977.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119977.log.gz 2023-07-13T22:33:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138855.log 2023-07-13T22:33:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160133.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.160133.log.gz 2023-07-13T22:33:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139899.log 2023-07-13T22:33:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138855.log: gzip 88.1%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.138855.log.gz/var/log/ceph/ceph-client.admin.134774.log 2023-07-13T22:33:59.552 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-07-13T22:33:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139899.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.139899.log.gz 2023-07-13T22:33:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117245.log 2023-07-13T22:33:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.134774.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134774.log.gz 2023-07-13T22:33:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129559.log 2023-07-13T22:33:59.555 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117245.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117245.log.gz 2023-07-13T22:33:59.555 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157360.log 2023-07-13T22:33:59.556 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129559.log: 87.6%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.129559.log.gz--verbose -- 2023-07-13T22:33:59.556 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151188.log 2023-07-13T22:33:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157360.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157360.log.gz 2023-07-13T22:33:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159915.log 2023-07-13T22:33:59.558 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150552.log 2023-07-13T22:33:59.558 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151188.log.gz 2023-07-13T22:33:59.559 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159915.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159915.log.gz 2023-07-13T22:33:59.559 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139702.log 2023-07-13T22:33:59.560 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168188.log 2023-07-13T22:33:59.560 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150552.log.gz 2023-07-13T22:33:59.561 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139702.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139702.log.gz 2023-07-13T22:33:59.561 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119732.log 2023-07-13T22:33:59.562 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168188.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.168188.log.gz 2023-07-13T22:33:59.562 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-07-13T22:33:59.563 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119732.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119732.log.gz 2023-07-13T22:33:59.564 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140931.log 2023-07-13T22:33:59.564 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131633.log 2023-07-13T22:33:59.565 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140931.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140931.log.gz 2023-07-13T22:33:59.566 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139292.log 2023-07-13T22:33:59.566 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131633.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.131633.log.gz 2023-07-13T22:33:59.567 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149606.log 2023-07-13T22:33:59.568 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139292.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139292.log.gz 2023-07-13T22:33:59.568 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149896.log 2023-07-13T22:33:59.569 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149606.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149606.log.gz 2023-07-13T22:33:59.569 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163755.log 2023-07-13T22:33:59.570 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149896.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149896.log.gz 2023-07-13T22:33:59.570 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152844.log 2023-07-13T22:33:59.571 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163755.log.gz 2023-07-13T22:33:59.572 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151386.log 2023-07-13T22:33:59.573 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152844.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152844.log.gzgzip 2023-07-13T22:33:59.573 INFO:teuthology.orchestra.run.smithi060.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.159768.log 2023-07-13T22:33:59.574 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151386.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151386.log.gz 2023-07-13T22:33:59.574 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163163.log 2023-07-13T22:33:59.575 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159768.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159768.log.gz 2023-07-13T22:33:59.575 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160468.log 2023-07-13T22:33:59.576 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163163.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163163.log.gz 2023-07-13T22:33:59.577 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156163.log 2023-07-13T22:33:59.578 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160468.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.160468.log.gz 2023-07-13T22:33:59.578 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129510.log 2023-07-13T22:33:59.579 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156163.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156163.log.gz 2023-07-13T22:33:59.579 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118631.log 2023-07-13T22:33:59.580 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129510.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129510.log.gz 2023-07-13T22:33:59.580 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168237.log 2023-07-13T22:33:59.581 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118631.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118631.log.gz 2023-07-13T22:33:59.581 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165835.log 2023-07-13T22:33:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168237.log: gzip 87.8%-5 -- replaced with /var/log/ceph/ceph-client.admin.168237.log.gz--verbose 2023-07-13T22:33:59.583 INFO:teuthology.orchestra.run.smithi060.stderr:-- /var/log/ceph/ceph-client.admin.137142.log 2023-07-13T22:33:59.583 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154844.log 2023-07-13T22:33:59.584 INFO:teuthology.orchestra.run.smithi060.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.165835.log.gz 2023-07-13T22:33:59.584 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.137142.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137142.log.gz 2023-07-13T22:33:59.585 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169147.log 2023-07-13T22:33:59.586 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154844.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.154844.log.gz 2023-07-13T22:33:59.586 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154260.log 2023-07-13T22:33:59.587 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169147.log: gzip 89.2%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.169147.log.gz-- 2023-07-13T22:33:59.587 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162968.log 2023-07-13T22:33:59.588 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154260.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.154260.log.gz 2023-07-13T22:33:59.588 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-07-13T22:33:59.589 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162968.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162968.log.gz 2023-07-13T22:33:59.590 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158700.log 2023-07-13T22:33:59.590 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149994.log 2023-07-13T22:33:59.591 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158700.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158700.log.gz 2023-07-13T22:33:59.592 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155206.log 2023-07-13T22:33:59.593 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164460.log 2023-07-13T22:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149994.log.gz 2023-07-13T22:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155206.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.155206.log.gz 2023-07-13T22:33:59.595 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153845.log 2023-07-13T22:33:59.596 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147141.log 2023-07-13T22:33:59.597 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164460.log.gz 2023-07-13T22:33:59.597 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153845.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.153845.log.gz 2023-07-13T22:33:59.597 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156106.log 2023-07-13T22:33:59.598 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.147141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.120832.log 2023-07-13T22:33:59.599 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.147141.log.gz 2023-07-13T22:33:59.600 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156106.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.156106.log.gz 2023-07-13T22:33:59.600 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148121.log 2023-07-13T22:33:59.601 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.120832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118617.log 2023-07-13T22:33:59.602 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.148121.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148121.log.gz 2023-07-13T22:33:59.602 INFO:teuthology.orchestra.run.smithi060.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.120832.log.gz 2023-07-13T22:33:59.603 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167918.log 2023-07-13T22:33:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139096.log 2023-07-13T22:33:59.604 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118617.log.gz 2023-07-13T22:33:59.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.167918.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.167918.log.gz 2023-07-13T22:33:59.606 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119282.log 2023-07-13T22:33:59.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139096.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139096.log.gz 2023-07-13T22:33:59.607 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146333.log 2023-07-13T22:33:59.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119282.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119282.log.gz 2023-07-13T22:33:59.609 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170895.log 2023-07-13T22:33:59.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146333.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146333.log.gz 2023-07-13T22:33:59.610 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118626.log 2023-07-13T22:33:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165111.log 2023-07-13T22:33:59.612 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.170895.log.gz 2023-07-13T22:33:59.616 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118626.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118626.log.gz 2023-07-13T22:33:59.617 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131188.log 2023-07-13T22:33:59.617 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165111.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165111.log.gz 2023-07-13T22:33:59.618 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121952.log 2023-07-13T22:33:59.618 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169004.log 2023-07-13T22:33:59.618 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131188.log.gz 2023-07-13T22:33:59.620 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.121952.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121952.log.gz 2023-07-13T22:33:59.620 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118630.log 2023-07-13T22:33:59.620 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159186.log 2023-07-13T22:33:59.621 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169004.log.gz 2023-07-13T22:33:59.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118630.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118630.log.gz 2023-07-13T22:33:59.622 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170846.log 2023-07-13T22:33:59.623 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159186.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159186.log.gz 2023-07-13T22:33:59.623 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130118.log 2023-07-13T22:33:59.624 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170846.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.170846.log.gz 2023-07-13T22:33:59.624 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170797.log 2023-07-13T22:33:59.625 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130118.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130118.log.gz 2023-07-13T22:33:59.625 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144417.log 2023-07-13T22:33:59.626 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170797.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170797.log.gz 2023-07-13T22:33:59.627 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126886.log 2023-07-13T22:33:59.627 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144417.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144417.log.gz 2023-07-13T22:33:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117812.log 2023-07-13T22:33:59.635 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126886.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126886.log.gz 2023-07-13T22:33:59.635 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111957.log 2023-07-13T22:33:59.635 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117812.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117812.log.gz 2023-07-13T22:33:59.635 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118998.log 2023-07-13T22:33:59.635 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.111957.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.111957.log.gz 2023-07-13T22:33:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158990.log 2023-07-13T22:33:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118998.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118998.log.gz 2023-07-13T22:33:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169753.log 2023-07-13T22:33:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158990.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158990.log.gz 2023-07-13T22:33:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130216.log 2023-07-13T22:33:59.638 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169753.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.169753.log.gz 2023-07-13T22:33:59.638 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156606.log 2023-07-13T22:33:59.639 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130216.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130216.log.gz 2023-07-13T22:33:59.640 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150733.log 2023-07-13T22:33:59.641 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119928.log 2023-07-13T22:33:59.641 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156606.log.gz 2023-07-13T22:33:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150733.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150733.log.gz 2023-07-13T22:33:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149411.log 2023-07-13T22:33:59.643 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119928.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119928.log.gz 2023-07-13T22:33:59.644 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143215.log 2023-07-13T22:33:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149411.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.149411.log.gz 2023-07-13T22:33:59.646 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129070.log 2023-07-13T22:33:59.647 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163706.log 2023-07-13T22:33:59.647 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143215.log.gz 2023-07-13T22:33:59.648 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129070.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129070.log.gz 2023-07-13T22:33:59.649 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135018.log 2023-07-13T22:33:59.650 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152640.log 2023-07-13T22:33:59.650 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163706.log.gz 2023-07-13T22:33:59.652 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135018.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135018.log.gz 2023-07-13T22:33:59.652 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144614.log 2023-07-13T22:33:59.653 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136303.log 2023-07-13T22:33:59.653 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152640.log.gz 2023-07-13T22:33:59.654 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144614.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144614.log.gz 2023-07-13T22:33:59.654 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161994.log 2023-07-13T22:33:59.655 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163951.log 2023-07-13T22:33:59.656 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136303.log.gz 2023-07-13T22:33:59.656 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161994.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.161994.log.gz 2023-07-13T22:33:59.657 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156312.log 2023-07-13T22:33:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163951.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163951.log.gz 2023-07-13T22:33:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135357.log 2023-07-13T22:33:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156312.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156312.log.gz 2023-07-13T22:33:59.660 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154403.log 2023-07-13T22:33:59.661 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135357.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135357.log.gz 2023-07-13T22:33:59.662 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143457.log 2023-07-13T22:33:59.663 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154403.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154403.log.gz 2023-07-13T22:33:59.663 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129315.log 2023-07-13T22:33:59.664 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143457.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143457.log.gz 2023-07-13T22:33:59.665 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119016.log 2023-07-13T22:33:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129315.log.gz 2023-07-13T22:33:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126053.log 2023-07-13T22:33:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119016.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119016.log.gz 2023-07-13T22:33:59.668 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124440.log 2023-07-13T22:33:59.669 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136687.log 2023-07-13T22:33:59.669 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126053.log.gz 2023-07-13T22:33:59.670 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124440.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124440.log.gz 2023-07-13T22:33:59.671 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170699.log 2023-07-13T22:33:59.672 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137411.log 2023-07-13T22:33:59.673 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136687.log.gz 2023-07-13T22:33:59.673 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170699.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.170699.log.gz 2023-07-13T22:33:59.674 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133126.log 2023-07-13T22:33:59.675 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.137411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125759.log 2023-07-13T22:33:59.675 INFO:teuthology.orchestra.run.smithi060.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.137411.log.gz 2023-07-13T22:33:59.676 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133126.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133126.log.gz 2023-07-13T22:33:59.677 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148995.log 2023-07-13T22:33:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125759.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125759.log.gz 2023-07-13T22:33:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164411.log 2023-07-13T22:33:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.148995.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148995.log.gz 2023-07-13T22:33:59.680 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142856.log 2023-07-13T22:33:59.681 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118239.log 2023-07-13T22:33:59.681 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.164411.log.gz 2023-07-13T22:33:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142856.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.142856.log.gz 2023-07-13T22:33:59.682 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145247.log 2023-07-13T22:33:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118239.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118239.log.gz 2023-07-13T22:33:59.684 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160370.log 2023-07-13T22:33:59.685 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145247.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145247.log.gz 2023-07-13T22:33:59.686 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145345.log 2023-07-13T22:33:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140674.log 2023-07-13T22:33:59.687 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.160370.log.gz 2023-07-13T22:33:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145345.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145345.log.gz 2023-07-13T22:33:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170626.log 2023-07-13T22:33:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140674.log: gzip 87.7%-5 -- replaced with /var/log/ceph/ceph-client.admin.140674.log.gz--verbose 2023-07-13T22:33:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:-- /var/log/ceph/ceph-client.admin.122197.log 2023-07-13T22:33:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170626.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.170626.log.gz 2023-07-13T22:33:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118236.log 2023-07-13T22:33:59.693 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122197.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122197.log.gz 2023-07-13T22:33:59.693 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128160.log 2023-07-13T22:33:59.694 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118236.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118236.log.gz 2023-07-13T22:33:59.695 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143707.log 2023-07-13T22:33:59.696 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.128160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117567.log 2023-07-13T22:33:59.697 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143707.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143707.log.gz 2023-07-13T22:33:59.698 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158798.log 2023-07-13T22:33:59.699 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126494.log 2023-07-13T22:33:59.700 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117567.log.gz 2023-07-13T22:33:59.700 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158798.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158798.log.gz 2023-07-13T22:33:59.701 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124903.log 2023-07-13T22:33:59.702 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126494.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126494.log.gz 2023-07-13T22:33:59.702 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169392.log 2023-07-13T22:33:59.703 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124903.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124903.log.gz 2023-07-13T22:33:59.704 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130069.log 2023-07-13T22:33:59.706 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138562.log 2023-07-13T22:33:59.706 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169392.log.gz 2023-07-13T22:33:59.706 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130069.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130069.log.gz 2023-07-13T22:33:59.707 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153136.log 2023-07-13T22:33:59.709 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138562.log: gzip 87.5%-5 -- replaced with /var/log/ceph/ceph-client.admin.138562.log.gz--verbose 2023-07-13T22:33:59.709 INFO:teuthology.orchestra.run.smithi060.stderr:-- /var/log/ceph/ceph-client.admin.139537.log 2023-07-13T22:33:59.709 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153136.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153136.log.gz 2023-07-13T22:33:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151765.log 2023-07-13T22:33:59.711 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165541.log 88.5% 2023-07-13T22:33:59.711 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.139537.log.gz 2023-07-13T22:33:59.712 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151765.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.151765.log.gz 2023-07-13T22:33:59.713 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135847.log 2023-07-13T22:33:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131584.log 2023-07-13T22:33:59.714 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165541.log.gz 2023-07-13T22:33:59.715 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135847.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135847.log.gz 2023-07-13T22:33:59.716 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124538.log 2023-07-13T22:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131584.log: gzip 88.2%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.131584.log.gz-- 2023-07-13T22:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169053.log 2023-07-13T22:33:59.718 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124538.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124538.log.gz 2023-07-13T22:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139948.log 2023-07-13T22:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169053.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169053.log.gz 2023-07-13T22:33:59.720 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161217.log 2023-07-13T22:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139948.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139948.log.gz 2023-07-13T22:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149557.log 2023-07-13T22:33:59.724 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149215.log 2023-07-13T22:33:59.724 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149557.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.161217.log.gz 2023-07-13T22:33:59.725 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.149557.log.gz 2023-07-13T22:33:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-13T22:33:59.727 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142692.log 2023-07-13T22:33:59.727 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149215.log.gz 2023-07-13T22:33:59.729 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133224.log 2023-07-13T22:33:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142692.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142692.log.gz 2023-07-13T22:33:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131731.log 2023-07-13T22:33:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133224.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-13T22:33:59.732 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133224.log.gz 2023-07-13T22:33:59.732 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136995.log 2023-07-13T22:33:59.733 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168040.log 2023-07-13T22:33:59.734 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.131731.log.gz 2023-07-13T22:33:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136995.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136995.log.gz 2023-07-13T22:33:59.735 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165013.log 2023-07-13T22:33:59.735 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141078.log 2023-07-13T22:33:59.736 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168040.log.gz 2023-07-13T22:33:59.736 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165013.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165013.log.gz 2023-07-13T22:33:59.736 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131535.log 2023-07-13T22:33:59.739 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.141078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161070.log 2023-07-13T22:33:59.739 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141078.log.gz 2023-07-13T22:33:59.739 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131535.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131535.log.gz 2023-07-13T22:33:59.740 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128387.log 2023-07-13T22:33:59.741 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119830.log 2023-07-13T22:33:59.741 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161070.log.gz 2023-07-13T22:33:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.128387.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128387.log.gz 2023-07-13T22:33:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143761.log 2023-07-13T22:33:59.742 INFO:teuthology.orchestra.run.smithi060.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-07-13T22:33:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140980.log 2023-07-13T22:33:59.743 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119830.log.gz 2023-07-13T22:33:59.744 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143761.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143761.log.gz 2023-07-13T22:33:59.745 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132428.log 2023-07-13T22:33:59.745 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168654.log 2023-07-13T22:33:59.746 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140980.log.gz 2023-07-13T22:33:59.747 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.132428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129119.log 2023-07-13T22:33:59.748 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168654.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.168654.log.gz 2023-07-13T22:33:59.749 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165908.log 2023-07-13T22:33:59.750 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131829.log 2023-07-13T22:33:59.750 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129119.log.gz 2023-07-13T22:33:59.751 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165908.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165908.log.gz 2023-07-13T22:33:59.751 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157309.log 2023-07-13T22:33:59.753 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147974.log 2023-07-13T22:33:59.753 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131829.log.gz 2023-07-13T22:33:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157309.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157309.log.gz 2023-07-13T22:33:59.754 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138139.log 2023-07-13T22:33:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.147974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159333.log 2023-07-13T22:33:59.756 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147974.log.gz 2023-07-13T22:33:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138139.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138139.log.gz 2023-07-13T22:33:59.757 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148072.log 2023-07-13T22:33:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159333.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159333.log.gz 2023-07-13T22:33:59.759 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163559.log 2023-07-13T22:33:59.760 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.148072.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148072.log.gz 2023-07-13T22:33:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117861.log 2023-07-13T22:33:59.761 INFO:teuthology.orchestra.run.smithi060.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.152462.log.gz 2023-07-13T22:33:59.761 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163559.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163559.log.gz 2023-07-13T22:33:59.762 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139243.log 2023-07-13T22:33:59.763 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151518.log 2023-07-13T22:33:59.764 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117861.log.gz 2023-07-13T22:33:59.765 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139243.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139243.log.gz 2023-07-13T22:33:59.765 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125220.log 2023-07-13T22:33:59.766 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151518.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.151518.log.gz 2023-07-13T22:33:59.767 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133344.log 2023-07-13T22:33:59.768 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125220.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125220.log.gz 2023-07-13T22:33:59.768 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137484.log 2023-07-13T22:33:59.769 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117665.log 2023-07-13T22:33:59.770 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133344.log.gz 2023-07-13T22:33:59.770 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.137484.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137484.log.gz 2023-07-13T22:33:59.771 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117075.log 2023-07-13T22:33:59.772 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133175.log 2023-07-13T22:33:59.772 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117665.log.gz 2023-07-13T22:33:59.773 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117075.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117075.log.gz 2023-07-13T22:33:59.773 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164000.log 2023-07-13T22:33:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139801.log 2023-07-13T22:33:59.775 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.133175.log.gz 2023-07-13T22:33:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164000.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164000.log.gz 2023-07-13T22:33:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163212.log 2023-07-13T22:33:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125122.log 2023-07-13T22:33:59.778 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.139801.log.gz 2023-07-13T22:33:59.778 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163212.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.163212.log.gz 2023-07-13T22:33:59.779 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149313.log 2023-07-13T22:33:59.780 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160774.log 2023-07-13T22:33:59.780 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125122.log.gz 2023-07-13T22:33:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149313.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.149313.log.gz 2023-07-13T22:33:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136895.log 2023-07-13T22:33:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130725.log 2023-07-13T22:33:59.783 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.160774.log.gz 2023-07-13T22:33:59.783 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136895.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136895.log.gz 2023-07-13T22:33:59.784 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144368.log 2023-07-13T22:33:59.785 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129412.log 2023-07-13T22:33:59.786 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130725.log.gz 2023-07-13T22:33:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144368.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144368.log.gz 2023-07-13T22:33:59.787 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137093.log 2023-07-13T22:33:59.788 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154746.log 2023-07-13T22:33:59.788 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129412.log.gz 2023-07-13T22:33:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.137093.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137093.log.gz 2023-07-13T22:33:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155353.log 2023-07-13T22:33:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134050.log 2023-07-13T22:33:59.791 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.154746.log.gz 2023-07-13T22:33:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155353.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.155353.log.gz 2023-07-13T22:33:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170944.log 2023-07-13T22:33:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.134050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125906.log 2023-07-13T22:33:59.794 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170944.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.170944.log.gz 2023-07-13T22:33:59.795 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128869.log 2023-07-13T22:33:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155402.log 2023-07-13T22:33:59.796 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125906.log.gz 2023-07-13T22:33:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.128869.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128869.log.gz 2023-07-13T22:33:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129657.log 2023-07-13T22:33:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127241.log 2023-07-13T22:33:59.799 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155402.log.gz 2023-07-13T22:33:59.800 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129657.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129657.log.gz 2023-07-13T22:33:59.800 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171091.log 2023-07-13T22:33:59.801 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136538.log 2023-07-13T22:33:59.802 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127241.log.gz 2023-07-13T22:33:59.803 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.171091.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.171091.log.gz 2023-07-13T22:33:59.803 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154550.log 2023-07-13T22:33:59.804 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136538.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136538.log.gz 2023-07-13T22:33:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142638.log 2023-07-13T22:33:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149264.log 2023-07-13T22:33:59.807 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.154550.log.gz 2023-07-13T22:33:59.807 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142638.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142638.log.gz 2023-07-13T22:33:59.808 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122148.log 2023-07-13T22:33:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157941.log 2023-07-13T22:33:59.809 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149264.log.gz 2023-07-13T22:33:59.810 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122148.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122148.log.gz 2023-07-13T22:33:59.811 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128338.log 2023-07-13T22:33:59.812 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142108.log 2023-07-13T22:33:59.812 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157941.log.gz 2023-07-13T22:33:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.128338.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128338.log.gz 2023-07-13T22:33:59.814 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163310.log 2023-07-13T22:33:59.815 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137240.log 2023-07-13T22:33:59.815 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142108.log.gz 2023-07-13T22:33:59.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163310.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163310.log.gz 2023-07-13T22:33:59.816 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-07-13T22:33:59.818 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.137240.log: gzip 88.4%-5 -- replaced with /var/log/ceph/ceph-client.admin.137240.log.gz--verbose 2023-07-13T22:33:59.818 INFO:teuthology.orchestra.run.smithi060.stderr:-- /var/log/ceph/ceph-client.admin.154040.log 2023-07-13T22:33:59.819 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155010.log 2023-07-13T22:33:59.823 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154040.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154040.log.gz 2023-07-13T22:33:59.823 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155808.log 2023-07-13T22:33:59.823 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155010.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155010.log.gz 2023-07-13T22:33:59.824 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127781.log 2023-07-13T22:33:59.825 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155808.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.155808.log.gz 2023-07-13T22:33:59.826 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127781.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127781.log.gz 2023-07-13T22:33:59.827 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151461.log 2023-07-13T22:33:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159137.log 2023-07-13T22:33:59.829 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151461.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.151461.log.gz 2023-07-13T22:33:59.831 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159137.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.159137.log.gz 2023-07-13T22:33:59.831 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123150.log 2023-07-13T22:33:59.832 INFO:teuthology.orchestra.run.smithi105.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-07-13T22:33:59.834 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125416.log 2023-07-13T22:33:59.835 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123150.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123150.log.gz 2023-07-13T22:33:59.836 INFO:teuthology.orchestra.run.smithi105.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-07-13T22:33:59.836 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125416.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125416.log.gz 2023-07-13T22:33:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123417.log 2023-07-13T22:33:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125073.log 2023-07-13T22:33:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123417.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123417.log.gz 2023-07-13T22:33:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125073.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125073.log.gz 2023-07-13T22:33:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112425.log 2023-07-13T22:33:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129873.log 2023-07-13T22:33:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.112425.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112425.log.gz 2023-07-13T22:33:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129873.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129873.log.gz 2023-07-13T22:33:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160923.log 2023-07-13T22:33:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161168.log 2023-07-13T22:33:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160923.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160923.log.gz 2023-07-13T22:33:59.850 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161168.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161168.log.gz 2023-07-13T22:33:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136036.log 2023-07-13T22:33:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146888.log 2023-07-13T22:33:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136036.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136036.log.gz 2023-07-13T22:33:59.855 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146888.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.146888.log.gz 2023-07-13T22:33:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141961.log 2023-07-13T22:33:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116759.log 2023-07-13T22:33:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.141961.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141961.log.gz 2023-07-13T22:33:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.116759.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116759.log.gz 2023-07-13T22:33:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171140.log 2023-07-13T22:33:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163902.log 2023-07-13T22:33:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.171140.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171140.log.gz 2023-07-13T22:33:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126788.log 2023-07-13T22:33:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163902.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.163902.log.gz 2023-07-13T22:33:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126788.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126788.log.gz 2023-07-13T22:33:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136944.log 2023-07-13T22:33:59.870 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135308.log 2023-07-13T22:33:59.871 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136944.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136944.log.gz 2023-07-13T22:33:59.872 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135308.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135308.log.gz 2023-07-13T22:33:59.872 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130529.log 2023-07-13T22:33:59.875 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136838.log 2023-07-13T22:33:59.875 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130529.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130529.log.gz 2023-07-13T22:33:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136838.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136838.log.gz 2023-07-13T22:33:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.148646.log 2023-07-13T22:33:59.884 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.148646.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-07-13T22:33:59.886 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149945.log 2023-07-13T22:33:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135953.log 2023-07-13T22:33:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149945.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.149945.log.gz 2023-07-13T22:33:59.894 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159431.log 2023-07-13T22:33:59.894 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135953.log.gz 2023-07-13T22:33:59.898 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162092.log 2023-07-13T22:33:59.898 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159431.log.gz 2023-07-13T22:33:59.901 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135455.log 2023-07-13T22:33:59.902 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162092.log.gz 2023-07-13T22:33:59.905 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146937.log 2023-07-13T22:33:59.906 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135455.log.gz 2023-07-13T22:33:59.909 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146096.log 2023-07-13T22:33:59.910 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146937.log.gz 2023-07-13T22:33:59.913 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169935.log 2023-07-13T22:33:59.914 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146096.log.gz 2023-07-13T22:33:59.917 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154795.log 2023-07-13T22:33:59.917 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169935.log.gz 2023-07-13T22:33:59.921 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127341.log 2023-07-13T22:33:59.921 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154795.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154795.log.gz 2023-07-13T22:33:59.924 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127341.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127341.log.gz 2023-07-13T22:33:59.925 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142801.log 2023-07-13T22:33:59.929 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122050.log 2023-07-13T22:33:59.929 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142801.log.gz 2023-07-13T22:33:59.932 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122687.log 2023-07-13T22:33:59.933 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122050.log.gz 2023-07-13T22:33:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144712.log 2023-07-13T22:33:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122687.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122687.log.gz 2023-07-13T22:33:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144712.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144712.log.gz 2023-07-13T22:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158553.log 2023-07-13T22:33:59.951 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129608.log 2023-07-13T22:33:59.951 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158553.log.gz 2023-07-13T22:33:59.955 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167967.log 2023-07-13T22:33:59.955 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129608.log.gz 2023-07-13T22:33:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.167967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150337.log 2023-07-13T22:33:59.959 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167967.log.gz 2023-07-13T22:33:59.963 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131486.log 2023-07-13T22:33:59.963 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.150337.log.gz 2023-07-13T22:33:59.967 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160013.log 2023-07-13T22:33:59.967 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.131486.log.gz 2023-07-13T22:33:59.970 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119017.log 2023-07-13T22:33:59.971 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.160013.log.gz 2023-07-13T22:33:59.974 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125612.log 2023-07-13T22:33:59.975 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119017.log.gz 2023-07-13T22:33:59.978 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145002.log 2023-07-13T22:33:59.979 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125612.log.gz 2023-07-13T22:33:59.983 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126935.log 2023-07-13T22:33:59.983 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145002.log.gz 2023-07-13T22:33:59.987 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145756.log 2023-07-13T22:33:59.987 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126935.log.gz 2023-07-13T22:33:59.991 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144319.log 2023-07-13T22:33:59.992 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.145756.log.gz 2023-07-13T22:33:59.992 INFO:teuthology.orchestra.run.smithi060.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.0.132428.log.gz 2023-07-13T22:33:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140782.log 2023-07-13T22:33:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144319.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144319.log.gz 2023-07-13T22:33:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157509.log 2023-07-13T22:33:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140782.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140782.log.gz 2023-07-13T22:33:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140623.log 2023-07-13T22:33:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157509.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.157509.log.gz 2023-07-13T22:34:00.000 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117188.log 2023-07-13T22:34:00.000 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140623.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140623.log.gz 2023-07-13T22:34:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136134.log 2023-07-13T22:34:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117188.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117188.log.gz 2023-07-13T22:34:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169294.log 2023-07-13T22:34:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136134.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136134.log.gz 2023-07-13T22:34:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162443.log 2023-07-13T22:34:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169294.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.169294.log.gz 2023-07-13T22:34:00.008 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152697.log 2023-07-13T22:34:00.009 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162443.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162443.log.gz 2023-07-13T22:34:00.010 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150684.log 2023-07-13T22:34:00.011 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152697.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152697.log.gz 2023-07-13T22:34:00.012 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145149.log 2023-07-13T22:34:00.014 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150684.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150684.log.gz 2023-07-13T22:34:00.015 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122491.log 2023-07-13T22:34:00.016 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145149.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.145149.log.gz 2023-07-13T22:34:00.017 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122001.log 2023-07-13T22:34:00.018 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122491.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122491.log.gz 2023-07-13T22:34:00.019 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137044.log 2023-07-13T22:34:00.020 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122001.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122001.log.gz 2023-07-13T22:34:00.021 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170283.log 2023-07-13T22:34:00.022 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.137044.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.137044.log.gz 2023-07-13T22:34:00.023 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125318.log 2023-07-13T22:34:00.025 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170283.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170283.log.gz 2023-07-13T22:34:00.026 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122736.log 2023-07-13T22:34:00.026 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125318.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125318.log.gz 2023-07-13T22:34:00.027 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153796.log 2023-07-13T22:34:00.029 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122736.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122736.log.gz 2023-07-13T22:34:00.030 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167869.log 2023-07-13T22:34:00.031 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153796.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.153796.log.gz 2023-07-13T22:34:00.034 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168605.log 2023-07-13T22:34:00.034 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.167869.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167869.log.gz 2023-07-13T22:34:00.035 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153991.log 2023-07-13T22:34:00.035 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168605.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168605.log.gz 2023-07-13T22:34:00.036 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.147642.log 2023-07-13T22:34:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153991.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153991.log.gz 2023-07-13T22:34:00.039 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118229.log 2023-07-13T22:34:00.040 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.147642.log: 89.3% -- replaced with /var/log/ceph/ceph-client.0.147642.log.gz 2023-07-13T22:34:00.045 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118629.log 2023-07-13T22:34:00.045 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118229.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118229.log.gz 2023-07-13T22:34:00.045 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163416.log 2023-07-13T22:34:00.045 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118629.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118629.log.gz 2023-07-13T22:34:00.046 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166766.log 2023-07-13T22:34:00.046 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163416.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163416.log.gz 2023-07-13T22:34:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131976.log 2023-07-13T22:34:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.166766.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.166766.log.gz 2023-07-13T22:34:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130894.log 2023-07-13T22:34:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131976.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.131976.log.gz 2023-07-13T22:34:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164962.log 2023-07-13T22:34:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130894.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130894.log.gz 2023-07-13T22:34:00.054 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138464.log 2023-07-13T22:34:00.055 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164962.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164962.log.gz 2023-07-13T22:34:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116948.log 2023-07-13T22:34:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138464.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138464.log.gz 2023-07-13T22:34:00.059 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167641.log 2023-07-13T22:34:00.059 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.116948.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116948.log.gz 2023-07-13T22:34:00.060 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168507.log 2023-07-13T22:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.167641.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167641.log.gz 2023-07-13T22:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145198.log 2023-07-13T22:34:00.063 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168507.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.168507.log.gz 2023-07-13T22:34:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149166.log 2023-07-13T22:34:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145198.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145198.log.gz 2023-07-13T22:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159670.log 2023-07-13T22:34:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149166.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149166.log.gz 2023-07-13T22:34:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162869.log 2023-07-13T22:34:00.070 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159670.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.159670.log.gz 2023-07-13T22:34:00.071 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129922.log 2023-07-13T22:34:00.072 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162869.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162869.log.gz 2023-07-13T22:34:00.073 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162820.log 2023-07-13T22:34:00.074 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129922.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129922.log.gz 2023-07-13T22:34:00.075 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169588.log 2023-07-13T22:34:00.076 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162820.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162820.log.gz 2023-07-13T22:34:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137191.log 2023-07-13T22:34:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169588.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.169588.log.gz 2023-07-13T22:34:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167543.log 2023-07-13T22:34:00.079 INFO:teuthology.orchestra.run.smithi060.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.0.134050.log.gz 2023-07-13T22:34:00.081 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.137191.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137191.log.gz 2023-07-13T22:34:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149655.log 2023-07-13T22:34:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.167543.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167543.log.gz 2023-07-13T22:34:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133295.log 2023-07-13T22:34:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149655.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149655.log.gz 2023-07-13T22:34:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169245.log 2023-07-13T22:34:00.087 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148170.log 2023-07-13T22:34:00.087 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.133295.log.gz 2023-07-13T22:34:00.087 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169245.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.169245.log.gz 2023-07-13T22:34:00.088 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158407.log 2023-07-13T22:34:00.089 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.148170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131878.log 2023-07-13T22:34:00.089 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148170.log.gz 2023-07-13T22:34:00.090 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158407.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158407.log.gz 2023-07-13T22:34:00.090 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123808.log 2023-07-13T22:34:00.092 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131878.log: gzip 88.7%-5 -- replaced with /var/log/ceph/ceph-client.admin.131878.log.gz--verbose 2023-07-13T22:34:00.092 INFO:teuthology.orchestra.run.smithi060.stderr:-- /var/log/ceph/ceph-client.admin.151863.log 2023-07-13T22:34:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123808.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123808.log.gz 2023-07-13T22:34:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130314.log 2023-07-13T22:34:00.095 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151090.log 2023-07-13T22:34:00.095 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151863.log.gz 2023-07-13T22:34:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130314.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130314.log.gz 2023-07-13T22:34:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168703.log 2023-07-13T22:34:00.098 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122442.log 2023-07-13T22:34:00.098 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151090.log.gz 2023-07-13T22:34:00.099 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168703.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.168703.log.gz 2023-07-13T22:34:00.099 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138660.log 2023-07-13T22:34:00.101 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123052.log 2023-07-13T22:34:00.101 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122442.log.gz 2023-07-13T22:34:00.102 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138660.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138660.log.gz 2023-07-13T22:34:00.102 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162694.log 2023-07-13T22:34:00.104 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124587.log 2023-07-13T22:34:00.104 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123052.log.gz 2023-07-13T22:34:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162694.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162694.log.gz 2023-07-13T22:34:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134676.log 2023-07-13T22:34:00.107 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160972.log 2023-07-13T22:34:00.107 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124587.log.gz 2023-07-13T22:34:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.134676.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134676.log.gz 2023-07-13T22:34:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134725.log 2023-07-13T22:34:00.110 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123515.log 2023-07-13T22:34:00.110 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.160972.log.gz 2023-07-13T22:34:00.111 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.134725.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.134725.log.gz 2023-07-13T22:34:00.111 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124854.log 2023-07-13T22:34:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123515.log: gzip 87.9%-5 -- replaced with /var/log/ceph/ceph-client.admin.123515.log.gz--verbose 2023-07-13T22:34:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:-- /var/log/ceph/ceph-client.admin.123101.log 2023-07-13T22:34:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124854.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.124854.log.gz 2023-07-13T22:34:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138904.log 2023-07-13T22:34:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142510.log 2023-07-13T22:34:00.116 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123101.log.gz 2023-07-13T22:34:00.117 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138904.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138904.log.gz 2023-07-13T22:34:00.117 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147263.log 2023-07-13T22:34:00.119 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145051.log 2023-07-13T22:34:00.119 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.142510.log.gz 2023-07-13T22:34:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.147263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147263.log.gz 2023-07-13T22:34:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154138.log 2023-07-13T22:34:00.122 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131335.log 2023-07-13T22:34:00.122 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145051.log.gz 2023-07-13T22:34:00.123 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154138.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154138.log.gz 2023-07-13T22:34:00.123 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156361.log 2023-07-13T22:34:00.125 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131335.log: gzip 87.4%-5 -- replaced with /var/log/ceph/ceph-client.admin.131335.log.gz--verbose 2023-07-13T22:34:00.126 INFO:teuthology.orchestra.run.smithi060.stderr:-- /var/log/ceph/ceph-client.admin.141127.log 2023-07-13T22:34:00.126 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156361.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156361.log.gz 2023-07-13T22:34:00.126 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127084.log 2023-07-13T22:34:00.128 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.141127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169441.log 2023-07-13T22:34:00.128 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141127.log.gz 2023-07-13T22:34:00.129 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127084.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127084.log.gz 2023-07-13T22:34:00.129 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124805.log 2023-07-13T22:34:00.131 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136183.log 2023-07-13T22:34:00.131 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.169441.log.gz 2023-07-13T22:34:00.132 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124805.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124805.log.gz 2023-07-13T22:34:00.132 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126405.log 2023-07-13T22:34:00.134 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155759.log 2023-07-13T22:34:00.135 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136183.log.gz 2023-07-13T22:34:00.135 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126405.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126405.log.gz 2023-07-13T22:34:00.136 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145461.log 2023-07-13T22:34:00.137 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167714.log 2023-07-13T22:34:00.137 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.155759.log.gz 2023-07-13T22:34:00.138 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145461.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.145461.log.gz 2023-07-13T22:34:00.138 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119781.log 2023-07-13T22:34:00.140 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.167714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162190.log 2023-07-13T22:34:00.140 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.167714.log.gz 2023-07-13T22:34:00.141 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119781.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119781.log.gz 2023-07-13T22:34:00.141 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160232.log 2023-07-13T22:34:00.143 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142433.log 2023-07-13T22:34:00.143 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.162190.log.gz 2023-07-13T22:34:00.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160232.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.160232.log.gz 2023-07-13T22:34:00.144 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147092.log 2023-07-13T22:34:00.146 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171239.log 2023-07-13T22:34:00.146 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142433.log.gz 2023-07-13T22:34:00.147 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.147092.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.147092.log.gz 2023-07-13T22:34:00.147 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145952.log 2023-07-13T22:34:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.171239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156557.log 2023-07-13T22:34:00.149 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.171239.log.gz 2023-07-13T22:34:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145952.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.145952.log.gz 2023-07-13T22:34:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149704.log 2023-07-13T22:34:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125171.log 2023-07-13T22:34:00.152 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.156557.log.gz 2023-07-13T22:34:00.153 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149704.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.149704.log.gz 2023-07-13T22:34:00.153 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166912.log 2023-07-13T22:34:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140476.log 2023-07-13T22:34:00.155 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125171.log.gz 2023-07-13T22:34:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.166912.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166912.log.gz 2023-07-13T22:34:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126739.log 2023-07-13T22:34:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160566.log 2023-07-13T22:34:00.158 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140476.log.gz 2023-07-13T22:34:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126739.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126739.log.gz 2023-07-13T22:34:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170234.log 2023-07-13T22:34:00.161 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165443.log 2023-07-13T22:34:00.161 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.160566.log.gz 2023-07-13T22:34:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170234.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.170234.log.gz 2023-07-13T22:34:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151001.log 2023-07-13T22:34:00.164 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135504.log 2023-07-13T22:34:00.164 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.165443.log.gz 2023-07-13T22:34:00.165 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151001.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151001.log.gz 2023-07-13T22:34:00.165 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139488.log 2023-07-13T22:34:00.167 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145903.log 2023-07-13T22:34:00.168 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135504.log.gz 2023-07-13T22:34:00.168 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139488.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139488.log.gz 2023-07-13T22:34:00.168 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146690.log 2023-07-13T22:34:00.170 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123564.log 2023-07-13T22:34:00.170 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145903.log.gz 2023-07-13T22:34:00.171 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146690.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146690.log.gz 2023-07-13T22:34:00.171 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135651.log 2023-07-13T22:34:00.173 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165343.log 2023-07-13T22:34:00.173 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123564.log.gz 2023-07-13T22:34:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135651.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135651.log.gz 2023-07-13T22:34:00.175 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164558.log 2023-07-13T22:34:00.176 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116998.log 2023-07-13T22:34:00.176 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165343.log.gz 2023-07-13T22:34:00.177 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164558.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164558.log.gz 2023-07-13T22:34:00.177 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157460.log 2023-07-13T22:34:00.179 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.116998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157879.log 2023-07-13T22:34:00.179 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116998.log.gz 2023-07-13T22:34:00.180 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157460.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157460.log.gz 2023-07-13T22:34:00.180 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155906.log 2023-07-13T22:34:00.182 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125367.log 2023-07-13T22:34:00.182 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157879.log.gz 2023-07-13T22:34:00.183 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155906.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.155906.log.gz 2023-07-13T22:34:00.183 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165492.log 2023-07-13T22:34:00.185 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125367.log: gzip -5 --verbose 87.9%-- /var/log/ceph/ceph-client.admin.145854.log -- replaced with /var/log/ceph/ceph-client.admin.125367.log.gz 2023-07-13T22:34:00.185 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-07-13T22:34:00.186 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165492.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.165492.log.gz 2023-07-13T22:34:00.186 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144003.log 2023-07-13T22:34:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164727.log 2023-07-13T22:34:00.188 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145854.log.gz 2023-07-13T22:34:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144003.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144003.log.gz 2023-07-13T22:34:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141282.log 2023-07-13T22:34:00.191 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125000.log 2023-07-13T22:34:00.192 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164727.log.gz 2023-07-13T22:34:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.141282.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.141282.log.gz 2023-07-13T22:34:00.193 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118228.log 2023-07-13T22:34:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125000.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125000.log.gz 2023-07-13T22:34:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146195.log 2023-07-13T22:34:00.196 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118228.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118228.log.gz 2023-07-13T22:34:00.196 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121903.log 2023-07-13T22:34:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146195.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146195.log.gz 2023-07-13T22:34:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125514.log 2023-07-13T22:34:00.201 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.121903.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.121903.log.gz 2023-07-13T22:34:00.202 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147925.log 2023-07-13T22:34:00.202 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150288.log 2023-07-13T22:34:00.202 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125514.log.gz 2023-07-13T22:34:00.203 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.147925.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.147925.log.gz 2023-07-13T22:34:00.203 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124756.log 2023-07-13T22:34:00.204 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124658.log 2023-07-13T22:34:00.205 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150288.log.gz 2023-07-13T22:34:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124756.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.124756.log.gz 2023-07-13T22:34:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151335.log 2023-07-13T22:34:00.207 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142059.log 2023-07-13T22:34:00.207 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124658.log.gz 2023-07-13T22:34:00.208 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151335.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151335.log.gz 2023-07-13T22:34:00.209 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125465.log 2023-07-13T22:34:00.212 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142059.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142059.log.gz 2023-07-13T22:34:00.212 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167763.log 2023-07-13T22:34:00.213 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124391.log 2023-07-13T22:34:00.213 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125465.log.gz 2023-07-13T22:34:00.215 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.167763.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.167763.log.gz 2023-07-13T22:34:00.215 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127439.log 2023-07-13T22:34:00.216 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164509.log 2023-07-13T22:34:00.216 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124391.log.gz 2023-07-13T22:34:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127439.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.127439.log.gz 2023-07-13T22:34:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148267.log 2023-07-13T22:34:00.218 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164509.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164509.log.gz 2023-07-13T22:34:00.219 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118237.log 2023-07-13T22:34:00.220 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.148267.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148267.log.gz 2023-07-13T22:34:00.221 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144515.log 2023-07-13T22:34:00.222 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118237.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118237.log.gz 2023-07-13T22:34:00.223 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169637.log 2023-07-13T22:34:00.224 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144515.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144515.log.gz 2023-07-13T22:34:00.225 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150386.log 2023-07-13T22:34:00.227 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169637.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169637.log.gz 2023-07-13T22:34:00.227 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129461.log 2023-07-13T22:34:00.228 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150386.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.150386.log.gz 2023-07-13T22:34:00.228 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146284.log 2023-07-13T22:34:00.229 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129461.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129461.log.gz 2023-07-13T22:34:00.230 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133028.log 2023-07-13T22:34:00.231 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146284.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146284.log.gz 2023-07-13T22:34:00.232 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163066.log 2023-07-13T22:34:00.232 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133028.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133028.log.gz 2023-07-13T22:34:00.233 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127033.log 2023-07-13T22:34:00.235 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163066.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163066.log.gz 2023-07-13T22:34:00.236 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162335.log 2023-07-13T22:34:00.237 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127033.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127033.log.gz 2023-07-13T22:34:00.237 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151961.log 2023-07-13T22:34:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162335.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162335.log.gz 2023-07-13T22:34:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145609.log 2023-07-13T22:34:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170381.log 2023-07-13T22:34:00.242 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151961.log.gz 2023-07-13T22:34:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145609.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145609.log.gz 2023-07-13T22:34:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165688.log 2023-07-13T22:34:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170381.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170381.log.gz 2023-07-13T22:34:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136736.log 2023-07-13T22:34:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165688.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.165688.log.gz 2023-07-13T22:34:00.245 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119018.log 2023-07-13T22:34:00.246 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136736.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136736.log.gz 2023-07-13T22:34:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152991.log 2023-07-13T22:34:00.248 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119018.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119018.log.gz 2023-07-13T22:34:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126151.log 2023-07-13T22:34:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152991.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152991.log.gz 2023-07-13T22:34:00.251 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151716.log 2023-07-13T22:34:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126151.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.126151.log.gz 2023-07-13T22:34:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130676.log 2023-07-13T22:34:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151716.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151716.log.gz 2023-07-13T22:34:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127488.log 2023-07-13T22:34:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144953.log 2023-07-13T22:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130676.log.gz 2023-07-13T22:34:00.258 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127488.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.127488.log.gz 2023-07-13T22:34:00.258 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130381.log 2023-07-13T22:34:00.259 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144953.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144953.log.gz 2023-07-13T22:34:00.259 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157558.log 2023-07-13T22:34:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130381.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.130381.log.gz 2023-07-13T22:34:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128945.log 2023-07-13T22:34:00.263 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157558.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.157558.log.gz 2023-07-13T22:34:00.263 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127659.log 2023-07-13T22:34:00.264 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.128945.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128945.log.gz 2023-07-13T22:34:00.264 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139002.log 2023-07-13T22:34:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127659.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127659.log.gz 2023-07-13T22:34:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138806.log 2023-07-13T22:34:00.268 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150043.log 2023-07-13T22:34:00.269 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139002.log.gz 2023-07-13T22:34:00.270 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138806.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138806.log.gz 2023-07-13T22:34:00.270 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146529.log 2023-07-13T22:34:00.271 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150043.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.150043.log.gz 2023-07-13T22:34:00.272 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125808.log 2023-07-13T22:34:00.273 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146529.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146529.log.gz 2023-07-13T22:34:00.273 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134969.log 2023-07-13T22:34:00.274 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125808.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.125808.log.gz 2023-07-13T22:34:00.275 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119429.log 2023-07-13T22:34:00.276 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.134969.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134969.log.gz 2023-07-13T22:34:00.277 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158084.log 2023-07-13T22:34:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119429.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119429.log.gz 2023-07-13T22:34:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139439.log 2023-07-13T22:34:00.280 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158084.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158084.log.gz 2023-07-13T22:34:00.280 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152010.log 2023-07-13T22:34:00.281 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139439.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139439.log.gz 2023-07-13T22:34:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159964.log 2023-07-13T22:34:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152010.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.152010.log.gz 2023-07-13T22:34:00.284 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164116.log 2023-07-13T22:34:00.285 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159964.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159964.log.gz 2023-07-13T22:34:00.286 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122589.log 2023-07-13T22:34:00.288 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164116.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164116.log.gz 2023-07-13T22:34:00.288 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168556.log 2023-07-13T22:34:00.289 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122589.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122589.log.gz 2023-07-13T22:34:00.290 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155304.log 2023-07-13T22:34:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168556.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168556.log.gz 2023-07-13T22:34:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162519.log 2023-07-13T22:34:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155304.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155304.log.gz 2023-07-13T22:34:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126837.log 2023-07-13T22:34:00.295 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162519.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162519.log.gz 2023-07-13T22:34:00.295 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122932.log 2023-07-13T22:34:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126837.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126837.log.gz 2023-07-13T22:34:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150092.log 2023-07-13T22:34:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167592.log 2023-07-13T22:34:00.299 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122932.log.gz 2023-07-13T22:34:00.300 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150092.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.150092.log.gz 2023-07-13T22:34:00.300 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171189.log 2023-07-13T22:34:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.167592.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167592.log.gz 2023-07-13T22:34:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143160.log 2023-07-13T22:34:00.303 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.171189.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171189.log.gz 2023-07-13T22:34:00.303 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146839.log 2023-07-13T22:34:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143160.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143160.log.gz 2023-07-13T22:34:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117294.log 2023-07-13T22:34:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146839.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146839.log.gz 2023-07-13T22:34:00.307 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165211.log 2023-07-13T22:34:00.308 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117294.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117294.log.gz 2023-07-13T22:34:00.308 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154697.log 2023-07-13T22:34:00.310 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165211.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165211.log.gz 2023-07-13T22:34:00.310 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141225.log 2023-07-13T22:34:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154697.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.154697.log.gz 2023-07-13T22:34:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151237.log 2023-07-13T22:34:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.141225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143293.log 2023-07-13T22:34:00.314 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141225.log.gz 2023-07-13T22:34:00.315 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151237.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.151237.log.gz 2023-07-13T22:34:00.315 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146994.log 2023-07-13T22:34:00.316 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143293.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.143293.log.gz 2023-07-13T22:34:00.317 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127586.log 2023-07-13T22:34:00.318 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146994.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146994.log.gz 2023-07-13T22:34:00.319 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129217.log 2023-07-13T22:34:00.320 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127586.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127586.log.gz 2023-07-13T22:34:00.321 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169852.log 2023-07-13T22:34:00.322 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129217.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129217.log.gz 2023-07-13T22:34:00.323 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157656.log 2023-07-13T22:34:00.324 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169852.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.169852.log.gz 2023-07-13T22:34:00.325 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123955.log 2023-07-13T22:34:00.326 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157656.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157656.log.gz 2023-07-13T22:34:00.327 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117494.log 2023-07-13T22:34:00.327 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123955.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123955.log.gz 2023-07-13T22:34:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131927.log 2023-07-13T22:34:00.331 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117494.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117494.log.gz 2023-07-13T22:34:00.331 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157801.log 2023-07-13T22:34:00.332 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139752.log 2023-07-13T22:34:00.332 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131927.log.gz 2023-07-13T22:34:00.333 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157801.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157801.log.gz 2023-07-13T22:34:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160874.log 2023-07-13T22:34:00.334 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139752.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139752.log.gz 2023-07-13T22:34:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129971.log 2023-07-13T22:34:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160874.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160874.log.gz 2023-07-13T22:34:00.337 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122344.log 2023-07-13T22:34:00.337 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129971.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129971.log.gz 2023-07-13T22:34:00.338 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130480.log 2023-07-13T22:34:00.340 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122344.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122344.log.gz 2023-07-13T22:34:00.340 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166668.log 2023-07-13T22:34:00.341 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130480.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.130480.log.gz 2023-07-13T22:34:00.342 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170577.log 2023-07-13T22:34:00.343 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.166668.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.166668.log.gz 2023-07-13T22:34:00.344 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170479.log 2023-07-13T22:34:00.345 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170577.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.170577.log.gz 2023-07-13T22:34:00.345 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126200.log 2023-07-13T22:34:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170479.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170479.log.gz 2023-07-13T22:34:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169539.log 2023-07-13T22:34:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126200.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126200.log.gz 2023-07-13T22:34:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164313.log 2023-07-13T22:34:00.350 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169539.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.169539.log.gz 2023-07-13T22:34:00.351 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133442.log 2023-07-13T22:34:00.354 INFO:teuthology.orchestra.run.smithi105.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-07-13T22:34:00.354 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164313.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164313.log.gz 2023-07-13T22:34:00.355 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142351.log 2023-07-13T22:34:00.355 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158941.log 2023-07-13T22:34:00.355 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133442.log.gz 2023-07-13T22:34:00.356 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142351.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142351.log.gz 2023-07-13T22:34:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124489.log 2023-07-13T22:34:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158941.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158941.log.gz 2023-07-13T22:34:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125955.log 2023-07-13T22:34:00.359 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124489.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124489.log.gz 2023-07-13T22:34:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-07-13T22:34:00.361 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125955.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.125955.log.gz 2023-07-13T22:34:00.361 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-07-13T22:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159548.log 2023-07-13T22:34:00.385 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161290.log 2023-07-13T22:34:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159548.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.159548.log.gz 2023-07-13T22:34:00.387 INFO:teuthology.orchestra.run.smithi105.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-07-13T22:34:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165062.log 2023-07-13T22:34:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161290.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.161290.log.gz 2023-07-13T22:34:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165062.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.165062.log.gz 2023-07-13T22:34:00.409 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119019.log 2023-07-13T22:34:00.413 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119019.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119019.log.gz 2023-07-13T22:34:00.420 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140725.log 2023-07-13T22:34:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140725.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140725.log.gz 2023-07-13T22:34:00.425 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124200.log 2023-07-13T22:34:00.428 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134627.log 2023-07-13T22:34:00.429 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124200.log.gz 2023-07-13T22:34:00.432 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.134627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155108.log 2023-07-13T22:34:00.433 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.134627.log.gz 2023-07-13T22:34:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127135.log 2023-07-13T22:34:00.437 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155108.log.gz 2023-07-13T22:34:00.440 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145511.log 2023-07-13T22:34:00.440 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127135.log.gz 2023-07-13T22:34:00.442 INFO:teuthology.orchestra.run.smithi105.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-07-13T22:34:00.443 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119331.log 2023-07-13T22:34:00.444 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145511.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145511.log.gz 2023-07-13T22:34:00.447 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130020.log 2023-07-13T22:34:00.455 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119331.log.gz 2023-07-13T22:34:00.456 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117763.log 2023-07-13T22:34:00.456 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130020.log.gz 2023-07-13T22:34:00.456 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117763.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117763.log.gz 2023-07-13T22:34:00.456 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159088.log 2023-07-13T22:34:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159088.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159088.log.gz 2023-07-13T22:34:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126298.log 2023-07-13T22:34:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126298.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126298.log.gz 2023-07-13T22:34:00.464 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151567.log 2023-07-13T22:34:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170993.log 2023-07-13T22:34:00.466 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.151567.log.gz 2023-07-13T22:34:00.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135896.log 2023-07-13T22:34:00.470 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.170993.log.gz 2023-07-13T22:34:00.474 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149362.log 2023-07-13T22:34:00.475 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135896.log.gz 2023-07-13T22:34:00.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149362.log: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-07-13T22:34:00.477 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134920.log 2023-07-13T22:34:00.478 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149362.log.gz 2023-07-13T22:34:00.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.134920.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.134920.log.gz 2023-07-13T22:34:00.481 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117910.log 2023-07-13T22:34:00.483 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117910.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117910.log.gz 2023-07-13T22:34:00.484 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140289.log 2023-07-13T22:34:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140289.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140289.log.gz 2023-07-13T22:34:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145100.log 2023-07-13T22:34:00.491 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145100.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145100.log.gz 2023-07-13T22:34:00.491 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123906.log 2023-07-13T22:34:00.493 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123906.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123906.log.gz 2023-07-13T22:34:00.494 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164826.log 2023-07-13T22:34:00.497 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164826.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164826.log.gz 2023-07-13T22:34:00.497 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-07-13T22:34:00.508 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159284.log 2023-07-13T22:34:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159284.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159284.log.gz 2023-07-13T22:34:00.516 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155157.log 2023-07-13T22:34:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155157.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155157.log.gz 2023-07-13T22:34:00.521 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153296.log 2023-07-13T22:34:00.524 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134823.log 2023-07-13T22:34:00.525 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153296.log.gz 2023-07-13T22:34:00.528 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.134823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151286.log 2023-07-13T22:34:00.528 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.134823.log.gz 2023-07-13T22:34:00.533 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156263.log 2023-07-13T22:34:00.534 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.151286.log.gz 2023-07-13T22:34:00.539 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163359.log 2023-07-13T22:34:00.539 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156263.log.gz 2023-07-13T22:34:00.540 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139145.log 2023-07-13T22:34:00.540 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163359.log.gz 2023-07-13T22:34:00.542 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139145.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139145.log.gz 2023-07-13T22:34:00.543 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131139.log 2023-07-13T22:34:00.547 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168089.log 2023-07-13T22:34:00.547 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131139.log.gz 2023-07-13T22:34:00.551 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171335.log 2023-07-13T22:34:00.551 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.168089.log.gz 2023-07-13T22:34:00.555 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.171335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160617.log 2023-07-13T22:34:00.555 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.171335.log.gz 2023-07-13T22:34:00.559 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142989.log 2023-07-13T22:34:00.559 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160617.log.gz 2023-07-13T22:34:00.563 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165786.log 2023-07-13T22:34:00.563 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142989.log.gz 2023-07-13T22:34:00.573 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122540.log 2023-07-13T22:34:00.574 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165786.log.gz 2023-07-13T22:34:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122540.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122540.log.gz 2023-07-13T22:34:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142010.log 2023-07-13T22:34:00.589 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142010.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142010.log.gz 2023-07-13T22:34:00.589 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122834.log 2023-07-13T22:34:00.590 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122834.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122834.log.gz 2023-07-13T22:34:00.590 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130167.log 2023-07-13T22:34:00.590 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130167.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.130167.log.gz 2023-07-13T22:34:00.591 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131090.log 2023-07-13T22:34:00.591 INFO:teuthology.orchestra.run.smithi105.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-07-13T22:34:00.593 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131090.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.131090.log.gz 2023-07-13T22:34:00.593 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123248.log 2023-07-13T22:34:00.596 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143043.log 2023-07-13T22:34:00.596 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123248.log.gz 2023-07-13T22:34:00.600 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143043.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.143043.log.gz 2023-07-13T22:34:00.601 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123199.log 2023-07-13T22:34:00.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162918.log 2023-07-13T22:34:00.605 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123199.log.gz 2023-07-13T22:34:00.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162569.log 2023-07-13T22:34:00.609 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.162918.log.gz 2023-07-13T22:34:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119577.log 2023-07-13T22:34:00.613 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162569.log.gz 2023-07-13T22:34:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145560.log 2023-07-13T22:34:00.617 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119577.log.gz 2023-07-13T22:34:00.619 INFO:teuthology.orchestra.run.smithi060.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.148646.log.gz 2023-07-13T22:34:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123297.log 2023-07-13T22:34:00.621 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.145560.log.gz 2023-07-13T22:34:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123297.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123297.log.gz 2023-07-13T22:34:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156508.log 2023-07-13T22:34:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156508.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156508.log.gz 2023-07-13T22:34:00.628 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-13T22:34:00.632 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.log: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-13T22:34:00.632 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126249.log 2023-07-13T22:34:00.635 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126249.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126249.log.gz 2023-07-13T22:34:00.636 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135116.log 2023-07-13T22:34:00.638 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135116.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135116.log.gz 2023-07-13T22:34:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140525.log 2023-07-13T22:34:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140525.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140525.log.gz 2023-07-13T22:34:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117394.log 2023-07-13T22:34:00.645 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117394.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117394.log.gz 2023-07-13T22:34:00.646 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.167291.log 2023-07-13T22:34:00.651 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.167291.log: 94.1% -- replaced with /var/log/ceph/ceph-client.0.167291.log.gz 2023-07-13T22:34:00.651 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124151.log 2023-07-13T22:34:00.655 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124151.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124151.log.gz 2023-07-13T22:34:00.655 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137362.log 2023-07-13T22:34:00.658 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.137362.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.137362.log.gz 2023-07-13T22:34:00.658 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.141735.log 2023-07-13T22:34:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.141735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122883.log 2023-07-13T22:34:00.671 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171042.log 2023-07-13T22:34:00.672 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122883.log.gz 2023-07-13T22:34:00.675 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.171042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142206.log 2023-07-13T22:34:00.676 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.171042.log.gz 2023-07-13T22:34:00.679 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117139.log 2023-07-13T22:34:00.679 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142206.log.gz 2023-07-13T22:34:00.683 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133077.log 2023-07-13T22:34:00.683 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117139.log.gz 2023-07-13T22:34:00.687 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163853.log 2023-07-13T22:34:00.687 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133077.log.gz 2023-07-13T22:34:00.690 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140574.log 2023-07-13T22:34:00.691 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163853.log.gz 2023-07-13T22:34:00.694 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140574.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140574.log.gz 2023-07-13T22:34:00.694 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170332.log 2023-07-13T22:34:00.698 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170332.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.170332.log.gz 2023-07-13T22:34:00.698 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.166287.log 2023-07-13T22:34:00.703 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.166287.log: 89.0% -- replaced with /var/log/ceph/ceph-client.0.166287.log.gz 2023-07-13T22:34:00.703 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163608.log 2023-07-13T22:34:00.708 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163608.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163608.log.gz 2023-07-13T22:34:00.708 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168457.log 2023-07-13T22:34:00.710 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168457.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.168457.log.gz 2023-07-13T22:34:00.710 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144171.log 2023-07-13T22:34:00.715 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144171.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.144171.log.gz 2023-07-13T22:34:00.715 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143511.log 2023-07-13T22:34:00.715 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143511.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143511.log.gz 2023-07-13T22:34:00.716 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145658.log 2023-07-13T22:34:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145658.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145658.log.gz 2023-07-13T22:34:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171286.log 2023-07-13T22:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.171286.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171286.log.gz 2023-07-13T22:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157411.log 2023-07-13T22:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.157411.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157411.log.gz 2023-07-13T22:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152746.log 2023-07-13T22:34:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152746.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.152746.log.gz 2023-07-13T22:34:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151667.log 2023-07-13T22:34:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151667.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151667.log.gz 2023-07-13T22:34:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122785.log 2023-07-13T22:34:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122785.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122785.log.gz 2023-07-13T22:34:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149068.log 2023-07-13T22:34:00.739 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149068.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149068.log.gz 2023-07-13T22:34:00.740 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135165.log 2023-07-13T22:34:00.742 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135165.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135165.log.gz 2023-07-13T22:34:00.743 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127708.log 2023-07-13T22:34:00.745 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127708.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.127708.log.gz 2023-07-13T22:34:00.746 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159382.log 2023-07-13T22:34:00.749 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159382.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159382.log.gz 2023-07-13T22:34:00.749 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140190.log 2023-07-13T22:34:00.752 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.140190.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140190.log.gz 2023-07-13T22:34:00.753 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160321.log 2023-07-13T22:34:00.756 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160321.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160321.log.gz 2023-07-13T22:34:00.757 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169490.log 2023-07-13T22:34:00.759 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169490.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.169490.log.gz 2023-07-13T22:34:00.760 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138757.log 2023-07-13T22:34:00.763 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138757.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138757.log.gz 2023-07-13T22:34:00.764 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146739.log 2023-07-13T22:34:00.766 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146739.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146739.log.gz 2023-07-13T22:34:00.767 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150635.log 2023-07-13T22:34:00.770 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150635.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150635.log.gz 2023-07-13T22:34:00.770 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153220.log 2023-07-13T22:34:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153220.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153220.log.gz 2023-07-13T22:34:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119015.log 2023-07-13T22:34:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143590.log 2023-07-13T22:34:00.778 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119015.log.gz 2023-07-13T22:34:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143590.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143590.log.gz 2023-07-13T22:34:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119380.log 2023-07-13T22:34:00.783 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149847.log 2023-07-13T22:34:00.786 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149847.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149847.log.gz 2023-07-13T22:34:00.786 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144221.log 2023-07-13T22:34:00.789 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144221.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144221.log.gz 2023-07-13T22:34:00.790 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136589.log 2023-07-13T22:34:00.793 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136589.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136589.log.gz 2023-07-13T22:34:00.795 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117959.log 2023-07-13T22:34:00.795 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119380.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119380.log.gz 2023-07-13T22:34:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117959.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117959.log.gz 2023-07-13T22:34:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127192.log 2023-07-13T22:34:00.801 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127192.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127192.log.gz 2023-07-13T22:34:00.802 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146480.log 2023-07-13T22:34:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146480.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.146480.log.gz 2023-07-13T22:34:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155255.log 2023-07-13T22:34:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155255.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155255.log.gz 2023-07-13T22:34:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151814.log 2023-07-13T22:34:00.811 INFO:teuthology.orchestra.run.smithi060.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.128160.log.gz 2023-07-13T22:34:00.812 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151814.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151814.log.gz 2023-07-13T22:34:00.812 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156655.log 2023-07-13T22:34:00.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156655.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.156655.log.gz 2023-07-13T22:34:00.816 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170430.log 2023-07-13T22:34:00.819 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170430.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.170430.log.gz 2023-07-13T22:34:00.819 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138513.log 2023-07-13T22:34:00.821 INFO:teuthology.orchestra.run.smithi105.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-07-13T22:34:00.822 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138513.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.138513.log.gz 2023-07-13T22:34:00.823 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165737.log 2023-07-13T22:34:00.824 INFO:teuthology.orchestra.run.smithi105.stderr: 2023-07-13T22:34:00.824 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m1.436s 2023-07-13T22:34:00.825 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m5.762s 2023-07-13T22:34:00.825 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m0.588s 2023-07-13T22:34:00.826 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165737.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.165737.log.gz 2023-07-13T22:34:00.827 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135749.log 2023-07-13T22:34:00.829 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135749.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135749.log.gz 2023-07-13T22:34:00.829 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130431.log 2023-07-13T22:34:00.832 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132049.log 2023-07-13T22:34:00.833 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130431.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.130431.log.gz 2023-07-13T22:34:00.834 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.132049.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132049.log.gz 2023-07-13T22:34:00.834 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138290.log 2023-07-13T22:34:00.837 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138290.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.138290.log.gz 2023-07-13T22:34:00.838 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138611.log 2023-07-13T22:34:00.841 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129824.log 2023-07-13T22:34:00.841 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138611.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138611.log.gz 2023-07-13T22:34:00.843 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129824.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129824.log.gz 2023-07-13T22:34:00.844 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133393.log 2023-07-13T22:34:00.847 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133393.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.133393.log.gz 2023-07-13T22:34:00.847 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122393.log 2023-07-13T22:34:00.850 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122393.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122393.log.gz 2023-07-13T22:34:00.851 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118008.log 2023-07-13T22:34:00.853 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118008.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118008.log.gz 2023-07-13T22:34:00.854 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166619.log 2023-07-13T22:34:00.860 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.166619.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166619.log.gz 2023-07-13T22:34:00.860 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159597.log 2023-07-13T22:34:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.161669.log 2023-07-13T22:34:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159597.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159597.log.gz 2023-07-13T22:34:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.161669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125563.log 2023-07-13T22:34:00.878 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125563.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125563.log.gz 2023-07-13T22:34:00.881 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136085.log 2023-07-13T22:34:00.894 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127292.log 2023-07-13T22:34:00.895 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136085.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136085.log.gz 2023-07-13T22:34:00.899 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127292.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127292.log.gz 2023-07-13T22:34:00.900 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170528.log 2023-07-13T22:34:00.903 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136638.log 2023-07-13T22:34:00.914 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170528.log.gz 2023-07-13T22:34:00.914 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.136638.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136638.log.gz 2023-07-13T22:34:00.914 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121737.log 2023-07-13T22:34:00.915 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.121737.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.121737.log.gz 2023-07-13T22:34:00.915 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155059.log 2023-07-13T22:34:00.915 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155059.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155059.log.gz 2023-07-13T22:34:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170185.log 2023-07-13T22:34:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153648.log 2023-07-13T22:34:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170185.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.170185.log.gz 2023-07-13T22:34:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153648.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.153648.log.gz 2023-07-13T22:34:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147190.log 2023-07-13T22:34:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.147190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170748.log 2023-07-13T22:34:00.928 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.147190.log.gz 2023-07-13T22:34:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.170748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164049.log 2023-07-13T22:34:00.931 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.170748.log.gz 2023-07-13T22:34:00.935 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131237.log 2023-07-13T22:34:00.935 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164049.log.gz 2023-07-13T22:34:00.939 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133540.log 2023-07-13T22:34:00.939 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131237.log.gz 2023-07-13T22:34:00.944 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144761.log 2023-07-13T22:34:00.944 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133540.log.gz 2023-07-13T22:34:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120075.log 2023-07-13T22:34:00.948 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144761.log.gz 2023-07-13T22:34:00.951 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.120075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144663.log 2023-07-13T22:34:00.952 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.120075.log.gz 2023-07-13T22:34:00.956 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153746.log 2023-07-13T22:34:00.956 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144663.log.gz 2023-07-13T22:34:00.958 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121688.log 2023-07-13T22:34:00.959 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153746.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.153746.log.gz 2023-07-13T22:34:00.963 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.121688.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.121688.log.gz 2023-07-13T22:34:00.964 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123368.log 2023-07-13T22:34:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123368.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123368.log.gz 2023-07-13T22:34:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118614.log 2023-07-13T22:34:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118614.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118614.log.gz 2023-07-13T22:34:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141029.log 2023-07-13T22:34:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.141029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152942.log 2023-07-13T22:34:00.976 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141029.log.gz 2023-07-13T22:34:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146790.log 2023-07-13T22:34:00.980 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.152942.log.gz 2023-07-13T22:34:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158210.log 2023-07-13T22:34:00.984 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146790.log.gz 2023-07-13T22:34:00.987 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151618.log 2023-07-13T22:34:00.987 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158210.log.gz 2023-07-13T22:34:00.991 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.151618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117343.log 2023-07-13T22:34:00.992 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.151618.log.gz 2023-07-13T22:34:00.996 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117445.log 2023-07-13T22:34:00.996 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117343.log.gz 2023-07-13T22:34:00.999 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119626.log 2023-07-13T22:34:01.000 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117445.log.gz 2023-07-13T22:34:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146581.log 2023-07-13T22:34:01.004 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119626.log.gz 2023-07-13T22:34:01.008 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.146581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168310.log 2023-07-13T22:34:01.009 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146581.log.gz 2023-07-13T22:34:01.012 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117714.log 2023-07-13T22:34:01.013 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.168310.log.gz 2023-07-13T22:34:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.117714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118621.log 2023-07-13T22:34:01.016 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117714.log.gz 2023-07-13T22:34:01.020 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130627.log 2023-07-13T22:34:01.021 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118621.log.gz 2023-07-13T22:34:01.024 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.130627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163804.log 2023-07-13T22:34:01.025 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130627.log.gz 2023-07-13T22:34:01.028 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131780.log 2023-07-13T22:34:01.028 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163804.log.gz 2023-07-13T22:34:01.032 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131780.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.131780.log.gz 2023-07-13T22:34:01.032 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163261.log 2023-07-13T22:34:01.036 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.163261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156004.log 2023-07-13T22:34:01.036 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163261.log.gz 2023-07-13T22:34:01.040 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124342.log 2023-07-13T22:34:01.040 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.156004.log.gz 2023-07-13T22:34:01.043 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126690.log 2023-07-13T22:34:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124342.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124342.log.gz 2023-07-13T22:34:01.048 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118233.log 2023-07-13T22:34:01.049 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126690.log.gz 2023-07-13T22:34:01.052 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161847.log 2023-07-13T22:34:01.052 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118233.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118233.log.gz 2023-07-13T22:34:01.056 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156055.log 2023-07-13T22:34:01.056 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.161847.log.gz 2023-07-13T22:34:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.156055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150902.log 2023-07-13T22:34:01.060 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.156055.log.gz 2023-07-13T22:34:01.064 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143949.log 2023-07-13T22:34:01.064 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150902.log.gz 2023-07-13T22:34:01.068 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158651.log 2023-07-13T22:34:01.068 INFO:teuthology.orchestra.run.smithi060.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.161669.log.gz 2023-07-13T22:34:01.069 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143949.log.gz 2023-07-13T22:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131682.log 2023-07-13T22:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158651.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158651.log.gz 2023-07-13T22:34:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131682.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131682.log.gz 2023-07-13T22:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169343.log 2023-07-13T22:34:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150239.log 2023-07-13T22:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169343.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.169343.log.gz 2023-07-13T22:34:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160725.log 2023-07-13T22:34:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150239.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150239.log.gz 2023-07-13T22:34:01.080 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160725.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160725.log.gz 2023-07-13T22:34:01.081 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128436.log 2023-07-13T22:34:01.084 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.128436.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128436.log.gz 2023-07-13T22:34:01.084 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135067.log 2023-07-13T22:34:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155451.log 2023-07-13T22:34:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135067.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135067.log.gz 2023-07-13T22:34:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155451.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155451.log.gz 2023-07-13T22:34:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127537.log 2023-07-13T22:34:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154648.log 2023-07-13T22:34:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.127537.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127537.log.gz 2023-07-13T22:34:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154648.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.154648.log.gz 2023-07-13T22:34:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162043.log 2023-07-13T22:34:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144904.log 2023-07-13T22:34:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162043.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162043.log.gz 2023-07-13T22:34:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144904.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144904.log.gz 2023-07-13T22:34:01.099 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158892.log 2023-07-13T22:34:01.101 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158892.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158892.log.gz 2023-07-13T22:34:01.102 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.104273.log 2023-07-13T22:34:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.tmp-client.admin.104273.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.104273.log.gz 2023-07-13T22:34:01.105 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145296.log 2023-07-13T22:34:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145296.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.145296.log.gz 2023-07-13T22:34:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158749.log 2023-07-13T22:34:01.111 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158749.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158749.log.gz 2023-07-13T22:34:01.111 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123710.log 2023-07-13T22:34:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145707.log 2023-07-13T22:34:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123710.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123710.log.gz 2023-07-13T22:34:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145707.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.145707.log.gz 2023-07-13T22:34:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150190.log 2023-07-13T22:34:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150190.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150190.log.gz 2023-07-13T22:34:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142747.log 2023-07-13T22:34:01.122 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142747.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142747.log.gz 2023-07-13T22:34:01.123 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160419.log 2023-07-13T22:34:01.126 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119680.log 2023-07-13T22:34:01.127 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160419.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160419.log.gz 2023-07-13T22:34:01.127 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119680.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119680.log.gz 2023-07-13T22:34:01.127 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150782.log 2023-07-13T22:34:01.133 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150782.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150782.log.gz 2023-07-13T22:34:01.134 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169196.log 2023-07-13T22:34:01.135 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169196.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.169196.log.gz 2023-07-13T22:34:01.135 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129751.log 2023-07-13T22:34:01.137 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.129751.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129751.log.gz 2023-07-13T22:34:01.137 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159039.log 2023-07-13T22:34:01.140 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159039.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159039.log.gz 2023-07-13T22:34:01.141 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153894.log 2023-07-13T22:34:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.153894.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.153894.log.gz 2023-07-13T22:34:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169686.log 2023-07-13T22:34:01.147 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169686.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.169686.log.gz 2023-07-13T22:34:01.147 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164362.log 2023-07-13T22:34:01.150 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.164362.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164362.log.gz 2023-07-13T22:34:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143644.log 2023-07-13T22:34:01.154 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143644.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143644.log.gz 2023-07-13T22:34:01.154 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152083.log 2023-07-13T22:34:01.158 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152893.log 2023-07-13T22:34:01.158 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152083.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152083.log.gz 2023-07-13T22:34:01.159 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.152893.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152893.log.gz 2023-07-13T22:34:01.159 INFO:teuthology.orchestra.run.smithi060.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-07-13T22:34:01.159 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131386.log 2023-07-13T22:34:01.161 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168800.log 2023-07-13T22:34:01.163 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.131386.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131386.log.gz 2023-07-13T22:34:01.164 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.168800.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168800.log.gz 2023-07-13T22:34:01.165 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126102.log 2023-07-13T22:34:01.166 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135602.log 2023-07-13T22:34:01.168 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126102.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126102.log.gz 2023-07-13T22:34:01.169 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135602.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135602.log.gz 2023-07-13T22:34:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.157131.log 2023-07-13T22:34:01.171 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125269.log 2023-07-13T22:34:01.174 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.157131.log: /var/log/ceph/ceph-client.admin.125269.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125269.log.gz 2023-07-13T22:34:01.175 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145394.log 2023-07-13T22:34:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.145394.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.145394.log.gz 2023-07-13T22:34:01.178 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-07-13T22:34:01.189 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137864.log 2023-07-13T22:34:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.137864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169984.log 2023-07-13T22:34:01.203 INFO:teuthology.orchestra.run.smithi060.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-07-13T22:34:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137305.log 2023-07-13T22:34:01.219 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.169984.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169984.log.gz 2023-07-13T22:34:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.137305.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137305.log.gz 2023-07-13T22:34:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159719.log 2023-07-13T22:34:01.233 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.159719.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159719.log.gz 2023-07-13T22:34:01.234 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154960.log 2023-07-13T22:34:01.234 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154960.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154960.log.gz 2023-07-13T22:34:01.234 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133671.log 2023-07-13T22:34:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.133671.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133671.log.gz 2023-07-13T22:34:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119478.log 2023-07-13T22:34:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.119478.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119478.log.gz 2023-07-13T22:34:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147820.log 2023-07-13T22:34:01.246 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.147820.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147820.log.gz 2023-07-13T22:34:01.247 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123613.log 2023-07-13T22:34:01.247 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.123613.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123613.log.gz 2023-07-13T22:34:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142157.log 2023-07-13T22:34:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.142157.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142157.log.gz 2023-07-13T22:34:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165639.log 2023-07-13T22:34:01.249 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165639.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165639.log.gz 2023-07-13T22:34:01.249 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126543.log 2023-07-13T22:34:01.251 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126543.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.126543.log.gz 2023-07-13T22:34:01.251 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154452.log 2023-07-13T22:34:01.252 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154452.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.154452.log.gz 2023-07-13T22:34:01.252 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160517.log 2023-07-13T22:34:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.160517.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160517.log.gz 2023-07-13T22:34:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143348.log 2023-07-13T22:34:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143348.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143348.log.gz 2023-07-13T22:34:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118235.log 2023-07-13T22:34:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.118235.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118235.log.gz 2023-07-13T22:34:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125857.log 2023-07-13T22:34:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.125857.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.125857.log.gz 2023-07-13T22:34:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155670.log 2023-07-13T22:34:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.155670.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.155670.log.gz 2023-07-13T22:34:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144564.log 2023-07-13T22:34:01.271 INFO:teuthology.orchestra.run.smithi060.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.0.157131.log.gz 2023-07-13T22:34:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.144564.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144564.log.gz 2023-07-13T22:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149753.log 2023-07-13T22:34:01.275 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.149753.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149753.log.gz 2023-07-13T22:34:01.276 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116860.log 2023-07-13T22:34:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.116860.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116860.log.gz 2023-07-13T22:34:01.279 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150503.log 2023-07-13T22:34:01.281 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150141.log 2023-07-13T22:34:01.281 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150503.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150503.log.gz 2023-07-13T22:34:01.284 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166717.log 2023-07-13T22:34:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.150141.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150141.log.gz 2023-07-13T22:34:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124707.log 2023-07-13T22:34:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.166717.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166717.log.gz 2023-07-13T22:34:01.289 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.124707.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124707.log.gz 2023-07-13T22:34:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122099.log 2023-07-13T22:34:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122099.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122099.log.gz 2023-07-13T22:34:01.293 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135700.log 2023-07-13T22:34:01.297 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138415.log 2023-07-13T22:34:01.297 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135700.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135700.log.gz 2023-07-13T22:34:01.297 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154089.log 2023-07-13T22:34:01.300 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.138415.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138415.log.gz 2023-07-13T22:34:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126004.log 2023-07-13T22:34:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.154089.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154089.log.gz 2023-07-13T22:34:01.302 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126004.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.126004.log.gz 2023-07-13T22:34:01.303 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122638.log 2023-07-13T22:34:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126984.log 2023-07-13T22:34:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122638.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122638.log.gz 2023-07-13T22:34:01.307 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.126984.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.126984.log.gz 2023-07-13T22:34:01.308 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162141.log 2023-07-13T22:34:01.311 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143870.log 2023-07-13T22:34:01.311 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.162141.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162141.log.gz 2023-07-13T22:34:01.313 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143870.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143870.log.gz 2023-07-13T22:34:01.313 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165590.log 2023-07-13T22:34:01.317 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158602.log 2023-07-13T22:34:01.317 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.165590.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165590.log.gz 2023-07-13T22:34:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.158602.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158602.log.gz 2023-07-13T22:34:01.319 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139194.log 2023-07-13T22:34:01.322 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122295.log 2023-07-13T22:34:01.322 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.139194.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139194.log.gz 2023-07-13T22:34:01.324 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.122295.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.122295.log.gz 2023-07-13T22:34:01.324 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135259.log 2023-07-13T22:34:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141912.log 2023-07-13T22:34:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.135259.log: 95.8% -- replaced with /var/log/ceph/ceph-client.0.141735.log.gz 2023-07-13T22:34:01.329 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135259.log.gz 2023-07-13T22:34:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.141912.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141912.log.gz 2023-07-13T22:34:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120963.log 2023-07-13T22:34:01.332 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161896.log 2023-07-13T22:34:01.333 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.120963.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120963.log.gz 2023-07-13T22:34:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161896.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.161896.log.gz 2023-07-13T22:34:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161021.log 2023-07-13T22:34:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143816.log 2023-07-13T22:34:01.337 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.161021.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161021.log.gz 2023-07-13T22:34:01.339 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.143816.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.143816.log.gz 2023-07-13T22:34:01.492 INFO:teuthology.orchestra.run.smithi060.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-07-13T22:34:01.606 INFO:teuthology.orchestra.run.smithi060.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.137864.log.gz 2023-07-13T22:34:02.129 INFO:teuthology.orchestra.run.smithi060.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-07-13T22:34:03.025 INFO:teuthology.orchestra.run.smithi060.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-07-13T22:34:03.607 INFO:teuthology.orchestra.run.smithi060.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-07-13T22:34:03.610 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-07-13T22:34:03.610 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m4.226s 2023-07-13T22:34:03.611 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m14.619s 2023-07-13T22:34:03.611 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m3.617s 2023-07-13T22:34:03.612 INFO:tasks.ceph:Archiving logs... 2023-07-13T22:34:03.613 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/remote/smithi060/log 2023-07-13T22:34:03.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-13T22:34:05.787 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/remote/smithi105/log 2023-07-13T22:34:05.788 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-13T22:34:06.487 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-13T22:34:06.496 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-13T22:34:06.497 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-13T22:34:06.502 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-13T22:34:06.633 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-13T22:34:06.653 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-13T22:34:06.654 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:34:06.660 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:34:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:bash: ntpq: command not found 2023-07-13T22:34:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:bash: ntpq: command not found 2023-07-13T22:34:06.711 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:34:06.711 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-07-13T22:34:06.712 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 +34us[ +26us] +/- 91ms 2023-07-13T22:34:06.712 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv02.front.sepia.ceph.com 3 6 377 50 -1526us[-1534us] +/- 76ms 2023-07-13T22:34:06.712 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +1791us[+1784us] +/- 90ms 2023-07-13T22:34:06.713 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:34:06.715 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:34:06.715 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2023-07-13T22:34:06.715 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 50 +53us[ +53us] +/- 91ms 2023-07-13T22:34:06.716 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -1500us[-1529us] +/- 76ms 2023-07-13T22:34:06.716 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +1811us[+1811us] +/- 90ms 2023-07-13T22:34:06.716 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:34:06.717 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-13T22:34:06.728 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-13T22:34:06.729 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-13T22:34:06.739 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-13T22:34:06.863 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-13T22:34:07.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T22:34:07.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:34:07.109 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T22:34:07.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:34:07.186 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-13T22:34:07.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-13T22:34:07.218 INFO:teuthology.task.internal:Duration was 2268.905793 seconds 2023-07-13T22:34:07.219 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-13T22:34:07.230 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-13T22:34:07.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-13T22:34:07.236 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-13T22:34:07.352 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T22:34:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T22:34:07.821 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-13T22:34:07.822 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2023-07-13T22:34:07.823 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-13T22:34:07.888 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2023-07-13T22:34:07.888 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-13T22:34:07.949 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-13T22:34:07.950 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-13T22:34:07.954 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-13T22:34:08.042 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-13T22:34:08.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-13T22:34:08.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-13T22:34:08.185 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-13T22:34:08.197 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-13T22:34:08.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-13T22:34:08.268 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-13T22:34:08.338 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-13T22:34:08.349 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-13T22:34:08.354 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-13T22:34:08.419 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2023-07-13T22:34:08.422 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2023-07-13T22:34:08.457 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-13T22:34:08.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:34:08.502 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-13T22:34:08.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:34:08.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-13T22:34:08.559 INFO:teuthology.task.internal:Transferring archived files... 2023-07-13T22:34:08.560 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/remote/smithi060 2023-07-13T22:34:08.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-13T22:34:08.643 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271/remote/smithi105 2023-07-13T22:34:08.644 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-13T22:34:08.731 INFO:teuthology.task.internal:Removing archive directory... 2023-07-13T22:34:08.732 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-13T22:34:08.738 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-13T22:34:08.783 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-13T22:34:08.795 INFO:teuthology.task.internal:Not uploading archives. 2023-07-13T22:34:08.796 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-13T22:34:08.809 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-13T22:34:08.810 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-13T22:34:08.824 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-13T22:34:08.866 INFO:teuthology.orchestra.run.smithi060.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 13 22:34 /home/ubuntu/cephtest 2023-07-13T22:34:08.870 INFO:teuthology.orchestra.run.smithi105.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 13 22:34 /home/ubuntu/cephtest 2023-07-13T22:34:08.873 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-13T22:34:08.885 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-13T22:34:08.910 INFO:teuthology.nuke:Checking targets against current locks 2023-07-13T22:34:08.941 DEBUG:teuthology.nuke:shortname: smithi060 2023-07-13T22:34:08.941 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-13T22:34:08.972 DEBUG:teuthology.nuke:shortname: smithi105 2023-07-13T22:34:08.972 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-13T22:34:08.976 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:15.314340', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8nVQIrQ7b3w8ZPuA5eRJy7ykeR6z2+B/1lmJQu3oCFhge/KRmZhTWueiqzcfJEpS3NJkyU0jvPF/5kuU+lvcWyfsI1+Z53LxbyncMhk8AZiJXJ5RVsxpymZZAkn39Ab+fnO8vGyx50dH5XAGBR5OWqruHYPpcWw4bH3ddIvGOpXjxR7IKEKAULhkf9YZ/j11O4uaHiN9Xzv+K5N/nMdk4kxCPS21Gtst3U0KGWrR+wC2WNSJVB7I/LL48GAX4X3Nlh4gvr3jtEKEvpNCduUi8vnEBDc3xhb+jOkMZtdZB96z0ZxT/LmBZvyDQ3Zr9eigrYDz7EzbAgOMRaaHMonPY3xrJM5LuQUzjFbga2yPTZDVkBATKlxGmlAxDgtuuKA9fYpsaXvMn7vpjK/jlReKedb2lS7heNL4EY1ZebMxhAudvqljlLgQeQMBRhZ4W+lShI8nssgvHLsBKf5IVmlcmMbCTAFnNOcw4cVZNECQrwVW6TbZ6V0YJ6WN+9SLkpG0='} 2023-07-13T22:34:09.005 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:15.316596', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLA2TVUdkdWf/GTZ+fWrjN14P+IxbNz6W3xXXjoshJQIYoB4jbdyPIdkjNaHeN8ANEoVUFDLZVy4k5cCqpwxTkEs2TZviwA0NyjGu+J660iXZM9f3izPw766fe8IE1duDu3xe0oyFZY30eJeh2CooQVeT11iLbr5ESZRGKlWRcnARsYAg52je2Rbcuus70/KQ5O07M+Y9lP4BlHkqg5/ycHSodgSe1TAr63oA2FZOsM5lYpsc+QdBIU4D4oOM48NBGuX0yzP7Fzga9QSKEYlzF1A5GWo4jmvj0EZygyx+hiRQEUjI7MlB+nTMM8qGVEB6ID30Xid9gCT8Mi7NhIgAvc5U6JH3Ey4wzGaV3x7vrz1bFvQhgK2xwR1ITNfQSafEOzr7fhI/mWcUzlUX/ouH8AbjkAtNYgO18ez7fzgIEtA+01SBhR6Q/rdDHSDA9Wb0dMzrnFEkLdcXjqF5fswqeTR7rA1JXG4jX9Cb2OXd6iSUqiNWtznHs4eI1bmBf2/E='} 2023-07-13T22:34:09.040 INFO:teuthology.orchestra.console:Power off smithi060 2023-07-13T22:34:09.041 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-13T22:34:09.065 INFO:teuthology.orchestra.console:Power off smithi105 2023-07-13T22:34:09.066 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-13T22:34:09.081 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:34:09.093 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:34:13.107 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:34:13.121 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:34:17.224 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:34:17.235 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:34:17.438 INFO:teuthology.orchestra.console:Power off for smithi060 completed 2023-07-13T22:34:17.447 INFO:teuthology.orchestra.console:Power off for smithi105 completed 2023-07-13T22:34:17.560 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} duration: 2268.905793428421 failure_reason: 'Test failure: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits)' flavor: default mon.a-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 mon.b-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=33cde7b3508c41d5911278b2b8da0289 status: fail success: false 2023-07-13T22:34:17.561 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-13T22:34:17.638 INFO:teuthology.run:FAIL